2024-09-10T09:33:21.458 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-10T09:33:21.482 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-10T09:33:21.572 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7898997 branch: wip-rishabh-testing-20240909.165402-squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-quiesce/with-quiesce 6-workunit/suites/fsx}} email: null first_in_suite: false job_id: '7898997' kernel: client: branch: testing kdb: 1 sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-rishabh-testing-20240909.165402-squid ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 mon warn on pool no app: false osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request sha1: 1cf819b7212a6048e92fc78c4eff963344f6d56b subvols: create: 2 subvol_options: --size 25000000000 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.dir_split install: ceph: flavor: default sha1: 1cf819b7212a6048e92fc78c4eff963344f6d56b kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=legacy - wsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-testing-20240909.165402-squid sha1: 1cf819b7212a6048e92fc78c4eff963344f6d56b owner: scheduled_rishabh@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 9893 sha1: 1cf819b7212a6048e92fc78c4eff963344f6d56b sleep_before_teardown: 0 subset: 197/1024 suite: fs suite_branch: wip-rishabh-testing-20240909.165402-squid suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 1cf819b7212a6048e92fc78c4eff963344f6d56b syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi077.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBGD0g8+fXAuauH7KXYeJhNPxZu/qoO7b3Kw7YWti587XYEGegMCHPr/QuklN44lj+Xr5Dzlne9fQZ4Hc4KWD8Y= smithi120.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEt06PKDP6C377N+xdWjSVDgVx9yV8MtXI2Z0zyBpI8HFMnK097siA7dgIDA9lOtk6PY8yr3Np4K0+SgVLs1rpU= smithi134.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOhSjakgYM9TuIi/Lzpqe7rqra/hhH/h73gVf3oyc7u0UCwfNG/xtCFpUO/zovYvsYySaZ82J6+rYQxSnjOEMNU= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - quiescer: null - workunit: clients: all: - suites/fsx.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-10_06:58:10 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-10T09:33:21.573 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa; will attempt to use it 2024-09-10T09:33:21.573 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa/tasks 2024-09-10T09:33:21.573 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-10T09:33:21.575 INFO:teuthology.task.internal:Checking packages... 2024-09-10T09:33:21.595 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '1cf819b7212a6048e92fc78c4eff963344f6d56b' 2024-09-10T09:33:21.596 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-10T09:33:21.596 INFO:teuthology.packaging:ref: None 2024-09-10T09:33:21.596 INFO:teuthology.packaging:tag: None 2024-09-10T09:33:21.596 INFO:teuthology.packaging:branch: wip-rishabh-testing-20240909.165402-squid 2024-09-10T09:33:21.596 INFO:teuthology.packaging:sha1: 1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T09:33:21.596 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-rishabh-testing-20240909.165402-squid 2024-09-10T09:33:21.738 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-305.g1cf819b7 2024-09-10T09:33:21.739 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-10T09:33:21.748 INFO:teuthology.task.internal:no buildpackages task found 2024-09-10T09:33:21.748 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-10T09:33:21.756 INFO:teuthology.task.internal:Saving configuration 2024-09-10T09:33:21.776 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-10T09:33:21.786 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-10T09:33:21.805 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi077.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7898997', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 09:28:21.763035', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBGD0g8+fXAuauH7KXYeJhNPxZu/qoO7b3Kw7YWti587XYEGegMCHPr/QuklN44lj+Xr5Dzlne9fQZ4Hc4KWD8Y='} 2024-09-10T09:33:21.821 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi120.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7898997', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 09:28:21.761890', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEt06PKDP6C377N+xdWjSVDgVx9yV8MtXI2Z0zyBpI8HFMnK097siA7dgIDA9lOtk6PY8yr3Np4K0+SgVLs1rpU='} 2024-09-10T09:33:21.841 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi134.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7898997', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 09:28:21.763945', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOhSjakgYM9TuIi/Lzpqe7rqra/hhH/h73gVf3oyc7u0UCwfNG/xtCFpUO/zovYvsYySaZ82J6+rYQxSnjOEMNU='} 2024-09-10T09:33:21.841 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-10T09:33:21.847 INFO:teuthology.task.internal:roles: ubuntu@smithi077.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-09-10T09:33:21.847 INFO:teuthology.task.internal:roles: ubuntu@smithi120.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-09-10T09:33:21.847 INFO:teuthology.task.internal:roles: ubuntu@smithi134.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-09-10T09:33:21.847 INFO:teuthology.run_tasks:Running task console_log... 2024-09-10T09:33:21.999 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fe28c793be0>, signals=[15]) 2024-09-10T09:33:21.999 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-10T09:33:22.005 INFO:teuthology.task.internal:Opening connections... 2024-09-10T09:33:22.005 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi077.front.sepia.ceph.com 2024-09-10T09:33:22.006 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi077.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T09:33:22.081 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi120.front.sepia.ceph.com 2024-09-10T09:33:22.082 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi120.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T09:33:22.155 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi134.front.sepia.ceph.com 2024-09-10T09:33:22.155 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi134.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T09:33:22.226 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-10T09:33:22.235 DEBUG:teuthology.orchestra.run.smithi077:> uname -m 2024-09-10T09:33:22.254 INFO:teuthology.orchestra.run.smithi077.stdout:x86_64 2024-09-10T09:33:22.254 DEBUG:teuthology.orchestra.run.smithi077:> cat /etc/os-release 2024-09-10T09:33:22.312 INFO:teuthology.orchestra.run.smithi077.stdout:NAME="CentOS Stream" 2024-09-10T09:33:22.312 INFO:teuthology.orchestra.run.smithi077.stdout:VERSION="9" 2024-09-10T09:33:22.312 INFO:teuthology.orchestra.run.smithi077.stdout:ID="centos" 2024-09-10T09:33:22.312 INFO:teuthology.orchestra.run.smithi077.stdout:ID_LIKE="rhel fedora" 2024-09-10T09:33:22.312 INFO:teuthology.orchestra.run.smithi077.stdout:VERSION_ID="9" 2024-09-10T09:33:22.313 INFO:teuthology.orchestra.run.smithi077.stdout:PLATFORM_ID="platform:el9" 2024-09-10T09:33:22.313 INFO:teuthology.orchestra.run.smithi077.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-10T09:33:22.313 INFO:teuthology.orchestra.run.smithi077.stdout:ANSI_COLOR="0;31" 2024-09-10T09:33:22.313 INFO:teuthology.orchestra.run.smithi077.stdout:LOGO="fedora-logo-icon" 2024-09-10T09:33:22.313 INFO:teuthology.orchestra.run.smithi077.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-10T09:33:22.313 INFO:teuthology.orchestra.run.smithi077.stdout:HOME_URL="https://centos.org/" 2024-09-10T09:33:22.313 INFO:teuthology.orchestra.run.smithi077.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-10T09:33:22.313 INFO:teuthology.orchestra.run.smithi077.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-10T09:33:22.313 INFO:teuthology.orchestra.run.smithi077.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-10T09:33:22.314 INFO:teuthology.lock.ops:Updating smithi077.front.sepia.ceph.com on lock server 2024-09-10T09:33:22.334 DEBUG:teuthology.orchestra.run.smithi120:> uname -m 2024-09-10T09:33:22.352 INFO:teuthology.orchestra.run.smithi120.stdout:x86_64 2024-09-10T09:33:22.353 DEBUG:teuthology.orchestra.run.smithi120:> cat /etc/os-release 2024-09-10T09:33:22.409 INFO:teuthology.orchestra.run.smithi120.stdout:NAME="CentOS Stream" 2024-09-10T09:33:22.410 INFO:teuthology.orchestra.run.smithi120.stdout:VERSION="9" 2024-09-10T09:33:22.410 INFO:teuthology.orchestra.run.smithi120.stdout:ID="centos" 2024-09-10T09:33:22.410 INFO:teuthology.orchestra.run.smithi120.stdout:ID_LIKE="rhel fedora" 2024-09-10T09:33:22.410 INFO:teuthology.orchestra.run.smithi120.stdout:VERSION_ID="9" 2024-09-10T09:33:22.410 INFO:teuthology.orchestra.run.smithi120.stdout:PLATFORM_ID="platform:el9" 2024-09-10T09:33:22.410 INFO:teuthology.orchestra.run.smithi120.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-10T09:33:22.410 INFO:teuthology.orchestra.run.smithi120.stdout:ANSI_COLOR="0;31" 2024-09-10T09:33:22.410 INFO:teuthology.orchestra.run.smithi120.stdout:LOGO="fedora-logo-icon" 2024-09-10T09:33:22.411 INFO:teuthology.orchestra.run.smithi120.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-10T09:33:22.411 INFO:teuthology.orchestra.run.smithi120.stdout:HOME_URL="https://centos.org/" 2024-09-10T09:33:22.411 INFO:teuthology.orchestra.run.smithi120.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-10T09:33:22.411 INFO:teuthology.orchestra.run.smithi120.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-10T09:33:22.411 INFO:teuthology.orchestra.run.smithi120.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-10T09:33:22.411 INFO:teuthology.lock.ops:Updating smithi120.front.sepia.ceph.com on lock server 2024-09-10T09:33:22.434 DEBUG:teuthology.orchestra.run.smithi134:> uname -m 2024-09-10T09:33:22.452 INFO:teuthology.orchestra.run.smithi134.stdout:x86_64 2024-09-10T09:33:22.452 DEBUG:teuthology.orchestra.run.smithi134:> cat /etc/os-release 2024-09-10T09:33:22.511 INFO:teuthology.orchestra.run.smithi134.stdout:NAME="CentOS Stream" 2024-09-10T09:33:22.511 INFO:teuthology.orchestra.run.smithi134.stdout:VERSION="9" 2024-09-10T09:33:22.511 INFO:teuthology.orchestra.run.smithi134.stdout:ID="centos" 2024-09-10T09:33:22.511 INFO:teuthology.orchestra.run.smithi134.stdout:ID_LIKE="rhel fedora" 2024-09-10T09:33:22.511 INFO:teuthology.orchestra.run.smithi134.stdout:VERSION_ID="9" 2024-09-10T09:33:22.511 INFO:teuthology.orchestra.run.smithi134.stdout:PLATFORM_ID="platform:el9" 2024-09-10T09:33:22.511 INFO:teuthology.orchestra.run.smithi134.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-10T09:33:22.511 INFO:teuthology.orchestra.run.smithi134.stdout:ANSI_COLOR="0;31" 2024-09-10T09:33:22.511 INFO:teuthology.orchestra.run.smithi134.stdout:LOGO="fedora-logo-icon" 2024-09-10T09:33:22.512 INFO:teuthology.orchestra.run.smithi134.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-10T09:33:22.512 INFO:teuthology.orchestra.run.smithi134.stdout:HOME_URL="https://centos.org/" 2024-09-10T09:33:22.512 INFO:teuthology.orchestra.run.smithi134.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-10T09:33:22.512 INFO:teuthology.orchestra.run.smithi134.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-10T09:33:22.512 INFO:teuthology.orchestra.run.smithi134.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-10T09:33:22.512 INFO:teuthology.lock.ops:Updating smithi134.front.sepia.ceph.com on lock server 2024-09-10T09:33:22.534 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-10T09:33:22.544 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-10T09:33:22.550 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-10T09:33:22.550 DEBUG:teuthology.orchestra.run.smithi077:> test '!' -e /home/ubuntu/cephtest 2024-09-10T09:33:22.553 DEBUG:teuthology.orchestra.run.smithi120:> test '!' -e /home/ubuntu/cephtest 2024-09-10T09:33:22.555 DEBUG:teuthology.orchestra.run.smithi134:> test '!' -e /home/ubuntu/cephtest 2024-09-10T09:33:22.572 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-10T09:33:22.578 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-10T09:33:22.578 DEBUG:teuthology.orchestra.run.smithi077:> test -z $(ls -A /var/lib/ceph) 2024-09-10T09:33:22.610 DEBUG:teuthology.orchestra.run.smithi120:> test -z $(ls -A /var/lib/ceph) 2024-09-10T09:33:22.613 DEBUG:teuthology.orchestra.run.smithi134:> test -z $(ls -A /var/lib/ceph) 2024-09-10T09:33:22.892 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-10T09:33:22.960 INFO:teuthology.run_tasks:Running task kernel... 2024-09-10T09:33:22.976 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'} 2024-09-10T09:33:22.976 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-09-10T09:33:22.976 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-09-10T09:33:22.976 DEBUG:teuthology.orchestra.run.smithi077:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T09:33:22.993 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T09:33:22.994 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-09-10T09:33:23.201 DEBUG:teuthology.task.kernel:sha1 for client.0 is 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T09:33:23.201 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "2ca44a2f35ca8f36983aaaa46d49312239ef519a"... 2024-09-10T09:33:23.201 DEBUG:teuthology.orchestra.run.smithi077:> uname -r 2024-09-10T09:33:23.219 INFO:teuthology.orchestra.run.smithi077.stdout:5.14.0-503.el9.x86_64 2024-09-10T09:33:23.219 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-503.el9.x86_64 vs 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T09:33:23.220 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-09-10T09:33:23.220 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum upgrade -y linux-firmware 2024-09-10T09:33:28.038 INFO:teuthology.orchestra.run.smithi077.stdout:CentOS Stream 9 - BaseOS 2.7 MB/s | 8.2 MB 00:03 2024-09-10T09:33:31.180 INFO:teuthology.orchestra.run.smithi077.stdout:CentOS Stream 9 - AppStream 10 MB/s | 20 MB 00:02 2024-09-10T09:33:37.656 INFO:teuthology.orchestra.run.smithi077.stdout:CentOS Stream 9 - CRB 2.4 MB/s | 6.5 MB 00:02 2024-09-10T09:33:39.057 INFO:teuthology.orchestra.run.smithi077.stdout:CentOS Stream 9 - Extras packages 99 kB/s | 19 kB 00:00 2024-09-10T09:33:39.944 INFO:teuthology.orchestra.run.smithi077.stdout:Extra Packages for Enterprise Linux 30 MB/s | 23 MB 00:00 2024-09-10T09:33:45.408 INFO:teuthology.orchestra.run.smithi077.stdout:lab-extras 34 kB/s | 1.7 kB 00:00 2024-09-10T09:33:47.487 INFO:teuthology.orchestra.run.smithi077.stdout:Dependencies resolved. 2024-09-10T09:33:47.488 INFO:teuthology.orchestra.run.smithi077.stdout:Nothing to do. 2024-09-10T09:33:47.488 INFO:teuthology.orchestra.run.smithi077.stdout:Complete! 2024-09-10T09:33:47.592 INFO:teuthology.task.kernel:Downloading kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.0... 2024-09-10T09:33:47.592 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T09:33:47.736 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/ 2024-09-10T09:33:47.737 DEBUG:teuthology.orchestra.run.smithi077:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.11.0_rc5_g2ca44a2f35ca-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/ --input-file=- 2024-09-10T09:33:47.739 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi077.front.sepia.ceph.com to complete... 2024-09-10T09:33:54.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10 09:33:54 URL:https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/kernel-6.11.0_rc5_g2ca44a2f35ca-1.x86_64.rpm [359633503/359633503] -> "/tmp/kernel.x86_64.rpm" [1] 2024-09-10T09:33:54.074 INFO:teuthology.orchestra.run.smithi077.stderr:FINISHED --2024-09-10 09:33:54-- 2024-09-10T09:33:54.074 INFO:teuthology.orchestra.run.smithi077.stderr:Total wall clock time: 6.3s 2024-09-10T09:33:54.074 INFO:teuthology.orchestra.run.smithi077.stderr:Downloaded: 1 files, 343M in 6.0s (57.1 MB/s) 2024-09-10T09:33:54.077 INFO:teuthology.task.kernel:Installing kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.0... 2024-09-10T09:33:54.077 DEBUG:teuthology.orchestra.run.smithi077:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-09-10T09:33:56.174 INFO:teuthology.orchestra.run.smithi077.stdout:Verifying... ######################################## 2024-09-10T09:33:56.438 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing... ######################################## 2024-09-10T09:33:56.553 INFO:teuthology.orchestra.run.smithi077.stdout:Updating / installing... 2024-09-10T09:34:04.573 INFO:teuthology.orchestra.run.smithi077.stdout:kernel-6.11.0_rc5_g2ca44a2f35ca-1 ######################################## 2024-09-10T09:34:44.361 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi077.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-09-10T09:34:44.361 DEBUG:teuthology.orchestra.run.smithi077:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-09-10T09:34:44.454 INFO:teuthology.orchestra.run.smithi077.stdout:/boot/System.map-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T09:34:44.454 INFO:teuthology.orchestra.run.smithi077.stdout:/boot/config-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T09:34:44.454 INFO:teuthology.orchestra.run.smithi077.stdout:/boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T09:34:44.454 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca 2024-09-10T09:34:44.454 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/System.map 2024-09-10T09:34:44.454 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/config 2024-09-10T09:34:44.454 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel 2024-09-10T09:34:44.454 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch 2024-09-10T09:34:44.455 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86 2024-09-10T09:34:44.455 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto 2024-09-10T09:34:44.455 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-09-10T09:34:44.455 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-09-10T09:34:44.455 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-09-10T09:34:44.455 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-09-10T09:34:44.455 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-09-10T09:34:44.455 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-09-10T09:34:44.455 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-09-10T09:34:44.455 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32c-intel.ko 2024-09-10T09:34:44.455 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-09-10T09:34:44.455 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-09-10T09:34:44.455 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-09-10T09:34:44.456 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-09-10T09:34:44.456 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx2.ko 2024-09-10T09:34:44.456 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-09-10T09:34:44.456 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-09-10T09:34:44.456 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-09-10T09:34:44.456 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-09-10T09:34:44.456 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-09-10T09:34:44.456 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-09-10T09:34:44.456 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-09-10T09:34:44.456 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel 2024-09-10T09:34:44.456 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu 2024-09-10T09:34:44.456 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce 2024-09-10T09:34:44.457 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-09-10T09:34:44.457 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm 2024-09-10T09:34:44.457 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-amd.ko 2024-09-10T09:34:44.457 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-intel.ko 2024-09-10T09:34:44.457 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm.ko 2024-09-10T09:34:44.457 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto 2024-09-10T09:34:44.457 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ansi_cprng.ko 2024-09-10T09:34:44.457 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/anubis.ko 2024-09-10T09:34:44.457 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/arc4.ko 2024-09-10T09:34:44.457 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx 2024-09-10T09:34:44.457 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_memcpy.ko 2024-09-10T09:34:44.457 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_pq.ko 2024-09-10T09:34:44.458 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_raid6_recov.ko 2024-09-10T09:34:44.458 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_tx.ko 2024-09-10T09:34:44.458 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_xor.ko 2024-09-10T09:34:44.458 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/raid6test.ko 2024-09-10T09:34:44.458 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authenc.ko 2024-09-10T09:34:44.458 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authencesn.ko 2024-09-10T09:34:44.458 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blake2b_generic.ko 2024-09-10T09:34:44.458 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_common.ko 2024-09-10T09:34:44.458 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_generic.ko 2024-09-10T09:34:44.458 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/camellia_generic.ko 2024-09-10T09:34:44.458 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast5_generic.ko 2024-09-10T09:34:44.458 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast6_generic.ko 2024-09-10T09:34:44.458 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast_common.ko 2024-09-10T09:34:44.458 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ccm.ko 2024-09-10T09:34:44.459 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cmac.ko 2024-09-10T09:34:44.459 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crc32_generic.ko 2024-09-10T09:34:44.459 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_engine.ko 2024-09-10T09:34:44.459 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_user.ko 2024-09-10T09:34:44.459 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cts.ko 2024-09-10T09:34:44.459 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/des_generic.ko 2024-09-10T09:34:44.459 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/dh_generic.ko 2024-09-10T09:34:44.459 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ecc.ko 2024-09-10T09:34:44.459 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ecdh_generic.ko 2024-09-10T09:34:44.460 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/echainiv.ko 2024-09-10T09:34:44.460 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/essiv.ko 2024-09-10T09:34:44.460 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/fcrypt.ko 2024-09-10T09:34:44.460 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/khazad.ko 2024-09-10T09:34:44.460 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4.ko 2024-09-10T09:34:44.460 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4hc.ko 2024-09-10T09:34:44.460 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/md4.ko 2024-09-10T09:34:44.460 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/michael_mic.ko 2024-09-10T09:34:44.461 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcbc.ko 2024-09-10T09:34:44.461 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcrypt.ko 2024-09-10T09:34:44.461 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/rmd160.ko 2024-09-10T09:34:44.461 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/seed.ko 2024-09-10T09:34:44.461 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/serpent_generic.ko 2024-09-10T09:34:44.461 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tcrypt.ko 2024-09-10T09:34:44.461 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tea.ko 2024-09-10T09:34:44.461 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_common.ko 2024-09-10T09:34:44.461 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_generic.ko 2024-09-10T09:34:44.461 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/vmac.ko 2024-09-10T09:34:44.461 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/wp512.ko 2024-09-10T09:34:44.461 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xcbc.ko 2024-09-10T09:34:44.462 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xor.ko 2024-09-10T09:34:44.462 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xxhash_generic.ko 2024-09-10T09:34:44.462 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers 2024-09-10T09:34:44.462 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi 2024-09-10T09:34:44.462 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_ipmi.ko 2024-09-10T09:34:44.462 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_pad.ko 2024-09-10T09:34:44.462 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ec_sys.ko 2024-09-10T09:34:44.462 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/platform_profile.ko 2024-09-10T09:34:44.462 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbs.ko 2024-09-10T09:34:44.462 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbshc.ko 2024-09-10T09:34:44.462 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/video.ko 2024-09-10T09:34:44.462 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata 2024-09-10T09:34:44.462 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/acard-ahci.ko 2024-09-10T09:34:44.463 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci_platform.ko 2024-09-10T09:34:44.463 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ata_generic.ko 2024-09-10T09:34:44.463 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci_platform.ko 2024-09-10T09:34:44.463 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_acpi.ko 2024-09-10T09:34:44.463 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ali.ko 2024-09-10T09:34:44.463 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_amd.ko 2024-09-10T09:34:44.463 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_artop.ko 2024-09-10T09:34:44.463 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atiixp.ko 2024-09-10T09:34:44.463 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atp867x.ko 2024-09-10T09:34:44.463 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd640.ko 2024-09-10T09:34:44.464 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd64x.ko 2024-09-10T09:34:44.464 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cypress.ko 2024-09-10T09:34:44.464 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_efar.ko 2024-09-10T09:34:44.464 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt366.ko 2024-09-10T09:34:44.464 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt37x.ko 2024-09-10T09:34:44.464 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x2n.ko 2024-09-10T09:34:44.464 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x3.ko 2024-09-10T09:34:44.464 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it8213.ko 2024-09-10T09:34:44.464 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it821x.ko 2024-09-10T09:34:44.464 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_jmicron.ko 2024-09-10T09:34:44.464 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_marvell.ko 2024-09-10T09:34:44.465 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_mpiix.ko 2024-09-10T09:34:44.465 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_netcell.ko 2024-09-10T09:34:44.465 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ninja32.ko 2024-09-10T09:34:44.465 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87410.ko 2024-09-10T09:34:44.465 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87415.ko 2024-09-10T09:34:44.465 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_oldpiix.ko 2024-09-10T09:34:44.465 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_opti.ko 2024-09-10T09:34:44.465 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_optidma.ko 2024-09-10T09:34:44.465 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pcmcia.ko 2024-09-10T09:34:44.465 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc2027x.ko 2024-09-10T09:34:44.465 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-09-10T09:34:44.465 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_piccolo.ko 2024-09-10T09:34:44.465 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rdc.ko 2024-09-10T09:34:44.466 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sch.ko 2024-09-10T09:34:44.466 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_serverworks.ko 2024-09-10T09:34:44.466 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sil680.ko 2024-09-10T09:34:44.466 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sis.ko 2024-09-10T09:34:44.466 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sl82c105.ko 2024-09-10T09:34:44.466 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_triflex.ko 2024-09-10T09:34:44.466 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_via.ko 2024-09-10T09:34:44.466 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pdc_adma.ko 2024-09-10T09:34:44.466 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_inic162x.ko 2024-09-10T09:34:44.466 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_mv.ko 2024-09-10T09:34:44.466 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_nv.ko 2024-09-10T09:34:44.466 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_promise.ko 2024-09-10T09:34:44.466 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_qstor.ko 2024-09-10T09:34:44.466 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil.ko 2024-09-10T09:34:44.467 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil24.ko 2024-09-10T09:34:44.467 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sis.ko 2024-09-10T09:34:44.467 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_svw.ko 2024-09-10T09:34:44.467 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sx4.ko 2024-09-10T09:34:44.467 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_uli.ko 2024-09-10T09:34:44.467 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_via.ko 2024-09-10T09:34:44.467 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_vsc.ko 2024-09-10T09:34:44.467 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay 2024-09-10T09:34:44.467 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ks0108.ko 2024-09-10T09:34:44.467 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base 2024-09-10T09:34:44.467 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap 2024-09-10T09:34:44.467 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap/regmap-i2c.ko 2024-09-10T09:34:44.467 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap/regmap-mmio.ko 2024-09-10T09:34:44.467 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma 2024-09-10T09:34:44.468 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/bcma.ko 2024-09-10T09:34:44.468 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block 2024-09-10T09:34:44.468 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe 2024-09-10T09:34:44.468 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/aoe.ko 2024-09-10T09:34:44.468 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/brd.ko 2024-09-10T09:34:44.468 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd 2024-09-10T09:34:44.468 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/drbd.ko 2024-09-10T09:34:44.468 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/floppy.ko 2024-09-10T09:34:44.468 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/loop.ko 2024-09-10T09:34:44.468 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx 2024-09-10T09:34:44.468 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-09-10T09:34:44.468 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/nbd.ko 2024-09-10T09:34:44.468 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk 2024-09-10T09:34:44.468 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/null_blk.ko 2024-09-10T09:34:44.469 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/pktcdvd.ko 2024-09-10T09:34:44.469 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/rbd.ko 2024-09-10T09:34:44.469 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/virtio_blk.ko 2024-09-10T09:34:44.469 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback 2024-09-10T09:34:44.469 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-09-10T09:34:44.469 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkfront.ko 2024-09-10T09:34:44.469 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram 2024-09-10T09:34:44.469 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/zram.ko 2024-09-10T09:34:44.469 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char 2024-09-10T09:34:44.469 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hangcheck-timer.ko 2024-09-10T09:34:44.469 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random 2024-09-10T09:34:44.469 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/amd-rng.ko 2024-09-10T09:34:44.469 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/intel-rng.ko 2024-09-10T09:34:44.470 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-09-10T09:34:44.470 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/via-rng.ko 2024-09-10T09:34:44.470 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/virtio-rng.ko 2024-09-10T09:34:44.470 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi 2024-09-10T09:34:44.470 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-09-10T09:34:44.470 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-09-10T09:34:44.470 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-09-10T09:34:44.470 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_si.ko 2024-09-10T09:34:44.470 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-09-10T09:34:44.470 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-09-10T09:34:44.470 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/lp.ko 2024-09-10T09:34:44.470 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave 2024-09-10T09:34:44.470 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/mwave.ko 2024-09-10T09:34:44.470 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ppdev.ko 2024-09-10T09:34:44.471 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tlclk.ko 2024-09-10T09:34:44.471 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm 2024-09-10T09:34:44.471 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm.ko 2024-09-10T09:34:44.471 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_atmel.ko 2024-09-10T09:34:44.471 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_infineon.ko 2024-09-10T09:34:44.471 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_nsc.ko 2024-09-10T09:34:44.471 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_tis.ko 2024-09-10T09:34:44.471 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-09-10T09:34:44.471 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/uv_mmtimer.ko 2024-09-10T09:34:44.471 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/virtio_console.ko 2024-09-10T09:34:44.471 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq 2024-09-10T09:34:44.471 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-09-10T09:34:44.471 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-09-10T09:34:44.472 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/p4-clockmod.ko 2024-09-10T09:34:44.472 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-09-10T09:34:44.472 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/powernow-k8.ko 2024-09-10T09:34:44.472 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/speedstep-lib.ko 2024-09-10T09:34:44.472 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto 2024-09-10T09:34:44.472 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp 2024-09-10T09:34:44.472 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-09-10T09:34:44.472 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp.ko 2024-09-10T09:34:44.472 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel 2024-09-10T09:34:44.472 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat 2024-09-10T09:34:44.472 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common 2024-09-10T09:34:44.472 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-09-10T09:34:44.472 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-09-10T09:34:44.473 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-09-10T09:34:44.473 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-aes.ko 2024-09-10T09:34:44.473 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-sha.ko 2024-09-10T09:34:44.473 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio 2024-09-10T09:34:44.473 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-09-10T09:34:44.473 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca 2024-09-10T09:34:44.473 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/dca.ko 2024-09-10T09:34:44.473 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/devfreq 2024-09-10T09:34:44.473 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-09-10T09:34:44.473 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma 2024-09-10T09:34:44.473 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw 2024-09-10T09:34:44.473 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/dw_dmac.ko 2024-09-10T09:34:44.473 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat 2024-09-10T09:34:44.474 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ioatdma.ko 2024-09-10T09:34:44.474 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac 2024-09-10T09:34:44.474 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/amd64_edac.ko 2024-09-10T09:34:44.474 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/e752x_edac.ko 2024-09-10T09:34:44.474 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/edac_mce_amd.ko 2024-09-10T09:34:44.474 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3000_edac.ko 2024-09-10T09:34:44.474 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3200_edac.ko 2024-09-10T09:34:44.474 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5100_edac.ko 2024-09-10T09:34:44.474 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5400_edac.ko 2024-09-10T09:34:44.474 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7300_edac.ko 2024-09-10T09:34:44.474 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7core_edac.ko 2024-09-10T09:34:44.474 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i82975x_edac.ko 2024-09-10T09:34:44.474 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ie31200_edac.ko 2024-09-10T09:34:44.475 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/sb_edac.ko 2024-09-10T09:34:44.475 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/x38_edac.ko 2024-09-10T09:34:44.475 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware 2024-09-10T09:34:44.475 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/edd.ko 2024-09-10T09:34:44.475 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/iscsi_ibft.ko 2024-09-10T09:34:44.475 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio 2024-09-10T09:34:44.475 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ich.ko 2024-09-10T09:34:44.475 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-viperboard.ko 2024-09-10T09:34:44.475 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid 2024-09-10T09:34:44.476 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-appleir.ko 2024-09-10T09:34:44.476 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-aureal.ko 2024-09-10T09:34:44.476 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-axff.ko 2024-09-10T09:34:44.476 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-dr.ko 2024-09-10T09:34:44.476 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elecom.ko 2024-09-10T09:34:44.476 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elo.ko 2024-09-10T09:34:44.476 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-emsff.ko 2024-09-10T09:34:44.476 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gaff.ko 2024-09-10T09:34:44.476 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gt683r.ko 2024-09-10T09:34:44.476 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gyration.ko 2024-09-10T09:34:44.476 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-kbd.ko 2024-09-10T09:34:44.476 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-mouse.ko 2024-09-10T09:34:44.476 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtekff.ko 2024-09-10T09:34:44.477 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-hyperv.ko 2024-09-10T09:34:44.477 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-icade.ko 2024-09-10T09:34:44.477 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-keytouch.ko 2024-09-10T09:34:44.477 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kye.ko 2024-09-10T09:34:44.477 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lcpower.ko 2024-09-10T09:34:44.477 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-led.ko 2024-09-10T09:34:44.477 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lenovo.ko 2024-09-10T09:34:44.477 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-dj.ko 2024-09-10T09:34:44.477 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-09-10T09:34:44.477 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-multitouch.ko 2024-09-10T09:34:44.477 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ortek.ko 2024-09-10T09:34:44.477 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-penmount.ko 2024-09-10T09:34:44.478 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-petalynx.ko 2024-09-10T09:34:44.478 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-picolcd.ko 2024-09-10T09:34:44.478 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-pl.ko 2024-09-10T09:34:44.478 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-plantronics.ko 2024-09-10T09:34:44.478 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-primax.ko 2024-09-10T09:34:44.478 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-rmi.ko 2024-09-10T09:34:44.478 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-arvo.ko 2024-09-10T09:34:44.478 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-common.ko 2024-09-10T09:34:44.478 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-isku.ko 2024-09-10T09:34:44.478 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kone.ko 2024-09-10T09:34:44.478 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-09-10T09:34:44.479 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-konepure.ko 2024-09-10T09:34:44.479 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-09-10T09:34:44.479 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-lua.ko 2024-09-10T09:34:44.479 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-pyra.ko 2024-09-10T09:34:44.479 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-ryos.ko 2024-09-10T09:34:44.479 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-savu.ko 2024-09-10T09:34:44.479 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat.ko 2024-09-10T09:34:44.479 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-saitek.ko 2024-09-10T09:34:44.479 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-samsung.ko 2024-09-10T09:34:44.479 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sensor-hub.ko 2024-09-10T09:34:44.479 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sjoy.ko 2024-09-10T09:34:44.479 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sony.ko 2024-09-10T09:34:44.479 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-speedlink.ko 2024-09-10T09:34:44.480 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-steelseries.ko 2024-09-10T09:34:44.480 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sunplus.ko 2024-09-10T09:34:44.480 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-thrustmaster.ko 2024-09-10T09:34:44.480 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tivo.ko 2024-09-10T09:34:44.480 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tmff.ko 2024-09-10T09:34:44.480 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-topseed.ko 2024-09-10T09:34:44.480 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-twinhan.ko 2024-09-10T09:34:44.480 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-uclogic.ko 2024-09-10T09:34:44.480 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-waltop.ko 2024-09-10T09:34:44.480 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-wiimote.ko 2024-09-10T09:34:44.481 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-xinmo.ko 2024-09-10T09:34:44.482 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zpff.ko 2024-09-10T09:34:44.482 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zydacron.ko 2024-09-10T09:34:44.482 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/uhid.ko 2024-09-10T09:34:44.482 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/wacom.ko 2024-09-10T09:34:44.482 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv 2024-09-10T09:34:44.482 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_balloon.ko 2024-09-10T09:34:44.482 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_utils.ko 2024-09-10T09:34:44.482 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_vmbus.ko 2024-09-10T09:34:44.482 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon 2024-09-10T09:34:44.482 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru.ko 2024-09-10T09:34:44.482 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru3.ko 2024-09-10T09:34:44.482 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/acpi_power_meter.ko 2024-09-10T09:34:44.483 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7414.ko 2024-09-10T09:34:44.483 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7418.ko 2024-09-10T09:34:44.483 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adc128d818.ko 2024-09-10T09:34:44.483 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1025.ko 2024-09-10T09:34:44.483 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1026.ko 2024-09-10T09:34:44.483 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1029.ko 2024-09-10T09:34:44.483 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1031.ko 2024-09-10T09:34:44.483 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm9240.ko 2024-09-10T09:34:44.483 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ads7828.ko 2024-09-10T09:34:44.483 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7410.ko 2024-09-10T09:34:44.483 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7411.ko 2024-09-10T09:34:44.483 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7462.ko 2024-09-10T09:34:44.483 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7470.ko 2024-09-10T09:34:44.483 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7475.ko 2024-09-10T09:34:44.484 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7x10.ko 2024-09-10T09:34:44.484 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/amc6821.ko 2024-09-10T09:34:44.484 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/applesmc.ko 2024-09-10T09:34:44.484 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asb100.ko 2024-09-10T09:34:44.484 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asc7621.ko 2024-09-10T09:34:44.484 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asus_atk0110.ko 2024-09-10T09:34:44.484 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/atxp1.ko 2024-09-10T09:34:44.484 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/coretemp.ko 2024-09-10T09:34:44.484 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-09-10T09:34:44.484 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dme1737.ko 2024-09-10T09:34:44.484 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds1621.ko 2024-09-10T09:34:44.484 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds620.ko 2024-09-10T09:34:44.484 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc1403.ko 2024-09-10T09:34:44.485 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc6w201.ko 2024-09-10T09:34:44.485 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71805f.ko 2024-09-10T09:34:44.485 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71882fg.ko 2024-09-10T09:34:44.485 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f75375s.ko 2024-09-10T09:34:44.485 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fam15h_power.ko 2024-09-10T09:34:44.485 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fschmd.ko 2024-09-10T09:34:44.485 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g760a.ko 2024-09-10T09:34:44.485 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g762.ko 2024-09-10T09:34:44.485 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl518sm.ko 2024-09-10T09:34:44.485 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl520sm.ko 2024-09-10T09:34:44.485 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hwmon-vid.ko 2024-09-10T09:34:44.485 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5500_temp.ko 2024-09-10T09:34:44.485 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5k_amb.ko 2024-09-10T09:34:44.485 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmaem.ko 2024-09-10T09:34:44.486 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmpex.ko 2024-09-10T09:34:44.486 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina209.ko 2024-09-10T09:34:44.486 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina2xx.ko 2024-09-10T09:34:44.486 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/it87.ko 2024-09-10T09:34:44.486 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k10temp.ko 2024-09-10T09:34:44.486 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k8temp.ko 2024-09-10T09:34:44.486 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lineage-pem.ko 2024-09-10T09:34:44.486 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm63.ko 2024-09-10T09:34:44.486 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm73.ko 2024-09-10T09:34:44.486 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm75.ko 2024-09-10T09:34:44.486 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm77.ko 2024-09-10T09:34:44.486 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm78.ko 2024-09-10T09:34:44.486 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm80.ko 2024-09-10T09:34:44.486 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm83.ko 2024-09-10T09:34:44.487 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm85.ko 2024-09-10T09:34:44.487 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm87.ko 2024-09-10T09:34:44.487 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm90.ko 2024-09-10T09:34:44.487 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm92.ko 2024-09-10T09:34:44.487 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm93.ko 2024-09-10T09:34:44.487 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95234.ko 2024-09-10T09:34:44.487 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95241.ko 2024-09-10T09:34:44.487 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95245.ko 2024-09-10T09:34:44.487 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc2945.ko 2024-09-10T09:34:44.487 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4151.ko 2024-09-10T09:34:44.487 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4215.ko 2024-09-10T09:34:44.487 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4222.ko 2024-09-10T09:34:44.487 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4245.ko 2024-09-10T09:34:44.487 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4260.ko 2024-09-10T09:34:44.488 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4261.ko 2024-09-10T09:34:44.488 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max16065.ko 2024-09-10T09:34:44.488 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1619.ko 2024-09-10T09:34:44.488 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1668.ko 2024-09-10T09:34:44.488 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max197.ko 2024-09-10T09:34:44.488 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6639.ko 2024-09-10T09:34:44.488 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6650.ko 2024-09-10T09:34:44.488 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6697.ko 2024-09-10T09:34:44.488 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mcp3021.ko 2024-09-10T09:34:44.488 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6683.ko 2024-09-10T09:34:44.488 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775-core.ko 2024-09-10T09:34:44.488 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775.ko 2024-09-10T09:34:44.488 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct7802.ko 2024-09-10T09:34:44.488 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87360.ko 2024-09-10T09:34:44.489 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87427.ko 2024-09-10T09:34:44.489 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pcf8591.ko 2024-09-10T09:34:44.489 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus 2024-09-10T09:34:44.489 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-09-10T09:34:44.489 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-09-10T09:34:44.489 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-09-10T09:34:44.489 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max16064.ko 2024-09-10T09:34:44.489 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max34440.ko 2024-09-10T09:34:44.489 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max8688.ko 2024-09-10T09:34:44.489 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-09-10T09:34:44.489 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-09-10T09:34:44.489 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-09-10T09:34:44.489 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-09-10T09:34:44.490 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-09-10T09:34:44.490 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-09-10T09:34:44.490 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/powr1220.ko 2024-09-10T09:34:44.490 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5627.ko 2024-09-10T09:34:44.490 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5636.ko 2024-09-10T09:34:44.490 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch56xx-common.ko 2024-09-10T09:34:44.490 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht15.ko 2024-09-10T09:34:44.490 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht21.ko 2024-09-10T09:34:44.490 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/shtc1.ko 2024-09-10T09:34:44.490 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sis5595.ko 2024-09-10T09:34:44.490 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47b397.ko 2024-09-10T09:34:44.490 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m1.ko 2024-09-10T09:34:44.490 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m192.ko 2024-09-10T09:34:44.490 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/thmc50.ko 2024-09-10T09:34:44.490 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp102.ko 2024-09-10T09:34:44.491 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp103.ko 2024-09-10T09:34:44.491 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp401.ko 2024-09-10T09:34:44.491 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp421.ko 2024-09-10T09:34:44.491 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via-cputemp.ko 2024-09-10T09:34:44.491 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via686a.ko 2024-09-10T09:34:44.491 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt1211.ko 2024-09-10T09:34:44.491 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt8231.ko 2024-09-10T09:34:44.491 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627ehf.ko 2024-09-10T09:34:44.491 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627hf.ko 2024-09-10T09:34:44.491 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83781d.ko 2024-09-10T09:34:44.491 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83791d.ko 2024-09-10T09:34:44.491 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83792d.ko 2024-09-10T09:34:44.491 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83793.ko 2024-09-10T09:34:44.491 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83795.ko 2024-09-10T09:34:44.492 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l785ts.ko 2024-09-10T09:34:44.492 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l786ng.ko 2024-09-10T09:34:44.492 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c 2024-09-10T09:34:44.492 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos 2024-09-10T09:34:44.492 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-09-10T09:34:44.492 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-09-10T09:34:44.492 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses 2024-09-10T09:34:44.492 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-09-10T09:34:44.492 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-09-10T09:34:44.492 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-09-10T09:34:44.492 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-09-10T09:34:44.492 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-09-10T09:34:44.492 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-09-10T09:34:44.492 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-09-10T09:34:44.493 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-09-10T09:34:44.493 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-i801.ko 2024-09-10T09:34:44.493 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-isch.ko 2024-09-10T09:34:44.493 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-09-10T09:34:44.493 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-09-10T09:34:44.493 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-09-10T09:34:44.493 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-parport.ko 2024-09-10T09:34:44.493 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-09-10T09:34:44.493 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-09-10T09:34:44.493 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-09-10T09:34:44.493 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-09-10T09:34:44.493 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-09-10T09:34:44.493 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-09-10T09:34:44.493 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-via.ko 2024-09-10T09:34:44.494 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-09-10T09:34:44.494 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-09-10T09:34:44.494 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-dev.ko 2024-09-10T09:34:44.494 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-mux.ko 2024-09-10T09:34:44.494 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-09-10T09:34:44.494 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-smbus.ko 2024-09-10T09:34:44.494 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-stub.ko 2024-09-10T09:34:44.494 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband 2024-09-10T09:34:44.494 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core 2024-09-10T09:34:44.494 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_cm.ko 2024-09-10T09:34:44.494 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_core.ko 2024-09-10T09:34:44.494 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_umad.ko 2024-09-10T09:34:44.494 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-09-10T09:34:44.494 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/iw_cm.ko 2024-09-10T09:34:44.494 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_cm.ko 2024-09-10T09:34:44.495 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-09-10T09:34:44.495 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw 2024-09-10T09:34:44.495 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4 2024-09-10T09:34:44.495 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-09-10T09:34:44.495 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4 2024-09-10T09:34:44.495 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-09-10T09:34:44.495 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5 2024-09-10T09:34:44.495 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-09-10T09:34:44.495 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca 2024-09-10T09:34:44.495 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-09-10T09:34:44.495 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp 2024-09-10T09:34:44.495 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib 2024-09-10T09:34:44.495 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-09-10T09:34:44.495 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser 2024-09-10T09:34:44.496 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-09-10T09:34:44.496 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert 2024-09-10T09:34:44.496 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-09-10T09:34:44.496 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp 2024-09-10T09:34:44.496 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-09-10T09:34:44.496 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt 2024-09-10T09:34:44.496 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-09-10T09:34:44.496 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input 2024-09-10T09:34:44.496 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport 2024-09-10T09:34:44.496 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-09-10T09:34:44.496 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/fm801-gp.ko 2024-09-10T09:34:44.496 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/gameport.ko 2024-09-10T09:34:44.496 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/joydev.ko 2024-09-10T09:34:44.496 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard 2024-09-10T09:34:44.497 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys.ko 2024-09-10T09:34:44.497 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse 2024-09-10T09:34:44.497 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/appletouch.ko 2024-09-10T09:34:44.497 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/bcm5974.ko 2024-09-10T09:34:44.497 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/cyapatp.ko 2024-09-10T09:34:44.497 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/elan_i2c.ko 2024-09-10T09:34:44.497 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/sermouse.ko 2024-09-10T09:34:44.497 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-09-10T09:34:44.497 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_usb.ko 2024-09-10T09:34:44.497 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/vsxxxaa.ko 2024-09-10T09:34:44.497 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4 2024-09-10T09:34:44.497 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/rmi_core.ko 2024-09-10T09:34:44.497 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio 2024-09-10T09:34:44.497 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/altera_ps2.ko 2024-09-10T09:34:44.497 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/arc_ps2.ko 2024-09-10T09:34:44.498 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-09-10T09:34:44.498 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serio_raw.ko 2024-09-10T09:34:44.498 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/sparse-keymap.ko 2024-09-10T09:34:44.498 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds 2024-09-10T09:34:44.498 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-blinkm.ko 2024-09-10T09:34:44.498 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3530.ko 2024-09-10T09:34:44.498 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp3944.ko 2024-09-10T09:34:44.498 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-ss4200.ko 2024-09-10T09:34:44.498 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger 2024-09-10T09:34:44.498 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-09-10T09:34:44.498 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-09-10T09:34:44.498 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-09-10T09:34:44.498 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-09-10T09:34:44.498 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-09-10T09:34:44.499 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-09-10T09:34:44.499 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-09-10T09:34:44.499 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-09-10T09:34:44.499 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md 2024-09-10T09:34:44.499 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache 2024-09-10T09:34:44.499 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/bcache.ko 2024-09-10T09:34:44.499 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bio-prison.ko 2024-09-10T09:34:44.499 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache-smq.ko 2024-09-10T09:34:44.499 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache.ko 2024-09-10T09:34:44.499 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-crypt.ko 2024-09-10T09:34:44.499 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-delay.ko 2024-09-10T09:34:44.499 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-flakey.ko 2024-09-10T09:34:44.499 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log-userspace.ko 2024-09-10T09:34:44.499 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-multipath.ko 2024-09-10T09:34:44.500 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-queue-length.ko 2024-09-10T09:34:44.500 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-raid.ko 2024-09-10T09:34:44.500 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-round-robin.ko 2024-09-10T09:34:44.500 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-service-time.ko 2024-09-10T09:34:44.500 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-switch.ko 2024-09-10T09:34:44.500 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-thin-pool.ko 2024-09-10T09:34:44.500 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-verity.ko 2024-09-10T09:34:44.500 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data 2024-09-10T09:34:44.500 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-09-10T09:34:44.500 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid0.ko 2024-09-10T09:34:44.500 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid1.ko 2024-09-10T09:34:44.500 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid10.ko 2024-09-10T09:34:44.500 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid456.ko 2024-09-10T09:34:44.501 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message 2024-09-10T09:34:44.501 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion 2024-09-10T09:34:44.501 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptbase.ko 2024-09-10T09:34:44.501 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptctl.ko 2024-09-10T09:34:44.501 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptfc.ko 2024-09-10T09:34:44.501 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptlan.ko 2024-09-10T09:34:44.501 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptsas.ko 2024-09-10T09:34:44.501 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptscsih.ko 2024-09-10T09:34:44.501 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptspi.ko 2024-09-10T09:34:44.501 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd 2024-09-10T09:34:44.501 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_ich.ko 2024-09-10T09:34:44.501 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_sch.ko 2024-09-10T09:34:44.501 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mfd-core.ko 2024-09-10T09:34:44.501 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/sm501.ko 2024-09-10T09:34:44.501 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/viperboard.ko 2024-09-10T09:34:44.502 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/vx855.ko 2024-09-10T09:34:44.502 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wl1273-core.ko 2024-09-10T09:34:44.502 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc 2024-09-10T09:34:44.502 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl 2024-09-10T09:34:44.502 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-09-10T09:34:44.502 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds9802als.ko 2024-09-10T09:34:44.502 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds990x.ko 2024-09-10T09:34:44.502 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/bh1770glc.ko 2024-09-10T09:34:44.502 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710 2024-09-10T09:34:44.502 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/cb710.ko 2024-09-10T09:34:44.502 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom 2024-09-10T09:34:44.502 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/at24.ko 2024-09-10T09:34:44.502 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-09-10T09:34:44.503 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/max6875.ko 2024-09-10T09:34:44.503 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/enclosure.ko 2024-09-10T09:34:44.503 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hpilo.ko 2024-09-10T09:34:44.503 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29003.ko 2024-09-10T09:34:44.503 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29020.ko 2024-09-10T09:34:44.503 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d 2024-09-10T09:34:44.503 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-09-10T09:34:44.503 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-09-10T09:34:44.503 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei 2024-09-10T09:34:44.503 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-me.ko 2024-09-10T09:34:44.503 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-txe.ko 2024-09-10T09:34:44.503 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei.ko 2024-09-10T09:34:44.503 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-gru 2024-09-10T09:34:44.503 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-gru/gru.ko 2024-09-10T09:34:44.504 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp 2024-09-10T09:34:44.504 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xp.ko 2024-09-10T09:34:44.504 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xpc.ko 2024-09-10T09:34:44.504 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-09-10T09:34:44.504 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_7xx1.ko 2024-09-10T09:34:44.504 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_core.ko 2024-09-10T09:34:44.504 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tsl2550.ko 2024-09-10T09:34:44.504 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_balloon.ko 2024-09-10T09:34:44.504 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci 2024-09-10T09:34:44.504 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-09-10T09:34:44.504 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net 2024-09-10T09:34:44.504 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding 2024-09-10T09:34:44.504 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/bonding.ko 2024-09-10T09:34:44.504 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa 2024-09-10T09:34:44.505 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53 2024-09-10T09:34:44.505 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/b53_common.ko 2024-09-10T09:34:44.505 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/bcm-sf2.ko 2024-09-10T09:34:44.505 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6060.ko 2024-09-10T09:34:44.505 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx 2024-09-10T09:34:44.505 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-09-10T09:34:44.505 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dummy.ko 2024-09-10T09:34:44.505 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/eql.ko 2024-09-10T09:34:44.505 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet 2024-09-10T09:34:44.505 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com 2024-09-10T09:34:44.505 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-09-10T09:34:44.505 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-09-10T09:34:44.505 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-09-10T09:34:44.505 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-09-10T09:34:44.506 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390 2024-09-10T09:34:44.506 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/8390.ko 2024-09-10T09:34:44.506 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-09-10T09:34:44.506 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-09-10T09:34:44.506 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-09-10T09:34:44.506 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec 2024-09-10T09:34:44.506 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-09-10T09:34:44.506 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere 2024-09-10T09:34:44.506 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/et131x.ko 2024-09-10T09:34:44.506 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon 2024-09-10T09:34:44.506 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-09-10T09:34:44.506 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera 2024-09-10T09:34:44.506 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-09-10T09:34:44.506 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd 2024-09-10T09:34:44.507 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-09-10T09:34:44.507 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-09-10T09:34:44.507 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-09-10T09:34:44.507 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros 2024-09-10T09:34:44.507 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx 2024-09-10T09:34:44.507 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-09-10T09:34:44.507 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c 2024-09-10T09:34:44.507 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-09-10T09:34:44.507 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e 2024-09-10T09:34:44.507 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-09-10T09:34:44.507 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx 2024-09-10T09:34:44.507 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-09-10T09:34:44.507 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-09-10T09:34:44.507 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom 2024-09-10T09:34:44.508 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-09-10T09:34:44.508 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-09-10T09:34:44.508 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-09-10T09:34:44.508 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-09-10T09:34:44.508 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-09-10T09:34:44.508 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet 2024-09-10T09:34:44.508 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-09-10T09:34:44.508 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-09-10T09:34:44.508 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade 2024-09-10T09:34:44.508 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna 2024-09-10T09:34:44.508 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-09-10T09:34:44.508 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio 2024-09-10T09:34:44.508 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb 2024-09-10T09:34:44.508 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-09-10T09:34:44.509 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-09-10T09:34:44.509 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-09-10T09:34:44.509 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-09-10T09:34:44.509 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-09-10T09:34:44.509 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-09-10T09:34:44.509 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-09-10T09:34:44.509 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-09-10T09:34:44.509 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-09-10T09:34:44.509 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco 2024-09-10T09:34:44.509 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic 2024-09-10T09:34:44.509 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-09-10T09:34:44.509 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec 2024-09-10T09:34:44.509 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip 2024-09-10T09:34:44.509 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-09-10T09:34:44.510 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-09-10T09:34:44.510 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-09-10T09:34:44.510 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-09-10T09:34:44.510 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-09-10T09:34:44.510 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-09-10T09:34:44.510 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink 2024-09-10T09:34:44.510 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-09-10T09:34:44.510 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-09-10T09:34:44.510 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dnet.ko 2024-09-10T09:34:44.510 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex 2024-09-10T09:34:44.510 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet 2024-09-10T09:34:44.510 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-09-10T09:34:44.510 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ethoc.ko 2024-09-10T09:34:44.510 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fealnx.ko 2024-09-10T09:34:44.511 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel 2024-09-10T09:34:44.511 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e100.ko 2024-09-10T09:34:44.511 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000 2024-09-10T09:34:44.511 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-09-10T09:34:44.511 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e 2024-09-10T09:34:44.511 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-09-10T09:34:44.511 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k 2024-09-10T09:34:44.511 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-09-10T09:34:44.511 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e 2024-09-10T09:34:44.511 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-09-10T09:34:44.511 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf 2024-09-10T09:34:44.511 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-09-10T09:34:44.511 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb 2024-09-10T09:34:44.511 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-09-10T09:34:44.511 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf 2024-09-10T09:34:44.512 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-09-10T09:34:44.512 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe 2024-09-10T09:34:44.512 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-09-10T09:34:44.512 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf 2024-09-10T09:34:44.512 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-09-10T09:34:44.512 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth 2024-09-10T09:34:44.512 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/libeth.ko 2024-09-10T09:34:44.512 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie 2024-09-10T09:34:44.512 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/libie.ko 2024-09-10T09:34:44.512 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/jme.ko 2024-09-10T09:34:44.512 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell 2024-09-10T09:34:44.512 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-09-10T09:34:44.512 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/skge.ko 2024-09-10T09:34:44.512 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-09-10T09:34:44.513 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox 2024-09-10T09:34:44.513 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4 2024-09-10T09:34:44.513 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-09-10T09:34:44.513 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-09-10T09:34:44.513 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5 2024-09-10T09:34:44.513 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-09-10T09:34:44.513 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-09-10T09:34:44.513 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel 2024-09-10T09:34:44.513 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-09-10T09:34:44.513 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom 2024-09-10T09:34:44.513 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge 2024-09-10T09:34:44.513 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-09-10T09:34:44.513 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi 2024-09-10T09:34:44.513 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-09-10T09:34:44.514 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-09-10T09:34:44.514 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion 2024-09-10T09:34:44.514 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-09-10T09:34:44.514 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia 2024-09-10T09:34:44.514 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-09-10T09:34:44.514 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines 2024-09-10T09:34:44.514 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-09-10T09:34:44.514 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-09-10T09:34:44.514 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic 2024-09-10T09:34:44.514 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen 2024-09-10T09:34:44.514 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-09-10T09:34:44.514 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-09-10T09:34:44.514 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-09-10T09:34:44.514 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-09-10T09:34:44.515 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc 2024-09-10T09:34:44.515 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-09-10T09:34:44.515 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek 2024-09-10T09:34:44.515 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-09-10T09:34:44.515 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-09-10T09:34:44.515 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/atp.ko 2024-09-10T09:34:44.515 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-09-10T09:34:44.515 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rocker 2024-09-10T09:34:44.515 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-09-10T09:34:44.515 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc 2024-09-10T09:34:44.515 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-09-10T09:34:44.515 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan 2024-09-10T09:34:44.515 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-09-10T09:34:44.515 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis 2024-09-10T09:34:44.515 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis190.ko 2024-09-10T09:34:44.516 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis900.ko 2024-09-10T09:34:44.516 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc 2024-09-10T09:34:44.516 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-09-10T09:34:44.516 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-09-10T09:34:44.516 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-09-10T09:34:44.516 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-09-10T09:34:44.516 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro 2024-09-10T09:34:44.516 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac 2024-09-10T09:34:44.516 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-09-10T09:34:44.516 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-09-10T09:34:44.516 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun 2024-09-10T09:34:44.516 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/cassini.ko 2024-09-10T09:34:44.516 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/niu.ko 2024-09-10T09:34:44.516 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sungem.ko 2024-09-10T09:34:44.517 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-09-10T09:34:44.517 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti 2024-09-10T09:34:44.517 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-09-10T09:34:44.517 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti 2024-09-10T09:34:44.517 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/tlan.ko 2024-09-10T09:34:44.517 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via 2024-09-10T09:34:44.517 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-09-10T09:34:44.517 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-09-10T09:34:44.517 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet 2024-09-10T09:34:44.517 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-09-10T09:34:44.517 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-09-10T09:34:44.517 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom 2024-09-10T09:34:44.517 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-09-10T09:34:44.517 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/geneve.ko 2024-09-10T09:34:44.518 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv 2024-09-10T09:34:44.518 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-09-10T09:34:44.518 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154 2024-09-10T09:34:44.518 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/fakelb.ko 2024-09-10T09:34:44.518 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ifb.ko 2024-09-10T09:34:44.518 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan 2024-09-10T09:34:44.518 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ipvlan.ko 2024-09-10T09:34:44.518 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvlan.ko 2024-09-10T09:34:44.518 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvtap.ko 2024-09-10T09:34:44.518 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio 2024-09-10T09:34:44.518 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio.ko 2024-09-10T09:34:44.518 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-09-10T09:34:44.518 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-09-10T09:34:44.518 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-regmap.ko 2024-09-10T09:34:44.518 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mii.ko 2024-09-10T09:34:44.519 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/net_failover.ko 2024-09-10T09:34:44.519 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/netconsole.ko 2024-09-10T09:34:44.519 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/nlmon.ko 2024-09-10T09:34:44.519 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs 2024-09-10T09:34:44.519 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs-lynx.ko 2024-09-10T09:34:44.519 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-09-10T09:34:44.519 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy 2024-09-10T09:34:44.519 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/amd.ko 2024-09-10T09:34:44.519 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-09-10T09:34:44.519 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-09-10T09:34:44.519 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm7xxx.ko 2024-09-10T09:34:44.519 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm87xx.ko 2024-09-10T09:34:44.519 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/broadcom.ko 2024-09-10T09:34:44.519 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/cicada.ko 2024-09-10T09:34:44.520 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/davicom.ko 2024-09-10T09:34:44.520 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/dp83640.ko 2024-09-10T09:34:44.520 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/et1011c.ko 2024-09-10T09:34:44.520 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/icplus.ko 2024-09-10T09:34:44.520 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/lxt.ko 2024-09-10T09:34:44.520 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/marvell.ko 2024-09-10T09:34:44.520 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/micrel.ko 2024-09-10T09:34:44.520 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/national.ko 2024-09-10T09:34:44.520 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/phylink.ko 2024-09-10T09:34:44.520 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/qsemi.ko 2024-09-10T09:34:44.520 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/realtek.ko 2024-09-10T09:34:44.520 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/smsc.ko 2024-09-10T09:34:44.520 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ste10Xp.ko 2024-09-10T09:34:44.520 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/vitesse.ko 2024-09-10T09:34:44.521 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp 2024-09-10T09:34:44.521 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/bsd_comp.ko 2024-09-10T09:34:44.521 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_async.ko 2024-09-10T09:34:44.521 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_deflate.ko 2024-09-10T09:34:44.521 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_generic.ko 2024-09-10T09:34:44.521 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_mppe.ko 2024-09-10T09:34:44.521 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_synctty.ko 2024-09-10T09:34:44.521 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppoe.ko 2024-09-10T09:34:44.521 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppox.ko 2024-09-10T09:34:44.521 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pptp.ko 2024-09-10T09:34:44.521 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip 2024-09-10T09:34:44.521 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slhc.ko 2024-09-10T09:34:44.521 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slip.ko 2024-09-10T09:34:44.521 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/sungem_phy.ko 2024-09-10T09:34:44.522 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tap.ko 2024-09-10T09:34:44.522 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team 2024-09-10T09:34:44.522 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team.ko 2024-09-10T09:34:44.522 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_activebackup.ko 2024-09-10T09:34:44.522 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_broadcast.ko 2024-09-10T09:34:44.522 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-09-10T09:34:44.522 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_random.ko 2024-09-10T09:34:44.522 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-09-10T09:34:44.522 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tun.ko 2024-09-10T09:34:44.522 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/veth.ko 2024-09-10T09:34:44.522 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/virtio_net.ko 2024-09-10T09:34:44.522 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3 2024-09-10T09:34:44.522 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-09-10T09:34:44.522 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan 2024-09-10T09:34:44.523 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/vxlan.ko 2024-09-10T09:34:44.523 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback 2024-09-10T09:34:44.523 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/xen-netback.ko 2024-09-10T09:34:44.523 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netfront.ko 2024-09-10T09:34:44.523 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb 2024-09-10T09:34:44.523 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ntb.ko 2024-09-10T09:34:44.523 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme 2024-09-10T09:34:44.523 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host 2024-09-10T09:34:44.523 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme-core.ko 2024-09-10T09:34:44.523 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme.ko 2024-09-10T09:34:44.524 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport 2024-09-10T09:34:44.524 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport.ko 2024-09-10T09:34:44.524 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_cs.ko 2024-09-10T09:34:44.524 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_pc.ko 2024-09-10T09:34:44.524 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_serial.ko 2024-09-10T09:34:44.524 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci 2024-09-10T09:34:44.524 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug 2024-09-10T09:34:44.524 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-09-10T09:34:44.524 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pcie 2024-09-10T09:34:44.524 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pcie/aer_inject.ko 2024-09-10T09:34:44.524 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/xen-pcifront.ko 2024-09-10T09:34:44.524 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia 2024-09-10T09:34:44.524 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/i82092.ko 2024-09-10T09:34:44.525 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pd6729.ko 2024-09-10T09:34:44.525 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/yenta_socket.ko 2024-09-10T09:34:44.525 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl 2024-09-10T09:34:44.525 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel 2024-09-10T09:34:44.525 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-09-10T09:34:44.525 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform 2024-09-10T09:34:44.525 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86 2024-09-10T09:34:44.525 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acer-wmi.ko 2024-09-10T09:34:44.525 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acerhdf.ko 2024-09-10T09:34:44.525 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/apple-gmux.ko 2024-09-10T09:34:44.525 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-laptop.ko 2024-09-10T09:34:44.525 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-09-10T09:34:44.525 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-wmi.ko 2024-09-10T09:34:44.525 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/classmate-laptop.ko 2024-09-10T09:34:44.525 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-09-10T09:34:44.526 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-09-10T09:34:44.526 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-09-10T09:34:44.526 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-09-10T09:34:44.526 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/hdaps.ko 2024-09-10T09:34:44.526 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel 2024-09-10T09:34:44.526 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-09-10T09:34:44.526 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel_ips.ko 2024-09-10T09:34:44.526 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/msi-wmi.ko 2024-09-10T09:34:44.526 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/mxm-wmi.ko 2024-09-10T09:34:44.526 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-09-10T09:34:44.526 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-laptop.ko 2024-09-10T09:34:44.526 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-q10.ko 2024-09-10T09:34:44.526 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/topstar-laptop.ko 2024-09-10T09:34:44.526 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-09-10T09:34:44.527 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_haps.ko 2024-09-10T09:34:44.527 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi-bmof.ko 2024-09-10T09:34:44.527 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi.ko 2024-09-10T09:34:44.527 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap 2024-09-10T09:34:44.527 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_common.ko 2024-09-10T09:34:44.527 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_msr.ko 2024-09-10T09:34:44.527 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps 2024-09-10T09:34:44.527 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients 2024-09-10T09:34:44.527 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-gpio.ko 2024-09-10T09:34:44.527 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-ldisc.ko 2024-09-10T09:34:44.527 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps_parport.ko 2024-09-10T09:34:44.527 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/pps_core.ko 2024-09-10T09:34:44.527 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp 2024-09-10T09:34:44.527 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp.ko 2024-09-10T09:34:44.527 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp_kvm.ko 2024-09-10T09:34:44.528 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras 2024-09-10T09:34:44.528 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd 2024-09-10T09:34:44.528 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl 2024-09-10T09:34:44.528 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-09-10T09:34:44.528 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/fmpm.ko 2024-09-10T09:34:44.528 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc 2024-09-10T09:34:44.528 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-bq32k.ko 2024-09-10T09:34:44.528 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1286.ko 2024-09-10T09:34:44.528 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1307.ko 2024-09-10T09:34:44.528 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1374.ko 2024-09-10T09:34:44.528 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1511.ko 2024-09-10T09:34:44.528 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1553.ko 2024-09-10T09:34:44.528 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1672.ko 2024-09-10T09:34:44.528 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1742.ko 2024-09-10T09:34:44.529 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds2404.ko 2024-09-10T09:34:44.529 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds3232.ko 2024-09-10T09:34:44.529 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-em3027.ko 2024-09-10T09:34:44.529 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-fm3130.ko 2024-09-10T09:34:44.529 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl12022.ko 2024-09-10T09:34:44.529 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl1208.ko 2024-09-10T09:34:44.529 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m41t80.ko 2024-09-10T09:34:44.529 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t35.ko 2024-09-10T09:34:44.529 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t59.ko 2024-09-10T09:34:44.529 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max6900.ko 2024-09-10T09:34:44.529 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-msm6242.ko 2024-09-10T09:34:44.529 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf2127.ko 2024-09-10T09:34:44.529 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf85063.ko 2024-09-10T09:34:44.529 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8523.ko 2024-09-10T09:34:44.529 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8563.ko 2024-09-10T09:34:44.530 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8583.ko 2024-09-10T09:34:44.530 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rp5c01.ko 2024-09-10T09:34:44.530 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rs5c372.ko 2024-09-10T09:34:44.530 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-09-10T09:34:44.530 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8025.ko 2024-09-10T09:34:44.530 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8581.ko 2024-09-10T09:34:44.530 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-09-10T09:34:44.530 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-x1205.ko 2024-09-10T09:34:44.530 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi 2024-09-10T09:34:44.530 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-9xxx.ko 2024-09-10T09:34:44.530 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-sas.ko 2024-09-10T09:34:44.530 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-xxxx.ko 2024-09-10T09:34:44.530 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/BusLogic.ko 2024-09-10T09:34:44.530 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/a100u2w.ko 2024-09-10T09:34:44.530 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid 2024-09-10T09:34:44.531 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/aacraid.ko 2024-09-10T09:34:44.531 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/advansys.ko 2024-09-10T09:34:44.531 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx 2024-09-10T09:34:44.531 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-09-10T09:34:44.531 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-09-10T09:34:44.531 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/am53c974.ko 2024-09-10T09:34:44.531 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr 2024-09-10T09:34:44.531 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-09-10T09:34:44.531 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/atp870u.ko 2024-09-10T09:34:44.531 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi 2024-09-10T09:34:44.531 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-09-10T09:34:44.531 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa 2024-09-10T09:34:44.531 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/bfa.ko 2024-09-10T09:34:44.531 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc 2024-09-10T09:34:44.532 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-09-10T09:34:44.532 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i 2024-09-10T09:34:44.532 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-09-10T09:34:44.532 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ch.ko 2024-09-10T09:34:44.532 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor 2024-09-10T09:34:44.532 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/csiostor.ko 2024-09-10T09:34:44.532 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi 2024-09-10T09:34:44.532 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i 2024-09-10T09:34:44.532 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-09-10T09:34:44.532 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i 2024-09-10T09:34:44.532 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-09-10T09:34:44.532 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-09-10T09:34:44.532 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dc395x.ko 2024-09-10T09:34:44.532 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler 2024-09-10T09:34:44.533 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-09-10T09:34:44.533 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-09-10T09:34:44.533 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-09-10T09:34:44.533 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-09-10T09:34:44.533 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dmx3191d.ko 2024-09-10T09:34:44.533 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r 2024-09-10T09:34:44.533 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/esas2r.ko 2024-09-10T09:34:44.533 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esp_scsi.ko 2024-09-10T09:34:44.533 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe 2024-09-10T09:34:44.533 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/fcoe.ko 2024-09-10T09:34:44.533 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-09-10T09:34:44.533 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic 2024-09-10T09:34:44.533 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/fnic.ko 2024-09-10T09:34:44.533 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hpsa.ko 2024-09-10T09:34:44.533 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hptiop.ko 2024-09-10T09:34:44.534 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hv_storvsc.ko 2024-09-10T09:34:44.534 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/initio.ko 2024-09-10T09:34:44.534 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ipr.ko 2024-09-10T09:34:44.534 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ips.ko 2024-09-10T09:34:44.534 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci 2024-09-10T09:34:44.534 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/isci.ko 2024-09-10T09:34:44.534 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-09-10T09:34:44.534 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_tcp.ko 2024-09-10T09:34:44.534 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc 2024-09-10T09:34:44.534 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/libfc.ko 2024-09-10T09:34:44.534 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi.ko 2024-09-10T09:34:44.534 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi_tcp.ko 2024-09-10T09:34:44.534 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas 2024-09-10T09:34:44.534 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/libsas.ko 2024-09-10T09:34:44.535 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc 2024-09-10T09:34:44.535 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/lpfc.ko 2024-09-10T09:34:44.535 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid 2024-09-10T09:34:44.535 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid.ko 2024-09-10T09:34:44.535 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-09-10T09:34:44.535 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-09-10T09:34:44.535 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-09-10T09:34:44.535 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas 2024-09-10T09:34:44.535 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-09-10T09:34:44.535 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas 2024-09-10T09:34:44.535 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/mvsas.ko 2024-09-10T09:34:44.535 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvumi.ko 2024-09-10T09:34:44.535 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001 2024-09-10T09:34:44.535 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-09-10T09:34:44.535 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pmcraid.ko 2024-09-10T09:34:44.536 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla1280.ko 2024-09-10T09:34:44.536 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx 2024-09-10T09:34:44.536 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-09-10T09:34:44.536 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-09-10T09:34:44.536 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx 2024-09-10T09:34:44.536 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-09-10T09:34:44.536 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/raid_class.ko 2024-09-10T09:34:44.536 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_debug.ko 2024-09-10T09:34:44.536 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_fc.ko 2024-09-10T09:34:44.536 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-09-10T09:34:44.536 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_sas.ko 2024-09-10T09:34:44.536 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_spi.ko 2024-09-10T09:34:44.536 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_srp.ko 2024-09-10T09:34:44.537 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ses.ko 2024-09-10T09:34:44.537 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/st.ko 2024-09-10T09:34:44.537 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/stex.ko 2024-09-10T09:34:44.537 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2 2024-09-10T09:34:44.537 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-09-10T09:34:44.537 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/virtio_scsi.ko 2024-09-10T09:34:44.537 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/vmw_pvscsi.ko 2024-09-10T09:34:44.537 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/wd719x.ko 2024-09-10T09:34:44.537 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb 2024-09-10T09:34:44.537 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ssb.ko 2024-09-10T09:34:44.537 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target 2024-09-10T09:34:44.537 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi 2024-09-10T09:34:44.537 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-09-10T09:34:44.537 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback 2024-09-10T09:34:44.538 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/tcm_loop.ko 2024-09-10T09:34:44.538 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_file.ko 2024-09-10T09:34:44.538 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_iblock.ko 2024-09-10T09:34:44.538 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_mod.ko 2024-09-10T09:34:44.538 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_pscsi.ko 2024-09-10T09:34:44.538 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_user.ko 2024-09-10T09:34:44.538 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc 2024-09-10T09:34:44.538 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-09-10T09:34:44.538 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal 2024-09-10T09:34:44.538 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel 2024-09-10T09:34:44.538 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal 2024-09-10T09:34:44.538 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-09-10T09:34:44.538 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-09-10T09:34:44.539 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-09-10T09:34:44.539 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-09-10T09:34:44.539 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-09-10T09:34:44.539 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-09-10T09:34:44.539 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-09-10T09:34:44.539 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-09-10T09:34:44.539 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-09-10T09:34:44.539 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-09-10T09:34:44.539 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-09-10T09:34:44.539 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-09-10T09:34:44.539 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-09-10T09:34:44.539 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-09-10T09:34:44.539 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-09-10T09:34:44.539 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-09-10T09:34:44.539 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-09-10T09:34:44.540 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-09-10T09:34:44.540 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty 2024-09-10T09:34:44.540 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless 2024-09-10T09:34:44.540 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-09-10T09:34:44.540 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_gsm.ko 2024-09-10T09:34:44.540 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_hdlc.ko 2024-09-10T09:34:44.540 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/nozomi.ko 2024-09-10T09:34:44.540 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial 2024-09-10T09:34:44.540 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250 2024-09-10T09:34:44.540 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-09-10T09:34:44.540 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/arc_uart.ko 2024-09-10T09:34:44.540 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm 2024-09-10T09:34:44.540 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/jsm.ko 2024-09-10T09:34:44.541 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs 2024-09-10T09:34:44.541 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core 2024-09-10T09:34:44.541 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ufshcd-core.ko 2024-09-10T09:34:44.541 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host 2024-09-10T09:34:44.541 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-09-10T09:34:44.541 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio 2024-09-10T09:34:44.541 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio.ko 2024-09-10T09:34:44.541 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_aec.ko 2024-09-10T09:34:44.541 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_cif.ko 2024-09-10T09:34:44.541 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pci_generic.ko 2024-09-10T09:34:44.541 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_sercos3.ko 2024-09-10T09:34:44.541 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb 2024-09-10T09:34:44.541 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class 2024-09-10T09:34:44.541 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-acm.ko 2024-09-10T09:34:44.542 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-wdm.ko 2024-09-10T09:34:44.542 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usblp.ko 2024-09-10T09:34:44.542 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usbtmc.ko 2024-09-10T09:34:44.542 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host 2024-09-10T09:34:44.542 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811-hcd.ko 2024-09-10T09:34:44.542 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image 2024-09-10T09:34:44.542 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/mdc800.ko 2024-09-10T09:34:44.542 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/microtek.ko 2024-09-10T09:34:44.542 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc 2024-09-10T09:34:44.542 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/adutux.ko 2024-09-10T09:34:44.542 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/appledisplay.ko 2024-09-10T09:34:44.542 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi26.ko 2024-09-10T09:34:44.542 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi62.ko 2024-09-10T09:34:44.542 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ezusb.ko 2024-09-10T09:34:44.543 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/idmouse.ko 2024-09-10T09:34:44.543 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/iowarrior.ko 2024-09-10T09:34:44.543 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/isight_firmware.ko 2024-09-10T09:34:44.543 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ldusb.ko 2024-09-10T09:34:44.543 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/legousbtower.ko 2024-09-10T09:34:44.543 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga 2024-09-10T09:34:44.543 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-09-10T09:34:44.543 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/trancevibrator.ko 2024-09-10T09:34:44.543 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usb3503.ko 2024-09-10T09:34:44.543 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usblcd.ko 2024-09-10T09:34:44.543 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbsevseg.ko 2024-09-10T09:34:44.543 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/uss720.ko 2024-09-10T09:34:44.543 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/yurex.ko 2024-09-10T09:34:44.543 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy 2024-09-10T09:34:44.543 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-generic.ko 2024-09-10T09:34:44.544 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial 2024-09-10T09:34:44.544 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/aircable.ko 2024-09-10T09:34:44.544 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ark3116.ko 2024-09-10T09:34:44.544 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/belkin_sa.ko 2024-09-10T09:34:44.544 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ch341.ko 2024-09-10T09:34:44.544 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cp210x.ko 2024-09-10T09:34:44.544 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cyberjack.ko 2024-09-10T09:34:44.544 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cypress_m8.ko 2024-09-10T09:34:44.544 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/digi_acceleport.ko 2024-09-10T09:34:44.544 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/empeg.ko 2024-09-10T09:34:44.544 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ftdi_sio.ko 2024-09-10T09:34:44.544 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/garmin_gps.ko 2024-09-10T09:34:44.544 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_edgeport.ko 2024-09-10T09:34:44.544 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_ti.ko 2024-09-10T09:34:44.545 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipaq.ko 2024-09-10T09:34:44.545 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipw.ko 2024-09-10T09:34:44.545 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ir-usb.ko 2024-09-10T09:34:44.545 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-09-10T09:34:44.545 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan.ko 2024-09-10T09:34:44.545 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan_pda.ko 2024-09-10T09:34:44.545 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kl5kusb105.ko 2024-09-10T09:34:44.545 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kobil_sct.ko 2024-09-10T09:34:44.545 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mct_u232.ko 2024-09-10T09:34:44.545 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7720.ko 2024-09-10T09:34:44.545 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7840.ko 2024-09-10T09:34:44.545 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/navman.ko 2024-09-10T09:34:44.545 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/omninet.ko 2024-09-10T09:34:44.545 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/opticon.ko 2024-09-10T09:34:44.546 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/option.ko 2024-09-10T09:34:44.546 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/oti6858.ko 2024-09-10T09:34:44.546 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/pl2303.ko 2024-09-10T09:34:44.546 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcaux.ko 2024-09-10T09:34:44.546 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcserial.ko 2024-09-10T09:34:44.546 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/quatech2.ko 2024-09-10T09:34:44.546 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/safe_serial.ko 2024-09-10T09:34:44.546 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/sierra.ko 2024-09-10T09:34:44.546 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/spcp8x5.ko 2024-09-10T09:34:44.546 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ssu100.ko 2024-09-10T09:34:44.546 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/symbolserial.ko 2024-09-10T09:34:44.546 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-09-10T09:34:44.546 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-09-10T09:34:44.546 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_debug.ko 2024-09-10T09:34:44.547 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_wwan.ko 2024-09-10T09:34:44.547 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/visor.ko 2024-09-10T09:34:44.547 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/whiteheat.ko 2024-09-10T09:34:44.547 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/xsens_mt.ko 2024-09-10T09:34:44.547 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage 2024-09-10T09:34:44.547 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/uas.ko 2024-09-10T09:34:44.547 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-alauda.ko 2024-09-10T09:34:44.547 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-cypress.ko 2024-09-10T09:34:44.547 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-datafab.ko 2024-09-10T09:34:44.547 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-09-10T09:34:44.547 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-freecom.ko 2024-09-10T09:34:44.547 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-isd200.ko 2024-09-10T09:34:44.547 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-09-10T09:34:44.547 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-karma.ko 2024-09-10T09:34:44.547 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-onetouch.ko 2024-09-10T09:34:44.548 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-realtek.ko 2024-09-10T09:34:44.548 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr09.ko 2024-09-10T09:34:44.548 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr55.ko 2024-09-10T09:34:44.548 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-usbat.ko 2024-09-10T09:34:44.548 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/usb-storage.ko 2024-09-10T09:34:44.548 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip 2024-09-10T09:34:44.548 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-core.ko 2024-09-10T09:34:44.548 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-host.ko 2024-09-10T09:34:44.548 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-09-10T09:34:44.548 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio 2024-09-10T09:34:44.548 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci 2024-09-10T09:34:44.548 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-09-10T09:34:44.548 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci.ko 2024-09-10T09:34:44.548 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio.ko 2024-09-10T09:34:44.549 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-09-10T09:34:44.549 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost 2024-09-10T09:34:44.549 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost.ko 2024-09-10T09:34:44.549 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_iotlb.ko 2024-09-10T09:34:44.549 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_net.ko 2024-09-10T09:34:44.549 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_scsi.ko 2024-09-10T09:34:44.549 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video 2024-09-10T09:34:44.549 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight 2024-09-10T09:34:44.549 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/apple_bl.ko 2024-09-10T09:34:44.549 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lcd.ko 2024-09-10T09:34:44.549 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/platform_lcd.ko 2024-09-10T09:34:44.549 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio 2024-09-10T09:34:44.549 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio.ko 2024-09-10T09:34:44.550 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_balloon.ko 2024-09-10T09:34:44.550 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_mmio.ko 2024-09-10T09:34:44.550 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci.ko 2024-09-10T09:34:44.550 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-09-10T09:34:44.550 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-09-10T09:34:44.550 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_ring.ko 2024-09-10T09:34:44.550 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1 2024-09-10T09:34:44.550 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters 2024-09-10T09:34:44.550 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2482.ko 2024-09-10T09:34:44.550 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2490.ko 2024-09-10T09:34:44.550 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves 2024-09-10T09:34:44.550 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-09-10T09:34:44.550 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-09-10T09:34:44.550 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-09-10T09:34:44.551 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-09-10T09:34:44.551 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-09-10T09:34:44.551 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-09-10T09:34:44.551 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-09-10T09:34:44.551 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-09-10T09:34:44.551 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-09-10T09:34:44.551 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_smem.ko 2024-09-10T09:34:44.551 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_therm.ko 2024-09-10T09:34:44.551 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/wire.ko 2024-09-10T09:34:44.551 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog 2024-09-10T09:34:44.551 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim1535_wdt.ko 2024-09-10T09:34:44.551 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim7101_wdt.ko 2024-09-10T09:34:44.551 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/f71808e_wdt.ko 2024-09-10T09:34:44.551 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/hpwdt.ko 2024-09-10T09:34:44.552 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/i6300esb.ko 2024-09-10T09:34:44.552 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-09-10T09:34:44.552 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_wdt.ko 2024-09-10T09:34:44.552 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ib700wdt.ko 2024-09-10T09:34:44.552 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ibmasr.ko 2024-09-10T09:34:44.552 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-09-10T09:34:44.552 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it8712f_wdt.ko 2024-09-10T09:34:44.552 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it87_wdt.ko 2024-09-10T09:34:44.552 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/machzwd.ko 2024-09-10T09:34:44.552 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/nv_tco.ko 2024-09-10T09:34:44.552 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_pci.ko 2024-09-10T09:34:44.552 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_usb.ko 2024-09-10T09:34:44.552 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-09-10T09:34:44.552 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sch311x_wdt.ko 2024-09-10T09:34:44.553 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/softdog.ko 2024-09-10T09:34:44.553 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sp5100_tco.ko 2024-09-10T09:34:44.553 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/via_wdt.ko 2024-09-10T09:34:44.553 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-09-10T09:34:44.553 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83877f_wdt.ko 2024-09-10T09:34:44.553 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83977f_wdt.ko 2024-09-10T09:34:44.553 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wdt_pci.ko 2024-09-10T09:34:44.553 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/xen_wdt.ko 2024-09-10T09:34:44.553 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen 2024-09-10T09:34:44.553 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-acpi-processor.ko 2024-09-10T09:34:44.553 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-evtchn.ko 2024-09-10T09:34:44.553 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntalloc.ko 2024-09-10T09:34:44.554 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntdev.ko 2024-09-10T09:34:44.554 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback 2024-09-10T09:34:44.554 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-09-10T09:34:44.554 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-privcmd.ko 2024-09-10T09:34:44.554 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs 2024-09-10T09:34:44.554 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/xenfs.ko 2024-09-10T09:34:44.554 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs 2024-09-10T09:34:44.554 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p 2024-09-10T09:34:44.554 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/9p.ko 2024-09-10T09:34:44.554 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs 2024-09-10T09:34:44.554 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/affs.ko 2024-09-10T09:34:44.554 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs 2024-09-10T09:34:44.554 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/befs.ko 2024-09-10T09:34:44.554 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/binfmt_misc.ko 2024-09-10T09:34:44.555 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs 2024-09-10T09:34:44.555 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/btrfs.ko 2024-09-10T09:34:44.555 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles 2024-09-10T09:34:44.555 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/cachefiles.ko 2024-09-10T09:34:44.555 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph 2024-09-10T09:34:44.555 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ceph.ko 2024-09-10T09:34:44.555 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda 2024-09-10T09:34:44.555 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/coda.ko 2024-09-10T09:34:44.555 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs 2024-09-10T09:34:44.555 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/cramfs.ko 2024-09-10T09:34:44.555 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm 2024-09-10T09:34:44.556 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/dlm.ko 2024-09-10T09:34:44.556 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ecryptfs 2024-09-10T09:34:44.556 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ecryptfs/ecryptfs.ko 2024-09-10T09:34:44.556 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs 2024-09-10T09:34:44.556 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/f2fs.ko 2024-09-10T09:34:44.556 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat 2024-09-10T09:34:44.556 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/fat.ko 2024-09-10T09:34:44.556 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/msdos.ko 2024-09-10T09:34:44.556 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/vfat.ko 2024-09-10T09:34:44.556 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse 2024-09-10T09:34:44.556 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/cuse.ko 2024-09-10T09:34:44.556 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/fuse.ko 2024-09-10T09:34:44.556 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2 2024-09-10T09:34:44.556 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/gfs2.ko 2024-09-10T09:34:44.557 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs 2024-09-10T09:34:44.557 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/hfs.ko 2024-09-10T09:34:44.557 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus 2024-09-10T09:34:44.557 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/hfsplus.ko 2024-09-10T09:34:44.557 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs 2024-09-10T09:34:44.557 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/isofs.ko 2024-09-10T09:34:44.557 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs 2024-09-10T09:34:44.557 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/jfs.ko 2024-09-10T09:34:44.557 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd 2024-09-10T09:34:44.557 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/lockd.ko 2024-09-10T09:34:44.557 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix 2024-09-10T09:34:44.557 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/minix.ko 2024-09-10T09:34:44.557 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs 2024-09-10T09:34:44.557 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/netfs.ko 2024-09-10T09:34:44.558 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs 2024-09-10T09:34:44.558 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout 2024-09-10T09:34:44.558 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-09-10T09:34:44.558 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout 2024-09-10T09:34:44.558 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-09-10T09:34:44.558 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout 2024-09-10T09:34:44.558 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-09-10T09:34:44.558 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfs.ko 2024-09-10T09:34:44.558 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv3.ko 2024-09-10T09:34:44.558 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv4.ko 2024-09-10T09:34:44.558 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common 2024-09-10T09:34:44.558 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/grace.ko 2024-09-10T09:34:44.558 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/nfs_acl.ko 2024-09-10T09:34:44.559 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd 2024-09-10T09:34:44.559 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/nfsd.ko 2024-09-10T09:34:44.559 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2 2024-09-10T09:34:44.559 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/nilfs2.ko 2024-09-10T09:34:44.559 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls 2024-09-10T09:34:44.559 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-celtic.ko 2024-09-10T09:34:44.559 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-centeuro.ko 2024-09-10T09:34:44.559 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-croatian.ko 2024-09-10T09:34:44.559 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-cyrillic.ko 2024-09-10T09:34:44.559 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-gaelic.ko 2024-09-10T09:34:44.559 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-greek.ko 2024-09-10T09:34:44.559 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-iceland.ko 2024-09-10T09:34:44.559 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-inuit.ko 2024-09-10T09:34:44.559 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-roman.ko 2024-09-10T09:34:44.560 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-romanian.ko 2024-09-10T09:34:44.560 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-turkish.ko 2024-09-10T09:34:44.560 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1250.ko 2024-09-10T09:34:44.560 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1251.ko 2024-09-10T09:34:44.560 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1255.ko 2024-09-10T09:34:44.560 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp737.ko 2024-09-10T09:34:44.560 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp775.ko 2024-09-10T09:34:44.560 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp850.ko 2024-09-10T09:34:44.560 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp852.ko 2024-09-10T09:34:44.560 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp855.ko 2024-09-10T09:34:44.560 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp857.ko 2024-09-10T09:34:44.561 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp860.ko 2024-09-10T09:34:44.561 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp861.ko 2024-09-10T09:34:44.561 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp862.ko 2024-09-10T09:34:44.561 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp863.ko 2024-09-10T09:34:44.561 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp864.ko 2024-09-10T09:34:44.561 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp865.ko 2024-09-10T09:34:44.561 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp866.ko 2024-09-10T09:34:44.561 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp869.ko 2024-09-10T09:34:44.561 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp874.ko 2024-09-10T09:34:44.561 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp932.ko 2024-09-10T09:34:44.561 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp936.ko 2024-09-10T09:34:44.561 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp949.ko 2024-09-10T09:34:44.561 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp950.ko 2024-09-10T09:34:44.562 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_euc-jp.ko 2024-09-10T09:34:44.562 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-1.ko 2024-09-10T09:34:44.562 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-13.ko 2024-09-10T09:34:44.562 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-14.ko 2024-09-10T09:34:44.562 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-15.ko 2024-09-10T09:34:44.562 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-2.ko 2024-09-10T09:34:44.562 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-3.ko 2024-09-10T09:34:44.562 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-4.ko 2024-09-10T09:34:44.562 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-5.ko 2024-09-10T09:34:44.562 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-6.ko 2024-09-10T09:34:44.562 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-7.ko 2024-09-10T09:34:44.562 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-9.ko 2024-09-10T09:34:44.562 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-r.ko 2024-09-10T09:34:44.562 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-ru.ko 2024-09-10T09:34:44.562 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-u.ko 2024-09-10T09:34:44.563 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ucs2_utils.ko 2024-09-10T09:34:44.563 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_utf8.ko 2024-09-10T09:34:44.563 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2 2024-09-10T09:34:44.563 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster 2024-09-10T09:34:44.563 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-09-10T09:34:44.563 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm 2024-09-10T09:34:44.563 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-09-10T09:34:44.563 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs 2024-09-10T09:34:44.563 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-09-10T09:34:44.563 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2.ko 2024-09-10T09:34:44.563 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-09-10T09:34:44.563 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-09-10T09:34:44.563 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-09-10T09:34:44.563 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs 2024-09-10T09:34:44.564 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/overlay.ko 2024-09-10T09:34:44.564 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore 2024-09-10T09:34:44.564 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ramoops.ko 2024-09-10T09:34:44.564 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs 2024-09-10T09:34:44.564 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/reiserfs.ko 2024-09-10T09:34:44.564 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs 2024-09-10T09:34:44.564 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/romfs.ko 2024-09-10T09:34:44.564 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb 2024-09-10T09:34:44.564 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client 2024-09-10T09:34:44.564 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/cifs.ko 2024-09-10T09:34:44.564 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common 2024-09-10T09:34:44.564 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_arc4.ko 2024-09-10T09:34:44.564 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_md4.ko 2024-09-10T09:34:44.564 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs 2024-09-10T09:34:44.565 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/squashfs.ko 2024-09-10T09:34:44.565 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv 2024-09-10T09:34:44.565 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/sysv.ko 2024-09-10T09:34:44.565 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf 2024-09-10T09:34:44.565 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/udf.ko 2024-09-10T09:34:44.565 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs 2024-09-10T09:34:44.565 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ufs.ko 2024-09-10T09:34:44.565 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs 2024-09-10T09:34:44.565 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/xfs.ko 2024-09-10T09:34:44.565 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel 2024-09-10T09:34:44.565 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/rcu 2024-09-10T09:34:44.565 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/rcu/rcutorture.ko 2024-09-10T09:34:44.565 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/torture.ko 2024-09-10T09:34:44.565 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/trace 2024-09-10T09:34:44.566 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-09-10T09:34:44.566 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib 2024-09-10T09:34:44.566 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/asn1_encoder.ko 2024-09-10T09:34:44.566 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc-itu-t.ko 2024-09-10T09:34:44.566 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc8.ko 2024-09-10T09:34:44.566 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto 2024-09-10T09:34:44.566 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libaescfb.ko 2024-09-10T09:34:44.566 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libarc4.ko 2024-09-10T09:34:44.566 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libdes.ko 2024-09-10T09:34:44.566 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/libcrc32c.ko 2024-09-10T09:34:44.566 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lru_cache.ko 2024-09-10T09:34:44.566 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4 2024-09-10T09:34:44.566 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4_compress.ko 2024-09-10T09:34:44.566 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4hc_compress.ko 2024-09-10T09:34:44.567 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math 2024-09-10T09:34:44.567 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/cordic.ko 2024-09-10T09:34:44.567 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6 2024-09-10T09:34:44.567 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/raid6_pq.ko 2024-09-10T09:34:44.567 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon 2024-09-10T09:34:44.567 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/reed_solomon.ko 2024-09-10T09:34:44.567 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_bm.ko 2024-09-10T09:34:44.567 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_fsm.ko 2024-09-10T09:34:44.567 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_kmp.ko 2024-09-10T09:34:44.567 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/zstd 2024-09-10T09:34:44.567 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/zstd/zstd_compress.ko 2024-09-10T09:34:44.567 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm 2024-09-10T09:34:44.567 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/hwpoison-inject.ko 2024-09-10T09:34:44.567 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net 2024-09-10T09:34:44.567 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan 2024-09-10T09:34:44.568 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/6lowpan.ko 2024-09-10T09:34:44.568 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_dest.ko 2024-09-10T09:34:44.568 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_fragment.ko 2024-09-10T09:34:44.568 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_hop.ko 2024-09-10T09:34:44.568 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_ipv6.ko 2024-09-10T09:34:44.568 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_mobility.ko 2024-09-10T09:34:44.568 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_routing.ko 2024-09-10T09:34:44.568 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_udp.ko 2024-09-10T09:34:44.568 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802 2024-09-10T09:34:44.568 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/garp.ko 2024-09-10T09:34:44.568 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/mrp.ko 2024-09-10T09:34:44.568 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/p8022.ko 2024-09-10T09:34:44.568 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/psnap.ko 2024-09-10T09:34:44.569 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/stp.ko 2024-09-10T09:34:44.569 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q 2024-09-10T09:34:44.569 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/8021q.ko 2024-09-10T09:34:44.569 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p 2024-09-10T09:34:44.569 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet.ko 2024-09-10T09:34:44.569 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_fd.ko 2024-09-10T09:34:44.569 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_rdma.ko 2024-09-10T09:34:44.569 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_virtio.ko 2024-09-10T09:34:44.569 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv 2024-09-10T09:34:44.569 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/batman-adv.ko 2024-09-10T09:34:44.569 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge 2024-09-10T09:34:44.569 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/br_netfilter.ko 2024-09-10T09:34:44.569 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/bridge.ko 2024-09-10T09:34:44.569 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter 2024-09-10T09:34:44.570 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-09-10T09:34:44.570 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_among.ko 2024-09-10T09:34:44.570 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arp.ko 2024-09-10T09:34:44.570 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-09-10T09:34:44.570 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-09-10T09:34:44.570 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip.ko 2024-09-10T09:34:44.570 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-09-10T09:34:44.570 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_limit.ko 2024-09-10T09:34:44.570 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_log.ko 2024-09-10T09:34:44.570 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark.ko 2024-09-10T09:34:44.570 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-09-10T09:34:44.570 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-09-10T09:34:44.570 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-09-10T09:34:44.570 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-09-10T09:34:44.571 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_snat.ko 2024-09-10T09:34:44.571 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_stp.ko 2024-09-10T09:34:44.571 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-09-10T09:34:44.571 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-09-10T09:34:44.571 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-09-10T09:34:44.571 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-09-10T09:34:44.571 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtables.ko 2024-09-10T09:34:44.571 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-09-10T09:34:44.571 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-09-10T09:34:44.571 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-09-10T09:34:44.571 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph 2024-09-10T09:34:44.571 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/libceph.ko 2024-09-10T09:34:44.571 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core 2024-09-10T09:34:44.571 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/failover.ko 2024-09-10T09:34:44.572 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/pktgen.ko 2024-09-10T09:34:44.572 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp 2024-09-10T09:34:44.572 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp.ko 2024-09-10T09:34:44.572 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_diag.ko 2024-09-10T09:34:44.572 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv4.ko 2024-09-10T09:34:44.572 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv6.ko 2024-09-10T09:34:44.572 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dns_resolver 2024-09-10T09:34:44.572 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dns_resolver/dns_resolver.ko 2024-09-10T09:34:44.572 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa 2024-09-10T09:34:44.572 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/dsa_core.ko 2024-09-10T09:34:44.572 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_brcm.ko 2024-09-10T09:34:44.572 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_dsa.ko 2024-09-10T09:34:44.572 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_none.ko 2024-09-10T09:34:44.573 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_trailer.ko 2024-09-10T09:34:44.573 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154 2024-09-10T09:34:44.573 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan 2024-09-10T09:34:44.573 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-09-10T09:34:44.573 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154.ko 2024-09-10T09:34:44.573 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154_socket.ko 2024-09-10T09:34:44.573 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4 2024-09-10T09:34:44.573 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ah4.ko 2024-09-10T09:34:44.573 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/esp4.ko 2024-09-10T09:34:44.573 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/fou.ko 2024-09-10T09:34:44.573 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/gre.ko 2024-09-10T09:34:44.573 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/inet_diag.ko 2024-09-10T09:34:44.573 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_gre.ko 2024-09-10T09:34:44.573 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_tunnel.ko 2024-09-10T09:34:44.574 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_vti.ko 2024-09-10T09:34:44.574 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipcomp.ko 2024-09-10T09:34:44.574 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipip.ko 2024-09-10T09:34:44.574 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter 2024-09-10T09:34:44.574 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arp_tables.ko 2024-09-10T09:34:44.574 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-09-10T09:34:44.574 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-09-10T09:34:44.574 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-09-10T09:34:44.574 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-09-10T09:34:44.574 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-09-10T09:34:44.574 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-09-10T09:34:44.574 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-09-10T09:34:44.574 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-09-10T09:34:44.574 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-09-10T09:34:44.575 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_security.ko 2024-09-10T09:34:44.575 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-09-10T09:34:44.575 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-09-10T09:34:44.575 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-09-10T09:34:44.575 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-09-10T09:34:44.575 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-09-10T09:34:44.575 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-09-10T09:34:44.575 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-09-10T09:34:44.575 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-09-10T09:34:44.575 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-09-10T09:34:44.575 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-09-10T09:34:44.575 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_bic.ko 2024-09-10T09:34:44.575 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_dctcp.ko 2024-09-10T09:34:44.575 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_diag.ko 2024-09-10T09:34:44.575 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_highspeed.ko 2024-09-10T09:34:44.576 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_htcp.ko 2024-09-10T09:34:44.576 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_hybla.ko 2024-09-10T09:34:44.576 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_illinois.ko 2024-09-10T09:34:44.576 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_lp.ko 2024-09-10T09:34:44.576 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_scalable.ko 2024-09-10T09:34:44.576 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_vegas.ko 2024-09-10T09:34:44.576 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_veno.ko 2024-09-10T09:34:44.576 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_westwood.ko 2024-09-10T09:34:44.576 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_yeah.ko 2024-09-10T09:34:44.576 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tunnel4.ko 2024-09-10T09:34:44.576 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_diag.ko 2024-09-10T09:34:44.576 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_tunnel.ko 2024-09-10T09:34:44.576 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/xfrm4_tunnel.ko 2024-09-10T09:34:44.576 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6 2024-09-10T09:34:44.577 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ah6.ko 2024-09-10T09:34:44.577 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/esp6.ko 2024-09-10T09:34:44.577 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/fou6.ko 2024-09-10T09:34:44.577 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_tunnel.ko 2024-09-10T09:34:44.577 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-09-10T09:34:44.577 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_vti.ko 2024-09-10T09:34:44.577 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ipcomp6.ko 2024-09-10T09:34:44.577 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter 2024-09-10T09:34:44.577 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-09-10T09:34:44.577 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-09-10T09:34:44.577 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-09-10T09:34:44.577 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-09-10T09:34:44.577 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-09-10T09:34:44.577 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-09-10T09:34:44.578 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-09-10T09:34:44.578 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-09-10T09:34:44.578 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-09-10T09:34:44.578 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-09-10T09:34:44.578 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-09-10T09:34:44.578 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-09-10T09:34:44.578 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-09-10T09:34:44.578 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-09-10T09:34:44.578 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-09-10T09:34:44.578 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-09-10T09:34:44.578 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-09-10T09:34:44.578 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-09-10T09:34:44.578 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-09-10T09:34:44.578 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-09-10T09:34:44.579 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-09-10T09:34:44.579 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-09-10T09:34:44.579 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-09-10T09:34:44.579 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-09-10T09:34:44.579 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/sit.ko 2024-09-10T09:34:44.579 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/tunnel6.ko 2024-09-10T09:34:44.579 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/xfrm6_tunnel.ko 2024-09-10T09:34:44.579 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key 2024-09-10T09:34:44.579 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/af_key.ko 2024-09-10T09:34:44.579 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp 2024-09-10T09:34:44.579 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_core.ko 2024-09-10T09:34:44.579 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_debugfs.ko 2024-09-10T09:34:44.579 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_eth.ko 2024-09-10T09:34:44.579 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip.ko 2024-09-10T09:34:44.580 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip6.ko 2024-09-10T09:34:44.580 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_netlink.ko 2024-09-10T09:34:44.580 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ppp.ko 2024-09-10T09:34:44.580 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc 2024-09-10T09:34:44.580 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc.ko 2024-09-10T09:34:44.580 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154 2024-09-10T09:34:44.580 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/mac802154.ko 2024-09-10T09:34:44.580 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls 2024-09-10T09:34:44.580 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/mpls_gso.ko 2024-09-10T09:34:44.580 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter 2024-09-10T09:34:44.580 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset 2024-09-10T09:34:44.580 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set.ko 2024-09-10T09:34:44.580 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-09-10T09:34:44.580 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-09-10T09:34:44.581 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-09-10T09:34:44.581 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-09-10T09:34:44.581 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-09-10T09:34:44.581 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-09-10T09:34:44.581 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-09-10T09:34:44.581 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-09-10T09:34:44.581 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-09-10T09:34:44.581 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-09-10T09:34:44.581 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-09-10T09:34:44.581 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-09-10T09:34:44.581 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-09-10T09:34:44.581 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-09-10T09:34:44.581 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-09-10T09:34:44.582 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs 2024-09-10T09:34:44.582 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs.ko 2024-09-10T09:34:44.582 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-09-10T09:34:44.582 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-09-10T09:34:44.582 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-09-10T09:34:44.582 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-09-10T09:34:44.582 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-09-10T09:34:44.582 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-09-10T09:34:44.582 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-09-10T09:34:44.582 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-09-10T09:34:44.582 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-09-10T09:34:44.582 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-09-10T09:34:44.582 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-09-10T09:34:44.582 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-09-10T09:34:44.582 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-09-10T09:34:44.583 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conncount.ko 2024-09-10T09:34:44.583 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack.ko 2024-09-10T09:34:44.583 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-09-10T09:34:44.583 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-09-10T09:34:44.583 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-09-10T09:34:44.583 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_h323.ko 2024-09-10T09:34:44.583 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_irc.ko 2024-09-10T09:34:44.583 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-09-10T09:34:44.583 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-09-10T09:34:44.583 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-09-10T09:34:44.583 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sane.ko 2024-09-10T09:34:44.583 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sip.ko 2024-09-10T09:34:44.583 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-09-10T09:34:44.583 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-09-10T09:34:44.584 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_dup_netdev.ko 2024-09-10T09:34:44.584 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table.ko 2024-09-10T09:34:44.584 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table_inet.ko 2024-09-10T09:34:44.584 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_log_syslog.ko 2024-09-10T09:34:44.584 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat.ko 2024-09-10T09:34:44.584 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_amanda.ko 2024-09-10T09:34:44.584 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_ftp.ko 2024-09-10T09:34:44.584 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_irc.ko 2024-09-10T09:34:44.584 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_sip.ko 2024-09-10T09:34:44.584 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_tftp.ko 2024-09-10T09:34:44.584 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_synproxy_core.ko 2024-09-10T09:34:44.584 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_tables.ko 2024-09-10T09:34:44.584 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink.ko 2024-09-10T09:34:44.584 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_acct.ko 2024-09-10T09:34:44.585 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_log.ko 2024-09-10T09:34:44.585 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_osf.ko 2024-09-10T09:34:44.585 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_queue.ko 2024-09-10T09:34:44.585 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_chain_nat.ko 2024-09-10T09:34:44.585 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_compat.ko 2024-09-10T09:34:44.585 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_ct.ko 2024-09-10T09:34:44.585 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_dup_netdev.ko 2024-09-10T09:34:44.585 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fib.ko 2024-09-10T09:34:44.585 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fwd_netdev.ko 2024-09-10T09:34:44.585 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_hash.ko 2024-09-10T09:34:44.585 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_limit.ko 2024-09-10T09:34:44.585 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_log.ko 2024-09-10T09:34:44.585 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_masq.ko 2024-09-10T09:34:44.585 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_nat.ko 2024-09-10T09:34:44.586 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_numgen.ko 2024-09-10T09:34:44.586 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_queue.ko 2024-09-10T09:34:44.586 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_quota.ko 2024-09-10T09:34:44.586 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_redir.ko 2024-09-10T09:34:44.586 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject.ko 2024-09-10T09:34:44.586 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject_inet.ko 2024-09-10T09:34:44.586 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_socket.ko 2024-09-10T09:34:44.586 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_tunnel.ko 2024-09-10T09:34:44.586 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_AUDIT.ko 2024-09-10T09:34:44.586 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CHECKSUM.ko 2024-09-10T09:34:44.586 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CLASSIFY.ko 2024-09-10T09:34:44.586 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-09-10T09:34:44.586 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CT.ko 2024-09-10T09:34:44.586 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_DSCP.ko 2024-09-10T09:34:44.587 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HL.ko 2024-09-10T09:34:44.587 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HMARK.ko 2024-09-10T09:34:44.587 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_IDLETIMER.ko 2024-09-10T09:34:44.587 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LED.ko 2024-09-10T09:34:44.587 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LOG.ko 2024-09-10T09:34:44.587 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_MASQUERADE.ko 2024-09-10T09:34:44.587 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NETMAP.ko 2024-09-10T09:34:44.587 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFLOG.ko 2024-09-10T09:34:44.587 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFQUEUE.ko 2024-09-10T09:34:44.587 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_RATEEST.ko 2024-09-10T09:34:44.587 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_REDIRECT.ko 2024-09-10T09:34:44.587 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_SECMARK.ko 2024-09-10T09:34:44.587 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPMSS.ko 2024-09-10T09:34:44.587 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-09-10T09:34:44.587 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TEE.ko 2024-09-10T09:34:44.587 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TPROXY.ko 2024-09-10T09:34:44.588 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TRACE.ko 2024-09-10T09:34:44.588 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_addrtype.ko 2024-09-10T09:34:44.588 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_bpf.ko 2024-09-10T09:34:44.588 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cgroup.ko 2024-09-10T09:34:44.588 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cluster.ko 2024-09-10T09:34:44.588 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_comment.ko 2024-09-10T09:34:44.588 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connbytes.ko 2024-09-10T09:34:44.588 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlabel.ko 2024-09-10T09:34:44.588 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlimit.ko 2024-09-10T09:34:44.588 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connmark.ko 2024-09-10T09:34:44.588 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_conntrack.ko 2024-09-10T09:34:44.588 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cpu.ko 2024-09-10T09:34:44.588 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dccp.ko 2024-09-10T09:34:44.588 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_devgroup.ko 2024-09-10T09:34:44.589 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dscp.ko 2024-09-10T09:34:44.589 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ecn.ko 2024-09-10T09:34:44.589 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_esp.ko 2024-09-10T09:34:44.589 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hashlimit.ko 2024-09-10T09:34:44.589 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_helper.ko 2024-09-10T09:34:44.589 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hl.ko 2024-09-10T09:34:44.589 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipcomp.ko 2024-09-10T09:34:44.589 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_iprange.ko 2024-09-10T09:34:44.589 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipvs.ko 2024-09-10T09:34:44.589 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_l2tp.ko 2024-09-10T09:34:44.589 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_length.ko 2024-09-10T09:34:44.589 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_limit.ko 2024-09-10T09:34:44.589 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mac.ko 2024-09-10T09:34:44.589 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mark.ko 2024-09-10T09:34:44.590 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_multiport.ko 2024-09-10T09:34:44.590 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nat.ko 2024-09-10T09:34:44.590 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nfacct.ko 2024-09-10T09:34:44.590 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_osf.ko 2024-09-10T09:34:44.590 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_owner.ko 2024-09-10T09:34:44.590 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_physdev.ko 2024-09-10T09:34:44.590 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_pkttype.ko 2024-09-10T09:34:44.590 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_policy.ko 2024-09-10T09:34:44.590 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_quota.ko 2024-09-10T09:34:44.590 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_rateest.ko 2024-09-10T09:34:44.590 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_realm.ko 2024-09-10T09:34:44.590 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_recent.ko 2024-09-10T09:34:44.590 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_sctp.ko 2024-09-10T09:34:44.590 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_set.ko 2024-09-10T09:34:44.590 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_socket.ko 2024-09-10T09:34:44.590 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_state.ko 2024-09-10T09:34:44.591 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_statistic.ko 2024-09-10T09:34:44.591 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_string.ko 2024-09-10T09:34:44.591 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpmss.ko 2024-09-10T09:34:44.591 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_time.ko 2024-09-10T09:34:44.591 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_u32.ko 2024-09-10T09:34:44.591 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink 2024-09-10T09:34:44.591 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/netlink_diag.ko 2024-09-10T09:34:44.591 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh 2024-09-10T09:34:44.591 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/nsh.ko 2024-09-10T09:34:44.591 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch 2024-09-10T09:34:44.591 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/openvswitch.ko 2024-09-10T09:34:44.591 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-geneve.ko 2024-09-10T09:34:44.591 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-gre.ko 2024-09-10T09:34:44.591 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-vxlan.ko 2024-09-10T09:34:44.591 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet 2024-09-10T09:34:44.591 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/af_packet_diag.ko 2024-09-10T09:34:44.592 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds 2024-09-10T09:34:44.592 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds.ko 2024-09-10T09:34:44.592 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_rdma.ko 2024-09-10T09:34:44.592 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_tcp.ko 2024-09-10T09:34:44.592 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched 2024-09-10T09:34:44.592 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_csum.ko 2024-09-10T09:34:44.592 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_gact.ko 2024-09-10T09:34:44.592 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_mirred.ko 2024-09-10T09:34:44.592 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_nat.ko 2024-09-10T09:34:44.592 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_pedit.ko 2024-09-10T09:34:44.592 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_police.ko 2024-09-10T09:34:44.592 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_simple.ko 2024-09-10T09:34:44.592 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_skbedit.ko 2024-09-10T09:34:44.592 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_vlan.ko 2024-09-10T09:34:44.592 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_basic.ko 2024-09-10T09:34:44.592 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_bpf.ko 2024-09-10T09:34:44.593 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_flow.ko 2024-09-10T09:34:44.593 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_fw.ko 2024-09-10T09:34:44.593 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_route.ko 2024-09-10T09:34:44.593 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_u32.ko 2024-09-10T09:34:44.593 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_cmp.ko 2024-09-10T09:34:44.593 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_ipset.ko 2024-09-10T09:34:44.593 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_meta.ko 2024-09-10T09:34:44.593 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_nbyte.ko 2024-09-10T09:34:44.593 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_text.ko 2024-09-10T09:34:44.593 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_u32.ko 2024-09-10T09:34:44.593 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_choke.ko 2024-09-10T09:34:44.593 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_codel.ko 2024-09-10T09:34:44.593 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_drr.ko 2024-09-10T09:34:44.593 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq.ko 2024-09-10T09:34:44.594 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_gred.ko 2024-09-10T09:34:44.594 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hfsc.ko 2024-09-10T09:34:44.594 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hhf.ko 2024-09-10T09:34:44.594 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_htb.ko 2024-09-10T09:34:44.594 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_ingress.ko 2024-09-10T09:34:44.594 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio.ko 2024-09-10T09:34:44.594 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio_lib.ko 2024-09-10T09:34:44.594 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_multiq.ko 2024-09-10T09:34:44.594 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_netem.ko 2024-09-10T09:34:44.594 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_pie.ko 2024-09-10T09:34:44.594 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_plug.ko 2024-09-10T09:34:44.594 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_prio.ko 2024-09-10T09:34:44.594 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_qfq.ko 2024-09-10T09:34:44.594 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_red.ko 2024-09-10T09:34:44.594 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfb.ko 2024-09-10T09:34:44.595 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfq.ko 2024-09-10T09:34:44.595 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_tbf.ko 2024-09-10T09:34:44.595 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_teql.ko 2024-09-10T09:34:44.595 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp 2024-09-10T09:34:44.595 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp.ko 2024-09-10T09:34:44.595 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp_diag.ko 2024-09-10T09:34:44.595 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc 2024-09-10T09:34:44.595 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss 2024-09-10T09:34:44.595 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-09-10T09:34:44.595 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-09-10T09:34:44.595 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/sunrpc.ko 2024-09-10T09:34:44.595 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma 2024-09-10T09:34:44.595 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-09-10T09:34:44.595 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc 2024-09-10T09:34:44.595 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc.ko 2024-09-10T09:34:44.596 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc_diag.ko 2024-09-10T09:34:44.596 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix 2024-09-10T09:34:44.596 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/unix_diag.ko 2024-09-10T09:34:44.596 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock 2024-09-10T09:34:44.596 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-09-10T09:34:44.596 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-09-10T09:34:44.596 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock.ko 2024-09-10T09:34:44.596 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_diag.ko 2024-09-10T09:34:44.596 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_loopback.ko 2024-09-10T09:34:44.596 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm 2024-09-10T09:34:44.596 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_ipcomp.ko 2024-09-10T09:34:44.596 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security 2024-09-10T09:34:44.596 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys 2024-09-10T09:34:44.597 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/encrypted-keys 2024-09-10T09:34:44.597 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-09-10T09:34:44.597 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/trusted-keys 2024-09-10T09:34:44.597 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/trusted-keys/trusted.ko 2024-09-10T09:34:44.597 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias 2024-09-10T09:34:44.597 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias.bin 2024-09-10T09:34:44.597 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin 2024-09-10T09:34:44.597 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.alias.bin 2024-09-10T09:34:44.597 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.bin 2024-09-10T09:34:44.597 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.modinfo 2024-09-10T09:34:44.597 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep 2024-09-10T09:34:44.597 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep.bin 2024-09-10T09:34:44.597 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.devname 2024-09-10T09:34:44.597 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.order 2024-09-10T09:34:44.598 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.softdep 2024-09-10T09:34:44.598 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols 2024-09-10T09:34:44.598 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols.bin 2024-09-10T09:34:44.598 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.weakdep 2024-09-10T09:34:44.598 INFO:teuthology.orchestra.run.smithi077.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/vmlinuz 2024-09-10T09:34:44.602 DEBUG:teuthology.task.kernel:get_image_version: 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T09:34:44.602 DEBUG:teuthology.orchestra.run.smithi077:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-09-10T09:34:44.626 INFO:teuthology.orchestra.run.smithi077.stdout:postinstall scriptlet (using /bin/sh): 2024-09-10T09:34:44.626 INFO:teuthology.orchestra.run.smithi077.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-10T09:34:44.626 INFO:teuthology.orchestra.run.smithi077.stdout: /usr/bin/kernel-install add 6.11.0-rc5-g2ca44a2f35ca /lib/modules/6.11.0-rc5-g2ca44a2f35ca/vmlinuz 2024-09-10T09:34:44.626 INFO:teuthology.orchestra.run.smithi077.stdout:fi 2024-09-10T09:34:44.626 INFO:teuthology.orchestra.run.smithi077.stdout:for file in vmlinuz System.map config; do 2024-09-10T09:34:44.626 INFO:teuthology.orchestra.run.smithi077.stdout: if ! cmp --silent "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/${file}" "/boot/${file}-6.11.0-rc5-g2ca44a2f35ca"; then 2024-09-10T09:34:44.626 INFO:teuthology.orchestra.run.smithi077.stdout: cp "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/${file}" "/boot/${file}-6.11.0-rc5-g2ca44a2f35ca" 2024-09-10T09:34:44.626 INFO:teuthology.orchestra.run.smithi077.stdout: fi 2024-09-10T09:34:44.626 INFO:teuthology.orchestra.run.smithi077.stdout:done 2024-09-10T09:34:44.627 INFO:teuthology.orchestra.run.smithi077.stdout:if [ -d "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" ] && \ 2024-09-10T09:34:44.627 INFO:teuthology.orchestra.run.smithi077.stdout: ! diff -rq "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" >/dev/null 2>&1; then 2024-09-10T09:34:44.627 INFO:teuthology.orchestra.run.smithi077.stdout: rm -rf "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" 2024-09-10T09:34:44.627 INFO:teuthology.orchestra.run.smithi077.stdout: cp -r "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" 2024-09-10T09:34:44.627 INFO:teuthology.orchestra.run.smithi077.stdout:fi 2024-09-10T09:34:44.627 INFO:teuthology.orchestra.run.smithi077.stdout:if [ ! -e "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep" ]; then 2024-09-10T09:34:44.627 INFO:teuthology.orchestra.run.smithi077.stdout: /usr/sbin/depmod 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T09:34:44.627 INFO:teuthology.orchestra.run.smithi077.stdout:fi 2024-09-10T09:34:44.627 INFO:teuthology.orchestra.run.smithi077.stdout:preuninstall scriptlet (using /bin/sh): 2024-09-10T09:34:44.627 INFO:teuthology.orchestra.run.smithi077.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-10T09:34:44.627 INFO:teuthology.orchestra.run.smithi077.stdout:kernel-install remove 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T09:34:44.627 INFO:teuthology.orchestra.run.smithi077.stdout:fi 2024-09-10T09:34:44.628 DEBUG:teuthology.orchestra.run.smithi077:> sudo rpm -qi grub2-tools 2024-09-10T09:34:44.707 INFO:teuthology.orchestra.run.smithi077.stdout:Name : grub2-tools 2024-09-10T09:34:44.707 INFO:teuthology.orchestra.run.smithi077.stdout:Epoch : 1 2024-09-10T09:34:44.707 INFO:teuthology.orchestra.run.smithi077.stdout:Version : 2.06 2024-09-10T09:34:44.707 INFO:teuthology.orchestra.run.smithi077.stdout:Release : 82.el9 2024-09-10T09:34:44.707 INFO:teuthology.orchestra.run.smithi077.stdout:Architecture: x86_64 2024-09-10T09:34:44.707 INFO:teuthology.orchestra.run.smithi077.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-10T09:34:44.707 INFO:teuthology.orchestra.run.smithi077.stdout:Group : Unspecified 2024-09-10T09:34:44.708 INFO:teuthology.orchestra.run.smithi077.stdout:Size : 8274670 2024-09-10T09:34:44.708 INFO:teuthology.orchestra.run.smithi077.stdout:License : GPLv3+ 2024-09-10T09:34:44.708 INFO:teuthology.orchestra.run.smithi077.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-10T09:34:44.708 INFO:teuthology.orchestra.run.smithi077.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-10T09:34:44.708 INFO:teuthology.orchestra.run.smithi077.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-10T09:34:44.708 INFO:teuthology.orchestra.run.smithi077.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-10T09:34:44.708 INFO:teuthology.orchestra.run.smithi077.stdout:Packager : builder@centos.org 2024-09-10T09:34:44.708 INFO:teuthology.orchestra.run.smithi077.stdout:Vendor : CentOS 2024-09-10T09:34:44.708 INFO:teuthology.orchestra.run.smithi077.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-10T09:34:44.708 INFO:teuthology.orchestra.run.smithi077.stdout:Summary : Support tools for GRUB. 2024-09-10T09:34:44.708 INFO:teuthology.orchestra.run.smithi077.stdout:Description : 2024-09-10T09:34:44.708 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:34:44.708 INFO:teuthology.orchestra.run.smithi077.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-10T09:34:44.709 INFO:teuthology.orchestra.run.smithi077.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-10T09:34:44.709 INFO:teuthology.orchestra.run.smithi077.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-10T09:34:44.709 INFO:teuthology.orchestra.run.smithi077.stdout:hardware devices. 2024-09-10T09:34:44.709 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:34:44.709 INFO:teuthology.orchestra.run.smithi077.stdout:This subpackage provides tools for support of all platforms. 2024-09-10T09:34:44.710 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-10T09:34:44.710 INFO:teuthology.task.kernel:Updating grub on smithi077 to boot 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T09:34:44.710 DEBUG:teuthology.orchestra.run.smithi077:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-10T09:34:45.263 INFO:teuthology.orchestra.run.smithi077.stderr:Generating grub configuration file ... 2024-09-10T09:34:46.641 INFO:teuthology.orchestra.run.smithi077.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-10T09:34:46.667 INFO:teuthology.orchestra.run.smithi077.stderr:done 2024-09-10T09:34:46.669 DEBUG:teuthology.orchestra.run.smithi077:> mktemp 2024-09-10T09:34:46.684 INFO:teuthology.orchestra.run.smithi077.stdout:/tmp/tmp.yuanZwWhwL 2024-09-10T09:34:46.684 DEBUG:teuthology.orchestra.run.smithi077:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.yuanZwWhwL 2024-09-10T09:34:46.751 DEBUG:teuthology.orchestra.run.smithi077:> sudo chmod 0666 /tmp/tmp.yuanZwWhwL 2024-09-10T09:34:46.896 DEBUG:teuthology.orchestra.remote:smithi077:/tmp/tmp.yuanZwWhwL is 6KB 2024-09-10T09:34:46.944 DEBUG:teuthology.orchestra.run.smithi077:> rm -fr /tmp/tmp.yuanZwWhwL 2024-09-10T09:34:46.958 DEBUG:teuthology.orchestra.run.smithi077:> sudo /bin/ls /boot/loader/entries || true 2024-09-10T09:34:47.020 INFO:teuthology.orchestra.run.smithi077.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-10T09:34:47.020 INFO:teuthology.orchestra.run.smithi077.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-10T09:34:47.021 INFO:teuthology.orchestra.run.smithi077.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-10T09:34:47.021 INFO:teuthology.orchestra.run.smithi077.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g2ca44a2f35ca.conf 2024-09-10T09:34:47.022 DEBUG:teuthology.orchestra.run.smithi077:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T09:34:47.177 DEBUG:teuthology.orchestra.run.smithi077:> sudo shutdown -r now 2024-09-10T09:35:17.180 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-10T09:35:17.181 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi077.front.sepia.ceph.com' 2024-09-10T09:35:17.182 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi077.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T09:35:35.741 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.77 2024-09-10T09:35:44.742 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi077.front.sepia.ceph.com' 2024-09-10T09:35:44.743 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi077.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T09:35:47.805 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.77 2024-09-10T09:35:59.810 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi077.front.sepia.ceph.com' 2024-09-10T09:35:59.811 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi077.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T09:36:18.237 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.77 2024-09-10T09:36:33.238 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi077.front.sepia.ceph.com' 2024-09-10T09:36:33.239 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi077.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T09:36:33.629 DEBUG:teuthology.orchestra.run.smithi077:> true 2024-09-10T09:36:34.116 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi077.front.sepia.ceph.com' 2024-09-10T09:36:34.116 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.11.0-rc5-g2ca44a2f35ca"... 2024-09-10T09:36:34.116 DEBUG:teuthology.orchestra.run.smithi077:> uname -r 2024-09-10T09:36:34.150 INFO:teuthology.orchestra.run.smithi077.stdout:6.11.0-rc5-g2ca44a2f35ca 2024-09-10T09:36:34.150 DEBUG:teuthology.task.kernel:current kernel version is 6.11.0-rc5-g2ca44a2f35ca vs 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T09:36:34.151 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-10T09:36:34.151 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-10T09:36:35.152 DEBUG:teuthology.parallel:result is None 2024-09-10T09:36:35.152 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-10T09:36:35.160 INFO:teuthology.task.internal:Creating test directory... 2024-09-10T09:36:35.161 DEBUG:teuthology.orchestra.run.smithi077:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-10T09:36:35.165 DEBUG:teuthology.orchestra.run.smithi120:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-10T09:36:35.167 DEBUG:teuthology.orchestra.run.smithi134:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-10T09:36:35.196 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-10T09:36:35.217 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-10T09:36:35.252 INFO:teuthology.task.internal:Creating archive directory... 2024-09-10T09:36:35.252 DEBUG:teuthology.orchestra.run.smithi077:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-10T09:36:35.256 DEBUG:teuthology.orchestra.run.smithi120:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-10T09:36:35.258 DEBUG:teuthology.orchestra.run.smithi134:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-10T09:36:35.300 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-10T09:36:35.307 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-10T09:36:35.307 DEBUG:teuthology.orchestra.run.smithi120:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T09:36:35.344 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T09:36:35.345 DEBUG:teuthology.orchestra.run.smithi134:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T09:36:35.361 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T09:36:35.362 DEBUG:teuthology.orchestra.run.smithi077:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-10T09:36:35.365 DEBUG:teuthology.orchestra.run.smithi120:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-10T09:36:35.387 DEBUG:teuthology.orchestra.run.smithi134:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-10T09:36:35.436 INFO:teuthology.orchestra.run.smithi120.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T09:36:35.438 INFO:teuthology.orchestra.run.smithi134.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T09:36:35.450 INFO:teuthology.orchestra.run.smithi120.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T09:36:35.471 INFO:teuthology.orchestra.run.smithi134.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T09:36:35.491 INFO:teuthology.orchestra.run.smithi077.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T09:36:35.565 INFO:teuthology.orchestra.run.smithi077.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T09:36:35.569 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-10T09:36:35.577 INFO:teuthology.task.internal:Configuring sudo... 2024-09-10T09:36:35.577 DEBUG:teuthology.orchestra.run.smithi077:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-10T09:36:35.580 DEBUG:teuthology.orchestra.run.smithi120:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-10T09:36:35.583 DEBUG:teuthology.orchestra.run.smithi134:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-10T09:36:35.634 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-10T09:36:35.642 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-10T09:36:35.643 DEBUG:teuthology.orchestra.run.smithi077:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-10T09:36:35.646 DEBUG:teuthology.orchestra.run.smithi120:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-10T09:36:35.653 DEBUG:teuthology.orchestra.run.smithi134:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-10T09:36:35.677 DEBUG:teuthology.orchestra.run.smithi077:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T09:36:35.774 DEBUG:teuthology.orchestra.run.smithi077:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T09:36:35.861 DEBUG:teuthology.orchestra.run.smithi077:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T09:36:35.918 DEBUG:teuthology.orchestra.run.smithi077:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T09:36:35.993 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-09-10T09:36:35.993 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-10T09:36:36.085 DEBUG:teuthology.orchestra.run.smithi120:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T09:36:36.126 DEBUG:teuthology.orchestra.run.smithi120:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T09:36:36.194 DEBUG:teuthology.orchestra.run.smithi120:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T09:36:36.236 DEBUG:teuthology.orchestra.run.smithi120:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T09:36:36.281 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-10T09:36:36.281 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-10T09:36:36.347 DEBUG:teuthology.orchestra.run.smithi134:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T09:36:36.392 DEBUG:teuthology.orchestra.run.smithi134:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T09:36:36.449 DEBUG:teuthology.orchestra.run.smithi134:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T09:36:36.533 DEBUG:teuthology.orchestra.run.smithi134:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T09:36:36.577 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-10T09:36:36.577 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-10T09:36:36.644 DEBUG:teuthology.orchestra.run.smithi077:> sudo service rsyslog restart 2024-09-10T09:36:36.648 DEBUG:teuthology.orchestra.run.smithi120:> sudo service rsyslog restart 2024-09-10T09:36:36.651 DEBUG:teuthology.orchestra.run.smithi134:> sudo service rsyslog restart 2024-09-10T09:36:36.689 INFO:teuthology.orchestra.run.smithi120.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T09:36:36.726 INFO:teuthology.orchestra.run.smithi077.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T09:36:36.726 INFO:teuthology.orchestra.run.smithi134.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T09:36:37.167 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-10T09:36:37.174 INFO:teuthology.task.internal:Starting timer... 2024-09-10T09:36:37.174 INFO:teuthology.run_tasks:Running task pcp... 2024-09-10T09:36:37.243 INFO:teuthology.run_tasks:Running task selinux... 2024-09-10T09:36:37.299 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-09-10T09:36:37.299 DEBUG:teuthology.orchestra.run.smithi077:> sudo service auditd rotate 2024-09-10T09:36:37.402 INFO:teuthology.orchestra.run.smithi077.stdout:Rotating logs: 2024-09-10T09:36:37.405 DEBUG:teuthology.orchestra.run.smithi120:> sudo service auditd rotate 2024-09-10T09:36:37.452 INFO:teuthology.orchestra.run.smithi120.stdout:Rotating logs: 2024-09-10T09:36:37.453 DEBUG:teuthology.orchestra.run.smithi134:> sudo service auditd rotate 2024-09-10T09:36:37.499 INFO:teuthology.orchestra.run.smithi134.stdout:Rotating logs: 2024-09-10T09:36:37.501 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-10T09:36:37.501 DEBUG:teuthology.orchestra.run.smithi077:> /usr/sbin/getenforce 2024-09-10T09:36:37.545 INFO:teuthology.orchestra.run.smithi077.stdout:Permissive 2024-09-10T09:36:37.545 DEBUG:teuthology.orchestra.run.smithi120:> /usr/sbin/getenforce 2024-09-10T09:36:37.573 INFO:teuthology.orchestra.run.smithi120.stdout:Permissive 2024-09-10T09:36:37.574 DEBUG:teuthology.orchestra.run.smithi134:> /usr/sbin/getenforce 2024-09-10T09:36:37.599 INFO:teuthology.orchestra.run.smithi134.stdout:Permissive 2024-09-10T09:36:37.599 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi077.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi120.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi134.front.sepia.ceph.com': 'permissive'} 2024-09-10T09:36:37.600 DEBUG:teuthology.orchestra.run.smithi077:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-10T09:36:37.655 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T09:36:37.656 DEBUG:teuthology.orchestra.run.smithi120:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-10T09:36:37.684 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T09:36:37.684 DEBUG:teuthology.orchestra.run.smithi134:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-10T09:36:37.711 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T09:36:37.711 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-10T09:36:37.711 DEBUG:teuthology.orchestra.run.smithi077:> sudo /usr/sbin/setenforce permissive 2024-09-10T09:36:37.844 DEBUG:teuthology.orchestra.run.smithi120:> sudo /usr/sbin/setenforce permissive 2024-09-10T09:36:37.872 DEBUG:teuthology.orchestra.run.smithi134:> sudo /usr/sbin/setenforce permissive 2024-09-10T09:36:37.902 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-10T09:36:37.955 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-10T09:36:37.966 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-10T09:36:37.967 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi077.front.sepia.ceph.com,smithi120.front.sepia.ceph.com,smithi134.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-10T09:41:15.095 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi077.front.sepia.ceph.com'), Remote(name='ubuntu@smithi120.front.sepia.ceph.com'), Remote(name='ubuntu@smithi134.front.sepia.ceph.com')] 2024-09-10T09:41:15.097 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi077.front.sepia.ceph.com' 2024-09-10T09:41:15.098 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi077.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T09:41:15.187 DEBUG:teuthology.orchestra.run.smithi077:> true 2024-09-10T09:41:15.290 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi077.front.sepia.ceph.com' 2024-09-10T09:41:15.316 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi120.front.sepia.ceph.com' 2024-09-10T09:41:15.316 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi120.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T09:41:15.394 DEBUG:teuthology.orchestra.run.smithi120:> true 2024-09-10T09:41:15.472 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi120.front.sepia.ceph.com' 2024-09-10T09:41:15.472 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi134.front.sepia.ceph.com' 2024-09-10T09:41:15.473 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi134.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T09:41:15.544 DEBUG:teuthology.orchestra.run.smithi134:> true 2024-09-10T09:41:15.586 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi134.front.sepia.ceph.com' 2024-09-10T09:41:15.587 INFO:teuthology.run_tasks:Running task clock... 2024-09-10T09:41:15.599 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-10T09:41:15.599 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-10T09:41:15.599 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T09:41:15.603 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-10T09:41:15.603 DEBUG:teuthology.orchestra.run.smithi120:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T09:41:15.606 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-10T09:41:15.606 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T09:41:15.633 INFO:teuthology.orchestra.run.smithi120.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-10T09:41:15.648 INFO:teuthology.orchestra.run.smithi120.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-10T09:41:15.654 INFO:teuthology.orchestra.run.smithi134.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-10T09:41:15.659 INFO:teuthology.orchestra.run.smithi077.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-10T09:41:15.669 INFO:teuthology.orchestra.run.smithi134.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-10T09:41:15.672 INFO:teuthology.orchestra.run.smithi120.stderr:sudo: ntpd: command not found 2024-09-10T09:41:15.683 INFO:teuthology.orchestra.run.smithi120.stdout:506 Cannot talk to daemon 2024-09-10T09:41:15.689 INFO:teuthology.orchestra.run.smithi077.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-10T09:41:15.692 INFO:teuthology.orchestra.run.smithi134.stderr:sudo: ntpd: command not found 2024-09-10T09:41:15.697 INFO:teuthology.orchestra.run.smithi120.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-10T09:41:15.704 INFO:teuthology.orchestra.run.smithi134.stdout:506 Cannot talk to daemon 2024-09-10T09:41:15.712 INFO:teuthology.orchestra.run.smithi120.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-10T09:41:15.718 INFO:teuthology.orchestra.run.smithi134.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-10T09:41:15.732 INFO:teuthology.orchestra.run.smithi134.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-10T09:41:15.739 INFO:teuthology.orchestra.run.smithi077.stderr:sudo: ntpd: command not found 2024-09-10T09:41:15.763 INFO:teuthology.orchestra.run.smithi077.stdout:506 Cannot talk to daemon 2024-09-10T09:41:15.773 INFO:teuthology.orchestra.run.smithi120.stderr:bash: line 1: ntpq: command not found 2024-09-10T09:41:15.776 INFO:teuthology.orchestra.run.smithi120.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T09:41:15.776 INFO:teuthology.orchestra.run.smithi120.stdout:=============================================================================== 2024-09-10T09:41:15.776 INFO:teuthology.orchestra.run.smithi120.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T09:41:15.776 INFO:teuthology.orchestra.run.smithi120.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T09:41:15.777 INFO:teuthology.orchestra.run.smithi120.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T09:41:15.777 INFO:teuthology.orchestra.run.smithi120.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T09:41:15.788 INFO:teuthology.orchestra.run.smithi134.stderr:bash: line 1: ntpq: command not found 2024-09-10T09:41:15.792 INFO:teuthology.orchestra.run.smithi134.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T09:41:15.792 INFO:teuthology.orchestra.run.smithi134.stdout:=============================================================================== 2024-09-10T09:41:15.792 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T09:41:15.792 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T09:41:15.792 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T09:41:15.792 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T09:41:15.793 INFO:teuthology.orchestra.run.smithi077.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-10T09:41:15.822 INFO:teuthology.orchestra.run.smithi077.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-10T09:41:15.925 INFO:teuthology.orchestra.run.smithi077.stderr:bash: line 1: ntpq: command not found 2024-09-10T09:41:15.932 INFO:teuthology.orchestra.run.smithi077.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T09:41:15.932 INFO:teuthology.orchestra.run.smithi077.stdout:=============================================================================== 2024-09-10T09:41:15.932 INFO:teuthology.orchestra.run.smithi077.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T09:41:15.932 INFO:teuthology.orchestra.run.smithi077.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T09:41:15.932 INFO:teuthology.orchestra.run.smithi077.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T09:41:15.932 INFO:teuthology.orchestra.run.smithi077.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T09:41:15.933 INFO:teuthology.run_tasks:Running task install... 2024-09-10T09:41:15.941 DEBUG:teuthology.task.install:project ceph 2024-09-10T09:41:15.942 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b'}} 2024-09-10T09:41:15.942 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b'} 2024-09-10T09:41:15.942 INFO:teuthology.task.install:Using flavor: default 2024-09-10T09:41:15.947 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-10T09:41:15.948 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-10T09:41:15.948 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b', 'tag': None, 'wait_for_package': False} 2024-09-10T09:41:15.948 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T09:41:15.950 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b', 'tag': None, 'wait_for_package': False} 2024-09-10T09:41:15.950 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T09:41:15.951 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b', 'tag': None, 'wait_for_package': False} 2024-09-10T09:41:15.951 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T09:41:16.099 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/ 2024-09-10T09:41:16.099 INFO:teuthology.task.install.rpm:Package version is 19.1.1-305.g1cf819b7 2024-09-10T09:41:16.102 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/ 2024-09-10T09:41:16.103 INFO:teuthology.task.install.rpm:Package version is 19.1.1-305.g1cf819b7 2024-09-10T09:41:16.120 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/ 2024-09-10T09:41:16.120 INFO:teuthology.task.install.rpm:Package version is 19.1.1-305.g1cf819b7 2024-09-10T09:41:16.211 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-10T09:41:16.211 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-09-10T09:41:16.211 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-10T09:41:16.216 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-10T09:41:16.216 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-10T09:41:16.216 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-10T09:41:16.231 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-10T09:41:16.231 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-10T09:41:16.231 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-10T09:41:16.243 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-10T09:41:16.243 DEBUG:teuthology.orchestra.run.smithi134:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1cf819b7212a6048e92fc78c4eff963344f6d56b/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-10T09:41:16.257 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-10T09:41:16.257 DEBUG:teuthology.orchestra.run.smithi120:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1cf819b7212a6048e92fc78c4eff963344f6d56b/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-10T09:41:16.267 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-10T09:41:16.267 DEBUG:teuthology.orchestra.run.smithi077:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1cf819b7212a6048e92fc78c4eff963344f6d56b/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-10T09:41:16.309 DEBUG:teuthology.orchestra.run.smithi134:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-10T09:41:16.323 DEBUG:teuthology.orchestra.run.smithi120:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-10T09:41:16.366 DEBUG:teuthology.orchestra.run.smithi077:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-10T09:41:16.384 DEBUG:teuthology.orchestra.run.smithi134:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-10T09:41:16.398 DEBUG:teuthology.orchestra.run.smithi120:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-10T09:41:16.444 DEBUG:teuthology.orchestra.run.smithi077:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-10T09:41:16.449 INFO:teuthology.orchestra.run.smithi134.stdout:check_obsoletes = 1 2024-09-10T09:41:16.450 DEBUG:teuthology.orchestra.run.smithi134:> sudo yum clean all 2024-09-10T09:41:16.461 INFO:teuthology.orchestra.run.smithi120.stdout:check_obsoletes = 1 2024-09-10T09:41:16.463 DEBUG:teuthology.orchestra.run.smithi120:> sudo yum clean all 2024-09-10T09:41:16.499 INFO:teuthology.orchestra.run.smithi077.stdout:check_obsoletes = 1 2024-09-10T09:41:16.502 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum clean all 2024-09-10T09:41:16.676 INFO:teuthology.orchestra.run.smithi134.stdout:45 files removed 2024-09-10T09:41:16.688 INFO:teuthology.orchestra.run.smithi120.stdout:45 files removed 2024-09-10T09:41:16.704 DEBUG:teuthology.orchestra.run.smithi134:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-10T09:41:16.715 DEBUG:teuthology.orchestra.run.smithi120:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-10T09:41:16.794 INFO:teuthology.orchestra.run.smithi077.stdout:45 files removed 2024-09-10T09:41:16.828 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-10T09:41:17.251 INFO:teuthology.orchestra.run.smithi134.stdout:ceph packages for x86_64 312 kB/s | 84 kB 00:00 2024-09-10T09:41:17.260 INFO:teuthology.orchestra.run.smithi120.stdout:ceph packages for x86_64 309 kB/s | 84 kB 00:00 2024-09-10T09:41:17.518 INFO:teuthology.orchestra.run.smithi134.stdout:ceph noarch packages 53 kB/s | 12 kB 00:00 2024-09-10T09:41:17.552 INFO:teuthology.orchestra.run.smithi120.stdout:ceph noarch packages 47 kB/s | 12 kB 00:00 2024-09-10T09:41:17.560 INFO:teuthology.orchestra.run.smithi077.stdout:ceph packages for x86_64 249 kB/s | 84 kB 00:00 2024-09-10T09:41:17.778 INFO:teuthology.orchestra.run.smithi134.stdout:ceph source packages 8.5 kB/s | 1.9 kB 00:00 2024-09-10T09:41:17.811 INFO:teuthology.orchestra.run.smithi120.stdout:ceph source packages 8.5 kB/s | 1.9 kB 00:00 2024-09-10T09:41:17.861 INFO:teuthology.orchestra.run.smithi077.stdout:ceph noarch packages 46 kB/s | 12 kB 00:00 2024-09-10T09:41:18.137 INFO:teuthology.orchestra.run.smithi077.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-10T09:41:18.528 INFO:teuthology.orchestra.run.smithi120.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.2 MB 00:00 2024-09-10T09:41:19.167 INFO:teuthology.orchestra.run.smithi077.stdout:CentOS Stream 9 - BaseOS 8.3 MB/s | 8.2 MB 00:00 2024-09-10T09:41:19.194 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 9 - BaseOS 6.0 MB/s | 8.2 MB 00:01 2024-09-10T09:41:21.038 INFO:teuthology.orchestra.run.smithi077.stdout:CentOS Stream 9 - AppStream 28 MB/s | 20 MB 00:00 2024-09-10T09:41:22.379 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 9 - AppStream 9.9 MB/s | 20 MB 00:02 2024-09-10T09:41:26.214 INFO:teuthology.orchestra.run.smithi077.stdout:CentOS Stream 9 - CRB 5.6 MB/s | 6.5 MB 00:01 2024-09-10T09:41:28.514 INFO:teuthology.orchestra.run.smithi077.stdout:CentOS Stream 9 - Extras packages 18 kB/s | 19 kB 00:01 2024-09-10T09:41:28.654 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 9 - CRB 2.6 MB/s | 6.5 MB 00:02 2024-09-10T09:41:29.629 INFO:teuthology.orchestra.run.smithi077.stdout:Extra Packages for Enterprise Linux 23 MB/s | 23 MB 00:00 2024-09-10T09:41:30.130 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 9 - Extras packages 70 kB/s | 19 kB 00:00 2024-09-10T09:41:31.109 INFO:teuthology.orchestra.run.smithi134.stdout:Extra Packages for Enterprise Linux 26 MB/s | 23 MB 00:00 2024-09-10T09:41:35.993 INFO:teuthology.orchestra.run.smithi077.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-10T09:41:36.489 INFO:teuthology.orchestra.run.smithi134.stdout:lab-extras 48 kB/s | 1.7 kB 00:00 2024-09-10T09:41:36.677 INFO:teuthology.orchestra.run.smithi120.stdout:CentOS Stream 9 - AppStream 1.2 MB/s | 20 MB 00:17 2024-09-10T09:41:38.465 INFO:teuthology.orchestra.run.smithi077.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T09:41:38.466 INFO:teuthology.orchestra.run.smithi077.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T09:41:38.475 INFO:teuthology.orchestra.run.smithi077.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-10T09:41:38.476 INFO:teuthology.orchestra.run.smithi077.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-10T09:41:38.477 INFO:teuthology.orchestra.run.smithi077.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-10T09:41:38.478 INFO:teuthology.orchestra.run.smithi077.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-10T09:41:38.480 INFO:teuthology.orchestra.run.smithi077.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-10T09:41:38.482 INFO:teuthology.orchestra.run.smithi077.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-10T09:41:38.483 INFO:teuthology.orchestra.run.smithi077.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-10T09:41:38.547 INFO:teuthology.orchestra.run.smithi077.stdout:Dependencies resolved. 2024-09-10T09:41:38.571 INFO:teuthology.orchestra.run.smithi077.stdout:====================================================================================== 2024-09-10T09:41:38.571 INFO:teuthology.orchestra.run.smithi077.stdout: Package Arch Version Repository Size 2024-09-10T09:41:38.571 INFO:teuthology.orchestra.run.smithi077.stdout:====================================================================================== 2024-09-10T09:41:38.571 INFO:teuthology.orchestra.run.smithi077.stdout:Installing: 2024-09-10T09:41:38.571 INFO:teuthology.orchestra.run.smithi077.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-10T09:41:38.571 INFO:teuthology.orchestra.run.smithi077.stdout: ceph x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 6.5 k 2024-09-10T09:41:38.571 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-base x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 5.4 M 2024-09-10T09:41:38.572 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-fuse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 897 k 2024-09-10T09:41:38.572 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 145 k 2024-09-10T09:41:38.572 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 1.5 M 2024-09-10T09:41:38.572 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-cephadm noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 146 k 2024-09-10T09:41:38.572 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-dashboard noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 3.6 M 2024-09-10T09:41:38.572 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 7.4 M 2024-09-10T09:41:38.572 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-rook noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 49 k 2024-09-10T09:41:38.572 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-radosgw x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 11 M 2024-09-10T09:41:38.572 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-test x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 49 M 2024-09-10T09:41:38.572 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-volume noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 285 k 2024-09-10T09:41:38.572 INFO:teuthology.orchestra.run.smithi077.stdout: cephadm noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 766 k 2024-09-10T09:41:38.572 INFO:teuthology.orchestra.run.smithi077.stdout: cephfs-mirror x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 228 k 2024-09-10T09:41:38.572 INFO:teuthology.orchestra.run.smithi077.stdout: cephfs-shell x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 27 k 2024-09-10T09:41:38.573 INFO:teuthology.orchestra.run.smithi077.stdout: cephfs-top noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 18 k 2024-09-10T09:41:38.573 INFO:teuthology.orchestra.run.smithi077.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-10T09:41:38.573 INFO:teuthology.orchestra.run.smithi077.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-10T09:41:38.573 INFO:teuthology.orchestra.run.smithi077.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-10T09:41:38.573 INFO:teuthology.orchestra.run.smithi077.stdout: libcephfs-devel x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 32 k 2024-09-10T09:41:38.573 INFO:teuthology.orchestra.run.smithi077.stdout: libcephfs2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 751 k 2024-09-10T09:41:38.573 INFO:teuthology.orchestra.run.smithi077.stdout: librados-devel x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 127 k 2024-09-10T09:41:38.573 INFO:teuthology.orchestra.run.smithi077.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-10T09:41:38.573 INFO:teuthology.orchestra.run.smithi077.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-10T09:41:38.573 INFO:teuthology.orchestra.run.smithi077.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-10T09:41:38.573 INFO:teuthology.orchestra.run.smithi077.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-10T09:41:38.573 INFO:teuthology.orchestra.run.smithi077.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-10T09:41:38.574 INFO:teuthology.orchestra.run.smithi077.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-10T09:41:38.574 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cephfs x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 161 k 2024-09-10T09:41:38.574 INFO:teuthology.orchestra.run.smithi077.stdout: python3-rados x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 320 k 2024-09-10T09:41:38.574 INFO:teuthology.orchestra.run.smithi077.stdout: python3-rbd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 301 k 2024-09-10T09:41:38.574 INFO:teuthology.orchestra.run.smithi077.stdout: python3-rgw x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 99 k 2024-09-10T09:41:38.574 INFO:teuthology.orchestra.run.smithi077.stdout: rbd-fuse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 85 k 2024-09-10T09:41:38.574 INFO:teuthology.orchestra.run.smithi077.stdout: rbd-mirror x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.1 M 2024-09-10T09:41:38.574 INFO:teuthology.orchestra.run.smithi077.stdout: rbd-nbd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 170 k 2024-09-10T09:41:38.574 INFO:teuthology.orchestra.run.smithi077.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-10T09:41:38.574 INFO:teuthology.orchestra.run.smithi077.stdout:Upgrading: 2024-09-10T09:41:38.574 INFO:teuthology.orchestra.run.smithi077.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-10T09:41:38.575 INFO:teuthology.orchestra.run.smithi077.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-10T09:41:38.575 INFO:teuthology.orchestra.run.smithi077.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-10T09:41:38.575 INFO:teuthology.orchestra.run.smithi077.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-10T09:41:38.575 INFO:teuthology.orchestra.run.smithi077.stdout: librados2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.4 M 2024-09-10T09:41:38.575 INFO:teuthology.orchestra.run.smithi077.stdout: librbd1 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.2 M 2024-09-10T09:41:38.575 INFO:teuthology.orchestra.run.smithi077.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-10T09:41:38.575 INFO:teuthology.orchestra.run.smithi077.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-10T09:41:38.575 INFO:teuthology.orchestra.run.smithi077.stdout:Installing dependencies: 2024-09-10T09:41:38.575 INFO:teuthology.orchestra.run.smithi077.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-10T09:41:38.575 INFO:teuthology.orchestra.run.smithi077.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-10T09:41:38.575 INFO:teuthology.orchestra.run.smithi077.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-10T09:41:38.575 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-common x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 21 M 2024-09-10T09:41:38.576 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-grafana-dashboards noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 32 k 2024-09-10T09:41:38.576 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mds x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 2.4 M 2024-09-10T09:41:38.576 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-modules-core noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 249 k 2024-09-10T09:41:38.576 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mon x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 4.7 M 2024-09-10T09:41:38.576 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-osd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 17 M 2024-09-10T09:41:38.576 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-prometheus-alerts noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 16 k 2024-09-10T09:41:38.576 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-selinux x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 25 k 2024-09-10T09:41:38.576 INFO:teuthology.orchestra.run.smithi077.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-10T09:41:38.576 INFO:teuthology.orchestra.run.smithi077.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-10T09:41:38.576 INFO:teuthology.orchestra.run.smithi077.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-10T09:41:38.576 INFO:teuthology.orchestra.run.smithi077.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-10T09:41:38.576 INFO:teuthology.orchestra.run.smithi077.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-10T09:41:38.577 INFO:teuthology.orchestra.run.smithi077.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-10T09:41:38.577 INFO:teuthology.orchestra.run.smithi077.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-10T09:41:38.577 INFO:teuthology.orchestra.run.smithi077.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-10T09:41:38.577 INFO:teuthology.orchestra.run.smithi077.stdout: libcephsqlite x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 164 k 2024-09-10T09:41:38.577 INFO:teuthology.orchestra.run.smithi077.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-10T09:41:38.577 INFO:teuthology.orchestra.run.smithi077.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-10T09:41:38.577 INFO:teuthology.orchestra.run.smithi077.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-10T09:41:38.577 INFO:teuthology.orchestra.run.smithi077.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-10T09:41:38.577 INFO:teuthology.orchestra.run.smithi077.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-10T09:41:38.577 INFO:teuthology.orchestra.run.smithi077.stdout: libradosstriper1 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 506 k 2024-09-10T09:41:38.577 INFO:teuthology.orchestra.run.smithi077.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-10T09:41:38.577 INFO:teuthology.orchestra.run.smithi077.stdout: librgw2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 5.3 M 2024-09-10T09:41:38.577 INFO:teuthology.orchestra.run.smithi077.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-10T09:41:38.578 INFO:teuthology.orchestra.run.smithi077.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-10T09:41:38.578 INFO:teuthology.orchestra.run.smithi077.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-10T09:41:38.578 INFO:teuthology.orchestra.run.smithi077.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-10T09:41:38.578 INFO:teuthology.orchestra.run.smithi077.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-10T09:41:38.578 INFO:teuthology.orchestra.run.smithi077.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-10T09:41:38.578 INFO:teuthology.orchestra.run.smithi077.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-10T09:41:38.578 INFO:teuthology.orchestra.run.smithi077.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-10T09:41:38.578 INFO:teuthology.orchestra.run.smithi077.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-10T09:41:38.578 INFO:teuthology.orchestra.run.smithi077.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-10T09:41:38.578 INFO:teuthology.orchestra.run.smithi077.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-10T09:41:38.578 INFO:teuthology.orchestra.run.smithi077.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-10T09:41:38.578 INFO:teuthology.orchestra.run.smithi077.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-10T09:41:38.578 INFO:teuthology.orchestra.run.smithi077.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-10T09:41:38.579 INFO:teuthology.orchestra.run.smithi077.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-10T09:41:38.579 INFO:teuthology.orchestra.run.smithi077.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-10T09:41:38.579 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-10T09:41:38.579 INFO:teuthology.orchestra.run.smithi077.stdout: python3-ceph-argparse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 45 k 2024-09-10T09:41:38.579 INFO:teuthology.orchestra.run.smithi077.stdout: python3-ceph-common x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 136 k 2024-09-10T09:41:38.579 INFO:teuthology.orchestra.run.smithi077.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-10T09:41:38.579 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-10T09:41:38.579 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-10T09:41:38.579 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-10T09:41:38.579 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-10T09:41:38.580 INFO:teuthology.orchestra.run.smithi077.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-10T09:41:38.580 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-10T09:41:38.580 INFO:teuthology.orchestra.run.smithi077.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-10T09:41:38.580 INFO:teuthology.orchestra.run.smithi077.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-10T09:41:38.580 INFO:teuthology.orchestra.run.smithi077.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-10T09:41:38.580 INFO:teuthology.orchestra.run.smithi077.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-10T09:41:38.580 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-10T09:41:38.580 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-10T09:41:38.580 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-10T09:41:38.580 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-10T09:41:38.580 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-10T09:41:38.580 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-10T09:41:38.580 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-10T09:41:38.581 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-10T09:41:38.581 INFO:teuthology.orchestra.run.smithi077.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-10T09:41:38.581 INFO:teuthology.orchestra.run.smithi077.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-10T09:41:38.581 INFO:teuthology.orchestra.run.smithi077.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-10T09:41:38.581 INFO:teuthology.orchestra.run.smithi077.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-10T09:41:38.581 INFO:teuthology.orchestra.run.smithi077.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-10T09:41:38.581 INFO:teuthology.orchestra.run.smithi077.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-10T09:41:38.581 INFO:teuthology.orchestra.run.smithi077.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-10T09:41:38.581 INFO:teuthology.orchestra.run.smithi077.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-10T09:41:38.581 INFO:teuthology.orchestra.run.smithi077.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-10T09:41:38.581 INFO:teuthology.orchestra.run.smithi077.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-10T09:41:38.581 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-10T09:41:38.581 INFO:teuthology.orchestra.run.smithi077.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-10T09:41:38.581 INFO:teuthology.orchestra.run.smithi077.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-10T09:41:38.582 INFO:teuthology.orchestra.run.smithi077.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-10T09:41:38.582 INFO:teuthology.orchestra.run.smithi077.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-10T09:41:38.582 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-10T09:41:38.582 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-10T09:41:38.582 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-10T09:41:38.582 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-10T09:41:38.582 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-10T09:41:38.582 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-10T09:41:38.582 INFO:teuthology.orchestra.run.smithi077.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-10T09:41:38.582 INFO:teuthology.orchestra.run.smithi077.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-10T09:41:38.582 INFO:teuthology.orchestra.run.smithi077.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-10T09:41:38.582 INFO:teuthology.orchestra.run.smithi077.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-10T09:41:38.582 INFO:teuthology.orchestra.run.smithi077.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-10T09:41:38.582 INFO:teuthology.orchestra.run.smithi077.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-10T09:41:38.583 INFO:teuthology.orchestra.run.smithi077.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-10T09:41:38.583 INFO:teuthology.orchestra.run.smithi077.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-10T09:41:38.583 INFO:teuthology.orchestra.run.smithi077.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-10T09:41:38.583 INFO:teuthology.orchestra.run.smithi077.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-10T09:41:38.583 INFO:teuthology.orchestra.run.smithi077.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-10T09:41:38.583 INFO:teuthology.orchestra.run.smithi077.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-10T09:41:38.583 INFO:teuthology.orchestra.run.smithi077.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-10T09:41:38.583 INFO:teuthology.orchestra.run.smithi077.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-10T09:41:38.583 INFO:teuthology.orchestra.run.smithi077.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-10T09:41:38.583 INFO:teuthology.orchestra.run.smithi077.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-10T09:41:38.583 INFO:teuthology.orchestra.run.smithi077.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-10T09:41:38.583 INFO:teuthology.orchestra.run.smithi077.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-10T09:41:38.583 INFO:teuthology.orchestra.run.smithi077.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-10T09:41:38.584 INFO:teuthology.orchestra.run.smithi077.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-10T09:41:38.584 INFO:teuthology.orchestra.run.smithi077.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-10T09:41:38.584 INFO:teuthology.orchestra.run.smithi077.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-10T09:41:38.584 INFO:teuthology.orchestra.run.smithi077.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-10T09:41:38.584 INFO:teuthology.orchestra.run.smithi077.stdout:Installing weak dependencies: 2024-09-10T09:41:38.584 INFO:teuthology.orchestra.run.smithi077.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-10T09:41:38.584 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:41:38.584 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction Summary 2024-09-10T09:41:38.584 INFO:teuthology.orchestra.run.smithi077.stdout:====================================================================================== 2024-09-10T09:41:38.584 INFO:teuthology.orchestra.run.smithi077.stdout:Install 148 Packages 2024-09-10T09:41:38.584 INFO:teuthology.orchestra.run.smithi077.stdout:Upgrade 8 Packages 2024-09-10T09:41:38.584 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:41:38.585 INFO:teuthology.orchestra.run.smithi077.stdout:Total download size: 226 M 2024-09-10T09:41:38.585 INFO:teuthology.orchestra.run.smithi077.stdout:Downloading Packages: 2024-09-10T09:41:38.816 INFO:teuthology.orchestra.run.smithi134.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T09:41:38.817 INFO:teuthology.orchestra.run.smithi134.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T09:41:38.828 INFO:teuthology.orchestra.run.smithi134.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-10T09:41:38.829 INFO:teuthology.orchestra.run.smithi134.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-10T09:41:38.830 INFO:teuthology.orchestra.run.smithi134.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-10T09:41:38.831 INFO:teuthology.orchestra.run.smithi134.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-10T09:41:38.833 INFO:teuthology.orchestra.run.smithi134.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-10T09:41:38.835 INFO:teuthology.orchestra.run.smithi134.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-10T09:41:38.836 INFO:teuthology.orchestra.run.smithi134.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-10T09:41:38.899 INFO:teuthology.orchestra.run.smithi134.stdout:Dependencies resolved. 2024-09-10T09:41:38.929 INFO:teuthology.orchestra.run.smithi134.stdout:====================================================================================== 2024-09-10T09:41:38.929 INFO:teuthology.orchestra.run.smithi134.stdout: Package Arch Version Repository Size 2024-09-10T09:41:38.929 INFO:teuthology.orchestra.run.smithi134.stdout:====================================================================================== 2024-09-10T09:41:38.929 INFO:teuthology.orchestra.run.smithi134.stdout:Installing: 2024-09-10T09:41:38.929 INFO:teuthology.orchestra.run.smithi134.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-10T09:41:38.930 INFO:teuthology.orchestra.run.smithi134.stdout: ceph x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 6.5 k 2024-09-10T09:41:38.930 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-base x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 5.4 M 2024-09-10T09:41:38.930 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-fuse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 897 k 2024-09-10T09:41:38.930 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 145 k 2024-09-10T09:41:38.930 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 1.5 M 2024-09-10T09:41:38.930 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-cephadm noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 146 k 2024-09-10T09:41:38.930 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-dashboard noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 3.6 M 2024-09-10T09:41:38.930 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 7.4 M 2024-09-10T09:41:38.930 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-rook noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 49 k 2024-09-10T09:41:38.930 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-radosgw x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 11 M 2024-09-10T09:41:38.930 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-test x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 49 M 2024-09-10T09:41:38.930 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-volume noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 285 k 2024-09-10T09:41:38.930 INFO:teuthology.orchestra.run.smithi134.stdout: cephadm noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 766 k 2024-09-10T09:41:38.930 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-mirror x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 228 k 2024-09-10T09:41:38.931 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-shell x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 27 k 2024-09-10T09:41:38.931 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-top noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 18 k 2024-09-10T09:41:38.931 INFO:teuthology.orchestra.run.smithi134.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-10T09:41:38.931 INFO:teuthology.orchestra.run.smithi134.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-10T09:41:38.931 INFO:teuthology.orchestra.run.smithi134.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-10T09:41:38.931 INFO:teuthology.orchestra.run.smithi134.stdout: libcephfs-devel x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 32 k 2024-09-10T09:41:38.931 INFO:teuthology.orchestra.run.smithi134.stdout: libcephfs2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 751 k 2024-09-10T09:41:38.931 INFO:teuthology.orchestra.run.smithi134.stdout: librados-devel x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 127 k 2024-09-10T09:41:38.931 INFO:teuthology.orchestra.run.smithi134.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-10T09:41:38.932 INFO:teuthology.orchestra.run.smithi134.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-10T09:41:38.932 INFO:teuthology.orchestra.run.smithi134.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-10T09:41:38.932 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-10T09:41:38.932 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-10T09:41:38.932 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-10T09:41:38.932 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cephfs x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 161 k 2024-09-10T09:41:38.932 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rados x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 320 k 2024-09-10T09:41:38.932 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rbd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 301 k 2024-09-10T09:41:38.933 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rgw x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 99 k 2024-09-10T09:41:38.933 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-fuse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 85 k 2024-09-10T09:41:38.933 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-mirror x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.1 M 2024-09-10T09:41:38.933 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-nbd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 170 k 2024-09-10T09:41:38.933 INFO:teuthology.orchestra.run.smithi134.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-10T09:41:38.933 INFO:teuthology.orchestra.run.smithi134.stdout:Upgrading: 2024-09-10T09:41:38.933 INFO:teuthology.orchestra.run.smithi134.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-10T09:41:38.933 INFO:teuthology.orchestra.run.smithi134.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-10T09:41:38.933 INFO:teuthology.orchestra.run.smithi134.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-10T09:41:38.933 INFO:teuthology.orchestra.run.smithi134.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-10T09:41:38.933 INFO:teuthology.orchestra.run.smithi134.stdout: librados2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.4 M 2024-09-10T09:41:38.933 INFO:teuthology.orchestra.run.smithi134.stdout: librbd1 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.2 M 2024-09-10T09:41:38.933 INFO:teuthology.orchestra.run.smithi134.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-10T09:41:38.934 INFO:teuthology.orchestra.run.smithi134.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-10T09:41:38.934 INFO:teuthology.orchestra.run.smithi134.stdout:Installing dependencies: 2024-09-10T09:41:38.934 INFO:teuthology.orchestra.run.smithi134.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-10T09:41:38.934 INFO:teuthology.orchestra.run.smithi134.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-10T09:41:38.934 INFO:teuthology.orchestra.run.smithi134.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-10T09:41:38.934 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-common x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 21 M 2024-09-10T09:41:38.934 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-grafana-dashboards noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 32 k 2024-09-10T09:41:38.934 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mds x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 2.4 M 2024-09-10T09:41:38.934 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-modules-core noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 249 k 2024-09-10T09:41:38.934 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mon x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 4.7 M 2024-09-10T09:41:38.934 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-osd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 17 M 2024-09-10T09:41:38.934 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-prometheus-alerts noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 16 k 2024-09-10T09:41:38.935 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-selinux x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 25 k 2024-09-10T09:41:38.935 INFO:teuthology.orchestra.run.smithi134.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-10T09:41:38.935 INFO:teuthology.orchestra.run.smithi134.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-10T09:41:38.935 INFO:teuthology.orchestra.run.smithi134.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-10T09:41:38.935 INFO:teuthology.orchestra.run.smithi134.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-10T09:41:38.935 INFO:teuthology.orchestra.run.smithi134.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-10T09:41:38.935 INFO:teuthology.orchestra.run.smithi134.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-10T09:41:38.935 INFO:teuthology.orchestra.run.smithi134.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-10T09:41:38.935 INFO:teuthology.orchestra.run.smithi134.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-10T09:41:38.935 INFO:teuthology.orchestra.run.smithi134.stdout: libcephsqlite x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 164 k 2024-09-10T09:41:38.935 INFO:teuthology.orchestra.run.smithi134.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-10T09:41:38.935 INFO:teuthology.orchestra.run.smithi134.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-10T09:41:38.935 INFO:teuthology.orchestra.run.smithi134.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-10T09:41:38.936 INFO:teuthology.orchestra.run.smithi134.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-10T09:41:38.936 INFO:teuthology.orchestra.run.smithi134.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-10T09:41:38.936 INFO:teuthology.orchestra.run.smithi134.stdout: libradosstriper1 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 506 k 2024-09-10T09:41:38.936 INFO:teuthology.orchestra.run.smithi134.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-10T09:41:38.936 INFO:teuthology.orchestra.run.smithi134.stdout: librgw2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 5.3 M 2024-09-10T09:41:38.936 INFO:teuthology.orchestra.run.smithi134.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-10T09:41:38.936 INFO:teuthology.orchestra.run.smithi134.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-10T09:41:38.936 INFO:teuthology.orchestra.run.smithi134.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-10T09:41:38.936 INFO:teuthology.orchestra.run.smithi134.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-10T09:41:38.936 INFO:teuthology.orchestra.run.smithi134.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-10T09:41:38.936 INFO:teuthology.orchestra.run.smithi134.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-10T09:41:38.936 INFO:teuthology.orchestra.run.smithi134.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-10T09:41:38.936 INFO:teuthology.orchestra.run.smithi134.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-10T09:41:38.937 INFO:teuthology.orchestra.run.smithi134.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-10T09:41:38.937 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-10T09:41:38.937 INFO:teuthology.orchestra.run.smithi134.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-10T09:41:38.937 INFO:teuthology.orchestra.run.smithi134.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-10T09:41:38.937 INFO:teuthology.orchestra.run.smithi134.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-10T09:41:38.937 INFO:teuthology.orchestra.run.smithi134.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-10T09:41:38.937 INFO:teuthology.orchestra.run.smithi134.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-10T09:41:38.937 INFO:teuthology.orchestra.run.smithi134.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-10T09:41:38.937 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-10T09:41:38.937 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ceph-argparse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 45 k 2024-09-10T09:41:38.937 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ceph-common x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 136 k 2024-09-10T09:41:38.937 INFO:teuthology.orchestra.run.smithi134.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-10T09:41:38.937 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-10T09:41:38.937 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-10T09:41:38.938 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-10T09:41:38.938 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-10T09:41:38.938 INFO:teuthology.orchestra.run.smithi134.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-10T09:41:38.938 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-10T09:41:38.938 INFO:teuthology.orchestra.run.smithi134.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-10T09:41:38.938 INFO:teuthology.orchestra.run.smithi134.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-10T09:41:38.938 INFO:teuthology.orchestra.run.smithi134.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-10T09:41:38.938 INFO:teuthology.orchestra.run.smithi134.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-10T09:41:38.938 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-10T09:41:38.938 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-10T09:41:38.938 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-10T09:41:38.938 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-10T09:41:38.938 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-10T09:41:38.939 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-10T09:41:38.939 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-10T09:41:38.939 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-10T09:41:38.939 INFO:teuthology.orchestra.run.smithi134.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-10T09:41:38.939 INFO:teuthology.orchestra.run.smithi134.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-10T09:41:38.939 INFO:teuthology.orchestra.run.smithi134.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-10T09:41:38.939 INFO:teuthology.orchestra.run.smithi134.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-10T09:41:38.939 INFO:teuthology.orchestra.run.smithi134.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-10T09:41:38.939 INFO:teuthology.orchestra.run.smithi134.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-10T09:41:38.939 INFO:teuthology.orchestra.run.smithi134.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-10T09:41:38.939 INFO:teuthology.orchestra.run.smithi134.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-10T09:41:38.939 INFO:teuthology.orchestra.run.smithi134.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-10T09:41:38.939 INFO:teuthology.orchestra.run.smithi134.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-10T09:41:38.939 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-10T09:41:38.939 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-10T09:41:38.940 INFO:teuthology.orchestra.run.smithi134.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-10T09:41:38.940 INFO:teuthology.orchestra.run.smithi134.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-10T09:41:38.940 INFO:teuthology.orchestra.run.smithi134.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-10T09:41:38.940 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-10T09:41:38.940 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-10T09:41:38.940 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-10T09:41:38.940 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-10T09:41:38.940 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-10T09:41:38.940 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-10T09:41:38.941 INFO:teuthology.orchestra.run.smithi134.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-10T09:41:38.941 INFO:teuthology.orchestra.run.smithi134.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-10T09:41:38.941 INFO:teuthology.orchestra.run.smithi134.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-10T09:41:38.941 INFO:teuthology.orchestra.run.smithi134.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-10T09:41:38.941 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-10T09:41:38.941 INFO:teuthology.orchestra.run.smithi134.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-10T09:41:38.941 INFO:teuthology.orchestra.run.smithi134.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-10T09:41:38.941 INFO:teuthology.orchestra.run.smithi134.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-10T09:41:38.941 INFO:teuthology.orchestra.run.smithi134.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-10T09:41:38.941 INFO:teuthology.orchestra.run.smithi134.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-10T09:41:38.941 INFO:teuthology.orchestra.run.smithi134.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-10T09:41:38.941 INFO:teuthology.orchestra.run.smithi134.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-10T09:41:38.942 INFO:teuthology.orchestra.run.smithi134.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-10T09:41:38.942 INFO:teuthology.orchestra.run.smithi134.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-10T09:41:38.942 INFO:teuthology.orchestra.run.smithi134.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-10T09:41:38.942 INFO:teuthology.orchestra.run.smithi134.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-10T09:41:38.942 INFO:teuthology.orchestra.run.smithi134.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-10T09:41:38.942 INFO:teuthology.orchestra.run.smithi134.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-10T09:41:38.942 INFO:teuthology.orchestra.run.smithi134.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-10T09:41:38.942 INFO:teuthology.orchestra.run.smithi134.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-10T09:41:38.942 INFO:teuthology.orchestra.run.smithi134.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-10T09:41:38.942 INFO:teuthology.orchestra.run.smithi134.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-10T09:41:38.942 INFO:teuthology.orchestra.run.smithi134.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-10T09:41:38.942 INFO:teuthology.orchestra.run.smithi134.stdout:Installing weak dependencies: 2024-09-10T09:41:38.942 INFO:teuthology.orchestra.run.smithi134.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-10T09:41:38.943 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-10T09:41:38.943 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction Summary 2024-09-10T09:41:38.943 INFO:teuthology.orchestra.run.smithi134.stdout:====================================================================================== 2024-09-10T09:41:38.943 INFO:teuthology.orchestra.run.smithi134.stdout:Install 148 Packages 2024-09-10T09:41:38.943 INFO:teuthology.orchestra.run.smithi134.stdout:Upgrade 8 Packages 2024-09-10T09:41:38.943 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-10T09:41:38.943 INFO:teuthology.orchestra.run.smithi134.stdout:Total download size: 226 M 2024-09-10T09:41:38.943 INFO:teuthology.orchestra.run.smithi134.stdout:Downloading Packages: 2024-09-10T09:41:39.186 INFO:teuthology.orchestra.run.smithi077.stdout:(1/156): ceph-19.1.1-305.g1cf819b7.el9.x86_64.r 22 kB/s | 6.5 kB 00:00 2024-09-10T09:41:39.416 INFO:teuthology.orchestra.run.smithi134.stdout:(1/156): ceph-19.1.1-305.g1cf819b7.el9.x86_64.r 25 kB/s | 6.5 kB 00:00 2024-09-10T09:41:40.183 INFO:teuthology.orchestra.run.smithi134.stdout:(2/156): ceph-fuse-19.1.1-305.g1cf819b7.el9.x86 1.1 MB/s | 897 kB 00:00 2024-09-10T09:41:40.203 INFO:teuthology.orchestra.run.smithi077.stdout:(2/156): ceph-fuse-19.1.1-305.g1cf819b7.el9.x86 882 kB/s | 897 kB 00:01 2024-09-10T09:41:40.299 INFO:teuthology.orchestra.run.smithi134.stdout:(3/156): ceph-immutable-object-cache-19.1.1-305 1.2 MB/s | 145 kB 00:00 2024-09-10T09:41:40.303 INFO:teuthology.orchestra.run.smithi077.stdout:(3/156): ceph-immutable-object-cache-19.1.1-305 1.4 MB/s | 145 kB 00:00 2024-09-10T09:41:40.791 INFO:teuthology.orchestra.run.smithi134.stdout:(4/156): ceph-base-19.1.1-305.g1cf819b7.el9.x86 3.3 MB/s | 5.4 MB 00:01 2024-09-10T09:41:41.150 INFO:teuthology.orchestra.run.smithi134.stdout:(5/156): ceph-mgr-19.1.1-305.g1cf819b7.el9.x86_ 4.2 MB/s | 1.5 MB 00:00 2024-09-10T09:41:41.545 INFO:teuthology.orchestra.run.smithi077.stdout:(4/156): ceph-base-19.1.1-305.g1cf819b7.el9.x86 2.0 MB/s | 5.4 MB 00:02 2024-09-10T09:41:41.633 INFO:teuthology.orchestra.run.smithi134.stdout:(6/156): ceph-common-19.1.1-305.g1cf819b7.el9.x 8.5 MB/s | 21 MB 00:02 2024-09-10T09:41:41.637 INFO:teuthology.orchestra.run.smithi077.stdout:(5/156): ceph-mds-19.1.1-305.g1cf819b7.el9.x86_ 1.8 MB/s | 2.4 MB 00:01 2024-09-10T09:41:41.675 INFO:teuthology.orchestra.run.smithi134.stdout:(7/156): ceph-mds-19.1.1-305.g1cf819b7.el9.x86_ 1.7 MB/s | 2.4 MB 00:01 2024-09-10T09:41:41.879 INFO:teuthology.orchestra.run.smithi077.stdout:(6/156): ceph-mgr-19.1.1-305.g1cf819b7.el9.x86_ 4.6 MB/s | 1.5 MB 00:00 2024-09-10T09:41:42.276 INFO:teuthology.orchestra.run.smithi134.stdout:(8/156): ceph-mon-19.1.1-305.g1cf819b7.el9.x86_ 4.2 MB/s | 4.7 MB 00:01 2024-09-10T09:41:42.367 INFO:teuthology.orchestra.run.smithi134.stdout:(9/156): ceph-selinux-19.1.1-305.g1cf819b7.el9. 275 kB/s | 25 kB 00:00 2024-09-10T09:41:42.559 INFO:teuthology.orchestra.run.smithi134.stdout:(10/156): ceph-osd-19.1.1-305.g1cf819b7.el9.x86 18 MB/s | 17 MB 00:00 2024-09-10T09:41:42.626 INFO:teuthology.orchestra.run.smithi134.stdout:(11/156): cephfs-mirror-19.1.1-305.g1cf819b7.el 3.4 MB/s | 228 kB 00:00 2024-09-10T09:41:42.685 INFO:teuthology.orchestra.run.smithi134.stdout:(12/156): cephfs-shell-19.1.1-305.g1cf819b7.el9 462 kB/s | 27 kB 00:00 2024-09-10T09:41:42.752 INFO:teuthology.orchestra.run.smithi134.stdout:(13/156): libcephfs-devel-19.1.1-305.g1cf819b7. 477 kB/s | 32 kB 00:00 2024-09-10T09:41:42.844 INFO:teuthology.orchestra.run.smithi134.stdout:(14/156): libcephfs2-19.1.1-305.g1cf819b7.el9.x 8.0 MB/s | 751 kB 00:00 2024-09-10T09:41:42.919 INFO:teuthology.orchestra.run.smithi134.stdout:(15/156): libcephsqlite-19.1.1-305.g1cf819b7.el 2.1 MB/s | 164 kB 00:00 2024-09-10T09:41:42.994 INFO:teuthology.orchestra.run.smithi134.stdout:(16/156): librados-devel-19.1.1-305.g1cf819b7.e 1.7 MB/s | 127 kB 00:00 2024-09-10T09:41:43.045 INFO:teuthology.orchestra.run.smithi120.stdout:CentOS Stream 9 - CRB 2.5 MB/s | 6.5 MB 00:02 2024-09-10T09:41:43.078 INFO:teuthology.orchestra.run.smithi134.stdout:(17/156): libradosstriper1-19.1.1-305.g1cf819b7 5.9 MB/s | 506 kB 00:00 2024-09-10T09:41:43.305 INFO:teuthology.orchestra.run.smithi077.stdout:(7/156): ceph-osd-19.1.1-305.g1cf819b7.el9.x86_ 12 MB/s | 17 MB 00:01 2024-09-10T09:41:43.412 INFO:teuthology.orchestra.run.smithi134.stdout:(18/156): librgw2-19.1.1-305.g1cf819b7.el9.x86_ 16 MB/s | 5.3 MB 00:00 2024-09-10T09:41:43.478 INFO:teuthology.orchestra.run.smithi134.stdout:(19/156): python3-ceph-argparse-19.1.1-305.g1cf 679 kB/s | 45 kB 00:00 2024-09-10T09:41:43.545 INFO:teuthology.orchestra.run.smithi134.stdout:(20/156): python3-ceph-common-19.1.1-305.g1cf81 2.0 MB/s | 136 kB 00:00 2024-09-10T09:41:43.621 INFO:teuthology.orchestra.run.smithi134.stdout:(21/156): python3-cephfs-19.1.1-305.g1cf819b7.e 2.1 MB/s | 161 kB 00:00 2024-09-10T09:41:43.704 INFO:teuthology.orchestra.run.smithi134.stdout:(22/156): python3-rados-19.1.1-305.g1cf819b7.el 3.8 MB/s | 320 kB 00:00 2024-09-10T09:41:43.755 INFO:teuthology.orchestra.run.smithi134.stdout:(23/156): python3-rbd-19.1.1-305.g1cf819b7.el9. 5.9 MB/s | 301 kB 00:00 2024-09-10T09:41:43.805 INFO:teuthology.orchestra.run.smithi077.stdout:(8/156): ceph-radosgw-19.1.1-305.g1cf819b7.el9. 21 MB/s | 11 MB 00:00 2024-09-10T09:41:43.813 INFO:teuthology.orchestra.run.smithi134.stdout:(24/156): python3-rgw-19.1.1-305.g1cf819b7.el9. 1.7 MB/s | 99 kB 00:00 2024-09-10T09:41:43.863 INFO:teuthology.orchestra.run.smithi077.stdout:(9/156): ceph-selinux-19.1.1-305.g1cf819b7.el9. 434 kB/s | 25 kB 00:00 2024-09-10T09:41:43.880 INFO:teuthology.orchestra.run.smithi134.stdout:(25/156): rbd-fuse-19.1.1-305.g1cf819b7.el9.x86 1.3 MB/s | 85 kB 00:00 2024-09-10T09:41:44.230 INFO:teuthology.orchestra.run.smithi134.stdout:(26/156): rbd-mirror-19.1.1-305.g1cf819b7.el9.x 8.9 MB/s | 3.1 MB 00:00 2024-09-10T09:41:44.297 INFO:teuthology.orchestra.run.smithi134.stdout:(27/156): rbd-nbd-19.1.1-305.g1cf819b7.el9.x86_ 2.5 MB/s | 170 kB 00:00 2024-09-10T09:41:44.305 INFO:teuthology.orchestra.run.smithi077.stdout:(10/156): ceph-mon-19.1.1-305.g1cf819b7.el9.x86 1.8 MB/s | 4.7 MB 00:02 2024-09-10T09:41:44.347 INFO:teuthology.orchestra.run.smithi134.stdout:(28/156): ceph-grafana-dashboards-19.1.1-305.g1 638 kB/s | 32 kB 00:00 2024-09-10T09:41:44.414 INFO:teuthology.orchestra.run.smithi134.stdout:(29/156): ceph-mgr-cephadm-19.1.1-305.g1cf819b7 2.1 MB/s | 146 kB 00:00 2024-09-10T09:41:44.455 INFO:teuthology.orchestra.run.smithi077.stdout:(11/156): cephfs-mirror-19.1.1-305.g1cf819b7.el 1.5 MB/s | 228 kB 00:00 2024-09-10T09:41:44.539 INFO:teuthology.orchestra.run.smithi077.stdout:(12/156): cephfs-shell-19.1.1-305.g1cf819b7.el9 325 kB/s | 27 kB 00:00 2024-09-10T09:41:44.616 INFO:teuthology.orchestra.run.smithi077.stdout:(13/156): libcephfs-devel-19.1.1-305.g1cf819b7. 423 kB/s | 32 kB 00:00 2024-09-10T09:41:44.646 INFO:teuthology.orchestra.run.smithi120.stdout:CentOS Stream 9 - Extras packages 49 kB/s | 19 kB 00:00 2024-09-10T09:41:44.698 INFO:teuthology.orchestra.run.smithi134.stdout:(30/156): ceph-mgr-dashboard-19.1.1-305.g1cf819 13 MB/s | 3.6 MB 00:00 2024-09-10T09:41:45.056 INFO:teuthology.orchestra.run.smithi077.stdout:(14/156): libcephfs2-19.1.1-305.g1cf819b7.el9.x 1.7 MB/s | 751 kB 00:00 2024-09-10T09:41:45.181 INFO:teuthology.orchestra.run.smithi134.stdout:(31/156): ceph-mgr-diskprediction-local-19.1.1- 15 MB/s | 7.4 MB 00:00 2024-09-10T09:41:45.190 INFO:teuthology.orchestra.run.smithi077.stdout:(15/156): libcephsqlite-19.1.1-305.g1cf819b7.el 1.2 MB/s | 164 kB 00:00 2024-09-10T09:41:45.232 INFO:teuthology.orchestra.run.smithi134.stdout:(32/156): ceph-mgr-modules-core-19.1.1-305.g1cf 4.9 MB/s | 249 kB 00:00 2024-09-10T09:41:45.298 INFO:teuthology.orchestra.run.smithi077.stdout:(16/156): librados-devel-19.1.1-305.g1cf819b7.e 1.1 MB/s | 127 kB 00:00 2024-09-10T09:41:45.299 INFO:teuthology.orchestra.run.smithi134.stdout:(33/156): ceph-mgr-rook-19.1.1-305.g1cf819b7.el 739 kB/s | 49 kB 00:00 2024-09-10T09:41:45.365 INFO:teuthology.orchestra.run.smithi134.stdout:(34/156): ceph-prometheus-alerts-19.1.1-305.g1c 247 kB/s | 16 kB 00:00 2024-09-10T09:41:45.433 INFO:teuthology.orchestra.run.smithi134.stdout:(35/156): ceph-volume-19.1.1-305.g1cf819b7.el9. 4.2 MB/s | 285 kB 00:00 2024-09-10T09:41:45.492 INFO:teuthology.orchestra.run.smithi120.stdout:Extra Packages for Enterprise Linux 31 MB/s | 23 MB 00:00 2024-09-10T09:41:45.533 INFO:teuthology.orchestra.run.smithi134.stdout:(36/156): cephadm-19.1.1-305.g1cf819b7.el9.noar 7.5 MB/s | 766 kB 00:00 2024-09-10T09:41:45.549 INFO:teuthology.orchestra.run.smithi077.stdout:(17/156): libradosstriper1-19.1.1-305.g1cf819b7 2.0 MB/s | 506 kB 00:00 2024-09-10T09:41:45.599 INFO:teuthology.orchestra.run.smithi134.stdout:(37/156): cephfs-top-19.1.1-305.g1cf819b7.el9.n 274 kB/s | 18 kB 00:00 2024-09-10T09:41:45.858 INFO:teuthology.orchestra.run.smithi134.stdout:(38/156): ceph-radosgw-19.1.1-305.g1cf819b7.el9 2.5 MB/s | 11 MB 00:04 2024-09-10T09:41:46.107 INFO:teuthology.orchestra.run.smithi077.stdout:(18/156): ceph-test-19.1.1-305.g1cf819b7.el9.x8 22 MB/s | 49 MB 00:02 2024-09-10T09:41:46.158 INFO:teuthology.orchestra.run.smithi077.stdout:(19/156): python3-ceph-argparse-19.1.1-305.g1cf 912 kB/s | 45 kB 00:00 2024-09-10T09:41:46.208 INFO:teuthology.orchestra.run.smithi077.stdout:(20/156): python3-ceph-common-19.1.1-305.g1cf81 2.7 MB/s | 136 kB 00:00 2024-09-10T09:41:46.278 INFO:teuthology.orchestra.run.smithi077.stdout:(21/156): python3-cephfs-19.1.1-305.g1cf819b7.e 2.3 MB/s | 161 kB 00:00 2024-09-10T09:41:46.300 INFO:teuthology.orchestra.run.smithi134.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 114 kB/s | 80 kB 00:00 2024-09-10T09:41:46.333 INFO:teuthology.orchestra.run.smithi077.stdout:(22/156): python3-rados-19.1.1-305.g1cf819b7.el 5.7 MB/s | 320 kB 00:00 2024-09-10T09:41:46.384 INFO:teuthology.orchestra.run.smithi077.stdout:(23/156): python3-rbd-19.1.1-305.g1cf819b7.el9. 5.9 MB/s | 301 kB 00:00 2024-09-10T09:41:46.434 INFO:teuthology.orchestra.run.smithi077.stdout:(24/156): python3-rgw-19.1.1-305.g1cf819b7.el9. 1.9 MB/s | 99 kB 00:00 2024-09-10T09:41:46.484 INFO:teuthology.orchestra.run.smithi077.stdout:(25/156): rbd-fuse-19.1.1-305.g1cf819b7.el9.x86 1.7 MB/s | 85 kB 00:00 2024-09-10T09:41:46.626 INFO:teuthology.orchestra.run.smithi077.stdout:(26/156): rbd-mirror-19.1.1-305.g1cf819b7.el9.x 22 MB/s | 3.1 MB 00:00 2024-09-10T09:41:46.685 INFO:teuthology.orchestra.run.smithi077.stdout:(27/156): rbd-nbd-19.1.1-305.g1cf819b7.el9.x86_ 2.9 MB/s | 170 kB 00:00 2024-09-10T09:41:46.735 INFO:teuthology.orchestra.run.smithi077.stdout:(28/156): ceph-grafana-dashboards-19.1.1-305.g1 639 kB/s | 32 kB 00:00 2024-09-10T09:41:46.750 INFO:teuthology.orchestra.run.smithi134.stdout:(40/156): libgfortran-11.5.0-2.el9.x86_64.rpm 906 kB/s | 801 kB 00:00 2024-09-10T09:41:46.785 INFO:teuthology.orchestra.run.smithi077.stdout:(29/156): ceph-mgr-cephadm-19.1.1-305.g1cf819b7 2.9 MB/s | 146 kB 00:00 2024-09-10T09:41:46.936 INFO:teuthology.orchestra.run.smithi077.stdout:(30/156): ceph-mgr-dashboard-19.1.1-305.g1cf819 24 MB/s | 3.6 MB 00:00 2024-09-10T09:41:47.051 INFO:teuthology.orchestra.run.smithi134.stdout:(41/156): libquadmath-11.5.0-2.el9.x86_64.rpm 255 kB/s | 191 kB 00:00 2024-09-10T09:41:47.203 INFO:teuthology.orchestra.run.smithi077.stdout:(31/156): ceph-mgr-diskprediction-local-19.1.1- 28 MB/s | 7.4 MB 00:00 2024-09-10T09:41:47.278 INFO:teuthology.orchestra.run.smithi077.stdout:(32/156): librgw2-19.1.1-305.g1cf819b7.el9.x86_ 3.1 MB/s | 5.3 MB 00:01 2024-09-10T09:41:47.303 INFO:teuthology.orchestra.run.smithi077.stdout:(33/156): ceph-mgr-modules-core-19.1.1-305.g1cf 2.4 MB/s | 249 kB 00:00 2024-09-10T09:41:47.328 INFO:teuthology.orchestra.run.smithi077.stdout:(34/156): ceph-mgr-rook-19.1.1-305.g1cf819b7.el 986 kB/s | 49 kB 00:00 2024-09-10T09:41:47.354 INFO:teuthology.orchestra.run.smithi077.stdout:(35/156): ceph-prometheus-alerts-19.1.1-305.g1c 328 kB/s | 16 kB 00:00 2024-09-10T09:41:47.526 INFO:teuthology.orchestra.run.smithi134.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 327 kB/s | 253 kB 00:00 2024-09-10T09:41:47.571 INFO:teuthology.orchestra.run.smithi077.stdout:(36/156): ceph-common-19.1.1-305.g1cf819b7.el9. 2.4 MB/s | 21 MB 00:08 2024-09-10T09:41:47.596 INFO:teuthology.orchestra.run.smithi077.stdout:(37/156): ceph-volume-19.1.1-305.g1cf819b7.el9. 1.0 MB/s | 285 kB 00:00 2024-09-10T09:41:47.621 INFO:teuthology.orchestra.run.smithi077.stdout:(38/156): cephfs-top-19.1.1-305.g1cf819b7.el9.n 367 kB/s | 18 kB 00:00 2024-09-10T09:41:47.655 INFO:teuthology.orchestra.run.smithi077.stdout:(39/156): cephadm-19.1.1-305.g1cf819b7.el9.noar 2.5 MB/s | 766 kB 00:00 2024-09-10T09:41:47.805 INFO:teuthology.orchestra.run.smithi077.stdout:(40/156): fuse-2.9.9-16.el9.x86_64.rpm 383 kB/s | 80 kB 00:00 2024-09-10T09:41:47.944 INFO:teuthology.orchestra.run.smithi077.stdout:(41/156): libgfortran-11.5.0-2.el9.x86_64.rpm 2.4 MB/s | 801 kB 00:00 2024-09-10T09:41:47.972 INFO:teuthology.orchestra.run.smithi077.stdout:(42/156): libquadmath-11.5.0-2.el9.x86_64.rpm 603 kB/s | 191 kB 00:00 2024-09-10T09:41:47.998 INFO:teuthology.orchestra.run.smithi077.stdout:(43/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.3 MB/s | 253 kB 00:00 2024-09-10T09:41:48.031 INFO:teuthology.orchestra.run.smithi077.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-09-10T09:41:48.056 INFO:teuthology.orchestra.run.smithi077.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-09-10T09:41:48.060 INFO:teuthology.orchestra.run.smithi134.stdout:(43/156): ceph-test-19.1.1-305.g1cf819b7.el9.x8 8.6 MB/s | 49 MB 00:05 2024-09-10T09:41:48.090 INFO:teuthology.orchestra.run.smithi077.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-09-10T09:41:48.124 INFO:teuthology.orchestra.run.smithi077.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 3.1 MB/s | 215 kB 00:00 2024-09-10T09:41:48.266 INFO:teuthology.orchestra.run.smithi077.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 3.8 MB/s | 689 kB 00:00 2024-09-10T09:41:48.299 INFO:teuthology.orchestra.run.smithi077.stdout:(49/156): python3-cryptography-36.0.1-4.el9.x86 3.5 MB/s | 1.2 MB 00:00 2024-09-10T09:41:48.325 INFO:teuthology.orchestra.run.smithi077.stdout:(50/156): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-09-10T09:41:48.358 INFO:teuthology.orchestra.run.smithi077.stdout:(51/156): bison-3.7.4-5.el9.x86_64.rpm 3.9 MB/s | 943 kB 00:00 2024-09-10T09:41:48.376 INFO:teuthology.orchestra.run.smithi134.stdout:(44/156): python3-cryptography-36.0.1-4.el9.x86 947 kB/s | 1.2 MB 00:01 2024-09-10T09:41:48.392 INFO:teuthology.orchestra.run.smithi077.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 4.6 MB/s | 313 kB 00:00 2024-09-10T09:41:48.417 INFO:teuthology.orchestra.run.smithi077.stdout:(53/156): elfutils-libelf-devel-0.191-4.el9.x86 185 kB/s | 22 kB 00:00 2024-09-10T09:41:48.442 INFO:teuthology.orchestra.run.smithi077.stdout:(54/156): flexiblas-3.0.4-8.el9.x86_64.rpm 391 kB/s | 33 kB 00:00 2024-09-10T09:41:48.476 INFO:teuthology.orchestra.run.smithi077.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 314 kB/s | 18 kB 00:00 2024-09-10T09:41:48.501 INFO:teuthology.orchestra.run.smithi077.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 191 kB/s | 11 kB 00:00 2024-09-10T09:41:48.527 INFO:teuthology.orchestra.run.smithi077.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-09-10T09:41:48.569 INFO:teuthology.orchestra.run.smithi077.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 677 kB/s | 45 kB 00:00 2024-09-10T09:41:48.593 INFO:teuthology.orchestra.run.smithi134.stdout:(45/156): python3-ply-3.11-14.el9.noarch.rpm 100 kB/s | 106 kB 00:01 2024-09-10T09:41:48.611 INFO:teuthology.orchestra.run.smithi077.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 7.8 MB/s | 662 kB 00:00 2024-09-10T09:41:48.669 INFO:teuthology.orchestra.run.smithi077.stdout:(60/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 11 MB/s | 3.0 MB 00:00 2024-09-10T09:41:48.694 INFO:teuthology.orchestra.run.smithi077.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 294 kB/s | 24 kB 00:00 2024-09-10T09:41:48.720 INFO:teuthology.orchestra.run.smithi077.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 947 kB/s | 47 kB 00:00 2024-09-10T09:41:48.770 INFO:teuthology.orchestra.run.smithi077.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.8 MB/s | 292 kB 00:00 2024-09-10T09:41:48.785 INFO:teuthology.orchestra.run.smithi134.stdout:(46/156): python3-pycparser-2.20-6.el9.noarch.r 186 kB/s | 135 kB 00:00 2024-09-10T09:41:48.795 INFO:teuthology.orchestra.run.smithi077.stdout:(64/156): libtool-2.4.6-46.el9.x86_64.rpm 2.5 MB/s | 581 kB 00:00 2024-09-10T09:41:48.820 INFO:teuthology.orchestra.run.smithi077.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 767 kB/s | 38 kB 00:00 2024-09-10T09:41:48.846 INFO:teuthology.orchestra.run.smithi077.stdout:(66/156): lua-5.4.4-4.el9.x86_64.rpm 1.5 MB/s | 188 kB 00:00 2024-09-10T09:41:48.921 INFO:teuthology.orchestra.run.smithi077.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 21 MB/s | 1.6 MB 00:00 2024-09-10T09:41:48.996 INFO:teuthology.orchestra.run.smithi077.stdout:(68/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 25 MB/s | 4.4 MB 00:00 2024-09-10T09:41:49.010 INFO:teuthology.orchestra.run.smithi134.stdout:(47/156): automake-1.16.2-8.el9.noarch.rpm 3.0 MB/s | 689 kB 00:00 2024-09-10T09:41:49.061 INFO:teuthology.orchestra.run.smithi134.stdout:(48/156): python3-requests-2.25.1-8.el9.noarch. 183 kB/s | 125 kB 00:00 2024-09-10T09:41:49.071 INFO:teuthology.orchestra.run.smithi077.stdout:(69/156): openblas-openmp-0.3.26-2.el9.x86_64.r 18 MB/s | 4.9 MB 00:00 2024-09-10T09:41:49.097 INFO:teuthology.orchestra.run.smithi077.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 1.3 MB/s | 137 kB 00:00 2024-09-10T09:41:49.139 INFO:teuthology.orchestra.run.smithi077.stdout:(71/156): postgresql-contrib-13.16-1.el9.x86_64 4.1 MB/s | 910 kB 00:00 2024-09-10T09:41:49.181 INFO:teuthology.orchestra.run.smithi077.stdout:(72/156): protobuf-3.14.0-13.el9.x86_64.rpm 12 MB/s | 1.0 MB 00:00 2024-09-10T09:41:49.202 INFO:teuthology.orchestra.run.smithi134.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 757 kB/s | 107 kB 00:00 2024-09-10T09:41:49.206 INFO:teuthology.orchestra.run.smithi077.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-09-10T09:41:49.248 INFO:teuthology.orchestra.run.smithi077.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.4 MB/s | 61 kB 00:00 2024-09-10T09:41:49.253 INFO:teuthology.orchestra.run.smithi134.stdout:(50/156): elfutils-libelf-devel-0.191-4.el9.x86 434 kB/s | 22 kB 00:00 2024-09-10T09:41:49.286 INFO:teuthology.orchestra.run.smithi134.stdout:(51/156): bison-3.7.4-5.el9.x86_64.rpm 3.3 MB/s | 943 kB 00:00 2024-09-10T09:41:49.337 INFO:teuthology.orchestra.run.smithi134.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 3.7 MB/s | 313 kB 00:00 2024-09-10T09:41:49.340 INFO:teuthology.orchestra.run.smithi077.stdout:(75/156): postgresql-server-13.16-1.el9.x86_64. 22 MB/s | 5.8 MB 00:00 2024-09-10T09:41:49.374 INFO:teuthology.orchestra.run.smithi077.stdout:(76/156): python3-devel-3.9.19-8.el9.x86_64.rpm 1.9 MB/s | 245 kB 00:00 2024-09-10T09:41:49.387 INFO:teuthology.orchestra.run.smithi134.stdout:(53/156): python3-urllib3-1.26.5-6.el9.noarch.r 273 kB/s | 215 kB 00:00 2024-09-10T09:41:49.416 INFO:teuthology.orchestra.run.smithi077.stdout:(77/156): python3-jinja2-2.11.3-6.el9.noarch.rp 3.2 MB/s | 248 kB 00:00 2024-09-10T09:41:49.441 INFO:teuthology.orchestra.run.smithi077.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 384 kB/s | 26 kB 00:00 2024-09-10T09:41:49.462 INFO:teuthology.orchestra.run.smithi134.stdout:(54/156): flexiblas-3.0.4-8.el9.x86_64.rpm 186 kB/s | 33 kB 00:00 2024-09-10T09:41:49.466 INFO:teuthology.orchestra.run.smithi077.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 384 kB/s | 19 kB 00:00 2024-09-10T09:41:49.487 INFO:teuthology.orchestra.run.smithi134.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 182 kB/s | 18 kB 00:00 2024-09-10T09:41:49.491 INFO:teuthology.orchestra.run.smithi077.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-09-10T09:41:49.508 INFO:teuthology.orchestra.run.smithi077.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 830 kB/s | 35 kB 00:00 2024-09-10T09:41:49.521 INFO:teuthology.orchestra.run.smithi134.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 190 kB/s | 11 kB 00:00 2024-09-10T09:41:49.571 INFO:teuthology.orchestra.run.smithi134.stdout:(57/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 910 kB/s | 45 kB 00:00 2024-09-10T09:41:49.592 INFO:teuthology.orchestra.run.smithi077.stdout:(82/156): python3-babel-2.9.1-2.el9.noarch.rpm 15 MB/s | 6.0 MB 00:00 2024-09-10T09:41:49.617 INFO:teuthology.orchestra.run.smithi077.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.3 MB/s | 250 kB 00:00 2024-09-10T09:41:49.621 INFO:teuthology.orchestra.run.smithi134.stdout:(58/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.2 MB/s | 160 kB 00:00 2024-09-10T09:41:49.642 INFO:teuthology.orchestra.run.smithi077.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 4.4 MB/s | 222 kB 00:00 2024-09-10T09:41:49.668 INFO:teuthology.orchestra.run.smithi077.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-10T09:41:49.693 INFO:teuthology.orchestra.run.smithi077.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 842 kB/s | 42 kB 00:00 2024-09-10T09:41:49.726 INFO:teuthology.orchestra.run.smithi077.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-09-10T09:41:49.738 INFO:teuthology.orchestra.run.smithi134.stdout:(59/156): libtool-2.4.6-46.el9.x86_64.rpm 4.9 MB/s | 581 kB 00:00 2024-09-10T09:41:49.752 INFO:teuthology.orchestra.run.smithi077.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 159 kB 00:00 2024-09-10T09:41:49.772 INFO:teuthology.orchestra.run.smithi134.stdout:(60/156): librdkafka-1.6.1-102.el9.x86_64.rpm 3.2 MB/s | 662 kB 00:00 2024-09-10T09:41:49.777 INFO:teuthology.orchestra.run.smithi077.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-09-10T09:41:49.797 INFO:teuthology.orchestra.run.smithi134.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 419 kB/s | 24 kB 00:00 2024-09-10T09:41:49.803 INFO:teuthology.orchestra.run.smithi077.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-10T09:41:49.827 INFO:teuthology.orchestra.run.smithi077.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-10T09:41:49.847 INFO:teuthology.orchestra.run.smithi134.stdout:(62/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 5.9 MB/s | 3.0 MB 00:00 2024-09-10T09:41:49.870 INFO:teuthology.orchestra.run.smithi077.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 1.0 MB/s | 42 kB 00:00 2024-09-10T09:41:49.872 INFO:teuthology.orchestra.run.smithi134.stdout:(63/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 473 kB/s | 47 kB 00:00 2024-09-10T09:41:49.898 INFO:teuthology.orchestra.run.smithi134.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.8 MB/s | 292 kB 00:00 2024-09-10T09:41:49.930 INFO:teuthology.orchestra.run.smithi134.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 670 kB/s | 38 kB 00:00 2024-09-10T09:41:49.945 INFO:teuthology.orchestra.run.smithi077.stdout:(93/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 11 MB/s | 5.1 MB 00:00 2024-09-10T09:41:49.955 INFO:teuthology.orchestra.run.smithi134.stdout:(66/156): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 188 kB 00:00 2024-09-10T09:41:49.970 INFO:teuthology.orchestra.run.smithi077.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 442 kB/s | 44 kB 00:00 2024-09-10T09:41:49.995 INFO:teuthology.orchestra.run.smithi077.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-09-10T09:41:50.021 INFO:teuthology.orchestra.run.smithi077.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 716 kB/s | 36 kB 00:00 2024-09-10T09:41:50.046 INFO:teuthology.orchestra.run.smithi077.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 944 kB/s | 47 kB 00:00 2024-09-10T09:41:50.071 INFO:teuthology.orchestra.run.smithi077.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 5.9 MB/s | 304 kB 00:00 2024-09-10T09:41:50.097 INFO:teuthology.orchestra.run.smithi077.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-10T09:41:50.130 INFO:teuthology.orchestra.run.smithi077.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.6 MB/s | 96 kB 00:00 2024-09-10T09:41:50.155 INFO:teuthology.orchestra.run.smithi077.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-09-10T09:41:50.181 INFO:teuthology.orchestra.run.smithi077.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 894 kB/s | 45 kB 00:00 2024-09-10T09:41:50.398 INFO:teuthology.orchestra.run.smithi134.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 3.6 MB/s | 1.6 MB 00:00 2024-09-10T09:41:50.414 INFO:teuthology.orchestra.run.smithi077.stdout:(103/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 27 MB/s | 16 MB 00:00 2024-09-10T09:41:50.473 INFO:teuthology.orchestra.run.smithi077.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 9.2 MB/s | 548 kB 00:00 2024-09-10T09:41:50.489 INFO:teuthology.orchestra.run.smithi134.stdout:(68/156): openblas-openmp-0.3.26-2.el9.x86_64.r 8.4 MB/s | 4.9 MB 00:00 2024-09-10T09:41:50.507 INFO:teuthology.orchestra.run.smithi077.stdout:(105/156): gperftools-libs-2.9.1-2.el9.x86_64.r 9.1 MB/s | 309 kB 00:00 2024-09-10T09:41:50.540 INFO:teuthology.orchestra.run.smithi134.stdout:(69/156): postgresql-private-libs-13.16-1.el9.x 2.7 MB/s | 137 kB 00:00 2024-09-10T09:41:50.540 INFO:teuthology.orchestra.run.smithi077.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 826 kB/s | 19 kB 00:00 2024-09-10T09:41:50.674 INFO:teuthology.orchestra.run.smithi077.stdout:(107/156): libarrow-9.0.0-11.el9.x86_64.rpm 33 MB/s | 4.4 MB 00:00 2024-09-10T09:41:50.699 INFO:teuthology.orchestra.run.smithi077.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.1 MB/s | 26 kB 00:00 2024-09-10T09:41:50.706 INFO:teuthology.orchestra.run.smithi134.stdout:(70/156): postgresql-contrib-13.16-1.el9.x86_64 2.9 MB/s | 910 kB 00:00 2024-09-10T09:41:50.725 INFO:teuthology.orchestra.run.smithi077.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-10T09:41:50.750 INFO:teuthology.orchestra.run.smithi077.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-10T09:41:50.775 INFO:teuthology.orchestra.run.smithi077.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 6.0 MB/s | 151 kB 00:00 2024-09-10T09:41:50.782 INFO:teuthology.orchestra.run.smithi134.stdout:(71/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 5.2 MB/s | 4.4 MB 00:00 2024-09-10T09:41:50.825 INFO:teuthology.orchestra.run.smithi077.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 16 MB/s | 839 kB 00:00 2024-09-10T09:41:50.832 INFO:teuthology.orchestra.run.smithi134.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.7 MB/s | 87 kB 00:00 2024-09-10T09:41:50.859 INFO:teuthology.orchestra.run.smithi077.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 16 MB/s | 548 kB 00:00 2024-09-10T09:41:50.884 INFO:teuthology.orchestra.run.smithi077.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-10T09:41:50.906 INFO:teuthology.orchestra.run.smithi120.stdout:lab-extras 36 kB/s | 1.7 kB 00:00 2024-09-10T09:41:50.910 INFO:teuthology.orchestra.run.smithi077.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-10T09:41:50.935 INFO:teuthology.orchestra.run.smithi077.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 571 kB/s | 14 kB 00:00 2024-09-10T09:41:50.960 INFO:teuthology.orchestra.run.smithi077.stdout:(117/156): lua-devel-5.4.4-4.el9.x86_64.rpm 28 kB/s | 22 kB 00:00 2024-09-10T09:41:50.994 INFO:teuthology.orchestra.run.smithi077.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-09-10T09:41:51.007 INFO:teuthology.orchestra.run.smithi134.stdout:(73/156): protobuf-3.14.0-13.el9.x86_64.rpm 3.4 MB/s | 1.0 MB 00:00 2024-09-10T09:41:51.027 INFO:teuthology.orchestra.run.smithi077.stdout:(119/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 9.4 MB/s | 303 kB 00:00 2024-09-10T09:41:51.057 INFO:teuthology.orchestra.run.smithi134.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.2 MB/s | 61 kB 00:00 2024-09-10T09:41:51.106 INFO:teuthology.orchestra.run.smithi077.stdout:(120/156): python3-cherrypy-18.6.1-2.el9.noarch 2.4 MB/s | 358 kB 00:00 2024-09-10T09:41:51.136 INFO:teuthology.orchestra.run.smithi077.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 2.0 MB/s | 220 kB 00:00 2024-09-10T09:41:51.174 INFO:teuthology.orchestra.run.smithi134.stdout:(75/156): python3-devel-3.9.19-8.el9.x86_64.rpm 2.0 MB/s | 245 kB 00:00 2024-09-10T09:41:51.178 INFO:teuthology.orchestra.run.smithi077.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 3.4 MB/s | 144 kB 00:00 2024-09-10T09:41:51.195 INFO:teuthology.orchestra.run.smithi077.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 656 kB/s | 11 kB 00:00 2024-09-10T09:41:51.212 INFO:teuthology.orchestra.run.smithi077.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 1.0 MB/s | 18 kB 00:00 2024-09-10T09:41:51.237 INFO:teuthology.orchestra.run.smithi077.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 939 kB/s | 23 kB 00:00 2024-09-10T09:41:51.254 INFO:teuthology.orchestra.run.smithi077.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 1.1 MB/s | 19 kB 00:00 2024-09-10T09:41:51.291 INFO:teuthology.orchestra.run.smithi134.stdout:(76/156): python3-jinja2-2.11.3-6.el9.noarch.rp 2.1 MB/s | 248 kB 00:00 2024-09-10T09:41:51.296 INFO:teuthology.orchestra.run.smithi077.stdout:(127/156): protobuf-compiler-3.14.0-13.el9.x86_ 774 kB/s | 863 kB 00:01 2024-09-10T09:41:51.313 INFO:teuthology.orchestra.run.smithi077.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 336 kB/s | 20 kB 00:00 2024-09-10T09:41:51.338 INFO:teuthology.orchestra.run.smithi077.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 3.0 MB/s | 46 kB 00:00 2024-09-10T09:41:51.341 INFO:teuthology.orchestra.run.smithi134.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 513 kB/s | 26 kB 00:00 2024-09-10T09:41:51.380 INFO:teuthology.orchestra.run.smithi077.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 1.9 MB/s | 79 kB 00:00 2024-09-10T09:41:51.392 INFO:teuthology.orchestra.run.smithi134.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 382 kB/s | 19 kB 00:00 2024-09-10T09:41:51.397 INFO:teuthology.orchestra.run.smithi077.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 3.4 MB/s | 58 kB 00:00 2024-09-10T09:41:51.456 INFO:teuthology.orchestra.run.smithi077.stdout:(132/156): python3-grpcio-1.46.7-10.el9.x86_64. 5.8 MB/s | 2.0 MB 00:00 2024-09-10T09:41:51.459 INFO:teuthology.orchestra.run.smithi134.stdout:(79/156): postgresql-server-13.16-1.el9.x86_64. 6.3 MB/s | 5.8 MB 00:00 2024-09-10T09:41:51.481 INFO:teuthology.orchestra.run.smithi077.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 656 kB/s | 16 kB 00:00 2024-09-10T09:41:51.507 INFO:teuthology.orchestra.run.smithi077.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.4 MB/s | 272 kB 00:00 2024-09-10T09:41:51.509 INFO:teuthology.orchestra.run.smithi134.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 1.4 MB/s | 172 kB 00:00 2024-09-10T09:41:51.526 INFO:teuthology.orchestra.run.smithi134.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 521 kB/s | 35 kB 00:00 2024-09-10T09:41:51.532 INFO:teuthology.orchestra.run.smithi077.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-09-10T09:41:51.557 INFO:teuthology.orchestra.run.smithi077.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-09-10T09:41:51.590 INFO:teuthology.orchestra.run.smithi077.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 533 kB/s | 31 kB 00:00 2024-09-10T09:41:51.609 INFO:teuthology.orchestra.run.smithi134.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.9 MB/s | 250 kB 00:00 2024-09-10T09:41:51.615 INFO:teuthology.orchestra.run.smithi077.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-10T09:41:51.641 INFO:teuthology.orchestra.run.smithi077.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 3.7 MB/s | 188 kB 00:00 2024-09-10T09:41:51.666 INFO:teuthology.orchestra.run.smithi077.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 712 kB/s | 36 kB 00:00 2024-09-10T09:41:51.668 INFO:teuthology.orchestra.run.smithi134.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-09-10T09:41:51.692 INFO:teuthology.orchestra.run.smithi077.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-09-10T09:41:51.717 INFO:teuthology.orchestra.run.smithi077.stdout:(142/156): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-10T09:41:51.750 INFO:teuthology.orchestra.run.smithi077.stdout:(143/156): python3-kubernetes-26.1.0-2.el9.noar 2.2 MB/s | 1.0 MB 00:00 2024-09-10T09:41:51.769 INFO:teuthology.orchestra.run.smithi134.stdout:(84/156): python3-babel-2.9.1-2.el9.noarch.rpm 6.4 MB/s | 6.0 MB 00:00 2024-09-10T09:41:51.776 INFO:teuthology.orchestra.run.smithi077.stdout:(144/156): python3-webob-1.8.8-2.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-09-10T09:41:51.794 INFO:teuthology.orchestra.run.smithi134.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 615 kB/s | 77 kB 00:00 2024-09-10T09:41:51.801 INFO:teuthology.orchestra.run.smithi077.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 398 kB/s | 20 kB 00:00 2024-09-10T09:41:51.810 INFO:teuthology.orchestra.run.smithi134.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 42 kB 00:00 2024-09-10T09:41:51.835 INFO:teuthology.orchestra.run.smithi077.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.2 MB/s | 191 kB 00:00 2024-09-10T09:41:51.861 INFO:teuthology.orchestra.run.smithi134.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-09-10T09:41:51.877 INFO:teuthology.orchestra.run.smithi077.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 2.6 MB/s | 427 kB 00:00 2024-09-10T09:41:51.886 INFO:teuthology.orchestra.run.smithi134.stdout:(88/156): python3-protobuf-3.14.0-13.el9.noarch 2.9 MB/s | 269 kB 00:00 2024-09-10T09:41:51.935 INFO:teuthology.orchestra.run.smithi077.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-09-10T09:41:51.945 INFO:teuthology.orchestra.run.smithi134.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 3.3 MB/s | 279 kB 00:00 2024-09-10T09:41:51.987 INFO:teuthology.orchestra.run.smithi134.stdout:(90/156): python3-requests-oauthlib-1.3.0-12.el 1.3 MB/s | 54 kB 00:00 2024-09-10T09:41:52.037 INFO:teuthology.orchestra.run.smithi134.stdout:(91/156): python3-pytz-2021.1-5.el9.noarch.rpm 339 kB/s | 51 kB 00:00 2024-09-10T09:41:52.079 INFO:teuthology.orchestra.run.smithi134.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 1.0 MB/s | 42 kB 00:00 2024-09-10T09:41:52.121 INFO:teuthology.orchestra.run.smithi134.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.0 MB/s | 44 kB 00:00 2024-09-10T09:41:52.169 INFO:teuthology.orchestra.run.smithi077.stdout:(149/156): librbd1-19.1.1-305.g1cf819b7.el9.x86 11 MB/s | 3.2 MB 00:00 2024-09-10T09:41:52.204 INFO:teuthology.orchestra.run.smithi134.stdout:(94/156): qatlib-24.02.0-1.el9.x86_64.rpm 2.6 MB/s | 221 kB 00:00 2024-09-10T09:41:52.363 INFO:teuthology.orchestra.run.smithi134.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 227 kB/s | 36 kB 00:00 2024-09-10T09:41:52.438 INFO:teuthology.orchestra.run.smithi134.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 632 kB/s | 47 kB 00:00 2024-09-10T09:41:52.478 INFO:teuthology.orchestra.run.smithi077.stdout:(150/156): NetworkManager-libnm-1.49.90-1.el9.x 5.9 MB/s | 1.8 MB 00:00 2024-09-10T09:41:52.528 INFO:teuthology.orchestra.run.smithi077.stdout:(151/156): NetworkManager-team-1.49.90-1.el9.x8 816 kB/s | 41 kB 00:00 2024-09-10T09:41:52.586 INFO:teuthology.orchestra.run.smithi077.stdout:(152/156): librados2-19.1.1-305.g1cf819b7.el9.x 4.6 MB/s | 3.4 MB 00:00 2024-09-10T09:41:52.589 INFO:teuthology.orchestra.run.smithi134.stdout:(97/156): socat-1.7.4.1-6.el9.x86_64.rpm 2.0 MB/s | 304 kB 00:00 2024-09-10T09:41:52.631 INFO:teuthology.orchestra.run.smithi134.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 1.4 MB/s | 58 kB 00:00 2024-09-10T09:41:52.679 INFO:teuthology.orchestra.run.smithi077.stdout:(153/156): NetworkManager-tui-1.49.90-1.el9.x86 1.6 MB/s | 249 kB 00:00 2024-09-10T09:41:52.681 INFO:teuthology.orchestra.run.smithi134.stdout:(99/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rpm 1.9 MB/s | 96 kB 00:00 2024-09-10T09:41:52.770 INFO:teuthology.orchestra.run.smithi077.stdout:(154/156): NetworkManager-1.49.90-1.el9.x86_64. 2.8 MB/s | 2.3 MB 00:00 2024-09-10T09:41:52.840 INFO:teuthology.orchestra.run.smithi134.stdout:(100/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 402 kB/s | 64 kB 00:00 2024-09-10T09:41:52.954 INFO:teuthology.orchestra.run.smithi077.stdout:(155/156): openssl-3.2.2-6.el9.x86_64.rpm 3.7 MB/s | 1.4 MB 00:00 2024-09-10T09:41:52.998 INFO:teuthology.orchestra.run.smithi134.stdout:(101/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 283 kB/s | 45 kB 00:00 2024-09-10T09:41:53.132 INFO:teuthology.orchestra.run.smithi134.stdout:(102/156): python3-numpy-1.20.1-5.el9.x86_64.rp 3.1 MB/s | 5.1 MB 00:01 2024-09-10T09:41:53.148 INFO:teuthology.orchestra.run.smithi134.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 149 kB/s | 22 kB 00:00 2024-09-10T09:41:53.192 INFO:teuthology.orchestra.run.smithi120.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T09:41:53.192 INFO:teuthology.orchestra.run.smithi120.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T09:41:53.199 INFO:teuthology.orchestra.run.smithi134.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-09-10T09:41:53.203 INFO:teuthology.orchestra.run.smithi120.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-10T09:41:53.204 INFO:teuthology.orchestra.run.smithi120.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-10T09:41:53.205 INFO:teuthology.orchestra.run.smithi120.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-10T09:41:53.206 INFO:teuthology.orchestra.run.smithi120.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-10T09:41:53.208 INFO:teuthology.orchestra.run.smithi120.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-10T09:41:53.210 INFO:teuthology.orchestra.run.smithi120.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-10T09:41:53.211 INFO:teuthology.orchestra.run.smithi120.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-10T09:41:53.233 INFO:teuthology.orchestra.run.smithi134.stdout:(105/156): gperftools-libs-2.9.1-2.el9.x86_64.r 9.1 MB/s | 309 kB 00:00 2024-09-10T09:41:53.257 INFO:teuthology.orchestra.run.smithi134.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 787 kB/s | 19 kB 00:00 2024-09-10T09:41:53.273 INFO:teuthology.orchestra.run.smithi120.stdout:Dependencies resolved. 2024-09-10T09:41:53.296 INFO:teuthology.orchestra.run.smithi120.stdout:====================================================================================== 2024-09-10T09:41:53.296 INFO:teuthology.orchestra.run.smithi120.stdout: Package Arch Version Repository Size 2024-09-10T09:41:53.297 INFO:teuthology.orchestra.run.smithi120.stdout:====================================================================================== 2024-09-10T09:41:53.297 INFO:teuthology.orchestra.run.smithi120.stdout:Installing: 2024-09-10T09:41:53.297 INFO:teuthology.orchestra.run.smithi120.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-10T09:41:53.297 INFO:teuthology.orchestra.run.smithi120.stdout: ceph x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 6.5 k 2024-09-10T09:41:53.297 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-base x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 5.4 M 2024-09-10T09:41:53.297 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-fuse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 897 k 2024-09-10T09:41:53.297 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 145 k 2024-09-10T09:41:53.297 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 1.5 M 2024-09-10T09:41:53.297 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-cephadm noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 146 k 2024-09-10T09:41:53.297 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-dashboard noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 3.6 M 2024-09-10T09:41:53.297 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 7.4 M 2024-09-10T09:41:53.297 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-rook noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 49 k 2024-09-10T09:41:53.298 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-radosgw x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 11 M 2024-09-10T09:41:53.298 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-test x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 49 M 2024-09-10T09:41:53.298 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-volume noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 285 k 2024-09-10T09:41:53.298 INFO:teuthology.orchestra.run.smithi120.stdout: cephadm noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 766 k 2024-09-10T09:41:53.298 INFO:teuthology.orchestra.run.smithi120.stdout: cephfs-mirror x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 228 k 2024-09-10T09:41:53.298 INFO:teuthology.orchestra.run.smithi120.stdout: cephfs-shell x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 27 k 2024-09-10T09:41:53.298 INFO:teuthology.orchestra.run.smithi120.stdout: cephfs-top noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 18 k 2024-09-10T09:41:53.298 INFO:teuthology.orchestra.run.smithi120.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-10T09:41:53.298 INFO:teuthology.orchestra.run.smithi120.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-10T09:41:53.298 INFO:teuthology.orchestra.run.smithi120.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-10T09:41:53.298 INFO:teuthology.orchestra.run.smithi120.stdout: libcephfs-devel x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 32 k 2024-09-10T09:41:53.298 INFO:teuthology.orchestra.run.smithi120.stdout: libcephfs2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 751 k 2024-09-10T09:41:53.298 INFO:teuthology.orchestra.run.smithi120.stdout: librados-devel x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 127 k 2024-09-10T09:41:53.299 INFO:teuthology.orchestra.run.smithi120.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-10T09:41:53.299 INFO:teuthology.orchestra.run.smithi120.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-10T09:41:53.299 INFO:teuthology.orchestra.run.smithi120.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-10T09:41:53.299 INFO:teuthology.orchestra.run.smithi120.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-10T09:41:53.299 INFO:teuthology.orchestra.run.smithi120.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-10T09:41:53.299 INFO:teuthology.orchestra.run.smithi120.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-10T09:41:53.299 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cephfs x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 161 k 2024-09-10T09:41:53.299 INFO:teuthology.orchestra.run.smithi120.stdout: python3-rados x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 320 k 2024-09-10T09:41:53.299 INFO:teuthology.orchestra.run.smithi120.stdout: python3-rbd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 301 k 2024-09-10T09:41:53.299 INFO:teuthology.orchestra.run.smithi120.stdout: python3-rgw x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 99 k 2024-09-10T09:41:53.299 INFO:teuthology.orchestra.run.smithi120.stdout: rbd-fuse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 85 k 2024-09-10T09:41:53.299 INFO:teuthology.orchestra.run.smithi120.stdout: rbd-mirror x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.1 M 2024-09-10T09:41:53.300 INFO:teuthology.orchestra.run.smithi120.stdout: rbd-nbd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 170 k 2024-09-10T09:41:53.300 INFO:teuthology.orchestra.run.smithi120.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-10T09:41:53.300 INFO:teuthology.orchestra.run.smithi120.stdout:Upgrading: 2024-09-10T09:41:53.300 INFO:teuthology.orchestra.run.smithi120.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-10T09:41:53.300 INFO:teuthology.orchestra.run.smithi120.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-10T09:41:53.300 INFO:teuthology.orchestra.run.smithi120.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-10T09:41:53.300 INFO:teuthology.orchestra.run.smithi120.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-10T09:41:53.300 INFO:teuthology.orchestra.run.smithi120.stdout: librados2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.4 M 2024-09-10T09:41:53.300 INFO:teuthology.orchestra.run.smithi120.stdout: librbd1 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.2 M 2024-09-10T09:41:53.300 INFO:teuthology.orchestra.run.smithi120.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-10T09:41:53.301 INFO:teuthology.orchestra.run.smithi120.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-10T09:41:53.301 INFO:teuthology.orchestra.run.smithi120.stdout:Installing dependencies: 2024-09-10T09:41:53.301 INFO:teuthology.orchestra.run.smithi120.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-10T09:41:53.301 INFO:teuthology.orchestra.run.smithi120.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-10T09:41:53.301 INFO:teuthology.orchestra.run.smithi120.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-10T09:41:53.301 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-common x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 21 M 2024-09-10T09:41:53.301 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-grafana-dashboards noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 32 k 2024-09-10T09:41:53.301 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mds x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 2.4 M 2024-09-10T09:41:53.301 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-modules-core noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 249 k 2024-09-10T09:41:53.301 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mon x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 4.7 M 2024-09-10T09:41:53.301 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-osd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 17 M 2024-09-10T09:41:53.301 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-prometheus-alerts noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 16 k 2024-09-10T09:41:53.301 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-selinux x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 25 k 2024-09-10T09:41:53.301 INFO:teuthology.orchestra.run.smithi120.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-10T09:41:53.302 INFO:teuthology.orchestra.run.smithi120.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-10T09:41:53.302 INFO:teuthology.orchestra.run.smithi120.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-10T09:41:53.302 INFO:teuthology.orchestra.run.smithi120.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-10T09:41:53.302 INFO:teuthology.orchestra.run.smithi120.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-10T09:41:53.302 INFO:teuthology.orchestra.run.smithi120.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-10T09:41:53.302 INFO:teuthology.orchestra.run.smithi120.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-10T09:41:53.302 INFO:teuthology.orchestra.run.smithi120.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-10T09:41:53.302 INFO:teuthology.orchestra.run.smithi120.stdout: libcephsqlite x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 164 k 2024-09-10T09:41:53.302 INFO:teuthology.orchestra.run.smithi120.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-10T09:41:53.302 INFO:teuthology.orchestra.run.smithi120.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-10T09:41:53.302 INFO:teuthology.orchestra.run.smithi120.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-10T09:41:53.302 INFO:teuthology.orchestra.run.smithi120.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-10T09:41:53.302 INFO:teuthology.orchestra.run.smithi120.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-10T09:41:53.303 INFO:teuthology.orchestra.run.smithi120.stdout: libradosstriper1 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 506 k 2024-09-10T09:41:53.303 INFO:teuthology.orchestra.run.smithi120.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-10T09:41:53.303 INFO:teuthology.orchestra.run.smithi120.stdout: librgw2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 5.3 M 2024-09-10T09:41:53.303 INFO:teuthology.orchestra.run.smithi120.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-10T09:41:53.303 INFO:teuthology.orchestra.run.smithi120.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-10T09:41:53.303 INFO:teuthology.orchestra.run.smithi120.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-10T09:41:53.303 INFO:teuthology.orchestra.run.smithi120.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-10T09:41:53.303 INFO:teuthology.orchestra.run.smithi120.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-10T09:41:53.303 INFO:teuthology.orchestra.run.smithi120.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-10T09:41:53.303 INFO:teuthology.orchestra.run.smithi120.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-10T09:41:53.303 INFO:teuthology.orchestra.run.smithi120.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-10T09:41:53.303 INFO:teuthology.orchestra.run.smithi120.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-10T09:41:53.303 INFO:teuthology.orchestra.run.smithi120.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-10T09:41:53.303 INFO:teuthology.orchestra.run.smithi120.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-10T09:41:53.304 INFO:teuthology.orchestra.run.smithi120.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-10T09:41:53.304 INFO:teuthology.orchestra.run.smithi120.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-10T09:41:53.304 INFO:teuthology.orchestra.run.smithi120.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-10T09:41:53.304 INFO:teuthology.orchestra.run.smithi120.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-10T09:41:53.304 INFO:teuthology.orchestra.run.smithi120.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-10T09:41:53.304 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-10T09:41:53.304 INFO:teuthology.orchestra.run.smithi120.stdout: python3-ceph-argparse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 45 k 2024-09-10T09:41:53.304 INFO:teuthology.orchestra.run.smithi120.stdout: python3-ceph-common x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 136 k 2024-09-10T09:41:53.304 INFO:teuthology.orchestra.run.smithi120.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-10T09:41:53.304 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-10T09:41:53.304 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-10T09:41:53.305 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-10T09:41:53.305 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-10T09:41:53.305 INFO:teuthology.orchestra.run.smithi120.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-10T09:41:53.305 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-10T09:41:53.305 INFO:teuthology.orchestra.run.smithi120.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-10T09:41:53.305 INFO:teuthology.orchestra.run.smithi120.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-10T09:41:53.305 INFO:teuthology.orchestra.run.smithi120.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-10T09:41:53.305 INFO:teuthology.orchestra.run.smithi120.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-10T09:41:53.305 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-10T09:41:53.305 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-10T09:41:53.305 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-10T09:41:53.305 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-10T09:41:53.305 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-10T09:41:53.306 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-10T09:41:53.306 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-10T09:41:53.306 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-10T09:41:53.306 INFO:teuthology.orchestra.run.smithi120.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-10T09:41:53.306 INFO:teuthology.orchestra.run.smithi120.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-10T09:41:53.306 INFO:teuthology.orchestra.run.smithi120.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-10T09:41:53.306 INFO:teuthology.orchestra.run.smithi120.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-10T09:41:53.306 INFO:teuthology.orchestra.run.smithi120.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-10T09:41:53.306 INFO:teuthology.orchestra.run.smithi120.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-10T09:41:53.306 INFO:teuthology.orchestra.run.smithi120.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-10T09:41:53.306 INFO:teuthology.orchestra.run.smithi120.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-10T09:41:53.306 INFO:teuthology.orchestra.run.smithi120.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-10T09:41:53.307 INFO:teuthology.orchestra.run.smithi120.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-10T09:41:53.307 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-10T09:41:53.307 INFO:teuthology.orchestra.run.smithi120.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-10T09:41:53.307 INFO:teuthology.orchestra.run.smithi120.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-10T09:41:53.307 INFO:teuthology.orchestra.run.smithi120.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-10T09:41:53.307 INFO:teuthology.orchestra.run.smithi120.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-10T09:41:53.307 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-10T09:41:53.307 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-10T09:41:53.307 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-10T09:41:53.307 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-10T09:41:53.307 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-10T09:41:53.307 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-10T09:41:53.307 INFO:teuthology.orchestra.run.smithi120.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-10T09:41:53.307 INFO:teuthology.orchestra.run.smithi120.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-10T09:41:53.308 INFO:teuthology.orchestra.run.smithi120.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-10T09:41:53.308 INFO:teuthology.orchestra.run.smithi120.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-10T09:41:53.308 INFO:teuthology.orchestra.run.smithi120.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-10T09:41:53.308 INFO:teuthology.orchestra.run.smithi120.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-10T09:41:53.308 INFO:teuthology.orchestra.run.smithi120.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-10T09:41:53.308 INFO:teuthology.orchestra.run.smithi120.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-10T09:41:53.308 INFO:teuthology.orchestra.run.smithi120.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-10T09:41:53.308 INFO:teuthology.orchestra.run.smithi120.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-10T09:41:53.308 INFO:teuthology.orchestra.run.smithi120.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-10T09:41:53.309 INFO:teuthology.orchestra.run.smithi120.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-10T09:41:53.309 INFO:teuthology.orchestra.run.smithi120.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-10T09:41:53.309 INFO:teuthology.orchestra.run.smithi120.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-10T09:41:53.309 INFO:teuthology.orchestra.run.smithi120.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-10T09:41:53.309 INFO:teuthology.orchestra.run.smithi120.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-10T09:41:53.309 INFO:teuthology.orchestra.run.smithi120.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-10T09:41:53.309 INFO:teuthology.orchestra.run.smithi120.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-10T09:41:53.309 INFO:teuthology.orchestra.run.smithi120.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-10T09:41:53.309 INFO:teuthology.orchestra.run.smithi120.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-10T09:41:53.309 INFO:teuthology.orchestra.run.smithi120.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-10T09:41:53.309 INFO:teuthology.orchestra.run.smithi120.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-10T09:41:53.309 INFO:teuthology.orchestra.run.smithi120.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-10T09:41:53.309 INFO:teuthology.orchestra.run.smithi120.stdout:Installing weak dependencies: 2024-09-10T09:41:53.310 INFO:teuthology.orchestra.run.smithi120.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-10T09:41:53.310 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T09:41:53.310 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction Summary 2024-09-10T09:41:53.310 INFO:teuthology.orchestra.run.smithi120.stdout:====================================================================================== 2024-09-10T09:41:53.310 INFO:teuthology.orchestra.run.smithi120.stdout:Install 148 Packages 2024-09-10T09:41:53.310 INFO:teuthology.orchestra.run.smithi120.stdout:Upgrade 8 Packages 2024-09-10T09:41:53.310 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T09:41:53.310 INFO:teuthology.orchestra.run.smithi120.stdout:Total download size: 226 M 2024-09-10T09:41:53.310 INFO:teuthology.orchestra.run.smithi120.stdout:Downloading Packages: 2024-09-10T09:41:53.366 INFO:teuthology.orchestra.run.smithi134.stdout:(107/156): libarrow-9.0.0-11.el9.x86_64.rpm 41 MB/s | 4.4 MB 00:00 2024-09-10T09:41:53.391 INFO:teuthology.orchestra.run.smithi134.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-10T09:41:53.417 INFO:teuthology.orchestra.run.smithi134.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-10T09:41:53.442 INFO:teuthology.orchestra.run.smithi134.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-10T09:41:53.467 INFO:teuthology.orchestra.run.smithi134.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 6.0 MB/s | 151 kB 00:00 2024-09-10T09:41:53.509 INFO:teuthology.orchestra.run.smithi134.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 20 MB/s | 839 kB 00:00 2024-09-10T09:41:53.535 INFO:teuthology.orchestra.run.smithi134.stdout:(113/156): protobuf-compiler-3.14.0-13.el9.x86_ 2.1 MB/s | 863 kB 00:00 2024-09-10T09:41:53.576 INFO:teuthology.orchestra.run.smithi134.stdout:(114/156): python3-asyncssh-2.13.2-5.el9.noarch 8.0 MB/s | 548 kB 00:00 2024-09-10T09:41:53.602 INFO:teuthology.orchestra.run.smithi134.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.4 MB/s | 32 kB 00:00 2024-09-10T09:41:53.627 INFO:teuthology.orchestra.run.smithi134.stdout:(116/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 471 kB/s | 43 kB 00:00 2024-09-10T09:41:53.652 INFO:teuthology.orchestra.run.smithi134.stdout:(117/156): python3-certifi-2023.05.07-4.el9.noa 281 kB/s | 14 kB 00:00 2024-09-10T09:41:53.678 INFO:teuthology.orchestra.run.smithi134.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-09-10T09:41:53.703 INFO:teuthology.orchestra.run.smithi134.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 7.0 MB/s | 358 kB 00:00 2024-09-10T09:41:53.728 INFO:teuthology.orchestra.run.smithi134.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.9 MB/s | 303 kB 00:00 2024-09-10T09:41:53.753 INFO:teuthology.orchestra.run.smithi134.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-10T09:41:53.779 INFO:teuthology.orchestra.run.smithi134.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 5.7 MB/s | 144 kB 00:00 2024-09-10T09:41:53.796 INFO:teuthology.orchestra.run.smithi134.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 642 kB/s | 11 kB 00:00 2024-09-10T09:41:53.813 INFO:teuthology.orchestra.run.smithi134.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 1.0 MB/s | 18 kB 00:00 2024-09-10T09:41:53.829 INFO:teuthology.orchestra.run.smithi134.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 1.4 MB/s | 23 kB 00:00 2024-09-10T09:41:53.846 INFO:teuthology.orchestra.run.smithi077.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 2.1 MB/s | 2.4 MB 00:01 2024-09-10T09:41:53.846 INFO:teuthology.orchestra.run.smithi134.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 1.1 MB/s | 19 kB 00:00 2024-09-10T09:41:53.852 INFO:teuthology.orchestra.run.smithi077.stdout:-------------------------------------------------------------------------------- 2024-09-10T09:41:53.852 INFO:teuthology.orchestra.run.smithi077.stdout:Total 15 MB/s | 226 MB 00:15 2024-09-10T09:41:53.863 INFO:teuthology.orchestra.run.smithi134.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 1.2 MB/s | 20 kB 00:00 2024-09-10T09:41:53.914 INFO:teuthology.orchestra.run.smithi134.stdout:(128/156): python3-grpcio-1.46.7-10.el9.x86_64. 11 MB/s | 2.0 MB 00:00 2024-09-10T09:41:53.931 INFO:teuthology.orchestra.run.smithi134.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 2.7 MB/s | 46 kB 00:00 2024-09-10T09:41:53.956 INFO:teuthology.orchestra.run.smithi134.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-10T09:41:53.981 INFO:teuthology.orchestra.run.smithi134.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-10T09:41:54.015 INFO:teuthology.orchestra.run.smithi134.stdout:(132/156): python3-kubernetes-26.1.0-2.el9.noar 6.8 MB/s | 1.0 MB 00:00 2024-09-10T09:41:54.032 INFO:teuthology.orchestra.run.smithi134.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 994 kB/s | 16 kB 00:00 2024-09-10T09:41:54.108 INFO:teuthology.orchestra.run.smithi120.stdout:(1/156): ceph-19.1.1-305.g1cf819b7.el9.x86_64.r 41 kB/s | 6.5 kB 00:00 2024-09-10T09:41:54.182 INFO:teuthology.orchestra.run.smithi134.stdout:(134/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 7.4 MB/s | 16 MB 00:02 2024-09-10T09:41:54.232 INFO:teuthology.orchestra.run.smithi134.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 449 kB/s | 90 kB 00:00 2024-09-10T09:41:54.262 INFO:teuthology.orchestra.run.smithi134.stdout:(136/156): python3-pecan-1.4.2-3.el9.noarch.rpm 987 kB/s | 272 kB 00:00 2024-09-10T09:41:54.274 INFO:teuthology.orchestra.run.smithi134.stdout:(137/156): python3-pyperclip-1.8.2-2.el9.noarch 307 kB/s | 28 kB 00:00 2024-09-10T09:41:54.291 INFO:teuthology.orchestra.run.smithi134.stdout:(138/156): python3-repoze-lru-0.7-16.el9.noarch 542 kB/s | 31 kB 00:00 2024-09-10T09:41:54.316 INFO:teuthology.orchestra.run.smithi134.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-10T09:41:54.342 INFO:teuthology.orchestra.run.smithi134.stdout:(140/156): python3-routes-2.5.1-5.el9.noarch.rp 2.2 MB/s | 188 kB 00:00 2024-09-10T09:41:54.359 INFO:teuthology.orchestra.run.smithi134.stdout:(141/156): python3-tempora-5.0.0-2.el9.noarch.r 536 kB/s | 36 kB 00:00 2024-09-10T09:41:54.376 INFO:teuthology.orchestra.run.smithi134.stdout:(142/156): python3-typing-extensions-4.12.2-2.e 1.2 MB/s | 74 kB 00:00 2024-09-10T09:41:54.401 INFO:teuthology.orchestra.run.smithi134.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 2.1 MB/s | 90 kB 00:00 2024-09-10T09:41:54.408 INFO:teuthology.orchestra.run.smithi120.stdout:(2/156): ceph-fuse-19.1.1-305.g1cf819b7.el9.x86 2.9 MB/s | 897 kB 00:00 2024-09-10T09:41:54.418 INFO:teuthology.orchestra.run.smithi134.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 1.2 MB/s | 20 kB 00:00 2024-09-10T09:41:54.443 INFO:teuthology.orchestra.run.smithi134.stdout:(145/156): python3-webob-1.8.8-2.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-09-10T09:41:54.468 INFO:teuthology.orchestra.run.smithi134.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-10T09:41:54.491 INFO:teuthology.orchestra.run.smithi120.stdout:(3/156): ceph-base-19.1.1-305.g1cf819b7.el9.x86 9.9 MB/s | 5.4 MB 00:00 2024-09-10T09:41:54.493 INFO:teuthology.orchestra.run.smithi134.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.5 MB/s | 427 kB 00:00 2024-09-10T09:41:54.517 INFO:teuthology.orchestra.run.smithi120.stdout:(4/156): ceph-immutable-object-cache-19.1.1-305 1.3 MB/s | 145 kB 00:00 2024-09-10T09:41:54.544 INFO:teuthology.orchestra.run.smithi134.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-10T09:41:54.659 INFO:teuthology.orchestra.run.smithi120.stdout:(5/156): ceph-mds-19.1.1-305.g1cf819b7.el9.x86_ 14 MB/s | 2.4 MB 00:00 2024-09-10T09:41:54.951 INFO:teuthology.orchestra.run.smithi120.stdout:(6/156): ceph-mon-19.1.1-305.g1cf819b7.el9.x86_ 16 MB/s | 4.7 MB 00:00 2024-09-10T09:41:54.984 INFO:teuthology.orchestra.run.smithi120.stdout:(7/156): ceph-mgr-19.1.1-305.g1cf819b7.el9.x86_ 3.3 MB/s | 1.5 MB 00:00 2024-09-10T09:41:55.169 INFO:teuthology.orchestra.run.smithi134.stdout:(149/156): librbd1-19.1.1-305.g1cf819b7.el9.x86 4.7 MB/s | 3.2 MB 00:00 2024-09-10T09:41:55.193 INFO:teuthology.orchestra.run.smithi120.stdout:(8/156): ceph-common-19.1.1-305.g1cf819b7.el9.x 17 MB/s | 21 MB 00:01 2024-09-10T09:41:55.235 INFO:teuthology.orchestra.run.smithi120.stdout:(9/156): ceph-selinux-19.1.1-305.g1cf819b7.el9. 610 kB/s | 25 kB 00:00 2024-09-10T09:41:55.544 INFO:teuthology.orchestra.run.smithi134.stdout:(150/156): NetworkManager-1.49.90-1.el9.x86_64. 2.3 MB/s | 2.3 MB 00:00 2024-09-10T09:41:55.669 INFO:teuthology.orchestra.run.smithi134.stdout:(151/156): librados2-19.1.1-305.g1cf819b7.el9.x 2.8 MB/s | 3.4 MB 00:01 2024-09-10T09:41:55.744 INFO:teuthology.orchestra.run.smithi120.stdout:(10/156): ceph-osd-19.1.1-305.g1cf819b7.el9.x86 21 MB/s | 17 MB 00:00 2024-09-10T09:41:55.794 INFO:teuthology.orchestra.run.smithi120.stdout:(11/156): cephfs-mirror-19.1.1-305.g1cf819b7.el 4.5 MB/s | 228 kB 00:00 2024-09-10T09:41:55.853 INFO:teuthology.orchestra.run.smithi120.stdout:(12/156): cephfs-shell-19.1.1-305.g1cf819b7.el9 465 kB/s | 27 kB 00:00 2024-09-10T09:41:55.911 INFO:teuthology.orchestra.run.smithi120.stdout:(13/156): libcephfs-devel-19.1.1-305.g1cf819b7. 543 kB/s | 32 kB 00:00 2024-09-10T09:41:55.927 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction check 2024-09-10T09:41:55.978 INFO:teuthology.orchestra.run.smithi120.stdout:(14/156): libcephfs2-19.1.1-305.g1cf819b7.el9.x 11 MB/s | 751 kB 00:00 2024-09-10T09:41:56.037 INFO:teuthology.orchestra.run.smithi120.stdout:(15/156): libcephsqlite-19.1.1-305.g1cf819b7.el 2.7 MB/s | 164 kB 00:00 2024-09-10T09:41:56.087 INFO:teuthology.orchestra.run.smithi120.stdout:(16/156): librados-devel-19.1.1-305.g1cf819b7.e 2.5 MB/s | 127 kB 00:00 2024-09-10T09:41:56.107 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction check succeeded. 2024-09-10T09:41:56.107 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction test 2024-09-10T09:41:56.154 INFO:teuthology.orchestra.run.smithi120.stdout:(17/156): libradosstriper1-19.1.1-305.g1cf819b7 7.4 MB/s | 506 kB 00:00 2024-09-10T09:41:56.161 INFO:teuthology.orchestra.run.smithi134.stdout:(152/156): NetworkManager-team-1.49.90-1.el9.x8 66 kB/s | 41 kB 00:00 2024-09-10T09:41:56.203 INFO:teuthology.orchestra.run.smithi134.stdout:(153/156): NetworkManager-libnm-1.49.90-1.el9.x 1.7 MB/s | 1.8 MB 00:01 2024-09-10T09:41:56.430 INFO:teuthology.orchestra.run.smithi120.stdout:(18/156): ceph-radosgw-19.1.1-305.g1cf819b7.el9 7.3 MB/s | 11 MB 00:01 2024-09-10T09:41:56.445 INFO:teuthology.orchestra.run.smithi134.stdout:(154/156): NetworkManager-tui-1.49.90-1.el9.x86 321 kB/s | 249 kB 00:00 2024-09-10T09:41:56.513 INFO:teuthology.orchestra.run.smithi120.stdout:(19/156): librgw2-19.1.1-305.g1cf819b7.el9.x86_ 15 MB/s | 5.3 MB 00:00 2024-09-10T09:41:56.538 INFO:teuthology.orchestra.run.smithi120.stdout:(20/156): python3-ceph-argparse-19.1.1-305.g1cf 416 kB/s | 45 kB 00:00 2024-09-10T09:41:56.564 INFO:teuthology.orchestra.run.smithi120.stdout:(21/156): python3-ceph-common-19.1.1-305.g1cf81 2.7 MB/s | 136 kB 00:00 2024-09-10T09:41:56.598 INFO:teuthology.orchestra.run.smithi120.stdout:(22/156): python3-cephfs-19.1.1-305.g1cf819b7.e 2.7 MB/s | 161 kB 00:00 2024-09-10T09:41:56.631 INFO:teuthology.orchestra.run.smithi120.stdout:(23/156): python3-rados-19.1.1-305.g1cf819b7.el 4.7 MB/s | 320 kB 00:00 2024-09-10T09:41:56.656 INFO:teuthology.orchestra.run.smithi120.stdout:(24/156): python3-rbd-19.1.1-305.g1cf819b7.el9. 5.0 MB/s | 301 kB 00:00 2024-09-10T09:41:56.682 INFO:teuthology.orchestra.run.smithi120.stdout:(25/156): python3-rgw-19.1.1-305.g1cf819b7.el9. 1.9 MB/s | 99 kB 00:00 2024-09-10T09:41:56.707 INFO:teuthology.orchestra.run.smithi120.stdout:(26/156): rbd-fuse-19.1.1-305.g1cf819b7.el9.x86 1.7 MB/s | 85 kB 00:00 2024-09-10T09:41:56.766 INFO:teuthology.orchestra.run.smithi120.stdout:(27/156): rbd-nbd-19.1.1-305.g1cf819b7.el9.x86_ 2.8 MB/s | 170 kB 00:00 2024-09-10T09:41:56.825 INFO:teuthology.orchestra.run.smithi120.stdout:(28/156): rbd-mirror-19.1.1-305.g1cf819b7.el9.x 22 MB/s | 3.1 MB 00:00 2024-09-10T09:41:56.850 INFO:teuthology.orchestra.run.smithi120.stdout:(29/156): ceph-grafana-dashboards-19.1.1-305.g1 381 kB/s | 32 kB 00:00 2024-09-10T09:41:56.875 INFO:teuthology.orchestra.run.smithi120.stdout:(30/156): ceph-mgr-cephadm-19.1.1-305.g1cf819b7 2.8 MB/s | 146 kB 00:00 2024-09-10T09:41:57.095 INFO:teuthology.orchestra.run.smithi134.stdout:(155/156): openssl-3.2.2-6.el9.x86_64.rpm 1.4 MB/s | 1.4 MB 00:00 2024-09-10T09:41:57.229 INFO:teuthology.orchestra.run.smithi134.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 2.3 MB/s | 2.4 MB 00:01 2024-09-10T09:41:57.236 INFO:teuthology.orchestra.run.smithi134.stdout:-------------------------------------------------------------------------------- 2024-09-10T09:41:57.236 INFO:teuthology.orchestra.run.smithi134.stdout:Total 12 MB/s | 226 MB 00:18 2024-09-10T09:41:57.242 INFO:teuthology.orchestra.run.smithi120.stdout:(31/156): ceph-mgr-diskprediction-local-19.1.1- 20 MB/s | 7.4 MB 00:00 2024-09-10T09:41:57.334 INFO:teuthology.orchestra.run.smithi120.stdout:(32/156): ceph-mgr-dashboard-19.1.1-305.g1cf819 7.4 MB/s | 3.6 MB 00:00 2024-09-10T09:41:57.359 INFO:teuthology.orchestra.run.smithi120.stdout:(33/156): ceph-mgr-modules-core-19.1.1-305.g1cf 2.1 MB/s | 249 kB 00:00 2024-09-10T09:41:57.385 INFO:teuthology.orchestra.run.smithi120.stdout:(34/156): ceph-mgr-rook-19.1.1-305.g1cf819b7.el 980 kB/s | 49 kB 00:00 2024-09-10T09:41:57.410 INFO:teuthology.orchestra.run.smithi120.stdout:(35/156): ceph-prometheus-alerts-19.1.1-305.g1c 324 kB/s | 16 kB 00:00 2024-09-10T09:41:57.436 INFO:teuthology.orchestra.run.smithi120.stdout:(36/156): ceph-volume-19.1.1-305.g1cf819b7.el9. 5.5 MB/s | 285 kB 00:00 2024-09-10T09:41:57.478 INFO:teuthology.orchestra.run.smithi120.stdout:(37/156): cephadm-19.1.1-305.g1cf819b7.el9.noar 11 MB/s | 766 kB 00:00 2024-09-10T09:41:57.503 INFO:teuthology.orchestra.run.smithi120.stdout:(38/156): cephfs-top-19.1.1-305.g1cf819b7.el9.n 273 kB/s | 18 kB 00:00 2024-09-10T09:41:57.620 INFO:teuthology.orchestra.run.smithi120.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 563 kB/s | 80 kB 00:00 2024-09-10T09:41:57.695 INFO:teuthology.orchestra.run.smithi120.stdout:(40/156): libquadmath-11.5.0-2.el9.x86_64.rpm 2.5 MB/s | 191 kB 00:00 2024-09-10T09:41:57.720 INFO:teuthology.orchestra.run.smithi120.stdout:(41/156): libgfortran-11.5.0-2.el9.x86_64.rpm 3.6 MB/s | 801 kB 00:00 2024-09-10T09:41:57.754 INFO:teuthology.orchestra.run.smithi120.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.2 MB/s | 253 kB 00:00 2024-09-10T09:41:58.213 INFO:teuthology.orchestra.run.smithi120.stdout:(43/156): ceph-test-19.1.1-305.g1cf819b7.el9.x8 16 MB/s | 49 MB 00:02 2024-09-10T09:41:58.246 INFO:teuthology.orchestra.run.smithi120.stdout:(44/156): python3-cryptography-36.0.1-4.el9.x86 2.3 MB/s | 1.2 MB 00:00 2024-09-10T09:41:58.272 INFO:teuthology.orchestra.run.smithi120.stdout:(45/156): python3-ply-3.11-14.el9.noarch.rpm 206 kB/s | 106 kB 00:00 2024-09-10T09:41:58.297 INFO:teuthology.orchestra.run.smithi120.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-09-10T09:41:58.339 INFO:teuthology.orchestra.run.smithi120.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 3.1 MB/s | 215 kB 00:00 2024-09-10T09:41:58.373 INFO:teuthology.orchestra.run.smithi120.stdout:(48/156): python3-pycparser-2.20-6.el9.noarch.r 848 kB/s | 135 kB 00:00 2024-09-10T09:41:58.474 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction check 2024-09-10T09:41:58.480 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction test succeeded. 2024-09-10T09:41:58.481 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction 2024-09-10T09:41:58.556 INFO:teuthology.orchestra.run.smithi120.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 584 kB/s | 107 kB 00:00 2024-09-10T09:41:58.624 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction check succeeded. 2024-09-10T09:41:58.624 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction test 2024-09-10T09:41:58.624 INFO:teuthology.orchestra.run.smithi120.stdout:(50/156): automake-1.16.2-8.el9.noarch.rpm 2.1 MB/s | 689 kB 00:00 2024-09-10T09:41:58.641 INFO:teuthology.orchestra.run.smithi120.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 259 kB/s | 22 kB 00:00 2024-09-10T09:41:58.682 INFO:teuthology.orchestra.run.smithi120.stdout:(52/156): bison-3.7.4-5.el9.x86_64.rpm 2.7 MB/s | 943 kB 00:00 2024-09-10T09:41:58.708 INFO:teuthology.orchestra.run.smithi120.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 489 kB/s | 33 kB 00:00 2024-09-10T09:41:58.741 INFO:teuthology.orchestra.run.smithi120.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 2.6 MB/s | 313 kB 00:00 2024-09-10T09:41:58.766 INFO:teuthology.orchestra.run.smithi120.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 310 kB/s | 18 kB 00:00 2024-09-10T09:41:58.792 INFO:teuthology.orchestra.run.smithi120.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-09-10T09:41:58.842 INFO:teuthology.orchestra.run.smithi120.stdout:(57/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 904 kB/s | 45 kB 00:00 2024-09-10T09:41:58.867 INFO:teuthology.orchestra.run.smithi120.stdout:(58/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.6 MB/s | 160 kB 00:00 2024-09-10T09:41:58.934 INFO:teuthology.orchestra.run.smithi120.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 7.0 MB/s | 662 kB 00:00 2024-09-10T09:41:58.993 INFO:teuthology.orchestra.run.smithi120.stdout:(60/156): libtool-2.4.6-46.el9.x86_64.rpm 4.5 MB/s | 581 kB 00:00 2024-09-10T09:41:59.019 INFO:teuthology.orchestra.run.smithi120.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 292 kB/s | 24 kB 00:00 2024-09-10T09:41:59.044 INFO:teuthology.orchestra.run.smithi120.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 940 kB/s | 47 kB 00:00 2024-09-10T09:41:59.103 INFO:teuthology.orchestra.run.smithi120.stdout:(63/156): lua-5.4.4-4.el9.x86_64.rpm 3.2 MB/s | 188 kB 00:00 2024-09-10T09:41:59.153 INFO:teuthology.orchestra.run.smithi120.stdout:(64/156): openblas-0.3.26-2.el9.x86_64.rpm 770 kB/s | 38 kB 00:00 2024-09-10T09:41:59.195 INFO:teuthology.orchestra.run.smithi120.stdout:(65/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.6 MB/s | 292 kB 00:00 2024-09-10T09:41:59.912 INFO:teuthology.orchestra.run.smithi120.stdout:(66/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.4 MB/s | 3.0 MB 00:01 2024-09-10T09:42:00.043 INFO:teuthology.orchestra.run.smithi120.stdout:(67/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 5.2 MB/s | 4.4 MB 00:00 2024-09-10T09:42:00.110 INFO:teuthology.orchestra.run.smithi120.stdout:(68/156): postgresql-contrib-13.16-1.el9.x86_64 13 MB/s | 910 kB 00:00 2024-09-10T09:42:00.177 INFO:teuthology.orchestra.run.smithi120.stdout:(69/156): postgresql-private-libs-13.16-1.el9.x 2.0 MB/s | 137 kB 00:00 2024-09-10T09:42:00.219 INFO:teuthology.orchestra.run.smithi120.stdout:(70/156): postgresql-13.16-1.el9.x86_64.rpm 5.2 MB/s | 1.6 MB 00:00 2024-09-10T09:42:00.294 INFO:teuthology.orchestra.run.smithi120.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 13 MB/s | 1.0 MB 00:00 2024-09-10T09:42:00.369 INFO:teuthology.orchestra.run.smithi120.stdout:(72/156): postgresql-server-13.16-1.el9.x86_64. 30 MB/s | 5.8 MB 00:00 2024-09-10T09:42:00.395 INFO:teuthology.orchestra.run.smithi120.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 872 kB/s | 87 kB 00:00 2024-09-10T09:42:00.462 INFO:teuthology.orchestra.run.smithi120.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 920 kB/s | 61 kB 00:00 2024-09-10T09:42:00.488 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction test succeeded. 2024-09-10T09:42:00.488 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction 2024-09-10T09:42:00.537 INFO:teuthology.orchestra.run.smithi120.stdout:(75/156): python3-babel-2.9.1-2.el9.noarch.rpm 36 MB/s | 6.0 MB 00:00 2024-09-10T09:42:00.562 INFO:teuthology.orchestra.run.smithi120.stdout:(76/156): python3-devel-3.9.19-8.el9.x86_64.rpm 2.4 MB/s | 245 kB 00:00 2024-09-10T09:42:00.596 INFO:teuthology.orchestra.run.smithi120.stdout:(77/156): python3-jinja2-2.11.3-6.el9.noarch.rp 4.1 MB/s | 248 kB 00:00 2024-09-10T09:42:00.621 INFO:teuthology.orchestra.run.smithi120.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 437 kB/s | 26 kB 00:00 2024-09-10T09:42:00.647 INFO:teuthology.orchestra.run.smithi120.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 380 kB/s | 19 kB 00:00 2024-09-10T09:42:00.672 INFO:teuthology.orchestra.run.smithi120.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-09-10T09:42:00.697 INFO:teuthology.orchestra.run.smithi120.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 693 kB/s | 35 kB 00:00 2024-09-10T09:42:00.756 INFO:teuthology.orchestra.run.smithi120.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.2 MB/s | 250 kB 00:00 2024-09-10T09:42:00.823 INFO:teuthology.orchestra.run.smithi120.stdout:(83/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 34 MB/s | 5.1 MB 00:00 2024-09-10T09:42:00.848 INFO:teuthology.orchestra.run.smithi120.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 2.4 MB/s | 222 kB 00:00 2024-09-10T09:42:00.882 INFO:teuthology.orchestra.run.smithi120.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-09-10T09:42:00.907 INFO:teuthology.orchestra.run.smithi120.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 718 kB/s | 42 kB 00:00 2024-09-10T09:42:00.949 INFO:teuthology.orchestra.run.smithi120.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 3.9 MB/s | 269 kB 00:00 2024-09-10T09:42:00.975 INFO:teuthology.orchestra.run.smithi120.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 159 kB 00:00 2024-09-10T09:42:01.008 INFO:teuthology.orchestra.run.smithi120.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 4.6 MB/s | 279 kB 00:00 2024-09-10T09:42:01.025 INFO:teuthology.orchestra.run.smithi120.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-10T09:42:01.076 INFO:teuthology.orchestra.run.smithi120.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 801 kB/s | 54 kB 00:00 2024-09-10T09:42:01.118 INFO:teuthology.orchestra.run.smithi120.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 995 kB/s | 42 kB 00:00 2024-09-10T09:42:01.160 INFO:teuthology.orchestra.run.smithi120.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.0 MB/s | 44 kB 00:00 2024-09-10T09:42:01.218 INFO:teuthology.orchestra.run.smithi120.stdout:(94/156): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-09-10T09:42:01.269 INFO:teuthology.orchestra.run.smithi120.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 714 kB/s | 36 kB 00:00 2024-09-10T09:42:01.427 INFO:teuthology.orchestra.run.smithi120.stdout:(96/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 40 MB/s | 16 MB 00:00 2024-09-10T09:42:01.452 INFO:teuthology.orchestra.run.smithi120.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 258 kB/s | 47 kB 00:00 2024-09-10T09:42:01.528 INFO:teuthology.orchestra.run.smithi120.stdout:(98/156): openblas-openmp-0.3.26-2.el9.x86_64.r 2.1 MB/s | 4.9 MB 00:02 2024-09-10T09:42:01.562 INFO:teuthology.orchestra.run.smithi120.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 530 kB/s | 58 kB 00:00 2024-09-10T09:42:01.587 INFO:teuthology.orchestra.run.smithi120.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.6 MB/s | 96 kB 00:00 2024-09-10T09:42:01.646 INFO:teuthology.orchestra.run.smithi120.stdout:(101/156): socat-1.7.4.1-6.el9.x86_64.rpm 1.4 MB/s | 304 kB 00:00 2024-09-10T09:42:01.671 INFO:teuthology.orchestra.run.smithi120.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 536 kB/s | 45 kB 00:00 2024-09-10T09:42:01.696 INFO:teuthology.orchestra.run.smithi120.stdout:(103/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 474 kB/s | 64 kB 00:00 2024-09-10T09:42:01.747 INFO:teuthology.orchestra.run.smithi120.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-09-10T09:42:01.772 INFO:teuthology.orchestra.run.smithi120.stdout:(105/156): lua-devel-5.4.4-4.el9.x86_64.rpm 177 kB/s | 22 kB 00:00 2024-09-10T09:42:01.805 INFO:teuthology.orchestra.run.smithi120.stdout:(106/156): gperftools-libs-2.9.1-2.el9.x86_64.r 5.2 MB/s | 309 kB 00:00 2024-09-10T09:42:01.831 INFO:teuthology.orchestra.run.smithi120.stdout:(107/156): grpc-data-1.46.7-10.el9.noarch.rpm 330 kB/s | 19 kB 00:00 2024-09-10T09:42:01.856 INFO:teuthology.orchestra.run.smithi120.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-10T09:42:01.881 INFO:teuthology.orchestra.run.smithi120.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-10T09:42:01.915 INFO:teuthology.orchestra.run.smithi120.stdout:(110/156): protobuf-compiler-3.14.0-13.el9.x86_ 3.5 MB/s | 863 kB 00:00 2024-09-10T09:42:01.949 INFO:teuthology.orchestra.run.smithi120.stdout:(111/156): libunwind-1.6.2-1.el9.x86_64.rpm 1.0 MB/s | 67 kB 00:00 2024-09-10T09:42:01.991 INFO:teuthology.orchestra.run.smithi120.stdout:(112/156): luarocks-3.9.2-5.el9.noarch.rpm 2.0 MB/s | 151 kB 00:00 2024-09-10T09:42:02.066 INFO:teuthology.orchestra.run.smithi120.stdout:(113/156): libarrow-9.0.0-11.el9.x86_64.rpm 17 MB/s | 4.4 MB 00:00 2024-09-10T09:42:02.091 INFO:teuthology.orchestra.run.smithi120.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-10T09:42:02.117 INFO:teuthology.orchestra.run.smithi120.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-10T09:42:02.150 INFO:teuthology.orchestra.run.smithi120.stdout:(116/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 4.1 MB/s | 839 kB 00:00 2024-09-10T09:42:02.176 INFO:teuthology.orchestra.run.smithi120.stdout:(117/156): python3-certifi-2023.05.07-4.el9.noa 240 kB/s | 14 kB 00:00 2024-09-10T09:42:02.201 INFO:teuthology.orchestra.run.smithi120.stdout:(118/156): python3-asyncssh-2.13.2-5.el9.noarch 2.6 MB/s | 548 kB 00:00 2024-09-10T09:42:02.226 INFO:teuthology.orchestra.run.smithi120.stdout:(119/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.2 MB/s | 172 kB 00:00 2024-09-10T09:42:02.260 INFO:teuthology.orchestra.run.smithi120.stdout:(120/156): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-09-10T09:42:02.293 INFO:teuthology.orchestra.run.smithi120.stdout:(121/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.2 MB/s | 303 kB 00:00 2024-09-10T09:42:02.319 INFO:teuthology.orchestra.run.smithi120.stdout:(122/156): python3-google-auth-2.34.0-1.el9.noa 2.3 MB/s | 220 kB 00:00 2024-09-10T09:42:02.344 INFO:teuthology.orchestra.run.smithi120.stdout:(123/156): python3-grpcio-tools-1.46.7-10.el9.x 2.8 MB/s | 144 kB 00:00 2024-09-10T09:42:02.361 INFO:teuthology.orchestra.run.smithi120.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 254 kB/s | 11 kB 00:00 2024-09-10T09:42:02.378 INFO:teuthology.orchestra.run.smithi120.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 519 kB/s | 18 kB 00:00 2024-09-10T09:42:02.395 INFO:teuthology.orchestra.run.smithi120.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 680 kB/s | 23 kB 00:00 2024-09-10T09:42:02.412 INFO:teuthology.orchestra.run.smithi120.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 582 kB/s | 19 kB 00:00 2024-09-10T09:42:02.438 INFO:teuthology.orchestra.run.smithi120.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 469 kB/s | 20 kB 00:00 2024-09-10T09:42:02.463 INFO:teuthology.orchestra.run.smithi120.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-10T09:42:02.488 INFO:teuthology.orchestra.run.smithi120.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-10T09:42:02.513 INFO:teuthology.orchestra.run.smithi120.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-10T09:42:02.547 INFO:teuthology.orchestra.run.smithi120.stdout:(132/156): python3-pecan-1.4.2-3.el9.noarch.rpm 8.0 MB/s | 272 kB 00:00 2024-09-10T09:42:02.573 INFO:teuthology.orchestra.run.smithi120.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 662 kB/s | 16 kB 00:00 2024-09-10T09:42:02.606 INFO:teuthology.orchestra.run.smithi120.stdout:(134/156): python3-kubernetes-26.1.0-2.el9.noar 5.3 MB/s | 1.0 MB 00:00 2024-09-10T09:42:02.632 INFO:teuthology.orchestra.run.smithi120.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-09-10T09:42:02.657 INFO:teuthology.orchestra.run.smithi120.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 563 kB/s | 28 kB 00:00 2024-09-10T09:42:02.682 INFO:teuthology.orchestra.run.smithi120.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 611 kB/s | 31 kB 00:00 2024-09-10T09:42:02.707 INFO:teuthology.orchestra.run.smithi120.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-10T09:42:02.733 INFO:teuthology.orchestra.run.smithi120.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-09-10T09:42:02.758 INFO:teuthology.orchestra.run.smithi120.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 710 kB/s | 36 kB 00:00 2024-09-10T09:42:02.783 INFO:teuthology.orchestra.run.smithi120.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-09-10T09:42:02.809 INFO:teuthology.orchestra.run.smithi120.stdout:(142/156): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-10T09:42:02.834 INFO:teuthology.orchestra.run.smithi120.stdout:(143/156): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-10T09:42:02.860 INFO:teuthology.orchestra.run.smithi120.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 802 kB/s | 20 kB 00:00 2024-09-10T09:42:02.901 INFO:teuthology.orchestra.run.smithi120.stdout:(145/156): python3-grpcio-1.46.7-10.el9.x86_64. 3.2 MB/s | 2.0 MB 00:00 2024-09-10T09:42:02.927 INFO:teuthology.orchestra.run.smithi120.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 2.8 MB/s | 191 kB 00:00 2024-09-10T09:42:02.952 INFO:teuthology.orchestra.run.smithi120.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 2.9 MB/s | 427 kB 00:00 2024-09-10T09:42:03.002 INFO:teuthology.orchestra.run.smithi120.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-10T09:42:03.269 INFO:teuthology.orchestra.run.smithi120.stdout:(149/156): librados2-19.1.1-305.g1cf819b7.el9.x 10 MB/s | 3.4 MB 00:00 2024-09-10T09:42:03.320 INFO:teuthology.orchestra.run.smithi120.stdout:(150/156): NetworkManager-1.49.90-1.el9.x86_64. 7.5 MB/s | 2.3 MB 00:00 2024-09-10T09:42:03.362 INFO:teuthology.orchestra.run.smithi120.stdout:(151/156): NetworkManager-team-1.49.90-1.el9.x8 975 kB/s | 41 kB 00:00 2024-09-10T09:42:03.404 INFO:teuthology.orchestra.run.smithi120.stdout:(152/156): NetworkManager-tui-1.49.90-1.el9.x86 5.8 MB/s | 249 kB 00:00 2024-09-10T09:42:03.454 INFO:teuthology.orchestra.run.smithi120.stdout:(153/156): NetworkManager-libnm-1.49.90-1.el9.x 9.8 MB/s | 1.8 MB 00:00 2024-09-10T09:42:03.496 INFO:teuthology.orchestra.run.smithi120.stdout:(154/156): openssl-3.2.2-6.el9.x86_64.rpm 15 MB/s | 1.4 MB 00:00 2024-09-10T09:42:03.546 INFO:teuthology.orchestra.run.smithi120.stdout:(155/156): librbd1-19.1.1-305.g1cf819b7.el9.x86 5.3 MB/s | 3.2 MB 00:00 2024-09-10T09:42:03.605 INFO:teuthology.orchestra.run.smithi120.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 16 MB/s | 2.4 MB 00:00 2024-09-10T09:42:03.612 INFO:teuthology.orchestra.run.smithi120.stdout:-------------------------------------------------------------------------------- 2024-09-10T09:42:03.612 INFO:teuthology.orchestra.run.smithi120.stdout:Total 22 MB/s | 226 MB 00:10 2024-09-10T09:42:04.858 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction check 2024-09-10T09:42:05.009 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction check succeeded. 2024-09-10T09:42:05.009 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction test 2024-09-10T09:42:05.082 INFO:teuthology.orchestra.run.smithi077.stdout: Preparing : 1/1 2024-09-10T09:42:05.259 INFO:teuthology.orchestra.run.smithi077.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-10T09:42:05.506 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-10T09:42:05.694 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-10T09:42:05.746 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-10T09:42:05.777 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-10T09:42:05.834 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-10T09:42:06.200 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-10T09:42:06.322 INFO:teuthology.orchestra.run.smithi077.stdout: Upgrading : librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T09:42:06.454 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T09:42:06.495 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T09:42:06.599 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T09:42:06.641 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 10/164 2024-09-10T09:42:06.811 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction test succeeded. 2024-09-10T09:42:06.812 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction 2024-09-10T09:42:06.862 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-10T09:42:06.923 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-10T09:42:06.949 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T09:42:06.977 INFO:teuthology.orchestra.run.smithi134.stdout: Preparing : 1/1 2024-09-10T09:42:07.076 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T09:42:07.099 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-10T09:42:07.145 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-10T09:42:07.150 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 14/164 2024-09-10T09:42:07.198 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-10T09:42:07.233 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-10T09:42:07.243 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 14/164 2024-09-10T09:42:07.258 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-10T09:42:07.276 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-10T09:42:07.300 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-10T09:42:07.319 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-10T09:42:07.524 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-10T09:42:07.625 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-10T09:42:07.738 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-10T09:42:07.751 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T09:42:07.872 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T09:42:07.900 INFO:teuthology.orchestra.run.smithi077.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-10T09:42:07.902 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T09:42:07.993 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T09:42:08.022 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 10/164 2024-09-10T09:42:08.060 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T09:42:08.159 INFO:teuthology.orchestra.run.smithi077.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T09:42:08.170 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-10T09:42:08.228 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-10T09:42:08.255 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T09:42:08.359 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T09:42:08.381 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 14/164 2024-09-10T09:42:08.444 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 14/164 2024-09-10T09:42:08.457 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T09:42:08.474 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-10T09:42:08.500 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-10T09:42:08.590 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-10T09:42:08.708 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-10T09:42:08.718 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-10T09:42:08.774 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-10T09:42:08.819 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-10T09:42:08.847 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-10T09:42:08.900 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-10T09:42:08.954 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-10T09:42:08.964 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-10T09:42:08.993 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-10T09:42:09.047 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-10T09:42:09.076 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T09:42:09.086 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-10T09:42:09.128 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-10T09:42:09.157 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T09:42:09.215 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-10T09:42:09.272 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-10T09:42:09.316 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-10T09:42:09.362 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-10T09:42:09.384 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T09:42:09.443 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-10T09:42:09.478 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x 34/164 2024-09-10T09:42:09.499 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-10T09:42:09.538 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9 35/164 2024-09-10T09:42:09.566 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 36/164 2024-09-10T09:42:09.593 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-10T09:42:09.629 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-10T09:42:09.654 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-10T09:42:09.671 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-10T09:42:09.718 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-10T09:42:09.753 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-10T09:42:09.797 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-10T09:42:09.809 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-10T09:42:09.829 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-10T09:42:09.873 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-10T09:42:09.905 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-10T09:42:09.934 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-10T09:42:09.969 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-10T09:42:10.001 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-10T09:42:10.029 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-10T09:42:10.055 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-10T09:42:10.095 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-10T09:42:10.138 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-10T09:42:10.213 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-10T09:42:10.249 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x 34/164 2024-09-10T09:42:10.292 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9 35/164 2024-09-10T09:42:10.321 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 36/164 2024-09-10T09:42:10.390 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-10T09:42:10.439 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-10T09:42:10.625 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-10T09:42:10.660 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-10T09:42:10.770 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-10T09:42:10.809 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-10T09:42:10.860 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-10T09:42:10.888 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-10T09:42:10.899 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 46/164 2024-09-10T09:42:10.941 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-10T09:42:10.952 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-10T09:42:11.135 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-10T09:42:11.210 INFO:teuthology.orchestra.run.smithi077.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-10T09:42:11.278 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-10T09:42:11.362 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-10T09:42:11.402 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-10T09:42:11.438 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-10T09:42:11.474 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-10T09:42:11.515 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-10T09:42:11.554 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-10T09:42:11.589 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-10T09:42:11.589 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-10T09:42:11.600 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-10T09:42:11.625 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-10T09:42:11.650 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-10T09:42:11.663 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 46/164 2024-09-10T09:42:11.705 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-10T09:42:11.819 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-10T09:42:11.860 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-10T09:42:11.888 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-10T09:42:11.916 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-10T09:42:11.938 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-10T09:42:11.973 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-10T09:42:12.043 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-10T09:42:12.055 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-10T09:42:12.076 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-10T09:42:12.104 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-10T09:42:12.112 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-10T09:42:12.145 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-10T09:42:12.174 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-10T09:42:12.175 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-09-10T09:42:12.200 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-10T09:42:12.211 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-10T09:42:12.254 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-10T09:42:12.296 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-10T09:42:12.308 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-10T09:42:12.438 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-10T09:42:12.459 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-10T09:42:12.489 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-10T09:42:12.500 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-10T09:42:12.536 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-10T09:42:12.578 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-10T09:42:12.620 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-10T09:42:12.658 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-10T09:42:12.721 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-10T09:42:12.739 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-10T09:42:12.755 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-10T09:42:12.789 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-10T09:42:12.832 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-10T09:42:12.840 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-10T09:42:12.865 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-10T09:42:12.872 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-10T09:42:12.911 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-10T09:42:12.937 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-09-10T09:42:12.962 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-10T09:42:12.971 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-10T09:42:13.036 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-10T09:42:13.056 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-10T09:42:13.093 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-10T09:42:13.141 INFO:teuthology.orchestra.run.smithi120.stdout: Preparing : 1/1 2024-09-10T09:42:13.170 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-10T09:42:13.214 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-10T09:42:13.219 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-10T09:42:13.233 INFO:teuthology.orchestra.run.smithi120.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-10T09:42:13.255 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-10T09:42:13.272 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-10T09:42:13.273 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-10T09:42:13.289 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-10T09:42:13.325 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-10T09:42:13.328 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-10T09:42:13.360 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-10T09:42:13.360 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-10T09:42:13.390 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-10T09:42:13.393 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-10T09:42:13.426 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-10T09:42:13.445 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-10T09:42:13.460 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-10T09:42:13.501 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-10T09:42:13.536 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-10T09:42:13.621 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-10T09:42:13.666 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-10T09:42:13.706 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-10T09:42:13.786 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-10T09:42:13.808 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-10T09:42:13.856 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-10T09:42:13.903 INFO:teuthology.orchestra.run.smithi120.stdout: Upgrading : librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T09:42:14.008 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T09:42:14.063 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T09:42:14.136 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T09:42:14.158 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-10T09:42:14.167 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 10/164 2024-09-10T09:42:14.224 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-10T09:42:14.258 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-10T09:42:14.315 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-10T09:42:14.364 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-10T09:42:14.370 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-10T09:42:14.392 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T09:42:14.433 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-10T09:42:14.486 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-10T09:42:14.496 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T09:42:14.518 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 14/164 2024-09-10T09:42:14.565 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-10T09:42:14.581 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 14/164 2024-09-10T09:42:14.603 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-10T09:42:14.605 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-10T09:42:14.612 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-10T09:42:14.627 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-10T09:42:14.646 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-10T09:42:14.752 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-10T09:42:14.796 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-10T09:42:14.817 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-10T09:42:14.862 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-10T09:42:14.923 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-10T09:42:14.940 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-10T09:42:14.944 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-10T09:42:14.973 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-10T09:42:15.044 INFO:teuthology.orchestra.run.smithi120.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-10T09:42:15.146 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T09:42:15.167 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-10T09:42:15.193 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-10T09:42:15.196 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-10T09:42:15.229 INFO:teuthology.orchestra.run.smithi120.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T09:42:15.232 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-10T09:42:15.273 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-10T09:42:15.336 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-10T09:42:15.450 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T09:42:15.563 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-10T09:42:15.656 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-10T09:42:15.693 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-10T09:42:15.728 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-10T09:42:15.793 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-10T09:42:15.837 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-10T09:42:15.886 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-10T09:42:15.916 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-10T09:42:15.938 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-10T09:42:15.971 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-10T09:42:15.999 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-10T09:42:16.042 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-10T09:42:16.068 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-10T09:42:16.121 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-10T09:42:16.147 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-10T09:42:16.152 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-10T09:42:16.187 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-10T09:42:16.263 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-10T09:42:16.295 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-10T09:42:16.314 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x 34/164 2024-09-10T09:42:16.358 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9 35/164 2024-09-10T09:42:16.387 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 36/164 2024-09-10T09:42:16.457 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-10T09:42:16.505 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-10T09:42:16.580 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-10T09:42:16.721 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-10T09:42:16.775 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-10T09:42:17.251 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-10T09:42:17.303 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-10T09:42:17.348 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-10T09:42:17.376 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-10T09:42:17.388 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-10T09:42:17.409 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-10T09:42:17.417 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-10T09:42:17.460 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-10T09:42:17.498 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 46/164 2024-09-10T09:42:17.540 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-10T09:42:17.702 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-10T09:42:17.729 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-10T09:42:17.759 INFO:teuthology.orchestra.run.smithi120.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-10T09:42:17.781 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 101/164 2024-09-10T09:42:17.814 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-10T09:42:17.880 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 101/164 2024-09-10T09:42:17.886 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-10T09:42:17.887 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-10T09:42:17.911 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 102/164 2024-09-10T09:42:17.919 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-10T09:42:17.947 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-10T09:42:17.955 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-10T09:42:17.978 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-10T09:42:18.003 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-10T09:42:18.014 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-10T09:42:18.038 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-10T09:42:18.047 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-10T09:42:18.090 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-10T09:42:18.140 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-10T09:42:18.277 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-10T09:42:18.332 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-10T09:42:18.373 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-10T09:42:18.392 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-10T09:42:18.428 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-10T09:42:18.442 INFO:teuthology.orchestra.run.smithi077.stdout: Upgrading : librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 101/164 2024-09-10T09:42:18.458 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 105/164 2024-09-10T09:42:18.486 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-10T09:42:18.521 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 105/164 2024-09-10T09:42:18.543 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 106/164 2024-09-10T09:42:18.563 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 101/164 2024-09-10T09:42:18.606 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 102/164 2024-09-10T09:42:18.655 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-10T09:42:18.752 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-10T09:42:18.790 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-10T09:42:18.823 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-09-10T09:42:18.847 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-10T09:42:18.928 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-10T09:42:19.061 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-10T09:42:19.107 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-10T09:42:19.144 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-10T09:42:19.173 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-10T09:42:19.178 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-10T09:42:19.204 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 105/164 2024-09-10T09:42:19.217 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-10T09:42:19.249 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-10T09:42:19.278 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 105/164 2024-09-10T09:42:19.279 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-10T09:42:19.305 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 106/164 2024-09-10T09:42:19.313 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-10T09:42:19.347 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-10T09:42:19.389 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-10T09:42:19.423 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-10T09:42:19.509 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-10T09:42:19.553 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-10T09:42:19.593 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-10T09:42:19.695 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-10T09:42:19.744 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-10T09:42:20.466 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-10T09:42:20.517 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-10T09:42:20.550 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-10T09:42:20.649 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-10T09:42:20.714 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-10T09:42:20.759 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-10T09:42:20.809 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T09:42:20.840 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T09:42:20.847 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-10T09:42:20.887 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-10T09:42:21.087 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-10T09:42:21.118 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-10T09:42:21.204 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-10T09:42:21.387 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T09:42:21.501 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 108/164 2024-09-10T09:42:21.598 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 108/164 2024-09-10T09:42:21.599 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-10T09:42:21.599 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-10T09:42:21.599 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-10T09:42:21.627 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 109/164 2024-09-10T09:42:21.837 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T09:42:21.847 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-10T09:42:21.885 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T09:42:21.967 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-10T09:42:22.511 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T09:42:22.555 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 108/164 2024-09-10T09:42:22.691 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 108/164 2024-09-10T09:42:22.691 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-10T09:42:22.691 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-10T09:42:22.691 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:42:22.739 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 109/164 2024-09-10T09:42:23.181 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-10T09:42:23.274 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-10T09:42:23.302 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-10T09:42:23.335 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-10T09:42:23.683 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-10T09:42:23.890 INFO:teuthology.orchestra.run.smithi120.stdout: Upgrading : librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 101/164 2024-09-10T09:42:23.974 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 101/164 2024-09-10T09:42:24.004 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 102/164 2024-09-10T09:42:24.048 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-10T09:42:24.533 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-10T09:42:24.569 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 105/164 2024-09-10T09:42:24.640 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 105/164 2024-09-10T09:42:24.679 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 106/164 2024-09-10T09:42:27.037 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T09:42:27.076 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T09:42:27.631 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T09:42:27.671 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 108/164 2024-09-10T09:42:27.754 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 108/164 2024-09-10T09:42:27.754 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-10T09:42:27.754 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-10T09:42:27.754 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T09:42:27.781 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 109/164 2024-09-10T09:42:33.495 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 109/164 2024-09-10T09:42:33.496 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /sys 2024-09-10T09:42:33.496 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /proc 2024-09-10T09:42:33.496 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /mnt 2024-09-10T09:42:33.496 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /var/tmp 2024-09-10T09:42:33.496 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /home 2024-09-10T09:42:33.496 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /root 2024-09-10T09:42:33.496 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /tmp 2024-09-10T09:42:33.496 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-10T09:42:33.702 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 110/164 2024-09-10T09:42:33.718 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 110/164 2024-09-10T09:42:33.719 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T09:42:33.719 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-10T09:42:33.719 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-10T09:42:35.182 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 111/164 2024-09-10T09:42:35.208 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 111/164 2024-09-10T09:42:35.209 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T09:42:35.209 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-10T09:42:35.209 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T09:42:35.209 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T09:42:35.209 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-10T09:42:35.241 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 109/164 2024-09-10T09:42:35.241 INFO:teuthology.orchestra.run.smithi077.stdout:skipping the directory /sys 2024-09-10T09:42:35.241 INFO:teuthology.orchestra.run.smithi077.stdout:skipping the directory /proc 2024-09-10T09:42:35.241 INFO:teuthology.orchestra.run.smithi077.stdout:skipping the directory /mnt 2024-09-10T09:42:35.241 INFO:teuthology.orchestra.run.smithi077.stdout:skipping the directory /var/tmp 2024-09-10T09:42:35.241 INFO:teuthology.orchestra.run.smithi077.stdout:skipping the directory /home 2024-09-10T09:42:35.241 INFO:teuthology.orchestra.run.smithi077.stdout:skipping the directory /root 2024-09-10T09:42:35.241 INFO:teuthology.orchestra.run.smithi077.stdout:skipping the directory /tmp 2024-09-10T09:42:35.241 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:42:35.433 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 112/164 2024-09-10T09:42:35.434 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 110/164 2024-09-10T09:42:35.456 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 112/164 2024-09-10T09:42:35.456 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T09:42:35.456 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-10T09:42:35.456 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T09:42:35.456 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T09:42:35.456 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-10T09:42:35.470 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 110/164 2024-09-10T09:42:35.470 INFO:teuthology.orchestra.run.smithi077.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T09:42:35.470 INFO:teuthology.orchestra.run.smithi077.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-10T09:42:35.470 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:42:35.899 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 113/164 2024-09-10T09:42:35.923 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 113/164 2024-09-10T09:42:35.923 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T09:42:35.924 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-10T09:42:35.924 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T09:42:35.924 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T09:42:35.924 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-10T09:42:36.029 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-10T09:42:36.078 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-10T09:42:36.132 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-10T09:42:36.318 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-10T09:42:36.366 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-10T09:42:36.436 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-10T09:42:36.525 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-10T09:42:36.662 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-10T09:42:36.708 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-10T09:42:36.748 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-10T09:42:36.776 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-10T09:42:37.105 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 111/164 2024-09-10T09:42:37.165 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 111/164 2024-09-10T09:42:37.166 INFO:teuthology.orchestra.run.smithi077.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T09:42:37.166 INFO:teuthology.orchestra.run.smithi077.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-10T09:42:37.166 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T09:42:37.166 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T09:42:37.166 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:42:37.230 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 125/164 2024-09-10T09:42:37.283 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 125/164 2024-09-10T09:42:37.314 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el 126/164 2024-09-10T09:42:37.376 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.e 127/164 2024-09-10T09:42:37.414 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 112/164 2024-09-10T09:42:37.414 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 128/164 2024-09-10T09:42:37.465 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 112/164 2024-09-10T09:42:37.465 INFO:teuthology.orchestra.run.smithi077.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T09:42:37.465 INFO:teuthology.orchestra.run.smithi077.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-10T09:42:37.465 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T09:42:37.466 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T09:42:37.466 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:42:37.939 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 113/164 2024-09-10T09:42:37.991 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 113/164 2024-09-10T09:42:37.991 INFO:teuthology.orchestra.run.smithi077.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T09:42:37.991 INFO:teuthology.orchestra.run.smithi077.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-10T09:42:37.991 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T09:42:37.991 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T09:42:37.991 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:42:38.114 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-10T09:42:38.183 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-10T09:42:38.254 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-10T09:42:38.457 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 128/164 2024-09-10T09:42:38.481 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-10T09:42:38.491 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 129/164 2024-09-10T09:42:38.543 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-10T09:42:38.636 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-10T09:42:38.745 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-10T09:42:38.919 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-10T09:42:38.977 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-10T09:42:39.024 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-10T09:42:39.050 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-10T09:42:39.454 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 129/164 2024-09-10T09:42:39.483 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 130/164 2024-09-10T09:42:39.585 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 130/164 2024-09-10T09:42:39.745 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9 131/164 2024-09-10T09:42:39.773 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 125/164 2024-09-10T09:42:39.776 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 132/164 2024-09-10T09:42:39.777 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 109/164 2024-09-10T09:42:39.777 INFO:teuthology.orchestra.run.smithi120.stdout:skipping the directory /sys 2024-09-10T09:42:39.778 INFO:teuthology.orchestra.run.smithi120.stdout:skipping the directory /proc 2024-09-10T09:42:39.778 INFO:teuthology.orchestra.run.smithi120.stdout:skipping the directory /mnt 2024-09-10T09:42:39.778 INFO:teuthology.orchestra.run.smithi120.stdout:skipping the directory /var/tmp 2024-09-10T09:42:39.778 INFO:teuthology.orchestra.run.smithi120.stdout:skipping the directory /home 2024-09-10T09:42:39.778 INFO:teuthology.orchestra.run.smithi120.stdout:skipping the directory /root 2024-09-10T09:42:39.778 INFO:teuthology.orchestra.run.smithi120.stdout:skipping the directory /tmp 2024-09-10T09:42:39.778 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T09:42:39.801 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 132/164 2024-09-10T09:42:39.801 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T09:42:39.801 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-10T09:42:39.801 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T09:42:39.801 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T09:42:39.801 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-10T09:42:39.816 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 125/164 2024-09-10T09:42:39.852 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 133/164 2024-09-10T09:42:39.857 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el 126/164 2024-09-10T09:42:39.864 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 133/164 2024-09-10T09:42:39.925 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.e 127/164 2024-09-10T09:42:39.955 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 110/164 2024-09-10T09:42:39.955 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 128/164 2024-09-10T09:42:39.961 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 134/164 2024-09-10T09:42:39.971 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 110/164 2024-09-10T09:42:39.971 INFO:teuthology.orchestra.run.smithi120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T09:42:39.971 INFO:teuthology.orchestra.run.smithi120.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-10T09:42:39.972 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T09:42:40.017 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 135/164 2024-09-10T09:42:40.952 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-10T09:42:40.988 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 137/164 2024-09-10T09:42:41.012 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 137/164 2024-09-10T09:42:41.013 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T09:42:41.013 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-10T09:42:41.013 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T09:42:41.013 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T09:42:41.013 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-10T09:42:41.055 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-305.g1cf819 138/164 2024-09-10T09:42:41.078 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-305.g1cf819 138/164 2024-09-10T09:42:41.078 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T09:42:41.078 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-10T09:42:41.078 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-10T09:42:41.123 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 139/164 2024-09-10T09:42:41.146 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 139/164 2024-09-10T09:42:41.146 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T09:42:41.146 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-10T09:42:41.147 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T09:42:41.147 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T09:42:41.147 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-10T09:42:41.169 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 128/164 2024-09-10T09:42:41.215 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 129/164 2024-09-10T09:42:41.457 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 140/164 2024-09-10T09:42:41.467 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 111/164 2024-09-10T09:42:41.480 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 140/164 2024-09-10T09:42:41.480 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T09:42:41.480 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-10T09:42:41.480 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T09:42:41.480 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T09:42:41.480 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-10T09:42:41.494 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 111/164 2024-09-10T09:42:41.494 INFO:teuthology.orchestra.run.smithi120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T09:42:41.494 INFO:teuthology.orchestra.run.smithi120.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-10T09:42:41.494 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T09:42:41.494 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T09:42:41.494 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T09:42:41.726 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 112/164 2024-09-10T09:42:41.750 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 112/164 2024-09-10T09:42:41.750 INFO:teuthology.orchestra.run.smithi120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T09:42:41.750 INFO:teuthology.orchestra.run.smithi120.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-10T09:42:41.750 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T09:42:41.750 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T09:42:41.750 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T09:42:42.185 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 113/164 2024-09-10T09:42:42.209 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 113/164 2024-09-10T09:42:42.209 INFO:teuthology.orchestra.run.smithi120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T09:42:42.209 INFO:teuthology.orchestra.run.smithi120.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-10T09:42:42.209 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T09:42:42.209 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T09:42:42.209 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T09:42:42.291 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 129/164 2024-09-10T09:42:42.323 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-10T09:42:42.324 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 130/164 2024-09-10T09:42:42.381 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-10T09:42:42.437 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-10T09:42:42.461 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 130/164 2024-09-10T09:42:42.623 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-10T09:42:42.636 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9 131/164 2024-09-10T09:42:42.668 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 132/164 2024-09-10T09:42:42.678 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-10T09:42:42.719 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 132/164 2024-09-10T09:42:42.719 INFO:teuthology.orchestra.run.smithi077.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T09:42:42.719 INFO:teuthology.orchestra.run.smithi077.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-10T09:42:42.719 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T09:42:42.719 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T09:42:42.719 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:42:42.756 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-10T09:42:42.768 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 133/164 2024-09-10T09:42:42.795 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 133/164 2024-09-10T09:42:42.845 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-10T09:42:42.901 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 134/164 2024-09-10T09:42:42.983 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 135/164 2024-09-10T09:42:42.983 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-10T09:42:43.029 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-10T09:42:43.069 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-10T09:42:43.105 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-10T09:42:43.500 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 125/164 2024-09-10T09:42:43.529 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 125/164 2024-09-10T09:42:43.561 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el 126/164 2024-09-10T09:42:43.648 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.e 127/164 2024-09-10T09:42:43.677 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 128/164 2024-09-10T09:42:43.988 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-10T09:42:44.021 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 137/164 2024-09-10T09:42:44.072 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 137/164 2024-09-10T09:42:44.073 INFO:teuthology.orchestra.run.smithi077.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T09:42:44.073 INFO:teuthology.orchestra.run.smithi077.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-10T09:42:44.073 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T09:42:44.073 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T09:42:44.073 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:42:44.130 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-305.g1cf819 138/164 2024-09-10T09:42:44.180 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-305.g1cf819 138/164 2024-09-10T09:42:44.180 INFO:teuthology.orchestra.run.smithi077.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T09:42:44.180 INFO:teuthology.orchestra.run.smithi077.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-10T09:42:44.180 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:42:44.239 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 139/164 2024-09-10T09:42:44.291 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 139/164 2024-09-10T09:42:44.291 INFO:teuthology.orchestra.run.smithi077.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T09:42:44.291 INFO:teuthology.orchestra.run.smithi077.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-10T09:42:44.291 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T09:42:44.291 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T09:42:44.291 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:42:44.623 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 140/164 2024-09-10T09:42:44.674 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 140/164 2024-09-10T09:42:44.674 INFO:teuthology.orchestra.run.smithi077.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T09:42:44.674 INFO:teuthology.orchestra.run.smithi077.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-10T09:42:44.674 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T09:42:44.674 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T09:42:44.674 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:42:44.724 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 128/164 2024-09-10T09:42:44.771 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 129/164 2024-09-10T09:42:45.734 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 129/164 2024-09-10T09:42:45.771 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 130/164 2024-09-10T09:42:45.875 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 130/164 2024-09-10T09:42:46.035 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 141/164 2024-09-10T09:42:46.044 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9 131/164 2024-09-10T09:42:46.082 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 132/164 2024-09-10T09:42:46.085 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 142/164 2024-09-10T09:42:46.107 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 132/164 2024-09-10T09:42:46.107 INFO:teuthology.orchestra.run.smithi120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T09:42:46.107 INFO:teuthology.orchestra.run.smithi120.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-10T09:42:46.107 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T09:42:46.107 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T09:42:46.107 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T09:42:46.133 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 143/164 2024-09-10T09:42:46.158 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 133/164 2024-09-10T09:42:46.166 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-10T09:42:46.170 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 133/164 2024-09-10T09:42:46.193 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-10T09:42:46.267 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 134/164 2024-09-10T09:42:46.315 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 135/164 2024-09-10T09:42:46.377 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 146/164 2024-09-10T09:42:46.423 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-10T09:42:46.453 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_6 148/164 2024-09-10T09:42:46.482 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-10T09:42:46.529 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-10T09:42:47.173 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T09:42:47.210 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T09:42:47.230 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T09:42:47.242 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-10T09:42:47.286 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 137/164 2024-09-10T09:42:47.310 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 137/164 2024-09-10T09:42:47.310 INFO:teuthology.orchestra.run.smithi120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T09:42:47.310 INFO:teuthology.orchestra.run.smithi120.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-10T09:42:47.310 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T09:42:47.310 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T09:42:47.310 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T09:42:47.353 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-305.g1cf819 138/164 2024-09-10T09:42:47.376 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-305.g1cf819 138/164 2024-09-10T09:42:47.376 INFO:teuthology.orchestra.run.smithi120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T09:42:47.376 INFO:teuthology.orchestra.run.smithi120.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-10T09:42:47.376 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T09:42:47.429 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 139/164 2024-09-10T09:42:47.452 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 139/164 2024-09-10T09:42:47.452 INFO:teuthology.orchestra.run.smithi120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T09:42:47.452 INFO:teuthology.orchestra.run.smithi120.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-10T09:42:47.452 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T09:42:47.452 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T09:42:47.453 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T09:42:47.763 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 140/164 2024-09-10T09:42:47.787 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 140/164 2024-09-10T09:42:47.787 INFO:teuthology.orchestra.run.smithi120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T09:42:47.787 INFO:teuthology.orchestra.run.smithi120.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-10T09:42:47.788 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T09:42:47.788 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T09:42:47.788 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T09:42:48.043 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 152/164 2024-09-10T09:42:48.122 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-10T09:42:48.163 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-10T09:42:48.243 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-10T09:42:48.275 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-10T09:42:48.301 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-10T09:42:48.335 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-10T09:42:48.335 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-10T09:42:48.419 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-10T09:42:48.419 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-10T09:42:48.503 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-10T09:42:48.528 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-10T09:42:48.533 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T09:42:48.534 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T09:42:48.722 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T09:42:48.722 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-10T09:42:48.814 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-10T09:42:48.878 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T09:42:49.653 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 141/164 2024-09-10T09:42:49.695 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 142/164 2024-09-10T09:42:49.738 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 143/164 2024-09-10T09:42:49.775 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-10T09:42:49.819 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-10T09:42:50.040 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 146/164 2024-09-10T09:42:50.091 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-10T09:42:50.120 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_6 148/164 2024-09-10T09:42:50.152 INFO:teuthology.orchestra.run.smithi077.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-10T09:42:50.196 INFO:teuthology.orchestra.run.smithi077.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-10T09:42:50.912 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T09:42:50.961 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T09:42:51.002 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T09:42:52.263 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 152/164 2024-09-10T09:42:52.357 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-10T09:42:52.376 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 141/164 2024-09-10T09:42:52.409 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-10T09:42:52.417 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 142/164 2024-09-10T09:42:52.458 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 143/164 2024-09-10T09:42:52.491 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-10T09:42:52.532 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-10T09:42:52.532 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-10T09:42:52.551 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-10T09:42:52.576 INFO:teuthology.orchestra.run.smithi077.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-10T09:42:52.622 INFO:teuthology.orchestra.run.smithi077.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-10T09:42:52.622 INFO:teuthology.orchestra.run.smithi077.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-10T09:42:52.708 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 146/164 2024-09-10T09:42:52.729 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-10T09:42:52.729 INFO:teuthology.orchestra.run.smithi077.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-10T09:42:52.748 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-10T09:42:52.778 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_6 148/164 2024-09-10T09:42:52.825 INFO:teuthology.orchestra.run.smithi120.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-10T09:42:52.855 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-10T09:42:52.863 INFO:teuthology.orchestra.run.smithi120.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-10T09:42:52.879 INFO:teuthology.orchestra.run.smithi077.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-10T09:42:52.895 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T09:42:52.895 INFO:teuthology.orchestra.run.smithi077.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T09:42:53.141 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T09:42:53.141 INFO:teuthology.orchestra.run.smithi077.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-10T09:42:53.233 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-10T09:42:53.306 INFO:teuthology.orchestra.run.smithi077.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T09:42:53.471 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T09:42:53.511 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T09:42:53.533 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T09:42:53.544 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T09:42:53.544 INFO:teuthology.orchestra.run.smithi077.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-09-10T09:42:53.544 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:42:54.384 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 152/164 2024-09-10T09:42:54.468 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-10T09:42:54.500 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T09:42:54.500 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 1/164 2024-09-10T09:42:54.500 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 2/164 2024-09-10T09:42:54.500 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 3/164 2024-09-10T09:42:54.500 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 4/164 2024-09-10T09:42:54.500 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-305.g1cf819 5/164 2024-09-10T09:42:54.500 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 6/164 2024-09-10T09:42:54.501 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 7/164 2024-09-10T09:42:54.501 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T09:42:54.501 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T09:42:54.501 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 10/164 2024-09-10T09:42:54.501 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 11/164 2024-09-10T09:42:54.501 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 12/164 2024-09-10T09:42:54.501 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T09:42:54.501 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 14/164 2024-09-10T09:42:54.501 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_6 15/164 2024-09-10T09:42:54.501 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 16/164 2024-09-10T09:42:54.501 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 17/164 2024-09-10T09:42:54.501 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 18/164 2024-09-10T09:42:54.501 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 19/164 2024-09-10T09:42:54.502 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 20/164 2024-09-10T09:42:54.502 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9 21/164 2024-09-10T09:42:54.502 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x 22/164 2024-09-10T09:42:54.502 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 23/164 2024-09-10T09:42:54.502 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 24/164 2024-09-10T09:42:54.502 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 25/164 2024-09-10T09:42:54.502 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 26/164 2024-09-10T09:42:54.502 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 27/164 2024-09-10T09:42:54.502 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 28/164 2024-09-10T09:42:54.502 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 29/164 2024-09-10T09:42:54.509 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-10T09:42:54.515 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.e 30/164 2024-09-10T09:42:54.515 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 31/164 2024-09-10T09:42:54.515 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 32/164 2024-09-10T09:42:54.515 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 33/164 2024-09-10T09:42:54.515 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9 34/164 2024-09-10T09:42:54.515 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 35/164 2024-09-10T09:42:54.515 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el 36/164 2024-09-10T09:42:54.515 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 37/164 2024-09-10T09:42:54.515 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 38/164 2024-09-10T09:42:54.515 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 39/164 2024-09-10T09:42:54.516 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-10T09:42:54.516 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-10T09:42:54.516 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-10T09:42:54.516 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-10T09:42:54.516 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-10T09:42:54.516 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-10T09:42:54.516 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-10T09:42:54.516 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-10T09:42:54.516 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-10T09:42:54.516 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-10T09:42:54.516 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-10T09:42:54.516 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-10T09:42:54.516 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-10T09:42:54.517 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-10T09:42:54.517 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-10T09:42:54.517 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-10T09:42:54.517 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-10T09:42:54.517 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-10T09:42:54.517 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-10T09:42:54.517 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-10T09:42:54.517 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-10T09:42:54.517 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-10T09:42:54.517 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-10T09:42:54.517 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-10T09:42:54.517 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-10T09:42:54.517 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-10T09:42:54.518 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-10T09:42:54.518 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-10T09:42:54.518 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-10T09:42:54.518 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-10T09:42:54.518 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-10T09:42:54.518 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-10T09:42:54.518 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-10T09:42:54.518 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-10T09:42:54.518 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-10T09:42:54.518 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-10T09:42:54.518 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-10T09:42:54.518 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-10T09:42:54.519 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-10T09:42:54.519 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-10T09:42:54.519 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-10T09:42:54.519 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-10T09:42:54.519 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-10T09:42:54.519 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-10T09:42:54.519 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-10T09:42:54.519 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-10T09:42:54.519 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-10T09:42:54.519 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-10T09:42:54.519 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-10T09:42:54.519 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-10T09:42:54.519 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-10T09:42:54.519 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-10T09:42:54.520 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-10T09:42:54.520 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-10T09:42:54.520 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-10T09:42:54.520 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-10T09:42:54.520 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-10T09:42:54.520 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-10T09:42:54.520 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-10T09:42:54.520 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-10T09:42:54.520 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-10T09:42:54.520 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-10T09:42:54.520 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-10T09:42:54.520 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-10T09:42:54.520 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-10T09:42:54.520 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-10T09:42:54.521 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-10T09:42:54.521 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/164 2024-09-10T09:42:54.521 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-10T09:42:54.521 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-10T09:42:54.521 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-10T09:42:54.521 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-10T09:42:54.521 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-10T09:42:54.521 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-10T09:42:54.521 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-10T09:42:54.521 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-10T09:42:54.521 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-10T09:42:54.521 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-10T09:42:54.521 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-10T09:42:54.521 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-10T09:42:54.522 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-10T09:42:54.522 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-10T09:42:54.522 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-10T09:42:54.522 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-10T09:42:54.522 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-10T09:42:54.522 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-10T09:42:54.522 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-10T09:42:54.522 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-10T09:42:54.522 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-10T09:42:54.522 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-10T09:42:54.522 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-10T09:42:54.522 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-10T09:42:54.522 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-10T09:42:54.522 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-10T09:42:54.523 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-10T09:42:54.523 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-10T09:42:54.523 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-10T09:42:54.523 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-10T09:42:54.523 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-10T09:42:54.523 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-10T09:42:54.523 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-10T09:42:54.523 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-10T09:42:54.523 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-10T09:42:54.523 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-10T09:42:54.523 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-10T09:42:54.523 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-10T09:42:54.523 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-10T09:42:54.524 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-10T09:42:54.524 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-10T09:42:54.524 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 149/164 2024-09-10T09:42:54.524 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-10T09:42:54.524 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 151/164 2024-09-10T09:42:54.524 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-10T09:42:54.524 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-10T09:42:54.524 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-10T09:42:54.524 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-10T09:42:54.524 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-10T09:42:54.524 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-10T09:42:54.524 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-10T09:42:54.524 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-10T09:42:54.524 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-10T09:42:54.525 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-10T09:42:54.525 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-10T09:42:54.525 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-10T09:42:54.589 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-10T09:42:54.621 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-10T09:42:54.647 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-10T09:42:54.673 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-10T09:42:54.673 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-10T09:42:54.740 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-10T09:42:54.740 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-10T09:42:54.807 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-10T09:42:54.841 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-10T09:42:54.847 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T09:42:54.847 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T09:42:55.035 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T09:42:55.035 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-10T09:42:55.119 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-10T09:42:55.183 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T09:42:57.429 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T09:42:57.429 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-10T09:42:57.429 INFO:teuthology.orchestra.run.smithi134.stdout:Upgraded: 2024-09-10T09:42:57.429 INFO:teuthology.orchestra.run.smithi134.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-10T09:42:57.429 INFO:teuthology.orchestra.run.smithi134.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-10T09:42:57.429 INFO:teuthology.orchestra.run.smithi134.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-10T09:42:57.429 INFO:teuthology.orchestra.run.smithi134.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-10T09:42:57.429 INFO:teuthology.orchestra.run.smithi134.stdout: librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:42:57.429 INFO:teuthology.orchestra.run.smithi134.stdout: librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:42:57.429 INFO:teuthology.orchestra.run.smithi134.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-10T09:42:57.429 INFO:teuthology.orchestra.run.smithi134.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-10T09:42:57.430 INFO:teuthology.orchestra.run.smithi134.stdout:Installed: 2024-09-10T09:42:57.430 INFO:teuthology.orchestra.run.smithi134.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-10T09:42:57.430 INFO:teuthology.orchestra.run.smithi134.stdout: automake-1.16.2-8.el9.noarch 2024-09-10T09:42:57.430 INFO:teuthology.orchestra.run.smithi134.stdout: bison-3.7.4-5.el9.x86_64 2024-09-10T09:42:57.430 INFO:teuthology.orchestra.run.smithi134.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-10T09:42:57.430 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:42:57.430 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:42:57.430 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:42:57.430 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:42:57.430 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T09:42:57.430 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-immutable-object-cache-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:42:57.430 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:42:57.430 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:42:57.430 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T09:42:57.431 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T09:42:57.431 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T09:42:57.431 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T09:42:57.431 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T09:42:57.431 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:42:57.431 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:42:57.431 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T09:42:57.431 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:42:57.431 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:42:57.431 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:42:57.431 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T09:42:57.431 INFO:teuthology.orchestra.run.smithi134.stdout: cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T09:42:57.431 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:42:57.432 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:42:57.432 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T09:42:57.432 INFO:teuthology.orchestra.run.smithi134.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-10T09:42:57.432 INFO:teuthology.orchestra.run.smithi134.stdout: flex-2.6.4-9.el9.x86_64 2024-09-10T09:42:57.432 INFO:teuthology.orchestra.run.smithi134.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-10T09:42:57.432 INFO:teuthology.orchestra.run.smithi134.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-10T09:42:57.432 INFO:teuthology.orchestra.run.smithi134.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-10T09:42:57.432 INFO:teuthology.orchestra.run.smithi134.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-10T09:42:57.432 INFO:teuthology.orchestra.run.smithi134.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-10T09:42:57.432 INFO:teuthology.orchestra.run.smithi134.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-10T09:42:57.432 INFO:teuthology.orchestra.run.smithi134.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-10T09:42:57.432 INFO:teuthology.orchestra.run.smithi134.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-10T09:42:57.432 INFO:teuthology.orchestra.run.smithi134.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-10T09:42:57.433 INFO:teuthology.orchestra.run.smithi134.stdout: libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:42:57.433 INFO:teuthology.orchestra.run.smithi134.stdout: libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:42:57.433 INFO:teuthology.orchestra.run.smithi134.stdout: libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:42:57.433 INFO:teuthology.orchestra.run.smithi134.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-10T09:42:57.433 INFO:teuthology.orchestra.run.smithi134.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-10T09:42:57.433 INFO:teuthology.orchestra.run.smithi134.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-10T09:42:57.433 INFO:teuthology.orchestra.run.smithi134.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-10T09:42:57.433 INFO:teuthology.orchestra.run.smithi134.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-10T09:42:57.433 INFO:teuthology.orchestra.run.smithi134.stdout: librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:42:57.433 INFO:teuthology.orchestra.run.smithi134.stdout: libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:42:57.433 INFO:teuthology.orchestra.run.smithi134.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-10T09:42:57.433 INFO:teuthology.orchestra.run.smithi134.stdout: librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:42:57.433 INFO:teuthology.orchestra.run.smithi134.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-10T09:42:57.433 INFO:teuthology.orchestra.run.smithi134.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-10T09:42:57.434 INFO:teuthology.orchestra.run.smithi134.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-10T09:42:57.434 INFO:teuthology.orchestra.run.smithi134.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-10T09:42:57.434 INFO:teuthology.orchestra.run.smithi134.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-10T09:42:57.434 INFO:teuthology.orchestra.run.smithi134.stdout: lua-5.4.4-4.el9.x86_64 2024-09-10T09:42:57.434 INFO:teuthology.orchestra.run.smithi134.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-10T09:42:57.434 INFO:teuthology.orchestra.run.smithi134.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-10T09:42:57.434 INFO:teuthology.orchestra.run.smithi134.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-10T09:42:57.434 INFO:teuthology.orchestra.run.smithi134.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-10T09:42:57.434 INFO:teuthology.orchestra.run.smithi134.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-10T09:42:57.434 INFO:teuthology.orchestra.run.smithi134.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-10T09:42:57.434 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-13.16-1.el9.x86_64 2024-09-10T09:42:57.434 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-10T09:42:57.434 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-10T09:42:57.435 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-10T09:42:57.435 INFO:teuthology.orchestra.run.smithi134.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-10T09:42:57.435 INFO:teuthology.orchestra.run.smithi134.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-10T09:42:57.435 INFO:teuthology.orchestra.run.smithi134.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-10T09:42:57.435 INFO:teuthology.orchestra.run.smithi134.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-10T09:42:57.435 INFO:teuthology.orchestra.run.smithi134.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-10T09:42:57.435 INFO:teuthology.orchestra.run.smithi134.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-10T09:42:57.435 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-10T09:42:57.435 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:42:57.435 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:42:57.435 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:42:57.435 INFO:teuthology.orchestra.run.smithi134.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-10T09:42:57.435 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-10T09:42:57.435 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-10T09:42:57.436 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-10T09:42:57.436 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-10T09:42:57.436 INFO:teuthology.orchestra.run.smithi134.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-10T09:42:57.436 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-10T09:42:57.436 INFO:teuthology.orchestra.run.smithi134.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-10T09:42:57.436 INFO:teuthology.orchestra.run.smithi134.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-10T09:42:57.436 INFO:teuthology.orchestra.run.smithi134.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-10T09:42:57.436 INFO:teuthology.orchestra.run.smithi134.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-10T09:42:57.436 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-10T09:42:57.436 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-10T09:42:57.436 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-10T09:42:57.436 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-10T09:42:57.436 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-10T09:42:57.437 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-10T09:42:57.437 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-10T09:42:57.437 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-10T09:42:57.437 INFO:teuthology.orchestra.run.smithi134.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-10T09:42:57.437 INFO:teuthology.orchestra.run.smithi134.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-10T09:42:57.437 INFO:teuthology.orchestra.run.smithi134.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-10T09:42:57.437 INFO:teuthology.orchestra.run.smithi134.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-10T09:42:57.437 INFO:teuthology.orchestra.run.smithi134.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-10T09:42:57.437 INFO:teuthology.orchestra.run.smithi134.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-10T09:42:57.437 INFO:teuthology.orchestra.run.smithi134.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-10T09:42:57.437 INFO:teuthology.orchestra.run.smithi134.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-10T09:42:57.437 INFO:teuthology.orchestra.run.smithi134.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-10T09:42:57.437 INFO:teuthology.orchestra.run.smithi134.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-10T09:42:57.437 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-10T09:42:57.438 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ply-3.11-14.el9.noarch 2024-09-10T09:42:57.438 INFO:teuthology.orchestra.run.smithi134.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-10T09:42:57.438 INFO:teuthology.orchestra.run.smithi134.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-10T09:42:57.438 INFO:teuthology.orchestra.run.smithi134.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-10T09:42:57.438 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-10T09:42:57.438 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-10T09:42:57.438 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-10T09:42:57.438 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-10T09:42:57.438 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-10T09:42:57.438 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-10T09:42:57.438 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:42:57.438 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:42:57.438 INFO:teuthology.orchestra.run.smithi134.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-10T09:42:57.438 INFO:teuthology.orchestra.run.smithi134.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-10T09:42:57.439 INFO:teuthology.orchestra.run.smithi134.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-10T09:42:57.439 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:42:57.439 INFO:teuthology.orchestra.run.smithi134.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-10T09:42:57.439 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-10T09:42:57.439 INFO:teuthology.orchestra.run.smithi134.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-10T09:42:57.439 INFO:teuthology.orchestra.run.smithi134.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-10T09:42:57.439 INFO:teuthology.orchestra.run.smithi134.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-10T09:42:57.439 INFO:teuthology.orchestra.run.smithi134.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-10T09:42:57.439 INFO:teuthology.orchestra.run.smithi134.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-10T09:42:57.439 INFO:teuthology.orchestra.run.smithi134.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-10T09:42:57.439 INFO:teuthology.orchestra.run.smithi134.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-10T09:42:57.439 INFO:teuthology.orchestra.run.smithi134.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-10T09:42:57.439 INFO:teuthology.orchestra.run.smithi134.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-10T09:42:57.439 INFO:teuthology.orchestra.run.smithi134.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-10T09:42:57.440 INFO:teuthology.orchestra.run.smithi134.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-10T09:42:57.440 INFO:teuthology.orchestra.run.smithi134.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-10T09:42:57.440 INFO:teuthology.orchestra.run.smithi134.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-10T09:42:57.440 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:42:57.440 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:42:57.440 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:42:57.440 INFO:teuthology.orchestra.run.smithi134.stdout: re2-1:20211101-20.el9.x86_64 2024-09-10T09:42:57.440 INFO:teuthology.orchestra.run.smithi134.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-10T09:42:57.440 INFO:teuthology.orchestra.run.smithi134.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-10T09:42:57.440 INFO:teuthology.orchestra.run.smithi134.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-10T09:42:57.440 INFO:teuthology.orchestra.run.smithi134.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-10T09:42:57.440 INFO:teuthology.orchestra.run.smithi134.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-10T09:42:57.441 INFO:teuthology.orchestra.run.smithi134.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-10T09:42:57.441 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-10T09:42:57.441 INFO:teuthology.orchestra.run.smithi134.stdout:Complete! 2024-09-10T09:42:57.878 DEBUG:teuthology.parallel:result is None 2024-09-10T09:42:59.340 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 1/164 2024-09-10T09:42:59.340 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 2/164 2024-09-10T09:42:59.340 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 3/164 2024-09-10T09:42:59.340 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 4/164 2024-09-10T09:42:59.340 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-305.g1cf819 5/164 2024-09-10T09:42:59.340 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 6/164 2024-09-10T09:42:59.341 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 7/164 2024-09-10T09:42:59.341 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T09:42:59.341 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T09:42:59.341 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 10/164 2024-09-10T09:42:59.341 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 11/164 2024-09-10T09:42:59.341 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 12/164 2024-09-10T09:42:59.341 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T09:42:59.341 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 14/164 2024-09-10T09:42:59.341 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_6 15/164 2024-09-10T09:42:59.341 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 16/164 2024-09-10T09:42:59.341 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 17/164 2024-09-10T09:42:59.341 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 18/164 2024-09-10T09:42:59.341 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 19/164 2024-09-10T09:42:59.341 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 20/164 2024-09-10T09:42:59.342 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9 21/164 2024-09-10T09:42:59.342 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x 22/164 2024-09-10T09:42:59.342 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 23/164 2024-09-10T09:42:59.342 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 24/164 2024-09-10T09:42:59.342 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 25/164 2024-09-10T09:42:59.342 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 26/164 2024-09-10T09:42:59.342 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 27/164 2024-09-10T09:42:59.342 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 28/164 2024-09-10T09:42:59.342 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 29/164 2024-09-10T09:42:59.342 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.e 30/164 2024-09-10T09:42:59.342 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 31/164 2024-09-10T09:42:59.342 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 32/164 2024-09-10T09:42:59.342 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 33/164 2024-09-10T09:42:59.342 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9 34/164 2024-09-10T09:42:59.343 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 35/164 2024-09-10T09:42:59.343 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el 36/164 2024-09-10T09:42:59.343 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 37/164 2024-09-10T09:42:59.343 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 38/164 2024-09-10T09:42:59.343 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 39/164 2024-09-10T09:42:59.343 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-10T09:42:59.343 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-10T09:42:59.343 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-10T09:42:59.344 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-10T09:42:59.344 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-10T09:42:59.344 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-10T09:42:59.344 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-10T09:42:59.344 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-10T09:42:59.344 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-10T09:42:59.344 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-10T09:42:59.344 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-10T09:42:59.344 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-10T09:42:59.344 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-10T09:42:59.344 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-10T09:42:59.344 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-10T09:42:59.344 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-10T09:42:59.344 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-10T09:42:59.345 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-10T09:42:59.345 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-10T09:42:59.345 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-10T09:42:59.345 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-10T09:42:59.345 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-10T09:42:59.345 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-10T09:42:59.345 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-10T09:42:59.345 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-10T09:42:59.345 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-10T09:42:59.345 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-10T09:42:59.345 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-10T09:42:59.345 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-10T09:42:59.345 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-10T09:42:59.346 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-10T09:42:59.346 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-10T09:42:59.346 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-10T09:42:59.346 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-10T09:42:59.346 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-10T09:42:59.346 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-10T09:42:59.346 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-10T09:42:59.346 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-10T09:42:59.346 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-10T09:42:59.346 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-10T09:42:59.346 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-10T09:42:59.346 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-10T09:42:59.346 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-10T09:42:59.346 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-10T09:42:59.346 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-10T09:42:59.347 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-10T09:42:59.347 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-10T09:42:59.347 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-10T09:42:59.347 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-10T09:42:59.347 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-10T09:42:59.347 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-10T09:42:59.347 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-10T09:42:59.347 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-10T09:42:59.347 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-10T09:42:59.347 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-10T09:42:59.347 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-10T09:42:59.347 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-10T09:42:59.347 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-10T09:42:59.347 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-10T09:42:59.348 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-10T09:42:59.348 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-10T09:42:59.348 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-10T09:42:59.348 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-10T09:42:59.348 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-10T09:42:59.348 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-10T09:42:59.348 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-10T09:42:59.348 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-10T09:42:59.348 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/164 2024-09-10T09:42:59.348 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-10T09:42:59.348 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-10T09:42:59.348 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-10T09:42:59.348 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-10T09:42:59.349 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-10T09:42:59.349 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-10T09:42:59.349 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-10T09:42:59.349 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-10T09:42:59.349 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-10T09:42:59.349 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-10T09:42:59.349 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-10T09:42:59.349 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-10T09:42:59.349 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-10T09:42:59.349 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-10T09:42:59.349 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-10T09:42:59.349 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-10T09:42:59.349 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-10T09:42:59.349 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-10T09:42:59.350 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-10T09:42:59.350 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-10T09:42:59.350 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-10T09:42:59.350 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-10T09:42:59.350 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-10T09:42:59.350 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-10T09:42:59.350 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-10T09:42:59.350 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-10T09:42:59.350 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-10T09:42:59.350 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-10T09:42:59.350 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-10T09:42:59.350 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-10T09:42:59.351 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-10T09:42:59.351 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-10T09:42:59.351 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-10T09:42:59.351 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-10T09:42:59.351 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-10T09:42:59.351 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-10T09:42:59.351 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-10T09:42:59.351 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-10T09:42:59.351 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-10T09:42:59.351 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-10T09:42:59.351 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-10T09:42:59.351 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 149/164 2024-09-10T09:42:59.351 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-10T09:42:59.351 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 151/164 2024-09-10T09:42:59.351 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-10T09:42:59.351 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-10T09:42:59.352 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-10T09:42:59.352 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-10T09:42:59.352 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-10T09:42:59.352 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-10T09:42:59.352 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-10T09:42:59.352 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-10T09:42:59.352 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-10T09:42:59.352 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-10T09:42:59.352 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-10T09:42:59.352 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-10T09:43:01.030 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T09:43:01.031 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 1/164 2024-09-10T09:43:01.031 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 2/164 2024-09-10T09:43:01.031 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 3/164 2024-09-10T09:43:01.031 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 4/164 2024-09-10T09:43:01.031 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-305.g1cf819 5/164 2024-09-10T09:43:01.031 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 6/164 2024-09-10T09:43:01.031 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 7/164 2024-09-10T09:43:01.031 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T09:43:01.031 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T09:43:01.031 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 10/164 2024-09-10T09:43:01.031 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 11/164 2024-09-10T09:43:01.032 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 12/164 2024-09-10T09:43:01.032 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T09:43:01.032 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 14/164 2024-09-10T09:43:01.032 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_6 15/164 2024-09-10T09:43:01.032 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 16/164 2024-09-10T09:43:01.032 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 17/164 2024-09-10T09:43:01.032 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 18/164 2024-09-10T09:43:01.032 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 19/164 2024-09-10T09:43:01.032 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 20/164 2024-09-10T09:43:01.032 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9 21/164 2024-09-10T09:43:01.032 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x 22/164 2024-09-10T09:43:01.032 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 23/164 2024-09-10T09:43:01.032 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 24/164 2024-09-10T09:43:01.033 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 25/164 2024-09-10T09:43:01.033 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 26/164 2024-09-10T09:43:01.033 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 27/164 2024-09-10T09:43:01.044 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 28/164 2024-09-10T09:43:01.045 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 29/164 2024-09-10T09:43:01.045 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.e 30/164 2024-09-10T09:43:01.045 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 31/164 2024-09-10T09:43:01.045 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 32/164 2024-09-10T09:43:01.045 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 33/164 2024-09-10T09:43:01.045 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9 34/164 2024-09-10T09:43:01.045 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 35/164 2024-09-10T09:43:01.045 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el 36/164 2024-09-10T09:43:01.045 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 37/164 2024-09-10T09:43:01.045 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 38/164 2024-09-10T09:43:01.045 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 39/164 2024-09-10T09:43:01.045 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-10T09:43:01.046 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-10T09:43:01.046 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-10T09:43:01.046 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-10T09:43:01.046 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-10T09:43:01.046 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-10T09:43:01.046 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-10T09:43:01.046 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-10T09:43:01.046 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-10T09:43:01.046 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-10T09:43:01.046 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-10T09:43:01.046 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-10T09:43:01.046 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-10T09:43:01.046 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-10T09:43:01.047 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-10T09:43:01.047 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-10T09:43:01.047 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-10T09:43:01.047 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-10T09:43:01.047 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-10T09:43:01.047 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-10T09:43:01.047 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-10T09:43:01.047 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-10T09:43:01.047 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-10T09:43:01.047 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-10T09:43:01.047 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-10T09:43:01.047 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-10T09:43:01.047 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-10T09:43:01.047 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-10T09:43:01.048 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-10T09:43:01.048 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-10T09:43:01.048 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-10T09:43:01.048 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-10T09:43:01.048 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-10T09:43:01.048 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-10T09:43:01.048 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-10T09:43:01.048 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-10T09:43:01.048 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-10T09:43:01.048 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-10T09:43:01.048 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-10T09:43:01.048 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-10T09:43:01.049 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-10T09:43:01.049 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-10T09:43:01.049 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-10T09:43:01.049 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-10T09:43:01.049 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-10T09:43:01.049 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-10T09:43:01.049 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-10T09:43:01.050 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-10T09:43:01.050 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-10T09:43:01.050 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-10T09:43:01.050 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-10T09:43:01.050 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-10T09:43:01.050 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-10T09:43:01.052 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-10T09:43:01.052 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-10T09:43:01.052 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-10T09:43:01.052 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-10T09:43:01.052 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-10T09:43:01.052 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-10T09:43:01.052 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-10T09:43:01.052 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-10T09:43:01.052 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-10T09:43:01.052 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-10T09:43:01.052 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-10T09:43:01.052 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-10T09:43:01.053 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-10T09:43:01.053 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-10T09:43:01.053 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/164 2024-09-10T09:43:01.053 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-10T09:43:01.053 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-10T09:43:01.053 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-10T09:43:01.053 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-10T09:43:01.053 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-10T09:43:01.053 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-10T09:43:01.053 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-10T09:43:01.053 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-10T09:43:01.053 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-10T09:43:01.053 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-10T09:43:01.053 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-10T09:43:01.054 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-10T09:43:01.054 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-10T09:43:01.054 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-10T09:43:01.054 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-10T09:43:01.054 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-10T09:43:01.054 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-10T09:43:01.054 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-10T09:43:01.054 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-10T09:43:01.054 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-10T09:43:01.054 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-10T09:43:01.054 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-10T09:43:01.054 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-10T09:43:01.054 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-10T09:43:01.055 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-10T09:43:01.055 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-10T09:43:01.055 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-10T09:43:01.055 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-10T09:43:01.055 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-10T09:43:01.055 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-10T09:43:01.055 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-10T09:43:01.055 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-10T09:43:01.055 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-10T09:43:01.055 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-10T09:43:01.055 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-10T09:43:01.055 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-10T09:43:01.055 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-10T09:43:01.055 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-10T09:43:01.056 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-10T09:43:01.056 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-10T09:43:01.056 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-10T09:43:01.056 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 149/164 2024-09-10T09:43:01.056 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-10T09:43:01.056 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 151/164 2024-09-10T09:43:01.056 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-10T09:43:01.056 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-10T09:43:01.056 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-10T09:43:01.056 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-10T09:43:01.056 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-10T09:43:01.056 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-10T09:43:01.056 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-10T09:43:01.056 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-10T09:43:01.057 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-10T09:43:01.057 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-10T09:43:01.057 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-10T09:43:01.057 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-10T09:43:01.519 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T09:43:01.519 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:43:01.519 INFO:teuthology.orchestra.run.smithi077.stdout:Upgraded: 2024-09-10T09:43:01.519 INFO:teuthology.orchestra.run.smithi077.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-10T09:43:01.519 INFO:teuthology.orchestra.run.smithi077.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-10T09:43:01.520 INFO:teuthology.orchestra.run.smithi077.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-10T09:43:01.520 INFO:teuthology.orchestra.run.smithi077.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-10T09:43:01.520 INFO:teuthology.orchestra.run.smithi077.stdout: librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:43:01.520 INFO:teuthology.orchestra.run.smithi077.stdout: librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:43:01.520 INFO:teuthology.orchestra.run.smithi077.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-10T09:43:01.520 INFO:teuthology.orchestra.run.smithi077.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-10T09:43:01.520 INFO:teuthology.orchestra.run.smithi077.stdout:Installed: 2024-09-10T09:43:01.520 INFO:teuthology.orchestra.run.smithi077.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-10T09:43:01.520 INFO:teuthology.orchestra.run.smithi077.stdout: automake-1.16.2-8.el9.noarch 2024-09-10T09:43:01.520 INFO:teuthology.orchestra.run.smithi077.stdout: bison-3.7.4-5.el9.x86_64 2024-09-10T09:43:01.520 INFO:teuthology.orchestra.run.smithi077.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-10T09:43:01.520 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:43:01.520 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:43:01.521 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:43:01.521 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:43:01.521 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T09:43:01.521 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-immutable-object-cache-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:43:01.521 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:43:01.521 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:43:01.521 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T09:43:01.521 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T09:43:01.521 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T09:43:01.521 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T09:43:01.521 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T09:43:01.521 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:43:01.521 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:43:01.522 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T09:43:01.522 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:43:01.522 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:43:01.522 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:43:01.522 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T09:43:01.522 INFO:teuthology.orchestra.run.smithi077.stdout: cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T09:43:01.522 INFO:teuthology.orchestra.run.smithi077.stdout: cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:43:01.522 INFO:teuthology.orchestra.run.smithi077.stdout: cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:43:01.522 INFO:teuthology.orchestra.run.smithi077.stdout: cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T09:43:01.522 INFO:teuthology.orchestra.run.smithi077.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-10T09:43:01.522 INFO:teuthology.orchestra.run.smithi077.stdout: flex-2.6.4-9.el9.x86_64 2024-09-10T09:43:01.522 INFO:teuthology.orchestra.run.smithi077.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-10T09:43:01.522 INFO:teuthology.orchestra.run.smithi077.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-10T09:43:01.523 INFO:teuthology.orchestra.run.smithi077.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-10T09:43:01.523 INFO:teuthology.orchestra.run.smithi077.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-10T09:43:01.523 INFO:teuthology.orchestra.run.smithi077.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-10T09:43:01.523 INFO:teuthology.orchestra.run.smithi077.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-10T09:43:01.523 INFO:teuthology.orchestra.run.smithi077.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-10T09:43:01.523 INFO:teuthology.orchestra.run.smithi077.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-10T09:43:01.523 INFO:teuthology.orchestra.run.smithi077.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-10T09:43:01.523 INFO:teuthology.orchestra.run.smithi077.stdout: libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:43:01.523 INFO:teuthology.orchestra.run.smithi077.stdout: libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:43:01.523 INFO:teuthology.orchestra.run.smithi077.stdout: libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:43:01.523 INFO:teuthology.orchestra.run.smithi077.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-10T09:43:01.523 INFO:teuthology.orchestra.run.smithi077.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-10T09:43:01.523 INFO:teuthology.orchestra.run.smithi077.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-10T09:43:01.524 INFO:teuthology.orchestra.run.smithi077.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-10T09:43:01.524 INFO:teuthology.orchestra.run.smithi077.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-10T09:43:01.524 INFO:teuthology.orchestra.run.smithi077.stdout: librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:43:01.524 INFO:teuthology.orchestra.run.smithi077.stdout: libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:43:01.524 INFO:teuthology.orchestra.run.smithi077.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-10T09:43:01.524 INFO:teuthology.orchestra.run.smithi077.stdout: librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:43:01.524 INFO:teuthology.orchestra.run.smithi077.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-10T09:43:01.524 INFO:teuthology.orchestra.run.smithi077.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-10T09:43:01.524 INFO:teuthology.orchestra.run.smithi077.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-10T09:43:01.524 INFO:teuthology.orchestra.run.smithi077.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-10T09:43:01.524 INFO:teuthology.orchestra.run.smithi077.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-10T09:43:01.524 INFO:teuthology.orchestra.run.smithi077.stdout: lua-5.4.4-4.el9.x86_64 2024-09-10T09:43:01.524 INFO:teuthology.orchestra.run.smithi077.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-10T09:43:01.525 INFO:teuthology.orchestra.run.smithi077.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-10T09:43:01.525 INFO:teuthology.orchestra.run.smithi077.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-10T09:43:01.525 INFO:teuthology.orchestra.run.smithi077.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-10T09:43:01.525 INFO:teuthology.orchestra.run.smithi077.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-10T09:43:01.525 INFO:teuthology.orchestra.run.smithi077.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-10T09:43:01.525 INFO:teuthology.orchestra.run.smithi077.stdout: postgresql-13.16-1.el9.x86_64 2024-09-10T09:43:01.525 INFO:teuthology.orchestra.run.smithi077.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-10T09:43:01.525 INFO:teuthology.orchestra.run.smithi077.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-10T09:43:01.525 INFO:teuthology.orchestra.run.smithi077.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-10T09:43:01.525 INFO:teuthology.orchestra.run.smithi077.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-10T09:43:01.525 INFO:teuthology.orchestra.run.smithi077.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-10T09:43:01.525 INFO:teuthology.orchestra.run.smithi077.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-10T09:43:01.525 INFO:teuthology.orchestra.run.smithi077.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-10T09:43:01.526 INFO:teuthology.orchestra.run.smithi077.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-10T09:43:01.526 INFO:teuthology.orchestra.run.smithi077.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-10T09:43:01.526 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-10T09:43:01.526 INFO:teuthology.orchestra.run.smithi077.stdout: python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:43:01.526 INFO:teuthology.orchestra.run.smithi077.stdout: python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:43:01.526 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:43:01.526 INFO:teuthology.orchestra.run.smithi077.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-10T09:43:01.526 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-10T09:43:01.526 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-10T09:43:01.526 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-10T09:43:01.526 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-10T09:43:01.526 INFO:teuthology.orchestra.run.smithi077.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-10T09:43:01.526 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-10T09:43:01.526 INFO:teuthology.orchestra.run.smithi077.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-10T09:43:01.527 INFO:teuthology.orchestra.run.smithi077.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-10T09:43:01.527 INFO:teuthology.orchestra.run.smithi077.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-10T09:43:01.527 INFO:teuthology.orchestra.run.smithi077.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-10T09:43:01.527 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-10T09:43:01.527 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-10T09:43:01.527 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-10T09:43:01.527 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-10T09:43:01.527 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-10T09:43:01.527 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-10T09:43:01.527 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-10T09:43:01.528 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-10T09:43:01.528 INFO:teuthology.orchestra.run.smithi077.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-10T09:43:01.528 INFO:teuthology.orchestra.run.smithi077.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-10T09:43:01.528 INFO:teuthology.orchestra.run.smithi077.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-10T09:43:01.528 INFO:teuthology.orchestra.run.smithi077.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-10T09:43:01.528 INFO:teuthology.orchestra.run.smithi077.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-10T09:43:01.528 INFO:teuthology.orchestra.run.smithi077.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-10T09:43:01.528 INFO:teuthology.orchestra.run.smithi077.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-10T09:43:01.528 INFO:teuthology.orchestra.run.smithi077.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-10T09:43:01.528 INFO:teuthology.orchestra.run.smithi077.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-10T09:43:01.528 INFO:teuthology.orchestra.run.smithi077.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-10T09:43:01.528 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-10T09:43:01.528 INFO:teuthology.orchestra.run.smithi077.stdout: python3-ply-3.11-14.el9.noarch 2024-09-10T09:43:01.529 INFO:teuthology.orchestra.run.smithi077.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-10T09:43:01.529 INFO:teuthology.orchestra.run.smithi077.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-10T09:43:01.529 INFO:teuthology.orchestra.run.smithi077.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-10T09:43:01.529 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-10T09:43:01.529 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-10T09:43:01.529 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-10T09:43:01.529 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-10T09:43:01.529 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-10T09:43:01.529 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-10T09:43:01.529 INFO:teuthology.orchestra.run.smithi077.stdout: python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:43:01.529 INFO:teuthology.orchestra.run.smithi077.stdout: python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:43:01.529 INFO:teuthology.orchestra.run.smithi077.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-10T09:43:01.529 INFO:teuthology.orchestra.run.smithi077.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-10T09:43:01.530 INFO:teuthology.orchestra.run.smithi077.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-10T09:43:01.530 INFO:teuthology.orchestra.run.smithi077.stdout: python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:43:01.530 INFO:teuthology.orchestra.run.smithi077.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-10T09:43:01.530 INFO:teuthology.orchestra.run.smithi077.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-10T09:43:01.530 INFO:teuthology.orchestra.run.smithi077.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-10T09:43:01.530 INFO:teuthology.orchestra.run.smithi077.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-10T09:43:01.530 INFO:teuthology.orchestra.run.smithi077.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-10T09:43:01.530 INFO:teuthology.orchestra.run.smithi077.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-10T09:43:01.530 INFO:teuthology.orchestra.run.smithi077.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-10T09:43:01.530 INFO:teuthology.orchestra.run.smithi077.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-10T09:43:01.530 INFO:teuthology.orchestra.run.smithi077.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-10T09:43:01.530 INFO:teuthology.orchestra.run.smithi077.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-10T09:43:01.530 INFO:teuthology.orchestra.run.smithi077.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-10T09:43:01.530 INFO:teuthology.orchestra.run.smithi077.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-10T09:43:01.531 INFO:teuthology.orchestra.run.smithi077.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-10T09:43:01.531 INFO:teuthology.orchestra.run.smithi077.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-10T09:43:01.531 INFO:teuthology.orchestra.run.smithi077.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-10T09:43:01.531 INFO:teuthology.orchestra.run.smithi077.stdout: rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:43:01.531 INFO:teuthology.orchestra.run.smithi077.stdout: rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:43:01.531 INFO:teuthology.orchestra.run.smithi077.stdout: rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:43:01.531 INFO:teuthology.orchestra.run.smithi077.stdout: re2-1:20211101-20.el9.x86_64 2024-09-10T09:43:01.531 INFO:teuthology.orchestra.run.smithi077.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-10T09:43:01.531 INFO:teuthology.orchestra.run.smithi077.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-10T09:43:01.531 INFO:teuthology.orchestra.run.smithi077.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-10T09:43:01.531 INFO:teuthology.orchestra.run.smithi077.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-10T09:43:01.531 INFO:teuthology.orchestra.run.smithi077.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-10T09:43:01.532 INFO:teuthology.orchestra.run.smithi077.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-10T09:43:01.532 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:43:01.532 INFO:teuthology.orchestra.run.smithi077.stdout:Complete! 2024-09-10T09:43:02.180 DEBUG:teuthology.parallel:result is None 2024-09-10T09:43:03.851 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T09:43:03.852 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T09:43:03.852 INFO:teuthology.orchestra.run.smithi120.stdout:Upgraded: 2024-09-10T09:43:03.852 INFO:teuthology.orchestra.run.smithi120.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-10T09:43:03.852 INFO:teuthology.orchestra.run.smithi120.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-10T09:43:03.852 INFO:teuthology.orchestra.run.smithi120.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-10T09:43:03.852 INFO:teuthology.orchestra.run.smithi120.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-10T09:43:03.852 INFO:teuthology.orchestra.run.smithi120.stdout: librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:43:03.852 INFO:teuthology.orchestra.run.smithi120.stdout: librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:43:03.852 INFO:teuthology.orchestra.run.smithi120.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-10T09:43:03.852 INFO:teuthology.orchestra.run.smithi120.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-10T09:43:03.852 INFO:teuthology.orchestra.run.smithi120.stdout:Installed: 2024-09-10T09:43:03.853 INFO:teuthology.orchestra.run.smithi120.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-10T09:43:03.853 INFO:teuthology.orchestra.run.smithi120.stdout: automake-1.16.2-8.el9.noarch 2024-09-10T09:43:03.853 INFO:teuthology.orchestra.run.smithi120.stdout: bison-3.7.4-5.el9.x86_64 2024-09-10T09:43:03.853 INFO:teuthology.orchestra.run.smithi120.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-10T09:43:03.853 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:43:03.853 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:43:03.853 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:43:03.853 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:43:03.853 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T09:43:03.853 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-immutable-object-cache-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:43:03.853 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:43:03.853 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:43:03.854 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T09:43:03.854 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T09:43:03.854 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T09:43:03.854 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T09:43:03.854 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T09:43:03.854 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:43:03.854 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:43:03.854 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T09:43:03.854 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:43:03.854 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:43:03.854 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:43:03.854 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T09:43:03.854 INFO:teuthology.orchestra.run.smithi120.stdout: cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T09:43:03.855 INFO:teuthology.orchestra.run.smithi120.stdout: cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:43:03.855 INFO:teuthology.orchestra.run.smithi120.stdout: cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:43:03.855 INFO:teuthology.orchestra.run.smithi120.stdout: cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T09:43:03.855 INFO:teuthology.orchestra.run.smithi120.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-10T09:43:03.855 INFO:teuthology.orchestra.run.smithi120.stdout: flex-2.6.4-9.el9.x86_64 2024-09-10T09:43:03.855 INFO:teuthology.orchestra.run.smithi120.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-10T09:43:03.855 INFO:teuthology.orchestra.run.smithi120.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-10T09:43:03.855 INFO:teuthology.orchestra.run.smithi120.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-10T09:43:03.855 INFO:teuthology.orchestra.run.smithi120.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-10T09:43:03.855 INFO:teuthology.orchestra.run.smithi120.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-10T09:43:03.855 INFO:teuthology.orchestra.run.smithi120.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-10T09:43:03.855 INFO:teuthology.orchestra.run.smithi120.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-10T09:43:03.855 INFO:teuthology.orchestra.run.smithi120.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-10T09:43:03.856 INFO:teuthology.orchestra.run.smithi120.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-10T09:43:03.856 INFO:teuthology.orchestra.run.smithi120.stdout: libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:43:03.856 INFO:teuthology.orchestra.run.smithi120.stdout: libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:43:03.856 INFO:teuthology.orchestra.run.smithi120.stdout: libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:43:03.856 INFO:teuthology.orchestra.run.smithi120.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-10T09:43:03.856 INFO:teuthology.orchestra.run.smithi120.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-10T09:43:03.856 INFO:teuthology.orchestra.run.smithi120.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-10T09:43:03.856 INFO:teuthology.orchestra.run.smithi120.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-10T09:43:03.856 INFO:teuthology.orchestra.run.smithi120.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-10T09:43:03.856 INFO:teuthology.orchestra.run.smithi120.stdout: librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:43:03.856 INFO:teuthology.orchestra.run.smithi120.stdout: libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:43:03.856 INFO:teuthology.orchestra.run.smithi120.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-10T09:43:03.856 INFO:teuthology.orchestra.run.smithi120.stdout: librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:43:03.857 INFO:teuthology.orchestra.run.smithi120.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-10T09:43:03.857 INFO:teuthology.orchestra.run.smithi120.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-10T09:43:03.857 INFO:teuthology.orchestra.run.smithi120.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-10T09:43:03.857 INFO:teuthology.orchestra.run.smithi120.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-10T09:43:03.857 INFO:teuthology.orchestra.run.smithi120.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-10T09:43:03.857 INFO:teuthology.orchestra.run.smithi120.stdout: lua-5.4.4-4.el9.x86_64 2024-09-10T09:43:03.857 INFO:teuthology.orchestra.run.smithi120.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-10T09:43:03.857 INFO:teuthology.orchestra.run.smithi120.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-10T09:43:03.857 INFO:teuthology.orchestra.run.smithi120.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-10T09:43:03.857 INFO:teuthology.orchestra.run.smithi120.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-10T09:43:03.857 INFO:teuthology.orchestra.run.smithi120.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-10T09:43:03.857 INFO:teuthology.orchestra.run.smithi120.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-10T09:43:03.858 INFO:teuthology.orchestra.run.smithi120.stdout: postgresql-13.16-1.el9.x86_64 2024-09-10T09:43:03.858 INFO:teuthology.orchestra.run.smithi120.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-10T09:43:03.858 INFO:teuthology.orchestra.run.smithi120.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-10T09:43:03.858 INFO:teuthology.orchestra.run.smithi120.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-10T09:43:03.858 INFO:teuthology.orchestra.run.smithi120.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-10T09:43:03.858 INFO:teuthology.orchestra.run.smithi120.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-10T09:43:03.858 INFO:teuthology.orchestra.run.smithi120.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-10T09:43:03.858 INFO:teuthology.orchestra.run.smithi120.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-10T09:43:03.858 INFO:teuthology.orchestra.run.smithi120.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-10T09:43:03.858 INFO:teuthology.orchestra.run.smithi120.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-10T09:43:03.858 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-10T09:43:03.858 INFO:teuthology.orchestra.run.smithi120.stdout: python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:43:03.858 INFO:teuthology.orchestra.run.smithi120.stdout: python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:43:03.859 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:43:03.859 INFO:teuthology.orchestra.run.smithi120.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-10T09:43:03.859 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-10T09:43:03.859 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-10T09:43:03.859 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-10T09:43:03.859 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-10T09:43:03.859 INFO:teuthology.orchestra.run.smithi120.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-10T09:43:03.859 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-10T09:43:03.859 INFO:teuthology.orchestra.run.smithi120.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-10T09:43:03.860 INFO:teuthology.orchestra.run.smithi120.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-10T09:43:03.860 INFO:teuthology.orchestra.run.smithi120.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-10T09:43:03.860 INFO:teuthology.orchestra.run.smithi120.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-10T09:43:03.860 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-10T09:43:03.860 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-10T09:43:03.860 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-10T09:43:03.860 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-10T09:43:03.860 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-10T09:43:03.860 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-10T09:43:03.860 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-10T09:43:03.860 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-10T09:43:03.860 INFO:teuthology.orchestra.run.smithi120.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-10T09:43:03.861 INFO:teuthology.orchestra.run.smithi120.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-10T09:43:03.861 INFO:teuthology.orchestra.run.smithi120.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-10T09:43:03.861 INFO:teuthology.orchestra.run.smithi120.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-10T09:43:03.861 INFO:teuthology.orchestra.run.smithi120.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-10T09:43:03.861 INFO:teuthology.orchestra.run.smithi120.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-10T09:43:03.861 INFO:teuthology.orchestra.run.smithi120.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-10T09:43:03.862 INFO:teuthology.orchestra.run.smithi120.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-10T09:43:03.862 INFO:teuthology.orchestra.run.smithi120.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-10T09:43:03.862 INFO:teuthology.orchestra.run.smithi120.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-10T09:43:03.862 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-10T09:43:03.862 INFO:teuthology.orchestra.run.smithi120.stdout: python3-ply-3.11-14.el9.noarch 2024-09-10T09:43:03.862 INFO:teuthology.orchestra.run.smithi120.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-10T09:43:03.862 INFO:teuthology.orchestra.run.smithi120.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-10T09:43:03.862 INFO:teuthology.orchestra.run.smithi120.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-10T09:43:03.862 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-10T09:43:03.862 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-10T09:43:03.863 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-10T09:43:03.863 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-10T09:43:03.863 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-10T09:43:03.863 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-10T09:43:03.863 INFO:teuthology.orchestra.run.smithi120.stdout: python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:43:03.863 INFO:teuthology.orchestra.run.smithi120.stdout: python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:43:03.863 INFO:teuthology.orchestra.run.smithi120.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-10T09:43:03.864 INFO:teuthology.orchestra.run.smithi120.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-10T09:43:03.864 INFO:teuthology.orchestra.run.smithi120.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-10T09:43:03.864 INFO:teuthology.orchestra.run.smithi120.stdout: python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:43:03.864 INFO:teuthology.orchestra.run.smithi120.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-10T09:43:03.864 INFO:teuthology.orchestra.run.smithi120.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-10T09:43:03.864 INFO:teuthology.orchestra.run.smithi120.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-10T09:43:03.864 INFO:teuthology.orchestra.run.smithi120.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-10T09:43:03.865 INFO:teuthology.orchestra.run.smithi120.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-10T09:43:03.865 INFO:teuthology.orchestra.run.smithi120.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-10T09:43:03.865 INFO:teuthology.orchestra.run.smithi120.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-10T09:43:03.865 INFO:teuthology.orchestra.run.smithi120.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-10T09:43:03.865 INFO:teuthology.orchestra.run.smithi120.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-10T09:43:03.865 INFO:teuthology.orchestra.run.smithi120.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-10T09:43:03.866 INFO:teuthology.orchestra.run.smithi120.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-10T09:43:03.866 INFO:teuthology.orchestra.run.smithi120.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-10T09:43:03.866 INFO:teuthology.orchestra.run.smithi120.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-10T09:43:03.866 INFO:teuthology.orchestra.run.smithi120.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-10T09:43:03.866 INFO:teuthology.orchestra.run.smithi120.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-10T09:43:03.867 INFO:teuthology.orchestra.run.smithi120.stdout: rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:43:03.867 INFO:teuthology.orchestra.run.smithi120.stdout: rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:43:03.867 INFO:teuthology.orchestra.run.smithi120.stdout: rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:43:03.867 INFO:teuthology.orchestra.run.smithi120.stdout: re2-1:20211101-20.el9.x86_64 2024-09-10T09:43:03.867 INFO:teuthology.orchestra.run.smithi120.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-10T09:43:03.867 INFO:teuthology.orchestra.run.smithi120.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-10T09:43:03.867 INFO:teuthology.orchestra.run.smithi120.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-10T09:43:03.868 INFO:teuthology.orchestra.run.smithi120.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-10T09:43:03.868 INFO:teuthology.orchestra.run.smithi120.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-10T09:43:03.868 INFO:teuthology.orchestra.run.smithi120.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-10T09:43:03.869 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T09:43:03.869 INFO:teuthology.orchestra.run.smithi120.stdout:Complete! 2024-09-10T09:43:04.285 DEBUG:teuthology.parallel:result is None 2024-09-10T09:43:04.286 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T09:43:04.441 DEBUG:teuthology.orchestra.run.smithi077:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-10T09:43:04.487 INFO:teuthology.orchestra.run.smithi077.stdout:19.1.1-305.g1cf819b7.el9 2024-09-10T09:43:04.488 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-305.g1cf819b7.el9 2024-09-10T09:43:04.488 INFO:teuthology.task.install:The correct ceph version 19.1.1-305.g1cf819b7 is installed. 2024-09-10T09:43:04.490 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T09:43:04.637 DEBUG:teuthology.orchestra.run.smithi120:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-10T09:43:04.660 INFO:teuthology.orchestra.run.smithi120.stdout:19.1.1-305.g1cf819b7.el9 2024-09-10T09:43:04.660 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-305.g1cf819b7.el9 2024-09-10T09:43:04.660 INFO:teuthology.task.install:The correct ceph version 19.1.1-305.g1cf819b7 is installed. 2024-09-10T09:43:04.662 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T09:43:04.810 DEBUG:teuthology.orchestra.run.smithi134:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-10T09:43:04.837 INFO:teuthology.orchestra.run.smithi134.stdout:19.1.1-305.g1cf819b7.el9 2024-09-10T09:43:04.837 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-305.g1cf819b7.el9 2024-09-10T09:43:04.837 INFO:teuthology.task.install:The correct ceph version 19.1.1-305.g1cf819b7 is installed. 2024-09-10T09:43:04.839 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-10T09:43:04.839 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-09-10T09:43:04.839 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-10T09:43:04.900 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-10T09:43:04.900 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-10T09:43:04.927 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-10T09:43:04.927 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-10T09:43:04.960 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-10T09:43:04.960 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-09-10T09:43:04.960 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/usr/bin/daemon-helper 2024-09-10T09:43:05.042 DEBUG:teuthology.orchestra.run.smithi077:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-10T09:43:05.137 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-10T09:43:05.137 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/usr/bin/daemon-helper 2024-09-10T09:43:05.162 DEBUG:teuthology.orchestra.run.smithi120:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-10T09:43:05.228 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-10T09:43:05.228 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/usr/bin/daemon-helper 2024-09-10T09:43:05.258 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-10T09:43:05.327 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-10T09:43:05.327 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-09-10T09:43:05.327 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-10T09:43:05.382 DEBUG:teuthology.orchestra.run.smithi077:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-10T09:43:05.477 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-10T09:43:05.478 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-10T09:43:05.502 DEBUG:teuthology.orchestra.run.smithi120:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-10T09:43:05.567 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-10T09:43:05.567 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-10T09:43:05.597 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-10T09:43:05.664 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-10T09:43:05.665 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-09-10T09:43:05.665 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/usr/bin/stdin-killer 2024-09-10T09:43:05.719 DEBUG:teuthology.orchestra.run.smithi077:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-10T09:43:05.815 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-10T09:43:05.815 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/usr/bin/stdin-killer 2024-09-10T09:43:05.839 DEBUG:teuthology.orchestra.run.smithi120:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-10T09:43:05.904 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-10T09:43:05.904 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/usr/bin/stdin-killer 2024-09-10T09:43:05.932 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-10T09:43:06.001 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-10T09:43:06.091 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b', 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-09-10T09:43:06.092 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T09:43:06.092 INFO:tasks.cephadm:Cluster fsid is 14a55226-6f59-11ef-bcea-c7b262605968 2024-09-10T09:43:06.092 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-10T09:43:06.093 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.77', 'mon.b': '172.21.15.120', 'mon.c': '172.21.15.134'} 2024-09-10T09:43:06.093 INFO:tasks.cephadm:First mon is mon.a on smithi077 2024-09-10T09:43:06.093 INFO:tasks.cephadm:First mgr is x 2024-09-10T09:43:06.093 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-10T09:43:06.093 DEBUG:teuthology.orchestra.run.smithi077:> sudo hostname $(hostname -s) 2024-09-10T09:43:06.150 DEBUG:teuthology.orchestra.run.smithi120:> sudo hostname $(hostname -s) 2024-09-10T09:43:06.176 DEBUG:teuthology.orchestra.run.smithi134:> sudo hostname $(hostname -s) 2024-09-10T09:43:06.206 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-10T09:43:06.207 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T09:43:06.352 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/82914/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.1.1-305-g1cf819b7', 'node_name': '172.21.5.35+adami05', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.1.1-305.g1cf819b7'}, 'url': 'https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-09 22:14:38.445020', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-rishabh-testing-20240909.165402-squid', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}, {'status': 'ready', 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/82912/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.1.1-305-g1cf819b7', 'node_name': '172.21.2.7+braggi07', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.1.1-305.g1cf819b7'}, 'url': 'https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-09 22:05:03.996701', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-rishabh-testing-20240909.165402-squid', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-09-10T09:43:06.503 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref wip-rishabh-testing-20240909.165402-squid, sha1 1cf819b7212a6048e92fc78c4eff963344f6d56b from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T09:43:06.505 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/x86_64/flavors/default/cephadm 2024-09-10T09:43:06.505 INFO:tasks.cephadm:Downloading cephadm from url: https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/x86_64/flavors/default/cephadm 2024-09-10T09:43:06.505 DEBUG:teuthology.orchestra.run.smithi077:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-10T09:43:06.918 INFO:teuthology.orchestra.run.smithi077.stdout:-rw-r--r--. 1 ubuntu ubuntu 784613 Sep 10 09:43 /home/ubuntu/cephtest/cephadm 2024-09-10T09:43:06.919 DEBUG:teuthology.orchestra.run.smithi120:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-10T09:43:07.309 INFO:teuthology.orchestra.run.smithi120.stdout:-rw-r--r--. 1 ubuntu ubuntu 784613 Sep 10 09:43 /home/ubuntu/cephtest/cephadm 2024-09-10T09:43:07.309 DEBUG:teuthology.orchestra.run.smithi134:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-10T09:43:07.704 INFO:teuthology.orchestra.run.smithi134.stdout:-rw-r--r--. 1 ubuntu ubuntu 784613 Sep 10 09:43 /home/ubuntu/cephtest/cephadm 2024-09-10T09:43:07.705 DEBUG:teuthology.orchestra.run.smithi077:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-10T09:43:07.742 DEBUG:teuthology.orchestra.run.smithi120:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-10T09:43:07.763 DEBUG:teuthology.orchestra.run.smithi134:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-10T09:43:07.800 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b on all hosts... 2024-09-10T09:43:07.800 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b pull 2024-09-10T09:43:07.805 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b pull 2024-09-10T09:43:07.808 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b pull 2024-09-10T09:43:09.442 INFO:teuthology.orchestra.run.smithi134.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b... 2024-09-10T09:43:09.531 INFO:teuthology.orchestra.run.smithi120.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b... 2024-09-10T09:43:09.734 INFO:teuthology.orchestra.run.smithi077.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b... 2024-09-10T09:43:51.575 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-10T09:43:51.575 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_version": "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)", 2024-09-10T09:43:51.575 INFO:teuthology.orchestra.run.smithi077.stdout: "image_id": "41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005", 2024-09-10T09:43:51.575 INFO:teuthology.orchestra.run.smithi077.stdout: "repo_digests": [ 2024-09-10T09:43:51.575 INFO:teuthology.orchestra.run.smithi077.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559" 2024-09-10T09:43:51.576 INFO:teuthology.orchestra.run.smithi077.stdout: ] 2024-09-10T09:43:51.576 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-10T09:43:54.980 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T09:43:54.981 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_version": "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)", 2024-09-10T09:43:54.981 INFO:teuthology.orchestra.run.smithi134.stdout: "image_id": "41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005", 2024-09-10T09:43:54.981 INFO:teuthology.orchestra.run.smithi134.stdout: "repo_digests": [ 2024-09-10T09:43:54.981 INFO:teuthology.orchestra.run.smithi134.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559" 2024-09-10T09:43:54.981 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T09:43:54.981 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T09:43:56.408 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T09:43:56.408 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_version": "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)", 2024-09-10T09:43:56.408 INFO:teuthology.orchestra.run.smithi120.stdout: "image_id": "41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005", 2024-09-10T09:43:56.408 INFO:teuthology.orchestra.run.smithi120.stdout: "repo_digests": [ 2024-09-10T09:43:56.408 INFO:teuthology.orchestra.run.smithi120.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559" 2024-09-10T09:43:56.408 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T09:43:56.408 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T09:43:56.433 DEBUG:teuthology.orchestra.run.smithi077:> sudo mkdir -p /etc/ceph 2024-09-10T09:43:56.498 DEBUG:teuthology.orchestra.run.smithi120:> sudo mkdir -p /etc/ceph 2024-09-10T09:43:56.525 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /etc/ceph 2024-09-10T09:43:56.559 DEBUG:teuthology.orchestra.run.smithi077:> sudo chmod 777 /etc/ceph 2024-09-10T09:43:56.617 DEBUG:teuthology.orchestra.run.smithi120:> sudo chmod 777 /etc/ceph 2024-09-10T09:43:56.642 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod 777 /etc/ceph 2024-09-10T09:43:56.671 INFO:tasks.cephadm:Writing seed config... 2024-09-10T09:43:56.672 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-09-10T09:43:56.672 INFO:tasks.cephadm: override: [client] debug client = 20 2024-09-10T09:43:56.672 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-09-10T09:43:56.672 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-09-10T09:43:56.672 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-09-10T09:43:56.672 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-09-10T09:43:56.672 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-09-10T09:43:56.672 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-09-10T09:43:56.672 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-09-10T09:43:56.672 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-09-10T09:43:56.673 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-09-10T09:43:56.673 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-09-10T09:43:56.673 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-09-10T09:43:56.673 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-09-10T09:43:56.673 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-09-10T09:43:56.673 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-09-10T09:43:56.673 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-09-10T09:43:56.673 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-09-10T09:43:56.673 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-09-10T09:43:56.673 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-09-10T09:43:56.673 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-09-10T09:43:56.673 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-09-10T09:43:56.673 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-10T09:43:56.673 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-10T09:43:56.674 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-10T09:43:56.674 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-10T09:43:56.674 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-10T09:43:56.674 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-10T09:43:56.674 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-09-10T09:43:56.674 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-09-10T09:43:56.674 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-10T09:43:56.674 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-09-10T09:43:56.674 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-10T09:43:56.674 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-09-10T09:43:56.674 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-09-10T09:43:56.674 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-10T09:43:56.674 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-10T09:43:56.674 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-09-10T09:43:56.674 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-10T09:43:56.675 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-10T09:43:56.675 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-10T09:43:56.675 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-10T09:43:56.675 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-10T09:43:56.675 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-09-10T09:43:56.675 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-10T09:43:56.675 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-09-10T09:43:56.676 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-09-10T09:43:56.676 DEBUG:teuthology.orchestra.run.smithi077:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-10T09:43:56.711 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 14a55226-6f59-11ef-bcea-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-09-10T09:43:56.713 DEBUG:teuthology.orchestra.run.smithi077:mon.a> sudo journalctl -f -n 0 -u ceph-14a55226-6f59-11ef-bcea-c7b262605968@mon.a.service 2024-09-10T09:43:56.756 DEBUG:teuthology.orchestra.run.smithi077:mgr.x> sudo journalctl -f -n 0 -u ceph-14a55226-6f59-11ef-bcea-c7b262605968@mgr.x.service 2024-09-10T09:43:56.801 INFO:tasks.cephadm:Bootstrapping... 2024-09-10T09:43:56.801 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b -v bootstrap --fsid 14a55226-6f59-11ef-bcea-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.77 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-10T09:43:57.059 INFO:teuthology.orchestra.run.smithi077.stdout:-------------------------------------------------------------------------------- 2024-09-10T09:43:57.060 INFO:teuthology.orchestra.run.smithi077.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b', '-v', 'bootstrap', '--fsid', '14a55226-6f59-11ef-bcea-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.77', '--skip-admin-label'] 2024-09-10T09:43:57.060 INFO:teuthology.orchestra.run.smithi077.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-10T09:43:57.060 INFO:teuthology.orchestra.run.smithi077.stdout:Verifying podman|docker is present... 2024-09-10T09:43:57.121 INFO:teuthology.orchestra.run.smithi077.stdout:/bin/podman: stdout 5.2.2 2024-09-10T09:43:57.121 INFO:teuthology.orchestra.run.smithi077.stdout:Verifying lvm2 is present... 2024-09-10T09:43:57.121 INFO:teuthology.orchestra.run.smithi077.stdout:Verifying time synchronization is in place... 2024-09-10T09:43:57.135 INFO:teuthology.orchestra.run.smithi077.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-10T09:43:57.135 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-10T09:43:57.147 INFO:teuthology.orchestra.run.smithi077.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-10T09:43:57.147 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stdout inactive 2024-09-10T09:43:57.161 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stdout enabled 2024-09-10T09:43:57.173 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stdout active 2024-09-10T09:43:57.174 INFO:teuthology.orchestra.run.smithi077.stdout:Unit chronyd.service is enabled and running 2024-09-10T09:43:57.174 INFO:teuthology.orchestra.run.smithi077.stdout:Repeating the final host check... 2024-09-10T09:43:57.218 INFO:teuthology.orchestra.run.smithi077.stdout:/bin/podman: stdout 5.2.2 2024-09-10T09:43:57.218 INFO:teuthology.orchestra.run.smithi077.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-10T09:43:57.218 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl is present 2024-09-10T09:43:57.218 INFO:teuthology.orchestra.run.smithi077.stdout:lvcreate is present 2024-09-10T09:43:57.231 INFO:teuthology.orchestra.run.smithi077.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-10T09:43:57.231 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-10T09:43:57.243 INFO:teuthology.orchestra.run.smithi077.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-10T09:43:57.243 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stdout inactive 2024-09-10T09:43:57.256 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stdout enabled 2024-09-10T09:43:57.269 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stdout active 2024-09-10T09:43:57.269 INFO:teuthology.orchestra.run.smithi077.stdout:Unit chronyd.service is enabled and running 2024-09-10T09:43:57.269 INFO:teuthology.orchestra.run.smithi077.stdout:Host looks OK 2024-09-10T09:43:57.269 INFO:teuthology.orchestra.run.smithi077.stdout:Cluster fsid: 14a55226-6f59-11ef-bcea-c7b262605968 2024-09-10T09:43:57.269 INFO:teuthology.orchestra.run.smithi077.stdout:Acquiring lock 139739506492416 on /run/cephadm/14a55226-6f59-11ef-bcea-c7b262605968.lock 2024-09-10T09:43:57.270 INFO:teuthology.orchestra.run.smithi077.stdout:Lock 139739506492416 acquired on /run/cephadm/14a55226-6f59-11ef-bcea-c7b262605968.lock 2024-09-10T09:43:57.270 INFO:teuthology.orchestra.run.smithi077.stdout:Verifying IP 172.21.15.77 port 3300 ... 2024-09-10T09:43:57.270 INFO:teuthology.orchestra.run.smithi077.stdout:Verifying IP 172.21.15.77 port 6789 ... 2024-09-10T09:43:57.271 INFO:teuthology.orchestra.run.smithi077.stdout:Base mon IP(s) is [172.21.15.77:3300, 172.21.15.77:6789], mon addrv is [v2:172.21.15.77:3300,v1:172.21.15.77:6789] 2024-09-10T09:43:57.277 INFO:teuthology.orchestra.run.smithi077.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.77 metric 100 2024-09-10T09:43:57.278 INFO:teuthology.orchestra.run.smithi077.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.77 metric 100 2024-09-10T09:43:57.283 INFO:teuthology.orchestra.run.smithi077.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-10T09:43:57.284 INFO:teuthology.orchestra.run.smithi077.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-09-10T09:43:57.284 INFO:teuthology.orchestra.run.smithi077.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-09-10T09:43:57.289 INFO:teuthology.orchestra.run.smithi077.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-10T09:43:57.289 INFO:teuthology.orchestra.run.smithi077.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-10T09:43:57.290 INFO:teuthology.orchestra.run.smithi077.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-10T09:43:57.290 INFO:teuthology.orchestra.run.smithi077.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-09-10T09:43:57.290 INFO:teuthology.orchestra.run.smithi077.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:d3d3/64 scope link noprefixroute 2024-09-10T09:43:57.290 INFO:teuthology.orchestra.run.smithi077.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-10T09:43:57.291 INFO:teuthology.orchestra.run.smithi077.stdout:Mon IP `172.21.15.77` is in CIDR network `172.21.0.0/20` 2024-09-10T09:43:57.291 INFO:teuthology.orchestra.run.smithi077.stdout:Mon IP `172.21.15.77` is in CIDR network `172.21.0.0/20` 2024-09-10T09:43:57.291 INFO:teuthology.orchestra.run.smithi077.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-10T09:43:57.291 INFO:teuthology.orchestra.run.smithi077.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-10T09:43:57.292 INFO:teuthology.orchestra.run.smithi077.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b... 2024-09-10T09:43:57.808 INFO:teuthology.orchestra.run.smithi077.stdout:/bin/podman: stdout 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 2024-09-10T09:43:57.809 INFO:teuthology.orchestra.run.smithi077.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b... 2024-09-10T09:43:57.809 INFO:teuthology.orchestra.run.smithi077.stdout:/bin/podman: stderr Getting image source signatures 2024-09-10T09:43:57.809 INFO:teuthology.orchestra.run.smithi077.stdout:/bin/podman: stderr Copying blob sha256:3965ca83761b6e221cd35b61f2ebac0e3b1f9d68a1ce39c1ada69153a1d73116 2024-09-10T09:43:57.809 INFO:teuthology.orchestra.run.smithi077.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-10T09:43:57.809 INFO:teuthology.orchestra.run.smithi077.stdout:/bin/podman: stderr Copying config sha256:41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 2024-09-10T09:43:57.809 INFO:teuthology.orchestra.run.smithi077.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-10T09:43:58.879 INFO:teuthology.orchestra.run.smithi077.stdout:ceph: stdout ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable) 2024-09-10T09:43:58.879 INFO:teuthology.orchestra.run.smithi077.stdout:Ceph version: ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable) 2024-09-10T09:43:58.879 INFO:teuthology.orchestra.run.smithi077.stdout:Extracting ceph user uid/gid from container image... 2024-09-10T09:43:59.951 INFO:teuthology.orchestra.run.smithi077.stdout:stat: stdout 167 167 2024-09-10T09:43:59.951 INFO:teuthology.orchestra.run.smithi077.stdout:Creating initial keys... 2024-09-10T09:44:01.040 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph-authtool: stdout AQDgFOBmDW/kFRAAAPIJ1OS4liSx9g4ilC+yew== 2024-09-10T09:44:02.181 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph-authtool: stdout AQDhFOBmSSksGRAAiqSP/oiYm/aDaMdTjzJATw== 2024-09-10T09:44:03.278 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph-authtool: stdout AQDiFOBmpKlwIhAA6fcPfMyxV7V1uuarSHceXg== 2024-09-10T09:44:03.280 INFO:teuthology.orchestra.run.smithi077.stdout:Creating initial monmap... 2024-09-10T09:44:04.275 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-10T09:44:04.275 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-10T09:44:04.275 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 14a55226-6f59-11ef-bcea-c7b262605968 2024-09-10T09:44:04.275 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-10T09:44:04.275 INFO:teuthology.orchestra.run.smithi077.stdout:monmaptool for a [v2:172.21.15.77:3300,v1:172.21.15.77:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-10T09:44:04.275 INFO:teuthology.orchestra.run.smithi077.stdout:setting min_mon_release = quincy 2024-09-10T09:44:04.276 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/monmaptool: set fsid to 14a55226-6f59-11ef-bcea-c7b262605968 2024-09-10T09:44:04.276 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-10T09:44:04.276 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:44:04.276 INFO:teuthology.orchestra.run.smithi077.stdout:Creating mon... 2024-09-10T09:44:05.313 INFO:teuthology.orchestra.run.smithi077.stdout:create mon.a on 2024-09-10T09:44:05.618 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-10T09:44:05.835 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-10T09:44:06.104 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-14a55226-6f59-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-14a55226-6f59-11ef-bcea-c7b262605968.target. 2024-09-10T09:44:06.104 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-14a55226-6f59-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-14a55226-6f59-11ef-bcea-c7b262605968.target. 2024-09-10T09:44:06.432 INFO:teuthology.orchestra.run.smithi077.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-14a55226-6f59-11ef-bcea-c7b262605968@mon.a 2024-09-10T09:44:06.433 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stderr Failed to reset failed state of unit ceph-14a55226-6f59-11ef-bcea-c7b262605968@mon.a.service: Unit ceph-14a55226-6f59-11ef-bcea-c7b262605968@mon.a.service not loaded. 2024-09-10T09:44:06.707 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-14a55226-6f59-11ef-bcea-c7b262605968.target.wants/ceph-14a55226-6f59-11ef-bcea-c7b262605968@mon.a.service → /etc/systemd/system/ceph-14a55226-6f59-11ef-bcea-c7b262605968@.service. 2024-09-10T09:44:07.296 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:07 smithi077 podman[29933]: 2024-09-10 09:44:06.950182655 +0000 UTC m=+0.026531942 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T09:44:07.296 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:07 smithi077 podman[29933]: 2024-09-10 09:44:07.295100394 +0000 UTC m=+0.371449682 container init b9644962f5694e87b0493a4e4da8899fab6abf736a9a1d8c3f878ffafbdce85e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-mon-a, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-10T09:44:07.430 INFO:teuthology.orchestra.run.smithi077.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T09:44:07.431 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stdout disabled 2024-09-10T09:44:07.444 INFO:teuthology.orchestra.run.smithi077.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T09:44:07.444 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stdout inactive 2024-09-10T09:44:07.444 INFO:teuthology.orchestra.run.smithi077.stdout:firewalld.service is not enabled 2024-09-10T09:44:07.444 INFO:teuthology.orchestra.run.smithi077.stdout:Not possible to enable service . firewalld.service is not available 2024-09-10T09:44:07.445 INFO:teuthology.orchestra.run.smithi077.stdout:Waiting for mon to start... 2024-09-10T09:44:07.445 INFO:teuthology.orchestra.run.smithi077.stdout:Waiting for mon... 2024-09-10T09:44:07.585 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:07 smithi077 podman[29933]: 2024-09-10 09:44:07.301341799 +0000 UTC m=+0.377691065 container start b9644962f5694e87b0493a4e4da8899fab6abf736a9a1d8c3f878ffafbdce85e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-mon-a, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True) 2024-09-10T09:44:07.585 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:07 smithi077 ceph-mon[29953]: mkfs 14a55226-6f59-11ef-bcea-c7b262605968 2024-09-10T09:44:07.585 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:07 smithi077 ceph-mon[29953]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-10T09:44:07.585 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:07 smithi077 bash[29933]: b9644962f5694e87b0493a4e4da8899fab6abf736a9a1d8c3f878ffafbdce85e 2024-09-10T09:44:07.586 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:07 smithi077 systemd[1]: Started Ceph mon.a for 14a55226-6f59-11ef-bcea-c7b262605968. 2024-09-10T09:44:08.611 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:08 smithi077 ceph-mon[29953]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-10T09:44:08.611 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:08 smithi077 ceph-mon[29953]: monmap epoch 1 2024-09-10T09:44:08.611 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:08 smithi077 ceph-mon[29953]: fsid 14a55226-6f59-11ef-bcea-c7b262605968 2024-09-10T09:44:08.611 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:08 smithi077 ceph-mon[29953]: last_changed 2024-09-10T09:44:03.655548+0000 2024-09-10T09:44:08.611 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:08 smithi077 ceph-mon[29953]: created 2024-09-10T09:44:03.655548+0000 2024-09-10T09:44:08.611 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:08 smithi077 ceph-mon[29953]: min_mon_release 19 (squid) 2024-09-10T09:44:08.612 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:08 smithi077 ceph-mon[29953]: election_strategy: 1 2024-09-10T09:44:08.612 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:08 smithi077 ceph-mon[29953]: 0: [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] mon.a 2024-09-10T09:44:08.612 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:08 smithi077 ceph-mon[29953]: fsmap 2024-09-10T09:44:08.612 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:08 smithi077 ceph-mon[29953]: osdmap e1: 0 total, 0 up, 0 in 2024-09-10T09:44:08.612 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:08 smithi077 ceph-mon[29953]: mgrmap e1: no daemons active 2024-09-10T09:44:08.612 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:08 smithi077 ceph-mon[29953]: from='client.? 172.21.15.77:0/861010745' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-10T09:44:08.617 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout cluster: 2024-09-10T09:44:08.623 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout id: 14a55226-6f59-11ef-bcea-c7b262605968 2024-09-10T09:44:08.623 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-10T09:44:08.623 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout 2024-09-10T09:44:08.623 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout services: 2024-09-10T09:44:08.623 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.654015s) 2024-09-10T09:44:08.623 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-10T09:44:08.623 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-10T09:44:08.623 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout 2024-09-10T09:44:08.623 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout data: 2024-09-10T09:44:08.623 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-10T09:44:08.624 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-10T09:44:08.624 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-10T09:44:08.624 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout pgs: 2024-09-10T09:44:08.624 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout 2024-09-10T09:44:08.624 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:07.937+0000 7f969f800640 1 Processor -- start 2024-09-10T09:44:08.624 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:07.937+0000 7f969f800640 1 -- start start 2024-09-10T09:44:08.624 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:07.937+0000 7f969f800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9698107560 0x7f9698107960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:08.624 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:07.937+0000 7f969f800640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9698107f30 con 0x7f9698107560 2024-09-10T09:44:08.624 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:07.938+0000 7f969ce00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9698107560 0x7f9698107960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:08.624 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:07.938+0000 7f969ce00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9698107560 0x7f9698107960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58478/0 (socket says 172.21.15.77:58478) 2024-09-10T09:44:08.624 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:07.938+0000 7f969ce00640 1 -- 172.21.15.77:0/3244140226 learned_addr learned my addr 172.21.15.77:0/3244140226 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:44:08.624 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:07.938+0000 7f969ce00640 1 -- 172.21.15.77:0/3244140226 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9698108760 con 0x7f9698107560 2024-09-10T09:44:08.624 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:07.941+0000 7f969ce00640 1 --2- 172.21.15.77:0/3244140226 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9698107560 0x7f9698107960 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f968c009820 tx=0x7f968c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f723a500add7546 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:08.625 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:07.941+0000 7f9697400640 1 -- 172.21.15.77:0/3244140226 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f968c037070 con 0x7f9698107560 2024-09-10T09:44:08.625 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:07.942+0000 7f9697400640 1 -- 172.21.15.77:0/3244140226 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f968c032440 con 0x7f9698107560 2024-09-10T09:44:08.625 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:07.942+0000 7f9697400640 1 -- 172.21.15.77:0/3244140226 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f968c032740 con 0x7f9698107560 2024-09-10T09:44:08.625 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:07.942+0000 7f969f800640 1 -- 172.21.15.77:0/3244140226 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9698107560 msgr2=0x7f9698107960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:08.625 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:07.942+0000 7f969f800640 1 --2- 172.21.15.77:0/3244140226 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9698107560 0x7f9698107960 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f968c009820 tx=0x7f968c02a270 comp rx=0 tx=0).stop 2024-09-10T09:44:08.625 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:07.942+0000 7f969f800640 1 -- 172.21.15.77:0/3244140226 shutdown_connections 2024-09-10T09:44:08.625 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:07.942+0000 7f969f800640 1 --2- 172.21.15.77:0/3244140226 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9698107560 0x7f9698107960 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:08.625 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:07.942+0000 7f969f800640 1 -- 172.21.15.77:0/3244140226 >> 172.21.15.77:0/3244140226 conn(0x7f9698102cf0 msgr2=0x7f9698105130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:08.625 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:07.942+0000 7f969f800640 1 -- 172.21.15.77:0/3244140226 shutdown_connections 2024-09-10T09:44:08.625 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:07.943+0000 7f969f800640 1 -- 172.21.15.77:0/3244140226 wait complete. 2024-09-10T09:44:08.625 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:07.943+0000 7f969f800640 1 Processor -- start 2024-09-10T09:44:08.625 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:07.944+0000 7f969f800640 1 -- start start 2024-09-10T09:44:08.625 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:07.944+0000 7f969f800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9698107560 0x7f969819d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:08.625 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:07.944+0000 7f969f800640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9698108d00 con 0x7f9698107560 2024-09-10T09:44:08.626 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:07.944+0000 7f969ce00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9698107560 0x7f969819d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:08.626 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:07.944+0000 7f969ce00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9698107560 0x7f969819d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58494/0 (socket says 172.21.15.77:58494) 2024-09-10T09:44:08.626 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:07.944+0000 7f969ce00640 1 -- 172.21.15.77:0/861010745 learned_addr learned my addr 172.21.15.77:0/861010745 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:44:08.626 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:07.945+0000 7f969ce00640 1 -- 172.21.15.77:0/861010745 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f969819d7a0 con 0x7f9698107560 2024-09-10T09:44:08.626 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:07.945+0000 7f969ce00640 1 --2- 172.21.15.77:0/861010745 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9698107560 0x7f969819d260 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f968c0090b0 tx=0x7f968c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:08.626 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:07.945+0000 7f9695600640 1 -- 172.21.15.77:0/861010745 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f968c041070 con 0x7f9698107560 2024-09-10T09:44:08.626 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:07.945+0000 7f9695600640 1 -- 172.21.15.77:0/861010745 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f968c03c7a0 con 0x7f9698107560 2024-09-10T09:44:08.626 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:07.945+0000 7f9695600640 1 -- 172.21.15.77:0/861010745 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f968c037040 con 0x7f9698107560 2024-09-10T09:44:08.626 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:07.945+0000 7f969f800640 1 -- 172.21.15.77:0/861010745 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f969819da30 con 0x7f9698107560 2024-09-10T09:44:08.626 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:07.946+0000 7f969f800640 1 -- 172.21.15.77:0/861010745 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96981a0720 con 0x7f9698107560 2024-09-10T09:44:08.626 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:07.946+0000 7f9695600640 1 -- 172.21.15.77:0/861010745 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f968c044070 con 0x7f9698107560 2024-09-10T09:44:08.626 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:07.946+0000 7f9695600640 1 -- 172.21.15.77:0/861010745 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f968c03d3f0 con 0x7f9698107560 2024-09-10T09:44:08.626 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:07.947+0000 7f969f800640 1 -- 172.21.15.77:0/861010745 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9698109350 con 0x7f9698107560 2024-09-10T09:44:08.627 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:07.950+0000 7f9695600640 1 -- 172.21.15.77:0/861010745 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f968c046450 con 0x7f9698107560 2024-09-10T09:44:08.627 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:08.011+0000 7f969f800640 1 -- 172.21.15.77:0/861010745 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f9698075f40 con 0x7f9698107560 2024-09-10T09:44:08.627 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:08.012+0000 7f9695600640 1 -- 172.21.15.77:0/861010745 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f968c046630 con 0x7f9698107560 2024-09-10T09:44:08.627 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:08.012+0000 7f969f800640 1 -- 172.21.15.77:0/861010745 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9698107560 msgr2=0x7f969819d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:08.627 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:08.012+0000 7f969f800640 1 --2- 172.21.15.77:0/861010745 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9698107560 0x7f969819d260 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f968c0090b0 tx=0x7f968c0046c0 comp rx=0 tx=0).stop 2024-09-10T09:44:08.627 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:08.012+0000 7f969f800640 1 -- 172.21.15.77:0/861010745 shutdown_connections 2024-09-10T09:44:08.627 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:08.012+0000 7f969f800640 1 --2- 172.21.15.77:0/861010745 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9698107560 0x7f969819d260 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:08.627 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:08.012+0000 7f969f800640 1 -- 172.21.15.77:0/861010745 >> 172.21.15.77:0/861010745 conn(0x7f9698102cf0 msgr2=0x7f9698193e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:08.627 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:08.013+0000 7f969f800640 1 -- 172.21.15.77:0/861010745 shutdown_connections 2024-09-10T09:44:08.627 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:08.013+0000 7f969f800640 1 -- 172.21.15.77:0/861010745 wait complete. 2024-09-10T09:44:08.627 INFO:teuthology.orchestra.run.smithi077.stdout:mon is available 2024-09-10T09:44:08.627 INFO:teuthology.orchestra.run.smithi077.stdout:Assimilating anything we can from ceph.conf... 2024-09-10T09:44:09.538 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:09 smithi077 ceph-mon[29953]: from='client.? 172.21.15.77:0/2706458163' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-10T09:44:09.538 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:09 smithi077 ceph-mon[29953]: from='client.? 172.21.15.77:0/2706458163' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-10T09:44:09.855 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout 2024-09-10T09:44:09.855 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout [global] 2024-09-10T09:44:09.855 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout fsid = 14a55226-6f59-11ef-bcea-c7b262605968 2024-09-10T09:44:09.855 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.77:3300,v1:172.21.15.77:6789] 2024-09-10T09:44:09.855 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-10T09:44:09.856 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-10T09:44:09.856 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-10T09:44:09.856 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-10T09:44:09.856 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout 2024-09-10T09:44:09.856 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-10T09:44:09.856 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-10T09:44:09.856 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout 2024-09-10T09:44:09.856 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout [osd] 2024-09-10T09:44:09.856 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-10T09:44:09.856 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-10T09:44:09.856 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-10T09:44:09.857 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-10T09:44:09.857 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-10T09:44:09.857 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:09.134+0000 7f46cec00640 1 Processor -- start 2024-09-10T09:44:09.857 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:09.135+0000 7f46cec00640 1 -- start start 2024-09-10T09:44:09.857 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:09.135+0000 7f46cec00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46c807de70 0x7f46c807c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:09.857 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:09.135+0000 7f46cec00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f46c807c8a0 con 0x7f46c807de70 2024-09-10T09:44:09.857 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:09.135+0000 7f46cd800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46c807de70 0x7f46c807c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:09.857 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:09.135+0000 7f46cd800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46c807de70 0x7f46c807c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58498/0 (socket says 172.21.15.77:58498) 2024-09-10T09:44:09.857 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:09.135+0000 7f46cd800640 1 -- 172.21.15.77:0/862472781 learned_addr learned my addr 172.21.15.77:0/862472781 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:44:09.857 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:09.136+0000 7f46cd800640 1 -- 172.21.15.77:0/862472781 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46c807ca20 con 0x7f46c807de70 2024-09-10T09:44:09.858 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:09.137+0000 7f46cd800640 1 --2- 172.21.15.77:0/862472781 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46c807de70 0x7f46c807c2d0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f46bc009820 tx=0x7f46bc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8fbd35915f511c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:09.858 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:09.137+0000 7f46c7e00640 1 -- 172.21.15.77:0/862472781 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f46bc037070 con 0x7f46c807de70 2024-09-10T09:44:09.858 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:09.137+0000 7f46c7e00640 1 -- 172.21.15.77:0/862472781 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46bc032440 con 0x7f46c807de70 2024-09-10T09:44:09.858 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:09.137+0000 7f46c7e00640 1 -- 172.21.15.77:0/862472781 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f46bc032740 con 0x7f46c807de70 2024-09-10T09:44:09.858 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:09.137+0000 7f46cec00640 1 -- 172.21.15.77:0/862472781 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46c807de70 msgr2=0x7f46c807c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:09.858 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:09.137+0000 7f46cec00640 1 --2- 172.21.15.77:0/862472781 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46c807de70 0x7f46c807c2d0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f46bc009820 tx=0x7f46bc02a270 comp rx=0 tx=0).stop 2024-09-10T09:44:09.858 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:09.137+0000 7f46cec00640 1 -- 172.21.15.77:0/862472781 shutdown_connections 2024-09-10T09:44:09.858 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:09.137+0000 7f46cec00640 1 --2- 172.21.15.77:0/862472781 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46c807de70 0x7f46c807c2d0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:09.858 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:09.137+0000 7f46cec00640 1 -- 172.21.15.77:0/862472781 >> 172.21.15.77:0/862472781 conn(0x7f46c8102b40 msgr2=0x7f46c8104fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:09.858 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:09.138+0000 7f46cec00640 1 -- 172.21.15.77:0/862472781 shutdown_connections 2024-09-10T09:44:09.858 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:09.138+0000 7f46cec00640 1 -- 172.21.15.77:0/862472781 wait complete. 2024-09-10T09:44:09.858 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:09.138+0000 7f46cec00640 1 Processor -- start 2024-09-10T09:44:09.859 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:09.139+0000 7f46cec00640 1 -- start start 2024-09-10T09:44:09.859 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:09.139+0000 7f46cec00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46c807de70 0x7f46c819d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:09.859 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:09.139+0000 7f46cec00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f46c807cf50 con 0x7f46c807de70 2024-09-10T09:44:09.859 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:09.139+0000 7f46cd800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46c807de70 0x7f46c819d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:09.859 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:09.139+0000 7f46cd800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46c807de70 0x7f46c819d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58500/0 (socket says 172.21.15.77:58500) 2024-09-10T09:44:09.859 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:09.139+0000 7f46cd800640 1 -- 172.21.15.77:0/2706458163 learned_addr learned my addr 172.21.15.77:0/2706458163 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:44:09.859 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:09.140+0000 7f46cd800640 1 -- 172.21.15.77:0/2706458163 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46c819d5e0 con 0x7f46c807de70 2024-09-10T09:44:09.859 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:09.140+0000 7f46cd800640 1 --2- 172.21.15.77:0/2706458163 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46c807de70 0x7f46c819d0a0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f46bc0090b0 tx=0x7f46bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:09.859 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:09.140+0000 7f46c6000640 1 -- 172.21.15.77:0/2706458163 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f46bc041070 con 0x7f46c807de70 2024-09-10T09:44:09.859 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:09.140+0000 7f46c6000640 1 -- 172.21.15.77:0/2706458163 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46bc03c7a0 con 0x7f46c807de70 2024-09-10T09:44:09.859 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:09.140+0000 7f46cec00640 1 -- 172.21.15.77:0/2706458163 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46c819d870 con 0x7f46c807de70 2024-09-10T09:44:09.859 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:09.140+0000 7f46c6000640 1 -- 172.21.15.77:0/2706458163 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f46bc037040 con 0x7f46c807de70 2024-09-10T09:44:09.859 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:09.140+0000 7f46cec00640 1 -- 172.21.15.77:0/2706458163 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46c81a0560 con 0x7f46c807de70 2024-09-10T09:44:09.859 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:09.140+0000 7f46c6000640 1 -- 172.21.15.77:0/2706458163 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f46bc044070 con 0x7f46c807de70 2024-09-10T09:44:09.860 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:09.141+0000 7f46c6000640 1 -- 172.21.15.77:0/2706458163 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f46bc03d3f0 con 0x7f46c807de70 2024-09-10T09:44:09.860 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:09.141+0000 7f46cec00640 1 -- 172.21.15.77:0/2706458163 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46c807c350 con 0x7f46c807de70 2024-09-10T09:44:09.860 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:09.144+0000 7f46c6000640 1 -- 172.21.15.77:0/2706458163 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f46bc046450 con 0x7f46c807de70 2024-09-10T09:44:09.860 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:09.204+0000 7f46cec00640 1 -- 172.21.15.77:0/2706458163 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f46c8075f40 con 0x7f46c807de70 2024-09-10T09:44:09.860 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:09.208+0000 7f46c6000640 1 -- 172.21.15.77:0/2706458163 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+429 (secure 0 0 0) 0x7f46bc046630 con 0x7f46c807de70 2024-09-10T09:44:09.860 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:09.209+0000 7f46c6000640 1 -- 172.21.15.77:0/2706458163 <== mon.0 v2:172.21.15.77:3300/0 8 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f46bc0584b0 con 0x7f46c807de70 2024-09-10T09:44:09.860 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:09.210+0000 7f46cec00640 1 -- 172.21.15.77:0/2706458163 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46c807de70 msgr2=0x7f46c819d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:09.860 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:09.210+0000 7f46cec00640 1 --2- 172.21.15.77:0/2706458163 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46c807de70 0x7f46c819d0a0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f46bc0090b0 tx=0x7f46bc0046c0 comp rx=0 tx=0).stop 2024-09-10T09:44:09.860 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:09.210+0000 7f46cec00640 1 -- 172.21.15.77:0/2706458163 shutdown_connections 2024-09-10T09:44:09.860 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:09.210+0000 7f46cec00640 1 --2- 172.21.15.77:0/2706458163 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46c807de70 0x7f46c819d0a0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:09.860 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:09.210+0000 7f46cec00640 1 -- 172.21.15.77:0/2706458163 >> 172.21.15.77:0/2706458163 conn(0x7f46c8102b40 msgr2=0x7f46c8104230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:09.860 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:09.210+0000 7f46cec00640 1 -- 172.21.15.77:0/2706458163 shutdown_connections 2024-09-10T09:44:09.860 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:09.210+0000 7f46cec00640 1 -- 172.21.15.77:0/2706458163 wait complete. 2024-09-10T09:44:09.861 INFO:teuthology.orchestra.run.smithi077.stdout:Generating new minimal ceph.conf... 2024-09-10T09:44:10.689 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:10 smithi077 ceph-mon[29953]: from='client.? 172.21.15.77:0/1062770598' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:44:11.001 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:10.315+0000 7fa3f6600640 1 Processor -- start 2024-09-10T09:44:11.001 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:10.316+0000 7fa3f6600640 1 -- start start 2024-09-10T09:44:11.001 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:10.316+0000 7fa3f6600640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa3f007de50 0x7fa3f007c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:11.001 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:10.316+0000 7fa3f6600640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f007c880 con 0x7fa3f007de50 2024-09-10T09:44:11.001 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:10.316+0000 7fa3ef400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa3f007de50 0x7fa3f007c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:11.001 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:10.316+0000 7fa3ef400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa3f007de50 0x7fa3f007c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58506/0 (socket says 172.21.15.77:58506) 2024-09-10T09:44:11.001 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:10.316+0000 7fa3ef400640 1 -- 172.21.15.77:0/3936778791 learned_addr learned my addr 172.21.15.77:0/3936778791 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:44:11.001 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:10.316+0000 7fa3ef400640 1 -- 172.21.15.77:0/3936778791 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3f007ca00 con 0x7fa3f007de50 2024-09-10T09:44:11.001 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:10.317+0000 7fa3ef400640 1 --2- 172.21.15.77:0/3936778791 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa3f007de50 0x7fa3f007c2b0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fa3e0009820 tx=0x7fa3e002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ad3aa6571628531 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:11.001 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:10.318+0000 7fa3eea00640 1 -- 172.21.15.77:0/3936778791 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa3e0037070 con 0x7fa3f007de50 2024-09-10T09:44:11.001 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:10.318+0000 7fa3eea00640 1 -- 172.21.15.77:0/3936778791 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa3e0032900 con 0x7fa3f007de50 2024-09-10T09:44:11.002 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:10.318+0000 7fa3eea00640 1 -- 172.21.15.77:0/3936778791 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa3e0030850 con 0x7fa3f007de50 2024-09-10T09:44:11.002 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:10.318+0000 7fa3f6600640 1 -- 172.21.15.77:0/3936778791 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa3f007de50 msgr2=0x7fa3f007c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:11.002 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:10.318+0000 7fa3f6600640 1 --2- 172.21.15.77:0/3936778791 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa3f007de50 0x7fa3f007c2b0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fa3e0009820 tx=0x7fa3e002a270 comp rx=0 tx=0).stop 2024-09-10T09:44:11.002 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:10.318+0000 7fa3f6600640 1 -- 172.21.15.77:0/3936778791 shutdown_connections 2024-09-10T09:44:11.002 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:10.318+0000 7fa3f6600640 1 --2- 172.21.15.77:0/3936778791 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa3f007de50 0x7fa3f007c2b0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:11.002 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:10.318+0000 7fa3f6600640 1 -- 172.21.15.77:0/3936778791 >> 172.21.15.77:0/3936778791 conn(0x7fa3f0102cf0 msgr2=0x7fa3f0105130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:11.002 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:10.319+0000 7fa3f6600640 1 -- 172.21.15.77:0/3936778791 shutdown_connections 2024-09-10T09:44:11.002 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:10.319+0000 7fa3f6600640 1 -- 172.21.15.77:0/3936778791 wait complete. 2024-09-10T09:44:11.002 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:10.319+0000 7fa3f6600640 1 Processor -- start 2024-09-10T09:44:11.002 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:10.320+0000 7fa3f6600640 1 -- start start 2024-09-10T09:44:11.002 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:10.320+0000 7fa3f6600640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa3f007de50 0x7fa3f019db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:11.002 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:10.320+0000 7fa3f6600640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f010dbd0 con 0x7fa3f007de50 2024-09-10T09:44:11.002 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:10.320+0000 7fa3ef400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa3f007de50 0x7fa3f019db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:11.003 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:10.320+0000 7fa3ef400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa3f007de50 0x7fa3f019db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58510/0 (socket says 172.21.15.77:58510) 2024-09-10T09:44:11.003 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:10.320+0000 7fa3ef400640 1 -- 172.21.15.77:0/1062770598 learned_addr learned my addr 172.21.15.77:0/1062770598 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:44:11.003 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:10.320+0000 7fa3ef400640 1 -- 172.21.15.77:0/1062770598 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3f019e040 con 0x7fa3f007de50 2024-09-10T09:44:11.003 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:10.321+0000 7fa3ef400640 1 --2- 172.21.15.77:0/1062770598 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa3f007de50 0x7fa3f019db00 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa3e00097f0 tx=0x7fa3e0030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:11.003 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:10.321+0000 7fa3ecc00640 1 -- 172.21.15.77:0/1062770598 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa3e0042070 con 0x7fa3f007de50 2024-09-10T09:44:11.003 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:10.321+0000 7fa3ecc00640 1 -- 172.21.15.77:0/1062770598 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa3e0032ca0 con 0x7fa3f007de50 2024-09-10T09:44:11.003 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:10.321+0000 7fa3ecc00640 1 -- 172.21.15.77:0/1062770598 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa3e0037040 con 0x7fa3f007de50 2024-09-10T09:44:11.003 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:10.321+0000 7fa3f6600640 1 -- 172.21.15.77:0/1062770598 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3f019e2d0 con 0x7fa3f007de50 2024-09-10T09:44:11.003 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:10.321+0000 7fa3f6600640 1 -- 172.21.15.77:0/1062770598 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3f01a0fc0 con 0x7fa3f007de50 2024-09-10T09:44:11.003 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:10.321+0000 7fa3ecc00640 1 -- 172.21.15.77:0/1062770598 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa3e0045070 con 0x7fa3f007de50 2024-09-10T09:44:11.003 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:10.322+0000 7fa3ecc00640 1 -- 172.21.15.77:0/1062770598 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa3e0047af0 con 0x7fa3f007de50 2024-09-10T09:44:11.003 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:10.322+0000 7fa3f6600640 1 -- 172.21.15.77:0/1062770598 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3f007c330 con 0x7fa3f007de50 2024-09-10T09:44:11.003 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:10.325+0000 7fa3ecc00640 1 -- 172.21.15.77:0/1062770598 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fa3e003db10 con 0x7fa3f007de50 2024-09-10T09:44:11.004 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:10.384+0000 7fa3f6600640 1 -- 172.21.15.77:0/1062770598 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fa3f0075f10 con 0x7fa3f007de50 2024-09-10T09:44:11.004 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:10.384+0000 7fa3ecc00640 1 -- 172.21.15.77:0/1062770598 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7fa3e003d430 con 0x7fa3f007de50 2024-09-10T09:44:11.004 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:10.385+0000 7fa3f6600640 1 -- 172.21.15.77:0/1062770598 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa3f007de50 msgr2=0x7fa3f019db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:11.004 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:10.385+0000 7fa3f6600640 1 --2- 172.21.15.77:0/1062770598 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa3f007de50 0x7fa3f019db00 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa3e00097f0 tx=0x7fa3e0030ec0 comp rx=0 tx=0).stop 2024-09-10T09:44:11.004 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:10.385+0000 7fa3f6600640 1 -- 172.21.15.77:0/1062770598 shutdown_connections 2024-09-10T09:44:11.004 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:10.385+0000 7fa3f6600640 1 --2- 172.21.15.77:0/1062770598 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa3f007de50 0x7fa3f019db00 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:11.004 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:10.385+0000 7fa3f6600640 1 -- 172.21.15.77:0/1062770598 >> 172.21.15.77:0/1062770598 conn(0x7fa3f0102cf0 msgr2=0x7fa3f0103ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:11.004 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:10.386+0000 7fa3f6600640 1 -- 172.21.15.77:0/1062770598 shutdown_connections 2024-09-10T09:44:11.004 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:10.386+0000 7fa3f6600640 1 -- 172.21.15.77:0/1062770598 wait complete. 2024-09-10T09:44:11.004 INFO:teuthology.orchestra.run.smithi077.stdout:Restarting the monitor... 2024-09-10T09:44:11.338 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:11 smithi077 systemd[1]: Stopping Ceph mon.a for 14a55226-6f59-11ef-bcea-c7b262605968... 2024-09-10T09:44:11.689 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:11 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mon-a[29949]: 2024-09-10T09:44:11.336+0000 7f023f200640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T09:44:11.689 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:11 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mon-a[29949]: 2024-09-10T09:44:11.336+0000 7f023f200640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-10T09:44:11.689 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:11 smithi077 podman[30194]: 2024-09-10 09:44:11.56261138 +0000 UTC m=+0.345846196 container died b9644962f5694e87b0493a4e4da8899fab6abf736a9a1d8c3f878ffafbdce85e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-mon-a, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T09:44:12.126 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:11 smithi077 podman[30194]: 2024-09-10 09:44:11.974294546 +0000 UTC m=+0.757529357 container cleanup b9644962f5694e87b0493a4e4da8899fab6abf736a9a1d8c3f878ffafbdce85e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-mon-a, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-10T09:44:12.126 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:11 smithi077 bash[30194]: ceph-14a55226-6f59-11ef-bcea-c7b262605968-mon-a 2024-09-10T09:44:12.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:12 smithi077 podman[30209]: 2024-09-10 09:44:12.325281937 +0000 UTC m=+0.747972024 container remove b9644962f5694e87b0493a4e4da8899fab6abf736a9a1d8c3f878ffafbdce85e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-mon-a, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-10T09:44:12.778 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:12 smithi077 systemd[1]: ceph-14a55226-6f59-11ef-bcea-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-10T09:44:12.778 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:12 smithi077 systemd[1]: Stopped Ceph mon.a for 14a55226-6f59-11ef-bcea-c7b262605968. 2024-09-10T09:44:12.779 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:12 smithi077 systemd[1]: ceph-14a55226-6f59-11ef-bcea-c7b262605968@mon.a.service: Consumed 1.207s CPU time. 2024-09-10T09:44:12.779 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:12 smithi077 systemd[1]: Starting Ceph mon.a for 14a55226-6f59-11ef-bcea-c7b262605968... 2024-09-10T09:44:13.029 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:12 smithi077 podman[30297]: 2024-09-10 09:44:12.685706056 +0000 UTC m=+0.031263531 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T09:44:13.029 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:12 smithi077 podman[30297]: 2024-09-10 09:44:12.784276145 +0000 UTC m=+0.129833603 container create f32e55b70c78a46742af30d82a80ca5f00f5d5478a5be5faf690e79bd873f941 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-mon-a, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-10T09:44:13.029 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 podman[30297]: 2024-09-10 09:44:13.028467278 +0000 UTC m=+0.374024750 container init f32e55b70c78a46742af30d82a80ca5f00f5d5478a5be5faf690e79bd873f941 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-mon-a, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux ) 2024-09-10T09:44:13.129 INFO:teuthology.orchestra.run.smithi077.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-10T09:44:13.361 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 podman[30297]: 2024-09-10 09:44:13.03525679 +0000 UTC m=+0.380814250 container start f32e55b70c78a46742af30d82a80ca5f00f5d5478a5be5faf690e79bd873f941 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-mon-a, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, CEPH_POINT_RELEASE=) 2024-09-10T09:44:13.362 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: set uid:gid to 167:167 (ceph:ceph) 2024-09-10T09:44:13.362 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable), process ceph-mon, pid 2 2024-09-10T09:44:13.362 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: pidfile_write: ignore empty --pid-file 2024-09-10T09:44:13.362 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: load: jerasure load: lrc 2024-09-10T09:44:13.362 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: RocksDB version: 7.9.2 2024-09-10T09:44:13.362 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Git sha 0 2024-09-10T09:44:13.362 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Compile date 2024-09-09 20:43:47 2024-09-10T09:44:13.362 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: DB SUMMARY 2024-09-10T09:44:13.362 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: DB Session ID: 4RZBB264KHI7KQGTL5U1 2024-09-10T09:44:13.362 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: CURRENT file: CURRENT 2024-09-10T09:44:13.362 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: IDENTITY file: IDENTITY 2024-09-10T09:44:13.362 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-10T09:44:13.363 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-10T09:44:13.363 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 104828 ; 2024-09-10T09:44:13.363 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.error_if_exists: 0 2024-09-10T09:44:13.363 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.create_if_missing: 0 2024-09-10T09:44:13.363 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.paranoid_checks: 1 2024-09-10T09:44:13.363 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-10T09:44:13.363 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-10T09:44:13.363 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-10T09:44:13.363 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.env: 0x563b57255940 2024-09-10T09:44:13.363 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.fs: PosixFileSystem 2024-09-10T09:44:13.363 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.info_log: 0x563b6a4036a0 2024-09-10T09:44:13.363 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.max_file_opening_threads: 16 2024-09-10T09:44:13.363 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.statistics: (nil) 2024-09-10T09:44:13.364 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.use_fsync: 0 2024-09-10T09:44:13.364 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.max_log_file_size: 0 2024-09-10T09:44:13.364 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-10T09:44:13.364 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-10T09:44:13.364 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.keep_log_file_num: 1000 2024-09-10T09:44:13.364 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.recycle_log_file_num: 0 2024-09-10T09:44:13.364 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.allow_fallocate: 1 2024-09-10T09:44:13.364 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.allow_mmap_reads: 0 2024-09-10T09:44:13.364 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.allow_mmap_writes: 0 2024-09-10T09:44:13.364 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.use_direct_reads: 0 2024-09-10T09:44:13.364 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-10T09:44:13.364 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.create_missing_column_families: 0 2024-09-10T09:44:13.365 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.db_log_dir: 2024-09-10T09:44:13.365 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.wal_dir: 2024-09-10T09:44:13.365 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-10T09:44:13.365 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-10T09:44:13.365 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-10T09:44:13.365 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-10T09:44:13.365 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-10T09:44:13.365 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-10T09:44:13.365 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.advise_random_on_open: 1 2024-09-10T09:44:13.365 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.db_write_buffer_size: 0 2024-09-10T09:44:13.365 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.write_buffer_manager: 0x563b6a412dc0 2024-09-10T09:44:13.365 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-10T09:44:13.365 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-10T09:44:13.366 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-10T09:44:13.366 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.rate_limiter: (nil) 2024-09-10T09:44:13.366 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-10T09:44:13.366 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.wal_recovery_mode: 2 2024-09-10T09:44:13.366 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.enable_thread_tracking: 0 2024-09-10T09:44:13.366 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.enable_pipelined_write: 0 2024-09-10T09:44:13.366 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.unordered_write: 0 2024-09-10T09:44:13.366 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-10T09:44:13.366 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-10T09:44:13.366 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-10T09:44:13.366 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-10T09:44:13.366 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.row_cache: None 2024-09-10T09:44:13.366 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.wal_filter: None 2024-09-10T09:44:13.366 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-10T09:44:13.367 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.allow_ingest_behind: 0 2024-09-10T09:44:13.367 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.two_write_queues: 0 2024-09-10T09:44:13.367 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.manual_wal_flush: 0 2024-09-10T09:44:13.367 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.wal_compression: 0 2024-09-10T09:44:13.367 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.atomic_flush: 0 2024-09-10T09:44:13.367 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-10T09:44:13.367 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-10T09:44:13.367 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-10T09:44:13.368 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.log_readahead_size: 0 2024-09-10T09:44:13.368 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-10T09:44:13.368 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.best_efforts_recovery: 0 2024-09-10T09:44:13.369 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-10T09:44:13.369 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-10T09:44:13.369 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.allow_data_in_errors: 0 2024-09-10T09:44:13.369 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.db_host_id: __hostname__ 2024-09-10T09:44:13.369 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-10T09:44:13.369 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.max_background_jobs: 2 2024-09-10T09:44:13.369 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.max_background_compactions: -1 2024-09-10T09:44:13.369 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.max_subcompactions: 1 2024-09-10T09:44:13.369 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-10T09:44:13.369 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-10T09:44:13.369 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-10T09:44:13.369 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.max_total_wal_size: 0 2024-09-10T09:44:13.369 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-10T09:44:13.370 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-10T09:44:13.370 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-10T09:44:13.370 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-10T09:44:13.370 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.max_open_files: -1 2024-09-10T09:44:13.370 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.bytes_per_sync: 0 2024-09-10T09:44:13.370 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-10T09:44:13.370 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-10T09:44:13.370 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.compaction_readahead_size: 0 2024-09-10T09:44:13.370 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.max_background_flushes: -1 2024-09-10T09:44:13.370 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Compression algorithms supported: 2024-09-10T09:44:13.370 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: kZSTD supported: 0 2024-09-10T09:44:13.370 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: kXpressCompression supported: 0 2024-09-10T09:44:13.370 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: kBZip2Compression supported: 0 2024-09-10T09:44:13.371 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-10T09:44:13.371 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: kLZ4Compression supported: 1 2024-09-10T09:44:13.371 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: kZlibCompression supported: 1 2024-09-10T09:44:13.371 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: kLZ4HCCompression supported: 1 2024-09-10T09:44:13.371 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: kSnappyCompression supported: 1 2024-09-10T09:44:13.371 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-10T09:44:13.371 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-10T09:44:13.371 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-10T09:44:13.371 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-10T09:44:13.371 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-10T09:44:13.371 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.merge_operator: 2024-09-10T09:44:13.371 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.compaction_filter: None 2024-09-10T09:44:13.371 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.compaction_filter_factory: None 2024-09-10T09:44:13.371 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.sst_partitioner_factory: None 2024-09-10T09:44:13.371 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-10T09:44:13.372 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-10T09:44:13.372 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x563b6a403300) 2024-09-10T09:44:13.372 INFO:journalctl@ceph.mon.a.smithi077.stdout: cache_index_and_filter_blocks: 1 2024-09-10T09:44:13.372 INFO:journalctl@ceph.mon.a.smithi077.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-10T09:44:13.372 INFO:journalctl@ceph.mon.a.smithi077.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-10T09:44:13.372 INFO:journalctl@ceph.mon.a.smithi077.stdout: pin_top_level_index_and_filter: 1 2024-09-10T09:44:13.372 INFO:journalctl@ceph.mon.a.smithi077.stdout: index_type: 0 2024-09-10T09:44:13.372 INFO:journalctl@ceph.mon.a.smithi077.stdout: data_block_index_type: 0 2024-09-10T09:44:13.372 INFO:journalctl@ceph.mon.a.smithi077.stdout: index_shortening: 1 2024-09-10T09:44:13.372 INFO:journalctl@ceph.mon.a.smithi077.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-10T09:44:13.372 INFO:journalctl@ceph.mon.a.smithi077.stdout: checksum: 4 2024-09-10T09:44:13.372 INFO:journalctl@ceph.mon.a.smithi077.stdout: no_block_cache: 0 2024-09-10T09:44:13.372 INFO:journalctl@ceph.mon.a.smithi077.stdout: block_cache: 0x563b6a3f71f0 2024-09-10T09:44:13.373 INFO:journalctl@ceph.mon.a.smithi077.stdout: block_cache_name: BinnedLRUCache 2024-09-10T09:44:13.373 INFO:journalctl@ceph.mon.a.smithi077.stdout: block_cache_options: 2024-09-10T09:44:13.373 INFO:journalctl@ceph.mon.a.smithi077.stdout: capacity : 536870912 2024-09-10T09:44:13.373 INFO:journalctl@ceph.mon.a.smithi077.stdout: num_shard_bits : 4 2024-09-10T09:44:13.373 INFO:journalctl@ceph.mon.a.smithi077.stdout: strict_capacity_limit : 0 2024-09-10T09:44:13.373 INFO:journalctl@ceph.mon.a.smithi077.stdout: high_pri_pool_ratio: 0.000 2024-09-10T09:44:13.373 INFO:journalctl@ceph.mon.a.smithi077.stdout: block_cache_compressed: (nil) 2024-09-10T09:44:13.373 INFO:journalctl@ceph.mon.a.smithi077.stdout: persistent_cache: (nil) 2024-09-10T09:44:13.373 INFO:journalctl@ceph.mon.a.smithi077.stdout: block_size: 4096 2024-09-10T09:44:13.373 INFO:journalctl@ceph.mon.a.smithi077.stdout: block_size_deviation: 10 2024-09-10T09:44:13.373 INFO:journalctl@ceph.mon.a.smithi077.stdout: block_restart_interval: 16 2024-09-10T09:44:13.373 INFO:journalctl@ceph.mon.a.smithi077.stdout: index_block_restart_interval: 1 2024-09-10T09:44:13.373 INFO:journalctl@ceph.mon.a.smithi077.stdout: metadata_block_size: 4096 2024-09-10T09:44:13.374 INFO:journalctl@ceph.mon.a.smithi077.stdout: partition_filters: 0 2024-09-10T09:44:13.374 INFO:journalctl@ceph.mon.a.smithi077.stdout: use_delta_encoding: 1 2024-09-10T09:44:13.374 INFO:journalctl@ceph.mon.a.smithi077.stdout: filter_policy: bloomfilter 2024-09-10T09:44:13.374 INFO:journalctl@ceph.mon.a.smithi077.stdout: whole_key_filtering: 1 2024-09-10T09:44:13.374 INFO:journalctl@ceph.mon.a.smithi077.stdout: verify_compression: 0 2024-09-10T09:44:13.374 INFO:journalctl@ceph.mon.a.smithi077.stdout: read_amp_bytes_per_bit: 0 2024-09-10T09:44:13.374 INFO:journalctl@ceph.mon.a.smithi077.stdout: format_version: 5 2024-09-10T09:44:13.374 INFO:journalctl@ceph.mon.a.smithi077.stdout: enable_index_compression: 1 2024-09-10T09:44:13.374 INFO:journalctl@ceph.mon.a.smithi077.stdout: block_align: 0 2024-09-10T09:44:13.374 INFO:journalctl@ceph.mon.a.smithi077.stdout: max_auto_readahead_size: 262144 2024-09-10T09:44:13.374 INFO:journalctl@ceph.mon.a.smithi077.stdout: prepopulate_block_cache: 0 2024-09-10T09:44:13.374 INFO:journalctl@ceph.mon.a.smithi077.stdout: initial_auto_readahead_size: 8192 2024-09-10T09:44:13.374 INFO:journalctl@ceph.mon.a.smithi077.stdout: num_file_reads_for_auto_readahead: 2 2024-09-10T09:44:13.374 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.write_buffer_size: 33554432 2024-09-10T09:44:13.374 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.max_write_buffer_number: 2 2024-09-10T09:44:13.375 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.compression: NoCompression 2024-09-10T09:44:13.375 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.bottommost_compression: Disabled 2024-09-10T09:44:13.375 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.prefix_extractor: nullptr 2024-09-10T09:44:13.375 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-10T09:44:13.375 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.num_levels: 7 2024-09-10T09:44:13.375 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-10T09:44:13.375 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-10T09:44:13.375 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-10T09:44:13.375 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-10T09:44:13.375 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-10T09:44:13.375 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-10T09:44:13.376 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-10T09:44:13.376 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-10T09:44:13.376 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-10T09:44:13.376 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-10T09:44:13.376 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-10T09:44:13.376 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-10T09:44:13.376 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-10T09:44:13.376 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.compression_opts.level: 32767 2024-09-10T09:44:13.376 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.compression_opts.strategy: 0 2024-09-10T09:44:13.376 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-10T09:44:13.376 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-10T09:44:13.376 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-10T09:44:13.376 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-10T09:44:13.377 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.compression_opts.enabled: false 2024-09-10T09:44:13.377 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-10T09:44:13.377 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-10T09:44:13.377 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-10T09:44:13.377 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-10T09:44:13.377 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.target_file_size_base: 67108864 2024-09-10T09:44:13.377 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-10T09:44:13.377 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-10T09:44:13.377 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-10T09:44:13.377 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-10T09:44:13.377 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-10T09:44:13.378 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-10T09:44:13.378 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-10T09:44:13.378 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-10T09:44:13.378 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-10T09:44:13.378 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-10T09:44:13.378 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-10T09:44:13.378 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-10T09:44:13.378 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-10T09:44:13.378 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-10T09:44:13.378 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.arena_block_size: 1048576 2024-09-10T09:44:13.378 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-10T09:44:13.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-10T09:44:13.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.disable_auto_compactions: 0 2024-09-10T09:44:13.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-10T09:44:13.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-10T09:44:13.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-10T09:44:13.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-10T09:44:13.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-10T09:44:13.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-10T09:44:13.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-10T09:44:13.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-10T09:44:13.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-10T09:44:13.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-10T09:44:13.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-10T09:44:13.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.inplace_update_support: 0 2024-09-10T09:44:13.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-10T09:44:13.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-10T09:44:13.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-10T09:44:13.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-10T09:44:13.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.bloom_locality: 0 2024-09-10T09:44:13.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.max_successive_merges: 0 2024-09-10T09:44:13.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-10T09:44:13.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.paranoid_file_checks: 0 2024-09-10T09:44:13.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.force_consistency_checks: 1 2024-09-10T09:44:13.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.report_bg_io_stats: 0 2024-09-10T09:44:13.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.ttl: 2592000 2024-09-10T09:44:13.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-10T09:44:13.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-10T09:44:13.381 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-10T09:44:13.381 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.enable_blob_files: false 2024-09-10T09:44:13.381 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.min_blob_size: 0 2024-09-10T09:44:13.381 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.blob_file_size: 268435456 2024-09-10T09:44:13.381 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-10T09:44:13.381 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-10T09:44:13.381 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-10T09:44:13.381 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-10T09:44:13.381 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-10T09:44:13.381 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.blob_file_starting_level: 0 2024-09-10T09:44:13.381 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-10T09:44:13.381 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-10T09:44:13.382 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-10T09:44:13.382 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 1d2bcc7c-d8f6-4e5f-ac8d-ec2c0e3a6d29 2024-09-10T09:44:13.382 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725961453079526, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-10T09:44:13.382 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-10T09:44:13.382 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725961453082234, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 99932, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 326, "table_properties": {"data_size": 97818, "index_size": 306, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14179, "raw_average_key_size": 49, "raw_value_size": 90154, "raw_average_value_size": 316, "num_data_blocks": 12, "num_entries": 285, "num_filter_entries": 285, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1725961453, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "1d2bcc7c-d8f6-4e5f-ac8d-ec2c0e3a6d29", "db_session_id": "4RZBB264KHI7KQGTL5U1", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-10T09:44:13.382 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725961453082611, "job": 1, "event": "recovery_finished"} 2024-09-10T09:44:13.382 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-10T09:44:13.382 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-10T09:44:13.382 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x563b6a426e00 2024-09-10T09:44:13.382 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: DB pointer 0x563b6a522000 2024-09-10T09:44:13.382 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-10T09:44:13.382 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-10T09:44:13.382 INFO:journalctl@ceph.mon.a.smithi077.stdout: ** DB Stats ** 2024-09-10T09:44:13.383 INFO:journalctl@ceph.mon.a.smithi077.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T09:44:13.383 INFO:journalctl@ceph.mon.a.smithi077.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-10T09:44:13.383 INFO:journalctl@ceph.mon.a.smithi077.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T09:44:13.383 INFO:journalctl@ceph.mon.a.smithi077.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T09:44:13.383 INFO:journalctl@ceph.mon.a.smithi077.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-10T09:44:13.383 INFO:journalctl@ceph.mon.a.smithi077.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T09:44:13.383 INFO:journalctl@ceph.mon.a.smithi077.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T09:44:13.383 INFO:journalctl@ceph.mon.a.smithi077.stdout: 2024-09-10T09:44:13.383 INFO:journalctl@ceph.mon.a.smithi077.stdout: ** Compaction Stats [default] ** 2024-09-10T09:44:13.384 INFO:journalctl@ceph.mon.a.smithi077.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-10T09:44:13.384 INFO:journalctl@ceph.mon.a.smithi077.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-10T09:44:13.384 INFO:journalctl@ceph.mon.a.smithi077.stdout: L0 2/0 99.45 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 60.2 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-09-10T09:44:13.384 INFO:journalctl@ceph.mon.a.smithi077.stdout: Sum 2/0 99.45 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 60.2 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-09-10T09:44:13.384 INFO:journalctl@ceph.mon.a.smithi077.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 60.2 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-09-10T09:44:13.384 INFO:journalctl@ceph.mon.a.smithi077.stdout: 2024-09-10T09:44:13.384 INFO:journalctl@ceph.mon.a.smithi077.stdout: ** Compaction Stats [default] ** 2024-09-10T09:44:13.385 INFO:journalctl@ceph.mon.a.smithi077.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-10T09:44:13.385 INFO:journalctl@ceph.mon.a.smithi077.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T09:44:13.385 INFO:journalctl@ceph.mon.a.smithi077.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 60.2 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-09-10T09:44:13.385 INFO:journalctl@ceph.mon.a.smithi077.stdout: 2024-09-10T09:44:13.385 INFO:journalctl@ceph.mon.a.smithi077.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-10T09:44:13.385 INFO:journalctl@ceph.mon.a.smithi077.stdout: 2024-09-10T09:44:13.385 INFO:journalctl@ceph.mon.a.smithi077.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T09:44:13.385 INFO:journalctl@ceph.mon.a.smithi077.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-10T09:44:13.385 INFO:journalctl@ceph.mon.a.smithi077.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-10T09:44:13.385 INFO:journalctl@ceph.mon.a.smithi077.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-10T09:44:13.385 INFO:journalctl@ceph.mon.a.smithi077.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-10T09:44:13.386 INFO:journalctl@ceph.mon.a.smithi077.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-10T09:44:13.386 INFO:journalctl@ceph.mon.a.smithi077.stdout: Cumulative compaction: 0.00 GB write, 10.23 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T09:44:13.386 INFO:journalctl@ceph.mon.a.smithi077.stdout: Interval compaction: 0.00 GB write, 10.23 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T09:44:13.386 INFO:journalctl@ceph.mon.a.smithi077.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-10T09:44:13.386 INFO:journalctl@ceph.mon.a.smithi077.stdout: Block cache BinnedLRUCache@0x563b6a3f71f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 3.8e-05 secs_since: 0 2024-09-10T09:44:13.388 INFO:journalctl@ceph.mon.a.smithi077.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-09-10T09:44:13.388 INFO:journalctl@ceph.mon.a.smithi077.stdout: 2024-09-10T09:44:13.388 INFO:journalctl@ceph.mon.a.smithi077.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-10T09:44:13.388 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: starting mon.a rank 0 at public addrs [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] at bind addrs [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 14a55226-6f59-11ef-bcea-c7b262605968 2024-09-10T09:44:13.388 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: mon.a@-1(???) e1 preinit fsid 14a55226-6f59-11ef-bcea-c7b262605968 2024-09-10T09:44:13.388 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: mon.a@-1(???).mds e1 new map 2024-09-10T09:44:13.388 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: mon.a@-1(???).mds e1 print_map 2024-09-10T09:44:13.388 INFO:journalctl@ceph.mon.a.smithi077.stdout: e1 2024-09-10T09:44:13.388 INFO:journalctl@ceph.mon.a.smithi077.stdout: btime 2024-09-10T09:44:07:359254+0000 2024-09-10T09:44:13.389 INFO:journalctl@ceph.mon.a.smithi077.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T09:44:13.389 INFO:journalctl@ceph.mon.a.smithi077.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T09:44:13.389 INFO:journalctl@ceph.mon.a.smithi077.stdout: legacy client fscid: -1 2024-09-10T09:44:13.389 INFO:journalctl@ceph.mon.a.smithi077.stdout: 2024-09-10T09:44:13.389 INFO:journalctl@ceph.mon.a.smithi077.stdout: No filesystems configured 2024-09-10T09:44:13.389 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-10T09:44:13.389 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T09:44:13.389 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T09:44:13.389 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T09:44:13.389 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-10T09:44:13.389 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 bash[30297]: f32e55b70c78a46742af30d82a80ca5f00f5d5478a5be5faf690e79bd873f941 2024-09-10T09:44:13.389 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 systemd[1]: Started Ceph mon.a for 14a55226-6f59-11ef-bcea-c7b262605968. 2024-09-10T09:44:13.390 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-10T09:44:13.390 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: monmap epoch 1 2024-09-10T09:44:13.390 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: fsid 14a55226-6f59-11ef-bcea-c7b262605968 2024-09-10T09:44:13.390 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: last_changed 2024-09-10T09:44:03.655548+0000 2024-09-10T09:44:13.390 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: created 2024-09-10T09:44:03.655548+0000 2024-09-10T09:44:13.390 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: min_mon_release 19 (squid) 2024-09-10T09:44:13.390 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: election_strategy: 1 2024-09-10T09:44:13.390 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: 0: [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] mon.a 2024-09-10T09:44:13.390 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: fsmap 2024-09-10T09:44:13.390 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: osdmap e1: 0 total, 0 up, 0 in 2024-09-10T09:44:13.390 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:13 smithi077 ceph-mon[30319]: mgrmap e1: no daemons active 2024-09-10T09:44:14.278 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:13.593+0000 7f364d600640 1 Processor -- start 2024-09-10T09:44:14.279 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:13.594+0000 7f364d600640 1 -- start start 2024-09-10T09:44:14.279 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:13.594+0000 7f364d600640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3648109870 0x7f3648109c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:14.279 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:13.594+0000 7f364d600640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f364810a240 con 0x7f3648109870 2024-09-10T09:44:14.279 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:13.594+0000 7f3646a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3648109870 0x7f3648109c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:14.279 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:13.595+0000 7f3646a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3648109870 0x7f3648109c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:37512/0 (socket says 172.21.15.77:37512) 2024-09-10T09:44:14.279 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:13.595+0000 7f3646a00640 1 -- 172.21.15.77:0/1221249480 learned_addr learned my addr 172.21.15.77:0/1221249480 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:44:14.279 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:13.595+0000 7f3646a00640 1 -- 172.21.15.77:0/1221249480 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f364810a980 con 0x7f3648109870 2024-09-10T09:44:14.279 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:13.598+0000 7f3646a00640 1 --2- 172.21.15.77:0/1221249480 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3648109870 0x7f3648109c70 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f3638009820 tx=0x7f363802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de685bc1b12240b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:14.279 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:13.598+0000 7f3645600640 1 -- 172.21.15.77:0/1221249480 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3638037070 con 0x7f3648109870 2024-09-10T09:44:14.279 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:13.598+0000 7f3645600640 1 -- 172.21.15.77:0/1221249480 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3638032900 con 0x7f3648109870 2024-09-10T09:44:14.280 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:13.598+0000 7f3645600640 1 -- 172.21.15.77:0/1221249480 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3638030850 con 0x7f3648109870 2024-09-10T09:44:14.280 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:13.599+0000 7f364d600640 1 -- 172.21.15.77:0/1221249480 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3648109870 msgr2=0x7f3648109c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:14.280 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:13.599+0000 7f364d600640 1 --2- 172.21.15.77:0/1221249480 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3648109870 0x7f3648109c70 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f3638009820 tx=0x7f363802a270 comp rx=0 tx=0).stop 2024-09-10T09:44:14.280 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:13.599+0000 7f364d600640 1 -- 172.21.15.77:0/1221249480 shutdown_connections 2024-09-10T09:44:14.280 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:13.599+0000 7f364d600640 1 --2- 172.21.15.77:0/1221249480 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3648109870 0x7f3648109c70 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:14.280 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:13.599+0000 7f364d600640 1 -- 172.21.15.77:0/1221249480 >> 172.21.15.77:0/1221249480 conn(0x7f364807cdc0 msgr2=0x7f364807d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:14.280 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:13.599+0000 7f364d600640 1 -- 172.21.15.77:0/1221249480 shutdown_connections 2024-09-10T09:44:14.280 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:13.599+0000 7f364d600640 1 -- 172.21.15.77:0/1221249480 wait complete. 2024-09-10T09:44:14.280 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:13.600+0000 7f364d600640 1 Processor -- start 2024-09-10T09:44:14.280 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:13.600+0000 7f364d600640 1 -- start start 2024-09-10T09:44:14.280 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:13.600+0000 7f364d600640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3648109870 0x7f364819daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:14.280 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:13.601+0000 7f364d600640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f364810dd30 con 0x7f3648109870 2024-09-10T09:44:14.280 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:13.601+0000 7f3646a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3648109870 0x7f364819daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:14.281 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:13.601+0000 7f3646a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3648109870 0x7f364819daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:37514/0 (socket says 172.21.15.77:37514) 2024-09-10T09:44:14.281 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:13.601+0000 7f3646a00640 1 -- 172.21.15.77:0/2705536406 learned_addr learned my addr 172.21.15.77:0/2705536406 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:44:14.281 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:13.601+0000 7f3646a00640 1 -- 172.21.15.77:0/2705536406 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f364819dfe0 con 0x7f3648109870 2024-09-10T09:44:14.281 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:13.602+0000 7f3646a00640 1 --2- 172.21.15.77:0/2705536406 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3648109870 0x7f364819daa0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f36380097f0 tx=0x7f3638030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:14.281 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:13.602+0000 7f363f400640 1 -- 172.21.15.77:0/2705536406 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3638037040 con 0x7f3648109870 2024-09-10T09:44:14.281 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:13.602+0000 7f363f400640 1 -- 172.21.15.77:0/2705536406 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3638040070 con 0x7f3648109870 2024-09-10T09:44:14.281 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:13.602+0000 7f364d600640 1 -- 172.21.15.77:0/2705536406 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f364819e270 con 0x7f3648109870 2024-09-10T09:44:14.281 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:13.602+0000 7f363f400640 1 -- 172.21.15.77:0/2705536406 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f363803a390 con 0x7f3648109870 2024-09-10T09:44:14.281 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:13.602+0000 7f364d600640 1 -- 172.21.15.77:0/2705536406 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36481a0f60 con 0x7f3648109870 2024-09-10T09:44:14.281 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:13.602+0000 7f363f400640 1 -- 172.21.15.77:0/2705536406 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f363803a650 con 0x7f3648109870 2024-09-10T09:44:14.281 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:13.603+0000 7f363f400640 1 -- 172.21.15.77:0/2705536406 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f3638045a00 con 0x7f3648109870 2024-09-10T09:44:14.281 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:13.603+0000 7f364d600640 1 -- 172.21.15.77:0/2705536406 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3648109cf0 con 0x7f3648109870 2024-09-10T09:44:14.281 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:13.606+0000 7f363f400640 1 -- 172.21.15.77:0/2705536406 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f3638045460 con 0x7f3648109870 2024-09-10T09:44:14.282 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:13.667+0000 7f364d600640 1 -- 172.21.15.77:0/2705536406 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f364802d7c0 con 0x7f3648109870 2024-09-10T09:44:14.282 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:13.670+0000 7f363f400640 1 -- 172.21.15.77:0/2705536406 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f363803b430 con 0x7f3648109870 2024-09-10T09:44:14.282 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:13.671+0000 7f364d600640 1 -- 172.21.15.77:0/2705536406 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3648109870 msgr2=0x7f364819daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:14.282 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:13.671+0000 7f364d600640 1 --2- 172.21.15.77:0/2705536406 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3648109870 0x7f364819daa0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f36380097f0 tx=0x7f3638030d80 comp rx=0 tx=0).stop 2024-09-10T09:44:14.282 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:13.671+0000 7f364d600640 1 -- 172.21.15.77:0/2705536406 shutdown_connections 2024-09-10T09:44:14.282 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:13.671+0000 7f364d600640 1 --2- 172.21.15.77:0/2705536406 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3648109870 0x7f364819daa0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:14.282 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:13.671+0000 7f364d600640 1 -- 172.21.15.77:0/2705536406 >> 172.21.15.77:0/2705536406 conn(0x7f364807cdc0 msgr2=0x7f36481076e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:14.282 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:13.672+0000 7f364d600640 1 -- 172.21.15.77:0/2705536406 shutdown_connections 2024-09-10T09:44:14.282 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:13.672+0000 7f364d600640 1 -- 172.21.15.77:0/2705536406 wait complete. 2024-09-10T09:44:14.282 INFO:teuthology.orchestra.run.smithi077.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-10T09:44:14.309 INFO:teuthology.orchestra.run.smithi077.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-10T09:44:14.309 INFO:teuthology.orchestra.run.smithi077.stdout:Creating mgr... 2024-09-10T09:44:14.310 INFO:teuthology.orchestra.run.smithi077.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-10T09:44:14.310 INFO:teuthology.orchestra.run.smithi077.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-10T09:44:14.648 INFO:teuthology.orchestra.run.smithi077.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-14a55226-6f59-11ef-bcea-c7b262605968@mgr.x 2024-09-10T09:44:14.648 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stderr Failed to reset failed state of unit ceph-14a55226-6f59-11ef-bcea-c7b262605968@mgr.x.service: Unit ceph-14a55226-6f59-11ef-bcea-c7b262605968@mgr.x.service not loaded. 2024-09-10T09:44:14.874 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-14a55226-6f59-11ef-bcea-c7b262605968.target.wants/ceph-14a55226-6f59-11ef-bcea-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-14a55226-6f59-11ef-bcea-c7b262605968@.service. 2024-09-10T09:44:14.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:14 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/2705536406' entity='client.admin' 2024-09-10T09:44:14.940 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:14 smithi077 systemd[1]: Starting Ceph mgr.x for 14a55226-6f59-11ef-bcea-c7b262605968... 2024-09-10T09:44:15.547 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:15 smithi077 podman[30536]: 2024-09-10 09:44:15.116178969 +0000 UTC m=+0.027153435 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T09:44:15.547 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:15 smithi077 podman[30536]: 2024-09-10 09:44:15.244630055 +0000 UTC m=+0.155604520 container create 688587491f1e07a2e7e61da7d709e0072094a3f2899b67821b5c7be390199740 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T09:44:15.548 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:15 smithi077 podman[30536]: 2024-09-10 09:44:15.455550026 +0000 UTC m=+0.366524518 container init 688587491f1e07a2e7e61da7d709e0072094a3f2899b67821b5c7be390199740 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-10T09:44:15.548 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:15 smithi077 podman[30536]: 2024-09-10 09:44:15.46165059 +0000 UTC m=+0.372625056 container start 688587491f1e07a2e7e61da7d709e0072094a3f2899b67821b5c7be390199740 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2) 2024-09-10T09:44:15.548 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:15 smithi077 bash[30536]: 688587491f1e07a2e7e61da7d709e0072094a3f2899b67821b5c7be390199740 2024-09-10T09:44:15.574 INFO:teuthology.orchestra.run.smithi077.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T09:44:15.574 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stdout disabled 2024-09-10T09:44:15.588 INFO:teuthology.orchestra.run.smithi077.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T09:44:15.588 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stdout inactive 2024-09-10T09:44:15.588 INFO:teuthology.orchestra.run.smithi077.stdout:firewalld.service is not enabled 2024-09-10T09:44:15.588 INFO:teuthology.orchestra.run.smithi077.stdout:Not possible to enable service . firewalld.service is not available 2024-09-10T09:44:15.605 INFO:teuthology.orchestra.run.smithi077.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T09:44:15.605 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stdout disabled 2024-09-10T09:44:15.619 INFO:teuthology.orchestra.run.smithi077.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T09:44:15.619 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stdout inactive 2024-09-10T09:44:15.619 INFO:teuthology.orchestra.run.smithi077.stdout:firewalld.service is not enabled 2024-09-10T09:44:15.619 INFO:teuthology.orchestra.run.smithi077.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-10T09:44:15.619 INFO:teuthology.orchestra.run.smithi077.stdout:Waiting for mgr to start... 2024-09-10T09:44:15.619 INFO:teuthology.orchestra.run.smithi077.stdout:Waiting for mgr... 2024-09-10T09:44:15.798 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:15 smithi077 systemd[1]: Started Ceph mgr.x for 14a55226-6f59-11ef-bcea-c7b262605968. 2024-09-10T09:44:15.798 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:15 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: 2024-09-10T09:44:15.629+0000 7ff6e202f040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T09:44:15.798 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:15 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: 2024-09-10T09:44:15.702+0000 7ff6e202f040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T09:44:16.060 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:15 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: 2024-09-10T09:44:15.889+0000 7ff6e202f040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-10T09:44:16.060 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:15 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: 2024-09-10T09:44:15.974+0000 7ff6e202f040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T09:44:16.329 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:16 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/552281556' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-10T09:44:16.329 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:16 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: 2024-09-10T09:44:16.149+0000 7ff6e202f040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T09:44:16.329 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:16 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: 2024-09-10T09:44:16.328+0000 7ff6e202f040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T09:44:16.939 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:16 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: 2024-09-10T09:44:16.713+0000 7ff6e202f040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T09:44:16.939 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:16 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: 2024-09-10T09:44:16.784+0000 7ff6e202f040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T09:44:17.044 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout 2024-09-10T09:44:17.047 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout { 2024-09-10T09:44:17.047 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "fsid": "14a55226-6f59-11ef-bcea-c7b262605968", 2024-09-10T09:44:17.047 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "health": { 2024-09-10T09:44:17.047 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-10T09:44:17.047 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-10T09:44:17.047 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-10T09:44:17.047 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2024-09-10T09:44:17.047 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-10T09:44:17.048 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-10T09:44:17.048 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout 0 2024-09-10T09:44:17.048 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout ], 2024-09-10T09:44:17.048 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-10T09:44:17.048 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "a" 2024-09-10T09:44:17.048 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout ], 2024-09-10T09:44:17.048 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-09-10T09:44:17.048 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-10T09:44:17.048 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T09:44:17.048 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-10T09:44:17.048 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-10T09:44:17.048 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2024-09-10T09:44:17.048 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-10T09:44:17.049 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T09:44:17.049 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-10T09:44:17.049 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-10T09:44:17.057 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-10T09:44:17.057 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-10T09:44:17.057 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-10T09:44:17.057 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-10T09:44:17.057 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2024-09-10T09:44:17.057 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-10T09:44:17.057 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-10T09:44:17.057 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-10T09:44:17.057 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-10T09:44:17.057 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-10T09:44:17.058 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-10T09:44:17.058 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-10T09:44:17.058 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-10T09:44:17.058 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-10T09:44:17.058 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2024-09-10T09:44:17.058 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-10T09:44:17.058 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T09:44:17.058 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "btime": "2024-09-10T09:44:07:359254+0000", 2024-09-10T09:44:17.058 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-10T09:44:17.058 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-10T09:44:17.058 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2024-09-10T09:44:17.058 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-10T09:44:17.058 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-10T09:44:17.059 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-10T09:44:17.059 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-10T09:44:17.059 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-10T09:44:17.059 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-10T09:44:17.059 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "restful" 2024-09-10T09:44:17.059 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout ], 2024-09-10T09:44:17.059 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T09:44:17.059 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2024-09-10T09:44:17.059 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-10T09:44:17.059 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T09:44:17.059 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "modified": "2024-09-10T09:44:07.360068+0000", 2024-09-10T09:44:17.059 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T09:44:17.059 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2024-09-10T09:44:17.060 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-10T09:44:17.060 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout } 2024-09-10T09:44:17.060 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:16.165+0000 7f09c3200640 1 Processor -- start 2024-09-10T09:44:17.060 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:16.166+0000 7f09c3200640 1 -- start start 2024-09-10T09:44:17.060 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:16.166+0000 7f09c3200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09c4107400 0x7f09c4107800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:17.060 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:16.166+0000 7f09c3200640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f09c4107dd0 con 0x7f09c4107400 2024-09-10T09:44:17.060 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:16.166+0000 7f09c1e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09c4107400 0x7f09c4107800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:17.060 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:16.166+0000 7f09c1e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09c4107400 0x7f09c4107800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:37538/0 (socket says 172.21.15.77:37538) 2024-09-10T09:44:17.060 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:16.166+0000 7f09c1e00640 1 -- 172.21.15.77:0/1990580939 learned_addr learned my addr 172.21.15.77:0/1990580939 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:44:17.060 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:16.167+0000 7f09c1e00640 1 -- 172.21.15.77:0/1990580939 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09c41085b0 con 0x7f09c4107400 2024-09-10T09:44:17.060 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:16.168+0000 7f09c1e00640 1 --2- 172.21.15.77:0/1990580939 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09c4107400 0x7f09c4107800 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f09b4009e80 tx=0x7f09b402a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4db9a9be79befec server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:17.061 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:16.168+0000 7f09c1400640 1 -- 172.21.15.77:0/1990580939 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f09b4036070 con 0x7f09c4107400 2024-09-10T09:44:17.061 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:16.168+0000 7f09c1400640 1 -- 172.21.15.77:0/1990580939 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f09b40318c0 con 0x7f09c4107400 2024-09-10T09:44:17.061 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:16.168+0000 7f09c1400640 1 -- 172.21.15.77:0/1990580939 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f09b4007ab0 con 0x7f09c4107400 2024-09-10T09:44:17.061 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:16.169+0000 7f09c3200640 1 -- 172.21.15.77:0/1990580939 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09c4107400 msgr2=0x7f09c4107800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:17.061 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:16.169+0000 7f09c3200640 1 --2- 172.21.15.77:0/1990580939 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09c4107400 0x7f09c4107800 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f09b4009e80 tx=0x7f09b402a7a0 comp rx=0 tx=0).stop 2024-09-10T09:44:17.061 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:16.169+0000 7f09c3200640 1 -- 172.21.15.77:0/1990580939 shutdown_connections 2024-09-10T09:44:17.061 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:16.169+0000 7f09c3200640 1 --2- 172.21.15.77:0/1990580939 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09c4107400 0x7f09c4107800 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:17.061 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:16.169+0000 7f09c3200640 1 -- 172.21.15.77:0/1990580939 >> 172.21.15.77:0/1990580939 conn(0x7f09c4102bb0 msgr2=0x7f09c4104fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:17.061 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:16.169+0000 7f09c3200640 1 -- 172.21.15.77:0/1990580939 shutdown_connections 2024-09-10T09:44:17.061 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:16.169+0000 7f09c3200640 1 -- 172.21.15.77:0/1990580939 wait complete. 2024-09-10T09:44:17.061 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:16.170+0000 7f09c3200640 1 Processor -- start 2024-09-10T09:44:17.061 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:16.170+0000 7f09c3200640 1 -- start start 2024-09-10T09:44:17.061 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:16.171+0000 7f09c3200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09c4107400 0x7f09c419b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:17.062 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:16.171+0000 7f09c3200640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f09c410b960 con 0x7f09c4107400 2024-09-10T09:44:17.062 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:16.171+0000 7f09c1e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09c4107400 0x7f09c419b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:17.062 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:16.171+0000 7f09c1e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09c4107400 0x7f09c419b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:37550/0 (socket says 172.21.15.77:37550) 2024-09-10T09:44:17.062 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:16.171+0000 7f09c1e00640 1 -- 172.21.15.77:0/552281556 learned_addr learned my addr 172.21.15.77:0/552281556 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:44:17.062 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:16.171+0000 7f09c1e00640 1 -- 172.21.15.77:0/552281556 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09c419bc10 con 0x7f09c4107400 2024-09-10T09:44:17.062 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:16.171+0000 7f09c1e00640 1 --2- 172.21.15.77:0/552281556 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09c4107400 0x7f09c419b6d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f09b402acc0 tx=0x7f09b4031ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:17.062 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:16.172+0000 7f09bb400640 1 -- 172.21.15.77:0/552281556 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f09b4036050 con 0x7f09c4107400 2024-09-10T09:44:17.062 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:16.172+0000 7f09bb400640 1 -- 172.21.15.77:0/552281556 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f09b4037040 con 0x7f09c4107400 2024-09-10T09:44:17.062 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:16.172+0000 7f09bb400640 1 -- 172.21.15.77:0/552281556 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f09b403d8c0 con 0x7f09c4107400 2024-09-10T09:44:17.062 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:16.172+0000 7f09c3200640 1 -- 172.21.15.77:0/552281556 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09c419bea0 con 0x7f09c4107400 2024-09-10T09:44:17.062 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:16.172+0000 7f09c3200640 1 -- 172.21.15.77:0/552281556 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09c419eb90 con 0x7f09c4107400 2024-09-10T09:44:17.062 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:16.172+0000 7f09bb400640 1 -- 172.21.15.77:0/552281556 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f09b4044020 con 0x7f09c4107400 2024-09-10T09:44:17.062 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:16.172+0000 7f09bb400640 1 -- 172.21.15.77:0/552281556 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f09b4042020 con 0x7f09c4107400 2024-09-10T09:44:17.062 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:16.173+0000 7f09c3200640 1 -- 172.21.15.77:0/552281556 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09c410bae0 con 0x7f09c4107400 2024-09-10T09:44:17.063 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:16.176+0000 7f09bb400640 1 -- 172.21.15.77:0/552281556 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f09b404b020 con 0x7f09c4107400 2024-09-10T09:44:17.063 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:16.238+0000 7f09c3200640 1 -- 172.21.15.77:0/552281556 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f09c4075e60 con 0x7f09c4107400 2024-09-10T09:44:17.063 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:16.238+0000 7f09bb400640 1 -- 172.21.15.77:0/552281556 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7f09b40073b0 con 0x7f09c4107400 2024-09-10T09:44:17.063 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:16.239+0000 7f09c3200640 1 -- 172.21.15.77:0/552281556 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09c4107400 msgr2=0x7f09c419b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:17.063 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:16.239+0000 7f09c3200640 1 --2- 172.21.15.77:0/552281556 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09c4107400 0x7f09c419b6d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f09b402acc0 tx=0x7f09b4031ca0 comp rx=0 tx=0).stop 2024-09-10T09:44:17.063 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:16.239+0000 7f09c3200640 1 -- 172.21.15.77:0/552281556 shutdown_connections 2024-09-10T09:44:17.063 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:16.239+0000 7f09c3200640 1 --2- 172.21.15.77:0/552281556 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09c4107400 0x7f09c419b6d0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:17.063 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:16.239+0000 7f09c3200640 1 -- 172.21.15.77:0/552281556 >> 172.21.15.77:0/552281556 conn(0x7f09c4102bb0 msgr2=0x7f09c4103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:17.063 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:16.240+0000 7f09c3200640 1 -- 172.21.15.77:0/552281556 shutdown_connections 2024-09-10T09:44:17.063 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:16.240+0000 7f09c3200640 1 -- 172.21.15.77:0/552281556 wait complete. 2024-09-10T09:44:17.063 INFO:teuthology.orchestra.run.smithi077.stdout:mgr not available, waiting (1/15)... 2024-09-10T09:44:17.439 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:17 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: 2024-09-10T09:44:16.995+0000 7ff6e202f040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T09:44:18.439 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:18 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: 2024-09-10T09:44:18.088+0000 7ff6e202f040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T09:44:18.439 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:18 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: 2024-09-10T09:44:18.167+0000 7ff6e202f040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T09:44:19.370 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:19 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: 2024-09-10T09:44:19.098+0000 7ff6e202f040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T09:44:19.370 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:19 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: 2024-09-10T09:44:19.236+0000 7ff6e202f040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T09:44:19.646 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:19 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: 2024-09-10T09:44:19.407+0000 7ff6e202f040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T09:44:19.646 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:19 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: 2024-09-10T09:44:19.480+0000 7ff6e202f040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T09:44:19.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:19 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/1539752904' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-10T09:44:19.939 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:19 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: 2024-09-10T09:44:19.664+0000 7ff6e202f040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T09:44:19.939 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:19 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: 2024-09-10T09:44:19.756+0000 7ff6e202f040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T09:44:19.939 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:19 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: 2024-09-10T09:44:19.827+0000 7ff6e202f040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T09:44:20.243 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:20 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: 2024-09-10T09:44:20.032+0000 7ff6e202f040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T09:44:20.243 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:20 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: 2024-09-10T09:44:20.109+0000 7ff6e202f040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T09:44:20.244 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:20 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: 2024-09-10T09:44:20.188+0000 7ff6e202f040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T09:44:20.335 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout 2024-09-10T09:44:20.335 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout { 2024-09-10T09:44:20.335 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "fsid": "14a55226-6f59-11ef-bcea-c7b262605968", 2024-09-10T09:44:20.336 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "health": { 2024-09-10T09:44:20.336 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-10T09:44:20.336 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-10T09:44:20.336 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-10T09:44:20.336 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2024-09-10T09:44:20.336 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-10T09:44:20.336 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-10T09:44:20.336 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout 0 2024-09-10T09:44:20.336 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout ], 2024-09-10T09:44:20.336 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-10T09:44:20.336 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "a" 2024-09-10T09:44:20.336 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout ], 2024-09-10T09:44:20.336 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2024-09-10T09:44:20.337 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-10T09:44:20.337 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T09:44:20.337 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-10T09:44:20.337 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-10T09:44:20.337 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2024-09-10T09:44:20.337 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-10T09:44:20.337 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T09:44:20.337 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-10T09:44:20.337 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-10T09:44:20.346 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-10T09:44:20.346 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-10T09:44:20.346 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-10T09:44:20.346 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-10T09:44:20.346 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2024-09-10T09:44:20.346 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-10T09:44:20.346 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-10T09:44:20.346 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-10T09:44:20.347 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-10T09:44:20.347 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-10T09:44:20.347 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-10T09:44:20.347 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-10T09:44:20.347 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-10T09:44:20.347 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-10T09:44:20.347 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2024-09-10T09:44:20.347 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-10T09:44:20.347 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T09:44:20.347 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "btime": "2024-09-10T09:44:07:359254+0000", 2024-09-10T09:44:20.347 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-10T09:44:20.347 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-10T09:44:20.347 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2024-09-10T09:44:20.347 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-10T09:44:20.348 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-10T09:44:20.348 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-10T09:44:20.348 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-10T09:44:20.348 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-10T09:44:20.348 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-10T09:44:20.348 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "restful" 2024-09-10T09:44:20.348 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout ], 2024-09-10T09:44:20.348 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T09:44:20.348 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2024-09-10T09:44:20.348 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-10T09:44:20.348 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T09:44:20.348 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "modified": "2024-09-10T09:44:07.360068+0000", 2024-09-10T09:44:20.348 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T09:44:20.349 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2024-09-10T09:44:20.349 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-10T09:44:20.349 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout } 2024-09-10T09:44:20.349 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:19.571+0000 7f66d8000640 1 Processor -- start 2024-09-10T09:44:20.349 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:19.572+0000 7f66d8000640 1 -- start start 2024-09-10T09:44:20.349 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:19.572+0000 7f66d8000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f66d0109870 0x7f66d0109c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:20.349 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:19.572+0000 7f66d8000640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f66d010a240 con 0x7f66d0109870 2024-09-10T09:44:20.349 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:19.572+0000 7f66d5600640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f66d0109870 0x7f66d0109c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:20.349 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:19.572+0000 7f66d5600640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f66d0109870 0x7f66d0109c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:37552/0 (socket says 172.21.15.77:37552) 2024-09-10T09:44:20.349 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:19.572+0000 7f66d5600640 1 -- 172.21.15.77:0/100156463 learned_addr learned my addr 172.21.15.77:0/100156463 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:44:20.349 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:19.573+0000 7f66d5600640 1 -- 172.21.15.77:0/100156463 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66d010a980 con 0x7f66d0109870 2024-09-10T09:44:20.349 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:19.574+0000 7f66d5600640 1 --2- 172.21.15.77:0/100156463 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f66d0109870 0x7f66d0109c70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f66c4009820 tx=0x7f66c402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0e2074f8b17323b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:20.350 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:19.574+0000 7f66cfe00640 1 -- 172.21.15.77:0/100156463 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f66c4037070 con 0x7f66d0109870 2024-09-10T09:44:20.350 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:19.574+0000 7f66cfe00640 1 -- 172.21.15.77:0/100156463 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f66c4032900 con 0x7f66d0109870 2024-09-10T09:44:20.350 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:19.574+0000 7f66cfe00640 1 -- 172.21.15.77:0/100156463 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f66c4030850 con 0x7f66d0109870 2024-09-10T09:44:20.350 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:19.575+0000 7f66d8000640 1 -- 172.21.15.77:0/100156463 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f66d0109870 msgr2=0x7f66d0109c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:20.350 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:19.575+0000 7f66d8000640 1 --2- 172.21.15.77:0/100156463 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f66d0109870 0x7f66d0109c70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f66c4009820 tx=0x7f66c402a270 comp rx=0 tx=0).stop 2024-09-10T09:44:20.350 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:19.575+0000 7f66d8000640 1 -- 172.21.15.77:0/100156463 shutdown_connections 2024-09-10T09:44:20.350 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:19.575+0000 7f66d8000640 1 --2- 172.21.15.77:0/100156463 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f66d0109870 0x7f66d0109c70 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:20.350 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:19.575+0000 7f66d8000640 1 -- 172.21.15.77:0/100156463 >> 172.21.15.77:0/100156463 conn(0x7f66d007cdc0 msgr2=0x7f66d007d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:20.350 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:19.575+0000 7f66d8000640 1 -- 172.21.15.77:0/100156463 shutdown_connections 2024-09-10T09:44:20.350 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:19.575+0000 7f66d8000640 1 -- 172.21.15.77:0/100156463 wait complete. 2024-09-10T09:44:20.350 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:19.576+0000 7f66d8000640 1 Processor -- start 2024-09-10T09:44:20.350 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:19.576+0000 7f66d8000640 1 -- start start 2024-09-10T09:44:20.350 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:19.576+0000 7f66d8000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f66d0109870 0x7f66d019daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:20.351 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:19.576+0000 7f66d8000640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f66d010dd30 con 0x7f66d0109870 2024-09-10T09:44:20.351 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:19.576+0000 7f66d5600640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f66d0109870 0x7f66d019daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:20.351 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:19.576+0000 7f66d5600640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f66d0109870 0x7f66d019daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:37558/0 (socket says 172.21.15.77:37558) 2024-09-10T09:44:20.351 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:19.576+0000 7f66d5600640 1 -- 172.21.15.77:0/1539752904 learned_addr learned my addr 172.21.15.77:0/1539752904 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:44:20.351 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:19.577+0000 7f66d5600640 1 -- 172.21.15.77:0/1539752904 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66d019dfe0 con 0x7f66d0109870 2024-09-10T09:44:20.351 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:19.577+0000 7f66d5600640 1 --2- 172.21.15.77:0/1539752904 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f66d0109870 0x7f66d019daa0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f66c40097f0 tx=0x7f66c4030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:20.351 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:19.577+0000 7f66ce000640 1 -- 172.21.15.77:0/1539752904 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f66c4037040 con 0x7f66d0109870 2024-09-10T09:44:20.351 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:19.577+0000 7f66ce000640 1 -- 172.21.15.77:0/1539752904 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f66c4040070 con 0x7f66d0109870 2024-09-10T09:44:20.351 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:19.577+0000 7f66d8000640 1 -- 172.21.15.77:0/1539752904 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66d019e270 con 0x7f66d0109870 2024-09-10T09:44:20.351 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:19.577+0000 7f66ce000640 1 -- 172.21.15.77:0/1539752904 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f66c403a390 con 0x7f66d0109870 2024-09-10T09:44:20.351 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:19.577+0000 7f66d8000640 1 -- 172.21.15.77:0/1539752904 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66d01a0f60 con 0x7f66d0109870 2024-09-10T09:44:20.351 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:19.578+0000 7f66ce000640 1 -- 172.21.15.77:0/1539752904 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f66c403a5e0 con 0x7f66d0109870 2024-09-10T09:44:20.351 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:19.578+0000 7f66ce000640 1 -- 172.21.15.77:0/1539752904 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f66c40459a0 con 0x7f66d0109870 2024-09-10T09:44:20.352 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:19.579+0000 7f66d8000640 1 -- 172.21.15.77:0/1539752904 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66d0109cf0 con 0x7f66d0109870 2024-09-10T09:44:20.352 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:19.581+0000 7f66ce000640 1 -- 172.21.15.77:0/1539752904 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f66c4045460 con 0x7f66d0109870 2024-09-10T09:44:20.352 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:19.645+0000 7f66d8000640 1 -- 172.21.15.77:0/1539752904 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f66d002d7c0 con 0x7f66d0109870 2024-09-10T09:44:20.352 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:19.646+0000 7f66ce000640 1 -- 172.21.15.77:0/1539752904 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7f66c403b430 con 0x7f66d0109870 2024-09-10T09:44:20.352 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:19.647+0000 7f66d8000640 1 -- 172.21.15.77:0/1539752904 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f66d0109870 msgr2=0x7f66d019daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:20.352 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:19.647+0000 7f66d8000640 1 --2- 172.21.15.77:0/1539752904 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f66d0109870 0x7f66d019daa0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f66c40097f0 tx=0x7f66c4030d80 comp rx=0 tx=0).stop 2024-09-10T09:44:20.352 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:19.647+0000 7f66d8000640 1 -- 172.21.15.77:0/1539752904 shutdown_connections 2024-09-10T09:44:20.352 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:19.647+0000 7f66d8000640 1 --2- 172.21.15.77:0/1539752904 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f66d0109870 0x7f66d019daa0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:20.352 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:19.647+0000 7f66d8000640 1 -- 172.21.15.77:0/1539752904 >> 172.21.15.77:0/1539752904 conn(0x7f66d007cdc0 msgr2=0x7f66d01076e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:20.352 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:19.647+0000 7f66d8000640 1 -- 172.21.15.77:0/1539752904 shutdown_connections 2024-09-10T09:44:20.352 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:19.647+0000 7f66d8000640 1 -- 172.21.15.77:0/1539752904 wait complete. 2024-09-10T09:44:20.352 INFO:teuthology.orchestra.run.smithi077.stdout:mgr not available, waiting (2/15)... 2024-09-10T09:44:20.689 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:20 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: 2024-09-10T09:44:20.289+0000 7ff6e202f040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T09:44:20.689 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:20 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: 2024-09-10T09:44:20.364+0000 7ff6e202f040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T09:44:20.689 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:20 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: 2024-09-10T09:44:20.432+0000 7ff6e202f040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T09:44:21.689 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:21 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: 2024-09-10T09:44:21.332+0000 7ff6e202f040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T09:44:21.689 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:21 smithi077 ceph-mon[30319]: Activating manager daemon x 2024-09-10T09:44:21.690 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:21 smithi077 ceph-mon[30319]: mgrmap e2: x(active, starting, since 0.00428989s) 2024-09-10T09:44:21.690 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:21 smithi077 ceph-mon[30319]: from='mgr.14100 172.21.15.77:0/2589495969' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T09:44:21.690 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:21 smithi077 ceph-mon[30319]: from='mgr.14100 172.21.15.77:0/2589495969' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T09:44:21.690 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:21 smithi077 ceph-mon[30319]: from='mgr.14100 172.21.15.77:0/2589495969' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T09:44:21.690 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:21 smithi077 ceph-mon[30319]: from='mgr.14100 172.21.15.77:0/2589495969' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T09:44:21.690 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:21 smithi077 ceph-mon[30319]: from='mgr.14100 172.21.15.77:0/2589495969' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-10T09:44:21.690 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:21 smithi077 ceph-mon[30319]: Manager daemon x is now available 2024-09-10T09:44:21.690 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:21 smithi077 ceph-mon[30319]: from='mgr.14100 172.21.15.77:0/2589495969' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-10T09:44:21.690 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:21 smithi077 ceph-mon[30319]: from='mgr.14100 172.21.15.77:0/2589495969' entity='mgr.x' 2024-09-10T09:44:21.690 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:21 smithi077 ceph-mon[30319]: from='mgr.14100 172.21.15.77:0/2589495969' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-10T09:44:21.690 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:21 smithi077 ceph-mon[30319]: from='mgr.14100 172.21.15.77:0/2589495969' entity='mgr.x' 2024-09-10T09:44:21.690 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:21 smithi077 ceph-mon[30319]: from='mgr.14100 172.21.15.77:0/2589495969' entity='mgr.x' 2024-09-10T09:44:23.689 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:23 smithi077 ceph-mon[30319]: mgrmap e3: x(active, since 1.00923s) 2024-09-10T09:44:23.689 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:23 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/1818961024' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-10T09:44:24.075 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout 2024-09-10T09:44:24.076 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout { 2024-09-10T09:44:24.076 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "fsid": "14a55226-6f59-11ef-bcea-c7b262605968", 2024-09-10T09:44:24.076 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "health": { 2024-09-10T09:44:24.076 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-10T09:44:24.076 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-10T09:44:24.076 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-10T09:44:24.076 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2024-09-10T09:44:24.076 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-10T09:44:24.076 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-10T09:44:24.076 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout 0 2024-09-10T09:44:24.077 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout ], 2024-09-10T09:44:24.077 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-10T09:44:24.077 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "a" 2024-09-10T09:44:24.077 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout ], 2024-09-10T09:44:24.077 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-09-10T09:44:24.077 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-10T09:44:24.077 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T09:44:24.077 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-10T09:44:24.077 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-10T09:44:24.078 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2024-09-10T09:44:24.078 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-10T09:44:24.078 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T09:44:24.078 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-10T09:44:24.078 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-10T09:44:24.078 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-10T09:44:24.078 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-10T09:44:24.078 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-10T09:44:24.078 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-10T09:44:24.078 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2024-09-10T09:44:24.078 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-10T09:44:24.078 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-10T09:44:24.079 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-10T09:44:24.079 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-10T09:44:24.079 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-10T09:44:24.079 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-10T09:44:24.079 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-10T09:44:24.079 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-10T09:44:24.079 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-10T09:44:24.079 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2024-09-10T09:44:24.079 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-10T09:44:24.080 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T09:44:24.080 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "btime": "2024-09-10T09:44:07:359254+0000", 2024-09-10T09:44:24.080 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-10T09:44:24.080 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-10T09:44:24.080 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2024-09-10T09:44:24.080 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-10T09:44:24.080 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-10T09:44:24.080 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-10T09:44:24.080 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-10T09:44:24.080 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-10T09:44:24.080 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-10T09:44:24.080 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "restful" 2024-09-10T09:44:24.081 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout ], 2024-09-10T09:44:24.081 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T09:44:24.081 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2024-09-10T09:44:24.081 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-10T09:44:24.081 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T09:44:24.081 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "modified": "2024-09-10T09:44:07.360068+0000", 2024-09-10T09:44:24.081 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T09:44:24.081 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2024-09-10T09:44:24.081 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-10T09:44:24.081 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout } 2024-09-10T09:44:24.082 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:22.824+0000 7f7767e00640 1 Processor -- start 2024-09-10T09:44:24.082 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:22.825+0000 7f7767e00640 1 -- start start 2024-09-10T09:44:24.082 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:22.825+0000 7f7767e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7768107e90 0x7f776810a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:24.082 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:22.825+0000 7f7767e00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f776807bda0 con 0x7f7768107e90 2024-09-10T09:44:24.082 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:22.825+0000 7f7766a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7768107e90 0x7f776810a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:24.082 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:22.825+0000 7f7766a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7768107e90 0x7f776810a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59214/0 (socket says 172.21.15.77:59214) 2024-09-10T09:44:24.082 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:22.825+0000 7f7766a00640 1 -- 172.21.15.77:0/1265853390 learned_addr learned my addr 172.21.15.77:0/1265853390 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:44:24.082 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:22.826+0000 7f7766a00640 1 -- 172.21.15.77:0/1265853390 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f776810a7c0 con 0x7f7768107e90 2024-09-10T09:44:24.082 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:22.827+0000 7f7766a00640 1 --2- 172.21.15.77:0/1265853390 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7768107e90 0x7f776810a280 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7758009820 tx=0x7f775802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af00188ea29ac563 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:24.082 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:22.827+0000 7f7765600640 1 -- 172.21.15.77:0/1265853390 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7758037070 con 0x7f7768107e90 2024-09-10T09:44:24.083 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:22.827+0000 7f7765600640 1 -- 172.21.15.77:0/1265853390 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7758032900 con 0x7f7768107e90 2024-09-10T09:44:24.083 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:22.827+0000 7f7765600640 1 -- 172.21.15.77:0/1265853390 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7758030850 con 0x7f7768107e90 2024-09-10T09:44:24.083 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:22.828+0000 7f7767e00640 1 -- 172.21.15.77:0/1265853390 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7768107e90 msgr2=0x7f776810a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:24.083 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:22.828+0000 7f7767e00640 1 --2- 172.21.15.77:0/1265853390 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7768107e90 0x7f776810a280 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7758009820 tx=0x7f775802a270 comp rx=0 tx=0).stop 2024-09-10T09:44:24.083 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:22.828+0000 7f7767e00640 1 -- 172.21.15.77:0/1265853390 shutdown_connections 2024-09-10T09:44:24.083 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:22.828+0000 7f7767e00640 1 --2- 172.21.15.77:0/1265853390 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7768107e90 0x7f776810a280 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:24.083 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:22.828+0000 7f7767e00640 1 -- 172.21.15.77:0/1265853390 >> 172.21.15.77:0/1265853390 conn(0x7f77681019b0 msgr2=0x7f7768103e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:24.083 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:22.828+0000 7f7767e00640 1 -- 172.21.15.77:0/1265853390 shutdown_connections 2024-09-10T09:44:24.083 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:22.828+0000 7f7767e00640 1 -- 172.21.15.77:0/1265853390 wait complete. 2024-09-10T09:44:24.083 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:22.829+0000 7f7767e00640 1 Processor -- start 2024-09-10T09:44:24.083 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:22.829+0000 7f7767e00640 1 -- start start 2024-09-10T09:44:24.083 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:22.829+0000 7f7767e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7768107e90 0x7f776819d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:24.083 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:22.829+0000 7f7767e00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f776810b7a0 con 0x7f7768107e90 2024-09-10T09:44:24.083 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:22.829+0000 7f7766a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7768107e90 0x7f776819d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:24.084 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:22.829+0000 7f7766a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7768107e90 0x7f776819d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59230/0 (socket says 172.21.15.77:59230) 2024-09-10T09:44:24.084 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:22.829+0000 7f7766a00640 1 -- 172.21.15.77:0/1818961024 learned_addr learned my addr 172.21.15.77:0/1818961024 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:44:24.084 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:22.830+0000 7f7766a00640 1 -- 172.21.15.77:0/1818961024 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f776819de20 con 0x7f7768107e90 2024-09-10T09:44:24.084 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:22.830+0000 7f7766a00640 1 --2- 172.21.15.77:0/1818961024 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7768107e90 0x7f776819d8e0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7758032ce0 tx=0x7f7758032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:24.084 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:22.830+0000 7f775f400640 1 -- 172.21.15.77:0/1818961024 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7758042070 con 0x7f7768107e90 2024-09-10T09:44:24.084 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:22.830+0000 7f775f400640 1 -- 172.21.15.77:0/1818961024 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f77580308c0 con 0x7f7768107e90 2024-09-10T09:44:24.084 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:22.830+0000 7f775f400640 1 -- 172.21.15.77:0/1818961024 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7758037040 con 0x7f7768107e90 2024-09-10T09:44:24.084 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:22.830+0000 7f7767e00640 1 -- 172.21.15.77:0/1818961024 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f776819e0b0 con 0x7f7768107e90 2024-09-10T09:44:24.084 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:22.830+0000 7f7767e00640 1 -- 172.21.15.77:0/1818961024 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77681a0d50 con 0x7f7768107e90 2024-09-10T09:44:24.084 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:22.831+0000 7f775f400640 1 -- 172.21.15.77:0/1818961024 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 3) ==== 46170+0+0 (secure 0 0 0) 0x7f7758045070 con 0x7f7768107e90 2024-09-10T09:44:24.084 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:22.831+0000 7f7767e00640 1 -- 172.21.15.77:0/1818961024 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77681082f0 con 0x7f7768107e90 2024-09-10T09:44:24.084 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:22.832+0000 7f775f400640 1 --2- 172.21.15.77:0/1818961024 >> [v2:172.21.15.77:6800/3235900503,v1:172.21.15.77:6801/3235900503] conn(0x7f774c038e60 0x7f774c03b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:24.085 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:22.832+0000 7f775f400640 1 -- 172.21.15.77:0/1818961024 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f775806bd60 con 0x7f7768107e90 2024-09-10T09:44:24.085 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:22.832+0000 7f7766000640 1 --2- 172.21.15.77:0/1818961024 >> [v2:172.21.15.77:6800/3235900503,v1:172.21.15.77:6801/3235900503] conn(0x7f774c038e60 0x7f774c03b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:24.085 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:22.833+0000 7f7766000640 1 --2- 172.21.15.77:0/1818961024 >> [v2:172.21.15.77:6800/3235900503,v1:172.21.15.77:6801/3235900503] conn(0x7f774c038e60 0x7f774c03b320 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f77540098c0 tx=0x7f7754006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:24.085 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:22.838+0000 7f775f400640 1 -- 172.21.15.77:0/1818961024 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f775804b170 con 0x7f7768107e90 2024-09-10T09:44:24.085 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:23.078+0000 7f7767e00640 1 -- 172.21.15.77:0/1818961024 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f776802d7c0 con 0x7f7768107e90 2024-09-10T09:44:24.085 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:23.078+0000 7f775f400640 1 -- 172.21.15.77:0/1818961024 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1290 (secure 0 0 0) 0x7f775803ccc0 con 0x7f7768107e90 2024-09-10T09:44:24.085 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:23.080+0000 7f7767e00640 1 -- 172.21.15.77:0/1818961024 >> [v2:172.21.15.77:6800/3235900503,v1:172.21.15.77:6801/3235900503] conn(0x7f774c038e60 msgr2=0x7f774c03b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:24.085 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:23.080+0000 7f7767e00640 1 --2- 172.21.15.77:0/1818961024 >> [v2:172.21.15.77:6800/3235900503,v1:172.21.15.77:6801/3235900503] conn(0x7f774c038e60 0x7f774c03b320 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f77540098c0 tx=0x7f7754006db0 comp rx=0 tx=0).stop 2024-09-10T09:44:24.085 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:23.080+0000 7f7767e00640 1 -- 172.21.15.77:0/1818961024 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7768107e90 msgr2=0x7f776819d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:24.085 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:23.080+0000 7f7767e00640 1 --2- 172.21.15.77:0/1818961024 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7768107e90 0x7f776819d8e0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7758032ce0 tx=0x7f7758032d10 comp rx=0 tx=0).stop 2024-09-10T09:44:24.085 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:23.081+0000 7f7767e00640 1 -- 172.21.15.77:0/1818961024 shutdown_connections 2024-09-10T09:44:24.085 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:23.081+0000 7f7767e00640 1 --2- 172.21.15.77:0/1818961024 >> [v2:172.21.15.77:6800/3235900503,v1:172.21.15.77:6801/3235900503] conn(0x7f774c038e60 0x7f774c03b320 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:24.085 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:23.081+0000 7f7767e00640 1 --2- 172.21.15.77:0/1818961024 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7768107e90 0x7f776819d8e0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:24.086 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:23.081+0000 7f7767e00640 1 -- 172.21.15.77:0/1818961024 >> 172.21.15.77:0/1818961024 conn(0x7f77681019b0 msgr2=0x7f7768102730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:24.086 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:23.081+0000 7f7767e00640 1 -- 172.21.15.77:0/1818961024 shutdown_connections 2024-09-10T09:44:24.086 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:23.081+0000 7f7767e00640 1 -- 172.21.15.77:0/1818961024 wait complete. 2024-09-10T09:44:24.086 INFO:teuthology.orchestra.run.smithi077.stdout:mgr is available 2024-09-10T09:44:24.689 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:24 smithi077 ceph-mon[30319]: mgrmap e4: x(active, since 2s) 2024-09-10T09:44:25.351 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout 2024-09-10T09:44:25.352 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout [global] 2024-09-10T09:44:25.352 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout fsid = 14a55226-6f59-11ef-bcea-c7b262605968 2024-09-10T09:44:25.352 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.77:3300,v1:172.21.15.77:6789] 2024-09-10T09:44:25.352 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-10T09:44:25.352 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-10T09:44:25.352 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-10T09:44:25.352 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-10T09:44:25.352 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout 2024-09-10T09:44:25.352 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-10T09:44:25.352 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-10T09:44:25.352 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout 2024-09-10T09:44:25.352 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout [osd] 2024-09-10T09:44:25.352 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-10T09:44:25.352 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-10T09:44:25.353 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-10T09:44:25.353 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-10T09:44:25.353 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-10T09:44:25.353 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:24.539+0000 7f5dd3e00640 1 Processor -- start 2024-09-10T09:44:25.353 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:24.540+0000 7f5dd3e00640 1 -- start start 2024-09-10T09:44:25.353 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:24.540+0000 7f5dd3e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5dd407de30 0x7f5dd407c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:25.353 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:24.540+0000 7f5dd3e00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5dd407c860 con 0x7f5dd407de30 2024-09-10T09:44:25.353 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:24.540+0000 7f5dd2a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5dd407de30 0x7f5dd407c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:25.353 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:24.540+0000 7f5dd2a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5dd407de30 0x7f5dd407c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59238/0 (socket says 172.21.15.77:59238) 2024-09-10T09:44:25.353 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:24.540+0000 7f5dd2a00640 1 -- 172.21.15.77:0/1834179645 learned_addr learned my addr 172.21.15.77:0/1834179645 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:44:25.353 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:24.541+0000 7f5dd2a00640 1 -- 172.21.15.77:0/1834179645 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dd407c9e0 con 0x7f5dd407de30 2024-09-10T09:44:25.353 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:24.541+0000 7f5dd2a00640 1 --2- 172.21.15.77:0/1834179645 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5dd407de30 0x7f5dd407c290 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5dc4009820 tx=0x7f5dc402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=584150a36f265ae3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:25.353 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:24.542+0000 7f5dd2000640 1 -- 172.21.15.77:0/1834179645 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5dc4037070 con 0x7f5dd407de30 2024-09-10T09:44:25.354 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:24.542+0000 7f5dd2000640 1 -- 172.21.15.77:0/1834179645 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5dc4032900 con 0x7f5dd407de30 2024-09-10T09:44:25.354 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:24.542+0000 7f5dd2000640 1 -- 172.21.15.77:0/1834179645 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5dc4030850 con 0x7f5dd407de30 2024-09-10T09:44:25.354 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:24.542+0000 7f5dd3e00640 1 -- 172.21.15.77:0/1834179645 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5dd407de30 msgr2=0x7f5dd407c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:25.354 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:24.542+0000 7f5dd3e00640 1 --2- 172.21.15.77:0/1834179645 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5dd407de30 0x7f5dd407c290 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5dc4009820 tx=0x7f5dc402a270 comp rx=0 tx=0).stop 2024-09-10T09:44:25.354 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:24.542+0000 7f5dd3e00640 1 -- 172.21.15.77:0/1834179645 shutdown_connections 2024-09-10T09:44:25.354 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:24.542+0000 7f5dd3e00640 1 --2- 172.21.15.77:0/1834179645 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5dd407de30 0x7f5dd407c290 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:25.354 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:24.542+0000 7f5dd3e00640 1 -- 172.21.15.77:0/1834179645 >> 172.21.15.77:0/1834179645 conn(0x7f5dd4102b70 msgr2=0x7f5dd4104fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:25.354 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:24.543+0000 7f5dd3e00640 1 -- 172.21.15.77:0/1834179645 shutdown_connections 2024-09-10T09:44:25.354 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:24.543+0000 7f5dd3e00640 1 -- 172.21.15.77:0/1834179645 wait complete. 2024-09-10T09:44:25.354 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:24.543+0000 7f5dd3e00640 1 Processor -- start 2024-09-10T09:44:25.354 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:24.544+0000 7f5dd3e00640 1 -- start start 2024-09-10T09:44:25.354 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:24.544+0000 7f5dd3e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5dd407de30 0x7f5dd419d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:25.354 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:24.544+0000 7f5dd3e00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5dd410d9b0 con 0x7f5dd407de30 2024-09-10T09:44:25.355 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:24.544+0000 7f5dd2a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5dd407de30 0x7f5dd419d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:25.355 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:24.544+0000 7f5dd2a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5dd407de30 0x7f5dd419d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59244/0 (socket says 172.21.15.77:59244) 2024-09-10T09:44:25.355 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:24.544+0000 7f5dd2a00640 1 -- 172.21.15.77:0/622162173 learned_addr learned my addr 172.21.15.77:0/622162173 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:44:25.355 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:24.544+0000 7f5dd2a00640 1 -- 172.21.15.77:0/622162173 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dd419dd70 con 0x7f5dd407de30 2024-09-10T09:44:25.355 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:24.545+0000 7f5dd2a00640 1 --2- 172.21.15.77:0/622162173 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5dd407de30 0x7f5dd419d830 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f5dc40097f0 tx=0x7f5dc4030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:25.355 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:24.545+0000 7f5dd8a00640 1 -- 172.21.15.77:0/622162173 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5dc4037040 con 0x7f5dd407de30 2024-09-10T09:44:25.355 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:24.545+0000 7f5dd8a00640 1 -- 172.21.15.77:0/622162173 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5dc4040070 con 0x7f5dd407de30 2024-09-10T09:44:25.355 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:24.545+0000 7f5dd3e00640 1 -- 172.21.15.77:0/622162173 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dd419e000 con 0x7f5dd407de30 2024-09-10T09:44:25.355 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:24.545+0000 7f5dd8a00640 1 -- 172.21.15.77:0/622162173 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5dc403a390 con 0x7f5dd407de30 2024-09-10T09:44:25.355 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:24.545+0000 7f5dd3e00640 1 -- 172.21.15.77:0/622162173 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dd41a0cf0 con 0x7f5dd407de30 2024-09-10T09:44:25.355 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:24.546+0000 7f5dd8a00640 1 -- 172.21.15.77:0/622162173 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f5dc403a5e0 con 0x7f5dd407de30 2024-09-10T09:44:25.355 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:24.546+0000 7f5dd3e00640 1 -- 172.21.15.77:0/622162173 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5dd407c310 con 0x7f5dd407de30 2024-09-10T09:44:25.355 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:24.546+0000 7f5dd8a00640 1 --2- 172.21.15.77:0/622162173 >> [v2:172.21.15.77:6800/3235900503,v1:172.21.15.77:6801/3235900503] conn(0x7f5db8038f70 0x7f5db803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:25.356 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:24.546+0000 7f5dd8a00640 1 -- 172.21.15.77:0/622162173 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5dc406c0f0 con 0x7f5dd407de30 2024-09-10T09:44:25.356 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:24.547+0000 7f5dca000640 1 --2- 172.21.15.77:0/622162173 >> [v2:172.21.15.77:6800/3235900503,v1:172.21.15.77:6801/3235900503] conn(0x7f5db8038f70 0x7f5db803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:25.356 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:24.547+0000 7f5dca000640 1 --2- 172.21.15.77:0/622162173 >> [v2:172.21.15.77:6800/3235900503,v1:172.21.15.77:6801/3235900503] conn(0x7f5db8038f70 0x7f5db803b430 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc00ad30 tx=0x7f5dbc0093f0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:25.356 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:24.551+0000 7f5dd8a00640 1 -- 172.21.15.77:0/622162173 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5dc406be50 con 0x7f5dd407de30 2024-09-10T09:44:25.356 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:24.728+0000 7f5dd3e00640 1 -- 172.21.15.77:0/622162173 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f5dd419e480 con 0x7f5dd407de30 2024-09-10T09:44:25.356 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:24.729+0000 7f5dd8a00640 1 -- 172.21.15.77:0/622162173 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+429 (secure 0 0 0) 0x7f5dc4043760 con 0x7f5dd407de30 2024-09-10T09:44:25.356 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:24.732+0000 7f5dd3e00640 1 -- 172.21.15.77:0/622162173 >> [v2:172.21.15.77:6800/3235900503,v1:172.21.15.77:6801/3235900503] conn(0x7f5db8038f70 msgr2=0x7f5db803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:25.356 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:24.732+0000 7f5dd3e00640 1 --2- 172.21.15.77:0/622162173 >> [v2:172.21.15.77:6800/3235900503,v1:172.21.15.77:6801/3235900503] conn(0x7f5db8038f70 0x7f5db803b430 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc00ad30 tx=0x7f5dbc0093f0 comp rx=0 tx=0).stop 2024-09-10T09:44:25.356 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:24.732+0000 7f5dd3e00640 1 -- 172.21.15.77:0/622162173 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5dd407de30 msgr2=0x7f5dd419d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:25.356 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:24.732+0000 7f5dd3e00640 1 --2- 172.21.15.77:0/622162173 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5dd407de30 0x7f5dd419d830 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f5dc40097f0 tx=0x7f5dc4030d80 comp rx=0 tx=0).stop 2024-09-10T09:44:25.356 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:24.732+0000 7f5dd3e00640 1 -- 172.21.15.77:0/622162173 shutdown_connections 2024-09-10T09:44:25.356 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:24.732+0000 7f5dd3e00640 1 --2- 172.21.15.77:0/622162173 >> [v2:172.21.15.77:6800/3235900503,v1:172.21.15.77:6801/3235900503] conn(0x7f5db8038f70 0x7f5db803b430 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:25.356 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:24.732+0000 7f5dd3e00640 1 --2- 172.21.15.77:0/622162173 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5dd407de30 0x7f5dd419d830 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:25.356 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:24.732+0000 7f5dd3e00640 1 -- 172.21.15.77:0/622162173 >> 172.21.15.77:0/622162173 conn(0x7f5dd4102b70 msgr2=0x7f5dd4103860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:25.357 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:24.732+0000 7f5dd3e00640 1 -- 172.21.15.77:0/622162173 shutdown_connections 2024-09-10T09:44:25.357 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:24.732+0000 7f5dd3e00640 1 -- 172.21.15.77:0/622162173 wait complete. 2024-09-10T09:44:25.357 INFO:teuthology.orchestra.run.smithi077.stdout:Enabling cephadm module... 2024-09-10T09:44:25.622 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:25 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/622162173' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-10T09:44:26.637 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:26 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/3318011633' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-10T09:44:26.638 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:26 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: ignoring --setuser ceph since I am not root 2024-09-10T09:44:26.638 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:26 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: ignoring --setgroup ceph since I am not root 2024-09-10T09:44:26.638 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:26 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: 2024-09-10T09:44:26.565+0000 7f8fa35de040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T09:44:26.638 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:26 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: 2024-09-10T09:44:26.636+0000 7f8fa35de040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T09:44:26.904 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:26 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: 2024-09-10T09:44:26.819+0000 7f8fa35de040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-10T09:44:26.904 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:26 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: 2024-09-10T09:44:26.902+0000 7f8fa35de040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T09:44:26.955 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:25.848+0000 7ff5bfe00640 1 Processor -- start 2024-09-10T09:44:26.956 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:25.848+0000 7ff5bfe00640 1 -- start start 2024-09-10T09:44:26.956 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:25.848+0000 7ff5bfe00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff5c0109920 0x7ff5c0109d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:26.956 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:25.848+0000 7ff5bfe00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c010a2f0 con 0x7ff5c0109920 2024-09-10T09:44:26.956 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:25.848+0000 7ff5bea00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff5c0109920 0x7ff5c0109d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:26.956 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:25.848+0000 7ff5bea00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff5c0109920 0x7ff5c0109d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59248/0 (socket says 172.21.15.77:59248) 2024-09-10T09:44:26.956 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:25.848+0000 7ff5bea00640 1 -- 172.21.15.77:0/703162943 learned_addr learned my addr 172.21.15.77:0/703162943 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:44:26.956 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:25.849+0000 7ff5bea00640 1 -- 172.21.15.77:0/703162943 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5c010a780 con 0x7ff5c0109920 2024-09-10T09:44:26.956 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:25.850+0000 7ff5bea00640 1 --2- 172.21.15.77:0/703162943 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff5c0109920 0x7ff5c0109d20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0009820 tx=0x7ff5b002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b97aec0e84567323 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:26.956 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:25.850+0000 7ff5be000640 1 -- 172.21.15.77:0/703162943 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5b0037070 con 0x7ff5c0109920 2024-09-10T09:44:26.957 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:25.850+0000 7ff5be000640 1 -- 172.21.15.77:0/703162943 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff5b0032900 con 0x7ff5c0109920 2024-09-10T09:44:26.957 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:25.850+0000 7ff5be000640 1 -- 172.21.15.77:0/703162943 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5b0030850 con 0x7ff5c0109920 2024-09-10T09:44:26.957 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:25.851+0000 7ff5bfe00640 1 -- 172.21.15.77:0/703162943 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff5c0109920 msgr2=0x7ff5c0109d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:26.957 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:25.851+0000 7ff5bfe00640 1 --2- 172.21.15.77:0/703162943 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff5c0109920 0x7ff5c0109d20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0009820 tx=0x7ff5b002a270 comp rx=0 tx=0).stop 2024-09-10T09:44:26.957 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:25.851+0000 7ff5bfe00640 1 -- 172.21.15.77:0/703162943 shutdown_connections 2024-09-10T09:44:26.957 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:25.851+0000 7ff5bfe00640 1 --2- 172.21.15.77:0/703162943 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff5c0109920 0x7ff5c0109d20 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:26.957 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:25.851+0000 7ff5bfe00640 1 -- 172.21.15.77:0/703162943 >> 172.21.15.77:0/703162943 conn(0x7ff5c007cbe0 msgr2=0x7ff5c007cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:26.957 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:25.851+0000 7ff5bfe00640 1 -- 172.21.15.77:0/703162943 shutdown_connections 2024-09-10T09:44:26.957 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:25.851+0000 7ff5bfe00640 1 -- 172.21.15.77:0/703162943 wait complete. 2024-09-10T09:44:26.957 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:25.852+0000 7ff5bfe00640 1 Processor -- start 2024-09-10T09:44:26.957 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:25.852+0000 7ff5bfe00640 1 -- start start 2024-09-10T09:44:26.957 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:25.852+0000 7ff5bfe00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff5c0109920 0x7ff5c019d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:26.957 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:25.852+0000 7ff5bfe00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c010db30 con 0x7ff5c0109920 2024-09-10T09:44:26.958 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:25.852+0000 7ff5bea00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff5c0109920 0x7ff5c019d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:26.958 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:25.852+0000 7ff5bea00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff5c0109920 0x7ff5c019d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59252/0 (socket says 172.21.15.77:59252) 2024-09-10T09:44:26.958 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:25.853+0000 7ff5bea00640 1 -- 172.21.15.77:0/3318011633 learned_addr learned my addr 172.21.15.77:0/3318011633 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:44:26.958 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:25.853+0000 7ff5bea00640 1 -- 172.21.15.77:0/3318011633 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5c019de00 con 0x7ff5c0109920 2024-09-10T09:44:26.958 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:25.853+0000 7ff5bea00640 1 --2- 172.21.15.77:0/3318011633 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff5c0109920 0x7ff5c019d8c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff5b00097f0 tx=0x7ff5b0030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:26.958 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:25.853+0000 7ff5b7400640 1 -- 172.21.15.77:0/3318011633 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5b0037040 con 0x7ff5c0109920 2024-09-10T09:44:26.958 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:25.853+0000 7ff5b7400640 1 -- 172.21.15.77:0/3318011633 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff5b0040070 con 0x7ff5c0109920 2024-09-10T09:44:26.958 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:25.853+0000 7ff5b7400640 1 -- 172.21.15.77:0/3318011633 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5b003a390 con 0x7ff5c0109920 2024-09-10T09:44:26.958 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:25.853+0000 7ff5bfe00640 1 -- 172.21.15.77:0/3318011633 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5c019e090 con 0x7ff5c0109920 2024-09-10T09:44:26.958 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:25.854+0000 7ff5bfe00640 1 -- 172.21.15.77:0/3318011633 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5c01a0d80 con 0x7ff5c0109920 2024-09-10T09:44:26.958 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:25.855+0000 7ff5b7400640 1 -- 172.21.15.77:0/3318011633 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7ff5b003b430 con 0x7ff5c0109920 2024-09-10T09:44:26.958 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:25.855+0000 7ff5bfe00640 1 -- 172.21.15.77:0/3318011633 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5c0109da0 con 0x7ff5c0109920 2024-09-10T09:44:26.959 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:25.855+0000 7ff5b7400640 1 --2- 172.21.15.77:0/3318011633 >> [v2:172.21.15.77:6800/3235900503,v1:172.21.15.77:6801/3235900503] conn(0x7ff5a4038f70 0x7ff5a403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:26.959 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:25.855+0000 7ff5b7400640 1 -- 172.21.15.77:0/3318011633 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff5b006c0d0 con 0x7ff5c0109920 2024-09-10T09:44:26.959 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:25.855+0000 7ff5b7e00640 1 --2- 172.21.15.77:0/3318011633 >> [v2:172.21.15.77:6800/3235900503,v1:172.21.15.77:6801/3235900503] conn(0x7ff5a4038f70 0x7ff5a403b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:26.959 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:25.856+0000 7ff5b7e00640 1 --2- 172.21.15.77:0/3318011633 >> [v2:172.21.15.77:6800/3235900503,v1:172.21.15.77:6801/3235900503] conn(0x7ff5a4038f70 0x7ff5a403b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff5a80098c0 tx=0x7ff5a8006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:26.959 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:25.860+0000 7ff5b7400640 1 -- 172.21.15.77:0/3318011633 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff5b00371e0 con 0x7ff5c0109920 2024-09-10T09:44:26.959 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:26.083+0000 7ff5bfe00640 1 -- 172.21.15.77:0/3318011633 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7ff5c019e510 con 0x7ff5c0109920 2024-09-10T09:44:26.959 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:26.347+0000 7ff5b7400640 1 -- 172.21.15.77:0/3318011633 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7ff5b0032dd0 con 0x7ff5c0109920 2024-09-10T09:44:26.959 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:26.348+0000 7ff5b7400640 1 -- 172.21.15.77:0/3318011633 <== mon.0 v2:172.21.15.77:3300/0 8 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7ff5b0032900 con 0x7ff5c0109920 2024-09-10T09:44:26.959 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:26.350+0000 7ff5bfe00640 1 -- 172.21.15.77:0/3318011633 >> [v2:172.21.15.77:6800/3235900503,v1:172.21.15.77:6801/3235900503] conn(0x7ff5a4038f70 msgr2=0x7ff5a403b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:26.959 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:26.350+0000 7ff5bfe00640 1 --2- 172.21.15.77:0/3318011633 >> [v2:172.21.15.77:6800/3235900503,v1:172.21.15.77:6801/3235900503] conn(0x7ff5a4038f70 0x7ff5a403b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff5a80098c0 tx=0x7ff5a8006db0 comp rx=0 tx=0).stop 2024-09-10T09:44:26.959 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:26.350+0000 7ff5bfe00640 1 -- 172.21.15.77:0/3318011633 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff5c0109920 msgr2=0x7ff5c019d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:26.959 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:26.350+0000 7ff5bfe00640 1 --2- 172.21.15.77:0/3318011633 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff5c0109920 0x7ff5c019d8c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff5b00097f0 tx=0x7ff5b0030d80 comp rx=0 tx=0).stop 2024-09-10T09:44:26.959 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:26.350+0000 7ff5bfe00640 1 -- 172.21.15.77:0/3318011633 shutdown_connections 2024-09-10T09:44:26.959 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:26.350+0000 7ff5bfe00640 1 --2- 172.21.15.77:0/3318011633 >> [v2:172.21.15.77:6800/3235900503,v1:172.21.15.77:6801/3235900503] conn(0x7ff5a4038f70 0x7ff5a403b430 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:26.960 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:26.350+0000 7ff5bfe00640 1 --2- 172.21.15.77:0/3318011633 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff5c0109920 0x7ff5c019d8c0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:26.960 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:26.350+0000 7ff5bfe00640 1 -- 172.21.15.77:0/3318011633 >> 172.21.15.77:0/3318011633 conn(0x7ff5c007cbe0 msgr2=0x7ff5c01077d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:26.960 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:26.351+0000 7ff5bfe00640 1 -- 172.21.15.77:0/3318011633 shutdown_connections 2024-09-10T09:44:26.960 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:26.351+0000 7ff5bfe00640 1 -- 172.21.15.77:0/3318011633 wait complete. 2024-09-10T09:44:27.189 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:27 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: 2024-09-10T09:44:27.082+0000 7f8fa35de040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T09:44:27.493 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:27 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: 2024-09-10T09:44:27.251+0000 7f8fa35de040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T09:44:27.493 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:27 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/3318011633' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-10T09:44:27.493 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:27 smithi077 ceph-mon[30319]: mgrmap e5: x(active, since 5s) 2024-09-10T09:44:27.930 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:27 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: 2024-09-10T09:44:27.629+0000 7f8fa35de040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T09:44:27.930 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:27 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: 2024-09-10T09:44:27.702+0000 7f8fa35de040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T09:44:27.930 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:27 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: 2024-09-10T09:44:27.928+0000 7f8fa35de040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T09:44:28.445 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout { 2024-09-10T09:44:28.445 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-10T09:44:28.445 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-10T09:44:28.445 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-10T09:44:28.445 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-10T09:44:28.446 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout } 2024-09-10T09:44:28.446 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:27.477+0000 7fa56f200640 1 Processor -- start 2024-09-10T09:44:28.446 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:27.477+0000 7fa56f200640 1 -- start start 2024-09-10T09:44:28.446 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:27.478+0000 7fa56f200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa56807de40 0x7fa56807c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:28.446 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:27.478+0000 7fa56f200640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa56807c8c0 con 0x7fa56807de40 2024-09-10T09:44:28.446 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:27.478+0000 7fa56de00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa56807de40 0x7fa56807c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:28.446 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:27.478+0000 7fa56de00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa56807de40 0x7fa56807c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59276/0 (socket says 172.21.15.77:59276) 2024-09-10T09:44:28.446 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:27.478+0000 7fa56de00640 1 -- 172.21.15.77:0/3636629027 learned_addr learned my addr 172.21.15.77:0/3636629027 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:44:28.446 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:27.478+0000 7fa56de00640 1 -- 172.21.15.77:0/3636629027 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa56807ca40 con 0x7fa56807de40 2024-09-10T09:44:28.446 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:27.479+0000 7fa56de00640 1 --2- 172.21.15.77:0/3636629027 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa56807de40 0x7fa56807c2f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fa55c009820 tx=0x7fa55c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8131ce6d5509c292 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:28.446 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:27.480+0000 7fa56ca00640 1 -- 172.21.15.77:0/3636629027 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa55c037070 con 0x7fa56807de40 2024-09-10T09:44:28.446 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:27.480+0000 7fa56ca00640 1 -- 172.21.15.77:0/3636629027 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa55c032900 con 0x7fa56807de40 2024-09-10T09:44:28.446 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:27.480+0000 7fa56ca00640 1 -- 172.21.15.77:0/3636629027 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa55c030850 con 0x7fa56807de40 2024-09-10T09:44:28.446 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:27.481+0000 7fa56f200640 1 -- 172.21.15.77:0/3636629027 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa56807de40 msgr2=0x7fa56807c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:28.447 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:27.481+0000 7fa56f200640 1 --2- 172.21.15.77:0/3636629027 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa56807de40 0x7fa56807c2f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fa55c009820 tx=0x7fa55c02a270 comp rx=0 tx=0).stop 2024-09-10T09:44:28.447 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:27.481+0000 7fa56f200640 1 -- 172.21.15.77:0/3636629027 shutdown_connections 2024-09-10T09:44:28.447 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:27.481+0000 7fa56f200640 1 --2- 172.21.15.77:0/3636629027 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa56807de40 0x7fa56807c2f0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:28.447 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:27.481+0000 7fa56f200640 1 -- 172.21.15.77:0/3636629027 >> 172.21.15.77:0/3636629027 conn(0x7fa568102bb0 msgr2=0x7fa568104fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:28.447 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:27.481+0000 7fa56f200640 1 -- 172.21.15.77:0/3636629027 shutdown_connections 2024-09-10T09:44:28.447 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:27.481+0000 7fa56f200640 1 -- 172.21.15.77:0/3636629027 wait complete. 2024-09-10T09:44:28.447 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:27.482+0000 7fa56f200640 1 Processor -- start 2024-09-10T09:44:28.447 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:27.482+0000 7fa56f200640 1 -- start start 2024-09-10T09:44:28.447 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:27.482+0000 7fa56f200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa56807de40 0x7fa5681a1c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:28.447 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:27.482+0000 7fa56f200640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa5681095c0 con 0x7fa56807de40 2024-09-10T09:44:28.447 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:27.483+0000 7fa56de00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa56807de40 0x7fa5681a1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:28.447 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:27.483+0000 7fa56de00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa56807de40 0x7fa5681a1c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59288/0 (socket says 172.21.15.77:59288) 2024-09-10T09:44:28.447 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:27.483+0000 7fa56de00640 1 -- 172.21.15.77:0/1348026825 learned_addr learned my addr 172.21.15.77:0/1348026825 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:44:28.448 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:27.483+0000 7fa56de00640 1 -- 172.21.15.77:0/1348026825 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5681a21b0 con 0x7fa56807de40 2024-09-10T09:44:28.448 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:27.483+0000 7fa56de00640 1 --2- 172.21.15.77:0/1348026825 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa56807de40 0x7fa5681a1c70 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fa55c032ce0 tx=0x7fa55c032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:28.448 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:27.484+0000 7fa562a00640 1 -- 172.21.15.77:0/1348026825 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa55c042070 con 0x7fa56807de40 2024-09-10T09:44:28.448 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:27.484+0000 7fa562a00640 1 -- 172.21.15.77:0/1348026825 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa55c0308c0 con 0x7fa56807de40 2024-09-10T09:44:28.448 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:27.484+0000 7fa562a00640 1 -- 172.21.15.77:0/1348026825 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa55c037040 con 0x7fa56807de40 2024-09-10T09:44:28.448 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:27.484+0000 7fa56f200640 1 -- 172.21.15.77:0/1348026825 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5681a2440 con 0x7fa56807de40 2024-09-10T09:44:28.448 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:27.484+0000 7fa56f200640 1 -- 172.21.15.77:0/1348026825 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5681a5130 con 0x7fa56807de40 2024-09-10T09:44:28.448 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:27.485+0000 7fa56f200640 1 -- 172.21.15.77:0/1348026825 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa56807c370 con 0x7fa56807de40 2024-09-10T09:44:28.448 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:27.485+0000 7fa562a00640 1 -- 172.21.15.77:0/1348026825 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7fa55c045070 con 0x7fa56807de40 2024-09-10T09:44:28.448 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:27.486+0000 7fa562a00640 1 --2- 172.21.15.77:0/1348026825 >> [v2:172.21.15.77:6800/3235900503,v1:172.21.15.77:6801/3235900503] conn(0x7fa550038fc0 0x7fa55003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:28.448 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:27.486+0000 7fa562a00640 1 -- 172.21.15.77:0/1348026825 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa55c06bfc0 con 0x7fa56807de40 2024-09-10T09:44:28.448 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:27.486+0000 7fa56d400640 1 -- 172.21.15.77:0/1348026825 >> [v2:172.21.15.77:6800/3235900503,v1:172.21.15.77:6801/3235900503] conn(0x7fa550038fc0 msgr2=0x7fa55003b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.77:6800/3235900503 2024-09-10T09:44:28.448 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:27.486+0000 7fa56d400640 1 --2- 172.21.15.77:0/1348026825 >> [v2:172.21.15.77:6800/3235900503,v1:172.21.15.77:6801/3235900503] conn(0x7fa550038fc0 0x7fa55003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T09:44:28.448 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:27.491+0000 7fa562a00640 1 -- 172.21.15.77:0/1348026825 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa55c04b170 con 0x7fa56807de40 2024-09-10T09:44:28.449 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:27.687+0000 7fa56d400640 1 -- 172.21.15.77:0/1348026825 >> [v2:172.21.15.77:6800/3235900503,v1:172.21.15.77:6801/3235900503] conn(0x7fa550038fc0 msgr2=0x7fa55003b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.77:6800/3235900503 2024-09-10T09:44:28.449 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:27.687+0000 7fa56d400640 1 --2- 172.21.15.77:0/1348026825 >> [v2:172.21.15.77:6800/3235900503,v1:172.21.15.77:6801/3235900503] conn(0x7fa550038fc0 0x7fa55003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-10T09:44:28.449 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:27.721+0000 7fa56f200640 1 -- 172.21.15.77:0/1348026825 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fa56802d7c0 con 0x7fa56807de40 2024-09-10T09:44:28.449 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:27.721+0000 7fa562a00640 1 -- 172.21.15.77:0/1348026825 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7fa55c06b9e0 con 0x7fa56807de40 2024-09-10T09:44:28.449 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:27.724+0000 7fa56f200640 1 -- 172.21.15.77:0/1348026825 >> [v2:172.21.15.77:6800/3235900503,v1:172.21.15.77:6801/3235900503] conn(0x7fa550038fc0 msgr2=0x7fa55003b480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T09:44:28.449 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:27.724+0000 7fa56f200640 1 --2- 172.21.15.77:0/1348026825 >> [v2:172.21.15.77:6800/3235900503,v1:172.21.15.77:6801/3235900503] conn(0x7fa550038fc0 0x7fa55003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:28.449 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:27.724+0000 7fa56f200640 1 -- 172.21.15.77:0/1348026825 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa56807de40 msgr2=0x7fa5681a1c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:28.449 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:27.724+0000 7fa56f200640 1 --2- 172.21.15.77:0/1348026825 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa56807de40 0x7fa5681a1c70 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fa55c032ce0 tx=0x7fa55c032d10 comp rx=0 tx=0).stop 2024-09-10T09:44:28.449 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:27.724+0000 7fa56f200640 1 -- 172.21.15.77:0/1348026825 shutdown_connections 2024-09-10T09:44:28.449 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:27.724+0000 7fa56f200640 1 --2- 172.21.15.77:0/1348026825 >> [v2:172.21.15.77:6800/3235900503,v1:172.21.15.77:6801/3235900503] conn(0x7fa550038fc0 0x7fa55003b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:28.449 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:27.724+0000 7fa56f200640 1 --2- 172.21.15.77:0/1348026825 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa56807de40 0x7fa5681a1c70 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:28.449 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:27.724+0000 7fa56f200640 1 -- 172.21.15.77:0/1348026825 >> 172.21.15.77:0/1348026825 conn(0x7fa568102bb0 msgr2=0x7fa5681986f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:28.449 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:27.724+0000 7fa56f200640 1 -- 172.21.15.77:0/1348026825 shutdown_connections 2024-09-10T09:44:28.449 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:27.724+0000 7fa56f200640 1 -- 172.21.15.77:0/1348026825 wait complete. 2024-09-10T09:44:28.450 INFO:teuthology.orchestra.run.smithi077.stdout:Waiting for the mgr to restart... 2024-09-10T09:44:28.450 INFO:teuthology.orchestra.run.smithi077.stdout:Waiting for mgr epoch 5... 2024-09-10T09:44:28.689 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:28 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/1348026825' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-10T09:44:29.439 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:29 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: 2024-09-10T09:44:29.058+0000 7f8fa35de040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T09:44:29.439 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:29 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: 2024-09-10T09:44:29.138+0000 7f8fa35de040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T09:44:30.331 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:30 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: 2024-09-10T09:44:30.035+0000 7f8fa35de040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T09:44:30.331 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:30 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: 2024-09-10T09:44:30.168+0000 7f8fa35de040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T09:44:30.652 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:30 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: 2024-09-10T09:44:30.330+0000 7f8fa35de040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T09:44:30.652 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:30 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: 2024-09-10T09:44:30.401+0000 7f8fa35de040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T09:44:30.652 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:30 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: 2024-09-10T09:44:30.569+0000 7f8fa35de040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T09:44:30.652 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:30 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: 2024-09-10T09:44:30.650+0000 7f8fa35de040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T09:44:30.939 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:30 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: 2024-09-10T09:44:30.721+0000 7f8fa35de040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T09:44:30.939 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:30 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: 2024-09-10T09:44:30.898+0000 7f8fa35de040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T09:44:31.286 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:30 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: 2024-09-10T09:44:30.974+0000 7f8fa35de040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T09:44:31.286 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:31 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: 2024-09-10T09:44:31.052+0000 7f8fa35de040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T09:44:31.286 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:31 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: 2024-09-10T09:44:31.146+0000 7f8fa35de040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T09:44:31.286 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:31 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: 2024-09-10T09:44:31.216+0000 7f8fa35de040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T09:44:31.286 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:31 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: 2024-09-10T09:44:31.284+0000 7f8fa35de040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T09:44:32.439 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:32 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: 2024-09-10T09:44:32.182+0000 7f8fa35de040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T09:44:32.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:32 smithi077 ceph-mon[30319]: Active manager daemon x restarted 2024-09-10T09:44:32.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:32 smithi077 ceph-mon[30319]: Activating manager daemon x 2024-09-10T09:44:32.440 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:32 smithi077 ceph-mon[30319]: osdmap e2: 0 total, 0 up, 0 in 2024-09-10T09:44:32.440 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:32 smithi077 ceph-mon[30319]: mgrmap e6: x(active, starting, since 0.00394286s) 2024-09-10T09:44:32.440 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:32 smithi077 ceph-mon[30319]: from='mgr.14118 172.21.15.77:0/1382121611' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T09:44:32.440 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:32 smithi077 ceph-mon[30319]: from='mgr.14118 172.21.15.77:0/1382121611' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-10T09:44:32.440 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:32 smithi077 ceph-mon[30319]: from='mgr.14118 172.21.15.77:0/1382121611' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T09:44:32.440 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:32 smithi077 ceph-mon[30319]: from='mgr.14118 172.21.15.77:0/1382121611' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T09:44:32.440 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:32 smithi077 ceph-mon[30319]: from='mgr.14118 172.21.15.77:0/1382121611' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T09:44:32.440 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:32 smithi077 ceph-mon[30319]: Manager daemon x is now available 2024-09-10T09:44:32.440 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:32 smithi077 ceph-mon[30319]: from='mgr.14118 172.21.15.77:0/1382121611' entity='mgr.x' 2024-09-10T09:44:32.440 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:32 smithi077 ceph-mon[30319]: from='mgr.14118 172.21.15.77:0/1382121611' entity='mgr.x' 2024-09-10T09:44:32.440 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:32 smithi077 ceph-mon[30319]: from='mgr.14118 172.21.15.77:0/1382121611' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:44:32.440 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:32 smithi077 ceph-mon[30319]: from='mgr.14118 172.21.15.77:0/1382121611' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:44:32.440 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:32 smithi077 ceph-mon[30319]: from='mgr.14118 172.21.15.77:0/1382121611' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-10T09:44:33.666 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:33 smithi077 ceph-mon[30319]: Found migration_current of "None". Setting to last migration. 2024-09-10T09:44:33.667 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:33 smithi077 ceph-mon[30319]: from='mgr.14118 172.21.15.77:0/1382121611' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-10T09:44:33.667 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:33 smithi077 ceph-mon[30319]: mgrmap e7: x(active, since 1.00657s) 2024-09-10T09:44:33.980 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout { 2024-09-10T09:44:33.980 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-10T09:44:33.981 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-10T09:44:33.981 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout } 2024-09-10T09:44:33.981 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:28.918+0000 7f5eaa000640 1 Processor -- start 2024-09-10T09:44:33.981 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:28.918+0000 7f5eaa000640 1 -- start start 2024-09-10T09:44:33.981 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:28.918+0000 7f5eaa000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ea4106230 0x7f5ea4106630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:33.981 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:28.918+0000 7f5eaa000640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5ea4106c00 con 0x7f5ea4106230 2024-09-10T09:44:33.981 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:28.919+0000 7f5ea8c00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ea4106230 0x7f5ea4106630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:33.981 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:28.919+0000 7f5ea8c00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ea4106230 0x7f5ea4106630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59304/0 (socket says 172.21.15.77:59304) 2024-09-10T09:44:33.981 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:28.919+0000 7f5ea8c00640 1 -- 172.21.15.77:0/3040306129 learned_addr learned my addr 172.21.15.77:0/3040306129 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:44:33.981 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:28.919+0000 7f5ea8c00640 1 -- 172.21.15.77:0/3040306129 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ea4107430 con 0x7f5ea4106230 2024-09-10T09:44:33.981 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:28.920+0000 7f5ea8c00640 1 --2- 172.21.15.77:0/3040306129 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ea4106230 0x7f5ea4106630 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f5e98009820 tx=0x7f5e9802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a83fc613450496 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:33.982 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:28.920+0000 7f5ea3400640 1 -- 172.21.15.77:0/3040306129 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5e98037070 con 0x7f5ea4106230 2024-09-10T09:44:33.982 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:28.920+0000 7f5ea3400640 1 -- 172.21.15.77:0/3040306129 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5e98032900 con 0x7f5ea4106230 2024-09-10T09:44:33.982 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:28.920+0000 7f5ea3400640 1 -- 172.21.15.77:0/3040306129 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5e98030850 con 0x7f5ea4106230 2024-09-10T09:44:33.982 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:28.921+0000 7f5eaa000640 1 -- 172.21.15.77:0/3040306129 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ea4106230 msgr2=0x7f5ea4106630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:33.982 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:28.921+0000 7f5eaa000640 1 --2- 172.21.15.77:0/3040306129 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ea4106230 0x7f5ea4106630 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f5e98009820 tx=0x7f5e9802a270 comp rx=0 tx=0).stop 2024-09-10T09:44:33.982 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:28.921+0000 7f5eaa000640 1 -- 172.21.15.77:0/3040306129 shutdown_connections 2024-09-10T09:44:33.982 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:28.921+0000 7f5eaa000640 1 --2- 172.21.15.77:0/3040306129 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ea4106230 0x7f5ea4106630 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:33.982 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:28.921+0000 7f5eaa000640 1 -- 172.21.15.77:0/3040306129 >> 172.21.15.77:0/3040306129 conn(0x7f5ea41019c0 msgr2=0x7f5ea4103e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:33.982 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:28.921+0000 7f5eaa000640 1 -- 172.21.15.77:0/3040306129 shutdown_connections 2024-09-10T09:44:33.982 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:28.921+0000 7f5eaa000640 1 -- 172.21.15.77:0/3040306129 wait complete. 2024-09-10T09:44:33.982 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:28.922+0000 7f5eaa000640 1 Processor -- start 2024-09-10T09:44:33.983 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:28.922+0000 7f5eaa000640 1 -- start start 2024-09-10T09:44:33.983 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:28.923+0000 7f5eaa000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ea4106230 0x7f5ea419d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:33.983 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:28.923+0000 7f5eaa000640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5ea407b100 con 0x7f5ea4106230 2024-09-10T09:44:33.983 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:28.923+0000 7f5ea8c00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ea4106230 0x7f5ea419d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:33.983 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:28.923+0000 7f5ea8c00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ea4106230 0x7f5ea419d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59316/0 (socket says 172.21.15.77:59316) 2024-09-10T09:44:33.983 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:28.923+0000 7f5ea8c00640 1 -- 172.21.15.77:0/4000591775 learned_addr learned my addr 172.21.15.77:0/4000591775 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:44:33.983 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:28.923+0000 7f5ea8c00640 1 -- 172.21.15.77:0/4000591775 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ea419ddd0 con 0x7f5ea4106230 2024-09-10T09:44:33.983 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:28.924+0000 7f5ea8c00640 1 --2- 172.21.15.77:0/4000591775 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ea4106230 0x7f5ea419d890 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5e980097f0 tx=0x7f5e98030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:33.983 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:28.924+0000 7f5ea1600640 1 -- 172.21.15.77:0/4000591775 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5e98037040 con 0x7f5ea4106230 2024-09-10T09:44:33.983 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:28.924+0000 7f5ea1600640 1 -- 172.21.15.77:0/4000591775 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5e98040070 con 0x7f5ea4106230 2024-09-10T09:44:33.983 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:28.924+0000 7f5ea1600640 1 -- 172.21.15.77:0/4000591775 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5e9803a390 con 0x7f5ea4106230 2024-09-10T09:44:33.984 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:28.924+0000 7f5eaa000640 1 -- 172.21.15.77:0/4000591775 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ea419e060 con 0x7f5ea4106230 2024-09-10T09:44:33.984 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:28.924+0000 7f5eaa000640 1 -- 172.21.15.77:0/4000591775 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ea41a0d50 con 0x7f5ea4106230 2024-09-10T09:44:33.984 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:28.925+0000 7f5ea1600640 1 -- 172.21.15.77:0/4000591775 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f5e98032900 con 0x7f5ea4106230 2024-09-10T09:44:33.984 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:28.926+0000 7f5ea1600640 1 --2- 172.21.15.77:0/4000591775 >> [v2:172.21.15.77:6800/3235900503,v1:172.21.15.77:6801/3235900503] conn(0x7f5e8c038fc0 0x7f5e8c03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:33.984 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:28.926+0000 7f5ea1600640 1 -- 172.21.15.77:0/4000591775 --> [v2:172.21.15.77:6800/3235900503,v1:172.21.15.77:6801/3235900503] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f5e8c03bb90 con 0x7f5e8c038fc0 2024-09-10T09:44:33.984 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:28.926+0000 7f5ea1600640 1 -- 172.21.15.77:0/4000591775 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5e9806c0f0 con 0x7f5ea4106230 2024-09-10T09:44:33.984 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:28.926+0000 7f5ea3e00640 1 -- 172.21.15.77:0/4000591775 >> [v2:172.21.15.77:6800/3235900503,v1:172.21.15.77:6801/3235900503] conn(0x7f5e8c038fc0 msgr2=0x7f5e8c03b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.77:6800/3235900503 2024-09-10T09:44:33.984 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:28.926+0000 7f5ea3e00640 1 --2- 172.21.15.77:0/4000591775 >> [v2:172.21.15.77:6800/3235900503,v1:172.21.15.77:6801/3235900503] conn(0x7f5e8c038fc0 0x7f5e8c03b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T09:44:33.984 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:29.127+0000 7f5ea3e00640 1 -- 172.21.15.77:0/4000591775 >> [v2:172.21.15.77:6800/3235900503,v1:172.21.15.77:6801/3235900503] conn(0x7f5e8c038fc0 msgr2=0x7f5e8c03b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.77:6800/3235900503 2024-09-10T09:44:33.984 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:29.127+0000 7f5ea3e00640 1 --2- 172.21.15.77:0/4000591775 >> [v2:172.21.15.77:6800/3235900503,v1:172.21.15.77:6801/3235900503] conn(0x7f5e8c038fc0 0x7f5e8c03b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-10T09:44:33.985 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:29.528+0000 7f5ea3e00640 1 -- 172.21.15.77:0/4000591775 >> [v2:172.21.15.77:6800/3235900503,v1:172.21.15.77:6801/3235900503] conn(0x7f5e8c038fc0 msgr2=0x7f5e8c03b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.77:6800/3235900503 2024-09-10T09:44:33.985 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:29.528+0000 7f5ea3e00640 1 --2- 172.21.15.77:0/4000591775 >> [v2:172.21.15.77:6800/3235900503,v1:172.21.15.77:6801/3235900503] conn(0x7f5e8c038fc0 0x7f5e8c03b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-10T09:44:33.985 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:30.329+0000 7f5ea3e00640 1 -- 172.21.15.77:0/4000591775 >> [v2:172.21.15.77:6800/3235900503,v1:172.21.15.77:6801/3235900503] conn(0x7f5e8c038fc0 msgr2=0x7f5e8c03b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.77:6800/3235900503 2024-09-10T09:44:33.985 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:30.329+0000 7f5ea3e00640 1 --2- 172.21.15.77:0/4000591775 >> [v2:172.21.15.77:6800/3235900503,v1:172.21.15.77:6801/3235900503] conn(0x7f5e8c038fc0 0x7f5e8c03b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-10T09:44:33.985 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:31.930+0000 7f5ea3e00640 1 -- 172.21.15.77:0/4000591775 >> [v2:172.21.15.77:6800/3235900503,v1:172.21.15.77:6801/3235900503] conn(0x7f5e8c038fc0 msgr2=0x7f5e8c03b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.77:6800/3235900503 2024-09-10T09:44:33.985 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:31.930+0000 7f5ea3e00640 1 --2- 172.21.15.77:0/4000591775 >> [v2:172.21.15.77:6800/3235900503,v1:172.21.15.77:6801/3235900503] conn(0x7f5e8c038fc0 0x7f5e8c03b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-09-10T09:44:33.985 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:32.186+0000 7f5ea1600640 1 -- 172.21.15.77:0/4000591775 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mgrmap(e 6) ==== 46054+0+0 (secure 0 0 0) 0x7f5e9803a5e0 con 0x7f5ea4106230 2024-09-10T09:44:33.985 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:32.186+0000 7f5ea1600640 1 -- 172.21.15.77:0/4000591775 >> [v2:172.21.15.77:6800/3235900503,v1:172.21.15.77:6801/3235900503] conn(0x7f5e8c038fc0 msgr2=0x7f5e8c03b480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T09:44:33.985 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:32.186+0000 7f5ea1600640 1 --2- 172.21.15.77:0/4000591775 >> [v2:172.21.15.77:6800/3235900503,v1:172.21.15.77:6801/3235900503] conn(0x7f5e8c038fc0 0x7f5e8c03b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:33.985 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:33.188+0000 7f5ea1600640 1 -- 172.21.15.77:0/4000591775 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7f5e9806d770 con 0x7f5ea4106230 2024-09-10T09:44:33.985 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:33.188+0000 7f5ea1600640 1 --2- 172.21.15.77:0/4000591775 >> [v2:172.21.15.77:6800/1527070442,v1:172.21.15.77:6801/1527070442] conn(0x7f5e8c03c8f0 0x7f5e8c03ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:33.986 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:33.188+0000 7f5ea1600640 1 -- 172.21.15.77:0/4000591775 --> [v2:172.21.15.77:6800/1527070442,v1:172.21.15.77:6801/1527070442] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f5e98046e50 con 0x7f5e8c03c8f0 2024-09-10T09:44:33.986 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:33.188+0000 7f5ea3e00640 1 --2- 172.21.15.77:0/4000591775 >> [v2:172.21.15.77:6800/1527070442,v1:172.21.15.77:6801/1527070442] conn(0x7f5e8c03c8f0 0x7f5e8c03ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:33.986 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:33.190+0000 7f5ea3e00640 1 --2- 172.21.15.77:0/4000591775 >> [v2:172.21.15.77:6800/1527070442,v1:172.21.15.77:6801/1527070442] conn(0x7f5e8c03c8f0 0x7f5e8c03ece0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f5e90003980 tx=0x7f5e900092b0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:33.986 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:33.190+0000 7f5ea1600640 1 -- 172.21.15.77:0/4000591775 <== mgr.14118 v2:172.21.15.77:6800/1527070442 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f5e98046e50 con 0x7f5e8c03c8f0 2024-09-10T09:44:33.986 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:33.203+0000 7f5eaa000640 1 -- 172.21.15.77:0/4000591775 --> [v2:172.21.15.77:6800/1527070442,v1:172.21.15.77:6801/1527070442] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f5ea4106630 con 0x7f5e8c03c8f0 2024-09-10T09:44:33.986 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:33.204+0000 7f5ea1600640 1 -- 172.21.15.77:0/4000591775 <== mgr.14118 v2:172.21.15.77:6800/1527070442 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f5ea4106630 con 0x7f5e8c03c8f0 2024-09-10T09:44:33.986 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:33.204+0000 7f5eaa000640 1 -- 172.21.15.77:0/4000591775 >> [v2:172.21.15.77:6800/1527070442,v1:172.21.15.77:6801/1527070442] conn(0x7f5e8c03c8f0 msgr2=0x7f5e8c03ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:33.986 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:33.204+0000 7f5eaa000640 1 --2- 172.21.15.77:0/4000591775 >> [v2:172.21.15.77:6800/1527070442,v1:172.21.15.77:6801/1527070442] conn(0x7f5e8c03c8f0 0x7f5e8c03ece0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f5e90003980 tx=0x7f5e900092b0 comp rx=0 tx=0).stop 2024-09-10T09:44:33.986 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:33.204+0000 7f5eaa000640 1 -- 172.21.15.77:0/4000591775 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ea4106230 msgr2=0x7f5ea419d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:33.986 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:33.204+0000 7f5eaa000640 1 --2- 172.21.15.77:0/4000591775 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ea4106230 0x7f5ea419d890 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5e980097f0 tx=0x7f5e98030d80 comp rx=0 tx=0).stop 2024-09-10T09:44:33.986 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:33.204+0000 7f5eaa000640 1 -- 172.21.15.77:0/4000591775 shutdown_connections 2024-09-10T09:44:33.987 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:33.204+0000 7f5eaa000640 1 --2- 172.21.15.77:0/4000591775 >> [v2:172.21.15.77:6800/1527070442,v1:172.21.15.77:6801/1527070442] conn(0x7f5e8c03c8f0 0x7f5e8c03ece0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:33.987 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:33.204+0000 7f5eaa000640 1 --2- 172.21.15.77:0/4000591775 >> [v2:172.21.15.77:6800/3235900503,v1:172.21.15.77:6801/3235900503] conn(0x7f5e8c038fc0 0x7f5e8c03b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:33.987 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:33.204+0000 7f5eaa000640 1 --2- 172.21.15.77:0/4000591775 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ea4106230 0x7f5ea419d890 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:33.987 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:33.204+0000 7f5eaa000640 1 -- 172.21.15.77:0/4000591775 >> 172.21.15.77:0/4000591775 conn(0x7f5ea41019c0 msgr2=0x7f5ea4193e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:33.987 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:33.204+0000 7f5eaa000640 1 -- 172.21.15.77:0/4000591775 shutdown_connections 2024-09-10T09:44:33.987 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:33.204+0000 7f5eaa000640 1 -- 172.21.15.77:0/4000591775 wait complete. 2024-09-10T09:44:33.987 INFO:teuthology.orchestra.run.smithi077.stdout:mgr epoch 5 is available 2024-09-10T09:44:33.987 INFO:teuthology.orchestra.run.smithi077.stdout:Setting orchestrator backend to cephadm... 2024-09-10T09:44:34.242 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:34 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: 2024-09-10T09:44:33.998+0000 7f8f01c00640 -1 log_channel(cephadm) log [ERR] : [10/Sep/2024:09:44:34] ENGINE Error in HTTPServer.serve 2024-09-10T09:44:34.242 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:34 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: Traceback (most recent call last): 2024-09-10T09:44:34.243 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:34 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T09:44:34.243 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:34 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: self._connections.run(self.expiration_interval) 2024-09-10T09:44:34.243 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:34 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T09:44:34.243 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:34 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: self._run(expiration_interval) 2024-09-10T09:44:34.243 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:34 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T09:44:34.243 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:34 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T09:44:34.243 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:34 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T09:44:34.243 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:34 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T09:44:34.243 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:34 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T09:44:34.243 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:34 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: s = self.context.wrap_socket( 2024-09-10T09:44:34.243 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:34 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T09:44:34.243 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:34 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: return self.sslsocket_class._create( 2024-09-10T09:44:34.243 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:34 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T09:44:34.244 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:34 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: self.do_handshake() 2024-09-10T09:44:34.244 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:34 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T09:44:34.244 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:34 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: self._sslobj.do_handshake() 2024-09-10T09:44:34.244 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:34 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T09:44:34.244 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:34 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: 2024-09-10T09:44:34.675 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:34 smithi077 ceph-mon[30319]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-10T09:44:34.675 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:34 smithi077 ceph-mon[30319]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-10T09:44:34.675 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:34 smithi077 ceph-mon[30319]: from='mgr.14118 172.21.15.77:0/1382121611' entity='mgr.x' 2024-09-10T09:44:34.676 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:34 smithi077 ceph-mon[30319]: from='mgr.14118 172.21.15.77:0/1382121611' entity='mgr.x' 2024-09-10T09:44:34.676 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:34 smithi077 ceph-mon[30319]: from='mgr.14118 172.21.15.77:0/1382121611' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:44:35.338 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:35 smithi077 ceph-mon[30319]: [10/Sep/2024:09:44:33] ENGINE Bus STARTING 2024-09-10T09:44:35.338 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:35 smithi077 ceph-mon[30319]: [10/Sep/2024:09:44:33] ENGINE Serving on https://172.21.15.77:7150 2024-09-10T09:44:35.338 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:35 smithi077 ceph-mon[30319]: [10/Sep/2024:09:44:34] ENGINE Error in HTTPServer.serve 2024-09-10T09:44:35.338 INFO:journalctl@ceph.mon.a.smithi077.stdout: Traceback (most recent call last): 2024-09-10T09:44:35.338 INFO:journalctl@ceph.mon.a.smithi077.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T09:44:35.339 INFO:journalctl@ceph.mon.a.smithi077.stdout: self._connections.run(self.expiration_interval) 2024-09-10T09:44:35.339 INFO:journalctl@ceph.mon.a.smithi077.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T09:44:35.339 INFO:journalctl@ceph.mon.a.smithi077.stdout: self._run(expiration_interval) 2024-09-10T09:44:35.339 INFO:journalctl@ceph.mon.a.smithi077.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T09:44:35.339 INFO:journalctl@ceph.mon.a.smithi077.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T09:44:35.339 INFO:journalctl@ceph.mon.a.smithi077.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T09:44:35.339 INFO:journalctl@ceph.mon.a.smithi077.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T09:44:35.339 INFO:journalctl@ceph.mon.a.smithi077.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T09:44:35.339 INFO:journalctl@ceph.mon.a.smithi077.stdout: s = self.context.wrap_socket( 2024-09-10T09:44:35.339 INFO:journalctl@ceph.mon.a.smithi077.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T09:44:35.339 INFO:journalctl@ceph.mon.a.smithi077.stdout: return self.sslsocket_class._create( 2024-09-10T09:44:35.340 INFO:journalctl@ceph.mon.a.smithi077.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T09:44:35.340 INFO:journalctl@ceph.mon.a.smithi077.stdout: self.do_handshake() 2024-09-10T09:44:35.340 INFO:journalctl@ceph.mon.a.smithi077.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T09:44:35.340 INFO:journalctl@ceph.mon.a.smithi077.stdout: self._sslobj.do_handshake() 2024-09-10T09:44:35.340 INFO:journalctl@ceph.mon.a.smithi077.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T09:44:35.340 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:35 smithi077 ceph-mon[30319]: [10/Sep/2024:09:44:34] ENGINE Serving on http://172.21.15.77:8765 2024-09-10T09:44:35.340 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:35 smithi077 ceph-mon[30319]: [10/Sep/2024:09:44:34] ENGINE Bus STARTED 2024-09-10T09:44:35.340 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:35 smithi077 ceph-mon[30319]: mgrmap e8: x(active, since 2s) 2024-09-10T09:44:35.340 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:35 smithi077 ceph-mon[30319]: from='mgr.14118 172.21.15.77:0/1382121611' entity='mgr.x' 2024-09-10T09:44:35.340 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:35 smithi077 ceph-mon[30319]: from='mgr.14118 172.21.15.77:0/1382121611' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:44:35.348 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:34.472+0000 7f614a200640 1 Processor -- start 2024-09-10T09:44:35.348 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:34.472+0000 7f614a200640 1 -- start start 2024-09-10T09:44:35.348 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:34.472+0000 7f614a200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6144105e60 0x7f6144106260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:35.348 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:34.472+0000 7f614a200640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f614407bd90 con 0x7f6144105e60 2024-09-10T09:44:35.349 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:34.473+0000 7f6143400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6144105e60 0x7f6144106260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:35.349 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:34.473+0000 7f6143400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6144105e60 0x7f6144106260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46922/0 (socket says 172.21.15.77:46922) 2024-09-10T09:44:35.349 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:34.473+0000 7f6143400640 1 -- 172.21.15.77:0/1022997745 learned_addr learned my addr 172.21.15.77:0/1022997745 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:44:35.350 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:34.473+0000 7f6143400640 1 -- 172.21.15.77:0/1022997745 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61441067a0 con 0x7f6144105e60 2024-09-10T09:44:35.350 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:34.474+0000 7f6143400640 1 --2- 172.21.15.77:0/1022997745 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6144105e60 0x7f6144106260 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f6134009e80 tx=0x7f613402a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dabfef5af2fa322d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:35.350 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:34.474+0000 7f6142a00640 1 -- 172.21.15.77:0/1022997745 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6134036070 con 0x7f6144105e60 2024-09-10T09:44:35.350 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:34.474+0000 7f6142a00640 1 -- 172.21.15.77:0/1022997745 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f61340318c0 con 0x7f6144105e60 2024-09-10T09:44:35.350 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:34.474+0000 7f6142a00640 1 -- 172.21.15.77:0/1022997745 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6134007ab0 con 0x7f6144105e60 2024-09-10T09:44:35.350 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:34.475+0000 7f614a200640 1 -- 172.21.15.77:0/1022997745 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6144105e60 msgr2=0x7f6144106260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:35.350 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:34.475+0000 7f614a200640 1 --2- 172.21.15.77:0/1022997745 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6144105e60 0x7f6144106260 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f6134009e80 tx=0x7f613402a7a0 comp rx=0 tx=0).stop 2024-09-10T09:44:35.350 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:34.475+0000 7f614a200640 1 -- 172.21.15.77:0/1022997745 shutdown_connections 2024-09-10T09:44:35.350 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:34.475+0000 7f614a200640 1 --2- 172.21.15.77:0/1022997745 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6144105e60 0x7f6144106260 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:35.350 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:34.475+0000 7f614a200640 1 -- 172.21.15.77:0/1022997745 >> 172.21.15.77:0/1022997745 conn(0x7f6144101bf0 msgr2=0x7f6144104050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:35.350 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:34.475+0000 7f614a200640 1 -- 172.21.15.77:0/1022997745 shutdown_connections 2024-09-10T09:44:35.351 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:34.475+0000 7f614a200640 1 -- 172.21.15.77:0/1022997745 wait complete. 2024-09-10T09:44:35.351 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:34.476+0000 7f614a200640 1 Processor -- start 2024-09-10T09:44:35.351 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:34.476+0000 7f614a200640 1 -- start start 2024-09-10T09:44:35.351 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:34.476+0000 7f614a200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6144105e60 0x7f614419da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:35.351 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:34.476+0000 7f614a200640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6144107720 con 0x7f6144105e60 2024-09-10T09:44:35.351 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:34.477+0000 7f6143400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6144105e60 0x7f614419da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:35.351 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:34.477+0000 7f6143400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6144105e60 0x7f614419da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46924/0 (socket says 172.21.15.77:46924) 2024-09-10T09:44:35.351 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:34.477+0000 7f6143400640 1 -- 172.21.15.77:0/1617394681 learned_addr learned my addr 172.21.15.77:0/1617394681 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:44:35.351 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:34.477+0000 7f6143400640 1 -- 172.21.15.77:0/1617394681 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f614419df60 con 0x7f6144105e60 2024-09-10T09:44:35.351 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:34.477+0000 7f6143400640 1 --2- 172.21.15.77:0/1617394681 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6144105e60 0x7f614419da20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f613402acc0 tx=0x7f6134031ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:35.351 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:34.477+0000 7f6140c00640 1 -- 172.21.15.77:0/1617394681 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6134036050 con 0x7f6144105e60 2024-09-10T09:44:35.351 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:34.477+0000 7f6140c00640 1 -- 172.21.15.77:0/1617394681 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6134037040 con 0x7f6144105e60 2024-09-10T09:44:35.351 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:34.478+0000 7f614a200640 1 -- 172.21.15.77:0/1617394681 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f614419e1f0 con 0x7f6144105e60 2024-09-10T09:44:35.351 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:34.478+0000 7f6140c00640 1 -- 172.21.15.77:0/1617394681 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f613403d850 con 0x7f6144105e60 2024-09-10T09:44:35.352 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:34.478+0000 7f614a200640 1 -- 172.21.15.77:0/1617394681 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61441a0ee0 con 0x7f6144105e60 2024-09-10T09:44:35.352 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:34.479+0000 7f614a200640 1 -- 172.21.15.77:0/1617394681 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61200052c0 con 0x7f6144105e60 2024-09-10T09:44:35.352 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:34.479+0000 7f6140c00640 1 -- 172.21.15.77:0/1617394681 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f6134044020 con 0x7f6144105e60 2024-09-10T09:44:35.352 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:34.479+0000 7f6140c00640 1 --2- 172.21.15.77:0/1617394681 >> [v2:172.21.15.77:6800/1527070442,v1:172.21.15.77:6801/1527070442] conn(0x7f6128038f70 0x7f612803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:35.352 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:34.479+0000 7f6140c00640 1 -- 172.21.15.77:0/1617394681 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f6134042020 con 0x7f6144105e60 2024-09-10T09:44:35.352 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:34.480+0000 7f613aa00640 1 --2- 172.21.15.77:0/1617394681 >> [v2:172.21.15.77:6800/1527070442,v1:172.21.15.77:6801/1527070442] conn(0x7f6128038f70 0x7f612803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:35.352 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:34.480+0000 7f613aa00640 1 --2- 172.21.15.77:0/1617394681 >> [v2:172.21.15.77:6800/1527070442,v1:172.21.15.77:6801/1527070442] conn(0x7f6128038f70 0x7f612803b430 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f612c0098c0 tx=0x7f612c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:35.352 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:34.485+0000 7f6140c00640 1 -- 172.21.15.77:0/1617394681 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f613404b020 con 0x7f6144105e60 2024-09-10T09:44:35.352 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:34.673+0000 7f614a200640 1 -- 172.21.15.77:0/1617394681 --> [v2:172.21.15.77:6800/1527070442,v1:172.21.15.77:6801/1527070442] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f6120000bf0 con 0x7f6128038f70 2024-09-10T09:44:35.352 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:34.682+0000 7f6140c00640 1 -- 172.21.15.77:0/1617394681 <== mgr.14118 v2:172.21.15.77:6800/1527070442 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6120000bf0 con 0x7f6128038f70 2024-09-10T09:44:35.352 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:34.684+0000 7f614a200640 1 -- 172.21.15.77:0/1617394681 >> [v2:172.21.15.77:6800/1527070442,v1:172.21.15.77:6801/1527070442] conn(0x7f6128038f70 msgr2=0x7f612803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:35.352 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:34.684+0000 7f614a200640 1 --2- 172.21.15.77:0/1617394681 >> [v2:172.21.15.77:6800/1527070442,v1:172.21.15.77:6801/1527070442] conn(0x7f6128038f70 0x7f612803b430 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f612c0098c0 tx=0x7f612c006db0 comp rx=0 tx=0).stop 2024-09-10T09:44:35.352 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:34.684+0000 7f614a200640 1 -- 172.21.15.77:0/1617394681 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6144105e60 msgr2=0x7f614419da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:35.353 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:34.684+0000 7f614a200640 1 --2- 172.21.15.77:0/1617394681 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6144105e60 0x7f614419da20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f613402acc0 tx=0x7f6134031ca0 comp rx=0 tx=0).stop 2024-09-10T09:44:35.353 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:34.685+0000 7f614a200640 1 -- 172.21.15.77:0/1617394681 shutdown_connections 2024-09-10T09:44:35.353 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:34.685+0000 7f614a200640 1 --2- 172.21.15.77:0/1617394681 >> [v2:172.21.15.77:6800/1527070442,v1:172.21.15.77:6801/1527070442] conn(0x7f6128038f70 0x7f612803b430 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:35.353 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:34.685+0000 7f614a200640 1 --2- 172.21.15.77:0/1617394681 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6144105e60 0x7f614419da20 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:35.353 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:34.685+0000 7f614a200640 1 -- 172.21.15.77:0/1617394681 >> 172.21.15.77:0/1617394681 conn(0x7f6144101bf0 msgr2=0x7f6144194410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:35.353 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:34.685+0000 7f614a200640 1 -- 172.21.15.77:0/1617394681 shutdown_connections 2024-09-10T09:44:35.353 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:34.685+0000 7f614a200640 1 -- 172.21.15.77:0/1617394681 wait complete. 2024-09-10T09:44:36.647 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:36 smithi077 ceph-mon[30319]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:44:36.659 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-10T09:44:36.659 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:35.803+0000 7fce37200640 1 Processor -- start 2024-09-10T09:44:36.659 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:35.804+0000 7fce37200640 1 -- start start 2024-09-10T09:44:36.659 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:35.804+0000 7fce37200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fce38109900 0x7fce38109d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:36.659 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:35.804+0000 7fce37200640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fce3810a2d0 con 0x7fce38109900 2024-09-10T09:44:36.659 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:35.804+0000 7fce35e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fce38109900 0x7fce38109d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:36.659 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:35.804+0000 7fce35e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fce38109900 0x7fce38109d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46936/0 (socket says 172.21.15.77:46936) 2024-09-10T09:44:36.659 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:35.804+0000 7fce35e00640 1 -- 172.21.15.77:0/1881749136 learned_addr learned my addr 172.21.15.77:0/1881749136 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:44:36.659 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:35.805+0000 7fce35e00640 1 -- 172.21.15.77:0/1881749136 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce3810a7a0 con 0x7fce38109900 2024-09-10T09:44:36.660 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:35.806+0000 7fce35e00640 1 --2- 172.21.15.77:0/1881749136 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fce38109900 0x7fce38109d00 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fce28009820 tx=0x7fce2802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d86c96e1209c702c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:36.660 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:35.806+0000 7fce35400640 1 -- 172.21.15.77:0/1881749136 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fce28037070 con 0x7fce38109900 2024-09-10T09:44:36.660 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:35.806+0000 7fce35400640 1 -- 172.21.15.77:0/1881749136 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fce28032900 con 0x7fce38109900 2024-09-10T09:44:36.660 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:35.806+0000 7fce35400640 1 -- 172.21.15.77:0/1881749136 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fce28030850 con 0x7fce38109900 2024-09-10T09:44:36.660 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:35.807+0000 7fce37200640 1 -- 172.21.15.77:0/1881749136 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fce38109900 msgr2=0x7fce38109d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:36.660 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:35.807+0000 7fce37200640 1 --2- 172.21.15.77:0/1881749136 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fce38109900 0x7fce38109d00 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fce28009820 tx=0x7fce2802a270 comp rx=0 tx=0).stop 2024-09-10T09:44:36.660 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:35.807+0000 7fce37200640 1 -- 172.21.15.77:0/1881749136 shutdown_connections 2024-09-10T09:44:36.660 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:35.807+0000 7fce37200640 1 --2- 172.21.15.77:0/1881749136 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fce38109900 0x7fce38109d00 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:36.660 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:35.807+0000 7fce37200640 1 -- 172.21.15.77:0/1881749136 >> 172.21.15.77:0/1881749136 conn(0x7fce3807cb40 msgr2=0x7fce3807cf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:36.660 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:35.807+0000 7fce37200640 1 -- 172.21.15.77:0/1881749136 shutdown_connections 2024-09-10T09:44:36.660 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:35.807+0000 7fce37200640 1 -- 172.21.15.77:0/1881749136 wait complete. 2024-09-10T09:44:36.660 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:35.808+0000 7fce37200640 1 Processor -- start 2024-09-10T09:44:36.660 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:35.808+0000 7fce37200640 1 -- start start 2024-09-10T09:44:36.661 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:35.808+0000 7fce37200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fce38109900 0x7fce3819d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:36.661 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:35.808+0000 7fce37200640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fce3810db50 con 0x7fce38109900 2024-09-10T09:44:36.661 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:35.808+0000 7fce35e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fce38109900 0x7fce3819d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:36.661 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:35.808+0000 7fce35e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fce38109900 0x7fce3819d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46948/0 (socket says 172.21.15.77:46948) 2024-09-10T09:44:36.661 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:35.808+0000 7fce35e00640 1 -- 172.21.15.77:0/776925080 learned_addr learned my addr 172.21.15.77:0/776925080 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:44:36.661 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:35.809+0000 7fce35e00640 1 -- 172.21.15.77:0/776925080 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce3819ddc0 con 0x7fce38109900 2024-09-10T09:44:36.661 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:35.809+0000 7fce35e00640 1 --2- 172.21.15.77:0/776925080 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fce38109900 0x7fce3819d880 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fce28032ce0 tx=0x7fce28032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:36.661 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:35.809+0000 7fce2f400640 1 -- 172.21.15.77:0/776925080 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fce28042070 con 0x7fce38109900 2024-09-10T09:44:36.661 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:35.809+0000 7fce2f400640 1 -- 172.21.15.77:0/776925080 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fce280308c0 con 0x7fce38109900 2024-09-10T09:44:36.661 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:35.809+0000 7fce2f400640 1 -- 172.21.15.77:0/776925080 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fce28037040 con 0x7fce38109900 2024-09-10T09:44:36.661 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:35.809+0000 7fce37200640 1 -- 172.21.15.77:0/776925080 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce3819e050 con 0x7fce38109900 2024-09-10T09:44:36.661 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:35.809+0000 7fce37200640 1 -- 172.21.15.77:0/776925080 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce381a0d40 con 0x7fce38109900 2024-09-10T09:44:36.662 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:35.811+0000 7fce2f400640 1 -- 172.21.15.77:0/776925080 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fce28045070 con 0x7fce38109900 2024-09-10T09:44:36.662 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:35.811+0000 7fce37200640 1 -- 172.21.15.77:0/776925080 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce38109d80 con 0x7fce38109900 2024-09-10T09:44:36.662 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:35.811+0000 7fce2f400640 1 --2- 172.21.15.77:0/776925080 >> [v2:172.21.15.77:6800/1527070442,v1:172.21.15.77:6801/1527070442] conn(0x7fce1c038fc0 0x7fce1c03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:36.662 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:35.811+0000 7fce2f400640 1 -- 172.21.15.77:0/776925080 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fce2806bfe0 con 0x7fce38109900 2024-09-10T09:44:36.662 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:35.811+0000 7fce2d400640 1 --2- 172.21.15.77:0/776925080 >> [v2:172.21.15.77:6800/1527070442,v1:172.21.15.77:6801/1527070442] conn(0x7fce1c038fc0 0x7fce1c03b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:36.662 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:35.812+0000 7fce2d400640 1 --2- 172.21.15.77:0/776925080 >> [v2:172.21.15.77:6800/1527070442,v1:172.21.15.77:6801/1527070442] conn(0x7fce1c038fc0 0x7fce1c03b480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fce2000ad30 tx=0x7fce200093f0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:36.662 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:35.816+0000 7fce2f400640 1 -- 172.21.15.77:0/776925080 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fce2803c720 con 0x7fce38109900 2024-09-10T09:44:36.662 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:35.996+0000 7fce37200640 1 -- 172.21.15.77:0/776925080 --> [v2:172.21.15.77:6800/1527070442,v1:172.21.15.77:6801/1527070442] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7fce38062630 con 0x7fce1c038fc0 2024-09-10T09:44:36.662 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:35.997+0000 7fce2f400640 1 -- 172.21.15.77:0/776925080 <== mgr.14118 v2:172.21.15.77:6800/1527070442 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7fce38062630 con 0x7fce1c038fc0 2024-09-10T09:44:36.662 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:35.999+0000 7fce37200640 1 -- 172.21.15.77:0/776925080 >> [v2:172.21.15.77:6800/1527070442,v1:172.21.15.77:6801/1527070442] conn(0x7fce1c038fc0 msgr2=0x7fce1c03b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:36.662 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:35.999+0000 7fce37200640 1 --2- 172.21.15.77:0/776925080 >> [v2:172.21.15.77:6800/1527070442,v1:172.21.15.77:6801/1527070442] conn(0x7fce1c038fc0 0x7fce1c03b480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fce2000ad30 tx=0x7fce200093f0 comp rx=0 tx=0).stop 2024-09-10T09:44:36.662 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:35.999+0000 7fce37200640 1 -- 172.21.15.77:0/776925080 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fce38109900 msgr2=0x7fce3819d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:36.662 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:35.999+0000 7fce37200640 1 --2- 172.21.15.77:0/776925080 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fce38109900 0x7fce3819d880 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fce28032ce0 tx=0x7fce28032d10 comp rx=0 tx=0).stop 2024-09-10T09:44:36.663 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:35.999+0000 7fce37200640 1 -- 172.21.15.77:0/776925080 shutdown_connections 2024-09-10T09:44:36.663 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:35.999+0000 7fce37200640 1 --2- 172.21.15.77:0/776925080 >> [v2:172.21.15.77:6800/1527070442,v1:172.21.15.77:6801/1527070442] conn(0x7fce1c038fc0 0x7fce1c03b480 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:36.663 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:35.999+0000 7fce37200640 1 --2- 172.21.15.77:0/776925080 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fce38109900 0x7fce3819d880 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:36.663 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:35.999+0000 7fce37200640 1 -- 172.21.15.77:0/776925080 >> 172.21.15.77:0/776925080 conn(0x7fce3807cb40 msgr2=0x7fce381075e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:36.663 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:36.000+0000 7fce37200640 1 -- 172.21.15.77:0/776925080 shutdown_connections 2024-09-10T09:44:36.663 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:36.000+0000 7fce37200640 1 -- 172.21.15.77:0/776925080 wait complete. 2024-09-10T09:44:36.663 INFO:teuthology.orchestra.run.smithi077.stdout:Generating ssh key... 2024-09-10T09:44:37.550 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:37 smithi077 ceph-mon[30319]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:44:37.892 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:37 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: Generating public/private rsa key pair. 2024-09-10T09:44:37.892 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:37 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: Your identification has been saved in /tmp/tmp3u1a5ze4/key 2024-09-10T09:44:37.892 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:37 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: Your public key has been saved in /tmp/tmp3u1a5ze4/key.pub 2024-09-10T09:44:37.892 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:37 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: The key fingerprint is: 2024-09-10T09:44:37.893 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:37 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: SHA256:NTsVkRP4n409pvFVOwL8Vi7GorJMFSVO6uUcMG0cMdw ceph-14a55226-6f59-11ef-bcea-c7b262605968 2024-09-10T09:44:37.893 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:37 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: The key's randomart image is: 2024-09-10T09:44:37.893 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:37 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: +---[RSA 3072]----+ 2024-09-10T09:44:37.893 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:37 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: | o+*+o++ | 2024-09-10T09:44:37.893 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:37 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: | *==Eo. | 2024-09-10T09:44:37.893 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:37 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: | ..=+... | 2024-09-10T09:44:37.893 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:37 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: | . +.o*. ..| 2024-09-10T09:44:37.893 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:37 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: | .S+o +.o=o| 2024-09-10T09:44:37.893 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:37 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: | . o O+*+| 2024-09-10T09:44:37.893 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:37 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: | . . + B +| 2024-09-10T09:44:37.893 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:37 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: | o. . . . | 2024-09-10T09:44:37.894 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:37 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: | oo | 2024-09-10T09:44:37.894 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:37 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: +----[SHA256]-----+ 2024-09-10T09:44:38.229 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:37.106+0000 7f92d9c00640 1 Processor -- start 2024-09-10T09:44:38.229 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:37.106+0000 7f92d9c00640 1 -- start start 2024-09-10T09:44:38.229 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:37.107+0000 7f92d9c00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f92d407de20 0x7f92d407c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:38.229 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:37.107+0000 7f92d9c00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f92d407c8a0 con 0x7f92d407de20 2024-09-10T09:44:38.229 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:37.107+0000 7f92d2a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f92d407de20 0x7f92d407c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:38.229 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:37.107+0000 7f92d2a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f92d407de20 0x7f92d407c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46952/0 (socket says 172.21.15.77:46952) 2024-09-10T09:44:38.229 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:37.107+0000 7f92d2a00640 1 -- 172.21.15.77:0/2859894387 learned_addr learned my addr 172.21.15.77:0/2859894387 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:44:38.229 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:37.108+0000 7f92d2a00640 1 -- 172.21.15.77:0/2859894387 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92d407ca20 con 0x7f92d407de20 2024-09-10T09:44:38.230 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:37.109+0000 7f92d2a00640 1 --2- 172.21.15.77:0/2859894387 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f92d407de20 0x7f92d407c2d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f92c4009820 tx=0x7f92c402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4d4d0704c30eac3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:38.230 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:37.109+0000 7f92d2000640 1 -- 172.21.15.77:0/2859894387 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f92c4037070 con 0x7f92d407de20 2024-09-10T09:44:38.230 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:37.109+0000 7f92d2000640 1 -- 172.21.15.77:0/2859894387 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f92c4032900 con 0x7f92d407de20 2024-09-10T09:44:38.230 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:37.109+0000 7f92d2000640 1 -- 172.21.15.77:0/2859894387 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f92c4030850 con 0x7f92d407de20 2024-09-10T09:44:38.230 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:37.109+0000 7f92d9c00640 1 -- 172.21.15.77:0/2859894387 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f92d407de20 msgr2=0x7f92d407c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:38.230 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:37.109+0000 7f92d9c00640 1 --2- 172.21.15.77:0/2859894387 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f92d407de20 0x7f92d407c2d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f92c4009820 tx=0x7f92c402a270 comp rx=0 tx=0).stop 2024-09-10T09:44:38.230 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:37.110+0000 7f92d9c00640 1 -- 172.21.15.77:0/2859894387 shutdown_connections 2024-09-10T09:44:38.230 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:37.110+0000 7f92d9c00640 1 --2- 172.21.15.77:0/2859894387 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f92d407de20 0x7f92d407c2d0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:38.230 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:37.110+0000 7f92d9c00640 1 -- 172.21.15.77:0/2859894387 >> 172.21.15.77:0/2859894387 conn(0x7f92d4102d20 msgr2=0x7f92d4105160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:38.230 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:37.110+0000 7f92d9c00640 1 -- 172.21.15.77:0/2859894387 shutdown_connections 2024-09-10T09:44:38.230 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:37.110+0000 7f92d9c00640 1 -- 172.21.15.77:0/2859894387 wait complete. 2024-09-10T09:44:38.231 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:37.111+0000 7f92d9c00640 1 Processor -- start 2024-09-10T09:44:38.231 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:37.111+0000 7f92d9c00640 1 -- start start 2024-09-10T09:44:38.231 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:37.111+0000 7f92d9c00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f92d407de20 0x7f92d419da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:38.231 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:37.111+0000 7f92d9c00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f92d410db80 con 0x7f92d407de20 2024-09-10T09:44:38.231 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:37.111+0000 7f92d2a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f92d407de20 0x7f92d419da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:38.231 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:37.111+0000 7f92d2a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f92d407de20 0x7f92d419da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46966/0 (socket says 172.21.15.77:46966) 2024-09-10T09:44:38.231 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:37.111+0000 7f92d2a00640 1 -- 172.21.15.77:0/3367491367 learned_addr learned my addr 172.21.15.77:0/3367491367 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:44:38.231 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:37.111+0000 7f92d2a00640 1 -- 172.21.15.77:0/3367491367 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92d419df70 con 0x7f92d407de20 2024-09-10T09:44:38.231 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:37.112+0000 7f92d2a00640 1 --2- 172.21.15.77:0/3367491367 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f92d407de20 0x7f92d419da30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f92c40097f0 tx=0x7f92c4030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:38.231 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:37.112+0000 7f92cbe00640 1 -- 172.21.15.77:0/3367491367 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f92c4037040 con 0x7f92d407de20 2024-09-10T09:44:38.231 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:37.112+0000 7f92cbe00640 1 -- 172.21.15.77:0/3367491367 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f92c4040070 con 0x7f92d407de20 2024-09-10T09:44:38.232 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:37.112+0000 7f92d9c00640 1 -- 172.21.15.77:0/3367491367 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92d419e200 con 0x7f92d407de20 2024-09-10T09:44:38.232 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:37.112+0000 7f92cbe00640 1 -- 172.21.15.77:0/3367491367 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f92c403a390 con 0x7f92d407de20 2024-09-10T09:44:38.232 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:37.112+0000 7f92d9c00640 1 -- 172.21.15.77:0/3367491367 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92d41a0ef0 con 0x7f92d407de20 2024-09-10T09:44:38.232 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:37.113+0000 7f92cbe00640 1 -- 172.21.15.77:0/3367491367 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f92c4032900 con 0x7f92d407de20 2024-09-10T09:44:38.232 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:37.113+0000 7f92d9c00640 1 -- 172.21.15.77:0/3367491367 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92d407c350 con 0x7f92d407de20 2024-09-10T09:44:38.232 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:37.114+0000 7f92cbe00640 1 --2- 172.21.15.77:0/3367491367 >> [v2:172.21.15.77:6800/1527070442,v1:172.21.15.77:6801/1527070442] conn(0x7f92b8038f70 0x7f92b803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:38.232 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:37.114+0000 7f92cbe00640 1 -- 172.21.15.77:0/3367491367 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f92c406c570 con 0x7f92d407de20 2024-09-10T09:44:38.232 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:37.114+0000 7f92ca000640 1 --2- 172.21.15.77:0/3367491367 >> [v2:172.21.15.77:6800/1527070442,v1:172.21.15.77:6801/1527070442] conn(0x7f92b8038f70 0x7f92b803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:38.232 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:37.115+0000 7f92ca000640 1 --2- 172.21.15.77:0/3367491367 >> [v2:172.21.15.77:6800/1527070442,v1:172.21.15.77:6801/1527070442] conn(0x7f92b8038f70 0x7f92b803b430 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f92bc0098c0 tx=0x7f92bc006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:38.232 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:37.119+0000 7f92cbe00640 1 -- 172.21.15.77:0/3367491367 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f92c406bec0 con 0x7f92d407de20 2024-09-10T09:44:38.233 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:37.294+0000 7f92d9c00640 1 -- 172.21.15.77:0/3367491367 --> [v2:172.21.15.77:6800/1527070442,v1:172.21.15.77:6801/1527070442] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f92d402d830 con 0x7f92b8038f70 2024-09-10T09:44:38.233 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:37.556+0000 7f92cbe00640 1 -- 172.21.15.77:0/3367491367 <== mgr.14118 v2:172.21.15.77:6800/1527070442 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f92d402d830 con 0x7f92b8038f70 2024-09-10T09:44:38.233 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:37.559+0000 7f92d9c00640 1 -- 172.21.15.77:0/3367491367 >> [v2:172.21.15.77:6800/1527070442,v1:172.21.15.77:6801/1527070442] conn(0x7f92b8038f70 msgr2=0x7f92b803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:38.233 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:37.559+0000 7f92d9c00640 1 --2- 172.21.15.77:0/3367491367 >> [v2:172.21.15.77:6800/1527070442,v1:172.21.15.77:6801/1527070442] conn(0x7f92b8038f70 0x7f92b803b430 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f92bc0098c0 tx=0x7f92bc006db0 comp rx=0 tx=0).stop 2024-09-10T09:44:38.233 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:37.559+0000 7f92d9c00640 1 -- 172.21.15.77:0/3367491367 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f92d407de20 msgr2=0x7f92d419da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:38.233 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:37.559+0000 7f92d9c00640 1 --2- 172.21.15.77:0/3367491367 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f92d407de20 0x7f92d419da30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f92c40097f0 tx=0x7f92c4030d80 comp rx=0 tx=0).stop 2024-09-10T09:44:38.233 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:37.559+0000 7f92d9c00640 1 -- 172.21.15.77:0/3367491367 shutdown_connections 2024-09-10T09:44:38.233 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:37.559+0000 7f92d9c00640 1 --2- 172.21.15.77:0/3367491367 >> [v2:172.21.15.77:6800/1527070442,v1:172.21.15.77:6801/1527070442] conn(0x7f92b8038f70 0x7f92b803b430 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:38.233 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:37.559+0000 7f92d9c00640 1 --2- 172.21.15.77:0/3367491367 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f92d407de20 0x7f92d419da30 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:38.233 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:37.559+0000 7f92d9c00640 1 -- 172.21.15.77:0/3367491367 >> 172.21.15.77:0/3367491367 conn(0x7f92d4102d20 msgr2=0x7f92d4103a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:38.233 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:37.559+0000 7f92d9c00640 1 -- 172.21.15.77:0/3367491367 shutdown_connections 2024-09-10T09:44:38.234 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:37.559+0000 7f92d9c00640 1 -- 172.21.15.77:0/3367491367 wait complete. 2024-09-10T09:44:38.543 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:38 smithi077 ceph-mon[30319]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:44:38.543 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:38 smithi077 ceph-mon[30319]: Generating ssh key... 2024-09-10T09:44:38.543 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:38 smithi077 ceph-mon[30319]: from='mgr.14118 172.21.15.77:0/1382121611' entity='mgr.x' 2024-09-10T09:44:38.543 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:38 smithi077 ceph-mon[30319]: from='mgr.14118 172.21.15.77:0/1382121611' entity='mgr.x' 2024-09-10T09:44:39.539 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:39 smithi077 ceph-mon[30319]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:44:39.550 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC17JjvNomGaHPZ1jKXRB2sCB5Kys7lr8ABNRYFrod4XZYGMm19CmJnRMLzWEHCkFO3CJZPB7JZfzZjZa5Aqh4vigtIl1Fh5jc92/MCRkmc6QpEi6if4zWdCIHXMJlcTq3XrtB0Beeb7J+9JGN+iRQzKbW4hFOtxJjmvgVNfjZfq/vS9V8XFHRFYeRu6BOKSGYOeBYeJjGkAzDO2aOxhq0dNiWT+0IxMeSPlWWZPo6x5WQ/HufAx1vTMaeGuCE88hBio2a+zua8bfYqVEJwzHszQlvmid7yhVPVCpLpWu11X2LDHlIobHCIp4c0oyVdrN5PqxbnWvMWXir6PXAQX9zoIR9wZpLDDmxbwusQ44oSrfAu8nn5qQ7PnATMnVRdKg+Hq0iciiBbtDPmiue4oXQhQm/0FiHXwTmcQeSrR2TKQ0hN5vFCvVWKsUbRdT0oRl/704wDTTqKSk/N0Uf5ukHMOW9jhpS4xoNYpF5nF/qZOmH+o6N3rHQCkk+fyp82odk= ceph-14a55226-6f59-11ef-bcea-c7b262605968 2024-09-10T09:44:39.550 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:38.756+0000 7f08b4000640 1 Processor -- start 2024-09-10T09:44:39.550 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:38.756+0000 7f08b4000640 1 -- start start 2024-09-10T09:44:39.550 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:38.756+0000 7f08b4000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f08ac07de10 0x7f08ac07c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:39.551 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:38.757+0000 7f08b4000640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f08ac07c7b0 con 0x7f08ac07de10 2024-09-10T09:44:39.551 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:38.757+0000 7f08b1600640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f08ac07de10 0x7f08ac07c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:39.551 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:38.757+0000 7f08b1600640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f08ac07de10 0x7f08ac07c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46980/0 (socket says 172.21.15.77:46980) 2024-09-10T09:44:39.551 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:38.757+0000 7f08b1600640 1 -- 172.21.15.77:0/2869914157 learned_addr learned my addr 172.21.15.77:0/2869914157 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:44:39.551 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:38.757+0000 7f08b1600640 1 -- 172.21.15.77:0/2869914157 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08ac07c930 con 0x7f08ac07de10 2024-09-10T09:44:39.551 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:38.758+0000 7f08b1600640 1 --2- 172.21.15.77:0/2869914157 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f08ac07de10 0x7f08ac07c270 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f08a0009820 tx=0x7f08a002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52c4900602860f3b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:39.551 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:38.758+0000 7f08abe00640 1 -- 172.21.15.77:0/2869914157 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f08a0037070 con 0x7f08ac07de10 2024-09-10T09:44:39.551 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:38.759+0000 7f08abe00640 1 -- 172.21.15.77:0/2869914157 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f08a0032900 con 0x7f08ac07de10 2024-09-10T09:44:39.551 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:38.759+0000 7f08abe00640 1 -- 172.21.15.77:0/2869914157 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f08a0030850 con 0x7f08ac07de10 2024-09-10T09:44:39.551 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:38.759+0000 7f08b4000640 1 -- 172.21.15.77:0/2869914157 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f08ac07de10 msgr2=0x7f08ac07c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:39.551 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:38.759+0000 7f08b4000640 1 --2- 172.21.15.77:0/2869914157 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f08ac07de10 0x7f08ac07c270 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f08a0009820 tx=0x7f08a002a270 comp rx=0 tx=0).stop 2024-09-10T09:44:39.551 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:38.759+0000 7f08b4000640 1 -- 172.21.15.77:0/2869914157 shutdown_connections 2024-09-10T09:44:39.551 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:38.759+0000 7f08b4000640 1 --2- 172.21.15.77:0/2869914157 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f08ac07de10 0x7f08ac07c270 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:39.552 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:38.759+0000 7f08b4000640 1 -- 172.21.15.77:0/2869914157 >> 172.21.15.77:0/2869914157 conn(0x7f08ac102cf0 msgr2=0x7f08ac105130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:39.552 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:38.759+0000 7f08b4000640 1 -- 172.21.15.77:0/2869914157 shutdown_connections 2024-09-10T09:44:39.552 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:38.759+0000 7f08b4000640 1 -- 172.21.15.77:0/2869914157 wait complete. 2024-09-10T09:44:39.552 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:38.760+0000 7f08b4000640 1 Processor -- start 2024-09-10T09:44:39.552 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:38.760+0000 7f08b4000640 1 -- start start 2024-09-10T09:44:39.552 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:38.761+0000 7f08b4000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f08ac07de10 0x7f08ac19da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:39.552 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:38.761+0000 7f08b4000640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f08ac10dbb0 con 0x7f08ac07de10 2024-09-10T09:44:39.552 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:38.761+0000 7f08b1600640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f08ac07de10 0x7f08ac19da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:39.552 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:38.761+0000 7f08b1600640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f08ac07de10 0x7f08ac19da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46996/0 (socket says 172.21.15.77:46996) 2024-09-10T09:44:39.552 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:38.761+0000 7f08b1600640 1 -- 172.21.15.77:0/4209956215 learned_addr learned my addr 172.21.15.77:0/4209956215 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:44:39.552 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:38.761+0000 7f08b1600640 1 -- 172.21.15.77:0/4209956215 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08ac19dfb0 con 0x7f08ac07de10 2024-09-10T09:44:39.552 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:38.761+0000 7f08b1600640 1 --2- 172.21.15.77:0/4209956215 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f08ac07de10 0x7f08ac19da70 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f08a00097f0 tx=0x7f08a0030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:39.552 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:38.762+0000 7f08aa000640 1 -- 172.21.15.77:0/4209956215 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f08a0042070 con 0x7f08ac07de10 2024-09-10T09:44:39.553 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:38.762+0000 7f08aa000640 1 -- 172.21.15.77:0/4209956215 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f08a0032ca0 con 0x7f08ac07de10 2024-09-10T09:44:39.553 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:38.762+0000 7f08aa000640 1 -- 172.21.15.77:0/4209956215 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f08a0037040 con 0x7f08ac07de10 2024-09-10T09:44:39.553 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:38.762+0000 7f08b4000640 1 -- 172.21.15.77:0/4209956215 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08ac19e240 con 0x7f08ac07de10 2024-09-10T09:44:39.553 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:38.762+0000 7f08b4000640 1 -- 172.21.15.77:0/4209956215 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08ac1a0f30 con 0x7f08ac07de10 2024-09-10T09:44:39.553 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:38.763+0000 7f08aa000640 1 -- 172.21.15.77:0/4209956215 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f08a0045070 con 0x7f08ac07de10 2024-09-10T09:44:39.553 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:38.763+0000 7f08b4000640 1 -- 172.21.15.77:0/4209956215 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08ac07c2f0 con 0x7f08ac07de10 2024-09-10T09:44:39.553 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:38.764+0000 7f08aa000640 1 --2- 172.21.15.77:0/4209956215 >> [v2:172.21.15.77:6800/1527070442,v1:172.21.15.77:6801/1527070442] conn(0x7f0894038fc0 0x7f089403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:39.553 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:38.764+0000 7f08aa000640 1 -- 172.21.15.77:0/4209956215 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f08a006bfc0 con 0x7f08ac07de10 2024-09-10T09:44:39.553 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:38.764+0000 7f08b0c00640 1 --2- 172.21.15.77:0/4209956215 >> [v2:172.21.15.77:6800/1527070442,v1:172.21.15.77:6801/1527070442] conn(0x7f0894038fc0 0x7f089403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:39.553 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:38.765+0000 7f08b0c00640 1 --2- 172.21.15.77:0/4209956215 >> [v2:172.21.15.77:6800/1527070442,v1:172.21.15.77:6801/1527070442] conn(0x7f0894038fc0 0x7f089403b480 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f08980098c0 tx=0x7f0898006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:39.553 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:38.769+0000 7f08aa000640 1 -- 172.21.15.77:0/4209956215 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f08a0045360 con 0x7f08ac07de10 2024-09-10T09:44:39.553 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:38.947+0000 7f08b4000640 1 -- 172.21.15.77:0/4209956215 --> [v2:172.21.15.77:6800/1527070442,v1:172.21.15.77:6801/1527070442] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f08ac062630 con 0x7f0894038fc0 2024-09-10T09:44:39.554 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:38.947+0000 7f08aa000640 1 -- 172.21.15.77:0/4209956215 <== mgr.14118 v2:172.21.15.77:6800/1527070442 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f08ac062630 con 0x7f0894038fc0 2024-09-10T09:44:39.554 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:38.950+0000 7f08b4000640 1 -- 172.21.15.77:0/4209956215 >> [v2:172.21.15.77:6800/1527070442,v1:172.21.15.77:6801/1527070442] conn(0x7f0894038fc0 msgr2=0x7f089403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:39.554 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:38.950+0000 7f08b4000640 1 --2- 172.21.15.77:0/4209956215 >> [v2:172.21.15.77:6800/1527070442,v1:172.21.15.77:6801/1527070442] conn(0x7f0894038fc0 0x7f089403b480 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f08980098c0 tx=0x7f0898006db0 comp rx=0 tx=0).stop 2024-09-10T09:44:39.554 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:38.950+0000 7f08b4000640 1 -- 172.21.15.77:0/4209956215 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f08ac07de10 msgr2=0x7f08ac19da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:39.554 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:38.950+0000 7f08b4000640 1 --2- 172.21.15.77:0/4209956215 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f08ac07de10 0x7f08ac19da70 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f08a00097f0 tx=0x7f08a0030ec0 comp rx=0 tx=0).stop 2024-09-10T09:44:39.554 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:38.950+0000 7f08b4000640 1 -- 172.21.15.77:0/4209956215 shutdown_connections 2024-09-10T09:44:39.554 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:38.950+0000 7f08b4000640 1 --2- 172.21.15.77:0/4209956215 >> [v2:172.21.15.77:6800/1527070442,v1:172.21.15.77:6801/1527070442] conn(0x7f0894038fc0 0x7f089403b480 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:39.554 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:38.950+0000 7f08b4000640 1 --2- 172.21.15.77:0/4209956215 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f08ac07de10 0x7f08ac19da70 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:39.554 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:38.950+0000 7f08b4000640 1 -- 172.21.15.77:0/4209956215 >> 172.21.15.77:0/4209956215 conn(0x7f08ac102cf0 msgr2=0x7f08ac103a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:39.554 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:38.950+0000 7f08b4000640 1 -- 172.21.15.77:0/4209956215 shutdown_connections 2024-09-10T09:44:39.554 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:38.950+0000 7f08b4000640 1 -- 172.21.15.77:0/4209956215 wait complete. 2024-09-10T09:44:39.554 INFO:teuthology.orchestra.run.smithi077.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-10T09:44:39.555 INFO:teuthology.orchestra.run.smithi077.stdout:Adding key to root@localhost authorized_keys... 2024-09-10T09:44:39.562 INFO:teuthology.orchestra.run.smithi077.stdout:Adding host smithi077... 2024-09-10T09:44:40.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:40 smithi077 ceph-mon[30319]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi077", "addr": "172.21.15.77", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:44:41.822 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:41 smithi077 ceph-mon[30319]: Deploying cephadm binary to smithi077 2024-09-10T09:44:44.768 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout Added host 'smithi077' with addr '172.21.15.77' 2024-09-10T09:44:44.769 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:40.065+0000 7f6716800640 1 Processor -- start 2024-09-10T09:44:44.769 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:40.065+0000 7f6716800640 1 -- start start 2024-09-10T09:44:44.769 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:40.066+0000 7f6716800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6710105e40 0x7f6710106240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:44.769 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:40.066+0000 7f6716800640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f671007bd90 con 0x7f6710105e40 2024-09-10T09:44:44.769 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:40.066+0000 7f670f400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6710105e40 0x7f6710106240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:44.769 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:40.066+0000 7f670f400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6710105e40 0x7f6710106240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46998/0 (socket says 172.21.15.77:46998) 2024-09-10T09:44:44.769 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:40.066+0000 7f670f400640 1 -- 172.21.15.77:0/3637891583 learned_addr learned my addr 172.21.15.77:0/3637891583 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:44:44.769 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:40.066+0000 7f670f400640 1 -- 172.21.15.77:0/3637891583 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6710106780 con 0x7f6710105e40 2024-09-10T09:44:44.769 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:40.067+0000 7f670f400640 1 --2- 172.21.15.77:0/3637891583 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6710105e40 0x7f6710106240 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6700009820 tx=0x7f670002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cd2d5bbd851ad9f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:44.769 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:40.068+0000 7f670e000640 1 -- 172.21.15.77:0/3637891583 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6700037070 con 0x7f6710105e40 2024-09-10T09:44:44.769 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:40.068+0000 7f670e000640 1 -- 172.21.15.77:0/3637891583 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6700032900 con 0x7f6710105e40 2024-09-10T09:44:44.769 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:40.068+0000 7f670e000640 1 -- 172.21.15.77:0/3637891583 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6700030850 con 0x7f6710105e40 2024-09-10T09:44:44.770 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:40.068+0000 7f6716800640 1 -- 172.21.15.77:0/3637891583 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6710105e40 msgr2=0x7f6710106240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:44.770 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:40.068+0000 7f6716800640 1 --2- 172.21.15.77:0/3637891583 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6710105e40 0x7f6710106240 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6700009820 tx=0x7f670002a270 comp rx=0 tx=0).stop 2024-09-10T09:44:44.770 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:40.068+0000 7f6716800640 1 -- 172.21.15.77:0/3637891583 shutdown_connections 2024-09-10T09:44:44.770 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:40.068+0000 7f6716800640 1 --2- 172.21.15.77:0/3637891583 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6710105e40 0x7f6710106240 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:44.770 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:40.068+0000 7f6716800640 1 -- 172.21.15.77:0/3637891583 >> 172.21.15.77:0/3637891583 conn(0x7f6710101bb0 msgr2=0x7f6710104030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:44.770 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:40.068+0000 7f6716800640 1 -- 172.21.15.77:0/3637891583 shutdown_connections 2024-09-10T09:44:44.770 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:40.068+0000 7f6716800640 1 -- 172.21.15.77:0/3637891583 wait complete. 2024-09-10T09:44:44.770 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:40.069+0000 7f6716800640 1 Processor -- start 2024-09-10T09:44:44.771 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:40.069+0000 7f6716800640 1 -- start start 2024-09-10T09:44:44.771 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:40.070+0000 7f6716800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6710105e40 0x7f671019da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:44.771 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:40.070+0000 7f6716800640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6710107700 con 0x7f6710105e40 2024-09-10T09:44:44.771 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:40.070+0000 7f670f400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6710105e40 0x7f671019da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:44.771 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:40.070+0000 7f670f400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6710105e40 0x7f671019da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47004/0 (socket says 172.21.15.77:47004) 2024-09-10T09:44:44.771 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:40.070+0000 7f670f400640 1 -- 172.21.15.77:0/749975669 learned_addr learned my addr 172.21.15.77:0/749975669 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:44:44.771 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:40.070+0000 7f670f400640 1 -- 172.21.15.77:0/749975669 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f671019df50 con 0x7f6710105e40 2024-09-10T09:44:44.772 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:40.070+0000 7f670f400640 1 --2- 172.21.15.77:0/749975669 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6710105e40 0x7f671019da10 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f67000097f0 tx=0x7f6700030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:44.772 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:40.071+0000 7f6707e00640 1 -- 172.21.15.77:0/749975669 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6700037040 con 0x7f6710105e40 2024-09-10T09:44:44.772 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:40.071+0000 7f6707e00640 1 -- 172.21.15.77:0/749975669 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6700040070 con 0x7f6710105e40 2024-09-10T09:44:44.772 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:40.071+0000 7f6707e00640 1 -- 172.21.15.77:0/749975669 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f670003a390 con 0x7f6710105e40 2024-09-10T09:44:44.772 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:40.071+0000 7f6716800640 1 -- 172.21.15.77:0/749975669 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f671019e1e0 con 0x7f6710105e40 2024-09-10T09:44:44.772 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:40.071+0000 7f6716800640 1 -- 172.21.15.77:0/749975669 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67101a0ed0 con 0x7f6710105e40 2024-09-10T09:44:44.772 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:40.072+0000 7f6716800640 1 -- 172.21.15.77:0/749975669 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6710107dd0 con 0x7f6710105e40 2024-09-10T09:44:44.772 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:40.072+0000 7f6707e00640 1 -- 172.21.15.77:0/749975669 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f670003b430 con 0x7f6710105e40 2024-09-10T09:44:44.772 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:40.073+0000 7f6707e00640 1 --2- 172.21.15.77:0/749975669 >> [v2:172.21.15.77:6800/1527070442,v1:172.21.15.77:6801/1527070442] conn(0x7f66f4038fc0 0x7f66f403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:44.772 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:40.073+0000 7f6707e00640 1 -- 172.21.15.77:0/749975669 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f670006c540 con 0x7f6710105e40 2024-09-10T09:44:44.772 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:40.073+0000 7f670ea00640 1 --2- 172.21.15.77:0/749975669 >> [v2:172.21.15.77:6800/1527070442,v1:172.21.15.77:6801/1527070442] conn(0x7f66f4038fc0 0x7f66f403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:44.773 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:40.074+0000 7f670ea00640 1 --2- 172.21.15.77:0/749975669 >> [v2:172.21.15.77:6800/1527070442,v1:172.21.15.77:6801/1527070442] conn(0x7f66f4038fc0 0x7f66f403b480 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f66fc0098c0 tx=0x7f66fc006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:44.773 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:40.078+0000 7f6707e00640 1 -- 172.21.15.77:0/749975669 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f67000371e0 con 0x7f6710105e40 2024-09-10T09:44:44.773 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:40.257+0000 7f6716800640 1 -- 172.21.15.77:0/749975669 --> [v2:172.21.15.77:6800/1527070442,v1:172.21.15.77:6801/1527070442] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi077", "addr": "172.21.15.77", "target": ["mon-mgr", ""]}) -- 0x7f6710062630 con 0x7f66f4038fc0 2024-09-10T09:44:44.773 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:44.136+0000 7f6707e00640 1 -- 172.21.15.77:0/749975669 <== mgr.14118 v2:172.21.15.77:6800/1527070442 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f6710062630 con 0x7f66f4038fc0 2024-09-10T09:44:44.773 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:44.139+0000 7f6716800640 1 -- 172.21.15.77:0/749975669 >> [v2:172.21.15.77:6800/1527070442,v1:172.21.15.77:6801/1527070442] conn(0x7f66f4038fc0 msgr2=0x7f66f403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:44.773 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:44.139+0000 7f6716800640 1 --2- 172.21.15.77:0/749975669 >> [v2:172.21.15.77:6800/1527070442,v1:172.21.15.77:6801/1527070442] conn(0x7f66f4038fc0 0x7f66f403b480 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f66fc0098c0 tx=0x7f66fc006db0 comp rx=0 tx=0).stop 2024-09-10T09:44:44.773 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:44.139+0000 7f6716800640 1 -- 172.21.15.77:0/749975669 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6710105e40 msgr2=0x7f671019da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:44.773 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:44.139+0000 7f6716800640 1 --2- 172.21.15.77:0/749975669 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6710105e40 0x7f671019da10 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f67000097f0 tx=0x7f6700030d80 comp rx=0 tx=0).stop 2024-09-10T09:44:44.773 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:44.140+0000 7f6716800640 1 -- 172.21.15.77:0/749975669 shutdown_connections 2024-09-10T09:44:44.773 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:44.140+0000 7f6716800640 1 --2- 172.21.15.77:0/749975669 >> [v2:172.21.15.77:6800/1527070442,v1:172.21.15.77:6801/1527070442] conn(0x7f66f4038fc0 0x7f66f403b480 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:44.774 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:44.140+0000 7f6716800640 1 --2- 172.21.15.77:0/749975669 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6710105e40 0x7f671019da10 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:44.774 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:44.140+0000 7f6716800640 1 -- 172.21.15.77:0/749975669 >> 172.21.15.77:0/749975669 conn(0x7f6710101bb0 msgr2=0x7f6710194440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:44.774 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:44.140+0000 7f6716800640 1 -- 172.21.15.77:0/749975669 shutdown_connections 2024-09-10T09:44:44.774 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:44.140+0000 7f6716800640 1 -- 172.21.15.77:0/749975669 wait complete. 2024-09-10T09:44:44.774 INFO:teuthology.orchestra.run.smithi077.stdout:Deploying unmanaged mon service... 2024-09-10T09:44:45.301 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:45 smithi077 ceph-mon[30319]: from='mgr.14118 172.21.15.77:0/1382121611' entity='mgr.x' 2024-09-10T09:44:45.301 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:45 smithi077 ceph-mon[30319]: Added host smithi077 2024-09-10T09:44:45.301 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:45 smithi077 ceph-mon[30319]: from='mgr.14118 172.21.15.77:0/1382121611' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:44:46.689 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:46 smithi077 ceph-mon[30319]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:44:46.689 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:46 smithi077 ceph-mon[30319]: Saving service mon spec with placement count:5 2024-09-10T09:44:46.689 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:46 smithi077 ceph-mon[30319]: from='mgr.14118 172.21.15.77:0/1382121611' entity='mgr.x' 2024-09-10T09:44:47.240 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-10T09:44:47.240 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:45.299+0000 7f13f5a00640 1 Processor -- start 2024-09-10T09:44:47.240 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:45.299+0000 7f13f5a00640 1 -- start start 2024-09-10T09:44:47.240 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:45.299+0000 7f13f5a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f13f0109900 0x7f13f0109d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:47.240 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:45.299+0000 7f13f5a00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f13f010a2d0 con 0x7f13f0109900 2024-09-10T09:44:47.240 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:45.300+0000 7f13efe00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f13f0109900 0x7f13f0109d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:47.240 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:45.300+0000 7f13efe00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f13f0109900 0x7f13f0109d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:32826/0 (socket says 172.21.15.77:32826) 2024-09-10T09:44:47.240 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:45.300+0000 7f13efe00640 1 -- 172.21.15.77:0/1406746550 learned_addr learned my addr 172.21.15.77:0/1406746550 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:44:47.240 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:45.300+0000 7f13efe00640 1 -- 172.21.15.77:0/1406746550 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13f010a7a0 con 0x7f13f0109900 2024-09-10T09:44:47.241 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:45.301+0000 7f13efe00640 1 --2- 172.21.15.77:0/1406746550 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f13f0109900 0x7f13f0109d00 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f13e0009820 tx=0x7f13e002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5b08f1c5e47bb4c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:47.241 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:45.302+0000 7f13eea00640 1 -- 172.21.15.77:0/1406746550 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f13e0037070 con 0x7f13f0109900 2024-09-10T09:44:47.241 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:45.302+0000 7f13eea00640 1 -- 172.21.15.77:0/1406746550 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f13e0032900 con 0x7f13f0109900 2024-09-10T09:44:47.241 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:45.302+0000 7f13eea00640 1 -- 172.21.15.77:0/1406746550 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f13e0030850 con 0x7f13f0109900 2024-09-10T09:44:47.241 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:45.302+0000 7f13f5a00640 1 -- 172.21.15.77:0/1406746550 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f13f0109900 msgr2=0x7f13f0109d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:47.241 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:45.302+0000 7f13f5a00640 1 --2- 172.21.15.77:0/1406746550 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f13f0109900 0x7f13f0109d00 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f13e0009820 tx=0x7f13e002a270 comp rx=0 tx=0).stop 2024-09-10T09:44:47.241 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:45.302+0000 7f13f5a00640 1 -- 172.21.15.77:0/1406746550 shutdown_connections 2024-09-10T09:44:47.241 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:45.302+0000 7f13f5a00640 1 --2- 172.21.15.77:0/1406746550 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f13f0109900 0x7f13f0109d00 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:47.241 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:45.302+0000 7f13f5a00640 1 -- 172.21.15.77:0/1406746550 >> 172.21.15.77:0/1406746550 conn(0x7f13f007cb40 msgr2=0x7f13f007cf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:47.241 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:45.303+0000 7f13f5a00640 1 -- 172.21.15.77:0/1406746550 shutdown_connections 2024-09-10T09:44:47.242 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:45.303+0000 7f13f5a00640 1 -- 172.21.15.77:0/1406746550 wait complete. 2024-09-10T09:44:47.242 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:45.303+0000 7f13f5a00640 1 Processor -- start 2024-09-10T09:44:47.242 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:45.304+0000 7f13f5a00640 1 -- start start 2024-09-10T09:44:47.242 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:45.304+0000 7f13f5a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f13f0109900 0x7f13f019d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:47.242 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:45.304+0000 7f13f5a00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f13f010db50 con 0x7f13f0109900 2024-09-10T09:44:47.242 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:45.304+0000 7f13efe00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f13f0109900 0x7f13f019d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:47.242 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:45.304+0000 7f13efe00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f13f0109900 0x7f13f019d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:32834/0 (socket says 172.21.15.77:32834) 2024-09-10T09:44:47.242 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:45.304+0000 7f13efe00640 1 -- 172.21.15.77:0/3491528144 learned_addr learned my addr 172.21.15.77:0/3491528144 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:44:47.242 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:45.304+0000 7f13efe00640 1 -- 172.21.15.77:0/3491528144 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13f019ddc0 con 0x7f13f0109900 2024-09-10T09:44:47.242 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:45.305+0000 7f13efe00640 1 --2- 172.21.15.77:0/3491528144 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f13f0109900 0x7f13f019d880 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f13e00097f0 tx=0x7f13e0030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:47.242 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:45.305+0000 7f13ecc00640 1 -- 172.21.15.77:0/3491528144 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f13e0037040 con 0x7f13f0109900 2024-09-10T09:44:47.242 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:45.305+0000 7f13ecc00640 1 -- 172.21.15.77:0/3491528144 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f13e0040070 con 0x7f13f0109900 2024-09-10T09:44:47.242 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:45.305+0000 7f13f5a00640 1 -- 172.21.15.77:0/3491528144 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13f019e050 con 0x7f13f0109900 2024-09-10T09:44:47.242 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:45.305+0000 7f13ecc00640 1 -- 172.21.15.77:0/3491528144 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f13e003a390 con 0x7f13f0109900 2024-09-10T09:44:47.243 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:45.305+0000 7f13f5a00640 1 -- 172.21.15.77:0/3491528144 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13f01a0d40 con 0x7f13f0109900 2024-09-10T09:44:47.243 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:45.306+0000 7f13ecc00640 1 -- 172.21.15.77:0/3491528144 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f13e003b430 con 0x7f13f0109900 2024-09-10T09:44:47.243 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:45.306+0000 7f13f5a00640 1 -- 172.21.15.77:0/3491528144 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13f0109d80 con 0x7f13f0109900 2024-09-10T09:44:47.243 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:45.306+0000 7f13ecc00640 1 --2- 172.21.15.77:0/3491528144 >> [v2:172.21.15.77:6800/1527070442,v1:172.21.15.77:6801/1527070442] conn(0x7f13d4038f70 0x7f13d403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:47.243 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:45.306+0000 7f13ecc00640 1 -- 172.21.15.77:0/3491528144 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f13e006c4e0 con 0x7f13f0109900 2024-09-10T09:44:47.243 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:45.307+0000 7f13ef400640 1 --2- 172.21.15.77:0/3491528144 >> [v2:172.21.15.77:6800/1527070442,v1:172.21.15.77:6801/1527070442] conn(0x7f13d4038f70 0x7f13d403b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:47.243 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:45.307+0000 7f13ef400640 1 --2- 172.21.15.77:0/3491528144 >> [v2:172.21.15.77:6800/1527070442,v1:172.21.15.77:6801/1527070442] conn(0x7f13d4038f70 0x7f13d403b430 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f13dc0098c0 tx=0x7f13dc006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:47.243 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:45.311+0000 7f13ecc00640 1 -- 172.21.15.77:0/3491528144 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f13e006be30 con 0x7f13f0109900 2024-09-10T09:44:47.243 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:45.527+0000 7f13f5a00640 1 -- 172.21.15.77:0/3491528144 --> [v2:172.21.15.77:6800/1527070442,v1:172.21.15.77:6801/1527070442] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f13f0062630 con 0x7f13d4038f70 2024-09-10T09:44:47.243 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:45.533+0000 7f13ecc00640 1 -- 172.21.15.77:0/3491528144 <== mgr.14118 v2:172.21.15.77:6800/1527070442 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f13f0062630 con 0x7f13d4038f70 2024-09-10T09:44:47.243 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:45.535+0000 7f13f5a00640 1 -- 172.21.15.77:0/3491528144 >> [v2:172.21.15.77:6800/1527070442,v1:172.21.15.77:6801/1527070442] conn(0x7f13d4038f70 msgr2=0x7f13d403b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:47.243 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:45.535+0000 7f13f5a00640 1 --2- 172.21.15.77:0/3491528144 >> [v2:172.21.15.77:6800/1527070442,v1:172.21.15.77:6801/1527070442] conn(0x7f13d4038f70 0x7f13d403b430 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f13dc0098c0 tx=0x7f13dc006db0 comp rx=0 tx=0).stop 2024-09-10T09:44:47.243 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:45.535+0000 7f13f5a00640 1 -- 172.21.15.77:0/3491528144 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f13f0109900 msgr2=0x7f13f019d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:47.243 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:45.535+0000 7f13f5a00640 1 --2- 172.21.15.77:0/3491528144 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f13f0109900 0x7f13f019d880 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f13e00097f0 tx=0x7f13e0030d80 comp rx=0 tx=0).stop 2024-09-10T09:44:47.243 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:45.535+0000 7f13f5a00640 1 -- 172.21.15.77:0/3491528144 shutdown_connections 2024-09-10T09:44:47.244 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:45.535+0000 7f13f5a00640 1 --2- 172.21.15.77:0/3491528144 >> [v2:172.21.15.77:6800/1527070442,v1:172.21.15.77:6801/1527070442] conn(0x7f13d4038f70 0x7f13d403b430 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:47.244 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:45.535+0000 7f13f5a00640 1 --2- 172.21.15.77:0/3491528144 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f13f0109900 0x7f13f019d880 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:47.244 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:45.535+0000 7f13f5a00640 1 -- 172.21.15.77:0/3491528144 >> 172.21.15.77:0/3491528144 conn(0x7f13f007cb40 msgr2=0x7f13f01075e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:47.244 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:45.536+0000 7f13f5a00640 1 -- 172.21.15.77:0/3491528144 shutdown_connections 2024-09-10T09:44:47.244 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:45.536+0000 7f13f5a00640 1 -- 172.21.15.77:0/3491528144 wait complete. 2024-09-10T09:44:47.244 INFO:teuthology.orchestra.run.smithi077.stdout:Deploying unmanaged mgr service... 2024-09-10T09:44:48.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:47 smithi077 ceph-mon[30319]: from='mgr.14118 172.21.15.77:0/1382121611' entity='mgr.x' 2024-09-10T09:44:48.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:47 smithi077 ceph-mon[30319]: from='mgr.14118 172.21.15.77:0/1382121611' entity='mgr.x' 2024-09-10T09:44:49.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:48 smithi077 ceph-mon[30319]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:44:49.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:48 smithi077 ceph-mon[30319]: Saving service mgr spec with placement count:2 2024-09-10T09:44:49.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:48 smithi077 ceph-mon[30319]: from='mgr.14118 172.21.15.77:0/1382121611' entity='mgr.x' 2024-09-10T09:44:49.569 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-10T09:44:49.569 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:47.745+0000 7f8a72200640 1 Processor -- start 2024-09-10T09:44:49.569 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:47.746+0000 7f8a72200640 1 -- start start 2024-09-10T09:44:49.569 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:47.746+0000 7f8a72200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8a6c1097e0 0x7f8a6c109be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:49.569 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:47.746+0000 7f8a72200640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8a6c10a1b0 con 0x7f8a6c1097e0 2024-09-10T09:44:49.570 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:47.746+0000 7f8a6b400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8a6c1097e0 0x7f8a6c109be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:49.570 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:47.746+0000 7f8a6b400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8a6c1097e0 0x7f8a6c109be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:32844/0 (socket says 172.21.15.77:32844) 2024-09-10T09:44:49.570 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:47.746+0000 7f8a6b400640 1 -- 172.21.15.77:0/3740850977 learned_addr learned my addr 172.21.15.77:0/3740850977 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:44:49.570 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:47.747+0000 7f8a6b400640 1 -- 172.21.15.77:0/3740850977 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a6c10a990 con 0x7f8a6c1097e0 2024-09-10T09:44:49.570 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:47.747+0000 7f8a6b400640 1 --2- 172.21.15.77:0/3740850977 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8a6c1097e0 0x7f8a6c109be0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c009820 tx=0x7f8a5c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=690dbc6165600cb0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:49.570 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:47.748+0000 7f8a6aa00640 1 -- 172.21.15.77:0/3740850977 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8a5c037070 con 0x7f8a6c1097e0 2024-09-10T09:44:49.570 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:47.748+0000 7f8a6aa00640 1 -- 172.21.15.77:0/3740850977 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a5c032920 con 0x7f8a6c1097e0 2024-09-10T09:44:49.570 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:47.748+0000 7f8a6aa00640 1 -- 172.21.15.77:0/3740850977 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8a5c030810 con 0x7f8a6c1097e0 2024-09-10T09:44:49.570 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:47.748+0000 7f8a72200640 1 -- 172.21.15.77:0/3740850977 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8a6c1097e0 msgr2=0x7f8a6c109be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:49.570 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:47.748+0000 7f8a72200640 1 --2- 172.21.15.77:0/3740850977 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8a6c1097e0 0x7f8a6c109be0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c009820 tx=0x7f8a5c02a270 comp rx=0 tx=0).stop 2024-09-10T09:44:49.570 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:47.749+0000 7f8a72200640 1 -- 172.21.15.77:0/3740850977 shutdown_connections 2024-09-10T09:44:49.570 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:47.749+0000 7f8a72200640 1 --2- 172.21.15.77:0/3740850977 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8a6c1097e0 0x7f8a6c109be0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:49.570 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:47.749+0000 7f8a72200640 1 -- 172.21.15.77:0/3740850977 >> 172.21.15.77:0/3740850977 conn(0x7f8a6c07cdf0 msgr2=0x7f8a6c07d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:49.571 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:47.749+0000 7f8a72200640 1 -- 172.21.15.77:0/3740850977 shutdown_connections 2024-09-10T09:44:49.571 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:47.749+0000 7f8a72200640 1 -- 172.21.15.77:0/3740850977 wait complete. 2024-09-10T09:44:49.571 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:47.750+0000 7f8a72200640 1 Processor -- start 2024-09-10T09:44:49.571 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:47.750+0000 7f8a72200640 1 -- start start 2024-09-10T09:44:49.571 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:47.750+0000 7f8a72200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8a6c1097e0 0x7f8a6c19da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:49.571 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:47.750+0000 7f8a72200640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8a6c10dd40 con 0x7f8a6c1097e0 2024-09-10T09:44:49.571 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:47.751+0000 7f8a6b400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8a6c1097e0 0x7f8a6c19da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:49.571 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:47.751+0000 7f8a6b400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8a6c1097e0 0x7f8a6c19da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:32858/0 (socket says 172.21.15.77:32858) 2024-09-10T09:44:49.571 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:47.751+0000 7f8a6b400640 1 -- 172.21.15.77:0/3126382362 learned_addr learned my addr 172.21.15.77:0/3126382362 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:44:49.571 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:47.751+0000 7f8a6b400640 1 -- 172.21.15.77:0/3126382362 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a6c19dfc0 con 0x7f8a6c1097e0 2024-09-10T09:44:49.571 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:47.751+0000 7f8a6b400640 1 --2- 172.21.15.77:0/3126382362 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8a6c1097e0 0x7f8a6c19da80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c030560 tx=0x7f8a5c032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:49.572 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:47.751+0000 7f8a68c00640 1 -- 172.21.15.77:0/3126382362 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8a5c037040 con 0x7f8a6c1097e0 2024-09-10T09:44:49.572 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:47.751+0000 7f8a68c00640 1 -- 172.21.15.77:0/3126382362 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a5c040070 con 0x7f8a6c1097e0 2024-09-10T09:44:49.572 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:47.751+0000 7f8a68c00640 1 -- 172.21.15.77:0/3126382362 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8a5c03a740 con 0x7f8a6c1097e0 2024-09-10T09:44:49.572 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:47.752+0000 7f8a72200640 1 -- 172.21.15.77:0/3126382362 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a6c19e250 con 0x7f8a6c1097e0 2024-09-10T09:44:49.572 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:47.752+0000 7f8a72200640 1 -- 172.21.15.77:0/3126382362 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a6c1a0f40 con 0x7f8a6c1097e0 2024-09-10T09:44:49.572 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:47.753+0000 7f8a68c00640 1 -- 172.21.15.77:0/3126382362 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f8a5c03bb90 con 0x7f8a6c1097e0 2024-09-10T09:44:49.572 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:47.753+0000 7f8a72200640 1 -- 172.21.15.77:0/3126382362 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a6c109be0 con 0x7f8a6c1097e0 2024-09-10T09:44:49.573 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:47.754+0000 7f8a68c00640 1 --2- 172.21.15.77:0/3126382362 >> [v2:172.21.15.77:6800/1527070442,v1:172.21.15.77:6801/1527070442] conn(0x7f8a50038fc0 0x7f8a5003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:49.573 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:47.754+0000 7f8a68c00640 1 -- 172.21.15.77:0/3126382362 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f8a5c06c600 con 0x7f8a6c1097e0 2024-09-10T09:44:49.573 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:47.754+0000 7f8a63e00640 1 --2- 172.21.15.77:0/3126382362 >> [v2:172.21.15.77:6800/1527070442,v1:172.21.15.77:6801/1527070442] conn(0x7f8a50038fc0 0x7f8a5003b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:49.573 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:47.754+0000 7f8a63e00640 1 --2- 172.21.15.77:0/3126382362 >> [v2:172.21.15.77:6800/1527070442,v1:172.21.15.77:6801/1527070442] conn(0x7f8a50038fc0 0x7f8a5003b480 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8a540098c0 tx=0x7f8a54006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:49.573 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:47.758+0000 7f8a68c00640 1 -- 172.21.15.77:0/3126382362 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8a5c04a200 con 0x7f8a6c1097e0 2024-09-10T09:44:49.573 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:47.965+0000 7f8a72200640 1 -- 172.21.15.77:0/3126382362 --> [v2:172.21.15.77:6800/1527070442,v1:172.21.15.77:6801/1527070442] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f8a6c107c90 con 0x7f8a50038fc0 2024-09-10T09:44:49.573 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:47.970+0000 7f8a68c00640 1 -- 172.21.15.77:0/3126382362 <== mgr.14118 v2:172.21.15.77:6800/1527070442 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f8a6c107c90 con 0x7f8a50038fc0 2024-09-10T09:44:49.573 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:47.972+0000 7f8a72200640 1 -- 172.21.15.77:0/3126382362 >> [v2:172.21.15.77:6800/1527070442,v1:172.21.15.77:6801/1527070442] conn(0x7f8a50038fc0 msgr2=0x7f8a5003b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:49.573 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:47.972+0000 7f8a72200640 1 --2- 172.21.15.77:0/3126382362 >> [v2:172.21.15.77:6800/1527070442,v1:172.21.15.77:6801/1527070442] conn(0x7f8a50038fc0 0x7f8a5003b480 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8a540098c0 tx=0x7f8a54006db0 comp rx=0 tx=0).stop 2024-09-10T09:44:49.573 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:47.972+0000 7f8a72200640 1 -- 172.21.15.77:0/3126382362 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8a6c1097e0 msgr2=0x7f8a6c19da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:49.574 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:47.972+0000 7f8a72200640 1 --2- 172.21.15.77:0/3126382362 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8a6c1097e0 0x7f8a6c19da80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c030560 tx=0x7f8a5c032670 comp rx=0 tx=0).stop 2024-09-10T09:44:49.574 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:47.972+0000 7f8a72200640 1 -- 172.21.15.77:0/3126382362 shutdown_connections 2024-09-10T09:44:49.574 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:47.972+0000 7f8a72200640 1 --2- 172.21.15.77:0/3126382362 >> [v2:172.21.15.77:6800/1527070442,v1:172.21.15.77:6801/1527070442] conn(0x7f8a50038fc0 0x7f8a5003b480 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:49.574 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:47.972+0000 7f8a72200640 1 --2- 172.21.15.77:0/3126382362 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8a6c1097e0 0x7f8a6c19da80 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:49.574 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:47.972+0000 7f8a72200640 1 -- 172.21.15.77:0/3126382362 >> 172.21.15.77:0/3126382362 conn(0x7f8a6c07cdf0 msgr2=0x7f8a6c107520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:49.574 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:47.973+0000 7f8a72200640 1 -- 172.21.15.77:0/3126382362 shutdown_connections 2024-09-10T09:44:49.574 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:47.973+0000 7f8a72200640 1 -- 172.21.15.77:0/3126382362 wait complete. 2024-09-10T09:44:50.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:50 smithi077 ceph-mon[30319]: from='mgr.14118 172.21.15.77:0/1382121611' entity='mgr.x' 2024-09-10T09:44:50.987 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:50.075+0000 7fef34c00640 1 Processor -- start 2024-09-10T09:44:50.987 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:50.076+0000 7fef34c00640 1 -- start start 2024-09-10T09:44:50.987 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:50.076+0000 7fef34c00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fef301097e0 0x7fef30109be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:50.987 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:50.076+0000 7fef34c00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fef3010a1b0 con 0x7fef301097e0 2024-09-10T09:44:50.987 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:50.076+0000 7fef2de00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fef301097e0 0x7fef30109be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:50.987 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:50.076+0000 7fef2de00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fef301097e0 0x7fef30109be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:32864/0 (socket says 172.21.15.77:32864) 2024-09-10T09:44:50.987 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:50.076+0000 7fef2de00640 1 -- 172.21.15.77:0/1514637638 learned_addr learned my addr 172.21.15.77:0/1514637638 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:44:50.987 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:50.077+0000 7fef2de00640 1 -- 172.21.15.77:0/1514637638 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef3010a990 con 0x7fef301097e0 2024-09-10T09:44:50.987 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:50.078+0000 7fef2de00640 1 --2- 172.21.15.77:0/1514637638 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fef301097e0 0x7fef30109be0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fef20009e80 tx=0x7fef2002a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5408e0e255a645c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:50.988 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:50.078+0000 7fef2ca00640 1 -- 172.21.15.77:0/1514637638 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fef20036070 con 0x7fef301097e0 2024-09-10T09:44:50.988 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:50.078+0000 7fef2ca00640 1 -- 172.21.15.77:0/1514637638 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef200318e0 con 0x7fef301097e0 2024-09-10T09:44:50.988 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:50.078+0000 7fef2ca00640 1 -- 172.21.15.77:0/1514637638 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fef20007a00 con 0x7fef301097e0 2024-09-10T09:44:50.988 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:50.079+0000 7fef34c00640 1 -- 172.21.15.77:0/1514637638 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fef301097e0 msgr2=0x7fef30109be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:50.988 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:50.079+0000 7fef34c00640 1 --2- 172.21.15.77:0/1514637638 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fef301097e0 0x7fef30109be0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fef20009e80 tx=0x7fef2002a7a0 comp rx=0 tx=0).stop 2024-09-10T09:44:50.988 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:50.079+0000 7fef34c00640 1 -- 172.21.15.77:0/1514637638 shutdown_connections 2024-09-10T09:44:50.988 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:50.079+0000 7fef34c00640 1 --2- 172.21.15.77:0/1514637638 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fef301097e0 0x7fef30109be0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:50.988 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:50.079+0000 7fef34c00640 1 -- 172.21.15.77:0/1514637638 >> 172.21.15.77:0/1514637638 conn(0x7fef3007cdf0 msgr2=0x7fef3007d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:50.988 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:50.079+0000 7fef34c00640 1 -- 172.21.15.77:0/1514637638 shutdown_connections 2024-09-10T09:44:50.988 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:50.079+0000 7fef34c00640 1 -- 172.21.15.77:0/1514637638 wait complete. 2024-09-10T09:44:50.988 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:50.080+0000 7fef34c00640 1 Processor -- start 2024-09-10T09:44:50.988 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:50.080+0000 7fef34c00640 1 -- start start 2024-09-10T09:44:50.989 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:50.080+0000 7fef34c00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fef301097e0 0x7fef3019da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:50.989 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:50.080+0000 7fef34c00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fef3010dd40 con 0x7fef301097e0 2024-09-10T09:44:50.989 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:50.081+0000 7fef2de00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fef301097e0 0x7fef3019da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:50.989 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:50.081+0000 7fef2de00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fef301097e0 0x7fef3019da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:32872/0 (socket says 172.21.15.77:32872) 2024-09-10T09:44:50.989 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:50.081+0000 7fef2de00640 1 -- 172.21.15.77:0/1913922463 learned_addr learned my addr 172.21.15.77:0/1913922463 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:44:50.989 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:50.081+0000 7fef2de00640 1 -- 172.21.15.77:0/1913922463 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef3019dfb0 con 0x7fef301097e0 2024-09-10T09:44:50.989 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:50.081+0000 7fef2de00640 1 --2- 172.21.15.77:0/1913922463 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fef301097e0 0x7fef3019da70 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fef2002acc0 tx=0x7fef20031cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:50.989 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:50.082+0000 7fef26a00640 1 -- 172.21.15.77:0/1913922463 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fef20036050 con 0x7fef301097e0 2024-09-10T09:44:50.989 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:50.082+0000 7fef26a00640 1 -- 172.21.15.77:0/1913922463 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef20037040 con 0x7fef301097e0 2024-09-10T09:44:50.989 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:50.082+0000 7fef26a00640 1 -- 172.21.15.77:0/1913922463 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fef2003b890 con 0x7fef301097e0 2024-09-10T09:44:50.989 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:50.082+0000 7fef34c00640 1 -- 172.21.15.77:0/1913922463 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef3019e240 con 0x7fef301097e0 2024-09-10T09:44:50.989 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:50.082+0000 7fef34c00640 1 -- 172.21.15.77:0/1913922463 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef301a0f30 con 0x7fef301097e0 2024-09-10T09:44:50.989 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:50.083+0000 7fef34c00640 1 -- 172.21.15.77:0/1913922463 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef30109be0 con 0x7fef301097e0 2024-09-10T09:44:50.989 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:50.084+0000 7fef26a00640 1 -- 172.21.15.77:0/1913922463 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fef20044020 con 0x7fef301097e0 2024-09-10T09:44:50.990 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:50.084+0000 7fef26a00640 1 --2- 172.21.15.77:0/1913922463 >> [v2:172.21.15.77:6800/1527070442,v1:172.21.15.77:6801/1527070442] conn(0x7fef14038fc0 0x7fef1403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:50.990 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:50.084+0000 7fef26a00640 1 -- 172.21.15.77:0/1913922463 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fef20042020 con 0x7fef301097e0 2024-09-10T09:44:50.990 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:50.084+0000 7fef2d400640 1 --2- 172.21.15.77:0/1913922463 >> [v2:172.21.15.77:6800/1527070442,v1:172.21.15.77:6801/1527070442] conn(0x7fef14038fc0 0x7fef1403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:50.990 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:50.085+0000 7fef2d400640 1 --2- 172.21.15.77:0/1913922463 >> [v2:172.21.15.77:6800/1527070442,v1:172.21.15.77:6801/1527070442] conn(0x7fef14038fc0 0x7fef1403b480 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fef180098c0 tx=0x7fef18006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:50.990 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:50.089+0000 7fef26a00640 1 -- 172.21.15.77:0/1913922463 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fef2004b020 con 0x7fef301097e0 2024-09-10T09:44:50.990 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:50.274+0000 7fef34c00640 1 -- 172.21.15.77:0/1913922463 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7fef301a1160 con 0x7fef301097e0 2024-09-10T09:44:50.990 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:50.277+0000 7fef26a00640 1 -- 172.21.15.77:0/1913922463 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7fef2004b200 con 0x7fef301097e0 2024-09-10T09:44:50.990 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:50.279+0000 7fef34c00640 1 -- 172.21.15.77:0/1913922463 >> [v2:172.21.15.77:6800/1527070442,v1:172.21.15.77:6801/1527070442] conn(0x7fef14038fc0 msgr2=0x7fef1403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:50.990 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:50.279+0000 7fef34c00640 1 --2- 172.21.15.77:0/1913922463 >> [v2:172.21.15.77:6800/1527070442,v1:172.21.15.77:6801/1527070442] conn(0x7fef14038fc0 0x7fef1403b480 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fef180098c0 tx=0x7fef18006db0 comp rx=0 tx=0).stop 2024-09-10T09:44:50.990 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:50.279+0000 7fef34c00640 1 -- 172.21.15.77:0/1913922463 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fef301097e0 msgr2=0x7fef3019da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:50.990 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:50.279+0000 7fef34c00640 1 --2- 172.21.15.77:0/1913922463 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fef301097e0 0x7fef3019da70 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fef2002acc0 tx=0x7fef20031cc0 comp rx=0 tx=0).stop 2024-09-10T09:44:50.990 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:50.280+0000 7fef34c00640 1 -- 172.21.15.77:0/1913922463 shutdown_connections 2024-09-10T09:44:50.990 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:50.280+0000 7fef34c00640 1 --2- 172.21.15.77:0/1913922463 >> [v2:172.21.15.77:6800/1527070442,v1:172.21.15.77:6801/1527070442] conn(0x7fef14038fc0 0x7fef1403b480 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:50.991 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:50.280+0000 7fef34c00640 1 --2- 172.21.15.77:0/1913922463 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fef301097e0 0x7fef3019da70 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:50.991 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:50.280+0000 7fef34c00640 1 -- 172.21.15.77:0/1913922463 >> 172.21.15.77:0/1913922463 conn(0x7fef3007cdf0 msgr2=0x7fef301074f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:50.991 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:50.280+0000 7fef34c00640 1 -- 172.21.15.77:0/1913922463 shutdown_connections 2024-09-10T09:44:50.991 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:50.280+0000 7fef34c00640 1 -- 172.21.15.77:0/1913922463 wait complete. 2024-09-10T09:44:51.502 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:51 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/1913922463' entity='client.admin' 2024-09-10T09:44:51.502 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:51 smithi077 ceph-mon[30319]: from='mgr.14118 172.21.15.77:0/1382121611' entity='mgr.x' 2024-09-10T09:44:52.418 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:51.501+0000 7f87f3200640 1 Processor -- start 2024-09-10T09:44:52.418 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:51.501+0000 7f87f3200640 1 -- start start 2024-09-10T09:44:52.418 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:51.501+0000 7f87f3200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87f4109920 0x7f87f4109d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:52.418 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:51.501+0000 7f87f3200640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f87f410a2f0 con 0x7f87f4109920 2024-09-10T09:44:52.418 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:51.502+0000 7f87f1e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87f4109920 0x7f87f4109d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:52.419 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:51.502+0000 7f87f1e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87f4109920 0x7f87f4109d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:32880/0 (socket says 172.21.15.77:32880) 2024-09-10T09:44:52.419 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:51.502+0000 7f87f1e00640 1 -- 172.21.15.77:0/3974082362 learned_addr learned my addr 172.21.15.77:0/3974082362 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:44:52.419 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:51.502+0000 7f87f1e00640 1 -- 172.21.15.77:0/3974082362 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87f410a780 con 0x7f87f4109920 2024-09-10T09:44:52.419 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:51.503+0000 7f87f1e00640 1 --2- 172.21.15.77:0/3974082362 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87f4109920 0x7f87f4109d20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f87e4009820 tx=0x7f87e402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84e1b5d0694a51f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:52.419 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:51.503+0000 7f87f1400640 1 -- 172.21.15.77:0/3974082362 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f87e4037070 con 0x7f87f4109920 2024-09-10T09:44:52.419 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:51.503+0000 7f87f1400640 1 -- 172.21.15.77:0/3974082362 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87e4032920 con 0x7f87f4109920 2024-09-10T09:44:52.419 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:51.503+0000 7f87f1400640 1 -- 172.21.15.77:0/3974082362 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f87e4030810 con 0x7f87f4109920 2024-09-10T09:44:52.419 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:51.504+0000 7f87f3200640 1 -- 172.21.15.77:0/3974082362 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87f4109920 msgr2=0x7f87f4109d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:52.419 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:51.504+0000 7f87f3200640 1 --2- 172.21.15.77:0/3974082362 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87f4109920 0x7f87f4109d20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f87e4009820 tx=0x7f87e402a270 comp rx=0 tx=0).stop 2024-09-10T09:44:52.419 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:51.504+0000 7f87f3200640 1 -- 172.21.15.77:0/3974082362 shutdown_connections 2024-09-10T09:44:52.419 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:51.504+0000 7f87f3200640 1 --2- 172.21.15.77:0/3974082362 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87f4109920 0x7f87f4109d20 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:52.419 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:51.504+0000 7f87f3200640 1 -- 172.21.15.77:0/3974082362 >> 172.21.15.77:0/3974082362 conn(0x7f87f407cbe0 msgr2=0x7f87f407cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:52.419 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:51.504+0000 7f87f3200640 1 -- 172.21.15.77:0/3974082362 shutdown_connections 2024-09-10T09:44:52.420 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:51.504+0000 7f87f3200640 1 -- 172.21.15.77:0/3974082362 wait complete. 2024-09-10T09:44:52.420 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:51.505+0000 7f87f3200640 1 Processor -- start 2024-09-10T09:44:52.420 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:51.505+0000 7f87f3200640 1 -- start start 2024-09-10T09:44:52.420 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:51.505+0000 7f87f3200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87f4109920 0x7f87f419d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:52.420 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:51.505+0000 7f87f3200640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f87f410db30 con 0x7f87f4109920 2024-09-10T09:44:52.420 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:51.506+0000 7f87f1e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87f4109920 0x7f87f419d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:52.420 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:51.506+0000 7f87f1e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87f4109920 0x7f87f419d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:32886/0 (socket says 172.21.15.77:32886) 2024-09-10T09:44:52.420 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:51.506+0000 7f87f1e00640 1 -- 172.21.15.77:0/1842458037 learned_addr learned my addr 172.21.15.77:0/1842458037 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:44:52.420 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:51.506+0000 7f87f1e00640 1 -- 172.21.15.77:0/1842458037 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87f419de40 con 0x7f87f4109920 2024-09-10T09:44:52.420 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:51.506+0000 7f87f1e00640 1 --2- 172.21.15.77:0/1842458037 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87f4109920 0x7f87f419d900 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f87e40097f0 tx=0x7f87e40327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:52.420 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:51.507+0000 7f87eb400640 1 -- 172.21.15.77:0/1842458037 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f87e4037040 con 0x7f87f4109920 2024-09-10T09:44:52.420 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:51.507+0000 7f87eb400640 1 -- 172.21.15.77:0/1842458037 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87e4030cc0 con 0x7f87f4109920 2024-09-10T09:44:52.420 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:51.507+0000 7f87f3200640 1 -- 172.21.15.77:0/1842458037 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87f419e0d0 con 0x7f87f4109920 2024-09-10T09:44:52.420 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:51.507+0000 7f87eb400640 1 -- 172.21.15.77:0/1842458037 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f87e403a640 con 0x7f87f4109920 2024-09-10T09:44:52.421 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:51.507+0000 7f87f3200640 1 -- 172.21.15.77:0/1842458037 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87f41a0dc0 con 0x7f87f4109920 2024-09-10T09:44:52.421 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:51.508+0000 7f87eb400640 1 -- 172.21.15.77:0/1842458037 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f87e403b3f0 con 0x7f87f4109920 2024-09-10T09:44:52.421 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:51.508+0000 7f87f3200640 1 -- 172.21.15.77:0/1842458037 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87f4109da0 con 0x7f87f4109920 2024-09-10T09:44:52.421 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:51.509+0000 7f87eb400640 1 --2- 172.21.15.77:0/1842458037 >> [v2:172.21.15.77:6800/1527070442,v1:172.21.15.77:6801/1527070442] conn(0x7f87d8038fc0 0x7f87d803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:52.421 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:51.509+0000 7f87eb400640 1 -- 172.21.15.77:0/1842458037 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f87e406c570 con 0x7f87f4109920 2024-09-10T09:44:52.421 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:51.509+0000 7f87e9400640 1 --2- 172.21.15.77:0/1842458037 >> [v2:172.21.15.77:6800/1527070442,v1:172.21.15.77:6801/1527070442] conn(0x7f87d8038fc0 0x7f87d803b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:52.421 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:51.510+0000 7f87e9400640 1 --2- 172.21.15.77:0/1842458037 >> [v2:172.21.15.77:6800/1527070442,v1:172.21.15.77:6801/1527070442] conn(0x7f87d8038fc0 0x7f87d803b480 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f87dc0098c0 tx=0x7f87dc006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:52.421 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:51.514+0000 7f87eb400640 1 -- 172.21.15.77:0/1842458037 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f87e4042e10 con 0x7f87f4109920 2024-09-10T09:44:52.421 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:51.692+0000 7f87f3200640 1 -- 172.21.15.77:0/1842458037 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f87f402d7c0 con 0x7f87f4109920 2024-09-10T09:44:52.421 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:51.695+0000 7f87eb400640 1 -- 172.21.15.77:0/1842458037 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f87e40328c0 con 0x7f87f4109920 2024-09-10T09:44:52.421 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:51.697+0000 7f87f3200640 1 -- 172.21.15.77:0/1842458037 >> [v2:172.21.15.77:6800/1527070442,v1:172.21.15.77:6801/1527070442] conn(0x7f87d8038fc0 msgr2=0x7f87d803b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:52.421 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:51.697+0000 7f87f3200640 1 --2- 172.21.15.77:0/1842458037 >> [v2:172.21.15.77:6800/1527070442,v1:172.21.15.77:6801/1527070442] conn(0x7f87d8038fc0 0x7f87d803b480 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f87dc0098c0 tx=0x7f87dc006db0 comp rx=0 tx=0).stop 2024-09-10T09:44:52.421 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:51.697+0000 7f87f3200640 1 -- 172.21.15.77:0/1842458037 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87f4109920 msgr2=0x7f87f419d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:52.421 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:51.697+0000 7f87f3200640 1 --2- 172.21.15.77:0/1842458037 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87f4109920 0x7f87f419d900 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f87e40097f0 tx=0x7f87e40327b0 comp rx=0 tx=0).stop 2024-09-10T09:44:52.422 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:51.697+0000 7f87f3200640 1 -- 172.21.15.77:0/1842458037 shutdown_connections 2024-09-10T09:44:52.422 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:51.697+0000 7f87f3200640 1 --2- 172.21.15.77:0/1842458037 >> [v2:172.21.15.77:6800/1527070442,v1:172.21.15.77:6801/1527070442] conn(0x7f87d8038fc0 0x7f87d803b480 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:52.422 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:51.697+0000 7f87f3200640 1 --2- 172.21.15.77:0/1842458037 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87f4109920 0x7f87f419d900 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:52.422 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:51.697+0000 7f87f3200640 1 -- 172.21.15.77:0/1842458037 >> 172.21.15.77:0/1842458037 conn(0x7f87f407cbe0 msgr2=0x7f87f4107710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:52.422 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:51.698+0000 7f87f3200640 1 -- 172.21.15.77:0/1842458037 shutdown_connections 2024-09-10T09:44:52.422 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:51.698+0000 7f87f3200640 1 -- 172.21.15.77:0/1842458037 wait complete. 2024-09-10T09:44:52.422 INFO:teuthology.orchestra.run.smithi077.stdout:Enabling the dashboard module... 2024-09-10T09:44:52.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:52 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/1842458037' entity='client.admin' 2024-09-10T09:44:52.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:52 smithi077 ceph-mon[30319]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:44:53.708 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:53 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/4134447241' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-10T09:44:54.013 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:53 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: ignoring --setuser ceph since I am not root 2024-09-10T09:44:54.014 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:53 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: ignoring --setgroup ceph since I am not root 2024-09-10T09:44:54.014 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:53 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: 2024-09-10T09:44:53.941+0000 7f28a962b040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T09:44:54.014 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:54 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: 2024-09-10T09:44:54.012+0000 7f28a962b040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T09:44:54.278 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:54 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: 2024-09-10T09:44:54.198+0000 7f28a962b040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-10T09:44:54.279 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:54 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: 2024-09-10T09:44:54.277+0000 7f28a962b040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T09:44:54.466 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:53.254+0000 7fa4ed000640 1 Processor -- start 2024-09-10T09:44:54.472 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:53.254+0000 7fa4ed000640 1 -- start start 2024-09-10T09:44:54.472 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:53.254+0000 7fa4ed000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa4e807de30 0x7fa4e807c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:54.472 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:53.254+0000 7fa4ed000640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e807c860 con 0x7fa4e807de30 2024-09-10T09:44:54.472 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:53.255+0000 7fa4e7400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa4e807de30 0x7fa4e807c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:54.472 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:53.255+0000 7fa4e7400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa4e807de30 0x7fa4e807c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57310/0 (socket says 172.21.15.77:57310) 2024-09-10T09:44:54.473 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:53.255+0000 7fa4e7400640 1 -- 172.21.15.77:0/37045640 learned_addr learned my addr 172.21.15.77:0/37045640 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:44:54.473 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:53.255+0000 7fa4e7400640 1 -- 172.21.15.77:0/37045640 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4e807c9e0 con 0x7fa4e807de30 2024-09-10T09:44:54.473 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:53.256+0000 7fa4e7400640 1 --2- 172.21.15.77:0/37045640 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa4e807de30 0x7fa4e807c290 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa4d8009820 tx=0x7fa4d802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df3d31c3d82b2422 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:54.473 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:53.256+0000 7fa4e6000640 1 -- 172.21.15.77:0/37045640 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa4d8037070 con 0x7fa4e807de30 2024-09-10T09:44:54.473 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:53.256+0000 7fa4e6000640 1 -- 172.21.15.77:0/37045640 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4d8032920 con 0x7fa4e807de30 2024-09-10T09:44:54.473 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:53.256+0000 7fa4e6000640 1 -- 172.21.15.77:0/37045640 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa4d8030810 con 0x7fa4e807de30 2024-09-10T09:44:54.474 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:53.257+0000 7fa4ed000640 1 -- 172.21.15.77:0/37045640 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa4e807de30 msgr2=0x7fa4e807c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:54.474 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:53.257+0000 7fa4ed000640 1 --2- 172.21.15.77:0/37045640 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa4e807de30 0x7fa4e807c290 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa4d8009820 tx=0x7fa4d802a270 comp rx=0 tx=0).stop 2024-09-10T09:44:54.474 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:53.257+0000 7fa4ed000640 1 -- 172.21.15.77:0/37045640 shutdown_connections 2024-09-10T09:44:54.474 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:53.257+0000 7fa4ed000640 1 --2- 172.21.15.77:0/37045640 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa4e807de30 0x7fa4e807c290 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:54.475 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:53.257+0000 7fa4ed000640 1 -- 172.21.15.77:0/37045640 >> 172.21.15.77:0/37045640 conn(0x7fa4e8102af0 msgr2=0x7fa4e8104f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:54.475 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:53.257+0000 7fa4ed000640 1 -- 172.21.15.77:0/37045640 shutdown_connections 2024-09-10T09:44:54.475 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:53.257+0000 7fa4ed000640 1 -- 172.21.15.77:0/37045640 wait complete. 2024-09-10T09:44:54.476 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:53.258+0000 7fa4ed000640 1 Processor -- start 2024-09-10T09:44:54.476 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:53.258+0000 7fa4ed000640 1 -- start start 2024-09-10T09:44:54.476 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:53.258+0000 7fa4ed000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa4e807de30 0x7fa4e819d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:54.476 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:53.258+0000 7fa4ed000640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e810da10 con 0x7fa4e807de30 2024-09-10T09:44:54.476 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:53.259+0000 7fa4e7400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa4e807de30 0x7fa4e819d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:54.476 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:53.259+0000 7fa4e7400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa4e807de30 0x7fa4e819d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57322/0 (socket says 172.21.15.77:57322) 2024-09-10T09:44:54.477 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:53.259+0000 7fa4e7400640 1 -- 172.21.15.77:0/4134447241 learned_addr learned my addr 172.21.15.77:0/4134447241 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:44:54.477 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:53.259+0000 7fa4e7400640 1 -- 172.21.15.77:0/4134447241 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4e819dde0 con 0x7fa4e807de30 2024-09-10T09:44:54.477 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:53.259+0000 7fa4e7400640 1 --2- 172.21.15.77:0/4134447241 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa4e807de30 0x7fa4e819d8a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa4d80097f0 tx=0x7fa4d80327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:54.477 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:53.260+0000 7fa4dfe00640 1 -- 172.21.15.77:0/4134447241 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa4d8037040 con 0x7fa4e807de30 2024-09-10T09:44:54.477 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:53.260+0000 7fa4dfe00640 1 -- 172.21.15.77:0/4134447241 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4d8030cc0 con 0x7fa4e807de30 2024-09-10T09:44:54.478 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:53.260+0000 7fa4dfe00640 1 -- 172.21.15.77:0/4134447241 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa4d803a6b0 con 0x7fa4e807de30 2024-09-10T09:44:54.478 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:53.260+0000 7fa4ed000640 1 -- 172.21.15.77:0/4134447241 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4e819e070 con 0x7fa4e807de30 2024-09-10T09:44:54.478 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:53.260+0000 7fa4ed000640 1 -- 172.21.15.77:0/4134447241 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4e81a0d60 con 0x7fa4e807de30 2024-09-10T09:44:54.478 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:53.261+0000 7fa4dfe00640 1 -- 172.21.15.77:0/4134447241 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fa4d803baf0 con 0x7fa4e807de30 2024-09-10T09:44:54.478 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:53.261+0000 7fa4ed000640 1 -- 172.21.15.77:0/4134447241 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4e807c310 con 0x7fa4e807de30 2024-09-10T09:44:54.478 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:53.262+0000 7fa4dfe00640 1 --2- 172.21.15.77:0/4134447241 >> [v2:172.21.15.77:6800/1527070442,v1:172.21.15.77:6801/1527070442] conn(0x7fa4cc038f70 0x7fa4cc03b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:54.478 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:53.262+0000 7fa4dfe00640 1 -- 172.21.15.77:0/4134447241 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa4d806c5c0 con 0x7fa4e807de30 2024-09-10T09:44:54.479 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:53.262+0000 7fa4e6a00640 1 --2- 172.21.15.77:0/4134447241 >> [v2:172.21.15.77:6800/1527070442,v1:172.21.15.77:6801/1527070442] conn(0x7fa4cc038f70 0x7fa4cc03b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:54.479 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:53.263+0000 7fa4e6a00640 1 --2- 172.21.15.77:0/4134447241 >> [v2:172.21.15.77:6800/1527070442,v1:172.21.15.77:6801/1527070442] conn(0x7fa4cc038f70 0x7fa4cc03b430 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa4d40098c0 tx=0x7fa4d4006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:54.479 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:53.267+0000 7fa4dfe00640 1 -- 172.21.15.77:0/4134447241 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa4d8042dc0 con 0x7fa4e807de30 2024-09-10T09:44:54.479 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:53.491+0000 7fa4ed000640 1 -- 172.21.15.77:0/4134447241 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7fa4e802d7c0 con 0x7fa4e807de30 2024-09-10T09:44:54.479 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:53.698+0000 7fa4dfe00640 1 -- 172.21.15.77:0/4134447241 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7fa4d806bf10 con 0x7fa4e807de30 2024-09-10T09:44:54.480 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:53.699+0000 7fa4dfe00640 1 -- 172.21.15.77:0/4134447241 <== mon.0 v2:172.21.15.77:3300/0 8 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7fa4d803b770 con 0x7fa4e807de30 2024-09-10T09:44:54.480 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:53.702+0000 7fa4ed000640 1 -- 172.21.15.77:0/4134447241 >> [v2:172.21.15.77:6800/1527070442,v1:172.21.15.77:6801/1527070442] conn(0x7fa4cc038f70 msgr2=0x7fa4cc03b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:54.480 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:53.702+0000 7fa4ed000640 1 --2- 172.21.15.77:0/4134447241 >> [v2:172.21.15.77:6800/1527070442,v1:172.21.15.77:6801/1527070442] conn(0x7fa4cc038f70 0x7fa4cc03b430 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa4d40098c0 tx=0x7fa4d4006db0 comp rx=0 tx=0).stop 2024-09-10T09:44:54.480 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:53.702+0000 7fa4ed000640 1 -- 172.21.15.77:0/4134447241 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa4e807de30 msgr2=0x7fa4e819d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:54.480 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:53.702+0000 7fa4ed000640 1 --2- 172.21.15.77:0/4134447241 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa4e807de30 0x7fa4e819d8a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa4d80097f0 tx=0x7fa4d80327b0 comp rx=0 tx=0).stop 2024-09-10T09:44:54.480 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:53.702+0000 7fa4ed000640 1 -- 172.21.15.77:0/4134447241 shutdown_connections 2024-09-10T09:44:54.481 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:53.702+0000 7fa4ed000640 1 --2- 172.21.15.77:0/4134447241 >> [v2:172.21.15.77:6800/1527070442,v1:172.21.15.77:6801/1527070442] conn(0x7fa4cc038f70 0x7fa4cc03b430 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:54.481 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:53.702+0000 7fa4ed000640 1 --2- 172.21.15.77:0/4134447241 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa4e807de30 0x7fa4e819d8a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:54.481 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:53.702+0000 7fa4ed000640 1 -- 172.21.15.77:0/4134447241 >> 172.21.15.77:0/4134447241 conn(0x7fa4e8102af0 msgr2=0x7fa4e8103810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:54.481 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:53.702+0000 7fa4ed000640 1 -- 172.21.15.77:0/4134447241 shutdown_connections 2024-09-10T09:44:54.481 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:53.702+0000 7fa4ed000640 1 -- 172.21.15.77:0/4134447241 wait complete. 2024-09-10T09:44:54.609 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:54 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: 2024-09-10T09:44:54.448+0000 7f28a962b040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T09:44:54.883 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:54 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: 2024-09-10T09:44:54.626+0000 7f28a962b040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T09:44:54.884 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:54 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/4134447241' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-10T09:44:54.884 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:54 smithi077 ceph-mon[30319]: mgrmap e9: x(active, since 21s) 2024-09-10T09:44:55.189 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:55 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: 2024-09-10T09:44:55.013+0000 7f28a962b040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T09:44:55.189 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:55 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: 2024-09-10T09:44:55.086+0000 7f28a962b040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T09:44:55.581 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:55 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: 2024-09-10T09:44:55.314+0000 7f28a962b040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T09:44:55.901 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:55 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/1978965995' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-10T09:44:55.916 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout { 2024-09-10T09:44:55.916 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-10T09:44:55.917 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-10T09:44:55.917 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-10T09:44:55.917 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-10T09:44:55.917 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout } 2024-09-10T09:44:55.917 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:54.990+0000 7fa4ff600640 1 Processor -- start 2024-09-10T09:44:55.917 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:54.991+0000 7fa4ff600640 1 -- start start 2024-09-10T09:44:55.917 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:54.991+0000 7fa4ff600640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa4f807de30 0x7fa4f807c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:55.917 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:54.991+0000 7fa4ff600640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f807c860 con 0x7fa4f807de30 2024-09-10T09:44:55.917 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:54.991+0000 7fa4fcc00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa4f807de30 0x7fa4f807c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:55.917 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:54.991+0000 7fa4fcc00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa4f807de30 0x7fa4f807c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57348/0 (socket says 172.21.15.77:57348) 2024-09-10T09:44:55.917 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:54.991+0000 7fa4fcc00640 1 -- 172.21.15.77:0/623283877 learned_addr learned my addr 172.21.15.77:0/623283877 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:44:55.917 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:54.992+0000 7fa4fcc00640 1 -- 172.21.15.77:0/623283877 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4f807c9e0 con 0x7fa4f807de30 2024-09-10T09:44:55.917 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:54.993+0000 7fa4fcc00640 1 --2- 172.21.15.77:0/623283877 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa4f807de30 0x7fa4f807c290 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec009820 tx=0x7fa4ec02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfdd9c767d234376 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:55.918 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:54.993+0000 7fa4f7400640 1 -- 172.21.15.77:0/623283877 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa4ec037070 con 0x7fa4f807de30 2024-09-10T09:44:55.918 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:54.993+0000 7fa4f7400640 1 -- 172.21.15.77:0/623283877 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4ec032920 con 0x7fa4f807de30 2024-09-10T09:44:55.918 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:54.993+0000 7fa4f7400640 1 -- 172.21.15.77:0/623283877 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa4ec030810 con 0x7fa4f807de30 2024-09-10T09:44:55.918 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:54.994+0000 7fa4ff600640 1 -- 172.21.15.77:0/623283877 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa4f807de30 msgr2=0x7fa4f807c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:55.918 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:54.994+0000 7fa4ff600640 1 --2- 172.21.15.77:0/623283877 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa4f807de30 0x7fa4f807c290 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec009820 tx=0x7fa4ec02a270 comp rx=0 tx=0).stop 2024-09-10T09:44:55.918 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:54.994+0000 7fa4ff600640 1 -- 172.21.15.77:0/623283877 shutdown_connections 2024-09-10T09:44:55.918 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:54.994+0000 7fa4ff600640 1 --2- 172.21.15.77:0/623283877 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa4f807de30 0x7fa4f807c290 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:55.918 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:54.994+0000 7fa4ff600640 1 -- 172.21.15.77:0/623283877 >> 172.21.15.77:0/623283877 conn(0x7fa4f8102cf0 msgr2=0x7fa4f8105130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:55.918 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:54.994+0000 7fa4ff600640 1 -- 172.21.15.77:0/623283877 shutdown_connections 2024-09-10T09:44:55.918 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:54.994+0000 7fa4ff600640 1 -- 172.21.15.77:0/623283877 wait complete. 2024-09-10T09:44:55.918 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:54.995+0000 7fa4ff600640 1 Processor -- start 2024-09-10T09:44:55.919 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:54.995+0000 7fa4ff600640 1 -- start start 2024-09-10T09:44:55.919 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:54.995+0000 7fa4ff600640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa4f807de30 0x7fa4f819da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:55.919 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:54.995+0000 7fa4ff600640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f810dbf0 con 0x7fa4f807de30 2024-09-10T09:44:55.919 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:54.996+0000 7fa4fcc00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa4f807de30 0x7fa4f819da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:55.919 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:54.996+0000 7fa4fcc00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa4f807de30 0x7fa4f819da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57350/0 (socket says 172.21.15.77:57350) 2024-09-10T09:44:55.919 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:54.996+0000 7fa4fcc00640 1 -- 172.21.15.77:0/1978965995 learned_addr learned my addr 172.21.15.77:0/1978965995 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:44:55.919 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:54.996+0000 7fa4fcc00640 1 -- 172.21.15.77:0/1978965995 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4f819dfa0 con 0x7fa4f807de30 2024-09-10T09:44:55.919 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:54.996+0000 7fa4fcc00640 1 --2- 172.21.15.77:0/1978965995 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa4f807de30 0x7fa4f819da60 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec030380 tx=0x7fa4ec032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:55.919 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:54.996+0000 7fa4f5600640 1 -- 172.21.15.77:0/1978965995 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa4ec037040 con 0x7fa4f807de30 2024-09-10T09:44:55.919 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:54.997+0000 7fa4f5600640 1 -- 172.21.15.77:0/1978965995 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4ec040070 con 0x7fa4f807de30 2024-09-10T09:44:55.919 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:54.997+0000 7fa4ff600640 1 -- 172.21.15.77:0/1978965995 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4f819e230 con 0x7fa4f807de30 2024-09-10T09:44:55.919 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:54.997+0000 7fa4f5600640 1 -- 172.21.15.77:0/1978965995 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa4ec03a7b0 con 0x7fa4f807de30 2024-09-10T09:44:55.919 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:54.997+0000 7fa4ff600640 1 -- 172.21.15.77:0/1978965995 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4f81a0f20 con 0x7fa4f807de30 2024-09-10T09:44:55.920 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:54.998+0000 7fa4ff600640 1 -- 172.21.15.77:0/1978965995 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4f807c310 con 0x7fa4f807de30 2024-09-10T09:44:55.920 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:54.998+0000 7fa4f5600640 1 -- 172.21.15.77:0/1978965995 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7fa4ec03bc00 con 0x7fa4f807de30 2024-09-10T09:44:55.920 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:54.999+0000 7fa4f5600640 1 --2- 172.21.15.77:0/1978965995 >> [v2:172.21.15.77:6800/1527070442,v1:172.21.15.77:6801/1527070442] conn(0x7fa4e0039020 0x7fa4e003b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:55.920 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:54.999+0000 7fa4f5600640 1 -- 172.21.15.77:0/1978965995 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa4ec06c700 con 0x7fa4f807de30 2024-09-10T09:44:55.920 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:54.999+0000 7fa4f7e00640 1 -- 172.21.15.77:0/1978965995 >> [v2:172.21.15.77:6800/1527070442,v1:172.21.15.77:6801/1527070442] conn(0x7fa4e0039020 msgr2=0x7fa4e003b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.77:6800/1527070442 2024-09-10T09:44:55.920 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:54.999+0000 7fa4f7e00640 1 --2- 172.21.15.77:0/1978965995 >> [v2:172.21.15.77:6800/1527070442,v1:172.21.15.77:6801/1527070442] conn(0x7fa4e0039020 0x7fa4e003b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T09:44:55.920 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:55.004+0000 7fa4f5600640 1 -- 172.21.15.77:0/1978965995 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa4ec06c050 con 0x7fa4f807de30 2024-09-10T09:44:55.920 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:55.200+0000 7fa4f7e00640 1 -- 172.21.15.77:0/1978965995 >> [v2:172.21.15.77:6800/1527070442,v1:172.21.15.77:6801/1527070442] conn(0x7fa4e0039020 msgr2=0x7fa4e003b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.77:6800/1527070442 2024-09-10T09:44:55.920 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:55.200+0000 7fa4f7e00640 1 --2- 172.21.15.77:0/1978965995 >> [v2:172.21.15.77:6800/1527070442,v1:172.21.15.77:6801/1527070442] conn(0x7fa4e0039020 0x7fa4e003b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-10T09:44:55.920 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:55.233+0000 7fa4ff600640 1 -- 172.21.15.77:0/1978965995 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fa4f802d7c0 con 0x7fa4f807de30 2024-09-10T09:44:55.920 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:55.233+0000 7fa4f5600640 1 -- 172.21.15.77:0/1978965995 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7fa4ec03b7a0 con 0x7fa4f807de30 2024-09-10T09:44:55.920 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:55.235+0000 7fa4ff600640 1 -- 172.21.15.77:0/1978965995 >> [v2:172.21.15.77:6800/1527070442,v1:172.21.15.77:6801/1527070442] conn(0x7fa4e0039020 msgr2=0x7fa4e003b4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T09:44:55.920 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:55.235+0000 7fa4ff600640 1 --2- 172.21.15.77:0/1978965995 >> [v2:172.21.15.77:6800/1527070442,v1:172.21.15.77:6801/1527070442] conn(0x7fa4e0039020 0x7fa4e003b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:55.921 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:55.235+0000 7fa4ff600640 1 -- 172.21.15.77:0/1978965995 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa4f807de30 msgr2=0x7fa4f819da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:55.921 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:55.235+0000 7fa4ff600640 1 --2- 172.21.15.77:0/1978965995 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa4f807de30 0x7fa4f819da60 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec030380 tx=0x7fa4ec032670 comp rx=0 tx=0).stop 2024-09-10T09:44:55.921 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:55.235+0000 7fa4ff600640 1 -- 172.21.15.77:0/1978965995 shutdown_connections 2024-09-10T09:44:55.921 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:55.235+0000 7fa4ff600640 1 --2- 172.21.15.77:0/1978965995 >> [v2:172.21.15.77:6800/1527070442,v1:172.21.15.77:6801/1527070442] conn(0x7fa4e0039020 0x7fa4e003b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:55.921 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:55.235+0000 7fa4ff600640 1 --2- 172.21.15.77:0/1978965995 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa4f807de30 0x7fa4f819da60 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:55.921 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:55.235+0000 7fa4ff600640 1 -- 172.21.15.77:0/1978965995 >> 172.21.15.77:0/1978965995 conn(0x7fa4f8102cf0 msgr2=0x7fa4f81039d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:55.921 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:55.236+0000 7fa4ff600640 1 -- 172.21.15.77:0/1978965995 shutdown_connections 2024-09-10T09:44:55.921 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:55.236+0000 7fa4ff600640 1 -- 172.21.15.77:0/1978965995 wait complete. 2024-09-10T09:44:55.921 INFO:teuthology.orchestra.run.smithi077.stdout:Waiting for the mgr to restart... 2024-09-10T09:44:55.921 INFO:teuthology.orchestra.run.smithi077.stdout:Waiting for mgr epoch 9... 2024-09-10T09:44:56.939 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:56 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: 2024-09-10T09:44:56.455+0000 7f28a962b040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T09:44:56.939 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:56 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: 2024-09-10T09:44:56.534+0000 7f28a962b040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T09:44:57.689 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:57 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: 2024-09-10T09:44:57.426+0000 7f28a962b040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T09:44:57.689 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:57 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: 2024-09-10T09:44:57.559+0000 7f28a962b040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T09:44:58.041 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:57 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: 2024-09-10T09:44:57.720+0000 7f28a962b040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T09:44:58.041 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:57 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: 2024-09-10T09:44:57.791+0000 7f28a962b040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T09:44:58.041 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:57 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: 2024-09-10T09:44:57.960+0000 7f28a962b040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T09:44:58.041 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:58 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: 2024-09-10T09:44:58.040+0000 7f28a962b040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T09:44:58.291 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:58 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: 2024-09-10T09:44:58.111+0000 7f28a962b040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T09:44:58.609 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:58 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: 2024-09-10T09:44:58.290+0000 7f28a962b040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T09:44:58.609 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:58 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: 2024-09-10T09:44:58.366+0000 7f28a962b040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T09:44:58.609 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:58 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: 2024-09-10T09:44:58.444+0000 7f28a962b040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T09:44:58.609 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:58 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: 2024-09-10T09:44:58.538+0000 7f28a962b040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T09:44:58.939 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:58 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: 2024-09-10T09:44:58.608+0000 7f28a962b040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T09:44:58.939 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:58 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: 2024-09-10T09:44:58.677+0000 7f28a962b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T09:44:59.689 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:44:59 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: 2024-09-10T09:44:59.573+0000 7f28a962b040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T09:44:59.689 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:59 smithi077 ceph-mon[30319]: Active manager daemon x restarted 2024-09-10T09:44:59.689 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:59 smithi077 ceph-mon[30319]: Activating manager daemon x 2024-09-10T09:44:59.689 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:59 smithi077 ceph-mon[30319]: osdmap e3: 0 total, 0 up, 0 in 2024-09-10T09:44:59.689 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:59 smithi077 ceph-mon[30319]: mgrmap e10: x(active, starting, since 0.00401538s) 2024-09-10T09:44:59.689 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:59 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T09:44:59.690 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:59 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-10T09:44:59.690 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:59 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T09:44:59.690 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:59 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T09:44:59.690 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:59 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T09:44:59.690 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:59 smithi077 ceph-mon[30319]: Manager daemon x is now available 2024-09-10T09:44:59.690 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:44:59 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:45:00.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:00 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-10T09:45:00.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:00 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-10T09:45:00.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:00 smithi077 ceph-mon[30319]: mgrmap e11: x(active, since 1.00754s) 2024-09-10T09:45:00.940 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:45:00 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: 2024-09-10T09:45:00.822+0000 7f2806e00640 -1 log_channel(cephadm) log [ERR] : [10/Sep/2024:09:45:00] ENGINE Error in HTTPServer.serve 2024-09-10T09:45:00.940 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:45:00 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: Traceback (most recent call last): 2024-09-10T09:45:00.940 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:45:00 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T09:45:00.940 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:45:00 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: self._connections.run(self.expiration_interval) 2024-09-10T09:45:00.940 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:45:00 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T09:45:00.940 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:45:00 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: self._run(expiration_interval) 2024-09-10T09:45:00.940 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:45:00 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T09:45:00.941 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:45:00 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T09:45:00.941 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:45:00 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T09:45:00.941 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:45:00 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T09:45:00.941 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:45:00 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T09:45:00.941 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:45:00 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: s = self.context.wrap_socket( 2024-09-10T09:45:00.941 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:45:00 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T09:45:00.941 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:45:00 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: return self.sslsocket_class._create( 2024-09-10T09:45:00.941 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:45:00 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T09:45:00.941 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:45:00 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: self.do_handshake() 2024-09-10T09:45:00.941 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:45:00 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T09:45:00.941 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:45:00 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: self._sslobj.do_handshake() 2024-09-10T09:45:00.941 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:45:00 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T09:45:00.941 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:45:00 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: 2024-09-10T09:45:01.826 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout { 2024-09-10T09:45:01.826 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-10T09:45:01.826 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-10T09:45:01.826 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout } 2024-09-10T09:45:01.826 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:56.391+0000 7f3e0f200640 1 Processor -- start 2024-09-10T09:45:01.826 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:56.391+0000 7f3e0f200640 1 -- start start 2024-09-10T09:45:01.826 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:56.391+0000 7f3e0f200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e081097e0 0x7f3e08109be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:01.826 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:56.391+0000 7f3e0f200640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3e0810a1b0 con 0x7f3e081097e0 2024-09-10T09:45:01.826 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:56.392+0000 7f3e07e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e081097e0 0x7f3e08109be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:01.827 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:56.392+0000 7f3e07e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e081097e0 0x7f3e08109be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57356/0 (socket says 172.21.15.77:57356) 2024-09-10T09:45:01.827 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:56.392+0000 7f3e07e00640 1 -- 172.21.15.77:0/3324877584 learned_addr learned my addr 172.21.15.77:0/3324877584 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:45:01.827 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:56.392+0000 7f3e07e00640 1 -- 172.21.15.77:0/3324877584 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e0810a980 con 0x7f3e081097e0 2024-09-10T09:45:01.827 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:56.393+0000 7f3e07e00640 1 --2- 172.21.15.77:0/3324877584 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e081097e0 0x7f3e08109be0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3df8009820 tx=0x7f3df802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f46895202a9ff9e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:01.827 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:56.393+0000 7f3e06a00640 1 -- 172.21.15.77:0/3324877584 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3df8037070 con 0x7f3e081097e0 2024-09-10T09:45:01.827 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:56.393+0000 7f3e06a00640 1 -- 172.21.15.77:0/3324877584 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3df8032920 con 0x7f3e081097e0 2024-09-10T09:45:01.827 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:56.393+0000 7f3e06a00640 1 -- 172.21.15.77:0/3324877584 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3df8030810 con 0x7f3e081097e0 2024-09-10T09:45:01.827 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:56.394+0000 7f3e0f200640 1 -- 172.21.15.77:0/3324877584 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e081097e0 msgr2=0x7f3e08109be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:01.827 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:56.394+0000 7f3e0f200640 1 --2- 172.21.15.77:0/3324877584 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e081097e0 0x7f3e08109be0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3df8009820 tx=0x7f3df802a270 comp rx=0 tx=0).stop 2024-09-10T09:45:01.827 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:56.394+0000 7f3e0f200640 1 -- 172.21.15.77:0/3324877584 shutdown_connections 2024-09-10T09:45:01.827 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:56.394+0000 7f3e0f200640 1 --2- 172.21.15.77:0/3324877584 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e081097e0 0x7f3e08109be0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:01.827 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:56.394+0000 7f3e0f200640 1 -- 172.21.15.77:0/3324877584 >> 172.21.15.77:0/3324877584 conn(0x7f3e0807cd30 msgr2=0x7f3e0807d160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:01.828 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:56.394+0000 7f3e0f200640 1 -- 172.21.15.77:0/3324877584 shutdown_connections 2024-09-10T09:45:01.828 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:56.394+0000 7f3e0f200640 1 -- 172.21.15.77:0/3324877584 wait complete. 2024-09-10T09:45:01.828 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:56.395+0000 7f3e0f200640 1 Processor -- start 2024-09-10T09:45:01.828 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:56.395+0000 7f3e0f200640 1 -- start start 2024-09-10T09:45:01.828 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:56.396+0000 7f3e0f200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e081097e0 0x7f3e0819dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:01.828 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:56.396+0000 7f3e0f200640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3e0810dd30 con 0x7f3e081097e0 2024-09-10T09:45:01.828 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:56.396+0000 7f3e07e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e081097e0 0x7f3e0819dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:01.828 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:56.396+0000 7f3e07e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e081097e0 0x7f3e0819dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57364/0 (socket says 172.21.15.77:57364) 2024-09-10T09:45:01.828 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:56.396+0000 7f3e07e00640 1 -- 172.21.15.77:0/2351370826 learned_addr learned my addr 172.21.15.77:0/2351370826 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:45:01.828 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:56.396+0000 7f3e07e00640 1 -- 172.21.15.77:0/2351370826 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e0819dff0 con 0x7f3e081097e0 2024-09-10T09:45:01.828 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:56.396+0000 7f3e07e00640 1 --2- 172.21.15.77:0/2351370826 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e081097e0 0x7f3e0819dab0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3df80097f0 tx=0x7f3df80327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:01.829 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:56.397+0000 7f3e04c00640 1 -- 172.21.15.77:0/2351370826 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3df8037040 con 0x7f3e081097e0 2024-09-10T09:45:01.829 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:56.397+0000 7f3e04c00640 1 -- 172.21.15.77:0/2351370826 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3df8030cc0 con 0x7f3e081097e0 2024-09-10T09:45:01.829 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:56.397+0000 7f3e04c00640 1 -- 172.21.15.77:0/2351370826 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3df803a640 con 0x7f3e081097e0 2024-09-10T09:45:01.829 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:56.397+0000 7f3e0f200640 1 -- 172.21.15.77:0/2351370826 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e0819e280 con 0x7f3e081097e0 2024-09-10T09:45:01.829 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:56.397+0000 7f3e0f200640 1 -- 172.21.15.77:0/2351370826 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e081a0f70 con 0x7f3e081097e0 2024-09-10T09:45:01.829 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:56.398+0000 7f3e04c00640 1 -- 172.21.15.77:0/2351370826 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f3df803b3f0 con 0x7f3e081097e0 2024-09-10T09:45:01.829 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:56.398+0000 7f3e04c00640 1 --2- 172.21.15.77:0/2351370826 >> [v2:172.21.15.77:6800/1527070442,v1:172.21.15.77:6801/1527070442] conn(0x7f3dec039020 0x7f3dec03b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:01.829 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:56.398+0000 7f3e04c00640 1 -- 172.21.15.77:0/2351370826 --> [v2:172.21.15.77:6800/1527070442,v1:172.21.15.77:6801/1527070442] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f3dec03bbf0 con 0x7f3dec039020 2024-09-10T09:45:01.829 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:56.398+0000 7f3e04c00640 1 -- 172.21.15.77:0/2351370826 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f3df806c5c0 con 0x7f3e081097e0 2024-09-10T09:45:01.829 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:56.399+0000 7f3e07400640 1 -- 172.21.15.77:0/2351370826 >> [v2:172.21.15.77:6800/1527070442,v1:172.21.15.77:6801/1527070442] conn(0x7f3dec039020 msgr2=0x7f3dec03b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.77:6800/1527070442 2024-09-10T09:45:01.829 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:56.399+0000 7f3e07400640 1 --2- 172.21.15.77:0/2351370826 >> [v2:172.21.15.77:6800/1527070442,v1:172.21.15.77:6801/1527070442] conn(0x7f3dec039020 0x7f3dec03b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T09:45:01.829 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:56.600+0000 7f3e07400640 1 -- 172.21.15.77:0/2351370826 >> [v2:172.21.15.77:6800/1527070442,v1:172.21.15.77:6801/1527070442] conn(0x7f3dec039020 msgr2=0x7f3dec03b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.77:6800/1527070442 2024-09-10T09:45:01.830 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:56.600+0000 7f3e07400640 1 --2- 172.21.15.77:0/2351370826 >> [v2:172.21.15.77:6800/1527070442,v1:172.21.15.77:6801/1527070442] conn(0x7f3dec039020 0x7f3dec03b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-10T09:45:01.830 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:57.001+0000 7f3e07400640 1 -- 172.21.15.77:0/2351370826 >> [v2:172.21.15.77:6800/1527070442,v1:172.21.15.77:6801/1527070442] conn(0x7f3dec039020 msgr2=0x7f3dec03b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.77:6800/1527070442 2024-09-10T09:45:01.830 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:57.001+0000 7f3e07400640 1 --2- 172.21.15.77:0/2351370826 >> [v2:172.21.15.77:6800/1527070442,v1:172.21.15.77:6801/1527070442] conn(0x7f3dec039020 0x7f3dec03b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-10T09:45:01.830 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:57.802+0000 7f3e07400640 1 -- 172.21.15.77:0/2351370826 >> [v2:172.21.15.77:6800/1527070442,v1:172.21.15.77:6801/1527070442] conn(0x7f3dec039020 msgr2=0x7f3dec03b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.77:6800/1527070442 2024-09-10T09:45:01.830 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:57.802+0000 7f3e07400640 1 --2- 172.21.15.77:0/2351370826 >> [v2:172.21.15.77:6800/1527070442,v1:172.21.15.77:6801/1527070442] conn(0x7f3dec039020 0x7f3dec03b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-10T09:45:01.830 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:59.403+0000 7f3e07400640 1 -- 172.21.15.77:0/2351370826 >> [v2:172.21.15.77:6800/1527070442,v1:172.21.15.77:6801/1527070442] conn(0x7f3dec039020 msgr2=0x7f3dec03b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.77:6800/1527070442 2024-09-10T09:45:01.830 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:59.404+0000 7f3e07400640 1 --2- 172.21.15.77:0/2351370826 >> [v2:172.21.15.77:6800/1527070442,v1:172.21.15.77:6801/1527070442] conn(0x7f3dec039020 0x7f3dec03b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-09-10T09:45:01.830 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:59.578+0000 7f3e04c00640 1 -- 172.21.15.77:0/2351370826 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mgrmap(e 10) ==== 46067+0+0 (secure 0 0 0) 0x7f3df803a8f0 con 0x7f3e081097e0 2024-09-10T09:45:01.830 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:59.578+0000 7f3e04c00640 1 -- 172.21.15.77:0/2351370826 >> [v2:172.21.15.77:6800/1527070442,v1:172.21.15.77:6801/1527070442] conn(0x7f3dec039020 msgr2=0x7f3dec03b4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T09:45:01.830 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:44:59.578+0000 7f3e04c00640 1 --2- 172.21.15.77:0/2351370826 >> [v2:172.21.15.77:6800/1527070442,v1:172.21.15.77:6801/1527070442] conn(0x7f3dec039020 0x7f3dec03b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:01.830 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:00.580+0000 7f3e04c00640 1 -- 172.21.15.77:0/2351370826 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7f3df806e8d0 con 0x7f3e081097e0 2024-09-10T09:45:01.831 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:00.581+0000 7f3e04c00640 1 --2- 172.21.15.77:0/2351370826 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f3dec03cae0 0x7f3dec03eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:01.831 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:00.581+0000 7f3e04c00640 1 -- 172.21.15.77:0/2351370826 --> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f3df8047e50 con 0x7f3dec03cae0 2024-09-10T09:45:01.831 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:00.581+0000 7f3e07400640 1 --2- 172.21.15.77:0/2351370826 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f3dec03cae0 0x7f3dec03eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:01.831 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:00.582+0000 7f3e07400640 1 --2- 172.21.15.77:0/2351370826 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f3dec03cae0 0x7f3dec03eed0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f3df4003980 tx=0x7f3df40092b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:01.831 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:00.583+0000 7f3e04c00640 1 -- 172.21.15.77:0/2351370826 <== mgr.14150 v2:172.21.15.77:6800/4261324650 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f3df8047e50 con 0x7f3dec03cae0 2024-09-10T09:45:01.831 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:00.591+0000 7f3e0f200640 1 -- 172.21.15.77:0/2351370826 --> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f3e08109be0 con 0x7f3dec03cae0 2024-09-10T09:45:01.831 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:00.591+0000 7f3e04c00640 1 -- 172.21.15.77:0/2351370826 <== mgr.14150 v2:172.21.15.77:6800/4261324650 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f3e08109be0 con 0x7f3dec03cae0 2024-09-10T09:45:01.831 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:00.592+0000 7f3e0f200640 1 -- 172.21.15.77:0/2351370826 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f3dec03cae0 msgr2=0x7f3dec03eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:01.831 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:00.592+0000 7f3e0f200640 1 --2- 172.21.15.77:0/2351370826 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f3dec03cae0 0x7f3dec03eed0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f3df4003980 tx=0x7f3df40092b0 comp rx=0 tx=0).stop 2024-09-10T09:45:01.831 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:00.592+0000 7f3e0f200640 1 -- 172.21.15.77:0/2351370826 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e081097e0 msgr2=0x7f3e0819dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:01.831 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:00.592+0000 7f3e0f200640 1 --2- 172.21.15.77:0/2351370826 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e081097e0 0x7f3e0819dab0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3df80097f0 tx=0x7f3df80327b0 comp rx=0 tx=0).stop 2024-09-10T09:45:01.832 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:00.592+0000 7f3e0f200640 1 -- 172.21.15.77:0/2351370826 shutdown_connections 2024-09-10T09:45:01.832 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:00.592+0000 7f3e0f200640 1 --2- 172.21.15.77:0/2351370826 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f3dec03cae0 0x7f3dec03eed0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:01.832 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:00.592+0000 7f3e0f200640 1 --2- 172.21.15.77:0/2351370826 >> [v2:172.21.15.77:6800/1527070442,v1:172.21.15.77:6801/1527070442] conn(0x7f3dec039020 0x7f3dec03b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:01.832 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:00.592+0000 7f3e0f200640 1 --2- 172.21.15.77:0/2351370826 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e081097e0 0x7f3e0819dab0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:01.832 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:00.592+0000 7f3e0f200640 1 -- 172.21.15.77:0/2351370826 >> 172.21.15.77:0/2351370826 conn(0x7f3e0807cd30 msgr2=0x7f3e081075a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:01.832 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:00.592+0000 7f3e0f200640 1 -- 172.21.15.77:0/2351370826 shutdown_connections 2024-09-10T09:45:01.832 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:00.592+0000 7f3e0f200640 1 -- 172.21.15.77:0/2351370826 wait complete. 2024-09-10T09:45:01.832 INFO:teuthology.orchestra.run.smithi077.stdout:mgr epoch 9 is available 2024-09-10T09:45:01.832 INFO:teuthology.orchestra.run.smithi077.stdout:Generating a dashboard self-signed certificate... 2024-09-10T09:45:01.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:01 smithi077 ceph-mon[30319]: [10/Sep/2024:09:45:00] ENGINE Bus STARTING 2024-09-10T09:45:01.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:01 smithi077 ceph-mon[30319]: [10/Sep/2024:09:45:00] ENGINE Serving on https://172.21.15.77:7150 2024-09-10T09:45:01.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:01 smithi077 ceph-mon[30319]: [10/Sep/2024:09:45:00] ENGINE Error in HTTPServer.serve 2024-09-10T09:45:01.939 INFO:journalctl@ceph.mon.a.smithi077.stdout: Traceback (most recent call last): 2024-09-10T09:45:01.940 INFO:journalctl@ceph.mon.a.smithi077.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T09:45:01.940 INFO:journalctl@ceph.mon.a.smithi077.stdout: self._connections.run(self.expiration_interval) 2024-09-10T09:45:01.940 INFO:journalctl@ceph.mon.a.smithi077.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T09:45:01.940 INFO:journalctl@ceph.mon.a.smithi077.stdout: self._run(expiration_interval) 2024-09-10T09:45:01.940 INFO:journalctl@ceph.mon.a.smithi077.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T09:45:01.940 INFO:journalctl@ceph.mon.a.smithi077.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T09:45:01.940 INFO:journalctl@ceph.mon.a.smithi077.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T09:45:01.940 INFO:journalctl@ceph.mon.a.smithi077.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T09:45:01.940 INFO:journalctl@ceph.mon.a.smithi077.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T09:45:01.940 INFO:journalctl@ceph.mon.a.smithi077.stdout: s = self.context.wrap_socket( 2024-09-10T09:45:01.940 INFO:journalctl@ceph.mon.a.smithi077.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T09:45:01.941 INFO:journalctl@ceph.mon.a.smithi077.stdout: return self.sslsocket_class._create( 2024-09-10T09:45:01.941 INFO:journalctl@ceph.mon.a.smithi077.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T09:45:01.941 INFO:journalctl@ceph.mon.a.smithi077.stdout: self.do_handshake() 2024-09-10T09:45:01.941 INFO:journalctl@ceph.mon.a.smithi077.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T09:45:01.941 INFO:journalctl@ceph.mon.a.smithi077.stdout: self._sslobj.do_handshake() 2024-09-10T09:45:01.941 INFO:journalctl@ceph.mon.a.smithi077.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T09:45:01.941 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:01 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:02.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:02 smithi077 ceph-mon[30319]: [10/Sep/2024:09:45:00] ENGINE Serving on http://172.21.15.77:8765 2024-09-10T09:45:02.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:02 smithi077 ceph-mon[30319]: [10/Sep/2024:09:45:00] ENGINE Bus STARTED 2024-09-10T09:45:02.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:02 smithi077 ceph-mon[30319]: mgrmap e12: x(active, since 2s) 2024-09-10T09:45:03.379 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-10T09:45:03.380 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:02.358+0000 7f71b9600640 1 Processor -- start 2024-09-10T09:45:03.380 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:02.358+0000 7f71b9600640 1 -- start start 2024-09-10T09:45:03.380 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:02.359+0000 7f71b9600640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f71b4107580 0x7f71b4107980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:03.380 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:02.359+0000 7f71b9600640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f71b4107f50 con 0x7f71b4107580 2024-09-10T09:45:03.380 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:02.359+0000 7f71b2a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f71b4107580 0x7f71b4107980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:03.380 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:02.359+0000 7f71b2a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f71b4107580 0x7f71b4107980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57448/0 (socket says 172.21.15.77:57448) 2024-09-10T09:45:03.380 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:02.359+0000 7f71b2a00640 1 -- 172.21.15.77:0/836192845 learned_addr learned my addr 172.21.15.77:0/836192845 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:45:03.380 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:02.360+0000 7f71b2a00640 1 -- 172.21.15.77:0/836192845 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71b4108780 con 0x7f71b4107580 2024-09-10T09:45:03.380 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:02.361+0000 7f71b2a00640 1 --2- 172.21.15.77:0/836192845 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f71b4107580 0x7f71b4107980 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f71a4009820 tx=0x7f71a402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74592da2c42359b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:03.380 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:02.361+0000 7f71b1600640 1 -- 172.21.15.77:0/836192845 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f71a4037070 con 0x7f71b4107580 2024-09-10T09:45:03.380 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:02.361+0000 7f71b1600640 1 -- 172.21.15.77:0/836192845 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71a4032920 con 0x7f71b4107580 2024-09-10T09:45:03.381 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:02.361+0000 7f71b1600640 1 -- 172.21.15.77:0/836192845 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f71a4030810 con 0x7f71b4107580 2024-09-10T09:45:03.381 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:02.362+0000 7f71b9600640 1 -- 172.21.15.77:0/836192845 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f71b4107580 msgr2=0x7f71b4107980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:03.381 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:02.362+0000 7f71b9600640 1 --2- 172.21.15.77:0/836192845 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f71b4107580 0x7f71b4107980 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f71a4009820 tx=0x7f71a402a270 comp rx=0 tx=0).stop 2024-09-10T09:45:03.381 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:02.362+0000 7f71b9600640 1 -- 172.21.15.77:0/836192845 shutdown_connections 2024-09-10T09:45:03.381 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:02.362+0000 7f71b9600640 1 --2- 172.21.15.77:0/836192845 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f71b4107580 0x7f71b4107980 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:03.381 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:02.362+0000 7f71b9600640 1 -- 172.21.15.77:0/836192845 >> 172.21.15.77:0/836192845 conn(0x7f71b4102d90 msgr2=0x7f71b41051b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:03.381 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:02.362+0000 7f71b9600640 1 -- 172.21.15.77:0/836192845 shutdown_connections 2024-09-10T09:45:03.381 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:02.362+0000 7f71b9600640 1 -- 172.21.15.77:0/836192845 wait complete. 2024-09-10T09:45:03.381 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:02.363+0000 7f71b9600640 1 Processor -- start 2024-09-10T09:45:03.381 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:02.364+0000 7f71b9600640 1 -- start start 2024-09-10T09:45:03.381 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:02.364+0000 7f71b9600640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f71b4107580 0x7f71b419b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:03.381 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:02.364+0000 7f71b9600640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f71b410bb30 con 0x7f71b4107580 2024-09-10T09:45:03.382 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:02.364+0000 7f71b2a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f71b4107580 0x7f71b419b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:03.382 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:02.364+0000 7f71b2a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f71b4107580 0x7f71b419b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57452/0 (socket says 172.21.15.77:57452) 2024-09-10T09:45:03.382 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:02.364+0000 7f71b2a00640 1 -- 172.21.15.77:0/612848605 learned_addr learned my addr 172.21.15.77:0/612848605 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:45:03.382 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:02.364+0000 7f71b2a00640 1 -- 172.21.15.77:0/612848605 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71b419bd90 con 0x7f71b4107580 2024-09-10T09:45:03.382 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:02.365+0000 7f71b2a00640 1 --2- 172.21.15.77:0/612848605 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f71b4107580 0x7f71b419b850 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f71a40097f0 tx=0x7f71a40327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:03.382 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:02.365+0000 7f71ab400640 1 -- 172.21.15.77:0/612848605 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f71a4037040 con 0x7f71b4107580 2024-09-10T09:45:03.382 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:02.365+0000 7f71b9600640 1 -- 172.21.15.77:0/612848605 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71b419c020 con 0x7f71b4107580 2024-09-10T09:45:03.382 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:02.365+0000 7f71ab400640 1 -- 172.21.15.77:0/612848605 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71a4030cc0 con 0x7f71b4107580 2024-09-10T09:45:03.382 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:02.365+0000 7f71ab400640 1 -- 172.21.15.77:0/612848605 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f71a403a640 con 0x7f71b4107580 2024-09-10T09:45:03.382 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:02.365+0000 7f71b9600640 1 -- 172.21.15.77:0/612848605 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71b419ed10 con 0x7f71b4107580 2024-09-10T09:45:03.382 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:02.366+0000 7f71ab400640 1 -- 172.21.15.77:0/612848605 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7f71a403a8f0 con 0x7f71b4107580 2024-09-10T09:45:03.382 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:02.366+0000 7f71b9600640 1 -- 172.21.15.77:0/612848605 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71b410bcb0 con 0x7f71b4107580 2024-09-10T09:45:03.382 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:02.367+0000 7f71ab400640 1 --2- 172.21.15.77:0/612848605 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f7198038f00 0x7f719803b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:03.382 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:02.367+0000 7f71ab400640 1 -- 172.21.15.77:0/612848605 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f71a406c660 con 0x7f71b4107580 2024-09-10T09:45:03.383 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:02.367+0000 7f71b2000640 1 --2- 172.21.15.77:0/612848605 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f7198038f00 0x7f719803b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:03.383 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:02.368+0000 7f71b2000640 1 --2- 172.21.15.77:0/612848605 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f7198038f00 0x7f719803b3c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f71a00098c0 tx=0x7f71a0006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:03.383 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:02.372+0000 7f71ab400640 1 -- 172.21.15.77:0/612848605 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f71a40328c0 con 0x7f71b4107580 2024-09-10T09:45:03.383 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:02.558+0000 7f71ab400640 1 -- 172.21.15.77:0/612848605 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f71a403ad30 con 0x7f71b4107580 2024-09-10T09:45:03.383 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:02.564+0000 7f71b9600640 1 -- 172.21.15.77:0/612848605 --> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f71b4062630 con 0x7f7198038f00 2024-09-10T09:45:03.383 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:02.642+0000 7f71ab400640 1 -- 172.21.15.77:0/612848605 <== mgr.14150 v2:172.21.15.77:6800/4261324650 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f71b4062630 con 0x7f7198038f00 2024-09-10T09:45:03.383 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:02.645+0000 7f71b9600640 1 -- 172.21.15.77:0/612848605 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f7198038f00 msgr2=0x7f719803b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:03.383 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:02.645+0000 7f71b9600640 1 --2- 172.21.15.77:0/612848605 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f7198038f00 0x7f719803b3c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f71a00098c0 tx=0x7f71a0006db0 comp rx=0 tx=0).stop 2024-09-10T09:45:03.383 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:02.645+0000 7f71b9600640 1 -- 172.21.15.77:0/612848605 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f71b4107580 msgr2=0x7f71b419b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:03.383 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:02.645+0000 7f71b9600640 1 --2- 172.21.15.77:0/612848605 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f71b4107580 0x7f71b419b850 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f71a40097f0 tx=0x7f71a40327b0 comp rx=0 tx=0).stop 2024-09-10T09:45:03.383 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:02.645+0000 7f71b9600640 1 -- 172.21.15.77:0/612848605 shutdown_connections 2024-09-10T09:45:03.383 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:02.645+0000 7f71b9600640 1 --2- 172.21.15.77:0/612848605 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f7198038f00 0x7f719803b3c0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:03.383 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:02.645+0000 7f71b9600640 1 --2- 172.21.15.77:0/612848605 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f71b4107580 0x7f71b419b850 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:03.383 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:02.645+0000 7f71b9600640 1 -- 172.21.15.77:0/612848605 >> 172.21.15.77:0/612848605 conn(0x7f71b4102d90 msgr2=0x7f71b4103a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:03.384 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:02.645+0000 7f71b9600640 1 -- 172.21.15.77:0/612848605 shutdown_connections 2024-09-10T09:45:03.384 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:02.645+0000 7f71b9600640 1 -- 172.21.15.77:0/612848605 wait complete. 2024-09-10T09:45:03.384 INFO:teuthology.orchestra.run.smithi077.stdout:Creating initial admin user... 2024-09-10T09:45:03.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:03 smithi077 ceph-mon[30319]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:45:03.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:03 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:03.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:03 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:03.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:03 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:05.689 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:05 smithi077 ceph-mon[30319]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:45:05.689 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:05 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:05.725 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$.A7.iGgt9Kyjr91Tu6njzOErK7SL2ZfAJ0x2wDRsValP3BTV/SviG", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1725961504, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-10T09:45:05.725 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:03.855+0000 7f4bd4600640 1 Processor -- start 2024-09-10T09:45:05.725 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:03.856+0000 7f4bd4600640 1 -- start start 2024-09-10T09:45:05.725 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:03.856+0000 7f4bd4600640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4bcc107f80 0x7f4bcc10a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:05.725 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:03.856+0000 7f4bd4600640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4bcc07be60 con 0x7f4bcc107f80 2024-09-10T09:45:05.725 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:03.856+0000 7f4bd1c00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4bcc107f80 0x7f4bcc10a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:05.726 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:03.856+0000 7f4bd1c00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4bcc107f80 0x7f4bcc10a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42266/0 (socket says 172.21.15.77:42266) 2024-09-10T09:45:05.726 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:03.856+0000 7f4bd1c00640 1 -- 172.21.15.77:0/4081242052 learned_addr learned my addr 172.21.15.77:0/4081242052 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:45:05.726 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:03.857+0000 7f4bd1c00640 1 -- 172.21.15.77:0/4081242052 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bcc10a8b0 con 0x7f4bcc107f80 2024-09-10T09:45:05.726 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:03.858+0000 7f4bd1c00640 1 --2- 172.21.15.77:0/4081242052 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4bcc107f80 0x7f4bcc10a370 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4bc0009820 tx=0x7f4bc002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f8a645faf5feed2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:05.726 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:03.858+0000 7f4bcbe00640 1 -- 172.21.15.77:0/4081242052 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4bc0037070 con 0x7f4bcc107f80 2024-09-10T09:45:05.726 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:03.859+0000 7f4bcbe00640 1 -- 172.21.15.77:0/4081242052 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4bc0032920 con 0x7f4bcc107f80 2024-09-10T09:45:05.726 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:03.859+0000 7f4bcbe00640 1 -- 172.21.15.77:0/4081242052 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4bc0030810 con 0x7f4bcc107f80 2024-09-10T09:45:05.726 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:03.859+0000 7f4bd4600640 1 -- 172.21.15.77:0/4081242052 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4bcc107f80 msgr2=0x7f4bcc10a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:05.726 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:03.859+0000 7f4bd4600640 1 --2- 172.21.15.77:0/4081242052 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4bcc107f80 0x7f4bcc10a370 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4bc0009820 tx=0x7f4bc002a270 comp rx=0 tx=0).stop 2024-09-10T09:45:05.726 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:03.860+0000 7f4bd4600640 1 -- 172.21.15.77:0/4081242052 shutdown_connections 2024-09-10T09:45:05.726 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:03.860+0000 7f4bd4600640 1 --2- 172.21.15.77:0/4081242052 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4bcc107f80 0x7f4bcc10a370 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:05.726 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:03.860+0000 7f4bd4600640 1 -- 172.21.15.77:0/4081242052 >> 172.21.15.77:0/4081242052 conn(0x7f4bcc101b80 msgr2=0x7f4bcc103fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:05.726 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:03.860+0000 7f4bd4600640 1 -- 172.21.15.77:0/4081242052 shutdown_connections 2024-09-10T09:45:05.726 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:03.860+0000 7f4bd4600640 1 -- 172.21.15.77:0/4081242052 wait complete. 2024-09-10T09:45:05.726 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:03.861+0000 7f4bd4600640 1 Processor -- start 2024-09-10T09:45:05.727 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:03.861+0000 7f4bd4600640 1 -- start start 2024-09-10T09:45:05.727 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:03.861+0000 7f4bd4600640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4bcc107f80 0x7f4bcc19da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:05.727 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:03.861+0000 7f4bd4600640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4bcc10b990 con 0x7f4bcc107f80 2024-09-10T09:45:05.727 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:03.861+0000 7f4bd1c00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4bcc107f80 0x7f4bcc19da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:05.727 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:03.862+0000 7f4bd1c00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4bcc107f80 0x7f4bcc19da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42278/0 (socket says 172.21.15.77:42278) 2024-09-10T09:45:05.727 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:03.862+0000 7f4bd1c00640 1 -- 172.21.15.77:0/1267261649 learned_addr learned my addr 172.21.15.77:0/1267261649 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:45:05.727 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:03.862+0000 7f4bd1c00640 1 -- 172.21.15.77:0/1267261649 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bcc19dfc0 con 0x7f4bcc107f80 2024-09-10T09:45:05.727 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:03.862+0000 7f4bd1c00640 1 --2- 172.21.15.77:0/1267261649 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4bcc107f80 0x7f4bcc19da80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4bc00097f0 tx=0x7f4bc00327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:05.727 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:03.863+0000 7f4bca000640 1 -- 172.21.15.77:0/1267261649 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4bc0037040 con 0x7f4bcc107f80 2024-09-10T09:45:05.727 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:03.863+0000 7f4bca000640 1 -- 172.21.15.77:0/1267261649 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4bc0030cc0 con 0x7f4bcc107f80 2024-09-10T09:45:05.727 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:03.863+0000 7f4bd4600640 1 -- 172.21.15.77:0/1267261649 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bcc19e250 con 0x7f4bcc107f80 2024-09-10T09:45:05.727 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:03.863+0000 7f4bca000640 1 -- 172.21.15.77:0/1267261649 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4bc003a6b0 con 0x7f4bcc107f80 2024-09-10T09:45:05.727 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:03.863+0000 7f4bd4600640 1 -- 172.21.15.77:0/1267261649 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bcc1a0ef0 con 0x7f4bcc107f80 2024-09-10T09:45:05.727 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:03.864+0000 7f4bca000640 1 -- 172.21.15.77:0/1267261649 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f4bc003a850 con 0x7f4bcc107f80 2024-09-10T09:45:05.728 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:03.864+0000 7f4bd4600640 1 -- 172.21.15.77:0/1267261649 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bcc1083e0 con 0x7f4bcc107f80 2024-09-10T09:45:05.728 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:03.864+0000 7f4bca000640 1 --2- 172.21.15.77:0/1267261649 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f4bb4039020 0x7f4bb403b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:05.728 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:03.864+0000 7f4bca000640 1 -- 172.21.15.77:0/1267261649 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f4bc006c6a0 con 0x7f4bcc107f80 2024-09-10T09:45:05.728 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:03.865+0000 7f4bd1200640 1 --2- 172.21.15.77:0/1267261649 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f4bb4039020 0x7f4bb403b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:05.728 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:03.865+0000 7f4bd1200640 1 --2- 172.21.15.77:0/1267261649 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f4bb4039020 0x7f4bb403b4e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4bb80098c0 tx=0x7f4bb8006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:05.728 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:03.870+0000 7f4bca000640 1 -- 172.21.15.77:0/1267261649 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4bc0042300 con 0x7f4bcc107f80 2024-09-10T09:45:05.728 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:04.078+0000 7f4bd4600640 1 -- 172.21.15.77:0/1267261649 --> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f4bcc19e730 con 0x7f4bb4039020 2024-09-10T09:45:05.728 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:04.306+0000 7f4bca000640 1 -- 172.21.15.77:0/1267261649 <== mgr.14150 v2:172.21.15.77:6800/4261324650 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f4bcc19e730 con 0x7f4bb4039020 2024-09-10T09:45:05.728 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:04.309+0000 7f4bd4600640 1 -- 172.21.15.77:0/1267261649 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f4bb4039020 msgr2=0x7f4bb403b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:05.728 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:04.309+0000 7f4bd4600640 1 --2- 172.21.15.77:0/1267261649 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f4bb4039020 0x7f4bb403b4e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4bb80098c0 tx=0x7f4bb8006db0 comp rx=0 tx=0).stop 2024-09-10T09:45:05.728 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:04.309+0000 7f4bd4600640 1 -- 172.21.15.77:0/1267261649 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4bcc107f80 msgr2=0x7f4bcc19da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:05.728 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:04.309+0000 7f4bd4600640 1 --2- 172.21.15.77:0/1267261649 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4bcc107f80 0x7f4bcc19da80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4bc00097f0 tx=0x7f4bc00327b0 comp rx=0 tx=0).stop 2024-09-10T09:45:05.728 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:04.309+0000 7f4bd4600640 1 -- 172.21.15.77:0/1267261649 shutdown_connections 2024-09-10T09:45:05.729 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:04.309+0000 7f4bd4600640 1 --2- 172.21.15.77:0/1267261649 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f4bb4039020 0x7f4bb403b4e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:05.729 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:04.309+0000 7f4bd4600640 1 --2- 172.21.15.77:0/1267261649 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4bcc107f80 0x7f4bcc19da80 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:05.729 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:04.309+0000 7f4bd4600640 1 -- 172.21.15.77:0/1267261649 >> 172.21.15.77:0/1267261649 conn(0x7f4bcc101b80 msgr2=0x7f4bcc1028b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:05.729 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:04.309+0000 7f4bd4600640 1 -- 172.21.15.77:0/1267261649 shutdown_connections 2024-09-10T09:45:05.729 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:04.309+0000 7f4bd4600640 1 -- 172.21.15.77:0/1267261649 wait complete. 2024-09-10T09:45:05.729 INFO:teuthology.orchestra.run.smithi077.stdout:Fetching dashboard port number... 2024-09-10T09:45:06.975 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout 8443 2024-09-10T09:45:06.975 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:06.185+0000 7f2fc9800640 1 Processor -- start 2024-09-10T09:45:06.975 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:06.185+0000 7f2fc9800640 1 -- start start 2024-09-10T09:45:06.976 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:06.186+0000 7f2fc9800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2fc4108050 0x7f2fc410a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:06.976 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:06.186+0000 7f2fc9800640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc407bd90 con 0x7f2fc4108050 2024-09-10T09:45:06.976 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:06.186+0000 7f2fc2a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2fc4108050 0x7f2fc410a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:06.976 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:06.186+0000 7f2fc2a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2fc4108050 0x7f2fc410a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42288/0 (socket says 172.21.15.77:42288) 2024-09-10T09:45:06.976 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:06.186+0000 7f2fc2a00640 1 -- 172.21.15.77:0/977935663 learned_addr learned my addr 172.21.15.77:0/977935663 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:45:06.976 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:06.186+0000 7f2fc2a00640 1 -- 172.21.15.77:0/977935663 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fc410a980 con 0x7f2fc4108050 2024-09-10T09:45:06.976 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:06.187+0000 7f2fc2a00640 1 --2- 172.21.15.77:0/977935663 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2fc4108050 0x7f2fc410a440 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2fb4009820 tx=0x7f2fb402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52a64274f8127032 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:06.976 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:06.188+0000 7f2fc2000640 1 -- 172.21.15.77:0/977935663 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2fb4037070 con 0x7f2fc4108050 2024-09-10T09:45:06.976 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:06.188+0000 7f2fc2000640 1 -- 172.21.15.77:0/977935663 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2fb4032920 con 0x7f2fc4108050 2024-09-10T09:45:06.976 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:06.188+0000 7f2fc2000640 1 -- 172.21.15.77:0/977935663 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2fb4030810 con 0x7f2fc4108050 2024-09-10T09:45:06.976 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:06.188+0000 7f2fc9800640 1 -- 172.21.15.77:0/977935663 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2fc4108050 msgr2=0x7f2fc410a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:06.976 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:06.188+0000 7f2fc9800640 1 --2- 172.21.15.77:0/977935663 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2fc4108050 0x7f2fc410a440 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2fb4009820 tx=0x7f2fb402a270 comp rx=0 tx=0).stop 2024-09-10T09:45:06.977 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:06.188+0000 7f2fc9800640 1 -- 172.21.15.77:0/977935663 shutdown_connections 2024-09-10T09:45:06.977 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:06.188+0000 7f2fc9800640 1 --2- 172.21.15.77:0/977935663 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2fc4108050 0x7f2fc410a440 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:06.977 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:06.188+0000 7f2fc9800640 1 -- 172.21.15.77:0/977935663 >> 172.21.15.77:0/977935663 conn(0x7f2fc4101bb0 msgr2=0x7f2fc4104030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:06.977 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:06.188+0000 7f2fc9800640 1 -- 172.21.15.77:0/977935663 shutdown_connections 2024-09-10T09:45:06.977 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:06.189+0000 7f2fc9800640 1 -- 172.21.15.77:0/977935663 wait complete. 2024-09-10T09:45:06.977 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:06.189+0000 7f2fc9800640 1 Processor -- start 2024-09-10T09:45:06.977 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:06.190+0000 7f2fc9800640 1 -- start start 2024-09-10T09:45:06.977 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:06.190+0000 7f2fc9800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2fc4108050 0x7f2fc419b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:06.977 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:06.190+0000 7f2fc9800640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc410b940 con 0x7f2fc4108050 2024-09-10T09:45:06.977 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:06.190+0000 7f2fc2a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2fc4108050 0x7f2fc419b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:06.977 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:06.190+0000 7f2fc2a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2fc4108050 0x7f2fc419b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42302/0 (socket says 172.21.15.77:42302) 2024-09-10T09:45:06.978 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:06.190+0000 7f2fc2a00640 1 -- 172.21.15.77:0/892214800 learned_addr learned my addr 172.21.15.77:0/892214800 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:45:06.978 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:06.190+0000 7f2fc2a00640 1 -- 172.21.15.77:0/892214800 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fc419bd70 con 0x7f2fc4108050 2024-09-10T09:45:06.978 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:06.191+0000 7f2fc2a00640 1 --2- 172.21.15.77:0/892214800 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2fc4108050 0x7f2fc419b830 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2fb40097f0 tx=0x7f2fb40327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:06.978 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:06.191+0000 7f2fbbe00640 1 -- 172.21.15.77:0/892214800 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2fb4037040 con 0x7f2fc4108050 2024-09-10T09:45:06.978 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:06.191+0000 7f2fbbe00640 1 -- 172.21.15.77:0/892214800 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2fb4030cc0 con 0x7f2fc4108050 2024-09-10T09:45:06.978 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:06.191+0000 7f2fbbe00640 1 -- 172.21.15.77:0/892214800 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2fb403a6b0 con 0x7f2fc4108050 2024-09-10T09:45:06.978 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:06.191+0000 7f2fc9800640 1 -- 172.21.15.77:0/892214800 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fc419c000 con 0x7f2fc4108050 2024-09-10T09:45:06.978 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:06.191+0000 7f2fc9800640 1 -- 172.21.15.77:0/892214800 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fc419ecf0 con 0x7f2fc4108050 2024-09-10T09:45:06.978 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:06.192+0000 7f2fc9800640 1 -- 172.21.15.77:0/892214800 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fc41084b0 con 0x7f2fc4108050 2024-09-10T09:45:06.978 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:06.192+0000 7f2fbbe00640 1 -- 172.21.15.77:0/892214800 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f2fb403baf0 con 0x7f2fc4108050 2024-09-10T09:45:06.978 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:06.193+0000 7f2fbbe00640 1 --2- 172.21.15.77:0/892214800 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f2fa8039020 0x7f2fa803b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:06.978 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:06.193+0000 7f2fbbe00640 1 -- 172.21.15.77:0/892214800 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f2fb406c680 con 0x7f2fc4108050 2024-09-10T09:45:06.978 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:06.193+0000 7f2fba000640 1 --2- 172.21.15.77:0/892214800 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f2fa8039020 0x7f2fa803b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:06.979 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:06.194+0000 7f2fba000640 1 --2- 172.21.15.77:0/892214800 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f2fa8039020 0x7f2fa803b4e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2fac0098c0 tx=0x7f2fac006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:06.979 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:06.198+0000 7f2fbbe00640 1 -- 172.21.15.77:0/892214800 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2fb4042e10 con 0x7f2fc4108050 2024-09-10T09:45:06.979 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:06.307+0000 7f2fbbe00640 1 -- 172.21.15.77:0/892214800 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f2fb403a850 con 0x7f2fc4108050 2024-09-10T09:45:06.979 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:06.376+0000 7f2fc9800640 1 -- 172.21.15.77:0/892214800 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f2fc402d7c0 con 0x7f2fc4108050 2024-09-10T09:45:06.979 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:06.376+0000 7f2fbbe00640 1 -- 172.21.15.77:0/892214800 <== mon.0 v2:172.21.15.77:3300/0 8 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f2fb4049200 con 0x7f2fc4108050 2024-09-10T09:45:06.979 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:06.378+0000 7f2fc9800640 1 -- 172.21.15.77:0/892214800 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f2fa8039020 msgr2=0x7f2fa803b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:06.979 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:06.378+0000 7f2fc9800640 1 --2- 172.21.15.77:0/892214800 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f2fa8039020 0x7f2fa803b4e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2fac0098c0 tx=0x7f2fac006db0 comp rx=0 tx=0).stop 2024-09-10T09:45:06.979 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:06.378+0000 7f2fc9800640 1 -- 172.21.15.77:0/892214800 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2fc4108050 msgr2=0x7f2fc419b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:06.979 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:06.378+0000 7f2fc9800640 1 --2- 172.21.15.77:0/892214800 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2fc4108050 0x7f2fc419b830 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2fb40097f0 tx=0x7f2fb40327b0 comp rx=0 tx=0).stop 2024-09-10T09:45:06.979 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:06.378+0000 7f2fc9800640 1 -- 172.21.15.77:0/892214800 shutdown_connections 2024-09-10T09:45:06.979 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:06.378+0000 7f2fc9800640 1 --2- 172.21.15.77:0/892214800 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f2fa8039020 0x7f2fa803b4e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:06.979 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:06.378+0000 7f2fc9800640 1 --2- 172.21.15.77:0/892214800 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2fc4108050 0x7f2fc419b830 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:06.979 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:06.378+0000 7f2fc9800640 1 -- 172.21.15.77:0/892214800 >> 172.21.15.77:0/892214800 conn(0x7f2fc4101bb0 msgr2=0x7f2fc4102980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:06.980 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:06.378+0000 7f2fc9800640 1 -- 172.21.15.77:0/892214800 shutdown_connections 2024-09-10T09:45:06.980 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:06.379+0000 7f2fc9800640 1 -- 172.21.15.77:0/892214800 wait complete. 2024-09-10T09:45:06.990 INFO:teuthology.orchestra.run.smithi077.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T09:45:06.991 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stdout disabled 2024-09-10T09:45:07.004 INFO:teuthology.orchestra.run.smithi077.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T09:45:07.004 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stdout inactive 2024-09-10T09:45:07.004 INFO:teuthology.orchestra.run.smithi077.stdout:firewalld.service is not enabled 2024-09-10T09:45:07.005 INFO:teuthology.orchestra.run.smithi077.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-10T09:45:07.006 INFO:teuthology.orchestra.run.smithi077.stdout:Ceph Dashboard is now available at: 2024-09-10T09:45:07.007 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:45:07.007 INFO:teuthology.orchestra.run.smithi077.stdout: URL: https://smithi077.front.sepia.ceph.com:8443/ 2024-09-10T09:45:07.007 INFO:teuthology.orchestra.run.smithi077.stdout: User: admin 2024-09-10T09:45:07.007 INFO:teuthology.orchestra.run.smithi077.stdout: Password: 9a2m5211ut 2024-09-10T09:45:07.007 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:45:07.007 INFO:teuthology.orchestra.run.smithi077.stdout:Saving cluster configuration to /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/config directory 2024-09-10T09:45:07.508 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:07 smithi077 ceph-mon[30319]: mgrmap e13: x(active, since 6s) 2024-09-10T09:45:07.509 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:07 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/892214800' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-10T09:45:08.357 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:07.507+0000 7f7331600640 1 Processor -- start 2024-09-10T09:45:08.357 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:07.507+0000 7f7331600640 1 -- start start 2024-09-10T09:45:08.357 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:07.508+0000 7f7331600640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f732c109870 0x7f732c109c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:08.357 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:07.508+0000 7f7331600640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f732c10a240 con 0x7f732c109870 2024-09-10T09:45:08.357 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:07.508+0000 7f732aa00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f732c109870 0x7f732c109c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:08.357 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:07.508+0000 7f732aa00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f732c109870 0x7f732c109c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42312/0 (socket says 172.21.15.77:42312) 2024-09-10T09:45:08.357 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:07.508+0000 7f732aa00640 1 -- 172.21.15.77:0/1447390984 learned_addr learned my addr 172.21.15.77:0/1447390984 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:45:08.358 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:07.509+0000 7f732aa00640 1 -- 172.21.15.77:0/1447390984 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f732c10a980 con 0x7f732c109870 2024-09-10T09:45:08.358 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:07.510+0000 7f732aa00640 1 --2- 172.21.15.77:0/1447390984 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f732c109870 0x7f732c109c70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f731c009820 tx=0x7f731c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f744d5f1b37a49e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:08.358 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:07.510+0000 7f7329600640 1 -- 172.21.15.77:0/1447390984 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f731c037070 con 0x7f732c109870 2024-09-10T09:45:08.358 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:07.510+0000 7f7329600640 1 -- 172.21.15.77:0/1447390984 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f731c032920 con 0x7f732c109870 2024-09-10T09:45:08.358 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:07.510+0000 7f7329600640 1 -- 172.21.15.77:0/1447390984 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f731c030810 con 0x7f732c109870 2024-09-10T09:45:08.358 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:07.511+0000 7f7331600640 1 -- 172.21.15.77:0/1447390984 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f732c109870 msgr2=0x7f732c109c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:08.358 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:07.511+0000 7f7331600640 1 --2- 172.21.15.77:0/1447390984 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f732c109870 0x7f732c109c70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f731c009820 tx=0x7f731c02a270 comp rx=0 tx=0).stop 2024-09-10T09:45:08.358 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:07.511+0000 7f7331600640 1 -- 172.21.15.77:0/1447390984 shutdown_connections 2024-09-10T09:45:08.358 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:07.511+0000 7f7331600640 1 --2- 172.21.15.77:0/1447390984 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f732c109870 0x7f732c109c70 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:08.359 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:07.511+0000 7f7331600640 1 -- 172.21.15.77:0/1447390984 >> 172.21.15.77:0/1447390984 conn(0x7f732c07cdc0 msgr2=0x7f732c07d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:08.359 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:07.511+0000 7f7331600640 1 -- 172.21.15.77:0/1447390984 shutdown_connections 2024-09-10T09:45:08.359 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:07.511+0000 7f7331600640 1 -- 172.21.15.77:0/1447390984 wait complete. 2024-09-10T09:45:08.359 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:07.512+0000 7f7331600640 1 Processor -- start 2024-09-10T09:45:08.359 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:07.512+0000 7f7331600640 1 -- start start 2024-09-10T09:45:08.359 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:07.512+0000 7f7331600640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f732c109870 0x7f732c19daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:08.359 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:07.512+0000 7f7331600640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f732c10dd30 con 0x7f732c109870 2024-09-10T09:45:08.359 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:07.512+0000 7f732aa00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f732c109870 0x7f732c19daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:08.359 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:07.513+0000 7f732aa00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f732c109870 0x7f732c19daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42318/0 (socket says 172.21.15.77:42318) 2024-09-10T09:45:08.359 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:07.513+0000 7f732aa00640 1 -- 172.21.15.77:0/1016675354 learned_addr learned my addr 172.21.15.77:0/1016675354 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:45:08.359 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:07.513+0000 7f732aa00640 1 -- 172.21.15.77:0/1016675354 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f732c19dfe0 con 0x7f732c109870 2024-09-10T09:45:08.359 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:07.513+0000 7f732aa00640 1 --2- 172.21.15.77:0/1016675354 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f732c109870 0x7f732c19daa0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f731c0097f0 tx=0x7f731c0327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:08.360 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:07.513+0000 7f7323400640 1 -- 172.21.15.77:0/1016675354 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f731c037040 con 0x7f732c109870 2024-09-10T09:45:08.360 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:07.513+0000 7f7323400640 1 -- 172.21.15.77:0/1016675354 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f731c030cc0 con 0x7f732c109870 2024-09-10T09:45:08.360 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:07.513+0000 7f7331600640 1 -- 172.21.15.77:0/1016675354 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f732c19e270 con 0x7f732c109870 2024-09-10T09:45:08.360 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:07.513+0000 7f7323400640 1 -- 172.21.15.77:0/1016675354 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f731c03a640 con 0x7f732c109870 2024-09-10T09:45:08.360 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:07.514+0000 7f7331600640 1 -- 172.21.15.77:0/1016675354 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f732c1a0f60 con 0x7f732c109870 2024-09-10T09:45:08.360 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:07.515+0000 7f7323400640 1 -- 172.21.15.77:0/1016675354 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f731c03b3f0 con 0x7f732c109870 2024-09-10T09:45:08.360 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:07.515+0000 7f7331600640 1 -- 172.21.15.77:0/1016675354 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f732c10deb0 con 0x7f732c109870 2024-09-10T09:45:08.360 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:07.515+0000 7f7323400640 1 --2- 172.21.15.77:0/1016675354 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f73100390c0 0x7f731003b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:08.360 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:07.515+0000 7f7323400640 1 -- 172.21.15.77:0/1016675354 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f731c06c760 con 0x7f732c109870 2024-09-10T09:45:08.360 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:07.516+0000 7f732a000640 1 --2- 172.21.15.77:0/1016675354 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f73100390c0 0x7f731003b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:08.360 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:07.516+0000 7f732a000640 1 --2- 172.21.15.77:0/1016675354 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f73100390c0 0x7f731003b580 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f73180098c0 tx=0x7f7318006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:08.360 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:07.520+0000 7f7323400640 1 -- 172.21.15.77:0/1016675354 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f731c041710 con 0x7f732c109870 2024-09-10T09:45:08.361 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:07.780+0000 7f7331600640 1 -- 172.21.15.77:0/1016675354 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f732c109cf0 con 0x7f732c109870 2024-09-10T09:45:08.361 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:07.783+0000 7f7323400640 1 -- 172.21.15.77:0/1016675354 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7f731c0328c0 con 0x7f732c109870 2024-09-10T09:45:08.361 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-10T09:45:08.361 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:07.785+0000 7f7331600640 1 -- 172.21.15.77:0/1016675354 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f73100390c0 msgr2=0x7f731003b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:08.361 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:07.785+0000 7f7331600640 1 --2- 172.21.15.77:0/1016675354 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f73100390c0 0x7f731003b580 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f73180098c0 tx=0x7f7318006db0 comp rx=0 tx=0).stop 2024-09-10T09:45:08.361 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:07.785+0000 7f7331600640 1 -- 172.21.15.77:0/1016675354 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f732c109870 msgr2=0x7f732c19daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:08.361 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:07.785+0000 7f7331600640 1 --2- 172.21.15.77:0/1016675354 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f732c109870 0x7f732c19daa0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f731c0097f0 tx=0x7f731c0327b0 comp rx=0 tx=0).stop 2024-09-10T09:45:08.361 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:07.785+0000 7f7331600640 1 -- 172.21.15.77:0/1016675354 shutdown_connections 2024-09-10T09:45:08.361 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:07.785+0000 7f7331600640 1 --2- 172.21.15.77:0/1016675354 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f73100390c0 0x7f731003b580 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:08.361 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:07.785+0000 7f7331600640 1 --2- 172.21.15.77:0/1016675354 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f732c109870 0x7f732c19daa0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:08.361 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:07.785+0000 7f7331600640 1 -- 172.21.15.77:0/1016675354 >> 172.21.15.77:0/1016675354 conn(0x7f732c07cdc0 msgr2=0x7f732c107650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:08.361 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:07.786+0000 7f7331600640 1 -- 172.21.15.77:0/1016675354 shutdown_connections 2024-09-10T09:45:08.362 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-10T09:45:07.786+0000 7f7331600640 1 -- 172.21.15.77:0/1016675354 wait complete. 2024-09-10T09:45:08.362 INFO:teuthology.orchestra.run.smithi077.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-10T09:45:08.362 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:45:08.362 INFO:teuthology.orchestra.run.smithi077.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-10T09:45:08.362 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:45:08.362 INFO:teuthology.orchestra.run.smithi077.stdout:Or, if you are only running a single cluster on this host: 2024-09-10T09:45:08.362 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:45:08.362 INFO:teuthology.orchestra.run.smithi077.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-10T09:45:08.362 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:45:08.362 INFO:teuthology.orchestra.run.smithi077.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-10T09:45:08.362 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:45:08.362 INFO:teuthology.orchestra.run.smithi077.stdout: ceph telemetry on 2024-09-10T09:45:08.363 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:45:08.363 INFO:teuthology.orchestra.run.smithi077.stdout:For more information see: 2024-09-10T09:45:08.363 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:45:08.363 INFO:teuthology.orchestra.run.smithi077.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-10T09:45:08.363 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:45:08.363 INFO:teuthology.orchestra.run.smithi077.stdout:Bootstrap complete. 2024-09-10T09:45:08.412 INFO:tasks.cephadm:Fetching config... 2024-09-10T09:45:08.412 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-09-10T09:45:08.412 DEBUG:teuthology.orchestra.run.smithi077:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-10T09:45:08.448 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-10T09:45:08.449 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-09-10T09:45:08.449 DEBUG:teuthology.orchestra.run.smithi077:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-10T09:45:08.525 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-10T09:45:08.526 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-09-10T09:45:08.526 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd if=/var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-10T09:45:08.621 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-10T09:45:08.622 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-09-10T09:45:08.622 DEBUG:teuthology.orchestra.run.smithi077:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-10T09:45:08.697 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-10T09:45:08.697 DEBUG:teuthology.orchestra.run.smithi077:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC17JjvNomGaHPZ1jKXRB2sCB5Kys7lr8ABNRYFrod4XZYGMm19CmJnRMLzWEHCkFO3CJZPB7JZfzZjZa5Aqh4vigtIl1Fh5jc92/MCRkmc6QpEi6if4zWdCIHXMJlcTq3XrtB0Beeb7J+9JGN+iRQzKbW4hFOtxJjmvgVNfjZfq/vS9V8XFHRFYeRu6BOKSGYOeBYeJjGkAzDO2aOxhq0dNiWT+0IxMeSPlWWZPo6x5WQ/HufAx1vTMaeGuCE88hBio2a+zua8bfYqVEJwzHszQlvmid7yhVPVCpLpWu11X2LDHlIobHCIp4c0oyVdrN5PqxbnWvMWXir6PXAQX9zoIR9wZpLDDmxbwusQ44oSrfAu8nn5qQ7PnATMnVRdKg+Hq0iciiBbtDPmiue4oXQhQm/0FiHXwTmcQeSrR2TKQ0hN5vFCvVWKsUbRdT0oRl/704wDTTqKSk/N0Uf5ukHMOW9jhpS4xoNYpF5nF/qZOmH+o6N3rHQCkk+fyp82odk= ceph-14a55226-6f59-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-10T09:45:08.817 INFO:teuthology.orchestra.run.smithi077.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC17JjvNomGaHPZ1jKXRB2sCB5Kys7lr8ABNRYFrod4XZYGMm19CmJnRMLzWEHCkFO3CJZPB7JZfzZjZa5Aqh4vigtIl1Fh5jc92/MCRkmc6QpEi6if4zWdCIHXMJlcTq3XrtB0Beeb7J+9JGN+iRQzKbW4hFOtxJjmvgVNfjZfq/vS9V8XFHRFYeRu6BOKSGYOeBYeJjGkAzDO2aOxhq0dNiWT+0IxMeSPlWWZPo6x5WQ/HufAx1vTMaeGuCE88hBio2a+zua8bfYqVEJwzHszQlvmid7yhVPVCpLpWu11X2LDHlIobHCIp4c0oyVdrN5PqxbnWvMWXir6PXAQX9zoIR9wZpLDDmxbwusQ44oSrfAu8nn5qQ7PnATMnVRdKg+Hq0iciiBbtDPmiue4oXQhQm/0FiHXwTmcQeSrR2TKQ0hN5vFCvVWKsUbRdT0oRl/704wDTTqKSk/N0Uf5ukHMOW9jhpS4xoNYpF5nF/qZOmH+o6N3rHQCkk+fyp82odk= ceph-14a55226-6f59-11ef-bcea-c7b262605968 2024-09-10T09:45:08.842 DEBUG:teuthology.orchestra.run.smithi120:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC17JjvNomGaHPZ1jKXRB2sCB5Kys7lr8ABNRYFrod4XZYGMm19CmJnRMLzWEHCkFO3CJZPB7JZfzZjZa5Aqh4vigtIl1Fh5jc92/MCRkmc6QpEi6if4zWdCIHXMJlcTq3XrtB0Beeb7J+9JGN+iRQzKbW4hFOtxJjmvgVNfjZfq/vS9V8XFHRFYeRu6BOKSGYOeBYeJjGkAzDO2aOxhq0dNiWT+0IxMeSPlWWZPo6x5WQ/HufAx1vTMaeGuCE88hBio2a+zua8bfYqVEJwzHszQlvmid7yhVPVCpLpWu11X2LDHlIobHCIp4c0oyVdrN5PqxbnWvMWXir6PXAQX9zoIR9wZpLDDmxbwusQ44oSrfAu8nn5qQ7PnATMnVRdKg+Hq0iciiBbtDPmiue4oXQhQm/0FiHXwTmcQeSrR2TKQ0hN5vFCvVWKsUbRdT0oRl/704wDTTqKSk/N0Uf5ukHMOW9jhpS4xoNYpF5nF/qZOmH+o6N3rHQCkk+fyp82odk= ceph-14a55226-6f59-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-10T09:45:08.883 INFO:teuthology.orchestra.run.smithi120.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC17JjvNomGaHPZ1jKXRB2sCB5Kys7lr8ABNRYFrod4XZYGMm19CmJnRMLzWEHCkFO3CJZPB7JZfzZjZa5Aqh4vigtIl1Fh5jc92/MCRkmc6QpEi6if4zWdCIHXMJlcTq3XrtB0Beeb7J+9JGN+iRQzKbW4hFOtxJjmvgVNfjZfq/vS9V8XFHRFYeRu6BOKSGYOeBYeJjGkAzDO2aOxhq0dNiWT+0IxMeSPlWWZPo6x5WQ/HufAx1vTMaeGuCE88hBio2a+zua8bfYqVEJwzHszQlvmid7yhVPVCpLpWu11X2LDHlIobHCIp4c0oyVdrN5PqxbnWvMWXir6PXAQX9zoIR9wZpLDDmxbwusQ44oSrfAu8nn5qQ7PnATMnVRdKg+Hq0iciiBbtDPmiue4oXQhQm/0FiHXwTmcQeSrR2TKQ0hN5vFCvVWKsUbRdT0oRl/704wDTTqKSk/N0Uf5ukHMOW9jhpS4xoNYpF5nF/qZOmH+o6N3rHQCkk+fyp82odk= ceph-14a55226-6f59-11ef-bcea-c7b262605968 2024-09-10T09:45:08.891 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:08 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/1016675354' entity='client.admin' 2024-09-10T09:45:08.896 DEBUG:teuthology.orchestra.run.smithi134:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC17JjvNomGaHPZ1jKXRB2sCB5Kys7lr8ABNRYFrod4XZYGMm19CmJnRMLzWEHCkFO3CJZPB7JZfzZjZa5Aqh4vigtIl1Fh5jc92/MCRkmc6QpEi6if4zWdCIHXMJlcTq3XrtB0Beeb7J+9JGN+iRQzKbW4hFOtxJjmvgVNfjZfq/vS9V8XFHRFYeRu6BOKSGYOeBYeJjGkAzDO2aOxhq0dNiWT+0IxMeSPlWWZPo6x5WQ/HufAx1vTMaeGuCE88hBio2a+zua8bfYqVEJwzHszQlvmid7yhVPVCpLpWu11X2LDHlIobHCIp4c0oyVdrN5PqxbnWvMWXir6PXAQX9zoIR9wZpLDDmxbwusQ44oSrfAu8nn5qQ7PnATMnVRdKg+Hq0iciiBbtDPmiue4oXQhQm/0FiHXwTmcQeSrR2TKQ0hN5vFCvVWKsUbRdT0oRl/704wDTTqKSk/N0Uf5ukHMOW9jhpS4xoNYpF5nF/qZOmH+o6N3rHQCkk+fyp82odk= ceph-14a55226-6f59-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-10T09:45:08.939 INFO:teuthology.orchestra.run.smithi134.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC17JjvNomGaHPZ1jKXRB2sCB5Kys7lr8ABNRYFrod4XZYGMm19CmJnRMLzWEHCkFO3CJZPB7JZfzZjZa5Aqh4vigtIl1Fh5jc92/MCRkmc6QpEi6if4zWdCIHXMJlcTq3XrtB0Beeb7J+9JGN+iRQzKbW4hFOtxJjmvgVNfjZfq/vS9V8XFHRFYeRu6BOKSGYOeBYeJjGkAzDO2aOxhq0dNiWT+0IxMeSPlWWZPo6x5WQ/HufAx1vTMaeGuCE88hBio2a+zua8bfYqVEJwzHszQlvmid7yhVPVCpLpWu11X2LDHlIobHCIp4c0oyVdrN5PqxbnWvMWXir6PXAQX9zoIR9wZpLDDmxbwusQ44oSrfAu8nn5qQ7PnATMnVRdKg+Hq0iciiBbtDPmiue4oXQhQm/0FiHXwTmcQeSrR2TKQ0hN5vFCvVWKsUbRdT0oRl/704wDTTqKSk/N0Uf5ukHMOW9jhpS4xoNYpF5nF/qZOmH+o6N3rHQCkk+fyp82odk= ceph-14a55226-6f59-11ef-bcea-c7b262605968 2024-09-10T09:45:08.951 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-10T09:45:09.314 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:45:09.878 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:09.877+0000 7fdcaa200640 1 -- 172.21.15.77:0/335948144 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdca41037f0 msgr2=0x7fdca4103bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:09.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:09.877+0000 7fdcaa200640 1 --2- 172.21.15.77:0/335948144 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdca41037f0 0x7fdca4103bf0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fdc94009880 tx=0x7fdc9402a5d0 comp rx=0 tx=0).stop 2024-09-10T09:45:09.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:09.877+0000 7fdcaa200640 1 -- 172.21.15.77:0/335948144 shutdown_connections 2024-09-10T09:45:09.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:09.877+0000 7fdcaa200640 1 --2- 172.21.15.77:0/335948144 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdca41037f0 0x7fdca4103bf0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:09.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:09.877+0000 7fdcaa200640 1 -- 172.21.15.77:0/335948144 >> 172.21.15.77:0/335948144 conn(0x7fdca40fefc0 msgr2=0x7fdca4101400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:09.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:09.877+0000 7fdcaa200640 1 -- 172.21.15.77:0/335948144 shutdown_connections 2024-09-10T09:45:09.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:09.877+0000 7fdcaa200640 1 -- 172.21.15.77:0/335948144 wait complete. 2024-09-10T09:45:09.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:09.879+0000 7fdcaa200640 1 Processor -- start 2024-09-10T09:45:09.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:09.879+0000 7fdcaa200640 1 -- start start 2024-09-10T09:45:09.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:09.879+0000 7fdcaa200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdca41037f0 0x7fdca4198f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:09.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:09.879+0000 7fdcaa200640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fdca4109900 con 0x7fdca41037f0 2024-09-10T09:45:09.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:09.879+0000 7fdca3400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdca41037f0 0x7fdca4198f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:09.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:09.880+0000 7fdca3400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdca41037f0 0x7fdca4198f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42328/0 (socket says 172.21.15.77:42328) 2024-09-10T09:45:09.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:09.880+0000 7fdca3400640 1 -- 172.21.15.77:0/1691133755 learned_addr learned my addr 172.21.15.77:0/1691133755 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:45:09.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:09.880+0000 7fdca3400640 1 -- 172.21.15.77:0/1691133755 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdca4199470 con 0x7fdca41037f0 2024-09-10T09:45:09.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:09.880+0000 7fdca3400640 1 --2- 172.21.15.77:0/1691133755 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdca41037f0 0x7fdca4198f30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fdc94004f60 tx=0x7fdc940048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:09.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:09.881+0000 7fdc9be00640 1 -- 172.21.15.77:0/1691133755 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc9403f070 con 0x7fdca41037f0 2024-09-10T09:45:09.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:09.881+0000 7fdc9be00640 1 -- 172.21.15.77:0/1691133755 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc94004cc0 con 0x7fdca41037f0 2024-09-10T09:45:09.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:09.881+0000 7fdc9be00640 1 -- 172.21.15.77:0/1691133755 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc94035030 con 0x7fdca41037f0 2024-09-10T09:45:09.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:09.881+0000 7fdcaa200640 1 -- 172.21.15.77:0/1691133755 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdca4199700 con 0x7fdca41037f0 2024-09-10T09:45:09.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:09.881+0000 7fdcaa200640 1 -- 172.21.15.77:0/1691133755 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdca419c3f0 con 0x7fdca41037f0 2024-09-10T09:45:09.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:09.882+0000 7fdc9be00640 1 -- 172.21.15.77:0/1691133755 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fdc94042070 con 0x7fdca41037f0 2024-09-10T09:45:09.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:09.882+0000 7fdc9be00640 1 --2- 172.21.15.77:0/1691133755 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fdc880390c0 0x7fdc8803b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:09.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:09.882+0000 7fdc9be00640 1 -- 172.21.15.77:0/1691133755 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fdc9406cdd0 con 0x7fdca41037f0 2024-09-10T09:45:09.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:09.882+0000 7fdcaa200640 1 -- 172.21.15.77:0/1691133755 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdca410a080 con 0x7fdca41037f0 2024-09-10T09:45:09.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:09.883+0000 7fdca2a00640 1 --2- 172.21.15.77:0/1691133755 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fdc880390c0 0x7fdc8803b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:09.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:09.883+0000 7fdca2a00640 1 --2- 172.21.15.77:0/1691133755 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fdc880390c0 0x7fdc8803b580 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fdc900098c0 tx=0x7fdc90006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:09.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:09.888+0000 7fdc9be00640 1 -- 172.21.15.77:0/1691133755 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdc94040260 con 0x7fdca41037f0 2024-09-10T09:45:10.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:10.062+0000 7fdcaa200640 1 -- 172.21.15.77:0/1691133755 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7fdca402d7c0 con 0x7fdca41037f0 2024-09-10T09:45:10.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:10.065+0000 7fdc9be00640 1 -- 172.21.15.77:0/1691133755 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7fdc94003dd0 con 0x7fdca41037f0 2024-09-10T09:45:10.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:10.068+0000 7fdcaa200640 1 -- 172.21.15.77:0/1691133755 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fdc880390c0 msgr2=0x7fdc8803b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:10.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:10.068+0000 7fdcaa200640 1 --2- 172.21.15.77:0/1691133755 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fdc880390c0 0x7fdc8803b580 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fdc900098c0 tx=0x7fdc90006db0 comp rx=0 tx=0).stop 2024-09-10T09:45:10.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:10.068+0000 7fdcaa200640 1 -- 172.21.15.77:0/1691133755 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdca41037f0 msgr2=0x7fdca4198f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:10.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:10.068+0000 7fdcaa200640 1 --2- 172.21.15.77:0/1691133755 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdca41037f0 0x7fdca4198f30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fdc94004f60 tx=0x7fdc940048f0 comp rx=0 tx=0).stop 2024-09-10T09:45:10.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:10.069+0000 7fdcaa200640 1 -- 172.21.15.77:0/1691133755 shutdown_connections 2024-09-10T09:45:10.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:10.069+0000 7fdcaa200640 1 --2- 172.21.15.77:0/1691133755 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fdc880390c0 0x7fdc8803b580 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:10.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:10.069+0000 7fdcaa200640 1 --2- 172.21.15.77:0/1691133755 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdca41037f0 0x7fdca4198f30 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:10.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:10.069+0000 7fdcaa200640 1 -- 172.21.15.77:0/1691133755 >> 172.21.15.77:0/1691133755 conn(0x7fdca40fefc0 msgr2=0x7fdca40ffa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:10.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:10.069+0000 7fdcaa200640 1 -- 172.21.15.77:0/1691133755 shutdown_connections 2024-09-10T09:45:10.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:10.069+0000 7fdcaa200640 1 -- 172.21.15.77:0/1691133755 wait complete. 2024-09-10T09:45:11.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:11 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/1691133755' entity='client.admin' 2024-09-10T09:45:11.331 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-10T09:45:11.331 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-10T09:45:11.936 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:45:12.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:12.469+0000 7f6d49c00640 1 -- 172.21.15.77:0/2844008661 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6d44076220 msgr2=0x7f6d44074680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:12.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:12.469+0000 7f6d49c00640 1 --2- 172.21.15.77:0/2844008661 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6d44076220 0x7f6d44074680 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6d34009880 tx=0x7f6d3402a5d0 comp rx=0 tx=0).stop 2024-09-10T09:45:12.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:12.469+0000 7f6d49c00640 1 -- 172.21.15.77:0/2844008661 shutdown_connections 2024-09-10T09:45:12.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:12.469+0000 7f6d49c00640 1 --2- 172.21.15.77:0/2844008661 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6d44076220 0x7f6d44074680 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:12.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:12.469+0000 7f6d49c00640 1 -- 172.21.15.77:0/2844008661 >> 172.21.15.77:0/2844008661 conn(0x7f6d440fcd80 msgr2=0x7f6d440ff1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:12.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:12.469+0000 7f6d49c00640 1 -- 172.21.15.77:0/2844008661 shutdown_connections 2024-09-10T09:45:12.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:12.469+0000 7f6d49c00640 1 -- 172.21.15.77:0/2844008661 wait complete. 2024-09-10T09:45:12.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:12.470+0000 7f6d49c00640 1 Processor -- start 2024-09-10T09:45:12.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:12.470+0000 7f6d49c00640 1 -- start start 2024-09-10T09:45:12.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:12.470+0000 7f6d49c00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6d44076220 0x7f6d4419d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:12.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:12.470+0000 7f6d49c00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6d4410dd50 con 0x7f6d44076220 2024-09-10T09:45:12.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:12.471+0000 7f6d42a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6d44076220 0x7f6d4419d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:12.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:12.471+0000 7f6d42a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6d44076220 0x7f6d4419d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42348/0 (socket says 172.21.15.77:42348) 2024-09-10T09:45:12.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:12.471+0000 7f6d42a00640 1 -- 172.21.15.77:0/3074591039 learned_addr learned my addr 172.21.15.77:0/3074591039 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:45:12.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:12.471+0000 7f6d42a00640 1 -- 172.21.15.77:0/3074591039 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d4419d8d0 con 0x7f6d44076220 2024-09-10T09:45:12.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:12.471+0000 7f6d42a00640 1 --2- 172.21.15.77:0/3074591039 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6d44076220 0x7f6d4419d390 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6d34004f60 tx=0x7f6d340048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:12.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:12.472+0000 7f6d3b400640 1 -- 172.21.15.77:0/3074591039 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6d3403f070 con 0x7f6d44076220 2024-09-10T09:45:12.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:12.472+0000 7f6d3b400640 1 -- 172.21.15.77:0/3074591039 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d34004cc0 con 0x7f6d44076220 2024-09-10T09:45:12.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:12.472+0000 7f6d3b400640 1 -- 172.21.15.77:0/3074591039 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6d34035030 con 0x7f6d44076220 2024-09-10T09:45:12.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:12.472+0000 7f6d49c00640 1 -- 172.21.15.77:0/3074591039 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d4419db60 con 0x7f6d44076220 2024-09-10T09:45:12.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:12.472+0000 7f6d49c00640 1 -- 172.21.15.77:0/3074591039 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d441a0850 con 0x7f6d44076220 2024-09-10T09:45:12.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:12.473+0000 7f6d3b400640 1 -- 172.21.15.77:0/3074591039 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f6d34042070 con 0x7f6d44076220 2024-09-10T09:45:12.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:12.473+0000 7f6d3b400640 1 --2- 172.21.15.77:0/3074591039 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f6d280390c0 0x7f6d2803b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:12.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:12.473+0000 7f6d49c00640 1 -- 172.21.15.77:0/3074591039 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d4410e490 con 0x7f6d44076220 2024-09-10T09:45:12.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:12.474+0000 7f6d3b400640 1 -- 172.21.15.77:0/3074591039 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f6d3406ce10 con 0x7f6d44076220 2024-09-10T09:45:12.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:12.474+0000 7f6d3be00640 1 --2- 172.21.15.77:0/3074591039 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f6d280390c0 0x7f6d2803b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:12.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:12.475+0000 7f6d3be00640 1 --2- 172.21.15.77:0/3074591039 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f6d280390c0 0x7f6d2803b580 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c0098c0 tx=0x7f6d2c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:12.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:12.479+0000 7f6d3b400640 1 -- 172.21.15.77:0/3074591039 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6d34031bf0 con 0x7f6d44076220 2024-09-10T09:45:12.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:12.724+0000 7f6d49c00640 1 -- 172.21.15.77:0/3074591039 --> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f6d4402d830 con 0x7f6d280390c0 2024-09-10T09:45:12.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:12.729+0000 7f6d3b400640 1 -- 172.21.15.77:0/3074591039 <== mgr.14150 v2:172.21.15.77:6800/4261324650 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6d4402d830 con 0x7f6d280390c0 2024-09-10T09:45:12.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:12.731+0000 7f6d49c00640 1 -- 172.21.15.77:0/3074591039 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f6d280390c0 msgr2=0x7f6d2803b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:12.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:12.731+0000 7f6d49c00640 1 --2- 172.21.15.77:0/3074591039 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f6d280390c0 0x7f6d2803b580 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c0098c0 tx=0x7f6d2c006db0 comp rx=0 tx=0).stop 2024-09-10T09:45:12.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:12.731+0000 7f6d49c00640 1 -- 172.21.15.77:0/3074591039 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6d44076220 msgr2=0x7f6d4419d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:12.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:12.731+0000 7f6d49c00640 1 --2- 172.21.15.77:0/3074591039 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6d44076220 0x7f6d4419d390 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6d34004f60 tx=0x7f6d340048f0 comp rx=0 tx=0).stop 2024-09-10T09:45:12.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:12.731+0000 7f6d49c00640 1 -- 172.21.15.77:0/3074591039 shutdown_connections 2024-09-10T09:45:12.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:12.731+0000 7f6d49c00640 1 --2- 172.21.15.77:0/3074591039 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f6d280390c0 0x7f6d2803b580 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:12.734 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:12.731+0000 7f6d49c00640 1 --2- 172.21.15.77:0/3074591039 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6d44076220 0x7f6d4419d390 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:12.734 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:12.731+0000 7f6d49c00640 1 -- 172.21.15.77:0/3074591039 >> 172.21.15.77:0/3074591039 conn(0x7f6d440fcd80 msgr2=0x7f6d440fd890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:12.734 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:12.731+0000 7f6d49c00640 1 -- 172.21.15.77:0/3074591039 shutdown_connections 2024-09-10T09:45:12.734 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:12.731+0000 7f6d49c00640 1 -- 172.21.15.77:0/3074591039 wait complete. 2024-09-10T09:45:13.706 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi120 2024-09-10T09:45:13.707 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-10T09:45:13.707 DEBUG:teuthology.orchestra.run.smithi120:> dd of=/etc/ceph/ceph.conf 2024-09-10T09:45:13.728 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-10T09:45:13.728 DEBUG:teuthology.orchestra.run.smithi120:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T09:45:13.787 INFO:tasks.cephadm:Adding host smithi120 to orchestrator... 2024-09-10T09:45:13.787 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph orch host add smithi120 2024-09-10T09:45:13.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:13 smithi077 ceph-mon[30319]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:45:13.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:13 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:14.187 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:45:14.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:14.793+0000 7f23cf400640 1 -- 172.21.15.77:0/2941907522 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f23c81037f0 msgr2=0x7f23c8103bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:14.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:14.793+0000 7f23cf400640 1 --2- 172.21.15.77:0/2941907522 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f23c81037f0 0x7f23c8103bf0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f23bc009880 tx=0x7f23bc02a5d0 comp rx=0 tx=0).stop 2024-09-10T09:45:14.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:14.793+0000 7f23cf400640 1 -- 172.21.15.77:0/2941907522 shutdown_connections 2024-09-10T09:45:14.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:14.793+0000 7f23cf400640 1 --2- 172.21.15.77:0/2941907522 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f23c81037f0 0x7f23c8103bf0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:14.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:14.793+0000 7f23cf400640 1 -- 172.21.15.77:0/2941907522 >> 172.21.15.77:0/2941907522 conn(0x7f23c80fefc0 msgr2=0x7f23c8101400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:14.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:14.793+0000 7f23cf400640 1 -- 172.21.15.77:0/2941907522 shutdown_connections 2024-09-10T09:45:14.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:14.793+0000 7f23cf400640 1 -- 172.21.15.77:0/2941907522 wait complete. 2024-09-10T09:45:14.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:14.794+0000 7f23cf400640 1 Processor -- start 2024-09-10T09:45:14.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:14.794+0000 7f23cf400640 1 -- start start 2024-09-10T09:45:14.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:14.795+0000 7f23cf400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f23c81037f0 0x7f23c8198f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:14.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:14.795+0000 7f23cf400640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f23c8109900 con 0x7f23c81037f0 2024-09-10T09:45:14.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:14.795+0000 7f23cca00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f23c81037f0 0x7f23c8198f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:14.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:14.795+0000 7f23cca00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f23c81037f0 0x7f23c8198f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:48512/0 (socket says 172.21.15.77:48512) 2024-09-10T09:45:14.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:14.795+0000 7f23cca00640 1 -- 172.21.15.77:0/3735686421 learned_addr learned my addr 172.21.15.77:0/3735686421 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:45:14.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:14.795+0000 7f23cca00640 1 -- 172.21.15.77:0/3735686421 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23c8199470 con 0x7f23c81037f0 2024-09-10T09:45:14.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:14.796+0000 7f23cca00640 1 --2- 172.21.15.77:0/3735686421 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f23c81037f0 0x7f23c8198f30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f23bc004f60 tx=0x7f23bc0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:14.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:14.796+0000 7f23c5600640 1 -- 172.21.15.77:0/3735686421 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f23bc03f070 con 0x7f23c81037f0 2024-09-10T09:45:14.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:14.796+0000 7f23c5600640 1 -- 172.21.15.77:0/3735686421 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23bc004cc0 con 0x7f23c81037f0 2024-09-10T09:45:14.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:14.796+0000 7f23c5600640 1 -- 172.21.15.77:0/3735686421 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f23bc035030 con 0x7f23c81037f0 2024-09-10T09:45:14.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:14.796+0000 7f23cf400640 1 -- 172.21.15.77:0/3735686421 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23c8199700 con 0x7f23c81037f0 2024-09-10T09:45:14.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:14.796+0000 7f23cf400640 1 -- 172.21.15.77:0/3735686421 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23c819c3f0 con 0x7f23c81037f0 2024-09-10T09:45:14.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:14.797+0000 7f23c5600640 1 -- 172.21.15.77:0/3735686421 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f23bc042070 con 0x7f23c81037f0 2024-09-10T09:45:14.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:14.797+0000 7f23cf400640 1 -- 172.21.15.77:0/3735686421 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23c810a080 con 0x7f23c81037f0 2024-09-10T09:45:14.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:14.798+0000 7f23c5600640 1 --2- 172.21.15.77:0/3735686421 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f23b00390c0 0x7f23b003b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:14.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:14.798+0000 7f23c5600640 1 -- 172.21.15.77:0/3735686421 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f23bc06cdd0 con 0x7f23c81037f0 2024-09-10T09:45:14.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:14.798+0000 7f23c7e00640 1 --2- 172.21.15.77:0/3735686421 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f23b00390c0 0x7f23b003b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:14.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:14.799+0000 7f23c7e00640 1 --2- 172.21.15.77:0/3735686421 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f23b00390c0 0x7f23b003b580 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f23b40098c0 tx=0x7f23b4006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:14.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:14.803+0000 7f23c5600640 1 -- 172.21.15.77:0/3735686421 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f23bc040260 con 0x7f23c81037f0 2024-09-10T09:45:15.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:15.019+0000 7f23cf400640 1 -- 172.21.15.77:0/3735686421 --> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi120", "target": ["mon-mgr", ""]}) -- 0x7f23c802d830 con 0x7f23b00390c0 2024-09-10T09:45:15.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:15 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:15.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:15 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:15.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:15 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi077", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:45:15.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:15 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:45:15.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:15 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:45:15.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:15 smithi077 ceph-mon[30319]: Updating smithi077:/etc/ceph/ceph.conf 2024-09-10T09:45:15.440 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:15 smithi077 ceph-mon[30319]: Updating smithi077:/var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T09:45:16.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:16 smithi077 ceph-mon[30319]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi120", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:45:16.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:16 smithi077 ceph-mon[30319]: Updating smithi077:/etc/ceph/ceph.client.admin.keyring 2024-09-10T09:45:16.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:16 smithi077 ceph-mon[30319]: Updating smithi077:/var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T09:45:16.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:16 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:16.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:16 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:16.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:16 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:17.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:17 smithi077 ceph-mon[30319]: Deploying cephadm binary to smithi120 2024-09-10T09:45:17.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:17 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:45:17.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:17 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:45:17.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:17 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:45:17.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:17 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:18.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:18.237+0000 7f23c5600640 1 -- 172.21.15.77:0/3735686421 <== mgr.14150 v2:172.21.15.77:6800/4261324650 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f23c802d830 con 0x7f23b00390c0 2024-09-10T09:45:18.239 INFO:teuthology.orchestra.run.smithi077.stdout:Added host 'smithi120' with addr '172.21.15.120' 2024-09-10T09:45:18.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:18.240+0000 7f23cf400640 1 -- 172.21.15.77:0/3735686421 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f23b00390c0 msgr2=0x7f23b003b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:18.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:18.240+0000 7f23cf400640 1 --2- 172.21.15.77:0/3735686421 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f23b00390c0 0x7f23b003b580 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f23b40098c0 tx=0x7f23b4006db0 comp rx=0 tx=0).stop 2024-09-10T09:45:18.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:18.240+0000 7f23cf400640 1 -- 172.21.15.77:0/3735686421 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f23c81037f0 msgr2=0x7f23c8198f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:18.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:18.240+0000 7f23cf400640 1 --2- 172.21.15.77:0/3735686421 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f23c81037f0 0x7f23c8198f30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f23bc004f60 tx=0x7f23bc0048f0 comp rx=0 tx=0).stop 2024-09-10T09:45:18.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:18.240+0000 7f23cf400640 1 -- 172.21.15.77:0/3735686421 shutdown_connections 2024-09-10T09:45:18.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:18.240+0000 7f23cf400640 1 --2- 172.21.15.77:0/3735686421 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f23b00390c0 0x7f23b003b580 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:18.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:18.240+0000 7f23cf400640 1 --2- 172.21.15.77:0/3735686421 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f23c81037f0 0x7f23c8198f30 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:18.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:18.240+0000 7f23cf400640 1 -- 172.21.15.77:0/3735686421 >> 172.21.15.77:0/3735686421 conn(0x7f23c80fefc0 msgr2=0x7f23c80ffa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:18.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:18.240+0000 7f23cf400640 1 -- 172.21.15.77:0/3735686421 shutdown_connections 2024-09-10T09:45:18.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:18.240+0000 7f23cf400640 1 -- 172.21.15.77:0/3735686421 wait complete. 2024-09-10T09:45:18.954 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-10T09:45:19.310 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:45:19.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:19 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:19.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:19 smithi077 ceph-mon[30319]: Added host smithi120 2024-09-10T09:45:19.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:19 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:45:19.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:19 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:19.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:19 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:19.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:19.830+0000 7fd9fbc00640 1 -- 172.21.15.77:0/2428155430 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd9f4109780 msgr2=0x7fd9f4109b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:19.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:19.830+0000 7fd9fbc00640 1 --2- 172.21.15.77:0/2428155430 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd9f4109780 0x7fd9f4109b60 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd9e4009880 tx=0x7fd9e402a5d0 comp rx=0 tx=0).stop 2024-09-10T09:45:19.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:19.831+0000 7fd9fbc00640 1 -- 172.21.15.77:0/2428155430 shutdown_connections 2024-09-10T09:45:19.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:19.831+0000 7fd9fbc00640 1 --2- 172.21.15.77:0/2428155430 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd9f4109780 0x7fd9f4109b60 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:19.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:19.831+0000 7fd9fbc00640 1 -- 172.21.15.77:0/2428155430 >> 172.21.15.77:0/2428155430 conn(0x7fd9f40ff480 msgr2=0x7fd9f41018a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:19.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:19.831+0000 7fd9fbc00640 1 -- 172.21.15.77:0/2428155430 shutdown_connections 2024-09-10T09:45:19.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:19.831+0000 7fd9fbc00640 1 -- 172.21.15.77:0/2428155430 wait complete. 2024-09-10T09:45:19.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:19.832+0000 7fd9fbc00640 1 Processor -- start 2024-09-10T09:45:19.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:19.832+0000 7fd9fbc00640 1 -- start start 2024-09-10T09:45:19.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:19.832+0000 7fd9fbc00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd9f4109780 0x7fd9f4199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:19.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:19.832+0000 7fd9fbc00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f410de40 con 0x7fd9f4109780 2024-09-10T09:45:19.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:19.832+0000 7fd9f9200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd9f4109780 0x7fd9f4199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:19.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:19.832+0000 7fd9f9200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd9f4109780 0x7fd9f4199320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:48524/0 (socket says 172.21.15.77:48524) 2024-09-10T09:45:19.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:19.832+0000 7fd9f9200640 1 -- 172.21.15.77:0/3727027010 learned_addr learned my addr 172.21.15.77:0/3727027010 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:45:19.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:19.833+0000 7fd9f9200640 1 -- 172.21.15.77:0/3727027010 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9f4199860 con 0x7fd9f4109780 2024-09-10T09:45:19.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:19.833+0000 7fd9f9200640 1 --2- 172.21.15.77:0/3727027010 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd9f4109780 0x7fd9f4199320 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd9e4009230 tx=0x7fd9e4003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:19.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:19.833+0000 7fd9f2000640 1 -- 172.21.15.77:0/3727027010 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd9e403f070 con 0x7fd9f4109780 2024-09-10T09:45:19.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:19.833+0000 7fd9f2000640 1 -- 172.21.15.77:0/3727027010 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9e403bc80 con 0x7fd9f4109780 2024-09-10T09:45:19.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:19.833+0000 7fd9fbc00640 1 -- 172.21.15.77:0/3727027010 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9f4199b50 con 0x7fd9f4109780 2024-09-10T09:45:19.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:19.833+0000 7fd9f2000640 1 -- 172.21.15.77:0/3727027010 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd9e4035030 con 0x7fd9f4109780 2024-09-10T09:45:19.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:19.834+0000 7fd9fbc00640 1 -- 172.21.15.77:0/3727027010 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9f419dbf0 con 0x7fd9f4109780 2024-09-10T09:45:19.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:19.835+0000 7fd9f2000640 1 -- 172.21.15.77:0/3727027010 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fd9e4042070 con 0x7fd9f4109780 2024-09-10T09:45:19.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:19.835+0000 7fd9fbc00640 1 -- 172.21.15.77:0/3727027010 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9f4104ec0 con 0x7fd9f4109780 2024-09-10T09:45:19.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:19.835+0000 7fd9f2000640 1 --2- 172.21.15.77:0/3727027010 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fd9d80390c0 0x7fd9d803b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:19.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:19.835+0000 7fd9f2000640 1 -- 172.21.15.77:0/3727027010 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd9e406cd60 con 0x7fd9f4109780 2024-09-10T09:45:19.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:19.835+0000 7fd9ebe00640 1 --2- 172.21.15.77:0/3727027010 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fd9d80390c0 0x7fd9d803b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:19.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:19.836+0000 7fd9ebe00640 1 --2- 172.21.15.77:0/3727027010 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fd9d80390c0 0x7fd9d803b580 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc0098c0 tx=0x7fd9dc006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:19.842 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:19.840+0000 7fd9f2000640 1 -- 172.21.15.77:0/3727027010 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd9e40372d0 con 0x7fd9f4109780 2024-09-10T09:45:20.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:20.021+0000 7fd9fbc00640 1 -- 172.21.15.77:0/3727027010 --> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd9f4109be0 con 0x7fd9d80390c0 2024-09-10T09:45:20.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:20.022+0000 7fd9f2000640 1 -- 172.21.15.77:0/3727027010 <== mgr.14150 v2:172.21.15.77:6800/4261324650 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+160 (secure 0 0 0) 0x7fd9f4109be0 con 0x7fd9d80390c0 2024-09-10T09:45:20.024 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:45:20.024 INFO:teuthology.orchestra.run.smithi077.stdout:[{"addr": "172.21.15.77", "hostname": "smithi077", "labels": [], "status": ""}, {"addr": "172.21.15.120", "hostname": "smithi120", "labels": [], "status": ""}] 2024-09-10T09:45:20.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:20.024+0000 7fd9fbc00640 1 -- 172.21.15.77:0/3727027010 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fd9d80390c0 msgr2=0x7fd9d803b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:20.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:20.024+0000 7fd9fbc00640 1 --2- 172.21.15.77:0/3727027010 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fd9d80390c0 0x7fd9d803b580 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc0098c0 tx=0x7fd9dc006db0 comp rx=0 tx=0).stop 2024-09-10T09:45:20.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:20.024+0000 7fd9fbc00640 1 -- 172.21.15.77:0/3727027010 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd9f4109780 msgr2=0x7fd9f4199320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:20.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:20.024+0000 7fd9fbc00640 1 --2- 172.21.15.77:0/3727027010 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd9f4109780 0x7fd9f4199320 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd9e4009230 tx=0x7fd9e4003d40 comp rx=0 tx=0).stop 2024-09-10T09:45:20.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:20.025+0000 7fd9fbc00640 1 -- 172.21.15.77:0/3727027010 shutdown_connections 2024-09-10T09:45:20.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:20.025+0000 7fd9fbc00640 1 --2- 172.21.15.77:0/3727027010 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fd9d80390c0 0x7fd9d803b580 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:20.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:20.025+0000 7fd9fbc00640 1 --2- 172.21.15.77:0/3727027010 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd9f4109780 0x7fd9f4199320 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:20.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:20.025+0000 7fd9fbc00640 1 -- 172.21.15.77:0/3727027010 >> 172.21.15.77:0/3727027010 conn(0x7fd9f40ff480 msgr2=0x7fd9f4100e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:20.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:20.025+0000 7fd9fbc00640 1 -- 172.21.15.77:0/3727027010 shutdown_connections 2024-09-10T09:45:20.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:20.025+0000 7fd9fbc00640 1 -- 172.21.15.77:0/3727027010 wait complete. 2024-09-10T09:45:20.683 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:20 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:20.684 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:20 smithi077 ceph-mon[30319]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:45:20.721 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi134 2024-09-10T09:45:20.721 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-10T09:45:20.721 DEBUG:teuthology.orchestra.run.smithi134:> dd of=/etc/ceph/ceph.conf 2024-09-10T09:45:20.738 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-10T09:45:20.738 DEBUG:teuthology.orchestra.run.smithi134:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T09:45:20.795 INFO:tasks.cephadm:Adding host smithi134 to orchestrator... 2024-09-10T09:45:20.796 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph orch host add smithi134 2024-09-10T09:45:21.150 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:45:21.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:21.738+0000 7f1c5ae00640 1 -- 172.21.15.77:0/506641286 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1c54109670 msgr2=0x7f1c54109a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:21.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:21.738+0000 7f1c5ae00640 1 --2- 172.21.15.77:0/506641286 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1c54109670 0x7f1c54109a50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1c48009880 tx=0x7f1c4802a5d0 comp rx=0 tx=0).stop 2024-09-10T09:45:21.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:21.738+0000 7f1c5ae00640 1 -- 172.21.15.77:0/506641286 shutdown_connections 2024-09-10T09:45:21.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:21.738+0000 7f1c5ae00640 1 --2- 172.21.15.77:0/506641286 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1c54109670 0x7f1c54109a50 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:21.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:21.738+0000 7f1c5ae00640 1 -- 172.21.15.77:0/506641286 >> 172.21.15.77:0/506641286 conn(0x7f1c540ff260 msgr2=0x7f1c54101680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:21.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:21.738+0000 7f1c5ae00640 1 -- 172.21.15.77:0/506641286 shutdown_connections 2024-09-10T09:45:21.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:21.738+0000 7f1c5ae00640 1 -- 172.21.15.77:0/506641286 wait complete. 2024-09-10T09:45:21.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:21.739+0000 7f1c5ae00640 1 Processor -- start 2024-09-10T09:45:21.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:21.739+0000 7f1c5ae00640 1 -- start start 2024-09-10T09:45:21.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:21.739+0000 7f1c5ae00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1c54109670 0x7f1c54199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:21.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:21.739+0000 7f1c5ae00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1c5410dca0 con 0x7f1c54109670 2024-09-10T09:45:21.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:21.739+0000 7f1c59a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1c54109670 0x7f1c54199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:21.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:21.739+0000 7f1c59a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1c54109670 0x7f1c54199180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:48542/0 (socket says 172.21.15.77:48542) 2024-09-10T09:45:21.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:21.739+0000 7f1c59a00640 1 -- 172.21.15.77:0/1313624434 learned_addr learned my addr 172.21.15.77:0/1313624434 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:45:21.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:21.740+0000 7f1c59a00640 1 -- 172.21.15.77:0/1313624434 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c541996c0 con 0x7f1c54109670 2024-09-10T09:45:21.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:21.740+0000 7f1c59a00640 1 --2- 172.21.15.77:0/1313624434 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1c54109670 0x7f1c54199180 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1c48009230 tx=0x7f1c48003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:21.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:21.740+0000 7f1c52000640 1 -- 172.21.15.77:0/1313624434 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1c4803f070 con 0x7f1c54109670 2024-09-10T09:45:21.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:21.740+0000 7f1c52000640 1 -- 172.21.15.77:0/1313624434 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c4803bc80 con 0x7f1c54109670 2024-09-10T09:45:21.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:21.740+0000 7f1c5ae00640 1 -- 172.21.15.77:0/1313624434 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c54199950 con 0x7f1c54109670 2024-09-10T09:45:21.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:21.740+0000 7f1c52000640 1 -- 172.21.15.77:0/1313624434 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1c48035030 con 0x7f1c54109670 2024-09-10T09:45:21.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:21.741+0000 7f1c5ae00640 1 -- 172.21.15.77:0/1313624434 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c5419da50 con 0x7f1c54109670 2024-09-10T09:45:21.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:21.742+0000 7f1c52000640 1 -- 172.21.15.77:0/1313624434 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f1c48042070 con 0x7f1c54109670 2024-09-10T09:45:21.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:21.742+0000 7f1c5ae00640 1 -- 172.21.15.77:0/1313624434 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c54104ce0 con 0x7f1c54109670 2024-09-10T09:45:21.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:21.742+0000 7f1c52000640 1 --2- 172.21.15.77:0/1313624434 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f1c3c0390c0 0x7f1c3c03b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:21.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:21.742+0000 7f1c52000640 1 -- 172.21.15.77:0/1313624434 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f1c4806cd80 con 0x7f1c54109670 2024-09-10T09:45:21.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:21.742+0000 7f1c59000640 1 --2- 172.21.15.77:0/1313624434 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f1c3c0390c0 0x7f1c3c03b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:21.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:21.743+0000 7f1c59000640 1 --2- 172.21.15.77:0/1313624434 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f1c3c0390c0 0x7f1c3c03b580 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f1c400098c0 tx=0x7f1c40006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:21.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:21.747+0000 7f1c52000640 1 -- 172.21.15.77:0/1313624434 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1c480372d0 con 0x7f1c54109670 2024-09-10T09:45:21.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:21.924+0000 7f1c5ae00640 1 -- 172.21.15.77:0/1313624434 --> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi134", "target": ["mon-mgr", ""]}) -- 0x7f1c5419a4e0 con 0x7f1c3c0390c0 2024-09-10T09:45:21.926 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:21 smithi077 ceph-mon[30319]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T09:45:22.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:22 smithi077 ceph-mon[30319]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:45:22.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:22 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:22.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:22 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:22.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:22 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:22.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:22 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:22.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:22 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi120", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:45:22.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:22 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:45:22.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:22 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:45:23.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:23 smithi077 ceph-mon[30319]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi134", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:45:23.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:23 smithi077 ceph-mon[30319]: Updating smithi120:/etc/ceph/ceph.conf 2024-09-10T09:45:23.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:23 smithi077 ceph-mon[30319]: Updating smithi120:/var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T09:45:23.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:23 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:23.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:23 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:23.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:23 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:24.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:24 smithi077 ceph-mon[30319]: Deploying cephadm binary to smithi134 2024-09-10T09:45:24.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:24 smithi077 ceph-mon[30319]: Updating smithi120:/etc/ceph/ceph.client.admin.keyring 2024-09-10T09:45:24.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:24 smithi077 ceph-mon[30319]: Updating smithi120:/var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T09:45:24.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:24 smithi077 ceph-mon[30319]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:45:25.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:25.177+0000 7f1c52000640 1 -- 172.21.15.77:0/1313624434 <== mgr.14150 v2:172.21.15.77:6800/4261324650 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f1c5419a4e0 con 0x7f1c3c0390c0 2024-09-10T09:45:25.179 INFO:teuthology.orchestra.run.smithi077.stdout:Added host 'smithi134' with addr '172.21.15.134' 2024-09-10T09:45:25.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:25.180+0000 7f1c5ae00640 1 -- 172.21.15.77:0/1313624434 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f1c3c0390c0 msgr2=0x7f1c3c03b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:25.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:25.180+0000 7f1c5ae00640 1 --2- 172.21.15.77:0/1313624434 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f1c3c0390c0 0x7f1c3c03b580 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f1c400098c0 tx=0x7f1c40006db0 comp rx=0 tx=0).stop 2024-09-10T09:45:25.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:25.180+0000 7f1c5ae00640 1 -- 172.21.15.77:0/1313624434 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1c54109670 msgr2=0x7f1c54199180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:25.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:25.180+0000 7f1c5ae00640 1 --2- 172.21.15.77:0/1313624434 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1c54109670 0x7f1c54199180 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1c48009230 tx=0x7f1c48003d40 comp rx=0 tx=0).stop 2024-09-10T09:45:25.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:25.180+0000 7f1c5ae00640 1 -- 172.21.15.77:0/1313624434 shutdown_connections 2024-09-10T09:45:25.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:25.180+0000 7f1c5ae00640 1 --2- 172.21.15.77:0/1313624434 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f1c3c0390c0 0x7f1c3c03b580 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:25.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:25.180+0000 7f1c5ae00640 1 --2- 172.21.15.77:0/1313624434 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1c54109670 0x7f1c54199180 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:25.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:25.181+0000 7f1c5ae00640 1 -- 172.21.15.77:0/1313624434 >> 172.21.15.77:0/1313624434 conn(0x7f1c540ff260 msgr2=0x7f1c54100ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:25.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:25.181+0000 7f1c5ae00640 1 -- 172.21.15.77:0/1313624434 shutdown_connections 2024-09-10T09:45:25.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:25.181+0000 7f1c5ae00640 1 -- 172.21.15.77:0/1313624434 wait complete. 2024-09-10T09:45:25.912 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-10T09:45:26.282 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:45:26.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:26 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:26.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:26 smithi077 ceph-mon[30319]: Added host smithi134 2024-09-10T09:45:26.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:26 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:45:26.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:26 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:26.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:26 smithi077 ceph-mon[30319]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:45:26.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:26 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:26.804 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:26.802+0000 7f6b71e00640 1 -- 172.21.15.77:0/3916118978 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b6c0745c0 msgr2=0x7f6b6c0749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:26.805 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:26.802+0000 7f6b71e00640 1 --2- 172.21.15.77:0/3916118978 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b6c0745c0 0x7f6b6c0749a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c009880 tx=0x7f6b5c02a670 comp rx=0 tx=0).stop 2024-09-10T09:45:26.805 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:26.802+0000 7f6b71e00640 1 -- 172.21.15.77:0/3916118978 shutdown_connections 2024-09-10T09:45:26.805 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:26.802+0000 7f6b71e00640 1 --2- 172.21.15.77:0/3916118978 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b6c0745c0 0x7f6b6c0749a0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:26.805 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:26.802+0000 7f6b71e00640 1 -- 172.21.15.77:0/3916118978 >> 172.21.15.77:0/3916118978 conn(0x7f6b6c0fd290 msgr2=0x7f6b6c0ff6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:26.805 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:26.802+0000 7f6b71e00640 1 -- 172.21.15.77:0/3916118978 shutdown_connections 2024-09-10T09:45:26.805 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:26.802+0000 7f6b71e00640 1 -- 172.21.15.77:0/3916118978 wait complete. 2024-09-10T09:45:26.806 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:26.803+0000 7f6b71e00640 1 Processor -- start 2024-09-10T09:45:26.806 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:26.803+0000 7f6b71e00640 1 -- start start 2024-09-10T09:45:26.806 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:26.804+0000 7f6b71e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b6c0745c0 0x7f6b6c1993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:26.806 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:26.804+0000 7f6b71e00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6b6c10deb0 con 0x7f6b6c0745c0 2024-09-10T09:45:26.806 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:26.804+0000 7f6b6aa00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b6c0745c0 0x7f6b6c1993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:26.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:26.804+0000 7f6b6aa00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b6c0745c0 0x7f6b6c1993b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:36166/0 (socket says 172.21.15.77:36166) 2024-09-10T09:45:26.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:26.804+0000 7f6b6aa00640 1 -- 172.21.15.77:0/914802010 learned_addr learned my addr 172.21.15.77:0/914802010 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:45:26.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:26.804+0000 7f6b6aa00640 1 -- 172.21.15.77:0/914802010 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b6c1998f0 con 0x7f6b6c0745c0 2024-09-10T09:45:26.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:26.805+0000 7f6b6aa00640 1 --2- 172.21.15.77:0/914802010 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b6c0745c0 0x7f6b6c1993b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c009230 tx=0x7f6b5c003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:26.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:26.805+0000 7f6b70a00640 1 -- 172.21.15.77:0/914802010 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b5c03f070 con 0x7f6b6c0745c0 2024-09-10T09:45:26.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:26.805+0000 7f6b70a00640 1 -- 172.21.15.77:0/914802010 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b5c03bd40 con 0x7f6b6c0745c0 2024-09-10T09:45:26.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:26.805+0000 7f6b70a00640 1 -- 172.21.15.77:0/914802010 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b5c035030 con 0x7f6b6c0745c0 2024-09-10T09:45:26.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:26.805+0000 7f6b71e00640 1 -- 172.21.15.77:0/914802010 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b6c199b80 con 0x7f6b6c0745c0 2024-09-10T09:45:26.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:26.805+0000 7f6b71e00640 1 -- 172.21.15.77:0/914802010 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b6c199fe0 con 0x7f6b6c0745c0 2024-09-10T09:45:26.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:26.806+0000 7f6b70a00640 1 -- 172.21.15.77:0/914802010 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f6b5c042070 con 0x7f6b6c0745c0 2024-09-10T09:45:26.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:26.806+0000 7f6b71e00640 1 -- 172.21.15.77:0/914802010 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b6c075c70 con 0x7f6b6c0745c0 2024-09-10T09:45:26.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:26.807+0000 7f6b70a00640 1 --2- 172.21.15.77:0/914802010 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f6b500390c0 0x7f6b5003b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:26.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:26.807+0000 7f6b70a00640 1 -- 172.21.15.77:0/914802010 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f6b5c06cc90 con 0x7f6b6c0745c0 2024-09-10T09:45:26.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:26.807+0000 7f6b63e00640 1 --2- 172.21.15.77:0/914802010 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f6b500390c0 0x7f6b5003b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:26.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:26.808+0000 7f6b63e00640 1 --2- 172.21.15.77:0/914802010 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f6b500390c0 0x7f6b5003b580 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6b540098c0 tx=0x7f6b54006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:26.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:26.812+0000 7f6b70a00640 1 -- 172.21.15.77:0/914802010 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6b5c036df0 con 0x7f6b6c0745c0 2024-09-10T09:45:26.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:26.994+0000 7f6b71e00640 1 -- 172.21.15.77:0/914802010 --> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6b6c19a770 con 0x7f6b500390c0 2024-09-10T09:45:26.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:26.995+0000 7f6b70a00640 1 -- 172.21.15.77:0/914802010 <== mgr.14150 v2:172.21.15.77:6800/4261324650 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+240 (secure 0 0 0) 0x7f6b6c19a770 con 0x7f6b500390c0 2024-09-10T09:45:26.997 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:45:26.997 INFO:teuthology.orchestra.run.smithi077.stdout:[{"addr": "172.21.15.77", "hostname": "smithi077", "labels": [], "status": ""}, {"addr": "172.21.15.120", "hostname": "smithi120", "labels": [], "status": ""}, {"addr": "172.21.15.134", "hostname": "smithi134", "labels": [], "status": ""}] 2024-09-10T09:45:27.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:26.998+0000 7f6b71e00640 1 -- 172.21.15.77:0/914802010 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f6b500390c0 msgr2=0x7f6b5003b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:27.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:26.998+0000 7f6b71e00640 1 --2- 172.21.15.77:0/914802010 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f6b500390c0 0x7f6b5003b580 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6b540098c0 tx=0x7f6b54006db0 comp rx=0 tx=0).stop 2024-09-10T09:45:27.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:26.998+0000 7f6b71e00640 1 -- 172.21.15.77:0/914802010 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b6c0745c0 msgr2=0x7f6b6c1993b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:27.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:26.998+0000 7f6b71e00640 1 --2- 172.21.15.77:0/914802010 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b6c0745c0 0x7f6b6c1993b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c009230 tx=0x7f6b5c003d80 comp rx=0 tx=0).stop 2024-09-10T09:45:27.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:26.998+0000 7f6b71e00640 1 -- 172.21.15.77:0/914802010 shutdown_connections 2024-09-10T09:45:27.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:26.998+0000 7f6b71e00640 1 --2- 172.21.15.77:0/914802010 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f6b500390c0 0x7f6b5003b580 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:27.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:26.998+0000 7f6b71e00640 1 --2- 172.21.15.77:0/914802010 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b6c0745c0 0x7f6b6c1993b0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:27.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:26.998+0000 7f6b71e00640 1 -- 172.21.15.77:0/914802010 >> 172.21.15.77:0/914802010 conn(0x7f6b6c0fd290 msgr2=0x7f6b6c105890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:27.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:26.998+0000 7f6b71e00640 1 -- 172.21.15.77:0/914802010 shutdown_connections 2024-09-10T09:45:27.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:26.998+0000 7f6b71e00640 1 -- 172.21.15.77:0/914802010 wait complete. 2024-09-10T09:45:27.781 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:27 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:27.817 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-10T09:45:27.817 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph osd crush tunables default 2024-09-10T09:45:28.170 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:45:28.576 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:28 smithi077 ceph-mon[30319]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T09:45:28.576 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:28 smithi077 ceph-mon[30319]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:45:28.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:28.676+0000 7ff11f800640 1 -- 172.21.15.77:0/896900177 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff118107640 msgr2=0x7ff118107a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:28.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:28.676+0000 7ff11f800640 1 --2- 172.21.15.77:0/896900177 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff118107640 0x7ff118107a20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff10c009880 tx=0x7ff10c02a5d0 comp rx=0 tx=0).stop 2024-09-10T09:45:28.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:28.677+0000 7ff11f800640 1 -- 172.21.15.77:0/896900177 shutdown_connections 2024-09-10T09:45:28.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:28.677+0000 7ff11f800640 1 --2- 172.21.15.77:0/896900177 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff118107640 0x7ff118107a20 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:28.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:28.677+0000 7ff11f800640 1 -- 172.21.15.77:0/896900177 >> 172.21.15.77:0/896900177 conn(0x7ff1180fd230 msgr2=0x7ff1180ff650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:28.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:28.677+0000 7ff11f800640 1 -- 172.21.15.77:0/896900177 shutdown_connections 2024-09-10T09:45:28.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:28.677+0000 7ff11f800640 1 -- 172.21.15.77:0/896900177 wait complete. 2024-09-10T09:45:28.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:28.678+0000 7ff11f800640 1 Processor -- start 2024-09-10T09:45:28.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:28.678+0000 7ff11f800640 1 -- start start 2024-09-10T09:45:28.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:28.678+0000 7ff11f800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff118107640 0x7ff118199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:28.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:28.678+0000 7ff11f800640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff11810bc70 con 0x7ff118107640 2024-09-10T09:45:28.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:28.678+0000 7ff11ce00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff118107640 0x7ff118199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:28.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:28.678+0000 7ff11ce00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff118107640 0x7ff118199330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:36184/0 (socket says 172.21.15.77:36184) 2024-09-10T09:45:28.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:28.678+0000 7ff11ce00640 1 -- 172.21.15.77:0/3142331239 learned_addr learned my addr 172.21.15.77:0/3142331239 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:45:28.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:28.679+0000 7ff11ce00640 1 -- 172.21.15.77:0/3142331239 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff118199870 con 0x7ff118107640 2024-09-10T09:45:28.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:28.679+0000 7ff11ce00640 1 --2- 172.21.15.77:0/3142331239 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff118107640 0x7ff118199330 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff10c009230 tx=0x7ff10c003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:28.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:28.679+0000 7ff115600640 1 -- 172.21.15.77:0/3142331239 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff10c03f070 con 0x7ff118107640 2024-09-10T09:45:28.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:28.679+0000 7ff115600640 1 -- 172.21.15.77:0/3142331239 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff10c03bd40 con 0x7ff118107640 2024-09-10T09:45:28.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:28.679+0000 7ff11f800640 1 -- 172.21.15.77:0/3142331239 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff118199b00 con 0x7ff118107640 2024-09-10T09:45:28.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:28.679+0000 7ff115600640 1 -- 172.21.15.77:0/3142331239 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff10c035030 con 0x7ff118107640 2024-09-10T09:45:28.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:28.679+0000 7ff11f800640 1 -- 172.21.15.77:0/3142331239 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff11819dc00 con 0x7ff118107640 2024-09-10T09:45:28.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:28.680+0000 7ff115600640 1 -- 172.21.15.77:0/3142331239 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7ff10c042070 con 0x7ff118107640 2024-09-10T09:45:28.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:28.680+0000 7ff11f800640 1 -- 172.21.15.77:0/3142331239 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff118102cb0 con 0x7ff118107640 2024-09-10T09:45:28.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:28.681+0000 7ff115600640 1 --2- 172.21.15.77:0/3142331239 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ff100039060 0x7ff10003b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:28.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:28.681+0000 7ff115600640 1 -- 172.21.15.77:0/3142331239 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff10c06cd20 con 0x7ff118107640 2024-09-10T09:45:28.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:28.681+0000 7ff117e00640 1 --2- 172.21.15.77:0/3142331239 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ff100039060 0x7ff10003b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:28.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:28.682+0000 7ff117e00640 1 --2- 172.21.15.77:0/3142331239 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ff100039060 0x7ff10003b520 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7ff1040098c0 tx=0x7ff104006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:28.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:28.686+0000 7ff115600640 1 -- 172.21.15.77:0/3142331239 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff10c030d60 con 0x7ff118107640 2024-09-10T09:45:28.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:28.864+0000 7ff11f800640 1 -- 172.21.15.77:0/3142331239 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7ff11819a690 con 0x7ff118107640 2024-09-10T09:45:29.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:29.562+0000 7ff115600640 1 -- 172.21.15.77:0/3142331239 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7ff10c03b860 con 0x7ff118107640 2024-09-10T09:45:29.564 INFO:teuthology.orchestra.run.smithi077.stderr:adjusted tunables profile to default 2024-09-10T09:45:29.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:29.564+0000 7ff11f800640 1 -- 172.21.15.77:0/3142331239 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ff100039060 msgr2=0x7ff10003b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:29.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:29.564+0000 7ff11f800640 1 --2- 172.21.15.77:0/3142331239 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ff100039060 0x7ff10003b520 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7ff1040098c0 tx=0x7ff104006db0 comp rx=0 tx=0).stop 2024-09-10T09:45:29.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:29.564+0000 7ff11f800640 1 -- 172.21.15.77:0/3142331239 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff118107640 msgr2=0x7ff118199330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:29.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:29.564+0000 7ff11f800640 1 --2- 172.21.15.77:0/3142331239 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff118107640 0x7ff118199330 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff10c009230 tx=0x7ff10c003d80 comp rx=0 tx=0).stop 2024-09-10T09:45:29.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:29.564+0000 7ff11f800640 1 -- 172.21.15.77:0/3142331239 shutdown_connections 2024-09-10T09:45:29.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:29.564+0000 7ff11f800640 1 --2- 172.21.15.77:0/3142331239 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ff100039060 0x7ff10003b520 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:29.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:29.564+0000 7ff11f800640 1 --2- 172.21.15.77:0/3142331239 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff118107640 0x7ff118199330 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:29.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:29.564+0000 7ff11f800640 1 -- 172.21.15.77:0/3142331239 >> 172.21.15.77:0/3142331239 conn(0x7ff1180fd230 msgr2=0x7ff118075e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:29.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:29.564+0000 7ff11f800640 1 -- 172.21.15.77:0/3142331239 shutdown_connections 2024-09-10T09:45:29.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:29.564+0000 7ff11f800640 1 -- 172.21.15.77:0/3142331239 wait complete. 2024-09-10T09:45:29.914 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:29 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/3142331239' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-10T09:45:29.914 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:29 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:29.914 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:29 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:29.914 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:29 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:29.914 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:29 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:29.914 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:29 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi134", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:45:29.914 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:29 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:45:29.914 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:29 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:45:30.307 INFO:tasks.cephadm:Adding mon.a on smithi077 2024-09-10T09:45:30.307 INFO:tasks.cephadm:Adding mon.b on smithi120 2024-09-10T09:45:30.307 INFO:tasks.cephadm:Adding mon.c on smithi134 2024-09-10T09:45:30.308 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph orch apply mon '3;smithi077:172.21.15.77=a;smithi120:172.21.15.120=b;smithi134:172.21.15.134=c' 2024-09-10T09:45:30.544 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T09:45:30.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:30 smithi077 ceph-mon[30319]: Updating smithi134:/etc/ceph/ceph.conf 2024-09-10T09:45:30.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:30 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/3142331239' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-10T09:45:30.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:30 smithi077 ceph-mon[30319]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T09:45:30.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:30 smithi077 ceph-mon[30319]: Updating smithi134:/var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T09:45:30.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:30 smithi077 ceph-mon[30319]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:45:30.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:30 smithi077 ceph-mon[30319]: Updating smithi134:/etc/ceph/ceph.client.admin.keyring 2024-09-10T09:45:30.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:30 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:30.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:30 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:30.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:30 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:30.942 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:30.940+0000 7fdc5c55a640 1 -- 172.21.15.134:0/2936962841 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdc54107650 msgr2=0x7fdc54107a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:30.942 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:30.940+0000 7fdc5c55a640 1 --2- 172.21.15.134:0/2936962841 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdc54107650 0x7fdc54107a30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fdc50009880 tx=0x7fdc5002a5d0 comp rx=0 tx=0).stop 2024-09-10T09:45:30.943 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:30.941+0000 7fdc5c55a640 1 -- 172.21.15.134:0/2936962841 shutdown_connections 2024-09-10T09:45:30.943 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:30.941+0000 7fdc5c55a640 1 --2- 172.21.15.134:0/2936962841 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdc54107650 0x7fdc54107a30 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:30.943 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:30.941+0000 7fdc5c55a640 1 -- 172.21.15.134:0/2936962841 >> 172.21.15.134:0/2936962841 conn(0x7fdc540fd260 msgr2=0x7fdc540ff680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:30.943 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:30.941+0000 7fdc5c55a640 1 -- 172.21.15.134:0/2936962841 shutdown_connections 2024-09-10T09:45:30.943 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:30.941+0000 7fdc5c55a640 1 -- 172.21.15.134:0/2936962841 wait complete. 2024-09-10T09:45:30.943 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:30.941+0000 7fdc5c55a640 1 Processor -- start 2024-09-10T09:45:30.943 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:30.941+0000 7fdc5c55a640 1 -- start start 2024-09-10T09:45:30.943 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:30.941+0000 7fdc5c55a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdc54107650 0x7fdc54197110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:30.943 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:30.941+0000 7fdc5c55a640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fdc5410bc80 con 0x7fdc54107650 2024-09-10T09:45:30.945 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:30.942+0000 7fdc5a2f6640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdc54107650 0x7fdc54197110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:30.945 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:30.942+0000 7fdc5a2f6640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdc54107650 0x7fdc54197110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.134:35062/0 (socket says 172.21.15.134:35062) 2024-09-10T09:45:30.945 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:30.942+0000 7fdc5a2f6640 1 -- 172.21.15.134:0/1463255152 learned_addr learned my addr 172.21.15.134:0/1463255152 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-10T09:45:30.945 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:30.942+0000 7fdc5a2f6640 1 -- 172.21.15.134:0/1463255152 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc54197650 con 0x7fdc54107650 2024-09-10T09:45:30.945 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:30.942+0000 7fdc5a2f6640 1 --2- 172.21.15.134:0/1463255152 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdc54107650 0x7fdc54197110 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fdc50009230 tx=0x7fdc50003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:30.945 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:30.943+0000 7fdc4b7fe640 1 -- 172.21.15.134:0/1463255152 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc5003f070 con 0x7fdc54107650 2024-09-10T09:45:30.945 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:30.943+0000 7fdc5c55a640 1 -- 172.21.15.134:0/1463255152 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc541978e0 con 0x7fdc54107650 2024-09-10T09:45:30.945 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:30.943+0000 7fdc4b7fe640 1 -- 172.21.15.134:0/1463255152 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc5003bd40 con 0x7fdc54107650 2024-09-10T09:45:30.945 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:30.943+0000 7fdc4b7fe640 1 -- 172.21.15.134:0/1463255152 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc50035030 con 0x7fdc54107650 2024-09-10T09:45:30.945 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:30.943+0000 7fdc5c55a640 1 -- 172.21.15.134:0/1463255152 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc5419b9e0 con 0x7fdc54107650 2024-09-10T09:45:30.946 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:30.944+0000 7fdc4b7fe640 1 -- 172.21.15.134:0/1463255152 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fdc50042070 con 0x7fdc54107650 2024-09-10T09:45:30.946 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:30.944+0000 7fdc5c55a640 1 -- 172.21.15.134:0/1463255152 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc54102cc0 con 0x7fdc54107650 2024-09-10T09:45:30.946 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:30.944+0000 7fdc4b7fe640 1 --2- 172.21.15.134:0/1463255152 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fdc40039060 0x7fdc4003b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:30.946 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:30.944+0000 7fdc4b7fe640 1 -- 172.21.15.134:0/1463255152 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fdc5006cc40 con 0x7fdc54107650 2024-09-10T09:45:30.946 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:30.944+0000 7fdc59af5640 1 --2- 172.21.15.134:0/1463255152 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fdc40039060 0x7fdc4003b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:30.946 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:30.945+0000 7fdc59af5640 1 --2- 172.21.15.134:0/1463255152 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fdc40039060 0x7fdc4003b520 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fdc440098c0 tx=0x7fdc44006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:30.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:30.949+0000 7fdc4b7fe640 1 -- 172.21.15.134:0/1463255152 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdc50036df0 con 0x7fdc54107650 2024-09-10T09:45:31.134 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:31.133+0000 7fdc5c55a640 1 -- 172.21.15.134:0/1463255152 --> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi077:172.21.15.77=a;smithi120:172.21.15.120=b;smithi134:172.21.15.134=c", "target": ["mon-mgr", ""]}) -- 0x7fdc54198470 con 0x7fdc40039060 2024-09-10T09:45:31.141 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:31.139+0000 7fdc4b7fe640 1 -- 172.21.15.134:0/1463255152 <== mgr.14150 v2:172.21.15.77:6800/4261324650 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fdc54198470 con 0x7fdc40039060 2024-09-10T09:45:31.141 INFO:teuthology.orchestra.run.smithi134.stdout:Scheduled mon update... 2024-09-10T09:45:31.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:31.142+0000 7fdc5c55a640 1 -- 172.21.15.134:0/1463255152 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fdc40039060 msgr2=0x7fdc4003b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:31.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:31.142+0000 7fdc5c55a640 1 --2- 172.21.15.134:0/1463255152 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fdc40039060 0x7fdc4003b520 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fdc440098c0 tx=0x7fdc44006db0 comp rx=0 tx=0).stop 2024-09-10T09:45:31.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:31.142+0000 7fdc5c55a640 1 -- 172.21.15.134:0/1463255152 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdc54107650 msgr2=0x7fdc54197110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:31.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:31.142+0000 7fdc5c55a640 1 --2- 172.21.15.134:0/1463255152 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdc54107650 0x7fdc54197110 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fdc50009230 tx=0x7fdc50003d80 comp rx=0 tx=0).stop 2024-09-10T09:45:31.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:31.142+0000 7fdc5c55a640 1 -- 172.21.15.134:0/1463255152 shutdown_connections 2024-09-10T09:45:31.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:31.142+0000 7fdc5c55a640 1 --2- 172.21.15.134:0/1463255152 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fdc40039060 0x7fdc4003b520 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:31.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:31.142+0000 7fdc5c55a640 1 --2- 172.21.15.134:0/1463255152 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdc54107650 0x7fdc54197110 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:31.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:31.142+0000 7fdc5c55a640 1 -- 172.21.15.134:0/1463255152 >> 172.21.15.134:0/1463255152 conn(0x7fdc540fd260 msgr2=0x7fdc540fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:31.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:31.142+0000 7fdc5c55a640 1 -- 172.21.15.134:0/1463255152 shutdown_connections 2024-09-10T09:45:31.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:31.142+0000 7fdc5c55a640 1 -- 172.21.15.134:0/1463255152 wait complete. 2024-09-10T09:45:31.743 DEBUG:teuthology.orchestra.run.smithi120:mon.b> sudo journalctl -f -n 0 -u ceph-14a55226-6f59-11ef-bcea-c7b262605968@mon.b.service 2024-09-10T09:45:31.746 DEBUG:teuthology.orchestra.run.smithi134:mon.c> sudo journalctl -f -n 0 -u ceph-14a55226-6f59-11ef-bcea-c7b262605968@mon.c.service 2024-09-10T09:45:31.748 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-10T09:45:31.748 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T09:45:31.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:31 smithi077 ceph-mon[30319]: Updating smithi134:/var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T09:45:31.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:31 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:31.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:31 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:45:31.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:31 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:45:31.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:31 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:45:31.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:31 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:31.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:31 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T09:45:31.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:31 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:45:32.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:32 smithi077 ceph-mon[30319]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi077:172.21.15.77=a;smithi120:172.21.15.120=b;smithi134:172.21.15.134=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:45:32.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:32 smithi077 ceph-mon[30319]: Saving service mon spec with placement smithi077:172.21.15.77=a;smithi120:172.21.15.120=b;smithi134:172.21.15.134=c;count:3 2024-09-10T09:45:32.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:32 smithi077 ceph-mon[30319]: Deploying daemon mon.c on smithi134 2024-09-10T09:45:32.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:32 smithi077 ceph-mon[30319]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:45:33.768 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T09:45:34.415 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:34.412+0000 7fd38f2fb640 1 -- 172.21.15.134:0/799620462 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd388106650 msgr2=0x7fd388106a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:34.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:34.412+0000 7fd38f2fb640 1 --2- 172.21.15.134:0/799620462 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd388106650 0x7fd388106a30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd384009880 tx=0x7fd38402a5d0 comp rx=0 tx=0).stop 2024-09-10T09:45:34.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:34.413+0000 7fd38f2fb640 1 -- 172.21.15.134:0/799620462 shutdown_connections 2024-09-10T09:45:34.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:34.413+0000 7fd38f2fb640 1 --2- 172.21.15.134:0/799620462 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd388106650 0x7fd388106a30 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:34.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:34.413+0000 7fd38f2fb640 1 -- 172.21.15.134:0/799620462 >> 172.21.15.134:0/799620462 conn(0x7fd3880fc280 msgr2=0x7fd3880fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:34.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:34.413+0000 7fd38f2fb640 1 -- 172.21.15.134:0/799620462 shutdown_connections 2024-09-10T09:45:34.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:34.413+0000 7fd38f2fb640 1 -- 172.21.15.134:0/799620462 wait complete. 2024-09-10T09:45:34.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:34.413+0000 7fd38f2fb640 1 Processor -- start 2024-09-10T09:45:34.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:34.413+0000 7fd38f2fb640 1 -- start start 2024-09-10T09:45:34.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:34.414+0000 7fd38f2fb640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd388106650 0x7fd388197200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:34.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:34.414+0000 7fd38f2fb640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd38810ac80 con 0x7fd388106650 2024-09-10T09:45:34.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:34.414+0000 7fd38d097640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd388106650 0x7fd388197200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:34.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:34.414+0000 7fd38d097640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd388106650 0x7fd388197200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.134:35084/0 (socket says 172.21.15.134:35084) 2024-09-10T09:45:34.417 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:34.414+0000 7fd38d097640 1 -- 172.21.15.134:0/971624609 learned_addr learned my addr 172.21.15.134:0/971624609 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-10T09:45:34.417 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:34.415+0000 7fd38d097640 1 -- 172.21.15.134:0/971624609 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd388197740 con 0x7fd388106650 2024-09-10T09:45:34.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:34.415+0000 7fd38d097640 1 --2- 172.21.15.134:0/971624609 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd388106650 0x7fd388197200 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd384009230 tx=0x7fd384003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:34.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:34.415+0000 7fd37e7fc640 1 -- 172.21.15.134:0/971624609 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd38403f070 con 0x7fd388106650 2024-09-10T09:45:34.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:34.415+0000 7fd38f2fb640 1 -- 172.21.15.134:0/971624609 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3881979d0 con 0x7fd388106650 2024-09-10T09:45:34.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:34.415+0000 7fd38f2fb640 1 -- 172.21.15.134:0/971624609 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd38819bad0 con 0x7fd388106650 2024-09-10T09:45:34.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:34.415+0000 7fd37e7fc640 1 -- 172.21.15.134:0/971624609 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd38403bc80 con 0x7fd388106650 2024-09-10T09:45:34.419 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:34.415+0000 7fd37e7fc640 1 -- 172.21.15.134:0/971624609 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd384049060 con 0x7fd388106650 2024-09-10T09:45:34.419 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:34.416+0000 7fd38f2fb640 1 -- 172.21.15.134:0/971624609 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd388101cc0 con 0x7fd388106650 2024-09-10T09:45:34.419 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:34.416+0000 7fd37e7fc640 1 -- 172.21.15.134:0/971624609 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fd384042070 con 0x7fd388106650 2024-09-10T09:45:34.419 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:34.416+0000 7fd37e7fc640 1 --2- 172.21.15.134:0/971624609 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fd374039060 0x7fd37403b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:34.419 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:34.417+0000 7fd37e7fc640 1 -- 172.21.15.134:0/971624609 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fd38406cd60 con 0x7fd388106650 2024-09-10T09:45:34.419 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:34.417+0000 7fd38c896640 1 --2- 172.21.15.134:0/971624609 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fd374039060 0x7fd37403b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:34.419 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:34.417+0000 7fd38c896640 1 --2- 172.21.15.134:0/971624609 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fd374039060 0x7fd37403b520 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd3780098c0 tx=0x7fd378006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:34.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:34.422+0000 7fd37e7fc640 1 -- 172.21.15.134:0/971624609 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd384030e20 con 0x7fd388106650 2024-09-10T09:45:34.699 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:34.698+0000 7fd38f2fb640 1 -- 172.21.15.134:0/971624609 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fd388198560 con 0x7fd388106650 2024-09-10T09:45:34.700 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:34.699+0000 7fd37e7fc640 1 -- 172.21.15.134:0/971624609 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7fd384041e50 con 0x7fd388106650 2024-09-10T09:45:34.700 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-10T09:45:34.700 INFO:teuthology.orchestra.run.smithi134.stdout:{"epoch":1,"fsid":"14a55226-6f59-11ef-bcea-c7b262605968","modified":"2024-09-10T09:44:03.655548Z","created":"2024-09-10T09:44:03.655548Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:3300","nonce":0},{"type":"v1","addr":"172.21.15.77:6789","nonce":0}]},"addr":"172.21.15.77:6789/0","public_addr":"172.21.15.77:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-10T09:45:34.700 INFO:teuthology.orchestra.run.smithi134.stderr:dumped monmap epoch 1 2024-09-10T09:45:34.702 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:34.701+0000 7fd38f2fb640 1 -- 172.21.15.134:0/971624609 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fd374039060 msgr2=0x7fd37403b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:34.702 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:34.701+0000 7fd38f2fb640 1 --2- 172.21.15.134:0/971624609 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fd374039060 0x7fd37403b520 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd3780098c0 tx=0x7fd378006db0 comp rx=0 tx=0).stop 2024-09-10T09:45:34.702 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:34.701+0000 7fd38f2fb640 1 -- 172.21.15.134:0/971624609 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd388106650 msgr2=0x7fd388197200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:34.702 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:34.701+0000 7fd38f2fb640 1 --2- 172.21.15.134:0/971624609 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd388106650 0x7fd388197200 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd384009230 tx=0x7fd384003d40 comp rx=0 tx=0).stop 2024-09-10T09:45:34.702 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:34.701+0000 7fd38f2fb640 1 -- 172.21.15.134:0/971624609 shutdown_connections 2024-09-10T09:45:34.702 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:34.701+0000 7fd38f2fb640 1 --2- 172.21.15.134:0/971624609 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fd374039060 0x7fd37403b520 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:34.702 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:34.701+0000 7fd38f2fb640 1 --2- 172.21.15.134:0/971624609 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd388106650 0x7fd388197200 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:34.702 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:34.701+0000 7fd38f2fb640 1 -- 172.21.15.134:0/971624609 >> 172.21.15.134:0/971624609 conn(0x7fd3880fc280 msgr2=0x7fd3880fdc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:34.702 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:34.701+0000 7fd38f2fb640 1 -- 172.21.15.134:0/971624609 shutdown_connections 2024-09-10T09:45:34.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:34.701+0000 7fd38f2fb640 1 -- 172.21.15.134:0/971624609 wait complete. 2024-09-10T09:45:34.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:34 smithi077 ceph-mon[30319]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:45:35.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:35 smithi134 systemd[1]: Started Ceph mon.c for 14a55226-6f59-11ef-bcea-c7b262605968. 2024-09-10T09:45:35.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:35 smithi077 ceph-mon[30319]: from='client.? 172.21.15.134:0/971624609' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T09:45:36.281 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-10T09:45:36.281 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T09:45:36.538 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T09:45:41.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:40 smithi134 ceph-mon[35674]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:45:41.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:40 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T09:45:41.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:40 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T09:45:41.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:40 smithi134 ceph-mon[35674]: mon.a calling monitor election 2024-09-10T09:45:41.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:40 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T09:45:41.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:40 smithi134 ceph-mon[35674]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:45:41.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:40 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T09:45:41.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:40 smithi134 ceph-mon[35674]: mon.c calling monitor election 2024-09-10T09:45:41.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:40 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T09:45:41.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:40 smithi134 ceph-mon[35674]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:45:41.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:40 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T09:45:41.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:40 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T09:45:41.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:40 smithi134 ceph-mon[35674]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-10T09:45:41.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:40 smithi134 ceph-mon[35674]: monmap epoch 2 2024-09-10T09:45:41.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:40 smithi134 ceph-mon[35674]: fsid 14a55226-6f59-11ef-bcea-c7b262605968 2024-09-10T09:45:41.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:40 smithi134 ceph-mon[35674]: last_changed 2024-09-10T09:45:35.685083+0000 2024-09-10T09:45:41.133 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:40 smithi134 ceph-mon[35674]: created 2024-09-10T09:44:03.655548+0000 2024-09-10T09:45:41.133 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:40 smithi134 ceph-mon[35674]: min_mon_release 19 (squid) 2024-09-10T09:45:41.133 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:40 smithi134 ceph-mon[35674]: election_strategy: 1 2024-09-10T09:45:41.133 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:40 smithi134 ceph-mon[35674]: 0: [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] mon.a 2024-09-10T09:45:41.133 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:40 smithi134 ceph-mon[35674]: 1: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.c 2024-09-10T09:45:41.133 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:40 smithi134 ceph-mon[35674]: fsmap 2024-09-10T09:45:41.133 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:40 smithi134 ceph-mon[35674]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T09:45:41.133 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:40 smithi134 ceph-mon[35674]: mgrmap e13: x(active, since 41s) 2024-09-10T09:45:41.133 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:40 smithi134 ceph-mon[35674]: overall HEALTH_OK 2024-09-10T09:45:41.133 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:40 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:41.133 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:40 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:41.133 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:40 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:41.133 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:40 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T09:45:41.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:40 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:45:41.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:40 smithi077 ceph-mon[30319]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:45:41.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:40 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T09:45:41.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:40 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T09:45:41.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:40 smithi077 ceph-mon[30319]: mon.a calling monitor election 2024-09-10T09:45:41.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:40 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T09:45:41.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:40 smithi077 ceph-mon[30319]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:45:41.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:40 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T09:45:41.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:40 smithi077 ceph-mon[30319]: mon.c calling monitor election 2024-09-10T09:45:41.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:40 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T09:45:41.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:40 smithi077 ceph-mon[30319]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:45:41.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:40 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T09:45:41.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:40 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T09:45:41.191 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:40 smithi077 ceph-mon[30319]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-10T09:45:41.191 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:40 smithi077 ceph-mon[30319]: monmap epoch 2 2024-09-10T09:45:41.191 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:40 smithi077 ceph-mon[30319]: fsid 14a55226-6f59-11ef-bcea-c7b262605968 2024-09-10T09:45:41.191 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:40 smithi077 ceph-mon[30319]: last_changed 2024-09-10T09:45:35.685083+0000 2024-09-10T09:45:41.191 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:40 smithi077 ceph-mon[30319]: created 2024-09-10T09:44:03.655548+0000 2024-09-10T09:45:41.191 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:40 smithi077 ceph-mon[30319]: min_mon_release 19 (squid) 2024-09-10T09:45:41.191 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:40 smithi077 ceph-mon[30319]: election_strategy: 1 2024-09-10T09:45:41.191 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:40 smithi077 ceph-mon[30319]: 0: [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] mon.a 2024-09-10T09:45:41.191 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:40 smithi077 ceph-mon[30319]: 1: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.c 2024-09-10T09:45:41.191 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:40 smithi077 ceph-mon[30319]: fsmap 2024-09-10T09:45:41.191 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:40 smithi077 ceph-mon[30319]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T09:45:41.191 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:40 smithi077 ceph-mon[30319]: mgrmap e13: x(active, since 41s) 2024-09-10T09:45:41.192 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:40 smithi077 ceph-mon[30319]: overall HEALTH_OK 2024-09-10T09:45:41.192 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:40 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:41.192 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:40 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:41.192 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:40 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:41.192 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:40 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T09:45:41.192 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:40 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:45:41.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:41.393+0000 7f7ffe14e640 1 -- 172.21.15.134:0/1871531577 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7ff8106630 msgr2=0x7f7fe0005580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:41.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:41.393+0000 7f7ffe14e640 1 --2- 172.21.15.134:0/1871531577 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7ff8106630 0x7f7fe0005580 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7fe8009db0 tx=0x7f7fe802c140 comp rx=0 tx=0).stop 2024-09-10T09:45:41.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:41.393+0000 7f7ffe14e640 1 -- 172.21.15.134:0/1871531577 shutdown_connections 2024-09-10T09:45:41.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:41.393+0000 7f7ffe14e640 1 --2- 172.21.15.134:0/1871531577 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7ff8106630 0x7f7fe0005580 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:41.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:41.393+0000 7f7ffe14e640 1 -- 172.21.15.134:0/1871531577 >> 172.21.15.134:0/1871531577 conn(0x7f7ff80fc220 msgr2=0x7f7ff80fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:41.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:41.393+0000 7f7ffe14e640 1 -- 172.21.15.134:0/1871531577 shutdown_connections 2024-09-10T09:45:41.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:41.393+0000 7f7ffe14e640 1 -- 172.21.15.134:0/1871531577 wait complete. 2024-09-10T09:45:41.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:41.394+0000 7f7ffe14e640 1 Processor -- start 2024-09-10T09:45:41.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:41.394+0000 7f7ffe14e640 1 -- start start 2024-09-10T09:45:41.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:41.395+0000 7f7ffe14e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ff8106630 0x7f7ff81971d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:41.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:41.395+0000 7f7ffe14e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7ff8197710 0x7f7ff819baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:41.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:41.395+0000 7f7ffe14e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7ff8108670 con 0x7f7ff8197710 2024-09-10T09:45:41.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:41.395+0000 7f7ffe14e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7ff81084f0 con 0x7f7ff8106630 2024-09-10T09:45:41.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:41.395+0000 7f7ff77fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ff8106630 0x7f7ff81971d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:41.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:41.395+0000 7f7ff77fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ff8106630 0x7f7ff81971d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:33192/0 (socket says 172.21.15.134:33192) 2024-09-10T09:45:41.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:41.395+0000 7f7ff77fe640 1 -- 172.21.15.134:0/4231504166 learned_addr learned my addr 172.21.15.134:0/4231504166 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-10T09:45:41.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:41.395+0000 7f7ff77fe640 1 -- 172.21.15.134:0/4231504166 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ff8106630 msgr2=0x7f7ff81971d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-10T09:45:41.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:41.395+0000 7f7ff77fe640 1 -- 172.21.15.134:0/4231504166 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ff8106630 msgr2=0x7f7ff81971d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T09:45:41.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:41.395+0000 7f7ff77fe640 1 --2- 172.21.15.134:0/4231504166 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ff8106630 0x7f7ff81971d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T09:45:41.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:41.395+0000 7f7ff77fe640 1 --2- 172.21.15.134:0/4231504166 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ff8106630 0x7f7ff81971d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T09:45:41.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:41.395+0000 7f7ff6ffd640 1 --2- 172.21.15.134:0/4231504166 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7ff8197710 0x7f7ff819baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:41.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:41.396+0000 7f7ff6ffd640 1 -- 172.21.15.134:0/4231504166 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ff8106630 msgr2=0x7f7ff81971d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T09:45:41.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:41.396+0000 7f7ff6ffd640 1 --2- 172.21.15.134:0/4231504166 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ff8106630 0x7f7ff81971d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:41.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:41.396+0000 7f7ff6ffd640 1 -- 172.21.15.134:0/4231504166 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ff8197d30 con 0x7f7ff8197710 2024-09-10T09:45:41.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:41.396+0000 7f7ff6ffd640 1 --2- 172.21.15.134:0/4231504166 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7ff8197710 0x7f7ff819baa0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7fec00b3d0 tx=0x7f7fec00b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:41.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:41.396+0000 7f7ff4ff9640 1 -- 172.21.15.134:0/4231504166 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f7fec011630 con 0x7f7ff8197710 2024-09-10T09:45:41.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:41.396+0000 7f7ffe14e640 1 -- 172.21.15.134:0/4231504166 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ff819bfe0 con 0x7f7ff8197710 2024-09-10T09:45:41.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:41.396+0000 7f7ff4ff9640 1 -- 172.21.15.134:0/4231504166 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fec011cb0 con 0x7f7ff8197710 2024-09-10T09:45:41.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:41.396+0000 7f7ff4ff9640 1 -- 172.21.15.134:0/4231504166 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f7fec010df0 con 0x7f7ff8197710 2024-09-10T09:45:41.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:41.396+0000 7f7ffe14e640 1 -- 172.21.15.134:0/4231504166 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ff819c5a0 con 0x7f7ff8197710 2024-09-10T09:45:41.401 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:41.397+0000 7f7ff4ff9640 1 -- 172.21.15.134:0/4231504166 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f7fec018020 con 0x7f7ff8197710 2024-09-10T09:45:41.401 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:41.397+0000 7f7ff4ff9640 1 --2- 172.21.15.134:0/4231504166 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f7fe4038e00 0x7f7fe403b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:41.401 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:41.397+0000 7f7ffe14e640 1 -- 172.21.15.134:0/4231504166 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ff819c8b0 con 0x7f7ff8197710 2024-09-10T09:45:41.401 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:41.398+0000 7f7ff4ff9640 1 -- 172.21.15.134:0/4231504166 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f7fec04bbd0 con 0x7f7ff8197710 2024-09-10T09:45:41.401 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:41.398+0000 7f7ff77fe640 1 --2- 172.21.15.134:0/4231504166 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f7fe4038e00 0x7f7fe403b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:41.401 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:41.399+0000 7f7ff77fe640 1 --2- 172.21.15.134:0/4231504166 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f7fe4038e00 0x7f7fe403b2c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7fe8005b40 tx=0x7f7fe802e040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:41.404 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:41.403+0000 7f7ff4ff9640 1 -- 172.21.15.134:0/4231504166 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7fec01f020 con 0x7f7ff8197710 2024-09-10T09:45:41.638 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:41.636+0000 7f7ffe14e640 1 -- 172.21.15.134:0/4231504166 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f7ff8198560 con 0x7f7ff8197710 2024-09-10T09:45:41.638 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:41.637+0000 7f7ff4ff9640 1 -- 172.21.15.134:0/4231504166 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f7fec018310 con 0x7f7ff8197710 2024-09-10T09:45:41.639 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-10T09:45:41.639 INFO:teuthology.orchestra.run.smithi134.stdout:{"epoch":2,"fsid":"14a55226-6f59-11ef-bcea-c7b262605968","modified":"2024-09-10T09:45:35.685083Z","created":"2024-09-10T09:44:03.655548Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:3300","nonce":0},{"type":"v1","addr":"172.21.15.77:6789","nonce":0}]},"addr":"172.21.15.77:6789/0","public_addr":"172.21.15.77:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-10T09:45:41.639 INFO:teuthology.orchestra.run.smithi134.stderr:dumped monmap epoch 2 2024-09-10T09:45:41.641 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:41.639+0000 7f7ffe14e640 1 -- 172.21.15.134:0/4231504166 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f7fe4038e00 msgr2=0x7f7fe403b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:41.641 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:41.639+0000 7f7ffe14e640 1 --2- 172.21.15.134:0/4231504166 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f7fe4038e00 0x7f7fe403b2c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7fe8005b40 tx=0x7f7fe802e040 comp rx=0 tx=0).stop 2024-09-10T09:45:41.641 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:41.639+0000 7f7ffe14e640 1 -- 172.21.15.134:0/4231504166 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7ff8197710 msgr2=0x7f7ff819baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:41.641 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:41.639+0000 7f7ffe14e640 1 --2- 172.21.15.134:0/4231504166 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7ff8197710 0x7f7ff819baa0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7fec00b3d0 tx=0x7f7fec00b890 comp rx=0 tx=0).stop 2024-09-10T09:45:41.641 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:41.639+0000 7f7ffe14e640 1 -- 172.21.15.134:0/4231504166 shutdown_connections 2024-09-10T09:45:41.641 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:41.639+0000 7f7ffe14e640 1 --2- 172.21.15.134:0/4231504166 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f7fe4038e00 0x7f7fe403b2c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:41.641 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:41.639+0000 7f7ffe14e640 1 --2- 172.21.15.134:0/4231504166 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7ff8197710 0x7f7ff819baa0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:41.641 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:41.639+0000 7f7ffe14e640 1 --2- 172.21.15.134:0/4231504166 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ff8106630 0x7f7ff81971d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:41.641 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:41.639+0000 7f7ffe14e640 1 -- 172.21.15.134:0/4231504166 >> 172.21.15.134:0/4231504166 conn(0x7f7ff80fc220 msgr2=0x7f7ff806a830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:41.641 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:41.640+0000 7f7ffe14e640 1 -- 172.21.15.134:0/4231504166 shutdown_connections 2024-09-10T09:45:41.641 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:41.640+0000 7f7ffe14e640 1 -- 172.21.15.134:0/4231504166 wait complete. 2024-09-10T09:45:42.097 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:41 smithi134 ceph-mon[35674]: Deploying daemon mon.b on smithi120 2024-09-10T09:45:42.097 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:41 smithi134 ceph-mon[35674]: from='client.? 172.21.15.134:0/4231504166' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T09:45:42.097 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:41 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T09:45:42.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:41 smithi077 ceph-mon[30319]: Deploying daemon mon.b on smithi120 2024-09-10T09:45:42.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:41 smithi077 ceph-mon[30319]: from='client.? 172.21.15.134:0/4231504166' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T09:45:42.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:41 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T09:45:42.838 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:42 smithi077 ceph-mon[30319]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:45:43.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:42 smithi134 ceph-mon[35674]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:45:43.399 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-10T09:45:43.400 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T09:45:43.642 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T09:45:44.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:44.102+0000 7f08d215b640 1 -- 172.21.15.134:0/2741121214 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f08cc107660 msgr2=0x7f08cc107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:44.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:44.102+0000 7f08d215b640 1 --2- 172.21.15.134:0/2741121214 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f08cc107660 0x7f08cc107a40 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f08bc009880 tx=0x7f08bc02a5d0 comp rx=0 tx=0).stop 2024-09-10T09:45:44.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:44.102+0000 7f08d215b640 1 -- 172.21.15.134:0/2741121214 shutdown_connections 2024-09-10T09:45:44.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:44.102+0000 7f08d215b640 1 --2- 172.21.15.134:0/2741121214 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f08cc107660 0x7f08cc107a40 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:44.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:44.102+0000 7f08d215b640 1 -- 172.21.15.134:0/2741121214 >> 172.21.15.134:0/2741121214 conn(0x7f08cc0fd320 msgr2=0x7f08cc0ff740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:44.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:44.102+0000 7f08d215b640 1 -- 172.21.15.134:0/2741121214 shutdown_connections 2024-09-10T09:45:44.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:44.102+0000 7f08d215b640 1 -- 172.21.15.134:0/2741121214 wait complete. 2024-09-10T09:45:44.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:44.103+0000 7f08d215b640 1 Processor -- start 2024-09-10T09:45:44.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:44.103+0000 7f08d215b640 1 -- start start 2024-09-10T09:45:44.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:44.103+0000 7f08d215b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f08cc107660 0x7f08cc1973c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:44.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:44.103+0000 7f08d215b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f08cc197900 0x7f08cc19bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:44.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:44.103+0000 7f08d215b640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f08cc10bd20 con 0x7f08cc107660 2024-09-10T09:45:44.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:44.103+0000 7f08d215b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f08cc10bba0 con 0x7f08cc197900 2024-09-10T09:45:44.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:44.103+0000 7f08cb7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f08cc107660 0x7f08cc1973c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:44.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:44.103+0000 7f08cb7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f08cc107660 0x7f08cc1973c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.134:38628/0 (socket says 172.21.15.134:38628) 2024-09-10T09:45:44.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:44.103+0000 7f08cb7fe640 1 -- 172.21.15.134:0/3845871286 learned_addr learned my addr 172.21.15.134:0/3845871286 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-10T09:45:44.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:44.103+0000 7f08caffd640 1 --2- 172.21.15.134:0/3845871286 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f08cc197900 0x7f08cc19bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:44.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:44.103+0000 7f08caffd640 1 -- 172.21.15.134:0/3845871286 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f08cc197900 msgr2=0x7f08cc19bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-10T09:45:44.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:44.103+0000 7f08caffd640 1 -- 172.21.15.134:0/3845871286 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f08cc197900 msgr2=0x7f08cc19bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T09:45:44.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:44.103+0000 7f08caffd640 1 --2- 172.21.15.134:0/3845871286 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f08cc197900 0x7f08cc19bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T09:45:44.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:44.103+0000 7f08caffd640 1 --2- 172.21.15.134:0/3845871286 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f08cc197900 0x7f08cc19bc90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T09:45:44.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:44.104+0000 7f08cb7fe640 1 -- 172.21.15.134:0/3845871286 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f08cc197900 msgr2=0x7f08cc19bc90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T09:45:44.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:44.104+0000 7f08cb7fe640 1 --2- 172.21.15.134:0/3845871286 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f08cc197900 0x7f08cc19bc90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:44.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:44.104+0000 7f08cb7fe640 1 -- 172.21.15.134:0/3845871286 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08cc197f20 con 0x7f08cc107660 2024-09-10T09:45:44.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:44.104+0000 7f08cb7fe640 1 --2- 172.21.15.134:0/3845871286 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f08cc107660 0x7f08cc1973c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f08bc004650 tx=0x7f08bc032c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:44.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:44.104+0000 7f08c8ff9640 1 -- 172.21.15.134:0/3845871286 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f08bc035070 con 0x7f08cc107660 2024-09-10T09:45:44.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:44.104+0000 7f08d215b640 1 -- 172.21.15.134:0/3845871286 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08cc19c1d0 con 0x7f08cc107660 2024-09-10T09:45:44.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:44.104+0000 7f08c8ff9640 1 -- 172.21.15.134:0/3845871286 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08bc003ce0 con 0x7f08cc107660 2024-09-10T09:45:44.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:44.104+0000 7f08c8ff9640 1 -- 172.21.15.134:0/3845871286 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f08bc03bbd0 con 0x7f08cc107660 2024-09-10T09:45:44.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:44.104+0000 7f08d215b640 1 -- 172.21.15.134:0/3845871286 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08cc19c710 con 0x7f08cc107660 2024-09-10T09:45:44.109 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:44.105+0000 7f08d215b640 1 -- 172.21.15.134:0/3845871286 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08cc102da0 con 0x7f08cc107660 2024-09-10T09:45:44.109 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:44.105+0000 7f08c8ff9640 1 -- 172.21.15.134:0/3845871286 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f08bc042030 con 0x7f08cc107660 2024-09-10T09:45:44.109 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:44.106+0000 7f08c8ff9640 1 --2- 172.21.15.134:0/3845871286 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f08b4038e00 0x7f08b403b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:44.109 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:44.106+0000 7f08c8ff9640 1 -- 172.21.15.134:0/3845871286 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f08bc040070 con 0x7f08cc107660 2024-09-10T09:45:44.109 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:44.106+0000 7f08caffd640 1 --2- 172.21.15.134:0/3845871286 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f08b4038e00 0x7f08b403b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:44.109 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:44.107+0000 7f08caffd640 1 --2- 172.21.15.134:0/3845871286 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f08b4038e00 0x7f08b403b2c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f08c0006fd0 tx=0x7f08c0008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:44.112 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:44.111+0000 7f08c8ff9640 1 -- 172.21.15.134:0/3845871286 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f08bc03bd70 con 0x7f08cc107660 2024-09-10T09:45:44.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:44.338+0000 7f08d215b640 1 -- 172.21.15.134:0/3845871286 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f08cc19c9c0 con 0x7f08cc107660 2024-09-10T09:45:44.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:44.339+0000 7f08c8ff9640 1 -- 172.21.15.134:0/3845871286 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f08bc0438e0 con 0x7f08cc107660 2024-09-10T09:45:44.340 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-10T09:45:44.340 INFO:teuthology.orchestra.run.smithi134.stdout:{"epoch":2,"fsid":"14a55226-6f59-11ef-bcea-c7b262605968","modified":"2024-09-10T09:45:35.685083Z","created":"2024-09-10T09:44:03.655548Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:3300","nonce":0},{"type":"v1","addr":"172.21.15.77:6789","nonce":0}]},"addr":"172.21.15.77:6789/0","public_addr":"172.21.15.77:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-10T09:45:44.341 INFO:teuthology.orchestra.run.smithi134.stderr:dumped monmap epoch 2 2024-09-10T09:45:44.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:44.342+0000 7f08d215b640 1 -- 172.21.15.134:0/3845871286 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f08b4038e00 msgr2=0x7f08b403b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:44.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:44.342+0000 7f08d215b640 1 --2- 172.21.15.134:0/3845871286 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f08b4038e00 0x7f08b403b2c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f08c0006fd0 tx=0x7f08c0008040 comp rx=0 tx=0).stop 2024-09-10T09:45:44.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:44.342+0000 7f08d215b640 1 -- 172.21.15.134:0/3845871286 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f08cc107660 msgr2=0x7f08cc1973c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:44.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:44.342+0000 7f08d215b640 1 --2- 172.21.15.134:0/3845871286 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f08cc107660 0x7f08cc1973c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f08bc004650 tx=0x7f08bc032c50 comp rx=0 tx=0).stop 2024-09-10T09:45:44.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:44.342+0000 7f08d215b640 1 -- 172.21.15.134:0/3845871286 shutdown_connections 2024-09-10T09:45:44.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:44.342+0000 7f08d215b640 1 --2- 172.21.15.134:0/3845871286 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f08b4038e00 0x7f08b403b2c0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:44.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:44.342+0000 7f08d215b640 1 --2- 172.21.15.134:0/3845871286 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f08cc197900 0x7f08cc19bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:44.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:44.342+0000 7f08d215b640 1 --2- 172.21.15.134:0/3845871286 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f08cc107660 0x7f08cc1973c0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:44.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:44.342+0000 7f08d215b640 1 -- 172.21.15.134:0/3845871286 >> 172.21.15.134:0/3845871286 conn(0x7f08cc0fd320 msgr2=0x7f08cc0fee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:44.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:44.342+0000 7f08d215b640 1 -- 172.21.15.134:0/3845871286 shutdown_connections 2024-09-10T09:45:44.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:44.342+0000 7f08d215b640 1 -- 172.21.15.134:0/3845871286 wait complete. 2024-09-10T09:45:44.461 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 podman[34751]: 2024-09-10 09:45:44.439228553 +0000 UTC m=+0.329497704 container start fc48b4c41882bdb7707c2ee05be81d6817e86874c6152a0c3f947b34ffc360b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-mon-b, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-10T09:45:44.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: set uid:gid to 167:167 (ceph:ceph) 2024-09-10T09:45:44.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable), process ceph-mon, pid 2 2024-09-10T09:45:44.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: pidfile_write: ignore empty --pid-file 2024-09-10T09:45:44.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: load: jerasure load: lrc 2024-09-10T09:45:44.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: RocksDB version: 7.9.2 2024-09-10T09:45:44.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Git sha 0 2024-09-10T09:45:44.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Compile date 2024-09-09 20:43:47 2024-09-10T09:45:44.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: DB SUMMARY 2024-09-10T09:45:44.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: DB Session ID: U4WJGNARQ1KB2D1QQKKM 2024-09-10T09:45:44.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: CURRENT file: CURRENT 2024-09-10T09:45:44.925 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: IDENTITY file: IDENTITY 2024-09-10T09:45:44.925 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-10T09:45:44.925 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-09-10T09:45:44.925 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-09-10T09:45:44.925 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.error_if_exists: 0 2024-09-10T09:45:44.925 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.create_if_missing: 0 2024-09-10T09:45:44.925 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.paranoid_checks: 1 2024-09-10T09:45:44.925 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-10T09:45:44.925 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-10T09:45:44.925 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-10T09:45:44.925 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.env: 0x55bebedeb940 2024-09-10T09:45:44.925 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.fs: PosixFileSystem 2024-09-10T09:45:44.925 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.info_log: 0x55bec03376e0 2024-09-10T09:45:44.926 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.max_file_opening_threads: 16 2024-09-10T09:45:44.926 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.statistics: (nil) 2024-09-10T09:45:44.926 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.use_fsync: 0 2024-09-10T09:45:44.926 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.max_log_file_size: 0 2024-09-10T09:45:44.926 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-10T09:45:44.926 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-10T09:45:44.926 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.keep_log_file_num: 1000 2024-09-10T09:45:44.926 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.recycle_log_file_num: 0 2024-09-10T09:45:44.926 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.allow_fallocate: 1 2024-09-10T09:45:44.926 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.allow_mmap_reads: 0 2024-09-10T09:45:44.926 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.allow_mmap_writes: 0 2024-09-10T09:45:44.926 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.use_direct_reads: 0 2024-09-10T09:45:44.926 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-10T09:45:44.927 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.create_missing_column_families: 0 2024-09-10T09:45:44.927 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.db_log_dir: 2024-09-10T09:45:44.927 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.wal_dir: 2024-09-10T09:45:44.927 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-10T09:45:44.929 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-10T09:45:44.929 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-10T09:45:44.929 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-10T09:45:44.929 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-10T09:45:44.929 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-10T09:45:44.929 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.advise_random_on_open: 1 2024-09-10T09:45:44.929 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.db_write_buffer_size: 0 2024-09-10T09:45:44.929 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.write_buffer_manager: 0x55bec0346dc0 2024-09-10T09:45:44.929 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-10T09:45:44.929 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-10T09:45:44.929 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-10T09:45:44.930 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.rate_limiter: (nil) 2024-09-10T09:45:44.930 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-10T09:45:44.930 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.wal_recovery_mode: 2 2024-09-10T09:45:44.930 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.enable_thread_tracking: 0 2024-09-10T09:45:44.930 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.enable_pipelined_write: 0 2024-09-10T09:45:44.930 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.unordered_write: 0 2024-09-10T09:45:44.930 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-10T09:45:44.930 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-10T09:45:44.930 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-10T09:45:44.930 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-10T09:45:44.930 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.row_cache: None 2024-09-10T09:45:44.930 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.wal_filter: None 2024-09-10T09:45:44.931 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-10T09:45:44.931 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.allow_ingest_behind: 0 2024-09-10T09:45:44.931 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.two_write_queues: 0 2024-09-10T09:45:44.931 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.manual_wal_flush: 0 2024-09-10T09:45:44.931 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.wal_compression: 0 2024-09-10T09:45:44.931 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.atomic_flush: 0 2024-09-10T09:45:44.931 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-10T09:45:44.931 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-10T09:45:44.931 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-10T09:45:44.931 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.log_readahead_size: 0 2024-09-10T09:45:44.931 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-10T09:45:44.931 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.best_efforts_recovery: 0 2024-09-10T09:45:44.932 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-10T09:45:44.932 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-10T09:45:44.932 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.allow_data_in_errors: 0 2024-09-10T09:45:44.932 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.db_host_id: __hostname__ 2024-09-10T09:45:44.932 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-10T09:45:44.932 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.max_background_jobs: 2 2024-09-10T09:45:44.932 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.max_background_compactions: -1 2024-09-10T09:45:44.932 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.max_subcompactions: 1 2024-09-10T09:45:44.932 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-10T09:45:44.932 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-10T09:45:44.932 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-10T09:45:44.932 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.max_total_wal_size: 0 2024-09-10T09:45:44.932 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-10T09:45:44.933 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-10T09:45:44.933 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-10T09:45:44.933 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-10T09:45:44.933 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.max_open_files: -1 2024-09-10T09:45:44.933 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.bytes_per_sync: 0 2024-09-10T09:45:44.933 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-10T09:45:44.933 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-10T09:45:44.933 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.compaction_readahead_size: 0 2024-09-10T09:45:44.933 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.max_background_flushes: -1 2024-09-10T09:45:44.933 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Compression algorithms supported: 2024-09-10T09:45:44.933 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: kZSTD supported: 0 2024-09-10T09:45:44.933 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: kXpressCompression supported: 0 2024-09-10T09:45:44.933 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: kBZip2Compression supported: 0 2024-09-10T09:45:44.934 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-10T09:45:44.934 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: kLZ4Compression supported: 1 2024-09-10T09:45:44.934 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: kZlibCompression supported: 1 2024-09-10T09:45:44.934 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: kLZ4HCCompression supported: 1 2024-09-10T09:45:44.934 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: kSnappyCompression supported: 1 2024-09-10T09:45:44.934 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-10T09:45:44.934 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-10T09:45:44.934 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-09-10T09:45:44.934 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-10T09:45:44.934 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-10T09:45:44.934 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.merge_operator: 2024-09-10T09:45:44.934 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.compaction_filter: None 2024-09-10T09:45:44.934 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.compaction_filter_factory: None 2024-09-10T09:45:44.934 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.sst_partitioner_factory: None 2024-09-10T09:45:44.935 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-10T09:45:44.935 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-10T09:45:44.935 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55bec0337320) 2024-09-10T09:45:44.935 INFO:journalctl@ceph.mon.b.smithi120.stdout: cache_index_and_filter_blocks: 1 2024-09-10T09:45:44.935 INFO:journalctl@ceph.mon.b.smithi120.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-10T09:45:44.935 INFO:journalctl@ceph.mon.b.smithi120.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-10T09:45:44.935 INFO:journalctl@ceph.mon.b.smithi120.stdout: pin_top_level_index_and_filter: 1 2024-09-10T09:45:44.935 INFO:journalctl@ceph.mon.b.smithi120.stdout: index_type: 0 2024-09-10T09:45:44.935 INFO:journalctl@ceph.mon.b.smithi120.stdout: data_block_index_type: 0 2024-09-10T09:45:44.935 INFO:journalctl@ceph.mon.b.smithi120.stdout: index_shortening: 1 2024-09-10T09:45:44.935 INFO:journalctl@ceph.mon.b.smithi120.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-10T09:45:44.935 INFO:journalctl@ceph.mon.b.smithi120.stdout: checksum: 4 2024-09-10T09:45:44.935 INFO:journalctl@ceph.mon.b.smithi120.stdout: no_block_cache: 0 2024-09-10T09:45:44.936 INFO:journalctl@ceph.mon.b.smithi120.stdout: block_cache: 0x55bec032b1f0 2024-09-10T09:45:44.936 INFO:journalctl@ceph.mon.b.smithi120.stdout: block_cache_name: BinnedLRUCache 2024-09-10T09:45:44.936 INFO:journalctl@ceph.mon.b.smithi120.stdout: block_cache_options: 2024-09-10T09:45:44.936 INFO:journalctl@ceph.mon.b.smithi120.stdout: capacity : 536870912 2024-09-10T09:45:44.936 INFO:journalctl@ceph.mon.b.smithi120.stdout: num_shard_bits : 4 2024-09-10T09:45:44.936 INFO:journalctl@ceph.mon.b.smithi120.stdout: strict_capacity_limit : 0 2024-09-10T09:45:44.936 INFO:journalctl@ceph.mon.b.smithi120.stdout: high_pri_pool_ratio: 0.000 2024-09-10T09:45:44.936 INFO:journalctl@ceph.mon.b.smithi120.stdout: block_cache_compressed: (nil) 2024-09-10T09:45:44.936 INFO:journalctl@ceph.mon.b.smithi120.stdout: persistent_cache: (nil) 2024-09-10T09:45:44.936 INFO:journalctl@ceph.mon.b.smithi120.stdout: block_size: 4096 2024-09-10T09:45:44.936 INFO:journalctl@ceph.mon.b.smithi120.stdout: block_size_deviation: 10 2024-09-10T09:45:44.936 INFO:journalctl@ceph.mon.b.smithi120.stdout: block_restart_interval: 16 2024-09-10T09:45:44.937 INFO:journalctl@ceph.mon.b.smithi120.stdout: index_block_restart_interval: 1 2024-09-10T09:45:44.937 INFO:journalctl@ceph.mon.b.smithi120.stdout: metadata_block_size: 4096 2024-09-10T09:45:44.937 INFO:journalctl@ceph.mon.b.smithi120.stdout: partition_filters: 0 2024-09-10T09:45:44.937 INFO:journalctl@ceph.mon.b.smithi120.stdout: use_delta_encoding: 1 2024-09-10T09:45:44.937 INFO:journalctl@ceph.mon.b.smithi120.stdout: filter_policy: bloomfilter 2024-09-10T09:45:44.937 INFO:journalctl@ceph.mon.b.smithi120.stdout: whole_key_filtering: 1 2024-09-10T09:45:44.937 INFO:journalctl@ceph.mon.b.smithi120.stdout: verify_compression: 0 2024-09-10T09:45:44.937 INFO:journalctl@ceph.mon.b.smithi120.stdout: read_amp_bytes_per_bit: 0 2024-09-10T09:45:44.937 INFO:journalctl@ceph.mon.b.smithi120.stdout: format_version: 5 2024-09-10T09:45:44.937 INFO:journalctl@ceph.mon.b.smithi120.stdout: enable_index_compression: 1 2024-09-10T09:45:44.937 INFO:journalctl@ceph.mon.b.smithi120.stdout: block_align: 0 2024-09-10T09:45:44.937 INFO:journalctl@ceph.mon.b.smithi120.stdout: max_auto_readahead_size: 262144 2024-09-10T09:45:44.937 INFO:journalctl@ceph.mon.b.smithi120.stdout: prepopulate_block_cache: 0 2024-09-10T09:45:44.938 INFO:journalctl@ceph.mon.b.smithi120.stdout: initial_auto_readahead_size: 8192 2024-09-10T09:45:44.938 INFO:journalctl@ceph.mon.b.smithi120.stdout: num_file_reads_for_auto_readahead: 2 2024-09-10T09:45:44.938 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.write_buffer_size: 33554432 2024-09-10T09:45:44.938 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.max_write_buffer_number: 2 2024-09-10T09:45:44.938 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.compression: NoCompression 2024-09-10T09:45:44.938 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.bottommost_compression: Disabled 2024-09-10T09:45:44.938 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.prefix_extractor: nullptr 2024-09-10T09:45:44.938 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-10T09:45:44.938 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.num_levels: 7 2024-09-10T09:45:44.938 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-10T09:45:44.938 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-10T09:45:44.938 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-10T09:45:44.938 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-10T09:45:44.938 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-10T09:45:44.939 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-10T09:45:44.939 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-10T09:45:44.939 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-10T09:45:44.939 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-10T09:45:44.939 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-10T09:45:44.939 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-10T09:45:44.939 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-10T09:45:44.939 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-10T09:45:44.939 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.compression_opts.level: 32767 2024-09-10T09:45:44.939 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.compression_opts.strategy: 0 2024-09-10T09:45:44.939 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-10T09:45:44.939 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-10T09:45:44.939 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-10T09:45:44.940 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-10T09:45:44.940 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.compression_opts.enabled: false 2024-09-10T09:45:44.940 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-10T09:45:44.940 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-10T09:45:44.940 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-10T09:45:44.940 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-10T09:45:44.940 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.target_file_size_base: 67108864 2024-09-10T09:45:44.940 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-10T09:45:44.940 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-10T09:45:44.940 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-10T09:45:44.941 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-10T09:45:44.941 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-10T09:45:44.941 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-10T09:45:44.941 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-10T09:45:44.941 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-10T09:45:44.941 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-10T09:45:44.941 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-10T09:45:44.941 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-10T09:45:44.941 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-10T09:45:44.941 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-10T09:45:44.941 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-10T09:45:44.941 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.arena_block_size: 1048576 2024-09-10T09:45:44.941 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-10T09:45:44.942 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-10T09:45:44.942 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.disable_auto_compactions: 0 2024-09-10T09:45:44.942 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-10T09:45:44.942 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-10T09:45:44.942 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-10T09:45:44.942 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-10T09:45:44.942 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-10T09:45:44.942 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-10T09:45:44.942 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-10T09:45:44.942 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-10T09:45:44.942 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-10T09:45:44.942 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-10T09:45:44.942 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-10T09:45:44.942 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.inplace_update_support: 0 2024-09-10T09:45:44.943 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-10T09:45:44.943 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-10T09:45:44.943 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-10T09:45:44.943 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-10T09:45:44.943 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.bloom_locality: 0 2024-09-10T09:45:44.943 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.max_successive_merges: 0 2024-09-10T09:45:44.943 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-10T09:45:44.943 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.paranoid_file_checks: 0 2024-09-10T09:45:44.943 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.force_consistency_checks: 1 2024-09-10T09:45:44.943 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.report_bg_io_stats: 0 2024-09-10T09:45:44.943 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.ttl: 2592000 2024-09-10T09:45:44.943 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-10T09:45:44.944 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-10T09:45:44.944 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-10T09:45:44.944 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.enable_blob_files: false 2024-09-10T09:45:44.944 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.min_blob_size: 0 2024-09-10T09:45:44.944 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.blob_file_size: 268435456 2024-09-10T09:45:44.944 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-10T09:45:44.944 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-10T09:45:44.944 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-10T09:45:44.944 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-10T09:45:44.944 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-10T09:45:44.944 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.blob_file_starting_level: 0 2024-09-10T09:45:44.944 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-10T09:45:44.944 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-10T09:45:44.945 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-10T09:45:44.945 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: d2547f74-d944-4f6f-b666-2b1e1a6650f9 2024-09-10T09:45:44.945 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725961544466484, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-10T09:45:44.945 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-10T09:45:44.945 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725961544466872, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1725961544, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d2547f74-d944-4f6f-b666-2b1e1a6650f9", "db_session_id": "U4WJGNARQ1KB2D1QQKKM", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-10T09:45:44.945 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725961544466963, "job": 1, "event": "recovery_finished"} 2024-09-10T09:45:44.945 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-10T09:45:44.945 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-10T09:45:44.945 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55bec0358e00 2024-09-10T09:45:44.945 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: DB pointer 0x55bec0454000 2024-09-10T09:45:44.945 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-10T09:45:44.945 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-10T09:45:44.945 INFO:journalctl@ceph.mon.b.smithi120.stdout: ** DB Stats ** 2024-09-10T09:45:44.946 INFO:journalctl@ceph.mon.b.smithi120.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T09:45:44.946 INFO:journalctl@ceph.mon.b.smithi120.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-10T09:45:44.946 INFO:journalctl@ceph.mon.b.smithi120.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T09:45:44.946 INFO:journalctl@ceph.mon.b.smithi120.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T09:45:44.946 INFO:journalctl@ceph.mon.b.smithi120.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-10T09:45:44.946 INFO:journalctl@ceph.mon.b.smithi120.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T09:45:44.946 INFO:journalctl@ceph.mon.b.smithi120.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T09:45:44.946 INFO:journalctl@ceph.mon.b.smithi120.stdout: 2024-09-10T09:45:44.946 INFO:journalctl@ceph.mon.b.smithi120.stdout: ** Compaction Stats [default] ** 2024-09-10T09:45:44.946 INFO:journalctl@ceph.mon.b.smithi120.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-10T09:45:44.946 INFO:journalctl@ceph.mon.b.smithi120.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-10T09:45:44.946 INFO:journalctl@ceph.mon.b.smithi120.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.5 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-10T09:45:44.946 INFO:journalctl@ceph.mon.b.smithi120.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.5 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-10T09:45:44.946 INFO:journalctl@ceph.mon.b.smithi120.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.5 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-10T09:45:44.947 INFO:journalctl@ceph.mon.b.smithi120.stdout: 2024-09-10T09:45:44.947 INFO:journalctl@ceph.mon.b.smithi120.stdout: ** Compaction Stats [default] ** 2024-09-10T09:45:44.947 INFO:journalctl@ceph.mon.b.smithi120.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-10T09:45:44.947 INFO:journalctl@ceph.mon.b.smithi120.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T09:45:44.947 INFO:journalctl@ceph.mon.b.smithi120.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.5 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-10T09:45:44.947 INFO:journalctl@ceph.mon.b.smithi120.stdout: 2024-09-10T09:45:44.947 INFO:journalctl@ceph.mon.b.smithi120.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-10T09:45:44.947 INFO:journalctl@ceph.mon.b.smithi120.stdout: 2024-09-10T09:45:44.947 INFO:journalctl@ceph.mon.b.smithi120.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T09:45:44.947 INFO:journalctl@ceph.mon.b.smithi120.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-10T09:45:44.947 INFO:journalctl@ceph.mon.b.smithi120.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-10T09:45:44.947 INFO:journalctl@ceph.mon.b.smithi120.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-10T09:45:44.947 INFO:journalctl@ceph.mon.b.smithi120.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-10T09:45:44.948 INFO:journalctl@ceph.mon.b.smithi120.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-10T09:45:44.948 INFO:journalctl@ceph.mon.b.smithi120.stdout: Cumulative compaction: 0.00 GB write, 0.65 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T09:45:44.948 INFO:journalctl@ceph.mon.b.smithi120.stdout: Interval compaction: 0.00 GB write, 0.65 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T09:45:44.948 INFO:journalctl@ceph.mon.b.smithi120.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-10T09:45:44.948 INFO:journalctl@ceph.mon.b.smithi120.stdout: Block cache BinnedLRUCache@0x55bec032b1f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-10T09:45:44.948 INFO:journalctl@ceph.mon.b.smithi120.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-10T09:45:44.948 INFO:journalctl@ceph.mon.b.smithi120.stdout: 2024-09-10T09:45:44.948 INFO:journalctl@ceph.mon.b.smithi120.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-10T09:45:44.948 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-09-10T09:45:44.948 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: using public_addr v2:172.21.15.120:0/0 -> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] 2024-09-10T09:45:44.948 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: starting mon.b rank -1 at public addrs [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] at bind addrs [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 14a55226-6f59-11ef-bcea-c7b262605968 2024-09-10T09:45:44.948 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: mon.b@-1(???) e0 preinit fsid 14a55226-6f59-11ef-bcea-c7b262605968 2024-09-10T09:45:44.948 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: mon.b@-1(synchronizing).mds e1 new map 2024-09-10T09:45:44.949 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: mon.b@-1(synchronizing).mds e1 print_map 2024-09-10T09:45:44.949 INFO:journalctl@ceph.mon.b.smithi120.stdout: e1 2024-09-10T09:45:44.949 INFO:journalctl@ceph.mon.b.smithi120.stdout: btime 2024-09-10T09:44:07:359254+0000 2024-09-10T09:45:44.949 INFO:journalctl@ceph.mon.b.smithi120.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T09:45:44.949 INFO:journalctl@ceph.mon.b.smithi120.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T09:45:44.949 INFO:journalctl@ceph.mon.b.smithi120.stdout: legacy client fscid: -1 2024-09-10T09:45:44.949 INFO:journalctl@ceph.mon.b.smithi120.stdout: 2024-09-10T09:45:44.949 INFO:journalctl@ceph.mon.b.smithi120.stdout: No filesystems configured 2024-09-10T09:45:44.949 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-10T09:45:44.949 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-10T09:45:44.949 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-10T09:45:44.949 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-10T09:45:44.949 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-10T09:45:44.950 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-10T09:45:44.950 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-10T09:45:44.950 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T09:45:44.950 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T09:45:44.950 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T09:45:44.950 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T09:45:44.950 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:45:44.950 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:44.950 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:44.950 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:44.950 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:44.950 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi120", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:45:44.950 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:45:44.951 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:45:44.951 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi134", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:45:44.951 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: Updating smithi120:/etc/ceph/ceph.conf 2024-09-10T09:45:44.951 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: Updating smithi120:/var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T09:45:44.951 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:44.951 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:44.951 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:44.951 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: Deploying cephadm binary to smithi134 2024-09-10T09:45:44.951 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: Updating smithi120:/etc/ceph/ceph.client.admin.keyring 2024-09-10T09:45:44.951 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: Updating smithi120:/var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T09:45:44.951 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:45:44.951 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:44.951 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: Added host smithi134 2024-09-10T09:45:44.951 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:45:44.952 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:44.952 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:45:44.952 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:44.952 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:44.952 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T09:45:44.952 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:45:44.952 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/3142331239' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-10T09:45:44.952 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:44.952 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:44.952 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:44.952 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:44.952 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi134", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:45:44.952 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:45:44.953 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:45:44.953 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: Updating smithi134:/etc/ceph/ceph.conf 2024-09-10T09:45:44.953 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/3142331239' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-10T09:45:44.953 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T09:45:44.953 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: Updating smithi134:/var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T09:45:44.953 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:45:44.953 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: Updating smithi134:/etc/ceph/ceph.client.admin.keyring 2024-09-10T09:45:44.953 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:44.953 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:44.953 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:44.956 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: Updating smithi134:/var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T09:45:44.956 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:44.956 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:45:44.956 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:45:44.956 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:45:44.956 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:44.956 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T09:45:44.956 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:45:44.956 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi077:172.21.15.77=a;smithi120:172.21.15.120=b;smithi134:172.21.15.134=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:45:44.957 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: Saving service mon spec with placement smithi077:172.21.15.77=a;smithi120:172.21.15.120=b;smithi134:172.21.15.134=c;count:3 2024-09-10T09:45:44.957 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: Deploying daemon mon.c on smithi134 2024-09-10T09:45:44.957 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:45:44.957 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:45:44.957 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: from='client.? 172.21.15.134:0/971624609' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T09:45:44.957 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:45:44.957 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T09:45:44.957 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T09:45:44.958 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: mon.a calling monitor election 2024-09-10T09:45:44.958 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T09:45:44.958 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:45:44.958 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T09:45:44.958 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: mon.c calling monitor election 2024-09-10T09:45:44.958 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T09:45:44.958 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:45:44.958 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T09:45:44.958 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T09:45:44.958 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-10T09:45:44.958 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: monmap epoch 2 2024-09-10T09:45:44.958 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: fsid 14a55226-6f59-11ef-bcea-c7b262605968 2024-09-10T09:45:44.958 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: last_changed 2024-09-10T09:45:35.685083+0000 2024-09-10T09:45:44.959 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: created 2024-09-10T09:44:03.655548+0000 2024-09-10T09:45:44.959 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: min_mon_release 19 (squid) 2024-09-10T09:45:44.959 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: election_strategy: 1 2024-09-10T09:45:44.959 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: 0: [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] mon.a 2024-09-10T09:45:44.959 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: 1: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.c 2024-09-10T09:45:44.959 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: fsmap 2024-09-10T09:45:44.959 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T09:45:44.959 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: mgrmap e13: x(active, since 41s) 2024-09-10T09:45:44.959 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: overall HEALTH_OK 2024-09-10T09:45:44.959 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:44.959 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:44.959 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:44.960 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T09:45:44.960 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:45:44.960 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: Deploying daemon mon.b on smithi120 2024-09-10T09:45:44.960 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: from='client.? 172.21.15.134:0/4231504166' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T09:45:44.960 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T09:45:44.960 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:45:44.960 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 ceph-mon[34770]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-10T09:45:44.960 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 bash[34751]: fc48b4c41882bdb7707c2ee05be81d6817e86874c6152a0c3f947b34ffc360b3 2024-09-10T09:45:44.960 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:44 smithi120 systemd[1]: Started Ceph mon.b for 14a55226-6f59-11ef-bcea-c7b262605968. 2024-09-10T09:45:45.906 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-10T09:45:45.906 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T09:45:46.155 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T09:45:49.861 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:49.858+0000 7f19408f5640 1 -- 172.21.15.134:0/2871184142 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f193c1066c0 msgr2=0x7f1924005580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:49.861 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:49.858+0000 7f19408f5640 1 --2- 172.21.15.134:0/2871184142 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f193c1066c0 0x7f1924005580 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f192c011c50 tx=0x7f192c02be50 comp rx=0 tx=0).stop 2024-09-10T09:45:49.861 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:49.858+0000 7f19408f5640 1 -- 172.21.15.134:0/2871184142 shutdown_connections 2024-09-10T09:45:49.861 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:49.858+0000 7f19408f5640 1 --2- 172.21.15.134:0/2871184142 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f193c1066c0 0x7f1924005580 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:49.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:49.858+0000 7f19408f5640 1 -- 172.21.15.134:0/2871184142 >> 172.21.15.134:0/2871184142 conn(0x7f193c0fb9a0 msgr2=0x7f193c0fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:49.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:49.859+0000 7f19408f5640 1 -- 172.21.15.134:0/2871184142 shutdown_connections 2024-09-10T09:45:49.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:49.859+0000 7f19408f5640 1 -- 172.21.15.134:0/2871184142 wait complete. 2024-09-10T09:45:49.862 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:49 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T09:45:49.863 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:49 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:45:49.863 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:49 smithi134 ceph-mon[35674]: mon.c calling monitor election 2024-09-10T09:45:49.863 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:49 smithi134 ceph-mon[35674]: mon.a calling monitor election 2024-09-10T09:45:49.863 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:49 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T09:45:49.863 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:49 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:45:49.864 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:49 smithi134 ceph-mon[35674]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:45:49.864 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:49 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:45:49.864 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:49 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:45:49.864 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:49 smithi134 ceph-mon[35674]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:45:49.865 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:49 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:45:49.865 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:49 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:45:49.865 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:49 smithi134 ceph-mon[35674]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-10T09:45:49.865 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:49 smithi134 ceph-mon[35674]: monmap epoch 3 2024-09-10T09:45:49.865 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:49 smithi134 ceph-mon[35674]: fsid 14a55226-6f59-11ef-bcea-c7b262605968 2024-09-10T09:45:49.865 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:49 smithi134 ceph-mon[35674]: last_changed 2024-09-10T09:45:44.490735+0000 2024-09-10T09:45:49.866 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:49 smithi134 ceph-mon[35674]: created 2024-09-10T09:44:03.655548+0000 2024-09-10T09:45:49.866 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:49 smithi134 ceph-mon[35674]: min_mon_release 19 (squid) 2024-09-10T09:45:49.866 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:49 smithi134 ceph-mon[35674]: election_strategy: 1 2024-09-10T09:45:49.866 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:49 smithi134 ceph-mon[35674]: 0: [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] mon.a 2024-09-10T09:45:49.866 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:49 smithi134 ceph-mon[35674]: 1: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.c 2024-09-10T09:45:49.866 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:49 smithi134 ceph-mon[35674]: 2: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-09-10T09:45:49.867 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:49 smithi134 ceph-mon[35674]: fsmap 2024-09-10T09:45:49.867 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:49 smithi134 ceph-mon[35674]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T09:45:49.867 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:49 smithi134 ceph-mon[35674]: mgrmap e13: x(active, since 49s) 2024-09-10T09:45:49.867 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:49 smithi134 ceph-mon[35674]: overall HEALTH_OK 2024-09-10T09:45:49.867 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:49 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:49.867 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:49 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:49.868 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:49 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:49.868 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:49 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:49.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:49.859+0000 7f19408f5640 1 Processor -- start 2024-09-10T09:45:49.873 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:49.859+0000 7f19408f5640 1 -- start start 2024-09-10T09:45:49.873 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:49.860+0000 7f19408f5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f193c1066c0 0x7f193c102200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:49.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:49.860+0000 7f19408f5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f193c102740 0x7f193c102ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:49.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:49.860+0000 7f19408f5640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f193c1a3510 0x7f193c1a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:49.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:49.860+0000 7f19408f5640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f193c108690 con 0x7f193c1066c0 2024-09-10T09:45:49.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:49.860+0000 7f19408f5640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f193c108510 con 0x7f193c1a3510 2024-09-10T09:45:49.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:49.860+0000 7f19408f5640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f193c108810 con 0x7f193c102740 2024-09-10T09:45:49.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:49.860+0000 7f1939d9b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f193c102740 0x7f193c102ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:49.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:49.860+0000 7f1939d9b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f193c102740 0x7f193c102ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:34676/0 (socket says 172.21.15.134:34676) 2024-09-10T09:45:49.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:49.860+0000 7f1939d9b640 1 -- 172.21.15.134:0/3500218316 learned_addr learned my addr 172.21.15.134:0/3500218316 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-10T09:45:49.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:49.860+0000 7f193ad9d640 1 --2- 172.21.15.134:0/3500218316 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f193c1a3510 0x7f193c1a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:49.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:49.860+0000 7f1939d9b640 1 -- 172.21.15.134:0/3500218316 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f193c102740 msgr2=0x7f193c102ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-10T09:45:49.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:49.860+0000 7f1939d9b640 1 -- 172.21.15.134:0/3500218316 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f193c102740 msgr2=0x7f193c102ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T09:45:49.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:49.860+0000 7f1939d9b640 1 --2- 172.21.15.134:0/3500218316 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f193c102740 0x7f193c102ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T09:45:49.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:49.860+0000 7f1939d9b640 1 --2- 172.21.15.134:0/3500218316 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f193c102740 0x7f193c102ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T09:45:49.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:49.860+0000 7f193a59c640 1 --2- 172.21.15.134:0/3500218316 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f193c1066c0 0x7f193c102200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:49.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:49.860+0000 7f193ad9d640 1 -- 172.21.15.134:0/3500218316 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f193c1a3510 msgr2=0x7f193c1a5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-09-10T09:45:49.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:49.860+0000 7f193ad9d640 1 -- 172.21.15.134:0/3500218316 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f193c1a3510 msgr2=0x7f193c1a5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T09:45:49.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:49.860+0000 7f193ad9d640 1 --2- 172.21.15.134:0/3500218316 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f193c1a3510 0x7f193c1a5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T09:45:49.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:49.860+0000 7f193ad9d640 1 --2- 172.21.15.134:0/3500218316 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f193c1a3510 0x7f193c1a5900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T09:45:49.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:49.860+0000 7f193a59c640 1 -- 172.21.15.134:0/3500218316 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f193c102740 msgr2=0x7f193c102ba0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T09:45:49.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:49.860+0000 7f193a59c640 1 --2- 172.21.15.134:0/3500218316 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f193c102740 0x7f193c102ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:49.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:49.860+0000 7f193a59c640 1 -- 172.21.15.134:0/3500218316 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f193c1a3510 msgr2=0x7f193c1a5900 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T09:45:49.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:49.860+0000 7f193a59c640 1 --2- 172.21.15.134:0/3500218316 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f193c1a3510 0x7f193c1a5900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:49.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:49.860+0000 7f193a59c640 1 -- 172.21.15.134:0/3500218316 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f193c1a5fd0 con 0x7f193c1066c0 2024-09-10T09:45:49.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:49.861+0000 7f193a59c640 1 --2- 172.21.15.134:0/3500218316 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f193c1066c0 0x7f193c102200 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f192c011c50 tx=0x7f192c02cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:49.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:49.861+0000 7f192b7fe640 1 -- 172.21.15.134:0/3500218316 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f192c032040 con 0x7f193c1066c0 2024-09-10T09:45:49.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:49.861+0000 7f192b7fe640 1 -- 172.21.15.134:0/3500218316 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f192c043070 con 0x7f193c1066c0 2024-09-10T09:45:49.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:49.861+0000 7f19408f5640 1 -- 172.21.15.134:0/3500218316 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f193c1a6260 con 0x7f193c1066c0 2024-09-10T09:45:49.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:49.861+0000 7f192b7fe640 1 -- 172.21.15.134:0/3500218316 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f192c004ab0 con 0x7f193c1066c0 2024-09-10T09:45:49.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:49.861+0000 7f19408f5640 1 -- 172.21.15.134:0/3500218316 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f193c1a67f0 con 0x7f193c1066c0 2024-09-10T09:45:49.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:49.862+0000 7f19408f5640 1 -- 172.21.15.134:0/3500218316 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f193c1097c0 con 0x7f193c1066c0 2024-09-10T09:45:49.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:49.862+0000 7f192b7fe640 1 -- 172.21.15.134:0/3500218316 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f192c004c50 con 0x7f193c1066c0 2024-09-10T09:45:49.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:49.862+0000 7f192b7fe640 1 --2- 172.21.15.134:0/3500218316 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f191c0394f0 0x7f191c03b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:49.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:49.862+0000 7f192b7fe640 1 -- 172.21.15.134:0/3500218316 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f192c06dbb0 con 0x7f193c1066c0 2024-09-10T09:45:49.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:49.863+0000 7f1939d9b640 1 --2- 172.21.15.134:0/3500218316 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f191c0394f0 0x7f191c03b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:49.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:49.863+0000 7f1939d9b640 1 --2- 172.21.15.134:0/3500218316 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f191c0394f0 0x7f191c03b9b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1930009100 tx=0x7f1930008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:49.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:49.867+0000 7f192b7fe640 1 -- 172.21.15.134:0/3500218316 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f192c046590 con 0x7f193c1066c0 2024-09-10T09:45:49.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:49 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T09:45:49.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:49 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:45:49.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:49 smithi077 ceph-mon[30319]: mon.c calling monitor election 2024-09-10T09:45:49.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:49 smithi077 ceph-mon[30319]: mon.a calling monitor election 2024-09-10T09:45:49.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:49 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T09:45:49.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:49 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:45:49.941 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:49 smithi077 ceph-mon[30319]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:45:49.941 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:49 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:45:49.941 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:49 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:45:49.941 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:49 smithi077 ceph-mon[30319]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:45:49.941 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:49 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:45:49.942 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:49 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:45:49.942 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:49 smithi077 ceph-mon[30319]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-10T09:45:49.942 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:49 smithi077 ceph-mon[30319]: monmap epoch 3 2024-09-10T09:45:49.942 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:49 smithi077 ceph-mon[30319]: fsid 14a55226-6f59-11ef-bcea-c7b262605968 2024-09-10T09:45:49.942 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:49 smithi077 ceph-mon[30319]: last_changed 2024-09-10T09:45:44.490735+0000 2024-09-10T09:45:49.943 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:49 smithi077 ceph-mon[30319]: created 2024-09-10T09:44:03.655548+0000 2024-09-10T09:45:49.943 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:49 smithi077 ceph-mon[30319]: min_mon_release 19 (squid) 2024-09-10T09:45:49.943 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:49 smithi077 ceph-mon[30319]: election_strategy: 1 2024-09-10T09:45:49.943 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:49 smithi077 ceph-mon[30319]: 0: [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] mon.a 2024-09-10T09:45:49.943 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:49 smithi077 ceph-mon[30319]: 1: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.c 2024-09-10T09:45:49.943 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:49 smithi077 ceph-mon[30319]: 2: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-09-10T09:45:49.944 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:49 smithi077 ceph-mon[30319]: fsmap 2024-09-10T09:45:49.944 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:49 smithi077 ceph-mon[30319]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T09:45:49.944 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:49 smithi077 ceph-mon[30319]: mgrmap e13: x(active, since 49s) 2024-09-10T09:45:49.944 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:49 smithi077 ceph-mon[30319]: overall HEALTH_OK 2024-09-10T09:45:49.944 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:49 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:49.944 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:49 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:49.945 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:49 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:49.945 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:49 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:50.137 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:50.135+0000 7f19408f5640 1 -- 172.21.15.134:0/3500218316 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f193c1a6b10 con 0x7f193c1066c0 2024-09-10T09:45:50.137 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:50.136+0000 7f192b7fe640 1 -- 172.21.15.134:0/3500218316 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1283 (secure 0 0 0) 0x7f192c03b240 con 0x7f193c1066c0 2024-09-10T09:45:50.137 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-10T09:45:50.137 INFO:teuthology.orchestra.run.smithi134.stdout:{"epoch":3,"fsid":"14a55226-6f59-11ef-bcea-c7b262605968","modified":"2024-09-10T09:45:44.490735Z","created":"2024-09-10T09:44:03.655548Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:3300","nonce":0},{"type":"v1","addr":"172.21.15.77:6789","nonce":0}]},"addr":"172.21.15.77:6789/0","public_addr":"172.21.15.77:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:3300","nonce":0},{"type":"v1","addr":"172.21.15.120:6789","nonce":0}]},"addr":"172.21.15.120:6789/0","public_addr":"172.21.15.120:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-10T09:45:50.138 INFO:teuthology.orchestra.run.smithi134.stderr:dumped monmap epoch 3 2024-09-10T09:45:50.139 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:50.138+0000 7f19408f5640 1 -- 172.21.15.134:0/3500218316 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f191c0394f0 msgr2=0x7f191c03b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:50.139 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:50.138+0000 7f19408f5640 1 --2- 172.21.15.134:0/3500218316 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f191c0394f0 0x7f191c03b9b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1930009100 tx=0x7f1930008040 comp rx=0 tx=0).stop 2024-09-10T09:45:50.139 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:50.138+0000 7f19408f5640 1 -- 172.21.15.134:0/3500218316 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f193c1066c0 msgr2=0x7f193c102200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:50.139 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:50.138+0000 7f19408f5640 1 --2- 172.21.15.134:0/3500218316 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f193c1066c0 0x7f193c102200 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f192c011c50 tx=0x7f192c02cc70 comp rx=0 tx=0).stop 2024-09-10T09:45:50.139 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:50.138+0000 7f19408f5640 1 -- 172.21.15.134:0/3500218316 shutdown_connections 2024-09-10T09:45:50.140 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:50.138+0000 7f19408f5640 1 --2- 172.21.15.134:0/3500218316 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f191c0394f0 0x7f191c03b9b0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:50.140 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:50.138+0000 7f19408f5640 1 --2- 172.21.15.134:0/3500218316 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f193c1a3510 0x7f193c1a5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:50.140 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:50.138+0000 7f19408f5640 1 --2- 172.21.15.134:0/3500218316 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f193c102740 0x7f193c102ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:50.140 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:50.138+0000 7f19408f5640 1 --2- 172.21.15.134:0/3500218316 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f193c1066c0 0x7f193c102200 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:50.140 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:50.138+0000 7f19408f5640 1 -- 172.21.15.134:0/3500218316 >> 172.21.15.134:0/3500218316 conn(0x7f193c0fb9a0 msgr2=0x7f193c105900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:50.140 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:50.138+0000 7f19408f5640 1 -- 172.21.15.134:0/3500218316 shutdown_connections 2024-09-10T09:45:50.140 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:50.138+0000 7f19408f5640 1 -- 172.21.15.134:0/3500218316 wait complete. 2024-09-10T09:45:50.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:50 smithi134 ceph-mon[35674]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:45:50.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:50 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:50.884 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:50 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:45:50.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:50 smithi134 ceph-mon[35674]: from='client.? 172.21.15.134:0/3500218316' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T09:45:50.885 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:50 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:45:50.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:50 smithi077 ceph-mon[30319]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:45:50.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:50 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:50.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:50 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:45:50.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:50 smithi077 ceph-mon[30319]: from='client.? 172.21.15.134:0/3500218316' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T09:45:50.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:50 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:45:51.266 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-10T09:45:51.267 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph config generate-minimal-conf 2024-09-10T09:45:51.655 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:45:51.909 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:51 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T09:45:51.909 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:51 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:45:51.909 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:51 smithi120 ceph-mon[34770]: mon.c calling monitor election 2024-09-10T09:45:51.909 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:51 smithi120 ceph-mon[34770]: mon.a calling monitor election 2024-09-10T09:45:51.909 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:51 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T09:45:51.909 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:51 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:45:51.909 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:51 smithi120 ceph-mon[34770]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:45:51.909 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:51 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:45:51.909 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:51 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:45:51.909 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:51 smithi120 ceph-mon[34770]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:45:51.910 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:51 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:45:51.910 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:51 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:45:51.910 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:51 smithi120 ceph-mon[34770]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-10T09:45:51.910 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:51 smithi120 ceph-mon[34770]: monmap epoch 3 2024-09-10T09:45:51.910 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:51 smithi120 ceph-mon[34770]: fsid 14a55226-6f59-11ef-bcea-c7b262605968 2024-09-10T09:45:51.910 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:51 smithi120 ceph-mon[34770]: last_changed 2024-09-10T09:45:44.490735+0000 2024-09-10T09:45:51.910 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:51 smithi120 ceph-mon[34770]: created 2024-09-10T09:44:03.655548+0000 2024-09-10T09:45:51.910 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:51 smithi120 ceph-mon[34770]: min_mon_release 19 (squid) 2024-09-10T09:45:51.910 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:51 smithi120 ceph-mon[34770]: election_strategy: 1 2024-09-10T09:45:51.910 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:51 smithi120 ceph-mon[34770]: 0: [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] mon.a 2024-09-10T09:45:51.910 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:51 smithi120 ceph-mon[34770]: 1: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.c 2024-09-10T09:45:51.910 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:51 smithi120 ceph-mon[34770]: 2: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-09-10T09:45:51.910 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:51 smithi120 ceph-mon[34770]: fsmap 2024-09-10T09:45:51.911 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:51 smithi120 ceph-mon[34770]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T09:45:51.911 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:51 smithi120 ceph-mon[34770]: mgrmap e13: x(active, since 49s) 2024-09-10T09:45:51.911 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:51 smithi120 ceph-mon[34770]: overall HEALTH_OK 2024-09-10T09:45:51.911 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:51 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:51.911 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:51 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:51.911 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:51 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:51.911 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:51 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:51.911 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:51 smithi120 ceph-mon[34770]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:45:51.911 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:51 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:51.911 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:51 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:45:51.911 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:51 smithi120 ceph-mon[34770]: from='client.? 172.21.15.134:0/3500218316' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T09:45:51.911 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:51 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:45:52.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:52.296+0000 7f7354a00640 1 -- 172.21.15.77:0/231616643 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7350101610 msgr2=0x7f73501019f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:52.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:52.296+0000 7f7354a00640 1 --2- 172.21.15.77:0/231616643 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7350101610 0x7f73501019f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7340009880 tx=0x7f734002a5d0 comp rx=0 tx=0).stop 2024-09-10T09:45:52.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:52.296+0000 7f7354a00640 1 -- 172.21.15.77:0/231616643 shutdown_connections 2024-09-10T09:45:52.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:52.296+0000 7f7354a00640 1 --2- 172.21.15.77:0/231616643 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7350101610 0x7f73501019f0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:52.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:52.296+0000 7f7354a00640 1 -- 172.21.15.77:0/231616643 >> 172.21.15.77:0/231616643 conn(0x7f73500fd0b0 msgr2=0x7f73500ff4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:52.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:52.296+0000 7f7354a00640 1 -- 172.21.15.77:0/231616643 shutdown_connections 2024-09-10T09:45:52.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:52.296+0000 7f7354a00640 1 -- 172.21.15.77:0/231616643 wait complete. 2024-09-10T09:45:52.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:52.297+0000 7f7354a00640 1 Processor -- start 2024-09-10T09:45:52.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:52.297+0000 7f7354a00640 1 -- start start 2024-09-10T09:45:52.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:52.298+0000 7f7354a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7350101610 0x7f73500744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:52.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:52.298+0000 7f7354a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7350076130 0x7f7350074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:52.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:52.298+0000 7f7354a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7350074f70 0x7f73501a9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:52.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:52.298+0000 7f7354a00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f735010ba10 con 0x7f7350076130 2024-09-10T09:45:52.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:52.298+0000 7f7354a00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f735010b890 con 0x7f7350074f70 2024-09-10T09:45:52.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:52.298+0000 7f7354a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f735010bb90 con 0x7f7350101610 2024-09-10T09:45:52.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:52.298+0000 7f734ea00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7350076130 0x7f7350074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:52.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:52.298+0000 7f734ea00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7350076130 0x7f7350074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60234/0 (socket says 172.21.15.77:60234) 2024-09-10T09:45:52.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:52.298+0000 7f734ea00640 1 -- 172.21.15.77:0/771588928 learned_addr learned my addr 172.21.15.77:0/771588928 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:45:52.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:52.298+0000 7f734fe00640 1 --2- 172.21.15.77:0/771588928 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7350074f70 0x7f73501a9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:52.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:52.298+0000 7f734f400640 1 --2- 172.21.15.77:0/771588928 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7350101610 0x7f73500744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:52.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:52.299+0000 7f734ea00640 1 -- 172.21.15.77:0/771588928 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7350101610 msgr2=0x7f73500744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:52.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:52.299+0000 7f734ea00640 1 --2- 172.21.15.77:0/771588928 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7350101610 0x7f73500744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:52.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:52.299+0000 7f734ea00640 1 -- 172.21.15.77:0/771588928 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7350074f70 msgr2=0x7f73501a9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:52.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:52.299+0000 7f734ea00640 1 --2- 172.21.15.77:0/771588928 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7350074f70 0x7f73501a9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:52.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:52.299+0000 7f734ea00640 1 -- 172.21.15.77:0/771588928 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73501aa280 con 0x7f7350076130 2024-09-10T09:45:52.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:52.300+0000 7f734ea00640 1 --2- 172.21.15.77:0/771588928 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7350076130 0x7f7350074a30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f733c00b520 tx=0x7f733c00b9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:52.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:52.300+0000 7f7347e00640 1 -- 172.21.15.77:0/771588928 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f733c004230 con 0x7f7350076130 2024-09-10T09:45:52.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:52.300+0000 7f7347e00640 1 -- 172.21.15.77:0/771588928 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f733c0043d0 con 0x7f7350076130 2024-09-10T09:45:52.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:52.300+0000 7f7354a00640 1 -- 172.21.15.77:0/771588928 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73501aa570 con 0x7f7350076130 2024-09-10T09:45:52.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:52.300+0000 7f7354a00640 1 -- 172.21.15.77:0/771588928 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73501aaab0 con 0x7f7350076130 2024-09-10T09:45:52.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:52.300+0000 7f7347e00640 1 -- 172.21.15.77:0/771588928 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f733c003d60 con 0x7f7350076130 2024-09-10T09:45:52.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:52.301+0000 7f7347e00640 1 -- 172.21.15.77:0/771588928 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f733c00e070 con 0x7f7350076130 2024-09-10T09:45:52.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:52.301+0000 7f7354a00640 1 -- 172.21.15.77:0/771588928 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7330005290 con 0x7f7350076130 2024-09-10T09:45:52.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:52.302+0000 7f7347e00640 1 --2- 172.21.15.77:0/771588928 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f73340394f0 0x7f733403b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:52.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:52.302+0000 7f7347e00640 1 -- 172.21.15.77:0/771588928 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f733c04b9c0 con 0x7f7350076130 2024-09-10T09:45:52.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:52.302+0000 7f734f400640 1 --2- 172.21.15.77:0/771588928 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f73340394f0 0x7f733403b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:52.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:52.303+0000 7f734f400640 1 --2- 172.21.15.77:0/771588928 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f73340394f0 0x7f733403b9b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7340009230 tx=0x7f7340005a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:52.313 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:52.311+0000 7f7347e00640 1 -- 172.21.15.77:0/771588928 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f733c014020 con 0x7f7350076130 2024-09-10T09:45:52.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:52 smithi077 ceph-mon[30319]: mon.b calling monitor election 2024-09-10T09:45:52.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:52 smithi077 ceph-mon[30319]: Updating smithi134:/var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T09:45:52.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:52 smithi077 ceph-mon[30319]: Updating smithi120:/var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T09:45:52.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:52 smithi077 ceph-mon[30319]: Updating smithi077:/var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T09:45:52.440 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:52 smithi077 ceph-mon[30319]: mon.b calling monitor election 2024-09-10T09:45:52.440 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:52 smithi077 ceph-mon[30319]: mon.c calling monitor election 2024-09-10T09:45:52.440 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:52 smithi077 ceph-mon[30319]: mon.a calling monitor election 2024-09-10T09:45:52.440 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:52 smithi077 ceph-mon[30319]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-10T09:45:52.440 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:52 smithi077 ceph-mon[30319]: monmap epoch 3 2024-09-10T09:45:52.440 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:52 smithi077 ceph-mon[30319]: fsid 14a55226-6f59-11ef-bcea-c7b262605968 2024-09-10T09:45:52.440 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:52 smithi077 ceph-mon[30319]: last_changed 2024-09-10T09:45:44.490735+0000 2024-09-10T09:45:52.440 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:52 smithi077 ceph-mon[30319]: created 2024-09-10T09:44:03.655548+0000 2024-09-10T09:45:52.440 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:52 smithi077 ceph-mon[30319]: min_mon_release 19 (squid) 2024-09-10T09:45:52.440 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:52 smithi077 ceph-mon[30319]: election_strategy: 1 2024-09-10T09:45:52.440 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:52 smithi077 ceph-mon[30319]: 0: [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] mon.a 2024-09-10T09:45:52.440 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:52 smithi077 ceph-mon[30319]: 1: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.c 2024-09-10T09:45:52.440 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:52 smithi077 ceph-mon[30319]: 2: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-09-10T09:45:52.441 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:52 smithi077 ceph-mon[30319]: fsmap 2024-09-10T09:45:52.441 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:52 smithi077 ceph-mon[30319]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T09:45:52.441 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:52 smithi077 ceph-mon[30319]: mgrmap e13: x(active, since 51s) 2024-09-10T09:45:52.441 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:52 smithi077 ceph-mon[30319]: overall HEALTH_OK 2024-09-10T09:45:52.441 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:52 smithi077 ceph-mon[30319]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:45:52.441 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:52 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:52.441 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:52 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:52.441 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:52 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:52.441 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:52 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:52.441 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:52 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:52.441 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:52 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:52.441 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:52 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:52.441 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:52 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T09:45:52.441 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:52 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T09:45:52.442 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:52 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:45:52.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:52.498+0000 7f7354a00640 1 -- 172.21.15.77:0/771588928 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f7330005580 con 0x7f7350076130 2024-09-10T09:45:52.499 INFO:teuthology.orchestra.run.smithi077.stdout:# minimal ceph.conf for 14a55226-6f59-11ef-bcea-c7b262605968 2024-09-10T09:45:52.499 INFO:teuthology.orchestra.run.smithi077.stdout:[global] 2024-09-10T09:45:52.500 INFO:teuthology.orchestra.run.smithi077.stdout: fsid = 14a55226-6f59-11ef-bcea-c7b262605968 2024-09-10T09:45:52.500 INFO:teuthology.orchestra.run.smithi077.stdout: mon_host = [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-10T09:45:52.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:52.498+0000 7f7347e00640 1 -- 172.21.15.77:0/771588928 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+275 (secure 0 0 0) 0x7f733c020350 con 0x7f7350076130 2024-09-10T09:45:52.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:52.500+0000 7f7354a00640 1 -- 172.21.15.77:0/771588928 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f73340394f0 msgr2=0x7f733403b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:52.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:52.500+0000 7f7354a00640 1 --2- 172.21.15.77:0/771588928 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f73340394f0 0x7f733403b9b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7340009230 tx=0x7f7340005a40 comp rx=0 tx=0).stop 2024-09-10T09:45:52.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:52.500+0000 7f7354a00640 1 -- 172.21.15.77:0/771588928 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7350076130 msgr2=0x7f7350074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:52.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:52.500+0000 7f7354a00640 1 --2- 172.21.15.77:0/771588928 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7350076130 0x7f7350074a30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f733c00b520 tx=0x7f733c00b9e0 comp rx=0 tx=0).stop 2024-09-10T09:45:52.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:52.501+0000 7f7354a00640 1 -- 172.21.15.77:0/771588928 shutdown_connections 2024-09-10T09:45:52.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:52.501+0000 7f7354a00640 1 --2- 172.21.15.77:0/771588928 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f73340394f0 0x7f733403b9b0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:52.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:52.501+0000 7f7354a00640 1 --2- 172.21.15.77:0/771588928 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7350074f70 0x7f73501a9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:52.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:52.501+0000 7f7354a00640 1 --2- 172.21.15.77:0/771588928 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7350076130 0x7f7350074a30 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:52.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:52.501+0000 7f7354a00640 1 --2- 172.21.15.77:0/771588928 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7350101610 0x7f73500744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:52.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:52.501+0000 7f7354a00640 1 -- 172.21.15.77:0/771588928 >> 172.21.15.77:0/771588928 conn(0x7f73500fd0b0 msgr2=0x7f7350108fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:52.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:52.501+0000 7f7354a00640 1 -- 172.21.15.77:0/771588928 shutdown_connections 2024-09-10T09:45:52.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:45:52.501+0000 7f7354a00640 1 -- 172.21.15.77:0/771588928 wait complete. 2024-09-10T09:45:52.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:52 smithi134 ceph-mon[35674]: mon.b calling monitor election 2024-09-10T09:45:52.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:52 smithi134 ceph-mon[35674]: Updating smithi134:/var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T09:45:52.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:52 smithi134 ceph-mon[35674]: Updating smithi120:/var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T09:45:52.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:52 smithi134 ceph-mon[35674]: Updating smithi077:/var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T09:45:52.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:52 smithi134 ceph-mon[35674]: mon.b calling monitor election 2024-09-10T09:45:52.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:52 smithi134 ceph-mon[35674]: mon.c calling monitor election 2024-09-10T09:45:52.632 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:52 smithi134 ceph-mon[35674]: mon.a calling monitor election 2024-09-10T09:45:52.632 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:52 smithi134 ceph-mon[35674]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-10T09:45:52.632 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:52 smithi134 ceph-mon[35674]: monmap epoch 3 2024-09-10T09:45:52.632 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:52 smithi134 ceph-mon[35674]: fsid 14a55226-6f59-11ef-bcea-c7b262605968 2024-09-10T09:45:52.632 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:52 smithi134 ceph-mon[35674]: last_changed 2024-09-10T09:45:44.490735+0000 2024-09-10T09:45:52.632 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:52 smithi134 ceph-mon[35674]: created 2024-09-10T09:44:03.655548+0000 2024-09-10T09:45:52.632 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:52 smithi134 ceph-mon[35674]: min_mon_release 19 (squid) 2024-09-10T09:45:52.632 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:52 smithi134 ceph-mon[35674]: election_strategy: 1 2024-09-10T09:45:52.632 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:52 smithi134 ceph-mon[35674]: 0: [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] mon.a 2024-09-10T09:45:52.632 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:52 smithi134 ceph-mon[35674]: 1: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.c 2024-09-10T09:45:52.632 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:52 smithi134 ceph-mon[35674]: 2: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-09-10T09:45:52.632 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:52 smithi134 ceph-mon[35674]: fsmap 2024-09-10T09:45:52.632 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:52 smithi134 ceph-mon[35674]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T09:45:52.632 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:52 smithi134 ceph-mon[35674]: mgrmap e13: x(active, since 51s) 2024-09-10T09:45:52.633 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:52 smithi134 ceph-mon[35674]: overall HEALTH_OK 2024-09-10T09:45:52.633 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:52 smithi134 ceph-mon[35674]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:45:52.633 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:52 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:52.633 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:52 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:52.633 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:52 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:52.633 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:52 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:52.633 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:52 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:52.633 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:52 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:52.633 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:52 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:52.633 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:52 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T09:45:52.633 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:52 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T09:45:52.633 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:52 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:45:52.674 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:52 smithi120 ceph-mon[34770]: mon.b calling monitor election 2024-09-10T09:45:52.674 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:52 smithi120 ceph-mon[34770]: Updating smithi134:/var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T09:45:52.674 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:52 smithi120 ceph-mon[34770]: Updating smithi120:/var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T09:45:52.674 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:52 smithi120 ceph-mon[34770]: Updating smithi077:/var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T09:45:52.674 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:52 smithi120 ceph-mon[34770]: mon.b calling monitor election 2024-09-10T09:45:52.674 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:52 smithi120 ceph-mon[34770]: mon.c calling monitor election 2024-09-10T09:45:52.674 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:52 smithi120 ceph-mon[34770]: mon.a calling monitor election 2024-09-10T09:45:52.674 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:52 smithi120 ceph-mon[34770]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-10T09:45:52.674 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:52 smithi120 ceph-mon[34770]: monmap epoch 3 2024-09-10T09:45:52.674 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:52 smithi120 ceph-mon[34770]: fsid 14a55226-6f59-11ef-bcea-c7b262605968 2024-09-10T09:45:52.674 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:52 smithi120 ceph-mon[34770]: last_changed 2024-09-10T09:45:44.490735+0000 2024-09-10T09:45:52.675 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:52 smithi120 ceph-mon[34770]: created 2024-09-10T09:44:03.655548+0000 2024-09-10T09:45:52.675 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:52 smithi120 ceph-mon[34770]: min_mon_release 19 (squid) 2024-09-10T09:45:52.675 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:52 smithi120 ceph-mon[34770]: election_strategy: 1 2024-09-10T09:45:52.675 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:52 smithi120 ceph-mon[34770]: 0: [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] mon.a 2024-09-10T09:45:52.675 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:52 smithi120 ceph-mon[34770]: 1: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.c 2024-09-10T09:45:52.675 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:52 smithi120 ceph-mon[34770]: 2: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-09-10T09:45:52.675 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:52 smithi120 ceph-mon[34770]: fsmap 2024-09-10T09:45:52.675 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:52 smithi120 ceph-mon[34770]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T09:45:52.675 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:52 smithi120 ceph-mon[34770]: mgrmap e13: x(active, since 51s) 2024-09-10T09:45:52.675 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:52 smithi120 ceph-mon[34770]: overall HEALTH_OK 2024-09-10T09:45:52.675 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:52 smithi120 ceph-mon[34770]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:45:52.675 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:52 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:52.675 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:52 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:52.675 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:52 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:52.676 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:52 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:52.676 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:52 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:52.676 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:52 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:52.676 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:52 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:52.676 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:52 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T09:45:52.676 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:52 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T09:45:52.676 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:52 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:45:53.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:53 smithi077 ceph-mon[30319]: Reconfiguring mon.a (unknown last config time)... 2024-09-10T09:45:53.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:53 smithi077 ceph-mon[30319]: Reconfiguring daemon mon.a on smithi077 2024-09-10T09:45:53.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:53 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:45:53.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:53 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/771588928' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:45:53.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:53 smithi134 ceph-mon[35674]: Reconfiguring mon.a (unknown last config time)... 2024-09-10T09:45:53.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:53 smithi134 ceph-mon[35674]: Reconfiguring daemon mon.a on smithi077 2024-09-10T09:45:53.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:53 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:45:53.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:53 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/771588928' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:45:53.673 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:53 smithi120 ceph-mon[34770]: Reconfiguring mon.a (unknown last config time)... 2024-09-10T09:45:53.673 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:53 smithi120 ceph-mon[34770]: Reconfiguring daemon mon.a on smithi077 2024-09-10T09:45:53.673 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:53 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:45:53.673 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:53 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/771588928' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:45:54.066 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-10T09:45:54.067 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-09-10T09:45:54.067 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/etc/ceph/ceph.conf 2024-09-10T09:45:54.125 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-09-10T09:45:54.125 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T09:45:54.222 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-10T09:45:54.222 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/etc/ceph/ceph.conf 2024-09-10T09:45:54.251 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-10T09:45:54.251 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T09:45:54.316 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-10T09:45:54.316 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/etc/ceph/ceph.conf 2024-09-10T09:45:54.349 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-10T09:45:54.349 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T09:45:54.418 INFO:tasks.cephadm:Adding mgr.x on smithi077 2024-09-10T09:45:54.419 INFO:tasks.cephadm:Adding mgr.y on smithi120 2024-09-10T09:45:54.419 INFO:tasks.cephadm:Adding mgr.z on smithi134 2024-09-10T09:45:54.419 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph orch apply mgr '3;smithi077=x;smithi120=y;smithi134=z' 2024-09-10T09:45:54.706 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T09:45:54.920 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:54 smithi120 ceph-mon[34770]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:45:54.921 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:54 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:54.921 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:54 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:54.921 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:54 smithi120 ceph-mon[34770]: Reconfiguring mon.b (monmap changed)... 2024-09-10T09:45:54.921 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:54 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T09:45:54.921 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:54 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T09:45:54.921 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:54 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:45:54.921 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:54 smithi120 ceph-mon[34770]: Reconfiguring daemon mon.b on smithi120 2024-09-10T09:45:54.934 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:54 smithi134 ceph-mon[35674]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:45:54.934 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:54 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:54.934 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:54 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:54.934 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:54 smithi134 ceph-mon[35674]: Reconfiguring mon.b (monmap changed)... 2024-09-10T09:45:54.934 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:54 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T09:45:54.934 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:54 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T09:45:54.934 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:54 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:45:54.934 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:54 smithi134 ceph-mon[35674]: Reconfiguring daemon mon.b on smithi120 2024-09-10T09:45:54.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:54 smithi077 ceph-mon[30319]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:45:54.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:54 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:54.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:54 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:54.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:54 smithi077 ceph-mon[30319]: Reconfiguring mon.b (monmap changed)... 2024-09-10T09:45:54.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:54 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T09:45:54.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:54 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T09:45:54.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:54 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:45:54.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:54 smithi077 ceph-mon[30319]: Reconfiguring daemon mon.b on smithi120 2024-09-10T09:45:55.093 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:55.091+0000 7f009c5b1640 1 -- 172.21.15.134:0/805962082 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f00941005e0 msgr2=0x7f00941009e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:55.093 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:55.091+0000 7f009c5b1640 1 --2- 172.21.15.134:0/805962082 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f00941005e0 0x7f00941009e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0090009880 tx=0x7f009002a5d0 comp rx=0 tx=0).stop 2024-09-10T09:45:55.093 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:55.091+0000 7f009c5b1640 1 -- 172.21.15.134:0/805962082 shutdown_connections 2024-09-10T09:45:55.093 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:55.091+0000 7f009c5b1640 1 --2- 172.21.15.134:0/805962082 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f00941005e0 0x7f00941009e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:55.093 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:55.091+0000 7f009c5b1640 1 -- 172.21.15.134:0/805962082 >> 172.21.15.134:0/805962082 conn(0x7f00940fbdb0 msgr2=0x7f00940fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:55.093 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:55.091+0000 7f009c5b1640 1 -- 172.21.15.134:0/805962082 shutdown_connections 2024-09-10T09:45:55.093 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:55.091+0000 7f009c5b1640 1 -- 172.21.15.134:0/805962082 wait complete. 2024-09-10T09:45:55.096 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:55.092+0000 7f009c5b1640 1 Processor -- start 2024-09-10T09:45:55.097 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:55.092+0000 7f009c5b1640 1 -- start start 2024-09-10T09:45:55.097 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:55.092+0000 7f009c5b1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f00941005e0 0x7f009406a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:55.097 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:55.092+0000 7f009c5b1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0094199d20 0x7f009419c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:55.097 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:55.092+0000 7f009c5b1640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f009419c650 0x7f009419eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:55.097 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:55.092+0000 7f009c5b1640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f00941066d0 con 0x7f009419c650 2024-09-10T09:45:55.097 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:55.092+0000 7f009c5b1640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0094106550 con 0x7f0094199d20 2024-09-10T09:45:55.097 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:55.092+0000 7f009c5b1640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0094106850 con 0x7f00941005e0 2024-09-10T09:45:55.097 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:55.093+0000 7f009a34d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f00941005e0 0x7f009406a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:55.097 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:55.093+0000 7f009a34d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f00941005e0 0x7f009406a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:34682/0 (socket says 172.21.15.134:34682) 2024-09-10T09:45:55.098 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:55.093+0000 7f009a34d640 1 -- 172.21.15.134:0/2341909461 learned_addr learned my addr 172.21.15.134:0/2341909461 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-10T09:45:55.098 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:55.093+0000 7f009ab4e640 1 --2- 172.21.15.134:0/2341909461 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f009419c650 0x7f009419eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:55.098 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:55.093+0000 7f0099b4c640 1 --2- 172.21.15.134:0/2341909461 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0094199d20 0x7f009419c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:55.098 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:55.093+0000 7f009a34d640 1 -- 172.21.15.134:0/2341909461 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0094199d20 msgr2=0x7f009419c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:55.098 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:55.093+0000 7f009a34d640 1 --2- 172.21.15.134:0/2341909461 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0094199d20 0x7f009419c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:55.098 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:55.093+0000 7f009a34d640 1 -- 172.21.15.134:0/2341909461 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f009419c650 msgr2=0x7f009419eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:55.098 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:55.093+0000 7f009a34d640 1 --2- 172.21.15.134:0/2341909461 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f009419c650 0x7f009419eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:55.098 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:55.093+0000 7f009a34d640 1 -- 172.21.15.134:0/2341909461 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f009419f420 con 0x7f00941005e0 2024-09-10T09:45:55.098 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:55.093+0000 7f009a34d640 1 --2- 172.21.15.134:0/2341909461 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f00941005e0 0x7f009406a770 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0090031610 tx=0x7f00900043c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:55.098 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:55.093+0000 7f008b7fe640 1 -- 172.21.15.134:0/2341909461 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f009003f030 con 0x7f00941005e0 2024-09-10T09:45:55.098 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:55.093+0000 7f009c5b1640 1 -- 172.21.15.134:0/2341909461 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f009419f650 con 0x7f00941005e0 2024-09-10T09:45:55.099 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:55.093+0000 7f008b7fe640 1 -- 172.21.15.134:0/2341909461 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0090003ce0 con 0x7f00941005e0 2024-09-10T09:45:55.099 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:55.093+0000 7f008b7fe640 1 -- 172.21.15.134:0/2341909461 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0090035070 con 0x7f00941005e0 2024-09-10T09:45:55.099 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:55.093+0000 7f009c5b1640 1 -- 172.21.15.134:0/2341909461 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f009419fc00 con 0x7f00941005e0 2024-09-10T09:45:55.099 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:55.094+0000 7f008b7fe640 1 -- 172.21.15.134:0/2341909461 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f0090042030 con 0x7f00941005e0 2024-09-10T09:45:55.099 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:55.094+0000 7f008b7fe640 1 --2- 172.21.15.134:0/2341909461 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f00800394f0 0x7f008003b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:55.099 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:55.094+0000 7f008b7fe640 1 -- 172.21.15.134:0/2341909461 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f009006db40 con 0x7f00941005e0 2024-09-10T09:45:55.099 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:55.095+0000 7f009c5b1640 1 -- 172.21.15.134:0/2341909461 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0094100a60 con 0x7f00941005e0 2024-09-10T09:45:55.099 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:55.095+0000 7f0099b4c640 1 --2- 172.21.15.134:0/2341909461 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f00800394f0 0x7f008003b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:55.099 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:55.095+0000 7f0099b4c640 1 --2- 172.21.15.134:0/2341909461 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f00800394f0 0x7f008003b9b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f009406acf0 tx=0x7f0084008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:55.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:55.104+0000 7f008b7fe640 1 -- 172.21.15.134:0/2341909461 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f009003ac70 con 0x7f00941005e0 2024-09-10T09:45:55.292 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:55.290+0000 7f009c5b1640 1 -- 172.21.15.134:0/2341909461 --> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi077=x;smithi120=y;smithi134=z", "target": ["mon-mgr", ""]}) -- 0x7f00940626c0 con 0x7f00800394f0 2024-09-10T09:45:55.297 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:55.295+0000 7f008b7fe640 1 -- 172.21.15.134:0/2341909461 <== mgr.14150 v2:172.21.15.77:6800/4261324650 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f00940626c0 con 0x7f00800394f0 2024-09-10T09:45:55.297 INFO:teuthology.orchestra.run.smithi134.stdout:Scheduled mgr update... 2024-09-10T09:45:55.301 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:55.298+0000 7f009c5b1640 1 -- 172.21.15.134:0/2341909461 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f00800394f0 msgr2=0x7f008003b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:55.301 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:55.298+0000 7f009c5b1640 1 --2- 172.21.15.134:0/2341909461 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f00800394f0 0x7f008003b9b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f009406acf0 tx=0x7f0084008040 comp rx=0 tx=0).stop 2024-09-10T09:45:55.301 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:55.298+0000 7f009c5b1640 1 -- 172.21.15.134:0/2341909461 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f00941005e0 msgr2=0x7f009406a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:55.301 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:55.298+0000 7f009c5b1640 1 --2- 172.21.15.134:0/2341909461 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f00941005e0 0x7f009406a770 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0090031610 tx=0x7f00900043c0 comp rx=0 tx=0).stop 2024-09-10T09:45:55.301 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:55.298+0000 7f009c5b1640 1 -- 172.21.15.134:0/2341909461 shutdown_connections 2024-09-10T09:45:55.301 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:55.298+0000 7f009c5b1640 1 --2- 172.21.15.134:0/2341909461 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f00800394f0 0x7f008003b9b0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:55.301 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:55.298+0000 7f009c5b1640 1 --2- 172.21.15.134:0/2341909461 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f009419c650 0x7f009419eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:55.301 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:55.298+0000 7f009c5b1640 1 --2- 172.21.15.134:0/2341909461 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0094199d20 0x7f009419c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:55.301 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:55.298+0000 7f009c5b1640 1 --2- 172.21.15.134:0/2341909461 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f00941005e0 0x7f009406a770 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:55.301 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:55.298+0000 7f009c5b1640 1 -- 172.21.15.134:0/2341909461 >> 172.21.15.134:0/2341909461 conn(0x7f00940fbdb0 msgr2=0x7f00940fc8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:55.301 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:55.298+0000 7f009c5b1640 1 -- 172.21.15.134:0/2341909461 shutdown_connections 2024-09-10T09:45:55.302 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:45:55.298+0000 7f009c5b1640 1 -- 172.21.15.134:0/2341909461 wait complete. 2024-09-10T09:45:56.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:55 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:56.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:55 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:56.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:55 smithi134 ceph-mon[35674]: Reconfiguring mon.c (monmap changed)... 2024-09-10T09:45:56.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:55 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T09:45:56.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:55 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T09:45:56.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:55 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:45:56.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:55 smithi134 ceph-mon[35674]: Reconfiguring daemon mon.c on smithi134 2024-09-10T09:45:56.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:55 smithi134 ceph-mon[35674]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi077=x;smithi120=y;smithi134=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:45:56.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:55 smithi134 ceph-mon[35674]: Saving service mgr spec with placement smithi077=x;smithi120=y;smithi134=z;count:3 2024-09-10T09:45:56.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:55 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:56.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:55 smithi134 ceph-mon[35674]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:45:56.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:55 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:56.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:55 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:56.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:55 smithi120 ceph-mon[34770]: Reconfiguring mon.c (monmap changed)... 2024-09-10T09:45:56.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:55 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T09:45:56.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:55 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T09:45:56.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:55 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:45:56.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:55 smithi120 ceph-mon[34770]: Reconfiguring daemon mon.c on smithi134 2024-09-10T09:45:56.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:55 smithi120 ceph-mon[34770]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi077=x;smithi120=y;smithi134=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:45:56.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:55 smithi120 ceph-mon[34770]: Saving service mgr spec with placement smithi077=x;smithi120=y;smithi134=z;count:3 2024-09-10T09:45:56.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:55 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:56.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:55 smithi120 ceph-mon[34770]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:45:56.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:55 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:56.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:55 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:56.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:55 smithi077 ceph-mon[30319]: Reconfiguring mon.c (monmap changed)... 2024-09-10T09:45:56.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:55 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T09:45:56.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:55 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T09:45:56.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:55 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:45:56.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:55 smithi077 ceph-mon[30319]: Reconfiguring daemon mon.c on smithi134 2024-09-10T09:45:56.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:55 smithi077 ceph-mon[30319]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi077=x;smithi120=y;smithi134=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:45:56.440 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:55 smithi077 ceph-mon[30319]: Saving service mgr spec with placement smithi077=x;smithi120=y;smithi134=z;count:3 2024-09-10T09:45:56.440 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:55 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:56.440 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:55 smithi077 ceph-mon[30319]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:45:56.493 DEBUG:teuthology.orchestra.run.smithi120:mgr.y> sudo journalctl -f -n 0 -u ceph-14a55226-6f59-11ef-bcea-c7b262605968@mgr.y.service 2024-09-10T09:45:56.495 DEBUG:teuthology.orchestra.run.smithi134:mgr.z> sudo journalctl -f -n 0 -u ceph-14a55226-6f59-11ef-bcea-c7b262605968@mgr.z.service 2024-09-10T09:45:56.498 INFO:tasks.cephadm:Deploying OSDs... 2024-09-10T09:45:56.498 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-09-10T09:45:56.498 DEBUG:teuthology.orchestra.run.smithi077:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T09:45:56.537 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-10T09:45:56.537 DEBUG:teuthology.orchestra.run.smithi077:> stat /dev/vg_nvme/lv_1 2024-09-10T09:45:56.615 INFO:teuthology.orchestra.run.smithi077.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-10T09:45:56.615 INFO:teuthology.orchestra.run.smithi077.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T09:45:56.615 INFO:teuthology.orchestra.run.smithi077.stdout:Device: 6h/6d Inode: 637 Links: 1 2024-09-10T09:45:56.615 INFO:teuthology.orchestra.run.smithi077.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T09:45:56.615 INFO:teuthology.orchestra.run.smithi077.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T09:45:56.615 INFO:teuthology.orchestra.run.smithi077.stdout:Access: 2024-09-10 09:45:12.557999106 +0000 2024-09-10T09:45:56.615 INFO:teuthology.orchestra.run.smithi077.stdout:Modify: 2024-09-10 09:40:51.120449909 +0000 2024-09-10T09:45:56.615 INFO:teuthology.orchestra.run.smithi077.stdout:Change: 2024-09-10 09:40:51.120449909 +0000 2024-09-10T09:45:56.615 INFO:teuthology.orchestra.run.smithi077.stdout: Birth: 2024-09-10 09:40:51.120449909 +0000 2024-09-10T09:45:56.616 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-10T09:45:56.711 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records in 2024-09-10T09:45:56.711 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records out 2024-09-10T09:45:56.711 INFO:teuthology.orchestra.run.smithi077.stderr:512 bytes copied, 0.000291769 s, 1.8 MB/s 2024-09-10T09:45:56.714 DEBUG:teuthology.orchestra.run.smithi077:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-10T09:45:56.751 DEBUG:teuthology.orchestra.run.smithi077:> stat /dev/vg_nvme/lv_2 2024-09-10T09:45:56.827 INFO:teuthology.orchestra.run.smithi077.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-10T09:45:56.867 INFO:teuthology.orchestra.run.smithi077.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T09:45:56.867 INFO:teuthology.orchestra.run.smithi077.stdout:Device: 6h/6d Inode: 656 Links: 1 2024-09-10T09:45:56.867 INFO:teuthology.orchestra.run.smithi077.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T09:45:56.867 INFO:teuthology.orchestra.run.smithi077.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T09:45:56.867 INFO:teuthology.orchestra.run.smithi077.stdout:Access: 2024-09-10 09:45:12.608998046 +0000 2024-09-10T09:45:56.867 INFO:teuthology.orchestra.run.smithi077.stdout:Modify: 2024-09-10 09:40:51.579440306 +0000 2024-09-10T09:45:56.867 INFO:teuthology.orchestra.run.smithi077.stdout:Change: 2024-09-10 09:40:51.579440306 +0000 2024-09-10T09:45:56.868 INFO:teuthology.orchestra.run.smithi077.stdout: Birth: 2024-09-10 09:40:51.579440306 +0000 2024-09-10T09:45:56.868 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-10T09:45:56.920 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records in 2024-09-10T09:45:56.921 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records out 2024-09-10T09:45:56.921 INFO:teuthology.orchestra.run.smithi077.stderr:512 bytes copied, 0.000353448 s, 1.4 MB/s 2024-09-10T09:45:56.923 DEBUG:teuthology.orchestra.run.smithi077:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-10T09:45:56.963 DEBUG:teuthology.orchestra.run.smithi077:> stat /dev/vg_nvme/lv_3 2024-09-10T09:45:57.039 INFO:teuthology.orchestra.run.smithi077.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-10T09:45:57.039 INFO:teuthology.orchestra.run.smithi077.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T09:45:57.039 INFO:teuthology.orchestra.run.smithi077.stdout:Device: 6h/6d Inode: 677 Links: 1 2024-09-10T09:45:57.039 INFO:teuthology.orchestra.run.smithi077.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T09:45:57.039 INFO:teuthology.orchestra.run.smithi077.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T09:45:57.039 INFO:teuthology.orchestra.run.smithi077.stdout:Access: 2024-09-10 09:45:12.671996736 +0000 2024-09-10T09:45:57.039 INFO:teuthology.orchestra.run.smithi077.stdout:Modify: 2024-09-10 09:40:52.058430284 +0000 2024-09-10T09:45:57.039 INFO:teuthology.orchestra.run.smithi077.stdout:Change: 2024-09-10 09:40:52.058430284 +0000 2024-09-10T09:45:57.039 INFO:teuthology.orchestra.run.smithi077.stdout: Birth: 2024-09-10 09:40:52.058430284 +0000 2024-09-10T09:45:57.040 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-10T09:45:57.132 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records in 2024-09-10T09:45:57.132 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records out 2024-09-10T09:45:57.132 INFO:teuthology.orchestra.run.smithi077.stderr:512 bytes copied, 0.000282534 s, 1.8 MB/s 2024-09-10T09:45:57.135 DEBUG:teuthology.orchestra.run.smithi077:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-10T09:45:57.170 DEBUG:teuthology.orchestra.run.smithi077:> stat /dev/vg_nvme/lv_4 2024-09-10T09:45:57.245 INFO:teuthology.orchestra.run.smithi077.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-10T09:45:57.245 INFO:teuthology.orchestra.run.smithi077.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T09:45:57.245 INFO:teuthology.orchestra.run.smithi077.stdout:Device: 6h/6d Inode: 696 Links: 1 2024-09-10T09:45:57.245 INFO:teuthology.orchestra.run.smithi077.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T09:45:57.246 INFO:teuthology.orchestra.run.smithi077.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T09:45:57.246 INFO:teuthology.orchestra.run.smithi077.stdout:Access: 2024-09-10 09:45:12.730995509 +0000 2024-09-10T09:45:57.246 INFO:teuthology.orchestra.run.smithi077.stdout:Modify: 2024-09-10 09:40:52.648417941 +0000 2024-09-10T09:45:57.246 INFO:teuthology.orchestra.run.smithi077.stdout:Change: 2024-09-10 09:40:52.648417941 +0000 2024-09-10T09:45:57.246 INFO:teuthology.orchestra.run.smithi077.stdout: Birth: 2024-09-10 09:40:52.648417941 +0000 2024-09-10T09:45:57.246 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-10T09:45:57.337 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records in 2024-09-10T09:45:57.338 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records out 2024-09-10T09:45:57.338 INFO:teuthology.orchestra.run.smithi077.stderr:512 bytes copied, 0.000280518 s, 1.8 MB/s 2024-09-10T09:45:57.341 DEBUG:teuthology.orchestra.run.smithi077:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-10T09:45:57.377 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-10T09:45:57.378 DEBUG:teuthology.orchestra.run.smithi120:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T09:45:57.397 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-10T09:45:57.397 DEBUG:teuthology.orchestra.run.smithi120:> stat /dev/vg_nvme/lv_1 2024-09-10T09:45:57.456 INFO:teuthology.orchestra.run.smithi120.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-10T09:45:57.456 INFO:teuthology.orchestra.run.smithi120.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T09:45:57.456 INFO:teuthology.orchestra.run.smithi120.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-09-10T09:45:57.456 INFO:teuthology.orchestra.run.smithi120.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T09:45:57.456 INFO:teuthology.orchestra.run.smithi120.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T09:45:57.457 INFO:teuthology.orchestra.run.smithi120.stdout:Access: 2024-09-10 09:45:43.919228339 +0000 2024-09-10T09:45:57.457 INFO:teuthology.orchestra.run.smithi120.stdout:Modify: 2024-09-10 09:40:30.064535236 +0000 2024-09-10T09:45:57.457 INFO:teuthology.orchestra.run.smithi120.stdout:Change: 2024-09-10 09:40:30.064535236 +0000 2024-09-10T09:45:57.457 INFO:teuthology.orchestra.run.smithi120.stdout: Birth: 2024-09-10 09:40:30.064535236 +0000 2024-09-10T09:45:57.457 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-10T09:45:57.523 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records in 2024-09-10T09:45:57.523 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records out 2024-09-10T09:45:57.524 INFO:teuthology.orchestra.run.smithi120.stderr:512 bytes copied, 0.000233946 s, 2.2 MB/s 2024-09-10T09:45:57.525 DEBUG:teuthology.orchestra.run.smithi120:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-10T09:45:57.584 DEBUG:teuthology.orchestra.run.smithi120:> stat /dev/vg_nvme/lv_2 2024-09-10T09:45:57.641 INFO:teuthology.orchestra.run.smithi120.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-10T09:45:57.642 INFO:teuthology.orchestra.run.smithi120.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T09:45:57.642 INFO:teuthology.orchestra.run.smithi120.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-09-10T09:45:57.642 INFO:teuthology.orchestra.run.smithi120.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T09:45:57.642 INFO:teuthology.orchestra.run.smithi120.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T09:45:57.642 INFO:teuthology.orchestra.run.smithi120.stdout:Access: 2024-09-10 09:45:43.919228339 +0000 2024-09-10T09:45:57.642 INFO:teuthology.orchestra.run.smithi120.stdout:Modify: 2024-09-10 09:40:30.388527626 +0000 2024-09-10T09:45:57.642 INFO:teuthology.orchestra.run.smithi120.stdout:Change: 2024-09-10 09:40:30.388527626 +0000 2024-09-10T09:45:57.642 INFO:teuthology.orchestra.run.smithi120.stdout: Birth: 2024-09-10 09:40:30.388527626 +0000 2024-09-10T09:45:57.642 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-10T09:45:57.708 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records in 2024-09-10T09:45:57.708 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records out 2024-09-10T09:45:57.708 INFO:teuthology.orchestra.run.smithi120.stderr:512 bytes copied, 0.000216691 s, 2.4 MB/s 2024-09-10T09:45:57.709 DEBUG:teuthology.orchestra.run.smithi120:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-10T09:45:57.768 DEBUG:teuthology.orchestra.run.smithi120:> stat /dev/vg_nvme/lv_3 2024-09-10T09:45:57.825 INFO:teuthology.orchestra.run.smithi120.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-10T09:45:57.825 INFO:teuthology.orchestra.run.smithi120.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T09:45:57.826 INFO:teuthology.orchestra.run.smithi120.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-09-10T09:45:57.826 INFO:teuthology.orchestra.run.smithi120.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T09:45:57.826 INFO:teuthology.orchestra.run.smithi120.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T09:45:57.826 INFO:teuthology.orchestra.run.smithi120.stdout:Access: 2024-09-10 09:45:43.919228339 +0000 2024-09-10T09:45:57.826 INFO:teuthology.orchestra.run.smithi120.stdout:Modify: 2024-09-10 09:40:30.713519992 +0000 2024-09-10T09:45:57.826 INFO:teuthology.orchestra.run.smithi120.stdout:Change: 2024-09-10 09:40:30.713519992 +0000 2024-09-10T09:45:57.826 INFO:teuthology.orchestra.run.smithi120.stdout: Birth: 2024-09-10 09:40:30.713519992 +0000 2024-09-10T09:45:57.826 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-10T09:45:57.890 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records in 2024-09-10T09:45:57.890 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records out 2024-09-10T09:45:57.891 INFO:teuthology.orchestra.run.smithi120.stderr:512 bytes copied, 0.000219698 s, 2.3 MB/s 2024-09-10T09:45:57.892 DEBUG:teuthology.orchestra.run.smithi120:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-10T09:45:57.950 DEBUG:teuthology.orchestra.run.smithi120:> stat /dev/vg_nvme/lv_4 2024-09-10T09:45:58.006 INFO:teuthology.orchestra.run.smithi120.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-10T09:45:58.007 INFO:teuthology.orchestra.run.smithi120.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T09:45:58.007 INFO:teuthology.orchestra.run.smithi120.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-09-10T09:45:58.007 INFO:teuthology.orchestra.run.smithi120.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T09:45:58.007 INFO:teuthology.orchestra.run.smithi120.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T09:45:58.007 INFO:teuthology.orchestra.run.smithi120.stdout:Access: 2024-09-10 09:45:43.920228316 +0000 2024-09-10T09:45:58.007 INFO:teuthology.orchestra.run.smithi120.stdout:Modify: 2024-09-10 09:40:31.051512053 +0000 2024-09-10T09:45:58.007 INFO:teuthology.orchestra.run.smithi120.stdout:Change: 2024-09-10 09:40:31.051512053 +0000 2024-09-10T09:45:58.007 INFO:teuthology.orchestra.run.smithi120.stdout: Birth: 2024-09-10 09:40:31.051512053 +0000 2024-09-10T09:45:58.007 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-10T09:45:58.071 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records in 2024-09-10T09:45:58.071 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records out 2024-09-10T09:45:58.071 INFO:teuthology.orchestra.run.smithi120.stderr:512 bytes copied, 0.000216763 s, 2.4 MB/s 2024-09-10T09:45:58.072 DEBUG:teuthology.orchestra.run.smithi120:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-10T09:45:58.129 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-10T09:45:58.129 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T09:45:58.146 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-10T09:45:58.146 DEBUG:teuthology.orchestra.run.smithi134:> stat /dev/vg_nvme/lv_1 2024-09-10T09:45:58.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:57 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:58.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:57 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:58.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:57 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:45:58.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:57 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:45:58.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:57 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:45:58.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:57 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:58.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:57 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T09:45:58.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:57 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T09:45:58.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:57 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T09:45:58.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:57 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:45:58.175 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:57 smithi120 ceph-mon[34770]: Deploying daemon mgr.z on smithi134 2024-09-10T09:45:58.175 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:45:57 smithi120 ceph-mon[34770]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:45:58.201 INFO:teuthology.orchestra.run.smithi134.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-10T09:45:58.201 INFO:teuthology.orchestra.run.smithi134.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T09:45:58.201 INFO:teuthology.orchestra.run.smithi134.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-09-10T09:45:58.202 INFO:teuthology.orchestra.run.smithi134.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T09:45:58.202 INFO:teuthology.orchestra.run.smithi134.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T09:45:58.202 INFO:teuthology.orchestra.run.smithi134.stdout:Access: 2024-09-10 09:45:34.817489856 +0000 2024-09-10T09:45:58.202 INFO:teuthology.orchestra.run.smithi134.stdout:Modify: 2024-09-10 09:40:30.743290676 +0000 2024-09-10T09:45:58.202 INFO:teuthology.orchestra.run.smithi134.stdout:Change: 2024-09-10 09:40:30.743290676 +0000 2024-09-10T09:45:58.202 INFO:teuthology.orchestra.run.smithi134.stdout: Birth: 2024-09-10 09:40:30.743290676 +0000 2024-09-10T09:45:58.202 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-10T09:45:58.264 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:57 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:58.264 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:57 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:58.264 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:57 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:45:58.264 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:57 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:45:58.264 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:57 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:45:58.265 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:57 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:58.265 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:57 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T09:45:58.265 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:57 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T09:45:58.265 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:57 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T09:45:58.265 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:57 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:45:58.265 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:57 smithi134 ceph-mon[35674]: Deploying daemon mgr.z on smithi134 2024-09-10T09:45:58.265 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:45:57 smithi134 ceph-mon[35674]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:45:58.266 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records in 2024-09-10T09:45:58.267 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records out 2024-09-10T09:45:58.267 INFO:teuthology.orchestra.run.smithi134.stderr:512 bytes copied, 0.000263221 s, 1.9 MB/s 2024-09-10T09:45:58.267 DEBUG:teuthology.orchestra.run.smithi134:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-10T09:45:58.324 DEBUG:teuthology.orchestra.run.smithi134:> stat /dev/vg_nvme/lv_2 2024-09-10T09:45:58.380 INFO:teuthology.orchestra.run.smithi134.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-10T09:45:58.380 INFO:teuthology.orchestra.run.smithi134.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T09:45:58.380 INFO:teuthology.orchestra.run.smithi134.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-09-10T09:45:58.380 INFO:teuthology.orchestra.run.smithi134.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T09:45:58.381 INFO:teuthology.orchestra.run.smithi134.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T09:45:58.381 INFO:teuthology.orchestra.run.smithi134.stdout:Access: 2024-09-10 09:45:34.817489856 +0000 2024-09-10T09:45:58.381 INFO:teuthology.orchestra.run.smithi134.stdout:Modify: 2024-09-10 09:40:31.066282823 +0000 2024-09-10T09:45:58.381 INFO:teuthology.orchestra.run.smithi134.stdout:Change: 2024-09-10 09:40:31.066282823 +0000 2024-09-10T09:45:58.381 INFO:teuthology.orchestra.run.smithi134.stdout: Birth: 2024-09-10 09:40:31.066282823 +0000 2024-09-10T09:45:58.381 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-10T09:45:58.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:57 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:58.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:57 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:58.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:57 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:45:58.440 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:57 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:45:58.440 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:57 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:45:58.440 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:57 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:45:58.440 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:57 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T09:45:58.440 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:57 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T09:45:58.440 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:57 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T09:45:58.440 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:57 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:45:58.440 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:57 smithi077 ceph-mon[30319]: Deploying daemon mgr.z on smithi134 2024-09-10T09:45:58.440 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:45:57 smithi077 ceph-mon[30319]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:45:58.523 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records in 2024-09-10T09:45:58.523 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records out 2024-09-10T09:45:58.523 INFO:teuthology.orchestra.run.smithi134.stderr:512 bytes copied, 0.000192787 s, 2.7 MB/s 2024-09-10T09:45:58.525 DEBUG:teuthology.orchestra.run.smithi134:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-10T09:45:58.541 DEBUG:teuthology.orchestra.run.smithi134:> stat /dev/vg_nvme/lv_3 2024-09-10T09:45:58.601 INFO:teuthology.orchestra.run.smithi134.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-10T09:45:58.601 INFO:teuthology.orchestra.run.smithi134.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T09:45:58.601 INFO:teuthology.orchestra.run.smithi134.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-09-10T09:45:58.601 INFO:teuthology.orchestra.run.smithi134.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T09:45:58.601 INFO:teuthology.orchestra.run.smithi134.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T09:45:58.602 INFO:teuthology.orchestra.run.smithi134.stdout:Access: 2024-09-10 09:45:58.474920879 +0000 2024-09-10T09:45:58.602 INFO:teuthology.orchestra.run.smithi134.stdout:Modify: 2024-09-10 09:40:31.389274970 +0000 2024-09-10T09:45:58.602 INFO:teuthology.orchestra.run.smithi134.stdout:Change: 2024-09-10 09:40:31.389274970 +0000 2024-09-10T09:45:58.602 INFO:teuthology.orchestra.run.smithi134.stdout: Birth: 2024-09-10 09:40:31.389274970 +0000 2024-09-10T09:45:58.602 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-10T09:45:58.707 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records in 2024-09-10T09:45:58.707 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records out 2024-09-10T09:45:58.707 INFO:teuthology.orchestra.run.smithi134.stderr:512 bytes copied, 0.000242344 s, 2.1 MB/s 2024-09-10T09:45:58.709 DEBUG:teuthology.orchestra.run.smithi134:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-10T09:45:58.724 DEBUG:teuthology.orchestra.run.smithi134:> stat /dev/vg_nvme/lv_4 2024-09-10T09:45:58.781 INFO:teuthology.orchestra.run.smithi134.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-10T09:45:58.781 INFO:teuthology.orchestra.run.smithi134.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T09:45:58.781 INFO:teuthology.orchestra.run.smithi134.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-09-10T09:45:58.781 INFO:teuthology.orchestra.run.smithi134.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T09:45:58.781 INFO:teuthology.orchestra.run.smithi134.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T09:45:58.781 INFO:teuthology.orchestra.run.smithi134.stdout:Access: 2024-09-10 09:45:58.662916351 +0000 2024-09-10T09:45:58.781 INFO:teuthology.orchestra.run.smithi134.stdout:Modify: 2024-09-10 09:40:31.712267117 +0000 2024-09-10T09:45:58.781 INFO:teuthology.orchestra.run.smithi134.stdout:Change: 2024-09-10 09:40:31.712267117 +0000 2024-09-10T09:45:58.781 INFO:teuthology.orchestra.run.smithi134.stdout: Birth: 2024-09-10 09:40:31.712267117 +0000 2024-09-10T09:45:58.782 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-10T09:45:58.844 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Sep 10 09:45:58 smithi134 systemd[1]: Starting Ceph mgr.z for 14a55226-6f59-11ef-bcea-c7b262605968... 2024-09-10T09:45:58.847 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records in 2024-09-10T09:45:58.847 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records out 2024-09-10T09:45:58.847 INFO:teuthology.orchestra.run.smithi134.stderr:512 bytes copied, 0.000169821 s, 3.0 MB/s 2024-09-10T09:45:58.849 DEBUG:teuthology.orchestra.run.smithi134:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-10T09:45:58.906 INFO:tasks.cephadm:Deploying osd.0 on smithi077 with /dev/vg_nvme/lv_4... 2024-09-10T09:45:58.906 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-10T09:45:59.131 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Sep 10 09:45:58 smithi134 podman[37425]: 2024-09-10 09:45:58.844376576 +0000 UTC m=+0.016408979 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T09:45:59.131 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Sep 10 09:45:58 smithi134 podman[37425]: 2024-09-10 09:45:58.950851592 +0000 UTC m=+0.122884023 container create 84161fdf796da7108462a7154931c9fcf956bb7068d7f5cb39404e71f5b08d9c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-z, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T09:45:59.266 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:45:59.505 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Sep 10 09:45:59 smithi134 podman[37425]: 2024-09-10 09:45:59.170163865 +0000 UTC m=+0.342196255 container init 84161fdf796da7108462a7154931c9fcf956bb7068d7f5cb39404e71f5b08d9c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-z, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-10T09:45:59.505 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Sep 10 09:45:59 smithi134 podman[37425]: 2024-09-10 09:45:59.172897668 +0000 UTC m=+0.344930060 container start 84161fdf796da7108462a7154931c9fcf956bb7068d7f5cb39404e71f5b08d9c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-z, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-10T09:45:59.505 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Sep 10 09:45:59 smithi134 ceph-mgr[37481]: -- 172.21.15.134:0/480925704 <== mon.1 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55575ebc10e0 con 0x55575eb9e800 2024-09-10T09:45:59.505 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Sep 10 09:45:59 smithi134 bash[37425]: 84161fdf796da7108462a7154931c9fcf956bb7068d7f5cb39404e71f5b08d9c 2024-09-10T09:45:59.505 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Sep 10 09:45:59 smithi134 systemd[1]: Started Ceph mgr.z for 14a55226-6f59-11ef-bcea-c7b262605968. 2024-09-10T09:45:59.505 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Sep 10 09:45:59 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-z[37477]: 2024-09-10T09:45:59.291+0000 7f0829d13040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T09:45:59.505 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Sep 10 09:45:59 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-z[37477]: 2024-09-10T09:45:59.355+0000 7f0829d13040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T09:45:59.505 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Sep 10 09:45:59 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-z[37477]: 2024-09-10T09:45:59.503+0000 7f0829d13040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-10T09:45:59.854 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Sep 10 09:45:59 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-z[37477]: 2024-09-10T09:45:59.568+0000 7f0829d13040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T09:45:59.854 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Sep 10 09:45:59 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-z[37477]: 2024-09-10T09:45:59.715+0000 7f0829d13040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T09:45:59.854 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Sep 10 09:45:59 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-z[37477]: 2024-09-10T09:45:59.852+0000 7f0829d13040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T09:46:00.563 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:00 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:00.563 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:00 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:00.563 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:00 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:00.563 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:00 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T09:46:00.563 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:00 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T09:46:00.563 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:00 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T09:46:00.563 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:00 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:46:00.563 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:00 smithi077 ceph-mon[30319]: Deploying daemon mgr.y on smithi120 2024-09-10T09:46:00.563 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:00 smithi077 ceph-mon[30319]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:46:00.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:00 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:00.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:00 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:00.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:00 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:00.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:00 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T09:46:00.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:00 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T09:46:00.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:00 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T09:46:00.632 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:00 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:46:00.632 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:00 smithi134 ceph-mon[35674]: Deploying daemon mgr.y on smithi120 2024-09-10T09:46:00.632 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:00 smithi134 ceph-mon[35674]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:46:00.632 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Sep 10 09:46:00 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-z[37477]: 2024-09-10T09:46:00.166+0000 7f0829d13040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T09:46:00.632 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Sep 10 09:46:00 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-z[37477]: 2024-09-10T09:46:00.225+0000 7f0829d13040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T09:46:00.632 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Sep 10 09:46:00 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-z[37477]: 2024-09-10T09:46:00.412+0000 7f0829d13040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T09:46:00.658 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:00 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:00.658 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:00 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:00.658 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:00 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:00.658 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:00 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T09:46:00.659 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:00 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T09:46:00.659 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:00 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T09:46:00.659 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:00 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:46:00.659 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:00 smithi120 ceph-mon[34770]: Deploying daemon mgr.y on smithi120 2024-09-10T09:46:00.659 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:00 smithi120 ceph-mon[34770]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:46:01.631 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Sep 10 09:46:01 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-z[37477]: 2024-09-10T09:46:01.281+0000 7f0829d13040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T09:46:01.631 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Sep 10 09:46:01 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-z[37477]: 2024-09-10T09:46:01.348+0000 7f0829d13040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T09:46:01.780 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Sep 10 09:46:01 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-y[36142]: 2024-09-10T09:46:01.778+0000 7f158a614040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-10T09:46:02.165 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Sep 10 09:46:01 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-y[36142]: 2024-09-10T09:46:01.848+0000 7f158a614040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T09:46:02.165 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Sep 10 09:46:02 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-y[36142]: 2024-09-10T09:46:02.005+0000 7f158a614040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T09:46:02.166 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Sep 10 09:46:02 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-y[36142]: 2024-09-10T09:46:02.164+0000 7f158a614040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T09:46:02.166 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:46:02.197 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi077:vg_nvme/lv_4 2024-09-10T09:46:02.358 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Sep 10 09:46:02 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-z[37477]: 2024-09-10T09:46:02.091+0000 7f0829d13040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T09:46:02.358 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Sep 10 09:46:02 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-z[37477]: 2024-09-10T09:46:02.211+0000 7f0829d13040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T09:46:02.358 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Sep 10 09:46:02 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-z[37477]: 2024-09-10T09:46:02.356+0000 7f0829d13040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T09:46:02.553 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:46:02.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:02 smithi134 ceph-mon[35674]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:46:02.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:02 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:02.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:02 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:02.632 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:02 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:02.632 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:02 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:02.632 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:02 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:46:02.632 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Sep 10 09:46:02 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-z[37477]: 2024-09-10T09:46:02.425+0000 7f0829d13040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T09:46:02.632 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Sep 10 09:46:02 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-z[37477]: 2024-09-10T09:46:02.576+0000 7f0829d13040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T09:46:02.752 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Sep 10 09:46:02 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-y[36142]: 2024-09-10T09:46:02.490+0000 7f158a614040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T09:46:02.752 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Sep 10 09:46:02 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-y[36142]: 2024-09-10T09:46:02.553+0000 7f158a614040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T09:46:02.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:02 smithi120 ceph-mon[34770]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:46:02.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:02 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:02.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:02 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:02.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:02 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:02.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:02 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:02.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:02 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:46:02.848 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:02 smithi077 ceph-mon[30319]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:46:02.848 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:02 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:02.848 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:02 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:02.849 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:02 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:02.849 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:02 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:02.849 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:02 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:46:02.947 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Sep 10 09:46:02 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-z[37477]: 2024-09-10T09:46:02.646+0000 7f0829d13040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T09:46:02.947 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Sep 10 09:46:02 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-z[37477]: 2024-09-10T09:46:02.714+0000 7f0829d13040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T09:46:02.947 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Sep 10 09:46:02 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-z[37477]: 2024-09-10T09:46:02.878+0000 7f0829d13040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T09:46:02.947 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Sep 10 09:46:02 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-z[37477]: 2024-09-10T09:46:02.946+0000 7f0829d13040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T09:46:03.173 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Sep 10 09:46:02 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-y[36142]: 2024-09-10T09:46:02.750+0000 7f158a614040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T09:46:03.208 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Sep 10 09:46:03 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-z[37477]: 2024-09-10T09:46:03.011+0000 7f0829d13040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T09:46:03.208 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Sep 10 09:46:03 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-z[37477]: 2024-09-10T09:46:03.091+0000 7f0829d13040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T09:46:03.208 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Sep 10 09:46:03 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-z[37477]: 2024-09-10T09:46:03.149+0000 7f0829d13040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T09:46:03.208 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Sep 10 09:46:03 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-z[37477]: 2024-09-10T09:46:03.207+0000 7f0829d13040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T09:46:03.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:46:03.235+0000 7f080f200640 1 -- 172.21.15.77:0/1233900210 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f08101098e0 msgr2=0x7f081010bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:03.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:46:03.235+0000 7f080f200640 1 --2- 172.21.15.77:0/1233900210 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f08101098e0 0x7f081010bd80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0808009ca0 tx=0x7f080802a950 comp rx=0 tx=0).stop 2024-09-10T09:46:03.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:46:03.235+0000 7f080f200640 1 -- 172.21.15.77:0/1233900210 shutdown_connections 2024-09-10T09:46:03.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:46:03.235+0000 7f080f200640 1 --2- 172.21.15.77:0/1233900210 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f08101098e0 0x7f081010bd80 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:03.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:46:03.235+0000 7f080f200640 1 --2- 172.21.15.77:0/1233900210 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0810074bf0 0x7f0810075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:03.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:46:03.235+0000 7f080f200640 1 --2- 172.21.15.77:0/1233900210 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f08100761c0 0x7f0810074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:03.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:46:03.235+0000 7f080f200640 1 -- 172.21.15.77:0/1233900210 >> 172.21.15.77:0/1233900210 conn(0x7f08100fd1f0 msgr2=0x7f08100ff610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:46:03.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:46:03.235+0000 7f080f200640 1 -- 172.21.15.77:0/1233900210 shutdown_connections 2024-09-10T09:46:03.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:46:03.235+0000 7f080f200640 1 -- 172.21.15.77:0/1233900210 wait complete. 2024-09-10T09:46:03.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:46:03.236+0000 7f080f200640 1 Processor -- start 2024-09-10T09:46:03.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:46:03.236+0000 7f080f200640 1 -- start start 2024-09-10T09:46:03.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:46:03.237+0000 7f080f200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0810074bf0 0x7f0810199ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:03.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:46:03.237+0000 7f080f200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f08100761c0 0x7f081019a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:03.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:46:03.237+0000 7f080f200640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f08101098e0 0x7f08101a1490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:03.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:46:03.237+0000 7f080f200640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f081010f8a0 con 0x7f08100761c0 2024-09-10T09:46:03.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:46:03.237+0000 7f080f200640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f081010f720 con 0x7f08101098e0 2024-09-10T09:46:03.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:46:03.237+0000 7f080f200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f081010fa20 con 0x7f0810074bf0 2024-09-10T09:46:03.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:46:03.237+0000 7f080de00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0810074bf0 0x7f0810199ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:03.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:46:03.237+0000 7f080de00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0810074bf0 0x7f0810199ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:56318/0 (socket says 172.21.15.77:56318) 2024-09-10T09:46:03.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:46:03.237+0000 7f080de00640 1 -- 172.21.15.77:0/3053529797 learned_addr learned my addr 172.21.15.77:0/3053529797 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:46:03.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:46:03.237+0000 7f080d400640 1 --2- 172.21.15.77:0/3053529797 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f08100761c0 0x7f081019a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:03.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:46:03.237+0000 7f080e800640 1 --2- 172.21.15.77:0/3053529797 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f08101098e0 0x7f08101a1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:03.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:46:03.237+0000 7f080de00640 1 -- 172.21.15.77:0/3053529797 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f08101098e0 msgr2=0x7f08101a1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:03.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:46:03.237+0000 7f080de00640 1 --2- 172.21.15.77:0/3053529797 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f08101098e0 0x7f08101a1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:03.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:46:03.237+0000 7f080de00640 1 -- 172.21.15.77:0/3053529797 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f08100761c0 msgr2=0x7f081019a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:03.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:46:03.237+0000 7f080de00640 1 --2- 172.21.15.77:0/3053529797 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f08100761c0 0x7f081019a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:03.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:46:03.237+0000 7f080de00640 1 -- 172.21.15.77:0/3053529797 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08101a1b00 con 0x7f0810074bf0 2024-09-10T09:46:03.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:46:03.238+0000 7f080de00640 1 --2- 172.21.15.77:0/3053529797 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0810074bf0 0x7f0810199ed0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f080000ea60 tx=0x7f080000ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:46:03.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:46:03.238+0000 7f0806a00640 1 -- 172.21.15.77:0/3053529797 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f080000cde0 con 0x7f0810074bf0 2024-09-10T09:46:03.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:46:03.238+0000 7f0806a00640 1 -- 172.21.15.77:0/3053529797 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08000035f0 con 0x7f0810074bf0 2024-09-10T09:46:03.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:46:03.238+0000 7f0806a00640 1 -- 172.21.15.77:0/3053529797 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0800010670 con 0x7f0810074bf0 2024-09-10T09:46:03.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:46:03.238+0000 7f080f200640 1 -- 172.21.15.77:0/3053529797 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08101a1df0 con 0x7f0810074bf0 2024-09-10T09:46:03.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:46:03.238+0000 7f080f200640 1 -- 172.21.15.77:0/3053529797 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08101a21f0 con 0x7f0810074bf0 2024-09-10T09:46:03.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:46:03.239+0000 7f0806a00640 1 -- 172.21.15.77:0/3053529797 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f0800010810 con 0x7f0810074bf0 2024-09-10T09:46:03.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:46:03.239+0000 7f080f200640 1 -- 172.21.15.77:0/3053529797 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08101086a0 con 0x7f0810074bf0 2024-09-10T09:46:03.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:46:03.240+0000 7f0806a00640 1 --2- 172.21.15.77:0/3053529797 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f07f40392c0 0x7f07f403b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:03.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:46:03.240+0000 7f0806a00640 1 -- 172.21.15.77:0/3053529797 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f080004c730 con 0x7f0810074bf0 2024-09-10T09:46:03.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:46:03.240+0000 7f080d400640 1 --2- 172.21.15.77:0/3053529797 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f07f40392c0 0x7f07f403b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:03.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:46:03.240+0000 7f080d400640 1 --2- 172.21.15.77:0/3053529797 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f07f40392c0 0x7f07f403b780 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f07fc0095f0 tx=0x7f07fc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:46:03.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:46:03.245+0000 7f0806a00640 1 -- 172.21.15.77:0/3053529797 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f08000173d0 con 0x7f0810074bf0 2024-09-10T09:46:03.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:46:03.463+0000 7f080f200640 1 -- 172.21.15.77:0/3053529797 --> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f0810062630 con 0x7f07f40392c0 2024-09-10T09:46:03.923 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Sep 10 09:46:03 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-y[36142]: 2024-09-10T09:46:03.634+0000 7f158a614040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T09:46:03.923 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Sep 10 09:46:03 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-y[36142]: 2024-09-10T09:46:03.701+0000 7f158a614040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T09:46:03.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:03 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:03.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:03 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:03.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:03 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:03.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:03 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:03.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:03 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:46:03.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:03 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:46:03.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:03 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:03.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:03 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T09:46:03.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:03 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T09:46:03.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:03 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:46:03.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:03 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:46:03.925 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:03 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:46:03.925 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:03 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:46:04.132 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Sep 10 09:46:03 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-z[37477]: 2024-09-10T09:46:03.987+0000 7f0829d13040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T09:46:04.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:03 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:04.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:03 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:04.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:03 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:04.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:03 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:04.133 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:03 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:46:04.133 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:03 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:46:04.133 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:03 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:04.133 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:03 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T09:46:04.133 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:03 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T09:46:04.133 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:03 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:46:04.133 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:03 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:46:04.133 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:03 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:46:04.133 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:03 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:46:04.187 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:03 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:04.188 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:03 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:04.188 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:03 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:04.188 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:03 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:04.188 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:03 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:46:04.188 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:03 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:46:04.188 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:03 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:04.188 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:03 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T09:46:04.188 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:03 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T09:46:04.188 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:03 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:46:04.188 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:03 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:46:04.188 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:03 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:46:04.188 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:03 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:46:04.670 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Sep 10 09:46:04 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-y[36142]: 2024-09-10T09:46:04.411+0000 7f158a614040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T09:46:04.670 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Sep 10 09:46:04 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-y[36142]: 2024-09-10T09:46:04.526+0000 7f158a614040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T09:46:04.670 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Sep 10 09:46:04 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-y[36142]: 2024-09-10T09:46:04.668+0000 7f158a614040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T09:46:04.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:46:04.779+0000 7f0806a00640 1 -- 172.21.15.77:0/3053529797 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mgrmap(e 14) ==== 92060+0+0 (secure 0 0 0) 0x7f0800021950 con 0x7f0810074bf0 2024-09-10T09:46:04.923 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Sep 10 09:46:04 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-y[36142]: 2024-09-10T09:46:04.729+0000 7f158a614040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T09:46:04.924 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Sep 10 09:46:04 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-y[36142]: 2024-09-10T09:46:04.878+0000 7f158a614040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T09:46:04.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:04 smithi120 ceph-mon[34770]: Reconfiguring mgr.x (unknown last config time)... 2024-09-10T09:46:04.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:04 smithi120 ceph-mon[34770]: Reconfiguring daemon mgr.x on smithi077 2024-09-10T09:46:04.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:04 smithi120 ceph-mon[34770]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:46:04.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:04 smithi120 ceph-mon[34770]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:46:04.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:04 smithi120 ceph-mon[34770]: Standby manager daemon z started 2024-09-10T09:46:04.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:04 smithi120 ceph-mon[34770]: from='mgr.? 172.21.15.134:0/4271691253' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-10T09:46:04.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:04 smithi120 ceph-mon[34770]: from='mgr.? 172.21.15.134:0/4271691253' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T09:46:04.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:04 smithi120 ceph-mon[34770]: from='mgr.? 172.21.15.134:0/4271691253' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-10T09:46:04.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:04 smithi120 ceph-mon[34770]: from='mgr.? 172.21.15.134:0/4271691253' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T09:46:04.925 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:04 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:04.925 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:04 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:04.925 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:04 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:46:04.925 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:04 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:05.124 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:04 smithi077 ceph-mon[30319]: Reconfiguring mgr.x (unknown last config time)... 2024-09-10T09:46:05.125 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:04 smithi077 ceph-mon[30319]: Reconfiguring daemon mgr.x on smithi077 2024-09-10T09:46:05.125 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:04 smithi077 ceph-mon[30319]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:46:05.125 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:04 smithi077 ceph-mon[30319]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:46:05.125 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:04 smithi077 ceph-mon[30319]: Standby manager daemon z started 2024-09-10T09:46:05.125 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:04 smithi077 ceph-mon[30319]: from='mgr.? 172.21.15.134:0/4271691253' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-10T09:46:05.125 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:04 smithi077 ceph-mon[30319]: from='mgr.? 172.21.15.134:0/4271691253' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T09:46:05.126 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:04 smithi077 ceph-mon[30319]: from='mgr.? 172.21.15.134:0/4271691253' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-10T09:46:05.126 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:04 smithi077 ceph-mon[30319]: from='mgr.? 172.21.15.134:0/4271691253' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T09:46:05.126 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:04 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:05.126 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:04 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:05.126 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:04 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:46:05.126 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:04 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:05.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:04 smithi134 ceph-mon[35674]: Reconfiguring mgr.x (unknown last config time)... 2024-09-10T09:46:05.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:04 smithi134 ceph-mon[35674]: Reconfiguring daemon mgr.x on smithi077 2024-09-10T09:46:05.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:04 smithi134 ceph-mon[35674]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:46:05.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:04 smithi134 ceph-mon[35674]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:46:05.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:04 smithi134 ceph-mon[35674]: Standby manager daemon z started 2024-09-10T09:46:05.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:04 smithi134 ceph-mon[35674]: from='mgr.? 172.21.15.134:0/4271691253' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-10T09:46:05.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:04 smithi134 ceph-mon[35674]: from='mgr.? 172.21.15.134:0/4271691253' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T09:46:05.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:04 smithi134 ceph-mon[35674]: from='mgr.? 172.21.15.134:0/4271691253' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-10T09:46:05.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:04 smithi134 ceph-mon[35674]: from='mgr.? 172.21.15.134:0/4271691253' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T09:46:05.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:04 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:05.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:04 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:05.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:04 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:46:05.133 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:04 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:05.227 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Sep 10 09:46:04 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-y[36142]: 2024-09-10T09:46:04.945+0000 7f158a614040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T09:46:05.228 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Sep 10 09:46:05 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-y[36142]: 2024-09-10T09:46:05.005+0000 7f158a614040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T09:46:05.228 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Sep 10 09:46:05 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-y[36142]: 2024-09-10T09:46:05.161+0000 7f158a614040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T09:46:05.228 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Sep 10 09:46:05 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-y[36142]: 2024-09-10T09:46:05.226+0000 7f158a614040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T09:46:05.497 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Sep 10 09:46:05 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-y[36142]: 2024-09-10T09:46:05.291+0000 7f158a614040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T09:46:05.497 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Sep 10 09:46:05 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-y[36142]: 2024-09-10T09:46:05.373+0000 7f158a614040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T09:46:05.497 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Sep 10 09:46:05 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-y[36142]: 2024-09-10T09:46:05.433+0000 7f158a614040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T09:46:05.497 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Sep 10 09:46:05 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-y[36142]: 2024-09-10T09:46:05.495+0000 7f158a614040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T09:46:06.104 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:05 smithi077 ceph-mon[30319]: mgrmap e14: x(active, since 65s), standbys: z 2024-09-10T09:46:06.104 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:05 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-10T09:46:06.104 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:05 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:46:06.104 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:05 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:46:06.104 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:05 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:06.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:05 smithi134 ceph-mon[35674]: mgrmap e14: x(active, since 65s), standbys: z 2024-09-10T09:46:06.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:05 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-10T09:46:06.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:05 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:46:06.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:05 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:46:06.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:05 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:06.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:05 smithi120 ceph-mon[34770]: mgrmap e14: x(active, since 65s), standbys: z 2024-09-10T09:46:06.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:05 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-10T09:46:06.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:05 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:46:06.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:05 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:46:06.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:05 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:06.673 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Sep 10 09:46:06 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-y[36142]: 2024-09-10T09:46:06.283+0000 7f158a614040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T09:46:06.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:46:06.800+0000 7f0806a00640 1 -- 172.21.15.77:0/3053529797 <== mon.1 v2:172.21.15.134:3300/0 8 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f080001d250 con 0x7f0810074bf0 2024-09-10T09:46:07.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:06 smithi134 ceph-mon[35674]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:46:07.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:06 smithi134 ceph-mon[35674]: Standby manager daemon y started 2024-09-10T09:46:07.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:06 smithi134 ceph-mon[35674]: from='mgr.? 172.21.15.120:0/1084676280' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-10T09:46:07.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:06 smithi134 ceph-mon[35674]: from='mgr.? 172.21.15.120:0/1084676280' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T09:46:07.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:06 smithi134 ceph-mon[35674]: from='mgr.? 172.21.15.120:0/1084676280' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-10T09:46:07.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:06 smithi134 ceph-mon[35674]: from='mgr.? 172.21.15.120:0/1084676280' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T09:46:07.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:06 smithi120 ceph-mon[34770]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:46:07.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:06 smithi120 ceph-mon[34770]: Standby manager daemon y started 2024-09-10T09:46:07.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:06 smithi120 ceph-mon[34770]: from='mgr.? 172.21.15.120:0/1084676280' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-10T09:46:07.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:06 smithi120 ceph-mon[34770]: from='mgr.? 172.21.15.120:0/1084676280' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T09:46:07.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:06 smithi120 ceph-mon[34770]: from='mgr.? 172.21.15.120:0/1084676280' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-10T09:46:07.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:06 smithi120 ceph-mon[34770]: from='mgr.? 172.21.15.120:0/1084676280' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T09:46:07.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:06 smithi077 ceph-mon[30319]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:46:07.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:06 smithi077 ceph-mon[30319]: Standby manager daemon y started 2024-09-10T09:46:07.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:06 smithi077 ceph-mon[30319]: from='mgr.? 172.21.15.120:0/1084676280' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-10T09:46:07.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:06 smithi077 ceph-mon[30319]: from='mgr.? 172.21.15.120:0/1084676280' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T09:46:07.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:06 smithi077 ceph-mon[30319]: from='mgr.? 172.21.15.120:0/1084676280' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-10T09:46:07.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:06 smithi077 ceph-mon[30319]: from='mgr.? 172.21.15.120:0/1084676280' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T09:46:08.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:07 smithi134 ceph-mon[35674]: mgrmap e15: x(active, since 67s), standbys: z, y 2024-09-10T09:46:08.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:07 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T09:46:08.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:07 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/975117075' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8d46f634-114c-4926-afdc-f8fa37e40336"}]: dispatch 2024-09-10T09:46:08.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:07 smithi134 ceph-mon[35674]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8d46f634-114c-4926-afdc-f8fa37e40336"}]: dispatch 2024-09-10T09:46:08.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:07 smithi134 ceph-mon[35674]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8d46f634-114c-4926-afdc-f8fa37e40336"}]': finished 2024-09-10T09:46:08.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:07 smithi134 ceph-mon[35674]: osdmap e5: 1 total, 0 up, 1 in 2024-09-10T09:46:08.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:07 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T09:46:08.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:07 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/3934299416' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:46:08.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:07 smithi120 ceph-mon[34770]: mgrmap e15: x(active, since 67s), standbys: z, y 2024-09-10T09:46:08.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:07 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T09:46:08.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:07 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/975117075' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8d46f634-114c-4926-afdc-f8fa37e40336"}]: dispatch 2024-09-10T09:46:08.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:07 smithi120 ceph-mon[34770]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8d46f634-114c-4926-afdc-f8fa37e40336"}]: dispatch 2024-09-10T09:46:08.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:07 smithi120 ceph-mon[34770]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8d46f634-114c-4926-afdc-f8fa37e40336"}]': finished 2024-09-10T09:46:08.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:07 smithi120 ceph-mon[34770]: osdmap e5: 1 total, 0 up, 1 in 2024-09-10T09:46:08.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:07 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T09:46:08.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:07 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/3934299416' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:46:08.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:07 smithi077 ceph-mon[30319]: mgrmap e15: x(active, since 67s), standbys: z, y 2024-09-10T09:46:08.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:07 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T09:46:08.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:07 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/975117075' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8d46f634-114c-4926-afdc-f8fa37e40336"}]: dispatch 2024-09-10T09:46:08.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:07 smithi077 ceph-mon[30319]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8d46f634-114c-4926-afdc-f8fa37e40336"}]: dispatch 2024-09-10T09:46:08.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:07 smithi077 ceph-mon[30319]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8d46f634-114c-4926-afdc-f8fa37e40336"}]': finished 2024-09-10T09:46:08.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:07 smithi077 ceph-mon[30319]: osdmap e5: 1 total, 0 up, 1 in 2024-09-10T09:46:08.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:07 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T09:46:08.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:07 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/3934299416' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:46:09.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:08 smithi134 ceph-mon[35674]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:46:09.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:08 smithi120 ceph-mon[34770]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:46:09.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:08 smithi077 ceph-mon[30319]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:46:11.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:10 smithi134 ceph-mon[35674]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:46:11.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:10 smithi120 ceph-mon[34770]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:46:11.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:10 smithi077 ceph-mon[30319]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:46:13.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:12 smithi134 ceph-mon[35674]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:46:13.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:12 smithi120 ceph-mon[34770]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:46:13.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:12 smithi077 ceph-mon[30319]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:46:15.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:14 smithi134 ceph-mon[35674]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:46:15.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:14 smithi120 ceph-mon[34770]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:46:15.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:14 smithi077 ceph-mon[30319]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:46:15.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:15 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-10T09:46:15.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:15 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:46:16.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:15 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-10T09:46:16.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:15 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:46:16.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:15 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-10T09:46:16.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:15 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:46:16.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:16 smithi077 ceph-mon[30319]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:46:16.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:16 smithi077 ceph-mon[30319]: Deploying daemon osd.0 on smithi077 2024-09-10T09:46:17.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:16 smithi134 ceph-mon[35674]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:46:17.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:16 smithi134 ceph-mon[35674]: Deploying daemon osd.0 on smithi077 2024-09-10T09:46:17.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:16 smithi120 ceph-mon[34770]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:46:17.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:16 smithi120 ceph-mon[34770]: Deploying daemon osd.0 on smithi077 2024-09-10T09:46:18.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:18 smithi077 ceph-mon[30319]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:46:19.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:18 smithi134 ceph-mon[35674]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:46:19.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:18 smithi120 ceph-mon[34770]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:46:21.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:20 smithi134 ceph-mon[35674]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:46:21.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:20 smithi120 ceph-mon[34770]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:46:21.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:20 smithi077 ceph-mon[30319]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:46:22.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:22 smithi077 ceph-mon[30319]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:46:23.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:22 smithi134 ceph-mon[35674]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:46:23.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:22 smithi120 ceph-mon[34770]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:46:23.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:23 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:46:23.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:23 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:23.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:23 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:24.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:23 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:46:24.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:23 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:24.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:23 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:24.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:23 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:46:24.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:23 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:24.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:23 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:24.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:24 smithi077 ceph-mon[30319]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:46:25.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:24 smithi134 ceph-mon[35674]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:46:25.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:24 smithi120 ceph-mon[34770]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:46:27.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:26 smithi134 ceph-mon[35674]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:46:27.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:26 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:27.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:26 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:27.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:26 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:46:27.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:26 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:46:27.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:26 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:27.134 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:26 smithi077 ceph-mon[30319]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:46:27.134 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:26 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:27.134 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:26 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:27.134 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:26 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:46:27.134 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:26 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:46:27.134 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:26 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:27.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:26 smithi120 ceph-mon[34770]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:46:27.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:26 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:27.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:26 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:27.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:26 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:46:27.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:26 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:46:27.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:26 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:28.642 INFO:teuthology.orchestra.run.smithi077.stdout:Created osd(s) 0 on host 'smithi077' 2024-09-10T09:46:28.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:46:28.641+0000 7f0806a00640 1 -- 172.21.15.77:0/3053529797 <== mgr.14150 v2:172.21.15.77:6800/4261324650 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0810062630 con 0x7f07f40392c0 2024-09-10T09:46:28.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:46:28.644+0000 7f080f200640 1 -- 172.21.15.77:0/3053529797 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f07f40392c0 msgr2=0x7f07f403b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:28.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:46:28.644+0000 7f080f200640 1 --2- 172.21.15.77:0/3053529797 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f07f40392c0 0x7f07f403b780 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f07fc0095f0 tx=0x7f07fc005f90 comp rx=0 tx=0).stop 2024-09-10T09:46:28.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:46:28.644+0000 7f080f200640 1 -- 172.21.15.77:0/3053529797 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0810074bf0 msgr2=0x7f0810199ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:28.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:46:28.644+0000 7f080f200640 1 --2- 172.21.15.77:0/3053529797 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0810074bf0 0x7f0810199ed0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f080000ea60 tx=0x7f080000ef20 comp rx=0 tx=0).stop 2024-09-10T09:46:28.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:46:28.644+0000 7f080f200640 1 -- 172.21.15.77:0/3053529797 shutdown_connections 2024-09-10T09:46:28.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:46:28.644+0000 7f080f200640 1 --2- 172.21.15.77:0/3053529797 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f07f40392c0 0x7f07f403b780 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:28.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:46:28.644+0000 7f080f200640 1 --2- 172.21.15.77:0/3053529797 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f08101098e0 0x7f08101a1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:28.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:46:28.644+0000 7f080f200640 1 --2- 172.21.15.77:0/3053529797 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f08100761c0 0x7f081019a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:28.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:46:28.644+0000 7f080f200640 1 --2- 172.21.15.77:0/3053529797 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0810074bf0 0x7f0810199ed0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:28.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:46:28.644+0000 7f080f200640 1 -- 172.21.15.77:0/3053529797 >> 172.21.15.77:0/3053529797 conn(0x7f08100fd1f0 msgr2=0x7f0810105870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:46:28.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:46:28.645+0000 7f080f200640 1 -- 172.21.15.77:0/3053529797 shutdown_connections 2024-09-10T09:46:28.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:46:28.645+0000 7f080f200640 1 -- 172.21.15.77:0/3053529797 wait complete. 2024-09-10T09:46:28.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:28 smithi077 ceph-mon[30319]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:46:28.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:28 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:46:28.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:28 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:28.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:28 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:29.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:28 smithi134 ceph-mon[35674]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:46:29.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:28 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:46:29.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:28 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:29.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:28 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:29.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:28 smithi120 ceph-mon[34770]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:46:29.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:28 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:46:29.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:28 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:29.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:28 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:29.294 DEBUG:teuthology.orchestra.run.smithi077:osd.0> sudo journalctl -f -n 0 -u ceph-14a55226-6f59-11ef-bcea-c7b262605968@osd.0.service 2024-09-10T09:46:29.298 INFO:tasks.cephadm:Deploying osd.1 on smithi120 with /dev/vg_nvme/lv_4... 2024-09-10T09:46:29.298 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-10T09:46:29.567 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T09:46:29.943 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:29 smithi120 ceph-mon[34770]: from='osd.0 [v2:172.21.15.77:6802/1509649079,v1:172.21.15.77:6803/1509649079]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-10T09:46:30.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:29 smithi134 ceph-mon[35674]: from='osd.0 [v2:172.21.15.77:6802/1509649079,v1:172.21.15.77:6803/1509649079]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-10T09:46:30.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:29 smithi077 ceph-mon[30319]: from='osd.0 [v2:172.21.15.77:6802/1509649079,v1:172.21.15.77:6803/1509649079]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-10T09:46:30.862 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:30 smithi077 ceph-mon[30319]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:46:30.863 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:30 smithi077 ceph-mon[30319]: from='osd.0 [v2:172.21.15.77:6802/1509649079,v1:172.21.15.77:6803/1509649079]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-10T09:46:30.863 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:30 smithi077 ceph-mon[30319]: osdmap e6: 1 total, 0 up, 1 in 2024-09-10T09:46:30.863 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:30 smithi077 ceph-mon[30319]: from='osd.0 [v2:172.21.15.77:6802/1509649079,v1:172.21.15.77:6803/1509649079]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-09-10T09:46:30.863 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:30 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T09:46:30.951 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:30 smithi120 ceph-mon[34770]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:46:30.951 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:30 smithi120 ceph-mon[34770]: from='osd.0 [v2:172.21.15.77:6802/1509649079,v1:172.21.15.77:6803/1509649079]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-10T09:46:30.951 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:30 smithi120 ceph-mon[34770]: osdmap e6: 1 total, 0 up, 1 in 2024-09-10T09:46:30.951 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:30 smithi120 ceph-mon[34770]: from='osd.0 [v2:172.21.15.77:6802/1509649079,v1:172.21.15.77:6803/1509649079]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-09-10T09:46:30.951 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:30 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T09:46:31.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:30 smithi134 ceph-mon[35674]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:46:31.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:30 smithi134 ceph-mon[35674]: from='osd.0 [v2:172.21.15.77:6802/1509649079,v1:172.21.15.77:6803/1509649079]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-10T09:46:31.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:30 smithi134 ceph-mon[35674]: osdmap e6: 1 total, 0 up, 1 in 2024-09-10T09:46:31.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:30 smithi134 ceph-mon[35674]: from='osd.0 [v2:172.21.15.77:6802/1509649079,v1:172.21.15.77:6803/1509649079]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-09-10T09:46:31.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:30 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T09:46:31.169 INFO:journalctl@ceph.osd.0.smithi077.stdout:Sep 10 09:46:30 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-0[44004]: 2024-09-10T09:46:30.861+0000 7f40eae00640 -1 osd.0 0 waiting for initial osdmap 2024-09-10T09:46:31.169 INFO:journalctl@ceph.osd.0.smithi077.stdout:Sep 10 09:46:30 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-0[44004]: 2024-09-10T09:46:30.864+0000 7f40e5e00640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T09:46:32.105 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:31 smithi120 ceph-mon[34770]: from='osd.0 [v2:172.21.15.77:6802/1509649079,v1:172.21.15.77:6803/1509649079]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-09-10T09:46:32.105 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:31 smithi120 ceph-mon[34770]: osdmap e7: 1 total, 0 up, 1 in 2024-09-10T09:46:32.105 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:31 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T09:46:32.105 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:31 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T09:46:32.105 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:31 smithi120 ceph-mon[34770]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:46:32.105 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T09:46:32.131 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi120:vg_nvme/lv_4 2024-09-10T09:46:32.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:31 smithi134 ceph-mon[35674]: from='osd.0 [v2:172.21.15.77:6802/1509649079,v1:172.21.15.77:6803/1509649079]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-09-10T09:46:32.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:31 smithi134 ceph-mon[35674]: osdmap e7: 1 total, 0 up, 1 in 2024-09-10T09:46:32.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:31 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T09:46:32.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:31 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T09:46:32.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:31 smithi134 ceph-mon[35674]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:46:32.134 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:31 smithi077 ceph-mon[30319]: from='osd.0 [v2:172.21.15.77:6802/1509649079,v1:172.21.15.77:6803/1509649079]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-09-10T09:46:32.135 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:31 smithi077 ceph-mon[30319]: osdmap e7: 1 total, 0 up, 1 in 2024-09-10T09:46:32.135 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:31 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T09:46:32.135 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:31 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T09:46:32.135 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:31 smithi077 ceph-mon[30319]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:46:32.377 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T09:46:32.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:46:32.856+0000 7ffaa909e640 1 -- 172.21.15.120:0/2100029368 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffaa4100ad0 msgr2=0x7ffaa4100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:32.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:46:32.856+0000 7ffaa909e640 1 --2- 172.21.15.120:0/2100029368 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffaa4100ad0 0x7ffaa4100ed0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ffa94009900 tx=0x7ffa9402a410 comp rx=0 tx=0).stop 2024-09-10T09:46:32.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:46:32.856+0000 7ffaa909e640 1 -- 172.21.15.120:0/2100029368 shutdown_connections 2024-09-10T09:46:32.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:46:32.856+0000 7ffaa909e640 1 --2- 172.21.15.120:0/2100029368 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffaa4102690 0x7ffaa4108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:32.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:46:32.856+0000 7ffaa909e640 1 --2- 172.21.15.120:0/2100029368 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffaa4101cd0 0x7ffaa4102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:32.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:46:32.856+0000 7ffaa909e640 1 --2- 172.21.15.120:0/2100029368 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffaa4100ad0 0x7ffaa4100ed0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:32.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:46:32.856+0000 7ffaa909e640 1 -- 172.21.15.120:0/2100029368 >> 172.21.15.120:0/2100029368 conn(0x7ffaa40fc280 msgr2=0x7ffaa40fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:46:32.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:46:32.856+0000 7ffaa909e640 1 -- 172.21.15.120:0/2100029368 shutdown_connections 2024-09-10T09:46:32.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:46:32.856+0000 7ffaa909e640 1 -- 172.21.15.120:0/2100029368 wait complete. 2024-09-10T09:46:32.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:46:32.857+0000 7ffaa909e640 1 Processor -- start 2024-09-10T09:46:32.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:46:32.857+0000 7ffaa909e640 1 -- start start 2024-09-10T09:46:32.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:46:32.858+0000 7ffaa909e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffaa4100ad0 0x7ffaa419a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:32.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:46:32.858+0000 7ffaa909e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffaa4101cd0 0x7ffaa419a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:32.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:46:32.858+0000 7ffaa909e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffaa4102690 0x7ffaa41a1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:32.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:46:32.858+0000 7ffaa909e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa410c800 con 0x7ffaa4102690 2024-09-10T09:46:32.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:46:32.858+0000 7ffaa909e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa410c680 con 0x7ffaa4101cd0 2024-09-10T09:46:32.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:46:32.858+0000 7ffaa909e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa410c980 con 0x7ffaa4100ad0 2024-09-10T09:46:32.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:46:32.858+0000 7ffaa259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffaa4101cd0 0x7ffaa419a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:32.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:46:32.858+0000 7ffaa259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffaa4101cd0 0x7ffaa419a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57824/0 (socket says 172.21.15.120:57824) 2024-09-10T09:46:32.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:46:32.858+0000 7ffaa259c640 1 -- 172.21.15.120:0/3587626522 learned_addr learned my addr 172.21.15.120:0/3587626522 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T09:46:32.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:46:32.858+0000 7ffaa2d9d640 1 --2- 172.21.15.120:0/3587626522 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffaa4100ad0 0x7ffaa419a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:32.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:46:32.858+0000 7ffaa359e640 1 --2- 172.21.15.120:0/3587626522 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffaa4102690 0x7ffaa41a1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:32.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:46:32.858+0000 7ffaa259c640 1 -- 172.21.15.120:0/3587626522 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffaa4100ad0 msgr2=0x7ffaa419a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:32.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:46:32.858+0000 7ffaa259c640 1 --2- 172.21.15.120:0/3587626522 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffaa4100ad0 0x7ffaa419a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:32.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:46:32.858+0000 7ffaa259c640 1 -- 172.21.15.120:0/3587626522 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffaa4102690 msgr2=0x7ffaa41a1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:32.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:46:32.858+0000 7ffaa259c640 1 --2- 172.21.15.120:0/3587626522 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffaa4102690 0x7ffaa41a1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:32.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:46:32.858+0000 7ffaa259c640 1 -- 172.21.15.120:0/3587626522 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffaa41a1dc0 con 0x7ffaa4101cd0 2024-09-10T09:46:32.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:46:32.858+0000 7ffaa359e640 1 --2- 172.21.15.120:0/3587626522 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffaa4102690 0x7ffaa41a1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:46:32.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:46:32.858+0000 7ffaa259c640 1 --2- 172.21.15.120:0/3587626522 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffaa4101cd0 0x7ffaa419a6d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ffa9800b4a0 tx=0x7ffa9800b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:46:32.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:46:32.859+0000 7ffa8bfff640 1 -- 172.21.15.120:0/3587626522 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa98014070 con 0x7ffaa4101cd0 2024-09-10T09:46:32.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:46:32.859+0000 7ffa8bfff640 1 -- 172.21.15.120:0/3587626522 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa98010af0 con 0x7ffaa4101cd0 2024-09-10T09:46:32.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:46:32.859+0000 7ffaa909e640 1 -- 172.21.15.120:0/3587626522 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffaa41a20b0 con 0x7ffaa4101cd0 2024-09-10T09:46:32.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:46:32.860+0000 7ffa8bfff640 1 -- 172.21.15.120:0/3587626522 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa98017d10 con 0x7ffaa4101cd0 2024-09-10T09:46:32.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:46:32.860+0000 7ffaa909e640 1 -- 172.21.15.120:0/3587626522 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffaa41a26c0 con 0x7ffaa4101cd0 2024-09-10T09:46:32.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:46:32.861+0000 7ffaa909e640 1 -- 172.21.15.120:0/3587626522 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffaa41a2980 con 0x7ffaa4101cd0 2024-09-10T09:46:32.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:46:32.862+0000 7ffa8bfff640 1 -- 172.21.15.120:0/3587626522 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7ffa98017070 con 0x7ffaa4101cd0 2024-09-10T09:46:32.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:46:32.862+0000 7ffa8bfff640 1 --2- 172.21.15.120:0/3587626522 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ffa8c0a2ba0 0x7ffa8c0a5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:32.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:46:32.862+0000 7ffa8bfff640 1 -- 172.21.15.120:0/3587626522 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7ffa980cc860 con 0x7ffaa4101cd0 2024-09-10T09:46:32.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:46:32.863+0000 7ffaa2d9d640 1 --2- 172.21.15.120:0/3587626522 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ffa8c0a2ba0 0x7ffa8c0a5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:32.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:46:32.863+0000 7ffaa2d9d640 1 --2- 172.21.15.120:0/3587626522 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ffa8c0a2ba0 0x7ffa8c0a5060 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ffa9402aa00 tx=0x7ffa940315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:46:32.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:46:32.869+0000 7ffa8bfff640 1 -- 172.21.15.120:0/3587626522 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ffa9809b230 con 0x7ffaa4101cd0 2024-09-10T09:46:33.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:46:33.048+0000 7ffaa909e640 1 -- 172.21.15.120:0/3587626522 --> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7ffaa406a6f0 con 0x7ffa8c0a2ba0 2024-09-10T09:46:33.049 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:32 smithi120 ceph-mon[34770]: purged_snaps scrub starts 2024-09-10T09:46:33.049 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:32 smithi120 ceph-mon[34770]: purged_snaps scrub ok 2024-09-10T09:46:33.049 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:32 smithi120 ceph-mon[34770]: osd.0 [v2:172.21.15.77:6802/1509649079,v1:172.21.15.77:6803/1509649079] boot 2024-09-10T09:46:33.049 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:32 smithi120 ceph-mon[34770]: osdmap e8: 1 total, 1 up, 1 in 2024-09-10T09:46:33.049 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:32 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T09:46:33.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:32 smithi134 ceph-mon[35674]: purged_snaps scrub starts 2024-09-10T09:46:33.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:32 smithi134 ceph-mon[35674]: purged_snaps scrub ok 2024-09-10T09:46:33.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:32 smithi134 ceph-mon[35674]: osd.0 [v2:172.21.15.77:6802/1509649079,v1:172.21.15.77:6803/1509649079] boot 2024-09-10T09:46:33.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:32 smithi134 ceph-mon[35674]: osdmap e8: 1 total, 1 up, 1 in 2024-09-10T09:46:33.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:32 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T09:46:33.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:32 smithi077 ceph-mon[30319]: purged_snaps scrub starts 2024-09-10T09:46:33.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:32 smithi077 ceph-mon[30319]: purged_snaps scrub ok 2024-09-10T09:46:33.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:32 smithi077 ceph-mon[30319]: osd.0 [v2:172.21.15.77:6802/1509649079,v1:172.21.15.77:6803/1509649079] boot 2024-09-10T09:46:33.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:32 smithi077 ceph-mon[30319]: osdmap e8: 1 total, 1 up, 1 in 2024-09-10T09:46:33.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:32 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T09:46:34.103 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:33 smithi120 ceph-mon[34770]: Detected new or changed devices on smithi077 2024-09-10T09:46:34.104 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:33 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:34.104 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:33 smithi120 ceph-mon[34770]: osdmap e9: 1 total, 1 up, 1 in 2024-09-10T09:46:34.104 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:33 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:34.104 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:33 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:46:34.104 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:33 smithi120 ceph-mon[34770]: Adjusting osd_memory_target on smithi077 to 17089M 2024-09-10T09:46:34.104 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:33 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:34.104 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:33 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:46:34.104 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:33 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:46:34.104 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:33 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:34.104 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:33 smithi120 ceph-mon[34770]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:46:34.104 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:33 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:46:34.105 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:33 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:46:34.105 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:33 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:46:34.105 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:33 smithi120 ceph-mon[34770]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:46:34.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:33 smithi134 ceph-mon[35674]: Detected new or changed devices on smithi077 2024-09-10T09:46:34.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:33 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:34.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:33 smithi134 ceph-mon[35674]: osdmap e9: 1 total, 1 up, 1 in 2024-09-10T09:46:34.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:33 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:34.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:33 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:46:34.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:33 smithi134 ceph-mon[35674]: Adjusting osd_memory_target on smithi077 to 17089M 2024-09-10T09:46:34.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:33 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:34.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:33 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:46:34.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:33 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:46:34.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:33 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:34.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:33 smithi134 ceph-mon[35674]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:46:34.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:33 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:46:34.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:33 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:46:34.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:33 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:46:34.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:33 smithi134 ceph-mon[35674]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:46:34.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:33 smithi077 ceph-mon[30319]: Detected new or changed devices on smithi077 2024-09-10T09:46:34.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:33 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:34.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:33 smithi077 ceph-mon[30319]: osdmap e9: 1 total, 1 up, 1 in 2024-09-10T09:46:34.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:33 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:34.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:33 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:46:34.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:33 smithi077 ceph-mon[30319]: Adjusting osd_memory_target on smithi077 to 17089M 2024-09-10T09:46:34.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:33 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:34.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:33 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:46:34.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:33 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:46:34.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:33 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:34.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:33 smithi077 ceph-mon[30319]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:46:34.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:33 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:46:34.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:33 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:46:34.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:33 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:46:34.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:33 smithi077 ceph-mon[30319]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:46:35.618 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:35 smithi120 ceph-mon[34770]: from='client.? 172.21.15.120:0/1146010902' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b6570764-736a-49e4-877a-1965e5812d1f"}]: dispatch 2024-09-10T09:46:35.618 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:35 smithi120 ceph-mon[34770]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b6570764-736a-49e4-877a-1965e5812d1f"}]: dispatch 2024-09-10T09:46:35.618 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:35 smithi120 ceph-mon[34770]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b6570764-736a-49e4-877a-1965e5812d1f"}]': finished 2024-09-10T09:46:35.618 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:35 smithi120 ceph-mon[34770]: osdmap e10: 2 total, 1 up, 2 in 2024-09-10T09:46:35.618 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:35 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T09:46:35.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:35 smithi134 ceph-mon[35674]: from='client.? 172.21.15.120:0/1146010902' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b6570764-736a-49e4-877a-1965e5812d1f"}]: dispatch 2024-09-10T09:46:35.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:35 smithi134 ceph-mon[35674]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b6570764-736a-49e4-877a-1965e5812d1f"}]: dispatch 2024-09-10T09:46:35.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:35 smithi134 ceph-mon[35674]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b6570764-736a-49e4-877a-1965e5812d1f"}]': finished 2024-09-10T09:46:35.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:35 smithi134 ceph-mon[35674]: osdmap e10: 2 total, 1 up, 2 in 2024-09-10T09:46:35.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:35 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T09:46:35.689 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:35 smithi077 ceph-mon[30319]: from='client.? 172.21.15.120:0/1146010902' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b6570764-736a-49e4-877a-1965e5812d1f"}]: dispatch 2024-09-10T09:46:35.689 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:35 smithi077 ceph-mon[30319]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b6570764-736a-49e4-877a-1965e5812d1f"}]: dispatch 2024-09-10T09:46:35.689 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:35 smithi077 ceph-mon[30319]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b6570764-736a-49e4-877a-1965e5812d1f"}]': finished 2024-09-10T09:46:35.689 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:35 smithi077 ceph-mon[30319]: osdmap e10: 2 total, 1 up, 2 in 2024-09-10T09:46:35.689 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:35 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T09:46:36.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:36 smithi134 ceph-mon[35674]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:46:36.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:36 smithi134 ceph-mon[35674]: from='client.? 172.21.15.120:0/4140692520' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:46:36.673 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:36 smithi120 ceph-mon[34770]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:46:36.673 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:36 smithi120 ceph-mon[34770]: from='client.? 172.21.15.120:0/4140692520' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:46:36.689 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:36 smithi077 ceph-mon[30319]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:46:36.689 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:36 smithi077 ceph-mon[30319]: from='client.? 172.21.15.120:0/4140692520' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:46:38.923 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:38 smithi120 ceph-mon[34770]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:46:38.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:38 smithi077 ceph-mon[30319]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:46:39.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:38 smithi134 ceph-mon[35674]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:46:40.923 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:40 smithi120 ceph-mon[34770]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:46:40.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:40 smithi077 ceph-mon[30319]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:46:41.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:40 smithi134 ceph-mon[35674]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:46:42.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:42 smithi120 ceph-mon[34770]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:46:42.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:42 smithi077 ceph-mon[30319]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:46:43.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:42 smithi134 ceph-mon[35674]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:46:43.874 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:43 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-10T09:46:43.874 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:43 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:46:43.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:43 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-10T09:46:43.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:43 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:46:44.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:43 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-10T09:46:44.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:43 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:46:44.800 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:44 smithi120 ceph-mon[34770]: Deploying daemon osd.1 on smithi120 2024-09-10T09:46:44.800 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:44 smithi120 ceph-mon[34770]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:46:44.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:44 smithi077 ceph-mon[30319]: Deploying daemon osd.1 on smithi120 2024-09-10T09:46:44.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:44 smithi077 ceph-mon[30319]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:46:45.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:44 smithi134 ceph-mon[35674]: Deploying daemon osd.1 on smithi120 2024-09-10T09:46:45.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:44 smithi134 ceph-mon[35674]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:46:46.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:46 smithi120 ceph-mon[34770]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:46:46.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:46 smithi077 ceph-mon[30319]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:46:47.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:46 smithi134 ceph-mon[35674]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:46:48.923 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:48 smithi120 ceph-mon[34770]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:46:48.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:48 smithi077 ceph-mon[30319]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:46:49.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:48 smithi134 ceph-mon[35674]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:46:49.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:49 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:46:49.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:49 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:49.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:49 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:49.970 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:49 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:46:49.970 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:49 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:49.970 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:49 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:50.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:49 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:46:50.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:49 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:50.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:49 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:50.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:50 smithi134 ceph-mon[35674]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:46:50.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:50 smithi077 ceph-mon[30319]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:46:50.973 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:50 smithi120 ceph-mon[34770]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:46:52.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:52 smithi077 ceph-mon[30319]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:46:52.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:52 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:52.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:52 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:52.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:52 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:46:52.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:52 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:46:52.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:52 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:53.036 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:52 smithi120 ceph-mon[34770]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:46:53.036 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:52 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:53.036 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:52 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:53.036 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:52 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:46:53.036 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:52 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:46:53.036 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:52 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:53.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:52 smithi134 ceph-mon[35674]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:46:53.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:52 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:53.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:52 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:53.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:52 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:46:53.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:52 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:46:53.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:52 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:53.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:46:53.654+0000 7ffa8bfff640 1 -- 172.21.15.120:0/3587626522 <== mgr.14150 v2:172.21.15.77:6800/4261324650 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ffaa406a6f0 con 0x7ffa8c0a2ba0 2024-09-10T09:46:53.656 INFO:teuthology.orchestra.run.smithi120.stdout:Created osd(s) 1 on host 'smithi120' 2024-09-10T09:46:53.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:46:53.657+0000 7ffaa909e640 1 -- 172.21.15.120:0/3587626522 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ffa8c0a2ba0 msgr2=0x7ffa8c0a5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:53.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:46:53.657+0000 7ffaa909e640 1 --2- 172.21.15.120:0/3587626522 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ffa8c0a2ba0 0x7ffa8c0a5060 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ffa9402aa00 tx=0x7ffa940315b0 comp rx=0 tx=0).stop 2024-09-10T09:46:53.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:46:53.657+0000 7ffaa909e640 1 -- 172.21.15.120:0/3587626522 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffaa4101cd0 msgr2=0x7ffaa419a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:53.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:46:53.657+0000 7ffaa909e640 1 --2- 172.21.15.120:0/3587626522 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffaa4101cd0 0x7ffaa419a6d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ffa9800b4a0 tx=0x7ffa9800b960 comp rx=0 tx=0).stop 2024-09-10T09:46:53.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:46:53.657+0000 7ffaa909e640 1 -- 172.21.15.120:0/3587626522 shutdown_connections 2024-09-10T09:46:53.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:46:53.657+0000 7ffaa909e640 1 --2- 172.21.15.120:0/3587626522 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ffa8c0a2ba0 0x7ffa8c0a5060 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:53.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:46:53.657+0000 7ffaa909e640 1 --2- 172.21.15.120:0/3587626522 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffaa4102690 0x7ffaa41a1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:53.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:46:53.657+0000 7ffaa909e640 1 --2- 172.21.15.120:0/3587626522 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffaa4101cd0 0x7ffaa419a6d0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:53.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:46:53.657+0000 7ffaa909e640 1 --2- 172.21.15.120:0/3587626522 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffaa4100ad0 0x7ffaa419a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:53.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:46:53.657+0000 7ffaa909e640 1 -- 172.21.15.120:0/3587626522 >> 172.21.15.120:0/3587626522 conn(0x7ffaa40fc280 msgr2=0x7ffaa4069fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:46:53.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:46:53.657+0000 7ffaa909e640 1 -- 172.21.15.120:0/3587626522 shutdown_connections 2024-09-10T09:46:53.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:46:53.657+0000 7ffaa909e640 1 -- 172.21.15.120:0/3587626522 wait complete. 2024-09-10T09:46:53.923 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:53 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:46:53.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:53 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:53.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:53 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:54.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:53 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:46:54.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:53 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:54.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:53 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:54.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:53 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:46:54.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:53 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:54.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:53 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:54.317 DEBUG:teuthology.orchestra.run.smithi120:osd.1> sudo journalctl -f -n 0 -u ceph-14a55226-6f59-11ef-bcea-c7b262605968@osd.1.service 2024-09-10T09:46:54.320 INFO:tasks.cephadm:Deploying osd.2 on smithi134 with /dev/vg_nvme/lv_4... 2024-09-10T09:46:54.320 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-10T09:46:54.572 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T09:46:54.861 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:54 smithi120 ceph-mon[34770]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:46:54.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:54 smithi077 ceph-mon[30319]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:46:54.974 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:54 smithi134 ceph-mon[35674]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:46:55.673 INFO:journalctl@ceph.osd.1.smithi120.stdout:Sep 10 09:46:55 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-1[39249]: 2024-09-10T09:46:55.583+0000 7f81c872c640 -1 osd.1 0 log_to_monitors true 2024-09-10T09:46:55.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:55 smithi134 ceph-mon[35674]: from='osd.1 [v2:172.21.15.120:6800/4243920698,v1:172.21.15.120:6801/4243920698]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-10T09:46:55.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:55 smithi134 ceph-mon[35674]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-10T09:46:56.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:55 smithi120 ceph-mon[34770]: from='osd.1 [v2:172.21.15.120:6800/4243920698,v1:172.21.15.120:6801/4243920698]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-10T09:46:56.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:55 smithi120 ceph-mon[34770]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-10T09:46:56.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:55 smithi077 ceph-mon[30319]: from='osd.1 [v2:172.21.15.120:6800/4243920698,v1:172.21.15.120:6801/4243920698]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-10T09:46:56.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:55 smithi077 ceph-mon[30319]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-10T09:46:56.924 INFO:journalctl@ceph.osd.1.smithi120.stdout:Sep 10 09:46:56 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-1[39249]: 2024-09-10T09:46:56.705+0000 7f81c46af640 -1 osd.1 0 waiting for initial osdmap 2024-09-10T09:46:56.924 INFO:journalctl@ceph.osd.1.smithi120.stdout:Sep 10 09:46:56 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-1[39249]: 2024-09-10T09:46:56.709+0000 7f81c04d7640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T09:46:56.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:56 smithi120 ceph-mon[34770]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:46:56.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:56 smithi120 ceph-mon[34770]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-10T09:46:56.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:56 smithi120 ceph-mon[34770]: from='osd.1 [v2:172.21.15.120:6800/4243920698,v1:172.21.15.120:6801/4243920698]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-09-10T09:46:56.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:56 smithi120 ceph-mon[34770]: osdmap e11: 2 total, 1 up, 2 in 2024-09-10T09:46:56.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:56 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T09:46:56.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:56 smithi120 ceph-mon[34770]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-09-10T09:46:56.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:56 smithi134 ceph-mon[35674]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:46:56.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:56 smithi134 ceph-mon[35674]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-10T09:46:56.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:56 smithi134 ceph-mon[35674]: from='osd.1 [v2:172.21.15.120:6800/4243920698,v1:172.21.15.120:6801/4243920698]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-09-10T09:46:56.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:56 smithi134 ceph-mon[35674]: osdmap e11: 2 total, 1 up, 2 in 2024-09-10T09:46:56.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:56 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T09:46:56.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:56 smithi134 ceph-mon[35674]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-09-10T09:46:56.929 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-10T09:46:56.955 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi134:vg_nvme/lv_4 2024-09-10T09:46:57.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:56 smithi077 ceph-mon[30319]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:46:57.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:56 smithi077 ceph-mon[30319]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-10T09:46:57.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:56 smithi077 ceph-mon[30319]: from='osd.1 [v2:172.21.15.120:6800/4243920698,v1:172.21.15.120:6801/4243920698]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-09-10T09:46:57.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:56 smithi077 ceph-mon[30319]: osdmap e11: 2 total, 1 up, 2 in 2024-09-10T09:46:57.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:56 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T09:46:57.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:56 smithi077 ceph-mon[30319]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-09-10T09:46:57.203 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T09:46:57.656 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:46:57.653+0000 7fc58550a640 1 -- 172.21.15.134:0/170528530 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc580100b10 msgr2=0x7fc580100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:57.656 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:46:57.653+0000 7fc58550a640 1 --2- 172.21.15.134:0/170528530 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc580100b10 0x7fc580100f10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc574009900 tx=0x7fc57402a410 comp rx=0 tx=0).stop 2024-09-10T09:46:57.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:46:57.653+0000 7fc58550a640 1 -- 172.21.15.134:0/170528530 shutdown_connections 2024-09-10T09:46:57.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:46:57.653+0000 7fc58550a640 1 --2- 172.21.15.134:0/170528530 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc5801026d0 0x7fc580108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:57.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:46:57.653+0000 7fc58550a640 1 --2- 172.21.15.134:0/170528530 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc580101d10 0x7fc580102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:57.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:46:57.653+0000 7fc58550a640 1 --2- 172.21.15.134:0/170528530 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc580100b10 0x7fc580100f10 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:57.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:46:57.653+0000 7fc58550a640 1 -- 172.21.15.134:0/170528530 >> 172.21.15.134:0/170528530 conn(0x7fc5800fc2a0 msgr2=0x7fc5800fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:46:57.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:46:57.653+0000 7fc58550a640 1 -- 172.21.15.134:0/170528530 shutdown_connections 2024-09-10T09:46:57.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:46:57.653+0000 7fc58550a640 1 -- 172.21.15.134:0/170528530 wait complete. 2024-09-10T09:46:57.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:46:57.654+0000 7fc58550a640 1 Processor -- start 2024-09-10T09:46:57.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:46:57.654+0000 7fc58550a640 1 -- start start 2024-09-10T09:46:57.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:46:57.654+0000 7fc58550a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc580100b10 0x7fc58010b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:57.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:46:57.654+0000 7fc58550a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc580101d10 0x7fc580109ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:57.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:46:57.654+0000 7fc58550a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc5801026d0 0x7fc58010a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:57.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:46:57.654+0000 7fc58550a640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc58010d980 con 0x7fc580100b10 2024-09-10T09:46:57.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:46:57.654+0000 7fc58550a640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc58010d800 con 0x7fc5801026d0 2024-09-10T09:46:57.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:46:57.654+0000 7fc58550a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc58010db00 con 0x7fc580101d10 2024-09-10T09:46:57.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:46:57.654+0000 7fc57e7fc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc580101d10 0x7fc580109ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:57.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:46:57.654+0000 7fc57e7fc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc580101d10 0x7fc580109ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:40500/0 (socket says 172.21.15.134:40500) 2024-09-10T09:46:57.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:46:57.654+0000 7fc57e7fc640 1 -- 172.21.15.134:0/1769674570 learned_addr learned my addr 172.21.15.134:0/1769674570 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-10T09:46:57.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:46:57.654+0000 7fc57effd640 1 --2- 172.21.15.134:0/1769674570 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc580100b10 0x7fc58010b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:57.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:46:57.654+0000 7fc57f7fe640 1 --2- 172.21.15.134:0/1769674570 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc5801026d0 0x7fc58010a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:57.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:46:57.655+0000 7fc57e7fc640 1 -- 172.21.15.134:0/1769674570 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc5801026d0 msgr2=0x7fc58010a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:57.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:46:57.655+0000 7fc57e7fc640 1 --2- 172.21.15.134:0/1769674570 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc5801026d0 0x7fc58010a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:57.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:46:57.655+0000 7fc57e7fc640 1 -- 172.21.15.134:0/1769674570 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc580100b10 msgr2=0x7fc58010b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:57.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:46:57.655+0000 7fc57e7fc640 1 --2- 172.21.15.134:0/1769674570 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc580100b10 0x7fc58010b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:57.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:46:57.655+0000 7fc57e7fc640 1 -- 172.21.15.134:0/1769674570 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc58010a8e0 con 0x7fc580101d10 2024-09-10T09:46:57.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:46:57.655+0000 7fc57e7fc640 1 --2- 172.21.15.134:0/1769674570 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc580101d10 0x7fc580109ae0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc57000b4a0 tx=0x7fc57000b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:46:57.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:46:57.655+0000 7fc567fff640 1 -- 172.21.15.134:0/1769674570 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc570014070 con 0x7fc580101d10 2024-09-10T09:46:57.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:46:57.655+0000 7fc58550a640 1 -- 172.21.15.134:0/1769674570 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5801a2220 con 0x7fc580101d10 2024-09-10T09:46:57.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:46:57.655+0000 7fc567fff640 1 -- 172.21.15.134:0/1769674570 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc570010af0 con 0x7fc580101d10 2024-09-10T09:46:57.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:46:57.655+0000 7fc567fff640 1 -- 172.21.15.134:0/1769674570 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc570017d10 con 0x7fc580101d10 2024-09-10T09:46:57.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:46:57.655+0000 7fc58550a640 1 -- 172.21.15.134:0/1769674570 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5801a27b0 con 0x7fc580101d10 2024-09-10T09:46:57.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:46:57.656+0000 7fc58550a640 1 -- 172.21.15.134:0/1769674570 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc580108000 con 0x7fc580101d10 2024-09-10T09:46:57.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:46:57.657+0000 7fc567fff640 1 -- 172.21.15.134:0/1769674570 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7fc570017070 con 0x7fc580101d10 2024-09-10T09:46:57.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:46:57.658+0000 7fc567fff640 1 --2- 172.21.15.134:0/1769674570 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fc5680a2b70 0x7fc5680a5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:57.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:46:57.658+0000 7fc567fff640 1 -- 172.21.15.134:0/1769674570 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2029+0+0 (secure 0 0 0) 0x7fc5700cc950 con 0x7fc580101d10 2024-09-10T09:46:57.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:46:57.658+0000 7fc57effd640 1 --2- 172.21.15.134:0/1769674570 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fc5680a2b70 0x7fc5680a5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:57.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:46:57.659+0000 7fc57effd640 1 --2- 172.21.15.134:0/1769674570 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fc5680a2b70 0x7fc5680a5030 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc574005ab0 tx=0x7fc574000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:46:57.663 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:46:57.662+0000 7fc567fff640 1 -- 172.21.15.134:0/1769674570 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc57009abd0 con 0x7fc580101d10 2024-09-10T09:46:57.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:46:57.837+0000 7fc58550a640 1 -- 172.21.15.134:0/1769674570 --> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fc580062630 con 0x7fc5680a2b70 2024-09-10T09:46:57.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:57 smithi134 ceph-mon[35674]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2024-09-10T09:46:57.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:57 smithi134 ceph-mon[35674]: osdmap e12: 2 total, 1 up, 2 in 2024-09-10T09:46:57.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:57 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T09:46:57.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:57 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T09:46:57.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:57 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:57.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:57 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:57.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:57 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:46:57.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:57 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:57.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:57 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:46:57.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:57 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:46:57.882 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:57 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:58.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:57 smithi120 ceph-mon[34770]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2024-09-10T09:46:58.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:57 smithi120 ceph-mon[34770]: osdmap e12: 2 total, 1 up, 2 in 2024-09-10T09:46:58.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:57 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T09:46:58.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:57 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T09:46:58.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:57 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:58.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:57 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:58.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:57 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:46:58.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:57 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:58.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:57 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:46:58.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:57 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:46:58.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:57 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:58.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:57 smithi077 ceph-mon[30319]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2024-09-10T09:46:58.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:57 smithi077 ceph-mon[30319]: osdmap e12: 2 total, 1 up, 2 in 2024-09-10T09:46:58.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:57 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T09:46:58.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:57 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T09:46:58.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:57 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:58.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:57 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:58.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:57 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:46:58.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:57 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:58.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:57 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:46:58.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:57 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:46:58.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:57 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:46:58.876 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:58 smithi134 ceph-mon[35674]: purged_snaps scrub starts 2024-09-10T09:46:58.876 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:58 smithi134 ceph-mon[35674]: purged_snaps scrub ok 2024-09-10T09:46:58.876 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:58 smithi134 ceph-mon[35674]: Detected new or changed devices on smithi120 2024-09-10T09:46:58.877 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:58 smithi134 ceph-mon[35674]: Adjusting osd_memory_target on smithi120 to 17088M 2024-09-10T09:46:58.877 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:58 smithi134 ceph-mon[35674]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:46:58.877 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:58 smithi134 ceph-mon[35674]: osd.1 [v2:172.21.15.120:6800/4243920698,v1:172.21.15.120:6801/4243920698] boot 2024-09-10T09:46:58.877 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:58 smithi134 ceph-mon[35674]: osdmap e13: 2 total, 2 up, 2 in 2024-09-10T09:46:58.877 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:58 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T09:46:58.877 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:58 smithi134 ceph-mon[35674]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:46:58.877 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:58 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:46:58.877 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:58 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:46:58.877 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:46:58 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:46:59.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:58 smithi120 ceph-mon[34770]: purged_snaps scrub starts 2024-09-10T09:46:59.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:58 smithi120 ceph-mon[34770]: purged_snaps scrub ok 2024-09-10T09:46:59.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:58 smithi120 ceph-mon[34770]: Detected new or changed devices on smithi120 2024-09-10T09:46:59.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:58 smithi120 ceph-mon[34770]: Adjusting osd_memory_target on smithi120 to 17088M 2024-09-10T09:46:59.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:58 smithi120 ceph-mon[34770]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:46:59.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:58 smithi120 ceph-mon[34770]: osd.1 [v2:172.21.15.120:6800/4243920698,v1:172.21.15.120:6801/4243920698] boot 2024-09-10T09:46:59.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:58 smithi120 ceph-mon[34770]: osdmap e13: 2 total, 2 up, 2 in 2024-09-10T09:46:59.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:58 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T09:46:59.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:58 smithi120 ceph-mon[34770]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:46:59.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:58 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:46:59.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:58 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:46:59.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:46:58 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:46:59.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:58 smithi077 ceph-mon[30319]: purged_snaps scrub starts 2024-09-10T09:46:59.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:58 smithi077 ceph-mon[30319]: purged_snaps scrub ok 2024-09-10T09:46:59.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:58 smithi077 ceph-mon[30319]: Detected new or changed devices on smithi120 2024-09-10T09:46:59.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:58 smithi077 ceph-mon[30319]: Adjusting osd_memory_target on smithi120 to 17088M 2024-09-10T09:46:59.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:58 smithi077 ceph-mon[30319]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:46:59.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:58 smithi077 ceph-mon[30319]: osd.1 [v2:172.21.15.120:6800/4243920698,v1:172.21.15.120:6801/4243920698] boot 2024-09-10T09:46:59.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:58 smithi077 ceph-mon[30319]: osdmap e13: 2 total, 2 up, 2 in 2024-09-10T09:46:59.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:58 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T09:46:59.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:58 smithi077 ceph-mon[30319]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:46:59.191 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:58 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:46:59.191 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:58 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:46:59.191 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:46:58 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:47:00.361 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:00 smithi134 ceph-mon[35674]: osdmap e14: 2 total, 2 up, 2 in 2024-09-10T09:47:00.361 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:00 smithi134 ceph-mon[35674]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:47:00.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:00 smithi120 ceph-mon[34770]: osdmap e14: 2 total, 2 up, 2 in 2024-09-10T09:47:00.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:00 smithi120 ceph-mon[34770]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:47:00.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:00 smithi077 ceph-mon[30319]: osdmap e14: 2 total, 2 up, 2 in 2024-09-10T09:47:00.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:00 smithi077 ceph-mon[30319]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:47:01.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:01 smithi134 ceph-mon[35674]: from='client.? 172.21.15.134:0/3524596030' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5e83fe13-d1ee-483e-a575-e8041ec836f3"}]: dispatch 2024-09-10T09:47:01.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:01 smithi134 ceph-mon[35674]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5e83fe13-d1ee-483e-a575-e8041ec836f3"}]: dispatch 2024-09-10T09:47:01.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:01 smithi134 ceph-mon[35674]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5e83fe13-d1ee-483e-a575-e8041ec836f3"}]': finished 2024-09-10T09:47:01.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:01 smithi134 ceph-mon[35674]: osdmap e15: 3 total, 2 up, 3 in 2024-09-10T09:47:01.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:01 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T09:47:01.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:01 smithi134 ceph-mon[35674]: from='client.? 172.21.15.134:0/1941102769' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:47:01.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:01 smithi120 ceph-mon[34770]: from='client.? 172.21.15.134:0/3524596030' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5e83fe13-d1ee-483e-a575-e8041ec836f3"}]: dispatch 2024-09-10T09:47:01.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:01 smithi120 ceph-mon[34770]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5e83fe13-d1ee-483e-a575-e8041ec836f3"}]: dispatch 2024-09-10T09:47:01.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:01 smithi120 ceph-mon[34770]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5e83fe13-d1ee-483e-a575-e8041ec836f3"}]': finished 2024-09-10T09:47:01.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:01 smithi120 ceph-mon[34770]: osdmap e15: 3 total, 2 up, 3 in 2024-09-10T09:47:01.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:01 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T09:47:01.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:01 smithi120 ceph-mon[34770]: from='client.? 172.21.15.134:0/1941102769' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:47:01.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:01 smithi077 ceph-mon[30319]: from='client.? 172.21.15.134:0/3524596030' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5e83fe13-d1ee-483e-a575-e8041ec836f3"}]: dispatch 2024-09-10T09:47:01.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:01 smithi077 ceph-mon[30319]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5e83fe13-d1ee-483e-a575-e8041ec836f3"}]: dispatch 2024-09-10T09:47:01.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:01 smithi077 ceph-mon[30319]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5e83fe13-d1ee-483e-a575-e8041ec836f3"}]': finished 2024-09-10T09:47:01.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:01 smithi077 ceph-mon[30319]: osdmap e15: 3 total, 2 up, 3 in 2024-09-10T09:47:01.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:01 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T09:47:01.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:01 smithi077 ceph-mon[30319]: from='client.? 172.21.15.134:0/1941102769' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:47:02.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:02 smithi134 ceph-mon[35674]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:47:02.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:02 smithi120 ceph-mon[34770]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:47:02.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:02 smithi077 ceph-mon[30319]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:47:04.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:04 smithi134 ceph-mon[35674]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:47:04.923 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:04 smithi120 ceph-mon[34770]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:47:04.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:04 smithi077 ceph-mon[30319]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:47:06.908 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:06 smithi134 ceph-mon[35674]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:47:06.923 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:06 smithi120 ceph-mon[34770]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:47:06.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:06 smithi077 ceph-mon[30319]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:47:08.851 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:08 smithi134 ceph-mon[35674]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:47:08.851 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:08 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-10T09:47:08.851 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:08 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:47:08.923 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:08 smithi120 ceph-mon[34770]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:47:08.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:08 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-10T09:47:08.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:08 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:47:08.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:08 smithi077 ceph-mon[30319]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:47:08.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:08 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-10T09:47:08.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:08 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:47:09.779 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:09 smithi134 ceph-mon[35674]: Deploying daemon osd.2 on smithi134 2024-09-10T09:47:09.923 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:09 smithi120 ceph-mon[34770]: Deploying daemon osd.2 on smithi134 2024-09-10T09:47:09.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:09 smithi077 ceph-mon[30319]: Deploying daemon osd.2 on smithi134 2024-09-10T09:47:10.923 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:10 smithi120 ceph-mon[34770]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:47:10.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:10 smithi077 ceph-mon[30319]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:47:11.063 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:10 smithi134 ceph-mon[35674]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:47:12.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:12 smithi077 ceph-mon[30319]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:47:12.997 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:12 smithi120 ceph-mon[34770]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:47:13.089 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:12 smithi134 ceph-mon[35674]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:47:14.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:14 smithi134 ceph-mon[35674]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:47:14.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:14 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:47:14.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:14 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:47:14.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:14 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:47:14.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:14 smithi077 ceph-mon[30319]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:47:14.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:14 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:47:14.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:14 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:47:14.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:14 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:47:15.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:14 smithi120 ceph-mon[34770]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:47:15.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:14 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:47:15.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:14 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:47:15.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:14 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:47:16.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:16 smithi134 ceph-mon[35674]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:47:16.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:16 smithi077 ceph-mon[30319]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:47:17.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:16 smithi120 ceph-mon[34770]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:47:17.910 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:47:17.908+0000 7fc567fff640 1 -- 172.21.15.134:0/1769674570 <== mgr.14150 v2:172.21.15.77:6800/4261324650 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc580062630 con 0x7fc5680a2b70 2024-09-10T09:47:17.910 INFO:teuthology.orchestra.run.smithi134.stdout:Created osd(s) 2 on host 'smithi134' 2024-09-10T09:47:17.912 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:47:17.911+0000 7fc58550a640 1 -- 172.21.15.134:0/1769674570 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fc5680a2b70 msgr2=0x7fc5680a5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:17.912 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:47:17.911+0000 7fc58550a640 1 --2- 172.21.15.134:0/1769674570 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fc5680a2b70 0x7fc5680a5030 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc574005ab0 tx=0x7fc574000bc0 comp rx=0 tx=0).stop 2024-09-10T09:47:17.912 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:47:17.911+0000 7fc58550a640 1 -- 172.21.15.134:0/1769674570 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc580101d10 msgr2=0x7fc580109ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:17.912 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:47:17.911+0000 7fc58550a640 1 --2- 172.21.15.134:0/1769674570 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc580101d10 0x7fc580109ae0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc57000b4a0 tx=0x7fc57000b960 comp rx=0 tx=0).stop 2024-09-10T09:47:17.913 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:47:17.911+0000 7fc58550a640 1 -- 172.21.15.134:0/1769674570 shutdown_connections 2024-09-10T09:47:17.913 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:47:17.911+0000 7fc58550a640 1 --2- 172.21.15.134:0/1769674570 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fc5680a2b70 0x7fc5680a5030 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:17.913 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:47:17.911+0000 7fc58550a640 1 --2- 172.21.15.134:0/1769674570 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc5801026d0 0x7fc58010a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:17.913 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:47:17.911+0000 7fc58550a640 1 --2- 172.21.15.134:0/1769674570 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc580101d10 0x7fc580109ae0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:17.913 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:47:17.911+0000 7fc58550a640 1 --2- 172.21.15.134:0/1769674570 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc580100b10 0x7fc58010b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:17.913 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:47:17.911+0000 7fc58550a640 1 -- 172.21.15.134:0/1769674570 >> 172.21.15.134:0/1769674570 conn(0x7fc5800fc2a0 msgr2=0x7fc5801051d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:47:17.913 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:47:17.911+0000 7fc58550a640 1 -- 172.21.15.134:0/1769674570 shutdown_connections 2024-09-10T09:47:17.913 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:47:17.911+0000 7fc58550a640 1 -- 172.21.15.134:0/1769674570 wait complete. 2024-09-10T09:47:18.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:18 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:47:18.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:18 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:47:18.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:18 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:47:18.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:18 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:47:18.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:18 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:47:18.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:18 smithi134 ceph-mon[35674]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:47:18.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:18 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:47:18.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:18 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:47:18.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:18 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:47:18.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:18 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:47:18.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:18 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:47:18.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:18 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:47:18.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:18 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:47:18.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:18 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:47:18.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:18 smithi120 ceph-mon[34770]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:47:18.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:18 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:47:18.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:18 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:47:18.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:18 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:47:18.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:18 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:47:18.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:18 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:47:18.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:18 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:47:18.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:18 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:47:18.440 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:18 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:47:18.440 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:18 smithi077 ceph-mon[30319]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:47:18.440 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:18 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:47:18.440 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:18 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:47:18.440 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:18 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:47:18.558 DEBUG:teuthology.orchestra.run.smithi134:osd.2> sudo journalctl -f -n 0 -u ceph-14a55226-6f59-11ef-bcea-c7b262605968@osd.2.service 2024-09-10T09:47:18.560 INFO:tasks.cephadm:Deploying osd.3 on smithi077 with /dev/vg_nvme/lv_3... 2024-09-10T09:47:18.560 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-10T09:47:18.926 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:47:20.808 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:20 smithi134 ceph-mon[35674]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:47:20.808 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:20 smithi134 ceph-mon[35674]: from='osd.2 [v2:172.21.15.134:6800/3280466823,v1:172.21.15.134:6801/3280466823]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T09:47:20.808 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:20 smithi134 ceph-mon[35674]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T09:47:20.808 INFO:journalctl@ceph.osd.2.smithi134.stdout:Sep 10 09:47:20 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-2[40655]: 2024-09-10T09:47:20.476+0000 7f3e6ea57640 -1 osd.2 0 log_to_monitors true 2024-09-10T09:47:20.923 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:20 smithi120 ceph-mon[34770]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:47:20.923 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:20 smithi120 ceph-mon[34770]: from='osd.2 [v2:172.21.15.134:6800/3280466823,v1:172.21.15.134:6801/3280466823]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T09:47:20.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:20 smithi120 ceph-mon[34770]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T09:47:20.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:20 smithi077 ceph-mon[30319]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:47:20.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:20 smithi077 ceph-mon[30319]: from='osd.2 [v2:172.21.15.134:6800/3280466823,v1:172.21.15.134:6801/3280466823]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T09:47:20.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:20 smithi077 ceph-mon[30319]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T09:47:21.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:21 smithi120 ceph-mon[34770]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-10T09:47:21.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:21 smithi120 ceph-mon[34770]: osdmap e16: 3 total, 2 up, 3 in 2024-09-10T09:47:21.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:21 smithi120 ceph-mon[34770]: from='osd.2 [v2:172.21.15.134:6800/3280466823,v1:172.21.15.134:6801/3280466823]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-09-10T09:47:21.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:21 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T09:47:21.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:21 smithi120 ceph-mon[34770]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-09-10T09:47:21.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:21 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:47:21.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:21 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:47:21.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:21 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:47:21.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:21 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:47:21.925 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:21 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:47:21.925 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:21 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:47:21.925 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:21 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:47:21.925 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:21 smithi120 ceph-mon[34770]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-09-10T09:47:21.925 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:21 smithi120 ceph-mon[34770]: osdmap e17: 3 total, 2 up, 3 in 2024-09-10T09:47:21.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:21 smithi077 ceph-mon[30319]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-10T09:47:21.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:21 smithi077 ceph-mon[30319]: osdmap e16: 3 total, 2 up, 3 in 2024-09-10T09:47:21.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:21 smithi077 ceph-mon[30319]: from='osd.2 [v2:172.21.15.134:6800/3280466823,v1:172.21.15.134:6801/3280466823]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-09-10T09:47:21.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:21 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T09:47:21.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:21 smithi077 ceph-mon[30319]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-09-10T09:47:21.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:21 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:47:21.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:21 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:47:21.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:21 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:47:21.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:21 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:47:21.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:21 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:47:21.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:21 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:47:21.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:21 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:47:21.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:21 smithi077 ceph-mon[30319]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-09-10T09:47:21.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:21 smithi077 ceph-mon[30319]: osdmap e17: 3 total, 2 up, 3 in 2024-09-10T09:47:21.980 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:47:22.011 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi077:vg_nvme/lv_3 2024-09-10T09:47:22.043 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:21 smithi134 ceph-mon[35674]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-10T09:47:22.043 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:21 smithi134 ceph-mon[35674]: osdmap e16: 3 total, 2 up, 3 in 2024-09-10T09:47:22.043 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:21 smithi134 ceph-mon[35674]: from='osd.2 [v2:172.21.15.134:6800/3280466823,v1:172.21.15.134:6801/3280466823]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-09-10T09:47:22.043 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:21 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T09:47:22.043 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:21 smithi134 ceph-mon[35674]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-09-10T09:47:22.043 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:21 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:47:22.043 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:21 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:47:22.044 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:21 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:47:22.044 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:21 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:47:22.044 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:21 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:47:22.044 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:21 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:47:22.044 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:21 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:47:22.044 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:21 smithi134 ceph-mon[35674]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-09-10T09:47:22.044 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:21 smithi134 ceph-mon[35674]: osdmap e17: 3 total, 2 up, 3 in 2024-09-10T09:47:22.372 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:47:22.381 INFO:journalctl@ceph.osd.2.smithi134.stdout:Sep 10 09:47:22 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-2[40655]: 2024-09-10T09:47:22.040+0000 7f3e6a9da640 -1 osd.2 0 waiting for initial osdmap 2024-09-10T09:47:22.381 INFO:journalctl@ceph.osd.2.smithi134.stdout:Sep 10 09:47:22 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-2[40655]: 2024-09-10T09:47:22.046+0000 7f3e627fa640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T09:47:22.859 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:22 smithi077 ceph-mon[30319]: Detected new or changed devices on smithi134 2024-09-10T09:47:22.859 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:22 smithi077 ceph-mon[30319]: Adjusting osd_memory_target on smithi134 to 17088M 2024-09-10T09:47:22.859 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:22 smithi077 ceph-mon[30319]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:47:22.859 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:22 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T09:47:22.859 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:22 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T09:47:22.859 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:22 smithi077 ceph-mon[30319]: from='osd.2 ' entity='osd.2' 2024-09-10T09:47:22.923 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:22 smithi120 ceph-mon[34770]: Detected new or changed devices on smithi134 2024-09-10T09:47:22.923 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:22 smithi120 ceph-mon[34770]: Adjusting osd_memory_target on smithi134 to 17088M 2024-09-10T09:47:22.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:22 smithi120 ceph-mon[34770]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:47:22.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:22 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T09:47:22.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:22 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T09:47:22.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:22 smithi120 ceph-mon[34770]: from='osd.2 ' entity='osd.2' 2024-09-10T09:47:22.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:47:22.971+0000 7f2f27000640 1 -- 172.21.15.77:0/178089529 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2f20104e50 msgr2=0x7f2f201052d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:22.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:47:22.971+0000 7f2f27000640 1 --2- 172.21.15.77:0/178089529 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2f20104e50 0x7f2f201052d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c009900 tx=0x7f2f0c02a3f0 comp rx=0 tx=0).stop 2024-09-10T09:47:22.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:47:22.971+0000 7f2f27000640 1 -- 172.21.15.77:0/178089529 shutdown_connections 2024-09-10T09:47:22.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:47:22.971+0000 7f2f27000640 1 --2- 172.21.15.77:0/178089529 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2f20105810 0x7f2f2010c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:22.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:47:22.971+0000 7f2f27000640 1 --2- 172.21.15.77:0/178089529 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2f20104e50 0x7f2f201052d0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:22.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:47:22.971+0000 7f2f27000640 1 --2- 172.21.15.77:0/178089529 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2f20103c50 0x7f2f20104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:22.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:47:22.971+0000 7f2f27000640 1 -- 172.21.15.77:0/178089529 >> 172.21.15.77:0/178089529 conn(0x7f2f200ff3e0 msgr2=0x7f2f20101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:47:22.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:47:22.971+0000 7f2f27000640 1 -- 172.21.15.77:0/178089529 shutdown_connections 2024-09-10T09:47:22.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:47:22.971+0000 7f2f27000640 1 -- 172.21.15.77:0/178089529 wait complete. 2024-09-10T09:47:22.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:47:22.972+0000 7f2f27000640 1 Processor -- start 2024-09-10T09:47:22.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:47:22.972+0000 7f2f27000640 1 -- start start 2024-09-10T09:47:22.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:47:22.973+0000 7f2f27000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2f20103c50 0x7f2f20199ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:22.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:47:22.973+0000 7f2f27000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2f20104e50 0x7f2f2019a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:22.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:47:22.973+0000 7f2f27000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2f20105810 0x7f2f201a14b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:22.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:47:22.973+0000 7f2f27000640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2f2010f920 con 0x7f2f20104e50 2024-09-10T09:47:22.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:47:22.973+0000 7f2f27000640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2f2010f7a0 con 0x7f2f20105810 2024-09-10T09:47:22.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:47:22.973+0000 7f2f27000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2f2010faa0 con 0x7f2f20103c50 2024-09-10T09:47:22.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:47:22.973+0000 7f2f25200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2f20104e50 0x7f2f2019a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:22.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:47:22.973+0000 7f2f25200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2f20104e50 0x7f2f2019a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:39948/0 (socket says 172.21.15.77:39948) 2024-09-10T09:47:22.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:47:22.973+0000 7f2f25200640 1 -- 172.21.15.77:0/250691927 learned_addr learned my addr 172.21.15.77:0/250691927 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:47:22.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:47:22.973+0000 7f2f25c00640 1 --2- 172.21.15.77:0/250691927 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2f20103c50 0x7f2f20199ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:22.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:47:22.973+0000 7f2f26600640 1 --2- 172.21.15.77:0/250691927 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2f20105810 0x7f2f201a14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:22.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:47:22.973+0000 7f2f25200640 1 -- 172.21.15.77:0/250691927 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2f20103c50 msgr2=0x7f2f20199ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:22.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:47:22.973+0000 7f2f25200640 1 --2- 172.21.15.77:0/250691927 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2f20103c50 0x7f2f20199ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:22.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:47:22.973+0000 7f2f25200640 1 -- 172.21.15.77:0/250691927 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2f20105810 msgr2=0x7f2f201a14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:22.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:47:22.973+0000 7f2f25200640 1 --2- 172.21.15.77:0/250691927 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2f20105810 0x7f2f201a14b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:22.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:47:22.973+0000 7f2f25200640 1 -- 172.21.15.77:0/250691927 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f201a1b20 con 0x7f2f20104e50 2024-09-10T09:47:22.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:47:22.974+0000 7f2f25200640 1 --2- 172.21.15.77:0/250691927 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2f20104e50 0x7f2f2019a430 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c031440 tx=0x7f2f0c032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:47:22.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:47:22.974+0000 7f2f1e000640 1 -- 172.21.15.77:0/250691927 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f0c03f030 con 0x7f2f20104e50 2024-09-10T09:47:22.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:47:22.974+0000 7f2f27000640 1 -- 172.21.15.77:0/250691927 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f201a1db0 con 0x7f2f20104e50 2024-09-10T09:47:22.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:47:22.974+0000 7f2f1e000640 1 -- 172.21.15.77:0/250691927 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f0c003ce0 con 0x7f2f20104e50 2024-09-10T09:47:22.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:47:22.974+0000 7f2f1e000640 1 -- 172.21.15.77:0/250691927 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f0c035070 con 0x7f2f20104e50 2024-09-10T09:47:22.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:47:22.974+0000 7f2f27000640 1 -- 172.21.15.77:0/250691927 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f201a2250 con 0x7f2f20104e50 2024-09-10T09:47:22.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:47:22.976+0000 7f2f27000640 1 -- 172.21.15.77:0/250691927 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f2010b000 con 0x7f2f20104e50 2024-09-10T09:47:22.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:47:22.977+0000 7f2f1e000640 1 -- 172.21.15.77:0/250691927 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f2f0c042030 con 0x7f2f20104e50 2024-09-10T09:47:22.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:47:22.979+0000 7f2f1e000640 1 --2- 172.21.15.77:0/250691927 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f2f080a2b20 0x7f2f080a4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:22.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:47:22.979+0000 7f2f1e000640 1 -- 172.21.15.77:0/250691927 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 2571+0+0 (secure 0 0 0) 0x7f2f0c0eda00 con 0x7f2f20104e50 2024-09-10T09:47:22.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:47:22.979+0000 7f2f25c00640 1 --2- 172.21.15.77:0/250691927 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f2f080a2b20 0x7f2f080a4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:22.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:47:22.980+0000 7f2f25c00640 1 --2- 172.21.15.77:0/250691927 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f2f080a2b20 0x7f2f080a4fe0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2f140095f0 tx=0x7f2f14005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:47:22.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:47:22.983+0000 7f2f1e000640 1 -- 172.21.15.77:0/250691927 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2f0c0bb9c0 con 0x7f2f20104e50 2024-09-10T09:47:23.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:22 smithi134 ceph-mon[35674]: Detected new or changed devices on smithi134 2024-09-10T09:47:23.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:22 smithi134 ceph-mon[35674]: Adjusting osd_memory_target on smithi134 to 17088M 2024-09-10T09:47:23.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:22 smithi134 ceph-mon[35674]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:47:23.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:22 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T09:47:23.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:22 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T09:47:23.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:22 smithi134 ceph-mon[35674]: from='osd.2 ' entity='osd.2' 2024-09-10T09:47:23.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:47:23.169+0000 7f2f27000640 1 -- 172.21.15.77:0/250691927 --> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f2f20062630 con 0x7f2f080a2b20 2024-09-10T09:47:24.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:24 smithi120 ceph-mon[34770]: purged_snaps scrub starts 2024-09-10T09:47:24.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:24 smithi120 ceph-mon[34770]: purged_snaps scrub ok 2024-09-10T09:47:24.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:24 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T09:47:24.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:24 smithi120 ceph-mon[34770]: osd.2 [v2:172.21.15.134:6800/3280466823,v1:172.21.15.134:6801/3280466823] boot 2024-09-10T09:47:24.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:24 smithi120 ceph-mon[34770]: osdmap e18: 3 total, 3 up, 3 in 2024-09-10T09:47:24.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:24 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T09:47:24.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:24 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:47:24.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:24 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:47:24.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:24 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:47:24.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:24 smithi077 ceph-mon[30319]: purged_snaps scrub starts 2024-09-10T09:47:24.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:24 smithi077 ceph-mon[30319]: purged_snaps scrub ok 2024-09-10T09:47:24.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:24 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T09:47:24.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:24 smithi077 ceph-mon[30319]: osd.2 [v2:172.21.15.134:6800/3280466823,v1:172.21.15.134:6801/3280466823] boot 2024-09-10T09:47:24.440 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:24 smithi077 ceph-mon[30319]: osdmap e18: 3 total, 3 up, 3 in 2024-09-10T09:47:24.440 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:24 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T09:47:24.440 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:24 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:47:24.440 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:24 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:47:24.440 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:24 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:47:24.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:24 smithi134 ceph-mon[35674]: purged_snaps scrub starts 2024-09-10T09:47:24.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:24 smithi134 ceph-mon[35674]: purged_snaps scrub ok 2024-09-10T09:47:24.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:24 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T09:47:24.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:24 smithi134 ceph-mon[35674]: osd.2 [v2:172.21.15.134:6800/3280466823,v1:172.21.15.134:6801/3280466823] boot 2024-09-10T09:47:24.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:24 smithi134 ceph-mon[35674]: osdmap e18: 3 total, 3 up, 3 in 2024-09-10T09:47:24.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:24 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T09:47:24.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:24 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:47:24.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:24 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:47:24.632 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:24 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:47:25.392 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:25 smithi077 ceph-mon[30319]: from='client.14262 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:47:25.392 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:25 smithi077 ceph-mon[30319]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:47:25.392 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:25 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-10T09:47:25.392 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:25 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-10T09:47:25.392 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:25 smithi077 ceph-mon[30319]: osdmap e19: 3 total, 3 up, 3 in 2024-09-10T09:47:25.392 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:25 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T09:47:25.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:25 smithi134 ceph-mon[35674]: from='client.14262 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:47:25.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:25 smithi134 ceph-mon[35674]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:47:25.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:25 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-10T09:47:25.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:25 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-10T09:47:25.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:25 smithi134 ceph-mon[35674]: osdmap e19: 3 total, 3 up, 3 in 2024-09-10T09:47:25.632 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:25 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T09:47:25.673 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:25 smithi120 ceph-mon[34770]: from='client.14262 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:47:25.673 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:25 smithi120 ceph-mon[34770]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:47:25.674 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:25 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-10T09:47:25.674 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:25 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-10T09:47:25.674 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:25 smithi120 ceph-mon[34770]: osdmap e19: 3 total, 3 up, 3 in 2024-09-10T09:47:25.674 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:25 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T09:47:26.189 INFO:journalctl@ceph.osd.0.smithi077.stdout:Sep 10 09:47:25 smithi077 sudo[48662]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T09:47:26.189 INFO:journalctl@ceph.osd.0.smithi077.stdout:Sep 10 09:47:25 smithi077 sudo[48662]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T09:47:26.189 INFO:journalctl@ceph.osd.0.smithi077.stdout:Sep 10 09:47:25 smithi077 sudo[48662]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T09:47:26.189 INFO:journalctl@ceph.osd.0.smithi077.stdout:Sep 10 09:47:25 smithi077 sudo[48662]: pam_unix(sudo:session): session closed for user root 2024-09-10T09:47:26.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:47:26.201+0000 7f2f1e000640 1 -- 172.21.15.77:0/250691927 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f2f0c03bcd0 con 0x7f2f20104e50 2024-09-10T09:47:26.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:26 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-10T09:47:26.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:26 smithi134 ceph-mon[35674]: osdmap e20: 3 total, 3 up, 3 in 2024-09-10T09:47:26.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:26 smithi134 ceph-mon[35674]: pgmap v83: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T09:47:26.638 INFO:journalctl@ceph.osd.0.smithi077.stdout:Sep 10 09:47:26 smithi077 sudo[48701]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T09:47:26.638 INFO:journalctl@ceph.osd.0.smithi077.stdout:Sep 10 09:47:26 smithi077 sudo[48701]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T09:47:26.639 INFO:journalctl@ceph.osd.0.smithi077.stdout:Sep 10 09:47:26 smithi077 sudo[48701]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T09:47:26.639 INFO:journalctl@ceph.osd.0.smithi077.stdout:Sep 10 09:47:26 smithi077 sudo[48701]: pam_unix(sudo:session): session closed for user root 2024-09-10T09:47:26.639 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:26 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-10T09:47:26.639 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:26 smithi077 ceph-mon[30319]: osdmap e20: 3 total, 3 up, 3 in 2024-09-10T09:47:26.639 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:26 smithi077 ceph-mon[30319]: pgmap v83: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T09:47:26.665 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:26 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-10T09:47:26.665 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:26 smithi120 ceph-mon[34770]: osdmap e20: 3 total, 3 up, 3 in 2024-09-10T09:47:26.665 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:26 smithi120 ceph-mon[34770]: pgmap v83: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T09:47:26.919 INFO:journalctl@ceph.osd.1.smithi120.stdout:Sep 10 09:47:26 smithi120 sudo[42950]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T09:47:26.919 INFO:journalctl@ceph.osd.1.smithi120.stdout:Sep 10 09:47:26 smithi120 sudo[42950]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T09:47:26.919 INFO:journalctl@ceph.osd.1.smithi120.stdout:Sep 10 09:47:26 smithi120 sudo[42950]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T09:47:26.919 INFO:journalctl@ceph.osd.1.smithi120.stdout:Sep 10 09:47:26 smithi120 sudo[42950]: pam_unix(sudo:session): session closed for user root 2024-09-10T09:47:27.173 INFO:journalctl@ceph.osd.1.smithi120.stdout:Sep 10 09:47:26 smithi120 sudo[42953]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T09:47:27.174 INFO:journalctl@ceph.osd.1.smithi120.stdout:Sep 10 09:47:26 smithi120 sudo[42953]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T09:47:27.174 INFO:journalctl@ceph.osd.1.smithi120.stdout:Sep 10 09:47:26 smithi120 sudo[42953]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T09:47:27.174 INFO:journalctl@ceph.osd.1.smithi120.stdout:Sep 10 09:47:26 smithi120 sudo[42953]: pam_unix(sudo:session): session closed for user root 2024-09-10T09:47:27.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:27 smithi077 ceph-mon[30319]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-10T09:47:27.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:27 smithi077 ceph-mon[30319]: osdmap e21: 3 total, 3 up, 3 in 2024-09-10T09:47:27.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:27 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/1969646715' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5affc6a7-1cae-48fe-b7db-befc2fcafecb"}]: dispatch 2024-09-10T09:47:27.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:27 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/1969646715' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5affc6a7-1cae-48fe-b7db-befc2fcafecb"}]': finished 2024-09-10T09:47:27.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:27 smithi077 ceph-mon[30319]: osdmap e22: 4 total, 3 up, 4 in 2024-09-10T09:47:27.440 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:27 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T09:47:27.440 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:27 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:47:27.440 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:27 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:47:27.440 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:27 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/4094564478' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:47:27.443 INFO:journalctl@ceph.osd.2.smithi134.stdout:Sep 10 09:47:27 smithi134 sudo[44725]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T09:47:27.443 INFO:journalctl@ceph.osd.2.smithi134.stdout:Sep 10 09:47:27 smithi134 sudo[44725]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T09:47:27.443 INFO:journalctl@ceph.osd.2.smithi134.stdout:Sep 10 09:47:27 smithi134 sudo[44725]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T09:47:27.443 INFO:journalctl@ceph.osd.2.smithi134.stdout:Sep 10 09:47:27 smithi134 sudo[44725]: pam_unix(sudo:session): session closed for user root 2024-09-10T09:47:27.444 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:27 smithi134 ceph-mon[35674]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-10T09:47:27.444 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:27 smithi134 ceph-mon[35674]: osdmap e21: 3 total, 3 up, 3 in 2024-09-10T09:47:27.444 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:27 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/1969646715' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5affc6a7-1cae-48fe-b7db-befc2fcafecb"}]: dispatch 2024-09-10T09:47:27.444 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:27 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/1969646715' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5affc6a7-1cae-48fe-b7db-befc2fcafecb"}]': finished 2024-09-10T09:47:27.444 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:27 smithi134 ceph-mon[35674]: osdmap e22: 4 total, 3 up, 4 in 2024-09-10T09:47:27.444 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:27 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T09:47:27.444 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:27 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:47:27.444 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:27 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:47:27.444 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:27 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/4094564478' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:47:27.673 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:27 smithi120 ceph-mon[34770]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-10T09:47:27.674 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:27 smithi120 ceph-mon[34770]: osdmap e21: 3 total, 3 up, 3 in 2024-09-10T09:47:27.674 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:27 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/1969646715' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5affc6a7-1cae-48fe-b7db-befc2fcafecb"}]: dispatch 2024-09-10T09:47:27.674 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:27 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/1969646715' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5affc6a7-1cae-48fe-b7db-befc2fcafecb"}]': finished 2024-09-10T09:47:27.674 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:27 smithi120 ceph-mon[34770]: osdmap e22: 4 total, 3 up, 4 in 2024-09-10T09:47:27.674 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:27 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T09:47:27.674 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:27 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:47:27.674 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:27 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:47:27.674 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:27 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/4094564478' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:47:27.881 INFO:journalctl@ceph.osd.2.smithi134.stdout:Sep 10 09:47:27 smithi134 sudo[44728]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T09:47:27.881 INFO:journalctl@ceph.osd.2.smithi134.stdout:Sep 10 09:47:27 smithi134 sudo[44728]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T09:47:27.881 INFO:journalctl@ceph.osd.2.smithi134.stdout:Sep 10 09:47:27 smithi134 sudo[44728]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T09:47:27.881 INFO:journalctl@ceph.osd.2.smithi134.stdout:Sep 10 09:47:27 smithi134 sudo[44728]: pam_unix(sudo:session): session closed for user root 2024-09-10T09:47:28.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:27 smithi077 sudo[48786]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T09:47:28.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:27 smithi077 sudo[48786]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T09:47:28.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:27 smithi077 sudo[48786]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T09:47:28.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:28 smithi077 sudo[48786]: pam_unix(sudo:session): session closed for user root 2024-09-10T09:47:28.812 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:28 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:47:28.812 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:28 smithi134 ceph-mon[35674]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T09:47:28.813 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:28 smithi134 sudo[44731]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T09:47:28.813 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:28 smithi134 sudo[44731]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T09:47:28.813 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:28 smithi134 sudo[44731]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T09:47:28.813 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:28 smithi134 sudo[44731]: pam_unix(sudo:session): session closed for user root 2024-09-10T09:47:28.923 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:28 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:47:28.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:28 smithi120 ceph-mon[34770]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T09:47:28.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:28 smithi077 sudo[49158]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T09:47:28.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:28 smithi077 sudo[49158]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T09:47:28.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:28 smithi077 sudo[49158]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T09:47:28.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:28 smithi077 sudo[49158]: pam_unix(sudo:session): session closed for user root 2024-09-10T09:47:28.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:28 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:47:28.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:28 smithi077 ceph-mon[30319]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T09:47:29.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:29 smithi134 sudo[44734]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T09:47:29.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:29 smithi134 sudo[44734]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T09:47:29.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:29 smithi134 sudo[44734]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T09:47:29.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:29 smithi134 sudo[44734]: pam_unix(sudo:session): session closed for user root 2024-09-10T09:47:29.547 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:29 smithi120 sudo[42956]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T09:47:29.548 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:29 smithi120 sudo[42956]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T09:47:29.548 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:29 smithi120 sudo[42956]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T09:47:29.548 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:29 smithi120 sudo[42956]: pam_unix(sudo:session): session closed for user root 2024-09-10T09:47:29.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:29 smithi134 ceph-mon[35674]: pgmap v86: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T09:47:29.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:29 smithi134 ceph-mon[35674]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T09:47:29.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:29 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T09:47:29.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:29 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:47:29.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:29 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T09:47:29.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:29 smithi134 ceph-mon[35674]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T09:47:29.882 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:29 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T09:47:29.882 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:29 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:47:29.882 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:29 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T09:47:29.882 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:29 smithi134 ceph-mon[35674]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T09:47:29.882 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:29 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T09:47:29.882 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:29 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:47:29.882 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:29 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T09:47:29.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:29 smithi120 sudo[42959]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T09:47:29.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:29 smithi120 sudo[42959]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T09:47:29.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:29 smithi120 sudo[42959]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T09:47:29.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:29 smithi120 sudo[42959]: pam_unix(sudo:session): session closed for user root 2024-09-10T09:47:29.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:29 smithi120 ceph-mon[34770]: pgmap v86: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T09:47:29.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:29 smithi120 ceph-mon[34770]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T09:47:29.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:29 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T09:47:29.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:29 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:47:29.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:29 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T09:47:29.925 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:29 smithi120 ceph-mon[34770]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T09:47:29.925 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:29 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T09:47:29.925 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:29 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:47:29.925 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:29 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T09:47:29.925 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:29 smithi120 ceph-mon[34770]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T09:47:29.925 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:29 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T09:47:29.925 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:29 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:47:29.925 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:29 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T09:47:29.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:29 smithi077 ceph-mon[30319]: pgmap v86: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T09:47:29.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:29 smithi077 ceph-mon[30319]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T09:47:29.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:29 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T09:47:29.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:29 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:47:29.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:29 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T09:47:29.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:29 smithi077 ceph-mon[30319]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T09:47:29.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:29 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T09:47:29.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:29 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:47:29.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:29 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T09:47:29.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:29 smithi077 ceph-mon[30319]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T09:47:29.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:29 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T09:47:29.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:29 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:47:29.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:29 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T09:47:30.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:30 smithi134 ceph-mon[35674]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T09:47:30.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:30 smithi134 ceph-mon[35674]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T09:47:30.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:30 smithi134 ceph-mon[35674]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T09:47:30.923 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:30 smithi120 ceph-mon[34770]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T09:47:30.923 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:30 smithi120 ceph-mon[34770]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T09:47:30.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:30 smithi120 ceph-mon[34770]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T09:47:30.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:30 smithi077 ceph-mon[30319]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T09:47:30.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:30 smithi077 ceph-mon[30319]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T09:47:30.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:30 smithi077 ceph-mon[30319]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T09:47:32.920 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:32 smithi077 ceph-mon[30319]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T09:47:32.923 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:32 smithi120 ceph-mon[34770]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T09:47:33.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:32 smithi134 ceph-mon[35674]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T09:47:34.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:34 smithi120 ceph-mon[34770]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T09:47:34.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:34 smithi077 ceph-mon[30319]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T09:47:35.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:34 smithi134 ceph-mon[35674]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T09:47:35.923 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:35 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-10T09:47:35.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:35 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:47:35.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:35 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-10T09:47:35.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:35 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:47:36.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:35 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-10T09:47:36.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:35 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:47:36.923 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:36 smithi120 ceph-mon[34770]: Deploying daemon osd.3 on smithi077 2024-09-10T09:47:36.923 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:36 smithi120 ceph-mon[34770]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-09-10T09:47:36.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:36 smithi077 ceph-mon[30319]: Deploying daemon osd.3 on smithi077 2024-09-10T09:47:36.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:36 smithi077 ceph-mon[30319]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-09-10T09:47:37.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:36 smithi134 ceph-mon[35674]: Deploying daemon osd.3 on smithi077 2024-09-10T09:47:37.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:36 smithi134 ceph-mon[35674]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-09-10T09:47:38.864 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:38 smithi077 ceph-mon[30319]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-09-10T09:47:38.923 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:38 smithi120 ceph-mon[34770]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-09-10T09:47:39.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:38 smithi134 ceph-mon[35674]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-09-10T09:47:40.933 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:40 smithi134 ceph-mon[35674]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-10T09:47:40.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:40 smithi077 ceph-mon[30319]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-10T09:47:41.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:40 smithi120 ceph-mon[34770]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-10T09:47:42.915 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:42 smithi077 ceph-mon[30319]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-10T09:47:42.975 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:42 smithi120 ceph-mon[34770]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-10T09:47:43.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:42 smithi134 ceph-mon[35674]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-10T09:47:43.923 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:43 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:47:43.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:43 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:47:43.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:43 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:47:43.936 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:43 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:47:43.936 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:43 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:47:43.936 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:43 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:47:44.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:43 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:47:44.133 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:43 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:47:44.133 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:43 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:47:44.890 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:44 smithi077 ceph-mon[30319]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-10T09:47:44.923 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:44 smithi120 ceph-mon[34770]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-10T09:47:45.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:44 smithi134 ceph-mon[35674]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-10T09:47:47.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:46 smithi134 ceph-mon[35674]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-10T09:47:47.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:46 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:47:47.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:46 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:47:47.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:46 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:47:47.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:46 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:47:47.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:46 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:47:47.141 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:46 smithi077 ceph-mon[30319]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-10T09:47:47.141 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:46 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:47:47.141 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:46 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:47:47.142 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:46 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:47:47.142 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:46 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:47:47.142 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:46 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:47:47.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:46 smithi120 ceph-mon[34770]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-10T09:47:47.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:46 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:47:47.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:46 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:47:47.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:46 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:47:47.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:46 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:47:47.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:46 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:47:48.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:47:48.121+0000 7f2f1e000640 1 -- 172.21.15.77:0/250691927 <== mgr.14150 v2:172.21.15.77:6800/4261324650 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2f20062630 con 0x7f2f080a2b20 2024-09-10T09:47:48.124 INFO:teuthology.orchestra.run.smithi077.stdout:Created osd(s) 3 on host 'smithi077' 2024-09-10T09:47:48.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:47:48.124+0000 7f2f27000640 1 -- 172.21.15.77:0/250691927 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f2f080a2b20 msgr2=0x7f2f080a4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:48.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:47:48.124+0000 7f2f27000640 1 --2- 172.21.15.77:0/250691927 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f2f080a2b20 0x7f2f080a4fe0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2f140095f0 tx=0x7f2f14005f90 comp rx=0 tx=0).stop 2024-09-10T09:47:48.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:47:48.124+0000 7f2f27000640 1 -- 172.21.15.77:0/250691927 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2f20104e50 msgr2=0x7f2f2019a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:48.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:47:48.124+0000 7f2f27000640 1 --2- 172.21.15.77:0/250691927 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2f20104e50 0x7f2f2019a430 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c031440 tx=0x7f2f0c032ce0 comp rx=0 tx=0).stop 2024-09-10T09:47:48.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:47:48.124+0000 7f2f27000640 1 -- 172.21.15.77:0/250691927 shutdown_connections 2024-09-10T09:47:48.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:47:48.124+0000 7f2f27000640 1 --2- 172.21.15.77:0/250691927 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f2f080a2b20 0x7f2f080a4fe0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:48.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:47:48.124+0000 7f2f27000640 1 --2- 172.21.15.77:0/250691927 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2f20105810 0x7f2f201a14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:48.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:47:48.124+0000 7f2f27000640 1 --2- 172.21.15.77:0/250691927 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2f20104e50 0x7f2f2019a430 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:48.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:47:48.124+0000 7f2f27000640 1 --2- 172.21.15.77:0/250691927 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2f20103c50 0x7f2f20199ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:48.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:47:48.124+0000 7f2f27000640 1 -- 172.21.15.77:0/250691927 >> 172.21.15.77:0/250691927 conn(0x7f2f200ff3e0 msgr2=0x7f2f201081d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:47:48.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:47:48.124+0000 7f2f27000640 1 -- 172.21.15.77:0/250691927 shutdown_connections 2024-09-10T09:47:48.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:47:48.124+0000 7f2f27000640 1 -- 172.21.15.77:0/250691927 wait complete. 2024-09-10T09:47:48.841 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:48 smithi077 ceph-mon[30319]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T09:47:48.841 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:48 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:47:48.841 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:48 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:47:48.842 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:48 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:47:48.877 DEBUG:teuthology.orchestra.run.smithi077:osd.3> sudo journalctl -f -n 0 -u ceph-14a55226-6f59-11ef-bcea-c7b262605968@osd.3.service 2024-09-10T09:47:48.881 INFO:tasks.cephadm:Deploying osd.4 on smithi120 with /dev/vg_nvme/lv_3... 2024-09-10T09:47:48.881 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-10T09:47:49.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:48 smithi134 ceph-mon[35674]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T09:47:49.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:48 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:47:49.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:48 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:47:49.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:48 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:47:49.150 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T09:47:49.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:48 smithi120 ceph-mon[34770]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T09:47:49.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:48 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:47:49.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:48 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:47:49.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:48 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:47:49.918 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:49 smithi120 ceph-mon[34770]: from='osd.3 [v2:172.21.15.77:6810/1629416786,v1:172.21.15.77:6811/1629416786]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-10T09:47:50.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:49 smithi134 ceph-mon[35674]: from='osd.3 [v2:172.21.15.77:6810/1629416786,v1:172.21.15.77:6811/1629416786]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-10T09:47:50.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:49 smithi077 ceph-mon[30319]: from='osd.3 [v2:172.21.15.77:6810/1629416786,v1:172.21.15.77:6811/1629416786]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-10T09:47:51.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:50 smithi134 ceph-mon[35674]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T09:47:51.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:50 smithi134 ceph-mon[35674]: from='osd.3 [v2:172.21.15.77:6810/1629416786,v1:172.21.15.77:6811/1629416786]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-10T09:47:51.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:50 smithi134 ceph-mon[35674]: osdmap e23: 4 total, 3 up, 4 in 2024-09-10T09:47:51.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:50 smithi134 ceph-mon[35674]: from='osd.3 [v2:172.21.15.77:6810/1629416786,v1:172.21.15.77:6811/1629416786]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-09-10T09:47:51.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:50 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T09:47:51.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:50 smithi120 ceph-mon[34770]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T09:47:51.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:50 smithi120 ceph-mon[34770]: from='osd.3 [v2:172.21.15.77:6810/1629416786,v1:172.21.15.77:6811/1629416786]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-10T09:47:51.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:50 smithi120 ceph-mon[34770]: osdmap e23: 4 total, 3 up, 4 in 2024-09-10T09:47:51.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:50 smithi120 ceph-mon[34770]: from='osd.3 [v2:172.21.15.77:6810/1629416786,v1:172.21.15.77:6811/1629416786]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-09-10T09:47:51.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:50 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T09:47:51.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:50 smithi077 ceph-mon[30319]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T09:47:51.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:50 smithi077 ceph-mon[30319]: from='osd.3 [v2:172.21.15.77:6810/1629416786,v1:172.21.15.77:6811/1629416786]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-10T09:47:51.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:50 smithi077 ceph-mon[30319]: osdmap e23: 4 total, 3 up, 4 in 2024-09-10T09:47:51.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:50 smithi077 ceph-mon[30319]: from='osd.3 [v2:172.21.15.77:6810/1629416786,v1:172.21.15.77:6811/1629416786]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-09-10T09:47:51.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:50 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T09:47:51.580 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T09:47:51.605 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi120:vg_nvme/lv_3 2024-09-10T09:47:51.689 INFO:journalctl@ceph.osd.3.smithi077.stdout:Sep 10 09:47:51 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-3[50752]: 2024-09-10T09:47:51.297+0000 7fceaf600640 -1 osd.3 0 waiting for initial osdmap 2024-09-10T09:47:51.689 INFO:journalctl@ceph.osd.3.smithi077.stdout:Sep 10 09:47:51 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-3[50752]: 2024-09-10T09:47:51.316+0000 7fcea5600640 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T09:47:51.849 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T09:47:51.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:51 smithi120 ceph-mon[34770]: from='osd.3 [v2:172.21.15.77:6810/1629416786,v1:172.21.15.77:6811/1629416786]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-09-10T09:47:51.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:51 smithi120 ceph-mon[34770]: osdmap e24: 4 total, 3 up, 4 in 2024-09-10T09:47:51.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:51 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T09:47:51.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:51 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T09:47:51.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:51 smithi120 ceph-mon[34770]: from='osd.3 [v2:172.21.15.77:6810/1629416786,v1:172.21.15.77:6811/1629416786]' entity='osd.3' 2024-09-10T09:47:52.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:51 smithi134 ceph-mon[35674]: from='osd.3 [v2:172.21.15.77:6810/1629416786,v1:172.21.15.77:6811/1629416786]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-09-10T09:47:52.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:51 smithi134 ceph-mon[35674]: osdmap e24: 4 total, 3 up, 4 in 2024-09-10T09:47:52.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:51 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T09:47:52.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:51 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T09:47:52.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:51 smithi134 ceph-mon[35674]: from='osd.3 [v2:172.21.15.77:6810/1629416786,v1:172.21.15.77:6811/1629416786]' entity='osd.3' 2024-09-10T09:47:52.158 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:51 smithi077 ceph-mon[30319]: from='osd.3 [v2:172.21.15.77:6810/1629416786,v1:172.21.15.77:6811/1629416786]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-09-10T09:47:52.158 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:51 smithi077 ceph-mon[30319]: osdmap e24: 4 total, 3 up, 4 in 2024-09-10T09:47:52.158 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:51 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T09:47:52.158 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:51 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T09:47:52.158 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:51 smithi077 ceph-mon[30319]: from='osd.3 [v2:172.21.15.77:6810/1629416786,v1:172.21.15.77:6811/1629416786]' entity='osd.3' 2024-09-10T09:47:52.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:47:52.304+0000 7f140ba8b640 1 -- 172.21.15.120:0/1320181340 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1404100490 msgr2=0x7f14041028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:52.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:47:52.304+0000 7f140ba8b640 1 --2- 172.21.15.120:0/1320181340 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1404100490 0x7f14041028a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f13f4009900 tx=0x7f13f402a410 comp rx=0 tx=0).stop 2024-09-10T09:47:52.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:47:52.304+0000 7f140ba8b640 1 -- 172.21.15.120:0/1320181340 shutdown_connections 2024-09-10T09:47:52.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:47:52.304+0000 7f140ba8b640 1 --2- 172.21.15.120:0/1320181340 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1404105790 0x7f1404105c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:52.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:47:52.304+0000 7f140ba8b640 1 --2- 172.21.15.120:0/1320181340 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1404102de0 0x7f1404105250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:52.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:47:52.304+0000 7f140ba8b640 1 --2- 172.21.15.120:0/1320181340 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1404100490 0x7f14041028a0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:52.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:47:52.304+0000 7f140ba8b640 1 -- 172.21.15.120:0/1320181340 >> 172.21.15.120:0/1320181340 conn(0x7f14040fc2a0 msgr2=0x7f14040fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:47:52.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:47:52.304+0000 7f140ba8b640 1 -- 172.21.15.120:0/1320181340 shutdown_connections 2024-09-10T09:47:52.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:47:52.304+0000 7f140ba8b640 1 -- 172.21.15.120:0/1320181340 wait complete. 2024-09-10T09:47:52.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:47:52.305+0000 7f140ba8b640 1 Processor -- start 2024-09-10T09:47:52.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:47:52.305+0000 7f140ba8b640 1 -- start start 2024-09-10T09:47:52.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:47:52.305+0000 7f140ba8b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1404100490 0x7f140419e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:52.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:47:52.305+0000 7f140ba8b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1404102de0 0x7f140419ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:52.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:47:52.305+0000 7f140ba8b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1404105790 0x7f14041a5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:52.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:47:52.305+0000 7f140ba8b640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1404113e70 con 0x7f1404102de0 2024-09-10T09:47:52.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:47:52.305+0000 7f140ba8b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1404113cf0 con 0x7f1404105790 2024-09-10T09:47:52.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:47:52.305+0000 7f140ba8b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1404113ff0 con 0x7f1404100490 2024-09-10T09:47:52.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:47:52.305+0000 7f140a028640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1404105790 0x7f14041a5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:52.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:47:52.305+0000 7f140a028640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1404105790 0x7f14041a5b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56834/0 (socket says 172.21.15.120:56834) 2024-09-10T09:47:52.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:47:52.305+0000 7f140a028640 1 -- 172.21.15.120:0/1073215139 learned_addr learned my addr 172.21.15.120:0/1073215139 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T09:47:52.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:47:52.306+0000 7f1409026640 1 --2- 172.21.15.120:0/1073215139 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1404102de0 0x7f140419ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:52.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:47:52.306+0000 7f1409827640 1 --2- 172.21.15.120:0/1073215139 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1404100490 0x7f140419e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:52.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:47:52.306+0000 7f140a028640 1 -- 172.21.15.120:0/1073215139 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1404100490 msgr2=0x7f140419e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:52.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:47:52.306+0000 7f140a028640 1 --2- 172.21.15.120:0/1073215139 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1404100490 0x7f140419e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:52.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:47:52.306+0000 7f140a028640 1 -- 172.21.15.120:0/1073215139 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1404102de0 msgr2=0x7f140419ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:52.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:47:52.306+0000 7f140a028640 1 --2- 172.21.15.120:0/1073215139 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1404102de0 0x7f140419ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:52.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:47:52.306+0000 7f140a028640 1 -- 172.21.15.120:0/1073215139 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14041a6170 con 0x7f1404105790 2024-09-10T09:47:52.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:47:52.306+0000 7f140a028640 1 --2- 172.21.15.120:0/1073215139 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1404105790 0x7f14041a5b00 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f13fc009fd0 tx=0x7f13fc00b260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:47:52.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:47:52.306+0000 7f13faffd640 1 -- 172.21.15.120:0/1073215139 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13fc017070 con 0x7f1404105790 2024-09-10T09:47:52.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:47:52.306+0000 7f13faffd640 1 -- 172.21.15.120:0/1073215139 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13fc003e50 con 0x7f1404105790 2024-09-10T09:47:52.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:47:52.306+0000 7f140ba8b640 1 -- 172.21.15.120:0/1073215139 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14041a6460 con 0x7f1404105790 2024-09-10T09:47:52.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:47:52.306+0000 7f13faffd640 1 -- 172.21.15.120:0/1073215139 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13fc0106c0 con 0x7f1404105790 2024-09-10T09:47:52.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:47:52.306+0000 7f140ba8b640 1 -- 172.21.15.120:0/1073215139 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14041a6950 con 0x7f1404105790 2024-09-10T09:47:52.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:47:52.307+0000 7f140ba8b640 1 -- 172.21.15.120:0/1073215139 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14040fdd90 con 0x7f1404105790 2024-09-10T09:47:52.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:47:52.308+0000 7f13faffd640 1 -- 172.21.15.120:0/1073215139 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f13fc003970 con 0x7f1404105790 2024-09-10T09:47:52.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:47:52.309+0000 7f13faffd640 1 --2- 172.21.15.120:0/1073215139 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f13f00a2bb0 0x7f13f00a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:52.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:47:52.309+0000 7f13faffd640 1 -- 172.21.15.120:0/1073215139 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 3702+0+0 (secure 0 0 0) 0x7f13fc0cdc00 con 0x7f1404105790 2024-09-10T09:47:52.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:47:52.309+0000 7f1409827640 1 --2- 172.21.15.120:0/1073215139 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f13f00a2bb0 0x7f13f00a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:52.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:47:52.310+0000 7f1409827640 1 --2- 172.21.15.120:0/1073215139 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f13f00a2bb0 0x7f13f00a5070 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f13f4000c00 tx=0x7f13f40315d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:47:52.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:47:52.312+0000 7f13faffd640 1 -- 172.21.15.120:0/1073215139 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f13fc09bd60 con 0x7f1404105790 2024-09-10T09:47:52.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:47:52.486+0000 7f140ba8b640 1 -- 172.21.15.120:0/1073215139 --> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f1404062630 con 0x7f13f00a2bb0 2024-09-10T09:47:52.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:52 smithi077 ceph-mon[30319]: purged_snaps scrub starts 2024-09-10T09:47:52.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:52 smithi077 ceph-mon[30319]: purged_snaps scrub ok 2024-09-10T09:47:52.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:52 smithi077 ceph-mon[30319]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T09:47:52.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:52 smithi077 ceph-mon[30319]: osd.3 [v2:172.21.15.77:6810/1629416786,v1:172.21.15.77:6811/1629416786] boot 2024-09-10T09:47:52.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:52 smithi077 ceph-mon[30319]: osdmap e25: 4 total, 4 up, 4 in 2024-09-10T09:47:52.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:52 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T09:47:52.941 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:52 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:47:52.941 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:52 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:47:52.941 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:52 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:47:52.941 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:52 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:47:52.941 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:52 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:47:52.941 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:52 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:47:52.942 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:52 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:47:52.942 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:52 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:47:52.942 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:52 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:47:52.942 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:52 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:47:52.942 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:52 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:47:52.942 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:52 smithi077 ceph-mon[30319]: osdmap e26: 4 total, 4 up, 4 in 2024-09-10T09:47:53.133 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:52 smithi134 ceph-mon[35674]: purged_snaps scrub starts 2024-09-10T09:47:53.133 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:52 smithi134 ceph-mon[35674]: purged_snaps scrub ok 2024-09-10T09:47:53.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:52 smithi134 ceph-mon[35674]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T09:47:53.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:52 smithi134 ceph-mon[35674]: osd.3 [v2:172.21.15.77:6810/1629416786,v1:172.21.15.77:6811/1629416786] boot 2024-09-10T09:47:53.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:52 smithi134 ceph-mon[35674]: osdmap e25: 4 total, 4 up, 4 in 2024-09-10T09:47:53.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:52 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T09:47:53.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:52 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:47:53.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:52 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:47:53.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:52 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:47:53.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:52 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:47:53.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:52 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:47:53.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:52 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:47:53.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:52 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:47:53.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:52 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:47:53.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:52 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:47:53.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:52 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:47:53.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:52 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:47:53.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:52 smithi134 ceph-mon[35674]: osdmap e26: 4 total, 4 up, 4 in 2024-09-10T09:47:53.135 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:52 smithi120 ceph-mon[34770]: purged_snaps scrub starts 2024-09-10T09:47:53.135 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:52 smithi120 ceph-mon[34770]: purged_snaps scrub ok 2024-09-10T09:47:53.135 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:52 smithi120 ceph-mon[34770]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T09:47:53.135 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:52 smithi120 ceph-mon[34770]: osd.3 [v2:172.21.15.77:6810/1629416786,v1:172.21.15.77:6811/1629416786] boot 2024-09-10T09:47:53.135 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:52 smithi120 ceph-mon[34770]: osdmap e25: 4 total, 4 up, 4 in 2024-09-10T09:47:53.135 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:52 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T09:47:53.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:52 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:47:53.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:52 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:47:53.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:52 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:47:53.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:52 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:47:53.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:52 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:47:53.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:52 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:47:53.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:52 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:47:53.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:52 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:47:53.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:52 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:47:53.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:52 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:47:53.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:52 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:47:53.136 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:52 smithi120 ceph-mon[34770]: osdmap e26: 4 total, 4 up, 4 in 2024-09-10T09:47:54.038 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:53 smithi120 ceph-mon[34770]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:47:54.039 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:53 smithi120 ceph-mon[34770]: Detected new or changed devices on smithi077 2024-09-10T09:47:54.039 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:53 smithi120 ceph-mon[34770]: Adjusting osd_memory_target on smithi077 to 8544M 2024-09-10T09:47:54.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:53 smithi134 ceph-mon[35674]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:47:54.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:53 smithi134 ceph-mon[35674]: Detected new or changed devices on smithi077 2024-09-10T09:47:54.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:53 smithi134 ceph-mon[35674]: Adjusting osd_memory_target on smithi077 to 8544M 2024-09-10T09:47:54.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:53 smithi077 ceph-mon[30319]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:47:54.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:53 smithi077 ceph-mon[30319]: Detected new or changed devices on smithi077 2024-09-10T09:47:54.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:53 smithi077 ceph-mon[30319]: Adjusting osd_memory_target on smithi077 to 8544M 2024-09-10T09:47:54.923 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:54 smithi120 ceph-mon[34770]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-10T09:47:54.923 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:54 smithi120 ceph-mon[34770]: osdmap e27: 4 total, 4 up, 4 in 2024-09-10T09:47:55.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:54 smithi134 ceph-mon[35674]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-10T09:47:55.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:54 smithi134 ceph-mon[35674]: osdmap e27: 4 total, 4 up, 4 in 2024-09-10T09:47:55.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:54 smithi077 ceph-mon[30319]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-10T09:47:55.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:54 smithi077 ceph-mon[30319]: osdmap e27: 4 total, 4 up, 4 in 2024-09-10T09:47:56.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:55 smithi134 ceph-mon[35674]: osdmap e28: 4 total, 4 up, 4 in 2024-09-10T09:47:56.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:55 smithi134 ceph-mon[35674]: from='client.? 172.21.15.120:0/3023728815' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c826978f-40b0-43ae-adbb-5c93b32a68ff"}]: dispatch 2024-09-10T09:47:56.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:55 smithi134 ceph-mon[35674]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c826978f-40b0-43ae-adbb-5c93b32a68ff"}]: dispatch 2024-09-10T09:47:56.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:55 smithi134 ceph-mon[35674]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c826978f-40b0-43ae-adbb-5c93b32a68ff"}]': finished 2024-09-10T09:47:56.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:55 smithi134 ceph-mon[35674]: osdmap e29: 5 total, 4 up, 5 in 2024-09-10T09:47:56.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:55 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T09:47:56.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:55 smithi134 ceph-mon[35674]: from='client.? 172.21.15.120:0/2928943075' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:47:56.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:55 smithi120 ceph-mon[34770]: osdmap e28: 4 total, 4 up, 4 in 2024-09-10T09:47:56.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:55 smithi120 ceph-mon[34770]: from='client.? 172.21.15.120:0/3023728815' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c826978f-40b0-43ae-adbb-5c93b32a68ff"}]: dispatch 2024-09-10T09:47:56.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:55 smithi120 ceph-mon[34770]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c826978f-40b0-43ae-adbb-5c93b32a68ff"}]: dispatch 2024-09-10T09:47:56.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:55 smithi120 ceph-mon[34770]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c826978f-40b0-43ae-adbb-5c93b32a68ff"}]': finished 2024-09-10T09:47:56.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:55 smithi120 ceph-mon[34770]: osdmap e29: 5 total, 4 up, 5 in 2024-09-10T09:47:56.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:55 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T09:47:56.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:55 smithi120 ceph-mon[34770]: from='client.? 172.21.15.120:0/2928943075' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:47:56.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:55 smithi077 ceph-mon[30319]: osdmap e28: 4 total, 4 up, 4 in 2024-09-10T09:47:56.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:55 smithi077 ceph-mon[30319]: from='client.? 172.21.15.120:0/3023728815' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c826978f-40b0-43ae-adbb-5c93b32a68ff"}]: dispatch 2024-09-10T09:47:56.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:55 smithi077 ceph-mon[30319]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c826978f-40b0-43ae-adbb-5c93b32a68ff"}]: dispatch 2024-09-10T09:47:56.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:55 smithi077 ceph-mon[30319]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c826978f-40b0-43ae-adbb-5c93b32a68ff"}]': finished 2024-09-10T09:47:56.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:55 smithi077 ceph-mon[30319]: osdmap e29: 5 total, 4 up, 5 in 2024-09-10T09:47:56.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:55 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T09:47:56.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:55 smithi077 ceph-mon[30319]: from='client.? 172.21.15.120:0/2928943075' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:47:57.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:56 smithi134 ceph-mon[35674]: pgmap v107: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-10T09:47:57.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:56 smithi120 ceph-mon[34770]: pgmap v107: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-10T09:47:57.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:56 smithi077 ceph-mon[30319]: pgmap v107: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-10T09:47:59.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:47:58 smithi134 ceph-mon[35674]: pgmap v108: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-10T09:47:59.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:47:58 smithi120 ceph-mon[34770]: pgmap v108: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-10T09:47:59.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:47:58 smithi077 ceph-mon[30319]: pgmap v108: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-10T09:48:01.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:00 smithi134 ceph-mon[35674]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T09:48:01.145 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:00 smithi120 ceph-mon[34770]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T09:48:01.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:00 smithi077 ceph-mon[30319]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T09:48:03.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:02 smithi134 ceph-mon[35674]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-09-10T09:48:03.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:02 smithi120 ceph-mon[34770]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-09-10T09:48:03.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:02 smithi077 ceph-mon[30319]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-09-10T09:48:03.923 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:03 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-10T09:48:03.923 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:03 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:48:04.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:03 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-10T09:48:04.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:03 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:48:04.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:03 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-10T09:48:04.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:03 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:48:04.961 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:04 smithi120 ceph-mon[34770]: Deploying daemon osd.4 on smithi120 2024-09-10T09:48:04.962 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:04 smithi120 ceph-mon[34770]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-10T09:48:05.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:04 smithi134 ceph-mon[35674]: Deploying daemon osd.4 on smithi120 2024-09-10T09:48:05.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:04 smithi134 ceph-mon[35674]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-10T09:48:05.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:04 smithi077 ceph-mon[30319]: Deploying daemon osd.4 on smithi120 2024-09-10T09:48:05.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:04 smithi077 ceph-mon[30319]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-10T09:48:06.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:06 smithi120 ceph-mon[34770]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-09-10T09:48:07.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:06 smithi134 ceph-mon[35674]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-09-10T09:48:07.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:06 smithi077 ceph-mon[30319]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-09-10T09:48:08.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:08 smithi120 ceph-mon[34770]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T09:48:09.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:08 smithi134 ceph-mon[35674]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T09:48:09.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:08 smithi077 ceph-mon[30319]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T09:48:09.959 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:09 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:48:09.959 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:09 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:48:09.959 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:09 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:48:10.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:09 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:48:10.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:09 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:48:10.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:09 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:48:10.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:09 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:48:10.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:09 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:48:10.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:09 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:48:10.976 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:10 smithi120 ceph-mon[34770]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T09:48:11.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:10 smithi134 ceph-mon[35674]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T09:48:11.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:10 smithi077 ceph-mon[30319]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T09:48:13.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:12 smithi134 ceph-mon[35674]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-10T09:48:13.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:12 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:48:13.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:12 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:48:13.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:12 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:48:13.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:12 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:48:13.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:12 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:48:13.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:12 smithi120 ceph-mon[34770]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-10T09:48:13.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:12 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:48:13.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:12 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:48:13.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:12 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:48:13.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:12 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:48:13.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:12 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:48:13.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:12 smithi077 ceph-mon[30319]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-10T09:48:13.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:12 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:48:13.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:12 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:48:13.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:12 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:48:13.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:12 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:48:13.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:12 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:48:13.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:48:13.275+0000 7f13faffd640 1 -- 172.21.15.120:0/1073215139 <== mgr.14150 v2:172.21.15.77:6800/4261324650 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1404062630 con 0x7f13f00a2bb0 2024-09-10T09:48:13.277 INFO:teuthology.orchestra.run.smithi120.stdout:Created osd(s) 4 on host 'smithi120' 2024-09-10T09:48:13.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:48:13.278+0000 7f140ba8b640 1 -- 172.21.15.120:0/1073215139 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f13f00a2bb0 msgr2=0x7f13f00a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:13.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:48:13.278+0000 7f140ba8b640 1 --2- 172.21.15.120:0/1073215139 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f13f00a2bb0 0x7f13f00a5070 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f13f4000c00 tx=0x7f13f40315d0 comp rx=0 tx=0).stop 2024-09-10T09:48:13.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:48:13.278+0000 7f140ba8b640 1 -- 172.21.15.120:0/1073215139 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1404105790 msgr2=0x7f14041a5b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:13.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:48:13.278+0000 7f140ba8b640 1 --2- 172.21.15.120:0/1073215139 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1404105790 0x7f14041a5b00 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f13fc009fd0 tx=0x7f13fc00b260 comp rx=0 tx=0).stop 2024-09-10T09:48:13.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:48:13.278+0000 7f140ba8b640 1 -- 172.21.15.120:0/1073215139 shutdown_connections 2024-09-10T09:48:13.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:48:13.278+0000 7f140ba8b640 1 --2- 172.21.15.120:0/1073215139 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f13f00a2bb0 0x7f13f00a5070 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:13.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:48:13.278+0000 7f140ba8b640 1 --2- 172.21.15.120:0/1073215139 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1404105790 0x7f14041a5b00 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:13.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:48:13.278+0000 7f140ba8b640 1 --2- 172.21.15.120:0/1073215139 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1404102de0 0x7f140419ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:13.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:48:13.278+0000 7f140ba8b640 1 --2- 172.21.15.120:0/1073215139 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1404100490 0x7f140419e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:13.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:48:13.278+0000 7f140ba8b640 1 -- 172.21.15.120:0/1073215139 >> 172.21.15.120:0/1073215139 conn(0x7f14040fc2a0 msgr2=0x7f14041037a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:48:13.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:48:13.278+0000 7f140ba8b640 1 -- 172.21.15.120:0/1073215139 shutdown_connections 2024-09-10T09:48:13.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:48:13.278+0000 7f140ba8b640 1 -- 172.21.15.120:0/1073215139 wait complete. 2024-09-10T09:48:13.923 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:13 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:48:13.923 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:13 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:48:13.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:13 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:48:13.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:13 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:48:13.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:13 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:48:13.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:13 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:48:13.994 DEBUG:teuthology.orchestra.run.smithi120:osd.4> sudo journalctl -f -n 0 -u ceph-14a55226-6f59-11ef-bcea-c7b262605968@osd.4.service 2024-09-10T09:48:13.996 INFO:tasks.cephadm:Deploying osd.5 on smithi134 with /dev/vg_nvme/lv_3... 2024-09-10T09:48:13.996 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-10T09:48:14.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:13 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:48:14.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:13 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:48:14.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:13 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:48:14.251 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T09:48:15.119 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:14 smithi120 ceph-mon[34770]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-10T09:48:15.130 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:14 smithi134 ceph-mon[35674]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-10T09:48:15.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:14 smithi077 ceph-mon[30319]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-10T09:48:15.377 INFO:journalctl@ceph.osd.4.smithi120.stdout:Sep 10 09:48:15 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-4[45711]: 2024-09-10T09:48:15.116+0000 7fcb1e472640 -1 osd.4 0 log_to_monitors true 2024-09-10T09:48:15.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:15 smithi134 ceph-mon[35674]: from='osd.4 [v2:172.21.15.120:6808/3162658038,v1:172.21.15.120:6809/3162658038]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T09:48:15.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:15 smithi134 ceph-mon[35674]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T09:48:16.083 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:15 smithi120 ceph-mon[34770]: from='osd.4 [v2:172.21.15.120:6808/3162658038,v1:172.21.15.120:6809/3162658038]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T09:48:16.083 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:15 smithi120 ceph-mon[34770]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T09:48:16.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:15 smithi077 ceph-mon[30319]: from='osd.4 [v2:172.21.15.120:6808/3162658038,v1:172.21.15.120:6809/3162658038]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T09:48:16.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:15 smithi077 ceph-mon[30319]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T09:48:16.774 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-10T09:48:16.801 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi134:vg_nvme/lv_3 2024-09-10T09:48:17.055 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T09:48:17.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:16 smithi134 ceph-mon[35674]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-10T09:48:17.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:16 smithi134 ceph-mon[35674]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-10T09:48:17.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:16 smithi134 ceph-mon[35674]: from='osd.4 [v2:172.21.15.120:6808/3162658038,v1:172.21.15.120:6809/3162658038]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-09-10T09:48:17.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:16 smithi134 ceph-mon[35674]: osdmap e30: 5 total, 4 up, 5 in 2024-09-10T09:48:17.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:16 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T09:48:17.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:16 smithi134 ceph-mon[35674]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-09-10T09:48:17.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:16 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:48:17.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:16 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:48:17.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:16 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:48:17.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:16 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:48:17.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:16 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:48:17.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:16 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:48:17.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:16 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:48:17.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:16 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:48:17.173 INFO:journalctl@ceph.osd.4.smithi120.stdout:Sep 10 09:48:16 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-4[45711]: 2024-09-10T09:48:16.838+0000 7fcb1a3f5640 -1 osd.4 0 waiting for initial osdmap 2024-09-10T09:48:17.173 INFO:journalctl@ceph.osd.4.smithi120.stdout:Sep 10 09:48:16 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-4[45711]: 2024-09-10T09:48:16.845+0000 7fcb1621d640 -1 osd.4 31 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T09:48:17.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:16 smithi120 ceph-mon[34770]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-10T09:48:17.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:16 smithi120 ceph-mon[34770]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-10T09:48:17.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:16 smithi120 ceph-mon[34770]: from='osd.4 [v2:172.21.15.120:6808/3162658038,v1:172.21.15.120:6809/3162658038]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-09-10T09:48:17.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:16 smithi120 ceph-mon[34770]: osdmap e30: 5 total, 4 up, 5 in 2024-09-10T09:48:17.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:16 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T09:48:17.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:16 smithi120 ceph-mon[34770]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-09-10T09:48:17.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:16 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:48:17.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:16 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:48:17.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:16 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:48:17.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:16 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:48:17.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:16 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:48:17.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:16 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:48:17.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:16 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:48:17.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:16 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:48:17.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:16 smithi077 ceph-mon[30319]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-10T09:48:17.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:16 smithi077 ceph-mon[30319]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-10T09:48:17.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:16 smithi077 ceph-mon[30319]: from='osd.4 [v2:172.21.15.120:6808/3162658038,v1:172.21.15.120:6809/3162658038]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-09-10T09:48:17.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:16 smithi077 ceph-mon[30319]: osdmap e30: 5 total, 4 up, 5 in 2024-09-10T09:48:17.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:16 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T09:48:17.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:16 smithi077 ceph-mon[30319]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-09-10T09:48:17.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:16 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:48:17.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:16 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:48:17.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:16 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:48:17.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:16 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:48:17.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:16 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:48:17.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:16 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:48:17.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:16 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:48:17.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:16 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:48:17.569 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:48:17.565+0000 7fefc5437640 1 -- 172.21.15.134:0/2754815645 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fefc0102700 msgr2=0x7fefc0104af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:17.569 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:48:17.565+0000 7fefc5437640 1 --2- 172.21.15.134:0/2754815645 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fefc0102700 0x7fefc0104af0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fefb0009900 tx=0x7fefb002a410 comp rx=0 tx=0).stop 2024-09-10T09:48:17.569 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:48:17.565+0000 7fefc5437640 1 -- 172.21.15.134:0/2754815645 shutdown_connections 2024-09-10T09:48:17.569 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:48:17.565+0000 7fefc5437640 1 --2- 172.21.15.134:0/2754815645 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fefc0107960 0x7fefc0109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:17.569 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:48:17.565+0000 7fefc5437640 1 --2- 172.21.15.134:0/2754815645 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fefc0105030 0x7fefc0107420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:17.569 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:48:17.565+0000 7fefc5437640 1 --2- 172.21.15.134:0/2754815645 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fefc0102700 0x7fefc0104af0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:17.569 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:48:17.565+0000 7fefc5437640 1 -- 172.21.15.134:0/2754815645 >> 172.21.15.134:0/2754815645 conn(0x7fefc00fc280 msgr2=0x7fefc00fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:48:17.570 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:48:17.566+0000 7fefc5437640 1 -- 172.21.15.134:0/2754815645 shutdown_connections 2024-09-10T09:48:17.570 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:48:17.566+0000 7fefc5437640 1 -- 172.21.15.134:0/2754815645 wait complete. 2024-09-10T09:48:17.570 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:48:17.566+0000 7fefc5437640 1 Processor -- start 2024-09-10T09:48:17.570 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:48:17.566+0000 7fefc5437640 1 -- start start 2024-09-10T09:48:17.570 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:48:17.566+0000 7fefc5437640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fefc0102700 0x7fefc0197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:17.570 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:48:17.566+0000 7fefc5437640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fefc0105030 0x7fefc01984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:17.570 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:48:17.566+0000 7fefc5437640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fefc0107960 0x7fefc019f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:17.570 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:48:17.566+0000 7fefc5437640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fefc010d900 con 0x7fefc0105030 2024-09-10T09:48:17.570 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:48:17.566+0000 7fefc5437640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fefc010d780 con 0x7fefc0102700 2024-09-10T09:48:17.570 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:48:17.566+0000 7fefc5437640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fefc010da80 con 0x7fefc0107960 2024-09-10T09:48:17.570 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:48:17.566+0000 7fefbf7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fefc0107960 0x7fefc019f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:17.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:48:17.567+0000 7fefbf7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fefc0107960 0x7fefc019f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:37040/0 (socket says 172.21.15.134:37040) 2024-09-10T09:48:17.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:48:17.567+0000 7fefbf7fe640 1 -- 172.21.15.134:0/118149857 learned_addr learned my addr 172.21.15.134:0/118149857 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-10T09:48:17.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:48:17.567+0000 7fefbeffd640 1 --2- 172.21.15.134:0/118149857 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fefc0102700 0x7fefc0197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:17.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:48:17.567+0000 7fefbf7fe640 1 -- 172.21.15.134:0/118149857 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fefc0102700 msgr2=0x7fefc0197fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:17.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:48:17.567+0000 7fefbf7fe640 1 --2- 172.21.15.134:0/118149857 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fefc0102700 0x7fefc0197fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:17.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:48:17.567+0000 7fefbf7fe640 1 -- 172.21.15.134:0/118149857 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fefc0105030 msgr2=0x7fefc01984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:17.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:48:17.567+0000 7fefbf7fe640 1 --2- 172.21.15.134:0/118149857 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fefc0105030 0x7fefc01984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:17.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:48:17.567+0000 7fefbf7fe640 1 -- 172.21.15.134:0/118149857 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefc019fbe0 con 0x7fefc0107960 2024-09-10T09:48:17.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:48:17.567+0000 7fefbf7fe640 1 --2- 172.21.15.134:0/118149857 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fefc0107960 0x7fefc019f570 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fefb800b970 tx=0x7fefb800be30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:48:17.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:48:17.567+0000 7fefbcff9640 1 -- 172.21.15.134:0/118149857 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb8019070 con 0x7fefc0107960 2024-09-10T09:48:17.572 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:48:17.567+0000 7fefc5437640 1 -- 172.21.15.134:0/118149857 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefc019fed0 con 0x7fefc0107960 2024-09-10T09:48:17.572 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:48:17.567+0000 7fefbcff9640 1 -- 172.21.15.134:0/118149857 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefb800cb40 con 0x7fefc0107960 2024-09-10T09:48:17.572 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:48:17.567+0000 7fefbcff9640 1 -- 172.21.15.134:0/118149857 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb8007650 con 0x7fefc0107960 2024-09-10T09:48:17.572 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:48:17.567+0000 7fefc5437640 1 -- 172.21.15.134:0/118149857 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefc01a0460 con 0x7fefc0107960 2024-09-10T09:48:17.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:48:17.568+0000 7fefc5437640 1 -- 172.21.15.134:0/118149857 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefc00fdd70 con 0x7fefc0107960 2024-09-10T09:48:17.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:48:17.569+0000 7fefbcff9640 1 -- 172.21.15.134:0/118149857 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fefb801c070 con 0x7fefc0107960 2024-09-10T09:48:17.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:48:17.570+0000 7fefbcff9640 1 --2- 172.21.15.134:0/118149857 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fefac0a2bb0 0x7fefac0a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:17.574 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:48:17.571+0000 7fefbcff9640 1 -- 172.21.15.134:0/118149857 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 3821+0+0 (secure 0 0 0) 0x7fefb80cd440 con 0x7fefc0107960 2024-09-10T09:48:17.574 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:48:17.571+0000 7fefbeffd640 1 --2- 172.21.15.134:0/118149857 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fefac0a2bb0 0x7fefac0a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:17.574 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:48:17.571+0000 7fefbeffd640 1 --2- 172.21.15.134:0/118149857 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fefac0a2bb0 0x7fefac0a5070 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fefb00314e0 tx=0x7fefb0000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:48:17.576 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:48:17.575+0000 7fefbcff9640 1 -- 172.21.15.134:0/118149857 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fefb809aee0 con 0x7fefc0107960 2024-09-10T09:48:17.748 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:48:17.747+0000 7fefc5437640 1 -- 172.21.15.134:0/118149857 --> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fefc0062630 con 0x7fefac0a2bb0 2024-09-10T09:48:18.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:17 smithi134 ceph-mon[35674]: Detected new or changed devices on smithi120 2024-09-10T09:48:18.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:17 smithi134 ceph-mon[35674]: Adjusting osd_memory_target on smithi120 to 8544M 2024-09-10T09:48:18.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:17 smithi134 ceph-mon[35674]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2024-09-10T09:48:18.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:17 smithi134 ceph-mon[35674]: osdmap e31: 5 total, 4 up, 5 in 2024-09-10T09:48:18.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:17 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T09:48:18.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:17 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T09:48:18.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:17 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:48:18.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:17 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:48:18.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:17 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:48:18.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:17 smithi120 ceph-mon[34770]: Detected new or changed devices on smithi120 2024-09-10T09:48:18.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:17 smithi120 ceph-mon[34770]: Adjusting osd_memory_target on smithi120 to 8544M 2024-09-10T09:48:18.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:17 smithi120 ceph-mon[34770]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2024-09-10T09:48:18.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:17 smithi120 ceph-mon[34770]: osdmap e31: 5 total, 4 up, 5 in 2024-09-10T09:48:18.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:17 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T09:48:18.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:17 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T09:48:18.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:17 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:48:18.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:17 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:48:18.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:17 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:48:18.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:17 smithi077 ceph-mon[30319]: Detected new or changed devices on smithi120 2024-09-10T09:48:18.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:17 smithi077 ceph-mon[30319]: Adjusting osd_memory_target on smithi120 to 8544M 2024-09-10T09:48:18.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:17 smithi077 ceph-mon[30319]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2024-09-10T09:48:18.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:17 smithi077 ceph-mon[30319]: osdmap e31: 5 total, 4 up, 5 in 2024-09-10T09:48:18.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:17 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T09:48:18.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:17 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T09:48:18.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:17 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:48:18.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:17 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:48:18.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:17 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:48:19.122 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:18 smithi134 ceph-mon[35674]: purged_snaps scrub starts 2024-09-10T09:48:19.122 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:18 smithi134 ceph-mon[35674]: purged_snaps scrub ok 2024-09-10T09:48:19.123 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:18 smithi134 ceph-mon[35674]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-10T09:48:19.123 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:18 smithi134 ceph-mon[35674]: from='client.24250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:48:19.123 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:18 smithi134 ceph-mon[35674]: osd.4 [v2:172.21.15.120:6808/3162658038,v1:172.21.15.120:6809/3162658038] boot 2024-09-10T09:48:19.123 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:18 smithi134 ceph-mon[35674]: osdmap e32: 5 total, 5 up, 5 in 2024-09-10T09:48:19.123 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:18 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T09:48:19.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:18 smithi120 ceph-mon[34770]: purged_snaps scrub starts 2024-09-10T09:48:19.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:18 smithi120 ceph-mon[34770]: purged_snaps scrub ok 2024-09-10T09:48:19.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:18 smithi120 ceph-mon[34770]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-10T09:48:19.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:18 smithi120 ceph-mon[34770]: from='client.24250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:48:19.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:18 smithi120 ceph-mon[34770]: osd.4 [v2:172.21.15.120:6808/3162658038,v1:172.21.15.120:6809/3162658038] boot 2024-09-10T09:48:19.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:18 smithi120 ceph-mon[34770]: osdmap e32: 5 total, 5 up, 5 in 2024-09-10T09:48:19.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:18 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T09:48:19.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:18 smithi077 ceph-mon[30319]: purged_snaps scrub starts 2024-09-10T09:48:19.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:18 smithi077 ceph-mon[30319]: purged_snaps scrub ok 2024-09-10T09:48:19.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:18 smithi077 ceph-mon[30319]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-10T09:48:19.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:18 smithi077 ceph-mon[30319]: from='client.24250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:48:19.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:18 smithi077 ceph-mon[30319]: osd.4 [v2:172.21.15.120:6808/3162658038,v1:172.21.15.120:6809/3162658038] boot 2024-09-10T09:48:19.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:18 smithi077 ceph-mon[30319]: osdmap e32: 5 total, 5 up, 5 in 2024-09-10T09:48:19.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:18 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T09:48:20.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:19 smithi134 ceph-mon[35674]: osdmap e33: 5 total, 5 up, 5 in 2024-09-10T09:48:20.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:19 smithi120 ceph-mon[34770]: osdmap e33: 5 total, 5 up, 5 in 2024-09-10T09:48:20.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:19 smithi077 ceph-mon[30319]: osdmap e33: 5 total, 5 up, 5 in 2024-09-10T09:48:21.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:20 smithi134 ceph-mon[35674]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:48:21.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:20 smithi134 ceph-mon[35674]: from='client.? 172.21.15.134:0/4180747021' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "be51c7fb-2765-46a3-9861-c950f63de148"}]: dispatch 2024-09-10T09:48:21.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:20 smithi134 ceph-mon[35674]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "be51c7fb-2765-46a3-9861-c950f63de148"}]: dispatch 2024-09-10T09:48:21.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:20 smithi134 ceph-mon[35674]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "be51c7fb-2765-46a3-9861-c950f63de148"}]': finished 2024-09-10T09:48:21.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:20 smithi134 ceph-mon[35674]: osdmap e34: 6 total, 5 up, 6 in 2024-09-10T09:48:21.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:20 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T09:48:21.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:20 smithi134 ceph-mon[35674]: from='client.? 172.21.15.134:0/761473976' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:48:21.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:20 smithi120 ceph-mon[34770]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:48:21.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:20 smithi120 ceph-mon[34770]: from='client.? 172.21.15.134:0/4180747021' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "be51c7fb-2765-46a3-9861-c950f63de148"}]: dispatch 2024-09-10T09:48:21.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:20 smithi120 ceph-mon[34770]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "be51c7fb-2765-46a3-9861-c950f63de148"}]: dispatch 2024-09-10T09:48:21.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:20 smithi120 ceph-mon[34770]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "be51c7fb-2765-46a3-9861-c950f63de148"}]': finished 2024-09-10T09:48:21.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:20 smithi120 ceph-mon[34770]: osdmap e34: 6 total, 5 up, 6 in 2024-09-10T09:48:21.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:20 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T09:48:21.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:20 smithi120 ceph-mon[34770]: from='client.? 172.21.15.134:0/761473976' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:48:21.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:20 smithi077 ceph-mon[30319]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:48:21.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:20 smithi077 ceph-mon[30319]: from='client.? 172.21.15.134:0/4180747021' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "be51c7fb-2765-46a3-9861-c950f63de148"}]: dispatch 2024-09-10T09:48:21.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:20 smithi077 ceph-mon[30319]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "be51c7fb-2765-46a3-9861-c950f63de148"}]: dispatch 2024-09-10T09:48:21.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:20 smithi077 ceph-mon[30319]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "be51c7fb-2765-46a3-9861-c950f63de148"}]': finished 2024-09-10T09:48:21.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:20 smithi077 ceph-mon[30319]: osdmap e34: 6 total, 5 up, 6 in 2024-09-10T09:48:21.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:20 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T09:48:21.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:20 smithi077 ceph-mon[30319]: from='client.? 172.21.15.134:0/761473976' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:48:22.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:21 smithi134 ceph-mon[35674]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:48:22.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:21 smithi120 ceph-mon[34770]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:48:22.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:21 smithi077 ceph-mon[30319]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:48:24.923 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:24 smithi120 ceph-mon[34770]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:48:24.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:24 smithi077 ceph-mon[30319]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:48:25.079 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:24 smithi134 ceph-mon[35674]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:48:26.862 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:26 smithi134 ceph-mon[35674]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:48:26.923 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:26 smithi120 ceph-mon[34770]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:48:26.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:26 smithi077 ceph-mon[30319]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:48:28.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:28 smithi134 ceph-mon[35674]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:48:28.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:28 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-10T09:48:28.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:28 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:48:28.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:28 smithi077 ceph-mon[30319]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:48:28.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:28 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-10T09:48:28.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:28 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:48:29.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:28 smithi120 ceph-mon[34770]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:48:29.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:28 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-10T09:48:29.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:28 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:48:29.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:29 smithi077 ceph-mon[30319]: Deploying daemon osd.5 on smithi134 2024-09-10T09:48:29.984 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:29 smithi134 ceph-mon[35674]: Deploying daemon osd.5 on smithi134 2024-09-10T09:48:30.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:29 smithi120 ceph-mon[34770]: Deploying daemon osd.5 on smithi134 2024-09-10T09:48:30.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:30 smithi134 ceph-mon[35674]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:48:30.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:30 smithi077 ceph-mon[30319]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:48:31.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:30 smithi120 ceph-mon[34770]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:48:32.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:32 smithi077 ceph-mon[30319]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:48:33.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:32 smithi134 ceph-mon[35674]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:48:33.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:32 smithi120 ceph-mon[34770]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:48:35.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:34 smithi134 ceph-mon[35674]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:48:35.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:34 smithi120 ceph-mon[34770]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:48:35.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:34 smithi077 ceph-mon[30319]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:48:35.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:35 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:48:35.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:35 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:48:35.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:35 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:48:36.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:35 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:48:36.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:35 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:48:36.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:35 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:48:36.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:35 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:48:36.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:35 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:48:36.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:35 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:48:37.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:36 smithi134 ceph-mon[35674]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:48:37.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:36 smithi120 ceph-mon[34770]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:48:37.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:36 smithi077 ceph-mon[30319]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:48:38.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:38 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:48:38.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:38 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:48:38.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:38 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:48:38.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:38 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:48:38.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:38 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:48:38.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:38 smithi134 ceph-mon[35674]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:48:38.923 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:38 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:48:38.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:38 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:48:38.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:38 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:48:38.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:38 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:48:38.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:38 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:48:38.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:38 smithi120 ceph-mon[34770]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:48:38.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:38 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:48:38.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:38 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:48:38.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:38 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:48:38.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:38 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:48:38.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:38 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:48:38.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:38 smithi077 ceph-mon[30319]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:48:38.971 INFO:teuthology.orchestra.run.smithi134.stdout:Created osd(s) 5 on host 'smithi134' 2024-09-10T09:48:38.972 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:48:38.970+0000 7fefbcff9640 1 -- 172.21.15.134:0/118149857 <== mgr.14150 v2:172.21.15.77:6800/4261324650 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fefc0062630 con 0x7fefac0a2bb0 2024-09-10T09:48:38.974 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:48:38.972+0000 7fefc5437640 1 -- 172.21.15.134:0/118149857 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fefac0a2bb0 msgr2=0x7fefac0a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:38.974 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:48:38.972+0000 7fefc5437640 1 --2- 172.21.15.134:0/118149857 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fefac0a2bb0 0x7fefac0a5070 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fefb00314e0 tx=0x7fefb0000bc0 comp rx=0 tx=0).stop 2024-09-10T09:48:38.974 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:48:38.972+0000 7fefc5437640 1 -- 172.21.15.134:0/118149857 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fefc0107960 msgr2=0x7fefc019f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:38.974 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:48:38.972+0000 7fefc5437640 1 --2- 172.21.15.134:0/118149857 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fefc0107960 0x7fefc019f570 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fefb800b970 tx=0x7fefb800be30 comp rx=0 tx=0).stop 2024-09-10T09:48:38.974 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:48:38.972+0000 7fefc5437640 1 -- 172.21.15.134:0/118149857 shutdown_connections 2024-09-10T09:48:38.974 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:48:38.972+0000 7fefc5437640 1 --2- 172.21.15.134:0/118149857 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fefac0a2bb0 0x7fefac0a5070 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:38.974 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:48:38.972+0000 7fefc5437640 1 --2- 172.21.15.134:0/118149857 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fefc0107960 0x7fefc019f570 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:38.974 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:48:38.972+0000 7fefc5437640 1 --2- 172.21.15.134:0/118149857 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fefc0105030 0x7fefc01984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:38.974 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:48:38.972+0000 7fefc5437640 1 --2- 172.21.15.134:0/118149857 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fefc0102700 0x7fefc0197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:38.975 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:48:38.972+0000 7fefc5437640 1 -- 172.21.15.134:0/118149857 >> 172.21.15.134:0/118149857 conn(0x7fefc00fc280 msgr2=0x7fefc01030c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:48:38.975 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:48:38.973+0000 7fefc5437640 1 -- 172.21.15.134:0/118149857 shutdown_connections 2024-09-10T09:48:38.975 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:48:38.973+0000 7fefc5437640 1 -- 172.21.15.134:0/118149857 wait complete. 2024-09-10T09:48:39.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:39 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:48:39.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:39 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:48:39.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:39 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:48:39.659 DEBUG:teuthology.orchestra.run.smithi134:osd.5> sudo journalctl -f -n 0 -u ceph-14a55226-6f59-11ef-bcea-c7b262605968@osd.5.service 2024-09-10T09:48:39.662 INFO:tasks.cephadm:Deploying osd.6 on smithi077 with /dev/vg_nvme/lv_2... 2024-09-10T09:48:39.662 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-10T09:48:39.923 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:39 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:48:39.923 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:39 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:48:39.923 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:39 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:48:39.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:39 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:48:39.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:39 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:48:39.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:39 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:48:40.030 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:48:40.245 INFO:journalctl@ceph.osd.5.smithi134.stdout:Sep 10 09:48:40 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-5[47540]: 2024-09-10T09:48:40.191+0000 7f3724552640 -1 osd.5 0 log_to_monitors true 2024-09-10T09:48:40.861 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:40 smithi134 ceph-mon[35674]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:48:40.861 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:40 smithi134 ceph-mon[35674]: from='osd.5 [v2:172.21.15.134:6808/4081444621,v1:172.21.15.134:6809/4081444621]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T09:48:40.861 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:40 smithi134 ceph-mon[35674]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T09:48:40.923 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:40 smithi120 ceph-mon[34770]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:48:40.923 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:40 smithi120 ceph-mon[34770]: from='osd.5 [v2:172.21.15.134:6808/4081444621,v1:172.21.15.134:6809/4081444621]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T09:48:40.923 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:40 smithi120 ceph-mon[34770]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T09:48:40.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:40 smithi077 ceph-mon[30319]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:48:40.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:40 smithi077 ceph-mon[30319]: from='osd.5 [v2:172.21.15.134:6808/4081444621,v1:172.21.15.134:6809/4081444621]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T09:48:40.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:40 smithi077 ceph-mon[30319]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T09:48:41.658 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:41 smithi077 ceph-mon[30319]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-10T09:48:41.658 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:41 smithi077 ceph-mon[30319]: from='osd.5 [v2:172.21.15.134:6808/4081444621,v1:172.21.15.134:6809/4081444621]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-09-10T09:48:41.658 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:41 smithi077 ceph-mon[30319]: osdmap e35: 6 total, 5 up, 6 in 2024-09-10T09:48:41.658 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:41 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T09:48:41.658 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:41 smithi077 ceph-mon[30319]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-09-10T09:48:41.881 INFO:journalctl@ceph.osd.5.smithi134.stdout:Sep 10 09:48:41 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-5[47540]: 2024-09-10T09:48:41.539+0000 7f37204d5640 -1 osd.5 0 waiting for initial osdmap 2024-09-10T09:48:41.881 INFO:journalctl@ceph.osd.5.smithi134.stdout:Sep 10 09:48:41 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-5[47540]: 2024-09-10T09:48:41.551+0000 7f371c2fd640 -1 osd.5 36 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T09:48:41.882 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:41 smithi134 ceph-mon[35674]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-10T09:48:41.882 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:41 smithi134 ceph-mon[35674]: from='osd.5 [v2:172.21.15.134:6808/4081444621,v1:172.21.15.134:6809/4081444621]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-09-10T09:48:41.882 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:41 smithi134 ceph-mon[35674]: osdmap e35: 6 total, 5 up, 6 in 2024-09-10T09:48:41.882 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:41 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T09:48:41.882 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:41 smithi134 ceph-mon[35674]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-09-10T09:48:41.923 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:41 smithi120 ceph-mon[34770]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-10T09:48:41.923 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:41 smithi120 ceph-mon[34770]: from='osd.5 [v2:172.21.15.134:6808/4081444621,v1:172.21.15.134:6809/4081444621]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-09-10T09:48:41.923 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:41 smithi120 ceph-mon[34770]: osdmap e35: 6 total, 5 up, 6 in 2024-09-10T09:48:41.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:41 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T09:48:41.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:41 smithi120 ceph-mon[34770]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-09-10T09:48:42.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:42 smithi134 ceph-mon[35674]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-09-10T09:48:42.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:42 smithi134 ceph-mon[35674]: osdmap e36: 6 total, 5 up, 6 in 2024-09-10T09:48:42.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:42 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T09:48:42.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:42 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T09:48:42.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:42 smithi134 ceph-mon[35674]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:48:42.824 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:42 smithi120 ceph-mon[34770]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-09-10T09:48:42.825 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:42 smithi120 ceph-mon[34770]: osdmap e36: 6 total, 5 up, 6 in 2024-09-10T09:48:42.825 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:42 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T09:48:42.825 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:42 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T09:48:42.825 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:42 smithi120 ceph-mon[34770]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:48:42.866 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:42 smithi077 ceph-mon[30319]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-09-10T09:48:42.866 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:42 smithi077 ceph-mon[30319]: osdmap e36: 6 total, 5 up, 6 in 2024-09-10T09:48:42.866 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:42 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T09:48:42.866 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:42 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T09:48:42.867 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:42 smithi077 ceph-mon[30319]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:48:43.256 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:48:43.289 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi077:vg_nvme/lv_2 2024-09-10T09:48:43.674 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:48:43.820 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:43 smithi077 ceph-mon[30319]: purged_snaps scrub starts 2024-09-10T09:48:43.820 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:43 smithi077 ceph-mon[30319]: purged_snaps scrub ok 2024-09-10T09:48:43.820 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:43 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T09:48:43.821 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:43 smithi077 ceph-mon[30319]: osd.5 [v2:172.21.15.134:6808/4081444621,v1:172.21.15.134:6809/4081444621] boot 2024-09-10T09:48:43.821 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:43 smithi077 ceph-mon[30319]: osdmap e37: 6 total, 6 up, 6 in 2024-09-10T09:48:43.821 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:43 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T09:48:43.821 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:43 smithi077 ceph-mon[30319]: Detected new or changed devices on smithi134 2024-09-10T09:48:43.821 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:43 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:48:43.821 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:43 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:48:43.821 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:43 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:48:43.821 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:43 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:48:43.821 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:43 smithi077 ceph-mon[30319]: Adjusting osd_memory_target on smithi134 to 8544M 2024-09-10T09:48:43.821 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:43 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:48:43.821 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:43 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:48:43.821 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:43 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:48:43.821 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:43 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:48:43.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:43 smithi134 ceph-mon[35674]: purged_snaps scrub starts 2024-09-10T09:48:43.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:43 smithi134 ceph-mon[35674]: purged_snaps scrub ok 2024-09-10T09:48:43.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:43 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T09:48:43.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:43 smithi134 ceph-mon[35674]: osd.5 [v2:172.21.15.134:6808/4081444621,v1:172.21.15.134:6809/4081444621] boot 2024-09-10T09:48:43.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:43 smithi134 ceph-mon[35674]: osdmap e37: 6 total, 6 up, 6 in 2024-09-10T09:48:43.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:43 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T09:48:43.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:43 smithi134 ceph-mon[35674]: Detected new or changed devices on smithi134 2024-09-10T09:48:43.882 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:43 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:48:43.882 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:43 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:48:43.882 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:43 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:48:43.882 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:43 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:48:43.882 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:43 smithi134 ceph-mon[35674]: Adjusting osd_memory_target on smithi134 to 8544M 2024-09-10T09:48:43.882 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:43 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:48:43.882 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:43 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:48:43.882 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:43 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:48:43.882 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:43 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:48:43.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:43 smithi120 ceph-mon[34770]: purged_snaps scrub starts 2024-09-10T09:48:43.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:43 smithi120 ceph-mon[34770]: purged_snaps scrub ok 2024-09-10T09:48:43.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:43 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T09:48:43.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:43 smithi120 ceph-mon[34770]: osd.5 [v2:172.21.15.134:6808/4081444621,v1:172.21.15.134:6809/4081444621] boot 2024-09-10T09:48:43.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:43 smithi120 ceph-mon[34770]: osdmap e37: 6 total, 6 up, 6 in 2024-09-10T09:48:43.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:43 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T09:48:43.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:43 smithi120 ceph-mon[34770]: Detected new or changed devices on smithi134 2024-09-10T09:48:43.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:43 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:48:43.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:43 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:48:43.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:43 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:48:43.925 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:43 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:48:43.925 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:43 smithi120 ceph-mon[34770]: Adjusting osd_memory_target on smithi134 to 8544M 2024-09-10T09:48:43.925 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:43 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:48:43.925 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:43 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:48:43.925 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:43 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:48:43.925 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:43 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:48:44.316 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:48:44.314+0000 7f6aebe00640 1 -- 172.21.15.77:0/759263280 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6aec101a40 msgr2=0x7f6aec101e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:44.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:48:44.314+0000 7f6aebe00640 1 --2- 172.21.15.77:0/759263280 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6aec101a40 0x7f6aec101e40 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6adc009900 tx=0x7f6adc02a410 comp rx=0 tx=0).stop 2024-09-10T09:48:44.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:48:44.314+0000 7f6aebe00640 1 -- 172.21.15.77:0/759263280 shutdown_connections 2024-09-10T09:48:44.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:48:44.314+0000 7f6aebe00640 1 --2- 172.21.15.77:0/759263280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6aec103600 0x7f6aec109e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:44.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:48:44.314+0000 7f6aebe00640 1 --2- 172.21.15.77:0/759263280 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6aec102c40 0x7f6aec1030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:44.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:48:44.314+0000 7f6aebe00640 1 --2- 172.21.15.77:0/759263280 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6aec101a40 0x7f6aec101e40 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:44.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:48:44.314+0000 7f6aebe00640 1 -- 172.21.15.77:0/759263280 >> 172.21.15.77:0/759263280 conn(0x7f6aec0fd1f0 msgr2=0x7f6aec0ff610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:48:44.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:48:44.314+0000 7f6aebe00640 1 -- 172.21.15.77:0/759263280 shutdown_connections 2024-09-10T09:48:44.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:48:44.314+0000 7f6aebe00640 1 -- 172.21.15.77:0/759263280 wait complete. 2024-09-10T09:48:44.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:48:44.315+0000 7f6aebe00640 1 Processor -- start 2024-09-10T09:48:44.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:48:44.315+0000 7f6aebe00640 1 -- start start 2024-09-10T09:48:44.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:48:44.315+0000 7f6aebe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6aec101a40 0x7f6aec199ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:44.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:48:44.315+0000 7f6aebe00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6aec102c40 0x7f6aec19a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:44.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:48:44.316+0000 7f6aebe00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6aec103600 0x7f6aec1a14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:44.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:48:44.316+0000 7f6aebe00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6aec10d720 con 0x7f6aec102c40 2024-09-10T09:48:44.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:48:44.316+0000 7f6aebe00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6aec10d5a0 con 0x7f6aec103600 2024-09-10T09:48:44.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:48:44.316+0000 7f6aebe00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6aec10d8a0 con 0x7f6aec101a40 2024-09-10T09:48:44.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:48:44.316+0000 7f6aeaa00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6aec101a40 0x7f6aec199ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:44.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:48:44.316+0000 7f6aea000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6aec102c40 0x7f6aec19a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:44.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:48:44.316+0000 7f6aeaa00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6aec101a40 0x7f6aec199ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:47316/0 (socket says 172.21.15.77:47316) 2024-09-10T09:48:44.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:48:44.316+0000 7f6aeaa00640 1 -- 172.21.15.77:0/1678528618 learned_addr learned my addr 172.21.15.77:0/1678528618 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:48:44.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:48:44.316+0000 7f6aeb400640 1 --2- 172.21.15.77:0/1678528618 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6aec103600 0x7f6aec1a14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:44.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:48:44.316+0000 7f6aeaa00640 1 -- 172.21.15.77:0/1678528618 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6aec103600 msgr2=0x7f6aec1a14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:44.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:48:44.316+0000 7f6aeaa00640 1 --2- 172.21.15.77:0/1678528618 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6aec103600 0x7f6aec1a14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:44.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:48:44.316+0000 7f6aeaa00640 1 -- 172.21.15.77:0/1678528618 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6aec102c40 msgr2=0x7f6aec19a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:44.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:48:44.316+0000 7f6aeaa00640 1 --2- 172.21.15.77:0/1678528618 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6aec102c40 0x7f6aec19a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:44.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:48:44.316+0000 7f6aeaa00640 1 -- 172.21.15.77:0/1678528618 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6aec1a1b10 con 0x7f6aec101a40 2024-09-10T09:48:44.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:48:44.316+0000 7f6aea000640 1 --2- 172.21.15.77:0/1678528618 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6aec102c40 0x7f6aec19a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:48:44.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:48:44.317+0000 7f6aeaa00640 1 --2- 172.21.15.77:0/1678528618 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6aec101a40 0x7f6aec199ee0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f6adc031440 tx=0x7f6adc032ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:48:44.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:48:44.317+0000 7f6ae3400640 1 -- 172.21.15.77:0/1678528618 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6adc03f030 con 0x7f6aec101a40 2024-09-10T09:48:44.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:48:44.317+0000 7f6ae3400640 1 -- 172.21.15.77:0/1678528618 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6adc003ce0 con 0x7f6aec101a40 2024-09-10T09:48:44.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:48:44.317+0000 7f6aebe00640 1 -- 172.21.15.77:0/1678528618 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6aec1a1da0 con 0x7f6aec101a40 2024-09-10T09:48:44.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:48:44.317+0000 7f6ae3400640 1 -- 172.21.15.77:0/1678528618 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6adc035070 con 0x7f6aec101a40 2024-09-10T09:48:44.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:48:44.317+0000 7f6aebe00640 1 -- 172.21.15.77:0/1678528618 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6aec1a21c0 con 0x7f6aec101a40 2024-09-10T09:48:44.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:48:44.318+0000 7f6aebe00640 1 -- 172.21.15.77:0/1678528618 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6aec1a23f0 con 0x7f6aec101a40 2024-09-10T09:48:44.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:48:44.320+0000 7f6ae3400640 1 -- 172.21.15.77:0/1678528618 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f6adc042030 con 0x7f6aec101a40 2024-09-10T09:48:44.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:48:44.321+0000 7f6ae3400640 1 --2- 172.21.15.77:0/1678528618 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f6ad00a2bb0 0x7f6ad00a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:44.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:48:44.321+0000 7f6ae3400640 1 -- 172.21.15.77:0/1678528618 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4562+0+0 (secure 0 0 0) 0x7f6adc0eea30 con 0x7f6aec101a40 2024-09-10T09:48:44.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:48:44.321+0000 7f6aea000640 1 --2- 172.21.15.77:0/1678528618 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f6ad00a2bb0 0x7f6ad00a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:44.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:48:44.322+0000 7f6aea000640 1 --2- 172.21.15.77:0/1678528618 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f6ad00a2bb0 0x7f6ad00a5070 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6aec19b400 tx=0x7f6ad8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:48:44.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:48:44.323+0000 7f6ae3400640 1 -- 172.21.15.77:0/1678528618 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6adc0bc140 con 0x7f6aec101a40 2024-09-10T09:48:44.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:48:44.508+0000 7f6aebe00640 1 -- 172.21.15.77:0/1678528618 --> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f6aec1a2680 con 0x7f6ad00a2bb0 2024-09-10T09:48:44.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:44 smithi134 ceph-mon[35674]: osdmap e38: 6 total, 6 up, 6 in 2024-09-10T09:48:44.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:44 smithi134 ceph-mon[35674]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-09-10T09:48:44.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:44 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:48:44.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:44 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:48:44.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:44 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:48:44.923 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:44 smithi120 ceph-mon[34770]: osdmap e38: 6 total, 6 up, 6 in 2024-09-10T09:48:44.923 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:44 smithi120 ceph-mon[34770]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-09-10T09:48:44.923 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:44 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:48:44.923 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:44 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:48:44.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:44 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:48:44.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:44 smithi077 ceph-mon[30319]: osdmap e38: 6 total, 6 up, 6 in 2024-09-10T09:48:44.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:44 smithi077 ceph-mon[30319]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-09-10T09:48:44.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:44 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:48:44.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:44 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:48:44.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:44 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:48:45.648 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:45 smithi077 ceph-mon[30319]: from='client.24277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:48:45.649 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:45 smithi077 ceph-mon[30319]: osdmap e39: 6 total, 6 up, 6 in 2024-09-10T09:48:45.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:45 smithi134 ceph-mon[35674]: from='client.24277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:48:45.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:45 smithi134 ceph-mon[35674]: osdmap e39: 6 total, 6 up, 6 in 2024-09-10T09:48:45.923 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:45 smithi120 ceph-mon[34770]: from='client.24277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:48:45.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:45 smithi120 ceph-mon[34770]: osdmap e39: 6 total, 6 up, 6 in 2024-09-10T09:48:46.662 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:46 smithi077 ceph-mon[30319]: osdmap e40: 6 total, 6 up, 6 in 2024-09-10T09:48:46.662 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:46 smithi077 ceph-mon[30319]: pgmap v143: 1 pgs: 1 remapped+peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-10T09:48:46.662 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:46 smithi077 ceph-mon[30319]: osdmap e41: 6 total, 6 up, 6 in 2024-09-10T09:48:46.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:46 smithi134 ceph-mon[35674]: osdmap e40: 6 total, 6 up, 6 in 2024-09-10T09:48:46.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:46 smithi134 ceph-mon[35674]: pgmap v143: 1 pgs: 1 remapped+peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-10T09:48:46.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:46 smithi134 ceph-mon[35674]: osdmap e41: 6 total, 6 up, 6 in 2024-09-10T09:48:46.923 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:46 smithi120 ceph-mon[34770]: osdmap e40: 6 total, 6 up, 6 in 2024-09-10T09:48:46.923 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:46 smithi120 ceph-mon[34770]: pgmap v143: 1 pgs: 1 remapped+peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-10T09:48:46.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:46 smithi120 ceph-mon[34770]: osdmap e41: 6 total, 6 up, 6 in 2024-09-10T09:48:48.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:48 smithi077 ceph-mon[30319]: pgmap v145: 1 pgs: 1 remapped+peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-10T09:48:48.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:48 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/1356017839' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4eb403ab-473b-41cc-803d-18c908f768a9"}]: dispatch 2024-09-10T09:48:48.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:48 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/1356017839' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4eb403ab-473b-41cc-803d-18c908f768a9"}]': finished 2024-09-10T09:48:48.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:48 smithi077 ceph-mon[30319]: osdmap e42: 7 total, 6 up, 7 in 2024-09-10T09:48:48.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:48 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T09:48:48.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:48 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/558319174' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:48:49.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:48 smithi134 ceph-mon[35674]: pgmap v145: 1 pgs: 1 remapped+peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-10T09:48:49.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:48 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/1356017839' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4eb403ab-473b-41cc-803d-18c908f768a9"}]: dispatch 2024-09-10T09:48:49.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:48 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/1356017839' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4eb403ab-473b-41cc-803d-18c908f768a9"}]': finished 2024-09-10T09:48:49.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:48 smithi134 ceph-mon[35674]: osdmap e42: 7 total, 6 up, 7 in 2024-09-10T09:48:49.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:48 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T09:48:49.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:48 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/558319174' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:48:49.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:48 smithi120 ceph-mon[34770]: pgmap v145: 1 pgs: 1 remapped+peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-10T09:48:49.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:48 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/1356017839' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4eb403ab-473b-41cc-803d-18c908f768a9"}]: dispatch 2024-09-10T09:48:49.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:48 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/1356017839' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4eb403ab-473b-41cc-803d-18c908f768a9"}]': finished 2024-09-10T09:48:49.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:48 smithi120 ceph-mon[34770]: osdmap e42: 7 total, 6 up, 7 in 2024-09-10T09:48:49.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:48 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T09:48:49.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:48 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/558319174' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:48:50.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:50 smithi077 ceph-mon[30319]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-09-10T09:48:51.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:50 smithi134 ceph-mon[35674]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-09-10T09:48:51.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:50 smithi120 ceph-mon[34770]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-09-10T09:48:52.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:52 smithi077 ceph-mon[30319]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-09-10T09:48:53.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:52 smithi134 ceph-mon[35674]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-09-10T09:48:53.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:52 smithi120 ceph-mon[34770]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-09-10T09:48:54.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:54 smithi077 ceph-mon[30319]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T09:48:55.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:54 smithi134 ceph-mon[35674]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T09:48:55.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:54 smithi120 ceph-mon[34770]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T09:48:56.896 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:56 smithi077 ceph-mon[30319]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-09-10T09:48:57.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:56 smithi134 ceph-mon[35674]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-09-10T09:48:57.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:56 smithi120 ceph-mon[34770]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-09-10T09:48:57.833 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:57 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-10T09:48:57.833 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:57 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:48:58.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:57 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-10T09:48:58.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:57 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:48:58.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:57 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-10T09:48:58.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:57 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:48:58.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:58 smithi077 ceph-mon[30319]: Deploying daemon osd.6 on smithi077 2024-09-10T09:48:58.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:48:58 smithi077 ceph-mon[30319]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T09:48:59.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:58 smithi134 ceph-mon[35674]: Deploying daemon osd.6 on smithi077 2024-09-10T09:48:59.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:48:58 smithi134 ceph-mon[35674]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T09:48:59.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:58 smithi120 ceph-mon[34770]: Deploying daemon osd.6 on smithi077 2024-09-10T09:48:59.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:48:58 smithi120 ceph-mon[34770]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T09:49:00.858 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:00 smithi077 ceph-mon[30319]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-09-10T09:49:01.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:00 smithi134 ceph-mon[35674]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-09-10T09:49:01.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:00 smithi120 ceph-mon[34770]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-09-10T09:49:03.077 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:02 smithi077 ceph-mon[30319]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-10T09:49:03.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:02 smithi134 ceph-mon[35674]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-10T09:49:03.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:02 smithi120 ceph-mon[34770]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-10T09:49:04.829 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:04 smithi077 ceph-mon[30319]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-10T09:49:05.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:04 smithi134 ceph-mon[35674]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-10T09:49:05.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:04 smithi120 ceph-mon[34770]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-10T09:49:05.917 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:05 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:49:05.918 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:05 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:49:05.918 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:05 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:49:06.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:05 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:49:06.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:05 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:49:06.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:05 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:49:06.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:05 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:49:06.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:05 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:49:06.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:05 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:49:07.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:06 smithi134 ceph-mon[35674]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-10T09:49:07.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:06 smithi120 ceph-mon[34770]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-10T09:49:07.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:06 smithi077 ceph-mon[30319]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-10T09:49:09.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:08 smithi134 ceph-mon[35674]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-10T09:49:09.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:08 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:49:09.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:08 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:49:09.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:08 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:49:09.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:08 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:49:09.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:08 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:49:09.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:08 smithi120 ceph-mon[34770]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-10T09:49:09.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:08 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:49:09.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:08 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:49:09.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:08 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:49:09.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:08 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:49:09.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:08 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:49:09.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:08 smithi077 ceph-mon[30319]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-10T09:49:09.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:08 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:49:09.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:08 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:49:09.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:08 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:49:09.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:08 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:49:09.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:08 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:49:09.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:49:09.739+0000 7f6ae3400640 1 -- 172.21.15.77:0/1678528618 <== mgr.14150 v2:172.21.15.77:6800/4261324650 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6aec1a2680 con 0x7f6ad00a2bb0 2024-09-10T09:49:09.740 INFO:teuthology.orchestra.run.smithi077.stdout:Created osd(s) 6 on host 'smithi077' 2024-09-10T09:49:09.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:49:09.742+0000 7f6aebe00640 1 -- 172.21.15.77:0/1678528618 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f6ad00a2bb0 msgr2=0x7f6ad00a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:09.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:49:09.742+0000 7f6aebe00640 1 --2- 172.21.15.77:0/1678528618 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f6ad00a2bb0 0x7f6ad00a5070 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6aec19b400 tx=0x7f6ad8005f90 comp rx=0 tx=0).stop 2024-09-10T09:49:09.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:49:09.742+0000 7f6aebe00640 1 -- 172.21.15.77:0/1678528618 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6aec101a40 msgr2=0x7f6aec199ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:09.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:49:09.742+0000 7f6aebe00640 1 --2- 172.21.15.77:0/1678528618 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6aec101a40 0x7f6aec199ee0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f6adc031440 tx=0x7f6adc032ce0 comp rx=0 tx=0).stop 2024-09-10T09:49:09.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:49:09.742+0000 7f6aebe00640 1 -- 172.21.15.77:0/1678528618 shutdown_connections 2024-09-10T09:49:09.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:49:09.742+0000 7f6aebe00640 1 --2- 172.21.15.77:0/1678528618 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f6ad00a2bb0 0x7f6ad00a5070 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:09.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:49:09.742+0000 7f6aebe00640 1 --2- 172.21.15.77:0/1678528618 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6aec103600 0x7f6aec1a14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:09.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:49:09.742+0000 7f6aebe00640 1 --2- 172.21.15.77:0/1678528618 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6aec102c40 0x7f6aec19a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:09.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:49:09.742+0000 7f6aebe00640 1 --2- 172.21.15.77:0/1678528618 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6aec101a40 0x7f6aec199ee0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:09.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:49:09.742+0000 7f6aebe00640 1 -- 172.21.15.77:0/1678528618 >> 172.21.15.77:0/1678528618 conn(0x7f6aec0fd1f0 msgr2=0x7f6aec074420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:49:09.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:49:09.742+0000 7f6aebe00640 1 -- 172.21.15.77:0/1678528618 shutdown_connections 2024-09-10T09:49:09.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:49:09.742+0000 7f6aebe00640 1 -- 172.21.15.77:0/1678528618 wait complete. 2024-09-10T09:49:10.607 DEBUG:teuthology.orchestra.run.smithi077:osd.6> sudo journalctl -f -n 0 -u ceph-14a55226-6f59-11ef-bcea-c7b262605968@osd.6.service 2024-09-10T09:49:10.611 INFO:tasks.cephadm:Deploying osd.7 on smithi120 with /dev/vg_nvme/lv_2... 2024-09-10T09:49:10.611 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-10T09:49:10.855 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:10 smithi077 ceph-mon[30319]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-10T09:49:10.856 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:10 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:49:10.856 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:10 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:49:10.856 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:10 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:49:10.856 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:10 smithi077 ceph-mon[30319]: from='osd.6 [v2:172.21.15.77:6818/879565639,v1:172.21.15.77:6819/879565639]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-10T09:49:10.856 INFO:journalctl@ceph.osd.6.smithi077.stdout:Sep 10 09:49:10 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-6[57286]: 2024-09-10T09:49:10.701+0000 7f175ba3c640 -1 osd.6 0 log_to_monitors true 2024-09-10T09:49:10.867 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T09:49:10.923 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:10 smithi120 ceph-mon[34770]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-10T09:49:10.923 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:10 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:49:10.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:10 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:49:10.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:10 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:49:10.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:10 smithi120 ceph-mon[34770]: from='osd.6 [v2:172.21.15.77:6818/879565639,v1:172.21.15.77:6819/879565639]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-10T09:49:11.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:10 smithi134 ceph-mon[35674]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-10T09:49:11.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:10 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:49:11.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:10 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:49:11.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:10 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:49:11.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:10 smithi134 ceph-mon[35674]: from='osd.6 [v2:172.21.15.77:6818/879565639,v1:172.21.15.77:6819/879565639]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-10T09:49:11.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:11 smithi077 ceph-mon[30319]: from='osd.6 [v2:172.21.15.77:6818/879565639,v1:172.21.15.77:6819/879565639]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-10T09:49:11.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:11 smithi077 ceph-mon[30319]: osdmap e43: 7 total, 6 up, 7 in 2024-09-10T09:49:11.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:11 smithi077 ceph-mon[30319]: from='osd.6 [v2:172.21.15.77:6818/879565639,v1:172.21.15.77:6819/879565639]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-09-10T09:49:11.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:11 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T09:49:12.002 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:11 smithi120 ceph-mon[34770]: from='osd.6 [v2:172.21.15.77:6818/879565639,v1:172.21.15.77:6819/879565639]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-10T09:49:12.002 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:11 smithi120 ceph-mon[34770]: osdmap e43: 7 total, 6 up, 7 in 2024-09-10T09:49:12.002 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:11 smithi120 ceph-mon[34770]: from='osd.6 [v2:172.21.15.77:6818/879565639,v1:172.21.15.77:6819/879565639]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-09-10T09:49:12.002 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:11 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T09:49:12.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:11 smithi134 ceph-mon[35674]: from='osd.6 [v2:172.21.15.77:6818/879565639,v1:172.21.15.77:6819/879565639]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-10T09:49:12.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:11 smithi134 ceph-mon[35674]: osdmap e43: 7 total, 6 up, 7 in 2024-09-10T09:49:12.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:11 smithi134 ceph-mon[35674]: from='osd.6 [v2:172.21.15.77:6818/879565639,v1:172.21.15.77:6819/879565639]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-09-10T09:49:12.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:11 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T09:49:12.536 INFO:journalctl@ceph.osd.6.smithi077.stdout:Sep 10 09:49:12 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-6[57286]: 2024-09-10T09:49:12.235+0000 7f1756c00640 -1 osd.6 0 waiting for initial osdmap 2024-09-10T09:49:12.536 INFO:journalctl@ceph.osd.6.smithi077.stdout:Sep 10 09:49:12 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-6[57286]: 2024-09-10T09:49:12.266+0000 7f174c200640 -1 osd.6 44 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T09:49:12.873 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:12 smithi120 ceph-mon[34770]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-10T09:49:12.873 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:12 smithi120 ceph-mon[34770]: from='osd.6 [v2:172.21.15.77:6818/879565639,v1:172.21.15.77:6819/879565639]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-09-10T09:49:12.873 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:12 smithi120 ceph-mon[34770]: osdmap e44: 7 total, 6 up, 7 in 2024-09-10T09:49:12.873 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:12 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T09:49:12.873 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:12 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T09:49:12.873 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:12 smithi120 ceph-mon[34770]: from='osd.6 [v2:172.21.15.77:6818/879565639,v1:172.21.15.77:6819/879565639]' entity='osd.6' 2024-09-10T09:49:12.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:12 smithi077 ceph-mon[30319]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-10T09:49:12.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:12 smithi077 ceph-mon[30319]: from='osd.6 [v2:172.21.15.77:6818/879565639,v1:172.21.15.77:6819/879565639]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-09-10T09:49:12.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:12 smithi077 ceph-mon[30319]: osdmap e44: 7 total, 6 up, 7 in 2024-09-10T09:49:12.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:12 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T09:49:12.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:12 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T09:49:12.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:12 smithi077 ceph-mon[30319]: from='osd.6 [v2:172.21.15.77:6818/879565639,v1:172.21.15.77:6819/879565639]' entity='osd.6' 2024-09-10T09:49:13.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:12 smithi134 ceph-mon[35674]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-10T09:49:13.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:12 smithi134 ceph-mon[35674]: from='osd.6 [v2:172.21.15.77:6818/879565639,v1:172.21.15.77:6819/879565639]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-09-10T09:49:13.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:12 smithi134 ceph-mon[35674]: osdmap e44: 7 total, 6 up, 7 in 2024-09-10T09:49:13.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:12 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T09:49:13.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:12 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T09:49:13.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:12 smithi134 ceph-mon[35674]: from='osd.6 [v2:172.21.15.77:6818/879565639,v1:172.21.15.77:6819/879565639]' entity='osd.6' 2024-09-10T09:49:13.536 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T09:49:13.564 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi120:vg_nvme/lv_2 2024-09-10T09:49:13.817 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T09:49:13.838 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:13 smithi120 ceph-mon[34770]: purged_snaps scrub starts 2024-09-10T09:49:13.838 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:13 smithi120 ceph-mon[34770]: purged_snaps scrub ok 2024-09-10T09:49:13.838 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:13 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T09:49:13.838 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:13 smithi120 ceph-mon[34770]: osd.6 [v2:172.21.15.77:6818/879565639,v1:172.21.15.77:6819/879565639] boot 2024-09-10T09:49:13.838 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:13 smithi120 ceph-mon[34770]: osdmap e45: 7 total, 7 up, 7 in 2024-09-10T09:49:13.838 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:13 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T09:49:13.924 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:13 smithi077 ceph-mon[30319]: purged_snaps scrub starts 2024-09-10T09:49:13.924 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:13 smithi077 ceph-mon[30319]: purged_snaps scrub ok 2024-09-10T09:49:13.924 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:13 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T09:49:13.924 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:13 smithi077 ceph-mon[30319]: osd.6 [v2:172.21.15.77:6818/879565639,v1:172.21.15.77:6819/879565639] boot 2024-09-10T09:49:13.924 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:13 smithi077 ceph-mon[30319]: osdmap e45: 7 total, 7 up, 7 in 2024-09-10T09:49:13.925 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:13 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T09:49:14.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:13 smithi134 ceph-mon[35674]: purged_snaps scrub starts 2024-09-10T09:49:14.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:13 smithi134 ceph-mon[35674]: purged_snaps scrub ok 2024-09-10T09:49:14.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:13 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T09:49:14.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:13 smithi134 ceph-mon[35674]: osd.6 [v2:172.21.15.77:6818/879565639,v1:172.21.15.77:6819/879565639] boot 2024-09-10T09:49:14.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:13 smithi134 ceph-mon[35674]: osdmap e45: 7 total, 7 up, 7 in 2024-09-10T09:49:14.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:13 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T09:49:14.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:49:14.264+0000 7f0661187640 1 -- 172.21.15.120:0/1595997101 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f065c101c00 msgr2=0x7f065c102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:14.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:49:14.264+0000 7f0661187640 1 --2- 172.21.15.120:0/1595997101 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f065c101c00 0x7f065c102000 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f064c009900 tx=0x7f064c02a410 comp rx=0 tx=0).stop 2024-09-10T09:49:14.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:49:14.264+0000 7f0661187640 1 -- 172.21.15.120:0/1595997101 shutdown_connections 2024-09-10T09:49:14.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:49:14.264+0000 7f0661187640 1 --2- 172.21.15.120:0/1595997101 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f065c1037c0 0x7f065c10a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:14.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:49:14.264+0000 7f0661187640 1 --2- 172.21.15.120:0/1595997101 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f065c102e00 0x7f065c103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:14.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:49:14.264+0000 7f0661187640 1 --2- 172.21.15.120:0/1595997101 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f065c101c00 0x7f065c102000 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:14.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:49:14.264+0000 7f0661187640 1 -- 172.21.15.120:0/1595997101 >> 172.21.15.120:0/1595997101 conn(0x7f065c0fd390 msgr2=0x7f065c0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:49:14.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:49:14.264+0000 7f0661187640 1 -- 172.21.15.120:0/1595997101 shutdown_connections 2024-09-10T09:49:14.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:49:14.264+0000 7f0661187640 1 -- 172.21.15.120:0/1595997101 wait complete. 2024-09-10T09:49:14.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:49:14.265+0000 7f0661187640 1 Processor -- start 2024-09-10T09:49:14.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:49:14.265+0000 7f0661187640 1 -- start start 2024-09-10T09:49:14.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:49:14.265+0000 7f0661187640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f065c101c00 0x7f065c197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:14.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:49:14.265+0000 7f0661187640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f065c102e00 0x7f065c198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:14.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:49:14.265+0000 7f0661187640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f065c1037c0 0x7f065c19f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:14.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:49:14.265+0000 7f0661187640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f065c10d860 con 0x7f065c102e00 2024-09-10T09:49:14.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:49:14.265+0000 7f0661187640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f065c10d6e0 con 0x7f065c101c00 2024-09-10T09:49:14.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:49:14.265+0000 7f0661187640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f065c10d9e0 con 0x7f065c1037c0 2024-09-10T09:49:14.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:49:14.265+0000 7f065ad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f065c101c00 0x7f065c197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:14.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:49:14.265+0000 7f065ad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f065c101c00 0x7f065c197f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:54266/0 (socket says 172.21.15.120:54266) 2024-09-10T09:49:14.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:49:14.265+0000 7f065ad9d640 1 -- 172.21.15.120:0/2555101936 learned_addr learned my addr 172.21.15.120:0/2555101936 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T09:49:14.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:49:14.265+0000 7f065a59c640 1 --2- 172.21.15.120:0/2555101936 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f065c102e00 0x7f065c198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:14.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:49:14.266+0000 7f065b59e640 1 --2- 172.21.15.120:0/2555101936 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f065c1037c0 0x7f065c19f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:14.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:49:14.266+0000 7f065ad9d640 1 -- 172.21.15.120:0/2555101936 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f065c1037c0 msgr2=0x7f065c19f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:14.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:49:14.266+0000 7f065ad9d640 1 --2- 172.21.15.120:0/2555101936 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f065c1037c0 0x7f065c19f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:14.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:49:14.266+0000 7f065ad9d640 1 -- 172.21.15.120:0/2555101936 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f065c102e00 msgr2=0x7f065c198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:14.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:49:14.266+0000 7f065ad9d640 1 --2- 172.21.15.120:0/2555101936 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f065c102e00 0x7f065c198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:14.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:49:14.266+0000 7f065ad9d640 1 -- 172.21.15.120:0/2555101936 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f065c19fb50 con 0x7f065c101c00 2024-09-10T09:49:14.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:49:14.266+0000 7f065b59e640 1 --2- 172.21.15.120:0/2555101936 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f065c1037c0 0x7f065c19f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:49:14.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:49:14.266+0000 7f065ad9d640 1 --2- 172.21.15.120:0/2555101936 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f065c101c00 0x7f065c197f20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f064c031610 tx=0x7f064c0043b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:49:14.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:49:14.266+0000 7f0643fff640 1 -- 172.21.15.120:0/2555101936 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f064c03f030 con 0x7f065c101c00 2024-09-10T09:49:14.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:49:14.266+0000 7f0661187640 1 -- 172.21.15.120:0/2555101936 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f065c19fde0 con 0x7f065c101c00 2024-09-10T09:49:14.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:49:14.266+0000 7f0643fff640 1 -- 172.21.15.120:0/2555101936 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f064c003ce0 con 0x7f065c101c00 2024-09-10T09:49:14.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:49:14.266+0000 7f0643fff640 1 -- 172.21.15.120:0/2555101936 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f064c035070 con 0x7f065c101c00 2024-09-10T09:49:14.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:49:14.266+0000 7f0661187640 1 -- 172.21.15.120:0/2555101936 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f065c1a0230 con 0x7f065c101c00 2024-09-10T09:49:14.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:49:14.267+0000 7f0661187640 1 -- 172.21.15.120:0/2555101936 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f065c109040 con 0x7f065c101c00 2024-09-10T09:49:14.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:49:14.268+0000 7f0643fff640 1 -- 172.21.15.120:0/2555101936 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f064c042030 con 0x7f065c101c00 2024-09-10T09:49:14.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:49:14.269+0000 7f0643fff640 1 --2- 172.21.15.120:0/2555101936 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f06440a2c30 0x7f06440a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:14.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:49:14.269+0000 7f0643fff640 1 -- 172.21.15.120:0/2555101936 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 4965+0+0 (secure 0 0 0) 0x7f064c0ee820 con 0x7f065c101c00 2024-09-10T09:49:14.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:49:14.269+0000 7f065a59c640 1 --2- 172.21.15.120:0/2555101936 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f06440a2c30 0x7f06440a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:14.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:49:14.270+0000 7f065a59c640 1 --2- 172.21.15.120:0/2555101936 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f06440a2c30 0x7f06440a50f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f065c199440 tx=0x7f065000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:49:14.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:49:14.273+0000 7f0643fff640 1 -- 172.21.15.120:0/2555101936 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f064c0bc490 con 0x7f065c101c00 2024-09-10T09:49:14.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:49:14.451+0000 7f0661187640 1 -- 172.21.15.120:0/2555101936 --> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f065c062630 con 0x7f06440a2c30 2024-09-10T09:49:15.031 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:14 smithi120 ceph-mon[34770]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-10T09:49:15.031 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:14 smithi120 ceph-mon[34770]: osdmap e46: 7 total, 7 up, 7 in 2024-09-10T09:49:15.031 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:14 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:49:15.031 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:14 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:49:15.031 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:14 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:49:15.031 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:14 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:49:15.031 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:14 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:49:15.031 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:14 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:49:15.032 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:14 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:49:15.032 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:14 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:49:15.032 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:14 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:49:15.032 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:14 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:49:15.032 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:14 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:49:15.032 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:14 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:49:15.092 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:14 smithi077 ceph-mon[30319]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-10T09:49:15.092 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:14 smithi077 ceph-mon[30319]: osdmap e46: 7 total, 7 up, 7 in 2024-09-10T09:49:15.092 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:14 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:49:15.092 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:14 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:49:15.092 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:14 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:49:15.092 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:14 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:49:15.093 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:14 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:49:15.093 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:14 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:49:15.093 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:14 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:49:15.093 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:14 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:49:15.093 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:14 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:49:15.093 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:14 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:49:15.093 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:14 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:49:15.093 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:14 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:49:15.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:14 smithi134 ceph-mon[35674]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-10T09:49:15.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:14 smithi134 ceph-mon[35674]: osdmap e46: 7 total, 7 up, 7 in 2024-09-10T09:49:15.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:14 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:49:15.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:14 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:49:15.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:14 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:49:15.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:14 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:49:15.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:14 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:49:15.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:14 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:49:15.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:14 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:49:15.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:14 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:49:15.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:14 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:49:15.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:14 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:49:15.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:14 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:49:15.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:14 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:49:15.896 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:15 smithi120 ceph-mon[34770]: from='client.24296 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:49:15.896 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:15 smithi120 ceph-mon[34770]: Detected new or changed devices on smithi077 2024-09-10T09:49:15.896 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:15 smithi120 ceph-mon[34770]: Adjusting osd_memory_target on smithi077 to 5696M 2024-09-10T09:49:16.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:15 smithi134 ceph-mon[35674]: from='client.24296 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:49:16.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:15 smithi134 ceph-mon[35674]: Detected new or changed devices on smithi077 2024-09-10T09:49:16.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:15 smithi134 ceph-mon[35674]: Adjusting osd_memory_target on smithi077 to 5696M 2024-09-10T09:49:16.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:15 smithi077 ceph-mon[30319]: from='client.24296 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:49:16.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:15 smithi077 ceph-mon[30319]: Detected new or changed devices on smithi077 2024-09-10T09:49:16.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:15 smithi077 ceph-mon[30319]: Adjusting osd_memory_target on smithi077 to 5696M 2024-09-10T09:49:16.923 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:16 smithi120 ceph-mon[34770]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:49:17.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:16 smithi134 ceph-mon[35674]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:49:17.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:16 smithi077 ceph-mon[30319]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:49:18.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:17 smithi134 ceph-mon[35674]: from='client.? 172.21.15.120:0/171466028' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ba982bd0-c236-456c-9889-61087d79c1aa"}]: dispatch 2024-09-10T09:49:18.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:17 smithi134 ceph-mon[35674]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ba982bd0-c236-456c-9889-61087d79c1aa"}]: dispatch 2024-09-10T09:49:18.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:17 smithi134 ceph-mon[35674]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ba982bd0-c236-456c-9889-61087d79c1aa"}]': finished 2024-09-10T09:49:18.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:17 smithi134 ceph-mon[35674]: osdmap e47: 8 total, 7 up, 8 in 2024-09-10T09:49:18.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:17 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T09:49:18.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:17 smithi134 ceph-mon[35674]: from='client.? 172.21.15.120:0/1495345426' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:49:18.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:17 smithi120 ceph-mon[34770]: from='client.? 172.21.15.120:0/171466028' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ba982bd0-c236-456c-9889-61087d79c1aa"}]: dispatch 2024-09-10T09:49:18.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:17 smithi120 ceph-mon[34770]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ba982bd0-c236-456c-9889-61087d79c1aa"}]: dispatch 2024-09-10T09:49:18.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:17 smithi120 ceph-mon[34770]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ba982bd0-c236-456c-9889-61087d79c1aa"}]': finished 2024-09-10T09:49:18.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:17 smithi120 ceph-mon[34770]: osdmap e47: 8 total, 7 up, 8 in 2024-09-10T09:49:18.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:17 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T09:49:18.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:17 smithi120 ceph-mon[34770]: from='client.? 172.21.15.120:0/1495345426' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:49:18.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:17 smithi077 ceph-mon[30319]: from='client.? 172.21.15.120:0/171466028' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ba982bd0-c236-456c-9889-61087d79c1aa"}]: dispatch 2024-09-10T09:49:18.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:17 smithi077 ceph-mon[30319]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ba982bd0-c236-456c-9889-61087d79c1aa"}]: dispatch 2024-09-10T09:49:18.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:17 smithi077 ceph-mon[30319]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ba982bd0-c236-456c-9889-61087d79c1aa"}]': finished 2024-09-10T09:49:18.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:17 smithi077 ceph-mon[30319]: osdmap e47: 8 total, 7 up, 8 in 2024-09-10T09:49:18.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:17 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T09:49:18.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:17 smithi077 ceph-mon[30319]: from='client.? 172.21.15.120:0/1495345426' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:49:19.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:18 smithi134 ceph-mon[35674]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:49:19.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:18 smithi120 ceph-mon[34770]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:49:19.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:18 smithi077 ceph-mon[30319]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:49:21.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:20 smithi134 ceph-mon[35674]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:49:21.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:20 smithi120 ceph-mon[34770]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:49:21.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:20 smithi077 ceph-mon[30319]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:49:22.923 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:22 smithi120 ceph-mon[34770]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:49:23.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:22 smithi134 ceph-mon[35674]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:49:23.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:22 smithi077 ceph-mon[30319]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:49:24.893 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:24 smithi120 ceph-mon[34770]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:49:24.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:24 smithi077 ceph-mon[30319]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:49:25.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:24 smithi134 ceph-mon[35674]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:49:26.117 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:25 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-10T09:49:26.117 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:25 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:49:26.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:25 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-10T09:49:26.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:25 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:49:26.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:25 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-10T09:49:26.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:25 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:49:27.067 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:26 smithi120 ceph-mon[34770]: Deploying daemon osd.7 on smithi120 2024-09-10T09:49:27.068 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:26 smithi120 ceph-mon[34770]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:49:27.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:26 smithi134 ceph-mon[35674]: Deploying daemon osd.7 on smithi120 2024-09-10T09:49:27.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:26 smithi134 ceph-mon[35674]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:49:27.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:26 smithi077 ceph-mon[30319]: Deploying daemon osd.7 on smithi120 2024-09-10T09:49:27.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:26 smithi077 ceph-mon[30319]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:49:28.923 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:28 smithi120 ceph-mon[34770]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:49:29.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:28 smithi134 ceph-mon[35674]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:49:29.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:28 smithi077 ceph-mon[30319]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:49:30.899 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:30 smithi120 ceph-mon[34770]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:49:31.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:30 smithi134 ceph-mon[35674]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:49:31.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:30 smithi077 ceph-mon[30319]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:49:32.111 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:31 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:49:32.112 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:31 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:49:32.112 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:31 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:49:32.130 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:31 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:49:32.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:31 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:49:32.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:31 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:49:32.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:31 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:49:32.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:31 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:49:32.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:31 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:49:33.081 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:32 smithi120 ceph-mon[34770]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:49:33.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:32 smithi134 ceph-mon[35674]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:49:33.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:32 smithi077 ceph-mon[30319]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:49:35.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:34 smithi134 ceph-mon[35674]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:49:35.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:34 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:49:35.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:34 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:49:35.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:34 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:49:35.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:34 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:49:35.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:34 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:49:35.143 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:34 smithi120 ceph-mon[34770]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:49:35.143 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:34 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:49:35.143 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:34 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:49:35.143 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:34 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:49:35.143 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:34 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:49:35.143 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:34 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:49:35.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:49:35.187+0000 7f0643fff640 1 -- 172.21.15.120:0/2555101936 <== mgr.14150 v2:172.21.15.77:6800/4261324650 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f065c062630 con 0x7f06440a2c30 2024-09-10T09:49:35.189 INFO:teuthology.orchestra.run.smithi120.stdout:Created osd(s) 7 on host 'smithi120' 2024-09-10T09:49:35.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:34 smithi077 ceph-mon[30319]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:49:35.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:34 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:49:35.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:34 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:49:35.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:34 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:49:35.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:34 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:49:35.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:34 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:49:35.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:49:35.189+0000 7f0661187640 1 -- 172.21.15.120:0/2555101936 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f06440a2c30 msgr2=0x7f06440a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:35.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:49:35.189+0000 7f0661187640 1 --2- 172.21.15.120:0/2555101936 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f06440a2c30 0x7f06440a50f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f065c199440 tx=0x7f065000a2d0 comp rx=0 tx=0).stop 2024-09-10T09:49:35.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:49:35.189+0000 7f0661187640 1 -- 172.21.15.120:0/2555101936 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f065c101c00 msgr2=0x7f065c197f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:35.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:49:35.189+0000 7f0661187640 1 --2- 172.21.15.120:0/2555101936 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f065c101c00 0x7f065c197f20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f064c031610 tx=0x7f064c0043b0 comp rx=0 tx=0).stop 2024-09-10T09:49:35.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:49:35.189+0000 7f0661187640 1 -- 172.21.15.120:0/2555101936 shutdown_connections 2024-09-10T09:49:35.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:49:35.189+0000 7f0661187640 1 --2- 172.21.15.120:0/2555101936 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f06440a2c30 0x7f06440a50f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:35.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:49:35.189+0000 7f0661187640 1 --2- 172.21.15.120:0/2555101936 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f065c1037c0 0x7f065c19f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:35.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:49:35.189+0000 7f0661187640 1 --2- 172.21.15.120:0/2555101936 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f065c102e00 0x7f065c198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:35.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:49:35.189+0000 7f0661187640 1 --2- 172.21.15.120:0/2555101936 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f065c101c00 0x7f065c197f20 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:35.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:49:35.189+0000 7f0661187640 1 -- 172.21.15.120:0/2555101936 >> 172.21.15.120:0/2555101936 conn(0x7f065c0fd390 msgr2=0x7f065c106210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:49:35.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:49:35.190+0000 7f0661187640 1 -- 172.21.15.120:0/2555101936 shutdown_connections 2024-09-10T09:49:35.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:49:35.190+0000 7f0661187640 1 -- 172.21.15.120:0/2555101936 wait complete. 2024-09-10T09:49:35.818 DEBUG:teuthology.orchestra.run.smithi120:osd.7> sudo journalctl -f -n 0 -u ceph-14a55226-6f59-11ef-bcea-c7b262605968@osd.7.service 2024-09-10T09:49:35.820 INFO:tasks.cephadm:Deploying osd.8 on smithi134 with /dev/vg_nvme/lv_2... 2024-09-10T09:49:35.821 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-10T09:49:36.040 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:35 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:49:36.041 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:35 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:49:36.041 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:35 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:49:36.077 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T09:49:36.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:35 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:49:36.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:35 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:49:36.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:35 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:49:36.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:35 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:49:36.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:35 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:49:36.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:35 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:49:37.050 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:36 smithi120 ceph-mon[34770]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:49:37.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:36 smithi134 ceph-mon[35674]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:49:37.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:36 smithi077 ceph-mon[30319]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:49:38.073 INFO:journalctl@ceph.osd.7.smithi120.stdout:Sep 10 09:49:37 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-7[52351]: 2024-09-10T09:49:37.809+0000 7f3f46136640 -1 osd.7 0 log_to_monitors true 2024-09-10T09:49:38.921 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-10T09:49:38.948 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi134:vg_nvme/lv_2 2024-09-10T09:49:39.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:38 smithi134 ceph-mon[35674]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:49:39.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:38 smithi134 ceph-mon[35674]: from='osd.7 [v2:172.21.15.120:6816/2688667400,v1:172.21.15.120:6817/2688667400]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T09:49:39.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:38 smithi134 ceph-mon[35674]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T09:49:39.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:38 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:49:39.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:38 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:49:39.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:38 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:49:39.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:38 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:49:39.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:38 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:49:39.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:38 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:49:39.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:38 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:49:39.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:38 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:49:39.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:38 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:49:39.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:38 smithi120 ceph-mon[34770]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:49:39.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:38 smithi120 ceph-mon[34770]: from='osd.7 [v2:172.21.15.120:6816/2688667400,v1:172.21.15.120:6817/2688667400]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T09:49:39.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:38 smithi120 ceph-mon[34770]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T09:49:39.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:38 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:49:39.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:38 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:49:39.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:38 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:49:39.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:38 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:49:39.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:38 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:49:39.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:38 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:49:39.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:38 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:49:39.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:38 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:49:39.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:38 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:49:39.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:38 smithi077 ceph-mon[30319]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:49:39.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:38 smithi077 ceph-mon[30319]: from='osd.7 [v2:172.21.15.120:6816/2688667400,v1:172.21.15.120:6817/2688667400]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T09:49:39.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:38 smithi077 ceph-mon[30319]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T09:49:39.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:38 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:49:39.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:38 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:49:39.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:38 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:49:39.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:38 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:49:39.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:38 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:49:39.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:38 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:49:39.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:38 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:49:39.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:38 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:49:39.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:38 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:49:39.192 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T09:49:39.754 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:49:39.740+0000 7f9adbeab640 1 -- 172.21.15.134:0/629998265 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ad4100290 msgr2=0x7f9ad4100690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:39.755 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:49:39.740+0000 7f9adbeab640 1 --2- 172.21.15.134:0/629998265 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ad4100290 0x7f9ad4100690 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f9ac800aa10 tx=0x7f9ac802b720 comp rx=0 tx=0).stop 2024-09-10T09:49:39.755 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:49:39.740+0000 7f9adbeab640 1 -- 172.21.15.134:0/629998265 shutdown_connections 2024-09-10T09:49:39.755 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:49:39.740+0000 7f9adbeab640 1 --2- 172.21.15.134:0/629998265 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9ad4101e50 0x7f9ad41086e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:39.755 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:49:39.740+0000 7f9adbeab640 1 --2- 172.21.15.134:0/629998265 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9ad4101490 0x7f9ad4101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:39.755 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:49:39.740+0000 7f9adbeab640 1 --2- 172.21.15.134:0/629998265 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ad4100290 0x7f9ad4100690 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:39.755 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:49:39.740+0000 7f9adbeab640 1 -- 172.21.15.134:0/629998265 >> 172.21.15.134:0/629998265 conn(0x7f9ad40fba40 msgr2=0x7f9ad40fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:49:39.755 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:49:39.740+0000 7f9adbeab640 1 -- 172.21.15.134:0/629998265 shutdown_connections 2024-09-10T09:49:39.755 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:49:39.740+0000 7f9adbeab640 1 -- 172.21.15.134:0/629998265 wait complete. 2024-09-10T09:49:39.755 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:49:39.741+0000 7f9adbeab640 1 Processor -- start 2024-09-10T09:49:39.755 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:49:39.741+0000 7f9adbeab640 1 -- start start 2024-09-10T09:49:39.755 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:49:39.741+0000 7f9adbeab640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ad4100290 0x7f9ad4195d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:39.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:49:39.741+0000 7f9adbeab640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9ad4101490 0x7f9ad4196290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:39.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:49:39.741+0000 7f9adbeab640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9ad4101e50 0x7f9ad419d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:39.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:49:39.741+0000 7f9adbeab640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad410b6b0 con 0x7f9ad4101e50 2024-09-10T09:49:39.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:49:39.741+0000 7f9adbeab640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad410b530 con 0x7f9ad4101490 2024-09-10T09:49:39.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:49:39.741+0000 7f9adbeab640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad410b830 con 0x7f9ad4100290 2024-09-10T09:49:39.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:49:39.741+0000 7f9ad9c47640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ad4100290 0x7f9ad4195d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:39.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:49:39.741+0000 7f9ad9c47640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ad4100290 0x7f9ad4195d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:35240/0 (socket says 172.21.15.134:35240) 2024-09-10T09:49:39.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:49:39.741+0000 7f9ad9c47640 1 -- 172.21.15.134:0/3117252279 learned_addr learned my addr 172.21.15.134:0/3117252279 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-10T09:49:39.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:49:39.741+0000 7f9ad9446640 1 --2- 172.21.15.134:0/3117252279 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9ad4101490 0x7f9ad4196290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:39.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:49:39.741+0000 7f9ad9c47640 1 -- 172.21.15.134:0/3117252279 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9ad4101490 msgr2=0x7f9ad4196290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:39.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:49:39.741+0000 7f9ad9c47640 1 --2- 172.21.15.134:0/3117252279 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9ad4101490 0x7f9ad4196290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:39.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:49:39.741+0000 7f9ad9c47640 1 -- 172.21.15.134:0/3117252279 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9ad4101e50 msgr2=0x7f9ad419d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:39.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:49:39.741+0000 7f9ad9c47640 1 --2- 172.21.15.134:0/3117252279 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9ad4101e50 0x7f9ad419d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:39.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:49:39.741+0000 7f9ad9c47640 1 -- 172.21.15.134:0/3117252279 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ad419d980 con 0x7f9ad4100290 2024-09-10T09:49:39.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:49:39.741+0000 7f9ad9c47640 1 --2- 172.21.15.134:0/3117252279 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ad4100290 0x7f9ad4195d50 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9ac8005ab0 tx=0x7f9ac80042d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:49:39.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:49:39.742+0000 7f9ac6ffd640 1 -- 172.21.15.134:0/3117252279 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ac8040070 con 0x7f9ad4100290 2024-09-10T09:49:39.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:49:39.742+0000 7f9ac6ffd640 1 -- 172.21.15.134:0/3117252279 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ac80044c0 con 0x7f9ad4100290 2024-09-10T09:49:39.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:49:39.742+0000 7f9adbeab640 1 -- 172.21.15.134:0/3117252279 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ad419dc10 con 0x7f9ad4100290 2024-09-10T09:49:39.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:49:39.742+0000 7f9ac6ffd640 1 -- 172.21.15.134:0/3117252279 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ac8036030 con 0x7f9ad4100290 2024-09-10T09:49:39.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:49:39.742+0000 7f9adbeab640 1 -- 172.21.15.134:0/3117252279 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ad419e140 con 0x7f9ad4100290 2024-09-10T09:49:39.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:49:39.743+0000 7f9adbeab640 1 -- 172.21.15.134:0/3117252279 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ad4107780 con 0x7f9ad4100290 2024-09-10T09:49:39.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:49:39.744+0000 7f9ac6ffd640 1 -- 172.21.15.134:0/3117252279 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f9ac8043070 con 0x7f9ad4100290 2024-09-10T09:49:39.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:49:39.744+0000 7f9ac6ffd640 1 --2- 172.21.15.134:0/3117252279 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f9ac00a2c30 0x7f9ac00a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:39.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:49:39.744+0000 7f9ac6ffd640 1 -- 172.21.15.134:0/3117252279 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5101+0+0 (secure 0 0 0) 0x7f9ac80efdf0 con 0x7f9ad4100290 2024-09-10T09:49:39.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:49:39.745+0000 7f9ad9446640 1 --2- 172.21.15.134:0/3117252279 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f9ac00a2c30 0x7f9ac00a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:39.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:49:39.745+0000 7f9ad9446640 1 --2- 172.21.15.134:0/3117252279 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f9ac00a2c30 0x7f9ac00a50f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9ad4197270 tx=0x7f9ad000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:49:39.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:49:39.749+0000 7f9ac6ffd640 1 -- 172.21.15.134:0/3117252279 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9ac80bd9c0 con 0x7f9ad4100290 2024-09-10T09:49:39.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:49:39.925+0000 7f9adbeab640 1 -- 172.21.15.134:0/3117252279 --> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f9ad4062630 con 0x7f9ac00a2c30 2024-09-10T09:49:39.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:39 smithi134 ceph-mon[35674]: Detected new or changed devices on smithi120 2024-09-10T09:49:39.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:39 smithi134 ceph-mon[35674]: Adjusting osd_memory_target on smithi120 to 5696M 2024-09-10T09:49:39.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:39 smithi134 ceph-mon[35674]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-10T09:49:39.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:39 smithi134 ceph-mon[35674]: from='osd.7 [v2:172.21.15.120:6816/2688667400,v1:172.21.15.120:6817/2688667400]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-09-10T09:49:39.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:39 smithi134 ceph-mon[35674]: osdmap e48: 8 total, 7 up, 8 in 2024-09-10T09:49:39.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:39 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T09:49:39.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:39 smithi134 ceph-mon[35674]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-09-10T09:49:39.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:39 smithi134 ceph-mon[35674]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2024-09-10T09:49:39.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:39 smithi134 ceph-mon[35674]: osdmap e49: 8 total, 7 up, 8 in 2024-09-10T09:49:39.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:39 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T09:49:40.173 INFO:journalctl@ceph.osd.7.smithi120.stdout:Sep 10 09:49:39 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-7[52351]: 2024-09-10T09:49:39.809+0000 7f3f420b9640 -1 osd.7 0 waiting for initial osdmap 2024-09-10T09:49:40.174 INFO:journalctl@ceph.osd.7.smithi120.stdout:Sep 10 09:49:39 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-7[52351]: 2024-09-10T09:49:39.820+0000 7f3f3dee1640 -1 osd.7 49 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T09:49:40.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:39 smithi120 ceph-mon[34770]: Detected new or changed devices on smithi120 2024-09-10T09:49:40.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:39 smithi120 ceph-mon[34770]: Adjusting osd_memory_target on smithi120 to 5696M 2024-09-10T09:49:40.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:39 smithi120 ceph-mon[34770]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-10T09:49:40.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:39 smithi120 ceph-mon[34770]: from='osd.7 [v2:172.21.15.120:6816/2688667400,v1:172.21.15.120:6817/2688667400]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-09-10T09:49:40.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:39 smithi120 ceph-mon[34770]: osdmap e48: 8 total, 7 up, 8 in 2024-09-10T09:49:40.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:39 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T09:49:40.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:39 smithi120 ceph-mon[34770]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-09-10T09:49:40.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:39 smithi120 ceph-mon[34770]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2024-09-10T09:49:40.175 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:39 smithi120 ceph-mon[34770]: osdmap e49: 8 total, 7 up, 8 in 2024-09-10T09:49:40.175 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:39 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T09:49:40.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:39 smithi077 ceph-mon[30319]: Detected new or changed devices on smithi120 2024-09-10T09:49:40.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:39 smithi077 ceph-mon[30319]: Adjusting osd_memory_target on smithi120 to 5696M 2024-09-10T09:49:40.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:39 smithi077 ceph-mon[30319]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-10T09:49:40.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:39 smithi077 ceph-mon[30319]: from='osd.7 [v2:172.21.15.120:6816/2688667400,v1:172.21.15.120:6817/2688667400]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-09-10T09:49:40.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:39 smithi077 ceph-mon[30319]: osdmap e48: 8 total, 7 up, 8 in 2024-09-10T09:49:40.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:39 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T09:49:40.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:39 smithi077 ceph-mon[30319]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-09-10T09:49:40.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:39 smithi077 ceph-mon[30319]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2024-09-10T09:49:40.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:39 smithi077 ceph-mon[30319]: osdmap e49: 8 total, 7 up, 8 in 2024-09-10T09:49:40.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:39 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T09:49:40.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:40 smithi134 ceph-mon[35674]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:49:40.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:40 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T09:49:40.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:40 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:49:40.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:40 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:49:40.882 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:40 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:49:41.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:40 smithi120 ceph-mon[34770]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:49:41.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:40 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T09:49:41.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:40 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:49:41.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:40 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:49:41.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:40 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:49:41.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:40 smithi077 ceph-mon[30319]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:49:41.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:40 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T09:49:41.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:40 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:49:41.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:40 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:49:41.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:40 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:49:42.051 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:41 smithi134 ceph-mon[35674]: purged_snaps scrub starts 2024-09-10T09:49:42.051 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:41 smithi134 ceph-mon[35674]: purged_snaps scrub ok 2024-09-10T09:49:42.051 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:41 smithi134 ceph-mon[35674]: from='client.24319 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:49:42.052 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:41 smithi134 ceph-mon[35674]: osd.7 [v2:172.21.15.120:6816/2688667400,v1:172.21.15.120:6817/2688667400] boot 2024-09-10T09:49:42.052 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:41 smithi134 ceph-mon[35674]: osdmap e50: 8 total, 8 up, 8 in 2024-09-10T09:49:42.052 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:41 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T09:49:42.075 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:41 smithi120 ceph-mon[34770]: purged_snaps scrub starts 2024-09-10T09:49:42.075 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:41 smithi120 ceph-mon[34770]: purged_snaps scrub ok 2024-09-10T09:49:42.075 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:41 smithi120 ceph-mon[34770]: from='client.24319 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:49:42.075 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:41 smithi120 ceph-mon[34770]: osd.7 [v2:172.21.15.120:6816/2688667400,v1:172.21.15.120:6817/2688667400] boot 2024-09-10T09:49:42.075 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:41 smithi120 ceph-mon[34770]: osdmap e50: 8 total, 8 up, 8 in 2024-09-10T09:49:42.075 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:41 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T09:49:42.091 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:41 smithi077 ceph-mon[30319]: purged_snaps scrub starts 2024-09-10T09:49:42.091 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:41 smithi077 ceph-mon[30319]: purged_snaps scrub ok 2024-09-10T09:49:42.091 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:41 smithi077 ceph-mon[30319]: from='client.24319 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:49:42.091 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:41 smithi077 ceph-mon[30319]: osd.7 [v2:172.21.15.120:6816/2688667400,v1:172.21.15.120:6817/2688667400] boot 2024-09-10T09:49:42.091 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:41 smithi077 ceph-mon[30319]: osdmap e50: 8 total, 8 up, 8 in 2024-09-10T09:49:42.091 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:41 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T09:49:42.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:42 smithi134 ceph-mon[35674]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:49:42.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:42 smithi134 ceph-mon[35674]: osdmap e51: 8 total, 8 up, 8 in 2024-09-10T09:49:42.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:42 smithi134 ceph-mon[35674]: from='client.? 172.21.15.134:0/1052329721' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f2e8ca43-57c9-4933-b5f7-73f4d1b3e08b"}]: dispatch 2024-09-10T09:49:42.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:42 smithi134 ceph-mon[35674]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f2e8ca43-57c9-4933-b5f7-73f4d1b3e08b"}]: dispatch 2024-09-10T09:49:42.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:42 smithi134 ceph-mon[35674]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f2e8ca43-57c9-4933-b5f7-73f4d1b3e08b"}]': finished 2024-09-10T09:49:42.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:42 smithi134 ceph-mon[35674]: osdmap e52: 9 total, 8 up, 9 in 2024-09-10T09:49:42.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:42 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T09:49:43.115 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:42 smithi077 ceph-mon[30319]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:49:43.115 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:42 smithi077 ceph-mon[30319]: osdmap e51: 8 total, 8 up, 8 in 2024-09-10T09:49:43.115 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:42 smithi077 ceph-mon[30319]: from='client.? 172.21.15.134:0/1052329721' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f2e8ca43-57c9-4933-b5f7-73f4d1b3e08b"}]: dispatch 2024-09-10T09:49:43.115 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:42 smithi077 ceph-mon[30319]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f2e8ca43-57c9-4933-b5f7-73f4d1b3e08b"}]: dispatch 2024-09-10T09:49:43.115 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:42 smithi077 ceph-mon[30319]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f2e8ca43-57c9-4933-b5f7-73f4d1b3e08b"}]': finished 2024-09-10T09:49:43.115 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:42 smithi077 ceph-mon[30319]: osdmap e52: 9 total, 8 up, 9 in 2024-09-10T09:49:43.116 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:42 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T09:49:43.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:42 smithi120 ceph-mon[34770]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:49:43.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:42 smithi120 ceph-mon[34770]: osdmap e51: 8 total, 8 up, 8 in 2024-09-10T09:49:43.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:42 smithi120 ceph-mon[34770]: from='client.? 172.21.15.134:0/1052329721' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f2e8ca43-57c9-4933-b5f7-73f4d1b3e08b"}]: dispatch 2024-09-10T09:49:43.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:42 smithi120 ceph-mon[34770]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f2e8ca43-57c9-4933-b5f7-73f4d1b3e08b"}]: dispatch 2024-09-10T09:49:43.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:42 smithi120 ceph-mon[34770]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f2e8ca43-57c9-4933-b5f7-73f4d1b3e08b"}]': finished 2024-09-10T09:49:43.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:42 smithi120 ceph-mon[34770]: osdmap e52: 9 total, 8 up, 9 in 2024-09-10T09:49:43.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:42 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T09:49:44.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:43 smithi134 ceph-mon[35674]: from='client.? 172.21.15.134:0/4130667861' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:49:44.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:43 smithi134 ceph-mon[35674]: osdmap e53: 9 total, 8 up, 9 in 2024-09-10T09:49:44.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:43 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T09:49:44.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:43 smithi120 ceph-mon[34770]: from='client.? 172.21.15.134:0/4130667861' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:49:44.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:43 smithi120 ceph-mon[34770]: osdmap e53: 9 total, 8 up, 9 in 2024-09-10T09:49:44.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:43 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T09:49:44.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:43 smithi077 ceph-mon[30319]: from='client.? 172.21.15.134:0/4130667861' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:49:44.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:43 smithi077 ceph-mon[30319]: osdmap e53: 9 total, 8 up, 9 in 2024-09-10T09:49:44.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:43 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T09:49:45.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:44 smithi134 ceph-mon[35674]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:49:45.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:44 smithi134 ceph-mon[35674]: osdmap e54: 9 total, 8 up, 9 in 2024-09-10T09:49:45.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:44 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T09:49:45.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:44 smithi120 ceph-mon[34770]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:49:45.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:44 smithi120 ceph-mon[34770]: osdmap e54: 9 total, 8 up, 9 in 2024-09-10T09:49:45.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:44 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T09:49:45.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:44 smithi077 ceph-mon[30319]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:49:45.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:44 smithi077 ceph-mon[30319]: osdmap e54: 9 total, 8 up, 9 in 2024-09-10T09:49:45.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:44 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T09:49:47.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:46 smithi134 ceph-mon[35674]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:49:47.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:46 smithi120 ceph-mon[34770]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:49:47.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:46 smithi077 ceph-mon[30319]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:49:48.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:48 smithi134 ceph-mon[35674]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:49:49.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:48 smithi120 ceph-mon[34770]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:49:49.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:48 smithi077 ceph-mon[30319]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:49:51.107 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:50 smithi134 ceph-mon[35674]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:49:51.107 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:50 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-10T09:49:51.107 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:50 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:49:51.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:50 smithi120 ceph-mon[34770]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:49:51.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:50 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-10T09:49:51.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:50 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:49:51.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:50 smithi077 ceph-mon[30319]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:49:51.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:50 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-10T09:49:51.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:50 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:49:52.122 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:51 smithi134 ceph-mon[35674]: Deploying daemon osd.8 on smithi134 2024-09-10T09:49:52.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:51 smithi120 ceph-mon[34770]: Deploying daemon osd.8 on smithi134 2024-09-10T09:49:52.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:51 smithi077 ceph-mon[30319]: Deploying daemon osd.8 on smithi134 2024-09-10T09:49:53.122 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:52 smithi134 ceph-mon[35674]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:49:53.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:52 smithi120 ceph-mon[34770]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:49:53.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:52 smithi077 ceph-mon[30319]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:49:54.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:54 smithi077 ceph-mon[30319]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:49:55.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:54 smithi134 ceph-mon[35674]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:49:55.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:54 smithi120 ceph-mon[34770]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:49:56.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:56 smithi134 ceph-mon[35674]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:49:56.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:56 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:49:56.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:56 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:49:56.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:56 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:49:57.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:56 smithi120 ceph-mon[34770]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:49:57.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:56 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:49:57.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:56 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:49:57.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:56 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:49:57.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:56 smithi077 ceph-mon[30319]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:49:57.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:56 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:49:57.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:56 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:49:57.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:56 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:49:59.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:49:58 smithi134 ceph-mon[35674]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:49:59.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:49:58 smithi120 ceph-mon[34770]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:49:59.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:49:58 smithi077 ceph-mon[30319]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:50:00.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:50:00.667+0000 7f9ac6ffd640 1 -- 172.21.15.134:0/3117252279 <== mgr.14150 v2:172.21.15.77:6800/4261324650 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f9ad4062630 con 0x7f9ac00a2c30 2024-09-10T09:50:00.669 INFO:teuthology.orchestra.run.smithi134.stdout:Created osd(s) 8 on host 'smithi134' 2024-09-10T09:50:00.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:50:00.669+0000 7f9adbeab640 1 -- 172.21.15.134:0/3117252279 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f9ac00a2c30 msgr2=0x7f9ac00a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:00.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:50:00.669+0000 7f9adbeab640 1 --2- 172.21.15.134:0/3117252279 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f9ac00a2c30 0x7f9ac00a50f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9ad4197270 tx=0x7f9ad000b390 comp rx=0 tx=0).stop 2024-09-10T09:50:00.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:50:00.669+0000 7f9adbeab640 1 -- 172.21.15.134:0/3117252279 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ad4100290 msgr2=0x7f9ad4195d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:00.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:50:00.669+0000 7f9adbeab640 1 --2- 172.21.15.134:0/3117252279 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ad4100290 0x7f9ad4195d50 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9ac8005ab0 tx=0x7f9ac80042d0 comp rx=0 tx=0).stop 2024-09-10T09:50:00.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:50:00.669+0000 7f9adbeab640 1 -- 172.21.15.134:0/3117252279 shutdown_connections 2024-09-10T09:50:00.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:50:00.669+0000 7f9adbeab640 1 --2- 172.21.15.134:0/3117252279 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f9ac00a2c30 0x7f9ac00a50f0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:00.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:50:00.669+0000 7f9adbeab640 1 --2- 172.21.15.134:0/3117252279 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9ad4101e50 0x7f9ad419d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:00.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:50:00.669+0000 7f9adbeab640 1 --2- 172.21.15.134:0/3117252279 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9ad4101490 0x7f9ad4196290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:00.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:50:00.669+0000 7f9adbeab640 1 --2- 172.21.15.134:0/3117252279 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ad4100290 0x7f9ad4195d50 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:00.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:50:00.669+0000 7f9adbeab640 1 -- 172.21.15.134:0/3117252279 >> 172.21.15.134:0/3117252279 conn(0x7f9ad40fba40 msgr2=0x7f9ad4104950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:50:00.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:50:00.670+0000 7f9adbeab640 1 -- 172.21.15.134:0/3117252279 shutdown_connections 2024-09-10T09:50:00.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:50:00.670+0000 7f9adbeab640 1 -- 172.21.15.134:0/3117252279 wait complete. 2024-09-10T09:50:00.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:00 smithi134 ceph-mon[35674]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:50:00.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:00 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:50:00.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:00 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:50:00.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:00 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:50:00.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:00 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:50:00.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:00 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:50:00.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:00 smithi134 ceph-mon[35674]: overall HEALTH_OK 2024-09-10T09:50:00.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:00 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:50:00.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:00 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:50:00.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:00 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:50:01.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:00 smithi120 ceph-mon[34770]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:50:01.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:00 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:50:01.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:00 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:50:01.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:00 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:50:01.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:00 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:50:01.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:00 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:50:01.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:00 smithi120 ceph-mon[34770]: overall HEALTH_OK 2024-09-10T09:50:01.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:00 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:50:01.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:00 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:50:01.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:00 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:50:01.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:00 smithi077 ceph-mon[30319]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:50:01.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:00 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:50:01.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:00 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:50:01.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:00 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:50:01.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:00 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:50:01.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:00 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:50:01.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:00 smithi077 ceph-mon[30319]: overall HEALTH_OK 2024-09-10T09:50:01.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:00 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:50:01.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:00 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:50:01.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:00 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:50:01.313 DEBUG:teuthology.orchestra.run.smithi134:osd.8> sudo journalctl -f -n 0 -u ceph-14a55226-6f59-11ef-bcea-c7b262605968@osd.8.service 2024-09-10T09:50:01.315 INFO:tasks.cephadm:Deploying osd.9 on smithi077 with /dev/vg_nvme/lv_1... 2024-09-10T09:50:01.316 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-10T09:50:01.689 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:50:02.988 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:02 smithi134 ceph-mon[35674]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:50:03.161 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:02 smithi077 ceph-mon[30319]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:50:03.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:02 smithi120 ceph-mon[34770]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:50:03.381 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 09:50:03 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8[54506]: 2024-09-10T09:50:03.083+0000 7fb0a7d36640 -1 osd.8 0 log_to_monitors true 2024-09-10T09:50:04.109 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:03 smithi077 ceph-mon[30319]: from='osd.8 [v2:172.21.15.134:6816/2081686544,v1:172.21.15.134:6817/2081686544]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-10T09:50:04.109 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:03 smithi077 ceph-mon[30319]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-10T09:50:04.125 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:03 smithi134 ceph-mon[35674]: from='osd.8 [v2:172.21.15.134:6816/2081686544,v1:172.21.15.134:6817/2081686544]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-10T09:50:04.125 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:03 smithi134 ceph-mon[35674]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-10T09:50:04.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:03 smithi120 ceph-mon[34770]: from='osd.8 [v2:172.21.15.134:6816/2081686544,v1:172.21.15.134:6817/2081686544]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-10T09:50:04.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:03 smithi120 ceph-mon[34770]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-10T09:50:04.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:04 smithi077 ceph-mon[30319]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:50:04.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:04 smithi077 ceph-mon[30319]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-10T09:50:04.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:04 smithi077 ceph-mon[30319]: from='osd.8 [v2:172.21.15.134:6816/2081686544,v1:172.21.15.134:6817/2081686544]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-09-10T09:50:04.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:04 smithi077 ceph-mon[30319]: osdmap e55: 9 total, 8 up, 9 in 2024-09-10T09:50:04.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:04 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T09:50:04.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:04 smithi077 ceph-mon[30319]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-09-10T09:50:04.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:04 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:50:04.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:04 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:50:04.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:04 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:50:04.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:04 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:50:04.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:04 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:50:04.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:04 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:50:04.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:04 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:50:04.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:04 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:50:04.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:04 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:50:05.123 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:04 smithi134 ceph-mon[35674]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:50:05.123 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:04 smithi134 ceph-mon[35674]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-10T09:50:05.123 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:04 smithi134 ceph-mon[35674]: from='osd.8 [v2:172.21.15.134:6816/2081686544,v1:172.21.15.134:6817/2081686544]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-09-10T09:50:05.123 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:04 smithi134 ceph-mon[35674]: osdmap e55: 9 total, 8 up, 9 in 2024-09-10T09:50:05.123 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:04 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T09:50:05.123 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:04 smithi134 ceph-mon[35674]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-09-10T09:50:05.123 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:04 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:50:05.123 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:04 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:50:05.123 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:04 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:50:05.123 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:04 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:50:05.123 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:04 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:50:05.124 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:04 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:50:05.124 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:04 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:50:05.124 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:04 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:50:05.124 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:04 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:50:05.161 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:50:05.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:04 smithi120 ceph-mon[34770]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:50:05.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:04 smithi120 ceph-mon[34770]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-10T09:50:05.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:04 smithi120 ceph-mon[34770]: from='osd.8 [v2:172.21.15.134:6816/2081686544,v1:172.21.15.134:6817/2081686544]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-09-10T09:50:05.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:04 smithi120 ceph-mon[34770]: osdmap e55: 9 total, 8 up, 9 in 2024-09-10T09:50:05.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:04 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T09:50:05.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:04 smithi120 ceph-mon[34770]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-09-10T09:50:05.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:04 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:50:05.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:04 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:50:05.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:04 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:50:05.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:04 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:50:05.175 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:04 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:50:05.175 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:04 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:50:05.175 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:04 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:50:05.175 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:04 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:50:05.175 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:04 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:50:05.193 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi077:vg_nvme/lv_1 2024-09-10T09:50:05.381 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 09:50:05 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8[54506]: 2024-09-10T09:50:05.121+0000 7fb0a3cb9640 -1 osd.8 0 waiting for initial osdmap 2024-09-10T09:50:05.381 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 09:50:05 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8[54506]: 2024-09-10T09:50:05.131+0000 7fb09bad9640 -1 osd.8 56 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T09:50:05.567 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:50:06.058 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:05 smithi077 ceph-mon[30319]: Detected new or changed devices on smithi134 2024-09-10T09:50:06.058 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:05 smithi077 ceph-mon[30319]: Adjusting osd_memory_target on smithi134 to 5696M 2024-09-10T09:50:06.058 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:05 smithi077 ceph-mon[30319]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-09-10T09:50:06.058 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:05 smithi077 ceph-mon[30319]: osdmap e56: 9 total, 8 up, 9 in 2024-09-10T09:50:06.058 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:05 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T09:50:06.058 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:05 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T09:50:06.058 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:05 smithi077 ceph-mon[30319]: from='osd.8 ' entity='osd.8' 2024-09-10T09:50:06.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:05 smithi134 ceph-mon[35674]: Detected new or changed devices on smithi134 2024-09-10T09:50:06.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:05 smithi134 ceph-mon[35674]: Adjusting osd_memory_target on smithi134 to 5696M 2024-09-10T09:50:06.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:05 smithi134 ceph-mon[35674]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-09-10T09:50:06.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:05 smithi134 ceph-mon[35674]: osdmap e56: 9 total, 8 up, 9 in 2024-09-10T09:50:06.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:05 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T09:50:06.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:05 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T09:50:06.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:05 smithi134 ceph-mon[35674]: from='osd.8 ' entity='osd.8' 2024-09-10T09:50:06.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:05 smithi120 ceph-mon[34770]: Detected new or changed devices on smithi134 2024-09-10T09:50:06.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:05 smithi120 ceph-mon[34770]: Adjusting osd_memory_target on smithi134 to 5696M 2024-09-10T09:50:06.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:05 smithi120 ceph-mon[34770]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-09-10T09:50:06.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:05 smithi120 ceph-mon[34770]: osdmap e56: 9 total, 8 up, 9 in 2024-09-10T09:50:06.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:05 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T09:50:06.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:05 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T09:50:06.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:05 smithi120 ceph-mon[34770]: from='osd.8 ' entity='osd.8' 2024-09-10T09:50:06.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:50:06.171+0000 7fa91ae00640 1 -- 172.21.15.77:0/3043354681 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa914101c00 msgr2=0x7fa914102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:06.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:50:06.171+0000 7fa91ae00640 1 --2- 172.21.15.77:0/3043354681 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa914101c00 0x7fa914102000 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa904009900 tx=0x7fa90402a400 comp rx=0 tx=0).stop 2024-09-10T09:50:06.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:50:06.171+0000 7fa91ae00640 1 -- 172.21.15.77:0/3043354681 shutdown_connections 2024-09-10T09:50:06.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:50:06.171+0000 7fa91ae00640 1 --2- 172.21.15.77:0/3043354681 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa9141037c0 0x7fa91410a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:06.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:50:06.171+0000 7fa91ae00640 1 --2- 172.21.15.77:0/3043354681 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa914102e00 0x7fa914103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:06.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:50:06.171+0000 7fa91ae00640 1 --2- 172.21.15.77:0/3043354681 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa914101c00 0x7fa914102000 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:06.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:50:06.171+0000 7fa91ae00640 1 -- 172.21.15.77:0/3043354681 >> 172.21.15.77:0/3043354681 conn(0x7fa9140fd390 msgr2=0x7fa9140ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:50:06.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:50:06.171+0000 7fa91ae00640 1 -- 172.21.15.77:0/3043354681 shutdown_connections 2024-09-10T09:50:06.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:50:06.171+0000 7fa91ae00640 1 -- 172.21.15.77:0/3043354681 wait complete. 2024-09-10T09:50:06.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:50:06.172+0000 7fa91ae00640 1 Processor -- start 2024-09-10T09:50:06.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:50:06.173+0000 7fa91ae00640 1 -- start start 2024-09-10T09:50:06.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:50:06.173+0000 7fa91ae00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa914101c00 0x7fa91419a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:06.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:50:06.173+0000 7fa91ae00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa914102e00 0x7fa91419a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:06.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:50:06.173+0000 7fa91ae00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa9141037c0 0x7fa9141a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:06.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:50:06.173+0000 7fa91ae00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa91410d8b0 con 0x7fa914101c00 2024-09-10T09:50:06.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:50:06.173+0000 7fa91ae00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa91410d730 con 0x7fa914102e00 2024-09-10T09:50:06.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:50:06.173+0000 7fa91ae00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa91410da30 con 0x7fa9141037c0 2024-09-10T09:50:06.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:50:06.173+0000 7fa913e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa914101c00 0x7fa91419a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:06.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:50:06.173+0000 7fa913e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa914101c00 0x7fa91419a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50726/0 (socket says 172.21.15.77:50726) 2024-09-10T09:50:06.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:50:06.173+0000 7fa913e00640 1 -- 172.21.15.77:0/2102620617 learned_addr learned my addr 172.21.15.77:0/2102620617 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:50:06.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:50:06.173+0000 7fa90b400640 1 --2- 172.21.15.77:0/2102620617 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa914102e00 0x7fa91419a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:06.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:50:06.173+0000 7fa918e00640 1 --2- 172.21.15.77:0/2102620617 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa9141037c0 0x7fa9141a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:06.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:50:06.174+0000 7fa913e00640 1 -- 172.21.15.77:0/2102620617 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa9141037c0 msgr2=0x7fa9141a16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:06.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:50:06.174+0000 7fa913e00640 1 --2- 172.21.15.77:0/2102620617 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa9141037c0 0x7fa9141a16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:06.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:50:06.174+0000 7fa913e00640 1 -- 172.21.15.77:0/2102620617 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa914102e00 msgr2=0x7fa91419a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:06.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:50:06.174+0000 7fa913e00640 1 --2- 172.21.15.77:0/2102620617 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa914102e00 0x7fa91419a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:06.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:50:06.174+0000 7fa913e00640 1 -- 172.21.15.77:0/2102620617 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9141a1d30 con 0x7fa914101c00 2024-09-10T09:50:06.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:50:06.174+0000 7fa913e00640 1 --2- 172.21.15.77:0/2102620617 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa914101c00 0x7fa91419a100 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa904031440 tx=0x7fa904032e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:50:06.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:50:06.174+0000 7fa911600640 1 -- 172.21.15.77:0/2102620617 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa90403f030 con 0x7fa914101c00 2024-09-10T09:50:06.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:50:06.174+0000 7fa911600640 1 -- 172.21.15.77:0/2102620617 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa904003ce0 con 0x7fa914101c00 2024-09-10T09:50:06.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:50:06.174+0000 7fa91ae00640 1 -- 172.21.15.77:0/2102620617 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9141a1fc0 con 0x7fa914101c00 2024-09-10T09:50:06.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:50:06.174+0000 7fa911600640 1 -- 172.21.15.77:0/2102620617 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa904035070 con 0x7fa914101c00 2024-09-10T09:50:06.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:50:06.174+0000 7fa91ae00640 1 -- 172.21.15.77:0/2102620617 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9141a23c0 con 0x7fa914101c00 2024-09-10T09:50:06.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:50:06.175+0000 7fa91ae00640 1 -- 172.21.15.77:0/2102620617 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9141a25f0 con 0x7fa914101c00 2024-09-10T09:50:06.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:50:06.177+0000 7fa911600640 1 -- 172.21.15.77:0/2102620617 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fa904042030 con 0x7fa914101c00 2024-09-10T09:50:06.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:50:06.178+0000 7fa911600640 1 --2- 172.21.15.77:0/2102620617 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fa8f80a2bb0 0x7fa8f80a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:06.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:50:06.179+0000 7fa911600640 1 -- 172.21.15.77:0/2102620617 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5862+0+0 (secure 0 0 0) 0x7fa9040ee970 con 0x7fa914101c00 2024-09-10T09:50:06.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:50:06.179+0000 7fa90b400640 1 --2- 172.21.15.77:0/2102620617 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fa8f80a2bb0 0x7fa8f80a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:06.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:50:06.179+0000 7fa90b400640 1 --2- 172.21.15.77:0/2102620617 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fa8f80a2bb0 0x7fa8f80a5070 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa91419b620 tx=0x7fa8fc00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:50:06.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:50:06.184+0000 7fa911600640 1 -- 172.21.15.77:0/2102620617 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa9040bbb70 con 0x7fa914101c00 2024-09-10T09:50:06.370 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:50:06.368+0000 7fa91ae00640 1 -- 172.21.15.77:0/2102620617 --> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fa914109ce0 con 0x7fa8f80a2bb0 2024-09-10T09:50:07.001 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:06 smithi077 ceph-mon[30319]: purged_snaps scrub starts 2024-09-10T09:50:07.001 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:06 smithi077 ceph-mon[30319]: purged_snaps scrub ok 2024-09-10T09:50:07.001 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:06 smithi077 ceph-mon[30319]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:50:07.001 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:06 smithi077 ceph-mon[30319]: osd.8 [v2:172.21.15.134:6816/2081686544,v1:172.21.15.134:6817/2081686544] boot 2024-09-10T09:50:07.001 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:06 smithi077 ceph-mon[30319]: osdmap e57: 9 total, 9 up, 9 in 2024-09-10T09:50:07.001 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:06 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T09:50:07.001 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:06 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:50:07.001 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:06 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:50:07.001 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:06 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:50:07.002 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:06 smithi077 ceph-mon[30319]: osdmap e58: 9 total, 9 up, 9 in 2024-09-10T09:50:07.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:06 smithi134 ceph-mon[35674]: purged_snaps scrub starts 2024-09-10T09:50:07.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:06 smithi134 ceph-mon[35674]: purged_snaps scrub ok 2024-09-10T09:50:07.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:06 smithi134 ceph-mon[35674]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:50:07.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:06 smithi134 ceph-mon[35674]: osd.8 [v2:172.21.15.134:6816/2081686544,v1:172.21.15.134:6817/2081686544] boot 2024-09-10T09:50:07.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:06 smithi134 ceph-mon[35674]: osdmap e57: 9 total, 9 up, 9 in 2024-09-10T09:50:07.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:06 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T09:50:07.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:06 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:50:07.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:06 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:50:07.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:06 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:50:07.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:06 smithi134 ceph-mon[35674]: osdmap e58: 9 total, 9 up, 9 in 2024-09-10T09:50:07.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:06 smithi120 ceph-mon[34770]: purged_snaps scrub starts 2024-09-10T09:50:07.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:06 smithi120 ceph-mon[34770]: purged_snaps scrub ok 2024-09-10T09:50:07.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:06 smithi120 ceph-mon[34770]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:50:07.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:06 smithi120 ceph-mon[34770]: osd.8 [v2:172.21.15.134:6816/2081686544,v1:172.21.15.134:6817/2081686544] boot 2024-09-10T09:50:07.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:06 smithi120 ceph-mon[34770]: osdmap e57: 9 total, 9 up, 9 in 2024-09-10T09:50:07.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:06 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T09:50:07.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:06 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:50:07.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:06 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:50:07.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:06 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:50:07.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:06 smithi120 ceph-mon[34770]: osdmap e58: 9 total, 9 up, 9 in 2024-09-10T09:50:08.005 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:07 smithi077 ceph-mon[30319]: from='client.14352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:50:08.005 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:07 smithi077 ceph-mon[30319]: osdmap e59: 9 total, 9 up, 9 in 2024-09-10T09:50:08.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:07 smithi134 ceph-mon[35674]: from='client.14352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:50:08.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:07 smithi134 ceph-mon[35674]: osdmap e59: 9 total, 9 up, 9 in 2024-09-10T09:50:08.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:07 smithi120 ceph-mon[34770]: from='client.14352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:50:08.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:07 smithi120 ceph-mon[34770]: osdmap e59: 9 total, 9 up, 9 in 2024-09-10T09:50:08.920 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:08 smithi077 ceph-mon[30319]: pgmap v202: 1 pgs: 1 remapped+peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-10T09:50:08.920 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:08 smithi077 ceph-mon[30319]: osdmap e60: 9 total, 9 up, 9 in 2024-09-10T09:50:09.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:08 smithi134 ceph-mon[35674]: pgmap v202: 1 pgs: 1 remapped+peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-10T09:50:09.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:08 smithi134 ceph-mon[35674]: osdmap e60: 9 total, 9 up, 9 in 2024-09-10T09:50:09.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:08 smithi120 ceph-mon[34770]: pgmap v202: 1 pgs: 1 remapped+peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-10T09:50:09.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:08 smithi120 ceph-mon[34770]: osdmap e60: 9 total, 9 up, 9 in 2024-09-10T09:50:10.689 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:10 smithi077 ceph-mon[30319]: osdmap e61: 9 total, 9 up, 9 in 2024-09-10T09:50:10.689 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:10 smithi077 ceph-mon[30319]: pgmap v206: 1 pgs: 1 remapped+peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-10T09:50:10.689 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:10 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/70301955' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9486f8b5-0614-4208-87ec-117741f6b622"}]: dispatch 2024-09-10T09:50:10.689 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:10 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/70301955' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9486f8b5-0614-4208-87ec-117741f6b622"}]': finished 2024-09-10T09:50:10.690 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:10 smithi077 ceph-mon[30319]: osdmap e62: 10 total, 9 up, 10 in 2024-09-10T09:50:10.690 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:10 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T09:50:10.690 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:10 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/130723980' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:50:10.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:10 smithi134 ceph-mon[35674]: osdmap e61: 9 total, 9 up, 9 in 2024-09-10T09:50:10.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:10 smithi134 ceph-mon[35674]: pgmap v206: 1 pgs: 1 remapped+peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-10T09:50:10.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:10 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/70301955' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9486f8b5-0614-4208-87ec-117741f6b622"}]: dispatch 2024-09-10T09:50:10.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:10 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/70301955' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9486f8b5-0614-4208-87ec-117741f6b622"}]': finished 2024-09-10T09:50:10.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:10 smithi134 ceph-mon[35674]: osdmap e62: 10 total, 9 up, 10 in 2024-09-10T09:50:10.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:10 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T09:50:10.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:10 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/130723980' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:50:10.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:10 smithi120 ceph-mon[34770]: osdmap e61: 9 total, 9 up, 9 in 2024-09-10T09:50:10.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:10 smithi120 ceph-mon[34770]: pgmap v206: 1 pgs: 1 remapped+peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-10T09:50:10.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:10 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/70301955' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9486f8b5-0614-4208-87ec-117741f6b622"}]: dispatch 2024-09-10T09:50:10.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:10 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/70301955' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9486f8b5-0614-4208-87ec-117741f6b622"}]': finished 2024-09-10T09:50:10.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:10 smithi120 ceph-mon[34770]: osdmap e62: 10 total, 9 up, 10 in 2024-09-10T09:50:10.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:10 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T09:50:10.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:10 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/130723980' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:50:13.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:12 smithi134 ceph-mon[35674]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-10T09:50:13.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:12 smithi120 ceph-mon[34770]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-10T09:50:13.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:12 smithi077 ceph-mon[30319]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-10T09:50:14.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:14 smithi077 ceph-mon[30319]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-09-10T09:50:15.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:14 smithi134 ceph-mon[35674]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-09-10T09:50:15.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:14 smithi120 ceph-mon[34770]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-09-10T09:50:16.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:16 smithi077 ceph-mon[30319]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-09-10T09:50:17.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:16 smithi134 ceph-mon[35674]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-09-10T09:50:17.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:16 smithi120 ceph-mon[34770]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-09-10T09:50:18.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:18 smithi077 ceph-mon[30319]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T09:50:19.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:18 smithi134 ceph-mon[35674]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T09:50:19.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:18 smithi120 ceph-mon[34770]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T09:50:19.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:19 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-10T09:50:19.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:19 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:50:20.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:19 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-10T09:50:20.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:19 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:50:20.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:19 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-10T09:50:20.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:19 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:50:20.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:20 smithi077 ceph-mon[30319]: Deploying daemon osd.9 on smithi077 2024-09-10T09:50:20.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:20 smithi077 ceph-mon[30319]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T09:50:21.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:20 smithi134 ceph-mon[35674]: Deploying daemon osd.9 on smithi077 2024-09-10T09:50:21.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:20 smithi134 ceph-mon[35674]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T09:50:21.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:20 smithi120 ceph-mon[34770]: Deploying daemon osd.9 on smithi077 2024-09-10T09:50:21.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:20 smithi120 ceph-mon[34770]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T09:50:22.797 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:22 smithi077 ceph-mon[30319]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T09:50:23.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:22 smithi134 ceph-mon[35674]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T09:50:23.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:22 smithi120 ceph-mon[34770]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T09:50:25.089 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:24 smithi077 ceph-mon[30319]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-10T09:50:25.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:24 smithi134 ceph-mon[35674]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-10T09:50:25.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:24 smithi120 ceph-mon[34770]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-10T09:50:26.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:26 smithi077 ceph-mon[30319]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-10T09:50:27.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:26 smithi134 ceph-mon[35674]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-10T09:50:27.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:26 smithi120 ceph-mon[34770]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-10T09:50:27.802 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:27 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:50:27.803 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:27 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:50:27.803 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:27 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:50:28.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:27 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:50:28.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:27 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:50:28.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:27 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:50:28.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:27 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:50:28.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:27 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:50:28.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:27 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:50:28.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:28 smithi077 ceph-mon[30319]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-10T09:50:29.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:28 smithi134 ceph-mon[35674]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-10T09:50:29.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:28 smithi120 ceph-mon[34770]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-10T09:50:31.127 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:30 smithi077 ceph-mon[30319]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-10T09:50:31.128 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:30 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:50:31.128 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:30 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:50:31.128 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:30 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:50:31.128 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:30 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:50:31.128 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:30 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:50:31.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:30 smithi134 ceph-mon[35674]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-10T09:50:31.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:30 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:50:31.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:30 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:50:31.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:30 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:50:31.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:30 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:50:31.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:30 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:50:31.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:30 smithi120 ceph-mon[34770]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-10T09:50:31.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:30 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:50:31.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:30 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:50:31.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:30 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:50:31.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:30 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:50:31.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:30 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:50:31.175 INFO:teuthology.orchestra.run.smithi077.stdout:Created osd(s) 9 on host 'smithi077' 2024-09-10T09:50:31.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:50:31.174+0000 7fa911600640 1 -- 172.21.15.77:0/2102620617 <== mgr.14150 v2:172.21.15.77:6800/4261324650 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa914109ce0 con 0x7fa8f80a2bb0 2024-09-10T09:50:31.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:50:31.176+0000 7fa91ae00640 1 -- 172.21.15.77:0/2102620617 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fa8f80a2bb0 msgr2=0x7fa8f80a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:31.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:50:31.176+0000 7fa91ae00640 1 --2- 172.21.15.77:0/2102620617 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fa8f80a2bb0 0x7fa8f80a5070 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa91419b620 tx=0x7fa8fc00b390 comp rx=0 tx=0).stop 2024-09-10T09:50:31.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:50:31.176+0000 7fa91ae00640 1 -- 172.21.15.77:0/2102620617 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa914101c00 msgr2=0x7fa91419a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:31.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:50:31.176+0000 7fa91ae00640 1 --2- 172.21.15.77:0/2102620617 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa914101c00 0x7fa91419a100 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa904031440 tx=0x7fa904032e20 comp rx=0 tx=0).stop 2024-09-10T09:50:31.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:50:31.176+0000 7fa91ae00640 1 -- 172.21.15.77:0/2102620617 shutdown_connections 2024-09-10T09:50:31.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:50:31.176+0000 7fa91ae00640 1 --2- 172.21.15.77:0/2102620617 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fa8f80a2bb0 0x7fa8f80a5070 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:31.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:50:31.176+0000 7fa91ae00640 1 --2- 172.21.15.77:0/2102620617 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa9141037c0 0x7fa9141a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:31.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:50:31.176+0000 7fa91ae00640 1 --2- 172.21.15.77:0/2102620617 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa914102e00 0x7fa91419a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:31.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:50:31.176+0000 7fa91ae00640 1 --2- 172.21.15.77:0/2102620617 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa914101c00 0x7fa91419a100 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:31.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:50:31.176+0000 7fa91ae00640 1 -- 172.21.15.77:0/2102620617 >> 172.21.15.77:0/2102620617 conn(0x7fa9140fd390 msgr2=0x7fa9141085c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:50:31.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:50:31.177+0000 7fa91ae00640 1 -- 172.21.15.77:0/2102620617 shutdown_connections 2024-09-10T09:50:31.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:50:31.177+0000 7fa91ae00640 1 -- 172.21.15.77:0/2102620617 wait complete. 2024-09-10T09:50:32.007 DEBUG:teuthology.orchestra.run.smithi077:osd.9> sudo journalctl -f -n 0 -u ceph-14a55226-6f59-11ef-bcea-c7b262605968@osd.9.service 2024-09-10T09:50:32.011 INFO:tasks.cephadm:Deploying osd.10 on smithi120 with /dev/vg_nvme/lv_1... 2024-09-10T09:50:32.011 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-10T09:50:32.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:31 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:50:32.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:31 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:50:32.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:31 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:50:32.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:31 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:50:32.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:31 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:50:32.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:31 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:50:32.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:31 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:50:32.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:31 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:50:32.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:31 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:50:32.271 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T09:50:32.923 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:32 smithi120 ceph-mon[34770]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-10T09:50:32.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:32 smithi120 ceph-mon[34770]: from='osd.9 [v2:172.21.15.77:6826/1210619387,v1:172.21.15.77:6827/1210619387]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-10T09:50:33.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:32 smithi134 ceph-mon[35674]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-10T09:50:33.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:32 smithi134 ceph-mon[35674]: from='osd.9 [v2:172.21.15.77:6826/1210619387,v1:172.21.15.77:6827/1210619387]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-10T09:50:33.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:32 smithi077 ceph-mon[30319]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-10T09:50:33.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:32 smithi077 ceph-mon[30319]: from='osd.9 [v2:172.21.15.77:6826/1210619387,v1:172.21.15.77:6827/1210619387]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-10T09:50:33.189 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 09:50:32 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T09:50:32.807+0000 7fcc3c897640 -1 osd.9 0 log_to_monitors true 2024-09-10T09:50:34.440 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:34 smithi077 ceph-mon[30319]: from='osd.9 [v2:172.21.15.77:6826/1210619387,v1:172.21.15.77:6827/1210619387]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-10T09:50:34.440 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:34 smithi077 ceph-mon[30319]: osdmap e63: 10 total, 9 up, 10 in 2024-09-10T09:50:34.440 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:34 smithi077 ceph-mon[30319]: from='osd.9 [v2:172.21.15.77:6826/1210619387,v1:172.21.15.77:6827/1210619387]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-09-10T09:50:34.440 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:34 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T09:50:34.440 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:34 smithi077 ceph-mon[30319]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-10T09:50:34.609 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:34 smithi120 ceph-mon[34770]: from='osd.9 [v2:172.21.15.77:6826/1210619387,v1:172.21.15.77:6827/1210619387]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-10T09:50:34.609 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:34 smithi120 ceph-mon[34770]: osdmap e63: 10 total, 9 up, 10 in 2024-09-10T09:50:34.609 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:34 smithi120 ceph-mon[34770]: from='osd.9 [v2:172.21.15.77:6826/1210619387,v1:172.21.15.77:6827/1210619387]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-09-10T09:50:34.609 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:34 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T09:50:34.609 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:34 smithi120 ceph-mon[34770]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-10T09:50:34.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:34 smithi134 ceph-mon[35674]: from='osd.9 [v2:172.21.15.77:6826/1210619387,v1:172.21.15.77:6827/1210619387]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-10T09:50:34.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:34 smithi134 ceph-mon[35674]: osdmap e63: 10 total, 9 up, 10 in 2024-09-10T09:50:34.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:34 smithi134 ceph-mon[35674]: from='osd.9 [v2:172.21.15.77:6826/1210619387,v1:172.21.15.77:6827/1210619387]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-09-10T09:50:34.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:34 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T09:50:34.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:34 smithi134 ceph-mon[35674]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-10T09:50:34.980 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 09:50:34 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T09:50:34.709+0000 7fcc38200640 -1 osd.9 0 waiting for initial osdmap 2024-09-10T09:50:34.980 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 09:50:34 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T09:50:34.747+0000 7fcc2d800640 -1 osd.9 64 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T09:50:35.027 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T09:50:35.054 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi120:vg_nvme/lv_1 2024-09-10T09:50:35.305 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T09:50:35.419 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:35 smithi077 ceph-mon[30319]: from='osd.9 [v2:172.21.15.77:6826/1210619387,v1:172.21.15.77:6827/1210619387]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-09-10T09:50:35.419 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:35 smithi077 ceph-mon[30319]: osdmap e64: 10 total, 9 up, 10 in 2024-09-10T09:50:35.419 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:35 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T09:50:35.419 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:35 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T09:50:35.419 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:35 smithi077 ceph-mon[30319]: from='osd.9 [v2:172.21.15.77:6826/1210619387,v1:172.21.15.77:6827/1210619387]' entity='osd.9' 2024-09-10T09:50:35.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:35 smithi120 ceph-mon[34770]: from='osd.9 [v2:172.21.15.77:6826/1210619387,v1:172.21.15.77:6827/1210619387]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-09-10T09:50:35.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:35 smithi120 ceph-mon[34770]: osdmap e64: 10 total, 9 up, 10 in 2024-09-10T09:50:35.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:35 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T09:50:35.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:35 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T09:50:35.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:35 smithi120 ceph-mon[34770]: from='osd.9 [v2:172.21.15.77:6826/1210619387,v1:172.21.15.77:6827/1210619387]' entity='osd.9' 2024-09-10T09:50:35.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:35 smithi134 ceph-mon[35674]: from='osd.9 [v2:172.21.15.77:6826/1210619387,v1:172.21.15.77:6827/1210619387]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-09-10T09:50:35.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:35 smithi134 ceph-mon[35674]: osdmap e64: 10 total, 9 up, 10 in 2024-09-10T09:50:35.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:35 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T09:50:35.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:35 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T09:50:35.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:35 smithi134 ceph-mon[35674]: from='osd.9 [v2:172.21.15.77:6826/1210619387,v1:172.21.15.77:6827/1210619387]' entity='osd.9' 2024-09-10T09:50:35.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:50:35.826+0000 7efcdffaf640 1 -- 172.21.15.120:0/2393879151 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efcd8101600 msgr2=0x7efcd8105ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:35.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:50:35.826+0000 7efcdffaf640 1 --2- 172.21.15.120:0/2393879151 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efcd8101600 0x7efcd8105ea0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7efcd0009ca0 tx=0x7efcd002a8c0 comp rx=0 tx=0).stop 2024-09-10T09:50:35.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:50:35.826+0000 7efcdffaf640 1 -- 172.21.15.120:0/2393879151 shutdown_connections 2024-09-10T09:50:35.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:50:35.826+0000 7efcdffaf640 1 --2- 172.21.15.120:0/2393879151 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efcd8101600 0x7efcd8105ea0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:35.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:50:35.826+0000 7efcdffaf640 1 --2- 172.21.15.120:0/2393879151 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efcd8100c40 0x7efcd81010c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:35.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:50:35.826+0000 7efcdffaf640 1 --2- 172.21.15.120:0/2393879151 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efcd80ffed0 0x7efcd81002d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:35.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:50:35.826+0000 7efcdffaf640 1 -- 172.21.15.120:0/2393879151 >> 172.21.15.120:0/2393879151 conn(0x7efcd80fba40 msgr2=0x7efcd80fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:50:35.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:50:35.826+0000 7efcdffaf640 1 -- 172.21.15.120:0/2393879151 shutdown_connections 2024-09-10T09:50:35.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:50:35.826+0000 7efcdffaf640 1 -- 172.21.15.120:0/2393879151 wait complete. 2024-09-10T09:50:35.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:50:35.827+0000 7efcdffaf640 1 Processor -- start 2024-09-10T09:50:35.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:50:35.827+0000 7efcdffaf640 1 -- start start 2024-09-10T09:50:35.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:50:35.827+0000 7efcdffaf640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efcd80ffed0 0x7efcd819e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:35.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:50:35.827+0000 7efcdffaf640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efcd8100c40 0x7efcd819e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:35.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:50:35.827+0000 7efcdffaf640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efcd8101600 0x7efcd81a5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:35.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:50:35.827+0000 7efcdffaf640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7efcd8113f10 con 0x7efcd8101600 2024-09-10T09:50:35.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:50:35.827+0000 7efcdffaf640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7efcd8113d90 con 0x7efcd8100c40 2024-09-10T09:50:35.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:50:35.827+0000 7efcdffaf640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7efcd8114090 con 0x7efcd80ffed0 2024-09-10T09:50:35.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:50:35.827+0000 7efcdd54a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efcd8100c40 0x7efcd819e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:35.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:50:35.827+0000 7efcdd54a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efcd8100c40 0x7efcd819e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47718/0 (socket says 172.21.15.120:47718) 2024-09-10T09:50:35.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:50:35.827+0000 7efcdd54a640 1 -- 172.21.15.120:0/1976783691 learned_addr learned my addr 172.21.15.120:0/1976783691 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T09:50:35.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:50:35.827+0000 7efcde54c640 1 --2- 172.21.15.120:0/1976783691 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efcd8101600 0x7efcd81a5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:35.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:50:35.827+0000 7efcddd4b640 1 --2- 172.21.15.120:0/1976783691 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efcd80ffed0 0x7efcd819e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:35.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:50:35.828+0000 7efcdd54a640 1 -- 172.21.15.120:0/1976783691 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efcd80ffed0 msgr2=0x7efcd819e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:35.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:50:35.828+0000 7efcdd54a640 1 --2- 172.21.15.120:0/1976783691 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efcd80ffed0 0x7efcd819e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:35.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:50:35.828+0000 7efcdd54a640 1 -- 172.21.15.120:0/1976783691 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efcd8101600 msgr2=0x7efcd81a5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:35.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:50:35.828+0000 7efcdd54a640 1 --2- 172.21.15.120:0/1976783691 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efcd8101600 0x7efcd81a5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:35.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:50:35.828+0000 7efcdd54a640 1 -- 172.21.15.120:0/1976783691 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcd81a60a0 con 0x7efcd8100c40 2024-09-10T09:50:35.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:50:35.828+0000 7efcddd4b640 1 --2- 172.21.15.120:0/1976783691 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efcd80ffed0 0x7efcd819e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:50:35.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:50:35.828+0000 7efcde54c640 1 --2- 172.21.15.120:0/1976783691 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efcd8101600 0x7efcd81a5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:50:35.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:50:35.828+0000 7efcdd54a640 1 --2- 172.21.15.120:0/1976783691 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efcd8100c40 0x7efcd819e9b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7efcc800d770 tx=0x7efcc800dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:50:35.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:50:35.828+0000 7efcceffd640 1 -- 172.21.15.120:0/1976783691 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc8017070 con 0x7efcd8100c40 2024-09-10T09:50:35.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:50:35.828+0000 7efcdffaf640 1 -- 172.21.15.120:0/1976783691 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcd81a6390 con 0x7efcd8100c40 2024-09-10T09:50:35.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:50:35.828+0000 7efcceffd640 1 -- 172.21.15.120:0/1976783691 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efcc8009c30 con 0x7efcd8100c40 2024-09-10T09:50:35.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:50:35.828+0000 7efcceffd640 1 -- 172.21.15.120:0/1976783691 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc8003dd0 con 0x7efcd8100c40 2024-09-10T09:50:35.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:50:35.828+0000 7efcdffaf640 1 -- 172.21.15.120:0/1976783691 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcd81a68d0 con 0x7efcd8100c40 2024-09-10T09:50:35.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:50:35.829+0000 7efcdffaf640 1 -- 172.21.15.120:0/1976783691 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcd81a6b30 con 0x7efcd8100c40 2024-09-10T09:50:35.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:50:35.830+0000 7efcceffd640 1 -- 172.21.15.120:0/1976783691 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7efcc8003f70 con 0x7efcd8100c40 2024-09-10T09:50:35.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:50:35.830+0000 7efcceffd640 1 --2- 172.21.15.120:0/1976783691 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7efcc40a2bb0 0x7efcc40a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:35.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:50:35.831+0000 7efcceffd640 1 -- 172.21.15.120:0/1976783691 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6261+0+0 (secure 0 0 0) 0x7efcc80ce250 con 0x7efcd8100c40 2024-09-10T09:50:35.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:50:35.831+0000 7efcddd4b640 1 --2- 172.21.15.120:0/1976783691 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7efcc40a2bb0 0x7efcc40a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:35.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:50:35.831+0000 7efcddd4b640 1 --2- 172.21.15.120:0/1976783691 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7efcc40a2bb0 0x7efcc40a5070 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7efcd400baa0 tx=0x7efcd4005d90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:50:35.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:50:35.835+0000 7efcceffd640 1 -- 172.21.15.120:0/1976783691 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efcc809b9b0 con 0x7efcd8100c40 2024-09-10T09:50:36.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:50:36.009+0000 7efcdffaf640 1 -- 172.21.15.120:0/1976783691 --> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7efcd80fdd60 con 0x7efcc40a2bb0 2024-09-10T09:50:36.412 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:36 smithi120 ceph-mon[34770]: purged_snaps scrub starts 2024-09-10T09:50:36.412 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:36 smithi120 ceph-mon[34770]: purged_snaps scrub ok 2024-09-10T09:50:36.412 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:36 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T09:50:36.412 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:36 smithi120 ceph-mon[34770]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-10T09:50:36.413 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:36 smithi120 ceph-mon[34770]: osd.9 [v2:172.21.15.77:6826/1210619387,v1:172.21.15.77:6827/1210619387] boot 2024-09-10T09:50:36.413 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:36 smithi120 ceph-mon[34770]: osdmap e65: 10 total, 10 up, 10 in 2024-09-10T09:50:36.413 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:36 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T09:50:36.413 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:36 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:50:36.413 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:36 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:50:36.413 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:36 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:50:36.413 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:36 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:50:36.413 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:36 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:50:36.413 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:36 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:50:36.413 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:36 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:50:36.413 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:36 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:50:36.413 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:36 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:50:36.414 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:36 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:50:36.414 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:36 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:50:36.414 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:36 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:50:36.414 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:36 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:50:36.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:36 smithi077 ceph-mon[30319]: purged_snaps scrub starts 2024-09-10T09:50:36.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:36 smithi077 ceph-mon[30319]: purged_snaps scrub ok 2024-09-10T09:50:36.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:36 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T09:50:36.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:36 smithi077 ceph-mon[30319]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-10T09:50:36.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:36 smithi077 ceph-mon[30319]: osd.9 [v2:172.21.15.77:6826/1210619387,v1:172.21.15.77:6827/1210619387] boot 2024-09-10T09:50:36.440 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:36 smithi077 ceph-mon[30319]: osdmap e65: 10 total, 10 up, 10 in 2024-09-10T09:50:36.440 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:36 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T09:50:36.440 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:36 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:50:36.440 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:36 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:50:36.440 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:36 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:50:36.440 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:36 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:50:36.440 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:36 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:50:36.440 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:36 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:50:36.440 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:36 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:50:36.440 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:36 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:50:36.440 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:36 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:50:36.440 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:36 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:50:36.440 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:36 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:50:36.441 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:36 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:50:36.441 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:36 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:50:36.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:36 smithi134 ceph-mon[35674]: purged_snaps scrub starts 2024-09-10T09:50:36.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:36 smithi134 ceph-mon[35674]: purged_snaps scrub ok 2024-09-10T09:50:36.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:36 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T09:50:36.632 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:36 smithi134 ceph-mon[35674]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-10T09:50:36.632 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:36 smithi134 ceph-mon[35674]: osd.9 [v2:172.21.15.77:6826/1210619387,v1:172.21.15.77:6827/1210619387] boot 2024-09-10T09:50:36.632 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:36 smithi134 ceph-mon[35674]: osdmap e65: 10 total, 10 up, 10 in 2024-09-10T09:50:36.632 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:36 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T09:50:36.632 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:36 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:50:36.632 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:36 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:50:36.632 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:36 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:50:36.632 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:36 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:50:36.632 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:36 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:50:36.632 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:36 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:50:36.632 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:36 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:50:36.633 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:36 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:50:36.633 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:36 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:50:36.633 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:36 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:50:36.633 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:36 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:50:36.633 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:36 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:50:36.633 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:36 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:50:37.974 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:37 smithi120 ceph-mon[34770]: from='client.24377 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:50:37.974 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:37 smithi120 ceph-mon[34770]: Detected new or changed devices on smithi077 2024-09-10T09:50:37.974 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:37 smithi120 ceph-mon[34770]: Adjusting osd_memory_target on smithi077 to 4272M 2024-09-10T09:50:37.974 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:37 smithi120 ceph-mon[34770]: osdmap e66: 10 total, 10 up, 10 in 2024-09-10T09:50:38.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:37 smithi134 ceph-mon[35674]: from='client.24377 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:50:38.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:37 smithi134 ceph-mon[35674]: Detected new or changed devices on smithi077 2024-09-10T09:50:38.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:37 smithi134 ceph-mon[35674]: Adjusting osd_memory_target on smithi077 to 4272M 2024-09-10T09:50:38.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:37 smithi134 ceph-mon[35674]: osdmap e66: 10 total, 10 up, 10 in 2024-09-10T09:50:38.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:37 smithi077 ceph-mon[30319]: from='client.24377 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:50:38.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:37 smithi077 ceph-mon[30319]: Detected new or changed devices on smithi077 2024-09-10T09:50:38.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:37 smithi077 ceph-mon[30319]: Adjusting osd_memory_target on smithi077 to 4272M 2024-09-10T09:50:38.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:37 smithi077 ceph-mon[30319]: osdmap e66: 10 total, 10 up, 10 in 2024-09-10T09:50:39.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:38 smithi120 ceph-mon[34770]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-10T09:50:39.004 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:38 smithi120 ceph-mon[34770]: from='client.? 172.21.15.120:0/2944889243' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8e8192b0-1398-4d4d-9756-762b2f874d63"}]: dispatch 2024-09-10T09:50:39.004 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:38 smithi120 ceph-mon[34770]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8e8192b0-1398-4d4d-9756-762b2f874d63"}]: dispatch 2024-09-10T09:50:39.004 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:38 smithi120 ceph-mon[34770]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8e8192b0-1398-4d4d-9756-762b2f874d63"}]': finished 2024-09-10T09:50:39.004 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:38 smithi120 ceph-mon[34770]: osdmap e67: 11 total, 10 up, 11 in 2024-09-10T09:50:39.004 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:38 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T09:50:39.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:38 smithi134 ceph-mon[35674]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-10T09:50:39.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:38 smithi134 ceph-mon[35674]: from='client.? 172.21.15.120:0/2944889243' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8e8192b0-1398-4d4d-9756-762b2f874d63"}]: dispatch 2024-09-10T09:50:39.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:38 smithi134 ceph-mon[35674]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8e8192b0-1398-4d4d-9756-762b2f874d63"}]: dispatch 2024-09-10T09:50:39.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:38 smithi134 ceph-mon[35674]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8e8192b0-1398-4d4d-9756-762b2f874d63"}]': finished 2024-09-10T09:50:39.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:38 smithi134 ceph-mon[35674]: osdmap e67: 11 total, 10 up, 11 in 2024-09-10T09:50:39.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:38 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T09:50:39.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:38 smithi077 ceph-mon[30319]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-10T09:50:39.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:38 smithi077 ceph-mon[30319]: from='client.? 172.21.15.120:0/2944889243' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8e8192b0-1398-4d4d-9756-762b2f874d63"}]: dispatch 2024-09-10T09:50:39.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:38 smithi077 ceph-mon[30319]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8e8192b0-1398-4d4d-9756-762b2f874d63"}]: dispatch 2024-09-10T09:50:39.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:38 smithi077 ceph-mon[30319]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8e8192b0-1398-4d4d-9756-762b2f874d63"}]': finished 2024-09-10T09:50:39.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:38 smithi077 ceph-mon[30319]: osdmap e67: 11 total, 10 up, 11 in 2024-09-10T09:50:39.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:38 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T09:50:40.032 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:39 smithi134 ceph-mon[35674]: from='client.? 172.21.15.120:0/1710101101' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:50:40.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:39 smithi120 ceph-mon[34770]: from='client.? 172.21.15.120:0/1710101101' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:50:40.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:39 smithi077 ceph-mon[30319]: from='client.? 172.21.15.120:0/1710101101' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:50:41.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:40 smithi134 ceph-mon[35674]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-10T09:50:41.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:40 smithi120 ceph-mon[34770]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-10T09:50:41.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:40 smithi077 ceph-mon[30319]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-10T09:50:42.987 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:42 smithi120 ceph-mon[34770]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-10T09:50:43.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:42 smithi134 ceph-mon[35674]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-10T09:50:43.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:42 smithi077 ceph-mon[30319]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-10T09:50:44.992 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:44 smithi077 ceph-mon[30319]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-10T09:50:45.068 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:44 smithi120 ceph-mon[34770]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-10T09:50:45.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:44 smithi134 ceph-mon[35674]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-10T09:50:46.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:46 smithi077 ceph-mon[30319]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-10T09:50:47.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:46 smithi134 ceph-mon[35674]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-10T09:50:47.157 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:46 smithi120 ceph-mon[34770]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-10T09:50:47.894 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:47 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-10T09:50:47.895 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:47 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:50:48.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:47 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-10T09:50:48.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:47 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:50:48.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:47 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-10T09:50:48.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:47 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:50:48.891 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:48 smithi120 ceph-mon[34770]: Deploying daemon osd.10 on smithi120 2024-09-10T09:50:48.891 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:48 smithi120 ceph-mon[34770]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-10T09:50:49.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:48 smithi134 ceph-mon[35674]: Deploying daemon osd.10 on smithi120 2024-09-10T09:50:49.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:48 smithi134 ceph-mon[35674]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-10T09:50:49.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:48 smithi077 ceph-mon[30319]: Deploying daemon osd.10 on smithi120 2024-09-10T09:50:49.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:48 smithi077 ceph-mon[30319]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-10T09:50:50.897 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:50 smithi120 ceph-mon[34770]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-10T09:50:51.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:50 smithi134 ceph-mon[35674]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-10T09:50:51.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:50 smithi077 ceph-mon[30319]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-10T09:50:53.021 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:52 smithi120 ceph-mon[34770]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-10T09:50:53.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:52 smithi134 ceph-mon[35674]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-10T09:50:53.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:52 smithi077 ceph-mon[30319]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-10T09:50:53.811 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:53 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:50:53.811 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:53 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:50:53.811 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:53 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:50:54.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:53 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:50:54.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:53 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:50:54.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:53 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:50:54.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:53 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:50:54.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:53 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:50:54.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:53 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:50:54.903 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:54 smithi120 ceph-mon[34770]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-10T09:50:55.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:54 smithi134 ceph-mon[35674]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-10T09:50:55.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:54 smithi077 ceph-mon[30319]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-10T09:50:56.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:50:56.805+0000 7efcceffd640 1 -- 172.21.15.120:0/1976783691 <== mgr.14150 v2:172.21.15.77:6800/4261324650 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7efcd80fdd60 con 0x7efcc40a2bb0 2024-09-10T09:50:56.806 INFO:teuthology.orchestra.run.smithi120.stdout:Created osd(s) 10 on host 'smithi120' 2024-09-10T09:50:56.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:50:56.807+0000 7efcdffaf640 1 -- 172.21.15.120:0/1976783691 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7efcc40a2bb0 msgr2=0x7efcc40a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:56.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:50:56.807+0000 7efcdffaf640 1 --2- 172.21.15.120:0/1976783691 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7efcc40a2bb0 0x7efcc40a5070 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7efcd400baa0 tx=0x7efcd4005d90 comp rx=0 tx=0).stop 2024-09-10T09:50:56.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:50:56.807+0000 7efcdffaf640 1 -- 172.21.15.120:0/1976783691 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efcd8100c40 msgr2=0x7efcd819e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:56.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:50:56.807+0000 7efcdffaf640 1 --2- 172.21.15.120:0/1976783691 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efcd8100c40 0x7efcd819e9b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7efcc800d770 tx=0x7efcc800dc30 comp rx=0 tx=0).stop 2024-09-10T09:50:56.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:50:56.807+0000 7efcdffaf640 1 -- 172.21.15.120:0/1976783691 shutdown_connections 2024-09-10T09:50:56.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:50:56.807+0000 7efcdffaf640 1 --2- 172.21.15.120:0/1976783691 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7efcc40a2bb0 0x7efcc40a5070 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:56.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:50:56.807+0000 7efcdffaf640 1 --2- 172.21.15.120:0/1976783691 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efcd8101600 0x7efcd81a5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:56.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:50:56.807+0000 7efcdffaf640 1 --2- 172.21.15.120:0/1976783691 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efcd8100c40 0x7efcd819e9b0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:56.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:50:56.807+0000 7efcdffaf640 1 --2- 172.21.15.120:0/1976783691 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efcd80ffed0 0x7efcd819e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:56.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:50:56.807+0000 7efcdffaf640 1 -- 172.21.15.120:0/1976783691 >> 172.21.15.120:0/1976783691 conn(0x7efcd80fba40 msgr2=0x7efcd80fc640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:50:56.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:50:56.807+0000 7efcdffaf640 1 -- 172.21.15.120:0/1976783691 shutdown_connections 2024-09-10T09:50:56.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T09:50:56.807+0000 7efcdffaf640 1 -- 172.21.15.120:0/1976783691 wait complete. 2024-09-10T09:50:57.110 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:56 smithi120 ceph-mon[34770]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-10T09:50:57.110 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:56 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:50:57.110 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:56 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:50:57.110 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:56 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:50:57.110 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:56 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:50:57.110 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:56 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:50:57.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:56 smithi134 ceph-mon[35674]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-10T09:50:57.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:56 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:50:57.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:56 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:50:57.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:56 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:50:57.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:56 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:50:57.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:56 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:50:57.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:56 smithi077 ceph-mon[30319]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-10T09:50:57.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:56 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:50:57.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:56 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:50:57.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:56 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:50:57.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:56 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:50:57.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:56 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:50:57.456 DEBUG:teuthology.orchestra.run.smithi120:osd.10> sudo journalctl -f -n 0 -u ceph-14a55226-6f59-11ef-bcea-c7b262605968@osd.10.service 2024-09-10T09:50:57.458 INFO:tasks.cephadm:Deploying osd.11 on smithi134 with /dev/vg_nvme/lv_1... 2024-09-10T09:50:57.458 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-10T09:50:57.713 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T09:50:58.092 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:57 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:50:58.092 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:57 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:50:58.092 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:57 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:50:58.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:57 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:50:58.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:57 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:50:58.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:57 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:50:58.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:57 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:50:58.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:57 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:50:58.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:57 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:50:59.038 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:58 smithi120 ceph-mon[34770]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-10T09:50:59.039 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 09:50:58 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T09:50:58.894+0000 7f7595c56640 -1 osd.10 0 log_to_monitors true 2024-09-10T09:50:59.121 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:58 smithi134 ceph-mon[35674]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-10T09:50:59.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:58 smithi077 ceph-mon[30319]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-10T09:51:00.071 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:59 smithi134 ceph-mon[35674]: from='osd.10 [v2:172.21.15.120:6824/754733885,v1:172.21.15.120:6825/754733885]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-10T09:51:00.071 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:50:59 smithi134 ceph-mon[35674]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-10T09:51:00.170 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:59 smithi120 ceph-mon[34770]: from='osd.10 [v2:172.21.15.120:6824/754733885,v1:172.21.15.120:6825/754733885]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-10T09:51:00.170 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:50:59 smithi120 ceph-mon[34770]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-10T09:51:00.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:59 smithi077 ceph-mon[30319]: from='osd.10 [v2:172.21.15.120:6824/754733885,v1:172.21.15.120:6825/754733885]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-10T09:51:00.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:50:59 smithi077 ceph-mon[30319]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-10T09:51:00.434 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-10T09:51:00.463 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi134:vg_nvme/lv_1 2024-09-10T09:51:00.722 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T09:51:00.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:00 smithi120 ceph-mon[34770]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-10T09:51:00.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:00 smithi120 ceph-mon[34770]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-10T09:51:00.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:00 smithi120 ceph-mon[34770]: from='osd.10 [v2:172.21.15.120:6824/754733885,v1:172.21.15.120:6825/754733885]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-09-10T09:51:00.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:00 smithi120 ceph-mon[34770]: osdmap e68: 11 total, 10 up, 11 in 2024-09-10T09:51:00.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:00 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T09:51:00.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:00 smithi120 ceph-mon[34770]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-09-10T09:51:00.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:00 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:00.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:00 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:00.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:00 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:51:00.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:00 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:51:00.925 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:00 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:51:00.925 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:00 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:51:00.925 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:00 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:00.925 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:00 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:51:00.925 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:00 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:51:00.925 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:00 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:01.110 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:00 smithi134 ceph-mon[35674]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-10T09:51:01.110 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:00 smithi134 ceph-mon[35674]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-10T09:51:01.110 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:00 smithi134 ceph-mon[35674]: from='osd.10 [v2:172.21.15.120:6824/754733885,v1:172.21.15.120:6825/754733885]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-09-10T09:51:01.110 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:00 smithi134 ceph-mon[35674]: osdmap e68: 11 total, 10 up, 11 in 2024-09-10T09:51:01.111 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:00 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T09:51:01.111 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:00 smithi134 ceph-mon[35674]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-09-10T09:51:01.111 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:00 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:01.111 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:00 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:01.111 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:00 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:51:01.111 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:00 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:51:01.112 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:00 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:51:01.112 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:00 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:51:01.112 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:00 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:01.112 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:00 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:51:01.112 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:00 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:51:01.113 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:00 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:01.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:00 smithi077 ceph-mon[30319]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-10T09:51:01.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:00 smithi077 ceph-mon[30319]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-10T09:51:01.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:00 smithi077 ceph-mon[30319]: from='osd.10 [v2:172.21.15.120:6824/754733885,v1:172.21.15.120:6825/754733885]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-09-10T09:51:01.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:00 smithi077 ceph-mon[30319]: osdmap e68: 11 total, 10 up, 11 in 2024-09-10T09:51:01.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:00 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T09:51:01.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:00 smithi077 ceph-mon[30319]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-09-10T09:51:01.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:00 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:01.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:00 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:01.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:00 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:51:01.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:00 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:51:01.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:00 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:51:01.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:00 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:51:01.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:00 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:01.191 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:00 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:51:01.191 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:00 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:51:01.191 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:00 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:01.201 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:01.193+0000 7fb9e3720640 1 -- 172.21.15.134:0/3182495796 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb9dc102e00 msgr2=0x7fb9dc103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:01.201 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:01.193+0000 7fb9e3720640 1 --2- 172.21.15.134:0/3182495796 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb9dc102e00 0x7fb9dc103280 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc009900 tx=0x7fb9cc02a410 comp rx=0 tx=0).stop 2024-09-10T09:51:01.201 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:01.193+0000 7fb9e3720640 1 -- 172.21.15.134:0/3182495796 shutdown_connections 2024-09-10T09:51:01.201 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:01.193+0000 7fb9e3720640 1 --2- 172.21.15.134:0/3182495796 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb9dc1037c0 0x7fb9dc10a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:01.201 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:01.193+0000 7fb9e3720640 1 --2- 172.21.15.134:0/3182495796 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb9dc102e00 0x7fb9dc103280 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:01.201 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:01.193+0000 7fb9e3720640 1 --2- 172.21.15.134:0/3182495796 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb9dc101c00 0x7fb9dc102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:01.201 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:01.193+0000 7fb9e3720640 1 -- 172.21.15.134:0/3182495796 >> 172.21.15.134:0/3182495796 conn(0x7fb9dc0fd3b0 msgr2=0x7fb9dc0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:01.201 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:01.193+0000 7fb9e3720640 1 -- 172.21.15.134:0/3182495796 shutdown_connections 2024-09-10T09:51:01.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:01.193+0000 7fb9e3720640 1 -- 172.21.15.134:0/3182495796 wait complete. 2024-09-10T09:51:01.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:01.194+0000 7fb9e3720640 1 Processor -- start 2024-09-10T09:51:01.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:01.194+0000 7fb9e3720640 1 -- start start 2024-09-10T09:51:01.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:01.194+0000 7fb9e3720640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb9dc101c00 0x7fb9dc197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:01.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:01.194+0000 7fb9e3720640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb9dc102e00 0x7fb9dc198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:01.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:01.194+0000 7fb9e3720640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb9dc1037c0 0x7fb9dc19f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:01.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:01.194+0000 7fb9e3720640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb9dc10d880 con 0x7fb9dc102e00 2024-09-10T09:51:01.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:01.194+0000 7fb9e3720640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb9dc10d700 con 0x7fb9dc1037c0 2024-09-10T09:51:01.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:01.194+0000 7fb9e3720640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb9dc10da00 con 0x7fb9dc101c00 2024-09-10T09:51:01.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:01.194+0000 7fb9e14bc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb9dc101c00 0x7fb9dc197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:01.203 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:01.194+0000 7fb9e14bc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb9dc101c00 0x7fb9dc197e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:60440/0 (socket says 172.21.15.134:60440) 2024-09-10T09:51:01.203 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:01.194+0000 7fb9e14bc640 1 -- 172.21.15.134:0/2119220417 learned_addr learned my addr 172.21.15.134:0/2119220417 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-10T09:51:01.203 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:01.195+0000 7fb9e1cbd640 1 --2- 172.21.15.134:0/2119220417 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb9dc1037c0 0x7fb9dc19f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:01.203 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:01.195+0000 7fb9e0cbb640 1 --2- 172.21.15.134:0/2119220417 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb9dc102e00 0x7fb9dc198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:01.203 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:01.195+0000 7fb9e14bc640 1 -- 172.21.15.134:0/2119220417 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb9dc1037c0 msgr2=0x7fb9dc19f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:01.203 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:01.195+0000 7fb9e14bc640 1 --2- 172.21.15.134:0/2119220417 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb9dc1037c0 0x7fb9dc19f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:01.203 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:01.195+0000 7fb9e14bc640 1 -- 172.21.15.134:0/2119220417 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb9dc102e00 msgr2=0x7fb9dc198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:01.204 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:01.195+0000 7fb9e14bc640 1 --2- 172.21.15.134:0/2119220417 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb9dc102e00 0x7fb9dc198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:01.204 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:01.195+0000 7fb9e14bc640 1 -- 172.21.15.134:0/2119220417 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9dc19fa50 con 0x7fb9dc101c00 2024-09-10T09:51:01.204 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:01.195+0000 7fb9e1cbd640 1 --2- 172.21.15.134:0/2119220417 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb9dc1037c0 0x7fb9dc19f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:51:01.204 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:01.195+0000 7fb9e14bc640 1 --2- 172.21.15.134:0/2119220417 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb9dc101c00 0x7fb9dc197e20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb9d800c970 tx=0x7fb9d800ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:01.204 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:01.195+0000 7fb9d27fc640 1 -- 172.21.15.134:0/2119220417 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9d8007c10 con 0x7fb9dc101c00 2024-09-10T09:51:01.204 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:01.195+0000 7fb9e3720640 1 -- 172.21.15.134:0/2119220417 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9dc19fd40 con 0x7fb9dc101c00 2024-09-10T09:51:01.204 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:01.195+0000 7fb9d27fc640 1 -- 172.21.15.134:0/2119220417 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9d8007db0 con 0x7fb9dc101c00 2024-09-10T09:51:01.204 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:01.195+0000 7fb9d27fc640 1 -- 172.21.15.134:0/2119220417 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9d800f8a0 con 0x7fb9dc101c00 2024-09-10T09:51:01.205 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:01.195+0000 7fb9e3720640 1 -- 172.21.15.134:0/2119220417 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9dc1a01e0 con 0x7fb9dc101c00 2024-09-10T09:51:01.205 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:01.196+0000 7fb9e3720640 1 -- 172.21.15.134:0/2119220417 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9dc108f30 con 0x7fb9dc101c00 2024-09-10T09:51:01.205 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:01.197+0000 7fb9d27fc640 1 -- 172.21.15.134:0/2119220417 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fb9d800fac0 con 0x7fb9dc101c00 2024-09-10T09:51:01.205 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:01.198+0000 7fb9d27fc640 1 --2- 172.21.15.134:0/2119220417 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fb9c80a2bb0 0x7fb9c80a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:01.205 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:01.198+0000 7fb9d27fc640 1 -- 172.21.15.134:0/2119220417 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6414+0+0 (secure 0 0 0) 0x7fb9d80cec60 con 0x7fb9dc101c00 2024-09-10T09:51:01.205 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:01.198+0000 7fb9e0cbb640 1 --2- 172.21.15.134:0/2119220417 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fb9c80a2bb0 0x7fb9c80a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:01.205 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:01.199+0000 7fb9e0cbb640 1 --2- 172.21.15.134:0/2119220417 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fb9c80a2bb0 0x7fb9c80a5070 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc000c00 tx=0x7fb9cc0315d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:01.212 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:01.211+0000 7fb9d27fc640 1 -- 172.21.15.134:0/2119220417 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb9d809c3c0 con 0x7fb9dc101c00 2024-09-10T09:51:01.394 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:01.392+0000 7fb9e3720640 1 -- 172.21.15.134:0/2119220417 --> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fb9dc062630 con 0x7fb9c80a2bb0 2024-09-10T09:51:01.673 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 09:51:01 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T09:51:01.194+0000 7f75923ed640 -1 osd.10 0 waiting for initial osdmap 2024-09-10T09:51:01.674 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 09:51:01 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T09:51:01.207+0000 7f75899f9640 -1 osd.10 69 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T09:51:02.088 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:01 smithi134 ceph-mon[35674]: Detected new or changed devices on smithi120 2024-09-10T09:51:02.088 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:01 smithi134 ceph-mon[35674]: Adjusting osd_memory_target on smithi120 to 4272M 2024-09-10T09:51:02.088 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:01 smithi134 ceph-mon[35674]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2024-09-10T09:51:02.088 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:01 smithi134 ceph-mon[35674]: osdmap e69: 11 total, 10 up, 11 in 2024-09-10T09:51:02.088 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:01 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T09:51:02.088 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:01 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T09:51:02.089 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:01 smithi134 ceph-mon[35674]: from='osd.10 ' entity='osd.10' 2024-09-10T09:51:02.089 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:01 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:51:02.089 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:01 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:51:02.089 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:01 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:51:02.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:01 smithi120 ceph-mon[34770]: Detected new or changed devices on smithi120 2024-09-10T09:51:02.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:01 smithi120 ceph-mon[34770]: Adjusting osd_memory_target on smithi120 to 4272M 2024-09-10T09:51:02.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:01 smithi120 ceph-mon[34770]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2024-09-10T09:51:02.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:01 smithi120 ceph-mon[34770]: osdmap e69: 11 total, 10 up, 11 in 2024-09-10T09:51:02.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:01 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T09:51:02.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:01 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T09:51:02.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:01 smithi120 ceph-mon[34770]: from='osd.10 ' entity='osd.10' 2024-09-10T09:51:02.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:01 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:51:02.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:01 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:51:02.175 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:01 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:51:02.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:01 smithi077 ceph-mon[30319]: Detected new or changed devices on smithi120 2024-09-10T09:51:02.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:01 smithi077 ceph-mon[30319]: Adjusting osd_memory_target on smithi120 to 4272M 2024-09-10T09:51:02.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:01 smithi077 ceph-mon[30319]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2024-09-10T09:51:02.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:01 smithi077 ceph-mon[30319]: osdmap e69: 11 total, 10 up, 11 in 2024-09-10T09:51:02.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:01 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T09:51:02.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:01 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T09:51:02.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:01 smithi077 ceph-mon[30319]: from='osd.10 ' entity='osd.10' 2024-09-10T09:51:02.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:01 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:51:02.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:01 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:51:02.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:01 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:51:03.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:02 smithi134 ceph-mon[35674]: purged_snaps scrub starts 2024-09-10T09:51:03.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:02 smithi134 ceph-mon[35674]: purged_snaps scrub ok 2024-09-10T09:51:03.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:02 smithi134 ceph-mon[35674]: from='client.24385 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:51:03.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:02 smithi134 ceph-mon[35674]: pgmap v240: 1 pgs: 1 peering; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-10T09:51:03.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:02 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T09:51:03.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:02 smithi134 ceph-mon[35674]: osd.10 [v2:172.21.15.120:6824/754733885,v1:172.21.15.120:6825/754733885] boot 2024-09-10T09:51:03.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:02 smithi134 ceph-mon[35674]: osdmap e70: 11 total, 11 up, 11 in 2024-09-10T09:51:03.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:02 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T09:51:03.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:02 smithi120 ceph-mon[34770]: purged_snaps scrub starts 2024-09-10T09:51:03.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:02 smithi120 ceph-mon[34770]: purged_snaps scrub ok 2024-09-10T09:51:03.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:02 smithi120 ceph-mon[34770]: from='client.24385 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:51:03.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:02 smithi120 ceph-mon[34770]: pgmap v240: 1 pgs: 1 peering; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-10T09:51:03.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:02 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T09:51:03.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:02 smithi120 ceph-mon[34770]: osd.10 [v2:172.21.15.120:6824/754733885,v1:172.21.15.120:6825/754733885] boot 2024-09-10T09:51:03.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:02 smithi120 ceph-mon[34770]: osdmap e70: 11 total, 11 up, 11 in 2024-09-10T09:51:03.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:02 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T09:51:03.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:02 smithi077 ceph-mon[30319]: purged_snaps scrub starts 2024-09-10T09:51:03.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:02 smithi077 ceph-mon[30319]: purged_snaps scrub ok 2024-09-10T09:51:03.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:02 smithi077 ceph-mon[30319]: from='client.24385 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:51:03.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:02 smithi077 ceph-mon[30319]: pgmap v240: 1 pgs: 1 peering; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-10T09:51:03.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:02 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T09:51:03.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:02 smithi077 ceph-mon[30319]: osd.10 [v2:172.21.15.120:6824/754733885,v1:172.21.15.120:6825/754733885] boot 2024-09-10T09:51:03.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:02 smithi077 ceph-mon[30319]: osdmap e70: 11 total, 11 up, 11 in 2024-09-10T09:51:03.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:02 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T09:51:04.106 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:03 smithi134 ceph-mon[35674]: osdmap e71: 11 total, 11 up, 11 in 2024-09-10T09:51:04.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:03 smithi120 ceph-mon[34770]: osdmap e71: 11 total, 11 up, 11 in 2024-09-10T09:51:04.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:03 smithi077 ceph-mon[30319]: osdmap e71: 11 total, 11 up, 11 in 2024-09-10T09:51:05.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:04 smithi134 ceph-mon[35674]: pgmap v243: 1 pgs: 1 peering; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-09-10T09:51:05.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:04 smithi134 ceph-mon[35674]: osdmap e72: 11 total, 11 up, 11 in 2024-09-10T09:51:05.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:04 smithi134 ceph-mon[35674]: from='client.? 172.21.15.134:0/1481486911' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "85c1ccd4-5147-4995-896e-fe977f73f749"}]: dispatch 2024-09-10T09:51:05.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:04 smithi134 ceph-mon[35674]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "85c1ccd4-5147-4995-896e-fe977f73f749"}]: dispatch 2024-09-10T09:51:05.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:04 smithi134 ceph-mon[35674]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "85c1ccd4-5147-4995-896e-fe977f73f749"}]': finished 2024-09-10T09:51:05.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:04 smithi134 ceph-mon[35674]: osdmap e73: 12 total, 11 up, 12 in 2024-09-10T09:51:05.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:04 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T09:51:05.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:04 smithi134 ceph-mon[35674]: from='client.? 172.21.15.134:0/665261783' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:51:05.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:04 smithi120 ceph-mon[34770]: pgmap v243: 1 pgs: 1 peering; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-09-10T09:51:05.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:04 smithi120 ceph-mon[34770]: osdmap e72: 11 total, 11 up, 11 in 2024-09-10T09:51:05.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:04 smithi120 ceph-mon[34770]: from='client.? 172.21.15.134:0/1481486911' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "85c1ccd4-5147-4995-896e-fe977f73f749"}]: dispatch 2024-09-10T09:51:05.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:04 smithi120 ceph-mon[34770]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "85c1ccd4-5147-4995-896e-fe977f73f749"}]: dispatch 2024-09-10T09:51:05.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:04 smithi120 ceph-mon[34770]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "85c1ccd4-5147-4995-896e-fe977f73f749"}]': finished 2024-09-10T09:51:05.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:04 smithi120 ceph-mon[34770]: osdmap e73: 12 total, 11 up, 12 in 2024-09-10T09:51:05.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:04 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T09:51:05.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:04 smithi120 ceph-mon[34770]: from='client.? 172.21.15.134:0/665261783' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:51:05.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:04 smithi077 ceph-mon[30319]: pgmap v243: 1 pgs: 1 peering; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-09-10T09:51:05.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:04 smithi077 ceph-mon[30319]: osdmap e72: 11 total, 11 up, 11 in 2024-09-10T09:51:05.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:04 smithi077 ceph-mon[30319]: from='client.? 172.21.15.134:0/1481486911' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "85c1ccd4-5147-4995-896e-fe977f73f749"}]: dispatch 2024-09-10T09:51:05.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:04 smithi077 ceph-mon[30319]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "85c1ccd4-5147-4995-896e-fe977f73f749"}]: dispatch 2024-09-10T09:51:05.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:04 smithi077 ceph-mon[30319]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "85c1ccd4-5147-4995-896e-fe977f73f749"}]': finished 2024-09-10T09:51:05.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:04 smithi077 ceph-mon[30319]: osdmap e73: 12 total, 11 up, 12 in 2024-09-10T09:51:05.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:04 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T09:51:05.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:04 smithi077 ceph-mon[30319]: from='client.? 172.21.15.134:0/665261783' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:51:06.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:05 smithi134 ceph-mon[35674]: osdmap e74: 12 total, 11 up, 12 in 2024-09-10T09:51:06.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:05 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T09:51:06.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:05 smithi120 ceph-mon[34770]: osdmap e74: 12 total, 11 up, 12 in 2024-09-10T09:51:06.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:05 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T09:51:06.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:05 smithi077 ceph-mon[30319]: osdmap e74: 12 total, 11 up, 12 in 2024-09-10T09:51:06.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:05 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T09:51:07.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:06 smithi134 ceph-mon[35674]: pgmap v247: 1 pgs: 1 peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-10T09:51:07.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:06 smithi134 ceph-mon[35674]: osdmap e75: 12 total, 11 up, 12 in 2024-09-10T09:51:07.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:06 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T09:51:07.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:06 smithi120 ceph-mon[34770]: pgmap v247: 1 pgs: 1 peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-10T09:51:07.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:06 smithi120 ceph-mon[34770]: osdmap e75: 12 total, 11 up, 12 in 2024-09-10T09:51:07.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:06 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T09:51:07.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:06 smithi077 ceph-mon[30319]: pgmap v247: 1 pgs: 1 peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-10T09:51:07.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:06 smithi077 ceph-mon[30319]: osdmap e75: 12 total, 11 up, 12 in 2024-09-10T09:51:07.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:06 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T09:51:08.996 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:08 smithi134 ceph-mon[35674]: pgmap v249: 1 pgs: 1 peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-10T09:51:09.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:08 smithi120 ceph-mon[34770]: pgmap v249: 1 pgs: 1 peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-10T09:51:09.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:08 smithi077 ceph-mon[30319]: pgmap v249: 1 pgs: 1 peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-10T09:51:11.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:10 smithi134 ceph-mon[35674]: pgmap v250: 1 pgs: 1 peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-09-10T09:51:11.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:10 smithi120 ceph-mon[34770]: pgmap v250: 1 pgs: 1 peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-09-10T09:51:11.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:10 smithi077 ceph-mon[30319]: pgmap v250: 1 pgs: 1 peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-09-10T09:51:13.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:12 smithi134 ceph-mon[35674]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-09-10T09:51:13.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:12 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-10T09:51:13.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:12 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:51:13.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:12 smithi120 ceph-mon[34770]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-09-10T09:51:13.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:12 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-10T09:51:13.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:12 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:51:13.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:12 smithi077 ceph-mon[30319]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-09-10T09:51:13.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:12 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-10T09:51:13.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:12 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:51:14.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:13 smithi120 ceph-mon[34770]: Deploying daemon osd.11 on smithi134 2024-09-10T09:51:14.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:13 smithi077 ceph-mon[30319]: Deploying daemon osd.11 on smithi134 2024-09-10T09:51:14.242 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:13 smithi134 ceph-mon[35674]: Deploying daemon osd.11 on smithi134 2024-09-10T09:51:15.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:14 smithi120 ceph-mon[34770]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-09-10T09:51:15.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:14 smithi077 ceph-mon[30319]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-09-10T09:51:15.300 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:14 smithi134 ceph-mon[35674]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-09-10T09:51:16.202 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:15 smithi134 ceph-mon[35674]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T09:51:16.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:15 smithi120 ceph-mon[34770]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T09:51:16.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:15 smithi077 ceph-mon[30319]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T09:51:19.091 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:18 smithi134 ceph-mon[35674]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-10T09:51:19.092 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:18 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:51:19.092 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:18 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:19.092 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:18 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:19.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:18 smithi120 ceph-mon[34770]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-10T09:51:19.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:18 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:51:19.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:18 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:19.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:18 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:19.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:18 smithi077 ceph-mon[30319]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-10T09:51:19.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:18 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:51:19.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:18 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:19.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:18 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:20.759 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:20 smithi134 ceph-mon[35674]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-10T09:51:21.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:20 smithi120 ceph-mon[34770]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-10T09:51:21.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:20 smithi077 ceph-mon[30319]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-10T09:51:22.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:22.273+0000 7fb9d27fc640 1 -- 172.21.15.134:0/2119220417 <== mgr.14150 v2:172.21.15.77:6800/4261324650 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fb9dc062630 con 0x7fb9c80a2bb0 2024-09-10T09:51:22.275 INFO:teuthology.orchestra.run.smithi134.stdout:Created osd(s) 11 on host 'smithi134' 2024-09-10T09:51:22.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:22.275+0000 7fb9e3720640 1 -- 172.21.15.134:0/2119220417 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fb9c80a2bb0 msgr2=0x7fb9c80a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:22.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:22.275+0000 7fb9e3720640 1 --2- 172.21.15.134:0/2119220417 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fb9c80a2bb0 0x7fb9c80a5070 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc000c00 tx=0x7fb9cc0315d0 comp rx=0 tx=0).stop 2024-09-10T09:51:22.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:22.275+0000 7fb9e3720640 1 -- 172.21.15.134:0/2119220417 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb9dc101c00 msgr2=0x7fb9dc197e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:22.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:22.275+0000 7fb9e3720640 1 --2- 172.21.15.134:0/2119220417 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb9dc101c00 0x7fb9dc197e20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb9d800c970 tx=0x7fb9d800ce30 comp rx=0 tx=0).stop 2024-09-10T09:51:22.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:22.275+0000 7fb9e3720640 1 -- 172.21.15.134:0/2119220417 shutdown_connections 2024-09-10T09:51:22.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:22.275+0000 7fb9e3720640 1 --2- 172.21.15.134:0/2119220417 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fb9c80a2bb0 0x7fb9c80a5070 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:22.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:22.275+0000 7fb9e3720640 1 --2- 172.21.15.134:0/2119220417 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb9dc1037c0 0x7fb9dc19f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:22.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:22.275+0000 7fb9e3720640 1 --2- 172.21.15.134:0/2119220417 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb9dc102e00 0x7fb9dc198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:22.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:22.275+0000 7fb9e3720640 1 --2- 172.21.15.134:0/2119220417 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb9dc101c00 0x7fb9dc197e20 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:22.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:22.275+0000 7fb9e3720640 1 -- 172.21.15.134:0/2119220417 >> 172.21.15.134:0/2119220417 conn(0x7fb9dc0fd3b0 msgr2=0x7fb9dc106100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:22.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:22.275+0000 7fb9e3720640 1 -- 172.21.15.134:0/2119220417 shutdown_connections 2024-09-10T09:51:22.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:22.275+0000 7fb9e3720640 1 -- 172.21.15.134:0/2119220417 wait complete. 2024-09-10T09:51:22.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:22 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:22.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:22 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:22.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:22 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:51:22.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:22 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:51:22.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:22 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:22.882 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:22 smithi134 ceph-mon[35674]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-10T09:51:22.882 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:22 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:51:22.882 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:22 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:22.882 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:22 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:22.923 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:22 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:22.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:22 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:22.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:22 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:51:22.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:22 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:51:22.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:22 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:22.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:22 smithi120 ceph-mon[34770]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-10T09:51:22.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:22 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:51:22.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:22 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:22.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:22 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:22.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:22 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:22.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:22 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:22.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:22 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:51:22.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:22 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:51:22.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:22 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:22.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:22 smithi077 ceph-mon[30319]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-10T09:51:22.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:22 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:51:22.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:22 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:22.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:22 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:22.957 DEBUG:teuthology.orchestra.run.smithi134:osd.11> sudo journalctl -f -n 0 -u ceph-14a55226-6f59-11ef-bcea-c7b262605968@osd.11.service 2024-09-10T09:51:22.959 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-09-10T09:51:22.959 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-10T09:51:23.331 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:51:23.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:23.929+0000 7f8fa5c00640 1 -- 172.21.15.77:0/2776978281 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8fa0103c50 msgr2=0x7f8fa0104050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:23.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:23.929+0000 7f8fa5c00640 1 --2- 172.21.15.77:0/2776978281 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8fa0103c50 0x7f8fa0104050 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8f90009900 tx=0x7f8f9002a410 comp rx=0 tx=0).stop 2024-09-10T09:51:23.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:23.929+0000 7f8fa5c00640 1 -- 172.21.15.77:0/2776978281 shutdown_connections 2024-09-10T09:51:23.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:23.929+0000 7f8fa5c00640 1 --2- 172.21.15.77:0/2776978281 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8fa0105810 0x7f8fa010c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:23.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:23.929+0000 7f8fa5c00640 1 --2- 172.21.15.77:0/2776978281 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8fa0104e50 0x7f8fa01052d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:23.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:23.929+0000 7f8fa5c00640 1 --2- 172.21.15.77:0/2776978281 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8fa0103c50 0x7f8fa0104050 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:23.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:23.929+0000 7f8fa5c00640 1 -- 172.21.15.77:0/2776978281 >> 172.21.15.77:0/2776978281 conn(0x7f8fa00ff3e0 msgr2=0x7f8fa0101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:23.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:23.929+0000 7f8fa5c00640 1 -- 172.21.15.77:0/2776978281 shutdown_connections 2024-09-10T09:51:23.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:23.929+0000 7f8fa5c00640 1 -- 172.21.15.77:0/2776978281 wait complete. 2024-09-10T09:51:23.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:23.930+0000 7f8fa5c00640 1 Processor -- start 2024-09-10T09:51:23.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:23.930+0000 7f8fa5c00640 1 -- start start 2024-09-10T09:51:23.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:23.930+0000 7f8fa5c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8fa0103c50 0x7f8fa0199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:23.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:23.930+0000 7f8fa5c00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8fa0104e50 0x7f8fa019a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:23.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:23.930+0000 7f8fa5c00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8fa0105810 0x7f8fa01a1510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:23.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:23.930+0000 7f8fa5c00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8fa010f920 con 0x7f8fa0105810 2024-09-10T09:51:23.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:23.930+0000 7f8fa5c00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8fa010f7a0 con 0x7f8fa0104e50 2024-09-10T09:51:23.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:23.930+0000 7f8fa5c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8fa010faa0 con 0x7f8fa0103c50 2024-09-10T09:51:23.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:23.931+0000 7f8f9fe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8fa0103c50 0x7f8fa0199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:23.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:23.931+0000 7f8fa5200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8fa0105810 0x7f8fa01a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:23.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:23.931+0000 7f8f9fe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8fa0103c50 0x7f8fa0199f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:60834/0 (socket says 172.21.15.77:60834) 2024-09-10T09:51:23.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:23.931+0000 7f8f9fe00640 1 -- 172.21.15.77:0/3696234329 learned_addr learned my addr 172.21.15.77:0/3696234329 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:51:23.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:23.931+0000 7f8f9f400640 1 --2- 172.21.15.77:0/3696234329 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8fa0104e50 0x7f8fa019a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:23.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:23.931+0000 7f8f9fe00640 1 -- 172.21.15.77:0/3696234329 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8fa0104e50 msgr2=0x7f8fa019a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:23.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:23.931+0000 7f8f9fe00640 1 --2- 172.21.15.77:0/3696234329 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8fa0104e50 0x7f8fa019a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:23.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:23.931+0000 7f8f9fe00640 1 -- 172.21.15.77:0/3696234329 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8fa0105810 msgr2=0x7f8fa01a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:23.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:23.931+0000 7f8f9fe00640 1 --2- 172.21.15.77:0/3696234329 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8fa0105810 0x7f8fa01a1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:23.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:23.931+0000 7f8f9fe00640 1 -- 172.21.15.77:0/3696234329 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fa01a1b80 con 0x7f8fa0103c50 2024-09-10T09:51:23.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:23.931+0000 7f8f9fe00640 1 --2- 172.21.15.77:0/3696234329 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8fa0103c50 0x7f8fa0199f50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8f90031440 tx=0x7f8f90032ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:23.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:23.932+0000 7f8f9cc00640 1 -- 172.21.15.77:0/3696234329 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f9003f030 con 0x7f8fa0103c50 2024-09-10T09:51:23.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:23.932+0000 7f8fa5c00640 1 -- 172.21.15.77:0/3696234329 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fa01a1e10 con 0x7f8fa0103c50 2024-09-10T09:51:23.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:23.932+0000 7f8f9cc00640 1 -- 172.21.15.77:0/3696234329 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f90003ce0 con 0x7f8fa0103c50 2024-09-10T09:51:23.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:23.932+0000 7f8f9cc00640 1 -- 172.21.15.77:0/3696234329 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f90035070 con 0x7f8fa0103c50 2024-09-10T09:51:23.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:23.932+0000 7f8fa5c00640 1 -- 172.21.15.77:0/3696234329 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fa01a2200 con 0x7f8fa0103c50 2024-09-10T09:51:23.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:23.933+0000 7f8fa5c00640 1 -- 172.21.15.77:0/3696234329 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fa010b050 con 0x7f8fa0103c50 2024-09-10T09:51:23.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:23.935+0000 7f8f9cc00640 1 -- 172.21.15.77:0/3696234329 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f8f90042030 con 0x7f8fa0103c50 2024-09-10T09:51:23.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:23.936+0000 7f8f9cc00640 1 --2- 172.21.15.77:0/3696234329 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f8f840a2bb0 0x7f8f840a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:23.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:23.936+0000 7f8f9cc00640 1 -- 172.21.15.77:0/3696234329 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6809+0+0 (secure 0 0 0) 0x7f8f900eef40 con 0x7f8fa0103c50 2024-09-10T09:51:23.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:23.936+0000 7f8f9f400640 1 --2- 172.21.15.77:0/3696234329 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f8f840a2bb0 0x7f8f840a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:23.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:23.937+0000 7f8f9f400640 1 --2- 172.21.15.77:0/3696234329 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f8f840a2bb0 0x7f8f840a5070 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8fa019b470 tx=0x7f8f8c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:23.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:23.940+0000 7f8f9cc00640 1 -- 172.21.15.77:0/3696234329 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8f900bbd90 con 0x7f8fa0103c50 2024-09-10T09:51:24.117 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:24.116+0000 7f8fa5c00640 1 -- 172.21.15.77:0/3696234329 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f8fa0062e20 con 0x7f8fa0103c50 2024-09-10T09:51:24.117 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:24.116+0000 7f8f9cc00640 1 -- 172.21.15.77:0/3696234329 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v75) ==== 74+0+133 (secure 0 0 0) 0x7f8f900c0090 con 0x7f8fa0103c50 2024-09-10T09:51:24.117 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:51:24.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:24.119+0000 7f8fa5c00640 1 -- 172.21.15.77:0/3696234329 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f8f840a2bb0 msgr2=0x7f8f840a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:24.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:24.119+0000 7f8fa5c00640 1 --2- 172.21.15.77:0/3696234329 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f8f840a2bb0 0x7f8f840a5070 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8fa019b470 tx=0x7f8f8c005f90 comp rx=0 tx=0).stop 2024-09-10T09:51:24.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:24.119+0000 7f8fa5c00640 1 -- 172.21.15.77:0/3696234329 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8fa0103c50 msgr2=0x7f8fa0199f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:24.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:24.119+0000 7f8fa5c00640 1 --2- 172.21.15.77:0/3696234329 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8fa0103c50 0x7f8fa0199f50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8f90031440 tx=0x7f8f90032ce0 comp rx=0 tx=0).stop 2024-09-10T09:51:24.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:24.119+0000 7f8fa5c00640 1 -- 172.21.15.77:0/3696234329 shutdown_connections 2024-09-10T09:51:24.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:24.119+0000 7f8fa5c00640 1 --2- 172.21.15.77:0/3696234329 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f8f840a2bb0 0x7f8f840a5070 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:24.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:24.119+0000 7f8fa5c00640 1 --2- 172.21.15.77:0/3696234329 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8fa0105810 0x7f8fa01a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:24.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:24.119+0000 7f8fa5c00640 1 --2- 172.21.15.77:0/3696234329 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8fa0104e50 0x7f8fa019a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:24.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:24.119+0000 7f8fa5c00640 1 --2- 172.21.15.77:0/3696234329 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8fa0103c50 0x7f8fa0199f50 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:24.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:24.119+0000 7f8fa5c00640 1 -- 172.21.15.77:0/3696234329 >> 172.21.15.77:0/3696234329 conn(0x7f8fa00ff3e0 msgr2=0x7f8fa0108220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:24.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:24.119+0000 7f8fa5c00640 1 -- 172.21.15.77:0/3696234329 shutdown_connections 2024-09-10T09:51:24.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:24.119+0000 7f8fa5c00640 1 -- 172.21.15.77:0/3696234329 wait complete. 2024-09-10T09:51:24.458 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 09:51:24 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T09:51:24.201+0000 7f5c019d1640 -1 osd.11 0 log_to_monitors true 2024-09-10T09:51:24.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:24 smithi077 ceph-mon[30319]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-10T09:51:24.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:24 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/3696234329' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T09:51:24.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:24 smithi077 ceph-mon[30319]: from='osd.11 [v2:172.21.15.134:6824/1602213143,v1:172.21.15.134:6825/1602213143]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-10T09:51:24.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:24 smithi077 ceph-mon[30319]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-10T09:51:25.102 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":75,"num_osds":12,"num_up_osds":11,"osd_up_since":1725961861,"num_in_osds":12,"osd_in_since":1725961863,"num_remapped_pgs":0} 2024-09-10T09:51:25.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:24 smithi134 ceph-mon[35674]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-10T09:51:25.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:24 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/3696234329' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T09:51:25.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:24 smithi134 ceph-mon[35674]: from='osd.11 [v2:172.21.15.134:6824/1602213143,v1:172.21.15.134:6825/1602213143]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-10T09:51:25.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:24 smithi134 ceph-mon[35674]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-10T09:51:25.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:24 smithi120 ceph-mon[34770]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-10T09:51:25.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:24 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/3696234329' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T09:51:25.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:24 smithi120 ceph-mon[34770]: from='osd.11 [v2:172.21.15.134:6824/1602213143,v1:172.21.15.134:6825/1602213143]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-10T09:51:25.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:24 smithi120 ceph-mon[34770]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-10T09:51:25.881 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 09:51:25 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T09:51:25.736+0000 7f5bfd954640 -1 osd.11 0 waiting for initial osdmap 2024-09-10T09:51:25.881 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 09:51:25 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T09:51:25.749+0000 7f5bf977c640 -1 osd.11 77 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T09:51:25.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:25 smithi134 ceph-mon[35674]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-10T09:51:25.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:25 smithi134 ceph-mon[35674]: osdmap e76: 12 total, 11 up, 12 in 2024-09-10T09:51:25.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:25 smithi134 ceph-mon[35674]: from='osd.11 [v2:172.21.15.134:6824/1602213143,v1:172.21.15.134:6825/1602213143]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-09-10T09:51:25.882 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:25 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T09:51:25.882 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:25 smithi134 ceph-mon[35674]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-09-10T09:51:26.045 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:25 smithi077 ceph-mon[30319]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-10T09:51:26.045 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:25 smithi077 ceph-mon[30319]: osdmap e76: 12 total, 11 up, 12 in 2024-09-10T09:51:26.045 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:25 smithi077 ceph-mon[30319]: from='osd.11 [v2:172.21.15.134:6824/1602213143,v1:172.21.15.134:6825/1602213143]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-09-10T09:51:26.045 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:25 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T09:51:26.045 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:25 smithi077 ceph-mon[30319]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-09-10T09:51:26.104 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-10T09:51:26.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:25 smithi120 ceph-mon[34770]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-10T09:51:26.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:25 smithi120 ceph-mon[34770]: osdmap e76: 12 total, 11 up, 12 in 2024-09-10T09:51:26.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:25 smithi120 ceph-mon[34770]: from='osd.11 [v2:172.21.15.134:6824/1602213143,v1:172.21.15.134:6825/1602213143]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-09-10T09:51:26.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:25 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T09:51:26.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:25 smithi120 ceph-mon[34770]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-09-10T09:51:26.476 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:51:26.911 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:26 smithi077 ceph-mon[30319]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-10T09:51:26.911 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:26 smithi077 ceph-mon[30319]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-09-10T09:51:26.912 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:26 smithi077 ceph-mon[30319]: osdmap e77: 12 total, 11 up, 12 in 2024-09-10T09:51:26.912 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:26 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T09:51:26.912 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:26 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T09:51:26.912 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:26 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:26.912 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:26 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:26.912 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:26 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:51:26.912 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:26 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:51:26.912 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:26 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:51:26.912 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:26 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:51:26.912 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:26 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:26.912 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:26 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:51:26.912 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:26 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:51:26.913 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:26 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:27.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:27.024+0000 7f7593200640 1 -- 172.21.15.77:0/559138937 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f758c109aa0 msgr2=0x7f758c10bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:27.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:27.024+0000 7f7593200640 1 --2- 172.21.15.77:0/559138937 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f758c109aa0 0x7f758c10bf40 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7584009ca0 tx=0x7f758402a950 comp rx=0 tx=0).stop 2024-09-10T09:51:27.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:27.024+0000 7f7593200640 1 -- 172.21.15.77:0/559138937 shutdown_connections 2024-09-10T09:51:27.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:27.024+0000 7f7593200640 1 --2- 172.21.15.77:0/559138937 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f758c109aa0 0x7f758c10bf40 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:27.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:27.024+0000 7f7593200640 1 --2- 172.21.15.77:0/559138937 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f758c074bf0 0x7f758c075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:27.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:27.024+0000 7f7593200640 1 --2- 172.21.15.77:0/559138937 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f758c0761c0 0x7f758c074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:27.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:27.024+0000 7f7593200640 1 -- 172.21.15.77:0/559138937 >> 172.21.15.77:0/559138937 conn(0x7f758c0fd390 msgr2=0x7f758c0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:27.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:27.024+0000 7f7593200640 1 -- 172.21.15.77:0/559138937 shutdown_connections 2024-09-10T09:51:27.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:27.024+0000 7f7593200640 1 -- 172.21.15.77:0/559138937 wait complete. 2024-09-10T09:51:27.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:27.025+0000 7f7593200640 1 Processor -- start 2024-09-10T09:51:27.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:27.025+0000 7f7593200640 1 -- start start 2024-09-10T09:51:27.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:27.026+0000 7f7593200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f758c074bf0 0x7f758c19a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:27.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:27.026+0000 7f7593200640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f758c0761c0 0x7f758c19a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:27.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:27.026+0000 7f7593200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f758c109aa0 0x7f758c1a1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:27.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:27.026+0000 7f7593200640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f758c10fad0 con 0x7f758c074bf0 2024-09-10T09:51:27.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:27.026+0000 7f7593200640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f758c10f950 con 0x7f758c0761c0 2024-09-10T09:51:27.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:27.026+0000 7f7593200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f758c10fc50 con 0x7f758c109aa0 2024-09-10T09:51:27.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:27.026+0000 7f758be00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f758c074bf0 0x7f758c19a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:27.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:27.026+0000 7f758be00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f758c074bf0 0x7f758c19a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34126/0 (socket says 172.21.15.77:34126) 2024-09-10T09:51:27.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:27.026+0000 7f7591200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f758c109aa0 0x7f758c1a1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:27.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:27.026+0000 7f758be00640 1 -- 172.21.15.77:0/236312220 learned_addr learned my addr 172.21.15.77:0/236312220 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:51:27.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:27.026+0000 7f758b400640 1 --2- 172.21.15.77:0/236312220 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f758c0761c0 0x7f758c19a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:27.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:27.026+0000 7f7591200640 1 -- 172.21.15.77:0/236312220 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f758c0761c0 msgr2=0x7f758c19a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:27.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:27.026+0000 7f7591200640 1 --2- 172.21.15.77:0/236312220 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f758c0761c0 0x7f758c19a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:27.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:27.026+0000 7f7591200640 1 -- 172.21.15.77:0/236312220 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f758c074bf0 msgr2=0x7f758c19a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:27.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:27.026+0000 7f7591200640 1 --2- 172.21.15.77:0/236312220 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f758c074bf0 0x7f758c19a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:27.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:27.026+0000 7f7591200640 1 -- 172.21.15.77:0/236312220 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f758c1a1ca0 con 0x7f758c109aa0 2024-09-10T09:51:27.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:27.027+0000 7f7591200640 1 --2- 172.21.15.77:0/236312220 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f758c109aa0 0x7f758c1a1630 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f758402af40 tx=0x7f758402fce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:27.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:27.027+0000 7f7588c00640 1 -- 172.21.15.77:0/236312220 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7584034040 con 0x7f758c109aa0 2024-09-10T09:51:27.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:27.027+0000 7f7588c00640 1 -- 172.21.15.77:0/236312220 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7584030920 con 0x7f758c109aa0 2024-09-10T09:51:27.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:27.027+0000 7f7593200640 1 -- 172.21.15.77:0/236312220 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f758c1a1f30 con 0x7f758c109aa0 2024-09-10T09:51:27.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:27.027+0000 7f7588c00640 1 -- 172.21.15.77:0/236312220 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7584004800 con 0x7f758c109aa0 2024-09-10T09:51:27.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:27.027+0000 7f7593200640 1 -- 172.21.15.77:0/236312220 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f758c1a2380 con 0x7f758c109aa0 2024-09-10T09:51:27.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:27.028+0000 7f7593200640 1 -- 172.21.15.77:0/236312220 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f758c108860 con 0x7f758c109aa0 2024-09-10T09:51:27.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:27.030+0000 7f7588c00640 1 -- 172.21.15.77:0/236312220 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f7584037020 con 0x7f758c109aa0 2024-09-10T09:51:27.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:27.032+0000 7f7588c00640 1 --2- 172.21.15.77:0/236312220 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f75700a2bb0 0x7f75700a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:27.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:27.032+0000 7f7588c00640 1 -- 172.21.15.77:0/236312220 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7f75840efa60 con 0x7f758c109aa0 2024-09-10T09:51:27.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:27.032+0000 7f758be00640 1 --2- 172.21.15.77:0/236312220 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f75700a2bb0 0x7f75700a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:27.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:27.032+0000 7f758be00640 1 --2- 172.21.15.77:0/236312220 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f75700a2bb0 0x7f75700a5070 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f757c0095f0 tx=0x7f757c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:27.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:27.034+0000 7f7588c00640 1 -- 172.21.15.77:0/236312220 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f75840bcdc0 con 0x7f758c109aa0 2024-09-10T09:51:27.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:26 smithi134 ceph-mon[35674]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-10T09:51:27.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:26 smithi134 ceph-mon[35674]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-09-10T09:51:27.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:26 smithi134 ceph-mon[35674]: osdmap e77: 12 total, 11 up, 12 in 2024-09-10T09:51:27.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:26 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T09:51:27.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:26 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T09:51:27.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:26 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:27.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:26 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:27.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:26 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:51:27.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:26 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:51:27.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:26 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:51:27.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:26 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:51:27.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:26 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:27.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:26 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:51:27.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:26 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:51:27.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:26 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:27.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:26 smithi120 ceph-mon[34770]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-10T09:51:27.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:26 smithi120 ceph-mon[34770]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-09-10T09:51:27.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:26 smithi120 ceph-mon[34770]: osdmap e77: 12 total, 11 up, 12 in 2024-09-10T09:51:27.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:26 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T09:51:27.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:26 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T09:51:27.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:26 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:27.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:26 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:27.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:26 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:51:27.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:26 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:51:27.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:26 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:51:27.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:26 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:51:27.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:26 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:27.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:26 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:51:27.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:26 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:51:27.175 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:26 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:27.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:27.219+0000 7f7593200640 1 -- 172.21.15.77:0/236312220 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f758c062e90 con 0x7f758c109aa0 2024-09-10T09:51:27.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:27.219+0000 7f7588c00640 1 -- 172.21.15.77:0/236312220 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v78) ==== 74+0+133 (secure 0 0 0) 0x7f75840bc6c0 con 0x7f758c109aa0 2024-09-10T09:51:27.220 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:51:27.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:27.222+0000 7f7593200640 1 -- 172.21.15.77:0/236312220 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f75700a2bb0 msgr2=0x7f75700a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:27.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:27.222+0000 7f7593200640 1 --2- 172.21.15.77:0/236312220 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f75700a2bb0 0x7f75700a5070 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f757c0095f0 tx=0x7f757c005f90 comp rx=0 tx=0).stop 2024-09-10T09:51:27.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:27.222+0000 7f7593200640 1 -- 172.21.15.77:0/236312220 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f758c109aa0 msgr2=0x7f758c1a1630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:27.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:27.222+0000 7f7593200640 1 --2- 172.21.15.77:0/236312220 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f758c109aa0 0x7f758c1a1630 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f758402af40 tx=0x7f758402fce0 comp rx=0 tx=0).stop 2024-09-10T09:51:27.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:27.222+0000 7f7593200640 1 -- 172.21.15.77:0/236312220 shutdown_connections 2024-09-10T09:51:27.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:27.222+0000 7f7593200640 1 --2- 172.21.15.77:0/236312220 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f75700a2bb0 0x7f75700a5070 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:27.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:27.222+0000 7f7593200640 1 --2- 172.21.15.77:0/236312220 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f758c109aa0 0x7f758c1a1630 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:27.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:27.222+0000 7f7593200640 1 --2- 172.21.15.77:0/236312220 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f758c0761c0 0x7f758c19a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:27.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:27.222+0000 7f7593200640 1 --2- 172.21.15.77:0/236312220 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f758c074bf0 0x7f758c19a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:27.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:27.222+0000 7f7593200640 1 -- 172.21.15.77:0/236312220 >> 172.21.15.77:0/236312220 conn(0x7f758c0fd390 msgr2=0x7f758c105a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:27.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:27.222+0000 7f7593200640 1 -- 172.21.15.77:0/236312220 shutdown_connections 2024-09-10T09:51:27.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:27.222+0000 7f7593200640 1 -- 172.21.15.77:0/236312220 wait complete. 2024-09-10T09:51:27.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:27 smithi077 ceph-mon[30319]: purged_snaps scrub starts 2024-09-10T09:51:27.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:27 smithi077 ceph-mon[30319]: purged_snaps scrub ok 2024-09-10T09:51:27.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:27 smithi077 ceph-mon[30319]: Detected new or changed devices on smithi134 2024-09-10T09:51:27.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:27 smithi077 ceph-mon[30319]: Adjusting osd_memory_target on smithi134 to 4272M 2024-09-10T09:51:27.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:27 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T09:51:27.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:27 smithi077 ceph-mon[30319]: osd.11 [v2:172.21.15.134:6824/1602213143,v1:172.21.15.134:6825/1602213143] boot 2024-09-10T09:51:27.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:27 smithi077 ceph-mon[30319]: osdmap e78: 12 total, 12 up, 12 in 2024-09-10T09:51:27.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:27 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T09:51:27.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:27 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/236312220' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T09:51:28.104 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":78,"num_osds":12,"num_up_osds":12,"osd_up_since":1725961886,"num_in_osds":12,"osd_in_since":1725961863,"num_remapped_pgs":0} 2024-09-10T09:51:28.105 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T09:51:28.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:27 smithi134 ceph-mon[35674]: purged_snaps scrub starts 2024-09-10T09:51:28.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:27 smithi134 ceph-mon[35674]: purged_snaps scrub ok 2024-09-10T09:51:28.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:27 smithi134 ceph-mon[35674]: Detected new or changed devices on smithi134 2024-09-10T09:51:28.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:27 smithi134 ceph-mon[35674]: Adjusting osd_memory_target on smithi134 to 4272M 2024-09-10T09:51:28.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:27 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T09:51:28.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:27 smithi134 ceph-mon[35674]: osd.11 [v2:172.21.15.134:6824/1602213143,v1:172.21.15.134:6825/1602213143] boot 2024-09-10T09:51:28.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:27 smithi134 ceph-mon[35674]: osdmap e78: 12 total, 12 up, 12 in 2024-09-10T09:51:28.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:27 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T09:51:28.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:27 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/236312220' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T09:51:28.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:27 smithi120 ceph-mon[34770]: purged_snaps scrub starts 2024-09-10T09:51:28.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:27 smithi120 ceph-mon[34770]: purged_snaps scrub ok 2024-09-10T09:51:28.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:27 smithi120 ceph-mon[34770]: Detected new or changed devices on smithi134 2024-09-10T09:51:28.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:27 smithi120 ceph-mon[34770]: Adjusting osd_memory_target on smithi134 to 4272M 2024-09-10T09:51:28.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:27 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T09:51:28.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:27 smithi120 ceph-mon[34770]: osd.11 [v2:172.21.15.134:6824/1602213143,v1:172.21.15.134:6825/1602213143] boot 2024-09-10T09:51:28.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:27 smithi120 ceph-mon[34770]: osdmap e78: 12 total, 12 up, 12 in 2024-09-10T09:51:28.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:27 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T09:51:28.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:27 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/236312220' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T09:51:28.470 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:51:28.911 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:28 smithi077 ceph-mon[30319]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-10T09:51:28.911 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:28 smithi077 ceph-mon[30319]: osdmap e79: 12 total, 12 up, 12 in 2024-09-10T09:51:29.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:29.018+0000 7fd1c3000640 1 -- 172.21.15.77:0/3829180454 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd1bc101de0 msgr2=0x7fd1bc1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:29.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:29.018+0000 7fd1c3000640 1 --2- 172.21.15.77:0/3829180454 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd1bc101de0 0x7fd1bc1021c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac009900 tx=0x7fd1ac02a410 comp rx=0 tx=0).stop 2024-09-10T09:51:29.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:29.018+0000 7fd1c3000640 1 -- 172.21.15.77:0/3829180454 shutdown_connections 2024-09-10T09:51:29.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:29.019+0000 7fd1c3000640 1 --2- 172.21.15.77:0/3829180454 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd1bc106f10 0x7fd1bc10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:29.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:29.019+0000 7fd1c3000640 1 --2- 172.21.15.77:0/3829180454 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd1bc102790 0x7fd1bc1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:29.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:29.019+0000 7fd1c3000640 1 --2- 172.21.15.77:0/3829180454 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd1bc101de0 0x7fd1bc1021c0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:29.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:29.019+0000 7fd1c3000640 1 -- 172.21.15.77:0/3829180454 >> 172.21.15.77:0/3829180454 conn(0x7fd1bc0fd8c0 msgr2=0x7fd1bc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:29.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:29.019+0000 7fd1c3000640 1 -- 172.21.15.77:0/3829180454 shutdown_connections 2024-09-10T09:51:29.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:29.019+0000 7fd1c3000640 1 -- 172.21.15.77:0/3829180454 wait complete. 2024-09-10T09:51:29.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:29.020+0000 7fd1c3000640 1 Processor -- start 2024-09-10T09:51:29.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:29.020+0000 7fd1c3000640 1 -- start start 2024-09-10T09:51:29.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:29.020+0000 7fd1c3000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd1bc101de0 0x7fd1bc198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:29.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:29.020+0000 7fd1c3000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd1bc102790 0x7fd1bc198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:29.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:29.020+0000 7fd1c3000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd1bc106f10 0x7fd1bc19cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:29.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:29.020+0000 7fd1c3000640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd1bc111f70 con 0x7fd1bc102790 2024-09-10T09:51:29.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:29.020+0000 7fd1c3000640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd1bc111df0 con 0x7fd1bc106f10 2024-09-10T09:51:29.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:29.020+0000 7fd1c3000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd1bc1120f0 con 0x7fd1bc101de0 2024-09-10T09:51:29.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:29.020+0000 7fd1b3400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd1bc102790 0x7fd1bc198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:29.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:29.020+0000 7fd1b3400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd1bc102790 0x7fd1bc198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34142/0 (socket says 172.21.15.77:34142) 2024-09-10T09:51:29.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:29.020+0000 7fd1b3400640 1 -- 172.21.15.77:0/282480837 learned_addr learned my addr 172.21.15.77:0/282480837 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:51:29.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:29.021+0000 7fd1bbe00640 1 --2- 172.21.15.77:0/282480837 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd1bc101de0 0x7fd1bc198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:29.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:29.021+0000 7fd1c1000640 1 --2- 172.21.15.77:0/282480837 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd1bc106f10 0x7fd1bc19cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:29.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:29.021+0000 7fd1b3400640 1 -- 172.21.15.77:0/282480837 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd1bc101de0 msgr2=0x7fd1bc198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:29.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:29.021+0000 7fd1b3400640 1 --2- 172.21.15.77:0/282480837 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd1bc101de0 0x7fd1bc198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:29.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:29.021+0000 7fd1b3400640 1 -- 172.21.15.77:0/282480837 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd1bc106f10 msgr2=0x7fd1bc19cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:29.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:29.021+0000 7fd1b3400640 1 --2- 172.21.15.77:0/282480837 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd1bc106f10 0x7fd1bc19cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:29.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:29.021+0000 7fd1b3400640 1 -- 172.21.15.77:0/282480837 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1bc19d3c0 con 0x7fd1bc102790 2024-09-10T09:51:29.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:29.021+0000 7fd1bbe00640 1 --2- 172.21.15.77:0/282480837 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd1bc101de0 0x7fd1bc198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:51:29.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:29.021+0000 7fd1b3400640 1 --2- 172.21.15.77:0/282480837 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd1bc102790 0x7fd1bc198950 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd1a400e9f0 tx=0x7fd1a400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:29.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:29.021+0000 7fd1b9600640 1 -- 172.21.15.77:0/282480837 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a400cd80 con 0x7fd1bc102790 2024-09-10T09:51:29.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:29.021+0000 7fd1b9600640 1 -- 172.21.15.77:0/282480837 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1a4005500 con 0x7fd1bc102790 2024-09-10T09:51:29.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:29.021+0000 7fd1c3000640 1 -- 172.21.15.77:0/282480837 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1bc19d6b0 con 0x7fd1bc102790 2024-09-10T09:51:29.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:29.021+0000 7fd1b9600640 1 -- 172.21.15.77:0/282480837 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a4010690 con 0x7fd1bc102790 2024-09-10T09:51:29.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:29.021+0000 7fd1c3000640 1 -- 172.21.15.77:0/282480837 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1bc1a4f90 con 0x7fd1bc102790 2024-09-10T09:51:29.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:29.022+0000 7fd1c3000640 1 -- 172.21.15.77:0/282480837 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1bc1021c0 con 0x7fd1bc102790 2024-09-10T09:51:29.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:29.024+0000 7fd1b9600640 1 -- 172.21.15.77:0/282480837 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd1a40046c0 con 0x7fd1bc102790 2024-09-10T09:51:29.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:29.026+0000 7fd1b9600640 1 --2- 172.21.15.77:0/282480837 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fd1a00a2b80 0x7fd1a00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:29.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:29.026+0000 7fd1b9600640 1 -- 172.21.15.77:0/282480837 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7fd1a40cf300 con 0x7fd1bc102790 2024-09-10T09:51:29.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:29.026+0000 7fd1bbe00640 1 --2- 172.21.15.77:0/282480837 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fd1a00a2b80 0x7fd1a00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:29.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:29.027+0000 7fd1bbe00640 1 --2- 172.21.15.77:0/282480837 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fd1a00a2b80 0x7fd1a00a5040 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac005ab0 tx=0x7fd1ac005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:29.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:29.029+0000 7fd1b9600640 1 -- 172.21.15.77:0/282480837 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd1a40541b0 con 0x7fd1bc102790 2024-09-10T09:51:29.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:28 smithi134 ceph-mon[35674]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-10T09:51:29.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:28 smithi134 ceph-mon[35674]: osdmap e79: 12 total, 12 up, 12 in 2024-09-10T09:51:29.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:28 smithi120 ceph-mon[34770]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-10T09:51:29.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:28 smithi120 ceph-mon[34770]: osdmap e79: 12 total, 12 up, 12 in 2024-09-10T09:51:29.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:29.205+0000 7fd1c3000640 1 -- 172.21.15.77:0/282480837 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd1bc109b60 con 0x7fd1bc102790 2024-09-10T09:51:29.206 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:29.206+0000 7fd1b9600640 1 -- 172.21.15.77:0/282480837 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+18492 (secure 0 0 0) 0x7fd1a409c660 con 0x7fd1bc102790 2024-09-10T09:51:29.207 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:51:29.207 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":79,"fsid":"14a55226-6f59-11ef-bcea-c7b262605968","created":"2024-09-10T09:44:07.359542+0000","modified":"2024-09-10T09:51:27.738927+0000","last_up_change":"2024-09-10T09:51:26.731971+0000","last_in_change":"2024-09-10T09:51:03.857873+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:47:24.174660+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8d46f634-114c-4926-afdc-f8fa37e40336","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":1509649079},{"type":"v1","addr":"172.21.15.77:6803","nonce":1509649079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":1509649079},{"type":"v1","addr":"172.21.15.77:6805","nonce":1509649079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":1509649079},{"type":"v1","addr":"172.21.15.77:6809","nonce":1509649079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":1509649079},{"type":"v1","addr":"172.21.15.77:6807","nonce":1509649079}]},"public_addr":"172.21.15.77:6803/1509649079","cluster_addr":"172.21.15.77:6805/1509649079","heartbeat_back_addr":"172.21.15.77:6809/1509649079","heartbeat_front_addr":"172.21.15.77:6807/1509649079","state":["exists","up"]},{"osd":1,"uuid":"b6570764-736a-49e4-877a-1965e5812d1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":4243920698},{"type":"v1","addr":"172.21.15.120:6801","nonce":4243920698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":4243920698},{"type":"v1","addr":"172.21.15.120:6803","nonce":4243920698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":4243920698},{"type":"v1","addr":"172.21.15.120:6807","nonce":4243920698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":4243920698},{"type":"v1","addr":"172.21.15.120:6805","nonce":4243920698}]},"public_addr":"172.21.15.120:6801/4243920698","cluster_addr":"172.21.15.120:6803/4243920698","heartbeat_back_addr":"172.21.15.120:6807/4243920698","heartbeat_front_addr":"172.21.15.120:6805/4243920698","state":["exists","up"]},{"osd":2,"uuid":"5e83fe13-d1ee-483e-a575-e8041ec836f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":3280466823},{"type":"v1","addr":"172.21.15.134:6801","nonce":3280466823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":3280466823},{"type":"v1","addr":"172.21.15.134:6803","nonce":3280466823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":3280466823},{"type":"v1","addr":"172.21.15.134:6807","nonce":3280466823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":3280466823},{"type":"v1","addr":"172.21.15.134:6805","nonce":3280466823}]},"public_addr":"172.21.15.134:6801/3280466823","cluster_addr":"172.21.15.134:6803/3280466823","heartbeat_back_addr":"172.21.15.134:6807/3280466823","heartbeat_front_addr":"172.21.15.134:6805/3280466823","state":["exists","up"]},{"osd":3,"uuid":"5affc6a7-1cae-48fe-b7db-befc2fcafecb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":1629416786},{"type":"v1","addr":"172.21.15.77:6811","nonce":1629416786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":1629416786},{"type":"v1","addr":"172.21.15.77:6813","nonce":1629416786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":1629416786},{"type":"v1","addr":"172.21.15.77:6817","nonce":1629416786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":1629416786},{"type":"v1","addr":"172.21.15.77:6815","nonce":1629416786}]},"public_addr":"172.21.15.77:6811/1629416786","cluster_addr":"172.21.15.77:6813/1629416786","heartbeat_back_addr":"172.21.15.77:6817/1629416786","heartbeat_front_addr":"172.21.15.77:6815/1629416786","state":["exists","up"]},{"osd":4,"uuid":"c826978f-40b0-43ae-adbb-5c93b32a68ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":3162658038},{"type":"v1","addr":"172.21.15.120:6809","nonce":3162658038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":3162658038},{"type":"v1","addr":"172.21.15.120:6811","nonce":3162658038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":3162658038},{"type":"v1","addr":"172.21.15.120:6815","nonce":3162658038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":3162658038},{"type":"v1","addr":"172.21.15.120:6813","nonce":3162658038}]},"public_addr":"172.21.15.120:6809/3162658038","cluster_addr":"172.21.15.120:6811/3162658038","heartbeat_back_addr":"172.21.15.120:6815/3162658038","heartbeat_front_addr":"172.21.15.120:6813/3162658038","state":["exists","up"]},{"osd":5,"uuid":"be51c7fb-2765-46a3-9861-c950f63de148","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":4081444621},{"type":"v1","addr":"172.21.15.134:6809","nonce":4081444621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":4081444621},{"type":"v1","addr":"172.21.15.134:6811","nonce":4081444621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":4081444621},{"type":"v1","addr":"172.21.15.134:6815","nonce":4081444621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":4081444621},{"type":"v1","addr":"172.21.15.134:6813","nonce":4081444621}]},"public_addr":"172.21.15.134:6809/4081444621","cluster_addr":"172.21.15.134:6811/4081444621","heartbeat_back_addr":"172.21.15.134:6815/4081444621","heartbeat_front_addr":"172.21.15.134:6813/4081444621","state":["exists","up"]},{"osd":6,"uuid":"4eb403ab-473b-41cc-803d-18c908f768a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":879565639},{"type":"v1","addr":"172.21.15.77:6819","nonce":879565639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":879565639},{"type":"v1","addr":"172.21.15.77:6821","nonce":879565639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":879565639},{"type":"v1","addr":"172.21.15.77:6825","nonce":879565639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":879565639},{"type":"v1","addr":"172.21.15.77:6823","nonce":879565639}]},"public_addr":"172.21.15.77:6819/879565639","cluster_addr":"172.21.15.77:6821/879565639","heartbeat_back_addr":"172.21.15.77:6825/879565639","heartbeat_front_addr":"172.21.15.77:6823/879565639","state":["exists","up"]},{"osd":7,"uuid":"ba982bd0-c236-456c-9889-61087d79c1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":2688667400},{"type":"v1","addr":"172.21.15.120:6817","nonce":2688667400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":2688667400},{"type":"v1","addr":"172.21.15.120:6819","nonce":2688667400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":2688667400},{"type":"v1","addr":"172.21.15.120:6823","nonce":2688667400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2688667400},{"type":"v1","addr":"172.21.15.120:6821","nonce":2688667400}]},"public_addr":"172.21.15.120:6817/2688667400","cluster_addr":"172.21.15.120:6819/2688667400","heartbeat_back_addr":"172.21.15.120:6823/2688667400","heartbeat_front_addr":"172.21.15.120:6821/2688667400","state":["exists","up"]},{"osd":8,"uuid":"f2e8ca43-57c9-4933-b5f7-73f4d1b3e08b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2081686544},{"type":"v1","addr":"172.21.15.134:6817","nonce":2081686544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2081686544},{"type":"v1","addr":"172.21.15.134:6819","nonce":2081686544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2081686544},{"type":"v1","addr":"172.21.15.134:6823","nonce":2081686544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2081686544},{"type":"v1","addr":"172.21.15.134:6821","nonce":2081686544}]},"public_addr":"172.21.15.134:6817/2081686544","cluster_addr":"172.21.15.134:6819/2081686544","heartbeat_back_addr":"172.21.15.134:6823/2081686544","heartbeat_front_addr":"172.21.15.134:6821/2081686544","state":["exists","up"]},{"osd":9,"uuid":"9486f8b5-0614-4208-87ec-117741f6b622","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":1210619387},{"type":"v1","addr":"172.21.15.77:6827","nonce":1210619387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":1210619387},{"type":"v1","addr":"172.21.15.77:6829","nonce":1210619387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":1210619387},{"type":"v1","addr":"172.21.15.77:6833","nonce":1210619387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":1210619387},{"type":"v1","addr":"172.21.15.77:6831","nonce":1210619387}]},"public_addr":"172.21.15.77:6827/1210619387","cluster_addr":"172.21.15.77:6829/1210619387","heartbeat_back_addr":"172.21.15.77:6833/1210619387","heartbeat_front_addr":"172.21.15.77:6831/1210619387","state":["exists","up"]},{"osd":10,"uuid":"8e8192b0-1398-4d4d-9756-762b2f874d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":754733885},{"type":"v1","addr":"172.21.15.120:6825","nonce":754733885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":754733885},{"type":"v1","addr":"172.21.15.120:6827","nonce":754733885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":754733885},{"type":"v1","addr":"172.21.15.120:6831","nonce":754733885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":754733885},{"type":"v1","addr":"172.21.15.120:6829","nonce":754733885}]},"public_addr":"172.21.15.120:6825/754733885","cluster_addr":"172.21.15.120:6827/754733885","heartbeat_back_addr":"172.21.15.120:6831/754733885","heartbeat_front_addr":"172.21.15.120:6829/754733885","state":["exists","up"]},{"osd":11,"uuid":"85c1ccd4-5147-4995-896e-fe977f73f749","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1602213143},{"type":"v1","addr":"172.21.15.134:6825","nonce":1602213143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1602213143},{"type":"v1","addr":"172.21.15.134:6827","nonce":1602213143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1602213143},{"type":"v1","addr":"172.21.15.134:6831","nonce":1602213143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1602213143},{"type":"v1","addr":"172.21.15.134:6829","nonce":1602213143}]},"public_addr":"172.21.15.134:6825/1602213143","cluster_addr":"172.21.15.134:6827/1602213143","heartbeat_back_addr":"172.21.15.134:6831/1602213143","heartbeat_front_addr":"172.21.15.134:6829/1602213143","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:46:30.293451+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:46:56.554801+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:47:21.476225+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:47:49.737332+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:48:16.092181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:48:41.240815+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:49:11.683950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:49:38.859255+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:50:04.105401+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:50:33.847810+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:50:59.901797+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:51:25.241507+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.77:0/1697982655":"2024-09-11T09:44:59.575663+0000","172.21.15.77:6801/1527070442":"2024-09-11T09:44:59.575663+0000","172.21.15.77:0/3146688196":"2024-09-11T09:44:32.185342+0000","172.21.15.77:0/371773177":"2024-09-11T09:44:59.575663+0000","172.21.15.77:0/1925456910":"2024-09-11T09:44:59.575663+0000","172.21.15.77:0/138545832":"2024-09-11T09:44:32.185342+0000","172.21.15.77:6801/3235900503":"2024-09-11T09:44:32.185342+0000","172.21.15.77:6800/1527070442":"2024-09-11T09:44:59.575663+0000","172.21.15.77:6800/3235900503":"2024-09-11T09:44:32.185342+0000","172.21.15.77:0/1647092548":"2024-09-11T09:44:32.185342+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T09:51:29.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:29.208+0000 7fd1c3000640 1 -- 172.21.15.77:0/282480837 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fd1a00a2b80 msgr2=0x7fd1a00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:29.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:29.208+0000 7fd1c3000640 1 --2- 172.21.15.77:0/282480837 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fd1a00a2b80 0x7fd1a00a5040 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac005ab0 tx=0x7fd1ac005a20 comp rx=0 tx=0).stop 2024-09-10T09:51:29.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:29.208+0000 7fd1c3000640 1 -- 172.21.15.77:0/282480837 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd1bc102790 msgr2=0x7fd1bc198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:29.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:29.208+0000 7fd1c3000640 1 --2- 172.21.15.77:0/282480837 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd1bc102790 0x7fd1bc198950 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd1a400e9f0 tx=0x7fd1a400edf0 comp rx=0 tx=0).stop 2024-09-10T09:51:29.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:29.208+0000 7fd1c3000640 1 -- 172.21.15.77:0/282480837 shutdown_connections 2024-09-10T09:51:29.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:29.208+0000 7fd1c3000640 1 --2- 172.21.15.77:0/282480837 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fd1a00a2b80 0x7fd1a00a5040 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:29.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:29.208+0000 7fd1c3000640 1 --2- 172.21.15.77:0/282480837 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd1bc106f10 0x7fd1bc19cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:29.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:29.208+0000 7fd1c3000640 1 --2- 172.21.15.77:0/282480837 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd1bc102790 0x7fd1bc198950 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:29.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:29.208+0000 7fd1c3000640 1 --2- 172.21.15.77:0/282480837 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd1bc101de0 0x7fd1bc198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:29.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:29.208+0000 7fd1c3000640 1 -- 172.21.15.77:0/282480837 >> 172.21.15.77:0/282480837 conn(0x7fd1bc0fd8c0 msgr2=0x7fd1bc0fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:29.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:29.208+0000 7fd1c3000640 1 -- 172.21.15.77:0/282480837 shutdown_connections 2024-09-10T09:51:29.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:29.208+0000 7fd1c3000640 1 -- 172.21.15.77:0/282480837 wait complete. 2024-09-10T09:51:30.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:29 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/282480837' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T09:51:30.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:29 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/282480837' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T09:51:30.175 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:29 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/282480837' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T09:51:30.215 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T09:47:24.174660+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T09:51:30.216 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-10T09:51:30.576 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:51:31.004 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:30 smithi077 ceph-mon[30319]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:51:31.117 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:31.116+0000 7fef78400640 1 -- 172.21.15.77:0/179256804 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fef70109140 msgr2=0x7fef70110cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:31.117 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:31.116+0000 7fef78400640 1 --2- 172.21.15.77:0/179256804 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fef70109140 0x7fef70110cc0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fef68009ca0 tx=0x7fef6802a8c0 comp rx=0 tx=0).stop 2024-09-10T09:51:31.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:31.116+0000 7fef78400640 1 -- 172.21.15.77:0/179256804 shutdown_connections 2024-09-10T09:51:31.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:31.116+0000 7fef78400640 1 --2- 172.21.15.77:0/179256804 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fef70109140 0x7fef70110cc0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:31.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:31.116+0000 7fef78400640 1 --2- 172.21.15.77:0/179256804 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef701049c0 0x7fef70108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:31.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:31.116+0000 7fef78400640 1 --2- 172.21.15.77:0/179256804 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fef70104010 0x7fef701043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:31.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:31.116+0000 7fef78400640 1 -- 172.21.15.77:0/179256804 >> 172.21.15.77:0/179256804 conn(0x7fef700ffa50 msgr2=0x7fef70101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:31.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:31.116+0000 7fef78400640 1 -- 172.21.15.77:0/179256804 shutdown_connections 2024-09-10T09:51:31.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:31.116+0000 7fef78400640 1 -- 172.21.15.77:0/179256804 wait complete. 2024-09-10T09:51:31.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:31.117+0000 7fef78400640 1 Processor -- start 2024-09-10T09:51:31.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:31.117+0000 7fef78400640 1 -- start start 2024-09-10T09:51:31.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:31.117+0000 7fef78400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fef70104010 0x7fef7019a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:31.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:31.117+0000 7fef78400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef701049c0 0x7fef7019abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:31.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:31.118+0000 7fef78400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fef70109140 0x7fef7019ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:31.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:31.118+0000 7fef78400640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fef701141a0 con 0x7fef70104010 2024-09-10T09:51:31.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:31.118+0000 7fef78400640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fef70114020 con 0x7fef70109140 2024-09-10T09:51:31.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:31.118+0000 7fef78400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fef70114320 con 0x7fef701049c0 2024-09-10T09:51:31.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:31.118+0000 7fef75a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fef70104010 0x7fef7019a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:31.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:31.118+0000 7fef75a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fef70104010 0x7fef7019a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34160/0 (socket says 172.21.15.77:34160) 2024-09-10T09:51:31.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:31.118+0000 7fef75a00640 1 -- 172.21.15.77:0/3122282602 learned_addr learned my addr 172.21.15.77:0/3122282602 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:51:31.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:31.118+0000 7fef75000640 1 --2- 172.21.15.77:0/3122282602 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef701049c0 0x7fef7019abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:31.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:31.118+0000 7fef76400640 1 --2- 172.21.15.77:0/3122282602 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fef70109140 0x7fef7019ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:31.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:31.118+0000 7fef75a00640 1 -- 172.21.15.77:0/3122282602 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef701049c0 msgr2=0x7fef7019abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:31.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:31.118+0000 7fef75a00640 1 --2- 172.21.15.77:0/3122282602 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef701049c0 0x7fef7019abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:31.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:31.118+0000 7fef75a00640 1 -- 172.21.15.77:0/3122282602 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fef70109140 msgr2=0x7fef7019ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:31.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:31.118+0000 7fef75a00640 1 --2- 172.21.15.77:0/3122282602 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fef70109140 0x7fef7019ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:31.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:31.118+0000 7fef75a00640 1 -- 172.21.15.77:0/3122282602 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef7019f650 con 0x7fef70104010 2024-09-10T09:51:31.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:31.119+0000 7fef75a00640 1 --2- 172.21.15.77:0/3122282602 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fef70104010 0x7fef7019a6a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fef6400e9f0 tx=0x7fef6400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:31.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:31.119+0000 7fef6e000640 1 -- 172.21.15.77:0/3122282602 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef6400cd80 con 0x7fef70104010 2024-09-10T09:51:31.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:31.119+0000 7fef6e000640 1 -- 172.21.15.77:0/3122282602 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef64010910 con 0x7fef70104010 2024-09-10T09:51:31.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:31.119+0000 7fef78400640 1 -- 172.21.15.77:0/3122282602 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef7019f940 con 0x7fef70104010 2024-09-10T09:51:31.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:31.119+0000 7fef6e000640 1 -- 172.21.15.77:0/3122282602 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef64018990 con 0x7fef70104010 2024-09-10T09:51:31.122 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:31.119+0000 7fef78400640 1 -- 172.21.15.77:0/3122282602 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef701a7220 con 0x7fef70104010 2024-09-10T09:51:31.122 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:31.120+0000 7fef78400640 1 -- 172.21.15.77:0/3122282602 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef701043f0 con 0x7fef70104010 2024-09-10T09:51:31.123 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:31.122+0000 7fef6e000640 1 -- 172.21.15.77:0/3122282602 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fef64010ac0 con 0x7fef70104010 2024-09-10T09:51:31.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:31.123+0000 7fef6e000640 1 --2- 172.21.15.77:0/3122282602 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fef580a2c00 0x7fef580a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:31.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:31.124+0000 7fef6e000640 1 -- 172.21.15.77:0/3122282602 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7fef640ceeb0 con 0x7fef70104010 2024-09-10T09:51:31.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:31.124+0000 7fef75000640 1 --2- 172.21.15.77:0/3122282602 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fef580a2c00 0x7fef580a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:31.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:31.124+0000 7fef75000640 1 --2- 172.21.15.77:0/3122282602 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fef580a2c00 0x7fef580a50c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fef7010ab10 tx=0x7fef5c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:31.128 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:31.127+0000 7fef6e000640 1 -- 172.21.15.77:0/3122282602 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fef6409bbc0 con 0x7fef70104010 2024-09-10T09:51:31.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:30 smithi134 ceph-mon[35674]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:51:31.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:30 smithi120 ceph-mon[34770]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:51:31.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:31.300+0000 7fef78400640 1 -- 172.21.15.77:0/3122282602 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fef7010bdf0 con 0x7fef70104010 2024-09-10T09:51:31.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:31.301+0000 7fef6e000640 1 -- 172.21.15.77:0/3122282602 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) ==== 93+0+10 (secure 0 0 0) 0x7fef6409fec0 con 0x7fef70104010 2024-09-10T09:51:31.302 INFO:teuthology.orchestra.run.smithi077.stdout:pg_num: 1 2024-09-10T09:51:31.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:31.303+0000 7fef78400640 1 -- 172.21.15.77:0/3122282602 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fef580a2c00 msgr2=0x7fef580a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:31.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:31.303+0000 7fef78400640 1 --2- 172.21.15.77:0/3122282602 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fef580a2c00 0x7fef580a50c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fef7010ab10 tx=0x7fef5c00b390 comp rx=0 tx=0).stop 2024-09-10T09:51:31.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:31.303+0000 7fef78400640 1 -- 172.21.15.77:0/3122282602 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fef70104010 msgr2=0x7fef7019a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:31.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:31.303+0000 7fef78400640 1 --2- 172.21.15.77:0/3122282602 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fef70104010 0x7fef7019a6a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fef6400e9f0 tx=0x7fef6400eeb0 comp rx=0 tx=0).stop 2024-09-10T09:51:31.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:31.303+0000 7fef78400640 1 -- 172.21.15.77:0/3122282602 shutdown_connections 2024-09-10T09:51:31.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:31.303+0000 7fef78400640 1 --2- 172.21.15.77:0/3122282602 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fef580a2c00 0x7fef580a50c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:31.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:31.303+0000 7fef78400640 1 --2- 172.21.15.77:0/3122282602 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fef70109140 0x7fef7019ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:31.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:31.303+0000 7fef78400640 1 --2- 172.21.15.77:0/3122282602 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef701049c0 0x7fef7019abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:31.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:31.303+0000 7fef78400640 1 --2- 172.21.15.77:0/3122282602 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fef70104010 0x7fef7019a6a0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:31.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:31.303+0000 7fef78400640 1 -- 172.21.15.77:0/3122282602 >> 172.21.15.77:0/3122282602 conn(0x7fef700ffa50 msgr2=0x7fef70100db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:31.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:31.304+0000 7fef78400640 1 -- 172.21.15.77:0/3122282602 shutdown_connections 2024-09-10T09:51:31.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:31.304+0000 7fef78400640 1 -- 172.21.15.77:0/3122282602 wait complete. 2024-09-10T09:51:31.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:31 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/3122282602' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T09:51:32.038 INFO:tasks.cephadm:Adding mds.a on smithi077 2024-09-10T09:51:32.038 INFO:tasks.cephadm:Adding mds.d on smithi077 2024-09-10T09:51:32.038 INFO:tasks.cephadm:Adding mds.g on smithi077 2024-09-10T09:51:32.038 INFO:tasks.cephadm:Adding mds.j on smithi077 2024-09-10T09:51:32.038 INFO:tasks.cephadm:Adding mds.b on smithi120 2024-09-10T09:51:32.038 INFO:tasks.cephadm:Adding mds.e on smithi120 2024-09-10T09:51:32.038 INFO:tasks.cephadm:Adding mds.h on smithi120 2024-09-10T09:51:32.039 INFO:tasks.cephadm:Adding mds.k on smithi120 2024-09-10T09:51:32.039 INFO:tasks.cephadm:Adding mds.c on smithi134 2024-09-10T09:51:32.039 INFO:tasks.cephadm:Adding mds.f on smithi134 2024-09-10T09:51:32.039 INFO:tasks.cephadm:Adding mds.i on smithi134 2024-09-10T09:51:32.039 INFO:tasks.cephadm:Adding mds.l on smithi134 2024-09-10T09:51:32.039 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph orch apply mds all '12;smithi077=a;smithi077=d;smithi077=g;smithi077=j;smithi120=b;smithi120=e;smithi120=h;smithi120=k;smithi134=c;smithi134=f;smithi134=i;smithi134=l' 2024-09-10T09:51:32.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:31 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/3122282602' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T09:51:32.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:31 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/3122282602' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T09:51:32.299 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T09:51:32.858 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:32.834+0000 7f60e59fa640 1 -- 172.21.15.134:0/2985771766 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60e01026e0 msgr2=0x7f60e0104ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:32.858 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:32.834+0000 7f60e59fa640 1 --2- 172.21.15.134:0/2985771766 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60e01026e0 0x7f60e0104ad0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f60d4009900 tx=0x7f60d402a410 comp rx=0 tx=0).stop 2024-09-10T09:51:32.858 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:32.834+0000 7f60e59fa640 1 -- 172.21.15.134:0/2985771766 shutdown_connections 2024-09-10T09:51:32.858 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:32.834+0000 7f60e59fa640 1 --2- 172.21.15.134:0/2985771766 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f60e0107940 0x7f60e0109d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:32.859 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:32.834+0000 7f60e59fa640 1 --2- 172.21.15.134:0/2985771766 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f60e0105010 0x7f60e0107400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:32.859 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:32.834+0000 7f60e59fa640 1 --2- 172.21.15.134:0/2985771766 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60e01026e0 0x7f60e0104ad0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:32.859 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:32.834+0000 7f60e59fa640 1 -- 172.21.15.134:0/2985771766 >> 172.21.15.134:0/2985771766 conn(0x7f60e00fc280 msgr2=0x7f60e00fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:32.859 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:32.834+0000 7f60e59fa640 1 -- 172.21.15.134:0/2985771766 shutdown_connections 2024-09-10T09:51:32.859 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:32.834+0000 7f60e59fa640 1 -- 172.21.15.134:0/2985771766 wait complete. 2024-09-10T09:51:32.859 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:32.834+0000 7f60e59fa640 1 Processor -- start 2024-09-10T09:51:32.859 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:32.835+0000 7f60e59fa640 1 -- start start 2024-09-10T09:51:32.859 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:32.835+0000 7f60e59fa640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60e01026e0 0x7f60e0102040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:32.859 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:32.835+0000 7f60e59fa640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f60e0105010 0x7f60e0100690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:32.859 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:32.835+0000 7f60e59fa640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f60e0107940 0x7f60e0100bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:32.859 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:32.835+0000 7f60e59fa640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f60e010d860 con 0x7f60e0107940 2024-09-10T09:51:32.859 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:32.835+0000 7f60e59fa640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f60e010d6e0 con 0x7f60e0105010 2024-09-10T09:51:32.859 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:32.835+0000 7f60e59fa640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f60e010d9e0 con 0x7f60e01026e0 2024-09-10T09:51:32.860 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:32.835+0000 7f60deffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60e01026e0 0x7f60e0102040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:32.860 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:32.835+0000 7f60deffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60e01026e0 0x7f60e0102040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:34908/0 (socket says 172.21.15.134:34908) 2024-09-10T09:51:32.860 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:32.835+0000 7f60deffd640 1 -- 172.21.15.134:0/4209124005 learned_addr learned my addr 172.21.15.134:0/4209124005 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-10T09:51:32.860 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:32.835+0000 7f60de7fc640 1 --2- 172.21.15.134:0/4209124005 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f60e0105010 0x7f60e0100690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:32.860 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:32.835+0000 7f60deffd640 1 -- 172.21.15.134:0/4209124005 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f60e0105010 msgr2=0x7f60e0100690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:32.860 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:32.835+0000 7f60deffd640 1 --2- 172.21.15.134:0/4209124005 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f60e0105010 0x7f60e0100690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:32.860 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:32.835+0000 7f60deffd640 1 -- 172.21.15.134:0/4209124005 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f60e0107940 msgr2=0x7f60e0100bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:32.860 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:32.835+0000 7f60df7fe640 1 --2- 172.21.15.134:0/4209124005 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f60e0107940 0x7f60e0100bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:32.860 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:32.835+0000 7f60deffd640 1 --2- 172.21.15.134:0/4209124005 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f60e0107940 0x7f60e0100bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:32.860 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:32.835+0000 7f60deffd640 1 -- 172.21.15.134:0/4209124005 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60e0101490 con 0x7f60e01026e0 2024-09-10T09:51:32.860 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:32.835+0000 7f60df7fe640 1 --2- 172.21.15.134:0/4209124005 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f60e0107940 0x7f60e0100bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:51:32.860 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:32.835+0000 7f60deffd640 1 --2- 172.21.15.134:0/4209124005 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60e01026e0 0x7f60e0102040 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f60d4009230 tx=0x7f60d4004430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:32.861 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:32.836+0000 7f60e49f8640 1 -- 172.21.15.134:0/4209124005 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d403f030 con 0x7f60e01026e0 2024-09-10T09:51:32.861 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:32.836+0000 7f60e59fa640 1 -- 172.21.15.134:0/4209124005 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60e01a1fe0 con 0x7f60e01026e0 2024-09-10T09:51:32.861 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:32.836+0000 7f60e49f8640 1 -- 172.21.15.134:0/4209124005 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60d4003ce0 con 0x7f60e01026e0 2024-09-10T09:51:32.861 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:32.836+0000 7f60e59fa640 1 -- 172.21.15.134:0/4209124005 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60e01a23d0 con 0x7f60e01026e0 2024-09-10T09:51:32.861 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:32.836+0000 7f60e49f8640 1 -- 172.21.15.134:0/4209124005 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d4049060 con 0x7f60e01026e0 2024-09-10T09:51:32.861 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:32.837+0000 7f60e59fa640 1 -- 172.21.15.134:0/4209124005 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60e01053f0 con 0x7f60e01026e0 2024-09-10T09:51:32.861 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:32.838+0000 7f60e49f8640 1 -- 172.21.15.134:0/4209124005 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f60d4042030 con 0x7f60e01026e0 2024-09-10T09:51:32.861 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:32.838+0000 7f60e49f8640 1 --2- 172.21.15.134:0/4209124005 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f60c80a2b80 0x7f60c80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:32.861 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:32.838+0000 7f60e49f8640 1 -- 172.21.15.134:0/4209124005 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7f60d40ef6e0 con 0x7f60e01026e0 2024-09-10T09:51:32.861 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:32.839+0000 7f60de7fc640 1 --2- 172.21.15.134:0/4209124005 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f60c80a2b80 0x7f60c80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:32.861 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:32.839+0000 7f60de7fc640 1 --2- 172.21.15.134:0/4209124005 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f60c80a2b80 0x7f60c80a5040 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f60d0005e10 tx=0x7f60d0005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:32.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:32.847+0000 7f60e49f8640 1 -- 172.21.15.134:0/4209124005 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f60d40bcae0 con 0x7f60e01026e0 2024-09-10T09:51:33.015 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:32 smithi134 ceph-mon[35674]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:51:33.028 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:33.026+0000 7f60e59fa640 1 -- 172.21.15.134:0/4209124005 --> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi077=a;smithi077=d;smithi077=g;smithi077=j;smithi120=b;smithi120=e;smithi120=h;smithi120=k;smithi134=c;smithi134=f;smithi134=i;smithi134=l", "target": ["mon-mgr", ""]}) -- 0x7f60e00628f0 con 0x7f60c80a2b80 2024-09-10T09:51:33.037 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:33.035+0000 7f60e49f8640 1 -- 172.21.15.134:0/4209124005 <== mgr.14150 v2:172.21.15.77:6800/4261324650 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f60e00628f0 con 0x7f60c80a2b80 2024-09-10T09:51:33.037 INFO:teuthology.orchestra.run.smithi134.stdout:Scheduled mds.all update... 2024-09-10T09:51:33.039 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:33.037+0000 7f60e59fa640 1 -- 172.21.15.134:0/4209124005 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f60c80a2b80 msgr2=0x7f60c80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:33.039 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:33.037+0000 7f60e59fa640 1 --2- 172.21.15.134:0/4209124005 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f60c80a2b80 0x7f60c80a5040 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f60d0005e10 tx=0x7f60d0005da0 comp rx=0 tx=0).stop 2024-09-10T09:51:33.040 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:33.037+0000 7f60e59fa640 1 -- 172.21.15.134:0/4209124005 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60e01026e0 msgr2=0x7f60e0102040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:33.040 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:33.037+0000 7f60e59fa640 1 --2- 172.21.15.134:0/4209124005 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60e01026e0 0x7f60e0102040 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f60d4009230 tx=0x7f60d4004430 comp rx=0 tx=0).stop 2024-09-10T09:51:33.040 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:33.037+0000 7f60e59fa640 1 -- 172.21.15.134:0/4209124005 shutdown_connections 2024-09-10T09:51:33.040 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:33.037+0000 7f60e59fa640 1 --2- 172.21.15.134:0/4209124005 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f60c80a2b80 0x7f60c80a5040 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:33.040 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:33.037+0000 7f60e59fa640 1 --2- 172.21.15.134:0/4209124005 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f60e0107940 0x7f60e0100bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:33.040 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:33.037+0000 7f60e59fa640 1 --2- 172.21.15.134:0/4209124005 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f60e0105010 0x7f60e0100690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:33.040 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:33.037+0000 7f60e59fa640 1 --2- 172.21.15.134:0/4209124005 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60e01026e0 0x7f60e0102040 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:33.040 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:33.037+0000 7f60e59fa640 1 -- 172.21.15.134:0/4209124005 >> 172.21.15.134:0/4209124005 conn(0x7f60e00fc280 msgr2=0x7f60e0103040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:33.040 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:33.037+0000 7f60e59fa640 1 -- 172.21.15.134:0/4209124005 shutdown_connections 2024-09-10T09:51:33.040 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T09:51:33.037+0000 7f60e59fa640 1 -- 172.21.15.134:0/4209124005 wait complete. 2024-09-10T09:51:33.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:32 smithi120 ceph-mon[34770]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:51:33.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:32 smithi077 ceph-mon[30319]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:51:33.743 DEBUG:teuthology.orchestra.run.smithi077:mds.a> sudo journalctl -f -n 0 -u ceph-14a55226-6f59-11ef-bcea-c7b262605968@mds.a.service 2024-09-10T09:51:33.748 DEBUG:teuthology.orchestra.run.smithi077:mds.d> sudo journalctl -f -n 0 -u ceph-14a55226-6f59-11ef-bcea-c7b262605968@mds.d.service 2024-09-10T09:51:33.792 DEBUG:teuthology.orchestra.run.smithi077:mds.g> sudo journalctl -f -n 0 -u ceph-14a55226-6f59-11ef-bcea-c7b262605968@mds.g.service 2024-09-10T09:51:33.838 DEBUG:teuthology.orchestra.run.smithi077:mds.j> sudo journalctl -f -n 0 -u ceph-14a55226-6f59-11ef-bcea-c7b262605968@mds.j.service 2024-09-10T09:51:33.884 DEBUG:teuthology.orchestra.run.smithi120:mds.b> sudo journalctl -f -n 0 -u ceph-14a55226-6f59-11ef-bcea-c7b262605968@mds.b.service 2024-09-10T09:51:33.887 DEBUG:teuthology.orchestra.run.smithi120:mds.e> sudo journalctl -f -n 0 -u ceph-14a55226-6f59-11ef-bcea-c7b262605968@mds.e.service 2024-09-10T09:51:33.929 DEBUG:teuthology.orchestra.run.smithi120:mds.h> sudo journalctl -f -n 0 -u ceph-14a55226-6f59-11ef-bcea-c7b262605968@mds.h.service 2024-09-10T09:51:33.972 DEBUG:teuthology.orchestra.run.smithi120:mds.k> sudo journalctl -f -n 0 -u ceph-14a55226-6f59-11ef-bcea-c7b262605968@mds.k.service 2024-09-10T09:51:34.015 DEBUG:teuthology.orchestra.run.smithi134:mds.c> sudo journalctl -f -n 0 -u ceph-14a55226-6f59-11ef-bcea-c7b262605968@mds.c.service 2024-09-10T09:51:34.018 DEBUG:teuthology.orchestra.run.smithi134:mds.f> sudo journalctl -f -n 0 -u ceph-14a55226-6f59-11ef-bcea-c7b262605968@mds.f.service 2024-09-10T09:51:34.061 DEBUG:teuthology.orchestra.run.smithi134:mds.i> sudo journalctl -f -n 0 -u ceph-14a55226-6f59-11ef-bcea-c7b262605968@mds.i.service 2024-09-10T09:51:34.104 DEBUG:teuthology.orchestra.run.smithi134:mds.l> sudo journalctl -f -n 0 -u ceph-14a55226-6f59-11ef-bcea-c7b262605968@mds.l.service 2024-09-10T09:51:34.148 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-09-10T09:51:34.149 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:51:34.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:34 smithi120 ceph-mon[34770]: from='client.24436 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi077=a;smithi077=d;smithi077=g;smithi077=j;smithi120=b;smithi120=e;smithi120=h;smithi120=k;smithi134=c;smithi134=f;smithi134=i;smithi134=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:51:34.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:34 smithi120 ceph-mon[34770]: Saving service mds.all spec with placement smithi077=a;smithi077=d;smithi077=g;smithi077=j;smithi120=b;smithi120=e;smithi120=h;smithi120=k;smithi134=c;smithi134=f;smithi134=i;smithi134=l;count:12 2024-09-10T09:51:34.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:34 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:34.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:34 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:51:34.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:34 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:51:34.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:34 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:51:34.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:34 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:34.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:34 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T09:51:34.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:34 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T09:51:34.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:34 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:51:34.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:34 smithi120 ceph-mon[34770]: Deploying daemon mds.l on smithi134 2024-09-10T09:51:34.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:34 smithi120 ceph-mon[34770]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:51:34.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:34 smithi077 ceph-mon[30319]: from='client.24436 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi077=a;smithi077=d;smithi077=g;smithi077=j;smithi120=b;smithi120=e;smithi120=h;smithi120=k;smithi134=c;smithi134=f;smithi134=i;smithi134=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:51:34.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:34 smithi077 ceph-mon[30319]: Saving service mds.all spec with placement smithi077=a;smithi077=d;smithi077=g;smithi077=j;smithi120=b;smithi120=e;smithi120=h;smithi120=k;smithi134=c;smithi134=f;smithi134=i;smithi134=l;count:12 2024-09-10T09:51:34.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:34 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:34.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:34 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:51:34.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:34 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:51:34.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:34 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:51:34.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:34 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:34.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:34 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T09:51:34.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:34 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T09:51:34.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:34 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:51:34.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:34 smithi077 ceph-mon[30319]: Deploying daemon mds.l on smithi134 2024-09-10T09:51:34.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:34 smithi077 ceph-mon[30319]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:51:34.309 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:34 smithi134 ceph-mon[35674]: from='client.24436 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi077=a;smithi077=d;smithi077=g;smithi077=j;smithi120=b;smithi120=e;smithi120=h;smithi120=k;smithi134=c;smithi134=f;smithi134=i;smithi134=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:51:34.309 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:34 smithi134 ceph-mon[35674]: Saving service mds.all spec with placement smithi077=a;smithi077=d;smithi077=g;smithi077=j;smithi120=b;smithi120=e;smithi120=h;smithi120=k;smithi134=c;smithi134=f;smithi134=i;smithi134=l;count:12 2024-09-10T09:51:34.309 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:34 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:34.309 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:34 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:51:34.309 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:34 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:51:34.309 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:34 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:51:34.309 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:34 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:34.309 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:34 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T09:51:34.309 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:34 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T09:51:34.309 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:34 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:51:34.309 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:34 smithi134 ceph-mon[35674]: Deploying daemon mds.l on smithi134 2024-09-10T09:51:34.310 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:34 smithi134 ceph-mon[35674]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:51:34.506 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:51:35.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:35.086+0000 7f60cba00640 1 -- 172.21.15.77:0/3944264181 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f60c4106f10 msgr2=0x7f60c410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:35.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:35.086+0000 7f60cba00640 1 --2- 172.21.15.77:0/3944264181 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f60c4106f10 0x7f60c410ea90 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f60bc009ca0 tx=0x7f60bc02a8c0 comp rx=0 tx=0).stop 2024-09-10T09:51:35.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:35.086+0000 7f60cba00640 1 -- 172.21.15.77:0/3944264181 shutdown_connections 2024-09-10T09:51:35.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:35.086+0000 7f60cba00640 1 --2- 172.21.15.77:0/3944264181 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f60c4106f10 0x7f60c410ea90 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:35.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:35.086+0000 7f60cba00640 1 --2- 172.21.15.77:0/3944264181 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f60c4102790 0x7f60c41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:35.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:35.086+0000 7f60cba00640 1 --2- 172.21.15.77:0/3944264181 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60c4101de0 0x7f60c41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:35.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:35.086+0000 7f60cba00640 1 -- 172.21.15.77:0/3944264181 >> 172.21.15.77:0/3944264181 conn(0x7f60c40fd8c0 msgr2=0x7f60c40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:35.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:35.086+0000 7f60cba00640 1 -- 172.21.15.77:0/3944264181 shutdown_connections 2024-09-10T09:51:35.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:35.086+0000 7f60cba00640 1 -- 172.21.15.77:0/3944264181 wait complete. 2024-09-10T09:51:35.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:35.087+0000 7f60cba00640 1 Processor -- start 2024-09-10T09:51:35.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:35.088+0000 7f60cba00640 1 -- start start 2024-09-10T09:51:35.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:35.088+0000 7f60cba00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f60c4101de0 0x7f60c419a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:35.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:35.088+0000 7f60cba00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f60c4102790 0x7f60c419ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:35.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:35.088+0000 7f60cba00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60c4106f10 0x7f60c419ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:35.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:35.088+0000 7f60cba00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f60c4111fc0 con 0x7f60c4101de0 2024-09-10T09:51:35.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:35.088+0000 7f60cba00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f60c4111e40 con 0x7f60c4102790 2024-09-10T09:51:35.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:35.088+0000 7f60cba00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f60c4112140 con 0x7f60c4106f10 2024-09-10T09:51:35.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:35.088+0000 7f60c9000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f60c4101de0 0x7f60c419a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:35.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:35.089+0000 7f60c9000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f60c4101de0 0x7f60c419a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56622/0 (socket says 172.21.15.77:56622) 2024-09-10T09:51:35.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:35.089+0000 7f60c9000640 1 -- 172.21.15.77:0/562621988 learned_addr learned my addr 172.21.15.77:0/562621988 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:51:35.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:35.089+0000 7f60c9a00640 1 --2- 172.21.15.77:0/562621988 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60c4106f10 0x7f60c419ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:35.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:35.089+0000 7f60c3e00640 1 --2- 172.21.15.77:0/562621988 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f60c4102790 0x7f60c419ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:35.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:35.089+0000 7f60c9000640 1 -- 172.21.15.77:0/562621988 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60c4106f10 msgr2=0x7f60c419ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:35.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:35.089+0000 7f60c9000640 1 --2- 172.21.15.77:0/562621988 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60c4106f10 0x7f60c419ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:35.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:35.089+0000 7f60c9000640 1 -- 172.21.15.77:0/562621988 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f60c4102790 msgr2=0x7f60c419ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:35.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:35.089+0000 7f60c9000640 1 --2- 172.21.15.77:0/562621988 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f60c4102790 0x7f60c419ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:35.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:35.089+0000 7f60c9000640 1 -- 172.21.15.77:0/562621988 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60c419f5e0 con 0x7f60c4101de0 2024-09-10T09:51:35.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:35.089+0000 7f60c9000640 1 --2- 172.21.15.77:0/562621988 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f60c4101de0 0x7f60c419a630 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f60b400d950 tx=0x7f60b400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:35.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:35.090+0000 7f60c1600640 1 -- 172.21.15.77:0/562621988 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60b4017070 con 0x7f60c4101de0 2024-09-10T09:51:35.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:35.090+0000 7f60c1600640 1 -- 172.21.15.77:0/562621988 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60b4009c30 con 0x7f60c4101de0 2024-09-10T09:51:35.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:35.090+0000 7f60c1600640 1 -- 172.21.15.77:0/562621988 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60b400f6b0 con 0x7f60c4101de0 2024-09-10T09:51:35.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:35.090+0000 7f60cba00640 1 -- 172.21.15.77:0/562621988 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60c419f8d0 con 0x7f60c4101de0 2024-09-10T09:51:35.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:35.090+0000 7f60cba00640 1 -- 172.21.15.77:0/562621988 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60c41a71b0 con 0x7f60c4101de0 2024-09-10T09:51:35.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:35.091+0000 7f60cba00640 1 -- 172.21.15.77:0/562621988 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60c41021c0 con 0x7f60c4101de0 2024-09-10T09:51:35.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:35.093+0000 7f60c1600640 1 -- 172.21.15.77:0/562621988 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f60b40109d0 con 0x7f60c4101de0 2024-09-10T09:51:35.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:35.094+0000 7f60c1600640 1 --2- 172.21.15.77:0/562621988 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f60a80a2c00 0x7f60a80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:35.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:35.094+0000 7f60c1600640 1 -- 172.21.15.77:0/562621988 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7f60b40ce780 con 0x7f60c4101de0 2024-09-10T09:51:35.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:35.094+0000 7f60c3e00640 1 --2- 172.21.15.77:0/562621988 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f60a80a2c00 0x7f60a80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:35.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:35.095+0000 7f60c3e00640 1 --2- 172.21.15.77:0/562621988 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f60a80a2c00 0x7f60a80a50c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f60c40fddc0 tx=0x7f60b0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:35.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:35.098+0000 7f60c1600640 1 -- 172.21.15.77:0/562621988 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f60b4009de0 con 0x7f60c4101de0 2024-09-10T09:51:35.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:35.307+0000 7f60cba00640 1 -- 172.21.15.77:0/562621988 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f60c41a73b0 con 0x7f60c4101de0 2024-09-10T09:51:35.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:35.308+0000 7f60c1600640 1 -- 172.21.15.77:0/562621988 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7f60b409b490 con 0x7f60c4101de0 2024-09-10T09:51:35.309 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:51:35.309 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":1,"btime":"2024-09-10T09:44:07:359254+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-09-10T09:51:35.309 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 1 2024-09-10T09:51:35.311 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:35.310+0000 7f60cba00640 1 -- 172.21.15.77:0/562621988 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f60a80a2c00 msgr2=0x7f60a80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:35.311 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:35.310+0000 7f60cba00640 1 --2- 172.21.15.77:0/562621988 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f60a80a2c00 0x7f60a80a50c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f60c40fddc0 tx=0x7f60b0005f90 comp rx=0 tx=0).stop 2024-09-10T09:51:35.311 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:35.310+0000 7f60cba00640 1 -- 172.21.15.77:0/562621988 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f60c4101de0 msgr2=0x7f60c419a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:35.311 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:35.310+0000 7f60cba00640 1 --2- 172.21.15.77:0/562621988 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f60c4101de0 0x7f60c419a630 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f60b400d950 tx=0x7f60b400de10 comp rx=0 tx=0).stop 2024-09-10T09:51:35.311 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:35.310+0000 7f60cba00640 1 -- 172.21.15.77:0/562621988 shutdown_connections 2024-09-10T09:51:35.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:35.310+0000 7f60cba00640 1 --2- 172.21.15.77:0/562621988 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f60a80a2c00 0x7f60a80a50c0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:35.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:35.310+0000 7f60cba00640 1 --2- 172.21.15.77:0/562621988 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60c4106f10 0x7f60c419ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:35.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:35.310+0000 7f60cba00640 1 --2- 172.21.15.77:0/562621988 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f60c4102790 0x7f60c419ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:35.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:35.310+0000 7f60cba00640 1 --2- 172.21.15.77:0/562621988 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f60c4101de0 0x7f60c419a630 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:35.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:35.310+0000 7f60cba00640 1 -- 172.21.15.77:0/562621988 >> 172.21.15.77:0/562621988 conn(0x7f60c40fd8c0 msgr2=0x7f60c40ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:35.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:35.310+0000 7f60cba00640 1 -- 172.21.15.77:0/562621988 shutdown_connections 2024-09-10T09:51:35.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:35.310+0000 7f60cba00640 1 -- 172.21.15.77:0/562621988 wait complete. 2024-09-10T09:51:35.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:35 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/562621988' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:51:35.631 INFO:journalctl@ceph.mds.l.smithi134.stdout:Sep 10 09:51:35 smithi134 systemd[1]: Starting Ceph mds.l for 14a55226-6f59-11ef-bcea-c7b262605968... 2024-09-10T09:51:35.689 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:35 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/562621988' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:51:35.923 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:35 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/562621988' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:51:35.985 INFO:journalctl@ceph.mds.l.smithi134.stdout:Sep 10 09:51:35 smithi134 podman[66256]: 2024-09-10 09:51:35.598332154 +0000 UTC m=+0.016045472 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T09:51:35.985 INFO:journalctl@ceph.mds.l.smithi134.stdout:Sep 10 09:51:35 smithi134 podman[66256]: 2024-09-10 09:51:35.730336218 +0000 UTC m=+0.148049520 container create 91a33ae0180337dc06f8e1a40d500adc71cca72b0402b1d9e8f6f7cef04ce34a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-l, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True) 2024-09-10T09:51:35.985 INFO:journalctl@ceph.mds.l.smithi134.stdout:Sep 10 09:51:35 smithi134 podman[66256]: 2024-09-10 09:51:35.983101522 +0000 UTC m=+0.400814827 container init 91a33ae0180337dc06f8e1a40d500adc71cca72b0402b1d9e8f6f7cef04ce34a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-10T09:51:36.381 INFO:journalctl@ceph.mds.l.smithi134.stdout:Sep 10 09:51:35 smithi134 podman[66256]: 2024-09-10 09:51:35.985971165 +0000 UTC m=+0.403684477 container start 91a33ae0180337dc06f8e1a40d500adc71cca72b0402b1d9e8f6f7cef04ce34a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-l, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T09:51:36.381 INFO:journalctl@ceph.mds.l.smithi134.stdout:Sep 10 09:51:36 smithi134 ceph-mds[66275]: -- 172.21.15.134:0/1135213420 <== mon.1 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55dd6b763860 con 0x55dd6b741400 2024-09-10T09:51:36.381 INFO:journalctl@ceph.mds.l.smithi134.stdout:Sep 10 09:51:36 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-l[66271]: starting mds.l at 2024-09-10T09:51:36.381 INFO:journalctl@ceph.mds.l.smithi134.stdout:Sep 10 09:51:36 smithi134 bash[66256]: 91a33ae0180337dc06f8e1a40d500adc71cca72b0402b1d9e8f6f7cef04ce34a 2024-09-10T09:51:36.381 INFO:journalctl@ceph.mds.l.smithi134.stdout:Sep 10 09:51:36 smithi134 systemd[1]: Started Ceph mds.l for 14a55226-6f59-11ef-bcea-c7b262605968. 2024-09-10T09:51:36.745 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:36 smithi120 ceph-mon[34770]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:51:36.746 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:36 smithi120 ceph-mon[34770]: mds.? [v2:172.21.15.134:6832/1005054423,v1:172.21.15.134:6833/1005054423] up:boot 2024-09-10T09:51:36.746 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:36 smithi120 ceph-mon[34770]: fsmap 1 up:standby 2024-09-10T09:51:36.746 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:36 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-10T09:51:36.746 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:36 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:36.746 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:36 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:36.746 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:36 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:36.746 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:36 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T09:51:36.746 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:36 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T09:51:36.746 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:36 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:51:36.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:36 smithi134 ceph-mon[35674]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:51:36.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:36 smithi134 ceph-mon[35674]: mds.? [v2:172.21.15.134:6832/1005054423,v1:172.21.15.134:6833/1005054423] up:boot 2024-09-10T09:51:36.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:36 smithi134 ceph-mon[35674]: fsmap 1 up:standby 2024-09-10T09:51:36.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:36 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-10T09:51:36.882 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:36 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:36.882 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:36 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:36.882 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:36 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:36.882 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:36 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T09:51:36.882 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:36 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T09:51:36.882 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:36 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:51:36.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:36 smithi077 ceph-mon[30319]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:51:36.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:36 smithi077 ceph-mon[30319]: mds.? [v2:172.21.15.134:6832/1005054423,v1:172.21.15.134:6833/1005054423] up:boot 2024-09-10T09:51:36.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:36 smithi077 ceph-mon[30319]: fsmap 1 up:standby 2024-09-10T09:51:36.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:36 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-10T09:51:36.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:36 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:36.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:36 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:36.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:36 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:36.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:36 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T09:51:36.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:36 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T09:51:36.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:36 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:51:37.673 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:37 smithi120 ceph-mon[34770]: Deploying daemon mds.k on smithi120 2024-09-10T09:51:37.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:37 smithi134 ceph-mon[35674]: Deploying daemon mds.k on smithi120 2024-09-10T09:51:37.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:37 smithi077 ceph-mon[30319]: Deploying daemon mds.k on smithi120 2024-09-10T09:51:38.200 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:51:38.391 INFO:journalctl@ceph.mds.k.smithi120.stdout:Sep 10 09:51:38 smithi120 systemd[1]: Starting Ceph mds.k for 14a55226-6f59-11ef-bcea-c7b262605968... 2024-09-10T09:51:38.569 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:51:38.673 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:38 smithi120 ceph-mon[34770]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:51:38.674 INFO:journalctl@ceph.mds.k.smithi120.stdout:Sep 10 09:51:38 smithi120 podman[63996]: 2024-09-10 09:51:38.293109387 +0000 UTC m=+0.015717729 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T09:51:38.674 INFO:journalctl@ceph.mds.k.smithi120.stdout:Sep 10 09:51:38 smithi120 podman[63996]: 2024-09-10 09:51:38.417637628 +0000 UTC m=+0.140245969 container create 4a976ae355b021cb76066d452e2b735d2242a5d69a95f336e7cf947107cc7626 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-k, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-10T09:51:38.689 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:38 smithi077 ceph-mon[30319]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:51:38.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:38 smithi134 ceph-mon[35674]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:51:39.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:39.158+0000 7f8c6dc00640 1 -- 172.21.15.77:0/2622033412 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8c68101450 msgr2=0x7f8c68105920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:39.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:39.158+0000 7f8c6dc00640 1 --2- 172.21.15.77:0/2622033412 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8c68101450 0x7f8c68105920 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8c54009900 tx=0x7f8c5402a470 comp rx=0 tx=0).stop 2024-09-10T09:51:39.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:39.158+0000 7f8c6dc00640 1 -- 172.21.15.77:0/2622033412 shutdown_connections 2024-09-10T09:51:39.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:39.158+0000 7f8c6dc00640 1 --2- 172.21.15.77:0/2622033412 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8c68105f40 0x7f8c681063d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:39.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:39.158+0000 7f8c6dc00640 1 --2- 172.21.15.77:0/2622033412 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8c68101450 0x7f8c68105920 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:39.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:39.158+0000 7f8c6dc00640 1 --2- 172.21.15.77:0/2622033412 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8c68100b30 0x7f8c68100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:39.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:39.158+0000 7f8c6dc00640 1 -- 172.21.15.77:0/2622033412 >> 172.21.15.77:0/2622033412 conn(0x7f8c680fc7a0 msgr2=0x7f8c680febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:39.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:39.158+0000 7f8c6dc00640 1 -- 172.21.15.77:0/2622033412 shutdown_connections 2024-09-10T09:51:39.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:39.158+0000 7f8c6dc00640 1 -- 172.21.15.77:0/2622033412 wait complete. 2024-09-10T09:51:39.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:39.159+0000 7f8c6dc00640 1 Processor -- start 2024-09-10T09:51:39.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:39.159+0000 7f8c6dc00640 1 -- start start 2024-09-10T09:51:39.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:39.160+0000 7f8c6dc00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8c68100b30 0x7f8c681a4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:39.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:39.160+0000 7f8c6dc00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8c68101450 0x7f8c681a4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:39.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:39.160+0000 7f8c6dc00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8c68105f40 0x7f8c6819e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:39.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:39.160+0000 7f8c6dc00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8c68118560 con 0x7f8c68100b30 2024-09-10T09:51:39.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:39.160+0000 7f8c6dc00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8c681183e0 con 0x7f8c68101450 2024-09-10T09:51:39.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:39.160+0000 7f8c6dc00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8c681186e0 con 0x7f8c68105f40 2024-09-10T09:51:39.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:39.160+0000 7f8c66a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8c68100b30 0x7f8c681a4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:39.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:39.160+0000 7f8c66a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8c68100b30 0x7f8c681a4750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56636/0 (socket says 172.21.15.77:56636) 2024-09-10T09:51:39.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:39.160+0000 7f8c66a00640 1 -- 172.21.15.77:0/1602715300 learned_addr learned my addr 172.21.15.77:0/1602715300 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:51:39.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:39.160+0000 7f8c66000640 1 --2- 172.21.15.77:0/1602715300 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8c68101450 0x7f8c681a4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:39.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:39.160+0000 7f8c67400640 1 --2- 172.21.15.77:0/1602715300 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8c68105f40 0x7f8c6819e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:39.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:39.160+0000 7f8c66000640 1 -- 172.21.15.77:0/1602715300 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8c68105f40 msgr2=0x7f8c6819e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:39.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:39.160+0000 7f8c66000640 1 --2- 172.21.15.77:0/1602715300 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8c68105f40 0x7f8c6819e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:39.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:39.160+0000 7f8c66000640 1 -- 172.21.15.77:0/1602715300 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8c68100b30 msgr2=0x7f8c681a4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:39.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:39.160+0000 7f8c66000640 1 --2- 172.21.15.77:0/1602715300 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8c68100b30 0x7f8c681a4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:39.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:39.160+0000 7f8c66000640 1 -- 172.21.15.77:0/1602715300 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c6819f040 con 0x7f8c68101450 2024-09-10T09:51:39.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:39.161+0000 7f8c66000640 1 --2- 172.21.15.77:0/1602715300 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8c68101450 0x7f8c681a4c90 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8c54005940 tx=0x7f8c540040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:39.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:39.161+0000 7f8c5f400640 1 -- 172.21.15.77:0/1602715300 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c5403f030 con 0x7f8c68101450 2024-09-10T09:51:39.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:39.161+0000 7f8c5f400640 1 -- 172.21.15.77:0/1602715300 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c540044c0 con 0x7f8c68101450 2024-09-10T09:51:39.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:39.161+0000 7f8c6dc00640 1 -- 172.21.15.77:0/1602715300 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c6819f2d0 con 0x7f8c68101450 2024-09-10T09:51:39.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:39.161+0000 7f8c5f400640 1 -- 172.21.15.77:0/1602715300 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c54035070 con 0x7f8c68101450 2024-09-10T09:51:39.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:39.161+0000 7f8c6dc00640 1 -- 172.21.15.77:0/1602715300 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c681ab620 con 0x7f8c68101450 2024-09-10T09:51:39.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:39.162+0000 7f8c6dc00640 1 -- 172.21.15.77:0/1602715300 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c68105920 con 0x7f8c68101450 2024-09-10T09:51:39.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:39.164+0000 7f8c5f400640 1 -- 172.21.15.77:0/1602715300 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f8c54042030 con 0x7f8c68101450 2024-09-10T09:51:39.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:39.165+0000 7f8c5f400640 1 --2- 172.21.15.77:0/1602715300 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f8c4c0a2c00 0x7f8c4c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:39.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:39.165+0000 7f8c5f400640 1 -- 172.21.15.77:0/1602715300 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7f8c540eeec0 con 0x7f8c68101450 2024-09-10T09:51:39.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:39.165+0000 7f8c66a00640 1 --2- 172.21.15.77:0/1602715300 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f8c4c0a2c00 0x7f8c4c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:39.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:39.166+0000 7f8c66a00640 1 --2- 172.21.15.77:0/1602715300 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f8c4c0a2c00 0x7f8c4c0a50c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8c6819fa50 tx=0x7f8c5800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:39.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:39.169+0000 7f8c5f400640 1 -- 172.21.15.77:0/1602715300 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8c540bbbd0 con 0x7f8c68101450 2024-09-10T09:51:39.175 INFO:journalctl@ceph.mds.k.smithi120.stdout:Sep 10 09:51:38 smithi120 podman[63996]: 2024-09-10 09:51:38.712544776 +0000 UTC m=+0.435153104 container init 4a976ae355b021cb76066d452e2b735d2242a5d69a95f336e7cf947107cc7626 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-k, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True) 2024-09-10T09:51:39.175 INFO:journalctl@ceph.mds.k.smithi120.stdout:Sep 10 09:51:38 smithi120 podman[63996]: 2024-09-10 09:51:38.715373573 +0000 UTC m=+0.437981911 container start 4a976ae355b021cb76066d452e2b735d2242a5d69a95f336e7cf947107cc7626 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-k, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0) 2024-09-10T09:51:39.175 INFO:journalctl@ceph.mds.k.smithi120.stdout:Sep 10 09:51:38 smithi120 ceph-mds[64016]: -- 172.21.15.120:0/552761141 <== mon.2 v2:172.21.15.120:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5648fdcb3860 con 0x5648fdc90800 2024-09-10T09:51:39.175 INFO:journalctl@ceph.mds.k.smithi120.stdout:Sep 10 09:51:38 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-k[64012]: starting mds.k at 2024-09-10T09:51:39.175 INFO:journalctl@ceph.mds.k.smithi120.stdout:Sep 10 09:51:38 smithi120 bash[63996]: 4a976ae355b021cb76066d452e2b735d2242a5d69a95f336e7cf947107cc7626 2024-09-10T09:51:39.175 INFO:journalctl@ceph.mds.k.smithi120.stdout:Sep 10 09:51:38 smithi120 systemd[1]: Started Ceph mds.k for 14a55226-6f59-11ef-bcea-c7b262605968. 2024-09-10T09:51:39.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:39.392+0000 7f8c6dc00640 1 -- 172.21.15.77:0/1602715300 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8c6819fe40 con 0x7f8c68101450 2024-09-10T09:51:39.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:39.393+0000 7f8c5f400640 1 -- 172.21.15.77:0/1602715300 <== mon.2 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1438 (secure 0 0 0) 0x7f8c540bfed0 con 0x7f8c68101450 2024-09-10T09:51:39.394 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:51:39.394 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":2,"btime":"2024-09-10T09:51:36:014053+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/1005054423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1005054423},{"type":"v1","addr":"172.21.15.134:6833","nonce":1005054423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-10T09:51:39.394 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 2 2024-09-10T09:51:39.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:39.395+0000 7f8c6dc00640 1 -- 172.21.15.77:0/1602715300 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f8c4c0a2c00 msgr2=0x7f8c4c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:39.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:39.395+0000 7f8c6dc00640 1 --2- 172.21.15.77:0/1602715300 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f8c4c0a2c00 0x7f8c4c0a50c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8c6819fa50 tx=0x7f8c5800b410 comp rx=0 tx=0).stop 2024-09-10T09:51:39.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:39.395+0000 7f8c6dc00640 1 -- 172.21.15.77:0/1602715300 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8c68101450 msgr2=0x7f8c681a4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:39.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:39.395+0000 7f8c6dc00640 1 --2- 172.21.15.77:0/1602715300 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8c68101450 0x7f8c681a4c90 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8c54005940 tx=0x7f8c540040c0 comp rx=0 tx=0).stop 2024-09-10T09:51:39.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:39.396+0000 7f8c6dc00640 1 -- 172.21.15.77:0/1602715300 shutdown_connections 2024-09-10T09:51:39.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:39.396+0000 7f8c6dc00640 1 --2- 172.21.15.77:0/1602715300 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f8c4c0a2c00 0x7f8c4c0a50c0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:39.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:39.396+0000 7f8c6dc00640 1 --2- 172.21.15.77:0/1602715300 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8c68105f40 0x7f8c6819e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:39.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:39.396+0000 7f8c6dc00640 1 --2- 172.21.15.77:0/1602715300 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8c68101450 0x7f8c681a4c90 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:39.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:39.396+0000 7f8c6dc00640 1 --2- 172.21.15.77:0/1602715300 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8c68100b30 0x7f8c681a4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:39.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:39.396+0000 7f8c6dc00640 1 -- 172.21.15.77:0/1602715300 >> 172.21.15.77:0/1602715300 conn(0x7f8c680fc7a0 msgr2=0x7f8c680fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:39.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:39.396+0000 7f8c6dc00640 1 -- 172.21.15.77:0/1602715300 shutdown_connections 2024-09-10T09:51:39.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:39.396+0000 7f8c6dc00640 1 -- 172.21.15.77:0/1602715300 wait complete. 2024-09-10T09:51:39.938 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:39 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:39.938 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:39 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:39.938 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:39 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:39.938 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:39 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T09:51:39.938 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:39 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T09:51:39.938 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:39 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:51:39.938 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:39 smithi077 ceph-mon[30319]: Deploying daemon mds.j on smithi077 2024-09-10T09:51:39.938 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:39 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/1602715300' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:51:39.938 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:39 smithi077 ceph-mon[30319]: mds.? [v2:172.21.15.120:6832/3716307744,v1:172.21.15.120:6833/3716307744] up:boot 2024-09-10T09:51:39.938 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:39 smithi077 ceph-mon[30319]: fsmap 2 up:standby 2024-09-10T09:51:39.938 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:39 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-10T09:51:40.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:39 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:40.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:39 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:40.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:39 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:40.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:39 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T09:51:40.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:39 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T09:51:40.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:39 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:51:40.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:39 smithi134 ceph-mon[35674]: Deploying daemon mds.j on smithi077 2024-09-10T09:51:40.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:39 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/1602715300' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:51:40.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:39 smithi134 ceph-mon[35674]: mds.? [v2:172.21.15.120:6832/3716307744,v1:172.21.15.120:6833/3716307744] up:boot 2024-09-10T09:51:40.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:39 smithi134 ceph-mon[35674]: fsmap 2 up:standby 2024-09-10T09:51:40.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:39 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-10T09:51:40.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:39 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:40.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:39 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:40.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:39 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:40.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:39 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T09:51:40.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:39 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T09:51:40.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:39 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:51:40.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:39 smithi120 ceph-mon[34770]: Deploying daemon mds.j on smithi077 2024-09-10T09:51:40.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:39 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/1602715300' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:51:40.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:39 smithi120 ceph-mon[34770]: mds.? [v2:172.21.15.120:6832/3716307744,v1:172.21.15.120:6833/3716307744] up:boot 2024-09-10T09:51:40.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:39 smithi120 ceph-mon[34770]: fsmap 2 up:standby 2024-09-10T09:51:40.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:39 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-10T09:51:41.127 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:40 smithi120 ceph-mon[34770]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:51:41.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:40 smithi134 ceph-mon[35674]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:51:41.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:40 smithi077 ceph-mon[30319]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:51:42.087 INFO:journalctl@ceph.mds.j.smithi077.stdout:Sep 10 09:51:42 smithi077 systemd[1]: Starting Ceph mds.j for 14a55226-6f59-11ef-bcea-c7b262605968... 2024-09-10T09:51:42.300 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:51:42.683 INFO:journalctl@ceph.mds.j.smithi077.stdout:Sep 10 09:51:42 smithi077 podman[69278]: 2024-09-10 09:51:42.300811976 +0000 UTC m=+0.027029061 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T09:51:42.683 INFO:journalctl@ceph.mds.j.smithi077.stdout:Sep 10 09:51:42 smithi077 podman[69278]: 2024-09-10 09:51:42.436988149 +0000 UTC m=+0.163205220 container create 3646333f4cb18cad1b3cb6d41960e086f202513768325b8e01c53f423a16f6dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-10T09:51:42.683 INFO:journalctl@ceph.mds.j.smithi077.stdout:Sep 10 09:51:42 smithi077 podman[69278]: 2024-09-10 09:51:42.682393409 +0000 UTC m=+0.408610482 container init 3646333f4cb18cad1b3cb6d41960e086f202513768325b8e01c53f423a16f6dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-j, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T09:51:42.900 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:51:42.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:42 smithi077 ceph-mon[30319]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:51:42.939 INFO:journalctl@ceph.mds.j.smithi077.stdout:Sep 10 09:51:42 smithi077 podman[69278]: 2024-09-10 09:51:42.68945445 +0000 UTC m=+0.415671531 container start 3646333f4cb18cad1b3cb6d41960e086f202513768325b8e01c53f423a16f6dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-j, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T09:51:42.940 INFO:journalctl@ceph.mds.j.smithi077.stdout:Sep 10 09:51:42 smithi077 ceph-mds[69340]: -- 172.21.15.77:0/2604951281 <== mon.2 v2:172.21.15.120:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c2e783f860 con 0x55c2e781d000 2024-09-10T09:51:42.940 INFO:journalctl@ceph.mds.j.smithi077.stdout:Sep 10 09:51:42 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-j[69324]: starting mds.j at 2024-09-10T09:51:42.940 INFO:journalctl@ceph.mds.j.smithi077.stdout:Sep 10 09:51:42 smithi077 bash[69278]: 3646333f4cb18cad1b3cb6d41960e086f202513768325b8e01c53f423a16f6dd 2024-09-10T09:51:42.940 INFO:journalctl@ceph.mds.j.smithi077.stdout:Sep 10 09:51:42 smithi077 systemd[1]: Started Ceph mds.j for 14a55226-6f59-11ef-bcea-c7b262605968. 2024-09-10T09:51:43.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:42 smithi134 ceph-mon[35674]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:51:43.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:42 smithi120 ceph-mon[34770]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:51:43.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:43.459+0000 7f9037800640 1 -- 172.21.15.77:0/4217974879 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9030076720 msgr2=0x7f9030076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:43.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:43.459+0000 7f9037800640 1 --2- 172.21.15.77:0/4217974879 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9030076720 0x7f9030076b00 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9024009900 tx=0x7f902402a410 comp rx=0 tx=0).stop 2024-09-10T09:51:43.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:43.459+0000 7f9037800640 1 -- 172.21.15.77:0/4217974879 shutdown_connections 2024-09-10T09:51:43.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:43.459+0000 7f9037800640 1 --2- 172.21.15.77:0/4217974879 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9030112a30 0x7f9030114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:43.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:43.459+0000 7f9037800640 1 --2- 172.21.15.77:0/4217974879 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9030077040 0x7f90301124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:43.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:43.459+0000 7f9037800640 1 --2- 172.21.15.77:0/4217974879 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9030076720 0x7f9030076b00 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:43.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:43.459+0000 7f9037800640 1 -- 172.21.15.77:0/4217974879 >> 172.21.15.77:0/4217974879 conn(0x7f90300ffa90 msgr2=0x7f9030101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:43.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:43.459+0000 7f9037800640 1 -- 172.21.15.77:0/4217974879 shutdown_connections 2024-09-10T09:51:43.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:43.459+0000 7f9037800640 1 -- 172.21.15.77:0/4217974879 wait complete. 2024-09-10T09:51:43.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:43.460+0000 7f9037800640 1 Processor -- start 2024-09-10T09:51:43.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:43.460+0000 7f9037800640 1 -- start start 2024-09-10T09:51:43.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:43.461+0000 7f9037800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9030076720 0x7f90301a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:43.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:43.461+0000 7f9037800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9030077040 0x7f90301a4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:43.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:43.461+0000 7f9037800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9030112a30 0x7f903019e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:43.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:43.461+0000 7f9037800640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f90301185c0 con 0x7f9030077040 2024-09-10T09:51:43.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:43.461+0000 7f9037800640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9030118440 con 0x7f9030076720 2024-09-10T09:51:43.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:43.461+0000 7f9037800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9030118740 con 0x7f9030112a30 2024-09-10T09:51:43.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:43.461+0000 7f9034e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9030076720 0x7f90301a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:43.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:43.461+0000 7f902fe00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9030077040 0x7f90301a4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:43.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:43.461+0000 7f9034e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9030076720 0x7f90301a46f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:47076/0 (socket says 172.21.15.77:47076) 2024-09-10T09:51:43.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:43.461+0000 7f9034e00640 1 -- 172.21.15.77:0/2556655156 learned_addr learned my addr 172.21.15.77:0/2556655156 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:51:43.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:43.461+0000 7f9035800640 1 --2- 172.21.15.77:0/2556655156 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9030112a30 0x7f903019e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:43.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:43.462+0000 7f902fe00640 1 -- 172.21.15.77:0/2556655156 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9030112a30 msgr2=0x7f903019e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:43.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:43.462+0000 7f902fe00640 1 --2- 172.21.15.77:0/2556655156 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9030112a30 0x7f903019e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:43.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:43.462+0000 7f902fe00640 1 -- 172.21.15.77:0/2556655156 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9030076720 msgr2=0x7f90301a46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:43.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:43.462+0000 7f902fe00640 1 --2- 172.21.15.77:0/2556655156 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9030076720 0x7f90301a46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:43.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:43.462+0000 7f902fe00640 1 -- 172.21.15.77:0/2556655156 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f903019efe0 con 0x7f9030077040 2024-09-10T09:51:43.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:43.462+0000 7f902fe00640 1 --2- 172.21.15.77:0/2556655156 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9030077040 0x7f90301a4c30 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f901c00d6d0 tx=0x7f901c00db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:43.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:43.462+0000 7f902d600640 1 -- 172.21.15.77:0/2556655156 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f901c017070 con 0x7f9030077040 2024-09-10T09:51:43.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:43.462+0000 7f902d600640 1 -- 172.21.15.77:0/2556655156 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f901c009820 con 0x7f9030077040 2024-09-10T09:51:43.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:43.462+0000 7f902d600640 1 -- 172.21.15.77:0/2556655156 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f901c004d00 con 0x7f9030077040 2024-09-10T09:51:43.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:43.462+0000 7f9037800640 1 -- 172.21.15.77:0/2556655156 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f903019f2d0 con 0x7f9030077040 2024-09-10T09:51:43.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:43.463+0000 7f9037800640 1 -- 172.21.15.77:0/2556655156 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90301ab5c0 con 0x7f9030077040 2024-09-10T09:51:43.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:43.464+0000 7f9037800640 1 -- 172.21.15.77:0/2556655156 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9030076b00 con 0x7f9030077040 2024-09-10T09:51:43.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:43.466+0000 7f902d600640 1 -- 172.21.15.77:0/2556655156 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f901c0099d0 con 0x7f9030077040 2024-09-10T09:51:43.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:43.467+0000 7f902d600640 1 --2- 172.21.15.77:0/2556655156 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f90180a2c00 0x7f90180a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:43.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:43.467+0000 7f902d600640 1 -- 172.21.15.77:0/2556655156 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7f901c0ce6c0 con 0x7f9030077040 2024-09-10T09:51:43.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:43.467+0000 7f9034e00640 1 --2- 172.21.15.77:0/2556655156 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f90180a2c00 0x7f90180a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:43.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:43.468+0000 7f9034e00640 1 --2- 172.21.15.77:0/2556655156 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f90180a2c00 0x7f90180a50c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f90240096c0 tx=0x7f9024005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:43.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:43.472+0000 7f902d600640 1 -- 172.21.15.77:0/2556655156 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f901c09b3d0 con 0x7f9030077040 2024-09-10T09:51:43.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:43.695+0000 7f9037800640 1 -- 172.21.15.77:0/2556655156 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f903002d7c0 con 0x7f9030077040 2024-09-10T09:51:43.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:43.695+0000 7f902d600640 1 -- 172.21.15.77:0/2556655156 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3091 (secure 0 0 0) 0x7f901c09f6d0 con 0x7f9030077040 2024-09-10T09:51:43.697 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:51:43.697 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":4,"btime":"2024-09-10T09:51:42:864468+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3159289675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3159289675},{"type":"v1","addr":"172.21.15.77:6835","nonce":3159289675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/1005054423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1005054423},{"type":"v1","addr":"172.21.15.134:6833","nonce":1005054423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/3716307744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":3716307744},{"type":"v1","addr":"172.21.15.120:6833","nonce":3716307744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-10T09:51:43.697 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 4 2024-09-10T09:51:43.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:43.697+0000 7f9037800640 1 -- 172.21.15.77:0/2556655156 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f90180a2c00 msgr2=0x7f90180a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:43.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:43.697+0000 7f9037800640 1 --2- 172.21.15.77:0/2556655156 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f90180a2c00 0x7f90180a50c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f90240096c0 tx=0x7f9024005a20 comp rx=0 tx=0).stop 2024-09-10T09:51:43.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:43.697+0000 7f9037800640 1 -- 172.21.15.77:0/2556655156 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9030077040 msgr2=0x7f90301a4c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:43.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:43.697+0000 7f9037800640 1 --2- 172.21.15.77:0/2556655156 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9030077040 0x7f90301a4c30 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f901c00d6d0 tx=0x7f901c00db90 comp rx=0 tx=0).stop 2024-09-10T09:51:43.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:43.698+0000 7f9037800640 1 -- 172.21.15.77:0/2556655156 shutdown_connections 2024-09-10T09:51:43.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:43.698+0000 7f9037800640 1 --2- 172.21.15.77:0/2556655156 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f90180a2c00 0x7f90180a50c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:43.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:43.698+0000 7f9037800640 1 --2- 172.21.15.77:0/2556655156 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9030112a30 0x7f903019e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:43.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:43.698+0000 7f9037800640 1 --2- 172.21.15.77:0/2556655156 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9030077040 0x7f90301a4c30 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:43.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:43.698+0000 7f9037800640 1 --2- 172.21.15.77:0/2556655156 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9030076720 0x7f90301a46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:43.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:43.698+0000 7f9037800640 1 -- 172.21.15.77:0/2556655156 >> 172.21.15.77:0/2556655156 conn(0x7f90300ffa90 msgr2=0x7f9030101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:43.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:43.698+0000 7f9037800640 1 -- 172.21.15.77:0/2556655156 shutdown_connections 2024-09-10T09:51:43.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:43.698+0000 7f9037800640 1 -- 172.21.15.77:0/2556655156 wait complete. 2024-09-10T09:51:43.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:43 smithi077 ceph-mon[30319]: mds.? [v2:172.21.15.77:6834/3159289675,v1:172.21.15.77:6835/3159289675] up:boot 2024-09-10T09:51:43.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:43 smithi077 ceph-mon[30319]: fsmap 3 up:standby 2024-09-10T09:51:43.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:43 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-10T09:51:43.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:43 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:43.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:43 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:43.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:43 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:43.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:43 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T09:51:43.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:43 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T09:51:43.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:43 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:51:43.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:43 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/2556655156' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:51:44.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:43 smithi134 ceph-mon[35674]: mds.? [v2:172.21.15.77:6834/3159289675,v1:172.21.15.77:6835/3159289675] up:boot 2024-09-10T09:51:44.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:43 smithi134 ceph-mon[35674]: fsmap 3 up:standby 2024-09-10T09:51:44.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:43 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-10T09:51:44.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:43 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:44.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:43 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:44.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:43 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:44.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:43 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T09:51:44.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:43 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T09:51:44.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:43 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:51:44.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:43 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/2556655156' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:51:44.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:43 smithi120 ceph-mon[34770]: mds.? [v2:172.21.15.77:6834/3159289675,v1:172.21.15.77:6835/3159289675] up:boot 2024-09-10T09:51:44.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:43 smithi120 ceph-mon[34770]: fsmap 3 up:standby 2024-09-10T09:51:44.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:43 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-10T09:51:44.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:43 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:44.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:43 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:44.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:43 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:44.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:43 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T09:51:44.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:43 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T09:51:44.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:43 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:51:44.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:43 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/2556655156' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:51:45.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:44 smithi134 ceph-mon[35674]: Deploying daemon mds.d on smithi077 2024-09-10T09:51:45.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:44 smithi134 ceph-mon[35674]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:51:45.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:44 smithi120 ceph-mon[34770]: Deploying daemon mds.d on smithi077 2024-09-10T09:51:45.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:44 smithi120 ceph-mon[34770]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:51:45.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:44 smithi077 ceph-mon[30319]: Deploying daemon mds.d on smithi077 2024-09-10T09:51:45.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:44 smithi077 ceph-mon[30319]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:51:46.550 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:51:46.879 INFO:journalctl@ceph.mds.d.smithi077.stdout:Sep 10 09:51:46 smithi077 podman[69696]: 2024-09-10 09:51:46.736341224 +0000 UTC m=+0.293017908 container create 3cf98e5ddefe69a9845bf0b41f0a16bad998e948b4adda79635a9cb4f1add85e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-d, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-10T09:51:47.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:46 smithi134 ceph-mon[35674]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:51:47.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:46 smithi120 ceph-mon[34770]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:51:47.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:46 smithi077 ceph-mon[30319]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:51:47.189 INFO:journalctl@ceph.mds.d.smithi077.stdout:Sep 10 09:51:47 smithi077 podman[69696]: 2024-09-10 09:51:47.014588733 +0000 UTC m=+0.571265412 container init 3cf98e5ddefe69a9845bf0b41f0a16bad998e948b4adda79635a9cb4f1add85e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-d, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-10T09:51:47.189 INFO:journalctl@ceph.mds.d.smithi077.stdout:Sep 10 09:51:47 smithi077 podman[69696]: 2024-09-10 09:51:47.022871246 +0000 UTC m=+0.579547929 container start 3cf98e5ddefe69a9845bf0b41f0a16bad998e948b4adda79635a9cb4f1add85e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-d, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True) 2024-09-10T09:51:47.189 INFO:journalctl@ceph.mds.d.smithi077.stdout:Sep 10 09:51:47 smithi077 ceph-mds[69761]: -- 172.21.15.77:0/1165215064 <== mon.0 v2:172.21.15.77:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5640f072bc20 con 0x5640f0709000 2024-09-10T09:51:47.189 INFO:journalctl@ceph.mds.d.smithi077.stdout:Sep 10 09:51:47 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-d[69746]: starting mds.d at 2024-09-10T09:51:47.190 INFO:journalctl@ceph.mds.d.smithi077.stdout:Sep 10 09:51:47 smithi077 bash[69696]: 3cf98e5ddefe69a9845bf0b41f0a16bad998e948b4adda79635a9cb4f1add85e 2024-09-10T09:51:47.190 INFO:journalctl@ceph.mds.d.smithi077.stdout:Sep 10 09:51:47 smithi077 systemd[1]: Started Ceph mds.d for 14a55226-6f59-11ef-bcea-c7b262605968. 2024-09-10T09:51:47.202 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:51:47.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:47.813+0000 7f822f000640 1 -- 172.21.15.77:0/630502493 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f82281049a0 msgr2=0x7f82281089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:47.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:47.813+0000 7f822f000640 1 --2- 172.21.15.77:0/630502493 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f82281049a0 0x7f82281089f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8214009860 tx=0x7f821402a340 comp rx=0 tx=0).stop 2024-09-10T09:51:47.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:47.813+0000 7f822f000640 1 -- 172.21.15.77:0/630502493 shutdown_connections 2024-09-10T09:51:47.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:47.813+0000 7f822f000640 1 --2- 172.21.15.77:0/630502493 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8228109120 0x7f8228110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:47.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:47.813+0000 7f822f000640 1 --2- 172.21.15.77:0/630502493 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f82281049a0 0x7f82281089f0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:47.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:47.813+0000 7f822f000640 1 --2- 172.21.15.77:0/630502493 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8228103ff0 0x7f82281043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:47.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:47.813+0000 7f822f000640 1 -- 172.21.15.77:0/630502493 >> 172.21.15.77:0/630502493 conn(0x7f82280ffa90 msgr2=0x7f8228101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:47.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:47.813+0000 7f822f000640 1 -- 172.21.15.77:0/630502493 shutdown_connections 2024-09-10T09:51:47.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:47.813+0000 7f822f000640 1 -- 172.21.15.77:0/630502493 wait complete. 2024-09-10T09:51:47.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:47.814+0000 7f822f000640 1 Processor -- start 2024-09-10T09:51:47.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:47.814+0000 7f822f000640 1 -- start start 2024-09-10T09:51:47.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:47.814+0000 7f822f000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8228103ff0 0x7f822819a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:47.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:47.814+0000 7f822f000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f82281049a0 0x7f822819ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:47.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:47.814+0000 7f822f000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8228109120 0x7f822819ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:47.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:47.814+0000 7f822f000640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8228114120 con 0x7f8228103ff0 2024-09-10T09:51:47.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:47.814+0000 7f822f000640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8228113fa0 con 0x7f8228109120 2024-09-10T09:51:47.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:47.814+0000 7f822f000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f82281142a0 con 0x7f82281049a0 2024-09-10T09:51:47.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:47.814+0000 7f8227e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8228103ff0 0x7f822819a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:47.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:47.814+0000 7f8227e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8228103ff0 0x7f822819a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:44302/0 (socket says 172.21.15.77:44302) 2024-09-10T09:51:47.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:47.814+0000 7f8227e00640 1 -- 172.21.15.77:0/3987503562 learned_addr learned my addr 172.21.15.77:0/3987503562 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:51:47.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:47.815+0000 7f8227400640 1 --2- 172.21.15.77:0/3987503562 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f82281049a0 0x7f822819ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:47.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:47.815+0000 7f822d000640 1 --2- 172.21.15.77:0/3987503562 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8228109120 0x7f822819ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:47.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:47.815+0000 7f8227e00640 1 -- 172.21.15.77:0/3987503562 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f82281049a0 msgr2=0x7f822819ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:47.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:47.815+0000 7f8227e00640 1 --2- 172.21.15.77:0/3987503562 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f82281049a0 0x7f822819ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:47.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:47.815+0000 7f8227e00640 1 -- 172.21.15.77:0/3987503562 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8228109120 msgr2=0x7f822819ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:47.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:47.815+0000 7f8227e00640 1 --2- 172.21.15.77:0/3987503562 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8228109120 0x7f822819ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:47.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:47.815+0000 7f8227e00640 1 -- 172.21.15.77:0/3987503562 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f822819f670 con 0x7f8228103ff0 2024-09-10T09:51:47.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:47.815+0000 7f8227e00640 1 --2- 172.21.15.77:0/3987503562 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8228103ff0 0x7f822819a6c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f821800c420 tx=0x7f8218009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:47.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:47.815+0000 7f8224c00640 1 -- 172.21.15.77:0/3987503562 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8218003970 con 0x7f8228103ff0 2024-09-10T09:51:47.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:47.815+0000 7f822f000640 1 -- 172.21.15.77:0/3987503562 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f822819f960 con 0x7f8228103ff0 2024-09-10T09:51:47.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:47.815+0000 7f8224c00640 1 -- 172.21.15.77:0/3987503562 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8218004020 con 0x7f8228103ff0 2024-09-10T09:51:47.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:47.816+0000 7f822f000640 1 -- 172.21.15.77:0/3987503562 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82281a7240 con 0x7f8228103ff0 2024-09-10T09:51:47.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:47.816+0000 7f8224c00640 1 -- 172.21.15.77:0/3987503562 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8218004d40 con 0x7f8228103ff0 2024-09-10T09:51:47.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:47.817+0000 7f822f000640 1 -- 172.21.15.77:0/3987503562 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82281043d0 con 0x7f8228103ff0 2024-09-10T09:51:47.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:47.819+0000 7f8224c00640 1 -- 172.21.15.77:0/3987503562 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f821801d020 con 0x7f8228103ff0 2024-09-10T09:51:47.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:47.820+0000 7f8224c00640 1 --2- 172.21.15.77:0/3987503562 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f820c0a2b80 0x7f820c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:47.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:47.820+0000 7f8224c00640 1 -- 172.21.15.77:0/3987503562 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7f82180cebb0 con 0x7f8228103ff0 2024-09-10T09:51:47.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:47.820+0000 7f8227400640 1 --2- 172.21.15.77:0/3987503562 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f820c0a2b80 0x7f820c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:47.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:47.821+0000 7f8227400640 1 --2- 172.21.15.77:0/3987503562 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f820c0a2b80 0x7f820c0a5040 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f821402a930 tx=0x7f82140058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:47.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:47.824+0000 7f8224c00640 1 -- 172.21.15.77:0/3987503562 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f821809bf10 con 0x7f8228103ff0 2024-09-10T09:51:48.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:48.035+0000 7f822f000640 1 -- 172.21.15.77:0/3987503562 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f822810c360 con 0x7f8228103ff0 2024-09-10T09:51:48.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:48.036+0000 7f8224c00640 1 -- 172.21.15.77:0/3987503562 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3916 (secure 0 0 0) 0x7f821809b810 con 0x7f8228103ff0 2024-09-10T09:51:48.038 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:51:48.038 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":5,"btime":"2024-09-10T09:51:47:878308+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3159289675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3159289675},{"type":"v1","addr":"172.21.15.77:6835","nonce":3159289675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1951962341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1951962341},{"type":"v1","addr":"172.21.15.77:6837","nonce":1951962341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/1005054423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1005054423},{"type":"v1","addr":"172.21.15.134:6833","nonce":1005054423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/3716307744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":3716307744},{"type":"v1","addr":"172.21.15.120:6833","nonce":3716307744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-10T09:51:48.038 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 5 2024-09-10T09:51:48.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:48.038+0000 7f822f000640 1 -- 172.21.15.77:0/3987503562 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f820c0a2b80 msgr2=0x7f820c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:48.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:48.038+0000 7f822f000640 1 --2- 172.21.15.77:0/3987503562 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f820c0a2b80 0x7f820c0a5040 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f821402a930 tx=0x7f82140058d0 comp rx=0 tx=0).stop 2024-09-10T09:51:48.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:48.038+0000 7f822f000640 1 -- 172.21.15.77:0/3987503562 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8228103ff0 msgr2=0x7f822819a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:48.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:48.038+0000 7f822f000640 1 --2- 172.21.15.77:0/3987503562 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8228103ff0 0x7f822819a6c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f821800c420 tx=0x7f8218009f90 comp rx=0 tx=0).stop 2024-09-10T09:51:48.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:48.038+0000 7f822f000640 1 -- 172.21.15.77:0/3987503562 shutdown_connections 2024-09-10T09:51:48.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:48.038+0000 7f822f000640 1 --2- 172.21.15.77:0/3987503562 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f820c0a2b80 0x7f820c0a5040 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:48.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:48.038+0000 7f822f000640 1 --2- 172.21.15.77:0/3987503562 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8228109120 0x7f822819ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:48.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:48.038+0000 7f822f000640 1 --2- 172.21.15.77:0/3987503562 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f82281049a0 0x7f822819ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:48.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:48.038+0000 7f822f000640 1 --2- 172.21.15.77:0/3987503562 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8228103ff0 0x7f822819a6c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:48.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:48.038+0000 7f822f000640 1 -- 172.21.15.77:0/3987503562 >> 172.21.15.77:0/3987503562 conn(0x7f82280ffa90 msgr2=0x7f8228100db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:48.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:48.038+0000 7f822f000640 1 -- 172.21.15.77:0/3987503562 shutdown_connections 2024-09-10T09:51:48.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:48.039+0000 7f822f000640 1 -- 172.21.15.77:0/3987503562 wait complete. 2024-09-10T09:51:48.406 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:48 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:48.407 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:48 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:48.407 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:48 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:48.407 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:48 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T09:51:48.407 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:48 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T09:51:48.407 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:48 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:51:48.407 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:48 smithi077 ceph-mon[30319]: Deploying daemon mds.e on smithi120 2024-09-10T09:51:48.407 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:48 smithi077 ceph-mon[30319]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:51:48.407 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:48 smithi077 ceph-mon[30319]: mds.? [v2:172.21.15.77:6836/1951962341,v1:172.21.15.77:6837/1951962341] up:boot 2024-09-10T09:51:48.407 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:48 smithi077 ceph-mon[30319]: fsmap 4 up:standby 2024-09-10T09:51:48.407 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:48 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-10T09:51:48.407 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:48 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/3987503562' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:51:48.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:48 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:48.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:48 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:48.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:48 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:48.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:48 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T09:51:48.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:48 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T09:51:48.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:48 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:51:48.632 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:48 smithi134 ceph-mon[35674]: Deploying daemon mds.e on smithi120 2024-09-10T09:51:48.632 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:48 smithi134 ceph-mon[35674]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:51:48.632 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:48 smithi134 ceph-mon[35674]: mds.? [v2:172.21.15.77:6836/1951962341,v1:172.21.15.77:6837/1951962341] up:boot 2024-09-10T09:51:48.632 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:48 smithi134 ceph-mon[35674]: fsmap 4 up:standby 2024-09-10T09:51:48.632 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:48 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-10T09:51:48.632 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:48 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/3987503562' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:51:48.674 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:48 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:48.674 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:48 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:48.674 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:48 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:48.674 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:48 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T09:51:48.674 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:48 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T09:51:48.674 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:48 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:51:48.674 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:48 smithi120 ceph-mon[34770]: Deploying daemon mds.e on smithi120 2024-09-10T09:51:48.674 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:48 smithi120 ceph-mon[34770]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:51:48.674 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:48 smithi120 ceph-mon[34770]: mds.? [v2:172.21.15.77:6836/1951962341,v1:172.21.15.77:6837/1951962341] up:boot 2024-09-10T09:51:48.674 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:48 smithi120 ceph-mon[34770]: fsmap 4 up:standby 2024-09-10T09:51:48.674 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:48 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-10T09:51:48.675 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:48 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/3987503562' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:51:49.423 INFO:journalctl@ceph.mds.e.smithi120.stdout:Sep 10 09:51:49 smithi120 systemd[1]: Starting Ceph mds.e for 14a55226-6f59-11ef-bcea-c7b262605968... 2024-09-10T09:51:49.729 INFO:journalctl@ceph.mds.e.smithi120.stdout:Sep 10 09:51:49 smithi120 podman[64314]: 2024-09-10 09:51:49.348161805 +0000 UTC m=+0.015784819 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T09:51:49.729 INFO:journalctl@ceph.mds.e.smithi120.stdout:Sep 10 09:51:49 smithi120 podman[64314]: 2024-09-10 09:51:49.467387422 +0000 UTC m=+0.135010437 container create 4041cb8ecc89493653c1ff68c88b19ee6003d773aff2273e897afab7369b59ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-e, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-10T09:51:50.174 INFO:journalctl@ceph.mds.e.smithi120.stdout:Sep 10 09:51:49 smithi120 podman[64314]: 2024-09-10 09:51:49.728371917 +0000 UTC m=+0.395994932 container init 4041cb8ecc89493653c1ff68c88b19ee6003d773aff2273e897afab7369b59ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-e, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-10T09:51:50.174 INFO:journalctl@ceph.mds.e.smithi120.stdout:Sep 10 09:51:49 smithi120 podman[64314]: 2024-09-10 09:51:49.73128924 +0000 UTC m=+0.398912263 container start 4041cb8ecc89493653c1ff68c88b19ee6003d773aff2273e897afab7369b59ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-e, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2) 2024-09-10T09:51:50.174 INFO:journalctl@ceph.mds.e.smithi120.stdout:Sep 10 09:51:49 smithi120 ceph-mds[64334]: -- 172.21.15.120:0/3961933700 <== mon.2 v2:172.21.15.120:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5634c7589860 con 0x5634c7567000 2024-09-10T09:51:50.174 INFO:journalctl@ceph.mds.e.smithi120.stdout:Sep 10 09:51:49 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-e[64330]: starting mds.e at 2024-09-10T09:51:50.175 INFO:journalctl@ceph.mds.e.smithi120.stdout:Sep 10 09:51:49 smithi120 bash[64314]: 4041cb8ecc89493653c1ff68c88b19ee6003d773aff2273e897afab7369b59ae 2024-09-10T09:51:50.175 INFO:journalctl@ceph.mds.e.smithi120.stdout:Sep 10 09:51:49 smithi120 systemd[1]: Started Ceph mds.e for 14a55226-6f59-11ef-bcea-c7b262605968. 2024-09-10T09:51:50.778 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:51:51.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:50 smithi134 ceph-mon[35674]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:51:51.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:50 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:51.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:50 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:51.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:50 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:51.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:50 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T09:51:51.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:50 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T09:51:51.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:50 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:51:51.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:50 smithi134 ceph-mon[35674]: Deploying daemon mds.f on smithi134 2024-09-10T09:51:51.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:50 smithi134 ceph-mon[35674]: mds.? [v2:172.21.15.120:6834/1319892544,v1:172.21.15.120:6835/1319892544] up:boot 2024-09-10T09:51:51.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:50 smithi134 ceph-mon[35674]: fsmap 5 up:standby 2024-09-10T09:51:51.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:50 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-10T09:51:51.160 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:51:51.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:50 smithi120 ceph-mon[34770]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:51:51.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:50 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:51.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:50 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:51.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:50 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:51.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:50 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T09:51:51.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:50 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T09:51:51.175 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:50 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:51:51.175 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:50 smithi120 ceph-mon[34770]: Deploying daemon mds.f on smithi134 2024-09-10T09:51:51.175 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:50 smithi120 ceph-mon[34770]: mds.? [v2:172.21.15.120:6834/1319892544,v1:172.21.15.120:6835/1319892544] up:boot 2024-09-10T09:51:51.175 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:50 smithi120 ceph-mon[34770]: fsmap 5 up:standby 2024-09-10T09:51:51.175 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:50 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-10T09:51:51.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:50 smithi077 ceph-mon[30319]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:51:51.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:50 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:51.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:50 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:51.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:50 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:51.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:50 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T09:51:51.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:50 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T09:51:51.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:50 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:51:51.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:50 smithi077 ceph-mon[30319]: Deploying daemon mds.f on smithi134 2024-09-10T09:51:51.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:50 smithi077 ceph-mon[30319]: mds.? [v2:172.21.15.120:6834/1319892544,v1:172.21.15.120:6835/1319892544] up:boot 2024-09-10T09:51:51.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:50 smithi077 ceph-mon[30319]: fsmap 5 up:standby 2024-09-10T09:51:51.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:50 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-10T09:51:51.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:51.743+0000 7fe8cf400640 1 -- 172.21.15.77:0/1773289399 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe8c8101de0 msgr2=0x7fe8c81021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:51.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:51.743+0000 7fe8cf400640 1 --2- 172.21.15.77:0/1773289399 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe8c8101de0 0x7fe8c81021c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8009900 tx=0x7fe8b802a3f0 comp rx=0 tx=0).stop 2024-09-10T09:51:51.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:51.744+0000 7fe8cf400640 1 -- 172.21.15.77:0/1773289399 shutdown_connections 2024-09-10T09:51:51.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:51.744+0000 7fe8cf400640 1 --2- 172.21.15.77:0/1773289399 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8c8106f10 0x7fe8c810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:51.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:51.744+0000 7fe8cf400640 1 --2- 172.21.15.77:0/1773289399 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe8c8102790 0x7fe8c81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:51.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:51.744+0000 7fe8cf400640 1 --2- 172.21.15.77:0/1773289399 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe8c8101de0 0x7fe8c81021c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:51.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:51.744+0000 7fe8cf400640 1 -- 172.21.15.77:0/1773289399 >> 172.21.15.77:0/1773289399 conn(0x7fe8c80fd8c0 msgr2=0x7fe8c80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:51.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:51.744+0000 7fe8cf400640 1 -- 172.21.15.77:0/1773289399 shutdown_connections 2024-09-10T09:51:51.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:51.744+0000 7fe8cf400640 1 -- 172.21.15.77:0/1773289399 wait complete. 2024-09-10T09:51:51.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:51.745+0000 7fe8cf400640 1 Processor -- start 2024-09-10T09:51:51.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:51.745+0000 7fe8cf400640 1 -- start start 2024-09-10T09:51:51.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:51.745+0000 7fe8cf400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe8c8101de0 0x7fe8c81983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:51.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:51.745+0000 7fe8cf400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8c8102790 0x7fe8c8198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:51.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:51.745+0000 7fe8cf400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe8c8106f10 0x7fe8c819ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:51.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:51.745+0000 7fe8cf400640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c8111f70 con 0x7fe8c8102790 2024-09-10T09:51:51.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:51.745+0000 7fe8cf400640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c8111df0 con 0x7fe8c8106f10 2024-09-10T09:51:51.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:51.745+0000 7fe8cf400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c81120f0 con 0x7fe8c8101de0 2024-09-10T09:51:51.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:51.745+0000 7fe8cca00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe8c8101de0 0x7fe8c81983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:51.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:51.745+0000 7fe8cca00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe8c8101de0 0x7fe8c81983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:54528/0 (socket says 172.21.15.77:54528) 2024-09-10T09:51:51.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:51.745+0000 7fe8cca00640 1 -- 172.21.15.77:0/3819889266 learned_addr learned my addr 172.21.15.77:0/3819889266 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:51:51.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:51.746+0000 7fe8cd400640 1 --2- 172.21.15.77:0/3819889266 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe8c8106f10 0x7fe8c819ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:51.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:51.746+0000 7fe8bfe00640 1 --2- 172.21.15.77:0/3819889266 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8c8102790 0x7fe8c8198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:51.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:51.746+0000 7fe8cca00640 1 -- 172.21.15.77:0/3819889266 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe8c8106f10 msgr2=0x7fe8c819ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:51.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:51.746+0000 7fe8cca00640 1 --2- 172.21.15.77:0/3819889266 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe8c8106f10 0x7fe8c819ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:51.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:51.746+0000 7fe8cca00640 1 -- 172.21.15.77:0/3819889266 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8c8102790 msgr2=0x7fe8c8198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:51.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:51.746+0000 7fe8cca00640 1 --2- 172.21.15.77:0/3819889266 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8c8102790 0x7fe8c8198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:51.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:51.746+0000 7fe8cca00640 1 -- 172.21.15.77:0/3819889266 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8c819d3a0 con 0x7fe8c8101de0 2024-09-10T09:51:51.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:51.746+0000 7fe8cd400640 1 --2- 172.21.15.77:0/3819889266 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe8c8106f10 0x7fe8c819ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:51:51.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:51.746+0000 7fe8cca00640 1 --2- 172.21.15.77:0/3819889266 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe8c8101de0 0x7fe8c81983f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8006010 tx=0x7fe8b8004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:51.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:51.746+0000 7fe8c6000640 1 -- 172.21.15.77:0/3819889266 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b803f030 con 0x7fe8c8101de0 2024-09-10T09:51:51.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:51.747+0000 7fe8cf400640 1 -- 172.21.15.77:0/3819889266 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8c819d630 con 0x7fe8c8101de0 2024-09-10T09:51:51.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:51.747+0000 7fe8c6000640 1 -- 172.21.15.77:0/3819889266 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8b8004440 con 0x7fe8c8101de0 2024-09-10T09:51:51.751 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:51.747+0000 7fe8c6000640 1 -- 172.21.15.77:0/3819889266 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b8035070 con 0x7fe8c8101de0 2024-09-10T09:51:51.751 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:51.747+0000 7fe8cf400640 1 -- 172.21.15.77:0/3819889266 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8c81a4f70 con 0x7fe8c8101de0 2024-09-10T09:51:51.751 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:51.748+0000 7fe8cf400640 1 -- 172.21.15.77:0/3819889266 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8c81032c0 con 0x7fe8c8101de0 2024-09-10T09:51:51.751 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:51.750+0000 7fe8c6000640 1 -- 172.21.15.77:0/3819889266 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe8b8042030 con 0x7fe8c8101de0 2024-09-10T09:51:51.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:51.751+0000 7fe8c6000640 1 --2- 172.21.15.77:0/3819889266 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fe8ac0a2b80 0x7fe8ac0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:51.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:51.751+0000 7fe8c6000640 1 -- 172.21.15.77:0/3819889266 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7fe8b806de00 con 0x7fe8c8101de0 2024-09-10T09:51:51.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:51.751+0000 7fe8bfe00640 1 --2- 172.21.15.77:0/3819889266 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fe8ac0a2b80 0x7fe8ac0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:51.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:51.752+0000 7fe8bfe00640 1 --2- 172.21.15.77:0/3819889266 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fe8ac0a2b80 0x7fe8ac0a5040 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe8c80fdd50 tx=0x7fe8b0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:51.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:51.757+0000 7fe8c6000640 1 -- 172.21.15.77:0/3819889266 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe8b80bbd60 con 0x7fe8c8101de0 2024-09-10T09:51:51.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:51.968+0000 7fe8cf400640 1 -- 172.21.15.77:0/3819889266 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe8c810a0f0 con 0x7fe8c8101de0 2024-09-10T09:51:51.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:51.969+0000 7fe8c6000640 1 -- 172.21.15.77:0/3819889266 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4744 (secure 0 0 0) 0x7fe8b80c0060 con 0x7fe8c8101de0 2024-09-10T09:51:51.971 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:51:51.971 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":6,"btime":"2024-09-10T09:51:50:716005+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3159289675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3159289675},{"type":"v1","addr":"172.21.15.77:6835","nonce":3159289675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1951962341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1951962341},{"type":"v1","addr":"172.21.15.77:6837","nonce":1951962341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/1005054423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1005054423},{"type":"v1","addr":"172.21.15.134:6833","nonce":1005054423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/3716307744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":3716307744},{"type":"v1","addr":"172.21.15.120:6833","nonce":3716307744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/1319892544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1319892544},{"type":"v1","addr":"172.21.15.120:6835","nonce":1319892544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-09-10T09:51:51.972 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 6 2024-09-10T09:51:51.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:51.972+0000 7fe8cf400640 1 -- 172.21.15.77:0/3819889266 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fe8ac0a2b80 msgr2=0x7fe8ac0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:51.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:51.972+0000 7fe8cf400640 1 --2- 172.21.15.77:0/3819889266 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fe8ac0a2b80 0x7fe8ac0a5040 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe8c80fdd50 tx=0x7fe8b0005f90 comp rx=0 tx=0).stop 2024-09-10T09:51:51.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:51.972+0000 7fe8cf400640 1 -- 172.21.15.77:0/3819889266 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe8c8101de0 msgr2=0x7fe8c81983f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:51.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:51.972+0000 7fe8cf400640 1 --2- 172.21.15.77:0/3819889266 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe8c8101de0 0x7fe8c81983f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8006010 tx=0x7fe8b8004040 comp rx=0 tx=0).stop 2024-09-10T09:51:51.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:51.972+0000 7fe8cf400640 1 -- 172.21.15.77:0/3819889266 shutdown_connections 2024-09-10T09:51:51.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:51.972+0000 7fe8cf400640 1 --2- 172.21.15.77:0/3819889266 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fe8ac0a2b80 0x7fe8ac0a5040 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:51.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:51.972+0000 7fe8cf400640 1 --2- 172.21.15.77:0/3819889266 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe8c8106f10 0x7fe8c819ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:51.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:51.972+0000 7fe8cf400640 1 --2- 172.21.15.77:0/3819889266 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8c8102790 0x7fe8c8198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:51.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:51.972+0000 7fe8cf400640 1 --2- 172.21.15.77:0/3819889266 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe8c8101de0 0x7fe8c81983f0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:51.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:51.972+0000 7fe8cf400640 1 -- 172.21.15.77:0/3819889266 >> 172.21.15.77:0/3819889266 conn(0x7fe8c80fd8c0 msgr2=0x7fe8c80fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:51.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:51.972+0000 7fe8cf400640 1 -- 172.21.15.77:0/3819889266 shutdown_connections 2024-09-10T09:51:51.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:51.972+0000 7fe8cf400640 1 -- 172.21.15.77:0/3819889266 wait complete. 2024-09-10T09:51:52.095 INFO:journalctl@ceph.mds.f.smithi134.stdout:Sep 10 09:51:51 smithi134 systemd[1]: Starting Ceph mds.f for 14a55226-6f59-11ef-bcea-c7b262605968... 2024-09-10T09:51:52.363 INFO:journalctl@ceph.mds.f.smithi134.stdout:Sep 10 09:51:52 smithi134 podman[66556]: 2024-09-10 09:51:51.996728573 +0000 UTC m=+0.015603623 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T09:51:52.363 INFO:journalctl@ceph.mds.f.smithi134.stdout:Sep 10 09:51:52 smithi134 podman[66556]: 2024-09-10 09:51:52.118360968 +0000 UTC m=+0.137236014 container create 221d5b459a693b4cf60023f43b887461723dd8066b6eb952f70c00ba012444ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-f, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T09:51:52.364 INFO:journalctl@ceph.mds.f.smithi134.stdout:Sep 10 09:51:52 smithi134 podman[66556]: 2024-09-10 09:51:52.363133312 +0000 UTC m=+0.382008364 container init 221d5b459a693b4cf60023f43b887461723dd8066b6eb952f70c00ba012444ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-f, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T09:51:52.631 INFO:journalctl@ceph.mds.f.smithi134.stdout:Sep 10 09:51:52 smithi134 podman[66556]: 2024-09-10 09:51:52.365956246 +0000 UTC m=+0.384831311 container start 221d5b459a693b4cf60023f43b887461723dd8066b6eb952f70c00ba012444ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-f, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-10T09:51:52.631 INFO:journalctl@ceph.mds.f.smithi134.stdout:Sep 10 09:51:52 smithi134 ceph-mds[66575]: -- 172.21.15.134:0/3450889596 <== mon.1 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559e46265860 con 0x559e46243000 2024-09-10T09:51:52.631 INFO:journalctl@ceph.mds.f.smithi134.stdout:Sep 10 09:51:52 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-f[66571]: starting mds.f at 2024-09-10T09:51:52.631 INFO:journalctl@ceph.mds.f.smithi134.stdout:Sep 10 09:51:52 smithi134 bash[66556]: 221d5b459a693b4cf60023f43b887461723dd8066b6eb952f70c00ba012444ba 2024-09-10T09:51:52.631 INFO:journalctl@ceph.mds.f.smithi134.stdout:Sep 10 09:51:52 smithi134 systemd[1]: Started Ceph mds.f for 14a55226-6f59-11ef-bcea-c7b262605968. 2024-09-10T09:51:53.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:52 smithi134 ceph-mon[35674]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:51:53.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:52 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/3819889266' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:51:53.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:52 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:53.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:52 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:53.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:52 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:53.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:52 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T09:51:53.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:52 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T09:51:53.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:52 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:51:53.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:52 smithi120 ceph-mon[34770]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:51:53.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:52 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/3819889266' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:51:53.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:52 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:53.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:52 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:53.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:52 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:53.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:52 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T09:51:53.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:52 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T09:51:53.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:52 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:51:53.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:52 smithi077 ceph-mon[30319]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:51:53.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:52 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/3819889266' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:51:53.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:52 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:53.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:52 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:53.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:52 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:53.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:52 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T09:51:53.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:52 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T09:51:53.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:52 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:51:53.991 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:53 smithi134 ceph-mon[35674]: Deploying daemon mds.c on smithi134 2024-09-10T09:51:53.991 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:53 smithi134 ceph-mon[35674]: mds.? [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] up:boot 2024-09-10T09:51:53.991 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:53 smithi134 ceph-mon[35674]: fsmap 6 up:standby 2024-09-10T09:51:53.991 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:53 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-10T09:51:54.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:53 smithi120 ceph-mon[34770]: Deploying daemon mds.c on smithi134 2024-09-10T09:51:54.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:53 smithi120 ceph-mon[34770]: mds.? [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] up:boot 2024-09-10T09:51:54.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:53 smithi120 ceph-mon[34770]: fsmap 6 up:standby 2024-09-10T09:51:54.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:53 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-10T09:51:54.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:53 smithi077 ceph-mon[30319]: Deploying daemon mds.c on smithi134 2024-09-10T09:51:54.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:53 smithi077 ceph-mon[30319]: mds.? [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] up:boot 2024-09-10T09:51:54.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:53 smithi077 ceph-mon[30319]: fsmap 6 up:standby 2024-09-10T09:51:54.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:53 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-10T09:51:54.631 INFO:journalctl@ceph.mds.c.smithi134.stdout:Sep 10 09:51:54 smithi134 systemd[1]: Starting Ceph mds.c for 14a55226-6f59-11ef-bcea-c7b262605968... 2024-09-10T09:51:54.856 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:51:54.963 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:54 smithi134 ceph-mon[35674]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:51:54.963 INFO:journalctl@ceph.mds.c.smithi134.stdout:Sep 10 09:51:54 smithi134 podman[66860]: 2024-09-10 09:51:54.606429941 +0000 UTC m=+0.016145962 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T09:51:54.964 INFO:journalctl@ceph.mds.c.smithi134.stdout:Sep 10 09:51:54 smithi134 podman[66860]: 2024-09-10 09:51:54.718326734 +0000 UTC m=+0.128042741 container create 5864e6ac715cecdacce9bff622395ca4ab8710a454503aee4714e943533ebe77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-c, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T09:51:54.964 INFO:journalctl@ceph.mds.c.smithi134.stdout:Sep 10 09:51:54 smithi134 podman[66860]: 2024-09-10 09:51:54.96305287 +0000 UTC m=+0.372768878 container init 5864e6ac715cecdacce9bff622395ca4ab8710a454503aee4714e943533ebe77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-c, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T09:51:55.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:54 smithi120 ceph-mon[34770]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:51:55.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:54 smithi077 ceph-mon[30319]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:51:55.236 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:51:55.381 INFO:journalctl@ceph.mds.c.smithi134.stdout:Sep 10 09:51:54 smithi134 podman[66860]: 2024-09-10 09:51:54.966108802 +0000 UTC m=+0.375824810 container start 5864e6ac715cecdacce9bff622395ca4ab8710a454503aee4714e943533ebe77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-c, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True) 2024-09-10T09:51:55.381 INFO:journalctl@ceph.mds.c.smithi134.stdout:Sep 10 09:51:54 smithi134 ceph-mds[66879]: -- 172.21.15.134:0/971113272 <== mon.1 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55dab3753860 con 0x55dab3731400 2024-09-10T09:51:55.381 INFO:journalctl@ceph.mds.c.smithi134.stdout:Sep 10 09:51:54 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-c[66875]: starting mds.c at 2024-09-10T09:51:55.381 INFO:journalctl@ceph.mds.c.smithi134.stdout:Sep 10 09:51:55 smithi134 bash[66860]: 5864e6ac715cecdacce9bff622395ca4ab8710a454503aee4714e943533ebe77 2024-09-10T09:51:55.381 INFO:journalctl@ceph.mds.c.smithi134.stdout:Sep 10 09:51:55 smithi134 systemd[1]: Started Ceph mds.c for 14a55226-6f59-11ef-bcea-c7b262605968. 2024-09-10T09:51:55.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:55.793+0000 7fdd9b200640 1 -- 172.21.15.77:0/3420804364 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdd9c1047e0 msgr2=0x7fdd9c108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:55.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:55.793+0000 7fdd9b200640 1 --2- 172.21.15.77:0/3420804364 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdd9c1047e0 0x7fdd9c108830 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fdd88009860 tx=0x7fdd8802a340 comp rx=0 tx=0).stop 2024-09-10T09:51:55.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:55.793+0000 7fdd9b200640 1 -- 172.21.15.77:0/3420804364 shutdown_connections 2024-09-10T09:51:55.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:55.793+0000 7fdd9b200640 1 --2- 172.21.15.77:0/3420804364 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdd9c108f60 0x7fdd9c110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:55.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:55.793+0000 7fdd9b200640 1 --2- 172.21.15.77:0/3420804364 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdd9c1047e0 0x7fdd9c108830 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:55.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:55.793+0000 7fdd9b200640 1 --2- 172.21.15.77:0/3420804364 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd9c103e30 0x7fdd9c104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:55.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:55.793+0000 7fdd9b200640 1 -- 172.21.15.77:0/3420804364 >> 172.21.15.77:0/3420804364 conn(0x7fdd9c0ff910 msgr2=0x7fdd9c101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:55.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:55.793+0000 7fdd9b200640 1 -- 172.21.15.77:0/3420804364 shutdown_connections 2024-09-10T09:51:55.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:55.793+0000 7fdd9b200640 1 -- 172.21.15.77:0/3420804364 wait complete. 2024-09-10T09:51:55.797 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:55.794+0000 7fdd9b200640 1 Processor -- start 2024-09-10T09:51:55.797 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:55.794+0000 7fdd9b200640 1 -- start start 2024-09-10T09:51:55.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:55.794+0000 7fdd9b200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd9c103e30 0x7fdd9c19a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:55.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:55.794+0000 7fdd9b200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdd9c1047e0 0x7fdd9c19aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:55.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:55.794+0000 7fdd9b200640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdd9c108f60 0x7fdd9c19ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:55.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:55.794+0000 7fdd9b200640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fdd9c113fc0 con 0x7fdd9c1047e0 2024-09-10T09:51:55.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:55.794+0000 7fdd9b200640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdd9c113e40 con 0x7fdd9c108f60 2024-09-10T09:51:55.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:55.794+0000 7fdd9b200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdd9c114140 con 0x7fdd9c103e30 2024-09-10T09:51:55.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:55.795+0000 7fdd99400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdd9c1047e0 0x7fdd9c19aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:55.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:55.795+0000 7fdd9a800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdd9c108f60 0x7fdd9c19ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:55.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:55.795+0000 7fdd99400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdd9c1047e0 0x7fdd9c19aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57874/0 (socket says 172.21.15.77:57874) 2024-09-10T09:51:55.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:55.795+0000 7fdd9a800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdd9c108f60 0x7fdd9c19ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:33378/0 (socket says 172.21.15.77:33378) 2024-09-10T09:51:55.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:55.795+0000 7fdd99400640 1 -- 172.21.15.77:0/3586808936 learned_addr learned my addr 172.21.15.77:0/3586808936 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:51:55.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:55.795+0000 7fdd99e00640 1 --2- 172.21.15.77:0/3586808936 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd9c103e30 0x7fdd9c19a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:55.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:55.795+0000 7fdd9a800640 1 -- 172.21.15.77:0/3586808936 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd9c103e30 msgr2=0x7fdd9c19a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:55.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:55.795+0000 7fdd9a800640 1 --2- 172.21.15.77:0/3586808936 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd9c103e30 0x7fdd9c19a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:55.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:55.795+0000 7fdd9a800640 1 -- 172.21.15.77:0/3586808936 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdd9c1047e0 msgr2=0x7fdd9c19aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:55.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:55.795+0000 7fdd9a800640 1 --2- 172.21.15.77:0/3586808936 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdd9c1047e0 0x7fdd9c19aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:55.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:55.795+0000 7fdd9a800640 1 -- 172.21.15.77:0/3586808936 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd9c19f470 con 0x7fdd9c108f60 2024-09-10T09:51:55.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:55.795+0000 7fdd99e00640 1 --2- 172.21.15.77:0/3586808936 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd9c103e30 0x7fdd9c19a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:51:55.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:55.795+0000 7fdd99400640 1 --2- 172.21.15.77:0/3586808936 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdd9c1047e0 0x7fdd9c19aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:51:55.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:55.795+0000 7fdd9a800640 1 --2- 172.21.15.77:0/3586808936 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdd9c108f60 0x7fdd9c19ed90 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fdd9400ef00 tx=0x7fdd9400c740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:55.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:55.796+0000 7fdd92a00640 1 -- 172.21.15.77:0/3586808936 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd94004b20 con 0x7fdd9c108f60 2024-09-10T09:51:55.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:55.796+0000 7fdd9b200640 1 -- 172.21.15.77:0/3586808936 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd9c19f760 con 0x7fdd9c108f60 2024-09-10T09:51:55.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:55.796+0000 7fdd92a00640 1 -- 172.21.15.77:0/3586808936 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd94003e50 con 0x7fdd9c108f60 2024-09-10T09:51:55.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:55.796+0000 7fdd92a00640 1 -- 172.21.15.77:0/3586808936 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd9401a020 con 0x7fdd9c108f60 2024-09-10T09:51:55.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:55.796+0000 7fdd9b200640 1 -- 172.21.15.77:0/3586808936 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd9c1a7040 con 0x7fdd9c108f60 2024-09-10T09:51:55.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:55.797+0000 7fdd9b200640 1 -- 172.21.15.77:0/3586808936 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd9c105310 con 0x7fdd9c108f60 2024-09-10T09:51:55.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:55.799+0000 7fdd92a00640 1 -- 172.21.15.77:0/3586808936 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fdd9401d020 con 0x7fdd9c108f60 2024-09-10T09:51:55.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:55.800+0000 7fdd92a00640 1 --2- 172.21.15.77:0/3586808936 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fdd800a2c00 0x7fdd800a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:55.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:55.800+0000 7fdd92a00640 1 -- 172.21.15.77:0/3586808936 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7fdd940ce9f0 con 0x7fdd9c108f60 2024-09-10T09:51:55.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:55.800+0000 7fdd99e00640 1 --2- 172.21.15.77:0/3586808936 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fdd800a2c00 0x7fdd800a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:55.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:55.801+0000 7fdd99e00640 1 --2- 172.21.15.77:0/3586808936 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fdd800a2c00 0x7fdd800a50c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c009fc0 tx=0x7fdd8c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:55.806 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:55.804+0000 7fdd92a00640 1 -- 172.21.15.77:0/3586808936 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdd9409bd50 con 0x7fdd9c108f60 2024-09-10T09:51:56.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:56.017+0000 7fdd9b200640 1 -- 172.21.15.77:0/3586808936 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdd9c10c200 con 0x7fdd9c108f60 2024-09-10T09:51:56.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:56.018+0000 7fdd92a00640 1 -- 172.21.15.77:0/3586808936 <== mon.2 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6400 (secure 0 0 0) 0x7fdd9409b650 con 0x7fdd9c108f60 2024-09-10T09:51:56.020 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:51:56.020 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":8,"btime":"2024-09-10T09:51:55:888178+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3159289675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3159289675},{"type":"v1","addr":"172.21.15.77:6835","nonce":3159289675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1951962341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1951962341},{"type":"v1","addr":"172.21.15.77:6837","nonce":1951962341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/1005054423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1005054423},{"type":"v1","addr":"172.21.15.134:6833","nonce":1005054423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/3716307744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":3716307744},{"type":"v1","addr":"172.21.15.120:6833","nonce":3716307744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/1319892544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1319892544},{"type":"v1","addr":"172.21.15.120:6835","nonce":1319892544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24496,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/3293933699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3293933699},{"type":"v1","addr":"172.21.15.134:6835","nonce":3293933699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24502,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/2461497091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2461497091},{"type":"v1","addr":"172.21.15.134:6837","nonce":2461497091}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-09-10T09:51:56.020 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 8 2024-09-10T09:51:56.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:56.020+0000 7fdd9b200640 1 -- 172.21.15.77:0/3586808936 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fdd800a2c00 msgr2=0x7fdd800a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:56.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:56.020+0000 7fdd9b200640 1 --2- 172.21.15.77:0/3586808936 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fdd800a2c00 0x7fdd800a50c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c009fc0 tx=0x7fdd8c005f90 comp rx=0 tx=0).stop 2024-09-10T09:51:56.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:56.020+0000 7fdd9b200640 1 -- 172.21.15.77:0/3586808936 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdd9c108f60 msgr2=0x7fdd9c19ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:56.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:56.020+0000 7fdd9b200640 1 --2- 172.21.15.77:0/3586808936 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdd9c108f60 0x7fdd9c19ed90 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fdd9400ef00 tx=0x7fdd9400c740 comp rx=0 tx=0).stop 2024-09-10T09:51:56.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:56.020+0000 7fdd9b200640 1 -- 172.21.15.77:0/3586808936 shutdown_connections 2024-09-10T09:51:56.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:56.020+0000 7fdd9b200640 1 --2- 172.21.15.77:0/3586808936 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fdd800a2c00 0x7fdd800a50c0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:56.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:56.020+0000 7fdd9b200640 1 --2- 172.21.15.77:0/3586808936 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdd9c108f60 0x7fdd9c19ed90 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:56.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:56.020+0000 7fdd9b200640 1 --2- 172.21.15.77:0/3586808936 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdd9c1047e0 0x7fdd9c19aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:56.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:56.020+0000 7fdd9b200640 1 --2- 172.21.15.77:0/3586808936 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd9c103e30 0x7fdd9c19a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:56.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:56.020+0000 7fdd9b200640 1 -- 172.21.15.77:0/3586808936 >> 172.21.15.77:0/3586808936 conn(0x7fdd9c0ff910 msgr2=0x7fdd9c100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:56.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:56.021+0000 7fdd9b200640 1 -- 172.21.15.77:0/3586808936 shutdown_connections 2024-09-10T09:51:56.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:56.021+0000 7fdd9b200640 1 -- 172.21.15.77:0/3586808936 wait complete. 2024-09-10T09:51:56.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:56 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:56.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:56 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:56.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:56 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:56.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:56 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T09:51:56.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:56 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T09:51:56.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:56 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:51:56.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:56 smithi134 ceph-mon[35674]: Deploying daemon mds.h on smithi120 2024-09-10T09:51:56.382 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:56 smithi134 ceph-mon[35674]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:51:56.382 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:56 smithi134 ceph-mon[35674]: mds.? [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] up:boot 2024-09-10T09:51:56.382 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:56 smithi134 ceph-mon[35674]: fsmap 7 up:standby 2024-09-10T09:51:56.382 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:56 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-10T09:51:56.382 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:56 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/3586808936' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:51:56.423 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:56 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:56.423 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:56 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:56.423 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:56 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:56.423 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:56 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T09:51:56.423 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:56 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T09:51:56.424 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:56 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:51:56.424 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:56 smithi077 ceph-mon[30319]: Deploying daemon mds.h on smithi120 2024-09-10T09:51:56.424 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:56 smithi077 ceph-mon[30319]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:51:56.424 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:56 smithi077 ceph-mon[30319]: mds.? [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] up:boot 2024-09-10T09:51:56.424 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:56 smithi077 ceph-mon[30319]: fsmap 7 up:standby 2024-09-10T09:51:56.424 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:56 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-10T09:51:56.424 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:56 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/3586808936' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:51:56.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:56 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:56.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:56 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:56.425 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:56 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:56.425 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:56 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T09:51:56.425 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:56 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T09:51:56.425 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:56 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:51:56.425 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:56 smithi120 ceph-mon[34770]: Deploying daemon mds.h on smithi120 2024-09-10T09:51:56.425 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:56 smithi120 ceph-mon[34770]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:51:56.425 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:56 smithi120 ceph-mon[34770]: mds.? [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] up:boot 2024-09-10T09:51:56.425 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:56 smithi120 ceph-mon[34770]: fsmap 7 up:standby 2024-09-10T09:51:56.425 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:56 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-10T09:51:56.425 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:56 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/3586808936' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:51:57.387 INFO:journalctl@ceph.mds.h.smithi120.stdout:Sep 10 09:51:57 smithi120 systemd[1]: Starting Ceph mds.h for 14a55226-6f59-11ef-bcea-c7b262605968... 2024-09-10T09:51:57.674 INFO:journalctl@ceph.mds.h.smithi120.stdout:Sep 10 09:51:57 smithi120 podman[64617]: 2024-09-10 09:51:57.288752732 +0000 UTC m=+0.016053528 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T09:51:57.674 INFO:journalctl@ceph.mds.h.smithi120.stdout:Sep 10 09:51:57 smithi120 podman[64617]: 2024-09-10 09:51:57.408674439 +0000 UTC m=+0.135975263 container create bfdb151a8d5157d138ac1536508a71cd6354770553ca2e908fced2a3fdc048a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-h, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-10T09:51:58.174 INFO:journalctl@ceph.mds.h.smithi120.stdout:Sep 10 09:51:57 smithi120 podman[64617]: 2024-09-10 09:51:57.67828461 +0000 UTC m=+0.405585397 container init bfdb151a8d5157d138ac1536508a71cd6354770553ca2e908fced2a3fdc048a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-h, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-10T09:51:58.174 INFO:journalctl@ceph.mds.h.smithi120.stdout:Sep 10 09:51:57 smithi120 podman[64617]: 2024-09-10 09:51:57.681172847 +0000 UTC m=+0.408473638 container start bfdb151a8d5157d138ac1536508a71cd6354770553ca2e908fced2a3fdc048a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T09:51:58.174 INFO:journalctl@ceph.mds.h.smithi120.stdout:Sep 10 09:51:57 smithi120 ceph-mds[64637]: -- 172.21.15.120:0/1089008670 <== mon.2 v2:172.21.15.120:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c0e0b65860 con 0x55c0e0b42800 2024-09-10T09:51:58.174 INFO:journalctl@ceph.mds.h.smithi120.stdout:Sep 10 09:51:57 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-h[64633]: starting mds.h at 2024-09-10T09:51:58.174 INFO:journalctl@ceph.mds.h.smithi120.stdout:Sep 10 09:51:57 smithi120 bash[64617]: bfdb151a8d5157d138ac1536508a71cd6354770553ca2e908fced2a3fdc048a6 2024-09-10T09:51:58.174 INFO:journalctl@ceph.mds.h.smithi120.stdout:Sep 10 09:51:57 smithi120 systemd[1]: Started Ceph mds.h for 14a55226-6f59-11ef-bcea-c7b262605968. 2024-09-10T09:51:58.804 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:51:58.856 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:58 smithi077 ceph-mon[30319]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:51:58.856 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:58 smithi077 ceph-mon[30319]: mds.? [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] up:boot 2024-09-10T09:51:58.856 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:58 smithi077 ceph-mon[30319]: fsmap 8 up:standby 2024-09-10T09:51:58.856 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:58 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-10T09:51:58.856 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:58 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:58.856 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:58 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:58.856 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:58 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:58.856 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:58 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T09:51:58.856 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:58 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T09:51:58.857 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:58 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:51:58.857 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:51:58 smithi077 ceph-mon[30319]: Deploying daemon mds.i on smithi134 2024-09-10T09:51:59.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:58 smithi134 ceph-mon[35674]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:51:59.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:58 smithi134 ceph-mon[35674]: mds.? [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] up:boot 2024-09-10T09:51:59.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:58 smithi134 ceph-mon[35674]: fsmap 8 up:standby 2024-09-10T09:51:59.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:58 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-10T09:51:59.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:58 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:59.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:58 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:59.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:58 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:59.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:58 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T09:51:59.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:58 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T09:51:59.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:58 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:51:59.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:51:58 smithi134 ceph-mon[35674]: Deploying daemon mds.i on smithi134 2024-09-10T09:51:59.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:58 smithi120 ceph-mon[34770]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:51:59.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:58 smithi120 ceph-mon[34770]: mds.? [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] up:boot 2024-09-10T09:51:59.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:58 smithi120 ceph-mon[34770]: fsmap 8 up:standby 2024-09-10T09:51:59.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:58 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-10T09:51:59.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:58 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:59.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:58 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:59.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:58 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:51:59.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:58 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T09:51:59.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:58 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T09:51:59.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:58 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:51:59.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:51:58 smithi120 ceph-mon[34770]: Deploying daemon mds.i on smithi134 2024-09-10T09:51:59.211 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:51:59.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:59.807+0000 7fc91f800640 1 -- 172.21.15.77:0/1987634062 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc918109140 msgr2=0x7fc918110cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:59.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:59.807+0000 7fc91f800640 1 --2- 172.21.15.77:0/1987634062 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc918109140 0x7fc918110cc0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc910009ca0 tx=0x7fc91002a8c0 comp rx=0 tx=0).stop 2024-09-10T09:51:59.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:59.808+0000 7fc91f800640 1 -- 172.21.15.77:0/1987634062 shutdown_connections 2024-09-10T09:51:59.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:59.808+0000 7fc91f800640 1 --2- 172.21.15.77:0/1987634062 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc918109140 0x7fc918110cc0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:59.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:59.808+0000 7fc91f800640 1 --2- 172.21.15.77:0/1987634062 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc9181049c0 0x7fc918108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:59.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:59.808+0000 7fc91f800640 1 --2- 172.21.15.77:0/1987634062 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc918104010 0x7fc9181043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:59.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:59.808+0000 7fc91f800640 1 -- 172.21.15.77:0/1987634062 >> 172.21.15.77:0/1987634062 conn(0x7fc9180ffa50 msgr2=0x7fc918101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:59.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:59.808+0000 7fc91f800640 1 -- 172.21.15.77:0/1987634062 shutdown_connections 2024-09-10T09:51:59.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:59.808+0000 7fc91f800640 1 -- 172.21.15.77:0/1987634062 wait complete. 2024-09-10T09:51:59.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:59.809+0000 7fc91f800640 1 Processor -- start 2024-09-10T09:51:59.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:59.809+0000 7fc91f800640 1 -- start start 2024-09-10T09:51:59.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:59.810+0000 7fc91f800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc918104010 0x7fc91819a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:59.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:59.810+0000 7fc91f800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc9181049c0 0x7fc91819ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:59.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:59.810+0000 7fc91f800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc918109140 0x7fc91819ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:59.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:59.810+0000 7fc91f800640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc9181141c0 con 0x7fc918104010 2024-09-10T09:51:59.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:59.810+0000 7fc91f800640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc918114040 con 0x7fc9181049c0 2024-09-10T09:51:59.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:59.810+0000 7fc91f800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc918114340 con 0x7fc918109140 2024-09-10T09:51:59.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:59.810+0000 7fc91ce00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc918104010 0x7fc91819a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:59.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:59.810+0000 7fc917e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc9181049c0 0x7fc91819ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:59.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:59.810+0000 7fc91ce00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc918104010 0x7fc91819a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57898/0 (socket says 172.21.15.77:57898) 2024-09-10T09:51:59.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:59.810+0000 7fc917e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc9181049c0 0x7fc91819ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:33392/0 (socket says 172.21.15.77:33392) 2024-09-10T09:51:59.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:59.810+0000 7fc91ce00640 1 -- 172.21.15.77:0/1233912988 learned_addr learned my addr 172.21.15.77:0/1233912988 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:51:59.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:59.810+0000 7fc91d800640 1 --2- 172.21.15.77:0/1233912988 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc918109140 0x7fc91819ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:59.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:59.811+0000 7fc917e00640 1 -- 172.21.15.77:0/1233912988 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc918109140 msgr2=0x7fc91819ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:59.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:59.811+0000 7fc917e00640 1 --2- 172.21.15.77:0/1233912988 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc918109140 0x7fc91819ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:59.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:59.811+0000 7fc917e00640 1 -- 172.21.15.77:0/1233912988 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc918104010 msgr2=0x7fc91819a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:59.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:59.811+0000 7fc917e00640 1 --2- 172.21.15.77:0/1233912988 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc918104010 0x7fc91819a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:59.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:59.811+0000 7fc917e00640 1 -- 172.21.15.77:0/1233912988 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc91819f670 con 0x7fc9181049c0 2024-09-10T09:51:59.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:59.811+0000 7fc91ce00640 1 --2- 172.21.15.77:0/1233912988 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc918104010 0x7fc91819a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:51:59.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:59.811+0000 7fc917e00640 1 --2- 172.21.15.77:0/1233912988 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc9181049c0 0x7fc91819ac00 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc90400e9f0 tx=0x7fc90400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:59.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:59.811+0000 7fc915600640 1 -- 172.21.15.77:0/1233912988 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc90400ccc0 con 0x7fc9181049c0 2024-09-10T09:51:59.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:59.811+0000 7fc91f800640 1 -- 172.21.15.77:0/1233912988 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc91819f960 con 0x7fc9181049c0 2024-09-10T09:51:59.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:59.811+0000 7fc915600640 1 -- 172.21.15.77:0/1233912988 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc904005020 con 0x7fc9181049c0 2024-09-10T09:51:59.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:59.811+0000 7fc915600640 1 -- 172.21.15.77:0/1233912988 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9040049e0 con 0x7fc9181049c0 2024-09-10T09:51:59.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:59.812+0000 7fc91f800640 1 -- 172.21.15.77:0/1233912988 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9181a7240 con 0x7fc9181049c0 2024-09-10T09:51:59.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:59.813+0000 7fc91f800640 1 -- 172.21.15.77:0/1233912988 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9181043f0 con 0x7fc9181049c0 2024-09-10T09:51:59.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:59.814+0000 7fc915600640 1 -- 172.21.15.77:0/1233912988 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fc904005560 con 0x7fc9181049c0 2024-09-10T09:51:59.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:59.816+0000 7fc915600640 1 --2- 172.21.15.77:0/1233912988 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fc9000a2b80 0x7fc9000a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:59.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:59.816+0000 7fc915600640 1 -- 172.21.15.77:0/1233912988 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7fc9040ceb30 con 0x7fc9181049c0 2024-09-10T09:51:59.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:59.816+0000 7fc91ce00640 1 --2- 172.21.15.77:0/1233912988 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fc9000a2b80 0x7fc9000a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:59.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:59.817+0000 7fc91ce00640 1 --2- 172.21.15.77:0/1233912988 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fc9000a2b80 0x7fc9000a5040 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc90c0095f0 tx=0x7fc90c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:59.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:51:59.820+0000 7fc915600640 1 -- 172.21.15.77:0/1233912988 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc9040051d0 con 0x7fc9181049c0 2024-09-10T09:52:00.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:00.038+0000 7fc91f800640 1 -- 172.21.15.77:0/1233912988 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc91810bdf0 con 0x7fc9181049c0 2024-09-10T09:52:00.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:00.039+0000 7fc915600640 1 -- 172.21.15.77:0/1233912988 <== mon.2 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7228 (secure 0 0 0) 0x7fc90409be90 con 0x7fc9181049c0 2024-09-10T09:52:00.041 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:52:00.041 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":9,"btime":"2024-09-10T09:51:57:759692+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3159289675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3159289675},{"type":"v1","addr":"172.21.15.77:6835","nonce":3159289675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1951962341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1951962341},{"type":"v1","addr":"172.21.15.77:6837","nonce":1951962341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/1005054423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1005054423},{"type":"v1","addr":"172.21.15.134:6833","nonce":1005054423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/3716307744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":3716307744},{"type":"v1","addr":"172.21.15.120:6833","nonce":3716307744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/1319892544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1319892544},{"type":"v1","addr":"172.21.15.120:6835","nonce":1319892544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24496,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/3293933699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3293933699},{"type":"v1","addr":"172.21.15.134:6835","nonce":3293933699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24502,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/2461497091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2461497091},{"type":"v1","addr":"172.21.15.134:6837","nonce":2461497091}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/3033280709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3033280709},{"type":"v1","addr":"172.21.15.120:6837","nonce":3033280709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-10T09:52:00.041 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 9 2024-09-10T09:52:00.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:00.041+0000 7fc91f800640 1 -- 172.21.15.77:0/1233912988 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fc9000a2b80 msgr2=0x7fc9000a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:00.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:00.041+0000 7fc91f800640 1 --2- 172.21.15.77:0/1233912988 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fc9000a2b80 0x7fc9000a5040 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc90c0095f0 tx=0x7fc90c005f90 comp rx=0 tx=0).stop 2024-09-10T09:52:00.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:00.041+0000 7fc91f800640 1 -- 172.21.15.77:0/1233912988 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc9181049c0 msgr2=0x7fc91819ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:00.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:00.041+0000 7fc91f800640 1 --2- 172.21.15.77:0/1233912988 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc9181049c0 0x7fc91819ac00 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc90400e9f0 tx=0x7fc90400eeb0 comp rx=0 tx=0).stop 2024-09-10T09:52:00.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:00.041+0000 7fc91f800640 1 -- 172.21.15.77:0/1233912988 shutdown_connections 2024-09-10T09:52:00.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:00.041+0000 7fc91f800640 1 --2- 172.21.15.77:0/1233912988 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fc9000a2b80 0x7fc9000a5040 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:00.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:00.041+0000 7fc91f800640 1 --2- 172.21.15.77:0/1233912988 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc918109140 0x7fc91819ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:00.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:00.041+0000 7fc91f800640 1 --2- 172.21.15.77:0/1233912988 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc9181049c0 0x7fc91819ac00 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:00.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:00.041+0000 7fc91f800640 1 --2- 172.21.15.77:0/1233912988 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc918104010 0x7fc91819a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:00.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:00.042+0000 7fc91f800640 1 -- 172.21.15.77:0/1233912988 >> 172.21.15.77:0/1233912988 conn(0x7fc9180ffa50 msgr2=0x7fc918100db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:00.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:00.042+0000 7fc91f800640 1 -- 172.21.15.77:0/1233912988 shutdown_connections 2024-09-10T09:52:00.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:00.042+0000 7fc91f800640 1 -- 172.21.15.77:0/1233912988 wait complete. 2024-09-10T09:52:00.075 INFO:journalctl@ceph.mds.i.smithi134.stdout:Sep 10 09:51:59 smithi134 systemd[1]: Starting Ceph mds.i for 14a55226-6f59-11ef-bcea-c7b262605968... 2024-09-10T09:52:00.370 INFO:journalctl@ceph.mds.i.smithi134.stdout:Sep 10 09:52:00 smithi134 podman[67162]: 2024-09-10 09:51:59.976930344 +0000 UTC m=+0.016263183 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T09:52:00.370 INFO:journalctl@ceph.mds.i.smithi134.stdout:Sep 10 09:52:00 smithi134 podman[67162]: 2024-09-10 09:52:00.108131509 +0000 UTC m=+0.147464340 container create 35a8e19d47456f286e9b6bf0c764fb254a082a8d381d58363cc581e45b94f2dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-i, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True) 2024-09-10T09:52:00.370 INFO:journalctl@ceph.mds.i.smithi134.stdout:Sep 10 09:52:00 smithi134 podman[67162]: 2024-09-10 09:52:00.369571584 +0000 UTC m=+0.408904424 container init 35a8e19d47456f286e9b6bf0c764fb254a082a8d381d58363cc581e45b94f2dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-i, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-10T09:52:00.631 INFO:journalctl@ceph.mds.i.smithi134.stdout:Sep 10 09:52:00 smithi134 podman[67162]: 2024-09-10 09:52:00.372434509 +0000 UTC m=+0.411767354 container start 35a8e19d47456f286e9b6bf0c764fb254a082a8d381d58363cc581e45b94f2dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-i, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T09:52:00.631 INFO:journalctl@ceph.mds.i.smithi134.stdout:Sep 10 09:52:00 smithi134 ceph-mds[67181]: -- 172.21.15.134:0/1620921267 <== mon.1 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5568d84ab860 con 0x5568d8488800 2024-09-10T09:52:00.631 INFO:journalctl@ceph.mds.i.smithi134.stdout:Sep 10 09:52:00 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-i[67177]: starting mds.i at 2024-09-10T09:52:00.631 INFO:journalctl@ceph.mds.i.smithi134.stdout:Sep 10 09:52:00 smithi134 bash[67162]: 35a8e19d47456f286e9b6bf0c764fb254a082a8d381d58363cc581e45b94f2dd 2024-09-10T09:52:00.631 INFO:journalctl@ceph.mds.i.smithi134.stdout:Sep 10 09:52:00 smithi134 systemd[1]: Started Ceph mds.i for 14a55226-6f59-11ef-bcea-c7b262605968. 2024-09-10T09:52:00.883 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:00 smithi077 ceph-mon[30319]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:52:00.884 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:00 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/1233912988' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:52:00.884 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:00 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:52:00.884 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:00 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:52:00.884 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:00 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:52:00.884 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:00 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T09:52:00.884 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:00 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T09:52:00.884 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:00 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:52:01.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:00 smithi134 ceph-mon[35674]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:52:01.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:00 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/1233912988' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:52:01.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:00 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:52:01.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:00 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:52:01.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:00 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:52:01.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:00 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T09:52:01.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:00 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T09:52:01.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:00 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:52:01.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:00 smithi120 ceph-mon[34770]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:52:01.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:00 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/1233912988' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:52:01.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:00 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:52:01.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:00 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:52:01.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:00 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:52:01.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:00 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T09:52:01.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:00 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T09:52:01.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:00 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:52:02.044 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:01 smithi077 ceph-mon[30319]: Deploying daemon mds.g on smithi077 2024-09-10T09:52:02.045 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:01 smithi077 ceph-mon[30319]: mds.? [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] up:boot 2024-09-10T09:52:02.045 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:01 smithi077 ceph-mon[30319]: fsmap 9 up:standby 2024-09-10T09:52:02.045 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:01 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-10T09:52:02.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:01 smithi134 ceph-mon[35674]: Deploying daemon mds.g on smithi077 2024-09-10T09:52:02.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:01 smithi134 ceph-mon[35674]: mds.? [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] up:boot 2024-09-10T09:52:02.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:01 smithi134 ceph-mon[35674]: fsmap 9 up:standby 2024-09-10T09:52:02.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:01 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-10T09:52:02.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:01 smithi120 ceph-mon[34770]: Deploying daemon mds.g on smithi077 2024-09-10T09:52:02.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:01 smithi120 ceph-mon[34770]: mds.? [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] up:boot 2024-09-10T09:52:02.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:01 smithi120 ceph-mon[34770]: fsmap 9 up:standby 2024-09-10T09:52:02.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:01 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-10T09:52:02.926 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:52:02.999 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:02 smithi077 ceph-mon[30319]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:52:03.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:02 smithi134 ceph-mon[35674]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:52:03.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:02 smithi120 ceph-mon[34770]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:52:03.359 INFO:journalctl@ceph.mds.g.smithi077.stdout:Sep 10 09:52:03 smithi077 systemd[1]: Starting Ceph mds.g for 14a55226-6f59-11ef-bcea-c7b262605968... 2024-09-10T09:52:03.680 INFO:journalctl@ceph.mds.g.smithi077.stdout:Sep 10 09:52:03 smithi077 podman[70484]: 2024-09-10 09:52:03.262455095 +0000 UTC m=+0.027561897 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T09:52:03.680 INFO:journalctl@ceph.mds.g.smithi077.stdout:Sep 10 09:52:03 smithi077 podman[70484]: 2024-09-10 09:52:03.38495601 +0000 UTC m=+0.150062841 container create 289cc8276997f59b4128aca7bd0d7dade1ef669fb28373c8bc7f212e388de7ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-g, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-10T09:52:03.680 INFO:journalctl@ceph.mds.g.smithi077.stdout:Sep 10 09:52:03 smithi077 podman[70484]: 2024-09-10 09:52:03.679384669 +0000 UTC m=+0.444491460 container init 289cc8276997f59b4128aca7bd0d7dade1ef669fb28373c8bc7f212e388de7ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-g, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T09:52:03.881 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:52:03.939 INFO:journalctl@ceph.mds.g.smithi077.stdout:Sep 10 09:52:03 smithi077 podman[70484]: 2024-09-10 09:52:03.686093074 +0000 UTC m=+0.451199865 container start 289cc8276997f59b4128aca7bd0d7dade1ef669fb28373c8bc7f212e388de7ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-g, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-10T09:52:03.939 INFO:journalctl@ceph.mds.g.smithi077.stdout:Sep 10 09:52:03 smithi077 ceph-mds[70515]: -- 172.21.15.77:0/13299828 <== mon.0 v2:172.21.15.77:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d8483ef860 con 0x55d8483cd000 2024-09-10T09:52:03.939 INFO:journalctl@ceph.mds.g.smithi077.stdout:Sep 10 09:52:03 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-g[70511]: starting mds.g at 2024-09-10T09:52:03.939 INFO:journalctl@ceph.mds.g.smithi077.stdout:Sep 10 09:52:03 smithi077 bash[70484]: 289cc8276997f59b4128aca7bd0d7dade1ef669fb28373c8bc7f212e388de7ee 2024-09-10T09:52:03.939 INFO:journalctl@ceph.mds.g.smithi077.stdout:Sep 10 09:52:03 smithi077 systemd[1]: Started Ceph mds.g for 14a55226-6f59-11ef-bcea-c7b262605968. 2024-09-10T09:52:04.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:04.491+0000 7ff2aa000640 1 -- 172.21.15.77:0/1575588705 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff2a4074620 msgr2=0x7ff2a4074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:04.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:04.491+0000 7ff2aa000640 1 --2- 172.21.15.77:0/1575588705 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff2a4074620 0x7ff2a4074a00 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff294009900 tx=0x7ff29402a480 comp rx=0 tx=0).stop 2024-09-10T09:52:04.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:04.492+0000 7ff2aa000640 1 -- 172.21.15.77:0/1575588705 shutdown_connections 2024-09-10T09:52:04.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:04.492+0000 7ff2aa000640 1 --2- 172.21.15.77:0/1575588705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff2a410e3c0 0x7ff2a4110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:04.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:04.492+0000 7ff2aa000640 1 --2- 172.21.15.77:0/1575588705 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff2a4074f40 0x7ff2a410de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:04.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:04.492+0000 7ff2aa000640 1 --2- 172.21.15.77:0/1575588705 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff2a4074620 0x7ff2a4074a00 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:04.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:04.492+0000 7ff2aa000640 1 -- 172.21.15.77:0/1575588705 >> 172.21.15.77:0/1575588705 conn(0x7ff2a40fd8c0 msgr2=0x7ff2a40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:04.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:04.492+0000 7ff2aa000640 1 -- 172.21.15.77:0/1575588705 shutdown_connections 2024-09-10T09:52:04.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:04.492+0000 7ff2aa000640 1 -- 172.21.15.77:0/1575588705 wait complete. 2024-09-10T09:52:04.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:04.493+0000 7ff2aa000640 1 Processor -- start 2024-09-10T09:52:04.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:04.493+0000 7ff2aa000640 1 -- start start 2024-09-10T09:52:04.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:04.493+0000 7ff2aa000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff2a4074620 0x7ff2a419a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:04.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:04.493+0000 7ff2aa000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff2a4074f40 0x7ff2a419abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:04.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:04.493+0000 7ff2aa000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff2a410e3c0 0x7ff2a419ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:04.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:04.493+0000 7ff2aa000640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a4114150 con 0x7ff2a4074620 2024-09-10T09:52:04.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:04.493+0000 7ff2aa000640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a4113fd0 con 0x7ff2a4074f40 2024-09-10T09:52:04.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:04.493+0000 7ff2aa000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a41142d0 con 0x7ff2a410e3c0 2024-09-10T09:52:04.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:04.493+0000 7ff2a3400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff2a4074620 0x7ff2a419a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:04.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:04.493+0000 7ff2a3400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff2a4074620 0x7ff2a419a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50978/0 (socket says 172.21.15.77:50978) 2024-09-10T09:52:04.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:04.493+0000 7ff2a3400640 1 -- 172.21.15.77:0/433632633 learned_addr learned my addr 172.21.15.77:0/433632633 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:52:04.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:04.493+0000 7ff2a2a00640 1 --2- 172.21.15.77:0/433632633 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff2a4074f40 0x7ff2a419abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:04.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:04.493+0000 7ff2a3e00640 1 --2- 172.21.15.77:0/433632633 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff2a410e3c0 0x7ff2a419ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:04.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:04.494+0000 7ff2a3400640 1 -- 172.21.15.77:0/433632633 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff2a410e3c0 msgr2=0x7ff2a419ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:04.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:04.494+0000 7ff2a3400640 1 --2- 172.21.15.77:0/433632633 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff2a410e3c0 0x7ff2a419ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:04.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:04.494+0000 7ff2a3400640 1 -- 172.21.15.77:0/433632633 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff2a4074f40 msgr2=0x7ff2a419abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:04.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:04.494+0000 7ff2a3400640 1 --2- 172.21.15.77:0/433632633 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff2a4074f40 0x7ff2a419abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:04.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:04.494+0000 7ff2a3400640 1 -- 172.21.15.77:0/433632633 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2a419f650 con 0x7ff2a4074620 2024-09-10T09:52:04.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:04.494+0000 7ff2a3e00640 1 --2- 172.21.15.77:0/433632633 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff2a410e3c0 0x7ff2a419ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:52:04.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:04.494+0000 7ff2a3400640 1 --2- 172.21.15.77:0/433632633 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff2a4074620 0x7ff2a419a6a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff294006010 tx=0x7ff2940042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:04.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:04.494+0000 7ff29be00640 1 -- 172.21.15.77:0/433632633 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff29403f030 con 0x7ff2a4074620 2024-09-10T09:52:04.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:04.494+0000 7ff29be00640 1 -- 172.21.15.77:0/433632633 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff294003ce0 con 0x7ff2a4074620 2024-09-10T09:52:04.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:04.494+0000 7ff29be00640 1 -- 172.21.15.77:0/433632633 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff294035070 con 0x7ff2a4074620 2024-09-10T09:52:04.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:04.494+0000 7ff2aa000640 1 -- 172.21.15.77:0/433632633 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2a419f8e0 con 0x7ff2a4074620 2024-09-10T09:52:04.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:04.495+0000 7ff2aa000640 1 -- 172.21.15.77:0/433632633 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2a419fd70 con 0x7ff2a4074620 2024-09-10T09:52:04.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:04.496+0000 7ff2aa000640 1 -- 172.21.15.77:0/433632633 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2a4074a00 con 0x7ff2a4074620 2024-09-10T09:52:04.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:04.497+0000 7ff29be00640 1 -- 172.21.15.77:0/433632633 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff294042030 con 0x7ff2a4074620 2024-09-10T09:52:04.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:04.499+0000 7ff29be00640 1 --2- 172.21.15.77:0/433632633 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ff2880a2c00 0x7ff2880a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:04.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:04.499+0000 7ff29be00640 1 -- 172.21.15.77:0/433632633 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7ff2940ef100 con 0x7ff2a4074620 2024-09-10T09:52:04.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:04.499+0000 7ff2a2a00640 1 --2- 172.21.15.77:0/433632633 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ff2880a2c00 0x7ff2880a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:04.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:04.499+0000 7ff2a2a00640 1 --2- 172.21.15.77:0/433632633 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ff2880a2c00 0x7ff2880a50c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff2a410f8b0 tx=0x7ff29000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:04.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:04.503+0000 7ff29be00640 1 -- 172.21.15.77:0/433632633 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff2940bc520 con 0x7ff2a4074620 2024-09-10T09:52:04.751 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:04.750+0000 7ff2aa000640 1 -- 172.21.15.77:0/433632633 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff2a402d7c0 con 0x7ff2a4074620 2024-09-10T09:52:04.751 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:04.751+0000 7ff29be00640 1 -- 172.21.15.77:0/433632633 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8884 (secure 0 0 0) 0x7ff2940bbe20 con 0x7ff2a4074620 2024-09-10T09:52:04.752 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:52:04.752 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":11,"btime":"2024-09-10T09:52:03:779574+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3159289675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3159289675},{"type":"v1","addr":"172.21.15.77:6835","nonce":3159289675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1951962341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1951962341},{"type":"v1","addr":"172.21.15.77:6837","nonce":1951962341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14487,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6839/4291121888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":4291121888},{"type":"v1","addr":"172.21.15.77:6839","nonce":4291121888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/1005054423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1005054423},{"type":"v1","addr":"172.21.15.134:6833","nonce":1005054423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/3716307744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":3716307744},{"type":"v1","addr":"172.21.15.120:6833","nonce":3716307744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/1319892544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1319892544},{"type":"v1","addr":"172.21.15.120:6835","nonce":1319892544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24496,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/3293933699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3293933699},{"type":"v1","addr":"172.21.15.134:6835","nonce":3293933699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24502,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/2461497091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2461497091},{"type":"v1","addr":"172.21.15.134:6837","nonce":2461497091}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24520,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6839/1965117680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1965117680},{"type":"v1","addr":"172.21.15.134:6839","nonce":1965117680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/3033280709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3033280709},{"type":"v1","addr":"172.21.15.120:6837","nonce":3033280709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-10T09:52:04.752 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 11 2024-09-10T09:52:04.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:04.753+0000 7ff2aa000640 1 -- 172.21.15.77:0/433632633 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ff2880a2c00 msgr2=0x7ff2880a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:04.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:04.753+0000 7ff2aa000640 1 --2- 172.21.15.77:0/433632633 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ff2880a2c00 0x7ff2880a50c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff2a410f8b0 tx=0x7ff29000b410 comp rx=0 tx=0).stop 2024-09-10T09:52:04.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:04.753+0000 7ff2aa000640 1 -- 172.21.15.77:0/433632633 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff2a4074620 msgr2=0x7ff2a419a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:04.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:04.753+0000 7ff2aa000640 1 --2- 172.21.15.77:0/433632633 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff2a4074620 0x7ff2a419a6a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff294006010 tx=0x7ff2940042f0 comp rx=0 tx=0).stop 2024-09-10T09:52:04.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:04.753+0000 7ff2aa000640 1 -- 172.21.15.77:0/433632633 shutdown_connections 2024-09-10T09:52:04.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:04.753+0000 7ff2aa000640 1 --2- 172.21.15.77:0/433632633 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ff2880a2c00 0x7ff2880a50c0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:04.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:04.753+0000 7ff2aa000640 1 --2- 172.21.15.77:0/433632633 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff2a410e3c0 0x7ff2a419ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:04.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:04.753+0000 7ff2aa000640 1 --2- 172.21.15.77:0/433632633 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff2a4074f40 0x7ff2a419abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:04.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:04.753+0000 7ff2aa000640 1 --2- 172.21.15.77:0/433632633 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff2a4074620 0x7ff2a419a6a0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:04.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:04.753+0000 7ff2aa000640 1 -- 172.21.15.77:0/433632633 >> 172.21.15.77:0/433632633 conn(0x7ff2a40fd8c0 msgr2=0x7ff2a40fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:04.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:04.753+0000 7ff2aa000640 1 -- 172.21.15.77:0/433632633 shutdown_connections 2024-09-10T09:52:04.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:04.753+0000 7ff2aa000640 1 -- 172.21.15.77:0/433632633 wait complete. 2024-09-10T09:52:05.054 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:04 smithi077 ceph-mon[30319]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:52:05.054 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:04 smithi077 ceph-mon[30319]: mds.? [v2:172.21.15.77:6838/4291121888,v1:172.21.15.77:6839/4291121888] up:boot 2024-09-10T09:52:05.054 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:04 smithi077 ceph-mon[30319]: fsmap 10 up:standby 2024-09-10T09:52:05.054 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:04 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-10T09:52:05.054 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:04 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:52:05.055 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:04 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:52:05.055 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:04 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:52:05.055 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:04 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T09:52:05.055 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:04 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T09:52:05.055 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:04 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:52:05.055 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:04 smithi077 ceph-mon[30319]: Deploying daemon mds.a on smithi077 2024-09-10T09:52:05.055 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:04 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/433632633' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:52:05.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:04 smithi134 ceph-mon[35674]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:52:05.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:04 smithi134 ceph-mon[35674]: mds.? [v2:172.21.15.77:6838/4291121888,v1:172.21.15.77:6839/4291121888] up:boot 2024-09-10T09:52:05.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:04 smithi134 ceph-mon[35674]: fsmap 10 up:standby 2024-09-10T09:52:05.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:04 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-10T09:52:05.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:04 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:52:05.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:04 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:52:05.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:04 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:52:05.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:04 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T09:52:05.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:04 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T09:52:05.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:04 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:52:05.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:04 smithi134 ceph-mon[35674]: Deploying daemon mds.a on smithi077 2024-09-10T09:52:05.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:04 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/433632633' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:52:05.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:04 smithi120 ceph-mon[34770]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:52:05.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:04 smithi120 ceph-mon[34770]: mds.? [v2:172.21.15.77:6838/4291121888,v1:172.21.15.77:6839/4291121888] up:boot 2024-09-10T09:52:05.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:04 smithi120 ceph-mon[34770]: fsmap 10 up:standby 2024-09-10T09:52:05.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:04 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-10T09:52:05.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:04 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:52:05.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:04 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:52:05.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:04 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:52:05.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:04 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T09:52:05.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:04 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T09:52:05.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:04 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:52:05.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:04 smithi120 ceph-mon[34770]: Deploying daemon mds.a on smithi077 2024-09-10T09:52:05.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:04 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/433632633' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:52:07.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:06 smithi134 ceph-mon[35674]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:52:07.165 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:06 smithi077 ceph-mon[30319]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:52:07.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:06 smithi120 ceph-mon[34770]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:52:07.752 INFO:journalctl@ceph.mds.a.smithi077.stdout:Sep 10 09:52:07 smithi077 systemd[1]: Starting Ceph mds.a for 14a55226-6f59-11ef-bcea-c7b262605968... 2024-09-10T09:52:08.053 INFO:journalctl@ceph.mds.a.smithi077.stdout:Sep 10 09:52:07 smithi077 podman[70865]: 2024-09-10 09:52:07.655363253 +0000 UTC m=+0.026963163 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T09:52:08.053 INFO:journalctl@ceph.mds.a.smithi077.stdout:Sep 10 09:52:07 smithi077 podman[70865]: 2024-09-10 09:52:07.774718773 +0000 UTC m=+0.146318676 container create 465fb4b642d38bf0e9dc99e652b20257d33bb7e915b79a1bb81ac473cf2126cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-a, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T09:52:08.053 INFO:journalctl@ceph.mds.a.smithi077.stdout:Sep 10 09:52:08 smithi077 podman[70865]: 2024-09-10 09:52:08.052316269 +0000 UTC m=+0.423916173 container init 465fb4b642d38bf0e9dc99e652b20257d33bb7e915b79a1bb81ac473cf2126cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-a, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-10T09:52:08.264 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:52:08.316 INFO:journalctl@ceph.mds.a.smithi077.stdout:Sep 10 09:52:08 smithi077 podman[70865]: 2024-09-10 09:52:08.058636094 +0000 UTC m=+0.430235990 container start 465fb4b642d38bf0e9dc99e652b20257d33bb7e915b79a1bb81ac473cf2126cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-a, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T09:52:08.317 INFO:journalctl@ceph.mds.a.smithi077.stdout:Sep 10 09:52:08 smithi077 ceph-mds[70884]: -- 172.21.15.77:0/3451521580 <== mon.0 v2:172.21.15.77:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5559a4d93860 con 0x5559a4d71000 2024-09-10T09:52:08.317 INFO:journalctl@ceph.mds.a.smithi077.stdout:Sep 10 09:52:08 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-a[70880]: starting mds.a at 2024-09-10T09:52:08.317 INFO:journalctl@ceph.mds.a.smithi077.stdout:Sep 10 09:52:08 smithi077 bash[70865]: 465fb4b642d38bf0e9dc99e652b20257d33bb7e915b79a1bb81ac473cf2126cd 2024-09-10T09:52:08.317 INFO:journalctl@ceph.mds.a.smithi077.stdout:Sep 10 09:52:08 smithi077 systemd[1]: Started Ceph mds.a for 14a55226-6f59-11ef-bcea-c7b262605968. 2024-09-10T09:52:08.641 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:52:09.018 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:08 smithi120 ceph-mon[34770]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:52:09.018 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:08 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:52:09.018 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:08 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:52:09.018 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:08 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:52:09.018 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:08 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T09:52:09.019 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:08 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T09:52:09.019 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:08 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:52:09.064 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:08 smithi077 ceph-mon[30319]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:52:09.065 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:08 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:52:09.065 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:08 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:52:09.065 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:08 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:52:09.065 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:08 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T09:52:09.065 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:08 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T09:52:09.065 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:08 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:52:09.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:08 smithi134 ceph-mon[35674]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:52:09.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:08 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:52:09.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:08 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:52:09.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:08 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:52:09.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:08 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T09:52:09.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:08 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T09:52:09.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:08 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:52:09.206 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:09.175+0000 7f5027000640 1 -- 172.21.15.77:0/2709903874 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f50200745c0 msgr2=0x7f50200749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:09.206 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:09.175+0000 7f5027000640 1 --2- 172.21.15.77:0/2709903874 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f50200745c0 0x7f50200749a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5014009900 tx=0x7f501402a480 comp rx=0 tx=0).stop 2024-09-10T09:52:09.206 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:09.175+0000 7f5027000640 1 -- 172.21.15.77:0/2709903874 shutdown_connections 2024-09-10T09:52:09.206 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:09.175+0000 7f5027000640 1 --2- 172.21.15.77:0/2709903874 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f502010e260 0x7f5020110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:09.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:09.175+0000 7f5027000640 1 --2- 172.21.15.77:0/2709903874 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5020074ee0 0x7f502010dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:09.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:09.175+0000 7f5027000640 1 --2- 172.21.15.77:0/2709903874 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f50200745c0 0x7f50200749a0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:09.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:09.175+0000 7f5027000640 1 -- 172.21.15.77:0/2709903874 >> 172.21.15.77:0/2709903874 conn(0x7f50200fd740 msgr2=0x7f50200ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:09.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:09.175+0000 7f5027000640 1 -- 172.21.15.77:0/2709903874 shutdown_connections 2024-09-10T09:52:09.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:09.175+0000 7f5027000640 1 -- 172.21.15.77:0/2709903874 wait complete. 2024-09-10T09:52:09.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:09.176+0000 7f5027000640 1 Processor -- start 2024-09-10T09:52:09.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:09.176+0000 7f5027000640 1 -- start start 2024-09-10T09:52:09.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:09.177+0000 7f5027000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f50200745c0 0x7f502019a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:09.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:09.177+0000 7f5027000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5020074ee0 0x7f502019aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:09.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:09.177+0000 7f5027000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f502010e260 0x7f502019edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:09.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:09.177+0000 7f5027000640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5020114000 con 0x7f502010e260 2024-09-10T09:52:09.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:09.177+0000 7f5027000640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5020113e80 con 0x7f50200745c0 2024-09-10T09:52:09.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:09.177+0000 7f5027000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5020114180 con 0x7f5020074ee0 2024-09-10T09:52:09.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:09.177+0000 7f5026600640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f502010e260 0x7f502019edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:09.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:09.177+0000 7f5025c00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f50200745c0 0x7f502019a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:09.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:09.177+0000 7f5025200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5020074ee0 0x7f502019aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:09.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:09.177+0000 7f5026600640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f502010e260 0x7f502019edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:51022/0 (socket says 172.21.15.77:51022) 2024-09-10T09:52:09.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:09.177+0000 7f5026600640 1 -- 172.21.15.77:0/1164154056 learned_addr learned my addr 172.21.15.77:0/1164154056 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:52:09.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:09.177+0000 7f5025c00640 1 -- 172.21.15.77:0/1164154056 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5020074ee0 msgr2=0x7f502019aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:09.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:09.177+0000 7f5025c00640 1 --2- 172.21.15.77:0/1164154056 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5020074ee0 0x7f502019aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:09.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:09.177+0000 7f5025c00640 1 -- 172.21.15.77:0/1164154056 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f502010e260 msgr2=0x7f502019edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:09.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:09.177+0000 7f5025c00640 1 --2- 172.21.15.77:0/1164154056 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f502010e260 0x7f502019edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:09.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:09.177+0000 7f5025c00640 1 -- 172.21.15.77:0/1164154056 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f502019f490 con 0x7f50200745c0 2024-09-10T09:52:09.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:09.177+0000 7f5026600640 1 --2- 172.21.15.77:0/1164154056 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f502010e260 0x7f502019edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:52:09.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:09.178+0000 7f5025c00640 1 --2- 172.21.15.77:0/1164154056 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f50200745c0 0x7f502019a4e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f501402a9a0 tx=0x7f5014003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:09.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:09.178+0000 7f501e000640 1 -- 172.21.15.77:0/1164154056 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f501403f070 con 0x7f50200745c0 2024-09-10T09:52:09.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:09.178+0000 7f501e000640 1 -- 172.21.15.77:0/1164154056 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5014031c90 con 0x7f50200745c0 2024-09-10T09:52:09.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:09.178+0000 7f5027000640 1 -- 172.21.15.77:0/1164154056 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f502019f720 con 0x7f50200745c0 2024-09-10T09:52:09.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:09.178+0000 7f501e000640 1 -- 172.21.15.77:0/1164154056 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5014035030 con 0x7f50200745c0 2024-09-10T09:52:09.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:09.178+0000 7f5027000640 1 -- 172.21.15.77:0/1164154056 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50201a7060 con 0x7f50200745c0 2024-09-10T09:52:09.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:09.179+0000 7f5027000640 1 -- 172.21.15.77:0/1164154056 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5020075c70 con 0x7f50200745c0 2024-09-10T09:52:09.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:09.181+0000 7f501e000640 1 -- 172.21.15.77:0/1164154056 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f5014042070 con 0x7f50200745c0 2024-09-10T09:52:09.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:09.182+0000 7f501e000640 1 --2- 172.21.15.77:0/1164154056 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f50080a2b80 0x7f50080a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:09.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:09.183+0000 7f501e000640 1 -- 172.21.15.77:0/1164154056 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7f50140ef780 con 0x7f50200745c0 2024-09-10T09:52:09.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:09.183+0000 7f5025200640 1 --2- 172.21.15.77:0/1164154056 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f50080a2b80 0x7f50080a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:09.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:09.183+0000 7f5025200640 1 --2- 172.21.15.77:0/1164154056 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f50080a2b80 0x7f50080a5040 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f502010f660 tx=0x7f500c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:09.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:09.184+0000 7f501e000640 1 -- 172.21.15.77:0/1164154056 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f50140bc490 con 0x7f50200745c0 2024-09-10T09:52:09.395 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:09.394+0000 7f5027000640 1 -- 172.21.15.77:0/1164154056 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f50201093e0 con 0x7f50200745c0 2024-09-10T09:52:09.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:09.395+0000 7f501e000640 1 -- 172.21.15.77:0/1164154056 <== mon.2 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9707 (secure 0 0 0) 0x7f50140c0790 con 0x7f50200745c0 2024-09-10T09:52:09.396 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:52:09.396 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":12,"btime":"2024-09-10T09:52:08:792535+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3159289675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3159289675},{"type":"v1","addr":"172.21.15.77:6835","nonce":3159289675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1951962341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1951962341},{"type":"v1","addr":"172.21.15.77:6837","nonce":1951962341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14487,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6839/4291121888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":4291121888},{"type":"v1","addr":"172.21.15.77:6839","nonce":4291121888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14499,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6841/445284194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":445284194},{"type":"v1","addr":"172.21.15.77:6841","nonce":445284194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/1005054423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1005054423},{"type":"v1","addr":"172.21.15.134:6833","nonce":1005054423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/3716307744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":3716307744},{"type":"v1","addr":"172.21.15.120:6833","nonce":3716307744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/1319892544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1319892544},{"type":"v1","addr":"172.21.15.120:6835","nonce":1319892544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24496,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/3293933699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3293933699},{"type":"v1","addr":"172.21.15.134:6835","nonce":3293933699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24502,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/2461497091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2461497091},{"type":"v1","addr":"172.21.15.134:6837","nonce":2461497091}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24520,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6839/1965117680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1965117680},{"type":"v1","addr":"172.21.15.134:6839","nonce":1965117680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/3033280709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3033280709},{"type":"v1","addr":"172.21.15.120:6837","nonce":3033280709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-10T09:52:09.396 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 12 2024-09-10T09:52:09.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:09.398+0000 7f5027000640 1 -- 172.21.15.77:0/1164154056 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f50080a2b80 msgr2=0x7f50080a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:09.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:09.398+0000 7f5027000640 1 --2- 172.21.15.77:0/1164154056 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f50080a2b80 0x7f50080a5040 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f502010f660 tx=0x7f500c00b390 comp rx=0 tx=0).stop 2024-09-10T09:52:09.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:09.398+0000 7f5027000640 1 -- 172.21.15.77:0/1164154056 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f50200745c0 msgr2=0x7f502019a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:09.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:09.398+0000 7f5027000640 1 --2- 172.21.15.77:0/1164154056 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f50200745c0 0x7f502019a4e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f501402a9a0 tx=0x7f5014003b90 comp rx=0 tx=0).stop 2024-09-10T09:52:09.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:09.398+0000 7f5027000640 1 -- 172.21.15.77:0/1164154056 shutdown_connections 2024-09-10T09:52:09.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:09.398+0000 7f5027000640 1 --2- 172.21.15.77:0/1164154056 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f50080a2b80 0x7f50080a5040 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:09.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:09.398+0000 7f5027000640 1 --2- 172.21.15.77:0/1164154056 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f502010e260 0x7f502019edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:09.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:09.398+0000 7f5027000640 1 --2- 172.21.15.77:0/1164154056 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5020074ee0 0x7f502019aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:09.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:09.398+0000 7f5027000640 1 --2- 172.21.15.77:0/1164154056 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f50200745c0 0x7f502019a4e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:09.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:09.398+0000 7f5027000640 1 -- 172.21.15.77:0/1164154056 >> 172.21.15.77:0/1164154056 conn(0x7f50200fd740 msgr2=0x7f50200fe440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:09.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:09.398+0000 7f5027000640 1 -- 172.21.15.77:0/1164154056 shutdown_connections 2024-09-10T09:52:09.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:09.398+0000 7f5027000640 1 -- 172.21.15.77:0/1164154056 wait complete. 2024-09-10T09:52:09.923 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:09 smithi120 ceph-mon[34770]: Deploying daemon mds.b on smithi120 2024-09-10T09:52:09.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:09 smithi120 ceph-mon[34770]: mds.? [v2:172.21.15.77:6840/445284194,v1:172.21.15.77:6841/445284194] up:boot 2024-09-10T09:52:09.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:09 smithi120 ceph-mon[34770]: fsmap 11 up:standby 2024-09-10T09:52:09.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:09 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-10T09:52:09.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:09 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/1164154056' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:52:10.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:09 smithi134 ceph-mon[35674]: Deploying daemon mds.b on smithi120 2024-09-10T09:52:10.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:09 smithi134 ceph-mon[35674]: mds.? [v2:172.21.15.77:6840/445284194,v1:172.21.15.77:6841/445284194] up:boot 2024-09-10T09:52:10.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:09 smithi134 ceph-mon[35674]: fsmap 11 up:standby 2024-09-10T09:52:10.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:09 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-10T09:52:10.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:09 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/1164154056' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:52:10.174 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:09 smithi077 ceph-mon[30319]: Deploying daemon mds.b on smithi120 2024-09-10T09:52:10.174 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:09 smithi077 ceph-mon[30319]: mds.? [v2:172.21.15.77:6840/445284194,v1:172.21.15.77:6841/445284194] up:boot 2024-09-10T09:52:10.175 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:09 smithi077 ceph-mon[30319]: fsmap 11 up:standby 2024-09-10T09:52:10.175 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:09 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-10T09:52:10.175 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:09 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/1164154056' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:52:10.359 INFO:journalctl@ceph.mds.b.smithi120.stdout:Sep 10 09:52:10 smithi120 systemd[1]: Starting Ceph mds.b for 14a55226-6f59-11ef-bcea-c7b262605968... 2024-09-10T09:52:10.655 INFO:journalctl@ceph.mds.b.smithi120.stdout:Sep 10 09:52:10 smithi120 podman[64917]: 2024-09-10 09:52:10.260850323 +0000 UTC m=+0.016230010 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T09:52:10.655 INFO:journalctl@ceph.mds.b.smithi120.stdout:Sep 10 09:52:10 smithi120 podman[64917]: 2024-09-10 09:52:10.399499503 +0000 UTC m=+0.154879185 container create cb2fab542c1d156ee13a0bcf13bd9a9fb1782e52e9e21d462c8dae3f46acc487 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-10T09:52:10.923 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:10 smithi120 ceph-mon[34770]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:52:10.923 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:10 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:52:10.923 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:10 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:52:10.924 INFO:journalctl@ceph.mds.b.smithi120.stdout:Sep 10 09:52:10 smithi120 podman[64917]: 2024-09-10 09:52:10.654760806 +0000 UTC m=+0.410140485 container init cb2fab542c1d156ee13a0bcf13bd9a9fb1782e52e9e21d462c8dae3f46acc487 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-b, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True) 2024-09-10T09:52:10.924 INFO:journalctl@ceph.mds.b.smithi120.stdout:Sep 10 09:52:10 smithi120 podman[64917]: 2024-09-10 09:52:10.657953121 +0000 UTC m=+0.413332824 container start cb2fab542c1d156ee13a0bcf13bd9a9fb1782e52e9e21d462c8dae3f46acc487 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-b, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-10T09:52:10.924 INFO:journalctl@ceph.mds.b.smithi120.stdout:Sep 10 09:52:10 smithi120 ceph-mds[64936]: -- 172.21.15.120:0/3588142744 <== mon.2 v2:172.21.15.120:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55aad254b860 con 0x55aad2529000 2024-09-10T09:52:10.924 INFO:journalctl@ceph.mds.b.smithi120.stdout:Sep 10 09:52:10 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-b[64932]: starting mds.b at 2024-09-10T09:52:10.924 INFO:journalctl@ceph.mds.b.smithi120.stdout:Sep 10 09:52:10 smithi120 bash[64917]: cb2fab542c1d156ee13a0bcf13bd9a9fb1782e52e9e21d462c8dae3f46acc487 2024-09-10T09:52:10.924 INFO:journalctl@ceph.mds.b.smithi120.stdout:Sep 10 09:52:10 smithi120 systemd[1]: Started Ceph mds.b for 14a55226-6f59-11ef-bcea-c7b262605968. 2024-09-10T09:52:11.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:10 smithi134 ceph-mon[35674]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:52:11.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:10 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:52:11.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:10 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:52:11.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:10 smithi077 ceph-mon[30319]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:52:11.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:10 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:52:11.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:10 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:52:11.848 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:11 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:52:11.848 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:11 smithi134 ceph-mon[35674]: mds.? [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] up:boot 2024-09-10T09:52:11.849 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:11 smithi134 ceph-mon[35674]: fsmap 12 up:standby 2024-09-10T09:52:11.849 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:11 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-10T09:52:11.849 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:11 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:52:11.849 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:11 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:52:11.849 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:11 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:52:12.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:11 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:52:12.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:11 smithi120 ceph-mon[34770]: mds.? [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] up:boot 2024-09-10T09:52:12.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:11 smithi120 ceph-mon[34770]: fsmap 12 up:standby 2024-09-10T09:52:12.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:11 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-10T09:52:12.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:11 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:52:12.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:11 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:52:12.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:11 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:52:12.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:11 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:52:12.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:11 smithi077 ceph-mon[30319]: mds.? [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] up:boot 2024-09-10T09:52:12.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:11 smithi077 ceph-mon[30319]: fsmap 12 up:standby 2024-09-10T09:52:12.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:11 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-10T09:52:12.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:11 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:52:12.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:11 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:52:12.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:11 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:52:12.309 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:52:12.714 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:52:13.076 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:12 smithi077 ceph-mon[30319]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:52:13.076 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:12 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:52:13.076 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:12 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:52:13.076 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:12 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:52:13.077 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:12 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:52:13.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:12 smithi134 ceph-mon[35674]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:52:13.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:12 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:52:13.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:12 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:52:13.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:12 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:52:13.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:12 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:52:13.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:12 smithi120 ceph-mon[34770]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:52:13.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:12 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:52:13.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:12 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:52:13.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:12 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:52:13.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:12 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:52:13.314 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:13.313+0000 7fc88de00640 1 -- 172.21.15.77:0/3675005683 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc888108f60 msgr2=0x7fc888110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:13.314 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:13.313+0000 7fc88de00640 1 --2- 172.21.15.77:0/3675005683 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc888108f60 0x7fc888110ae0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc880009ca0 tx=0x7fc88002a8c0 comp rx=0 tx=0).stop 2024-09-10T09:52:13.314 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:13.313+0000 7fc88de00640 1 -- 172.21.15.77:0/3675005683 shutdown_connections 2024-09-10T09:52:13.314 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:13.313+0000 7fc88de00640 1 --2- 172.21.15.77:0/3675005683 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc888108f60 0x7fc888110ae0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:13.314 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:13.313+0000 7fc88de00640 1 --2- 172.21.15.77:0/3675005683 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc8881047e0 0x7fc888108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:13.315 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:13.313+0000 7fc88de00640 1 --2- 172.21.15.77:0/3675005683 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc888103e30 0x7fc888104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:13.315 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:13.313+0000 7fc88de00640 1 -- 172.21.15.77:0/3675005683 >> 172.21.15.77:0/3675005683 conn(0x7fc8880ff8d0 msgr2=0x7fc888101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:13.315 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:13.313+0000 7fc88de00640 1 -- 172.21.15.77:0/3675005683 shutdown_connections 2024-09-10T09:52:13.315 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:13.313+0000 7fc88de00640 1 -- 172.21.15.77:0/3675005683 wait complete. 2024-09-10T09:52:13.316 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:13.314+0000 7fc88de00640 1 Processor -- start 2024-09-10T09:52:13.316 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:13.314+0000 7fc88de00640 1 -- start start 2024-09-10T09:52:13.316 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:13.315+0000 7fc88de00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc888103e30 0x7fc88819a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:13.316 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:13.315+0000 7fc88de00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc8881047e0 0x7fc88819a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:13.316 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:13.315+0000 7fc88de00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc888108f60 0x7fc88819ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:13.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:13.315+0000 7fc88de00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc888113fc0 con 0x7fc888108f60 2024-09-10T09:52:13.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:13.315+0000 7fc88de00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc888113e40 con 0x7fc8881047e0 2024-09-10T09:52:13.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:13.315+0000 7fc88de00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc888114140 con 0x7fc888103e30 2024-09-10T09:52:13.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:13.315+0000 7fc88ca00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc888103e30 0x7fc88819a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:13.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:13.315+0000 7fc88d400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc888108f60 0x7fc88819ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:13.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:13.315+0000 7fc88ca00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc888103e30 0x7fc88819a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:59448/0 (socket says 172.21.15.77:59448) 2024-09-10T09:52:13.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:13.315+0000 7fc87fe00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc8881047e0 0x7fc88819a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:13.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:13.315+0000 7fc88ca00640 1 -- 172.21.15.77:0/224474930 learned_addr learned my addr 172.21.15.77:0/224474930 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:52:13.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:13.315+0000 7fc88ca00640 1 -- 172.21.15.77:0/224474930 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc8881047e0 msgr2=0x7fc88819a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:13.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:13.315+0000 7fc88ca00640 1 --2- 172.21.15.77:0/224474930 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc8881047e0 0x7fc88819a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:13.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:13.315+0000 7fc88ca00640 1 -- 172.21.15.77:0/224474930 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc888108f60 msgr2=0x7fc88819ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:13.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:13.315+0000 7fc88ca00640 1 --2- 172.21.15.77:0/224474930 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc888108f60 0x7fc88819ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:13.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:13.315+0000 7fc88ca00640 1 -- 172.21.15.77:0/224474930 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc88819f3b0 con 0x7fc888103e30 2024-09-10T09:52:13.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:13.316+0000 7fc88ca00640 1 --2- 172.21.15.77:0/224474930 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc888103e30 0x7fc88819a400 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc87800e9f0 tx=0x7fc87800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:13.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:13.316+0000 7fc886000640 1 -- 172.21.15.77:0/224474930 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc87800cd80 con 0x7fc888103e30 2024-09-10T09:52:13.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:13.316+0000 7fc88de00640 1 -- 172.21.15.77:0/224474930 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc88819f6a0 con 0x7fc888103e30 2024-09-10T09:52:13.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:13.316+0000 7fc886000640 1 -- 172.21.15.77:0/224474930 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc878005500 con 0x7fc888103e30 2024-09-10T09:52:13.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:13.316+0000 7fc886000640 1 -- 172.21.15.77:0/224474930 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc878010690 con 0x7fc888103e30 2024-09-10T09:52:13.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:13.316+0000 7fc88de00640 1 -- 172.21.15.77:0/224474930 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8881a6f80 con 0x7fc888103e30 2024-09-10T09:52:13.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:13.317+0000 7fc88de00640 1 -- 172.21.15.77:0/224474930 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc888104210 con 0x7fc888103e30 2024-09-10T09:52:13.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:13.319+0000 7fc886000640 1 -- 172.21.15.77:0/224474930 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fc8780046c0 con 0x7fc888103e30 2024-09-10T09:52:13.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:13.320+0000 7fc886000640 1 --2- 172.21.15.77:0/224474930 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fc86c0a2b80 0x7fc86c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:13.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:13.320+0000 7fc886000640 1 -- 172.21.15.77:0/224474930 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7fc8780cf060 con 0x7fc888103e30 2024-09-10T09:52:13.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:13.320+0000 7fc87fe00640 1 --2- 172.21.15.77:0/224474930 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fc86c0a2b80 0x7fc86c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:13.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:13.321+0000 7fc87fe00640 1 --2- 172.21.15.77:0/224474930 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fc86c0a2b80 0x7fc86c0a5040 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc88810a930 tx=0x7fc87000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:13.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:13.324+0000 7fc886000640 1 -- 172.21.15.77:0/224474930 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc87809c3c0 con 0x7fc888103e30 2024-09-10T09:52:13.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:13.559+0000 7fc88de00640 1 -- 172.21.15.77:0/224474930 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc88810c1a0 con 0x7fc888103e30 2024-09-10T09:52:13.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:13.563+0000 7fc886000640 1 -- 172.21.15.77:0/224474930 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10536 (secure 0 0 0) 0x7fc87809bcc0 con 0x7fc888103e30 2024-09-10T09:52:13.564 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:52:13.564 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":13,"btime":"2024-09-10T09:52:10:805412+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3159289675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3159289675},{"type":"v1","addr":"172.21.15.77:6835","nonce":3159289675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1951962341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1951962341},{"type":"v1","addr":"172.21.15.77:6837","nonce":1951962341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14487,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6839/4291121888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":4291121888},{"type":"v1","addr":"172.21.15.77:6839","nonce":4291121888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14499,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6841/445284194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":445284194},{"type":"v1","addr":"172.21.15.77:6841","nonce":445284194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/1005054423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1005054423},{"type":"v1","addr":"172.21.15.134:6833","nonce":1005054423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/3716307744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":3716307744},{"type":"v1","addr":"172.21.15.120:6833","nonce":3716307744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/1319892544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1319892544},{"type":"v1","addr":"172.21.15.120:6835","nonce":1319892544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24496,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/3293933699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3293933699},{"type":"v1","addr":"172.21.15.134:6835","nonce":3293933699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24502,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/2461497091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2461497091},{"type":"v1","addr":"172.21.15.134:6837","nonce":2461497091}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24520,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6839/1965117680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1965117680},{"type":"v1","addr":"172.21.15.134:6839","nonce":1965117680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/3033280709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3033280709},{"type":"v1","addr":"172.21.15.120:6837","nonce":3033280709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24563,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6839/1777710157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1777710157},{"type":"v1","addr":"172.21.15.120:6839","nonce":1777710157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-09-10T09:52:13.564 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 13 2024-09-10T09:52:13.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:13.565+0000 7fc88de00640 1 -- 172.21.15.77:0/224474930 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fc86c0a2b80 msgr2=0x7fc86c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:13.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:13.565+0000 7fc88de00640 1 --2- 172.21.15.77:0/224474930 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fc86c0a2b80 0x7fc86c0a5040 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc88810a930 tx=0x7fc87000b390 comp rx=0 tx=0).stop 2024-09-10T09:52:13.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:13.565+0000 7fc88de00640 1 -- 172.21.15.77:0/224474930 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc888103e30 msgr2=0x7fc88819a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:13.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:13.565+0000 7fc88de00640 1 --2- 172.21.15.77:0/224474930 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc888103e30 0x7fc88819a400 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc87800e9f0 tx=0x7fc87800edf0 comp rx=0 tx=0).stop 2024-09-10T09:52:13.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:13.565+0000 7fc88de00640 1 -- 172.21.15.77:0/224474930 shutdown_connections 2024-09-10T09:52:13.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:13.565+0000 7fc88de00640 1 --2- 172.21.15.77:0/224474930 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fc86c0a2b80 0x7fc86c0a5040 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:13.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:13.565+0000 7fc88de00640 1 --2- 172.21.15.77:0/224474930 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc888108f60 0x7fc88819ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:13.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:13.565+0000 7fc88de00640 1 --2- 172.21.15.77:0/224474930 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc8881047e0 0x7fc88819a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:13.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:13.565+0000 7fc88de00640 1 --2- 172.21.15.77:0/224474930 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc888103e30 0x7fc88819a400 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:13.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:13.565+0000 7fc88de00640 1 -- 172.21.15.77:0/224474930 >> 172.21.15.77:0/224474930 conn(0x7fc8880ff8d0 msgr2=0x7fc888100bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:13.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:13.565+0000 7fc88de00640 1 -- 172.21.15.77:0/224474930 shutdown_connections 2024-09-10T09:52:13.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:13.565+0000 7fc88de00640 1 -- 172.21.15.77:0/224474930 wait complete. 2024-09-10T09:52:14.347 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-09-10T09:52:14.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:14 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:52:14.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:14 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:52:14.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:14 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:52:14.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:14 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:52:14.632 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:14 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/224474930' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:52:14.632 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:14 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:52:14.632 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:14 smithi134 ceph-mon[35674]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:52:14.632 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:14 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:52:14.673 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:14 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:52:14.674 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:14 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:52:14.674 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:14 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:52:14.674 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:14 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:52:14.674 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:14 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/224474930' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:52:14.674 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:14 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:52:14.674 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:14 smithi120 ceph-mon[34770]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:52:14.674 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:14 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:52:14.689 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:14 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:52:14.689 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:14 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:52:14.689 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:14 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:52:14.689 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:14 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:52:14.689 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:14 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/224474930' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:52:14.689 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:14 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:52:14.689 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:14 smithi077 ceph-mon[30319]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:52:14.689 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:14 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:52:15.509 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:52:15.681 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:15 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:52:15.681 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:15 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:52:15.681 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:15 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:52:15.681 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:15 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:52:15.681 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:15 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:52:15.923 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:15 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:52:15.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:15 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:52:15.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:15 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:52:15.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:15 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:52:15.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:15 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:52:16.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:16.080+0000 7ffa90200640 1 -- 172.21.15.77:0/3306974774 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa88106f10 msgr2=0x7ffa8810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:16.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:16.080+0000 7ffa90200640 1 --2- 172.21.15.77:0/3306974774 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa88106f10 0x7ffa8810ea90 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ffa80009ca0 tx=0x7ffa8002a8c0 comp rx=0 tx=0).stop 2024-09-10T09:52:16.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:16.080+0000 7ffa90200640 1 -- 172.21.15.77:0/3306974774 shutdown_connections 2024-09-10T09:52:16.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:16.080+0000 7ffa90200640 1 --2- 172.21.15.77:0/3306974774 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa88106f10 0x7ffa8810ea90 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:16.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:16.080+0000 7ffa90200640 1 --2- 172.21.15.77:0/3306974774 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa88102790 0x7ffa881067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:16.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:16.080+0000 7ffa90200640 1 --2- 172.21.15.77:0/3306974774 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa88101de0 0x7ffa881021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:16.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:16.080+0000 7ffa90200640 1 -- 172.21.15.77:0/3306974774 >> 172.21.15.77:0/3306974774 conn(0x7ffa880fd8c0 msgr2=0x7ffa880ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:16.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:16.081+0000 7ffa90200640 1 -- 172.21.15.77:0/3306974774 shutdown_connections 2024-09-10T09:52:16.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:16.081+0000 7ffa90200640 1 -- 172.21.15.77:0/3306974774 wait complete. 2024-09-10T09:52:16.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:16.082+0000 7ffa90200640 1 Processor -- start 2024-09-10T09:52:16.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:16.082+0000 7ffa90200640 1 -- start start 2024-09-10T09:52:16.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:16.082+0000 7ffa90200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa88101de0 0x7ffa880744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:16.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:16.082+0000 7ffa90200640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa88102790 0x7ffa88074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:16.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:16.082+0000 7ffa90200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa88106f10 0x7ffa88074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:16.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:16.082+0000 7ffa90200640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ffa88111f70 con 0x7ffa88106f10 2024-09-10T09:52:16.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:16.082+0000 7ffa90200640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ffa88111df0 con 0x7ffa88102790 2024-09-10T09:52:16.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:16.082+0000 7ffa90200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ffa881120f0 con 0x7ffa88101de0 2024-09-10T09:52:16.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:16.082+0000 7ffa8d800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa88101de0 0x7ffa880744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:16.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:16.082+0000 7ffa8d800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa88101de0 0x7ffa880744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:59470/0 (socket says 172.21.15.77:59470) 2024-09-10T09:52:16.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:16.082+0000 7ffa8d800640 1 -- 172.21.15.77:0/3157183612 learned_addr learned my addr 172.21.15.77:0/3157183612 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:52:16.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:16.082+0000 7ffa8e200640 1 --2- 172.21.15.77:0/3157183612 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa88106f10 0x7ffa88074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:16.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:16.083+0000 7ffa8ce00640 1 --2- 172.21.15.77:0/3157183612 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa88102790 0x7ffa88074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:16.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:16.083+0000 7ffa8d800640 1 -- 172.21.15.77:0/3157183612 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa88102790 msgr2=0x7ffa88074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:16.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:16.083+0000 7ffa8d800640 1 --2- 172.21.15.77:0/3157183612 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa88102790 0x7ffa88074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:16.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:16.083+0000 7ffa8d800640 1 -- 172.21.15.77:0/3157183612 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa88106f10 msgr2=0x7ffa88074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:16.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:16.083+0000 7ffa8d800640 1 --2- 172.21.15.77:0/3157183612 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa88106f10 0x7ffa88074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:16.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:16.083+0000 7ffa8d800640 1 -- 172.21.15.77:0/3157183612 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa881a68b0 con 0x7ffa88101de0 2024-09-10T09:52:16.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:16.083+0000 7ffa8d800640 1 --2- 172.21.15.77:0/3157183612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa88101de0 0x7ffa880744f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c00e9f0 tx=0x7ffa7c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:16.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:16.084+0000 7ffa86000640 1 -- 172.21.15.77:0/3157183612 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa7c00cd80 con 0x7ffa88101de0 2024-09-10T09:52:16.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:16.084+0000 7ffa86000640 1 -- 172.21.15.77:0/3157183612 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa7c010910 con 0x7ffa88101de0 2024-09-10T09:52:16.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:16.084+0000 7ffa90200640 1 -- 172.21.15.77:0/3157183612 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa881a6ba0 con 0x7ffa88101de0 2024-09-10T09:52:16.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:16.084+0000 7ffa86000640 1 -- 172.21.15.77:0/3157183612 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa7c018990 con 0x7ffa88101de0 2024-09-10T09:52:16.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:16.084+0000 7ffa90200640 1 -- 172.21.15.77:0/3157183612 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa881a7130 con 0x7ffa88101de0 2024-09-10T09:52:16.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:16.085+0000 7ffa90200640 1 -- 172.21.15.77:0/3157183612 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa881021c0 con 0x7ffa88101de0 2024-09-10T09:52:16.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:16.087+0000 7ffa86000640 1 -- 172.21.15.77:0/3157183612 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ffa7c010ac0 con 0x7ffa88101de0 2024-09-10T09:52:16.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:16.089+0000 7ffa86000640 1 --2- 172.21.15.77:0/3157183612 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ffa700a2c00 0x7ffa700a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:16.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:16.089+0000 7ffa86000640 1 -- 172.21.15.77:0/3157183612 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7ffa7c0ceeb0 con 0x7ffa88101de0 2024-09-10T09:52:16.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:16.089+0000 7ffa8ce00640 1 --2- 172.21.15.77:0/3157183612 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ffa700a2c00 0x7ffa700a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:16.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:16.090+0000 7ffa8ce00640 1 --2- 172.21.15.77:0/3157183612 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ffa700a2c00 0x7ffa700a50c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ffa88075f80 tx=0x7ffa7400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:16.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:16.092+0000 7ffa86000640 1 -- 172.21.15.77:0/3157183612 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ffa7c09bbc0 con 0x7ffa88101de0 2024-09-10T09:52:16.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:15 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:52:16.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:15 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:52:16.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:15 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:52:16.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:15 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:52:16.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:15 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:52:16.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:16.285+0000 7ffa90200640 1 -- 172.21.15.77:0/3157183612 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7ffa88075a20 con 0x7ffa88101de0 2024-09-10T09:52:16.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:16.286+0000 7ffa86000640 1 -- 172.21.15.77:0/3157183612 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v79) ==== 84+0+65 (secure 0 0 0) 0x7ffa7c09fec0 con 0x7ffa88101de0 2024-09-10T09:52:16.287 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:52:16.287 INFO:teuthology.orchestra.run.smithi077.stdout:[ 2024-09-10T09:52:16.287 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T09:52:16.287 INFO:teuthology.orchestra.run.smithi077.stdout: "poolnum": 1, 2024-09-10T09:52:16.287 INFO:teuthology.orchestra.run.smithi077.stdout: "poolname": ".mgr" 2024-09-10T09:52:16.288 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T09:52:16.288 INFO:teuthology.orchestra.run.smithi077.stdout:] 2024-09-10T09:52:16.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:16.288+0000 7ffa90200640 1 -- 172.21.15.77:0/3157183612 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ffa700a2c00 msgr2=0x7ffa700a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:16.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:16.288+0000 7ffa90200640 1 --2- 172.21.15.77:0/3157183612 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ffa700a2c00 0x7ffa700a50c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ffa88075f80 tx=0x7ffa7400b390 comp rx=0 tx=0).stop 2024-09-10T09:52:16.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:16.288+0000 7ffa90200640 1 -- 172.21.15.77:0/3157183612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa88101de0 msgr2=0x7ffa880744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:16.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:16.288+0000 7ffa90200640 1 --2- 172.21.15.77:0/3157183612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa88101de0 0x7ffa880744f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c00e9f0 tx=0x7ffa7c00eeb0 comp rx=0 tx=0).stop 2024-09-10T09:52:16.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:16.289+0000 7ffa90200640 1 -- 172.21.15.77:0/3157183612 shutdown_connections 2024-09-10T09:52:16.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:16.289+0000 7ffa90200640 1 --2- 172.21.15.77:0/3157183612 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ffa700a2c00 0x7ffa700a50c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:16.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:16.289+0000 7ffa90200640 1 --2- 172.21.15.77:0/3157183612 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa88106f10 0x7ffa88074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:16.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:16.289+0000 7ffa90200640 1 --2- 172.21.15.77:0/3157183612 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa88102790 0x7ffa88074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:16.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:16.289+0000 7ffa90200640 1 --2- 172.21.15.77:0/3157183612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa88101de0 0x7ffa880744f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:16.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:16.289+0000 7ffa90200640 1 -- 172.21.15.77:0/3157183612 >> 172.21.15.77:0/3157183612 conn(0x7ffa880fd8c0 msgr2=0x7ffa880feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:16.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:16.289+0000 7ffa90200640 1 -- 172.21.15.77:0/3157183612 shutdown_connections 2024-09-10T09:52:16.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:16.289+0000 7ffa90200640 1 -- 172.21.15.77:0/3157183612 wait complete. 2024-09-10T09:52:16.923 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:16 smithi120 ceph-mon[34770]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:52:16.923 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:16 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/3157183612' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-10T09:52:16.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:16 smithi077 ceph-mon[30319]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:52:16.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:16 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/3157183612' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-10T09:52:17.107 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T09:52:17.107 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T09:52:17.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:16 smithi134 ceph-mon[35674]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:52:17.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:16 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/3157183612' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-10T09:52:17.493 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:52:18.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:18.081+0000 7f909f400640 1 -- 172.21.15.77:0/1323471130 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f909810e3c0 msgr2=0x7f9098110810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:18.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:18.081+0000 7f909f400640 1 --2- 172.21.15.77:0/1323471130 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f909810e3c0 0x7f9098110810 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9090009ca0 tx=0x7f909002a950 comp rx=0 tx=0).stop 2024-09-10T09:52:18.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:18.081+0000 7f909f400640 1 -- 172.21.15.77:0/1323471130 shutdown_connections 2024-09-10T09:52:18.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:18.081+0000 7f909f400640 1 --2- 172.21.15.77:0/1323471130 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f909810e3c0 0x7f9098110810 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:18.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:18.081+0000 7f909f400640 1 --2- 172.21.15.77:0/1323471130 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9098074f40 0x7f909810de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:18.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:18.081+0000 7f909f400640 1 --2- 172.21.15.77:0/1323471130 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9098074620 0x7f9098074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:18.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:18.081+0000 7f909f400640 1 -- 172.21.15.77:0/1323471130 >> 172.21.15.77:0/1323471130 conn(0x7f90980fd8c0 msgr2=0x7f90980ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:18.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:18.081+0000 7f909f400640 1 -- 172.21.15.77:0/1323471130 shutdown_connections 2024-09-10T09:52:18.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:18.081+0000 7f909f400640 1 -- 172.21.15.77:0/1323471130 wait complete. 2024-09-10T09:52:18.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:18.082+0000 7f909f400640 1 Processor -- start 2024-09-10T09:52:18.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:18.082+0000 7f909f400640 1 -- start start 2024-09-10T09:52:18.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:18.083+0000 7f909f400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9098074620 0x7f909819a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:18.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:18.083+0000 7f909f400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9098074f40 0x7f909819abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:18.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:18.083+0000 7f909f400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f909810e3c0 0x7f909819ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:18.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:18.083+0000 7f909f400640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f90981141b0 con 0x7f9098074620 2024-09-10T09:52:18.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:18.083+0000 7f909f400640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9098114030 con 0x7f9098074f40 2024-09-10T09:52:18.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:18.083+0000 7f909f400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9098114330 con 0x7f909810e3c0 2024-09-10T09:52:18.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:18.083+0000 7f909ca00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9098074620 0x7f909819a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:18.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:18.083+0000 7f909ca00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9098074620 0x7f909819a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46320/0 (socket says 172.21.15.77:46320) 2024-09-10T09:52:18.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:18.083+0000 7f909ca00640 1 -- 172.21.15.77:0/189545633 learned_addr learned my addr 172.21.15.77:0/189545633 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:52:18.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:18.083+0000 7f9097e00640 1 --2- 172.21.15.77:0/189545633 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9098074f40 0x7f909819abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:18.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:18.083+0000 7f909d400640 1 --2- 172.21.15.77:0/189545633 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f909810e3c0 0x7f909819ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:18.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:18.083+0000 7f909ca00640 1 -- 172.21.15.77:0/189545633 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f909810e3c0 msgr2=0x7f909819ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:18.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:18.083+0000 7f909ca00640 1 --2- 172.21.15.77:0/189545633 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f909810e3c0 0x7f909819ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:18.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:18.083+0000 7f909ca00640 1 -- 172.21.15.77:0/189545633 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9098074f40 msgr2=0x7f909819abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:18.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:18.083+0000 7f909ca00640 1 --2- 172.21.15.77:0/189545633 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9098074f40 0x7f909819abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:18.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:18.083+0000 7f909ca00640 1 -- 172.21.15.77:0/189545633 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f909819f660 con 0x7f9098074620 2024-09-10T09:52:18.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:18.083+0000 7f9097e00640 1 --2- 172.21.15.77:0/189545633 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9098074f40 0x7f909819abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:52:18.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:18.084+0000 7f909ca00640 1 --2- 172.21.15.77:0/189545633 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9098074620 0x7f909819a6b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f908800e9f0 tx=0x7f908800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:18.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:18.084+0000 7f9095600640 1 -- 172.21.15.77:0/189545633 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f908800cd80 con 0x7f9098074620 2024-09-10T09:52:18.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:18.084+0000 7f909f400640 1 -- 172.21.15.77:0/189545633 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f909819f950 con 0x7f9098074620 2024-09-10T09:52:18.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:18.084+0000 7f9095600640 1 -- 172.21.15.77:0/189545633 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9088005500 con 0x7f9098074620 2024-09-10T09:52:18.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:18.084+0000 7f9095600640 1 -- 172.21.15.77:0/189545633 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9088010690 con 0x7f9098074620 2024-09-10T09:52:18.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:18.084+0000 7f909f400640 1 -- 172.21.15.77:0/189545633 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90981a7230 con 0x7f9098074620 2024-09-10T09:52:18.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:18.085+0000 7f909f400640 1 -- 172.21.15.77:0/189545633 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9098074a00 con 0x7f9098074620 2024-09-10T09:52:18.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:18.087+0000 7f9095600640 1 -- 172.21.15.77:0/189545633 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f90880046c0 con 0x7f9098074620 2024-09-10T09:52:18.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:18.088+0000 7f9095600640 1 --2- 172.21.15.77:0/189545633 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f907c0a2b80 0x7f907c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:18.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:18.088+0000 7f9095600640 1 -- 172.21.15.77:0/189545633 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7f90880cefe0 con 0x7f9098074620 2024-09-10T09:52:18.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:18.088+0000 7f9097e00640 1 --2- 172.21.15.77:0/189545633 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f907c0a2b80 0x7f907c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:18.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:18.089+0000 7f9097e00640 1 --2- 172.21.15.77:0/189545633 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f907c0a2b80 0x7f907c0a5040 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9098110ce0 tx=0x7f9084005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:18.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:18.092+0000 7f9095600640 1 -- 172.21.15.77:0/189545633 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f908809c380 con 0x7f9098074620 2024-09-10T09:52:18.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:18.267+0000 7f909f400640 1 -- 172.21.15.77:0/189545633 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f90981a74f0 con 0x7f9098074620 2024-09-10T09:52:18.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:18.818+0000 7f9095600640 1 -- 172.21.15.77:0/189545633 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v80) ==== 138+0+0 (secure 0 0 0) 0x7f908809bc80 con 0x7f9098074620 2024-09-10T09:52:18.819 INFO:teuthology.orchestra.run.smithi077.stderr:pool 'cephfs_metadata' created 2024-09-10T09:52:18.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:18.821+0000 7f909f400640 1 -- 172.21.15.77:0/189545633 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f907c0a2b80 msgr2=0x7f907c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:18.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:18.821+0000 7f909f400640 1 --2- 172.21.15.77:0/189545633 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f907c0a2b80 0x7f907c0a5040 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9098110ce0 tx=0x7f9084005f90 comp rx=0 tx=0).stop 2024-09-10T09:52:18.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:18.821+0000 7f909f400640 1 -- 172.21.15.77:0/189545633 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9098074620 msgr2=0x7f909819a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:18.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:18.821+0000 7f909f400640 1 --2- 172.21.15.77:0/189545633 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9098074620 0x7f909819a6b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f908800e9f0 tx=0x7f908800edf0 comp rx=0 tx=0).stop 2024-09-10T09:52:18.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:18.821+0000 7f909f400640 1 -- 172.21.15.77:0/189545633 shutdown_connections 2024-09-10T09:52:18.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:18.821+0000 7f909f400640 1 --2- 172.21.15.77:0/189545633 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f907c0a2b80 0x7f907c0a5040 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:18.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:18.821+0000 7f909f400640 1 --2- 172.21.15.77:0/189545633 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f909810e3c0 0x7f909819ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:18.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:18.821+0000 7f909f400640 1 --2- 172.21.15.77:0/189545633 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9098074f40 0x7f909819abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:18.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:18.821+0000 7f909f400640 1 --2- 172.21.15.77:0/189545633 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9098074620 0x7f909819a6b0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:18.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:18.821+0000 7f909f400640 1 -- 172.21.15.77:0/189545633 >> 172.21.15.77:0/189545633 conn(0x7f90980fd8c0 msgr2=0x7f90980fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:18.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:18.821+0000 7f909f400640 1 -- 172.21.15.77:0/189545633 shutdown_connections 2024-09-10T09:52:18.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:18.821+0000 7f909f400640 1 -- 172.21.15.77:0/189545633 wait complete. 2024-09-10T09:52:19.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:18 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:52:19.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:18 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:52:19.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:18 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:52:19.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:18 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:52:19.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:18 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:52:19.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:18 smithi134 ceph-mon[35674]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:52:19.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:18 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/189545633' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-10T09:52:19.163 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:18 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:52:19.164 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:18 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:52:19.164 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:18 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:52:19.164 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:18 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:52:19.164 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:18 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:52:19.164 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:18 smithi077 ceph-mon[30319]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:52:19.164 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:18 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/189545633' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-10T09:52:19.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:18 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:52:19.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:18 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:52:19.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:18 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:52:19.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:18 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:52:19.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:18 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:52:19.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:18 smithi120 ceph-mon[34770]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:52:19.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:18 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/189545633' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-10T09:52:19.669 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-09-10T09:52:19.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:19 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/189545633' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-10T09:52:19.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:19 smithi077 ceph-mon[30319]: osdmap e80: 12 total, 12 up, 12 in 2024-09-10T09:52:20.048 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:52:20.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:19 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/189545633' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-10T09:52:20.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:19 smithi134 ceph-mon[35674]: osdmap e80: 12 total, 12 up, 12 in 2024-09-10T09:52:20.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:19 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/189545633' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-10T09:52:20.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:19 smithi120 ceph-mon[34770]: osdmap e80: 12 total, 12 up, 12 in 2024-09-10T09:52:20.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:20.602+0000 7f05d5600640 1 -- 172.21.15.77:0/2844293711 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f05d0102360 msgr2=0x7f05d010e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:20.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:20.602+0000 7f05d5600640 1 --2- 172.21.15.77:0/2844293711 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f05d0102360 0x7f05d010e890 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f05c8009cd0 tx=0x7f05c802a8e0 comp rx=0 tx=0).stop 2024-09-10T09:52:20.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:20.602+0000 7f05d5600640 1 -- 172.21.15.77:0/2844293711 shutdown_connections 2024-09-10T09:52:20.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:20.602+0000 7f05d5600640 1 --2- 172.21.15.77:0/2844293711 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f05d0102360 0x7f05d010e890 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:20.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:20.602+0000 7f05d5600640 1 --2- 172.21.15.77:0/2844293711 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f05d01019c0 0x7f05d0101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:20.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:20.602+0000 7f05d5600640 1 --2- 172.21.15.77:0/2844293711 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05d01079c0 0x7f05d0107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:20.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:20.602+0000 7f05d5600640 1 -- 172.21.15.77:0/2844293711 >> 172.21.15.77:0/2844293711 conn(0x7f05d00fd700 msgr2=0x7f05d00ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:20.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:20.602+0000 7f05d5600640 1 -- 172.21.15.77:0/2844293711 shutdown_connections 2024-09-10T09:52:20.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:20.602+0000 7f05d5600640 1 -- 172.21.15.77:0/2844293711 wait complete. 2024-09-10T09:52:20.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:20.603+0000 7f05d5600640 1 Processor -- start 2024-09-10T09:52:20.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:20.603+0000 7f05d5600640 1 -- start start 2024-09-10T09:52:20.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:20.603+0000 7f05d5600640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f05d01019c0 0x7f05d00744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:20.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:20.603+0000 7f05d5600640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f05d0102360 0x7f05d0074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:20.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:20.604+0000 7f05d5600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05d01079c0 0x7f05d0074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:20.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:20.604+0000 7f05d5600640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f05d0111e10 con 0x7f05d01019c0 2024-09-10T09:52:20.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:20.604+0000 7f05d5600640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f05d0111c90 con 0x7f05d0102360 2024-09-10T09:52:20.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:20.604+0000 7f05d5600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f05d0111f90 con 0x7f05d01079c0 2024-09-10T09:52:20.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:20.604+0000 7f05cfe00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f05d01019c0 0x7f05d00744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:20.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:20.604+0000 7f05cfe00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f05d01019c0 0x7f05d00744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46342/0 (socket says 172.21.15.77:46342) 2024-09-10T09:52:20.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:20.604+0000 7f05cfe00640 1 -- 172.21.15.77:0/1252803073 learned_addr learned my addr 172.21.15.77:0/1252803073 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:52:20.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:20.604+0000 7f05c7400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f05d0102360 0x7f05d0074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:20.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:20.604+0000 7f05d4c00640 1 --2- 172.21.15.77:0/1252803073 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05d01079c0 0x7f05d0074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:20.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:20.604+0000 7f05cfe00640 1 -- 172.21.15.77:0/1252803073 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05d01079c0 msgr2=0x7f05d0074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:20.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:20.604+0000 7f05cfe00640 1 --2- 172.21.15.77:0/1252803073 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05d01079c0 0x7f05d0074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:20.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:20.604+0000 7f05cfe00640 1 -- 172.21.15.77:0/1252803073 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f05d0102360 msgr2=0x7f05d0074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:20.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:20.604+0000 7f05cfe00640 1 --2- 172.21.15.77:0/1252803073 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f05d0102360 0x7f05d0074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:20.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:20.604+0000 7f05cfe00640 1 -- 172.21.15.77:0/1252803073 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05d01a6750 con 0x7f05d01019c0 2024-09-10T09:52:20.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:20.604+0000 7f05c7400640 1 --2- 172.21.15.77:0/1252803073 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f05d0102360 0x7f05d0074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:52:20.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:20.605+0000 7f05cfe00640 1 --2- 172.21.15.77:0/1252803073 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f05d01019c0 0x7f05d00744f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f05c000e9f0 tx=0x7f05c000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:20.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:20.605+0000 7f05cd600640 1 -- 172.21.15.77:0/1252803073 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c000cd80 con 0x7f05d01019c0 2024-09-10T09:52:20.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:20.605+0000 7f05cd600640 1 -- 172.21.15.77:0/1252803073 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05c0005500 con 0x7f05d01019c0 2024-09-10T09:52:20.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:20.605+0000 7f05cd600640 1 -- 172.21.15.77:0/1252803073 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c0010690 con 0x7f05d01019c0 2024-09-10T09:52:20.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:20.605+0000 7f05d5600640 1 -- 172.21.15.77:0/1252803073 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05d01a6a40 con 0x7f05d01019c0 2024-09-10T09:52:20.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:20.605+0000 7f05d5600640 1 -- 172.21.15.77:0/1252803073 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05d01a6f30 con 0x7f05d01019c0 2024-09-10T09:52:20.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:20.606+0000 7f05d5600640 1 -- 172.21.15.77:0/1252803073 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05d0101e20 con 0x7f05d01019c0 2024-09-10T09:52:20.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:20.608+0000 7f05cd600640 1 -- 172.21.15.77:0/1252803073 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f05c00046c0 con 0x7f05d01019c0 2024-09-10T09:52:20.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:20.610+0000 7f05cd600640 1 --2- 172.21.15.77:0/1252803073 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f05b40a2c00 0x7f05b40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:20.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:20.611+0000 7f05cd600640 1 -- 172.21.15.77:0/1252803073 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7497+0+0 (secure 0 0 0) 0x7f05c00cf2b0 con 0x7f05d01019c0 2024-09-10T09:52:20.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:20.611+0000 7f05c7400640 1 --2- 172.21.15.77:0/1252803073 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f05b40a2c00 0x7f05b40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:20.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:20.611+0000 7f05c7400640 1 --2- 172.21.15.77:0/1252803073 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f05b40a2c00 0x7f05b40a50c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f05d0075f80 tx=0x7f05b8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:20.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:20.613+0000 7f05cd600640 1 -- 172.21.15.77:0/1252803073 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f05c00541b0 con 0x7f05d01019c0 2024-09-10T09:52:20.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:20.786+0000 7f05d5600640 1 -- 172.21.15.77:0/1252803073 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f05d0075a20 con 0x7f05d01019c0 2024-09-10T09:52:21.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:21.292+0000 7f05cd600640 1 -- 172.21.15.77:0/1252803073 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v82) ==== 171+0+0 (secure 0 0 0) 0x7f05d0075a20 con 0x7f05d01019c0 2024-09-10T09:52:21.293 INFO:teuthology.orchestra.run.smithi077.stderr:pool 'cephfs_data' created 2024-09-10T09:52:21.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:21.295+0000 7f05d5600640 1 -- 172.21.15.77:0/1252803073 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f05b40a2c00 msgr2=0x7f05b40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:21.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:21.295+0000 7f05d5600640 1 --2- 172.21.15.77:0/1252803073 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f05b40a2c00 0x7f05b40a50c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f05d0075f80 tx=0x7f05b8005f90 comp rx=0 tx=0).stop 2024-09-10T09:52:21.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:21.295+0000 7f05d5600640 1 -- 172.21.15.77:0/1252803073 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f05d01019c0 msgr2=0x7f05d00744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:21.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:21.295+0000 7f05d5600640 1 --2- 172.21.15.77:0/1252803073 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f05d01019c0 0x7f05d00744f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f05c000e9f0 tx=0x7f05c000edf0 comp rx=0 tx=0).stop 2024-09-10T09:52:21.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:21.295+0000 7f05d5600640 1 -- 172.21.15.77:0/1252803073 shutdown_connections 2024-09-10T09:52:21.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:21.295+0000 7f05d5600640 1 --2- 172.21.15.77:0/1252803073 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f05b40a2c00 0x7f05b40a50c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:21.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:21.295+0000 7f05d5600640 1 --2- 172.21.15.77:0/1252803073 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05d01079c0 0x7f05d0074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:21.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:21.295+0000 7f05d5600640 1 --2- 172.21.15.77:0/1252803073 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f05d0102360 0x7f05d0074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:21.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:21.295+0000 7f05d5600640 1 --2- 172.21.15.77:0/1252803073 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f05d01019c0 0x7f05d00744f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:21.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:21.295+0000 7f05d5600640 1 -- 172.21.15.77:0/1252803073 >> 172.21.15.77:0/1252803073 conn(0x7f05d00fd700 msgr2=0x7f05d00fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:21.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:21.295+0000 7f05d5600640 1 -- 172.21.15.77:0/1252803073 shutdown_connections 2024-09-10T09:52:21.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:21.295+0000 7f05d5600640 1 -- 172.21.15.77:0/1252803073 wait complete. 2024-09-10T09:52:21.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:21 smithi134 ceph-mon[35674]: pgmap v290: 33 pgs: 3 creating+peering, 29 unknown, 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:52:21.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:21 smithi134 ceph-mon[35674]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T09:52:21.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:21 smithi134 ceph-mon[35674]: osdmap e81: 12 total, 12 up, 12 in 2024-09-10T09:52:21.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:21 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/1252803073' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-10T09:52:21.673 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:21 smithi077 ceph-mon[30319]: pgmap v290: 33 pgs: 3 creating+peering, 29 unknown, 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:52:21.673 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:21 smithi077 ceph-mon[30319]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T09:52:21.673 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:21 smithi077 ceph-mon[30319]: osdmap e81: 12 total, 12 up, 12 in 2024-09-10T09:52:21.673 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:21 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/1252803073' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-10T09:52:21.673 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:21 smithi120 ceph-mon[34770]: pgmap v290: 33 pgs: 3 creating+peering, 29 unknown, 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:52:21.673 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:21 smithi120 ceph-mon[34770]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T09:52:21.673 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:21 smithi120 ceph-mon[34770]: osdmap e81: 12 total, 12 up, 12 in 2024-09-10T09:52:21.674 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:21 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/1252803073' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-10T09:52:22.145 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T09:52:22.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:22 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/1252803073' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-10T09:52:22.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:22 smithi077 ceph-mon[30319]: osdmap e82: 12 total, 12 up, 12 in 2024-09-10T09:52:22.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:22 smithi077 ceph-mon[30319]: pgmap v293: 97 pgs: 7 active+clean, 3 creating+peering, 87 unknown; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:52:22.530 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:52:22.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:22 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/1252803073' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-10T09:52:22.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:22 smithi134 ceph-mon[35674]: osdmap e82: 12 total, 12 up, 12 in 2024-09-10T09:52:22.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:22 smithi134 ceph-mon[35674]: pgmap v293: 97 pgs: 7 active+clean, 3 creating+peering, 87 unknown; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:52:22.673 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:22 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/1252803073' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-10T09:52:22.673 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:22 smithi120 ceph-mon[34770]: osdmap e82: 12 total, 12 up, 12 in 2024-09-10T09:52:22.673 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:22 smithi120 ceph-mon[34770]: pgmap v293: 97 pgs: 7 active+clean, 3 creating+peering, 87 unknown; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:52:23.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:23.138+0000 7f8ec3200640 1 -- 172.21.15.77:0/2539783776 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ec4103e30 msgr2=0x7f8ec4104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:23.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:23.138+0000 7f8ec3200640 1 --2- 172.21.15.77:0/2539783776 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ec4103e30 0x7f8ec4104210 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8eb4009900 tx=0x7f8eb402a410 comp rx=0 tx=0).stop 2024-09-10T09:52:23.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:23.139+0000 7f8ec3200640 1 -- 172.21.15.77:0/2539783776 shutdown_connections 2024-09-10T09:52:23.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:23.139+0000 7f8ec3200640 1 --2- 172.21.15.77:0/2539783776 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8ec4108f60 0x7f8ec4110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:23.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:23.139+0000 7f8ec3200640 1 --2- 172.21.15.77:0/2539783776 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ec41047e0 0x7f8ec4108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:23.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:23.139+0000 7f8ec3200640 1 --2- 172.21.15.77:0/2539783776 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ec4103e30 0x7f8ec4104210 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:23.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:23.139+0000 7f8ec3200640 1 -- 172.21.15.77:0/2539783776 >> 172.21.15.77:0/2539783776 conn(0x7f8ec40ff8d0 msgr2=0x7f8ec4101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:23.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:23.139+0000 7f8ec3200640 1 -- 172.21.15.77:0/2539783776 shutdown_connections 2024-09-10T09:52:23.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:23.139+0000 7f8ec3200640 1 -- 172.21.15.77:0/2539783776 wait complete. 2024-09-10T09:52:23.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:23.140+0000 7f8ec3200640 1 Processor -- start 2024-09-10T09:52:23.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:23.140+0000 7f8ec3200640 1 -- start start 2024-09-10T09:52:23.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:23.140+0000 7f8ec3200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ec4103e30 0x7f8ec419a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:23.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:23.140+0000 7f8ec3200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ec41047e0 0x7f8ec419a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:23.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:23.140+0000 7f8ec3200640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8ec4108f60 0x7f8ec419ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:23.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:23.140+0000 7f8ec3200640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec4114010 con 0x7f8ec41047e0 2024-09-10T09:52:23.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:23.140+0000 7f8ec3200640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec4113e90 con 0x7f8ec4108f60 2024-09-10T09:52:23.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:23.140+0000 7f8ec3200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec4114190 con 0x7f8ec4103e30 2024-09-10T09:52:23.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:23.140+0000 7f8ec1400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ec41047e0 0x7f8ec419a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:23.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:23.140+0000 7f8ec1e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ec4103e30 0x7f8ec419a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:23.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:23.140+0000 7f8ec1400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ec41047e0 0x7f8ec419a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:32954/0 (socket says 172.21.15.77:32954) 2024-09-10T09:52:23.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:23.140+0000 7f8ec1e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ec4103e30 0x7f8ec419a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:33170/0 (socket says 172.21.15.77:33170) 2024-09-10T09:52:23.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:23.141+0000 7f8ec1400640 1 -- 172.21.15.77:0/2369495091 learned_addr learned my addr 172.21.15.77:0/2369495091 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:52:23.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:23.141+0000 7f8ec2800640 1 --2- 172.21.15.77:0/2369495091 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8ec4108f60 0x7f8ec419ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:23.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:23.141+0000 7f8ec1e00640 1 -- 172.21.15.77:0/2369495091 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8ec4108f60 msgr2=0x7f8ec419ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:23.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:23.141+0000 7f8ec1e00640 1 --2- 172.21.15.77:0/2369495091 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8ec4108f60 0x7f8ec419ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:23.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:23.141+0000 7f8ec1e00640 1 -- 172.21.15.77:0/2369495091 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ec41047e0 msgr2=0x7f8ec419a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:23.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:23.141+0000 7f8ec1e00640 1 --2- 172.21.15.77:0/2369495091 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ec41047e0 0x7f8ec419a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:23.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:23.141+0000 7f8ec1e00640 1 -- 172.21.15.77:0/2369495091 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ec419f400 con 0x7f8ec4103e30 2024-09-10T09:52:23.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:23.141+0000 7f8ec2800640 1 --2- 172.21.15.77:0/2369495091 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8ec4108f60 0x7f8ec419ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:52:23.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:23.141+0000 7f8ec1e00640 1 --2- 172.21.15.77:0/2369495091 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ec4103e30 0x7f8ec419a450 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8eb402a930 tx=0x7f8eb4003b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:23.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:23.141+0000 7f8ebaa00640 1 -- 172.21.15.77:0/2369495091 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eb403f070 con 0x7f8ec4103e30 2024-09-10T09:52:23.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:23.142+0000 7f8ec3200640 1 -- 172.21.15.77:0/2369495091 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ec419f690 con 0x7f8ec4103e30 2024-09-10T09:52:23.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:23.142+0000 7f8ebaa00640 1 -- 172.21.15.77:0/2369495091 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8eb4031c90 con 0x7f8ec4103e30 2024-09-10T09:52:23.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:23.142+0000 7f8ebaa00640 1 -- 172.21.15.77:0/2369495091 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eb4035030 con 0x7f8ec4103e30 2024-09-10T09:52:23.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:23.142+0000 7f8ec3200640 1 -- 172.21.15.77:0/2369495091 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ec41a6fd0 con 0x7f8ec4103e30 2024-09-10T09:52:23.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:23.143+0000 7f8ec3200640 1 -- 172.21.15.77:0/2369495091 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ec4105310 con 0x7f8ec4103e30 2024-09-10T09:52:23.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:23.145+0000 7f8ebaa00640 1 -- 172.21.15.77:0/2369495091 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f8eb4042070 con 0x7f8ec4103e30 2024-09-10T09:52:23.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:23.146+0000 7f8ebaa00640 1 --2- 172.21.15.77:0/2369495091 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f8ea80a2b80 0x7f8ea80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:23.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:23.146+0000 7f8ebaa00640 1 -- 172.21.15.77:0/2369495091 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7879+0+0 (secure 0 0 0) 0x7f8eb40efe10 con 0x7f8ec4103e30 2024-09-10T09:52:23.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:23.146+0000 7f8ec1400640 1 --2- 172.21.15.77:0/2369495091 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f8ea80a2b80 0x7f8ea80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:23.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:23.147+0000 7f8ec1400640 1 --2- 172.21.15.77:0/2369495091 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f8ea80a2b80 0x7f8ea80a5040 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8ec40ffcf0 tx=0x7f8eb0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:23.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:23.148+0000 7f8ebaa00640 1 -- 172.21.15.77:0/2369495091 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8eb40bc830 con 0x7f8ec4103e30 2024-09-10T09:52:23.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:23.366+0000 7f8ec3200640 1 -- 172.21.15.77:0/2369495091 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f8ec410bbb0 con 0x7f8ec4103e30 2024-09-10T09:52:23.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:23 smithi134 ceph-mon[35674]: osdmap e83: 12 total, 12 up, 12 in 2024-09-10T09:52:23.673 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:23 smithi120 ceph-mon[34770]: osdmap e83: 12 total, 12 up, 12 in 2024-09-10T09:52:23.689 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:23 smithi077 ceph-mon[30319]: osdmap e83: 12 total, 12 up, 12 in 2024-09-10T09:52:24.315 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:24.314+0000 7f8ebaa00640 1 -- 172.21.15.77:0/2369495091 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f8eb40c0b30 con 0x7f8ec4103e30 2024-09-10T09:52:24.315 INFO:teuthology.orchestra.run.smithi077.stderr:new fs with metadata pool 2 and data pool 3 2024-09-10T09:52:24.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:24.317+0000 7f8ec3200640 1 -- 172.21.15.77:0/2369495091 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f8ea80a2b80 msgr2=0x7f8ea80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:24.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:24.317+0000 7f8ec3200640 1 --2- 172.21.15.77:0/2369495091 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f8ea80a2b80 0x7f8ea80a5040 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8ec40ffcf0 tx=0x7f8eb0005f90 comp rx=0 tx=0).stop 2024-09-10T09:52:24.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:24.317+0000 7f8ec3200640 1 -- 172.21.15.77:0/2369495091 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ec4103e30 msgr2=0x7f8ec419a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:24.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:24.317+0000 7f8ec3200640 1 --2- 172.21.15.77:0/2369495091 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ec4103e30 0x7f8ec419a450 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8eb402a930 tx=0x7f8eb4003b20 comp rx=0 tx=0).stop 2024-09-10T09:52:24.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:24.317+0000 7f8ec3200640 1 -- 172.21.15.77:0/2369495091 shutdown_connections 2024-09-10T09:52:24.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:24.317+0000 7f8ec3200640 1 --2- 172.21.15.77:0/2369495091 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f8ea80a2b80 0x7f8ea80a5040 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:24.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:24.317+0000 7f8ec3200640 1 --2- 172.21.15.77:0/2369495091 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8ec4108f60 0x7f8ec419ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:24.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:24.317+0000 7f8ec3200640 1 --2- 172.21.15.77:0/2369495091 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ec41047e0 0x7f8ec419a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:24.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:24.317+0000 7f8ec3200640 1 --2- 172.21.15.77:0/2369495091 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ec4103e30 0x7f8ec419a450 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:24.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:24.317+0000 7f8ec3200640 1 -- 172.21.15.77:0/2369495091 >> 172.21.15.77:0/2369495091 conn(0x7f8ec40ff8d0 msgr2=0x7f8ec4100b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:24.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:24.317+0000 7f8ec3200640 1 -- 172.21.15.77:0/2369495091 shutdown_connections 2024-09-10T09:52:24.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:24.318+0000 7f8ec3200640 1 -- 172.21.15.77:0/2369495091 wait complete. 2024-09-10T09:52:24.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:24 smithi134 ceph-mon[35674]: osdmap e84: 12 total, 12 up, 12 in 2024-09-10T09:52:24.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:24 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/2369495091' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-10T09:52:24.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:24 smithi134 ceph-mon[35674]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-10T09:52:24.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:24 smithi134 ceph-mon[35674]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-10T09:52:24.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:24 smithi134 ceph-mon[35674]: pgmap v296: 97 pgs: 47 active+clean, 15 creating+peering, 35 unknown; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:52:24.673 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:24 smithi120 ceph-mon[34770]: osdmap e84: 12 total, 12 up, 12 in 2024-09-10T09:52:24.673 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:24 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/2369495091' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-10T09:52:24.674 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:24 smithi120 ceph-mon[34770]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-10T09:52:24.674 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:24 smithi120 ceph-mon[34770]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-10T09:52:24.674 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:24 smithi120 ceph-mon[34770]: pgmap v296: 97 pgs: 47 active+clean, 15 creating+peering, 35 unknown; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:52:24.674 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:24 smithi077 ceph-mon[30319]: osdmap e84: 12 total, 12 up, 12 in 2024-09-10T09:52:24.674 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:24 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/2369495091' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-10T09:52:24.674 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:24 smithi077 ceph-mon[30319]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-10T09:52:24.674 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:24 smithi077 ceph-mon[30319]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-10T09:52:24.674 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:24 smithi077 ceph-mon[30319]: pgmap v296: 97 pgs: 47 active+clean, 15 creating+peering, 35 unknown; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:52:25.204 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-09-10T09:52:25.204 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-09-10T09:52:25.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:25 smithi077 ceph-mon[30319]: osdmap e85: 12 total, 12 up, 12 in 2024-09-10T09:52:25.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:25 smithi077 ceph-mon[30319]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-10T09:52:25.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:25 smithi077 ceph-mon[30319]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-10T09:52:25.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:25 smithi077 ceph-mon[30319]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-10T09:52:25.573 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:52:25.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:25 smithi134 ceph-mon[35674]: osdmap e85: 12 total, 12 up, 12 in 2024-09-10T09:52:25.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:25 smithi134 ceph-mon[35674]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-10T09:52:25.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:25 smithi134 ceph-mon[35674]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-10T09:52:25.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:25 smithi134 ceph-mon[35674]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-10T09:52:25.673 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:25 smithi120 ceph-mon[34770]: osdmap e85: 12 total, 12 up, 12 in 2024-09-10T09:52:25.673 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:25 smithi120 ceph-mon[34770]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-10T09:52:25.674 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:25 smithi120 ceph-mon[34770]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-10T09:52:25.674 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:25 smithi120 ceph-mon[34770]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-10T09:52:26.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:26.125+0000 7f842ee00640 1 -- 172.21.15.77:0/862308973 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8428103ff0 msgr2=0x7f84281043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:26.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:26.125+0000 7f842ee00640 1 --2- 172.21.15.77:0/862308973 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8428103ff0 0x7f84281043d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8418009ca0 tx=0x7f841802a8c0 comp rx=0 tx=0).stop 2024-09-10T09:52:26.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:26.125+0000 7f842ee00640 1 -- 172.21.15.77:0/862308973 shutdown_connections 2024-09-10T09:52:26.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:26.125+0000 7f842ee00640 1 --2- 172.21.15.77:0/862308973 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8428109120 0x7f8428110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:26.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:26.125+0000 7f842ee00640 1 --2- 172.21.15.77:0/862308973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84281049a0 0x7f84281089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:26.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:26.125+0000 7f842ee00640 1 --2- 172.21.15.77:0/862308973 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8428103ff0 0x7f84281043d0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:26.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:26.125+0000 7f842ee00640 1 -- 172.21.15.77:0/862308973 >> 172.21.15.77:0/862308973 conn(0x7f84280ffa90 msgr2=0x7f8428101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:26.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:26.125+0000 7f842ee00640 1 -- 172.21.15.77:0/862308973 shutdown_connections 2024-09-10T09:52:26.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:26.125+0000 7f842ee00640 1 -- 172.21.15.77:0/862308973 wait complete. 2024-09-10T09:52:26.128 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:26.126+0000 7f842ee00640 1 Processor -- start 2024-09-10T09:52:26.128 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:26.126+0000 7f842ee00640 1 -- start start 2024-09-10T09:52:26.128 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:26.126+0000 7f842ee00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8428103ff0 0x7f842819a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:26.128 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:26.126+0000 7f842ee00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f84281049a0 0x7f842819ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:26.128 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:26.126+0000 7f842ee00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8428109120 0x7f842819eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:26.128 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:26.126+0000 7f842ee00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8428114140 con 0x7f8428103ff0 2024-09-10T09:52:26.128 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:26.126+0000 7f842ee00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8428113fc0 con 0x7f84281049a0 2024-09-10T09:52:26.128 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:26.126+0000 7f842ee00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f84281142c0 con 0x7f8428109120 2024-09-10T09:52:26.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:26.127+0000 7f8427e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8428103ff0 0x7f842819a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:26.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:26.127+0000 7f8427e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8428103ff0 0x7f842819a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:32976/0 (socket says 172.21.15.77:32976) 2024-09-10T09:52:26.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:26.127+0000 7f8427e00640 1 -- 172.21.15.77:0/882239943 learned_addr learned my addr 172.21.15.77:0/882239943 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:52:26.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:26.127+0000 7f8427400640 1 --2- 172.21.15.77:0/882239943 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f84281049a0 0x7f842819ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:26.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:26.127+0000 7f842ce00640 1 --2- 172.21.15.77:0/882239943 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8428109120 0x7f842819eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:26.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:26.127+0000 7f8427e00640 1 -- 172.21.15.77:0/882239943 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8428109120 msgr2=0x7f842819eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:26.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:26.127+0000 7f8427e00640 1 --2- 172.21.15.77:0/882239943 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8428109120 0x7f842819eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:26.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:26.127+0000 7f8427e00640 1 -- 172.21.15.77:0/882239943 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f84281049a0 msgr2=0x7f842819ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:26.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:26.127+0000 7f8427e00640 1 --2- 172.21.15.77:0/882239943 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f84281049a0 0x7f842819ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:26.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:26.127+0000 7f8427e00640 1 -- 172.21.15.77:0/882239943 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f842819f5b0 con 0x7f8428103ff0 2024-09-10T09:52:26.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:26.127+0000 7f8427400640 1 --2- 172.21.15.77:0/882239943 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f84281049a0 0x7f842819ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:52:26.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:26.127+0000 7f8427e00640 1 --2- 172.21.15.77:0/882239943 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8428103ff0 0x7f842819a600 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f841802ade0 tx=0x7f84180319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:26.130 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:26.128+0000 7f8424c00640 1 -- 172.21.15.77:0/882239943 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f841802fa20 con 0x7f8428103ff0 2024-09-10T09:52:26.130 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:26.128+0000 7f8424c00640 1 -- 172.21.15.77:0/882239943 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f841802fbc0 con 0x7f8428103ff0 2024-09-10T09:52:26.130 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:26.128+0000 7f842ee00640 1 -- 172.21.15.77:0/882239943 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f842819f840 con 0x7f8428103ff0 2024-09-10T09:52:26.130 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:26.128+0000 7f8424c00640 1 -- 172.21.15.77:0/882239943 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84180466e0 con 0x7f8428103ff0 2024-09-10T09:52:26.130 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:26.128+0000 7f842ee00640 1 -- 172.21.15.77:0/882239943 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84281a7180 con 0x7f8428103ff0 2024-09-10T09:52:26.130 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:26.129+0000 7f842ee00640 1 -- 172.21.15.77:0/882239943 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84281043d0 con 0x7f8428103ff0 2024-09-10T09:52:26.133 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:26.132+0000 7f8424c00640 1 -- 172.21.15.77:0/882239943 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f8418046940 con 0x7f8428103ff0 2024-09-10T09:52:26.135 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:26.134+0000 7f8424c00640 1 --2- 172.21.15.77:0/882239943 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f840c0a2c00 0x7f840c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:26.135 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:26.134+0000 7f8424c00640 1 -- 172.21.15.77:0/882239943 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f84180f05a0 con 0x7f8428103ff0 2024-09-10T09:52:26.135 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:26.134+0000 7f8427400640 1 --2- 172.21.15.77:0/882239943 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f840c0a2c00 0x7f840c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:26.135 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:26.135+0000 7f8427400640 1 --2- 172.21.15.77:0/882239943 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f840c0a2c00 0x7f840c0a50c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f84280fff90 tx=0x7f8414005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:26.137 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:26.136+0000 7f8424c00640 1 -- 172.21.15.77:0/882239943 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f84180bd640 con 0x7f8428103ff0 2024-09-10T09:52:26.173 INFO:journalctl@ceph.mds.b.smithi120.stdout:Sep 10 09:52:25 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-b[64932]: 2024-09-10T09:52:25.816+0000 7f2860ae5640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T09:52:26.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:26.320+0000 7f842ee00640 1 -- 172.21.15.77:0/882239943 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f842819b960 con 0x7f8428103ff0 2024-09-10T09:52:26.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:26 smithi134 ceph-mon[35674]: mds.? [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] up:active 2024-09-10T09:52:26.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:26 smithi134 ceph-mon[35674]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T09:52:26.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:26 smithi134 ceph-mon[35674]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T09:52:26.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:26 smithi134 ceph-mon[35674]: pgmap v298: 97 pgs: 75 active+clean, 12 creating+peering, 10 unknown; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:52:26.673 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:26 smithi120 ceph-mon[34770]: mds.? [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] up:active 2024-09-10T09:52:26.674 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:26 smithi120 ceph-mon[34770]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T09:52:26.674 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:26 smithi120 ceph-mon[34770]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T09:52:26.674 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:26 smithi120 ceph-mon[34770]: pgmap v298: 97 pgs: 75 active+clean, 12 creating+peering, 10 unknown; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:52:26.689 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:26 smithi077 ceph-mon[30319]: mds.? [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] up:active 2024-09-10T09:52:26.689 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:26 smithi077 ceph-mon[30319]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T09:52:26.689 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:26 smithi077 ceph-mon[30319]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T09:52:26.689 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:26 smithi077 ceph-mon[30319]: pgmap v298: 97 pgs: 75 active+clean, 12 creating+peering, 10 unknown; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:52:27.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:27.321+0000 7f8424c00640 1 -- 172.21.15.77:0/882239943 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v86) ==== 157+0+0 (secure 0 0 0) 0x7f842819b960 con 0x7f8428103ff0 2024-09-10T09:52:27.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:27.324+0000 7f842ee00640 1 -- 172.21.15.77:0/882239943 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f840c0a2c00 msgr2=0x7f840c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:27.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:27.324+0000 7f842ee00640 1 --2- 172.21.15.77:0/882239943 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f840c0a2c00 0x7f840c0a50c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f84280fff90 tx=0x7f8414005f90 comp rx=0 tx=0).stop 2024-09-10T09:52:27.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:27.324+0000 7f842ee00640 1 -- 172.21.15.77:0/882239943 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8428103ff0 msgr2=0x7f842819a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:27.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:27.324+0000 7f842ee00640 1 --2- 172.21.15.77:0/882239943 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8428103ff0 0x7f842819a600 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f841802ade0 tx=0x7f84180319f0 comp rx=0 tx=0).stop 2024-09-10T09:52:27.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:27.325+0000 7f842ee00640 1 -- 172.21.15.77:0/882239943 shutdown_connections 2024-09-10T09:52:27.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:27.325+0000 7f842ee00640 1 --2- 172.21.15.77:0/882239943 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f840c0a2c00 0x7f840c0a50c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:27.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:27.325+0000 7f842ee00640 1 --2- 172.21.15.77:0/882239943 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8428109120 0x7f842819eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:27.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:27.325+0000 7f842ee00640 1 --2- 172.21.15.77:0/882239943 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f84281049a0 0x7f842819ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:27.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:27.325+0000 7f842ee00640 1 --2- 172.21.15.77:0/882239943 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8428103ff0 0x7f842819a600 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:27.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:27.325+0000 7f842ee00640 1 -- 172.21.15.77:0/882239943 >> 172.21.15.77:0/882239943 conn(0x7f84280ffa90 msgr2=0x7f8428100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:27.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:27.325+0000 7f842ee00640 1 -- 172.21.15.77:0/882239943 shutdown_connections 2024-09-10T09:52:27.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:27.325+0000 7f842ee00640 1 -- 172.21.15.77:0/882239943 wait complete. 2024-09-10T09:52:27.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:27 smithi134 ceph-mon[35674]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-10T09:52:27.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:27 smithi134 ceph-mon[35674]: Cluster is now healthy 2024-09-10T09:52:27.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:27 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/882239943' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T09:52:27.667 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:27 smithi077 ceph-mon[30319]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-10T09:52:27.667 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:27 smithi077 ceph-mon[30319]: Cluster is now healthy 2024-09-10T09:52:27.667 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:27 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/882239943' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T09:52:27.673 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:27 smithi120 ceph-mon[34770]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-10T09:52:27.673 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:27 smithi120 ceph-mon[34770]: Cluster is now healthy 2024-09-10T09:52:27.673 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:27 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/882239943' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T09:52:28.070 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T09:52:28.439 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:52:28.609 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:28 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/882239943' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-10T09:52:28.609 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:28 smithi077 ceph-mon[30319]: osdmap e86: 12 total, 12 up, 12 in 2024-09-10T09:52:28.609 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:28 smithi077 ceph-mon[30319]: pgmap v300: 97 pgs: 97 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 381 B/s wr, 4 op/s 2024-09-10T09:52:28.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:28 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/882239943' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-10T09:52:28.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:28 smithi134 ceph-mon[35674]: osdmap e86: 12 total, 12 up, 12 in 2024-09-10T09:52:28.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:28 smithi134 ceph-mon[35674]: pgmap v300: 97 pgs: 97 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 381 B/s wr, 4 op/s 2024-09-10T09:52:28.673 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:28 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/882239943' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-10T09:52:28.673 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:28 smithi120 ceph-mon[34770]: osdmap e86: 12 total, 12 up, 12 in 2024-09-10T09:52:28.673 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:28 smithi120 ceph-mon[34770]: pgmap v300: 97 pgs: 97 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 381 B/s wr, 4 op/s 2024-09-10T09:52:29.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:29.027+0000 7fe85b600640 1 -- 172.21.15.77:0/206352105 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe854074f40 msgr2=0x7fe85410de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:29.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:29.027+0000 7fe85b600640 1 --2- 172.21.15.77:0/206352105 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe854074f40 0x7fe85410de80 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe840009900 tx=0x7fe84002a480 comp rx=0 tx=0).stop 2024-09-10T09:52:29.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:29.027+0000 7fe85b600640 1 -- 172.21.15.77:0/206352105 shutdown_connections 2024-09-10T09:52:29.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:29.027+0000 7fe85b600640 1 --2- 172.21.15.77:0/206352105 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe85410e3c0 0x7fe854110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:29.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:29.027+0000 7fe85b600640 1 --2- 172.21.15.77:0/206352105 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe854074f40 0x7fe85410de80 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:29.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:29.027+0000 7fe85b600640 1 --2- 172.21.15.77:0/206352105 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe854074620 0x7fe854074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:29.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:29.027+0000 7fe85b600640 1 -- 172.21.15.77:0/206352105 >> 172.21.15.77:0/206352105 conn(0x7fe8540fd8c0 msgr2=0x7fe8540ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:29.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:29.027+0000 7fe85b600640 1 -- 172.21.15.77:0/206352105 shutdown_connections 2024-09-10T09:52:29.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:29.027+0000 7fe85b600640 1 -- 172.21.15.77:0/206352105 wait complete. 2024-09-10T09:52:29.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:29.028+0000 7fe85b600640 1 Processor -- start 2024-09-10T09:52:29.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:29.029+0000 7fe85b600640 1 -- start start 2024-09-10T09:52:29.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:29.029+0000 7fe85b600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe854074620 0x7fe85410dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:29.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:29.029+0000 7fe85b600640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe854074f40 0x7fe85410ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:29.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:29.029+0000 7fe85b600640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe85410e3c0 0x7fe85410b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:29.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:29.029+0000 7fe85b600640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe854114160 con 0x7fe854074f40 2024-09-10T09:52:29.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:29.029+0000 7fe85b600640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe854113fe0 con 0x7fe85410e3c0 2024-09-10T09:52:29.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:29.029+0000 7fe85b600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe8541142e0 con 0x7fe854074620 2024-09-10T09:52:29.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:29.029+0000 7fe853e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe854074f40 0x7fe85410ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:29.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:29.029+0000 7fe853e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe854074f40 0x7fe85410ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:32990/0 (socket says 172.21.15.77:32990) 2024-09-10T09:52:29.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:29.029+0000 7fe853e00640 1 -- 172.21.15.77:0/2733612872 learned_addr learned my addr 172.21.15.77:0/2733612872 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:52:29.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:29.029+0000 7fe858c00640 1 --2- 172.21.15.77:0/2733612872 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe854074620 0x7fe85410dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:29.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:29.029+0000 7fe859600640 1 --2- 172.21.15.77:0/2733612872 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe85410e3c0 0x7fe85410b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:29.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:29.030+0000 7fe853e00640 1 -- 172.21.15.77:0/2733612872 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe854074620 msgr2=0x7fe85410dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:29.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:29.030+0000 7fe853e00640 1 --2- 172.21.15.77:0/2733612872 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe854074620 0x7fe85410dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:29.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:29.030+0000 7fe853e00640 1 -- 172.21.15.77:0/2733612872 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe85410e3c0 msgr2=0x7fe85410b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:29.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:29.030+0000 7fe853e00640 1 --2- 172.21.15.77:0/2733612872 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe85410e3c0 0x7fe85410b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:29.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:29.030+0000 7fe853e00640 1 -- 172.21.15.77:0/2733612872 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe85410bc60 con 0x7fe854074f40 2024-09-10T09:52:29.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:29.030+0000 7fe858c00640 1 --2- 172.21.15.77:0/2733612872 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe854074620 0x7fe85410dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:52:29.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:29.030+0000 7fe853e00640 1 --2- 172.21.15.77:0/2733612872 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe854074f40 0x7fe85410ae20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe840006010 tx=0x7fe8400042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:29.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:29.030+0000 7fe851600640 1 -- 172.21.15.77:0/2733612872 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe84003f030 con 0x7fe854074f40 2024-09-10T09:52:29.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:29.030+0000 7fe85b600640 1 -- 172.21.15.77:0/2733612872 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8541aaf60 con 0x7fe854074f40 2024-09-10T09:52:29.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:29.030+0000 7fe851600640 1 -- 172.21.15.77:0/2733612872 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe840003ce0 con 0x7fe854074f40 2024-09-10T09:52:29.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:29.030+0000 7fe851600640 1 -- 172.21.15.77:0/2733612872 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe840035070 con 0x7fe854074f40 2024-09-10T09:52:29.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:29.030+0000 7fe85b600640 1 -- 172.21.15.77:0/2733612872 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8541ab460 con 0x7fe854074f40 2024-09-10T09:52:29.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:29.032+0000 7fe85b600640 1 -- 172.21.15.77:0/2733612872 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe854075cd0 con 0x7fe854074f40 2024-09-10T09:52:29.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:29.033+0000 7fe851600640 1 -- 172.21.15.77:0/2733612872 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe840042030 con 0x7fe854074f40 2024-09-10T09:52:29.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:29.035+0000 7fe851600640 1 --2- 172.21.15.77:0/2733612872 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fe83c0a2b80 0x7fe83c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:29.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:29.035+0000 7fe851600640 1 -- 172.21.15.77:0/2733612872 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8277+0+0 (secure 0 0 0) 0x7fe8400ef490 con 0x7fe854074f40 2024-09-10T09:52:29.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:29.035+0000 7fe858c00640 1 --2- 172.21.15.77:0/2733612872 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fe83c0a2b80 0x7fe83c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:29.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:29.036+0000 7fe858c00640 1 --2- 172.21.15.77:0/2733612872 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fe83c0a2b80 0x7fe83c0a5040 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe848009fc0 tx=0x7fe848005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:29.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:29.039+0000 7fe851600640 1 -- 172.21.15.77:0/2733612872 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe8400bbdd0 con 0x7fe854074f40 2024-09-10T09:52:29.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:29.219+0000 7fe85b600640 1 -- 172.21.15.77:0/2733612872 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fe85402d7c0 con 0x7fe854074f40 2024-09-10T09:52:29.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:29 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/2733612872' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-10T09:52:29.673 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:29 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/2733612872' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-10T09:52:29.689 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:29 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/2733612872' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-10T09:52:30.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:30.345+0000 7fe851600640 1 -- 172.21.15.77:0/2733612872 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v88) ==== 228+0+0 (secure 0 0 0) 0x7fe8400c00d0 con 0x7fe854074f40 2024-09-10T09:52:30.347 INFO:teuthology.orchestra.run.smithi077.stderr:pool 'cephfs_data_ec' created 2024-09-10T09:52:30.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:30.349+0000 7fe85b600640 1 -- 172.21.15.77:0/2733612872 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fe83c0a2b80 msgr2=0x7fe83c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:30.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:30.349+0000 7fe85b600640 1 --2- 172.21.15.77:0/2733612872 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fe83c0a2b80 0x7fe83c0a5040 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe848009fc0 tx=0x7fe848005f90 comp rx=0 tx=0).stop 2024-09-10T09:52:30.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:30.349+0000 7fe85b600640 1 -- 172.21.15.77:0/2733612872 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe854074f40 msgr2=0x7fe85410ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:30.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:30.349+0000 7fe85b600640 1 --2- 172.21.15.77:0/2733612872 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe854074f40 0x7fe85410ae20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe840006010 tx=0x7fe8400042f0 comp rx=0 tx=0).stop 2024-09-10T09:52:30.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:30.350+0000 7fe85b600640 1 -- 172.21.15.77:0/2733612872 shutdown_connections 2024-09-10T09:52:30.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:30.350+0000 7fe85b600640 1 --2- 172.21.15.77:0/2733612872 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fe83c0a2b80 0x7fe83c0a5040 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:30.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:30.350+0000 7fe85b600640 1 --2- 172.21.15.77:0/2733612872 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe85410e3c0 0x7fe85410b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:30.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:30.350+0000 7fe85b600640 1 --2- 172.21.15.77:0/2733612872 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe854074f40 0x7fe85410ae20 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:30.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:30.350+0000 7fe85b600640 1 --2- 172.21.15.77:0/2733612872 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe854074620 0x7fe85410dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:30.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:30.350+0000 7fe85b600640 1 -- 172.21.15.77:0/2733612872 >> 172.21.15.77:0/2733612872 conn(0x7fe8540fd8c0 msgr2=0x7fe8540fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:30.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:30.350+0000 7fe85b600640 1 -- 172.21.15.77:0/2733612872 shutdown_connections 2024-09-10T09:52:30.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:30.350+0000 7fe85b600640 1 -- 172.21.15.77:0/2733612872 wait complete. 2024-09-10T09:52:30.374 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:30 smithi077 ceph-mon[30319]: pgmap v302: 97 pgs: 97 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-09-10T09:52:30.374 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:30 smithi077 ceph-mon[30319]: osdmap e87: 12 total, 12 up, 12 in 2024-09-10T09:52:30.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:30 smithi134 ceph-mon[35674]: pgmap v302: 97 pgs: 97 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-09-10T09:52:30.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:30 smithi134 ceph-mon[35674]: osdmap e87: 12 total, 12 up, 12 in 2024-09-10T09:52:30.673 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:30 smithi120 ceph-mon[34770]: pgmap v302: 97 pgs: 97 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-09-10T09:52:30.673 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:30 smithi120 ceph-mon[34770]: osdmap e87: 12 total, 12 up, 12 in 2024-09-10T09:52:31.276 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-09-10T09:52:31.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:31 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/2733612872' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-10T09:52:31.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:31 smithi134 ceph-mon[35674]: osdmap e88: 12 total, 12 up, 12 in 2024-09-10T09:52:31.660 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:52:31.673 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:31 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/2733612872' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-10T09:52:31.673 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:31 smithi120 ceph-mon[34770]: osdmap e88: 12 total, 12 up, 12 in 2024-09-10T09:52:31.689 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:31 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/2733612872' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-10T09:52:31.689 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:31 smithi077 ceph-mon[30319]: osdmap e88: 12 total, 12 up, 12 in 2024-09-10T09:52:32.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:32.274+0000 7f71bf200640 1 -- 172.21.15.77:0/3718296557 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f71c01079c0 msgr2=0x7f71c0107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:32.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:32.274+0000 7f71bf200640 1 --2- 172.21.15.77:0/3718296557 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f71c01079c0 0x7f71c0107da0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f71b0009930 tx=0x7f71b002a420 comp rx=0 tx=0).stop 2024-09-10T09:52:32.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:32.274+0000 7f71bf200640 1 -- 172.21.15.77:0/3718296557 shutdown_connections 2024-09-10T09:52:32.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:32.274+0000 7f71bf200640 1 --2- 172.21.15.77:0/3718296557 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71c0102360 0x7f71c010e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:32.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:32.274+0000 7f71bf200640 1 --2- 172.21.15.77:0/3718296557 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f71c01019c0 0x7f71c0101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:32.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:32.274+0000 7f71bf200640 1 --2- 172.21.15.77:0/3718296557 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f71c01079c0 0x7f71c0107da0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:32.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:32.274+0000 7f71bf200640 1 -- 172.21.15.77:0/3718296557 >> 172.21.15.77:0/3718296557 conn(0x7f71c00fd740 msgr2=0x7f71c00ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:32.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:32.274+0000 7f71bf200640 1 -- 172.21.15.77:0/3718296557 shutdown_connections 2024-09-10T09:52:32.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:32.274+0000 7f71bf200640 1 -- 172.21.15.77:0/3718296557 wait complete. 2024-09-10T09:52:32.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:32.275+0000 7f71bf200640 1 Processor -- start 2024-09-10T09:52:32.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:32.275+0000 7f71bf200640 1 -- start start 2024-09-10T09:52:32.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:32.275+0000 7f71bf200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f71c01019c0 0x7f71c0198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:32.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:32.275+0000 7f71bf200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71c0102360 0x7f71c01987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:32.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:32.276+0000 7f71bf200640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f71c01079c0 0x7f71c019cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:32.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:32.276+0000 7f71bf200640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f71c0111e10 con 0x7f71c01019c0 2024-09-10T09:52:32.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:32.276+0000 7f71bf200640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f71c0111c90 con 0x7f71c01079c0 2024-09-10T09:52:32.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:32.276+0000 7f71bf200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f71c0111f90 con 0x7f71c0102360 2024-09-10T09:52:32.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:32.276+0000 7f71bde00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f71c01019c0 0x7f71c0198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:32.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:32.276+0000 7f71bde00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f71c01019c0 0x7f71c0198270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33006/0 (socket says 172.21.15.77:33006) 2024-09-10T09:52:32.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:32.276+0000 7f71bde00640 1 -- 172.21.15.77:0/3081715916 learned_addr learned my addr 172.21.15.77:0/3081715916 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:52:32.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:32.276+0000 7f71b7e00640 1 --2- 172.21.15.77:0/3081715916 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71c0102360 0x7f71c01987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:32.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:32.276+0000 7f71be800640 1 --2- 172.21.15.77:0/3081715916 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f71c01079c0 0x7f71c019cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:32.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:32.276+0000 7f71bde00640 1 -- 172.21.15.77:0/3081715916 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71c0102360 msgr2=0x7f71c01987b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:32.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:32.276+0000 7f71bde00640 1 --2- 172.21.15.77:0/3081715916 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71c0102360 0x7f71c01987b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:32.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:32.276+0000 7f71bde00640 1 -- 172.21.15.77:0/3081715916 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f71c01079c0 msgr2=0x7f71c019cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:32.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:32.276+0000 7f71bde00640 1 --2- 172.21.15.77:0/3081715916 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f71c01079c0 0x7f71c019cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:32.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:32.276+0000 7f71bde00640 1 -- 172.21.15.77:0/3081715916 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71c019d220 con 0x7f71c01019c0 2024-09-10T09:52:32.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:32.276+0000 7f71be800640 1 --2- 172.21.15.77:0/3081715916 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f71c01079c0 0x7f71c019cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:52:32.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:32.277+0000 7f71bde00640 1 --2- 172.21.15.77:0/3081715916 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f71c01019c0 0x7f71c0198270 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f71b002a940 tx=0x7f71b0003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:32.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:32.277+0000 7f71b6a00640 1 -- 172.21.15.77:0/3081715916 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b003f070 con 0x7f71c01019c0 2024-09-10T09:52:32.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:32.277+0000 7f71b6a00640 1 -- 172.21.15.77:0/3081715916 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71b0031c90 con 0x7f71c01019c0 2024-09-10T09:52:32.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:32.277+0000 7f71bf200640 1 -- 172.21.15.77:0/3081715916 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71c019d4b0 con 0x7f71c01019c0 2024-09-10T09:52:32.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:32.277+0000 7f71b6a00640 1 -- 172.21.15.77:0/3081715916 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b0035030 con 0x7f71c01019c0 2024-09-10T09:52:32.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:32.277+0000 7f71bf200640 1 -- 172.21.15.77:0/3081715916 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71c01a4df0 con 0x7f71c01019c0 2024-09-10T09:52:32.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:32.278+0000 7f71bf200640 1 -- 172.21.15.77:0/3081715916 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71c0103100 con 0x7f71c01019c0 2024-09-10T09:52:32.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:32.280+0000 7f71b6a00640 1 -- 172.21.15.77:0/3081715916 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f71b0042070 con 0x7f71c01019c0 2024-09-10T09:52:32.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:32.281+0000 7f71b6a00640 1 --2- 172.21.15.77:0/3081715916 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f71a40a2b80 0x7f71a40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:32.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:32.281+0000 7f71b6a00640 1 -- 172.21.15.77:0/3081715916 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8770+0+0 (secure 0 0 0) 0x7f71b00efa70 con 0x7f71c01019c0 2024-09-10T09:52:32.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:32.282+0000 7f71b7e00640 1 --2- 172.21.15.77:0/3081715916 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f71a40a2b80 0x7f71a40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:32.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:32.282+0000 7f71b7e00640 1 --2- 172.21.15.77:0/3081715916 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f71a40a2b80 0x7f71a40a5040 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f71c0106fb0 tx=0x7f71a8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:32.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:32.286+0000 7f71b6a00640 1 -- 172.21.15.77:0/3081715916 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f71b00bc750 con 0x7f71c01019c0 2024-09-10T09:52:32.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:32 smithi077 ceph-mon[30319]: osdmap e89: 12 total, 12 up, 12 in 2024-09-10T09:52:32.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:32 smithi077 ceph-mon[30319]: pgmap v305: 129 pgs: 5 creating+peering, 27 unknown, 97 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s wr, 4 op/s 2024-09-10T09:52:32.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:32.465+0000 7f71bf200640 1 -- 172.21.15.77:0/3081715916 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f71c0109f30 con 0x7f71c01019c0 2024-09-10T09:52:32.632 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:32 smithi134 ceph-mon[35674]: osdmap e89: 12 total, 12 up, 12 in 2024-09-10T09:52:32.632 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:32 smithi134 ceph-mon[35674]: pgmap v305: 129 pgs: 5 creating+peering, 27 unknown, 97 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s wr, 4 op/s 2024-09-10T09:52:32.673 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:32 smithi120 ceph-mon[34770]: osdmap e89: 12 total, 12 up, 12 in 2024-09-10T09:52:32.673 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:32 smithi120 ceph-mon[34770]: pgmap v305: 129 pgs: 5 creating+peering, 27 unknown, 97 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s wr, 4 op/s 2024-09-10T09:52:33.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:33.369+0000 7f71b6a00640 1 -- 172.21.15.77:0/3081715916 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v91) ==== 169+0+0 (secure 0 0 0) 0x7f71b00dc110 con 0x7f71c01019c0 2024-09-10T09:52:33.372 INFO:teuthology.orchestra.run.smithi077.stderr:set pool 4 allow_ec_overwrites to true 2024-09-10T09:52:33.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:33.373+0000 7f71bf200640 1 -- 172.21.15.77:0/3081715916 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f71a40a2b80 msgr2=0x7f71a40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:33.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:33.373+0000 7f71bf200640 1 --2- 172.21.15.77:0/3081715916 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f71a40a2b80 0x7f71a40a5040 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f71c0106fb0 tx=0x7f71a8005f90 comp rx=0 tx=0).stop 2024-09-10T09:52:33.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:33.373+0000 7f71bf200640 1 -- 172.21.15.77:0/3081715916 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f71c01019c0 msgr2=0x7f71c0198270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:33.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:33.373+0000 7f71bf200640 1 --2- 172.21.15.77:0/3081715916 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f71c01019c0 0x7f71c0198270 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f71b002a940 tx=0x7f71b0003b90 comp rx=0 tx=0).stop 2024-09-10T09:52:33.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:33.373+0000 7f71bf200640 1 -- 172.21.15.77:0/3081715916 shutdown_connections 2024-09-10T09:52:33.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:33.373+0000 7f71bf200640 1 --2- 172.21.15.77:0/3081715916 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f71a40a2b80 0x7f71a40a5040 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:33.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:33.373+0000 7f71bf200640 1 --2- 172.21.15.77:0/3081715916 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f71c01079c0 0x7f71c019cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:33.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:33.373+0000 7f71bf200640 1 --2- 172.21.15.77:0/3081715916 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71c0102360 0x7f71c01987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:33.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:33.373+0000 7f71bf200640 1 --2- 172.21.15.77:0/3081715916 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f71c01019c0 0x7f71c0198270 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:33.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:33.373+0000 7f71bf200640 1 -- 172.21.15.77:0/3081715916 >> 172.21.15.77:0/3081715916 conn(0x7f71c00fd740 msgr2=0x7f71c00fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:33.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:33.374+0000 7f71bf200640 1 -- 172.21.15.77:0/3081715916 shutdown_connections 2024-09-10T09:52:33.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:33.374+0000 7f71bf200640 1 -- 172.21.15.77:0/3081715916 wait complete. 2024-09-10T09:52:33.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:33 smithi134 ceph-mon[35674]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T09:52:33.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:33 smithi134 ceph-mon[35674]: osdmap e90: 12 total, 12 up, 12 in 2024-09-10T09:52:33.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:33 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/3081715916' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-10T09:52:33.673 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:33 smithi120 ceph-mon[34770]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T09:52:33.673 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:33 smithi120 ceph-mon[34770]: osdmap e90: 12 total, 12 up, 12 in 2024-09-10T09:52:33.673 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:33 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/3081715916' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-10T09:52:33.689 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:33 smithi077 ceph-mon[30319]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T09:52:33.689 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:33 smithi077 ceph-mon[30319]: osdmap e90: 12 total, 12 up, 12 in 2024-09-10T09:52:33.689 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:33 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/3081715916' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-10T09:52:34.213 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph fs add_data_pool cephfs cephfs_data_ec 2024-09-10T09:52:34.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:34 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/3081715916' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-10T09:52:34.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:34 smithi077 ceph-mon[30319]: osdmap e91: 12 total, 12 up, 12 in 2024-09-10T09:52:34.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:34 smithi077 ceph-mon[30319]: pgmap v308: 129 pgs: 5 creating+peering, 15 unknown, 109 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:52:34.583 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:52:34.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:34 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/3081715916' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-10T09:52:34.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:34 smithi134 ceph-mon[35674]: osdmap e91: 12 total, 12 up, 12 in 2024-09-10T09:52:34.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:34 smithi134 ceph-mon[35674]: pgmap v308: 129 pgs: 5 creating+peering, 15 unknown, 109 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:52:34.673 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:34 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/3081715916' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-10T09:52:34.674 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:34 smithi120 ceph-mon[34770]: osdmap e91: 12 total, 12 up, 12 in 2024-09-10T09:52:34.674 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:34 smithi120 ceph-mon[34770]: pgmap v308: 129 pgs: 5 creating+peering, 15 unknown, 109 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:52:35.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:35.155+0000 7fe461000640 1 -- 172.21.15.77:0/2819880462 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe45c101de0 msgr2=0x7fe45c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:35.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:35.155+0000 7fe461000640 1 --2- 172.21.15.77:0/2819880462 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe45c101de0 0x7fe45c1021c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe44c009900 tx=0x7fe44c02a410 comp rx=0 tx=0).stop 2024-09-10T09:52:35.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:35.155+0000 7fe461000640 1 -- 172.21.15.77:0/2819880462 shutdown_connections 2024-09-10T09:52:35.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:35.155+0000 7fe461000640 1 --2- 172.21.15.77:0/2819880462 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe45c106f10 0x7fe45c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:35.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:35.155+0000 7fe461000640 1 --2- 172.21.15.77:0/2819880462 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe45c102790 0x7fe45c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:35.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:35.155+0000 7fe461000640 1 --2- 172.21.15.77:0/2819880462 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe45c101de0 0x7fe45c1021c0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:35.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:35.155+0000 7fe461000640 1 -- 172.21.15.77:0/2819880462 >> 172.21.15.77:0/2819880462 conn(0x7fe45c0fd8c0 msgr2=0x7fe45c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:35.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:35.155+0000 7fe461000640 1 -- 172.21.15.77:0/2819880462 shutdown_connections 2024-09-10T09:52:35.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:35.155+0000 7fe461000640 1 -- 172.21.15.77:0/2819880462 wait complete. 2024-09-10T09:52:35.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:35.156+0000 7fe461000640 1 Processor -- start 2024-09-10T09:52:35.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:35.156+0000 7fe461000640 1 -- start start 2024-09-10T09:52:35.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:35.157+0000 7fe461000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe45c101de0 0x7fe45c19a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:35.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:35.157+0000 7fe461000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe45c102790 0x7fe45c19abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:35.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:35.157+0000 7fe461000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe45c106f10 0x7fe45c19ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:35.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:35.157+0000 7fe461000640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe45c111f80 con 0x7fe45c106f10 2024-09-10T09:52:35.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:35.157+0000 7fe461000640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe45c111e00 con 0x7fe45c101de0 2024-09-10T09:52:35.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:35.157+0000 7fe461000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe45c112100 con 0x7fe45c102790 2024-09-10T09:52:35.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:35.157+0000 7fe45a800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe45c106f10 0x7fe45c19ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:35.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:35.157+0000 7fe45a800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe45c106f10 0x7fe45c19ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:39102/0 (socket says 172.21.15.77:39102) 2024-09-10T09:52:35.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:35.157+0000 7fe45a800640 1 -- 172.21.15.77:0/1247221706 learned_addr learned my addr 172.21.15.77:0/1247221706 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:52:35.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:35.157+0000 7fe459e00640 1 --2- 172.21.15.77:0/1247221706 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe45c101de0 0x7fe45c19a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:35.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:35.157+0000 7fe459400640 1 --2- 172.21.15.77:0/1247221706 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe45c102790 0x7fe45c19abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:35.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:35.158+0000 7fe45a800640 1 -- 172.21.15.77:0/1247221706 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe45c102790 msgr2=0x7fe45c19abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:35.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:35.158+0000 7fe45a800640 1 --2- 172.21.15.77:0/1247221706 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe45c102790 0x7fe45c19abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:35.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:35.158+0000 7fe45a800640 1 -- 172.21.15.77:0/1247221706 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe45c101de0 msgr2=0x7fe45c19a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:35.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:35.158+0000 7fe45a800640 1 --2- 172.21.15.77:0/1247221706 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe45c101de0 0x7fe45c19a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:35.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:35.158+0000 7fe45a800640 1 -- 172.21.15.77:0/1247221706 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe45c19f620 con 0x7fe45c106f10 2024-09-10T09:52:35.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:35.158+0000 7fe45a800640 1 --2- 172.21.15.77:0/1247221706 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe45c106f10 0x7fe45c19ef40 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe45400eaa0 tx=0x7fe45400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:35.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:35.158+0000 7fe452a00640 1 -- 172.21.15.77:0/1247221706 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe45400cdb0 con 0x7fe45c106f10 2024-09-10T09:52:35.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:35.158+0000 7fe452a00640 1 -- 172.21.15.77:0/1247221706 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe454005020 con 0x7fe45c106f10 2024-09-10T09:52:35.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:35.158+0000 7fe461000640 1 -- 172.21.15.77:0/1247221706 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe45c19f910 con 0x7fe45c106f10 2024-09-10T09:52:35.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:35.158+0000 7fe452a00640 1 -- 172.21.15.77:0/1247221706 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4540116b0 con 0x7fe45c106f10 2024-09-10T09:52:35.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:35.158+0000 7fe461000640 1 -- 172.21.15.77:0/1247221706 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe45c1a71f0 con 0x7fe45c106f10 2024-09-10T09:52:35.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:35.159+0000 7fe461000640 1 -- 172.21.15.77:0/1247221706 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe45c1021c0 con 0x7fe45c106f10 2024-09-10T09:52:35.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:35.161+0000 7fe452a00640 1 -- 172.21.15.77:0/1247221706 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe454005560 con 0x7fe45c106f10 2024-09-10T09:52:35.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:35.162+0000 7fe452a00640 1 --2- 172.21.15.77:0/1247221706 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fe4400a2c00 0x7fe4400a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:35.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:35.163+0000 7fe452a00640 1 -- 172.21.15.77:0/1247221706 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8770+0+0 (secure 0 0 0) 0x7fe4540cf440 con 0x7fe45c106f10 2024-09-10T09:52:35.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:35.163+0000 7fe459e00640 1 --2- 172.21.15.77:0/1247221706 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fe4400a2c00 0x7fe4400a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:35.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:35.163+0000 7fe459e00640 1 --2- 172.21.15.77:0/1247221706 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fe4400a2c00 0x7fe4400a50c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe44c000c00 tx=0x7fe44c005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:35.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:35.167+0000 7fe452a00640 1 -- 172.21.15.77:0/1247221706 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe454016030 con 0x7fe45c106f10 2024-09-10T09:52:35.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:35.382+0000 7fe461000640 1 -- 172.21.15.77:0/1247221706 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7fe45c1a7520 con 0x7fe45c106f10 2024-09-10T09:52:35.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:35.435+0000 7fe452a00640 1 -- 172.21.15.77:0/1247221706 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) ==== 137+0+0 (secure 0 0 0) 0x7fe45409c120 con 0x7fe45c106f10 2024-09-10T09:52:35.437 INFO:teuthology.orchestra.run.smithi077.stderr:added data pool 4 to fsmap 2024-09-10T09:52:35.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:35.438+0000 7fe461000640 1 -- 172.21.15.77:0/1247221706 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fe4400a2c00 msgr2=0x7fe4400a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:35.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:35.438+0000 7fe461000640 1 --2- 172.21.15.77:0/1247221706 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fe4400a2c00 0x7fe4400a50c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe44c000c00 tx=0x7fe44c005b50 comp rx=0 tx=0).stop 2024-09-10T09:52:35.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:35.438+0000 7fe461000640 1 -- 172.21.15.77:0/1247221706 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe45c106f10 msgr2=0x7fe45c19ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:35.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:35.438+0000 7fe461000640 1 --2- 172.21.15.77:0/1247221706 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe45c106f10 0x7fe45c19ef40 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe45400eaa0 tx=0x7fe45400ef60 comp rx=0 tx=0).stop 2024-09-10T09:52:35.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:35.438+0000 7fe461000640 1 -- 172.21.15.77:0/1247221706 shutdown_connections 2024-09-10T09:52:35.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:35.438+0000 7fe461000640 1 --2- 172.21.15.77:0/1247221706 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fe4400a2c00 0x7fe4400a50c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:35.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:35.438+0000 7fe461000640 1 --2- 172.21.15.77:0/1247221706 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe45c106f10 0x7fe45c19ef40 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:35.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:35.438+0000 7fe461000640 1 --2- 172.21.15.77:0/1247221706 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe45c102790 0x7fe45c19abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:35.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:35.438+0000 7fe461000640 1 --2- 172.21.15.77:0/1247221706 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe45c101de0 0x7fe45c19a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:35.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:35.438+0000 7fe461000640 1 -- 172.21.15.77:0/1247221706 >> 172.21.15.77:0/1247221706 conn(0x7fe45c0fd8c0 msgr2=0x7fe45c0feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:35.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:35.439+0000 7fe461000640 1 -- 172.21.15.77:0/1247221706 shutdown_connections 2024-09-10T09:52:35.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:35.439+0000 7fe461000640 1 -- 172.21.15.77:0/1247221706 wait complete. 2024-09-10T09:52:35.689 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:35 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/1247221706' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-10T09:52:35.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:35 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/1247221706' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-10T09:52:35.923 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:35 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/1247221706' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-10T09:52:36.430 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:52:36.689 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:36 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/1247221706' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-10T09:52:36.689 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:36 smithi077 ceph-mon[30319]: osdmap e92: 12 total, 12 up, 12 in 2024-09-10T09:52:36.689 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:36 smithi077 ceph-mon[30319]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T09:52:36.689 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:36 smithi077 ceph-mon[30319]: pgmap v310: 129 pgs: 5 creating+peering, 124 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:52:36.799 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:52:36.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:36 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/1247221706' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-10T09:52:36.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:36 smithi134 ceph-mon[35674]: osdmap e92: 12 total, 12 up, 12 in 2024-09-10T09:52:36.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:36 smithi134 ceph-mon[35674]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T09:52:36.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:36 smithi134 ceph-mon[35674]: pgmap v310: 129 pgs: 5 creating+peering, 124 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:52:36.923 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:36 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/1247221706' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-10T09:52:36.923 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:36 smithi120 ceph-mon[34770]: osdmap e92: 12 total, 12 up, 12 in 2024-09-10T09:52:36.923 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:36 smithi120 ceph-mon[34770]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T09:52:36.923 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:36 smithi120 ceph-mon[34770]: pgmap v310: 129 pgs: 5 creating+peering, 124 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:52:37.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:37.328+0000 7f92e0400640 1 -- 172.21.15.77:0/4034522031 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f92d8106f10 msgr2=0x7f92d810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:37.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:37.328+0000 7f92e0400640 1 --2- 172.21.15.77:0/4034522031 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f92d8106f10 0x7f92d810ea90 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f92d0009ca0 tx=0x7f92d002a8c0 comp rx=0 tx=0).stop 2024-09-10T09:52:37.331 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:37.328+0000 7f92e0400640 1 -- 172.21.15.77:0/4034522031 shutdown_connections 2024-09-10T09:52:37.331 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:37.328+0000 7f92e0400640 1 --2- 172.21.15.77:0/4034522031 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f92d8106f10 0x7f92d810ea90 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:37.331 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:37.329+0000 7f92e0400640 1 --2- 172.21.15.77:0/4034522031 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f92d8102790 0x7f92d81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:37.331 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:37.329+0000 7f92e0400640 1 --2- 172.21.15.77:0/4034522031 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92d8101de0 0x7f92d81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:37.331 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:37.329+0000 7f92e0400640 1 -- 172.21.15.77:0/4034522031 >> 172.21.15.77:0/4034522031 conn(0x7f92d80fd8c0 msgr2=0x7f92d80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:37.331 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:37.329+0000 7f92e0400640 1 -- 172.21.15.77:0/4034522031 shutdown_connections 2024-09-10T09:52:37.331 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:37.329+0000 7f92e0400640 1 -- 172.21.15.77:0/4034522031 wait complete. 2024-09-10T09:52:37.332 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:37.330+0000 7f92e0400640 1 Processor -- start 2024-09-10T09:52:37.332 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:37.330+0000 7f92e0400640 1 -- start start 2024-09-10T09:52:37.332 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:37.330+0000 7f92e0400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f92d8101de0 0x7f92d819a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:37.332 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:37.330+0000 7f92e0400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f92d8102790 0x7f92d819ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:37.332 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:37.330+0000 7f92e0400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92d8106f10 0x7f92d819ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:37.332 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:37.330+0000 7f92e0400640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f92d8111fc0 con 0x7f92d8101de0 2024-09-10T09:52:37.333 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:37.330+0000 7f92e0400640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f92d8111e40 con 0x7f92d8102790 2024-09-10T09:52:37.333 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:37.330+0000 7f92e0400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f92d8112140 con 0x7f92d8106f10 2024-09-10T09:52:37.333 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:37.330+0000 7f92dda00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f92d8101de0 0x7f92d819a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:37.333 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:37.330+0000 7f92dda00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f92d8101de0 0x7f92d819a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:39118/0 (socket says 172.21.15.77:39118) 2024-09-10T09:52:37.333 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:37.330+0000 7f92dda00640 1 -- 172.21.15.77:0/3976312630 learned_addr learned my addr 172.21.15.77:0/3976312630 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:52:37.333 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:37.331+0000 7f92de400640 1 --2- 172.21.15.77:0/3976312630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92d8106f10 0x7f92d819ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:37.333 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:37.331+0000 7f92dd000640 1 --2- 172.21.15.77:0/3976312630 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f92d8102790 0x7f92d819ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:37.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:37.331+0000 7f92dda00640 1 -- 172.21.15.77:0/3976312630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92d8106f10 msgr2=0x7f92d819ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:37.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:37.331+0000 7f92dda00640 1 --2- 172.21.15.77:0/3976312630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92d8106f10 0x7f92d819ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:37.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:37.331+0000 7f92dda00640 1 -- 172.21.15.77:0/3976312630 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f92d8102790 msgr2=0x7f92d819ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:37.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:37.331+0000 7f92dda00640 1 --2- 172.21.15.77:0/3976312630 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f92d8102790 0x7f92d819ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:37.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:37.331+0000 7f92dda00640 1 -- 172.21.15.77:0/3976312630 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92d819f5f0 con 0x7f92d8101de0 2024-09-10T09:52:37.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:37.331+0000 7f92dda00640 1 --2- 172.21.15.77:0/3976312630 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f92d8101de0 0x7f92d819a640 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f92cc00e9f0 tx=0x7f92cc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:37.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:37.331+0000 7f92d6000640 1 -- 172.21.15.77:0/3976312630 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92cc00cd80 con 0x7f92d8101de0 2024-09-10T09:52:37.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:37.331+0000 7f92d6000640 1 -- 172.21.15.77:0/3976312630 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92cc005500 con 0x7f92d8101de0 2024-09-10T09:52:37.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:37.332+0000 7f92d6000640 1 -- 172.21.15.77:0/3976312630 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92cc010690 con 0x7f92d8101de0 2024-09-10T09:52:37.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:37.332+0000 7f92e0400640 1 -- 172.21.15.77:0/3976312630 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92d819f8e0 con 0x7f92d8101de0 2024-09-10T09:52:37.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:37.332+0000 7f92e0400640 1 -- 172.21.15.77:0/3976312630 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92d81a71c0 con 0x7f92d8101de0 2024-09-10T09:52:37.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:37.333+0000 7f92e0400640 1 -- 172.21.15.77:0/3976312630 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92d81021c0 con 0x7f92d8101de0 2024-09-10T09:52:37.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:37.334+0000 7f92d6000640 1 -- 172.21.15.77:0/3976312630 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f92cc0046c0 con 0x7f92d8101de0 2024-09-10T09:52:37.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:37.336+0000 7f92d6000640 1 --2- 172.21.15.77:0/3976312630 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f92c00a2b80 0x7f92c00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:37.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:37.336+0000 7f92d6000640 1 -- 172.21.15.77:0/3976312630 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7f92cc0cf660 con 0x7f92d8101de0 2024-09-10T09:52:37.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:37.336+0000 7f92dd000640 1 --2- 172.21.15.77:0/3976312630 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f92c00a2b80 0x7f92c00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:37.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:37.337+0000 7f92dd000640 1 --2- 172.21.15.77:0/3976312630 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f92c00a2b80 0x7f92c00a5040 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f92d8076480 tx=0x7f92c400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:37.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:37.340+0000 7f92d6000640 1 -- 172.21.15.77:0/3976312630 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f92cc09c380 con 0x7f92d8101de0 2024-09-10T09:52:37.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:37.547+0000 7f92e0400640 1 -- 172.21.15.77:0/3976312630 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f92d819b9a0 con 0x7f92d8101de0 2024-09-10T09:52:37.550 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:37 smithi077 ceph-mon[30319]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-10T09:52:37.550 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:37 smithi077 ceph-mon[30319]: Cluster is now healthy 2024-09-10T09:52:37.550 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:37.548+0000 7f92d6000640 1 -- 172.21.15.77:0/3976312630 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11854 (secure 0 0 0) 0x7f92cc09bc80 con 0x7f92d8101de0 2024-09-10T09:52:37.551 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:52:37.551 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":17,"btime":"2024-09-10T09:52:35:434708+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3159289675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3159289675},{"type":"v1","addr":"172.21.15.77:6835","nonce":3159289675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1951962341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1951962341},{"type":"v1","addr":"172.21.15.77:6837","nonce":1951962341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14487,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6839/4291121888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":4291121888},{"type":"v1","addr":"172.21.15.77:6839","nonce":4291121888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14499,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6841/445284194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":445284194},{"type":"v1","addr":"172.21.15.77:6841","nonce":445284194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/1005054423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1005054423},{"type":"v1","addr":"172.21.15.134:6833","nonce":1005054423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/3716307744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":3716307744},{"type":"v1","addr":"172.21.15.120:6833","nonce":3716307744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/1319892544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1319892544},{"type":"v1","addr":"172.21.15.120:6835","nonce":1319892544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24496,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/3293933699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3293933699},{"type":"v1","addr":"172.21.15.134:6835","nonce":3293933699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24502,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/2461497091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2461497091},{"type":"v1","addr":"172.21.15.134:6837","nonce":2461497091}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24520,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6839/1965117680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1965117680},{"type":"v1","addr":"172.21.15.134:6839","nonce":1965117680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/3033280709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3033280709},{"type":"v1","addr":"172.21.15.120:6837","nonce":3033280709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T09:52:23.368252+0000","modified":"2024-09-10T09:52:35.384462+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24563},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.120:6839/1777710157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1777710157},{"type":"v1","addr":"172.21.15.120:6839","nonce":1777710157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24563,"qdb_cluster":[24563]},"id":1}]} 2024-09-10T09:52:37.551 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 17 2024-09-10T09:52:37.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:37.552+0000 7f92e0400640 1 -- 172.21.15.77:0/3976312630 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f92c00a2b80 msgr2=0x7f92c00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:37.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:37.552+0000 7f92e0400640 1 --2- 172.21.15.77:0/3976312630 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f92c00a2b80 0x7f92c00a5040 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f92d8076480 tx=0x7f92c400b410 comp rx=0 tx=0).stop 2024-09-10T09:52:37.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:37.552+0000 7f92e0400640 1 -- 172.21.15.77:0/3976312630 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f92d8101de0 msgr2=0x7f92d819a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:37.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:37.552+0000 7f92e0400640 1 --2- 172.21.15.77:0/3976312630 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f92d8101de0 0x7f92d819a640 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f92cc00e9f0 tx=0x7f92cc00edf0 comp rx=0 tx=0).stop 2024-09-10T09:52:37.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:37.552+0000 7f92e0400640 1 -- 172.21.15.77:0/3976312630 shutdown_connections 2024-09-10T09:52:37.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:37.552+0000 7f92e0400640 1 --2- 172.21.15.77:0/3976312630 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f92c00a2b80 0x7f92c00a5040 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:37.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:37.552+0000 7f92e0400640 1 --2- 172.21.15.77:0/3976312630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92d8106f10 0x7f92d819ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:37.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:37.552+0000 7f92e0400640 1 --2- 172.21.15.77:0/3976312630 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f92d8102790 0x7f92d819ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:37.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:37.552+0000 7f92e0400640 1 --2- 172.21.15.77:0/3976312630 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f92d8101de0 0x7f92d819a640 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:37.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:37.552+0000 7f92e0400640 1 -- 172.21.15.77:0/3976312630 >> 172.21.15.77:0/3976312630 conn(0x7f92d80fd8c0 msgr2=0x7f92d80fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:37.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:37.552+0000 7f92e0400640 1 -- 172.21.15.77:0/3976312630 shutdown_connections 2024-09-10T09:52:37.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:37.552+0000 7f92e0400640 1 -- 172.21.15.77:0/3976312630 wait complete. 2024-09-10T09:52:37.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:37 smithi134 ceph-mon[35674]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-10T09:52:37.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:37 smithi134 ceph-mon[35674]: Cluster is now healthy 2024-09-10T09:52:37.923 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:37 smithi120 ceph-mon[34770]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-10T09:52:37.923 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:37 smithi120 ceph-mon[34770]: Cluster is now healthy 2024-09-10T09:52:38.510 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T09:52:38.876 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:52:38.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:38 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/3976312630' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:52:38.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:38 smithi134 ceph-mon[35674]: pgmap v311: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:52:38.923 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:38 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/3976312630' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:52:38.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:38 smithi120 ceph-mon[34770]: pgmap v311: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:52:38.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:38 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/3976312630' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:52:38.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:38 smithi077 ceph-mon[30319]: pgmap v311: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:52:39.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:39.458+0000 7fe345a00640 1 -- 172.21.15.77:0/429799 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe340101de0 msgr2=0x7fe3401021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:39.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:39.458+0000 7fe345a00640 1 --2- 172.21.15.77:0/429799 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe340101de0 0x7fe3401021c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe330009900 tx=0x7fe33002a3f0 comp rx=0 tx=0).stop 2024-09-10T09:52:39.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:39.459+0000 7fe345a00640 1 -- 172.21.15.77:0/429799 shutdown_connections 2024-09-10T09:52:39.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:39.459+0000 7fe345a00640 1 --2- 172.21.15.77:0/429799 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe340106f10 0x7fe34010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:39.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:39.459+0000 7fe345a00640 1 --2- 172.21.15.77:0/429799 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe340102790 0x7fe3401067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:39.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:39.459+0000 7fe345a00640 1 --2- 172.21.15.77:0/429799 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe340101de0 0x7fe3401021c0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:39.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:39.459+0000 7fe345a00640 1 -- 172.21.15.77:0/429799 >> 172.21.15.77:0/429799 conn(0x7fe3400fd8c0 msgr2=0x7fe3400ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:39.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:39.459+0000 7fe345a00640 1 -- 172.21.15.77:0/429799 shutdown_connections 2024-09-10T09:52:39.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:39.459+0000 7fe345a00640 1 -- 172.21.15.77:0/429799 wait complete. 2024-09-10T09:52:39.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:39.460+0000 7fe345a00640 1 Processor -- start 2024-09-10T09:52:39.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:39.460+0000 7fe345a00640 1 -- start start 2024-09-10T09:52:39.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:39.460+0000 7fe345a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe340101de0 0x7fe3400744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:39.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:39.460+0000 7fe345a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe340102790 0x7fe340074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:39.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:39.460+0000 7fe345a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe340106f10 0x7fe340074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:39.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:39.460+0000 7fe345a00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe340111fc0 con 0x7fe340102790 2024-09-10T09:52:39.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:39.460+0000 7fe345a00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe340111e40 con 0x7fe340106f10 2024-09-10T09:52:39.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:39.460+0000 7fe345a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe340112140 con 0x7fe340101de0 2024-09-10T09:52:39.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:39.460+0000 7fe33ea00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe340101de0 0x7fe3400744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:39.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:39.460+0000 7fe33e000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe340102790 0x7fe340074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:39.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:39.460+0000 7fe33ea00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe340101de0 0x7fe3400744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:59112/0 (socket says 172.21.15.77:59112) 2024-09-10T09:52:39.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:39.460+0000 7fe33ea00640 1 -- 172.21.15.77:0/1409326476 learned_addr learned my addr 172.21.15.77:0/1409326476 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:52:39.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:39.460+0000 7fe33f400640 1 --2- 172.21.15.77:0/1409326476 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe340106f10 0x7fe340074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:39.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:39.461+0000 7fe33ea00640 1 -- 172.21.15.77:0/1409326476 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe340106f10 msgr2=0x7fe340074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:39.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:39.461+0000 7fe33ea00640 1 --2- 172.21.15.77:0/1409326476 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe340106f10 0x7fe340074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:39.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:39.461+0000 7fe33ea00640 1 -- 172.21.15.77:0/1409326476 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe340102790 msgr2=0x7fe340074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:39.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:39.461+0000 7fe33ea00640 1 --2- 172.21.15.77:0/1409326476 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe340102790 0x7fe340074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:39.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:39.461+0000 7fe33ea00640 1 -- 172.21.15.77:0/1409326476 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3401a6900 con 0x7fe340101de0 2024-09-10T09:52:39.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:39.461+0000 7fe33e000640 1 --2- 172.21.15.77:0/1409326476 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe340102790 0x7fe340074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:52:39.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:39.461+0000 7fe33ea00640 1 --2- 172.21.15.77:0/1409326476 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe340101de0 0x7fe3400744f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe330004770 tx=0x7fe330003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:39.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:39.461+0000 7fe337400640 1 -- 172.21.15.77:0/1409326476 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe33003f070 con 0x7fe340101de0 2024-09-10T09:52:39.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:39.461+0000 7fe345a00640 1 -- 172.21.15.77:0/1409326476 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3401a6b90 con 0x7fe340101de0 2024-09-10T09:52:39.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:39.461+0000 7fe337400640 1 -- 172.21.15.77:0/1409326476 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe330031ca0 con 0x7fe340101de0 2024-09-10T09:52:39.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:39.461+0000 7fe337400640 1 -- 172.21.15.77:0/1409326476 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe330035030 con 0x7fe340101de0 2024-09-10T09:52:39.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:39.461+0000 7fe345a00640 1 -- 172.21.15.77:0/1409326476 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3401a7070 con 0x7fe340101de0 2024-09-10T09:52:39.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:39.462+0000 7fe345a00640 1 -- 172.21.15.77:0/1409326476 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3401021c0 con 0x7fe340101de0 2024-09-10T09:52:39.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:39.465+0000 7fe337400640 1 -- 172.21.15.77:0/1409326476 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe330042070 con 0x7fe340101de0 2024-09-10T09:52:39.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:39.466+0000 7fe337400640 1 --2- 172.21.15.77:0/1409326476 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fe3240a2b80 0x7fe3240a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:39.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:39.466+0000 7fe337400640 1 -- 172.21.15.77:0/1409326476 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7fe3300efe80 con 0x7fe340101de0 2024-09-10T09:52:39.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:39.466+0000 7fe33e000640 1 --2- 172.21.15.77:0/1409326476 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fe3240a2b80 0x7fe3240a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:39.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:39.467+0000 7fe33e000640 1 --2- 172.21.15.77:0/1409326476 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fe3240a2b80 0x7fe3240a5040 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe340075f80 tx=0x7fe32c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:39.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:39.468+0000 7fe337400640 1 -- 172.21.15.77:0/1409326476 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe3300bc4f0 con 0x7fe340101de0 2024-09-10T09:52:39.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:39.645+0000 7fe345a00640 1 -- 172.21.15.77:0/1409326476 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe340075a20 con 0x7fe340101de0 2024-09-10T09:52:39.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:39.647+0000 7fe337400640 1 -- 172.21.15.77:0/1409326476 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24236 (secure 0 0 0) 0x7fe340075a20 con 0x7fe340101de0 2024-09-10T09:52:39.649 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:52:39.650 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":92,"fsid":"14a55226-6f59-11ef-bcea-c7b262605968","created":"2024-09-10T09:44:07.359542+0000","modified":"2024-09-10T09:52:35.384242+0000","last_up_change":"2024-09-10T09:51:26.731971+0000","last_in_change":"2024-09-10T09:51:03.857873+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:47:24.174660+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T09:52:18.766738+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T09:52:21.284745+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T09:52:29.846594+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8d46f634-114c-4926-afdc-f8fa37e40336","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":1509649079},{"type":"v1","addr":"172.21.15.77:6803","nonce":1509649079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":1509649079},{"type":"v1","addr":"172.21.15.77:6805","nonce":1509649079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":1509649079},{"type":"v1","addr":"172.21.15.77:6809","nonce":1509649079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":1509649079},{"type":"v1","addr":"172.21.15.77:6807","nonce":1509649079}]},"public_addr":"172.21.15.77:6803/1509649079","cluster_addr":"172.21.15.77:6805/1509649079","heartbeat_back_addr":"172.21.15.77:6809/1509649079","heartbeat_front_addr":"172.21.15.77:6807/1509649079","state":["exists","up"]},{"osd":1,"uuid":"b6570764-736a-49e4-877a-1965e5812d1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":4243920698},{"type":"v1","addr":"172.21.15.120:6801","nonce":4243920698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":4243920698},{"type":"v1","addr":"172.21.15.120:6803","nonce":4243920698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":4243920698},{"type":"v1","addr":"172.21.15.120:6807","nonce":4243920698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":4243920698},{"type":"v1","addr":"172.21.15.120:6805","nonce":4243920698}]},"public_addr":"172.21.15.120:6801/4243920698","cluster_addr":"172.21.15.120:6803/4243920698","heartbeat_back_addr":"172.21.15.120:6807/4243920698","heartbeat_front_addr":"172.21.15.120:6805/4243920698","state":["exists","up"]},{"osd":2,"uuid":"5e83fe13-d1ee-483e-a575-e8041ec836f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":3280466823},{"type":"v1","addr":"172.21.15.134:6801","nonce":3280466823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":3280466823},{"type":"v1","addr":"172.21.15.134:6803","nonce":3280466823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":3280466823},{"type":"v1","addr":"172.21.15.134:6807","nonce":3280466823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":3280466823},{"type":"v1","addr":"172.21.15.134:6805","nonce":3280466823}]},"public_addr":"172.21.15.134:6801/3280466823","cluster_addr":"172.21.15.134:6803/3280466823","heartbeat_back_addr":"172.21.15.134:6807/3280466823","heartbeat_front_addr":"172.21.15.134:6805/3280466823","state":["exists","up"]},{"osd":3,"uuid":"5affc6a7-1cae-48fe-b7db-befc2fcafecb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":1629416786},{"type":"v1","addr":"172.21.15.77:6811","nonce":1629416786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":1629416786},{"type":"v1","addr":"172.21.15.77:6813","nonce":1629416786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":1629416786},{"type":"v1","addr":"172.21.15.77:6817","nonce":1629416786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":1629416786},{"type":"v1","addr":"172.21.15.77:6815","nonce":1629416786}]},"public_addr":"172.21.15.77:6811/1629416786","cluster_addr":"172.21.15.77:6813/1629416786","heartbeat_back_addr":"172.21.15.77:6817/1629416786","heartbeat_front_addr":"172.21.15.77:6815/1629416786","state":["exists","up"]},{"osd":4,"uuid":"c826978f-40b0-43ae-adbb-5c93b32a68ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":3162658038},{"type":"v1","addr":"172.21.15.120:6809","nonce":3162658038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":3162658038},{"type":"v1","addr":"172.21.15.120:6811","nonce":3162658038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":3162658038},{"type":"v1","addr":"172.21.15.120:6815","nonce":3162658038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":3162658038},{"type":"v1","addr":"172.21.15.120:6813","nonce":3162658038}]},"public_addr":"172.21.15.120:6809/3162658038","cluster_addr":"172.21.15.120:6811/3162658038","heartbeat_back_addr":"172.21.15.120:6815/3162658038","heartbeat_front_addr":"172.21.15.120:6813/3162658038","state":["exists","up"]},{"osd":5,"uuid":"be51c7fb-2765-46a3-9861-c950f63de148","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":4081444621},{"type":"v1","addr":"172.21.15.134:6809","nonce":4081444621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":4081444621},{"type":"v1","addr":"172.21.15.134:6811","nonce":4081444621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":4081444621},{"type":"v1","addr":"172.21.15.134:6815","nonce":4081444621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":4081444621},{"type":"v1","addr":"172.21.15.134:6813","nonce":4081444621}]},"public_addr":"172.21.15.134:6809/4081444621","cluster_addr":"172.21.15.134:6811/4081444621","heartbeat_back_addr":"172.21.15.134:6815/4081444621","heartbeat_front_addr":"172.21.15.134:6813/4081444621","state":["exists","up"]},{"osd":6,"uuid":"4eb403ab-473b-41cc-803d-18c908f768a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":879565639},{"type":"v1","addr":"172.21.15.77:6819","nonce":879565639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":879565639},{"type":"v1","addr":"172.21.15.77:6821","nonce":879565639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":879565639},{"type":"v1","addr":"172.21.15.77:6825","nonce":879565639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":879565639},{"type":"v1","addr":"172.21.15.77:6823","nonce":879565639}]},"public_addr":"172.21.15.77:6819/879565639","cluster_addr":"172.21.15.77:6821/879565639","heartbeat_back_addr":"172.21.15.77:6825/879565639","heartbeat_front_addr":"172.21.15.77:6823/879565639","state":["exists","up"]},{"osd":7,"uuid":"ba982bd0-c236-456c-9889-61087d79c1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":2688667400},{"type":"v1","addr":"172.21.15.120:6817","nonce":2688667400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":2688667400},{"type":"v1","addr":"172.21.15.120:6819","nonce":2688667400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":2688667400},{"type":"v1","addr":"172.21.15.120:6823","nonce":2688667400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2688667400},{"type":"v1","addr":"172.21.15.120:6821","nonce":2688667400}]},"public_addr":"172.21.15.120:6817/2688667400","cluster_addr":"172.21.15.120:6819/2688667400","heartbeat_back_addr":"172.21.15.120:6823/2688667400","heartbeat_front_addr":"172.21.15.120:6821/2688667400","state":["exists","up"]},{"osd":8,"uuid":"f2e8ca43-57c9-4933-b5f7-73f4d1b3e08b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2081686544},{"type":"v1","addr":"172.21.15.134:6817","nonce":2081686544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2081686544},{"type":"v1","addr":"172.21.15.134:6819","nonce":2081686544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2081686544},{"type":"v1","addr":"172.21.15.134:6823","nonce":2081686544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2081686544},{"type":"v1","addr":"172.21.15.134:6821","nonce":2081686544}]},"public_addr":"172.21.15.134:6817/2081686544","cluster_addr":"172.21.15.134:6819/2081686544","heartbeat_back_addr":"172.21.15.134:6823/2081686544","heartbeat_front_addr":"172.21.15.134:6821/2081686544","state":["exists","up"]},{"osd":9,"uuid":"9486f8b5-0614-4208-87ec-117741f6b622","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":1210619387},{"type":"v1","addr":"172.21.15.77:6827","nonce":1210619387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":1210619387},{"type":"v1","addr":"172.21.15.77:6829","nonce":1210619387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":1210619387},{"type":"v1","addr":"172.21.15.77:6833","nonce":1210619387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":1210619387},{"type":"v1","addr":"172.21.15.77:6831","nonce":1210619387}]},"public_addr":"172.21.15.77:6827/1210619387","cluster_addr":"172.21.15.77:6829/1210619387","heartbeat_back_addr":"172.21.15.77:6833/1210619387","heartbeat_front_addr":"172.21.15.77:6831/1210619387","state":["exists","up"]},{"osd":10,"uuid":"8e8192b0-1398-4d4d-9756-762b2f874d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":754733885},{"type":"v1","addr":"172.21.15.120:6825","nonce":754733885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":754733885},{"type":"v1","addr":"172.21.15.120:6827","nonce":754733885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":754733885},{"type":"v1","addr":"172.21.15.120:6831","nonce":754733885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":754733885},{"type":"v1","addr":"172.21.15.120:6829","nonce":754733885}]},"public_addr":"172.21.15.120:6825/754733885","cluster_addr":"172.21.15.120:6827/754733885","heartbeat_back_addr":"172.21.15.120:6831/754733885","heartbeat_front_addr":"172.21.15.120:6829/754733885","state":["exists","up"]},{"osd":11,"uuid":"85c1ccd4-5147-4995-896e-fe977f73f749","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1602213143},{"type":"v1","addr":"172.21.15.134:6825","nonce":1602213143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1602213143},{"type":"v1","addr":"172.21.15.134:6827","nonce":1602213143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1602213143},{"type":"v1","addr":"172.21.15.134:6831","nonce":1602213143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1602213143},{"type":"v1","addr":"172.21.15.134:6829","nonce":1602213143}]},"public_addr":"172.21.15.134:6825/1602213143","cluster_addr":"172.21.15.134:6827/1602213143","heartbeat_back_addr":"172.21.15.134:6831/1602213143","heartbeat_front_addr":"172.21.15.134:6829/1602213143","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:46:30.293451+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:46:56.554801+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:47:21.476225+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:47:49.737332+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:48:16.092181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:48:41.240815+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:49:11.683950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:49:38.859255+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:50:04.105401+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:50:33.847810+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:50:59.901797+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:51:25.241507+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.77:0/1697982655":"2024-09-11T09:44:59.575663+0000","172.21.15.77:6801/1527070442":"2024-09-11T09:44:59.575663+0000","172.21.15.77:0/3146688196":"2024-09-11T09:44:32.185342+0000","172.21.15.77:0/371773177":"2024-09-11T09:44:59.575663+0000","172.21.15.77:0/1925456910":"2024-09-11T09:44:59.575663+0000","172.21.15.77:0/138545832":"2024-09-11T09:44:32.185342+0000","172.21.15.77:6801/3235900503":"2024-09-11T09:44:32.185342+0000","172.21.15.77:6800/1527070442":"2024-09-11T09:44:59.575663+0000","172.21.15.77:6800/3235900503":"2024-09-11T09:44:32.185342+0000","172.21.15.77:0/1647092548":"2024-09-11T09:44:32.185342+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T09:52:39.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:39.650+0000 7fe345a00640 1 -- 172.21.15.77:0/1409326476 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fe3240a2b80 msgr2=0x7fe3240a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:39.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:39.650+0000 7fe345a00640 1 --2- 172.21.15.77:0/1409326476 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fe3240a2b80 0x7fe3240a5040 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe340075f80 tx=0x7fe32c00b390 comp rx=0 tx=0).stop 2024-09-10T09:52:39.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:39.650+0000 7fe345a00640 1 -- 172.21.15.77:0/1409326476 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe340101de0 msgr2=0x7fe3400744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:39.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:39.650+0000 7fe345a00640 1 --2- 172.21.15.77:0/1409326476 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe340101de0 0x7fe3400744f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe330004770 tx=0x7fe330003c10 comp rx=0 tx=0).stop 2024-09-10T09:52:39.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:39.650+0000 7fe345a00640 1 -- 172.21.15.77:0/1409326476 shutdown_connections 2024-09-10T09:52:39.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:39.650+0000 7fe345a00640 1 --2- 172.21.15.77:0/1409326476 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fe3240a2b80 0x7fe3240a5040 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:39.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:39.650+0000 7fe345a00640 1 --2- 172.21.15.77:0/1409326476 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe340106f10 0x7fe340074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:39.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:39.650+0000 7fe345a00640 1 --2- 172.21.15.77:0/1409326476 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe340102790 0x7fe340074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:39.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:39.650+0000 7fe345a00640 1 --2- 172.21.15.77:0/1409326476 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe340101de0 0x7fe3400744f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:39.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:39.650+0000 7fe345a00640 1 -- 172.21.15.77:0/1409326476 >> 172.21.15.77:0/1409326476 conn(0x7fe3400fd8c0 msgr2=0x7fe3400febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:39.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:39.651+0000 7fe345a00640 1 -- 172.21.15.77:0/1409326476 shutdown_connections 2024-09-10T09:52:39.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:39.651+0000 7fe345a00640 1 -- 172.21.15.77:0/1409326476 wait complete. 2024-09-10T09:52:39.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:39 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/1409326476' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T09:52:40.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:39 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/1409326476' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T09:52:40.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:39 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/1409326476' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T09:52:40.410 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T09:52:40.778 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:52:41.032 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:40 smithi134 ceph-mon[35674]: pgmap v312: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:52:41.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:40 smithi077 ceph-mon[30319]: pgmap v312: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:52:41.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:40 smithi120 ceph-mon[34770]: pgmap v312: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:52:41.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:41.341+0000 7f71a9400640 1 -- 172.21.15.77:0/4119730243 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f71a4104010 msgr2=0x7f71a41043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:41.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:41.341+0000 7f71a9400640 1 --2- 172.21.15.77:0/4119730243 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f71a4104010 0x7f71a41043f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7194009900 tx=0x7f719402a3f0 comp rx=0 tx=0).stop 2024-09-10T09:52:41.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:41.341+0000 7f71a9400640 1 -- 172.21.15.77:0/4119730243 shutdown_connections 2024-09-10T09:52:41.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:41.341+0000 7f71a9400640 1 --2- 172.21.15.77:0/4119730243 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f71a4109140 0x7f71a4110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:41.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:41.341+0000 7f71a9400640 1 --2- 172.21.15.77:0/4119730243 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71a41049c0 0x7f71a4108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:41.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:41.341+0000 7f71a9400640 1 --2- 172.21.15.77:0/4119730243 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f71a4104010 0x7f71a41043f0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:41.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:41.341+0000 7f71a9400640 1 -- 172.21.15.77:0/4119730243 >> 172.21.15.77:0/4119730243 conn(0x7f71a40ffa50 msgr2=0x7f71a4101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:41.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:41.341+0000 7f71a9400640 1 -- 172.21.15.77:0/4119730243 shutdown_connections 2024-09-10T09:52:41.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:41.341+0000 7f71a9400640 1 -- 172.21.15.77:0/4119730243 wait complete. 2024-09-10T09:52:41.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:41.342+0000 7f71a9400640 1 Processor -- start 2024-09-10T09:52:41.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:41.342+0000 7f71a9400640 1 -- start start 2024-09-10T09:52:41.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:41.343+0000 7f71a9400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f71a4104010 0x7f71a419a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:41.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:41.343+0000 7f71a9400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f71a41049c0 0x7f71a419ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:41.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:41.343+0000 7f71a9400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71a4109140 0x7f71a419ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:41.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:41.343+0000 7f71a9400640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f71a41141b0 con 0x7f71a41049c0 2024-09-10T09:52:41.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:41.343+0000 7f71a9400640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f71a4114030 con 0x7f71a4104010 2024-09-10T09:52:41.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:41.343+0000 7f71a9400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f71a4114330 con 0x7f71a4109140 2024-09-10T09:52:41.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:41.343+0000 7f71a1e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f71a41049c0 0x7f71a419ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:41.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:41.343+0000 7f71a2800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f71a4104010 0x7f71a419a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:41.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:41.343+0000 7f71a2800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f71a4104010 0x7f71a419a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:54880/0 (socket says 172.21.15.77:54880) 2024-09-10T09:52:41.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:41.343+0000 7f71a2800640 1 -- 172.21.15.77:0/2543343688 learned_addr learned my addr 172.21.15.77:0/2543343688 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:52:41.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:41.343+0000 7f71a3200640 1 --2- 172.21.15.77:0/2543343688 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71a4109140 0x7f71a419ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:41.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:41.344+0000 7f71a2800640 1 -- 172.21.15.77:0/2543343688 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71a4109140 msgr2=0x7f71a419ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:41.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:41.344+0000 7f71a2800640 1 --2- 172.21.15.77:0/2543343688 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71a4109140 0x7f71a419ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:41.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:41.344+0000 7f71a2800640 1 -- 172.21.15.77:0/2543343688 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f71a41049c0 msgr2=0x7f71a419ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:41.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:41.344+0000 7f71a2800640 1 --2- 172.21.15.77:0/2543343688 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f71a41049c0 0x7f71a419ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:41.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:41.344+0000 7f71a2800640 1 -- 172.21.15.77:0/2543343688 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71a419f5e0 con 0x7f71a4104010 2024-09-10T09:52:41.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:41.344+0000 7f71a3200640 1 --2- 172.21.15.77:0/2543343688 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71a4109140 0x7f71a419ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:52:41.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:41.344+0000 7f71a2800640 1 --2- 172.21.15.77:0/2543343688 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f71a4104010 0x7f71a419a630 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7194004770 tx=0x7f7194003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:41.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:41.344+0000 7f719b400640 1 -- 172.21.15.77:0/2543343688 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f719403f070 con 0x7f71a4104010 2024-09-10T09:52:41.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:41.344+0000 7f719b400640 1 -- 172.21.15.77:0/2543343688 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7194031ca0 con 0x7f71a4104010 2024-09-10T09:52:41.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:41.344+0000 7f71a9400640 1 -- 172.21.15.77:0/2543343688 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71a419f870 con 0x7f71a4104010 2024-09-10T09:52:41.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:41.344+0000 7f719b400640 1 -- 172.21.15.77:0/2543343688 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7194035030 con 0x7f71a4104010 2024-09-10T09:52:41.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:41.344+0000 7f71a9400640 1 -- 172.21.15.77:0/2543343688 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71a41a71b0 con 0x7f71a4104010 2024-09-10T09:52:41.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:41.346+0000 7f71a9400640 1 -- 172.21.15.77:0/2543343688 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71a41054f0 con 0x7f71a4104010 2024-09-10T09:52:41.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:41.347+0000 7f719b400640 1 -- 172.21.15.77:0/2543343688 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f7194042070 con 0x7f71a4104010 2024-09-10T09:52:41.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:41.349+0000 7f719b400640 1 --2- 172.21.15.77:0/2543343688 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f71880a2b80 0x7f71880a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:41.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:41.349+0000 7f719b400640 1 -- 172.21.15.77:0/2543343688 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7f71940efba0 con 0x7f71a4104010 2024-09-10T09:52:41.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:41.349+0000 7f71a1e00640 1 --2- 172.21.15.77:0/2543343688 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f71880a2b80 0x7f71880a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:41.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:41.350+0000 7f71a1e00640 1 --2- 172.21.15.77:0/2543343688 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f71880a2b80 0x7f71880a5040 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f71a40ffe70 tx=0x7f7190005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:41.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:41.351+0000 7f719b400640 1 -- 172.21.15.77:0/2543343688 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f71940bc860 con 0x7f71a4104010 2024-09-10T09:52:41.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:41.540+0000 7f71a9400640 1 -- 172.21.15.77:0/2543343688 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f71a410c320 con 0x7f71a4104010 2024-09-10T09:52:41.543 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:41.542+0000 7f719b400640 1 -- 172.21.15.77:0/2543343688 <== mon.2 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24236 (secure 0 0 0) 0x7f71940bc160 con 0x7f71a4104010 2024-09-10T09:52:41.544 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:52:41.544 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":92,"fsid":"14a55226-6f59-11ef-bcea-c7b262605968","created":"2024-09-10T09:44:07.359542+0000","modified":"2024-09-10T09:52:35.384242+0000","last_up_change":"2024-09-10T09:51:26.731971+0000","last_in_change":"2024-09-10T09:51:03.857873+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:47:24.174660+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T09:52:18.766738+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T09:52:21.284745+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T09:52:29.846594+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8d46f634-114c-4926-afdc-f8fa37e40336","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":1509649079},{"type":"v1","addr":"172.21.15.77:6803","nonce":1509649079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":1509649079},{"type":"v1","addr":"172.21.15.77:6805","nonce":1509649079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":1509649079},{"type":"v1","addr":"172.21.15.77:6809","nonce":1509649079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":1509649079},{"type":"v1","addr":"172.21.15.77:6807","nonce":1509649079}]},"public_addr":"172.21.15.77:6803/1509649079","cluster_addr":"172.21.15.77:6805/1509649079","heartbeat_back_addr":"172.21.15.77:6809/1509649079","heartbeat_front_addr":"172.21.15.77:6807/1509649079","state":["exists","up"]},{"osd":1,"uuid":"b6570764-736a-49e4-877a-1965e5812d1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":4243920698},{"type":"v1","addr":"172.21.15.120:6801","nonce":4243920698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":4243920698},{"type":"v1","addr":"172.21.15.120:6803","nonce":4243920698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":4243920698},{"type":"v1","addr":"172.21.15.120:6807","nonce":4243920698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":4243920698},{"type":"v1","addr":"172.21.15.120:6805","nonce":4243920698}]},"public_addr":"172.21.15.120:6801/4243920698","cluster_addr":"172.21.15.120:6803/4243920698","heartbeat_back_addr":"172.21.15.120:6807/4243920698","heartbeat_front_addr":"172.21.15.120:6805/4243920698","state":["exists","up"]},{"osd":2,"uuid":"5e83fe13-d1ee-483e-a575-e8041ec836f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":3280466823},{"type":"v1","addr":"172.21.15.134:6801","nonce":3280466823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":3280466823},{"type":"v1","addr":"172.21.15.134:6803","nonce":3280466823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":3280466823},{"type":"v1","addr":"172.21.15.134:6807","nonce":3280466823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":3280466823},{"type":"v1","addr":"172.21.15.134:6805","nonce":3280466823}]},"public_addr":"172.21.15.134:6801/3280466823","cluster_addr":"172.21.15.134:6803/3280466823","heartbeat_back_addr":"172.21.15.134:6807/3280466823","heartbeat_front_addr":"172.21.15.134:6805/3280466823","state":["exists","up"]},{"osd":3,"uuid":"5affc6a7-1cae-48fe-b7db-befc2fcafecb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":1629416786},{"type":"v1","addr":"172.21.15.77:6811","nonce":1629416786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":1629416786},{"type":"v1","addr":"172.21.15.77:6813","nonce":1629416786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":1629416786},{"type":"v1","addr":"172.21.15.77:6817","nonce":1629416786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":1629416786},{"type":"v1","addr":"172.21.15.77:6815","nonce":1629416786}]},"public_addr":"172.21.15.77:6811/1629416786","cluster_addr":"172.21.15.77:6813/1629416786","heartbeat_back_addr":"172.21.15.77:6817/1629416786","heartbeat_front_addr":"172.21.15.77:6815/1629416786","state":["exists","up"]},{"osd":4,"uuid":"c826978f-40b0-43ae-adbb-5c93b32a68ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":3162658038},{"type":"v1","addr":"172.21.15.120:6809","nonce":3162658038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":3162658038},{"type":"v1","addr":"172.21.15.120:6811","nonce":3162658038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":3162658038},{"type":"v1","addr":"172.21.15.120:6815","nonce":3162658038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":3162658038},{"type":"v1","addr":"172.21.15.120:6813","nonce":3162658038}]},"public_addr":"172.21.15.120:6809/3162658038","cluster_addr":"172.21.15.120:6811/3162658038","heartbeat_back_addr":"172.21.15.120:6815/3162658038","heartbeat_front_addr":"172.21.15.120:6813/3162658038","state":["exists","up"]},{"osd":5,"uuid":"be51c7fb-2765-46a3-9861-c950f63de148","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":4081444621},{"type":"v1","addr":"172.21.15.134:6809","nonce":4081444621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":4081444621},{"type":"v1","addr":"172.21.15.134:6811","nonce":4081444621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":4081444621},{"type":"v1","addr":"172.21.15.134:6815","nonce":4081444621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":4081444621},{"type":"v1","addr":"172.21.15.134:6813","nonce":4081444621}]},"public_addr":"172.21.15.134:6809/4081444621","cluster_addr":"172.21.15.134:6811/4081444621","heartbeat_back_addr":"172.21.15.134:6815/4081444621","heartbeat_front_addr":"172.21.15.134:6813/4081444621","state":["exists","up"]},{"osd":6,"uuid":"4eb403ab-473b-41cc-803d-18c908f768a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":879565639},{"type":"v1","addr":"172.21.15.77:6819","nonce":879565639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":879565639},{"type":"v1","addr":"172.21.15.77:6821","nonce":879565639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":879565639},{"type":"v1","addr":"172.21.15.77:6825","nonce":879565639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":879565639},{"type":"v1","addr":"172.21.15.77:6823","nonce":879565639}]},"public_addr":"172.21.15.77:6819/879565639","cluster_addr":"172.21.15.77:6821/879565639","heartbeat_back_addr":"172.21.15.77:6825/879565639","heartbeat_front_addr":"172.21.15.77:6823/879565639","state":["exists","up"]},{"osd":7,"uuid":"ba982bd0-c236-456c-9889-61087d79c1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":2688667400},{"type":"v1","addr":"172.21.15.120:6817","nonce":2688667400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":2688667400},{"type":"v1","addr":"172.21.15.120:6819","nonce":2688667400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":2688667400},{"type":"v1","addr":"172.21.15.120:6823","nonce":2688667400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2688667400},{"type":"v1","addr":"172.21.15.120:6821","nonce":2688667400}]},"public_addr":"172.21.15.120:6817/2688667400","cluster_addr":"172.21.15.120:6819/2688667400","heartbeat_back_addr":"172.21.15.120:6823/2688667400","heartbeat_front_addr":"172.21.15.120:6821/2688667400","state":["exists","up"]},{"osd":8,"uuid":"f2e8ca43-57c9-4933-b5f7-73f4d1b3e08b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2081686544},{"type":"v1","addr":"172.21.15.134:6817","nonce":2081686544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2081686544},{"type":"v1","addr":"172.21.15.134:6819","nonce":2081686544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2081686544},{"type":"v1","addr":"172.21.15.134:6823","nonce":2081686544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2081686544},{"type":"v1","addr":"172.21.15.134:6821","nonce":2081686544}]},"public_addr":"172.21.15.134:6817/2081686544","cluster_addr":"172.21.15.134:6819/2081686544","heartbeat_back_addr":"172.21.15.134:6823/2081686544","heartbeat_front_addr":"172.21.15.134:6821/2081686544","state":["exists","up"]},{"osd":9,"uuid":"9486f8b5-0614-4208-87ec-117741f6b622","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":1210619387},{"type":"v1","addr":"172.21.15.77:6827","nonce":1210619387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":1210619387},{"type":"v1","addr":"172.21.15.77:6829","nonce":1210619387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":1210619387},{"type":"v1","addr":"172.21.15.77:6833","nonce":1210619387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":1210619387},{"type":"v1","addr":"172.21.15.77:6831","nonce":1210619387}]},"public_addr":"172.21.15.77:6827/1210619387","cluster_addr":"172.21.15.77:6829/1210619387","heartbeat_back_addr":"172.21.15.77:6833/1210619387","heartbeat_front_addr":"172.21.15.77:6831/1210619387","state":["exists","up"]},{"osd":10,"uuid":"8e8192b0-1398-4d4d-9756-762b2f874d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":754733885},{"type":"v1","addr":"172.21.15.120:6825","nonce":754733885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":754733885},{"type":"v1","addr":"172.21.15.120:6827","nonce":754733885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":754733885},{"type":"v1","addr":"172.21.15.120:6831","nonce":754733885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":754733885},{"type":"v1","addr":"172.21.15.120:6829","nonce":754733885}]},"public_addr":"172.21.15.120:6825/754733885","cluster_addr":"172.21.15.120:6827/754733885","heartbeat_back_addr":"172.21.15.120:6831/754733885","heartbeat_front_addr":"172.21.15.120:6829/754733885","state":["exists","up"]},{"osd":11,"uuid":"85c1ccd4-5147-4995-896e-fe977f73f749","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1602213143},{"type":"v1","addr":"172.21.15.134:6825","nonce":1602213143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1602213143},{"type":"v1","addr":"172.21.15.134:6827","nonce":1602213143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1602213143},{"type":"v1","addr":"172.21.15.134:6831","nonce":1602213143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1602213143},{"type":"v1","addr":"172.21.15.134:6829","nonce":1602213143}]},"public_addr":"172.21.15.134:6825/1602213143","cluster_addr":"172.21.15.134:6827/1602213143","heartbeat_back_addr":"172.21.15.134:6831/1602213143","heartbeat_front_addr":"172.21.15.134:6829/1602213143","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:46:30.293451+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:46:56.554801+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:47:21.476225+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:47:49.737332+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:48:16.092181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:48:41.240815+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:49:11.683950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:49:38.859255+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:50:04.105401+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:50:33.847810+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:50:59.901797+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:51:25.241507+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.77:0/1697982655":"2024-09-11T09:44:59.575663+0000","172.21.15.77:6801/1527070442":"2024-09-11T09:44:59.575663+0000","172.21.15.77:0/3146688196":"2024-09-11T09:44:32.185342+0000","172.21.15.77:0/371773177":"2024-09-11T09:44:59.575663+0000","172.21.15.77:0/1925456910":"2024-09-11T09:44:59.575663+0000","172.21.15.77:0/138545832":"2024-09-11T09:44:32.185342+0000","172.21.15.77:6801/3235900503":"2024-09-11T09:44:32.185342+0000","172.21.15.77:6800/1527070442":"2024-09-11T09:44:59.575663+0000","172.21.15.77:6800/3235900503":"2024-09-11T09:44:32.185342+0000","172.21.15.77:0/1647092548":"2024-09-11T09:44:32.185342+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T09:52:41.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:41.544+0000 7f71a9400640 1 -- 172.21.15.77:0/2543343688 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f71880a2b80 msgr2=0x7f71880a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:41.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:41.544+0000 7f71a9400640 1 --2- 172.21.15.77:0/2543343688 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f71880a2b80 0x7f71880a5040 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f71a40ffe70 tx=0x7f7190005f90 comp rx=0 tx=0).stop 2024-09-10T09:52:41.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:41.544+0000 7f71a9400640 1 -- 172.21.15.77:0/2543343688 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f71a4104010 msgr2=0x7f71a419a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:41.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:41.544+0000 7f71a9400640 1 --2- 172.21.15.77:0/2543343688 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f71a4104010 0x7f71a419a630 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7194004770 tx=0x7f7194003c10 comp rx=0 tx=0).stop 2024-09-10T09:52:41.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:41.544+0000 7f71a9400640 1 -- 172.21.15.77:0/2543343688 shutdown_connections 2024-09-10T09:52:41.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:41.544+0000 7f71a9400640 1 --2- 172.21.15.77:0/2543343688 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f71880a2b80 0x7f71880a5040 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:41.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:41.544+0000 7f71a9400640 1 --2- 172.21.15.77:0/2543343688 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71a4109140 0x7f71a419ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:41.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:41.544+0000 7f71a9400640 1 --2- 172.21.15.77:0/2543343688 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f71a41049c0 0x7f71a419ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:41.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:41.544+0000 7f71a9400640 1 --2- 172.21.15.77:0/2543343688 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f71a4104010 0x7f71a419a630 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:41.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:41.544+0000 7f71a9400640 1 -- 172.21.15.77:0/2543343688 >> 172.21.15.77:0/2543343688 conn(0x7f71a40ffa50 msgr2=0x7f71a4100dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:41.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:41.545+0000 7f71a9400640 1 -- 172.21.15.77:0/2543343688 shutdown_connections 2024-09-10T09:52:41.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:41.545+0000 7f71a9400640 1 -- 172.21.15.77:0/2543343688 wait complete. 2024-09-10T09:52:42.049 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:41 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/2543343688' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T09:52:42.074 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:41 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/2543343688' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T09:52:42.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:41 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/2543343688' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T09:52:42.393 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:52:42.760 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:52:43.093 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:42 smithi077 ceph-mon[30319]: pgmap v313: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:52:43.130 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:42 smithi134 ceph-mon[35674]: pgmap v313: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:52:43.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:42 smithi120 ceph-mon[34770]: pgmap v313: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:52:43.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:43.349+0000 7f5775600640 1 -- 172.21.15.77:0/2006270699 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5770101de0 msgr2=0x7f57701021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:43.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:43.349+0000 7f5775600640 1 --2- 172.21.15.77:0/2006270699 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5770101de0 0x7f57701021c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5760009900 tx=0x7f576002a400 comp rx=0 tx=0).stop 2024-09-10T09:52:43.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:43.349+0000 7f5775600640 1 -- 172.21.15.77:0/2006270699 shutdown_connections 2024-09-10T09:52:43.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:43.349+0000 7f5775600640 1 --2- 172.21.15.77:0/2006270699 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5770106f10 0x7f577010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:43.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:43.349+0000 7f5775600640 1 --2- 172.21.15.77:0/2006270699 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5770102790 0x7f57701067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:43.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:43.349+0000 7f5775600640 1 --2- 172.21.15.77:0/2006270699 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5770101de0 0x7f57701021c0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:43.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:43.349+0000 7f5775600640 1 -- 172.21.15.77:0/2006270699 >> 172.21.15.77:0/2006270699 conn(0x7f57700fd8c0 msgr2=0x7f57700ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:43.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:43.349+0000 7f5775600640 1 -- 172.21.15.77:0/2006270699 shutdown_connections 2024-09-10T09:52:43.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:43.349+0000 7f5775600640 1 -- 172.21.15.77:0/2006270699 wait complete. 2024-09-10T09:52:43.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:43.350+0000 7f5775600640 1 Processor -- start 2024-09-10T09:52:43.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:43.350+0000 7f5775600640 1 -- start start 2024-09-10T09:52:43.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:43.351+0000 7f5775600640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5770101de0 0x7f577019a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:43.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:43.351+0000 7f5775600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5770102790 0x7f577019abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:43.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:43.351+0000 7f5775600640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5770106f10 0x7f577019ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:43.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:43.351+0000 7f5775600640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5770111fc0 con 0x7f5770106f10 2024-09-10T09:52:43.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:43.351+0000 7f5775600640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5770111e40 con 0x7f5770101de0 2024-09-10T09:52:43.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:43.351+0000 7f5775600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5770112140 con 0x7f5770102790 2024-09-10T09:52:43.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:43.351+0000 7f576e000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5770102790 0x7f577019abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:43.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:43.351+0000 7f576e000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5770102790 0x7f577019abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:52916/0 (socket says 172.21.15.77:52916) 2024-09-10T09:52:43.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:43.351+0000 7f576e000640 1 -- 172.21.15.77:0/751350883 learned_addr learned my addr 172.21.15.77:0/751350883 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:52:43.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:43.351+0000 7f576f400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5770106f10 0x7f577019ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:43.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:43.351+0000 7f576ea00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5770101de0 0x7f577019a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:43.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:43.351+0000 7f576e000640 1 -- 172.21.15.77:0/751350883 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5770101de0 msgr2=0x7f577019a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:43.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:43.351+0000 7f576e000640 1 --2- 172.21.15.77:0/751350883 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5770101de0 0x7f577019a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:43.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:43.351+0000 7f576e000640 1 -- 172.21.15.77:0/751350883 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5770106f10 msgr2=0x7f577019ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:43.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:43.351+0000 7f576e000640 1 --2- 172.21.15.77:0/751350883 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5770106f10 0x7f577019ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:43.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:43.351+0000 7f576e000640 1 -- 172.21.15.77:0/751350883 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f577019f630 con 0x7f5770102790 2024-09-10T09:52:43.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:43.352+0000 7f576e000640 1 --2- 172.21.15.77:0/751350883 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5770102790 0x7f577019abc0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f575c00e9f0 tx=0x7f575c00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:43.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:43.352+0000 7f5767400640 1 -- 172.21.15.77:0/751350883 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f575c00ccc0 con 0x7f5770102790 2024-09-10T09:52:43.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:43.352+0000 7f5767400640 1 -- 172.21.15.77:0/751350883 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f575c00ce60 con 0x7f5770102790 2024-09-10T09:52:43.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:43.352+0000 7f5775600640 1 -- 172.21.15.77:0/751350883 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f577019f920 con 0x7f5770102790 2024-09-10T09:52:43.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:43.352+0000 7f5767400640 1 -- 172.21.15.77:0/751350883 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f575c010690 con 0x7f5770102790 2024-09-10T09:52:43.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:43.352+0000 7f5775600640 1 -- 172.21.15.77:0/751350883 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57701a7200 con 0x7f5770102790 2024-09-10T09:52:43.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:43.353+0000 7f5775600640 1 -- 172.21.15.77:0/751350883 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57701021c0 con 0x7f5770102790 2024-09-10T09:52:43.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:43.355+0000 7f5767400640 1 -- 172.21.15.77:0/751350883 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f575c0046c0 con 0x7f5770102790 2024-09-10T09:52:43.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:43.356+0000 7f5767400640 1 --2- 172.21.15.77:0/751350883 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f57540a2c00 0x7f57540a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:43.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:43.356+0000 7f5767400640 1 -- 172.21.15.77:0/751350883 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7f575c0cf640 con 0x7f5770102790 2024-09-10T09:52:43.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:43.356+0000 7f576ea00640 1 --2- 172.21.15.77:0/751350883 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f57540a2c00 0x7f57540a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:43.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:43.357+0000 7f576ea00640 1 --2- 172.21.15.77:0/751350883 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f57540a2c00 0x7f57540a50c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5760000c00 tx=0x7f5760005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:43.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:43.360+0000 7f5767400640 1 -- 172.21.15.77:0/751350883 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f575c09c300 con 0x7f5770102790 2024-09-10T09:52:43.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:43.566+0000 7f5775600640 1 -- 172.21.15.77:0/751350883 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f57701a7530 con 0x7f5770102790 2024-09-10T09:52:43.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:43.567+0000 7f5767400640 1 -- 172.21.15.77:0/751350883 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11854 (secure 0 0 0) 0x7f575c09bc00 con 0x7f5770102790 2024-09-10T09:52:43.570 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:52:43.570 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":17,"btime":"2024-09-10T09:52:35:434708+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3159289675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3159289675},{"type":"v1","addr":"172.21.15.77:6835","nonce":3159289675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1951962341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1951962341},{"type":"v1","addr":"172.21.15.77:6837","nonce":1951962341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14487,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6839/4291121888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":4291121888},{"type":"v1","addr":"172.21.15.77:6839","nonce":4291121888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14499,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6841/445284194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":445284194},{"type":"v1","addr":"172.21.15.77:6841","nonce":445284194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/1005054423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1005054423},{"type":"v1","addr":"172.21.15.134:6833","nonce":1005054423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/3716307744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":3716307744},{"type":"v1","addr":"172.21.15.120:6833","nonce":3716307744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/1319892544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1319892544},{"type":"v1","addr":"172.21.15.120:6835","nonce":1319892544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24496,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/3293933699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3293933699},{"type":"v1","addr":"172.21.15.134:6835","nonce":3293933699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24502,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/2461497091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2461497091},{"type":"v1","addr":"172.21.15.134:6837","nonce":2461497091}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24520,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6839/1965117680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1965117680},{"type":"v1","addr":"172.21.15.134:6839","nonce":1965117680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/3033280709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3033280709},{"type":"v1","addr":"172.21.15.120:6837","nonce":3033280709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T09:52:23.368252+0000","modified":"2024-09-10T09:52:35.384462+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24563},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.120:6839/1777710157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1777710157},{"type":"v1","addr":"172.21.15.120:6839","nonce":1777710157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24563,"qdb_cluster":[24563]},"id":1}]} 2024-09-10T09:52:43.570 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 17 2024-09-10T09:52:43.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:43.570+0000 7f5775600640 1 -- 172.21.15.77:0/751350883 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f57540a2c00 msgr2=0x7f57540a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:43.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:43.570+0000 7f5775600640 1 --2- 172.21.15.77:0/751350883 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f57540a2c00 0x7f57540a50c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5760000c00 tx=0x7f5760005b50 comp rx=0 tx=0).stop 2024-09-10T09:52:43.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:43.570+0000 7f5775600640 1 -- 172.21.15.77:0/751350883 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5770102790 msgr2=0x7f577019abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:43.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:43.570+0000 7f5775600640 1 --2- 172.21.15.77:0/751350883 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5770102790 0x7f577019abc0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f575c00e9f0 tx=0x7f575c00edf0 comp rx=0 tx=0).stop 2024-09-10T09:52:43.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:43.570+0000 7f5775600640 1 -- 172.21.15.77:0/751350883 shutdown_connections 2024-09-10T09:52:43.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:43.570+0000 7f5775600640 1 --2- 172.21.15.77:0/751350883 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f57540a2c00 0x7f57540a50c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:43.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:43.570+0000 7f5775600640 1 --2- 172.21.15.77:0/751350883 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5770106f10 0x7f577019ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:43.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:43.570+0000 7f5775600640 1 --2- 172.21.15.77:0/751350883 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5770102790 0x7f577019abc0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:43.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:43.570+0000 7f5775600640 1 --2- 172.21.15.77:0/751350883 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5770101de0 0x7f577019a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:43.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:43.570+0000 7f5775600640 1 -- 172.21.15.77:0/751350883 >> 172.21.15.77:0/751350883 conn(0x7f57700fd8c0 msgr2=0x7f57700ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:43.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:43.570+0000 7f5775600640 1 -- 172.21.15.77:0/751350883 shutdown_connections 2024-09-10T09:52:43.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:43.570+0000 7f5775600640 1 -- 172.21.15.77:0/751350883 wait complete. 2024-09-10T09:52:43.957 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:43 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/751350883' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:52:44.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:43 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/751350883' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:52:44.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:43 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/751350883' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:52:44.438 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T09:52:44.806 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:52:45.100 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:44 smithi077 ceph-mon[30319]: pgmap v314: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:52:45.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:44 smithi134 ceph-mon[35674]: pgmap v314: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:52:45.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:44 smithi120 ceph-mon[34770]: pgmap v314: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:52:45.395 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:45.355+0000 7f0405800640 1 -- 172.21.15.77:0/3938201774 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0400074ee0 msgr2=0x7f040010dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:45.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:45.355+0000 7f0405800640 1 --2- 172.21.15.77:0/3938201774 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0400074ee0 0x7f040010dd20 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f03ec009860 tx=0x7f03ec02a3d0 comp rx=0 tx=0).stop 2024-09-10T09:52:45.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:45.355+0000 7f0405800640 1 -- 172.21.15.77:0/3938201774 shutdown_connections 2024-09-10T09:52:45.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:45.355+0000 7f0405800640 1 --2- 172.21.15.77:0/3938201774 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f040010e260 0x7f0400110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:45.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:45.355+0000 7f0405800640 1 --2- 172.21.15.77:0/3938201774 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0400074ee0 0x7f040010dd20 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:45.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:45.355+0000 7f0405800640 1 --2- 172.21.15.77:0/3938201774 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f04000745c0 0x7f04000749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:45.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:45.355+0000 7f0405800640 1 -- 172.21.15.77:0/3938201774 >> 172.21.15.77:0/3938201774 conn(0x7f04000fd740 msgr2=0x7f04000ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:45.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:45.355+0000 7f0405800640 1 -- 172.21.15.77:0/3938201774 shutdown_connections 2024-09-10T09:52:45.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:45.355+0000 7f0405800640 1 -- 172.21.15.77:0/3938201774 wait complete. 2024-09-10T09:52:45.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:45.356+0000 7f0405800640 1 Processor -- start 2024-09-10T09:52:45.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:45.356+0000 7f0405800640 1 -- start start 2024-09-10T09:52:45.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:45.357+0000 7f0405800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f04000745c0 0x7f040019a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:45.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:45.357+0000 7f0405800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0400074ee0 0x7f040019aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:45.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:45.357+0000 7f0405800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f040010e260 0x7f040019ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:45.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:45.357+0000 7f0405800640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0400114000 con 0x7f040010e260 2024-09-10T09:52:45.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:45.357+0000 7f0405800640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0400113e80 con 0x7f0400074ee0 2024-09-10T09:52:45.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:45.357+0000 7f0405800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0400114180 con 0x7f04000745c0 2024-09-10T09:52:45.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:45.357+0000 7f03ffe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f04000745c0 0x7f040019a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:45.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:45.357+0000 7f03ffe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f04000745c0 0x7f040019a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:52940/0 (socket says 172.21.15.77:52940) 2024-09-10T09:52:45.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:45.357+0000 7f03ffe00640 1 -- 172.21.15.77:0/574406470 learned_addr learned my addr 172.21.15.77:0/574406470 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:52:45.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:45.357+0000 7f0404e00640 1 --2- 172.21.15.77:0/574406470 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f040010e260 0x7f040019ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:45.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:45.357+0000 7f03ff400640 1 --2- 172.21.15.77:0/574406470 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0400074ee0 0x7f040019aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:45.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:45.357+0000 7f03ffe00640 1 -- 172.21.15.77:0/574406470 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0400074ee0 msgr2=0x7f040019aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:45.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:45.357+0000 7f03ffe00640 1 --2- 172.21.15.77:0/574406470 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0400074ee0 0x7f040019aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:45.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:45.357+0000 7f03ffe00640 1 -- 172.21.15.77:0/574406470 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f040010e260 msgr2=0x7f040019ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:45.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:45.357+0000 7f03ffe00640 1 --2- 172.21.15.77:0/574406470 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f040010e260 0x7f040019ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:45.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:45.357+0000 7f03ffe00640 1 -- 172.21.15.77:0/574406470 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f040019f4f0 con 0x7f04000745c0 2024-09-10T09:52:45.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:45.358+0000 7f03ffe00640 1 --2- 172.21.15.77:0/574406470 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f04000745c0 0x7f040019a540 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f03f000e9f0 tx=0x7f03f000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:45.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:45.358+0000 7f03fcc00640 1 -- 172.21.15.77:0/574406470 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f000cd80 con 0x7f04000745c0 2024-09-10T09:52:45.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:45.358+0000 7f03fcc00640 1 -- 172.21.15.77:0/574406470 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03f0005500 con 0x7f04000745c0 2024-09-10T09:52:45.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:45.358+0000 7f0405800640 1 -- 172.21.15.77:0/574406470 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f040019f7e0 con 0x7f04000745c0 2024-09-10T09:52:45.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:45.358+0000 7f03fcc00640 1 -- 172.21.15.77:0/574406470 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f0010690 con 0x7f04000745c0 2024-09-10T09:52:45.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:45.358+0000 7f0405800640 1 -- 172.21.15.77:0/574406470 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04001a70c0 con 0x7f04000745c0 2024-09-10T09:52:45.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:45.359+0000 7f0405800640 1 -- 172.21.15.77:0/574406470 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04000749a0 con 0x7f04000745c0 2024-09-10T09:52:45.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:45.363+0000 7f03fcc00640 1 -- 172.21.15.77:0/574406470 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f03f00046c0 con 0x7f04000745c0 2024-09-10T09:52:45.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:45.364+0000 7f03fcc00640 1 --2- 172.21.15.77:0/574406470 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f03e40a2b80 0x7f03e40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:45.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:45.364+0000 7f03fcc00640 1 -- 172.21.15.77:0/574406470 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7f03f00cf6c0 con 0x7f04000745c0 2024-09-10T09:52:45.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:45.364+0000 7f03ff400640 1 --2- 172.21.15.77:0/574406470 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f03e40a2b80 0x7f03e40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:45.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:45.365+0000 7f03ff400640 1 --2- 172.21.15.77:0/574406470 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f03e40a2b80 0x7f03e40a5040 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f040010f530 tx=0x7f03ec005c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:45.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:45.368+0000 7f03fcc00640 1 -- 172.21.15.77:0/574406470 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f03f009c380 con 0x7f04000745c0 2024-09-10T09:52:45.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:45.552+0000 7f0405800640 1 -- 172.21.15.77:0/574406470 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f04001093e0 con 0x7f04000745c0 2024-09-10T09:52:45.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:45.554+0000 7f03fcc00640 1 -- 172.21.15.77:0/574406470 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24236 (secure 0 0 0) 0x7f03f009bc80 con 0x7f04000745c0 2024-09-10T09:52:45.556 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:52:45.557 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":92,"fsid":"14a55226-6f59-11ef-bcea-c7b262605968","created":"2024-09-10T09:44:07.359542+0000","modified":"2024-09-10T09:52:35.384242+0000","last_up_change":"2024-09-10T09:51:26.731971+0000","last_in_change":"2024-09-10T09:51:03.857873+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:47:24.174660+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T09:52:18.766738+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T09:52:21.284745+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T09:52:29.846594+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8d46f634-114c-4926-afdc-f8fa37e40336","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":1509649079},{"type":"v1","addr":"172.21.15.77:6803","nonce":1509649079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":1509649079},{"type":"v1","addr":"172.21.15.77:6805","nonce":1509649079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":1509649079},{"type":"v1","addr":"172.21.15.77:6809","nonce":1509649079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":1509649079},{"type":"v1","addr":"172.21.15.77:6807","nonce":1509649079}]},"public_addr":"172.21.15.77:6803/1509649079","cluster_addr":"172.21.15.77:6805/1509649079","heartbeat_back_addr":"172.21.15.77:6809/1509649079","heartbeat_front_addr":"172.21.15.77:6807/1509649079","state":["exists","up"]},{"osd":1,"uuid":"b6570764-736a-49e4-877a-1965e5812d1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":4243920698},{"type":"v1","addr":"172.21.15.120:6801","nonce":4243920698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":4243920698},{"type":"v1","addr":"172.21.15.120:6803","nonce":4243920698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":4243920698},{"type":"v1","addr":"172.21.15.120:6807","nonce":4243920698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":4243920698},{"type":"v1","addr":"172.21.15.120:6805","nonce":4243920698}]},"public_addr":"172.21.15.120:6801/4243920698","cluster_addr":"172.21.15.120:6803/4243920698","heartbeat_back_addr":"172.21.15.120:6807/4243920698","heartbeat_front_addr":"172.21.15.120:6805/4243920698","state":["exists","up"]},{"osd":2,"uuid":"5e83fe13-d1ee-483e-a575-e8041ec836f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":3280466823},{"type":"v1","addr":"172.21.15.134:6801","nonce":3280466823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":3280466823},{"type":"v1","addr":"172.21.15.134:6803","nonce":3280466823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":3280466823},{"type":"v1","addr":"172.21.15.134:6807","nonce":3280466823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":3280466823},{"type":"v1","addr":"172.21.15.134:6805","nonce":3280466823}]},"public_addr":"172.21.15.134:6801/3280466823","cluster_addr":"172.21.15.134:6803/3280466823","heartbeat_back_addr":"172.21.15.134:6807/3280466823","heartbeat_front_addr":"172.21.15.134:6805/3280466823","state":["exists","up"]},{"osd":3,"uuid":"5affc6a7-1cae-48fe-b7db-befc2fcafecb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":1629416786},{"type":"v1","addr":"172.21.15.77:6811","nonce":1629416786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":1629416786},{"type":"v1","addr":"172.21.15.77:6813","nonce":1629416786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":1629416786},{"type":"v1","addr":"172.21.15.77:6817","nonce":1629416786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":1629416786},{"type":"v1","addr":"172.21.15.77:6815","nonce":1629416786}]},"public_addr":"172.21.15.77:6811/1629416786","cluster_addr":"172.21.15.77:6813/1629416786","heartbeat_back_addr":"172.21.15.77:6817/1629416786","heartbeat_front_addr":"172.21.15.77:6815/1629416786","state":["exists","up"]},{"osd":4,"uuid":"c826978f-40b0-43ae-adbb-5c93b32a68ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":3162658038},{"type":"v1","addr":"172.21.15.120:6809","nonce":3162658038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":3162658038},{"type":"v1","addr":"172.21.15.120:6811","nonce":3162658038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":3162658038},{"type":"v1","addr":"172.21.15.120:6815","nonce":3162658038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":3162658038},{"type":"v1","addr":"172.21.15.120:6813","nonce":3162658038}]},"public_addr":"172.21.15.120:6809/3162658038","cluster_addr":"172.21.15.120:6811/3162658038","heartbeat_back_addr":"172.21.15.120:6815/3162658038","heartbeat_front_addr":"172.21.15.120:6813/3162658038","state":["exists","up"]},{"osd":5,"uuid":"be51c7fb-2765-46a3-9861-c950f63de148","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":4081444621},{"type":"v1","addr":"172.21.15.134:6809","nonce":4081444621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":4081444621},{"type":"v1","addr":"172.21.15.134:6811","nonce":4081444621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":4081444621},{"type":"v1","addr":"172.21.15.134:6815","nonce":4081444621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":4081444621},{"type":"v1","addr":"172.21.15.134:6813","nonce":4081444621}]},"public_addr":"172.21.15.134:6809/4081444621","cluster_addr":"172.21.15.134:6811/4081444621","heartbeat_back_addr":"172.21.15.134:6815/4081444621","heartbeat_front_addr":"172.21.15.134:6813/4081444621","state":["exists","up"]},{"osd":6,"uuid":"4eb403ab-473b-41cc-803d-18c908f768a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":879565639},{"type":"v1","addr":"172.21.15.77:6819","nonce":879565639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":879565639},{"type":"v1","addr":"172.21.15.77:6821","nonce":879565639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":879565639},{"type":"v1","addr":"172.21.15.77:6825","nonce":879565639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":879565639},{"type":"v1","addr":"172.21.15.77:6823","nonce":879565639}]},"public_addr":"172.21.15.77:6819/879565639","cluster_addr":"172.21.15.77:6821/879565639","heartbeat_back_addr":"172.21.15.77:6825/879565639","heartbeat_front_addr":"172.21.15.77:6823/879565639","state":["exists","up"]},{"osd":7,"uuid":"ba982bd0-c236-456c-9889-61087d79c1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":2688667400},{"type":"v1","addr":"172.21.15.120:6817","nonce":2688667400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":2688667400},{"type":"v1","addr":"172.21.15.120:6819","nonce":2688667400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":2688667400},{"type":"v1","addr":"172.21.15.120:6823","nonce":2688667400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2688667400},{"type":"v1","addr":"172.21.15.120:6821","nonce":2688667400}]},"public_addr":"172.21.15.120:6817/2688667400","cluster_addr":"172.21.15.120:6819/2688667400","heartbeat_back_addr":"172.21.15.120:6823/2688667400","heartbeat_front_addr":"172.21.15.120:6821/2688667400","state":["exists","up"]},{"osd":8,"uuid":"f2e8ca43-57c9-4933-b5f7-73f4d1b3e08b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2081686544},{"type":"v1","addr":"172.21.15.134:6817","nonce":2081686544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2081686544},{"type":"v1","addr":"172.21.15.134:6819","nonce":2081686544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2081686544},{"type":"v1","addr":"172.21.15.134:6823","nonce":2081686544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2081686544},{"type":"v1","addr":"172.21.15.134:6821","nonce":2081686544}]},"public_addr":"172.21.15.134:6817/2081686544","cluster_addr":"172.21.15.134:6819/2081686544","heartbeat_back_addr":"172.21.15.134:6823/2081686544","heartbeat_front_addr":"172.21.15.134:6821/2081686544","state":["exists","up"]},{"osd":9,"uuid":"9486f8b5-0614-4208-87ec-117741f6b622","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":1210619387},{"type":"v1","addr":"172.21.15.77:6827","nonce":1210619387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":1210619387},{"type":"v1","addr":"172.21.15.77:6829","nonce":1210619387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":1210619387},{"type":"v1","addr":"172.21.15.77:6833","nonce":1210619387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":1210619387},{"type":"v1","addr":"172.21.15.77:6831","nonce":1210619387}]},"public_addr":"172.21.15.77:6827/1210619387","cluster_addr":"172.21.15.77:6829/1210619387","heartbeat_back_addr":"172.21.15.77:6833/1210619387","heartbeat_front_addr":"172.21.15.77:6831/1210619387","state":["exists","up"]},{"osd":10,"uuid":"8e8192b0-1398-4d4d-9756-762b2f874d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":754733885},{"type":"v1","addr":"172.21.15.120:6825","nonce":754733885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":754733885},{"type":"v1","addr":"172.21.15.120:6827","nonce":754733885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":754733885},{"type":"v1","addr":"172.21.15.120:6831","nonce":754733885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":754733885},{"type":"v1","addr":"172.21.15.120:6829","nonce":754733885}]},"public_addr":"172.21.15.120:6825/754733885","cluster_addr":"172.21.15.120:6827/754733885","heartbeat_back_addr":"172.21.15.120:6831/754733885","heartbeat_front_addr":"172.21.15.120:6829/754733885","state":["exists","up"]},{"osd":11,"uuid":"85c1ccd4-5147-4995-896e-fe977f73f749","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1602213143},{"type":"v1","addr":"172.21.15.134:6825","nonce":1602213143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1602213143},{"type":"v1","addr":"172.21.15.134:6827","nonce":1602213143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1602213143},{"type":"v1","addr":"172.21.15.134:6831","nonce":1602213143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1602213143},{"type":"v1","addr":"172.21.15.134:6829","nonce":1602213143}]},"public_addr":"172.21.15.134:6825/1602213143","cluster_addr":"172.21.15.134:6827/1602213143","heartbeat_back_addr":"172.21.15.134:6831/1602213143","heartbeat_front_addr":"172.21.15.134:6829/1602213143","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:46:30.293451+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:46:56.554801+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:47:21.476225+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:47:49.737332+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:48:16.092181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:48:41.240815+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:49:11.683950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:49:38.859255+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:50:04.105401+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:50:33.847810+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:50:59.901797+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:51:25.241507+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.77:0/1697982655":"2024-09-11T09:44:59.575663+0000","172.21.15.77:6801/1527070442":"2024-09-11T09:44:59.575663+0000","172.21.15.77:0/3146688196":"2024-09-11T09:44:32.185342+0000","172.21.15.77:0/371773177":"2024-09-11T09:44:59.575663+0000","172.21.15.77:0/1925456910":"2024-09-11T09:44:59.575663+0000","172.21.15.77:0/138545832":"2024-09-11T09:44:32.185342+0000","172.21.15.77:6801/3235900503":"2024-09-11T09:44:32.185342+0000","172.21.15.77:6800/1527070442":"2024-09-11T09:44:59.575663+0000","172.21.15.77:6800/3235900503":"2024-09-11T09:44:32.185342+0000","172.21.15.77:0/1647092548":"2024-09-11T09:44:32.185342+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T09:52:45.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:45.557+0000 7f0405800640 1 -- 172.21.15.77:0/574406470 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f03e40a2b80 msgr2=0x7f03e40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:45.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:45.557+0000 7f0405800640 1 --2- 172.21.15.77:0/574406470 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f03e40a2b80 0x7f03e40a5040 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f040010f530 tx=0x7f03ec005c90 comp rx=0 tx=0).stop 2024-09-10T09:52:45.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:45.557+0000 7f0405800640 1 -- 172.21.15.77:0/574406470 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f04000745c0 msgr2=0x7f040019a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:45.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:45.557+0000 7f0405800640 1 --2- 172.21.15.77:0/574406470 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f04000745c0 0x7f040019a540 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f03f000e9f0 tx=0x7f03f000edf0 comp rx=0 tx=0).stop 2024-09-10T09:52:45.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:45.557+0000 7f0405800640 1 -- 172.21.15.77:0/574406470 shutdown_connections 2024-09-10T09:52:45.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:45.557+0000 7f0405800640 1 --2- 172.21.15.77:0/574406470 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f03e40a2b80 0x7f03e40a5040 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:45.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:45.557+0000 7f0405800640 1 --2- 172.21.15.77:0/574406470 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f040010e260 0x7f040019ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:45.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:45.557+0000 7f0405800640 1 --2- 172.21.15.77:0/574406470 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0400074ee0 0x7f040019aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:45.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:45.557+0000 7f0405800640 1 --2- 172.21.15.77:0/574406470 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f04000745c0 0x7f040019a540 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:45.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:45.557+0000 7f0405800640 1 -- 172.21.15.77:0/574406470 >> 172.21.15.77:0/574406470 conn(0x7f04000fd740 msgr2=0x7f04000fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:45.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:45.557+0000 7f0405800640 1 -- 172.21.15.77:0/574406470 shutdown_connections 2024-09-10T09:52:45.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:45.557+0000 7f0405800640 1 -- 172.21.15.77:0/574406470 wait complete. 2024-09-10T09:52:45.980 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:45 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/574406470' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T09:52:46.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:45 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/574406470' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T09:52:46.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:45 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/574406470' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T09:52:46.304 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T09:52:23.368252+0000', 'modified': '2024-09-10T09:52:35.384462+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24563}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24563': {'gid': 24563, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.120:6839/1777710157', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6838', 'nonce': 1777710157}, {'type': 'v1', 'addr': '172.21.15.120:6839', 'nonce': 1777710157}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24563, 'qdb_cluster': [24563]} 2024-09-10T09:52:46.304 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T09:52:46.305 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-10T09:52:46.692 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:52:46.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:46 smithi077 ceph-mon[30319]: pgmap v315: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:52:47.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:46 smithi134 ceph-mon[35674]: pgmap v315: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:52:47.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:46 smithi120 ceph-mon[34770]: pgmap v315: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:52:47.258 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.255+0000 7f21a5e00640 1 -- 172.21.15.77:0/2640942348 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f21a0106f10 msgr2=0x7f21a010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:47.258 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.255+0000 7f21a5e00640 1 --2- 172.21.15.77:0/2640942348 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f21a0106f10 0x7f21a010ea90 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2198009ca0 tx=0x7f219802a8c0 comp rx=0 tx=0).stop 2024-09-10T09:52:47.258 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.256+0000 7f21a5e00640 1 -- 172.21.15.77:0/2640942348 shutdown_connections 2024-09-10T09:52:47.258 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.256+0000 7f21a5e00640 1 --2- 172.21.15.77:0/2640942348 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f21a0106f10 0x7f21a010ea90 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:47.258 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.256+0000 7f21a5e00640 1 --2- 172.21.15.77:0/2640942348 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21a0102790 0x7f21a01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:47.258 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.256+0000 7f21a5e00640 1 --2- 172.21.15.77:0/2640942348 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f21a0101de0 0x7f21a01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:47.258 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.256+0000 7f21a5e00640 1 -- 172.21.15.77:0/2640942348 >> 172.21.15.77:0/2640942348 conn(0x7f21a00fd8c0 msgr2=0x7f21a00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:47.258 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.256+0000 7f21a5e00640 1 -- 172.21.15.77:0/2640942348 shutdown_connections 2024-09-10T09:52:47.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.256+0000 7f21a5e00640 1 -- 172.21.15.77:0/2640942348 wait complete. 2024-09-10T09:52:47.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.257+0000 7f21a5e00640 1 Processor -- start 2024-09-10T09:52:47.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.257+0000 7f21a5e00640 1 -- start start 2024-09-10T09:52:47.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.257+0000 7f21a5e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f21a0101de0 0x7f21a019a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:47.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.257+0000 7f21a5e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f21a0102790 0x7f21a019ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:47.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.257+0000 7f21a5e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21a0106f10 0x7f21a019eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:47.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.257+0000 7f21a5e00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f21a0111fc0 con 0x7f21a0101de0 2024-09-10T09:52:47.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.257+0000 7f21a5e00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f21a0111e40 con 0x7f21a0102790 2024-09-10T09:52:47.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.257+0000 7f21a5e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f21a0112140 con 0x7f21a0106f10 2024-09-10T09:52:47.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.257+0000 7f219ea00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f21a0101de0 0x7f21a019a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:47.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.257+0000 7f219ea00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f21a0101de0 0x7f21a019a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42970/0 (socket says 172.21.15.77:42970) 2024-09-10T09:52:47.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.257+0000 7f219ea00640 1 -- 172.21.15.77:0/387777695 learned_addr learned my addr 172.21.15.77:0/387777695 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:52:47.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.257+0000 7f219f400640 1 --2- 172.21.15.77:0/387777695 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21a0106f10 0x7f21a019eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:47.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.258+0000 7f2196000640 1 --2- 172.21.15.77:0/387777695 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f21a0102790 0x7f21a019ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:47.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.258+0000 7f219ea00640 1 -- 172.21.15.77:0/387777695 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21a0106f10 msgr2=0x7f21a019eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:47.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.258+0000 7f219ea00640 1 --2- 172.21.15.77:0/387777695 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21a0106f10 0x7f21a019eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:47.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.258+0000 7f219ea00640 1 -- 172.21.15.77:0/387777695 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f21a0102790 msgr2=0x7f21a019ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:47.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.258+0000 7f219ea00640 1 --2- 172.21.15.77:0/387777695 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f21a0102790 0x7f21a019ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:47.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.258+0000 7f219ea00640 1 -- 172.21.15.77:0/387777695 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21a019f590 con 0x7f21a0101de0 2024-09-10T09:52:47.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.258+0000 7f219f400640 1 --2- 172.21.15.77:0/387777695 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21a0106f10 0x7f21a019eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:52:47.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.258+0000 7f219ea00640 1 --2- 172.21.15.77:0/387777695 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f21a0101de0 0x7f21a019a5e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f219000e9f0 tx=0x7f219000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:47.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.258+0000 7f21a4a00640 1 -- 172.21.15.77:0/387777695 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f219000cd80 con 0x7f21a0101de0 2024-09-10T09:52:47.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.258+0000 7f21a4a00640 1 -- 172.21.15.77:0/387777695 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2190010910 con 0x7f21a0101de0 2024-09-10T09:52:47.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.258+0000 7f21a4a00640 1 -- 172.21.15.77:0/387777695 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2190018990 con 0x7f21a0101de0 2024-09-10T09:52:47.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.258+0000 7f21a5e00640 1 -- 172.21.15.77:0/387777695 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21a019f880 con 0x7f21a0101de0 2024-09-10T09:52:47.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.259+0000 7f21a5e00640 1 -- 172.21.15.77:0/387777695 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21a01a7160 con 0x7f21a0101de0 2024-09-10T09:52:47.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.262+0000 7f21a4a00640 1 -- 172.21.15.77:0/387777695 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f2190010ac0 con 0x7f21a0101de0 2024-09-10T09:52:47.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.262+0000 7f21a8880740 1 Processor -- start 2024-09-10T09:52:47.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.262+0000 7f21a8880740 1 -- start start 2024-09-10T09:52:47.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.262+0000 7f21a8880740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55875589d330 0x55875589d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:47.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.262+0000 7f21a8880740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55875589dc50 0x5587558a7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:47.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.262+0000 7f21a8880740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5587558a8510 0x5587558aa900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:47.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.262+0000 7f21a8880740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5587558030b0 con 0x55875589dc50 2024-09-10T09:52:47.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.262+0000 7f21a8880740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5587557aaab0 con 0x5587558a8510 2024-09-10T09:52:47.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.262+0000 7f21a8880740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5587557ff070 con 0x55875589d330 2024-09-10T09:52:47.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.263+0000 7f219f400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55875589d330 0x55875589d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:47.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.263+0000 7f219ea00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55875589dc50 0x5587558a7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:47.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.263+0000 7f219f400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55875589d330 0x55875589d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:52962/0 (socket says 172.21.15.77:52962) 2024-09-10T09:52:47.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.263+0000 7f219f400640 1 -- 172.21.15.77:0/1289398075 learned_addr learned my addr 172.21.15.77:0/1289398075 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:52:47.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.263+0000 7f2196000640 1 --2- 172.21.15.77:0/1289398075 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5587558a8510 0x5587558aa900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:47.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.263+0000 7f219f400640 1 -- 172.21.15.77:0/1289398075 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5587558a8510 msgr2=0x5587558aa900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:52:47.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.263+0000 7f219f400640 1 --2- 172.21.15.77:0/1289398075 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5587558a8510 0x5587558aa900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:47.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.263+0000 7f219f400640 1 -- 172.21.15.77:0/1289398075 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55875589dc50 msgr2=0x5587558a7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:52:47.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.263+0000 7f219f400640 1 --2- 172.21.15.77:0/1289398075 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55875589dc50 0x5587558a7fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:47.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.263+0000 7f219f400640 1 -- 172.21.15.77:0/1289398075 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587557b74a0 con 0x55875589d330 2024-09-10T09:52:47.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.263+0000 7f2196000640 1 --2- 172.21.15.77:0/1289398075 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5587558a8510 0x5587558aa900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:52:47.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.263+0000 7f219ea00640 1 --2- 172.21.15.77:0/1289398075 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55875589dc50 0x5587558a7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:52:47.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.264+0000 7f21a4a00640 1 --2- 172.21.15.77:0/387777695 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f21840a2b80 0x7f21840a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:47.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.264+0000 7f219f400640 1 --2- 172.21.15.77:0/1289398075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55875589d330 0x55875589d710 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2198035000 tx=0x7f2198009800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=399842018fca974f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:47.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.264+0000 7f21a4a00640 1 -- 172.21.15.77:0/387777695 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7f21900cf6a0 con 0x7f21a0101de0 2024-09-10T09:52:47.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.264+0000 7f219ea00640 1 --2- 172.21.15.77:0/387777695 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f21840a2b80 0x7f21840a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:47.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.264+0000 7f218fe00640 1 -- 172.21.15.77:0/1289398075 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2198031db0 con 0x55875589d330 2024-09-10T09:52:47.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.264+0000 7f218fe00640 1 -- 172.21.15.77:0/1289398075 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2198036040 con 0x55875589d330 2024-09-10T09:52:47.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.264+0000 7f218fe00640 1 -- 172.21.15.77:0/1289398075 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2198004330 con 0x55875589d330 2024-09-10T09:52:47.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.265+0000 7f219ea00640 1 --2- 172.21.15.77:0/387777695 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f21840a2b80 0x7f21840a5040 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f219000c6a0 tx=0x7f21900d5530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:47.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.265+0000 7f21a8880740 1 -- 172.21.15.77:0/1289398075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55875589d330 msgr2=0x55875589d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:47.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.265+0000 7f21a8880740 1 --2- 172.21.15.77:0/1289398075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55875589d330 0x55875589d710 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2198035000 tx=0x7f2198009800 comp rx=0 tx=0).stop 2024-09-10T09:52:47.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.265+0000 7f21a8880740 1 -- 172.21.15.77:0/1289398075 shutdown_connections 2024-09-10T09:52:47.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.265+0000 7f21a8880740 1 --2- 172.21.15.77:0/1289398075 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5587558a8510 0x5587558aa900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:47.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.265+0000 7f21a8880740 1 --2- 172.21.15.77:0/1289398075 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55875589dc50 0x5587558a7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:47.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.265+0000 7f21a8880740 1 --2- 172.21.15.77:0/1289398075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55875589d330 0x55875589d710 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:47.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.265+0000 7f21a8880740 1 -- 172.21.15.77:0/1289398075 >> 172.21.15.77:0/1289398075 conn(0x5587558998c0 msgr2=0x558755899ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:47.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.266+0000 7f21a8880740 1 -- 172.21.15.77:0/1289398075 shutdown_connections 2024-09-10T09:52:47.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.266+0000 7f21a8880740 1 -- 172.21.15.77:0/1289398075 wait complete. 2024-09-10T09:52:47.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.266+0000 7f21a8880740 1 -- start start 2024-09-10T09:52:47.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.266+0000 7f21a8880740 1 Processor -- start 2024-09-10T09:52:47.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.267+0000 7f21a8880740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55875589d330 0x5587558a55a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:47.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.267+0000 7f21a8880740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55875589dc50 0x5587558a5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:47.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.267+0000 7f21a8880740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587558a8510 0x5587558a6020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:47.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.267+0000 7f21a8880740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5587558ab060 con 0x5587558a8510 2024-09-10T09:52:47.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.267+0000 7f21a8880740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55875578b920 con 0x55875589dc50 2024-09-10T09:52:47.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.267+0000 7f21a8880740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5587558ab1e0 con 0x55875589d330 2024-09-10T09:52:47.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.267+0000 7f21a8880740 10 client.0 _ll_register_callbacks cb 0x5587557af8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:52:47.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.267+0000 7f219f400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587558a8510 0x5587558a6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:47.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.267+0000 7f2196000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55875589dc50 0x5587558a5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:47.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.267+0000 7f2196000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55875589dc50 0x5587558a5ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:50058/0 (socket says 172.21.15.77:50058) 2024-09-10T09:52:47.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.267+0000 7f219f400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587558a8510 0x5587558a6020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42996/0 (socket says 172.21.15.77:42996) 2024-09-10T09:52:47.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.267+0000 7f2196000640 1 -- 172.21.15.77:0/1211075102 learned_addr learned my addr 172.21.15.77:0/1211075102 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:52:47.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.267+0000 7f219f400640 1 --2- 172.21.15.77:0/1211075102 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55875589d330 0x5587558a55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:47.272 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.268+0000 7f2196000640 1 -- 172.21.15.77:0/1211075102 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55875589d330 msgr2=0x5587558a55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:52:47.272 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.268+0000 7f2196000640 1 --2- 172.21.15.77:0/1211075102 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55875589d330 0x5587558a55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:47.272 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.268+0000 7f2196000640 1 -- 172.21.15.77:0/1211075102 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587558a8510 msgr2=0x5587558a6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:52:47.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.268+0000 7f2196000640 1 --2- 172.21.15.77:0/1211075102 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587558a8510 0x5587558a6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:47.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.268+0000 7f2196000640 1 -- 172.21.15.77:0/1211075102 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587555742b0 con 0x55875589dc50 2024-09-10T09:52:47.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.268+0000 7f2196000640 1 --2- 172.21.15.77:0/1211075102 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55875589dc50 0x5587558a5ae0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f218800bed0 tx=0x7f21880117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5aec364b28495df server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:47.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.268+0000 7f218cc00640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T09:52:47.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.268+0000 7f218cc00640 1 -- 172.21.15.77:0/1211075102 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2188012920 con 0x55875589dc50 2024-09-10T09:52:47.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.268+0000 7f21a8880740 1 -- 172.21.15.77:0/1211075102 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2198036040 con 0x55875589dc50 2024-09-10T09:52:47.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.268+0000 7f218cc00640 1 -- 172.21.15.77:0/1211075102 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7f2188012ac0 con 0x55875589dc50 2024-09-10T09:52:47.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.269+0000 7f218cc00640 1 -- 172.21.15.77:0/1211075102 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f2188012cc0 con 0x55875589dc50 2024-09-10T09:52:47.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.269+0000 7f21a8880740 10 client.24656 fetch_fsmap learned FSMap version 17 2024-09-10T09:52:47.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.269+0000 7f21a8880740 1 -- 172.21.15.77:0/1211075102 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x5587558b6ea0 con 0x55875589dc50 2024-09-10T09:52:47.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.269+0000 7f218cc00640 1 -- 172.21.15.77:0/1211075102 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2198036040 con 0x55875589dc50 2024-09-10T09:52:47.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.269+0000 7f218cc00640 1 -- 172.21.15.77:0/1211075102 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 17) ==== 8350+0+0 (secure 0 0 0) 0x7f218800f440 con 0x55875589dc50 2024-09-10T09:52:47.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.269+0000 7f21a8880740 10 client.24656 fetch_fsmap finished waiting for FSMap version 17 2024-09-10T09:52:47.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.269+0000 7f21a8880740 10 client.24656 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T09:52:47.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.269+0000 7f21a8880740 20 client.24656 populate_metadata read hostname 'smithi077' 2024-09-10T09:52:47.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.270+0000 7f21a8880740 1 --2- 172.21.15.77:0/1211075102 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x5587558b9490 0x5587558ba900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:47.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.270+0000 7f21a8880740 4 client.24656 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:52:47.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.270+0000 7f21a8880740 1 -- 172.21.15.77:0/1211075102 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558755810620 con 0x5587558b9490 2024-09-10T09:52:47.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.270+0000 7f219f400640 1 --2- 172.21.15.77:0/1211075102 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x5587558b9490 0x5587558ba900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:47.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.270+0000 7f219f400640 1 --2- 172.21.15.77:0/1211075102 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x5587558b9490 0x5587558ba900 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d516f8ba5a14f7 server_cookie=fa2d9e86a952bac8 in_seq=0 out_seq=0 2024-09-10T09:52:47.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.270+0000 7f218cc00640 10 client.24656 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T09:52:47.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.271+0000 7f218cc00640 1 -- 172.21.15.77:0/1211075102 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558755810620 con 0x5587558b9490 2024-09-10T09:52:47.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.271+0000 7f218cc00640 10 client.24656 handle_command_reply: tid=0 2024-09-10T09:52:47.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.271+0000 7f21a8880740 1 client.24656 shutdown 2024-09-10T09:52:47.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.272+0000 7f21a8880740 1 -- 172.21.15.77:0/1211075102 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55875589dc50 msgr2=0x5587558a5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:47.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.272+0000 7f21a8880740 1 --2- 172.21.15.77:0/1211075102 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55875589dc50 0x5587558a5ae0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f218800bed0 tx=0x7f21880117b0 comp rx=0 tx=0).stop 2024-09-10T09:52:47.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.272+0000 7f21a8880740 1 -- 172.21.15.77:0/1211075102 shutdown_connections 2024-09-10T09:52:47.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.272+0000 7f21a8880740 1 --2- 172.21.15.77:0/1211075102 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x5587558b9490 0x5587558ba900 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:47.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.272+0000 7f21a8880740 1 --2- 172.21.15.77:0/1211075102 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587558a8510 0x5587558a6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:47.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.272+0000 7f21a8880740 1 --2- 172.21.15.77:0/1211075102 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55875589dc50 0x5587558a5ae0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:47.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.272+0000 7f21a8880740 1 --2- 172.21.15.77:0/1211075102 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55875589d330 0x5587558a55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:47.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.272+0000 7f21a8880740 1 -- 172.21.15.77:0/1211075102 >> 172.21.15.77:0/1211075102 conn(0x5587558998c0 msgr2=0x55875589f5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:47.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.272+0000 7f218cc00640 0 client.24656 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T09:52:47.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.272+0000 7f21a8880740 1 -- 172.21.15.77:0/1211075102 shutdown_connections 2024-09-10T09:52:47.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.272+0000 7f21a8880740 1 -- 172.21.15.77:0/1211075102 wait complete. 2024-09-10T09:52:47.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.272+0000 7f21a8880740 20 client.24656 trim_cache size 0 max 16384 2024-09-10T09:52:47.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.290+0000 7f21a8880740 1 Processor -- start 2024-09-10T09:52:47.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.291+0000 7f21a8880740 1 -- start start 2024-09-10T09:52:47.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.291+0000 7f21a8880740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55875589d330 0x5587558b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:47.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.291+0000 7f21a8880740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55875589dc50 0x5587558a3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:47.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.291+0000 7f21a8880740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587558a8510 0x5587558a43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:47.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.291+0000 7f21a8880740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5587558b8c40 con 0x5587558a8510 2024-09-10T09:52:47.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.291+0000 7f21a8880740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5587558b9310 con 0x55875589dc50 2024-09-10T09:52:47.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.291+0000 7f21a8880740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5587558b99e0 con 0x55875589d330 2024-09-10T09:52:47.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.291+0000 7f219f400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587558a8510 0x5587558a43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:47.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.291+0000 7f219f400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587558a8510 0x5587558a43f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:43002/0 (socket says 172.21.15.77:43002) 2024-09-10T09:52:47.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.291+0000 7f219f400640 1 -- 172.21.15.77:0/151651475 learned_addr learned my addr 172.21.15.77:0/151651475 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:52:47.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.291+0000 7f2196000640 1 --2- 172.21.15.77:0/151651475 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55875589dc50 0x5587558a3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:47.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.291+0000 7f219f400640 1 --2- 172.21.15.77:0/151651475 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55875589d330 0x5587558b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:47.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.292+0000 7f219f400640 1 -- 172.21.15.77:0/151651475 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55875589d330 msgr2=0x5587558b4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:52:47.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.292+0000 7f219f400640 1 --2- 172.21.15.77:0/151651475 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55875589d330 0x5587558b4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:47.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.292+0000 7f219f400640 1 -- 172.21.15.77:0/151651475 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55875589dc50 msgr2=0x5587558a3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:52:47.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.292+0000 7f219f400640 1 --2- 172.21.15.77:0/151651475 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55875589dc50 0x5587558a3eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:47.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.292+0000 7f219f400640 1 -- 172.21.15.77:0/151651475 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55875589da80 con 0x5587558a8510 2024-09-10T09:52:47.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.292+0000 7f2196000640 1 --2- 172.21.15.77:0/151651475 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55875589dc50 0x5587558a3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:52:47.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.292+0000 7f219f400640 1 --2- 172.21.15.77:0/151651475 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587558a8510 0x5587558a43f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2198031a90 tx=0x7f2198031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60cc16346a5cb290 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:47.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.292+0000 7f218f400640 1 -- 172.21.15.77:0/151651475 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2198032cd0 con 0x5587558a8510 2024-09-10T09:52:47.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.292+0000 7f218f400640 1 -- 172.21.15.77:0/151651475 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2198036040 con 0x5587558a8510 2024-09-10T09:52:47.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.292+0000 7f218f400640 1 -- 172.21.15.77:0/151651475 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2198004620 con 0x5587558a8510 2024-09-10T09:52:47.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.293+0000 7f21a8880740 1 -- 172.21.15.77:0/151651475 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587558a8510 msgr2=0x5587558a43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:47.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.293+0000 7f21a8880740 1 --2- 172.21.15.77:0/151651475 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587558a8510 0x5587558a43f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2198031a90 tx=0x7f2198031a20 comp rx=0 tx=0).stop 2024-09-10T09:52:47.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.293+0000 7f21a8880740 1 -- 172.21.15.77:0/151651475 shutdown_connections 2024-09-10T09:52:47.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.293+0000 7f21a8880740 1 --2- 172.21.15.77:0/151651475 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587558a8510 0x5587558a43f0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:47.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.293+0000 7f21a8880740 1 --2- 172.21.15.77:0/151651475 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55875589dc50 0x5587558a3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:47.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.293+0000 7f21a8880740 1 --2- 172.21.15.77:0/151651475 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55875589d330 0x5587558b4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:47.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.293+0000 7f21a8880740 1 -- 172.21.15.77:0/151651475 >> 172.21.15.77:0/151651475 conn(0x5587558998c0 msgr2=0x558755899ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:47.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.293+0000 7f21a8880740 1 -- 172.21.15.77:0/151651475 shutdown_connections 2024-09-10T09:52:47.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.293+0000 7f21a8880740 1 -- 172.21.15.77:0/151651475 wait complete. 2024-09-10T09:52:47.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.293+0000 7f21a8880740 1 -- start start 2024-09-10T09:52:47.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.294+0000 7f21a8880740 1 Processor -- start 2024-09-10T09:52:47.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.294+0000 7f21a8880740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55875589d330 0x5587558c3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:47.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.294+0000 7f21a8880740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55875589dc50 0x5587558a2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:47.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.294+0000 7f21a8880740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5587558a8510 0x5587558a26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:47.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.294+0000 7f21a8880740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5587558ba780 con 0x55875589d330 2024-09-10T09:52:47.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.294+0000 7f21a8880740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5587558ba0b0 con 0x5587558a8510 2024-09-10T09:52:47.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.294+0000 7f21a8880740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5587558bae50 con 0x55875589dc50 2024-09-10T09:52:47.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.294+0000 7f21a8880740 10 client.0 _ll_register_callbacks cb 0x7f218800bae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:52:47.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.294+0000 7f219f400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55875589d330 0x5587558c3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:47.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.294+0000 7f219f400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55875589d330 0x5587558c3050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:43018/0 (socket says 172.21.15.77:43018) 2024-09-10T09:52:47.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.294+0000 7f219f400640 1 -- 172.21.15.77:0/3974348881 learned_addr learned my addr 172.21.15.77:0/3974348881 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:52:47.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.294+0000 7f2196000640 1 --2- 172.21.15.77:0/3974348881 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55875589dc50 0x5587558a2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:47.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.295+0000 7f219f400640 1 --2- 172.21.15.77:0/3974348881 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5587558a8510 0x5587558a26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:47.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.295+0000 7f219f400640 1 -- 172.21.15.77:0/3974348881 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55875589dc50 msgr2=0x5587558a2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:52:47.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.295+0000 7f219f400640 1 --2- 172.21.15.77:0/3974348881 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55875589dc50 0x5587558a2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:47.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.295+0000 7f219f400640 1 -- 172.21.15.77:0/3974348881 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5587558a8510 msgr2=0x5587558a26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:52:47.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.295+0000 7f219f400640 1 --2- 172.21.15.77:0/3974348881 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5587558a8510 0x5587558a26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:47.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.295+0000 7f219f400640 1 -- 172.21.15.77:0/3974348881 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587558a2c10 con 0x55875589d330 2024-09-10T09:52:47.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.295+0000 7f219f400640 1 --2- 172.21.15.77:0/3974348881 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55875589d330 0x5587558c3050 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f219802f470 tx=0x7f2198009540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6bcc8294d018f12 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:47.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.295+0000 7f218cc00640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-10T09:52:47.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.295+0000 7f218cc00640 1 -- 172.21.15.77:0/3974348881 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2198004140 con 0x55875589d330 2024-09-10T09:52:47.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.295+0000 7f21a8880740 1 -- 172.21.15.77:0/3974348881 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2198036040 con 0x55875589d330 2024-09-10T09:52:47.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.295+0000 7f218cc00640 1 -- 172.21.15.77:0/3974348881 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7f2198032cd0 con 0x55875589d330 2024-09-10T09:52:47.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.296+0000 7f218cc00640 1 -- 172.21.15.77:0/3974348881 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f2198032ed0 con 0x55875589d330 2024-09-10T09:52:47.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.296+0000 7f21a8880740 10 client.14607 fetch_fsmap learned FSMap version 17 2024-09-10T09:52:47.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.296+0000 7f21a8880740 1 -- 172.21.15.77:0/3974348881 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x5587558c1990 con 0x55875589d330 2024-09-10T09:52:47.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.296+0000 7f218cc00640 1 -- 172.21.15.77:0/3974348881 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2198036040 con 0x55875589d330 2024-09-10T09:52:47.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.296+0000 7f218cc00640 1 -- 172.21.15.77:0/3974348881 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 17) ==== 8350+0+0 (secure 0 0 0) 0x7f2198003b70 con 0x55875589d330 2024-09-10T09:52:47.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.296+0000 7f21a8880740 10 client.14607 fetch_fsmap finished waiting for FSMap version 17 2024-09-10T09:52:47.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.296+0000 7f21a8880740 10 client.14607 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T09:52:47.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.296+0000 7f21a8880740 20 client.14607 populate_metadata read hostname 'smithi077' 2024-09-10T09:52:47.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.296+0000 7f21a8880740 1 --2- 172.21.15.77:0/3974348881 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x558755927f00 0x558755926c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:47.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.296+0000 7f21a8880740 4 client.14607 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T09:52:47.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.296+0000 7f21a8880740 1 -- 172.21.15.77:0/3974348881 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "status"}) -- 0x55875580ab30 con 0x558755927f00 2024-09-10T09:52:47.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.297+0000 7f2196000640 1 --2- 172.21.15.77:0/3974348881 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x558755927f00 0x558755926c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:47.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.297+0000 7f2196000640 1 --2- 172.21.15.77:0/3974348881 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x558755927f00 0x558755926c10 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad62feb246f5747e server_cookie=7aaa810db5553e1e in_seq=0 out_seq=0 2024-09-10T09:52:47.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.297+0000 7f218cc00640 10 client.14607 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T09:52:47.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.297+0000 7f218cc00640 1 -- 172.21.15.77:0/3974348881 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x55875580ab30 con 0x558755927f00 2024-09-10T09:52:47.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.297+0000 7f218cc00640 10 client.14607 handle_command_reply: tid=0 2024-09-10T09:52:47.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.297+0000 7f21a8880740 1 client.14607 shutdown 2024-09-10T09:52:47.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.297+0000 7f21a8880740 1 -- 172.21.15.77:0/3974348881 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55875589d330 msgr2=0x5587558c3050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:47.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.297+0000 7f21a8880740 1 --2- 172.21.15.77:0/3974348881 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55875589d330 0x5587558c3050 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f219802f470 tx=0x7f2198009540 comp rx=0 tx=0).stop 2024-09-10T09:52:47.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.298+0000 7f21a8880740 1 -- 172.21.15.77:0/3974348881 shutdown_connections 2024-09-10T09:52:47.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.298+0000 7f21a8880740 1 --2- 172.21.15.77:0/3974348881 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x558755927f00 0x558755926c10 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:47.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.298+0000 7f21a8880740 1 --2- 172.21.15.77:0/3974348881 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5587558a8510 0x5587558a26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:47.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.298+0000 7f21a8880740 1 --2- 172.21.15.77:0/3974348881 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55875589dc50 0x5587558a2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:47.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.298+0000 7f21a8880740 1 --2- 172.21.15.77:0/3974348881 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55875589d330 0x5587558c3050 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:47.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.298+0000 7f218cc00640 0 client.14607 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T09:52:47.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.298+0000 7f21a8880740 1 -- 172.21.15.77:0/3974348881 >> 172.21.15.77:0/3974348881 conn(0x5587558998c0 msgr2=0x5587558a3e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:47.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.298+0000 7f21a8880740 1 -- 172.21.15.77:0/3974348881 shutdown_connections 2024-09-10T09:52:47.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.298+0000 7f21a8880740 1 -- 172.21.15.77:0/3974348881 wait complete. 2024-09-10T09:52:47.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.298+0000 7f21a8880740 20 client.14607 trim_cache size 0 max 16384 2024-09-10T09:52:47.304 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-10T09:52:47.304 INFO:teuthology.orchestra.run.smithi077.stdout: "cluster_fsid": "14a55226-6f59-11ef-bcea-c7b262605968", 2024-09-10T09:52:47.304 INFO:teuthology.orchestra.run.smithi077.stdout: "whoami": 0, 2024-09-10T09:52:47.304 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 24563, 2024-09-10T09:52:47.304 INFO:teuthology.orchestra.run.smithi077.stdout: "want_state": "up:active", 2024-09-10T09:52:47.304 INFO:teuthology.orchestra.run.smithi077.stdout: "state": "up:active", 2024-09-10T09:52:47.304 INFO:teuthology.orchestra.run.smithi077.stdout: "fs_name": "cephfs", 2024-09-10T09:52:47.304 INFO:teuthology.orchestra.run.smithi077.stdout: "rank_uptime": 22.985352788, 2024-09-10T09:52:47.304 INFO:teuthology.orchestra.run.smithi077.stdout: "mdsmap_epoch": 17, 2024-09-10T09:52:47.304 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch": 92, 2024-09-10T09:52:47.305 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch_barrier": 91, 2024-09-10T09:52:47.305 INFO:teuthology.orchestra.run.smithi077.stdout: "uptime": 36.615656284000003 2024-09-10T09:52:47.305 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-10T09:52:47.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.298+0000 7f21a5e00640 1 -- 172.21.15.77:0/387777695 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f21840a2b80 msgr2=0x7f21840a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:47.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.298+0000 7f21a5e00640 1 --2- 172.21.15.77:0/387777695 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f21840a2b80 0x7f21840a5040 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f219000c6a0 tx=0x7f21900d5530 comp rx=0 tx=0).stop 2024-09-10T09:52:47.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.298+0000 7f21a5e00640 1 -- 172.21.15.77:0/387777695 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f21a0101de0 msgr2=0x7f21a019a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:47.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.298+0000 7f21a5e00640 1 --2- 172.21.15.77:0/387777695 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f21a0101de0 0x7f21a019a5e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f219000e9f0 tx=0x7f219000eeb0 comp rx=0 tx=0).stop 2024-09-10T09:52:47.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.299+0000 7f21a5e00640 1 -- 172.21.15.77:0/387777695 shutdown_connections 2024-09-10T09:52:47.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.299+0000 7f21a5e00640 1 --2- 172.21.15.77:0/387777695 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f21840a2b80 0x7f21840a5040 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:47.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.299+0000 7f21a5e00640 1 --2- 172.21.15.77:0/387777695 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21a0106f10 0x7f21a019eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:47.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.299+0000 7f21a5e00640 1 --2- 172.21.15.77:0/387777695 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f21a0102790 0x7f21a019ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:47.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.299+0000 7f21a5e00640 1 --2- 172.21.15.77:0/387777695 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f21a0101de0 0x7f21a019a5e0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:47.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.299+0000 7f21a5e00640 1 -- 172.21.15.77:0/387777695 >> 172.21.15.77:0/387777695 conn(0x7f21a00fd8c0 msgr2=0x7f21a00fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:47.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.299+0000 7f21a5e00640 1 -- 172.21.15.77:0/387777695 shutdown_connections 2024-09-10T09:52:47.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:47.299+0000 7f21a5e00640 1 -- 172.21.15.77:0/387777695 wait complete. 2024-09-10T09:52:48.141 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-10T09:52:48.142 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-10T09:52:48.142 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph fs ls 2024-09-10T09:52:48.511 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:52:48.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:48 smithi077 ceph-mon[30319]: pgmap v316: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:52:49.122 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:49.120+0000 7faaea400640 1 -- 172.21.15.77:0/1080749504 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faae4102790 msgr2=0x7faae41067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:49.122 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:49.120+0000 7faaea400640 1 --2- 172.21.15.77:0/1080749504 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faae4102790 0x7faae41067e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7faacc009900 tx=0x7faacc02a410 comp rx=0 tx=0).stop 2024-09-10T09:52:49.122 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:49.120+0000 7faaea400640 1 -- 172.21.15.77:0/1080749504 shutdown_connections 2024-09-10T09:52:49.123 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:49.120+0000 7faaea400640 1 --2- 172.21.15.77:0/1080749504 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faae4106f10 0x7faae410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:49.123 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:49.120+0000 7faaea400640 1 --2- 172.21.15.77:0/1080749504 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faae4102790 0x7faae41067e0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:49.123 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:49.120+0000 7faaea400640 1 --2- 172.21.15.77:0/1080749504 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faae4101de0 0x7faae41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:49.123 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:49.120+0000 7faaea400640 1 -- 172.21.15.77:0/1080749504 >> 172.21.15.77:0/1080749504 conn(0x7faae40fd8c0 msgr2=0x7faae40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:49.123 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:49.120+0000 7faaea400640 1 -- 172.21.15.77:0/1080749504 shutdown_connections 2024-09-10T09:52:49.123 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:49.120+0000 7faaea400640 1 -- 172.21.15.77:0/1080749504 wait complete. 2024-09-10T09:52:49.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:49.121+0000 7faaea400640 1 Processor -- start 2024-09-10T09:52:49.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:49.122+0000 7faaea400640 1 -- start start 2024-09-10T09:52:49.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:49.122+0000 7faaea400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faae4101de0 0x7faae419a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:49.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:49.122+0000 7faaea400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faae4102790 0x7faae419ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:49.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:49.122+0000 7faaea400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faae4106f10 0x7faae419ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:49.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:49.122+0000 7faaea400640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7faae4111f70 con 0x7faae4101de0 2024-09-10T09:52:49.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:49.122+0000 7faaea400640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7faae4111df0 con 0x7faae4106f10 2024-09-10T09:52:49.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:49.122+0000 7faaea400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7faae41120f0 con 0x7faae4102790 2024-09-10T09:52:49.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:49.122+0000 7faae3400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faae4101de0 0x7faae419a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:49.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:49.122+0000 7faae3400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faae4101de0 0x7faae419a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:43038/0 (socket says 172.21.15.77:43038) 2024-09-10T09:52:49.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:49.122+0000 7faae3400640 1 -- 172.21.15.77:0/4112917354 learned_addr learned my addr 172.21.15.77:0/4112917354 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:52:49.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:49.122+0000 7faadbe00640 1 --2- 172.21.15.77:0/4112917354 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faae4102790 0x7faae419ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:49.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:49.122+0000 7faae3e00640 1 --2- 172.21.15.77:0/4112917354 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faae4106f10 0x7faae419ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:49.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:49.123+0000 7faae3400640 1 -- 172.21.15.77:0/4112917354 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faae4102790 msgr2=0x7faae419ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:49.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:49.123+0000 7faae3400640 1 --2- 172.21.15.77:0/4112917354 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faae4102790 0x7faae419ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:49.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:49.123+0000 7faae3400640 1 -- 172.21.15.77:0/4112917354 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faae4106f10 msgr2=0x7faae419ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:49.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:49.123+0000 7faae3400640 1 --2- 172.21.15.77:0/4112917354 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faae4106f10 0x7faae419ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:49.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:49.123+0000 7faae3400640 1 -- 172.21.15.77:0/4112917354 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faae419f600 con 0x7faae4101de0 2024-09-10T09:52:49.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:49.123+0000 7faae3400640 1 --2- 172.21.15.77:0/4112917354 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faae4101de0 0x7faae419a650 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7faad400e9f0 tx=0x7faad400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:49.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:49.123+0000 7faae0c00640 1 -- 172.21.15.77:0/4112917354 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faad400cd80 con 0x7faae4101de0 2024-09-10T09:52:49.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:49.123+0000 7faae0c00640 1 -- 172.21.15.77:0/4112917354 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faad4010910 con 0x7faae4101de0 2024-09-10T09:52:49.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:49.123+0000 7faaea400640 1 -- 172.21.15.77:0/4112917354 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faae419f8f0 con 0x7faae4101de0 2024-09-10T09:52:49.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:49.123+0000 7faae0c00640 1 -- 172.21.15.77:0/4112917354 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faad4018990 con 0x7faae4101de0 2024-09-10T09:52:49.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:49.123+0000 7faaea400640 1 -- 172.21.15.77:0/4112917354 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faae41a71d0 con 0x7faae4101de0 2024-09-10T09:52:49.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:49.125+0000 7faaea400640 1 -- 172.21.15.77:0/4112917354 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faae41021c0 con 0x7faae4101de0 2024-09-10T09:52:49.128 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:49.126+0000 7faae0c00640 1 -- 172.21.15.77:0/4112917354 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7faad4010ac0 con 0x7faae4101de0 2024-09-10T09:52:49.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:49.127+0000 7faae0c00640 1 --2- 172.21.15.77:0/4112917354 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7faac80a2c00 0x7faac80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:49.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:49.128+0000 7faae0c00640 1 -- 172.21.15.77:0/4112917354 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7faad40cf550 con 0x7faae4101de0 2024-09-10T09:52:49.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:49.128+0000 7faadbe00640 1 --2- 172.21.15.77:0/4112917354 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7faac80a2c00 0x7faac80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:49.130 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:49.128+0000 7faadbe00640 1 --2- 172.21.15.77:0/4112917354 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7faac80a2c00 0x7faac80a50c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7faacc009230 tx=0x7faacc0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:49.130 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:48 smithi134 ceph-mon[35674]: pgmap v316: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:52:49.133 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:49.132+0000 7faae0c00640 1 -- 172.21.15.77:0/4112917354 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7faad409bbc0 con 0x7faae4101de0 2024-09-10T09:52:49.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:48 smithi120 ceph-mon[34770]: pgmap v316: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:52:49.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:49.342+0000 7faaea400640 1 -- 172.21.15.77:0/4112917354 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7faae41a7400 con 0x7faae4101de0 2024-09-10T09:52:49.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:49.343+0000 7faae0c00640 1 -- 172.21.15.77:0/4112917354 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+88 (secure 0 0 0) 0x7faad409fec0 con 0x7faae4101de0 2024-09-10T09:52:49.345 INFO:teuthology.orchestra.run.smithi077.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-10T09:52:49.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:49.345+0000 7faaea400640 1 -- 172.21.15.77:0/4112917354 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7faac80a2c00 msgr2=0x7faac80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:49.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:49.345+0000 7faaea400640 1 --2- 172.21.15.77:0/4112917354 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7faac80a2c00 0x7faac80a50c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7faacc009230 tx=0x7faacc0058d0 comp rx=0 tx=0).stop 2024-09-10T09:52:49.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:49.345+0000 7faaea400640 1 -- 172.21.15.77:0/4112917354 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faae4101de0 msgr2=0x7faae419a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:49.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:49.345+0000 7faaea400640 1 --2- 172.21.15.77:0/4112917354 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faae4101de0 0x7faae419a650 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7faad400e9f0 tx=0x7faad400eeb0 comp rx=0 tx=0).stop 2024-09-10T09:52:49.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:49.345+0000 7faaea400640 1 -- 172.21.15.77:0/4112917354 shutdown_connections 2024-09-10T09:52:49.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:49.345+0000 7faaea400640 1 --2- 172.21.15.77:0/4112917354 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7faac80a2c00 0x7faac80a50c0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:49.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:49.345+0000 7faaea400640 1 --2- 172.21.15.77:0/4112917354 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faae4106f10 0x7faae419ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:49.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:49.345+0000 7faaea400640 1 --2- 172.21.15.77:0/4112917354 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faae4102790 0x7faae419ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:49.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:49.345+0000 7faaea400640 1 --2- 172.21.15.77:0/4112917354 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faae4101de0 0x7faae419a650 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:49.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:49.345+0000 7faaea400640 1 -- 172.21.15.77:0/4112917354 >> 172.21.15.77:0/4112917354 conn(0x7faae40fd8c0 msgr2=0x7faae40feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:49.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:49.345+0000 7faaea400640 1 -- 172.21.15.77:0/4112917354 shutdown_connections 2024-09-10T09:52:49.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:49.345+0000 7faaea400640 1 -- 172.21.15.77:0/4112917354 wait complete. 2024-09-10T09:52:50.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:49 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/4112917354' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-10T09:52:50.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:49 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/4112917354' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-10T09:52:50.188 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:49 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/4112917354' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-10T09:52:50.226 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T09:52:50.226 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T09:52:50.226 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi077.front.sepia.ceph.com 2024-09-10T09:52:50.226 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-09-10T09:52:50.226 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-10T09:52:50.226 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T09:52:50.226 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T09:52:50.226 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T09:52:50.226 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-09-10T09:52:50.227 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:52:50.227 DEBUG:teuthology.orchestra.run.smithi077:> ip addr 2024-09-10T09:52:50.263 INFO:teuthology.orchestra.run.smithi077.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T09:52:50.263 INFO:teuthology.orchestra.run.smithi077.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T09:52:50.263 INFO:teuthology.orchestra.run.smithi077.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T09:52:50.263 INFO:teuthology.orchestra.run.smithi077.stdout: valid_lft forever preferred_lft forever 2024-09-10T09:52:50.263 INFO:teuthology.orchestra.run.smithi077.stdout: inet6 ::1/128 scope host 2024-09-10T09:52:50.263 INFO:teuthology.orchestra.run.smithi077.stdout: valid_lft forever preferred_lft forever 2024-09-10T09:52:50.263 INFO:teuthology.orchestra.run.smithi077.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T09:52:50.264 INFO:teuthology.orchestra.run.smithi077.stdout: link/ether 0c:c4:7a:d8:d9:d8 brd ff:ff:ff:ff:ff:ff 2024-09-10T09:52:50.264 INFO:teuthology.orchestra.run.smithi077.stdout: altname enp7s0f0 2024-09-10T09:52:50.264 INFO:teuthology.orchestra.run.smithi077.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T09:52:50.264 INFO:teuthology.orchestra.run.smithi077.stdout: link/ether 0c:c4:7a:d8:d9:d9 brd ff:ff:ff:ff:ff:ff 2024-09-10T09:52:50.264 INFO:teuthology.orchestra.run.smithi077.stdout: altname enp7s0f1 2024-09-10T09:52:50.264 INFO:teuthology.orchestra.run.smithi077.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T09:52:50.264 INFO:teuthology.orchestra.run.smithi077.stdout: link/ether 0c:c4:7a:8f:d3:d2 brd ff:ff:ff:ff:ff:ff 2024-09-10T09:52:50.264 INFO:teuthology.orchestra.run.smithi077.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T09:52:50.264 INFO:teuthology.orchestra.run.smithi077.stdout: link/ether 0c:c4:7a:8f:d3:d3 brd ff:ff:ff:ff:ff:ff 2024-09-10T09:52:50.264 INFO:teuthology.orchestra.run.smithi077.stdout: inet 172.21.15.77/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-10T09:52:50.264 INFO:teuthology.orchestra.run.smithi077.stdout: valid_lft 42220sec preferred_lft 42220sec 2024-09-10T09:52:50.264 INFO:teuthology.orchestra.run.smithi077.stdout: inet6 fe80::ec4:7aff:fe8f:d3d3/64 scope link noprefixroute 2024-09-10T09:52:50.264 INFO:teuthology.orchestra.run.smithi077.stdout: valid_lft forever preferred_lft forever 2024-09-10T09:52:50.265 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-10T09:52:50.265 DEBUG:teuthology.orchestra.run.smithi077:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T09:52:50.265 DEBUG:teuthology.orchestra.run.smithi077:> set -e 2024-09-10T09:52:50.266 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip link add name ceph-brx type bridge 2024-09-10T09:52:50.266 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip addr flush dev ceph-brx 2024-09-10T09:52:50.266 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip link set ceph-brx up 2024-09-10T09:52:50.266 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-10T09:52:50.266 DEBUG:teuthology.orchestra.run.smithi077:> ') 2024-09-10T09:52:50.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T09:52:50.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T09:52:50.695 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:52:50.695 DEBUG:teuthology.orchestra.run.smithi077:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T09:52:50.751 INFO:teuthology.orchestra.run.smithi077.stdout:1 2024-09-10T09:52:50.755 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:52:50.755 DEBUG:teuthology.orchestra.run.smithi077:> route 2024-09-10T09:52:50.817 INFO:teuthology.orchestra.run.smithi077.stdout:Kernel IP routing table 2024-09-10T09:52:50.817 INFO:teuthology.orchestra.run.smithi077.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T09:52:50.817 INFO:teuthology.orchestra.run.smithi077.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T09:52:50.817 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T09:52:50.817 INFO:teuthology.orchestra.run.smithi077.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T09:52:50.817 DEBUG:teuthology.orchestra.run.smithi077:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T09:52:50.818 DEBUG:teuthology.orchestra.run.smithi077:> set -e 2024-09-10T09:52:50.818 DEBUG:teuthology.orchestra.run.smithi077:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T09:52:50.818 DEBUG:teuthology.orchestra.run.smithi077:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T09:52:50.818 DEBUG:teuthology.orchestra.run.smithi077:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T09:52:50.818 DEBUG:teuthology.orchestra.run.smithi077:> ') 2024-09-10T09:52:50.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T09:52:50.913 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:50 smithi077 ceph-mon[30319]: pgmap v317: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:52:51.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:50 smithi134 ceph-mon[35674]: pgmap v317: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:52:51.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:50 smithi120 ceph-mon[34770]: pgmap v317: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:52:51.384 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T09:52:51.392 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:52:51.392 DEBUG:teuthology.orchestra.run.smithi077:> ip netns list 2024-09-10T09:52:51.428 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:52:51.428 DEBUG:teuthology.orchestra.run.smithi077:> ip netns list-id 2024-09-10T09:52:51.504 DEBUG:teuthology.orchestra.run.smithi077:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T09:52:51.504 DEBUG:teuthology.orchestra.run.smithi077:> set -e 2024-09-10T09:52:51.504 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T09:52:51.504 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-09-10T09:52:51.504 DEBUG:teuthology.orchestra.run.smithi077:> ') 2024-09-10T09:52:51.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T09:52:51.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T09:52:51.683 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-09-10T09:52:51.683 DEBUG:teuthology.orchestra.run.smithi077:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T09:52:51.683 DEBUG:teuthology.orchestra.run.smithi077:> set -e 2024-09-10T09:52:51.683 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-09-10T09:52:51.683 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-10T09:52:51.683 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-09-10T09:52:51.683 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-09-10T09:52:51.684 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-09-10T09:52:51.684 DEBUG:teuthology.orchestra.run.smithi077:> ') 2024-09-10T09:52:51.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T09:52:51.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T09:52:51.925 DEBUG:teuthology.orchestra.run.smithi077:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T09:52:51.926 DEBUG:teuthology.orchestra.run.smithi077:> set -e 2024-09-10T09:52:51.926 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip link set brx.0 up 2024-09-10T09:52:51.926 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip link set dev brx.0 master ceph-brx 2024-09-10T09:52:51.926 DEBUG:teuthology.orchestra.run.smithi077:> ') 2024-09-10T09:52:52.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T09:52:52.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T09:52:52.092 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-09-10T09:52:52.092 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T09:52:52.092 DEBUG:teuthology.orchestra.run.smithi077:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-09-10T09:52:52.129 INFO:teuthology.orchestra.run.smithi077.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-09-10T09:52:52.129 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T09:52:52.129 DEBUG:teuthology.orchestra.run.smithi077:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-09-10T09:52:52.204 DEBUG:teuthology.orchestra.run.smithi077:> sudo modprobe fuse 2024-09-10T09:52:52.301 DEBUG:teuthology.orchestra.run.smithi077:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T09:52:52.337 INFO:teuthology.orchestra.run.smithi077.stdout:/proc 2024-09-10T09:52:52.337 INFO:teuthology.orchestra.run.smithi077.stdout:/sys 2024-09-10T09:52:52.337 INFO:teuthology.orchestra.run.smithi077.stdout:/dev 2024-09-10T09:52:52.337 INFO:teuthology.orchestra.run.smithi077.stdout:/sys/kernel/security 2024-09-10T09:52:52.337 INFO:teuthology.orchestra.run.smithi077.stdout:/dev/shm 2024-09-10T09:52:52.337 INFO:teuthology.orchestra.run.smithi077.stdout:/dev/pts 2024-09-10T09:52:52.338 INFO:teuthology.orchestra.run.smithi077.stdout:/run 2024-09-10T09:52:52.338 INFO:teuthology.orchestra.run.smithi077.stdout:/sys/fs/cgroup 2024-09-10T09:52:52.338 INFO:teuthology.orchestra.run.smithi077.stdout:/sys/fs/pstore 2024-09-10T09:52:52.338 INFO:teuthology.orchestra.run.smithi077.stdout:/sys/fs/bpf 2024-09-10T09:52:52.338 INFO:teuthology.orchestra.run.smithi077.stdout:/ 2024-09-10T09:52:52.338 INFO:teuthology.orchestra.run.smithi077.stdout:/sys/fs/selinux 2024-09-10T09:52:52.338 INFO:teuthology.orchestra.run.smithi077.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T09:52:52.338 INFO:teuthology.orchestra.run.smithi077.stdout:/dev/hugepages 2024-09-10T09:52:52.338 INFO:teuthology.orchestra.run.smithi077.stdout:/dev/mqueue 2024-09-10T09:52:52.338 INFO:teuthology.orchestra.run.smithi077.stdout:/sys/kernel/debug 2024-09-10T09:52:52.338 INFO:teuthology.orchestra.run.smithi077.stdout:/sys/kernel/tracing 2024-09-10T09:52:52.338 INFO:teuthology.orchestra.run.smithi077.stdout:/sys/fs/fuse/connections 2024-09-10T09:52:52.338 INFO:teuthology.orchestra.run.smithi077.stdout:/sys/kernel/config 2024-09-10T09:52:52.338 INFO:teuthology.orchestra.run.smithi077.stdout:/run/credentials/systemd-sysctl.service 2024-09-10T09:52:52.339 INFO:teuthology.orchestra.run.smithi077.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-10T09:52:52.339 INFO:teuthology.orchestra.run.smithi077.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-10T09:52:52.339 INFO:teuthology.orchestra.run.smithi077.stdout:/var/lib/nfs/rpc_pipefs 2024-09-10T09:52:52.339 INFO:teuthology.orchestra.run.smithi077.stdout:/run/user/1000 2024-09-10T09:52:52.339 INFO:teuthology.orchestra.run.smithi077.stdout:/var/lib/ceph 2024-09-10T09:52:52.339 INFO:teuthology.orchestra.run.smithi077.stdout:/var/lib/containers/storage/overlay 2024-09-10T09:52:52.339 INFO:teuthology.orchestra.run.smithi077.stdout:/var/lib/containers/storage/overlay/35721519d96d163578ce1de5316d869b3092221fa3253a979f83484306626b78/merged 2024-09-10T09:52:52.339 INFO:teuthology.orchestra.run.smithi077.stdout:/var/lib/containers/storage/overlay/5785a63d1b9c229507112ef2753db50a385d6252c16ff542681164cb671a5707/merged 2024-09-10T09:52:52.339 INFO:teuthology.orchestra.run.smithi077.stdout:/run/user/0 2024-09-10T09:52:52.339 INFO:teuthology.orchestra.run.smithi077.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T09:52:52.339 INFO:teuthology.orchestra.run.smithi077.stdout:/var/lib/containers/storage/overlay/41b64b3628739daba4cbb6d77a4ddfb145f6f7699875e23275a91c9be15386e3/merged 2024-09-10T09:52:52.339 INFO:teuthology.orchestra.run.smithi077.stdout:/var/lib/containers/storage/overlay/d17c0011112205a5a48f96177d3c97238119a6b77a5000947d0b6a223e1cdf9c/merged 2024-09-10T09:52:52.339 INFO:teuthology.orchestra.run.smithi077.stdout:/var/lib/containers/storage/overlay/97e251b4f99a6a0c9c823978b8e51e9486fe8390170b8466d0f10444ed87dfff/merged 2024-09-10T09:52:52.340 INFO:teuthology.orchestra.run.smithi077.stdout:/var/lib/containers/storage/overlay/7ea00b20361f535ea8f0990ec8aa2e58ca01f1131bd42866386f40b226449824/merged 2024-09-10T09:52:52.340 INFO:teuthology.orchestra.run.smithi077.stdout:/var/lib/containers/storage/overlay/48a00f26b4ff0dcef267c91eb25c67e83b444bcf16a28ac29231e284da93d129/merged 2024-09-10T09:52:52.340 INFO:teuthology.orchestra.run.smithi077.stdout:/var/lib/containers/storage/overlay/a8b236d11f9808583b8b1278e30bbd7db4b27a9a8839b1e25e0d9775340a7600/merged 2024-09-10T09:52:52.340 INFO:teuthology.orchestra.run.smithi077.stdout:/var/lib/containers/storage/overlay/8215a331b16080617c90b5ada9cacfb91bd20f4445f610f539ae6343b6c2c878/merged 2024-09-10T09:52:52.340 INFO:teuthology.orchestra.run.smithi077.stdout:/var/lib/containers/storage/overlay/cd0edb7b4111edbb04b3238eae7be4e65b578ca12524ee453f78f3a4753b1844/merged 2024-09-10T09:52:52.340 INFO:teuthology.orchestra.run.smithi077.stdout:/run/netns 2024-09-10T09:52:52.340 INFO:teuthology.orchestra.run.smithi077.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T09:52:52.340 INFO:teuthology.orchestra.run.smithi077.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T09:52:52.341 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:52:52.341 DEBUG:teuthology.orchestra.run.smithi077:> ls /sys/fs/fuse/connections 2024-09-10T09:52:52.417 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-10T09:52:52.417 DEBUG:teuthology.orchestra.run.smithi077:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-09-10T09:52:52.461 DEBUG:teuthology.orchestra.run.smithi077:> sudo modprobe fuse 2024-09-10T09:52:52.518 DEBUG:teuthology.orchestra.run.smithi077:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T09:52:52.555 INFO:teuthology.orchestra.run.smithi077.stdout:/proc 2024-09-10T09:52:52.555 INFO:teuthology.orchestra.run.smithi077.stdout:/sys 2024-09-10T09:52:52.555 INFO:teuthology.orchestra.run.smithi077.stdout:/dev 2024-09-10T09:52:52.555 INFO:teuthology.orchestra.run.smithi077.stdout:/sys/kernel/security 2024-09-10T09:52:52.555 INFO:teuthology.orchestra.run.smithi077.stdout:/dev/shm 2024-09-10T09:52:52.555 INFO:teuthology.orchestra.run.smithi077.stdout:/dev/pts 2024-09-10T09:52:52.555 INFO:teuthology.orchestra.run.smithi077.stdout:/run 2024-09-10T09:52:52.555 INFO:teuthology.orchestra.run.smithi077.stdout:/sys/fs/cgroup 2024-09-10T09:52:52.555 INFO:teuthology.orchestra.run.smithi077.stdout:/sys/fs/pstore 2024-09-10T09:52:52.555 INFO:teuthology.orchestra.run.smithi077.stdout:/sys/fs/bpf 2024-09-10T09:52:52.555 INFO:teuthology.orchestra.run.smithi077.stdout:/ 2024-09-10T09:52:52.555 INFO:teuthology.orchestra.run.smithi077.stdout:/sys/fs/selinux 2024-09-10T09:52:52.556 INFO:teuthology.orchestra.run.smithi077.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T09:52:52.556 INFO:teuthology.orchestra.run.smithi077.stdout:/dev/hugepages 2024-09-10T09:52:52.556 INFO:teuthology.orchestra.run.smithi077.stdout:/dev/mqueue 2024-09-10T09:52:52.556 INFO:teuthology.orchestra.run.smithi077.stdout:/sys/kernel/debug 2024-09-10T09:52:52.556 INFO:teuthology.orchestra.run.smithi077.stdout:/sys/kernel/tracing 2024-09-10T09:52:52.556 INFO:teuthology.orchestra.run.smithi077.stdout:/sys/fs/fuse/connections 2024-09-10T09:52:52.556 INFO:teuthology.orchestra.run.smithi077.stdout:/sys/kernel/config 2024-09-10T09:52:52.556 INFO:teuthology.orchestra.run.smithi077.stdout:/run/credentials/systemd-sysctl.service 2024-09-10T09:52:52.556 INFO:teuthology.orchestra.run.smithi077.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-10T09:52:52.556 INFO:teuthology.orchestra.run.smithi077.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-10T09:52:52.556 INFO:teuthology.orchestra.run.smithi077.stdout:/var/lib/nfs/rpc_pipefs 2024-09-10T09:52:52.556 INFO:teuthology.orchestra.run.smithi077.stdout:/run/user/1000 2024-09-10T09:52:52.556 INFO:teuthology.orchestra.run.smithi077.stdout:/var/lib/ceph 2024-09-10T09:52:52.557 INFO:teuthology.orchestra.run.smithi077.stdout:/var/lib/containers/storage/overlay 2024-09-10T09:52:52.557 INFO:teuthology.orchestra.run.smithi077.stdout:/var/lib/containers/storage/overlay/35721519d96d163578ce1de5316d869b3092221fa3253a979f83484306626b78/merged 2024-09-10T09:52:52.557 INFO:teuthology.orchestra.run.smithi077.stdout:/var/lib/containers/storage/overlay/5785a63d1b9c229507112ef2753db50a385d6252c16ff542681164cb671a5707/merged 2024-09-10T09:52:52.557 INFO:teuthology.orchestra.run.smithi077.stdout:/run/user/0 2024-09-10T09:52:52.557 INFO:teuthology.orchestra.run.smithi077.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T09:52:52.557 INFO:teuthology.orchestra.run.smithi077.stdout:/var/lib/containers/storage/overlay/41b64b3628739daba4cbb6d77a4ddfb145f6f7699875e23275a91c9be15386e3/merged 2024-09-10T09:52:52.557 INFO:teuthology.orchestra.run.smithi077.stdout:/var/lib/containers/storage/overlay/d17c0011112205a5a48f96177d3c97238119a6b77a5000947d0b6a223e1cdf9c/merged 2024-09-10T09:52:52.557 INFO:teuthology.orchestra.run.smithi077.stdout:/var/lib/containers/storage/overlay/97e251b4f99a6a0c9c823978b8e51e9486fe8390170b8466d0f10444ed87dfff/merged 2024-09-10T09:52:52.557 INFO:teuthology.orchestra.run.smithi077.stdout:/var/lib/containers/storage/overlay/7ea00b20361f535ea8f0990ec8aa2e58ca01f1131bd42866386f40b226449824/merged 2024-09-10T09:52:52.557 INFO:teuthology.orchestra.run.smithi077.stdout:/var/lib/containers/storage/overlay/48a00f26b4ff0dcef267c91eb25c67e83b444bcf16a28ac29231e284da93d129/merged 2024-09-10T09:52:52.557 INFO:teuthology.orchestra.run.smithi077.stdout:/var/lib/containers/storage/overlay/a8b236d11f9808583b8b1278e30bbd7db4b27a9a8839b1e25e0d9775340a7600/merged 2024-09-10T09:52:52.557 INFO:teuthology.orchestra.run.smithi077.stdout:/var/lib/containers/storage/overlay/8215a331b16080617c90b5ada9cacfb91bd20f4445f610f539ae6343b6c2c878/merged 2024-09-10T09:52:52.557 INFO:teuthology.orchestra.run.smithi077.stdout:/var/lib/containers/storage/overlay/cd0edb7b4111edbb04b3238eae7be4e65b578ca12524ee453f78f3a4753b1844/merged 2024-09-10T09:52:52.557 INFO:teuthology.orchestra.run.smithi077.stdout:/run/netns 2024-09-10T09:52:52.558 INFO:teuthology.orchestra.run.smithi077.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T09:52:52.558 INFO:teuthology.orchestra.run.smithi077.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T09:52:52.558 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:52:52.559 DEBUG:teuthology.orchestra.run.smithi077:> ls /sys/fs/fuse/connections 2024-09-10T09:52:52.629 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi077.stderr:2024-09-10T09:52:52.627+0000 7f380530d480 -1 init, newargv = 0x5620364e73f0 newargc=15 2024-09-10T09:52:52.629 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi077.stderr:ceph-fuse[74050]: starting ceph client 2024-09-10T09:52:52.639 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi077.stderr:ceph-fuse[74050]: starting fuse 2024-09-10T09:52:53.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:52 smithi134 ceph-mon[35674]: pgmap v318: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:52:53.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:52 smithi120 ceph-mon[34770]: pgmap v318: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:52:53.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:52 smithi077 ceph-mon[30319]: pgmap v318: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:52:53.738 DEBUG:teuthology.orchestra.run.smithi077:> sudo modprobe fuse 2024-09-10T09:52:53.803 DEBUG:teuthology.orchestra.run.smithi077:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T09:52:53.841 INFO:teuthology.orchestra.run.smithi077.stdout:/proc 2024-09-10T09:52:53.842 INFO:teuthology.orchestra.run.smithi077.stdout:/sys 2024-09-10T09:52:53.842 INFO:teuthology.orchestra.run.smithi077.stdout:/dev 2024-09-10T09:52:53.842 INFO:teuthology.orchestra.run.smithi077.stdout:/sys/kernel/security 2024-09-10T09:52:53.842 INFO:teuthology.orchestra.run.smithi077.stdout:/dev/shm 2024-09-10T09:52:53.842 INFO:teuthology.orchestra.run.smithi077.stdout:/dev/pts 2024-09-10T09:52:53.842 INFO:teuthology.orchestra.run.smithi077.stdout:/run 2024-09-10T09:52:53.842 INFO:teuthology.orchestra.run.smithi077.stdout:/sys/fs/cgroup 2024-09-10T09:52:53.842 INFO:teuthology.orchestra.run.smithi077.stdout:/sys/fs/pstore 2024-09-10T09:52:53.842 INFO:teuthology.orchestra.run.smithi077.stdout:/sys/fs/bpf 2024-09-10T09:52:53.842 INFO:teuthology.orchestra.run.smithi077.stdout:/ 2024-09-10T09:52:53.842 INFO:teuthology.orchestra.run.smithi077.stdout:/sys/fs/selinux 2024-09-10T09:52:53.842 INFO:teuthology.orchestra.run.smithi077.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T09:52:53.843 INFO:teuthology.orchestra.run.smithi077.stdout:/dev/hugepages 2024-09-10T09:52:53.843 INFO:teuthology.orchestra.run.smithi077.stdout:/dev/mqueue 2024-09-10T09:52:53.843 INFO:teuthology.orchestra.run.smithi077.stdout:/sys/kernel/debug 2024-09-10T09:52:53.843 INFO:teuthology.orchestra.run.smithi077.stdout:/sys/kernel/tracing 2024-09-10T09:52:53.843 INFO:teuthology.orchestra.run.smithi077.stdout:/sys/fs/fuse/connections 2024-09-10T09:52:53.843 INFO:teuthology.orchestra.run.smithi077.stdout:/sys/kernel/config 2024-09-10T09:52:53.843 INFO:teuthology.orchestra.run.smithi077.stdout:/run/credentials/systemd-sysctl.service 2024-09-10T09:52:53.843 INFO:teuthology.orchestra.run.smithi077.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-10T09:52:53.843 INFO:teuthology.orchestra.run.smithi077.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-10T09:52:53.843 INFO:teuthology.orchestra.run.smithi077.stdout:/var/lib/nfs/rpc_pipefs 2024-09-10T09:52:53.843 INFO:teuthology.orchestra.run.smithi077.stdout:/run/user/1000 2024-09-10T09:52:53.843 INFO:teuthology.orchestra.run.smithi077.stdout:/var/lib/ceph 2024-09-10T09:52:53.843 INFO:teuthology.orchestra.run.smithi077.stdout:/var/lib/containers/storage/overlay 2024-09-10T09:52:53.843 INFO:teuthology.orchestra.run.smithi077.stdout:/var/lib/containers/storage/overlay/35721519d96d163578ce1de5316d869b3092221fa3253a979f83484306626b78/merged 2024-09-10T09:52:53.843 INFO:teuthology.orchestra.run.smithi077.stdout:/var/lib/containers/storage/overlay/5785a63d1b9c229507112ef2753db50a385d6252c16ff542681164cb671a5707/merged 2024-09-10T09:52:53.844 INFO:teuthology.orchestra.run.smithi077.stdout:/run/user/0 2024-09-10T09:52:53.844 INFO:teuthology.orchestra.run.smithi077.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T09:52:53.844 INFO:teuthology.orchestra.run.smithi077.stdout:/var/lib/containers/storage/overlay/41b64b3628739daba4cbb6d77a4ddfb145f6f7699875e23275a91c9be15386e3/merged 2024-09-10T09:52:53.844 INFO:teuthology.orchestra.run.smithi077.stdout:/var/lib/containers/storage/overlay/d17c0011112205a5a48f96177d3c97238119a6b77a5000947d0b6a223e1cdf9c/merged 2024-09-10T09:52:53.844 INFO:teuthology.orchestra.run.smithi077.stdout:/var/lib/containers/storage/overlay/97e251b4f99a6a0c9c823978b8e51e9486fe8390170b8466d0f10444ed87dfff/merged 2024-09-10T09:52:53.844 INFO:teuthology.orchestra.run.smithi077.stdout:/var/lib/containers/storage/overlay/7ea00b20361f535ea8f0990ec8aa2e58ca01f1131bd42866386f40b226449824/merged 2024-09-10T09:52:53.844 INFO:teuthology.orchestra.run.smithi077.stdout:/var/lib/containers/storage/overlay/48a00f26b4ff0dcef267c91eb25c67e83b444bcf16a28ac29231e284da93d129/merged 2024-09-10T09:52:53.844 INFO:teuthology.orchestra.run.smithi077.stdout:/var/lib/containers/storage/overlay/a8b236d11f9808583b8b1278e30bbd7db4b27a9a8839b1e25e0d9775340a7600/merged 2024-09-10T09:52:53.844 INFO:teuthology.orchestra.run.smithi077.stdout:/var/lib/containers/storage/overlay/8215a331b16080617c90b5ada9cacfb91bd20f4445f610f539ae6343b6c2c878/merged 2024-09-10T09:52:53.844 INFO:teuthology.orchestra.run.smithi077.stdout:/var/lib/containers/storage/overlay/cd0edb7b4111edbb04b3238eae7be4e65b578ca12524ee453f78f3a4753b1844/merged 2024-09-10T09:52:53.844 INFO:teuthology.orchestra.run.smithi077.stdout:/run/netns 2024-09-10T09:52:53.844 INFO:teuthology.orchestra.run.smithi077.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T09:52:53.844 INFO:teuthology.orchestra.run.smithi077.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T09:52:53.845 INFO:teuthology.orchestra.run.smithi077.stdout:/home/ubuntu/cephtest/mnt.admin 2024-09-10T09:52:53.845 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:52:53.845 DEBUG:teuthology.orchestra.run.smithi077:> ls /sys/fs/fuse/connections 2024-09-10T09:52:53.922 INFO:teuthology.orchestra.run.smithi077.stdout:120 2024-09-10T09:52:53.923 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [120] 2024-09-10T09:52:53.923 DEBUG:teuthology.orchestra.run.smithi077:> sudo stdin-killer -- python3 -c ' 2024-09-10T09:52:53.923 DEBUG:teuthology.orchestra.run.smithi077:> import glob 2024-09-10T09:52:53.923 DEBUG:teuthology.orchestra.run.smithi077:> import re 2024-09-10T09:52:53.923 DEBUG:teuthology.orchestra.run.smithi077:> import os 2024-09-10T09:52:53.923 DEBUG:teuthology.orchestra.run.smithi077:> import subprocess 2024-09-10T09:52:53.923 DEBUG:teuthology.orchestra.run.smithi077:> 2024-09-10T09:52:53.923 DEBUG:teuthology.orchestra.run.smithi077:> def _find_admin_socket(client_name): 2024-09-10T09:52:53.923 DEBUG:teuthology.orchestra.run.smithi077:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-09-10T09:52:53.924 DEBUG:teuthology.orchestra.run.smithi077:> files = glob.glob(asok_path) 2024-09-10T09:52:53.924 DEBUG:teuthology.orchestra.run.smithi077:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-09-10T09:52:53.924 DEBUG:teuthology.orchestra.run.smithi077:> 2024-09-10T09:52:53.924 DEBUG:teuthology.orchestra.run.smithi077:> # Given a non-glob path, it better be there 2024-09-10T09:52:53.924 DEBUG:teuthology.orchestra.run.smithi077:> if "*" not in asok_path: 2024-09-10T09:52:53.924 DEBUG:teuthology.orchestra.run.smithi077:> assert(len(files) == 1) 2024-09-10T09:52:53.924 DEBUG:teuthology.orchestra.run.smithi077:> return files[0] 2024-09-10T09:52:53.924 DEBUG:teuthology.orchestra.run.smithi077:> 2024-09-10T09:52:53.924 DEBUG:teuthology.orchestra.run.smithi077:> for f in files: 2024-09-10T09:52:53.924 DEBUG:teuthology.orchestra.run.smithi077:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-10T09:52:53.924 DEBUG:teuthology.orchestra.run.smithi077:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-10T09:52:53.924 DEBUG:teuthology.orchestra.run.smithi077:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-10T09:52:53.924 DEBUG:teuthology.orchestra.run.smithi077:> contents = proc_f.read() 2024-09-10T09:52:53.924 DEBUG:teuthology.orchestra.run.smithi077:> if mountpoint in contents: 2024-09-10T09:52:53.925 DEBUG:teuthology.orchestra.run.smithi077:> return f 2024-09-10T09:52:53.925 DEBUG:teuthology.orchestra.run.smithi077:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-10T09:52:53.925 DEBUG:teuthology.orchestra.run.smithi077:> 2024-09-10T09:52:53.925 DEBUG:teuthology.orchestra.run.smithi077:> print(_find_admin_socket("client.admin")) 2024-09-10T09:52:53.925 DEBUG:teuthology.orchestra.run.smithi077:> ' 2024-09-10T09:52:54.084 INFO:teuthology.orchestra.run.smithi077.stdout:/var/run/ceph/ceph-client.admin.74050.asok 2024-09-10T09:52:54.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T09:52:54.098 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.74050.asok 2024-09-10T09:52:54.098 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:52:54.099 DEBUG:teuthology.orchestra.run.smithi077:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.74050.asok status 2024-09-10T09:52:54.280 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-10T09:52:54.280 INFO:teuthology.orchestra.run.smithi077.stdout: "metadata": { 2024-09-10T09:52:54.280 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_sha1": "1cf819b7212a6048e92fc78c4eff963344f6d56b", 2024-09-10T09:52:54.280 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_version": "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)", 2024-09-10T09:52:54.280 INFO:teuthology.orchestra.run.smithi077.stdout: "entity_id": "admin", 2024-09-10T09:52:54.280 INFO:teuthology.orchestra.run.smithi077.stdout: "hostname": "smithi077", 2024-09-10T09:52:54.280 INFO:teuthology.orchestra.run.smithi077.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-09-10T09:52:54.280 INFO:teuthology.orchestra.run.smithi077.stdout: "pid": "74050", 2024-09-10T09:52:54.280 INFO:teuthology.orchestra.run.smithi077.stdout: "root": "/" 2024-09-10T09:52:54.281 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T09:52:54.281 INFO:teuthology.orchestra.run.smithi077.stdout: "dentry_count": 0, 2024-09-10T09:52:54.281 INFO:teuthology.orchestra.run.smithi077.stdout: "dentry_pinned_count": 0, 2024-09-10T09:52:54.281 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 14619, 2024-09-10T09:52:54.281 INFO:teuthology.orchestra.run.smithi077.stdout: "inst": { 2024-09-10T09:52:54.281 INFO:teuthology.orchestra.run.smithi077.stdout: "name": { 2024-09-10T09:52:54.281 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "client", 2024-09-10T09:52:54.281 INFO:teuthology.orchestra.run.smithi077.stdout: "num": 14619 2024-09-10T09:52:54.281 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T09:52:54.281 INFO:teuthology.orchestra.run.smithi077.stdout: "addr": { 2024-09-10T09:52:54.281 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "v1", 2024-09-10T09:52:54.281 INFO:teuthology.orchestra.run.smithi077.stdout: "addr": "192.168.0.1:0", 2024-09-10T09:52:54.281 INFO:teuthology.orchestra.run.smithi077.stdout: "nonce": 1146469536 2024-09-10T09:52:54.281 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T09:52:54.282 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T09:52:54.282 INFO:teuthology.orchestra.run.smithi077.stdout: "addr": { 2024-09-10T09:52:54.282 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "v1", 2024-09-10T09:52:54.282 INFO:teuthology.orchestra.run.smithi077.stdout: "addr": "192.168.0.1:0", 2024-09-10T09:52:54.282 INFO:teuthology.orchestra.run.smithi077.stdout: "nonce": 1146469536 2024-09-10T09:52:54.282 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T09:52:54.282 INFO:teuthology.orchestra.run.smithi077.stdout: "inst_str": "client.14619 192.168.0.1:0/1146469536", 2024-09-10T09:52:54.282 INFO:teuthology.orchestra.run.smithi077.stdout: "addr_str": "192.168.0.1:0/1146469536", 2024-09-10T09:52:54.282 INFO:teuthology.orchestra.run.smithi077.stdout: "inode_count": 1, 2024-09-10T09:52:54.282 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_epoch": 17, 2024-09-10T09:52:54.282 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_epoch": 92, 2024-09-10T09:52:54.282 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_epoch_barrier": 0, 2024-09-10T09:52:54.282 INFO:teuthology.orchestra.run.smithi077.stdout: "blocklisted": false, 2024-09-10T09:52:54.282 INFO:teuthology.orchestra.run.smithi077.stdout: "fs_name": "cephfs" 2024-09-10T09:52:54.283 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-10T09:52:54.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:52:54.292 DEBUG:teuthology.orchestra.run.smithi077:> stat --file-system '--printf=%T 2024-09-10T09:52:54.292 DEBUG:teuthology.orchestra.run.smithi077:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-09-10T09:52:54.331 INFO:teuthology.orchestra.run.smithi077.stdout:fuseblk 2024-09-10T09:52:54.331 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-09-10T09:52:54.331 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:52:54.331 DEBUG:teuthology.orchestra.run.smithi077:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-09-10T09:52:54.429 DEBUG:teuthology.orchestra.run.smithi077:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-09-10T09:52:54.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:54 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T09:52:54.535 INFO:teuthology.orchestra.run.smithi077.stdout:# file: . 2024-09-10T09:52:54.535 INFO:teuthology.orchestra.run.smithi077.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-09-10T09:52:54.535 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:52:54.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T09:52:54.543 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-09-10T09:52:54.543 DEBUG:teuthology.orchestra.run.smithi077:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T09:52:54.579 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph osd blocklist ls 2024-09-10T09:52:54.990 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:52:55.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:54 smithi077 ceph-mon[30319]: pgmap v319: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-09-10T09:52:55.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:54 smithi134 ceph-mon[35674]: pgmap v319: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-09-10T09:52:55.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:54 smithi120 ceph-mon[34770]: pgmap v319: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-09-10T09:52:55.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:55.529+0000 7fb928400640 1 -- 172.21.15.77:0/3563328106 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb920112a50 msgr2=0x7fb920114e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:55.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:55.529+0000 7fb928400640 1 --2- 172.21.15.77:0/3563328106 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb920112a50 0x7fb920114e40 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb918009ca0 tx=0x7fb91802a8c0 comp rx=0 tx=0).stop 2024-09-10T09:52:55.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:55.529+0000 7fb928400640 1 -- 172.21.15.77:0/3563328106 shutdown_connections 2024-09-10T09:52:55.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:55.529+0000 7fb928400640 1 --2- 172.21.15.77:0/3563328106 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb920112a50 0x7fb920114e40 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:55.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:55.529+0000 7fb928400640 1 --2- 172.21.15.77:0/3563328106 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb920077040 0x7fb920112510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:55.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:55.529+0000 7fb928400640 1 --2- 172.21.15.77:0/3563328106 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb920076720 0x7fb920076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:55.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:55.529+0000 7fb928400640 1 -- 172.21.15.77:0/3563328106 >> 172.21.15.77:0/3563328106 conn(0x7fb9200ffa50 msgr2=0x7fb920101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:55.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:55.530+0000 7fb928400640 1 -- 172.21.15.77:0/3563328106 shutdown_connections 2024-09-10T09:52:55.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:55.530+0000 7fb928400640 1 -- 172.21.15.77:0/3563328106 wait complete. 2024-09-10T09:52:55.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:55.531+0000 7fb928400640 1 Processor -- start 2024-09-10T09:52:55.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:55.531+0000 7fb928400640 1 -- start start 2024-09-10T09:52:55.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:55.531+0000 7fb928400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb920076720 0x7fb9201a4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:55.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:55.531+0000 7fb928400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb920077040 0x7fb9201a4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:55.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:55.531+0000 7fb928400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb920112a50 0x7fb92019e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:55.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:55.531+0000 7fb928400640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb9201185e0 con 0x7fb920077040 2024-09-10T09:52:55.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:55.531+0000 7fb928400640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb920118460 con 0x7fb920076720 2024-09-10T09:52:55.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:55.531+0000 7fb928400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb920118760 con 0x7fb920112a50 2024-09-10T09:52:55.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:55.531+0000 7fb925000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb920077040 0x7fb9201a4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:55.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:55.532+0000 7fb925000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb920077040 0x7fb9201a4c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:39788/0 (socket says 172.21.15.77:39788) 2024-09-10T09:52:55.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:55.532+0000 7fb926400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb920112a50 0x7fb92019e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:55.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:55.532+0000 7fb925000640 1 -- 172.21.15.77:0/3789491293 learned_addr learned my addr 172.21.15.77:0/3789491293 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:52:55.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:55.532+0000 7fb925a00640 1 --2- 172.21.15.77:0/3789491293 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb920076720 0x7fb9201a4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:55.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:55.532+0000 7fb926400640 1 -- 172.21.15.77:0/3789491293 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb920076720 msgr2=0x7fb9201a4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:55.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:55.532+0000 7fb926400640 1 --2- 172.21.15.77:0/3789491293 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb920076720 0x7fb9201a4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:55.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:55.532+0000 7fb926400640 1 -- 172.21.15.77:0/3789491293 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb920077040 msgr2=0x7fb9201a4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:55.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:55.532+0000 7fb926400640 1 --2- 172.21.15.77:0/3789491293 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb920077040 0x7fb9201a4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:55.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:55.532+0000 7fb926400640 1 -- 172.21.15.77:0/3789491293 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb92019f020 con 0x7fb920112a50 2024-09-10T09:52:55.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:55.532+0000 7fb926400640 1 --2- 172.21.15.77:0/3789491293 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb920112a50 0x7fb92019e8f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb91802aeb0 tx=0x7fb918003f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:55.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:55.532+0000 7fb91e000640 1 -- 172.21.15.77:0/3789491293 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb918031650 con 0x7fb920112a50 2024-09-10T09:52:55.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:55.533+0000 7fb91e000640 1 -- 172.21.15.77:0/3789491293 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9180317f0 con 0x7fb920112a50 2024-09-10T09:52:55.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:55.533+0000 7fb91e000640 1 -- 172.21.15.77:0/3789491293 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb91802fa30 con 0x7fb920112a50 2024-09-10T09:52:55.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:55.533+0000 7fb928400640 1 -- 172.21.15.77:0/3789491293 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb92019f2b0 con 0x7fb920112a50 2024-09-10T09:52:55.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:55.533+0000 7fb928400640 1 -- 172.21.15.77:0/3789491293 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9201ab600 con 0x7fb920112a50 2024-09-10T09:52:55.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:55.534+0000 7fb928400640 1 -- 172.21.15.77:0/3789491293 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb920076b00 con 0x7fb920112a50 2024-09-10T09:52:55.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:55.536+0000 7fb91e000640 1 -- 172.21.15.77:0/3789491293 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fb91802fbd0 con 0x7fb920112a50 2024-09-10T09:52:55.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:55.537+0000 7fb91e000640 1 --2- 172.21.15.77:0/3789491293 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fb9080a2c00 0x7fb9080a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:55.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:55.537+0000 7fb91e000640 1 -- 172.21.15.77:0/3789491293 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7fb9180f06a0 con 0x7fb920112a50 2024-09-10T09:52:55.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:55.537+0000 7fb925a00640 1 --2- 172.21.15.77:0/3789491293 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fb9080a2c00 0x7fb9080a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:55.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:55.538+0000 7fb925a00640 1 --2- 172.21.15.77:0/3789491293 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fb9080a2c00 0x7fb9080a50c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb92019fa30 tx=0x7fb91400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:55.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:55.539+0000 7fb91e000640 1 -- 172.21.15.77:0/3789491293 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb9180bd360 con 0x7fb920112a50 2024-09-10T09:52:55.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:55.728+0000 7fb928400640 1 -- 172.21.15.77:0/3789491293 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb92010dbe0 con 0x7fb920112a50 2024-09-10T09:52:55.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:55.729+0000 7fb91e000640 1 -- 172.21.15.77:0/3789491293 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v92) ==== 81+0+590 (secure 0 0 0) 0x7fb9180319a0 con 0x7fb920112a50 2024-09-10T09:52:55.730 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:0/1697982655 2024-09-11T09:44:59.575663+0000 2024-09-10T09:52:55.730 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:6801/1527070442 2024-09-11T09:44:59.575663+0000 2024-09-10T09:52:55.730 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:0/3146688196 2024-09-11T09:44:32.185342+0000 2024-09-10T09:52:55.730 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:0/371773177 2024-09-11T09:44:59.575663+0000 2024-09-10T09:52:55.730 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:0/1925456910 2024-09-11T09:44:59.575663+0000 2024-09-10T09:52:55.730 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:0/138545832 2024-09-11T09:44:32.185342+0000 2024-09-10T09:52:55.730 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:6801/3235900503 2024-09-11T09:44:32.185342+0000 2024-09-10T09:52:55.730 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:6800/1527070442 2024-09-11T09:44:59.575663+0000 2024-09-10T09:52:55.730 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:6800/3235900503 2024-09-11T09:44:32.185342+0000 2024-09-10T09:52:55.730 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:0/1647092548 2024-09-11T09:44:32.185342+0000 2024-09-10T09:52:55.731 INFO:teuthology.orchestra.run.smithi077.stderr:listed 10 entries 2024-09-10T09:52:55.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:55.731+0000 7fb928400640 1 -- 172.21.15.77:0/3789491293 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fb9080a2c00 msgr2=0x7fb9080a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:55.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:55.731+0000 7fb928400640 1 --2- 172.21.15.77:0/3789491293 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fb9080a2c00 0x7fb9080a50c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb92019fa30 tx=0x7fb91400b410 comp rx=0 tx=0).stop 2024-09-10T09:52:55.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:55.731+0000 7fb928400640 1 -- 172.21.15.77:0/3789491293 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb920112a50 msgr2=0x7fb92019e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:55.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:55.731+0000 7fb928400640 1 --2- 172.21.15.77:0/3789491293 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb920112a50 0x7fb92019e8f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb91802aeb0 tx=0x7fb918003f10 comp rx=0 tx=0).stop 2024-09-10T09:52:55.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:55.731+0000 7fb928400640 1 -- 172.21.15.77:0/3789491293 shutdown_connections 2024-09-10T09:52:55.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:55.731+0000 7fb928400640 1 --2- 172.21.15.77:0/3789491293 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fb9080a2c00 0x7fb9080a50c0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:55.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:55.731+0000 7fb928400640 1 --2- 172.21.15.77:0/3789491293 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb920112a50 0x7fb92019e8f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:55.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:55.731+0000 7fb928400640 1 --2- 172.21.15.77:0/3789491293 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb920077040 0x7fb9201a4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:55.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:55.731+0000 7fb928400640 1 --2- 172.21.15.77:0/3789491293 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb920076720 0x7fb9201a4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:55.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:55.731+0000 7fb928400640 1 -- 172.21.15.77:0/3789491293 >> 172.21.15.77:0/3789491293 conn(0x7fb9200ffa50 msgr2=0x7fb920113650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:55.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:55.731+0000 7fb928400640 1 -- 172.21.15.77:0/3789491293 shutdown_connections 2024-09-10T09:52:55.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:55.731+0000 7fb928400640 1 -- 172.21.15.77:0/3789491293 wait complete. 2024-09-10T09:52:56.084 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:55 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/3789491293' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T09:52:56.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:55 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/3789491293' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T09:52:56.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:55 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/3789491293' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T09:52:56.604 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-09-10T09:52:56.605 DEBUG:teuthology.orchestra.run.smithi077:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T09:52:56.642 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph osd blocklist ls 2024-09-10T09:52:56.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:56 smithi077 ceph-mon[30319]: pgmap v320: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-09-10T09:52:57.058 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:52:57.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:56 smithi134 ceph-mon[35674]: pgmap v320: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-09-10T09:52:57.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:56 smithi120 ceph-mon[34770]: pgmap v320: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-09-10T09:52:57.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:57.797+0000 7fca7b800640 1 -- 172.21.15.77:0/4263887572 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca74074590 msgr2=0x7fca74074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:57.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:57.797+0000 7fca7b800640 1 --2- 172.21.15.77:0/4263887572 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca74074590 0x7fca74074970 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fca68009900 tx=0x7fca6802a480 comp rx=0 tx=0).stop 2024-09-10T09:52:57.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:57.797+0000 7fca7b800640 1 -- 172.21.15.77:0/4263887572 shutdown_connections 2024-09-10T09:52:57.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:57.797+0000 7fca7b800640 1 --2- 172.21.15.77:0/4263887572 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca74112b90 0x7fca74114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:57.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:57.797+0000 7fca7b800640 1 --2- 172.21.15.77:0/4263887572 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fca74074eb0 0x7fca74112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:57.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:57.797+0000 7fca7b800640 1 --2- 172.21.15.77:0/4263887572 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca74074590 0x7fca74074970 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:57.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:57.797+0000 7fca7b800640 1 -- 172.21.15.77:0/4263887572 >> 172.21.15.77:0/4263887572 conn(0x7fca740fd8c0 msgr2=0x7fca740ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:57.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:57.797+0000 7fca7b800640 1 -- 172.21.15.77:0/4263887572 shutdown_connections 2024-09-10T09:52:57.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:57.797+0000 7fca7b800640 1 -- 172.21.15.77:0/4263887572 wait complete. 2024-09-10T09:52:57.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:57.798+0000 7fca7b800640 1 Processor -- start 2024-09-10T09:52:57.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:57.798+0000 7fca7b800640 1 -- start start 2024-09-10T09:52:57.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:57.798+0000 7fca7b800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca74074590 0x7fca741a4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:57.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:57.798+0000 7fca7b800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fca74074eb0 0x7fca741a4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:57.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:57.798+0000 7fca7b800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca74112b90 0x7fca7419e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:57.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:57.798+0000 7fca7b800640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fca74118570 con 0x7fca74074590 2024-09-10T09:52:57.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:57.798+0000 7fca7b800640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fca741183f0 con 0x7fca74074eb0 2024-09-10T09:52:57.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:57.798+0000 7fca7b800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fca741186f0 con 0x7fca74112b90 2024-09-10T09:52:57.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:57.799+0000 7fca78e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca74074590 0x7fca741a4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:57.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:57.799+0000 7fca78e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca74074590 0x7fca741a4700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:39808/0 (socket says 172.21.15.77:39808) 2024-09-10T09:52:57.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:57.799+0000 7fca78e00640 1 -- 172.21.15.77:0/3483297184 learned_addr learned my addr 172.21.15.77:0/3483297184 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:52:57.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:57.799+0000 7fca79800640 1 --2- 172.21.15.77:0/3483297184 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca74112b90 0x7fca7419e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:57.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:57.799+0000 7fca73e00640 1 --2- 172.21.15.77:0/3483297184 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fca74074eb0 0x7fca741a4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:57.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:57.799+0000 7fca78e00640 1 -- 172.21.15.77:0/3483297184 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca74112b90 msgr2=0x7fca7419e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:57.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:57.799+0000 7fca78e00640 1 --2- 172.21.15.77:0/3483297184 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca74112b90 0x7fca7419e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:57.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:57.799+0000 7fca78e00640 1 -- 172.21.15.77:0/3483297184 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fca74074eb0 msgr2=0x7fca741a4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:57.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:57.799+0000 7fca78e00640 1 --2- 172.21.15.77:0/3483297184 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fca74074eb0 0x7fca741a4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:57.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:57.799+0000 7fca78e00640 1 -- 172.21.15.77:0/3483297184 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca7419eff0 con 0x7fca74074590 2024-09-10T09:52:57.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:57.799+0000 7fca78e00640 1 --2- 172.21.15.77:0/3483297184 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca74074590 0x7fca741a4700 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fca68006010 tx=0x7fca680042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:57.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:57.800+0000 7fca71600640 1 -- 172.21.15.77:0/3483297184 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca6803f030 con 0x7fca74074590 2024-09-10T09:52:57.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:57.800+0000 7fca71600640 1 -- 172.21.15.77:0/3483297184 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca68003ce0 con 0x7fca74074590 2024-09-10T09:52:57.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:57.800+0000 7fca7b800640 1 -- 172.21.15.77:0/3483297184 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca7419f280 con 0x7fca74074590 2024-09-10T09:52:57.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:57.800+0000 7fca71600640 1 -- 172.21.15.77:0/3483297184 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca68035070 con 0x7fca74074590 2024-09-10T09:52:57.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:57.800+0000 7fca7b800640 1 -- 172.21.15.77:0/3483297184 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca741ab5d0 con 0x7fca74074590 2024-09-10T09:52:57.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:57.801+0000 7fca7b800640 1 -- 172.21.15.77:0/3483297184 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca74074970 con 0x7fca74074590 2024-09-10T09:52:57.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:57.803+0000 7fca71600640 1 -- 172.21.15.77:0/3483297184 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fca68042030 con 0x7fca74074590 2024-09-10T09:52:57.805 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:57.804+0000 7fca71600640 1 --2- 172.21.15.77:0/3483297184 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fca5c0a2b80 0x7fca5c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:57.805 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:57.804+0000 7fca71600640 1 -- 172.21.15.77:0/3483297184 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7fca680ef9e0 con 0x7fca74074590 2024-09-10T09:52:57.805 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:57.805+0000 7fca73e00640 1 --2- 172.21.15.77:0/3483297184 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fca5c0a2b80 0x7fca5c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:57.806 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:57.805+0000 7fca73e00640 1 --2- 172.21.15.77:0/3483297184 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fca5c0a2b80 0x7fca5c0a5040 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fca74114c10 tx=0x7fca6000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:57.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:57.807+0000 7fca71600640 1 -- 172.21.15.77:0/3483297184 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fca680bc110 con 0x7fca74074590 2024-09-10T09:52:57.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:57.994+0000 7fca7b800640 1 -- 172.21.15.77:0/3483297184 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fca741ab7d0 con 0x7fca74074590 2024-09-10T09:52:57.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:57.994+0000 7fca71600640 1 -- 172.21.15.77:0/3483297184 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v92) ==== 81+0+590 (secure 0 0 0) 0x7fca680c0410 con 0x7fca74074590 2024-09-10T09:52:57.995 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:0/1697982655 2024-09-11T09:44:59.575663+0000 2024-09-10T09:52:57.995 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:6801/1527070442 2024-09-11T09:44:59.575663+0000 2024-09-10T09:52:57.995 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:0/3146688196 2024-09-11T09:44:32.185342+0000 2024-09-10T09:52:57.995 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:0/371773177 2024-09-11T09:44:59.575663+0000 2024-09-10T09:52:57.996 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:0/1925456910 2024-09-11T09:44:59.575663+0000 2024-09-10T09:52:57.996 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:0/138545832 2024-09-11T09:44:32.185342+0000 2024-09-10T09:52:57.996 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:6801/3235900503 2024-09-11T09:44:32.185342+0000 2024-09-10T09:52:57.996 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:6800/1527070442 2024-09-11T09:44:59.575663+0000 2024-09-10T09:52:57.996 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:6800/3235900503 2024-09-11T09:44:32.185342+0000 2024-09-10T09:52:57.996 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:0/1647092548 2024-09-11T09:44:32.185342+0000 2024-09-10T09:52:57.996 INFO:teuthology.orchestra.run.smithi077.stderr:listed 10 entries 2024-09-10T09:52:57.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:57.996+0000 7fca7b800640 1 -- 172.21.15.77:0/3483297184 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fca5c0a2b80 msgr2=0x7fca5c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:57.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:57.996+0000 7fca7b800640 1 --2- 172.21.15.77:0/3483297184 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fca5c0a2b80 0x7fca5c0a5040 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fca74114c10 tx=0x7fca6000b410 comp rx=0 tx=0).stop 2024-09-10T09:52:57.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:57.996+0000 7fca7b800640 1 -- 172.21.15.77:0/3483297184 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca74074590 msgr2=0x7fca741a4700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:57.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:57.996+0000 7fca7b800640 1 --2- 172.21.15.77:0/3483297184 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca74074590 0x7fca741a4700 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fca68006010 tx=0x7fca680042f0 comp rx=0 tx=0).stop 2024-09-10T09:52:57.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:57.996+0000 7fca7b800640 1 -- 172.21.15.77:0/3483297184 shutdown_connections 2024-09-10T09:52:57.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:57.996+0000 7fca7b800640 1 --2- 172.21.15.77:0/3483297184 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fca5c0a2b80 0x7fca5c0a5040 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:57.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:57.996+0000 7fca7b800640 1 --2- 172.21.15.77:0/3483297184 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca74112b90 0x7fca7419e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:57.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:57.996+0000 7fca7b800640 1 --2- 172.21.15.77:0/3483297184 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fca74074eb0 0x7fca741a4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:57.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:57.996+0000 7fca7b800640 1 --2- 172.21.15.77:0/3483297184 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca74074590 0x7fca741a4700 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:57.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:57.996+0000 7fca7b800640 1 -- 172.21.15.77:0/3483297184 >> 172.21.15.77:0/3483297184 conn(0x7fca740fd8c0 msgr2=0x7fca740ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:57.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:57.997+0000 7fca7b800640 1 -- 172.21.15.77:0/3483297184 shutdown_connections 2024-09-10T09:52:57.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:52:57.997+0000 7fca7b800640 1 -- 172.21.15.77:0/3483297184 wait complete. 2024-09-10T09:52:58.749 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi077.front.sepia.ceph.com... 2024-09-10T09:52:58.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:52:58.750 DEBUG:teuthology.orchestra.run.smithi077:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-09-10T09:52:58.806 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi077.stderr:ceph-fuse[74050]: fuse finished with error 0 and tester_r 0 2024-09-10T09:52:58.809 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T09:52:59.006 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:58 smithi077 ceph-mon[30319]: pgmap v321: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-09-10T09:52:59.006 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:58 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/3483297184' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T09:52:59.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:58 smithi134 ceph-mon[35674]: pgmap v321: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-09-10T09:52:59.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:58 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/3483297184' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T09:52:59.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:58 smithi120 ceph-mon[34770]: pgmap v321: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-09-10T09:52:59.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:58 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/3483297184' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T09:53:00.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:59 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.2", "id": [7, 11]}]: dispatch 2024-09-10T09:53:00.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:59 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-09-10T09:53:00.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:59 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-09-10T09:53:00.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:59 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]: dispatch 2024-09-10T09:53:00.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:59 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]: dispatch 2024-09-10T09:53:00.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:59 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-09-10T09:53:00.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:59 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [5, 8]}]: dispatch 2024-09-10T09:53:00.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:59 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 8]}]: dispatch 2024-09-10T09:53:00.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:59 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]: dispatch 2024-09-10T09:53:00.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:52:59 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]: dispatch 2024-09-10T09:53:00.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:59 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.2", "id": [7, 11]}]: dispatch 2024-09-10T09:53:00.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:59 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-09-10T09:53:00.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:59 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-09-10T09:53:00.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:59 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]: dispatch 2024-09-10T09:53:00.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:59 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]: dispatch 2024-09-10T09:53:00.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:59 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-09-10T09:53:00.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:59 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [5, 8]}]: dispatch 2024-09-10T09:53:00.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:59 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 8]}]: dispatch 2024-09-10T09:53:00.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:59 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]: dispatch 2024-09-10T09:53:00.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:52:59 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]: dispatch 2024-09-10T09:53:00.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:59 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.2", "id": [7, 11]}]: dispatch 2024-09-10T09:53:00.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:59 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-09-10T09:53:00.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:59 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-09-10T09:53:00.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:59 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]: dispatch 2024-09-10T09:53:00.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:59 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]: dispatch 2024-09-10T09:53:00.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:59 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-09-10T09:53:00.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:59 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [5, 8]}]: dispatch 2024-09-10T09:53:00.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:59 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 8]}]: dispatch 2024-09-10T09:53:00.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:59 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]: dispatch 2024-09-10T09:53:00.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:52:59 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]: dispatch 2024-09-10T09:53:00.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:00 smithi077 ceph-mon[30319]: pgmap v322: 129 pgs: 129 active+clean; 581 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-10T09:53:00.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:00 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.2", "id": [7, 11]}]': finished 2024-09-10T09:53:00.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:00 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-09-10T09:53:00.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:00 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-09-10T09:53:00.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:00 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]': finished 2024-09-10T09:53:00.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:00 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]': finished 2024-09-10T09:53:00.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:00 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-09-10T09:53:00.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:00 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [5, 8]}]': finished 2024-09-10T09:53:00.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:00 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 8]}]': finished 2024-09-10T09:53:00.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:00 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2024-09-10T09:53:00.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:00 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]': finished 2024-09-10T09:53:00.941 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:00 smithi077 ceph-mon[30319]: osdmap e93: 12 total, 12 up, 12 in 2024-09-10T09:53:01.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:00 smithi134 ceph-mon[35674]: pgmap v322: 129 pgs: 129 active+clean; 581 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-10T09:53:01.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:00 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.2", "id": [7, 11]}]': finished 2024-09-10T09:53:01.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:00 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-09-10T09:53:01.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:00 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-09-10T09:53:01.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:00 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]': finished 2024-09-10T09:53:01.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:00 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]': finished 2024-09-10T09:53:01.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:00 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-09-10T09:53:01.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:00 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [5, 8]}]': finished 2024-09-10T09:53:01.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:00 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 8]}]': finished 2024-09-10T09:53:01.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:00 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2024-09-10T09:53:01.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:00 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]': finished 2024-09-10T09:53:01.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:00 smithi134 ceph-mon[35674]: osdmap e93: 12 total, 12 up, 12 in 2024-09-10T09:53:01.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:00 smithi120 ceph-mon[34770]: pgmap v322: 129 pgs: 129 active+clean; 581 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-10T09:53:01.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:00 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.2", "id": [7, 11]}]': finished 2024-09-10T09:53:01.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:00 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-09-10T09:53:01.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:00 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-09-10T09:53:01.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:00 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]': finished 2024-09-10T09:53:01.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:00 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]': finished 2024-09-10T09:53:01.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:00 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-09-10T09:53:01.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:00 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [5, 8]}]': finished 2024-09-10T09:53:01.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:00 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 8]}]': finished 2024-09-10T09:53:01.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:00 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2024-09-10T09:53:01.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:00 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]': finished 2024-09-10T09:53:01.175 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:00 smithi120 ceph-mon[34770]: osdmap e93: 12 total, 12 up, 12 in 2024-09-10T09:53:02.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:01 smithi134 ceph-mon[35674]: osdmap e94: 12 total, 12 up, 12 in 2024-09-10T09:53:02.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:01 smithi120 ceph-mon[34770]: osdmap e94: 12 total, 12 up, 12 in 2024-09-10T09:53:02.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:01 smithi077 ceph-mon[30319]: osdmap e94: 12 total, 12 up, 12 in 2024-09-10T09:53:03.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:02 smithi134 ceph-mon[35674]: pgmap v325: 129 pgs: 129 active+clean; 581 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s; 72 B/s, 0 objects/s recovering 2024-09-10T09:53:03.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:02 smithi120 ceph-mon[34770]: pgmap v325: 129 pgs: 129 active+clean; 581 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s; 72 B/s, 0 objects/s recovering 2024-09-10T09:53:03.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:02 smithi077 ceph-mon[30319]: pgmap v325: 129 pgs: 129 active+clean; 581 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s; 72 B/s, 0 objects/s recovering 2024-09-10T09:53:04.912 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi077.front.sepia.ceph.com 2024-09-10T09:53:04.912 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:53:04.912 DEBUG:teuthology.orchestra.run.smithi077:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-09-10T09:53:04.970 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T09:53:05.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:04 smithi134 ceph-mon[35674]: pgmap v326: 129 pgs: 129 active+clean; 581 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s wr, 0 op/s; 553 B/s, 0 objects/s recovering 2024-09-10T09:53:05.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:04 smithi120 ceph-mon[34770]: pgmap v326: 129 pgs: 129 active+clean; 581 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s wr, 0 op/s; 553 B/s, 0 objects/s recovering 2024-09-10T09:53:05.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:04 smithi077 ceph-mon[30319]: pgmap v326: 129 pgs: 129 active+clean; 581 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s wr, 0 op/s; 553 B/s, 0 objects/s recovering 2024-09-10T09:53:05.346 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:53:05.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:05.903+0000 7f4c0ae00640 1 -- 172.21.15.77:0/1285746835 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4c04074f40 msgr2=0x7f4c0410de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:05.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:05.903+0000 7f4c0ae00640 1 --2- 172.21.15.77:0/1285746835 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4c04074f40 0x7f4c0410de80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4bf0009860 tx=0x7f4bf002a3d0 comp rx=0 tx=0).stop 2024-09-10T09:53:05.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:05.903+0000 7f4c0ae00640 1 -- 172.21.15.77:0/1285746835 shutdown_connections 2024-09-10T09:53:05.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:05.903+0000 7f4c0ae00640 1 --2- 172.21.15.77:0/1285746835 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4c0410e3c0 0x7f4c04110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:05.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:05.903+0000 7f4c0ae00640 1 --2- 172.21.15.77:0/1285746835 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4c04074f40 0x7f4c0410de80 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:05.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:05.903+0000 7f4c0ae00640 1 --2- 172.21.15.77:0/1285746835 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4c04074620 0x7f4c04074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:05.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:05.903+0000 7f4c0ae00640 1 -- 172.21.15.77:0/1285746835 >> 172.21.15.77:0/1285746835 conn(0x7f4c040fd8c0 msgr2=0x7f4c040ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:05.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:05.903+0000 7f4c0ae00640 1 -- 172.21.15.77:0/1285746835 shutdown_connections 2024-09-10T09:53:05.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:05.903+0000 7f4c0ae00640 1 -- 172.21.15.77:0/1285746835 wait complete. 2024-09-10T09:53:05.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:05.904+0000 7f4c0ae00640 1 Processor -- start 2024-09-10T09:53:05.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:05.904+0000 7f4c0ae00640 1 -- start start 2024-09-10T09:53:05.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:05.904+0000 7f4c0ae00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4c04074620 0x7f4c0419a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:05.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:05.904+0000 7f4c0ae00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4c04074f40 0x7f4c0419abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:05.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:05.904+0000 7f4c0ae00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4c0410e3c0 0x7f4c0419ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:05.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:05.904+0000 7f4c0ae00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4c041141b0 con 0x7f4c04074620 2024-09-10T09:53:05.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:05.904+0000 7f4c0ae00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4c04114030 con 0x7f4c0410e3c0 2024-09-10T09:53:05.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:05.904+0000 7f4c0ae00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4c04114330 con 0x7f4c04074f40 2024-09-10T09:53:05.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:05.904+0000 7f4c03e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4c04074620 0x7f4c0419a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:05.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:05.904+0000 7f4c03e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4c04074620 0x7f4c0419a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57596/0 (socket says 172.21.15.77:57596) 2024-09-10T09:53:05.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:05.905+0000 7f4c03e00640 1 -- 172.21.15.77:0/2121257065 learned_addr learned my addr 172.21.15.77:0/2121257065 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:53:05.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:05.905+0000 7f4c03400640 1 --2- 172.21.15.77:0/2121257065 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4c04074f40 0x7f4c0419abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:05.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:05.905+0000 7f4c08e00640 1 --2- 172.21.15.77:0/2121257065 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4c0410e3c0 0x7f4c0419ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:05.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:05.905+0000 7f4c03e00640 1 -- 172.21.15.77:0/2121257065 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4c04074f40 msgr2=0x7f4c0419abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:05.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:05.905+0000 7f4c03e00640 1 --2- 172.21.15.77:0/2121257065 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4c04074f40 0x7f4c0419abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:05.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:05.905+0000 7f4c03e00640 1 -- 172.21.15.77:0/2121257065 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4c0410e3c0 msgr2=0x7f4c0419ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:05.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:05.905+0000 7f4c03e00640 1 --2- 172.21.15.77:0/2121257065 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4c0410e3c0 0x7f4c0419ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:05.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:05.905+0000 7f4c03e00640 1 -- 172.21.15.77:0/2121257065 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c0419f660 con 0x7f4c04074620 2024-09-10T09:53:05.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:05.905+0000 7f4c03e00640 1 --2- 172.21.15.77:0/2121257065 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4c04074620 0x7f4c0419a6b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4bf400e9f0 tx=0x7f4bf400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:05.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:05.905+0000 7f4c00c00640 1 -- 172.21.15.77:0/2121257065 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bf400cd80 con 0x7f4c04074620 2024-09-10T09:53:05.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:05.905+0000 7f4c00c00640 1 -- 172.21.15.77:0/2121257065 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4bf4005500 con 0x7f4c04074620 2024-09-10T09:53:05.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:05.905+0000 7f4c0ae00640 1 -- 172.21.15.77:0/2121257065 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c0419f950 con 0x7f4c04074620 2024-09-10T09:53:05.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:05.906+0000 7f4c00c00640 1 -- 172.21.15.77:0/2121257065 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bf4010690 con 0x7f4c04074620 2024-09-10T09:53:05.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:05.906+0000 7f4c0ae00640 1 -- 172.21.15.77:0/2121257065 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c041a7230 con 0x7f4c04074620 2024-09-10T09:53:05.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:05.907+0000 7f4c0ae00640 1 -- 172.21.15.77:0/2121257065 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c04074a00 con 0x7f4c04074620 2024-09-10T09:53:05.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:05.909+0000 7f4c00c00640 1 -- 172.21.15.77:0/2121257065 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f4bf40046c0 con 0x7f4c04074620 2024-09-10T09:53:05.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:05.911+0000 7f4c00c00640 1 --2- 172.21.15.77:0/2121257065 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f4be80a2c00 0x7f4be80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:05.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:05.911+0000 7f4c00c00640 1 -- 172.21.15.77:0/2121257065 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f4bf40cf780 con 0x7f4c04074620 2024-09-10T09:53:05.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:05.911+0000 7f4c03400640 1 --2- 172.21.15.77:0/2121257065 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f4be80a2c00 0x7f4be80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:05.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:05.912+0000 7f4c03400640 1 --2- 172.21.15.77:0/2121257065 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f4be80a2c00 0x7f4be80a50c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4c0410f7b0 tx=0x7f4bf0005c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:05.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:05.915+0000 7f4c00c00640 1 -- 172.21.15.77:0/2121257065 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4bf409c320 con 0x7f4c04074620 2024-09-10T09:53:06.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:06.092+0000 7f4c0ae00640 1 -- 172.21.15.77:0/2121257065 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4c04108fb0 con 0x7f4c04074620 2024-09-10T09:53:06.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:06.094+0000 7f4c00c00640 1 -- 172.21.15.77:0/2121257065 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24707 (secure 0 0 0) 0x7f4bf4086020 con 0x7f4c04074620 2024-09-10T09:53:06.095 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:53:06.095 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":94,"fsid":"14a55226-6f59-11ef-bcea-c7b262605968","created":"2024-09-10T09:44:07.359542+0000","modified":"2024-09-10T09:53:00.787197+0000","last_up_change":"2024-09-10T09:51:26.731971+0000","last_in_change":"2024-09-10T09:51:03.857873+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:47:24.174660+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T09:52:18.766738+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T09:52:21.284745+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T09:52:29.846594+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8d46f634-114c-4926-afdc-f8fa37e40336","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":1509649079},{"type":"v1","addr":"172.21.15.77:6803","nonce":1509649079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":1509649079},{"type":"v1","addr":"172.21.15.77:6805","nonce":1509649079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":1509649079},{"type":"v1","addr":"172.21.15.77:6809","nonce":1509649079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":1509649079},{"type":"v1","addr":"172.21.15.77:6807","nonce":1509649079}]},"public_addr":"172.21.15.77:6803/1509649079","cluster_addr":"172.21.15.77:6805/1509649079","heartbeat_back_addr":"172.21.15.77:6809/1509649079","heartbeat_front_addr":"172.21.15.77:6807/1509649079","state":["exists","up"]},{"osd":1,"uuid":"b6570764-736a-49e4-877a-1965e5812d1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":4243920698},{"type":"v1","addr":"172.21.15.120:6801","nonce":4243920698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":4243920698},{"type":"v1","addr":"172.21.15.120:6803","nonce":4243920698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":4243920698},{"type":"v1","addr":"172.21.15.120:6807","nonce":4243920698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":4243920698},{"type":"v1","addr":"172.21.15.120:6805","nonce":4243920698}]},"public_addr":"172.21.15.120:6801/4243920698","cluster_addr":"172.21.15.120:6803/4243920698","heartbeat_back_addr":"172.21.15.120:6807/4243920698","heartbeat_front_addr":"172.21.15.120:6805/4243920698","state":["exists","up"]},{"osd":2,"uuid":"5e83fe13-d1ee-483e-a575-e8041ec836f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":3280466823},{"type":"v1","addr":"172.21.15.134:6801","nonce":3280466823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":3280466823},{"type":"v1","addr":"172.21.15.134:6803","nonce":3280466823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":3280466823},{"type":"v1","addr":"172.21.15.134:6807","nonce":3280466823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":3280466823},{"type":"v1","addr":"172.21.15.134:6805","nonce":3280466823}]},"public_addr":"172.21.15.134:6801/3280466823","cluster_addr":"172.21.15.134:6803/3280466823","heartbeat_back_addr":"172.21.15.134:6807/3280466823","heartbeat_front_addr":"172.21.15.134:6805/3280466823","state":["exists","up"]},{"osd":3,"uuid":"5affc6a7-1cae-48fe-b7db-befc2fcafecb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":1629416786},{"type":"v1","addr":"172.21.15.77:6811","nonce":1629416786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":1629416786},{"type":"v1","addr":"172.21.15.77:6813","nonce":1629416786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":1629416786},{"type":"v1","addr":"172.21.15.77:6817","nonce":1629416786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":1629416786},{"type":"v1","addr":"172.21.15.77:6815","nonce":1629416786}]},"public_addr":"172.21.15.77:6811/1629416786","cluster_addr":"172.21.15.77:6813/1629416786","heartbeat_back_addr":"172.21.15.77:6817/1629416786","heartbeat_front_addr":"172.21.15.77:6815/1629416786","state":["exists","up"]},{"osd":4,"uuid":"c826978f-40b0-43ae-adbb-5c93b32a68ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":3162658038},{"type":"v1","addr":"172.21.15.120:6809","nonce":3162658038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":3162658038},{"type":"v1","addr":"172.21.15.120:6811","nonce":3162658038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":3162658038},{"type":"v1","addr":"172.21.15.120:6815","nonce":3162658038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":3162658038},{"type":"v1","addr":"172.21.15.120:6813","nonce":3162658038}]},"public_addr":"172.21.15.120:6809/3162658038","cluster_addr":"172.21.15.120:6811/3162658038","heartbeat_back_addr":"172.21.15.120:6815/3162658038","heartbeat_front_addr":"172.21.15.120:6813/3162658038","state":["exists","up"]},{"osd":5,"uuid":"be51c7fb-2765-46a3-9861-c950f63de148","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":4081444621},{"type":"v1","addr":"172.21.15.134:6809","nonce":4081444621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":4081444621},{"type":"v1","addr":"172.21.15.134:6811","nonce":4081444621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":4081444621},{"type":"v1","addr":"172.21.15.134:6815","nonce":4081444621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":4081444621},{"type":"v1","addr":"172.21.15.134:6813","nonce":4081444621}]},"public_addr":"172.21.15.134:6809/4081444621","cluster_addr":"172.21.15.134:6811/4081444621","heartbeat_back_addr":"172.21.15.134:6815/4081444621","heartbeat_front_addr":"172.21.15.134:6813/4081444621","state":["exists","up"]},{"osd":6,"uuid":"4eb403ab-473b-41cc-803d-18c908f768a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":879565639},{"type":"v1","addr":"172.21.15.77:6819","nonce":879565639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":879565639},{"type":"v1","addr":"172.21.15.77:6821","nonce":879565639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":879565639},{"type":"v1","addr":"172.21.15.77:6825","nonce":879565639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":879565639},{"type":"v1","addr":"172.21.15.77:6823","nonce":879565639}]},"public_addr":"172.21.15.77:6819/879565639","cluster_addr":"172.21.15.77:6821/879565639","heartbeat_back_addr":"172.21.15.77:6825/879565639","heartbeat_front_addr":"172.21.15.77:6823/879565639","state":["exists","up"]},{"osd":7,"uuid":"ba982bd0-c236-456c-9889-61087d79c1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":2688667400},{"type":"v1","addr":"172.21.15.120:6817","nonce":2688667400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":2688667400},{"type":"v1","addr":"172.21.15.120:6819","nonce":2688667400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":2688667400},{"type":"v1","addr":"172.21.15.120:6823","nonce":2688667400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2688667400},{"type":"v1","addr":"172.21.15.120:6821","nonce":2688667400}]},"public_addr":"172.21.15.120:6817/2688667400","cluster_addr":"172.21.15.120:6819/2688667400","heartbeat_back_addr":"172.21.15.120:6823/2688667400","heartbeat_front_addr":"172.21.15.120:6821/2688667400","state":["exists","up"]},{"osd":8,"uuid":"f2e8ca43-57c9-4933-b5f7-73f4d1b3e08b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2081686544},{"type":"v1","addr":"172.21.15.134:6817","nonce":2081686544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2081686544},{"type":"v1","addr":"172.21.15.134:6819","nonce":2081686544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2081686544},{"type":"v1","addr":"172.21.15.134:6823","nonce":2081686544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2081686544},{"type":"v1","addr":"172.21.15.134:6821","nonce":2081686544}]},"public_addr":"172.21.15.134:6817/2081686544","cluster_addr":"172.21.15.134:6819/2081686544","heartbeat_back_addr":"172.21.15.134:6823/2081686544","heartbeat_front_addr":"172.21.15.134:6821/2081686544","state":["exists","up"]},{"osd":9,"uuid":"9486f8b5-0614-4208-87ec-117741f6b622","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":1210619387},{"type":"v1","addr":"172.21.15.77:6827","nonce":1210619387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":1210619387},{"type":"v1","addr":"172.21.15.77:6829","nonce":1210619387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":1210619387},{"type":"v1","addr":"172.21.15.77:6833","nonce":1210619387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":1210619387},{"type":"v1","addr":"172.21.15.77:6831","nonce":1210619387}]},"public_addr":"172.21.15.77:6827/1210619387","cluster_addr":"172.21.15.77:6829/1210619387","heartbeat_back_addr":"172.21.15.77:6833/1210619387","heartbeat_front_addr":"172.21.15.77:6831/1210619387","state":["exists","up"]},{"osd":10,"uuid":"8e8192b0-1398-4d4d-9756-762b2f874d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":754733885},{"type":"v1","addr":"172.21.15.120:6825","nonce":754733885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":754733885},{"type":"v1","addr":"172.21.15.120:6827","nonce":754733885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":754733885},{"type":"v1","addr":"172.21.15.120:6831","nonce":754733885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":754733885},{"type":"v1","addr":"172.21.15.120:6829","nonce":754733885}]},"public_addr":"172.21.15.120:6825/754733885","cluster_addr":"172.21.15.120:6827/754733885","heartbeat_back_addr":"172.21.15.120:6831/754733885","heartbeat_front_addr":"172.21.15.120:6829/754733885","state":["exists","up"]},{"osd":11,"uuid":"85c1ccd4-5147-4995-896e-fe977f73f749","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1602213143},{"type":"v1","addr":"172.21.15.134:6825","nonce":1602213143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1602213143},{"type":"v1","addr":"172.21.15.134:6827","nonce":1602213143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1602213143},{"type":"v1","addr":"172.21.15.134:6831","nonce":1602213143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1602213143},{"type":"v1","addr":"172.21.15.134:6829","nonce":1602213143}]},"public_addr":"172.21.15.134:6825/1602213143","cluster_addr":"172.21.15.134:6827/1602213143","heartbeat_back_addr":"172.21.15.134:6831/1602213143","heartbeat_front_addr":"172.21.15.134:6829/1602213143","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:46:30.293451+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:46:56.554801+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:47:21.476225+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:47:49.737332+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:48:16.092181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:48:41.240815+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:49:11.683950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:49:38.859255+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:50:04.105401+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:50:33.847810+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:50:59.901797+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:51:25.241507+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.2","mappings":[{"from":7,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":5,"to":8}]},{"pgid":"4.6","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.77:0/1697982655":"2024-09-11T09:44:59.575663+0000","172.21.15.77:6801/1527070442":"2024-09-11T09:44:59.575663+0000","172.21.15.77:0/3146688196":"2024-09-11T09:44:32.185342+0000","172.21.15.77:0/371773177":"2024-09-11T09:44:59.575663+0000","172.21.15.77:0/1925456910":"2024-09-11T09:44:59.575663+0000","172.21.15.77:0/138545832":"2024-09-11T09:44:32.185342+0000","172.21.15.77:6801/3235900503":"2024-09-11T09:44:32.185342+0000","172.21.15.77:6800/1527070442":"2024-09-11T09:44:59.575663+0000","172.21.15.77:6800/3235900503":"2024-09-11T09:44:32.185342+0000","172.21.15.77:0/1647092548":"2024-09-11T09:44:32.185342+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T09:53:06.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:06.097+0000 7f4c0ae00640 1 -- 172.21.15.77:0/2121257065 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f4be80a2c00 msgr2=0x7f4be80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:06.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:06.097+0000 7f4c0ae00640 1 --2- 172.21.15.77:0/2121257065 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f4be80a2c00 0x7f4be80a50c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4c0410f7b0 tx=0x7f4bf0005c90 comp rx=0 tx=0).stop 2024-09-10T09:53:06.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:06.097+0000 7f4c0ae00640 1 -- 172.21.15.77:0/2121257065 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4c04074620 msgr2=0x7f4c0419a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:06.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:06.097+0000 7f4c0ae00640 1 --2- 172.21.15.77:0/2121257065 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4c04074620 0x7f4c0419a6b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4bf400e9f0 tx=0x7f4bf400edf0 comp rx=0 tx=0).stop 2024-09-10T09:53:06.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:06.097+0000 7f4c0ae00640 1 -- 172.21.15.77:0/2121257065 shutdown_connections 2024-09-10T09:53:06.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:06.097+0000 7f4c0ae00640 1 --2- 172.21.15.77:0/2121257065 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f4be80a2c00 0x7f4be80a50c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:06.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:06.097+0000 7f4c0ae00640 1 --2- 172.21.15.77:0/2121257065 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4c0410e3c0 0x7f4c0419ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:06.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:06.097+0000 7f4c0ae00640 1 --2- 172.21.15.77:0/2121257065 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4c04074f40 0x7f4c0419abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:06.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:06.097+0000 7f4c0ae00640 1 --2- 172.21.15.77:0/2121257065 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4c04074620 0x7f4c0419a6b0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:06.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:06.097+0000 7f4c0ae00640 1 -- 172.21.15.77:0/2121257065 >> 172.21.15.77:0/2121257065 conn(0x7f4c040fd8c0 msgr2=0x7f4c040fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:06.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:06.098+0000 7f4c0ae00640 1 -- 172.21.15.77:0/2121257065 shutdown_connections 2024-09-10T09:53:06.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:06.098+0000 7f4c0ae00640 1 -- 172.21.15.77:0/2121257065 wait complete. 2024-09-10T09:53:06.838 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T09:53:07.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:06 smithi134 ceph-mon[35674]: pgmap v327: 129 pgs: 129 active+clean; 581 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s wr, 0 op/s; 553 B/s, 0 objects/s recovering 2024-09-10T09:53:07.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:06 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/2121257065' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T09:53:07.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:06 smithi120 ceph-mon[34770]: pgmap v327: 129 pgs: 129 active+clean; 581 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s wr, 0 op/s; 553 B/s, 0 objects/s recovering 2024-09-10T09:53:07.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:06 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/2121257065' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T09:53:07.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:06 smithi077 ceph-mon[30319]: pgmap v327: 129 pgs: 129 active+clean; 581 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s wr, 0 op/s; 553 B/s, 0 objects/s recovering 2024-09-10T09:53:07.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:06 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/2121257065' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T09:53:07.204 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:53:07.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:07.762+0000 7f0d4f400640 1 -- 172.21.15.77:0/4004356872 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0d48101df0 msgr2=0x7f0d481021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:07.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:07.762+0000 7f0d4f400640 1 --2- 172.21.15.77:0/4004356872 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0d48101df0 0x7f0d481021d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0d38009900 tx=0x7f0d3802a400 comp rx=0 tx=0).stop 2024-09-10T09:53:07.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:07.763+0000 7f0d4f400640 1 -- 172.21.15.77:0/4004356872 shutdown_connections 2024-09-10T09:53:07.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:07.763+0000 7f0d4f400640 1 --2- 172.21.15.77:0/4004356872 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d48106f20 0x7f0d4810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:07.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:07.763+0000 7f0d4f400640 1 --2- 172.21.15.77:0/4004356872 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0d481027a0 0x7f0d481067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:07.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:07.763+0000 7f0d4f400640 1 --2- 172.21.15.77:0/4004356872 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0d48101df0 0x7f0d481021d0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:07.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:07.763+0000 7f0d4f400640 1 -- 172.21.15.77:0/4004356872 >> 172.21.15.77:0/4004356872 conn(0x7f0d480fd8b0 msgr2=0x7f0d480ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:07.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:07.763+0000 7f0d4f400640 1 -- 172.21.15.77:0/4004356872 shutdown_connections 2024-09-10T09:53:07.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:07.763+0000 7f0d4f400640 1 -- 172.21.15.77:0/4004356872 wait complete. 2024-09-10T09:53:07.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:07.764+0000 7f0d4f400640 1 Processor -- start 2024-09-10T09:53:07.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:07.764+0000 7f0d4f400640 1 -- start start 2024-09-10T09:53:07.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:07.764+0000 7f0d4f400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0d48101df0 0x7f0d480744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:07.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:07.764+0000 7f0d4f400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d481027a0 0x7f0d48074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:07.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:07.764+0000 7f0d4f400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0d48106f20 0x7f0d48074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:07.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:07.764+0000 7f0d4f400640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0d48111f90 con 0x7f0d48101df0 2024-09-10T09:53:07.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:07.764+0000 7f0d4f400640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0d48111e10 con 0x7f0d48106f20 2024-09-10T09:53:07.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:07.764+0000 7f0d4f400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0d48112110 con 0x7f0d481027a0 2024-09-10T09:53:07.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:07.764+0000 7f0d4ca00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0d48101df0 0x7f0d480744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:07.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:07.764+0000 7f0d4ca00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0d48101df0 0x7f0d480744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57622/0 (socket says 172.21.15.77:57622) 2024-09-10T09:53:07.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:07.764+0000 7f0d4ca00640 1 -- 172.21.15.77:0/776895757 learned_addr learned my addr 172.21.15.77:0/776895757 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:53:07.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:07.764+0000 7f0d3fe00640 1 --2- 172.21.15.77:0/776895757 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d481027a0 0x7f0d48074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:07.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:07.765+0000 7f0d4d400640 1 --2- 172.21.15.77:0/776895757 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0d48106f20 0x7f0d48074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:07.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:07.765+0000 7f0d4ca00640 1 -- 172.21.15.77:0/776895757 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d481027a0 msgr2=0x7f0d48074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:07.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:07.765+0000 7f0d4ca00640 1 --2- 172.21.15.77:0/776895757 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d481027a0 0x7f0d48074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:07.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:07.765+0000 7f0d4ca00640 1 -- 172.21.15.77:0/776895757 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0d48106f20 msgr2=0x7f0d48074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:07.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:07.765+0000 7f0d4ca00640 1 --2- 172.21.15.77:0/776895757 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0d48106f20 0x7f0d48074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:07.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:07.765+0000 7f0d4ca00640 1 -- 172.21.15.77:0/776895757 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d481a6930 con 0x7f0d48101df0 2024-09-10T09:53:07.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:07.765+0000 7f0d4ca00640 1 --2- 172.21.15.77:0/776895757 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0d48101df0 0x7f0d480744f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0d38004770 tx=0x7f0d38003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:07.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:07.765+0000 7f0d46000640 1 -- 172.21.15.77:0/776895757 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d3803f070 con 0x7f0d48101df0 2024-09-10T09:53:07.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:07.765+0000 7f0d4f400640 1 -- 172.21.15.77:0/776895757 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d481a6bc0 con 0x7f0d48101df0 2024-09-10T09:53:07.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:07.765+0000 7f0d46000640 1 -- 172.21.15.77:0/776895757 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d38031ca0 con 0x7f0d48101df0 2024-09-10T09:53:07.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:07.765+0000 7f0d46000640 1 -- 172.21.15.77:0/776895757 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d38035030 con 0x7f0d48101df0 2024-09-10T09:53:07.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:07.766+0000 7f0d4f400640 1 -- 172.21.15.77:0/776895757 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d481a70a0 con 0x7f0d48101df0 2024-09-10T09:53:07.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:07.767+0000 7f0d4f400640 1 -- 172.21.15.77:0/776895757 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d481021d0 con 0x7f0d48101df0 2024-09-10T09:53:07.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:07.768+0000 7f0d46000640 1 -- 172.21.15.77:0/776895757 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f0d38042070 con 0x7f0d48101df0 2024-09-10T09:53:07.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:07.770+0000 7f0d46000640 1 --2- 172.21.15.77:0/776895757 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f0d2c0a2b80 0x7f0d2c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:07.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:07.770+0000 7f0d46000640 1 -- 172.21.15.77:0/776895757 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f0d380ef7d0 con 0x7f0d48101df0 2024-09-10T09:53:07.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:07.770+0000 7f0d3fe00640 1 --2- 172.21.15.77:0/776895757 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f0d2c0a2b80 0x7f0d2c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:07.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:07.771+0000 7f0d3fe00640 1 --2- 172.21.15.77:0/776895757 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f0d2c0a2b80 0x7f0d2c0a5040 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0d48075f80 tx=0x7f0d3000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:07.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:07.774+0000 7f0d46000640 1 -- 172.21.15.77:0/776895757 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0d380bbd20 con 0x7f0d48101df0 2024-09-10T09:53:07.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:07.954+0000 7f0d4f400640 1 -- 172.21.15.77:0/776895757 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0d4810a130 con 0x7f0d48101df0 2024-09-10T09:53:07.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:07.955+0000 7f0d46000640 1 -- 172.21.15.77:0/776895757 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24707 (secure 0 0 0) 0x7f0d380c0020 con 0x7f0d48101df0 2024-09-10T09:53:07.957 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:53:07.957 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":94,"fsid":"14a55226-6f59-11ef-bcea-c7b262605968","created":"2024-09-10T09:44:07.359542+0000","modified":"2024-09-10T09:53:00.787197+0000","last_up_change":"2024-09-10T09:51:26.731971+0000","last_in_change":"2024-09-10T09:51:03.857873+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:47:24.174660+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T09:52:18.766738+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T09:52:21.284745+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T09:52:29.846594+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8d46f634-114c-4926-afdc-f8fa37e40336","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":1509649079},{"type":"v1","addr":"172.21.15.77:6803","nonce":1509649079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":1509649079},{"type":"v1","addr":"172.21.15.77:6805","nonce":1509649079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":1509649079},{"type":"v1","addr":"172.21.15.77:6809","nonce":1509649079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":1509649079},{"type":"v1","addr":"172.21.15.77:6807","nonce":1509649079}]},"public_addr":"172.21.15.77:6803/1509649079","cluster_addr":"172.21.15.77:6805/1509649079","heartbeat_back_addr":"172.21.15.77:6809/1509649079","heartbeat_front_addr":"172.21.15.77:6807/1509649079","state":["exists","up"]},{"osd":1,"uuid":"b6570764-736a-49e4-877a-1965e5812d1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":4243920698},{"type":"v1","addr":"172.21.15.120:6801","nonce":4243920698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":4243920698},{"type":"v1","addr":"172.21.15.120:6803","nonce":4243920698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":4243920698},{"type":"v1","addr":"172.21.15.120:6807","nonce":4243920698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":4243920698},{"type":"v1","addr":"172.21.15.120:6805","nonce":4243920698}]},"public_addr":"172.21.15.120:6801/4243920698","cluster_addr":"172.21.15.120:6803/4243920698","heartbeat_back_addr":"172.21.15.120:6807/4243920698","heartbeat_front_addr":"172.21.15.120:6805/4243920698","state":["exists","up"]},{"osd":2,"uuid":"5e83fe13-d1ee-483e-a575-e8041ec836f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":3280466823},{"type":"v1","addr":"172.21.15.134:6801","nonce":3280466823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":3280466823},{"type":"v1","addr":"172.21.15.134:6803","nonce":3280466823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":3280466823},{"type":"v1","addr":"172.21.15.134:6807","nonce":3280466823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":3280466823},{"type":"v1","addr":"172.21.15.134:6805","nonce":3280466823}]},"public_addr":"172.21.15.134:6801/3280466823","cluster_addr":"172.21.15.134:6803/3280466823","heartbeat_back_addr":"172.21.15.134:6807/3280466823","heartbeat_front_addr":"172.21.15.134:6805/3280466823","state":["exists","up"]},{"osd":3,"uuid":"5affc6a7-1cae-48fe-b7db-befc2fcafecb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":1629416786},{"type":"v1","addr":"172.21.15.77:6811","nonce":1629416786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":1629416786},{"type":"v1","addr":"172.21.15.77:6813","nonce":1629416786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":1629416786},{"type":"v1","addr":"172.21.15.77:6817","nonce":1629416786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":1629416786},{"type":"v1","addr":"172.21.15.77:6815","nonce":1629416786}]},"public_addr":"172.21.15.77:6811/1629416786","cluster_addr":"172.21.15.77:6813/1629416786","heartbeat_back_addr":"172.21.15.77:6817/1629416786","heartbeat_front_addr":"172.21.15.77:6815/1629416786","state":["exists","up"]},{"osd":4,"uuid":"c826978f-40b0-43ae-adbb-5c93b32a68ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":3162658038},{"type":"v1","addr":"172.21.15.120:6809","nonce":3162658038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":3162658038},{"type":"v1","addr":"172.21.15.120:6811","nonce":3162658038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":3162658038},{"type":"v1","addr":"172.21.15.120:6815","nonce":3162658038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":3162658038},{"type":"v1","addr":"172.21.15.120:6813","nonce":3162658038}]},"public_addr":"172.21.15.120:6809/3162658038","cluster_addr":"172.21.15.120:6811/3162658038","heartbeat_back_addr":"172.21.15.120:6815/3162658038","heartbeat_front_addr":"172.21.15.120:6813/3162658038","state":["exists","up"]},{"osd":5,"uuid":"be51c7fb-2765-46a3-9861-c950f63de148","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":4081444621},{"type":"v1","addr":"172.21.15.134:6809","nonce":4081444621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":4081444621},{"type":"v1","addr":"172.21.15.134:6811","nonce":4081444621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":4081444621},{"type":"v1","addr":"172.21.15.134:6815","nonce":4081444621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":4081444621},{"type":"v1","addr":"172.21.15.134:6813","nonce":4081444621}]},"public_addr":"172.21.15.134:6809/4081444621","cluster_addr":"172.21.15.134:6811/4081444621","heartbeat_back_addr":"172.21.15.134:6815/4081444621","heartbeat_front_addr":"172.21.15.134:6813/4081444621","state":["exists","up"]},{"osd":6,"uuid":"4eb403ab-473b-41cc-803d-18c908f768a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":879565639},{"type":"v1","addr":"172.21.15.77:6819","nonce":879565639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":879565639},{"type":"v1","addr":"172.21.15.77:6821","nonce":879565639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":879565639},{"type":"v1","addr":"172.21.15.77:6825","nonce":879565639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":879565639},{"type":"v1","addr":"172.21.15.77:6823","nonce":879565639}]},"public_addr":"172.21.15.77:6819/879565639","cluster_addr":"172.21.15.77:6821/879565639","heartbeat_back_addr":"172.21.15.77:6825/879565639","heartbeat_front_addr":"172.21.15.77:6823/879565639","state":["exists","up"]},{"osd":7,"uuid":"ba982bd0-c236-456c-9889-61087d79c1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":2688667400},{"type":"v1","addr":"172.21.15.120:6817","nonce":2688667400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":2688667400},{"type":"v1","addr":"172.21.15.120:6819","nonce":2688667400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":2688667400},{"type":"v1","addr":"172.21.15.120:6823","nonce":2688667400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2688667400},{"type":"v1","addr":"172.21.15.120:6821","nonce":2688667400}]},"public_addr":"172.21.15.120:6817/2688667400","cluster_addr":"172.21.15.120:6819/2688667400","heartbeat_back_addr":"172.21.15.120:6823/2688667400","heartbeat_front_addr":"172.21.15.120:6821/2688667400","state":["exists","up"]},{"osd":8,"uuid":"f2e8ca43-57c9-4933-b5f7-73f4d1b3e08b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2081686544},{"type":"v1","addr":"172.21.15.134:6817","nonce":2081686544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2081686544},{"type":"v1","addr":"172.21.15.134:6819","nonce":2081686544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2081686544},{"type":"v1","addr":"172.21.15.134:6823","nonce":2081686544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2081686544},{"type":"v1","addr":"172.21.15.134:6821","nonce":2081686544}]},"public_addr":"172.21.15.134:6817/2081686544","cluster_addr":"172.21.15.134:6819/2081686544","heartbeat_back_addr":"172.21.15.134:6823/2081686544","heartbeat_front_addr":"172.21.15.134:6821/2081686544","state":["exists","up"]},{"osd":9,"uuid":"9486f8b5-0614-4208-87ec-117741f6b622","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":1210619387},{"type":"v1","addr":"172.21.15.77:6827","nonce":1210619387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":1210619387},{"type":"v1","addr":"172.21.15.77:6829","nonce":1210619387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":1210619387},{"type":"v1","addr":"172.21.15.77:6833","nonce":1210619387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":1210619387},{"type":"v1","addr":"172.21.15.77:6831","nonce":1210619387}]},"public_addr":"172.21.15.77:6827/1210619387","cluster_addr":"172.21.15.77:6829/1210619387","heartbeat_back_addr":"172.21.15.77:6833/1210619387","heartbeat_front_addr":"172.21.15.77:6831/1210619387","state":["exists","up"]},{"osd":10,"uuid":"8e8192b0-1398-4d4d-9756-762b2f874d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":754733885},{"type":"v1","addr":"172.21.15.120:6825","nonce":754733885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":754733885},{"type":"v1","addr":"172.21.15.120:6827","nonce":754733885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":754733885},{"type":"v1","addr":"172.21.15.120:6831","nonce":754733885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":754733885},{"type":"v1","addr":"172.21.15.120:6829","nonce":754733885}]},"public_addr":"172.21.15.120:6825/754733885","cluster_addr":"172.21.15.120:6827/754733885","heartbeat_back_addr":"172.21.15.120:6831/754733885","heartbeat_front_addr":"172.21.15.120:6829/754733885","state":["exists","up"]},{"osd":11,"uuid":"85c1ccd4-5147-4995-896e-fe977f73f749","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1602213143},{"type":"v1","addr":"172.21.15.134:6825","nonce":1602213143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1602213143},{"type":"v1","addr":"172.21.15.134:6827","nonce":1602213143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1602213143},{"type":"v1","addr":"172.21.15.134:6831","nonce":1602213143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1602213143},{"type":"v1","addr":"172.21.15.134:6829","nonce":1602213143}]},"public_addr":"172.21.15.134:6825/1602213143","cluster_addr":"172.21.15.134:6827/1602213143","heartbeat_back_addr":"172.21.15.134:6831/1602213143","heartbeat_front_addr":"172.21.15.134:6829/1602213143","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:46:30.293451+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:46:56.554801+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:47:21.476225+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:47:49.737332+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:48:16.092181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:48:41.240815+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:49:11.683950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:49:38.859255+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:50:04.105401+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:50:33.847810+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:50:59.901797+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:51:25.241507+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.2","mappings":[{"from":7,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":5,"to":8}]},{"pgid":"4.6","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.77:0/1697982655":"2024-09-11T09:44:59.575663+0000","172.21.15.77:6801/1527070442":"2024-09-11T09:44:59.575663+0000","172.21.15.77:0/3146688196":"2024-09-11T09:44:32.185342+0000","172.21.15.77:0/371773177":"2024-09-11T09:44:59.575663+0000","172.21.15.77:0/1925456910":"2024-09-11T09:44:59.575663+0000","172.21.15.77:0/138545832":"2024-09-11T09:44:32.185342+0000","172.21.15.77:6801/3235900503":"2024-09-11T09:44:32.185342+0000","172.21.15.77:6800/1527070442":"2024-09-11T09:44:59.575663+0000","172.21.15.77:6800/3235900503":"2024-09-11T09:44:32.185342+0000","172.21.15.77:0/1647092548":"2024-09-11T09:44:32.185342+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T09:53:07.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:07.957+0000 7f0d4f400640 1 -- 172.21.15.77:0/776895757 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f0d2c0a2b80 msgr2=0x7f0d2c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:07.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:07.957+0000 7f0d4f400640 1 --2- 172.21.15.77:0/776895757 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f0d2c0a2b80 0x7f0d2c0a5040 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0d48075f80 tx=0x7f0d3000b390 comp rx=0 tx=0).stop 2024-09-10T09:53:07.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:07.957+0000 7f0d4f400640 1 -- 172.21.15.77:0/776895757 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0d48101df0 msgr2=0x7f0d480744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:07.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:07.957+0000 7f0d4f400640 1 --2- 172.21.15.77:0/776895757 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0d48101df0 0x7f0d480744f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0d38004770 tx=0x7f0d38003c10 comp rx=0 tx=0).stop 2024-09-10T09:53:07.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:07.958+0000 7f0d4f400640 1 -- 172.21.15.77:0/776895757 shutdown_connections 2024-09-10T09:53:07.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:07.958+0000 7f0d4f400640 1 --2- 172.21.15.77:0/776895757 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f0d2c0a2b80 0x7f0d2c0a5040 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:07.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:07.958+0000 7f0d4f400640 1 --2- 172.21.15.77:0/776895757 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0d48106f20 0x7f0d48074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:07.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:07.958+0000 7f0d4f400640 1 --2- 172.21.15.77:0/776895757 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d481027a0 0x7f0d48074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:07.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:07.958+0000 7f0d4f400640 1 --2- 172.21.15.77:0/776895757 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0d48101df0 0x7f0d480744f0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:07.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:07.958+0000 7f0d4f400640 1 -- 172.21.15.77:0/776895757 >> 172.21.15.77:0/776895757 conn(0x7f0d480fd8b0 msgr2=0x7f0d480feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:07.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:07.958+0000 7f0d4f400640 1 -- 172.21.15.77:0/776895757 shutdown_connections 2024-09-10T09:53:07.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:07.958+0000 7f0d4f400640 1 -- 172.21.15.77:0/776895757 wait complete. 2024-09-10T09:53:08.713 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-09-10T09:53:08.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:08 smithi077 ceph-mon[30319]: pgmap v328: 129 pgs: 129 active+clean; 581 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s; 553 B/s, 0 objects/s recovering 2024-09-10T09:53:08.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:08 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/776895757' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T09:53:09.085 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:53:09.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:08 smithi134 ceph-mon[35674]: pgmap v328: 129 pgs: 129 active+clean; 581 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s; 553 B/s, 0 objects/s recovering 2024-09-10T09:53:09.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:08 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/776895757' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T09:53:09.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:08 smithi120 ceph-mon[34770]: pgmap v328: 129 pgs: 129 active+clean; 581 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s; 553 B/s, 0 objects/s recovering 2024-09-10T09:53:09.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:08 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/776895757' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T09:53:09.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:09.632+0000 7f37df400640 1 -- 172.21.15.77:0/3136657909 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f37d810e3c0 msgr2=0x7f37d8110810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:09.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:09.632+0000 7f37df400640 1 --2- 172.21.15.77:0/3136657909 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f37d810e3c0 0x7f37d8110810 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f37d0009ca0 tx=0x7f37d002a950 comp rx=0 tx=0).stop 2024-09-10T09:53:09.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:09.632+0000 7f37df400640 1 -- 172.21.15.77:0/3136657909 shutdown_connections 2024-09-10T09:53:09.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:09.632+0000 7f37df400640 1 --2- 172.21.15.77:0/3136657909 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f37d810e3c0 0x7f37d8110810 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:09.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:09.632+0000 7f37df400640 1 --2- 172.21.15.77:0/3136657909 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37d8074f40 0x7f37d810de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:09.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:09.632+0000 7f37df400640 1 --2- 172.21.15.77:0/3136657909 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f37d8074620 0x7f37d8074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:09.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:09.632+0000 7f37df400640 1 -- 172.21.15.77:0/3136657909 >> 172.21.15.77:0/3136657909 conn(0x7f37d80fd8c0 msgr2=0x7f37d80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:09.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:09.632+0000 7f37df400640 1 -- 172.21.15.77:0/3136657909 shutdown_connections 2024-09-10T09:53:09.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:09.632+0000 7f37df400640 1 -- 172.21.15.77:0/3136657909 wait complete. 2024-09-10T09:53:09.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:09.633+0000 7f37df400640 1 Processor -- start 2024-09-10T09:53:09.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:09.633+0000 7f37df400640 1 -- start start 2024-09-10T09:53:09.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:09.634+0000 7f37df400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37d8074620 0x7f37d819a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:09.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:09.634+0000 7f37df400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f37d8074f40 0x7f37d819aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:09.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:09.634+0000 7f37df400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f37d810e3c0 0x7f37d819ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:09.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:09.634+0000 7f37df400640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f37d81141c0 con 0x7f37d8074f40 2024-09-10T09:53:09.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:09.634+0000 7f37df400640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f37d8114040 con 0x7f37d810e3c0 2024-09-10T09:53:09.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:09.634+0000 7f37df400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f37d8114340 con 0x7f37d8074620 2024-09-10T09:53:09.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:09.634+0000 7f37d7e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f37d8074f40 0x7f37d819aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:09.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:09.634+0000 7f37d7e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f37d8074f40 0x7f37d819aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57630/0 (socket says 172.21.15.77:57630) 2024-09-10T09:53:09.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:09.634+0000 7f37d7e00640 1 -- 172.21.15.77:0/1886092695 learned_addr learned my addr 172.21.15.77:0/1886092695 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:53:09.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:09.634+0000 7f37dd400640 1 --2- 172.21.15.77:0/1886092695 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f37d810e3c0 0x7f37d819ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:09.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:09.634+0000 7f37dca00640 1 --2- 172.21.15.77:0/1886092695 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37d8074620 0x7f37d819a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:09.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:09.635+0000 7f37d7e00640 1 -- 172.21.15.77:0/1886092695 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37d8074620 msgr2=0x7f37d819a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:09.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:09.635+0000 7f37d7e00640 1 --2- 172.21.15.77:0/1886092695 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37d8074620 0x7f37d819a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:09.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:09.635+0000 7f37d7e00640 1 -- 172.21.15.77:0/1886092695 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f37d810e3c0 msgr2=0x7f37d819ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:09.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:09.635+0000 7f37d7e00640 1 --2- 172.21.15.77:0/1886092695 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f37d810e3c0 0x7f37d819ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:09.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:09.635+0000 7f37d7e00640 1 -- 172.21.15.77:0/1886092695 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37d819f610 con 0x7f37d8074f40 2024-09-10T09:53:09.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:09.635+0000 7f37d7e00640 1 --2- 172.21.15.77:0/1886092695 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f37d8074f40 0x7f37d819aba0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f37c400e9f0 tx=0x7f37c400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:09.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:09.635+0000 7f37d5600640 1 -- 172.21.15.77:0/1886092695 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c400cd80 con 0x7f37d8074f40 2024-09-10T09:53:09.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:09.635+0000 7f37d5600640 1 -- 172.21.15.77:0/1886092695 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37c4005500 con 0x7f37d8074f40 2024-09-10T09:53:09.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:09.635+0000 7f37d5600640 1 -- 172.21.15.77:0/1886092695 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c4010690 con 0x7f37d8074f40 2024-09-10T09:53:09.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:09.635+0000 7f37df400640 1 -- 172.21.15.77:0/1886092695 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37d819f900 con 0x7f37d8074f40 2024-09-10T09:53:09.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:09.635+0000 7f37df400640 1 -- 172.21.15.77:0/1886092695 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37d81a71e0 con 0x7f37d8074f40 2024-09-10T09:53:09.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:09.636+0000 7f37df400640 1 -- 172.21.15.77:0/1886092695 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37d8074a00 con 0x7f37d8074f40 2024-09-10T09:53:09.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:09.638+0000 7f37d5600640 1 -- 172.21.15.77:0/1886092695 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f37c40046c0 con 0x7f37d8074f40 2024-09-10T09:53:09.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:09.640+0000 7f37d5600640 1 --2- 172.21.15.77:0/1886092695 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f37c00a2b80 0x7f37c00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:09.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:09.640+0000 7f37d5600640 1 -- 172.21.15.77:0/1886092695 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f37c40cf920 con 0x7f37d8074f40 2024-09-10T09:53:09.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:09.640+0000 7f37dca00640 1 --2- 172.21.15.77:0/1886092695 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f37c00a2b80 0x7f37c00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:09.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:09.641+0000 7f37dca00640 1 --2- 172.21.15.77:0/1886092695 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f37c00a2b80 0x7f37c00a5040 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f37cc009bd0 tx=0x7f37cc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:09.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:09.642+0000 7f37d5600640 1 -- 172.21.15.77:0/1886092695 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f37c40541b0 con 0x7f37d8074f40 2024-09-10T09:53:09.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:09.854+0000 7f37df400640 1 -- 172.21.15.77:0/1886092695 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f37d802d7c0 con 0x7f37d8074f40 2024-09-10T09:53:10.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:10.824+0000 7f37d5600640 1 -- 172.21.15.77:0/1886092695 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7f37c409c520 con 0x7f37d8074f40 2024-09-10T09:53:10.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:10.827+0000 7f37df400640 1 -- 172.21.15.77:0/1886092695 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f37c00a2b80 msgr2=0x7f37c00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:10.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:10.827+0000 7f37df400640 1 --2- 172.21.15.77:0/1886092695 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f37c00a2b80 0x7f37c00a5040 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f37cc009bd0 tx=0x7f37cc005f90 comp rx=0 tx=0).stop 2024-09-10T09:53:10.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:10.827+0000 7f37df400640 1 -- 172.21.15.77:0/1886092695 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f37d8074f40 msgr2=0x7f37d819aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:10.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:10.827+0000 7f37df400640 1 --2- 172.21.15.77:0/1886092695 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f37d8074f40 0x7f37d819aba0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f37c400e9f0 tx=0x7f37c400edf0 comp rx=0 tx=0).stop 2024-09-10T09:53:10.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:10.827+0000 7f37df400640 1 -- 172.21.15.77:0/1886092695 shutdown_connections 2024-09-10T09:53:10.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:10.827+0000 7f37df400640 1 --2- 172.21.15.77:0/1886092695 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f37c00a2b80 0x7f37c00a5040 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:10.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:10.827+0000 7f37df400640 1 --2- 172.21.15.77:0/1886092695 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f37d810e3c0 0x7f37d819ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:10.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:10.827+0000 7f37df400640 1 --2- 172.21.15.77:0/1886092695 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f37d8074f40 0x7f37d819aba0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:10.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:10.827+0000 7f37df400640 1 --2- 172.21.15.77:0/1886092695 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37d8074620 0x7f37d819a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:10.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:10.827+0000 7f37df400640 1 -- 172.21.15.77:0/1886092695 >> 172.21.15.77:0/1886092695 conn(0x7f37d80fd8c0 msgr2=0x7f37d80fe520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:10.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:10.827+0000 7f37df400640 1 -- 172.21.15.77:0/1886092695 shutdown_connections 2024-09-10T09:53:10.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:10.827+0000 7f37df400640 1 -- 172.21.15.77:0/1886092695 wait complete. 2024-09-10T09:53:11.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:10 smithi134 ceph-mon[35674]: pgmap v329: 129 pgs: 129 active+clean; 581 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 103 B/s wr, 0 op/s; 447 B/s, 0 objects/s recovering 2024-09-10T09:53:11.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:10 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/1886092695' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-10T09:53:11.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:10 smithi120 ceph-mon[34770]: pgmap v329: 129 pgs: 129 active+clean; 581 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 103 B/s wr, 0 op/s; 447 B/s, 0 objects/s recovering 2024-09-10T09:53:11.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:10 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/1886092695' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-10T09:53:11.185 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:10 smithi077 ceph-mon[30319]: pgmap v329: 129 pgs: 129 active+clean; 581 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 103 B/s wr, 0 op/s; 447 B/s, 0 objects/s recovering 2024-09-10T09:53:11.185 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:10 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/1886092695' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-10T09:53:11.572 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-09-10T09:53:11.573 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph fs set cephfs max_mds 5 2024-09-10T09:53:11.946 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:53:12.085 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:11 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/1886092695' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-10T09:53:12.085 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:11 smithi077 ceph-mon[30319]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T09:53:12.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:11 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/1886092695' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-10T09:53:12.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:11 smithi134 ceph-mon[35674]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T09:53:12.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:11 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/1886092695' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-10T09:53:12.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:11 smithi120 ceph-mon[34770]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T09:53:12.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:12.517+0000 7f3e50e00640 1 -- 172.21.15.77:0/3860473324 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e4c101de0 msgr2=0x7f3e4c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:12.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:12.517+0000 7f3e50e00640 1 --2- 172.21.15.77:0/3860473324 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e4c101de0 0x7f3e4c1021c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c009900 tx=0x7f3e3c02a3f0 comp rx=0 tx=0).stop 2024-09-10T09:53:12.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:12.517+0000 7f3e50e00640 1 -- 172.21.15.77:0/3860473324 shutdown_connections 2024-09-10T09:53:12.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:12.517+0000 7f3e50e00640 1 --2- 172.21.15.77:0/3860473324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e4c106f10 0x7f3e4c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:12.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:12.517+0000 7f3e50e00640 1 --2- 172.21.15.77:0/3860473324 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3e4c102790 0x7f3e4c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:12.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:12.517+0000 7f3e50e00640 1 --2- 172.21.15.77:0/3860473324 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e4c101de0 0x7f3e4c1021c0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:12.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:12.517+0000 7f3e50e00640 1 -- 172.21.15.77:0/3860473324 >> 172.21.15.77:0/3860473324 conn(0x7f3e4c0fd8c0 msgr2=0x7f3e4c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:12.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:12.517+0000 7f3e50e00640 1 -- 172.21.15.77:0/3860473324 shutdown_connections 2024-09-10T09:53:12.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:12.517+0000 7f3e50e00640 1 -- 172.21.15.77:0/3860473324 wait complete. 2024-09-10T09:53:12.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:12.518+0000 7f3e50e00640 1 Processor -- start 2024-09-10T09:53:12.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:12.518+0000 7f3e50e00640 1 -- start start 2024-09-10T09:53:12.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:12.518+0000 7f3e50e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e4c101de0 0x7f3e4c198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:12.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:12.518+0000 7f3e50e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e4c102790 0x7f3e4c198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:12.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:12.518+0000 7f3e50e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3e4c106f10 0x7f3e4c19cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:12.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:12.518+0000 7f3e50e00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3e4c111f70 con 0x7f3e4c102790 2024-09-10T09:53:12.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:12.518+0000 7f3e50e00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3e4c111df0 con 0x7f3e4c106f10 2024-09-10T09:53:12.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:12.518+0000 7f3e50e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3e4c1120f0 con 0x7f3e4c101de0 2024-09-10T09:53:12.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:12.519+0000 7f3e49400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e4c102790 0x7f3e4c198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:12.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:12.519+0000 7f3e49400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e4c102790 0x7f3e4c198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57656/0 (socket says 172.21.15.77:57656) 2024-09-10T09:53:12.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:12.519+0000 7f3e49400640 1 -- 172.21.15.77:0/2580165984 learned_addr learned my addr 172.21.15.77:0/2580165984 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:53:12.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:12.519+0000 7f3e49e00640 1 --2- 172.21.15.77:0/2580165984 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e4c101de0 0x7f3e4c198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:12.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:12.519+0000 7f3e4a800640 1 --2- 172.21.15.77:0/2580165984 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3e4c106f10 0x7f3e4c19cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:12.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:12.519+0000 7f3e49400640 1 -- 172.21.15.77:0/2580165984 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e4c101de0 msgr2=0x7f3e4c198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:12.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:12.519+0000 7f3e49400640 1 --2- 172.21.15.77:0/2580165984 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e4c101de0 0x7f3e4c198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:12.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:12.519+0000 7f3e49400640 1 -- 172.21.15.77:0/2580165984 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3e4c106f10 msgr2=0x7f3e4c19cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:12.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:12.519+0000 7f3e49400640 1 --2- 172.21.15.77:0/2580165984 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3e4c106f10 0x7f3e4c19cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:12.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:12.519+0000 7f3e49400640 1 -- 172.21.15.77:0/2580165984 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e4c19d3c0 con 0x7f3e4c102790 2024-09-10T09:53:12.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:12.519+0000 7f3e49400640 1 --2- 172.21.15.77:0/2580165984 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e4c102790 0x7f3e4c198950 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3e3800e9f0 tx=0x7f3e3800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:12.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:12.520+0000 7f3e42a00640 1 -- 172.21.15.77:0/2580165984 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3800cd80 con 0x7f3e4c102790 2024-09-10T09:53:12.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:12.520+0000 7f3e50e00640 1 -- 172.21.15.77:0/2580165984 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e4c19d6b0 con 0x7f3e4c102790 2024-09-10T09:53:12.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:12.520+0000 7f3e42a00640 1 -- 172.21.15.77:0/2580165984 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e38010910 con 0x7f3e4c102790 2024-09-10T09:53:12.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:12.520+0000 7f3e50e00640 1 -- 172.21.15.77:0/2580165984 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e4c1a4f90 con 0x7f3e4c102790 2024-09-10T09:53:12.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:12.520+0000 7f3e42a00640 1 -- 172.21.15.77:0/2580165984 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e38018910 con 0x7f3e4c102790 2024-09-10T09:53:12.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:12.521+0000 7f3e50e00640 1 -- 172.21.15.77:0/2580165984 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e4c1021c0 con 0x7f3e4c102790 2024-09-10T09:53:12.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:12.523+0000 7f3e42a00640 1 -- 172.21.15.77:0/2580165984 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f3e38010ac0 con 0x7f3e4c102790 2024-09-10T09:53:12.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:12.525+0000 7f3e42a00640 1 --2- 172.21.15.77:0/2580165984 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f3e300a2b80 0x7f3e300a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:12.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:12.526+0000 7f3e42a00640 1 -- 172.21.15.77:0/2580165984 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f3e380cf780 con 0x7f3e4c102790 2024-09-10T09:53:12.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:12.526+0000 7f3e49e00640 1 --2- 172.21.15.77:0/2580165984 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f3e300a2b80 0x7f3e300a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:12.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:12.526+0000 7f3e49e00640 1 --2- 172.21.15.77:0/2580165984 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f3e300a2b80 0x7f3e300a5040 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c009230 tx=0x7f3e3c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:12.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:12.529+0000 7f3e42a00640 1 -- 172.21.15.77:0/2580165984 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3e3809bcd0 con 0x7f3e4c102790 2024-09-10T09:53:12.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:12.741+0000 7f3e50e00640 1 -- 172.21.15.77:0/2580165984 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7f3e4c02d7c0 con 0x7f3e4c102790 2024-09-10T09:53:12.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:12.831+0000 7f3e42a00640 1 -- 172.21.15.77:0/2580165984 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v19) ==== 105+0+0 (secure 0 0 0) 0x7f3e3809ffd0 con 0x7f3e4c102790 2024-09-10T09:53:12.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:12.834+0000 7f3e50e00640 1 -- 172.21.15.77:0/2580165984 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f3e300a2b80 msgr2=0x7f3e300a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:12.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:12.834+0000 7f3e50e00640 1 --2- 172.21.15.77:0/2580165984 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f3e300a2b80 0x7f3e300a5040 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c009230 tx=0x7f3e3c0058d0 comp rx=0 tx=0).stop 2024-09-10T09:53:12.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:12.834+0000 7f3e50e00640 1 -- 172.21.15.77:0/2580165984 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e4c102790 msgr2=0x7f3e4c198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:12.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:12.834+0000 7f3e50e00640 1 --2- 172.21.15.77:0/2580165984 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e4c102790 0x7f3e4c198950 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3e3800e9f0 tx=0x7f3e3800eeb0 comp rx=0 tx=0).stop 2024-09-10T09:53:12.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:12.834+0000 7f3e50e00640 1 -- 172.21.15.77:0/2580165984 shutdown_connections 2024-09-10T09:53:12.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:12.834+0000 7f3e50e00640 1 --2- 172.21.15.77:0/2580165984 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f3e300a2b80 0x7f3e300a5040 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:12.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:12.834+0000 7f3e50e00640 1 --2- 172.21.15.77:0/2580165984 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3e4c106f10 0x7f3e4c19cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:12.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:12.834+0000 7f3e50e00640 1 --2- 172.21.15.77:0/2580165984 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e4c102790 0x7f3e4c198950 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:12.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:12.834+0000 7f3e50e00640 1 --2- 172.21.15.77:0/2580165984 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e4c101de0 0x7f3e4c198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:12.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:12.834+0000 7f3e50e00640 1 -- 172.21.15.77:0/2580165984 >> 172.21.15.77:0/2580165984 conn(0x7f3e4c0fd8c0 msgr2=0x7f3e4c0fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:12.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:12.835+0000 7f3e50e00640 1 -- 172.21.15.77:0/2580165984 shutdown_connections 2024-09-10T09:53:12.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:12.835+0000 7f3e50e00640 1 -- 172.21.15.77:0/2580165984 wait complete. 2024-09-10T09:53:13.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:12 smithi134 ceph-mon[35674]: pgmap v330: 129 pgs: 129 active+clean; 581 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 94 B/s wr, 0 op/s; 406 B/s, 0 objects/s recovering 2024-09-10T09:53:13.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:12 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/2580165984' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-10T09:53:13.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:12 smithi120 ceph-mon[34770]: pgmap v330: 129 pgs: 129 active+clean; 581 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 94 B/s wr, 0 op/s; 406 B/s, 0 objects/s recovering 2024-09-10T09:53:13.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:12 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/2580165984' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-10T09:53:13.185 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:12 smithi077 ceph-mon[30319]: pgmap v330: 129 pgs: 129 active+clean; 581 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 94 B/s wr, 0 op/s; 406 B/s, 0 objects/s recovering 2024-09-10T09:53:13.185 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:12 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/2580165984' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-10T09:53:13.574 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-09-10T09:53:13.946 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:53:14.128 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:13 smithi077 ceph-mon[30319]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-10T09:53:14.128 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:13 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/2580165984' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-10T09:53:14.128 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:13 smithi077 ceph-mon[30319]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-10T09:53:14.128 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:13 smithi077 ceph-mon[30319]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T09:53:14.128 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:13 smithi077 ceph-mon[30319]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-09-10T09:53:14.128 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:13 smithi077 ceph-mon[30319]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-09-10T09:53:14.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:13 smithi134 ceph-mon[35674]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-10T09:53:14.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:13 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/2580165984' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-10T09:53:14.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:13 smithi134 ceph-mon[35674]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-10T09:53:14.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:13 smithi134 ceph-mon[35674]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T09:53:14.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:13 smithi134 ceph-mon[35674]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-09-10T09:53:14.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:13 smithi134 ceph-mon[35674]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-09-10T09:53:14.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:13 smithi120 ceph-mon[34770]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-10T09:53:14.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:13 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/2580165984' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-10T09:53:14.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:13 smithi120 ceph-mon[34770]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-10T09:53:14.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:13 smithi120 ceph-mon[34770]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T09:53:14.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:13 smithi120 ceph-mon[34770]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-09-10T09:53:14.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:13 smithi120 ceph-mon[34770]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-09-10T09:53:14.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:14.537+0000 7f5d55400640 1 -- 172.21.15.77:0/963937414 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d50102360 msgr2=0x7f5d5010e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:14.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:14.537+0000 7f5d55400640 1 --2- 172.21.15.77:0/963937414 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d50102360 0x7f5d5010e890 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5d48009cd0 tx=0x7f5d4802a8e0 comp rx=0 tx=0).stop 2024-09-10T09:53:14.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:14.537+0000 7f5d55400640 1 -- 172.21.15.77:0/963937414 shutdown_connections 2024-09-10T09:53:14.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:14.537+0000 7f5d55400640 1 --2- 172.21.15.77:0/963937414 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d50102360 0x7f5d5010e890 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:14.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:14.537+0000 7f5d55400640 1 --2- 172.21.15.77:0/963937414 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d501019c0 0x7f5d50101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:14.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:14.537+0000 7f5d55400640 1 --2- 172.21.15.77:0/963937414 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d501079c0 0x7f5d50107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:14.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:14.537+0000 7f5d55400640 1 -- 172.21.15.77:0/963937414 >> 172.21.15.77:0/963937414 conn(0x7f5d500fd740 msgr2=0x7f5d500ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:14.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:14.537+0000 7f5d55400640 1 -- 172.21.15.77:0/963937414 shutdown_connections 2024-09-10T09:53:14.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:14.537+0000 7f5d55400640 1 -- 172.21.15.77:0/963937414 wait complete. 2024-09-10T09:53:14.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:14.538+0000 7f5d55400640 1 Processor -- start 2024-09-10T09:53:14.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:14.538+0000 7f5d55400640 1 -- start start 2024-09-10T09:53:14.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:14.538+0000 7f5d55400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d501019c0 0x7f5d500744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:14.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:14.538+0000 7f5d55400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d50102360 0x7f5d50074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:14.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:14.539+0000 7f5d55400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d501079c0 0x7f5d50074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:14.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:14.539+0000 7f5d55400640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5d50111e10 con 0x7f5d50102360 2024-09-10T09:53:14.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:14.539+0000 7f5d55400640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5d50111c90 con 0x7f5d501079c0 2024-09-10T09:53:14.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:14.539+0000 7f5d55400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5d50111f90 con 0x7f5d501019c0 2024-09-10T09:53:14.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:14.539+0000 7f5d4fe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d501019c0 0x7f5d500744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:14.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:14.539+0000 7f5d47e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d50102360 0x7f5d50074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:14.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:14.539+0000 7f5d4fe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d501019c0 0x7f5d500744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:53272/0 (socket says 172.21.15.77:53272) 2024-09-10T09:53:14.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:14.539+0000 7f5d4fe00640 1 -- 172.21.15.77:0/3656826687 learned_addr learned my addr 172.21.15.77:0/3656826687 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:53:14.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:14.539+0000 7f5d54a00640 1 --2- 172.21.15.77:0/3656826687 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d501079c0 0x7f5d50074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:14.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:14.539+0000 7f5d47e00640 1 -- 172.21.15.77:0/3656826687 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d501019c0 msgr2=0x7f5d500744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:14.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:14.539+0000 7f5d47e00640 1 --2- 172.21.15.77:0/3656826687 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d501019c0 0x7f5d500744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:14.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:14.539+0000 7f5d47e00640 1 -- 172.21.15.77:0/3656826687 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d501079c0 msgr2=0x7f5d50074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:14.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:14.539+0000 7f5d47e00640 1 --2- 172.21.15.77:0/3656826687 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d501079c0 0x7f5d50074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:14.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:14.539+0000 7f5d47e00640 1 -- 172.21.15.77:0/3656826687 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d501a66f0 con 0x7f5d50102360 2024-09-10T09:53:14.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:14.539+0000 7f5d4fe00640 1 --2- 172.21.15.77:0/3656826687 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d501019c0 0x7f5d500744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:53:14.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:14.540+0000 7f5d47e00640 1 --2- 172.21.15.77:0/3656826687 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d50102360 0x7f5d50074a30 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5d3800e9f0 tx=0x7f5d3800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:14.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:14.540+0000 7f5d4d600640 1 -- 172.21.15.77:0/3656826687 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d3800ccc0 con 0x7f5d50102360 2024-09-10T09:53:14.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:14.540+0000 7f5d4d600640 1 -- 172.21.15.77:0/3656826687 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d38010910 con 0x7f5d50102360 2024-09-10T09:53:14.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:14.540+0000 7f5d4d600640 1 -- 172.21.15.77:0/3656826687 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d38018960 con 0x7f5d50102360 2024-09-10T09:53:14.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:14.540+0000 7f5d55400640 1 -- 172.21.15.77:0/3656826687 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d501a69e0 con 0x7f5d50102360 2024-09-10T09:53:14.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:14.540+0000 7f5d55400640 1 -- 172.21.15.77:0/3656826687 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d501a6ed0 con 0x7f5d50102360 2024-09-10T09:53:14.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:14.541+0000 7f5d55400640 1 -- 172.21.15.77:0/3656826687 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d50101e20 con 0x7f5d50102360 2024-09-10T09:53:14.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:14.543+0000 7f5d4d600640 1 -- 172.21.15.77:0/3656826687 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f5d38010ac0 con 0x7f5d50102360 2024-09-10T09:53:14.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:14.544+0000 7f5d4d600640 1 --2- 172.21.15.77:0/3656826687 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f5d340a2c00 0x7f5d340a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:14.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:14.544+0000 7f5d4d600640 1 -- 172.21.15.77:0/3656826687 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f5d380cf6f0 con 0x7f5d50102360 2024-09-10T09:53:14.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:14.545+0000 7f5d4fe00640 1 --2- 172.21.15.77:0/3656826687 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f5d340a2c00 0x7f5d340a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:14.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:14.545+0000 7f5d4fe00640 1 --2- 172.21.15.77:0/3656826687 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f5d340a2c00 0x7f5d340a50c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5d40011490 tx=0x7f5d40005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:14.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:14.548+0000 7f5d4d600640 1 -- 172.21.15.77:0/3656826687 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5d3809bc40 con 0x7f5d50102360 2024-09-10T09:53:14.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:14.770+0000 7f5d55400640 1 -- 172.21.15.77:0/3656826687 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f5d501a7100 con 0x7f5d50102360 2024-09-10T09:53:14.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:14.850+0000 7f5d4d600640 1 -- 172.21.15.77:0/3656826687 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v23) ==== 121+0+0 (secure 0 0 0) 0x7f5d3809ff40 con 0x7f5d50102360 2024-09-10T09:53:14.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:14.852+0000 7f5d55400640 1 -- 172.21.15.77:0/3656826687 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f5d340a2c00 msgr2=0x7f5d340a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:14.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:14.852+0000 7f5d55400640 1 --2- 172.21.15.77:0/3656826687 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f5d340a2c00 0x7f5d340a50c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5d40011490 tx=0x7f5d40005f90 comp rx=0 tx=0).stop 2024-09-10T09:53:14.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:14.852+0000 7f5d55400640 1 -- 172.21.15.77:0/3656826687 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d50102360 msgr2=0x7f5d50074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:14.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:14.852+0000 7f5d55400640 1 --2- 172.21.15.77:0/3656826687 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d50102360 0x7f5d50074a30 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5d3800e9f0 tx=0x7f5d3800eeb0 comp rx=0 tx=0).stop 2024-09-10T09:53:14.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:14.853+0000 7f5d55400640 1 -- 172.21.15.77:0/3656826687 shutdown_connections 2024-09-10T09:53:14.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:14.853+0000 7f5d55400640 1 --2- 172.21.15.77:0/3656826687 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f5d340a2c00 0x7f5d340a50c0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:14.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:14.853+0000 7f5d55400640 1 --2- 172.21.15.77:0/3656826687 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d501079c0 0x7f5d50074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:14.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:14.853+0000 7f5d55400640 1 --2- 172.21.15.77:0/3656826687 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d50102360 0x7f5d50074a30 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:14.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:14.853+0000 7f5d55400640 1 --2- 172.21.15.77:0/3656826687 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d501019c0 0x7f5d500744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:14.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:14.853+0000 7f5d55400640 1 -- 172.21.15.77:0/3656826687 >> 172.21.15.77:0/3656826687 conn(0x7f5d500fd740 msgr2=0x7f5d500fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:14.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:14.853+0000 7f5d55400640 1 -- 172.21.15.77:0/3656826687 shutdown_connections 2024-09-10T09:53:14.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:14.853+0000 7f5d55400640 1 -- 172.21.15.77:0/3656826687 wait complete. 2024-09-10T09:53:15.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:14 smithi134 ceph-mon[35674]: pgmap v331: 129 pgs: 129 active+clean; 582 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s; 320 B/s, 0 objects/s recovering 2024-09-10T09:53:15.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:14 smithi134 ceph-mon[35674]: mds.? [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] up:active 2024-09-10T09:53:15.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:14 smithi134 ceph-mon[35674]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-10T09:53:15.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:14 smithi134 ceph-mon[35674]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-09-10T09:53:15.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:14 smithi134 ceph-mon[35674]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-09-10T09:53:15.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:14 smithi134 ceph-mon[35674]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-09-10T09:53:15.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:14 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/3656826687' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-10T09:53:15.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:14 smithi120 ceph-mon[34770]: pgmap v331: 129 pgs: 129 active+clean; 582 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s; 320 B/s, 0 objects/s recovering 2024-09-10T09:53:15.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:14 smithi120 ceph-mon[34770]: mds.? [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] up:active 2024-09-10T09:53:15.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:14 smithi120 ceph-mon[34770]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-10T09:53:15.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:14 smithi120 ceph-mon[34770]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-09-10T09:53:15.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:14 smithi120 ceph-mon[34770]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-09-10T09:53:15.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:14 smithi120 ceph-mon[34770]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-09-10T09:53:15.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:14 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/3656826687' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-10T09:53:15.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:14 smithi077 ceph-mon[30319]: pgmap v331: 129 pgs: 129 active+clean; 582 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s; 320 B/s, 0 objects/s recovering 2024-09-10T09:53:15.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:14 smithi077 ceph-mon[30319]: mds.? [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] up:active 2024-09-10T09:53:15.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:14 smithi077 ceph-mon[30319]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-10T09:53:15.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:14 smithi077 ceph-mon[30319]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-09-10T09:53:15.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:14 smithi077 ceph-mon[30319]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-09-10T09:53:15.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:14 smithi077 ceph-mon[30319]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-09-10T09:53:15.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:14 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/3656826687' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-10T09:53:15.644 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-09-10T09:53:15.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:15 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/3656826687' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-10T09:53:15.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:15 smithi077 ceph-mon[30319]: mds.? [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] up:active 2024-09-10T09:53:15.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:15 smithi077 ceph-mon[30319]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-10T09:53:15.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:15 smithi077 ceph-mon[30319]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-09-10T09:53:15.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:15 smithi077 ceph-mon[30319]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=c=up:creating} 8 up:standby 2024-09-10T09:53:15.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:15 smithi077 ceph-mon[30319]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-09-10T09:53:16.027 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:53:16.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:15 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/3656826687' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-10T09:53:16.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:15 smithi134 ceph-mon[35674]: mds.? [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] up:active 2024-09-10T09:53:16.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:15 smithi134 ceph-mon[35674]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-10T09:53:16.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:15 smithi134 ceph-mon[35674]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-09-10T09:53:16.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:15 smithi134 ceph-mon[35674]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=c=up:creating} 8 up:standby 2024-09-10T09:53:16.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:15 smithi134 ceph-mon[35674]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-09-10T09:53:16.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:15 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/3656826687' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-10T09:53:16.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:15 smithi120 ceph-mon[34770]: mds.? [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] up:active 2024-09-10T09:53:16.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:15 smithi120 ceph-mon[34770]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-10T09:53:16.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:15 smithi120 ceph-mon[34770]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-09-10T09:53:16.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:15 smithi120 ceph-mon[34770]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=c=up:creating} 8 up:standby 2024-09-10T09:53:16.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:15 smithi120 ceph-mon[34770]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-09-10T09:53:16.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:16.612+0000 7fc5e8600640 1 -- 172.21.15.77:0/3402430480 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc5e010e3c0 msgr2=0x7fc5e0110810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:16.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:16.612+0000 7fc5e8600640 1 --2- 172.21.15.77:0/3402430480 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc5e010e3c0 0x7fc5e0110810 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc5d8009be0 tx=0x7fc5d802a750 comp rx=0 tx=0).stop 2024-09-10T09:53:16.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:16.612+0000 7fc5e8600640 1 -- 172.21.15.77:0/3402430480 shutdown_connections 2024-09-10T09:53:16.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:16.612+0000 7fc5e8600640 1 --2- 172.21.15.77:0/3402430480 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc5e010e3c0 0x7fc5e0110810 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:16.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:16.612+0000 7fc5e8600640 1 --2- 172.21.15.77:0/3402430480 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc5e0074f40 0x7fc5e010de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:16.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:16.612+0000 7fc5e8600640 1 --2- 172.21.15.77:0/3402430480 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc5e0074620 0x7fc5e0074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:16.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:16.612+0000 7fc5e8600640 1 -- 172.21.15.77:0/3402430480 >> 172.21.15.77:0/3402430480 conn(0x7fc5e00fd8c0 msgr2=0x7fc5e00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:16.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:16.612+0000 7fc5e8600640 1 -- 172.21.15.77:0/3402430480 shutdown_connections 2024-09-10T09:53:16.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:16.612+0000 7fc5e8600640 1 -- 172.21.15.77:0/3402430480 wait complete. 2024-09-10T09:53:16.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:16.613+0000 7fc5e8600640 1 Processor -- start 2024-09-10T09:53:16.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:16.613+0000 7fc5e8600640 1 -- start start 2024-09-10T09:53:16.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:16.614+0000 7fc5e8600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc5e0074620 0x7fc5e019a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:16.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:16.614+0000 7fc5e8600640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc5e0074f40 0x7fc5e019abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:16.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:16.614+0000 7fc5e8600640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc5e010e3c0 0x7fc5e019ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:16.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:16.614+0000 7fc5e8600640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e0114110 con 0x7fc5e0074f40 2024-09-10T09:53:16.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:16.614+0000 7fc5e8600640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e0113f90 con 0x7fc5e010e3c0 2024-09-10T09:53:16.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:16.614+0000 7fc5e8600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e0114290 con 0x7fc5e0074620 2024-09-10T09:53:16.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:16.614+0000 7fc5e5200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc5e0074f40 0x7fc5e019abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:16.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:16.614+0000 7fc5e6600640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc5e010e3c0 0x7fc5e019ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:16.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:16.614+0000 7fc5e5200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc5e0074f40 0x7fc5e019abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:48264/0 (socket says 172.21.15.77:48264) 2024-09-10T09:53:16.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:16.614+0000 7fc5e5200640 1 -- 172.21.15.77:0/1139343878 learned_addr learned my addr 172.21.15.77:0/1139343878 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:53:16.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:16.614+0000 7fc5e5c00640 1 --2- 172.21.15.77:0/1139343878 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc5e0074620 0x7fc5e019a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:16.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:16.614+0000 7fc5e6600640 1 -- 172.21.15.77:0/1139343878 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc5e0074620 msgr2=0x7fc5e019a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:16.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:16.614+0000 7fc5e6600640 1 --2- 172.21.15.77:0/1139343878 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc5e0074620 0x7fc5e019a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:16.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:16.614+0000 7fc5e6600640 1 -- 172.21.15.77:0/1139343878 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc5e0074f40 msgr2=0x7fc5e019abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:16.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:16.614+0000 7fc5e6600640 1 --2- 172.21.15.77:0/1139343878 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc5e0074f40 0x7fc5e019abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:16.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:16.614+0000 7fc5e6600640 1 -- 172.21.15.77:0/1139343878 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5e019f660 con 0x7fc5e010e3c0 2024-09-10T09:53:16.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:16.615+0000 7fc5e6600640 1 --2- 172.21.15.77:0/1139343878 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc5e010e3c0 0x7fc5e019ef80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc5d8034040 tx=0x7fc5d80305d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:16.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:16.615+0000 7fc5de000640 1 -- 172.21.15.77:0/1139343878 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d8041020 con 0x7fc5e010e3c0 2024-09-10T09:53:16.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:16.615+0000 7fc5de000640 1 -- 172.21.15.77:0/1139343878 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5d8037040 con 0x7fc5e010e3c0 2024-09-10T09:53:16.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:16.615+0000 7fc5e8600640 1 -- 172.21.15.77:0/1139343878 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5e019f8f0 con 0x7fc5e010e3c0 2024-09-10T09:53:16.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:16.615+0000 7fc5de000640 1 -- 172.21.15.77:0/1139343878 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d8036070 con 0x7fc5e010e3c0 2024-09-10T09:53:16.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:16.615+0000 7fc5e8600640 1 -- 172.21.15.77:0/1139343878 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5e01a7230 con 0x7fc5e010e3c0 2024-09-10T09:53:16.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:16.616+0000 7fc5e8600640 1 -- 172.21.15.77:0/1139343878 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5e0074a00 con 0x7fc5e010e3c0 2024-09-10T09:53:16.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:16.618+0000 7fc5de000640 1 -- 172.21.15.77:0/1139343878 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fc5d8044020 con 0x7fc5e010e3c0 2024-09-10T09:53:16.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:16.620+0000 7fc5de000640 1 --2- 172.21.15.77:0/1139343878 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fc5c80a2c00 0x7fc5c80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:16.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:16.620+0000 7fc5de000640 1 -- 172.21.15.77:0/1139343878 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7fc5d80f0080 con 0x7fc5e010e3c0 2024-09-10T09:53:16.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:16.620+0000 7fc5e5c00640 1 --2- 172.21.15.77:0/1139343878 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fc5c80a2c00 0x7fc5c80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:16.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:16.621+0000 7fc5e5c00640 1 --2- 172.21.15.77:0/1139343878 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fc5c80a2c00 0x7fc5c80a50c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc5d4005fd0 tx=0x7fc5d4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:16.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:16.622+0000 7fc5de000640 1 -- 172.21.15.77:0/1139343878 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc5d804b020 con 0x7fc5e010e3c0 2024-09-10T09:53:16.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:16.830+0000 7fc5e8600640 1 -- 172.21.15.77:0/1139343878 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7fc5e019ba10 con 0x7fc5e010e3c0 2024-09-10T09:53:16.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:16.874+0000 7fc5de000640 1 -- 172.21.15.77:0/1139343878 <== mon.2 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v27) ==== 115+0+0 (secure 0 0 0) 0x7fc5e019ba10 con 0x7fc5e010e3c0 2024-09-10T09:53:16.877 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:16.876+0000 7fc5e8600640 1 -- 172.21.15.77:0/1139343878 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fc5c80a2c00 msgr2=0x7fc5c80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:16.878 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:16.876+0000 7fc5e8600640 1 --2- 172.21.15.77:0/1139343878 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fc5c80a2c00 0x7fc5c80a50c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc5d4005fd0 tx=0x7fc5d4009260 comp rx=0 tx=0).stop 2024-09-10T09:53:16.878 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:16.876+0000 7fc5e8600640 1 -- 172.21.15.77:0/1139343878 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc5e010e3c0 msgr2=0x7fc5e019ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:16.878 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:16.876+0000 7fc5e8600640 1 --2- 172.21.15.77:0/1139343878 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc5e010e3c0 0x7fc5e019ef80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc5d8034040 tx=0x7fc5d80305d0 comp rx=0 tx=0).stop 2024-09-10T09:53:16.878 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:16.877+0000 7fc5e8600640 1 -- 172.21.15.77:0/1139343878 shutdown_connections 2024-09-10T09:53:16.878 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:16.877+0000 7fc5e8600640 1 --2- 172.21.15.77:0/1139343878 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fc5c80a2c00 0x7fc5c80a50c0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:16.878 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:16.877+0000 7fc5e8600640 1 --2- 172.21.15.77:0/1139343878 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc5e010e3c0 0x7fc5e019ef80 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:16.878 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:16.877+0000 7fc5e8600640 1 --2- 172.21.15.77:0/1139343878 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc5e0074f40 0x7fc5e019abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:16.878 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:16.877+0000 7fc5e8600640 1 --2- 172.21.15.77:0/1139343878 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc5e0074620 0x7fc5e019a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:16.878 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:16.877+0000 7fc5e8600640 1 -- 172.21.15.77:0/1139343878 >> 172.21.15.77:0/1139343878 conn(0x7fc5e00fd8c0 msgr2=0x7fc5e00fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:16.878 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:16.877+0000 7fc5e8600640 1 -- 172.21.15.77:0/1139343878 shutdown_connections 2024-09-10T09:53:16.878 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:16.877+0000 7fc5e8600640 1 -- 172.21.15.77:0/1139343878 wait complete. 2024-09-10T09:53:17.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:16 smithi134 ceph-mon[35674]: pgmap v332: 129 pgs: 129 active+clean; 583 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s wr, 1 op/s 2024-09-10T09:53:17.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:16 smithi134 ceph-mon[35674]: mds.? [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] up:active 2024-09-10T09:53:17.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:16 smithi134 ceph-mon[35674]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-10T09:53:17.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:16 smithi134 ceph-mon[35674]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-10T09:53:17.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:16 smithi134 ceph-mon[35674]: Cluster is now healthy 2024-09-10T09:53:17.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:16 smithi134 ceph-mon[35674]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=c=up:active} 8 up:standby 2024-09-10T09:53:17.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:16 smithi134 ceph-mon[35674]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-10T09:53:17.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:16 smithi134 ceph-mon[35674]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-09-10T09:53:17.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:16 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/1139343878' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-10T09:53:17.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:16 smithi134 ceph-mon[35674]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-10T09:53:17.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:16 smithi120 ceph-mon[34770]: pgmap v332: 129 pgs: 129 active+clean; 583 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s wr, 1 op/s 2024-09-10T09:53:17.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:16 smithi120 ceph-mon[34770]: mds.? [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] up:active 2024-09-10T09:53:17.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:16 smithi120 ceph-mon[34770]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-10T09:53:17.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:16 smithi120 ceph-mon[34770]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-10T09:53:17.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:16 smithi120 ceph-mon[34770]: Cluster is now healthy 2024-09-10T09:53:17.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:16 smithi120 ceph-mon[34770]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=c=up:active} 8 up:standby 2024-09-10T09:53:17.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:16 smithi120 ceph-mon[34770]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-10T09:53:17.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:16 smithi120 ceph-mon[34770]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-09-10T09:53:17.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:16 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/1139343878' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-10T09:53:17.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:16 smithi120 ceph-mon[34770]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-10T09:53:17.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:16 smithi077 ceph-mon[30319]: pgmap v332: 129 pgs: 129 active+clean; 583 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s wr, 1 op/s 2024-09-10T09:53:17.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:16 smithi077 ceph-mon[30319]: mds.? [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] up:active 2024-09-10T09:53:17.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:16 smithi077 ceph-mon[30319]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-10T09:53:17.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:16 smithi077 ceph-mon[30319]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-10T09:53:17.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:16 smithi077 ceph-mon[30319]: Cluster is now healthy 2024-09-10T09:53:17.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:16 smithi077 ceph-mon[30319]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=c=up:active} 8 up:standby 2024-09-10T09:53:17.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:16 smithi077 ceph-mon[30319]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-10T09:53:17.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:16 smithi077 ceph-mon[30319]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-09-10T09:53:17.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:16 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/1139343878' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-10T09:53:17.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:16 smithi077 ceph-mon[30319]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-10T09:53:17.683 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--size 25000000000'} subvols for filesystem 'cephfs' 2024-09-10T09:53:17.683 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-09-10T09:53:17.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:17 smithi077 ceph-mon[30319]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-10T09:53:17.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:17 smithi077 ceph-mon[30319]: mds.? [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] up:active 2024-09-10T09:53:17.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:17 smithi077 ceph-mon[30319]: fsmap cephfs:5 7 up:standby 5 up:active 2024-09-10T09:53:17.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:17 smithi077 ceph-mon[30319]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T09:53:17.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:17 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:53:18.091 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:53:18.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:17 smithi134 ceph-mon[35674]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-10T09:53:18.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:17 smithi134 ceph-mon[35674]: mds.? [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] up:active 2024-09-10T09:53:18.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:17 smithi134 ceph-mon[35674]: fsmap cephfs:5 7 up:standby 5 up:active 2024-09-10T09:53:18.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:17 smithi134 ceph-mon[35674]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T09:53:18.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:17 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:53:18.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:17 smithi120 ceph-mon[34770]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-10T09:53:18.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:17 smithi120 ceph-mon[34770]: mds.? [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] up:active 2024-09-10T09:53:18.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:17 smithi120 ceph-mon[34770]: fsmap cephfs:5 7 up:standby 5 up:active 2024-09-10T09:53:18.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:17 smithi120 ceph-mon[34770]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T09:53:18.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:17 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:53:19.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:18 smithi134 ceph-mon[35674]: pgmap v333: 129 pgs: 129 active+clean; 584 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 7 op/s 2024-09-10T09:53:19.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:18 smithi120 ceph-mon[34770]: pgmap v333: 129 pgs: 129 active+clean; 584 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 7 op/s 2024-09-10T09:53:19.187 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:18 smithi077 ceph-mon[30319]: pgmap v333: 129 pgs: 129 active+clean; 584 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 7 op/s 2024-09-10T09:53:19.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:19.445+0000 7fee2cc00640 1 -- 172.21.15.77:0/3953144943 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fee28102790 msgr2=0x7fee281067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:19.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:19.445+0000 7fee2cc00640 1 --2- 172.21.15.77:0/3953144943 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fee28102790 0x7fee281067e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fee14009900 tx=0x7fee1402a410 comp rx=0 tx=0).stop 2024-09-10T09:53:19.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:19.445+0000 7fee2cc00640 1 -- 172.21.15.77:0/3953144943 shutdown_connections 2024-09-10T09:53:19.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:19.445+0000 7fee2cc00640 1 --2- 172.21.15.77:0/3953144943 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fee28106f10 0x7fee2810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:19.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:19.445+0000 7fee2cc00640 1 --2- 172.21.15.77:0/3953144943 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fee28102790 0x7fee281067e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:19.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:19.445+0000 7fee2cc00640 1 --2- 172.21.15.77:0/3953144943 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fee28101de0 0x7fee281021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:19.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:19.445+0000 7fee2cc00640 1 -- 172.21.15.77:0/3953144943 >> 172.21.15.77:0/3953144943 conn(0x7fee280fd8c0 msgr2=0x7fee280ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:19.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:19.445+0000 7fee2cc00640 1 -- 172.21.15.77:0/3953144943 shutdown_connections 2024-09-10T09:53:19.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:19.445+0000 7fee2cc00640 1 -- 172.21.15.77:0/3953144943 wait complete. 2024-09-10T09:53:19.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:19.446+0000 7fee2cc00640 1 Processor -- start 2024-09-10T09:53:19.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:19.446+0000 7fee2cc00640 1 -- start start 2024-09-10T09:53:19.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:19.447+0000 7fee2cc00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fee28101de0 0x7fee2819a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:19.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:19.447+0000 7fee2cc00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fee28102790 0x7fee2819ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:19.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:19.447+0000 7fee2cc00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fee28106f10 0x7fee2819efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:19.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:19.447+0000 7fee2cc00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fee28111fc0 con 0x7fee28102790 2024-09-10T09:53:19.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:19.447+0000 7fee2cc00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fee28111e40 con 0x7fee28106f10 2024-09-10T09:53:19.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:19.447+0000 7fee2cc00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fee28112140 con 0x7fee28101de0 2024-09-10T09:53:19.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:19.447+0000 7fee25400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fee28102790 0x7fee2819ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:19.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:19.447+0000 7fee25e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fee28101de0 0x7fee2819a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:19.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:19.447+0000 7fee25400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fee28102790 0x7fee2819ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:48274/0 (socket says 172.21.15.77:48274) 2024-09-10T09:53:19.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:19.447+0000 7fee25400640 1 -- 172.21.15.77:0/2302886938 learned_addr learned my addr 172.21.15.77:0/2302886938 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:53:19.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:19.447+0000 7fee26800640 1 --2- 172.21.15.77:0/2302886938 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fee28106f10 0x7fee2819efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:19.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:19.447+0000 7fee25e00640 1 -- 172.21.15.77:0/2302886938 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fee28106f10 msgr2=0x7fee2819efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:19.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:19.447+0000 7fee25e00640 1 --2- 172.21.15.77:0/2302886938 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fee28106f10 0x7fee2819efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:19.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:19.447+0000 7fee25e00640 1 -- 172.21.15.77:0/2302886938 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fee28102790 msgr2=0x7fee2819ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:19.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:19.447+0000 7fee25e00640 1 --2- 172.21.15.77:0/2302886938 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fee28102790 0x7fee2819ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:19.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:19.447+0000 7fee25e00640 1 -- 172.21.15.77:0/2302886938 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee2819f690 con 0x7fee28101de0 2024-09-10T09:53:19.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:19.447+0000 7fee25400640 1 --2- 172.21.15.77:0/2302886938 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fee28102790 0x7fee2819ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:53:19.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:19.447+0000 7fee26800640 1 --2- 172.21.15.77:0/2302886938 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fee28106f10 0x7fee2819efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:53:19.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:19.448+0000 7fee25e00640 1 --2- 172.21.15.77:0/2302886938 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fee28101de0 0x7fee2819a6e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fee1800e9f0 tx=0x7fee1800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:19.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:19.448+0000 7fee1ea00640 1 -- 172.21.15.77:0/2302886938 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee1800ccc0 con 0x7fee28101de0 2024-09-10T09:53:19.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:19.448+0000 7fee1ea00640 1 -- 172.21.15.77:0/2302886938 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee18010910 con 0x7fee28101de0 2024-09-10T09:53:19.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:19.448+0000 7fee2cc00640 1 -- 172.21.15.77:0/2302886938 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee2819f980 con 0x7fee28101de0 2024-09-10T09:53:19.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:19.448+0000 7fee1ea00640 1 -- 172.21.15.77:0/2302886938 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee18018960 con 0x7fee28101de0 2024-09-10T09:53:19.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:19.448+0000 7fee2cc00640 1 -- 172.21.15.77:0/2302886938 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee281a7260 con 0x7fee28101de0 2024-09-10T09:53:19.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:19.449+0000 7fee2cc00640 1 -- 172.21.15.77:0/2302886938 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee281021c0 con 0x7fee28101de0 2024-09-10T09:53:19.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:19.451+0000 7fee1ea00640 1 -- 172.21.15.77:0/2302886938 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fee18010ac0 con 0x7fee28101de0 2024-09-10T09:53:19.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:19.452+0000 7fee1ea00640 1 --2- 172.21.15.77:0/2302886938 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fee0c0a2c00 0x7fee0c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:19.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:19.452+0000 7fee1ea00640 1 -- 172.21.15.77:0/2302886938 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7fee180cf6f0 con 0x7fee28101de0 2024-09-10T09:53:19.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:19.453+0000 7fee25400640 1 --2- 172.21.15.77:0/2302886938 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fee0c0a2c00 0x7fee0c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:19.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:19.453+0000 7fee25400640 1 --2- 172.21.15.77:0/2302886938 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fee0c0a2c00 0x7fee0c0a50c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fee140096c0 tx=0x7fee140046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:19.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:19.456+0000 7fee1ea00640 1 -- 172.21.15.77:0/2302886938 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fee1809bc40 con 0x7fee28101de0 2024-09-10T09:53:19.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:19.667+0000 7fee2cc00640 1 -- 172.21.15.77:0/2302886938 --> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fee28075430 con 0x7fee0c0a2c00 2024-09-10T09:53:19.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:19.697+0000 7fee1ea00640 1 -- 172.21.15.77:0/2302886938 <== mgr.14150 v2:172.21.15.77:6800/4261324650 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fee28075430 con 0x7fee0c0a2c00 2024-09-10T09:53:19.699 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:53:19 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: 2024-09-10T09:53:19.675+0000 7f2807800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T09:53:19.699 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:53:19 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: 2024-09-10T09:53:19.675+0000 7f2807800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T09:53:19.699 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:53:19 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: 2024-09-10T09:53:19.675+0000 7f2807800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T09:53:19.699 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:53:19 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: 2024-09-10T09:53:19.675+0000 7f2807800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T09:53:19.700 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 09:53:19 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: 2024-09-10T09:53:19.675+0000 7f2807800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T09:53:19.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:19.700+0000 7fee2cc00640 1 -- 172.21.15.77:0/2302886938 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fee0c0a2c00 msgr2=0x7fee0c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:19.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:19.700+0000 7fee2cc00640 1 --2- 172.21.15.77:0/2302886938 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fee0c0a2c00 0x7fee0c0a50c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fee140096c0 tx=0x7fee140046c0 comp rx=0 tx=0).stop 2024-09-10T09:53:19.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:19.700+0000 7fee2cc00640 1 -- 172.21.15.77:0/2302886938 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fee28101de0 msgr2=0x7fee2819a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:19.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:19.700+0000 7fee2cc00640 1 --2- 172.21.15.77:0/2302886938 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fee28101de0 0x7fee2819a6e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fee1800e9f0 tx=0x7fee1800eeb0 comp rx=0 tx=0).stop 2024-09-10T09:53:19.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:19.700+0000 7fee2cc00640 1 -- 172.21.15.77:0/2302886938 shutdown_connections 2024-09-10T09:53:19.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:19.700+0000 7fee2cc00640 1 --2- 172.21.15.77:0/2302886938 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fee0c0a2c00 0x7fee0c0a50c0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:19.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:19.700+0000 7fee2cc00640 1 --2- 172.21.15.77:0/2302886938 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fee28106f10 0x7fee2819efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:19.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:19.700+0000 7fee2cc00640 1 --2- 172.21.15.77:0/2302886938 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fee28102790 0x7fee2819ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:19.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:19.700+0000 7fee2cc00640 1 --2- 172.21.15.77:0/2302886938 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fee28101de0 0x7fee2819a6e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:19.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:19.700+0000 7fee2cc00640 1 -- 172.21.15.77:0/2302886938 >> 172.21.15.77:0/2302886938 conn(0x7fee280fd8c0 msgr2=0x7fee280feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:19.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:19.700+0000 7fee2cc00640 1 -- 172.21.15.77:0/2302886938 shutdown_connections 2024-09-10T09:53:19.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:19.700+0000 7fee2cc00640 1 -- 172.21.15.77:0/2302886938 wait complete. 2024-09-10T09:53:20.045 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:19 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:53:20.045 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:19 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:53:20.045 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:19 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:53:20.046 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:19 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:53:20.046 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:19 smithi077 ceph-mon[30319]: from='client.24712 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:53:20.046 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:19 smithi077 ceph-mon[30319]: pgmap v334: 129 pgs: 129 active+clean; 587 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 4.0 KiB/s wr, 15 op/s 2024-09-10T09:53:20.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:19 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:53:20.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:19 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:53:20.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:19 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:53:20.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:19 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:53:20.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:19 smithi134 ceph-mon[35674]: from='client.24712 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:53:20.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:19 smithi134 ceph-mon[35674]: pgmap v334: 129 pgs: 129 active+clean; 587 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 4.0 KiB/s wr, 15 op/s 2024-09-10T09:53:20.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:19 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:53:20.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:19 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:53:20.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:19 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:53:20.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:19 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:53:20.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:19 smithi120 ceph-mon[34770]: from='client.24712 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:53:20.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:19 smithi120 ceph-mon[34770]: pgmap v334: 129 pgs: 129 active+clean; 587 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 4.0 KiB/s wr, 15 op/s 2024-09-10T09:53:20.458 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-09-10T09:53:20.847 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:53:21.213 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:20 smithi077 ceph-mon[30319]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-09-10T09:53:21.213 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:20 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:53:21.213 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:20 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:53:21.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:20 smithi134 ceph-mon[35674]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-09-10T09:53:21.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:20 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:53:21.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:20 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:53:21.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:20 smithi120 ceph-mon[34770]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-09-10T09:53:21.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:20 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:53:21.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:20 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:53:21.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:21.433+0000 7f639b200640 1 -- 172.21.15.77:0/124338671 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f639c104800 msgr2=0x7f639c108960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:21.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:21.433+0000 7f639b200640 1 --2- 172.21.15.77:0/124338671 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f639c104800 0x7f639c108960 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6388009900 tx=0x7f638802a3f0 comp rx=0 tx=0).stop 2024-09-10T09:53:21.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:21.433+0000 7f639b200640 1 -- 172.21.15.77:0/124338671 shutdown_connections 2024-09-10T09:53:21.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:21.433+0000 7f639b200640 1 --2- 172.21.15.77:0/124338671 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f639c109090 0x7f639c110c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:21.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:21.433+0000 7f639b200640 1 --2- 172.21.15.77:0/124338671 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f639c104800 0x7f639c108960 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:21.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:21.433+0000 7f639b200640 1 --2- 172.21.15.77:0/124338671 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f639c103e50 0x7f639c104230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:21.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:21.433+0000 7f639b200640 1 -- 172.21.15.77:0/124338671 >> 172.21.15.77:0/124338671 conn(0x7f639c0ff8b0 msgr2=0x7f639c101cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:21.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:21.433+0000 7f639b200640 1 -- 172.21.15.77:0/124338671 shutdown_connections 2024-09-10T09:53:21.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:21.433+0000 7f639b200640 1 -- 172.21.15.77:0/124338671 wait complete. 2024-09-10T09:53:21.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:21.434+0000 7f639b200640 1 Processor -- start 2024-09-10T09:53:21.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:21.434+0000 7f639b200640 1 -- start start 2024-09-10T09:53:21.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:21.435+0000 7f639b200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f639c103e50 0x7f639c19a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:21.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:21.435+0000 7f639b200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f639c104800 0x7f639c19aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:21.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:21.435+0000 7f639b200640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f639c109090 0x7f639c19ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:21.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:21.435+0000 7f639b200640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f639c114100 con 0x7f639c104800 2024-09-10T09:53:21.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:21.435+0000 7f639b200640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f639c113f80 con 0x7f639c109090 2024-09-10T09:53:21.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:21.435+0000 7f639b200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f639c114280 con 0x7f639c103e50 2024-09-10T09:53:21.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:21.435+0000 7f6399400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f639c104800 0x7f639c19aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:21.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:21.435+0000 7f6399400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f639c104800 0x7f639c19aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:48326/0 (socket says 172.21.15.77:48326) 2024-09-10T09:53:21.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:21.435+0000 7f6399400640 1 -- 172.21.15.77:0/2208956015 learned_addr learned my addr 172.21.15.77:0/2208956015 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:53:21.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:21.435+0000 7f639a800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f639c109090 0x7f639c19ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:21.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:21.435+0000 7f6399e00640 1 --2- 172.21.15.77:0/2208956015 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f639c103e50 0x7f639c19a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:21.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:21.435+0000 7f639a800640 1 -- 172.21.15.77:0/2208956015 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f639c103e50 msgr2=0x7f639c19a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:21.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:21.435+0000 7f639a800640 1 --2- 172.21.15.77:0/2208956015 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f639c103e50 0x7f639c19a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:21.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:21.435+0000 7f639a800640 1 -- 172.21.15.77:0/2208956015 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f639c104800 msgr2=0x7f639c19aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:21.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:21.435+0000 7f639a800640 1 --2- 172.21.15.77:0/2208956015 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f639c104800 0x7f639c19aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:21.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:21.435+0000 7f639a800640 1 -- 172.21.15.77:0/2208956015 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f639c19f550 con 0x7f639c109090 2024-09-10T09:53:21.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:21.436+0000 7f639a800640 1 --2- 172.21.15.77:0/2208956015 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f639c109090 0x7f639c19ee70 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6394009fd0 tx=0x7f639400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:21.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:21.436+0000 7f6392a00640 1 -- 172.21.15.77:0/2208956015 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f639400ce20 con 0x7f639c109090 2024-09-10T09:53:21.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:21.436+0000 7f6392a00640 1 -- 172.21.15.77:0/2208956015 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6394005020 con 0x7f639c109090 2024-09-10T09:53:21.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:21.436+0000 7f639b200640 1 -- 172.21.15.77:0/2208956015 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f639c19f840 con 0x7f639c109090 2024-09-10T09:53:21.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:21.436+0000 7f6392a00640 1 -- 172.21.15.77:0/2208956015 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6394010720 con 0x7f639c109090 2024-09-10T09:53:21.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:21.436+0000 7f639b200640 1 -- 172.21.15.77:0/2208956015 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f639c1a7120 con 0x7f639c109090 2024-09-10T09:53:21.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:21.437+0000 7f639b200640 1 -- 172.21.15.77:0/2208956015 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f639c105330 con 0x7f639c109090 2024-09-10T09:53:21.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:21.439+0000 7f6392a00640 1 -- 172.21.15.77:0/2208956015 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f6394005540 con 0x7f639c109090 2024-09-10T09:53:21.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:21.440+0000 7f6392a00640 1 --2- 172.21.15.77:0/2208956015 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f63800a2c10 0x7f63800a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:21.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:21.440+0000 7f6392a00640 1 -- 172.21.15.77:0/2208956015 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f63940cfbc0 con 0x7f639c109090 2024-09-10T09:53:21.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:21.441+0000 7f6399e00640 1 --2- 172.21.15.77:0/2208956015 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f63800a2c10 0x7f63800a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:21.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:21.441+0000 7f6399e00640 1 --2- 172.21.15.77:0/2208956015 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f63800a2c10 0x7f63800a50d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f638c009bd0 tx=0x7f638c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:21.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:21.444+0000 7f6392a00640 1 -- 172.21.15.77:0/2208956015 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f639409c660 con 0x7f639c109090 2024-09-10T09:53:21.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:21.657+0000 7f639b200640 1 -- 172.21.15.77:0/2208956015 --> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f639c10c340 con 0x7f63800a2c10 2024-09-10T09:53:21.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:21.684+0000 7f6392a00640 1 -- 172.21.15.77:0/2208956015 <== mgr.14150 v2:172.21.15.77:6800/4261324650 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f638c004030 con 0x7f63800a2c10 2024-09-10T09:53:21.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:21.686+0000 7f639b200640 1 -- 172.21.15.77:0/2208956015 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f63800a2c10 msgr2=0x7f63800a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:21.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:21.686+0000 7f639b200640 1 --2- 172.21.15.77:0/2208956015 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f63800a2c10 0x7f63800a50d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f638c009bd0 tx=0x7f638c005f90 comp rx=0 tx=0).stop 2024-09-10T09:53:21.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:21.686+0000 7f639b200640 1 -- 172.21.15.77:0/2208956015 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f639c109090 msgr2=0x7f639c19ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:21.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:21.686+0000 7f639b200640 1 --2- 172.21.15.77:0/2208956015 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f639c109090 0x7f639c19ee70 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6394009fd0 tx=0x7f639400ef10 comp rx=0 tx=0).stop 2024-09-10T09:53:21.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:21.686+0000 7f639b200640 1 -- 172.21.15.77:0/2208956015 shutdown_connections 2024-09-10T09:53:21.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:21.686+0000 7f639b200640 1 --2- 172.21.15.77:0/2208956015 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f63800a2c10 0x7f63800a50d0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:21.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:21.686+0000 7f639b200640 1 --2- 172.21.15.77:0/2208956015 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f639c109090 0x7f639c19ee70 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:21.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:21.686+0000 7f639b200640 1 --2- 172.21.15.77:0/2208956015 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f639c104800 0x7f639c19aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:21.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:21.686+0000 7f639b200640 1 --2- 172.21.15.77:0/2208956015 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f639c103e50 0x7f639c19a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:21.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:21.686+0000 7f639b200640 1 -- 172.21.15.77:0/2208956015 >> 172.21.15.77:0/2208956015 conn(0x7f639c0ff8b0 msgr2=0x7f639c100b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:21.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:21.686+0000 7f639b200640 1 -- 172.21.15.77:0/2208956015 shutdown_connections 2024-09-10T09:53:21.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:21.686+0000 7f639b200640 1 -- 172.21.15.77:0/2208956015 wait complete. 2024-09-10T09:53:22.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:21 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:53:22.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:21 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:53:22.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:21 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:53:22.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:21 smithi134 ceph-mon[35674]: from='client.24740 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:53:22.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:21 smithi134 ceph-mon[35674]: pgmap v335: 129 pgs: 129 active+clean; 587 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 4.1 KiB/s wr, 16 op/s 2024-09-10T09:53:22.398 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:21 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:53:22.398 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:21 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:53:22.398 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:21 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:53:22.398 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:21 smithi077 ceph-mon[30319]: from='client.24740 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:53:22.398 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:21 smithi077 ceph-mon[30319]: pgmap v335: 129 pgs: 129 active+clean; 587 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 4.1 KiB/s wr, 16 op/s 2024-09-10T09:53:22.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:21 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:53:22.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:21 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:53:22.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:21 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:53:22.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:21 smithi120 ceph-mon[34770]: from='client.24740 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:53:22.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:21 smithi120 ceph-mon[34770]: pgmap v335: 129 pgs: 129 active+clean; 587 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 4.1 KiB/s wr, 16 op/s 2024-09-10T09:53:22.434 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-09-10T09:53:22.804 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:53:23.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:23.339+0000 7f7915800640 1 -- 172.21.15.77:0/1248200286 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7910104010 msgr2=0x7f79101043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:23.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:23.339+0000 7f7915800640 1 --2- 172.21.15.77:0/1248200286 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7910104010 0x7f79101043f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f7900009900 tx=0x7f790002a3f0 comp rx=0 tx=0).stop 2024-09-10T09:53:23.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:23.339+0000 7f7915800640 1 -- 172.21.15.77:0/1248200286 shutdown_connections 2024-09-10T09:53:23.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:23.339+0000 7f7915800640 1 --2- 172.21.15.77:0/1248200286 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7910109140 0x7f7910110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:23.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:23.339+0000 7f7915800640 1 --2- 172.21.15.77:0/1248200286 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f79101049c0 0x7f7910108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:23.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:23.339+0000 7f7915800640 1 --2- 172.21.15.77:0/1248200286 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7910104010 0x7f79101043f0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:23.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:23.339+0000 7f7915800640 1 -- 172.21.15.77:0/1248200286 >> 172.21.15.77:0/1248200286 conn(0x7f79100ffa50 msgr2=0x7f7910101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:23.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:23.339+0000 7f7915800640 1 -- 172.21.15.77:0/1248200286 shutdown_connections 2024-09-10T09:53:23.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:23.339+0000 7f7915800640 1 -- 172.21.15.77:0/1248200286 wait complete. 2024-09-10T09:53:23.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:23.340+0000 7f7915800640 1 Processor -- start 2024-09-10T09:53:23.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:23.340+0000 7f7915800640 1 -- start start 2024-09-10T09:53:23.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:23.340+0000 7f7915800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7910104010 0x7f791019a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:23.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:23.340+0000 7f7915800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79101049c0 0x7f791019ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:23.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:23.340+0000 7f7915800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7910109140 0x7f791019ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:23.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:23.340+0000 7f7915800640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f79101141c0 con 0x7f7910104010 2024-09-10T09:53:23.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:23.340+0000 7f7915800640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7910114040 con 0x7f7910109140 2024-09-10T09:53:23.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:23.340+0000 7f7915800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7910114340 con 0x7f79101049c0 2024-09-10T09:53:23.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:23.341+0000 7f790ea00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7910104010 0x7f791019a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:23.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:23.341+0000 7f790ea00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7910104010 0x7f791019a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:51104/0 (socket says 172.21.15.77:51104) 2024-09-10T09:53:23.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:23.341+0000 7f790ea00640 1 -- 172.21.15.77:0/3770448671 learned_addr learned my addr 172.21.15.77:0/3770448671 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:53:23.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:23.341+0000 7f790f400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7910109140 0x7f791019ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:23.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:23.341+0000 7f7907e00640 1 --2- 172.21.15.77:0/3770448671 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79101049c0 0x7f791019ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:23.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:23.341+0000 7f790f400640 1 -- 172.21.15.77:0/3770448671 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79101049c0 msgr2=0x7f791019ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:23.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:23.341+0000 7f790f400640 1 --2- 172.21.15.77:0/3770448671 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79101049c0 0x7f791019ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:23.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:23.341+0000 7f790f400640 1 -- 172.21.15.77:0/3770448671 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7910104010 msgr2=0x7f791019a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:23.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:23.341+0000 7f790f400640 1 --2- 172.21.15.77:0/3770448671 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7910104010 0x7f791019a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:23.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:23.341+0000 7f790f400640 1 -- 172.21.15.77:0/3770448671 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f791019f5f0 con 0x7f7910109140 2024-09-10T09:53:23.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:23.341+0000 7f790ea00640 1 --2- 172.21.15.77:0/3770448671 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7910104010 0x7f791019a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:53:23.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:23.342+0000 7f790f400640 1 --2- 172.21.15.77:0/3770448671 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7910109140 0x7f791019ef10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7908009fd0 tx=0x7f790800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:23.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:23.342+0000 7f7907400640 1 -- 172.21.15.77:0/3770448671 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f790800ce20 con 0x7f7910109140 2024-09-10T09:53:23.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:23.342+0000 7f7915800640 1 -- 172.21.15.77:0/3770448671 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f791019f8e0 con 0x7f7910109140 2024-09-10T09:53:23.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:23.342+0000 7f7907400640 1 -- 172.21.15.77:0/3770448671 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7908005020 con 0x7f7910109140 2024-09-10T09:53:23.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:23.342+0000 7f7907400640 1 -- 172.21.15.77:0/3770448671 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7908010720 con 0x7f7910109140 2024-09-10T09:53:23.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:23.342+0000 7f7915800640 1 -- 172.21.15.77:0/3770448671 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79101a71c0 con 0x7f7910109140 2024-09-10T09:53:23.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:23.343+0000 7f7915800640 1 -- 172.21.15.77:0/3770448671 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79101043f0 con 0x7f7910109140 2024-09-10T09:53:23.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:23.345+0000 7f7907400640 1 -- 172.21.15.77:0/3770448671 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f7908005540 con 0x7f7910109140 2024-09-10T09:53:23.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:23.346+0000 7f7907400640 1 --2- 172.21.15.77:0/3770448671 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f78f40a2c10 0x7f78f40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:23.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:23.346+0000 7f7907400640 1 -- 172.21.15.77:0/3770448671 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f79080cfb20 con 0x7f7910109140 2024-09-10T09:53:23.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:23.346+0000 7f790ea00640 1 --2- 172.21.15.77:0/3770448671 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f78f40a2c10 0x7f78f40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:23.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:23.347+0000 7f790ea00640 1 --2- 172.21.15.77:0/3770448671 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f78f40a2c10 0x7f78f40a50d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f79000096c0 tx=0x7f79000046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:23.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:23.350+0000 7f7907400640 1 -- 172.21.15.77:0/3770448671 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f790809c560 con 0x7f7910109140 2024-09-10T09:53:23.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:23.552+0000 7f7915800640 1 -- 172.21.15.77:0/3770448671 --> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f7910062630 con 0x7f78f40a2c10 2024-09-10T09:53:23.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:23.558+0000 7f7907400640 1 -- 172.21.15.77:0/3770448671 <== mgr.14150 v2:172.21.15.77:6800/4261324650 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f7910062630 con 0x7f78f40a2c10 2024-09-10T09:53:23.559 INFO:teuthology.orchestra.run.smithi077.stdout:/volumes/qa/sv_0/fc494107-dc51-49b3-9fbc-6ee4ea1914ef 2024-09-10T09:53:23.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:23.560+0000 7f7915800640 1 -- 172.21.15.77:0/3770448671 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f78f40a2c10 msgr2=0x7f78f40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:23.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:23.560+0000 7f7915800640 1 --2- 172.21.15.77:0/3770448671 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f78f40a2c10 0x7f78f40a50d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f79000096c0 tx=0x7f79000046c0 comp rx=0 tx=0).stop 2024-09-10T09:53:23.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:23.560+0000 7f7915800640 1 -- 172.21.15.77:0/3770448671 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7910109140 msgr2=0x7f791019ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:23.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:23.560+0000 7f7915800640 1 --2- 172.21.15.77:0/3770448671 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7910109140 0x7f791019ef10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7908009fd0 tx=0x7f790800ef10 comp rx=0 tx=0).stop 2024-09-10T09:53:23.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:23.561+0000 7f7915800640 1 -- 172.21.15.77:0/3770448671 shutdown_connections 2024-09-10T09:53:23.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:23.561+0000 7f7915800640 1 --2- 172.21.15.77:0/3770448671 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f78f40a2c10 0x7f78f40a50d0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:23.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:23.561+0000 7f7915800640 1 --2- 172.21.15.77:0/3770448671 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7910109140 0x7f791019ef10 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:23.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:23.561+0000 7f7915800640 1 --2- 172.21.15.77:0/3770448671 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79101049c0 0x7f791019ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:23.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:23.561+0000 7f7915800640 1 --2- 172.21.15.77:0/3770448671 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7910104010 0x7f791019a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:23.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:23.561+0000 7f7915800640 1 -- 172.21.15.77:0/3770448671 >> 172.21.15.77:0/3770448671 conn(0x7f79100ffa50 msgr2=0x7f7910100dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:23.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:23.561+0000 7f7915800640 1 -- 172.21.15.77:0/3770448671 shutdown_connections 2024-09-10T09:53:23.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:23.561+0000 7f7915800640 1 -- 172.21.15.77:0/3770448671 wait complete. 2024-09-10T09:53:24.319 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-09-10T09:53:24.686 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:53:25.014 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:24 smithi077 ceph-mon[30319]: from='client.24746 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:53:25.014 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:24 smithi077 ceph-mon[30319]: pgmap v336: 129 pgs: 129 active+clean; 607 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 6.2 KiB/s wr, 21 op/s 2024-09-10T09:53:25.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:24 smithi134 ceph-mon[35674]: from='client.24746 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:53:25.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:24 smithi134 ceph-mon[35674]: pgmap v336: 129 pgs: 129 active+clean; 607 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 6.2 KiB/s wr, 21 op/s 2024-09-10T09:53:25.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:24 smithi120 ceph-mon[34770]: from='client.24746 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:53:25.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:24 smithi120 ceph-mon[34770]: pgmap v336: 129 pgs: 129 active+clean; 607 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 6.2 KiB/s wr, 21 op/s 2024-09-10T09:53:25.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:25.259+0000 7f6888800640 1 -- 172.21.15.77:0/256952333 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f688010e420 msgr2=0x7f6880110810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:25.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:25.259+0000 7f6888800640 1 --2- 172.21.15.77:0/256952333 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f688010e420 0x7f6880110810 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f687c009ca0 tx=0x7f687c02a950 comp rx=0 tx=0).stop 2024-09-10T09:53:25.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:25.259+0000 7f6888800640 1 -- 172.21.15.77:0/256952333 shutdown_connections 2024-09-10T09:53:25.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:25.259+0000 7f6888800640 1 --2- 172.21.15.77:0/256952333 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f688010e420 0x7f6880110810 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:25.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:25.259+0000 7f6888800640 1 --2- 172.21.15.77:0/256952333 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6880074ee0 0x7f688010dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:25.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:25.259+0000 7f6888800640 1 --2- 172.21.15.77:0/256952333 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f68800745c0 0x7f68800749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:25.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:25.259+0000 7f6888800640 1 -- 172.21.15.77:0/256952333 >> 172.21.15.77:0/256952333 conn(0x7f68800fd8c0 msgr2=0x7f68800ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:25.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:25.259+0000 7f6888800640 1 -- 172.21.15.77:0/256952333 shutdown_connections 2024-09-10T09:53:25.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:25.259+0000 7f6888800640 1 -- 172.21.15.77:0/256952333 wait complete. 2024-09-10T09:53:25.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:25.260+0000 7f6888800640 1 Processor -- start 2024-09-10T09:53:25.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:25.260+0000 7f6888800640 1 -- start start 2024-09-10T09:53:25.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:25.261+0000 7f6888800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f68800745c0 0x7f688019a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:25.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:25.261+0000 7f6888800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6880074ee0 0x7f688019ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:25.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:25.261+0000 7f6888800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f688010e420 0x7f688019eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:25.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:25.261+0000 7f6888800640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6880114170 con 0x7f68800745c0 2024-09-10T09:53:25.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:25.261+0000 7f6888800640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6880113ff0 con 0x7f6880074ee0 2024-09-10T09:53:25.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:25.261+0000 7f6888800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f68801142f0 con 0x7f688010e420 2024-09-10T09:53:25.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:25.261+0000 7f6885e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f68800745c0 0x7f688019a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:25.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:25.261+0000 7f6885400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6880074ee0 0x7f688019ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:25.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:25.261+0000 7f6885e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f68800745c0 0x7f688019a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:51132/0 (socket says 172.21.15.77:51132) 2024-09-10T09:53:25.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:25.261+0000 7f6885e00640 1 -- 172.21.15.77:0/3885178568 learned_addr learned my addr 172.21.15.77:0/3885178568 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:53:25.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:25.261+0000 7f6886800640 1 --2- 172.21.15.77:0/3885178568 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f688010e420 0x7f688019eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:25.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:25.261+0000 7f6885400640 1 -- 172.21.15.77:0/3885178568 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f688010e420 msgr2=0x7f688019eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:25.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:25.261+0000 7f6885400640 1 --2- 172.21.15.77:0/3885178568 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f688010e420 0x7f688019eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:25.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:25.261+0000 7f6885400640 1 -- 172.21.15.77:0/3885178568 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f68800745c0 msgr2=0x7f688019a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:25.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:25.261+0000 7f6885400640 1 --2- 172.21.15.77:0/3885178568 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f68800745c0 0x7f688019a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:25.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:25.261+0000 7f6885400640 1 -- 172.21.15.77:0/3885178568 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f688019f6d0 con 0x7f6880074ee0 2024-09-10T09:53:25.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:25.261+0000 7f6886800640 1 --2- 172.21.15.77:0/3885178568 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f688010e420 0x7f688019eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:53:25.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:25.262+0000 7f6885400640 1 --2- 172.21.15.77:0/3885178568 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6880074ee0 0x7f688019ac60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f687000e9f0 tx=0x7f687000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:25.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:25.262+0000 7f687aa00640 1 -- 172.21.15.77:0/3885178568 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f687000ccc0 con 0x7f6880074ee0 2024-09-10T09:53:25.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:25.262+0000 7f6888800640 1 -- 172.21.15.77:0/3885178568 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f688019f9c0 con 0x7f6880074ee0 2024-09-10T09:53:25.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:25.262+0000 7f687aa00640 1 -- 172.21.15.77:0/3885178568 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6870005020 con 0x7f6880074ee0 2024-09-10T09:53:25.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:25.262+0000 7f687aa00640 1 -- 172.21.15.77:0/3885178568 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68700049e0 con 0x7f6880074ee0 2024-09-10T09:53:25.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:25.262+0000 7f6888800640 1 -- 172.21.15.77:0/3885178568 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68801a72a0 con 0x7f6880074ee0 2024-09-10T09:53:25.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:25.263+0000 7f6888800640 1 -- 172.21.15.77:0/3885178568 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68800749a0 con 0x7f6880074ee0 2024-09-10T09:53:25.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:25.266+0000 7f687aa00640 1 -- 172.21.15.77:0/3885178568 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f6870005560 con 0x7f6880074ee0 2024-09-10T09:53:25.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:25.268+0000 7f687aa00640 1 --2- 172.21.15.77:0/3885178568 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f68680a2c10 0x7f68680a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:25.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:25.268+0000 7f687aa00640 1 -- 172.21.15.77:0/3885178568 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f68700cf530 con 0x7f6880074ee0 2024-09-10T09:53:25.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:25.268+0000 7f6885e00640 1 --2- 172.21.15.77:0/3885178568 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f68680a2c10 0x7f68680a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:25.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:25.269+0000 7f6885e00640 1 --2- 172.21.15.77:0/3885178568 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f68680a2c10 0x7f68680a50d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6874009c90 tx=0x7f6874005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:25.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:25.272+0000 7f687aa00640 1 -- 172.21.15.77:0/3885178568 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f68700051d0 con 0x7f6880074ee0 2024-09-10T09:53:25.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:25.472+0000 7f6888800640 1 -- 172.21.15.77:0/3885178568 --> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f6880062630 con 0x7f68680a2c10 2024-09-10T09:53:25.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:25.500+0000 7f687aa00640 1 -- 172.21.15.77:0/3885178568 <== mgr.14150 v2:172.21.15.77:6800/4261324650 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6880062630 con 0x7f68680a2c10 2024-09-10T09:53:25.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:25.503+0000 7f6888800640 1 -- 172.21.15.77:0/3885178568 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f68680a2c10 msgr2=0x7f68680a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:25.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:25.503+0000 7f6888800640 1 --2- 172.21.15.77:0/3885178568 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f68680a2c10 0x7f68680a50d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6874009c90 tx=0x7f6874005f90 comp rx=0 tx=0).stop 2024-09-10T09:53:25.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:25.503+0000 7f6888800640 1 -- 172.21.15.77:0/3885178568 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6880074ee0 msgr2=0x7f688019ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:25.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:25.503+0000 7f6888800640 1 --2- 172.21.15.77:0/3885178568 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6880074ee0 0x7f688019ac60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f687000e9f0 tx=0x7f687000eeb0 comp rx=0 tx=0).stop 2024-09-10T09:53:25.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:25.503+0000 7f6888800640 1 -- 172.21.15.77:0/3885178568 shutdown_connections 2024-09-10T09:53:25.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:25.503+0000 7f6888800640 1 --2- 172.21.15.77:0/3885178568 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f68680a2c10 0x7f68680a50d0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:25.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:25.503+0000 7f6888800640 1 --2- 172.21.15.77:0/3885178568 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f688010e420 0x7f688019eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:25.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:25.503+0000 7f6888800640 1 --2- 172.21.15.77:0/3885178568 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6880074ee0 0x7f688019ac60 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:25.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:25.503+0000 7f6888800640 1 --2- 172.21.15.77:0/3885178568 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f68800745c0 0x7f688019a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:25.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:25.503+0000 7f6888800640 1 -- 172.21.15.77:0/3885178568 >> 172.21.15.77:0/3885178568 conn(0x7f68800fd8c0 msgr2=0x7f68800ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:25.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:25.503+0000 7f6888800640 1 -- 172.21.15.77:0/3885178568 shutdown_connections 2024-09-10T09:53:25.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:25.503+0000 7f6888800640 1 -- 172.21.15.77:0/3885178568 wait complete. 2024-09-10T09:53:26.273 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-09-10T09:53:26.643 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:53:27.086 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:26 smithi077 ceph-mon[30319]: from='client.24752 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:53:27.087 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:26 smithi077 ceph-mon[30319]: pgmap v337: 129 pgs: 129 active+clean; 608 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 6.0 KiB/s wr, 22 op/s 2024-09-10T09:53:27.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:26 smithi134 ceph-mon[35674]: from='client.24752 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:53:27.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:26 smithi134 ceph-mon[35674]: pgmap v337: 129 pgs: 129 active+clean; 608 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 6.0 KiB/s wr, 22 op/s 2024-09-10T09:53:27.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:26 smithi120 ceph-mon[34770]: from='client.24752 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:53:27.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:26 smithi120 ceph-mon[34770]: pgmap v337: 129 pgs: 129 active+clean; 608 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 6.0 KiB/s wr, 22 op/s 2024-09-10T09:53:27.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:27.199+0000 7f8b71400640 1 -- 172.21.15.77:0/3444189909 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8b6c103ff0 msgr2=0x7f8b6c1043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:27.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:27.199+0000 7f8b71400640 1 --2- 172.21.15.77:0/3444189909 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8b6c103ff0 0x7f8b6c1043d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c009900 tx=0x7f8b5c02a3f0 comp rx=0 tx=0).stop 2024-09-10T09:53:27.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:27.199+0000 7f8b71400640 1 -- 172.21.15.77:0/3444189909 shutdown_connections 2024-09-10T09:53:27.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:27.199+0000 7f8b71400640 1 --2- 172.21.15.77:0/3444189909 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8b6c109120 0x7f8b6c110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:27.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:27.199+0000 7f8b71400640 1 --2- 172.21.15.77:0/3444189909 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b6c1049a0 0x7f8b6c1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:27.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:27.199+0000 7f8b71400640 1 --2- 172.21.15.77:0/3444189909 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8b6c103ff0 0x7f8b6c1043d0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:27.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:27.199+0000 7f8b71400640 1 -- 172.21.15.77:0/3444189909 >> 172.21.15.77:0/3444189909 conn(0x7f8b6c0ffa90 msgr2=0x7f8b6c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:27.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:27.199+0000 7f8b71400640 1 -- 172.21.15.77:0/3444189909 shutdown_connections 2024-09-10T09:53:27.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:27.199+0000 7f8b71400640 1 -- 172.21.15.77:0/3444189909 wait complete. 2024-09-10T09:53:27.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:27.201+0000 7f8b71400640 1 Processor -- start 2024-09-10T09:53:27.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:27.201+0000 7f8b71400640 1 -- start start 2024-09-10T09:53:27.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:27.201+0000 7f8b71400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8b6c103ff0 0x7f8b6c19a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:27.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:27.201+0000 7f8b71400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b6c1049a0 0x7f8b6c19ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:27.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:27.201+0000 7f8b71400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8b6c109120 0x7f8b6c19eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:27.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:27.201+0000 7f8b71400640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8b6c114140 con 0x7f8b6c109120 2024-09-10T09:53:27.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:27.201+0000 7f8b71400640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8b6c113fc0 con 0x7f8b6c103ff0 2024-09-10T09:53:27.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:27.201+0000 7f8b71400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8b6c1142c0 con 0x7f8b6c1049a0 2024-09-10T09:53:27.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:27.201+0000 7f8b6b200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8b6c109120 0x7f8b6c19eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:27.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:27.202+0000 7f8b6b200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8b6c109120 0x7f8b6c19eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:51158/0 (socket says 172.21.15.77:51158) 2024-09-10T09:53:27.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:27.202+0000 7f8b6b200640 1 -- 172.21.15.77:0/670772236 learned_addr learned my addr 172.21.15.77:0/670772236 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:53:27.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:27.202+0000 7f8b63e00640 1 --2- 172.21.15.77:0/670772236 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b6c1049a0 0x7f8b6c19ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:27.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:27.202+0000 7f8b6a800640 1 --2- 172.21.15.77:0/670772236 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8b6c103ff0 0x7f8b6c19a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:27.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:27.202+0000 7f8b6b200640 1 -- 172.21.15.77:0/670772236 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b6c1049a0 msgr2=0x7f8b6c19ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:27.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:27.202+0000 7f8b6b200640 1 --2- 172.21.15.77:0/670772236 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b6c1049a0 0x7f8b6c19ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:27.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:27.202+0000 7f8b6b200640 1 -- 172.21.15.77:0/670772236 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8b6c103ff0 msgr2=0x7f8b6c19a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:27.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:27.202+0000 7f8b6b200640 1 --2- 172.21.15.77:0/670772236 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8b6c103ff0 0x7f8b6c19a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:27.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:27.202+0000 7f8b6b200640 1 -- 172.21.15.77:0/670772236 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b6c19f5b0 con 0x7f8b6c109120 2024-09-10T09:53:27.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:27.202+0000 7f8b6b200640 1 --2- 172.21.15.77:0/670772236 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8b6c109120 0x7f8b6c19eed0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8b6400eaa0 tx=0x7f8b6400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:27.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:27.203+0000 7f8b63400640 1 -- 172.21.15.77:0/670772236 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b6400cdb0 con 0x7f8b6c109120 2024-09-10T09:53:27.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:27.203+0000 7f8b63400640 1 -- 172.21.15.77:0/670772236 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b64005020 con 0x7f8b6c109120 2024-09-10T09:53:27.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:27.203+0000 7f8b71400640 1 -- 172.21.15.77:0/670772236 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b6c19f8a0 con 0x7f8b6c109120 2024-09-10T09:53:27.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:27.203+0000 7f8b63400640 1 -- 172.21.15.77:0/670772236 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b640116b0 con 0x7f8b6c109120 2024-09-10T09:53:27.206 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:27.203+0000 7f8b71400640 1 -- 172.21.15.77:0/670772236 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b6c1a7180 con 0x7f8b6c109120 2024-09-10T09:53:27.206 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:27.204+0000 7f8b71400640 1 -- 172.21.15.77:0/670772236 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b6c1043d0 con 0x7f8b6c109120 2024-09-10T09:53:27.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:27.206+0000 7f8b63400640 1 -- 172.21.15.77:0/670772236 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f8b64005560 con 0x7f8b6c109120 2024-09-10T09:53:27.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:27.207+0000 7f8b63400640 1 --2- 172.21.15.77:0/670772236 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f8b500a2c90 0x7f8b500a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:27.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:27.207+0000 7f8b63400640 1 -- 172.21.15.77:0/670772236 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f8b640cf660 con 0x7f8b6c109120 2024-09-10T09:53:27.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:27.207+0000 7f8b6a800640 1 --2- 172.21.15.77:0/670772236 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f8b500a2c90 0x7f8b500a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:27.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:27.208+0000 7f8b6a800640 1 --2- 172.21.15.77:0/670772236 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f8b500a2c90 0x7f8b500a5150 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c009230 tx=0x7f8b5c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:27.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:27.211+0000 7f8b63400640 1 -- 172.21.15.77:0/670772236 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8b64016030 con 0x7f8b6c109120 2024-09-10T09:53:27.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:27.415+0000 7f8b71400640 1 -- 172.21.15.77:0/670772236 --> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f8b6c062630 con 0x7f8b500a2c90 2024-09-10T09:53:27.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:27.421+0000 7f8b63400640 1 -- 172.21.15.77:0/670772236 <== mgr.14150 v2:172.21.15.77:6800/4261324650 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f8b6c062630 con 0x7f8b500a2c90 2024-09-10T09:53:27.423 INFO:teuthology.orchestra.run.smithi077.stdout:/volumes/qa/sv_1/10f21c5c-5a81-465f-abea-54bb63745c38 2024-09-10T09:53:27.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:27.423+0000 7f8b71400640 1 -- 172.21.15.77:0/670772236 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f8b500a2c90 msgr2=0x7f8b500a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:27.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:27.423+0000 7f8b71400640 1 --2- 172.21.15.77:0/670772236 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f8b500a2c90 0x7f8b500a5150 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c009230 tx=0x7f8b5c0058d0 comp rx=0 tx=0).stop 2024-09-10T09:53:27.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:27.423+0000 7f8b71400640 1 -- 172.21.15.77:0/670772236 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8b6c109120 msgr2=0x7f8b6c19eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:27.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:27.423+0000 7f8b71400640 1 --2- 172.21.15.77:0/670772236 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8b6c109120 0x7f8b6c19eed0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8b6400eaa0 tx=0x7f8b6400ef60 comp rx=0 tx=0).stop 2024-09-10T09:53:27.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:27.423+0000 7f8b71400640 1 -- 172.21.15.77:0/670772236 shutdown_connections 2024-09-10T09:53:27.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:27.423+0000 7f8b71400640 1 --2- 172.21.15.77:0/670772236 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f8b500a2c90 0x7f8b500a5150 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:27.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:27.423+0000 7f8b71400640 1 --2- 172.21.15.77:0/670772236 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8b6c109120 0x7f8b6c19eed0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:27.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:27.423+0000 7f8b71400640 1 --2- 172.21.15.77:0/670772236 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b6c1049a0 0x7f8b6c19ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:27.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:27.423+0000 7f8b71400640 1 --2- 172.21.15.77:0/670772236 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8b6c103ff0 0x7f8b6c19a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:27.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:27.423+0000 7f8b71400640 1 -- 172.21.15.77:0/670772236 >> 172.21.15.77:0/670772236 conn(0x7f8b6c0ffa90 msgr2=0x7f8b6c100db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:27.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:27.423+0000 7f8b71400640 1 -- 172.21.15.77:0/670772236 shutdown_connections 2024-09-10T09:53:27.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:27.423+0000 7f8b71400640 1 -- 172.21.15.77:0/670772236 wait complete. 2024-09-10T09:53:28.171 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:53:28.538 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:53:28.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:28 smithi077 ceph-mon[30319]: from='client.14703 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:53:28.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:28 smithi077 ceph-mon[30319]: pgmap v338: 129 pgs: 129 active+clean; 608 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.5 KiB/s wr, 21 op/s 2024-09-10T09:53:29.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:29.107+0000 7f99b3200640 1 -- 172.21.15.77:0/977843421 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f99b4076720 msgr2=0x7f99b4076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:29.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:29.107+0000 7f99b3200640 1 --2- 172.21.15.77:0/977843421 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f99b4076720 0x7f99b4076b00 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f99a4009900 tx=0x7f99a402a3f0 comp rx=0 tx=0).stop 2024-09-10T09:53:29.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:29.107+0000 7f99b3200640 1 -- 172.21.15.77:0/977843421 shutdown_connections 2024-09-10T09:53:29.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:29.107+0000 7f99b3200640 1 --2- 172.21.15.77:0/977843421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99b4112870 0x7f99b4114c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:29.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:29.107+0000 7f99b3200640 1 --2- 172.21.15.77:0/977843421 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f99b4077040 0x7f99b4112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:29.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:29.107+0000 7f99b3200640 1 --2- 172.21.15.77:0/977843421 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f99b4076720 0x7f99b4076b00 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:29.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:29.107+0000 7f99b3200640 1 -- 172.21.15.77:0/977843421 >> 172.21.15.77:0/977843421 conn(0x7f99b40ff910 msgr2=0x7f99b4101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:29.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:29.107+0000 7f99b3200640 1 -- 172.21.15.77:0/977843421 shutdown_connections 2024-09-10T09:53:29.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:29.107+0000 7f99b3200640 1 -- 172.21.15.77:0/977843421 wait complete. 2024-09-10T09:53:29.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:29.108+0000 7f99b3200640 1 Processor -- start 2024-09-10T09:53:29.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:29.108+0000 7f99b3200640 1 -- start start 2024-09-10T09:53:29.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:29.109+0000 7f99b3200640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f99b4076720 0x7f99b41a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:29.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:29.109+0000 7f99b3200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99b4077040 0x7f99b41a4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:29.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:29.109+0000 7f99b3200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f99b4112870 0x7f99b419e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:29.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:29.109+0000 7f99b3200640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f99b4118450 con 0x7f99b4112870 2024-09-10T09:53:29.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:29.109+0000 7f99b3200640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f99b41182d0 con 0x7f99b4076720 2024-09-10T09:53:29.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:29.109+0000 7f99b3200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f99b41185d0 con 0x7f99b4077040 2024-09-10T09:53:29.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:29.109+0000 7f99b2800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f99b4112870 0x7f99b419e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:29.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:29.109+0000 7f99b1400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99b4077040 0x7f99b41a4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:29.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:29.109+0000 7f99b2800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f99b4112870 0x7f99b419e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:51162/0 (socket says 172.21.15.77:51162) 2024-09-10T09:53:29.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:29.109+0000 7f99b2800640 1 -- 172.21.15.77:0/3585842160 learned_addr learned my addr 172.21.15.77:0/3585842160 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:53:29.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:29.109+0000 7f99b1e00640 1 --2- 172.21.15.77:0/3585842160 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f99b4076720 0x7f99b41a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:29.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:29.110+0000 7f99b2800640 1 -- 172.21.15.77:0/3585842160 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99b4077040 msgr2=0x7f99b41a4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:29.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:29.110+0000 7f99b2800640 1 --2- 172.21.15.77:0/3585842160 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99b4077040 0x7f99b41a4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:29.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:29.110+0000 7f99b2800640 1 -- 172.21.15.77:0/3585842160 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f99b4076720 msgr2=0x7f99b41a46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:29.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:29.110+0000 7f99b2800640 1 --2- 172.21.15.77:0/3585842160 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f99b4076720 0x7f99b41a46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:29.113 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:29.110+0000 7f99b2800640 1 -- 172.21.15.77:0/3585842160 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99b419efe0 con 0x7f99b4112870 2024-09-10T09:53:29.113 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:29.110+0000 7f99b2800640 1 --2- 172.21.15.77:0/3585842160 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f99b4112870 0x7f99b419e770 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f99ac00eb10 tx=0x7f99ac00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:29.113 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:29.110+0000 7f99aaa00640 1 -- 172.21.15.77:0/3585842160 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99ac004b20 con 0x7f99b4112870 2024-09-10T09:53:29.113 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:29.110+0000 7f99aaa00640 1 -- 172.21.15.77:0/3585842160 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99ac004cc0 con 0x7f99b4112870 2024-09-10T09:53:29.113 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:29.110+0000 7f99aaa00640 1 -- 172.21.15.77:0/3585842160 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99ac010790 con 0x7f99b4112870 2024-09-10T09:53:29.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:29.110+0000 7f99b3200640 1 -- 172.21.15.77:0/3585842160 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99b419f2d0 con 0x7f99b4112870 2024-09-10T09:53:29.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:29.110+0000 7f99b3200640 1 -- 172.21.15.77:0/3585842160 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99b41ab450 con 0x7f99b4112870 2024-09-10T09:53:29.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:29.111+0000 7f99b3200640 1 -- 172.21.15.77:0/3585842160 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99b4077e60 con 0x7f99b4112870 2024-09-10T09:53:29.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:29.113+0000 7f99aaa00640 1 -- 172.21.15.77:0/3585842160 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f99ac005020 con 0x7f99b4112870 2024-09-10T09:53:29.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:29.114+0000 7f99aaa00640 1 --2- 172.21.15.77:0/3585842160 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f99980a2c90 0x7f99980a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:29.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:29.115+0000 7f99aaa00640 1 -- 172.21.15.77:0/3585842160 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f99ac0cfb80 con 0x7f99b4112870 2024-09-10T09:53:29.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:29.115+0000 7f99b1e00640 1 --2- 172.21.15.77:0/3585842160 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f99980a2c90 0x7f99980a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:29.117 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:29.115+0000 7f99b1e00640 1 --2- 172.21.15.77:0/3585842160 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f99980a2c90 0x7f99980a5150 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f99a4000c00 tx=0x7f99a4005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:29.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:29.119+0000 7f99aaa00640 1 -- 172.21.15.77:0/3585842160 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f99ac09c620 con 0x7f99b4112870 2024-09-10T09:53:29.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:28 smithi134 ceph-mon[35674]: from='client.14703 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:53:29.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:28 smithi134 ceph-mon[35674]: pgmap v338: 129 pgs: 129 active+clean; 608 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.5 KiB/s wr, 21 op/s 2024-09-10T09:53:29.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:28 smithi120 ceph-mon[34770]: from='client.14703 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:53:29.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:28 smithi120 ceph-mon[34770]: pgmap v338: 129 pgs: 129 active+clean; 608 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.5 KiB/s wr, 21 op/s 2024-09-10T09:53:29.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:29.328+0000 7f99b3200640 1 -- 172.21.15.77:0/3585842160 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f99b410d470 con 0x7f99b4112870 2024-09-10T09:53:29.331 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:29.329+0000 7f99aaa00640 1 -- 172.21.15.77:0/3585842160 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11989 (secure 0 0 0) 0x7f99ac09bf20 con 0x7f99b4112870 2024-09-10T09:53:29.331 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:53:29.331 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":28,"btime":"2024-09-10T09:53:16:873716+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3159289675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3159289675},{"type":"v1","addr":"172.21.15.77:6835","nonce":3159289675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1951962341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1951962341},{"type":"v1","addr":"172.21.15.77:6837","nonce":1951962341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:52:23.368252+0000","modified":"2024-09-10T09:53:16.873698+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24563,"mds_1":24524,"mds_2":24520,"mds_3":24502,"mds_4":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14487":{"gid":14487,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/4291121888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":4291121888},{"type":"v1","addr":"172.21.15.77:6839","nonce":4291121888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14499":{"gid":14499,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6841/445284194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":445284194},{"type":"v1","addr":"172.21.15.77:6841","nonce":445284194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1005054423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1005054423},{"type":"v1","addr":"172.21.15.134:6833","nonce":1005054423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6833/3716307744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":3716307744},{"type":"v1","addr":"172.21.15.120:6833","nonce":3716307744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/1319892544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1319892544},{"type":"v1","addr":"172.21.15.120:6835","nonce":1319892544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.134:6835/3293933699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3293933699},{"type":"v1","addr":"172.21.15.134:6835","nonce":3293933699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.134:6837/2461497091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2461497091},{"type":"v1","addr":"172.21.15.134:6837","nonce":2461497091}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":20,"addr":"172.21.15.134:6839/1965117680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1965117680},{"type":"v1","addr":"172.21.15.134:6839","nonce":1965117680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":20,"addr":"172.21.15.120:6837/3033280709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3033280709},{"type":"v1","addr":"172.21.15.120:6837","nonce":3033280709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.120:6839/1777710157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1777710157},{"type":"v1","addr":"172.21.15.120:6839","nonce":1777710157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24563,"qdb_cluster":[24563,24524,24520,24502,24496]},"id":1}]} 2024-09-10T09:53:29.332 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 28 2024-09-10T09:53:29.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:29.332+0000 7f99b3200640 1 -- 172.21.15.77:0/3585842160 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f99980a2c90 msgr2=0x7f99980a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:29.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:29.332+0000 7f99b3200640 1 --2- 172.21.15.77:0/3585842160 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f99980a2c90 0x7f99980a5150 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f99a4000c00 tx=0x7f99a4005b50 comp rx=0 tx=0).stop 2024-09-10T09:53:29.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:29.332+0000 7f99b3200640 1 -- 172.21.15.77:0/3585842160 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f99b4112870 msgr2=0x7f99b419e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:29.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:29.332+0000 7f99b3200640 1 --2- 172.21.15.77:0/3585842160 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f99b4112870 0x7f99b419e770 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f99ac00eb10 tx=0x7f99ac00c2d0 comp rx=0 tx=0).stop 2024-09-10T09:53:29.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:29.332+0000 7f99b3200640 1 -- 172.21.15.77:0/3585842160 shutdown_connections 2024-09-10T09:53:29.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:29.332+0000 7f99b3200640 1 --2- 172.21.15.77:0/3585842160 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f99980a2c90 0x7f99980a5150 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:29.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:29.332+0000 7f99b3200640 1 --2- 172.21.15.77:0/3585842160 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f99b4112870 0x7f99b419e770 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:29.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:29.332+0000 7f99b3200640 1 --2- 172.21.15.77:0/3585842160 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99b4077040 0x7f99b41a4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:29.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:29.332+0000 7f99b3200640 1 --2- 172.21.15.77:0/3585842160 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f99b4076720 0x7f99b41a46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:29.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:29.332+0000 7f99b3200640 1 -- 172.21.15.77:0/3585842160 >> 172.21.15.77:0/3585842160 conn(0x7f99b40ff910 msgr2=0x7f99b4101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:29.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:29.332+0000 7f99b3200640 1 -- 172.21.15.77:0/3585842160 shutdown_connections 2024-09-10T09:53:29.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:29.332+0000 7f99b3200640 1 -- 172.21.15.77:0/3585842160 wait complete. 2024-09-10T09:53:29.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:29 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/3585842160' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:53:30.093 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T09:53:30.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:29 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/3585842160' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:53:30.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:29 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/3585842160' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:53:30.475 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:53:30.877 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:30 smithi077 ceph-mon[30319]: pgmap v339: 129 pgs: 129 active+clean; 632 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.7 KiB/s wr, 20 op/s 2024-09-10T09:53:31.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:30 smithi134 ceph-mon[35674]: pgmap v339: 129 pgs: 129 active+clean; 632 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.7 KiB/s wr, 20 op/s 2024-09-10T09:53:31.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:30 smithi120 ceph-mon[34770]: pgmap v339: 129 pgs: 129 active+clean; 632 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.7 KiB/s wr, 20 op/s 2024-09-10T09:53:31.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:31.222+0000 7f6d7dc00640 1 -- 172.21.15.77:0/2581179389 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6d781079c0 msgr2=0x7f6d78107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:31.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:31.222+0000 7f6d7dc00640 1 --2- 172.21.15.77:0/2581179389 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6d781079c0 0x7f6d78107da0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6d68009930 tx=0x7f6d6802a420 comp rx=0 tx=0).stop 2024-09-10T09:53:31.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:31.222+0000 7f6d7dc00640 1 -- 172.21.15.77:0/2581179389 shutdown_connections 2024-09-10T09:53:31.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:31.222+0000 7f6d7dc00640 1 --2- 172.21.15.77:0/2581179389 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d78102360 0x7f6d7810e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:31.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:31.222+0000 7f6d7dc00640 1 --2- 172.21.15.77:0/2581179389 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6d781019c0 0x7f6d78101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:31.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:31.222+0000 7f6d7dc00640 1 --2- 172.21.15.77:0/2581179389 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6d781079c0 0x7f6d78107da0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:31.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:31.222+0000 7f6d7dc00640 1 -- 172.21.15.77:0/2581179389 >> 172.21.15.77:0/2581179389 conn(0x7f6d780fd740 msgr2=0x7f6d780ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:31.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:31.222+0000 7f6d7dc00640 1 -- 172.21.15.77:0/2581179389 shutdown_connections 2024-09-10T09:53:31.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:31.222+0000 7f6d7dc00640 1 -- 172.21.15.77:0/2581179389 wait complete. 2024-09-10T09:53:31.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:31.223+0000 7f6d7dc00640 1 Processor -- start 2024-09-10T09:53:31.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:31.223+0000 7f6d7dc00640 1 -- start start 2024-09-10T09:53:31.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:31.224+0000 7f6d7dc00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6d781019c0 0x7f6d7819a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:31.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:31.224+0000 7f6d7dc00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6d78102360 0x7f6d7819a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:31.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:31.224+0000 7f6d7dc00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d781079c0 0x7f6d7819ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:31.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:31.224+0000 7f6d7dc00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6d78111e10 con 0x7f6d781019c0 2024-09-10T09:53:31.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:31.224+0000 7f6d7dc00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6d78111c90 con 0x7f6d78102360 2024-09-10T09:53:31.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:31.224+0000 7f6d7dc00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6d78111f90 con 0x7f6d781079c0 2024-09-10T09:53:31.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:31.224+0000 7f6d77e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6d781019c0 0x7f6d7819a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:31.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:31.224+0000 7f6d77e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6d781019c0 0x7f6d7819a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:51180/0 (socket says 172.21.15.77:51180) 2024-09-10T09:53:31.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:31.224+0000 7f6d77e00640 1 -- 172.21.15.77:0/2818509545 learned_addr learned my addr 172.21.15.77:0/2818509545 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:53:31.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:31.224+0000 7f6d7d200640 1 --2- 172.21.15.77:0/2818509545 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d781079c0 0x7f6d7819ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:31.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:31.224+0000 7f6d77400640 1 --2- 172.21.15.77:0/2818509545 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6d78102360 0x7f6d7819a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:31.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:31.225+0000 7f6d77e00640 1 -- 172.21.15.77:0/2818509545 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d781079c0 msgr2=0x7f6d7819ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:31.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:31.225+0000 7f6d77e00640 1 --2- 172.21.15.77:0/2818509545 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d781079c0 0x7f6d7819ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:31.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:31.225+0000 7f6d77e00640 1 -- 172.21.15.77:0/2818509545 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6d78102360 msgr2=0x7f6d7819a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:31.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:31.225+0000 7f6d77e00640 1 --2- 172.21.15.77:0/2818509545 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6d78102360 0x7f6d7819a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:31.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:31.225+0000 7f6d77e00640 1 -- 172.21.15.77:0/2818509545 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d7819f430 con 0x7f6d781019c0 2024-09-10T09:53:31.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:31.225+0000 7f6d77e00640 1 --2- 172.21.15.77:0/2818509545 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6d781019c0 0x7f6d7819a480 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6d6802a940 tx=0x7f6d68003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:31.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:31.225+0000 7f6d74c00640 1 -- 172.21.15.77:0/2818509545 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d6803f070 con 0x7f6d781019c0 2024-09-10T09:53:31.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:31.225+0000 7f6d74c00640 1 -- 172.21.15.77:0/2818509545 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d68031c90 con 0x7f6d781019c0 2024-09-10T09:53:31.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:31.225+0000 7f6d7dc00640 1 -- 172.21.15.77:0/2818509545 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d7819f6c0 con 0x7f6d781019c0 2024-09-10T09:53:31.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:31.225+0000 7f6d74c00640 1 -- 172.21.15.77:0/2818509545 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d68035030 con 0x7f6d781019c0 2024-09-10T09:53:31.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:31.225+0000 7f6d7dc00640 1 -- 172.21.15.77:0/2818509545 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d781a7000 con 0x7f6d781019c0 2024-09-10T09:53:31.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:31.226+0000 7f6d7dc00640 1 -- 172.21.15.77:0/2818509545 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d78101e20 con 0x7f6d781019c0 2024-09-10T09:53:31.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:31.228+0000 7f6d74c00640 1 -- 172.21.15.77:0/2818509545 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f6d68042070 con 0x7f6d781019c0 2024-09-10T09:53:31.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:31.229+0000 7f6d74c00640 1 --2- 172.21.15.77:0/2818509545 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f6d5c0a2c90 0x7f6d5c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:31.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:31.230+0000 7f6d74c00640 1 -- 172.21.15.77:0/2818509545 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f6d680efbf0 con 0x7f6d781019c0 2024-09-10T09:53:31.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:31.230+0000 7f6d77400640 1 --2- 172.21.15.77:0/2818509545 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f6d5c0a2c90 0x7f6d5c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:31.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:31.230+0000 7f6d77400640 1 --2- 172.21.15.77:0/2818509545 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f6d5c0a2c90 0x7f6d5c0a5150 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6d7810ea70 tx=0x7f6d6000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:31.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:31.233+0000 7f6d74c00640 1 -- 172.21.15.77:0/2818509545 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6d680bbfe0 con 0x7f6d781019c0 2024-09-10T09:53:31.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:31.411+0000 7f6d7dc00640 1 -- 172.21.15.77:0/2818509545 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6d781a71d0 con 0x7f6d781019c0 2024-09-10T09:53:31.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:31.413+0000 7f6d74c00640 1 -- 172.21.15.77:0/2818509545 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24707 (secure 0 0 0) 0x7f6d680c02e0 con 0x7f6d781019c0 2024-09-10T09:53:31.415 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:53:31.416 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":94,"fsid":"14a55226-6f59-11ef-bcea-c7b262605968","created":"2024-09-10T09:44:07.359542+0000","modified":"2024-09-10T09:53:00.787197+0000","last_up_change":"2024-09-10T09:51:26.731971+0000","last_in_change":"2024-09-10T09:51:03.857873+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:47:24.174660+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T09:52:18.766738+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T09:52:21.284745+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T09:52:29.846594+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8d46f634-114c-4926-afdc-f8fa37e40336","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":1509649079},{"type":"v1","addr":"172.21.15.77:6803","nonce":1509649079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":1509649079},{"type":"v1","addr":"172.21.15.77:6805","nonce":1509649079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":1509649079},{"type":"v1","addr":"172.21.15.77:6809","nonce":1509649079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":1509649079},{"type":"v1","addr":"172.21.15.77:6807","nonce":1509649079}]},"public_addr":"172.21.15.77:6803/1509649079","cluster_addr":"172.21.15.77:6805/1509649079","heartbeat_back_addr":"172.21.15.77:6809/1509649079","heartbeat_front_addr":"172.21.15.77:6807/1509649079","state":["exists","up"]},{"osd":1,"uuid":"b6570764-736a-49e4-877a-1965e5812d1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":4243920698},{"type":"v1","addr":"172.21.15.120:6801","nonce":4243920698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":4243920698},{"type":"v1","addr":"172.21.15.120:6803","nonce":4243920698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":4243920698},{"type":"v1","addr":"172.21.15.120:6807","nonce":4243920698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":4243920698},{"type":"v1","addr":"172.21.15.120:6805","nonce":4243920698}]},"public_addr":"172.21.15.120:6801/4243920698","cluster_addr":"172.21.15.120:6803/4243920698","heartbeat_back_addr":"172.21.15.120:6807/4243920698","heartbeat_front_addr":"172.21.15.120:6805/4243920698","state":["exists","up"]},{"osd":2,"uuid":"5e83fe13-d1ee-483e-a575-e8041ec836f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":3280466823},{"type":"v1","addr":"172.21.15.134:6801","nonce":3280466823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":3280466823},{"type":"v1","addr":"172.21.15.134:6803","nonce":3280466823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":3280466823},{"type":"v1","addr":"172.21.15.134:6807","nonce":3280466823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":3280466823},{"type":"v1","addr":"172.21.15.134:6805","nonce":3280466823}]},"public_addr":"172.21.15.134:6801/3280466823","cluster_addr":"172.21.15.134:6803/3280466823","heartbeat_back_addr":"172.21.15.134:6807/3280466823","heartbeat_front_addr":"172.21.15.134:6805/3280466823","state":["exists","up"]},{"osd":3,"uuid":"5affc6a7-1cae-48fe-b7db-befc2fcafecb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":1629416786},{"type":"v1","addr":"172.21.15.77:6811","nonce":1629416786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":1629416786},{"type":"v1","addr":"172.21.15.77:6813","nonce":1629416786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":1629416786},{"type":"v1","addr":"172.21.15.77:6817","nonce":1629416786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":1629416786},{"type":"v1","addr":"172.21.15.77:6815","nonce":1629416786}]},"public_addr":"172.21.15.77:6811/1629416786","cluster_addr":"172.21.15.77:6813/1629416786","heartbeat_back_addr":"172.21.15.77:6817/1629416786","heartbeat_front_addr":"172.21.15.77:6815/1629416786","state":["exists","up"]},{"osd":4,"uuid":"c826978f-40b0-43ae-adbb-5c93b32a68ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":3162658038},{"type":"v1","addr":"172.21.15.120:6809","nonce":3162658038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":3162658038},{"type":"v1","addr":"172.21.15.120:6811","nonce":3162658038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":3162658038},{"type":"v1","addr":"172.21.15.120:6815","nonce":3162658038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":3162658038},{"type":"v1","addr":"172.21.15.120:6813","nonce":3162658038}]},"public_addr":"172.21.15.120:6809/3162658038","cluster_addr":"172.21.15.120:6811/3162658038","heartbeat_back_addr":"172.21.15.120:6815/3162658038","heartbeat_front_addr":"172.21.15.120:6813/3162658038","state":["exists","up"]},{"osd":5,"uuid":"be51c7fb-2765-46a3-9861-c950f63de148","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":4081444621},{"type":"v1","addr":"172.21.15.134:6809","nonce":4081444621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":4081444621},{"type":"v1","addr":"172.21.15.134:6811","nonce":4081444621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":4081444621},{"type":"v1","addr":"172.21.15.134:6815","nonce":4081444621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":4081444621},{"type":"v1","addr":"172.21.15.134:6813","nonce":4081444621}]},"public_addr":"172.21.15.134:6809/4081444621","cluster_addr":"172.21.15.134:6811/4081444621","heartbeat_back_addr":"172.21.15.134:6815/4081444621","heartbeat_front_addr":"172.21.15.134:6813/4081444621","state":["exists","up"]},{"osd":6,"uuid":"4eb403ab-473b-41cc-803d-18c908f768a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":879565639},{"type":"v1","addr":"172.21.15.77:6819","nonce":879565639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":879565639},{"type":"v1","addr":"172.21.15.77:6821","nonce":879565639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":879565639},{"type":"v1","addr":"172.21.15.77:6825","nonce":879565639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":879565639},{"type":"v1","addr":"172.21.15.77:6823","nonce":879565639}]},"public_addr":"172.21.15.77:6819/879565639","cluster_addr":"172.21.15.77:6821/879565639","heartbeat_back_addr":"172.21.15.77:6825/879565639","heartbeat_front_addr":"172.21.15.77:6823/879565639","state":["exists","up"]},{"osd":7,"uuid":"ba982bd0-c236-456c-9889-61087d79c1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":2688667400},{"type":"v1","addr":"172.21.15.120:6817","nonce":2688667400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":2688667400},{"type":"v1","addr":"172.21.15.120:6819","nonce":2688667400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":2688667400},{"type":"v1","addr":"172.21.15.120:6823","nonce":2688667400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2688667400},{"type":"v1","addr":"172.21.15.120:6821","nonce":2688667400}]},"public_addr":"172.21.15.120:6817/2688667400","cluster_addr":"172.21.15.120:6819/2688667400","heartbeat_back_addr":"172.21.15.120:6823/2688667400","heartbeat_front_addr":"172.21.15.120:6821/2688667400","state":["exists","up"]},{"osd":8,"uuid":"f2e8ca43-57c9-4933-b5f7-73f4d1b3e08b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2081686544},{"type":"v1","addr":"172.21.15.134:6817","nonce":2081686544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2081686544},{"type":"v1","addr":"172.21.15.134:6819","nonce":2081686544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2081686544},{"type":"v1","addr":"172.21.15.134:6823","nonce":2081686544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2081686544},{"type":"v1","addr":"172.21.15.134:6821","nonce":2081686544}]},"public_addr":"172.21.15.134:6817/2081686544","cluster_addr":"172.21.15.134:6819/2081686544","heartbeat_back_addr":"172.21.15.134:6823/2081686544","heartbeat_front_addr":"172.21.15.134:6821/2081686544","state":["exists","up"]},{"osd":9,"uuid":"9486f8b5-0614-4208-87ec-117741f6b622","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":1210619387},{"type":"v1","addr":"172.21.15.77:6827","nonce":1210619387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":1210619387},{"type":"v1","addr":"172.21.15.77:6829","nonce":1210619387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":1210619387},{"type":"v1","addr":"172.21.15.77:6833","nonce":1210619387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":1210619387},{"type":"v1","addr":"172.21.15.77:6831","nonce":1210619387}]},"public_addr":"172.21.15.77:6827/1210619387","cluster_addr":"172.21.15.77:6829/1210619387","heartbeat_back_addr":"172.21.15.77:6833/1210619387","heartbeat_front_addr":"172.21.15.77:6831/1210619387","state":["exists","up"]},{"osd":10,"uuid":"8e8192b0-1398-4d4d-9756-762b2f874d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":754733885},{"type":"v1","addr":"172.21.15.120:6825","nonce":754733885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":754733885},{"type":"v1","addr":"172.21.15.120:6827","nonce":754733885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":754733885},{"type":"v1","addr":"172.21.15.120:6831","nonce":754733885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":754733885},{"type":"v1","addr":"172.21.15.120:6829","nonce":754733885}]},"public_addr":"172.21.15.120:6825/754733885","cluster_addr":"172.21.15.120:6827/754733885","heartbeat_back_addr":"172.21.15.120:6831/754733885","heartbeat_front_addr":"172.21.15.120:6829/754733885","state":["exists","up"]},{"osd":11,"uuid":"85c1ccd4-5147-4995-896e-fe977f73f749","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1602213143},{"type":"v1","addr":"172.21.15.134:6825","nonce":1602213143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1602213143},{"type":"v1","addr":"172.21.15.134:6827","nonce":1602213143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1602213143},{"type":"v1","addr":"172.21.15.134:6831","nonce":1602213143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1602213143},{"type":"v1","addr":"172.21.15.134:6829","nonce":1602213143}]},"public_addr":"172.21.15.134:6825/1602213143","cluster_addr":"172.21.15.134:6827/1602213143","heartbeat_back_addr":"172.21.15.134:6831/1602213143","heartbeat_front_addr":"172.21.15.134:6829/1602213143","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:46:30.293451+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:46:56.554801+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:47:21.476225+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:47:49.737332+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:48:16.092181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:48:41.240815+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:49:11.683950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:49:38.859255+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:50:04.105401+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:50:33.847810+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:50:59.901797+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:51:25.241507+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.2","mappings":[{"from":7,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":5,"to":8}]},{"pgid":"4.6","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.77:0/1697982655":"2024-09-11T09:44:59.575663+0000","172.21.15.77:6801/1527070442":"2024-09-11T09:44:59.575663+0000","172.21.15.77:0/3146688196":"2024-09-11T09:44:32.185342+0000","172.21.15.77:0/371773177":"2024-09-11T09:44:59.575663+0000","172.21.15.77:0/1925456910":"2024-09-11T09:44:59.575663+0000","172.21.15.77:0/138545832":"2024-09-11T09:44:32.185342+0000","172.21.15.77:6801/3235900503":"2024-09-11T09:44:32.185342+0000","172.21.15.77:6800/1527070442":"2024-09-11T09:44:59.575663+0000","172.21.15.77:6800/3235900503":"2024-09-11T09:44:32.185342+0000","172.21.15.77:0/1647092548":"2024-09-11T09:44:32.185342+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T09:53:31.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:31.415+0000 7f6d7dc00640 1 -- 172.21.15.77:0/2818509545 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f6d5c0a2c90 msgr2=0x7f6d5c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:31.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:31.415+0000 7f6d7dc00640 1 --2- 172.21.15.77:0/2818509545 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f6d5c0a2c90 0x7f6d5c0a5150 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6d7810ea70 tx=0x7f6d6000b410 comp rx=0 tx=0).stop 2024-09-10T09:53:31.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:31.415+0000 7f6d7dc00640 1 -- 172.21.15.77:0/2818509545 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6d781019c0 msgr2=0x7f6d7819a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:31.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:31.415+0000 7f6d7dc00640 1 --2- 172.21.15.77:0/2818509545 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6d781019c0 0x7f6d7819a480 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6d6802a940 tx=0x7f6d68003b90 comp rx=0 tx=0).stop 2024-09-10T09:53:31.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:31.415+0000 7f6d7dc00640 1 -- 172.21.15.77:0/2818509545 shutdown_connections 2024-09-10T09:53:31.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:31.415+0000 7f6d7dc00640 1 --2- 172.21.15.77:0/2818509545 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f6d5c0a2c90 0x7f6d5c0a5150 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:31.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:31.415+0000 7f6d7dc00640 1 --2- 172.21.15.77:0/2818509545 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d781079c0 0x7f6d7819ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:31.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:31.415+0000 7f6d7dc00640 1 --2- 172.21.15.77:0/2818509545 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6d78102360 0x7f6d7819a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:31.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:31.415+0000 7f6d7dc00640 1 --2- 172.21.15.77:0/2818509545 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6d781019c0 0x7f6d7819a480 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:31.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:31.415+0000 7f6d7dc00640 1 -- 172.21.15.77:0/2818509545 >> 172.21.15.77:0/2818509545 conn(0x7f6d780fd740 msgr2=0x7f6d7810cf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:31.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:31.416+0000 7f6d7dc00640 1 -- 172.21.15.77:0/2818509545 shutdown_connections 2024-09-10T09:53:31.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:31.416+0000 7f6d7dc00640 1 -- 172.21.15.77:0/2818509545 wait complete. 2024-09-10T09:53:32.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:31 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/2818509545' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T09:53:32.168 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:31 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/2818509545' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T09:53:32.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:31 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/2818509545' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T09:53:32.205 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-10T09:53:32.205 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T09:53:32.572 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:53:33.038 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:32 smithi077 ceph-mon[30319]: pgmap v340: 129 pgs: 129 active+clean; 632 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 4.8 KiB/s wr, 12 op/s 2024-09-10T09:53:33.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:32 smithi134 ceph-mon[35674]: pgmap v340: 129 pgs: 129 active+clean; 632 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 4.8 KiB/s wr, 12 op/s 2024-09-10T09:53:33.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:33.146+0000 7f7928a00640 1 -- 172.21.15.77:0/1632075120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f792410bfc0 msgr2=0x7f792410e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:33.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:33.146+0000 7f7928a00640 1 --2- 172.21.15.77:0/1632075120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f792410bfc0 0x7f792410e460 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f791c009ca0 tx=0x7f791c02a950 comp rx=0 tx=0).stop 2024-09-10T09:53:33.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:33.146+0000 7f7928a00640 1 -- 172.21.15.77:0/1632075120 shutdown_connections 2024-09-10T09:53:33.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:33.146+0000 7f7928a00640 1 --2- 172.21.15.77:0/1632075120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f792410bfc0 0x7f792410e460 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:33.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:33.146+0000 7f7928a00640 1 --2- 172.21.15.77:0/1632075120 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f79241034b0 0x7f792410b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:33.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:33.146+0000 7f7928a00640 1 --2- 172.21.15.77:0/1632075120 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7924102b90 0x7f7924102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:33.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:33.146+0000 7f7928a00640 1 -- 172.21.15.77:0/1632075120 >> 172.21.15.77:0/1632075120 conn(0x7f79240fc610 msgr2=0x7f79240fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:33.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:33.146+0000 7f7928a00640 1 -- 172.21.15.77:0/1632075120 shutdown_connections 2024-09-10T09:53:33.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:33.146+0000 7f7928a00640 1 -- 172.21.15.77:0/1632075120 wait complete. 2024-09-10T09:53:33.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:33.147+0000 7f7928a00640 1 Processor -- start 2024-09-10T09:53:33.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:33.148+0000 7f7928a00640 1 -- start start 2024-09-10T09:53:33.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:33.148+0000 7f7928a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7924102b90 0x7f79241982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:33.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:33.148+0000 7f7928a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f79241034b0 0x7f7924198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:33.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:33.148+0000 7f7928a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f792410bfc0 0x7f792419cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:33.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:33.148+0000 7f7928a00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7924111dc0 con 0x7f79241034b0 2024-09-10T09:53:33.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:33.148+0000 7f7928a00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7924111c40 con 0x7f7924102b90 2024-09-10T09:53:33.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:33.148+0000 7f7928a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7924111f40 con 0x7f792410bfc0 2024-09-10T09:53:33.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:33.148+0000 7f791aa00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f79241034b0 0x7f7924198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:33.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:33.148+0000 7f791aa00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f79241034b0 0x7f7924198800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42346/0 (socket says 172.21.15.77:42346) 2024-09-10T09:53:33.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:33.148+0000 7f791aa00640 1 -- 172.21.15.77:0/3446411185 learned_addr learned my addr 172.21.15.77:0/3446411185 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:53:33.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:33.148+0000 7f7923400640 1 --2- 172.21.15.77:0/3446411185 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7924102b90 0x7f79241982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:33.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:33.149+0000 7f7923e00640 1 --2- 172.21.15.77:0/3446411185 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f792410bfc0 0x7f792419cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:33.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:33.149+0000 7f791aa00640 1 -- 172.21.15.77:0/3446411185 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f792410bfc0 msgr2=0x7f792419cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:33.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:33.149+0000 7f791aa00640 1 --2- 172.21.15.77:0/3446411185 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f792410bfc0 0x7f792419cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:33.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:33.149+0000 7f791aa00640 1 -- 172.21.15.77:0/3446411185 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7924102b90 msgr2=0x7f79241982c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:33.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:33.149+0000 7f791aa00640 1 --2- 172.21.15.77:0/3446411185 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7924102b90 0x7f79241982c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:33.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:33.149+0000 7f791aa00640 1 -- 172.21.15.77:0/3446411185 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f792419d270 con 0x7f79241034b0 2024-09-10T09:53:33.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:33.149+0000 7f7923400640 1 --2- 172.21.15.77:0/3446411185 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7924102b90 0x7f79241982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:53:33.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:33.149+0000 7f791aa00640 1 --2- 172.21.15.77:0/3446411185 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f79241034b0 0x7f7924198800 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f790c00e9f0 tx=0x7f790c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:33.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:33.149+0000 7f7920c00640 1 -- 172.21.15.77:0/3446411185 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f790c00ccc0 con 0x7f79241034b0 2024-09-10T09:53:33.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:33.150+0000 7f7928a00640 1 -- 172.21.15.77:0/3446411185 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f792419d560 con 0x7f79241034b0 2024-09-10T09:53:33.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:33.150+0000 7f7920c00640 1 -- 172.21.15.77:0/3446411185 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f790c010910 con 0x7f79241034b0 2024-09-10T09:53:33.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:33.150+0000 7f7920c00640 1 -- 172.21.15.77:0/3446411185 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f790c018920 con 0x7f79241034b0 2024-09-10T09:53:33.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:33.150+0000 7f7928a00640 1 -- 172.21.15.77:0/3446411185 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79241a4e40 con 0x7f79241034b0 2024-09-10T09:53:33.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:33.151+0000 7f7928a00640 1 -- 172.21.15.77:0/3446411185 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7924102f70 con 0x7f79241034b0 2024-09-10T09:53:33.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:33.153+0000 7f7920c00640 1 -- 172.21.15.77:0/3446411185 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f790c010430 con 0x7f79241034b0 2024-09-10T09:53:33.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:33.155+0000 7f7920c00640 1 --2- 172.21.15.77:0/3446411185 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f79080a2c10 0x7f79080a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:33.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:33.155+0000 7f7920c00640 1 -- 172.21.15.77:0/3446411185 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f790c0cfc70 con 0x7f79241034b0 2024-09-10T09:53:33.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:33.155+0000 7f7923400640 1 --2- 172.21.15.77:0/3446411185 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f79080a2c10 0x7f79080a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:33.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:33.156+0000 7f7923400640 1 --2- 172.21.15.77:0/3446411185 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f79080a2c10 0x7f79080a50d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7914009fc0 tx=0x7f7914005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:33.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:33.156+0000 7f7920c00640 1 -- 172.21.15.77:0/3446411185 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f790c09c060 con 0x7f79241034b0 2024-09-10T09:53:33.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:32 smithi120 ceph-mon[34770]: pgmap v340: 129 pgs: 129 active+clean; 632 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 4.8 KiB/s wr, 12 op/s 2024-09-10T09:53:33.333 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:33.332+0000 7f7928a00640 1 -- 172.21.15.77:0/3446411185 --> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7924062630 con 0x7f79080a2c10 2024-09-10T09:53:33.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:33.342+0000 7f7920c00640 1 -- 172.21.15.77:0/3446411185 <== mgr.14150 v2:172.21.15.77:6800/4261324650 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340727 (secure 0 0 0) 0x7f7924062630 con 0x7f79080a2c10 2024-09-10T09:53:33.348 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:53:33.350 INFO:teuthology.orchestra.run.smithi077.stderr:dumped all 2024-09-10T09:53:33.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:33.347+0000 7f7928a00640 1 -- 172.21.15.77:0/3446411185 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f79080a2c10 msgr2=0x7f79080a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:33.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:33.347+0000 7f7928a00640 1 --2- 172.21.15.77:0/3446411185 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f79080a2c10 0x7f79080a50d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7914009fc0 tx=0x7f7914005f90 comp rx=0 tx=0).stop 2024-09-10T09:53:33.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:33.347+0000 7f7928a00640 1 -- 172.21.15.77:0/3446411185 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f79241034b0 msgr2=0x7f7924198800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:33.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:33.347+0000 7f7928a00640 1 --2- 172.21.15.77:0/3446411185 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f79241034b0 0x7f7924198800 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f790c00e9f0 tx=0x7f790c00eeb0 comp rx=0 tx=0).stop 2024-09-10T09:53:33.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:33.347+0000 7f7928a00640 1 -- 172.21.15.77:0/3446411185 shutdown_connections 2024-09-10T09:53:33.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:33.347+0000 7f7928a00640 1 --2- 172.21.15.77:0/3446411185 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f79080a2c10 0x7f79080a50d0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:33.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:33.347+0000 7f7928a00640 1 --2- 172.21.15.77:0/3446411185 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f792410bfc0 0x7f792419cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:33.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:33.347+0000 7f7928a00640 1 --2- 172.21.15.77:0/3446411185 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f79241034b0 0x7f7924198800 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:33.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:33.347+0000 7f7928a00640 1 --2- 172.21.15.77:0/3446411185 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7924102b90 0x7f79241982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:33.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:33.347+0000 7f7928a00640 1 -- 172.21.15.77:0/3446411185 >> 172.21.15.77:0/3446411185 conn(0x7f79240fc610 msgr2=0x7f79240fd310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:33.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:33.348+0000 7f7928a00640 1 -- 172.21.15.77:0/3446411185 shutdown_connections 2024-09-10T09:53:33.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:33.348+0000 7f7928a00640 1 -- 172.21.15.77:0/3446411185 wait complete. 2024-09-10T09:53:34.167 INFO:teuthology.orchestra.run.smithi077.stdout:{"pg_ready":true,"pg_map":{"version":340,"stamp":"2024-09-10T09:53:31.688628+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":646908,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":281,"num_read_kb":251,"num_write":418,"num_write_kb":3063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1185162,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":343,"ondisk_log_size":343,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":366136,"kb_used_data":12728,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124575688,"statfs":{"total":1151940427776,"available":1151565504512,"internally_reserved":0,"allocated":13033472,"data_stored":8328362,"data_compressed":433053,"data_compressed_allocated":589824,"data_compressed_original":2678784,"omap_allocated":19143,"internal_metadata":361674041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":46274,"num_objects":4,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":119,"num_read_kb":111,"num_write":32,"num_write_kb":58,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003615"},"pg_stats":[{"pgid":"2.1f","version":"94'13","reported_seq":77,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:29.905424+0000","last_change":"2024-09-10T09:53:00.803590+0000","last_active":"2024-09-10T09:53:29.905424+0000","last_peered":"2024-09-10T09:53:29.905424+0000","last_clean":"2024-09-10T09:53:29.905424+0000","last_became_active":"2024-09-10T09:53:00.798189+0000","last_became_peered":"2024-09-10T09:53:00.798189+0000","last_unstale":"2024-09-10T09:53:29.905424+0000","last_undegraded":"2024-09-10T09:53:29.905424+0000","last_fullsized":"2024-09-10T09:53:29.905424+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:18:48.808836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.792133+0000","last_change":"2024-09-10T09:52:22.318195+0000","last_active":"2024-09-10T09:53:00.792133+0000","last_peered":"2024-09-10T09:53:00.792133+0000","last_clean":"2024-09-10T09:53:00.792133+0000","last_became_active":"2024-09-10T09:52:22.315905+0000","last_became_peered":"2024-09-10T09:52:22.315905+0000","last_unstale":"2024-09-10T09:53:00.792133+0000","last_undegraded":"2024-09-10T09:53:00.792133+0000","last_fullsized":"2024-09-10T09:53:00.792133+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:53:55.515984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.791326+0000","last_change":"2024-09-10T09:52:31.369780+0000","last_active":"2024-09-10T09:53:00.791326+0000","last_peered":"2024-09-10T09:53:00.791326+0000","last_clean":"2024-09-10T09:53:00.791326+0000","last_became_active":"2024-09-10T09:52:31.369238+0000","last_became_peered":"2024-09-10T09:52:31.369238+0000","last_unstale":"2024-09-10T09:53:00.791326+0000","last_undegraded":"2024-09-10T09:53:00.791326+0000","last_fullsized":"2024-09-10T09:53:00.791326+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:39:05.318493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"94'2","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.792494+0000","last_change":"2024-09-10T09:52:19.912276+0000","last_active":"2024-09-10T09:53:00.792494+0000","last_peered":"2024-09-10T09:53:00.792494+0000","last_clean":"2024-09-10T09:53:00.792494+0000","last_became_active":"2024-09-10T09:52:19.911452+0000","last_became_peered":"2024-09-10T09:52:19.911452+0000","last_unstale":"2024-09-10T09:53:00.792494+0000","last_undegraded":"2024-09-10T09:53:00.792494+0000","last_fullsized":"2024-09-10T09:53:00.792494+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:24:31.262207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.845076+0000","last_change":"2024-09-10T09:52:22.327641+0000","last_active":"2024-09-10T09:53:00.845076+0000","last_peered":"2024-09-10T09:53:00.845076+0000","last_clean":"2024-09-10T09:53:00.845076+0000","last_became_active":"2024-09-10T09:52:22.326969+0000","last_became_peered":"2024-09-10T09:52:22.326969+0000","last_unstale":"2024-09-10T09:53:00.845076+0000","last_undegraded":"2024-09-10T09:53:00.845076+0000","last_fullsized":"2024-09-10T09:53:00.845076+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:35:27.825534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:01.217030+0000","last_change":"2024-09-10T09:52:31.372360+0000","last_active":"2024-09-10T09:53:01.217030+0000","last_peered":"2024-09-10T09:53:01.217030+0000","last_clean":"2024-09-10T09:53:01.217030+0000","last_became_active":"2024-09-10T09:52:31.371957+0000","last_became_peered":"2024-09-10T09:52:31.371957+0000","last_unstale":"2024-09-10T09:53:01.217030+0000","last_undegraded":"2024-09-10T09:53:01.217030+0000","last_fullsized":"2024-09-10T09:53:01.217030+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:49:34.776185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"94'12","reported_seq":49,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:16.883154+0000","last_change":"2024-09-10T09:52:19.843532+0000","last_active":"2024-09-10T09:53:16.883154+0000","last_peered":"2024-09-10T09:53:16.883154+0000","last_clean":"2024-09-10T09:53:16.883154+0000","last_became_active":"2024-09-10T09:52:19.843070+0000","last_became_peered":"2024-09-10T09:52:19.843070+0000","last_unstale":"2024-09-10T09:53:16.883154+0000","last_undegraded":"2024-09-10T09:53:16.883154+0000","last_fullsized":"2024-09-10T09:53:16.883154+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:08:02.223373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.794657+0000","last_change":"2024-09-10T09:52:22.316981+0000","last_active":"2024-09-10T09:53:00.794657+0000","last_peered":"2024-09-10T09:53:00.794657+0000","last_clean":"2024-09-10T09:53:00.794657+0000","last_became_active":"2024-09-10T09:52:22.316648+0000","last_became_peered":"2024-09-10T09:52:22.316648+0000","last_unstale":"2024-09-10T09:53:00.794657+0000","last_undegraded":"2024-09-10T09:53:00.794657+0000","last_fullsized":"2024-09-10T09:53:00.794657+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:28:56.742496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:01.217444+0000","last_change":"2024-09-10T09:52:31.376811+0000","last_active":"2024-09-10T09:53:01.217444+0000","last_peered":"2024-09-10T09:53:01.217444+0000","last_clean":"2024-09-10T09:53:01.217444+0000","last_became_active":"2024-09-10T09:52:31.376007+0000","last_became_peered":"2024-09-10T09:52:31.376007+0000","last_unstale":"2024-09-10T09:53:01.217444+0000","last_undegraded":"2024-09-10T09:53:01.217444+0000","last_fullsized":"2024-09-10T09:53:01.217444+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:29:26.213281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"94'5","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:15.875175+0000","last_change":"2024-09-10T09:52:19.837848+0000","last_active":"2024-09-10T09:53:15.875175+0000","last_peered":"2024-09-10T09:53:15.875175+0000","last_clean":"2024-09-10T09:53:15.875175+0000","last_became_active":"2024-09-10T09:52:19.837383+0000","last_became_peered":"2024-09-10T09:52:19.837383+0000","last_unstale":"2024-09-10T09:53:15.875175+0000","last_undegraded":"2024-09-10T09:53:15.875175+0000","last_fullsized":"2024-09-10T09:53:15.875175+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:50:42.813665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.796806+0000","last_change":"2024-09-10T09:52:22.318517+0000","last_active":"2024-09-10T09:53:00.796806+0000","last_peered":"2024-09-10T09:53:00.796806+0000","last_clean":"2024-09-10T09:53:00.796806+0000","last_became_active":"2024-09-10T09:52:22.318207+0000","last_became_peered":"2024-09-10T09:52:22.318207+0000","last_unstale":"2024-09-10T09:53:00.796806+0000","last_undegraded":"2024-09-10T09:53:00.796806+0000","last_fullsized":"2024-09-10T09:53:00.796806+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:03:42.840690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:01.217522+0000","last_change":"2024-09-10T09:52:31.370910+0000","last_active":"2024-09-10T09:53:01.217522+0000","last_peered":"2024-09-10T09:53:01.217522+0000","last_clean":"2024-09-10T09:53:01.217522+0000","last_became_active":"2024-09-10T09:52:31.370467+0000","last_became_peered":"2024-09-10T09:52:31.370467+0000","last_unstale":"2024-09-10T09:53:01.217522+0000","last_undegraded":"2024-09-10T09:53:01.217522+0000","last_fullsized":"2024-09-10T09:53:01.217522+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:49:25.037820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.797176+0000","last_change":"2024-09-10T09:52:22.310708+0000","last_active":"2024-09-10T09:53:00.797176+0000","last_peered":"2024-09-10T09:53:00.797176+0000","last_clean":"2024-09-10T09:53:00.797176+0000","last_became_active":"2024-09-10T09:52:22.310387+0000","last_became_peered":"2024-09-10T09:52:22.310387+0000","last_unstale":"2024-09-10T09:53:00.797176+0000","last_undegraded":"2024-09-10T09:53:00.797176+0000","last_fullsized":"2024-09-10T09:53:00.797176+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:35:03.115408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"94'4","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:12.848949+0000","last_change":"2024-09-10T09:52:19.841417+0000","last_active":"2024-09-10T09:53:12.848949+0000","last_peered":"2024-09-10T09:53:12.848949+0000","last_clean":"2024-09-10T09:53:12.848949+0000","last_became_active":"2024-09-10T09:52:19.841083+0000","last_became_peered":"2024-09-10T09:52:19.841083+0000","last_unstale":"2024-09-10T09:53:12.848949+0000","last_undegraded":"2024-09-10T09:53:12.848949+0000","last_fullsized":"2024-09-10T09:53:12.848949+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:01:19.008345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:01.217723+0000","last_change":"2024-09-10T09:52:31.367406+0000","last_active":"2024-09-10T09:53:01.217723+0000","last_peered":"2024-09-10T09:53:01.217723+0000","last_clean":"2024-09-10T09:53:01.217723+0000","last_became_active":"2024-09-10T09:52:31.366961+0000","last_became_peered":"2024-09-10T09:52:31.366961+0000","last_unstale":"2024-09-10T09:53:01.217723+0000","last_undegraded":"2024-09-10T09:53:01.217723+0000","last_fullsized":"2024-09-10T09:53:01.217723+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:04:51.714465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.791633+0000","last_change":"2024-09-10T09:52:22.329843+0000","last_active":"2024-09-10T09:53:00.791633+0000","last_peered":"2024-09-10T09:53:00.791633+0000","last_clean":"2024-09-10T09:53:00.791633+0000","last_became_active":"2024-09-10T09:52:22.328933+0000","last_became_peered":"2024-09-10T09:52:22.328933+0000","last_unstale":"2024-09-10T09:53:00.791633+0000","last_undegraded":"2024-09-10T09:53:00.791633+0000","last_fullsized":"2024-09-10T09:53:00.791633+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:18:03.409246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"94'3","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:14.865092+0000","last_change":"2024-09-10T09:53:00.800421+0000","last_active":"2024-09-10T09:53:14.865092+0000","last_peered":"2024-09-10T09:53:14.865092+0000","last_clean":"2024-09-10T09:53:14.865092+0000","last_became_active":"2024-09-10T09:53:00.800026+0000","last_became_peered":"2024-09-10T09:53:00.800026+0000","last_unstale":"2024-09-10T09:53:14.865092+0000","last_undegraded":"2024-09-10T09:53:14.865092+0000","last_fullsized":"2024-09-10T09:53:14.865092+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:57:45.252420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"94'1","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:01.217875+0000","last_change":"2024-09-10T09:52:31.376605+0000","last_active":"2024-09-10T09:53:01.217875+0000","last_peered":"2024-09-10T09:53:01.217875+0000","last_clean":"2024-09-10T09:53:01.217875+0000","last_became_active":"2024-09-10T09:52:31.376083+0000","last_became_peered":"2024-09-10T09:52:31.376083+0000","last_unstale":"2024-09-10T09:53:01.217875+0000","last_undegraded":"2024-09-10T09:53:01.217875+0000","last_fullsized":"2024-09-10T09:53:01.217875+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:05:10.030673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.794858+0000","last_change":"2024-09-10T09:52:22.327303+0000","last_active":"2024-09-10T09:53:00.794858+0000","last_peered":"2024-09-10T09:53:00.794858+0000","last_clean":"2024-09-10T09:53:00.794858+0000","last_became_active":"2024-09-10T09:52:22.326753+0000","last_became_peered":"2024-09-10T09:52:22.326753+0000","last_unstale":"2024-09-10T09:53:00.794858+0000","last_undegraded":"2024-09-10T09:53:00.794858+0000","last_fullsized":"2024-09-10T09:53:00.794858+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:27:26.668151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"94'7","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:15.874371+0000","last_change":"2024-09-10T09:52:19.836484+0000","last_active":"2024-09-10T09:53:15.874371+0000","last_peered":"2024-09-10T09:53:15.874371+0000","last_clean":"2024-09-10T09:53:15.874371+0000","last_became_active":"2024-09-10T09:52:19.836130+0000","last_became_peered":"2024-09-10T09:52:19.836130+0000","last_unstale":"2024-09-10T09:53:15.874371+0000","last_undegraded":"2024-09-10T09:53:15.874371+0000","last_fullsized":"2024-09-10T09:53:15.874371+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:31:58.731981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.794698+0000","last_change":"2024-09-10T09:52:31.375279+0000","last_active":"2024-09-10T09:53:00.794698+0000","last_peered":"2024-09-10T09:53:00.794698+0000","last_clean":"2024-09-10T09:53:00.794698+0000","last_became_active":"2024-09-10T09:52:31.374181+0000","last_became_peered":"2024-09-10T09:52:31.374181+0000","last_unstale":"2024-09-10T09:53:00.794698+0000","last_undegraded":"2024-09-10T09:53:00.794698+0000","last_fullsized":"2024-09-10T09:53:00.794698+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:54:51.299224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.797330+0000","last_change":"2024-09-10T09:52:22.314817+0000","last_active":"2024-09-10T09:53:00.797330+0000","last_peered":"2024-09-10T09:53:00.797330+0000","last_clean":"2024-09-10T09:53:00.797330+0000","last_became_active":"2024-09-10T09:52:22.314491+0000","last_became_peered":"2024-09-10T09:52:22.314491+0000","last_unstale":"2024-09-10T09:53:00.797330+0000","last_undegraded":"2024-09-10T09:53:00.797330+0000","last_fullsized":"2024-09-10T09:53:00.797330+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:49:41.941667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"94'2","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:12.848559+0000","last_change":"2024-09-10T09:52:19.911730+0000","last_active":"2024-09-10T09:53:12.848559+0000","last_peered":"2024-09-10T09:53:12.848559+0000","last_clean":"2024-09-10T09:53:12.848559+0000","last_became_active":"2024-09-10T09:52:19.911393+0000","last_became_peered":"2024-09-10T09:52:19.911393+0000","last_unstale":"2024-09-10T09:53:12.848559+0000","last_undegraded":"2024-09-10T09:53:12.848559+0000","last_fullsized":"2024-09-10T09:53:12.848559+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:46:17.943129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:01.217751+0000","last_change":"2024-09-10T09:52:31.376498+0000","last_active":"2024-09-10T09:53:01.217751+0000","last_peered":"2024-09-10T09:53:01.217751+0000","last_clean":"2024-09-10T09:53:01.217751+0000","last_became_active":"2024-09-10T09:52:31.375966+0000","last_became_peered":"2024-09-10T09:52:31.375966+0000","last_unstale":"2024-09-10T09:53:01.217751+0000","last_undegraded":"2024-09-10T09:53:01.217751+0000","last_fullsized":"2024-09-10T09:53:01.217751+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:08:30.649399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.844976+0000","last_change":"2024-09-10T09:52:22.331611+0000","last_active":"2024-09-10T09:53:00.844976+0000","last_peered":"2024-09-10T09:53:00.844976+0000","last_clean":"2024-09-10T09:53:00.844976+0000","last_became_active":"2024-09-10T09:52:22.331268+0000","last_became_peered":"2024-09-10T09:52:22.331268+0000","last_unstale":"2024-09-10T09:53:00.844976+0000","last_undegraded":"2024-09-10T09:53:00.844976+0000","last_fullsized":"2024-09-10T09:53:00.844976+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:01:05.433994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"94'7","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:15.875105+0000","last_change":"2024-09-10T09:52:19.838387+0000","last_active":"2024-09-10T09:53:15.875105+0000","last_peered":"2024-09-10T09:53:15.875105+0000","last_clean":"2024-09-10T09:53:15.875105+0000","last_became_active":"2024-09-10T09:52:19.837889+0000","last_became_peered":"2024-09-10T09:52:19.837889+0000","last_unstale":"2024-09-10T09:53:15.875105+0000","last_undegraded":"2024-09-10T09:53:15.875105+0000","last_fullsized":"2024-09-10T09:53:15.875105+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:50:00.187243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.791474+0000","last_change":"2024-09-10T09:52:31.363262+0000","last_active":"2024-09-10T09:53:00.791474+0000","last_peered":"2024-09-10T09:53:00.791474+0000","last_clean":"2024-09-10T09:53:00.791474+0000","last_became_active":"2024-09-10T09:52:31.362762+0000","last_became_peered":"2024-09-10T09:52:31.362762+0000","last_unstale":"2024-09-10T09:53:00.791474+0000","last_undegraded":"2024-09-10T09:53:00.791474+0000","last_fullsized":"2024-09-10T09:53:00.791474+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:48:28.275041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.792719+0000","last_change":"2024-09-10T09:52:22.312472+0000","last_active":"2024-09-10T09:53:00.792719+0000","last_peered":"2024-09-10T09:53:00.792719+0000","last_clean":"2024-09-10T09:53:00.792719+0000","last_became_active":"2024-09-10T09:52:22.312117+0000","last_became_peered":"2024-09-10T09:52:22.312117+0000","last_unstale":"2024-09-10T09:53:00.792719+0000","last_undegraded":"2024-09-10T09:53:00.792719+0000","last_fullsized":"2024-09-10T09:53:00.792719+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:39:10.742137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"94'12","reported_seq":48,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:28.327159+0000","last_change":"2024-09-10T09:52:19.835543+0000","last_active":"2024-09-10T09:53:28.327159+0000","last_peered":"2024-09-10T09:53:28.327159+0000","last_clean":"2024-09-10T09:53:28.327159+0000","last_became_active":"2024-09-10T09:52:19.835170+0000","last_became_peered":"2024-09-10T09:52:19.835170+0000","last_unstale":"2024-09-10T09:53:28.327159+0000","last_undegraded":"2024-09-10T09:53:28.327159+0000","last_fullsized":"2024-09-10T09:53:28.327159+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:37.969277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.791386+0000","last_change":"2024-09-10T09:52:31.372398+0000","last_active":"2024-09-10T09:53:00.791386+0000","last_peered":"2024-09-10T09:53:00.791386+0000","last_clean":"2024-09-10T09:53:00.791386+0000","last_became_active":"2024-09-10T09:52:31.371874+0000","last_became_peered":"2024-09-10T09:52:31.371874+0000","last_unstale":"2024-09-10T09:53:00.791386+0000","last_undegraded":"2024-09-10T09:53:00.791386+0000","last_fullsized":"2024-09-10T09:53:00.791386+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:09:45.374320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.795543+0000","last_change":"2024-09-10T09:52:22.319592+0000","last_active":"2024-09-10T09:53:00.795543+0000","last_peered":"2024-09-10T09:53:00.795543+0000","last_clean":"2024-09-10T09:53:00.795543+0000","last_became_active":"2024-09-10T09:52:22.319298+0000","last_became_peered":"2024-09-10T09:52:22.319298+0000","last_unstale":"2024-09-10T09:53:00.795543+0000","last_undegraded":"2024-09-10T09:53:00.795543+0000","last_fullsized":"2024-09-10T09:53:00.795543+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:56:59.787638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"94'4","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:15.875681+0000","last_change":"2024-09-10T09:52:19.912923+0000","last_active":"2024-09-10T09:53:15.875681+0000","last_peered":"2024-09-10T09:53:15.875681+0000","last_clean":"2024-09-10T09:53:15.875681+0000","last_became_active":"2024-09-10T09:52:19.912268+0000","last_became_peered":"2024-09-10T09:52:19.912268+0000","last_unstale":"2024-09-10T09:53:15.875681+0000","last_undegraded":"2024-09-10T09:53:15.875681+0000","last_fullsized":"2024-09-10T09:53:15.875681+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:06:45.421004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.794358+0000","last_change":"2024-09-10T09:52:31.372761+0000","last_active":"2024-09-10T09:53:00.794358+0000","last_peered":"2024-09-10T09:53:00.794358+0000","last_clean":"2024-09-10T09:53:00.794358+0000","last_became_active":"2024-09-10T09:52:31.372293+0000","last_became_peered":"2024-09-10T09:52:31.372293+0000","last_unstale":"2024-09-10T09:53:00.794358+0000","last_undegraded":"2024-09-10T09:53:00.794358+0000","last_fullsized":"2024-09-10T09:53:00.794358+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:36:41.446810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.791685+0000","last_change":"2024-09-10T09:52:22.326291+0000","last_active":"2024-09-10T09:53:00.791685+0000","last_peered":"2024-09-10T09:53:00.791685+0000","last_clean":"2024-09-10T09:53:00.791685+0000","last_became_active":"2024-09-10T09:52:22.325985+0000","last_became_peered":"2024-09-10T09:52:22.325985+0000","last_unstale":"2024-09-10T09:53:00.791685+0000","last_undegraded":"2024-09-10T09:53:00.791685+0000","last_fullsized":"2024-09-10T09:53:00.791685+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:06:21.871502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"94'1","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.794398+0000","last_change":"2024-09-10T09:52:19.912851+0000","last_active":"2024-09-10T09:53:00.794398+0000","last_peered":"2024-09-10T09:53:00.794398+0000","last_clean":"2024-09-10T09:53:00.794398+0000","last_became_active":"2024-09-10T09:52:19.912200+0000","last_became_peered":"2024-09-10T09:52:19.912200+0000","last_unstale":"2024-09-10T09:53:00.794398+0000","last_undegraded":"2024-09-10T09:53:00.794398+0000","last_fullsized":"2024-09-10T09:53:00.794398+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:00:47.331755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.798335+0000","last_change":"2024-09-10T09:52:31.371329+0000","last_active":"2024-09-10T09:53:00.798335+0000","last_peered":"2024-09-10T09:53:00.798335+0000","last_clean":"2024-09-10T09:53:00.798335+0000","last_became_active":"2024-09-10T09:52:31.370971+0000","last_became_peered":"2024-09-10T09:52:31.370971+0000","last_unstale":"2024-09-10T09:53:00.798335+0000","last_undegraded":"2024-09-10T09:53:00.798335+0000","last_fullsized":"2024-09-10T09:53:00.798335+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:07:12.928421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.791638+0000","last_change":"2024-09-10T09:52:22.328039+0000","last_active":"2024-09-10T09:53:00.791638+0000","last_peered":"2024-09-10T09:53:00.791638+0000","last_clean":"2024-09-10T09:53:00.791638+0000","last_became_active":"2024-09-10T09:52:22.327588+0000","last_became_peered":"2024-09-10T09:52:22.327588+0000","last_unstale":"2024-09-10T09:53:00.791638+0000","last_undegraded":"2024-09-10T09:53:00.791638+0000","last_fullsized":"2024-09-10T09:53:00.791638+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:35:39.219544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"94'5","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:25.901073+0000","last_change":"2024-09-10T09:53:00.797651+0000","last_active":"2024-09-10T09:53:25.901073+0000","last_peered":"2024-09-10T09:53:25.901073+0000","last_clean":"2024-09-10T09:53:25.901073+0000","last_became_active":"2024-09-10T09:53:00.797285+0000","last_became_peered":"2024-09-10T09:53:00.797285+0000","last_unstale":"2024-09-10T09:53:25.901073+0000","last_undegraded":"2024-09-10T09:53:25.901073+0000","last_fullsized":"2024-09-10T09:53:25.901073+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:18:02.320032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,6,1],"acting":[11,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.795291+0000","last_change":"2024-09-10T09:52:31.379215+0000","last_active":"2024-09-10T09:53:00.795291+0000","last_peered":"2024-09-10T09:53:00.795291+0000","last_clean":"2024-09-10T09:53:00.795291+0000","last_became_active":"2024-09-10T09:52:31.377107+0000","last_became_peered":"2024-09-10T09:52:31.377107+0000","last_unstale":"2024-09-10T09:53:00.795291+0000","last_undegraded":"2024-09-10T09:53:00.795291+0000","last_fullsized":"2024-09-10T09:53:00.795291+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:09:39.474077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.795690+0000","last_change":"2024-09-10T09:52:22.316474+0000","last_active":"2024-09-10T09:53:00.795690+0000","last_peered":"2024-09-10T09:53:00.795690+0000","last_clean":"2024-09-10T09:53:00.795690+0000","last_became_active":"2024-09-10T09:52:22.316054+0000","last_became_peered":"2024-09-10T09:52:22.316054+0000","last_unstale":"2024-09-10T09:53:00.795690+0000","last_undegraded":"2024-09-10T09:53:00.795690+0000","last_fullsized":"2024-09-10T09:53:00.795690+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:11:31.334780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"94'5","reported_seq":55,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:26.900488+0000","last_change":"2024-09-10T09:52:19.911601+0000","last_active":"2024-09-10T09:53:26.900488+0000","last_peered":"2024-09-10T09:53:26.900488+0000","last_clean":"2024-09-10T09:53:26.900488+0000","last_became_active":"2024-09-10T09:52:19.911268+0000","last_became_peered":"2024-09-10T09:52:19.911268+0000","last_unstale":"2024-09-10T09:53:26.900488+0000","last_undegraded":"2024-09-10T09:53:26.900488+0000","last_fullsized":"2024-09-10T09:53:26.900488+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:46:38.158784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.796184+0000","last_change":"2024-09-10T09:52:31.375938+0000","last_active":"2024-09-10T09:53:00.796184+0000","last_peered":"2024-09-10T09:53:00.796184+0000","last_clean":"2024-09-10T09:53:00.796184+0000","last_became_active":"2024-09-10T09:52:31.375536+0000","last_became_peered":"2024-09-10T09:52:31.375536+0000","last_unstale":"2024-09-10T09:53:00.796184+0000","last_undegraded":"2024-09-10T09:53:00.796184+0000","last_fullsized":"2024-09-10T09:53:00.796184+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:55:17.303809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.795033+0000","last_change":"2024-09-10T09:52:22.315912+0000","last_active":"2024-09-10T09:53:00.795033+0000","last_peered":"2024-09-10T09:53:00.795033+0000","last_clean":"2024-09-10T09:53:00.795033+0000","last_became_active":"2024-09-10T09:52:22.315433+0000","last_became_peered":"2024-09-10T09:52:22.315433+0000","last_unstale":"2024-09-10T09:53:00.795033+0000","last_undegraded":"2024-09-10T09:53:00.795033+0000","last_fullsized":"2024-09-10T09:53:00.795033+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:57:10.454716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"94'4","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:13.859070+0000","last_change":"2024-09-10T09:52:19.838966+0000","last_active":"2024-09-10T09:53:13.859070+0000","last_peered":"2024-09-10T09:53:13.859070+0000","last_clean":"2024-09-10T09:53:13.859070+0000","last_became_active":"2024-09-10T09:52:19.838545+0000","last_became_peered":"2024-09-10T09:52:19.838545+0000","last_unstale":"2024-09-10T09:53:13.859070+0000","last_undegraded":"2024-09-10T09:53:13.859070+0000","last_fullsized":"2024-09-10T09:53:13.859070+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:07:00.930146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.802039+0000","last_change":"2024-09-10T09:53:00.802039+0000","last_active":"2024-09-10T09:53:00.802039+0000","last_peered":"2024-09-10T09:53:00.802039+0000","last_clean":"2024-09-10T09:53:00.802039+0000","last_became_active":"2024-09-10T09:53:00.800223+0000","last_became_peered":"2024-09-10T09:53:00.800223+0000","last_unstale":"2024-09-10T09:53:00.802039+0000","last_undegraded":"2024-09-10T09:53:00.802039+0000","last_fullsized":"2024-09-10T09:53:00.802039+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:50:16.026631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,6,10],"acting":[5,8,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.791782+0000","last_change":"2024-09-10T09:52:22.313819+0000","last_active":"2024-09-10T09:53:00.791782+0000","last_peered":"2024-09-10T09:53:00.791782+0000","last_clean":"2024-09-10T09:53:00.791782+0000","last_became_active":"2024-09-10T09:52:22.313473+0000","last_became_peered":"2024-09-10T09:52:22.313473+0000","last_unstale":"2024-09-10T09:53:00.791782+0000","last_undegraded":"2024-09-10T09:53:00.791782+0000","last_fullsized":"2024-09-10T09:53:00.791782+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:44:10.003187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"94'6","reported_seq":46,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:16.883322+0000","last_change":"2024-09-10T09:52:19.837376+0000","last_active":"2024-09-10T09:53:16.883322+0000","last_peered":"2024-09-10T09:53:16.883322+0000","last_clean":"2024-09-10T09:53:16.883322+0000","last_became_active":"2024-09-10T09:52:19.836880+0000","last_became_peered":"2024-09-10T09:52:19.836880+0000","last_unstale":"2024-09-10T09:53:16.883322+0000","last_undegraded":"2024-09-10T09:53:16.883322+0000","last_fullsized":"2024-09-10T09:53:16.883322+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:48:58.198188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":356,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.796687+0000","last_change":"2024-09-10T09:51:05.871529+0000","last_active":"2024-09-10T09:53:00.796687+0000","last_peered":"2024-09-10T09:53:00.796687+0000","last_clean":"2024-09-10T09:53:00.796687+0000","last_became_active":"2024-09-10T09:51:05.871115+0000","last_became_peered":"2024-09-10T09:51:05.871115+0000","last_unstale":"2024-09-10T09:53:00.796687+0000","last_undegraded":"2024-09-10T09:53:00.796687+0000","last_fullsized":"2024-09-10T09:53:00.796687+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:47:24.175358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:47:24.175358+0000","last_clean_scrub_stamp":"2024-09-10T09:47:24.175358+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:58:30.247077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"94'1","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.801000+0000","last_change":"2024-09-10T09:53:00.800474+0000","last_active":"2024-09-10T09:53:00.801000+0000","last_peered":"2024-09-10T09:53:00.801000+0000","last_clean":"2024-09-10T09:53:00.801000+0000","last_became_active":"2024-09-10T09:53:00.799932+0000","last_became_peered":"2024-09-10T09:53:00.799932+0000","last_unstale":"2024-09-10T09:53:00.801000+0000","last_undegraded":"2024-09-10T09:53:00.801000+0000","last_fullsized":"2024-09-10T09:53:00.801000+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:14:50.998952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,11,9],"acting":[7,8,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.797158+0000","last_change":"2024-09-10T09:52:22.312521+0000","last_active":"2024-09-10T09:53:00.797158+0000","last_peered":"2024-09-10T09:53:00.797158+0000","last_clean":"2024-09-10T09:53:00.797158+0000","last_became_active":"2024-09-10T09:52:22.312107+0000","last_became_peered":"2024-09-10T09:52:22.312107+0000","last_unstale":"2024-09-10T09:53:00.797158+0000","last_undegraded":"2024-09-10T09:53:00.797158+0000","last_fullsized":"2024-09-10T09:53:00.797158+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:14:19.799259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"94'5","reported_seq":56,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:26.900351+0000","last_change":"2024-09-10T09:52:19.911454+0000","last_active":"2024-09-10T09:53:26.900351+0000","last_peered":"2024-09-10T09:53:26.900351+0000","last_clean":"2024-09-10T09:53:26.900351+0000","last_became_active":"2024-09-10T09:52:19.911079+0000","last_became_peered":"2024-09-10T09:52:19.911079+0000","last_unstale":"2024-09-10T09:53:26.900351+0000","last_undegraded":"2024-09-10T09:53:26.900351+0000","last_fullsized":"2024-09-10T09:53:26.900351+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:43:38.787689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":14,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.794164+0000","last_change":"2024-09-10T09:52:31.381877+0000","last_active":"2024-09-10T09:53:00.794164+0000","last_peered":"2024-09-10T09:53:00.794164+0000","last_clean":"2024-09-10T09:53:00.794164+0000","last_became_active":"2024-09-10T09:52:31.381472+0000","last_became_peered":"2024-09-10T09:52:31.381472+0000","last_unstale":"2024-09-10T09:53:00.794164+0000","last_undegraded":"2024-09-10T09:53:00.794164+0000","last_fullsized":"2024-09-10T09:53:00.794164+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:41:24.686325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.797216+0000","last_change":"2024-09-10T09:52:22.313902+0000","last_active":"2024-09-10T09:53:00.797216+0000","last_peered":"2024-09-10T09:53:00.797216+0000","last_clean":"2024-09-10T09:53:00.797216+0000","last_became_active":"2024-09-10T09:52:22.313515+0000","last_became_peered":"2024-09-10T09:52:22.313515+0000","last_unstale":"2024-09-10T09:53:00.797216+0000","last_undegraded":"2024-09-10T09:53:00.797216+0000","last_fullsized":"2024-09-10T09:53:00.797216+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:25:00.238307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"94'3","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:15.875823+0000","last_change":"2024-09-10T09:52:19.837736+0000","last_active":"2024-09-10T09:53:15.875823+0000","last_peered":"2024-09-10T09:53:15.875823+0000","last_clean":"2024-09-10T09:53:15.875823+0000","last_became_active":"2024-09-10T09:52:19.837370+0000","last_became_peered":"2024-09-10T09:52:19.837370+0000","last_unstale":"2024-09-10T09:53:15.875823+0000","last_undegraded":"2024-09-10T09:53:15.875823+0000","last_fullsized":"2024-09-10T09:53:15.875823+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:54:35.753027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.844672+0000","last_change":"2024-09-10T09:52:31.378945+0000","last_active":"2024-09-10T09:53:00.844672+0000","last_peered":"2024-09-10T09:53:00.844672+0000","last_clean":"2024-09-10T09:53:00.844672+0000","last_became_active":"2024-09-10T09:52:31.378144+0000","last_became_peered":"2024-09-10T09:52:31.378144+0000","last_unstale":"2024-09-10T09:53:00.844672+0000","last_undegraded":"2024-09-10T09:53:00.844672+0000","last_fullsized":"2024-09-10T09:53:00.844672+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:36:06.288293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.797422+0000","last_change":"2024-09-10T09:52:22.316400+0000","last_active":"2024-09-10T09:53:00.797422+0000","last_peered":"2024-09-10T09:53:00.797422+0000","last_clean":"2024-09-10T09:53:00.797422+0000","last_became_active":"2024-09-10T09:52:22.315974+0000","last_became_peered":"2024-09-10T09:52:22.315974+0000","last_unstale":"2024-09-10T09:53:00.797422+0000","last_undegraded":"2024-09-10T09:53:00.797422+0000","last_fullsized":"2024-09-10T09:53:00.797422+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:31:05.538237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"94'5","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:15.874169+0000","last_change":"2024-09-10T09:52:19.837942+0000","last_active":"2024-09-10T09:53:15.874169+0000","last_peered":"2024-09-10T09:53:15.874169+0000","last_clean":"2024-09-10T09:53:15.874169+0000","last_became_active":"2024-09-10T09:52:19.837629+0000","last_became_peered":"2024-09-10T09:52:19.837629+0000","last_unstale":"2024-09-10T09:53:15.874169+0000","last_undegraded":"2024-09-10T09:53:15.874169+0000","last_fullsized":"2024-09-10T09:53:15.874169+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:34:13.207770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.794626+0000","last_change":"2024-09-10T09:52:31.364231+0000","last_active":"2024-09-10T09:53:00.794626+0000","last_peered":"2024-09-10T09:53:00.794626+0000","last_clean":"2024-09-10T09:53:00.794626+0000","last_became_active":"2024-09-10T09:52:31.363833+0000","last_became_peered":"2024-09-10T09:52:31.363833+0000","last_unstale":"2024-09-10T09:53:00.794626+0000","last_undegraded":"2024-09-10T09:53:00.794626+0000","last_fullsized":"2024-09-10T09:53:00.794626+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:53:51.435700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.795271+0000","last_change":"2024-09-10T09:52:22.319530+0000","last_active":"2024-09-10T09:53:00.795271+0000","last_peered":"2024-09-10T09:53:00.795271+0000","last_clean":"2024-09-10T09:53:00.795271+0000","last_became_active":"2024-09-10T09:52:22.318742+0000","last_became_peered":"2024-09-10T09:52:22.318742+0000","last_unstale":"2024-09-10T09:53:00.795271+0000","last_undegraded":"2024-09-10T09:53:00.795271+0000","last_fullsized":"2024-09-10T09:53:00.795271+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:44:35.050454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"94'2","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:14.854723+0000","last_change":"2024-09-10T09:52:19.836398+0000","last_active":"2024-09-10T09:53:14.854723+0000","last_peered":"2024-09-10T09:53:14.854723+0000","last_clean":"2024-09-10T09:53:14.854723+0000","last_became_active":"2024-09-10T09:52:19.836055+0000","last_became_peered":"2024-09-10T09:52:19.836055+0000","last_unstale":"2024-09-10T09:53:14.854723+0000","last_undegraded":"2024-09-10T09:53:14.854723+0000","last_fullsized":"2024-09-10T09:53:14.854723+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:49:26.923367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.794659+0000","last_change":"2024-09-10T09:52:31.371109+0000","last_active":"2024-09-10T09:53:00.794659+0000","last_peered":"2024-09-10T09:53:00.794659+0000","last_clean":"2024-09-10T09:53:00.794659+0000","last_became_active":"2024-09-10T09:52:31.370592+0000","last_became_peered":"2024-09-10T09:52:31.370592+0000","last_unstale":"2024-09-10T09:53:00.794659+0000","last_undegraded":"2024-09-10T09:53:00.794659+0000","last_fullsized":"2024-09-10T09:53:00.794659+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:20:23.585187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.794797+0000","last_change":"2024-09-10T09:52:22.319365+0000","last_active":"2024-09-10T09:53:00.794797+0000","last_peered":"2024-09-10T09:53:00.794797+0000","last_clean":"2024-09-10T09:53:00.794797+0000","last_became_active":"2024-09-10T09:52:22.319064+0000","last_became_peered":"2024-09-10T09:52:22.319064+0000","last_unstale":"2024-09-10T09:53:00.794797+0000","last_undegraded":"2024-09-10T09:53:00.794797+0000","last_fullsized":"2024-09-10T09:53:00.794797+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:14:50.089566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"94'17","reported_seq":54,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:15.875370+0000","last_change":"2024-09-10T09:52:19.911698+0000","last_active":"2024-09-10T09:53:15.875370+0000","last_peered":"2024-09-10T09:53:15.875370+0000","last_clean":"2024-09-10T09:53:15.875370+0000","last_became_active":"2024-09-10T09:52:19.911369+0000","last_became_peered":"2024-09-10T09:52:19.911369+0000","last_unstale":"2024-09-10T09:53:15.875370+0000","last_undegraded":"2024-09-10T09:53:15.875370+0000","last_fullsized":"2024-09-10T09:53:15.875370+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:40:39.045797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.796024+0000","last_change":"2024-09-10T09:52:31.380463+0000","last_active":"2024-09-10T09:53:00.796024+0000","last_peered":"2024-09-10T09:53:00.796024+0000","last_clean":"2024-09-10T09:53:00.796024+0000","last_became_active":"2024-09-10T09:52:31.380114+0000","last_became_peered":"2024-09-10T09:52:31.380114+0000","last_unstale":"2024-09-10T09:53:00.796024+0000","last_undegraded":"2024-09-10T09:53:00.796024+0000","last_fullsized":"2024-09-10T09:53:00.796024+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:19:50.869160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.796355+0000","last_change":"2024-09-10T09:52:22.313928+0000","last_active":"2024-09-10T09:53:00.796355+0000","last_peered":"2024-09-10T09:53:00.796355+0000","last_clean":"2024-09-10T09:53:00.796355+0000","last_became_active":"2024-09-10T09:52:22.313510+0000","last_became_peered":"2024-09-10T09:52:22.313510+0000","last_unstale":"2024-09-10T09:53:00.796355+0000","last_undegraded":"2024-09-10T09:53:00.796355+0000","last_fullsized":"2024-09-10T09:53:00.796355+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:58:19.031178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"94'6","reported_seq":58,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:26.901026+0000","last_change":"2024-09-10T09:52:19.839451+0000","last_active":"2024-09-10T09:53:26.901026+0000","last_peered":"2024-09-10T09:53:26.901026+0000","last_clean":"2024-09-10T09:53:26.901026+0000","last_became_active":"2024-09-10T09:52:19.839124+0000","last_became_peered":"2024-09-10T09:52:19.839124+0000","last_unstale":"2024-09-10T09:53:26.901026+0000","last_undegraded":"2024-09-10T09:53:26.901026+0000","last_fullsized":"2024-09-10T09:53:26.901026+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:51:33.489830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":16,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.794814+0000","last_change":"2024-09-10T09:52:31.376386+0000","last_active":"2024-09-10T09:53:00.794814+0000","last_peered":"2024-09-10T09:53:00.794814+0000","last_clean":"2024-09-10T09:53:00.794814+0000","last_became_active":"2024-09-10T09:52:31.376066+0000","last_became_peered":"2024-09-10T09:52:31.376066+0000","last_unstale":"2024-09-10T09:53:00.794814+0000","last_undegraded":"2024-09-10T09:53:00.794814+0000","last_fullsized":"2024-09-10T09:53:00.794814+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:23:07.191842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.796616+0000","last_change":"2024-09-10T09:52:22.323590+0000","last_active":"2024-09-10T09:53:00.796616+0000","last_peered":"2024-09-10T09:53:00.796616+0000","last_clean":"2024-09-10T09:53:00.796616+0000","last_became_active":"2024-09-10T09:52:22.323264+0000","last_became_peered":"2024-09-10T09:52:22.323264+0000","last_unstale":"2024-09-10T09:53:00.796616+0000","last_undegraded":"2024-09-10T09:53:00.796616+0000","last_fullsized":"2024-09-10T09:53:00.796616+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:07:56.572337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"94'5","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:15.875891+0000","last_change":"2024-09-10T09:52:19.840620+0000","last_active":"2024-09-10T09:53:15.875891+0000","last_peered":"2024-09-10T09:53:15.875891+0000","last_clean":"2024-09-10T09:53:15.875891+0000","last_became_active":"2024-09-10T09:52:19.840217+0000","last_became_peered":"2024-09-10T09:52:19.840217+0000","last_unstale":"2024-09-10T09:53:15.875891+0000","last_undegraded":"2024-09-10T09:53:15.875891+0000","last_fullsized":"2024-09-10T09:53:15.875891+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:55:13.111993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.794521+0000","last_change":"2024-09-10T09:52:31.373895+0000","last_active":"2024-09-10T09:53:00.794521+0000","last_peered":"2024-09-10T09:53:00.794521+0000","last_clean":"2024-09-10T09:53:00.794521+0000","last_became_active":"2024-09-10T09:52:31.373474+0000","last_became_peered":"2024-09-10T09:52:31.373474+0000","last_unstale":"2024-09-10T09:53:00.794521+0000","last_undegraded":"2024-09-10T09:53:00.794521+0000","last_fullsized":"2024-09-10T09:53:00.794521+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:39:16.049859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.796069+0000","last_change":"2024-09-10T09:52:22.319294+0000","last_active":"2024-09-10T09:53:00.796069+0000","last_peered":"2024-09-10T09:53:00.796069+0000","last_clean":"2024-09-10T09:53:00.796069+0000","last_became_active":"2024-09-10T09:52:22.318918+0000","last_became_peered":"2024-09-10T09:52:22.318918+0000","last_unstale":"2024-09-10T09:53:00.796069+0000","last_undegraded":"2024-09-10T09:53:00.796069+0000","last_fullsized":"2024-09-10T09:53:00.796069+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:52:46.434182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"94'9","reported_seq":48,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:16.883211+0000","last_change":"2024-09-10T09:52:19.842286+0000","last_active":"2024-09-10T09:53:16.883211+0000","last_peered":"2024-09-10T09:53:16.883211+0000","last_clean":"2024-09-10T09:53:16.883211+0000","last_became_active":"2024-09-10T09:52:19.841930+0000","last_became_peered":"2024-09-10T09:52:19.841930+0000","last_unstale":"2024-09-10T09:53:16.883211+0000","last_undegraded":"2024-09-10T09:53:16.883211+0000","last_fullsized":"2024-09-10T09:53:16.883211+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:38:46.358389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.796684+0000","last_change":"2024-09-10T09:52:31.370961+0000","last_active":"2024-09-10T09:53:00.796684+0000","last_peered":"2024-09-10T09:53:00.796684+0000","last_clean":"2024-09-10T09:53:00.796684+0000","last_became_active":"2024-09-10T09:52:31.370545+0000","last_became_peered":"2024-09-10T09:52:31.370545+0000","last_unstale":"2024-09-10T09:53:00.796684+0000","last_undegraded":"2024-09-10T09:53:00.796684+0000","last_fullsized":"2024-09-10T09:53:00.796684+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:17.461447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.798764+0000","last_change":"2024-09-10T09:52:22.313215+0000","last_active":"2024-09-10T09:53:00.798764+0000","last_peered":"2024-09-10T09:53:00.798764+0000","last_clean":"2024-09-10T09:53:00.798764+0000","last_became_active":"2024-09-10T09:52:22.312918+0000","last_became_peered":"2024-09-10T09:52:22.312918+0000","last_unstale":"2024-09-10T09:53:00.798764+0000","last_undegraded":"2024-09-10T09:53:00.798764+0000","last_fullsized":"2024-09-10T09:53:00.798764+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:48:09.568004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"94'28","reported_seq":109,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:26.900802+0000","last_change":"2024-09-10T09:53:01.226146+0000","last_active":"2024-09-10T09:53:26.900802+0000","last_peered":"2024-09-10T09:53:26.900802+0000","last_clean":"2024-09-10T09:53:26.900802+0000","last_became_active":"2024-09-10T09:53:01.218622+0000","last_became_peered":"2024-09-10T09:53:01.218622+0000","last_unstale":"2024-09-10T09:53:26.900802+0000","last_undegraded":"2024-09-10T09:53:26.900802+0000","last_fullsized":"2024-09-10T09:53:26.900802+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:43:11.927029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49098,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":61,"num_write":30,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":3844,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.801292+0000","last_change":"2024-09-10T09:53:00.800650+0000","last_active":"2024-09-10T09:53:00.801292+0000","last_peered":"2024-09-10T09:53:00.801292+0000","last_clean":"2024-09-10T09:53:00.801292+0000","last_became_active":"2024-09-10T09:53:00.800104+0000","last_became_peered":"2024-09-10T09:53:00.800104+0000","last_unstale":"2024-09-10T09:53:00.801292+0000","last_undegraded":"2024-09-10T09:53:00.801292+0000","last_fullsized":"2024-09-10T09:53:00.801292+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:41:16.022884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.794669+0000","last_change":"2024-09-10T09:52:22.317174+0000","last_active":"2024-09-10T09:53:00.794669+0000","last_peered":"2024-09-10T09:53:00.794669+0000","last_clean":"2024-09-10T09:53:00.794669+0000","last_became_active":"2024-09-10T09:52:22.316851+0000","last_became_peered":"2024-09-10T09:52:22.316851+0000","last_unstale":"2024-09-10T09:53:00.794669+0000","last_undegraded":"2024-09-10T09:53:00.794669+0000","last_fullsized":"2024-09-10T09:53:00.794669+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:55:18.742393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"94'5","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:14.863925+0000","last_change":"2024-09-10T09:52:19.912093+0000","last_active":"2024-09-10T09:53:14.863925+0000","last_peered":"2024-09-10T09:53:14.863925+0000","last_clean":"2024-09-10T09:53:14.863925+0000","last_became_active":"2024-09-10T09:52:19.911611+0000","last_became_peered":"2024-09-10T09:52:19.911611+0000","last_unstale":"2024-09-10T09:53:14.863925+0000","last_undegraded":"2024-09-10T09:53:14.863925+0000","last_fullsized":"2024-09-10T09:53:14.863925+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:01:10.859310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:01.217603+0000","last_change":"2024-09-10T09:52:31.362884+0000","last_active":"2024-09-10T09:53:01.217603+0000","last_peered":"2024-09-10T09:53:01.217603+0000","last_clean":"2024-09-10T09:53:01.217603+0000","last_became_active":"2024-09-10T09:52:31.362489+0000","last_became_peered":"2024-09-10T09:52:31.362489+0000","last_unstale":"2024-09-10T09:53:01.217603+0000","last_undegraded":"2024-09-10T09:53:01.217603+0000","last_fullsized":"2024-09-10T09:53:01.217603+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:02:33.084795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.845137+0000","last_change":"2024-09-10T09:52:22.325871+0000","last_active":"2024-09-10T09:53:00.845137+0000","last_peered":"2024-09-10T09:53:00.845137+0000","last_clean":"2024-09-10T09:53:00.845137+0000","last_became_active":"2024-09-10T09:52:22.324980+0000","last_became_peered":"2024-09-10T09:52:22.324980+0000","last_unstale":"2024-09-10T09:53:00.845137+0000","last_undegraded":"2024-09-10T09:53:00.845137+0000","last_fullsized":"2024-09-10T09:53:00.845137+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:31:55.713444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"94'4","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:15.874898+0000","last_change":"2024-09-10T09:52:19.839491+0000","last_active":"2024-09-10T09:53:15.874898+0000","last_peered":"2024-09-10T09:53:15.874898+0000","last_clean":"2024-09-10T09:53:15.874898+0000","last_became_active":"2024-09-10T09:52:19.839092+0000","last_became_peered":"2024-09-10T09:52:19.839092+0000","last_unstale":"2024-09-10T09:53:15.874898+0000","last_undegraded":"2024-09-10T09:53:15.874898+0000","last_fullsized":"2024-09-10T09:53:15.874898+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:42:13.453281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.793758+0000","last_change":"2024-09-10T09:52:31.371346+0000","last_active":"2024-09-10T09:53:00.793758+0000","last_peered":"2024-09-10T09:53:00.793758+0000","last_clean":"2024-09-10T09:53:00.793758+0000","last_became_active":"2024-09-10T09:52:31.370929+0000","last_became_peered":"2024-09-10T09:52:31.370929+0000","last_unstale":"2024-09-10T09:53:00.793758+0000","last_undegraded":"2024-09-10T09:53:00.793758+0000","last_fullsized":"2024-09-10T09:53:00.793758+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:44:22.925082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.794905+0000","last_change":"2024-09-10T09:52:22.317453+0000","last_active":"2024-09-10T09:53:00.794905+0000","last_peered":"2024-09-10T09:53:00.794905+0000","last_clean":"2024-09-10T09:53:00.794905+0000","last_became_active":"2024-09-10T09:52:22.315556+0000","last_became_peered":"2024-09-10T09:52:22.315556+0000","last_unstale":"2024-09-10T09:53:00.794905+0000","last_undegraded":"2024-09-10T09:53:00.794905+0000","last_fullsized":"2024-09-10T09:53:00.794905+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:56:05.860275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"94'4","reported_seq":64,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:26.899540+0000","last_change":"2024-09-10T09:52:19.835985+0000","last_active":"2024-09-10T09:53:26.899540+0000","last_peered":"2024-09-10T09:53:26.899540+0000","last_clean":"2024-09-10T09:53:26.899540+0000","last_became_active":"2024-09-10T09:52:19.834835+0000","last_became_peered":"2024-09-10T09:52:19.834835+0000","last_unstale":"2024-09-10T09:53:26.899540+0000","last_undegraded":"2024-09-10T09:53:26.899540+0000","last_fullsized":"2024-09-10T09:53:26.899540+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:09:11.590443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":22,"num_read_kb":22,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.791473+0000","last_change":"2024-09-10T09:52:31.368663+0000","last_active":"2024-09-10T09:53:00.791473+0000","last_peered":"2024-09-10T09:53:00.791473+0000","last_clean":"2024-09-10T09:53:00.791473+0000","last_became_active":"2024-09-10T09:52:31.368004+0000","last_became_peered":"2024-09-10T09:52:31.368004+0000","last_unstale":"2024-09-10T09:53:00.791473+0000","last_undegraded":"2024-09-10T09:53:00.791473+0000","last_fullsized":"2024-09-10T09:53:00.791473+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:56:15.100515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.844866+0000","last_change":"2024-09-10T09:52:31.370992+0000","last_active":"2024-09-10T09:53:00.844866+0000","last_peered":"2024-09-10T09:53:00.844866+0000","last_clean":"2024-09-10T09:53:00.844866+0000","last_became_active":"2024-09-10T09:52:31.370357+0000","last_became_peered":"2024-09-10T09:52:31.370357+0000","last_unstale":"2024-09-10T09:53:00.844866+0000","last_undegraded":"2024-09-10T09:53:00.844866+0000","last_fullsized":"2024-09-10T09:53:00.844866+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:02:49.482532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.793913+0000","last_change":"2024-09-10T09:52:22.319363+0000","last_active":"2024-09-10T09:53:00.793913+0000","last_peered":"2024-09-10T09:53:00.793913+0000","last_clean":"2024-09-10T09:53:00.793913+0000","last_became_active":"2024-09-10T09:52:22.319044+0000","last_became_peered":"2024-09-10T09:52:22.319044+0000","last_unstale":"2024-09-10T09:53:00.793913+0000","last_undegraded":"2024-09-10T09:53:00.793913+0000","last_fullsized":"2024-09-10T09:53:00.793913+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:30:51.328413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"94'3","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:15.874728+0000","last_change":"2024-09-10T09:52:19.841745+0000","last_active":"2024-09-10T09:53:15.874728+0000","last_peered":"2024-09-10T09:53:15.874728+0000","last_clean":"2024-09-10T09:53:15.874728+0000","last_became_active":"2024-09-10T09:52:19.841404+0000","last_became_peered":"2024-09-10T09:52:19.841404+0000","last_unstale":"2024-09-10T09:53:15.874728+0000","last_undegraded":"2024-09-10T09:53:15.874728+0000","last_fullsized":"2024-09-10T09:53:15.874728+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:44:36.346785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:01.218412+0000","last_change":"2024-09-10T09:53:01.218412+0000","last_active":"2024-09-10T09:53:01.218412+0000","last_peered":"2024-09-10T09:53:01.218412+0000","last_clean":"2024-09-10T09:53:01.218412+0000","last_became_active":"2024-09-10T09:53:01.217966+0000","last_became_peered":"2024-09-10T09:53:01.217966+0000","last_unstale":"2024-09-10T09:53:01.218412+0000","last_undegraded":"2024-09-10T09:53:01.218412+0000","last_fullsized":"2024-09-10T09:53:01.218412+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:53:25.743913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.791538+0000","last_change":"2024-09-10T09:52:22.316120+0000","last_active":"2024-09-10T09:53:00.791538+0000","last_peered":"2024-09-10T09:53:00.791538+0000","last_clean":"2024-09-10T09:53:00.791538+0000","last_became_active":"2024-09-10T09:52:22.315769+0000","last_became_peered":"2024-09-10T09:52:22.315769+0000","last_unstale":"2024-09-10T09:53:00.791538+0000","last_undegraded":"2024-09-10T09:53:00.791538+0000","last_fullsized":"2024-09-10T09:53:00.791538+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:16:41.866587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"94'3","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:13.861889+0000","last_change":"2024-09-10T09:52:19.835038+0000","last_active":"2024-09-10T09:53:13.861889+0000","last_peered":"2024-09-10T09:53:13.861889+0000","last_clean":"2024-09-10T09:53:13.861889+0000","last_became_active":"2024-09-10T09:52:19.834329+0000","last_became_peered":"2024-09-10T09:52:19.834329+0000","last_unstale":"2024-09-10T09:53:13.861889+0000","last_undegraded":"2024-09-10T09:53:13.861889+0000","last_fullsized":"2024-09-10T09:53:13.861889+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:58:24.084373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"94'7","reported_seq":51,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:14.865882+0000","last_change":"2024-09-10T09:53:01.219122+0000","last_active":"2024-09-10T09:53:14.865882+0000","last_peered":"2024-09-10T09:53:14.865882+0000","last_clean":"2024-09-10T09:53:14.865882+0000","last_became_active":"2024-09-10T09:53:01.218599+0000","last_became_peered":"2024-09-10T09:53:01.218599+0000","last_unstale":"2024-09-10T09:53:14.865882+0000","last_undegraded":"2024-09-10T09:53:14.865882+0000","last_fullsized":"2024-09-10T09:53:14.865882+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:31:10.092264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.795295+0000","last_change":"2024-09-10T09:52:22.316688+0000","last_active":"2024-09-10T09:53:00.795295+0000","last_peered":"2024-09-10T09:53:00.795295+0000","last_clean":"2024-09-10T09:53:00.795295+0000","last_became_active":"2024-09-10T09:52:22.316388+0000","last_became_peered":"2024-09-10T09:52:22.316388+0000","last_unstale":"2024-09-10T09:53:00.795295+0000","last_undegraded":"2024-09-10T09:53:00.795295+0000","last_fullsized":"2024-09-10T09:53:00.795295+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:30:46.219846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.791663+0000","last_change":"2024-09-10T09:52:31.371612+0000","last_active":"2024-09-10T09:53:00.791663+0000","last_peered":"2024-09-10T09:53:00.791663+0000","last_clean":"2024-09-10T09:53:00.791663+0000","last_became_active":"2024-09-10T09:52:31.371163+0000","last_became_peered":"2024-09-10T09:52:31.371163+0000","last_unstale":"2024-09-10T09:53:00.791663+0000","last_undegraded":"2024-09-10T09:53:00.791663+0000","last_fullsized":"2024-09-10T09:53:00.791663+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:11:43.681524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"94'6","reported_seq":59,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:26.900278+0000","last_change":"2024-09-10T09:52:19.841873+0000","last_active":"2024-09-10T09:53:26.900278+0000","last_peered":"2024-09-10T09:53:26.900278+0000","last_clean":"2024-09-10T09:53:26.900278+0000","last_became_active":"2024-09-10T09:52:19.841513+0000","last_became_peered":"2024-09-10T09:52:19.841513+0000","last_unstale":"2024-09-10T09:53:26.900278+0000","last_undegraded":"2024-09-10T09:53:26.900278+0000","last_fullsized":"2024-09-10T09:53:26.900278+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:53:31.854185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.798538+0000","last_change":"2024-09-10T09:52:22.314781+0000","last_active":"2024-09-10T09:53:00.798538+0000","last_peered":"2024-09-10T09:53:00.798538+0000","last_clean":"2024-09-10T09:53:00.798538+0000","last_became_active":"2024-09-10T09:52:22.313346+0000","last_became_peered":"2024-09-10T09:52:22.313346+0000","last_unstale":"2024-09-10T09:53:00.798538+0000","last_undegraded":"2024-09-10T09:53:00.798538+0000","last_fullsized":"2024-09-10T09:53:00.798538+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:15:16.921963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.791726+0000","last_change":"2024-09-10T09:52:31.365544+0000","last_active":"2024-09-10T09:53:00.791726+0000","last_peered":"2024-09-10T09:53:00.791726+0000","last_clean":"2024-09-10T09:53:00.791726+0000","last_became_active":"2024-09-10T09:52:31.365151+0000","last_became_peered":"2024-09-10T09:52:31.365151+0000","last_unstale":"2024-09-10T09:53:00.791726+0000","last_undegraded":"2024-09-10T09:53:00.791726+0000","last_fullsized":"2024-09-10T09:53:00.791726+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:36:55.247890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.793781+0000","last_change":"2024-09-10T09:52:22.314117+0000","last_active":"2024-09-10T09:53:00.793781+0000","last_peered":"2024-09-10T09:53:00.793781+0000","last_clean":"2024-09-10T09:53:00.793781+0000","last_became_active":"2024-09-10T09:52:22.313772+0000","last_became_peered":"2024-09-10T09:52:22.313772+0000","last_unstale":"2024-09-10T09:53:00.793781+0000","last_undegraded":"2024-09-10T09:53:00.793781+0000","last_fullsized":"2024-09-10T09:53:00.793781+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:19:37.373598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.792282+0000","last_change":"2024-09-10T09:52:22.314362+0000","last_active":"2024-09-10T09:53:00.792282+0000","last_peered":"2024-09-10T09:53:00.792282+0000","last_clean":"2024-09-10T09:53:00.792282+0000","last_became_active":"2024-09-10T09:52:22.314050+0000","last_became_peered":"2024-09-10T09:52:22.314050+0000","last_unstale":"2024-09-10T09:53:00.792282+0000","last_undegraded":"2024-09-10T09:53:00.792282+0000","last_fullsized":"2024-09-10T09:53:00.792282+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:37:07.218656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.791411+0000","last_change":"2024-09-10T09:52:22.322363+0000","last_active":"2024-09-10T09:53:00.791411+0000","last_peered":"2024-09-10T09:53:00.791411+0000","last_clean":"2024-09-10T09:53:00.791411+0000","last_became_active":"2024-09-10T09:52:22.322045+0000","last_became_peered":"2024-09-10T09:52:22.322045+0000","last_unstale":"2024-09-10T09:53:00.791411+0000","last_undegraded":"2024-09-10T09:53:00.791411+0000","last_fullsized":"2024-09-10T09:53:00.791411+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:10:27.918769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.844741+0000","last_change":"2024-09-10T09:52:22.331594+0000","last_active":"2024-09-10T09:53:00.844741+0000","last_peered":"2024-09-10T09:53:00.844741+0000","last_clean":"2024-09-10T09:53:00.844741+0000","last_became_active":"2024-09-10T09:52:22.330585+0000","last_became_peered":"2024-09-10T09:52:22.330585+0000","last_unstale":"2024-09-10T09:53:00.844741+0000","last_undegraded":"2024-09-10T09:53:00.844741+0000","last_fullsized":"2024-09-10T09:53:00.844741+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:45:56.412313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.795657+0000","last_change":"2024-09-10T09:52:22.325753+0000","last_active":"2024-09-10T09:53:00.795657+0000","last_peered":"2024-09-10T09:53:00.795657+0000","last_clean":"2024-09-10T09:53:00.795657+0000","last_became_active":"2024-09-10T09:52:22.325385+0000","last_became_peered":"2024-09-10T09:52:22.325385+0000","last_unstale":"2024-09-10T09:53:00.795657+0000","last_undegraded":"2024-09-10T09:53:00.795657+0000","last_fullsized":"2024-09-10T09:53:00.795657+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:18:28.776789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.795435+0000","last_change":"2024-09-10T09:52:22.319069+0000","last_active":"2024-09-10T09:53:00.795435+0000","last_peered":"2024-09-10T09:53:00.795435+0000","last_clean":"2024-09-10T09:53:00.795435+0000","last_became_active":"2024-09-10T09:52:22.318667+0000","last_became_peered":"2024-09-10T09:52:22.318667+0000","last_unstale":"2024-09-10T09:53:00.795435+0000","last_undegraded":"2024-09-10T09:53:00.795435+0000","last_fullsized":"2024-09-10T09:53:00.795435+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:21:32.028235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.792031+0000","last_change":"2024-09-10T09:52:22.312890+0000","last_active":"2024-09-10T09:53:00.792031+0000","last_peered":"2024-09-10T09:53:00.792031+0000","last_clean":"2024-09-10T09:53:00.792031+0000","last_became_active":"2024-09-10T09:52:22.312608+0000","last_became_peered":"2024-09-10T09:52:22.312608+0000","last_unstale":"2024-09-10T09:53:00.792031+0000","last_undegraded":"2024-09-10T09:53:00.792031+0000","last_fullsized":"2024-09-10T09:53:00.792031+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:07:46.046915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.792630+0000","last_change":"2024-09-10T09:52:22.327379+0000","last_active":"2024-09-10T09:53:00.792630+0000","last_peered":"2024-09-10T09:53:00.792630+0000","last_clean":"2024-09-10T09:53:00.792630+0000","last_became_active":"2024-09-10T09:52:22.326921+0000","last_became_peered":"2024-09-10T09:52:22.326921+0000","last_unstale":"2024-09-10T09:53:00.792630+0000","last_undegraded":"2024-09-10T09:53:00.792630+0000","last_fullsized":"2024-09-10T09:53:00.792630+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:01:53.732805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.794884+0000","last_change":"2024-09-10T09:52:22.321731+0000","last_active":"2024-09-10T09:53:00.794884+0000","last_peered":"2024-09-10T09:53:00.794884+0000","last_clean":"2024-09-10T09:53:00.794884+0000","last_became_active":"2024-09-10T09:52:22.321408+0000","last_became_peered":"2024-09-10T09:52:22.321408+0000","last_unstale":"2024-09-10T09:53:00.794884+0000","last_undegraded":"2024-09-10T09:53:00.794884+0000","last_fullsized":"2024-09-10T09:53:00.794884+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:19:57.800678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.791571+0000","last_change":"2024-09-10T09:52:22.313538+0000","last_active":"2024-09-10T09:53:00.791571+0000","last_peered":"2024-09-10T09:53:00.791571+0000","last_clean":"2024-09-10T09:53:00.791571+0000","last_became_active":"2024-09-10T09:52:22.313196+0000","last_became_peered":"2024-09-10T09:52:22.313196+0000","last_unstale":"2024-09-10T09:53:00.791571+0000","last_undegraded":"2024-09-10T09:53:00.791571+0000","last_fullsized":"2024-09-10T09:53:00.791571+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:53:23.387167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.794521+0000","last_change":"2024-09-10T09:52:22.316518+0000","last_active":"2024-09-10T09:53:00.794521+0000","last_peered":"2024-09-10T09:53:00.794521+0000","last_clean":"2024-09-10T09:53:00.794521+0000","last_became_active":"2024-09-10T09:52:22.316193+0000","last_became_peered":"2024-09-10T09:52:22.316193+0000","last_unstale":"2024-09-10T09:53:00.794521+0000","last_undegraded":"2024-09-10T09:53:00.794521+0000","last_fullsized":"2024-09-10T09:53:00.794521+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:49:55.903928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.796514+0000","last_change":"2024-09-10T09:52:22.329163+0000","last_active":"2024-09-10T09:53:00.796514+0000","last_peered":"2024-09-10T09:53:00.796514+0000","last_clean":"2024-09-10T09:53:00.796514+0000","last_became_active":"2024-09-10T09:52:22.328453+0000","last_became_peered":"2024-09-10T09:52:22.328453+0000","last_unstale":"2024-09-10T09:53:00.796514+0000","last_undegraded":"2024-09-10T09:53:00.796514+0000","last_fullsized":"2024-09-10T09:53:00.796514+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:53:33.286055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.794654+0000","last_change":"2024-09-10T09:52:22.314302+0000","last_active":"2024-09-10T09:53:00.794654+0000","last_peered":"2024-09-10T09:53:00.794654+0000","last_clean":"2024-09-10T09:53:00.794654+0000","last_became_active":"2024-09-10T09:52:22.313992+0000","last_became_peered":"2024-09-10T09:52:22.313992+0000","last_unstale":"2024-09-10T09:53:00.794654+0000","last_undegraded":"2024-09-10T09:53:00.794654+0000","last_fullsized":"2024-09-10T09:53:00.794654+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:07:35.764015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.791948+0000","last_change":"2024-09-10T09:52:22.316400+0000","last_active":"2024-09-10T09:53:00.791948+0000","last_peered":"2024-09-10T09:53:00.791948+0000","last_clean":"2024-09-10T09:53:00.791948+0000","last_became_active":"2024-09-10T09:52:22.316110+0000","last_became_peered":"2024-09-10T09:52:22.316110+0000","last_unstale":"2024-09-10T09:53:00.791948+0000","last_undegraded":"2024-09-10T09:53:00.791948+0000","last_fullsized":"2024-09-10T09:53:00.791948+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:17:26.528899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.794833+0000","last_change":"2024-09-10T09:52:22.317914+0000","last_active":"2024-09-10T09:53:00.794833+0000","last_peered":"2024-09-10T09:53:00.794833+0000","last_clean":"2024-09-10T09:53:00.794833+0000","last_became_active":"2024-09-10T09:52:22.317575+0000","last_became_peered":"2024-09-10T09:52:22.317575+0000","last_unstale":"2024-09-10T09:53:00.794833+0000","last_undegraded":"2024-09-10T09:53:00.794833+0000","last_fullsized":"2024-09-10T09:53:00.794833+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:17:51.446233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.795937+0000","last_change":"2024-09-10T09:52:22.327749+0000","last_active":"2024-09-10T09:53:00.795937+0000","last_peered":"2024-09-10T09:53:00.795937+0000","last_clean":"2024-09-10T09:53:00.795937+0000","last_became_active":"2024-09-10T09:52:22.327461+0000","last_became_peered":"2024-09-10T09:52:22.327461+0000","last_unstale":"2024-09-10T09:53:00.795937+0000","last_undegraded":"2024-09-10T09:53:00.795937+0000","last_fullsized":"2024-09-10T09:53:00.795937+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:54.208064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.794053+0000","last_change":"2024-09-10T09:52:22.326982+0000","last_active":"2024-09-10T09:53:00.794053+0000","last_peered":"2024-09-10T09:53:00.794053+0000","last_clean":"2024-09-10T09:53:00.794053+0000","last_became_active":"2024-09-10T09:52:22.326635+0000","last_became_peered":"2024-09-10T09:52:22.326635+0000","last_unstale":"2024-09-10T09:53:00.794053+0000","last_undegraded":"2024-09-10T09:53:00.794053+0000","last_fullsized":"2024-09-10T09:53:00.794053+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:50:54.835899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.844561+0000","last_change":"2024-09-10T09:52:22.323410+0000","last_active":"2024-09-10T09:53:00.844561+0000","last_peered":"2024-09-10T09:53:00.844561+0000","last_clean":"2024-09-10T09:53:00.844561+0000","last_became_active":"2024-09-10T09:52:22.323116+0000","last_became_peered":"2024-09-10T09:52:22.323116+0000","last_unstale":"2024-09-10T09:53:00.844561+0000","last_undegraded":"2024-09-10T09:53:00.844561+0000","last_fullsized":"2024-09-10T09:53:00.844561+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:09:15.115162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.844569+0000","last_change":"2024-09-10T09:52:22.324908+0000","last_active":"2024-09-10T09:53:00.844569+0000","last_peered":"2024-09-10T09:53:00.844569+0000","last_clean":"2024-09-10T09:53:00.844569+0000","last_became_active":"2024-09-10T09:52:22.324449+0000","last_became_peered":"2024-09-10T09:52:22.324449+0000","last_unstale":"2024-09-10T09:53:00.844569+0000","last_undegraded":"2024-09-10T09:53:00.844569+0000","last_fullsized":"2024-09-10T09:53:00.844569+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:43:24.535721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.796442+0000","last_change":"2024-09-10T09:52:22.322157+0000","last_active":"2024-09-10T09:53:00.796442+0000","last_peered":"2024-09-10T09:53:00.796442+0000","last_clean":"2024-09-10T09:53:00.796442+0000","last_became_active":"2024-09-10T09:52:22.321856+0000","last_became_peered":"2024-09-10T09:52:22.321856+0000","last_unstale":"2024-09-10T09:53:00.796442+0000","last_undegraded":"2024-09-10T09:53:00.796442+0000","last_fullsized":"2024-09-10T09:53:00.796442+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:36:38.395370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.801027+0000","last_change":"2024-09-10T09:53:00.800337+0000","last_active":"2024-09-10T09:53:00.801027+0000","last_peered":"2024-09-10T09:53:00.801027+0000","last_clean":"2024-09-10T09:53:00.801027+0000","last_became_active":"2024-09-10T09:53:00.799863+0000","last_became_peered":"2024-09-10T09:53:00.799863+0000","last_unstale":"2024-09-10T09:53:00.801027+0000","last_undegraded":"2024-09-10T09:53:00.801027+0000","last_fullsized":"2024-09-10T09:53:00.801027+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:18.017672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.797150+0000","last_change":"2024-09-10T09:52:22.310735+0000","last_active":"2024-09-10T09:53:00.797150+0000","last_peered":"2024-09-10T09:53:00.797150+0000","last_clean":"2024-09-10T09:53:00.797150+0000","last_became_active":"2024-09-10T09:52:22.310376+0000","last_became_peered":"2024-09-10T09:52:22.310376+0000","last_unstale":"2024-09-10T09:53:00.797150+0000","last_undegraded":"2024-09-10T09:53:00.797150+0000","last_fullsized":"2024-09-10T09:53:00.797150+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:38:08.637588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.792545+0000","last_change":"2024-09-10T09:52:22.321113+0000","last_active":"2024-09-10T09:53:00.792545+0000","last_peered":"2024-09-10T09:53:00.792545+0000","last_clean":"2024-09-10T09:53:00.792545+0000","last_became_active":"2024-09-10T09:52:22.320767+0000","last_became_peered":"2024-09-10T09:52:22.320767+0000","last_unstale":"2024-09-10T09:53:00.792545+0000","last_undegraded":"2024-09-10T09:53:00.792545+0000","last_fullsized":"2024-09-10T09:53:00.792545+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:37.549322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.794810+0000","last_change":"2024-09-10T09:52:22.318634+0000","last_active":"2024-09-10T09:53:00.794810+0000","last_peered":"2024-09-10T09:53:00.794810+0000","last_clean":"2024-09-10T09:53:00.794810+0000","last_became_active":"2024-09-10T09:52:22.318294+0000","last_became_peered":"2024-09-10T09:52:22.318294+0000","last_unstale":"2024-09-10T09:53:00.794810+0000","last_undegraded":"2024-09-10T09:53:00.794810+0000","last_fullsized":"2024-09-10T09:53:00.794810+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:19:31.262055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.791534+0000","last_change":"2024-09-10T09:52:22.324567+0000","last_active":"2024-09-10T09:53:00.791534+0000","last_peered":"2024-09-10T09:53:00.791534+0000","last_clean":"2024-09-10T09:53:00.791534+0000","last_became_active":"2024-09-10T09:52:22.321803+0000","last_became_peered":"2024-09-10T09:52:22.321803+0000","last_unstale":"2024-09-10T09:53:00.791534+0000","last_undegraded":"2024-09-10T09:53:00.791534+0000","last_fullsized":"2024-09-10T09:53:00.791534+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:08:11.099428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.796590+0000","last_change":"2024-09-10T09:52:22.318163+0000","last_active":"2024-09-10T09:53:00.796590+0000","last_peered":"2024-09-10T09:53:00.796590+0000","last_clean":"2024-09-10T09:53:00.796590+0000","last_became_active":"2024-09-10T09:52:22.317808+0000","last_became_peered":"2024-09-10T09:52:22.317808+0000","last_unstale":"2024-09-10T09:53:00.796590+0000","last_undegraded":"2024-09-10T09:53:00.796590+0000","last_fullsized":"2024-09-10T09:53:00.796590+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:19.214291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.791566+0000","last_change":"2024-09-10T09:52:22.323035+0000","last_active":"2024-09-10T09:53:00.791566+0000","last_peered":"2024-09-10T09:53:00.791566+0000","last_clean":"2024-09-10T09:53:00.791566+0000","last_became_active":"2024-09-10T09:52:22.322679+0000","last_became_peered":"2024-09-10T09:52:22.322679+0000","last_unstale":"2024-09-10T09:53:00.791566+0000","last_undegraded":"2024-09-10T09:53:00.791566+0000","last_fullsized":"2024-09-10T09:53:00.791566+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:23:13.762612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.797958+0000","last_change":"2024-09-10T09:52:22.315887+0000","last_active":"2024-09-10T09:53:00.797958+0000","last_peered":"2024-09-10T09:53:00.797958+0000","last_clean":"2024-09-10T09:53:00.797958+0000","last_became_active":"2024-09-10T09:52:22.315569+0000","last_became_peered":"2024-09-10T09:52:22.315569+0000","last_unstale":"2024-09-10T09:53:00.797958+0000","last_undegraded":"2024-09-10T09:53:00.797958+0000","last_fullsized":"2024-09-10T09:53:00.797958+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:46:57.094320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:01.217981+0000","last_change":"2024-09-10T09:52:22.314060+0000","last_active":"2024-09-10T09:53:01.217981+0000","last_peered":"2024-09-10T09:53:01.217981+0000","last_clean":"2024-09-10T09:53:01.217981+0000","last_became_active":"2024-09-10T09:52:22.313572+0000","last_became_peered":"2024-09-10T09:52:22.313572+0000","last_unstale":"2024-09-10T09:53:01.217981+0000","last_undegraded":"2024-09-10T09:53:01.217981+0000","last_fullsized":"2024-09-10T09:53:01.217981+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:11:17.158625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.844889+0000","last_change":"2024-09-10T09:52:22.327233+0000","last_active":"2024-09-10T09:53:00.844889+0000","last_peered":"2024-09-10T09:53:00.844889+0000","last_clean":"2024-09-10T09:53:00.844889+0000","last_became_active":"2024-09-10T09:52:22.326945+0000","last_became_peered":"2024-09-10T09:52:22.326945+0000","last_unstale":"2024-09-10T09:53:00.844889+0000","last_undegraded":"2024-09-10T09:53:00.844889+0000","last_fullsized":"2024-09-10T09:53:00.844889+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:31:13.490147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.844880+0000","last_change":"2024-09-10T09:52:22.330912+0000","last_active":"2024-09-10T09:53:00.844880+0000","last_peered":"2024-09-10T09:53:00.844880+0000","last_clean":"2024-09-10T09:53:00.844880+0000","last_became_active":"2024-09-10T09:52:22.330149+0000","last_became_peered":"2024-09-10T09:52:22.330149+0000","last_unstale":"2024-09-10T09:53:00.844880+0000","last_undegraded":"2024-09-10T09:53:00.844880+0000","last_fullsized":"2024-09-10T09:53:00.844880+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:10:48.842499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.795892+0000","last_change":"2024-09-10T09:52:22.327035+0000","last_active":"2024-09-10T09:53:00.795892+0000","last_peered":"2024-09-10T09:53:00.795892+0000","last_clean":"2024-09-10T09:53:00.795892+0000","last_became_active":"2024-09-10T09:52:22.326741+0000","last_became_peered":"2024-09-10T09:52:22.326741+0000","last_unstale":"2024-09-10T09:53:00.795892+0000","last_undegraded":"2024-09-10T09:53:00.795892+0000","last_fullsized":"2024-09-10T09:53:00.795892+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:21:05.026614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":56306,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":155,"num_read_kb":142,"num_write":235,"num_write_kb":123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":4426,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":516096,"data_stored":168918,"data_compressed":5661,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":204,"ondisk_log_size":204,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282980,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449115,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1048576,"data_stored":535507,"data_compressed":26618,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":70,"seq":300647710750,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32332,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712820,"statfs":{"total":95995035648,"available":95961927680,"internally_reserved":0,"allocated":1175552,"data_stored":1171226,"data_compressed":64497,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":65,"seq":279172874278,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32232,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712920,"statfs":{"total":95995035648,"available":95962030080,"internally_reserved":0,"allocated":1073152,"data_stored":538945,"data_compressed":26618,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135915,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32220,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712932,"statfs":{"total":95995035648,"available":95962042368,"internally_reserved":0,"allocated":1060864,"data_stored":538651,"data_compressed":26618,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364848,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28112,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717040,"statfs":{"total":95995035648,"available":95966248960,"internally_reserved":0,"allocated":1114112,"data_stored":1131531,"data_compressed":62610,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528374,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1044480,"data_stored":530353,"data_compressed":26618,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790012,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28120,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717032,"statfs":{"total":95995035648,"available":95966240768,"internally_reserved":0,"allocated":1122304,"data_stored":1123212,"data_compressed":62610,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953535,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1105920,"data_stored":583918,"data_compressed":28505,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182468,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32216,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712936,"statfs":{"total":95995035648,"available":95962046464,"internally_reserved":0,"allocated":1056768,"data_stored":530061,"data_compressed":26618,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411403,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36380,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708772,"statfs":{"total":95995035648,"available":95957782528,"internally_reserved":0,"allocated":1060864,"data_stored":531996,"data_compressed":26618,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574929,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1101824,"data_stored":581153,"data_compressed":28505,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738454,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1069056,"data_stored":531809,"data_compressed":26618,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34340,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34340,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34340,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35992,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35992,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35992,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35992,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35992,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":51522,"data_compressed":1887,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":50191,"data_compressed":1887,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":51227,"data_compressed":1887,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T09:53:34.173 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T09:53:34.544 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:53:34.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:34 smithi077 ceph-mon[30319]: from='client.14721 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T09:53:34.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:34 smithi077 ceph-mon[30319]: pgmap v341: 129 pgs: 129 active+clean; 639 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.5 KiB/s wr, 14 op/s 2024-09-10T09:53:35.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:35.122+0000 7f1344200640 1 -- 172.21.15.77:0/4170899036 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f133c106f10 msgr2=0x7f133c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:35.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:35.122+0000 7f1344200640 1 --2- 172.21.15.77:0/4170899036 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f133c106f10 0x7f133c10ea90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1334009ca0 tx=0x7f133402a8c0 comp rx=0 tx=0).stop 2024-09-10T09:53:35.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:35.122+0000 7f1344200640 1 -- 172.21.15.77:0/4170899036 shutdown_connections 2024-09-10T09:53:35.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:35.122+0000 7f1344200640 1 --2- 172.21.15.77:0/4170899036 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f133c106f10 0x7f133c10ea90 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:35.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:35.122+0000 7f1344200640 1 --2- 172.21.15.77:0/4170899036 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f133c102790 0x7f133c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:35.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:35.122+0000 7f1344200640 1 --2- 172.21.15.77:0/4170899036 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f133c101de0 0x7f133c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:35.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:35.122+0000 7f1344200640 1 -- 172.21.15.77:0/4170899036 >> 172.21.15.77:0/4170899036 conn(0x7f133c0fd8c0 msgr2=0x7f133c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:35.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:35.122+0000 7f1344200640 1 -- 172.21.15.77:0/4170899036 shutdown_connections 2024-09-10T09:53:35.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:35.122+0000 7f1344200640 1 -- 172.21.15.77:0/4170899036 wait complete. 2024-09-10T09:53:35.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:35.123+0000 7f1344200640 1 Processor -- start 2024-09-10T09:53:35.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:35.123+0000 7f1344200640 1 -- start start 2024-09-10T09:53:35.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:35.123+0000 7f1344200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f133c101de0 0x7f133c19a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:35.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:35.123+0000 7f1344200640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f133c102790 0x7f133c19ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:35.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:35.124+0000 7f1344200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f133c106f10 0x7f133c19eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:35.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:35.124+0000 7f1344200640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f133c111fc0 con 0x7f133c101de0 2024-09-10T09:53:35.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:35.124+0000 7f1344200640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f133c111e40 con 0x7f133c102790 2024-09-10T09:53:35.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:35.124+0000 7f1344200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f133c112140 con 0x7f133c106f10 2024-09-10T09:53:35.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:35.124+0000 7f1340e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f133c102790 0x7f133c19ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:35.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:35.124+0000 7f1341800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f133c101de0 0x7f133c19a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:35.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:35.124+0000 7f1340e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f133c102790 0x7f133c19ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:58392/0 (socket says 172.21.15.77:58392) 2024-09-10T09:53:35.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:35.124+0000 7f1340e00640 1 -- 172.21.15.77:0/1825464911 learned_addr learned my addr 172.21.15.77:0/1825464911 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:53:35.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:35.124+0000 7f1342200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f133c106f10 0x7f133c19eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:35.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:35.124+0000 7f1340e00640 1 -- 172.21.15.77:0/1825464911 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f133c106f10 msgr2=0x7f133c19eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:35.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:35.124+0000 7f1340e00640 1 --2- 172.21.15.77:0/1825464911 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f133c106f10 0x7f133c19eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:35.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:35.124+0000 7f1340e00640 1 -- 172.21.15.77:0/1825464911 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f133c101de0 msgr2=0x7f133c19a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:35.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:35.124+0000 7f1340e00640 1 --2- 172.21.15.77:0/1825464911 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f133c101de0 0x7f133c19a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:35.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:35.124+0000 7f1340e00640 1 -- 172.21.15.77:0/1825464911 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f133c19f5a0 con 0x7f133c102790 2024-09-10T09:53:35.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:35.124+0000 7f1341800640 1 --2- 172.21.15.77:0/1825464911 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f133c101de0 0x7f133c19a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:53:35.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:35.125+0000 7f1340e00640 1 --2- 172.21.15.77:0/1825464911 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f133c102790 0x7f133c19ab30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f132800ea60 tx=0x7f132800ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:35.128 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:35.125+0000 7f133a000640 1 -- 172.21.15.77:0/1825464911 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f132800cde0 con 0x7f133c102790 2024-09-10T09:53:35.128 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:35.125+0000 7f1344200640 1 -- 172.21.15.77:0/1825464911 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f133c19f890 con 0x7f133c102790 2024-09-10T09:53:35.128 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:35.125+0000 7f133a000640 1 -- 172.21.15.77:0/1825464911 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13280035f0 con 0x7f133c102790 2024-09-10T09:53:35.128 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:35.125+0000 7f133a000640 1 -- 172.21.15.77:0/1825464911 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1328010670 con 0x7f133c102790 2024-09-10T09:53:35.128 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:35.125+0000 7f1344200640 1 -- 172.21.15.77:0/1825464911 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f133c1a7170 con 0x7f133c102790 2024-09-10T09:53:35.128 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:35.126+0000 7f1344200640 1 -- 172.21.15.77:0/1825464911 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f133c1021c0 con 0x7f133c102790 2024-09-10T09:53:35.130 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:35.128+0000 7f133a000640 1 -- 172.21.15.77:0/1825464911 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f13280037a0 con 0x7f133c102790 2024-09-10T09:53:35.130 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:34 smithi134 ceph-mon[35674]: from='client.14721 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T09:53:35.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:34 smithi134 ceph-mon[35674]: pgmap v341: 129 pgs: 129 active+clean; 639 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.5 KiB/s wr, 14 op/s 2024-09-10T09:53:35.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:35.130+0000 7f133a000640 1 --2- 172.21.15.77:0/1825464911 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f13240a2c90 0x7f13240a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:35.132 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:35.130+0000 7f133a000640 1 -- 172.21.15.77:0/1825464911 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f13280cf760 con 0x7f133c102790 2024-09-10T09:53:35.132 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:35.130+0000 7f1341800640 1 --2- 172.21.15.77:0/1825464911 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f13240a2c90 0x7f13240a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:35.132 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:35.131+0000 7f1341800640 1 --2- 172.21.15.77:0/1825464911 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f13240a2c90 0x7f13240a5150 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1330009fc0 tx=0x7f1330005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:35.135 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:35.134+0000 7f133a000640 1 -- 172.21.15.77:0/1825464911 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f132809c200 con 0x7f133c102790 2024-09-10T09:53:35.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:34 smithi120 ceph-mon[34770]: from='client.14721 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T09:53:35.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:34 smithi120 ceph-mon[34770]: pgmap v341: 129 pgs: 129 active+clean; 639 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.5 KiB/s wr, 14 op/s 2024-09-10T09:53:35.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:35.306+0000 7f1344200640 1 -- 172.21.15.77:0/1825464911 --> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f133c075430 con 0x7f13240a2c90 2024-09-10T09:53:35.316 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:35.315+0000 7f133a000640 1 -- 172.21.15.77:0/1825464911 <== mgr.14150 v2:172.21.15.77:6800/4261324650 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340728 (secure 0 0 0) 0x7f133c075430 con 0x7f13240a2c90 2024-09-10T09:53:35.321 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:53:35.322 INFO:teuthology.orchestra.run.smithi077.stderr:dumped all 2024-09-10T09:53:35.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:35.319+0000 7f1344200640 1 -- 172.21.15.77:0/1825464911 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f13240a2c90 msgr2=0x7f13240a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:35.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:35.319+0000 7f1344200640 1 --2- 172.21.15.77:0/1825464911 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f13240a2c90 0x7f13240a5150 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1330009fc0 tx=0x7f1330005f90 comp rx=0 tx=0).stop 2024-09-10T09:53:35.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:35.320+0000 7f1344200640 1 -- 172.21.15.77:0/1825464911 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f133c102790 msgr2=0x7f133c19ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:35.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:35.320+0000 7f1344200640 1 --2- 172.21.15.77:0/1825464911 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f133c102790 0x7f133c19ab30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f132800ea60 tx=0x7f132800ef20 comp rx=0 tx=0).stop 2024-09-10T09:53:35.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:35.320+0000 7f1344200640 1 -- 172.21.15.77:0/1825464911 shutdown_connections 2024-09-10T09:53:35.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:35.320+0000 7f1344200640 1 --2- 172.21.15.77:0/1825464911 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f13240a2c90 0x7f13240a5150 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:35.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:35.320+0000 7f1344200640 1 --2- 172.21.15.77:0/1825464911 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f133c106f10 0x7f133c19eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:35.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:35.320+0000 7f1344200640 1 --2- 172.21.15.77:0/1825464911 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f133c102790 0x7f133c19ab30 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:35.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:35.320+0000 7f1344200640 1 --2- 172.21.15.77:0/1825464911 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f133c101de0 0x7f133c19a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:35.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:35.320+0000 7f1344200640 1 -- 172.21.15.77:0/1825464911 >> 172.21.15.77:0/1825464911 conn(0x7f133c0fd8c0 msgr2=0x7f133c0fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:35.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:35.320+0000 7f1344200640 1 -- 172.21.15.77:0/1825464911 shutdown_connections 2024-09-10T09:53:35.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:35.320+0000 7f1344200640 1 -- 172.21.15.77:0/1825464911 wait complete. 2024-09-10T09:53:36.207 INFO:teuthology.orchestra.run.smithi077.stdout:{"pg_ready":true,"pg_map":{"version":341,"stamp":"2024-09-10T09:53:33.689234+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654559,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":312,"num_read_kb":274,"num_write":421,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1185162,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":346,"ondisk_log_size":346,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":366144,"kb_used_data":12736,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124575680,"statfs":{"total":1151940427776,"available":1151565496320,"internally_reserved":0,"allocated":13041664,"data_stored":8336013,"data_compressed":433053,"data_compressed_allocated":589824,"data_compressed_original":2678784,"omap_allocated":19143,"internal_metadata":361674041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":53925,"num_objects":3,"num_object_clones":0,"num_object_copies":11,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":144,"num_read_kb":128,"num_write":30,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003645"},"pg_stats":[{"pgid":"2.1f","version":"94'13","reported_seq":77,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:29.905424+0000","last_change":"2024-09-10T09:53:00.803590+0000","last_active":"2024-09-10T09:53:29.905424+0000","last_peered":"2024-09-10T09:53:29.905424+0000","last_clean":"2024-09-10T09:53:29.905424+0000","last_became_active":"2024-09-10T09:53:00.798189+0000","last_became_peered":"2024-09-10T09:53:00.798189+0000","last_unstale":"2024-09-10T09:53:29.905424+0000","last_undegraded":"2024-09-10T09:53:29.905424+0000","last_fullsized":"2024-09-10T09:53:29.905424+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:18:48.808836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.792133+0000","last_change":"2024-09-10T09:52:22.318195+0000","last_active":"2024-09-10T09:53:00.792133+0000","last_peered":"2024-09-10T09:53:00.792133+0000","last_clean":"2024-09-10T09:53:00.792133+0000","last_became_active":"2024-09-10T09:52:22.315905+0000","last_became_peered":"2024-09-10T09:52:22.315905+0000","last_unstale":"2024-09-10T09:53:00.792133+0000","last_undegraded":"2024-09-10T09:53:00.792133+0000","last_fullsized":"2024-09-10T09:53:00.792133+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:53:55.515984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.791326+0000","last_change":"2024-09-10T09:52:31.369780+0000","last_active":"2024-09-10T09:53:00.791326+0000","last_peered":"2024-09-10T09:53:00.791326+0000","last_clean":"2024-09-10T09:53:00.791326+0000","last_became_active":"2024-09-10T09:52:31.369238+0000","last_became_peered":"2024-09-10T09:52:31.369238+0000","last_unstale":"2024-09-10T09:53:00.791326+0000","last_undegraded":"2024-09-10T09:53:00.791326+0000","last_fullsized":"2024-09-10T09:53:00.791326+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:39:05.318493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"94'2","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.792494+0000","last_change":"2024-09-10T09:52:19.912276+0000","last_active":"2024-09-10T09:53:00.792494+0000","last_peered":"2024-09-10T09:53:00.792494+0000","last_clean":"2024-09-10T09:53:00.792494+0000","last_became_active":"2024-09-10T09:52:19.911452+0000","last_became_peered":"2024-09-10T09:52:19.911452+0000","last_unstale":"2024-09-10T09:53:00.792494+0000","last_undegraded":"2024-09-10T09:53:00.792494+0000","last_fullsized":"2024-09-10T09:53:00.792494+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:24:31.262207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.845076+0000","last_change":"2024-09-10T09:52:22.327641+0000","last_active":"2024-09-10T09:53:00.845076+0000","last_peered":"2024-09-10T09:53:00.845076+0000","last_clean":"2024-09-10T09:53:00.845076+0000","last_became_active":"2024-09-10T09:52:22.326969+0000","last_became_peered":"2024-09-10T09:52:22.326969+0000","last_unstale":"2024-09-10T09:53:00.845076+0000","last_undegraded":"2024-09-10T09:53:00.845076+0000","last_fullsized":"2024-09-10T09:53:00.845076+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:35:27.825534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:01.217030+0000","last_change":"2024-09-10T09:52:31.372360+0000","last_active":"2024-09-10T09:53:01.217030+0000","last_peered":"2024-09-10T09:53:01.217030+0000","last_clean":"2024-09-10T09:53:01.217030+0000","last_became_active":"2024-09-10T09:52:31.371957+0000","last_became_peered":"2024-09-10T09:52:31.371957+0000","last_unstale":"2024-09-10T09:53:01.217030+0000","last_undegraded":"2024-09-10T09:53:01.217030+0000","last_fullsized":"2024-09-10T09:53:01.217030+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:49:34.776185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"94'12","reported_seq":49,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:16.883154+0000","last_change":"2024-09-10T09:52:19.843532+0000","last_active":"2024-09-10T09:53:16.883154+0000","last_peered":"2024-09-10T09:53:16.883154+0000","last_clean":"2024-09-10T09:53:16.883154+0000","last_became_active":"2024-09-10T09:52:19.843070+0000","last_became_peered":"2024-09-10T09:52:19.843070+0000","last_unstale":"2024-09-10T09:53:16.883154+0000","last_undegraded":"2024-09-10T09:53:16.883154+0000","last_fullsized":"2024-09-10T09:53:16.883154+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:08:02.223373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.794657+0000","last_change":"2024-09-10T09:52:22.316981+0000","last_active":"2024-09-10T09:53:00.794657+0000","last_peered":"2024-09-10T09:53:00.794657+0000","last_clean":"2024-09-10T09:53:00.794657+0000","last_became_active":"2024-09-10T09:52:22.316648+0000","last_became_peered":"2024-09-10T09:52:22.316648+0000","last_unstale":"2024-09-10T09:53:00.794657+0000","last_undegraded":"2024-09-10T09:53:00.794657+0000","last_fullsized":"2024-09-10T09:53:00.794657+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:28:56.742496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:01.217444+0000","last_change":"2024-09-10T09:52:31.376811+0000","last_active":"2024-09-10T09:53:01.217444+0000","last_peered":"2024-09-10T09:53:01.217444+0000","last_clean":"2024-09-10T09:53:01.217444+0000","last_became_active":"2024-09-10T09:52:31.376007+0000","last_became_peered":"2024-09-10T09:52:31.376007+0000","last_unstale":"2024-09-10T09:53:01.217444+0000","last_undegraded":"2024-09-10T09:53:01.217444+0000","last_fullsized":"2024-09-10T09:53:01.217444+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:29:26.213281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"94'5","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:15.875175+0000","last_change":"2024-09-10T09:52:19.837848+0000","last_active":"2024-09-10T09:53:15.875175+0000","last_peered":"2024-09-10T09:53:15.875175+0000","last_clean":"2024-09-10T09:53:15.875175+0000","last_became_active":"2024-09-10T09:52:19.837383+0000","last_became_peered":"2024-09-10T09:52:19.837383+0000","last_unstale":"2024-09-10T09:53:15.875175+0000","last_undegraded":"2024-09-10T09:53:15.875175+0000","last_fullsized":"2024-09-10T09:53:15.875175+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:50:42.813665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.796806+0000","last_change":"2024-09-10T09:52:22.318517+0000","last_active":"2024-09-10T09:53:00.796806+0000","last_peered":"2024-09-10T09:53:00.796806+0000","last_clean":"2024-09-10T09:53:00.796806+0000","last_became_active":"2024-09-10T09:52:22.318207+0000","last_became_peered":"2024-09-10T09:52:22.318207+0000","last_unstale":"2024-09-10T09:53:00.796806+0000","last_undegraded":"2024-09-10T09:53:00.796806+0000","last_fullsized":"2024-09-10T09:53:00.796806+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:03:42.840690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:01.217522+0000","last_change":"2024-09-10T09:52:31.370910+0000","last_active":"2024-09-10T09:53:01.217522+0000","last_peered":"2024-09-10T09:53:01.217522+0000","last_clean":"2024-09-10T09:53:01.217522+0000","last_became_active":"2024-09-10T09:52:31.370467+0000","last_became_peered":"2024-09-10T09:52:31.370467+0000","last_unstale":"2024-09-10T09:53:01.217522+0000","last_undegraded":"2024-09-10T09:53:01.217522+0000","last_fullsized":"2024-09-10T09:53:01.217522+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:49:25.037820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.797176+0000","last_change":"2024-09-10T09:52:22.310708+0000","last_active":"2024-09-10T09:53:00.797176+0000","last_peered":"2024-09-10T09:53:00.797176+0000","last_clean":"2024-09-10T09:53:00.797176+0000","last_became_active":"2024-09-10T09:52:22.310387+0000","last_became_peered":"2024-09-10T09:52:22.310387+0000","last_unstale":"2024-09-10T09:53:00.797176+0000","last_undegraded":"2024-09-10T09:53:00.797176+0000","last_fullsized":"2024-09-10T09:53:00.797176+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:35:03.115408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"94'4","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:12.848949+0000","last_change":"2024-09-10T09:52:19.841417+0000","last_active":"2024-09-10T09:53:12.848949+0000","last_peered":"2024-09-10T09:53:12.848949+0000","last_clean":"2024-09-10T09:53:12.848949+0000","last_became_active":"2024-09-10T09:52:19.841083+0000","last_became_peered":"2024-09-10T09:52:19.841083+0000","last_unstale":"2024-09-10T09:53:12.848949+0000","last_undegraded":"2024-09-10T09:53:12.848949+0000","last_fullsized":"2024-09-10T09:53:12.848949+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:01:19.008345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:01.217723+0000","last_change":"2024-09-10T09:52:31.367406+0000","last_active":"2024-09-10T09:53:01.217723+0000","last_peered":"2024-09-10T09:53:01.217723+0000","last_clean":"2024-09-10T09:53:01.217723+0000","last_became_active":"2024-09-10T09:52:31.366961+0000","last_became_peered":"2024-09-10T09:52:31.366961+0000","last_unstale":"2024-09-10T09:53:01.217723+0000","last_undegraded":"2024-09-10T09:53:01.217723+0000","last_fullsized":"2024-09-10T09:53:01.217723+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:04:51.714465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.791633+0000","last_change":"2024-09-10T09:52:22.329843+0000","last_active":"2024-09-10T09:53:00.791633+0000","last_peered":"2024-09-10T09:53:00.791633+0000","last_clean":"2024-09-10T09:53:00.791633+0000","last_became_active":"2024-09-10T09:52:22.328933+0000","last_became_peered":"2024-09-10T09:52:22.328933+0000","last_unstale":"2024-09-10T09:53:00.791633+0000","last_undegraded":"2024-09-10T09:53:00.791633+0000","last_fullsized":"2024-09-10T09:53:00.791633+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:18:03.409246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"94'3","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:14.865092+0000","last_change":"2024-09-10T09:53:00.800421+0000","last_active":"2024-09-10T09:53:14.865092+0000","last_peered":"2024-09-10T09:53:14.865092+0000","last_clean":"2024-09-10T09:53:14.865092+0000","last_became_active":"2024-09-10T09:53:00.800026+0000","last_became_peered":"2024-09-10T09:53:00.800026+0000","last_unstale":"2024-09-10T09:53:14.865092+0000","last_undegraded":"2024-09-10T09:53:14.865092+0000","last_fullsized":"2024-09-10T09:53:14.865092+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:57:45.252420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"94'1","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:01.217875+0000","last_change":"2024-09-10T09:52:31.376605+0000","last_active":"2024-09-10T09:53:01.217875+0000","last_peered":"2024-09-10T09:53:01.217875+0000","last_clean":"2024-09-10T09:53:01.217875+0000","last_became_active":"2024-09-10T09:52:31.376083+0000","last_became_peered":"2024-09-10T09:52:31.376083+0000","last_unstale":"2024-09-10T09:53:01.217875+0000","last_undegraded":"2024-09-10T09:53:01.217875+0000","last_fullsized":"2024-09-10T09:53:01.217875+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:05:10.030673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.794858+0000","last_change":"2024-09-10T09:52:22.327303+0000","last_active":"2024-09-10T09:53:00.794858+0000","last_peered":"2024-09-10T09:53:00.794858+0000","last_clean":"2024-09-10T09:53:00.794858+0000","last_became_active":"2024-09-10T09:52:22.326753+0000","last_became_peered":"2024-09-10T09:52:22.326753+0000","last_unstale":"2024-09-10T09:53:00.794858+0000","last_undegraded":"2024-09-10T09:53:00.794858+0000","last_fullsized":"2024-09-10T09:53:00.794858+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:27:26.668151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"94'7","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:15.874371+0000","last_change":"2024-09-10T09:52:19.836484+0000","last_active":"2024-09-10T09:53:15.874371+0000","last_peered":"2024-09-10T09:53:15.874371+0000","last_clean":"2024-09-10T09:53:15.874371+0000","last_became_active":"2024-09-10T09:52:19.836130+0000","last_became_peered":"2024-09-10T09:52:19.836130+0000","last_unstale":"2024-09-10T09:53:15.874371+0000","last_undegraded":"2024-09-10T09:53:15.874371+0000","last_fullsized":"2024-09-10T09:53:15.874371+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:31:58.731981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.794698+0000","last_change":"2024-09-10T09:52:31.375279+0000","last_active":"2024-09-10T09:53:00.794698+0000","last_peered":"2024-09-10T09:53:00.794698+0000","last_clean":"2024-09-10T09:53:00.794698+0000","last_became_active":"2024-09-10T09:52:31.374181+0000","last_became_peered":"2024-09-10T09:52:31.374181+0000","last_unstale":"2024-09-10T09:53:00.794698+0000","last_undegraded":"2024-09-10T09:53:00.794698+0000","last_fullsized":"2024-09-10T09:53:00.794698+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:54:51.299224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.797330+0000","last_change":"2024-09-10T09:52:22.314817+0000","last_active":"2024-09-10T09:53:00.797330+0000","last_peered":"2024-09-10T09:53:00.797330+0000","last_clean":"2024-09-10T09:53:00.797330+0000","last_became_active":"2024-09-10T09:52:22.314491+0000","last_became_peered":"2024-09-10T09:52:22.314491+0000","last_unstale":"2024-09-10T09:53:00.797330+0000","last_undegraded":"2024-09-10T09:53:00.797330+0000","last_fullsized":"2024-09-10T09:53:00.797330+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:49:41.941667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"94'2","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:12.848559+0000","last_change":"2024-09-10T09:52:19.911730+0000","last_active":"2024-09-10T09:53:12.848559+0000","last_peered":"2024-09-10T09:53:12.848559+0000","last_clean":"2024-09-10T09:53:12.848559+0000","last_became_active":"2024-09-10T09:52:19.911393+0000","last_became_peered":"2024-09-10T09:52:19.911393+0000","last_unstale":"2024-09-10T09:53:12.848559+0000","last_undegraded":"2024-09-10T09:53:12.848559+0000","last_fullsized":"2024-09-10T09:53:12.848559+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:46:17.943129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:01.217751+0000","last_change":"2024-09-10T09:52:31.376498+0000","last_active":"2024-09-10T09:53:01.217751+0000","last_peered":"2024-09-10T09:53:01.217751+0000","last_clean":"2024-09-10T09:53:01.217751+0000","last_became_active":"2024-09-10T09:52:31.375966+0000","last_became_peered":"2024-09-10T09:52:31.375966+0000","last_unstale":"2024-09-10T09:53:01.217751+0000","last_undegraded":"2024-09-10T09:53:01.217751+0000","last_fullsized":"2024-09-10T09:53:01.217751+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:08:30.649399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.844976+0000","last_change":"2024-09-10T09:52:22.331611+0000","last_active":"2024-09-10T09:53:00.844976+0000","last_peered":"2024-09-10T09:53:00.844976+0000","last_clean":"2024-09-10T09:53:00.844976+0000","last_became_active":"2024-09-10T09:52:22.331268+0000","last_became_peered":"2024-09-10T09:52:22.331268+0000","last_unstale":"2024-09-10T09:53:00.844976+0000","last_undegraded":"2024-09-10T09:53:00.844976+0000","last_fullsized":"2024-09-10T09:53:00.844976+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:01:05.433994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"94'7","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:15.875105+0000","last_change":"2024-09-10T09:52:19.838387+0000","last_active":"2024-09-10T09:53:15.875105+0000","last_peered":"2024-09-10T09:53:15.875105+0000","last_clean":"2024-09-10T09:53:15.875105+0000","last_became_active":"2024-09-10T09:52:19.837889+0000","last_became_peered":"2024-09-10T09:52:19.837889+0000","last_unstale":"2024-09-10T09:53:15.875105+0000","last_undegraded":"2024-09-10T09:53:15.875105+0000","last_fullsized":"2024-09-10T09:53:15.875105+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:50:00.187243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.791474+0000","last_change":"2024-09-10T09:52:31.363262+0000","last_active":"2024-09-10T09:53:00.791474+0000","last_peered":"2024-09-10T09:53:00.791474+0000","last_clean":"2024-09-10T09:53:00.791474+0000","last_became_active":"2024-09-10T09:52:31.362762+0000","last_became_peered":"2024-09-10T09:52:31.362762+0000","last_unstale":"2024-09-10T09:53:00.791474+0000","last_undegraded":"2024-09-10T09:53:00.791474+0000","last_fullsized":"2024-09-10T09:53:00.791474+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:48:28.275041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.792719+0000","last_change":"2024-09-10T09:52:22.312472+0000","last_active":"2024-09-10T09:53:00.792719+0000","last_peered":"2024-09-10T09:53:00.792719+0000","last_clean":"2024-09-10T09:53:00.792719+0000","last_became_active":"2024-09-10T09:52:22.312117+0000","last_became_peered":"2024-09-10T09:52:22.312117+0000","last_unstale":"2024-09-10T09:53:00.792719+0000","last_undegraded":"2024-09-10T09:53:00.792719+0000","last_fullsized":"2024-09-10T09:53:00.792719+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:39:10.742137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"94'12","reported_seq":48,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:28.327159+0000","last_change":"2024-09-10T09:52:19.835543+0000","last_active":"2024-09-10T09:53:28.327159+0000","last_peered":"2024-09-10T09:53:28.327159+0000","last_clean":"2024-09-10T09:53:28.327159+0000","last_became_active":"2024-09-10T09:52:19.835170+0000","last_became_peered":"2024-09-10T09:52:19.835170+0000","last_unstale":"2024-09-10T09:53:28.327159+0000","last_undegraded":"2024-09-10T09:53:28.327159+0000","last_fullsized":"2024-09-10T09:53:28.327159+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:37.969277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.791386+0000","last_change":"2024-09-10T09:52:31.372398+0000","last_active":"2024-09-10T09:53:00.791386+0000","last_peered":"2024-09-10T09:53:00.791386+0000","last_clean":"2024-09-10T09:53:00.791386+0000","last_became_active":"2024-09-10T09:52:31.371874+0000","last_became_peered":"2024-09-10T09:52:31.371874+0000","last_unstale":"2024-09-10T09:53:00.791386+0000","last_undegraded":"2024-09-10T09:53:00.791386+0000","last_fullsized":"2024-09-10T09:53:00.791386+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:09:45.374320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.795543+0000","last_change":"2024-09-10T09:52:22.319592+0000","last_active":"2024-09-10T09:53:00.795543+0000","last_peered":"2024-09-10T09:53:00.795543+0000","last_clean":"2024-09-10T09:53:00.795543+0000","last_became_active":"2024-09-10T09:52:22.319298+0000","last_became_peered":"2024-09-10T09:52:22.319298+0000","last_unstale":"2024-09-10T09:53:00.795543+0000","last_undegraded":"2024-09-10T09:53:00.795543+0000","last_fullsized":"2024-09-10T09:53:00.795543+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:56:59.787638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"94'4","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:15.875681+0000","last_change":"2024-09-10T09:52:19.912923+0000","last_active":"2024-09-10T09:53:15.875681+0000","last_peered":"2024-09-10T09:53:15.875681+0000","last_clean":"2024-09-10T09:53:15.875681+0000","last_became_active":"2024-09-10T09:52:19.912268+0000","last_became_peered":"2024-09-10T09:52:19.912268+0000","last_unstale":"2024-09-10T09:53:15.875681+0000","last_undegraded":"2024-09-10T09:53:15.875681+0000","last_fullsized":"2024-09-10T09:53:15.875681+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:06:45.421004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.794358+0000","last_change":"2024-09-10T09:52:31.372761+0000","last_active":"2024-09-10T09:53:00.794358+0000","last_peered":"2024-09-10T09:53:00.794358+0000","last_clean":"2024-09-10T09:53:00.794358+0000","last_became_active":"2024-09-10T09:52:31.372293+0000","last_became_peered":"2024-09-10T09:52:31.372293+0000","last_unstale":"2024-09-10T09:53:00.794358+0000","last_undegraded":"2024-09-10T09:53:00.794358+0000","last_fullsized":"2024-09-10T09:53:00.794358+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:36:41.446810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.791685+0000","last_change":"2024-09-10T09:52:22.326291+0000","last_active":"2024-09-10T09:53:00.791685+0000","last_peered":"2024-09-10T09:53:00.791685+0000","last_clean":"2024-09-10T09:53:00.791685+0000","last_became_active":"2024-09-10T09:52:22.325985+0000","last_became_peered":"2024-09-10T09:52:22.325985+0000","last_unstale":"2024-09-10T09:53:00.791685+0000","last_undegraded":"2024-09-10T09:53:00.791685+0000","last_fullsized":"2024-09-10T09:53:00.791685+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:06:21.871502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"94'1","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.794398+0000","last_change":"2024-09-10T09:52:19.912851+0000","last_active":"2024-09-10T09:53:00.794398+0000","last_peered":"2024-09-10T09:53:00.794398+0000","last_clean":"2024-09-10T09:53:00.794398+0000","last_became_active":"2024-09-10T09:52:19.912200+0000","last_became_peered":"2024-09-10T09:52:19.912200+0000","last_unstale":"2024-09-10T09:53:00.794398+0000","last_undegraded":"2024-09-10T09:53:00.794398+0000","last_fullsized":"2024-09-10T09:53:00.794398+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:00:47.331755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.798335+0000","last_change":"2024-09-10T09:52:31.371329+0000","last_active":"2024-09-10T09:53:00.798335+0000","last_peered":"2024-09-10T09:53:00.798335+0000","last_clean":"2024-09-10T09:53:00.798335+0000","last_became_active":"2024-09-10T09:52:31.370971+0000","last_became_peered":"2024-09-10T09:52:31.370971+0000","last_unstale":"2024-09-10T09:53:00.798335+0000","last_undegraded":"2024-09-10T09:53:00.798335+0000","last_fullsized":"2024-09-10T09:53:00.798335+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:07:12.928421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.791638+0000","last_change":"2024-09-10T09:52:22.328039+0000","last_active":"2024-09-10T09:53:00.791638+0000","last_peered":"2024-09-10T09:53:00.791638+0000","last_clean":"2024-09-10T09:53:00.791638+0000","last_became_active":"2024-09-10T09:52:22.327588+0000","last_became_peered":"2024-09-10T09:52:22.327588+0000","last_unstale":"2024-09-10T09:53:00.791638+0000","last_undegraded":"2024-09-10T09:53:00.791638+0000","last_fullsized":"2024-09-10T09:53:00.791638+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:35:39.219544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"94'5","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:30.907026+0000","last_change":"2024-09-10T09:53:00.797651+0000","last_active":"2024-09-10T09:53:30.907026+0000","last_peered":"2024-09-10T09:53:30.907026+0000","last_clean":"2024-09-10T09:53:30.907026+0000","last_became_active":"2024-09-10T09:53:00.797285+0000","last_became_peered":"2024-09-10T09:53:00.797285+0000","last_unstale":"2024-09-10T09:53:30.907026+0000","last_undegraded":"2024-09-10T09:53:30.907026+0000","last_fullsized":"2024-09-10T09:53:30.907026+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:18:02.320032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":17,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,6,1],"acting":[11,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.795291+0000","last_change":"2024-09-10T09:52:31.379215+0000","last_active":"2024-09-10T09:53:00.795291+0000","last_peered":"2024-09-10T09:53:00.795291+0000","last_clean":"2024-09-10T09:53:00.795291+0000","last_became_active":"2024-09-10T09:52:31.377107+0000","last_became_peered":"2024-09-10T09:52:31.377107+0000","last_unstale":"2024-09-10T09:53:00.795291+0000","last_undegraded":"2024-09-10T09:53:00.795291+0000","last_fullsized":"2024-09-10T09:53:00.795291+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:09:39.474077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.795690+0000","last_change":"2024-09-10T09:52:22.316474+0000","last_active":"2024-09-10T09:53:00.795690+0000","last_peered":"2024-09-10T09:53:00.795690+0000","last_clean":"2024-09-10T09:53:00.795690+0000","last_became_active":"2024-09-10T09:52:22.316054+0000","last_became_peered":"2024-09-10T09:52:22.316054+0000","last_unstale":"2024-09-10T09:53:00.795690+0000","last_undegraded":"2024-09-10T09:53:00.795690+0000","last_fullsized":"2024-09-10T09:53:00.795690+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:11:31.334780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"94'5","reported_seq":55,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:26.900488+0000","last_change":"2024-09-10T09:52:19.911601+0000","last_active":"2024-09-10T09:53:26.900488+0000","last_peered":"2024-09-10T09:53:26.900488+0000","last_clean":"2024-09-10T09:53:26.900488+0000","last_became_active":"2024-09-10T09:52:19.911268+0000","last_became_peered":"2024-09-10T09:52:19.911268+0000","last_unstale":"2024-09-10T09:53:26.900488+0000","last_undegraded":"2024-09-10T09:53:26.900488+0000","last_fullsized":"2024-09-10T09:53:26.900488+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:46:38.158784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.796184+0000","last_change":"2024-09-10T09:52:31.375938+0000","last_active":"2024-09-10T09:53:00.796184+0000","last_peered":"2024-09-10T09:53:00.796184+0000","last_clean":"2024-09-10T09:53:00.796184+0000","last_became_active":"2024-09-10T09:52:31.375536+0000","last_became_peered":"2024-09-10T09:52:31.375536+0000","last_unstale":"2024-09-10T09:53:00.796184+0000","last_undegraded":"2024-09-10T09:53:00.796184+0000","last_fullsized":"2024-09-10T09:53:00.796184+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:55:17.303809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.795033+0000","last_change":"2024-09-10T09:52:22.315912+0000","last_active":"2024-09-10T09:53:00.795033+0000","last_peered":"2024-09-10T09:53:00.795033+0000","last_clean":"2024-09-10T09:53:00.795033+0000","last_became_active":"2024-09-10T09:52:22.315433+0000","last_became_peered":"2024-09-10T09:52:22.315433+0000","last_unstale":"2024-09-10T09:53:00.795033+0000","last_undegraded":"2024-09-10T09:53:00.795033+0000","last_fullsized":"2024-09-10T09:53:00.795033+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:57:10.454716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"94'4","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:13.859070+0000","last_change":"2024-09-10T09:52:19.838966+0000","last_active":"2024-09-10T09:53:13.859070+0000","last_peered":"2024-09-10T09:53:13.859070+0000","last_clean":"2024-09-10T09:53:13.859070+0000","last_became_active":"2024-09-10T09:52:19.838545+0000","last_became_peered":"2024-09-10T09:52:19.838545+0000","last_unstale":"2024-09-10T09:53:13.859070+0000","last_undegraded":"2024-09-10T09:53:13.859070+0000","last_fullsized":"2024-09-10T09:53:13.859070+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:07:00.930146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.802039+0000","last_change":"2024-09-10T09:53:00.802039+0000","last_active":"2024-09-10T09:53:00.802039+0000","last_peered":"2024-09-10T09:53:00.802039+0000","last_clean":"2024-09-10T09:53:00.802039+0000","last_became_active":"2024-09-10T09:53:00.800223+0000","last_became_peered":"2024-09-10T09:53:00.800223+0000","last_unstale":"2024-09-10T09:53:00.802039+0000","last_undegraded":"2024-09-10T09:53:00.802039+0000","last_fullsized":"2024-09-10T09:53:00.802039+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:50:16.026631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,6,10],"acting":[5,8,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.791782+0000","last_change":"2024-09-10T09:52:22.313819+0000","last_active":"2024-09-10T09:53:00.791782+0000","last_peered":"2024-09-10T09:53:00.791782+0000","last_clean":"2024-09-10T09:53:00.791782+0000","last_became_active":"2024-09-10T09:52:22.313473+0000","last_became_peered":"2024-09-10T09:52:22.313473+0000","last_unstale":"2024-09-10T09:53:00.791782+0000","last_undegraded":"2024-09-10T09:53:00.791782+0000","last_fullsized":"2024-09-10T09:53:00.791782+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:44:10.003187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"94'6","reported_seq":46,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:16.883322+0000","last_change":"2024-09-10T09:52:19.837376+0000","last_active":"2024-09-10T09:53:16.883322+0000","last_peered":"2024-09-10T09:53:16.883322+0000","last_clean":"2024-09-10T09:53:16.883322+0000","last_became_active":"2024-09-10T09:52:19.836880+0000","last_became_peered":"2024-09-10T09:52:19.836880+0000","last_unstale":"2024-09-10T09:53:16.883322+0000","last_undegraded":"2024-09-10T09:53:16.883322+0000","last_fullsized":"2024-09-10T09:53:16.883322+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:48:58.198188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":356,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.796687+0000","last_change":"2024-09-10T09:51:05.871529+0000","last_active":"2024-09-10T09:53:00.796687+0000","last_peered":"2024-09-10T09:53:00.796687+0000","last_clean":"2024-09-10T09:53:00.796687+0000","last_became_active":"2024-09-10T09:51:05.871115+0000","last_became_peered":"2024-09-10T09:51:05.871115+0000","last_unstale":"2024-09-10T09:53:00.796687+0000","last_undegraded":"2024-09-10T09:53:00.796687+0000","last_fullsized":"2024-09-10T09:53:00.796687+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:47:24.175358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:47:24.175358+0000","last_clean_scrub_stamp":"2024-09-10T09:47:24.175358+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:58:30.247077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"94'1","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.801000+0000","last_change":"2024-09-10T09:53:00.800474+0000","last_active":"2024-09-10T09:53:00.801000+0000","last_peered":"2024-09-10T09:53:00.801000+0000","last_clean":"2024-09-10T09:53:00.801000+0000","last_became_active":"2024-09-10T09:53:00.799932+0000","last_became_peered":"2024-09-10T09:53:00.799932+0000","last_unstale":"2024-09-10T09:53:00.801000+0000","last_undegraded":"2024-09-10T09:53:00.801000+0000","last_fullsized":"2024-09-10T09:53:00.801000+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:14:50.998952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,11,9],"acting":[7,8,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.797158+0000","last_change":"2024-09-10T09:52:22.312521+0000","last_active":"2024-09-10T09:53:00.797158+0000","last_peered":"2024-09-10T09:53:00.797158+0000","last_clean":"2024-09-10T09:53:00.797158+0000","last_became_active":"2024-09-10T09:52:22.312107+0000","last_became_peered":"2024-09-10T09:52:22.312107+0000","last_unstale":"2024-09-10T09:53:00.797158+0000","last_undegraded":"2024-09-10T09:53:00.797158+0000","last_fullsized":"2024-09-10T09:53:00.797158+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:14:19.799259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"94'5","reported_seq":56,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:26.900351+0000","last_change":"2024-09-10T09:52:19.911454+0000","last_active":"2024-09-10T09:53:26.900351+0000","last_peered":"2024-09-10T09:53:26.900351+0000","last_clean":"2024-09-10T09:53:26.900351+0000","last_became_active":"2024-09-10T09:52:19.911079+0000","last_became_peered":"2024-09-10T09:52:19.911079+0000","last_unstale":"2024-09-10T09:53:26.900351+0000","last_undegraded":"2024-09-10T09:53:26.900351+0000","last_fullsized":"2024-09-10T09:53:26.900351+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:43:38.787689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":14,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.794164+0000","last_change":"2024-09-10T09:52:31.381877+0000","last_active":"2024-09-10T09:53:00.794164+0000","last_peered":"2024-09-10T09:53:00.794164+0000","last_clean":"2024-09-10T09:53:00.794164+0000","last_became_active":"2024-09-10T09:52:31.381472+0000","last_became_peered":"2024-09-10T09:52:31.381472+0000","last_unstale":"2024-09-10T09:53:00.794164+0000","last_undegraded":"2024-09-10T09:53:00.794164+0000","last_fullsized":"2024-09-10T09:53:00.794164+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:41:24.686325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.797216+0000","last_change":"2024-09-10T09:52:22.313902+0000","last_active":"2024-09-10T09:53:00.797216+0000","last_peered":"2024-09-10T09:53:00.797216+0000","last_clean":"2024-09-10T09:53:00.797216+0000","last_became_active":"2024-09-10T09:52:22.313515+0000","last_became_peered":"2024-09-10T09:52:22.313515+0000","last_unstale":"2024-09-10T09:53:00.797216+0000","last_undegraded":"2024-09-10T09:53:00.797216+0000","last_fullsized":"2024-09-10T09:53:00.797216+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:25:00.238307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"94'3","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:15.875823+0000","last_change":"2024-09-10T09:52:19.837736+0000","last_active":"2024-09-10T09:53:15.875823+0000","last_peered":"2024-09-10T09:53:15.875823+0000","last_clean":"2024-09-10T09:53:15.875823+0000","last_became_active":"2024-09-10T09:52:19.837370+0000","last_became_peered":"2024-09-10T09:52:19.837370+0000","last_unstale":"2024-09-10T09:53:15.875823+0000","last_undegraded":"2024-09-10T09:53:15.875823+0000","last_fullsized":"2024-09-10T09:53:15.875823+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:54:35.753027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.844672+0000","last_change":"2024-09-10T09:52:31.378945+0000","last_active":"2024-09-10T09:53:00.844672+0000","last_peered":"2024-09-10T09:53:00.844672+0000","last_clean":"2024-09-10T09:53:00.844672+0000","last_became_active":"2024-09-10T09:52:31.378144+0000","last_became_peered":"2024-09-10T09:52:31.378144+0000","last_unstale":"2024-09-10T09:53:00.844672+0000","last_undegraded":"2024-09-10T09:53:00.844672+0000","last_fullsized":"2024-09-10T09:53:00.844672+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:36:06.288293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.797422+0000","last_change":"2024-09-10T09:52:22.316400+0000","last_active":"2024-09-10T09:53:00.797422+0000","last_peered":"2024-09-10T09:53:00.797422+0000","last_clean":"2024-09-10T09:53:00.797422+0000","last_became_active":"2024-09-10T09:52:22.315974+0000","last_became_peered":"2024-09-10T09:52:22.315974+0000","last_unstale":"2024-09-10T09:53:00.797422+0000","last_undegraded":"2024-09-10T09:53:00.797422+0000","last_fullsized":"2024-09-10T09:53:00.797422+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:31:05.538237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"94'5","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:15.874169+0000","last_change":"2024-09-10T09:52:19.837942+0000","last_active":"2024-09-10T09:53:15.874169+0000","last_peered":"2024-09-10T09:53:15.874169+0000","last_clean":"2024-09-10T09:53:15.874169+0000","last_became_active":"2024-09-10T09:52:19.837629+0000","last_became_peered":"2024-09-10T09:52:19.837629+0000","last_unstale":"2024-09-10T09:53:15.874169+0000","last_undegraded":"2024-09-10T09:53:15.874169+0000","last_fullsized":"2024-09-10T09:53:15.874169+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:34:13.207770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.794626+0000","last_change":"2024-09-10T09:52:31.364231+0000","last_active":"2024-09-10T09:53:00.794626+0000","last_peered":"2024-09-10T09:53:00.794626+0000","last_clean":"2024-09-10T09:53:00.794626+0000","last_became_active":"2024-09-10T09:52:31.363833+0000","last_became_peered":"2024-09-10T09:52:31.363833+0000","last_unstale":"2024-09-10T09:53:00.794626+0000","last_undegraded":"2024-09-10T09:53:00.794626+0000","last_fullsized":"2024-09-10T09:53:00.794626+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:53:51.435700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.795271+0000","last_change":"2024-09-10T09:52:22.319530+0000","last_active":"2024-09-10T09:53:00.795271+0000","last_peered":"2024-09-10T09:53:00.795271+0000","last_clean":"2024-09-10T09:53:00.795271+0000","last_became_active":"2024-09-10T09:52:22.318742+0000","last_became_peered":"2024-09-10T09:52:22.318742+0000","last_unstale":"2024-09-10T09:53:00.795271+0000","last_undegraded":"2024-09-10T09:53:00.795271+0000","last_fullsized":"2024-09-10T09:53:00.795271+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:44:35.050454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"94'2","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:14.854723+0000","last_change":"2024-09-10T09:52:19.836398+0000","last_active":"2024-09-10T09:53:14.854723+0000","last_peered":"2024-09-10T09:53:14.854723+0000","last_clean":"2024-09-10T09:53:14.854723+0000","last_became_active":"2024-09-10T09:52:19.836055+0000","last_became_peered":"2024-09-10T09:52:19.836055+0000","last_unstale":"2024-09-10T09:53:14.854723+0000","last_undegraded":"2024-09-10T09:53:14.854723+0000","last_fullsized":"2024-09-10T09:53:14.854723+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:49:26.923367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.794659+0000","last_change":"2024-09-10T09:52:31.371109+0000","last_active":"2024-09-10T09:53:00.794659+0000","last_peered":"2024-09-10T09:53:00.794659+0000","last_clean":"2024-09-10T09:53:00.794659+0000","last_became_active":"2024-09-10T09:52:31.370592+0000","last_became_peered":"2024-09-10T09:52:31.370592+0000","last_unstale":"2024-09-10T09:53:00.794659+0000","last_undegraded":"2024-09-10T09:53:00.794659+0000","last_fullsized":"2024-09-10T09:53:00.794659+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:20:23.585187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.794797+0000","last_change":"2024-09-10T09:52:22.319365+0000","last_active":"2024-09-10T09:53:00.794797+0000","last_peered":"2024-09-10T09:53:00.794797+0000","last_clean":"2024-09-10T09:53:00.794797+0000","last_became_active":"2024-09-10T09:52:22.319064+0000","last_became_peered":"2024-09-10T09:52:22.319064+0000","last_unstale":"2024-09-10T09:53:00.794797+0000","last_undegraded":"2024-09-10T09:53:00.794797+0000","last_fullsized":"2024-09-10T09:53:00.794797+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:14:50.089566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"94'17","reported_seq":54,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:15.875370+0000","last_change":"2024-09-10T09:52:19.911698+0000","last_active":"2024-09-10T09:53:15.875370+0000","last_peered":"2024-09-10T09:53:15.875370+0000","last_clean":"2024-09-10T09:53:15.875370+0000","last_became_active":"2024-09-10T09:52:19.911369+0000","last_became_peered":"2024-09-10T09:52:19.911369+0000","last_unstale":"2024-09-10T09:53:15.875370+0000","last_undegraded":"2024-09-10T09:53:15.875370+0000","last_fullsized":"2024-09-10T09:53:15.875370+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:40:39.045797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.796024+0000","last_change":"2024-09-10T09:52:31.380463+0000","last_active":"2024-09-10T09:53:00.796024+0000","last_peered":"2024-09-10T09:53:00.796024+0000","last_clean":"2024-09-10T09:53:00.796024+0000","last_became_active":"2024-09-10T09:52:31.380114+0000","last_became_peered":"2024-09-10T09:52:31.380114+0000","last_unstale":"2024-09-10T09:53:00.796024+0000","last_undegraded":"2024-09-10T09:53:00.796024+0000","last_fullsized":"2024-09-10T09:53:00.796024+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:19:50.869160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.796355+0000","last_change":"2024-09-10T09:52:22.313928+0000","last_active":"2024-09-10T09:53:00.796355+0000","last_peered":"2024-09-10T09:53:00.796355+0000","last_clean":"2024-09-10T09:53:00.796355+0000","last_became_active":"2024-09-10T09:52:22.313510+0000","last_became_peered":"2024-09-10T09:52:22.313510+0000","last_unstale":"2024-09-10T09:53:00.796355+0000","last_undegraded":"2024-09-10T09:53:00.796355+0000","last_fullsized":"2024-09-10T09:53:00.796355+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:58:19.031178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"94'6","reported_seq":58,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:26.901026+0000","last_change":"2024-09-10T09:52:19.839451+0000","last_active":"2024-09-10T09:53:26.901026+0000","last_peered":"2024-09-10T09:53:26.901026+0000","last_clean":"2024-09-10T09:53:26.901026+0000","last_became_active":"2024-09-10T09:52:19.839124+0000","last_became_peered":"2024-09-10T09:52:19.839124+0000","last_unstale":"2024-09-10T09:53:26.901026+0000","last_undegraded":"2024-09-10T09:53:26.901026+0000","last_fullsized":"2024-09-10T09:53:26.901026+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:51:33.489830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":16,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.794814+0000","last_change":"2024-09-10T09:52:31.376386+0000","last_active":"2024-09-10T09:53:00.794814+0000","last_peered":"2024-09-10T09:53:00.794814+0000","last_clean":"2024-09-10T09:53:00.794814+0000","last_became_active":"2024-09-10T09:52:31.376066+0000","last_became_peered":"2024-09-10T09:52:31.376066+0000","last_unstale":"2024-09-10T09:53:00.794814+0000","last_undegraded":"2024-09-10T09:53:00.794814+0000","last_fullsized":"2024-09-10T09:53:00.794814+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:23:07.191842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.796616+0000","last_change":"2024-09-10T09:52:22.323590+0000","last_active":"2024-09-10T09:53:00.796616+0000","last_peered":"2024-09-10T09:53:00.796616+0000","last_clean":"2024-09-10T09:53:00.796616+0000","last_became_active":"2024-09-10T09:52:22.323264+0000","last_became_peered":"2024-09-10T09:52:22.323264+0000","last_unstale":"2024-09-10T09:53:00.796616+0000","last_undegraded":"2024-09-10T09:53:00.796616+0000","last_fullsized":"2024-09-10T09:53:00.796616+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:07:56.572337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"94'5","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:15.875891+0000","last_change":"2024-09-10T09:52:19.840620+0000","last_active":"2024-09-10T09:53:15.875891+0000","last_peered":"2024-09-10T09:53:15.875891+0000","last_clean":"2024-09-10T09:53:15.875891+0000","last_became_active":"2024-09-10T09:52:19.840217+0000","last_became_peered":"2024-09-10T09:52:19.840217+0000","last_unstale":"2024-09-10T09:53:15.875891+0000","last_undegraded":"2024-09-10T09:53:15.875891+0000","last_fullsized":"2024-09-10T09:53:15.875891+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:55:13.111993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.794521+0000","last_change":"2024-09-10T09:52:31.373895+0000","last_active":"2024-09-10T09:53:00.794521+0000","last_peered":"2024-09-10T09:53:00.794521+0000","last_clean":"2024-09-10T09:53:00.794521+0000","last_became_active":"2024-09-10T09:52:31.373474+0000","last_became_peered":"2024-09-10T09:52:31.373474+0000","last_unstale":"2024-09-10T09:53:00.794521+0000","last_undegraded":"2024-09-10T09:53:00.794521+0000","last_fullsized":"2024-09-10T09:53:00.794521+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:39:16.049859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.796069+0000","last_change":"2024-09-10T09:52:22.319294+0000","last_active":"2024-09-10T09:53:00.796069+0000","last_peered":"2024-09-10T09:53:00.796069+0000","last_clean":"2024-09-10T09:53:00.796069+0000","last_became_active":"2024-09-10T09:52:22.318918+0000","last_became_peered":"2024-09-10T09:52:22.318918+0000","last_unstale":"2024-09-10T09:53:00.796069+0000","last_undegraded":"2024-09-10T09:53:00.796069+0000","last_fullsized":"2024-09-10T09:53:00.796069+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:52:46.434182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"94'9","reported_seq":48,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:16.883211+0000","last_change":"2024-09-10T09:52:19.842286+0000","last_active":"2024-09-10T09:53:16.883211+0000","last_peered":"2024-09-10T09:53:16.883211+0000","last_clean":"2024-09-10T09:53:16.883211+0000","last_became_active":"2024-09-10T09:52:19.841930+0000","last_became_peered":"2024-09-10T09:52:19.841930+0000","last_unstale":"2024-09-10T09:53:16.883211+0000","last_undegraded":"2024-09-10T09:53:16.883211+0000","last_fullsized":"2024-09-10T09:53:16.883211+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:38:46.358389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.796684+0000","last_change":"2024-09-10T09:52:31.370961+0000","last_active":"2024-09-10T09:53:00.796684+0000","last_peered":"2024-09-10T09:53:00.796684+0000","last_clean":"2024-09-10T09:53:00.796684+0000","last_became_active":"2024-09-10T09:52:31.370545+0000","last_became_peered":"2024-09-10T09:52:31.370545+0000","last_unstale":"2024-09-10T09:53:00.796684+0000","last_undegraded":"2024-09-10T09:53:00.796684+0000","last_fullsized":"2024-09-10T09:53:00.796684+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:17.461447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.798764+0000","last_change":"2024-09-10T09:52:22.313215+0000","last_active":"2024-09-10T09:53:00.798764+0000","last_peered":"2024-09-10T09:53:00.798764+0000","last_clean":"2024-09-10T09:53:00.798764+0000","last_became_active":"2024-09-10T09:52:22.312918+0000","last_became_peered":"2024-09-10T09:52:22.312918+0000","last_unstale":"2024-09-10T09:53:00.798764+0000","last_undegraded":"2024-09-10T09:53:00.798764+0000","last_fullsized":"2024-09-10T09:53:00.798764+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:48:09.568004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"94'31","reported_seq":123,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:31.907970+0000","last_change":"2024-09-10T09:53:01.226146+0000","last_active":"2024-09-10T09:53:31.907970+0000","last_peered":"2024-09-10T09:53:31.907970+0000","last_clean":"2024-09-10T09:53:31.907970+0000","last_became_active":"2024-09-10T09:53:01.218622+0000","last_became_peered":"2024-09-10T09:53:01.218622+0000","last_unstale":"2024-09-10T09:53:31.907970+0000","last_undegraded":"2024-09-10T09:53:31.907970+0000","last_fullsized":"2024-09-10T09:53:31.907970+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:43:11.927029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56749,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":38,"num_read_kb":74,"num_write":33,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":3844,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.801292+0000","last_change":"2024-09-10T09:53:00.800650+0000","last_active":"2024-09-10T09:53:00.801292+0000","last_peered":"2024-09-10T09:53:00.801292+0000","last_clean":"2024-09-10T09:53:00.801292+0000","last_became_active":"2024-09-10T09:53:00.800104+0000","last_became_peered":"2024-09-10T09:53:00.800104+0000","last_unstale":"2024-09-10T09:53:00.801292+0000","last_undegraded":"2024-09-10T09:53:00.801292+0000","last_fullsized":"2024-09-10T09:53:00.801292+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:41:16.022884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.794669+0000","last_change":"2024-09-10T09:52:22.317174+0000","last_active":"2024-09-10T09:53:00.794669+0000","last_peered":"2024-09-10T09:53:00.794669+0000","last_clean":"2024-09-10T09:53:00.794669+0000","last_became_active":"2024-09-10T09:52:22.316851+0000","last_became_peered":"2024-09-10T09:52:22.316851+0000","last_unstale":"2024-09-10T09:53:00.794669+0000","last_undegraded":"2024-09-10T09:53:00.794669+0000","last_fullsized":"2024-09-10T09:53:00.794669+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:55:18.742393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"94'5","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:14.863925+0000","last_change":"2024-09-10T09:52:19.912093+0000","last_active":"2024-09-10T09:53:14.863925+0000","last_peered":"2024-09-10T09:53:14.863925+0000","last_clean":"2024-09-10T09:53:14.863925+0000","last_became_active":"2024-09-10T09:52:19.911611+0000","last_became_peered":"2024-09-10T09:52:19.911611+0000","last_unstale":"2024-09-10T09:53:14.863925+0000","last_undegraded":"2024-09-10T09:53:14.863925+0000","last_fullsized":"2024-09-10T09:53:14.863925+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:01:10.859310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:01.217603+0000","last_change":"2024-09-10T09:52:31.362884+0000","last_active":"2024-09-10T09:53:01.217603+0000","last_peered":"2024-09-10T09:53:01.217603+0000","last_clean":"2024-09-10T09:53:01.217603+0000","last_became_active":"2024-09-10T09:52:31.362489+0000","last_became_peered":"2024-09-10T09:52:31.362489+0000","last_unstale":"2024-09-10T09:53:01.217603+0000","last_undegraded":"2024-09-10T09:53:01.217603+0000","last_fullsized":"2024-09-10T09:53:01.217603+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:02:33.084795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.845137+0000","last_change":"2024-09-10T09:52:22.325871+0000","last_active":"2024-09-10T09:53:00.845137+0000","last_peered":"2024-09-10T09:53:00.845137+0000","last_clean":"2024-09-10T09:53:00.845137+0000","last_became_active":"2024-09-10T09:52:22.324980+0000","last_became_peered":"2024-09-10T09:52:22.324980+0000","last_unstale":"2024-09-10T09:53:00.845137+0000","last_undegraded":"2024-09-10T09:53:00.845137+0000","last_fullsized":"2024-09-10T09:53:00.845137+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:31:55.713444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"94'4","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:15.874898+0000","last_change":"2024-09-10T09:52:19.839491+0000","last_active":"2024-09-10T09:53:15.874898+0000","last_peered":"2024-09-10T09:53:15.874898+0000","last_clean":"2024-09-10T09:53:15.874898+0000","last_became_active":"2024-09-10T09:52:19.839092+0000","last_became_peered":"2024-09-10T09:52:19.839092+0000","last_unstale":"2024-09-10T09:53:15.874898+0000","last_undegraded":"2024-09-10T09:53:15.874898+0000","last_fullsized":"2024-09-10T09:53:15.874898+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:42:13.453281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.793758+0000","last_change":"2024-09-10T09:52:31.371346+0000","last_active":"2024-09-10T09:53:00.793758+0000","last_peered":"2024-09-10T09:53:00.793758+0000","last_clean":"2024-09-10T09:53:00.793758+0000","last_became_active":"2024-09-10T09:52:31.370929+0000","last_became_peered":"2024-09-10T09:52:31.370929+0000","last_unstale":"2024-09-10T09:53:00.793758+0000","last_undegraded":"2024-09-10T09:53:00.793758+0000","last_fullsized":"2024-09-10T09:53:00.793758+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:44:22.925082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.794905+0000","last_change":"2024-09-10T09:52:22.317453+0000","last_active":"2024-09-10T09:53:00.794905+0000","last_peered":"2024-09-10T09:53:00.794905+0000","last_clean":"2024-09-10T09:53:00.794905+0000","last_became_active":"2024-09-10T09:52:22.315556+0000","last_became_peered":"2024-09-10T09:52:22.315556+0000","last_unstale":"2024-09-10T09:53:00.794905+0000","last_undegraded":"2024-09-10T09:53:00.794905+0000","last_fullsized":"2024-09-10T09:53:00.794905+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:56:05.860275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"94'4","reported_seq":74,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:31.907740+0000","last_change":"2024-09-10T09:52:19.835985+0000","last_active":"2024-09-10T09:53:31.907740+0000","last_peered":"2024-09-10T09:53:31.907740+0000","last_clean":"2024-09-10T09:53:31.907740+0000","last_became_active":"2024-09-10T09:52:19.834835+0000","last_became_peered":"2024-09-10T09:52:19.834835+0000","last_unstale":"2024-09-10T09:53:31.907740+0000","last_undegraded":"2024-09-10T09:53:31.907740+0000","last_fullsized":"2024-09-10T09:53:31.907740+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:09:11.590443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":32,"num_read_kb":32,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.791473+0000","last_change":"2024-09-10T09:52:31.368663+0000","last_active":"2024-09-10T09:53:00.791473+0000","last_peered":"2024-09-10T09:53:00.791473+0000","last_clean":"2024-09-10T09:53:00.791473+0000","last_became_active":"2024-09-10T09:52:31.368004+0000","last_became_peered":"2024-09-10T09:52:31.368004+0000","last_unstale":"2024-09-10T09:53:00.791473+0000","last_undegraded":"2024-09-10T09:53:00.791473+0000","last_fullsized":"2024-09-10T09:53:00.791473+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:56:15.100515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.844866+0000","last_change":"2024-09-10T09:52:31.370992+0000","last_active":"2024-09-10T09:53:00.844866+0000","last_peered":"2024-09-10T09:53:00.844866+0000","last_clean":"2024-09-10T09:53:00.844866+0000","last_became_active":"2024-09-10T09:52:31.370357+0000","last_became_peered":"2024-09-10T09:52:31.370357+0000","last_unstale":"2024-09-10T09:53:00.844866+0000","last_undegraded":"2024-09-10T09:53:00.844866+0000","last_fullsized":"2024-09-10T09:53:00.844866+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:02:49.482532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.793913+0000","last_change":"2024-09-10T09:52:22.319363+0000","last_active":"2024-09-10T09:53:00.793913+0000","last_peered":"2024-09-10T09:53:00.793913+0000","last_clean":"2024-09-10T09:53:00.793913+0000","last_became_active":"2024-09-10T09:52:22.319044+0000","last_became_peered":"2024-09-10T09:52:22.319044+0000","last_unstale":"2024-09-10T09:53:00.793913+0000","last_undegraded":"2024-09-10T09:53:00.793913+0000","last_fullsized":"2024-09-10T09:53:00.793913+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:30:51.328413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"94'3","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:15.874728+0000","last_change":"2024-09-10T09:52:19.841745+0000","last_active":"2024-09-10T09:53:15.874728+0000","last_peered":"2024-09-10T09:53:15.874728+0000","last_clean":"2024-09-10T09:53:15.874728+0000","last_became_active":"2024-09-10T09:52:19.841404+0000","last_became_peered":"2024-09-10T09:52:19.841404+0000","last_unstale":"2024-09-10T09:53:15.874728+0000","last_undegraded":"2024-09-10T09:53:15.874728+0000","last_fullsized":"2024-09-10T09:53:15.874728+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:44:36.346785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:01.218412+0000","last_change":"2024-09-10T09:53:01.218412+0000","last_active":"2024-09-10T09:53:01.218412+0000","last_peered":"2024-09-10T09:53:01.218412+0000","last_clean":"2024-09-10T09:53:01.218412+0000","last_became_active":"2024-09-10T09:53:01.217966+0000","last_became_peered":"2024-09-10T09:53:01.217966+0000","last_unstale":"2024-09-10T09:53:01.218412+0000","last_undegraded":"2024-09-10T09:53:01.218412+0000","last_fullsized":"2024-09-10T09:53:01.218412+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:53:25.743913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.791538+0000","last_change":"2024-09-10T09:52:22.316120+0000","last_active":"2024-09-10T09:53:00.791538+0000","last_peered":"2024-09-10T09:53:00.791538+0000","last_clean":"2024-09-10T09:53:00.791538+0000","last_became_active":"2024-09-10T09:52:22.315769+0000","last_became_peered":"2024-09-10T09:52:22.315769+0000","last_unstale":"2024-09-10T09:53:00.791538+0000","last_undegraded":"2024-09-10T09:53:00.791538+0000","last_fullsized":"2024-09-10T09:53:00.791538+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:16:41.866587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"94'3","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:13.861889+0000","last_change":"2024-09-10T09:52:19.835038+0000","last_active":"2024-09-10T09:53:13.861889+0000","last_peered":"2024-09-10T09:53:13.861889+0000","last_clean":"2024-09-10T09:53:13.861889+0000","last_became_active":"2024-09-10T09:52:19.834329+0000","last_became_peered":"2024-09-10T09:52:19.834329+0000","last_unstale":"2024-09-10T09:53:13.861889+0000","last_undegraded":"2024-09-10T09:53:13.861889+0000","last_fullsized":"2024-09-10T09:53:13.861889+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:58:24.084373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"94'7","reported_seq":51,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:14.865882+0000","last_change":"2024-09-10T09:53:01.219122+0000","last_active":"2024-09-10T09:53:14.865882+0000","last_peered":"2024-09-10T09:53:14.865882+0000","last_clean":"2024-09-10T09:53:14.865882+0000","last_became_active":"2024-09-10T09:53:01.218599+0000","last_became_peered":"2024-09-10T09:53:01.218599+0000","last_unstale":"2024-09-10T09:53:14.865882+0000","last_undegraded":"2024-09-10T09:53:14.865882+0000","last_fullsized":"2024-09-10T09:53:14.865882+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:31:10.092264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.795295+0000","last_change":"2024-09-10T09:52:22.316688+0000","last_active":"2024-09-10T09:53:00.795295+0000","last_peered":"2024-09-10T09:53:00.795295+0000","last_clean":"2024-09-10T09:53:00.795295+0000","last_became_active":"2024-09-10T09:52:22.316388+0000","last_became_peered":"2024-09-10T09:52:22.316388+0000","last_unstale":"2024-09-10T09:53:00.795295+0000","last_undegraded":"2024-09-10T09:53:00.795295+0000","last_fullsized":"2024-09-10T09:53:00.795295+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:30:46.219846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.791663+0000","last_change":"2024-09-10T09:52:31.371612+0000","last_active":"2024-09-10T09:53:00.791663+0000","last_peered":"2024-09-10T09:53:00.791663+0000","last_clean":"2024-09-10T09:53:00.791663+0000","last_became_active":"2024-09-10T09:52:31.371163+0000","last_became_peered":"2024-09-10T09:52:31.371163+0000","last_unstale":"2024-09-10T09:53:00.791663+0000","last_undegraded":"2024-09-10T09:53:00.791663+0000","last_fullsized":"2024-09-10T09:53:00.791663+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:11:43.681524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"94'6","reported_seq":64,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:31.908330+0000","last_change":"2024-09-10T09:52:19.841873+0000","last_active":"2024-09-10T09:53:31.908330+0000","last_peered":"2024-09-10T09:53:31.908330+0000","last_clean":"2024-09-10T09:53:31.908330+0000","last_became_active":"2024-09-10T09:52:19.841513+0000","last_became_peered":"2024-09-10T09:52:19.841513+0000","last_unstale":"2024-09-10T09:53:31.908330+0000","last_undegraded":"2024-09-10T09:53:31.908330+0000","last_fullsized":"2024-09-10T09:53:31.908330+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:53:31.854185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":20,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.798538+0000","last_change":"2024-09-10T09:52:22.314781+0000","last_active":"2024-09-10T09:53:00.798538+0000","last_peered":"2024-09-10T09:53:00.798538+0000","last_clean":"2024-09-10T09:53:00.798538+0000","last_became_active":"2024-09-10T09:52:22.313346+0000","last_became_peered":"2024-09-10T09:52:22.313346+0000","last_unstale":"2024-09-10T09:53:00.798538+0000","last_undegraded":"2024-09-10T09:53:00.798538+0000","last_fullsized":"2024-09-10T09:53:00.798538+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:15:16.921963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.791726+0000","last_change":"2024-09-10T09:52:31.365544+0000","last_active":"2024-09-10T09:53:00.791726+0000","last_peered":"2024-09-10T09:53:00.791726+0000","last_clean":"2024-09-10T09:53:00.791726+0000","last_became_active":"2024-09-10T09:52:31.365151+0000","last_became_peered":"2024-09-10T09:52:31.365151+0000","last_unstale":"2024-09-10T09:53:00.791726+0000","last_undegraded":"2024-09-10T09:53:00.791726+0000","last_fullsized":"2024-09-10T09:53:00.791726+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:36:55.247890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.793781+0000","last_change":"2024-09-10T09:52:22.314117+0000","last_active":"2024-09-10T09:53:00.793781+0000","last_peered":"2024-09-10T09:53:00.793781+0000","last_clean":"2024-09-10T09:53:00.793781+0000","last_became_active":"2024-09-10T09:52:22.313772+0000","last_became_peered":"2024-09-10T09:52:22.313772+0000","last_unstale":"2024-09-10T09:53:00.793781+0000","last_undegraded":"2024-09-10T09:53:00.793781+0000","last_fullsized":"2024-09-10T09:53:00.793781+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:19:37.373598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.792282+0000","last_change":"2024-09-10T09:52:22.314362+0000","last_active":"2024-09-10T09:53:00.792282+0000","last_peered":"2024-09-10T09:53:00.792282+0000","last_clean":"2024-09-10T09:53:00.792282+0000","last_became_active":"2024-09-10T09:52:22.314050+0000","last_became_peered":"2024-09-10T09:52:22.314050+0000","last_unstale":"2024-09-10T09:53:00.792282+0000","last_undegraded":"2024-09-10T09:53:00.792282+0000","last_fullsized":"2024-09-10T09:53:00.792282+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:37:07.218656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.791411+0000","last_change":"2024-09-10T09:52:22.322363+0000","last_active":"2024-09-10T09:53:00.791411+0000","last_peered":"2024-09-10T09:53:00.791411+0000","last_clean":"2024-09-10T09:53:00.791411+0000","last_became_active":"2024-09-10T09:52:22.322045+0000","last_became_peered":"2024-09-10T09:52:22.322045+0000","last_unstale":"2024-09-10T09:53:00.791411+0000","last_undegraded":"2024-09-10T09:53:00.791411+0000","last_fullsized":"2024-09-10T09:53:00.791411+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:10:27.918769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.844741+0000","last_change":"2024-09-10T09:52:22.331594+0000","last_active":"2024-09-10T09:53:00.844741+0000","last_peered":"2024-09-10T09:53:00.844741+0000","last_clean":"2024-09-10T09:53:00.844741+0000","last_became_active":"2024-09-10T09:52:22.330585+0000","last_became_peered":"2024-09-10T09:52:22.330585+0000","last_unstale":"2024-09-10T09:53:00.844741+0000","last_undegraded":"2024-09-10T09:53:00.844741+0000","last_fullsized":"2024-09-10T09:53:00.844741+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:45:56.412313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.795657+0000","last_change":"2024-09-10T09:52:22.325753+0000","last_active":"2024-09-10T09:53:00.795657+0000","last_peered":"2024-09-10T09:53:00.795657+0000","last_clean":"2024-09-10T09:53:00.795657+0000","last_became_active":"2024-09-10T09:52:22.325385+0000","last_became_peered":"2024-09-10T09:52:22.325385+0000","last_unstale":"2024-09-10T09:53:00.795657+0000","last_undegraded":"2024-09-10T09:53:00.795657+0000","last_fullsized":"2024-09-10T09:53:00.795657+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:18:28.776789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.795435+0000","last_change":"2024-09-10T09:52:22.319069+0000","last_active":"2024-09-10T09:53:00.795435+0000","last_peered":"2024-09-10T09:53:00.795435+0000","last_clean":"2024-09-10T09:53:00.795435+0000","last_became_active":"2024-09-10T09:52:22.318667+0000","last_became_peered":"2024-09-10T09:52:22.318667+0000","last_unstale":"2024-09-10T09:53:00.795435+0000","last_undegraded":"2024-09-10T09:53:00.795435+0000","last_fullsized":"2024-09-10T09:53:00.795435+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:21:32.028235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.792031+0000","last_change":"2024-09-10T09:52:22.312890+0000","last_active":"2024-09-10T09:53:00.792031+0000","last_peered":"2024-09-10T09:53:00.792031+0000","last_clean":"2024-09-10T09:53:00.792031+0000","last_became_active":"2024-09-10T09:52:22.312608+0000","last_became_peered":"2024-09-10T09:52:22.312608+0000","last_unstale":"2024-09-10T09:53:00.792031+0000","last_undegraded":"2024-09-10T09:53:00.792031+0000","last_fullsized":"2024-09-10T09:53:00.792031+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:07:46.046915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.792630+0000","last_change":"2024-09-10T09:52:22.327379+0000","last_active":"2024-09-10T09:53:00.792630+0000","last_peered":"2024-09-10T09:53:00.792630+0000","last_clean":"2024-09-10T09:53:00.792630+0000","last_became_active":"2024-09-10T09:52:22.326921+0000","last_became_peered":"2024-09-10T09:52:22.326921+0000","last_unstale":"2024-09-10T09:53:00.792630+0000","last_undegraded":"2024-09-10T09:53:00.792630+0000","last_fullsized":"2024-09-10T09:53:00.792630+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:01:53.732805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.794884+0000","last_change":"2024-09-10T09:52:22.321731+0000","last_active":"2024-09-10T09:53:00.794884+0000","last_peered":"2024-09-10T09:53:00.794884+0000","last_clean":"2024-09-10T09:53:00.794884+0000","last_became_active":"2024-09-10T09:52:22.321408+0000","last_became_peered":"2024-09-10T09:52:22.321408+0000","last_unstale":"2024-09-10T09:53:00.794884+0000","last_undegraded":"2024-09-10T09:53:00.794884+0000","last_fullsized":"2024-09-10T09:53:00.794884+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:19:57.800678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.791571+0000","last_change":"2024-09-10T09:52:22.313538+0000","last_active":"2024-09-10T09:53:00.791571+0000","last_peered":"2024-09-10T09:53:00.791571+0000","last_clean":"2024-09-10T09:53:00.791571+0000","last_became_active":"2024-09-10T09:52:22.313196+0000","last_became_peered":"2024-09-10T09:52:22.313196+0000","last_unstale":"2024-09-10T09:53:00.791571+0000","last_undegraded":"2024-09-10T09:53:00.791571+0000","last_fullsized":"2024-09-10T09:53:00.791571+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:53:23.387167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.794521+0000","last_change":"2024-09-10T09:52:22.316518+0000","last_active":"2024-09-10T09:53:00.794521+0000","last_peered":"2024-09-10T09:53:00.794521+0000","last_clean":"2024-09-10T09:53:00.794521+0000","last_became_active":"2024-09-10T09:52:22.316193+0000","last_became_peered":"2024-09-10T09:52:22.316193+0000","last_unstale":"2024-09-10T09:53:00.794521+0000","last_undegraded":"2024-09-10T09:53:00.794521+0000","last_fullsized":"2024-09-10T09:53:00.794521+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:49:55.903928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.796514+0000","last_change":"2024-09-10T09:52:22.329163+0000","last_active":"2024-09-10T09:53:00.796514+0000","last_peered":"2024-09-10T09:53:00.796514+0000","last_clean":"2024-09-10T09:53:00.796514+0000","last_became_active":"2024-09-10T09:52:22.328453+0000","last_became_peered":"2024-09-10T09:52:22.328453+0000","last_unstale":"2024-09-10T09:53:00.796514+0000","last_undegraded":"2024-09-10T09:53:00.796514+0000","last_fullsized":"2024-09-10T09:53:00.796514+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:53:33.286055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.794654+0000","last_change":"2024-09-10T09:52:22.314302+0000","last_active":"2024-09-10T09:53:00.794654+0000","last_peered":"2024-09-10T09:53:00.794654+0000","last_clean":"2024-09-10T09:53:00.794654+0000","last_became_active":"2024-09-10T09:52:22.313992+0000","last_became_peered":"2024-09-10T09:52:22.313992+0000","last_unstale":"2024-09-10T09:53:00.794654+0000","last_undegraded":"2024-09-10T09:53:00.794654+0000","last_fullsized":"2024-09-10T09:53:00.794654+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:07:35.764015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.791948+0000","last_change":"2024-09-10T09:52:22.316400+0000","last_active":"2024-09-10T09:53:00.791948+0000","last_peered":"2024-09-10T09:53:00.791948+0000","last_clean":"2024-09-10T09:53:00.791948+0000","last_became_active":"2024-09-10T09:52:22.316110+0000","last_became_peered":"2024-09-10T09:52:22.316110+0000","last_unstale":"2024-09-10T09:53:00.791948+0000","last_undegraded":"2024-09-10T09:53:00.791948+0000","last_fullsized":"2024-09-10T09:53:00.791948+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:17:26.528899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.794833+0000","last_change":"2024-09-10T09:52:22.317914+0000","last_active":"2024-09-10T09:53:00.794833+0000","last_peered":"2024-09-10T09:53:00.794833+0000","last_clean":"2024-09-10T09:53:00.794833+0000","last_became_active":"2024-09-10T09:52:22.317575+0000","last_became_peered":"2024-09-10T09:52:22.317575+0000","last_unstale":"2024-09-10T09:53:00.794833+0000","last_undegraded":"2024-09-10T09:53:00.794833+0000","last_fullsized":"2024-09-10T09:53:00.794833+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:17:51.446233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.795937+0000","last_change":"2024-09-10T09:52:22.327749+0000","last_active":"2024-09-10T09:53:00.795937+0000","last_peered":"2024-09-10T09:53:00.795937+0000","last_clean":"2024-09-10T09:53:00.795937+0000","last_became_active":"2024-09-10T09:52:22.327461+0000","last_became_peered":"2024-09-10T09:52:22.327461+0000","last_unstale":"2024-09-10T09:53:00.795937+0000","last_undegraded":"2024-09-10T09:53:00.795937+0000","last_fullsized":"2024-09-10T09:53:00.795937+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:54.208064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.794053+0000","last_change":"2024-09-10T09:52:22.326982+0000","last_active":"2024-09-10T09:53:00.794053+0000","last_peered":"2024-09-10T09:53:00.794053+0000","last_clean":"2024-09-10T09:53:00.794053+0000","last_became_active":"2024-09-10T09:52:22.326635+0000","last_became_peered":"2024-09-10T09:52:22.326635+0000","last_unstale":"2024-09-10T09:53:00.794053+0000","last_undegraded":"2024-09-10T09:53:00.794053+0000","last_fullsized":"2024-09-10T09:53:00.794053+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:50:54.835899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.844561+0000","last_change":"2024-09-10T09:52:22.323410+0000","last_active":"2024-09-10T09:53:00.844561+0000","last_peered":"2024-09-10T09:53:00.844561+0000","last_clean":"2024-09-10T09:53:00.844561+0000","last_became_active":"2024-09-10T09:52:22.323116+0000","last_became_peered":"2024-09-10T09:52:22.323116+0000","last_unstale":"2024-09-10T09:53:00.844561+0000","last_undegraded":"2024-09-10T09:53:00.844561+0000","last_fullsized":"2024-09-10T09:53:00.844561+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:09:15.115162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.844569+0000","last_change":"2024-09-10T09:52:22.324908+0000","last_active":"2024-09-10T09:53:00.844569+0000","last_peered":"2024-09-10T09:53:00.844569+0000","last_clean":"2024-09-10T09:53:00.844569+0000","last_became_active":"2024-09-10T09:52:22.324449+0000","last_became_peered":"2024-09-10T09:52:22.324449+0000","last_unstale":"2024-09-10T09:53:00.844569+0000","last_undegraded":"2024-09-10T09:53:00.844569+0000","last_fullsized":"2024-09-10T09:53:00.844569+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:43:24.535721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.796442+0000","last_change":"2024-09-10T09:52:22.322157+0000","last_active":"2024-09-10T09:53:00.796442+0000","last_peered":"2024-09-10T09:53:00.796442+0000","last_clean":"2024-09-10T09:53:00.796442+0000","last_became_active":"2024-09-10T09:52:22.321856+0000","last_became_peered":"2024-09-10T09:52:22.321856+0000","last_unstale":"2024-09-10T09:53:00.796442+0000","last_undegraded":"2024-09-10T09:53:00.796442+0000","last_fullsized":"2024-09-10T09:53:00.796442+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:36:38.395370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.801027+0000","last_change":"2024-09-10T09:53:00.800337+0000","last_active":"2024-09-10T09:53:00.801027+0000","last_peered":"2024-09-10T09:53:00.801027+0000","last_clean":"2024-09-10T09:53:00.801027+0000","last_became_active":"2024-09-10T09:53:00.799863+0000","last_became_peered":"2024-09-10T09:53:00.799863+0000","last_unstale":"2024-09-10T09:53:00.801027+0000","last_undegraded":"2024-09-10T09:53:00.801027+0000","last_fullsized":"2024-09-10T09:53:00.801027+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:18.017672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.797150+0000","last_change":"2024-09-10T09:52:22.310735+0000","last_active":"2024-09-10T09:53:00.797150+0000","last_peered":"2024-09-10T09:53:00.797150+0000","last_clean":"2024-09-10T09:53:00.797150+0000","last_became_active":"2024-09-10T09:52:22.310376+0000","last_became_peered":"2024-09-10T09:52:22.310376+0000","last_unstale":"2024-09-10T09:53:00.797150+0000","last_undegraded":"2024-09-10T09:53:00.797150+0000","last_fullsized":"2024-09-10T09:53:00.797150+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:38:08.637588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.792545+0000","last_change":"2024-09-10T09:52:22.321113+0000","last_active":"2024-09-10T09:53:00.792545+0000","last_peered":"2024-09-10T09:53:00.792545+0000","last_clean":"2024-09-10T09:53:00.792545+0000","last_became_active":"2024-09-10T09:52:22.320767+0000","last_became_peered":"2024-09-10T09:52:22.320767+0000","last_unstale":"2024-09-10T09:53:00.792545+0000","last_undegraded":"2024-09-10T09:53:00.792545+0000","last_fullsized":"2024-09-10T09:53:00.792545+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:37.549322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.794810+0000","last_change":"2024-09-10T09:52:22.318634+0000","last_active":"2024-09-10T09:53:00.794810+0000","last_peered":"2024-09-10T09:53:00.794810+0000","last_clean":"2024-09-10T09:53:00.794810+0000","last_became_active":"2024-09-10T09:52:22.318294+0000","last_became_peered":"2024-09-10T09:52:22.318294+0000","last_unstale":"2024-09-10T09:53:00.794810+0000","last_undegraded":"2024-09-10T09:53:00.794810+0000","last_fullsized":"2024-09-10T09:53:00.794810+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:19:31.262055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.791534+0000","last_change":"2024-09-10T09:52:22.324567+0000","last_active":"2024-09-10T09:53:00.791534+0000","last_peered":"2024-09-10T09:53:00.791534+0000","last_clean":"2024-09-10T09:53:00.791534+0000","last_became_active":"2024-09-10T09:52:22.321803+0000","last_became_peered":"2024-09-10T09:52:22.321803+0000","last_unstale":"2024-09-10T09:53:00.791534+0000","last_undegraded":"2024-09-10T09:53:00.791534+0000","last_fullsized":"2024-09-10T09:53:00.791534+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:08:11.099428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.796590+0000","last_change":"2024-09-10T09:52:22.318163+0000","last_active":"2024-09-10T09:53:00.796590+0000","last_peered":"2024-09-10T09:53:00.796590+0000","last_clean":"2024-09-10T09:53:00.796590+0000","last_became_active":"2024-09-10T09:52:22.317808+0000","last_became_peered":"2024-09-10T09:52:22.317808+0000","last_unstale":"2024-09-10T09:53:00.796590+0000","last_undegraded":"2024-09-10T09:53:00.796590+0000","last_fullsized":"2024-09-10T09:53:00.796590+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:19.214291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.791566+0000","last_change":"2024-09-10T09:52:22.323035+0000","last_active":"2024-09-10T09:53:00.791566+0000","last_peered":"2024-09-10T09:53:00.791566+0000","last_clean":"2024-09-10T09:53:00.791566+0000","last_became_active":"2024-09-10T09:52:22.322679+0000","last_became_peered":"2024-09-10T09:52:22.322679+0000","last_unstale":"2024-09-10T09:53:00.791566+0000","last_undegraded":"2024-09-10T09:53:00.791566+0000","last_fullsized":"2024-09-10T09:53:00.791566+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:23:13.762612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.797958+0000","last_change":"2024-09-10T09:52:22.315887+0000","last_active":"2024-09-10T09:53:00.797958+0000","last_peered":"2024-09-10T09:53:00.797958+0000","last_clean":"2024-09-10T09:53:00.797958+0000","last_became_active":"2024-09-10T09:52:22.315569+0000","last_became_peered":"2024-09-10T09:52:22.315569+0000","last_unstale":"2024-09-10T09:53:00.797958+0000","last_undegraded":"2024-09-10T09:53:00.797958+0000","last_fullsized":"2024-09-10T09:53:00.797958+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:46:57.094320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:01.217981+0000","last_change":"2024-09-10T09:52:22.314060+0000","last_active":"2024-09-10T09:53:01.217981+0000","last_peered":"2024-09-10T09:53:01.217981+0000","last_clean":"2024-09-10T09:53:01.217981+0000","last_became_active":"2024-09-10T09:52:22.313572+0000","last_became_peered":"2024-09-10T09:52:22.313572+0000","last_unstale":"2024-09-10T09:53:01.217981+0000","last_undegraded":"2024-09-10T09:53:01.217981+0000","last_fullsized":"2024-09-10T09:53:01.217981+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:11:17.158625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.844889+0000","last_change":"2024-09-10T09:52:22.327233+0000","last_active":"2024-09-10T09:53:00.844889+0000","last_peered":"2024-09-10T09:53:00.844889+0000","last_clean":"2024-09-10T09:53:00.844889+0000","last_became_active":"2024-09-10T09:52:22.326945+0000","last_became_peered":"2024-09-10T09:52:22.326945+0000","last_unstale":"2024-09-10T09:53:00.844889+0000","last_undegraded":"2024-09-10T09:53:00.844889+0000","last_fullsized":"2024-09-10T09:53:00.844889+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:31:13.490147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.844880+0000","last_change":"2024-09-10T09:52:22.330912+0000","last_active":"2024-09-10T09:53:00.844880+0000","last_peered":"2024-09-10T09:53:00.844880+0000","last_clean":"2024-09-10T09:53:00.844880+0000","last_became_active":"2024-09-10T09:52:22.330149+0000","last_became_peered":"2024-09-10T09:52:22.330149+0000","last_unstale":"2024-09-10T09:53:00.844880+0000","last_undegraded":"2024-09-10T09:53:00.844880+0000","last_fullsized":"2024-09-10T09:53:00.844880+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:10:48.842499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:53:00.795892+0000","last_change":"2024-09-10T09:52:22.327035+0000","last_active":"2024-09-10T09:53:00.795892+0000","last_peered":"2024-09-10T09:53:00.795892+0000","last_clean":"2024-09-10T09:53:00.795892+0000","last_became_active":"2024-09-10T09:52:22.326741+0000","last_became_peered":"2024-09-10T09:52:22.326741+0000","last_unstale":"2024-09-10T09:53:00.795892+0000","last_undegraded":"2024-09-10T09:53:00.795892+0000","last_fullsized":"2024-09-10T09:53:00.795892+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:21:05.026614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":63957,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":186,"num_read_kb":165,"num_write":238,"num_write_kb":132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":4426,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":524288,"data_stored":176569,"data_compressed":5661,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":207,"ondisk_log_size":207,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282980,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449116,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1048576,"data_stored":535507,"data_compressed":26618,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":70,"seq":300647710751,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32340,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712812,"statfs":{"total":95995035648,"available":95961919488,"internally_reserved":0,"allocated":1183744,"data_stored":1178877,"data_compressed":64497,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":65,"seq":279172874278,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32232,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712920,"statfs":{"total":95995035648,"available":95962030080,"internally_reserved":0,"allocated":1073152,"data_stored":538945,"data_compressed":26618,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135916,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32220,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712932,"statfs":{"total":95995035648,"available":95962042368,"internally_reserved":0,"allocated":1060864,"data_stored":538651,"data_compressed":26618,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364848,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28112,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717040,"statfs":{"total":95995035648,"available":95966248960,"internally_reserved":0,"allocated":1114112,"data_stored":1131531,"data_compressed":62610,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528374,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1044480,"data_stored":530353,"data_compressed":26618,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790013,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28120,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717032,"statfs":{"total":95995035648,"available":95966240768,"internally_reserved":0,"allocated":1122304,"data_stored":1123212,"data_compressed":62610,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953535,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1105920,"data_stored":583918,"data_compressed":28505,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182469,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32216,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712936,"statfs":{"total":95995035648,"available":95962046464,"internally_reserved":0,"allocated":1056768,"data_stored":530061,"data_compressed":26618,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411403,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36380,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708772,"statfs":{"total":95995035648,"available":95957782528,"internally_reserved":0,"allocated":1060864,"data_stored":531996,"data_compressed":26618,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574929,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1101824,"data_stored":581153,"data_compressed":28505,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738455,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1069056,"data_stored":531809,"data_compressed":26618,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34340,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34340,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34340,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35992,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35992,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35992,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35992,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35992,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":51522,"data_compressed":1887,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":50191,"data_compressed":1887,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":58878,"data_compressed":1887,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T09:53:36.212 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-10T09:53:36.213 INFO:tasks.cephadm:Setting up client nodes... 2024-09-10T09:53:36.213 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-10T09:53:36.575 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:53:37.031 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:36 smithi077 ceph-mon[30319]: from='client.24782 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T09:53:37.031 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:36 smithi077 ceph-mon[30319]: pgmap v342: 129 pgs: 129 active+clean; 639 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 3.4 KiB/s wr, 11 op/s 2024-09-10T09:53:37.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:36 smithi134 ceph-mon[35674]: from='client.24782 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T09:53:37.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:36 smithi134 ceph-mon[35674]: pgmap v342: 129 pgs: 129 active+clean; 639 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 3.4 KiB/s wr, 11 op/s 2024-09-10T09:53:37.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:37.148+0000 7fe3eae00640 1 -- 172.21.15.77:0/3602538786 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe3e4104e50 msgr2=0x7fe3e41052d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:37.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:37.148+0000 7fe3eae00640 1 --2- 172.21.15.77:0/3602538786 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe3e4104e50 0x7fe3e41052d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe3d0009900 tx=0x7fe3d002a3f0 comp rx=0 tx=0).stop 2024-09-10T09:53:37.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:37.148+0000 7fe3eae00640 1 -- 172.21.15.77:0/3602538786 shutdown_connections 2024-09-10T09:53:37.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:37.148+0000 7fe3eae00640 1 --2- 172.21.15.77:0/3602538786 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe3e4105810 0x7fe3e410c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:37.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:37.148+0000 7fe3eae00640 1 --2- 172.21.15.77:0/3602538786 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe3e4104e50 0x7fe3e41052d0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:37.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:37.148+0000 7fe3eae00640 1 --2- 172.21.15.77:0/3602538786 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe3e4103c50 0x7fe3e4104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:37.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:37.148+0000 7fe3eae00640 1 -- 172.21.15.77:0/3602538786 >> 172.21.15.77:0/3602538786 conn(0x7fe3e40ff3e0 msgr2=0x7fe3e4101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:37.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:37.148+0000 7fe3eae00640 1 -- 172.21.15.77:0/3602538786 shutdown_connections 2024-09-10T09:53:37.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:37.148+0000 7fe3eae00640 1 -- 172.21.15.77:0/3602538786 wait complete. 2024-09-10T09:53:37.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:37.149+0000 7fe3eae00640 1 Processor -- start 2024-09-10T09:53:37.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:37.149+0000 7fe3eae00640 1 -- start start 2024-09-10T09:53:37.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:37.149+0000 7fe3eae00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe3e4103c50 0x7fe3e4199f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:37.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:37.149+0000 7fe3eae00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe3e4104e50 0x7fe3e419a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:37.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:37.149+0000 7fe3eae00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe3e4105810 0x7fe3e41a14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:37.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:37.149+0000 7fe3eae00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e410f8b0 con 0x7fe3e4104e50 2024-09-10T09:53:37.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:37.149+0000 7fe3eae00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e410f730 con 0x7fe3e4105810 2024-09-10T09:53:37.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:37.149+0000 7fe3eae00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e410fa30 con 0x7fe3e4103c50 2024-09-10T09:53:37.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:37.150+0000 7fe3e9a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe3e4103c50 0x7fe3e4199f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:37.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:37.150+0000 7fe3e9000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe3e4104e50 0x7fe3e419a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:37.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:37.150+0000 7fe3e9a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe3e4103c50 0x7fe3e4199f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:46798/0 (socket says 172.21.15.77:46798) 2024-09-10T09:53:37.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:37.150+0000 7fe3e9a00640 1 -- 172.21.15.77:0/3970306198 learned_addr learned my addr 172.21.15.77:0/3970306198 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:53:37.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:37.150+0000 7fe3ea400640 1 --2- 172.21.15.77:0/3970306198 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe3e4105810 0x7fe3e41a14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:37.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:37.150+0000 7fe3e9a00640 1 -- 172.21.15.77:0/3970306198 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe3e4105810 msgr2=0x7fe3e41a14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:37.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:37.150+0000 7fe3e9a00640 1 --2- 172.21.15.77:0/3970306198 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe3e4105810 0x7fe3e41a14f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:37.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:37.150+0000 7fe3e9a00640 1 -- 172.21.15.77:0/3970306198 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe3e4104e50 msgr2=0x7fe3e419a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:37.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:37.150+0000 7fe3e9a00640 1 --2- 172.21.15.77:0/3970306198 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe3e4104e50 0x7fe3e419a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:37.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:37.150+0000 7fe3e9a00640 1 -- 172.21.15.77:0/3970306198 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3e41a1b60 con 0x7fe3e4103c50 2024-09-10T09:53:37.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:37.150+0000 7fe3e9a00640 1 --2- 172.21.15.77:0/3970306198 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe3e4103c50 0x7fe3e4199f30 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe3d800e9c0 tx=0x7fe3d800edc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:37.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:37.151+0000 7fe3e2000640 1 -- 172.21.15.77:0/3970306198 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d800ccc0 con 0x7fe3e4103c50 2024-09-10T09:53:37.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:37.151+0000 7fe3e2000640 1 -- 172.21.15.77:0/3970306198 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3d80035f0 con 0x7fe3e4103c50 2024-09-10T09:53:37.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:37.151+0000 7fe3eae00640 1 -- 172.21.15.77:0/3970306198 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3e41a1e50 con 0x7fe3e4103c50 2024-09-10T09:53:37.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:37.151+0000 7fe3e2000640 1 -- 172.21.15.77:0/3970306198 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d800ce60 con 0x7fe3e4103c50 2024-09-10T09:53:37.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:37.151+0000 7fe3eae00640 1 -- 172.21.15.77:0/3970306198 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3e41a22a0 con 0x7fe3e4103c50 2024-09-10T09:53:37.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:37.152+0000 7fe3eae00640 1 -- 172.21.15.77:0/3970306198 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3e410b0a0 con 0x7fe3e4103c50 2024-09-10T09:53:37.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:37.154+0000 7fe3e2000640 1 -- 172.21.15.77:0/3970306198 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fe3d80037a0 con 0x7fe3e4103c50 2024-09-10T09:53:37.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:37.155+0000 7fe3e2000640 1 --2- 172.21.15.77:0/3970306198 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fe3cc0a2c40 0x7fe3cc0a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:37.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:37.155+0000 7fe3e2000640 1 -- 172.21.15.77:0/3970306198 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7fe3d80cf8f0 con 0x7fe3e4103c50 2024-09-10T09:53:37.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:37.155+0000 7fe3e9000640 1 --2- 172.21.15.77:0/3970306198 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fe3cc0a2c40 0x7fe3cc0a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:37.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:37.156+0000 7fe3e9000640 1 --2- 172.21.15.77:0/3970306198 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fe3cc0a2c40 0x7fe3cc0a5100 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe3d00314e0 tx=0x7fe3d0000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:37.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:37.157+0000 7fe3e2000640 1 -- 172.21.15.77:0/3970306198 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe3d809c330 con 0x7fe3e4103c50 2024-09-10T09:53:37.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:36 smithi120 ceph-mon[34770]: from='client.24782 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T09:53:37.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:36 smithi120 ceph-mon[34770]: pgmap v342: 129 pgs: 129 active+clean; 639 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 3.4 KiB/s wr, 11 op/s 2024-09-10T09:53:37.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:37.394+0000 7fe3eae00640 1 -- 172.21.15.77:0/3970306198 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7fe3e402d7c0 con 0x7fe3e4103c50 2024-09-10T09:53:37.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:37.400+0000 7fe3e2000640 1 -- 172.21.15.77:0/3970306198 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7fe3d809bc30 con 0x7fe3e4103c50 2024-09-10T09:53:37.402 INFO:teuthology.orchestra.run.smithi077.stdout:[client.0] 2024-09-10T09:53:37.402 INFO:teuthology.orchestra.run.smithi077.stdout: key = AQAhF+BmjBioFxAA9uk58Ab0VO3eHS9piNVpPA== 2024-09-10T09:53:37.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:37.402+0000 7fe3eae00640 1 -- 172.21.15.77:0/3970306198 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fe3cc0a2c40 msgr2=0x7fe3cc0a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:37.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:37.402+0000 7fe3eae00640 1 --2- 172.21.15.77:0/3970306198 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fe3cc0a2c40 0x7fe3cc0a5100 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe3d00314e0 tx=0x7fe3d0000bc0 comp rx=0 tx=0).stop 2024-09-10T09:53:37.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:37.402+0000 7fe3eae00640 1 -- 172.21.15.77:0/3970306198 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe3e4103c50 msgr2=0x7fe3e4199f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:37.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:37.402+0000 7fe3eae00640 1 --2- 172.21.15.77:0/3970306198 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe3e4103c50 0x7fe3e4199f30 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe3d800e9c0 tx=0x7fe3d800edc0 comp rx=0 tx=0).stop 2024-09-10T09:53:37.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:37.402+0000 7fe3eae00640 1 -- 172.21.15.77:0/3970306198 shutdown_connections 2024-09-10T09:53:37.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:37.402+0000 7fe3eae00640 1 --2- 172.21.15.77:0/3970306198 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fe3cc0a2c40 0x7fe3cc0a5100 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:37.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:37.402+0000 7fe3eae00640 1 --2- 172.21.15.77:0/3970306198 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe3e4105810 0x7fe3e41a14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:37.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:37.402+0000 7fe3eae00640 1 --2- 172.21.15.77:0/3970306198 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe3e4104e50 0x7fe3e419a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:37.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:37.402+0000 7fe3eae00640 1 --2- 172.21.15.77:0/3970306198 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe3e4103c50 0x7fe3e4199f30 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:37.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:37.402+0000 7fe3eae00640 1 -- 172.21.15.77:0/3970306198 >> 172.21.15.77:0/3970306198 conn(0x7fe3e40ff3e0 msgr2=0x7fe3e4108270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:37.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:37.402+0000 7fe3eae00640 1 -- 172.21.15.77:0/3970306198 shutdown_connections 2024-09-10T09:53:37.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:37.403+0000 7fe3eae00640 1 -- 172.21.15.77:0/3970306198 wait complete. 2024-09-10T09:53:38.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:37 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/3970306198' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T09:53:38.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:37 smithi134 ceph-mon[35674]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T09:53:38.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:37 smithi134 ceph-mon[35674]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T09:53:38.136 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:37 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/3970306198' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T09:53:38.136 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:37 smithi077 ceph-mon[30319]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T09:53:38.136 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:37 smithi077 ceph-mon[30319]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T09:53:38.174 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-09-10T09:53:38.174 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-10T09:53:38.174 DEBUG:teuthology.orchestra.run.smithi077:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-10T09:53:38.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:37 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/3970306198' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T09:53:38.175 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:37 smithi120 ceph-mon[34770]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T09:53:38.175 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:37 smithi120 ceph-mon[34770]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T09:53:38.255 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-10T09:53:38.255 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-10T09:53:38.255 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph mgr dump --format=json 2024-09-10T09:53:38.660 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:53:39.126 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:38 smithi077 ceph-mon[30319]: pgmap v343: 129 pgs: 129 active+clean; 639 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.3 KiB/s wr, 10 op/s 2024-09-10T09:53:39.130 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:38 smithi134 ceph-mon[35674]: pgmap v343: 129 pgs: 129 active+clean; 639 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.3 KiB/s wr, 10 op/s 2024-09-10T09:53:39.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:38 smithi120 ceph-mon[34770]: pgmap v343: 129 pgs: 129 active+clean; 639 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.3 KiB/s wr, 10 op/s 2024-09-10T09:53:39.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:39.234+0000 7fa23f600640 1 -- 172.21.15.77:0/3763318169 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa238102790 msgr2=0x7fa2381067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:39.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:39.234+0000 7fa23f600640 1 --2- 172.21.15.77:0/3763318169 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa238102790 0x7fa2381067e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa224009860 tx=0x7fa22402a340 comp rx=0 tx=0).stop 2024-09-10T09:53:39.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:39.234+0000 7fa23f600640 1 -- 172.21.15.77:0/3763318169 shutdown_connections 2024-09-10T09:53:39.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:39.234+0000 7fa23f600640 1 --2- 172.21.15.77:0/3763318169 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa238106f10 0x7fa23810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:39.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:39.234+0000 7fa23f600640 1 --2- 172.21.15.77:0/3763318169 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa238102790 0x7fa2381067e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:39.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:39.234+0000 7fa23f600640 1 --2- 172.21.15.77:0/3763318169 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa238101de0 0x7fa2381021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:39.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:39.234+0000 7fa23f600640 1 -- 172.21.15.77:0/3763318169 >> 172.21.15.77:0/3763318169 conn(0x7fa2380fd8c0 msgr2=0x7fa2380ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:39.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:39.234+0000 7fa23f600640 1 -- 172.21.15.77:0/3763318169 shutdown_connections 2024-09-10T09:53:39.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:39.234+0000 7fa23f600640 1 -- 172.21.15.77:0/3763318169 wait complete. 2024-09-10T09:53:39.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:39.235+0000 7fa23f600640 1 Processor -- start 2024-09-10T09:53:39.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:39.235+0000 7fa23f600640 1 -- start start 2024-09-10T09:53:39.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:39.235+0000 7fa23f600640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa238101de0 0x7fa238198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:39.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:39.235+0000 7fa23f600640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa238102790 0x7fa238198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:39.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:39.235+0000 7fa23f600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa238106f10 0x7fa23819cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:39.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:39.235+0000 7fa23f600640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa238111f70 con 0x7fa238102790 2024-09-10T09:53:39.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:39.235+0000 7fa23f600640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa238111df0 con 0x7fa238101de0 2024-09-10T09:53:39.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:39.235+0000 7fa23f600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa2381120f0 con 0x7fa238106f10 2024-09-10T09:53:39.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:39.236+0000 7fa237e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa238102790 0x7fa238198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:39.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:39.236+0000 7fa237e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa238102790 0x7fa238198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42412/0 (socket says 172.21.15.77:42412) 2024-09-10T09:53:39.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:39.236+0000 7fa237e00640 1 -- 172.21.15.77:0/736268079 learned_addr learned my addr 172.21.15.77:0/736268079 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:53:39.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:39.236+0000 7fa23cc00640 1 --2- 172.21.15.77:0/736268079 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa238101de0 0x7fa238198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:39.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:39.236+0000 7fa23d600640 1 --2- 172.21.15.77:0/736268079 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa238106f10 0x7fa23819cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:39.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:39.236+0000 7fa23cc00640 1 -- 172.21.15.77:0/736268079 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa238106f10 msgr2=0x7fa23819cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:39.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:39.236+0000 7fa23cc00640 1 --2- 172.21.15.77:0/736268079 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa238106f10 0x7fa23819cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:39.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:39.236+0000 7fa23cc00640 1 -- 172.21.15.77:0/736268079 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa238102790 msgr2=0x7fa238198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:39.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:39.236+0000 7fa23cc00640 1 --2- 172.21.15.77:0/736268079 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa238102790 0x7fa238198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:39.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:39.236+0000 7fa23cc00640 1 -- 172.21.15.77:0/736268079 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa23819d3e0 con 0x7fa238101de0 2024-09-10T09:53:39.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:39.236+0000 7fa237e00640 1 --2- 172.21.15.77:0/736268079 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa238102790 0x7fa238198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:53:39.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:39.237+0000 7fa23cc00640 1 --2- 172.21.15.77:0/736268079 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa238101de0 0x7fa238198430 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa22c00e9f0 tx=0x7fa22c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:39.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:39.237+0000 7fa235600640 1 -- 172.21.15.77:0/736268079 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa22c00ccc0 con 0x7fa238101de0 2024-09-10T09:53:39.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:39.237+0000 7fa235600640 1 -- 172.21.15.77:0/736268079 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa22c010910 con 0x7fa238101de0 2024-09-10T09:53:39.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:39.237+0000 7fa235600640 1 -- 172.21.15.77:0/736268079 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa22c018960 con 0x7fa238101de0 2024-09-10T09:53:39.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:39.237+0000 7fa23f600640 1 -- 172.21.15.77:0/736268079 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa23819d6d0 con 0x7fa238101de0 2024-09-10T09:53:39.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:39.237+0000 7fa23f600640 1 -- 172.21.15.77:0/736268079 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2381a4fb0 con 0x7fa238101de0 2024-09-10T09:53:39.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:39.238+0000 7fa23f600640 1 -- 172.21.15.77:0/736268079 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2381032c0 con 0x7fa238101de0 2024-09-10T09:53:39.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:39.240+0000 7fa235600640 1 -- 172.21.15.77:0/736268079 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fa22c010ac0 con 0x7fa238101de0 2024-09-10T09:53:39.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:39.241+0000 7fa235600640 1 --2- 172.21.15.77:0/736268079 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fa2200a2c10 0x7fa2200a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:39.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:39.242+0000 7fa235600640 1 -- 172.21.15.77:0/736268079 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7fa22c0cfde0 con 0x7fa238101de0 2024-09-10T09:53:39.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:39.242+0000 7fa237e00640 1 --2- 172.21.15.77:0/736268079 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fa2200a2c10 0x7fa2200a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:39.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:39.242+0000 7fa237e00640 1 --2- 172.21.15.77:0/736268079 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fa2200a2c10 0x7fa2200a50d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa2240057c0 tx=0x7fa2240056d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:39.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:39.243+0000 7fa235600640 1 -- 172.21.15.77:0/736268079 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa22c09c1d0 con 0x7fa238101de0 2024-09-10T09:53:39.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:39.467+0000 7fa23f600640 1 -- 172.21.15.77:0/736268079 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fa23810a150 con 0x7fa238101de0 2024-09-10T09:53:39.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:39.472+0000 7fa235600640 1 -- 172.21.15.77:0/736268079 <== mon.2 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+263200 (secure 0 0 0) 0x7fa22c0a04d0 con 0x7fa238101de0 2024-09-10T09:53:39.478 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:53:39.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:39.478+0000 7fa23f600640 1 -- 172.21.15.77:0/736268079 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fa2200a2c10 msgr2=0x7fa2200a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:39.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:39.478+0000 7fa23f600640 1 --2- 172.21.15.77:0/736268079 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fa2200a2c10 0x7fa2200a50d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa2240057c0 tx=0x7fa2240056d0 comp rx=0 tx=0).stop 2024-09-10T09:53:39.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:39.478+0000 7fa23f600640 1 -- 172.21.15.77:0/736268079 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa238101de0 msgr2=0x7fa238198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:39.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:39.478+0000 7fa23f600640 1 --2- 172.21.15.77:0/736268079 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa238101de0 0x7fa238198430 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa22c00e9f0 tx=0x7fa22c00eeb0 comp rx=0 tx=0).stop 2024-09-10T09:53:39.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:39.478+0000 7fa23f600640 1 -- 172.21.15.77:0/736268079 shutdown_connections 2024-09-10T09:53:39.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:39.478+0000 7fa23f600640 1 --2- 172.21.15.77:0/736268079 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fa2200a2c10 0x7fa2200a50d0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:39.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:39.478+0000 7fa23f600640 1 --2- 172.21.15.77:0/736268079 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa238106f10 0x7fa23819cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:39.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:39.478+0000 7fa23f600640 1 --2- 172.21.15.77:0/736268079 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa238102790 0x7fa238198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:39.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:39.478+0000 7fa23f600640 1 --2- 172.21.15.77:0/736268079 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa238101de0 0x7fa238198430 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:39.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:39.478+0000 7fa23f600640 1 -- 172.21.15.77:0/736268079 >> 172.21.15.77:0/736268079 conn(0x7fa2380fd8c0 msgr2=0x7fa2380fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:39.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:39.478+0000 7fa23f600640 1 -- 172.21.15.77:0/736268079 shutdown_connections 2024-09-10T09:53:39.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:39.478+0000 7fa23f600640 1 -- 172.21.15.77:0/736268079 wait complete. 2024-09-10T09:53:40.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:39 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/736268079' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-10T09:53:40.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:39 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/736268079' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-10T09:53:40.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:39 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/736268079' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-10T09:53:40.369 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6800","nonce":4261324650},{"type":"v1","addr":"172.21.15.77:6801","nonce":4261324650}]},"active_addr":"172.21.15.77:6801/4261324650","active_change":"2024-09-10T09:44:59.575771+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.77:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.77:0","nonce":1084675112}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.77:0","nonce":1630899152}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.77:0","nonce":589729730}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.77:0","nonce":2998673251}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.77:0","nonce":1063494935}]}]} 2024-09-10T09:53:40.375 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-10T09:53:40.375 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-10T09:53:40.376 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T09:53:40.741 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:53:41.047 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:40 smithi077 ceph-mon[30319]: pgmap v344: 129 pgs: 129 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 3.7 KiB/s wr, 14 op/s 2024-09-10T09:53:41.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:40 smithi134 ceph-mon[35674]: pgmap v344: 129 pgs: 129 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 3.7 KiB/s wr, 14 op/s 2024-09-10T09:53:41.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:40 smithi120 ceph-mon[34770]: pgmap v344: 129 pgs: 129 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 3.7 KiB/s wr, 14 op/s 2024-09-10T09:53:41.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:41.426+0000 7fe540e00640 1 -- 172.21.15.77:0/3097267823 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe53c1049a0 msgr2=0x7fe53c1089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:41.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:41.426+0000 7fe540e00640 1 --2- 172.21.15.77:0/3097267823 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe53c1049a0 0x7fe53c1089f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe524009900 tx=0x7fe52402a3f0 comp rx=0 tx=0).stop 2024-09-10T09:53:41.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:41.426+0000 7fe540e00640 1 -- 172.21.15.77:0/3097267823 shutdown_connections 2024-09-10T09:53:41.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:41.426+0000 7fe540e00640 1 --2- 172.21.15.77:0/3097267823 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe53c109120 0x7fe53c110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:41.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:41.426+0000 7fe540e00640 1 --2- 172.21.15.77:0/3097267823 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe53c1049a0 0x7fe53c1089f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:41.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:41.426+0000 7fe540e00640 1 --2- 172.21.15.77:0/3097267823 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe53c103ff0 0x7fe53c1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:41.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:41.426+0000 7fe540e00640 1 -- 172.21.15.77:0/3097267823 >> 172.21.15.77:0/3097267823 conn(0x7fe53c0ffa90 msgr2=0x7fe53c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:41.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:41.427+0000 7fe540e00640 1 -- 172.21.15.77:0/3097267823 shutdown_connections 2024-09-10T09:53:41.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:41.427+0000 7fe540e00640 1 -- 172.21.15.77:0/3097267823 wait complete. 2024-09-10T09:53:41.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:41.427+0000 7fe540e00640 1 Processor -- start 2024-09-10T09:53:41.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:41.428+0000 7fe540e00640 1 -- start start 2024-09-10T09:53:41.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:41.428+0000 7fe540e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe53c103ff0 0x7fe53c19a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:41.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:41.428+0000 7fe540e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe53c1049a0 0x7fe53c19ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:41.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:41.428+0000 7fe540e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe53c109120 0x7fe53c19eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:41.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:41.428+0000 7fe540e00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe53c114190 con 0x7fe53c103ff0 2024-09-10T09:53:41.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:41.428+0000 7fe540e00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe53c114010 con 0x7fe53c1049a0 2024-09-10T09:53:41.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:41.428+0000 7fe540e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe53c114310 con 0x7fe53c109120 2024-09-10T09:53:41.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:41.428+0000 7fe539e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe53c103ff0 0x7fe53c19a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:41.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:41.428+0000 7fe539e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe53c103ff0 0x7fe53c19a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42432/0 (socket says 172.21.15.77:42432) 2024-09-10T09:53:41.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:41.428+0000 7fe539e00640 1 -- 172.21.15.77:0/3840972533 learned_addr learned my addr 172.21.15.77:0/3840972533 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:53:41.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:41.428+0000 7fe53a800640 1 --2- 172.21.15.77:0/3840972533 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe53c109120 0x7fe53c19eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:41.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:41.428+0000 7fe531400640 1 --2- 172.21.15.77:0/3840972533 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe53c1049a0 0x7fe53c19ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:41.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:41.429+0000 7fe539e00640 1 -- 172.21.15.77:0/3840972533 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe53c109120 msgr2=0x7fe53c19eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:41.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:41.429+0000 7fe539e00640 1 --2- 172.21.15.77:0/3840972533 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe53c109120 0x7fe53c19eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:41.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:41.429+0000 7fe539e00640 1 -- 172.21.15.77:0/3840972533 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe53c1049a0 msgr2=0x7fe53c19ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:41.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:41.429+0000 7fe539e00640 1 --2- 172.21.15.77:0/3840972533 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe53c1049a0 0x7fe53c19ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:41.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:41.429+0000 7fe539e00640 1 -- 172.21.15.77:0/3840972533 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe53c19f5a0 con 0x7fe53c103ff0 2024-09-10T09:53:41.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:41.429+0000 7fe539e00640 1 --2- 172.21.15.77:0/3840972533 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe53c103ff0 0x7fe53c19a5f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe52c00d950 tx=0x7fe52c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:41.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:41.429+0000 7fe533400640 1 -- 172.21.15.77:0/3840972533 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe52c017070 con 0x7fe53c103ff0 2024-09-10T09:53:41.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:41.429+0000 7fe533400640 1 -- 172.21.15.77:0/3840972533 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe52c009c30 con 0x7fe53c103ff0 2024-09-10T09:53:41.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:41.429+0000 7fe540e00640 1 -- 172.21.15.77:0/3840972533 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe53c19f890 con 0x7fe53c103ff0 2024-09-10T09:53:41.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:41.429+0000 7fe533400640 1 -- 172.21.15.77:0/3840972533 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe52c00f6b0 con 0x7fe53c103ff0 2024-09-10T09:53:41.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:41.429+0000 7fe540e00640 1 -- 172.21.15.77:0/3840972533 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe53c1a7170 con 0x7fe53c103ff0 2024-09-10T09:53:41.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:41.430+0000 7fe540e00640 1 -- 172.21.15.77:0/3840972533 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe53c1043d0 con 0x7fe53c103ff0 2024-09-10T09:53:41.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:41.432+0000 7fe533400640 1 -- 172.21.15.77:0/3840972533 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fe52c0109d0 con 0x7fe53c103ff0 2024-09-10T09:53:41.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:41.434+0000 7fe533400640 1 --2- 172.21.15.77:0/3840972533 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fe5200a2c10 0x7fe5200a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:41.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:41.434+0000 7fe533400640 1 -- 172.21.15.77:0/3840972533 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7fe52c0cf170 con 0x7fe53c103ff0 2024-09-10T09:53:41.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:41.435+0000 7fe531400640 1 --2- 172.21.15.77:0/3840972533 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fe5200a2c10 0x7fe5200a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:41.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:41.435+0000 7fe531400640 1 --2- 172.21.15.77:0/3840972533 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fe5200a2c10 0x7fe5200a50d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe524000c00 tx=0x7fe524005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:41.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:41.439+0000 7fe533400640 1 -- 172.21.15.77:0/3840972533 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe52c009de0 con 0x7fe53c103ff0 2024-09-10T09:53:41.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:41.614+0000 7fe540e00640 1 -- 172.21.15.77:0/3840972533 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe53c1a73e0 con 0x7fe53c103ff0 2024-09-10T09:53:41.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:41.616+0000 7fe533400640 1 -- 172.21.15.77:0/3840972533 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24707 (secure 0 0 0) 0x7fe52c09b560 con 0x7fe53c103ff0 2024-09-10T09:53:41.617 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:53:41.617 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":94,"fsid":"14a55226-6f59-11ef-bcea-c7b262605968","created":"2024-09-10T09:44:07.359542+0000","modified":"2024-09-10T09:53:00.787197+0000","last_up_change":"2024-09-10T09:51:26.731971+0000","last_in_change":"2024-09-10T09:51:03.857873+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:47:24.174660+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T09:52:18.766738+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T09:52:21.284745+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T09:52:29.846594+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8d46f634-114c-4926-afdc-f8fa37e40336","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":1509649079},{"type":"v1","addr":"172.21.15.77:6803","nonce":1509649079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":1509649079},{"type":"v1","addr":"172.21.15.77:6805","nonce":1509649079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":1509649079},{"type":"v1","addr":"172.21.15.77:6809","nonce":1509649079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":1509649079},{"type":"v1","addr":"172.21.15.77:6807","nonce":1509649079}]},"public_addr":"172.21.15.77:6803/1509649079","cluster_addr":"172.21.15.77:6805/1509649079","heartbeat_back_addr":"172.21.15.77:6809/1509649079","heartbeat_front_addr":"172.21.15.77:6807/1509649079","state":["exists","up"]},{"osd":1,"uuid":"b6570764-736a-49e4-877a-1965e5812d1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":4243920698},{"type":"v1","addr":"172.21.15.120:6801","nonce":4243920698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":4243920698},{"type":"v1","addr":"172.21.15.120:6803","nonce":4243920698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":4243920698},{"type":"v1","addr":"172.21.15.120:6807","nonce":4243920698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":4243920698},{"type":"v1","addr":"172.21.15.120:6805","nonce":4243920698}]},"public_addr":"172.21.15.120:6801/4243920698","cluster_addr":"172.21.15.120:6803/4243920698","heartbeat_back_addr":"172.21.15.120:6807/4243920698","heartbeat_front_addr":"172.21.15.120:6805/4243920698","state":["exists","up"]},{"osd":2,"uuid":"5e83fe13-d1ee-483e-a575-e8041ec836f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":3280466823},{"type":"v1","addr":"172.21.15.134:6801","nonce":3280466823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":3280466823},{"type":"v1","addr":"172.21.15.134:6803","nonce":3280466823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":3280466823},{"type":"v1","addr":"172.21.15.134:6807","nonce":3280466823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":3280466823},{"type":"v1","addr":"172.21.15.134:6805","nonce":3280466823}]},"public_addr":"172.21.15.134:6801/3280466823","cluster_addr":"172.21.15.134:6803/3280466823","heartbeat_back_addr":"172.21.15.134:6807/3280466823","heartbeat_front_addr":"172.21.15.134:6805/3280466823","state":["exists","up"]},{"osd":3,"uuid":"5affc6a7-1cae-48fe-b7db-befc2fcafecb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":1629416786},{"type":"v1","addr":"172.21.15.77:6811","nonce":1629416786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":1629416786},{"type":"v1","addr":"172.21.15.77:6813","nonce":1629416786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":1629416786},{"type":"v1","addr":"172.21.15.77:6817","nonce":1629416786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":1629416786},{"type":"v1","addr":"172.21.15.77:6815","nonce":1629416786}]},"public_addr":"172.21.15.77:6811/1629416786","cluster_addr":"172.21.15.77:6813/1629416786","heartbeat_back_addr":"172.21.15.77:6817/1629416786","heartbeat_front_addr":"172.21.15.77:6815/1629416786","state":["exists","up"]},{"osd":4,"uuid":"c826978f-40b0-43ae-adbb-5c93b32a68ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":3162658038},{"type":"v1","addr":"172.21.15.120:6809","nonce":3162658038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":3162658038},{"type":"v1","addr":"172.21.15.120:6811","nonce":3162658038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":3162658038},{"type":"v1","addr":"172.21.15.120:6815","nonce":3162658038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":3162658038},{"type":"v1","addr":"172.21.15.120:6813","nonce":3162658038}]},"public_addr":"172.21.15.120:6809/3162658038","cluster_addr":"172.21.15.120:6811/3162658038","heartbeat_back_addr":"172.21.15.120:6815/3162658038","heartbeat_front_addr":"172.21.15.120:6813/3162658038","state":["exists","up"]},{"osd":5,"uuid":"be51c7fb-2765-46a3-9861-c950f63de148","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":4081444621},{"type":"v1","addr":"172.21.15.134:6809","nonce":4081444621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":4081444621},{"type":"v1","addr":"172.21.15.134:6811","nonce":4081444621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":4081444621},{"type":"v1","addr":"172.21.15.134:6815","nonce":4081444621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":4081444621},{"type":"v1","addr":"172.21.15.134:6813","nonce":4081444621}]},"public_addr":"172.21.15.134:6809/4081444621","cluster_addr":"172.21.15.134:6811/4081444621","heartbeat_back_addr":"172.21.15.134:6815/4081444621","heartbeat_front_addr":"172.21.15.134:6813/4081444621","state":["exists","up"]},{"osd":6,"uuid":"4eb403ab-473b-41cc-803d-18c908f768a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":879565639},{"type":"v1","addr":"172.21.15.77:6819","nonce":879565639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":879565639},{"type":"v1","addr":"172.21.15.77:6821","nonce":879565639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":879565639},{"type":"v1","addr":"172.21.15.77:6825","nonce":879565639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":879565639},{"type":"v1","addr":"172.21.15.77:6823","nonce":879565639}]},"public_addr":"172.21.15.77:6819/879565639","cluster_addr":"172.21.15.77:6821/879565639","heartbeat_back_addr":"172.21.15.77:6825/879565639","heartbeat_front_addr":"172.21.15.77:6823/879565639","state":["exists","up"]},{"osd":7,"uuid":"ba982bd0-c236-456c-9889-61087d79c1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":2688667400},{"type":"v1","addr":"172.21.15.120:6817","nonce":2688667400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":2688667400},{"type":"v1","addr":"172.21.15.120:6819","nonce":2688667400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":2688667400},{"type":"v1","addr":"172.21.15.120:6823","nonce":2688667400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2688667400},{"type":"v1","addr":"172.21.15.120:6821","nonce":2688667400}]},"public_addr":"172.21.15.120:6817/2688667400","cluster_addr":"172.21.15.120:6819/2688667400","heartbeat_back_addr":"172.21.15.120:6823/2688667400","heartbeat_front_addr":"172.21.15.120:6821/2688667400","state":["exists","up"]},{"osd":8,"uuid":"f2e8ca43-57c9-4933-b5f7-73f4d1b3e08b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2081686544},{"type":"v1","addr":"172.21.15.134:6817","nonce":2081686544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2081686544},{"type":"v1","addr":"172.21.15.134:6819","nonce":2081686544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2081686544},{"type":"v1","addr":"172.21.15.134:6823","nonce":2081686544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2081686544},{"type":"v1","addr":"172.21.15.134:6821","nonce":2081686544}]},"public_addr":"172.21.15.134:6817/2081686544","cluster_addr":"172.21.15.134:6819/2081686544","heartbeat_back_addr":"172.21.15.134:6823/2081686544","heartbeat_front_addr":"172.21.15.134:6821/2081686544","state":["exists","up"]},{"osd":9,"uuid":"9486f8b5-0614-4208-87ec-117741f6b622","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":1210619387},{"type":"v1","addr":"172.21.15.77:6827","nonce":1210619387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":1210619387},{"type":"v1","addr":"172.21.15.77:6829","nonce":1210619387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":1210619387},{"type":"v1","addr":"172.21.15.77:6833","nonce":1210619387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":1210619387},{"type":"v1","addr":"172.21.15.77:6831","nonce":1210619387}]},"public_addr":"172.21.15.77:6827/1210619387","cluster_addr":"172.21.15.77:6829/1210619387","heartbeat_back_addr":"172.21.15.77:6833/1210619387","heartbeat_front_addr":"172.21.15.77:6831/1210619387","state":["exists","up"]},{"osd":10,"uuid":"8e8192b0-1398-4d4d-9756-762b2f874d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":754733885},{"type":"v1","addr":"172.21.15.120:6825","nonce":754733885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":754733885},{"type":"v1","addr":"172.21.15.120:6827","nonce":754733885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":754733885},{"type":"v1","addr":"172.21.15.120:6831","nonce":754733885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":754733885},{"type":"v1","addr":"172.21.15.120:6829","nonce":754733885}]},"public_addr":"172.21.15.120:6825/754733885","cluster_addr":"172.21.15.120:6827/754733885","heartbeat_back_addr":"172.21.15.120:6831/754733885","heartbeat_front_addr":"172.21.15.120:6829/754733885","state":["exists","up"]},{"osd":11,"uuid":"85c1ccd4-5147-4995-896e-fe977f73f749","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1602213143},{"type":"v1","addr":"172.21.15.134:6825","nonce":1602213143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1602213143},{"type":"v1","addr":"172.21.15.134:6827","nonce":1602213143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1602213143},{"type":"v1","addr":"172.21.15.134:6831","nonce":1602213143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1602213143},{"type":"v1","addr":"172.21.15.134:6829","nonce":1602213143}]},"public_addr":"172.21.15.134:6825/1602213143","cluster_addr":"172.21.15.134:6827/1602213143","heartbeat_back_addr":"172.21.15.134:6831/1602213143","heartbeat_front_addr":"172.21.15.134:6829/1602213143","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:46:30.293451+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:46:56.554801+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:47:21.476225+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:47:49.737332+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:48:16.092181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:48:41.240815+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:49:11.683950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:49:38.859255+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:50:04.105401+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:50:33.847810+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:50:59.901797+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:51:25.241507+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.2","mappings":[{"from":7,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":5,"to":8}]},{"pgid":"4.6","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.77:0/1697982655":"2024-09-11T09:44:59.575663+0000","172.21.15.77:6801/1527070442":"2024-09-11T09:44:59.575663+0000","172.21.15.77:0/3146688196":"2024-09-11T09:44:32.185342+0000","172.21.15.77:0/371773177":"2024-09-11T09:44:59.575663+0000","172.21.15.77:0/1925456910":"2024-09-11T09:44:59.575663+0000","172.21.15.77:0/138545832":"2024-09-11T09:44:32.185342+0000","172.21.15.77:6801/3235900503":"2024-09-11T09:44:32.185342+0000","172.21.15.77:6800/1527070442":"2024-09-11T09:44:59.575663+0000","172.21.15.77:6800/3235900503":"2024-09-11T09:44:32.185342+0000","172.21.15.77:0/1647092548":"2024-09-11T09:44:32.185342+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T09:53:41.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:41.619+0000 7fe540e00640 1 -- 172.21.15.77:0/3840972533 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fe5200a2c10 msgr2=0x7fe5200a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:41.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:41.619+0000 7fe540e00640 1 --2- 172.21.15.77:0/3840972533 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fe5200a2c10 0x7fe5200a50d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe524000c00 tx=0x7fe524005b50 comp rx=0 tx=0).stop 2024-09-10T09:53:41.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:41.619+0000 7fe540e00640 1 -- 172.21.15.77:0/3840972533 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe53c103ff0 msgr2=0x7fe53c19a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:41.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:41.619+0000 7fe540e00640 1 --2- 172.21.15.77:0/3840972533 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe53c103ff0 0x7fe53c19a5f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe52c00d950 tx=0x7fe52c00de10 comp rx=0 tx=0).stop 2024-09-10T09:53:41.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:41.619+0000 7fe540e00640 1 -- 172.21.15.77:0/3840972533 shutdown_connections 2024-09-10T09:53:41.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:41.619+0000 7fe540e00640 1 --2- 172.21.15.77:0/3840972533 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fe5200a2c10 0x7fe5200a50d0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:41.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:41.619+0000 7fe540e00640 1 --2- 172.21.15.77:0/3840972533 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe53c109120 0x7fe53c19eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:41.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:41.619+0000 7fe540e00640 1 --2- 172.21.15.77:0/3840972533 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe53c1049a0 0x7fe53c19ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:41.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:41.619+0000 7fe540e00640 1 --2- 172.21.15.77:0/3840972533 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe53c103ff0 0x7fe53c19a5f0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:41.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:41.619+0000 7fe540e00640 1 -- 172.21.15.77:0/3840972533 >> 172.21.15.77:0/3840972533 conn(0x7fe53c0ffa90 msgr2=0x7fe53c101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:41.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:41.619+0000 7fe540e00640 1 -- 172.21.15.77:0/3840972533 shutdown_connections 2024-09-10T09:53:41.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:41.619+0000 7fe540e00640 1 -- 172.21.15.77:0/3840972533 wait complete. 2024-09-10T09:53:41.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:41 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/3840972533' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T09:53:42.075 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:41 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/3840972533' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T09:53:42.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:41 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/3840972533' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T09:53:42.464 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-10T09:53:42.464 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T09:53:42.834 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:53:43.091 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:42 smithi120 ceph-mon[34770]: pgmap v345: 129 pgs: 129 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-09-10T09:53:43.092 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:42 smithi077 ceph-mon[30319]: pgmap v345: 129 pgs: 129 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-09-10T09:53:43.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:42 smithi134 ceph-mon[35674]: pgmap v345: 129 pgs: 129 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-09-10T09:53:43.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:43.362+0000 7f381ce00640 1 -- 172.21.15.77:0/3416770901 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3818106f10 msgr2=0x7f381810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:43.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:43.362+0000 7f381ce00640 1 --2- 172.21.15.77:0/3416770901 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3818106f10 0x7f381810ea90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3810009900 tx=0x7f381002a3f0 comp rx=0 tx=0).stop 2024-09-10T09:53:43.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:43.362+0000 7f381ce00640 1 -- 172.21.15.77:0/3416770901 shutdown_connections 2024-09-10T09:53:43.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:43.362+0000 7f381ce00640 1 --2- 172.21.15.77:0/3416770901 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3818106f10 0x7f381810ea90 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:43.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:43.362+0000 7f381ce00640 1 --2- 172.21.15.77:0/3416770901 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3818102790 0x7f38181067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:43.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:43.362+0000 7f381ce00640 1 --2- 172.21.15.77:0/3416770901 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3818101de0 0x7f38181021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:43.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:43.362+0000 7f381ce00640 1 -- 172.21.15.77:0/3416770901 >> 172.21.15.77:0/3416770901 conn(0x7f38180fd8c0 msgr2=0x7f38180ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:43.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:43.362+0000 7f381ce00640 1 -- 172.21.15.77:0/3416770901 shutdown_connections 2024-09-10T09:53:43.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:43.362+0000 7f381ce00640 1 -- 172.21.15.77:0/3416770901 wait complete. 2024-09-10T09:53:43.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:43.363+0000 7f381ce00640 1 Processor -- start 2024-09-10T09:53:43.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:43.363+0000 7f381ce00640 1 -- start start 2024-09-10T09:53:43.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:43.363+0000 7f381ce00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3818101de0 0x7f381819a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:43.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:43.363+0000 7f381ce00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3818102790 0x7f381819ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:43.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:43.363+0000 7f381ce00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3818106f10 0x7f381819eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:43.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:43.363+0000 7f381ce00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3818111f70 con 0x7f3818102790 2024-09-10T09:53:43.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:43.363+0000 7f381ce00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3818111df0 con 0x7f3818106f10 2024-09-10T09:53:43.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:43.363+0000 7f381ce00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f38181120f0 con 0x7f3818101de0 2024-09-10T09:53:43.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:43.364+0000 7f3815400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3818102790 0x7f381819ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:43.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:43.364+0000 7f3816800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3818106f10 0x7f381819eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:43.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:43.364+0000 7f3815400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3818102790 0x7f381819ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:53124/0 (socket says 172.21.15.77:53124) 2024-09-10T09:53:43.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:43.364+0000 7f3815400640 1 -- 172.21.15.77:0/2539698227 learned_addr learned my addr 172.21.15.77:0/2539698227 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:53:43.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:43.364+0000 7f3815e00640 1 --2- 172.21.15.77:0/2539698227 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3818101de0 0x7f381819a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:43.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:43.364+0000 7f3815400640 1 -- 172.21.15.77:0/2539698227 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3818101de0 msgr2=0x7f381819a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:43.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:43.364+0000 7f3815400640 1 --2- 172.21.15.77:0/2539698227 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3818101de0 0x7f381819a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:43.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:43.364+0000 7f3815400640 1 -- 172.21.15.77:0/2539698227 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3818106f10 msgr2=0x7f381819eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:43.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:43.364+0000 7f3815400640 1 --2- 172.21.15.77:0/2539698227 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3818106f10 0x7f381819eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:43.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:43.364+0000 7f3815400640 1 -- 172.21.15.77:0/2539698227 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f381819f5a0 con 0x7f3818102790 2024-09-10T09:53:43.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:43.364+0000 7f3816800640 1 --2- 172.21.15.77:0/2539698227 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3818106f10 0x7f381819eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:53:43.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:43.364+0000 7f3815400640 1 --2- 172.21.15.77:0/2539698227 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3818102790 0x7f381819ab30 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f380400e9f0 tx=0x7f380400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:43.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:43.365+0000 7f380ea00640 1 -- 172.21.15.77:0/2539698227 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f380400cd80 con 0x7f3818102790 2024-09-10T09:53:43.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:43.365+0000 7f380ea00640 1 -- 172.21.15.77:0/2539698227 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3804010910 con 0x7f3818102790 2024-09-10T09:53:43.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:43.365+0000 7f381ce00640 1 -- 172.21.15.77:0/2539698227 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f381819f890 con 0x7f3818102790 2024-09-10T09:53:43.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:43.365+0000 7f381ce00640 1 -- 172.21.15.77:0/2539698227 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38181a7170 con 0x7f3818102790 2024-09-10T09:53:43.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:43.365+0000 7f380ea00640 1 -- 172.21.15.77:0/2539698227 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3804018990 con 0x7f3818102790 2024-09-10T09:53:43.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:43.366+0000 7f381ce00640 1 -- 172.21.15.77:0/2539698227 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37f8005290 con 0x7f3818102790 2024-09-10T09:53:43.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:43.369+0000 7f380ea00640 1 -- 172.21.15.77:0/2539698227 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f3804018b30 con 0x7f3818102790 2024-09-10T09:53:43.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:43.370+0000 7f380ea00640 1 --2- 172.21.15.77:0/2539698227 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f37fc0a2c10 0x7f37fc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:43.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:43.370+0000 7f380ea00640 1 -- 172.21.15.77:0/2539698227 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f38040cf680 con 0x7f3818102790 2024-09-10T09:53:43.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:43.371+0000 7f3815e00640 1 --2- 172.21.15.77:0/2539698227 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f37fc0a2c10 0x7f37fc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:43.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:43.372+0000 7f3815e00640 1 --2- 172.21.15.77:0/2539698227 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f37fc0a2c10 0x7f37fc0a50d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3808005fd0 tx=0x7f3808009680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:43.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:43.375+0000 7f380ea00640 1 -- 172.21.15.77:0/2539698227 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f380409ba70 con 0x7f3818102790 2024-09-10T09:53:43.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:43.551+0000 7f381ce00640 1 -- 172.21.15.77:0/2539698227 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f37f8005850 con 0x7f3818102790 2024-09-10T09:53:43.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:43.552+0000 7f380ea00640 1 -- 172.21.15.77:0/2539698227 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24707 (secure 0 0 0) 0x7f380409fd70 con 0x7f3818102790 2024-09-10T09:53:43.554 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:53:43.554 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":94,"fsid":"14a55226-6f59-11ef-bcea-c7b262605968","created":"2024-09-10T09:44:07.359542+0000","modified":"2024-09-10T09:53:00.787197+0000","last_up_change":"2024-09-10T09:51:26.731971+0000","last_in_change":"2024-09-10T09:51:03.857873+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:47:24.174660+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T09:52:18.766738+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T09:52:21.284745+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T09:52:29.846594+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8d46f634-114c-4926-afdc-f8fa37e40336","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":1509649079},{"type":"v1","addr":"172.21.15.77:6803","nonce":1509649079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":1509649079},{"type":"v1","addr":"172.21.15.77:6805","nonce":1509649079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":1509649079},{"type":"v1","addr":"172.21.15.77:6809","nonce":1509649079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":1509649079},{"type":"v1","addr":"172.21.15.77:6807","nonce":1509649079}]},"public_addr":"172.21.15.77:6803/1509649079","cluster_addr":"172.21.15.77:6805/1509649079","heartbeat_back_addr":"172.21.15.77:6809/1509649079","heartbeat_front_addr":"172.21.15.77:6807/1509649079","state":["exists","up"]},{"osd":1,"uuid":"b6570764-736a-49e4-877a-1965e5812d1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":4243920698},{"type":"v1","addr":"172.21.15.120:6801","nonce":4243920698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":4243920698},{"type":"v1","addr":"172.21.15.120:6803","nonce":4243920698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":4243920698},{"type":"v1","addr":"172.21.15.120:6807","nonce":4243920698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":4243920698},{"type":"v1","addr":"172.21.15.120:6805","nonce":4243920698}]},"public_addr":"172.21.15.120:6801/4243920698","cluster_addr":"172.21.15.120:6803/4243920698","heartbeat_back_addr":"172.21.15.120:6807/4243920698","heartbeat_front_addr":"172.21.15.120:6805/4243920698","state":["exists","up"]},{"osd":2,"uuid":"5e83fe13-d1ee-483e-a575-e8041ec836f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":3280466823},{"type":"v1","addr":"172.21.15.134:6801","nonce":3280466823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":3280466823},{"type":"v1","addr":"172.21.15.134:6803","nonce":3280466823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":3280466823},{"type":"v1","addr":"172.21.15.134:6807","nonce":3280466823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":3280466823},{"type":"v1","addr":"172.21.15.134:6805","nonce":3280466823}]},"public_addr":"172.21.15.134:6801/3280466823","cluster_addr":"172.21.15.134:6803/3280466823","heartbeat_back_addr":"172.21.15.134:6807/3280466823","heartbeat_front_addr":"172.21.15.134:6805/3280466823","state":["exists","up"]},{"osd":3,"uuid":"5affc6a7-1cae-48fe-b7db-befc2fcafecb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":1629416786},{"type":"v1","addr":"172.21.15.77:6811","nonce":1629416786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":1629416786},{"type":"v1","addr":"172.21.15.77:6813","nonce":1629416786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":1629416786},{"type":"v1","addr":"172.21.15.77:6817","nonce":1629416786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":1629416786},{"type":"v1","addr":"172.21.15.77:6815","nonce":1629416786}]},"public_addr":"172.21.15.77:6811/1629416786","cluster_addr":"172.21.15.77:6813/1629416786","heartbeat_back_addr":"172.21.15.77:6817/1629416786","heartbeat_front_addr":"172.21.15.77:6815/1629416786","state":["exists","up"]},{"osd":4,"uuid":"c826978f-40b0-43ae-adbb-5c93b32a68ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":3162658038},{"type":"v1","addr":"172.21.15.120:6809","nonce":3162658038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":3162658038},{"type":"v1","addr":"172.21.15.120:6811","nonce":3162658038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":3162658038},{"type":"v1","addr":"172.21.15.120:6815","nonce":3162658038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":3162658038},{"type":"v1","addr":"172.21.15.120:6813","nonce":3162658038}]},"public_addr":"172.21.15.120:6809/3162658038","cluster_addr":"172.21.15.120:6811/3162658038","heartbeat_back_addr":"172.21.15.120:6815/3162658038","heartbeat_front_addr":"172.21.15.120:6813/3162658038","state":["exists","up"]},{"osd":5,"uuid":"be51c7fb-2765-46a3-9861-c950f63de148","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":4081444621},{"type":"v1","addr":"172.21.15.134:6809","nonce":4081444621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":4081444621},{"type":"v1","addr":"172.21.15.134:6811","nonce":4081444621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":4081444621},{"type":"v1","addr":"172.21.15.134:6815","nonce":4081444621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":4081444621},{"type":"v1","addr":"172.21.15.134:6813","nonce":4081444621}]},"public_addr":"172.21.15.134:6809/4081444621","cluster_addr":"172.21.15.134:6811/4081444621","heartbeat_back_addr":"172.21.15.134:6815/4081444621","heartbeat_front_addr":"172.21.15.134:6813/4081444621","state":["exists","up"]},{"osd":6,"uuid":"4eb403ab-473b-41cc-803d-18c908f768a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":879565639},{"type":"v1","addr":"172.21.15.77:6819","nonce":879565639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":879565639},{"type":"v1","addr":"172.21.15.77:6821","nonce":879565639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":879565639},{"type":"v1","addr":"172.21.15.77:6825","nonce":879565639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":879565639},{"type":"v1","addr":"172.21.15.77:6823","nonce":879565639}]},"public_addr":"172.21.15.77:6819/879565639","cluster_addr":"172.21.15.77:6821/879565639","heartbeat_back_addr":"172.21.15.77:6825/879565639","heartbeat_front_addr":"172.21.15.77:6823/879565639","state":["exists","up"]},{"osd":7,"uuid":"ba982bd0-c236-456c-9889-61087d79c1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":2688667400},{"type":"v1","addr":"172.21.15.120:6817","nonce":2688667400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":2688667400},{"type":"v1","addr":"172.21.15.120:6819","nonce":2688667400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":2688667400},{"type":"v1","addr":"172.21.15.120:6823","nonce":2688667400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2688667400},{"type":"v1","addr":"172.21.15.120:6821","nonce":2688667400}]},"public_addr":"172.21.15.120:6817/2688667400","cluster_addr":"172.21.15.120:6819/2688667400","heartbeat_back_addr":"172.21.15.120:6823/2688667400","heartbeat_front_addr":"172.21.15.120:6821/2688667400","state":["exists","up"]},{"osd":8,"uuid":"f2e8ca43-57c9-4933-b5f7-73f4d1b3e08b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2081686544},{"type":"v1","addr":"172.21.15.134:6817","nonce":2081686544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2081686544},{"type":"v1","addr":"172.21.15.134:6819","nonce":2081686544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2081686544},{"type":"v1","addr":"172.21.15.134:6823","nonce":2081686544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2081686544},{"type":"v1","addr":"172.21.15.134:6821","nonce":2081686544}]},"public_addr":"172.21.15.134:6817/2081686544","cluster_addr":"172.21.15.134:6819/2081686544","heartbeat_back_addr":"172.21.15.134:6823/2081686544","heartbeat_front_addr":"172.21.15.134:6821/2081686544","state":["exists","up"]},{"osd":9,"uuid":"9486f8b5-0614-4208-87ec-117741f6b622","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":1210619387},{"type":"v1","addr":"172.21.15.77:6827","nonce":1210619387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":1210619387},{"type":"v1","addr":"172.21.15.77:6829","nonce":1210619387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":1210619387},{"type":"v1","addr":"172.21.15.77:6833","nonce":1210619387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":1210619387},{"type":"v1","addr":"172.21.15.77:6831","nonce":1210619387}]},"public_addr":"172.21.15.77:6827/1210619387","cluster_addr":"172.21.15.77:6829/1210619387","heartbeat_back_addr":"172.21.15.77:6833/1210619387","heartbeat_front_addr":"172.21.15.77:6831/1210619387","state":["exists","up"]},{"osd":10,"uuid":"8e8192b0-1398-4d4d-9756-762b2f874d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":754733885},{"type":"v1","addr":"172.21.15.120:6825","nonce":754733885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":754733885},{"type":"v1","addr":"172.21.15.120:6827","nonce":754733885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":754733885},{"type":"v1","addr":"172.21.15.120:6831","nonce":754733885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":754733885},{"type":"v1","addr":"172.21.15.120:6829","nonce":754733885}]},"public_addr":"172.21.15.120:6825/754733885","cluster_addr":"172.21.15.120:6827/754733885","heartbeat_back_addr":"172.21.15.120:6831/754733885","heartbeat_front_addr":"172.21.15.120:6829/754733885","state":["exists","up"]},{"osd":11,"uuid":"85c1ccd4-5147-4995-896e-fe977f73f749","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1602213143},{"type":"v1","addr":"172.21.15.134:6825","nonce":1602213143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1602213143},{"type":"v1","addr":"172.21.15.134:6827","nonce":1602213143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1602213143},{"type":"v1","addr":"172.21.15.134:6831","nonce":1602213143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1602213143},{"type":"v1","addr":"172.21.15.134:6829","nonce":1602213143}]},"public_addr":"172.21.15.134:6825/1602213143","cluster_addr":"172.21.15.134:6827/1602213143","heartbeat_back_addr":"172.21.15.134:6831/1602213143","heartbeat_front_addr":"172.21.15.134:6829/1602213143","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:46:30.293451+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:46:56.554801+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:47:21.476225+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:47:49.737332+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:48:16.092181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:48:41.240815+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:49:11.683950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:49:38.859255+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:50:04.105401+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:50:33.847810+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:50:59.901797+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:51:25.241507+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.2","mappings":[{"from":7,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":5,"to":8}]},{"pgid":"4.6","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.77:0/1697982655":"2024-09-11T09:44:59.575663+0000","172.21.15.77:6801/1527070442":"2024-09-11T09:44:59.575663+0000","172.21.15.77:0/3146688196":"2024-09-11T09:44:32.185342+0000","172.21.15.77:0/371773177":"2024-09-11T09:44:59.575663+0000","172.21.15.77:0/1925456910":"2024-09-11T09:44:59.575663+0000","172.21.15.77:0/138545832":"2024-09-11T09:44:32.185342+0000","172.21.15.77:6801/3235900503":"2024-09-11T09:44:32.185342+0000","172.21.15.77:6800/1527070442":"2024-09-11T09:44:59.575663+0000","172.21.15.77:6800/3235900503":"2024-09-11T09:44:32.185342+0000","172.21.15.77:0/1647092548":"2024-09-11T09:44:32.185342+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T09:53:43.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:43.555+0000 7f381ce00640 1 -- 172.21.15.77:0/2539698227 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f37fc0a2c10 msgr2=0x7f37fc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:43.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:43.555+0000 7f381ce00640 1 --2- 172.21.15.77:0/2539698227 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f37fc0a2c10 0x7f37fc0a50d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3808005fd0 tx=0x7f3808009680 comp rx=0 tx=0).stop 2024-09-10T09:53:43.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:43.555+0000 7f381ce00640 1 -- 172.21.15.77:0/2539698227 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3818102790 msgr2=0x7f381819ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:43.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:43.555+0000 7f381ce00640 1 --2- 172.21.15.77:0/2539698227 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3818102790 0x7f381819ab30 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f380400e9f0 tx=0x7f380400eeb0 comp rx=0 tx=0).stop 2024-09-10T09:53:43.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:43.555+0000 7f381ce00640 1 -- 172.21.15.77:0/2539698227 shutdown_connections 2024-09-10T09:53:43.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:43.555+0000 7f381ce00640 1 --2- 172.21.15.77:0/2539698227 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f37fc0a2c10 0x7f37fc0a50d0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:43.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:43.555+0000 7f381ce00640 1 --2- 172.21.15.77:0/2539698227 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3818106f10 0x7f381819eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:43.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:43.555+0000 7f381ce00640 1 --2- 172.21.15.77:0/2539698227 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3818102790 0x7f381819ab30 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:43.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:43.555+0000 7f381ce00640 1 --2- 172.21.15.77:0/2539698227 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3818101de0 0x7f381819a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:43.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:43.555+0000 7f381ce00640 1 -- 172.21.15.77:0/2539698227 >> 172.21.15.77:0/2539698227 conn(0x7f38180fd8c0 msgr2=0x7f38180feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:43.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:43.555+0000 7f381ce00640 1 -- 172.21.15.77:0/2539698227 shutdown_connections 2024-09-10T09:53:43.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:43.555+0000 7f381ce00640 1 -- 172.21.15.77:0/2539698227 wait complete. 2024-09-10T09:53:43.964 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:43 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/2539698227' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T09:53:44.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:43 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/2539698227' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T09:53:44.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:43 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/2539698227' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T09:53:44.380 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-10T09:53:44.381 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-10T09:53:44.381 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-10T09:53:44.382 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-10T09:53:44.382 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-10T09:53:44.383 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-10T09:53:44.383 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-10T09:53:44.383 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-10T09:53:44.384 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-09-10T09:53:44.384 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-09-10T09:53:44.385 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-09-10T09:53:44.385 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-09-10T09:53:45.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:44 smithi134 ceph-mon[35674]: pgmap v346: 129 pgs: 129 active+clean; 645 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2024-09-10T09:53:45.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:44 smithi120 ceph-mon[34770]: pgmap v346: 129 pgs: 129 active+clean; 645 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2024-09-10T09:53:45.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:44 smithi077 ceph-mon[30319]: pgmap v346: 129 pgs: 129 active+clean; 645 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2024-09-10T09:53:45.249 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:53:45.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:45.818+0000 7ffa5a000640 1 -- 172.21.15.77:0/3584153563 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa54103ff0 msgr2=0x7ffa541043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:45.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:45.818+0000 7ffa5a000640 1 --2- 172.21.15.77:0/3584153563 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa54103ff0 0x7ffa541043d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ffa44009900 tx=0x7ffa4402a3f0 comp rx=0 tx=0).stop 2024-09-10T09:53:45.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:45.818+0000 7ffa5a000640 1 -- 172.21.15.77:0/3584153563 shutdown_connections 2024-09-10T09:53:45.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:45.818+0000 7ffa5a000640 1 --2- 172.21.15.77:0/3584153563 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa54109120 0x7ffa54110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:45.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:45.818+0000 7ffa5a000640 1 --2- 172.21.15.77:0/3584153563 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa541049a0 0x7ffa541089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:45.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:45.818+0000 7ffa5a000640 1 --2- 172.21.15.77:0/3584153563 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa54103ff0 0x7ffa541043d0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:45.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:45.818+0000 7ffa5a000640 1 -- 172.21.15.77:0/3584153563 >> 172.21.15.77:0/3584153563 conn(0x7ffa540ffa90 msgr2=0x7ffa54101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:45.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:45.818+0000 7ffa5a000640 1 -- 172.21.15.77:0/3584153563 shutdown_connections 2024-09-10T09:53:45.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:45.819+0000 7ffa5a000640 1 -- 172.21.15.77:0/3584153563 wait complete. 2024-09-10T09:53:45.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:45.819+0000 7ffa5a000640 1 Processor -- start 2024-09-10T09:53:45.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:45.820+0000 7ffa5a000640 1 -- start start 2024-09-10T09:53:45.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:45.820+0000 7ffa5a000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa54103ff0 0x7ffa5419a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:45.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:45.820+0000 7ffa5a000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa541049a0 0x7ffa5419ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:45.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:45.820+0000 7ffa5a000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa54109120 0x7ffa5419eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:45.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:45.820+0000 7ffa5a000640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ffa54114170 con 0x7ffa541049a0 2024-09-10T09:53:45.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:45.820+0000 7ffa5a000640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ffa54113ff0 con 0x7ffa54109120 2024-09-10T09:53:45.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:45.820+0000 7ffa5a000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ffa541142f0 con 0x7ffa54103ff0 2024-09-10T09:53:45.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:45.820+0000 7ffa52a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa541049a0 0x7ffa5419ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:45.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:45.820+0000 7ffa53400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa54103ff0 0x7ffa5419a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:45.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:45.820+0000 7ffa52a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa541049a0 0x7ffa5419ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:53142/0 (socket says 172.21.15.77:53142) 2024-09-10T09:53:45.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:45.820+0000 7ffa52a00640 1 -- 172.21.15.77:0/1074333158 learned_addr learned my addr 172.21.15.77:0/1074333158 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:53:45.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:45.821+0000 7ffa53e00640 1 --2- 172.21.15.77:0/1074333158 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa54109120 0x7ffa5419eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:45.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:45.821+0000 7ffa52a00640 1 -- 172.21.15.77:0/1074333158 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa54103ff0 msgr2=0x7ffa5419a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:45.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:45.821+0000 7ffa52a00640 1 --2- 172.21.15.77:0/1074333158 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa54103ff0 0x7ffa5419a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:45.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:45.821+0000 7ffa52a00640 1 -- 172.21.15.77:0/1074333158 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa54109120 msgr2=0x7ffa5419eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:45.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:45.821+0000 7ffa52a00640 1 --2- 172.21.15.77:0/1074333158 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa54109120 0x7ffa5419eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:45.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:45.821+0000 7ffa52a00640 1 -- 172.21.15.77:0/1074333158 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa5419f5b0 con 0x7ffa541049a0 2024-09-10T09:53:45.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:45.821+0000 7ffa53400640 1 --2- 172.21.15.77:0/1074333158 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa54103ff0 0x7ffa5419a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:53:45.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:45.821+0000 7ffa52a00640 1 --2- 172.21.15.77:0/1074333158 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa541049a0 0x7ffa5419ab40 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ffa4000e9f0 tx=0x7ffa4000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:45.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:45.821+0000 7ffa4be00640 1 -- 172.21.15.77:0/1074333158 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4000cd80 con 0x7ffa541049a0 2024-09-10T09:53:45.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:45.821+0000 7ffa4be00640 1 -- 172.21.15.77:0/1074333158 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa40010910 con 0x7ffa541049a0 2024-09-10T09:53:45.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:45.821+0000 7ffa4be00640 1 -- 172.21.15.77:0/1074333158 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa40018990 con 0x7ffa541049a0 2024-09-10T09:53:45.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:45.821+0000 7ffa5a000640 1 -- 172.21.15.77:0/1074333158 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa5419f8a0 con 0x7ffa541049a0 2024-09-10T09:53:45.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:45.822+0000 7ffa5a000640 1 -- 172.21.15.77:0/1074333158 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa541a7180 con 0x7ffa541049a0 2024-09-10T09:53:45.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:45.823+0000 7ffa5a000640 1 -- 172.21.15.77:0/1074333158 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ffa54062980 con 0x7ffa541049a0 2024-09-10T09:53:45.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:45.827+0000 7ffa4be00640 1 -- 172.21.15.77:0/1074333158 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ffa40010ac0 con 0x7ffa541049a0 2024-09-10T09:53:45.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:45.828+0000 7ffa4be00640 1 --2- 172.21.15.77:0/1074333158 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ffa380a2c90 0x7ffa380a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:45.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:45.828+0000 7ffa4be00640 1 -- 172.21.15.77:0/1074333158 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7ffa400cfec0 con 0x7ffa541049a0 2024-09-10T09:53:45.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:45.829+0000 7ffa53400640 1 --2- 172.21.15.77:0/1074333158 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ffa380a2c90 0x7ffa380a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:45.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:45.829+0000 7ffa4be00640 1 --2- 172.21.15.77:0/1074333158 >> [v2:172.21.15.120:6800/4243920698,v1:172.21.15.120:6801/4243920698] conn(0x7ffa380ade60 0x7ffa380b0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:45.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:45.829+0000 7ffa4be00640 1 -- 172.21.15.77:0/1074333158 --> [v2:172.21.15.120:6800/4243920698,v1:172.21.15.120:6801/4243920698] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ffa40019ec0 con 0x7ffa380ade60 2024-09-10T09:53:45.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:45.829+0000 7ffa4be00640 1 -- 172.21.15.77:0/1074333158 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7ffa400d0270 con 0x7ffa541049a0 2024-09-10T09:53:45.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:45.829+0000 7ffa53e00640 1 --2- 172.21.15.77:0/1074333158 >> [v2:172.21.15.120:6800/4243920698,v1:172.21.15.120:6801/4243920698] conn(0x7ffa380ade60 0x7ffa380b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:45.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:45.829+0000 7ffa53400640 1 --2- 172.21.15.77:0/1074333158 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ffa380a2c90 0x7ffa380a5150 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ffa44000c00 tx=0x7ffa44031510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:45.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:45.830+0000 7ffa53e00640 1 --2- 172.21.15.77:0/1074333158 >> [v2:172.21.15.120:6800/4243920698,v1:172.21.15.120:6801/4243920698] conn(0x7ffa380ade60 0x7ffa380b0300 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:45.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:45.831+0000 7ffa4be00640 1 -- 172.21.15.77:0/1074333158 <== osd.1 v2:172.21.15.120:6800/4243920698 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ffa40019ec0 con 0x7ffa380ade60 2024-09-10T09:53:45.845 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:45.845+0000 7ffa5a000640 1 -- 172.21.15.77:0/1074333158 --> [v2:172.21.15.120:6800/4243920698,v1:172.21.15.120:6801/4243920698] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ffa54062d80 con 0x7ffa380ade60 2024-09-10T09:53:45.846 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:45.845+0000 7ffa4be00640 1 -- 172.21.15.77:0/1074333158 <== osd.1 v2:172.21.15.120:6800/4243920698 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ffa54062d80 con 0x7ffa380ade60 2024-09-10T09:53:45.846 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:45.846+0000 7ffa5a000640 1 -- 172.21.15.77:0/1074333158 >> [v2:172.21.15.120:6800/4243920698,v1:172.21.15.120:6801/4243920698] conn(0x7ffa380ade60 msgr2=0x7ffa380b0300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:45.846 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:45.846+0000 7ffa5a000640 1 --2- 172.21.15.77:0/1074333158 >> [v2:172.21.15.120:6800/4243920698,v1:172.21.15.120:6801/4243920698] conn(0x7ffa380ade60 0x7ffa380b0300 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:45.847 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:45.846+0000 7ffa5a000640 1 -- 172.21.15.77:0/1074333158 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ffa380a2c90 msgr2=0x7ffa380a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:45.847 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:45.846+0000 7ffa5a000640 1 --2- 172.21.15.77:0/1074333158 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ffa380a2c90 0x7ffa380a5150 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ffa44000c00 tx=0x7ffa44031510 comp rx=0 tx=0).stop 2024-09-10T09:53:45.847 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:45.846+0000 7ffa5a000640 1 -- 172.21.15.77:0/1074333158 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa541049a0 msgr2=0x7ffa5419ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:45.847 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:45.846+0000 7ffa5a000640 1 --2- 172.21.15.77:0/1074333158 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa541049a0 0x7ffa5419ab40 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ffa4000e9f0 tx=0x7ffa4000eeb0 comp rx=0 tx=0).stop 2024-09-10T09:53:45.847 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:45.846+0000 7ffa53e00640 1 -- 172.21.15.77:0/1074333158 reap_dead start 2024-09-10T09:53:45.848 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:45.846+0000 7ffa5a000640 1 -- 172.21.15.77:0/1074333158 shutdown_connections 2024-09-10T09:53:45.848 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:45.846+0000 7ffa5a000640 1 -- 172.21.15.77:0/1074333158 >> 172.21.15.77:0/1074333158 conn(0x7ffa540ffa90 msgr2=0x7ffa54100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:45.848 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:45.846+0000 7ffa5a000640 1 -- 172.21.15.77:0/1074333158 shutdown_connections 2024-09-10T09:53:45.848 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:45.846+0000 7ffa5a000640 1 -- 172.21.15.77:0/1074333158 wait complete. 2024-09-10T09:53:46.782 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:53:46.782 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:53:46.786 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:53:46.804 INFO:teuthology.orchestra.run.smithi077.stdout:55834574933 2024-09-10T09:53:46.804 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-10T09:53:46.879 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:53:46.881 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:53:47.009 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:46 smithi077 ceph-mon[30319]: pgmap v347: 129 pgs: 129 active+clean; 645 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 10 op/s 2024-09-10T09:53:47.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:46 smithi134 ceph-mon[35674]: pgmap v347: 129 pgs: 129 active+clean; 645 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 10 op/s 2024-09-10T09:53:47.173 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:53:47.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:46 smithi120 ceph-mon[34770]: pgmap v347: 129 pgs: 129 active+clean; 645 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 10 op/s 2024-09-10T09:53:47.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:47.528+0000 7f7b1d600640 1 -- 172.21.15.77:0/467005139 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7b18101df0 msgr2=0x7f7b181021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:47.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:47.528+0000 7f7b1d600640 1 --2- 172.21.15.77:0/467005139 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7b18101df0 0x7f7b181021d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7b08009900 tx=0x7f7b0802a3f0 comp rx=0 tx=0).stop 2024-09-10T09:53:47.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:47.528+0000 7f7b1d600640 1 -- 172.21.15.77:0/467005139 shutdown_connections 2024-09-10T09:53:47.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:47.528+0000 7f7b1d600640 1 --2- 172.21.15.77:0/467005139 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7b18106f20 0x7f7b1810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:47.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:47.528+0000 7f7b1d600640 1 --2- 172.21.15.77:0/467005139 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b181027a0 0x7f7b181067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:47.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:47.528+0000 7f7b1d600640 1 --2- 172.21.15.77:0/467005139 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7b18101df0 0x7f7b181021d0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:47.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:47.528+0000 7f7b1d600640 1 -- 172.21.15.77:0/467005139 >> 172.21.15.77:0/467005139 conn(0x7f7b180fd8b0 msgr2=0x7f7b180ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:47.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:47.528+0000 7f7b1d600640 1 -- 172.21.15.77:0/467005139 shutdown_connections 2024-09-10T09:53:47.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:47.528+0000 7f7b1d600640 1 -- 172.21.15.77:0/467005139 wait complete. 2024-09-10T09:53:47.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:47.529+0000 7f7b1d600640 1 Processor -- start 2024-09-10T09:53:47.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:47.530+0000 7f7b1d600640 1 -- start start 2024-09-10T09:53:47.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:47.530+0000 7f7b1d600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b18101df0 0x7f7b1819a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:47.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:47.530+0000 7f7b1d600640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7b181027a0 0x7f7b1819ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:47.550 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:47.530+0000 7f7b1d600640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7b18106f20 0x7f7b1819eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:47.550 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:47.530+0000 7f7b1d600640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7b18111f80 con 0x7f7b18106f20 2024-09-10T09:53:47.550 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:47.530+0000 7f7b1d600640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7b18111e00 con 0x7f7b181027a0 2024-09-10T09:53:47.550 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:47.530+0000 7f7b1d600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7b18112100 con 0x7f7b18101df0 2024-09-10T09:53:47.550 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:47.530+0000 7f7b17400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7b18106f20 0x7f7b1819eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:47.550 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:47.530+0000 7f7b16a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b18101df0 0x7f7b1819a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:47.550 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:47.530+0000 7f7b17400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7b18106f20 0x7f7b1819eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:53162/0 (socket says 172.21.15.77:53162) 2024-09-10T09:53:47.550 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:47.530+0000 7f7b17400640 1 -- 172.21.15.77:0/3080873915 learned_addr learned my addr 172.21.15.77:0/3080873915 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:53:47.550 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:47.530+0000 7f7b16000640 1 --2- 172.21.15.77:0/3080873915 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7b181027a0 0x7f7b1819ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:47.550 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:47.531+0000 7f7b16a00640 1 -- 172.21.15.77:0/3080873915 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7b181027a0 msgr2=0x7f7b1819ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:47.550 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:47.531+0000 7f7b16a00640 1 --2- 172.21.15.77:0/3080873915 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7b181027a0 0x7f7b1819ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:47.550 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:47.531+0000 7f7b16a00640 1 -- 172.21.15.77:0/3080873915 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7b18106f20 msgr2=0x7f7b1819eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:47.550 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:47.531+0000 7f7b16a00640 1 --2- 172.21.15.77:0/3080873915 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7b18106f20 0x7f7b1819eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:47.550 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:47.531+0000 7f7b16a00640 1 -- 172.21.15.77:0/3080873915 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b1819f5c0 con 0x7f7b18101df0 2024-09-10T09:53:47.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:47.531+0000 7f7b16a00640 1 --2- 172.21.15.77:0/3080873915 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b18101df0 0x7f7b1819a610 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7b08005940 tx=0x7f7b080040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:47.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:47.531+0000 7f7b0f400640 1 -- 172.21.15.77:0/3080873915 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b0803f030 con 0x7f7b18101df0 2024-09-10T09:53:47.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:47.531+0000 7f7b1d600640 1 -- 172.21.15.77:0/3080873915 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b1819f850 con 0x7f7b18101df0 2024-09-10T09:53:47.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:47.531+0000 7f7b0f400640 1 -- 172.21.15.77:0/3080873915 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b080044c0 con 0x7f7b18101df0 2024-09-10T09:53:47.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:47.531+0000 7f7b0f400640 1 -- 172.21.15.77:0/3080873915 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b08035070 con 0x7f7b18101df0 2024-09-10T09:53:47.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:47.531+0000 7f7b1d600640 1 -- 172.21.15.77:0/3080873915 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b181a7190 con 0x7f7b18101df0 2024-09-10T09:53:47.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:47.532+0000 7f7b1d600640 1 -- 172.21.15.77:0/3080873915 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7b180fef30 con 0x7f7b18101df0 2024-09-10T09:53:47.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:47.534+0000 7f7b0f400640 1 -- 172.21.15.77:0/3080873915 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f7b08042030 con 0x7f7b18101df0 2024-09-10T09:53:47.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:47.535+0000 7f7b0f400640 1 --2- 172.21.15.77:0/3080873915 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f7afc0a2c10 0x7f7afc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:47.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:47.536+0000 7f7b0f400640 1 -- 172.21.15.77:0/3080873915 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f7b080eff40 con 0x7f7b18101df0 2024-09-10T09:53:47.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:47.536+0000 7f7b16000640 1 --2- 172.21.15.77:0/3080873915 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f7afc0a2c10 0x7f7afc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:47.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:47.536+0000 7f7b0f400640 1 --2- 172.21.15.77:0/3080873915 >> [v2:172.21.15.77:6802/1509649079,v1:172.21.15.77:6803/1509649079] conn(0x7f7afc0adde0 0x7f7afc0b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:47.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:47.536+0000 7f7b0f400640 1 -- 172.21.15.77:0/3080873915 --> [v2:172.21.15.77:6802/1509649079,v1:172.21.15.77:6803/1509649079] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7b08040e70 con 0x7f7afc0adde0 2024-09-10T09:53:47.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:47.536+0000 7f7b0f400640 1 -- 172.21.15.77:0/3080873915 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7f7b080f02f0 con 0x7f7b18101df0 2024-09-10T09:53:47.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:47.536+0000 7f7b17400640 1 --2- 172.21.15.77:0/3080873915 >> [v2:172.21.15.77:6802/1509649079,v1:172.21.15.77:6803/1509649079] conn(0x7f7afc0adde0 0x7f7afc0b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:47.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:47.536+0000 7f7b16000640 1 --2- 172.21.15.77:0/3080873915 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f7afc0a2c10 0x7f7afc0a50d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7b1810eef0 tx=0x7f7b04005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:47.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:47.537+0000 7f7b17400640 1 --2- 172.21.15.77:0/3080873915 >> [v2:172.21.15.77:6802/1509649079,v1:172.21.15.77:6803/1509649079] conn(0x7f7afc0adde0 0x7f7afc0b0280 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:47.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:47.538+0000 7f7b0f400640 1 -- 172.21.15.77:0/3080873915 <== osd.0 v2:172.21.15.77:6802/1509649079 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7b08040e70 con 0x7f7afc0adde0 2024-09-10T09:53:47.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:47.552+0000 7f7b1d600640 1 -- 172.21.15.77:0/3080873915 --> [v2:172.21.15.77:6802/1509649079,v1:172.21.15.77:6803/1509649079] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7b181032d0 con 0x7f7afc0adde0 2024-09-10T09:53:47.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:47.552+0000 7f7b0f400640 1 -- 172.21.15.77:0/3080873915 <== osd.0 v2:172.21.15.77:6802/1509649079 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f7b181032d0 con 0x7f7afc0adde0 2024-09-10T09:53:47.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:47.553+0000 7f7b1d600640 1 -- 172.21.15.77:0/3080873915 >> [v2:172.21.15.77:6802/1509649079,v1:172.21.15.77:6803/1509649079] conn(0x7f7afc0adde0 msgr2=0x7f7afc0b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:47.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:47.553+0000 7f7b1d600640 1 --2- 172.21.15.77:0/3080873915 >> [v2:172.21.15.77:6802/1509649079,v1:172.21.15.77:6803/1509649079] conn(0x7f7afc0adde0 0x7f7afc0b0280 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:47.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:47.553+0000 7f7b1d600640 1 -- 172.21.15.77:0/3080873915 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f7afc0a2c10 msgr2=0x7f7afc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:47.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:47.553+0000 7f7b1d600640 1 --2- 172.21.15.77:0/3080873915 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f7afc0a2c10 0x7f7afc0a50d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7b1810eef0 tx=0x7f7b04005f90 comp rx=0 tx=0).stop 2024-09-10T09:53:47.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:47.553+0000 7f7b1d600640 1 -- 172.21.15.77:0/3080873915 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b18101df0 msgr2=0x7f7b1819a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:47.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:47.553+0000 7f7b1d600640 1 --2- 172.21.15.77:0/3080873915 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b18101df0 0x7f7b1819a610 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7b08005940 tx=0x7f7b080040c0 comp rx=0 tx=0).stop 2024-09-10T09:53:47.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:47.553+0000 7f7b17400640 1 -- 172.21.15.77:0/3080873915 reap_dead start 2024-09-10T09:53:47.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:47.553+0000 7f7b1d600640 1 -- 172.21.15.77:0/3080873915 shutdown_connections 2024-09-10T09:53:47.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:47.553+0000 7f7b1d600640 1 -- 172.21.15.77:0/3080873915 >> 172.21.15.77:0/3080873915 conn(0x7f7b180fd8b0 msgr2=0x7f7b180ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:47.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:47.553+0000 7f7b1d600640 1 -- 172.21.15.77:0/3080873915 shutdown_connections 2024-09-10T09:53:47.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:47.553+0000 7f7b1d600640 1 -- 172.21.15.77:0/3080873915 wait complete. 2024-09-10T09:53:47.678 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:53:47.678 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:53:47.683 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:53:47.683 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:53:47.683 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:53:48.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:48.579+0000 7fbad7200640 1 -- 172.21.15.77:0/3602378341 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbad0103e30 msgr2=0x7fbad0104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:48.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:48.579+0000 7fbad7200640 1 --2- 172.21.15.77:0/3602378341 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbad0103e30 0x7fbad0104210 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fbac4009900 tx=0x7fbac402a410 comp rx=0 tx=0).stop 2024-09-10T09:53:48.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:48.579+0000 7fbad7200640 1 -- 172.21.15.77:0/3602378341 shutdown_connections 2024-09-10T09:53:48.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:48.579+0000 7fbad7200640 1 --2- 172.21.15.77:0/3602378341 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbad0108f60 0x7fbad0110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:48.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:48.579+0000 7fbad7200640 1 --2- 172.21.15.77:0/3602378341 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbad01047e0 0x7fbad0108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:48.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:48.579+0000 7fbad7200640 1 --2- 172.21.15.77:0/3602378341 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbad0103e30 0x7fbad0104210 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:48.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:48.579+0000 7fbad7200640 1 -- 172.21.15.77:0/3602378341 >> 172.21.15.77:0/3602378341 conn(0x7fbad00ff910 msgr2=0x7fbad0101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:48.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:48.579+0000 7fbad7200640 1 -- 172.21.15.77:0/3602378341 shutdown_connections 2024-09-10T09:53:48.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:48.580+0000 7fbad7200640 1 -- 172.21.15.77:0/3602378341 wait complete. 2024-09-10T09:53:48.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:48.580+0000 7fbad7200640 1 Processor -- start 2024-09-10T09:53:48.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:48.581+0000 7fbad7200640 1 -- start start 2024-09-10T09:53:48.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:48.581+0000 7fbad7200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbad0103e30 0x7fbad019a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:48.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:48.581+0000 7fbad7200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbad01047e0 0x7fbad019a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:48.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:48.581+0000 7fbad7200640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbad0108f60 0x7fbad019ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:48.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:48.581+0000 7fbad7200640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbad0113fc0 con 0x7fbad01047e0 2024-09-10T09:53:48.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:48.581+0000 7fbad7200640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbad0113e40 con 0x7fbad0108f60 2024-09-10T09:53:48.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:48.581+0000 7fbad7200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbad0114140 con 0x7fbad0103e30 2024-09-10T09:53:48.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:48.581+0000 7fbad5e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbad0103e30 0x7fbad019a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:48.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:48.581+0000 7fbad5400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbad01047e0 0x7fbad019a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:48.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:48.581+0000 7fbad5e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbad0103e30 0x7fbad019a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:55862/0 (socket says 172.21.15.77:55862) 2024-09-10T09:53:48.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:48.581+0000 7fbad5e00640 1 -- 172.21.15.77:0/3191337531 learned_addr learned my addr 172.21.15.77:0/3191337531 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:53:48.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:48.582+0000 7fbad6800640 1 --2- 172.21.15.77:0/3191337531 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbad0108f60 0x7fbad019ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:48.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:48.582+0000 7fbad5e00640 1 -- 172.21.15.77:0/3191337531 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbad0108f60 msgr2=0x7fbad019ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:48.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:48.582+0000 7fbad5e00640 1 --2- 172.21.15.77:0/3191337531 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbad0108f60 0x7fbad019ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:48.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:48.582+0000 7fbad5e00640 1 -- 172.21.15.77:0/3191337531 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbad01047e0 msgr2=0x7fbad019a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:48.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:48.582+0000 7fbad5e00640 1 --2- 172.21.15.77:0/3191337531 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbad01047e0 0x7fbad019a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:48.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:48.582+0000 7fbad5e00640 1 -- 172.21.15.77:0/3191337531 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbad019f3f0 con 0x7fbad0103e30 2024-09-10T09:53:48.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:48.582+0000 7fbad5400640 1 --2- 172.21.15.77:0/3191337531 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbad01047e0 0x7fbad019a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:53:48.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:48.582+0000 7fbad5e00640 1 --2- 172.21.15.77:0/3191337531 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbad0103e30 0x7fbad019a440 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fbac4006010 tx=0x7fbac4004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:48.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:48.582+0000 7fbacaa00640 1 -- 172.21.15.77:0/3191337531 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac403f030 con 0x7fbad0103e30 2024-09-10T09:53:48.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:48.583+0000 7fbad7200640 1 -- 172.21.15.77:0/3191337531 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbad019f680 con 0x7fbad0103e30 2024-09-10T09:53:48.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:48.583+0000 7fbad7200640 1 -- 172.21.15.77:0/3191337531 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbad01a6fc0 con 0x7fbad0103e30 2024-09-10T09:53:48.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:48.583+0000 7fbacaa00640 1 -- 172.21.15.77:0/3191337531 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbac4004440 con 0x7fbad0103e30 2024-09-10T09:53:48.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:48.583+0000 7fbacaa00640 1 -- 172.21.15.77:0/3191337531 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac4035070 con 0x7fbad0103e30 2024-09-10T09:53:48.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:48.584+0000 7fbabfe00640 1 -- 172.21.15.77:0/3191337531 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbad0062940 con 0x7fbad0103e30 2024-09-10T09:53:48.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:48.586+0000 7fbacaa00640 1 -- 172.21.15.77:0/3191337531 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fbac4042030 con 0x7fbad0103e30 2024-09-10T09:53:48.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:48.587+0000 7fbacaa00640 1 --2- 172.21.15.77:0/3191337531 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fbab80a2c90 0x7fbab80a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:48.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:48.588+0000 7fbacaa00640 1 -- 172.21.15.77:0/3191337531 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7fbac40eff20 con 0x7fbad0103e30 2024-09-10T09:53:48.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:48.588+0000 7fbad5400640 1 --2- 172.21.15.77:0/3191337531 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fbab80a2c90 0x7fbab80a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:48.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:48.588+0000 7fbacaa00640 1 --2- 172.21.15.77:0/3191337531 >> [v2:172.21.15.134:6824/1602213143,v1:172.21.15.134:6825/1602213143] conn(0x7fbab80ade60 0x7fbab80b0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:48.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:48.588+0000 7fbacaa00640 1 -- 172.21.15.77:0/3191337531 --> [v2:172.21.15.134:6824/1602213143,v1:172.21.15.134:6825/1602213143] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbac4001d30 con 0x7fbab80ade60 2024-09-10T09:53:48.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:48.588+0000 7fbacaa00640 1 -- 172.21.15.77:0/3191337531 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7fbac40f02d0 con 0x7fbad0103e30 2024-09-10T09:53:48.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:48.588+0000 7fbad6800640 1 --2- 172.21.15.77:0/3191337531 >> [v2:172.21.15.134:6824/1602213143,v1:172.21.15.134:6825/1602213143] conn(0x7fbab80ade60 0x7fbab80b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:48.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:48.589+0000 7fbad5400640 1 --2- 172.21.15.77:0/3191337531 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fbab80a2c90 0x7fbab80a5150 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fbad00ffd30 tx=0x7fbac0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:48.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:48.589+0000 7fbad6800640 1 --2- 172.21.15.77:0/3191337531 >> [v2:172.21.15.134:6824/1602213143,v1:172.21.15.134:6825/1602213143] conn(0x7fbab80ade60 0x7fbab80b0300 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:48.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:48.590+0000 7fbacaa00640 1 -- 172.21.15.77:0/3191337531 <== osd.11 v2:172.21.15.134:6824/1602213143 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbac4001d30 con 0x7fbab80ade60 2024-09-10T09:53:48.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:48.606+0000 7fbabfe00640 1 -- 172.21.15.77:0/3191337531 --> [v2:172.21.15.134:6824/1602213143,v1:172.21.15.134:6825/1602213143] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fbad0062d70 con 0x7fbab80ade60 2024-09-10T09:53:48.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:48.606+0000 7fbacaa00640 1 -- 172.21.15.77:0/3191337531 <== osd.11 v2:172.21.15.134:6824/1602213143 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fbad0062d70 con 0x7fbab80ade60 2024-09-10T09:53:48.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:48.607+0000 7fbabfe00640 1 -- 172.21.15.77:0/3191337531 >> [v2:172.21.15.134:6824/1602213143,v1:172.21.15.134:6825/1602213143] conn(0x7fbab80ade60 msgr2=0x7fbab80b0300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:48.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:48.607+0000 7fbabfe00640 1 --2- 172.21.15.77:0/3191337531 >> [v2:172.21.15.134:6824/1602213143,v1:172.21.15.134:6825/1602213143] conn(0x7fbab80ade60 0x7fbab80b0300 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:48.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:48.607+0000 7fbabfe00640 1 -- 172.21.15.77:0/3191337531 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fbab80a2c90 msgr2=0x7fbab80a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:48.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:48.607+0000 7fbabfe00640 1 --2- 172.21.15.77:0/3191337531 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fbab80a2c90 0x7fbab80a5150 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fbad00ffd30 tx=0x7fbac0005f90 comp rx=0 tx=0).stop 2024-09-10T09:53:48.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:48.607+0000 7fbabfe00640 1 -- 172.21.15.77:0/3191337531 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbad0103e30 msgr2=0x7fbad019a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:48.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:48.607+0000 7fbabfe00640 1 --2- 172.21.15.77:0/3191337531 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbad0103e30 0x7fbad019a440 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fbac4006010 tx=0x7fbac4004040 comp rx=0 tx=0).stop 2024-09-10T09:53:48.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:48.607+0000 7fbad6800640 1 -- 172.21.15.77:0/3191337531 reap_dead start 2024-09-10T09:53:48.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:48.607+0000 7fbabfe00640 1 -- 172.21.15.77:0/3191337531 shutdown_connections 2024-09-10T09:53:48.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:48.607+0000 7fbabfe00640 1 -- 172.21.15.77:0/3191337531 >> 172.21.15.77:0/3191337531 conn(0x7fbad00ff910 msgr2=0x7fbad0100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:48.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:48.608+0000 7fbabfe00640 1 -- 172.21.15.77:0/3191337531 shutdown_connections 2024-09-10T09:53:48.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:48.608+0000 7fbabfe00640 1 -- 172.21.15.77:0/3191337531 wait complete. 2024-09-10T09:53:48.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:48 smithi077 ceph-mon[30319]: pgmap v348: 129 pgs: 129 active+clean; 645 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-10T09:53:49.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:48 smithi134 ceph-mon[35674]: pgmap v348: 129 pgs: 129 active+clean; 645 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-10T09:53:49.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:48 smithi120 ceph-mon[34770]: pgmap v348: 129 pgs: 129 active+clean; 645 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-10T09:53:50.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.278+0000 7fa246e00640 1 -- 172.21.15.77:0/3454562752 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa2401079c0 msgr2=0x7fa240107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:50.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.278+0000 7fa246e00640 1 --2- 172.21.15.77:0/3454562752 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa2401079c0 0x7fa240107da0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa234009930 tx=0x7fa23402a420 comp rx=0 tx=0).stop 2024-09-10T09:53:50.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.278+0000 7fa246e00640 1 -- 172.21.15.77:0/3454562752 shutdown_connections 2024-09-10T09:53:50.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.278+0000 7fa246e00640 1 --2- 172.21.15.77:0/3454562752 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa240102360 0x7fa24010e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:50.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.278+0000 7fa246e00640 1 --2- 172.21.15.77:0/3454562752 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa2401019c0 0x7fa240101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:50.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.278+0000 7fa246e00640 1 --2- 172.21.15.77:0/3454562752 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa2401079c0 0x7fa240107da0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:50.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.278+0000 7fa246e00640 1 -- 172.21.15.77:0/3454562752 >> 172.21.15.77:0/3454562752 conn(0x7fa2400fd740 msgr2=0x7fa2400ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:50.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.278+0000 7fa246e00640 1 -- 172.21.15.77:0/3454562752 shutdown_connections 2024-09-10T09:53:50.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.278+0000 7fa246e00640 1 -- 172.21.15.77:0/3454562752 wait complete. 2024-09-10T09:53:50.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.279+0000 7fa246e00640 1 Processor -- start 2024-09-10T09:53:50.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.279+0000 7fa246e00640 1 -- start start 2024-09-10T09:53:50.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.280+0000 7fa246e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2401019c0 0x7fa24019a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:50.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.280+0000 7fa246e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa240102360 0x7fa24019aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:50.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.280+0000 7fa246e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa2401079c0 0x7fa24019edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:50.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.280+0000 7fa246e00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa240111dc0 con 0x7fa2401079c0 2024-09-10T09:53:50.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.280+0000 7fa246e00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa240111c40 con 0x7fa240102360 2024-09-10T09:53:50.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.280+0000 7fa246e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa240111f40 con 0x7fa2401019c0 2024-09-10T09:53:50.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.280+0000 7fa245000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa240102360 0x7fa24019aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:50.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.280+0000 7fa246400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa2401079c0 0x7fa24019edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:50.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.280+0000 7fa245000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa240102360 0x7fa24019aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:41520/0 (socket says 172.21.15.77:41520) 2024-09-10T09:53:50.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.280+0000 7fa245000640 1 -- 172.21.15.77:0/981000206 learned_addr learned my addr 172.21.15.77:0/981000206 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:53:50.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.280+0000 7fa245a00640 1 --2- 172.21.15.77:0/981000206 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2401019c0 0x7fa24019a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:50.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.280+0000 7fa246400640 1 -- 172.21.15.77:0/981000206 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2401019c0 msgr2=0x7fa24019a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:50.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.280+0000 7fa246400640 1 --2- 172.21.15.77:0/981000206 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2401019c0 0x7fa24019a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:50.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.280+0000 7fa246400640 1 -- 172.21.15.77:0/981000206 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa240102360 msgr2=0x7fa24019aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:50.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.280+0000 7fa246400640 1 --2- 172.21.15.77:0/981000206 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa240102360 0x7fa24019aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:50.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.280+0000 7fa246400640 1 -- 172.21.15.77:0/981000206 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa24019f4b0 con 0x7fa2401079c0 2024-09-10T09:53:50.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.280+0000 7fa245000640 1 --2- 172.21.15.77:0/981000206 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa240102360 0x7fa24019aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:53:50.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.281+0000 7fa246400640 1 --2- 172.21.15.77:0/981000206 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa2401079c0 0x7fa24019edd0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa238009fd0 tx=0x7fa23800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:50.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.281+0000 7fa23e000640 1 -- 172.21.15.77:0/981000206 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa23800ce20 con 0x7fa2401079c0 2024-09-10T09:53:50.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.281+0000 7fa246e00640 1 -- 172.21.15.77:0/981000206 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa24019f7a0 con 0x7fa2401079c0 2024-09-10T09:53:50.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.281+0000 7fa23e000640 1 -- 172.21.15.77:0/981000206 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa238005020 con 0x7fa2401079c0 2024-09-10T09:53:50.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.281+0000 7fa23e000640 1 -- 172.21.15.77:0/981000206 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa238010720 con 0x7fa2401079c0 2024-09-10T09:53:50.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.281+0000 7fa246e00640 1 -- 172.21.15.77:0/981000206 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2401a7080 con 0x7fa2401079c0 2024-09-10T09:53:50.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.282+0000 7fa246e00640 1 -- 172.21.15.77:0/981000206 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa2400ff6f0 con 0x7fa2401079c0 2024-09-10T09:53:50.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.284+0000 7fa23e000640 1 -- 172.21.15.77:0/981000206 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fa238005540 con 0x7fa2401079c0 2024-09-10T09:53:50.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.285+0000 7fa23e000640 1 --2- 172.21.15.77:0/981000206 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fa2280a2c90 0x7fa2280a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:50.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.285+0000 7fa23e000640 1 -- 172.21.15.77:0/981000206 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7fa2380cfca0 con 0x7fa2401079c0 2024-09-10T09:53:50.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.285+0000 7fa245a00640 1 --2- 172.21.15.77:0/981000206 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fa2280a2c90 0x7fa2280a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:50.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.286+0000 7fa23e000640 1 --2- 172.21.15.77:0/981000206 >> [v2:172.21.15.77:6818/879565639,v1:172.21.15.77:6819/879565639] conn(0x7fa2280ade60 0x7fa2280b0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:50.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.286+0000 7fa23e000640 1 -- 172.21.15.77:0/981000206 --> [v2:172.21.15.77:6818/879565639,v1:172.21.15.77:6819/879565639] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa2280b09d0 con 0x7fa2280ade60 2024-09-10T09:53:50.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.286+0000 7fa23e000640 1 -- 172.21.15.77:0/981000206 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7fa2380d0050 con 0x7fa2401079c0 2024-09-10T09:53:50.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.286+0000 7fa245000640 1 --2- 172.21.15.77:0/981000206 >> [v2:172.21.15.77:6818/879565639,v1:172.21.15.77:6819/879565639] conn(0x7fa2280ade60 0x7fa2280b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:50.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.286+0000 7fa245a00640 1 --2- 172.21.15.77:0/981000206 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fa2280a2c90 0x7fa2280a5150 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa234009900 tx=0x7fa2340058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:50.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.286+0000 7fa245000640 1 --2- 172.21.15.77:0/981000206 >> [v2:172.21.15.77:6818/879565639,v1:172.21.15.77:6819/879565639] conn(0x7fa2280ade60 0x7fa2280b0300 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:50.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.287+0000 7fa23e000640 1 -- 172.21.15.77:0/981000206 <== osd.6 v2:172.21.15.77:6818/879565639 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa2280b09d0 con 0x7fa2280ade60 2024-09-10T09:53:50.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.302+0000 7fa246e00640 1 -- 172.21.15.77:0/981000206 --> [v2:172.21.15.77:6818/879565639,v1:172.21.15.77:6819/879565639] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa240103100 con 0x7fa2280ade60 2024-09-10T09:53:50.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.302+0000 7fa23e000640 1 -- 172.21.15.77:0/981000206 <== osd.6 v2:172.21.15.77:6818/879565639 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa240103100 con 0x7fa2280ade60 2024-09-10T09:53:50.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.302+0000 7fa246e00640 1 -- 172.21.15.77:0/981000206 >> [v2:172.21.15.77:6818/879565639,v1:172.21.15.77:6819/879565639] conn(0x7fa2280ade60 msgr2=0x7fa2280b0300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:50.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.302+0000 7fa246e00640 1 --2- 172.21.15.77:0/981000206 >> [v2:172.21.15.77:6818/879565639,v1:172.21.15.77:6819/879565639] conn(0x7fa2280ade60 0x7fa2280b0300 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:50.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.303+0000 7fa246e00640 1 -- 172.21.15.77:0/981000206 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fa2280a2c90 msgr2=0x7fa2280a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:50.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.303+0000 7fa246e00640 1 --2- 172.21.15.77:0/981000206 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fa2280a2c90 0x7fa2280a5150 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa234009900 tx=0x7fa2340058d0 comp rx=0 tx=0).stop 2024-09-10T09:53:50.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.303+0000 7fa246e00640 1 -- 172.21.15.77:0/981000206 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa2401079c0 msgr2=0x7fa24019edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:50.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.303+0000 7fa246e00640 1 --2- 172.21.15.77:0/981000206 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa2401079c0 0x7fa24019edd0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa238009fd0 tx=0x7fa23800ef10 comp rx=0 tx=0).stop 2024-09-10T09:53:50.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.303+0000 7fa246400640 1 -- 172.21.15.77:0/981000206 reap_dead start 2024-09-10T09:53:50.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.303+0000 7fa246e00640 1 -- 172.21.15.77:0/981000206 shutdown_connections 2024-09-10T09:53:50.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.303+0000 7fa246e00640 1 -- 172.21.15.77:0/981000206 >> 172.21.15.77:0/981000206 conn(0x7fa2400fd740 msgr2=0x7fa2400fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:50.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.303+0000 7fa246e00640 1 -- 172.21.15.77:0/981000206 shutdown_connections 2024-09-10T09:53:50.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.303+0000 7fa246e00640 1 -- 172.21.15.77:0/981000206 wait complete. 2024-09-10T09:53:50.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.924+0000 7f20da200640 1 -- 172.21.15.77:0/420196136 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f20d41047e0 msgr2=0x7f20d4108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:50.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.924+0000 7f20da200640 1 --2- 172.21.15.77:0/420196136 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f20d41047e0 0x7f20d4108830 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f20c0009900 tx=0x7f20c002a3f0 comp rx=0 tx=0).stop 2024-09-10T09:53:50.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.924+0000 7f20da200640 1 -- 172.21.15.77:0/420196136 shutdown_connections 2024-09-10T09:53:50.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.924+0000 7f20da200640 1 --2- 172.21.15.77:0/420196136 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f20d4108f60 0x7f20d4110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:50.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.924+0000 7f20da200640 1 --2- 172.21.15.77:0/420196136 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f20d41047e0 0x7f20d4108830 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:50.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.924+0000 7f20da200640 1 --2- 172.21.15.77:0/420196136 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f20d4103e30 0x7f20d4104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:50.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.924+0000 7f20da200640 1 -- 172.21.15.77:0/420196136 >> 172.21.15.77:0/420196136 conn(0x7f20d40ff8d0 msgr2=0x7f20d4101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:50.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.924+0000 7f20da200640 1 -- 172.21.15.77:0/420196136 shutdown_connections 2024-09-10T09:53:50.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.924+0000 7f20da200640 1 -- 172.21.15.77:0/420196136 wait complete. 2024-09-10T09:53:50.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.925+0000 7f20da200640 1 Processor -- start 2024-09-10T09:53:50.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.925+0000 7f20da200640 1 -- start start 2024-09-10T09:53:50.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.926+0000 7f20da200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f20d4103e30 0x7f20d419a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:50.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.926+0000 7f20da200640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f20d41047e0 0x7f20d419a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:50.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.926+0000 7f20da200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f20d4108f60 0x7f20d419ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:50.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.926+0000 7f20da200640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f20d4114010 con 0x7f20d4103e30 2024-09-10T09:53:50.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.926+0000 7f20da200640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f20d4113e90 con 0x7f20d41047e0 2024-09-10T09:53:50.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.926+0000 7f20da200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f20d4114190 con 0x7f20d4108f60 2024-09-10T09:53:50.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.926+0000 7f20d8e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f20d4103e30 0x7f20d419a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:50.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.926+0000 7f20d8e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f20d4103e30 0x7f20d419a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:53246/0 (socket says 172.21.15.77:53246) 2024-09-10T09:53:50.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.926+0000 7f20d8e00640 1 -- 172.21.15.77:0/2467272942 learned_addr learned my addr 172.21.15.77:0/2467272942 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:53:50.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.926+0000 7f20d3e00640 1 --2- 172.21.15.77:0/2467272942 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f20d41047e0 0x7f20d419a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:50.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.926+0000 7f20d9800640 1 --2- 172.21.15.77:0/2467272942 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f20d4108f60 0x7f20d419ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:50.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.927+0000 7f20d3e00640 1 -- 172.21.15.77:0/2467272942 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f20d4108f60 msgr2=0x7f20d419ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:50.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.927+0000 7f20d3e00640 1 --2- 172.21.15.77:0/2467272942 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f20d4108f60 0x7f20d419ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:50.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.927+0000 7f20d3e00640 1 -- 172.21.15.77:0/2467272942 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f20d4103e30 msgr2=0x7f20d419a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:50.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.927+0000 7f20d3e00640 1 --2- 172.21.15.77:0/2467272942 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f20d4103e30 0x7f20d419a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:50.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.927+0000 7f20d3e00640 1 -- 172.21.15.77:0/2467272942 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20d419f440 con 0x7f20d41047e0 2024-09-10T09:53:50.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.927+0000 7f20d8e00640 1 --2- 172.21.15.77:0/2467272942 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f20d4103e30 0x7f20d419a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:53:50.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.927+0000 7f20d9800640 1 --2- 172.21.15.77:0/2467272942 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f20d4108f60 0x7f20d419ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:53:50.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.927+0000 7f20d3e00640 1 --2- 172.21.15.77:0/2467272942 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f20d41047e0 0x7f20d419a9d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f20c002a910 tx=0x7f20c0003b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:50.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.927+0000 7f20d1600640 1 -- 172.21.15.77:0/2467272942 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c003f070 con 0x7f20d41047e0 2024-09-10T09:53:50.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.927+0000 7f20da200640 1 -- 172.21.15.77:0/2467272942 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20d419f6d0 con 0x7f20d41047e0 2024-09-10T09:53:50.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.927+0000 7f20d1600640 1 -- 172.21.15.77:0/2467272942 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20c0031c90 con 0x7f20d41047e0 2024-09-10T09:53:50.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.927+0000 7f20d1600640 1 -- 172.21.15.77:0/2467272942 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c0035030 con 0x7f20d41047e0 2024-09-10T09:53:50.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.927+0000 7f20da200640 1 -- 172.21.15.77:0/2467272942 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20d41a7010 con 0x7f20d41047e0 2024-09-10T09:53:50.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.928+0000 7f20da200640 1 -- 172.21.15.77:0/2467272942 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f20d4062940 con 0x7f20d41047e0 2024-09-10T09:53:50.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.930+0000 7f20d1600640 1 -- 172.21.15.77:0/2467272942 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f20c0042070 con 0x7f20d41047e0 2024-09-10T09:53:50.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.932+0000 7f20d1600640 1 --2- 172.21.15.77:0/2467272942 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f20bc0a2c90 0x7f20bc0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:50.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.932+0000 7f20d1600640 1 -- 172.21.15.77:0/2467272942 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f20c00f0390 con 0x7f20d41047e0 2024-09-10T09:53:50.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.932+0000 7f20d8e00640 1 --2- 172.21.15.77:0/2467272942 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f20bc0a2c90 0x7f20bc0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:50.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.932+0000 7f20d1600640 1 --2- 172.21.15.77:0/2467272942 >> [v2:172.21.15.120:6816/2688667400,v1:172.21.15.120:6817/2688667400] conn(0x7f20bc0ade60 0x7f20bc0b0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:50.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.932+0000 7f20d1600640 1 -- 172.21.15.77:0/2467272942 --> [v2:172.21.15.120:6816/2688667400,v1:172.21.15.120:6817/2688667400] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f20c0034e50 con 0x7f20bc0ade60 2024-09-10T09:53:50.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.932+0000 7f20d1600640 1 -- 172.21.15.77:0/2467272942 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7f20c00f07e0 con 0x7f20d41047e0 2024-09-10T09:53:50.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.933+0000 7f20d8e00640 1 --2- 172.21.15.77:0/2467272942 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f20bc0a2c90 0x7f20bc0a5150 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f20c8011550 tx=0x7f20c8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:50.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.933+0000 7f20d9800640 1 --2- 172.21.15.77:0/2467272942 >> [v2:172.21.15.120:6816/2688667400,v1:172.21.15.120:6817/2688667400] conn(0x7f20bc0ade60 0x7f20bc0b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:50.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.933+0000 7f20d9800640 1 --2- 172.21.15.77:0/2467272942 >> [v2:172.21.15.120:6816/2688667400,v1:172.21.15.120:6817/2688667400] conn(0x7f20bc0ade60 0x7f20bc0b0300 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:50.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.934+0000 7f20d1600640 1 -- 172.21.15.77:0/2467272942 <== osd.7 v2:172.21.15.120:6816/2688667400 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f20c0034e50 con 0x7f20bc0ade60 2024-09-10T09:53:50.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.949+0000 7f20da200640 1 -- 172.21.15.77:0/2467272942 --> [v2:172.21.15.120:6816/2688667400,v1:172.21.15.120:6817/2688667400] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f20d4062d40 con 0x7f20bc0ade60 2024-09-10T09:53:50.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.950+0000 7f20d1600640 1 -- 172.21.15.77:0/2467272942 <== osd.7 v2:172.21.15.120:6816/2688667400 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f20d4062d40 con 0x7f20bc0ade60 2024-09-10T09:53:50.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.950+0000 7f20da200640 1 -- 172.21.15.77:0/2467272942 >> [v2:172.21.15.120:6816/2688667400,v1:172.21.15.120:6817/2688667400] conn(0x7f20bc0ade60 msgr2=0x7f20bc0b0300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:50.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.950+0000 7f20da200640 1 --2- 172.21.15.77:0/2467272942 >> [v2:172.21.15.120:6816/2688667400,v1:172.21.15.120:6817/2688667400] conn(0x7f20bc0ade60 0x7f20bc0b0300 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:50.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.950+0000 7f20da200640 1 -- 172.21.15.77:0/2467272942 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f20bc0a2c90 msgr2=0x7f20bc0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:50.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.950+0000 7f20da200640 1 --2- 172.21.15.77:0/2467272942 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f20bc0a2c90 0x7f20bc0a5150 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f20c8011550 tx=0x7f20c8005f90 comp rx=0 tx=0).stop 2024-09-10T09:53:50.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.950+0000 7f20da200640 1 -- 172.21.15.77:0/2467272942 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f20d41047e0 msgr2=0x7f20d419a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:50.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.950+0000 7f20da200640 1 --2- 172.21.15.77:0/2467272942 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f20d41047e0 0x7f20d419a9d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f20c002a910 tx=0x7f20c0003b20 comp rx=0 tx=0).stop 2024-09-10T09:53:50.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.950+0000 7f20d9800640 1 -- 172.21.15.77:0/2467272942 reap_dead start 2024-09-10T09:53:50.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.950+0000 7f20da200640 1 -- 172.21.15.77:0/2467272942 shutdown_connections 2024-09-10T09:53:50.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.950+0000 7f20da200640 1 -- 172.21.15.77:0/2467272942 >> 172.21.15.77:0/2467272942 conn(0x7f20d40ff8d0 msgr2=0x7f20d4100bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:50.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.951+0000 7f20da200640 1 -- 172.21.15.77:0/2467272942 shutdown_connections 2024-09-10T09:53:50.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.951+0000 7f20da200640 1 -- 172.21.15.77:0/2467272942 wait complete. 2024-09-10T09:53:51.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:50 smithi134 ceph-mon[35674]: pgmap v349: 129 pgs: 129 active+clean; 645 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 682 B/s wr, 12 op/s 2024-09-10T09:53:51.141 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:50 smithi077 ceph-mon[30319]: pgmap v349: 129 pgs: 129 active+clean; 645 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 682 B/s wr, 12 op/s 2024-09-10T09:53:51.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:50 smithi120 ceph-mon[34770]: pgmap v349: 129 pgs: 129 active+clean; 645 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 682 B/s wr, 12 op/s 2024-09-10T09:53:51.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.947+0000 7f376ec00640 1 -- 172.21.15.77:0/2420930763 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3768103ff0 msgr2=0x7f37681043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:51.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.947+0000 7f376ec00640 1 --2- 172.21.15.77:0/2420930763 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3768103ff0 0x7f37681043d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f3758009900 tx=0x7f375802a410 comp rx=0 tx=0).stop 2024-09-10T09:53:51.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.948+0000 7f376ec00640 1 -- 172.21.15.77:0/2420930763 shutdown_connections 2024-09-10T09:53:51.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.948+0000 7f376ec00640 1 --2- 172.21.15.77:0/2420930763 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3768109120 0x7f3768110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:51.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.948+0000 7f376ec00640 1 --2- 172.21.15.77:0/2420930763 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f37681049a0 0x7f37681089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:51.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.948+0000 7f376ec00640 1 --2- 172.21.15.77:0/2420930763 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3768103ff0 0x7f37681043d0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:51.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.948+0000 7f376ec00640 1 -- 172.21.15.77:0/2420930763 >> 172.21.15.77:0/2420930763 conn(0x7f37680ffa90 msgr2=0x7f3768101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:51.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.948+0000 7f376ec00640 1 -- 172.21.15.77:0/2420930763 shutdown_connections 2024-09-10T09:53:51.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.948+0000 7f376ec00640 1 -- 172.21.15.77:0/2420930763 wait complete. 2024-09-10T09:53:51.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.949+0000 7f376ec00640 1 Processor -- start 2024-09-10T09:53:51.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.949+0000 7f376ec00640 1 -- start start 2024-09-10T09:53:51.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.949+0000 7f376ec00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3768103ff0 0x7f376819a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:51.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.949+0000 7f376ec00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37681049a0 0x7f376819ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:51.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.949+0000 7f376ec00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3768109120 0x7f376819eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:51.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.949+0000 7f376ec00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3768114170 con 0x7f3768109120 2024-09-10T09:53:51.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.949+0000 7f376ec00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3768113ff0 con 0x7f3768103ff0 2024-09-10T09:53:51.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.949+0000 7f376ec00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f37681142f0 con 0x7f37681049a0 2024-09-10T09:53:51.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.949+0000 7f376cc00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3768109120 0x7f376819eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:51.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.950+0000 7f376cc00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3768109120 0x7f376819eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:53268/0 (socket says 172.21.15.77:53268) 2024-09-10T09:53:51.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.950+0000 7f376cc00640 1 -- 172.21.15.77:0/1406281162 learned_addr learned my addr 172.21.15.77:0/1406281162 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:53:51.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.950+0000 7f3767e00640 1 --2- 172.21.15.77:0/1406281162 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3768103ff0 0x7f376819a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:51.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.950+0000 7f3767400640 1 --2- 172.21.15.77:0/1406281162 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37681049a0 0x7f376819ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:51.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.950+0000 7f3767400640 1 -- 172.21.15.77:0/1406281162 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3768103ff0 msgr2=0x7f376819a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:51.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.950+0000 7f3767400640 1 --2- 172.21.15.77:0/1406281162 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3768103ff0 0x7f376819a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:51.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.950+0000 7f3767400640 1 -- 172.21.15.77:0/1406281162 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3768109120 msgr2=0x7f376819eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:51.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.950+0000 7f3767400640 1 --2- 172.21.15.77:0/1406281162 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3768109120 0x7f376819eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:51.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.950+0000 7f3767400640 1 -- 172.21.15.77:0/1406281162 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f376819f590 con 0x7f37681049a0 2024-09-10T09:53:51.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.950+0000 7f376cc00640 1 --2- 172.21.15.77:0/1406281162 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3768109120 0x7f376819eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:53:51.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.950+0000 7f3767e00640 1 --2- 172.21.15.77:0/1406281162 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3768103ff0 0x7f376819a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:53:51.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.950+0000 7f3767400640 1 --2- 172.21.15.77:0/1406281162 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37681049a0 0x7f376819ab20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f375400e9f0 tx=0x7f375400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:51.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.951+0000 7f3764c00640 1 -- 172.21.15.77:0/1406281162 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375400ccc0 con 0x7f37681049a0 2024-09-10T09:53:51.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.951+0000 7f376ec00640 1 -- 172.21.15.77:0/1406281162 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f376819f880 con 0x7f37681049a0 2024-09-10T09:53:51.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.951+0000 7f3764c00640 1 -- 172.21.15.77:0/1406281162 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3754010910 con 0x7f37681049a0 2024-09-10T09:53:51.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.951+0000 7f3764c00640 1 -- 172.21.15.77:0/1406281162 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3754018960 con 0x7f37681049a0 2024-09-10T09:53:51.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.951+0000 7f376ec00640 1 -- 172.21.15.77:0/1406281162 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37681a7160 con 0x7f37681049a0 2024-09-10T09:53:51.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.952+0000 7f376ec00640 1 -- 172.21.15.77:0/1406281162 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3768062980 con 0x7f37681049a0 2024-09-10T09:53:51.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.954+0000 7f3764c00640 1 -- 172.21.15.77:0/1406281162 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f3754010ac0 con 0x7f37681049a0 2024-09-10T09:53:51.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.956+0000 7f3764c00640 1 --2- 172.21.15.77:0/1406281162 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f374c0a2c90 0x7f374c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:51.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.956+0000 7f3764c00640 1 -- 172.21.15.77:0/1406281162 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f37540d00b0 con 0x7f37681049a0 2024-09-10T09:53:51.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.956+0000 7f3767e00640 1 --2- 172.21.15.77:0/1406281162 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f374c0a2c90 0x7f374c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:51.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.956+0000 7f3764c00640 1 --2- 172.21.15.77:0/1406281162 >> [v2:172.21.15.134:6800/3280466823,v1:172.21.15.134:6801/3280466823] conn(0x7f374c0ade60 0x7f374c0b0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:51.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.956+0000 7f3764c00640 1 -- 172.21.15.77:0/1406281162 --> [v2:172.21.15.134:6800/3280466823,v1:172.21.15.134:6801/3280466823] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3754028ea0 con 0x7f374c0ade60 2024-09-10T09:53:51.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.956+0000 7f3764c00640 1 -- 172.21.15.77:0/1406281162 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7f37540d0460 con 0x7f37681049a0 2024-09-10T09:53:51.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.956+0000 7f376cc00640 1 --2- 172.21.15.77:0/1406281162 >> [v2:172.21.15.134:6800/3280466823,v1:172.21.15.134:6801/3280466823] conn(0x7f374c0ade60 0x7f374c0b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:51.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.957+0000 7f3767e00640 1 --2- 172.21.15.77:0/1406281162 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f374c0a2c90 0x7f374c0a5150 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f3758000c00 tx=0x7f3758031510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:51.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.957+0000 7f376cc00640 1 --2- 172.21.15.77:0/1406281162 >> [v2:172.21.15.134:6800/3280466823,v1:172.21.15.134:6801/3280466823] conn(0x7f374c0ade60 0x7f374c0b0300 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:51.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.958+0000 7f3764c00640 1 -- 172.21.15.77:0/1406281162 <== osd.2 v2:172.21.15.134:6800/3280466823 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3754028ea0 con 0x7f374c0ade60 2024-09-10T09:53:51.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.972+0000 7f376ec00640 1 -- 172.21.15.77:0/1406281162 --> [v2:172.21.15.134:6800/3280466823,v1:172.21.15.134:6801/3280466823] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3768062d80 con 0x7f374c0ade60 2024-09-10T09:53:51.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.972+0000 7f3764c00640 1 -- 172.21.15.77:0/1406281162 <== osd.2 v2:172.21.15.134:6800/3280466823 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f3768062d80 con 0x7f374c0ade60 2024-09-10T09:53:51.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.973+0000 7f376ec00640 1 -- 172.21.15.77:0/1406281162 >> [v2:172.21.15.134:6800/3280466823,v1:172.21.15.134:6801/3280466823] conn(0x7f374c0ade60 msgr2=0x7f374c0b0300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:51.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.973+0000 7f376ec00640 1 --2- 172.21.15.77:0/1406281162 >> [v2:172.21.15.134:6800/3280466823,v1:172.21.15.134:6801/3280466823] conn(0x7f374c0ade60 0x7f374c0b0300 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:51.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.973+0000 7f376ec00640 1 -- 172.21.15.77:0/1406281162 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f374c0a2c90 msgr2=0x7f374c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:51.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.973+0000 7f376ec00640 1 --2- 172.21.15.77:0/1406281162 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f374c0a2c90 0x7f374c0a5150 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f3758000c00 tx=0x7f3758031510 comp rx=0 tx=0).stop 2024-09-10T09:53:51.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.973+0000 7f376ec00640 1 -- 172.21.15.77:0/1406281162 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37681049a0 msgr2=0x7f376819ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:51.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.973+0000 7f376ec00640 1 --2- 172.21.15.77:0/1406281162 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37681049a0 0x7f376819ab20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f375400e9f0 tx=0x7f375400eeb0 comp rx=0 tx=0).stop 2024-09-10T09:53:51.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.973+0000 7f376cc00640 1 -- 172.21.15.77:0/1406281162 reap_dead start 2024-09-10T09:53:51.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.973+0000 7f376ec00640 1 -- 172.21.15.77:0/1406281162 shutdown_connections 2024-09-10T09:53:51.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.973+0000 7f376ec00640 1 -- 172.21.15.77:0/1406281162 >> 172.21.15.77:0/1406281162 conn(0x7f37680ffa90 msgr2=0x7f3768100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:51.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.973+0000 7f376ec00640 1 -- 172.21.15.77:0/1406281162 shutdown_connections 2024-09-10T09:53:51.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.973+0000 7f376ec00640 1 -- 172.21.15.77:0/1406281162 wait complete. 2024-09-10T09:53:52.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.814+0000 7f5f41800640 1 -- 172.21.15.77:0/3707806854 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5f3c074e30 msgr2=0x7f5f3c10dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:52.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.814+0000 7f5f41800640 1 --2- 172.21.15.77:0/3707806854 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5f3c074e30 0x7f5f3c10dd40 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5f24009900 tx=0x7f5f2402a480 comp rx=0 tx=0).stop 2024-09-10T09:53:52.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.814+0000 7f5f41800640 1 -- 172.21.15.77:0/3707806854 shutdown_connections 2024-09-10T09:53:52.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.814+0000 7f5f41800640 1 --2- 172.21.15.77:0/3707806854 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5f3c10e280 0x7f5f3c110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:52.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.814+0000 7f5f41800640 1 --2- 172.21.15.77:0/3707806854 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5f3c074e30 0x7f5f3c10dd40 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:52.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.814+0000 7f5f41800640 1 --2- 172.21.15.77:0/3707806854 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5f3c074510 0x7f5f3c0748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:52.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.814+0000 7f5f41800640 1 -- 172.21.15.77:0/3707806854 >> 172.21.15.77:0/3707806854 conn(0x7f5f3c0fd6e0 msgr2=0x7f5f3c0ffb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:52.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.814+0000 7f5f41800640 1 -- 172.21.15.77:0/3707806854 shutdown_connections 2024-09-10T09:53:52.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.815+0000 7f5f41800640 1 -- 172.21.15.77:0/3707806854 wait complete. 2024-09-10T09:53:52.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.815+0000 7f5f41800640 1 Processor -- start 2024-09-10T09:53:52.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.816+0000 7f5f41800640 1 -- start start 2024-09-10T09:53:52.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.816+0000 7f5f41800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5f3c074510 0x7f5f3c19a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:52.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.816+0000 7f5f41800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5f3c074e30 0x7f5f3c19a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:52.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.816+0000 7f5f41800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5f3c10e280 0x7f5f3c19ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:52.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.816+0000 7f5f41800640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5f3c113f70 con 0x7f5f3c074e30 2024-09-10T09:53:52.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.816+0000 7f5f41800640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5f3c113df0 con 0x7f5f3c074510 2024-09-10T09:53:52.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.816+0000 7f5f41800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5f3c1140f0 con 0x7f5f3c10e280 2024-09-10T09:53:52.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.816+0000 7f5f33400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5f3c074e30 0x7f5f3c19a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:52.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.816+0000 7f5f33400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5f3c074e30 0x7f5f3c19a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:53258/0 (socket says 172.21.15.77:53258) 2024-09-10T09:53:52.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.816+0000 7f5f33400640 1 -- 172.21.15.77:0/996297929 learned_addr learned my addr 172.21.15.77:0/996297929 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:53:52.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.816+0000 7f5f40e00640 1 --2- 172.21.15.77:0/996297929 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5f3c10e280 0x7f5f3c19ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:52.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.816+0000 7f5f3be00640 1 --2- 172.21.15.77:0/996297929 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5f3c074510 0x7f5f3c19a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:52.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.817+0000 7f5f33400640 1 -- 172.21.15.77:0/996297929 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5f3c10e280 msgr2=0x7f5f3c19ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:52.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.817+0000 7f5f33400640 1 --2- 172.21.15.77:0/996297929 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5f3c10e280 0x7f5f3c19ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:52.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.817+0000 7f5f33400640 1 -- 172.21.15.77:0/996297929 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5f3c074510 msgr2=0x7f5f3c19a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:52.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.817+0000 7f5f33400640 1 --2- 172.21.15.77:0/996297929 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5f3c074510 0x7f5f3c19a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:52.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.817+0000 7f5f33400640 1 -- 172.21.15.77:0/996297929 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f3c19f460 con 0x7f5f3c074e30 2024-09-10T09:53:52.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.817+0000 7f5f33400640 1 --2- 172.21.15.77:0/996297929 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5f3c074e30 0x7f5f3c19a9f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5f24006010 tx=0x7f5f240042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:52.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.817+0000 7f5f39600640 1 -- 172.21.15.77:0/996297929 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f2403f030 con 0x7f5f3c074e30 2024-09-10T09:53:52.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.817+0000 7f5f39600640 1 -- 172.21.15.77:0/996297929 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f24003ce0 con 0x7f5f3c074e30 2024-09-10T09:53:52.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.817+0000 7f5f41800640 1 -- 172.21.15.77:0/996297929 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f3c19f6f0 con 0x7f5f3c074e30 2024-09-10T09:53:52.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.817+0000 7f5f39600640 1 -- 172.21.15.77:0/996297929 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f24035070 con 0x7f5f3c074e30 2024-09-10T09:53:52.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.817+0000 7f5f41800640 1 -- 172.21.15.77:0/996297929 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f3c1a7030 con 0x7f5f3c074e30 2024-09-10T09:53:52.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.818+0000 7f5f41800640 1 -- 172.21.15.77:0/996297929 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5f3c062980 con 0x7f5f3c074e30 2024-09-10T09:53:52.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.820+0000 7f5f39600640 1 -- 172.21.15.77:0/996297929 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f5f24042030 con 0x7f5f3c074e30 2024-09-10T09:53:52.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.822+0000 7f5f39600640 1 --2- 172.21.15.77:0/996297929 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f5f200a2c10 0x7f5f200a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:52.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.822+0000 7f5f39600640 1 -- 172.21.15.77:0/996297929 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f5f240f00e0 con 0x7f5f3c074e30 2024-09-10T09:53:52.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.822+0000 7f5f3be00640 1 --2- 172.21.15.77:0/996297929 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f5f200a2c10 0x7f5f200a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:52.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.822+0000 7f5f39600640 1 --2- 172.21.15.77:0/996297929 >> [v2:172.21.15.120:6808/3162658038,v1:172.21.15.120:6809/3162658038] conn(0x7f5f200adde0 0x7f5f200b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:52.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.822+0000 7f5f39600640 1 -- 172.21.15.77:0/996297929 --> [v2:172.21.15.120:6808/3162658038,v1:172.21.15.120:6809/3162658038] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5f24040e70 con 0x7f5f200adde0 2024-09-10T09:53:52.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.822+0000 7f5f39600640 1 -- 172.21.15.77:0/996297929 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7f5f240f0530 con 0x7f5f3c074e30 2024-09-10T09:53:52.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.823+0000 7f5f40e00640 1 --2- 172.21.15.77:0/996297929 >> [v2:172.21.15.120:6808/3162658038,v1:172.21.15.120:6809/3162658038] conn(0x7f5f200adde0 0x7f5f200b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:52.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.823+0000 7f5f3be00640 1 --2- 172.21.15.77:0/996297929 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f5f200a2c10 0x7f5f200a50d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c005d00 tx=0x7f5f2c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:52.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.823+0000 7f5f40e00640 1 --2- 172.21.15.77:0/996297929 >> [v2:172.21.15.120:6808/3162658038,v1:172.21.15.120:6809/3162658038] conn(0x7f5f200adde0 0x7f5f200b0280 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:52.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.824+0000 7f5f39600640 1 -- 172.21.15.77:0/996297929 <== osd.4 v2:172.21.15.120:6808/3162658038 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5f24040e70 con 0x7f5f200adde0 2024-09-10T09:53:52.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.842+0000 7f5f41800640 1 -- 172.21.15.77:0/996297929 --> [v2:172.21.15.120:6808/3162658038,v1:172.21.15.120:6809/3162658038] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5f3c062e00 con 0x7f5f200adde0 2024-09-10T09:53:52.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.842+0000 7f5f39600640 1 -- 172.21.15.77:0/996297929 <== osd.4 v2:172.21.15.120:6808/3162658038 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f5f3c062e00 con 0x7f5f200adde0 2024-09-10T09:53:52.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.843+0000 7f5f41800640 1 -- 172.21.15.77:0/996297929 >> [v2:172.21.15.120:6808/3162658038,v1:172.21.15.120:6809/3162658038] conn(0x7f5f200adde0 msgr2=0x7f5f200b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:52.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.843+0000 7f5f41800640 1 --2- 172.21.15.77:0/996297929 >> [v2:172.21.15.120:6808/3162658038,v1:172.21.15.120:6809/3162658038] conn(0x7f5f200adde0 0x7f5f200b0280 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:52.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.843+0000 7f5f41800640 1 -- 172.21.15.77:0/996297929 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f5f200a2c10 msgr2=0x7f5f200a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:52.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.843+0000 7f5f41800640 1 --2- 172.21.15.77:0/996297929 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f5f200a2c10 0x7f5f200a50d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c005d00 tx=0x7f5f2c00b410 comp rx=0 tx=0).stop 2024-09-10T09:53:52.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.843+0000 7f5f41800640 1 -- 172.21.15.77:0/996297929 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5f3c074e30 msgr2=0x7f5f3c19a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:52.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.843+0000 7f5f41800640 1 --2- 172.21.15.77:0/996297929 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5f3c074e30 0x7f5f3c19a9f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5f24006010 tx=0x7f5f240042f0 comp rx=0 tx=0).stop 2024-09-10T09:53:52.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.843+0000 7f5f40e00640 1 -- 172.21.15.77:0/996297929 reap_dead start 2024-09-10T09:53:52.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.843+0000 7f5f41800640 1 -- 172.21.15.77:0/996297929 shutdown_connections 2024-09-10T09:53:52.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.843+0000 7f5f41800640 1 -- 172.21.15.77:0/996297929 >> 172.21.15.77:0/996297929 conn(0x7f5f3c0fd6e0 msgr2=0x7f5f3c0fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:52.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.843+0000 7f5f41800640 1 -- 172.21.15.77:0/996297929 shutdown_connections 2024-09-10T09:53:52.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:51.843+0000 7f5f41800640 1 -- 172.21.15.77:0/996297929 wait complete. 2024-09-10T09:53:52.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:52 smithi077 ceph-mon[30319]: pgmap v350: 129 pgs: 129 active+clean; 645 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-10T09:53:53.125 INFO:teuthology.orchestra.run.smithi077.stdout:137438953540 2024-09-10T09:53:53.126 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-10T09:53:53.130 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:52 smithi134 ceph-mon[35674]: pgmap v350: 129 pgs: 129 active+clean; 645 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-10T09:53:53.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:52 smithi120 ceph-mon[34770]: pgmap v350: 129 pgs: 129 active+clean; 645 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-10T09:53:53.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.381+0000 7fcebc200640 1 -- 172.21.15.77:0/3342263592 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fceb41049a0 msgr2=0x7fceb41089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:53.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.381+0000 7fcebc200640 1 --2- 172.21.15.77:0/3342263592 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fceb41049a0 0x7fceb41089f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fcea0009860 tx=0x7fcea002a340 comp rx=0 tx=0).stop 2024-09-10T09:53:53.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.381+0000 7fcebc200640 1 -- 172.21.15.77:0/3342263592 shutdown_connections 2024-09-10T09:53:53.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.381+0000 7fcebc200640 1 --2- 172.21.15.77:0/3342263592 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fceb4109120 0x7fceb4110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:53.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.381+0000 7fcebc200640 1 --2- 172.21.15.77:0/3342263592 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fceb41049a0 0x7fceb41089f0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:53.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.381+0000 7fcebc200640 1 --2- 172.21.15.77:0/3342263592 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fceb4103ff0 0x7fceb41043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:53.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.381+0000 7fcebc200640 1 -- 172.21.15.77:0/3342263592 >> 172.21.15.77:0/3342263592 conn(0x7fceb40ffa90 msgr2=0x7fceb4101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:53.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.381+0000 7fcebc200640 1 -- 172.21.15.77:0/3342263592 shutdown_connections 2024-09-10T09:53:53.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.381+0000 7fcebc200640 1 -- 172.21.15.77:0/3342263592 wait complete. 2024-09-10T09:53:53.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.382+0000 7fcebc200640 1 Processor -- start 2024-09-10T09:53:53.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.383+0000 7fcebc200640 1 -- start start 2024-09-10T09:53:53.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.383+0000 7fcebc200640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fceb4103ff0 0x7fceb419a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:53.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.383+0000 7fcebc200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fceb41049a0 0x7fceb419abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:53.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.383+0000 7fcebc200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fceb4109120 0x7fceb419ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:53.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.383+0000 7fcebc200640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fceb4114120 con 0x7fceb41049a0 2024-09-10T09:53:53.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.383+0000 7fcebc200640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fceb4113fa0 con 0x7fceb4103ff0 2024-09-10T09:53:53.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.383+0000 7fcebc200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fceb41142a0 con 0x7fceb4109120 2024-09-10T09:53:53.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.383+0000 7fceb9800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fceb4103ff0 0x7fceb419a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:53.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.383+0000 7fceb9800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fceb4103ff0 0x7fceb419a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:41538/0 (socket says 172.21.15.77:41538) 2024-09-10T09:53:53.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.383+0000 7fceb8e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fceb41049a0 0x7fceb419abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:53.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.383+0000 7fceb9800640 1 -- 172.21.15.77:0/1584490437 learned_addr learned my addr 172.21.15.77:0/1584490437 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:53:53.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.383+0000 7fceba200640 1 --2- 172.21.15.77:0/1584490437 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fceb4109120 0x7fceb419ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:53.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.384+0000 7fceb9800640 1 -- 172.21.15.77:0/1584490437 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fceb4109120 msgr2=0x7fceb419ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:53.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.384+0000 7fceb9800640 1 --2- 172.21.15.77:0/1584490437 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fceb4109120 0x7fceb419ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:53.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.384+0000 7fceb9800640 1 -- 172.21.15.77:0/1584490437 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fceb41049a0 msgr2=0x7fceb419abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:53.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.384+0000 7fceb9800640 1 --2- 172.21.15.77:0/1584490437 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fceb41049a0 0x7fceb419abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:53.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.384+0000 7fceb9800640 1 -- 172.21.15.77:0/1584490437 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fceb419f650 con 0x7fceb4103ff0 2024-09-10T09:53:53.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.384+0000 7fceb9800640 1 --2- 172.21.15.77:0/1584490437 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fceb4103ff0 0x7fceb419a6a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fcea800e9c0 tx=0x7fcea800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:53.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.384+0000 7fceb2000640 1 -- 172.21.15.77:0/1584490437 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea800ccc0 con 0x7fceb4103ff0 2024-09-10T09:53:53.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.384+0000 7fcebc200640 1 -- 172.21.15.77:0/1584490437 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fceb419f940 con 0x7fceb4103ff0 2024-09-10T09:53:53.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.384+0000 7fceb2000640 1 -- 172.21.15.77:0/1584490437 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcea8005500 con 0x7fceb4103ff0 2024-09-10T09:53:53.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.384+0000 7fceb2000640 1 -- 172.21.15.77:0/1584490437 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea8010430 con 0x7fceb4103ff0 2024-09-10T09:53:53.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.384+0000 7fcebc200640 1 -- 172.21.15.77:0/1584490437 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fceb41a7220 con 0x7fceb4103ff0 2024-09-10T09:53:53.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.385+0000 7fcebc200640 1 -- 172.21.15.77:0/1584490437 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fceb40628c0 con 0x7fceb4103ff0 2024-09-10T09:53:53.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.388+0000 7fceb2000640 1 -- 172.21.15.77:0/1584490437 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fcea80046c0 con 0x7fceb4103ff0 2024-09-10T09:53:53.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.389+0000 7fceb2000640 1 --2- 172.21.15.77:0/1584490437 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fce9c0a2c10 0x7fce9c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:53.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.389+0000 7fceb2000640 1 -- 172.21.15.77:0/1584490437 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7fcea80cfca0 con 0x7fceb4103ff0 2024-09-10T09:53:53.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.389+0000 7fceb8e00640 1 --2- 172.21.15.77:0/1584490437 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fce9c0a2c10 0x7fce9c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:53.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.389+0000 7fceb2000640 1 --2- 172.21.15.77:0/1584490437 >> [v2:172.21.15.77:6826/1210619387,v1:172.21.15.77:6827/1210619387] conn(0x7fce9c0adde0 0x7fce9c0b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:53.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.389+0000 7fceb2000640 1 -- 172.21.15.77:0/1584490437 --> [v2:172.21.15.77:6826/1210619387,v1:172.21.15.77:6827/1210619387] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fce9c0b0950 con 0x7fce9c0adde0 2024-09-10T09:53:53.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.389+0000 7fceb2000640 1 -- 172.21.15.77:0/1584490437 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7fcea80d0050 con 0x7fceb4103ff0 2024-09-10T09:53:53.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.390+0000 7fceba200640 1 --2- 172.21.15.77:0/1584490437 >> [v2:172.21.15.77:6826/1210619387,v1:172.21.15.77:6827/1210619387] conn(0x7fce9c0adde0 0x7fce9c0b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:53.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.390+0000 7fceb8e00640 1 --2- 172.21.15.77:0/1584490437 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fce9c0a2c10 0x7fce9c0a50d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fcea002a930 tx=0x7fcea00056d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:53.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.390+0000 7fceba200640 1 --2- 172.21.15.77:0/1584490437 >> [v2:172.21.15.77:6826/1210619387,v1:172.21.15.77:6827/1210619387] conn(0x7fce9c0adde0 0x7fce9c0b0280 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:53.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.391+0000 7fceb2000640 1 -- 172.21.15.77:0/1584490437 <== osd.9 v2:172.21.15.77:6826/1210619387 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fce9c0b0950 con 0x7fce9c0adde0 2024-09-10T09:53:53.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.406+0000 7fcebc200640 1 -- 172.21.15.77:0/1584490437 --> [v2:172.21.15.77:6826/1210619387,v1:172.21.15.77:6827/1210619387] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fceb4062cc0 con 0x7fce9c0adde0 2024-09-10T09:53:53.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.407+0000 7fceb2000640 1 -- 172.21.15.77:0/1584490437 <== osd.9 v2:172.21.15.77:6826/1210619387 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fceb4062cc0 con 0x7fce9c0adde0 2024-09-10T09:53:53.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.407+0000 7fcebc200640 1 -- 172.21.15.77:0/1584490437 >> [v2:172.21.15.77:6826/1210619387,v1:172.21.15.77:6827/1210619387] conn(0x7fce9c0adde0 msgr2=0x7fce9c0b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:53.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.407+0000 7fcebc200640 1 --2- 172.21.15.77:0/1584490437 >> [v2:172.21.15.77:6826/1210619387,v1:172.21.15.77:6827/1210619387] conn(0x7fce9c0adde0 0x7fce9c0b0280 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:53.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.407+0000 7fcebc200640 1 -- 172.21.15.77:0/1584490437 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fce9c0a2c10 msgr2=0x7fce9c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:53.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.407+0000 7fcebc200640 1 --2- 172.21.15.77:0/1584490437 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fce9c0a2c10 0x7fce9c0a50d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fcea002a930 tx=0x7fcea00056d0 comp rx=0 tx=0).stop 2024-09-10T09:53:53.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.407+0000 7fcebc200640 1 -- 172.21.15.77:0/1584490437 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fceb4103ff0 msgr2=0x7fceb419a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:53.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.407+0000 7fcebc200640 1 --2- 172.21.15.77:0/1584490437 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fceb4103ff0 0x7fceb419a6a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fcea800e9c0 tx=0x7fcea800ee80 comp rx=0 tx=0).stop 2024-09-10T09:53:53.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.407+0000 7fceba200640 1 -- 172.21.15.77:0/1584490437 reap_dead start 2024-09-10T09:53:53.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.407+0000 7fcebc200640 1 -- 172.21.15.77:0/1584490437 shutdown_connections 2024-09-10T09:53:53.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.407+0000 7fcebc200640 1 -- 172.21.15.77:0/1584490437 >> 172.21.15.77:0/1584490437 conn(0x7fceb40ffa90 msgr2=0x7fceb4100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:53.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.408+0000 7fcebc200640 1 -- 172.21.15.77:0/1584490437 shutdown_connections 2024-09-10T09:53:53.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.408+0000 7fcebc200640 1 -- 172.21.15.77:0/1584490437 wait complete. 2024-09-10T09:53:54.080 INFO:teuthology.orchestra.run.smithi077.stdout:279172874282 2024-09-10T09:53:54.081 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-09-10T09:53:54.511 INFO:teuthology.orchestra.run.smithi077.stdout:77309411408 2024-09-10T09:53:54.511 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-10T09:53:54.615 INFO:teuthology.orchestra.run.smithi077.stdout:34359738458 2024-09-10T09:53:54.616 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-10T09:53:55.124 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:54 smithi077 ceph-mon[30319]: pgmap v351: 129 pgs: 129 active+clean; 645 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-10T09:53:55.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:54 smithi134 ceph-mon[35674]: pgmap v351: 129 pgs: 129 active+clean; 645 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-10T09:53:55.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:54 smithi120 ceph-mon[34770]: pgmap v351: 129 pgs: 129 active+clean; 645 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-10T09:53:55.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:55.234+0000 7f940b200640 1 -- 172.21.15.77:0/3144228006 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9404074590 msgr2=0x7f9404074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:55.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:55.234+0000 7f940b200640 1 --2- 172.21.15.77:0/3144228006 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9404074590 0x7f9404074970 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f93f4009900 tx=0x7f93f402a480 comp rx=0 tx=0).stop 2024-09-10T09:53:55.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:55.234+0000 7f940b200640 1 -- 172.21.15.77:0/3144228006 shutdown_connections 2024-09-10T09:53:55.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:55.234+0000 7f940b200640 1 --2- 172.21.15.77:0/3144228006 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9404112b90 0x7f9404114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:55.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:55.234+0000 7f940b200640 1 --2- 172.21.15.77:0/3144228006 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9404074eb0 0x7f9404112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:55.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:55.234+0000 7f940b200640 1 --2- 172.21.15.77:0/3144228006 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9404074590 0x7f9404074970 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:55.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:55.234+0000 7f940b200640 1 -- 172.21.15.77:0/3144228006 >> 172.21.15.77:0/3144228006 conn(0x7f94040fd8c0 msgr2=0x7f94040ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:55.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:55.234+0000 7f940b200640 1 -- 172.21.15.77:0/3144228006 shutdown_connections 2024-09-10T09:53:55.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:55.234+0000 7f940b200640 1 -- 172.21.15.77:0/3144228006 wait complete. 2024-09-10T09:53:55.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:55.235+0000 7f940b200640 1 Processor -- start 2024-09-10T09:53:55.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:55.236+0000 7f940b200640 1 -- start start 2024-09-10T09:53:55.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:55.236+0000 7f940b200640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9404074590 0x7f940419eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:55.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:55.236+0000 7f940b200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9404074eb0 0x7f940419f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:55.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:55.236+0000 7f940b200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9404112b90 0x7f94041a33f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:55.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:55.236+0000 7f940b200640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f94041185a0 con 0x7f9404112b90 2024-09-10T09:53:55.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:55.236+0000 7f940b200640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9404118420 con 0x7f9404074590 2024-09-10T09:53:55.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:55.236+0000 7f940b200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9404118720 con 0x7f9404074eb0 2024-09-10T09:53:55.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:55.236+0000 7f9409200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9404112b90 0x7f94041a33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:55.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:55.236+0000 7f9409200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9404112b90 0x7f94041a33f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:55932/0 (socket says 172.21.15.77:55932) 2024-09-10T09:53:55.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:55.236+0000 7f9403400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9404074eb0 0x7f940419f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:55.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:55.236+0000 7f9409200640 1 -- 172.21.15.77:0/4283467381 learned_addr learned my addr 172.21.15.77:0/4283467381 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:53:55.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:55.236+0000 7f9403e00640 1 --2- 172.21.15.77:0/4283467381 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9404074590 0x7f940419eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:55.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:55.237+0000 7f9409200640 1 -- 172.21.15.77:0/4283467381 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9404074eb0 msgr2=0x7f940419f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:55.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:55.237+0000 7f9409200640 1 --2- 172.21.15.77:0/4283467381 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9404074eb0 0x7f940419f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:55.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:55.237+0000 7f9409200640 1 -- 172.21.15.77:0/4283467381 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9404074590 msgr2=0x7f940419eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:55.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:55.237+0000 7f9409200640 1 --2- 172.21.15.77:0/4283467381 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9404074590 0x7f940419eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:55.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:55.237+0000 7f9409200640 1 -- 172.21.15.77:0/4283467381 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94041a3b20 con 0x7f9404112b90 2024-09-10T09:53:55.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:55.237+0000 7f9409200640 1 --2- 172.21.15.77:0/4283467381 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9404112b90 0x7f94041a33f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f93fc00dac0 tx=0x7f93fc00df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:55.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:55.237+0000 7f9400c00640 1 -- 172.21.15.77:0/4283467381 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93fc017070 con 0x7f9404112b90 2024-09-10T09:53:55.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:55.237+0000 7f940b200640 1 -- 172.21.15.77:0/4283467381 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94041a3e10 con 0x7f9404112b90 2024-09-10T09:53:55.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:55.237+0000 7f9400c00640 1 -- 172.21.15.77:0/4283467381 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93fc00be30 con 0x7f9404112b90 2024-09-10T09:53:55.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:55.237+0000 7f9400c00640 1 -- 172.21.15.77:0/4283467381 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93fc020060 con 0x7f9404112b90 2024-09-10T09:53:55.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:55.237+0000 7f940b200640 1 -- 172.21.15.77:0/4283467381 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94041ab6a0 con 0x7f9404112b90 2024-09-10T09:53:55.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:55.238+0000 7f940b200640 1 -- 172.21.15.77:0/4283467381 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f94040628c0 con 0x7f9404112b90 2024-09-10T09:53:55.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:55.240+0000 7f9400c00640 1 -- 172.21.15.77:0/4283467381 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f93fc010460 con 0x7f9404112b90 2024-09-10T09:53:55.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:55.242+0000 7f9400c00640 1 --2- 172.21.15.77:0/4283467381 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f93e80a2c10 0x7f93e80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:55.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:55.242+0000 7f9400c00640 1 -- 172.21.15.77:0/4283467381 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f93fc0cf7f0 con 0x7f9404112b90 2024-09-10T09:53:55.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:55.242+0000 7f9403e00640 1 --2- 172.21.15.77:0/4283467381 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f93e80a2c10 0x7f93e80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:55.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:55.242+0000 7f9400c00640 1 --2- 172.21.15.77:0/4283467381 >> [v2:172.21.15.120:6824/754733885,v1:172.21.15.120:6825/754733885] conn(0x7f93e80adde0 0x7f93e80b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:55.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:55.242+0000 7f9400c00640 1 -- 172.21.15.77:0/4283467381 --> [v2:172.21.15.120:6824/754733885,v1:172.21.15.120:6825/754733885] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f93e80b0950 con 0x7f93e80adde0 2024-09-10T09:53:55.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:55.242+0000 7f9400c00640 1 -- 172.21.15.77:0/4283467381 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7f93fc0cfba0 con 0x7f9404112b90 2024-09-10T09:53:55.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:55.243+0000 7f9403400640 1 --2- 172.21.15.77:0/4283467381 >> [v2:172.21.15.120:6824/754733885,v1:172.21.15.120:6825/754733885] conn(0x7f93e80adde0 0x7f93e80b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:55.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:55.243+0000 7f9403e00640 1 --2- 172.21.15.77:0/4283467381 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f93e80a2c10 0x7f93e80a50d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f93f4000c00 tx=0x7f93f4005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:55.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:55.243+0000 7f9403400640 1 --2- 172.21.15.77:0/4283467381 >> [v2:172.21.15.120:6824/754733885,v1:172.21.15.120:6825/754733885] conn(0x7f93e80adde0 0x7f93e80b0280 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:55.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:55.244+0000 7f9400c00640 1 -- 172.21.15.77:0/4283467381 <== osd.10 v2:172.21.15.120:6824/754733885 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f93e80b0950 con 0x7f93e80adde0 2024-09-10T09:53:55.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:55.258+0000 7f940b200640 1 -- 172.21.15.77:0/4283467381 --> [v2:172.21.15.120:6824/754733885,v1:172.21.15.120:6825/754733885] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9404062d40 con 0x7f93e80adde0 2024-09-10T09:53:55.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:55.259+0000 7f9400c00640 1 -- 172.21.15.77:0/4283467381 <== osd.10 v2:172.21.15.120:6824/754733885 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9404062d40 con 0x7f93e80adde0 2024-09-10T09:53:55.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:55.259+0000 7f940b200640 1 -- 172.21.15.77:0/4283467381 >> [v2:172.21.15.120:6824/754733885,v1:172.21.15.120:6825/754733885] conn(0x7f93e80adde0 msgr2=0x7f93e80b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:55.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:55.259+0000 7f940b200640 1 --2- 172.21.15.77:0/4283467381 >> [v2:172.21.15.120:6824/754733885,v1:172.21.15.120:6825/754733885] conn(0x7f93e80adde0 0x7f93e80b0280 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:55.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:55.259+0000 7f940b200640 1 -- 172.21.15.77:0/4283467381 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f93e80a2c10 msgr2=0x7f93e80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:55.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:55.259+0000 7f940b200640 1 --2- 172.21.15.77:0/4283467381 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f93e80a2c10 0x7f93e80a50d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f93f4000c00 tx=0x7f93f4005b50 comp rx=0 tx=0).stop 2024-09-10T09:53:55.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:55.259+0000 7f940b200640 1 -- 172.21.15.77:0/4283467381 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9404112b90 msgr2=0x7f94041a33f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:55.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:55.259+0000 7f940b200640 1 --2- 172.21.15.77:0/4283467381 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9404112b90 0x7f94041a33f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f93fc00dac0 tx=0x7f93fc00df80 comp rx=0 tx=0).stop 2024-09-10T09:53:55.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:55.259+0000 7f9409200640 1 -- 172.21.15.77:0/4283467381 reap_dead start 2024-09-10T09:53:55.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:55.260+0000 7f940b200640 1 -- 172.21.15.77:0/4283467381 shutdown_connections 2024-09-10T09:53:55.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:55.260+0000 7f940b200640 1 -- 172.21.15.77:0/4283467381 >> 172.21.15.77:0/4283467381 conn(0x7f94040fd8c0 msgr2=0x7f94040ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:55.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:55.260+0000 7f940b200640 1 -- 172.21.15.77:0/4283467381 shutdown_connections 2024-09-10T09:53:55.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:55.260+0000 7f940b200640 1 -- 172.21.15.77:0/4283467381 wait complete. 2024-09-10T09:53:55.618 INFO:teuthology.orchestra.run.smithi077.stdout:214748364853 2024-09-10T09:53:55.619 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-10T09:53:55.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.481+0000 7f39f5600640 1 -- 172.21.15.77:0/4141788960 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39f0074510 msgr2=0x7f39f00748f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:55.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.481+0000 7f39f5600640 1 --2- 172.21.15.77:0/4141788960 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39f0074510 0x7f39f00748f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f39e0009900 tx=0x7f39e002a480 comp rx=0 tx=0).stop 2024-09-10T09:53:55.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.481+0000 7f39f5600640 1 -- 172.21.15.77:0/4141788960 shutdown_connections 2024-09-10T09:53:55.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.481+0000 7f39f5600640 1 --2- 172.21.15.77:0/4141788960 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f39f010e280 0x7f39f0110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:55.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.481+0000 7f39f5600640 1 --2- 172.21.15.77:0/4141788960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39f0074e30 0x7f39f010dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:55.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.481+0000 7f39f5600640 1 --2- 172.21.15.77:0/4141788960 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39f0074510 0x7f39f00748f0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:55.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.481+0000 7f39f5600640 1 -- 172.21.15.77:0/4141788960 >> 172.21.15.77:0/4141788960 conn(0x7f39f00fd740 msgr2=0x7f39f00ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:55.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.481+0000 7f39f5600640 1 -- 172.21.15.77:0/4141788960 shutdown_connections 2024-09-10T09:53:55.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.481+0000 7f39f5600640 1 -- 172.21.15.77:0/4141788960 wait complete. 2024-09-10T09:53:55.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.482+0000 7f39f5600640 1 Processor -- start 2024-09-10T09:53:55.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.482+0000 7f39f5600640 1 -- start start 2024-09-10T09:53:55.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.482+0000 7f39f5600640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39f0074510 0x7f39f019a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:55.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.482+0000 7f39f5600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39f0074e30 0x7f39f019a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:55.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.482+0000 7f39f5600640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f39f010e280 0x7f39f019ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:55.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.482+0000 7f39f5600640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f39f0114000 con 0x7f39f0074510 2024-09-10T09:53:55.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.482+0000 7f39f5600640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f39f0113e80 con 0x7f39f010e280 2024-09-10T09:53:55.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.482+0000 7f39f5600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f39f0114180 con 0x7f39f0074e30 2024-09-10T09:53:55.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.483+0000 7f39efe00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39f0074510 0x7f39f019a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:55.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.483+0000 7f39f4c00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f39f010e280 0x7f39f019ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:55.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.483+0000 7f39f4c00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f39f010e280 0x7f39f019ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:41566/0 (socket says 172.21.15.77:41566) 2024-09-10T09:53:55.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.483+0000 7f39efe00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39f0074510 0x7f39f019a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:53232/0 (socket says 172.21.15.77:53232) 2024-09-10T09:53:55.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.483+0000 7f39f4c00640 1 -- 172.21.15.77:0/4029575189 learned_addr learned my addr 172.21.15.77:0/4029575189 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:53:55.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.483+0000 7f39ef400640 1 --2- 172.21.15.77:0/4029575189 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39f0074e30 0x7f39f019a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:55.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.483+0000 7f39f4c00640 1 -- 172.21.15.77:0/4029575189 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39f0074e30 msgr2=0x7f39f019a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:55.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.483+0000 7f39f4c00640 1 --2- 172.21.15.77:0/4029575189 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39f0074e30 0x7f39f019a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:55.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.483+0000 7f39f4c00640 1 -- 172.21.15.77:0/4029575189 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39f0074510 msgr2=0x7f39f019a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:55.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.483+0000 7f39f4c00640 1 --2- 172.21.15.77:0/4029575189 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39f0074510 0x7f39f019a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:55.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.483+0000 7f39f4c00640 1 -- 172.21.15.77:0/4029575189 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39f019f450 con 0x7f39f010e280 2024-09-10T09:53:55.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.483+0000 7f39efe00640 1 --2- 172.21.15.77:0/4029575189 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39f0074510 0x7f39f019a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:53:55.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.484+0000 7f39f4c00640 1 --2- 172.21.15.77:0/4029575189 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f39f010e280 0x7f39f019ed70 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f39e800dac0 tx=0x7f39e800df80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:55.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.484+0000 7f39ecc00640 1 -- 172.21.15.77:0/4029575189 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e8017070 con 0x7f39f010e280 2024-09-10T09:53:55.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.484+0000 7f39ecc00640 1 -- 172.21.15.77:0/4029575189 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39e800be30 con 0x7f39f010e280 2024-09-10T09:53:55.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.484+0000 7f39f5600640 1 -- 172.21.15.77:0/4029575189 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39f019f740 con 0x7f39f010e280 2024-09-10T09:53:55.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.484+0000 7f39ecc00640 1 -- 172.21.15.77:0/4029575189 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e800f890 con 0x7f39f010e280 2024-09-10T09:53:55.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.484+0000 7f39f5600640 1 -- 172.21.15.77:0/4029575189 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39f01a7020 con 0x7f39f010e280 2024-09-10T09:53:55.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.485+0000 7f39f5600640 1 -- 172.21.15.77:0/4029575189 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f39f0062980 con 0x7f39f010e280 2024-09-10T09:53:55.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.487+0000 7f39ecc00640 1 -- 172.21.15.77:0/4029575189 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f39e800fa30 con 0x7f39f010e280 2024-09-10T09:53:55.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.488+0000 7f39ecc00640 1 --2- 172.21.15.77:0/4029575189 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f39d40a2c10 0x7f39d40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:55.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.488+0000 7f39ecc00640 1 -- 172.21.15.77:0/4029575189 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f39e80cf790 con 0x7f39f010e280 2024-09-10T09:53:55.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.489+0000 7f39efe00640 1 --2- 172.21.15.77:0/4029575189 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f39d40a2c10 0x7f39d40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:55.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.489+0000 7f39ecc00640 1 --2- 172.21.15.77:0/4029575189 >> [v2:172.21.15.134:6816/2081686544,v1:172.21.15.134:6817/2081686544] conn(0x7f39d40adde0 0x7f39d40b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:55.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.489+0000 7f39ecc00640 1 -- 172.21.15.77:0/4029575189 --> [v2:172.21.15.134:6816/2081686544,v1:172.21.15.134:6817/2081686544] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f39e800fec0 con 0x7f39d40adde0 2024-09-10T09:53:55.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.489+0000 7f39ecc00640 1 -- 172.21.15.77:0/4029575189 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7f39e80cfb40 con 0x7f39f010e280 2024-09-10T09:53:55.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.489+0000 7f39ef400640 1 --2- 172.21.15.77:0/4029575189 >> [v2:172.21.15.134:6816/2081686544,v1:172.21.15.134:6817/2081686544] conn(0x7f39d40adde0 0x7f39d40b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:55.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.489+0000 7f39efe00640 1 --2- 172.21.15.77:0/4029575189 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f39d40a2c10 0x7f39d40a50d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f39e0000c00 tx=0x7f39e0031510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:55.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.490+0000 7f39ef400640 1 --2- 172.21.15.77:0/4029575189 >> [v2:172.21.15.134:6816/2081686544,v1:172.21.15.134:6817/2081686544] conn(0x7f39d40adde0 0x7f39d40b0280 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:55.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.491+0000 7f39ecc00640 1 -- 172.21.15.77:0/4029575189 <== osd.8 v2:172.21.15.134:6816/2081686544 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f39e800fec0 con 0x7f39d40adde0 2024-09-10T09:53:55.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.505+0000 7f39f5600640 1 -- 172.21.15.77:0/4029575189 --> [v2:172.21.15.134:6816/2081686544,v1:172.21.15.134:6817/2081686544] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f39f0062e00 con 0x7f39d40adde0 2024-09-10T09:53:55.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.506+0000 7f39ecc00640 1 -- 172.21.15.77:0/4029575189 <== osd.8 v2:172.21.15.134:6816/2081686544 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f39f0062e00 con 0x7f39d40adde0 2024-09-10T09:53:55.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.506+0000 7f39f5600640 1 -- 172.21.15.77:0/4029575189 >> [v2:172.21.15.134:6816/2081686544,v1:172.21.15.134:6817/2081686544] conn(0x7f39d40adde0 msgr2=0x7f39d40b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:55.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.506+0000 7f39f5600640 1 --2- 172.21.15.77:0/4029575189 >> [v2:172.21.15.134:6816/2081686544,v1:172.21.15.134:6817/2081686544] conn(0x7f39d40adde0 0x7f39d40b0280 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:55.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.506+0000 7f39f5600640 1 -- 172.21.15.77:0/4029575189 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f39d40a2c10 msgr2=0x7f39d40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:55.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.506+0000 7f39f5600640 1 --2- 172.21.15.77:0/4029575189 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f39d40a2c10 0x7f39d40a50d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f39e0000c00 tx=0x7f39e0031510 comp rx=0 tx=0).stop 2024-09-10T09:53:55.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.506+0000 7f39f5600640 1 -- 172.21.15.77:0/4029575189 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f39f010e280 msgr2=0x7f39f019ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:55.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.506+0000 7f39f5600640 1 --2- 172.21.15.77:0/4029575189 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f39f010e280 0x7f39f019ed70 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f39e800dac0 tx=0x7f39e800df80 comp rx=0 tx=0).stop 2024-09-10T09:53:55.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.506+0000 7f39f4c00640 1 -- 172.21.15.77:0/4029575189 reap_dead start 2024-09-10T09:53:55.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.506+0000 7f39f5600640 1 -- 172.21.15.77:0/4029575189 shutdown_connections 2024-09-10T09:53:55.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.506+0000 7f39f5600640 1 -- 172.21.15.77:0/4029575189 >> 172.21.15.77:0/4029575189 conn(0x7f39f00fd740 msgr2=0x7f39f00fee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:55.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.506+0000 7f39f5600640 1 -- 172.21.15.77:0/4029575189 shutdown_connections 2024-09-10T09:53:55.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:50.506+0000 7f39f5600640 1 -- 172.21.15.77:0/4029575189 wait complete. 2024-09-10T09:53:56.800 INFO:teuthology.orchestra.run.smithi077.stdout:244813135920 2024-09-10T09:53:56.801 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-09-10T09:53:57.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:56 smithi134 ceph-mon[35674]: pgmap v352: 129 pgs: 129 active+clean; 645 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-09-10T09:53:57.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:56 smithi120 ceph-mon[34770]: pgmap v352: 129 pgs: 129 active+clean; 645 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-09-10T09:53:57.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:56 smithi077 ceph-mon[30319]: pgmap v352: 129 pgs: 129 active+clean; 645 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-09-10T09:53:57.555 INFO:teuthology.orchestra.run.smithi077.stdout:193273528379 2024-09-10T09:53:57.556 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-10T09:53:57.600 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:53:58.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.642+0000 7f78c5e00640 1 -- 172.21.15.77:0/3407605193 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f78c0102360 msgr2=0x7f78c010e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:58.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.642+0000 7f78c5e00640 1 --2- 172.21.15.77:0/3407605193 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f78c0102360 0x7f78c010e890 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f78b8009cd0 tx=0x7f78b802a910 comp rx=0 tx=0).stop 2024-09-10T09:53:58.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.643+0000 7f78c5e00640 1 -- 172.21.15.77:0/3407605193 shutdown_connections 2024-09-10T09:53:58.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.643+0000 7f78c5e00640 1 --2- 172.21.15.77:0/3407605193 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f78c0102360 0x7f78c010e890 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:58.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.643+0000 7f78c5e00640 1 --2- 172.21.15.77:0/3407605193 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78c01019c0 0x7f78c0101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:58.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.643+0000 7f78c5e00640 1 --2- 172.21.15.77:0/3407605193 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f78c01079c0 0x7f78c0107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:58.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.643+0000 7f78c5e00640 1 -- 172.21.15.77:0/3407605193 >> 172.21.15.77:0/3407605193 conn(0x7f78c00fd740 msgr2=0x7f78c00ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:58.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.643+0000 7f78c5e00640 1 -- 172.21.15.77:0/3407605193 shutdown_connections 2024-09-10T09:53:58.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.643+0000 7f78c5e00640 1 -- 172.21.15.77:0/3407605193 wait complete. 2024-09-10T09:53:58.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.644+0000 7f78c5e00640 1 Processor -- start 2024-09-10T09:53:58.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.644+0000 7f78c5e00640 1 -- start start 2024-09-10T09:53:58.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.644+0000 7f78c5e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f78c01019c0 0x7f78c019a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:58.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.644+0000 7f78c5e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78c0102360 0x7f78c019aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:58.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.644+0000 7f78c5e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f78c01079c0 0x7f78c019edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:58.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.644+0000 7f78c5e00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f78c0111e00 con 0x7f78c01079c0 2024-09-10T09:53:58.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.644+0000 7f78c5e00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f78c0111c80 con 0x7f78c01019c0 2024-09-10T09:53:58.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.644+0000 7f78c5e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f78c0111f80 con 0x7f78c0102360 2024-09-10T09:53:58.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.644+0000 7f78bfe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78c0102360 0x7f78c019aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:58.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.644+0000 7f78bfe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78c0102360 0x7f78c019aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:52018/0 (socket says 172.21.15.77:52018) 2024-09-10T09:53:58.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.644+0000 7f78bfe00640 1 -- 172.21.15.77:0/3132265623 learned_addr learned my addr 172.21.15.77:0/3132265623 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:53:58.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.644+0000 7f78c5400640 1 --2- 172.21.15.77:0/3132265623 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f78c01079c0 0x7f78c019edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:58.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.644+0000 7f78c4a00640 1 --2- 172.21.15.77:0/3132265623 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f78c01019c0 0x7f78c019a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:58.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.645+0000 7f78bfe00640 1 -- 172.21.15.77:0/3132265623 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f78c01019c0 msgr2=0x7f78c019a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:58.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.645+0000 7f78bfe00640 1 --2- 172.21.15.77:0/3132265623 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f78c01019c0 0x7f78c019a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:58.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.645+0000 7f78bfe00640 1 -- 172.21.15.77:0/3132265623 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f78c01079c0 msgr2=0x7f78c019edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:58.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.645+0000 7f78bfe00640 1 --2- 172.21.15.77:0/3132265623 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f78c01079c0 0x7f78c019edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:58.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.645+0000 7f78bfe00640 1 -- 172.21.15.77:0/3132265623 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78c019f4b0 con 0x7f78c0102360 2024-09-10T09:53:58.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.649+0000 7f78bfe00640 1 --2- 172.21.15.77:0/3132265623 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78c0102360 0x7f78c019aa40 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f78ac00b450 tx=0x7f78ac00b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:58.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.649+0000 7f78bd600640 1 -- 172.21.15.77:0/3132265623 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78ac015020 con 0x7f78c0102360 2024-09-10T09:53:58.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.650+0000 7f78c5e00640 1 -- 172.21.15.77:0/3132265623 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78c019f7a0 con 0x7f78c0102360 2024-09-10T09:53:58.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.650+0000 7f78bd600640 1 -- 172.21.15.77:0/3132265623 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78ac011d70 con 0x7f78c0102360 2024-09-10T09:53:58.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.650+0000 7f78bd600640 1 -- 172.21.15.77:0/3132265623 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78ac00f340 con 0x7f78c0102360 2024-09-10T09:53:58.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.650+0000 7f78c5e00640 1 -- 172.21.15.77:0/3132265623 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78c01a7080 con 0x7f78c0102360 2024-09-10T09:53:58.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.651+0000 7f78c5e00640 1 -- 172.21.15.77:0/3132265623 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78c0101e20 con 0x7f78c0102360 2024-09-10T09:53:58.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.653+0000 7f78bd600640 1 -- 172.21.15.77:0/3132265623 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f78ac018020 con 0x7f78c0102360 2024-09-10T09:53:58.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.655+0000 7f78bd600640 1 --2- 172.21.15.77:0/3132265623 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f78a80a2c10 0x7f78a80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:58.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.655+0000 7f78bd600640 1 -- 172.21.15.77:0/3132265623 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f78ac0cf3c0 con 0x7f78c0102360 2024-09-10T09:53:58.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.655+0000 7f78c4a00640 1 --2- 172.21.15.77:0/3132265623 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f78a80a2c10 0x7f78a80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:58.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.656+0000 7f78c4a00640 1 --2- 172.21.15.77:0/3132265623 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f78a80a2c10 0x7f78a80a50d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f78b4005d00 tx=0x7f78b400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:58.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.656+0000 7f78bd600640 1 -- 172.21.15.77:0/3132265623 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f78ac01f020 con 0x7f78c0102360 2024-09-10T09:53:58.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.839+0000 7f78c5e00640 1 -- 172.21.15.77:0/3132265623 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f78c019b860 con 0x7f78c0102360 2024-09-10T09:53:58.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.839+0000 7f78bd600640 1 -- 172.21.15.77:0/3132265623 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f78c019b860 con 0x7f78c0102360 2024-09-10T09:53:58.840 INFO:teuthology.orchestra.run.smithi077.stdout:55834574935 2024-09-10T09:53:58.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.842+0000 7f78c5e00640 1 -- 172.21.15.77:0/3132265623 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f78a80a2c10 msgr2=0x7f78a80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:58.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.842+0000 7f78c5e00640 1 --2- 172.21.15.77:0/3132265623 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f78a80a2c10 0x7f78a80a50d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f78b4005d00 tx=0x7f78b400b410 comp rx=0 tx=0).stop 2024-09-10T09:53:58.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.842+0000 7f78c5e00640 1 -- 172.21.15.77:0/3132265623 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78c0102360 msgr2=0x7f78c019aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:58.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.842+0000 7f78c5e00640 1 --2- 172.21.15.77:0/3132265623 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78c0102360 0x7f78c019aa40 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f78ac00b450 tx=0x7f78ac00b910 comp rx=0 tx=0).stop 2024-09-10T09:53:58.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.842+0000 7f78c5e00640 1 -- 172.21.15.77:0/3132265623 shutdown_connections 2024-09-10T09:53:58.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.842+0000 7f78c5e00640 1 --2- 172.21.15.77:0/3132265623 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f78a80a2c10 0x7f78a80a50d0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:58.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.842+0000 7f78c5e00640 1 --2- 172.21.15.77:0/3132265623 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f78c01079c0 0x7f78c019edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:58.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.842+0000 7f78c5e00640 1 --2- 172.21.15.77:0/3132265623 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78c0102360 0x7f78c019aa40 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:58.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.842+0000 7f78c5e00640 1 --2- 172.21.15.77:0/3132265623 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f78c01019c0 0x7f78c019a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:58.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.842+0000 7f78c5e00640 1 -- 172.21.15.77:0/3132265623 >> 172.21.15.77:0/3132265623 conn(0x7f78c00fd740 msgr2=0x7f78c010d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:58.844 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.842+0000 7f78c5e00640 1 -- 172.21.15.77:0/3132265623 shutdown_connections 2024-09-10T09:53:58.844 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.842+0000 7f78c5e00640 1 -- 172.21.15.77:0/3132265623 wait complete. 2024-09-10T09:53:59.125 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:58 smithi077 ceph-mon[30319]: pgmap v353: 129 pgs: 129 active+clean; 645 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2024-09-10T09:53:59.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:58 smithi134 ceph-mon[35674]: pgmap v353: 129 pgs: 129 active+clean; 645 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2024-09-10T09:53:59.164 INFO:teuthology.orchestra.run.smithi077.stdout:300647710756 2024-09-10T09:53:59.164 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-09-10T09:53:59.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:58 smithi120 ceph-mon[34770]: pgmap v353: 129 pgs: 129 active+clean; 645 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2024-09-10T09:53:59.624 INFO:teuthology.orchestra.run.smithi077.stdout:335007449120 2024-09-10T09:53:59.625 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-09-10T09:54:00.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:59 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/3132265623' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-10T09:54:00.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:59 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.b", "id": [3, 9]}]: dispatch 2024-09-10T09:54:00.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:59 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]: dispatch 2024-09-10T09:54:00.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:59 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 7]}]: dispatch 2024-09-10T09:54:00.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:59 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]: dispatch 2024-09-10T09:54:00.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:53:59 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 6]}]: dispatch 2024-09-10T09:54:00.148 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:59 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/3132265623' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-10T09:54:00.148 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:59 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.b", "id": [3, 9]}]: dispatch 2024-09-10T09:54:00.149 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:59 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]: dispatch 2024-09-10T09:54:00.149 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:59 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 7]}]: dispatch 2024-09-10T09:54:00.149 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:59 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]: dispatch 2024-09-10T09:54:00.149 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:53:59 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 6]}]: dispatch 2024-09-10T09:54:00.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:59 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/3132265623' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-10T09:54:00.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:59 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.b", "id": [3, 9]}]: dispatch 2024-09-10T09:54:00.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:59 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]: dispatch 2024-09-10T09:54:00.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:59 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 7]}]: dispatch 2024-09-10T09:54:00.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:59 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]: dispatch 2024-09-10T09:54:00.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:53:59 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 6]}]: dispatch 2024-09-10T09:54:00.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.530+0000 7f9173200640 1 -- 172.21.15.77:0/3829877475 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9174074e30 msgr2=0x7f917410dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:00.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.530+0000 7f9173200640 1 --2- 172.21.15.77:0/3829877475 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9174074e30 0x7f917410dd50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9160009900 tx=0x7f916002a480 comp rx=0 tx=0).stop 2024-09-10T09:54:00.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.530+0000 7f9173200640 1 -- 172.21.15.77:0/3829877475 shutdown_connections 2024-09-10T09:54:00.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.530+0000 7f9173200640 1 --2- 172.21.15.77:0/3829877475 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f917410e290 0x7f9174110680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:00.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.530+0000 7f9173200640 1 --2- 172.21.15.77:0/3829877475 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9174074e30 0x7f917410dd50 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:00.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.530+0000 7f9173200640 1 --2- 172.21.15.77:0/3829877475 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9174074510 0x7f91740748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:00.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.530+0000 7f9173200640 1 -- 172.21.15.77:0/3829877475 >> 172.21.15.77:0/3829877475 conn(0x7f91740fd710 msgr2=0x7f91740ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:00.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.530+0000 7f9173200640 1 -- 172.21.15.77:0/3829877475 shutdown_connections 2024-09-10T09:54:00.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.530+0000 7f9173200640 1 -- 172.21.15.77:0/3829877475 wait complete. 2024-09-10T09:54:00.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.531+0000 7f9173200640 1 Processor -- start 2024-09-10T09:54:00.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.531+0000 7f9173200640 1 -- start start 2024-09-10T09:54:00.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.532+0000 7f9173200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9174074510 0x7f917419a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:00.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.532+0000 7f9173200640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9174074e30 0x7f917419aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:00.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.532+0000 7f9173200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f917410e290 0x7f917419ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:00.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.532+0000 7f9173200640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9174113fc0 con 0x7f9174074510 2024-09-10T09:54:00.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.532+0000 7f9173200640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9174113e40 con 0x7f9174074e30 2024-09-10T09:54:00.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.532+0000 7f9173200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9174114140 con 0x7f917410e290 2024-09-10T09:54:00.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.532+0000 7f9171e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9174074510 0x7f917419a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:00.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.532+0000 7f9171e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9174074510 0x7f917419a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:55946/0 (socket says 172.21.15.77:55946) 2024-09-10T09:54:00.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.532+0000 7f9171400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9174074e30 0x7f917419aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:00.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.532+0000 7f9171e00640 1 -- 172.21.15.77:0/2524663512 learned_addr learned my addr 172.21.15.77:0/2524663512 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:54:00.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.532+0000 7f9172800640 1 --2- 172.21.15.77:0/2524663512 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f917410e290 0x7f917419ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:00.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.533+0000 7f9171400640 1 -- 172.21.15.77:0/2524663512 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f917410e290 msgr2=0x7f917419ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:00.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.533+0000 7f9171400640 1 --2- 172.21.15.77:0/2524663512 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f917410e290 0x7f917419ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:00.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.533+0000 7f9171400640 1 -- 172.21.15.77:0/2524663512 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9174074510 msgr2=0x7f917419a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:00.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.533+0000 7f9171400640 1 --2- 172.21.15.77:0/2524663512 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9174074510 0x7f917419a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:00.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.533+0000 7f9171400640 1 -- 172.21.15.77:0/2524663512 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f917419f470 con 0x7f9174074e30 2024-09-10T09:54:00.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.533+0000 7f9171400640 1 --2- 172.21.15.77:0/2524663512 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9174074e30 0x7f917419aa00 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9160005940 tx=0x7f91600040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:00.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.533+0000 7f916aa00640 1 -- 172.21.15.77:0/2524663512 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f916003f030 con 0x7f9174074e30 2024-09-10T09:54:00.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.533+0000 7f916aa00640 1 -- 172.21.15.77:0/2524663512 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91600044c0 con 0x7f9174074e30 2024-09-10T09:54:00.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.533+0000 7f9173200640 1 -- 172.21.15.77:0/2524663512 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f917419f700 con 0x7f9174074e30 2024-09-10T09:54:00.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.533+0000 7f916aa00640 1 -- 172.21.15.77:0/2524663512 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9160035070 con 0x7f9174074e30 2024-09-10T09:54:00.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.533+0000 7f9173200640 1 -- 172.21.15.77:0/2524663512 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91741a7040 con 0x7f9174074e30 2024-09-10T09:54:00.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.534+0000 7f9173200640 1 -- 172.21.15.77:0/2524663512 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9174062980 con 0x7f9174074e30 2024-09-10T09:54:00.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.536+0000 7f916aa00640 1 -- 172.21.15.77:0/2524663512 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f9160042030 con 0x7f9174074e30 2024-09-10T09:54:00.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.538+0000 7f916aa00640 1 --2- 172.21.15.77:0/2524663512 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f91580a2c10 0x7f91580a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:00.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.538+0000 7f916aa00640 1 -- 172.21.15.77:0/2524663512 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f91600eff40 con 0x7f9174074e30 2024-09-10T09:54:00.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.538+0000 7f9171e00640 1 --2- 172.21.15.77:0/2524663512 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f91580a2c10 0x7f91580a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:00.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.538+0000 7f916aa00640 1 --2- 172.21.15.77:0/2524663512 >> [v2:172.21.15.77:6810/1629416786,v1:172.21.15.77:6811/1629416786] conn(0x7f91580adde0 0x7f91580b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:00.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.538+0000 7f916aa00640 1 -- 172.21.15.77:0/2524663512 --> [v2:172.21.15.77:6810/1629416786,v1:172.21.15.77:6811/1629416786] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9160040e70 con 0x7f91580adde0 2024-09-10T09:54:00.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.538+0000 7f916aa00640 1 -- 172.21.15.77:0/2524663512 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7f91600f02f0 con 0x7f9174074e30 2024-09-10T09:54:00.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.538+0000 7f9172800640 1 --2- 172.21.15.77:0/2524663512 >> [v2:172.21.15.77:6810/1629416786,v1:172.21.15.77:6811/1629416786] conn(0x7f91580adde0 0x7f91580b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:00.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.539+0000 7f9171e00640 1 --2- 172.21.15.77:0/2524663512 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f91580a2c10 0x7f91580a50d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9164009c90 tx=0x7f9164005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:00.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.539+0000 7f9172800640 1 --2- 172.21.15.77:0/2524663512 >> [v2:172.21.15.77:6810/1629416786,v1:172.21.15.77:6811/1629416786] conn(0x7f91580adde0 0x7f91580b0280 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:00.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.540+0000 7f916aa00640 1 -- 172.21.15.77:0/2524663512 <== osd.3 v2:172.21.15.77:6810/1629416786 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9160040e70 con 0x7f91580adde0 2024-09-10T09:54:00.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.554+0000 7f9173200640 1 -- 172.21.15.77:0/2524663512 --> [v2:172.21.15.77:6810/1629416786,v1:172.21.15.77:6811/1629416786] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9174062e00 con 0x7f91580adde0 2024-09-10T09:54:00.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.554+0000 7f916aa00640 1 -- 172.21.15.77:0/2524663512 <== osd.3 v2:172.21.15.77:6810/1629416786 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9174062e00 con 0x7f91580adde0 2024-09-10T09:54:00.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.555+0000 7f9173200640 1 -- 172.21.15.77:0/2524663512 >> [v2:172.21.15.77:6810/1629416786,v1:172.21.15.77:6811/1629416786] conn(0x7f91580adde0 msgr2=0x7f91580b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:00.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.555+0000 7f9173200640 1 --2- 172.21.15.77:0/2524663512 >> [v2:172.21.15.77:6810/1629416786,v1:172.21.15.77:6811/1629416786] conn(0x7f91580adde0 0x7f91580b0280 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:00.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.555+0000 7f9173200640 1 -- 172.21.15.77:0/2524663512 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f91580a2c10 msgr2=0x7f91580a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:00.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.555+0000 7f9173200640 1 --2- 172.21.15.77:0/2524663512 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f91580a2c10 0x7f91580a50d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9164009c90 tx=0x7f9164005f90 comp rx=0 tx=0).stop 2024-09-10T09:54:00.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.555+0000 7f9173200640 1 -- 172.21.15.77:0/2524663512 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9174074e30 msgr2=0x7f917419aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:00.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.555+0000 7f9173200640 1 --2- 172.21.15.77:0/2524663512 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9174074e30 0x7f917419aa00 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9160005940 tx=0x7f91600040c0 comp rx=0 tx=0).stop 2024-09-10T09:54:00.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.555+0000 7f9172800640 1 -- 172.21.15.77:0/2524663512 reap_dead start 2024-09-10T09:54:00.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.555+0000 7f9173200640 1 -- 172.21.15.77:0/2524663512 shutdown_connections 2024-09-10T09:54:00.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.555+0000 7f9173200640 1 -- 172.21.15.77:0/2524663512 >> 172.21.15.77:0/2524663512 conn(0x7f91740fd710 msgr2=0x7f91740fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:00.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.555+0000 7f9173200640 1 -- 172.21.15.77:0/2524663512 shutdown_connections 2024-09-10T09:54:00.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:53:58.555+0000 7f9173200640 1 -- 172.21.15.77:0/2524663512 wait complete. 2024-09-10T09:54:01.033 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:54:00 smithi077 ceph-mon[30319]: pgmap v354: 129 pgs: 129 active+clean; 645 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-09-10T09:54:01.034 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:54:00 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.b", "id": [3, 9]}]': finished 2024-09-10T09:54:01.034 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:54:00 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2024-09-10T09:54:01.034 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:54:00 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 7]}]': finished 2024-09-10T09:54:01.034 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:54:00 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]': finished 2024-09-10T09:54:01.034 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:54:00 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 6]}]': finished 2024-09-10T09:54:01.034 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:54:00 smithi077 ceph-mon[30319]: osdmap e95: 12 total, 12 up, 12 in 2024-09-10T09:54:01.074 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574933 got 55834574935 for osd.1 2024-09-10T09:54:01.074 DEBUG:teuthology.parallel:result is None 2024-09-10T09:54:01.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:54:00 smithi134 ceph-mon[35674]: pgmap v354: 129 pgs: 129 active+clean; 645 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-09-10T09:54:01.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:54:00 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.b", "id": [3, 9]}]': finished 2024-09-10T09:54:01.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:54:00 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2024-09-10T09:54:01.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:54:00 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 7]}]': finished 2024-09-10T09:54:01.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:54:00 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]': finished 2024-09-10T09:54:01.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:54:00 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 6]}]': finished 2024-09-10T09:54:01.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:54:00 smithi134 ceph-mon[35674]: osdmap e95: 12 total, 12 up, 12 in 2024-09-10T09:54:01.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:54:00 smithi120 ceph-mon[34770]: pgmap v354: 129 pgs: 129 active+clean; 645 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-09-10T09:54:01.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:54:00 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.b", "id": [3, 9]}]': finished 2024-09-10T09:54:01.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:54:00 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2024-09-10T09:54:01.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:54:00 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 7]}]': finished 2024-09-10T09:54:01.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:54:00 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]': finished 2024-09-10T09:54:01.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:54:00 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 6]}]': finished 2024-09-10T09:54:01.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:54:00 smithi120 ceph-mon[34770]: osdmap e95: 12 total, 12 up, 12 in 2024-09-10T09:54:01.416 INFO:teuthology.orchestra.run.smithi077.stdout:107374182475 2024-09-10T09:54:01.417 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-10T09:54:01.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:01.626+0000 7f5258800640 1 -- 172.21.15.77:0/3202430387 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5250109120 msgr2=0x7f5250110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:01.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:01.626+0000 7f5258800640 1 --2- 172.21.15.77:0/3202430387 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5250109120 0x7f5250110ca0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f524c009ca0 tx=0x7f524c02a8c0 comp rx=0 tx=0).stop 2024-09-10T09:54:01.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:01.626+0000 7f5258800640 1 -- 172.21.15.77:0/3202430387 shutdown_connections 2024-09-10T09:54:01.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:01.626+0000 7f5258800640 1 --2- 172.21.15.77:0/3202430387 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5250109120 0x7f5250110ca0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:01.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:01.626+0000 7f5258800640 1 --2- 172.21.15.77:0/3202430387 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52501049a0 0x7f52501089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:01.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:01.626+0000 7f5258800640 1 --2- 172.21.15.77:0/3202430387 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5250103ff0 0x7f52501043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:01.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:01.626+0000 7f5258800640 1 -- 172.21.15.77:0/3202430387 >> 172.21.15.77:0/3202430387 conn(0x7f52500ffa90 msgr2=0x7f5250101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:01.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:01.626+0000 7f5258800640 1 -- 172.21.15.77:0/3202430387 shutdown_connections 2024-09-10T09:54:01.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:01.627+0000 7f5258800640 1 -- 172.21.15.77:0/3202430387 wait complete. 2024-09-10T09:54:01.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:01.628+0000 7f5258800640 1 Processor -- start 2024-09-10T09:54:01.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:01.628+0000 7f5258800640 1 -- start start 2024-09-10T09:54:01.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:01.628+0000 7f5258800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5250103ff0 0x7f525019a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:01.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:01.628+0000 7f5258800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f52501049a0 0x7f525019ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:01.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:01.628+0000 7f5258800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5250109120 0x7f525019eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:01.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:01.628+0000 7f5258800640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5250114150 con 0x7f52501049a0 2024-09-10T09:54:01.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:01.628+0000 7f5258800640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5250113fd0 con 0x7f5250109120 2024-09-10T09:54:01.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:01.628+0000 7f5258800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f52501142d0 con 0x7f5250103ff0 2024-09-10T09:54:01.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:01.628+0000 7f5255e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5250103ff0 0x7f525019a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:01.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:01.628+0000 7f5255e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5250103ff0 0x7f525019a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:52022/0 (socket says 172.21.15.77:52022) 2024-09-10T09:54:01.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:01.629+0000 7f5255e00640 1 -- 172.21.15.77:0/915546037 learned_addr learned my addr 172.21.15.77:0/915546037 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:54:01.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:01.629+0000 7f5255400640 1 --2- 172.21.15.77:0/915546037 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f52501049a0 0x7f525019ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:01.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:01.629+0000 7f5256800640 1 --2- 172.21.15.77:0/915546037 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5250109120 0x7f525019eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:01.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:01.629+0000 7f5255e00640 1 -- 172.21.15.77:0/915546037 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5250109120 msgr2=0x7f525019eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:01.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:01.629+0000 7f5255e00640 1 --2- 172.21.15.77:0/915546037 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5250109120 0x7f525019eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:01.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:01.629+0000 7f5255e00640 1 -- 172.21.15.77:0/915546037 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f52501049a0 msgr2=0x7f525019ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:01.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:01.629+0000 7f5255e00640 1 --2- 172.21.15.77:0/915546037 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f52501049a0 0x7f525019ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:01.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:01.629+0000 7f5255e00640 1 -- 172.21.15.77:0/915546037 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f525019f5a0 con 0x7f5250103ff0 2024-09-10T09:54:01.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:01.629+0000 7f5255e00640 1 --2- 172.21.15.77:0/915546037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5250103ff0 0x7f525019a5f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f524400e9f0 tx=0x7f524400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:01.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:01.630+0000 7f524aa00640 1 -- 172.21.15.77:0/915546037 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f524400cd80 con 0x7f5250103ff0 2024-09-10T09:54:01.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:01.630+0000 7f5258800640 1 -- 172.21.15.77:0/915546037 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f525019f890 con 0x7f5250103ff0 2024-09-10T09:54:01.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:01.630+0000 7f524aa00640 1 -- 172.21.15.77:0/915546037 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5244010910 con 0x7f5250103ff0 2024-09-10T09:54:01.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:01.630+0000 7f524aa00640 1 -- 172.21.15.77:0/915546037 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52440188f0 con 0x7f5250103ff0 2024-09-10T09:54:01.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:01.630+0000 7f5258800640 1 -- 172.21.15.77:0/915546037 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52501a7170 con 0x7f5250103ff0 2024-09-10T09:54:01.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:01.631+0000 7f5258800640 1 -- 172.21.15.77:0/915546037 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5250062940 con 0x7f5250103ff0 2024-09-10T09:54:01.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:01.633+0000 7f524aa00640 1 -- 172.21.15.77:0/915546037 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f5244010ac0 con 0x7f5250103ff0 2024-09-10T09:54:01.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:01.634+0000 7f524aa00640 1 --2- 172.21.15.77:0/915546037 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f52380a2c90 0x7f52380a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:01.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:01.634+0000 7f524aa00640 1 -- 172.21.15.77:0/915546037 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f52440d0430 con 0x7f5250103ff0 2024-09-10T09:54:01.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:01.634+0000 7f5255400640 1 --2- 172.21.15.77:0/915546037 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f52380a2c90 0x7f52380a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:01.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:01.635+0000 7f524aa00640 1 --2- 172.21.15.77:0/915546037 >> [v2:172.21.15.134:6808/4081444621,v1:172.21.15.134:6809/4081444621] conn(0x7f52380ae0e0 0x7f52380b0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:01.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:01.635+0000 7f524aa00640 1 -- 172.21.15.77:0/915546037 --> [v2:172.21.15.134:6808/4081444621,v1:172.21.15.134:6809/4081444621] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f524401bec0 con 0x7f52380ae0e0 2024-09-10T09:54:01.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:01.635+0000 7f524aa00640 1 -- 172.21.15.77:0/915546037 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_get_version_reply(handle=1 version=96) ==== 24+0+0 (secure 0 0 0) 0x7f52440d0870 con 0x7f5250103ff0 2024-09-10T09:54:01.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:01.635+0000 7f5256800640 1 --2- 172.21.15.77:0/915546037 >> [v2:172.21.15.134:6808/4081444621,v1:172.21.15.134:6809/4081444621] conn(0x7f52380ae0e0 0x7f52380b0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:01.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:01.635+0000 7f5255400640 1 --2- 172.21.15.77:0/915546037 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f52380a2c90 0x7f52380a5150 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f52500ffeb0 tx=0x7f5240005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:01.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:01.636+0000 7f5256800640 1 --2- 172.21.15.77:0/915546037 >> [v2:172.21.15.134:6808/4081444621,v1:172.21.15.134:6809/4081444621] conn(0x7f52380ae0e0 0x7f52380b0580 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:01.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:01.637+0000 7f524aa00640 1 -- 172.21.15.77:0/915546037 <== osd.5 v2:172.21.15.134:6808/4081444621 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f524401bec0 con 0x7f52380ae0e0 2024-09-10T09:54:01.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:01.651+0000 7f5258800640 1 -- 172.21.15.77:0/915546037 --> [v2:172.21.15.134:6808/4081444621,v1:172.21.15.134:6809/4081444621] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5250062d40 con 0x7f52380ae0e0 2024-09-10T09:54:01.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:01.651+0000 7f524aa00640 1 -- 172.21.15.77:0/915546037 <== osd.5 v2:172.21.15.134:6808/4081444621 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f5250062d40 con 0x7f52380ae0e0 2024-09-10T09:54:01.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:01.652+0000 7f5258800640 1 -- 172.21.15.77:0/915546037 >> [v2:172.21.15.134:6808/4081444621,v1:172.21.15.134:6809/4081444621] conn(0x7f52380ae0e0 msgr2=0x7f52380b0580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:01.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:01.652+0000 7f5258800640 1 --2- 172.21.15.77:0/915546037 >> [v2:172.21.15.134:6808/4081444621,v1:172.21.15.134:6809/4081444621] conn(0x7f52380ae0e0 0x7f52380b0580 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:01.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:01.652+0000 7f5258800640 1 -- 172.21.15.77:0/915546037 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f52380a2c90 msgr2=0x7f52380a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:01.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:01.652+0000 7f5258800640 1 --2- 172.21.15.77:0/915546037 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f52380a2c90 0x7f52380a5150 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f52500ffeb0 tx=0x7f5240005f90 comp rx=0 tx=0).stop 2024-09-10T09:54:01.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:01.652+0000 7f5258800640 1 -- 172.21.15.77:0/915546037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5250103ff0 msgr2=0x7f525019a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:01.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:01.652+0000 7f5258800640 1 --2- 172.21.15.77:0/915546037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5250103ff0 0x7f525019a5f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f524400e9f0 tx=0x7f524400eeb0 comp rx=0 tx=0).stop 2024-09-10T09:54:01.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:01.652+0000 7f5256800640 1 -- 172.21.15.77:0/915546037 reap_dead start 2024-09-10T09:54:01.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:01.653+0000 7f5258800640 1 -- 172.21.15.77:0/915546037 shutdown_connections 2024-09-10T09:54:01.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:01.653+0000 7f5258800640 1 -- 172.21.15.77:0/915546037 >> 172.21.15.77:0/915546037 conn(0x7f52500ffa90 msgr2=0x7f5250100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:01.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:01.653+0000 7f5258800640 1 -- 172.21.15.77:0/915546037 shutdown_connections 2024-09-10T09:54:01.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:01.653+0000 7f5258800640 1 -- 172.21.15.77:0/915546037 wait complete. 2024-09-10T09:54:01.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:54:01 smithi077 ceph-mon[30319]: osdmap e96: 12 total, 12 up, 12 in 2024-09-10T09:54:02.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:54:01 smithi134 ceph-mon[35674]: osdmap e96: 12 total, 12 up, 12 in 2024-09-10T09:54:02.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:54:01 smithi120 ceph-mon[34770]: osdmap e96: 12 total, 12 up, 12 in 2024-09-10T09:54:02.529 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:54:02.529 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:54:02.530 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:54:02.533 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:54:02.559 INFO:teuthology.orchestra.run.smithi077.stdout:158913790019 2024-09-10T09:54:02.560 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-10T09:54:03.063 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:54:02 smithi077 ceph-mon[30319]: pgmap v357: 129 pgs: 1 activating, 128 active+clean; 645 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 7 op/s 2024-09-10T09:54:03.064 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:54:02 smithi077 ceph-mon[30319]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-09-10T09:54:03.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:54:02 smithi134 ceph-mon[35674]: pgmap v357: 129 pgs: 1 activating, 128 active+clean; 645 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 7 op/s 2024-09-10T09:54:03.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:54:02 smithi134 ceph-mon[35674]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-09-10T09:54:03.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:54:02 smithi120 ceph-mon[34770]: pgmap v357: 129 pgs: 1 activating, 128 active+clean; 645 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 7 op/s 2024-09-10T09:54:03.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:54:02 smithi120 ceph-mon[34770]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-09-10T09:54:03.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:03.178+0000 7f7f59a00640 1 -- 172.21.15.77:0/1375532255 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f54101de0 msgr2=0x7f7f541021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:03.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:03.178+0000 7f7f59a00640 1 --2- 172.21.15.77:0/1375532255 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f54101de0 0x7f7f541021c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f7f44009900 tx=0x7f7f4402a410 comp rx=0 tx=0).stop 2024-09-10T09:54:03.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:03.178+0000 7f7f59a00640 1 -- 172.21.15.77:0/1375532255 shutdown_connections 2024-09-10T09:54:03.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:03.178+0000 7f7f59a00640 1 --2- 172.21.15.77:0/1375532255 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7f54106f10 0x7f7f5410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:03.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:03.178+0000 7f7f59a00640 1 --2- 172.21.15.77:0/1375532255 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f54102790 0x7f7f541067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:03.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:03.178+0000 7f7f59a00640 1 --2- 172.21.15.77:0/1375532255 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f54101de0 0x7f7f541021c0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:03.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:03.178+0000 7f7f59a00640 1 -- 172.21.15.77:0/1375532255 >> 172.21.15.77:0/1375532255 conn(0x7f7f540fd8c0 msgr2=0x7f7f540ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:03.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:03.178+0000 7f7f59a00640 1 -- 172.21.15.77:0/1375532255 shutdown_connections 2024-09-10T09:54:03.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:03.178+0000 7f7f59a00640 1 -- 172.21.15.77:0/1375532255 wait complete. 2024-09-10T09:54:03.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:03.179+0000 7f7f59a00640 1 Processor -- start 2024-09-10T09:54:03.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:03.179+0000 7f7f59a00640 1 -- start start 2024-09-10T09:54:03.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:03.179+0000 7f7f59a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f54101de0 0x7f7f5419a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:03.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:03.179+0000 7f7f59a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7f54102790 0x7f7f5419abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:03.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:03.179+0000 7f7f59a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f54106f10 0x7f7f5419ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:03.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:03.179+0000 7f7f59a00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7f54111fc0 con 0x7f7f54106f10 2024-09-10T09:54:03.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:03.179+0000 7f7f59a00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7f54111e40 con 0x7f7f54102790 2024-09-10T09:54:03.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:03.179+0000 7f7f59a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7f54112140 con 0x7f7f54101de0 2024-09-10T09:54:03.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:03.180+0000 7f7f52000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7f54102790 0x7f7f5419abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:03.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:03.180+0000 7f7f53400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f54106f10 0x7f7f5419ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:03.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:03.180+0000 7f7f52000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7f54102790 0x7f7f5419abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:39140/0 (socket says 172.21.15.77:39140) 2024-09-10T09:54:03.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:03.180+0000 7f7f52a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f54101de0 0x7f7f5419a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:03.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:03.180+0000 7f7f52000640 1 -- 172.21.15.77:0/371262096 learned_addr learned my addr 172.21.15.77:0/371262096 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:54:03.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:03.180+0000 7f7f52000640 1 -- 172.21.15.77:0/371262096 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f54101de0 msgr2=0x7f7f5419a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:03.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:03.180+0000 7f7f52000640 1 --2- 172.21.15.77:0/371262096 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f54101de0 0x7f7f5419a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:03.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:03.180+0000 7f7f52000640 1 -- 172.21.15.77:0/371262096 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f54106f10 msgr2=0x7f7f5419ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:03.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:03.180+0000 7f7f52000640 1 --2- 172.21.15.77:0/371262096 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f54106f10 0x7f7f5419ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:03.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:03.180+0000 7f7f52000640 1 -- 172.21.15.77:0/371262096 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f5419f650 con 0x7f7f54102790 2024-09-10T09:54:03.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:03.181+0000 7f7f52000640 1 --2- 172.21.15.77:0/371262096 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7f54102790 0x7f7f5419abe0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7f4000e9f0 tx=0x7f7f4000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:03.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:03.181+0000 7f7f4b400640 1 -- 172.21.15.77:0/371262096 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f4000cd80 con 0x7f7f54102790 2024-09-10T09:54:03.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:03.182+0000 7f7f59a00640 1 -- 172.21.15.77:0/371262096 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f5419f940 con 0x7f7f54102790 2024-09-10T09:54:03.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:03.182+0000 7f7f4b400640 1 -- 172.21.15.77:0/371262096 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f40005500 con 0x7f7f54102790 2024-09-10T09:54:03.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:03.182+0000 7f7f4b400640 1 -- 172.21.15.77:0/371262096 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f40010690 con 0x7f7f54102790 2024-09-10T09:54:03.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:03.182+0000 7f7f59a00640 1 -- 172.21.15.77:0/371262096 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f541a7220 con 0x7f7f54102790 2024-09-10T09:54:03.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:03.183+0000 7f7f59a00640 1 -- 172.21.15.77:0/371262096 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f541021c0 con 0x7f7f54102790 2024-09-10T09:54:03.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:03.185+0000 7f7f4b400640 1 -- 172.21.15.77:0/371262096 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f7f400046c0 con 0x7f7f54102790 2024-09-10T09:54:03.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:03.187+0000 7f7f4b400640 1 --2- 172.21.15.77:0/371262096 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f7f380a2c90 0x7f7f380a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:03.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:03.187+0000 7f7f4b400640 1 -- 172.21.15.77:0/371262096 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f7f400cf9b0 con 0x7f7f54102790 2024-09-10T09:54:03.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:03.187+0000 7f7f52a00640 1 --2- 172.21.15.77:0/371262096 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f7f380a2c90 0x7f7f380a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:03.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:03.188+0000 7f7f52a00640 1 --2- 172.21.15.77:0/371262096 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f7f380a2c90 0x7f7f380a5150 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7f44000c00 tx=0x7f7f44005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:03.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:03.188+0000 7f7f4b400640 1 -- 172.21.15.77:0/371262096 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7f4009c360 con 0x7f7f54102790 2024-09-10T09:54:03.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:03.375+0000 7f7f59a00640 1 -- 172.21.15.77:0/371262096 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f7f5419ba00 con 0x7f7f54102790 2024-09-10T09:54:03.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:03.375+0000 7f7f4b400640 1 -- 172.21.15.77:0/371262096 <== mon.2 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f7f4009bc60 con 0x7f7f54102790 2024-09-10T09:54:03.376 INFO:teuthology.orchestra.run.smithi077.stdout:214748364855 2024-09-10T09:54:03.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:03.378+0000 7f7f59a00640 1 -- 172.21.15.77:0/371262096 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f7f380a2c90 msgr2=0x7f7f380a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:03.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:03.378+0000 7f7f59a00640 1 --2- 172.21.15.77:0/371262096 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f7f380a2c90 0x7f7f380a5150 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7f44000c00 tx=0x7f7f44005b50 comp rx=0 tx=0).stop 2024-09-10T09:54:03.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:03.378+0000 7f7f59a00640 1 -- 172.21.15.77:0/371262096 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7f54102790 msgr2=0x7f7f5419abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:03.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:03.378+0000 7f7f59a00640 1 --2- 172.21.15.77:0/371262096 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7f54102790 0x7f7f5419abe0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7f4000e9f0 tx=0x7f7f4000edf0 comp rx=0 tx=0).stop 2024-09-10T09:54:03.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:03.378+0000 7f7f59a00640 1 -- 172.21.15.77:0/371262096 shutdown_connections 2024-09-10T09:54:03.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:03.378+0000 7f7f59a00640 1 --2- 172.21.15.77:0/371262096 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f7f380a2c90 0x7f7f380a5150 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:03.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:03.378+0000 7f7f59a00640 1 --2- 172.21.15.77:0/371262096 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f54106f10 0x7f7f5419ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:03.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:03.378+0000 7f7f59a00640 1 --2- 172.21.15.77:0/371262096 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7f54102790 0x7f7f5419abe0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:03.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:03.378+0000 7f7f59a00640 1 --2- 172.21.15.77:0/371262096 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f54101de0 0x7f7f5419a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:03.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:03.378+0000 7f7f59a00640 1 -- 172.21.15.77:0/371262096 >> 172.21.15.77:0/371262096 conn(0x7f7f540fd8c0 msgr2=0x7f7f540ffc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:03.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:03.378+0000 7f7f59a00640 1 -- 172.21.15.77:0/371262096 shutdown_connections 2024-09-10T09:54:03.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:03.378+0000 7f7f59a00640 1 -- 172.21.15.77:0/371262096 wait complete. 2024-09-10T09:54:03.853 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:54:03.857 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:54:03.858 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:54:03.858 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:54:03.867 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:54:03.883 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:54:03.982 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:54:03 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/371262096' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-10T09:54:04.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:54:03 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/371262096' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-10T09:54:04.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:54:03 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/371262096' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-10T09:54:04.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:04.235+0000 7fe403e00640 1 -- 172.21.15.77:0/101389720 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe404103e50 msgr2=0x7fe404104230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:04.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:04.235+0000 7fe403e00640 1 --2- 172.21.15.77:0/101389720 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe404103e50 0x7fe404104230 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe3f4009900 tx=0x7fe3f402a410 comp rx=0 tx=0).stop 2024-09-10T09:54:04.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:04.235+0000 7fe403e00640 1 -- 172.21.15.77:0/101389720 shutdown_connections 2024-09-10T09:54:04.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:04.235+0000 7fe403e00640 1 --2- 172.21.15.77:0/101389720 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe404109090 0x7fe404110c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:04.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:04.235+0000 7fe403e00640 1 --2- 172.21.15.77:0/101389720 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe404104800 0x7fe404108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:04.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:04.235+0000 7fe403e00640 1 --2- 172.21.15.77:0/101389720 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe404103e50 0x7fe404104230 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:04.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:04.235+0000 7fe403e00640 1 -- 172.21.15.77:0/101389720 >> 172.21.15.77:0/101389720 conn(0x7fe4040ff8b0 msgr2=0x7fe404101cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:04.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:04.235+0000 7fe403e00640 1 -- 172.21.15.77:0/101389720 shutdown_connections 2024-09-10T09:54:04.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:04.235+0000 7fe403e00640 1 -- 172.21.15.77:0/101389720 wait complete. 2024-09-10T09:54:04.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:04.236+0000 7fe403e00640 1 Processor -- start 2024-09-10T09:54:04.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:04.236+0000 7fe403e00640 1 -- start start 2024-09-10T09:54:04.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:04.236+0000 7fe403e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe404103e50 0x7fe40419a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:04.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:04.237+0000 7fe403e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe404104800 0x7fe40419aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:04.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:04.237+0000 7fe403e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe404109090 0x7fe40419ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:04.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:04.237+0000 7fe403e00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe4041140e0 con 0x7fe404109090 2024-09-10T09:54:04.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:04.237+0000 7fe403e00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe404113f60 con 0x7fe404104800 2024-09-10T09:54:04.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:04.237+0000 7fe403e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe404114260 con 0x7fe404103e50 2024-09-10T09:54:04.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:04.237+0000 7fe402a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe404103e50 0x7fe40419a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:04.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:04.237+0000 7fe402a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe404103e50 0x7fe40419a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:44798/0 (socket says 172.21.15.77:44798) 2024-09-10T09:54:04.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:04.237+0000 7fe402a00640 1 -- 172.21.15.77:0/3155490344 learned_addr learned my addr 172.21.15.77:0/3155490344 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:54:04.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:04.237+0000 7fe403400640 1 --2- 172.21.15.77:0/3155490344 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe404109090 0x7fe40419ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:04.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:04.237+0000 7fe402000640 1 --2- 172.21.15.77:0/3155490344 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe404104800 0x7fe40419aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:04.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:04.237+0000 7fe402a00640 1 -- 172.21.15.77:0/3155490344 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe404104800 msgr2=0x7fe40419aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:04.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:04.237+0000 7fe402a00640 1 --2- 172.21.15.77:0/3155490344 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe404104800 0x7fe40419aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:04.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:04.237+0000 7fe402a00640 1 -- 172.21.15.77:0/3155490344 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe404109090 msgr2=0x7fe40419ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:04.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:04.237+0000 7fe402a00640 1 --2- 172.21.15.77:0/3155490344 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe404109090 0x7fe40419ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:04.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:04.237+0000 7fe402a00640 1 -- 172.21.15.77:0/3155490344 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe40419f550 con 0x7fe404103e50 2024-09-10T09:54:04.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:04.238+0000 7fe402a00640 1 --2- 172.21.15.77:0/3155490344 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe404103e50 0x7fe40419a5a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe3f4004770 tx=0x7fe3f4003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:04.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:04.238+0000 7fe3fb400640 1 -- 172.21.15.77:0/3155490344 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3f403f070 con 0x7fe404103e50 2024-09-10T09:54:04.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:04.238+0000 7fe3fb400640 1 -- 172.21.15.77:0/3155490344 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3f4031ca0 con 0x7fe404103e50 2024-09-10T09:54:04.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:04.238+0000 7fe403e00640 1 -- 172.21.15.77:0/3155490344 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe40419f7e0 con 0x7fe404103e50 2024-09-10T09:54:04.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:04.238+0000 7fe3fb400640 1 -- 172.21.15.77:0/3155490344 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3f4035030 con 0x7fe404103e50 2024-09-10T09:54:04.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:04.238+0000 7fe403e00640 1 -- 172.21.15.77:0/3155490344 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4041a7120 con 0x7fe404103e50 2024-09-10T09:54:04.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:04.239+0000 7fe403e00640 1 -- 172.21.15.77:0/3155490344 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe404105330 con 0x7fe404103e50 2024-09-10T09:54:04.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:04.241+0000 7fe3fb400640 1 -- 172.21.15.77:0/3155490344 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fe3f4042070 con 0x7fe404103e50 2024-09-10T09:54:04.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:04.243+0000 7fe3fb400640 1 --2- 172.21.15.77:0/3155490344 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fe3e80a2c90 0x7fe3e80a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:04.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:04.243+0000 7fe3fb400640 1 -- 172.21.15.77:0/3155490344 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fe3f40f0210 con 0x7fe404103e50 2024-09-10T09:54:04.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:04.243+0000 7fe402000640 1 --2- 172.21.15.77:0/3155490344 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fe3e80a2c90 0x7fe3e80a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:04.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:04.244+0000 7fe402000640 1 --2- 172.21.15.77:0/3155490344 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fe3e80a2c90 0x7fe3e80a5150 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe4040ffd60 tx=0x7fe3f0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:04.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:04.245+0000 7fe3fb400640 1 -- 172.21.15.77:0/3155490344 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe3f40bc570 con 0x7fe404103e50 2024-09-10T09:54:04.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:04.423+0000 7fe403e00640 1 -- 172.21.15.77:0/3155490344 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7fe404104230 con 0x7fe404103e50 2024-09-10T09:54:04.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:04.424+0000 7fe3fb400640 1 -- 172.21.15.77:0/3155490344 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fe404104230 con 0x7fe404103e50 2024-09-10T09:54:04.572 INFO:teuthology.orchestra.run.smithi077.stdout:279172874285 2024-09-10T09:54:04.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:04.427+0000 7fe403e00640 1 -- 172.21.15.77:0/3155490344 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fe3e80a2c90 msgr2=0x7fe3e80a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:04.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:04.427+0000 7fe403e00640 1 --2- 172.21.15.77:0/3155490344 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fe3e80a2c90 0x7fe3e80a5150 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe4040ffd60 tx=0x7fe3f0005f90 comp rx=0 tx=0).stop 2024-09-10T09:54:04.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:04.427+0000 7fe403e00640 1 -- 172.21.15.77:0/3155490344 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe404103e50 msgr2=0x7fe40419a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:04.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:04.427+0000 7fe403e00640 1 --2- 172.21.15.77:0/3155490344 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe404103e50 0x7fe40419a5a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe3f4004770 tx=0x7fe3f4003c10 comp rx=0 tx=0).stop 2024-09-10T09:54:04.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:04.427+0000 7fe403e00640 1 -- 172.21.15.77:0/3155490344 shutdown_connections 2024-09-10T09:54:04.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:04.427+0000 7fe403e00640 1 --2- 172.21.15.77:0/3155490344 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fe3e80a2c90 0x7fe3e80a5150 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:04.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:04.427+0000 7fe403e00640 1 --2- 172.21.15.77:0/3155490344 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe404109090 0x7fe40419ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:04.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:04.427+0000 7fe403e00640 1 --2- 172.21.15.77:0/3155490344 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe404104800 0x7fe40419aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:04.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:04.427+0000 7fe403e00640 1 --2- 172.21.15.77:0/3155490344 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe404103e50 0x7fe40419a5a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:04.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:04.427+0000 7fe403e00640 1 -- 172.21.15.77:0/3155490344 >> 172.21.15.77:0/3155490344 conn(0x7fe4040ff8b0 msgr2=0x7fe404100bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:04.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:04.427+0000 7fe403e00640 1 -- 172.21.15.77:0/3155490344 shutdown_connections 2024-09-10T09:54:04.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:04.427+0000 7fe403e00640 1 -- 172.21.15.77:0/3155490344 wait complete. 2024-09-10T09:54:05.121 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:54:04 smithi077 ceph-mon[30319]: pgmap v358: 129 pgs: 1 activating, 128 active+clean; 645 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-09-10T09:54:05.121 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:54:04 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/3155490344' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-10T09:54:05.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:54:04 smithi134 ceph-mon[35674]: pgmap v358: 129 pgs: 1 activating, 128 active+clean; 645 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-09-10T09:54:05.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:54:04 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/3155490344' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-10T09:54:05.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:54:04 smithi120 ceph-mon[34770]: pgmap v358: 129 pgs: 1 activating, 128 active+clean; 645 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-09-10T09:54:05.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:54:04 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/3155490344' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-10T09:54:05.326 INFO:tasks.cephadm.ceph_manager.ceph:need seq 214748364853 got 214748364855 for osd.7 2024-09-10T09:54:05.327 DEBUG:teuthology.parallel:result is None 2024-09-10T09:54:06.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.118+0000 7ffb37200640 1 -- 172.21.15.77:0/3319033206 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb38074620 msgr2=0x7ffb38074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:06.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.118+0000 7ffb37200640 1 --2- 172.21.15.77:0/3319033206 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb38074620 0x7ffb38074a00 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ffb28009900 tx=0x7ffb2802a480 comp rx=0 tx=0).stop 2024-09-10T09:54:06.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.118+0000 7ffb37200640 1 -- 172.21.15.77:0/3319033206 shutdown_connections 2024-09-10T09:54:06.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.118+0000 7ffb37200640 1 --2- 172.21.15.77:0/3319033206 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffb3810e210 0x7ffb38110660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:06.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.118+0000 7ffb37200640 1 --2- 172.21.15.77:0/3319033206 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffb38074f40 0x7ffb3810dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:06.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.118+0000 7ffb37200640 1 --2- 172.21.15.77:0/3319033206 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb38074620 0x7ffb38074a00 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:06.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.118+0000 7ffb37200640 1 -- 172.21.15.77:0/3319033206 >> 172.21.15.77:0/3319033206 conn(0x7ffb380fd710 msgr2=0x7ffb380ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:06.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.118+0000 7ffb37200640 1 -- 172.21.15.77:0/3319033206 shutdown_connections 2024-09-10T09:54:06.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.118+0000 7ffb37200640 1 -- 172.21.15.77:0/3319033206 wait complete. 2024-09-10T09:54:06.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.119+0000 7ffb37200640 1 Processor -- start 2024-09-10T09:54:06.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.119+0000 7ffb37200640 1 -- start start 2024-09-10T09:54:06.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.119+0000 7ffb37200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffb38074620 0x7ffb3819a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:06.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.119+0000 7ffb37200640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffb38074f40 0x7ffb3819aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:06.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.119+0000 7ffb37200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb3810e210 0x7ffb3819ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:06.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.119+0000 7ffb37200640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ffb38113f80 con 0x7ffb38074620 2024-09-10T09:54:06.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.119+0000 7ffb37200640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ffb38113e00 con 0x7ffb38074f40 2024-09-10T09:54:06.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.119+0000 7ffb37200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ffb38114100 con 0x7ffb3810e210 2024-09-10T09:54:06.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.119+0000 7ffb35e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffb38074620 0x7ffb3819a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:06.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.120+0000 7ffb2fe00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffb38074f40 0x7ffb3819aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:06.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.120+0000 7ffb35e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffb38074620 0x7ffb3819a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:44318/0 (socket says 172.21.15.77:44318) 2024-09-10T09:54:06.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.120+0000 7ffb35e00640 1 -- 172.21.15.77:0/3817661690 learned_addr learned my addr 172.21.15.77:0/3817661690 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:54:06.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.120+0000 7ffb36800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb3810e210 0x7ffb3819ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:06.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.120+0000 7ffb2fe00640 1 -- 172.21.15.77:0/3817661690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb3810e210 msgr2=0x7ffb3819ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:06.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.120+0000 7ffb2fe00640 1 --2- 172.21.15.77:0/3817661690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb3810e210 0x7ffb3819ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:06.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.120+0000 7ffb2fe00640 1 -- 172.21.15.77:0/3817661690 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffb38074620 msgr2=0x7ffb3819a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:06.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.120+0000 7ffb2fe00640 1 --2- 172.21.15.77:0/3817661690 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffb38074620 0x7ffb3819a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:06.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.120+0000 7ffb2fe00640 1 -- 172.21.15.77:0/3817661690 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb3819f4e0 con 0x7ffb38074f40 2024-09-10T09:54:06.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.120+0000 7ffb35e00640 1 --2- 172.21.15.77:0/3817661690 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffb38074620 0x7ffb3819a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:54:06.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.120+0000 7ffb2fe00640 1 --2- 172.21.15.77:0/3817661690 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffb38074f40 0x7ffb3819aa70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ffb2000e9f0 tx=0x7ffb2000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:06.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.121+0000 7ffb2ea00640 1 -- 172.21.15.77:0/3817661690 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb2000cd80 con 0x7ffb38074f40 2024-09-10T09:54:06.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.121+0000 7ffb2ea00640 1 -- 172.21.15.77:0/3817661690 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb200035f0 con 0x7ffb38074f40 2024-09-10T09:54:06.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.121+0000 7ffb2ea00640 1 -- 172.21.15.77:0/3817661690 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb20010690 con 0x7ffb38074f40 2024-09-10T09:54:06.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.121+0000 7ffb37200640 1 -- 172.21.15.77:0/3817661690 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb3819f7d0 con 0x7ffb38074f40 2024-09-10T09:54:06.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.121+0000 7ffb37200640 1 -- 172.21.15.77:0/3817661690 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb381a70b0 con 0x7ffb38074f40 2024-09-10T09:54:06.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.122+0000 7ffb37200640 1 -- 172.21.15.77:0/3817661690 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb38074a00 con 0x7ffb38074f40 2024-09-10T09:54:06.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.124+0000 7ffb2ea00640 1 -- 172.21.15.77:0/3817661690 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ffb20003b30 con 0x7ffb38074f40 2024-09-10T09:54:06.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.126+0000 7ffb2ea00640 1 --2- 172.21.15.77:0/3817661690 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ffb1c0a2c90 0x7ffb1c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:06.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.126+0000 7ffb2ea00640 1 -- 172.21.15.77:0/3817661690 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7ffb200cf830 con 0x7ffb38074f40 2024-09-10T09:54:06.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.126+0000 7ffb35e00640 1 --2- 172.21.15.77:0/3817661690 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ffb1c0a2c90 0x7ffb1c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:06.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.127+0000 7ffb35e00640 1 --2- 172.21.15.77:0/3817661690 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ffb1c0a2c90 0x7ffb1c0a5150 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ffb280096c0 tx=0x7ffb28005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:06.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.127+0000 7ffb2ea00640 1 -- 172.21.15.77:0/3817661690 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ffb2009c1e0 con 0x7ffb38074f40 2024-09-10T09:54:06.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.321+0000 7ffb37200640 1 -- 172.21.15.77:0/3817661690 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7ffb3819b890 con 0x7ffb38074f40 2024-09-10T09:54:06.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.322+0000 7ffb2ea00640 1 -- 172.21.15.77:0/3817661690 <== mon.2 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7ffb2009bae0 con 0x7ffb38074f40 2024-09-10T09:54:06.324 INFO:teuthology.orchestra.run.smithi077.stdout:137438953543 2024-09-10T09:54:06.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.325+0000 7ffb37200640 1 -- 172.21.15.77:0/3817661690 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ffb1c0a2c90 msgr2=0x7ffb1c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:06.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.325+0000 7ffb37200640 1 --2- 172.21.15.77:0/3817661690 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ffb1c0a2c90 0x7ffb1c0a5150 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ffb280096c0 tx=0x7ffb28005a20 comp rx=0 tx=0).stop 2024-09-10T09:54:06.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.325+0000 7ffb37200640 1 -- 172.21.15.77:0/3817661690 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffb38074f40 msgr2=0x7ffb3819aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:06.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.325+0000 7ffb37200640 1 --2- 172.21.15.77:0/3817661690 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffb38074f40 0x7ffb3819aa70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ffb2000e9f0 tx=0x7ffb2000eeb0 comp rx=0 tx=0).stop 2024-09-10T09:54:06.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.325+0000 7ffb37200640 1 -- 172.21.15.77:0/3817661690 shutdown_connections 2024-09-10T09:54:06.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.325+0000 7ffb37200640 1 --2- 172.21.15.77:0/3817661690 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ffb1c0a2c90 0x7ffb1c0a5150 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:06.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.325+0000 7ffb37200640 1 --2- 172.21.15.77:0/3817661690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb3810e210 0x7ffb3819ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:06.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.325+0000 7ffb37200640 1 --2- 172.21.15.77:0/3817661690 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffb38074f40 0x7ffb3819aa70 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:06.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.325+0000 7ffb37200640 1 --2- 172.21.15.77:0/3817661690 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffb38074620 0x7ffb3819a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:06.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.325+0000 7ffb37200640 1 -- 172.21.15.77:0/3817661690 >> 172.21.15.77:0/3817661690 conn(0x7ffb380fd710 msgr2=0x7ffb380fef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:06.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.325+0000 7ffb37200640 1 -- 172.21.15.77:0/3817661690 shutdown_connections 2024-09-10T09:54:06.328 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.325+0000 7ffb37200640 1 -- 172.21.15.77:0/3817661690 wait complete. 2024-09-10T09:54:06.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.019+0000 7f7d63200640 1 -- 172.21.15.77:0/2570901984 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7d64102b90 msgr2=0x7f7d64102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:06.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.019+0000 7f7d63200640 1 --2- 172.21.15.77:0/2570901984 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7d64102b90 0x7f7d64102f70 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7d54009900 tx=0x7f7d5402a480 comp rx=0 tx=0).stop 2024-09-10T09:54:06.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.019+0000 7f7d63200640 1 -- 172.21.15.77:0/2570901984 shutdown_connections 2024-09-10T09:54:06.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.019+0000 7f7d63200640 1 --2- 172.21.15.77:0/2570901984 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d6410bfc0 0x7f7d6410e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:06.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.019+0000 7f7d63200640 1 --2- 172.21.15.77:0/2570901984 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d641034b0 0x7f7d6410b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:06.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.019+0000 7f7d63200640 1 --2- 172.21.15.77:0/2570901984 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7d64102b90 0x7f7d64102f70 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:06.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.019+0000 7f7d63200640 1 -- 172.21.15.77:0/2570901984 >> 172.21.15.77:0/2570901984 conn(0x7f7d640fc610 msgr2=0x7f7d640fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:06.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.019+0000 7f7d63200640 1 -- 172.21.15.77:0/2570901984 shutdown_connections 2024-09-10T09:54:06.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.019+0000 7f7d63200640 1 -- 172.21.15.77:0/2570901984 wait complete. 2024-09-10T09:54:06.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.020+0000 7f7d63200640 1 Processor -- start 2024-09-10T09:54:06.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.021+0000 7f7d63200640 1 -- start start 2024-09-10T09:54:06.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.021+0000 7f7d63200640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d64102b90 0x7f7d6419a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:06.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.021+0000 7f7d63200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7d641034b0 0x7f7d6419aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:06.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.021+0000 7f7d63200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d6410bfc0 0x7f7d6419ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:06.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.021+0000 7f7d63200640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7d64111e10 con 0x7f7d641034b0 2024-09-10T09:54:06.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.021+0000 7f7d63200640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7d64111c90 con 0x7f7d64102b90 2024-09-10T09:54:06.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.021+0000 7f7d63200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7d64111f90 con 0x7f7d6410bfc0 2024-09-10T09:54:06.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.021+0000 7f7d61400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7d641034b0 0x7f7d6419aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:06.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.021+0000 7f7d62800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d6410bfc0 0x7f7d6419ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:06.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.021+0000 7f7d62800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d6410bfc0 0x7f7d6419ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:44818/0 (socket says 172.21.15.77:44818) 2024-09-10T09:54:06.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.021+0000 7f7d62800640 1 -- 172.21.15.77:0/3571121227 learned_addr learned my addr 172.21.15.77:0/3571121227 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:54:06.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.021+0000 7f7d61e00640 1 --2- 172.21.15.77:0/3571121227 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d64102b90 0x7f7d6419a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:06.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.022+0000 7f7d62800640 1 -- 172.21.15.77:0/3571121227 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d64102b90 msgr2=0x7f7d6419a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:06.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.022+0000 7f7d62800640 1 --2- 172.21.15.77:0/3571121227 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d64102b90 0x7f7d6419a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:06.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.022+0000 7f7d62800640 1 -- 172.21.15.77:0/3571121227 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7d641034b0 msgr2=0x7f7d6419aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:06.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.022+0000 7f7d62800640 1 --2- 172.21.15.77:0/3571121227 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7d641034b0 0x7f7d6419aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:06.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.022+0000 7f7d62800640 1 -- 172.21.15.77:0/3571121227 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d6419f4c0 con 0x7f7d6410bfc0 2024-09-10T09:54:06.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.022+0000 7f7d62800640 1 --2- 172.21.15.77:0/3571121227 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d6410bfc0 0x7f7d6419ede0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c00eb10 tx=0x7f7d5c00c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:06.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.022+0000 7f7d5aa00640 1 -- 172.21.15.77:0/3571121227 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d5c004b20 con 0x7f7d6410bfc0 2024-09-10T09:54:06.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.022+0000 7f7d63200640 1 -- 172.21.15.77:0/3571121227 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d6419f7b0 con 0x7f7d6410bfc0 2024-09-10T09:54:06.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.022+0000 7f7d5aa00640 1 -- 172.21.15.77:0/3571121227 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d5c004cc0 con 0x7f7d6410bfc0 2024-09-10T09:54:06.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.022+0000 7f7d5aa00640 1 -- 172.21.15.77:0/3571121227 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d5c010760 con 0x7f7d6410bfc0 2024-09-10T09:54:06.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.023+0000 7f7d63200640 1 -- 172.21.15.77:0/3571121227 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d641a7090 con 0x7f7d6410bfc0 2024-09-10T09:54:06.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.024+0000 7f7d63200640 1 -- 172.21.15.77:0/3571121227 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d64102f70 con 0x7f7d6410bfc0 2024-09-10T09:54:06.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.026+0000 7f7d5aa00640 1 -- 172.21.15.77:0/3571121227 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f7d5c010900 con 0x7f7d6410bfc0 2024-09-10T09:54:06.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.027+0000 7f7d5aa00640 1 --2- 172.21.15.77:0/3571121227 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f7d480a2c10 0x7f7d480a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:06.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.027+0000 7f7d5aa00640 1 -- 172.21.15.77:0/3571121227 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f7d5c0d0080 con 0x7f7d6410bfc0 2024-09-10T09:54:06.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.027+0000 7f7d61e00640 1 --2- 172.21.15.77:0/3571121227 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f7d480a2c10 0x7f7d480a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:06.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.028+0000 7f7d61e00640 1 --2- 172.21.15.77:0/3571121227 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f7d480a2c10 0x7f7d480a50d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7d54000c00 tx=0x7f7d54005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:06.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.029+0000 7f7d5aa00640 1 -- 172.21.15.77:0/3571121227 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7d5c09ca30 con 0x7f7d6410bfc0 2024-09-10T09:54:06.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.217+0000 7f7d63200640 1 -- 172.21.15.77:0/3571121227 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f7d6419b870 con 0x7f7d6410bfc0 2024-09-10T09:54:06.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.218+0000 7f7d5aa00640 1 -- 172.21.15.77:0/3571121227 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f7d6419b870 con 0x7f7d6410bfc0 2024-09-10T09:54:06.381 INFO:teuthology.orchestra.run.smithi077.stdout:244813135923 2024-09-10T09:54:06.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.220+0000 7f7d63200640 1 -- 172.21.15.77:0/3571121227 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f7d480a2c10 msgr2=0x7f7d480a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:06.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.220+0000 7f7d63200640 1 --2- 172.21.15.77:0/3571121227 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f7d480a2c10 0x7f7d480a50d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7d54000c00 tx=0x7f7d54005b50 comp rx=0 tx=0).stop 2024-09-10T09:54:06.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.220+0000 7f7d63200640 1 -- 172.21.15.77:0/3571121227 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d6410bfc0 msgr2=0x7f7d6419ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:06.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.220+0000 7f7d63200640 1 --2- 172.21.15.77:0/3571121227 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d6410bfc0 0x7f7d6419ede0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c00eb10 tx=0x7f7d5c00c2d0 comp rx=0 tx=0).stop 2024-09-10T09:54:06.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.220+0000 7f7d63200640 1 -- 172.21.15.77:0/3571121227 shutdown_connections 2024-09-10T09:54:06.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.220+0000 7f7d63200640 1 --2- 172.21.15.77:0/3571121227 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f7d480a2c10 0x7f7d480a50d0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:06.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.220+0000 7f7d63200640 1 --2- 172.21.15.77:0/3571121227 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d6410bfc0 0x7f7d6419ede0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:06.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.220+0000 7f7d63200640 1 --2- 172.21.15.77:0/3571121227 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7d641034b0 0x7f7d6419aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:06.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.220+0000 7f7d63200640 1 --2- 172.21.15.77:0/3571121227 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d64102b90 0x7f7d6419a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:06.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.220+0000 7f7d63200640 1 -- 172.21.15.77:0/3571121227 >> 172.21.15.77:0/3571121227 conn(0x7f7d640fc610 msgr2=0x7f7d640fd210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:06.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.220+0000 7f7d63200640 1 -- 172.21.15.77:0/3571121227 shutdown_connections 2024-09-10T09:54:06.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.220+0000 7f7d63200640 1 -- 172.21.15.77:0/3571121227 wait complete. 2024-09-10T09:54:07.090 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:54:06 smithi077 ceph-mon[30319]: pgmap v359: 129 pgs: 1 activating, 128 active+clean; 645 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 10 op/s 2024-09-10T09:54:07.090 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:54:06 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/3571121227' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-10T09:54:07.090 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:54:06 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/3817661690' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-10T09:54:07.090 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:54:06 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/326713419' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-10T09:54:07.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:54:06 smithi134 ceph-mon[35674]: pgmap v359: 129 pgs: 1 activating, 128 active+clean; 645 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 10 op/s 2024-09-10T09:54:07.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:54:06 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/3571121227' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-10T09:54:07.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:54:06 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/3817661690' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-10T09:54:07.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:54:06 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/326713419' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-10T09:54:07.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:54:06 smithi120 ceph-mon[34770]: pgmap v359: 129 pgs: 1 activating, 128 active+clean; 645 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 10 op/s 2024-09-10T09:54:07.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:54:06 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/3571121227' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-10T09:54:07.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:54:06 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/3817661690' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-10T09:54:07.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:54:06 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/326713419' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-10T09:54:07.519 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.404+0000 7f81a7000640 1 -- 172.21.15.77:0/931530731 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f81a0109120 msgr2=0x7f81a0110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:07.519 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.404+0000 7f81a7000640 1 --2- 172.21.15.77:0/931530731 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f81a0109120 0x7f81a0110ca0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8198009900 tx=0x7f819802a3f0 comp rx=0 tx=0).stop 2024-09-10T09:54:07.519 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.404+0000 7f81a7000640 1 -- 172.21.15.77:0/931530731 shutdown_connections 2024-09-10T09:54:07.519 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.404+0000 7f81a7000640 1 --2- 172.21.15.77:0/931530731 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f81a0109120 0x7f81a0110ca0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:07.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.404+0000 7f81a7000640 1 --2- 172.21.15.77:0/931530731 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f81a01049a0 0x7f81a01089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:07.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.404+0000 7f81a7000640 1 --2- 172.21.15.77:0/931530731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f81a0103ff0 0x7f81a01043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:07.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.404+0000 7f81a7000640 1 -- 172.21.15.77:0/931530731 >> 172.21.15.77:0/931530731 conn(0x7f81a00ffa90 msgr2=0x7f81a0101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:07.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.404+0000 7f81a7000640 1 -- 172.21.15.77:0/931530731 shutdown_connections 2024-09-10T09:54:07.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.405+0000 7f81a7000640 1 -- 172.21.15.77:0/931530731 wait complete. 2024-09-10T09:54:07.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.405+0000 7f81a7000640 1 Processor -- start 2024-09-10T09:54:07.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.405+0000 7f81a7000640 1 -- start start 2024-09-10T09:54:07.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.406+0000 7f81a7000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f81a0103ff0 0x7f81a019a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:07.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.406+0000 7f81a7000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f81a01049a0 0x7f81a019ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:07.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.406+0000 7f81a7000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f81a0109120 0x7f81a019ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:07.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.406+0000 7f81a7000640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f81a01141f0 con 0x7f81a01049a0 2024-09-10T09:54:07.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.406+0000 7f81a7000640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f81a0114070 con 0x7f81a0103ff0 2024-09-10T09:54:07.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.406+0000 7f81a7000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f81a0114370 con 0x7f81a0109120 2024-09-10T09:54:07.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.406+0000 7f819f400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f81a01049a0 0x7f81a019ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:07.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.406+0000 7f819f400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f81a01049a0 0x7f81a019ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:44348/0 (socket says 172.21.15.77:44348) 2024-09-10T09:54:07.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.406+0000 7f81a5000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f81a0109120 0x7f81a019ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:07.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.406+0000 7f819f400640 1 -- 172.21.15.77:0/759786634 learned_addr learned my addr 172.21.15.77:0/759786634 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:54:07.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.406+0000 7f819fe00640 1 --2- 172.21.15.77:0/759786634 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f81a0103ff0 0x7f81a019a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:07.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.407+0000 7f819f400640 1 -- 172.21.15.77:0/759786634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f81a0109120 msgr2=0x7f81a019ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:07.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.407+0000 7f819f400640 1 --2- 172.21.15.77:0/759786634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f81a0109120 0x7f81a019ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:07.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.407+0000 7f819f400640 1 -- 172.21.15.77:0/759786634 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f81a0103ff0 msgr2=0x7f81a019a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:07.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.407+0000 7f819f400640 1 --2- 172.21.15.77:0/759786634 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f81a0103ff0 0x7f81a019a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:07.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.407+0000 7f819f400640 1 -- 172.21.15.77:0/759786634 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81a019f570 con 0x7f81a01049a0 2024-09-10T09:54:07.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.407+0000 7f819f400640 1 --2- 172.21.15.77:0/759786634 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f81a01049a0 0x7f81a019ab00 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f818c00b4a0 tx=0x7f818c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:07.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.407+0000 7f819cc00640 1 -- 172.21.15.77:0/759786634 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f818c014070 con 0x7f81a01049a0 2024-09-10T09:54:07.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.407+0000 7f819cc00640 1 -- 172.21.15.77:0/759786634 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f818c010af0 con 0x7f81a01049a0 2024-09-10T09:54:07.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.407+0000 7f81a7000640 1 -- 172.21.15.77:0/759786634 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81a019f860 con 0x7f81a01049a0 2024-09-10T09:54:07.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.407+0000 7f819cc00640 1 -- 172.21.15.77:0/759786634 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f818c017d10 con 0x7f81a01049a0 2024-09-10T09:54:07.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.407+0000 7f81a7000640 1 -- 172.21.15.77:0/759786634 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81a01a7140 con 0x7f81a01049a0 2024-09-10T09:54:07.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.408+0000 7f81a7000640 1 -- 172.21.15.77:0/759786634 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81a01043d0 con 0x7f81a01049a0 2024-09-10T09:54:07.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.411+0000 7f819cc00640 1 -- 172.21.15.77:0/759786634 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f818c017070 con 0x7f81a01049a0 2024-09-10T09:54:07.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.412+0000 7f819cc00640 1 --2- 172.21.15.77:0/759786634 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f81840a2c90 0x7f81840a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:07.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.412+0000 7f819cc00640 1 -- 172.21.15.77:0/759786634 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f818c0ce750 con 0x7f81a01049a0 2024-09-10T09:54:07.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.412+0000 7f819fe00640 1 --2- 172.21.15.77:0/759786634 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f81840a2c90 0x7f81840a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:07.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.413+0000 7f819fe00640 1 --2- 172.21.15.77:0/759786634 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f81840a2c90 0x7f81840a5150 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8190007ec0 tx=0x7f81900075b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:07.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.417+0000 7f819cc00640 1 -- 172.21.15.77:0/759786634 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f818c09abc0 con 0x7f81a01049a0 2024-09-10T09:54:07.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.608+0000 7f81a7000640 1 -- 172.21.15.77:0/759786634 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f81a01a74d0 con 0x7f81a01049a0 2024-09-10T09:54:07.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.609+0000 7f819cc00640 1 -- 172.21.15.77:0/759786634 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f818c09eec0 con 0x7f81a01049a0 2024-09-10T09:54:07.611 INFO:teuthology.orchestra.run.smithi077.stdout:193273528382 2024-09-10T09:54:07.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.611+0000 7f81a7000640 1 -- 172.21.15.77:0/759786634 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f81840a2c90 msgr2=0x7f81840a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:07.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.611+0000 7f81a7000640 1 --2- 172.21.15.77:0/759786634 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f81840a2c90 0x7f81840a5150 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8190007ec0 tx=0x7f81900075b0 comp rx=0 tx=0).stop 2024-09-10T09:54:07.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.611+0000 7f81a7000640 1 -- 172.21.15.77:0/759786634 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f81a01049a0 msgr2=0x7f81a019ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:07.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.611+0000 7f81a7000640 1 --2- 172.21.15.77:0/759786634 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f81a01049a0 0x7f81a019ab00 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f818c00b4a0 tx=0x7f818c00b960 comp rx=0 tx=0).stop 2024-09-10T09:54:07.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.611+0000 7f81a7000640 1 -- 172.21.15.77:0/759786634 shutdown_connections 2024-09-10T09:54:07.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.611+0000 7f81a7000640 1 --2- 172.21.15.77:0/759786634 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f81840a2c90 0x7f81840a5150 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:07.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.611+0000 7f81a7000640 1 --2- 172.21.15.77:0/759786634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f81a0109120 0x7f81a019ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:07.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.611+0000 7f81a7000640 1 --2- 172.21.15.77:0/759786634 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f81a01049a0 0x7f81a019ab00 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:07.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.611+0000 7f81a7000640 1 --2- 172.21.15.77:0/759786634 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f81a0103ff0 0x7f81a019a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:07.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.611+0000 7f81a7000640 1 -- 172.21.15.77:0/759786634 >> 172.21.15.77:0/759786634 conn(0x7f81a00ffa90 msgr2=0x7f81a0100560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:07.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.611+0000 7f81a7000640 1 -- 172.21.15.77:0/759786634 shutdown_connections 2024-09-10T09:54:07.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.612+0000 7f81a7000640 1 -- 172.21.15.77:0/759786634 wait complete. 2024-09-10T09:54:07.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.534+0000 7f4f9f200640 1 -- 172.21.15.77:0/555972484 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4fa010bfc0 msgr2=0x7f4fa010e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:07.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.534+0000 7f4f9f200640 1 --2- 172.21.15.77:0/555972484 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4fa010bfc0 0x7f4fa010e460 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4f98009be0 tx=0x7f4f9802a750 comp rx=0 tx=0).stop 2024-09-10T09:54:07.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.534+0000 7f4f9f200640 1 -- 172.21.15.77:0/555972484 shutdown_connections 2024-09-10T09:54:07.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.534+0000 7f4f9f200640 1 --2- 172.21.15.77:0/555972484 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4fa010bfc0 0x7f4fa010e460 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:07.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.534+0000 7f4f9f200640 1 --2- 172.21.15.77:0/555972484 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4fa01034b0 0x7f4fa010b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:07.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.534+0000 7f4f9f200640 1 --2- 172.21.15.77:0/555972484 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fa0102b90 0x7f4fa0102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:07.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.534+0000 7f4f9f200640 1 -- 172.21.15.77:0/555972484 >> 172.21.15.77:0/555972484 conn(0x7f4fa00fc610 msgr2=0x7f4fa00fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:07.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.534+0000 7f4f9f200640 1 -- 172.21.15.77:0/555972484 shutdown_connections 2024-09-10T09:54:07.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.535+0000 7f4f9f200640 1 -- 172.21.15.77:0/555972484 wait complete. 2024-09-10T09:54:07.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.536+0000 7f4f9f200640 1 Processor -- start 2024-09-10T09:54:07.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.536+0000 7f4f9f200640 1 -- start start 2024-09-10T09:54:07.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.537+0000 7f4f9f200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fa0102b90 0x7f4fa0198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:07.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.537+0000 7f4f9f200640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4fa01034b0 0x7f4fa0198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:07.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.537+0000 7f4f9f200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4fa010bfc0 0x7f4fa019cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:07.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.537+0000 7f4f9f200640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa0111d40 con 0x7f4fa0102b90 2024-09-10T09:54:07.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.537+0000 7f4f9f200640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa0111bc0 con 0x7f4fa01034b0 2024-09-10T09:54:07.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.537+0000 7f4f9f200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa0111ec0 con 0x7f4fa010bfc0 2024-09-10T09:54:07.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.537+0000 7f4f9de00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fa0102b90 0x7f4fa0198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:07.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.537+0000 7f4f9de00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fa0102b90 0x7f4fa0198240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:44372/0 (socket says 172.21.15.77:44372) 2024-09-10T09:54:07.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.537+0000 7f4f9de00640 1 -- 172.21.15.77:0/3962309663 learned_addr learned my addr 172.21.15.77:0/3962309663 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:54:07.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.537+0000 7f4f9d400640 1 --2- 172.21.15.77:0/3962309663 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4fa01034b0 0x7f4fa0198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:07.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.537+0000 7f4f9e800640 1 --2- 172.21.15.77:0/3962309663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4fa010bfc0 0x7f4fa019cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:07.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.537+0000 7f4f9d400640 1 -- 172.21.15.77:0/3962309663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4fa010bfc0 msgr2=0x7f4fa019cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:07.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.537+0000 7f4f9d400640 1 --2- 172.21.15.77:0/3962309663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4fa010bfc0 0x7f4fa019cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:07.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.537+0000 7f4f9d400640 1 -- 172.21.15.77:0/3962309663 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fa0102b90 msgr2=0x7f4fa0198240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:07.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.537+0000 7f4f9d400640 1 --2- 172.21.15.77:0/3962309663 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fa0102b90 0x7f4fa0198240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:07.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.537+0000 7f4f9d400640 1 -- 172.21.15.77:0/3962309663 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fa019d1f0 con 0x7f4fa01034b0 2024-09-10T09:54:07.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.537+0000 7f4f9de00640 1 --2- 172.21.15.77:0/3962309663 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fa0102b90 0x7f4fa0198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:54:07.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.538+0000 7f4f9d400640 1 --2- 172.21.15.77:0/3962309663 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4fa01034b0 0x7f4fa0198780 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c00e7a0 tx=0x7f4f8c00ec60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:07.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.538+0000 7f4f96a00640 1 -- 172.21.15.77:0/3962309663 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c00ccc0 con 0x7f4fa01034b0 2024-09-10T09:54:07.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.538+0000 7f4f96a00640 1 -- 172.21.15.77:0/3962309663 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f8c009530 con 0x7f4fa01034b0 2024-09-10T09:54:07.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.538+0000 7f4f9f200640 1 -- 172.21.15.77:0/3962309663 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fa019d4e0 con 0x7f4fa01034b0 2024-09-10T09:54:07.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.538+0000 7f4f96a00640 1 -- 172.21.15.77:0/3962309663 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c018880 con 0x7f4fa01034b0 2024-09-10T09:54:07.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.538+0000 7f4f9f200640 1 -- 172.21.15.77:0/3962309663 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fa01a4dc0 con 0x7f4fa01034b0 2024-09-10T09:54:07.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.539+0000 7f4f9f200640 1 -- 172.21.15.77:0/3962309663 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fa0102f70 con 0x7f4fa01034b0 2024-09-10T09:54:07.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.541+0000 7f4f96a00640 1 -- 172.21.15.77:0/3962309663 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f4f8c018a20 con 0x7f4fa01034b0 2024-09-10T09:54:07.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.543+0000 7f4f96a00640 1 --2- 172.21.15.77:0/3962309663 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f4f840a2c10 0x7f4f840a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:07.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.543+0000 7f4f96a00640 1 -- 172.21.15.77:0/3962309663 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f4f8c0cfc20 con 0x7f4fa01034b0 2024-09-10T09:54:07.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.543+0000 7f4f9de00640 1 --2- 172.21.15.77:0/3962309663 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f4f840a2c10 0x7f4f840a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:07.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.543+0000 7f4f9de00640 1 --2- 172.21.15.77:0/3962309663 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f4f840a2c10 0x7f4f840a50d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4f900096c0 tx=0x7f4f90009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:07.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.545+0000 7f4f96a00640 1 -- 172.21.15.77:0/3962309663 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4f8c09bf80 con 0x7f4fa01034b0 2024-09-10T09:54:07.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.733+0000 7f4f9f200640 1 -- 172.21.15.77:0/3962309663 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f4fa002d7c0 con 0x7f4fa01034b0 2024-09-10T09:54:07.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.734+0000 7f4f96a00640 1 -- 172.21.15.77:0/3962309663 <== mon.2 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f4f8c0a0280 con 0x7f4fa01034b0 2024-09-10T09:54:07.736 INFO:teuthology.orchestra.run.smithi077.stdout:335007449123 2024-09-10T09:54:07.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.736+0000 7f4f9f200640 1 -- 172.21.15.77:0/3962309663 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f4f840a2c10 msgr2=0x7f4f840a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:07.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.736+0000 7f4f9f200640 1 --2- 172.21.15.77:0/3962309663 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f4f840a2c10 0x7f4f840a50d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4f900096c0 tx=0x7f4f90009260 comp rx=0 tx=0).stop 2024-09-10T09:54:07.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.736+0000 7f4f9f200640 1 -- 172.21.15.77:0/3962309663 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4fa01034b0 msgr2=0x7f4fa0198780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:07.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.736+0000 7f4f9f200640 1 --2- 172.21.15.77:0/3962309663 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4fa01034b0 0x7f4fa0198780 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c00e7a0 tx=0x7f4f8c00ec60 comp rx=0 tx=0).stop 2024-09-10T09:54:07.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.737+0000 7f4f9f200640 1 -- 172.21.15.77:0/3962309663 shutdown_connections 2024-09-10T09:54:07.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.737+0000 7f4f9f200640 1 --2- 172.21.15.77:0/3962309663 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f4f840a2c10 0x7f4f840a50d0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:07.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.737+0000 7f4f9f200640 1 --2- 172.21.15.77:0/3962309663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4fa010bfc0 0x7f4fa019cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:07.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.737+0000 7f4f9f200640 1 --2- 172.21.15.77:0/3962309663 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4fa01034b0 0x7f4fa0198780 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:07.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.737+0000 7f4f9f200640 1 --2- 172.21.15.77:0/3962309663 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fa0102b90 0x7f4fa0198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:07.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.737+0000 7f4f9f200640 1 -- 172.21.15.77:0/3962309663 >> 172.21.15.77:0/3962309663 conn(0x7f4fa00fc610 msgr2=0x7f4fa00fd310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:07.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.737+0000 7f4f9f200640 1 -- 172.21.15.77:0/3962309663 shutdown_connections 2024-09-10T09:54:07.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:07.737+0000 7f4f9f200640 1 -- 172.21.15.77:0/3962309663 wait complete. 2024-09-10T09:54:08.007 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:54:07 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/759786634' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-10T09:54:08.007 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:54:07 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/3962309663' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-10T09:54:08.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:54:07 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/759786634' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-10T09:54:08.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:54:07 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/3962309663' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-10T09:54:08.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:54:07 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/759786634' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-10T09:54:08.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:54:07 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/3962309663' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-10T09:54:08.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.319+0000 7f45d7400640 1 -- 172.21.15.77:0/2275886689 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f45d0103ff0 msgr2=0x7f45d01043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:08.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.319+0000 7f45d7400640 1 --2- 172.21.15.77:0/2275886689 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f45d0103ff0 0x7f45d01043d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f45c4009900 tx=0x7f45c402a3f0 comp rx=0 tx=0).stop 2024-09-10T09:54:08.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.319+0000 7f45d7400640 1 -- 172.21.15.77:0/2275886689 shutdown_connections 2024-09-10T09:54:08.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.319+0000 7f45d7400640 1 --2- 172.21.15.77:0/2275886689 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45d0109120 0x7f45d0110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:08.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.319+0000 7f45d7400640 1 --2- 172.21.15.77:0/2275886689 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f45d01049a0 0x7f45d01089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:08.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.319+0000 7f45d7400640 1 --2- 172.21.15.77:0/2275886689 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f45d0103ff0 0x7f45d01043d0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:08.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.319+0000 7f45d7400640 1 -- 172.21.15.77:0/2275886689 >> 172.21.15.77:0/2275886689 conn(0x7f45d00ffa90 msgr2=0x7f45d0101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:08.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.319+0000 7f45d7400640 1 -- 172.21.15.77:0/2275886689 shutdown_connections 2024-09-10T09:54:08.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.319+0000 7f45d7400640 1 -- 172.21.15.77:0/2275886689 wait complete. 2024-09-10T09:54:08.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.320+0000 7f45d7400640 1 Processor -- start 2024-09-10T09:54:08.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.320+0000 7f45d7400640 1 -- start start 2024-09-10T09:54:08.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.320+0000 7f45d7400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f45d0103ff0 0x7f45d019a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:08.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.320+0000 7f45d7400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f45d01049a0 0x7f45d019ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:08.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.320+0000 7f45d7400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45d0109120 0x7f45d019eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:08.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.320+0000 7f45d7400640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f45d01141a0 con 0x7f45d0103ff0 2024-09-10T09:54:08.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.320+0000 7f45d7400640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f45d0114020 con 0x7f45d01049a0 2024-09-10T09:54:08.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.320+0000 7f45d7400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f45d0114320 con 0x7f45d0109120 2024-09-10T09:54:08.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.321+0000 7f45d4a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f45d0103ff0 0x7f45d019a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:08.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.321+0000 7f45d4a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f45d0103ff0 0x7f45d019a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:44378/0 (socket says 172.21.15.77:44378) 2024-09-10T09:54:08.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.321+0000 7f45d4a00640 1 -- 172.21.15.77:0/2418052200 learned_addr learned my addr 172.21.15.77:0/2418052200 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:54:08.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.321+0000 7f45cfe00640 1 --2- 172.21.15.77:0/2418052200 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f45d01049a0 0x7f45d019ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:08.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.321+0000 7f45d5400640 1 --2- 172.21.15.77:0/2418052200 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45d0109120 0x7f45d019eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:08.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.321+0000 7f45d4a00640 1 -- 172.21.15.77:0/2418052200 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45d0109120 msgr2=0x7f45d019eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:08.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.321+0000 7f45d4a00640 1 --2- 172.21.15.77:0/2418052200 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45d0109120 0x7f45d019eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:08.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.321+0000 7f45d4a00640 1 -- 172.21.15.77:0/2418052200 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f45d01049a0 msgr2=0x7f45d019ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:08.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.321+0000 7f45d4a00640 1 --2- 172.21.15.77:0/2418052200 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f45d01049a0 0x7f45d019ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:08.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.321+0000 7f45d4a00640 1 -- 172.21.15.77:0/2418052200 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45d019f5d0 con 0x7f45d0103ff0 2024-09-10T09:54:08.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.321+0000 7f45cfe00640 1 --2- 172.21.15.77:0/2418052200 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f45d01049a0 0x7f45d019ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:54:08.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.322+0000 7f45d4a00640 1 --2- 172.21.15.77:0/2418052200 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f45d0103ff0 0x7f45d019a620 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f45c4004770 tx=0x7f45c4003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:08.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.322+0000 7f45cd600640 1 -- 172.21.15.77:0/2418052200 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c403f070 con 0x7f45d0103ff0 2024-09-10T09:54:08.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.322+0000 7f45cd600640 1 -- 172.21.15.77:0/2418052200 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45c4031ca0 con 0x7f45d0103ff0 2024-09-10T09:54:08.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.322+0000 7f45d7400640 1 -- 172.21.15.77:0/2418052200 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45d019f860 con 0x7f45d0103ff0 2024-09-10T09:54:08.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.322+0000 7f45cd600640 1 -- 172.21.15.77:0/2418052200 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c4035030 con 0x7f45d0103ff0 2024-09-10T09:54:08.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.322+0000 7f45d7400640 1 -- 172.21.15.77:0/2418052200 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45d01a71a0 con 0x7f45d0103ff0 2024-09-10T09:54:08.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.323+0000 7f45d7400640 1 -- 172.21.15.77:0/2418052200 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45d01054d0 con 0x7f45d0103ff0 2024-09-10T09:54:08.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.325+0000 7f45cd600640 1 -- 172.21.15.77:0/2418052200 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f45c4042070 con 0x7f45d0103ff0 2024-09-10T09:54:08.328 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.326+0000 7f45cd600640 1 --2- 172.21.15.77:0/2418052200 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f45b80a2c10 0x7f45b80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:08.328 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.326+0000 7f45cd600640 1 -- 172.21.15.77:0/2418052200 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f45c40efca0 con 0x7f45d0103ff0 2024-09-10T09:54:08.328 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.327+0000 7f45cfe00640 1 --2- 172.21.15.77:0/2418052200 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f45b80a2c10 0x7f45b80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:08.329 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.327+0000 7f45cfe00640 1 --2- 172.21.15.77:0/2418052200 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f45b80a2c10 0x7f45b80a50d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f45d00fff90 tx=0x7f45bc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:08.332 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.330+0000 7f45cd600640 1 -- 172.21.15.77:0/2418052200 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f45c40bc000 con 0x7f45d0103ff0 2024-09-10T09:54:08.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.512+0000 7f45d7400640 1 -- 172.21.15.77:0/2418052200 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f45d010c300 con 0x7f45d0103ff0 2024-09-10T09:54:08.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.512+0000 7f45cd600640 1 -- 172.21.15.77:0/2418052200 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f45c40c0300 con 0x7f45d0103ff0 2024-09-10T09:54:08.514 INFO:teuthology.orchestra.run.smithi077.stdout:107374182477 2024-09-10T09:54:08.517 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.515+0000 7f45d7400640 1 -- 172.21.15.77:0/2418052200 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f45b80a2c10 msgr2=0x7f45b80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:08.517 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.515+0000 7f45d7400640 1 --2- 172.21.15.77:0/2418052200 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f45b80a2c10 0x7f45b80a50d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f45d00fff90 tx=0x7f45bc005f90 comp rx=0 tx=0).stop 2024-09-10T09:54:08.517 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.515+0000 7f45d7400640 1 -- 172.21.15.77:0/2418052200 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f45d0103ff0 msgr2=0x7f45d019a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:08.517 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.515+0000 7f45d7400640 1 --2- 172.21.15.77:0/2418052200 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f45d0103ff0 0x7f45d019a620 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f45c4004770 tx=0x7f45c4003c10 comp rx=0 tx=0).stop 2024-09-10T09:54:08.517 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.515+0000 7f45d7400640 1 -- 172.21.15.77:0/2418052200 shutdown_connections 2024-09-10T09:54:08.517 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.515+0000 7f45d7400640 1 --2- 172.21.15.77:0/2418052200 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f45b80a2c10 0x7f45b80a50d0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:08.517 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.515+0000 7f45d7400640 1 --2- 172.21.15.77:0/2418052200 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45d0109120 0x7f45d019eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:08.517 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.515+0000 7f45d7400640 1 --2- 172.21.15.77:0/2418052200 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f45d01049a0 0x7f45d019ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:08.517 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.515+0000 7f45d7400640 1 --2- 172.21.15.77:0/2418052200 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f45d0103ff0 0x7f45d019a620 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:08.517 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.515+0000 7f45d7400640 1 -- 172.21.15.77:0/2418052200 >> 172.21.15.77:0/2418052200 conn(0x7f45d00ffa90 msgr2=0x7f45d0100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:08.517 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.515+0000 7f45d7400640 1 -- 172.21.15.77:0/2418052200 shutdown_connections 2024-09-10T09:54:08.517 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.515+0000 7f45d7400640 1 -- 172.21.15.77:0/2418052200 wait complete. 2024-09-10T09:54:08.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.532+0000 7fe600a00640 1 -- 172.21.15.77:0/3703749296 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe5fc1049a0 msgr2=0x7fe5fc1089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:08.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.532+0000 7fe600a00640 1 --2- 172.21.15.77:0/3703749296 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe5fc1049a0 0x7fe5fc1089f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe5e8009900 tx=0x7fe5e802a3f0 comp rx=0 tx=0).stop 2024-09-10T09:54:08.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.532+0000 7fe600a00640 1 -- 172.21.15.77:0/3703749296 shutdown_connections 2024-09-10T09:54:08.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.532+0000 7fe600a00640 1 --2- 172.21.15.77:0/3703749296 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe5fc109120 0x7fe5fc110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:08.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.532+0000 7fe600a00640 1 --2- 172.21.15.77:0/3703749296 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe5fc1049a0 0x7fe5fc1089f0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:08.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.532+0000 7fe600a00640 1 --2- 172.21.15.77:0/3703749296 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5fc103ff0 0x7fe5fc1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:08.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.532+0000 7fe600a00640 1 -- 172.21.15.77:0/3703749296 >> 172.21.15.77:0/3703749296 conn(0x7fe5fc0ffa90 msgr2=0x7fe5fc101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:08.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.532+0000 7fe600a00640 1 -- 172.21.15.77:0/3703749296 shutdown_connections 2024-09-10T09:54:08.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.532+0000 7fe600a00640 1 -- 172.21.15.77:0/3703749296 wait complete. 2024-09-10T09:54:08.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.533+0000 7fe600a00640 1 Processor -- start 2024-09-10T09:54:08.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.533+0000 7fe600a00640 1 -- start start 2024-09-10T09:54:08.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.534+0000 7fe600a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe5fc103ff0 0x7fe5fc19a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:08.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.534+0000 7fe600a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5fc1049a0 0x7fe5fc19ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:08.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.534+0000 7fe600a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe5fc109120 0x7fe5fc19efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:08.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.534+0000 7fe600a00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe5fc114170 con 0x7fe5fc103ff0 2024-09-10T09:54:08.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.534+0000 7fe600a00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe5fc113ff0 con 0x7fe5fc1049a0 2024-09-10T09:54:08.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.534+0000 7fe600a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe5fc1142f0 con 0x7fe5fc109120 2024-09-10T09:54:08.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.534+0000 7fe5f9e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe5fc103ff0 0x7fe5fc19a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:08.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.534+0000 7fe5f9e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe5fc103ff0 0x7fe5fc19a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:44400/0 (socket says 172.21.15.77:44400) 2024-09-10T09:54:08.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.534+0000 7fe5f9e00640 1 -- 172.21.15.77:0/3864670970 learned_addr learned my addr 172.21.15.77:0/3864670970 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:54:08.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.534+0000 7fe5f9400640 1 --2- 172.21.15.77:0/3864670970 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5fc1049a0 0x7fe5fc19ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:08.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.534+0000 7fe5fa800640 1 --2- 172.21.15.77:0/3864670970 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe5fc109120 0x7fe5fc19efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:08.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.534+0000 7fe5f9e00640 1 -- 172.21.15.77:0/3864670970 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe5fc109120 msgr2=0x7fe5fc19efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:08.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.534+0000 7fe5f9e00640 1 --2- 172.21.15.77:0/3864670970 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe5fc109120 0x7fe5fc19efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:08.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.534+0000 7fe5f9e00640 1 -- 172.21.15.77:0/3864670970 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5fc1049a0 msgr2=0x7fe5fc19ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:08.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.534+0000 7fe5f9e00640 1 --2- 172.21.15.77:0/3864670970 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5fc1049a0 0x7fe5fc19ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:08.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.534+0000 7fe5f9e00640 1 -- 172.21.15.77:0/3864670970 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5fc19f680 con 0x7fe5fc103ff0 2024-09-10T09:54:08.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.535+0000 7fe5f9e00640 1 --2- 172.21.15.77:0/3864670970 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe5fc103ff0 0x7fe5fc19a6d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec00e9f0 tx=0x7fe5ec00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:08.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.535+0000 7fe5f2a00640 1 -- 172.21.15.77:0/3864670970 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5ec00ccc0 con 0x7fe5fc103ff0 2024-09-10T09:54:08.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.535+0000 7fe600a00640 1 -- 172.21.15.77:0/3864670970 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5fc19f970 con 0x7fe5fc103ff0 2024-09-10T09:54:08.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.535+0000 7fe5f2a00640 1 -- 172.21.15.77:0/3864670970 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5ec010910 con 0x7fe5fc103ff0 2024-09-10T09:54:08.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.535+0000 7fe5f2a00640 1 -- 172.21.15.77:0/3864670970 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5ec0188a0 con 0x7fe5fc103ff0 2024-09-10T09:54:08.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.535+0000 7fe600a00640 1 -- 172.21.15.77:0/3864670970 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5fc1a7250 con 0x7fe5fc103ff0 2024-09-10T09:54:08.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.537+0000 7fe600a00640 1 -- 172.21.15.77:0/3864670970 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5fc1043d0 con 0x7fe5fc103ff0 2024-09-10T09:54:08.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.538+0000 7fe5f2a00640 1 -- 172.21.15.77:0/3864670970 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fe5ec010ac0 con 0x7fe5fc103ff0 2024-09-10T09:54:08.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.540+0000 7fe5f2a00640 1 --2- 172.21.15.77:0/3864670970 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fe5e00a2c90 0x7fe5e00a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:08.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.541+0000 7fe5f2a00640 1 -- 172.21.15.77:0/3864670970 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fe5ec0cf9d0 con 0x7fe5fc103ff0 2024-09-10T09:54:08.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.541+0000 7fe5f9400640 1 --2- 172.21.15.77:0/3864670970 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fe5e00a2c90 0x7fe5e00a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:08.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.542+0000 7fe5f9400640 1 --2- 172.21.15.77:0/3864670970 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fe5e00a2c90 0x7fe5e00a5150 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe5e8000c00 tx=0x7fe5e8005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:08.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.545+0000 7fe5f2a00640 1 -- 172.21.15.77:0/3864670970 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe5ec09c380 con 0x7fe5fc103ff0 2024-09-10T09:54:08.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.725+0000 7fe600a00640 1 -- 172.21.15.77:0/3864670970 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7fe5fc10c390 con 0x7fe5fc103ff0 2024-09-10T09:54:08.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.726+0000 7fe5f2a00640 1 -- 172.21.15.77:0/3864670970 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fe5ec09bc80 con 0x7fe5fc103ff0 2024-09-10T09:54:08.728 INFO:teuthology.orchestra.run.smithi077.stdout:300647710758 2024-09-10T09:54:08.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.728+0000 7fe600a00640 1 -- 172.21.15.77:0/3864670970 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fe5e00a2c90 msgr2=0x7fe5e00a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:08.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.728+0000 7fe600a00640 1 --2- 172.21.15.77:0/3864670970 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fe5e00a2c90 0x7fe5e00a5150 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe5e8000c00 tx=0x7fe5e8005b50 comp rx=0 tx=0).stop 2024-09-10T09:54:08.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.728+0000 7fe600a00640 1 -- 172.21.15.77:0/3864670970 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe5fc103ff0 msgr2=0x7fe5fc19a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:08.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.728+0000 7fe600a00640 1 --2- 172.21.15.77:0/3864670970 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe5fc103ff0 0x7fe5fc19a6d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec00e9f0 tx=0x7fe5ec00edf0 comp rx=0 tx=0).stop 2024-09-10T09:54:08.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.728+0000 7fe600a00640 1 -- 172.21.15.77:0/3864670970 shutdown_connections 2024-09-10T09:54:08.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.728+0000 7fe600a00640 1 --2- 172.21.15.77:0/3864670970 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fe5e00a2c90 0x7fe5e00a5150 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:08.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.728+0000 7fe600a00640 1 --2- 172.21.15.77:0/3864670970 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe5fc109120 0x7fe5fc19efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:08.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.728+0000 7fe600a00640 1 --2- 172.21.15.77:0/3864670970 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5fc1049a0 0x7fe5fc19ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:08.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.728+0000 7fe600a00640 1 --2- 172.21.15.77:0/3864670970 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe5fc103ff0 0x7fe5fc19a6d0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:08.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.728+0000 7fe600a00640 1 -- 172.21.15.77:0/3864670970 >> 172.21.15.77:0/3864670970 conn(0x7fe5fc0ffa90 msgr2=0x7fe5fc100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:08.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.729+0000 7fe600a00640 1 -- 172.21.15.77:0/3864670970 shutdown_connections 2024-09-10T09:54:08.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:08.729+0000 7fe600a00640 1 -- 172.21.15.77:0/3864670970 wait complete. 2024-09-10T09:54:09.128 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:54:08 smithi077 ceph-mon[30319]: pgmap v360: 129 pgs: 129 active+clean; 645 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-10T09:54:09.128 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:54:08 smithi077 ceph-mon[30319]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-09-10T09:54:09.128 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:54:08 smithi077 ceph-mon[30319]: Cluster is now healthy 2024-09-10T09:54:09.128 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:54:08 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/2418052200' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-10T09:54:09.128 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:54:08 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/3864670970' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-10T09:54:09.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:54:08 smithi134 ceph-mon[35674]: pgmap v360: 129 pgs: 129 active+clean; 645 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-10T09:54:09.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:54:08 smithi134 ceph-mon[35674]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-09-10T09:54:09.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:54:08 smithi134 ceph-mon[35674]: Cluster is now healthy 2024-09-10T09:54:09.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:54:08 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/2418052200' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-10T09:54:09.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:54:08 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/3864670970' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-10T09:54:09.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:54:08 smithi120 ceph-mon[34770]: pgmap v360: 129 pgs: 129 active+clean; 645 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-10T09:54:09.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:54:08 smithi120 ceph-mon[34770]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-09-10T09:54:09.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:54:08 smithi120 ceph-mon[34770]: Cluster is now healthy 2024-09-10T09:54:09.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:54:08 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/2418052200' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-10T09:54:09.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:54:08 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/3864670970' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-10T09:54:09.682 INFO:tasks.cephadm.ceph_manager.ceph:need seq 193273528379 got 193273528382 for osd.6 2024-09-10T09:54:09.683 DEBUG:teuthology.parallel:result is None 2024-09-10T09:54:10.667 INFO:tasks.cephadm.ceph_manager.ceph:need seq 300647710756 got 300647710758 for osd.10 2024-09-10T09:54:10.667 DEBUG:teuthology.parallel:result is None 2024-09-10T09:54:10.974 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182475 got 107374182477 for osd.3 2024-09-10T09:54:10.974 DEBUG:teuthology.parallel:result is None 2024-09-10T09:54:11.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:54:10 smithi120 ceph-mon[34770]: pgmap v361: 129 pgs: 129 active+clean; 645 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-09-10T09:54:11.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:54:10 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/2034471539' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-10T09:54:11.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:54:10 smithi077 ceph-mon[30319]: pgmap v361: 129 pgs: 129 active+clean; 645 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-09-10T09:54:11.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:54:10 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/2034471539' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-10T09:54:11.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:54:10 smithi134 ceph-mon[35674]: pgmap v361: 129 pgs: 129 active+clean; 645 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-09-10T09:54:11.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:54:10 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/2034471539' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-10T09:54:11.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:10.409+0000 7f1ffe400640 1 -- 172.21.15.77:0/1125525793 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1ff8102360 msgr2=0x7f1ff810e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:11.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:10.409+0000 7f1ffe400640 1 --2- 172.21.15.77:0/1125525793 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1ff8102360 0x7f1ff810e890 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f1ff0009cd0 tx=0x7f1ff002a8f0 comp rx=0 tx=0).stop 2024-09-10T09:54:11.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:10.409+0000 7f1ffe400640 1 -- 172.21.15.77:0/1125525793 shutdown_connections 2024-09-10T09:54:11.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:10.409+0000 7f1ffe400640 1 --2- 172.21.15.77:0/1125525793 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1ff8102360 0x7f1ff810e890 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:11.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:10.409+0000 7f1ffe400640 1 --2- 172.21.15.77:0/1125525793 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1ff81019c0 0x7f1ff8101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:11.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:10.409+0000 7f1ffe400640 1 --2- 172.21.15.77:0/1125525793 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ff81079c0 0x7f1ff8107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:11.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:10.409+0000 7f1ffe400640 1 -- 172.21.15.77:0/1125525793 >> 172.21.15.77:0/1125525793 conn(0x7f1ff80fd740 msgr2=0x7f1ff80ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:11.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:10.409+0000 7f1ffe400640 1 -- 172.21.15.77:0/1125525793 shutdown_connections 2024-09-10T09:54:11.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:10.409+0000 7f1ffe400640 1 -- 172.21.15.77:0/1125525793 wait complete. 2024-09-10T09:54:11.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:10.410+0000 7f1ffe400640 1 Processor -- start 2024-09-10T09:54:11.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:10.410+0000 7f1ffe400640 1 -- start start 2024-09-10T09:54:11.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:10.410+0000 7f1ffe400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1ff81019c0 0x7f1ff819a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:11.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:10.410+0000 7f1ffe400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1ff8102360 0x7f1ff819a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:11.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:10.410+0000 7f1ffe400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ff81079c0 0x7f1ff819ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:11.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:10.410+0000 7f1ffe400640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff8111dc0 con 0x7f1ff8102360 2024-09-10T09:54:11.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:10.410+0000 7f1ffe400640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff8111c40 con 0x7f1ff81019c0 2024-09-10T09:54:11.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:10.410+0000 7f1ffe400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff8111f40 con 0x7f1ff81079c0 2024-09-10T09:54:11.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:10.410+0000 7f1ff7e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1ff8102360 0x7f1ff819a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:11.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:10.411+0000 7f1ff7e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1ff8102360 0x7f1ff819a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:44420/0 (socket says 172.21.15.77:44420) 2024-09-10T09:54:11.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:10.411+0000 7f1ff7e00640 1 -- 172.21.15.77:0/2034471539 learned_addr learned my addr 172.21.15.77:0/2034471539 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:54:11.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:10.411+0000 7f1ffda00640 1 --2- 172.21.15.77:0/2034471539 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ff81079c0 0x7f1ff819ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:11.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:10.411+0000 7f1ffd000640 1 --2- 172.21.15.77:0/2034471539 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1ff81019c0 0x7f1ff819a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:11.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:10.411+0000 7f1ff7e00640 1 -- 172.21.15.77:0/2034471539 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ff81079c0 msgr2=0x7f1ff819ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:11.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:10.411+0000 7f1ff7e00640 1 --2- 172.21.15.77:0/2034471539 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ff81079c0 0x7f1ff819ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:11.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:10.411+0000 7f1ff7e00640 1 -- 172.21.15.77:0/2034471539 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1ff81019c0 msgr2=0x7f1ff819a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:11.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:10.411+0000 7f1ff7e00640 1 --2- 172.21.15.77:0/2034471539 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1ff81019c0 0x7f1ff819a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:11.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:10.411+0000 7f1ff7e00640 1 -- 172.21.15.77:0/2034471539 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ff819f3e0 con 0x7f1ff8102360 2024-09-10T09:54:11.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:10.411+0000 7f1ffd000640 1 --2- 172.21.15.77:0/2034471539 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1ff81019c0 0x7f1ff819a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:54:11.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:10.411+0000 7f1ff7e00640 1 --2- 172.21.15.77:0/2034471539 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1ff8102360 0x7f1ff819a970 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f1fe400e9f0 tx=0x7f1fe400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:11.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:10.412+0000 7f1ff5600640 1 -- 172.21.15.77:0/2034471539 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe400cd80 con 0x7f1ff8102360 2024-09-10T09:54:11.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:10.412+0000 7f1ff5600640 1 -- 172.21.15.77:0/2034471539 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fe4010910 con 0x7f1ff8102360 2024-09-10T09:54:11.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:10.412+0000 7f1ffe400640 1 -- 172.21.15.77:0/2034471539 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ff819f6d0 con 0x7f1ff8102360 2024-09-10T09:54:11.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:10.412+0000 7f1ff5600640 1 -- 172.21.15.77:0/2034471539 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe4018990 con 0x7f1ff8102360 2024-09-10T09:54:11.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:10.412+0000 7f1ffe400640 1 -- 172.21.15.77:0/2034471539 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ff81a6fb0 con 0x7f1ff8102360 2024-09-10T09:54:11.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:10.413+0000 7f1ffe400640 1 -- 172.21.15.77:0/2034471539 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ff8101e20 con 0x7f1ff8102360 2024-09-10T09:54:11.982 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:10.415+0000 7f1ff5600640 1 -- 172.21.15.77:0/2034471539 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f1fe4010ac0 con 0x7f1ff8102360 2024-09-10T09:54:11.982 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:10.416+0000 7f1ff5600640 1 --2- 172.21.15.77:0/2034471539 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f1fdc0a2c90 0x7f1fdc0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:11.982 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:10.416+0000 7f1ff5600640 1 -- 172.21.15.77:0/2034471539 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f1fe40cf830 con 0x7f1ff8102360 2024-09-10T09:54:11.982 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:10.416+0000 7f1ffd000640 1 --2- 172.21.15.77:0/2034471539 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f1fdc0a2c90 0x7f1fdc0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:11.982 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:10.417+0000 7f1ffd000640 1 --2- 172.21.15.77:0/2034471539 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f1fdc0a2c90 0x7f1fdc0a5150 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1fe8009f40 tx=0x7f1fe8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:11.982 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:10.420+0000 7f1ff5600640 1 -- 172.21.15.77:0/2034471539 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1fe409bbf0 con 0x7f1ff8102360 2024-09-10T09:54:11.982 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:10.602+0000 7f1ffe400640 1 -- 172.21.15.77:0/2034471539 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f1ff81a7250 con 0x7f1ff8102360 2024-09-10T09:54:11.982 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:10.603+0000 7f1ff5600640 1 -- 172.21.15.77:0/2034471539 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1fe409fef0 con 0x7f1ff8102360 2024-09-10T09:54:11.982 INFO:teuthology.orchestra.run.smithi077.stdout:77309411412 2024-09-10T09:54:11.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:10.605+0000 7f1ffe400640 1 -- 172.21.15.77:0/2034471539 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f1fdc0a2c90 msgr2=0x7f1fdc0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:11.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:10.605+0000 7f1ffe400640 1 --2- 172.21.15.77:0/2034471539 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f1fdc0a2c90 0x7f1fdc0a5150 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1fe8009f40 tx=0x7f1fe8005f90 comp rx=0 tx=0).stop 2024-09-10T09:54:11.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:10.605+0000 7f1ffe400640 1 -- 172.21.15.77:0/2034471539 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1ff8102360 msgr2=0x7f1ff819a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:11.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:10.605+0000 7f1ffe400640 1 --2- 172.21.15.77:0/2034471539 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1ff8102360 0x7f1ff819a970 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f1fe400e9f0 tx=0x7f1fe400eeb0 comp rx=0 tx=0).stop 2024-09-10T09:54:11.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:10.605+0000 7f1ffe400640 1 -- 172.21.15.77:0/2034471539 shutdown_connections 2024-09-10T09:54:11.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:10.605+0000 7f1ffe400640 1 --2- 172.21.15.77:0/2034471539 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f1fdc0a2c90 0x7f1fdc0a5150 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:11.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:10.605+0000 7f1ffe400640 1 --2- 172.21.15.77:0/2034471539 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ff81079c0 0x7f1ff819ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:11.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:10.605+0000 7f1ffe400640 1 --2- 172.21.15.77:0/2034471539 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1ff8102360 0x7f1ff819a970 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:11.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:10.605+0000 7f1ffe400640 1 --2- 172.21.15.77:0/2034471539 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1ff81019c0 0x7f1ff819a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:11.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:10.605+0000 7f1ffe400640 1 -- 172.21.15.77:0/2034471539 >> 172.21.15.77:0/2034471539 conn(0x7f1ff80fd740 msgr2=0x7f1ff80fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:11.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:10.605+0000 7f1ffe400640 1 -- 172.21.15.77:0/2034471539 shutdown_connections 2024-09-10T09:54:11.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:10.605+0000 7f1ffe400640 1 -- 172.21.15.77:0/2034471539 wait complete. 2024-09-10T09:54:12.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.251+0000 7f1cce000640 1 -- 172.21.15.77:0/2613783010 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1cc8104010 msgr2=0x7f1cc81043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:12.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.251+0000 7f1cce000640 1 --2- 172.21.15.77:0/2613783010 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1cc8104010 0x7f1cc81043f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1cb8009900 tx=0x7f1cb802a410 comp rx=0 tx=0).stop 2024-09-10T09:54:12.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.251+0000 7f1cce000640 1 -- 172.21.15.77:0/2613783010 shutdown_connections 2024-09-10T09:54:12.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.251+0000 7f1cce000640 1 --2- 172.21.15.77:0/2613783010 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1cc8109140 0x7f1cc8110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:12.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.251+0000 7f1cce000640 1 --2- 172.21.15.77:0/2613783010 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1cc81049c0 0x7f1cc8108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:12.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.251+0000 7f1cce000640 1 --2- 172.21.15.77:0/2613783010 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1cc8104010 0x7f1cc81043f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:12.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.251+0000 7f1cce000640 1 -- 172.21.15.77:0/2613783010 >> 172.21.15.77:0/2613783010 conn(0x7f1cc80ffa50 msgr2=0x7f1cc8101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:12.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.251+0000 7f1cce000640 1 -- 172.21.15.77:0/2613783010 shutdown_connections 2024-09-10T09:54:12.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.251+0000 7f1cce000640 1 -- 172.21.15.77:0/2613783010 wait complete. 2024-09-10T09:54:12.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.252+0000 7f1cce000640 1 Processor -- start 2024-09-10T09:54:12.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.253+0000 7f1cce000640 1 -- start start 2024-09-10T09:54:12.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.253+0000 7f1cce000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1cc8104010 0x7f1cc819a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:12.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.253+0000 7f1cce000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1cc81049c0 0x7f1cc819ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:12.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.253+0000 7f1cce000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1cc8109140 0x7f1cc819eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:12.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.253+0000 7f1cce000640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc8114160 con 0x7f1cc8109140 2024-09-10T09:54:12.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.253+0000 7f1cce000640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc8113fe0 con 0x7f1cc81049c0 2024-09-10T09:54:12.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.253+0000 7f1cce000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc81142e0 con 0x7f1cc8104010 2024-09-10T09:54:12.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.253+0000 7f1cc7400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1cc8104010 0x7f1cc819a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:12.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.253+0000 7f1cc7400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1cc8104010 0x7f1cc819a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:44858/0 (socket says 172.21.15.77:44858) 2024-09-10T09:54:12.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.253+0000 7f1cc7400640 1 -- 172.21.15.77:0/326713419 learned_addr learned my addr 172.21.15.77:0/326713419 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:54:12.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.253+0000 7f1cc7e00640 1 --2- 172.21.15.77:0/326713419 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1cc8109140 0x7f1cc819eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:12.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.254+0000 7f1cc6a00640 1 --2- 172.21.15.77:0/326713419 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1cc81049c0 0x7f1cc819ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:12.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.254+0000 7f1cc7400640 1 -- 172.21.15.77:0/326713419 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1cc81049c0 msgr2=0x7f1cc819ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:12.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.254+0000 7f1cc7400640 1 --2- 172.21.15.77:0/326713419 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1cc81049c0 0x7f1cc819ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:12.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.254+0000 7f1cc7400640 1 -- 172.21.15.77:0/326713419 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1cc8109140 msgr2=0x7f1cc819eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:12.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.254+0000 7f1cc7400640 1 --2- 172.21.15.77:0/326713419 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1cc8109140 0x7f1cc819eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:12.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.254+0000 7f1cc7400640 1 -- 172.21.15.77:0/326713419 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cc819f5d0 con 0x7f1cc8104010 2024-09-10T09:54:12.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.254+0000 7f1cc7400640 1 --2- 172.21.15.77:0/326713419 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1cc8104010 0x7f1cc819a620 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1cb8006010 tx=0x7f1cb8004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:12.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.255+0000 7f1cbfe00640 1 -- 172.21.15.77:0/326713419 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb803f030 con 0x7f1cc8104010 2024-09-10T09:54:12.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.255+0000 7f1cce000640 1 -- 172.21.15.77:0/326713419 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cc819f860 con 0x7f1cc8104010 2024-09-10T09:54:12.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.255+0000 7f1cbfe00640 1 -- 172.21.15.77:0/326713419 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cb8004440 con 0x7f1cc8104010 2024-09-10T09:54:12.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.255+0000 7f1cbfe00640 1 -- 172.21.15.77:0/326713419 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb8035070 con 0x7f1cc8104010 2024-09-10T09:54:12.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.255+0000 7f1cce000640 1 -- 172.21.15.77:0/326713419 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cc81a71a0 con 0x7f1cc8104010 2024-09-10T09:54:12.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.256+0000 7f1cce000640 1 -- 172.21.15.77:0/326713419 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cc81043f0 con 0x7f1cc8104010 2024-09-10T09:54:12.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.258+0000 7f1cbfe00640 1 -- 172.21.15.77:0/326713419 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f1cb8042030 con 0x7f1cc8104010 2024-09-10T09:54:12.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.259+0000 7f1cbfe00640 1 --2- 172.21.15.77:0/326713419 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f1cac0a2c90 0x7f1cac0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:12.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.260+0000 7f1cbfe00640 1 -- 172.21.15.77:0/326713419 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f1cb80efc10 con 0x7f1cc8104010 2024-09-10T09:54:12.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.260+0000 7f1cc6a00640 1 --2- 172.21.15.77:0/326713419 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f1cac0a2c90 0x7f1cac0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:12.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.261+0000 7f1cc6a00640 1 --2- 172.21.15.77:0/326713419 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f1cac0a2c90 0x7f1cac0a5150 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1cc80ffe70 tx=0x7f1cb4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:12.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.261+0000 7f1cbfe00640 1 -- 172.21.15.77:0/326713419 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1cb80bc030 con 0x7f1cc8104010 2024-09-10T09:54:12.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.455+0000 7f1cce000640 1 -- 172.21.15.77:0/326713419 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f1cc810c320 con 0x7f1cc8104010 2024-09-10T09:54:12.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.456+0000 7f1cbfe00640 1 -- 172.21.15.77:0/326713419 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1cb80c0330 con 0x7f1cc8104010 2024-09-10T09:54:12.074 INFO:teuthology.orchestra.run.smithi077.stdout:34359738462 2024-09-10T09:54:12.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.458+0000 7f1cce000640 1 -- 172.21.15.77:0/326713419 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f1cac0a2c90 msgr2=0x7f1cac0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:12.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.458+0000 7f1cce000640 1 --2- 172.21.15.77:0/326713419 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f1cac0a2c90 0x7f1cac0a5150 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1cc80ffe70 tx=0x7f1cb4005f90 comp rx=0 tx=0).stop 2024-09-10T09:54:12.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.458+0000 7f1cce000640 1 -- 172.21.15.77:0/326713419 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1cc8104010 msgr2=0x7f1cc819a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:12.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.458+0000 7f1cce000640 1 --2- 172.21.15.77:0/326713419 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1cc8104010 0x7f1cc819a620 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1cb8006010 tx=0x7f1cb8004040 comp rx=0 tx=0).stop 2024-09-10T09:54:12.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.458+0000 7f1cce000640 1 -- 172.21.15.77:0/326713419 shutdown_connections 2024-09-10T09:54:12.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.458+0000 7f1cce000640 1 --2- 172.21.15.77:0/326713419 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f1cac0a2c90 0x7f1cac0a5150 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:12.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.458+0000 7f1cce000640 1 --2- 172.21.15.77:0/326713419 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1cc8109140 0x7f1cc819eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:12.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.458+0000 7f1cce000640 1 --2- 172.21.15.77:0/326713419 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1cc81049c0 0x7f1cc819ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:12.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.458+0000 7f1cce000640 1 --2- 172.21.15.77:0/326713419 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1cc8104010 0x7f1cc819a620 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:12.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.458+0000 7f1cce000640 1 -- 172.21.15.77:0/326713419 >> 172.21.15.77:0/326713419 conn(0x7f1cc80ffa50 msgr2=0x7f1cc8100e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:12.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.458+0000 7f1cce000640 1 -- 172.21.15.77:0/326713419 shutdown_connections 2024-09-10T09:54:12.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:06.458+0000 7f1cce000640 1 -- 172.21.15.77:0/326713419 wait complete. 2024-09-10T09:54:12.696 INFO:tasks.cephadm.ceph_manager.ceph:need seq 335007449120 got 335007449123 for osd.11 2024-09-10T09:54:12.696 DEBUG:teuthology.parallel:result is None 2024-09-10T09:54:12.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:54:12 smithi077 ceph-mon[30319]: pgmap v362: 129 pgs: 129 active+clean; 645 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s 2024-09-10T09:54:13.049 INFO:tasks.cephadm.ceph_manager.ceph:need seq 279172874282 got 279172874285 for osd.9 2024-09-10T09:54:13.049 DEBUG:teuthology.parallel:result is None 2024-09-10T09:54:13.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:54:12 smithi120 ceph-mon[34770]: pgmap v362: 129 pgs: 129 active+clean; 645 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s 2024-09-10T09:54:13.183 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:54:12 smithi134 ceph-mon[35674]: pgmap v362: 129 pgs: 129 active+clean; 645 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s 2024-09-10T09:54:13.459 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738458 got 34359738462 for osd.0 2024-09-10T09:54:13.459 DEBUG:teuthology.parallel:result is None 2024-09-10T09:54:13.730 INFO:tasks.cephadm.ceph_manager.ceph:need seq 244813135920 got 244813135923 for osd.8 2024-09-10T09:54:13.730 DEBUG:teuthology.parallel:result is None 2024-09-10T09:54:14.023 INFO:tasks.cephadm.ceph_manager.ceph:need seq 137438953540 got 137438953543 for osd.4 2024-09-10T09:54:14.023 DEBUG:teuthology.parallel:result is None 2024-09-10T09:54:14.217 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:54:14.687 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411408 got 77309411412 for osd.2 2024-09-10T09:54:14.687 DEBUG:teuthology.parallel:result is None 2024-09-10T09:54:14.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:54:14 smithi077 ceph-mon[30319]: pgmap v363: 129 pgs: 129 active+clean; 645 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-10T09:54:15.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:54:14 smithi120 ceph-mon[34770]: pgmap v363: 129 pgs: 129 active+clean; 645 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-10T09:54:15.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:15.243+0000 7efe43200640 1 -- 172.21.15.77:0/1854665618 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efe3c102370 msgr2=0x7efe3c10e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:15.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:15.243+0000 7efe43200640 1 --2- 172.21.15.77:0/1854665618 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efe3c102370 0x7efe3c10e8a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7efe38009cd0 tx=0x7efe3802a8e0 comp rx=0 tx=0).stop 2024-09-10T09:54:15.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:15.243+0000 7efe43200640 1 -- 172.21.15.77:0/1854665618 shutdown_connections 2024-09-10T09:54:15.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:15.243+0000 7efe43200640 1 --2- 172.21.15.77:0/1854665618 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efe3c102370 0x7efe3c10e8a0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:15.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:15.243+0000 7efe43200640 1 --2- 172.21.15.77:0/1854665618 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efe3c1019d0 0x7efe3c101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:15.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:15.243+0000 7efe43200640 1 --2- 172.21.15.77:0/1854665618 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe3c1079d0 0x7efe3c107db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:15.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:15.243+0000 7efe43200640 1 -- 172.21.15.77:0/1854665618 >> 172.21.15.77:0/1854665618 conn(0x7efe3c0fd710 msgr2=0x7efe3c0ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:15.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:15.243+0000 7efe43200640 1 -- 172.21.15.77:0/1854665618 shutdown_connections 2024-09-10T09:54:15.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:15.243+0000 7efe43200640 1 -- 172.21.15.77:0/1854665618 wait complete. 2024-09-10T09:54:15.248 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:15.244+0000 7efe43200640 1 Processor -- start 2024-09-10T09:54:15.248 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:15.244+0000 7efe43200640 1 -- start start 2024-09-10T09:54:15.248 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:15.245+0000 7efe43200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efe3c1019d0 0x7efe3c19a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:15.248 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:15.245+0000 7efe43200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efe3c102370 0x7efe3c19a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:15.248 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:15.245+0000 7efe43200640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe3c1079d0 0x7efe3c19ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:15.248 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:15.245+0000 7efe43200640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7efe3c111e20 con 0x7efe3c1019d0 2024-09-10T09:54:15.248 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:15.245+0000 7efe43200640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7efe3c111ca0 con 0x7efe3c1079d0 2024-09-10T09:54:15.248 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:15.245+0000 7efe43200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7efe3c111fa0 con 0x7efe3c102370 2024-09-10T09:54:15.248 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:15.245+0000 7efe41e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efe3c1019d0 0x7efe3c19a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:15.248 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:15.245+0000 7efe41e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efe3c1019d0 0x7efe3c19a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:35354/0 (socket says 172.21.15.77:35354) 2024-09-10T09:54:15.249 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:15.245+0000 7efe41e00640 1 -- 172.21.15.77:0/1258692682 learned_addr learned my addr 172.21.15.77:0/1258692682 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:54:15.249 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:15.245+0000 7efe41400640 1 --2- 172.21.15.77:0/1258692682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efe3c102370 0x7efe3c19a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:15.249 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:15.245+0000 7efe42800640 1 --2- 172.21.15.77:0/1258692682 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe3c1079d0 0x7efe3c19ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:15.249 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:15.246+0000 7efe41e00640 1 -- 172.21.15.77:0/1258692682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efe3c102370 msgr2=0x7efe3c19a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:15.249 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:15.246+0000 7efe41e00640 1 --2- 172.21.15.77:0/1258692682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efe3c102370 0x7efe3c19a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:15.249 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:15.246+0000 7efe41e00640 1 -- 172.21.15.77:0/1258692682 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe3c1079d0 msgr2=0x7efe3c19ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:15.249 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:15.246+0000 7efe41e00640 1 --2- 172.21.15.77:0/1258692682 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe3c1079d0 0x7efe3c19ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:15.249 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:15.246+0000 7efe41e00640 1 -- 172.21.15.77:0/1258692682 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe3c19f410 con 0x7efe3c1019d0 2024-09-10T09:54:15.249 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:15.246+0000 7efe41e00640 1 --2- 172.21.15.77:0/1258692682 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efe3c1019d0 0x7efe3c19a460 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7efe3000e9f0 tx=0x7efe3000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:15.250 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:15.246+0000 7efe36a00640 1 -- 172.21.15.77:0/1258692682 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe3000cd80 con 0x7efe3c1019d0 2024-09-10T09:54:15.250 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:15.247+0000 7efe43200640 1 -- 172.21.15.77:0/1258692682 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe3c19f700 con 0x7efe3c1019d0 2024-09-10T09:54:15.250 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:15.247+0000 7efe36a00640 1 -- 172.21.15.77:0/1258692682 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe30010910 con 0x7efe3c1019d0 2024-09-10T09:54:15.250 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:15.247+0000 7efe43200640 1 -- 172.21.15.77:0/1258692682 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe3c1a6fe0 con 0x7efe3c1019d0 2024-09-10T09:54:15.250 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:15.247+0000 7efe36a00640 1 -- 172.21.15.77:0/1258692682 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe30018930 con 0x7efe3c1019d0 2024-09-10T09:54:15.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:15.248+0000 7efe43200640 1 -- 172.21.15.77:0/1258692682 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe3c101e30 con 0x7efe3c1019d0 2024-09-10T09:54:15.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:15.250+0000 7efe36a00640 1 -- 172.21.15.77:0/1258692682 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7efe30010ac0 con 0x7efe3c1019d0 2024-09-10T09:54:15.253 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:15.251+0000 7efe36a00640 1 --2- 172.21.15.77:0/1258692682 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7efe240a2c10 0x7efe240a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:15.253 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:15.251+0000 7efe36a00640 1 -- 172.21.15.77:0/1258692682 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7efe300cfc70 con 0x7efe3c1019d0 2024-09-10T09:54:15.253 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:15.252+0000 7efe41400640 1 --2- 172.21.15.77:0/1258692682 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7efe240a2c10 0x7efe240a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:15.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:15.252+0000 7efe41400640 1 --2- 172.21.15.77:0/1258692682 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7efe240a2c10 0x7efe240a50d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7efe3c10ed30 tx=0x7efe2c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:15.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:15.254+0000 7efe36a00640 1 -- 172.21.15.77:0/1258692682 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efe3009bfd0 con 0x7efe3c1019d0 2024-09-10T09:54:15.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:54:14 smithi134 ceph-mon[35674]: pgmap v363: 129 pgs: 129 active+clean; 645 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-10T09:54:15.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:15.440+0000 7efe43200640 1 -- 172.21.15.77:0/1258692682 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7efe3c1a72d0 con 0x7efe3c1019d0 2024-09-10T09:54:15.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:15.441+0000 7efe36a00640 1 -- 172.21.15.77:0/1258692682 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7efe300a02d0 con 0x7efe3c1019d0 2024-09-10T09:54:15.443 INFO:teuthology.orchestra.run.smithi077.stdout:158913790022 2024-09-10T09:54:15.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:15.444+0000 7efe43200640 1 -- 172.21.15.77:0/1258692682 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7efe240a2c10 msgr2=0x7efe240a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:15.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:15.444+0000 7efe43200640 1 --2- 172.21.15.77:0/1258692682 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7efe240a2c10 0x7efe240a50d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7efe3c10ed30 tx=0x7efe2c005f90 comp rx=0 tx=0).stop 2024-09-10T09:54:15.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:15.444+0000 7efe43200640 1 -- 172.21.15.77:0/1258692682 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efe3c1019d0 msgr2=0x7efe3c19a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:15.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:15.444+0000 7efe43200640 1 --2- 172.21.15.77:0/1258692682 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efe3c1019d0 0x7efe3c19a460 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7efe3000e9f0 tx=0x7efe3000eeb0 comp rx=0 tx=0).stop 2024-09-10T09:54:15.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:15.444+0000 7efe43200640 1 -- 172.21.15.77:0/1258692682 shutdown_connections 2024-09-10T09:54:15.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:15.444+0000 7efe43200640 1 --2- 172.21.15.77:0/1258692682 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7efe240a2c10 0x7efe240a50d0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:15.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:15.444+0000 7efe43200640 1 --2- 172.21.15.77:0/1258692682 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe3c1079d0 0x7efe3c19ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:15.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:15.444+0000 7efe43200640 1 --2- 172.21.15.77:0/1258692682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efe3c102370 0x7efe3c19a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:15.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:15.444+0000 7efe43200640 1 --2- 172.21.15.77:0/1258692682 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efe3c1019d0 0x7efe3c19a460 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:15.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:15.444+0000 7efe43200640 1 -- 172.21.15.77:0/1258692682 >> 172.21.15.77:0/1258692682 conn(0x7efe3c0fd710 msgr2=0x7efe3c0fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:15.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:15.444+0000 7efe43200640 1 -- 172.21.15.77:0/1258692682 shutdown_connections 2024-09-10T09:54:15.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:15.444+0000 7efe43200640 1 -- 172.21.15.77:0/1258692682 wait complete. 2024-09-10T09:54:16.143 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:54:15 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/1258692682' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-10T09:54:16.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:54:15 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/1258692682' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-10T09:54:16.184 INFO:tasks.cephadm.ceph_manager.ceph:need seq 158913790019 got 158913790022 for osd.5 2024-09-10T09:54:16.184 DEBUG:teuthology.parallel:result is None 2024-09-10T09:54:16.184 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-10T09:54:16.184 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T09:54:16.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:54:15 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/1258692682' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-10T09:54:16.559 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:54:17.047 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:54:16 smithi077 ceph-mon[30319]: pgmap v364: 129 pgs: 129 active+clean; 645 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T09:54:17.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:17.155+0000 7fa127200640 1 -- 172.21.15.77:0/3475099945 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa1281019e0 msgr2=0x7fa128101e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:17.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:17.155+0000 7fa127200640 1 --2- 172.21.15.77:0/3475099945 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa1281019e0 0x7fa128101e40 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa110009930 tx=0x7fa11002a420 comp rx=0 tx=0).stop 2024-09-10T09:54:17.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:17.155+0000 7fa127200640 1 -- 172.21.15.77:0/3475099945 shutdown_connections 2024-09-10T09:54:17.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:17.155+0000 7fa127200640 1 --2- 172.21.15.77:0/3475099945 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa128102380 0x7fa12810e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:17.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:17.155+0000 7fa127200640 1 --2- 172.21.15.77:0/3475099945 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa1281019e0 0x7fa128101e40 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:17.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:17.155+0000 7fa127200640 1 --2- 172.21.15.77:0/3475099945 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa128107af0 0x7fa128107ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:17.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:17.155+0000 7fa127200640 1 -- 172.21.15.77:0/3475099945 >> 172.21.15.77:0/3475099945 conn(0x7fa1280fd700 msgr2=0x7fa1280ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:17.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:17.156+0000 7fa127200640 1 -- 172.21.15.77:0/3475099945 shutdown_connections 2024-09-10T09:54:17.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:17.156+0000 7fa127200640 1 -- 172.21.15.77:0/3475099945 wait complete. 2024-09-10T09:54:17.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:17.156+0000 7fa127200640 1 Processor -- start 2024-09-10T09:54:17.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:17.157+0000 7fa127200640 1 -- start start 2024-09-10T09:54:17.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:17.157+0000 7fa127200640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa1281019e0 0x7fa1280744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:17.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:17.157+0000 7fa127200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa128102380 0x7fa128074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:17.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:17.157+0000 7fa127200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa128107af0 0x7fa128074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:17.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:17.157+0000 7fa127200640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa128111f40 con 0x7fa128102380 2024-09-10T09:54:17.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:17.157+0000 7fa127200640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa128111dc0 con 0x7fa1281019e0 2024-09-10T09:54:17.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:17.157+0000 7fa127200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa1281120c0 con 0x7fa128107af0 2024-09-10T09:54:17.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:17.157+0000 7fa126800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa128107af0 0x7fa128074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:17.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:17.157+0000 7fa11fe00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa128102380 0x7fa128074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:17.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:17.157+0000 7fa125e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa1281019e0 0x7fa1280744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:17.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:17.157+0000 7fa126800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa128107af0 0x7fa128074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:51942/0 (socket says 172.21.15.77:51942) 2024-09-10T09:54:17.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:17.157+0000 7fa126800640 1 -- 172.21.15.77:0/3343255527 learned_addr learned my addr 172.21.15.77:0/3343255527 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:54:17.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:17.158+0000 7fa126800640 1 -- 172.21.15.77:0/3343255527 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa1281019e0 msgr2=0x7fa1280744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:17.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:17.158+0000 7fa126800640 1 --2- 172.21.15.77:0/3343255527 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa1281019e0 0x7fa1280744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:17.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:17.158+0000 7fa126800640 1 -- 172.21.15.77:0/3343255527 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa128102380 msgr2=0x7fa128074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:17.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:17.158+0000 7fa126800640 1 --2- 172.21.15.77:0/3343255527 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa128102380 0x7fa128074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:17.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:17.158+0000 7fa126800640 1 -- 172.21.15.77:0/3343255527 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1281a68c0 con 0x7fa128107af0 2024-09-10T09:54:17.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:17.158+0000 7fa125e00640 1 --2- 172.21.15.77:0/3343255527 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa1281019e0 0x7fa1280744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:54:17.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:17.158+0000 7fa11fe00640 1 --2- 172.21.15.77:0/3343255527 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa128102380 0x7fa128074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:54:17.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:17.158+0000 7fa126800640 1 --2- 172.21.15.77:0/3343255527 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa128107af0 0x7fa128074f70 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa12000eb10 tx=0x7fa12000c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:17.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:17.159+0000 7fa11ea00640 1 -- 172.21.15.77:0/3343255527 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa120004b20 con 0x7fa128107af0 2024-09-10T09:54:17.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:17.159+0000 7fa11ea00640 1 -- 172.21.15.77:0/3343255527 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa120004cc0 con 0x7fa128107af0 2024-09-10T09:54:17.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:17.159+0000 7fa11ea00640 1 -- 172.21.15.77:0/3343255527 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa120010760 con 0x7fa128107af0 2024-09-10T09:54:17.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:17.159+0000 7fa127200640 1 -- 172.21.15.77:0/3343255527 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1281a6bb0 con 0x7fa128107af0 2024-09-10T09:54:17.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:17.159+0000 7fa127200640 1 -- 172.21.15.77:0/3343255527 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1281a7070 con 0x7fa128107af0 2024-09-10T09:54:17.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:17.160+0000 7fa127200640 1 -- 172.21.15.77:0/3343255527 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa128101e40 con 0x7fa128107af0 2024-09-10T09:54:17.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:17.162+0000 7fa11ea00640 1 -- 172.21.15.77:0/3343255527 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fa120010900 con 0x7fa128107af0 2024-09-10T09:54:17.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:17.163+0000 7fa11ea00640 1 --2- 172.21.15.77:0/3343255527 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fa10c0a2d60 0x7fa10c0a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:17.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:17.164+0000 7fa11ea00640 1 -- 172.21.15.77:0/3343255527 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fa1200cffe0 con 0x7fa128107af0 2024-09-10T09:54:17.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:17.164+0000 7fa125e00640 1 --2- 172.21.15.77:0/3343255527 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fa10c0a2d60 0x7fa10c0a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:17.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:17.164+0000 7fa125e00640 1 --2- 172.21.15.77:0/3343255527 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fa10c0a2d60 0x7fa10c0a5220 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa118009f40 tx=0x7fa118005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:17.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:17.166+0000 7fa11ea00640 1 -- 172.21.15.77:0/3343255527 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa12009ca30 con 0x7fa128107af0 2024-09-10T09:54:17.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:54:16 smithi120 ceph-mon[34770]: pgmap v364: 129 pgs: 129 active+clean; 645 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T09:54:17.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:17.338+0000 7fa127200640 1 -- 172.21.15.77:0/3343255527 --> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa1280fe590 con 0x7fa10c0a2d60 2024-09-10T09:54:17.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:17.346+0000 7fa11ea00640 1 -- 172.21.15.77:0/3343255527 <== mgr.14150 v2:172.21.15.77:6800/4261324650 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340732 (secure 0 0 0) 0x7fa1280fe590 con 0x7fa10c0a2d60 2024-09-10T09:54:17.353 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:54:17.355 INFO:teuthology.orchestra.run.smithi077.stderr:dumped all 2024-09-10T09:54:17.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:17.351+0000 7fa127200640 1 -- 172.21.15.77:0/3343255527 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fa10c0a2d60 msgr2=0x7fa10c0a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:17.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:17.351+0000 7fa127200640 1 --2- 172.21.15.77:0/3343255527 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fa10c0a2d60 0x7fa10c0a5220 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa118009f40 tx=0x7fa118005f90 comp rx=0 tx=0).stop 2024-09-10T09:54:17.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:17.351+0000 7fa127200640 1 -- 172.21.15.77:0/3343255527 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa128107af0 msgr2=0x7fa128074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:17.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:17.351+0000 7fa127200640 1 --2- 172.21.15.77:0/3343255527 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa128107af0 0x7fa128074f70 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa12000eb10 tx=0x7fa12000c2d0 comp rx=0 tx=0).stop 2024-09-10T09:54:17.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:17.351+0000 7fa127200640 1 -- 172.21.15.77:0/3343255527 shutdown_connections 2024-09-10T09:54:17.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:17.351+0000 7fa127200640 1 --2- 172.21.15.77:0/3343255527 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fa10c0a2d60 0x7fa10c0a5220 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:17.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:17.351+0000 7fa127200640 1 --2- 172.21.15.77:0/3343255527 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa128107af0 0x7fa128074f70 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:17.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:17.351+0000 7fa127200640 1 --2- 172.21.15.77:0/3343255527 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa128102380 0x7fa128074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:17.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:17.351+0000 7fa127200640 1 --2- 172.21.15.77:0/3343255527 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa1281019e0 0x7fa1280744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:17.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:17.351+0000 7fa127200640 1 -- 172.21.15.77:0/3343255527 >> 172.21.15.77:0/3343255527 conn(0x7fa1280fd700 msgr2=0x7fa1280fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:17.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:17.352+0000 7fa127200640 1 -- 172.21.15.77:0/3343255527 shutdown_connections 2024-09-10T09:54:17.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:17.352+0000 7fa127200640 1 -- 172.21.15.77:0/3343255527 wait complete. 2024-09-10T09:54:17.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:54:16 smithi134 ceph-mon[35674]: pgmap v364: 129 pgs: 129 active+clean; 645 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T09:54:18.142 INFO:teuthology.orchestra.run.smithi077.stdout:{"pg_ready":true,"pg_map":{"version":364,"stamp":"2024-09-10T09:54:15.700945+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660812,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":729,"num_read_kb":486,"num_write":428,"num_write_kb":3080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1185162,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":351,"ondisk_log_size":351,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":366552,"kb_used_data":13064,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124575272,"statfs":{"total":1151940427776,"available":1151565078528,"internally_reserved":0,"allocated":13377536,"data_stored":8598686,"data_compressed":504633,"data_compressed_allocated":688128,"data_compressed_original":2875392,"omap_allocated":19143,"internal_metadata":361674041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":115,"num_read_kb":55,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003378"},"pg_stats":[{"pgid":"2.1f","version":"94'13","reported_seq":121,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:09.966830+0000","last_change":"2024-09-10T09:53:00.803590+0000","last_active":"2024-09-10T09:54:09.966830+0000","last_peered":"2024-09-10T09:54:09.966830+0000","last_clean":"2024-09-10T09:54:09.966830+0000","last_became_active":"2024-09-10T09:53:00.798189+0000","last_became_peered":"2024-09-10T09:53:00.798189+0000","last_unstale":"2024-09-10T09:54:09.966830+0000","last_undegraded":"2024-09-10T09:54:09.966830+0000","last_fullsized":"2024-09-10T09:54:09.966830+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:18:48.808836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":56,"num_read_kb":56,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.846944+0000","last_change":"2024-09-10T09:52:22.318195+0000","last_active":"2024-09-10T09:54:00.846944+0000","last_peered":"2024-09-10T09:54:00.846944+0000","last_clean":"2024-09-10T09:54:00.846944+0000","last_became_active":"2024-09-10T09:52:22.315905+0000","last_became_peered":"2024-09-10T09:52:22.315905+0000","last_unstale":"2024-09-10T09:54:00.846944+0000","last_undegraded":"2024-09-10T09:54:00.846944+0000","last_fullsized":"2024-09-10T09:54:00.846944+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:53:55.515984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.843917+0000","last_change":"2024-09-10T09:52:31.369780+0000","last_active":"2024-09-10T09:54:00.843917+0000","last_peered":"2024-09-10T09:54:00.843917+0000","last_clean":"2024-09-10T09:54:00.843917+0000","last_became_active":"2024-09-10T09:52:31.369238+0000","last_became_peered":"2024-09-10T09:52:31.369238+0000","last_unstale":"2024-09-10T09:54:00.843917+0000","last_undegraded":"2024-09-10T09:54:00.843917+0000","last_fullsized":"2024-09-10T09:54:00.843917+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:39:05.318493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"94'2","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.847155+0000","last_change":"2024-09-10T09:52:19.912276+0000","last_active":"2024-09-10T09:54:00.847155+0000","last_peered":"2024-09-10T09:54:00.847155+0000","last_clean":"2024-09-10T09:54:00.847155+0000","last_became_active":"2024-09-10T09:52:19.911452+0000","last_became_peered":"2024-09-10T09:52:19.911452+0000","last_unstale":"2024-09-10T09:54:00.847155+0000","last_undegraded":"2024-09-10T09:54:00.847155+0000","last_fullsized":"2024-09-10T09:54:00.847155+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:24:31.262207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.248210+0000","last_change":"2024-09-10T09:52:22.327641+0000","last_active":"2024-09-10T09:54:01.248210+0000","last_peered":"2024-09-10T09:54:01.248210+0000","last_clean":"2024-09-10T09:54:01.248210+0000","last_became_active":"2024-09-10T09:52:22.326969+0000","last_became_peered":"2024-09-10T09:52:22.326969+0000","last_unstale":"2024-09-10T09:54:01.248210+0000","last_undegraded":"2024-09-10T09:54:01.248210+0000","last_fullsized":"2024-09-10T09:54:01.248210+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:35:27.825534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.950921+0000","last_change":"2024-09-10T09:52:31.372360+0000","last_active":"2024-09-10T09:54:00.950921+0000","last_peered":"2024-09-10T09:54:00.950921+0000","last_clean":"2024-09-10T09:54:00.950921+0000","last_became_active":"2024-09-10T09:52:31.371957+0000","last_became_peered":"2024-09-10T09:52:31.371957+0000","last_unstale":"2024-09-10T09:54:00.950921+0000","last_undegraded":"2024-09-10T09:54:00.950921+0000","last_fullsized":"2024-09-10T09:54:00.950921+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:49:34.776185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"94'12","reported_seq":53,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.149941+0000","last_change":"2024-09-10T09:52:19.843532+0000","last_active":"2024-09-10T09:54:01.149941+0000","last_peered":"2024-09-10T09:54:01.149941+0000","last_clean":"2024-09-10T09:54:01.149941+0000","last_became_active":"2024-09-10T09:52:19.843070+0000","last_became_peered":"2024-09-10T09:52:19.843070+0000","last_unstale":"2024-09-10T09:54:01.149941+0000","last_undegraded":"2024-09-10T09:54:01.149941+0000","last_fullsized":"2024-09-10T09:54:01.149941+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:08:02.223373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.988750+0000","last_change":"2024-09-10T09:52:22.316981+0000","last_active":"2024-09-10T09:54:00.988750+0000","last_peered":"2024-09-10T09:54:00.988750+0000","last_clean":"2024-09-10T09:54:00.988750+0000","last_became_active":"2024-09-10T09:52:22.316648+0000","last_became_peered":"2024-09-10T09:52:22.316648+0000","last_unstale":"2024-09-10T09:54:00.988750+0000","last_undegraded":"2024-09-10T09:54:00.988750+0000","last_fullsized":"2024-09-10T09:54:00.988750+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:28:56.742496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.951407+0000","last_change":"2024-09-10T09:52:31.376811+0000","last_active":"2024-09-10T09:54:00.951407+0000","last_peered":"2024-09-10T09:54:00.951407+0000","last_clean":"2024-09-10T09:54:00.951407+0000","last_became_active":"2024-09-10T09:52:31.376007+0000","last_became_peered":"2024-09-10T09:52:31.376007+0000","last_unstale":"2024-09-10T09:54:00.951407+0000","last_undegraded":"2024-09-10T09:54:00.951407+0000","last_fullsized":"2024-09-10T09:54:00.951407+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:29:26.213281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"94'5","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.149714+0000","last_change":"2024-09-10T09:52:19.837848+0000","last_active":"2024-09-10T09:54:01.149714+0000","last_peered":"2024-09-10T09:54:01.149714+0000","last_clean":"2024-09-10T09:54:01.149714+0000","last_became_active":"2024-09-10T09:52:19.837383+0000","last_became_peered":"2024-09-10T09:52:19.837383+0000","last_unstale":"2024-09-10T09:54:01.149714+0000","last_undegraded":"2024-09-10T09:54:01.149714+0000","last_fullsized":"2024-09-10T09:54:01.149714+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:50:42.813665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.060407+0000","last_change":"2024-09-10T09:54:01.060407+0000","last_active":"2024-09-10T09:54:01.060407+0000","last_peered":"2024-09-10T09:54:01.060407+0000","last_clean":"2024-09-10T09:54:01.060407+0000","last_became_active":"2024-09-10T09:54:01.060126+0000","last_became_peered":"2024-09-10T09:54:01.060126+0000","last_unstale":"2024-09-10T09:54:01.060407+0000","last_undegraded":"2024-09-10T09:54:01.060407+0000","last_fullsized":"2024-09-10T09:54:01.060407+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:59:29.765375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.951061+0000","last_change":"2024-09-10T09:52:31.370910+0000","last_active":"2024-09-10T09:54:00.951061+0000","last_peered":"2024-09-10T09:54:00.951061+0000","last_clean":"2024-09-10T09:54:00.951061+0000","last_became_active":"2024-09-10T09:52:31.370467+0000","last_became_peered":"2024-09-10T09:52:31.370467+0000","last_unstale":"2024-09-10T09:54:00.951061+0000","last_undegraded":"2024-09-10T09:54:00.951061+0000","last_fullsized":"2024-09-10T09:54:00.951061+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:49:25.037820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.059228+0000","last_change":"2024-09-10T09:52:22.310708+0000","last_active":"2024-09-10T09:54:01.059228+0000","last_peered":"2024-09-10T09:54:01.059228+0000","last_clean":"2024-09-10T09:54:01.059228+0000","last_became_active":"2024-09-10T09:52:22.310387+0000","last_became_peered":"2024-09-10T09:52:22.310387+0000","last_unstale":"2024-09-10T09:54:01.059228+0000","last_undegraded":"2024-09-10T09:54:01.059228+0000","last_fullsized":"2024-09-10T09:54:01.059228+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:35:03.115408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"94'4","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.149376+0000","last_change":"2024-09-10T09:52:19.841417+0000","last_active":"2024-09-10T09:54:01.149376+0000","last_peered":"2024-09-10T09:54:01.149376+0000","last_clean":"2024-09-10T09:54:01.149376+0000","last_became_active":"2024-09-10T09:52:19.841083+0000","last_became_peered":"2024-09-10T09:52:19.841083+0000","last_unstale":"2024-09-10T09:54:01.149376+0000","last_undegraded":"2024-09-10T09:54:01.149376+0000","last_fullsized":"2024-09-10T09:54:01.149376+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:01:19.008345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.951244+0000","last_change":"2024-09-10T09:52:31.367406+0000","last_active":"2024-09-10T09:54:00.951244+0000","last_peered":"2024-09-10T09:54:00.951244+0000","last_clean":"2024-09-10T09:54:00.951244+0000","last_became_active":"2024-09-10T09:52:31.366961+0000","last_became_peered":"2024-09-10T09:52:31.366961+0000","last_unstale":"2024-09-10T09:54:00.951244+0000","last_undegraded":"2024-09-10T09:54:00.951244+0000","last_fullsized":"2024-09-10T09:54:00.951244+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:04:51.714465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.249212+0000","last_change":"2024-09-10T09:54:01.249212+0000","last_active":"2024-09-10T09:54:01.249212+0000","last_peered":"2024-09-10T09:54:01.249212+0000","last_clean":"2024-09-10T09:54:01.249212+0000","last_became_active":"2024-09-10T09:54:01.248840+0000","last_became_peered":"2024-09-10T09:54:01.248840+0000","last_unstale":"2024-09-10T09:54:01.249212+0000","last_undegraded":"2024-09-10T09:54:01.249212+0000","last_fullsized":"2024-09-10T09:54:01.249212+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:39:13.167435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,10],"acting":[9,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.a","version":"94'3","reported_seq":18,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.954891+0000","last_change":"2024-09-10T09:53:00.800421+0000","last_active":"2024-09-10T09:54:00.954891+0000","last_peered":"2024-09-10T09:54:00.954891+0000","last_clean":"2024-09-10T09:54:00.954891+0000","last_became_active":"2024-09-10T09:53:00.800026+0000","last_became_peered":"2024-09-10T09:53:00.800026+0000","last_unstale":"2024-09-10T09:54:00.954891+0000","last_undegraded":"2024-09-10T09:54:00.954891+0000","last_fullsized":"2024-09-10T09:54:00.954891+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:57:45.252420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"94'1","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.951331+0000","last_change":"2024-09-10T09:52:31.376605+0000","last_active":"2024-09-10T09:54:00.951331+0000","last_peered":"2024-09-10T09:54:00.951331+0000","last_clean":"2024-09-10T09:54:00.951331+0000","last_became_active":"2024-09-10T09:52:31.376083+0000","last_became_peered":"2024-09-10T09:52:31.376083+0000","last_unstale":"2024-09-10T09:54:00.951331+0000","last_undegraded":"2024-09-10T09:54:00.951331+0000","last_fullsized":"2024-09-10T09:54:00.951331+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:05:10.030673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.954956+0000","last_change":"2024-09-10T09:52:22.327303+0000","last_active":"2024-09-10T09:54:00.954956+0000","last_peered":"2024-09-10T09:54:00.954956+0000","last_clean":"2024-09-10T09:54:00.954956+0000","last_became_active":"2024-09-10T09:52:22.326753+0000","last_became_peered":"2024-09-10T09:52:22.326753+0000","last_unstale":"2024-09-10T09:54:00.954956+0000","last_undegraded":"2024-09-10T09:54:00.954956+0000","last_fullsized":"2024-09-10T09:54:00.954956+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:27:26.668151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"94'7","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.845909+0000","last_change":"2024-09-10T09:52:19.836484+0000","last_active":"2024-09-10T09:54:00.845909+0000","last_peered":"2024-09-10T09:54:00.845909+0000","last_clean":"2024-09-10T09:54:00.845909+0000","last_became_active":"2024-09-10T09:52:19.836130+0000","last_became_peered":"2024-09-10T09:52:19.836130+0000","last_unstale":"2024-09-10T09:54:00.845909+0000","last_undegraded":"2024-09-10T09:54:00.845909+0000","last_fullsized":"2024-09-10T09:54:00.845909+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:31:58.731981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.844246+0000","last_change":"2024-09-10T09:52:31.375279+0000","last_active":"2024-09-10T09:54:00.844246+0000","last_peered":"2024-09-10T09:54:00.844246+0000","last_clean":"2024-09-10T09:54:00.844246+0000","last_became_active":"2024-09-10T09:52:31.374181+0000","last_became_peered":"2024-09-10T09:52:31.374181+0000","last_unstale":"2024-09-10T09:54:00.844246+0000","last_undegraded":"2024-09-10T09:54:00.844246+0000","last_fullsized":"2024-09-10T09:54:00.844246+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:54:51.299224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.149225+0000","last_change":"2024-09-10T09:52:22.314817+0000","last_active":"2024-09-10T09:54:01.149225+0000","last_peered":"2024-09-10T09:54:01.149225+0000","last_clean":"2024-09-10T09:54:01.149225+0000","last_became_active":"2024-09-10T09:52:22.314491+0000","last_became_peered":"2024-09-10T09:52:22.314491+0000","last_unstale":"2024-09-10T09:54:01.149225+0000","last_undegraded":"2024-09-10T09:54:01.149225+0000","last_fullsized":"2024-09-10T09:54:01.149225+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:49:41.941667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"94'2","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.988492+0000","last_change":"2024-09-10T09:52:19.911730+0000","last_active":"2024-09-10T09:54:00.988492+0000","last_peered":"2024-09-10T09:54:00.988492+0000","last_clean":"2024-09-10T09:54:00.988492+0000","last_became_active":"2024-09-10T09:52:19.911393+0000","last_became_peered":"2024-09-10T09:52:19.911393+0000","last_unstale":"2024-09-10T09:54:00.988492+0000","last_undegraded":"2024-09-10T09:54:00.988492+0000","last_fullsized":"2024-09-10T09:54:00.988492+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:46:17.943129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.951225+0000","last_change":"2024-09-10T09:52:31.376498+0000","last_active":"2024-09-10T09:54:00.951225+0000","last_peered":"2024-09-10T09:54:00.951225+0000","last_clean":"2024-09-10T09:54:00.951225+0000","last_became_active":"2024-09-10T09:52:31.375966+0000","last_became_peered":"2024-09-10T09:52:31.375966+0000","last_unstale":"2024-09-10T09:54:00.951225+0000","last_undegraded":"2024-09-10T09:54:00.951225+0000","last_fullsized":"2024-09-10T09:54:00.951225+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:08:30.649399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.248013+0000","last_change":"2024-09-10T09:52:22.331611+0000","last_active":"2024-09-10T09:54:01.248013+0000","last_peered":"2024-09-10T09:54:01.248013+0000","last_clean":"2024-09-10T09:54:01.248013+0000","last_became_active":"2024-09-10T09:52:22.331268+0000","last_became_peered":"2024-09-10T09:52:22.331268+0000","last_unstale":"2024-09-10T09:54:01.248013+0000","last_undegraded":"2024-09-10T09:54:01.248013+0000","last_fullsized":"2024-09-10T09:54:01.248013+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:01:05.433994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"94'7","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.951878+0000","last_change":"2024-09-10T09:52:19.838387+0000","last_active":"2024-09-10T09:54:00.951878+0000","last_peered":"2024-09-10T09:54:00.951878+0000","last_clean":"2024-09-10T09:54:00.951878+0000","last_became_active":"2024-09-10T09:52:19.837889+0000","last_became_peered":"2024-09-10T09:52:19.837889+0000","last_unstale":"2024-09-10T09:54:00.951878+0000","last_undegraded":"2024-09-10T09:54:00.951878+0000","last_fullsized":"2024-09-10T09:54:00.951878+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:50:00.187243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.844136+0000","last_change":"2024-09-10T09:52:31.363262+0000","last_active":"2024-09-10T09:54:00.844136+0000","last_peered":"2024-09-10T09:54:00.844136+0000","last_clean":"2024-09-10T09:54:00.844136+0000","last_became_active":"2024-09-10T09:52:31.362762+0000","last_became_peered":"2024-09-10T09:52:31.362762+0000","last_unstale":"2024-09-10T09:54:00.844136+0000","last_undegraded":"2024-09-10T09:54:00.844136+0000","last_fullsized":"2024-09-10T09:54:00.844136+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:48:28.275041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.847229+0000","last_change":"2024-09-10T09:52:22.312472+0000","last_active":"2024-09-10T09:54:00.847229+0000","last_peered":"2024-09-10T09:54:00.847229+0000","last_clean":"2024-09-10T09:54:00.847229+0000","last_became_active":"2024-09-10T09:52:22.312117+0000","last_became_peered":"2024-09-10T09:52:22.312117+0000","last_unstale":"2024-09-10T09:54:00.847229+0000","last_undegraded":"2024-09-10T09:54:00.847229+0000","last_fullsized":"2024-09-10T09:54:00.847229+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:39:10.742137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"94'14","reported_seq":54,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.844383+0000","last_change":"2024-09-10T09:52:19.835543+0000","last_active":"2024-09-10T09:54:00.844383+0000","last_peered":"2024-09-10T09:54:00.844383+0000","last_clean":"2024-09-10T09:54:00.844383+0000","last_became_active":"2024-09-10T09:52:19.835170+0000","last_became_peered":"2024-09-10T09:52:19.835170+0000","last_unstale":"2024-09-10T09:54:00.844383+0000","last_undegraded":"2024-09-10T09:54:00.844383+0000","last_fullsized":"2024-09-10T09:54:00.844383+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:37.969277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.845397+0000","last_change":"2024-09-10T09:52:31.372398+0000","last_active":"2024-09-10T09:54:00.845397+0000","last_peered":"2024-09-10T09:54:00.845397+0000","last_clean":"2024-09-10T09:54:00.845397+0000","last_became_active":"2024-09-10T09:52:31.371874+0000","last_became_peered":"2024-09-10T09:52:31.371874+0000","last_unstale":"2024-09-10T09:54:00.845397+0000","last_undegraded":"2024-09-10T09:54:00.845397+0000","last_fullsized":"2024-09-10T09:54:00.845397+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:09:45.374320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.841950+0000","last_change":"2024-09-10T09:52:22.319592+0000","last_active":"2024-09-10T09:54:00.841950+0000","last_peered":"2024-09-10T09:54:00.841950+0000","last_clean":"2024-09-10T09:54:00.841950+0000","last_became_active":"2024-09-10T09:52:22.319298+0000","last_became_peered":"2024-09-10T09:52:22.319298+0000","last_unstale":"2024-09-10T09:54:00.841950+0000","last_undegraded":"2024-09-10T09:54:00.841950+0000","last_fullsized":"2024-09-10T09:54:00.841950+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:56:59.787638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"94'4","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.149952+0000","last_change":"2024-09-10T09:52:19.912923+0000","last_active":"2024-09-10T09:54:01.149952+0000","last_peered":"2024-09-10T09:54:01.149952+0000","last_clean":"2024-09-10T09:54:01.149952+0000","last_became_active":"2024-09-10T09:52:19.912268+0000","last_became_peered":"2024-09-10T09:52:19.912268+0000","last_unstale":"2024-09-10T09:54:01.149952+0000","last_undegraded":"2024-09-10T09:54:01.149952+0000","last_fullsized":"2024-09-10T09:54:01.149952+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:06:45.421004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.955008+0000","last_change":"2024-09-10T09:52:31.372761+0000","last_active":"2024-09-10T09:54:00.955008+0000","last_peered":"2024-09-10T09:54:00.955008+0000","last_clean":"2024-09-10T09:54:00.955008+0000","last_became_active":"2024-09-10T09:52:31.372293+0000","last_became_peered":"2024-09-10T09:52:31.372293+0000","last_unstale":"2024-09-10T09:54:00.955008+0000","last_undegraded":"2024-09-10T09:54:00.955008+0000","last_fullsized":"2024-09-10T09:54:00.955008+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:36:41.446810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.841648+0000","last_change":"2024-09-10T09:52:22.326291+0000","last_active":"2024-09-10T09:54:00.841648+0000","last_peered":"2024-09-10T09:54:00.841648+0000","last_clean":"2024-09-10T09:54:00.841648+0000","last_became_active":"2024-09-10T09:52:22.325985+0000","last_became_peered":"2024-09-10T09:52:22.325985+0000","last_unstale":"2024-09-10T09:54:00.841648+0000","last_undegraded":"2024-09-10T09:54:00.841648+0000","last_fullsized":"2024-09-10T09:54:00.841648+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:06:21.871502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"94'1","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.844016+0000","last_change":"2024-09-10T09:52:19.912851+0000","last_active":"2024-09-10T09:54:00.844016+0000","last_peered":"2024-09-10T09:54:00.844016+0000","last_clean":"2024-09-10T09:54:00.844016+0000","last_became_active":"2024-09-10T09:52:19.912200+0000","last_became_peered":"2024-09-10T09:52:19.912200+0000","last_unstale":"2024-09-10T09:54:00.844016+0000","last_undegraded":"2024-09-10T09:54:00.844016+0000","last_fullsized":"2024-09-10T09:54:00.844016+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:00:47.331755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.149237+0000","last_change":"2024-09-10T09:52:31.371329+0000","last_active":"2024-09-10T09:54:01.149237+0000","last_peered":"2024-09-10T09:54:01.149237+0000","last_clean":"2024-09-10T09:54:01.149237+0000","last_became_active":"2024-09-10T09:52:31.370971+0000","last_became_peered":"2024-09-10T09:52:31.370971+0000","last_unstale":"2024-09-10T09:54:01.149237+0000","last_undegraded":"2024-09-10T09:54:01.149237+0000","last_fullsized":"2024-09-10T09:54:01.149237+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:07:12.928421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.841648+0000","last_change":"2024-09-10T09:52:22.328039+0000","last_active":"2024-09-10T09:54:00.841648+0000","last_peered":"2024-09-10T09:54:00.841648+0000","last_clean":"2024-09-10T09:54:00.841648+0000","last_became_active":"2024-09-10T09:52:22.327588+0000","last_became_peered":"2024-09-10T09:52:22.327588+0000","last_unstale":"2024-09-10T09:54:00.841648+0000","last_undegraded":"2024-09-10T09:54:00.841648+0000","last_fullsized":"2024-09-10T09:54:00.841648+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:35:39.219544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"94'5","reported_seq":77,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:10.968454+0000","last_change":"2024-09-10T09:53:00.797651+0000","last_active":"2024-09-10T09:54:10.968454+0000","last_peered":"2024-09-10T09:54:10.968454+0000","last_clean":"2024-09-10T09:54:10.968454+0000","last_became_active":"2024-09-10T09:53:00.797285+0000","last_became_peered":"2024-09-10T09:53:00.797285+0000","last_unstale":"2024-09-10T09:54:10.968454+0000","last_undegraded":"2024-09-10T09:54:10.968454+0000","last_fullsized":"2024-09-10T09:54:10.968454+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:18:02.320032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":57,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,6,1],"acting":[11,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.841739+0000","last_change":"2024-09-10T09:52:31.379215+0000","last_active":"2024-09-10T09:54:00.841739+0000","last_peered":"2024-09-10T09:54:00.841739+0000","last_clean":"2024-09-10T09:54:00.841739+0000","last_became_active":"2024-09-10T09:52:31.377107+0000","last_became_peered":"2024-09-10T09:52:31.377107+0000","last_unstale":"2024-09-10T09:54:00.841739+0000","last_undegraded":"2024-09-10T09:54:00.841739+0000","last_fullsized":"2024-09-10T09:54:00.841739+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:09:39.474077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.847190+0000","last_change":"2024-09-10T09:52:22.316474+0000","last_active":"2024-09-10T09:54:00.847190+0000","last_peered":"2024-09-10T09:54:00.847190+0000","last_clean":"2024-09-10T09:54:00.847190+0000","last_became_active":"2024-09-10T09:52:22.316054+0000","last_became_peered":"2024-09-10T09:52:22.316054+0000","last_unstale":"2024-09-10T09:54:00.847190+0000","last_undegraded":"2024-09-10T09:54:00.847190+0000","last_fullsized":"2024-09-10T09:54:00.847190+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:11:31.334780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"94'5","reported_seq":104,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:11.969631+0000","last_change":"2024-09-10T09:52:19.911601+0000","last_active":"2024-09-10T09:54:11.969631+0000","last_peered":"2024-09-10T09:54:11.969631+0000","last_clean":"2024-09-10T09:54:11.969631+0000","last_became_active":"2024-09-10T09:52:19.911268+0000","last_became_peered":"2024-09-10T09:52:19.911268+0000","last_unstale":"2024-09-10T09:54:11.969631+0000","last_undegraded":"2024-09-10T09:54:11.969631+0000","last_fullsized":"2024-09-10T09:54:11.969631+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:46:38.158784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":58,"num_read_kb":58,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.841751+0000","last_change":"2024-09-10T09:52:31.375938+0000","last_active":"2024-09-10T09:54:00.841751+0000","last_peered":"2024-09-10T09:54:00.841751+0000","last_clean":"2024-09-10T09:54:00.841751+0000","last_became_active":"2024-09-10T09:52:31.375536+0000","last_became_peered":"2024-09-10T09:52:31.375536+0000","last_unstale":"2024-09-10T09:54:00.841751+0000","last_undegraded":"2024-09-10T09:54:00.841751+0000","last_fullsized":"2024-09-10T09:54:00.841751+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:55:17.303809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.840977+0000","last_change":"2024-09-10T09:52:22.315912+0000","last_active":"2024-09-10T09:54:00.840977+0000","last_peered":"2024-09-10T09:54:00.840977+0000","last_clean":"2024-09-10T09:54:00.840977+0000","last_became_active":"2024-09-10T09:52:22.315433+0000","last_became_peered":"2024-09-10T09:52:22.315433+0000","last_unstale":"2024-09-10T09:54:00.840977+0000","last_undegraded":"2024-09-10T09:54:00.840977+0000","last_fullsized":"2024-09-10T09:54:00.840977+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:57:10.454716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"94'4","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.059073+0000","last_change":"2024-09-10T09:52:19.838966+0000","last_active":"2024-09-10T09:54:01.059073+0000","last_peered":"2024-09-10T09:54:01.059073+0000","last_clean":"2024-09-10T09:54:01.059073+0000","last_became_active":"2024-09-10T09:52:19.838545+0000","last_became_peered":"2024-09-10T09:52:19.838545+0000","last_unstale":"2024-09-10T09:54:01.059073+0000","last_undegraded":"2024-09-10T09:54:01.059073+0000","last_fullsized":"2024-09-10T09:54:01.059073+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:07:00.930146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.847547+0000","last_change":"2024-09-10T09:53:00.802039+0000","last_active":"2024-09-10T09:54:00.847547+0000","last_peered":"2024-09-10T09:54:00.847547+0000","last_clean":"2024-09-10T09:54:00.847547+0000","last_became_active":"2024-09-10T09:53:00.800223+0000","last_became_peered":"2024-09-10T09:53:00.800223+0000","last_unstale":"2024-09-10T09:54:00.847547+0000","last_undegraded":"2024-09-10T09:54:00.847547+0000","last_fullsized":"2024-09-10T09:54:00.847547+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:50:16.026631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,6,10],"acting":[5,8,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.846149+0000","last_change":"2024-09-10T09:52:22.313819+0000","last_active":"2024-09-10T09:54:00.846149+0000","last_peered":"2024-09-10T09:54:00.846149+0000","last_clean":"2024-09-10T09:54:00.846149+0000","last_became_active":"2024-09-10T09:52:22.313473+0000","last_became_peered":"2024-09-10T09:52:22.313473+0000","last_unstale":"2024-09-10T09:54:00.846149+0000","last_undegraded":"2024-09-10T09:54:00.846149+0000","last_fullsized":"2024-09-10T09:54:00.846149+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:44:10.003187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"94'6","reported_seq":50,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.951708+0000","last_change":"2024-09-10T09:52:19.837376+0000","last_active":"2024-09-10T09:54:00.951708+0000","last_peered":"2024-09-10T09:54:00.951708+0000","last_clean":"2024-09-10T09:54:00.951708+0000","last_became_active":"2024-09-10T09:52:19.836880+0000","last_became_peered":"2024-09-10T09:52:19.836880+0000","last_unstale":"2024-09-10T09:54:00.951708+0000","last_undegraded":"2024-09-10T09:54:00.951708+0000","last_fullsized":"2024-09-10T09:54:00.951708+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:48:58.198188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":361,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.059392+0000","last_change":"2024-09-10T09:51:05.871529+0000","last_active":"2024-09-10T09:54:01.059392+0000","last_peered":"2024-09-10T09:54:01.059392+0000","last_clean":"2024-09-10T09:54:01.059392+0000","last_became_active":"2024-09-10T09:51:05.871115+0000","last_became_peered":"2024-09-10T09:51:05.871115+0000","last_unstale":"2024-09-10T09:54:01.059392+0000","last_undegraded":"2024-09-10T09:54:01.059392+0000","last_fullsized":"2024-09-10T09:54:01.059392+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:47:24.175358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:47:24.175358+0000","last_clean_scrub_stamp":"2024-09-10T09:47:24.175358+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:58:30.247077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"94'1","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.059487+0000","last_change":"2024-09-10T09:53:00.800474+0000","last_active":"2024-09-10T09:54:01.059487+0000","last_peered":"2024-09-10T09:54:01.059487+0000","last_clean":"2024-09-10T09:54:01.059487+0000","last_became_active":"2024-09-10T09:53:00.799932+0000","last_became_peered":"2024-09-10T09:53:00.799932+0000","last_unstale":"2024-09-10T09:54:01.059487+0000","last_undegraded":"2024-09-10T09:54:01.059487+0000","last_fullsized":"2024-09-10T09:54:01.059487+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:14:50.998952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,11,9],"acting":[7,8,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.059603+0000","last_change":"2024-09-10T09:52:22.312521+0000","last_active":"2024-09-10T09:54:01.059603+0000","last_peered":"2024-09-10T09:54:01.059603+0000","last_clean":"2024-09-10T09:54:01.059603+0000","last_became_active":"2024-09-10T09:52:22.312107+0000","last_became_peered":"2024-09-10T09:52:22.312107+0000","last_unstale":"2024-09-10T09:54:01.059603+0000","last_undegraded":"2024-09-10T09:54:01.059603+0000","last_fullsized":"2024-09-10T09:54:01.059603+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:14:19.799259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"94'5","reported_seq":105,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:11.965971+0000","last_change":"2024-09-10T09:52:19.911454+0000","last_active":"2024-09-10T09:54:11.965971+0000","last_peered":"2024-09-10T09:54:11.965971+0000","last_clean":"2024-09-10T09:54:11.965971+0000","last_became_active":"2024-09-10T09:52:19.911079+0000","last_became_peered":"2024-09-10T09:52:19.911079+0000","last_unstale":"2024-09-10T09:54:11.965971+0000","last_undegraded":"2024-09-10T09:54:11.965971+0000","last_fullsized":"2024-09-10T09:54:11.965971+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:43:38.787689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":59,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.988483+0000","last_change":"2024-09-10T09:52:31.381877+0000","last_active":"2024-09-10T09:54:00.988483+0000","last_peered":"2024-09-10T09:54:00.988483+0000","last_clean":"2024-09-10T09:54:00.988483+0000","last_became_active":"2024-09-10T09:52:31.381472+0000","last_became_peered":"2024-09-10T09:52:31.381472+0000","last_unstale":"2024-09-10T09:54:00.988483+0000","last_undegraded":"2024-09-10T09:54:00.988483+0000","last_fullsized":"2024-09-10T09:54:00.988483+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:41:24.686325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.059412+0000","last_change":"2024-09-10T09:52:22.313902+0000","last_active":"2024-09-10T09:54:01.059412+0000","last_peered":"2024-09-10T09:54:01.059412+0000","last_clean":"2024-09-10T09:54:01.059412+0000","last_became_active":"2024-09-10T09:52:22.313515+0000","last_became_peered":"2024-09-10T09:52:22.313515+0000","last_unstale":"2024-09-10T09:54:01.059412+0000","last_undegraded":"2024-09-10T09:54:01.059412+0000","last_fullsized":"2024-09-10T09:54:01.059412+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:25:00.238307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"94'3","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.847808+0000","last_change":"2024-09-10T09:52:19.837736+0000","last_active":"2024-09-10T09:54:00.847808+0000","last_peered":"2024-09-10T09:54:00.847808+0000","last_clean":"2024-09-10T09:54:00.847808+0000","last_became_active":"2024-09-10T09:52:19.837370+0000","last_became_peered":"2024-09-10T09:52:19.837370+0000","last_unstale":"2024-09-10T09:54:00.847808+0000","last_undegraded":"2024-09-10T09:54:00.847808+0000","last_fullsized":"2024-09-10T09:54:00.847808+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:54:35.753027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.248170+0000","last_change":"2024-09-10T09:52:31.378945+0000","last_active":"2024-09-10T09:54:01.248170+0000","last_peered":"2024-09-10T09:54:01.248170+0000","last_clean":"2024-09-10T09:54:01.248170+0000","last_became_active":"2024-09-10T09:52:31.378144+0000","last_became_peered":"2024-09-10T09:52:31.378144+0000","last_unstale":"2024-09-10T09:54:01.248170+0000","last_undegraded":"2024-09-10T09:54:01.248170+0000","last_fullsized":"2024-09-10T09:54:01.248170+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:36:06.288293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.059676+0000","last_change":"2024-09-10T09:52:22.316400+0000","last_active":"2024-09-10T09:54:01.059676+0000","last_peered":"2024-09-10T09:54:01.059676+0000","last_clean":"2024-09-10T09:54:01.059676+0000","last_became_active":"2024-09-10T09:52:22.315974+0000","last_became_peered":"2024-09-10T09:52:22.315974+0000","last_unstale":"2024-09-10T09:54:01.059676+0000","last_undegraded":"2024-09-10T09:54:01.059676+0000","last_fullsized":"2024-09-10T09:54:01.059676+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:31:05.538237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"94'5","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.841919+0000","last_change":"2024-09-10T09:52:19.837942+0000","last_active":"2024-09-10T09:54:00.841919+0000","last_peered":"2024-09-10T09:54:00.841919+0000","last_clean":"2024-09-10T09:54:00.841919+0000","last_became_active":"2024-09-10T09:52:19.837629+0000","last_became_peered":"2024-09-10T09:52:19.837629+0000","last_unstale":"2024-09-10T09:54:00.841919+0000","last_undegraded":"2024-09-10T09:54:00.841919+0000","last_fullsized":"2024-09-10T09:54:00.841919+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:34:13.207770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.846958+0000","last_change":"2024-09-10T09:52:31.364231+0000","last_active":"2024-09-10T09:54:00.846958+0000","last_peered":"2024-09-10T09:54:00.846958+0000","last_clean":"2024-09-10T09:54:00.846958+0000","last_became_active":"2024-09-10T09:52:31.363833+0000","last_became_peered":"2024-09-10T09:52:31.363833+0000","last_unstale":"2024-09-10T09:54:00.846958+0000","last_undegraded":"2024-09-10T09:54:00.846958+0000","last_fullsized":"2024-09-10T09:54:00.846958+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:53:51.435700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.955436+0000","last_change":"2024-09-10T09:52:22.319530+0000","last_active":"2024-09-10T09:54:00.955436+0000","last_peered":"2024-09-10T09:54:00.955436+0000","last_clean":"2024-09-10T09:54:00.955436+0000","last_became_active":"2024-09-10T09:52:22.318742+0000","last_became_peered":"2024-09-10T09:52:22.318742+0000","last_unstale":"2024-09-10T09:54:00.955436+0000","last_undegraded":"2024-09-10T09:54:00.955436+0000","last_fullsized":"2024-09-10T09:54:00.955436+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:44:35.050454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"94'2","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.149987+0000","last_change":"2024-09-10T09:52:19.836398+0000","last_active":"2024-09-10T09:54:01.149987+0000","last_peered":"2024-09-10T09:54:01.149987+0000","last_clean":"2024-09-10T09:54:01.149987+0000","last_became_active":"2024-09-10T09:52:19.836055+0000","last_became_peered":"2024-09-10T09:52:19.836055+0000","last_unstale":"2024-09-10T09:54:01.149987+0000","last_undegraded":"2024-09-10T09:54:01.149987+0000","last_fullsized":"2024-09-10T09:54:01.149987+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:49:26.923367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.847151+0000","last_change":"2024-09-10T09:52:31.371109+0000","last_active":"2024-09-10T09:54:00.847151+0000","last_peered":"2024-09-10T09:54:00.847151+0000","last_clean":"2024-09-10T09:54:00.847151+0000","last_became_active":"2024-09-10T09:52:31.370592+0000","last_became_peered":"2024-09-10T09:52:31.370592+0000","last_unstale":"2024-09-10T09:54:00.847151+0000","last_undegraded":"2024-09-10T09:54:00.847151+0000","last_fullsized":"2024-09-10T09:54:00.847151+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:20:23.585187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.847241+0000","last_change":"2024-09-10T09:52:22.319365+0000","last_active":"2024-09-10T09:54:00.847241+0000","last_peered":"2024-09-10T09:54:00.847241+0000","last_clean":"2024-09-10T09:54:00.847241+0000","last_became_active":"2024-09-10T09:52:22.319064+0000","last_became_peered":"2024-09-10T09:52:22.319064+0000","last_unstale":"2024-09-10T09:54:00.847241+0000","last_undegraded":"2024-09-10T09:54:00.847241+0000","last_fullsized":"2024-09-10T09:54:00.847241+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:14:50.089566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"94'17","reported_seq":58,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.149249+0000","last_change":"2024-09-10T09:52:19.911698+0000","last_active":"2024-09-10T09:54:01.149249+0000","last_peered":"2024-09-10T09:54:01.149249+0000","last_clean":"2024-09-10T09:54:01.149249+0000","last_became_active":"2024-09-10T09:52:19.911369+0000","last_became_peered":"2024-09-10T09:52:19.911369+0000","last_unstale":"2024-09-10T09:54:01.149249+0000","last_undegraded":"2024-09-10T09:54:01.149249+0000","last_fullsized":"2024-09-10T09:54:01.149249+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:40:39.045797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.841907+0000","last_change":"2024-09-10T09:52:31.380463+0000","last_active":"2024-09-10T09:54:00.841907+0000","last_peered":"2024-09-10T09:54:00.841907+0000","last_clean":"2024-09-10T09:54:00.841907+0000","last_became_active":"2024-09-10T09:52:31.380114+0000","last_became_peered":"2024-09-10T09:52:31.380114+0000","last_unstale":"2024-09-10T09:54:00.841907+0000","last_undegraded":"2024-09-10T09:54:00.841907+0000","last_fullsized":"2024-09-10T09:54:00.841907+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:19:50.869160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.058932+0000","last_change":"2024-09-10T09:52:22.313928+0000","last_active":"2024-09-10T09:54:01.058932+0000","last_peered":"2024-09-10T09:54:01.058932+0000","last_clean":"2024-09-10T09:54:01.058932+0000","last_became_active":"2024-09-10T09:52:22.313510+0000","last_became_peered":"2024-09-10T09:52:22.313510+0000","last_unstale":"2024-09-10T09:54:01.058932+0000","last_undegraded":"2024-09-10T09:54:01.058932+0000","last_fullsized":"2024-09-10T09:54:01.058932+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:58:19.031178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"94'6","reported_seq":107,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:11.970409+0000","last_change":"2024-09-10T09:52:19.839451+0000","last_active":"2024-09-10T09:54:11.970409+0000","last_peered":"2024-09-10T09:54:11.970409+0000","last_clean":"2024-09-10T09:54:11.970409+0000","last_became_active":"2024-09-10T09:52:19.839124+0000","last_became_peered":"2024-09-10T09:52:19.839124+0000","last_unstale":"2024-09-10T09:54:11.970409+0000","last_undegraded":"2024-09-10T09:54:11.970409+0000","last_fullsized":"2024-09-10T09:54:11.970409+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:51:33.489830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":61,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.844303+0000","last_change":"2024-09-10T09:52:31.376386+0000","last_active":"2024-09-10T09:54:00.844303+0000","last_peered":"2024-09-10T09:54:00.844303+0000","last_clean":"2024-09-10T09:54:00.844303+0000","last_became_active":"2024-09-10T09:52:31.376066+0000","last_became_peered":"2024-09-10T09:52:31.376066+0000","last_unstale":"2024-09-10T09:54:00.844303+0000","last_undegraded":"2024-09-10T09:54:00.844303+0000","last_fullsized":"2024-09-10T09:54:00.844303+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:23:07.191842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.059245+0000","last_change":"2024-09-10T09:52:22.323590+0000","last_active":"2024-09-10T09:54:01.059245+0000","last_peered":"2024-09-10T09:54:01.059245+0000","last_clean":"2024-09-10T09:54:01.059245+0000","last_became_active":"2024-09-10T09:52:22.323264+0000","last_became_peered":"2024-09-10T09:52:22.323264+0000","last_unstale":"2024-09-10T09:54:01.059245+0000","last_undegraded":"2024-09-10T09:54:01.059245+0000","last_fullsized":"2024-09-10T09:54:01.059245+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:07:56.572337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"94'5","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.847453+0000","last_change":"2024-09-10T09:52:19.840620+0000","last_active":"2024-09-10T09:54:00.847453+0000","last_peered":"2024-09-10T09:54:00.847453+0000","last_clean":"2024-09-10T09:54:00.847453+0000","last_became_active":"2024-09-10T09:52:19.840217+0000","last_became_peered":"2024-09-10T09:52:19.840217+0000","last_unstale":"2024-09-10T09:54:00.847453+0000","last_undegraded":"2024-09-10T09:54:00.847453+0000","last_fullsized":"2024-09-10T09:54:00.847453+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:55:13.111993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.955012+0000","last_change":"2024-09-10T09:52:31.373895+0000","last_active":"2024-09-10T09:54:00.955012+0000","last_peered":"2024-09-10T09:54:00.955012+0000","last_clean":"2024-09-10T09:54:00.955012+0000","last_became_active":"2024-09-10T09:52:31.373474+0000","last_became_peered":"2024-09-10T09:52:31.373474+0000","last_unstale":"2024-09-10T09:54:00.955012+0000","last_undegraded":"2024-09-10T09:54:00.955012+0000","last_fullsized":"2024-09-10T09:54:00.955012+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:39:16.049859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.841146+0000","last_change":"2024-09-10T09:52:22.319294+0000","last_active":"2024-09-10T09:54:00.841146+0000","last_peered":"2024-09-10T09:54:00.841146+0000","last_clean":"2024-09-10T09:54:00.841146+0000","last_became_active":"2024-09-10T09:52:22.318918+0000","last_became_peered":"2024-09-10T09:52:22.318918+0000","last_unstale":"2024-09-10T09:54:00.841146+0000","last_undegraded":"2024-09-10T09:54:00.841146+0000","last_fullsized":"2024-09-10T09:54:00.841146+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:52:46.434182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"94'9","reported_seq":52,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.844181+0000","last_change":"2024-09-10T09:52:19.842286+0000","last_active":"2024-09-10T09:54:00.844181+0000","last_peered":"2024-09-10T09:54:00.844181+0000","last_clean":"2024-09-10T09:54:00.844181+0000","last_became_active":"2024-09-10T09:52:19.841930+0000","last_became_peered":"2024-09-10T09:52:19.841930+0000","last_unstale":"2024-09-10T09:54:00.844181+0000","last_undegraded":"2024-09-10T09:54:00.844181+0000","last_fullsized":"2024-09-10T09:54:00.844181+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:38:46.358389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.847452+0000","last_change":"2024-09-10T09:52:31.370961+0000","last_active":"2024-09-10T09:54:00.847452+0000","last_peered":"2024-09-10T09:54:00.847452+0000","last_clean":"2024-09-10T09:54:00.847452+0000","last_became_active":"2024-09-10T09:52:31.370545+0000","last_became_peered":"2024-09-10T09:52:31.370545+0000","last_unstale":"2024-09-10T09:54:00.847452+0000","last_undegraded":"2024-09-10T09:54:00.847452+0000","last_fullsized":"2024-09-10T09:54:00.847452+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:17.461447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.149789+0000","last_change":"2024-09-10T09:52:22.313215+0000","last_active":"2024-09-10T09:54:01.149789+0000","last_peered":"2024-09-10T09:54:01.149789+0000","last_clean":"2024-09-10T09:54:01.149789+0000","last_became_active":"2024-09-10T09:52:22.312918+0000","last_became_peered":"2024-09-10T09:52:22.312918+0000","last_unstale":"2024-09-10T09:54:01.149789+0000","last_undegraded":"2024-09-10T09:54:01.149789+0000","last_fullsized":"2024-09-10T09:54:01.149789+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:48:09.568004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"94'34","reported_seq":212,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:11.965210+0000","last_change":"2024-09-10T09:53:01.226146+0000","last_active":"2024-09-10T09:54:11.965210+0000","last_peered":"2024-09-10T09:54:11.965210+0000","last_clean":"2024-09-10T09:54:11.965210+0000","last_became_active":"2024-09-10T09:53:01.218622+0000","last_became_peered":"2024-09-10T09:53:01.218622+0000","last_unstale":"2024-09-10T09:54:11.965210+0000","last_undegraded":"2024-09-10T09:54:11.965210+0000","last_fullsized":"2024-09-10T09:54:11.965210+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:43:11.927029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63002,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":120,"num_read_kb":121,"num_write":36,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":3844,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":16,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.841392+0000","last_change":"2024-09-10T09:53:00.800650+0000","last_active":"2024-09-10T09:54:00.841392+0000","last_peered":"2024-09-10T09:54:00.841392+0000","last_clean":"2024-09-10T09:54:00.841392+0000","last_became_active":"2024-09-10T09:53:00.800104+0000","last_became_peered":"2024-09-10T09:53:00.800104+0000","last_unstale":"2024-09-10T09:54:00.841392+0000","last_undegraded":"2024-09-10T09:54:00.841392+0000","last_fullsized":"2024-09-10T09:54:00.841392+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:41:16.022884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.988810+0000","last_change":"2024-09-10T09:52:22.317174+0000","last_active":"2024-09-10T09:54:00.988810+0000","last_peered":"2024-09-10T09:54:00.988810+0000","last_clean":"2024-09-10T09:54:00.988810+0000","last_became_active":"2024-09-10T09:52:22.316851+0000","last_became_peered":"2024-09-10T09:52:22.316851+0000","last_unstale":"2024-09-10T09:54:00.988810+0000","last_undegraded":"2024-09-10T09:54:00.988810+0000","last_fullsized":"2024-09-10T09:54:00.988810+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:55:18.742393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"94'5","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.847663+0000","last_change":"2024-09-10T09:52:19.912093+0000","last_active":"2024-09-10T09:54:00.847663+0000","last_peered":"2024-09-10T09:54:00.847663+0000","last_clean":"2024-09-10T09:54:00.847663+0000","last_became_active":"2024-09-10T09:52:19.911611+0000","last_became_peered":"2024-09-10T09:52:19.911611+0000","last_unstale":"2024-09-10T09:54:00.847663+0000","last_undegraded":"2024-09-10T09:54:00.847663+0000","last_fullsized":"2024-09-10T09:54:00.847663+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:01:10.859310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.951597+0000","last_change":"2024-09-10T09:52:31.362884+0000","last_active":"2024-09-10T09:54:00.951597+0000","last_peered":"2024-09-10T09:54:00.951597+0000","last_clean":"2024-09-10T09:54:00.951597+0000","last_became_active":"2024-09-10T09:52:31.362489+0000","last_became_peered":"2024-09-10T09:52:31.362489+0000","last_unstale":"2024-09-10T09:54:00.951597+0000","last_undegraded":"2024-09-10T09:54:00.951597+0000","last_fullsized":"2024-09-10T09:54:00.951597+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:02:33.084795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.248236+0000","last_change":"2024-09-10T09:52:22.325871+0000","last_active":"2024-09-10T09:54:01.248236+0000","last_peered":"2024-09-10T09:54:01.248236+0000","last_clean":"2024-09-10T09:54:01.248236+0000","last_became_active":"2024-09-10T09:52:22.324980+0000","last_became_peered":"2024-09-10T09:52:22.324980+0000","last_unstale":"2024-09-10T09:54:01.248236+0000","last_undegraded":"2024-09-10T09:54:01.248236+0000","last_fullsized":"2024-09-10T09:54:01.248236+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:31:55.713444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"94'4","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.059600+0000","last_change":"2024-09-10T09:52:19.839491+0000","last_active":"2024-09-10T09:54:01.059600+0000","last_peered":"2024-09-10T09:54:01.059600+0000","last_clean":"2024-09-10T09:54:01.059600+0000","last_became_active":"2024-09-10T09:52:19.839092+0000","last_became_peered":"2024-09-10T09:52:19.839092+0000","last_unstale":"2024-09-10T09:54:01.059600+0000","last_undegraded":"2024-09-10T09:54:01.059600+0000","last_fullsized":"2024-09-10T09:54:01.059600+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:42:13.453281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.843235+0000","last_change":"2024-09-10T09:52:31.371346+0000","last_active":"2024-09-10T09:54:00.843235+0000","last_peered":"2024-09-10T09:54:00.843235+0000","last_clean":"2024-09-10T09:54:00.843235+0000","last_became_active":"2024-09-10T09:52:31.370929+0000","last_became_peered":"2024-09-10T09:52:31.370929+0000","last_unstale":"2024-09-10T09:54:00.843235+0000","last_undegraded":"2024-09-10T09:54:00.843235+0000","last_fullsized":"2024-09-10T09:54:00.843235+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:44:22.925082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.988772+0000","last_change":"2024-09-10T09:52:22.317453+0000","last_active":"2024-09-10T09:54:00.988772+0000","last_peered":"2024-09-10T09:54:00.988772+0000","last_clean":"2024-09-10T09:54:00.988772+0000","last_became_active":"2024-09-10T09:52:22.315556+0000","last_became_peered":"2024-09-10T09:52:22.315556+0000","last_unstale":"2024-09-10T09:54:00.988772+0000","last_undegraded":"2024-09-10T09:54:00.988772+0000","last_fullsized":"2024-09-10T09:54:00.988772+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:56:05.860275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"94'4","reported_seq":158,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:11.965187+0000","last_change":"2024-09-10T09:52:19.835985+0000","last_active":"2024-09-10T09:54:11.965187+0000","last_peered":"2024-09-10T09:54:11.965187+0000","last_clean":"2024-09-10T09:54:11.965187+0000","last_became_active":"2024-09-10T09:52:19.834835+0000","last_became_peered":"2024-09-10T09:52:19.834835+0000","last_unstale":"2024-09-10T09:54:11.965187+0000","last_undegraded":"2024-09-10T09:54:11.965187+0000","last_fullsized":"2024-09-10T09:54:11.965187+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:09:11.590443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":112,"num_read_kb":112,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.845677+0000","last_change":"2024-09-10T09:52:31.368663+0000","last_active":"2024-09-10T09:54:00.845677+0000","last_peered":"2024-09-10T09:54:00.845677+0000","last_clean":"2024-09-10T09:54:00.845677+0000","last_became_active":"2024-09-10T09:52:31.368004+0000","last_became_peered":"2024-09-10T09:52:31.368004+0000","last_unstale":"2024-09-10T09:54:00.845677+0000","last_undegraded":"2024-09-10T09:54:00.845677+0000","last_fullsized":"2024-09-10T09:54:00.845677+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:56:15.100515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.247874+0000","last_change":"2024-09-10T09:52:31.370992+0000","last_active":"2024-09-10T09:54:01.247874+0000","last_peered":"2024-09-10T09:54:01.247874+0000","last_clean":"2024-09-10T09:54:01.247874+0000","last_became_active":"2024-09-10T09:52:31.370357+0000","last_became_peered":"2024-09-10T09:52:31.370357+0000","last_unstale":"2024-09-10T09:54:01.247874+0000","last_undegraded":"2024-09-10T09:54:01.247874+0000","last_fullsized":"2024-09-10T09:54:01.247874+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:02:49.482532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.843524+0000","last_change":"2024-09-10T09:52:22.319363+0000","last_active":"2024-09-10T09:54:00.843524+0000","last_peered":"2024-09-10T09:54:00.843524+0000","last_clean":"2024-09-10T09:54:00.843524+0000","last_became_active":"2024-09-10T09:52:22.319044+0000","last_became_peered":"2024-09-10T09:52:22.319044+0000","last_unstale":"2024-09-10T09:54:00.843524+0000","last_undegraded":"2024-09-10T09:54:00.843524+0000","last_fullsized":"2024-09-10T09:54:00.843524+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:30:51.328413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"94'3","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.847564+0000","last_change":"2024-09-10T09:52:19.841745+0000","last_active":"2024-09-10T09:54:00.847564+0000","last_peered":"2024-09-10T09:54:00.847564+0000","last_clean":"2024-09-10T09:54:00.847564+0000","last_became_active":"2024-09-10T09:52:19.841404+0000","last_became_peered":"2024-09-10T09:52:19.841404+0000","last_unstale":"2024-09-10T09:54:00.847564+0000","last_undegraded":"2024-09-10T09:54:00.847564+0000","last_fullsized":"2024-09-10T09:54:00.847564+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:44:36.346785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.955474+0000","last_change":"2024-09-10T09:53:01.218412+0000","last_active":"2024-09-10T09:54:00.955474+0000","last_peered":"2024-09-10T09:54:00.955474+0000","last_clean":"2024-09-10T09:54:00.955474+0000","last_became_active":"2024-09-10T09:53:01.217966+0000","last_became_peered":"2024-09-10T09:53:01.217966+0000","last_unstale":"2024-09-10T09:54:00.955474+0000","last_undegraded":"2024-09-10T09:54:00.955474+0000","last_fullsized":"2024-09-10T09:54:00.955474+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:53:25.743913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.845534+0000","last_change":"2024-09-10T09:52:22.316120+0000","last_active":"2024-09-10T09:54:00.845534+0000","last_peered":"2024-09-10T09:54:00.845534+0000","last_clean":"2024-09-10T09:54:00.845534+0000","last_became_active":"2024-09-10T09:52:22.315769+0000","last_became_peered":"2024-09-10T09:52:22.315769+0000","last_unstale":"2024-09-10T09:54:00.845534+0000","last_undegraded":"2024-09-10T09:54:00.845534+0000","last_fullsized":"2024-09-10T09:54:00.845534+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:16:41.866587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"94'3","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.840993+0000","last_change":"2024-09-10T09:52:19.835038+0000","last_active":"2024-09-10T09:54:00.840993+0000","last_peered":"2024-09-10T09:54:00.840993+0000","last_clean":"2024-09-10T09:54:00.840993+0000","last_became_active":"2024-09-10T09:52:19.834329+0000","last_became_peered":"2024-09-10T09:52:19.834329+0000","last_unstale":"2024-09-10T09:54:00.840993+0000","last_undegraded":"2024-09-10T09:54:00.840993+0000","last_fullsized":"2024-09-10T09:54:00.840993+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:58:24.084373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"94'7","reported_seq":55,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.841698+0000","last_change":"2024-09-10T09:53:01.219122+0000","last_active":"2024-09-10T09:54:00.841698+0000","last_peered":"2024-09-10T09:54:00.841698+0000","last_clean":"2024-09-10T09:54:00.841698+0000","last_became_active":"2024-09-10T09:53:01.218599+0000","last_became_peered":"2024-09-10T09:53:01.218599+0000","last_unstale":"2024-09-10T09:54:00.841698+0000","last_undegraded":"2024-09-10T09:54:00.841698+0000","last_fullsized":"2024-09-10T09:54:00.841698+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:31:10.092264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.953227+0000","last_change":"2024-09-10T09:54:00.952617+0000","last_active":"2024-09-10T09:54:00.953227+0000","last_peered":"2024-09-10T09:54:00.953227+0000","last_clean":"2024-09-10T09:54:00.953227+0000","last_became_active":"2024-09-10T09:54:00.952202+0000","last_became_peered":"2024-09-10T09:54:00.952202+0000","last_unstale":"2024-09-10T09:54:00.953227+0000","last_undegraded":"2024-09-10T09:54:00.953227+0000","last_fullsized":"2024-09-10T09:54:00.953227+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:35:40.179591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.844105+0000","last_change":"2024-09-10T09:52:31.371612+0000","last_active":"2024-09-10T09:54:00.844105+0000","last_peered":"2024-09-10T09:54:00.844105+0000","last_clean":"2024-09-10T09:54:00.844105+0000","last_became_active":"2024-09-10T09:52:31.371163+0000","last_became_peered":"2024-09-10T09:52:31.371163+0000","last_unstale":"2024-09-10T09:54:00.844105+0000","last_undegraded":"2024-09-10T09:54:00.844105+0000","last_fullsized":"2024-09-10T09:54:00.844105+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:11:43.681524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"94'6","reported_seq":108,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:11.965955+0000","last_change":"2024-09-10T09:52:19.841873+0000","last_active":"2024-09-10T09:54:11.965955+0000","last_peered":"2024-09-10T09:54:11.965955+0000","last_clean":"2024-09-10T09:54:11.965955+0000","last_became_active":"2024-09-10T09:52:19.841513+0000","last_became_peered":"2024-09-10T09:52:19.841513+0000","last_unstale":"2024-09-10T09:54:11.965955+0000","last_undegraded":"2024-09-10T09:54:11.965955+0000","last_fullsized":"2024-09-10T09:54:11.965955+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:53:31.854185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":60,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.149395+0000","last_change":"2024-09-10T09:52:22.314781+0000","last_active":"2024-09-10T09:54:01.149395+0000","last_peered":"2024-09-10T09:54:01.149395+0000","last_clean":"2024-09-10T09:54:01.149395+0000","last_became_active":"2024-09-10T09:52:22.313346+0000","last_became_peered":"2024-09-10T09:52:22.313346+0000","last_unstale":"2024-09-10T09:54:01.149395+0000","last_undegraded":"2024-09-10T09:54:01.149395+0000","last_fullsized":"2024-09-10T09:54:01.149395+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:15:16.921963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.844346+0000","last_change":"2024-09-10T09:52:31.365544+0000","last_active":"2024-09-10T09:54:00.844346+0000","last_peered":"2024-09-10T09:54:00.844346+0000","last_clean":"2024-09-10T09:54:00.844346+0000","last_became_active":"2024-09-10T09:52:31.365151+0000","last_became_peered":"2024-09-10T09:52:31.365151+0000","last_unstale":"2024-09-10T09:54:00.844346+0000","last_undegraded":"2024-09-10T09:54:00.844346+0000","last_fullsized":"2024-09-10T09:54:00.844346+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:36:55.247890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.954850+0000","last_change":"2024-09-10T09:52:22.314117+0000","last_active":"2024-09-10T09:54:00.954850+0000","last_peered":"2024-09-10T09:54:00.954850+0000","last_clean":"2024-09-10T09:54:00.954850+0000","last_became_active":"2024-09-10T09:52:22.313772+0000","last_became_peered":"2024-09-10T09:52:22.313772+0000","last_unstale":"2024-09-10T09:54:00.954850+0000","last_undegraded":"2024-09-10T09:54:00.954850+0000","last_fullsized":"2024-09-10T09:54:00.954850+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:19:37.373598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.844246+0000","last_change":"2024-09-10T09:52:22.314362+0000","last_active":"2024-09-10T09:54:00.844246+0000","last_peered":"2024-09-10T09:54:00.844246+0000","last_clean":"2024-09-10T09:54:00.844246+0000","last_became_active":"2024-09-10T09:52:22.314050+0000","last_became_peered":"2024-09-10T09:52:22.314050+0000","last_unstale":"2024-09-10T09:54:00.844246+0000","last_undegraded":"2024-09-10T09:54:00.844246+0000","last_fullsized":"2024-09-10T09:54:00.844246+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:37:07.218656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.843976+0000","last_change":"2024-09-10T09:52:22.322363+0000","last_active":"2024-09-10T09:54:00.843976+0000","last_peered":"2024-09-10T09:54:00.843976+0000","last_clean":"2024-09-10T09:54:00.843976+0000","last_became_active":"2024-09-10T09:52:22.322045+0000","last_became_peered":"2024-09-10T09:52:22.322045+0000","last_unstale":"2024-09-10T09:54:00.843976+0000","last_undegraded":"2024-09-10T09:54:00.843976+0000","last_fullsized":"2024-09-10T09:54:00.843976+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:10:27.918769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.247998+0000","last_change":"2024-09-10T09:52:22.331594+0000","last_active":"2024-09-10T09:54:01.247998+0000","last_peered":"2024-09-10T09:54:01.247998+0000","last_clean":"2024-09-10T09:54:01.247998+0000","last_became_active":"2024-09-10T09:52:22.330585+0000","last_became_peered":"2024-09-10T09:52:22.330585+0000","last_unstale":"2024-09-10T09:54:01.247998+0000","last_undegraded":"2024-09-10T09:54:01.247998+0000","last_fullsized":"2024-09-10T09:54:01.247998+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:45:56.412313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.841849+0000","last_change":"2024-09-10T09:52:22.325753+0000","last_active":"2024-09-10T09:54:00.841849+0000","last_peered":"2024-09-10T09:54:00.841849+0000","last_clean":"2024-09-10T09:54:00.841849+0000","last_became_active":"2024-09-10T09:52:22.325385+0000","last_became_peered":"2024-09-10T09:52:22.325385+0000","last_unstale":"2024-09-10T09:54:00.841849+0000","last_undegraded":"2024-09-10T09:54:00.841849+0000","last_fullsized":"2024-09-10T09:54:00.841849+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:18:28.776789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.955325+0000","last_change":"2024-09-10T09:52:22.319069+0000","last_active":"2024-09-10T09:54:00.955325+0000","last_peered":"2024-09-10T09:54:00.955325+0000","last_clean":"2024-09-10T09:54:00.955325+0000","last_became_active":"2024-09-10T09:52:22.318667+0000","last_became_peered":"2024-09-10T09:52:22.318667+0000","last_unstale":"2024-09-10T09:54:00.955325+0000","last_undegraded":"2024-09-10T09:54:00.955325+0000","last_fullsized":"2024-09-10T09:54:00.955325+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:21:32.028235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.844567+0000","last_change":"2024-09-10T09:52:22.312890+0000","last_active":"2024-09-10T09:54:00.844567+0000","last_peered":"2024-09-10T09:54:00.844567+0000","last_clean":"2024-09-10T09:54:00.844567+0000","last_became_active":"2024-09-10T09:52:22.312608+0000","last_became_peered":"2024-09-10T09:52:22.312608+0000","last_unstale":"2024-09-10T09:54:00.844567+0000","last_undegraded":"2024-09-10T09:54:00.844567+0000","last_fullsized":"2024-09-10T09:54:00.844567+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:07:46.046915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.151385+0000","last_change":"2024-09-10T09:54:01.151385+0000","last_active":"2024-09-10T09:54:01.151385+0000","last_peered":"2024-09-10T09:54:01.151385+0000","last_clean":"2024-09-10T09:54:01.151385+0000","last_became_active":"2024-09-10T09:54:01.150971+0000","last_became_peered":"2024-09-10T09:54:01.150971+0000","last_unstale":"2024-09-10T09:54:01.151385+0000","last_undegraded":"2024-09-10T09:54:01.151385+0000","last_fullsized":"2024-09-10T09:54:01.151385+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:27:10.492071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.849203+0000","last_change":"2024-09-10T09:54:00.848273+0000","last_active":"2024-09-10T09:54:00.849203+0000","last_peered":"2024-09-10T09:54:00.849203+0000","last_clean":"2024-09-10T09:54:00.849203+0000","last_became_active":"2024-09-10T09:54:00.847492+0000","last_became_peered":"2024-09-10T09:54:00.847492+0000","last_unstale":"2024-09-10T09:54:00.849203+0000","last_undegraded":"2024-09-10T09:54:00.849203+0000","last_fullsized":"2024-09-10T09:54:00.849203+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:02:24.749291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,3],"acting":[6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.845839+0000","last_change":"2024-09-10T09:52:22.313538+0000","last_active":"2024-09-10T09:54:00.845839+0000","last_peered":"2024-09-10T09:54:00.845839+0000","last_clean":"2024-09-10T09:54:00.845839+0000","last_became_active":"2024-09-10T09:52:22.313196+0000","last_became_peered":"2024-09-10T09:52:22.313196+0000","last_unstale":"2024-09-10T09:54:00.845839+0000","last_undegraded":"2024-09-10T09:54:00.845839+0000","last_fullsized":"2024-09-10T09:54:00.845839+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:53:23.387167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.988773+0000","last_change":"2024-09-10T09:52:22.316518+0000","last_active":"2024-09-10T09:54:00.988773+0000","last_peered":"2024-09-10T09:54:00.988773+0000","last_clean":"2024-09-10T09:54:00.988773+0000","last_became_active":"2024-09-10T09:52:22.316193+0000","last_became_peered":"2024-09-10T09:52:22.316193+0000","last_unstale":"2024-09-10T09:54:00.988773+0000","last_undegraded":"2024-09-10T09:54:00.988773+0000","last_fullsized":"2024-09-10T09:54:00.988773+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:49:55.903928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.841553+0000","last_change":"2024-09-10T09:52:22.329163+0000","last_active":"2024-09-10T09:54:00.841553+0000","last_peered":"2024-09-10T09:54:00.841553+0000","last_clean":"2024-09-10T09:54:00.841553+0000","last_became_active":"2024-09-10T09:52:22.328453+0000","last_became_peered":"2024-09-10T09:52:22.328453+0000","last_unstale":"2024-09-10T09:54:00.841553+0000","last_undegraded":"2024-09-10T09:54:00.841553+0000","last_fullsized":"2024-09-10T09:54:00.841553+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:53:33.286055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.955022+0000","last_change":"2024-09-10T09:52:22.314302+0000","last_active":"2024-09-10T09:54:00.955022+0000","last_peered":"2024-09-10T09:54:00.955022+0000","last_clean":"2024-09-10T09:54:00.955022+0000","last_became_active":"2024-09-10T09:52:22.313992+0000","last_became_peered":"2024-09-10T09:52:22.313992+0000","last_unstale":"2024-09-10T09:54:00.955022+0000","last_undegraded":"2024-09-10T09:54:00.955022+0000","last_fullsized":"2024-09-10T09:54:00.955022+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:07:35.764015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.844450+0000","last_change":"2024-09-10T09:52:22.316400+0000","last_active":"2024-09-10T09:54:00.844450+0000","last_peered":"2024-09-10T09:54:00.844450+0000","last_clean":"2024-09-10T09:54:00.844450+0000","last_became_active":"2024-09-10T09:52:22.316110+0000","last_became_peered":"2024-09-10T09:52:22.316110+0000","last_unstale":"2024-09-10T09:54:00.844450+0000","last_undegraded":"2024-09-10T09:54:00.844450+0000","last_fullsized":"2024-09-10T09:54:00.844450+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:17:26.528899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.844292+0000","last_change":"2024-09-10T09:52:22.317914+0000","last_active":"2024-09-10T09:54:00.844292+0000","last_peered":"2024-09-10T09:54:00.844292+0000","last_clean":"2024-09-10T09:54:00.844292+0000","last_became_active":"2024-09-10T09:52:22.317575+0000","last_became_peered":"2024-09-10T09:52:22.317575+0000","last_unstale":"2024-09-10T09:54:00.844292+0000","last_undegraded":"2024-09-10T09:54:00.844292+0000","last_fullsized":"2024-09-10T09:54:00.844292+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:17:51.446233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.955646+0000","last_change":"2024-09-10T09:52:22.327749+0000","last_active":"2024-09-10T09:54:00.955646+0000","last_peered":"2024-09-10T09:54:00.955646+0000","last_clean":"2024-09-10T09:54:00.955646+0000","last_became_active":"2024-09-10T09:52:22.327461+0000","last_became_peered":"2024-09-10T09:52:22.327461+0000","last_unstale":"2024-09-10T09:54:00.955646+0000","last_undegraded":"2024-09-10T09:54:00.955646+0000","last_fullsized":"2024-09-10T09:54:00.955646+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:54.208064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.988358+0000","last_change":"2024-09-10T09:52:22.326982+0000","last_active":"2024-09-10T09:54:00.988358+0000","last_peered":"2024-09-10T09:54:00.988358+0000","last_clean":"2024-09-10T09:54:00.988358+0000","last_became_active":"2024-09-10T09:52:22.326635+0000","last_became_peered":"2024-09-10T09:52:22.326635+0000","last_unstale":"2024-09-10T09:54:00.988358+0000","last_undegraded":"2024-09-10T09:54:00.988358+0000","last_fullsized":"2024-09-10T09:54:00.988358+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:50:54.835899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.247654+0000","last_change":"2024-09-10T09:52:22.323410+0000","last_active":"2024-09-10T09:54:01.247654+0000","last_peered":"2024-09-10T09:54:01.247654+0000","last_clean":"2024-09-10T09:54:01.247654+0000","last_became_active":"2024-09-10T09:52:22.323116+0000","last_became_peered":"2024-09-10T09:52:22.323116+0000","last_unstale":"2024-09-10T09:54:01.247654+0000","last_undegraded":"2024-09-10T09:54:01.247654+0000","last_fullsized":"2024-09-10T09:54:01.247654+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:09:15.115162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.247721+0000","last_change":"2024-09-10T09:52:22.324908+0000","last_active":"2024-09-10T09:54:01.247721+0000","last_peered":"2024-09-10T09:54:01.247721+0000","last_clean":"2024-09-10T09:54:01.247721+0000","last_became_active":"2024-09-10T09:52:22.324449+0000","last_became_peered":"2024-09-10T09:52:22.324449+0000","last_unstale":"2024-09-10T09:54:01.247721+0000","last_undegraded":"2024-09-10T09:54:01.247721+0000","last_fullsized":"2024-09-10T09:54:01.247721+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:43:24.535721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.841549+0000","last_change":"2024-09-10T09:52:22.322157+0000","last_active":"2024-09-10T09:54:00.841549+0000","last_peered":"2024-09-10T09:54:00.841549+0000","last_clean":"2024-09-10T09:54:00.841549+0000","last_became_active":"2024-09-10T09:52:22.321856+0000","last_became_peered":"2024-09-10T09:52:22.321856+0000","last_unstale":"2024-09-10T09:54:00.841549+0000","last_undegraded":"2024-09-10T09:54:00.841549+0000","last_fullsized":"2024-09-10T09:54:00.841549+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:36:38.395370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":15,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.843379+0000","last_change":"2024-09-10T09:53:00.800337+0000","last_active":"2024-09-10T09:54:00.843379+0000","last_peered":"2024-09-10T09:54:00.843379+0000","last_clean":"2024-09-10T09:54:00.843379+0000","last_became_active":"2024-09-10T09:53:00.799863+0000","last_became_peered":"2024-09-10T09:53:00.799863+0000","last_unstale":"2024-09-10T09:54:00.843379+0000","last_undegraded":"2024-09-10T09:54:00.843379+0000","last_fullsized":"2024-09-10T09:54:00.843379+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:18.017672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.059591+0000","last_change":"2024-09-10T09:52:22.310735+0000","last_active":"2024-09-10T09:54:01.059591+0000","last_peered":"2024-09-10T09:54:01.059591+0000","last_clean":"2024-09-10T09:54:01.059591+0000","last_became_active":"2024-09-10T09:52:22.310376+0000","last_became_peered":"2024-09-10T09:52:22.310376+0000","last_unstale":"2024-09-10T09:54:01.059591+0000","last_undegraded":"2024-09-10T09:54:01.059591+0000","last_fullsized":"2024-09-10T09:54:01.059591+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:38:08.637588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.841944+0000","last_change":"2024-09-10T09:52:22.321113+0000","last_active":"2024-09-10T09:54:00.841944+0000","last_peered":"2024-09-10T09:54:00.841944+0000","last_clean":"2024-09-10T09:54:00.841944+0000","last_became_active":"2024-09-10T09:52:22.320767+0000","last_became_peered":"2024-09-10T09:52:22.320767+0000","last_unstale":"2024-09-10T09:54:00.841944+0000","last_undegraded":"2024-09-10T09:54:00.841944+0000","last_fullsized":"2024-09-10T09:54:00.841944+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:37.549322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.989072+0000","last_change":"2024-09-10T09:52:22.318634+0000","last_active":"2024-09-10T09:54:00.989072+0000","last_peered":"2024-09-10T09:54:00.989072+0000","last_clean":"2024-09-10T09:54:00.989072+0000","last_became_active":"2024-09-10T09:52:22.318294+0000","last_became_peered":"2024-09-10T09:52:22.318294+0000","last_unstale":"2024-09-10T09:54:00.989072+0000","last_undegraded":"2024-09-10T09:54:00.989072+0000","last_fullsized":"2024-09-10T09:54:00.989072+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:19:31.262055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.846113+0000","last_change":"2024-09-10T09:52:22.324567+0000","last_active":"2024-09-10T09:54:00.846113+0000","last_peered":"2024-09-10T09:54:00.846113+0000","last_clean":"2024-09-10T09:54:00.846113+0000","last_became_active":"2024-09-10T09:52:22.321803+0000","last_became_peered":"2024-09-10T09:52:22.321803+0000","last_unstale":"2024-09-10T09:54:00.846113+0000","last_undegraded":"2024-09-10T09:54:00.846113+0000","last_fullsized":"2024-09-10T09:54:00.846113+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:08:11.099428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.059075+0000","last_change":"2024-09-10T09:52:22.318163+0000","last_active":"2024-09-10T09:54:01.059075+0000","last_peered":"2024-09-10T09:54:01.059075+0000","last_clean":"2024-09-10T09:54:01.059075+0000","last_became_active":"2024-09-10T09:52:22.317808+0000","last_became_peered":"2024-09-10T09:52:22.317808+0000","last_unstale":"2024-09-10T09:54:01.059075+0000","last_undegraded":"2024-09-10T09:54:01.059075+0000","last_fullsized":"2024-09-10T09:54:01.059075+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:19.214291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.841550+0000","last_change":"2024-09-10T09:52:22.323035+0000","last_active":"2024-09-10T09:54:00.841550+0000","last_peered":"2024-09-10T09:54:00.841550+0000","last_clean":"2024-09-10T09:54:00.841550+0000","last_became_active":"2024-09-10T09:52:22.322679+0000","last_became_peered":"2024-09-10T09:52:22.322679+0000","last_unstale":"2024-09-10T09:54:00.841550+0000","last_undegraded":"2024-09-10T09:54:00.841550+0000","last_fullsized":"2024-09-10T09:54:00.841550+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:23:13.762612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.149321+0000","last_change":"2024-09-10T09:52:22.315887+0000","last_active":"2024-09-10T09:54:01.149321+0000","last_peered":"2024-09-10T09:54:01.149321+0000","last_clean":"2024-09-10T09:54:01.149321+0000","last_became_active":"2024-09-10T09:52:22.315569+0000","last_became_peered":"2024-09-10T09:52:22.315569+0000","last_unstale":"2024-09-10T09:54:01.149321+0000","last_undegraded":"2024-09-10T09:54:01.149321+0000","last_fullsized":"2024-09-10T09:54:01.149321+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:46:57.094320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.951738+0000","last_change":"2024-09-10T09:52:22.314060+0000","last_active":"2024-09-10T09:54:00.951738+0000","last_peered":"2024-09-10T09:54:00.951738+0000","last_clean":"2024-09-10T09:54:00.951738+0000","last_became_active":"2024-09-10T09:52:22.313572+0000","last_became_peered":"2024-09-10T09:52:22.313572+0000","last_unstale":"2024-09-10T09:54:00.951738+0000","last_undegraded":"2024-09-10T09:54:00.951738+0000","last_fullsized":"2024-09-10T09:54:00.951738+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:11:17.158625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.247866+0000","last_change":"2024-09-10T09:52:22.327233+0000","last_active":"2024-09-10T09:54:01.247866+0000","last_peered":"2024-09-10T09:54:01.247866+0000","last_clean":"2024-09-10T09:54:01.247866+0000","last_became_active":"2024-09-10T09:52:22.326945+0000","last_became_peered":"2024-09-10T09:52:22.326945+0000","last_unstale":"2024-09-10T09:54:01.247866+0000","last_undegraded":"2024-09-10T09:54:01.247866+0000","last_fullsized":"2024-09-10T09:54:01.247866+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:31:13.490147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.247963+0000","last_change":"2024-09-10T09:52:22.330912+0000","last_active":"2024-09-10T09:54:01.247963+0000","last_peered":"2024-09-10T09:54:01.247963+0000","last_clean":"2024-09-10T09:54:01.247963+0000","last_became_active":"2024-09-10T09:52:22.330149+0000","last_became_peered":"2024-09-10T09:52:22.330149+0000","last_unstale":"2024-09-10T09:54:01.247963+0000","last_undegraded":"2024-09-10T09:54:01.247963+0000","last_fullsized":"2024-09-10T09:54:01.247963+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:10:48.842499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.955621+0000","last_change":"2024-09-10T09:52:22.327035+0000","last_active":"2024-09-10T09:54:00.955621+0000","last_peered":"2024-09-10T09:54:00.955621+0000","last_clean":"2024-09-10T09:54:00.955621+0000","last_became_active":"2024-09-10T09:52:22.326741+0000","last_became_peered":"2024-09-10T09:52:22.326741+0000","last_unstale":"2024-09-10T09:54:00.955621+0000","last_undegraded":"2024-09-10T09:54:00.955621+0000","last_fullsized":"2024-09-10T09:54:00.955621+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:21:05.026614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70210,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":603,"num_read_kb":377,"num_write":245,"num_write_kb":140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":4426,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":210630,"data_compressed":5661,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":212,"ondisk_log_size":212,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282980,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449125,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1073152,"data_stored":554558,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":70,"seq":300647710760,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32380,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712772,"statfs":{"total":95995035648,"available":95961878528,"internally_reserved":0,"allocated":1216512,"data_stored":1204181,"data_compressed":70462,"data_compressed_allocated":94208,"data_compressed_original":688128,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":65,"seq":279172874287,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32264,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712888,"statfs":{"total":95995035648,"available":95961997312,"internally_reserved":0,"allocated":1097728,"data_stored":557996,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135925,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32252,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712900,"statfs":{"total":95995035648,"available":95962009600,"internally_reserved":0,"allocated":1085440,"data_stored":557702,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364857,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28144,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717008,"statfs":{"total":95995035648,"available":95966216192,"internally_reserved":0,"allocated":1138688,"data_stored":1150582,"data_compressed":68575,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528384,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32236,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712916,"statfs":{"total":95995035648,"available":95962025984,"internally_reserved":0,"allocated":1069056,"data_stored":549404,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790022,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28152,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717000,"statfs":{"total":95995035648,"available":95966208000,"internally_reserved":0,"allocated":1146880,"data_stored":1142263,"data_compressed":68575,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953545,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28144,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717008,"statfs":{"total":95995035648,"available":95966216192,"internally_reserved":0,"allocated":1146880,"data_stored":616873,"data_compressed":34470,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182478,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32248,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712904,"statfs":{"total":95995035648,"available":95962013696,"internally_reserved":0,"allocated":1081344,"data_stored":549112,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411413,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36412,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708740,"statfs":{"total":95995035648,"available":95957749760,"internally_reserved":0,"allocated":1085440,"data_stored":551047,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574939,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28144,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717008,"statfs":{"total":95995035648,"available":95966216192,"internally_reserved":0,"allocated":1142784,"data_stored":614108,"data_compressed":34470,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738464,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1093632,"data_stored":550860,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34340,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34340,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34340,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35992,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35992,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35992,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35992,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35992,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":65426,"data_compressed":1887,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":64095,"data_compressed":1887,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":65131,"data_compressed":1887,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T09:54:18.147 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T09:54:18.514 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:54:18.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:54:18 smithi077 ceph-mon[30319]: from='client.24934 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T09:54:18.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:54:18 smithi077 ceph-mon[30319]: pgmap v365: 129 pgs: 129 active+clean; 645 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-10T09:54:19.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:19.091+0000 7f94a7200640 1 -- 172.21.15.77:0/9494938 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f94a8108f60 msgr2=0x7f94a8110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:19.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:19.091+0000 7f94a7200640 1 --2- 172.21.15.77:0/9494938 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f94a8108f60 0x7f94a8110ae0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f94a0009ca0 tx=0x7f94a002a8b0 comp rx=0 tx=0).stop 2024-09-10T09:54:19.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:19.091+0000 7f94a7200640 1 -- 172.21.15.77:0/9494938 shutdown_connections 2024-09-10T09:54:19.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:19.091+0000 7f94a7200640 1 --2- 172.21.15.77:0/9494938 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f94a8108f60 0x7f94a8110ae0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:19.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:19.091+0000 7f94a7200640 1 --2- 172.21.15.77:0/9494938 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f94a81047e0 0x7f94a8108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:19.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:19.091+0000 7f94a7200640 1 --2- 172.21.15.77:0/9494938 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94a8103e30 0x7f94a8104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:19.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:19.091+0000 7f94a7200640 1 -- 172.21.15.77:0/9494938 >> 172.21.15.77:0/9494938 conn(0x7f94a80ff8d0 msgr2=0x7f94a8101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:19.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:19.091+0000 7f94a7200640 1 -- 172.21.15.77:0/9494938 shutdown_connections 2024-09-10T09:54:19.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:19.091+0000 7f94a7200640 1 -- 172.21.15.77:0/9494938 wait complete. 2024-09-10T09:54:19.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:19.092+0000 7f94a7200640 1 Processor -- start 2024-09-10T09:54:19.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:19.092+0000 7f94a7200640 1 -- start start 2024-09-10T09:54:19.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:19.092+0000 7f94a7200640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f94a8103e30 0x7f94a819a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:19.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:19.093+0000 7f94a7200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f94a81047e0 0x7f94a819a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:19.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:19.093+0000 7f94a7200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94a8108f60 0x7f94a819ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:19.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:19.093+0000 7f94a7200640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f94a8113fc0 con 0x7f94a81047e0 2024-09-10T09:54:19.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:19.093+0000 7f94a7200640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f94a8113e40 con 0x7f94a8103e30 2024-09-10T09:54:19.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:19.093+0000 7f94a7200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f94a8114140 con 0x7f94a8108f60 2024-09-10T09:54:19.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:19.093+0000 7f94a5400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f94a81047e0 0x7f94a819a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:19.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:19.093+0000 7f94a5e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f94a8103e30 0x7f94a819a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:19.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:19.093+0000 7f94a5e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f94a8103e30 0x7f94a819a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:57488/0 (socket says 172.21.15.77:57488) 2024-09-10T09:54:19.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:19.093+0000 7f94a5400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f94a81047e0 0x7f94a819a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:35398/0 (socket says 172.21.15.77:35398) 2024-09-10T09:54:19.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:19.093+0000 7f94a5e00640 1 -- 172.21.15.77:0/2379921604 learned_addr learned my addr 172.21.15.77:0/2379921604 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:54:19.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:19.093+0000 7f94a6800640 1 --2- 172.21.15.77:0/2379921604 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94a8108f60 0x7f94a819ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:19.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:19.093+0000 7f94a5e00640 1 -- 172.21.15.77:0/2379921604 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94a8108f60 msgr2=0x7f94a819ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:19.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:19.093+0000 7f94a5e00640 1 --2- 172.21.15.77:0/2379921604 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94a8108f60 0x7f94a819ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:19.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:19.093+0000 7f94a5e00640 1 -- 172.21.15.77:0/2379921604 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f94a81047e0 msgr2=0x7f94a819a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:19.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:19.093+0000 7f94a5e00640 1 --2- 172.21.15.77:0/2379921604 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f94a81047e0 0x7f94a819a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:19.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:19.093+0000 7f94a5e00640 1 -- 172.21.15.77:0/2379921604 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94a819f3b0 con 0x7f94a8103e30 2024-09-10T09:54:19.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:19.093+0000 7f94a5400640 1 --2- 172.21.15.77:0/2379921604 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f94a81047e0 0x7f94a819a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:54:19.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:19.093+0000 7f94a6800640 1 --2- 172.21.15.77:0/2379921604 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94a8108f60 0x7f94a819ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:54:19.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:19.094+0000 7f94a5e00640 1 --2- 172.21.15.77:0/2379921604 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f94a8103e30 0x7f94a819a400 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f949800e9f0 tx=0x7f949800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:19.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:19.094+0000 7f949ea00640 1 -- 172.21.15.77:0/2379921604 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f949800cd80 con 0x7f94a8103e30 2024-09-10T09:54:19.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:19.094+0000 7f949ea00640 1 -- 172.21.15.77:0/2379921604 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9498010910 con 0x7f94a8103e30 2024-09-10T09:54:19.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:19.094+0000 7f94a7200640 1 -- 172.21.15.77:0/2379921604 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94a819f6a0 con 0x7f94a8103e30 2024-09-10T09:54:19.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:19.094+0000 7f949ea00640 1 -- 172.21.15.77:0/2379921604 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9498018990 con 0x7f94a8103e30 2024-09-10T09:54:19.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:19.094+0000 7f94a7200640 1 -- 172.21.15.77:0/2379921604 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94a81a6f80 con 0x7f94a8103e30 2024-09-10T09:54:19.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:19.095+0000 7f94a7200640 1 -- 172.21.15.77:0/2379921604 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94a8104210 con 0x7f94a8103e30 2024-09-10T09:54:19.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:19.097+0000 7f949ea00640 1 -- 172.21.15.77:0/2379921604 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f9498010ac0 con 0x7f94a8103e30 2024-09-10T09:54:19.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:19.099+0000 7f949ea00640 1 --2- 172.21.15.77:0/2379921604 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f948c0a2c10 0x7f948c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:19.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:19.099+0000 7f949ea00640 1 -- 172.21.15.77:0/2379921604 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f94980cfe20 con 0x7f94a8103e30 2024-09-10T09:54:19.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:19.099+0000 7f94a5400640 1 --2- 172.21.15.77:0/2379921604 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f948c0a2c10 0x7f948c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:19.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:19.099+0000 7f94a5400640 1 --2- 172.21.15.77:0/2379921604 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f948c0a2c10 0x7f948c0a50d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f94a810a930 tx=0x7f9494005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:19.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:19.101+0000 7f949ea00640 1 -- 172.21.15.77:0/2379921604 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f949809c7d0 con 0x7f94a8103e30 2024-09-10T09:54:19.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:54:18 smithi120 ceph-mon[34770]: from='client.24934 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T09:54:19.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:54:18 smithi120 ceph-mon[34770]: pgmap v365: 129 pgs: 129 active+clean; 645 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-10T09:54:19.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:19.274+0000 7f94a7200640 1 -- 172.21.15.77:0/2379921604 --> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f94a8062630 con 0x7f948c0a2c10 2024-09-10T09:54:19.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:19.282+0000 7f949ea00640 1 -- 172.21.15.77:0/2379921604 <== mgr.14150 v2:172.21.15.77:6800/4261324650 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340732 (secure 0 0 0) 0x7f94a8062630 con 0x7f948c0a2c10 2024-09-10T09:54:19.288 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:54:19.289 INFO:teuthology.orchestra.run.smithi077.stderr:dumped all 2024-09-10T09:54:19.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:19.288+0000 7f94a7200640 1 -- 172.21.15.77:0/2379921604 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f948c0a2c10 msgr2=0x7f948c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:19.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:19.288+0000 7f94a7200640 1 --2- 172.21.15.77:0/2379921604 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f948c0a2c10 0x7f948c0a50d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f94a810a930 tx=0x7f9494005f90 comp rx=0 tx=0).stop 2024-09-10T09:54:19.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:19.288+0000 7f94a7200640 1 -- 172.21.15.77:0/2379921604 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f94a8103e30 msgr2=0x7f94a819a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:19.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:19.288+0000 7f94a7200640 1 --2- 172.21.15.77:0/2379921604 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f94a8103e30 0x7f94a819a400 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f949800e9f0 tx=0x7f949800eeb0 comp rx=0 tx=0).stop 2024-09-10T09:54:19.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:19.288+0000 7f94a7200640 1 -- 172.21.15.77:0/2379921604 shutdown_connections 2024-09-10T09:54:19.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:19.288+0000 7f94a7200640 1 --2- 172.21.15.77:0/2379921604 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f948c0a2c10 0x7f948c0a50d0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:19.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:19.288+0000 7f94a7200640 1 --2- 172.21.15.77:0/2379921604 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94a8108f60 0x7f94a819ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:19.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:19.288+0000 7f94a7200640 1 --2- 172.21.15.77:0/2379921604 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f94a81047e0 0x7f94a819a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:19.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:19.288+0000 7f94a7200640 1 --2- 172.21.15.77:0/2379921604 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f94a8103e30 0x7f94a819a400 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:19.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:19.288+0000 7f94a7200640 1 -- 172.21.15.77:0/2379921604 >> 172.21.15.77:0/2379921604 conn(0x7f94a80ff8d0 msgr2=0x7f94a8100bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:19.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:19.288+0000 7f94a7200640 1 -- 172.21.15.77:0/2379921604 shutdown_connections 2024-09-10T09:54:19.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:19.288+0000 7f94a7200640 1 -- 172.21.15.77:0/2379921604 wait complete. 2024-09-10T09:54:19.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:54:18 smithi134 ceph-mon[35674]: from='client.24934 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T09:54:19.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:54:18 smithi134 ceph-mon[35674]: pgmap v365: 129 pgs: 129 active+clean; 645 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-10T09:54:20.031 INFO:teuthology.orchestra.run.smithi077.stdout:{"pg_ready":true,"pg_map":{"version":365,"stamp":"2024-09-10T09:54:17.701445+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660812,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":734,"num_read_kb":491,"num_write":428,"num_write_kb":3080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1185162,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":351,"ondisk_log_size":351,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":366552,"kb_used_data":13064,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124575272,"statfs":{"total":1151940427776,"available":1151565078528,"internally_reserved":0,"allocated":13377536,"data_stored":8598686,"data_compressed":504633,"data_compressed_allocated":688128,"data_compressed_original":2875392,"omap_allocated":19143,"internal_metadata":361674041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":105,"num_read_kb":55,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003215"},"pg_stats":[{"pgid":"2.1f","version":"94'13","reported_seq":126,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:14.972883+0000","last_change":"2024-09-10T09:53:00.803590+0000","last_active":"2024-09-10T09:54:14.972883+0000","last_peered":"2024-09-10T09:54:14.972883+0000","last_clean":"2024-09-10T09:54:14.972883+0000","last_became_active":"2024-09-10T09:53:00.798189+0000","last_became_peered":"2024-09-10T09:53:00.798189+0000","last_unstale":"2024-09-10T09:54:14.972883+0000","last_undegraded":"2024-09-10T09:54:14.972883+0000","last_fullsized":"2024-09-10T09:54:14.972883+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:18:48.808836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":61,"num_read_kb":61,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.846944+0000","last_change":"2024-09-10T09:52:22.318195+0000","last_active":"2024-09-10T09:54:00.846944+0000","last_peered":"2024-09-10T09:54:00.846944+0000","last_clean":"2024-09-10T09:54:00.846944+0000","last_became_active":"2024-09-10T09:52:22.315905+0000","last_became_peered":"2024-09-10T09:52:22.315905+0000","last_unstale":"2024-09-10T09:54:00.846944+0000","last_undegraded":"2024-09-10T09:54:00.846944+0000","last_fullsized":"2024-09-10T09:54:00.846944+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:53:55.515984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.843917+0000","last_change":"2024-09-10T09:52:31.369780+0000","last_active":"2024-09-10T09:54:00.843917+0000","last_peered":"2024-09-10T09:54:00.843917+0000","last_clean":"2024-09-10T09:54:00.843917+0000","last_became_active":"2024-09-10T09:52:31.369238+0000","last_became_peered":"2024-09-10T09:52:31.369238+0000","last_unstale":"2024-09-10T09:54:00.843917+0000","last_undegraded":"2024-09-10T09:54:00.843917+0000","last_fullsized":"2024-09-10T09:54:00.843917+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:39:05.318493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"94'2","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.847155+0000","last_change":"2024-09-10T09:52:19.912276+0000","last_active":"2024-09-10T09:54:00.847155+0000","last_peered":"2024-09-10T09:54:00.847155+0000","last_clean":"2024-09-10T09:54:00.847155+0000","last_became_active":"2024-09-10T09:52:19.911452+0000","last_became_peered":"2024-09-10T09:52:19.911452+0000","last_unstale":"2024-09-10T09:54:00.847155+0000","last_undegraded":"2024-09-10T09:54:00.847155+0000","last_fullsized":"2024-09-10T09:54:00.847155+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:24:31.262207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.248210+0000","last_change":"2024-09-10T09:52:22.327641+0000","last_active":"2024-09-10T09:54:01.248210+0000","last_peered":"2024-09-10T09:54:01.248210+0000","last_clean":"2024-09-10T09:54:01.248210+0000","last_became_active":"2024-09-10T09:52:22.326969+0000","last_became_peered":"2024-09-10T09:52:22.326969+0000","last_unstale":"2024-09-10T09:54:01.248210+0000","last_undegraded":"2024-09-10T09:54:01.248210+0000","last_fullsized":"2024-09-10T09:54:01.248210+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:35:27.825534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.950921+0000","last_change":"2024-09-10T09:52:31.372360+0000","last_active":"2024-09-10T09:54:00.950921+0000","last_peered":"2024-09-10T09:54:00.950921+0000","last_clean":"2024-09-10T09:54:00.950921+0000","last_became_active":"2024-09-10T09:52:31.371957+0000","last_became_peered":"2024-09-10T09:52:31.371957+0000","last_unstale":"2024-09-10T09:54:00.950921+0000","last_undegraded":"2024-09-10T09:54:00.950921+0000","last_fullsized":"2024-09-10T09:54:00.950921+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:49:34.776185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"94'12","reported_seq":53,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.149941+0000","last_change":"2024-09-10T09:52:19.843532+0000","last_active":"2024-09-10T09:54:01.149941+0000","last_peered":"2024-09-10T09:54:01.149941+0000","last_clean":"2024-09-10T09:54:01.149941+0000","last_became_active":"2024-09-10T09:52:19.843070+0000","last_became_peered":"2024-09-10T09:52:19.843070+0000","last_unstale":"2024-09-10T09:54:01.149941+0000","last_undegraded":"2024-09-10T09:54:01.149941+0000","last_fullsized":"2024-09-10T09:54:01.149941+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:08:02.223373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.988750+0000","last_change":"2024-09-10T09:52:22.316981+0000","last_active":"2024-09-10T09:54:00.988750+0000","last_peered":"2024-09-10T09:54:00.988750+0000","last_clean":"2024-09-10T09:54:00.988750+0000","last_became_active":"2024-09-10T09:52:22.316648+0000","last_became_peered":"2024-09-10T09:52:22.316648+0000","last_unstale":"2024-09-10T09:54:00.988750+0000","last_undegraded":"2024-09-10T09:54:00.988750+0000","last_fullsized":"2024-09-10T09:54:00.988750+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:28:56.742496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.951407+0000","last_change":"2024-09-10T09:52:31.376811+0000","last_active":"2024-09-10T09:54:00.951407+0000","last_peered":"2024-09-10T09:54:00.951407+0000","last_clean":"2024-09-10T09:54:00.951407+0000","last_became_active":"2024-09-10T09:52:31.376007+0000","last_became_peered":"2024-09-10T09:52:31.376007+0000","last_unstale":"2024-09-10T09:54:00.951407+0000","last_undegraded":"2024-09-10T09:54:00.951407+0000","last_fullsized":"2024-09-10T09:54:00.951407+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:29:26.213281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"94'5","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.149714+0000","last_change":"2024-09-10T09:52:19.837848+0000","last_active":"2024-09-10T09:54:01.149714+0000","last_peered":"2024-09-10T09:54:01.149714+0000","last_clean":"2024-09-10T09:54:01.149714+0000","last_became_active":"2024-09-10T09:52:19.837383+0000","last_became_peered":"2024-09-10T09:52:19.837383+0000","last_unstale":"2024-09-10T09:54:01.149714+0000","last_undegraded":"2024-09-10T09:54:01.149714+0000","last_fullsized":"2024-09-10T09:54:01.149714+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:50:42.813665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.060407+0000","last_change":"2024-09-10T09:54:01.060407+0000","last_active":"2024-09-10T09:54:01.060407+0000","last_peered":"2024-09-10T09:54:01.060407+0000","last_clean":"2024-09-10T09:54:01.060407+0000","last_became_active":"2024-09-10T09:54:01.060126+0000","last_became_peered":"2024-09-10T09:54:01.060126+0000","last_unstale":"2024-09-10T09:54:01.060407+0000","last_undegraded":"2024-09-10T09:54:01.060407+0000","last_fullsized":"2024-09-10T09:54:01.060407+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:59:29.765375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.951061+0000","last_change":"2024-09-10T09:52:31.370910+0000","last_active":"2024-09-10T09:54:00.951061+0000","last_peered":"2024-09-10T09:54:00.951061+0000","last_clean":"2024-09-10T09:54:00.951061+0000","last_became_active":"2024-09-10T09:52:31.370467+0000","last_became_peered":"2024-09-10T09:52:31.370467+0000","last_unstale":"2024-09-10T09:54:00.951061+0000","last_undegraded":"2024-09-10T09:54:00.951061+0000","last_fullsized":"2024-09-10T09:54:00.951061+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:49:25.037820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.059228+0000","last_change":"2024-09-10T09:52:22.310708+0000","last_active":"2024-09-10T09:54:01.059228+0000","last_peered":"2024-09-10T09:54:01.059228+0000","last_clean":"2024-09-10T09:54:01.059228+0000","last_became_active":"2024-09-10T09:52:22.310387+0000","last_became_peered":"2024-09-10T09:52:22.310387+0000","last_unstale":"2024-09-10T09:54:01.059228+0000","last_undegraded":"2024-09-10T09:54:01.059228+0000","last_fullsized":"2024-09-10T09:54:01.059228+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:35:03.115408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"94'4","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.149376+0000","last_change":"2024-09-10T09:52:19.841417+0000","last_active":"2024-09-10T09:54:01.149376+0000","last_peered":"2024-09-10T09:54:01.149376+0000","last_clean":"2024-09-10T09:54:01.149376+0000","last_became_active":"2024-09-10T09:52:19.841083+0000","last_became_peered":"2024-09-10T09:52:19.841083+0000","last_unstale":"2024-09-10T09:54:01.149376+0000","last_undegraded":"2024-09-10T09:54:01.149376+0000","last_fullsized":"2024-09-10T09:54:01.149376+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:01:19.008345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.951244+0000","last_change":"2024-09-10T09:52:31.367406+0000","last_active":"2024-09-10T09:54:00.951244+0000","last_peered":"2024-09-10T09:54:00.951244+0000","last_clean":"2024-09-10T09:54:00.951244+0000","last_became_active":"2024-09-10T09:52:31.366961+0000","last_became_peered":"2024-09-10T09:52:31.366961+0000","last_unstale":"2024-09-10T09:54:00.951244+0000","last_undegraded":"2024-09-10T09:54:00.951244+0000","last_fullsized":"2024-09-10T09:54:00.951244+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:04:51.714465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.249212+0000","last_change":"2024-09-10T09:54:01.249212+0000","last_active":"2024-09-10T09:54:01.249212+0000","last_peered":"2024-09-10T09:54:01.249212+0000","last_clean":"2024-09-10T09:54:01.249212+0000","last_became_active":"2024-09-10T09:54:01.248840+0000","last_became_peered":"2024-09-10T09:54:01.248840+0000","last_unstale":"2024-09-10T09:54:01.249212+0000","last_undegraded":"2024-09-10T09:54:01.249212+0000","last_fullsized":"2024-09-10T09:54:01.249212+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:39:13.167435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,10],"acting":[9,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.a","version":"94'3","reported_seq":18,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.954891+0000","last_change":"2024-09-10T09:53:00.800421+0000","last_active":"2024-09-10T09:54:00.954891+0000","last_peered":"2024-09-10T09:54:00.954891+0000","last_clean":"2024-09-10T09:54:00.954891+0000","last_became_active":"2024-09-10T09:53:00.800026+0000","last_became_peered":"2024-09-10T09:53:00.800026+0000","last_unstale":"2024-09-10T09:54:00.954891+0000","last_undegraded":"2024-09-10T09:54:00.954891+0000","last_fullsized":"2024-09-10T09:54:00.954891+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:57:45.252420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"94'1","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.951331+0000","last_change":"2024-09-10T09:52:31.376605+0000","last_active":"2024-09-10T09:54:00.951331+0000","last_peered":"2024-09-10T09:54:00.951331+0000","last_clean":"2024-09-10T09:54:00.951331+0000","last_became_active":"2024-09-10T09:52:31.376083+0000","last_became_peered":"2024-09-10T09:52:31.376083+0000","last_unstale":"2024-09-10T09:54:00.951331+0000","last_undegraded":"2024-09-10T09:54:00.951331+0000","last_fullsized":"2024-09-10T09:54:00.951331+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:05:10.030673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.954956+0000","last_change":"2024-09-10T09:52:22.327303+0000","last_active":"2024-09-10T09:54:00.954956+0000","last_peered":"2024-09-10T09:54:00.954956+0000","last_clean":"2024-09-10T09:54:00.954956+0000","last_became_active":"2024-09-10T09:52:22.326753+0000","last_became_peered":"2024-09-10T09:52:22.326753+0000","last_unstale":"2024-09-10T09:54:00.954956+0000","last_undegraded":"2024-09-10T09:54:00.954956+0000","last_fullsized":"2024-09-10T09:54:00.954956+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:27:26.668151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"94'7","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.845909+0000","last_change":"2024-09-10T09:52:19.836484+0000","last_active":"2024-09-10T09:54:00.845909+0000","last_peered":"2024-09-10T09:54:00.845909+0000","last_clean":"2024-09-10T09:54:00.845909+0000","last_became_active":"2024-09-10T09:52:19.836130+0000","last_became_peered":"2024-09-10T09:52:19.836130+0000","last_unstale":"2024-09-10T09:54:00.845909+0000","last_undegraded":"2024-09-10T09:54:00.845909+0000","last_fullsized":"2024-09-10T09:54:00.845909+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:31:58.731981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.844246+0000","last_change":"2024-09-10T09:52:31.375279+0000","last_active":"2024-09-10T09:54:00.844246+0000","last_peered":"2024-09-10T09:54:00.844246+0000","last_clean":"2024-09-10T09:54:00.844246+0000","last_became_active":"2024-09-10T09:52:31.374181+0000","last_became_peered":"2024-09-10T09:52:31.374181+0000","last_unstale":"2024-09-10T09:54:00.844246+0000","last_undegraded":"2024-09-10T09:54:00.844246+0000","last_fullsized":"2024-09-10T09:54:00.844246+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:54:51.299224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.149225+0000","last_change":"2024-09-10T09:52:22.314817+0000","last_active":"2024-09-10T09:54:01.149225+0000","last_peered":"2024-09-10T09:54:01.149225+0000","last_clean":"2024-09-10T09:54:01.149225+0000","last_became_active":"2024-09-10T09:52:22.314491+0000","last_became_peered":"2024-09-10T09:52:22.314491+0000","last_unstale":"2024-09-10T09:54:01.149225+0000","last_undegraded":"2024-09-10T09:54:01.149225+0000","last_fullsized":"2024-09-10T09:54:01.149225+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:49:41.941667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"94'2","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.988492+0000","last_change":"2024-09-10T09:52:19.911730+0000","last_active":"2024-09-10T09:54:00.988492+0000","last_peered":"2024-09-10T09:54:00.988492+0000","last_clean":"2024-09-10T09:54:00.988492+0000","last_became_active":"2024-09-10T09:52:19.911393+0000","last_became_peered":"2024-09-10T09:52:19.911393+0000","last_unstale":"2024-09-10T09:54:00.988492+0000","last_undegraded":"2024-09-10T09:54:00.988492+0000","last_fullsized":"2024-09-10T09:54:00.988492+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:46:17.943129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.951225+0000","last_change":"2024-09-10T09:52:31.376498+0000","last_active":"2024-09-10T09:54:00.951225+0000","last_peered":"2024-09-10T09:54:00.951225+0000","last_clean":"2024-09-10T09:54:00.951225+0000","last_became_active":"2024-09-10T09:52:31.375966+0000","last_became_peered":"2024-09-10T09:52:31.375966+0000","last_unstale":"2024-09-10T09:54:00.951225+0000","last_undegraded":"2024-09-10T09:54:00.951225+0000","last_fullsized":"2024-09-10T09:54:00.951225+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:08:30.649399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.248013+0000","last_change":"2024-09-10T09:52:22.331611+0000","last_active":"2024-09-10T09:54:01.248013+0000","last_peered":"2024-09-10T09:54:01.248013+0000","last_clean":"2024-09-10T09:54:01.248013+0000","last_became_active":"2024-09-10T09:52:22.331268+0000","last_became_peered":"2024-09-10T09:52:22.331268+0000","last_unstale":"2024-09-10T09:54:01.248013+0000","last_undegraded":"2024-09-10T09:54:01.248013+0000","last_fullsized":"2024-09-10T09:54:01.248013+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:01:05.433994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"94'7","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.951878+0000","last_change":"2024-09-10T09:52:19.838387+0000","last_active":"2024-09-10T09:54:00.951878+0000","last_peered":"2024-09-10T09:54:00.951878+0000","last_clean":"2024-09-10T09:54:00.951878+0000","last_became_active":"2024-09-10T09:52:19.837889+0000","last_became_peered":"2024-09-10T09:52:19.837889+0000","last_unstale":"2024-09-10T09:54:00.951878+0000","last_undegraded":"2024-09-10T09:54:00.951878+0000","last_fullsized":"2024-09-10T09:54:00.951878+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:50:00.187243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.844136+0000","last_change":"2024-09-10T09:52:31.363262+0000","last_active":"2024-09-10T09:54:00.844136+0000","last_peered":"2024-09-10T09:54:00.844136+0000","last_clean":"2024-09-10T09:54:00.844136+0000","last_became_active":"2024-09-10T09:52:31.362762+0000","last_became_peered":"2024-09-10T09:52:31.362762+0000","last_unstale":"2024-09-10T09:54:00.844136+0000","last_undegraded":"2024-09-10T09:54:00.844136+0000","last_fullsized":"2024-09-10T09:54:00.844136+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:48:28.275041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.847229+0000","last_change":"2024-09-10T09:52:22.312472+0000","last_active":"2024-09-10T09:54:00.847229+0000","last_peered":"2024-09-10T09:54:00.847229+0000","last_clean":"2024-09-10T09:54:00.847229+0000","last_became_active":"2024-09-10T09:52:22.312117+0000","last_became_peered":"2024-09-10T09:52:22.312117+0000","last_unstale":"2024-09-10T09:54:00.847229+0000","last_undegraded":"2024-09-10T09:54:00.847229+0000","last_fullsized":"2024-09-10T09:54:00.847229+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:39:10.742137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"94'14","reported_seq":54,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.844383+0000","last_change":"2024-09-10T09:52:19.835543+0000","last_active":"2024-09-10T09:54:00.844383+0000","last_peered":"2024-09-10T09:54:00.844383+0000","last_clean":"2024-09-10T09:54:00.844383+0000","last_became_active":"2024-09-10T09:52:19.835170+0000","last_became_peered":"2024-09-10T09:52:19.835170+0000","last_unstale":"2024-09-10T09:54:00.844383+0000","last_undegraded":"2024-09-10T09:54:00.844383+0000","last_fullsized":"2024-09-10T09:54:00.844383+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:37.969277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.845397+0000","last_change":"2024-09-10T09:52:31.372398+0000","last_active":"2024-09-10T09:54:00.845397+0000","last_peered":"2024-09-10T09:54:00.845397+0000","last_clean":"2024-09-10T09:54:00.845397+0000","last_became_active":"2024-09-10T09:52:31.371874+0000","last_became_peered":"2024-09-10T09:52:31.371874+0000","last_unstale":"2024-09-10T09:54:00.845397+0000","last_undegraded":"2024-09-10T09:54:00.845397+0000","last_fullsized":"2024-09-10T09:54:00.845397+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:09:45.374320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.841950+0000","last_change":"2024-09-10T09:52:22.319592+0000","last_active":"2024-09-10T09:54:00.841950+0000","last_peered":"2024-09-10T09:54:00.841950+0000","last_clean":"2024-09-10T09:54:00.841950+0000","last_became_active":"2024-09-10T09:52:22.319298+0000","last_became_peered":"2024-09-10T09:52:22.319298+0000","last_unstale":"2024-09-10T09:54:00.841950+0000","last_undegraded":"2024-09-10T09:54:00.841950+0000","last_fullsized":"2024-09-10T09:54:00.841950+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:56:59.787638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"94'4","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.149952+0000","last_change":"2024-09-10T09:52:19.912923+0000","last_active":"2024-09-10T09:54:01.149952+0000","last_peered":"2024-09-10T09:54:01.149952+0000","last_clean":"2024-09-10T09:54:01.149952+0000","last_became_active":"2024-09-10T09:52:19.912268+0000","last_became_peered":"2024-09-10T09:52:19.912268+0000","last_unstale":"2024-09-10T09:54:01.149952+0000","last_undegraded":"2024-09-10T09:54:01.149952+0000","last_fullsized":"2024-09-10T09:54:01.149952+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:06:45.421004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.955008+0000","last_change":"2024-09-10T09:52:31.372761+0000","last_active":"2024-09-10T09:54:00.955008+0000","last_peered":"2024-09-10T09:54:00.955008+0000","last_clean":"2024-09-10T09:54:00.955008+0000","last_became_active":"2024-09-10T09:52:31.372293+0000","last_became_peered":"2024-09-10T09:52:31.372293+0000","last_unstale":"2024-09-10T09:54:00.955008+0000","last_undegraded":"2024-09-10T09:54:00.955008+0000","last_fullsized":"2024-09-10T09:54:00.955008+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:36:41.446810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.841648+0000","last_change":"2024-09-10T09:52:22.326291+0000","last_active":"2024-09-10T09:54:00.841648+0000","last_peered":"2024-09-10T09:54:00.841648+0000","last_clean":"2024-09-10T09:54:00.841648+0000","last_became_active":"2024-09-10T09:52:22.325985+0000","last_became_peered":"2024-09-10T09:52:22.325985+0000","last_unstale":"2024-09-10T09:54:00.841648+0000","last_undegraded":"2024-09-10T09:54:00.841648+0000","last_fullsized":"2024-09-10T09:54:00.841648+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:06:21.871502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"94'1","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.844016+0000","last_change":"2024-09-10T09:52:19.912851+0000","last_active":"2024-09-10T09:54:00.844016+0000","last_peered":"2024-09-10T09:54:00.844016+0000","last_clean":"2024-09-10T09:54:00.844016+0000","last_became_active":"2024-09-10T09:52:19.912200+0000","last_became_peered":"2024-09-10T09:52:19.912200+0000","last_unstale":"2024-09-10T09:54:00.844016+0000","last_undegraded":"2024-09-10T09:54:00.844016+0000","last_fullsized":"2024-09-10T09:54:00.844016+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:00:47.331755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.149237+0000","last_change":"2024-09-10T09:52:31.371329+0000","last_active":"2024-09-10T09:54:01.149237+0000","last_peered":"2024-09-10T09:54:01.149237+0000","last_clean":"2024-09-10T09:54:01.149237+0000","last_became_active":"2024-09-10T09:52:31.370971+0000","last_became_peered":"2024-09-10T09:52:31.370971+0000","last_unstale":"2024-09-10T09:54:01.149237+0000","last_undegraded":"2024-09-10T09:54:01.149237+0000","last_fullsized":"2024-09-10T09:54:01.149237+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:07:12.928421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.841648+0000","last_change":"2024-09-10T09:52:22.328039+0000","last_active":"2024-09-10T09:54:00.841648+0000","last_peered":"2024-09-10T09:54:00.841648+0000","last_clean":"2024-09-10T09:54:00.841648+0000","last_became_active":"2024-09-10T09:52:22.327588+0000","last_became_peered":"2024-09-10T09:52:22.327588+0000","last_unstale":"2024-09-10T09:54:00.841648+0000","last_undegraded":"2024-09-10T09:54:00.841648+0000","last_fullsized":"2024-09-10T09:54:00.841648+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:35:39.219544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"94'5","reported_seq":77,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:10.968454+0000","last_change":"2024-09-10T09:53:00.797651+0000","last_active":"2024-09-10T09:54:10.968454+0000","last_peered":"2024-09-10T09:54:10.968454+0000","last_clean":"2024-09-10T09:54:10.968454+0000","last_became_active":"2024-09-10T09:53:00.797285+0000","last_became_peered":"2024-09-10T09:53:00.797285+0000","last_unstale":"2024-09-10T09:54:10.968454+0000","last_undegraded":"2024-09-10T09:54:10.968454+0000","last_fullsized":"2024-09-10T09:54:10.968454+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:18:02.320032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":57,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,6,1],"acting":[11,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.841739+0000","last_change":"2024-09-10T09:52:31.379215+0000","last_active":"2024-09-10T09:54:00.841739+0000","last_peered":"2024-09-10T09:54:00.841739+0000","last_clean":"2024-09-10T09:54:00.841739+0000","last_became_active":"2024-09-10T09:52:31.377107+0000","last_became_peered":"2024-09-10T09:52:31.377107+0000","last_unstale":"2024-09-10T09:54:00.841739+0000","last_undegraded":"2024-09-10T09:54:00.841739+0000","last_fullsized":"2024-09-10T09:54:00.841739+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:09:39.474077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.847190+0000","last_change":"2024-09-10T09:52:22.316474+0000","last_active":"2024-09-10T09:54:00.847190+0000","last_peered":"2024-09-10T09:54:00.847190+0000","last_clean":"2024-09-10T09:54:00.847190+0000","last_became_active":"2024-09-10T09:52:22.316054+0000","last_became_peered":"2024-09-10T09:52:22.316054+0000","last_unstale":"2024-09-10T09:54:00.847190+0000","last_undegraded":"2024-09-10T09:54:00.847190+0000","last_fullsized":"2024-09-10T09:54:00.847190+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:11:31.334780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"94'5","reported_seq":104,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:11.969631+0000","last_change":"2024-09-10T09:52:19.911601+0000","last_active":"2024-09-10T09:54:11.969631+0000","last_peered":"2024-09-10T09:54:11.969631+0000","last_clean":"2024-09-10T09:54:11.969631+0000","last_became_active":"2024-09-10T09:52:19.911268+0000","last_became_peered":"2024-09-10T09:52:19.911268+0000","last_unstale":"2024-09-10T09:54:11.969631+0000","last_undegraded":"2024-09-10T09:54:11.969631+0000","last_fullsized":"2024-09-10T09:54:11.969631+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:46:38.158784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":58,"num_read_kb":58,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.841751+0000","last_change":"2024-09-10T09:52:31.375938+0000","last_active":"2024-09-10T09:54:00.841751+0000","last_peered":"2024-09-10T09:54:00.841751+0000","last_clean":"2024-09-10T09:54:00.841751+0000","last_became_active":"2024-09-10T09:52:31.375536+0000","last_became_peered":"2024-09-10T09:52:31.375536+0000","last_unstale":"2024-09-10T09:54:00.841751+0000","last_undegraded":"2024-09-10T09:54:00.841751+0000","last_fullsized":"2024-09-10T09:54:00.841751+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:55:17.303809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.840977+0000","last_change":"2024-09-10T09:52:22.315912+0000","last_active":"2024-09-10T09:54:00.840977+0000","last_peered":"2024-09-10T09:54:00.840977+0000","last_clean":"2024-09-10T09:54:00.840977+0000","last_became_active":"2024-09-10T09:52:22.315433+0000","last_became_peered":"2024-09-10T09:52:22.315433+0000","last_unstale":"2024-09-10T09:54:00.840977+0000","last_undegraded":"2024-09-10T09:54:00.840977+0000","last_fullsized":"2024-09-10T09:54:00.840977+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:57:10.454716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"94'4","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.059073+0000","last_change":"2024-09-10T09:52:19.838966+0000","last_active":"2024-09-10T09:54:01.059073+0000","last_peered":"2024-09-10T09:54:01.059073+0000","last_clean":"2024-09-10T09:54:01.059073+0000","last_became_active":"2024-09-10T09:52:19.838545+0000","last_became_peered":"2024-09-10T09:52:19.838545+0000","last_unstale":"2024-09-10T09:54:01.059073+0000","last_undegraded":"2024-09-10T09:54:01.059073+0000","last_fullsized":"2024-09-10T09:54:01.059073+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:07:00.930146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.847547+0000","last_change":"2024-09-10T09:53:00.802039+0000","last_active":"2024-09-10T09:54:00.847547+0000","last_peered":"2024-09-10T09:54:00.847547+0000","last_clean":"2024-09-10T09:54:00.847547+0000","last_became_active":"2024-09-10T09:53:00.800223+0000","last_became_peered":"2024-09-10T09:53:00.800223+0000","last_unstale":"2024-09-10T09:54:00.847547+0000","last_undegraded":"2024-09-10T09:54:00.847547+0000","last_fullsized":"2024-09-10T09:54:00.847547+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:50:16.026631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,6,10],"acting":[5,8,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.846149+0000","last_change":"2024-09-10T09:52:22.313819+0000","last_active":"2024-09-10T09:54:00.846149+0000","last_peered":"2024-09-10T09:54:00.846149+0000","last_clean":"2024-09-10T09:54:00.846149+0000","last_became_active":"2024-09-10T09:52:22.313473+0000","last_became_peered":"2024-09-10T09:52:22.313473+0000","last_unstale":"2024-09-10T09:54:00.846149+0000","last_undegraded":"2024-09-10T09:54:00.846149+0000","last_fullsized":"2024-09-10T09:54:00.846149+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:44:10.003187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"94'6","reported_seq":50,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.951708+0000","last_change":"2024-09-10T09:52:19.837376+0000","last_active":"2024-09-10T09:54:00.951708+0000","last_peered":"2024-09-10T09:54:00.951708+0000","last_clean":"2024-09-10T09:54:00.951708+0000","last_became_active":"2024-09-10T09:52:19.836880+0000","last_became_peered":"2024-09-10T09:52:19.836880+0000","last_unstale":"2024-09-10T09:54:00.951708+0000","last_undegraded":"2024-09-10T09:54:00.951708+0000","last_fullsized":"2024-09-10T09:54:00.951708+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:48:58.198188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":361,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.059392+0000","last_change":"2024-09-10T09:51:05.871529+0000","last_active":"2024-09-10T09:54:01.059392+0000","last_peered":"2024-09-10T09:54:01.059392+0000","last_clean":"2024-09-10T09:54:01.059392+0000","last_became_active":"2024-09-10T09:51:05.871115+0000","last_became_peered":"2024-09-10T09:51:05.871115+0000","last_unstale":"2024-09-10T09:54:01.059392+0000","last_undegraded":"2024-09-10T09:54:01.059392+0000","last_fullsized":"2024-09-10T09:54:01.059392+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:47:24.175358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:47:24.175358+0000","last_clean_scrub_stamp":"2024-09-10T09:47:24.175358+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:58:30.247077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"94'1","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.059487+0000","last_change":"2024-09-10T09:53:00.800474+0000","last_active":"2024-09-10T09:54:01.059487+0000","last_peered":"2024-09-10T09:54:01.059487+0000","last_clean":"2024-09-10T09:54:01.059487+0000","last_became_active":"2024-09-10T09:53:00.799932+0000","last_became_peered":"2024-09-10T09:53:00.799932+0000","last_unstale":"2024-09-10T09:54:01.059487+0000","last_undegraded":"2024-09-10T09:54:01.059487+0000","last_fullsized":"2024-09-10T09:54:01.059487+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:14:50.998952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,11,9],"acting":[7,8,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.059603+0000","last_change":"2024-09-10T09:52:22.312521+0000","last_active":"2024-09-10T09:54:01.059603+0000","last_peered":"2024-09-10T09:54:01.059603+0000","last_clean":"2024-09-10T09:54:01.059603+0000","last_became_active":"2024-09-10T09:52:22.312107+0000","last_became_peered":"2024-09-10T09:52:22.312107+0000","last_unstale":"2024-09-10T09:54:01.059603+0000","last_undegraded":"2024-09-10T09:54:01.059603+0000","last_fullsized":"2024-09-10T09:54:01.059603+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:14:19.799259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"94'5","reported_seq":105,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:11.965971+0000","last_change":"2024-09-10T09:52:19.911454+0000","last_active":"2024-09-10T09:54:11.965971+0000","last_peered":"2024-09-10T09:54:11.965971+0000","last_clean":"2024-09-10T09:54:11.965971+0000","last_became_active":"2024-09-10T09:52:19.911079+0000","last_became_peered":"2024-09-10T09:52:19.911079+0000","last_unstale":"2024-09-10T09:54:11.965971+0000","last_undegraded":"2024-09-10T09:54:11.965971+0000","last_fullsized":"2024-09-10T09:54:11.965971+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:43:38.787689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":59,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.988483+0000","last_change":"2024-09-10T09:52:31.381877+0000","last_active":"2024-09-10T09:54:00.988483+0000","last_peered":"2024-09-10T09:54:00.988483+0000","last_clean":"2024-09-10T09:54:00.988483+0000","last_became_active":"2024-09-10T09:52:31.381472+0000","last_became_peered":"2024-09-10T09:52:31.381472+0000","last_unstale":"2024-09-10T09:54:00.988483+0000","last_undegraded":"2024-09-10T09:54:00.988483+0000","last_fullsized":"2024-09-10T09:54:00.988483+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:41:24.686325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.059412+0000","last_change":"2024-09-10T09:52:22.313902+0000","last_active":"2024-09-10T09:54:01.059412+0000","last_peered":"2024-09-10T09:54:01.059412+0000","last_clean":"2024-09-10T09:54:01.059412+0000","last_became_active":"2024-09-10T09:52:22.313515+0000","last_became_peered":"2024-09-10T09:52:22.313515+0000","last_unstale":"2024-09-10T09:54:01.059412+0000","last_undegraded":"2024-09-10T09:54:01.059412+0000","last_fullsized":"2024-09-10T09:54:01.059412+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:25:00.238307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"94'3","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.847808+0000","last_change":"2024-09-10T09:52:19.837736+0000","last_active":"2024-09-10T09:54:00.847808+0000","last_peered":"2024-09-10T09:54:00.847808+0000","last_clean":"2024-09-10T09:54:00.847808+0000","last_became_active":"2024-09-10T09:52:19.837370+0000","last_became_peered":"2024-09-10T09:52:19.837370+0000","last_unstale":"2024-09-10T09:54:00.847808+0000","last_undegraded":"2024-09-10T09:54:00.847808+0000","last_fullsized":"2024-09-10T09:54:00.847808+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:54:35.753027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.248170+0000","last_change":"2024-09-10T09:52:31.378945+0000","last_active":"2024-09-10T09:54:01.248170+0000","last_peered":"2024-09-10T09:54:01.248170+0000","last_clean":"2024-09-10T09:54:01.248170+0000","last_became_active":"2024-09-10T09:52:31.378144+0000","last_became_peered":"2024-09-10T09:52:31.378144+0000","last_unstale":"2024-09-10T09:54:01.248170+0000","last_undegraded":"2024-09-10T09:54:01.248170+0000","last_fullsized":"2024-09-10T09:54:01.248170+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:36:06.288293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.059676+0000","last_change":"2024-09-10T09:52:22.316400+0000","last_active":"2024-09-10T09:54:01.059676+0000","last_peered":"2024-09-10T09:54:01.059676+0000","last_clean":"2024-09-10T09:54:01.059676+0000","last_became_active":"2024-09-10T09:52:22.315974+0000","last_became_peered":"2024-09-10T09:52:22.315974+0000","last_unstale":"2024-09-10T09:54:01.059676+0000","last_undegraded":"2024-09-10T09:54:01.059676+0000","last_fullsized":"2024-09-10T09:54:01.059676+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:31:05.538237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"94'5","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.841919+0000","last_change":"2024-09-10T09:52:19.837942+0000","last_active":"2024-09-10T09:54:00.841919+0000","last_peered":"2024-09-10T09:54:00.841919+0000","last_clean":"2024-09-10T09:54:00.841919+0000","last_became_active":"2024-09-10T09:52:19.837629+0000","last_became_peered":"2024-09-10T09:52:19.837629+0000","last_unstale":"2024-09-10T09:54:00.841919+0000","last_undegraded":"2024-09-10T09:54:00.841919+0000","last_fullsized":"2024-09-10T09:54:00.841919+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:34:13.207770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.846958+0000","last_change":"2024-09-10T09:52:31.364231+0000","last_active":"2024-09-10T09:54:00.846958+0000","last_peered":"2024-09-10T09:54:00.846958+0000","last_clean":"2024-09-10T09:54:00.846958+0000","last_became_active":"2024-09-10T09:52:31.363833+0000","last_became_peered":"2024-09-10T09:52:31.363833+0000","last_unstale":"2024-09-10T09:54:00.846958+0000","last_undegraded":"2024-09-10T09:54:00.846958+0000","last_fullsized":"2024-09-10T09:54:00.846958+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:53:51.435700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.955436+0000","last_change":"2024-09-10T09:52:22.319530+0000","last_active":"2024-09-10T09:54:00.955436+0000","last_peered":"2024-09-10T09:54:00.955436+0000","last_clean":"2024-09-10T09:54:00.955436+0000","last_became_active":"2024-09-10T09:52:22.318742+0000","last_became_peered":"2024-09-10T09:52:22.318742+0000","last_unstale":"2024-09-10T09:54:00.955436+0000","last_undegraded":"2024-09-10T09:54:00.955436+0000","last_fullsized":"2024-09-10T09:54:00.955436+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:44:35.050454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"94'2","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.149987+0000","last_change":"2024-09-10T09:52:19.836398+0000","last_active":"2024-09-10T09:54:01.149987+0000","last_peered":"2024-09-10T09:54:01.149987+0000","last_clean":"2024-09-10T09:54:01.149987+0000","last_became_active":"2024-09-10T09:52:19.836055+0000","last_became_peered":"2024-09-10T09:52:19.836055+0000","last_unstale":"2024-09-10T09:54:01.149987+0000","last_undegraded":"2024-09-10T09:54:01.149987+0000","last_fullsized":"2024-09-10T09:54:01.149987+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:49:26.923367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.847151+0000","last_change":"2024-09-10T09:52:31.371109+0000","last_active":"2024-09-10T09:54:00.847151+0000","last_peered":"2024-09-10T09:54:00.847151+0000","last_clean":"2024-09-10T09:54:00.847151+0000","last_became_active":"2024-09-10T09:52:31.370592+0000","last_became_peered":"2024-09-10T09:52:31.370592+0000","last_unstale":"2024-09-10T09:54:00.847151+0000","last_undegraded":"2024-09-10T09:54:00.847151+0000","last_fullsized":"2024-09-10T09:54:00.847151+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:20:23.585187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.847241+0000","last_change":"2024-09-10T09:52:22.319365+0000","last_active":"2024-09-10T09:54:00.847241+0000","last_peered":"2024-09-10T09:54:00.847241+0000","last_clean":"2024-09-10T09:54:00.847241+0000","last_became_active":"2024-09-10T09:52:22.319064+0000","last_became_peered":"2024-09-10T09:52:22.319064+0000","last_unstale":"2024-09-10T09:54:00.847241+0000","last_undegraded":"2024-09-10T09:54:00.847241+0000","last_fullsized":"2024-09-10T09:54:00.847241+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:14:50.089566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"94'17","reported_seq":58,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.149249+0000","last_change":"2024-09-10T09:52:19.911698+0000","last_active":"2024-09-10T09:54:01.149249+0000","last_peered":"2024-09-10T09:54:01.149249+0000","last_clean":"2024-09-10T09:54:01.149249+0000","last_became_active":"2024-09-10T09:52:19.911369+0000","last_became_peered":"2024-09-10T09:52:19.911369+0000","last_unstale":"2024-09-10T09:54:01.149249+0000","last_undegraded":"2024-09-10T09:54:01.149249+0000","last_fullsized":"2024-09-10T09:54:01.149249+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:40:39.045797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.841907+0000","last_change":"2024-09-10T09:52:31.380463+0000","last_active":"2024-09-10T09:54:00.841907+0000","last_peered":"2024-09-10T09:54:00.841907+0000","last_clean":"2024-09-10T09:54:00.841907+0000","last_became_active":"2024-09-10T09:52:31.380114+0000","last_became_peered":"2024-09-10T09:52:31.380114+0000","last_unstale":"2024-09-10T09:54:00.841907+0000","last_undegraded":"2024-09-10T09:54:00.841907+0000","last_fullsized":"2024-09-10T09:54:00.841907+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:19:50.869160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.058932+0000","last_change":"2024-09-10T09:52:22.313928+0000","last_active":"2024-09-10T09:54:01.058932+0000","last_peered":"2024-09-10T09:54:01.058932+0000","last_clean":"2024-09-10T09:54:01.058932+0000","last_became_active":"2024-09-10T09:52:22.313510+0000","last_became_peered":"2024-09-10T09:52:22.313510+0000","last_unstale":"2024-09-10T09:54:01.058932+0000","last_undegraded":"2024-09-10T09:54:01.058932+0000","last_fullsized":"2024-09-10T09:54:01.058932+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:58:19.031178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"94'6","reported_seq":107,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:11.970409+0000","last_change":"2024-09-10T09:52:19.839451+0000","last_active":"2024-09-10T09:54:11.970409+0000","last_peered":"2024-09-10T09:54:11.970409+0000","last_clean":"2024-09-10T09:54:11.970409+0000","last_became_active":"2024-09-10T09:52:19.839124+0000","last_became_peered":"2024-09-10T09:52:19.839124+0000","last_unstale":"2024-09-10T09:54:11.970409+0000","last_undegraded":"2024-09-10T09:54:11.970409+0000","last_fullsized":"2024-09-10T09:54:11.970409+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:51:33.489830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":61,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.844303+0000","last_change":"2024-09-10T09:52:31.376386+0000","last_active":"2024-09-10T09:54:00.844303+0000","last_peered":"2024-09-10T09:54:00.844303+0000","last_clean":"2024-09-10T09:54:00.844303+0000","last_became_active":"2024-09-10T09:52:31.376066+0000","last_became_peered":"2024-09-10T09:52:31.376066+0000","last_unstale":"2024-09-10T09:54:00.844303+0000","last_undegraded":"2024-09-10T09:54:00.844303+0000","last_fullsized":"2024-09-10T09:54:00.844303+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:23:07.191842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.059245+0000","last_change":"2024-09-10T09:52:22.323590+0000","last_active":"2024-09-10T09:54:01.059245+0000","last_peered":"2024-09-10T09:54:01.059245+0000","last_clean":"2024-09-10T09:54:01.059245+0000","last_became_active":"2024-09-10T09:52:22.323264+0000","last_became_peered":"2024-09-10T09:52:22.323264+0000","last_unstale":"2024-09-10T09:54:01.059245+0000","last_undegraded":"2024-09-10T09:54:01.059245+0000","last_fullsized":"2024-09-10T09:54:01.059245+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:07:56.572337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"94'5","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.847453+0000","last_change":"2024-09-10T09:52:19.840620+0000","last_active":"2024-09-10T09:54:00.847453+0000","last_peered":"2024-09-10T09:54:00.847453+0000","last_clean":"2024-09-10T09:54:00.847453+0000","last_became_active":"2024-09-10T09:52:19.840217+0000","last_became_peered":"2024-09-10T09:52:19.840217+0000","last_unstale":"2024-09-10T09:54:00.847453+0000","last_undegraded":"2024-09-10T09:54:00.847453+0000","last_fullsized":"2024-09-10T09:54:00.847453+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:55:13.111993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.955012+0000","last_change":"2024-09-10T09:52:31.373895+0000","last_active":"2024-09-10T09:54:00.955012+0000","last_peered":"2024-09-10T09:54:00.955012+0000","last_clean":"2024-09-10T09:54:00.955012+0000","last_became_active":"2024-09-10T09:52:31.373474+0000","last_became_peered":"2024-09-10T09:52:31.373474+0000","last_unstale":"2024-09-10T09:54:00.955012+0000","last_undegraded":"2024-09-10T09:54:00.955012+0000","last_fullsized":"2024-09-10T09:54:00.955012+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:39:16.049859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.841146+0000","last_change":"2024-09-10T09:52:22.319294+0000","last_active":"2024-09-10T09:54:00.841146+0000","last_peered":"2024-09-10T09:54:00.841146+0000","last_clean":"2024-09-10T09:54:00.841146+0000","last_became_active":"2024-09-10T09:52:22.318918+0000","last_became_peered":"2024-09-10T09:52:22.318918+0000","last_unstale":"2024-09-10T09:54:00.841146+0000","last_undegraded":"2024-09-10T09:54:00.841146+0000","last_fullsized":"2024-09-10T09:54:00.841146+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:52:46.434182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"94'9","reported_seq":52,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.844181+0000","last_change":"2024-09-10T09:52:19.842286+0000","last_active":"2024-09-10T09:54:00.844181+0000","last_peered":"2024-09-10T09:54:00.844181+0000","last_clean":"2024-09-10T09:54:00.844181+0000","last_became_active":"2024-09-10T09:52:19.841930+0000","last_became_peered":"2024-09-10T09:52:19.841930+0000","last_unstale":"2024-09-10T09:54:00.844181+0000","last_undegraded":"2024-09-10T09:54:00.844181+0000","last_fullsized":"2024-09-10T09:54:00.844181+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:38:46.358389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.847452+0000","last_change":"2024-09-10T09:52:31.370961+0000","last_active":"2024-09-10T09:54:00.847452+0000","last_peered":"2024-09-10T09:54:00.847452+0000","last_clean":"2024-09-10T09:54:00.847452+0000","last_became_active":"2024-09-10T09:52:31.370545+0000","last_became_peered":"2024-09-10T09:52:31.370545+0000","last_unstale":"2024-09-10T09:54:00.847452+0000","last_undegraded":"2024-09-10T09:54:00.847452+0000","last_fullsized":"2024-09-10T09:54:00.847452+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:17.461447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.149789+0000","last_change":"2024-09-10T09:52:22.313215+0000","last_active":"2024-09-10T09:54:01.149789+0000","last_peered":"2024-09-10T09:54:01.149789+0000","last_clean":"2024-09-10T09:54:01.149789+0000","last_became_active":"2024-09-10T09:52:22.312918+0000","last_became_peered":"2024-09-10T09:52:22.312918+0000","last_unstale":"2024-09-10T09:54:01.149789+0000","last_undegraded":"2024-09-10T09:54:01.149789+0000","last_fullsized":"2024-09-10T09:54:01.149789+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:48:09.568004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"94'34","reported_seq":212,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:11.965210+0000","last_change":"2024-09-10T09:53:01.226146+0000","last_active":"2024-09-10T09:54:11.965210+0000","last_peered":"2024-09-10T09:54:11.965210+0000","last_clean":"2024-09-10T09:54:11.965210+0000","last_became_active":"2024-09-10T09:53:01.218622+0000","last_became_peered":"2024-09-10T09:53:01.218622+0000","last_unstale":"2024-09-10T09:54:11.965210+0000","last_undegraded":"2024-09-10T09:54:11.965210+0000","last_fullsized":"2024-09-10T09:54:11.965210+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:43:11.927029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63002,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":120,"num_read_kb":121,"num_write":36,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":3844,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":16,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.841392+0000","last_change":"2024-09-10T09:53:00.800650+0000","last_active":"2024-09-10T09:54:00.841392+0000","last_peered":"2024-09-10T09:54:00.841392+0000","last_clean":"2024-09-10T09:54:00.841392+0000","last_became_active":"2024-09-10T09:53:00.800104+0000","last_became_peered":"2024-09-10T09:53:00.800104+0000","last_unstale":"2024-09-10T09:54:00.841392+0000","last_undegraded":"2024-09-10T09:54:00.841392+0000","last_fullsized":"2024-09-10T09:54:00.841392+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:41:16.022884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.988810+0000","last_change":"2024-09-10T09:52:22.317174+0000","last_active":"2024-09-10T09:54:00.988810+0000","last_peered":"2024-09-10T09:54:00.988810+0000","last_clean":"2024-09-10T09:54:00.988810+0000","last_became_active":"2024-09-10T09:52:22.316851+0000","last_became_peered":"2024-09-10T09:52:22.316851+0000","last_unstale":"2024-09-10T09:54:00.988810+0000","last_undegraded":"2024-09-10T09:54:00.988810+0000","last_fullsized":"2024-09-10T09:54:00.988810+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:55:18.742393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"94'5","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.847663+0000","last_change":"2024-09-10T09:52:19.912093+0000","last_active":"2024-09-10T09:54:00.847663+0000","last_peered":"2024-09-10T09:54:00.847663+0000","last_clean":"2024-09-10T09:54:00.847663+0000","last_became_active":"2024-09-10T09:52:19.911611+0000","last_became_peered":"2024-09-10T09:52:19.911611+0000","last_unstale":"2024-09-10T09:54:00.847663+0000","last_undegraded":"2024-09-10T09:54:00.847663+0000","last_fullsized":"2024-09-10T09:54:00.847663+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:01:10.859310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.951597+0000","last_change":"2024-09-10T09:52:31.362884+0000","last_active":"2024-09-10T09:54:00.951597+0000","last_peered":"2024-09-10T09:54:00.951597+0000","last_clean":"2024-09-10T09:54:00.951597+0000","last_became_active":"2024-09-10T09:52:31.362489+0000","last_became_peered":"2024-09-10T09:52:31.362489+0000","last_unstale":"2024-09-10T09:54:00.951597+0000","last_undegraded":"2024-09-10T09:54:00.951597+0000","last_fullsized":"2024-09-10T09:54:00.951597+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:02:33.084795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.248236+0000","last_change":"2024-09-10T09:52:22.325871+0000","last_active":"2024-09-10T09:54:01.248236+0000","last_peered":"2024-09-10T09:54:01.248236+0000","last_clean":"2024-09-10T09:54:01.248236+0000","last_became_active":"2024-09-10T09:52:22.324980+0000","last_became_peered":"2024-09-10T09:52:22.324980+0000","last_unstale":"2024-09-10T09:54:01.248236+0000","last_undegraded":"2024-09-10T09:54:01.248236+0000","last_fullsized":"2024-09-10T09:54:01.248236+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:31:55.713444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"94'4","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.059600+0000","last_change":"2024-09-10T09:52:19.839491+0000","last_active":"2024-09-10T09:54:01.059600+0000","last_peered":"2024-09-10T09:54:01.059600+0000","last_clean":"2024-09-10T09:54:01.059600+0000","last_became_active":"2024-09-10T09:52:19.839092+0000","last_became_peered":"2024-09-10T09:52:19.839092+0000","last_unstale":"2024-09-10T09:54:01.059600+0000","last_undegraded":"2024-09-10T09:54:01.059600+0000","last_fullsized":"2024-09-10T09:54:01.059600+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:42:13.453281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.843235+0000","last_change":"2024-09-10T09:52:31.371346+0000","last_active":"2024-09-10T09:54:00.843235+0000","last_peered":"2024-09-10T09:54:00.843235+0000","last_clean":"2024-09-10T09:54:00.843235+0000","last_became_active":"2024-09-10T09:52:31.370929+0000","last_became_peered":"2024-09-10T09:52:31.370929+0000","last_unstale":"2024-09-10T09:54:00.843235+0000","last_undegraded":"2024-09-10T09:54:00.843235+0000","last_fullsized":"2024-09-10T09:54:00.843235+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:44:22.925082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.988772+0000","last_change":"2024-09-10T09:52:22.317453+0000","last_active":"2024-09-10T09:54:00.988772+0000","last_peered":"2024-09-10T09:54:00.988772+0000","last_clean":"2024-09-10T09:54:00.988772+0000","last_became_active":"2024-09-10T09:52:22.315556+0000","last_became_peered":"2024-09-10T09:52:22.315556+0000","last_unstale":"2024-09-10T09:54:00.988772+0000","last_undegraded":"2024-09-10T09:54:00.988772+0000","last_fullsized":"2024-09-10T09:54:00.988772+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:56:05.860275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"94'4","reported_seq":158,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:11.965187+0000","last_change":"2024-09-10T09:52:19.835985+0000","last_active":"2024-09-10T09:54:11.965187+0000","last_peered":"2024-09-10T09:54:11.965187+0000","last_clean":"2024-09-10T09:54:11.965187+0000","last_became_active":"2024-09-10T09:52:19.834835+0000","last_became_peered":"2024-09-10T09:52:19.834835+0000","last_unstale":"2024-09-10T09:54:11.965187+0000","last_undegraded":"2024-09-10T09:54:11.965187+0000","last_fullsized":"2024-09-10T09:54:11.965187+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:09:11.590443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":112,"num_read_kb":112,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.845677+0000","last_change":"2024-09-10T09:52:31.368663+0000","last_active":"2024-09-10T09:54:00.845677+0000","last_peered":"2024-09-10T09:54:00.845677+0000","last_clean":"2024-09-10T09:54:00.845677+0000","last_became_active":"2024-09-10T09:52:31.368004+0000","last_became_peered":"2024-09-10T09:52:31.368004+0000","last_unstale":"2024-09-10T09:54:00.845677+0000","last_undegraded":"2024-09-10T09:54:00.845677+0000","last_fullsized":"2024-09-10T09:54:00.845677+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:56:15.100515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.247874+0000","last_change":"2024-09-10T09:52:31.370992+0000","last_active":"2024-09-10T09:54:01.247874+0000","last_peered":"2024-09-10T09:54:01.247874+0000","last_clean":"2024-09-10T09:54:01.247874+0000","last_became_active":"2024-09-10T09:52:31.370357+0000","last_became_peered":"2024-09-10T09:52:31.370357+0000","last_unstale":"2024-09-10T09:54:01.247874+0000","last_undegraded":"2024-09-10T09:54:01.247874+0000","last_fullsized":"2024-09-10T09:54:01.247874+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:02:49.482532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.843524+0000","last_change":"2024-09-10T09:52:22.319363+0000","last_active":"2024-09-10T09:54:00.843524+0000","last_peered":"2024-09-10T09:54:00.843524+0000","last_clean":"2024-09-10T09:54:00.843524+0000","last_became_active":"2024-09-10T09:52:22.319044+0000","last_became_peered":"2024-09-10T09:52:22.319044+0000","last_unstale":"2024-09-10T09:54:00.843524+0000","last_undegraded":"2024-09-10T09:54:00.843524+0000","last_fullsized":"2024-09-10T09:54:00.843524+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:30:51.328413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"94'3","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.847564+0000","last_change":"2024-09-10T09:52:19.841745+0000","last_active":"2024-09-10T09:54:00.847564+0000","last_peered":"2024-09-10T09:54:00.847564+0000","last_clean":"2024-09-10T09:54:00.847564+0000","last_became_active":"2024-09-10T09:52:19.841404+0000","last_became_peered":"2024-09-10T09:52:19.841404+0000","last_unstale":"2024-09-10T09:54:00.847564+0000","last_undegraded":"2024-09-10T09:54:00.847564+0000","last_fullsized":"2024-09-10T09:54:00.847564+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:44:36.346785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.955474+0000","last_change":"2024-09-10T09:53:01.218412+0000","last_active":"2024-09-10T09:54:00.955474+0000","last_peered":"2024-09-10T09:54:00.955474+0000","last_clean":"2024-09-10T09:54:00.955474+0000","last_became_active":"2024-09-10T09:53:01.217966+0000","last_became_peered":"2024-09-10T09:53:01.217966+0000","last_unstale":"2024-09-10T09:54:00.955474+0000","last_undegraded":"2024-09-10T09:54:00.955474+0000","last_fullsized":"2024-09-10T09:54:00.955474+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:53:25.743913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.845534+0000","last_change":"2024-09-10T09:52:22.316120+0000","last_active":"2024-09-10T09:54:00.845534+0000","last_peered":"2024-09-10T09:54:00.845534+0000","last_clean":"2024-09-10T09:54:00.845534+0000","last_became_active":"2024-09-10T09:52:22.315769+0000","last_became_peered":"2024-09-10T09:52:22.315769+0000","last_unstale":"2024-09-10T09:54:00.845534+0000","last_undegraded":"2024-09-10T09:54:00.845534+0000","last_fullsized":"2024-09-10T09:54:00.845534+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:16:41.866587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"94'3","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.840993+0000","last_change":"2024-09-10T09:52:19.835038+0000","last_active":"2024-09-10T09:54:00.840993+0000","last_peered":"2024-09-10T09:54:00.840993+0000","last_clean":"2024-09-10T09:54:00.840993+0000","last_became_active":"2024-09-10T09:52:19.834329+0000","last_became_peered":"2024-09-10T09:52:19.834329+0000","last_unstale":"2024-09-10T09:54:00.840993+0000","last_undegraded":"2024-09-10T09:54:00.840993+0000","last_fullsized":"2024-09-10T09:54:00.840993+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:58:24.084373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"94'7","reported_seq":55,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.841698+0000","last_change":"2024-09-10T09:53:01.219122+0000","last_active":"2024-09-10T09:54:00.841698+0000","last_peered":"2024-09-10T09:54:00.841698+0000","last_clean":"2024-09-10T09:54:00.841698+0000","last_became_active":"2024-09-10T09:53:01.218599+0000","last_became_peered":"2024-09-10T09:53:01.218599+0000","last_unstale":"2024-09-10T09:54:00.841698+0000","last_undegraded":"2024-09-10T09:54:00.841698+0000","last_fullsized":"2024-09-10T09:54:00.841698+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:31:10.092264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.953227+0000","last_change":"2024-09-10T09:54:00.952617+0000","last_active":"2024-09-10T09:54:00.953227+0000","last_peered":"2024-09-10T09:54:00.953227+0000","last_clean":"2024-09-10T09:54:00.953227+0000","last_became_active":"2024-09-10T09:54:00.952202+0000","last_became_peered":"2024-09-10T09:54:00.952202+0000","last_unstale":"2024-09-10T09:54:00.953227+0000","last_undegraded":"2024-09-10T09:54:00.953227+0000","last_fullsized":"2024-09-10T09:54:00.953227+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:35:40.179591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.844105+0000","last_change":"2024-09-10T09:52:31.371612+0000","last_active":"2024-09-10T09:54:00.844105+0000","last_peered":"2024-09-10T09:54:00.844105+0000","last_clean":"2024-09-10T09:54:00.844105+0000","last_became_active":"2024-09-10T09:52:31.371163+0000","last_became_peered":"2024-09-10T09:52:31.371163+0000","last_unstale":"2024-09-10T09:54:00.844105+0000","last_undegraded":"2024-09-10T09:54:00.844105+0000","last_fullsized":"2024-09-10T09:54:00.844105+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:11:43.681524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"94'6","reported_seq":108,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:11.965955+0000","last_change":"2024-09-10T09:52:19.841873+0000","last_active":"2024-09-10T09:54:11.965955+0000","last_peered":"2024-09-10T09:54:11.965955+0000","last_clean":"2024-09-10T09:54:11.965955+0000","last_became_active":"2024-09-10T09:52:19.841513+0000","last_became_peered":"2024-09-10T09:52:19.841513+0000","last_unstale":"2024-09-10T09:54:11.965955+0000","last_undegraded":"2024-09-10T09:54:11.965955+0000","last_fullsized":"2024-09-10T09:54:11.965955+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:53:31.854185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":60,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.149395+0000","last_change":"2024-09-10T09:52:22.314781+0000","last_active":"2024-09-10T09:54:01.149395+0000","last_peered":"2024-09-10T09:54:01.149395+0000","last_clean":"2024-09-10T09:54:01.149395+0000","last_became_active":"2024-09-10T09:52:22.313346+0000","last_became_peered":"2024-09-10T09:52:22.313346+0000","last_unstale":"2024-09-10T09:54:01.149395+0000","last_undegraded":"2024-09-10T09:54:01.149395+0000","last_fullsized":"2024-09-10T09:54:01.149395+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:15:16.921963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.844346+0000","last_change":"2024-09-10T09:52:31.365544+0000","last_active":"2024-09-10T09:54:00.844346+0000","last_peered":"2024-09-10T09:54:00.844346+0000","last_clean":"2024-09-10T09:54:00.844346+0000","last_became_active":"2024-09-10T09:52:31.365151+0000","last_became_peered":"2024-09-10T09:52:31.365151+0000","last_unstale":"2024-09-10T09:54:00.844346+0000","last_undegraded":"2024-09-10T09:54:00.844346+0000","last_fullsized":"2024-09-10T09:54:00.844346+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:36:55.247890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.954850+0000","last_change":"2024-09-10T09:52:22.314117+0000","last_active":"2024-09-10T09:54:00.954850+0000","last_peered":"2024-09-10T09:54:00.954850+0000","last_clean":"2024-09-10T09:54:00.954850+0000","last_became_active":"2024-09-10T09:52:22.313772+0000","last_became_peered":"2024-09-10T09:52:22.313772+0000","last_unstale":"2024-09-10T09:54:00.954850+0000","last_undegraded":"2024-09-10T09:54:00.954850+0000","last_fullsized":"2024-09-10T09:54:00.954850+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:19:37.373598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.844246+0000","last_change":"2024-09-10T09:52:22.314362+0000","last_active":"2024-09-10T09:54:00.844246+0000","last_peered":"2024-09-10T09:54:00.844246+0000","last_clean":"2024-09-10T09:54:00.844246+0000","last_became_active":"2024-09-10T09:52:22.314050+0000","last_became_peered":"2024-09-10T09:52:22.314050+0000","last_unstale":"2024-09-10T09:54:00.844246+0000","last_undegraded":"2024-09-10T09:54:00.844246+0000","last_fullsized":"2024-09-10T09:54:00.844246+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:37:07.218656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.843976+0000","last_change":"2024-09-10T09:52:22.322363+0000","last_active":"2024-09-10T09:54:00.843976+0000","last_peered":"2024-09-10T09:54:00.843976+0000","last_clean":"2024-09-10T09:54:00.843976+0000","last_became_active":"2024-09-10T09:52:22.322045+0000","last_became_peered":"2024-09-10T09:52:22.322045+0000","last_unstale":"2024-09-10T09:54:00.843976+0000","last_undegraded":"2024-09-10T09:54:00.843976+0000","last_fullsized":"2024-09-10T09:54:00.843976+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:10:27.918769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.247998+0000","last_change":"2024-09-10T09:52:22.331594+0000","last_active":"2024-09-10T09:54:01.247998+0000","last_peered":"2024-09-10T09:54:01.247998+0000","last_clean":"2024-09-10T09:54:01.247998+0000","last_became_active":"2024-09-10T09:52:22.330585+0000","last_became_peered":"2024-09-10T09:52:22.330585+0000","last_unstale":"2024-09-10T09:54:01.247998+0000","last_undegraded":"2024-09-10T09:54:01.247998+0000","last_fullsized":"2024-09-10T09:54:01.247998+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:45:56.412313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.841849+0000","last_change":"2024-09-10T09:52:22.325753+0000","last_active":"2024-09-10T09:54:00.841849+0000","last_peered":"2024-09-10T09:54:00.841849+0000","last_clean":"2024-09-10T09:54:00.841849+0000","last_became_active":"2024-09-10T09:52:22.325385+0000","last_became_peered":"2024-09-10T09:52:22.325385+0000","last_unstale":"2024-09-10T09:54:00.841849+0000","last_undegraded":"2024-09-10T09:54:00.841849+0000","last_fullsized":"2024-09-10T09:54:00.841849+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:18:28.776789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.955325+0000","last_change":"2024-09-10T09:52:22.319069+0000","last_active":"2024-09-10T09:54:00.955325+0000","last_peered":"2024-09-10T09:54:00.955325+0000","last_clean":"2024-09-10T09:54:00.955325+0000","last_became_active":"2024-09-10T09:52:22.318667+0000","last_became_peered":"2024-09-10T09:52:22.318667+0000","last_unstale":"2024-09-10T09:54:00.955325+0000","last_undegraded":"2024-09-10T09:54:00.955325+0000","last_fullsized":"2024-09-10T09:54:00.955325+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:21:32.028235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.844567+0000","last_change":"2024-09-10T09:52:22.312890+0000","last_active":"2024-09-10T09:54:00.844567+0000","last_peered":"2024-09-10T09:54:00.844567+0000","last_clean":"2024-09-10T09:54:00.844567+0000","last_became_active":"2024-09-10T09:52:22.312608+0000","last_became_peered":"2024-09-10T09:52:22.312608+0000","last_unstale":"2024-09-10T09:54:00.844567+0000","last_undegraded":"2024-09-10T09:54:00.844567+0000","last_fullsized":"2024-09-10T09:54:00.844567+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:07:46.046915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.151385+0000","last_change":"2024-09-10T09:54:01.151385+0000","last_active":"2024-09-10T09:54:01.151385+0000","last_peered":"2024-09-10T09:54:01.151385+0000","last_clean":"2024-09-10T09:54:01.151385+0000","last_became_active":"2024-09-10T09:54:01.150971+0000","last_became_peered":"2024-09-10T09:54:01.150971+0000","last_unstale":"2024-09-10T09:54:01.151385+0000","last_undegraded":"2024-09-10T09:54:01.151385+0000","last_fullsized":"2024-09-10T09:54:01.151385+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:27:10.492071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.849203+0000","last_change":"2024-09-10T09:54:00.848273+0000","last_active":"2024-09-10T09:54:00.849203+0000","last_peered":"2024-09-10T09:54:00.849203+0000","last_clean":"2024-09-10T09:54:00.849203+0000","last_became_active":"2024-09-10T09:54:00.847492+0000","last_became_peered":"2024-09-10T09:54:00.847492+0000","last_unstale":"2024-09-10T09:54:00.849203+0000","last_undegraded":"2024-09-10T09:54:00.849203+0000","last_fullsized":"2024-09-10T09:54:00.849203+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:02:24.749291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,3],"acting":[6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.845839+0000","last_change":"2024-09-10T09:52:22.313538+0000","last_active":"2024-09-10T09:54:00.845839+0000","last_peered":"2024-09-10T09:54:00.845839+0000","last_clean":"2024-09-10T09:54:00.845839+0000","last_became_active":"2024-09-10T09:52:22.313196+0000","last_became_peered":"2024-09-10T09:52:22.313196+0000","last_unstale":"2024-09-10T09:54:00.845839+0000","last_undegraded":"2024-09-10T09:54:00.845839+0000","last_fullsized":"2024-09-10T09:54:00.845839+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:53:23.387167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.988773+0000","last_change":"2024-09-10T09:52:22.316518+0000","last_active":"2024-09-10T09:54:00.988773+0000","last_peered":"2024-09-10T09:54:00.988773+0000","last_clean":"2024-09-10T09:54:00.988773+0000","last_became_active":"2024-09-10T09:52:22.316193+0000","last_became_peered":"2024-09-10T09:52:22.316193+0000","last_unstale":"2024-09-10T09:54:00.988773+0000","last_undegraded":"2024-09-10T09:54:00.988773+0000","last_fullsized":"2024-09-10T09:54:00.988773+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:49:55.903928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.841553+0000","last_change":"2024-09-10T09:52:22.329163+0000","last_active":"2024-09-10T09:54:00.841553+0000","last_peered":"2024-09-10T09:54:00.841553+0000","last_clean":"2024-09-10T09:54:00.841553+0000","last_became_active":"2024-09-10T09:52:22.328453+0000","last_became_peered":"2024-09-10T09:52:22.328453+0000","last_unstale":"2024-09-10T09:54:00.841553+0000","last_undegraded":"2024-09-10T09:54:00.841553+0000","last_fullsized":"2024-09-10T09:54:00.841553+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:53:33.286055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.955022+0000","last_change":"2024-09-10T09:52:22.314302+0000","last_active":"2024-09-10T09:54:00.955022+0000","last_peered":"2024-09-10T09:54:00.955022+0000","last_clean":"2024-09-10T09:54:00.955022+0000","last_became_active":"2024-09-10T09:52:22.313992+0000","last_became_peered":"2024-09-10T09:52:22.313992+0000","last_unstale":"2024-09-10T09:54:00.955022+0000","last_undegraded":"2024-09-10T09:54:00.955022+0000","last_fullsized":"2024-09-10T09:54:00.955022+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:07:35.764015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.844450+0000","last_change":"2024-09-10T09:52:22.316400+0000","last_active":"2024-09-10T09:54:00.844450+0000","last_peered":"2024-09-10T09:54:00.844450+0000","last_clean":"2024-09-10T09:54:00.844450+0000","last_became_active":"2024-09-10T09:52:22.316110+0000","last_became_peered":"2024-09-10T09:52:22.316110+0000","last_unstale":"2024-09-10T09:54:00.844450+0000","last_undegraded":"2024-09-10T09:54:00.844450+0000","last_fullsized":"2024-09-10T09:54:00.844450+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:17:26.528899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.844292+0000","last_change":"2024-09-10T09:52:22.317914+0000","last_active":"2024-09-10T09:54:00.844292+0000","last_peered":"2024-09-10T09:54:00.844292+0000","last_clean":"2024-09-10T09:54:00.844292+0000","last_became_active":"2024-09-10T09:52:22.317575+0000","last_became_peered":"2024-09-10T09:52:22.317575+0000","last_unstale":"2024-09-10T09:54:00.844292+0000","last_undegraded":"2024-09-10T09:54:00.844292+0000","last_fullsized":"2024-09-10T09:54:00.844292+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:17:51.446233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.955646+0000","last_change":"2024-09-10T09:52:22.327749+0000","last_active":"2024-09-10T09:54:00.955646+0000","last_peered":"2024-09-10T09:54:00.955646+0000","last_clean":"2024-09-10T09:54:00.955646+0000","last_became_active":"2024-09-10T09:52:22.327461+0000","last_became_peered":"2024-09-10T09:52:22.327461+0000","last_unstale":"2024-09-10T09:54:00.955646+0000","last_undegraded":"2024-09-10T09:54:00.955646+0000","last_fullsized":"2024-09-10T09:54:00.955646+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:54.208064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.988358+0000","last_change":"2024-09-10T09:52:22.326982+0000","last_active":"2024-09-10T09:54:00.988358+0000","last_peered":"2024-09-10T09:54:00.988358+0000","last_clean":"2024-09-10T09:54:00.988358+0000","last_became_active":"2024-09-10T09:52:22.326635+0000","last_became_peered":"2024-09-10T09:52:22.326635+0000","last_unstale":"2024-09-10T09:54:00.988358+0000","last_undegraded":"2024-09-10T09:54:00.988358+0000","last_fullsized":"2024-09-10T09:54:00.988358+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:50:54.835899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.247654+0000","last_change":"2024-09-10T09:52:22.323410+0000","last_active":"2024-09-10T09:54:01.247654+0000","last_peered":"2024-09-10T09:54:01.247654+0000","last_clean":"2024-09-10T09:54:01.247654+0000","last_became_active":"2024-09-10T09:52:22.323116+0000","last_became_peered":"2024-09-10T09:52:22.323116+0000","last_unstale":"2024-09-10T09:54:01.247654+0000","last_undegraded":"2024-09-10T09:54:01.247654+0000","last_fullsized":"2024-09-10T09:54:01.247654+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:09:15.115162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.247721+0000","last_change":"2024-09-10T09:52:22.324908+0000","last_active":"2024-09-10T09:54:01.247721+0000","last_peered":"2024-09-10T09:54:01.247721+0000","last_clean":"2024-09-10T09:54:01.247721+0000","last_became_active":"2024-09-10T09:52:22.324449+0000","last_became_peered":"2024-09-10T09:52:22.324449+0000","last_unstale":"2024-09-10T09:54:01.247721+0000","last_undegraded":"2024-09-10T09:54:01.247721+0000","last_fullsized":"2024-09-10T09:54:01.247721+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:43:24.535721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.841549+0000","last_change":"2024-09-10T09:52:22.322157+0000","last_active":"2024-09-10T09:54:00.841549+0000","last_peered":"2024-09-10T09:54:00.841549+0000","last_clean":"2024-09-10T09:54:00.841549+0000","last_became_active":"2024-09-10T09:52:22.321856+0000","last_became_peered":"2024-09-10T09:52:22.321856+0000","last_unstale":"2024-09-10T09:54:00.841549+0000","last_undegraded":"2024-09-10T09:54:00.841549+0000","last_fullsized":"2024-09-10T09:54:00.841549+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:36:38.395370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":15,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.843379+0000","last_change":"2024-09-10T09:53:00.800337+0000","last_active":"2024-09-10T09:54:00.843379+0000","last_peered":"2024-09-10T09:54:00.843379+0000","last_clean":"2024-09-10T09:54:00.843379+0000","last_became_active":"2024-09-10T09:53:00.799863+0000","last_became_peered":"2024-09-10T09:53:00.799863+0000","last_unstale":"2024-09-10T09:54:00.843379+0000","last_undegraded":"2024-09-10T09:54:00.843379+0000","last_fullsized":"2024-09-10T09:54:00.843379+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:18.017672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.059591+0000","last_change":"2024-09-10T09:52:22.310735+0000","last_active":"2024-09-10T09:54:01.059591+0000","last_peered":"2024-09-10T09:54:01.059591+0000","last_clean":"2024-09-10T09:54:01.059591+0000","last_became_active":"2024-09-10T09:52:22.310376+0000","last_became_peered":"2024-09-10T09:52:22.310376+0000","last_unstale":"2024-09-10T09:54:01.059591+0000","last_undegraded":"2024-09-10T09:54:01.059591+0000","last_fullsized":"2024-09-10T09:54:01.059591+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:38:08.637588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.841944+0000","last_change":"2024-09-10T09:52:22.321113+0000","last_active":"2024-09-10T09:54:00.841944+0000","last_peered":"2024-09-10T09:54:00.841944+0000","last_clean":"2024-09-10T09:54:00.841944+0000","last_became_active":"2024-09-10T09:52:22.320767+0000","last_became_peered":"2024-09-10T09:52:22.320767+0000","last_unstale":"2024-09-10T09:54:00.841944+0000","last_undegraded":"2024-09-10T09:54:00.841944+0000","last_fullsized":"2024-09-10T09:54:00.841944+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:37.549322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.989072+0000","last_change":"2024-09-10T09:52:22.318634+0000","last_active":"2024-09-10T09:54:00.989072+0000","last_peered":"2024-09-10T09:54:00.989072+0000","last_clean":"2024-09-10T09:54:00.989072+0000","last_became_active":"2024-09-10T09:52:22.318294+0000","last_became_peered":"2024-09-10T09:52:22.318294+0000","last_unstale":"2024-09-10T09:54:00.989072+0000","last_undegraded":"2024-09-10T09:54:00.989072+0000","last_fullsized":"2024-09-10T09:54:00.989072+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:19:31.262055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.846113+0000","last_change":"2024-09-10T09:52:22.324567+0000","last_active":"2024-09-10T09:54:00.846113+0000","last_peered":"2024-09-10T09:54:00.846113+0000","last_clean":"2024-09-10T09:54:00.846113+0000","last_became_active":"2024-09-10T09:52:22.321803+0000","last_became_peered":"2024-09-10T09:52:22.321803+0000","last_unstale":"2024-09-10T09:54:00.846113+0000","last_undegraded":"2024-09-10T09:54:00.846113+0000","last_fullsized":"2024-09-10T09:54:00.846113+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:08:11.099428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.059075+0000","last_change":"2024-09-10T09:52:22.318163+0000","last_active":"2024-09-10T09:54:01.059075+0000","last_peered":"2024-09-10T09:54:01.059075+0000","last_clean":"2024-09-10T09:54:01.059075+0000","last_became_active":"2024-09-10T09:52:22.317808+0000","last_became_peered":"2024-09-10T09:52:22.317808+0000","last_unstale":"2024-09-10T09:54:01.059075+0000","last_undegraded":"2024-09-10T09:54:01.059075+0000","last_fullsized":"2024-09-10T09:54:01.059075+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:19.214291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.841550+0000","last_change":"2024-09-10T09:52:22.323035+0000","last_active":"2024-09-10T09:54:00.841550+0000","last_peered":"2024-09-10T09:54:00.841550+0000","last_clean":"2024-09-10T09:54:00.841550+0000","last_became_active":"2024-09-10T09:52:22.322679+0000","last_became_peered":"2024-09-10T09:52:22.322679+0000","last_unstale":"2024-09-10T09:54:00.841550+0000","last_undegraded":"2024-09-10T09:54:00.841550+0000","last_fullsized":"2024-09-10T09:54:00.841550+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:23:13.762612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.149321+0000","last_change":"2024-09-10T09:52:22.315887+0000","last_active":"2024-09-10T09:54:01.149321+0000","last_peered":"2024-09-10T09:54:01.149321+0000","last_clean":"2024-09-10T09:54:01.149321+0000","last_became_active":"2024-09-10T09:52:22.315569+0000","last_became_peered":"2024-09-10T09:52:22.315569+0000","last_unstale":"2024-09-10T09:54:01.149321+0000","last_undegraded":"2024-09-10T09:54:01.149321+0000","last_fullsized":"2024-09-10T09:54:01.149321+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:46:57.094320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.951738+0000","last_change":"2024-09-10T09:52:22.314060+0000","last_active":"2024-09-10T09:54:00.951738+0000","last_peered":"2024-09-10T09:54:00.951738+0000","last_clean":"2024-09-10T09:54:00.951738+0000","last_became_active":"2024-09-10T09:52:22.313572+0000","last_became_peered":"2024-09-10T09:52:22.313572+0000","last_unstale":"2024-09-10T09:54:00.951738+0000","last_undegraded":"2024-09-10T09:54:00.951738+0000","last_fullsized":"2024-09-10T09:54:00.951738+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:11:17.158625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.247866+0000","last_change":"2024-09-10T09:52:22.327233+0000","last_active":"2024-09-10T09:54:01.247866+0000","last_peered":"2024-09-10T09:54:01.247866+0000","last_clean":"2024-09-10T09:54:01.247866+0000","last_became_active":"2024-09-10T09:52:22.326945+0000","last_became_peered":"2024-09-10T09:52:22.326945+0000","last_unstale":"2024-09-10T09:54:01.247866+0000","last_undegraded":"2024-09-10T09:54:01.247866+0000","last_fullsized":"2024-09-10T09:54:01.247866+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:31:13.490147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.247963+0000","last_change":"2024-09-10T09:52:22.330912+0000","last_active":"2024-09-10T09:54:01.247963+0000","last_peered":"2024-09-10T09:54:01.247963+0000","last_clean":"2024-09-10T09:54:01.247963+0000","last_became_active":"2024-09-10T09:52:22.330149+0000","last_became_peered":"2024-09-10T09:52:22.330149+0000","last_unstale":"2024-09-10T09:54:01.247963+0000","last_undegraded":"2024-09-10T09:54:01.247963+0000","last_fullsized":"2024-09-10T09:54:01.247963+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:10:48.842499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.955621+0000","last_change":"2024-09-10T09:52:22.327035+0000","last_active":"2024-09-10T09:54:00.955621+0000","last_peered":"2024-09-10T09:54:00.955621+0000","last_clean":"2024-09-10T09:54:00.955621+0000","last_became_active":"2024-09-10T09:52:22.326741+0000","last_became_peered":"2024-09-10T09:52:22.326741+0000","last_unstale":"2024-09-10T09:54:00.955621+0000","last_undegraded":"2024-09-10T09:54:00.955621+0000","last_fullsized":"2024-09-10T09:54:00.955621+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:21:05.026614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70210,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":608,"num_read_kb":382,"num_write":245,"num_write_kb":140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":4426,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":210630,"data_compressed":5661,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":212,"ondisk_log_size":212,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282980,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449125,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1073152,"data_stored":554558,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":70,"seq":300647710760,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32380,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712772,"statfs":{"total":95995035648,"available":95961878528,"internally_reserved":0,"allocated":1216512,"data_stored":1204181,"data_compressed":70462,"data_compressed_allocated":94208,"data_compressed_original":688128,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":65,"seq":279172874288,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32264,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712888,"statfs":{"total":95995035648,"available":95961997312,"internally_reserved":0,"allocated":1097728,"data_stored":557996,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135926,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32252,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712900,"statfs":{"total":95995035648,"available":95962009600,"internally_reserved":0,"allocated":1085440,"data_stored":557702,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364858,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28144,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717008,"statfs":{"total":95995035648,"available":95966216192,"internally_reserved":0,"allocated":1138688,"data_stored":1150582,"data_compressed":68575,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528384,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32236,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712916,"statfs":{"total":95995035648,"available":95962025984,"internally_reserved":0,"allocated":1069056,"data_stored":549404,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790022,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28152,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717000,"statfs":{"total":95995035648,"available":95966208000,"internally_reserved":0,"allocated":1146880,"data_stored":1142263,"data_compressed":68575,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953545,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28144,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717008,"statfs":{"total":95995035648,"available":95966216192,"internally_reserved":0,"allocated":1146880,"data_stored":616873,"data_compressed":34470,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182479,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32248,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712904,"statfs":{"total":95995035648,"available":95962013696,"internally_reserved":0,"allocated":1081344,"data_stored":549112,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411413,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36412,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708740,"statfs":{"total":95995035648,"available":95957749760,"internally_reserved":0,"allocated":1085440,"data_stored":551047,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574939,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28144,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717008,"statfs":{"total":95995035648,"available":95966216192,"internally_reserved":0,"allocated":1142784,"data_stored":614108,"data_compressed":34470,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738464,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1093632,"data_stored":550860,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34340,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34340,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34340,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35992,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35992,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35992,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35992,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35992,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":65426,"data_compressed":1887,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":64095,"data_compressed":1887,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":65131,"data_compressed":1887,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T09:54:20.036 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-10T09:54:20.037 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-10T09:54:20.037 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-10T09:54:20.037 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph health --format=json 2024-09-10T09:54:20.437 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:54:21.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:21.015+0000 7f0dc3e00640 1 -- 172.21.15.77:0/4081375821 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0dc4108f60 msgr2=0x7f0dc4110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:21.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:21.015+0000 7f0dc3e00640 1 --2- 172.21.15.77:0/4081375821 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0dc4108f60 0x7f0dc4110ae0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc009ca0 tx=0x7f0dbc02a8c0 comp rx=0 tx=0).stop 2024-09-10T09:54:21.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:21.015+0000 7f0dc3e00640 1 -- 172.21.15.77:0/4081375821 shutdown_connections 2024-09-10T09:54:21.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:21.015+0000 7f0dc3e00640 1 --2- 172.21.15.77:0/4081375821 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0dc4108f60 0x7f0dc4110ae0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:21.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:21.015+0000 7f0dc3e00640 1 --2- 172.21.15.77:0/4081375821 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0dc41047e0 0x7f0dc4108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:21.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:21.015+0000 7f0dc3e00640 1 --2- 172.21.15.77:0/4081375821 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0dc4103e30 0x7f0dc4104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:21.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:21.015+0000 7f0dc3e00640 1 -- 172.21.15.77:0/4081375821 >> 172.21.15.77:0/4081375821 conn(0x7f0dc40ff910 msgr2=0x7f0dc4101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:21.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:21.015+0000 7f0dc3e00640 1 -- 172.21.15.77:0/4081375821 shutdown_connections 2024-09-10T09:54:21.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:21.015+0000 7f0dc3e00640 1 -- 172.21.15.77:0/4081375821 wait complete. 2024-09-10T09:54:21.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:21.016+0000 7f0dc3e00640 1 Processor -- start 2024-09-10T09:54:21.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:21.016+0000 7f0dc3e00640 1 -- start start 2024-09-10T09:54:21.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:21.016+0000 7f0dc3e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0dc4103e30 0x7f0dc419a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:21.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:21.016+0000 7f0dc3e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0dc41047e0 0x7f0dc419a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:21.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:21.016+0000 7f0dc3e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0dc4108f60 0x7f0dc419ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:21.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:21.016+0000 7f0dc3e00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc4113fe0 con 0x7f0dc4108f60 2024-09-10T09:54:21.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:21.016+0000 7f0dc3e00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc4113e60 con 0x7f0dc4103e30 2024-09-10T09:54:21.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:21.016+0000 7f0dc3e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc4114160 con 0x7f0dc41047e0 2024-09-10T09:54:21.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:21.016+0000 7f0dc3400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0dc4108f60 0x7f0dc419ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:21.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:21.016+0000 7f0dc3400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0dc4108f60 0x7f0dc419ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:35412/0 (socket says 172.21.15.77:35412) 2024-09-10T09:54:21.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:21.016+0000 7f0dc3400640 1 -- 172.21.15.77:0/2011088387 learned_addr learned my addr 172.21.15.77:0/2011088387 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:54:21.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:21.017+0000 7f0dc2a00640 1 --2- 172.21.15.77:0/2011088387 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0dc4103e30 0x7f0dc419a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:21.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:21.017+0000 7f0dba000640 1 --2- 172.21.15.77:0/2011088387 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0dc41047e0 0x7f0dc419a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:21.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:21.017+0000 7f0dc3400640 1 -- 172.21.15.77:0/2011088387 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0dc41047e0 msgr2=0x7f0dc419a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:21.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:21.017+0000 7f0dc3400640 1 --2- 172.21.15.77:0/2011088387 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0dc41047e0 0x7f0dc419a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:21.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:21.017+0000 7f0dc3400640 1 -- 172.21.15.77:0/2011088387 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0dc4103e30 msgr2=0x7f0dc419a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:21.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:21.017+0000 7f0dc3400640 1 --2- 172.21.15.77:0/2011088387 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0dc4103e30 0x7f0dc419a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:21.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:21.017+0000 7f0dc3400640 1 -- 172.21.15.77:0/2011088387 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dc419f450 con 0x7f0dc4108f60 2024-09-10T09:54:21.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:21.017+0000 7f0dc2a00640 1 --2- 172.21.15.77:0/2011088387 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0dc4103e30 0x7f0dc419a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:54:21.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:21.017+0000 7f0dc3400640 1 --2- 172.21.15.77:0/2011088387 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0dc4108f60 0x7f0dc419ed70 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc031a20 tx=0x7f0dbc031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:21.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:21.018+0000 7f0dbbe00640 1 -- 172.21.15.77:0/2011088387 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dbc003b70 con 0x7f0dc4108f60 2024-09-10T09:54:21.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:21.018+0000 7f0dbbe00640 1 -- 172.21.15.77:0/2011088387 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0dbc003d10 con 0x7f0dc4108f60 2024-09-10T09:54:21.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:21.018+0000 7f0dbbe00640 1 -- 172.21.15.77:0/2011088387 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dbc02fbd0 con 0x7f0dc4108f60 2024-09-10T09:54:21.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:21.018+0000 7f0dc3e00640 1 -- 172.21.15.77:0/2011088387 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dc419f6e0 con 0x7f0dc4108f60 2024-09-10T09:54:21.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:21.018+0000 7f0dc3e00640 1 -- 172.21.15.77:0/2011088387 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dc41a7020 con 0x7f0dc4108f60 2024-09-10T09:54:21.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:21.019+0000 7f0dc3e00640 1 -- 172.21.15.77:0/2011088387 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0dc4105310 con 0x7f0dc4108f60 2024-09-10T09:54:21.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:21.021+0000 7f0dbbe00640 1 -- 172.21.15.77:0/2011088387 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f0dbc02fd70 con 0x7f0dc4108f60 2024-09-10T09:54:21.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:21.023+0000 7f0dbbe00640 1 --2- 172.21.15.77:0/2011088387 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f0da80a2c90 0x7f0da80a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:21.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:21.023+0000 7f0dbbe00640 1 -- 172.21.15.77:0/2011088387 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f0dbc0f05e0 con 0x7f0dc4108f60 2024-09-10T09:54:21.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:21.023+0000 7f0dc2a00640 1 --2- 172.21.15.77:0/2011088387 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f0da80a2c90 0x7f0da80a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:21.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:21.024+0000 7f0dc2a00640 1 --2- 172.21.15.77:0/2011088387 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f0da80a2c90 0x7f0da80a5150 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0db4009d60 tx=0x7f0db4009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:21.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:21.027+0000 7f0dbbe00640 1 -- 172.21.15.77:0/2011088387 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0dbc0bd250 con 0x7f0dc4108f60 2024-09-10T09:54:21.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:54:20 smithi120 ceph-mon[34770]: from='client.24959 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T09:54:21.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:54:20 smithi120 ceph-mon[34770]: pgmap v366: 129 pgs: 129 active+clean; 645 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-09-10T09:54:21.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:54:20 smithi077 ceph-mon[30319]: from='client.24959 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T09:54:21.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:54:20 smithi077 ceph-mon[30319]: pgmap v366: 129 pgs: 129 active+clean; 645 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-09-10T09:54:21.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:21.242+0000 7f0dc3e00640 1 -- 172.21.15.77:0/2011088387 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f0dc4104210 con 0x7f0dc4108f60 2024-09-10T09:54:21.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:21.243+0000 7f0dbbe00640 1 -- 172.21.15.77:0/2011088387 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f0dc4104210 con 0x7f0dc4108f60 2024-09-10T09:54:21.245 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:54:21.245 INFO:teuthology.orchestra.run.smithi077.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-10T09:54:21.247 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:21.245+0000 7f0dc3e00640 1 -- 172.21.15.77:0/2011088387 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f0da80a2c90 msgr2=0x7f0da80a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:21.247 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:21.245+0000 7f0dc3e00640 1 --2- 172.21.15.77:0/2011088387 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f0da80a2c90 0x7f0da80a5150 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0db4009d60 tx=0x7f0db4009110 comp rx=0 tx=0).stop 2024-09-10T09:54:21.247 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:21.245+0000 7f0dc3e00640 1 -- 172.21.15.77:0/2011088387 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0dc4108f60 msgr2=0x7f0dc419ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:21.247 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:21.245+0000 7f0dc3e00640 1 --2- 172.21.15.77:0/2011088387 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0dc4108f60 0x7f0dc419ed70 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc031a20 tx=0x7f0dbc031a50 comp rx=0 tx=0).stop 2024-09-10T09:54:21.247 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:21.245+0000 7f0dc3e00640 1 -- 172.21.15.77:0/2011088387 shutdown_connections 2024-09-10T09:54:21.247 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:21.245+0000 7f0dc3e00640 1 --2- 172.21.15.77:0/2011088387 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f0da80a2c90 0x7f0da80a5150 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:21.247 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:21.245+0000 7f0dc3e00640 1 --2- 172.21.15.77:0/2011088387 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0dc4108f60 0x7f0dc419ed70 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:21.247 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:21.245+0000 7f0dc3e00640 1 --2- 172.21.15.77:0/2011088387 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0dc41047e0 0x7f0dc419a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:21.247 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:21.245+0000 7f0dc3e00640 1 --2- 172.21.15.77:0/2011088387 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0dc4103e30 0x7f0dc419a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:21.247 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:21.245+0000 7f0dc3e00640 1 -- 172.21.15.77:0/2011088387 >> 172.21.15.77:0/2011088387 conn(0x7f0dc40ff910 msgr2=0x7f0dc4100c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:21.248 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:21.245+0000 7f0dc3e00640 1 -- 172.21.15.77:0/2011088387 shutdown_connections 2024-09-10T09:54:21.248 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:21.245+0000 7f0dc3e00640 1 -- 172.21.15.77:0/2011088387 wait complete. 2024-09-10T09:54:21.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:54:20 smithi134 ceph-mon[35674]: from='client.24959 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T09:54:21.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:54:20 smithi134 ceph-mon[35674]: pgmap v366: 129 pgs: 129 active+clean; 645 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-09-10T09:54:22.010 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-10T09:54:22.010 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:54:22.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:54:21 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/2011088387' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-10T09:54:22.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:54:21 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:54:22.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:54:21 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/2011088387' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-10T09:54:22.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:54:21 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:54:22.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:54:21 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/2011088387' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-10T09:54:22.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:54:21 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:54:22.743 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:54:23.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:54:22 smithi120 ceph-mon[34770]: pgmap v367: 129 pgs: 129 active+clean; 645 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-10T09:54:23.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:54:22 smithi077 ceph-mon[30319]: pgmap v367: 129 pgs: 129 active+clean; 645 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-10T09:54:23.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:54:22 smithi134 ceph-mon[35674]: pgmap v367: 129 pgs: 129 active+clean; 645 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-10T09:54:23.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:23.714+0000 7f3097400640 1 -- 172.21.15.77:0/3794814855 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3090074620 msgr2=0x7f3090074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:23.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:23.714+0000 7f3097400640 1 --2- 172.21.15.77:0/3794814855 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3090074620 0x7f3090074a00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3080009900 tx=0x7f308002a480 comp rx=0 tx=0).stop 2024-09-10T09:54:23.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:23.714+0000 7f3097400640 1 -- 172.21.15.77:0/3794814855 shutdown_connections 2024-09-10T09:54:23.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:23.714+0000 7f3097400640 1 --2- 172.21.15.77:0/3794814855 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f309010e3c0 0x7f3090110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:23.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:23.714+0000 7f3097400640 1 --2- 172.21.15.77:0/3794814855 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3090074f40 0x7f309010de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:23.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:23.714+0000 7f3097400640 1 --2- 172.21.15.77:0/3794814855 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3090074620 0x7f3090074a00 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:23.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:23.714+0000 7f3097400640 1 -- 172.21.15.77:0/3794814855 >> 172.21.15.77:0/3794814855 conn(0x7f30900fd8c0 msgr2=0x7f30900ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:23.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:23.714+0000 7f3097400640 1 -- 172.21.15.77:0/3794814855 shutdown_connections 2024-09-10T09:54:23.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:23.715+0000 7f3097400640 1 -- 172.21.15.77:0/3794814855 wait complete. 2024-09-10T09:54:23.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:23.716+0000 7f3097400640 1 Processor -- start 2024-09-10T09:54:23.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:23.716+0000 7f3097400640 1 -- start start 2024-09-10T09:54:23.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:23.716+0000 7f3097400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3090074620 0x7f309019a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:23.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:23.716+0000 7f3097400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3090074f40 0x7f309019abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:23.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:23.716+0000 7f3097400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f309010e3c0 0x7f309019ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:23.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:23.716+0000 7f3097400640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f30901141c0 con 0x7f309010e3c0 2024-09-10T09:54:23.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:23.716+0000 7f3097400640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3090114040 con 0x7f3090074620 2024-09-10T09:54:23.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:23.716+0000 7f3097400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3090114340 con 0x7f3090074f40 2024-09-10T09:54:23.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:23.717+0000 7f3094a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3090074620 0x7f309019a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:23.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:23.717+0000 7f3087e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3090074f40 0x7f309019abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:23.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:23.717+0000 7f3094a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3090074620 0x7f309019a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:59672/0 (socket says 172.21.15.77:59672) 2024-09-10T09:54:23.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:23.717+0000 7f3087e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3090074f40 0x7f309019abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:42872/0 (socket says 172.21.15.77:42872) 2024-09-10T09:54:23.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:23.717+0000 7f3094a00640 1 -- 172.21.15.77:0/1822555352 learned_addr learned my addr 172.21.15.77:0/1822555352 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:54:23.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:23.717+0000 7f3095400640 1 --2- 172.21.15.77:0/1822555352 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f309010e3c0 0x7f309019ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:23.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:23.717+0000 7f3094a00640 1 -- 172.21.15.77:0/1822555352 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3090074f40 msgr2=0x7f309019abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:23.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:23.717+0000 7f3094a00640 1 --2- 172.21.15.77:0/1822555352 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3090074f40 0x7f309019abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:23.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:23.717+0000 7f3094a00640 1 -- 172.21.15.77:0/1822555352 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f309010e3c0 msgr2=0x7f309019ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:23.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:23.717+0000 7f3094a00640 1 --2- 172.21.15.77:0/1822555352 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f309010e3c0 0x7f309019ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:23.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:23.717+0000 7f3094a00640 1 -- 172.21.15.77:0/1822555352 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f309019f630 con 0x7f3090074620 2024-09-10T09:54:23.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:23.717+0000 7f3094a00640 1 --2- 172.21.15.77:0/1822555352 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3090074620 0x7f309019a680 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3080004770 tx=0x7f3080003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:23.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:23.718+0000 7f308e000640 1 -- 172.21.15.77:0/1822555352 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f308003f070 con 0x7f3090074620 2024-09-10T09:54:23.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:23.718+0000 7f308e000640 1 -- 172.21.15.77:0/1822555352 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3080031ca0 con 0x7f3090074620 2024-09-10T09:54:23.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:23.718+0000 7f308e000640 1 -- 172.21.15.77:0/1822555352 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3080035030 con 0x7f3090074620 2024-09-10T09:54:23.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:23.718+0000 7f3097400640 1 -- 172.21.15.77:0/1822555352 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f309019f8c0 con 0x7f3090074620 2024-09-10T09:54:23.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:23.718+0000 7f3097400640 1 -- 172.21.15.77:0/1822555352 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f309019fd50 con 0x7f3090074620 2024-09-10T09:54:23.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:23.719+0000 7f3097400640 1 -- 172.21.15.77:0/1822555352 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3090074a00 con 0x7f3090074620 2024-09-10T09:54:23.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:23.721+0000 7f308e000640 1 -- 172.21.15.77:0/1822555352 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f3080042070 con 0x7f3090074620 2024-09-10T09:54:23.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:23.722+0000 7f308e000640 1 --2- 172.21.15.77:0/1822555352 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f30740a2c10 0x7f30740a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:23.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:23.722+0000 7f308e000640 1 -- 172.21.15.77:0/1822555352 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f30800eff50 con 0x7f3090074620 2024-09-10T09:54:23.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:23.722+0000 7f3087e00640 1 --2- 172.21.15.77:0/1822555352 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f30740a2c10 0x7f30740a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:23.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:23.723+0000 7f3087e00640 1 --2- 172.21.15.77:0/1822555352 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f30740a2c10 0x7f30740a50d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f30780095f0 tx=0x7f3078005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:23.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:23.724+0000 7f308e000640 1 -- 172.21.15.77:0/1822555352 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f30800bc370 con 0x7f3090074620 2024-09-10T09:54:23.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:23.965+0000 7f3097400640 1 -- 172.21.15.77:0/1822555352 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3090109540 con 0x7f3090074620 2024-09-10T09:54:23.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:23.966+0000 7f308e000640 1 -- 172.21.15.77:0/1822555352 <== mon.2 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11989 (secure 0 0 0) 0x7f30800c0670 con 0x7f3090074620 2024-09-10T09:54:23.968 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:54:23.968 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":28,"btime":"2024-09-10T09:53:16:873716+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3159289675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3159289675},{"type":"v1","addr":"172.21.15.77:6835","nonce":3159289675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1951962341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1951962341},{"type":"v1","addr":"172.21.15.77:6837","nonce":1951962341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:52:23.368252+0000","modified":"2024-09-10T09:53:16.873698+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24563,"mds_1":24524,"mds_2":24520,"mds_3":24502,"mds_4":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14487":{"gid":14487,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/4291121888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":4291121888},{"type":"v1","addr":"172.21.15.77:6839","nonce":4291121888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14499":{"gid":14499,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6841/445284194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":445284194},{"type":"v1","addr":"172.21.15.77:6841","nonce":445284194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1005054423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1005054423},{"type":"v1","addr":"172.21.15.134:6833","nonce":1005054423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6833/3716307744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":3716307744},{"type":"v1","addr":"172.21.15.120:6833","nonce":3716307744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/1319892544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1319892544},{"type":"v1","addr":"172.21.15.120:6835","nonce":1319892544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.134:6835/3293933699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3293933699},{"type":"v1","addr":"172.21.15.134:6835","nonce":3293933699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.134:6837/2461497091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2461497091},{"type":"v1","addr":"172.21.15.134:6837","nonce":2461497091}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":20,"addr":"172.21.15.134:6839/1965117680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1965117680},{"type":"v1","addr":"172.21.15.134:6839","nonce":1965117680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":20,"addr":"172.21.15.120:6837/3033280709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3033280709},{"type":"v1","addr":"172.21.15.120:6837","nonce":3033280709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.120:6839/1777710157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1777710157},{"type":"v1","addr":"172.21.15.120:6839","nonce":1777710157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24563,"qdb_cluster":[24563,24524,24520,24502,24496]},"id":1}]} 2024-09-10T09:54:23.969 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 28 2024-09-10T09:54:23.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:23.969+0000 7f3097400640 1 -- 172.21.15.77:0/1822555352 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f30740a2c10 msgr2=0x7f30740a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:23.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:23.969+0000 7f3097400640 1 --2- 172.21.15.77:0/1822555352 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f30740a2c10 0x7f30740a50d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f30780095f0 tx=0x7f3078005f90 comp rx=0 tx=0).stop 2024-09-10T09:54:23.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:23.969+0000 7f3097400640 1 -- 172.21.15.77:0/1822555352 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3090074620 msgr2=0x7f309019a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:23.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:23.969+0000 7f3097400640 1 --2- 172.21.15.77:0/1822555352 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3090074620 0x7f309019a680 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3080004770 tx=0x7f3080003c10 comp rx=0 tx=0).stop 2024-09-10T09:54:23.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:23.969+0000 7f3097400640 1 -- 172.21.15.77:0/1822555352 shutdown_connections 2024-09-10T09:54:23.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:23.969+0000 7f3097400640 1 --2- 172.21.15.77:0/1822555352 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f30740a2c10 0x7f30740a50d0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:23.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:23.969+0000 7f3097400640 1 --2- 172.21.15.77:0/1822555352 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f309010e3c0 0x7f309019ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:23.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:23.969+0000 7f3097400640 1 --2- 172.21.15.77:0/1822555352 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3090074f40 0x7f309019abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:23.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:23.969+0000 7f3097400640 1 --2- 172.21.15.77:0/1822555352 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3090074620 0x7f309019a680 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:23.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:23.969+0000 7f3097400640 1 -- 172.21.15.77:0/1822555352 >> 172.21.15.77:0/1822555352 conn(0x7f30900fd8c0 msgr2=0x7f30900fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:23.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:23.969+0000 7f3097400640 1 -- 172.21.15.77:0/1822555352 shutdown_connections 2024-09-10T09:54:23.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:23.969+0000 7f3097400640 1 -- 172.21.15.77:0/1822555352 wait complete. 2024-09-10T09:54:24.218 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:54:24 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:54:24.468 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:54:24 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:54:24.468 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:54:24 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:54:24.469 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:54:24 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:54:24.469 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:54:24 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:54:24.469 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:54:24 smithi134 ceph-mon[35674]: pgmap v368: 129 pgs: 129 active+clean; 645 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-10T09:54:24.469 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:54:24 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/1822555352' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:54:24.673 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:54:24 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:54:24.673 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:54:24 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:54:24.673 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:54:24 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:54:24.674 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:54:24 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:54:24.674 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:54:24 smithi120 ceph-mon[34770]: pgmap v368: 129 pgs: 129 active+clean; 645 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-10T09:54:24.674 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:54:24 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/1822555352' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:54:24.689 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:54:24 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:54:24.689 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:54:24 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:54:24.689 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:54:24 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:54:24.689 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:54:24 smithi077 ceph-mon[30319]: pgmap v368: 129 pgs: 129 active+clean; 645 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-10T09:54:24.689 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:54:24 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/1822555352' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:54:24.745 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:54:25.147 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:54:25.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:25.753+0000 7f134c400640 1 -- 172.21.15.77:0/1263399298 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1344076720 msgr2=0x7f1344076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:25.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:25.753+0000 7f134c400640 1 --2- 172.21.15.77:0/1263399298 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1344076720 0x7f1344076b00 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1338009860 tx=0x7f133802a340 comp rx=0 tx=0).stop 2024-09-10T09:54:25.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:25.753+0000 7f134c400640 1 -- 172.21.15.77:0/1263399298 shutdown_connections 2024-09-10T09:54:25.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:25.753+0000 7f134c400640 1 --2- 172.21.15.77:0/1263399298 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1344112a50 0x7f1344114e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:25.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:25.753+0000 7f134c400640 1 --2- 172.21.15.77:0/1263399298 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1344077040 0x7f1344112510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:25.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:25.753+0000 7f134c400640 1 --2- 172.21.15.77:0/1263399298 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1344076720 0x7f1344076b00 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:25.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:25.753+0000 7f134c400640 1 -- 172.21.15.77:0/1263399298 >> 172.21.15.77:0/1263399298 conn(0x7f13440ffa50 msgr2=0x7f1344101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:25.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:25.753+0000 7f134c400640 1 -- 172.21.15.77:0/1263399298 shutdown_connections 2024-09-10T09:54:25.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:25.753+0000 7f134c400640 1 -- 172.21.15.77:0/1263399298 wait complete. 2024-09-10T09:54:25.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:25.754+0000 7f134c400640 1 Processor -- start 2024-09-10T09:54:25.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:25.755+0000 7f134c400640 1 -- start start 2024-09-10T09:54:25.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:25.755+0000 7f134c400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1344076720 0x7f13441a47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:25.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:25.755+0000 7f134c400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1344077040 0x7f13441a4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:25.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:25.755+0000 7f134c400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1344112a50 0x7f134419e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:25.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:25.755+0000 7f134c400640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f13441185e0 con 0x7f1344077040 2024-09-10T09:54:25.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:25.755+0000 7f134c400640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1344118460 con 0x7f1344076720 2024-09-10T09:54:25.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:25.755+0000 7f134c400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1344118760 con 0x7f1344112a50 2024-09-10T09:54:25.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:25.755+0000 7f1349000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1344077040 0x7f13441a4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:25.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:25.755+0000 7f1349a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1344076720 0x7f13441a47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:25.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:25.756+0000 7f1349000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1344077040 0x7f13441a4d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:48470/0 (socket says 172.21.15.77:48470) 2024-09-10T09:54:25.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:25.756+0000 7f1349000640 1 -- 172.21.15.77:0/102637465 learned_addr learned my addr 172.21.15.77:0/102637465 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:54:25.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:25.756+0000 7f134a400640 1 --2- 172.21.15.77:0/102637465 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1344112a50 0x7f134419e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:25.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:25.756+0000 7f1349000640 1 -- 172.21.15.77:0/102637465 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1344112a50 msgr2=0x7f134419e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:25.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:25.756+0000 7f1349000640 1 --2- 172.21.15.77:0/102637465 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1344112a50 0x7f134419e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:25.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:25.756+0000 7f1349000640 1 -- 172.21.15.77:0/102637465 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1344076720 msgr2=0x7f13441a47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:25.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:25.756+0000 7f1349000640 1 --2- 172.21.15.77:0/102637465 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1344076720 0x7f13441a47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:25.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:25.756+0000 7f1349000640 1 -- 172.21.15.77:0/102637465 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f134419f0e0 con 0x7f1344077040 2024-09-10T09:54:25.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:25.756+0000 7f1349a00640 1 --2- 172.21.15.77:0/102637465 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1344076720 0x7f13441a47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:54:25.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:25.756+0000 7f1349000640 1 --2- 172.21.15.77:0/102637465 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1344077040 0x7f13441a4d30 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f133000e9f0 tx=0x7f133000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:25.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:25.757+0000 7f1342000640 1 -- 172.21.15.77:0/102637465 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f133000cd80 con 0x7f1344077040 2024-09-10T09:54:25.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:25.757+0000 7f1342000640 1 -- 172.21.15.77:0/102637465 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1330005500 con 0x7f1344077040 2024-09-10T09:54:25.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:25.757+0000 7f134c400640 1 -- 172.21.15.77:0/102637465 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f134419f3d0 con 0x7f1344077040 2024-09-10T09:54:25.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:25.757+0000 7f1342000640 1 -- 172.21.15.77:0/102637465 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1330010690 con 0x7f1344077040 2024-09-10T09:54:25.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:25.757+0000 7f134c400640 1 -- 172.21.15.77:0/102637465 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13441ab6c0 con 0x7f1344077040 2024-09-10T09:54:25.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:25.758+0000 7f134c400640 1 -- 172.21.15.77:0/102637465 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1344076b00 con 0x7f1344077040 2024-09-10T09:54:25.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:25.761+0000 7f1342000640 1 -- 172.21.15.77:0/102637465 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f13300046c0 con 0x7f1344077040 2024-09-10T09:54:25.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:25.762+0000 7f1342000640 1 --2- 172.21.15.77:0/102637465 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f132c0a2c10 0x7f132c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:25.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:25.763+0000 7f1342000640 1 -- 172.21.15.77:0/102637465 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f13300cfc10 con 0x7f1344077040 2024-09-10T09:54:25.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:25.763+0000 7f1349a00640 1 --2- 172.21.15.77:0/102637465 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f132c0a2c10 0x7f132c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:25.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:25.764+0000 7f1349a00640 1 --2- 172.21.15.77:0/102637465 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f132c0a2c10 0x7f132c0a50d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1338009830 tx=0x7f13380058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:25.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:25.765+0000 7f1342000640 1 -- 172.21.15.77:0/102637465 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f13300541b0 con 0x7f1344077040 2024-09-10T09:54:25.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:25.993+0000 7f134c400640 1 -- 172.21.15.77:0/102637465 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f134410dc40 con 0x7f1344077040 2024-09-10T09:54:25.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:25.994+0000 7f1342000640 1 -- 172.21.15.77:0/102637465 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11989 (secure 0 0 0) 0x7f133009c5c0 con 0x7f1344077040 2024-09-10T09:54:25.996 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:54:25.996 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":28,"btime":"2024-09-10T09:53:16:873716+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3159289675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3159289675},{"type":"v1","addr":"172.21.15.77:6835","nonce":3159289675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1951962341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1951962341},{"type":"v1","addr":"172.21.15.77:6837","nonce":1951962341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:52:23.368252+0000","modified":"2024-09-10T09:53:16.873698+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24563,"mds_1":24524,"mds_2":24520,"mds_3":24502,"mds_4":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14487":{"gid":14487,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/4291121888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":4291121888},{"type":"v1","addr":"172.21.15.77:6839","nonce":4291121888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14499":{"gid":14499,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6841/445284194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":445284194},{"type":"v1","addr":"172.21.15.77:6841","nonce":445284194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1005054423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1005054423},{"type":"v1","addr":"172.21.15.134:6833","nonce":1005054423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6833/3716307744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":3716307744},{"type":"v1","addr":"172.21.15.120:6833","nonce":3716307744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/1319892544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1319892544},{"type":"v1","addr":"172.21.15.120:6835","nonce":1319892544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.134:6835/3293933699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3293933699},{"type":"v1","addr":"172.21.15.134:6835","nonce":3293933699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.134:6837/2461497091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2461497091},{"type":"v1","addr":"172.21.15.134:6837","nonce":2461497091}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":20,"addr":"172.21.15.134:6839/1965117680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1965117680},{"type":"v1","addr":"172.21.15.134:6839","nonce":1965117680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":20,"addr":"172.21.15.120:6837/3033280709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3033280709},{"type":"v1","addr":"172.21.15.120:6837","nonce":3033280709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.120:6839/1777710157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1777710157},{"type":"v1","addr":"172.21.15.120:6839","nonce":1777710157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24563,"qdb_cluster":[24563,24524,24520,24502,24496]},"id":1}]} 2024-09-10T09:54:25.997 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 28 2024-09-10T09:54:25.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:25.997+0000 7f134c400640 1 -- 172.21.15.77:0/102637465 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f132c0a2c10 msgr2=0x7f132c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:25.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:25.997+0000 7f134c400640 1 --2- 172.21.15.77:0/102637465 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f132c0a2c10 0x7f132c0a50d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1338009830 tx=0x7f13380058d0 comp rx=0 tx=0).stop 2024-09-10T09:54:25.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:25.997+0000 7f134c400640 1 -- 172.21.15.77:0/102637465 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1344077040 msgr2=0x7f13441a4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:25.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:25.997+0000 7f134c400640 1 --2- 172.21.15.77:0/102637465 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1344077040 0x7f13441a4d30 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f133000e9f0 tx=0x7f133000edf0 comp rx=0 tx=0).stop 2024-09-10T09:54:25.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:25.997+0000 7f134c400640 1 -- 172.21.15.77:0/102637465 shutdown_connections 2024-09-10T09:54:25.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:25.997+0000 7f134c400640 1 --2- 172.21.15.77:0/102637465 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f132c0a2c10 0x7f132c0a50d0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:25.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:25.997+0000 7f134c400640 1 --2- 172.21.15.77:0/102637465 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1344112a50 0x7f134419e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:25.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:25.997+0000 7f134c400640 1 --2- 172.21.15.77:0/102637465 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1344077040 0x7f13441a4d30 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:25.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:25.997+0000 7f134c400640 1 --2- 172.21.15.77:0/102637465 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1344076720 0x7f13441a47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:25.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:25.997+0000 7f134c400640 1 -- 172.21.15.77:0/102637465 >> 172.21.15.77:0/102637465 conn(0x7f13440ffa50 msgr2=0x7f1344101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:25.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:25.997+0000 7f134c400640 1 -- 172.21.15.77:0/102637465 shutdown_connections 2024-09-10T09:54:25.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:25.997+0000 7f134c400640 1 -- 172.21.15.77:0/102637465 wait complete. 2024-09-10T09:54:26.289 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:54:26 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:54:26.289 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:54:26 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:54:26.289 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:54:26 smithi077 ceph-mon[30319]: pgmap v369: 129 pgs: 129 active+clean; 645 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T09:54:26.289 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:54:26 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:54:26.289 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:54:26 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:54:26.290 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:54:26 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:54:26.290 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:54:26 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/102637465' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:54:26.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:54:26 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:54:26.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:54:26 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:54:26.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:54:26 smithi134 ceph-mon[35674]: pgmap v369: 129 pgs: 129 active+clean; 645 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T09:54:26.382 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:54:26 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:54:26.382 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:54:26 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:54:26.382 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:54:26 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:54:26.382 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:54:26 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/102637465' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:54:26.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:54:26 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:54:26.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:54:26 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:54:26.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:54:26 smithi120 ceph-mon[34770]: pgmap v369: 129 pgs: 129 active+clean; 645 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T09:54:26.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:54:26 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:54:26.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:54:26 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:54:26.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:54:26 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:54:26.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:54:26 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/102637465' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:54:26.768 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T09:54:27.136 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:54:27.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:27.688+0000 7ff08b200640 1 -- 172.21.15.77:0/793629453 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff08c103e30 msgr2=0x7ff08c104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:27.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:27.688+0000 7ff08b200640 1 --2- 172.21.15.77:0/793629453 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff08c103e30 0x7ff08c104210 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff07c009900 tx=0x7ff07c02a410 comp rx=0 tx=0).stop 2024-09-10T09:54:27.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:27.688+0000 7ff08b200640 1 -- 172.21.15.77:0/793629453 shutdown_connections 2024-09-10T09:54:27.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:27.688+0000 7ff08b200640 1 --2- 172.21.15.77:0/793629453 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff08c108f60 0x7ff08c110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:27.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:27.688+0000 7ff08b200640 1 --2- 172.21.15.77:0/793629453 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff08c1047e0 0x7ff08c108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:27.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:27.688+0000 7ff08b200640 1 --2- 172.21.15.77:0/793629453 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff08c103e30 0x7ff08c104210 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:27.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:27.688+0000 7ff08b200640 1 -- 172.21.15.77:0/793629453 >> 172.21.15.77:0/793629453 conn(0x7ff08c0ff910 msgr2=0x7ff08c101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:27.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:27.688+0000 7ff08b200640 1 -- 172.21.15.77:0/793629453 shutdown_connections 2024-09-10T09:54:27.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:27.688+0000 7ff08b200640 1 -- 172.21.15.77:0/793629453 wait complete. 2024-09-10T09:54:27.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:27.689+0000 7ff08b200640 1 Processor -- start 2024-09-10T09:54:27.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:27.689+0000 7ff08b200640 1 -- start start 2024-09-10T09:54:27.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:27.689+0000 7ff08b200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff08c103e30 0x7ff08c19a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:27.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:27.689+0000 7ff08b200640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff08c1047e0 0x7ff08c19a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:27.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:27.689+0000 7ff08b200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff08c108f60 0x7ff08c19ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:27.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:27.689+0000 7ff08b200640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff08c114010 con 0x7ff08c108f60 2024-09-10T09:54:27.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:27.689+0000 7ff08b200640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff08c113e90 con 0x7ff08c1047e0 2024-09-10T09:54:27.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:27.689+0000 7ff08b200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff08c114190 con 0x7ff08c103e30 2024-09-10T09:54:27.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:27.690+0000 7ff08a800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff08c108f60 0x7ff08c19ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:27.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:27.690+0000 7ff08a800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff08c108f60 0x7ff08c19ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:48480/0 (socket says 172.21.15.77:48480) 2024-09-10T09:54:27.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:27.690+0000 7ff08a800640 1 -- 172.21.15.77:0/56526798 learned_addr learned my addr 172.21.15.77:0/56526798 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:54:27.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:27.690+0000 7ff089e00640 1 --2- 172.21.15.77:0/56526798 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff08c103e30 0x7ff08c19a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:27.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:27.690+0000 7ff083e00640 1 --2- 172.21.15.77:0/56526798 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff08c1047e0 0x7ff08c19a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:27.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:27.690+0000 7ff08a800640 1 -- 172.21.15.77:0/56526798 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff08c103e30 msgr2=0x7ff08c19a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:27.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:27.690+0000 7ff08a800640 1 --2- 172.21.15.77:0/56526798 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff08c103e30 0x7ff08c19a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:27.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:27.690+0000 7ff08a800640 1 -- 172.21.15.77:0/56526798 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff08c1047e0 msgr2=0x7ff08c19a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:27.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:27.690+0000 7ff08a800640 1 --2- 172.21.15.77:0/56526798 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff08c1047e0 0x7ff08c19a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:27.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:27.690+0000 7ff08a800640 1 -- 172.21.15.77:0/56526798 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff08c19f460 con 0x7ff08c108f60 2024-09-10T09:54:27.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:27.691+0000 7ff08a800640 1 --2- 172.21.15.77:0/56526798 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff08c108f60 0x7ff08c19ed80 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff084009fd0 tx=0x7ff08400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:27.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:27.691+0000 7ff082a00640 1 -- 172.21.15.77:0/56526798 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff084017070 con 0x7ff08c108f60 2024-09-10T09:54:27.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:27.691+0000 7ff082a00640 1 -- 172.21.15.77:0/56526798 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff084003e50 con 0x7ff08c108f60 2024-09-10T09:54:27.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:27.691+0000 7ff08b200640 1 -- 172.21.15.77:0/56526798 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff08c19f750 con 0x7ff08c108f60 2024-09-10T09:54:27.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:27.691+0000 7ff082a00640 1 -- 172.21.15.77:0/56526798 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0840106c0 con 0x7ff08c108f60 2024-09-10T09:54:27.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:27.691+0000 7ff08b200640 1 -- 172.21.15.77:0/56526798 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff08c1a7030 con 0x7ff08c108f60 2024-09-10T09:54:27.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:27.692+0000 7ff08b200640 1 -- 172.21.15.77:0/56526798 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff08c105310 con 0x7ff08c108f60 2024-09-10T09:54:27.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:27.694+0000 7ff082a00640 1 -- 172.21.15.77:0/56526798 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ff084003970 con 0x7ff08c108f60 2024-09-10T09:54:27.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:27.695+0000 7ff082a00640 1 --2- 172.21.15.77:0/56526798 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ff0700a2c10 0x7ff0700a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:27.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:27.695+0000 7ff082a00640 1 -- 172.21.15.77:0/56526798 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7ff0840cf2e0 con 0x7ff08c108f60 2024-09-10T09:54:27.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:27.695+0000 7ff089e00640 1 --2- 172.21.15.77:0/56526798 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ff0700a2c10 0x7ff0700a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:27.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:27.696+0000 7ff089e00640 1 --2- 172.21.15.77:0/56526798 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ff0700a2c10 0x7ff0700a50d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff07c000c00 tx=0x7ff07c005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:27.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:27.699+0000 7ff082a00640 1 -- 172.21.15.77:0/56526798 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff08409b640 con 0x7ff08c108f60 2024-09-10T09:54:27.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:27.877+0000 7ff08b200640 1 -- 172.21.15.77:0/56526798 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff08c10c1a0 con 0x7ff08c108f60 2024-09-10T09:54:27.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:27.879+0000 7ff082a00640 1 -- 172.21.15.77:0/56526798 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+24941 (secure 0 0 0) 0x7ff084097020 con 0x7ff08c108f60 2024-09-10T09:54:27.881 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:54:27.882 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":96,"fsid":"14a55226-6f59-11ef-bcea-c7b262605968","created":"2024-09-10T09:44:07.359542+0000","modified":"2024-09-10T09:54:00.837074+0000","last_up_change":"2024-09-10T09:51:26.731971+0000","last_in_change":"2024-09-10T09:51:03.857873+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:47:24.174660+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T09:52:18.766738+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T09:52:21.284745+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T09:52:29.846594+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8d46f634-114c-4926-afdc-f8fa37e40336","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":1509649079},{"type":"v1","addr":"172.21.15.77:6803","nonce":1509649079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":1509649079},{"type":"v1","addr":"172.21.15.77:6805","nonce":1509649079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":1509649079},{"type":"v1","addr":"172.21.15.77:6809","nonce":1509649079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":1509649079},{"type":"v1","addr":"172.21.15.77:6807","nonce":1509649079}]},"public_addr":"172.21.15.77:6803/1509649079","cluster_addr":"172.21.15.77:6805/1509649079","heartbeat_back_addr":"172.21.15.77:6809/1509649079","heartbeat_front_addr":"172.21.15.77:6807/1509649079","state":["exists","up"]},{"osd":1,"uuid":"b6570764-736a-49e4-877a-1965e5812d1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":4243920698},{"type":"v1","addr":"172.21.15.120:6801","nonce":4243920698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":4243920698},{"type":"v1","addr":"172.21.15.120:6803","nonce":4243920698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":4243920698},{"type":"v1","addr":"172.21.15.120:6807","nonce":4243920698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":4243920698},{"type":"v1","addr":"172.21.15.120:6805","nonce":4243920698}]},"public_addr":"172.21.15.120:6801/4243920698","cluster_addr":"172.21.15.120:6803/4243920698","heartbeat_back_addr":"172.21.15.120:6807/4243920698","heartbeat_front_addr":"172.21.15.120:6805/4243920698","state":["exists","up"]},{"osd":2,"uuid":"5e83fe13-d1ee-483e-a575-e8041ec836f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":3280466823},{"type":"v1","addr":"172.21.15.134:6801","nonce":3280466823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":3280466823},{"type":"v1","addr":"172.21.15.134:6803","nonce":3280466823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":3280466823},{"type":"v1","addr":"172.21.15.134:6807","nonce":3280466823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":3280466823},{"type":"v1","addr":"172.21.15.134:6805","nonce":3280466823}]},"public_addr":"172.21.15.134:6801/3280466823","cluster_addr":"172.21.15.134:6803/3280466823","heartbeat_back_addr":"172.21.15.134:6807/3280466823","heartbeat_front_addr":"172.21.15.134:6805/3280466823","state":["exists","up"]},{"osd":3,"uuid":"5affc6a7-1cae-48fe-b7db-befc2fcafecb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":1629416786},{"type":"v1","addr":"172.21.15.77:6811","nonce":1629416786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":1629416786},{"type":"v1","addr":"172.21.15.77:6813","nonce":1629416786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":1629416786},{"type":"v1","addr":"172.21.15.77:6817","nonce":1629416786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":1629416786},{"type":"v1","addr":"172.21.15.77:6815","nonce":1629416786}]},"public_addr":"172.21.15.77:6811/1629416786","cluster_addr":"172.21.15.77:6813/1629416786","heartbeat_back_addr":"172.21.15.77:6817/1629416786","heartbeat_front_addr":"172.21.15.77:6815/1629416786","state":["exists","up"]},{"osd":4,"uuid":"c826978f-40b0-43ae-adbb-5c93b32a68ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":3162658038},{"type":"v1","addr":"172.21.15.120:6809","nonce":3162658038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":3162658038},{"type":"v1","addr":"172.21.15.120:6811","nonce":3162658038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":3162658038},{"type":"v1","addr":"172.21.15.120:6815","nonce":3162658038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":3162658038},{"type":"v1","addr":"172.21.15.120:6813","nonce":3162658038}]},"public_addr":"172.21.15.120:6809/3162658038","cluster_addr":"172.21.15.120:6811/3162658038","heartbeat_back_addr":"172.21.15.120:6815/3162658038","heartbeat_front_addr":"172.21.15.120:6813/3162658038","state":["exists","up"]},{"osd":5,"uuid":"be51c7fb-2765-46a3-9861-c950f63de148","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":4081444621},{"type":"v1","addr":"172.21.15.134:6809","nonce":4081444621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":4081444621},{"type":"v1","addr":"172.21.15.134:6811","nonce":4081444621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":4081444621},{"type":"v1","addr":"172.21.15.134:6815","nonce":4081444621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":4081444621},{"type":"v1","addr":"172.21.15.134:6813","nonce":4081444621}]},"public_addr":"172.21.15.134:6809/4081444621","cluster_addr":"172.21.15.134:6811/4081444621","heartbeat_back_addr":"172.21.15.134:6815/4081444621","heartbeat_front_addr":"172.21.15.134:6813/4081444621","state":["exists","up"]},{"osd":6,"uuid":"4eb403ab-473b-41cc-803d-18c908f768a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":879565639},{"type":"v1","addr":"172.21.15.77:6819","nonce":879565639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":879565639},{"type":"v1","addr":"172.21.15.77:6821","nonce":879565639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":879565639},{"type":"v1","addr":"172.21.15.77:6825","nonce":879565639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":879565639},{"type":"v1","addr":"172.21.15.77:6823","nonce":879565639}]},"public_addr":"172.21.15.77:6819/879565639","cluster_addr":"172.21.15.77:6821/879565639","heartbeat_back_addr":"172.21.15.77:6825/879565639","heartbeat_front_addr":"172.21.15.77:6823/879565639","state":["exists","up"]},{"osd":7,"uuid":"ba982bd0-c236-456c-9889-61087d79c1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":2688667400},{"type":"v1","addr":"172.21.15.120:6817","nonce":2688667400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":2688667400},{"type":"v1","addr":"172.21.15.120:6819","nonce":2688667400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":2688667400},{"type":"v1","addr":"172.21.15.120:6823","nonce":2688667400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2688667400},{"type":"v1","addr":"172.21.15.120:6821","nonce":2688667400}]},"public_addr":"172.21.15.120:6817/2688667400","cluster_addr":"172.21.15.120:6819/2688667400","heartbeat_back_addr":"172.21.15.120:6823/2688667400","heartbeat_front_addr":"172.21.15.120:6821/2688667400","state":["exists","up"]},{"osd":8,"uuid":"f2e8ca43-57c9-4933-b5f7-73f4d1b3e08b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2081686544},{"type":"v1","addr":"172.21.15.134:6817","nonce":2081686544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2081686544},{"type":"v1","addr":"172.21.15.134:6819","nonce":2081686544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2081686544},{"type":"v1","addr":"172.21.15.134:6823","nonce":2081686544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2081686544},{"type":"v1","addr":"172.21.15.134:6821","nonce":2081686544}]},"public_addr":"172.21.15.134:6817/2081686544","cluster_addr":"172.21.15.134:6819/2081686544","heartbeat_back_addr":"172.21.15.134:6823/2081686544","heartbeat_front_addr":"172.21.15.134:6821/2081686544","state":["exists","up"]},{"osd":9,"uuid":"9486f8b5-0614-4208-87ec-117741f6b622","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":1210619387},{"type":"v1","addr":"172.21.15.77:6827","nonce":1210619387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":1210619387},{"type":"v1","addr":"172.21.15.77:6829","nonce":1210619387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":1210619387},{"type":"v1","addr":"172.21.15.77:6833","nonce":1210619387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":1210619387},{"type":"v1","addr":"172.21.15.77:6831","nonce":1210619387}]},"public_addr":"172.21.15.77:6827/1210619387","cluster_addr":"172.21.15.77:6829/1210619387","heartbeat_back_addr":"172.21.15.77:6833/1210619387","heartbeat_front_addr":"172.21.15.77:6831/1210619387","state":["exists","up"]},{"osd":10,"uuid":"8e8192b0-1398-4d4d-9756-762b2f874d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":754733885},{"type":"v1","addr":"172.21.15.120:6825","nonce":754733885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":754733885},{"type":"v1","addr":"172.21.15.120:6827","nonce":754733885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":754733885},{"type":"v1","addr":"172.21.15.120:6831","nonce":754733885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":754733885},{"type":"v1","addr":"172.21.15.120:6829","nonce":754733885}]},"public_addr":"172.21.15.120:6825/754733885","cluster_addr":"172.21.15.120:6827/754733885","heartbeat_back_addr":"172.21.15.120:6831/754733885","heartbeat_front_addr":"172.21.15.120:6829/754733885","state":["exists","up"]},{"osd":11,"uuid":"85c1ccd4-5147-4995-896e-fe977f73f749","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1602213143},{"type":"v1","addr":"172.21.15.134:6825","nonce":1602213143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1602213143},{"type":"v1","addr":"172.21.15.134:6827","nonce":1602213143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1602213143},{"type":"v1","addr":"172.21.15.134:6831","nonce":1602213143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1602213143},{"type":"v1","addr":"172.21.15.134:6829","nonce":1602213143}]},"public_addr":"172.21.15.134:6825/1602213143","cluster_addr":"172.21.15.134:6827/1602213143","heartbeat_back_addr":"172.21.15.134:6831/1602213143","heartbeat_front_addr":"172.21.15.134:6829/1602213143","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:46:30.293451+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:46:56.554801+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:47:21.476225+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:47:49.737332+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:48:16.092181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:48:41.240815+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:49:11.683950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:49:38.859255+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:50:04.105401+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:50:33.847810+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:50:59.901797+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:51:25.241507+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.2","mappings":[{"from":7,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":7}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":5,"to":8}]},{"pgid":"4.6","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.77:0/1697982655":"2024-09-11T09:44:59.575663+0000","172.21.15.77:6801/1527070442":"2024-09-11T09:44:59.575663+0000","172.21.15.77:0/3146688196":"2024-09-11T09:44:32.185342+0000","172.21.15.77:0/371773177":"2024-09-11T09:44:59.575663+0000","172.21.15.77:0/1925456910":"2024-09-11T09:44:59.575663+0000","172.21.15.77:0/138545832":"2024-09-11T09:44:32.185342+0000","172.21.15.77:6801/3235900503":"2024-09-11T09:44:32.185342+0000","172.21.15.77:6800/1527070442":"2024-09-11T09:44:59.575663+0000","172.21.15.77:6800/3235900503":"2024-09-11T09:44:32.185342+0000","172.21.15.77:0/1647092548":"2024-09-11T09:44:32.185342+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T09:54:27.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:27.881+0000 7ff08b200640 1 -- 172.21.15.77:0/56526798 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ff0700a2c10 msgr2=0x7ff0700a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:27.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:27.881+0000 7ff08b200640 1 --2- 172.21.15.77:0/56526798 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ff0700a2c10 0x7ff0700a50d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff07c000c00 tx=0x7ff07c005b50 comp rx=0 tx=0).stop 2024-09-10T09:54:27.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:27.881+0000 7ff08b200640 1 -- 172.21.15.77:0/56526798 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff08c108f60 msgr2=0x7ff08c19ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:27.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:27.881+0000 7ff08b200640 1 --2- 172.21.15.77:0/56526798 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff08c108f60 0x7ff08c19ed80 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff084009fd0 tx=0x7ff08400b260 comp rx=0 tx=0).stop 2024-09-10T09:54:27.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:27.882+0000 7ff08b200640 1 -- 172.21.15.77:0/56526798 shutdown_connections 2024-09-10T09:54:27.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:27.882+0000 7ff08b200640 1 --2- 172.21.15.77:0/56526798 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ff0700a2c10 0x7ff0700a50d0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:27.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:27.882+0000 7ff08b200640 1 --2- 172.21.15.77:0/56526798 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff08c108f60 0x7ff08c19ed80 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:27.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:27.882+0000 7ff08b200640 1 --2- 172.21.15.77:0/56526798 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff08c1047e0 0x7ff08c19a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:27.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:27.882+0000 7ff08b200640 1 --2- 172.21.15.77:0/56526798 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff08c103e30 0x7ff08c19a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:27.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:27.882+0000 7ff08b200640 1 -- 172.21.15.77:0/56526798 >> 172.21.15.77:0/56526798 conn(0x7ff08c0ff910 msgr2=0x7ff08c100c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:27.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:27.882+0000 7ff08b200640 1 -- 172.21.15.77:0/56526798 shutdown_connections 2024-09-10T09:54:27.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:27.882+0000 7ff08b200640 1 -- 172.21.15.77:0/56526798 wait complete. 2024-09-10T09:54:28.575 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:54:28.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:54:28 smithi077 ceph-mon[30319]: pgmap v370: 129 pgs: 129 active+clean; 645 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-10T09:54:28.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:54:28 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/56526798' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T09:54:28.943 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:54:29.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:54:28 smithi134 ceph-mon[35674]: pgmap v370: 129 pgs: 129 active+clean; 645 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-10T09:54:29.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:54:28 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/56526798' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T09:54:29.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:54:28 smithi120 ceph-mon[34770]: pgmap v370: 129 pgs: 129 active+clean; 645 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-10T09:54:29.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:54:28 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/56526798' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T09:54:29.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:29.520+0000 7fb776200640 1 -- 172.21.15.77:0/3335129983 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb770109120 msgr2=0x7fb770110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:29.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:29.520+0000 7fb776200640 1 --2- 172.21.15.77:0/3335129983 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb770109120 0x7fb770110ca0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb768009ca0 tx=0x7fb76802a8c0 comp rx=0 tx=0).stop 2024-09-10T09:54:29.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:29.520+0000 7fb776200640 1 -- 172.21.15.77:0/3335129983 shutdown_connections 2024-09-10T09:54:29.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:29.520+0000 7fb776200640 1 --2- 172.21.15.77:0/3335129983 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb770109120 0x7fb770110ca0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:29.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:29.520+0000 7fb776200640 1 --2- 172.21.15.77:0/3335129983 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb7701049a0 0x7fb7701089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:29.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:29.520+0000 7fb776200640 1 --2- 172.21.15.77:0/3335129983 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb770103ff0 0x7fb7701043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:29.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:29.520+0000 7fb776200640 1 -- 172.21.15.77:0/3335129983 >> 172.21.15.77:0/3335129983 conn(0x7fb7700ffa90 msgr2=0x7fb770101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:29.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:29.520+0000 7fb776200640 1 -- 172.21.15.77:0/3335129983 shutdown_connections 2024-09-10T09:54:29.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:29.520+0000 7fb776200640 1 -- 172.21.15.77:0/3335129983 wait complete. 2024-09-10T09:54:29.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:29.521+0000 7fb776200640 1 Processor -- start 2024-09-10T09:54:29.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:29.521+0000 7fb776200640 1 -- start start 2024-09-10T09:54:29.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:29.522+0000 7fb776200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb770103ff0 0x7fb77019a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:29.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:29.522+0000 7fb776200640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb7701049a0 0x7fb77019ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:29.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:29.522+0000 7fb776200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb770109120 0x7fb77019ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:29.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:29.522+0000 7fb776200640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb7701141f0 con 0x7fb770103ff0 2024-09-10T09:54:29.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:29.522+0000 7fb776200640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb770114070 con 0x7fb7701049a0 2024-09-10T09:54:29.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:29.522+0000 7fb776200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb770114370 con 0x7fb770109120 2024-09-10T09:54:29.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:29.522+0000 7fb76f400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb770103ff0 0x7fb77019a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:29.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:29.522+0000 7fb76f400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb770103ff0 0x7fb77019a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:48498/0 (socket says 172.21.15.77:48498) 2024-09-10T09:54:29.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:29.522+0000 7fb76f400640 1 -- 172.21.15.77:0/2743078659 learned_addr learned my addr 172.21.15.77:0/2743078659 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:54:29.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:29.522+0000 7fb76ea00640 1 --2- 172.21.15.77:0/2743078659 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb7701049a0 0x7fb77019ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:29.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:29.522+0000 7fb76fe00640 1 --2- 172.21.15.77:0/2743078659 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb770109120 0x7fb77019ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:29.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:29.523+0000 7fb76f400640 1 -- 172.21.15.77:0/2743078659 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb770109120 msgr2=0x7fb77019ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:29.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:29.523+0000 7fb76f400640 1 --2- 172.21.15.77:0/2743078659 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb770109120 0x7fb77019ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:29.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:29.523+0000 7fb76f400640 1 -- 172.21.15.77:0/2743078659 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb7701049a0 msgr2=0x7fb77019ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:29.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:29.523+0000 7fb76f400640 1 --2- 172.21.15.77:0/2743078659 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb7701049a0 0x7fb77019ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:29.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:29.523+0000 7fb76f400640 1 -- 172.21.15.77:0/2743078659 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb77019f5e0 con 0x7fb770103ff0 2024-09-10T09:54:29.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:29.523+0000 7fb76ea00640 1 --2- 172.21.15.77:0/2743078659 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb7701049a0 0x7fb77019ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:54:29.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:29.523+0000 7fb76f400640 1 --2- 172.21.15.77:0/2743078659 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb770103ff0 0x7fb77019a630 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb76000b4a0 tx=0x7fb76000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:29.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:29.523+0000 7fb767e00640 1 -- 172.21.15.77:0/2743078659 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb760014070 con 0x7fb770103ff0 2024-09-10T09:54:29.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:29.523+0000 7fb767e00640 1 -- 172.21.15.77:0/2743078659 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb760010af0 con 0x7fb770103ff0 2024-09-10T09:54:29.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:29.523+0000 7fb776200640 1 -- 172.21.15.77:0/2743078659 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb77019f8d0 con 0x7fb770103ff0 2024-09-10T09:54:29.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:29.523+0000 7fb767e00640 1 -- 172.21.15.77:0/2743078659 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb760017d10 con 0x7fb770103ff0 2024-09-10T09:54:29.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:29.524+0000 7fb776200640 1 -- 172.21.15.77:0/2743078659 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7701a71b0 con 0x7fb770103ff0 2024-09-10T09:54:29.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:29.525+0000 7fb776200640 1 -- 172.21.15.77:0/2743078659 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7701043d0 con 0x7fb770103ff0 2024-09-10T09:54:29.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:29.527+0000 7fb767e00640 1 -- 172.21.15.77:0/2743078659 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fb760017070 con 0x7fb770103ff0 2024-09-10T09:54:29.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:29.530+0000 7fb767e00640 1 --2- 172.21.15.77:0/2743078659 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fb7540a2c90 0x7fb7540a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:29.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:29.530+0000 7fb767e00640 1 -- 172.21.15.77:0/2743078659 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fb7600cebc0 con 0x7fb770103ff0 2024-09-10T09:54:29.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:29.530+0000 7fb76ea00640 1 --2- 172.21.15.77:0/2743078659 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fb7540a2c90 0x7fb7540a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:29.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:29.531+0000 7fb76ea00640 1 --2- 172.21.15.77:0/2743078659 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fb7540a2c90 0x7fb7540a5150 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb75c0095f0 tx=0x7fb75c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:29.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:29.532+0000 7fb767e00640 1 -- 172.21.15.77:0/2743078659 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb76009b030 con 0x7fb770103ff0 2024-09-10T09:54:29.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:29.787+0000 7fb776200640 1 -- 172.21.15.77:0/2743078659 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb77010c360 con 0x7fb770103ff0 2024-09-10T09:54:29.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:29.788+0000 7fb767e00640 1 -- 172.21.15.77:0/2743078659 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11989 (secure 0 0 0) 0x7fb76009f330 con 0x7fb770103ff0 2024-09-10T09:54:29.789 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:54:29.789 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":28,"btime":"2024-09-10T09:53:16:873716+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3159289675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3159289675},{"type":"v1","addr":"172.21.15.77:6835","nonce":3159289675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1951962341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1951962341},{"type":"v1","addr":"172.21.15.77:6837","nonce":1951962341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:52:23.368252+0000","modified":"2024-09-10T09:53:16.873698+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24563,"mds_1":24524,"mds_2":24520,"mds_3":24502,"mds_4":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14487":{"gid":14487,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/4291121888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":4291121888},{"type":"v1","addr":"172.21.15.77:6839","nonce":4291121888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14499":{"gid":14499,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6841/445284194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":445284194},{"type":"v1","addr":"172.21.15.77:6841","nonce":445284194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1005054423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1005054423},{"type":"v1","addr":"172.21.15.134:6833","nonce":1005054423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6833/3716307744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":3716307744},{"type":"v1","addr":"172.21.15.120:6833","nonce":3716307744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/1319892544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1319892544},{"type":"v1","addr":"172.21.15.120:6835","nonce":1319892544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.134:6835/3293933699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3293933699},{"type":"v1","addr":"172.21.15.134:6835","nonce":3293933699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.134:6837/2461497091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2461497091},{"type":"v1","addr":"172.21.15.134:6837","nonce":2461497091}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":20,"addr":"172.21.15.134:6839/1965117680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1965117680},{"type":"v1","addr":"172.21.15.134:6839","nonce":1965117680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":20,"addr":"172.21.15.120:6837/3033280709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3033280709},{"type":"v1","addr":"172.21.15.120:6837","nonce":3033280709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.120:6839/1777710157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1777710157},{"type":"v1","addr":"172.21.15.120:6839","nonce":1777710157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24563,"qdb_cluster":[24563,24524,24520,24502,24496]},"id":1}]} 2024-09-10T09:54:29.789 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 28 2024-09-10T09:54:29.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:29.790+0000 7fb776200640 1 -- 172.21.15.77:0/2743078659 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fb7540a2c90 msgr2=0x7fb7540a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:29.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:29.790+0000 7fb776200640 1 --2- 172.21.15.77:0/2743078659 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fb7540a2c90 0x7fb7540a5150 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb75c0095f0 tx=0x7fb75c005f90 comp rx=0 tx=0).stop 2024-09-10T09:54:29.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:29.790+0000 7fb776200640 1 -- 172.21.15.77:0/2743078659 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb770103ff0 msgr2=0x7fb77019a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:29.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:29.790+0000 7fb776200640 1 --2- 172.21.15.77:0/2743078659 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb770103ff0 0x7fb77019a630 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb76000b4a0 tx=0x7fb76000b960 comp rx=0 tx=0).stop 2024-09-10T09:54:29.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:29.790+0000 7fb776200640 1 -- 172.21.15.77:0/2743078659 shutdown_connections 2024-09-10T09:54:29.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:29.790+0000 7fb776200640 1 --2- 172.21.15.77:0/2743078659 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fb7540a2c90 0x7fb7540a5150 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:29.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:29.790+0000 7fb776200640 1 --2- 172.21.15.77:0/2743078659 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb770109120 0x7fb77019ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:29.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:29.790+0000 7fb776200640 1 --2- 172.21.15.77:0/2743078659 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb7701049a0 0x7fb77019ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:29.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:29.790+0000 7fb776200640 1 --2- 172.21.15.77:0/2743078659 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb770103ff0 0x7fb77019a630 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:29.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:29.790+0000 7fb776200640 1 -- 172.21.15.77:0/2743078659 >> 172.21.15.77:0/2743078659 conn(0x7fb7700ffa90 msgr2=0x7fb770100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:29.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:29.790+0000 7fb776200640 1 -- 172.21.15.77:0/2743078659 shutdown_connections 2024-09-10T09:54:29.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:29.790+0000 7fb776200640 1 -- 172.21.15.77:0/2743078659 wait complete. 2024-09-10T09:54:30.557 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-10T09:52:23.368252+0000', 'modified': '2024-09-10T09:53:16.873698+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24563, 'mds_1': 24524, 'mds_2': 24520, 'mds_3': 24502, 'mds_4': 24496}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14487': {'gid': 14487, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.77:6839/4291121888', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.77:6838', 'nonce': 4291121888}, {'type': 'v1', 'addr': '172.21.15.77:6839', 'nonce': 4291121888}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14499': {'gid': 14499, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.77:6841/445284194', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.77:6840', 'nonce': 445284194}, {'type': 'v1', 'addr': '172.21.15.77:6841', 'nonce': 445284194}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24448': {'gid': 24448, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.134:6833/1005054423', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6832', 'nonce': 1005054423}, {'type': 'v1', 'addr': '172.21.15.134:6833', 'nonce': 1005054423}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24458': {'gid': 24458, 'name': 'k', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.120:6833/3716307744', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6832', 'nonce': 3716307744}, {'type': 'v1', 'addr': '172.21.15.120:6833', 'nonce': 3716307744}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24494': {'gid': 24494, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.120:6835/1319892544', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6834', 'nonce': 1319892544}, {'type': 'v1', 'addr': '172.21.15.120:6835', 'nonce': 1319892544}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24496': {'gid': 24496, 'name': 'f', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 22, 'addr': '172.21.15.134:6835/3293933699', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6834', 'nonce': 3293933699}, {'type': 'v1', 'addr': '172.21.15.134:6835', 'nonce': 3293933699}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24502': {'gid': 24502, 'name': 'c', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 21, 'addr': '172.21.15.134:6837/2461497091', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6836', 'nonce': 2461497091}, {'type': 'v1', 'addr': '172.21.15.134:6837', 'nonce': 2461497091}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24520': {'gid': 24520, 'name': 'i', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 20, 'addr': '172.21.15.134:6839/1965117680', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6838', 'nonce': 1965117680}, {'type': 'v1', 'addr': '172.21.15.134:6839', 'nonce': 1965117680}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24524': {'gid': 24524, 'name': 'h', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 20, 'addr': '172.21.15.120:6837/3033280709', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6836', 'nonce': 3033280709}, {'type': 'v1', 'addr': '172.21.15.120:6837', 'nonce': 3033280709}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24563': {'gid': 24563, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.120:6839/1777710157', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6838', 'nonce': 1777710157}, {'type': 'v1', 'addr': '172.21.15.120:6839', 'nonce': 1777710157}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24563, 'qdb_cluster': [24563, 24524, 24520, 24502, 24496]} 2024-09-10T09:54:30.557 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-10T09:54:30.557 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.f status 2024-09-10T09:54:30.803 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:54:30 smithi077 ceph-mon[30319]: pgmap v371: 129 pgs: 129 active+clean; 645 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-09-10T09:54:30.803 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:54:30 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/2743078659' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:54:30.937 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:54:31.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:54:30 smithi134 ceph-mon[35674]: pgmap v371: 129 pgs: 129 active+clean; 645 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-09-10T09:54:31.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:54:30 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/2743078659' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:54:31.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:54:30 smithi120 ceph-mon[34770]: pgmap v371: 129 pgs: 129 active+clean; 645 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-09-10T09:54:31.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:54:30 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/2743078659' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:54:31.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.553+0000 7f99c7e00640 1 -- 172.21.15.77:0/2411509089 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f99c81047e0 msgr2=0x7f99c8108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:31.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.553+0000 7f99c7e00640 1 --2- 172.21.15.77:0/2411509089 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f99c81047e0 0x7f99c8108830 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f99b0009900 tx=0x7f99b002a410 comp rx=0 tx=0).stop 2024-09-10T09:54:31.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.553+0000 7f99c7e00640 1 -- 172.21.15.77:0/2411509089 shutdown_connections 2024-09-10T09:54:31.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.553+0000 7f99c7e00640 1 --2- 172.21.15.77:0/2411509089 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f99c8108f60 0x7f99c8110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:31.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.553+0000 7f99c7e00640 1 --2- 172.21.15.77:0/2411509089 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f99c81047e0 0x7f99c8108830 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:31.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.553+0000 7f99c7e00640 1 --2- 172.21.15.77:0/2411509089 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99c8103e30 0x7f99c8104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:31.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.553+0000 7f99c7e00640 1 -- 172.21.15.77:0/2411509089 >> 172.21.15.77:0/2411509089 conn(0x7f99c80ff8d0 msgr2=0x7f99c8101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:31.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.553+0000 7f99c7e00640 1 -- 172.21.15.77:0/2411509089 shutdown_connections 2024-09-10T09:54:31.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.553+0000 7f99c7e00640 1 -- 172.21.15.77:0/2411509089 wait complete. 2024-09-10T09:54:31.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.554+0000 7f99c7e00640 1 Processor -- start 2024-09-10T09:54:31.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.554+0000 7f99c7e00640 1 -- start start 2024-09-10T09:54:31.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.555+0000 7f99c7e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f99c8103e30 0x7f99c819a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:31.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.555+0000 7f99c7e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99c81047e0 0x7f99c819aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:31.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.555+0000 7f99c7e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f99c8108f60 0x7f99c819ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:31.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.555+0000 7f99c7e00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f99c8113fc0 con 0x7f99c8103e30 2024-09-10T09:54:31.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.555+0000 7f99c7e00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f99c8113e40 con 0x7f99c8108f60 2024-09-10T09:54:31.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.555+0000 7f99c7e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f99c8114140 con 0x7f99c81047e0 2024-09-10T09:54:31.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.555+0000 7f99c6a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f99c8103e30 0x7f99c819a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:31.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.555+0000 7f99c6a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f99c8103e30 0x7f99c819a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:48502/0 (socket says 172.21.15.77:48502) 2024-09-10T09:54:31.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.555+0000 7f99c6a00640 1 -- 172.21.15.77:0/3514279949 learned_addr learned my addr 172.21.15.77:0/3514279949 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:54:31.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.555+0000 7f99c7400640 1 --2- 172.21.15.77:0/3514279949 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f99c8108f60 0x7f99c819ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:31.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.555+0000 7f99be000640 1 --2- 172.21.15.77:0/3514279949 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99c81047e0 0x7f99c819aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:31.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.556+0000 7f99c6a00640 1 -- 172.21.15.77:0/3514279949 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99c81047e0 msgr2=0x7f99c819aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:31.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.556+0000 7f99c6a00640 1 --2- 172.21.15.77:0/3514279949 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99c81047e0 0x7f99c819aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:31.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.556+0000 7f99c6a00640 1 -- 172.21.15.77:0/3514279949 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f99c8108f60 msgr2=0x7f99c819ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:31.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.556+0000 7f99c6a00640 1 --2- 172.21.15.77:0/3514279949 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f99c8108f60 0x7f99c819ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:31.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.556+0000 7f99c6a00640 1 -- 172.21.15.77:0/3514279949 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99c819f470 con 0x7f99c8103e30 2024-09-10T09:54:31.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.556+0000 7f99c6a00640 1 --2- 172.21.15.77:0/3514279949 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f99c8103e30 0x7f99c819a4c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f99b800e9f0 tx=0x7f99b800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:31.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.556+0000 7f99bfe00640 1 -- 172.21.15.77:0/3514279949 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b800cd80 con 0x7f99c8103e30 2024-09-10T09:54:31.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.556+0000 7f99bfe00640 1 -- 172.21.15.77:0/3514279949 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99b8005500 con 0x7f99c8103e30 2024-09-10T09:54:31.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.556+0000 7f99c7e00640 1 -- 172.21.15.77:0/3514279949 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99c819f760 con 0x7f99c8103e30 2024-09-10T09:54:31.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.556+0000 7f99bfe00640 1 -- 172.21.15.77:0/3514279949 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b8010690 con 0x7f99c8103e30 2024-09-10T09:54:31.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.556+0000 7f99c7e00640 1 -- 172.21.15.77:0/3514279949 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99c81a7040 con 0x7f99c8103e30 2024-09-10T09:54:31.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.559+0000 7f99d060a740 1 Processor -- start 2024-09-10T09:54:31.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.559+0000 7f99bfe00640 1 -- 172.21.15.77:0/3514279949 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f99b80046c0 con 0x7f99c8103e30 2024-09-10T09:54:31.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.560+0000 7f99d060a740 1 -- start start 2024-09-10T09:54:31.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.560+0000 7f99d060a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652e9e0a4e0 0x5652e9e0a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:31.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.560+0000 7f99d060a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5652e9e0ae00 0x5652e9e15180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:31.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.560+0000 7f99d060a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652e9e156c0 0x5652e9e17ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:31.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.560+0000 7f99d060a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5652e9d700b0 con 0x5652e9e0a4e0 2024-09-10T09:54:31.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.560+0000 7f99d060a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5652e9d17ab0 con 0x5652e9e0ae00 2024-09-10T09:54:31.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.560+0000 7f99d060a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5652e9d6c070 con 0x5652e9e156c0 2024-09-10T09:54:31.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.560+0000 7f99c7400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652e9e0a4e0 0x5652e9e0a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:31.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.560+0000 7f99c7400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652e9e0a4e0 0x5652e9e0a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:48518/0 (socket says 172.21.15.77:48518) 2024-09-10T09:54:31.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.560+0000 7f99c7400640 1 -- 172.21.15.77:0/738711281 learned_addr learned my addr 172.21.15.77:0/738711281 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:54:31.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.560+0000 7f99be000640 1 --2- 172.21.15.77:0/738711281 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652e9e156c0 0x5652e9e17ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:31.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.560+0000 7f99c6a00640 1 --2- 172.21.15.77:0/738711281 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5652e9e0ae00 0x5652e9e15180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:31.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.561+0000 7f99c7400640 1 -- 172.21.15.77:0/738711281 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652e9e156c0 msgr2=0x5652e9e17ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:31.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.561+0000 7f99c7400640 1 --2- 172.21.15.77:0/738711281 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652e9e156c0 0x5652e9e17ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:31.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.561+0000 7f99c7400640 1 -- 172.21.15.77:0/738711281 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5652e9e0ae00 msgr2=0x5652e9e15180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:31.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.561+0000 7f99c7400640 1 --2- 172.21.15.77:0/738711281 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5652e9e0ae00 0x5652e9e15180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:31.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.561+0000 7f99c7400640 1 -- 172.21.15.77:0/738711281 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652e9d244a0 con 0x5652e9e0a4e0 2024-09-10T09:54:31.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.561+0000 7f99bfe00640 1 --2- 172.21.15.77:0/3514279949 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f99ac0a2b80 0x7f99ac0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:31.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.561+0000 7f99c7400640 1 --2- 172.21.15.77:0/738711281 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652e9e0a4e0 0x5652e9e0a8c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f99c0009f50 tx=0x7f99c00118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=284d184b19ba209 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:31.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.561+0000 7f99bfe00640 1 -- 172.21.15.77:0/3514279949 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f99b80cf870 con 0x7f99c8103e30 2024-09-10T09:54:31.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.561+0000 7f99c6a00640 1 --2- 172.21.15.77:0/3514279949 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f99ac0a2b80 0x7f99ac0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:31.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.561+0000 7f99b7e00640 1 -- 172.21.15.77:0/738711281 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c0004b20 con 0x5652e9e0a4e0 2024-09-10T09:54:31.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.562+0000 7f99b7e00640 1 -- 172.21.15.77:0/738711281 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99c0004cc0 con 0x5652e9e0a4e0 2024-09-10T09:54:31.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.562+0000 7f99b7e00640 1 -- 172.21.15.77:0/738711281 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c0005020 con 0x5652e9e0a4e0 2024-09-10T09:54:31.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.562+0000 7f99d060a740 1 -- 172.21.15.77:0/738711281 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652e9e0a4e0 msgr2=0x5652e9e0a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:31.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.562+0000 7f99c6a00640 1 --2- 172.21.15.77:0/3514279949 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f99ac0a2b80 0x7f99ac0a5040 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f99b8098650 tx=0x7f99b80d5530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:31.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.562+0000 7f99d060a740 1 --2- 172.21.15.77:0/738711281 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652e9e0a4e0 0x5652e9e0a8c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f99c0009f50 tx=0x7f99c00118d0 comp rx=0 tx=0).stop 2024-09-10T09:54:31.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.562+0000 7f99d060a740 1 -- 172.21.15.77:0/738711281 shutdown_connections 2024-09-10T09:54:31.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.562+0000 7f99d060a740 1 --2- 172.21.15.77:0/738711281 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652e9e156c0 0x5652e9e17ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:31.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.562+0000 7f99d060a740 1 --2- 172.21.15.77:0/738711281 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5652e9e0ae00 0x5652e9e15180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:31.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.562+0000 7f99d060a740 1 --2- 172.21.15.77:0/738711281 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652e9e0a4e0 0x5652e9e0a8c0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:31.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.562+0000 7f99d060a740 1 -- 172.21.15.77:0/738711281 >> 172.21.15.77:0/738711281 conn(0x5652e9e06a70 msgr2=0x5652e9e06e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:31.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.563+0000 7f99d060a740 1 -- 172.21.15.77:0/738711281 shutdown_connections 2024-09-10T09:54:31.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.563+0000 7f99d060a740 1 -- 172.21.15.77:0/738711281 wait complete. 2024-09-10T09:54:31.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.563+0000 7f99d060a740 1 -- start start 2024-09-10T09:54:31.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.563+0000 7f99d060a740 1 Processor -- start 2024-09-10T09:54:31.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.564+0000 7f99d060a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5652e9e0a4e0 0x5652e9e125e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:31.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.564+0000 7f99d060a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652e9e0ae00 0x5652e9e12b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:31.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.564+0000 7f99d060a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652e9e156c0 0x5652e9e13060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:31.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.564+0000 7f99d060a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5652e9e18330 con 0x5652e9e0ae00 2024-09-10T09:54:31.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.564+0000 7f99d060a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5652e9cf8920 con 0x5652e9e0a4e0 2024-09-10T09:54:31.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.564+0000 7f99d060a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5652e9e184b0 con 0x5652e9e156c0 2024-09-10T09:54:31.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.564+0000 7f99d060a740 10 client.0 _ll_register_callbacks cb 0x5652e9d1c8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:54:31.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.564+0000 7f99be000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652e9e0ae00 0x5652e9e12b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:31.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.564+0000 7f99c7400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5652e9e0a4e0 0x5652e9e125e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:31.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.564+0000 7f99be000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652e9e0ae00 0x5652e9e12b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:48528/0 (socket says 172.21.15.77:48528) 2024-09-10T09:54:31.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.564+0000 7f99be000640 1 -- 172.21.15.77:0/1478892171 learned_addr learned my addr 172.21.15.77:0/1478892171 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:54:31.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.564+0000 7f99c7400640 1 --2- 172.21.15.77:0/1478892171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652e9e156c0 0x5652e9e13060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:31.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.564+0000 7f99be000640 1 -- 172.21.15.77:0/1478892171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652e9e156c0 msgr2=0x5652e9e13060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:31.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.564+0000 7f99be000640 1 --2- 172.21.15.77:0/1478892171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652e9e156c0 0x5652e9e13060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:31.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.564+0000 7f99be000640 1 -- 172.21.15.77:0/1478892171 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5652e9e0a4e0 msgr2=0x5652e9e125e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:31.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.564+0000 7f99be000640 1 --2- 172.21.15.77:0/1478892171 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5652e9e0a4e0 0x5652e9e125e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:31.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.564+0000 7f99be000640 1 -- 172.21.15.77:0/1478892171 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652e9ae12b0 con 0x5652e9e0ae00 2024-09-10T09:54:31.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.564+0000 7f99c7400640 1 --2- 172.21.15.77:0/1478892171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652e9e156c0 0x5652e9e13060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:54:31.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.565+0000 7f99be000640 1 --2- 172.21.15.77:0/1478892171 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652e9e0ae00 0x5652e9e12b20 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f99b002a930 tx=0x7f99b00317e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4663c214e527e639 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:31.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.565+0000 7f99b4c00640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-10T09:54:31.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.565+0000 7f99b4c00640 1 -- 172.21.15.77:0/1478892171 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b0035040 con 0x5652e9e0ae00 2024-09-10T09:54:31.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.565+0000 7f99b4c00640 1 -- 172.21.15.77:0/1478892171 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f99b0004240 con 0x5652e9e0ae00 2024-09-10T09:54:31.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.565+0000 7f99d060a740 1 -- 172.21.15.77:0/1478892171 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99c0004cc0 con 0x5652e9e0ae00 2024-09-10T09:54:31.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.565+0000 7f99b4c00640 1 -- 172.21.15.77:0/1478892171 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f99b0004440 con 0x5652e9e0ae00 2024-09-10T09:54:31.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.565+0000 7f99d060a740 10 client.14949 fetch_fsmap learned FSMap version 28 2024-09-10T09:54:31.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.565+0000 7f99d060a740 1 -- 172.21.15.77:0/1478892171 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5652e9e24010 con 0x5652e9e0ae00 2024-09-10T09:54:31.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.566+0000 7f99b4c00640 1 -- 172.21.15.77:0/1478892171 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99c0004cc0 con 0x5652e9e0ae00 2024-09-10T09:54:31.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.566+0000 7f99b4c00640 1 -- 172.21.15.77:0/1478892171 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f99b003d5c0 con 0x5652e9e0ae00 2024-09-10T09:54:31.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.566+0000 7f99d060a740 10 client.14949 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T09:54:31.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.566+0000 7f99d060a740 10 client.14949 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-10T09:54:31.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.566+0000 7f99d060a740 20 client.14949 populate_metadata read hostname 'smithi077' 2024-09-10T09:54:31.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.566+0000 7f99d060a740 1 --2- 172.21.15.77:0/1478892171 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x5652e9e26670 0x5652e9e27ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:31.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.566+0000 7f99d060a740 4 client.14949 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:54:31.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.566+0000 7f99d060a740 1 -- 172.21.15.77:0/1478892171 --> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5652e9d7d620 con 0x5652e9e26670 2024-09-10T09:54:31.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.567+0000 7f99c7400640 1 --2- 172.21.15.77:0/1478892171 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x5652e9e26670 0x5652e9e27ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:31.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.567+0000 7f99c7400640 1 --2- 172.21.15.77:0/1478892171 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x5652e9e26670 0x5652e9e27ae0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c7da223e64973f0e server_cookie=12fe940f435b101e in_seq=0 out_seq=0 2024-09-10T09:54:31.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.567+0000 7f99b4c00640 10 client.14949 ms_handle_connect on v2:172.21.15.134:6834/3293933699 2024-09-10T09:54:31.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.568+0000 7f99b4c00640 1 -- 172.21.15.77:0/1478892171 <== mds.4 v2:172.21.15.134:6834/3293933699 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5652e9d7d620 con 0x5652e9e26670 2024-09-10T09:54:31.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.568+0000 7f99b4c00640 10 client.14949 handle_command_reply: tid=0 2024-09-10T09:54:31.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.568+0000 7f99d060a740 1 client.14949 shutdown 2024-09-10T09:54:31.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.568+0000 7f99d060a740 1 -- 172.21.15.77:0/1478892171 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652e9e0ae00 msgr2=0x5652e9e12b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:31.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.568+0000 7f99d060a740 1 --2- 172.21.15.77:0/1478892171 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652e9e0ae00 0x5652e9e12b20 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f99b002a930 tx=0x7f99b00317e0 comp rx=0 tx=0).stop 2024-09-10T09:54:31.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.568+0000 7f99d060a740 1 -- 172.21.15.77:0/1478892171 shutdown_connections 2024-09-10T09:54:31.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.568+0000 7f99d060a740 1 --2- 172.21.15.77:0/1478892171 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x5652e9e26670 0x5652e9e27ae0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:31.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.568+0000 7f99d060a740 1 --2- 172.21.15.77:0/1478892171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652e9e156c0 0x5652e9e13060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:31.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.568+0000 7f99d060a740 1 --2- 172.21.15.77:0/1478892171 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652e9e0ae00 0x5652e9e12b20 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:31.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.568+0000 7f99d060a740 1 --2- 172.21.15.77:0/1478892171 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5652e9e0a4e0 0x5652e9e125e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:31.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.568+0000 7f99d060a740 1 -- 172.21.15.77:0/1478892171 >> 172.21.15.77:0/1478892171 conn(0x5652e9e06a70 msgr2=0x5652e9e0c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:31.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.568+0000 7f99b4c00640 0 client.14949 ms_handle_reset on v2:172.21.15.134:6834/3293933699 2024-09-10T09:54:31.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.568+0000 7f99d060a740 1 -- 172.21.15.77:0/1478892171 shutdown_connections 2024-09-10T09:54:31.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.569+0000 7f99d060a740 1 -- 172.21.15.77:0/1478892171 wait complete. 2024-09-10T09:54:31.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.569+0000 7f99d060a740 20 client.14949 trim_cache size 0 max 16384 2024-09-10T09:54:31.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.587+0000 7f99d060a740 1 Processor -- start 2024-09-10T09:54:31.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.587+0000 7f99d060a740 1 -- start start 2024-09-10T09:54:31.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.587+0000 7f99d060a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652e9e0a4e0 0x5652e9e26d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:31.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.587+0000 7f99d060a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5652e9e0ae00 0x5652e9e12eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:31.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.587+0000 7f99d060a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652e9e156c0 0x5652e9e10ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:31.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.587+0000 7f99d060a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5652e9e25e20 con 0x5652e9e0a4e0 2024-09-10T09:54:31.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.587+0000 7f99d060a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5652e9e264f0 con 0x5652e9e0ae00 2024-09-10T09:54:31.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.587+0000 7f99d060a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5652e9e26bc0 con 0x5652e9e156c0 2024-09-10T09:54:31.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.587+0000 7f99c7400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652e9e0a4e0 0x5652e9e26d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:31.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.588+0000 7f99c7400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652e9e0a4e0 0x5652e9e26d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:48540/0 (socket says 172.21.15.77:48540) 2024-09-10T09:54:31.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.588+0000 7f99c7400640 1 -- 172.21.15.77:0/2251655144 learned_addr learned my addr 172.21.15.77:0/2251655144 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:54:31.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.588+0000 7f99be000640 1 --2- 172.21.15.77:0/2251655144 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5652e9e0ae00 0x5652e9e12eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:31.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.588+0000 7f99c7400640 1 --2- 172.21.15.77:0/2251655144 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652e9e156c0 0x5652e9e10ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:31.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.588+0000 7f99c7400640 1 -- 172.21.15.77:0/2251655144 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652e9e156c0 msgr2=0x5652e9e10ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:31.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.588+0000 7f99c7400640 1 --2- 172.21.15.77:0/2251655144 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652e9e156c0 0x5652e9e10ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:31.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.588+0000 7f99c7400640 1 -- 172.21.15.77:0/2251655144 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5652e9e0ae00 msgr2=0x5652e9e12eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:31.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.588+0000 7f99c7400640 1 --2- 172.21.15.77:0/2251655144 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5652e9e0ae00 0x5652e9e12eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:31.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.588+0000 7f99c7400640 1 -- 172.21.15.77:0/2251655144 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652e9e10b70 con 0x5652e9e0a4e0 2024-09-10T09:54:31.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.588+0000 7f99be000640 1 --2- 172.21.15.77:0/2251655144 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5652e9e0ae00 0x5652e9e12eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:54:31.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.588+0000 7f99c7400640 1 --2- 172.21.15.77:0/2251655144 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652e9e0a4e0 0x5652e9e26d40 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f99c0006010 tx=0x7f99c0000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eefab08d5ced60d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:31.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.589+0000 7f99b7400640 1 -- 172.21.15.77:0/2251655144 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c0033070 con 0x5652e9e0a4e0 2024-09-10T09:54:31.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.589+0000 7f99b7400640 1 -- 172.21.15.77:0/2251655144 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99c0004c60 con 0x5652e9e0a4e0 2024-09-10T09:54:31.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.589+0000 7f99b7400640 1 -- 172.21.15.77:0/2251655144 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c00387c0 con 0x5652e9e0a4e0 2024-09-10T09:54:31.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.589+0000 7f99d060a740 1 -- 172.21.15.77:0/2251655144 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652e9e0a4e0 msgr2=0x5652e9e26d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:31.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.589+0000 7f99d060a740 1 --2- 172.21.15.77:0/2251655144 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652e9e0a4e0 0x5652e9e26d40 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f99c0006010 tx=0x7f99c0000f40 comp rx=0 tx=0).stop 2024-09-10T09:54:31.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.589+0000 7f99d060a740 1 -- 172.21.15.77:0/2251655144 shutdown_connections 2024-09-10T09:54:31.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.589+0000 7f99d060a740 1 --2- 172.21.15.77:0/2251655144 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652e9e156c0 0x5652e9e10ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:31.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.589+0000 7f99d060a740 1 --2- 172.21.15.77:0/2251655144 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5652e9e0ae00 0x5652e9e12eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:31.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.589+0000 7f99d060a740 1 --2- 172.21.15.77:0/2251655144 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652e9e0a4e0 0x5652e9e26d40 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:31.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.589+0000 7f99d060a740 1 -- 172.21.15.77:0/2251655144 >> 172.21.15.77:0/2251655144 conn(0x5652e9e06a70 msgr2=0x5652e9e06e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:31.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.589+0000 7f99d060a740 1 -- 172.21.15.77:0/2251655144 shutdown_connections 2024-09-10T09:54:31.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.589+0000 7f99d060a740 1 -- 172.21.15.77:0/2251655144 wait complete. 2024-09-10T09:54:31.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.590+0000 7f99d060a740 1 -- start start 2024-09-10T09:54:31.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.590+0000 7f99d060a740 1 Processor -- start 2024-09-10T09:54:31.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.590+0000 7f99d060a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652e9e0a4e0 0x5652e9e0f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:31.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.590+0000 7f99d060a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5652e9e0ae00 0x5652e9e0f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:31.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.590+0000 7f99d060a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652e9e156c0 0x5652e9e2ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:31.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.590+0000 7f99d060a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5652e9e27960 con 0x5652e9e0a4e0 2024-09-10T09:54:31.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.590+0000 7f99d060a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5652e9e27290 con 0x5652e9e0ae00 2024-09-10T09:54:31.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.590+0000 7f99d060a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5652e9e28030 con 0x5652e9e156c0 2024-09-10T09:54:31.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.590+0000 7f99d060a740 10 client.0 _ll_register_callbacks cb 0x7f99b0005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:54:31.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.591+0000 7f99be000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5652e9e0ae00 0x5652e9e0f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:31.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.591+0000 7f99c7400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652e9e0a4e0 0x5652e9e0f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:31.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.591+0000 7f99be000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5652e9e0ae00 0x5652e9e0f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:59764/0 (socket says 172.21.15.77:59764) 2024-09-10T09:54:31.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.591+0000 7f99be000640 1 -- 172.21.15.77:0/3172212700 learned_addr learned my addr 172.21.15.77:0/3172212700 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:54:31.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.591+0000 7f99be000640 1 -- 172.21.15.77:0/3172212700 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652e9e156c0 msgr2=0x5652e9e2ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:31.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.591+0000 7f99be000640 1 --2- 172.21.15.77:0/3172212700 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652e9e156c0 0x5652e9e2ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:31.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.591+0000 7f99be000640 1 -- 172.21.15.77:0/3172212700 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652e9e0a4e0 msgr2=0x5652e9e0f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:31.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.591+0000 7f99be000640 1 --2- 172.21.15.77:0/3172212700 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652e9e0a4e0 0x5652e9e0f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:31.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.591+0000 7f99be000640 1 -- 172.21.15.77:0/3172212700 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652e9e2f270 con 0x5652e9e0ae00 2024-09-10T09:54:31.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.591+0000 7f99c7400640 1 --2- 172.21.15.77:0/3172212700 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652e9e0a4e0 0x5652e9e0f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:54:31.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.591+0000 7f99be000640 1 --2- 172.21.15.77:0/3172212700 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5652e9e0ae00 0x5652e9e0f550 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f99b0006010 tx=0x7f99b0009680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1de636b7728c17dd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:31.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.591+0000 7f99b4c00640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T09:54:31.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.592+0000 7f99b4c00640 1 -- 172.21.15.77:0/3172212700 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b0035040 con 0x5652e9e0ae00 2024-09-10T09:54:31.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.592+0000 7f99d060a740 1 -- 172.21.15.77:0/3172212700 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99c0004c60 con 0x5652e9e0ae00 2024-09-10T09:54:31.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.592+0000 7f99b4c00640 1 -- 172.21.15.77:0/3172212700 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f99b0005030 con 0x5652e9e0ae00 2024-09-10T09:54:31.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.592+0000 7f99b4c00640 1 -- 172.21.15.77:0/3172212700 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f99b0063050 con 0x5652e9e0ae00 2024-09-10T09:54:31.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.592+0000 7f99d060a740 10 client.25007 fetch_fsmap learned FSMap version 28 2024-09-10T09:54:31.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.592+0000 7f99d060a740 1 -- 172.21.15.77:0/3172212700 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5652e9e30570 con 0x5652e9e0ae00 2024-09-10T09:54:31.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.592+0000 7f99b4c00640 1 -- 172.21.15.77:0/3172212700 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99c0004c60 con 0x5652e9e0ae00 2024-09-10T09:54:31.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.593+0000 7f99b4c00640 1 -- 172.21.15.77:0/3172212700 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f99b0004970 con 0x5652e9e0ae00 2024-09-10T09:54:31.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.593+0000 7f99d060a740 10 client.25007 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T09:54:31.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.593+0000 7f99d060a740 10 client.25007 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-10T09:54:31.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.593+0000 7f99d060a740 20 client.25007 populate_metadata read hostname 'smithi077' 2024-09-10T09:54:31.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.593+0000 7f99d060a740 1 --2- 172.21.15.77:0/3172212700 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x5652e9e95290 0x5652e9e93fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:31.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.593+0000 7f99d060a740 4 client.25007 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T09:54:31.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.593+0000 7f99d060a740 1 -- 172.21.15.77:0/3172212700 --> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] -- command(tid 0: {"prefix": "status"}) -- 0x5652e9d77b30 con 0x5652e9e95290 2024-09-10T09:54:31.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.593+0000 7f99c7400640 1 --2- 172.21.15.77:0/3172212700 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x5652e9e95290 0x5652e9e93fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:31.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.594+0000 7f99c7400640 1 --2- 172.21.15.77:0/3172212700 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x5652e9e95290 0x5652e9e93fa0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=55361d15d288085d server_cookie=4b1072379d34be23 in_seq=0 out_seq=0 2024-09-10T09:54:31.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.594+0000 7f99b4c00640 10 client.25007 ms_handle_connect on v2:172.21.15.134:6834/3293933699 2024-09-10T09:54:31.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.594+0000 7f99b4c00640 1 -- 172.21.15.77:0/3172212700 <== mds.4 v2:172.21.15.134:6834/3293933699 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5652e9d77b30 con 0x5652e9e95290 2024-09-10T09:54:31.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.594+0000 7f99b4c00640 10 client.25007 handle_command_reply: tid=0 2024-09-10T09:54:31.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.594+0000 7f99d060a740 1 client.25007 shutdown 2024-09-10T09:54:31.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.594+0000 7f99d060a740 1 -- 172.21.15.77:0/3172212700 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5652e9e0ae00 msgr2=0x5652e9e0f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:31.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.594+0000 7f99d060a740 1 --2- 172.21.15.77:0/3172212700 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5652e9e0ae00 0x5652e9e0f550 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f99b0006010 tx=0x7f99b0009680 comp rx=0 tx=0).stop 2024-09-10T09:54:31.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.594+0000 7f99d060a740 1 -- 172.21.15.77:0/3172212700 shutdown_connections 2024-09-10T09:54:31.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.594+0000 7f99d060a740 1 --2- 172.21.15.77:0/3172212700 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x5652e9e95290 0x5652e9e93fa0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:31.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.594+0000 7f99d060a740 1 --2- 172.21.15.77:0/3172212700 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652e9e156c0 0x5652e9e2ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:31.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.594+0000 7f99d060a740 1 --2- 172.21.15.77:0/3172212700 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5652e9e0ae00 0x5652e9e0f550 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:31.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.594+0000 7f99b4c00640 0 client.25007 ms_handle_reset on v2:172.21.15.134:6834/3293933699 2024-09-10T09:54:31.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.594+0000 7f99d060a740 1 --2- 172.21.15.77:0/3172212700 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652e9e0a4e0 0x5652e9e0f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:31.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.594+0000 7f99d060a740 1 -- 172.21.15.77:0/3172212700 >> 172.21.15.77:0/3172212700 conn(0x5652e9e06a70 msgr2=0x5652e9e10d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:31.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.595+0000 7f99d060a740 1 -- 172.21.15.77:0/3172212700 shutdown_connections 2024-09-10T09:54:31.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.595+0000 7f99d060a740 1 -- 172.21.15.77:0/3172212700 wait complete. 2024-09-10T09:54:31.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.595+0000 7f99d060a740 20 client.25007 trim_cache size 0 max 16384 2024-09-10T09:54:31.599 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-10T09:54:31.599 INFO:teuthology.orchestra.run.smithi077.stdout: "cluster_fsid": "14a55226-6f59-11ef-bcea-c7b262605968", 2024-09-10T09:54:31.600 INFO:teuthology.orchestra.run.smithi077.stdout: "whoami": 4, 2024-09-10T09:54:31.600 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 24496, 2024-09-10T09:54:31.600 INFO:teuthology.orchestra.run.smithi077.stdout: "want_state": "up:active", 2024-09-10T09:54:31.600 INFO:teuthology.orchestra.run.smithi077.stdout: "state": "up:active", 2024-09-10T09:54:31.600 INFO:teuthology.orchestra.run.smithi077.stdout: "fs_name": "cephfs", 2024-09-10T09:54:31.600 INFO:teuthology.orchestra.run.smithi077.stdout: "rank_uptime": 75.728685628999997, 2024-09-10T09:54:31.600 INFO:teuthology.orchestra.run.smithi077.stdout: "mdsmap_epoch": 28, 2024-09-10T09:54:31.600 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch": 96, 2024-09-10T09:54:31.600 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch_barrier": 94, 2024-09-10T09:54:31.600 INFO:teuthology.orchestra.run.smithi077.stdout: "uptime": 159.203009342 2024-09-10T09:54:31.600 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-10T09:54:31.601 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.595+0000 7f99c7e00640 1 -- 172.21.15.77:0/3514279949 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f99ac0a2b80 msgr2=0x7f99ac0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:31.601 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.595+0000 7f99c7e00640 1 --2- 172.21.15.77:0/3514279949 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f99ac0a2b80 0x7f99ac0a5040 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f99b8098650 tx=0x7f99b80d5530 comp rx=0 tx=0).stop 2024-09-10T09:54:31.601 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.595+0000 7f99c7e00640 1 -- 172.21.15.77:0/3514279949 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f99c8103e30 msgr2=0x7f99c819a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:31.601 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.595+0000 7f99c7e00640 1 --2- 172.21.15.77:0/3514279949 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f99c8103e30 0x7f99c819a4c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f99b800e9f0 tx=0x7f99b800edf0 comp rx=0 tx=0).stop 2024-09-10T09:54:31.601 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.595+0000 7f99c7e00640 1 -- 172.21.15.77:0/3514279949 shutdown_connections 2024-09-10T09:54:31.601 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.595+0000 7f99c7e00640 1 --2- 172.21.15.77:0/3514279949 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f99ac0a2b80 0x7f99ac0a5040 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:31.601 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.595+0000 7f99c7e00640 1 --2- 172.21.15.77:0/3514279949 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f99c8108f60 0x7f99c819ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:31.602 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.595+0000 7f99c7e00640 1 --2- 172.21.15.77:0/3514279949 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99c81047e0 0x7f99c819aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:31.602 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.595+0000 7f99c7e00640 1 --2- 172.21.15.77:0/3514279949 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f99c8103e30 0x7f99c819a4c0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:31.602 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.595+0000 7f99c7e00640 1 -- 172.21.15.77:0/3514279949 >> 172.21.15.77:0/3514279949 conn(0x7f99c80ff8d0 msgr2=0x7f99c8100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:31.602 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.596+0000 7f99c7e00640 1 -- 172.21.15.77:0/3514279949 shutdown_connections 2024-09-10T09:54:31.602 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:31.596+0000 7f99c7e00640 1 -- 172.21.15.77:0/3514279949 wait complete. 2024-09-10T09:54:32.104 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:54:31 smithi077 ceph-mon[30319]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-10T09:54:32.130 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:54:31 smithi134 ceph-mon[35674]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-10T09:54:32.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:54:31 smithi120 ceph-mon[34770]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-10T09:54:32.445 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.c status 2024-09-10T09:54:32.811 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:54:33.085 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:54:32 smithi077 ceph-mon[30319]: pgmap v372: 129 pgs: 129 active+clean; 645 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-10T09:54:33.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:54:32 smithi134 ceph-mon[35674]: pgmap v372: 129 pgs: 129 active+clean; 645 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-10T09:54:33.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:54:32 smithi120 ceph-mon[34770]: pgmap v372: 129 pgs: 129 active+clean; 645 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-10T09:54:33.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.328+0000 7ffa15800640 1 -- 172.21.15.77:0/2793070720 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa10074590 msgr2=0x7ffa10074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:33.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.328+0000 7ffa15800640 1 --2- 172.21.15.77:0/2793070720 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa10074590 0x7ffa10074970 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ffa00009900 tx=0x7ffa0002a480 comp rx=0 tx=0).stop 2024-09-10T09:54:33.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.329+0000 7ffa15800640 1 -- 172.21.15.77:0/2793070720 shutdown_connections 2024-09-10T09:54:33.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.329+0000 7ffa15800640 1 --2- 172.21.15.77:0/2793070720 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa10112b90 0x7ffa10114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:33.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.329+0000 7ffa15800640 1 --2- 172.21.15.77:0/2793070720 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa10074eb0 0x7ffa10112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:33.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.329+0000 7ffa15800640 1 --2- 172.21.15.77:0/2793070720 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa10074590 0x7ffa10074970 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:33.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.329+0000 7ffa15800640 1 -- 172.21.15.77:0/2793070720 >> 172.21.15.77:0/2793070720 conn(0x7ffa100fd8c0 msgr2=0x7ffa100ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:33.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.329+0000 7ffa15800640 1 -- 172.21.15.77:0/2793070720 shutdown_connections 2024-09-10T09:54:33.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.329+0000 7ffa15800640 1 -- 172.21.15.77:0/2793070720 wait complete. 2024-09-10T09:54:33.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.330+0000 7ffa15800640 1 Processor -- start 2024-09-10T09:54:33.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.330+0000 7ffa15800640 1 -- start start 2024-09-10T09:54:33.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.330+0000 7ffa15800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa10074590 0x7ffa101a4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:33.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.330+0000 7ffa15800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa10074eb0 0x7ffa101a4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:33.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.330+0000 7ffa15800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa10112b90 0x7ffa1019e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:33.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.330+0000 7ffa15800640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ffa101185c0 con 0x7ffa10074eb0 2024-09-10T09:54:33.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.330+0000 7ffa15800640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ffa10118440 con 0x7ffa10112b90 2024-09-10T09:54:33.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.330+0000 7ffa15800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ffa10118740 con 0x7ffa10074590 2024-09-10T09:54:33.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.330+0000 7ffa0e000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa10074eb0 0x7ffa101a4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:33.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.330+0000 7ffa0ea00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa10074590 0x7ffa101a4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:33.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.330+0000 7ffa0e000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa10074eb0 0x7ffa101a4c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34170/0 (socket says 172.21.15.77:34170) 2024-09-10T09:54:33.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.330+0000 7ffa0ea00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa10074590 0x7ffa101a4730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:59760/0 (socket says 172.21.15.77:59760) 2024-09-10T09:54:33.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.330+0000 7ffa0e000640 1 -- 172.21.15.77:0/4219877544 learned_addr learned my addr 172.21.15.77:0/4219877544 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:54:33.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.331+0000 7ffa0f400640 1 --2- 172.21.15.77:0/4219877544 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa10112b90 0x7ffa1019e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:33.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.331+0000 7ffa0ea00640 1 -- 172.21.15.77:0/4219877544 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa10112b90 msgr2=0x7ffa1019e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:33.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.331+0000 7ffa0ea00640 1 --2- 172.21.15.77:0/4219877544 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa10112b90 0x7ffa1019e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:33.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.331+0000 7ffa0ea00640 1 -- 172.21.15.77:0/4219877544 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa10074eb0 msgr2=0x7ffa101a4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:33.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.331+0000 7ffa0ea00640 1 --2- 172.21.15.77:0/4219877544 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa10074eb0 0x7ffa101a4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:33.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.331+0000 7ffa0ea00640 1 -- 172.21.15.77:0/4219877544 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa1019f020 con 0x7ffa10074590 2024-09-10T09:54:33.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.331+0000 7ffa0e000640 1 --2- 172.21.15.77:0/4219877544 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa10074eb0 0x7ffa101a4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:54:33.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.331+0000 7ffa0ea00640 1 --2- 172.21.15.77:0/4219877544 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa10074590 0x7ffa101a4730 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ffa0002a9a0 tx=0x7ffa00003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:33.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.331+0000 7ffa07400640 1 -- 172.21.15.77:0/4219877544 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa0003f070 con 0x7ffa10074590 2024-09-10T09:54:33.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.331+0000 7ffa07400640 1 -- 172.21.15.77:0/4219877544 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa00031ca0 con 0x7ffa10074590 2024-09-10T09:54:33.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.331+0000 7ffa15800640 1 -- 172.21.15.77:0/4219877544 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa1019f2b0 con 0x7ffa10074590 2024-09-10T09:54:33.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.331+0000 7ffa07400640 1 -- 172.21.15.77:0/4219877544 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa00035030 con 0x7ffa10074590 2024-09-10T09:54:33.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.332+0000 7ffa15800640 1 -- 172.21.15.77:0/4219877544 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa101ab600 con 0x7ffa10074590 2024-09-10T09:54:33.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.335+0000 7ffa07400640 1 -- 172.21.15.77:0/4219877544 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ffa00042070 con 0x7ffa10074590 2024-09-10T09:54:33.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.335+0000 7ffa183da740 1 Processor -- start 2024-09-10T09:54:33.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.335+0000 7ffa183da740 1 -- start start 2024-09-10T09:54:33.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.335+0000 7ffa183da740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558c14851330 0x558c14851710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:33.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.335+0000 7ffa183da740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558c14851c50 0x558c1485bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:33.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.335+0000 7ffa183da740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c1485c510 0x558c1485e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:33.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.335+0000 7ffa183da740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558c147b70b0 con 0x558c1485c510 2024-09-10T09:54:33.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.335+0000 7ffa183da740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558c1475eab0 con 0x558c14851330 2024-09-10T09:54:33.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.335+0000 7ffa183da740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558c147b3070 con 0x558c14851c50 2024-09-10T09:54:33.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.336+0000 7ffa0ea00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558c14851c50 0x558c1485bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:33.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.336+0000 7ffa0ea00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558c14851c50 0x558c1485bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:59762/0 (socket says 172.21.15.77:59762) 2024-09-10T09:54:33.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.336+0000 7ffa0ea00640 1 -- 172.21.15.77:0/3507156005 learned_addr learned my addr 172.21.15.77:0/3507156005 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:54:33.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.336+0000 7ffa0f400640 1 --2- 172.21.15.77:0/3507156005 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558c14851330 0x558c14851710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:33.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.336+0000 7ffa0e000640 1 --2- 172.21.15.77:0/3507156005 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c1485c510 0x558c1485e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:33.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.336+0000 7ffa0ea00640 1 -- 172.21.15.77:0/3507156005 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558c14851330 msgr2=0x558c14851710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:33.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.336+0000 7ffa0ea00640 1 --2- 172.21.15.77:0/3507156005 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558c14851330 0x558c14851710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:33.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.336+0000 7ffa0ea00640 1 -- 172.21.15.77:0/3507156005 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c1485c510 msgr2=0x558c1485e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:33.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.336+0000 7ffa0ea00640 1 --2- 172.21.15.77:0/3507156005 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c1485c510 0x558c1485e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:33.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.336+0000 7ffa0ea00640 1 -- 172.21.15.77:0/3507156005 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c1476b4a0 con 0x558c14851c50 2024-09-10T09:54:33.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.336+0000 7ffa0f400640 1 --2- 172.21.15.77:0/3507156005 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558c14851330 0x558c14851710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:54:33.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.336+0000 7ffa07400640 1 --2- 172.21.15.77:0/4219877544 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ff9f40a2c00 0x7ff9f40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:33.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.337+0000 7ffa0ea00640 1 --2- 172.21.15.77:0/3507156005 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558c14851c50 0x558c1485bfd0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ffa000f5c20 tx=0x7ffa000f5d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f699011f93d140b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:33.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.337+0000 7ffa07400640 1 -- 172.21.15.77:0/4219877544 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7ffa000ef9f0 con 0x7ffa10074590 2024-09-10T09:54:33.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.337+0000 7ff9fbe00640 1 -- 172.21.15.77:0/3507156005 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa000b8150 con 0x558c14851c50 2024-09-10T09:54:33.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.337+0000 7ffa0f400640 1 --2- 172.21.15.77:0/4219877544 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ff9f40a2c00 0x7ff9f40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:33.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.337+0000 7ff9fbe00640 1 -- 172.21.15.77:0/3507156005 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa00031e50 con 0x558c14851c50 2024-09-10T09:54:33.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.337+0000 7ff9fbe00640 1 -- 172.21.15.77:0/3507156005 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa000bb810 con 0x558c14851c50 2024-09-10T09:54:33.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.338+0000 7ffa183da740 1 -- 172.21.15.77:0/3507156005 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558c14851c50 msgr2=0x558c1485bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:33.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.338+0000 7ffa0f400640 1 --2- 172.21.15.77:0/4219877544 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ff9f40a2c00 0x7ff9f40a50c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ffa10076600 tx=0x7ffa080096c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:33.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.338+0000 7ffa183da740 1 --2- 172.21.15.77:0/3507156005 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558c14851c50 0x558c1485bfd0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ffa000f5c20 tx=0x7ffa000f5d10 comp rx=0 tx=0).stop 2024-09-10T09:54:33.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.338+0000 7ffa183da740 1 -- 172.21.15.77:0/3507156005 shutdown_connections 2024-09-10T09:54:33.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.338+0000 7ffa183da740 1 --2- 172.21.15.77:0/3507156005 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c1485c510 0x558c1485e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:33.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.338+0000 7ffa183da740 1 --2- 172.21.15.77:0/3507156005 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558c14851c50 0x558c1485bfd0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:33.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.338+0000 7ffa183da740 1 --2- 172.21.15.77:0/3507156005 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558c14851330 0x558c14851710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:33.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.338+0000 7ffa183da740 1 -- 172.21.15.77:0/3507156005 >> 172.21.15.77:0/3507156005 conn(0x558c1484d8c0 msgr2=0x558c1484dca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:33.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.338+0000 7ffa183da740 1 -- 172.21.15.77:0/3507156005 shutdown_connections 2024-09-10T09:54:33.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.338+0000 7ffa183da740 1 -- 172.21.15.77:0/3507156005 wait complete. 2024-09-10T09:54:33.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.339+0000 7ffa183da740 1 -- start start 2024-09-10T09:54:33.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.339+0000 7ffa183da740 1 Processor -- start 2024-09-10T09:54:33.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.339+0000 7ffa183da740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558c14851330 0x558c14859470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:33.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.340+0000 7ffa183da740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558c14851c50 0x558c148599b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:33.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.340+0000 7ffa183da740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c1485c510 0x558c14859ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:33.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.340+0000 7ffa183da740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558c1485eea0 con 0x558c1485c510 2024-09-10T09:54:33.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.340+0000 7ffa183da740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558c1473f920 con 0x558c14851330 2024-09-10T09:54:33.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.340+0000 7ffa183da740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558c1485f020 con 0x558c14851c50 2024-09-10T09:54:33.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.340+0000 7ffa183da740 10 client.0 _ll_register_callbacks cb 0x558c147638a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:54:33.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.340+0000 7ffa0f400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c1485c510 0x558c14859ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:33.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.340+0000 7ffa0ea00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558c14851330 0x558c14859470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:33.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.340+0000 7ffa0f400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c1485c510 0x558c14859ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34192/0 (socket says 172.21.15.77:34192) 2024-09-10T09:54:33.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.340+0000 7ffa0f400640 1 -- 172.21.15.77:0/2401791970 learned_addr learned my addr 172.21.15.77:0/2401791970 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:54:33.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.340+0000 7ffa0e000640 1 --2- 172.21.15.77:0/2401791970 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558c14851c50 0x558c148599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:33.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.340+0000 7ffa0ea00640 1 -- 172.21.15.77:0/2401791970 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558c14851c50 msgr2=0x558c148599b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:33.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.340+0000 7ffa0ea00640 1 --2- 172.21.15.77:0/2401791970 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558c14851c50 0x558c148599b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:33.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.340+0000 7ffa0ea00640 1 -- 172.21.15.77:0/2401791970 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c1485c510 msgr2=0x558c14859ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:33.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.340+0000 7ffa0ea00640 1 --2- 172.21.15.77:0/2401791970 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c1485c510 0x558c14859ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:33.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.340+0000 7ffa0ea00640 1 -- 172.21.15.77:0/2401791970 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c145282b0 con 0x558c14851330 2024-09-10T09:54:33.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.341+0000 7ffa0ea00640 1 --2- 172.21.15.77:0/2401791970 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558c14851330 0x558c14859470 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ffa00005d00 tx=0x7ffa00000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a85a7d72f85cd61f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:33.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.341+0000 7ff9f8c00640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T09:54:33.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.341+0000 7ff9f8c00640 1 -- 172.21.15.77:0/2401791970 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa00049070 con 0x558c14851330 2024-09-10T09:54:33.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.341+0000 7ffa183da740 1 -- 172.21.15.77:0/2401791970 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa00031e50 con 0x558c14851330 2024-09-10T09:54:33.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.341+0000 7ff9f8c00640 1 -- 172.21.15.77:0/2401791970 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7ffa0003b460 con 0x558c14851330 2024-09-10T09:54:33.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.341+0000 7ff9f8c00640 1 -- 172.21.15.77:0/2401791970 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ffa000bb9f0 con 0x558c14851330 2024-09-10T09:54:33.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.341+0000 7ffa183da740 10 client.25016 fetch_fsmap learned FSMap version 28 2024-09-10T09:54:33.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.341+0000 7ffa183da740 1 -- 172.21.15.77:0/2401791970 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558c1485eea0 con 0x558c14851330 2024-09-10T09:54:33.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.342+0000 7ff9f8c00640 1 -- 172.21.15.77:0/2401791970 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa00031e50 con 0x558c14851330 2024-09-10T09:54:33.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.342+0000 7ff9f8c00640 1 -- 172.21.15.77:0/2401791970 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7ffa000bc110 con 0x558c14851330 2024-09-10T09:54:33.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.342+0000 7ffa183da740 10 client.25016 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T09:54:33.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.342+0000 7ffa183da740 10 client.25016 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T09:54:33.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.342+0000 7ffa183da740 20 client.25016 populate_metadata read hostname 'smithi077' 2024-09-10T09:54:33.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.342+0000 7ffa183da740 1 --2- 172.21.15.77:0/2401791970 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x558c1486cde0 0x558c1486e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:33.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.342+0000 7ffa183da740 4 client.25016 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:54:33.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.342+0000 7ffa183da740 1 -- 172.21.15.77:0/2401791970 --> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558c147c4620 con 0x558c1486cde0 2024-09-10T09:54:33.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.342+0000 7ffa0e000640 1 --2- 172.21.15.77:0/2401791970 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x558c1486cde0 0x558c1486e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:33.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.343+0000 7ffa0e000640 1 --2- 172.21.15.77:0/2401791970 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x558c1486cde0 0x558c1486e250 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4c6739b0685d537d server_cookie=d5cb2fa846226bff in_seq=0 out_seq=0 2024-09-10T09:54:33.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.343+0000 7ff9f8c00640 10 client.25016 ms_handle_connect on v2:172.21.15.134:6836/2461497091 2024-09-10T09:54:33.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.344+0000 7ff9f8c00640 1 -- 172.21.15.77:0/2401791970 <== mds.3 v2:172.21.15.134:6836/2461497091 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558c147c4620 con 0x558c1486cde0 2024-09-10T09:54:33.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.344+0000 7ff9f8c00640 10 client.25016 handle_command_reply: tid=0 2024-09-10T09:54:33.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.344+0000 7ffa183da740 1 client.25016 shutdown 2024-09-10T09:54:33.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.344+0000 7ffa183da740 1 -- 172.21.15.77:0/2401791970 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558c14851330 msgr2=0x558c14859470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:33.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.344+0000 7ffa183da740 1 --2- 172.21.15.77:0/2401791970 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558c14851330 0x558c14859470 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ffa00005d00 tx=0x7ffa00000bc0 comp rx=0 tx=0).stop 2024-09-10T09:54:33.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.344+0000 7ffa183da740 1 -- 172.21.15.77:0/2401791970 shutdown_connections 2024-09-10T09:54:33.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.344+0000 7ffa183da740 1 --2- 172.21.15.77:0/2401791970 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x558c1486cde0 0x558c1486e250 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:33.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.344+0000 7ffa183da740 1 --2- 172.21.15.77:0/2401791970 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c1485c510 0x558c14859ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:33.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.344+0000 7ffa183da740 1 --2- 172.21.15.77:0/2401791970 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558c14851c50 0x558c148599b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:33.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.344+0000 7ff9f8c00640 0 client.25016 ms_handle_reset on v2:172.21.15.134:6836/2461497091 2024-09-10T09:54:33.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.344+0000 7ffa183da740 1 --2- 172.21.15.77:0/2401791970 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558c14851330 0x558c14859470 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:33.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.344+0000 7ffa183da740 1 -- 172.21.15.77:0/2401791970 >> 172.21.15.77:0/2401791970 conn(0x558c1484d8c0 msgr2=0x558c148535a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:33.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.344+0000 7ffa183da740 1 -- 172.21.15.77:0/2401791970 shutdown_connections 2024-09-10T09:54:33.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.345+0000 7ffa183da740 1 -- 172.21.15.77:0/2401791970 wait complete. 2024-09-10T09:54:33.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.345+0000 7ffa183da740 20 client.25016 trim_cache size 0 max 16384 2024-09-10T09:54:33.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.363+0000 7ffa183da740 1 Processor -- start 2024-09-10T09:54:33.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.363+0000 7ffa183da740 1 -- start start 2024-09-10T09:54:33.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.363+0000 7ffa183da740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558c14851330 0x558c14858450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:33.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.363+0000 7ffa183da740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558c14851c50 0x558c14859d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:33.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.363+0000 7ffa183da740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c1485c510 0x558c148566d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:33.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.363+0000 7ffa183da740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558c1486c590 con 0x558c1485c510 2024-09-10T09:54:33.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.363+0000 7ffa183da740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558c1486cc60 con 0x558c14851c50 2024-09-10T09:54:33.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.363+0000 7ffa183da740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558c1486d330 con 0x558c14851330 2024-09-10T09:54:33.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.364+0000 7ffa0f400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c1485c510 0x558c148566d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:33.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.364+0000 7ffa0f400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c1485c510 0x558c148566d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34194/0 (socket says 172.21.15.77:34194) 2024-09-10T09:54:33.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.364+0000 7ffa0f400640 1 -- 172.21.15.77:0/55050819 learned_addr learned my addr 172.21.15.77:0/55050819 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:54:33.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.364+0000 7ffa0e000640 1 --2- 172.21.15.77:0/55050819 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558c14851c50 0x558c14859d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:33.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.364+0000 7ffa0ea00640 1 --2- 172.21.15.77:0/55050819 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558c14851330 0x558c14858450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:33.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.364+0000 7ffa0e000640 1 -- 172.21.15.77:0/55050819 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558c14851330 msgr2=0x558c14858450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:33.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.364+0000 7ffa0e000640 1 --2- 172.21.15.77:0/55050819 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558c14851330 0x558c14858450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:33.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.364+0000 7ffa0e000640 1 -- 172.21.15.77:0/55050819 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c1485c510 msgr2=0x558c148566d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:33.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.364+0000 7ffa0e000640 1 --2- 172.21.15.77:0/55050819 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c1485c510 0x558c148566d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:33.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.364+0000 7ffa0e000640 1 -- 172.21.15.77:0/55050819 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c148579c0 con 0x558c14851c50 2024-09-10T09:54:33.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.365+0000 7ffa0e000640 1 --2- 172.21.15.77:0/55050819 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558c14851c50 0x558c14859d40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x558c1485aab0 tx=0x7ff9fc005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f3703238c847135 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:33.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.365+0000 7ff9fb400640 1 -- 172.21.15.77:0/55050819 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc010ae0 con 0x558c14851c50 2024-09-10T09:54:33.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.365+0000 7ff9fb400640 1 -- 172.21.15.77:0/55050819 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9fc010c80 con 0x558c14851c50 2024-09-10T09:54:33.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.365+0000 7ff9fb400640 1 -- 172.21.15.77:0/55050819 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc026630 con 0x558c14851c50 2024-09-10T09:54:33.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.365+0000 7ffa183da740 1 -- 172.21.15.77:0/55050819 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558c14851c50 msgr2=0x558c14859d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:33.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.365+0000 7ffa183da740 1 --2- 172.21.15.77:0/55050819 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558c14851c50 0x558c14859d40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x558c1485aab0 tx=0x7ff9fc005f90 comp rx=0 tx=0).stop 2024-09-10T09:54:33.370 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.365+0000 7ffa183da740 1 -- 172.21.15.77:0/55050819 shutdown_connections 2024-09-10T09:54:33.370 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.365+0000 7ffa183da740 1 --2- 172.21.15.77:0/55050819 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c1485c510 0x558c148566d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:33.370 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.365+0000 7ffa183da740 1 --2- 172.21.15.77:0/55050819 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558c14851c50 0x558c14859d40 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:33.370 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.365+0000 7ffa183da740 1 --2- 172.21.15.77:0/55050819 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558c14851330 0x558c14858450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:33.370 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.365+0000 7ffa183da740 1 -- 172.21.15.77:0/55050819 >> 172.21.15.77:0/55050819 conn(0x558c1484d8c0 msgr2=0x558c14857d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:33.370 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.366+0000 7ffa183da740 1 -- 172.21.15.77:0/55050819 shutdown_connections 2024-09-10T09:54:33.370 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.366+0000 7ffa183da740 1 -- 172.21.15.77:0/55050819 wait complete. 2024-09-10T09:54:33.370 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.366+0000 7ffa183da740 1 -- start start 2024-09-10T09:54:33.370 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.366+0000 7ffa183da740 1 Processor -- start 2024-09-10T09:54:33.370 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.366+0000 7ffa183da740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c14851330 0x558c148703c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:33.370 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.366+0000 7ffa183da740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558c14851c50 0x558c14870900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:33.370 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.367+0000 7ffa183da740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558c1485c510 0x558c14870e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:33.370 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.367+0000 7ffa183da740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558c1486e0d0 con 0x558c14851330 2024-09-10T09:54:33.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.367+0000 7ffa183da740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558c1486da00 con 0x558c14851c50 2024-09-10T09:54:33.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.367+0000 7ffa183da740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558c1486e7a0 con 0x558c1485c510 2024-09-10T09:54:33.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.367+0000 7ffa183da740 10 client.0 _ll_register_callbacks cb 0x7ffa000bbfb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:54:33.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.367+0000 7ffa0ea00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c14851330 0x558c148703c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:33.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.367+0000 7ffa0e000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558c14851c50 0x558c14870900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:33.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.367+0000 7ffa0ea00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c14851330 0x558c148703c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34202/0 (socket says 172.21.15.77:34202) 2024-09-10T09:54:33.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.367+0000 7ffa0ea00640 1 -- 172.21.15.77:0/2626673410 learned_addr learned my addr 172.21.15.77:0/2626673410 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:54:33.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.367+0000 7ffa0e000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558c14851c50 0x558c14870900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:34042/0 (socket says 172.21.15.77:34042) 2024-09-10T09:54:33.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.367+0000 7ffa0f400640 1 --2- 172.21.15.77:0/2626673410 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558c1485c510 0x558c14870e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:33.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.367+0000 7ffa0e000640 1 -- 172.21.15.77:0/2626673410 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558c1485c510 msgr2=0x558c14870e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:33.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.367+0000 7ffa0e000640 1 --2- 172.21.15.77:0/2626673410 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558c1485c510 0x558c14870e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:33.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.367+0000 7ffa0e000640 1 -- 172.21.15.77:0/2626673410 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c14851330 msgr2=0x558c148703c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:33.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.367+0000 7ffa0e000640 1 --2- 172.21.15.77:0/2626673410 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c14851330 0x558c148703c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:33.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.367+0000 7ffa0e000640 1 -- 172.21.15.77:0/2626673410 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c14874cc0 con 0x558c14851c50 2024-09-10T09:54:33.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.367+0000 7ffa0ea00640 1 --2- 172.21.15.77:0/2626673410 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c14851330 0x558c148703c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:54:33.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.368+0000 7ffa0e000640 1 --2- 172.21.15.77:0/2626673410 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558c14851c50 0x558c14870900 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x558c148690c0 tx=0x7ff9fc012070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71ef6082ed96abfb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:33.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.368+0000 7ff9f8c00640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T09:54:33.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.368+0000 7ff9f8c00640 1 -- 172.21.15.77:0/2626673410 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc0108e0 con 0x558c14851c50 2024-09-10T09:54:33.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.368+0000 7ff9f8c00640 1 -- 172.21.15.77:0/2626673410 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7ff9fc010a80 con 0x558c14851c50 2024-09-10T09:54:33.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.368+0000 7ffa183da740 1 -- 172.21.15.77:0/2626673410 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9fc010c80 con 0x558c14851c50 2024-09-10T09:54:33.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.368+0000 7ff9f8c00640 1 -- 172.21.15.77:0/2626673410 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff9fc005040 con 0x558c14851c50 2024-09-10T09:54:33.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.368+0000 7ffa183da740 10 client.25022 fetch_fsmap learned FSMap version 28 2024-09-10T09:54:33.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.368+0000 7ffa183da740 1 -- 172.21.15.77:0/2626673410 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558c148d7950 con 0x558c14851c50 2024-09-10T09:54:33.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.369+0000 7ff9f8c00640 1 -- 172.21.15.77:0/2626673410 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9fc010c80 con 0x558c14851c50 2024-09-10T09:54:33.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.369+0000 7ff9f8c00640 1 -- 172.21.15.77:0/2626673410 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7ff9fc01e3f0 con 0x558c14851c50 2024-09-10T09:54:33.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.369+0000 7ffa183da740 10 client.25022 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T09:54:33.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.369+0000 7ffa183da740 10 client.25022 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T09:54:33.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.369+0000 7ffa183da740 20 client.25022 populate_metadata read hostname 'smithi077' 2024-09-10T09:54:33.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.369+0000 7ffa183da740 1 --2- 172.21.15.77:0/2626673410 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x558c148dc340 0x558c148db050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:33.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.369+0000 7ffa183da740 4 client.25022 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T09:54:33.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.369+0000 7ffa183da740 1 -- 172.21.15.77:0/2626673410 --> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] -- command(tid 0: {"prefix": "status"}) -- 0x558c147beb30 con 0x558c148dc340 2024-09-10T09:54:33.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.369+0000 7ffa0ea00640 1 --2- 172.21.15.77:0/2626673410 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x558c148dc340 0x558c148db050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:33.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.370+0000 7ffa0ea00640 1 --2- 172.21.15.77:0/2626673410 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x558c148dc340 0x558c148db050 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ef302a61c9a02dd0 server_cookie=83d5a901f141335a in_seq=0 out_seq=0 2024-09-10T09:54:33.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.370+0000 7ff9f8c00640 10 client.25022 ms_handle_connect on v2:172.21.15.134:6836/2461497091 2024-09-10T09:54:33.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.370+0000 7ff9f8c00640 1 -- 172.21.15.77:0/2626673410 <== mds.3 v2:172.21.15.134:6836/2461497091 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x558c147beb30 con 0x558c148dc340 2024-09-10T09:54:33.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.370+0000 7ff9f8c00640 10 client.25022 handle_command_reply: tid=0 2024-09-10T09:54:33.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.370+0000 7ffa183da740 1 client.25022 shutdown 2024-09-10T09:54:33.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.370+0000 7ffa183da740 1 -- 172.21.15.77:0/2626673410 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558c14851c50 msgr2=0x558c14870900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:33.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.370+0000 7ffa183da740 1 --2- 172.21.15.77:0/2626673410 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558c14851c50 0x558c14870900 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x558c148690c0 tx=0x7ff9fc012070 comp rx=0 tx=0).stop 2024-09-10T09:54:33.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.370+0000 7ffa183da740 1 -- 172.21.15.77:0/2626673410 shutdown_connections 2024-09-10T09:54:33.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.370+0000 7ffa183da740 1 --2- 172.21.15.77:0/2626673410 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558c1485c510 0x558c14870e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:33.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.370+0000 7ffa183da740 1 --2- 172.21.15.77:0/2626673410 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558c14851c50 0x558c14870900 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:33.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.370+0000 7ffa183da740 1 --2- 172.21.15.77:0/2626673410 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x558c148dc340 0x558c148db050 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:33.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.370+0000 7ffa183da740 1 --2- 172.21.15.77:0/2626673410 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c14851330 0x558c148703c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:33.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.370+0000 7ff9f8c00640 0 client.25022 ms_handle_reset on v2:172.21.15.134:6836/2461497091 2024-09-10T09:54:33.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.370+0000 7ffa183da740 1 -- 172.21.15.77:0/2626673410 >> 172.21.15.77:0/2626673410 conn(0x558c1484d8c0 msgr2=0x558c14867990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:33.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.370+0000 7ffa183da740 1 -- 172.21.15.77:0/2626673410 shutdown_connections 2024-09-10T09:54:33.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.371+0000 7ffa183da740 1 -- 172.21.15.77:0/2626673410 wait complete. 2024-09-10T09:54:33.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.371+0000 7ffa183da740 20 client.25022 trim_cache size 0 max 16384 2024-09-10T09:54:33.375 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-10T09:54:33.376 INFO:teuthology.orchestra.run.smithi077.stdout: "cluster_fsid": "14a55226-6f59-11ef-bcea-c7b262605968", 2024-09-10T09:54:33.376 INFO:teuthology.orchestra.run.smithi077.stdout: "whoami": 3, 2024-09-10T09:54:33.376 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 24502, 2024-09-10T09:54:33.376 INFO:teuthology.orchestra.run.smithi077.stdout: "want_state": "up:active", 2024-09-10T09:54:33.376 INFO:teuthology.orchestra.run.smithi077.stdout: "state": "up:active", 2024-09-10T09:54:33.376 INFO:teuthology.orchestra.run.smithi077.stdout: "fs_name": "cephfs", 2024-09-10T09:54:33.376 INFO:teuthology.orchestra.run.smithi077.stdout: "rank_uptime": 78.514497817000006, 2024-09-10T09:54:33.376 INFO:teuthology.orchestra.run.smithi077.stdout: "mdsmap_epoch": 28, 2024-09-10T09:54:33.376 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch": 96, 2024-09-10T09:54:33.376 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch_barrier": 94, 2024-09-10T09:54:33.376 INFO:teuthology.orchestra.run.smithi077.stdout: "uptime": 158.37901323099999 2024-09-10T09:54:33.376 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-10T09:54:33.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.371+0000 7ffa15800640 1 -- 172.21.15.77:0/4219877544 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ff9f40a2c00 msgr2=0x7ff9f40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:33.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.371+0000 7ffa15800640 1 --2- 172.21.15.77:0/4219877544 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ff9f40a2c00 0x7ff9f40a50c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ffa10076600 tx=0x7ffa080096c0 comp rx=0 tx=0).stop 2024-09-10T09:54:33.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.371+0000 7ffa15800640 1 -- 172.21.15.77:0/4219877544 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa10074590 msgr2=0x7ffa101a4730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:33.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.371+0000 7ffa15800640 1 --2- 172.21.15.77:0/4219877544 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa10074590 0x7ffa101a4730 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ffa0002a9a0 tx=0x7ffa00003c10 comp rx=0 tx=0).stop 2024-09-10T09:54:33.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.371+0000 7ffa15800640 1 -- 172.21.15.77:0/4219877544 shutdown_connections 2024-09-10T09:54:33.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.371+0000 7ffa15800640 1 --2- 172.21.15.77:0/4219877544 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ff9f40a2c00 0x7ff9f40a50c0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:33.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.371+0000 7ffa15800640 1 --2- 172.21.15.77:0/4219877544 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa10112b90 0x7ffa1019e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:33.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.371+0000 7ffa15800640 1 --2- 172.21.15.77:0/4219877544 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa10074eb0 0x7ffa101a4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:33.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.371+0000 7ffa15800640 1 --2- 172.21.15.77:0/4219877544 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa10074590 0x7ffa101a4730 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:33.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.371+0000 7ffa15800640 1 -- 172.21.15.77:0/4219877544 >> 172.21.15.77:0/4219877544 conn(0x7ffa100fd8c0 msgr2=0x7ffa100fefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:33.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.372+0000 7ffa15800640 1 -- 172.21.15.77:0/4219877544 shutdown_connections 2024-09-10T09:54:33.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:33.372+0000 7ffa15800640 1 -- 172.21.15.77:0/4219877544 wait complete. 2024-09-10T09:54:34.129 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.i status 2024-09-10T09:54:34.495 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:54:34.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:54:34 smithi077 ceph-mon[30319]: pgmap v373: 129 pgs: 129 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-10T09:54:35.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.090+0000 7f45d4a00640 1 -- 172.21.15.77:0/1606397829 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f45d010e420 msgr2=0x7f45d0110810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:35.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.090+0000 7f45d4a00640 1 --2- 172.21.15.77:0/1606397829 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f45d010e420 0x7f45d0110810 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f45c8009ca0 tx=0x7f45c802a950 comp rx=0 tx=0).stop 2024-09-10T09:54:35.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.090+0000 7f45d4a00640 1 -- 172.21.15.77:0/1606397829 shutdown_connections 2024-09-10T09:54:35.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.090+0000 7f45d4a00640 1 --2- 172.21.15.77:0/1606397829 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f45d010e420 0x7f45d0110810 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:35.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.090+0000 7f45d4a00640 1 --2- 172.21.15.77:0/1606397829 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f45d0074ee0 0x7f45d010dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:35.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.090+0000 7f45d4a00640 1 --2- 172.21.15.77:0/1606397829 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45d00745c0 0x7f45d00749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:35.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.090+0000 7f45d4a00640 1 -- 172.21.15.77:0/1606397829 >> 172.21.15.77:0/1606397829 conn(0x7f45d00fd8c0 msgr2=0x7f45d00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:35.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.090+0000 7f45d4a00640 1 -- 172.21.15.77:0/1606397829 shutdown_connections 2024-09-10T09:54:35.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.090+0000 7f45d4a00640 1 -- 172.21.15.77:0/1606397829 wait complete. 2024-09-10T09:54:35.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.091+0000 7f45d4a00640 1 Processor -- start 2024-09-10T09:54:35.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.091+0000 7f45d4a00640 1 -- start start 2024-09-10T09:54:35.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.092+0000 7f45d4a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f45d00745c0 0x7f45d019a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:35.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.092+0000 7f45d4a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45d0074ee0 0x7f45d019ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:35.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.092+0000 7f45d4a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f45d010e420 0x7f45d019eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:35.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.092+0000 7f45d4a00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f45d0114150 con 0x7f45d010e420 2024-09-10T09:54:35.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.092+0000 7f45d4a00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f45d0113fd0 con 0x7f45d00745c0 2024-09-10T09:54:35.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.092+0000 7f45d4a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f45d01142d0 con 0x7f45d0074ee0 2024-09-10T09:54:35.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.092+0000 7f45ce800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f45d010e420 0x7f45d019eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:35.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.092+0000 7f45ce800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f45d010e420 0x7f45d019eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34228/0 (socket says 172.21.15.77:34228) 2024-09-10T09:54:35.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.092+0000 7f45ce800640 1 -- 172.21.15.77:0/3958431593 learned_addr learned my addr 172.21.15.77:0/3958431593 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:54:35.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.092+0000 7f45cde00640 1 --2- 172.21.15.77:0/3958431593 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f45d00745c0 0x7f45d019a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:35.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.092+0000 7f45c5400640 1 --2- 172.21.15.77:0/3958431593 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45d0074ee0 0x7f45d019ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:35.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.093+0000 7f45ce800640 1 -- 172.21.15.77:0/3958431593 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45d0074ee0 msgr2=0x7f45d019ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:35.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.093+0000 7f45ce800640 1 --2- 172.21.15.77:0/3958431593 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45d0074ee0 0x7f45d019ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:35.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.093+0000 7f45ce800640 1 -- 172.21.15.77:0/3958431593 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f45d00745c0 msgr2=0x7f45d019a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:35.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.093+0000 7f45ce800640 1 --2- 172.21.15.77:0/3958431593 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f45d00745c0 0x7f45d019a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:35.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.093+0000 7f45ce800640 1 -- 172.21.15.77:0/3958431593 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45d019f5c0 con 0x7f45d010e420 2024-09-10T09:54:35.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.093+0000 7f45ce800640 1 --2- 172.21.15.77:0/3958431593 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f45d010e420 0x7f45d019eee0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f45c802ae70 tx=0x7f45c802f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:35.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.093+0000 7f45c7400640 1 -- 172.21.15.77:0/3958431593 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c8034040 con 0x7f45d010e420 2024-09-10T09:54:35.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.093+0000 7f45c7400640 1 -- 172.21.15.77:0/3958431593 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45c8030920 con 0x7f45d010e420 2024-09-10T09:54:35.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.093+0000 7f45d4a00640 1 -- 172.21.15.77:0/3958431593 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45d019f850 con 0x7f45d010e420 2024-09-10T09:54:35.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.093+0000 7f45c7400640 1 -- 172.21.15.77:0/3958431593 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c8045480 con 0x7f45d010e420 2024-09-10T09:54:35.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.093+0000 7f45d4a00640 1 -- 172.21.15.77:0/3958431593 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45d01a7190 con 0x7f45d010e420 2024-09-10T09:54:35.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.096+0000 7f45d75f4740 1 Processor -- start 2024-09-10T09:54:35.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.097+0000 7f45c7400640 1 -- 172.21.15.77:0/3958431593 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f45c8037020 con 0x7f45d010e420 2024-09-10T09:54:35.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.097+0000 7f45d75f4740 1 -- start start 2024-09-10T09:54:35.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.097+0000 7f45d75f4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56371a87d330 0x56371a87d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:35.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.097+0000 7f45d75f4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56371a87dc50 0x56371a887fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:35.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.097+0000 7f45d75f4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56371a888510 0x56371a88a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:35.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.097+0000 7f45d75f4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56371a7e30b0 con 0x56371a87d330 2024-09-10T09:54:35.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.097+0000 7f45d75f4740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56371a78aab0 con 0x56371a87dc50 2024-09-10T09:54:35.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.097+0000 7f45d75f4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56371a7df070 con 0x56371a888510 2024-09-10T09:54:35.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.097+0000 7f45c5400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56371a87d330 0x56371a87d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:35.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.098+0000 7f45c5400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56371a87d330 0x56371a87d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34240/0 (socket says 172.21.15.77:34240) 2024-09-10T09:54:35.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.098+0000 7f45c5400640 1 -- 172.21.15.77:0/3930883600 learned_addr learned my addr 172.21.15.77:0/3930883600 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:54:35.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.098+0000 7f45cde00640 1 --2- 172.21.15.77:0/3930883600 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56371a87dc50 0x56371a887fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:35.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.098+0000 7f45c5400640 1 --2- 172.21.15.77:0/3930883600 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56371a888510 0x56371a88a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:35.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.098+0000 7f45cde00640 1 -- 172.21.15.77:0/3930883600 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56371a888510 msgr2=0x56371a88a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:35.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.098+0000 7f45cde00640 1 --2- 172.21.15.77:0/3930883600 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56371a888510 0x56371a88a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:35.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.098+0000 7f45cde00640 1 -- 172.21.15.77:0/3930883600 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56371a87d330 msgr2=0x56371a87d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:35.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.098+0000 7f45cde00640 1 --2- 172.21.15.77:0/3930883600 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56371a87d330 0x56371a87d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:35.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.098+0000 7f45cde00640 1 -- 172.21.15.77:0/3930883600 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56371a7974a0 con 0x56371a87dc50 2024-09-10T09:54:35.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.098+0000 7f45c5400640 1 --2- 172.21.15.77:0/3930883600 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56371a87d330 0x56371a87d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:54:35.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.098+0000 7f45c7400640 1 --2- 172.21.15.77:0/3958431593 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f45b40a2c00 0x7f45b40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:35.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.098+0000 7f45c7400640 1 -- 172.21.15.77:0/3958431593 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f45c80f0290 con 0x7f45d010e420 2024-09-10T09:54:35.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.098+0000 7f45cde00640 1 --2- 172.21.15.77:0/3930883600 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56371a87dc50 0x56371a887fd0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f45c0005fd0 tx=0x7f45c00107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dbf88f71c7443f14 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:35.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.099+0000 7f45c5400640 1 --2- 172.21.15.77:0/3958431593 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f45b40a2c00 0x7f45b40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:35.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.099+0000 7f45bf000640 1 -- 172.21.15.77:0/3930883600 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c001c480 con 0x56371a87dc50 2024-09-10T09:54:35.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.099+0000 7f45bf000640 1 -- 172.21.15.77:0/3930883600 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45c001cb00 con 0x56371a87dc50 2024-09-10T09:54:35.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.099+0000 7f45bf000640 1 -- 172.21.15.77:0/3930883600 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c0011c10 con 0x56371a87dc50 2024-09-10T09:54:35.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.100+0000 7f45d75f4740 1 -- 172.21.15.77:0/3930883600 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56371a87dc50 msgr2=0x56371a887fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:35.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.100+0000 7f45c5400640 1 --2- 172.21.15.77:0/3958431593 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f45b40a2c00 0x7f45b40a50c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f45b80058a0 tx=0x7f45b800b710 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:35.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.100+0000 7f45d75f4740 1 --2- 172.21.15.77:0/3930883600 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56371a87dc50 0x56371a887fd0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f45c0005fd0 tx=0x7f45c00107a0 comp rx=0 tx=0).stop 2024-09-10T09:54:35.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.100+0000 7f45d75f4740 1 -- 172.21.15.77:0/3930883600 shutdown_connections 2024-09-10T09:54:35.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.100+0000 7f45d75f4740 1 --2- 172.21.15.77:0/3930883600 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56371a888510 0x56371a88a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:35.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.100+0000 7f45d75f4740 1 --2- 172.21.15.77:0/3930883600 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56371a87dc50 0x56371a887fd0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:35.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.100+0000 7f45d75f4740 1 --2- 172.21.15.77:0/3930883600 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56371a87d330 0x56371a87d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:35.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.100+0000 7f45d75f4740 1 -- 172.21.15.77:0/3930883600 >> 172.21.15.77:0/3930883600 conn(0x56371a8798c0 msgr2=0x56371a879ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:35.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.100+0000 7f45d75f4740 1 -- 172.21.15.77:0/3930883600 shutdown_connections 2024-09-10T09:54:35.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.100+0000 7f45d75f4740 1 -- 172.21.15.77:0/3930883600 wait complete. 2024-09-10T09:54:35.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.100+0000 7f45d75f4740 1 -- start start 2024-09-10T09:54:35.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.100+0000 7f45d75f4740 1 Processor -- start 2024-09-10T09:54:35.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.101+0000 7f45d75f4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56371a87d330 0x56371a885650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:35.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.101+0000 7f45d75f4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56371a87dc50 0x56371a885b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:35.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.101+0000 7f45d75f4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56371a888510 0x56371a8860f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:35.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.101+0000 7f45d75f4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56371a88b060 con 0x56371a87dc50 2024-09-10T09:54:35.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.101+0000 7f45d75f4740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56371a76b920 con 0x56371a888510 2024-09-10T09:54:35.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.101+0000 7f45d75f4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56371a88b1e0 con 0x56371a87d330 2024-09-10T09:54:35.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.101+0000 7f45d75f4740 10 client.0 _ll_register_callbacks cb 0x56371a78f8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:54:35.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.101+0000 7f45c5400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56371a87dc50 0x56371a885b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:35.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.101+0000 7f45cde00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56371a87d330 0x56371a885650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:35.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.101+0000 7f45ce800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56371a888510 0x56371a8860f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:35.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.101+0000 7f45cde00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56371a87d330 0x56371a885650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:59838/0 (socket says 172.21.15.77:59838) 2024-09-10T09:54:35.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.101+0000 7f45cde00640 1 -- 172.21.15.77:0/143346319 learned_addr learned my addr 172.21.15.77:0/143346319 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:54:35.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.102+0000 7f45cde00640 1 -- 172.21.15.77:0/143346319 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56371a888510 msgr2=0x56371a8860f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:35.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.102+0000 7f45cde00640 1 --2- 172.21.15.77:0/143346319 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56371a888510 0x56371a8860f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:35.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.102+0000 7f45cde00640 1 -- 172.21.15.77:0/143346319 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56371a87dc50 msgr2=0x56371a885b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:35.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.102+0000 7f45cde00640 1 --2- 172.21.15.77:0/143346319 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56371a87dc50 0x56371a885b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:35.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.102+0000 7f45cde00640 1 -- 172.21.15.77:0/143346319 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56371a5542b0 con 0x56371a87d330 2024-09-10T09:54:35.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.102+0000 7f45ce800640 1 --2- 172.21.15.77:0/143346319 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56371a888510 0x56371a8860f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:54:35.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.102+0000 7f45cde00640 1 --2- 172.21.15.77:0/143346319 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56371a87d330 0x56371a885650 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f45c0005d00 tx=0x7f45c00058c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d54f91f560b43cc2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:35.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.102+0000 7f45b3400640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-10T09:54:35.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.102+0000 7f45b3400640 1 -- 172.21.15.77:0/143346319 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c002b060 con 0x56371a87d330 2024-09-10T09:54:35.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.102+0000 7f45d75f4740 1 -- 172.21.15.77:0/143346319 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45c001cb00 con 0x56371a87d330 2024-09-10T09:54:35.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.102+0000 7f45b3400640 1 -- 172.21.15.77:0/143346319 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f45c001c690 con 0x56371a87d330 2024-09-10T09:54:35.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.103+0000 7f45b3400640 1 -- 172.21.15.77:0/143346319 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f45c0032050 con 0x56371a87d330 2024-09-10T09:54:35.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.103+0000 7f45d75f4740 10 client.25009 fetch_fsmap learned FSMap version 28 2024-09-10T09:54:35.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.103+0000 7f45d75f4740 1 -- 172.21.15.77:0/143346319 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56371a896ed0 con 0x56371a87d330 2024-09-10T09:54:35.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.103+0000 7f45b3400640 1 -- 172.21.15.77:0/143346319 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45c001cb00 con 0x56371a87d330 2024-09-10T09:54:35.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.103+0000 7f45b3400640 1 -- 172.21.15.77:0/143346319 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f45c001a790 con 0x56371a87d330 2024-09-10T09:54:35.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.103+0000 7f45d75f4740 10 client.25009 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T09:54:35.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.103+0000 7f45d75f4740 10 client.25009 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-10T09:54:35.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.103+0000 7f45d75f4740 20 client.25009 populate_metadata read hostname 'smithi077' 2024-09-10T09:54:35.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.103+0000 7f45d75f4740 1 --2- 172.21.15.77:0/143346319 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x56371a899140 0x56371a89a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:35.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.103+0000 7f45d75f4740 4 client.25009 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:54:35.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.103+0000 7f45d75f4740 1 -- 172.21.15.77:0/143346319 --> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56371a7f0620 con 0x56371a899140 2024-09-10T09:54:35.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.104+0000 7f45c5400640 1 --2- 172.21.15.77:0/143346319 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x56371a899140 0x56371a89a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:35.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.104+0000 7f45c5400640 1 --2- 172.21.15.77:0/143346319 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x56371a899140 0x56371a89a5b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f0e16860f9959842 server_cookie=3e7651e69a1da525 in_seq=0 out_seq=0 2024-09-10T09:54:35.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.104+0000 7f45b3400640 10 client.25009 ms_handle_connect on v2:172.21.15.134:6838/1965117680 2024-09-10T09:54:35.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.105+0000 7f45b3400640 1 -- 172.21.15.77:0/143346319 <== mds.2 v2:172.21.15.134:6838/1965117680 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56371a7f0620 con 0x56371a899140 2024-09-10T09:54:35.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.105+0000 7f45b3400640 10 client.25009 handle_command_reply: tid=0 2024-09-10T09:54:35.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.105+0000 7f45d75f4740 1 client.25009 shutdown 2024-09-10T09:54:35.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.105+0000 7f45d75f4740 1 -- 172.21.15.77:0/143346319 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56371a87d330 msgr2=0x56371a885650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:35.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.105+0000 7f45d75f4740 1 --2- 172.21.15.77:0/143346319 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56371a87d330 0x56371a885650 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f45c0005d00 tx=0x7f45c00058c0 comp rx=0 tx=0).stop 2024-09-10T09:54:35.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.105+0000 7f45d75f4740 1 -- 172.21.15.77:0/143346319 shutdown_connections 2024-09-10T09:54:35.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.105+0000 7f45d75f4740 1 --2- 172.21.15.77:0/143346319 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x56371a899140 0x56371a89a5b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:35.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.105+0000 7f45d75f4740 1 --2- 172.21.15.77:0/143346319 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56371a888510 0x56371a8860f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:35.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.105+0000 7f45d75f4740 1 --2- 172.21.15.77:0/143346319 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56371a87dc50 0x56371a885b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:35.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.105+0000 7f45b3400640 0 client.25009 ms_handle_reset on v2:172.21.15.134:6838/1965117680 2024-09-10T09:54:35.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.105+0000 7f45d75f4740 1 --2- 172.21.15.77:0/143346319 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56371a87d330 0x56371a885650 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:35.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.105+0000 7f45d75f4740 1 -- 172.21.15.77:0/143346319 >> 172.21.15.77:0/143346319 conn(0x56371a8798c0 msgr2=0x56371a87f5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:35.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.106+0000 7f45d75f4740 1 -- 172.21.15.77:0/143346319 shutdown_connections 2024-09-10T09:54:35.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.106+0000 7f45d75f4740 1 -- 172.21.15.77:0/143346319 wait complete. 2024-09-10T09:54:35.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.106+0000 7f45d75f4740 20 client.25009 trim_cache size 0 max 16384 2024-09-10T09:54:35.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.124+0000 7f45d75f4740 1 Processor -- start 2024-09-10T09:54:35.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.124+0000 7f45d75f4740 1 -- start start 2024-09-10T09:54:35.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.124+0000 7f45d75f4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56371a87d330 0x56371a899810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:35.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.124+0000 7f45d75f4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56371a87dc50 0x56371a885f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:35.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.124+0000 7f45d75f4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56371a888510 0x56371a882930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:35.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.124+0000 7f45d75f4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56371a8988f0 con 0x56371a888510 2024-09-10T09:54:35.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.124+0000 7f45d75f4740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56371a898fc0 con 0x56371a87dc50 2024-09-10T09:54:35.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.124+0000 7f45d75f4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56371a899690 con 0x56371a87d330 2024-09-10T09:54:35.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.125+0000 7f45cde00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56371a87d330 0x56371a899810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:35.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.125+0000 7f45cde00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56371a87d330 0x56371a899810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:59846/0 (socket says 172.21.15.77:59846) 2024-09-10T09:54:35.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.125+0000 7f45c5400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56371a87dc50 0x56371a885f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:35.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.125+0000 7f45cde00640 1 -- 172.21.15.77:0/1568946883 learned_addr learned my addr 172.21.15.77:0/1568946883 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:54:35.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.125+0000 7f45ce800640 1 --2- 172.21.15.77:0/1568946883 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56371a888510 0x56371a882930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:35.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.125+0000 7f45cde00640 1 -- 172.21.15.77:0/1568946883 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56371a87dc50 msgr2=0x56371a885f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:35.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.125+0000 7f45cde00640 1 --2- 172.21.15.77:0/1568946883 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56371a87dc50 0x56371a885f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:35.128 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.125+0000 7f45cde00640 1 -- 172.21.15.77:0/1568946883 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56371a888510 msgr2=0x56371a882930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:35.128 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.125+0000 7f45cde00640 1 --2- 172.21.15.77:0/1568946883 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56371a888510 0x56371a882930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:35.128 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.125+0000 7f45cde00640 1 -- 172.21.15.77:0/1568946883 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56371a87da80 con 0x56371a87d330 2024-09-10T09:54:35.128 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.125+0000 7f45c5400640 1 --2- 172.21.15.77:0/1568946883 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56371a87dc50 0x56371a885f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:54:35.128 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.125+0000 7f45cde00640 1 --2- 172.21.15.77:0/1568946883 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56371a87d330 0x56371a899810 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f45c001b000 tx=0x7f45c000b520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a90d30f19dde74e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:35.128 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.126+0000 7f45be600640 1 -- 172.21.15.77:0/1568946883 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c0010ca0 con 0x56371a87d330 2024-09-10T09:54:35.128 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.126+0000 7f45be600640 1 -- 172.21.15.77:0/1568946883 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45c0011cc0 con 0x56371a87d330 2024-09-10T09:54:35.128 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.126+0000 7f45be600640 1 -- 172.21.15.77:0/1568946883 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c001a5c0 con 0x56371a87d330 2024-09-10T09:54:35.128 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.126+0000 7f45d75f4740 1 -- 172.21.15.77:0/1568946883 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56371a87d330 msgr2=0x56371a899810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:35.128 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.126+0000 7f45d75f4740 1 --2- 172.21.15.77:0/1568946883 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56371a87d330 0x56371a899810 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f45c001b000 tx=0x7f45c000b520 comp rx=0 tx=0).stop 2024-09-10T09:54:35.128 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.126+0000 7f45d75f4740 1 -- 172.21.15.77:0/1568946883 shutdown_connections 2024-09-10T09:54:35.128 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.126+0000 7f45d75f4740 1 --2- 172.21.15.77:0/1568946883 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56371a888510 0x56371a882930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:35.128 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.126+0000 7f45d75f4740 1 --2- 172.21.15.77:0/1568946883 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56371a87dc50 0x56371a885f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:35.128 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.126+0000 7f45d75f4740 1 --2- 172.21.15.77:0/1568946883 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56371a87d330 0x56371a899810 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:35.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.126+0000 7f45d75f4740 1 -- 172.21.15.77:0/1568946883 >> 172.21.15.77:0/1568946883 conn(0x56371a8798c0 msgr2=0x56371a883f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:35.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.126+0000 7f45d75f4740 1 -- 172.21.15.77:0/1568946883 shutdown_connections 2024-09-10T09:54:35.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.126+0000 7f45d75f4740 1 -- 172.21.15.77:0/1568946883 wait complete. 2024-09-10T09:54:35.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.127+0000 7f45d75f4740 1 -- start start 2024-09-10T09:54:35.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.127+0000 7f45d75f4740 1 Processor -- start 2024-09-10T09:54:35.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.127+0000 7f45d75f4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56371a87d330 0x56371a8a29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:35.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.127+0000 7f45d75f4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56371a87dc50 0x56371a8a2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:35.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.127+0000 7f45d75f4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56371a888510 0x56371a881f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:35.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.127+0000 7f45d75f4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56371a89a430 con 0x56371a87dc50 2024-09-10T09:54:35.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.127+0000 7f45d75f4740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56371a899d60 con 0x56371a87d330 2024-09-10T09:54:35.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.127+0000 7f45d75f4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56371a89ab00 con 0x56371a888510 2024-09-10T09:54:35.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.128+0000 7f45d75f4740 10 client.0 _ll_register_callbacks cb 0x7f45c000bae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:54:35.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.128+0000 7f45cde00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56371a87d330 0x56371a8a29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:35.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.128+0000 7f45ce800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56371a888510 0x56371a881f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:35.132 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.128+0000 7f45cde00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56371a87d330 0x56371a8a29b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:34086/0 (socket says 172.21.15.77:34086) 2024-09-10T09:54:35.132 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.128+0000 7f45cde00640 1 -- 172.21.15.77:0/3274692365 learned_addr learned my addr 172.21.15.77:0/3274692365 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:54:35.132 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.128+0000 7f45c5400640 1 --2- 172.21.15.77:0/3274692365 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56371a87dc50 0x56371a8a2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:35.132 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.128+0000 7f45cde00640 1 -- 172.21.15.77:0/3274692365 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56371a888510 msgr2=0x56371a881f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:35.132 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.128+0000 7f45cde00640 1 --2- 172.21.15.77:0/3274692365 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56371a888510 0x56371a881f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:35.132 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.128+0000 7f45cde00640 1 -- 172.21.15.77:0/3274692365 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56371a87dc50 msgr2=0x56371a8a2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:35.132 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.128+0000 7f45cde00640 1 --2- 172.21.15.77:0/3274692365 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56371a87dc50 0x56371a8a2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:35.132 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.128+0000 7f45cde00640 1 -- 172.21.15.77:0/3274692365 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56371a8826c0 con 0x56371a87d330 2024-09-10T09:54:35.132 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.128+0000 7f45cde00640 1 --2- 172.21.15.77:0/3274692365 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56371a87d330 0x56371a8a29b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f45c00107e0 tx=0x7f45c0011e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e92e10bf79e6972d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:35.132 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.128+0000 7f45b3400640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T09:54:35.132 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.129+0000 7f45b3400640 1 -- 172.21.15.77:0/3274692365 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c001cb00 con 0x56371a87d330 2024-09-10T09:54:35.132 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.129+0000 7f45d75f4740 1 -- 172.21.15.77:0/3274692365 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45c0011cc0 con 0x56371a87d330 2024-09-10T09:54:35.132 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.129+0000 7f45b3400640 1 -- 172.21.15.77:0/3274692365 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f45c0022040 con 0x56371a87d330 2024-09-10T09:54:35.133 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.129+0000 7f45b3400640 1 -- 172.21.15.77:0/3274692365 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f45c004b050 con 0x56371a87d330 2024-09-10T09:54:35.133 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.129+0000 7f45d75f4740 10 client.25040 fetch_fsmap learned FSMap version 28 2024-09-10T09:54:35.133 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.129+0000 7f45d75f4740 1 -- 172.21.15.77:0/3274692365 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56371a8a2120 con 0x56371a87d330 2024-09-10T09:54:35.133 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.129+0000 7f45b3400640 1 -- 172.21.15.77:0/3274692365 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45c0011cc0 con 0x56371a87d330 2024-09-10T09:54:35.133 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.130+0000 7f45b3400640 1 -- 172.21.15.77:0/3274692365 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f45c0046030 con 0x56371a87d330 2024-09-10T09:54:35.133 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.130+0000 7f45d75f4740 10 client.25040 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T09:54:35.133 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.130+0000 7f45d75f4740 10 client.25040 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-10T09:54:35.133 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.130+0000 7f45d75f4740 20 client.25040 populate_metadata read hostname 'smithi077' 2024-09-10T09:54:35.133 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.130+0000 7f45d75f4740 1 --2- 172.21.15.77:0/3274692365 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x56371a907a30 0x56371a906740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:35.133 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.130+0000 7f45d75f4740 4 client.25040 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T09:54:35.133 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.130+0000 7f45d75f4740 1 -- 172.21.15.77:0/3274692365 --> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] -- command(tid 0: {"prefix": "status"}) -- 0x56371a7eab30 con 0x56371a907a30 2024-09-10T09:54:35.133 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.130+0000 7f45c5400640 1 --2- 172.21.15.77:0/3274692365 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x56371a907a30 0x56371a906740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:35.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:54:34 smithi134 ceph-mon[35674]: pgmap v373: 129 pgs: 129 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-10T09:54:35.135 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.131+0000 7f45c5400640 1 --2- 172.21.15.77:0/3274692365 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x56371a907a30 0x56371a906740 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9915bb51df258446 server_cookie=ad5dac930c65da83 in_seq=0 out_seq=0 2024-09-10T09:54:35.135 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.131+0000 7f45b3400640 10 client.25040 ms_handle_connect on v2:172.21.15.134:6838/1965117680 2024-09-10T09:54:35.135 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.131+0000 7f45b3400640 1 -- 172.21.15.77:0/3274692365 <== mds.2 v2:172.21.15.134:6838/1965117680 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x56371a7eab30 con 0x56371a907a30 2024-09-10T09:54:35.135 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.131+0000 7f45b3400640 10 client.25040 handle_command_reply: tid=0 2024-09-10T09:54:35.135 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.131+0000 7f45d75f4740 1 client.25040 shutdown 2024-09-10T09:54:35.135 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.131+0000 7f45d75f4740 1 -- 172.21.15.77:0/3274692365 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56371a87d330 msgr2=0x56371a8a29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:35.135 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.131+0000 7f45d75f4740 1 --2- 172.21.15.77:0/3274692365 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56371a87d330 0x56371a8a29b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f45c00107e0 tx=0x7f45c0011e70 comp rx=0 tx=0).stop 2024-09-10T09:54:35.135 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.131+0000 7f45d75f4740 1 -- 172.21.15.77:0/3274692365 shutdown_connections 2024-09-10T09:54:35.135 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.131+0000 7f45d75f4740 1 --2- 172.21.15.77:0/3274692365 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x56371a907a30 0x56371a906740 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:35.135 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.131+0000 7f45d75f4740 1 --2- 172.21.15.77:0/3274692365 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56371a888510 0x56371a881f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:35.135 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.131+0000 7f45d75f4740 1 --2- 172.21.15.77:0/3274692365 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56371a87dc50 0x56371a8a2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:35.136 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.131+0000 7f45b3400640 0 client.25040 ms_handle_reset on v2:172.21.15.134:6838/1965117680 2024-09-10T09:54:35.136 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.131+0000 7f45d75f4740 1 --2- 172.21.15.77:0/3274692365 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56371a87d330 0x56371a8a29b0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:35.136 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.131+0000 7f45d75f4740 1 -- 172.21.15.77:0/3274692365 >> 172.21.15.77:0/3274692365 conn(0x56371a8798c0 msgr2=0x56371a878200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:35.136 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.131+0000 7f45d75f4740 1 -- 172.21.15.77:0/3274692365 shutdown_connections 2024-09-10T09:54:35.136 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.131+0000 7f45d75f4740 1 -- 172.21.15.77:0/3274692365 wait complete. 2024-09-10T09:54:35.136 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.132+0000 7f45d75f4740 20 client.25040 trim_cache size 0 max 16384 2024-09-10T09:54:35.136 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-10T09:54:35.136 INFO:teuthology.orchestra.run.smithi077.stdout: "cluster_fsid": "14a55226-6f59-11ef-bcea-c7b262605968", 2024-09-10T09:54:35.136 INFO:teuthology.orchestra.run.smithi077.stdout: "whoami": 2, 2024-09-10T09:54:35.136 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 24520, 2024-09-10T09:54:35.137 INFO:teuthology.orchestra.run.smithi077.stdout: "want_state": "up:active", 2024-09-10T09:54:35.137 INFO:teuthology.orchestra.run.smithi077.stdout: "state": "up:active", 2024-09-10T09:54:35.137 INFO:teuthology.orchestra.run.smithi077.stdout: "fs_name": "cephfs", 2024-09-10T09:54:35.137 INFO:teuthology.orchestra.run.smithi077.stdout: "rank_uptime": 81.281641119, 2024-09-10T09:54:35.137 INFO:teuthology.orchestra.run.smithi077.stdout: "mdsmap_epoch": 28, 2024-09-10T09:54:35.137 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch": 96, 2024-09-10T09:54:35.137 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch_barrier": 94, 2024-09-10T09:54:35.137 INFO:teuthology.orchestra.run.smithi077.stdout: "uptime": 154.73323655499999 2024-09-10T09:54:35.137 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-10T09:54:35.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.132+0000 7f45d4a00640 1 -- 172.21.15.77:0/3958431593 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f45b40a2c00 msgr2=0x7f45b40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:35.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.132+0000 7f45d4a00640 1 --2- 172.21.15.77:0/3958431593 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f45b40a2c00 0x7f45b40a50c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f45b80058a0 tx=0x7f45b800b710 comp rx=0 tx=0).stop 2024-09-10T09:54:35.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.132+0000 7f45d4a00640 1 -- 172.21.15.77:0/3958431593 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f45d010e420 msgr2=0x7f45d019eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:35.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.132+0000 7f45d4a00640 1 --2- 172.21.15.77:0/3958431593 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f45d010e420 0x7f45d019eee0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f45c802ae70 tx=0x7f45c802f5b0 comp rx=0 tx=0).stop 2024-09-10T09:54:35.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.132+0000 7f45d4a00640 1 -- 172.21.15.77:0/3958431593 shutdown_connections 2024-09-10T09:54:35.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.132+0000 7f45d4a00640 1 --2- 172.21.15.77:0/3958431593 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f45b40a2c00 0x7f45b40a50c0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:35.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.132+0000 7f45d4a00640 1 --2- 172.21.15.77:0/3958431593 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f45d010e420 0x7f45d019eee0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:35.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.132+0000 7f45d4a00640 1 --2- 172.21.15.77:0/3958431593 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45d0074ee0 0x7f45d019ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:35.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.132+0000 7f45d4a00640 1 --2- 172.21.15.77:0/3958431593 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f45d00745c0 0x7f45d019a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:35.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.132+0000 7f45d4a00640 1 -- 172.21.15.77:0/3958431593 >> 172.21.15.77:0/3958431593 conn(0x7f45d00fd8c0 msgr2=0x7f45d00fe520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:35.139 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.132+0000 7f45d4a00640 1 -- 172.21.15.77:0/3958431593 shutdown_connections 2024-09-10T09:54:35.139 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:35.133+0000 7f45d4a00640 1 -- 172.21.15.77:0/3958431593 wait complete. 2024-09-10T09:54:35.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:54:34 smithi120 ceph-mon[34770]: pgmap v373: 129 pgs: 129 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-10T09:54:35.973 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.h status 2024-09-10T09:54:36.342 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:54:36.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.894+0000 7f8991200640 1 -- 172.21.15.77:0/2933563992 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f898c0745c0 msgr2=0x7f898c0749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:36.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.894+0000 7f8991200640 1 --2- 172.21.15.77:0/2933563992 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f898c0745c0 0x7f898c0749a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f897c009ca0 tx=0x7f897c02a950 comp rx=0 tx=0).stop 2024-09-10T09:54:36.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.895+0000 7f8991200640 1 -- 172.21.15.77:0/2933563992 shutdown_connections 2024-09-10T09:54:36.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.895+0000 7f8991200640 1 --2- 172.21.15.77:0/2933563992 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f898c10e260 0x7f898c110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:36.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.895+0000 7f8991200640 1 --2- 172.21.15.77:0/2933563992 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f898c074ee0 0x7f898c10dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:36.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.895+0000 7f8991200640 1 --2- 172.21.15.77:0/2933563992 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f898c0745c0 0x7f898c0749a0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:36.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.895+0000 7f8991200640 1 -- 172.21.15.77:0/2933563992 >> 172.21.15.77:0/2933563992 conn(0x7f898c0fd6e0 msgr2=0x7f898c0ffb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:36.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.895+0000 7f8991200640 1 -- 172.21.15.77:0/2933563992 shutdown_connections 2024-09-10T09:54:36.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.895+0000 7f8991200640 1 -- 172.21.15.77:0/2933563992 wait complete. 2024-09-10T09:54:36.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.896+0000 7f8991200640 1 Processor -- start 2024-09-10T09:54:36.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.896+0000 7f8991200640 1 -- start start 2024-09-10T09:54:36.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.896+0000 7f8991200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f898c0745c0 0x7f898c19a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:36.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.896+0000 7f8991200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f898c074ee0 0x7f898c19aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:36.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.896+0000 7f8991200640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f898c10e260 0x7f898c19ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:36.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.896+0000 7f8991200640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f898c114000 con 0x7f898c074ee0 2024-09-10T09:54:36.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.896+0000 7f8991200640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f898c113e80 con 0x7f898c10e260 2024-09-10T09:54:36.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.896+0000 7f8991200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f898c114180 con 0x7f898c0745c0 2024-09-10T09:54:36.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.896+0000 7f8982a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f898c074ee0 0x7f898c19aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:36.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.896+0000 7f898b400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f898c0745c0 0x7f898c19a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:36.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.896+0000 7f8982a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f898c074ee0 0x7f898c19aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34298/0 (socket says 172.21.15.77:34298) 2024-09-10T09:54:36.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.896+0000 7f8982a00640 1 -- 172.21.15.77:0/2998849336 learned_addr learned my addr 172.21.15.77:0/2998849336 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:54:36.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.896+0000 7f898b400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f898c0745c0 0x7f898c19a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:59882/0 (socket says 172.21.15.77:59882) 2024-09-10T09:54:36.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.896+0000 7f898be00640 1 --2- 172.21.15.77:0/2998849336 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f898c10e260 0x7f898c19ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:36.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.897+0000 7f898b400640 1 -- 172.21.15.77:0/2998849336 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f898c10e260 msgr2=0x7f898c19ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:36.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.897+0000 7f898b400640 1 --2- 172.21.15.77:0/2998849336 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f898c10e260 0x7f898c19ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:36.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.897+0000 7f898b400640 1 -- 172.21.15.77:0/2998849336 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f898c074ee0 msgr2=0x7f898c19aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:36.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.897+0000 7f898b400640 1 --2- 172.21.15.77:0/2998849336 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f898c074ee0 0x7f898c19aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:36.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.897+0000 7f898b400640 1 -- 172.21.15.77:0/2998849336 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f898c19f470 con 0x7f898c0745c0 2024-09-10T09:54:36.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.897+0000 7f8982a00640 1 --2- 172.21.15.77:0/2998849336 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f898c074ee0 0x7f898c19aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:54:36.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.897+0000 7f898b400640 1 --2- 172.21.15.77:0/2998849336 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f898c0745c0 0x7f898c19a4c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f897c005e30 tx=0x7f897c003c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:36.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.898+0000 7f8988c00640 1 -- 172.21.15.77:0/2998849336 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f897c034040 con 0x7f898c0745c0 2024-09-10T09:54:36.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.898+0000 7f8988c00640 1 -- 172.21.15.77:0/2998849336 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f897c02f940 con 0x7f898c0745c0 2024-09-10T09:54:36.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.898+0000 7f8988c00640 1 -- 172.21.15.77:0/2998849336 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f897c0046e0 con 0x7f898c0745c0 2024-09-10T09:54:36.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.898+0000 7f8991200640 1 -- 172.21.15.77:0/2998849336 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f898c19f700 con 0x7f898c0745c0 2024-09-10T09:54:36.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.898+0000 7f8991200640 1 -- 172.21.15.77:0/2998849336 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f898c1a7040 con 0x7f898c0745c0 2024-09-10T09:54:36.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.902+0000 7f899544e740 1 Processor -- start 2024-09-10T09:54:36.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.902+0000 7f899544e740 1 -- start start 2024-09-10T09:54:36.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.902+0000 7f8988c00640 1 -- 172.21.15.77:0/2998849336 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f897c037020 con 0x7f898c0745c0 2024-09-10T09:54:36.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.902+0000 7f899544e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a3619d4e0 0x558a3619d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:36.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.902+0000 7f899544e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a3619de00 0x558a361a8180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:36.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.902+0000 7f899544e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a361a86c0 0x558a361aaab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:36.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.902+0000 7f899544e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558a361030b0 con 0x558a3619d4e0 2024-09-10T09:54:36.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.902+0000 7f899544e740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558a360aaab0 con 0x558a3619de00 2024-09-10T09:54:36.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.902+0000 7f899544e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558a360ff070 con 0x558a361a86c0 2024-09-10T09:54:36.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.903+0000 7f898be00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a3619d4e0 0x558a3619d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:36.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.903+0000 7f898b400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a3619de00 0x558a361a8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:36.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.903+0000 7f898be00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a3619d4e0 0x558a3619d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34306/0 (socket says 172.21.15.77:34306) 2024-09-10T09:54:36.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.903+0000 7f898be00640 1 -- 172.21.15.77:0/66532498 learned_addr learned my addr 172.21.15.77:0/66532498 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:54:36.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.903+0000 7f8982a00640 1 --2- 172.21.15.77:0/66532498 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a361a86c0 0x558a361aaab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:36.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.903+0000 7f898be00640 1 -- 172.21.15.77:0/66532498 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a361a86c0 msgr2=0x558a361aaab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:36.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.903+0000 7f898be00640 1 --2- 172.21.15.77:0/66532498 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a361a86c0 0x558a361aaab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:36.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.903+0000 7f898be00640 1 -- 172.21.15.77:0/66532498 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a3619de00 msgr2=0x558a361a8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:36.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.903+0000 7f898be00640 1 --2- 172.21.15.77:0/66532498 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a3619de00 0x558a361a8180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:36.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.903+0000 7f898be00640 1 -- 172.21.15.77:0/66532498 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a360b74a0 con 0x558a3619d4e0 2024-09-10T09:54:36.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.903+0000 7f898be00640 1 --2- 172.21.15.77:0/66532498 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a3619d4e0 0x558a3619d8c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8984009b80 tx=0x7f8984011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ce3acee3b4f1c43 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:36.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.904+0000 7f8980c00640 1 -- 172.21.15.77:0/66532498 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8984004030 con 0x558a3619d4e0 2024-09-10T09:54:36.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.904+0000 7f8980c00640 1 -- 172.21.15.77:0/66532498 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8984005500 con 0x558a3619d4e0 2024-09-10T09:54:36.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.904+0000 7f8980c00640 1 -- 172.21.15.77:0/66532498 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8984004e10 con 0x558a3619d4e0 2024-09-10T09:54:36.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.904+0000 7f8988c00640 1 --2- 172.21.15.77:0/2998849336 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f89700a2b80 0x7f89700a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:36.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.904+0000 7f8988c00640 1 -- 172.21.15.77:0/2998849336 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f897c0f0150 con 0x7f898c0745c0 2024-09-10T09:54:36.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.904+0000 7f898b400640 1 --2- 172.21.15.77:0/2998849336 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f89700a2b80 0x7f89700a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:36.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.905+0000 7f899544e740 1 -- 172.21.15.77:0/66532498 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a3619d4e0 msgr2=0x558a3619d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:36.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.905+0000 7f899544e740 1 --2- 172.21.15.77:0/66532498 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a3619d4e0 0x558a3619d8c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8984009b80 tx=0x7f8984011810 comp rx=0 tx=0).stop 2024-09-10T09:54:36.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.905+0000 7f899544e740 1 -- 172.21.15.77:0/66532498 shutdown_connections 2024-09-10T09:54:36.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.905+0000 7f899544e740 1 --2- 172.21.15.77:0/66532498 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a361a86c0 0x558a361aaab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:36.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.905+0000 7f899544e740 1 --2- 172.21.15.77:0/66532498 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a3619de00 0x558a361a8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:36.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.905+0000 7f899544e740 1 --2- 172.21.15.77:0/66532498 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a3619d4e0 0x558a3619d8c0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:36.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.905+0000 7f899544e740 1 -- 172.21.15.77:0/66532498 >> 172.21.15.77:0/66532498 conn(0x558a36199a70 msgr2=0x558a36199e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:36.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.905+0000 7f898b400640 1 --2- 172.21.15.77:0/2998849336 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f89700a2b80 0x7f89700a5040 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f897c0edbd0 tx=0x7f897c0f3c30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:36.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.905+0000 7f899544e740 1 -- 172.21.15.77:0/66532498 shutdown_connections 2024-09-10T09:54:36.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.905+0000 7f899544e740 1 -- 172.21.15.77:0/66532498 wait complete. 2024-09-10T09:54:36.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.905+0000 7f899544e740 1 -- start start 2024-09-10T09:54:36.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.906+0000 7f899544e740 1 Processor -- start 2024-09-10T09:54:36.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.906+0000 7f899544e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a3619d4e0 0x558a361a5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:36.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.906+0000 7f899544e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a3619de00 0x558a361a5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:36.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.906+0000 7f899544e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a361a86c0 0x558a361a6080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:36.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.906+0000 7f899544e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558a361ab330 con 0x558a3619d4e0 2024-09-10T09:54:36.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.906+0000 7f899544e740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558a3608b920 con 0x558a3619de00 2024-09-10T09:54:36.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.906+0000 7f899544e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558a361ab4b0 con 0x558a361a86c0 2024-09-10T09:54:36.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.906+0000 7f899544e740 10 client.0 _ll_register_callbacks cb 0x558a360af8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:54:36.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.906+0000 7f898be00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a3619d4e0 0x558a361a5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:36.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.906+0000 7f898be00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a3619d4e0 0x558a361a5600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34310/0 (socket says 172.21.15.77:34310) 2024-09-10T09:54:36.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.906+0000 7f898be00640 1 -- 172.21.15.77:0/2063704405 learned_addr learned my addr 172.21.15.77:0/2063704405 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:54:36.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.906+0000 7f8982a00640 1 --2- 172.21.15.77:0/2063704405 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a3619de00 0x558a361a5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:36.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.907+0000 7f898be00640 1 --2- 172.21.15.77:0/2063704405 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a361a86c0 0x558a361a6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:36.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.907+0000 7f898be00640 1 -- 172.21.15.77:0/2063704405 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a361a86c0 msgr2=0x558a361a6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:36.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.907+0000 7f898be00640 1 --2- 172.21.15.77:0/2063704405 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a361a86c0 0x558a361a6080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:36.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.907+0000 7f898be00640 1 -- 172.21.15.77:0/2063704405 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a3619de00 msgr2=0x558a361a5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:36.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.907+0000 7f898be00640 1 --2- 172.21.15.77:0/2063704405 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a3619de00 0x558a361a5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:36.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.907+0000 7f898be00640 1 -- 172.21.15.77:0/2063704405 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a35e742b0 con 0x558a3619d4e0 2024-09-10T09:54:36.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.907+0000 7f898be00640 1 --2- 172.21.15.77:0/2063704405 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a3619d4e0 0x558a361a5600 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f89840054b0 tx=0x7f8984004a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5199676a16447cf0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:36.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.907+0000 7f8979600640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-10T09:54:36.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.907+0000 7f8979600640 1 -- 172.21.15.77:0/2063704405 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f898402e060 con 0x558a3619d4e0 2024-09-10T09:54:36.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.907+0000 7f8979600640 1 -- 172.21.15.77:0/2063704405 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f8984029440 con 0x558a3619d4e0 2024-09-10T09:54:36.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.907+0000 7f899544e740 1 -- 172.21.15.77:0/2063704405 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8984005500 con 0x558a3619d4e0 2024-09-10T09:54:36.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.908+0000 7f8979600640 1 -- 172.21.15.77:0/2063704405 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8984035050 con 0x558a3619d4e0 2024-09-10T09:54:36.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.908+0000 7f899544e740 10 client.15003 fetch_fsmap learned FSMap version 28 2024-09-10T09:54:36.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.908+0000 7f899544e740 1 -- 172.21.15.77:0/2063704405 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558a361b7060 con 0x558a3619d4e0 2024-09-10T09:54:36.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.908+0000 7f8979600640 1 -- 172.21.15.77:0/2063704405 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8984005500 con 0x558a3619d4e0 2024-09-10T09:54:36.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.908+0000 7f8979600640 1 -- 172.21.15.77:0/2063704405 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f8984018070 con 0x558a3619d4e0 2024-09-10T09:54:36.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.908+0000 7f899544e740 10 client.15003 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T09:54:36.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.908+0000 7f899544e740 10 client.15003 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-10T09:54:36.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.908+0000 7f899544e740 20 client.15003 populate_metadata read hostname 'smithi077' 2024-09-10T09:54:36.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.908+0000 7f899544e740 1 --2- 172.21.15.77:0/2063704405 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x558a361b94a0 0x558a361ba910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:36.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.908+0000 7f899544e740 4 client.15003 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:54:36.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.908+0000 7f899544e740 1 -- 172.21.15.77:0/2063704405 --> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558a36110620 con 0x558a361b94a0 2024-09-10T09:54:36.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.909+0000 7f8982a00640 1 --2- 172.21.15.77:0/2063704405 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x558a361b94a0 0x558a361ba910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:36.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.909+0000 7f8982a00640 1 --2- 172.21.15.77:0/2063704405 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x558a361b94a0 0x558a361ba910 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=74ce5472d8dd024d server_cookie=7f614f111d989a77 in_seq=0 out_seq=0 2024-09-10T09:54:36.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.909+0000 7f8979600640 10 client.15003 ms_handle_connect on v2:172.21.15.120:6836/3033280709 2024-09-10T09:54:36.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.910+0000 7f8979600640 1 -- 172.21.15.77:0/2063704405 <== mds.1 v2:172.21.15.120:6836/3033280709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558a36110620 con 0x558a361b94a0 2024-09-10T09:54:36.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.910+0000 7f8979600640 10 client.15003 handle_command_reply: tid=0 2024-09-10T09:54:36.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.910+0000 7f899544e740 1 client.15003 shutdown 2024-09-10T09:54:36.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.911+0000 7f899544e740 1 -- 172.21.15.77:0/2063704405 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a3619d4e0 msgr2=0x558a361a5600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:36.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.911+0000 7f899544e740 1 --2- 172.21.15.77:0/2063704405 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a3619d4e0 0x558a361a5600 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f89840054b0 tx=0x7f8984004a10 comp rx=0 tx=0).stop 2024-09-10T09:54:36.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.911+0000 7f899544e740 1 -- 172.21.15.77:0/2063704405 shutdown_connections 2024-09-10T09:54:36.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.911+0000 7f899544e740 1 --2- 172.21.15.77:0/2063704405 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x558a361b94a0 0x558a361ba910 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:36.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.911+0000 7f899544e740 1 --2- 172.21.15.77:0/2063704405 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a361a86c0 0x558a361a6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:36.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.911+0000 7f899544e740 1 --2- 172.21.15.77:0/2063704405 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a3619de00 0x558a361a5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:36.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.911+0000 7f899544e740 1 --2- 172.21.15.77:0/2063704405 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a3619d4e0 0x558a361a5600 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:36.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.911+0000 7f8979600640 0 client.15003 ms_handle_reset on v2:172.21.15.120:6836/3033280709 2024-09-10T09:54:36.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.911+0000 7f899544e740 1 -- 172.21.15.77:0/2063704405 >> 172.21.15.77:0/2063704405 conn(0x558a36199a70 msgr2=0x558a3619f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:36.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.911+0000 7f899544e740 1 -- 172.21.15.77:0/2063704405 shutdown_connections 2024-09-10T09:54:36.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.911+0000 7f899544e740 1 -- 172.21.15.77:0/2063704405 wait complete. 2024-09-10T09:54:36.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.911+0000 7f899544e740 20 client.15003 trim_cache size 0 max 16384 2024-09-10T09:54:36.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.929+0000 7f899544e740 1 Processor -- start 2024-09-10T09:54:36.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.929+0000 7f899544e740 1 -- start start 2024-09-10T09:54:36.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.930+0000 7f899544e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a3619d4e0 0x558a361a2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:36.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.930+0000 7f899544e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a3619de00 0x558a361a2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:36.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.930+0000 7f899544e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a361a86c0 0x558a361b3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:36.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.930+0000 7f899544e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558a361b8c50 con 0x558a3619d4e0 2024-09-10T09:54:36.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.930+0000 7f899544e740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558a361b9320 con 0x558a3619de00 2024-09-10T09:54:36.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.930+0000 7f899544e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558a361b99f0 con 0x558a361a86c0 2024-09-10T09:54:36.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.930+0000 7f898be00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a3619d4e0 0x558a361a2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:36.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.930+0000 7f898be00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a3619d4e0 0x558a361a2860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34314/0 (socket says 172.21.15.77:34314) 2024-09-10T09:54:36.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.930+0000 7f898be00640 1 -- 172.21.15.77:0/2886483662 learned_addr learned my addr 172.21.15.77:0/2886483662 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:54:36.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.930+0000 7f8982a00640 1 --2- 172.21.15.77:0/2886483662 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a3619de00 0x558a361a2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:36.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.930+0000 7f898be00640 1 --2- 172.21.15.77:0/2886483662 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a361a86c0 0x558a361b3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:36.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.931+0000 7f898be00640 1 -- 172.21.15.77:0/2886483662 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a361a86c0 msgr2=0x558a361b3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:36.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.931+0000 7f898be00640 1 --2- 172.21.15.77:0/2886483662 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a361a86c0 0x558a361b3730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:36.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.931+0000 7f898be00640 1 -- 172.21.15.77:0/2886483662 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a3619de00 msgr2=0x558a361a2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:36.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.931+0000 7f898be00640 1 --2- 172.21.15.77:0/2886483662 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a3619de00 0x558a361a2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:36.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.931+0000 7f898be00640 1 -- 172.21.15.77:0/2886483662 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a3619d310 con 0x558a3619d4e0 2024-09-10T09:54:36.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.931+0000 7f8982a00640 1 --2- 172.21.15.77:0/2886483662 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a3619de00 0x558a361a2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:54:36.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.931+0000 7f898be00640 1 --2- 172.21.15.77:0/2886483662 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a3619d4e0 0x558a361a2860 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f8984009b80 tx=0x7f89840059b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dd87d75e8c80660 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:36.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.931+0000 7f897be00640 1 -- 172.21.15.77:0/2886483662 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89840054e0 con 0x558a3619d4e0 2024-09-10T09:54:36.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.931+0000 7f897be00640 1 -- 172.21.15.77:0/2886483662 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f898401f040 con 0x558a3619d4e0 2024-09-10T09:54:36.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.931+0000 7f897be00640 1 -- 172.21.15.77:0/2886483662 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8984004410 con 0x558a3619d4e0 2024-09-10T09:54:36.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.932+0000 7f899544e740 1 -- 172.21.15.77:0/2886483662 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a3619d4e0 msgr2=0x558a361a2860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:36.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.932+0000 7f899544e740 1 --2- 172.21.15.77:0/2886483662 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a3619d4e0 0x558a361a2860 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f8984009b80 tx=0x7f89840059b0 comp rx=0 tx=0).stop 2024-09-10T09:54:36.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.932+0000 7f899544e740 1 -- 172.21.15.77:0/2886483662 shutdown_connections 2024-09-10T09:54:36.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.932+0000 7f899544e740 1 --2- 172.21.15.77:0/2886483662 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a361a86c0 0x558a361b3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:36.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.932+0000 7f899544e740 1 --2- 172.21.15.77:0/2886483662 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a3619de00 0x558a361a2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:36.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.932+0000 7f899544e740 1 --2- 172.21.15.77:0/2886483662 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a3619d4e0 0x558a361a2860 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:36.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.932+0000 7f899544e740 1 -- 172.21.15.77:0/2886483662 >> 172.21.15.77:0/2886483662 conn(0x558a36199a70 msgr2=0x558a361a3ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:36.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.932+0000 7f899544e740 1 -- 172.21.15.77:0/2886483662 shutdown_connections 2024-09-10T09:54:36.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.932+0000 7f899544e740 1 -- 172.21.15.77:0/2886483662 wait complete. 2024-09-10T09:54:36.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.932+0000 7f899544e740 1 -- start start 2024-09-10T09:54:36.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.933+0000 7f899544e740 1 Processor -- start 2024-09-10T09:54:36.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.933+0000 7f899544e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a3619d4e0 0x558a361c2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:36.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.933+0000 7f899544e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a3619de00 0x558a361bc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:36.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.933+0000 7f899544e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a361a86c0 0x558a361bc850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:36.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.933+0000 7f899544e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558a361ba790 con 0x558a361a86c0 2024-09-10T09:54:36.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.933+0000 7f899544e740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558a361ba0c0 con 0x558a3619de00 2024-09-10T09:54:36.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.933+0000 7f899544e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558a361bae60 con 0x558a3619d4e0 2024-09-10T09:54:36.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.933+0000 7f899544e740 10 client.0 _ll_register_callbacks cb 0x7f8984009a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:54:36.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.933+0000 7f898be00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a3619d4e0 0x558a361c2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:36.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.933+0000 7f898be00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a3619d4e0 0x558a361c2fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:59924/0 (socket says 172.21.15.77:59924) 2024-09-10T09:54:36.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.933+0000 7f898be00640 1 -- 172.21.15.77:0/1938270327 learned_addr learned my addr 172.21.15.77:0/1938270327 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:54:36.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.933+0000 7f898be00640 1 --2- 172.21.15.77:0/1938270327 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a361a86c0 0x558a361bc850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:36.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.933+0000 7f8982a00640 1 --2- 172.21.15.77:0/1938270327 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a3619de00 0x558a361bc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:36.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.934+0000 7f898be00640 1 -- 172.21.15.77:0/1938270327 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a3619de00 msgr2=0x558a361bc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:36.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.934+0000 7f898be00640 1 --2- 172.21.15.77:0/1938270327 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a3619de00 0x558a361bc310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:36.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.934+0000 7f898be00640 1 -- 172.21.15.77:0/1938270327 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a361a86c0 msgr2=0x558a361bc850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:36.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.934+0000 7f898be00640 1 --2- 172.21.15.77:0/1938270327 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a361a86c0 0x558a361bc850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:36.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.934+0000 7f898be00640 1 -- 172.21.15.77:0/1938270327 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a361bcd90 con 0x558a3619d4e0 2024-09-10T09:54:36.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.934+0000 7f898be00640 1 --2- 172.21.15.77:0/1938270327 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a3619d4e0 0x558a361c2fe0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8984019920 tx=0x7f89840059b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9106d5206b0daa7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:36.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.934+0000 7f8979600640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-10T09:54:36.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.935+0000 7f8979600640 1 -- 172.21.15.77:0/1938270327 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8984043040 con 0x558a3619d4e0 2024-09-10T09:54:36.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.935+0000 7f899544e740 1 -- 172.21.15.77:0/1938270327 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f898401f040 con 0x558a3619d4e0 2024-09-10T09:54:36.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.935+0000 7f8979600640 1 -- 172.21.15.77:0/1938270327 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f89840199e0 con 0x558a3619d4e0 2024-09-10T09:54:36.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.935+0000 7f8979600640 1 -- 172.21.15.77:0/1938270327 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8984019be0 con 0x558a3619d4e0 2024-09-10T09:54:36.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.935+0000 7f899544e740 10 client.25033 fetch_fsmap learned FSMap version 28 2024-09-10T09:54:36.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.935+0000 7f899544e740 1 -- 172.21.15.77:0/1938270327 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558a361c1810 con 0x558a3619d4e0 2024-09-10T09:54:36.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.935+0000 7f8979600640 1 -- 172.21.15.77:0/1938270327 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f898401f040 con 0x558a3619d4e0 2024-09-10T09:54:36.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.935+0000 7f8979600640 1 -- 172.21.15.77:0/1938270327 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f8984018030 con 0x558a3619d4e0 2024-09-10T09:54:36.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.935+0000 7f899544e740 10 client.25033 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T09:54:36.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.935+0000 7f899544e740 10 client.25033 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-10T09:54:36.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.935+0000 7f899544e740 20 client.25033 populate_metadata read hostname 'smithi077' 2024-09-10T09:54:36.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.935+0000 7f899544e740 1 --2- 172.21.15.77:0/1938270327 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x558a362308f0 0x558a3622f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:36.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.936+0000 7f899544e740 4 client.25033 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T09:54:36.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.936+0000 7f899544e740 1 -- 172.21.15.77:0/1938270327 --> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] -- command(tid 0: {"prefix": "status"}) -- 0x558a3610ab30 con 0x558a362308f0 2024-09-10T09:54:36.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.936+0000 7f8982a00640 1 --2- 172.21.15.77:0/1938270327 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x558a362308f0 0x558a3622f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:36.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.936+0000 7f8982a00640 1 --2- 172.21.15.77:0/1938270327 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x558a362308f0 0x558a3622f600 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=90c84cf05949bc5 server_cookie=82293c6139d36818 in_seq=0 out_seq=0 2024-09-10T09:54:36.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.936+0000 7f8979600640 10 client.25033 ms_handle_connect on v2:172.21.15.120:6836/3033280709 2024-09-10T09:54:36.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.937+0000 7f8979600640 1 -- 172.21.15.77:0/1938270327 <== mds.1 v2:172.21.15.120:6836/3033280709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x558a3610ab30 con 0x558a362308f0 2024-09-10T09:54:36.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.937+0000 7f8979600640 10 client.25033 handle_command_reply: tid=0 2024-09-10T09:54:36.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.937+0000 7f899544e740 1 client.25033 shutdown 2024-09-10T09:54:36.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.937+0000 7f899544e740 1 -- 172.21.15.77:0/1938270327 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a3619d4e0 msgr2=0x558a361c2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:36.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.937+0000 7f899544e740 1 --2- 172.21.15.77:0/1938270327 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a3619d4e0 0x558a361c2fe0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8984019920 tx=0x7f89840059b0 comp rx=0 tx=0).stop 2024-09-10T09:54:36.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.937+0000 7f899544e740 1 -- 172.21.15.77:0/1938270327 shutdown_connections 2024-09-10T09:54:36.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.937+0000 7f899544e740 1 --2- 172.21.15.77:0/1938270327 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x558a362308f0 0x558a3622f600 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:36.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.937+0000 7f899544e740 1 --2- 172.21.15.77:0/1938270327 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a361a86c0 0x558a361bc850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:36.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.937+0000 7f899544e740 1 --2- 172.21.15.77:0/1938270327 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a3619de00 0x558a361bc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:36.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.937+0000 7f899544e740 1 --2- 172.21.15.77:0/1938270327 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a3619d4e0 0x558a361c2fe0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:36.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.937+0000 7f8979600640 0 client.25033 ms_handle_reset on v2:172.21.15.120:6836/3033280709 2024-09-10T09:54:36.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.937+0000 7f899544e740 1 -- 172.21.15.77:0/1938270327 >> 172.21.15.77:0/1938270327 conn(0x558a36199a70 msgr2=0x558a361a42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:36.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.937+0000 7f899544e740 1 -- 172.21.15.77:0/1938270327 shutdown_connections 2024-09-10T09:54:36.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.937+0000 7f899544e740 1 -- 172.21.15.77:0/1938270327 wait complete. 2024-09-10T09:54:36.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.937+0000 7f899544e740 20 client.25033 trim_cache size 0 max 16384 2024-09-10T09:54:36.942 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-10T09:54:36.943 INFO:teuthology.orchestra.run.smithi077.stdout: "cluster_fsid": "14a55226-6f59-11ef-bcea-c7b262605968", 2024-09-10T09:54:36.943 INFO:teuthology.orchestra.run.smithi077.stdout: "whoami": 1, 2024-09-10T09:54:36.943 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 24524, 2024-09-10T09:54:36.943 INFO:teuthology.orchestra.run.smithi077.stdout: "want_state": "up:active", 2024-09-10T09:54:36.943 INFO:teuthology.orchestra.run.smithi077.stdout: "state": "up:active", 2024-09-10T09:54:36.943 INFO:teuthology.orchestra.run.smithi077.stdout: "fs_name": "cephfs", 2024-09-10T09:54:36.943 INFO:teuthology.orchestra.run.smithi077.stdout: "rank_uptime": 84.098832337999994, 2024-09-10T09:54:36.943 INFO:teuthology.orchestra.run.smithi077.stdout: "mdsmap_epoch": 28, 2024-09-10T09:54:36.943 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch": 96, 2024-09-10T09:54:36.943 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch_barrier": 94, 2024-09-10T09:54:36.943 INFO:teuthology.orchestra.run.smithi077.stdout: "uptime": 159.22998980899999 2024-09-10T09:54:36.943 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-10T09:54:36.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.938+0000 7f8991200640 1 -- 172.21.15.77:0/2998849336 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f89700a2b80 msgr2=0x7f89700a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:36.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.938+0000 7f8991200640 1 --2- 172.21.15.77:0/2998849336 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f89700a2b80 0x7f89700a5040 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f897c0edbd0 tx=0x7f897c0f3c30 comp rx=0 tx=0).stop 2024-09-10T09:54:36.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.938+0000 7f8991200640 1 -- 172.21.15.77:0/2998849336 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f898c0745c0 msgr2=0x7f898c19a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:36.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.938+0000 7f8991200640 1 --2- 172.21.15.77:0/2998849336 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f898c0745c0 0x7f898c19a4c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f897c005e30 tx=0x7f897c003c00 comp rx=0 tx=0).stop 2024-09-10T09:54:36.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.938+0000 7f8991200640 1 -- 172.21.15.77:0/2998849336 shutdown_connections 2024-09-10T09:54:36.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.938+0000 7f8991200640 1 --2- 172.21.15.77:0/2998849336 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f89700a2b80 0x7f89700a5040 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:36.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.938+0000 7f8991200640 1 --2- 172.21.15.77:0/2998849336 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f898c10e260 0x7f898c19ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:36.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.938+0000 7f8991200640 1 --2- 172.21.15.77:0/2998849336 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f898c074ee0 0x7f898c19aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:36.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.938+0000 7f8991200640 1 --2- 172.21.15.77:0/2998849336 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f898c0745c0 0x7f898c19a4c0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:36.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.938+0000 7f8991200640 1 -- 172.21.15.77:0/2998849336 >> 172.21.15.77:0/2998849336 conn(0x7f898c0fd6e0 msgr2=0x7f898c0fe380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:36.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.938+0000 7f8991200640 1 -- 172.21.15.77:0/2998849336 shutdown_connections 2024-09-10T09:54:36.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:36.938+0000 7f8991200640 1 -- 172.21.15.77:0/2998849336 wait complete. 2024-09-10T09:54:37.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:54:36 smithi134 ceph-mon[35674]: pgmap v374: 129 pgs: 129 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-10T09:54:37.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:54:36 smithi120 ceph-mon[34770]: pgmap v374: 129 pgs: 129 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-10T09:54:37.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:54:36 smithi077 ceph-mon[30319]: pgmap v374: 129 pgs: 129 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-10T09:54:37.750 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-10T09:54:38.118 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:54:38.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.674+0000 7fd692200640 1 -- 172.21.15.77:0/2369764592 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd68c1079c0 msgr2=0x7fd68c107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:38.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.674+0000 7fd692200640 1 --2- 172.21.15.77:0/2369764592 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd68c1079c0 0x7fd68c107da0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd680009890 tx=0x7fd68002a360 comp rx=0 tx=0).stop 2024-09-10T09:54:38.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.674+0000 7fd692200640 1 -- 172.21.15.77:0/2369764592 shutdown_connections 2024-09-10T09:54:38.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.675+0000 7fd692200640 1 --2- 172.21.15.77:0/2369764592 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd68c102360 0x7fd68c10e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:38.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.675+0000 7fd692200640 1 --2- 172.21.15.77:0/2369764592 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd68c1019c0 0x7fd68c101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:38.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.675+0000 7fd692200640 1 --2- 172.21.15.77:0/2369764592 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd68c1079c0 0x7fd68c107da0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:38.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.675+0000 7fd692200640 1 -- 172.21.15.77:0/2369764592 >> 172.21.15.77:0/2369764592 conn(0x7fd68c0fd740 msgr2=0x7fd68c0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:38.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.675+0000 7fd692200640 1 -- 172.21.15.77:0/2369764592 shutdown_connections 2024-09-10T09:54:38.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.675+0000 7fd692200640 1 -- 172.21.15.77:0/2369764592 wait complete. 2024-09-10T09:54:38.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.675+0000 7fd692200640 1 Processor -- start 2024-09-10T09:54:38.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.676+0000 7fd692200640 1 -- start start 2024-09-10T09:54:38.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.676+0000 7fd692200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd68c1019c0 0x7fd68c19a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:38.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.676+0000 7fd692200640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd68c102360 0x7fd68c19a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:38.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.676+0000 7fd692200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd68c1079c0 0x7fd68c19ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:38.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.676+0000 7fd692200640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd68c111d90 con 0x7fd68c1019c0 2024-09-10T09:54:38.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.676+0000 7fd692200640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd68c111c10 con 0x7fd68c102360 2024-09-10T09:54:38.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.676+0000 7fd692200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd68c111f10 con 0x7fd68c1079c0 2024-09-10T09:54:38.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.676+0000 7fd690e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd68c1019c0 0x7fd68c19a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:38.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.676+0000 7fd690e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd68c1019c0 0x7fd68c19a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34358/0 (socket says 172.21.15.77:34358) 2024-09-10T09:54:38.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.676+0000 7fd690e00640 1 -- 172.21.15.77:0/3287603186 learned_addr learned my addr 172.21.15.77:0/3287603186 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:54:38.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.676+0000 7fd691800640 1 --2- 172.21.15.77:0/3287603186 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd68c1079c0 0x7fd68c19ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:38.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.676+0000 7fd68be00640 1 --2- 172.21.15.77:0/3287603186 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd68c102360 0x7fd68c19a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:38.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.677+0000 7fd690e00640 1 -- 172.21.15.77:0/3287603186 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd68c1079c0 msgr2=0x7fd68c19ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:38.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.677+0000 7fd690e00640 1 --2- 172.21.15.77:0/3287603186 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd68c1079c0 0x7fd68c19ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:38.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.677+0000 7fd690e00640 1 -- 172.21.15.77:0/3287603186 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd68c102360 msgr2=0x7fd68c19a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:38.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.677+0000 7fd690e00640 1 --2- 172.21.15.77:0/3287603186 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd68c102360 0x7fd68c19a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:38.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.677+0000 7fd690e00640 1 -- 172.21.15.77:0/3287603186 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd68c19f440 con 0x7fd68c1019c0 2024-09-10T09:54:38.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.677+0000 7fd68be00640 1 --2- 172.21.15.77:0/3287603186 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd68c102360 0x7fd68c19a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:54:38.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.677+0000 7fd690e00640 1 --2- 172.21.15.77:0/3287603186 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd68c1019c0 0x7fd68c19a490 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd680035040 tx=0x7fd680033b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:38.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.677+0000 7fd689600640 1 -- 172.21.15.77:0/3287603186 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd680037040 con 0x7fd68c1019c0 2024-09-10T09:54:38.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.677+0000 7fd692200640 1 -- 172.21.15.77:0/3287603186 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd68c19f6d0 con 0x7fd68c1019c0 2024-09-10T09:54:38.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.677+0000 7fd689600640 1 -- 172.21.15.77:0/3287603186 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd680040070 con 0x7fd68c1019c0 2024-09-10T09:54:38.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.677+0000 7fd689600640 1 -- 172.21.15.77:0/3287603186 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd680003970 con 0x7fd68c1019c0 2024-09-10T09:54:38.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.677+0000 7fd692200640 1 -- 172.21.15.77:0/3287603186 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd68c1a7010 con 0x7fd68c1019c0 2024-09-10T09:54:38.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.680+0000 7fd689600640 1 -- 172.21.15.77:0/3287603186 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd6800306a0 con 0x7fd68c1019c0 2024-09-10T09:54:38.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.680+0000 7fd696454740 1 Processor -- start 2024-09-10T09:54:38.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.680+0000 7fd696454740 1 -- start start 2024-09-10T09:54:38.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.681+0000 7fd696454740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56125f7954e0 0x56125f7958c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:38.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.681+0000 7fd696454740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56125f795e00 0x56125f7a0180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:38.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.681+0000 7fd696454740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56125f7a06c0 0x56125f7a2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:38.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.681+0000 7fd696454740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56125f6fb0b0 con 0x56125f795e00 2024-09-10T09:54:38.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.681+0000 7fd696454740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56125f6a2ab0 con 0x56125f7954e0 2024-09-10T09:54:38.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.681+0000 7fd696454740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56125f6f7070 con 0x56125f7a06c0 2024-09-10T09:54:38.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.681+0000 7fd690e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56125f795e00 0x56125f7a0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:38.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.681+0000 7fd691800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56125f7954e0 0x56125f7958c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:38.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.681+0000 7fd690e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56125f795e00 0x56125f7a0180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34366/0 (socket says 172.21.15.77:34366) 2024-09-10T09:54:38.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.681+0000 7fd690e00640 1 -- 172.21.15.77:0/916251013 learned_addr learned my addr 172.21.15.77:0/916251013 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:54:38.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.681+0000 7fd68be00640 1 --2- 172.21.15.77:0/916251013 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56125f7a06c0 0x56125f7a2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:38.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.681+0000 7fd691800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56125f7954e0 0x56125f7958c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:34182/0 (socket says 172.21.15.77:34182) 2024-09-10T09:54:38.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.682+0000 7fd691800640 1 -- 172.21.15.77:0/916251013 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56125f7a06c0 msgr2=0x56125f7a2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:38.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.682+0000 7fd691800640 1 --2- 172.21.15.77:0/916251013 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56125f7a06c0 0x56125f7a2ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:38.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.682+0000 7fd691800640 1 -- 172.21.15.77:0/916251013 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56125f795e00 msgr2=0x56125f7a0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:38.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.682+0000 7fd691800640 1 --2- 172.21.15.77:0/916251013 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56125f795e00 0x56125f7a0180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:38.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.682+0000 7fd691800640 1 -- 172.21.15.77:0/916251013 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56125f6af4a0 con 0x56125f7954e0 2024-09-10T09:54:38.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.682+0000 7fd68be00640 1 --2- 172.21.15.77:0/916251013 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56125f7a06c0 0x56125f7a2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:54:38.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.682+0000 7fd689600640 1 --2- 172.21.15.77:0/3287603186 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fd6740a2c00 0x7fd6740a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:38.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.682+0000 7fd689600640 1 -- 172.21.15.77:0/3287603186 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fd6800ef420 con 0x7fd68c1019c0 2024-09-10T09:54:38.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.682+0000 7fd691800640 1 --2- 172.21.15.77:0/916251013 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56125f7954e0 0x56125f7958c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd684009f00 tx=0x7fd684011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64171506be70164a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:38.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.682+0000 7fd690e00640 1 --2- 172.21.15.77:0/3287603186 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fd6740a2c00 0x7fd6740a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:38.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.682+0000 7fd67e000640 1 -- 172.21.15.77:0/916251013 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd684004d20 con 0x56125f7954e0 2024-09-10T09:54:38.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.682+0000 7fd67e000640 1 -- 172.21.15.77:0/916251013 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd684004050 con 0x56125f7954e0 2024-09-10T09:54:38.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.682+0000 7fd67e000640 1 -- 172.21.15.77:0/916251013 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd684005260 con 0x56125f7954e0 2024-09-10T09:54:38.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.683+0000 7fd696454740 1 -- 172.21.15.77:0/916251013 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56125f7954e0 msgr2=0x56125f7958c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:38.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.683+0000 7fd696454740 1 --2- 172.21.15.77:0/916251013 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56125f7954e0 0x56125f7958c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd684009f00 tx=0x7fd684011810 comp rx=0 tx=0).stop 2024-09-10T09:54:38.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.683+0000 7fd690e00640 1 --2- 172.21.15.77:0/3287603186 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fd6740a2c00 0x7fd6740a50c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd6800ef6c0 tx=0x7fd6800f2e60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:38.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.683+0000 7fd696454740 1 -- 172.21.15.77:0/916251013 shutdown_connections 2024-09-10T09:54:38.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.683+0000 7fd696454740 1 --2- 172.21.15.77:0/916251013 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56125f7a06c0 0x56125f7a2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:38.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.683+0000 7fd696454740 1 --2- 172.21.15.77:0/916251013 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56125f795e00 0x56125f7a0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:38.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.683+0000 7fd696454740 1 --2- 172.21.15.77:0/916251013 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56125f7954e0 0x56125f7958c0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:38.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.683+0000 7fd696454740 1 -- 172.21.15.77:0/916251013 >> 172.21.15.77:0/916251013 conn(0x56125f791a70 msgr2=0x56125f791e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:38.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.683+0000 7fd696454740 1 -- 172.21.15.77:0/916251013 shutdown_connections 2024-09-10T09:54:38.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.683+0000 7fd696454740 1 -- 172.21.15.77:0/916251013 wait complete. 2024-09-10T09:54:38.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.684+0000 7fd696454740 1 -- start start 2024-09-10T09:54:38.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.684+0000 7fd696454740 1 Processor -- start 2024-09-10T09:54:38.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.684+0000 7fd696454740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56125f7954e0 0x56125f79d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:38.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.684+0000 7fd696454740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56125f795e00 0x56125f79da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:38.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.684+0000 7fd696454740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56125f7a06c0 0x56125f79dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:38.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.684+0000 7fd696454740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56125f7a3330 con 0x56125f7954e0 2024-09-10T09:54:38.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.684+0000 7fd696454740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56125f683920 con 0x56125f7a06c0 2024-09-10T09:54:38.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.684+0000 7fd696454740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56125f7a34b0 con 0x56125f795e00 2024-09-10T09:54:38.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.684+0000 7fd696454740 10 client.0 _ll_register_callbacks cb 0x56125f6a78a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:54:38.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.685+0000 7fd68be00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56125f795e00 0x56125f79da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:38.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.685+0000 7fd68be00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56125f795e00 0x56125f79da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:59948/0 (socket says 172.21.15.77:59948) 2024-09-10T09:54:38.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.685+0000 7fd68be00640 1 -- 172.21.15.77:0/667687172 learned_addr learned my addr 172.21.15.77:0/667687172 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:54:38.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.685+0000 7fd691800640 1 --2- 172.21.15.77:0/667687172 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56125f7954e0 0x56125f79d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:38.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.685+0000 7fd691800640 1 --2- 172.21.15.77:0/667687172 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56125f7a06c0 0x56125f79dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:38.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.685+0000 7fd68be00640 1 -- 172.21.15.77:0/667687172 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56125f7a06c0 msgr2=0x56125f79dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:38.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.685+0000 7fd68be00640 1 --2- 172.21.15.77:0/667687172 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56125f7a06c0 0x56125f79dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:38.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.685+0000 7fd68be00640 1 -- 172.21.15.77:0/667687172 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56125f7954e0 msgr2=0x56125f79d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:38.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.685+0000 7fd68be00640 1 --2- 172.21.15.77:0/667687172 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56125f7954e0 0x56125f79d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:38.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.685+0000 7fd68be00640 1 -- 172.21.15.77:0/667687172 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56125f46c2b0 con 0x56125f795e00 2024-09-10T09:54:38.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.685+0000 7fd68be00640 1 --2- 172.21.15.77:0/667687172 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56125f795e00 0x56125f79da70 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd678013710 tx=0x7fd678013bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92b46c46bb92038d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:38.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.685+0000 7fd672a00640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-10T09:54:38.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.685+0000 7fd672a00640 1 -- 172.21.15.77:0/667687172 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd678011db0 con 0x56125f795e00 2024-09-10T09:54:38.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.686+0000 7fd696454740 1 -- 172.21.15.77:0/667687172 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd684004050 con 0x56125f795e00 2024-09-10T09:54:38.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.686+0000 7fd672a00640 1 -- 172.21.15.77:0/667687172 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fd678005020 con 0x56125f795e00 2024-09-10T09:54:38.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.686+0000 7fd672a00640 1 -- 172.21.15.77:0/667687172 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd678026050 con 0x56125f795e00 2024-09-10T09:54:38.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.686+0000 7fd696454740 10 client.25045 fetch_fsmap learned FSMap version 28 2024-09-10T09:54:38.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.686+0000 7fd696454740 1 -- 172.21.15.77:0/667687172 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56125f7af040 con 0x56125f795e00 2024-09-10T09:54:38.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.686+0000 7fd672a00640 1 -- 172.21.15.77:0/667687172 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd684004050 con 0x56125f795e00 2024-09-10T09:54:38.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.687+0000 7fd672a00640 1 -- 172.21.15.77:0/667687172 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fd67801a030 con 0x56125f795e00 2024-09-10T09:54:38.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.687+0000 7fd696454740 10 client.25045 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T09:54:38.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.687+0000 7fd696454740 10 client.25045 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T09:54:38.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.687+0000 7fd696454740 20 client.25045 populate_metadata read hostname 'smithi077' 2024-09-10T09:54:38.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.687+0000 7fd696454740 1 --2- 172.21.15.77:0/667687172 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x56125f7b1670 0x56125f7b2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:38.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.687+0000 7fd696454740 4 client.25045 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:54:38.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.687+0000 7fd696454740 1 -- 172.21.15.77:0/667687172 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56125f708620 con 0x56125f7b1670 2024-09-10T09:54:38.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.687+0000 7fd691800640 1 --2- 172.21.15.77:0/667687172 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x56125f7b1670 0x56125f7b2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:38.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.688+0000 7fd691800640 1 --2- 172.21.15.77:0/667687172 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x56125f7b1670 0x56125f7b2ae0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9a3cb26fa5079e2 server_cookie=650fd71751a6a18e in_seq=0 out_seq=0 2024-09-10T09:54:38.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.688+0000 7fd672a00640 10 client.25045 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T09:54:38.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.688+0000 7fd672a00640 1 -- 172.21.15.77:0/667687172 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56125f708620 con 0x56125f7b1670 2024-09-10T09:54:38.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.689+0000 7fd672a00640 10 client.25045 handle_command_reply: tid=0 2024-09-10T09:54:38.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.689+0000 7fd696454740 1 client.25045 shutdown 2024-09-10T09:54:38.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.689+0000 7fd696454740 1 -- 172.21.15.77:0/667687172 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56125f795e00 msgr2=0x56125f79da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:38.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.689+0000 7fd696454740 1 --2- 172.21.15.77:0/667687172 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56125f795e00 0x56125f79da70 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd678013710 tx=0x7fd678013bd0 comp rx=0 tx=0).stop 2024-09-10T09:54:38.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.689+0000 7fd696454740 1 -- 172.21.15.77:0/667687172 shutdown_connections 2024-09-10T09:54:38.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.689+0000 7fd696454740 1 --2- 172.21.15.77:0/667687172 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x56125f7b1670 0x56125f7b2ae0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:38.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.689+0000 7fd696454740 1 --2- 172.21.15.77:0/667687172 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56125f7a06c0 0x56125f79dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:38.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.689+0000 7fd696454740 1 --2- 172.21.15.77:0/667687172 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56125f795e00 0x56125f79da70 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:38.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.689+0000 7fd696454740 1 --2- 172.21.15.77:0/667687172 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56125f7954e0 0x56125f79d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:38.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.689+0000 7fd672a00640 0 client.25045 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T09:54:38.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.689+0000 7fd696454740 1 -- 172.21.15.77:0/667687172 >> 172.21.15.77:0/667687172 conn(0x56125f791a70 msgr2=0x56125f7977a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:38.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.689+0000 7fd696454740 1 -- 172.21.15.77:0/667687172 shutdown_connections 2024-09-10T09:54:38.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.689+0000 7fd696454740 1 -- 172.21.15.77:0/667687172 wait complete. 2024-09-10T09:54:38.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.689+0000 7fd696454740 20 client.25045 trim_cache size 0 max 16384 2024-09-10T09:54:38.708 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.707+0000 7fd696454740 1 Processor -- start 2024-09-10T09:54:38.708 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.708+0000 7fd696454740 1 -- start start 2024-09-10T09:54:38.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.708+0000 7fd696454740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56125f7954e0 0x56125f7b2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:38.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.708+0000 7fd696454740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56125f795e00 0x56125f797f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:38.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.708+0000 7fd696454740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56125f7a06c0 0x56125f7abe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:38.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.708+0000 7fd696454740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56125f7b0e20 con 0x56125f7a06c0 2024-09-10T09:54:38.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.708+0000 7fd696454740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56125f7b14f0 con 0x56125f7954e0 2024-09-10T09:54:38.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.708+0000 7fd696454740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56125f7b1bc0 con 0x56125f795e00 2024-09-10T09:54:38.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.708+0000 7fd691800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56125f7a06c0 0x56125f7abe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:38.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.708+0000 7fd691800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56125f7a06c0 0x56125f7abe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34380/0 (socket says 172.21.15.77:34380) 2024-09-10T09:54:38.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.708+0000 7fd691800640 1 -- 172.21.15.77:0/458144007 learned_addr learned my addr 172.21.15.77:0/458144007 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:54:38.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.708+0000 7fd691800640 1 --2- 172.21.15.77:0/458144007 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56125f7954e0 0x56125f7b2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:38.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.708+0000 7fd68be00640 1 --2- 172.21.15.77:0/458144007 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56125f795e00 0x56125f797f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:38.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.709+0000 7fd691800640 1 -- 172.21.15.77:0/458144007 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56125f795e00 msgr2=0x56125f797f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:38.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.709+0000 7fd691800640 1 --2- 172.21.15.77:0/458144007 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56125f795e00 0x56125f797f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:38.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.709+0000 7fd691800640 1 -- 172.21.15.77:0/458144007 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56125f7954e0 msgr2=0x56125f7b2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:38.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.709+0000 7fd691800640 1 --2- 172.21.15.77:0/458144007 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56125f7954e0 0x56125f7b2ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:38.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.709+0000 7fd691800640 1 -- 172.21.15.77:0/458144007 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56125f79fea0 con 0x56125f7a06c0 2024-09-10T09:54:38.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.709+0000 7fd691800640 1 --2- 172.21.15.77:0/458144007 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56125f7a06c0 0x56125f7abe40 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd684006010 tx=0x7fd684009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80bdc1a850f54229 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:38.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.709+0000 7fd67d600640 1 -- 172.21.15.77:0/458144007 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd684029960 con 0x56125f7a06c0 2024-09-10T09:54:38.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.709+0000 7fd67d600640 1 -- 172.21.15.77:0/458144007 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd684019430 con 0x56125f7a06c0 2024-09-10T09:54:38.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.709+0000 7fd67d600640 1 -- 172.21.15.77:0/458144007 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd684003e00 con 0x56125f7a06c0 2024-09-10T09:54:38.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.710+0000 7fd696454740 1 -- 172.21.15.77:0/458144007 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56125f7a06c0 msgr2=0x56125f7abe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:38.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.710+0000 7fd696454740 1 --2- 172.21.15.77:0/458144007 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56125f7a06c0 0x56125f7abe40 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd684006010 tx=0x7fd684009500 comp rx=0 tx=0).stop 2024-09-10T09:54:38.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.710+0000 7fd696454740 1 -- 172.21.15.77:0/458144007 shutdown_connections 2024-09-10T09:54:38.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.710+0000 7fd696454740 1 --2- 172.21.15.77:0/458144007 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56125f7a06c0 0x56125f7abe40 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:38.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.710+0000 7fd696454740 1 --2- 172.21.15.77:0/458144007 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56125f795e00 0x56125f797f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:38.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.710+0000 7fd696454740 1 --2- 172.21.15.77:0/458144007 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56125f7954e0 0x56125f7b2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:38.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.710+0000 7fd696454740 1 -- 172.21.15.77:0/458144007 >> 172.21.15.77:0/458144007 conn(0x56125f791a70 msgr2=0x56125f7ab730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:38.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.710+0000 7fd696454740 1 -- 172.21.15.77:0/458144007 shutdown_connections 2024-09-10T09:54:38.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.710+0000 7fd696454740 1 -- 172.21.15.77:0/458144007 wait complete. 2024-09-10T09:54:38.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.711+0000 7fd696454740 1 -- start start 2024-09-10T09:54:38.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.711+0000 7fd696454740 1 Processor -- start 2024-09-10T09:54:38.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.711+0000 7fd696454740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56125f7954e0 0x56125f7bb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:38.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.711+0000 7fd696454740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56125f795e00 0x56125f7b4920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:38.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.711+0000 7fd696454740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56125f7a06c0 0x56125f7b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:38.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.711+0000 7fd696454740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56125f7b2960 con 0x56125f795e00 2024-09-10T09:54:38.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.711+0000 7fd696454740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56125f7b2290 con 0x56125f7a06c0 2024-09-10T09:54:38.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.711+0000 7fd696454740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56125f7b3030 con 0x56125f7954e0 2024-09-10T09:54:38.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.711+0000 7fd696454740 10 client.0 _ll_register_callbacks cb 0x7fd678009e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:54:38.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.711+0000 7fd68be00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56125f795e00 0x56125f7b4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:38.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.712+0000 7fd68be00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56125f795e00 0x56125f7b4920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34388/0 (socket says 172.21.15.77:34388) 2024-09-10T09:54:38.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.712+0000 7fd68be00640 1 -- 172.21.15.77:0/998376614 learned_addr learned my addr 172.21.15.77:0/998376614 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:54:38.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.712+0000 7fd691800640 1 --2- 172.21.15.77:0/998376614 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56125f7954e0 0x56125f7bb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:38.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.712+0000 7fd691800640 1 --2- 172.21.15.77:0/998376614 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56125f7a06c0 0x56125f7b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:38.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.712+0000 7fd68be00640 1 -- 172.21.15.77:0/998376614 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56125f7954e0 msgr2=0x56125f7bb5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:38.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.712+0000 7fd68be00640 1 --2- 172.21.15.77:0/998376614 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56125f7954e0 0x56125f7bb5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:38.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.712+0000 7fd68be00640 1 -- 172.21.15.77:0/998376614 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56125f7a06c0 msgr2=0x56125f7b4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:38.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.712+0000 7fd68be00640 1 --2- 172.21.15.77:0/998376614 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56125f7a06c0 0x56125f7b4f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:38.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.712+0000 7fd68be00640 1 -- 172.21.15.77:0/998376614 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56125f7b9110 con 0x56125f795e00 2024-09-10T09:54:38.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.712+0000 7fd691800640 1 --2- 172.21.15.77:0/998376614 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56125f7954e0 0x56125f7bb5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:54:38.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.712+0000 7fd68be00640 1 --2- 172.21.15.77:0/998376614 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56125f795e00 0x56125f7b4920 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd678000c00 tx=0x7fd678011de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cde4441f52db334a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:38.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.712+0000 7fd672a00640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-10T09:54:38.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.712+0000 7fd672a00640 1 -- 172.21.15.77:0/998376614 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd678004450 con 0x56125f795e00 2024-09-10T09:54:38.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.713+0000 7fd672a00640 1 -- 172.21.15.77:0/998376614 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fd678005040 con 0x56125f795e00 2024-09-10T09:54:38.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.713+0000 7fd696454740 1 -- 172.21.15.77:0/998376614 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd684019430 con 0x56125f795e00 2024-09-10T09:54:38.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.713+0000 7fd672a00640 1 -- 172.21.15.77:0/998376614 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd678005240 con 0x56125f795e00 2024-09-10T09:54:38.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.713+0000 7fd696454740 10 client.15027 fetch_fsmap learned FSMap version 28 2024-09-10T09:54:38.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.713+0000 7fd696454740 1 -- 172.21.15.77:0/998376614 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56125f81bf00 con 0x56125f795e00 2024-09-10T09:54:38.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.713+0000 7fd672a00640 1 -- 172.21.15.77:0/998376614 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd684019430 con 0x56125f795e00 2024-09-10T09:54:38.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.713+0000 7fd672a00640 1 -- 172.21.15.77:0/998376614 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fd67801a040 con 0x56125f795e00 2024-09-10T09:54:38.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.713+0000 7fd696454740 10 client.15027 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T09:54:38.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.713+0000 7fd696454740 10 client.15027 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T09:54:38.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.713+0000 7fd696454740 20 client.15027 populate_metadata read hostname 'smithi077' 2024-09-10T09:54:38.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.713+0000 7fd696454740 1 --2- 172.21.15.77:0/998376614 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x56125f820ad0 0x56125f81f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:38.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.713+0000 7fd696454740 4 client.15027 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T09:54:38.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.713+0000 7fd696454740 1 -- 172.21.15.77:0/998376614 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "status"}) -- 0x56125f702b30 con 0x56125f820ad0 2024-09-10T09:54:38.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.714+0000 7fd691800640 1 --2- 172.21.15.77:0/998376614 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x56125f820ad0 0x56125f81f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:38.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.714+0000 7fd691800640 1 --2- 172.21.15.77:0/998376614 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x56125f820ad0 0x56125f81f7e0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81d7f2c089a6fde0 server_cookie=ee89dd07fff8c321 in_seq=0 out_seq=0 2024-09-10T09:54:38.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.714+0000 7fd672a00640 10 client.15027 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T09:54:38.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.714+0000 7fd672a00640 1 -- 172.21.15.77:0/998376614 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x56125f702b30 con 0x56125f820ad0 2024-09-10T09:54:38.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.714+0000 7fd672a00640 10 client.15027 handle_command_reply: tid=0 2024-09-10T09:54:38.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.714+0000 7fd696454740 1 client.15027 shutdown 2024-09-10T09:54:38.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.715+0000 7fd696454740 1 -- 172.21.15.77:0/998376614 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56125f795e00 msgr2=0x56125f7b4920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:38.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.715+0000 7fd696454740 1 --2- 172.21.15.77:0/998376614 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56125f795e00 0x56125f7b4920 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd678000c00 tx=0x7fd678011de0 comp rx=0 tx=0).stop 2024-09-10T09:54:38.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.715+0000 7fd696454740 1 -- 172.21.15.77:0/998376614 shutdown_connections 2024-09-10T09:54:38.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.715+0000 7fd696454740 1 --2- 172.21.15.77:0/998376614 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x56125f820ad0 0x56125f81f7e0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:38.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.715+0000 7fd696454740 1 --2- 172.21.15.77:0/998376614 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56125f7a06c0 0x56125f7b4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:38.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.715+0000 7fd696454740 1 --2- 172.21.15.77:0/998376614 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56125f795e00 0x56125f7b4920 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:38.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.715+0000 7fd696454740 1 --2- 172.21.15.77:0/998376614 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56125f7954e0 0x56125f7bb5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:38.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.715+0000 7fd696454740 1 -- 172.21.15.77:0/998376614 >> 172.21.15.77:0/998376614 conn(0x56125f791a70 msgr2=0x56125f79f5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:38.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.715+0000 7fd672a00640 0 client.15027 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T09:54:38.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.715+0000 7fd696454740 1 -- 172.21.15.77:0/998376614 shutdown_connections 2024-09-10T09:54:38.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.715+0000 7fd696454740 1 -- 172.21.15.77:0/998376614 wait complete. 2024-09-10T09:54:38.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.715+0000 7fd696454740 20 client.15027 trim_cache size 0 max 16384 2024-09-10T09:54:38.722 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-10T09:54:38.722 INFO:teuthology.orchestra.run.smithi077.stdout: "cluster_fsid": "14a55226-6f59-11ef-bcea-c7b262605968", 2024-09-10T09:54:38.722 INFO:teuthology.orchestra.run.smithi077.stdout: "whoami": 0, 2024-09-10T09:54:38.722 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 24563, 2024-09-10T09:54:38.722 INFO:teuthology.orchestra.run.smithi077.stdout: "want_state": "up:active", 2024-09-10T09:54:38.722 INFO:teuthology.orchestra.run.smithi077.stdout: "state": "up:active", 2024-09-10T09:54:38.722 INFO:teuthology.orchestra.run.smithi077.stdout: "fs_name": "cephfs", 2024-09-10T09:54:38.722 INFO:teuthology.orchestra.run.smithi077.stdout: "rank_uptime": 134.401500129, 2024-09-10T09:54:38.722 INFO:teuthology.orchestra.run.smithi077.stdout: "mdsmap_epoch": 28, 2024-09-10T09:54:38.722 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch": 96, 2024-09-10T09:54:38.723 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch_barrier": 94, 2024-09-10T09:54:38.723 INFO:teuthology.orchestra.run.smithi077.stdout: "uptime": 148.031803704 2024-09-10T09:54:38.723 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-10T09:54:38.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.716+0000 7fd692200640 1 -- 172.21.15.77:0/3287603186 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fd6740a2c00 msgr2=0x7fd6740a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:38.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.716+0000 7fd692200640 1 --2- 172.21.15.77:0/3287603186 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fd6740a2c00 0x7fd6740a50c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd6800ef6c0 tx=0x7fd6800f2e60 comp rx=0 tx=0).stop 2024-09-10T09:54:38.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.716+0000 7fd692200640 1 -- 172.21.15.77:0/3287603186 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd68c1019c0 msgr2=0x7fd68c19a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:38.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.716+0000 7fd692200640 1 --2- 172.21.15.77:0/3287603186 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd68c1019c0 0x7fd68c19a490 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd680035040 tx=0x7fd680033b60 comp rx=0 tx=0).stop 2024-09-10T09:54:38.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.716+0000 7fd692200640 1 -- 172.21.15.77:0/3287603186 shutdown_connections 2024-09-10T09:54:38.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.716+0000 7fd692200640 1 --2- 172.21.15.77:0/3287603186 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fd6740a2c00 0x7fd6740a50c0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:38.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.716+0000 7fd692200640 1 --2- 172.21.15.77:0/3287603186 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd68c1079c0 0x7fd68c19ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:38.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.716+0000 7fd692200640 1 --2- 172.21.15.77:0/3287603186 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd68c102360 0x7fd68c19a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:38.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.716+0000 7fd692200640 1 --2- 172.21.15.77:0/3287603186 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd68c1019c0 0x7fd68c19a490 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:38.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.716+0000 7fd692200640 1 -- 172.21.15.77:0/3287603186 >> 172.21.15.77:0/3287603186 conn(0x7fd68c0fd740 msgr2=0x7fd68c0fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:38.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.717+0000 7fd692200640 1 -- 172.21.15.77:0/3287603186 shutdown_connections 2024-09-10T09:54:38.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:38.717+0000 7fd692200640 1 -- 172.21.15.77:0/3287603186 wait complete. 2024-09-10T09:54:38.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:54:38 smithi077 ceph-mon[30319]: pgmap v375: 129 pgs: 129 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-10T09:54:39.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:54:38 smithi134 ceph-mon[35674]: pgmap v375: 129 pgs: 129 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-10T09:54:39.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:54:38 smithi120 ceph-mon[34770]: pgmap v375: 129 pgs: 129 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-10T09:54:39.441 INFO:tasks.cephadm:Setup complete, yielding 2024-09-10T09:54:39.441 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-10T09:54:39.451 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi077.front.sepia.ceph.com 2024-09-10T09:54:39.451 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- bash -c 'ceph orch status' 2024-09-10T09:54:39.818 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:54:40.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:40.511+0000 7f5ef5800640 1 -- 172.21.15.77:0/2714178488 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ef0103e00 msgr2=0x7f5ef0104200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:40.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:40.511+0000 7f5ef5800640 1 --2- 172.21.15.77:0/2714178488 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ef0103e00 0x7f5ef0104200 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5ee0009900 tx=0x7f5ee002a410 comp rx=0 tx=0).stop 2024-09-10T09:54:40.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:40.511+0000 7f5ef5800640 1 -- 172.21.15.77:0/2714178488 shutdown_connections 2024-09-10T09:54:40.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:40.511+0000 7f5ef5800640 1 --2- 172.21.15.77:0/2714178488 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ef0105920 0x7f5ef010c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:40.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:40.511+0000 7f5ef5800640 1 --2- 172.21.15.77:0/2714178488 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5ef0104f60 0x7f5ef01053e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:40.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:40.511+0000 7f5ef5800640 1 --2- 172.21.15.77:0/2714178488 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ef0103e00 0x7f5ef0104200 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:40.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:40.511+0000 7f5ef5800640 1 -- 172.21.15.77:0/2714178488 >> 172.21.15.77:0/2714178488 conn(0x7f5ef00ff5b0 msgr2=0x7f5ef01019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:40.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:40.511+0000 7f5ef5800640 1 -- 172.21.15.77:0/2714178488 shutdown_connections 2024-09-10T09:54:40.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:40.512+0000 7f5ef5800640 1 -- 172.21.15.77:0/2714178488 wait complete. 2024-09-10T09:54:40.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:40.512+0000 7f5ef5800640 1 Processor -- start 2024-09-10T09:54:40.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:40.513+0000 7f5ef5800640 1 -- start start 2024-09-10T09:54:40.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:40.513+0000 7f5ef5800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ef0103e00 0x7f5ef019a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:40.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:40.513+0000 7f5ef5800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5ef0104f60 0x7f5ef019a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:40.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:40.513+0000 7f5ef5800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ef0105920 0x7f5ef01a1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:40.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:40.513+0000 7f5ef5800640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef010fb00 con 0x7f5ef0103e00 2024-09-10T09:54:40.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:40.513+0000 7f5ef5800640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef010f980 con 0x7f5ef0104f60 2024-09-10T09:54:40.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:40.513+0000 7f5ef5800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef010fc80 con 0x7f5ef0105920 2024-09-10T09:54:40.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:40.513+0000 7f5eeea00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ef0103e00 0x7f5ef019a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:40.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:40.513+0000 7f5eeea00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ef0103e00 0x7f5ef019a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34412/0 (socket says 172.21.15.77:34412) 2024-09-10T09:54:40.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:40.513+0000 7f5eeea00640 1 -- 172.21.15.77:0/733566716 learned_addr learned my addr 172.21.15.77:0/733566716 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:54:40.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:40.513+0000 7f5eef400640 1 --2- 172.21.15.77:0/733566716 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ef0105920 0x7f5ef01a1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:40.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:40.513+0000 7f5eee000640 1 --2- 172.21.15.77:0/733566716 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5ef0104f60 0x7f5ef019a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:40.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:40.514+0000 7f5eeea00640 1 -- 172.21.15.77:0/733566716 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ef0105920 msgr2=0x7f5ef01a1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:40.517 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:40.514+0000 7f5eeea00640 1 --2- 172.21.15.77:0/733566716 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ef0105920 0x7f5ef01a1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:40.517 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:40.514+0000 7f5eeea00640 1 -- 172.21.15.77:0/733566716 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5ef0104f60 msgr2=0x7f5ef019a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:40.517 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:40.514+0000 7f5eeea00640 1 --2- 172.21.15.77:0/733566716 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5ef0104f60 0x7f5ef019a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:40.517 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:40.514+0000 7f5eeea00640 1 -- 172.21.15.77:0/733566716 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ef01a1cf0 con 0x7f5ef0103e00 2024-09-10T09:54:40.517 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:40.514+0000 7f5eef400640 1 --2- 172.21.15.77:0/733566716 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ef0105920 0x7f5ef01a1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:54:40.517 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:40.514+0000 7f5eeea00640 1 --2- 172.21.15.77:0/733566716 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ef0103e00 0x7f5ef019a0c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5ee0031440 tx=0x7f5ee0032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:40.517 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:40.514+0000 7f5ee7400640 1 -- 172.21.15.77:0/733566716 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee003f030 con 0x7f5ef0103e00 2024-09-10T09:54:40.517 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:40.514+0000 7f5ef5800640 1 -- 172.21.15.77:0/733566716 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ef01a1f80 con 0x7f5ef0103e00 2024-09-10T09:54:40.517 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:40.514+0000 7f5ee7400640 1 -- 172.21.15.77:0/733566716 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ee0003ce0 con 0x7f5ef0103e00 2024-09-10T09:54:40.517 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:40.515+0000 7f5ee7400640 1 -- 172.21.15.77:0/733566716 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee0035070 con 0x7f5ef0103e00 2024-09-10T09:54:40.518 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:40.515+0000 7f5ef5800640 1 -- 172.21.15.77:0/733566716 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ef01a23d0 con 0x7f5ef0103e00 2024-09-10T09:54:40.518 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:40.516+0000 7f5ef5800640 1 -- 172.21.15.77:0/733566716 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ef010b1a0 con 0x7f5ef0103e00 2024-09-10T09:54:40.518 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:40.517+0000 7f5ee7400640 1 -- 172.21.15.77:0/733566716 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5ee0042030 con 0x7f5ef0103e00 2024-09-10T09:54:40.519 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:40.519+0000 7f5ee7400640 1 --2- 172.21.15.77:0/733566716 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f5ed40a2c30 0x7f5ed40a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:40.519 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:40.519+0000 7f5ee7400640 1 -- 172.21.15.77:0/733566716 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f5ee00ef900 con 0x7f5ef0103e00 2024-09-10T09:54:40.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:40.519+0000 7f5eee000640 1 --2- 172.21.15.77:0/733566716 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f5ed40a2c30 0x7f5ed40a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:40.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:40.519+0000 7f5eee000640 1 --2- 172.21.15.77:0/733566716 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f5ed40a2c30 0x7f5ed40a50f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5ef019b5e0 tx=0x7f5edc00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:40.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:40.523+0000 7f5ee7400640 1 -- 172.21.15.77:0/733566716 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5ee00bbdc0 con 0x7f5ef0103e00 2024-09-10T09:54:40.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:40.714+0000 7f5ef5800640 1 -- 172.21.15.77:0/733566716 --> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f5ef0062630 con 0x7f5ed40a2c30 2024-09-10T09:54:40.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:40.716+0000 7f5ee7400640 1 -- 172.21.15.77:0/733566716 <== mgr.14150 v2:172.21.15.77:6800/4261324650 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f5ef0062630 con 0x7f5ed40a2c30 2024-09-10T09:54:40.717 INFO:teuthology.orchestra.run.smithi077.stdout:Backend: cephadm 2024-09-10T09:54:40.717 INFO:teuthology.orchestra.run.smithi077.stdout:Available: Yes 2024-09-10T09:54:40.717 INFO:teuthology.orchestra.run.smithi077.stdout:Paused: No 2024-09-10T09:54:40.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:40.718+0000 7f5ef5800640 1 -- 172.21.15.77:0/733566716 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f5ed40a2c30 msgr2=0x7f5ed40a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:40.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:40.718+0000 7f5ef5800640 1 --2- 172.21.15.77:0/733566716 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f5ed40a2c30 0x7f5ed40a50f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5ef019b5e0 tx=0x7f5edc00b410 comp rx=0 tx=0).stop 2024-09-10T09:54:40.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:40.718+0000 7f5ef5800640 1 -- 172.21.15.77:0/733566716 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ef0103e00 msgr2=0x7f5ef019a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:40.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:40.718+0000 7f5ef5800640 1 --2- 172.21.15.77:0/733566716 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ef0103e00 0x7f5ef019a0c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5ee0031440 tx=0x7f5ee0032ce0 comp rx=0 tx=0).stop 2024-09-10T09:54:40.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:40.718+0000 7f5ef5800640 1 -- 172.21.15.77:0/733566716 shutdown_connections 2024-09-10T09:54:40.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:40.718+0000 7f5ef5800640 1 --2- 172.21.15.77:0/733566716 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f5ed40a2c30 0x7f5ed40a50f0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:40.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:40.718+0000 7f5ef5800640 1 --2- 172.21.15.77:0/733566716 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ef0105920 0x7f5ef01a1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:40.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:40.718+0000 7f5ef5800640 1 --2- 172.21.15.77:0/733566716 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5ef0104f60 0x7f5ef019a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:40.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:40.718+0000 7f5ef5800640 1 --2- 172.21.15.77:0/733566716 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ef0103e00 0x7f5ef019a0c0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:40.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:40.718+0000 7f5ef5800640 1 -- 172.21.15.77:0/733566716 >> 172.21.15.77:0/733566716 conn(0x7f5ef00ff5b0 msgr2=0x7f5ef0108370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:40.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:40.718+0000 7f5ef5800640 1 -- 172.21.15.77:0/733566716 shutdown_connections 2024-09-10T09:54:40.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:40.718+0000 7f5ef5800640 1 -- 172.21.15.77:0/733566716 wait complete. 2024-09-10T09:54:41.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:54:40 smithi134 ceph-mon[35674]: pgmap v376: 129 pgs: 129 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 13 op/s 2024-09-10T09:54:41.148 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:54:40 smithi077 ceph-mon[30319]: pgmap v376: 129 pgs: 129 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 13 op/s 2024-09-10T09:54:41.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:54:40 smithi120 ceph-mon[34770]: pgmap v376: 129 pgs: 129 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 13 op/s 2024-09-10T09:54:41.501 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- bash -c 'ceph orch ps' 2024-09-10T09:54:41.871 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:54:42.075 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:54:41 smithi120 ceph-mon[34770]: from='client.15033 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:54:42.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:54:41 smithi134 ceph-mon[35674]: from='client.15033 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:54:42.177 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:54:41 smithi077 ceph-mon[30319]: from='client.15033 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:54:42.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:42.437+0000 7fae75400640 1 -- 172.21.15.77:0/1944771315 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fae70104e50 msgr2=0x7fae701052d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:42.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:42.437+0000 7fae75400640 1 --2- 172.21.15.77:0/1944771315 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fae70104e50 0x7fae701052d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fae5c009900 tx=0x7fae5c02a3f0 comp rx=0 tx=0).stop 2024-09-10T09:54:42.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:42.437+0000 7fae75400640 1 -- 172.21.15.77:0/1944771315 shutdown_connections 2024-09-10T09:54:42.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:42.437+0000 7fae75400640 1 --2- 172.21.15.77:0/1944771315 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae70105810 0x7fae7010c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:42.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:42.437+0000 7fae75400640 1 --2- 172.21.15.77:0/1944771315 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fae70104e50 0x7fae701052d0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:42.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:42.437+0000 7fae75400640 1 --2- 172.21.15.77:0/1944771315 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fae70103c50 0x7fae70104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:42.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:42.437+0000 7fae75400640 1 -- 172.21.15.77:0/1944771315 >> 172.21.15.77:0/1944771315 conn(0x7fae700ff3e0 msgr2=0x7fae70101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:42.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:42.437+0000 7fae75400640 1 -- 172.21.15.77:0/1944771315 shutdown_connections 2024-09-10T09:54:42.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:42.437+0000 7fae75400640 1 -- 172.21.15.77:0/1944771315 wait complete. 2024-09-10T09:54:42.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:42.438+0000 7fae75400640 1 Processor -- start 2024-09-10T09:54:42.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:42.438+0000 7fae75400640 1 -- start start 2024-09-10T09:54:42.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:42.438+0000 7fae75400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae70103c50 0x7fae70199f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:42.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:42.438+0000 7fae75400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fae70104e50 0x7fae7019a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:42.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:42.438+0000 7fae75400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fae70105810 0x7fae701a14e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:42.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:42.438+0000 7fae75400640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fae7010f920 con 0x7fae70105810 2024-09-10T09:54:42.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:42.438+0000 7fae75400640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fae7010f7a0 con 0x7fae70103c50 2024-09-10T09:54:42.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:42.438+0000 7fae75400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fae7010faa0 con 0x7fae70104e50 2024-09-10T09:54:42.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:42.439+0000 7fae6fe00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae70103c50 0x7fae70199f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:42.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:42.439+0000 7fae6fe00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae70103c50 0x7fae70199f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:34248/0 (socket says 172.21.15.77:34248) 2024-09-10T09:54:42.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:42.439+0000 7fae6fe00640 1 -- 172.21.15.77:0/1716735986 learned_addr learned my addr 172.21.15.77:0/1716735986 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:54:42.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:42.439+0000 7fae74a00640 1 --2- 172.21.15.77:0/1716735986 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fae70105810 0x7fae701a14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:42.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:42.439+0000 7fae6f400640 1 --2- 172.21.15.77:0/1716735986 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fae70104e50 0x7fae7019a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:42.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:42.439+0000 7fae6fe00640 1 -- 172.21.15.77:0/1716735986 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fae70104e50 msgr2=0x7fae7019a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:42.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:42.439+0000 7fae6fe00640 1 --2- 172.21.15.77:0/1716735986 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fae70104e50 0x7fae7019a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:42.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:42.439+0000 7fae6fe00640 1 -- 172.21.15.77:0/1716735986 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fae70105810 msgr2=0x7fae701a14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:42.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:42.439+0000 7fae6fe00640 1 --2- 172.21.15.77:0/1716735986 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fae70105810 0x7fae701a14e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:42.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:42.439+0000 7fae6fe00640 1 -- 172.21.15.77:0/1716735986 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae701a1b50 con 0x7fae70103c50 2024-09-10T09:54:42.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:42.439+0000 7fae6fe00640 1 --2- 172.21.15.77:0/1716735986 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae70103c50 0x7fae70199f20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fae6000e9c0 tx=0x7fae6000edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:42.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:42.440+0000 7fae6cc00640 1 -- 172.21.15.77:0/1716735986 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae6000ccc0 con 0x7fae70103c50 2024-09-10T09:54:42.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:42.440+0000 7fae75400640 1 -- 172.21.15.77:0/1716735986 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae701a1e40 con 0x7fae70103c50 2024-09-10T09:54:42.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:42.440+0000 7fae6cc00640 1 -- 172.21.15.77:0/1716735986 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae600035f0 con 0x7fae70103c50 2024-09-10T09:54:42.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:42.440+0000 7fae6cc00640 1 -- 172.21.15.77:0/1716735986 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae6000ce60 con 0x7fae70103c50 2024-09-10T09:54:42.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:42.440+0000 7fae75400640 1 -- 172.21.15.77:0/1716735986 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae701a2240 con 0x7fae70103c50 2024-09-10T09:54:42.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:42.441+0000 7fae75400640 1 -- 172.21.15.77:0/1716735986 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae7010b000 con 0x7fae70103c50 2024-09-10T09:54:42.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:42.443+0000 7fae6cc00640 1 -- 172.21.15.77:0/1716735986 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fae600037a0 con 0x7fae70103c50 2024-09-10T09:54:42.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:42.444+0000 7fae6cc00640 1 --2- 172.21.15.77:0/1716735986 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fae540a2bb0 0x7fae540a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:42.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:42.445+0000 7fae6cc00640 1 -- 172.21.15.77:0/1716735986 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fae600cf7a0 con 0x7fae70103c50 2024-09-10T09:54:42.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:42.445+0000 7fae6f400640 1 --2- 172.21.15.77:0/1716735986 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fae540a2bb0 0x7fae540a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:42.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:42.445+0000 7fae6f400640 1 --2- 172.21.15.77:0/1716735986 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fae540a2bb0 0x7fae540a5070 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fae5c031440 tx=0x7fae5c0315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:42.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:42.452+0000 7fae6cc00640 1 -- 172.21.15.77:0/1716735986 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fae6009c2b0 con 0x7fae70103c50 2024-09-10T09:54:42.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:42.634+0000 7fae75400640 1 -- 172.21.15.77:0/1716735986 --> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fae70062630 con 0x7fae540a2bb0 2024-09-10T09:54:42.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:42.646+0000 7fae6cc00640 1 -- 172.21.15.77:0/1716735986 <== mgr.14150 v2:172.21.15.77:6800/4261324650 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4092 (secure 0 0 0) 0x7fae70062630 con 0x7fae540a2bb0 2024-09-10T09:54:42.647 INFO:teuthology.orchestra.run.smithi077.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T09:54:42.647 INFO:teuthology.orchestra.run.smithi077.stdout:mds.a smithi077 running (2m) 17s ago 2m 16.3M - 19.1.1-305-g1cf819b7 41d671128357 465fb4b642d3 2024-09-10T09:54:42.647 INFO:teuthology.orchestra.run.smithi077.stdout:mds.b smithi120 running (2m) 19s ago 2m 26.8M - 19.1.1-305-g1cf819b7 41d671128357 cb2fab542c1d 2024-09-10T09:54:42.647 INFO:teuthology.orchestra.run.smithi077.stdout:mds.c smithi134 running (2m) 19s ago 2m 18.7M - 19.1.1-305-g1cf819b7 41d671128357 5864e6ac715c 2024-09-10T09:54:42.647 INFO:teuthology.orchestra.run.smithi077.stdout:mds.d smithi077 running (2m) 17s ago 2m 13.1M - 19.1.1-305-g1cf819b7 41d671128357 3cf98e5ddefe 2024-09-10T09:54:42.647 INFO:teuthology.orchestra.run.smithi077.stdout:mds.e smithi120 running (2m) 19s ago 2m 19.3M - 19.1.1-305-g1cf819b7 41d671128357 4041cb8ecc89 2024-09-10T09:54:42.647 INFO:teuthology.orchestra.run.smithi077.stdout:mds.f smithi134 running (2m) 19s ago 2m 18.2M - 19.1.1-305-g1cf819b7 41d671128357 221d5b459a69 2024-09-10T09:54:42.647 INFO:teuthology.orchestra.run.smithi077.stdout:mds.g smithi077 running (2m) 17s ago 2m 16.4M - 19.1.1-305-g1cf819b7 41d671128357 289cc8276997 2024-09-10T09:54:42.647 INFO:teuthology.orchestra.run.smithi077.stdout:mds.h smithi120 running (2m) 19s ago 2m 18.9M - 19.1.1-305-g1cf819b7 41d671128357 bfdb151a8d51 2024-09-10T09:54:42.647 INFO:teuthology.orchestra.run.smithi077.stdout:mds.i smithi134 running (2m) 19s ago 2m 20.5M - 19.1.1-305-g1cf819b7 41d671128357 35a8e19d4745 2024-09-10T09:54:42.647 INFO:teuthology.orchestra.run.smithi077.stdout:mds.j smithi077 running (3m) 17s ago 2m 13.0M - 19.1.1-305-g1cf819b7 41d671128357 3646333f4cb1 2024-09-10T09:54:42.648 INFO:teuthology.orchestra.run.smithi077.stdout:mds.k smithi120 running (3m) 19s ago 3m 19.7M - 19.1.1-305-g1cf819b7 41d671128357 4a976ae355b0 2024-09-10T09:54:42.648 INFO:teuthology.orchestra.run.smithi077.stdout:mds.l smithi134 running (3m) 19s ago 3m 18.2M - 19.1.1-305-g1cf819b7 41d671128357 91a33ae01803 2024-09-10T09:54:42.648 INFO:teuthology.orchestra.run.smithi077.stdout:mgr.x smithi077 *:9283,8765 running (10m) 17s ago 10m 510M - 19.1.1-305-g1cf819b7 41d671128357 688587491f1e 2024-09-10T09:54:42.648 INFO:teuthology.orchestra.run.smithi077.stdout:mgr.y smithi120 *:8443,8765 running (8m) 19s ago 8m 442M - 19.1.1-305-g1cf819b7 41d671128357 09458c250d41 2024-09-10T09:54:42.648 INFO:teuthology.orchestra.run.smithi077.stdout:mgr.z smithi134 *:8443,8765 running (8m) 19s ago 8m 442M - 19.1.1-305-g1cf819b7 41d671128357 84161fdf796d 2024-09-10T09:54:42.648 INFO:teuthology.orchestra.run.smithi077.stdout:mon.a smithi077 running (10m) 17s ago 10m 50.5M 2048M 19.1.1-305-g1cf819b7 41d671128357 f32e55b70c78 2024-09-10T09:54:42.648 INFO:teuthology.orchestra.run.smithi077.stdout:mon.b smithi120 running (8m) 19s ago 8m 40.5M 2048M 19.1.1-305-g1cf819b7 41d671128357 fc48b4c41882 2024-09-10T09:54:42.648 INFO:teuthology.orchestra.run.smithi077.stdout:mon.c smithi134 running (9m) 19s ago 9m 42.1M 2048M 19.1.1-305-g1cf819b7 41d671128357 aa9effc066da 2024-09-10T09:54:42.648 INFO:teuthology.orchestra.run.smithi077.stdout:osd.0 smithi077 running (8m) 17s ago 8m 46.6M 4272M 19.1.1-305-g1cf819b7 41d671128357 9f906f9f1f3e 2024-09-10T09:54:42.648 INFO:teuthology.orchestra.run.smithi077.stdout:osd.1 smithi120 running (7m) 19s ago 7m 47.7M 4272M 19.1.1-305-g1cf819b7 41d671128357 9f3aa9a398fd 2024-09-10T09:54:42.648 INFO:teuthology.orchestra.run.smithi077.stdout:osd.2 smithi134 running (7m) 19s ago 7m 63.5M 4272M 19.1.1-305-g1cf819b7 41d671128357 9fea96d4a0b4 2024-09-10T09:54:42.648 INFO:teuthology.orchestra.run.smithi077.stdout:osd.3 smithi077 running (6m) 17s ago 6m 57.5M 4272M 19.1.1-305-g1cf819b7 41d671128357 1e8f5b36d293 2024-09-10T09:54:42.648 INFO:teuthology.orchestra.run.smithi077.stdout:osd.4 smithi120 running (6m) 19s ago 6m 46.0M 4272M 19.1.1-305-g1cf819b7 41d671128357 e8252a80001d 2024-09-10T09:54:42.649 INFO:teuthology.orchestra.run.smithi077.stdout:osd.5 smithi134 running (6m) 19s ago 6m 44.4M 4272M 19.1.1-305-g1cf819b7 41d671128357 e64a17ce88a2 2024-09-10T09:54:42.649 INFO:teuthology.orchestra.run.smithi077.stdout:osd.6 smithi077 running (5m) 17s ago 5m 54.5M 4272M 19.1.1-305-g1cf819b7 41d671128357 fed7996e61fc 2024-09-10T09:54:42.649 INFO:teuthology.orchestra.run.smithi077.stdout:osd.7 smithi120 running (5m) 19s ago 5m 44.8M 4272M 19.1.1-305-g1cf819b7 41d671128357 c7c192ebe569 2024-09-10T09:54:42.649 INFO:teuthology.orchestra.run.smithi077.stdout:osd.8 smithi134 running (4m) 19s ago 4m 55.6M 4272M 19.1.1-305-g1cf819b7 41d671128357 86c47bc8e576 2024-09-10T09:54:42.649 INFO:teuthology.orchestra.run.smithi077.stdout:osd.9 smithi077 running (4m) 17s ago 4m 54.9M 4272M 19.1.1-305-g1cf819b7 41d671128357 ac095d943f80 2024-09-10T09:54:42.649 INFO:teuthology.orchestra.run.smithi077.stdout:osd.10 smithi120 running (3m) 19s ago 3m 59.2M 4272M 19.1.1-305-g1cf819b7 41d671128357 fc2b0b00e001 2024-09-10T09:54:42.649 INFO:teuthology.orchestra.run.smithi077.stdout:osd.11 smithi134 running (3m) 19s ago 3m 42.8M 4272M 19.1.1-305-g1cf819b7 41d671128357 3288454b0514 2024-09-10T09:54:42.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:42.649+0000 7fae75400640 1 -- 172.21.15.77:0/1716735986 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fae540a2bb0 msgr2=0x7fae540a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:42.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:42.649+0000 7fae75400640 1 --2- 172.21.15.77:0/1716735986 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fae540a2bb0 0x7fae540a5070 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fae5c031440 tx=0x7fae5c0315b0 comp rx=0 tx=0).stop 2024-09-10T09:54:42.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:42.649+0000 7fae75400640 1 -- 172.21.15.77:0/1716735986 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae70103c50 msgr2=0x7fae70199f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:42.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:42.649+0000 7fae75400640 1 --2- 172.21.15.77:0/1716735986 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae70103c50 0x7fae70199f20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fae6000e9c0 tx=0x7fae6000edc0 comp rx=0 tx=0).stop 2024-09-10T09:54:42.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:42.649+0000 7fae75400640 1 -- 172.21.15.77:0/1716735986 shutdown_connections 2024-09-10T09:54:42.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:42.649+0000 7fae75400640 1 --2- 172.21.15.77:0/1716735986 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fae540a2bb0 0x7fae540a5070 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:42.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:42.649+0000 7fae75400640 1 --2- 172.21.15.77:0/1716735986 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fae70105810 0x7fae701a14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:42.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:42.649+0000 7fae75400640 1 --2- 172.21.15.77:0/1716735986 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fae70104e50 0x7fae7019a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:42.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:42.649+0000 7fae75400640 1 --2- 172.21.15.77:0/1716735986 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae70103c50 0x7fae70199f20 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:42.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:42.649+0000 7fae75400640 1 -- 172.21.15.77:0/1716735986 >> 172.21.15.77:0/1716735986 conn(0x7fae700ff3e0 msgr2=0x7fae701081d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:42.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:42.649+0000 7fae75400640 1 -- 172.21.15.77:0/1716735986 shutdown_connections 2024-09-10T09:54:42.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:42.649+0000 7fae75400640 1 -- 172.21.15.77:0/1716735986 wait complete. 2024-09-10T09:54:43.054 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:54:42 smithi077 ceph-mon[30319]: pgmap v377: 129 pgs: 129 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-10T09:54:43.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:54:42 smithi134 ceph-mon[35674]: pgmap v377: 129 pgs: 129 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-10T09:54:43.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:54:42 smithi120 ceph-mon[34770]: pgmap v377: 129 pgs: 129 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-10T09:54:43.395 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- bash -c 'ceph orch ls' 2024-09-10T09:54:43.759 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:54:44.061 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:54:43 smithi077 ceph-mon[30319]: from='client.25088 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:54:44.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:54:43 smithi134 ceph-mon[35674]: from='client.25088 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:54:44.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:54:43 smithi120 ceph-mon[34770]: from='client.25088 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:54:44.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:44.333+0000 7efe69400640 1 -- 172.21.15.77:0/447272037 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efe64105920 msgr2=0x7efe6410c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:44.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:44.333+0000 7efe69400640 1 --2- 172.21.15.77:0/447272037 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efe64105920 0x7efe6410c240 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7efe5c009ca0 tx=0x7efe5c02a8c0 comp rx=0 tx=0).stop 2024-09-10T09:54:44.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:44.333+0000 7efe69400640 1 -- 172.21.15.77:0/447272037 shutdown_connections 2024-09-10T09:54:44.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:44.333+0000 7efe69400640 1 --2- 172.21.15.77:0/447272037 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efe64105920 0x7efe6410c240 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:44.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:44.333+0000 7efe69400640 1 --2- 172.21.15.77:0/447272037 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe64104f60 0x7efe641053e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:44.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:44.333+0000 7efe69400640 1 --2- 172.21.15.77:0/447272037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efe64103e00 0x7efe64104200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:44.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:44.334+0000 7efe69400640 1 -- 172.21.15.77:0/447272037 >> 172.21.15.77:0/447272037 conn(0x7efe640ff5b0 msgr2=0x7efe641019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:44.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:44.334+0000 7efe69400640 1 -- 172.21.15.77:0/447272037 shutdown_connections 2024-09-10T09:54:44.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:44.334+0000 7efe69400640 1 -- 172.21.15.77:0/447272037 wait complete. 2024-09-10T09:54:44.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:44.335+0000 7efe69400640 1 Processor -- start 2024-09-10T09:54:44.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:44.335+0000 7efe69400640 1 -- start start 2024-09-10T09:54:44.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:44.335+0000 7efe69400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe64103e00 0x7efe6419a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:44.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:44.335+0000 7efe69400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efe64104f60 0x7efe6419a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:44.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:44.335+0000 7efe69400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efe64105920 0x7efe641a1660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:44.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:44.335+0000 7efe69400640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7efe6410fb00 con 0x7efe64105920 2024-09-10T09:54:44.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:44.335+0000 7efe69400640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7efe6410f980 con 0x7efe64103e00 2024-09-10T09:54:44.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:44.335+0000 7efe69400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7efe6410fc80 con 0x7efe64104f60 2024-09-10T09:54:44.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:44.336+0000 7efe62800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe64103e00 0x7efe6419a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:44.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:44.336+0000 7efe62800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe64103e00 0x7efe6419a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:49710/0 (socket says 172.21.15.77:49710) 2024-09-10T09:54:44.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:44.336+0000 7efe62800640 1 -- 172.21.15.77:0/1773318826 learned_addr learned my addr 172.21.15.77:0/1773318826 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:54:44.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:44.336+0000 7efe61e00640 1 --2- 172.21.15.77:0/1773318826 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efe64104f60 0x7efe6419a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:44.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:44.336+0000 7efe63200640 1 --2- 172.21.15.77:0/1773318826 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efe64105920 0x7efe641a1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:44.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:44.336+0000 7efe62800640 1 -- 172.21.15.77:0/1773318826 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efe64104f60 msgr2=0x7efe6419a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:44.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:44.336+0000 7efe62800640 1 --2- 172.21.15.77:0/1773318826 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efe64104f60 0x7efe6419a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:44.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:44.336+0000 7efe62800640 1 -- 172.21.15.77:0/1773318826 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efe64105920 msgr2=0x7efe641a1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:44.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:44.336+0000 7efe62800640 1 --2- 172.21.15.77:0/1773318826 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efe64105920 0x7efe641a1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:44.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:44.336+0000 7efe62800640 1 -- 172.21.15.77:0/1773318826 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe641a1cd0 con 0x7efe64103e00 2024-09-10T09:54:44.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:44.337+0000 7efe62800640 1 --2- 172.21.15.77:0/1773318826 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe64103e00 0x7efe6419a0a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7efe5400e9f0 tx=0x7efe5400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:44.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:44.337+0000 7efe5b400640 1 -- 172.21.15.77:0/1773318826 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe5400cd80 con 0x7efe64103e00 2024-09-10T09:54:44.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:44.337+0000 7efe5b400640 1 -- 172.21.15.77:0/1773318826 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe54005500 con 0x7efe64103e00 2024-09-10T09:54:44.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:44.337+0000 7efe69400640 1 -- 172.21.15.77:0/1773318826 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe641a1fc0 con 0x7efe64103e00 2024-09-10T09:54:44.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:44.337+0000 7efe5b400640 1 -- 172.21.15.77:0/1773318826 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe54010690 con 0x7efe64103e00 2024-09-10T09:54:44.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:44.337+0000 7efe69400640 1 -- 172.21.15.77:0/1773318826 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe641a2410 con 0x7efe64103e00 2024-09-10T09:54:44.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:44.338+0000 7efe69400640 1 -- 172.21.15.77:0/1773318826 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe6410b120 con 0x7efe64103e00 2024-09-10T09:54:44.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:44.340+0000 7efe5b400640 1 -- 172.21.15.77:0/1773318826 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7efe540046c0 con 0x7efe64103e00 2024-09-10T09:54:44.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:44.342+0000 7efe5b400640 1 --2- 172.21.15.77:0/1773318826 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7efe480a2bb0 0x7efe480a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:44.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:44.342+0000 7efe5b400640 1 -- 172.21.15.77:0/1773318826 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7efe540cfaf0 con 0x7efe64103e00 2024-09-10T09:54:44.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:44.342+0000 7efe61e00640 1 --2- 172.21.15.77:0/1773318826 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7efe480a2bb0 0x7efe480a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:44.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:44.342+0000 7efe61e00640 1 --2- 172.21.15.77:0/1773318826 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7efe480a2bb0 0x7efe480a5070 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7efe500095f0 tx=0x7efe50005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:44.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:44.344+0000 7efe5b400640 1 -- 172.21.15.77:0/1773318826 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efe540541b0 con 0x7efe64103e00 2024-09-10T09:54:44.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:44.535+0000 7efe69400640 1 -- 172.21.15.77:0/1773318826 --> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7efe64062630 con 0x7efe480a2bb0 2024-09-10T09:54:44.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:44.541+0000 7efe5b400640 1 -- 172.21.15.77:0/1773318826 <== mgr.14150 v2:172.21.15.77:6800/4261324650 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7efe64062630 con 0x7efe480a2bb0 2024-09-10T09:54:44.542 INFO:teuthology.orchestra.run.smithi077.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-10T09:54:44.542 INFO:teuthology.orchestra.run.smithi077.stdout:mds.all 12/12 21s ago 3m smithi077=a;smithi077=d;smithi077=g;smithi077=j;smithi120=b;smithi120=e;smithi120=h;smithi120=k;smithi134=c;smithi134=f;smithi134=i;smithi134=l;count:12 2024-09-10T09:54:44.542 INFO:teuthology.orchestra.run.smithi077.stdout:mgr 3/3 21s ago 8m smithi077=x;smithi120=y;smithi134=z;count:3 2024-09-10T09:54:44.542 INFO:teuthology.orchestra.run.smithi077.stdout:mon 3/3 21s ago 9m smithi077:172.21.15.77=a;smithi120:172.21.15.120=b;smithi134:172.21.15.134=c;count:3 2024-09-10T09:54:44.542 INFO:teuthology.orchestra.run.smithi077.stdout:osd 12 21s ago - 2024-09-10T09:54:44.544 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:44.543+0000 7efe69400640 1 -- 172.21.15.77:0/1773318826 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7efe480a2bb0 msgr2=0x7efe480a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:44.544 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:44.543+0000 7efe69400640 1 --2- 172.21.15.77:0/1773318826 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7efe480a2bb0 0x7efe480a5070 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7efe500095f0 tx=0x7efe50005f90 comp rx=0 tx=0).stop 2024-09-10T09:54:44.545 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:44.543+0000 7efe69400640 1 -- 172.21.15.77:0/1773318826 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe64103e00 msgr2=0x7efe6419a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:44.545 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:44.543+0000 7efe69400640 1 --2- 172.21.15.77:0/1773318826 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe64103e00 0x7efe6419a0a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7efe5400e9f0 tx=0x7efe5400edf0 comp rx=0 tx=0).stop 2024-09-10T09:54:44.545 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:44.543+0000 7efe69400640 1 -- 172.21.15.77:0/1773318826 shutdown_connections 2024-09-10T09:54:44.545 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:44.543+0000 7efe69400640 1 --2- 172.21.15.77:0/1773318826 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7efe480a2bb0 0x7efe480a5070 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:44.545 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:44.543+0000 7efe69400640 1 --2- 172.21.15.77:0/1773318826 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efe64105920 0x7efe641a1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:44.545 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:44.543+0000 7efe69400640 1 --2- 172.21.15.77:0/1773318826 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efe64104f60 0x7efe6419a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:44.545 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:44.543+0000 7efe69400640 1 --2- 172.21.15.77:0/1773318826 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe64103e00 0x7efe6419a0a0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:44.545 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:44.543+0000 7efe69400640 1 -- 172.21.15.77:0/1773318826 >> 172.21.15.77:0/1773318826 conn(0x7efe640ff5b0 msgr2=0x7efe641082f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:44.545 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:44.544+0000 7efe69400640 1 -- 172.21.15.77:0/1773318826 shutdown_connections 2024-09-10T09:54:44.545 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:44.544+0000 7efe69400640 1 -- 172.21.15.77:0/1773318826 wait complete. 2024-09-10T09:54:44.978 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:54:44 smithi077 ceph-mon[30319]: pgmap v378: 129 pgs: 129 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-10T09:54:45.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:54:44 smithi134 ceph-mon[35674]: pgmap v378: 129 pgs: 129 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-10T09:54:45.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:54:44 smithi120 ceph-mon[34770]: pgmap v378: 129 pgs: 129 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-10T09:54:45.530 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- bash -c 'ceph orch host ls' 2024-09-10T09:54:45.905 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:54:46.077 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:54:45 smithi077 ceph-mon[30319]: from='client.25094 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:54:46.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:54:45 smithi134 ceph-mon[35674]: from='client.25094 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:54:46.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:54:45 smithi120 ceph-mon[34770]: from='client.25094 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:54:46.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:46.502+0000 7fbad4e00640 1 -- 172.21.15.77:0/1107820013 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbad010e1e0 msgr2=0x7fbad01105d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:46.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:46.502+0000 7fbad4e00640 1 --2- 172.21.15.77:0/1107820013 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbad010e1e0 0x7fbad01105d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fbac8009ca0 tx=0x7fbac802a8c0 comp rx=0 tx=0).stop 2024-09-10T09:54:46.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:46.503+0000 7fbad4e00640 1 -- 172.21.15.77:0/1107820013 shutdown_connections 2024-09-10T09:54:46.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:46.503+0000 7fbad4e00640 1 --2- 172.21.15.77:0/1107820013 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbad010e1e0 0x7fbad01105d0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:46.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:46.503+0000 7fbad4e00640 1 --2- 172.21.15.77:0/1107820013 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbad0074c50 0x7fbad00750b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:46.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:46.503+0000 7fbad4e00640 1 --2- 172.21.15.77:0/1107820013 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbad0076220 0x7fbad0074680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:46.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:46.503+0000 7fbad4e00640 1 -- 172.21.15.77:0/1107820013 >> 172.21.15.77:0/1107820013 conn(0x7fbad00fd390 msgr2=0x7fbad00ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:46.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:46.503+0000 7fbad4e00640 1 -- 172.21.15.77:0/1107820013 shutdown_connections 2024-09-10T09:54:46.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:46.503+0000 7fbad4e00640 1 -- 172.21.15.77:0/1107820013 wait complete. 2024-09-10T09:54:46.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:46.503+0000 7fbad4e00640 1 Processor -- start 2024-09-10T09:54:46.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:46.504+0000 7fbad4e00640 1 -- start start 2024-09-10T09:54:46.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:46.504+0000 7fbad4e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbad0074c50 0x7fbad019e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:46.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:46.504+0000 7fbad4e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbad0076220 0x7fbad019ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:46.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:46.504+0000 7fbad4e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbad010e1e0 0x7fbad01a5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:46.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:46.504+0000 7fbad4e00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbad0113ea0 con 0x7fbad0074c50 2024-09-10T09:54:46.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:46.504+0000 7fbad4e00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbad0113d20 con 0x7fbad010e1e0 2024-09-10T09:54:46.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:46.504+0000 7fbad4e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbad0114020 con 0x7fbad0076220 2024-09-10T09:54:46.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:46.504+0000 7fbacde00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbad0074c50 0x7fbad019e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:46.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:46.504+0000 7fbacde00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbad0074c50 0x7fbad019e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:36464/0 (socket says 172.21.15.77:36464) 2024-09-10T09:54:46.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:46.504+0000 7fbacde00640 1 -- 172.21.15.77:0/598066452 learned_addr learned my addr 172.21.15.77:0/598066452 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:54:46.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:46.504+0000 7fbacd400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbad0076220 0x7fbad019ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:46.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:46.504+0000 7fbace800640 1 --2- 172.21.15.77:0/598066452 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbad010e1e0 0x7fbad01a5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:46.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:46.505+0000 7fbacd400640 1 -- 172.21.15.77:0/598066452 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbad010e1e0 msgr2=0x7fbad01a5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:46.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:46.505+0000 7fbacd400640 1 --2- 172.21.15.77:0/598066452 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbad010e1e0 0x7fbad01a5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:46.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:46.505+0000 7fbacd400640 1 -- 172.21.15.77:0/598066452 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbad0074c50 msgr2=0x7fbad019e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:46.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:46.505+0000 7fbacd400640 1 --2- 172.21.15.77:0/598066452 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbad0074c50 0x7fbad019e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:46.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:46.505+0000 7fbacd400640 1 -- 172.21.15.77:0/598066452 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbad01a6100 con 0x7fbad0076220 2024-09-10T09:54:46.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:46.505+0000 7fbacd400640 1 --2- 172.21.15.77:0/598066452 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbad0076220 0x7fbad019ea10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fbabc00e9f0 tx=0x7fbabc00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:46.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:46.505+0000 7fbac6a00640 1 -- 172.21.15.77:0/598066452 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbabc00ccc0 con 0x7fbad0076220 2024-09-10T09:54:46.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:46.505+0000 7fbad4e00640 1 -- 172.21.15.77:0/598066452 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbad01a63f0 con 0x7fbad0076220 2024-09-10T09:54:46.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:46.505+0000 7fbac6a00640 1 -- 172.21.15.77:0/598066452 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbabc005020 con 0x7fbad0076220 2024-09-10T09:54:46.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:46.505+0000 7fbac6a00640 1 -- 172.21.15.77:0/598066452 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbabc0049e0 con 0x7fbad0076220 2024-09-10T09:54:46.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:46.505+0000 7fbad4e00640 1 -- 172.21.15.77:0/598066452 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbad01a6930 con 0x7fbad0076220 2024-09-10T09:54:46.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:46.506+0000 7fbad4e00640 1 -- 172.21.15.77:0/598066452 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbad01043b0 con 0x7fbad0076220 2024-09-10T09:54:46.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:46.508+0000 7fbac6a00640 1 -- 172.21.15.77:0/598066452 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fbabc005560 con 0x7fbad0076220 2024-09-10T09:54:46.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:46.510+0000 7fbac6a00640 1 --2- 172.21.15.77:0/598066452 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fbab40a2bb0 0x7fbab40a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:46.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:46.510+0000 7fbac6a00640 1 -- 172.21.15.77:0/598066452 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fbabc0cf470 con 0x7fbad0076220 2024-09-10T09:54:46.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:46.510+0000 7fbacde00640 1 --2- 172.21.15.77:0/598066452 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fbab40a2bb0 0x7fbab40a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:46.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:46.511+0000 7fbacde00640 1 --2- 172.21.15.77:0/598066452 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fbab40a2bb0 0x7fbab40a5070 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fbac00058c0 tx=0x7fbac0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:46.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:46.512+0000 7fbac6a00640 1 -- 172.21.15.77:0/598066452 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbabc0051d0 con 0x7fbad0076220 2024-09-10T09:54:46.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:46.691+0000 7fbad4e00640 1 -- 172.21.15.77:0/598066452 --> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7fbad0062630 con 0x7fbab40a2bb0 2024-09-10T09:54:46.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:46.693+0000 7fbac6a00640 1 -- 172.21.15.77:0/598066452 <== mgr.14150 v2:172.21.15.77:6800/4261324650 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7fbad0062630 con 0x7fbab40a2bb0 2024-09-10T09:54:46.694 INFO:teuthology.orchestra.run.smithi077.stdout:HOST ADDR LABELS STATUS 2024-09-10T09:54:46.694 INFO:teuthology.orchestra.run.smithi077.stdout:smithi077 172.21.15.77 2024-09-10T09:54:46.694 INFO:teuthology.orchestra.run.smithi077.stdout:smithi120 172.21.15.120 2024-09-10T09:54:46.694 INFO:teuthology.orchestra.run.smithi077.stdout:smithi134 172.21.15.134 2024-09-10T09:54:46.694 INFO:teuthology.orchestra.run.smithi077.stdout:3 hosts in cluster 2024-09-10T09:54:46.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:46.695+0000 7fbad4e00640 1 -- 172.21.15.77:0/598066452 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fbab40a2bb0 msgr2=0x7fbab40a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:46.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:46.695+0000 7fbad4e00640 1 --2- 172.21.15.77:0/598066452 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fbab40a2bb0 0x7fbab40a5070 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fbac00058c0 tx=0x7fbac0005f90 comp rx=0 tx=0).stop 2024-09-10T09:54:46.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:46.695+0000 7fbad4e00640 1 -- 172.21.15.77:0/598066452 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbad0076220 msgr2=0x7fbad019ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:46.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:46.695+0000 7fbad4e00640 1 --2- 172.21.15.77:0/598066452 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbad0076220 0x7fbad019ea10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fbabc00e9f0 tx=0x7fbabc00eeb0 comp rx=0 tx=0).stop 2024-09-10T09:54:46.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:46.695+0000 7fbad4e00640 1 -- 172.21.15.77:0/598066452 shutdown_connections 2024-09-10T09:54:46.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:46.695+0000 7fbad4e00640 1 --2- 172.21.15.77:0/598066452 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fbab40a2bb0 0x7fbab40a5070 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:46.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:46.695+0000 7fbad4e00640 1 --2- 172.21.15.77:0/598066452 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbad010e1e0 0x7fbad01a5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:46.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:46.695+0000 7fbad4e00640 1 --2- 172.21.15.77:0/598066452 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbad0076220 0x7fbad019ea10 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:46.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:46.695+0000 7fbad4e00640 1 --2- 172.21.15.77:0/598066452 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbad0074c50 0x7fbad019e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:46.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:46.695+0000 7fbad4e00640 1 -- 172.21.15.77:0/598066452 >> 172.21.15.77:0/598066452 conn(0x7fbad00fd390 msgr2=0x7fbad0101580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:46.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:46.695+0000 7fbad4e00640 1 -- 172.21.15.77:0/598066452 shutdown_connections 2024-09-10T09:54:46.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:46.696+0000 7fbad4e00640 1 -- 172.21.15.77:0/598066452 wait complete. 2024-09-10T09:54:47.046 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:54:46 smithi077 ceph-mon[30319]: pgmap v379: 129 pgs: 129 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-10T09:54:47.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:54:46 smithi134 ceph-mon[35674]: pgmap v379: 129 pgs: 129 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-10T09:54:47.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:54:46 smithi120 ceph-mon[34770]: pgmap v379: 129 pgs: 129 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-10T09:54:47.440 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- bash -c 'ceph orch device ls' 2024-09-10T09:54:47.808 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:54:48.090 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:54:47 smithi077 ceph-mon[30319]: from='client.25075 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:54:48.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:54:47 smithi134 ceph-mon[35674]: from='client.25075 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:54:48.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:54:47 smithi120 ceph-mon[34770]: from='client.25075 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:54:48.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:48.373+0000 7fd9a7e00640 1 -- 172.21.15.77:0/2678977276 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd9a8105810 msgr2=0x7fd9a810c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:48.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:48.373+0000 7fd9a7e00640 1 --2- 172.21.15.77:0/2678977276 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd9a8105810 0x7fd9a810c0a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd9a0009ca0 tx=0x7fd9a002a8c0 comp rx=0 tx=0).stop 2024-09-10T09:54:48.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:48.373+0000 7fd9a7e00640 1 -- 172.21.15.77:0/2678977276 shutdown_connections 2024-09-10T09:54:48.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:48.373+0000 7fd9a7e00640 1 --2- 172.21.15.77:0/2678977276 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd9a8105810 0x7fd9a810c0a0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:48.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:48.373+0000 7fd9a7e00640 1 --2- 172.21.15.77:0/2678977276 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9a8104e50 0x7fd9a81052d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:48.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:48.373+0000 7fd9a7e00640 1 --2- 172.21.15.77:0/2678977276 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd9a8103c50 0x7fd9a8104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:48.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:48.373+0000 7fd9a7e00640 1 -- 172.21.15.77:0/2678977276 >> 172.21.15.77:0/2678977276 conn(0x7fd9a80ff3e0 msgr2=0x7fd9a8101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:48.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:48.373+0000 7fd9a7e00640 1 -- 172.21.15.77:0/2678977276 shutdown_connections 2024-09-10T09:54:48.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:48.373+0000 7fd9a7e00640 1 -- 172.21.15.77:0/2678977276 wait complete. 2024-09-10T09:54:48.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:48.374+0000 7fd9a7e00640 1 Processor -- start 2024-09-10T09:54:48.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:48.374+0000 7fd9a7e00640 1 -- start start 2024-09-10T09:54:48.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:48.374+0000 7fd9a7e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd9a8103c50 0x7fd9a8199ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:48.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:48.374+0000 7fd9a7e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9a8104e50 0x7fd9a819a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:48.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:48.375+0000 7fd9a7e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd9a8105810 0x7fd9a81a1480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:48.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:48.375+0000 7fd9a7e00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a810f8d0 con 0x7fd9a8105810 2024-09-10T09:54:48.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:48.375+0000 7fd9a7e00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a810f750 con 0x7fd9a8103c50 2024-09-10T09:54:48.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:48.375+0000 7fd9a7e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a810fa50 con 0x7fd9a8104e50 2024-09-10T09:54:48.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:48.375+0000 7fd9a6a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd9a8103c50 0x7fd9a8199ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:48.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:48.375+0000 7fd9a6a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd9a8103c50 0x7fd9a8199ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:49740/0 (socket says 172.21.15.77:49740) 2024-09-10T09:54:48.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:48.375+0000 7fd9a7400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd9a8105810 0x7fd9a81a1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:48.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:48.375+0000 7fd9a6a00640 1 -- 172.21.15.77:0/1589803359 learned_addr learned my addr 172.21.15.77:0/1589803359 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:54:48.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:48.375+0000 7fd9a6000640 1 --2- 172.21.15.77:0/1589803359 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9a8104e50 0x7fd9a819a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:48.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:48.375+0000 7fd9a6a00640 1 -- 172.21.15.77:0/1589803359 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9a8104e50 msgr2=0x7fd9a819a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:48.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:48.375+0000 7fd9a6a00640 1 --2- 172.21.15.77:0/1589803359 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9a8104e50 0x7fd9a819a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:48.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:48.375+0000 7fd9a6a00640 1 -- 172.21.15.77:0/1589803359 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd9a8105810 msgr2=0x7fd9a81a1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:48.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:48.375+0000 7fd9a6a00640 1 --2- 172.21.15.77:0/1589803359 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd9a8105810 0x7fd9a81a1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:48.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:48.375+0000 7fd9a6a00640 1 -- 172.21.15.77:0/1589803359 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9a81a1af0 con 0x7fd9a8103c50 2024-09-10T09:54:48.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:48.376+0000 7fd9a6a00640 1 --2- 172.21.15.77:0/1589803359 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd9a8103c50 0x7fd9a8199ec0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd99800e9c0 tx=0x7fd99800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:48.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:48.376+0000 7fd99f400640 1 -- 172.21.15.77:0/1589803359 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd99800cd80 con 0x7fd9a8103c50 2024-09-10T09:54:48.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:48.376+0000 7fd9a7e00640 1 -- 172.21.15.77:0/1589803359 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9a81a1de0 con 0x7fd9a8103c50 2024-09-10T09:54:48.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:48.376+0000 7fd99f400640 1 -- 172.21.15.77:0/1589803359 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9980035f0 con 0x7fd9a8103c50 2024-09-10T09:54:48.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:48.376+0000 7fd99f400640 1 -- 172.21.15.77:0/1589803359 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd998010690 con 0x7fd9a8103c50 2024-09-10T09:54:48.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:48.376+0000 7fd9a7e00640 1 -- 172.21.15.77:0/1589803359 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9a81a2240 con 0x7fd9a8103c50 2024-09-10T09:54:48.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:48.377+0000 7fd9a7e00640 1 -- 172.21.15.77:0/1589803359 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9a810b000 con 0x7fd9a8103c50 2024-09-10T09:54:48.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:48.379+0000 7fd99f400640 1 -- 172.21.15.77:0/1589803359 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd9980037a0 con 0x7fd9a8103c50 2024-09-10T09:54:48.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:48.380+0000 7fd99f400640 1 --2- 172.21.15.77:0/1589803359 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fd98c0a2bb0 0x7fd98c0a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:48.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:48.381+0000 7fd99f400640 1 -- 172.21.15.77:0/1589803359 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fd9980cf830 con 0x7fd9a8103c50 2024-09-10T09:54:48.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:48.381+0000 7fd9a6000640 1 --2- 172.21.15.77:0/1589803359 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fd98c0a2bb0 0x7fd98c0a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:48.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:48.381+0000 7fd9a6000640 1 --2- 172.21.15.77:0/1589803359 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fd98c0a2bb0 0x7fd98c0a5070 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd9a819b3e0 tx=0x7fd99400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:48.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:48.385+0000 7fd99f400640 1 -- 172.21.15.77:0/1589803359 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd99809c340 con 0x7fd9a8103c50 2024-09-10T09:54:48.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:48.566+0000 7fd9a7e00640 1 -- 172.21.15.77:0/1589803359 --> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7fd9a8062630 con 0x7fd98c0a2bb0 2024-09-10T09:54:48.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:48.567+0000 7fd99f400640 1 -- 172.21.15.77:0/1589803359 <== mgr.14150 v2:172.21.15.77:6800/4261324650 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7fd9a8062630 con 0x7fd98c0a2bb0 2024-09-10T09:54:48.568 INFO:teuthology.orchestra.run.smithi077.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-10T09:54:48.568 INFO:teuthology.orchestra.run.smithi077.stdout:smithi077 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400ZJ400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-09-10T09:54:48.568 INFO:teuthology.orchestra.run.smithi077.stdout:smithi120 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400XV400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-10T09:54:48.568 INFO:teuthology.orchestra.run.smithi077.stdout:smithi134 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT62420002400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-10T09:54:48.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:48.569+0000 7fd9a7e00640 1 -- 172.21.15.77:0/1589803359 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fd98c0a2bb0 msgr2=0x7fd98c0a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:48.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:48.569+0000 7fd9a7e00640 1 --2- 172.21.15.77:0/1589803359 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fd98c0a2bb0 0x7fd98c0a5070 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd9a819b3e0 tx=0x7fd99400b390 comp rx=0 tx=0).stop 2024-09-10T09:54:48.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:48.569+0000 7fd9a7e00640 1 -- 172.21.15.77:0/1589803359 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd9a8103c50 msgr2=0x7fd9a8199ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:48.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:48.569+0000 7fd9a7e00640 1 --2- 172.21.15.77:0/1589803359 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd9a8103c50 0x7fd9a8199ec0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd99800e9c0 tx=0x7fd99800ee80 comp rx=0 tx=0).stop 2024-09-10T09:54:48.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:48.569+0000 7fd9a7e00640 1 -- 172.21.15.77:0/1589803359 shutdown_connections 2024-09-10T09:54:48.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:48.569+0000 7fd9a7e00640 1 --2- 172.21.15.77:0/1589803359 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fd98c0a2bb0 0x7fd98c0a5070 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:48.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:48.569+0000 7fd9a7e00640 1 --2- 172.21.15.77:0/1589803359 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd9a8105810 0x7fd9a81a1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:48.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:48.569+0000 7fd9a7e00640 1 --2- 172.21.15.77:0/1589803359 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9a8104e50 0x7fd9a819a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:48.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:48.570+0000 7fd9a7e00640 1 --2- 172.21.15.77:0/1589803359 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd9a8103c50 0x7fd9a8199ec0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:48.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:48.570+0000 7fd9a7e00640 1 -- 172.21.15.77:0/1589803359 >> 172.21.15.77:0/1589803359 conn(0x7fd9a80ff3e0 msgr2=0x7fd9a81081d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:48.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:48.570+0000 7fd9a7e00640 1 -- 172.21.15.77:0/1589803359 shutdown_connections 2024-09-10T09:54:48.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:48.570+0000 7fd9a7e00640 1 -- 172.21.15.77:0/1589803359 wait complete. 2024-09-10T09:54:49.096 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:54:48 smithi077 ceph-mon[30319]: pgmap v380: 129 pgs: 129 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-10T09:54:49.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:54:48 smithi134 ceph-mon[35674]: pgmap v380: 129 pgs: 129 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-10T09:54:49.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:54:48 smithi120 ceph-mon[34770]: pgmap v380: 129 pgs: 129 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-10T09:54:49.304 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-10T09:54:49.314 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi077.front.sepia.ceph.com 2024-09-10T09:54:49.314 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- bash -c 'ceph fs dump' 2024-09-10T09:54:49.683 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:54:50.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:54:49 smithi134 ceph-mon[35674]: from='client.25106 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:54:50.137 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:54:49 smithi077 ceph-mon[30319]: from='client.25106 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:54:50.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:54:49 smithi120 ceph-mon[34770]: from='client.25106 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:54:50.248 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:50.247+0000 7f0309800640 1 -- 172.21.15.77:0/792767120 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0304101a40 msgr2=0x7f0304101e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:50.248 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:50.247+0000 7f0309800640 1 --2- 172.21.15.77:0/792767120 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0304101a40 0x7f0304101e40 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f02f4009900 tx=0x7f02f402a410 comp rx=0 tx=0).stop 2024-09-10T09:54:50.248 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:50.247+0000 7f0309800640 1 -- 172.21.15.77:0/792767120 shutdown_connections 2024-09-10T09:54:50.248 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:50.247+0000 7f0309800640 1 --2- 172.21.15.77:0/792767120 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0304103600 0x7f0304109e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:50.248 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:50.247+0000 7f0309800640 1 --2- 172.21.15.77:0/792767120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0304102c40 0x7f03041030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:50.248 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:50.247+0000 7f0309800640 1 --2- 172.21.15.77:0/792767120 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0304101a40 0x7f0304101e40 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:50.248 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:50.247+0000 7f0309800640 1 -- 172.21.15.77:0/792767120 >> 172.21.15.77:0/792767120 conn(0x7f03040fd1f0 msgr2=0x7f03040ff610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:50.248 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:50.247+0000 7f0309800640 1 -- 172.21.15.77:0/792767120 shutdown_connections 2024-09-10T09:54:50.249 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:50.247+0000 7f0309800640 1 -- 172.21.15.77:0/792767120 wait complete. 2024-09-10T09:54:50.250 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:50.248+0000 7f0309800640 1 Processor -- start 2024-09-10T09:54:50.250 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:50.248+0000 7f0309800640 1 -- start start 2024-09-10T09:54:50.250 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:50.248+0000 7f0309800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0304101a40 0x7f0304197d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:50.250 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:50.249+0000 7f0309800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0304102c40 0x7f0304198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:50.250 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:50.249+0000 7f0309800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0304103600 0x7f030419f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:50.250 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:50.249+0000 7f0309800640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f030410d6f0 con 0x7f0304101a40 2024-09-10T09:54:50.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:50.249+0000 7f0309800640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f030410d570 con 0x7f0304102c40 2024-09-10T09:54:50.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:50.249+0000 7f0309800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f030410d870 con 0x7f0304103600 2024-09-10T09:54:50.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:50.249+0000 7f0303400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0304102c40 0x7f0304198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:50.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:50.249+0000 7f0303e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0304101a40 0x7f0304197d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:50.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:50.249+0000 7f0303400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0304102c40 0x7f0304198270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:49762/0 (socket says 172.21.15.77:49762) 2024-09-10T09:54:50.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:50.249+0000 7f0303400640 1 -- 172.21.15.77:0/2656433455 learned_addr learned my addr 172.21.15.77:0/2656433455 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:54:50.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:50.249+0000 7f0308e00640 1 --2- 172.21.15.77:0/2656433455 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0304103600 0x7f030419f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:50.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:50.249+0000 7f0303400640 1 -- 172.21.15.77:0/2656433455 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0304103600 msgr2=0x7f030419f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:50.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:50.249+0000 7f0303400640 1 --2- 172.21.15.77:0/2656433455 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0304103600 0x7f030419f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:50.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:50.249+0000 7f0303400640 1 -- 172.21.15.77:0/2656433455 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0304101a40 msgr2=0x7f0304197d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:50.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:50.249+0000 7f0303400640 1 --2- 172.21.15.77:0/2656433455 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0304101a40 0x7f0304197d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:50.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:50.249+0000 7f0303400640 1 -- 172.21.15.77:0/2656433455 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f030419f950 con 0x7f0304102c40 2024-09-10T09:54:50.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:50.249+0000 7f0308e00640 1 --2- 172.21.15.77:0/2656433455 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0304103600 0x7f030419f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:54:50.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:50.250+0000 7f0303400640 1 --2- 172.21.15.77:0/2656433455 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0304102c40 0x7f0304198270 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f02f000e9f0 tx=0x7f02f000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:50.252 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:50.250+0000 7f0300c00640 1 -- 172.21.15.77:0/2656433455 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f000cd80 con 0x7f0304102c40 2024-09-10T09:54:50.252 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:50.250+0000 7f0309800640 1 -- 172.21.15.77:0/2656433455 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f030419fc40 con 0x7f0304102c40 2024-09-10T09:54:50.252 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:50.250+0000 7f0300c00640 1 -- 172.21.15.77:0/2656433455 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02f0010910 con 0x7f0304102c40 2024-09-10T09:54:50.252 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:50.250+0000 7f0300c00640 1 -- 172.21.15.77:0/2656433455 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f00188f0 con 0x7f0304102c40 2024-09-10T09:54:50.253 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:50.250+0000 7f0309800640 1 -- 172.21.15.77:0/2656433455 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03041a01a0 con 0x7f0304102c40 2024-09-10T09:54:50.253 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:50.251+0000 7f0309800640 1 -- 172.21.15.77:0/2656433455 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0304108e10 con 0x7f0304102c40 2024-09-10T09:54:50.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:50.254+0000 7f0300c00640 1 -- 172.21.15.77:0/2656433455 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f02f0010ac0 con 0x7f0304102c40 2024-09-10T09:54:50.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:50.255+0000 7f0300c00640 1 --2- 172.21.15.77:0/2656433455 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f02e80a2b80 0x7f02e80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:50.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:50.255+0000 7f0300c00640 1 -- 172.21.15.77:0/2656433455 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f02f00cfcd0 con 0x7f0304102c40 2024-09-10T09:54:50.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:50.255+0000 7f0303e00640 1 --2- 172.21.15.77:0/2656433455 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f02e80a2b80 0x7f02e80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:50.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:50.256+0000 7f0303e00640 1 --2- 172.21.15.77:0/2656433455 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f02e80a2b80 0x7f02e80a5040 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f02f402aa00 tx=0x7f02f40315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:50.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:50.257+0000 7f0300c00640 1 -- 172.21.15.77:0/2656433455 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f02f009c7e0 con 0x7f0304102c40 2024-09-10T09:54:50.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:50.466+0000 7f0309800640 1 -- 172.21.15.77:0/2656433455 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f030402d9c0 con 0x7f0304102c40 2024-09-10T09:54:50.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:50.467+0000 7f0300c00640 1 -- 172.21.15.77:0/2656433455 <== mon.2 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) ==== 76+0+3140 (secure 0 0 0) 0x7f02f009c0e0 con 0x7f0304102c40 2024-09-10T09:54:50.468 INFO:teuthology.orchestra.run.smithi077.stdout:e28 2024-09-10T09:54:50.468 INFO:teuthology.orchestra.run.smithi077.stdout:btime 2024-09-10T09:53:16:873716+0000 2024-09-10T09:54:50.468 INFO:teuthology.orchestra.run.smithi077.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T09:54:50.468 INFO:teuthology.orchestra.run.smithi077.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T09:54:50.468 INFO:teuthology.orchestra.run.smithi077.stdout:legacy client fscid: 1 2024-09-10T09:54:50.468 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:54:50.469 INFO:teuthology.orchestra.run.smithi077.stdout:Filesystem 'cephfs' (1) 2024-09-10T09:54:50.469 INFO:teuthology.orchestra.run.smithi077.stdout:fs_name cephfs 2024-09-10T09:54:50.469 INFO:teuthology.orchestra.run.smithi077.stdout:epoch 28 2024-09-10T09:54:50.469 INFO:teuthology.orchestra.run.smithi077.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-10T09:54:50.469 INFO:teuthology.orchestra.run.smithi077.stdout:created 2024-09-10T09:52:23.368252+0000 2024-09-10T09:54:50.469 INFO:teuthology.orchestra.run.smithi077.stdout:modified 2024-09-10T09:53:16.873698+0000 2024-09-10T09:54:50.469 INFO:teuthology.orchestra.run.smithi077.stdout:tableserver 0 2024-09-10T09:54:50.469 INFO:teuthology.orchestra.run.smithi077.stdout:root 0 2024-09-10T09:54:50.469 INFO:teuthology.orchestra.run.smithi077.stdout:session_timeout 300 2024-09-10T09:54:50.469 INFO:teuthology.orchestra.run.smithi077.stdout:session_autoclose 300 2024-09-10T09:54:50.469 INFO:teuthology.orchestra.run.smithi077.stdout:max_file_size 1099511627776 2024-09-10T09:54:50.469 INFO:teuthology.orchestra.run.smithi077.stdout:max_xattr_size 65536 2024-09-10T09:54:50.469 INFO:teuthology.orchestra.run.smithi077.stdout:required_client_features {} 2024-09-10T09:54:50.469 INFO:teuthology.orchestra.run.smithi077.stdout:last_failure 0 2024-09-10T09:54:50.470 INFO:teuthology.orchestra.run.smithi077.stdout:last_failure_osd_epoch 0 2024-09-10T09:54:50.470 INFO:teuthology.orchestra.run.smithi077.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T09:54:50.470 INFO:teuthology.orchestra.run.smithi077.stdout:max_mds 5 2024-09-10T09:54:50.470 INFO:teuthology.orchestra.run.smithi077.stdout:in 0,1,2,3,4 2024-09-10T09:54:50.470 INFO:teuthology.orchestra.run.smithi077.stdout:up {0=24563,1=24524,2=24520,3=24502,4=24496} 2024-09-10T09:54:50.470 INFO:teuthology.orchestra.run.smithi077.stdout:failed 2024-09-10T09:54:50.470 INFO:teuthology.orchestra.run.smithi077.stdout:damaged 2024-09-10T09:54:50.470 INFO:teuthology.orchestra.run.smithi077.stdout:stopped 2024-09-10T09:54:50.470 INFO:teuthology.orchestra.run.smithi077.stdout:data_pools [3,4] 2024-09-10T09:54:50.470 INFO:teuthology.orchestra.run.smithi077.stdout:metadata_pool 2 2024-09-10T09:54:50.470 INFO:teuthology.orchestra.run.smithi077.stdout:inline_data disabled 2024-09-10T09:54:50.470 INFO:teuthology.orchestra.run.smithi077.stdout:balancer 2024-09-10T09:54:50.470 INFO:teuthology.orchestra.run.smithi077.stdout:bal_rank_mask -1 2024-09-10T09:54:50.470 INFO:teuthology.orchestra.run.smithi077.stdout:standby_count_wanted 0 2024-09-10T09:54:50.470 INFO:teuthology.orchestra.run.smithi077.stdout:qdb_cluster leader: 24563 members: 24563,24524,24520,24502,24496 2024-09-10T09:54:50.470 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.b{0:24563} state up:active seq 5 addr [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T09:54:50.471 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.e{0:24494} state up:standby-replay seq 1 addr [v2:172.21.15.120:6834/1319892544,v1:172.21.15.120:6835/1319892544] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T09:54:50.471 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.h{1:24524} state up:active seq 20 addr [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T09:54:50.471 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.k{1:24458} state up:standby-replay seq 1 addr [v2:172.21.15.120:6832/3716307744,v1:172.21.15.120:6833/3716307744] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T09:54:50.471 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.i{2:24520} state up:active seq 20 addr [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T09:54:50.471 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.l{2:24448} state up:standby-replay seq 1 addr [v2:172.21.15.134:6832/1005054423,v1:172.21.15.134:6833/1005054423] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T09:54:50.471 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.c{3:24502} state up:active seq 21 addr [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T09:54:50.471 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.a{3:14499} state up:standby-replay seq 1 addr [v2:172.21.15.77:6840/445284194,v1:172.21.15.77:6841/445284194] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T09:54:50.471 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.f{4:24496} state up:active seq 22 addr [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T09:54:50.471 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.g{4:14487} state up:standby-replay seq 1 addr [v2:172.21.15.77:6838/4291121888,v1:172.21.15.77:6839/4291121888] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T09:54:50.471 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:54:50.471 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:54:50.471 INFO:teuthology.orchestra.run.smithi077.stdout:Standby daemons: 2024-09-10T09:54:50.471 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:54:50.471 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.j{-1:14436} state up:standby seq 1 addr [v2:172.21.15.77:6834/3159289675,v1:172.21.15.77:6835/3159289675] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T09:54:50.471 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.d{-1:14448} state up:standby seq 1 addr [v2:172.21.15.77:6836/1951962341,v1:172.21.15.77:6837/1951962341] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T09:54:50.472 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 28 2024-09-10T09:54:50.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:50.469+0000 7f0309800640 1 -- 172.21.15.77:0/2656433455 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f02e80a2b80 msgr2=0x7f02e80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:50.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:50.469+0000 7f0309800640 1 --2- 172.21.15.77:0/2656433455 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f02e80a2b80 0x7f02e80a5040 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f02f402aa00 tx=0x7f02f40315b0 comp rx=0 tx=0).stop 2024-09-10T09:54:50.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:50.469+0000 7f0309800640 1 -- 172.21.15.77:0/2656433455 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0304102c40 msgr2=0x7f0304198270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:50.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:50.469+0000 7f0309800640 1 --2- 172.21.15.77:0/2656433455 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0304102c40 0x7f0304198270 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f02f000e9f0 tx=0x7f02f000eeb0 comp rx=0 tx=0).stop 2024-09-10T09:54:50.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:50.469+0000 7f0309800640 1 -- 172.21.15.77:0/2656433455 shutdown_connections 2024-09-10T09:54:50.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:50.469+0000 7f0309800640 1 --2- 172.21.15.77:0/2656433455 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f02e80a2b80 0x7f02e80a5040 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:50.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:50.469+0000 7f0309800640 1 --2- 172.21.15.77:0/2656433455 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0304103600 0x7f030419f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:50.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:50.469+0000 7f0309800640 1 --2- 172.21.15.77:0/2656433455 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0304102c40 0x7f0304198270 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:50.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:50.469+0000 7f0309800640 1 --2- 172.21.15.77:0/2656433455 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0304101a40 0x7f0304197d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:50.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:50.469+0000 7f0309800640 1 -- 172.21.15.77:0/2656433455 >> 172.21.15.77:0/2656433455 conn(0x7f03040fd1f0 msgr2=0x7f0304105fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:50.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:50.469+0000 7f0309800640 1 -- 172.21.15.77:0/2656433455 shutdown_connections 2024-09-10T09:54:50.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:50.469+0000 7f0309800640 1 -- 172.21.15.77:0/2656433455 wait complete. 2024-09-10T09:54:51.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:54:50 smithi134 ceph-mon[35674]: pgmap v381: 129 pgs: 129 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-09-10T09:54:51.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:54:50 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/2656433455' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T09:54:51.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:54:50 smithi120 ceph-mon[34770]: pgmap v381: 129 pgs: 129 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-09-10T09:54:51.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:54:50 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/2656433455' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T09:54:51.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:54:50 smithi077 ceph-mon[30319]: pgmap v381: 129 pgs: 129 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-09-10T09:54:51.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:54:50 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/2656433455' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T09:54:51.356 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- bash -c 'ceph osd dump' 2024-09-10T09:54:51.719 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:54:52.332 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:52.328+0000 7f6b31a00640 1 -- 172.21.15.77:0/2306985241 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b2c076220 msgr2=0x7f6b2c074680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:52.333 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:52.328+0000 7f6b31a00640 1 --2- 172.21.15.77:0/2306985241 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b2c076220 0x7f6b2c074680 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c009900 tx=0x7f6b1c02a410 comp rx=0 tx=0).stop 2024-09-10T09:54:52.333 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:52.328+0000 7f6b31a00640 1 -- 172.21.15.77:0/2306985241 shutdown_connections 2024-09-10T09:54:52.333 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:52.328+0000 7f6b31a00640 1 --2- 172.21.15.77:0/2306985241 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b2c10e1e0 0x7f6b2c1105d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:52.333 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:52.328+0000 7f6b31a00640 1 --2- 172.21.15.77:0/2306985241 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6b2c074c50 0x7f6b2c0750b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:52.333 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:52.328+0000 7f6b31a00640 1 --2- 172.21.15.77:0/2306985241 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b2c076220 0x7f6b2c074680 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:52.333 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:52.328+0000 7f6b31a00640 1 -- 172.21.15.77:0/2306985241 >> 172.21.15.77:0/2306985241 conn(0x7f6b2c0fd390 msgr2=0x7f6b2c0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:52.333 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:52.328+0000 7f6b31a00640 1 -- 172.21.15.77:0/2306985241 shutdown_connections 2024-09-10T09:54:52.333 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:52.328+0000 7f6b31a00640 1 -- 172.21.15.77:0/2306985241 wait complete. 2024-09-10T09:54:52.333 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:52.330+0000 7f6b31a00640 1 Processor -- start 2024-09-10T09:54:52.333 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:52.330+0000 7f6b31a00640 1 -- start start 2024-09-10T09:54:52.333 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:52.330+0000 7f6b31a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b2c074c50 0x7f6b2c19e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:52.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:52.330+0000 7f6b31a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6b2c076220 0x7f6b2c19ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:52.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:52.330+0000 7f6b31a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b2c10e1e0 0x7f6b2c1a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:52.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:52.330+0000 7f6b31a00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6b2c113ef0 con 0x7f6b2c10e1e0 2024-09-10T09:54:52.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:52.330+0000 7f6b31a00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6b2c113d70 con 0x7f6b2c076220 2024-09-10T09:54:52.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:52.330+0000 7f6b31a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6b2c114070 con 0x7f6b2c074c50 2024-09-10T09:54:52.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:52.330+0000 7f6b2aa00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b2c074c50 0x7f6b2c19e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:52.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:52.330+0000 7f6b2b400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b2c10e1e0 0x7f6b2c1a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:52.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:52.330+0000 7f6b2aa00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b2c074c50 0x7f6b2c19e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:49204/0 (socket says 172.21.15.77:49204) 2024-09-10T09:54:52.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:52.330+0000 7f6b2aa00640 1 -- 172.21.15.77:0/4275450002 learned_addr learned my addr 172.21.15.77:0/4275450002 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:54:52.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:52.331+0000 7f6b2a000640 1 --2- 172.21.15.77:0/4275450002 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6b2c076220 0x7f6b2c19ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:52.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:52.331+0000 7f6b2aa00640 1 -- 172.21.15.77:0/4275450002 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6b2c076220 msgr2=0x7f6b2c19ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:52.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:52.331+0000 7f6b2aa00640 1 --2- 172.21.15.77:0/4275450002 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6b2c076220 0x7f6b2c19ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:52.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:52.331+0000 7f6b2aa00640 1 -- 172.21.15.77:0/4275450002 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b2c10e1e0 msgr2=0x7f6b2c1a5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:52.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:52.331+0000 7f6b2aa00640 1 --2- 172.21.15.77:0/4275450002 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b2c10e1e0 0x7f6b2c1a5b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:52.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:52.331+0000 7f6b2aa00640 1 -- 172.21.15.77:0/4275450002 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b2c1a61c0 con 0x7f6b2c074c50 2024-09-10T09:54:52.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:52.331+0000 7f6b2b400640 1 --2- 172.21.15.77:0/4275450002 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b2c10e1e0 0x7f6b2c1a5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:54:52.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:52.331+0000 7f6b2aa00640 1 --2- 172.21.15.77:0/4275450002 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b2c074c50 0x7f6b2c19e590 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c031440 tx=0x7f6b1c032ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:52.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:52.332+0000 7f6b23400640 1 -- 172.21.15.77:0/4275450002 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b1c03f030 con 0x7f6b2c074c50 2024-09-10T09:54:52.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:52.332+0000 7f6b23400640 1 -- 172.21.15.77:0/4275450002 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b1c003ce0 con 0x7f6b2c074c50 2024-09-10T09:54:52.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:52.332+0000 7f6b31a00640 1 -- 172.21.15.77:0/4275450002 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b2c1a6450 con 0x7f6b2c074c50 2024-09-10T09:54:52.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:52.332+0000 7f6b23400640 1 -- 172.21.15.77:0/4275450002 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b1c035070 con 0x7f6b2c074c50 2024-09-10T09:54:52.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:52.332+0000 7f6b31a00640 1 -- 172.21.15.77:0/4275450002 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b2c1a6930 con 0x7f6b2c074c50 2024-09-10T09:54:52.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:52.333+0000 7f6b31a00640 1 -- 172.21.15.77:0/4275450002 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b2c103b80 con 0x7f6b2c074c50 2024-09-10T09:54:52.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:52.335+0000 7f6b23400640 1 -- 172.21.15.77:0/4275450002 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6b1c042030 con 0x7f6b2c074c50 2024-09-10T09:54:52.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:52.336+0000 7f6b23400640 1 --2- 172.21.15.77:0/4275450002 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f6b100a2bb0 0x7f6b100a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:52.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:52.336+0000 7f6b23400640 1 -- 172.21.15.77:0/4275450002 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f6b1c0efd90 con 0x7f6b2c074c50 2024-09-10T09:54:52.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:52.337+0000 7f6b2a000640 1 --2- 172.21.15.77:0/4275450002 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f6b100a2bb0 0x7f6b100a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:52.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:52.337+0000 7f6b2a000640 1 --2- 172.21.15.77:0/4275450002 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f6b100a2bb0 0x7f6b100a5070 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c19fab0 tx=0x7f6b18005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:52.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:52.338+0000 7f6b23400640 1 -- 172.21.15.77:0/4275450002 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6b1c0bc250 con 0x7f6b2c074c50 2024-09-10T09:54:52.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:52.513+0000 7f6b31a00640 1 -- 172.21.15.77:0/4275450002 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f6b2c02d7c0 con 0x7f6b2c074c50 2024-09-10T09:54:52.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:52.514+0000 7f6b23400640 1 -- 172.21.15.77:0/4275450002 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v96) ==== 56+0+5735 (secure 0 0 0) 0x7f6b1c0c0550 con 0x7f6b2c074c50 2024-09-10T09:54:52.515 INFO:teuthology.orchestra.run.smithi077.stdout:epoch 96 2024-09-10T09:54:52.516 INFO:teuthology.orchestra.run.smithi077.stdout:fsid 14a55226-6f59-11ef-bcea-c7b262605968 2024-09-10T09:54:52.516 INFO:teuthology.orchestra.run.smithi077.stdout:created 2024-09-10T09:44:07.359542+0000 2024-09-10T09:54:52.516 INFO:teuthology.orchestra.run.smithi077.stdout:modified 2024-09-10T09:54:00.837074+0000 2024-09-10T09:54:52.516 INFO:teuthology.orchestra.run.smithi077.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-10T09:54:52.516 INFO:teuthology.orchestra.run.smithi077.stdout:crush_version 27 2024-09-10T09:54:52.516 INFO:teuthology.orchestra.run.smithi077.stdout:full_ratio 0.95 2024-09-10T09:54:52.516 INFO:teuthology.orchestra.run.smithi077.stdout:backfillfull_ratio 0.9 2024-09-10T09:54:52.516 INFO:teuthology.orchestra.run.smithi077.stdout:nearfull_ratio 0.85 2024-09-10T09:54:52.516 INFO:teuthology.orchestra.run.smithi077.stdout:require_min_compat_client luminous 2024-09-10T09:54:52.516 INFO:teuthology.orchestra.run.smithi077.stdout:min_compat_client luminous 2024-09-10T09:54:52.516 INFO:teuthology.orchestra.run.smithi077.stdout:require_osd_release squid 2024-09-10T09:54:52.516 INFO:teuthology.orchestra.run.smithi077.stdout:stretch_mode_enabled false 2024-09-10T09:54:52.516 INFO:teuthology.orchestra.run.smithi077.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-09-10T09:54:52.516 INFO:teuthology.orchestra.run.smithi077.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 2.25 2024-09-10T09:54:52.517 INFO:teuthology.orchestra.run.smithi077.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-09-10T09:54:52.517 INFO:teuthology.orchestra.run.smithi077.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 92 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-09-10T09:54:52.517 INFO:teuthology.orchestra.run.smithi077.stdout:max_osd 12 2024-09-10T09:54:52.517 INFO:teuthology.orchestra.run.smithi077.stdout:osd.0 up in weight 1 up_from 8 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.77:6802/1509649079,v1:172.21.15.77:6803/1509649079] [v2:172.21.15.77:6804/1509649079,v1:172.21.15.77:6805/1509649079] exists,up 8d46f634-114c-4926-afdc-f8fa37e40336 2024-09-10T09:54:52.517 INFO:teuthology.orchestra.run.smithi077.stdout:osd.1 up in weight 1 up_from 13 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.120:6800/4243920698,v1:172.21.15.120:6801/4243920698] [v2:172.21.15.120:6802/4243920698,v1:172.21.15.120:6803/4243920698] exists,up b6570764-736a-49e4-877a-1965e5812d1f 2024-09-10T09:54:52.517 INFO:teuthology.orchestra.run.smithi077.stdout:osd.2 up in weight 1 up_from 18 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.134:6800/3280466823,v1:172.21.15.134:6801/3280466823] [v2:172.21.15.134:6802/3280466823,v1:172.21.15.134:6803/3280466823] exists,up 5e83fe13-d1ee-483e-a575-e8041ec836f3 2024-09-10T09:54:52.517 INFO:teuthology.orchestra.run.smithi077.stdout:osd.3 up in weight 1 up_from 25 up_thru 95 down_at 0 last_clean_interval [0,0) [v2:172.21.15.77:6810/1629416786,v1:172.21.15.77:6811/1629416786] [v2:172.21.15.77:6812/1629416786,v1:172.21.15.77:6813/1629416786] exists,up 5affc6a7-1cae-48fe-b7db-befc2fcafecb 2024-09-10T09:54:52.517 INFO:teuthology.orchestra.run.smithi077.stdout:osd.4 up in weight 1 up_from 32 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.120:6808/3162658038,v1:172.21.15.120:6809/3162658038] [v2:172.21.15.120:6810/3162658038,v1:172.21.15.120:6811/3162658038] exists,up c826978f-40b0-43ae-adbb-5c93b32a68ff 2024-09-10T09:54:52.517 INFO:teuthology.orchestra.run.smithi077.stdout:osd.5 up in weight 1 up_from 37 up_thru 95 down_at 0 last_clean_interval [0,0) [v2:172.21.15.134:6808/4081444621,v1:172.21.15.134:6809/4081444621] [v2:172.21.15.134:6810/4081444621,v1:172.21.15.134:6811/4081444621] exists,up be51c7fb-2765-46a3-9861-c950f63de148 2024-09-10T09:54:52.517 INFO:teuthology.orchestra.run.smithi077.stdout:osd.6 up in weight 1 up_from 45 up_thru 95 down_at 0 last_clean_interval [0,0) [v2:172.21.15.77:6818/879565639,v1:172.21.15.77:6819/879565639] [v2:172.21.15.77:6820/879565639,v1:172.21.15.77:6821/879565639] exists,up 4eb403ab-473b-41cc-803d-18c908f768a9 2024-09-10T09:54:52.517 INFO:teuthology.orchestra.run.smithi077.stdout:osd.7 up in weight 1 up_from 50 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.120:6816/2688667400,v1:172.21.15.120:6817/2688667400] [v2:172.21.15.120:6818/2688667400,v1:172.21.15.120:6819/2688667400] exists,up ba982bd0-c236-456c-9889-61087d79c1aa 2024-09-10T09:54:52.517 INFO:teuthology.orchestra.run.smithi077.stdout:osd.8 up in weight 1 up_from 57 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.134:6816/2081686544,v1:172.21.15.134:6817/2081686544] [v2:172.21.15.134:6818/2081686544,v1:172.21.15.134:6819/2081686544] exists,up f2e8ca43-57c9-4933-b5f7-73f4d1b3e08b 2024-09-10T09:54:52.517 INFO:teuthology.orchestra.run.smithi077.stdout:osd.9 up in weight 1 up_from 65 up_thru 95 down_at 0 last_clean_interval [0,0) [v2:172.21.15.77:6826/1210619387,v1:172.21.15.77:6827/1210619387] [v2:172.21.15.77:6828/1210619387,v1:172.21.15.77:6829/1210619387] exists,up 9486f8b5-0614-4208-87ec-117741f6b622 2024-09-10T09:54:52.517 INFO:teuthology.orchestra.run.smithi077.stdout:osd.10 up in weight 1 up_from 70 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.120:6824/754733885,v1:172.21.15.120:6825/754733885] [v2:172.21.15.120:6826/754733885,v1:172.21.15.120:6827/754733885] exists,up 8e8192b0-1398-4d4d-9756-762b2f874d63 2024-09-10T09:54:52.518 INFO:teuthology.orchestra.run.smithi077.stdout:osd.11 up in weight 1 up_from 78 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.134:6824/1602213143,v1:172.21.15.134:6825/1602213143] [v2:172.21.15.134:6826/1602213143,v1:172.21.15.134:6827/1602213143] exists,up 85c1ccd4-5147-4995-896e-fe977f73f749 2024-09-10T09:54:52.518 INFO:teuthology.orchestra.run.smithi077.stdout:pg_upmap_items 2.2 [7,11] 2024-09-10T09:54:52.518 INFO:teuthology.orchestra.run.smithi077.stdout:pg_upmap_items 2.a [2,11] 2024-09-10T09:54:52.518 INFO:teuthology.orchestra.run.smithi077.stdout:pg_upmap_items 2.14 [2,4] 2024-09-10T09:54:52.518 INFO:teuthology.orchestra.run.smithi077.stdout:pg_upmap_items 2.1a [8,4] 2024-09-10T09:54:52.518 INFO:teuthology.orchestra.run.smithi077.stdout:pg_upmap_items 2.1f [8,1] 2024-09-10T09:54:52.518 INFO:teuthology.orchestra.run.smithi077.stdout:pg_upmap_items 3.b [3,9] 2024-09-10T09:54:52.518 INFO:teuthology.orchestra.run.smithi077.stdout:pg_upmap_items 3.1b [5,9] 2024-09-10T09:54:52.518 INFO:teuthology.orchestra.run.smithi077.stdout:pg_upmap_items 3.1d [4,7] 2024-09-10T09:54:52.518 INFO:teuthology.orchestra.run.smithi077.stdout:pg_upmap_items 3.27 [4,8] 2024-09-10T09:54:52.519 INFO:teuthology.orchestra.run.smithi077.stdout:pg_upmap_items 3.28 [5,6] 2024-09-10T09:54:52.519 INFO:teuthology.orchestra.run.smithi077.stdout:pg_upmap_items 3.34 [4,9] 2024-09-10T09:54:52.519 INFO:teuthology.orchestra.run.smithi077.stdout:pg_upmap_items 4.5 [5,8] 2024-09-10T09:54:52.519 INFO:teuthology.orchestra.run.smithi077.stdout:pg_upmap_items 4.6 [9,8] 2024-09-10T09:54:52.519 INFO:teuthology.orchestra.run.smithi077.stdout:pg_upmap_items 4.12 [9,10] 2024-09-10T09:54:52.519 INFO:teuthology.orchestra.run.smithi077.stdout:pg_upmap_items 4.1f [9,8] 2024-09-10T09:54:52.519 INFO:teuthology.orchestra.run.smithi077.stdout:blocklist 172.21.15.77:0/1697982655 expires 2024-09-11T09:44:59.575663+0000 2024-09-10T09:54:52.519 INFO:teuthology.orchestra.run.smithi077.stdout:blocklist 172.21.15.77:6801/1527070442 expires 2024-09-11T09:44:59.575663+0000 2024-09-10T09:54:52.519 INFO:teuthology.orchestra.run.smithi077.stdout:blocklist 172.21.15.77:0/3146688196 expires 2024-09-11T09:44:32.185342+0000 2024-09-10T09:54:52.519 INFO:teuthology.orchestra.run.smithi077.stdout:blocklist 172.21.15.77:0/371773177 expires 2024-09-11T09:44:59.575663+0000 2024-09-10T09:54:52.519 INFO:teuthology.orchestra.run.smithi077.stdout:blocklist 172.21.15.77:0/1925456910 expires 2024-09-11T09:44:59.575663+0000 2024-09-10T09:54:52.519 INFO:teuthology.orchestra.run.smithi077.stdout:blocklist 172.21.15.77:0/138545832 expires 2024-09-11T09:44:32.185342+0000 2024-09-10T09:54:52.519 INFO:teuthology.orchestra.run.smithi077.stdout:blocklist 172.21.15.77:6801/3235900503 expires 2024-09-11T09:44:32.185342+0000 2024-09-10T09:54:52.519 INFO:teuthology.orchestra.run.smithi077.stdout:blocklist 172.21.15.77:6800/1527070442 expires 2024-09-11T09:44:59.575663+0000 2024-09-10T09:54:52.519 INFO:teuthology.orchestra.run.smithi077.stdout:blocklist 172.21.15.77:6800/3235900503 expires 2024-09-11T09:44:32.185342+0000 2024-09-10T09:54:52.520 INFO:teuthology.orchestra.run.smithi077.stdout:blocklist 172.21.15.77:0/1647092548 expires 2024-09-11T09:44:32.185342+0000 2024-09-10T09:54:52.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:52.516+0000 7f6b31a00640 1 -- 172.21.15.77:0/4275450002 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f6b100a2bb0 msgr2=0x7f6b100a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:52.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:52.516+0000 7f6b31a00640 1 --2- 172.21.15.77:0/4275450002 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f6b100a2bb0 0x7f6b100a5070 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c19fab0 tx=0x7f6b18005f90 comp rx=0 tx=0).stop 2024-09-10T09:54:52.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:52.516+0000 7f6b31a00640 1 -- 172.21.15.77:0/4275450002 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b2c074c50 msgr2=0x7f6b2c19e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:52.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:52.516+0000 7f6b31a00640 1 --2- 172.21.15.77:0/4275450002 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b2c074c50 0x7f6b2c19e590 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c031440 tx=0x7f6b1c032ce0 comp rx=0 tx=0).stop 2024-09-10T09:54:52.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:52.517+0000 7f6b31a00640 1 -- 172.21.15.77:0/4275450002 shutdown_connections 2024-09-10T09:54:52.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:52.517+0000 7f6b31a00640 1 --2- 172.21.15.77:0/4275450002 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f6b100a2bb0 0x7f6b100a5070 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:52.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:52.517+0000 7f6b31a00640 1 --2- 172.21.15.77:0/4275450002 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b2c10e1e0 0x7f6b2c1a5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:52.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:52.517+0000 7f6b31a00640 1 --2- 172.21.15.77:0/4275450002 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6b2c076220 0x7f6b2c19ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:52.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:52.517+0000 7f6b31a00640 1 --2- 172.21.15.77:0/4275450002 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b2c074c50 0x7f6b2c19e590 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:52.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:52.517+0000 7f6b31a00640 1 -- 172.21.15.77:0/4275450002 >> 172.21.15.77:0/4275450002 conn(0x7f6b2c0fd390 msgr2=0x7f6b2c0ff780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:52.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:52.517+0000 7f6b31a00640 1 -- 172.21.15.77:0/4275450002 shutdown_connections 2024-09-10T09:54:52.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:52.517+0000 7f6b31a00640 1 -- 172.21.15.77:0/4275450002 wait complete. 2024-09-10T09:54:53.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:54:52 smithi134 ceph-mon[35674]: pgmap v382: 129 pgs: 129 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-10T09:54:53.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:54:52 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/4275450002' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-10T09:54:53.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:54:52 smithi120 ceph-mon[34770]: pgmap v382: 129 pgs: 129 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-10T09:54:53.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:54:52 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/4275450002' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-10T09:54:53.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:54:52 smithi077 ceph-mon[30319]: pgmap v382: 129 pgs: 129 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-10T09:54:53.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:54:52 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/4275450002' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-10T09:54:53.255 INFO:teuthology.run_tasks:Running task fs.ready... 2024-09-10T09:54:53.267 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:54:53.632 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:54:54.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:54.209+0000 7fcbe0600640 1 -- 172.21.15.77:0/1085858927 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcbd810e3c0 msgr2=0x7fcbd8110810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:54.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:54.209+0000 7fcbe0600640 1 --2- 172.21.15.77:0/1085858927 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcbd810e3c0 0x7fcbd8110810 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fcbd0009ca0 tx=0x7fcbd002a950 comp rx=0 tx=0).stop 2024-09-10T09:54:54.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:54.209+0000 7fcbe0600640 1 -- 172.21.15.77:0/1085858927 shutdown_connections 2024-09-10T09:54:54.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:54.209+0000 7fcbe0600640 1 --2- 172.21.15.77:0/1085858927 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcbd810e3c0 0x7fcbd8110810 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:54.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:54.209+0000 7fcbe0600640 1 --2- 172.21.15.77:0/1085858927 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcbd8074f40 0x7fcbd810de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:54.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:54.209+0000 7fcbe0600640 1 --2- 172.21.15.77:0/1085858927 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcbd8074620 0x7fcbd8074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:54.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:54.209+0000 7fcbe0600640 1 -- 172.21.15.77:0/1085858927 >> 172.21.15.77:0/1085858927 conn(0x7fcbd80fd8c0 msgr2=0x7fcbd80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:54.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:54.209+0000 7fcbe0600640 1 -- 172.21.15.77:0/1085858927 shutdown_connections 2024-09-10T09:54:54.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:54.209+0000 7fcbe0600640 1 -- 172.21.15.77:0/1085858927 wait complete. 2024-09-10T09:54:54.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:54.210+0000 7fcbe0600640 1 Processor -- start 2024-09-10T09:54:54.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:54.210+0000 7fcbe0600640 1 -- start start 2024-09-10T09:54:54.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:54.211+0000 7fcbe0600640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcbd8074620 0x7fcbd819a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:54.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:54.211+0000 7fcbe0600640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcbd8074f40 0x7fcbd819ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:54.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:54.211+0000 7fcbe0600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcbd810e3c0 0x7fcbd819eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:54.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:54.211+0000 7fcbe0600640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fcbd8114150 con 0x7fcbd8074620 2024-09-10T09:54:54.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:54.211+0000 7fcbe0600640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fcbd8113fd0 con 0x7fcbd8074f40 2024-09-10T09:54:54.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:54.211+0000 7fcbe0600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcbd81142d0 con 0x7fcbd810e3c0 2024-09-10T09:54:54.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:54.211+0000 7fcbddc00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcbd8074620 0x7fcbd819a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:54.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:54.211+0000 7fcbddc00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcbd8074620 0x7fcbd819a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:48996/0 (socket says 172.21.15.77:48996) 2024-09-10T09:54:54.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:54.211+0000 7fcbddc00640 1 -- 172.21.15.77:0/4043663222 learned_addr learned my addr 172.21.15.77:0/4043663222 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:54:54.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:54.211+0000 7fcbdd200640 1 --2- 172.21.15.77:0/4043663222 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcbd8074f40 0x7fcbd819ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:54.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:54.211+0000 7fcbde600640 1 --2- 172.21.15.77:0/4043663222 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcbd810e3c0 0x7fcbd819eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:54.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:54.212+0000 7fcbddc00640 1 -- 172.21.15.77:0/4043663222 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcbd810e3c0 msgr2=0x7fcbd819eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:54.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:54.212+0000 7fcbddc00640 1 --2- 172.21.15.77:0/4043663222 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcbd810e3c0 0x7fcbd819eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:54.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:54.212+0000 7fcbddc00640 1 -- 172.21.15.77:0/4043663222 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcbd8074f40 msgr2=0x7fcbd819ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:54.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:54.212+0000 7fcbddc00640 1 --2- 172.21.15.77:0/4043663222 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcbd8074f40 0x7fcbd819ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:54.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:54.212+0000 7fcbddc00640 1 -- 172.21.15.77:0/4043663222 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbd819f5c0 con 0x7fcbd8074620 2024-09-10T09:54:54.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:54.212+0000 7fcbddc00640 1 --2- 172.21.15.77:0/4043663222 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcbd8074620 0x7fcbd819a610 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc00d950 tx=0x7fcbcc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:54.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:54.212+0000 7fcbd6000640 1 -- 172.21.15.77:0/4043663222 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbcc017070 con 0x7fcbd8074620 2024-09-10T09:54:54.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:54.212+0000 7fcbd6000640 1 -- 172.21.15.77:0/4043663222 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbcc009c30 con 0x7fcbd8074620 2024-09-10T09:54:54.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:54.212+0000 7fcbe0600640 1 -- 172.21.15.77:0/4043663222 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbd819f8b0 con 0x7fcbd8074620 2024-09-10T09:54:54.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:54.212+0000 7fcbd6000640 1 -- 172.21.15.77:0/4043663222 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbcc00f6b0 con 0x7fcbd8074620 2024-09-10T09:54:54.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:54.212+0000 7fcbe0600640 1 -- 172.21.15.77:0/4043663222 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbd81a7190 con 0x7fcbd8074620 2024-09-10T09:54:54.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:54.213+0000 7fcbe0600640 1 -- 172.21.15.77:0/4043663222 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbd8074a00 con 0x7fcbd8074620 2024-09-10T09:54:54.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:54.215+0000 7fcbd6000640 1 -- 172.21.15.77:0/4043663222 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fcbcc0109d0 con 0x7fcbd8074620 2024-09-10T09:54:54.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:54.216+0000 7fcbd6000640 1 --2- 172.21.15.77:0/4043663222 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fcbc00a2b80 0x7fcbc00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:54.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:54.217+0000 7fcbd6000640 1 -- 172.21.15.77:0/4043663222 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fcbcc0cf070 con 0x7fcbd8074620 2024-09-10T09:54:54.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:54.217+0000 7fcbdd200640 1 --2- 172.21.15.77:0/4043663222 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fcbc00a2b80 0x7fcbc00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:54.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:54.217+0000 7fcbdd200640 1 --2- 172.21.15.77:0/4043663222 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fcbc00a2b80 0x7fcbc00a5040 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fcbd810f750 tx=0x7fcbc400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:54.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:54.221+0000 7fcbd6000640 1 -- 172.21.15.77:0/4043663222 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcbcc009de0 con 0x7fcbd8074620 2024-09-10T09:54:54.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:54.433+0000 7fcbe0600640 1 -- 172.21.15.77:0/4043663222 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcbd802d7c0 con 0x7fcbd8074620 2024-09-10T09:54:54.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:54.434+0000 7fcbd6000640 1 -- 172.21.15.77:0/4043663222 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11989 (secure 0 0 0) 0x7fcbcc09b530 con 0x7fcbd8074620 2024-09-10T09:54:54.436 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:54:54.436 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":28,"btime":"2024-09-10T09:53:16:873716+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3159289675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3159289675},{"type":"v1","addr":"172.21.15.77:6835","nonce":3159289675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1951962341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1951962341},{"type":"v1","addr":"172.21.15.77:6837","nonce":1951962341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:52:23.368252+0000","modified":"2024-09-10T09:53:16.873698+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24563,"mds_1":24524,"mds_2":24520,"mds_3":24502,"mds_4":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14487":{"gid":14487,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/4291121888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":4291121888},{"type":"v1","addr":"172.21.15.77:6839","nonce":4291121888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14499":{"gid":14499,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6841/445284194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":445284194},{"type":"v1","addr":"172.21.15.77:6841","nonce":445284194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1005054423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1005054423},{"type":"v1","addr":"172.21.15.134:6833","nonce":1005054423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6833/3716307744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":3716307744},{"type":"v1","addr":"172.21.15.120:6833","nonce":3716307744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/1319892544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1319892544},{"type":"v1","addr":"172.21.15.120:6835","nonce":1319892544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.134:6835/3293933699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3293933699},{"type":"v1","addr":"172.21.15.134:6835","nonce":3293933699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.134:6837/2461497091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2461497091},{"type":"v1","addr":"172.21.15.134:6837","nonce":2461497091}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":20,"addr":"172.21.15.134:6839/1965117680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1965117680},{"type":"v1","addr":"172.21.15.134:6839","nonce":1965117680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":20,"addr":"172.21.15.120:6837/3033280709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3033280709},{"type":"v1","addr":"172.21.15.120:6837","nonce":3033280709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.120:6839/1777710157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1777710157},{"type":"v1","addr":"172.21.15.120:6839","nonce":1777710157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24563,"qdb_cluster":[24563,24524,24520,24502,24496]},"id":1}]} 2024-09-10T09:54:54.437 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 28 2024-09-10T09:54:54.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:54.436+0000 7fcbe0600640 1 -- 172.21.15.77:0/4043663222 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fcbc00a2b80 msgr2=0x7fcbc00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:54.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:54.436+0000 7fcbe0600640 1 --2- 172.21.15.77:0/4043663222 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fcbc00a2b80 0x7fcbc00a5040 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fcbd810f750 tx=0x7fcbc400b410 comp rx=0 tx=0).stop 2024-09-10T09:54:54.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:54.436+0000 7fcbe0600640 1 -- 172.21.15.77:0/4043663222 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcbd8074620 msgr2=0x7fcbd819a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:54.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:54.436+0000 7fcbe0600640 1 --2- 172.21.15.77:0/4043663222 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcbd8074620 0x7fcbd819a610 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc00d950 tx=0x7fcbcc00de10 comp rx=0 tx=0).stop 2024-09-10T09:54:54.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:54.436+0000 7fcbe0600640 1 -- 172.21.15.77:0/4043663222 shutdown_connections 2024-09-10T09:54:54.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:54.436+0000 7fcbe0600640 1 --2- 172.21.15.77:0/4043663222 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fcbc00a2b80 0x7fcbc00a5040 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:54.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:54.436+0000 7fcbe0600640 1 --2- 172.21.15.77:0/4043663222 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcbd810e3c0 0x7fcbd819eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:54.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:54.436+0000 7fcbe0600640 1 --2- 172.21.15.77:0/4043663222 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcbd8074f40 0x7fcbd819ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:54.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:54.436+0000 7fcbe0600640 1 --2- 172.21.15.77:0/4043663222 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcbd8074620 0x7fcbd819a610 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:54.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:54.437+0000 7fcbe0600640 1 -- 172.21.15.77:0/4043663222 >> 172.21.15.77:0/4043663222 conn(0x7fcbd80fd8c0 msgr2=0x7fcbd80fe520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:54.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:54.437+0000 7fcbe0600640 1 -- 172.21.15.77:0/4043663222 shutdown_connections 2024-09-10T09:54:54.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:54.437+0000 7fcbe0600640 1 -- 172.21.15.77:0/4043663222 wait complete. 2024-09-10T09:54:55.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:54:54 smithi134 ceph-mon[35674]: pgmap v383: 129 pgs: 129 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-10T09:54:55.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:54:54 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/4043663222' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:54:55.158 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:54:54 smithi077 ceph-mon[30319]: pgmap v383: 129 pgs: 129 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-10T09:54:55.158 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:54:54 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/4043663222' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:54:55.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:54:54 smithi120 ceph-mon[34770]: pgmap v383: 129 pgs: 129 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-10T09:54:55.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:54:54 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/4043663222' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:54:55.197 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:54:55.567 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:54:56.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:56.105+0000 7fcbaf200640 1 -- 172.21.15.77:0/2281987304 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcbb0108f60 msgr2=0x7fcbb0110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:56.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:56.105+0000 7fcbaf200640 1 --2- 172.21.15.77:0/2281987304 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcbb0108f60 0x7fcbb0110ae0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fcba8009ca0 tx=0x7fcba802a8c0 comp rx=0 tx=0).stop 2024-09-10T09:54:56.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:56.105+0000 7fcbaf200640 1 -- 172.21.15.77:0/2281987304 shutdown_connections 2024-09-10T09:54:56.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:56.105+0000 7fcbaf200640 1 --2- 172.21.15.77:0/2281987304 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcbb0108f60 0x7fcbb0110ae0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:56.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:56.105+0000 7fcbaf200640 1 --2- 172.21.15.77:0/2281987304 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcbb01047e0 0x7fcbb0108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:56.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:56.105+0000 7fcbaf200640 1 --2- 172.21.15.77:0/2281987304 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcbb0103e30 0x7fcbb0104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:56.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:56.105+0000 7fcbaf200640 1 -- 172.21.15.77:0/2281987304 >> 172.21.15.77:0/2281987304 conn(0x7fcbb00ff8d0 msgr2=0x7fcbb0101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:56.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:56.105+0000 7fcbaf200640 1 -- 172.21.15.77:0/2281987304 shutdown_connections 2024-09-10T09:54:56.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:56.105+0000 7fcbaf200640 1 -- 172.21.15.77:0/2281987304 wait complete. 2024-09-10T09:54:56.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:56.106+0000 7fcbaf200640 1 Processor -- start 2024-09-10T09:54:56.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:56.106+0000 7fcbaf200640 1 -- start start 2024-09-10T09:54:56.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:56.106+0000 7fcbaf200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcbb0103e30 0x7fcbb019a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:56.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:56.106+0000 7fcbaf200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcbb01047e0 0x7fcbb019a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:56.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:56.106+0000 7fcbaf200640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcbb0108f60 0x7fcbb019ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:56.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:56.106+0000 7fcbaf200640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb0113fc0 con 0x7fcbb01047e0 2024-09-10T09:54:56.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:56.106+0000 7fcbaf200640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb0113e40 con 0x7fcbb0108f60 2024-09-10T09:54:56.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:56.107+0000 7fcbaf200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb0114140 con 0x7fcbb0103e30 2024-09-10T09:54:56.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:56.107+0000 7fcba5400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcbb01047e0 0x7fcbb019a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:56.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:56.107+0000 7fcba5400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcbb01047e0 0x7fcbb019a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49010/0 (socket says 172.21.15.77:49010) 2024-09-10T09:54:56.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:56.107+0000 7fcbade00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcbb0103e30 0x7fcbb019a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:56.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:56.107+0000 7fcba5400640 1 -- 172.21.15.77:0/1229830805 learned_addr learned my addr 172.21.15.77:0/1229830805 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:54:56.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:56.107+0000 7fcbae800640 1 --2- 172.21.15.77:0/1229830805 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcbb0108f60 0x7fcbb019ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:56.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:56.107+0000 7fcba5400640 1 -- 172.21.15.77:0/1229830805 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcbb0103e30 msgr2=0x7fcbb019a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:56.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:56.107+0000 7fcba5400640 1 --2- 172.21.15.77:0/1229830805 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcbb0103e30 0x7fcbb019a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:56.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:56.107+0000 7fcba5400640 1 -- 172.21.15.77:0/1229830805 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcbb0108f60 msgr2=0x7fcbb019ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:56.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:56.107+0000 7fcba5400640 1 --2- 172.21.15.77:0/1229830805 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcbb0108f60 0x7fcbb019ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:56.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:56.107+0000 7fcba5400640 1 -- 172.21.15.77:0/1229830805 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbb019f3d0 con 0x7fcbb01047e0 2024-09-10T09:54:56.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:56.107+0000 7fcbae800640 1 --2- 172.21.15.77:0/1229830805 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcbb0108f60 0x7fcbb019ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:54:56.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:56.108+0000 7fcba5400640 1 --2- 172.21.15.77:0/1229830805 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcbb01047e0 0x7fcbb019a960 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fcb9800e9f0 tx=0x7fcb9800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:56.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:56.108+0000 7fcba7400640 1 -- 172.21.15.77:0/1229830805 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb9800cd80 con 0x7fcbb01047e0 2024-09-10T09:54:56.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:56.108+0000 7fcba7400640 1 -- 172.21.15.77:0/1229830805 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb98005500 con 0x7fcbb01047e0 2024-09-10T09:54:56.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:56.108+0000 7fcbaf200640 1 -- 172.21.15.77:0/1229830805 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbb019f6c0 con 0x7fcbb01047e0 2024-09-10T09:54:56.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:56.108+0000 7fcba7400640 1 -- 172.21.15.77:0/1229830805 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb98010690 con 0x7fcbb01047e0 2024-09-10T09:54:56.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:56.108+0000 7fcbaf200640 1 -- 172.21.15.77:0/1229830805 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbb01a6fa0 con 0x7fcbb01047e0 2024-09-10T09:54:56.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:56.109+0000 7fcbaf200640 1 -- 172.21.15.77:0/1229830805 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbb0104210 con 0x7fcbb01047e0 2024-09-10T09:54:56.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:56.111+0000 7fcba7400640 1 -- 172.21.15.77:0/1229830805 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fcb980046c0 con 0x7fcbb01047e0 2024-09-10T09:54:56.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:56.112+0000 7fcba7400640 1 --2- 172.21.15.77:0/1229830805 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fcb940a2c00 0x7fcb940a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:56.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:56.112+0000 7fcba7400640 1 -- 172.21.15.77:0/1229830805 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fcb980cf890 con 0x7fcbb01047e0 2024-09-10T09:54:56.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:56.112+0000 7fcbade00640 1 --2- 172.21.15.77:0/1229830805 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fcb940a2c00 0x7fcb940a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:56.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:56.113+0000 7fcbade00640 1 --2- 172.21.15.77:0/1229830805 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fcb940a2c00 0x7fcb940a50c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fcba0009bd0 tx=0x7fcba0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:56.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:56.116+0000 7fcba7400640 1 -- 172.21.15.77:0/1229830805 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcb9809c3a0 con 0x7fcbb01047e0 2024-09-10T09:54:56.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:56.323+0000 7fcbaf200640 1 -- 172.21.15.77:0/1229830805 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcbb019b780 con 0x7fcbb01047e0 2024-09-10T09:54:56.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:56.324+0000 7fcba7400640 1 -- 172.21.15.77:0/1229830805 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11989 (secure 0 0 0) 0x7fcbb019b780 con 0x7fcbb01047e0 2024-09-10T09:54:56.326 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:54:56.326 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":28,"btime":"2024-09-10T09:53:16:873716+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3159289675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3159289675},{"type":"v1","addr":"172.21.15.77:6835","nonce":3159289675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1951962341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1951962341},{"type":"v1","addr":"172.21.15.77:6837","nonce":1951962341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:52:23.368252+0000","modified":"2024-09-10T09:53:16.873698+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24563,"mds_1":24524,"mds_2":24520,"mds_3":24502,"mds_4":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14487":{"gid":14487,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/4291121888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":4291121888},{"type":"v1","addr":"172.21.15.77:6839","nonce":4291121888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14499":{"gid":14499,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6841/445284194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":445284194},{"type":"v1","addr":"172.21.15.77:6841","nonce":445284194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1005054423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1005054423},{"type":"v1","addr":"172.21.15.134:6833","nonce":1005054423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6833/3716307744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":3716307744},{"type":"v1","addr":"172.21.15.120:6833","nonce":3716307744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/1319892544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1319892544},{"type":"v1","addr":"172.21.15.120:6835","nonce":1319892544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.134:6835/3293933699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3293933699},{"type":"v1","addr":"172.21.15.134:6835","nonce":3293933699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.134:6837/2461497091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2461497091},{"type":"v1","addr":"172.21.15.134:6837","nonce":2461497091}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":20,"addr":"172.21.15.134:6839/1965117680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1965117680},{"type":"v1","addr":"172.21.15.134:6839","nonce":1965117680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":20,"addr":"172.21.15.120:6837/3033280709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3033280709},{"type":"v1","addr":"172.21.15.120:6837","nonce":3033280709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.120:6839/1777710157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1777710157},{"type":"v1","addr":"172.21.15.120:6839","nonce":1777710157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24563,"qdb_cluster":[24563,24524,24520,24502,24496]},"id":1}]} 2024-09-10T09:54:56.326 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 28 2024-09-10T09:54:56.328 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:56.326+0000 7fcbaf200640 1 -- 172.21.15.77:0/1229830805 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fcb940a2c00 msgr2=0x7fcb940a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:56.329 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:56.326+0000 7fcbaf200640 1 --2- 172.21.15.77:0/1229830805 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fcb940a2c00 0x7fcb940a50c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fcba0009bd0 tx=0x7fcba0005f90 comp rx=0 tx=0).stop 2024-09-10T09:54:56.329 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:56.326+0000 7fcbaf200640 1 -- 172.21.15.77:0/1229830805 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcbb01047e0 msgr2=0x7fcbb019a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:56.329 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:56.326+0000 7fcbaf200640 1 --2- 172.21.15.77:0/1229830805 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcbb01047e0 0x7fcbb019a960 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fcb9800e9f0 tx=0x7fcb9800edf0 comp rx=0 tx=0).stop 2024-09-10T09:54:56.329 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:56.327+0000 7fcbaf200640 1 -- 172.21.15.77:0/1229830805 shutdown_connections 2024-09-10T09:54:56.329 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:56.327+0000 7fcbaf200640 1 --2- 172.21.15.77:0/1229830805 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fcb940a2c00 0x7fcb940a50c0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:56.329 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:56.327+0000 7fcbaf200640 1 --2- 172.21.15.77:0/1229830805 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcbb0108f60 0x7fcbb019ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:56.329 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:56.327+0000 7fcbaf200640 1 --2- 172.21.15.77:0/1229830805 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcbb01047e0 0x7fcbb019a960 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:56.329 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:56.327+0000 7fcbaf200640 1 --2- 172.21.15.77:0/1229830805 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcbb0103e30 0x7fcbb019a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:56.329 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:56.327+0000 7fcbaf200640 1 -- 172.21.15.77:0/1229830805 >> 172.21.15.77:0/1229830805 conn(0x7fcbb00ff8d0 msgr2=0x7fcbb0100bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:56.329 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:56.327+0000 7fcbaf200640 1 -- 172.21.15.77:0/1229830805 shutdown_connections 2024-09-10T09:54:56.329 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:56.327+0000 7fcbaf200640 1 -- 172.21.15.77:0/1229830805 wait complete. 2024-09-10T09:54:57.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:54:56 smithi134 ceph-mon[35674]: pgmap v384: 129 pgs: 129 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T09:54:57.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:54:56 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/1229830805' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:54:57.168 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:54:56 smithi077 ceph-mon[30319]: pgmap v384: 129 pgs: 129 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T09:54:57.168 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:54:56 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/1229830805' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:54:57.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:54:56 smithi120 ceph-mon[34770]: pgmap v384: 129 pgs: 129 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T09:54:57.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:54:56 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/1229830805' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:54:57.206 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T09:54:57.589 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:54:58.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:58.172+0000 7f2cb9e00640 1 -- 172.21.15.77:0/3433348449 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2cb4108f60 msgr2=0x7f2cb4110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:58.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:58.172+0000 7f2cb9e00640 1 --2- 172.21.15.77:0/3433348449 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2cb4108f60 0x7f2cb4110ae0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f2cac009ca0 tx=0x7f2cac02a8c0 comp rx=0 tx=0).stop 2024-09-10T09:54:58.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:58.172+0000 7f2cb9e00640 1 -- 172.21.15.77:0/3433348449 shutdown_connections 2024-09-10T09:54:58.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:58.172+0000 7f2cb9e00640 1 --2- 172.21.15.77:0/3433348449 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2cb4108f60 0x7f2cb4110ae0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:58.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:58.172+0000 7f2cb9e00640 1 --2- 172.21.15.77:0/3433348449 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2cb41047e0 0x7f2cb4108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:58.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:58.172+0000 7f2cb9e00640 1 --2- 172.21.15.77:0/3433348449 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2cb4103e30 0x7f2cb4104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:58.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:58.172+0000 7f2cb9e00640 1 -- 172.21.15.77:0/3433348449 >> 172.21.15.77:0/3433348449 conn(0x7f2cb40ff910 msgr2=0x7f2cb4101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:58.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:58.172+0000 7f2cb9e00640 1 -- 172.21.15.77:0/3433348449 shutdown_connections 2024-09-10T09:54:58.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:58.172+0000 7f2cb9e00640 1 -- 172.21.15.77:0/3433348449 wait complete. 2024-09-10T09:54:58.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:58.174+0000 7f2cb9e00640 1 Processor -- start 2024-09-10T09:54:58.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:58.174+0000 7f2cb9e00640 1 -- start start 2024-09-10T09:54:58.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:58.174+0000 7f2cb9e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2cb4103e30 0x7f2cb419a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:58.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:58.174+0000 7f2cb9e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2cb41047e0 0x7f2cb419a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:58.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:58.174+0000 7f2cb9e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2cb4108f60 0x7f2cb419ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:58.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:58.174+0000 7f2cb9e00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb4114010 con 0x7f2cb4103e30 2024-09-10T09:54:58.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:58.174+0000 7f2cb9e00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb4113e90 con 0x7f2cb41047e0 2024-09-10T09:54:58.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:58.174+0000 7f2cb9e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb4114190 con 0x7f2cb4108f60 2024-09-10T09:54:58.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:58.174+0000 7f2cb8a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2cb4103e30 0x7f2cb419a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:58.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:58.174+0000 7f2cb8a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2cb4103e30 0x7f2cb419a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49028/0 (socket says 172.21.15.77:49028) 2024-09-10T09:54:58.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:58.174+0000 7f2cb8a00640 1 -- 172.21.15.77:0/2115159879 learned_addr learned my addr 172.21.15.77:0/2115159879 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:54:58.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:58.174+0000 7f2cb9400640 1 --2- 172.21.15.77:0/2115159879 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2cb4108f60 0x7f2cb419ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:58.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:58.175+0000 7f2cb3e00640 1 --2- 172.21.15.77:0/2115159879 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2cb41047e0 0x7f2cb419a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:58.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:58.175+0000 7f2cb9400640 1 -- 172.21.15.77:0/2115159879 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2cb41047e0 msgr2=0x7f2cb419a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:58.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:58.175+0000 7f2cb9400640 1 --2- 172.21.15.77:0/2115159879 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2cb41047e0 0x7f2cb419a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:58.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:58.175+0000 7f2cb9400640 1 -- 172.21.15.77:0/2115159879 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2cb4103e30 msgr2=0x7f2cb419a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:58.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:58.175+0000 7f2cb9400640 1 --2- 172.21.15.77:0/2115159879 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2cb4103e30 0x7f2cb419a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:58.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:58.175+0000 7f2cb9400640 1 -- 172.21.15.77:0/2115159879 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cb419f3b0 con 0x7f2cb4108f60 2024-09-10T09:54:58.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:58.175+0000 7f2cb9400640 1 --2- 172.21.15.77:0/2115159879 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2cb4108f60 0x7f2cb419ecd0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2cac005e30 tx=0x7f2cac003f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:58.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:58.176+0000 7f2cb1600640 1 -- 172.21.15.77:0/2115159879 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cac031650 con 0x7f2cb4108f60 2024-09-10T09:54:58.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:58.176+0000 7f2cb9e00640 1 -- 172.21.15.77:0/2115159879 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cb419f640 con 0x7f2cb4108f60 2024-09-10T09:54:58.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:58.176+0000 7f2cb1600640 1 -- 172.21.15.77:0/2115159879 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cac0317f0 con 0x7f2cb4108f60 2024-09-10T09:54:58.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:58.176+0000 7f2cb1600640 1 -- 172.21.15.77:0/2115159879 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cac02f990 con 0x7f2cb4108f60 2024-09-10T09:54:58.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:58.176+0000 7f2cb9e00640 1 -- 172.21.15.77:0/2115159879 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cb41a6f80 con 0x7f2cb4108f60 2024-09-10T09:54:58.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:58.177+0000 7f2cb9e00640 1 -- 172.21.15.77:0/2115159879 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cb4105310 con 0x7f2cb4108f60 2024-09-10T09:54:58.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:58.179+0000 7f2cb1600640 1 -- 172.21.15.77:0/2115159879 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2cac004b40 con 0x7f2cb4108f60 2024-09-10T09:54:58.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:58.180+0000 7f2cb1600640 1 --2- 172.21.15.77:0/2115159879 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f2c9c0a2b80 0x7f2c9c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:58.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:58.180+0000 7f2cb1600640 1 -- 172.21.15.77:0/2115159879 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f2cac0f07d0 con 0x7f2cb4108f60 2024-09-10T09:54:58.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:58.180+0000 7f2cb8a00640 1 --2- 172.21.15.77:0/2115159879 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f2c9c0a2b80 0x7f2c9c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:58.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:58.181+0000 7f2cb8a00640 1 --2- 172.21.15.77:0/2115159879 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f2c9c0a2b80 0x7f2c9c0a5040 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2ca8009cd0 tx=0x7f2ca8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:58.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:58.182+0000 7f2cb1600640 1 -- 172.21.15.77:0/2115159879 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2cac0319a0 con 0x7f2cb4108f60 2024-09-10T09:54:58.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:58.357+0000 7f2cb9e00640 1 -- 172.21.15.77:0/2115159879 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2cb4104210 con 0x7f2cb4108f60 2024-09-10T09:54:58.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:58.359+0000 7f2cb1600640 1 -- 172.21.15.77:0/2115159879 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+24941 (secure 0 0 0) 0x7f2cb4104210 con 0x7f2cb4108f60 2024-09-10T09:54:58.361 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:54:58.361 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":96,"fsid":"14a55226-6f59-11ef-bcea-c7b262605968","created":"2024-09-10T09:44:07.359542+0000","modified":"2024-09-10T09:54:00.837074+0000","last_up_change":"2024-09-10T09:51:26.731971+0000","last_in_change":"2024-09-10T09:51:03.857873+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:47:24.174660+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T09:52:18.766738+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T09:52:21.284745+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T09:52:29.846594+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8d46f634-114c-4926-afdc-f8fa37e40336","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":1509649079},{"type":"v1","addr":"172.21.15.77:6803","nonce":1509649079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":1509649079},{"type":"v1","addr":"172.21.15.77:6805","nonce":1509649079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":1509649079},{"type":"v1","addr":"172.21.15.77:6809","nonce":1509649079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":1509649079},{"type":"v1","addr":"172.21.15.77:6807","nonce":1509649079}]},"public_addr":"172.21.15.77:6803/1509649079","cluster_addr":"172.21.15.77:6805/1509649079","heartbeat_back_addr":"172.21.15.77:6809/1509649079","heartbeat_front_addr":"172.21.15.77:6807/1509649079","state":["exists","up"]},{"osd":1,"uuid":"b6570764-736a-49e4-877a-1965e5812d1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":4243920698},{"type":"v1","addr":"172.21.15.120:6801","nonce":4243920698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":4243920698},{"type":"v1","addr":"172.21.15.120:6803","nonce":4243920698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":4243920698},{"type":"v1","addr":"172.21.15.120:6807","nonce":4243920698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":4243920698},{"type":"v1","addr":"172.21.15.120:6805","nonce":4243920698}]},"public_addr":"172.21.15.120:6801/4243920698","cluster_addr":"172.21.15.120:6803/4243920698","heartbeat_back_addr":"172.21.15.120:6807/4243920698","heartbeat_front_addr":"172.21.15.120:6805/4243920698","state":["exists","up"]},{"osd":2,"uuid":"5e83fe13-d1ee-483e-a575-e8041ec836f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":3280466823},{"type":"v1","addr":"172.21.15.134:6801","nonce":3280466823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":3280466823},{"type":"v1","addr":"172.21.15.134:6803","nonce":3280466823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":3280466823},{"type":"v1","addr":"172.21.15.134:6807","nonce":3280466823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":3280466823},{"type":"v1","addr":"172.21.15.134:6805","nonce":3280466823}]},"public_addr":"172.21.15.134:6801/3280466823","cluster_addr":"172.21.15.134:6803/3280466823","heartbeat_back_addr":"172.21.15.134:6807/3280466823","heartbeat_front_addr":"172.21.15.134:6805/3280466823","state":["exists","up"]},{"osd":3,"uuid":"5affc6a7-1cae-48fe-b7db-befc2fcafecb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":1629416786},{"type":"v1","addr":"172.21.15.77:6811","nonce":1629416786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":1629416786},{"type":"v1","addr":"172.21.15.77:6813","nonce":1629416786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":1629416786},{"type":"v1","addr":"172.21.15.77:6817","nonce":1629416786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":1629416786},{"type":"v1","addr":"172.21.15.77:6815","nonce":1629416786}]},"public_addr":"172.21.15.77:6811/1629416786","cluster_addr":"172.21.15.77:6813/1629416786","heartbeat_back_addr":"172.21.15.77:6817/1629416786","heartbeat_front_addr":"172.21.15.77:6815/1629416786","state":["exists","up"]},{"osd":4,"uuid":"c826978f-40b0-43ae-adbb-5c93b32a68ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":3162658038},{"type":"v1","addr":"172.21.15.120:6809","nonce":3162658038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":3162658038},{"type":"v1","addr":"172.21.15.120:6811","nonce":3162658038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":3162658038},{"type":"v1","addr":"172.21.15.120:6815","nonce":3162658038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":3162658038},{"type":"v1","addr":"172.21.15.120:6813","nonce":3162658038}]},"public_addr":"172.21.15.120:6809/3162658038","cluster_addr":"172.21.15.120:6811/3162658038","heartbeat_back_addr":"172.21.15.120:6815/3162658038","heartbeat_front_addr":"172.21.15.120:6813/3162658038","state":["exists","up"]},{"osd":5,"uuid":"be51c7fb-2765-46a3-9861-c950f63de148","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":4081444621},{"type":"v1","addr":"172.21.15.134:6809","nonce":4081444621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":4081444621},{"type":"v1","addr":"172.21.15.134:6811","nonce":4081444621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":4081444621},{"type":"v1","addr":"172.21.15.134:6815","nonce":4081444621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":4081444621},{"type":"v1","addr":"172.21.15.134:6813","nonce":4081444621}]},"public_addr":"172.21.15.134:6809/4081444621","cluster_addr":"172.21.15.134:6811/4081444621","heartbeat_back_addr":"172.21.15.134:6815/4081444621","heartbeat_front_addr":"172.21.15.134:6813/4081444621","state":["exists","up"]},{"osd":6,"uuid":"4eb403ab-473b-41cc-803d-18c908f768a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":879565639},{"type":"v1","addr":"172.21.15.77:6819","nonce":879565639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":879565639},{"type":"v1","addr":"172.21.15.77:6821","nonce":879565639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":879565639},{"type":"v1","addr":"172.21.15.77:6825","nonce":879565639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":879565639},{"type":"v1","addr":"172.21.15.77:6823","nonce":879565639}]},"public_addr":"172.21.15.77:6819/879565639","cluster_addr":"172.21.15.77:6821/879565639","heartbeat_back_addr":"172.21.15.77:6825/879565639","heartbeat_front_addr":"172.21.15.77:6823/879565639","state":["exists","up"]},{"osd":7,"uuid":"ba982bd0-c236-456c-9889-61087d79c1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":2688667400},{"type":"v1","addr":"172.21.15.120:6817","nonce":2688667400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":2688667400},{"type":"v1","addr":"172.21.15.120:6819","nonce":2688667400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":2688667400},{"type":"v1","addr":"172.21.15.120:6823","nonce":2688667400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2688667400},{"type":"v1","addr":"172.21.15.120:6821","nonce":2688667400}]},"public_addr":"172.21.15.120:6817/2688667400","cluster_addr":"172.21.15.120:6819/2688667400","heartbeat_back_addr":"172.21.15.120:6823/2688667400","heartbeat_front_addr":"172.21.15.120:6821/2688667400","state":["exists","up"]},{"osd":8,"uuid":"f2e8ca43-57c9-4933-b5f7-73f4d1b3e08b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2081686544},{"type":"v1","addr":"172.21.15.134:6817","nonce":2081686544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2081686544},{"type":"v1","addr":"172.21.15.134:6819","nonce":2081686544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2081686544},{"type":"v1","addr":"172.21.15.134:6823","nonce":2081686544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2081686544},{"type":"v1","addr":"172.21.15.134:6821","nonce":2081686544}]},"public_addr":"172.21.15.134:6817/2081686544","cluster_addr":"172.21.15.134:6819/2081686544","heartbeat_back_addr":"172.21.15.134:6823/2081686544","heartbeat_front_addr":"172.21.15.134:6821/2081686544","state":["exists","up"]},{"osd":9,"uuid":"9486f8b5-0614-4208-87ec-117741f6b622","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":1210619387},{"type":"v1","addr":"172.21.15.77:6827","nonce":1210619387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":1210619387},{"type":"v1","addr":"172.21.15.77:6829","nonce":1210619387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":1210619387},{"type":"v1","addr":"172.21.15.77:6833","nonce":1210619387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":1210619387},{"type":"v1","addr":"172.21.15.77:6831","nonce":1210619387}]},"public_addr":"172.21.15.77:6827/1210619387","cluster_addr":"172.21.15.77:6829/1210619387","heartbeat_back_addr":"172.21.15.77:6833/1210619387","heartbeat_front_addr":"172.21.15.77:6831/1210619387","state":["exists","up"]},{"osd":10,"uuid":"8e8192b0-1398-4d4d-9756-762b2f874d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":754733885},{"type":"v1","addr":"172.21.15.120:6825","nonce":754733885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":754733885},{"type":"v1","addr":"172.21.15.120:6827","nonce":754733885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":754733885},{"type":"v1","addr":"172.21.15.120:6831","nonce":754733885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":754733885},{"type":"v1","addr":"172.21.15.120:6829","nonce":754733885}]},"public_addr":"172.21.15.120:6825/754733885","cluster_addr":"172.21.15.120:6827/754733885","heartbeat_back_addr":"172.21.15.120:6831/754733885","heartbeat_front_addr":"172.21.15.120:6829/754733885","state":["exists","up"]},{"osd":11,"uuid":"85c1ccd4-5147-4995-896e-fe977f73f749","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1602213143},{"type":"v1","addr":"172.21.15.134:6825","nonce":1602213143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1602213143},{"type":"v1","addr":"172.21.15.134:6827","nonce":1602213143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1602213143},{"type":"v1","addr":"172.21.15.134:6831","nonce":1602213143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1602213143},{"type":"v1","addr":"172.21.15.134:6829","nonce":1602213143}]},"public_addr":"172.21.15.134:6825/1602213143","cluster_addr":"172.21.15.134:6827/1602213143","heartbeat_back_addr":"172.21.15.134:6831/1602213143","heartbeat_front_addr":"172.21.15.134:6829/1602213143","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:46:30.293451+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:46:56.554801+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:47:21.476225+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:47:49.737332+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:48:16.092181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:48:41.240815+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:49:11.683950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:49:38.859255+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:50:04.105401+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:50:33.847810+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:50:59.901797+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:51:25.241507+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.2","mappings":[{"from":7,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":7}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":5,"to":8}]},{"pgid":"4.6","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.77:0/1697982655":"2024-09-11T09:44:59.575663+0000","172.21.15.77:6801/1527070442":"2024-09-11T09:44:59.575663+0000","172.21.15.77:0/3146688196":"2024-09-11T09:44:32.185342+0000","172.21.15.77:0/371773177":"2024-09-11T09:44:59.575663+0000","172.21.15.77:0/1925456910":"2024-09-11T09:44:59.575663+0000","172.21.15.77:0/138545832":"2024-09-11T09:44:32.185342+0000","172.21.15.77:6801/3235900503":"2024-09-11T09:44:32.185342+0000","172.21.15.77:6800/1527070442":"2024-09-11T09:44:59.575663+0000","172.21.15.77:6800/3235900503":"2024-09-11T09:44:32.185342+0000","172.21.15.77:0/1647092548":"2024-09-11T09:44:32.185342+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T09:54:58.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:58.362+0000 7f2cb9e00640 1 -- 172.21.15.77:0/2115159879 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f2c9c0a2b80 msgr2=0x7f2c9c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:58.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:58.362+0000 7f2cb9e00640 1 --2- 172.21.15.77:0/2115159879 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f2c9c0a2b80 0x7f2c9c0a5040 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2ca8009cd0 tx=0x7f2ca8005f90 comp rx=0 tx=0).stop 2024-09-10T09:54:58.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:58.362+0000 7f2cb9e00640 1 -- 172.21.15.77:0/2115159879 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2cb4108f60 msgr2=0x7f2cb419ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:58.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:58.362+0000 7f2cb9e00640 1 --2- 172.21.15.77:0/2115159879 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2cb4108f60 0x7f2cb419ecd0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2cac005e30 tx=0x7f2cac003f10 comp rx=0 tx=0).stop 2024-09-10T09:54:58.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:58.362+0000 7f2cb9e00640 1 -- 172.21.15.77:0/2115159879 shutdown_connections 2024-09-10T09:54:58.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:58.362+0000 7f2cb9e00640 1 --2- 172.21.15.77:0/2115159879 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f2c9c0a2b80 0x7f2c9c0a5040 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:58.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:58.362+0000 7f2cb9e00640 1 --2- 172.21.15.77:0/2115159879 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2cb4108f60 0x7f2cb419ecd0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:58.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:58.362+0000 7f2cb9e00640 1 --2- 172.21.15.77:0/2115159879 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2cb41047e0 0x7f2cb419a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:58.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:58.362+0000 7f2cb9e00640 1 --2- 172.21.15.77:0/2115159879 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2cb4103e30 0x7f2cb419a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:58.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:58.362+0000 7f2cb9e00640 1 -- 172.21.15.77:0/2115159879 >> 172.21.15.77:0/2115159879 conn(0x7f2cb40ff910 msgr2=0x7f2cb4100bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:58.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:58.362+0000 7f2cb9e00640 1 -- 172.21.15.77:0/2115159879 shutdown_connections 2024-09-10T09:54:58.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:54:58.362+0000 7f2cb9e00640 1 -- 172.21.15.77:0/2115159879 wait complete. 2024-09-10T09:54:59.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:54:58 smithi134 ceph-mon[35674]: pgmap v385: 129 pgs: 129 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-10T09:54:59.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:54:58 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/2115159879' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T09:54:59.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:54:58 smithi120 ceph-mon[34770]: pgmap v385: 129 pgs: 129 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-10T09:54:59.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:54:58 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/2115159879' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T09:54:59.178 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:54:58 smithi077 ceph-mon[30319]: pgmap v385: 129 pgs: 129 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-10T09:54:59.178 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:54:58 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/2115159879' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T09:54:59.214 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-10T09:52:23.368252+0000', 'modified': '2024-09-10T09:53:16.873698+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24563, 'mds_1': 24524, 'mds_2': 24520, 'mds_3': 24502, 'mds_4': 24496}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14487': {'gid': 14487, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.77:6839/4291121888', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.77:6838', 'nonce': 4291121888}, {'type': 'v1', 'addr': '172.21.15.77:6839', 'nonce': 4291121888}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14499': {'gid': 14499, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.77:6841/445284194', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.77:6840', 'nonce': 445284194}, {'type': 'v1', 'addr': '172.21.15.77:6841', 'nonce': 445284194}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24448': {'gid': 24448, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.134:6833/1005054423', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6832', 'nonce': 1005054423}, {'type': 'v1', 'addr': '172.21.15.134:6833', 'nonce': 1005054423}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24458': {'gid': 24458, 'name': 'k', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.120:6833/3716307744', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6832', 'nonce': 3716307744}, {'type': 'v1', 'addr': '172.21.15.120:6833', 'nonce': 3716307744}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24494': {'gid': 24494, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.120:6835/1319892544', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6834', 'nonce': 1319892544}, {'type': 'v1', 'addr': '172.21.15.120:6835', 'nonce': 1319892544}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24496': {'gid': 24496, 'name': 'f', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 22, 'addr': '172.21.15.134:6835/3293933699', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6834', 'nonce': 3293933699}, {'type': 'v1', 'addr': '172.21.15.134:6835', 'nonce': 3293933699}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24502': {'gid': 24502, 'name': 'c', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 21, 'addr': '172.21.15.134:6837/2461497091', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6836', 'nonce': 2461497091}, {'type': 'v1', 'addr': '172.21.15.134:6837', 'nonce': 2461497091}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24520': {'gid': 24520, 'name': 'i', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 20, 'addr': '172.21.15.134:6839/1965117680', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6838', 'nonce': 1965117680}, {'type': 'v1', 'addr': '172.21.15.134:6839', 'nonce': 1965117680}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24524': {'gid': 24524, 'name': 'h', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 20, 'addr': '172.21.15.120:6837/3033280709', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6836', 'nonce': 3033280709}, {'type': 'v1', 'addr': '172.21.15.120:6837', 'nonce': 3033280709}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24563': {'gid': 24563, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.120:6839/1777710157', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6838', 'nonce': 1777710157}, {'type': 'v1', 'addr': '172.21.15.120:6839', 'nonce': 1777710157}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24563, 'qdb_cluster': [24563, 24524, 24520, 24502, 24496]} 2024-09-10T09:54:59.215 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-10T09:54:59.215 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.f status 2024-09-10T09:54:59.578 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:55:00.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.151+0000 7f8332800640 1 -- 172.21.15.77:0/3358058712 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f832c102790 msgr2=0x7f832c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:00.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.151+0000 7f8332800640 1 --2- 172.21.15.77:0/3358058712 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f832c102790 0x7f832c1067e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f8318009900 tx=0x7f831802a410 comp rx=0 tx=0).stop 2024-09-10T09:55:00.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.151+0000 7f8332800640 1 -- 172.21.15.77:0/3358058712 shutdown_connections 2024-09-10T09:55:00.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.151+0000 7f8332800640 1 --2- 172.21.15.77:0/3358058712 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f832c106f10 0x7f832c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:00.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.151+0000 7f8332800640 1 --2- 172.21.15.77:0/3358058712 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f832c102790 0x7f832c1067e0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:00.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.151+0000 7f8332800640 1 --2- 172.21.15.77:0/3358058712 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f832c101de0 0x7f832c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:00.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.151+0000 7f8332800640 1 -- 172.21.15.77:0/3358058712 >> 172.21.15.77:0/3358058712 conn(0x7f832c0fd8c0 msgr2=0x7f832c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:00.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.151+0000 7f8332800640 1 -- 172.21.15.77:0/3358058712 shutdown_connections 2024-09-10T09:55:00.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.151+0000 7f8332800640 1 -- 172.21.15.77:0/3358058712 wait complete. 2024-09-10T09:55:00.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.152+0000 7f8332800640 1 Processor -- start 2024-09-10T09:55:00.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.152+0000 7f8332800640 1 -- start start 2024-09-10T09:55:00.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.152+0000 7f8332800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f832c101de0 0x7f832c19a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:00.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.152+0000 7f8332800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f832c102790 0x7f832c19abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:00.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.152+0000 7f8332800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f832c106f10 0x7f832c19ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:00.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.152+0000 7f8332800640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f832c111f30 con 0x7f832c101de0 2024-09-10T09:55:00.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.152+0000 7f8332800640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f832c111db0 con 0x7f832c106f10 2024-09-10T09:55:00.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.152+0000 7f8332800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f832c1120b0 con 0x7f832c102790 2024-09-10T09:55:00.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.153+0000 7f832b400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f832c101de0 0x7f832c19a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:00.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.153+0000 7f832b400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f832c101de0 0x7f832c19a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49056/0 (socket says 172.21.15.77:49056) 2024-09-10T09:55:00.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.153+0000 7f832b400640 1 -- 172.21.15.77:0/1115376486 learned_addr learned my addr 172.21.15.77:0/1115376486 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:55:00.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.153+0000 7f832aa00640 1 --2- 172.21.15.77:0/1115376486 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f832c102790 0x7f832c19abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:00.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.153+0000 7f832be00640 1 --2- 172.21.15.77:0/1115376486 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f832c106f10 0x7f832c19ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:00.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.153+0000 7f832be00640 1 -- 172.21.15.77:0/1115376486 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f832c102790 msgr2=0x7f832c19abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:00.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.153+0000 7f832be00640 1 --2- 172.21.15.77:0/1115376486 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f832c102790 0x7f832c19abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:00.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.153+0000 7f832be00640 1 -- 172.21.15.77:0/1115376486 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f832c101de0 msgr2=0x7f832c19a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:00.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.153+0000 7f832be00640 1 --2- 172.21.15.77:0/1115376486 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f832c101de0 0x7f832c19a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:00.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.153+0000 7f832be00640 1 -- 172.21.15.77:0/1115376486 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f832c19f620 con 0x7f832c106f10 2024-09-10T09:55:00.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.153+0000 7f832b400640 1 --2- 172.21.15.77:0/1115376486 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f832c101de0 0x7f832c19a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:55:00.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.154+0000 7f832be00640 1 --2- 172.21.15.77:0/1115376486 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f832c106f10 0x7f832c19ef40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8324009fd0 tx=0x7f832400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:00.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.154+0000 7f8323e00640 1 -- 172.21.15.77:0/1115376486 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f832400ce20 con 0x7f832c106f10 2024-09-10T09:55:00.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.154+0000 7f8323e00640 1 -- 172.21.15.77:0/1115376486 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8324005020 con 0x7f832c106f10 2024-09-10T09:55:00.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.154+0000 7f8332800640 1 -- 172.21.15.77:0/1115376486 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f832c19f910 con 0x7f832c106f10 2024-09-10T09:55:00.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.154+0000 7f8323e00640 1 -- 172.21.15.77:0/1115376486 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8324010720 con 0x7f832c106f10 2024-09-10T09:55:00.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.154+0000 7f8332800640 1 -- 172.21.15.77:0/1115376486 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f832c1a71f0 con 0x7f832c106f10 2024-09-10T09:55:00.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.157+0000 7f8335426740 1 Processor -- start 2024-09-10T09:55:00.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.157+0000 7f8323e00640 1 -- 172.21.15.77:0/1115376486 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8324005540 con 0x7f832c106f10 2024-09-10T09:55:00.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.157+0000 7f8335426740 1 -- start start 2024-09-10T09:55:00.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.158+0000 7f8335426740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f3a227330 0x559f3a227710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:00.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.158+0000 7f8335426740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f3a227c50 0x559f3a231fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:00.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.158+0000 7f8335426740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f3a232510 0x559f3a234900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:00.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.158+0000 7f8335426740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x559f3a18d0b0 con 0x559f3a227330 2024-09-10T09:55:00.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.158+0000 7f8335426740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x559f3a134ab0 con 0x559f3a232510 2024-09-10T09:55:00.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.158+0000 7f8335426740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559f3a189070 con 0x559f3a227c50 2024-09-10T09:55:00.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.158+0000 7f832aa00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f3a227330 0x559f3a227710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:00.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.158+0000 7f832b400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f3a227c50 0x559f3a231fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:00.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.158+0000 7f832aa00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f3a227330 0x559f3a227710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49064/0 (socket says 172.21.15.77:49064) 2024-09-10T09:55:00.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.158+0000 7f832aa00640 1 -- 172.21.15.77:0/2780469475 learned_addr learned my addr 172.21.15.77:0/2780469475 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:55:00.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.158+0000 7f832aa00640 1 --2- 172.21.15.77:0/2780469475 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f3a232510 0x559f3a234900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:00.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.158+0000 7f832b400640 1 -- 172.21.15.77:0/2780469475 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f3a232510 msgr2=0x559f3a234900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:00.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.158+0000 7f832b400640 1 --2- 172.21.15.77:0/2780469475 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f3a232510 0x559f3a234900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:00.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.159+0000 7f832b400640 1 -- 172.21.15.77:0/2780469475 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f3a227330 msgr2=0x559f3a227710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:00.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.159+0000 7f832b400640 1 --2- 172.21.15.77:0/2780469475 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f3a227330 0x559f3a227710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:00.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.159+0000 7f832b400640 1 -- 172.21.15.77:0/2780469475 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f3a1414a0 con 0x559f3a227c50 2024-09-10T09:55:00.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.159+0000 7f832aa00640 1 --2- 172.21.15.77:0/2780469475 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f3a227330 0x559f3a227710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:55:00.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.159+0000 7f8323e00640 1 --2- 172.21.15.77:0/1115376486 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f83100a2b80 0x7f83100a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:00.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.159+0000 7f832b400640 1 --2- 172.21.15.77:0/2780469475 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f3a227c50 0x559f3a231fd0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f831c0114f0 tx=0x7f831c0119b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1850f6de75bb9269 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:00.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.159+0000 7f8323e00640 1 -- 172.21.15.77:0/1115376486 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f83240cfb10 con 0x7f832c106f10 2024-09-10T09:55:00.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.159+0000 7f8320c00640 1 -- 172.21.15.77:0/2780469475 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f831c01e070 con 0x559f3a227c50 2024-09-10T09:55:00.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.159+0000 7f832aa00640 1 --2- 172.21.15.77:0/1115376486 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f83100a2b80 0x7f83100a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:00.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.159+0000 7f8320c00640 1 -- 172.21.15.77:0/2780469475 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f831c0035f0 con 0x559f3a227c50 2024-09-10T09:55:00.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.159+0000 7f8320c00640 1 -- 172.21.15.77:0/2780469475 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f831c00f820 con 0x559f3a227c50 2024-09-10T09:55:00.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.160+0000 7f8335426740 1 -- 172.21.15.77:0/2780469475 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f3a227c50 msgr2=0x559f3a231fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:00.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.160+0000 7f832aa00640 1 --2- 172.21.15.77:0/1115376486 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f83100a2b80 0x7f83100a5040 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f83180096c0 tx=0x7f8318000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:00.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.160+0000 7f8335426740 1 --2- 172.21.15.77:0/2780469475 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f3a227c50 0x559f3a231fd0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f831c0114f0 tx=0x7f831c0119b0 comp rx=0 tx=0).stop 2024-09-10T09:55:00.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.160+0000 7f8335426740 1 -- 172.21.15.77:0/2780469475 shutdown_connections 2024-09-10T09:55:00.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.160+0000 7f8335426740 1 --2- 172.21.15.77:0/2780469475 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f3a232510 0x559f3a234900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:00.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.160+0000 7f8335426740 1 --2- 172.21.15.77:0/2780469475 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f3a227c50 0x559f3a231fd0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:00.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.160+0000 7f8335426740 1 --2- 172.21.15.77:0/2780469475 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f3a227330 0x559f3a227710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:00.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.160+0000 7f8335426740 1 -- 172.21.15.77:0/2780469475 >> 172.21.15.77:0/2780469475 conn(0x559f3a2238c0 msgr2=0x559f3a223ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:00.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.160+0000 7f8335426740 1 -- 172.21.15.77:0/2780469475 shutdown_connections 2024-09-10T09:55:00.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.160+0000 7f8335426740 1 -- 172.21.15.77:0/2780469475 wait complete. 2024-09-10T09:55:00.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.161+0000 7f8335426740 1 -- start start 2024-09-10T09:55:00.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.161+0000 7f8335426740 1 Processor -- start 2024-09-10T09:55:00.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.161+0000 7f8335426740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f3a227330 0x559f3a22f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:00.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.161+0000 7f8335426740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f3a227c50 0x559f3a22fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:00.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.161+0000 7f8335426740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f3a232510 0x559f3a2300a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:00.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.161+0000 7f8335426740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x559f3a234fc0 con 0x559f3a227330 2024-09-10T09:55:00.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.161+0000 7f8335426740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x559f3a115920 con 0x559f3a227c50 2024-09-10T09:55:00.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.161+0000 7f8335426740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559f3a235140 con 0x559f3a232510 2024-09-10T09:55:00.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.162+0000 7f8335426740 10 client.0 _ll_register_callbacks cb 0x559f3a1398a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:55:00.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.162+0000 7f832b400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f3a227330 0x559f3a22f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:00.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.162+0000 7f832b400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f3a227330 0x559f3a22f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49074/0 (socket says 172.21.15.77:49074) 2024-09-10T09:55:00.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.162+0000 7f832b400640 1 -- 172.21.15.77:0/2146665975 learned_addr learned my addr 172.21.15.77:0/2146665975 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:55:00.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.162+0000 7f832aa00640 1 --2- 172.21.15.77:0/2146665975 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f3a227c50 0x559f3a22fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:00.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.162+0000 7f832be00640 1 --2- 172.21.15.77:0/2146665975 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f3a232510 0x559f3a2300a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:00.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.162+0000 7f832b400640 1 -- 172.21.15.77:0/2146665975 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f3a232510 msgr2=0x559f3a2300a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:00.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.162+0000 7f832b400640 1 --2- 172.21.15.77:0/2146665975 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f3a232510 0x559f3a2300a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:00.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.162+0000 7f832b400640 1 -- 172.21.15.77:0/2146665975 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f3a227c50 msgr2=0x559f3a22fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:00.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.162+0000 7f832b400640 1 --2- 172.21.15.77:0/2146665975 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f3a227c50 0x559f3a22fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:00.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.162+0000 7f832b400640 1 -- 172.21.15.77:0/2146665975 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f39efe2b0 con 0x559f3a227330 2024-09-10T09:55:00.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.162+0000 7f832b400640 1 --2- 172.21.15.77:0/2146665975 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f3a227330 0x559f3a22f620 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f831c0097c0 tx=0x7f831c005280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2407c793656091bf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:00.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.162+0000 7f8315600640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-10T09:55:00.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.163+0000 7f8315600640 1 -- 172.21.15.77:0/2146665975 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f831c01e040 con 0x559f3a227330 2024-09-10T09:55:00.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.163+0000 7f8335426740 1 -- 172.21.15.77:0/2146665975 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f831c0035f0 con 0x559f3a227330 2024-09-10T09:55:00.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.163+0000 7f8315600640 1 -- 172.21.15.77:0/2146665975 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f831c005470 con 0x559f3a227330 2024-09-10T09:55:00.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.163+0000 7f8315600640 1 -- 172.21.15.77:0/2146665975 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f831c030050 con 0x559f3a227330 2024-09-10T09:55:00.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.163+0000 7f8335426740 10 client.15099 fetch_fsmap learned FSMap version 28 2024-09-10T09:55:00.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.163+0000 7f8335426740 1 -- 172.21.15.77:0/2146665975 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559f3a240e10 con 0x559f3a227330 2024-09-10T09:55:00.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.163+0000 7f8315600640 1 -- 172.21.15.77:0/2146665975 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f831c0035f0 con 0x559f3a227330 2024-09-10T09:55:00.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.164+0000 7f8315600640 1 -- 172.21.15.77:0/2146665975 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f831c021070 con 0x559f3a227330 2024-09-10T09:55:00.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.164+0000 7f8335426740 10 client.15099 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T09:55:00.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.164+0000 7f8335426740 10 client.15099 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-10T09:55:00.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.164+0000 7f8335426740 20 client.15099 populate_metadata read hostname 'smithi077' 2024-09-10T09:55:00.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.164+0000 7f8335426740 1 --2- 172.21.15.77:0/2146665975 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x559f3a2431c0 0x559f3a244630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:00.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.164+0000 7f8335426740 4 client.15099 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:55:00.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.164+0000 7f8335426740 1 -- 172.21.15.77:0/2146665975 --> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f3a19a620 con 0x559f3a2431c0 2024-09-10T09:55:00.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.164+0000 7f832aa00640 1 --2- 172.21.15.77:0/2146665975 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x559f3a2431c0 0x559f3a244630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:00.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.165+0000 7f832aa00640 1 --2- 172.21.15.77:0/2146665975 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x559f3a2431c0 0x559f3a244630 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=de606f3220ca70b1 server_cookie=1d8201488e3d1a06 in_seq=0 out_seq=0 2024-09-10T09:55:00.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.165+0000 7f8315600640 10 client.15099 ms_handle_connect on v2:172.21.15.134:6834/3293933699 2024-09-10T09:55:00.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.166+0000 7f8315600640 1 -- 172.21.15.77:0/2146665975 <== mds.4 v2:172.21.15.134:6834/3293933699 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559f3a19a620 con 0x559f3a2431c0 2024-09-10T09:55:00.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.166+0000 7f8315600640 10 client.15099 handle_command_reply: tid=0 2024-09-10T09:55:00.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.166+0000 7f8335426740 1 client.15099 shutdown 2024-09-10T09:55:00.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.166+0000 7f8335426740 1 -- 172.21.15.77:0/2146665975 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f3a227330 msgr2=0x559f3a22f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:00.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.166+0000 7f8335426740 1 --2- 172.21.15.77:0/2146665975 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f3a227330 0x559f3a22f620 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f831c0097c0 tx=0x7f831c005280 comp rx=0 tx=0).stop 2024-09-10T09:55:00.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.166+0000 7f8335426740 1 -- 172.21.15.77:0/2146665975 shutdown_connections 2024-09-10T09:55:00.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.166+0000 7f8335426740 1 --2- 172.21.15.77:0/2146665975 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x559f3a2431c0 0x559f3a244630 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:00.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.166+0000 7f8335426740 1 --2- 172.21.15.77:0/2146665975 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f3a232510 0x559f3a2300a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:00.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.166+0000 7f8335426740 1 --2- 172.21.15.77:0/2146665975 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f3a227c50 0x559f3a22fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:00.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.166+0000 7f8335426740 1 --2- 172.21.15.77:0/2146665975 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f3a227330 0x559f3a22f620 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:00.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.166+0000 7f8315600640 0 client.15099 ms_handle_reset on v2:172.21.15.134:6834/3293933699 2024-09-10T09:55:00.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.166+0000 7f8335426740 1 -- 172.21.15.77:0/2146665975 >> 172.21.15.77:0/2146665975 conn(0x559f3a2238c0 msgr2=0x559f3a2295f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:00.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.166+0000 7f8335426740 1 -- 172.21.15.77:0/2146665975 shutdown_connections 2024-09-10T09:55:00.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.166+0000 7f8335426740 1 -- 172.21.15.77:0/2146665975 wait complete. 2024-09-10T09:55:00.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.167+0000 7f8335426740 20 client.15099 trim_cache size 0 max 16384 2024-09-10T09:55:00.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.185+0000 7f8335426740 1 Processor -- start 2024-09-10T09:55:00.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.185+0000 7f8335426740 1 -- start start 2024-09-10T09:55:00.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.185+0000 7f8335426740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f3a227330 0x559f3a229d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:00.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.185+0000 7f8335426740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f3a227c50 0x559f3a22fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:00.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.185+0000 7f8335426740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f3a232510 0x559f3a230110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:00.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.185+0000 7f8335426740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x559f3a242970 con 0x559f3a232510 2024-09-10T09:55:00.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.185+0000 7f8335426740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x559f3a243040 con 0x559f3a227c50 2024-09-10T09:55:00.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.185+0000 7f8335426740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559f3a243710 con 0x559f3a227330 2024-09-10T09:55:00.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.185+0000 7f832be00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f3a232510 0x559f3a230110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:00.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.185+0000 7f832be00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f3a232510 0x559f3a230110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49082/0 (socket says 172.21.15.77:49082) 2024-09-10T09:55:00.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.185+0000 7f832be00640 1 -- 172.21.15.77:0/898022218 learned_addr learned my addr 172.21.15.77:0/898022218 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:55:00.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.186+0000 7f832b400640 1 --2- 172.21.15.77:0/898022218 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f3a227330 0x559f3a229d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:00.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.186+0000 7f832aa00640 1 --2- 172.21.15.77:0/898022218 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f3a227c50 0x559f3a22fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:00.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.186+0000 7f832be00640 1 -- 172.21.15.77:0/898022218 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f3a227330 msgr2=0x559f3a229d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:00.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.186+0000 7f832be00640 1 --2- 172.21.15.77:0/898022218 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f3a227330 0x559f3a229d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:00.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.186+0000 7f832be00640 1 -- 172.21.15.77:0/898022218 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f3a227c50 msgr2=0x559f3a22fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:00.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.186+0000 7f832be00640 1 --2- 172.21.15.77:0/898022218 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f3a227c50 0x559f3a22fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:00.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.186+0000 7f832be00640 1 -- 172.21.15.77:0/898022218 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f3a224a30 con 0x559f3a232510 2024-09-10T09:55:00.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.186+0000 7f832aa00640 1 --2- 172.21.15.77:0/898022218 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f3a227c50 0x559f3a22fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:55:00.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.186+0000 7f832be00640 1 --2- 172.21.15.77:0/898022218 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f3a232510 0x559f3a230110 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f83240cd1c0 tx=0x7f83240cd090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=415772c523034591 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:00.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.186+0000 7f8317e00640 1 -- 172.21.15.77:0/898022218 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83240a4b90 con 0x559f3a232510 2024-09-10T09:55:00.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.186+0000 7f8317e00640 1 -- 172.21.15.77:0/898022218 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f832409a220 con 0x559f3a232510 2024-09-10T09:55:00.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.186+0000 7f8317e00640 1 -- 172.21.15.77:0/898022218 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83240bab60 con 0x559f3a232510 2024-09-10T09:55:00.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.187+0000 7f8335426740 1 -- 172.21.15.77:0/898022218 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f3a232510 msgr2=0x559f3a230110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:00.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.187+0000 7f8335426740 1 --2- 172.21.15.77:0/898022218 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f3a232510 0x559f3a230110 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f83240cd1c0 tx=0x7f83240cd090 comp rx=0 tx=0).stop 2024-09-10T09:55:00.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.187+0000 7f8335426740 1 -- 172.21.15.77:0/898022218 shutdown_connections 2024-09-10T09:55:00.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.187+0000 7f8335426740 1 --2- 172.21.15.77:0/898022218 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f3a232510 0x559f3a230110 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:00.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.187+0000 7f8335426740 1 --2- 172.21.15.77:0/898022218 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f3a227c50 0x559f3a22fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:00.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.187+0000 7f8335426740 1 --2- 172.21.15.77:0/898022218 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f3a227330 0x559f3a229d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:00.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.187+0000 7f8335426740 1 -- 172.21.15.77:0/898022218 >> 172.21.15.77:0/898022218 conn(0x559f3a2238c0 msgr2=0x559f3a23dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:00.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.187+0000 7f8335426740 1 -- 172.21.15.77:0/898022218 shutdown_connections 2024-09-10T09:55:00.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.187+0000 7f8335426740 1 -- 172.21.15.77:0/898022218 wait complete. 2024-09-10T09:55:00.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.188+0000 7f8335426740 1 -- start start 2024-09-10T09:55:00.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.188+0000 7f8335426740 1 Processor -- start 2024-09-10T09:55:00.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.188+0000 7f8335426740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f3a227330 0x559f3a2476e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:00.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.188+0000 7f8335426740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f3a227c50 0x559f3a247c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:00.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.188+0000 7f8335426740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f3a232510 0x559f3a24c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:00.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.188+0000 7f8335426740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x559f3a2444b0 con 0x559f3a227330 2024-09-10T09:55:00.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.188+0000 7f8335426740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x559f3a243de0 con 0x559f3a232510 2024-09-10T09:55:00.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.188+0000 7f8335426740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559f3a244b80 con 0x559f3a227c50 2024-09-10T09:55:00.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.188+0000 7f8335426740 10 client.0 _ll_register_callbacks cb 0x7f831c009ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:55:00.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.189+0000 7f832b400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f3a227330 0x559f3a2476e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:00.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.189+0000 7f832b400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f3a227330 0x559f3a2476e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49092/0 (socket says 172.21.15.77:49092) 2024-09-10T09:55:00.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.189+0000 7f832b400640 1 -- 172.21.15.77:0/3322842649 learned_addr learned my addr 172.21.15.77:0/3322842649 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:55:00.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.189+0000 7f832be00640 1 --2- 172.21.15.77:0/3322842649 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f3a232510 0x559f3a24c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:00.194 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.189+0000 7f832aa00640 1 --2- 172.21.15.77:0/3322842649 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f3a227c50 0x559f3a247c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:00.194 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.189+0000 7f832b400640 1 -- 172.21.15.77:0/3322842649 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f3a227c50 msgr2=0x559f3a247c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:00.194 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.189+0000 7f832b400640 1 --2- 172.21.15.77:0/3322842649 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f3a227c50 0x559f3a247c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:00.194 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.189+0000 7f832b400640 1 -- 172.21.15.77:0/3322842649 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f3a232510 msgr2=0x559f3a24c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:00.194 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.189+0000 7f832b400640 1 --2- 172.21.15.77:0/3322842649 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f3a232510 0x559f3a24c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:00.194 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.189+0000 7f832b400640 1 -- 172.21.15.77:0/3322842649 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f3a24cc70 con 0x559f3a227330 2024-09-10T09:55:00.194 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.189+0000 7f832be00640 1 --2- 172.21.15.77:0/3322842649 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f3a232510 0x559f3a24c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:55:00.194 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.189+0000 7f832b400640 1 --2- 172.21.15.77:0/3322842649 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f3a227330 0x559f3a2476e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f831c01c040 tx=0x7f831c005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9da315e1855924ec server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:00.194 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.189+0000 7f8315600640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-10T09:55:00.194 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.190+0000 7f8315600640 1 -- 172.21.15.77:0/3322842649 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f831c01e040 con 0x559f3a227330 2024-09-10T09:55:00.194 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.190+0000 7f8315600640 1 -- 172.21.15.77:0/3322842649 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f831c0052f0 con 0x559f3a227330 2024-09-10T09:55:00.194 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.190+0000 7f8335426740 1 -- 172.21.15.77:0/3322842649 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f832409a220 con 0x559f3a227330 2024-09-10T09:55:00.194 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.190+0000 7f8315600640 1 -- 172.21.15.77:0/3322842649 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f831c0035f0 con 0x559f3a227330 2024-09-10T09:55:00.194 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.190+0000 7f8335426740 10 client.15105 fetch_fsmap learned FSMap version 28 2024-09-10T09:55:00.194 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.190+0000 7f8335426740 1 -- 172.21.15.77:0/3322842649 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559f3a2adfd0 con 0x559f3a227330 2024-09-10T09:55:00.195 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.190+0000 7f8315600640 1 -- 172.21.15.77:0/3322842649 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f832409a220 con 0x559f3a227330 2024-09-10T09:55:00.195 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.190+0000 7f8315600640 1 -- 172.21.15.77:0/3322842649 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f831c021040 con 0x559f3a227330 2024-09-10T09:55:00.195 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.190+0000 7f8335426740 10 client.15105 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T09:55:00.195 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.191+0000 7f8335426740 10 client.15105 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-10T09:55:00.195 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.191+0000 7f8335426740 20 client.15105 populate_metadata read hostname 'smithi077' 2024-09-10T09:55:00.195 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.191+0000 7f8335426740 1 --2- 172.21.15.77:0/3322842649 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x559f3a2b2c10 0x559f3a2b1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:00.195 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.191+0000 7f8335426740 4 client.15105 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T09:55:00.195 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.191+0000 7f8335426740 1 -- 172.21.15.77:0/3322842649 --> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] -- command(tid 0: {"prefix": "status"}) -- 0x559f3a194b30 con 0x559f3a2b2c10 2024-09-10T09:55:00.196 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.191+0000 7f832aa00640 1 --2- 172.21.15.77:0/3322842649 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x559f3a2b2c10 0x559f3a2b1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:00.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.192+0000 7f832aa00640 1 --2- 172.21.15.77:0/3322842649 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x559f3a2b2c10 0x559f3a2b1920 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8033812a69fe232f server_cookie=ede6d1f3e22c52b2 in_seq=0 out_seq=0 2024-09-10T09:55:00.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.192+0000 7f8315600640 10 client.15105 ms_handle_connect on v2:172.21.15.134:6834/3293933699 2024-09-10T09:55:00.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.192+0000 7f8315600640 1 -- 172.21.15.77:0/3322842649 <== mds.4 v2:172.21.15.134:6834/3293933699 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x559f3a194b30 con 0x559f3a2b2c10 2024-09-10T09:55:00.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.192+0000 7f8315600640 10 client.15105 handle_command_reply: tid=0 2024-09-10T09:55:00.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.192+0000 7f8335426740 1 client.15105 shutdown 2024-09-10T09:55:00.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.192+0000 7f8335426740 1 -- 172.21.15.77:0/3322842649 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f3a227330 msgr2=0x559f3a2476e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:00.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.192+0000 7f8335426740 1 --2- 172.21.15.77:0/3322842649 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f3a227330 0x559f3a2476e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f831c01c040 tx=0x7f831c005e70 comp rx=0 tx=0).stop 2024-09-10T09:55:00.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.192+0000 7f8335426740 1 -- 172.21.15.77:0/3322842649 shutdown_connections 2024-09-10T09:55:00.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.192+0000 7f8335426740 1 --2- 172.21.15.77:0/3322842649 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x559f3a2b2c10 0x559f3a2b1920 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:00.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.192+0000 7f8335426740 1 --2- 172.21.15.77:0/3322842649 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f3a232510 0x559f3a24c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:00.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.192+0000 7f8335426740 1 --2- 172.21.15.77:0/3322842649 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f3a227c50 0x559f3a247c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:00.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.192+0000 7f8335426740 1 --2- 172.21.15.77:0/3322842649 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f3a227330 0x559f3a2476e0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:00.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.192+0000 7f8335426740 1 -- 172.21.15.77:0/3322842649 >> 172.21.15.77:0/3322842649 conn(0x559f3a2238c0 msgr2=0x559f3a222200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:00.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.192+0000 7f8315600640 0 client.15105 ms_handle_reset on v2:172.21.15.134:6834/3293933699 2024-09-10T09:55:00.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.192+0000 7f8335426740 1 -- 172.21.15.77:0/3322842649 shutdown_connections 2024-09-10T09:55:00.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.192+0000 7f8335426740 1 -- 172.21.15.77:0/3322842649 wait complete. 2024-09-10T09:55:00.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.193+0000 7f8335426740 20 client.15105 trim_cache size 0 max 16384 2024-09-10T09:55:00.198 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-10T09:55:00.198 INFO:teuthology.orchestra.run.smithi077.stdout: "cluster_fsid": "14a55226-6f59-11ef-bcea-c7b262605968", 2024-09-10T09:55:00.198 INFO:teuthology.orchestra.run.smithi077.stdout: "whoami": 4, 2024-09-10T09:55:00.198 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 24496, 2024-09-10T09:55:00.198 INFO:teuthology.orchestra.run.smithi077.stdout: "want_state": "up:active", 2024-09-10T09:55:00.198 INFO:teuthology.orchestra.run.smithi077.stdout: "state": "up:active", 2024-09-10T09:55:00.198 INFO:teuthology.orchestra.run.smithi077.stdout: "fs_name": "cephfs", 2024-09-10T09:55:00.198 INFO:teuthology.orchestra.run.smithi077.stdout: "rank_uptime": 104.327464881, 2024-09-10T09:55:00.198 INFO:teuthology.orchestra.run.smithi077.stdout: "mdsmap_epoch": 28, 2024-09-10T09:55:00.198 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch": 96, 2024-09-10T09:55:00.199 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch_barrier": 94, 2024-09-10T09:55:00.199 INFO:teuthology.orchestra.run.smithi077.stdout: "uptime": 187.80178745200001 2024-09-10T09:55:00.199 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-10T09:55:00.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.193+0000 7f8332800640 1 -- 172.21.15.77:0/1115376486 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f83100a2b80 msgr2=0x7f83100a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:00.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.193+0000 7f8332800640 1 --2- 172.21.15.77:0/1115376486 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f83100a2b80 0x7f83100a5040 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f83180096c0 tx=0x7f8318000bc0 comp rx=0 tx=0).stop 2024-09-10T09:55:00.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.193+0000 7f8332800640 1 -- 172.21.15.77:0/1115376486 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f832c106f10 msgr2=0x7f832c19ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:00.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.193+0000 7f8332800640 1 --2- 172.21.15.77:0/1115376486 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f832c106f10 0x7f832c19ef40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8324009fd0 tx=0x7f832400ef10 comp rx=0 tx=0).stop 2024-09-10T09:55:00.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.193+0000 7f8332800640 1 -- 172.21.15.77:0/1115376486 shutdown_connections 2024-09-10T09:55:00.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.193+0000 7f8332800640 1 --2- 172.21.15.77:0/1115376486 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f83100a2b80 0x7f83100a5040 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:00.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.193+0000 7f8332800640 1 --2- 172.21.15.77:0/1115376486 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f832c106f10 0x7f832c19ef40 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:00.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.193+0000 7f8332800640 1 --2- 172.21.15.77:0/1115376486 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f832c102790 0x7f832c19abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:00.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.193+0000 7f8332800640 1 --2- 172.21.15.77:0/1115376486 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f832c101de0 0x7f832c19a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:00.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.193+0000 7f8332800640 1 -- 172.21.15.77:0/1115376486 >> 172.21.15.77:0/1115376486 conn(0x7f832c0fd8c0 msgr2=0x7f832c0fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:00.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.193+0000 7f8332800640 1 -- 172.21.15.77:0/1115376486 shutdown_connections 2024-09-10T09:55:00.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:00.193+0000 7f8332800640 1 -- 172.21.15.77:0/1115376486 wait complete. 2024-09-10T09:55:01.019 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.c status 2024-09-10T09:55:01.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:55:00 smithi134 ceph-mon[35674]: pgmap v386: 129 pgs: 129 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-09-10T09:55:01.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:55:00 smithi120 ceph-mon[34770]: pgmap v386: 129 pgs: 129 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-09-10T09:55:01.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:55:00 smithi077 ceph-mon[30319]: pgmap v386: 129 pgs: 129 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-09-10T09:55:01.400 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:55:02.128 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.119+0000 7f6c04000640 1 -- 172.21.15.77:0/2178195831 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6bfc10e3c0 msgr2=0x7f6bfc110810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:02.128 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.119+0000 7f6c04000640 1 --2- 172.21.15.77:0/2178195831 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6bfc10e3c0 0x7f6bfc110810 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f6bf4009ca0 tx=0x7f6bf402a950 comp rx=0 tx=0).stop 2024-09-10T09:55:02.128 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.119+0000 7f6c04000640 1 -- 172.21.15.77:0/2178195831 shutdown_connections 2024-09-10T09:55:02.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.119+0000 7f6c04000640 1 --2- 172.21.15.77:0/2178195831 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6bfc10e3c0 0x7f6bfc110810 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:02.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.119+0000 7f6c04000640 1 --2- 172.21.15.77:0/2178195831 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6bfc074f40 0x7f6bfc10de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:02.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.119+0000 7f6c04000640 1 --2- 172.21.15.77:0/2178195831 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6bfc074620 0x7f6bfc074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:02.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.120+0000 7f6c04000640 1 -- 172.21.15.77:0/2178195831 >> 172.21.15.77:0/2178195831 conn(0x7f6bfc0fd8c0 msgr2=0x7f6bfc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:02.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.120+0000 7f6c04000640 1 -- 172.21.15.77:0/2178195831 shutdown_connections 2024-09-10T09:55:02.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.120+0000 7f6c04000640 1 -- 172.21.15.77:0/2178195831 wait complete. 2024-09-10T09:55:02.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.121+0000 7f6c04000640 1 Processor -- start 2024-09-10T09:55:02.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.121+0000 7f6c04000640 1 -- start start 2024-09-10T09:55:02.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.121+0000 7f6c04000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6bfc074620 0x7f6bfc19a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:02.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.121+0000 7f6c04000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6bfc074f40 0x7f6bfc19ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:02.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.121+0000 7f6c04000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6bfc10e3c0 0x7f6bfc19eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:02.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.121+0000 7f6c04000640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6bfc1141a0 con 0x7f6bfc074f40 2024-09-10T09:55:02.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.121+0000 7f6c04000640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6bfc114020 con 0x7f6bfc10e3c0 2024-09-10T09:55:02.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.121+0000 7f6c04000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6bfc114320 con 0x7f6bfc074620 2024-09-10T09:55:02.130 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.121+0000 7f6c00c00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6bfc074f40 0x7f6bfc19ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:02.130 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.121+0000 7f6c01600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6bfc074620 0x7f6bfc19a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:02.130 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.121+0000 7f6c00c00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6bfc074f40 0x7f6bfc19ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49118/0 (socket says 172.21.15.77:49118) 2024-09-10T09:55:02.130 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.121+0000 7f6c01600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6bfc074620 0x7f6bfc19a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:48890/0 (socket says 172.21.15.77:48890) 2024-09-10T09:55:02.130 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.121+0000 7f6c00c00640 1 -- 172.21.15.77:0/3921890368 learned_addr learned my addr 172.21.15.77:0/3921890368 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:55:02.130 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.121+0000 7f6c02000640 1 --2- 172.21.15.77:0/3921890368 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6bfc10e3c0 0x7f6bfc19eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:02.130 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.122+0000 7f6c00c00640 1 -- 172.21.15.77:0/3921890368 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6bfc074620 msgr2=0x7f6bfc19a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:02.130 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.122+0000 7f6c00c00640 1 --2- 172.21.15.77:0/3921890368 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6bfc074620 0x7f6bfc19a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:02.130 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.122+0000 7f6c00c00640 1 -- 172.21.15.77:0/3921890368 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6bfc10e3c0 msgr2=0x7f6bfc19eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:02.130 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.122+0000 7f6c00c00640 1 --2- 172.21.15.77:0/3921890368 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6bfc10e3c0 0x7f6bfc19eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:02.130 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.122+0000 7f6c00c00640 1 -- 172.21.15.77:0/3921890368 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bfc19f5d0 con 0x7f6bfc074f40 2024-09-10T09:55:02.130 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.122+0000 7f6c00c00640 1 --2- 172.21.15.77:0/3921890368 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6bfc074f40 0x7f6bfc19ab60 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f6be800d950 tx=0x7f6be800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:02.130 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.122+0000 7f6bfa000640 1 -- 172.21.15.77:0/3921890368 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6be8017030 con 0x7f6bfc074f40 2024-09-10T09:55:02.130 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.122+0000 7f6bfa000640 1 -- 172.21.15.77:0/3921890368 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6be8004020 con 0x7f6bfc074f40 2024-09-10T09:55:02.130 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.122+0000 7f6c04000640 1 -- 172.21.15.77:0/3921890368 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bfc19f8c0 con 0x7f6bfc074f40 2024-09-10T09:55:02.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.122+0000 7f6bfa000640 1 -- 172.21.15.77:0/3921890368 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6be80047a0 con 0x7f6bfc074f40 2024-09-10T09:55:02.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.122+0000 7f6c04000640 1 -- 172.21.15.77:0/3921890368 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bfc1a71a0 con 0x7f6bfc074f40 2024-09-10T09:55:02.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.125+0000 7f6bfa000640 1 -- 172.21.15.77:0/3921890368 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6be80041d0 con 0x7f6bfc074f40 2024-09-10T09:55:02.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.126+0000 7f6c06be7740 1 Processor -- start 2024-09-10T09:55:02.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.126+0000 7f6c06be7740 1 -- start start 2024-09-10T09:55:02.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.126+0000 7f6c06be7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604fbd12330 0x5604fbd12710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:02.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.126+0000 7f6c06be7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5604fbd12c50 0x5604fbd1cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:02.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.126+0000 7f6c06be7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5604fbd1d510 0x5604fbd1f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:02.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.126+0000 7f6c06be7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5604fbc780b0 con 0x5604fbd12c50 2024-09-10T09:55:02.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.126+0000 7f6c06be7740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5604fbc1fab0 con 0x5604fbd1d510 2024-09-10T09:55:02.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.126+0000 7f6c06be7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5604fbc74070 con 0x5604fbd12330 2024-09-10T09:55:02.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.127+0000 7f6c02000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604fbd12330 0x5604fbd12710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:02.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.127+0000 7f6c01600640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5604fbd12c50 0x5604fbd1cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:02.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.127+0000 7f6c02000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604fbd12330 0x5604fbd12710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:48896/0 (socket says 172.21.15.77:48896) 2024-09-10T09:55:02.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.127+0000 7f6c02000640 1 -- 172.21.15.77:0/628004586 learned_addr learned my addr 172.21.15.77:0/628004586 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:55:02.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.127+0000 7f6c00c00640 1 --2- 172.21.15.77:0/628004586 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5604fbd1d510 0x5604fbd1f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:02.135 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.127+0000 7f6c01600640 1 -- 172.21.15.77:0/628004586 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604fbd12330 msgr2=0x5604fbd12710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:02.136 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.127+0000 7f6c01600640 1 --2- 172.21.15.77:0/628004586 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604fbd12330 0x5604fbd12710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:02.136 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.127+0000 7f6bfa000640 1 --2- 172.21.15.77:0/3921890368 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f6be40a2b80 0x7f6be40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:02.136 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.127+0000 7f6c01600640 1 -- 172.21.15.77:0/628004586 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5604fbd1d510 msgr2=0x5604fbd1f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:02.136 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.127+0000 7f6c01600640 1 --2- 172.21.15.77:0/628004586 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5604fbd1d510 0x5604fbd1f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:02.136 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.127+0000 7f6c01600640 1 -- 172.21.15.77:0/628004586 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604fbc2c4a0 con 0x5604fbd12c50 2024-09-10T09:55:02.137 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.127+0000 7f6c00c00640 1 --2- 172.21.15.77:0/628004586 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5604fbd1d510 0x5604fbd1f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:55:02.137 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.127+0000 7f6c02000640 1 --2- 172.21.15.77:0/628004586 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604fbd12330 0x5604fbd12710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:55:02.137 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.127+0000 7f6bfa000640 1 -- 172.21.15.77:0/3921890368 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f6be80ced80 con 0x7f6bfc074f40 2024-09-10T09:55:02.137 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.128+0000 7f6c02000640 1 --2- 172.21.15.77:0/3921890368 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f6be40a2b80 0x7f6be40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:02.137 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.128+0000 7f6c01600640 1 --2- 172.21.15.77:0/628004586 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5604fbd12c50 0x5604fbd1cfd0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f6bf0009c20 tx=0x7f6bf0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b19b9dd5f8c5f91 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:02.137 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.128+0000 7f6bee600640 1 -- 172.21.15.77:0/628004586 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf0004050 con 0x5604fbd12c50 2024-09-10T09:55:02.137 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.128+0000 7f6bee600640 1 -- 172.21.15.77:0/628004586 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6bf0005500 con 0x5604fbd12c50 2024-09-10T09:55:02.137 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.128+0000 7f6bee600640 1 -- 172.21.15.77:0/628004586 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf00035f0 con 0x5604fbd12c50 2024-09-10T09:55:02.137 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.129+0000 7f6c06be7740 1 -- 172.21.15.77:0/628004586 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5604fbd12c50 msgr2=0x5604fbd1cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:02.137 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.129+0000 7f6c06be7740 1 --2- 172.21.15.77:0/628004586 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5604fbd12c50 0x5604fbd1cfd0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f6bf0009c20 tx=0x7f6bf0011810 comp rx=0 tx=0).stop 2024-09-10T09:55:02.137 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.129+0000 7f6c02000640 1 --2- 172.21.15.77:0/3921890368 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f6be40a2b80 0x7f6be40a5040 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6bf402ae70 tx=0x7f6bf4005d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:02.137 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.129+0000 7f6c06be7740 1 -- 172.21.15.77:0/628004586 shutdown_connections 2024-09-10T09:55:02.137 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.129+0000 7f6c06be7740 1 --2- 172.21.15.77:0/628004586 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5604fbd1d510 0x5604fbd1f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:02.137 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.129+0000 7f6c06be7740 1 --2- 172.21.15.77:0/628004586 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5604fbd12c50 0x5604fbd1cfd0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:02.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.129+0000 7f6c06be7740 1 --2- 172.21.15.77:0/628004586 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604fbd12330 0x5604fbd12710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:02.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.129+0000 7f6c06be7740 1 -- 172.21.15.77:0/628004586 >> 172.21.15.77:0/628004586 conn(0x5604fbd0e8c0 msgr2=0x5604fbd0eca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:02.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.129+0000 7f6c06be7740 1 -- 172.21.15.77:0/628004586 shutdown_connections 2024-09-10T09:55:02.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.129+0000 7f6c06be7740 1 -- 172.21.15.77:0/628004586 wait complete. 2024-09-10T09:55:02.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.129+0000 7f6c06be7740 1 -- start start 2024-09-10T09:55:02.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.130+0000 7f6c06be7740 1 Processor -- start 2024-09-10T09:55:02.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.130+0000 7f6c06be7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604fbd12330 0x5604fbd1a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:02.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.130+0000 7f6c06be7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5604fbd12c50 0x5604fbd1aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:02.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.130+0000 7f6c06be7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5604fbd1d510 0x5604fbd1afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:02.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.130+0000 7f6c06be7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5604fbd20060 con 0x5604fbd12c50 2024-09-10T09:55:02.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.130+0000 7f6c06be7740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5604fbc00920 con 0x5604fbd1d510 2024-09-10T09:55:02.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.130+0000 7f6c06be7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5604fbd201e0 con 0x5604fbd12330 2024-09-10T09:55:02.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.130+0000 7f6c06be7740 10 client.0 _ll_register_callbacks cb 0x5604fbc248a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:55:02.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.130+0000 7f6c01600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604fbd12330 0x5604fbd1a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:02.139 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.130+0000 7f6c01600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604fbd12330 0x5604fbd1a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:48908/0 (socket says 172.21.15.77:48908) 2024-09-10T09:55:02.139 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.130+0000 7f6c01600640 1 -- 172.21.15.77:0/1645964287 learned_addr learned my addr 172.21.15.77:0/1645964287 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:55:02.139 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.130+0000 7f6c00c00640 1 --2- 172.21.15.77:0/1645964287 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5604fbd12c50 0x5604fbd1aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:02.139 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.130+0000 7f6c02000640 1 --2- 172.21.15.77:0/1645964287 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5604fbd1d510 0x5604fbd1afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:02.139 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.131+0000 7f6c01600640 1 -- 172.21.15.77:0/1645964287 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5604fbd1d510 msgr2=0x5604fbd1afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:02.139 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.131+0000 7f6c01600640 1 --2- 172.21.15.77:0/1645964287 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5604fbd1d510 0x5604fbd1afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:02.139 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.131+0000 7f6c01600640 1 -- 172.21.15.77:0/1645964287 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5604fbd12c50 msgr2=0x5604fbd1aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:02.139 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.131+0000 7f6c01600640 1 --2- 172.21.15.77:0/1645964287 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5604fbd12c50 0x5604fbd1aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:02.139 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.131+0000 7f6c01600640 1 -- 172.21.15.77:0/1645964287 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604fb9e92b0 con 0x5604fbd12330 2024-09-10T09:55:02.139 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.131+0000 7f6c00c00640 1 --2- 172.21.15.77:0/1645964287 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5604fbd12c50 0x5604fbd1aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:55:02.139 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.131+0000 7f6c01600640 1 --2- 172.21.15.77:0/1645964287 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604fbd12330 0x5604fbd1a530 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6bf0004a20 tx=0x7f6bf0004d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a027a43094965c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:02.139 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.131+0000 7f6be2a00640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-10T09:55:02.139 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.131+0000 7f6be2a00640 1 -- 172.21.15.77:0/1645964287 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf0029070 con 0x5604fbd12330 2024-09-10T09:55:02.139 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.132+0000 7f6c06be7740 1 -- 172.21.15.77:0/1645964287 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6bf0005500 con 0x5604fbd12330 2024-09-10T09:55:02.139 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.132+0000 7f6be2a00640 1 -- 172.21.15.77:0/1645964287 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f6bf0005180 con 0x5604fbd12330 2024-09-10T09:55:02.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.132+0000 7f6be2a00640 1 -- 172.21.15.77:0/1645964287 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6bf0035050 con 0x5604fbd12330 2024-09-10T09:55:02.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.132+0000 7f6c06be7740 10 client.25141 fetch_fsmap learned FSMap version 28 2024-09-10T09:55:02.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.132+0000 7f6c06be7740 1 -- 172.21.15.77:0/1645964287 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5604fbd2bdb0 con 0x5604fbd12330 2024-09-10T09:55:02.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.132+0000 7f6be2a00640 1 -- 172.21.15.77:0/1645964287 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6bf0005500 con 0x5604fbd12330 2024-09-10T09:55:02.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.132+0000 7f6be2a00640 1 -- 172.21.15.77:0/1645964287 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f6bf0023430 con 0x5604fbd12330 2024-09-10T09:55:02.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.132+0000 7f6c06be7740 10 client.25141 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T09:55:02.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.133+0000 7f6c06be7740 10 client.25141 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T09:55:02.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.133+0000 7f6c06be7740 20 client.25141 populate_metadata read hostname 'smithi077' 2024-09-10T09:55:02.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.133+0000 7f6c06be7740 1 --2- 172.21.15.77:0/1645964287 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x5604fbd2e160 0x5604fbd2f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:02.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.133+0000 7f6c06be7740 4 client.25141 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:55:02.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.133+0000 7f6c06be7740 1 -- 172.21.15.77:0/1645964287 --> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5604fbc85620 con 0x5604fbd2e160 2024-09-10T09:55:02.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.133+0000 7f6c00c00640 1 --2- 172.21.15.77:0/1645964287 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x5604fbd2e160 0x5604fbd2f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:02.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.133+0000 7f6c00c00640 1 --2- 172.21.15.77:0/1645964287 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x5604fbd2e160 0x5604fbd2f5d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=30fd6e8758d1ff38 server_cookie=849cf9de293f27be in_seq=0 out_seq=0 2024-09-10T09:55:02.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.133+0000 7f6be2a00640 10 client.25141 ms_handle_connect on v2:172.21.15.134:6836/2461497091 2024-09-10T09:55:02.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.134+0000 7f6be2a00640 1 -- 172.21.15.77:0/1645964287 <== mds.3 v2:172.21.15.134:6836/2461497091 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5604fbc85620 con 0x5604fbd2e160 2024-09-10T09:55:02.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.134+0000 7f6be2a00640 10 client.25141 handle_command_reply: tid=0 2024-09-10T09:55:02.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.134+0000 7f6c06be7740 1 client.25141 shutdown 2024-09-10T09:55:02.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.135+0000 7f6c06be7740 1 -- 172.21.15.77:0/1645964287 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604fbd12330 msgr2=0x5604fbd1a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:02.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.135+0000 7f6c06be7740 1 --2- 172.21.15.77:0/1645964287 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604fbd12330 0x5604fbd1a530 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6bf0004a20 tx=0x7f6bf0004d50 comp rx=0 tx=0).stop 2024-09-10T09:55:02.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.135+0000 7f6c06be7740 1 -- 172.21.15.77:0/1645964287 shutdown_connections 2024-09-10T09:55:02.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.135+0000 7f6c06be7740 1 --2- 172.21.15.77:0/1645964287 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5604fbd1d510 0x5604fbd1afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:02.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.135+0000 7f6c06be7740 1 --2- 172.21.15.77:0/1645964287 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x5604fbd2e160 0x5604fbd2f5d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:02.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.135+0000 7f6c06be7740 1 --2- 172.21.15.77:0/1645964287 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5604fbd12c50 0x5604fbd1aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:02.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.135+0000 7f6c06be7740 1 --2- 172.21.15.77:0/1645964287 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604fbd12330 0x5604fbd1a530 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:02.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.135+0000 7f6be2a00640 0 client.25141 ms_handle_reset on v2:172.21.15.134:6836/2461497091 2024-09-10T09:55:02.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.135+0000 7f6c06be7740 1 -- 172.21.15.77:0/1645964287 >> 172.21.15.77:0/1645964287 conn(0x5604fbd0e8c0 msgr2=0x5604fbd145a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:02.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.135+0000 7f6c06be7740 1 -- 172.21.15.77:0/1645964287 shutdown_connections 2024-09-10T09:55:02.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.135+0000 7f6c06be7740 1 -- 172.21.15.77:0/1645964287 wait complete. 2024-09-10T09:55:02.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.135+0000 7f6c06be7740 20 client.25141 trim_cache size 0 max 16384 2024-09-10T09:55:02.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.153+0000 7f6c06be7740 1 Processor -- start 2024-09-10T09:55:02.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.153+0000 7f6c06be7740 1 -- start start 2024-09-10T09:55:02.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.154+0000 7f6c06be7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5604fbd0e8c0 0x5604fbd0eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:02.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.154+0000 7f6c06be7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604fbd12330 0x5604fbd14d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:02.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.154+0000 7f6c06be7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5604fbd12c50 0x5604fbd1c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:02.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.154+0000 7f6c06be7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5604fbd2d910 con 0x5604fbd0e8c0 2024-09-10T09:55:02.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.154+0000 7f6c06be7740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5604fbd2dfe0 con 0x5604fbd12c50 2024-09-10T09:55:02.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.154+0000 7f6c06be7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5604fbd2e6b0 con 0x5604fbd12330 2024-09-10T09:55:02.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.154+0000 7f6c01600640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5604fbd0e8c0 0x5604fbd0eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:02.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.154+0000 7f6c01600640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5604fbd0e8c0 0x5604fbd0eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49154/0 (socket says 172.21.15.77:49154) 2024-09-10T09:55:02.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.154+0000 7f6c00c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604fbd12330 0x5604fbd14d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:02.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.154+0000 7f6c01600640 1 -- 172.21.15.77:0/141891351 learned_addr learned my addr 172.21.15.77:0/141891351 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:55:02.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.154+0000 7f6c02000640 1 --2- 172.21.15.77:0/141891351 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5604fbd12c50 0x5604fbd1c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:02.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.154+0000 7f6c00c00640 1 -- 172.21.15.77:0/141891351 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5604fbd12c50 msgr2=0x5604fbd1c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:02.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.154+0000 7f6c00c00640 1 --2- 172.21.15.77:0/141891351 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5604fbd12c50 0x5604fbd1c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:02.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.154+0000 7f6c00c00640 1 -- 172.21.15.77:0/141891351 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5604fbd0e8c0 msgr2=0x5604fbd0eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:02.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.154+0000 7f6c00c00640 1 --2- 172.21.15.77:0/141891351 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5604fbd0e8c0 0x5604fbd0eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:02.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.154+0000 7f6c00c00640 1 -- 172.21.15.77:0/141891351 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604fbd12160 con 0x5604fbd12330 2024-09-10T09:55:02.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.154+0000 7f6c01600640 1 --2- 172.21.15.77:0/141891351 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5604fbd0e8c0 0x5604fbd0eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:55:02.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.155+0000 7f6c00c00640 1 --2- 172.21.15.77:0/141891351 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604fbd12330 0x5604fbd14d50 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x5604fbd0fc40 tx=0x7f6be80d27c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20b2e76c80240fb4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:02.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.155+0000 7f6bedc00640 1 -- 172.21.15.77:0/141891351 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6be80d0040 con 0x5604fbd12330 2024-09-10T09:55:02.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.155+0000 7f6bedc00640 1 -- 172.21.15.77:0/141891351 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6be809b890 con 0x5604fbd12330 2024-09-10T09:55:02.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.155+0000 7f6bedc00640 1 -- 172.21.15.77:0/141891351 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6be80cc540 con 0x5604fbd12330 2024-09-10T09:55:02.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.156+0000 7f6c06be7740 1 -- 172.21.15.77:0/141891351 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604fbd12330 msgr2=0x5604fbd14d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:02.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.156+0000 7f6c06be7740 1 --2- 172.21.15.77:0/141891351 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604fbd12330 0x5604fbd14d50 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x5604fbd0fc40 tx=0x7f6be80d27c0 comp rx=0 tx=0).stop 2024-09-10T09:55:02.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.156+0000 7f6c06be7740 1 -- 172.21.15.77:0/141891351 shutdown_connections 2024-09-10T09:55:02.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.156+0000 7f6c06be7740 1 --2- 172.21.15.77:0/141891351 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5604fbd12c50 0x5604fbd1c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:02.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.156+0000 7f6c06be7740 1 --2- 172.21.15.77:0/141891351 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604fbd12330 0x5604fbd14d50 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:02.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.156+0000 7f6c06be7740 1 --2- 172.21.15.77:0/141891351 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5604fbd0e8c0 0x5604fbd0eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:02.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.156+0000 7f6c06be7740 1 -- 172.21.15.77:0/141891351 >> 172.21.15.77:0/141891351 conn(0x5604fbd2d530 msgr2=0x5604fbd1bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:02.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.156+0000 7f6c06be7740 1 -- 172.21.15.77:0/141891351 shutdown_connections 2024-09-10T09:55:02.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.156+0000 7f6c06be7740 1 -- 172.21.15.77:0/141891351 wait complete. 2024-09-10T09:55:02.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.156+0000 7f6c06be7740 1 -- start start 2024-09-10T09:55:02.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.156+0000 7f6c06be7740 1 Processor -- start 2024-09-10T09:55:02.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.157+0000 7f6c06be7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5604fbd0e8c0 0x5604fbd31860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:02.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.157+0000 7f6c06be7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5604fbd12330 0x5604fbd31da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:02.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.157+0000 7f6c06be7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604fbd12c50 0x5604fbd32320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:02.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.157+0000 7f6c06be7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5604fbd2f450 con 0x5604fbd0e8c0 2024-09-10T09:55:02.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.157+0000 7f6c06be7740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5604fbd2ed80 con 0x5604fbd12330 2024-09-10T09:55:02.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.157+0000 7f6c06be7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5604fbd2fb20 con 0x5604fbd12c50 2024-09-10T09:55:02.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.157+0000 7f6c06be7740 10 client.0 _ll_register_callbacks cb 0x7f6bf0009a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:55:02.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.157+0000 7f6c01600640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5604fbd0e8c0 0x5604fbd31860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:02.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.157+0000 7f6c01600640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5604fbd0e8c0 0x5604fbd31860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49160/0 (socket says 172.21.15.77:49160) 2024-09-10T09:55:02.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.157+0000 7f6c01600640 1 -- 172.21.15.77:0/1550971934 learned_addr learned my addr 172.21.15.77:0/1550971934 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:55:02.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.157+0000 7f6c00c00640 1 --2- 172.21.15.77:0/1550971934 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5604fbd12330 0x5604fbd31da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:02.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.157+0000 7f6c02000640 1 --2- 172.21.15.77:0/1550971934 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604fbd12c50 0x5604fbd32320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:02.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.158+0000 7f6c01600640 1 -- 172.21.15.77:0/1550971934 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604fbd12c50 msgr2=0x5604fbd32320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:02.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.158+0000 7f6c01600640 1 --2- 172.21.15.77:0/1550971934 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604fbd12c50 0x5604fbd32320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:02.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.158+0000 7f6c01600640 1 -- 172.21.15.77:0/1550971934 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5604fbd12330 msgr2=0x5604fbd31da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:02.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.158+0000 7f6c01600640 1 --2- 172.21.15.77:0/1550971934 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5604fbd12330 0x5604fbd31da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:02.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.158+0000 7f6c01600640 1 -- 172.21.15.77:0/1550971934 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604fbd36340 con 0x5604fbd0e8c0 2024-09-10T09:55:02.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.158+0000 7f6c01600640 1 --2- 172.21.15.77:0/1550971934 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5604fbd0e8c0 0x5604fbd31860 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6bf0000c00 tx=0x7f6bf0004dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=def2ba91a7a61962 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:02.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.158+0000 7f6be2a00640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-10T09:55:02.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.158+0000 7f6be2a00640 1 -- 172.21.15.77:0/1550971934 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf0029040 con 0x5604fbd0e8c0 2024-09-10T09:55:02.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.158+0000 7f6c06be7740 1 -- 172.21.15.77:0/1550971934 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6be809b890 con 0x5604fbd0e8c0 2024-09-10T09:55:02.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.158+0000 7f6be2a00640 1 -- 172.21.15.77:0/1550971934 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f6bf0023a40 con 0x5604fbd0e8c0 2024-09-10T09:55:02.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.158+0000 7f6be2a00640 1 -- 172.21.15.77:0/1550971934 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6bf0023630 con 0x5604fbd0e8c0 2024-09-10T09:55:02.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.158+0000 7f6c06be7740 10 client.15123 fetch_fsmap learned FSMap version 28 2024-09-10T09:55:02.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.158+0000 7f6c06be7740 1 -- 172.21.15.77:0/1550971934 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5604fbd98e60 con 0x5604fbd0e8c0 2024-09-10T09:55:02.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.159+0000 7f6be2a00640 1 -- 172.21.15.77:0/1550971934 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6be809b890 con 0x5604fbd0e8c0 2024-09-10T09:55:02.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.159+0000 7f6be2a00640 1 -- 172.21.15.77:0/1550971934 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f6bf00035f0 con 0x5604fbd0e8c0 2024-09-10T09:55:02.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.159+0000 7f6c06be7740 10 client.15123 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T09:55:02.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.159+0000 7f6c06be7740 10 client.15123 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T09:55:02.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.159+0000 7f6c06be7740 20 client.15123 populate_metadata read hostname 'smithi077' 2024-09-10T09:55:02.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.159+0000 7f6c06be7740 1 --2- 172.21.15.77:0/1550971934 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x5604fbd1d510 0x5604fbd9c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:02.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.159+0000 7f6c06be7740 4 client.15123 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T09:55:02.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.159+0000 7f6c06be7740 1 -- 172.21.15.77:0/1550971934 --> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] -- command(tid 0: {"prefix": "status"}) -- 0x5604fbc7fb30 con 0x5604fbd1d510 2024-09-10T09:55:02.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.159+0000 7f6c00c00640 1 --2- 172.21.15.77:0/1550971934 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x5604fbd1d510 0x5604fbd9c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:02.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.160+0000 7f6c00c00640 1 --2- 172.21.15.77:0/1550971934 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x5604fbd1d510 0x5604fbd9c6c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d291ef6c0bb6e1b3 server_cookie=8f277df596e167d9 in_seq=0 out_seq=0 2024-09-10T09:55:02.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.160+0000 7f6be2a00640 10 client.15123 ms_handle_connect on v2:172.21.15.134:6836/2461497091 2024-09-10T09:55:02.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.160+0000 7f6be2a00640 1 -- 172.21.15.77:0/1550971934 <== mds.3 v2:172.21.15.134:6836/2461497091 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5604fbc7fb30 con 0x5604fbd1d510 2024-09-10T09:55:02.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.160+0000 7f6be2a00640 10 client.15123 handle_command_reply: tid=0 2024-09-10T09:55:02.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.160+0000 7f6c06be7740 1 client.15123 shutdown 2024-09-10T09:55:02.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.160+0000 7f6c06be7740 1 -- 172.21.15.77:0/1550971934 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5604fbd0e8c0 msgr2=0x5604fbd31860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:02.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.160+0000 7f6c06be7740 1 --2- 172.21.15.77:0/1550971934 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5604fbd0e8c0 0x5604fbd31860 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6bf0000c00 tx=0x7f6bf0004dc0 comp rx=0 tx=0).stop 2024-09-10T09:55:02.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.160+0000 7f6c06be7740 1 -- 172.21.15.77:0/1550971934 shutdown_connections 2024-09-10T09:55:02.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.160+0000 7f6c06be7740 1 --2- 172.21.15.77:0/1550971934 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604fbd12c50 0x5604fbd32320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:02.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.160+0000 7f6c06be7740 1 --2- 172.21.15.77:0/1550971934 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5604fbd12330 0x5604fbd31da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:02.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.160+0000 7f6c06be7740 1 --2- 172.21.15.77:0/1550971934 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x5604fbd1d510 0x5604fbd9c6c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:02.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.160+0000 7f6c06be7740 1 --2- 172.21.15.77:0/1550971934 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5604fbd0e8c0 0x5604fbd31860 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:02.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.160+0000 7f6be2a00640 0 client.15123 ms_handle_reset on v2:172.21.15.134:6836/2461497091 2024-09-10T09:55:02.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.160+0000 7f6c06be7740 1 -- 172.21.15.77:0/1550971934 >> 172.21.15.77:0/1550971934 conn(0x5604fbd2d530 msgr2=0x5604fbd18a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:02.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.161+0000 7f6c06be7740 1 -- 172.21.15.77:0/1550971934 shutdown_connections 2024-09-10T09:55:02.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.161+0000 7f6c06be7740 1 -- 172.21.15.77:0/1550971934 wait complete. 2024-09-10T09:55:02.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.161+0000 7f6c06be7740 20 client.15123 trim_cache size 0 max 16384 2024-09-10T09:55:02.167 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-10T09:55:02.167 INFO:teuthology.orchestra.run.smithi077.stdout: "cluster_fsid": "14a55226-6f59-11ef-bcea-c7b262605968", 2024-09-10T09:55:02.167 INFO:teuthology.orchestra.run.smithi077.stdout: "whoami": 3, 2024-09-10T09:55:02.167 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 24502, 2024-09-10T09:55:02.167 INFO:teuthology.orchestra.run.smithi077.stdout: "want_state": "up:active", 2024-09-10T09:55:02.167 INFO:teuthology.orchestra.run.smithi077.stdout: "state": "up:active", 2024-09-10T09:55:02.167 INFO:teuthology.orchestra.run.smithi077.stdout: "fs_name": "cephfs", 2024-09-10T09:55:02.167 INFO:teuthology.orchestra.run.smithi077.stdout: "rank_uptime": 107.305667767, 2024-09-10T09:55:02.167 INFO:teuthology.orchestra.run.smithi077.stdout: "mdsmap_epoch": 28, 2024-09-10T09:55:02.167 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch": 96, 2024-09-10T09:55:02.167 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch_barrier": 94, 2024-09-10T09:55:02.167 INFO:teuthology.orchestra.run.smithi077.stdout: "uptime": 187.17018257199999 2024-09-10T09:55:02.167 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-10T09:55:02.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.161+0000 7f6c04000640 1 -- 172.21.15.77:0/3921890368 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f6be40a2b80 msgr2=0x7f6be40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:02.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.161+0000 7f6c04000640 1 --2- 172.21.15.77:0/3921890368 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f6be40a2b80 0x7f6be40a5040 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6bf402ae70 tx=0x7f6bf4005d80 comp rx=0 tx=0).stop 2024-09-10T09:55:02.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.161+0000 7f6c04000640 1 -- 172.21.15.77:0/3921890368 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6bfc074f40 msgr2=0x7f6bfc19ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:02.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.161+0000 7f6c04000640 1 --2- 172.21.15.77:0/3921890368 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6bfc074f40 0x7f6bfc19ab60 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f6be800d950 tx=0x7f6be800de10 comp rx=0 tx=0).stop 2024-09-10T09:55:02.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.162+0000 7f6c04000640 1 -- 172.21.15.77:0/3921890368 shutdown_connections 2024-09-10T09:55:02.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.162+0000 7f6c04000640 1 --2- 172.21.15.77:0/3921890368 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f6be40a2b80 0x7f6be40a5040 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:02.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.162+0000 7f6c04000640 1 --2- 172.21.15.77:0/3921890368 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6bfc10e3c0 0x7f6bfc19eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:02.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.162+0000 7f6c04000640 1 --2- 172.21.15.77:0/3921890368 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6bfc074f40 0x7f6bfc19ab60 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:02.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.162+0000 7f6c04000640 1 --2- 172.21.15.77:0/3921890368 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6bfc074620 0x7f6bfc19a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:02.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.162+0000 7f6c04000640 1 -- 172.21.15.77:0/3921890368 >> 172.21.15.77:0/3921890368 conn(0x7f6bfc0fd8c0 msgr2=0x7f6bfc0fe520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:02.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.162+0000 7f6c04000640 1 -- 172.21.15.77:0/3921890368 shutdown_connections 2024-09-10T09:55:02.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:02.162+0000 7f6c04000640 1 -- 172.21.15.77:0/3921890368 wait complete. 2024-09-10T09:55:03.058 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.i status 2024-09-10T09:55:03.108 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:55:02 smithi077 ceph-mon[30319]: pgmap v387: 129 pgs: 129 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-10T09:55:03.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:55:02 smithi134 ceph-mon[35674]: pgmap v387: 129 pgs: 129 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-10T09:55:03.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:55:02 smithi120 ceph-mon[34770]: pgmap v387: 129 pgs: 129 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-10T09:55:03.428 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:55:03.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:03.995+0000 7f0589800640 1 -- 172.21.15.77:0/2325930224 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0584101de0 msgr2=0x7f05841021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:03.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:03.995+0000 7f0589800640 1 --2- 172.21.15.77:0/2325930224 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0584101de0 0x7f05841021c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f0574009900 tx=0x7f057402a410 comp rx=0 tx=0).stop 2024-09-10T09:55:03.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:03.995+0000 7f0589800640 1 -- 172.21.15.77:0/2325930224 shutdown_connections 2024-09-10T09:55:03.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:03.995+0000 7f0589800640 1 --2- 172.21.15.77:0/2325930224 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0584106f10 0x7f058410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:03.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:03.995+0000 7f0589800640 1 --2- 172.21.15.77:0/2325930224 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0584102790 0x7f05841067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:03.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:03.995+0000 7f0589800640 1 --2- 172.21.15.77:0/2325930224 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0584101de0 0x7f05841021c0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:03.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:03.995+0000 7f0589800640 1 -- 172.21.15.77:0/2325930224 >> 172.21.15.77:0/2325930224 conn(0x7f05840fd8c0 msgr2=0x7f05840ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:03.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:03.995+0000 7f0589800640 1 -- 172.21.15.77:0/2325930224 shutdown_connections 2024-09-10T09:55:03.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:03.995+0000 7f0589800640 1 -- 172.21.15.77:0/2325930224 wait complete. 2024-09-10T09:55:04.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:03.996+0000 7f0589800640 1 Processor -- start 2024-09-10T09:55:04.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:03.996+0000 7f0589800640 1 -- start start 2024-09-10T09:55:04.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:03.996+0000 7f0589800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0584101de0 0x7f0584198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:04.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:03.996+0000 7f0589800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0584102790 0x7f0584198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:04.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:03.996+0000 7f0589800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0584106f10 0x7f058419cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:04.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:03.996+0000 7f0589800640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0584111fc0 con 0x7f0584106f10 2024-09-10T09:55:04.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:03.996+0000 7f0589800640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0584111e40 con 0x7f0584102790 2024-09-10T09:55:04.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:03.996+0000 7f0589800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0584112140 con 0x7f0584101de0 2024-09-10T09:55:04.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:03.997+0000 7f0583400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0584106f10 0x7f058419cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:04.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:03.997+0000 7f0583400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0584106f10 0x7f058419cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57484/0 (socket says 172.21.15.77:57484) 2024-09-10T09:55:04.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:03.997+0000 7f0583400640 1 -- 172.21.15.77:0/89914809 learned_addr learned my addr 172.21.15.77:0/89914809 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:55:04.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:03.997+0000 7f0582a00640 1 --2- 172.21.15.77:0/89914809 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0584101de0 0x7f0584198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:04.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:03.997+0000 7f057a000640 1 --2- 172.21.15.77:0/89914809 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0584102790 0x7f0584198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:04.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:03.997+0000 7f0583400640 1 -- 172.21.15.77:0/89914809 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0584101de0 msgr2=0x7f0584198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:04.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:03.997+0000 7f0583400640 1 --2- 172.21.15.77:0/89914809 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0584101de0 0x7f0584198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:04.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:03.997+0000 7f0583400640 1 -- 172.21.15.77:0/89914809 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0584102790 msgr2=0x7f0584198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:04.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:03.997+0000 7f0583400640 1 --2- 172.21.15.77:0/89914809 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0584102790 0x7f0584198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:04.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:03.997+0000 7f0583400640 1 -- 172.21.15.77:0/89914809 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f058419d3f0 con 0x7f0584106f10 2024-09-10T09:55:04.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:03.998+0000 7f0583400640 1 --2- 172.21.15.77:0/89914809 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0584106f10 0x7f058419cd10 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f057c00eb10 tx=0x7f057c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:04.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:03.998+0000 7f057be00640 1 -- 172.21.15.77:0/89914809 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f057c004b20 con 0x7f0584106f10 2024-09-10T09:55:04.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:03.998+0000 7f057be00640 1 -- 172.21.15.77:0/89914809 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f057c004cc0 con 0x7f0584106f10 2024-09-10T09:55:04.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:03.998+0000 7f057be00640 1 -- 172.21.15.77:0/89914809 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f057c010760 con 0x7f0584106f10 2024-09-10T09:55:04.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:03.998+0000 7f0589800640 1 -- 172.21.15.77:0/89914809 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f058419d6e0 con 0x7f0584106f10 2024-09-10T09:55:04.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:03.998+0000 7f0589800640 1 -- 172.21.15.77:0/89914809 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05841a4fc0 con 0x7f0584106f10 2024-09-10T09:55:04.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.001+0000 7f057be00640 1 -- 172.21.15.77:0/89914809 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f057c005020 con 0x7f0584106f10 2024-09-10T09:55:04.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.001+0000 7f058c23f740 1 Processor -- start 2024-09-10T09:55:04.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.001+0000 7f058c23f740 1 -- start start 2024-09-10T09:55:04.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.002+0000 7f058c23f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56120eb84230 0x56120eb84610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:04.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.002+0000 7f058c23f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56120eb84b50 0x56120eb933e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:04.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.002+0000 7f058c23f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56120eb93920 0x56120eb95d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:04.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.002+0000 7f058c23f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56120eaea0b0 con 0x56120eb93920 2024-09-10T09:55:04.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.002+0000 7f058c23f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56120ea91ab0 con 0x56120eb84230 2024-09-10T09:55:04.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.002+0000 7f058c23f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56120eae6070 con 0x56120eb84b50 2024-09-10T09:55:04.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.002+0000 7f057a000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56120eb84230 0x56120eb84610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:04.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.002+0000 7f0582a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56120eb84b50 0x56120eb933e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:04.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.002+0000 7f0582a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56120eb84b50 0x56120eb933e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:39062/0 (socket says 172.21.15.77:39062) 2024-09-10T09:55:04.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.002+0000 7f057a000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56120eb84230 0x56120eb84610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:43114/0 (socket says 172.21.15.77:43114) 2024-09-10T09:55:04.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.002+0000 7f0582a00640 1 -- 172.21.15.77:0/2985723910 learned_addr learned my addr 172.21.15.77:0/2985723910 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:55:04.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.002+0000 7f057a000640 1 --2- 172.21.15.77:0/2985723910 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56120eb93920 0x56120eb95d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:04.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.002+0000 7f0582a00640 1 -- 172.21.15.77:0/2985723910 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56120eb84230 msgr2=0x56120eb84610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:04.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.002+0000 7f0582a00640 1 --2- 172.21.15.77:0/2985723910 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56120eb84230 0x56120eb84610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:04.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.003+0000 7f057be00640 1 --2- 172.21.15.77:0/89914809 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f05680a2c00 0x7f05680a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:04.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.003+0000 7f0582a00640 1 -- 172.21.15.77:0/2985723910 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56120eb93920 msgr2=0x56120eb95d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:04.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.003+0000 7f0582a00640 1 --2- 172.21.15.77:0/2985723910 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56120eb93920 0x56120eb95d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:04.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.003+0000 7f0582a00640 1 -- 172.21.15.77:0/2985723910 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56120ea9e4a0 con 0x56120eb84b50 2024-09-10T09:55:04.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.003+0000 7f057be00640 1 -- 172.21.15.77:0/89914809 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f057c0cfa40 con 0x7f0584106f10 2024-09-10T09:55:04.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.003+0000 7f0582a00640 1 --2- 172.21.15.77:0/2985723910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56120eb84b50 0x56120eb933e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0574009900 tx=0x7f0574005de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e862a797a84d495 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:04.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.003+0000 7f0573e00640 1 -- 172.21.15.77:0/2985723910 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0574035040 con 0x56120eb84b50 2024-09-10T09:55:04.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.003+0000 7f0583400640 1 --2- 172.21.15.77:0/89914809 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f05680a2c00 0x7f05680a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:04.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.003+0000 7f0573e00640 1 -- 172.21.15.77:0/2985723910 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0574004400 con 0x56120eb84b50 2024-09-10T09:55:04.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.003+0000 7f0573e00640 1 -- 172.21.15.77:0/2985723910 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05740035f0 con 0x56120eb84b50 2024-09-10T09:55:04.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.004+0000 7f058c23f740 1 -- 172.21.15.77:0/2985723910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56120eb84b50 msgr2=0x56120eb933e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:04.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.004+0000 7f058c23f740 1 --2- 172.21.15.77:0/2985723910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56120eb84b50 0x56120eb933e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0574009900 tx=0x7f0574005de0 comp rx=0 tx=0).stop 2024-09-10T09:55:04.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.004+0000 7f0583400640 1 --2- 172.21.15.77:0/89914809 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f05680a2c00 0x7f05680a50c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f057c000f80 tx=0x7f057c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:04.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.004+0000 7f058c23f740 1 -- 172.21.15.77:0/2985723910 shutdown_connections 2024-09-10T09:55:04.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.004+0000 7f058c23f740 1 --2- 172.21.15.77:0/2985723910 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56120eb93920 0x56120eb95d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:04.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.004+0000 7f058c23f740 1 --2- 172.21.15.77:0/2985723910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56120eb84b50 0x56120eb933e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:04.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.004+0000 7f058c23f740 1 --2- 172.21.15.77:0/2985723910 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56120eb84230 0x56120eb84610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:04.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.004+0000 7f058c23f740 1 -- 172.21.15.77:0/2985723910 >> 172.21.15.77:0/2985723910 conn(0x56120eb808c0 msgr2=0x56120eb80ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:04.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.004+0000 7f058c23f740 1 -- 172.21.15.77:0/2985723910 shutdown_connections 2024-09-10T09:55:04.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.004+0000 7f058c23f740 1 -- 172.21.15.77:0/2985723910 wait complete. 2024-09-10T09:55:04.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.005+0000 7f058c23f740 1 -- start start 2024-09-10T09:55:04.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.005+0000 7f058c23f740 1 Processor -- start 2024-09-10T09:55:04.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.006+0000 7f058c23f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56120eb84230 0x56120eb90880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:04.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.006+0000 7f058c23f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56120eb84b50 0x56120eb90dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:04.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.006+0000 7f058c23f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56120eb93920 0x56120eb91300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:04.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.006+0000 7f058c23f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56120eb96590 con 0x56120eb84230 2024-09-10T09:55:04.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.006+0000 7f058c23f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56120ea72920 con 0x56120eb84b50 2024-09-10T09:55:04.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.006+0000 7f058c23f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56120eb96710 con 0x56120eb93920 2024-09-10T09:55:04.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.006+0000 7f058c23f740 10 client.0 _ll_register_callbacks cb 0x56120ea968a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:55:04.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.006+0000 7f057a000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56120eb84230 0x56120eb90880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:04.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.006+0000 7f057a000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56120eb84230 0x56120eb90880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57492/0 (socket says 172.21.15.77:57492) 2024-09-10T09:55:04.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.006+0000 7f057a000640 1 -- 172.21.15.77:0/955439463 learned_addr learned my addr 172.21.15.77:0/955439463 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:55:04.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.006+0000 7f0582a00640 1 --2- 172.21.15.77:0/955439463 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56120eb84b50 0x56120eb90dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:04.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.006+0000 7f057a000640 1 --2- 172.21.15.77:0/955439463 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56120eb93920 0x56120eb91300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:04.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.006+0000 7f0582a00640 1 -- 172.21.15.77:0/955439463 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56120eb93920 msgr2=0x56120eb91300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:04.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.006+0000 7f0582a00640 1 --2- 172.21.15.77:0/955439463 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56120eb93920 0x56120eb91300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:04.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.006+0000 7f0582a00640 1 -- 172.21.15.77:0/955439463 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56120eb84230 msgr2=0x56120eb90880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:04.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.006+0000 7f0582a00640 1 --2- 172.21.15.77:0/955439463 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56120eb84230 0x56120eb90880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:04.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.006+0000 7f0582a00640 1 -- 172.21.15.77:0/955439463 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56120e85b2b0 con 0x56120eb84b50 2024-09-10T09:55:04.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.006+0000 7f057a000640 1 --2- 172.21.15.77:0/955439463 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56120eb84230 0x56120eb90880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:55:04.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.007+0000 7f0582a00640 1 --2- 172.21.15.77:0/955439463 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56120eb84b50 0x56120eb90dc0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f057402a930 tx=0x7f0574004a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36880f0f630a0437 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:04.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.007+0000 7f0570c00640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T09:55:04.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.007+0000 7f0570c00640 1 -- 172.21.15.77:0/955439463 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0574052060 con 0x56120eb84b50 2024-09-10T09:55:04.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.007+0000 7f058c23f740 1 -- 172.21.15.77:0/955439463 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0574004400 con 0x56120eb84b50 2024-09-10T09:55:04.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.007+0000 7f0570c00640 1 -- 172.21.15.77:0/955439463 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f0574004200 con 0x56120eb84b50 2024-09-10T09:55:04.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.007+0000 7f0570c00640 1 -- 172.21.15.77:0/955439463 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0574035070 con 0x56120eb84b50 2024-09-10T09:55:04.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.007+0000 7f058c23f740 10 client.25184 fetch_fsmap learned FSMap version 28 2024-09-10T09:55:04.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.007+0000 7f058c23f740 1 -- 172.21.15.77:0/955439463 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56120eba2300 con 0x56120eb84b50 2024-09-10T09:55:04.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.008+0000 7f0570c00640 1 -- 172.21.15.77:0/955439463 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0574004400 con 0x56120eb84b50 2024-09-10T09:55:04.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.008+0000 7f0570c00640 1 -- 172.21.15.77:0/955439463 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f05740318b0 con 0x56120eb84b50 2024-09-10T09:55:04.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.008+0000 7f058c23f740 10 client.25184 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T09:55:04.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.008+0000 7f058c23f740 10 client.25184 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-10T09:55:04.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.008+0000 7f058c23f740 20 client.25184 populate_metadata read hostname 'smithi077' 2024-09-10T09:55:04.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.008+0000 7f058c23f740 1 --2- 172.21.15.77:0/955439463 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x56120eba3650 0x56120eba5830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:04.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.008+0000 7f058c23f740 4 client.25184 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:55:04.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.008+0000 7f058c23f740 1 -- 172.21.15.77:0/955439463 --> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56120eaf7620 con 0x56120eba3650 2024-09-10T09:55:04.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.009+0000 7f057a000640 1 --2- 172.21.15.77:0/955439463 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x56120eba3650 0x56120eba5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:04.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.009+0000 7f0570c00640 10 client.25184 ms_handle_connect on v2:172.21.15.134:6838/1965117680 2024-09-10T09:55:04.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.009+0000 7f057a000640 1 --2- 172.21.15.77:0/955439463 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x56120eba3650 0x56120eba5830 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e3a0af6e063bbb6c server_cookie=8135afc2d277a5cd in_seq=0 out_seq=0 2024-09-10T09:55:04.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.010+0000 7f0570c00640 1 -- 172.21.15.77:0/955439463 <== mds.2 v2:172.21.15.134:6838/1965117680 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56120eaf7620 con 0x56120eba3650 2024-09-10T09:55:04.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.010+0000 7f0570c00640 10 client.25184 handle_command_reply: tid=0 2024-09-10T09:55:04.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.010+0000 7f058c23f740 1 client.25184 shutdown 2024-09-10T09:55:04.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.010+0000 7f058c23f740 1 -- 172.21.15.77:0/955439463 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56120eb84b50 msgr2=0x56120eb90dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:04.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.010+0000 7f058c23f740 1 --2- 172.21.15.77:0/955439463 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56120eb84b50 0x56120eb90dc0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f057402a930 tx=0x7f0574004a80 comp rx=0 tx=0).stop 2024-09-10T09:55:04.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.010+0000 7f058c23f740 1 -- 172.21.15.77:0/955439463 shutdown_connections 2024-09-10T09:55:04.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.010+0000 7f058c23f740 1 --2- 172.21.15.77:0/955439463 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x56120eba3650 0x56120eba5830 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:04.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.010+0000 7f058c23f740 1 --2- 172.21.15.77:0/955439463 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56120eb93920 0x56120eb91300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:04.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.010+0000 7f058c23f740 1 --2- 172.21.15.77:0/955439463 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56120eb84b50 0x56120eb90dc0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:04.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.010+0000 7f0570c00640 0 client.25184 ms_handle_reset on v2:172.21.15.134:6838/1965117680 2024-09-10T09:55:04.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.010+0000 7f058c23f740 1 --2- 172.21.15.77:0/955439463 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56120eb84230 0x56120eb90880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:04.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.010+0000 7f058c23f740 1 -- 172.21.15.77:0/955439463 >> 172.21.15.77:0/955439463 conn(0x56120eb808c0 msgr2=0x56120eb86620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:04.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.010+0000 7f058c23f740 1 -- 172.21.15.77:0/955439463 shutdown_connections 2024-09-10T09:55:04.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.010+0000 7f058c23f740 1 -- 172.21.15.77:0/955439463 wait complete. 2024-09-10T09:55:04.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.010+0000 7f058c23f740 20 client.25184 trim_cache size 0 max 16384 2024-09-10T09:55:04.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.029+0000 7f058c23f740 1 Processor -- start 2024-09-10T09:55:04.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.029+0000 7f058c23f740 1 -- start start 2024-09-10T09:55:04.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.029+0000 7f058c23f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56120eb84230 0x56120eba5830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:04.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.029+0000 7f058c23f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56120eb84b50 0x56120eb91150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:04.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.029+0000 7f058c23f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56120eb93920 0x56120eb927f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:04.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.029+0000 7f058c23f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56120eba3b70 con 0x56120eb84b50 2024-09-10T09:55:04.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.029+0000 7f058c23f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56120eba4240 con 0x56120eb84230 2024-09-10T09:55:04.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.029+0000 7f058c23f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56120eba4910 con 0x56120eb93920 2024-09-10T09:55:04.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.029+0000 7f0582a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56120eb84b50 0x56120eb91150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:04.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.030+0000 7f0582a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56120eb84b50 0x56120eb91150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57496/0 (socket says 172.21.15.77:57496) 2024-09-10T09:55:04.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.030+0000 7f0582a00640 1 -- 172.21.15.77:0/1598442077 learned_addr learned my addr 172.21.15.77:0/1598442077 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:55:04.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.030+0000 7f057a000640 1 --2- 172.21.15.77:0/1598442077 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56120eb93920 0x56120eb927f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:04.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.030+0000 7f057a000640 1 --2- 172.21.15.77:0/1598442077 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56120eb84230 0x56120eba5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:04.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.030+0000 7f0582a00640 1 -- 172.21.15.77:0/1598442077 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56120eb93920 msgr2=0x56120eb927f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:04.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.030+0000 7f0582a00640 1 --2- 172.21.15.77:0/1598442077 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56120eb93920 0x56120eb927f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:04.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.030+0000 7f0582a00640 1 -- 172.21.15.77:0/1598442077 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56120eb84230 msgr2=0x56120eba5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:04.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.030+0000 7f0582a00640 1 --2- 172.21.15.77:0/1598442077 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56120eb84230 0x56120eba5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:04.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.030+0000 7f0582a00640 1 -- 172.21.15.77:0/1598442077 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56120eb84980 con 0x56120eb84b50 2024-09-10T09:55:04.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.030+0000 7f057a000640 1 --2- 172.21.15.77:0/1598442077 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56120eb84230 0x56120eba5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:55:04.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.030+0000 7f0582a00640 1 --2- 172.21.15.77:0/1598442077 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56120eb84b50 0x56120eb91150 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f0574031e40 tx=0x7f0574001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b7380c22a48348a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:04.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.030+0000 7f0573400640 1 -- 172.21.15.77:0/1598442077 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0574052030 con 0x56120eb84b50 2024-09-10T09:55:04.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.030+0000 7f0573400640 1 -- 172.21.15.77:0/1598442077 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0574005850 con 0x56120eb84b50 2024-09-10T09:55:04.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.031+0000 7f0573400640 1 -- 172.21.15.77:0/1598442077 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0574003a60 con 0x56120eb84b50 2024-09-10T09:55:04.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.031+0000 7f058c23f740 1 -- 172.21.15.77:0/1598442077 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56120eb84b50 msgr2=0x56120eb91150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:04.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.031+0000 7f058c23f740 1 --2- 172.21.15.77:0/1598442077 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56120eb84b50 0x56120eb91150 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f0574031e40 tx=0x7f0574001e50 comp rx=0 tx=0).stop 2024-09-10T09:55:04.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.031+0000 7f058c23f740 1 -- 172.21.15.77:0/1598442077 shutdown_connections 2024-09-10T09:55:04.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.031+0000 7f058c23f740 1 --2- 172.21.15.77:0/1598442077 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56120eb93920 0x56120eb927f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:04.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.031+0000 7f058c23f740 1 --2- 172.21.15.77:0/1598442077 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56120eb84b50 0x56120eb91150 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:04.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.031+0000 7f058c23f740 1 --2- 172.21.15.77:0/1598442077 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56120eb84230 0x56120eba5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:04.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.031+0000 7f058c23f740 1 -- 172.21.15.77:0/1598442077 >> 172.21.15.77:0/1598442077 conn(0x56120eb808c0 msgr2=0x56120eb920e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:04.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.031+0000 7f058c23f740 1 -- 172.21.15.77:0/1598442077 shutdown_connections 2024-09-10T09:55:04.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.031+0000 7f058c23f740 1 -- 172.21.15.77:0/1598442077 wait complete. 2024-09-10T09:55:04.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.032+0000 7f058c23f740 1 -- start start 2024-09-10T09:55:04.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.032+0000 7f058c23f740 1 Processor -- start 2024-09-10T09:55:04.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.032+0000 7f058c23f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56120eb84230 0x56120ebadcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:04.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.032+0000 7f058c23f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56120eb84b50 0x56120ebae210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:04.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.032+0000 7f058c23f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56120eb93920 0x56120eb8d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:04.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.032+0000 7f058c23f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56120eba56b0 con 0x56120eb84230 2024-09-10T09:55:04.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.032+0000 7f058c23f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56120eba4fe0 con 0x56120eb93920 2024-09-10T09:55:04.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.032+0000 7f058c23f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56120eba5d80 con 0x56120eb84b50 2024-09-10T09:55:04.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.032+0000 7f058c23f740 10 client.0 _ll_register_callbacks cb 0x7f0574009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:55:04.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.033+0000 7f057a000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56120eb84230 0x56120ebadcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:04.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.033+0000 7f057a000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56120eb84230 0x56120ebadcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57504/0 (socket says 172.21.15.77:57504) 2024-09-10T09:55:04.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.033+0000 7f057a000640 1 -- 172.21.15.77:0/194288151 learned_addr learned my addr 172.21.15.77:0/194288151 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:55:04.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.033+0000 7f057a000640 1 --2- 172.21.15.77:0/194288151 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56120eb93920 0x56120eb8d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:04.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.033+0000 7f0582a00640 1 --2- 172.21.15.77:0/194288151 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56120eb84b50 0x56120ebae210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:04.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.033+0000 7f057a000640 1 -- 172.21.15.77:0/194288151 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56120eb84b50 msgr2=0x56120ebae210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:04.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.033+0000 7f057a000640 1 --2- 172.21.15.77:0/194288151 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56120eb84b50 0x56120ebae210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:04.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.033+0000 7f057a000640 1 -- 172.21.15.77:0/194288151 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56120eb93920 msgr2=0x56120eb8d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:04.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.033+0000 7f057a000640 1 --2- 172.21.15.77:0/194288151 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56120eb93920 0x56120eb8d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:04.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.033+0000 7f057a000640 1 -- 172.21.15.77:0/194288151 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56120eb8d9b0 con 0x56120eb84230 2024-09-10T09:55:04.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.033+0000 7f057a000640 1 --2- 172.21.15.77:0/194288151 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56120eb84230 0x56120ebadcd0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f056c004700 tx=0x7f056c009770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76a71cc2d007db52 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:04.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.033+0000 7f0570c00640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-10T09:55:04.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.034+0000 7f0570c00640 1 -- 172.21.15.77:0/194288151 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f056c003c80 con 0x56120eb84230 2024-09-10T09:55:04.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.034+0000 7f058c23f740 1 -- 172.21.15.77:0/194288151 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0574005850 con 0x56120eb84230 2024-09-10T09:55:04.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.034+0000 7f0570c00640 1 -- 172.21.15.77:0/194288151 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f056c009040 con 0x56120eb84230 2024-09-10T09:55:04.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.034+0000 7f0570c00640 1 -- 172.21.15.77:0/194288151 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f056c003e20 con 0x56120eb84230 2024-09-10T09:55:04.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.034+0000 7f058c23f740 10 client.15141 fetch_fsmap learned FSMap version 28 2024-09-10T09:55:04.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.034+0000 7f058c23f740 1 -- 172.21.15.77:0/194288151 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56120ebad550 con 0x56120eb84230 2024-09-10T09:55:04.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.034+0000 7f0570c00640 1 -- 172.21.15.77:0/194288151 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0574005850 con 0x56120eb84230 2024-09-10T09:55:04.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.034+0000 7f0570c00640 1 -- 172.21.15.77:0/194288151 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f056c02a040 con 0x56120eb84230 2024-09-10T09:55:04.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.034+0000 7f058c23f740 10 client.15141 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T09:55:04.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.034+0000 7f058c23f740 10 client.15141 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-10T09:55:04.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.034+0000 7f058c23f740 20 client.15141 populate_metadata read hostname 'smithi077' 2024-09-10T09:55:04.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.034+0000 7f058c23f740 1 --2- 172.21.15.77:0/194288151 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x56120ec131b0 0x56120ec11ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:04.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.035+0000 7f058c23f740 4 client.15141 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T09:55:04.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.035+0000 7f058c23f740 1 -- 172.21.15.77:0/194288151 --> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] -- command(tid 0: {"prefix": "status"}) -- 0x56120eaf1b30 con 0x56120ec131b0 2024-09-10T09:55:04.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.035+0000 7f0582a00640 1 --2- 172.21.15.77:0/194288151 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x56120ec131b0 0x56120ec11ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:04.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.035+0000 7f0582a00640 1 --2- 172.21.15.77:0/194288151 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x56120ec131b0 0x56120ec11ec0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e805f3ba9d74a38b server_cookie=e0c6b3e0a77f9918 in_seq=0 out_seq=0 2024-09-10T09:55:04.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.035+0000 7f0570c00640 10 client.15141 ms_handle_connect on v2:172.21.15.134:6838/1965117680 2024-09-10T09:55:04.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.036+0000 7f0570c00640 1 -- 172.21.15.77:0/194288151 <== mds.2 v2:172.21.15.134:6838/1965117680 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x56120eaf1b30 con 0x56120ec131b0 2024-09-10T09:55:04.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.036+0000 7f0570c00640 10 client.15141 handle_command_reply: tid=0 2024-09-10T09:55:04.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.036+0000 7f058c23f740 1 client.15141 shutdown 2024-09-10T09:55:04.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.036+0000 7f058c23f740 1 -- 172.21.15.77:0/194288151 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56120eb84230 msgr2=0x56120ebadcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:04.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.036+0000 7f058c23f740 1 --2- 172.21.15.77:0/194288151 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56120eb84230 0x56120ebadcd0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f056c004700 tx=0x7f056c009770 comp rx=0 tx=0).stop 2024-09-10T09:55:04.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.036+0000 7f058c23f740 1 -- 172.21.15.77:0/194288151 shutdown_connections 2024-09-10T09:55:04.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.036+0000 7f058c23f740 1 --2- 172.21.15.77:0/194288151 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x56120ec131b0 0x56120ec11ec0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:04.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.036+0000 7f058c23f740 1 --2- 172.21.15.77:0/194288151 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56120eb93920 0x56120eb8d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:04.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.036+0000 7f058c23f740 1 --2- 172.21.15.77:0/194288151 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56120eb84b50 0x56120ebae210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:04.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.036+0000 7f058c23f740 1 --2- 172.21.15.77:0/194288151 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56120eb84230 0x56120ebadcd0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:04.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.036+0000 7f058c23f740 1 -- 172.21.15.77:0/194288151 >> 172.21.15.77:0/194288151 conn(0x56120eb808c0 msgr2=0x56120eb85d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:04.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.036+0000 7f0570c00640 0 client.15141 ms_handle_reset on v2:172.21.15.134:6838/1965117680 2024-09-10T09:55:04.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.036+0000 7f058c23f740 1 -- 172.21.15.77:0/194288151 shutdown_connections 2024-09-10T09:55:04.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.036+0000 7f058c23f740 1 -- 172.21.15.77:0/194288151 wait complete. 2024-09-10T09:55:04.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.036+0000 7f058c23f740 20 client.15141 trim_cache size 0 max 16384 2024-09-10T09:55:04.043 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-10T09:55:04.043 INFO:teuthology.orchestra.run.smithi077.stdout: "cluster_fsid": "14a55226-6f59-11ef-bcea-c7b262605968", 2024-09-10T09:55:04.043 INFO:teuthology.orchestra.run.smithi077.stdout: "whoami": 2, 2024-09-10T09:55:04.043 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 24520, 2024-09-10T09:55:04.043 INFO:teuthology.orchestra.run.smithi077.stdout: "want_state": "up:active", 2024-09-10T09:55:04.043 INFO:teuthology.orchestra.run.smithi077.stdout: "state": "up:active", 2024-09-10T09:55:04.043 INFO:teuthology.orchestra.run.smithi077.stdout: "fs_name": "cephfs", 2024-09-10T09:55:04.043 INFO:teuthology.orchestra.run.smithi077.stdout: "rank_uptime": 110.187274926, 2024-09-10T09:55:04.043 INFO:teuthology.orchestra.run.smithi077.stdout: "mdsmap_epoch": 28, 2024-09-10T09:55:04.043 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch": 96, 2024-09-10T09:55:04.043 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch_barrier": 94, 2024-09-10T09:55:04.043 INFO:teuthology.orchestra.run.smithi077.stdout: "uptime": 183.63887100299999 2024-09-10T09:55:04.044 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-10T09:55:04.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.037+0000 7f0589800640 1 -- 172.21.15.77:0/89914809 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f05680a2c00 msgr2=0x7f05680a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:04.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.037+0000 7f0589800640 1 --2- 172.21.15.77:0/89914809 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f05680a2c00 0x7f05680a50c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f057c000f80 tx=0x7f057c0058d0 comp rx=0 tx=0).stop 2024-09-10T09:55:04.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.037+0000 7f0589800640 1 -- 172.21.15.77:0/89914809 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0584106f10 msgr2=0x7f058419cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:04.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.037+0000 7f0589800640 1 --2- 172.21.15.77:0/89914809 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0584106f10 0x7f058419cd10 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f057c00eb10 tx=0x7f057c00c2d0 comp rx=0 tx=0).stop 2024-09-10T09:55:04.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.037+0000 7f0589800640 1 -- 172.21.15.77:0/89914809 shutdown_connections 2024-09-10T09:55:04.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.037+0000 7f0589800640 1 --2- 172.21.15.77:0/89914809 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f05680a2c00 0x7f05680a50c0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:04.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.037+0000 7f0589800640 1 --2- 172.21.15.77:0/89914809 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0584106f10 0x7f058419cd10 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:04.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.037+0000 7f0589800640 1 --2- 172.21.15.77:0/89914809 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0584102790 0x7f0584198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:04.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.037+0000 7f0589800640 1 --2- 172.21.15.77:0/89914809 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0584101de0 0x7f0584198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:04.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.037+0000 7f0589800640 1 -- 172.21.15.77:0/89914809 >> 172.21.15.77:0/89914809 conn(0x7f05840fd8c0 msgr2=0x7f05840febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:04.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.037+0000 7f0589800640 1 -- 172.21.15.77:0/89914809 shutdown_connections 2024-09-10T09:55:04.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:04.037+0000 7f0589800640 1 -- 172.21.15.77:0/89914809 wait complete. 2024-09-10T09:55:04.856 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.h status 2024-09-10T09:55:05.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:55:04 smithi134 ceph-mon[35674]: pgmap v388: 129 pgs: 129 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-10T09:55:05.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:55:04 smithi120 ceph-mon[34770]: pgmap v388: 129 pgs: 129 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-10T09:55:05.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:55:04 smithi077 ceph-mon[30319]: pgmap v388: 129 pgs: 129 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-10T09:55:05.222 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:55:05.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.766+0000 7f5736a00640 1 -- 172.21.15.77:0/289570294 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f57300745c0 msgr2=0x7f57300749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:05.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.766+0000 7f5736a00640 1 --2- 172.21.15.77:0/289570294 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f57300745c0 0x7f57300749a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f5720009ca0 tx=0x7f572002a950 comp rx=0 tx=0).stop 2024-09-10T09:55:05.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.766+0000 7f5736a00640 1 -- 172.21.15.77:0/289570294 shutdown_connections 2024-09-10T09:55:05.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.766+0000 7f5736a00640 1 --2- 172.21.15.77:0/289570294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f573010e420 0x7f5730110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:05.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.766+0000 7f5736a00640 1 --2- 172.21.15.77:0/289570294 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5730074ee0 0x7f573010dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:05.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.766+0000 7f5736a00640 1 --2- 172.21.15.77:0/289570294 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f57300745c0 0x7f57300749a0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:05.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.766+0000 7f5736a00640 1 -- 172.21.15.77:0/289570294 >> 172.21.15.77:0/289570294 conn(0x7f57300fd8c0 msgr2=0x7f57300ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:05.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.766+0000 7f5736a00640 1 -- 172.21.15.77:0/289570294 shutdown_connections 2024-09-10T09:55:05.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.766+0000 7f5736a00640 1 -- 172.21.15.77:0/289570294 wait complete. 2024-09-10T09:55:05.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.767+0000 7f5736a00640 1 Processor -- start 2024-09-10T09:55:05.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.767+0000 7f5736a00640 1 -- start start 2024-09-10T09:55:05.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.767+0000 7f5736a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f57300745c0 0x7f573019a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:05.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.768+0000 7f5736a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5730074ee0 0x7f573019aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:05.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.768+0000 7f5736a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f573010e420 0x7f573019ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:05.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.768+0000 7f5736a00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f57301141c0 con 0x7f573010e420 2024-09-10T09:55:05.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.768+0000 7f5736a00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5730114040 con 0x7f57300745c0 2024-09-10T09:55:05.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.768+0000 7f5736a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5730114340 con 0x7f5730074ee0 2024-09-10T09:55:05.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.768+0000 7f572fe00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f57300745c0 0x7f573019a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:05.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.768+0000 7f572f400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5730074ee0 0x7f573019aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:05.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.768+0000 7f572fe00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f57300745c0 0x7f573019a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:43156/0 (socket says 172.21.15.77:43156) 2024-09-10T09:55:05.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.768+0000 7f572fe00640 1 -- 172.21.15.77:0/3211631808 learned_addr learned my addr 172.21.15.77:0/3211631808 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:55:05.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.768+0000 7f572f400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5730074ee0 0x7f573019aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:39106/0 (socket says 172.21.15.77:39106) 2024-09-10T09:55:05.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.768+0000 7f5734a00640 1 --2- 172.21.15.77:0/3211631808 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f573010e420 0x7f573019ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:05.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.768+0000 7f572f400640 1 -- 172.21.15.77:0/3211631808 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f57300745c0 msgr2=0x7f573019a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:05.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.768+0000 7f572f400640 1 --2- 172.21.15.77:0/3211631808 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f57300745c0 0x7f573019a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:05.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.768+0000 7f572f400640 1 -- 172.21.15.77:0/3211631808 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f573010e420 msgr2=0x7f573019ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:05.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.768+0000 7f572f400640 1 --2- 172.21.15.77:0/3211631808 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f573010e420 0x7f573019ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:05.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.768+0000 7f572f400640 1 -- 172.21.15.77:0/3211631808 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f573019f610 con 0x7f5730074ee0 2024-09-10T09:55:05.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.768+0000 7f572fe00640 1 --2- 172.21.15.77:0/3211631808 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f57300745c0 0x7f573019a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:55:05.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.769+0000 7f572f400640 1 --2- 172.21.15.77:0/3211631808 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5730074ee0 0x7f573019aba0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f571c00e9f0 tx=0x7f571c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:05.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.769+0000 7f572cc00640 1 -- 172.21.15.77:0/3211631808 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f571c00ccc0 con 0x7f5730074ee0 2024-09-10T09:55:05.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.769+0000 7f5736a00640 1 -- 172.21.15.77:0/3211631808 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f573019f900 con 0x7f5730074ee0 2024-09-10T09:55:05.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.769+0000 7f572cc00640 1 -- 172.21.15.77:0/3211631808 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f571c010910 con 0x7f5730074ee0 2024-09-10T09:55:05.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.769+0000 7f572cc00640 1 -- 172.21.15.77:0/3211631808 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f571c018960 con 0x7f5730074ee0 2024-09-10T09:55:05.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.769+0000 7f5736a00640 1 -- 172.21.15.77:0/3211631808 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57301a71e0 con 0x7f5730074ee0 2024-09-10T09:55:05.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.772+0000 7f57393d2740 1 Processor -- start 2024-09-10T09:55:05.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.772+0000 7f57393d2740 1 -- start start 2024-09-10T09:55:05.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.773+0000 7f57393d2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c62746330 0x556c62746710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:05.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.773+0000 7f57393d2740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556c62746c50 0x556c62750fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:05.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.773+0000 7f57393d2740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556c62751510 0x556c62753900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:05.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.773+0000 7f57393d2740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x556c626ac0b0 con 0x556c62746c50 2024-09-10T09:55:05.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.773+0000 7f57393d2740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x556c62653ab0 con 0x556c62751510 2024-09-10T09:55:05.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.773+0000 7f57393d2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556c626a8070 con 0x556c62746330 2024-09-10T09:55:05.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.773+0000 7f572cc00640 1 -- 172.21.15.77:0/3211631808 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f571c010ac0 con 0x7f5730074ee0 2024-09-10T09:55:05.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.773+0000 7f572fe00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556c62746c50 0x556c62750fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:05.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.773+0000 7f5734a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c62746330 0x556c62746710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:05.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.773+0000 7f5734a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c62746330 0x556c62746710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:39114/0 (socket says 172.21.15.77:39114) 2024-09-10T09:55:05.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.773+0000 7f572fe00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556c62746c50 0x556c62750fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57528/0 (socket says 172.21.15.77:57528) 2024-09-10T09:55:05.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.773+0000 7f5734a00640 1 -- 172.21.15.77:0/122601251 learned_addr learned my addr 172.21.15.77:0/122601251 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:55:05.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.773+0000 7f572f400640 1 --2- 172.21.15.77:0/122601251 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556c62751510 0x556c62753900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:05.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.774+0000 7f5734a00640 1 -- 172.21.15.77:0/122601251 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556c62751510 msgr2=0x556c62753900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:05.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.774+0000 7f5734a00640 1 --2- 172.21.15.77:0/122601251 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556c62751510 0x556c62753900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:05.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.774+0000 7f5734a00640 1 -- 172.21.15.77:0/122601251 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556c62746c50 msgr2=0x556c62750fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:05.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.774+0000 7f5734a00640 1 --2- 172.21.15.77:0/122601251 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556c62746c50 0x556c62750fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:05.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.774+0000 7f5734a00640 1 -- 172.21.15.77:0/122601251 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c626604a0 con 0x556c62746330 2024-09-10T09:55:05.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.774+0000 7f572fe00640 1 --2- 172.21.15.77:0/122601251 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556c62746c50 0x556c62750fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:55:05.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.774+0000 7f572f400640 1 --2- 172.21.15.77:0/122601251 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556c62751510 0x556c62753900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:55:05.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.774+0000 7f5734a00640 1 --2- 172.21.15.77:0/122601251 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c62746330 0x556c62746710 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f572800b600 tx=0x7f572800bac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c191bfb3893d9b6e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:05.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.774+0000 7f5725200640 1 -- 172.21.15.77:0/122601251 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5728003b20 con 0x556c62746330 2024-09-10T09:55:05.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.774+0000 7f5725200640 1 -- 172.21.15.77:0/122601251 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5728015070 con 0x556c62746330 2024-09-10T09:55:05.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.774+0000 7f5725200640 1 -- 172.21.15.77:0/122601251 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57280116a0 con 0x556c62746330 2024-09-10T09:55:05.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.775+0000 7f572cc00640 1 --2- 172.21.15.77:0/3211631808 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f57140a2c00 0x7f57140a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:05.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.775+0000 7f57393d2740 1 -- 172.21.15.77:0/122601251 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c62746330 msgr2=0x556c62746710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:05.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.775+0000 7f57393d2740 1 --2- 172.21.15.77:0/122601251 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c62746330 0x556c62746710 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f572800b600 tx=0x7f572800bac0 comp rx=0 tx=0).stop 2024-09-10T09:55:05.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.775+0000 7f572cc00640 1 -- 172.21.15.77:0/3211631808 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f571c0cf780 con 0x7f5730074ee0 2024-09-10T09:55:05.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.775+0000 7f572fe00640 1 --2- 172.21.15.77:0/3211631808 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f57140a2c00 0x7f57140a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:05.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.775+0000 7f57393d2740 1 -- 172.21.15.77:0/122601251 shutdown_connections 2024-09-10T09:55:05.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.775+0000 7f57393d2740 1 --2- 172.21.15.77:0/122601251 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556c62751510 0x556c62753900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:05.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.775+0000 7f57393d2740 1 --2- 172.21.15.77:0/122601251 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556c62746c50 0x556c62750fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:05.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.775+0000 7f57393d2740 1 --2- 172.21.15.77:0/122601251 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c62746330 0x556c62746710 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:05.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.775+0000 7f57393d2740 1 -- 172.21.15.77:0/122601251 >> 172.21.15.77:0/122601251 conn(0x556c627428c0 msgr2=0x556c62742ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:05.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.775+0000 7f57393d2740 1 -- 172.21.15.77:0/122601251 shutdown_connections 2024-09-10T09:55:05.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.775+0000 7f57393d2740 1 -- 172.21.15.77:0/122601251 wait complete. 2024-09-10T09:55:05.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.776+0000 7f57393d2740 1 -- start start 2024-09-10T09:55:05.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.776+0000 7f57393d2740 1 Processor -- start 2024-09-10T09:55:05.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.776+0000 7f572fe00640 1 --2- 172.21.15.77:0/3211631808 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f57140a2c00 0x7f57140a50c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x556c62743720 tx=0x7f5720005dc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:05.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.777+0000 7f57393d2740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556c62746330 0x556c6274e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:05.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.777+0000 7f57393d2740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556c62746c50 0x556c6274eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:05.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.777+0000 7f57393d2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c62751510 0x556c6274f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:05.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.777+0000 7f57393d2740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x556c62754060 con 0x556c62746330 2024-09-10T09:55:05.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.777+0000 7f57393d2740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x556c62634920 con 0x556c62746c50 2024-09-10T09:55:05.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.777+0000 7f57393d2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556c627541e0 con 0x556c62751510 2024-09-10T09:55:05.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.777+0000 7f57393d2740 10 client.0 _ll_register_callbacks cb 0x556c626588a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:55:05.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.777+0000 7f572fe00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556c62746330 0x556c6274e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:05.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.777+0000 7f572fe00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556c62746330 0x556c6274e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57544/0 (socket says 172.21.15.77:57544) 2024-09-10T09:55:05.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.777+0000 7f572fe00640 1 -- 172.21.15.77:0/2116006978 learned_addr learned my addr 172.21.15.77:0/2116006978 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:55:05.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.777+0000 7f572f400640 1 --2- 172.21.15.77:0/2116006978 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556c62746c50 0x556c6274eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:05.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.777+0000 7f5734a00640 1 --2- 172.21.15.77:0/2116006978 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c62751510 0x556c6274f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:05.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.777+0000 7f572f400640 1 -- 172.21.15.77:0/2116006978 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c62751510 msgr2=0x556c6274f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:05.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.777+0000 7f572f400640 1 --2- 172.21.15.77:0/2116006978 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c62751510 0x556c6274f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:05.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.777+0000 7f572f400640 1 -- 172.21.15.77:0/2116006978 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556c62746330 msgr2=0x556c6274e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:05.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.777+0000 7f572f400640 1 --2- 172.21.15.77:0/2116006978 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556c62746330 0x556c6274e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:05.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.777+0000 7f572f400640 1 -- 172.21.15.77:0/2116006978 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c6241d2b0 con 0x556c62746c50 2024-09-10T09:55:05.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.777+0000 7f572fe00640 1 --2- 172.21.15.77:0/2116006978 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556c62746330 0x556c6274e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:55:05.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.778+0000 7f572f400640 1 --2- 172.21.15.77:0/2116006978 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556c62746c50 0x556c6274eaf0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x556c62743c40 tx=0x7f571c09bb00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e18cc53c3ba270d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:05.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.778+0000 7f5719600640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T09:55:05.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.778+0000 7f5719600640 1 -- 172.21.15.77:0/2116006978 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f571c09ff40 con 0x556c62746c50 2024-09-10T09:55:05.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.778+0000 7f57393d2740 1 -- 172.21.15.77:0/2116006978 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5728015070 con 0x556c62746c50 2024-09-10T09:55:05.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.778+0000 7f5719600640 1 -- 172.21.15.77:0/2116006978 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f571c0cf430 con 0x556c62746c50 2024-09-10T09:55:05.784 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.778+0000 7f5719600640 1 -- 172.21.15.77:0/2116006978 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f571c0a4800 con 0x556c62746c50 2024-09-10T09:55:05.784 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.778+0000 7f57393d2740 10 client.25202 fetch_fsmap learned FSMap version 28 2024-09-10T09:55:05.784 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.778+0000 7f57393d2740 1 -- 172.21.15.77:0/2116006978 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556c6275fe90 con 0x556c62746c50 2024-09-10T09:55:05.784 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.779+0000 7f5719600640 1 -- 172.21.15.77:0/2116006978 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5728015070 con 0x556c62746c50 2024-09-10T09:55:05.784 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.779+0000 7f5719600640 1 -- 172.21.15.77:0/2116006978 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f571c0ce070 con 0x556c62746c50 2024-09-10T09:55:05.784 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.779+0000 7f57393d2740 10 client.25202 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T09:55:05.784 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.779+0000 7f57393d2740 10 client.25202 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-10T09:55:05.784 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.779+0000 7f57393d2740 20 client.25202 populate_metadata read hostname 'smithi077' 2024-09-10T09:55:05.784 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.779+0000 7f57393d2740 1 --2- 172.21.15.77:0/2116006978 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x556c62762470 0x556c627638e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:05.784 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.779+0000 7f57393d2740 4 client.25202 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:55:05.784 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.779+0000 7f57393d2740 1 -- 172.21.15.77:0/2116006978 --> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556c626b9620 con 0x556c62762470 2024-09-10T09:55:05.784 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.779+0000 7f572fe00640 1 --2- 172.21.15.77:0/2116006978 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x556c62762470 0x556c627638e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:05.784 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.780+0000 7f572fe00640 1 --2- 172.21.15.77:0/2116006978 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x556c62762470 0x556c627638e0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a26a7b6e6ff3ab95 server_cookie=4160c97c6d15fb80 in_seq=0 out_seq=0 2024-09-10T09:55:05.785 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.780+0000 7f5719600640 10 client.25202 ms_handle_connect on v2:172.21.15.120:6836/3033280709 2024-09-10T09:55:05.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.781+0000 7f5719600640 1 -- 172.21.15.77:0/2116006978 <== mds.1 v2:172.21.15.120:6836/3033280709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556c626b9620 con 0x556c62762470 2024-09-10T09:55:05.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.781+0000 7f5719600640 10 client.25202 handle_command_reply: tid=0 2024-09-10T09:55:05.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.781+0000 7f57393d2740 1 client.25202 shutdown 2024-09-10T09:55:05.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.781+0000 7f57393d2740 1 -- 172.21.15.77:0/2116006978 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556c62746c50 msgr2=0x556c6274eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:05.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.781+0000 7f57393d2740 1 --2- 172.21.15.77:0/2116006978 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556c62746c50 0x556c6274eaf0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x556c62743c40 tx=0x7f571c09bb00 comp rx=0 tx=0).stop 2024-09-10T09:55:05.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.781+0000 7f57393d2740 1 -- 172.21.15.77:0/2116006978 shutdown_connections 2024-09-10T09:55:05.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.781+0000 7f57393d2740 1 --2- 172.21.15.77:0/2116006978 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x556c62762470 0x556c627638e0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:05.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.781+0000 7f57393d2740 1 --2- 172.21.15.77:0/2116006978 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c62751510 0x556c6274f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:05.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.781+0000 7f57393d2740 1 --2- 172.21.15.77:0/2116006978 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556c62746c50 0x556c6274eaf0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:05.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.781+0000 7f57393d2740 1 --2- 172.21.15.77:0/2116006978 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556c62746330 0x556c6274e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:05.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.781+0000 7f5719600640 0 client.25202 ms_handle_reset on v2:172.21.15.120:6836/3033280709 2024-09-10T09:55:05.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.781+0000 7f57393d2740 1 -- 172.21.15.77:0/2116006978 >> 172.21.15.77:0/2116006978 conn(0x556c627428c0 msgr2=0x556c627485f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:05.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.781+0000 7f57393d2740 1 -- 172.21.15.77:0/2116006978 shutdown_connections 2024-09-10T09:55:05.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.781+0000 7f57393d2740 1 -- 172.21.15.77:0/2116006978 wait complete. 2024-09-10T09:55:05.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.782+0000 7f57393d2740 20 client.25202 trim_cache size 0 max 16384 2024-09-10T09:55:05.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.800+0000 7f57393d2740 1 Processor -- start 2024-09-10T09:55:05.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.800+0000 7f57393d2740 1 -- start start 2024-09-10T09:55:05.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.800+0000 7f57393d2740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556c62746330 0x556c62742210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:05.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.800+0000 7f57393d2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c62746c50 0x556c62748de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:05.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.800+0000 7f57393d2740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556c62751510 0x556c6274b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:05.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.800+0000 7f57393d2740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x556c62761c20 con 0x556c62746330 2024-09-10T09:55:05.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.800+0000 7f57393d2740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x556c627622f0 con 0x556c62751510 2024-09-10T09:55:05.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.800+0000 7f57393d2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556c627629c0 con 0x556c62746c50 2024-09-10T09:55:05.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.801+0000 7f572fe00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556c62746330 0x556c62742210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:05.804 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.801+0000 7f572fe00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556c62746330 0x556c62742210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57556/0 (socket says 172.21.15.77:57556) 2024-09-10T09:55:05.804 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.801+0000 7f572fe00640 1 -- 172.21.15.77:0/2291300726 learned_addr learned my addr 172.21.15.77:0/2291300726 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:55:05.804 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.801+0000 7f5734a00640 1 --2- 172.21.15.77:0/2291300726 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556c62751510 0x556c6274b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:05.804 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.801+0000 7f572f400640 1 --2- 172.21.15.77:0/2291300726 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c62746c50 0x556c62748de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:05.804 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.801+0000 7f572fe00640 1 -- 172.21.15.77:0/2291300726 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c62746c50 msgr2=0x556c62748de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:05.804 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.801+0000 7f572fe00640 1 --2- 172.21.15.77:0/2291300726 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c62746c50 0x556c62748de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:05.804 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.801+0000 7f572fe00640 1 -- 172.21.15.77:0/2291300726 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556c62751510 msgr2=0x556c6274b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:05.804 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.801+0000 7f572fe00640 1 --2- 172.21.15.77:0/2291300726 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556c62751510 0x556c6274b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:05.804 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.801+0000 7f572fe00640 1 -- 172.21.15.77:0/2291300726 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c6274cb40 con 0x556c62746330 2024-09-10T09:55:05.804 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.801+0000 7f572fe00640 1 --2- 172.21.15.77:0/2291300726 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556c62746330 0x556c62742210 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f5720005e30 tx=0x7f5720043000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=508b82a7ac30d422 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:05.804 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.802+0000 7f571be00640 1 -- 172.21.15.77:0/2291300726 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5720034040 con 0x556c62746330 2024-09-10T09:55:05.804 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.802+0000 7f571be00640 1 -- 172.21.15.77:0/2291300726 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5720046bd0 con 0x556c62746330 2024-09-10T09:55:05.804 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.802+0000 7f571be00640 1 -- 172.21.15.77:0/2291300726 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f572003a690 con 0x556c62746330 2024-09-10T09:55:05.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.802+0000 7f57393d2740 1 -- 172.21.15.77:0/2291300726 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556c62746330 msgr2=0x556c62742210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:05.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.802+0000 7f57393d2740 1 --2- 172.21.15.77:0/2291300726 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556c62746330 0x556c62742210 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f5720005e30 tx=0x7f5720043000 comp rx=0 tx=0).stop 2024-09-10T09:55:05.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.802+0000 7f57393d2740 1 -- 172.21.15.77:0/2291300726 shutdown_connections 2024-09-10T09:55:05.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.802+0000 7f57393d2740 1 --2- 172.21.15.77:0/2291300726 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556c62751510 0x556c6274b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:05.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.802+0000 7f57393d2740 1 --2- 172.21.15.77:0/2291300726 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c62746c50 0x556c62748de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:05.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.802+0000 7f57393d2740 1 --2- 172.21.15.77:0/2291300726 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556c62746330 0x556c62742210 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:05.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.802+0000 7f57393d2740 1 -- 172.21.15.77:0/2291300726 >> 172.21.15.77:0/2291300726 conn(0x556c627428c0 msgr2=0x556c62742ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:05.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.803+0000 7f57393d2740 1 -- 172.21.15.77:0/2291300726 shutdown_connections 2024-09-10T09:55:05.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.803+0000 7f57393d2740 1 -- 172.21.15.77:0/2291300726 wait complete. 2024-09-10T09:55:05.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.803+0000 7f57393d2740 1 -- start start 2024-09-10T09:55:05.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.803+0000 7f57393d2740 1 Processor -- start 2024-09-10T09:55:05.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.804+0000 7f57393d2740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556c62746330 0x556c6276bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:05.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.804+0000 7f57393d2740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556c62746c50 0x556c6276c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:05.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.804+0000 7f57393d2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c62751510 0x556c62766a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:05.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.804+0000 7f57393d2740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x556c62763760 con 0x556c62746330 2024-09-10T09:55:05.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.804+0000 7f57393d2740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x556c62763090 con 0x556c62746c50 2024-09-10T09:55:05.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.804+0000 7f57393d2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556c62763e30 con 0x556c62751510 2024-09-10T09:55:05.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.804+0000 7f57393d2740 10 client.0 _ll_register_callbacks cb 0x7f571c097fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:55:05.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.804+0000 7f572fe00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556c62746330 0x556c6276bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:05.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.804+0000 7f572f400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556c62746c50 0x556c6276c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:05.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.804+0000 7f572fe00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556c62746330 0x556c6276bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57562/0 (socket says 172.21.15.77:57562) 2024-09-10T09:55:05.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.804+0000 7f572fe00640 1 -- 172.21.15.77:0/1629704565 learned_addr learned my addr 172.21.15.77:0/1629704565 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:55:05.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.804+0000 7f5734a00640 1 --2- 172.21.15.77:0/1629704565 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c62751510 0x556c62766a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:05.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.804+0000 7f572fe00640 1 -- 172.21.15.77:0/1629704565 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c62751510 msgr2=0x556c62766a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:05.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.804+0000 7f572fe00640 1 --2- 172.21.15.77:0/1629704565 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c62751510 0x556c62766a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:05.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.804+0000 7f572fe00640 1 -- 172.21.15.77:0/1629704565 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556c62746c50 msgr2=0x556c6276c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:05.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.804+0000 7f572fe00640 1 --2- 172.21.15.77:0/1629704565 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556c62746c50 0x556c6276c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:05.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.804+0000 7f572fe00640 1 -- 172.21.15.77:0/1629704565 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c627670f0 con 0x556c62746330 2024-09-10T09:55:05.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.804+0000 7f572f400640 1 --2- 172.21.15.77:0/1629704565 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556c62746c50 0x556c6276c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:55:05.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.805+0000 7f572fe00640 1 --2- 172.21.15.77:0/1629704565 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556c62746330 0x556c6276bc30 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f5720000f80 tx=0x7f572002a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65de4033ad1a9a1f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:05.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.805+0000 7f5719600640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-10T09:55:05.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.805+0000 7f5719600640 1 -- 172.21.15.77:0/1629704565 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5720063030 con 0x556c62746330 2024-09-10T09:55:05.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.805+0000 7f5719600640 1 -- 172.21.15.77:0/1629704565 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f57200469d0 con 0x556c62746330 2024-09-10T09:55:05.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.805+0000 7f57393d2740 1 -- 172.21.15.77:0/1629704565 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5720046bd0 con 0x556c62746330 2024-09-10T09:55:05.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.805+0000 7f5719600640 1 -- 172.21.15.77:0/1629704565 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5720070050 con 0x556c62746330 2024-09-10T09:55:05.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.805+0000 7f57393d2740 10 client.15159 fetch_fsmap learned FSMap version 28 2024-09-10T09:55:05.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.805+0000 7f57393d2740 1 -- 172.21.15.77:0/1629704565 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556c627cd520 con 0x556c62746330 2024-09-10T09:55:05.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.806+0000 7f5719600640 1 -- 172.21.15.77:0/1629704565 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5720046bd0 con 0x556c62746330 2024-09-10T09:55:05.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.806+0000 7f5719600640 1 -- 172.21.15.77:0/1629704565 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f572005cb60 con 0x556c62746330 2024-09-10T09:55:05.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.806+0000 7f57393d2740 10 client.15159 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T09:55:05.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.806+0000 7f57393d2740 10 client.15159 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-10T09:55:05.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.806+0000 7f57393d2740 20 client.15159 populate_metadata read hostname 'smithi077' 2024-09-10T09:55:05.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.806+0000 7f57393d2740 1 --2- 172.21.15.77:0/1629704565 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x556c627d1c00 0x556c627d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:05.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.806+0000 7f57393d2740 4 client.15159 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T09:55:05.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.806+0000 7f57393d2740 1 -- 172.21.15.77:0/1629704565 --> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] -- command(tid 0: {"prefix": "status"}) -- 0x556c626b3b30 con 0x556c627d1c00 2024-09-10T09:55:05.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.806+0000 7f572f400640 1 --2- 172.21.15.77:0/1629704565 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x556c627d1c00 0x556c627d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:05.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.807+0000 7f572f400640 1 --2- 172.21.15.77:0/1629704565 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x556c627d1c00 0x556c627d0910 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=37723fb27d9f7d45 server_cookie=6ce3db92933b27c9 in_seq=0 out_seq=0 2024-09-10T09:55:05.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.807+0000 7f5719600640 10 client.15159 ms_handle_connect on v2:172.21.15.120:6836/3033280709 2024-09-10T09:55:05.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.807+0000 7f5719600640 1 -- 172.21.15.77:0/1629704565 <== mds.1 v2:172.21.15.120:6836/3033280709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x556c626b3b30 con 0x556c627d1c00 2024-09-10T09:55:05.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.807+0000 7f5719600640 10 client.15159 handle_command_reply: tid=0 2024-09-10T09:55:05.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.807+0000 7f57393d2740 1 client.15159 shutdown 2024-09-10T09:55:05.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.807+0000 7f57393d2740 1 -- 172.21.15.77:0/1629704565 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556c62746330 msgr2=0x556c6276bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:05.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.807+0000 7f57393d2740 1 --2- 172.21.15.77:0/1629704565 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556c62746330 0x556c6276bc30 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f5720000f80 tx=0x7f572002a950 comp rx=0 tx=0).stop 2024-09-10T09:55:05.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.807+0000 7f57393d2740 1 -- 172.21.15.77:0/1629704565 shutdown_connections 2024-09-10T09:55:05.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.807+0000 7f57393d2740 1 --2- 172.21.15.77:0/1629704565 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x556c627d1c00 0x556c627d0910 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:05.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.807+0000 7f57393d2740 1 --2- 172.21.15.77:0/1629704565 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c62751510 0x556c62766a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:05.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.807+0000 7f57393d2740 1 --2- 172.21.15.77:0/1629704565 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556c62746c50 0x556c6276c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:05.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.807+0000 7f5719600640 0 client.15159 ms_handle_reset on v2:172.21.15.120:6836/3033280709 2024-09-10T09:55:05.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.807+0000 7f57393d2740 1 --2- 172.21.15.77:0/1629704565 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556c62746330 0x556c6276bc30 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:05.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.807+0000 7f57393d2740 1 -- 172.21.15.77:0/1629704565 >> 172.21.15.77:0/1629704565 conn(0x556c627428c0 msgr2=0x556c6275c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:05.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.808+0000 7f57393d2740 1 -- 172.21.15.77:0/1629704565 shutdown_connections 2024-09-10T09:55:05.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.808+0000 7f57393d2740 1 -- 172.21.15.77:0/1629704565 wait complete. 2024-09-10T09:55:05.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.808+0000 7f57393d2740 20 client.15159 trim_cache size 0 max 16384 2024-09-10T09:55:05.813 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-10T09:55:05.813 INFO:teuthology.orchestra.run.smithi077.stdout: "cluster_fsid": "14a55226-6f59-11ef-bcea-c7b262605968", 2024-09-10T09:55:05.813 INFO:teuthology.orchestra.run.smithi077.stdout: "whoami": 1, 2024-09-10T09:55:05.813 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 24524, 2024-09-10T09:55:05.813 INFO:teuthology.orchestra.run.smithi077.stdout: "want_state": "up:active", 2024-09-10T09:55:05.813 INFO:teuthology.orchestra.run.smithi077.stdout: "state": "up:active", 2024-09-10T09:55:05.813 INFO:teuthology.orchestra.run.smithi077.stdout: "fs_name": "cephfs", 2024-09-10T09:55:05.813 INFO:teuthology.orchestra.run.smithi077.stdout: "rank_uptime": 112.97026513500001, 2024-09-10T09:55:05.813 INFO:teuthology.orchestra.run.smithi077.stdout: "mdsmap_epoch": 28, 2024-09-10T09:55:05.814 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch": 96, 2024-09-10T09:55:05.814 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch_barrier": 94, 2024-09-10T09:55:05.814 INFO:teuthology.orchestra.run.smithi077.stdout: "uptime": 188.10142235800001 2024-09-10T09:55:05.814 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-10T09:55:05.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.808+0000 7f5736a00640 1 -- 172.21.15.77:0/3211631808 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f57140a2c00 msgr2=0x7f57140a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:05.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.808+0000 7f5736a00640 1 --2- 172.21.15.77:0/3211631808 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f57140a2c00 0x7f57140a50c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x556c62743720 tx=0x7f5720005dc0 comp rx=0 tx=0).stop 2024-09-10T09:55:05.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.809+0000 7f5736a00640 1 -- 172.21.15.77:0/3211631808 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5730074ee0 msgr2=0x7f573019aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:05.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.809+0000 7f5736a00640 1 --2- 172.21.15.77:0/3211631808 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5730074ee0 0x7f573019aba0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f571c00e9f0 tx=0x7f571c00eeb0 comp rx=0 tx=0).stop 2024-09-10T09:55:05.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.809+0000 7f5736a00640 1 -- 172.21.15.77:0/3211631808 shutdown_connections 2024-09-10T09:55:05.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.809+0000 7f5736a00640 1 --2- 172.21.15.77:0/3211631808 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f57140a2c00 0x7f57140a50c0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:05.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.809+0000 7f5736a00640 1 --2- 172.21.15.77:0/3211631808 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f573010e420 0x7f573019ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:05.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.809+0000 7f5736a00640 1 --2- 172.21.15.77:0/3211631808 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5730074ee0 0x7f573019aba0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:05.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.809+0000 7f5736a00640 1 --2- 172.21.15.77:0/3211631808 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f57300745c0 0x7f573019a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:05.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.809+0000 7f5736a00640 1 -- 172.21.15.77:0/3211631808 >> 172.21.15.77:0/3211631808 conn(0x7f57300fd8c0 msgr2=0x7f57300fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:05.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.809+0000 7f5736a00640 1 -- 172.21.15.77:0/3211631808 shutdown_connections 2024-09-10T09:55:05.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:05.809+0000 7f5736a00640 1 -- 172.21.15.77:0/3211631808 wait complete. 2024-09-10T09:55:06.657 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-10T09:55:06.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:55:06 smithi077 ceph-mon[30319]: pgmap v389: 129 pgs: 129 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T09:55:07.028 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:55:07.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:55:06 smithi134 ceph-mon[35674]: pgmap v389: 129 pgs: 129 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T09:55:07.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:55:06 smithi120 ceph-mon[34770]: pgmap v389: 129 pgs: 129 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T09:55:07.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.564+0000 7f9c70000640 1 -- 172.21.15.77:0/2547623253 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9c68106f10 msgr2=0x7f9c6810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:07.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.564+0000 7f9c70000640 1 --2- 172.21.15.77:0/2547623253 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9c68106f10 0x7f9c6810ea90 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f9c60009ca0 tx=0x7f9c6002a8c0 comp rx=0 tx=0).stop 2024-09-10T09:55:07.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.565+0000 7f9c70000640 1 -- 172.21.15.77:0/2547623253 shutdown_connections 2024-09-10T09:55:07.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.565+0000 7f9c70000640 1 --2- 172.21.15.77:0/2547623253 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9c68106f10 0x7f9c6810ea90 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:07.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.565+0000 7f9c70000640 1 --2- 172.21.15.77:0/2547623253 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c68102790 0x7f9c681067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:07.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.565+0000 7f9c70000640 1 --2- 172.21.15.77:0/2547623253 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9c68101de0 0x7f9c681021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:07.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.565+0000 7f9c70000640 1 -- 172.21.15.77:0/2547623253 >> 172.21.15.77:0/2547623253 conn(0x7f9c680fd8c0 msgr2=0x7f9c680ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:07.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.565+0000 7f9c70000640 1 -- 172.21.15.77:0/2547623253 shutdown_connections 2024-09-10T09:55:07.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.565+0000 7f9c70000640 1 -- 172.21.15.77:0/2547623253 wait complete. 2024-09-10T09:55:07.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.566+0000 7f9c70000640 1 Processor -- start 2024-09-10T09:55:07.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.566+0000 7f9c70000640 1 -- start start 2024-09-10T09:55:07.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.566+0000 7f9c70000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9c68101de0 0x7f9c6819a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:07.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.566+0000 7f9c70000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9c68102790 0x7f9c6819ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:07.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.566+0000 7f9c70000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c68106f10 0x7f9c6819ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:07.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.566+0000 7f9c70000640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9c68111f70 con 0x7f9c68102790 2024-09-10T09:55:07.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.566+0000 7f9c70000640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9c68111df0 con 0x7f9c68101de0 2024-09-10T09:55:07.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.566+0000 7f9c70000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9c681120f0 con 0x7f9c68106f10 2024-09-10T09:55:07.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.566+0000 7f9c6cc00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9c68102790 0x7f9c6819ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:07.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.566+0000 7f9c6cc00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9c68102790 0x7f9c6819ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57594/0 (socket says 172.21.15.77:57594) 2024-09-10T09:55:07.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.566+0000 7f9c6cc00640 1 -- 172.21.15.77:0/3538640058 learned_addr learned my addr 172.21.15.77:0/3538640058 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:55:07.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.566+0000 7f9c6d600640 1 --2- 172.21.15.77:0/3538640058 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9c68101de0 0x7f9c6819a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:07.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.566+0000 7f9c6e000640 1 --2- 172.21.15.77:0/3538640058 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c68106f10 0x7f9c6819ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:07.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.567+0000 7f9c6cc00640 1 -- 172.21.15.77:0/3538640058 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c68106f10 msgr2=0x7f9c6819ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:07.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.567+0000 7f9c6cc00640 1 --2- 172.21.15.77:0/3538640058 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c68106f10 0x7f9c6819ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:07.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.567+0000 7f9c6cc00640 1 -- 172.21.15.77:0/3538640058 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9c68101de0 msgr2=0x7f9c6819a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:07.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.567+0000 7f9c6cc00640 1 --2- 172.21.15.77:0/3538640058 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9c68101de0 0x7f9c6819a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:07.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.567+0000 7f9c6cc00640 1 -- 172.21.15.77:0/3538640058 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c6819f600 con 0x7f9c68102790 2024-09-10T09:55:07.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.567+0000 7f9c6cc00640 1 --2- 172.21.15.77:0/3538640058 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9c68102790 0x7f9c6819ab90 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9c5400e9f0 tx=0x7f9c5400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:07.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.567+0000 7f9c66000640 1 -- 172.21.15.77:0/3538640058 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c5400cd80 con 0x7f9c68102790 2024-09-10T09:55:07.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.567+0000 7f9c66000640 1 -- 172.21.15.77:0/3538640058 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c54005500 con 0x7f9c68102790 2024-09-10T09:55:07.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.567+0000 7f9c66000640 1 -- 172.21.15.77:0/3538640058 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c54010690 con 0x7f9c68102790 2024-09-10T09:55:07.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.567+0000 7f9c70000640 1 -- 172.21.15.77:0/3538640058 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c6819f8f0 con 0x7f9c68102790 2024-09-10T09:55:07.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.568+0000 7f9c70000640 1 -- 172.21.15.77:0/3538640058 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c681a71d0 con 0x7f9c68102790 2024-09-10T09:55:07.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.571+0000 7f9c66000640 1 -- 172.21.15.77:0/3538640058 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9c540046c0 con 0x7f9c68102790 2024-09-10T09:55:07.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.571+0000 7f9c72bdc740 1 Processor -- start 2024-09-10T09:55:07.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.572+0000 7f9c72bdc740 1 -- start start 2024-09-10T09:55:07.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.572+0000 7f9c72bdc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55719ea68330 0x55719ea68710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:07.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.572+0000 7f9c72bdc740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55719ea68c50 0x55719ea72fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:07.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.572+0000 7f9c72bdc740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55719ea73510 0x55719ea75900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:07.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.572+0000 7f9c72bdc740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55719e9ce0b0 con 0x55719ea68c50 2024-09-10T09:55:07.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.572+0000 7f9c72bdc740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55719e975ab0 con 0x55719ea73510 2024-09-10T09:55:07.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.572+0000 7f9c72bdc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55719e9ca070 con 0x55719ea68330 2024-09-10T09:55:07.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.572+0000 7f9c6e000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55719ea68330 0x55719ea68710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:07.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.572+0000 7f9c6e000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55719ea68330 0x55719ea68710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:39158/0 (socket says 172.21.15.77:39158) 2024-09-10T09:55:07.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.572+0000 7f9c6e000640 1 -- 172.21.15.77:0/599045519 learned_addr learned my addr 172.21.15.77:0/599045519 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:55:07.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.572+0000 7f9c6cc00640 1 --2- 172.21.15.77:0/599045519 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55719ea73510 0x55719ea75900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:07.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.572+0000 7f9c6d600640 1 --2- 172.21.15.77:0/599045519 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55719ea68c50 0x55719ea72fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:07.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.573+0000 7f9c6e000640 1 -- 172.21.15.77:0/599045519 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55719ea73510 msgr2=0x55719ea75900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:07.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.573+0000 7f9c6e000640 1 --2- 172.21.15.77:0/599045519 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55719ea73510 0x55719ea75900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:07.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.573+0000 7f9c6e000640 1 -- 172.21.15.77:0/599045519 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55719ea68c50 msgr2=0x55719ea72fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:07.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.573+0000 7f9c6e000640 1 --2- 172.21.15.77:0/599045519 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55719ea68c50 0x55719ea72fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:07.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.573+0000 7f9c6e000640 1 -- 172.21.15.77:0/599045519 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55719e9824a0 con 0x55719ea68330 2024-09-10T09:55:07.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.573+0000 7f9c66000640 1 --2- 172.21.15.77:0/3538640058 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f9c500a2b80 0x7f9c500a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:07.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.573+0000 7f9c6e000640 1 --2- 172.21.15.77:0/599045519 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55719ea68330 0x55719ea68710 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9c60009ca0 tx=0x7f9c60009800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87491a24038450b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:07.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.573+0000 7f9c66000640 1 -- 172.21.15.77:0/3538640058 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f9c540cf830 con 0x7f9c68102790 2024-09-10T09:55:07.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.573+0000 7f9c6d600640 1 --2- 172.21.15.77:0/3538640058 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f9c500a2b80 0x7f9c500a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:07.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.574+0000 7f9c5a600640 1 -- 172.21.15.77:0/599045519 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c60031db0 con 0x55719ea68330 2024-09-10T09:55:07.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.574+0000 7f9c5a600640 1 -- 172.21.15.77:0/599045519 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c60036040 con 0x55719ea68330 2024-09-10T09:55:07.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.574+0000 7f9c5a600640 1 -- 172.21.15.77:0/599045519 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c600043c0 con 0x55719ea68330 2024-09-10T09:55:07.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.574+0000 7f9c72bdc740 1 -- 172.21.15.77:0/599045519 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55719ea68330 msgr2=0x55719ea68710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:07.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.574+0000 7f9c6d600640 1 --2- 172.21.15.77:0/3538640058 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f9c500a2b80 0x7f9c500a5040 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x55719ea65720 tx=0x7f9c5c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:07.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.574+0000 7f9c72bdc740 1 --2- 172.21.15.77:0/599045519 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55719ea68330 0x55719ea68710 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9c60009ca0 tx=0x7f9c60009800 comp rx=0 tx=0).stop 2024-09-10T09:55:07.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.575+0000 7f9c72bdc740 1 -- 172.21.15.77:0/599045519 shutdown_connections 2024-09-10T09:55:07.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.575+0000 7f9c72bdc740 1 --2- 172.21.15.77:0/599045519 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55719ea73510 0x55719ea75900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:07.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.575+0000 7f9c72bdc740 1 --2- 172.21.15.77:0/599045519 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55719ea68c50 0x55719ea72fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:07.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.575+0000 7f9c72bdc740 1 --2- 172.21.15.77:0/599045519 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55719ea68330 0x55719ea68710 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:07.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.575+0000 7f9c72bdc740 1 -- 172.21.15.77:0/599045519 >> 172.21.15.77:0/599045519 conn(0x55719ea648c0 msgr2=0x55719ea64ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:07.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.575+0000 7f9c72bdc740 1 -- 172.21.15.77:0/599045519 shutdown_connections 2024-09-10T09:55:07.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.575+0000 7f9c72bdc740 1 -- 172.21.15.77:0/599045519 wait complete. 2024-09-10T09:55:07.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.575+0000 7f9c72bdc740 1 -- start start 2024-09-10T09:55:07.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.575+0000 7f9c72bdc740 1 Processor -- start 2024-09-10T09:55:07.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.576+0000 7f9c72bdc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55719ea68330 0x55719ea704a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:07.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.576+0000 7f9c72bdc740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55719ea68c50 0x55719ea709e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:07.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.576+0000 7f9c72bdc740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55719ea73510 0x55719ea70f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:07.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.576+0000 7f9c72bdc740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55719ea76000 con 0x55719ea73510 2024-09-10T09:55:07.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.576+0000 7f9c72bdc740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55719e956920 con 0x55719ea68c50 2024-09-10T09:55:07.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.576+0000 7f9c72bdc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55719ea76180 con 0x55719ea68330 2024-09-10T09:55:07.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.576+0000 7f9c72bdc740 10 client.0 _ll_register_callbacks cb 0x55719e97a8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:55:07.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.576+0000 7f9c6e000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55719ea73510 0x55719ea70f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:07.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.576+0000 7f9c6cc00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55719ea68c50 0x55719ea709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:07.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.576+0000 7f9c6e000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55719ea73510 0x55719ea70f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57602/0 (socket says 172.21.15.77:57602) 2024-09-10T09:55:07.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.576+0000 7f9c6d600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55719ea68330 0x55719ea704a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:07.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.576+0000 7f9c6e000640 1 -- 172.21.15.77:0/3670053614 learned_addr learned my addr 172.21.15.77:0/3670053614 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:55:07.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.577+0000 7f9c6e000640 1 -- 172.21.15.77:0/3670053614 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55719ea68330 msgr2=0x55719ea704a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:07.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.577+0000 7f9c6e000640 1 --2- 172.21.15.77:0/3670053614 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55719ea68330 0x55719ea704a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:07.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.577+0000 7f9c6e000640 1 -- 172.21.15.77:0/3670053614 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55719ea68c50 msgr2=0x55719ea709e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:07.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.577+0000 7f9c6e000640 1 --2- 172.21.15.77:0/3670053614 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55719ea68c50 0x55719ea709e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:07.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.577+0000 7f9c6e000640 1 -- 172.21.15.77:0/3670053614 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55719e73f2b0 con 0x55719ea73510 2024-09-10T09:55:07.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.577+0000 7f9c6cc00640 1 --2- 172.21.15.77:0/3670053614 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55719ea68c50 0x55719ea709e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:55:07.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.577+0000 7f9c6e000640 1 --2- 172.21.15.77:0/3670053614 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55719ea73510 0x55719ea70f20 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f9c60009a30 tx=0x7f9c60031e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2a00795c6275be6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:07.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.577+0000 7f9c4ea00640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-10T09:55:07.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.577+0000 7f9c4ea00640 1 -- 172.21.15.77:0/3670053614 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c60052060 con 0x55719ea73510 2024-09-10T09:55:07.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.577+0000 7f9c72bdc740 1 -- 172.21.15.77:0/3670053614 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c60036040 con 0x55719ea73510 2024-09-10T09:55:07.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.577+0000 7f9c4ea00640 1 -- 172.21.15.77:0/3670053614 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f9c60003b80 con 0x55719ea73510 2024-09-10T09:55:07.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.577+0000 7f9c4ea00640 1 -- 172.21.15.77:0/3670053614 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9c60059050 con 0x55719ea73510 2024-09-10T09:55:07.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.578+0000 7f9c72bdc740 10 client.15171 fetch_fsmap learned FSMap version 28 2024-09-10T09:55:07.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.578+0000 7f9c72bdc740 1 -- 172.21.15.77:0/3670053614 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55719ea76000 con 0x55719ea73510 2024-09-10T09:55:07.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.578+0000 7f9c4ea00640 1 -- 172.21.15.77:0/3670053614 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c60036040 con 0x55719ea73510 2024-09-10T09:55:07.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.578+0000 7f9c4ea00640 1 -- 172.21.15.77:0/3670053614 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f9c60043070 con 0x55719ea73510 2024-09-10T09:55:07.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.578+0000 7f9c72bdc740 10 client.15171 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T09:55:07.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.578+0000 7f9c72bdc740 10 client.15171 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T09:55:07.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.578+0000 7f9c72bdc740 20 client.15171 populate_metadata read hostname 'smithi077' 2024-09-10T09:55:07.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.578+0000 7f9c72bdc740 1 --2- 172.21.15.77:0/3670053614 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55719ea83f00 0x55719ea85370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:07.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.578+0000 7f9c72bdc740 4 client.15171 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:55:07.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.578+0000 7f9c72bdc740 1 -- 172.21.15.77:0/3670053614 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55719e9db620 con 0x55719ea83f00 2024-09-10T09:55:07.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.579+0000 7f9c6d600640 1 --2- 172.21.15.77:0/3670053614 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55719ea83f00 0x55719ea85370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:07.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.579+0000 7f9c6d600640 1 --2- 172.21.15.77:0/3670053614 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55719ea83f00 0x55719ea85370 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dfb88d5e9bb3f06f server_cookie=1c45a373cec5fa2f in_seq=0 out_seq=0 2024-09-10T09:55:07.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.579+0000 7f9c4ea00640 10 client.15171 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T09:55:07.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.580+0000 7f9c4ea00640 1 -- 172.21.15.77:0/3670053614 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55719e9db620 con 0x55719ea83f00 2024-09-10T09:55:07.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.580+0000 7f9c4ea00640 10 client.15171 handle_command_reply: tid=0 2024-09-10T09:55:07.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.580+0000 7f9c72bdc740 1 client.15171 shutdown 2024-09-10T09:55:07.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.580+0000 7f9c72bdc740 1 -- 172.21.15.77:0/3670053614 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55719ea73510 msgr2=0x55719ea70f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:07.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.580+0000 7f9c72bdc740 1 --2- 172.21.15.77:0/3670053614 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55719ea73510 0x55719ea70f20 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f9c60009a30 tx=0x7f9c60031e60 comp rx=0 tx=0).stop 2024-09-10T09:55:07.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.580+0000 7f9c72bdc740 1 -- 172.21.15.77:0/3670053614 shutdown_connections 2024-09-10T09:55:07.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.580+0000 7f9c72bdc740 1 --2- 172.21.15.77:0/3670053614 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55719ea83f00 0x55719ea85370 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:07.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.580+0000 7f9c72bdc740 1 --2- 172.21.15.77:0/3670053614 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55719ea73510 0x55719ea70f20 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:07.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.580+0000 7f9c72bdc740 1 --2- 172.21.15.77:0/3670053614 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55719ea68c50 0x55719ea709e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:07.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.580+0000 7f9c72bdc740 1 --2- 172.21.15.77:0/3670053614 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55719ea68330 0x55719ea704a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:07.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.580+0000 7f9c4ea00640 0 client.15171 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T09:55:07.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.580+0000 7f9c72bdc740 1 -- 172.21.15.77:0/3670053614 >> 172.21.15.77:0/3670053614 conn(0x55719ea648c0 msgr2=0x55719ea6a5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:07.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.580+0000 7f9c72bdc740 1 -- 172.21.15.77:0/3670053614 shutdown_connections 2024-09-10T09:55:07.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.580+0000 7f9c72bdc740 1 -- 172.21.15.77:0/3670053614 wait complete. 2024-09-10T09:55:07.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.581+0000 7f9c72bdc740 20 client.15171 trim_cache size 0 max 16384 2024-09-10T09:55:07.601 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.599+0000 7f9c72bdc740 1 Processor -- start 2024-09-10T09:55:07.601 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.599+0000 7f9c72bdc740 1 -- start start 2024-09-10T09:55:07.601 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.599+0000 7f9c72bdc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55719ea68330 0x55719ea85370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:07.601 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.599+0000 7f9c72bdc740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55719ea68c50 0x55719ea70d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:07.601 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.599+0000 7f9c72bdc740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55719ea73510 0x55719ea6d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:07.601 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.599+0000 7f9c72bdc740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55719ea836b0 con 0x55719ea68c50 2024-09-10T09:55:07.601 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.599+0000 7f9c72bdc740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55719ea83d80 con 0x55719ea73510 2024-09-10T09:55:07.601 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.599+0000 7f9c72bdc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55719ea84450 con 0x55719ea68330 2024-09-10T09:55:07.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.599+0000 7f9c6cc00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55719ea68c50 0x55719ea70d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:07.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.600+0000 7f9c6cc00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55719ea68c50 0x55719ea70d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57614/0 (socket says 172.21.15.77:57614) 2024-09-10T09:55:07.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.600+0000 7f9c6cc00640 1 -- 172.21.15.77:0/2079420164 learned_addr learned my addr 172.21.15.77:0/2079420164 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:55:07.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.600+0000 7f9c6e000640 1 --2- 172.21.15.77:0/2079420164 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55719ea73510 0x55719ea6d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:07.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.600+0000 7f9c6d600640 1 --2- 172.21.15.77:0/2079420164 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55719ea68330 0x55719ea85370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:07.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.600+0000 7f9c6e000640 1 -- 172.21.15.77:0/2079420164 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55719ea68330 msgr2=0x55719ea85370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:07.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.600+0000 7f9c6e000640 1 --2- 172.21.15.77:0/2079420164 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55719ea68330 0x55719ea85370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:07.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.600+0000 7f9c6e000640 1 -- 172.21.15.77:0/2079420164 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55719ea68c50 msgr2=0x55719ea70d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:07.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.600+0000 7f9c6e000640 1 --2- 172.21.15.77:0/2079420164 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55719ea68c50 0x55719ea70d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:07.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.600+0000 7f9c6e000640 1 -- 172.21.15.77:0/2079420164 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55719ea68160 con 0x55719ea73510 2024-09-10T09:55:07.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.600+0000 7f9c6cc00640 1 --2- 172.21.15.77:0/2079420164 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55719ea68c50 0x55719ea70d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:55:07.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.600+0000 7f9c6e000640 1 --2- 172.21.15.77:0/2079420164 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55719ea73510 0x55719ea6d720 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9c6002ade0 tx=0x7f9c60009b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e340383d0d5b33df server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:07.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.600+0000 7f9c59c00640 1 -- 172.21.15.77:0/2079420164 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c60052040 con 0x55719ea73510 2024-09-10T09:55:07.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.600+0000 7f9c59c00640 1 -- 172.21.15.77:0/2079420164 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c60036040 con 0x55719ea73510 2024-09-10T09:55:07.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.600+0000 7f9c59c00640 1 -- 172.21.15.77:0/2079420164 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c6002fd30 con 0x55719ea73510 2024-09-10T09:55:07.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.601+0000 7f9c72bdc740 1 -- 172.21.15.77:0/2079420164 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55719ea73510 msgr2=0x55719ea6d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:07.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.601+0000 7f9c72bdc740 1 --2- 172.21.15.77:0/2079420164 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55719ea73510 0x55719ea6d720 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9c6002ade0 tx=0x7f9c60009b20 comp rx=0 tx=0).stop 2024-09-10T09:55:07.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.601+0000 7f9c72bdc740 1 -- 172.21.15.77:0/2079420164 shutdown_connections 2024-09-10T09:55:07.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.601+0000 7f9c72bdc740 1 --2- 172.21.15.77:0/2079420164 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55719ea73510 0x55719ea6d720 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:07.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.601+0000 7f9c72bdc740 1 --2- 172.21.15.77:0/2079420164 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55719ea68c50 0x55719ea70d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:07.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.601+0000 7f9c72bdc740 1 --2- 172.21.15.77:0/2079420164 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55719ea68330 0x55719ea85370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:07.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.601+0000 7f9c72bdc740 1 -- 172.21.15.77:0/2079420164 >> 172.21.15.77:0/2079420164 conn(0x55719ea648c0 msgr2=0x55719ea6ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:07.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.601+0000 7f9c72bdc740 1 -- 172.21.15.77:0/2079420164 shutdown_connections 2024-09-10T09:55:07.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.601+0000 7f9c72bdc740 1 -- 172.21.15.77:0/2079420164 wait complete. 2024-09-10T09:55:07.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.602+0000 7f9c72bdc740 1 -- start start 2024-09-10T09:55:07.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.602+0000 7f9c72bdc740 1 Processor -- start 2024-09-10T09:55:07.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.602+0000 7f9c72bdc740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55719ea68330 0x55719ea8dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:07.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.602+0000 7f9c72bdc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55719ea68c50 0x55719ea6d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:07.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.602+0000 7f9c72bdc740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55719ea73510 0x55719ea6d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:07.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.602+0000 7f9c72bdc740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55719ea851f0 con 0x55719ea73510 2024-09-10T09:55:07.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.602+0000 7f9c72bdc740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55719ea84b20 con 0x55719ea68330 2024-09-10T09:55:07.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.602+0000 7f9c72bdc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55719ea858c0 con 0x55719ea68c50 2024-09-10T09:55:07.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.602+0000 7f9c72bdc740 10 client.0 _ll_register_callbacks cb 0x7f9c60031780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:55:07.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.603+0000 7f9c6d600640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55719ea68330 0x55719ea8dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:07.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.603+0000 7f9c6cc00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55719ea68c50 0x55719ea6d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:07.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.603+0000 7f9c6e000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55719ea73510 0x55719ea6d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:07.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.603+0000 7f9c6d600640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55719ea68330 0x55719ea8dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:43246/0 (socket says 172.21.15.77:43246) 2024-09-10T09:55:07.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.603+0000 7f9c6cc00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55719ea68c50 0x55719ea6d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:39172/0 (socket says 172.21.15.77:39172) 2024-09-10T09:55:07.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.603+0000 7f9c6d600640 1 -- 172.21.15.77:0/1425905526 learned_addr learned my addr 172.21.15.77:0/1425905526 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:55:07.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.603+0000 7f9c6cc00640 1 -- 172.21.15.77:0/1425905526 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55719ea68330 msgr2=0x55719ea8dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:07.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.603+0000 7f9c6cc00640 1 --2- 172.21.15.77:0/1425905526 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55719ea68330 0x55719ea8dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:07.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.603+0000 7f9c6cc00640 1 -- 172.21.15.77:0/1425905526 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55719ea73510 msgr2=0x55719ea6d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:07.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.603+0000 7f9c6cc00640 1 --2- 172.21.15.77:0/1425905526 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55719ea73510 0x55719ea6d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:07.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.603+0000 7f9c6cc00640 1 -- 172.21.15.77:0/1425905526 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55719ea6df00 con 0x55719ea68c50 2024-09-10T09:55:07.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.603+0000 7f9c6d600640 1 --2- 172.21.15.77:0/1425905526 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55719ea68330 0x55719ea8dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:55:07.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.603+0000 7f9c6cc00640 1 --2- 172.21.15.77:0/1425905526 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55719ea68c50 0x55719ea6d3d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9c540985c0 tx=0x7f9c5409b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7e107d1ee10999 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:07.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.603+0000 7f9c4ea00640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-10T09:55:07.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.604+0000 7f9c4ea00640 1 -- 172.21.15.77:0/1425905526 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c5409fff0 con 0x55719ea68c50 2024-09-10T09:55:07.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.604+0000 7f9c72bdc740 1 -- 172.21.15.77:0/1425905526 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c60036040 con 0x55719ea68c50 2024-09-10T09:55:07.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.604+0000 7f9c4ea00640 1 -- 172.21.15.77:0/1425905526 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f9c5409b090 con 0x55719ea68c50 2024-09-10T09:55:07.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.604+0000 7f9c4ea00640 1 -- 172.21.15.77:0/1425905526 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9c540a48b0 con 0x55719ea68c50 2024-09-10T09:55:07.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.604+0000 7f9c72bdc740 10 client.25201 fetch_fsmap learned FSMap version 28 2024-09-10T09:55:07.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.604+0000 7f9c72bdc740 1 -- 172.21.15.77:0/1425905526 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55719eaee610 con 0x55719ea68c50 2024-09-10T09:55:07.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.604+0000 7f9c4ea00640 1 -- 172.21.15.77:0/1425905526 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c60036040 con 0x55719ea68c50 2024-09-10T09:55:07.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.604+0000 7f9c4ea00640 1 -- 172.21.15.77:0/1425905526 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f9c540ba2a0 con 0x55719ea68c50 2024-09-10T09:55:07.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.604+0000 7f9c72bdc740 10 client.25201 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T09:55:07.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.604+0000 7f9c72bdc740 10 client.25201 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T09:55:07.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.604+0000 7f9c72bdc740 20 client.25201 populate_metadata read hostname 'smithi077' 2024-09-10T09:55:07.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.604+0000 7f9c72bdc740 1 --2- 172.21.15.77:0/1425905526 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55719eaf2ff0 0x55719eaf1d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:07.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.604+0000 7f9c72bdc740 4 client.25201 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T09:55:07.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.604+0000 7f9c72bdc740 1 -- 172.21.15.77:0/1425905526 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "status"}) -- 0x55719e9d5b30 con 0x55719eaf2ff0 2024-09-10T09:55:07.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.605+0000 7f9c6d600640 1 --2- 172.21.15.77:0/1425905526 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55719eaf2ff0 0x55719eaf1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:07.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.605+0000 7f9c6d600640 1 --2- 172.21.15.77:0/1425905526 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55719eaf2ff0 0x55719eaf1d00 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49eb77eb5c6f7eff server_cookie=5e7e20125ae0245f in_seq=0 out_seq=0 2024-09-10T09:55:07.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.605+0000 7f9c4ea00640 10 client.25201 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T09:55:07.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.605+0000 7f9c4ea00640 1 -- 172.21.15.77:0/1425905526 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55719e9d5b30 con 0x55719eaf2ff0 2024-09-10T09:55:07.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.605+0000 7f9c4ea00640 10 client.25201 handle_command_reply: tid=0 2024-09-10T09:55:07.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.605+0000 7f9c72bdc740 1 client.25201 shutdown 2024-09-10T09:55:07.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.606+0000 7f9c72bdc740 1 -- 172.21.15.77:0/1425905526 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55719ea68c50 msgr2=0x55719ea6d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:07.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.606+0000 7f9c72bdc740 1 --2- 172.21.15.77:0/1425905526 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55719ea68c50 0x55719ea6d3d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9c540985c0 tx=0x7f9c5409b840 comp rx=0 tx=0).stop 2024-09-10T09:55:07.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.606+0000 7f9c72bdc740 1 -- 172.21.15.77:0/1425905526 shutdown_connections 2024-09-10T09:55:07.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.606+0000 7f9c72bdc740 1 --2- 172.21.15.77:0/1425905526 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55719eaf2ff0 0x55719eaf1d00 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:07.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.606+0000 7f9c72bdc740 1 --2- 172.21.15.77:0/1425905526 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55719ea73510 0x55719ea6d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:07.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.606+0000 7f9c72bdc740 1 --2- 172.21.15.77:0/1425905526 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55719ea68c50 0x55719ea6d3d0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:07.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.606+0000 7f9c4ea00640 0 client.25201 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T09:55:07.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.606+0000 7f9c72bdc740 1 --2- 172.21.15.77:0/1425905526 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55719ea68330 0x55719ea8dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:07.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.606+0000 7f9c72bdc740 1 -- 172.21.15.77:0/1425905526 >> 172.21.15.77:0/1425905526 conn(0x55719ea648c0 msgr2=0x55719ea71d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:07.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.606+0000 7f9c72bdc740 1 -- 172.21.15.77:0/1425905526 shutdown_connections 2024-09-10T09:55:07.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.606+0000 7f9c72bdc740 1 -- 172.21.15.77:0/1425905526 wait complete. 2024-09-10T09:55:07.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.606+0000 7f9c72bdc740 20 client.25201 trim_cache size 0 max 16384 2024-09-10T09:55:07.613 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-10T09:55:07.613 INFO:teuthology.orchestra.run.smithi077.stdout: "cluster_fsid": "14a55226-6f59-11ef-bcea-c7b262605968", 2024-09-10T09:55:07.613 INFO:teuthology.orchestra.run.smithi077.stdout: "whoami": 0, 2024-09-10T09:55:07.613 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 24563, 2024-09-10T09:55:07.613 INFO:teuthology.orchestra.run.smithi077.stdout: "want_state": "up:active", 2024-09-10T09:55:07.613 INFO:teuthology.orchestra.run.smithi077.stdout: "state": "up:active", 2024-09-10T09:55:07.613 INFO:teuthology.orchestra.run.smithi077.stdout: "fs_name": "cephfs", 2024-09-10T09:55:07.613 INFO:teuthology.orchestra.run.smithi077.stdout: "rank_uptime": 163.29350914700001, 2024-09-10T09:55:07.613 INFO:teuthology.orchestra.run.smithi077.stdout: "mdsmap_epoch": 28, 2024-09-10T09:55:07.613 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch": 96, 2024-09-10T09:55:07.613 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch_barrier": 94, 2024-09-10T09:55:07.613 INFO:teuthology.orchestra.run.smithi077.stdout: "uptime": 176.923812803 2024-09-10T09:55:07.614 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-10T09:55:07.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.607+0000 7f9c70000640 1 -- 172.21.15.77:0/3538640058 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f9c500a2b80 msgr2=0x7f9c500a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:07.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.607+0000 7f9c70000640 1 --2- 172.21.15.77:0/3538640058 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f9c500a2b80 0x7f9c500a5040 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x55719ea65720 tx=0x7f9c5c00b390 comp rx=0 tx=0).stop 2024-09-10T09:55:07.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.607+0000 7f9c70000640 1 -- 172.21.15.77:0/3538640058 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9c68102790 msgr2=0x7f9c6819ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:07.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.607+0000 7f9c70000640 1 --2- 172.21.15.77:0/3538640058 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9c68102790 0x7f9c6819ab90 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9c5400e9f0 tx=0x7f9c5400edf0 comp rx=0 tx=0).stop 2024-09-10T09:55:07.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.607+0000 7f9c70000640 1 -- 172.21.15.77:0/3538640058 shutdown_connections 2024-09-10T09:55:07.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.607+0000 7f9c70000640 1 --2- 172.21.15.77:0/3538640058 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f9c500a2b80 0x7f9c500a5040 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:07.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.607+0000 7f9c70000640 1 --2- 172.21.15.77:0/3538640058 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c68106f10 0x7f9c6819ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:07.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.607+0000 7f9c70000640 1 --2- 172.21.15.77:0/3538640058 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9c68102790 0x7f9c6819ab90 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:07.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.607+0000 7f9c70000640 1 --2- 172.21.15.77:0/3538640058 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9c68101de0 0x7f9c6819a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:07.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.607+0000 7f9c70000640 1 -- 172.21.15.77:0/3538640058 >> 172.21.15.77:0/3538640058 conn(0x7f9c680fd8c0 msgr2=0x7f9c680ffc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:07.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.607+0000 7f9c70000640 1 -- 172.21.15.77:0/3538640058 shutdown_connections 2024-09-10T09:55:07.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:07.607+0000 7f9c70000640 1 -- 172.21.15.77:0/3538640058 wait complete. 2024-09-10T09:55:08.443 INFO:teuthology.run_tasks:Running task sequential... 2024-09-10T09:55:08.453 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-09-10T09:55:08.453 INFO:teuthology.task.sequential:In sequential, running task print... 2024-09-10T09:55:08.454 INFO:teuthology.task.print:Enabling mgr modules 2024-09-10T09:55:08.454 INFO:teuthology.run_tasks:Running task kclient... 2024-09-10T09:55:08.472 INFO:tasks.kclient:Mounting kernel clients... 2024-09-10T09:55:08.472 INFO:tasks.kclient:config is {'client.0': {}} 2024-09-10T09:55:08.472 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:55:08.472 DEBUG:teuthology.orchestra.run.smithi077:> ip netns list 2024-09-10T09:55:08.509 INFO:teuthology.orchestra.run.smithi077.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-09-10T09:55:08.509 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:55:08.510 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T09:55:08.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:55:08.607 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip link delete ceph-brx 2024-09-10T09:55:08.901 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=legacy', 'wsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-09-10T09:55:08.902 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-10T09:55:08.902 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T09:55:08.902 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph fs ls 2024-09-10T09:55:09.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:55:08 smithi134 ceph-mon[35674]: pgmap v390: 129 pgs: 129 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-10T09:55:09.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:55:08 smithi120 ceph-mon[34770]: pgmap v390: 129 pgs: 129 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-10T09:55:09.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:55:08 smithi077 ceph-mon[30319]: pgmap v390: 129 pgs: 129 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-10T09:55:09.267 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:55:09.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:09.794+0000 7ffa55400640 1 -- 172.21.15.77:0/1019670823 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa501049a0 msgr2=0x7ffa501089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:09.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:09.794+0000 7ffa55400640 1 --2- 172.21.15.77:0/1019670823 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa501049a0 0x7ffa501089f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ffa38009860 tx=0x7ffa3802a340 comp rx=0 tx=0).stop 2024-09-10T09:55:09.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:09.794+0000 7ffa55400640 1 -- 172.21.15.77:0/1019670823 shutdown_connections 2024-09-10T09:55:09.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:09.794+0000 7ffa55400640 1 --2- 172.21.15.77:0/1019670823 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa50109120 0x7ffa50110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:09.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:09.794+0000 7ffa55400640 1 --2- 172.21.15.77:0/1019670823 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa501049a0 0x7ffa501089f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:09.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:09.794+0000 7ffa55400640 1 --2- 172.21.15.77:0/1019670823 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa50103ff0 0x7ffa501043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:09.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:09.794+0000 7ffa55400640 1 -- 172.21.15.77:0/1019670823 >> 172.21.15.77:0/1019670823 conn(0x7ffa500ffa90 msgr2=0x7ffa50101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:09.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:09.794+0000 7ffa55400640 1 -- 172.21.15.77:0/1019670823 shutdown_connections 2024-09-10T09:55:09.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:09.794+0000 7ffa55400640 1 -- 172.21.15.77:0/1019670823 wait complete. 2024-09-10T09:55:09.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:09.796+0000 7ffa55400640 1 Processor -- start 2024-09-10T09:55:09.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:09.796+0000 7ffa55400640 1 -- start start 2024-09-10T09:55:09.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:09.796+0000 7ffa55400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa50103ff0 0x7ffa5019a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:09.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:09.796+0000 7ffa55400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa501049a0 0x7ffa5019ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:09.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:09.796+0000 7ffa55400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa50109120 0x7ffa5019eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:09.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:09.796+0000 7ffa55400640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ffa50114120 con 0x7ffa501049a0 2024-09-10T09:55:09.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:09.796+0000 7ffa55400640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ffa50113fa0 con 0x7ffa50103ff0 2024-09-10T09:55:09.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:09.796+0000 7ffa55400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ffa501142a0 con 0x7ffa50109120 2024-09-10T09:55:09.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:09.797+0000 7ffa45e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa501049a0 0x7ffa5019ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:09.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:09.797+0000 7ffa45e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa501049a0 0x7ffa5019ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57638/0 (socket says 172.21.15.77:57638) 2024-09-10T09:55:09.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:09.797+0000 7ffa45e00640 1 -- 172.21.15.77:0/1981956711 learned_addr learned my addr 172.21.15.77:0/1981956711 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:55:09.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:09.797+0000 7ffa4f200640 1 --2- 172.21.15.77:0/1981956711 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa50109120 0x7ffa5019eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:09.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:09.797+0000 7ffa4e800640 1 --2- 172.21.15.77:0/1981956711 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa50103ff0 0x7ffa5019a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:09.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:09.797+0000 7ffa45e00640 1 -- 172.21.15.77:0/1981956711 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa50109120 msgr2=0x7ffa5019eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:09.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:09.797+0000 7ffa45e00640 1 --2- 172.21.15.77:0/1981956711 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa50109120 0x7ffa5019eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:09.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:09.797+0000 7ffa45e00640 1 -- 172.21.15.77:0/1981956711 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa50103ff0 msgr2=0x7ffa5019a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:09.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:09.797+0000 7ffa45e00640 1 --2- 172.21.15.77:0/1981956711 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa50103ff0 0x7ffa5019a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:09.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:09.797+0000 7ffa45e00640 1 -- 172.21.15.77:0/1981956711 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa5019f5d0 con 0x7ffa501049a0 2024-09-10T09:55:09.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:09.797+0000 7ffa45e00640 1 --2- 172.21.15.77:0/1981956711 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa501049a0 0x7ffa5019ab60 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ffa38035040 tx=0x7ffa38033b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:09.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:09.798+0000 7ffa47e00640 1 -- 172.21.15.77:0/1981956711 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa38037040 con 0x7ffa501049a0 2024-09-10T09:55:09.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:09.798+0000 7ffa55400640 1 -- 172.21.15.77:0/1981956711 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa5019f860 con 0x7ffa501049a0 2024-09-10T09:55:09.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:09.798+0000 7ffa47e00640 1 -- 172.21.15.77:0/1981956711 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa38040070 con 0x7ffa501049a0 2024-09-10T09:55:09.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:09.798+0000 7ffa47e00640 1 -- 172.21.15.77:0/1981956711 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa380039b0 con 0x7ffa501049a0 2024-09-10T09:55:09.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:09.798+0000 7ffa55400640 1 -- 172.21.15.77:0/1981956711 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa501a71a0 con 0x7ffa501049a0 2024-09-10T09:55:09.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:09.799+0000 7ffa55400640 1 -- 172.21.15.77:0/1981956711 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa501054d0 con 0x7ffa501049a0 2024-09-10T09:55:09.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:09.801+0000 7ffa47e00640 1 -- 172.21.15.77:0/1981956711 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ffa380305e0 con 0x7ffa501049a0 2024-09-10T09:55:09.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:09.802+0000 7ffa47e00640 1 --2- 172.21.15.77:0/1981956711 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ffa340a2b80 0x7ffa340a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:09.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:09.802+0000 7ffa47e00640 1 -- 172.21.15.77:0/1981956711 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7ffa380ef3e0 con 0x7ffa501049a0 2024-09-10T09:55:09.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:09.802+0000 7ffa4e800640 1 --2- 172.21.15.77:0/1981956711 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ffa340a2b80 0x7ffa340a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:09.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:09.803+0000 7ffa4e800640 1 --2- 172.21.15.77:0/1981956711 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ffa340a2b80 0x7ffa340a5040 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ffa40005fd0 tx=0x7ffa40009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:09.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:09.806+0000 7ffa47e00640 1 -- 172.21.15.77:0/1981956711 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ffa380bbef0 con 0x7ffa501049a0 2024-09-10T09:55:10.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:10.021+0000 7ffa55400640 1 -- 172.21.15.77:0/1981956711 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ffa5010c360 con 0x7ffa501049a0 2024-09-10T09:55:10.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:10.021+0000 7ffa47e00640 1 -- 172.21.15.77:0/1981956711 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v28) ==== 53+0+88 (secure 0 0 0) 0x7ffa380bb7f0 con 0x7ffa501049a0 2024-09-10T09:55:10.023 INFO:teuthology.orchestra.run.smithi077.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-10T09:55:10.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:10.023+0000 7ffa55400640 1 -- 172.21.15.77:0/1981956711 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ffa340a2b80 msgr2=0x7ffa340a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:10.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:10.023+0000 7ffa55400640 1 --2- 172.21.15.77:0/1981956711 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ffa340a2b80 0x7ffa340a5040 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ffa40005fd0 tx=0x7ffa40009260 comp rx=0 tx=0).stop 2024-09-10T09:55:10.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:10.024+0000 7ffa55400640 1 -- 172.21.15.77:0/1981956711 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa501049a0 msgr2=0x7ffa5019ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:10.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:10.024+0000 7ffa55400640 1 --2- 172.21.15.77:0/1981956711 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa501049a0 0x7ffa5019ab60 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ffa38035040 tx=0x7ffa38033b60 comp rx=0 tx=0).stop 2024-09-10T09:55:10.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:10.024+0000 7ffa55400640 1 -- 172.21.15.77:0/1981956711 shutdown_connections 2024-09-10T09:55:10.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:10.024+0000 7ffa55400640 1 --2- 172.21.15.77:0/1981956711 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ffa340a2b80 0x7ffa340a5040 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:10.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:10.024+0000 7ffa55400640 1 --2- 172.21.15.77:0/1981956711 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa50109120 0x7ffa5019eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:10.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:10.024+0000 7ffa55400640 1 --2- 172.21.15.77:0/1981956711 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa501049a0 0x7ffa5019ab60 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:10.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:10.024+0000 7ffa55400640 1 --2- 172.21.15.77:0/1981956711 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa50103ff0 0x7ffa5019a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:10.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:10.024+0000 7ffa55400640 1 -- 172.21.15.77:0/1981956711 >> 172.21.15.77:0/1981956711 conn(0x7ffa500ffa90 msgr2=0x7ffa50100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:10.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:10.024+0000 7ffa55400640 1 -- 172.21.15.77:0/1981956711 shutdown_connections 2024-09-10T09:55:10.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:10.024+0000 7ffa55400640 1 -- 172.21.15.77:0/1981956711 wait complete. 2024-09-10T09:55:10.747 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T09:55:10.747 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T09:55:10.747 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi077.front.sepia.ceph.com 2024-09-10T09:55:10.747 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T09:55:10.747 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T09:55:10.747 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T09:55:10.747 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T09:55:10.748 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T09:55:10.748 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T09:55:10.748 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:55:10.748 DEBUG:teuthology.orchestra.run.smithi077:> ip addr 2024-09-10T09:55:10.785 INFO:teuthology.orchestra.run.smithi077.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T09:55:10.785 INFO:teuthology.orchestra.run.smithi077.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T09:55:10.785 INFO:teuthology.orchestra.run.smithi077.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T09:55:10.786 INFO:teuthology.orchestra.run.smithi077.stdout: valid_lft forever preferred_lft forever 2024-09-10T09:55:10.786 INFO:teuthology.orchestra.run.smithi077.stdout: inet6 ::1/128 scope host 2024-09-10T09:55:10.786 INFO:teuthology.orchestra.run.smithi077.stdout: valid_lft forever preferred_lft forever 2024-09-10T09:55:10.786 INFO:teuthology.orchestra.run.smithi077.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T09:55:10.786 INFO:teuthology.orchestra.run.smithi077.stdout: link/ether 0c:c4:7a:d8:d9:d8 brd ff:ff:ff:ff:ff:ff 2024-09-10T09:55:10.786 INFO:teuthology.orchestra.run.smithi077.stdout: altname enp7s0f0 2024-09-10T09:55:10.786 INFO:teuthology.orchestra.run.smithi077.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T09:55:10.786 INFO:teuthology.orchestra.run.smithi077.stdout: link/ether 0c:c4:7a:d8:d9:d9 brd ff:ff:ff:ff:ff:ff 2024-09-10T09:55:10.786 INFO:teuthology.orchestra.run.smithi077.stdout: altname enp7s0f1 2024-09-10T09:55:10.786 INFO:teuthology.orchestra.run.smithi077.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T09:55:10.786 INFO:teuthology.orchestra.run.smithi077.stdout: link/ether 0c:c4:7a:8f:d3:d2 brd ff:ff:ff:ff:ff:ff 2024-09-10T09:55:10.786 INFO:teuthology.orchestra.run.smithi077.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T09:55:10.786 INFO:teuthology.orchestra.run.smithi077.stdout: link/ether 0c:c4:7a:8f:d3:d3 brd ff:ff:ff:ff:ff:ff 2024-09-10T09:55:10.787 INFO:teuthology.orchestra.run.smithi077.stdout: inet 172.21.15.77/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-10T09:55:10.787 INFO:teuthology.orchestra.run.smithi077.stdout: valid_lft 42080sec preferred_lft 42080sec 2024-09-10T09:55:10.787 INFO:teuthology.orchestra.run.smithi077.stdout: inet6 fe80::ec4:7aff:fe8f:d3d3/64 scope link noprefixroute 2024-09-10T09:55:10.787 INFO:teuthology.orchestra.run.smithi077.stdout: valid_lft forever preferred_lft forever 2024-09-10T09:55:10.787 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-10T09:55:10.787 DEBUG:teuthology.orchestra.run.smithi077:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T09:55:10.787 DEBUG:teuthology.orchestra.run.smithi077:> set -e 2024-09-10T09:55:10.787 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip link add name ceph-brx type bridge 2024-09-10T09:55:10.787 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip addr flush dev ceph-brx 2024-09-10T09:55:10.787 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip link set ceph-brx up 2024-09-10T09:55:10.788 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-10T09:55:10.788 DEBUG:teuthology.orchestra.run.smithi077:> ') 2024-09-10T09:55:10.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:10 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T09:55:10.970 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:55:10 smithi077 ceph-mon[30319]: pgmap v391: 129 pgs: 129 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-09-10T09:55:10.970 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:55:10 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/1981956711' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-10T09:55:11.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T09:55:11.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:55:11.012 DEBUG:teuthology.orchestra.run.smithi077:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T09:55:11.066 INFO:teuthology.orchestra.run.smithi077.stdout:1 2024-09-10T09:55:11.069 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:55:11.069 DEBUG:teuthology.orchestra.run.smithi077:> route 2024-09-10T09:55:11.106 INFO:teuthology.orchestra.run.smithi077.stdout:Kernel IP routing table 2024-09-10T09:55:11.106 INFO:teuthology.orchestra.run.smithi077.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T09:55:11.106 INFO:teuthology.orchestra.run.smithi077.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T09:55:11.106 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T09:55:11.106 INFO:teuthology.orchestra.run.smithi077.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T09:55:11.107 DEBUG:teuthology.orchestra.run.smithi077:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T09:55:11.107 DEBUG:teuthology.orchestra.run.smithi077:> set -e 2024-09-10T09:55:11.107 DEBUG:teuthology.orchestra.run.smithi077:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T09:55:11.107 DEBUG:teuthology.orchestra.run.smithi077:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T09:55:11.107 DEBUG:teuthology.orchestra.run.smithi077:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T09:55:11.107 DEBUG:teuthology.orchestra.run.smithi077:> ') 2024-09-10T09:55:11.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:55:10 smithi134 ceph-mon[35674]: pgmap v391: 129 pgs: 129 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-09-10T09:55:11.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:55:10 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/1981956711' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-10T09:55:11.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:55:10 smithi120 ceph-mon[34770]: pgmap v391: 129 pgs: 129 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-09-10T09:55:11.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:55:10 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/1981956711' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-10T09:55:11.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T09:55:11.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T09:55:11.306 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:55:11.306 DEBUG:teuthology.orchestra.run.smithi077:> ip netns list 2024-09-10T09:55:11.341 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:55:11.342 DEBUG:teuthology.orchestra.run.smithi077:> ip netns list-id 2024-09-10T09:55:11.417 DEBUG:teuthology.orchestra.run.smithi077:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T09:55:11.417 DEBUG:teuthology.orchestra.run.smithi077:> set -e 2024-09-10T09:55:11.417 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T09:55:11.417 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-10T09:55:11.417 DEBUG:teuthology.orchestra.run.smithi077:> ') 2024-09-10T09:55:11.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T09:55:11.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T09:55:11.596 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-10T09:55:11.596 DEBUG:teuthology.orchestra.run.smithi077:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T09:55:11.597 DEBUG:teuthology.orchestra.run.smithi077:> set -e 2024-09-10T09:55:11.597 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-10T09:55:11.597 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-10T09:55:11.597 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-10T09:55:11.597 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-10T09:55:11.597 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-10T09:55:11.597 DEBUG:teuthology.orchestra.run.smithi077:> ') 2024-09-10T09:55:11.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T09:55:11.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T09:55:11.819 DEBUG:teuthology.orchestra.run.smithi077:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T09:55:11.819 DEBUG:teuthology.orchestra.run.smithi077:> set -e 2024-09-10T09:55:11.819 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip link set brx.0 up 2024-09-10T09:55:11.819 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip link set dev brx.0 master ceph-brx 2024-09-10T09:55:11.820 DEBUG:teuthology.orchestra.run.smithi077:> ') 2024-09-10T09:55:11.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T09:55:11.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T09:55:11.987 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T09:55:11.987 DEBUG:teuthology.orchestra.run.smithi077:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T09:55:12.022 INFO:teuthology.orchestra.run.smithi077.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T09:55:12.022 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T09:55:12.023 DEBUG:teuthology.orchestra.run.smithi077:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T09:55:12.098 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-09-10T09:55:12.098 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-09-10T09:55:12.098 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/10f21c5c-5a81-465f-abea-54bb63745c38 2024-09-10T09:55:12.098 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:55:12.098 DEBUG:teuthology.orchestra.run.smithi077:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/10f21c5c-5a81-465f-abea-54bb63745c38 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,wsync,nofallback 2024-09-10T09:55:12.226 INFO:teuthology.orchestra.run.smithi077.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,wsync,nofallback 2024-09-10T09:55:12.226 INFO:teuthology.orchestra.run.smithi077.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,wsync". 2024-09-10T09:55:12.868 INFO:teuthology.orchestra.run.smithi077.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,wsync,nofallback 2024-09-10T09:55:12.868 INFO:teuthology.orchestra.run.smithi077.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,wsync". 2024-09-10T09:55:12.868 INFO:teuthology.orchestra.run.smithi077.stdout:mount.ceph: resolved to: "172.21.15.77:6789,172.21.15.120:6789,172.21.15.134:6789" 2024-09-10T09:55:12.868 INFO:teuthology.orchestra.run.smithi077.stdout:mount.ceph: trying mount with new device syntax: 0@14a55226-6f59-11ef-bcea-c7b262605968.cephfs=/volumes/qa/sv_1/10f21c5c-5a81-465f-abea-54bb63745c38 2024-09-10T09:55:12.868 INFO:teuthology.orchestra.run.smithi077.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,wsync,name=0,key=0,mon_addr=172.21.15.77:6789/172.21.15.120:6789/172.21.15.134:6789" will pass to kernel 2024-09-10T09:55:12.870 INFO:teuthology.orchestra.run.smithi077.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-10T09:55:12.870 INFO:teuthology.orchestra.run.smithi077.stdout: You just mounted a file system that supports labels which does not 2024-09-10T09:55:12.870 INFO:teuthology.orchestra.run.smithi077.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-10T09:55:12.870 INFO:teuthology.orchestra.run.smithi077.stdout: applications will generate AVC messages and not be allowed access to 2024-09-10T09:55:12.870 INFO:teuthology.orchestra.run.smithi077.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-10T09:55:12.873 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T09:55:12.874 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:55:12.874 DEBUG:teuthology.orchestra.run.smithi077:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T09:55:12.936 DEBUG:teuthology.orchestra.run.smithi077:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T09:55:13.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:13 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T09:55:13.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T09:55:13.027 INFO:teuthology.orchestra.run.smithi077.stdout:client25210 2024-09-10T09:55:13.028 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-09-10T09:55:13.028 DEBUG:teuthology.orchestra.run.smithi077:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T09:55:13.064 DEBUG:teuthology.orchestra.run.smithi077:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T09:55:13.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:55:12 smithi134 ceph-mon[35674]: pgmap v392: 129 pgs: 129 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-10T09:55:13.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:55:12 smithi120 ceph-mon[34770]: pgmap v392: 129 pgs: 129 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-10T09:55:13.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:13 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T09:55:13.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T09:55:13.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:55:12 smithi077 ceph-mon[30319]: pgmap v392: 129 pgs: 129 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-10T09:55:13.194 INFO:teuthology.orchestra.run.smithi077.stdout:client25210 2024-09-10T09:55:13.194 DEBUG:teuthology.orchestra.run.smithi077:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T09:55:13.272 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:13 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T09:55:13.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T09:55:13.283 INFO:teuthology.orchestra.run.smithi077.stdout:client25210 2024-09-10T09:55:13.283 DEBUG:teuthology.orchestra.run.smithi077:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/14a55226-6f59-11ef-bcea-c7b262605968.client25210/status') 2024-09-10T09:55:13.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T09:55:13.383 INFO:teuthology.orchestra.run.smithi077.stdout:instance: client.25210 (3)172.21.15.77:0/4123468726 2024-09-10T09:55:13.383 INFO:teuthology.orchestra.run.smithi077.stdout:blocklisted: false 2024-09-10T09:55:13.383 INFO:teuthology.orchestra.run.smithi077.stderr:0+1 records in 2024-09-10T09:55:13.383 INFO:teuthology.orchestra.run.smithi077.stderr:0+1 records out 2024-09-10T09:55:13.384 INFO:teuthology.orchestra.run.smithi077.stderr:71 bytes copied, 7.0764e-05 s, 1.0 MB/s 2024-09-10T09:55:13.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T09:55:13.394 INFO:teuthology.run_tasks:Running task exec... 2024-09-10T09:55:13.409 INFO:teuthology.task.exec:Executing custom commands... 2024-09-10T09:55:13.409 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi077.front.sepia.ceph.com 2024-09-10T09:55:13.410 DEBUG:teuthology.orchestra.run.smithi077:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-09-10T09:55:13.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:13.561+0000 7f4a11c00640 1 -- 172.21.15.77:0/2217970274 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4a0c104d60 msgr2=0x7f4a0c1051e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:13.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:13.561+0000 7f4a11c00640 1 --2- 172.21.15.77:0/2217970274 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4a0c104d60 0x7f4a0c1051e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f49f4009900 tx=0x7f49f402a410 comp rx=0 tx=0).stop 2024-09-10T09:55:13.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:13.561+0000 7f4a11c00640 1 -- 172.21.15.77:0/2217970274 shutdown_connections 2024-09-10T09:55:13.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:13.561+0000 7f4a11c00640 1 --2- 172.21.15.77:0/2217970274 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4a0c105720 0x7f4a0c10bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:13.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:13.561+0000 7f4a11c00640 1 --2- 172.21.15.77:0/2217970274 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4a0c104d60 0x7f4a0c1051e0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:13.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:13.561+0000 7f4a11c00640 1 --2- 172.21.15.77:0/2217970274 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a0c103b70 0x7f4a0c103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:13.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:13.561+0000 7f4a11c00640 1 -- 172.21.15.77:0/2217970274 >> 172.21.15.77:0/2217970274 conn(0x7f4a0c0ff310 msgr2=0x7f4a0c101730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:13.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:13.561+0000 7f4a11c00640 1 -- 172.21.15.77:0/2217970274 shutdown_connections 2024-09-10T09:55:13.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:13.561+0000 7f4a11c00640 1 -- 172.21.15.77:0/2217970274 wait complete. 2024-09-10T09:55:13.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:13.562+0000 7f4a11c00640 1 Processor -- start 2024-09-10T09:55:13.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:13.562+0000 7f4a11c00640 1 -- start start 2024-09-10T09:55:13.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:13.562+0000 7f4a11c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a0c103b70 0x7f4a0c199ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:13.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:13.562+0000 7f4a11c00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4a0c104d60 0x7f4a0c19a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:13.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:13.562+0000 7f4a11c00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4a0c105720 0x7f4a0c1a1450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:13.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:13.562+0000 7f4a11c00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4a0c10f840 con 0x7f4a0c105720 2024-09-10T09:55:13.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:13.562+0000 7f4a11c00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4a0c10f6c0 con 0x7f4a0c104d60 2024-09-10T09:55:13.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:13.562+0000 7f4a11c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4a0c10f9c0 con 0x7f4a0c103b70 2024-09-10T09:55:13.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:13.562+0000 7f4a11200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4a0c105720 0x7f4a0c1a1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:13.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:13.562+0000 7f4a11200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4a0c105720 0x7f4a0c1a1450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52600/0 (socket says 172.21.15.77:52600) 2024-09-10T09:55:13.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:13.562+0000 7f4a11200640 1 -- 172.21.15.77:0/2324759926 learned_addr learned my addr 172.21.15.77:0/2324759926 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:55:13.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:13.562+0000 7f4a0be00640 1 --2- 172.21.15.77:0/2324759926 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a0c103b70 0x7f4a0c199ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:13.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:13.563+0000 7f4a03e00640 1 --2- 172.21.15.77:0/2324759926 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4a0c104d60 0x7f4a0c19a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:13.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:13.563+0000 7f4a11200640 1 -- 172.21.15.77:0/2324759926 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a0c103b70 msgr2=0x7f4a0c199ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:13.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:13.563+0000 7f4a11200640 1 --2- 172.21.15.77:0/2324759926 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a0c103b70 0x7f4a0c199ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:13.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:13.563+0000 7f4a11200640 1 -- 172.21.15.77:0/2324759926 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4a0c104d60 msgr2=0x7f4a0c19a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:13.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:13.563+0000 7f4a11200640 1 --2- 172.21.15.77:0/2324759926 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4a0c104d60 0x7f4a0c19a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:13.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:13.563+0000 7f4a11200640 1 -- 172.21.15.77:0/2324759926 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a0c1a1ac0 con 0x7f4a0c105720 2024-09-10T09:55:13.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:13.563+0000 7f4a03e00640 1 --2- 172.21.15.77:0/2324759926 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4a0c104d60 0x7f4a0c19a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:55:13.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:13.563+0000 7f4a11200640 1 --2- 172.21.15.77:0/2324759926 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4a0c105720 0x7f4a0c1a1450 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f4a0400eb10 tx=0x7f4a0400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:13.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:13.563+0000 7f4a09600640 1 -- 172.21.15.77:0/2324759926 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a04004b20 con 0x7f4a0c105720 2024-09-10T09:55:13.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:13.563+0000 7f4a11c00640 1 -- 172.21.15.77:0/2324759926 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a0c1a1db0 con 0x7f4a0c105720 2024-09-10T09:55:13.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:13.563+0000 7f4a09600640 1 -- 172.21.15.77:0/2324759926 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a04004cc0 con 0x7f4a0c105720 2024-09-10T09:55:13.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:13.563+0000 7f4a09600640 1 -- 172.21.15.77:0/2324759926 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a04010700 con 0x7f4a0c105720 2024-09-10T09:55:13.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:13.563+0000 7f4a11c00640 1 -- 172.21.15.77:0/2324759926 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a0c1a2220 con 0x7f4a0c105720 2024-09-10T09:55:13.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:13.564+0000 7f4a11c00640 1 -- 172.21.15.77:0/2324759926 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a0c10af60 con 0x7f4a0c105720 2024-09-10T09:55:13.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:13.566+0000 7f4a09600640 1 -- 172.21.15.77:0/2324759926 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4a04005000 con 0x7f4a0c105720 2024-09-10T09:55:13.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:13.567+0000 7f4a09600640 1 --2- 172.21.15.77:0/2324759926 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f49f00a2b80 0x7f49f00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:13.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:13.568+0000 7f4a09600640 1 -- 172.21.15.77:0/2324759926 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f4a040cfc00 con 0x7f4a0c105720 2024-09-10T09:55:13.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:13.568+0000 7f4a0be00640 1 --2- 172.21.15.77:0/2324759926 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f49f00a2b80 0x7f49f00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:13.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:13.569+0000 7f4a0be00640 1 --2- 172.21.15.77:0/2324759926 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f49f00a2b80 0x7f49f00a5040 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c104bc0 tx=0x7f49fc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:13.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:13.572+0000 7f4a09600640 1 -- 172.21.15.77:0/2324759926 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4a0409c710 con 0x7f4a0c105720 2024-09-10T09:55:13.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:13.785+0000 7f4a11c00640 1 -- 172.21.15.77:0/2324759926 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7f4a0c062df0 con 0x7f4a0c105720 2024-09-10T09:55:13.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:13.868+0000 7f4a09600640 1 -- 172.21.15.77:0/2324759926 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v29) ==== 117+0+0 (secure 0 0 0) 0x7f4a040bc0d0 con 0x7f4a0c105720 2024-09-10T09:55:13.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:13.871+0000 7f4a11c00640 1 -- 172.21.15.77:0/2324759926 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f49f00a2b80 msgr2=0x7f49f00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:13.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:13.871+0000 7f4a11c00640 1 --2- 172.21.15.77:0/2324759926 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f49f00a2b80 0x7f49f00a5040 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c104bc0 tx=0x7f49fc005f90 comp rx=0 tx=0).stop 2024-09-10T09:55:13.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:13.871+0000 7f4a11c00640 1 -- 172.21.15.77:0/2324759926 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4a0c105720 msgr2=0x7f4a0c1a1450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:13.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:13.871+0000 7f4a11c00640 1 --2- 172.21.15.77:0/2324759926 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4a0c105720 0x7f4a0c1a1450 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f4a0400eb10 tx=0x7f4a0400c2d0 comp rx=0 tx=0).stop 2024-09-10T09:55:13.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:13.871+0000 7f4a11c00640 1 -- 172.21.15.77:0/2324759926 shutdown_connections 2024-09-10T09:55:13.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:13.871+0000 7f4a11c00640 1 --2- 172.21.15.77:0/2324759926 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f49f00a2b80 0x7f49f00a5040 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:13.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:13.871+0000 7f4a11c00640 1 --2- 172.21.15.77:0/2324759926 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4a0c105720 0x7f4a0c1a1450 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:13.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:13.871+0000 7f4a11c00640 1 --2- 172.21.15.77:0/2324759926 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4a0c104d60 0x7f4a0c19a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:13.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:13.871+0000 7f4a11c00640 1 --2- 172.21.15.77:0/2324759926 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a0c103b70 0x7f4a0c199ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:13.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:13.871+0000 7f4a11c00640 1 -- 172.21.15.77:0/2324759926 >> 172.21.15.77:0/2324759926 conn(0x7f4a0c0ff310 msgr2=0x7f4a0c108130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:13.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:13.871+0000 7f4a11c00640 1 -- 172.21.15.77:0/2324759926 shutdown_connections 2024-09-10T09:55:13.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:13.871+0000 7f4a11c00640 1 -- 172.21.15.77:0/2324759926 wait complete. 2024-09-10T09:55:13.890 INFO:teuthology.run_tasks:Running task check-counter... 2024-09-10T09:55:13.901 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-09-10T09:55:13.911 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T09:55:14.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.065+0000 7f87c9000640 1 -- 172.21.15.77:0/717461921 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87c4101f40 msgr2=0x7f87c4109db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:14.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.065+0000 7f87c9000640 1 --2- 172.21.15.77:0/717461921 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87c4101f40 0x7f87c4109db0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f87bc009ca0 tx=0x7f87bc02a8c0 comp rx=0 tx=0).stop 2024-09-10T09:55:14.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.065+0000 7f87c9000640 1 -- 172.21.15.77:0/717461921 shutdown_connections 2024-09-10T09:55:14.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.065+0000 7f87c9000640 1 --2- 172.21.15.77:0/717461921 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87c4101f40 0x7f87c4109db0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:14.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.065+0000 7f87c9000640 1 --2- 172.21.15.77:0/717461921 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f87c4101580 0x7f87c4101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:14.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.065+0000 7f87c9000640 1 --2- 172.21.15.77:0/717461921 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f87c4103050 0x7f87c4103450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:14.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.065+0000 7f87c9000640 1 -- 172.21.15.77:0/717461921 >> 172.21.15.77:0/717461921 conn(0x7f87c40fd110 msgr2=0x7f87c40ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:14.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.065+0000 7f87c9000640 1 -- 172.21.15.77:0/717461921 shutdown_connections 2024-09-10T09:55:14.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.065+0000 7f87c9000640 1 -- 172.21.15.77:0/717461921 wait complete. 2024-09-10T09:55:14.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.066+0000 7f87c9000640 1 Processor -- start 2024-09-10T09:55:14.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.066+0000 7f87c9000640 1 -- start start 2024-09-10T09:55:14.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.066+0000 7f87c9000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f87c4101580 0x7f87c4075fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:14.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.066+0000 7f87c9000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87c4101f40 0x7f87c40745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:14.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.066+0000 7f87c9000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f87c4103050 0x7f87c4074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:14.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.066+0000 7f87c9000640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f87c410d6f0 con 0x7f87c4101f40 2024-09-10T09:55:14.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.066+0000 7f87c9000640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f87c410d570 con 0x7f87c4101580 2024-09-10T09:55:14.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.066+0000 7f87c9000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f87c410d870 con 0x7f87c4103050 2024-09-10T09:55:14.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.066+0000 7f87c2a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87c4101f40 0x7f87c40745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:14.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.066+0000 7f87c3400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f87c4101580 0x7f87c4075fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:14.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.066+0000 7f87c2a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87c4101f40 0x7f87c40745e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52626/0 (socket says 172.21.15.77:52626) 2024-09-10T09:55:14.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.066+0000 7f87c2a00640 1 -- 172.21.15.77:0/2532794997 learned_addr learned my addr 172.21.15.77:0/2532794997 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:55:14.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.066+0000 7f87c3e00640 1 --2- 172.21.15.77:0/2532794997 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f87c4103050 0x7f87c4074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:14.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.067+0000 7f87c3400640 1 -- 172.21.15.77:0/2532794997 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f87c4103050 msgr2=0x7f87c4074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:14.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.067+0000 7f87c3400640 1 --2- 172.21.15.77:0/2532794997 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f87c4103050 0x7f87c4074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:14.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.067+0000 7f87c3400640 1 -- 172.21.15.77:0/2532794997 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87c4101f40 msgr2=0x7f87c40745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:14.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.067+0000 7f87c3400640 1 --2- 172.21.15.77:0/2532794997 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87c4101f40 0x7f87c40745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:14.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.067+0000 7f87c3400640 1 -- 172.21.15.77:0/2532794997 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87c40753b0 con 0x7f87c4101580 2024-09-10T09:55:14.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.067+0000 7f87c2a00640 1 --2- 172.21.15.77:0/2532794997 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87c4101f40 0x7f87c40745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:55:14.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.067+0000 7f87c3400640 1 --2- 172.21.15.77:0/2532794997 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f87c4101580 0x7f87c4075fa0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f87b400e9c0 tx=0x7f87b400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:14.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.067+0000 7f87bbe00640 1 -- 172.21.15.77:0/2532794997 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b400ccc0 con 0x7f87c4101580 2024-09-10T09:55:14.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.067+0000 7f87bbe00640 1 -- 172.21.15.77:0/2532794997 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87b40035f0 con 0x7f87c4101580 2024-09-10T09:55:14.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.067+0000 7f87c9000640 1 -- 172.21.15.77:0/2532794997 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87c41a1e60 con 0x7f87c4101580 2024-09-10T09:55:14.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.068+0000 7f87bbe00640 1 -- 172.21.15.77:0/2532794997 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b400ce60 con 0x7f87c4101580 2024-09-10T09:55:14.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.068+0000 7f87c9000640 1 -- 172.21.15.77:0/2532794997 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87c41a2270 con 0x7f87c4101580 2024-09-10T09:55:14.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.069+0000 7f87c9000640 1 -- 172.21.15.77:0/2532794997 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87c4108db0 con 0x7f87c4101580 2024-09-10T09:55:14.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.071+0000 7f87bbe00640 1 -- 172.21.15.77:0/2532794997 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f87b40037a0 con 0x7f87c4101580 2024-09-10T09:55:14.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.072+0000 7f87bbe00640 1 --2- 172.21.15.77:0/2532794997 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f87a80a2b90 0x7f87a80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:14.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.072+0000 7f87bbe00640 1 -- 172.21.15.77:0/2532794997 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f87b40cf8d0 con 0x7f87c4101580 2024-09-10T09:55:14.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.072+0000 7f87c2a00640 1 --2- 172.21.15.77:0/2532794997 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f87a80a2b90 0x7f87a80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:14.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.073+0000 7f87c2a00640 1 --2- 172.21.15.77:0/2532794997 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f87a80a2b90 0x7f87a80a5050 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f87c4075d10 tx=0x7f87b000b4a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:14.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.074+0000 7f87bbe00640 1 -- 172.21.15.77:0/2532794997 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f87b409c3e0 con 0x7f87c4101580 2024-09-10T09:55:14.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:55:13 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/2324759926' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-10T09:55:14.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:55:13 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/2324759926' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-10T09:55:14.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:55:13 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/2324759926' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-10T09:55:14.252 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.251+0000 7f87c9000640 1 -- 172.21.15.77:0/2532794997 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f87c402d990 con 0x7f87c4101580 2024-09-10T09:55:14.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.254+0000 7f87bbe00640 1 -- 172.21.15.77:0/2532794997 <== mon.2 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+24941 (secure 0 0 0) 0x7f87b409bce0 con 0x7f87c4101580 2024-09-10T09:55:14.255 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:55:14.255 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":96,"fsid":"14a55226-6f59-11ef-bcea-c7b262605968","created":"2024-09-10T09:44:07.359542+0000","modified":"2024-09-10T09:54:00.837074+0000","last_up_change":"2024-09-10T09:51:26.731971+0000","last_in_change":"2024-09-10T09:51:03.857873+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:47:24.174660+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T09:52:18.766738+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T09:52:21.284745+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T09:52:29.846594+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8d46f634-114c-4926-afdc-f8fa37e40336","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":1509649079},{"type":"v1","addr":"172.21.15.77:6803","nonce":1509649079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":1509649079},{"type":"v1","addr":"172.21.15.77:6805","nonce":1509649079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":1509649079},{"type":"v1","addr":"172.21.15.77:6809","nonce":1509649079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":1509649079},{"type":"v1","addr":"172.21.15.77:6807","nonce":1509649079}]},"public_addr":"172.21.15.77:6803/1509649079","cluster_addr":"172.21.15.77:6805/1509649079","heartbeat_back_addr":"172.21.15.77:6809/1509649079","heartbeat_front_addr":"172.21.15.77:6807/1509649079","state":["exists","up"]},{"osd":1,"uuid":"b6570764-736a-49e4-877a-1965e5812d1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":4243920698},{"type":"v1","addr":"172.21.15.120:6801","nonce":4243920698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":4243920698},{"type":"v1","addr":"172.21.15.120:6803","nonce":4243920698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":4243920698},{"type":"v1","addr":"172.21.15.120:6807","nonce":4243920698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":4243920698},{"type":"v1","addr":"172.21.15.120:6805","nonce":4243920698}]},"public_addr":"172.21.15.120:6801/4243920698","cluster_addr":"172.21.15.120:6803/4243920698","heartbeat_back_addr":"172.21.15.120:6807/4243920698","heartbeat_front_addr":"172.21.15.120:6805/4243920698","state":["exists","up"]},{"osd":2,"uuid":"5e83fe13-d1ee-483e-a575-e8041ec836f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":3280466823},{"type":"v1","addr":"172.21.15.134:6801","nonce":3280466823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":3280466823},{"type":"v1","addr":"172.21.15.134:6803","nonce":3280466823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":3280466823},{"type":"v1","addr":"172.21.15.134:6807","nonce":3280466823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":3280466823},{"type":"v1","addr":"172.21.15.134:6805","nonce":3280466823}]},"public_addr":"172.21.15.134:6801/3280466823","cluster_addr":"172.21.15.134:6803/3280466823","heartbeat_back_addr":"172.21.15.134:6807/3280466823","heartbeat_front_addr":"172.21.15.134:6805/3280466823","state":["exists","up"]},{"osd":3,"uuid":"5affc6a7-1cae-48fe-b7db-befc2fcafecb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":1629416786},{"type":"v1","addr":"172.21.15.77:6811","nonce":1629416786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":1629416786},{"type":"v1","addr":"172.21.15.77:6813","nonce":1629416786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":1629416786},{"type":"v1","addr":"172.21.15.77:6817","nonce":1629416786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":1629416786},{"type":"v1","addr":"172.21.15.77:6815","nonce":1629416786}]},"public_addr":"172.21.15.77:6811/1629416786","cluster_addr":"172.21.15.77:6813/1629416786","heartbeat_back_addr":"172.21.15.77:6817/1629416786","heartbeat_front_addr":"172.21.15.77:6815/1629416786","state":["exists","up"]},{"osd":4,"uuid":"c826978f-40b0-43ae-adbb-5c93b32a68ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":3162658038},{"type":"v1","addr":"172.21.15.120:6809","nonce":3162658038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":3162658038},{"type":"v1","addr":"172.21.15.120:6811","nonce":3162658038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":3162658038},{"type":"v1","addr":"172.21.15.120:6815","nonce":3162658038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":3162658038},{"type":"v1","addr":"172.21.15.120:6813","nonce":3162658038}]},"public_addr":"172.21.15.120:6809/3162658038","cluster_addr":"172.21.15.120:6811/3162658038","heartbeat_back_addr":"172.21.15.120:6815/3162658038","heartbeat_front_addr":"172.21.15.120:6813/3162658038","state":["exists","up"]},{"osd":5,"uuid":"be51c7fb-2765-46a3-9861-c950f63de148","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":4081444621},{"type":"v1","addr":"172.21.15.134:6809","nonce":4081444621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":4081444621},{"type":"v1","addr":"172.21.15.134:6811","nonce":4081444621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":4081444621},{"type":"v1","addr":"172.21.15.134:6815","nonce":4081444621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":4081444621},{"type":"v1","addr":"172.21.15.134:6813","nonce":4081444621}]},"public_addr":"172.21.15.134:6809/4081444621","cluster_addr":"172.21.15.134:6811/4081444621","heartbeat_back_addr":"172.21.15.134:6815/4081444621","heartbeat_front_addr":"172.21.15.134:6813/4081444621","state":["exists","up"]},{"osd":6,"uuid":"4eb403ab-473b-41cc-803d-18c908f768a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":879565639},{"type":"v1","addr":"172.21.15.77:6819","nonce":879565639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":879565639},{"type":"v1","addr":"172.21.15.77:6821","nonce":879565639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":879565639},{"type":"v1","addr":"172.21.15.77:6825","nonce":879565639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":879565639},{"type":"v1","addr":"172.21.15.77:6823","nonce":879565639}]},"public_addr":"172.21.15.77:6819/879565639","cluster_addr":"172.21.15.77:6821/879565639","heartbeat_back_addr":"172.21.15.77:6825/879565639","heartbeat_front_addr":"172.21.15.77:6823/879565639","state":["exists","up"]},{"osd":7,"uuid":"ba982bd0-c236-456c-9889-61087d79c1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":2688667400},{"type":"v1","addr":"172.21.15.120:6817","nonce":2688667400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":2688667400},{"type":"v1","addr":"172.21.15.120:6819","nonce":2688667400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":2688667400},{"type":"v1","addr":"172.21.15.120:6823","nonce":2688667400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2688667400},{"type":"v1","addr":"172.21.15.120:6821","nonce":2688667400}]},"public_addr":"172.21.15.120:6817/2688667400","cluster_addr":"172.21.15.120:6819/2688667400","heartbeat_back_addr":"172.21.15.120:6823/2688667400","heartbeat_front_addr":"172.21.15.120:6821/2688667400","state":["exists","up"]},{"osd":8,"uuid":"f2e8ca43-57c9-4933-b5f7-73f4d1b3e08b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2081686544},{"type":"v1","addr":"172.21.15.134:6817","nonce":2081686544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2081686544},{"type":"v1","addr":"172.21.15.134:6819","nonce":2081686544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2081686544},{"type":"v1","addr":"172.21.15.134:6823","nonce":2081686544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2081686544},{"type":"v1","addr":"172.21.15.134:6821","nonce":2081686544}]},"public_addr":"172.21.15.134:6817/2081686544","cluster_addr":"172.21.15.134:6819/2081686544","heartbeat_back_addr":"172.21.15.134:6823/2081686544","heartbeat_front_addr":"172.21.15.134:6821/2081686544","state":["exists","up"]},{"osd":9,"uuid":"9486f8b5-0614-4208-87ec-117741f6b622","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":1210619387},{"type":"v1","addr":"172.21.15.77:6827","nonce":1210619387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":1210619387},{"type":"v1","addr":"172.21.15.77:6829","nonce":1210619387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":1210619387},{"type":"v1","addr":"172.21.15.77:6833","nonce":1210619387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":1210619387},{"type":"v1","addr":"172.21.15.77:6831","nonce":1210619387}]},"public_addr":"172.21.15.77:6827/1210619387","cluster_addr":"172.21.15.77:6829/1210619387","heartbeat_back_addr":"172.21.15.77:6833/1210619387","heartbeat_front_addr":"172.21.15.77:6831/1210619387","state":["exists","up"]},{"osd":10,"uuid":"8e8192b0-1398-4d4d-9756-762b2f874d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":754733885},{"type":"v1","addr":"172.21.15.120:6825","nonce":754733885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":754733885},{"type":"v1","addr":"172.21.15.120:6827","nonce":754733885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":754733885},{"type":"v1","addr":"172.21.15.120:6831","nonce":754733885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":754733885},{"type":"v1","addr":"172.21.15.120:6829","nonce":754733885}]},"public_addr":"172.21.15.120:6825/754733885","cluster_addr":"172.21.15.120:6827/754733885","heartbeat_back_addr":"172.21.15.120:6831/754733885","heartbeat_front_addr":"172.21.15.120:6829/754733885","state":["exists","up"]},{"osd":11,"uuid":"85c1ccd4-5147-4995-896e-fe977f73f749","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1602213143},{"type":"v1","addr":"172.21.15.134:6825","nonce":1602213143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1602213143},{"type":"v1","addr":"172.21.15.134:6827","nonce":1602213143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1602213143},{"type":"v1","addr":"172.21.15.134:6831","nonce":1602213143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1602213143},{"type":"v1","addr":"172.21.15.134:6829","nonce":1602213143}]},"public_addr":"172.21.15.134:6825/1602213143","cluster_addr":"172.21.15.134:6827/1602213143","heartbeat_back_addr":"172.21.15.134:6831/1602213143","heartbeat_front_addr":"172.21.15.134:6829/1602213143","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:46:30.293451+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:46:56.554801+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:47:21.476225+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:47:49.737332+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:48:16.092181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:48:41.240815+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:49:11.683950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:49:38.859255+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:50:04.105401+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:50:33.847810+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:50:59.901797+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:51:25.241507+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.2","mappings":[{"from":7,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":7}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":5,"to":8}]},{"pgid":"4.6","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.77:0/1697982655":"2024-09-11T09:44:59.575663+0000","172.21.15.77:6801/1527070442":"2024-09-11T09:44:59.575663+0000","172.21.15.77:0/3146688196":"2024-09-11T09:44:32.185342+0000","172.21.15.77:0/371773177":"2024-09-11T09:44:59.575663+0000","172.21.15.77:0/1925456910":"2024-09-11T09:44:59.575663+0000","172.21.15.77:0/138545832":"2024-09-11T09:44:32.185342+0000","172.21.15.77:6801/3235900503":"2024-09-11T09:44:32.185342+0000","172.21.15.77:6800/1527070442":"2024-09-11T09:44:59.575663+0000","172.21.15.77:6800/3235900503":"2024-09-11T09:44:32.185342+0000","172.21.15.77:0/1647092548":"2024-09-11T09:44:32.185342+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T09:55:14.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.255+0000 7f87c9000640 1 -- 172.21.15.77:0/2532794997 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f87a80a2b90 msgr2=0x7f87a80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:14.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.255+0000 7f87c9000640 1 --2- 172.21.15.77:0/2532794997 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f87a80a2b90 0x7f87a80a5050 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f87c4075d10 tx=0x7f87b000b4a0 comp rx=0 tx=0).stop 2024-09-10T09:55:14.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.255+0000 7f87c9000640 1 -- 172.21.15.77:0/2532794997 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f87c4101580 msgr2=0x7f87c4075fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:14.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.255+0000 7f87c9000640 1 --2- 172.21.15.77:0/2532794997 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f87c4101580 0x7f87c4075fa0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f87b400e9c0 tx=0x7f87b400ee80 comp rx=0 tx=0).stop 2024-09-10T09:55:14.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.255+0000 7f87c9000640 1 -- 172.21.15.77:0/2532794997 shutdown_connections 2024-09-10T09:55:14.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.255+0000 7f87c9000640 1 --2- 172.21.15.77:0/2532794997 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f87a80a2b90 0x7f87a80a5050 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:14.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.255+0000 7f87c9000640 1 --2- 172.21.15.77:0/2532794997 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f87c4103050 0x7f87c4074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:14.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.255+0000 7f87c9000640 1 --2- 172.21.15.77:0/2532794997 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87c4101f40 0x7f87c40745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:14.258 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.255+0000 7f87c9000640 1 --2- 172.21.15.77:0/2532794997 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f87c4101580 0x7f87c4075fa0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:14.258 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.255+0000 7f87c9000640 1 -- 172.21.15.77:0/2532794997 >> 172.21.15.77:0/2532794997 conn(0x7f87c40fd110 msgr2=0x7f87c4105f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:14.258 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.255+0000 7f87c9000640 1 -- 172.21.15.77:0/2532794997 shutdown_connections 2024-09-10T09:55:14.258 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.255+0000 7f87c9000640 1 -- 172.21.15.77:0/2532794997 wait complete. 2024-09-10T09:55:14.274 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T09:47:24.174660+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-10T09:52:18.766738+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-10T09:52:21.284745+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-10T09:52:29.846594+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '92', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-10T09:55:14.274 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T09:55:14.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.434+0000 7f349f200640 1 -- 172.21.15.77:0/1114160323 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f34a0100270 msgr2=0x7f34a0100670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:14.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.434+0000 7f349f200640 1 --2- 172.21.15.77:0/1114160323 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f34a0100270 0x7f34a0100670 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f3490009900 tx=0x7f349002a410 comp rx=0 tx=0).stop 2024-09-10T09:55:14.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.434+0000 7f349f200640 1 -- 172.21.15.77:0/1114160323 shutdown_connections 2024-09-10T09:55:14.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.434+0000 7f349f200640 1 --2- 172.21.15.77:0/1114160323 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34a0101550 0x7f34a0101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:14.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.434+0000 7f349f200640 1 --2- 172.21.15.77:0/1114160323 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f34a0100bb0 0x7f34a0101010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:14.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.434+0000 7f349f200640 1 --2- 172.21.15.77:0/1114160323 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f34a0100270 0x7f34a0100670 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:14.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.434+0000 7f349f200640 1 -- 172.21.15.77:0/1114160323 >> 172.21.15.77:0/1114160323 conn(0x7f34a00fc0a0 msgr2=0x7f34a00fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:14.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.434+0000 7f349f200640 1 -- 172.21.15.77:0/1114160323 shutdown_connections 2024-09-10T09:55:14.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.434+0000 7f349f200640 1 -- 172.21.15.77:0/1114160323 wait complete. 2024-09-10T09:55:14.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.435+0000 7f349f200640 1 Processor -- start 2024-09-10T09:55:14.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.435+0000 7f349f200640 1 -- start start 2024-09-10T09:55:14.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.435+0000 7f349f200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f34a0100270 0x7f34a0199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:14.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.435+0000 7f349f200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34a0100bb0 0x7f34a019a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:14.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.435+0000 7f349f200640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f34a0101550 0x7f34a01a1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:14.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.435+0000 7f349f200640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f34a010f8a0 con 0x7f34a0100270 2024-09-10T09:55:14.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.435+0000 7f349f200640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f34a010f720 con 0x7f34a0101550 2024-09-10T09:55:14.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.435+0000 7f349f200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f34a010fa20 con 0x7f34a0100bb0 2024-09-10T09:55:14.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.436+0000 7f349de00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f34a0100270 0x7f34a0199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:14.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.436+0000 7f349de00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f34a0100270 0x7f34a0199f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52632/0 (socket says 172.21.15.77:52632) 2024-09-10T09:55:14.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.436+0000 7f349de00640 1 -- 172.21.15.77:0/1128886523 learned_addr learned my addr 172.21.15.77:0/1128886523 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:55:14.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.436+0000 7f349d400640 1 --2- 172.21.15.77:0/1128886523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34a0100bb0 0x7f34a019a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:14.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.436+0000 7f349e800640 1 --2- 172.21.15.77:0/1128886523 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f34a0101550 0x7f34a01a1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:14.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.436+0000 7f349de00640 1 -- 172.21.15.77:0/1128886523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34a0100bb0 msgr2=0x7f34a019a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:14.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.436+0000 7f349de00640 1 --2- 172.21.15.77:0/1128886523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34a0100bb0 0x7f34a019a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:14.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.436+0000 7f349de00640 1 -- 172.21.15.77:0/1128886523 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f34a0101550 msgr2=0x7f34a01a1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:14.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.436+0000 7f349de00640 1 --2- 172.21.15.77:0/1128886523 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f34a0101550 0x7f34a01a1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:14.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.436+0000 7f349de00640 1 -- 172.21.15.77:0/1128886523 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34a01a1b70 con 0x7f34a0100270 2024-09-10T09:55:14.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.436+0000 7f349de00640 1 --2- 172.21.15.77:0/1128886523 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f34a0100270 0x7f34a0199f50 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f3490031440 tx=0x7f3490032e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:14.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.436+0000 7f3496a00640 1 -- 172.21.15.77:0/1128886523 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f349003f030 con 0x7f34a0100270 2024-09-10T09:55:14.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.437+0000 7f3496a00640 1 -- 172.21.15.77:0/1128886523 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3490003ce0 con 0x7f34a0100270 2024-09-10T09:55:14.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.437+0000 7f349f200640 1 -- 172.21.15.77:0/1128886523 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34a01a1e00 con 0x7f34a0100270 2024-09-10T09:55:14.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.437+0000 7f3496a00640 1 -- 172.21.15.77:0/1128886523 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3490035070 con 0x7f34a0100270 2024-09-10T09:55:14.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.437+0000 7f349f200640 1 -- 172.21.15.77:0/1128886523 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34a01a2260 con 0x7f34a0100270 2024-09-10T09:55:14.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.438+0000 7f349f200640 1 -- 172.21.15.77:0/1128886523 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34a010afc0 con 0x7f34a0100270 2024-09-10T09:55:14.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.441+0000 7f3496a00640 1 -- 172.21.15.77:0/1128886523 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3490042030 con 0x7f34a0100270 2024-09-10T09:55:14.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.443+0000 7f3496a00640 1 --2- 172.21.15.77:0/1128886523 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f34840a2c10 0x7f34840a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:14.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.443+0000 7f3496a00640 1 -- 172.21.15.77:0/1128886523 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f34900ef960 con 0x7f34a0100270 2024-09-10T09:55:14.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.443+0000 7f349d400640 1 --2- 172.21.15.77:0/1128886523 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f34840a2c10 0x7f34840a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:14.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.444+0000 7f349d400640 1 --2- 172.21.15.77:0/1128886523 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f34840a2c10 0x7f34840a50d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f34a019b460 tx=0x7f348c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:14.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.448+0000 7f3496a00640 1 -- 172.21.15.77:0/1128886523 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f34900bbe80 con 0x7f34a0100270 2024-09-10T09:55:14.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.624+0000 7f349f200640 1 -- 172.21.15.77:0/1128886523 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f34a0062e90 con 0x7f34a0100270 2024-09-10T09:55:14.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.625+0000 7f3496a00640 1 -- 172.21.15.77:0/1128886523 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v96) ==== 93+0+10 (secure 0 0 0) 0x7f34900c0180 con 0x7f34a0100270 2024-09-10T09:55:14.627 INFO:teuthology.orchestra.run.smithi077.stdout:pg_num: 1 2024-09-10T09:55:14.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.627+0000 7f349f200640 1 -- 172.21.15.77:0/1128886523 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f34840a2c10 msgr2=0x7f34840a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:14.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.627+0000 7f349f200640 1 --2- 172.21.15.77:0/1128886523 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f34840a2c10 0x7f34840a50d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f34a019b460 tx=0x7f348c00b410 comp rx=0 tx=0).stop 2024-09-10T09:55:14.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.627+0000 7f349f200640 1 -- 172.21.15.77:0/1128886523 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f34a0100270 msgr2=0x7f34a0199f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:14.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.627+0000 7f349f200640 1 --2- 172.21.15.77:0/1128886523 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f34a0100270 0x7f34a0199f50 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f3490031440 tx=0x7f3490032e20 comp rx=0 tx=0).stop 2024-09-10T09:55:14.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.627+0000 7f349f200640 1 -- 172.21.15.77:0/1128886523 shutdown_connections 2024-09-10T09:55:14.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.627+0000 7f349f200640 1 --2- 172.21.15.77:0/1128886523 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f34840a2c10 0x7f34840a50d0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:14.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.627+0000 7f349f200640 1 --2- 172.21.15.77:0/1128886523 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f34a0101550 0x7f34a01a1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:14.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.627+0000 7f349f200640 1 --2- 172.21.15.77:0/1128886523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34a0100bb0 0x7f34a019a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:14.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.627+0000 7f349f200640 1 --2- 172.21.15.77:0/1128886523 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f34a0100270 0x7f34a0199f50 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:14.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.627+0000 7f349f200640 1 -- 172.21.15.77:0/1128886523 >> 172.21.15.77:0/1128886523 conn(0x7f34a00fc0a0 msgr2=0x7f34a00fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:14.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.627+0000 7f349f200640 1 -- 172.21.15.77:0/1128886523 shutdown_connections 2024-09-10T09:55:14.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.627+0000 7f349f200640 1 -- 172.21.15.77:0/1128886523 wait complete. 2024-09-10T09:55:14.645 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-10T09:55:14.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.806+0000 7fd731c00640 1 -- 172.21.15.77:0/3769589980 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd72c076230 msgr2=0x7fd72c074680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:14.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.806+0000 7fd731c00640 1 --2- 172.21.15.77:0/3769589980 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd72c076230 0x7fd72c074680 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd71c009900 tx=0x7fd71c02a410 comp rx=0 tx=0).stop 2024-09-10T09:55:14.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.806+0000 7fd731c00640 1 -- 172.21.15.77:0/3769589980 shutdown_connections 2024-09-10T09:55:14.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.806+0000 7fd731c00640 1 --2- 172.21.15.77:0/3769589980 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd72c10e0f0 0x7fd72c1104e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:14.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.806+0000 7fd731c00640 1 --2- 172.21.15.77:0/3769589980 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd72c074c50 0x7fd72c0750b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:14.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.806+0000 7fd731c00640 1 --2- 172.21.15.77:0/3769589980 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd72c076230 0x7fd72c074680 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:14.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.806+0000 7fd731c00640 1 -- 172.21.15.77:0/3769589980 >> 172.21.15.77:0/3769589980 conn(0x7fd72c0fd2a0 msgr2=0x7fd72c0ff6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:14.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.806+0000 7fd731c00640 1 -- 172.21.15.77:0/3769589980 shutdown_connections 2024-09-10T09:55:14.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.806+0000 7fd731c00640 1 -- 172.21.15.77:0/3769589980 wait complete. 2024-09-10T09:55:14.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.807+0000 7fd731c00640 1 Processor -- start 2024-09-10T09:55:14.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.807+0000 7fd731c00640 1 -- start start 2024-09-10T09:55:14.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.807+0000 7fd731c00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd72c074c50 0x7fd72c19e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:14.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.807+0000 7fd731c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd72c076230 0x7fd72c19e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:14.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.807+0000 7fd731c00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd72c10e0f0 0x7fd72c1a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:14.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.807+0000 7fd731c00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd72c113e00 con 0x7fd72c10e0f0 2024-09-10T09:55:14.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.807+0000 7fd731c00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd72c113c80 con 0x7fd72c074c50 2024-09-10T09:55:14.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.807+0000 7fd731c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd72c113f80 con 0x7fd72c076230 2024-09-10T09:55:14.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.808+0000 7fd72b400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd72c10e0f0 0x7fd72c1a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:14.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.808+0000 7fd72b400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd72c10e0f0 0x7fd72c1a5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52658/0 (socket says 172.21.15.77:52658) 2024-09-10T09:55:14.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.808+0000 7fd72b400640 1 -- 172.21.15.77:0/245356223 learned_addr learned my addr 172.21.15.77:0/245356223 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:55:14.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.808+0000 7fd72aa00640 1 --2- 172.21.15.77:0/245356223 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd72c074c50 0x7fd72c19e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:14.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.808+0000 7fd722000640 1 --2- 172.21.15.77:0/245356223 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd72c076230 0x7fd72c19e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:14.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.808+0000 7fd72b400640 1 -- 172.21.15.77:0/245356223 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd72c076230 msgr2=0x7fd72c19e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:14.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.808+0000 7fd72b400640 1 --2- 172.21.15.77:0/245356223 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd72c076230 0x7fd72c19e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:14.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.808+0000 7fd72b400640 1 -- 172.21.15.77:0/245356223 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd72c074c50 msgr2=0x7fd72c19e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:14.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.808+0000 7fd72b400640 1 --2- 172.21.15.77:0/245356223 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd72c074c50 0x7fd72c19e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:14.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.808+0000 7fd72b400640 1 -- 172.21.15.77:0/245356223 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd72c1a6080 con 0x7fd72c10e0f0 2024-09-10T09:55:14.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.808+0000 7fd72b400640 1 --2- 172.21.15.77:0/245356223 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd72c10e0f0 0x7fd72c1a5a10 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd72400eb10 tx=0x7fd72400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:14.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.809+0000 7fd723e00640 1 -- 172.21.15.77:0/245356223 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd724004b20 con 0x7fd72c10e0f0 2024-09-10T09:55:14.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.809+0000 7fd723e00640 1 -- 172.21.15.77:0/245356223 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd724004cc0 con 0x7fd72c10e0f0 2024-09-10T09:55:14.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.809+0000 7fd731c00640 1 -- 172.21.15.77:0/245356223 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd72c1a6370 con 0x7fd72c10e0f0 2024-09-10T09:55:14.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.809+0000 7fd723e00640 1 -- 172.21.15.77:0/245356223 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd724010760 con 0x7fd72c10e0f0 2024-09-10T09:55:14.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.809+0000 7fd731c00640 1 -- 172.21.15.77:0/245356223 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd72c1a68c0 con 0x7fd72c10e0f0 2024-09-10T09:55:14.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.810+0000 7fd731c00640 1 -- 172.21.15.77:0/245356223 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd72c103a90 con 0x7fd72c10e0f0 2024-09-10T09:55:14.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.812+0000 7fd723e00640 1 -- 172.21.15.77:0/245356223 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd724005020 con 0x7fd72c10e0f0 2024-09-10T09:55:14.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.814+0000 7fd723e00640 1 --2- 172.21.15.77:0/245356223 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fd7100a2c10 0x7fd7100a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:14.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.814+0000 7fd723e00640 1 -- 172.21.15.77:0/245356223 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fd7240cf960 con 0x7fd72c10e0f0 2024-09-10T09:55:14.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.814+0000 7fd72aa00640 1 --2- 172.21.15.77:0/245356223 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fd7100a2c10 0x7fd7100a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:14.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.815+0000 7fd72aa00640 1 --2- 172.21.15.77:0/245356223 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fd7100a2c10 0x7fd7100a50d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd71c031440 tx=0x7fd71c0315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:14.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:14.818+0000 7fd723e00640 1 -- 172.21.15.77:0/245356223 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd72409c4d0 con 0x7fd72c10e0f0 2024-09-10T09:55:15.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.027+0000 7fd731c00640 1 -- 172.21.15.77:0/245356223 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd72c062df0 con 0x7fd72c10e0f0 2024-09-10T09:55:15.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.027+0000 7fd723e00640 1 -- 172.21.15.77:0/245356223 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v96) ==== 104+0+11 (secure 0 0 0) 0x7fd72409bdd0 con 0x7fd72c10e0f0 2024-09-10T09:55:15.029 INFO:teuthology.orchestra.run.smithi077.stdout:pg_num: 32 2024-09-10T09:55:15.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.029+0000 7fd731c00640 1 -- 172.21.15.77:0/245356223 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fd7100a2c10 msgr2=0x7fd7100a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:15.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.029+0000 7fd731c00640 1 --2- 172.21.15.77:0/245356223 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fd7100a2c10 0x7fd7100a50d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd71c031440 tx=0x7fd71c0315b0 comp rx=0 tx=0).stop 2024-09-10T09:55:15.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.029+0000 7fd731c00640 1 -- 172.21.15.77:0/245356223 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd72c10e0f0 msgr2=0x7fd72c1a5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:15.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.029+0000 7fd731c00640 1 --2- 172.21.15.77:0/245356223 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd72c10e0f0 0x7fd72c1a5a10 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd72400eb10 tx=0x7fd72400c2d0 comp rx=0 tx=0).stop 2024-09-10T09:55:15.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.030+0000 7fd731c00640 1 -- 172.21.15.77:0/245356223 shutdown_connections 2024-09-10T09:55:15.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.030+0000 7fd731c00640 1 --2- 172.21.15.77:0/245356223 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fd7100a2c10 0x7fd7100a50d0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:15.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.030+0000 7fd731c00640 1 --2- 172.21.15.77:0/245356223 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd72c10e0f0 0x7fd72c1a5a10 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:15.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.030+0000 7fd731c00640 1 --2- 172.21.15.77:0/245356223 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd72c076230 0x7fd72c19e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:15.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.030+0000 7fd731c00640 1 --2- 172.21.15.77:0/245356223 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd72c074c50 0x7fd72c19e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:15.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.030+0000 7fd731c00640 1 -- 172.21.15.77:0/245356223 >> 172.21.15.77:0/245356223 conn(0x7fd72c0fd2a0 msgr2=0x7fd72c0ff690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:15.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.030+0000 7fd731c00640 1 -- 172.21.15.77:0/245356223 shutdown_connections 2024-09-10T09:55:15.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.030+0000 7fd731c00640 1 -- 172.21.15.77:0/245356223 wait complete. 2024-09-10T09:55:15.047 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-10T09:55:15.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:55:14 smithi134 ceph-mon[35674]: pgmap v393: 129 pgs: 129 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-10T09:55:15.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:55:14 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/2324759926' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-10T09:55:15.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:55:14 smithi134 ceph-mon[35674]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T09:55:15.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:55:14 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/2532794997' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T09:55:15.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:55:14 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/1128886523' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T09:55:15.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:55:14 smithi120 ceph-mon[34770]: pgmap v393: 129 pgs: 129 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-10T09:55:15.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:55:14 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/2324759926' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-10T09:55:15.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:55:14 smithi120 ceph-mon[34770]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T09:55:15.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:55:14 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/2532794997' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T09:55:15.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:55:14 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/1128886523' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T09:55:15.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:55:14 smithi077 ceph-mon[30319]: pgmap v393: 129 pgs: 129 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-10T09:55:15.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:55:14 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/2324759926' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-10T09:55:15.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:55:14 smithi077 ceph-mon[30319]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T09:55:15.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:55:14 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/2532794997' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T09:55:15.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:55:14 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/1128886523' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T09:55:15.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.212+0000 7fde27200640 1 -- 172.21.15.77:0/1839208442 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde20103d30 msgr2=0x7fde20104130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:15.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.212+0000 7fde27200640 1 --2- 172.21.15.77:0/1839208442 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde20103d30 0x7fde20104130 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fde10009860 tx=0x7fde1002a340 comp rx=0 tx=0).stop 2024-09-10T09:55:15.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.212+0000 7fde27200640 1 -- 172.21.15.77:0/1839208442 shutdown_connections 2024-09-10T09:55:15.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.212+0000 7fde27200640 1 --2- 172.21.15.77:0/1839208442 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fde201058e0 0x7fde2010c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:15.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.212+0000 7fde27200640 1 --2- 172.21.15.77:0/1839208442 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde20104f20 0x7fde201053a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:15.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.212+0000 7fde27200640 1 --2- 172.21.15.77:0/1839208442 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde20103d30 0x7fde20104130 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:15.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.212+0000 7fde27200640 1 -- 172.21.15.77:0/1839208442 >> 172.21.15.77:0/1839208442 conn(0x7fde200ff470 msgr2=0x7fde201018d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:15.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.212+0000 7fde27200640 1 -- 172.21.15.77:0/1839208442 shutdown_connections 2024-09-10T09:55:15.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.212+0000 7fde27200640 1 -- 172.21.15.77:0/1839208442 wait complete. 2024-09-10T09:55:15.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.213+0000 7fde27200640 1 Processor -- start 2024-09-10T09:55:15.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.213+0000 7fde27200640 1 -- start start 2024-09-10T09:55:15.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.214+0000 7fde27200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fde20103d30 0x7fde2019a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:15.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.214+0000 7fde27200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde20104f20 0x7fde2019a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:15.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.214+0000 7fde27200640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde201058e0 0x7fde201a16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:15.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.214+0000 7fde27200640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fde2010f9b0 con 0x7fde20103d30 2024-09-10T09:55:15.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.214+0000 7fde27200640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fde2010f830 con 0x7fde201058e0 2024-09-10T09:55:15.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.214+0000 7fde27200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fde2010fb30 con 0x7fde20104f20 2024-09-10T09:55:15.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.214+0000 7fde1fe00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fde20103d30 0x7fde2019a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:15.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.214+0000 7fde1f400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde20104f20 0x7fde2019a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:15.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.214+0000 7fde25200640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde201058e0 0x7fde201a16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:15.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.214+0000 7fde1fe00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fde20103d30 0x7fde2019a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52672/0 (socket says 172.21.15.77:52672) 2024-09-10T09:55:15.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.214+0000 7fde1fe00640 1 -- 172.21.15.77:0/2291698456 learned_addr learned my addr 172.21.15.77:0/2291698456 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:55:15.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.215+0000 7fde25200640 1 -- 172.21.15.77:0/2291698456 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde20104f20 msgr2=0x7fde2019a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:15.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.215+0000 7fde25200640 1 --2- 172.21.15.77:0/2291698456 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde20104f20 0x7fde2019a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:15.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.215+0000 7fde25200640 1 -- 172.21.15.77:0/2291698456 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fde20103d30 msgr2=0x7fde2019a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:15.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.215+0000 7fde25200640 1 --2- 172.21.15.77:0/2291698456 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fde20103d30 0x7fde2019a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:15.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.215+0000 7fde25200640 1 -- 172.21.15.77:0/2291698456 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde201a1d10 con 0x7fde201058e0 2024-09-10T09:55:15.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.215+0000 7fde1f400640 1 --2- 172.21.15.77:0/2291698456 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde20104f20 0x7fde2019a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:55:15.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.215+0000 7fde25200640 1 --2- 172.21.15.77:0/2291698456 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde201058e0 0x7fde201a16a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fde1800efc0 tx=0x7fde1800c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:15.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.215+0000 7fde1cc00640 1 -- 172.21.15.77:0/2291698456 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde18003970 con 0x7fde201058e0 2024-09-10T09:55:15.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.215+0000 7fde1cc00640 1 -- 172.21.15.77:0/2291698456 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde18004020 con 0x7fde201058e0 2024-09-10T09:55:15.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.215+0000 7fde27200640 1 -- 172.21.15.77:0/2291698456 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde201a1fa0 con 0x7fde201058e0 2024-09-10T09:55:15.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.215+0000 7fde1cc00640 1 -- 172.21.15.77:0/2291698456 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde1800b9f0 con 0x7fde201058e0 2024-09-10T09:55:15.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.215+0000 7fde27200640 1 -- 172.21.15.77:0/2291698456 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde201a24c0 con 0x7fde201058e0 2024-09-10T09:55:15.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.216+0000 7fde27200640 1 -- 172.21.15.77:0/2291698456 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde2010b190 con 0x7fde201058e0 2024-09-10T09:55:15.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.218+0000 7fde1cc00640 1 -- 172.21.15.77:0/2291698456 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fde1801d020 con 0x7fde201058e0 2024-09-10T09:55:15.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.219+0000 7fde1cc00640 1 --2- 172.21.15.77:0/2291698456 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fde040a2c10 0x7fde040a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:15.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.220+0000 7fde1cc00640 1 -- 172.21.15.77:0/2291698456 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fde180cf3f0 con 0x7fde201058e0 2024-09-10T09:55:15.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.220+0000 7fde1fe00640 1 --2- 172.21.15.77:0/2291698456 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fde040a2c10 0x7fde040a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:15.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.220+0000 7fde1fe00640 1 --2- 172.21.15.77:0/2291698456 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fde040a2c10 0x7fde040a50d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fde1002a930 tx=0x7fde100058b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:15.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.222+0000 7fde1cc00640 1 -- 172.21.15.77:0/2291698456 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fde1809bf00 con 0x7fde201058e0 2024-09-10T09:55:15.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.405+0000 7fde27200640 1 -- 172.21.15.77:0/2291698456 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fde20062df0 con 0x7fde201058e0 2024-09-10T09:55:15.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.405+0000 7fde1cc00640 1 -- 172.21.15.77:0/2291698456 <== mon.2 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v96) ==== 100+0+11 (secure 0 0 0) 0x7fde1809b800 con 0x7fde201058e0 2024-09-10T09:55:15.407 INFO:teuthology.orchestra.run.smithi077.stdout:pg_num: 64 2024-09-10T09:55:15.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.407+0000 7fde27200640 1 -- 172.21.15.77:0/2291698456 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fde040a2c10 msgr2=0x7fde040a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:15.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.407+0000 7fde27200640 1 --2- 172.21.15.77:0/2291698456 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fde040a2c10 0x7fde040a50d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fde1002a930 tx=0x7fde100058b0 comp rx=0 tx=0).stop 2024-09-10T09:55:15.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.407+0000 7fde27200640 1 -- 172.21.15.77:0/2291698456 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde201058e0 msgr2=0x7fde201a16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:15.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.407+0000 7fde27200640 1 --2- 172.21.15.77:0/2291698456 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde201058e0 0x7fde201a16a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fde1800efc0 tx=0x7fde1800c800 comp rx=0 tx=0).stop 2024-09-10T09:55:15.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.408+0000 7fde27200640 1 -- 172.21.15.77:0/2291698456 shutdown_connections 2024-09-10T09:55:15.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.408+0000 7fde27200640 1 --2- 172.21.15.77:0/2291698456 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fde040a2c10 0x7fde040a50d0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:15.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.408+0000 7fde27200640 1 --2- 172.21.15.77:0/2291698456 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde201058e0 0x7fde201a16a0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:15.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.408+0000 7fde27200640 1 --2- 172.21.15.77:0/2291698456 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde20104f20 0x7fde2019a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:15.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.408+0000 7fde27200640 1 --2- 172.21.15.77:0/2291698456 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fde20103d30 0x7fde2019a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:15.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.408+0000 7fde27200640 1 -- 172.21.15.77:0/2291698456 >> 172.21.15.77:0/2291698456 conn(0x7fde200ff470 msgr2=0x7fde20108360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:15.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.408+0000 7fde27200640 1 -- 172.21.15.77:0/2291698456 shutdown_connections 2024-09-10T09:55:15.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.408+0000 7fde27200640 1 -- 172.21.15.77:0/2291698456 wait complete. 2024-09-10T09:55:15.425 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-10T09:55:15.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.583+0000 7f0e6ee00640 1 -- 172.21.15.77:0/104951973 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0e68076230 msgr2=0x7f0e68074680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:15.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.583+0000 7f0e6ee00640 1 --2- 172.21.15.77:0/104951973 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0e68076230 0x7f0e68074680 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c009900 tx=0x7f0e5c02a410 comp rx=0 tx=0).stop 2024-09-10T09:55:15.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.584+0000 7f0e6ee00640 1 -- 172.21.15.77:0/104951973 shutdown_connections 2024-09-10T09:55:15.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.584+0000 7f0e6ee00640 1 --2- 172.21.15.77:0/104951973 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0e68109870 0x7f0e6810bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:15.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.584+0000 7f0e6ee00640 1 --2- 172.21.15.77:0/104951973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0e68074c50 0x7f0e680750b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:15.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.584+0000 7f0e6ee00640 1 --2- 172.21.15.77:0/104951973 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0e68076230 0x7f0e68074680 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:15.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.584+0000 7f0e6ee00640 1 -- 172.21.15.77:0/104951973 >> 172.21.15.77:0/104951973 conn(0x7f0e680fd180 msgr2=0x7f0e680ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:15.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.584+0000 7f0e6ee00640 1 -- 172.21.15.77:0/104951973 shutdown_connections 2024-09-10T09:55:15.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.584+0000 7f0e6ee00640 1 -- 172.21.15.77:0/104951973 wait complete. 2024-09-10T09:55:15.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.585+0000 7f0e6ee00640 1 Processor -- start 2024-09-10T09:55:15.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.585+0000 7f0e6ee00640 1 -- start start 2024-09-10T09:55:15.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.585+0000 7f0e6ee00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0e68074c50 0x7f0e68199f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:15.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.585+0000 7f0e6ee00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0e68076230 0x7f0e6819a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:15.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.585+0000 7f0e6ee00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0e68109870 0x7f0e681a14e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:15.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.585+0000 7f0e6ee00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0e6810f8a0 con 0x7f0e68074c50 2024-09-10T09:55:15.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.585+0000 7f0e6ee00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0e6810f720 con 0x7f0e68076230 2024-09-10T09:55:15.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.585+0000 7f0e6ee00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0e6810fa20 con 0x7f0e68109870 2024-09-10T09:55:15.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.585+0000 7f0e6da00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0e68074c50 0x7f0e68199f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:15.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.585+0000 7f0e6da00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0e68074c50 0x7f0e68199f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52684/0 (socket says 172.21.15.77:52684) 2024-09-10T09:55:15.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.585+0000 7f0e6da00640 1 -- 172.21.15.77:0/3330318158 learned_addr learned my addr 172.21.15.77:0/3330318158 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:55:15.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.585+0000 7f0e6e400640 1 --2- 172.21.15.77:0/3330318158 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0e68109870 0x7f0e681a14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:15.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.586+0000 7f0e6d000640 1 --2- 172.21.15.77:0/3330318158 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0e68076230 0x7f0e6819a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:15.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.586+0000 7f0e6e400640 1 -- 172.21.15.77:0/3330318158 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0e68076230 msgr2=0x7f0e6819a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:15.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.586+0000 7f0e6e400640 1 --2- 172.21.15.77:0/3330318158 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0e68076230 0x7f0e6819a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:15.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.586+0000 7f0e6e400640 1 -- 172.21.15.77:0/3330318158 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0e68074c50 msgr2=0x7f0e68199f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:15.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.586+0000 7f0e6e400640 1 --2- 172.21.15.77:0/3330318158 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0e68074c50 0x7f0e68199f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:15.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.586+0000 7f0e6e400640 1 -- 172.21.15.77:0/3330318158 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e681a1b50 con 0x7f0e68109870 2024-09-10T09:55:15.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.586+0000 7f0e6da00640 1 --2- 172.21.15.77:0/3330318158 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0e68074c50 0x7f0e68199f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:55:15.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.586+0000 7f0e6e400640 1 --2- 172.21.15.77:0/3330318158 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0e68109870 0x7f0e681a14e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0e6000ec50 tx=0x7f0e6000c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:15.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.586+0000 7f0e66000640 1 -- 172.21.15.77:0/3330318158 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e60003b70 con 0x7f0e68109870 2024-09-10T09:55:15.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.586+0000 7f0e66000640 1 -- 172.21.15.77:0/3330318158 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e60003d10 con 0x7f0e68109870 2024-09-10T09:55:15.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.586+0000 7f0e6ee00640 1 -- 172.21.15.77:0/3330318158 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e681a1e40 con 0x7f0e68109870 2024-09-10T09:55:15.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.586+0000 7f0e66000640 1 -- 172.21.15.77:0/3330318158 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e60010890 con 0x7f0e68109870 2024-09-10T09:55:15.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.587+0000 7f0e6ee00640 1 -- 172.21.15.77:0/3330318158 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e681a22b0 con 0x7f0e68109870 2024-09-10T09:55:15.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.587+0000 7f0e6ee00640 1 -- 172.21.15.77:0/3330318158 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e681085f0 con 0x7f0e68109870 2024-09-10T09:55:15.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.590+0000 7f0e66000640 1 -- 172.21.15.77:0/3330318158 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0e60004b40 con 0x7f0e68109870 2024-09-10T09:55:15.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.591+0000 7f0e66000640 1 --2- 172.21.15.77:0/3330318158 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f0e500a2b90 0x7f0e500a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:15.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.591+0000 7f0e66000640 1 -- 172.21.15.77:0/3330318158 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f0e600cfac0 con 0x7f0e68109870 2024-09-10T09:55:15.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.591+0000 7f0e6da00640 1 --2- 172.21.15.77:0/3330318158 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f0e500a2b90 0x7f0e500a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:15.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.592+0000 7f0e6da00640 1 --2- 172.21.15.77:0/3330318158 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f0e500a2b90 0x7f0e500a5050 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c0096c0 tx=0x7f0e5c0315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:15.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.593+0000 7f0e66000640 1 -- 172.21.15.77:0/3330318158 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0e6009c660 con 0x7f0e68109870 2024-09-10T09:55:15.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.774+0000 7f0e6ee00640 1 -- 172.21.15.77:0/3330318158 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f0e6802da10 con 0x7f0e68109870 2024-09-10T09:55:15.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.774+0000 7f0e66000640 1 -- 172.21.15.77:0/3330318158 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v96) ==== 103+0+11 (secure 0 0 0) 0x7f0e6009bf60 con 0x7f0e68109870 2024-09-10T09:55:15.776 INFO:teuthology.orchestra.run.smithi077.stdout:pg_num: 32 2024-09-10T09:55:15.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.776+0000 7f0e6ee00640 1 -- 172.21.15.77:0/3330318158 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f0e500a2b90 msgr2=0x7f0e500a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:15.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.776+0000 7f0e6ee00640 1 --2- 172.21.15.77:0/3330318158 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f0e500a2b90 0x7f0e500a5050 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c0096c0 tx=0x7f0e5c0315b0 comp rx=0 tx=0).stop 2024-09-10T09:55:15.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.776+0000 7f0e6ee00640 1 -- 172.21.15.77:0/3330318158 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0e68109870 msgr2=0x7f0e681a14e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:15.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.776+0000 7f0e6ee00640 1 --2- 172.21.15.77:0/3330318158 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0e68109870 0x7f0e681a14e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0e6000ec50 tx=0x7f0e6000c6a0 comp rx=0 tx=0).stop 2024-09-10T09:55:15.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.777+0000 7f0e6ee00640 1 -- 172.21.15.77:0/3330318158 shutdown_connections 2024-09-10T09:55:15.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.777+0000 7f0e6ee00640 1 --2- 172.21.15.77:0/3330318158 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f0e500a2b90 0x7f0e500a5050 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:15.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.777+0000 7f0e6ee00640 1 --2- 172.21.15.77:0/3330318158 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0e68109870 0x7f0e681a14e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:15.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.777+0000 7f0e6ee00640 1 --2- 172.21.15.77:0/3330318158 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0e68076230 0x7f0e6819a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:15.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.777+0000 7f0e6ee00640 1 --2- 172.21.15.77:0/3330318158 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0e68074c50 0x7f0e68199f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:15.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.777+0000 7f0e6ee00640 1 -- 172.21.15.77:0/3330318158 >> 172.21.15.77:0/3330318158 conn(0x7f0e680fd180 msgr2=0x7f0e681057c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:15.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.777+0000 7f0e6ee00640 1 -- 172.21.15.77:0/3330318158 shutdown_connections 2024-09-10T09:55:15.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:15.777+0000 7f0e6ee00640 1 -- 172.21.15.77:0/3330318158 wait complete. 2024-09-10T09:55:15.794 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-09-10T09:55:15.794 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:55:16.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:55:15 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/245356223' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-10T09:55:16.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:55:15 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/2291698456' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-10T09:55:16.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:55:15 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/3330318158' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-10T09:55:16.167 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:55:16.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:55:15 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/245356223' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-10T09:55:16.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:55:15 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/2291698456' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-10T09:55:16.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:55:15 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/3330318158' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-10T09:55:16.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:55:15 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/245356223' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-10T09:55:16.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:55:15 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/2291698456' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-10T09:55:16.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:55:15 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/3330318158' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-10T09:55:16.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:16.747+0000 7fc45ca00640 1 -- 172.21.15.77:0/172205507 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc4581047e0 msgr2=0x7fc458108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:16.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:16.747+0000 7fc45ca00640 1 --2- 172.21.15.77:0/172205507 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc4581047e0 0x7fc458108830 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc440009900 tx=0x7fc44002a3f0 comp rx=0 tx=0).stop 2024-09-10T09:55:16.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:16.747+0000 7fc45ca00640 1 -- 172.21.15.77:0/172205507 shutdown_connections 2024-09-10T09:55:16.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:16.747+0000 7fc45ca00640 1 --2- 172.21.15.77:0/172205507 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc458108f60 0x7fc458110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:16.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:16.747+0000 7fc45ca00640 1 --2- 172.21.15.77:0/172205507 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc4581047e0 0x7fc458108830 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:16.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:16.747+0000 7fc45ca00640 1 --2- 172.21.15.77:0/172205507 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc458103e30 0x7fc458104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:16.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:16.747+0000 7fc45ca00640 1 -- 172.21.15.77:0/172205507 >> 172.21.15.77:0/172205507 conn(0x7fc4580ff8d0 msgr2=0x7fc458101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:16.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:16.747+0000 7fc45ca00640 1 -- 172.21.15.77:0/172205507 shutdown_connections 2024-09-10T09:55:16.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:16.747+0000 7fc45ca00640 1 -- 172.21.15.77:0/172205507 wait complete. 2024-09-10T09:55:16.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:16.748+0000 7fc45ca00640 1 Processor -- start 2024-09-10T09:55:16.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:16.748+0000 7fc45ca00640 1 -- start start 2024-09-10T09:55:16.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:16.749+0000 7fc45ca00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc458103e30 0x7fc45819a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:16.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:16.749+0000 7fc45ca00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4581047e0 0x7fc45819a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:16.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:16.749+0000 7fc45ca00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc458108f60 0x7fc45819ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:16.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:16.749+0000 7fc45ca00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc458114010 con 0x7fc458103e30 2024-09-10T09:55:16.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:16.749+0000 7fc45ca00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc458113e90 con 0x7fc458108f60 2024-09-10T09:55:16.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:16.749+0000 7fc45ca00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc458114190 con 0x7fc4581047e0 2024-09-10T09:55:16.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:16.749+0000 7fc456800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc458103e30 0x7fc45819a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:16.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:16.749+0000 7fc456800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc458103e30 0x7fc45819a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52698/0 (socket says 172.21.15.77:52698) 2024-09-10T09:55:16.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:16.749+0000 7fc456800640 1 -- 172.21.15.77:0/2339218315 learned_addr learned my addr 172.21.15.77:0/2339218315 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:55:16.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:16.749+0000 7fc44fe00640 1 --2- 172.21.15.77:0/2339218315 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4581047e0 0x7fc45819a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:16.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:16.749+0000 7fc457200640 1 --2- 172.21.15.77:0/2339218315 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc458108f60 0x7fc45819ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:16.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:16.750+0000 7fc456800640 1 -- 172.21.15.77:0/2339218315 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4581047e0 msgr2=0x7fc45819a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:16.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:16.750+0000 7fc456800640 1 --2- 172.21.15.77:0/2339218315 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4581047e0 0x7fc45819a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:16.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:16.750+0000 7fc456800640 1 -- 172.21.15.77:0/2339218315 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc458108f60 msgr2=0x7fc45819ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:16.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:16.750+0000 7fc456800640 1 --2- 172.21.15.77:0/2339218315 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc458108f60 0x7fc45819ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:16.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:16.750+0000 7fc456800640 1 -- 172.21.15.77:0/2339218315 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc45819f420 con 0x7fc458103e30 2024-09-10T09:55:16.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:16.750+0000 7fc456800640 1 --2- 172.21.15.77:0/2339218315 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc458103e30 0x7fc45819a470 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fc44800e9f0 tx=0x7fc44800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:16.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:16.750+0000 7fc44f400640 1 -- 172.21.15.77:0/2339218315 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc44800cd80 con 0x7fc458103e30 2024-09-10T09:55:16.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:16.750+0000 7fc45ca00640 1 -- 172.21.15.77:0/2339218315 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc45819f710 con 0x7fc458103e30 2024-09-10T09:55:16.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:16.750+0000 7fc44f400640 1 -- 172.21.15.77:0/2339218315 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc448010910 con 0x7fc458103e30 2024-09-10T09:55:16.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:16.750+0000 7fc44f400640 1 -- 172.21.15.77:0/2339218315 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc448018910 con 0x7fc458103e30 2024-09-10T09:55:16.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:16.750+0000 7fc45ca00640 1 -- 172.21.15.77:0/2339218315 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4581a6ff0 con 0x7fc458103e30 2024-09-10T09:55:16.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:16.752+0000 7fc45ca00640 1 -- 172.21.15.77:0/2339218315 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc458104210 con 0x7fc458103e30 2024-09-10T09:55:16.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:16.753+0000 7fc44f400640 1 -- 172.21.15.77:0/2339218315 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc448010ac0 con 0x7fc458103e30 2024-09-10T09:55:16.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:16.755+0000 7fc44f400640 1 --2- 172.21.15.77:0/2339218315 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fc43c0a2b80 0x7fc43c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:16.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:16.755+0000 7fc44f400640 1 -- 172.21.15.77:0/2339218315 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fc4480cf810 con 0x7fc458103e30 2024-09-10T09:55:16.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:16.755+0000 7fc44fe00640 1 --2- 172.21.15.77:0/2339218315 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fc43c0a2b80 0x7fc43c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:16.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:16.756+0000 7fc44fe00640 1 --2- 172.21.15.77:0/2339218315 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fc43c0a2b80 0x7fc43c0a5040 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc440000c00 tx=0x7fc440005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:16.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:16.759+0000 7fc44f400640 1 -- 172.21.15.77:0/2339218315 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc44809bcd0 con 0x7fc458103e30 2024-09-10T09:55:16.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:55:16 smithi077 ceph-mon[30319]: pgmap v394: 129 pgs: 129 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T09:55:16.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:16.971+0000 7fc45ca00640 1 -- 172.21.15.77:0/2339218315 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc45810c140 con 0x7fc458103e30 2024-09-10T09:55:16.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:16.972+0000 7fc44f400640 1 -- 172.21.15.77:0/2339218315 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11989 (secure 0 0 0) 0x7fc44809ffd0 con 0x7fc458103e30 2024-09-10T09:55:16.974 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:55:16.974 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":29,"btime":"2024-09-10T09:55:13:863715+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3159289675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3159289675},{"type":"v1","addr":"172.21.15.77:6835","nonce":3159289675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1951962341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1951962341},{"type":"v1","addr":"172.21.15.77:6837","nonce":1951962341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:52:23.368252+0000","modified":"2024-09-10T09:55:13.786975+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24563,"mds_1":24524,"mds_2":24520,"mds_3":24502,"mds_4":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14487":{"gid":14487,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/4291121888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":4291121888},{"type":"v1","addr":"172.21.15.77:6839","nonce":4291121888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14499":{"gid":14499,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6841/445284194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":445284194},{"type":"v1","addr":"172.21.15.77:6841","nonce":445284194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1005054423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1005054423},{"type":"v1","addr":"172.21.15.134:6833","nonce":1005054423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6833/3716307744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":3716307744},{"type":"v1","addr":"172.21.15.120:6833","nonce":3716307744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/1319892544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1319892544},{"type":"v1","addr":"172.21.15.120:6835","nonce":1319892544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.134:6835/3293933699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3293933699},{"type":"v1","addr":"172.21.15.134:6835","nonce":3293933699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.134:6837/2461497091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2461497091},{"type":"v1","addr":"172.21.15.134:6837","nonce":2461497091}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":20,"addr":"172.21.15.134:6839/1965117680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1965117680},{"type":"v1","addr":"172.21.15.134:6839","nonce":1965117680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":20,"addr":"172.21.15.120:6837/3033280709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3033280709},{"type":"v1","addr":"172.21.15.120:6837","nonce":3033280709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.120:6839/1777710157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1777710157},{"type":"v1","addr":"172.21.15.120:6839","nonce":1777710157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24563,"qdb_cluster":[24496,24502,24520,24524,24563]},"id":1}]} 2024-09-10T09:55:16.974 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 29 2024-09-10T09:55:16.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:16.974+0000 7fc45ca00640 1 -- 172.21.15.77:0/2339218315 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fc43c0a2b80 msgr2=0x7fc43c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:16.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:16.974+0000 7fc45ca00640 1 --2- 172.21.15.77:0/2339218315 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fc43c0a2b80 0x7fc43c0a5040 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc440000c00 tx=0x7fc440005b50 comp rx=0 tx=0).stop 2024-09-10T09:55:16.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:16.974+0000 7fc45ca00640 1 -- 172.21.15.77:0/2339218315 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc458103e30 msgr2=0x7fc45819a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:16.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:16.974+0000 7fc45ca00640 1 --2- 172.21.15.77:0/2339218315 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc458103e30 0x7fc45819a470 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fc44800e9f0 tx=0x7fc44800eeb0 comp rx=0 tx=0).stop 2024-09-10T09:55:16.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:16.974+0000 7fc45ca00640 1 -- 172.21.15.77:0/2339218315 shutdown_connections 2024-09-10T09:55:16.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:16.974+0000 7fc45ca00640 1 --2- 172.21.15.77:0/2339218315 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fc43c0a2b80 0x7fc43c0a5040 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:16.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:16.974+0000 7fc45ca00640 1 --2- 172.21.15.77:0/2339218315 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc458108f60 0x7fc45819ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:16.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:16.974+0000 7fc45ca00640 1 --2- 172.21.15.77:0/2339218315 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4581047e0 0x7fc45819a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:16.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:16.974+0000 7fc45ca00640 1 --2- 172.21.15.77:0/2339218315 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc458103e30 0x7fc45819a470 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:16.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:16.974+0000 7fc45ca00640 1 -- 172.21.15.77:0/2339218315 >> 172.21.15.77:0/2339218315 conn(0x7fc4580ff8d0 msgr2=0x7fc458100bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:16.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:16.974+0000 7fc45ca00640 1 -- 172.21.15.77:0/2339218315 shutdown_connections 2024-09-10T09:55:16.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:16.974+0000 7fc45ca00640 1 -- 172.21.15.77:0/2339218315 wait complete. 2024-09-10T09:55:17.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:55:16 smithi134 ceph-mon[35674]: pgmap v394: 129 pgs: 129 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T09:55:17.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:55:16 smithi120 ceph-mon[34770]: pgmap v394: 129 pgs: 129 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T09:55:17.790 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-09-10T09:55:17.790 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-09-10T09:55:17.790 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T09:55:17.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:17.950+0000 7f4107e00640 1 -- 172.21.15.77:0/1757489879 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4108102b60 msgr2=0x7f4108102fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:17.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:17.950+0000 7f4107e00640 1 --2- 172.21.15.77:0/1757489879 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4108102b60 0x7f4108102fe0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f40f4009900 tx=0x7f40f402a3f0 comp rx=0 tx=0).stop 2024-09-10T09:55:17.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:17.950+0000 7f4107e00640 1 -- 172.21.15.77:0/1757489879 shutdown_connections 2024-09-10T09:55:17.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:17.950+0000 7f4107e00640 1 --2- 172.21.15.77:0/1757489879 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4108103520 0x7f4108109db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:17.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:17.950+0000 7f4107e00640 1 --2- 172.21.15.77:0/1757489879 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4108102b60 0x7f4108102fe0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:17.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:17.950+0000 7f4107e00640 1 --2- 172.21.15.77:0/1757489879 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4108101970 0x7f4108101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:17.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:17.950+0000 7f4107e00640 1 -- 172.21.15.77:0/1757489879 >> 172.21.15.77:0/1757489879 conn(0x7f41080fd110 msgr2=0x7f41080ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:17.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:17.950+0000 7f4107e00640 1 -- 172.21.15.77:0/1757489879 shutdown_connections 2024-09-10T09:55:17.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:17.950+0000 7f4107e00640 1 -- 172.21.15.77:0/1757489879 wait complete. 2024-09-10T09:55:17.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:17.951+0000 7f4107e00640 1 Processor -- start 2024-09-10T09:55:17.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:17.951+0000 7f4107e00640 1 -- start start 2024-09-10T09:55:17.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:17.951+0000 7f4107e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4108101970 0x7f4108199e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:17.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:17.952+0000 7f4107e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4108102b60 0x7f410819a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:17.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:17.952+0000 7f4107e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4108103520 0x7f41081a13e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:17.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:17.952+0000 7f4107e00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f410810d640 con 0x7f4108101970 2024-09-10T09:55:17.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:17.952+0000 7f4107e00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f410810d4c0 con 0x7f4108102b60 2024-09-10T09:55:17.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:17.952+0000 7f4107e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f410810d7c0 con 0x7f4108103520 2024-09-10T09:55:17.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:17.952+0000 7f4106a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4108101970 0x7f4108199e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:17.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:17.952+0000 7f4106000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4108102b60 0x7f410819a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:17.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:17.952+0000 7f4106000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4108102b60 0x7f410819a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:39222/0 (socket says 172.21.15.77:39222) 2024-09-10T09:55:17.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:17.952+0000 7f4106a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4108101970 0x7f4108199e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52714/0 (socket says 172.21.15.77:52714) 2024-09-10T09:55:17.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:17.952+0000 7f4106000640 1 -- 172.21.15.77:0/3646063138 learned_addr learned my addr 172.21.15.77:0/3646063138 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:55:17.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:17.952+0000 7f4107400640 1 --2- 172.21.15.77:0/3646063138 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4108103520 0x7f41081a13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:17.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:17.952+0000 7f4106a00640 1 -- 172.21.15.77:0/3646063138 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4108103520 msgr2=0x7f41081a13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:17.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:17.952+0000 7f4106a00640 1 --2- 172.21.15.77:0/3646063138 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4108103520 0x7f41081a13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:17.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:17.952+0000 7f4106a00640 1 -- 172.21.15.77:0/3646063138 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4108102b60 msgr2=0x7f410819a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:17.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:17.952+0000 7f4106a00640 1 --2- 172.21.15.77:0/3646063138 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4108102b60 0x7f410819a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:17.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:17.952+0000 7f4106a00640 1 -- 172.21.15.77:0/3646063138 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41081a1a50 con 0x7f4108101970 2024-09-10T09:55:17.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:17.953+0000 7f4106a00640 1 --2- 172.21.15.77:0/3646063138 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4108101970 0x7f4108199e30 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f40f800e9c0 tx=0x7f40f800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:17.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:17.953+0000 7f40ff400640 1 -- 172.21.15.77:0/3646063138 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f800ccc0 con 0x7f4108101970 2024-09-10T09:55:17.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:17.953+0000 7f40ff400640 1 -- 172.21.15.77:0/3646063138 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40f80035f0 con 0x7f4108101970 2024-09-10T09:55:17.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:17.953+0000 7f4107e00640 1 -- 172.21.15.77:0/3646063138 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41081a1d40 con 0x7f4108101970 2024-09-10T09:55:17.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:17.953+0000 7f40ff400640 1 -- 172.21.15.77:0/3646063138 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f800ce60 con 0x7f4108101970 2024-09-10T09:55:17.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:17.953+0000 7f4107e00640 1 -- 172.21.15.77:0/3646063138 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41081a21d0 con 0x7f4108101970 2024-09-10T09:55:17.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:17.954+0000 7f4107e00640 1 -- 172.21.15.77:0/3646063138 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41081a2460 con 0x7f4108101970 2024-09-10T09:55:17.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:17.956+0000 7f40ff400640 1 -- 172.21.15.77:0/3646063138 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f40f80037a0 con 0x7f4108101970 2024-09-10T09:55:17.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:17.957+0000 7f40ff400640 1 --2- 172.21.15.77:0/3646063138 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f40ec0a2b90 0x7f40ec0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:17.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:17.957+0000 7f40ff400640 1 -- 172.21.15.77:0/3646063138 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f40f80cf8a0 con 0x7f4108101970 2024-09-10T09:55:17.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:17.957+0000 7f4106000640 1 --2- 172.21.15.77:0/3646063138 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f40ec0a2b90 0x7f40ec0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:17.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:17.958+0000 7f4106000640 1 --2- 172.21.15.77:0/3646063138 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f40ec0a2b90 0x7f40ec0a5050 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f410819b340 tx=0x7f40f40315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:17.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:17.961+0000 7f40ff400640 1 -- 172.21.15.77:0/3646063138 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f40f809c3b0 con 0x7f4108101970 2024-09-10T09:55:18.134 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.133+0000 7f4107e00640 1 -- 172.21.15.77:0/3646063138 --> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f41081a2630 con 0x7f40ec0a2b90 2024-09-10T09:55:18.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.142+0000 7f40ff400640 1 -- 172.21.15.77:0/3646063138 <== mgr.14150 v2:172.21.15.77:6800/4261324650 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340852 (secure 0 0 0) 0x7f41081a2630 con 0x7f40ec0a2b90 2024-09-10T09:55:18.147 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:55:18.149 INFO:teuthology.orchestra.run.smithi077.stderr:dumped all 2024-09-10T09:55:18.151 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.146+0000 7f4107e00640 1 -- 172.21.15.77:0/3646063138 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f40ec0a2b90 msgr2=0x7f40ec0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:18.151 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.146+0000 7f4107e00640 1 --2- 172.21.15.77:0/3646063138 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f40ec0a2b90 0x7f40ec0a5050 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f410819b340 tx=0x7f40f40315b0 comp rx=0 tx=0).stop 2024-09-10T09:55:18.151 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.146+0000 7f4107e00640 1 -- 172.21.15.77:0/3646063138 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4108101970 msgr2=0x7f4108199e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:18.151 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.146+0000 7f4107e00640 1 --2- 172.21.15.77:0/3646063138 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4108101970 0x7f4108199e30 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f40f800e9c0 tx=0x7f40f800ee80 comp rx=0 tx=0).stop 2024-09-10T09:55:18.151 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.146+0000 7f4107e00640 1 -- 172.21.15.77:0/3646063138 shutdown_connections 2024-09-10T09:55:18.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.146+0000 7f4107e00640 1 --2- 172.21.15.77:0/3646063138 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f40ec0a2b90 0x7f40ec0a5050 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:18.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.146+0000 7f4107e00640 1 --2- 172.21.15.77:0/3646063138 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4108103520 0x7f41081a13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:18.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.146+0000 7f4107e00640 1 --2- 172.21.15.77:0/3646063138 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4108102b60 0x7f410819a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:18.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.146+0000 7f4107e00640 1 --2- 172.21.15.77:0/3646063138 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4108101970 0x7f4108199e30 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:18.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.146+0000 7f4107e00640 1 -- 172.21.15.77:0/3646063138 >> 172.21.15.77:0/3646063138 conn(0x7f41080fd110 msgr2=0x7f4108074420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:18.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.146+0000 7f4107e00640 1 -- 172.21.15.77:0/3646063138 shutdown_connections 2024-09-10T09:55:18.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.146+0000 7f4107e00640 1 -- 172.21.15.77:0/3646063138 wait complete. 2024-09-10T09:55:18.162 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:55:17 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/2339218315' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:55:18.164 INFO:teuthology.orchestra.run.smithi077.stdout:{"pg_ready":true,"pg_map":{"version":395,"stamp":"2024-09-10T09:55:17.719147+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":661343,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1337,"num_read_kb":794,"num_write":442,"num_write_kb":3090,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1185162,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":862,"ondisk_log_size":862,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":366472,"kb_used_data":13064,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124575352,"statfs":{"total":1151940427776,"available":1151565160448,"internally_reserved":0,"allocated":13377536,"data_stored":8601523,"data_compressed":504633,"data_compressed_allocated":688128,"data_compressed_original":2875392,"omap_allocated":19143,"internal_metadata":361674041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":105,"num_read_kb":55,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003408"},"pg_stats":[{"pgid":"2.1f","version":"96'14","reported_seq":188,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:55:15.056536+0000","last_change":"2024-09-10T09:53:00.803590+0000","last_active":"2024-09-10T09:55:15.056536+0000","last_peered":"2024-09-10T09:55:15.056536+0000","last_clean":"2024-09-10T09:55:15.056536+0000","last_became_active":"2024-09-10T09:53:00.798189+0000","last_became_peered":"2024-09-10T09:53:00.798189+0000","last_unstale":"2024-09-10T09:55:15.056536+0000","last_undegraded":"2024-09-10T09:55:15.056536+0000","last_fullsized":"2024-09-10T09:55:15.056536+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:18:48.808836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":121,"num_read_kb":121,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"96'5","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.682459+0000","last_change":"2024-09-10T09:52:22.318195+0000","last_active":"2024-09-10T09:54:29.682459+0000","last_peered":"2024-09-10T09:54:29.682459+0000","last_clean":"2024-09-10T09:54:29.682459+0000","last_became_active":"2024-09-10T09:52:22.315905+0000","last_became_peered":"2024-09-10T09:52:22.315905+0000","last_unstale":"2024-09-10T09:54:29.682459+0000","last_undegraded":"2024-09-10T09:54:29.682459+0000","last_fullsized":"2024-09-10T09:54:29.682459+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:53:55.515984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.843988+0000","last_change":"2024-09-10T09:52:31.369780+0000","last_active":"2024-09-10T09:54:00.843988+0000","last_peered":"2024-09-10T09:54:00.843988+0000","last_clean":"2024-09-10T09:54:00.843988+0000","last_became_active":"2024-09-10T09:52:31.369238+0000","last_became_peered":"2024-09-10T09:52:31.369238+0000","last_unstale":"2024-09-10T09:54:00.843988+0000","last_undegraded":"2024-09-10T09:54:00.843988+0000","last_fullsized":"2024-09-10T09:54:00.843988+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:39:05.318493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"94'2","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.847203+0000","last_change":"2024-09-10T09:52:19.912276+0000","last_active":"2024-09-10T09:54:00.847203+0000","last_peered":"2024-09-10T09:54:00.847203+0000","last_clean":"2024-09-10T09:54:00.847203+0000","last_became_active":"2024-09-10T09:52:19.911452+0000","last_became_peered":"2024-09-10T09:52:19.911452+0000","last_unstale":"2024-09-10T09:54:00.847203+0000","last_undegraded":"2024-09-10T09:54:00.847203+0000","last_fullsized":"2024-09-10T09:54:00.847203+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:24:31.262207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"96'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.705105+0000","last_change":"2024-09-10T09:52:22.327641+0000","last_active":"2024-09-10T09:54:29.705105+0000","last_peered":"2024-09-10T09:54:29.705105+0000","last_clean":"2024-09-10T09:54:29.705105+0000","last_became_active":"2024-09-10T09:52:22.326969+0000","last_became_peered":"2024-09-10T09:52:22.326969+0000","last_unstale":"2024-09-10T09:54:29.705105+0000","last_undegraded":"2024-09-10T09:54:29.705105+0000","last_fullsized":"2024-09-10T09:54:29.705105+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:35:27.825534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.951014+0000","last_change":"2024-09-10T09:52:31.372360+0000","last_active":"2024-09-10T09:54:00.951014+0000","last_peered":"2024-09-10T09:54:00.951014+0000","last_clean":"2024-09-10T09:54:00.951014+0000","last_became_active":"2024-09-10T09:52:31.371957+0000","last_became_peered":"2024-09-10T09:52:31.371957+0000","last_unstale":"2024-09-10T09:54:00.951014+0000","last_undegraded":"2024-09-10T09:54:00.951014+0000","last_fullsized":"2024-09-10T09:54:00.951014+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:49:34.776185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"96'13","reported_seq":55,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:51.750986+0000","last_change":"2024-09-10T09:52:19.843532+0000","last_active":"2024-09-10T09:54:51.750986+0000","last_peered":"2024-09-10T09:54:51.750986+0000","last_clean":"2024-09-10T09:54:51.750986+0000","last_became_active":"2024-09-10T09:52:19.843070+0000","last_became_peered":"2024-09-10T09:52:19.843070+0000","last_unstale":"2024-09-10T09:54:51.750986+0000","last_undegraded":"2024-09-10T09:54:51.750986+0000","last_fullsized":"2024-09-10T09:54:51.750986+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:08:02.223373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"96'9","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.692662+0000","last_change":"2024-09-10T09:52:22.316981+0000","last_active":"2024-09-10T09:54:29.692662+0000","last_peered":"2024-09-10T09:54:29.692662+0000","last_clean":"2024-09-10T09:54:29.692662+0000","last_became_active":"2024-09-10T09:52:22.316648+0000","last_became_peered":"2024-09-10T09:52:22.316648+0000","last_unstale":"2024-09-10T09:54:29.692662+0000","last_undegraded":"2024-09-10T09:54:29.692662+0000","last_fullsized":"2024-09-10T09:54:29.692662+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:28:56.742496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.951485+0000","last_change":"2024-09-10T09:52:31.376811+0000","last_active":"2024-09-10T09:54:00.951485+0000","last_peered":"2024-09-10T09:54:00.951485+0000","last_clean":"2024-09-10T09:54:00.951485+0000","last_became_active":"2024-09-10T09:52:31.376007+0000","last_became_peered":"2024-09-10T09:52:31.376007+0000","last_unstale":"2024-09-10T09:54:00.951485+0000","last_undegraded":"2024-09-10T09:54:00.951485+0000","last_fullsized":"2024-09-10T09:54:00.951485+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:29:26.213281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"94'5","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.149772+0000","last_change":"2024-09-10T09:52:19.837848+0000","last_active":"2024-09-10T09:54:01.149772+0000","last_peered":"2024-09-10T09:54:01.149772+0000","last_clean":"2024-09-10T09:54:01.149772+0000","last_became_active":"2024-09-10T09:52:19.837383+0000","last_became_peered":"2024-09-10T09:52:19.837383+0000","last_unstale":"2024-09-10T09:54:01.149772+0000","last_undegraded":"2024-09-10T09:54:01.149772+0000","last_fullsized":"2024-09-10T09:54:01.149772+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:50:42.813665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"96'15","reported_seq":60,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.685844+0000","last_change":"2024-09-10T09:54:01.060407+0000","last_active":"2024-09-10T09:54:29.685844+0000","last_peered":"2024-09-10T09:54:29.685844+0000","last_clean":"2024-09-10T09:54:29.685844+0000","last_became_active":"2024-09-10T09:54:01.060126+0000","last_became_peered":"2024-09-10T09:54:01.060126+0000","last_unstale":"2024-09-10T09:54:29.685844+0000","last_undegraded":"2024-09-10T09:54:29.685844+0000","last_fullsized":"2024-09-10T09:54:29.685844+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:59:29.765375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.951119+0000","last_change":"2024-09-10T09:52:31.370910+0000","last_active":"2024-09-10T09:54:00.951119+0000","last_peered":"2024-09-10T09:54:00.951119+0000","last_clean":"2024-09-10T09:54:00.951119+0000","last_became_active":"2024-09-10T09:52:31.370467+0000","last_became_peered":"2024-09-10T09:52:31.370467+0000","last_unstale":"2024-09-10T09:54:00.951119+0000","last_undegraded":"2024-09-10T09:54:00.951119+0000","last_fullsized":"2024-09-10T09:54:00.951119+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:49:25.037820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"96'5","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.702762+0000","last_change":"2024-09-10T09:52:22.310708+0000","last_active":"2024-09-10T09:54:29.702762+0000","last_peered":"2024-09-10T09:54:29.702762+0000","last_clean":"2024-09-10T09:54:29.702762+0000","last_became_active":"2024-09-10T09:52:22.310387+0000","last_became_peered":"2024-09-10T09:52:22.310387+0000","last_unstale":"2024-09-10T09:54:29.702762+0000","last_undegraded":"2024-09-10T09:54:29.702762+0000","last_fullsized":"2024-09-10T09:54:29.702762+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:35:03.115408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"94'4","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.149453+0000","last_change":"2024-09-10T09:52:19.841417+0000","last_active":"2024-09-10T09:54:01.149453+0000","last_peered":"2024-09-10T09:54:01.149453+0000","last_clean":"2024-09-10T09:54:01.149453+0000","last_became_active":"2024-09-10T09:52:19.841083+0000","last_became_peered":"2024-09-10T09:52:19.841083+0000","last_unstale":"2024-09-10T09:54:01.149453+0000","last_undegraded":"2024-09-10T09:54:01.149453+0000","last_fullsized":"2024-09-10T09:54:01.149453+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:01:19.008345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.951294+0000","last_change":"2024-09-10T09:52:31.367406+0000","last_active":"2024-09-10T09:54:00.951294+0000","last_peered":"2024-09-10T09:54:00.951294+0000","last_clean":"2024-09-10T09:54:00.951294+0000","last_became_active":"2024-09-10T09:52:31.366961+0000","last_became_peered":"2024-09-10T09:52:31.366961+0000","last_unstale":"2024-09-10T09:54:00.951294+0000","last_undegraded":"2024-09-10T09:54:00.951294+0000","last_fullsized":"2024-09-10T09:54:00.951294+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:04:51.714465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"96'8","reported_seq":19,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.705252+0000","last_change":"2024-09-10T09:54:01.249212+0000","last_active":"2024-09-10T09:54:29.705252+0000","last_peered":"2024-09-10T09:54:29.705252+0000","last_clean":"2024-09-10T09:54:29.705252+0000","last_became_active":"2024-09-10T09:54:01.248840+0000","last_became_peered":"2024-09-10T09:54:01.248840+0000","last_unstale":"2024-09-10T09:54:29.705252+0000","last_undegraded":"2024-09-10T09:54:29.705252+0000","last_fullsized":"2024-09-10T09:54:29.705252+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:39:13.167435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,10],"acting":[9,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.a","version":"94'3","reported_seq":19,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.954964+0000","last_change":"2024-09-10T09:53:00.800421+0000","last_active":"2024-09-10T09:54:00.954964+0000","last_peered":"2024-09-10T09:54:00.954964+0000","last_clean":"2024-09-10T09:54:00.954964+0000","last_became_active":"2024-09-10T09:53:00.800026+0000","last_became_peered":"2024-09-10T09:53:00.800026+0000","last_unstale":"2024-09-10T09:54:00.954964+0000","last_undegraded":"2024-09-10T09:54:00.954964+0000","last_fullsized":"2024-09-10T09:54:00.954964+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:57:45.252420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"94'1","reported_seq":30,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.951388+0000","last_change":"2024-09-10T09:52:31.376605+0000","last_active":"2024-09-10T09:54:00.951388+0000","last_peered":"2024-09-10T09:54:00.951388+0000","last_clean":"2024-09-10T09:54:00.951388+0000","last_became_active":"2024-09-10T09:52:31.376083+0000","last_became_peered":"2024-09-10T09:52:31.376083+0000","last_unstale":"2024-09-10T09:54:00.951388+0000","last_undegraded":"2024-09-10T09:54:00.951388+0000","last_fullsized":"2024-09-10T09:54:00.951388+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:05:10.030673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"96'9","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.697029+0000","last_change":"2024-09-10T09:52:22.327303+0000","last_active":"2024-09-10T09:54:29.697029+0000","last_peered":"2024-09-10T09:54:29.697029+0000","last_clean":"2024-09-10T09:54:29.697029+0000","last_became_active":"2024-09-10T09:52:22.326753+0000","last_became_peered":"2024-09-10T09:52:22.326753+0000","last_unstale":"2024-09-10T09:54:29.697029+0000","last_undegraded":"2024-09-10T09:54:29.697029+0000","last_fullsized":"2024-09-10T09:54:29.697029+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:27:26.668151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"94'7","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.845956+0000","last_change":"2024-09-10T09:52:19.836484+0000","last_active":"2024-09-10T09:54:00.845956+0000","last_peered":"2024-09-10T09:54:00.845956+0000","last_clean":"2024-09-10T09:54:00.845956+0000","last_became_active":"2024-09-10T09:52:19.836130+0000","last_became_peered":"2024-09-10T09:52:19.836130+0000","last_unstale":"2024-09-10T09:54:00.845956+0000","last_undegraded":"2024-09-10T09:54:00.845956+0000","last_fullsized":"2024-09-10T09:54:00.845956+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:31:58.731981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.844293+0000","last_change":"2024-09-10T09:52:31.375279+0000","last_active":"2024-09-10T09:54:00.844293+0000","last_peered":"2024-09-10T09:54:00.844293+0000","last_clean":"2024-09-10T09:54:00.844293+0000","last_became_active":"2024-09-10T09:52:31.374181+0000","last_became_peered":"2024-09-10T09:52:31.374181+0000","last_unstale":"2024-09-10T09:54:00.844293+0000","last_undegraded":"2024-09-10T09:54:00.844293+0000","last_fullsized":"2024-09-10T09:54:00.844293+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:54:51.299224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"96'2","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.676489+0000","last_change":"2024-09-10T09:52:22.314817+0000","last_active":"2024-09-10T09:54:29.676489+0000","last_peered":"2024-09-10T09:54:29.676489+0000","last_clean":"2024-09-10T09:54:29.676489+0000","last_became_active":"2024-09-10T09:52:22.314491+0000","last_became_peered":"2024-09-10T09:52:22.314491+0000","last_unstale":"2024-09-10T09:54:29.676489+0000","last_undegraded":"2024-09-10T09:54:29.676489+0000","last_fullsized":"2024-09-10T09:54:29.676489+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:49:41.941667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"94'2","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.988558+0000","last_change":"2024-09-10T09:52:19.911730+0000","last_active":"2024-09-10T09:54:00.988558+0000","last_peered":"2024-09-10T09:54:00.988558+0000","last_clean":"2024-09-10T09:54:00.988558+0000","last_became_active":"2024-09-10T09:52:19.911393+0000","last_became_peered":"2024-09-10T09:52:19.911393+0000","last_unstale":"2024-09-10T09:54:00.988558+0000","last_undegraded":"2024-09-10T09:54:00.988558+0000","last_fullsized":"2024-09-10T09:54:00.988558+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:46:17.943129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.951301+0000","last_change":"2024-09-10T09:52:31.376498+0000","last_active":"2024-09-10T09:54:00.951301+0000","last_peered":"2024-09-10T09:54:00.951301+0000","last_clean":"2024-09-10T09:54:00.951301+0000","last_became_active":"2024-09-10T09:52:31.375966+0000","last_became_peered":"2024-09-10T09:52:31.375966+0000","last_unstale":"2024-09-10T09:54:00.951301+0000","last_undegraded":"2024-09-10T09:54:00.951301+0000","last_fullsized":"2024-09-10T09:54:00.951301+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:08:30.649399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.705573+0000","last_change":"2024-09-10T09:52:22.331611+0000","last_active":"2024-09-10T09:54:29.705573+0000","last_peered":"2024-09-10T09:54:29.705573+0000","last_clean":"2024-09-10T09:54:29.705573+0000","last_became_active":"2024-09-10T09:52:22.331268+0000","last_became_peered":"2024-09-10T09:52:22.331268+0000","last_unstale":"2024-09-10T09:54:29.705573+0000","last_undegraded":"2024-09-10T09:54:29.705573+0000","last_fullsized":"2024-09-10T09:54:29.705573+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:01:05.433994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"94'7","reported_seq":50,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.951927+0000","last_change":"2024-09-10T09:52:19.838387+0000","last_active":"2024-09-10T09:54:00.951927+0000","last_peered":"2024-09-10T09:54:00.951927+0000","last_clean":"2024-09-10T09:54:00.951927+0000","last_became_active":"2024-09-10T09:52:19.837889+0000","last_became_peered":"2024-09-10T09:52:19.837889+0000","last_unstale":"2024-09-10T09:54:00.951927+0000","last_undegraded":"2024-09-10T09:54:00.951927+0000","last_fullsized":"2024-09-10T09:54:00.951927+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:50:00.187243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.844187+0000","last_change":"2024-09-10T09:52:31.363262+0000","last_active":"2024-09-10T09:54:00.844187+0000","last_peered":"2024-09-10T09:54:00.844187+0000","last_clean":"2024-09-10T09:54:00.844187+0000","last_became_active":"2024-09-10T09:52:31.362762+0000","last_became_peered":"2024-09-10T09:52:31.362762+0000","last_unstale":"2024-09-10T09:54:00.844187+0000","last_undegraded":"2024-09-10T09:54:00.844187+0000","last_fullsized":"2024-09-10T09:54:00.844187+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:48:28.275041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"96'5","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.702169+0000","last_change":"2024-09-10T09:52:22.312472+0000","last_active":"2024-09-10T09:54:29.702169+0000","last_peered":"2024-09-10T09:54:29.702169+0000","last_clean":"2024-09-10T09:54:29.702169+0000","last_became_active":"2024-09-10T09:52:22.312117+0000","last_became_peered":"2024-09-10T09:52:22.312117+0000","last_unstale":"2024-09-10T09:54:29.702169+0000","last_undegraded":"2024-09-10T09:54:29.702169+0000","last_fullsized":"2024-09-10T09:54:29.702169+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:39:10.742137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"96'16","reported_seq":57,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:31.212993+0000","last_change":"2024-09-10T09:52:19.835543+0000","last_active":"2024-09-10T09:54:31.212993+0000","last_peered":"2024-09-10T09:54:31.212993+0000","last_clean":"2024-09-10T09:54:31.212993+0000","last_became_active":"2024-09-10T09:52:19.835170+0000","last_became_peered":"2024-09-10T09:52:19.835170+0000","last_unstale":"2024-09-10T09:54:31.212993+0000","last_undegraded":"2024-09-10T09:54:31.212993+0000","last_fullsized":"2024-09-10T09:54:31.212993+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:37.969277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.845447+0000","last_change":"2024-09-10T09:52:31.372398+0000","last_active":"2024-09-10T09:54:00.845447+0000","last_peered":"2024-09-10T09:54:00.845447+0000","last_clean":"2024-09-10T09:54:00.845447+0000","last_became_active":"2024-09-10T09:52:31.371874+0000","last_became_peered":"2024-09-10T09:52:31.371874+0000","last_unstale":"2024-09-10T09:54:00.845447+0000","last_undegraded":"2024-09-10T09:54:00.845447+0000","last_fullsized":"2024-09-10T09:54:00.845447+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:09:45.374320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"96'9","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.700381+0000","last_change":"2024-09-10T09:52:22.319592+0000","last_active":"2024-09-10T09:54:29.700381+0000","last_peered":"2024-09-10T09:54:29.700381+0000","last_clean":"2024-09-10T09:54:29.700381+0000","last_became_active":"2024-09-10T09:52:22.319298+0000","last_became_peered":"2024-09-10T09:52:22.319298+0000","last_unstale":"2024-09-10T09:54:29.700381+0000","last_undegraded":"2024-09-10T09:54:29.700381+0000","last_fullsized":"2024-09-10T09:54:29.700381+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:56:59.787638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"94'4","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.150011+0000","last_change":"2024-09-10T09:52:19.912923+0000","last_active":"2024-09-10T09:54:01.150011+0000","last_peered":"2024-09-10T09:54:01.150011+0000","last_clean":"2024-09-10T09:54:01.150011+0000","last_became_active":"2024-09-10T09:52:19.912268+0000","last_became_peered":"2024-09-10T09:52:19.912268+0000","last_unstale":"2024-09-10T09:54:01.150011+0000","last_undegraded":"2024-09-10T09:54:01.150011+0000","last_fullsized":"2024-09-10T09:54:01.150011+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:06:45.421004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.955091+0000","last_change":"2024-09-10T09:52:31.372761+0000","last_active":"2024-09-10T09:54:00.955091+0000","last_peered":"2024-09-10T09:54:00.955091+0000","last_clean":"2024-09-10T09:54:00.955091+0000","last_became_active":"2024-09-10T09:52:31.372293+0000","last_became_peered":"2024-09-10T09:52:31.372293+0000","last_unstale":"2024-09-10T09:54:00.955091+0000","last_undegraded":"2024-09-10T09:54:00.955091+0000","last_fullsized":"2024-09-10T09:54:00.955091+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:36:41.446810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"96'4","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.688817+0000","last_change":"2024-09-10T09:52:22.326291+0000","last_active":"2024-09-10T09:54:29.688817+0000","last_peered":"2024-09-10T09:54:29.688817+0000","last_clean":"2024-09-10T09:54:29.688817+0000","last_became_active":"2024-09-10T09:52:22.325985+0000","last_became_peered":"2024-09-10T09:52:22.325985+0000","last_unstale":"2024-09-10T09:54:29.688817+0000","last_undegraded":"2024-09-10T09:54:29.688817+0000","last_fullsized":"2024-09-10T09:54:29.688817+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:06:21.871502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"94'1","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.844068+0000","last_change":"2024-09-10T09:52:19.912851+0000","last_active":"2024-09-10T09:54:00.844068+0000","last_peered":"2024-09-10T09:54:00.844068+0000","last_clean":"2024-09-10T09:54:00.844068+0000","last_became_active":"2024-09-10T09:52:19.912200+0000","last_became_peered":"2024-09-10T09:52:19.912200+0000","last_unstale":"2024-09-10T09:54:00.844068+0000","last_undegraded":"2024-09-10T09:54:00.844068+0000","last_fullsized":"2024-09-10T09:54:00.844068+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:00:47.331755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.149311+0000","last_change":"2024-09-10T09:52:31.371329+0000","last_active":"2024-09-10T09:54:01.149311+0000","last_peered":"2024-09-10T09:54:01.149311+0000","last_clean":"2024-09-10T09:54:01.149311+0000","last_became_active":"2024-09-10T09:52:31.370971+0000","last_became_peered":"2024-09-10T09:52:31.370971+0000","last_unstale":"2024-09-10T09:54:01.149311+0000","last_undegraded":"2024-09-10T09:54:01.149311+0000","last_fullsized":"2024-09-10T09:54:01.149311+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:07:12.928421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"96'7","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.686209+0000","last_change":"2024-09-10T09:52:22.328039+0000","last_active":"2024-09-10T09:54:29.686209+0000","last_peered":"2024-09-10T09:54:29.686209+0000","last_clean":"2024-09-10T09:54:29.686209+0000","last_became_active":"2024-09-10T09:52:22.327588+0000","last_became_peered":"2024-09-10T09:52:22.327588+0000","last_unstale":"2024-09-10T09:54:29.686209+0000","last_undegraded":"2024-09-10T09:54:29.686209+0000","last_fullsized":"2024-09-10T09:54:29.686209+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:35:39.219544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"94'5","reported_seq":138,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:55:11.050835+0000","last_change":"2024-09-10T09:53:00.797651+0000","last_active":"2024-09-10T09:55:11.050835+0000","last_peered":"2024-09-10T09:55:11.050835+0000","last_clean":"2024-09-10T09:55:11.050835+0000","last_became_active":"2024-09-10T09:53:00.797285+0000","last_became_peered":"2024-09-10T09:53:00.797285+0000","last_unstale":"2024-09-10T09:55:11.050835+0000","last_undegraded":"2024-09-10T09:55:11.050835+0000","last_fullsized":"2024-09-10T09:55:11.050835+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:18:02.320032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,6,1],"acting":[11,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.841787+0000","last_change":"2024-09-10T09:52:31.379215+0000","last_active":"2024-09-10T09:54:00.841787+0000","last_peered":"2024-09-10T09:54:00.841787+0000","last_clean":"2024-09-10T09:54:00.841787+0000","last_became_active":"2024-09-10T09:52:31.377107+0000","last_became_peered":"2024-09-10T09:52:31.377107+0000","last_unstale":"2024-09-10T09:54:00.841787+0000","last_undegraded":"2024-09-10T09:54:00.841787+0000","last_fullsized":"2024-09-10T09:54:00.841787+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:09:39.474077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"96'8","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.673518+0000","last_change":"2024-09-10T09:52:22.316474+0000","last_active":"2024-09-10T09:54:29.673518+0000","last_peered":"2024-09-10T09:54:29.673518+0000","last_clean":"2024-09-10T09:54:29.673518+0000","last_became_active":"2024-09-10T09:52:22.316054+0000","last_became_peered":"2024-09-10T09:52:22.316054+0000","last_unstale":"2024-09-10T09:54:29.673518+0000","last_undegraded":"2024-09-10T09:54:29.673518+0000","last_fullsized":"2024-09-10T09:54:29.673518+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:11:31.334780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"96'6","reported_seq":166,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:55:12.082651+0000","last_change":"2024-09-10T09:52:19.911601+0000","last_active":"2024-09-10T09:55:12.082651+0000","last_peered":"2024-09-10T09:55:12.082651+0000","last_clean":"2024-09-10T09:55:12.082651+0000","last_became_active":"2024-09-10T09:52:19.911268+0000","last_became_peered":"2024-09-10T09:52:19.911268+0000","last_unstale":"2024-09-10T09:55:12.082651+0000","last_undegraded":"2024-09-10T09:55:12.082651+0000","last_fullsized":"2024-09-10T09:55:12.082651+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:46:38.158784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":118,"num_read_kb":118,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.841796+0000","last_change":"2024-09-10T09:52:31.375938+0000","last_active":"2024-09-10T09:54:00.841796+0000","last_peered":"2024-09-10T09:54:00.841796+0000","last_clean":"2024-09-10T09:54:00.841796+0000","last_became_active":"2024-09-10T09:52:31.375536+0000","last_became_peered":"2024-09-10T09:52:31.375536+0000","last_unstale":"2024-09-10T09:54:00.841796+0000","last_undegraded":"2024-09-10T09:54:00.841796+0000","last_fullsized":"2024-09-10T09:54:00.841796+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:55:17.303809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"96'5","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.688934+0000","last_change":"2024-09-10T09:52:22.315912+0000","last_active":"2024-09-10T09:54:29.688934+0000","last_peered":"2024-09-10T09:54:29.688934+0000","last_clean":"2024-09-10T09:54:29.688934+0000","last_became_active":"2024-09-10T09:52:22.315433+0000","last_became_peered":"2024-09-10T09:52:22.315433+0000","last_unstale":"2024-09-10T09:54:29.688934+0000","last_undegraded":"2024-09-10T09:54:29.688934+0000","last_fullsized":"2024-09-10T09:54:29.688934+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:57:10.454716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"94'4","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.059138+0000","last_change":"2024-09-10T09:52:19.838966+0000","last_active":"2024-09-10T09:54:01.059138+0000","last_peered":"2024-09-10T09:54:01.059138+0000","last_clean":"2024-09-10T09:54:01.059138+0000","last_became_active":"2024-09-10T09:52:19.838545+0000","last_became_peered":"2024-09-10T09:52:19.838545+0000","last_unstale":"2024-09-10T09:54:01.059138+0000","last_undegraded":"2024-09-10T09:54:01.059138+0000","last_fullsized":"2024-09-10T09:54:01.059138+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:07:00.930146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.847601+0000","last_change":"2024-09-10T09:53:00.802039+0000","last_active":"2024-09-10T09:54:00.847601+0000","last_peered":"2024-09-10T09:54:00.847601+0000","last_clean":"2024-09-10T09:54:00.847601+0000","last_became_active":"2024-09-10T09:53:00.800223+0000","last_became_peered":"2024-09-10T09:53:00.800223+0000","last_unstale":"2024-09-10T09:54:00.847601+0000","last_undegraded":"2024-09-10T09:54:00.847601+0000","last_fullsized":"2024-09-10T09:54:00.847601+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:50:16.026631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,6,10],"acting":[5,8,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"96'8","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.698863+0000","last_change":"2024-09-10T09:52:22.313819+0000","last_active":"2024-09-10T09:54:29.698863+0000","last_peered":"2024-09-10T09:54:29.698863+0000","last_clean":"2024-09-10T09:54:29.698863+0000","last_became_active":"2024-09-10T09:52:22.313473+0000","last_became_peered":"2024-09-10T09:52:22.313473+0000","last_unstale":"2024-09-10T09:54:29.698863+0000","last_undegraded":"2024-09-10T09:54:29.698863+0000","last_fullsized":"2024-09-10T09:54:29.698863+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:44:10.003187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"94'6","reported_seq":51,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.951818+0000","last_change":"2024-09-10T09:52:19.837376+0000","last_active":"2024-09-10T09:54:00.951818+0000","last_peered":"2024-09-10T09:54:00.951818+0000","last_clean":"2024-09-10T09:54:00.951818+0000","last_became_active":"2024-09-10T09:52:19.836880+0000","last_became_peered":"2024-09-10T09:52:19.836880+0000","last_unstale":"2024-09-10T09:54:00.951818+0000","last_undegraded":"2024-09-10T09:54:00.951818+0000","last_fullsized":"2024-09-10T09:54:00.951818+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:48:58.198188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":361,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.059392+0000","last_change":"2024-09-10T09:51:05.871529+0000","last_active":"2024-09-10T09:54:01.059392+0000","last_peered":"2024-09-10T09:54:01.059392+0000","last_clean":"2024-09-10T09:54:01.059392+0000","last_became_active":"2024-09-10T09:51:05.871115+0000","last_became_peered":"2024-09-10T09:51:05.871115+0000","last_unstale":"2024-09-10T09:54:01.059392+0000","last_undegraded":"2024-09-10T09:54:01.059392+0000","last_fullsized":"2024-09-10T09:54:01.059392+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:47:24.175358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:47:24.175358+0000","last_clean_scrub_stamp":"2024-09-10T09:47:24.175358+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:58:30.247077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"94'1","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.059534+0000","last_change":"2024-09-10T09:53:00.800474+0000","last_active":"2024-09-10T09:54:01.059534+0000","last_peered":"2024-09-10T09:54:01.059534+0000","last_clean":"2024-09-10T09:54:01.059534+0000","last_became_active":"2024-09-10T09:53:00.799932+0000","last_became_peered":"2024-09-10T09:53:00.799932+0000","last_unstale":"2024-09-10T09:54:01.059534+0000","last_undegraded":"2024-09-10T09:54:01.059534+0000","last_fullsized":"2024-09-10T09:54:01.059534+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:14:50.998952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,11,9],"acting":[7,8,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.693609+0000","last_change":"2024-09-10T09:52:22.312521+0000","last_active":"2024-09-10T09:54:29.693609+0000","last_peered":"2024-09-10T09:54:29.693609+0000","last_clean":"2024-09-10T09:54:29.693609+0000","last_became_active":"2024-09-10T09:52:22.312107+0000","last_became_peered":"2024-09-10T09:52:22.312107+0000","last_unstale":"2024-09-10T09:54:29.693609+0000","last_undegraded":"2024-09-10T09:54:29.693609+0000","last_fullsized":"2024-09-10T09:54:29.693609+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:14:19.799259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"94'5","reported_seq":166,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:55:12.052387+0000","last_change":"2024-09-10T09:52:19.911454+0000","last_active":"2024-09-10T09:55:12.052387+0000","last_peered":"2024-09-10T09:55:12.052387+0000","last_clean":"2024-09-10T09:55:12.052387+0000","last_became_active":"2024-09-10T09:52:19.911079+0000","last_became_peered":"2024-09-10T09:52:19.911079+0000","last_unstale":"2024-09-10T09:55:12.052387+0000","last_undegraded":"2024-09-10T09:55:12.052387+0000","last_fullsized":"2024-09-10T09:55:12.052387+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:43:38.787689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":119,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.988531+0000","last_change":"2024-09-10T09:52:31.381877+0000","last_active":"2024-09-10T09:54:00.988531+0000","last_peered":"2024-09-10T09:54:00.988531+0000","last_clean":"2024-09-10T09:54:00.988531+0000","last_became_active":"2024-09-10T09:52:31.381472+0000","last_became_peered":"2024-09-10T09:52:31.381472+0000","last_unstale":"2024-09-10T09:54:00.988531+0000","last_undegraded":"2024-09-10T09:54:00.988531+0000","last_fullsized":"2024-09-10T09:54:00.988531+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:41:24.686325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"96'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.702544+0000","last_change":"2024-09-10T09:52:22.313902+0000","last_active":"2024-09-10T09:54:29.702544+0000","last_peered":"2024-09-10T09:54:29.702544+0000","last_clean":"2024-09-10T09:54:29.702544+0000","last_became_active":"2024-09-10T09:52:22.313515+0000","last_became_peered":"2024-09-10T09:52:22.313515+0000","last_unstale":"2024-09-10T09:54:29.702544+0000","last_undegraded":"2024-09-10T09:54:29.702544+0000","last_fullsized":"2024-09-10T09:54:29.702544+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:25:00.238307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"94'3","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.847873+0000","last_change":"2024-09-10T09:52:19.837736+0000","last_active":"2024-09-10T09:54:00.847873+0000","last_peered":"2024-09-10T09:54:00.847873+0000","last_clean":"2024-09-10T09:54:00.847873+0000","last_became_active":"2024-09-10T09:52:19.837370+0000","last_became_peered":"2024-09-10T09:52:19.837370+0000","last_unstale":"2024-09-10T09:54:00.847873+0000","last_undegraded":"2024-09-10T09:54:00.847873+0000","last_fullsized":"2024-09-10T09:54:00.847873+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:54:35.753027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.248213+0000","last_change":"2024-09-10T09:52:31.378945+0000","last_active":"2024-09-10T09:54:01.248213+0000","last_peered":"2024-09-10T09:54:01.248213+0000","last_clean":"2024-09-10T09:54:01.248213+0000","last_became_active":"2024-09-10T09:52:31.378144+0000","last_became_peered":"2024-09-10T09:52:31.378144+0000","last_unstale":"2024-09-10T09:54:01.248213+0000","last_undegraded":"2024-09-10T09:54:01.248213+0000","last_fullsized":"2024-09-10T09:54:01.248213+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:36:06.288293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"96'9","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.692039+0000","last_change":"2024-09-10T09:52:22.316400+0000","last_active":"2024-09-10T09:54:29.692039+0000","last_peered":"2024-09-10T09:54:29.692039+0000","last_clean":"2024-09-10T09:54:29.692039+0000","last_became_active":"2024-09-10T09:52:22.315974+0000","last_became_peered":"2024-09-10T09:52:22.315974+0000","last_unstale":"2024-09-10T09:54:29.692039+0000","last_undegraded":"2024-09-10T09:54:29.692039+0000","last_fullsized":"2024-09-10T09:54:29.692039+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:31:05.538237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"94'5","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.841976+0000","last_change":"2024-09-10T09:52:19.837942+0000","last_active":"2024-09-10T09:54:00.841976+0000","last_peered":"2024-09-10T09:54:00.841976+0000","last_clean":"2024-09-10T09:54:00.841976+0000","last_became_active":"2024-09-10T09:52:19.837629+0000","last_became_peered":"2024-09-10T09:52:19.837629+0000","last_unstale":"2024-09-10T09:54:00.841976+0000","last_undegraded":"2024-09-10T09:54:00.841976+0000","last_fullsized":"2024-09-10T09:54:00.841976+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:34:13.207770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.847019+0000","last_change":"2024-09-10T09:52:31.364231+0000","last_active":"2024-09-10T09:54:00.847019+0000","last_peered":"2024-09-10T09:54:00.847019+0000","last_clean":"2024-09-10T09:54:00.847019+0000","last_became_active":"2024-09-10T09:52:31.363833+0000","last_became_peered":"2024-09-10T09:52:31.363833+0000","last_unstale":"2024-09-10T09:54:00.847019+0000","last_undegraded":"2024-09-10T09:54:00.847019+0000","last_fullsized":"2024-09-10T09:54:00.847019+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:53:51.435700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"96'5","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.680562+0000","last_change":"2024-09-10T09:52:22.319530+0000","last_active":"2024-09-10T09:54:29.680562+0000","last_peered":"2024-09-10T09:54:29.680562+0000","last_clean":"2024-09-10T09:54:29.680562+0000","last_became_active":"2024-09-10T09:52:22.318742+0000","last_became_peered":"2024-09-10T09:52:22.318742+0000","last_unstale":"2024-09-10T09:54:29.680562+0000","last_undegraded":"2024-09-10T09:54:29.680562+0000","last_fullsized":"2024-09-10T09:54:29.680562+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:44:35.050454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"94'2","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.150048+0000","last_change":"2024-09-10T09:52:19.836398+0000","last_active":"2024-09-10T09:54:01.150048+0000","last_peered":"2024-09-10T09:54:01.150048+0000","last_clean":"2024-09-10T09:54:01.150048+0000","last_became_active":"2024-09-10T09:52:19.836055+0000","last_became_peered":"2024-09-10T09:52:19.836055+0000","last_unstale":"2024-09-10T09:54:01.150048+0000","last_undegraded":"2024-09-10T09:54:01.150048+0000","last_fullsized":"2024-09-10T09:54:01.150048+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:49:26.923367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.847215+0000","last_change":"2024-09-10T09:52:31.371109+0000","last_active":"2024-09-10T09:54:00.847215+0000","last_peered":"2024-09-10T09:54:00.847215+0000","last_clean":"2024-09-10T09:54:00.847215+0000","last_became_active":"2024-09-10T09:52:31.370592+0000","last_became_peered":"2024-09-10T09:52:31.370592+0000","last_unstale":"2024-09-10T09:54:00.847215+0000","last_undegraded":"2024-09-10T09:54:00.847215+0000","last_fullsized":"2024-09-10T09:54:00.847215+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:20:23.585187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"96'6","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.685236+0000","last_change":"2024-09-10T09:52:22.319365+0000","last_active":"2024-09-10T09:54:29.685236+0000","last_peered":"2024-09-10T09:54:29.685236+0000","last_clean":"2024-09-10T09:54:29.685236+0000","last_became_active":"2024-09-10T09:52:22.319064+0000","last_became_peered":"2024-09-10T09:52:22.319064+0000","last_unstale":"2024-09-10T09:54:29.685236+0000","last_undegraded":"2024-09-10T09:54:29.685236+0000","last_fullsized":"2024-09-10T09:54:29.685236+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:14:50.089566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"94'17","reported_seq":59,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.149324+0000","last_change":"2024-09-10T09:52:19.911698+0000","last_active":"2024-09-10T09:54:01.149324+0000","last_peered":"2024-09-10T09:54:01.149324+0000","last_clean":"2024-09-10T09:54:01.149324+0000","last_became_active":"2024-09-10T09:52:19.911369+0000","last_became_peered":"2024-09-10T09:52:19.911369+0000","last_unstale":"2024-09-10T09:54:01.149324+0000","last_undegraded":"2024-09-10T09:54:01.149324+0000","last_fullsized":"2024-09-10T09:54:01.149324+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:40:39.045797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.841947+0000","last_change":"2024-09-10T09:52:31.380463+0000","last_active":"2024-09-10T09:54:00.841947+0000","last_peered":"2024-09-10T09:54:00.841947+0000","last_clean":"2024-09-10T09:54:00.841947+0000","last_became_active":"2024-09-10T09:52:31.380114+0000","last_became_peered":"2024-09-10T09:52:31.380114+0000","last_unstale":"2024-09-10T09:54:00.841947+0000","last_undegraded":"2024-09-10T09:54:00.841947+0000","last_fullsized":"2024-09-10T09:54:00.841947+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:19:50.869160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"96'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.682831+0000","last_change":"2024-09-10T09:52:22.313928+0000","last_active":"2024-09-10T09:54:29.682831+0000","last_peered":"2024-09-10T09:54:29.682831+0000","last_clean":"2024-09-10T09:54:29.682831+0000","last_became_active":"2024-09-10T09:52:22.313510+0000","last_became_peered":"2024-09-10T09:52:22.313510+0000","last_unstale":"2024-09-10T09:54:29.682831+0000","last_undegraded":"2024-09-10T09:54:29.682831+0000","last_fullsized":"2024-09-10T09:54:29.682831+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:58:19.031178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"96'7","reported_seq":170,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:55:12.083520+0000","last_change":"2024-09-10T09:52:19.839451+0000","last_active":"2024-09-10T09:55:12.083520+0000","last_peered":"2024-09-10T09:55:12.083520+0000","last_clean":"2024-09-10T09:55:12.083520+0000","last_became_active":"2024-09-10T09:52:19.839124+0000","last_became_peered":"2024-09-10T09:52:19.839124+0000","last_unstale":"2024-09-10T09:55:12.083520+0000","last_undegraded":"2024-09-10T09:55:12.083520+0000","last_fullsized":"2024-09-10T09:55:12.083520+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:51:33.489830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":122,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.844344+0000","last_change":"2024-09-10T09:52:31.376386+0000","last_active":"2024-09-10T09:54:00.844344+0000","last_peered":"2024-09-10T09:54:00.844344+0000","last_clean":"2024-09-10T09:54:00.844344+0000","last_became_active":"2024-09-10T09:52:31.376066+0000","last_became_peered":"2024-09-10T09:52:31.376066+0000","last_unstale":"2024-09-10T09:54:00.844344+0000","last_undegraded":"2024-09-10T09:54:00.844344+0000","last_fullsized":"2024-09-10T09:54:00.844344+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:23:07.191842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"96'15","reported_seq":52,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.702901+0000","last_change":"2024-09-10T09:52:22.323590+0000","last_active":"2024-09-10T09:54:29.702901+0000","last_peered":"2024-09-10T09:54:29.702901+0000","last_clean":"2024-09-10T09:54:29.702901+0000","last_became_active":"2024-09-10T09:52:22.323264+0000","last_became_peered":"2024-09-10T09:52:22.323264+0000","last_unstale":"2024-09-10T09:54:29.702901+0000","last_undegraded":"2024-09-10T09:54:29.702901+0000","last_fullsized":"2024-09-10T09:54:29.702901+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:07:56.572337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"94'5","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.847940+0000","last_change":"2024-09-10T09:52:19.840620+0000","last_active":"2024-09-10T09:54:00.847940+0000","last_peered":"2024-09-10T09:54:00.847940+0000","last_clean":"2024-09-10T09:54:00.847940+0000","last_became_active":"2024-09-10T09:52:19.840217+0000","last_became_peered":"2024-09-10T09:52:19.840217+0000","last_unstale":"2024-09-10T09:54:00.847940+0000","last_undegraded":"2024-09-10T09:54:00.847940+0000","last_fullsized":"2024-09-10T09:54:00.847940+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:55:13.111993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.955071+0000","last_change":"2024-09-10T09:52:31.373895+0000","last_active":"2024-09-10T09:54:00.955071+0000","last_peered":"2024-09-10T09:54:00.955071+0000","last_clean":"2024-09-10T09:54:00.955071+0000","last_became_active":"2024-09-10T09:52:31.373474+0000","last_became_peered":"2024-09-10T09:52:31.373474+0000","last_unstale":"2024-09-10T09:54:00.955071+0000","last_undegraded":"2024-09-10T09:54:00.955071+0000","last_fullsized":"2024-09-10T09:54:00.955071+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:39:16.049859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"96'9","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.704377+0000","last_change":"2024-09-10T09:52:22.319294+0000","last_active":"2024-09-10T09:54:29.704377+0000","last_peered":"2024-09-10T09:54:29.704377+0000","last_clean":"2024-09-10T09:54:29.704377+0000","last_became_active":"2024-09-10T09:52:22.318918+0000","last_became_peered":"2024-09-10T09:52:22.318918+0000","last_unstale":"2024-09-10T09:54:29.704377+0000","last_undegraded":"2024-09-10T09:54:29.704377+0000","last_fullsized":"2024-09-10T09:54:29.704377+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:52:46.434182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"94'9","reported_seq":53,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.844244+0000","last_change":"2024-09-10T09:52:19.842286+0000","last_active":"2024-09-10T09:54:00.844244+0000","last_peered":"2024-09-10T09:54:00.844244+0000","last_clean":"2024-09-10T09:54:00.844244+0000","last_became_active":"2024-09-10T09:52:19.841930+0000","last_became_peered":"2024-09-10T09:52:19.841930+0000","last_unstale":"2024-09-10T09:54:00.844244+0000","last_undegraded":"2024-09-10T09:54:00.844244+0000","last_fullsized":"2024-09-10T09:54:00.844244+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:38:46.358389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.847531+0000","last_change":"2024-09-10T09:52:31.370961+0000","last_active":"2024-09-10T09:54:00.847531+0000","last_peered":"2024-09-10T09:54:00.847531+0000","last_clean":"2024-09-10T09:54:00.847531+0000","last_became_active":"2024-09-10T09:52:31.370545+0000","last_became_peered":"2024-09-10T09:52:31.370545+0000","last_unstale":"2024-09-10T09:54:00.847531+0000","last_undegraded":"2024-09-10T09:54:00.847531+0000","last_fullsized":"2024-09-10T09:54:00.847531+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:17.461447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"96'6","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.685895+0000","last_change":"2024-09-10T09:52:22.313215+0000","last_active":"2024-09-10T09:54:29.685895+0000","last_peered":"2024-09-10T09:54:29.685895+0000","last_clean":"2024-09-10T09:54:29.685895+0000","last_became_active":"2024-09-10T09:52:22.312918+0000","last_became_peered":"2024-09-10T09:52:22.312918+0000","last_unstale":"2024-09-10T09:54:29.685895+0000","last_undegraded":"2024-09-10T09:54:29.685895+0000","last_fullsized":"2024-09-10T09:54:29.685895+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:48:09.568004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"96'37","reported_seq":337,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:55:12.051648+0000","last_change":"2024-09-10T09:53:01.226146+0000","last_active":"2024-09-10T09:55:12.051648+0000","last_peered":"2024-09-10T09:55:12.051648+0000","last_clean":"2024-09-10T09:55:12.051648+0000","last_became_active":"2024-09-10T09:53:01.218622+0000","last_became_peered":"2024-09-10T09:53:01.218622+0000","last_unstale":"2024-09-10T09:55:12.051648+0000","last_undegraded":"2024-09-10T09:55:12.051648+0000","last_fullsized":"2024-09-10T09:55:12.051648+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:43:11.927029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63251,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":241,"num_read_kb":182,"num_write":39,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":3844,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":16,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.841392+0000","last_change":"2024-09-10T09:53:00.800650+0000","last_active":"2024-09-10T09:54:00.841392+0000","last_peered":"2024-09-10T09:54:00.841392+0000","last_clean":"2024-09-10T09:54:00.841392+0000","last_became_active":"2024-09-10T09:53:00.800104+0000","last_became_peered":"2024-09-10T09:53:00.800104+0000","last_unstale":"2024-09-10T09:54:00.841392+0000","last_undegraded":"2024-09-10T09:54:00.841392+0000","last_fullsized":"2024-09-10T09:54:00.841392+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:41:16.022884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.14","version":"96'11","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.690121+0000","last_change":"2024-09-10T09:52:22.317174+0000","last_active":"2024-09-10T09:54:29.690121+0000","last_peered":"2024-09-10T09:54:29.690121+0000","last_clean":"2024-09-10T09:54:29.690121+0000","last_became_active":"2024-09-10T09:52:22.316851+0000","last_became_peered":"2024-09-10T09:52:22.316851+0000","last_unstale":"2024-09-10T09:54:29.690121+0000","last_undegraded":"2024-09-10T09:54:29.690121+0000","last_fullsized":"2024-09-10T09:54:29.690121+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:55:18.742393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"94'5","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.847731+0000","last_change":"2024-09-10T09:52:19.912093+0000","last_active":"2024-09-10T09:54:00.847731+0000","last_peered":"2024-09-10T09:54:00.847731+0000","last_clean":"2024-09-10T09:54:00.847731+0000","last_became_active":"2024-09-10T09:52:19.911611+0000","last_became_peered":"2024-09-10T09:52:19.911611+0000","last_unstale":"2024-09-10T09:54:00.847731+0000","last_undegraded":"2024-09-10T09:54:00.847731+0000","last_fullsized":"2024-09-10T09:54:00.847731+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:01:10.859310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.951658+0000","last_change":"2024-09-10T09:52:31.362884+0000","last_active":"2024-09-10T09:54:00.951658+0000","last_peered":"2024-09-10T09:54:00.951658+0000","last_clean":"2024-09-10T09:54:00.951658+0000","last_became_active":"2024-09-10T09:52:31.362489+0000","last_became_peered":"2024-09-10T09:52:31.362489+0000","last_unstale":"2024-09-10T09:54:00.951658+0000","last_undegraded":"2024-09-10T09:54:00.951658+0000","last_fullsized":"2024-09-10T09:54:00.951658+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:02:33.084795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"96'9","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.692830+0000","last_change":"2024-09-10T09:52:22.325871+0000","last_active":"2024-09-10T09:54:29.692830+0000","last_peered":"2024-09-10T09:54:29.692830+0000","last_clean":"2024-09-10T09:54:29.692830+0000","last_became_active":"2024-09-10T09:52:22.324980+0000","last_became_peered":"2024-09-10T09:52:22.324980+0000","last_unstale":"2024-09-10T09:54:29.692830+0000","last_undegraded":"2024-09-10T09:54:29.692830+0000","last_fullsized":"2024-09-10T09:54:29.692830+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:31:55.713444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"94'4","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.059691+0000","last_change":"2024-09-10T09:52:19.839491+0000","last_active":"2024-09-10T09:54:01.059691+0000","last_peered":"2024-09-10T09:54:01.059691+0000","last_clean":"2024-09-10T09:54:01.059691+0000","last_became_active":"2024-09-10T09:52:19.839092+0000","last_became_peered":"2024-09-10T09:52:19.839092+0000","last_unstale":"2024-09-10T09:54:01.059691+0000","last_undegraded":"2024-09-10T09:54:01.059691+0000","last_fullsized":"2024-09-10T09:54:01.059691+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:42:13.453281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.843285+0000","last_change":"2024-09-10T09:52:31.371346+0000","last_active":"2024-09-10T09:54:00.843285+0000","last_peered":"2024-09-10T09:54:00.843285+0000","last_clean":"2024-09-10T09:54:00.843285+0000","last_became_active":"2024-09-10T09:52:31.370929+0000","last_became_peered":"2024-09-10T09:52:31.370929+0000","last_unstale":"2024-09-10T09:54:00.843285+0000","last_undegraded":"2024-09-10T09:54:00.843285+0000","last_fullsized":"2024-09-10T09:54:00.843285+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:44:22.925082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"96'10","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.694062+0000","last_change":"2024-09-10T09:52:22.317453+0000","last_active":"2024-09-10T09:54:29.694062+0000","last_peered":"2024-09-10T09:54:29.694062+0000","last_clean":"2024-09-10T09:54:29.694062+0000","last_became_active":"2024-09-10T09:52:22.315556+0000","last_became_peered":"2024-09-10T09:52:22.315556+0000","last_unstale":"2024-09-10T09:54:29.694062+0000","last_undegraded":"2024-09-10T09:54:29.694062+0000","last_fullsized":"2024-09-10T09:54:29.694062+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:56:05.860275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"96'5","reported_seq":280,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:55:12.051699+0000","last_change":"2024-09-10T09:52:19.835985+0000","last_active":"2024-09-10T09:55:12.051699+0000","last_peered":"2024-09-10T09:55:12.051699+0000","last_clean":"2024-09-10T09:55:12.051699+0000","last_became_active":"2024-09-10T09:52:19.834835+0000","last_became_peered":"2024-09-10T09:52:19.834835+0000","last_unstale":"2024-09-10T09:55:12.051699+0000","last_undegraded":"2024-09-10T09:55:12.051699+0000","last_fullsized":"2024-09-10T09:55:12.051699+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:09:11.590443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":232,"num_read_kb":232,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.845724+0000","last_change":"2024-09-10T09:52:31.368663+0000","last_active":"2024-09-10T09:54:00.845724+0000","last_peered":"2024-09-10T09:54:00.845724+0000","last_clean":"2024-09-10T09:54:00.845724+0000","last_became_active":"2024-09-10T09:52:31.368004+0000","last_became_peered":"2024-09-10T09:52:31.368004+0000","last_unstale":"2024-09-10T09:54:00.845724+0000","last_undegraded":"2024-09-10T09:54:00.845724+0000","last_fullsized":"2024-09-10T09:54:00.845724+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:56:15.100515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.247917+0000","last_change":"2024-09-10T09:52:31.370992+0000","last_active":"2024-09-10T09:54:01.247917+0000","last_peered":"2024-09-10T09:54:01.247917+0000","last_clean":"2024-09-10T09:54:01.247917+0000","last_became_active":"2024-09-10T09:52:31.370357+0000","last_became_peered":"2024-09-10T09:52:31.370357+0000","last_unstale":"2024-09-10T09:54:01.247917+0000","last_undegraded":"2024-09-10T09:54:01.247917+0000","last_fullsized":"2024-09-10T09:54:01.247917+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:02:49.482532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"96'11","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.706426+0000","last_change":"2024-09-10T09:52:22.319363+0000","last_active":"2024-09-10T09:54:29.706426+0000","last_peered":"2024-09-10T09:54:29.706426+0000","last_clean":"2024-09-10T09:54:29.706426+0000","last_became_active":"2024-09-10T09:52:22.319044+0000","last_became_peered":"2024-09-10T09:52:22.319044+0000","last_unstale":"2024-09-10T09:54:29.706426+0000","last_undegraded":"2024-09-10T09:54:29.706426+0000","last_fullsized":"2024-09-10T09:54:29.706426+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:30:51.328413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"94'3","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.847831+0000","last_change":"2024-09-10T09:52:19.841745+0000","last_active":"2024-09-10T09:54:00.847831+0000","last_peered":"2024-09-10T09:54:00.847831+0000","last_clean":"2024-09-10T09:54:00.847831+0000","last_became_active":"2024-09-10T09:52:19.841404+0000","last_became_peered":"2024-09-10T09:52:19.841404+0000","last_unstale":"2024-09-10T09:54:00.847831+0000","last_undegraded":"2024-09-10T09:54:00.847831+0000","last_fullsized":"2024-09-10T09:54:00.847831+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:44:36.346785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.955521+0000","last_change":"2024-09-10T09:53:01.218412+0000","last_active":"2024-09-10T09:54:00.955521+0000","last_peered":"2024-09-10T09:54:00.955521+0000","last_clean":"2024-09-10T09:54:00.955521+0000","last_became_active":"2024-09-10T09:53:01.217966+0000","last_became_peered":"2024-09-10T09:53:01.217966+0000","last_unstale":"2024-09-10T09:54:00.955521+0000","last_undegraded":"2024-09-10T09:54:00.955521+0000","last_fullsized":"2024-09-10T09:54:00.955521+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:53:25.743913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"96'8","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.703441+0000","last_change":"2024-09-10T09:52:22.316120+0000","last_active":"2024-09-10T09:54:29.703441+0000","last_peered":"2024-09-10T09:54:29.703441+0000","last_clean":"2024-09-10T09:54:29.703441+0000","last_became_active":"2024-09-10T09:52:22.315769+0000","last_became_peered":"2024-09-10T09:52:22.315769+0000","last_unstale":"2024-09-10T09:54:29.703441+0000","last_undegraded":"2024-09-10T09:54:29.703441+0000","last_fullsized":"2024-09-10T09:54:29.703441+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:16:41.866587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"94'3","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.841046+0000","last_change":"2024-09-10T09:52:19.835038+0000","last_active":"2024-09-10T09:54:00.841046+0000","last_peered":"2024-09-10T09:54:00.841046+0000","last_clean":"2024-09-10T09:54:00.841046+0000","last_became_active":"2024-09-10T09:52:19.834329+0000","last_became_peered":"2024-09-10T09:52:19.834329+0000","last_unstale":"2024-09-10T09:54:00.841046+0000","last_undegraded":"2024-09-10T09:54:00.841046+0000","last_fullsized":"2024-09-10T09:54:00.841046+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:58:24.084373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"94'7","reported_seq":56,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.841764+0000","last_change":"2024-09-10T09:53:01.219122+0000","last_active":"2024-09-10T09:54:00.841764+0000","last_peered":"2024-09-10T09:54:00.841764+0000","last_clean":"2024-09-10T09:54:00.841764+0000","last_became_active":"2024-09-10T09:53:01.218599+0000","last_became_peered":"2024-09-10T09:53:01.218599+0000","last_unstale":"2024-09-10T09:54:00.841764+0000","last_undegraded":"2024-09-10T09:54:00.841764+0000","last_fullsized":"2024-09-10T09:54:00.841764+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:31:10.092264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"96'8","reported_seq":19,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.706393+0000","last_change":"2024-09-10T09:54:00.952617+0000","last_active":"2024-09-10T09:54:29.706393+0000","last_peered":"2024-09-10T09:54:29.706393+0000","last_clean":"2024-09-10T09:54:29.706393+0000","last_became_active":"2024-09-10T09:54:00.952202+0000","last_became_peered":"2024-09-10T09:54:00.952202+0000","last_unstale":"2024-09-10T09:54:29.706393+0000","last_undegraded":"2024-09-10T09:54:29.706393+0000","last_fullsized":"2024-09-10T09:54:29.706393+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:35:40.179591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.844169+0000","last_change":"2024-09-10T09:52:31.371612+0000","last_active":"2024-09-10T09:54:00.844169+0000","last_peered":"2024-09-10T09:54:00.844169+0000","last_clean":"2024-09-10T09:54:00.844169+0000","last_became_active":"2024-09-10T09:52:31.371163+0000","last_became_peered":"2024-09-10T09:52:31.371163+0000","last_unstale":"2024-09-10T09:54:00.844169+0000","last_undegraded":"2024-09-10T09:54:00.844169+0000","last_fullsized":"2024-09-10T09:54:00.844169+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:11:43.681524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"96'7","reported_seq":171,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:55:12.052584+0000","last_change":"2024-09-10T09:52:19.841873+0000","last_active":"2024-09-10T09:55:12.052584+0000","last_peered":"2024-09-10T09:55:12.052584+0000","last_clean":"2024-09-10T09:55:12.052584+0000","last_became_active":"2024-09-10T09:52:19.841513+0000","last_became_peered":"2024-09-10T09:52:19.841513+0000","last_unstale":"2024-09-10T09:55:12.052584+0000","last_undegraded":"2024-09-10T09:55:12.052584+0000","last_fullsized":"2024-09-10T09:55:12.052584+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:53:31.854185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1156,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":121,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"96'7","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.697738+0000","last_change":"2024-09-10T09:52:22.314781+0000","last_active":"2024-09-10T09:54:29.697738+0000","last_peered":"2024-09-10T09:54:29.697738+0000","last_clean":"2024-09-10T09:54:29.697738+0000","last_became_active":"2024-09-10T09:52:22.313346+0000","last_became_peered":"2024-09-10T09:52:22.313346+0000","last_unstale":"2024-09-10T09:54:29.697738+0000","last_undegraded":"2024-09-10T09:54:29.697738+0000","last_fullsized":"2024-09-10T09:54:29.697738+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:15:16.921963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.844401+0000","last_change":"2024-09-10T09:52:31.365544+0000","last_active":"2024-09-10T09:54:00.844401+0000","last_peered":"2024-09-10T09:54:00.844401+0000","last_clean":"2024-09-10T09:54:00.844401+0000","last_became_active":"2024-09-10T09:52:31.365151+0000","last_became_peered":"2024-09-10T09:52:31.365151+0000","last_unstale":"2024-09-10T09:54:00.844401+0000","last_undegraded":"2024-09-10T09:54:00.844401+0000","last_fullsized":"2024-09-10T09:54:00.844401+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:36:55.247890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"96'9","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.683641+0000","last_change":"2024-09-10T09:52:22.314117+0000","last_active":"2024-09-10T09:54:29.683641+0000","last_peered":"2024-09-10T09:54:29.683641+0000","last_clean":"2024-09-10T09:54:29.683641+0000","last_became_active":"2024-09-10T09:52:22.313772+0000","last_became_peered":"2024-09-10T09:52:22.313772+0000","last_unstale":"2024-09-10T09:54:29.683641+0000","last_undegraded":"2024-09-10T09:54:29.683641+0000","last_fullsized":"2024-09-10T09:54:29.683641+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:19:37.373598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"96'12","reported_seq":50,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.682477+0000","last_change":"2024-09-10T09:52:22.314362+0000","last_active":"2024-09-10T09:54:29.682477+0000","last_peered":"2024-09-10T09:54:29.682477+0000","last_clean":"2024-09-10T09:54:29.682477+0000","last_became_active":"2024-09-10T09:52:22.314050+0000","last_became_peered":"2024-09-10T09:52:22.314050+0000","last_unstale":"2024-09-10T09:54:29.682477+0000","last_undegraded":"2024-09-10T09:54:29.682477+0000","last_fullsized":"2024-09-10T09:54:29.682477+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:37:07.218656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"96'8","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.699844+0000","last_change":"2024-09-10T09:52:22.322363+0000","last_active":"2024-09-10T09:54:29.699844+0000","last_peered":"2024-09-10T09:54:29.699844+0000","last_clean":"2024-09-10T09:54:29.699844+0000","last_became_active":"2024-09-10T09:52:22.322045+0000","last_became_peered":"2024-09-10T09:52:22.322045+0000","last_unstale":"2024-09-10T09:54:29.699844+0000","last_undegraded":"2024-09-10T09:54:29.699844+0000","last_fullsized":"2024-09-10T09:54:29.699844+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:10:27.918769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.700329+0000","last_change":"2024-09-10T09:52:22.331594+0000","last_active":"2024-09-10T09:54:29.700329+0000","last_peered":"2024-09-10T09:54:29.700329+0000","last_clean":"2024-09-10T09:54:29.700329+0000","last_became_active":"2024-09-10T09:52:22.330585+0000","last_became_peered":"2024-09-10T09:52:22.330585+0000","last_unstale":"2024-09-10T09:54:29.700329+0000","last_undegraded":"2024-09-10T09:54:29.700329+0000","last_fullsized":"2024-09-10T09:54:29.700329+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:45:56.412313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"96'4","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.705719+0000","last_change":"2024-09-10T09:52:22.325753+0000","last_active":"2024-09-10T09:54:29.705719+0000","last_peered":"2024-09-10T09:54:29.705719+0000","last_clean":"2024-09-10T09:54:29.705719+0000","last_became_active":"2024-09-10T09:52:22.325385+0000","last_became_peered":"2024-09-10T09:52:22.325385+0000","last_unstale":"2024-09-10T09:54:29.705719+0000","last_undegraded":"2024-09-10T09:54:29.705719+0000","last_fullsized":"2024-09-10T09:54:29.705719+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:18:28.776789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"96'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.681378+0000","last_change":"2024-09-10T09:52:22.319069+0000","last_active":"2024-09-10T09:54:29.681378+0000","last_peered":"2024-09-10T09:54:29.681378+0000","last_clean":"2024-09-10T09:54:29.681378+0000","last_became_active":"2024-09-10T09:52:22.318667+0000","last_became_peered":"2024-09-10T09:52:22.318667+0000","last_unstale":"2024-09-10T09:54:29.681378+0000","last_undegraded":"2024-09-10T09:54:29.681378+0000","last_fullsized":"2024-09-10T09:54:29.681378+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:21:32.028235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"96'15","reported_seq":53,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.681487+0000","last_change":"2024-09-10T09:52:22.312890+0000","last_active":"2024-09-10T09:54:29.681487+0000","last_peered":"2024-09-10T09:54:29.681487+0000","last_clean":"2024-09-10T09:54:29.681487+0000","last_became_active":"2024-09-10T09:52:22.312608+0000","last_became_peered":"2024-09-10T09:52:22.312608+0000","last_unstale":"2024-09-10T09:54:29.681487+0000","last_undegraded":"2024-09-10T09:54:29.681487+0000","last_fullsized":"2024-09-10T09:54:29.681487+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:07:46.046915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"96'2","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.689884+0000","last_change":"2024-09-10T09:54:01.151385+0000","last_active":"2024-09-10T09:54:29.689884+0000","last_peered":"2024-09-10T09:54:29.689884+0000","last_clean":"2024-09-10T09:54:29.689884+0000","last_became_active":"2024-09-10T09:54:01.150971+0000","last_became_peered":"2024-09-10T09:54:01.150971+0000","last_unstale":"2024-09-10T09:54:29.689884+0000","last_undegraded":"2024-09-10T09:54:29.689884+0000","last_fullsized":"2024-09-10T09:54:29.689884+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:27:10.492071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"96'6","reported_seq":17,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.704565+0000","last_change":"2024-09-10T09:54:00.848273+0000","last_active":"2024-09-10T09:54:29.704565+0000","last_peered":"2024-09-10T09:54:29.704565+0000","last_clean":"2024-09-10T09:54:29.704565+0000","last_became_active":"2024-09-10T09:54:00.847492+0000","last_became_peered":"2024-09-10T09:54:00.847492+0000","last_unstale":"2024-09-10T09:54:29.704565+0000","last_undegraded":"2024-09-10T09:54:29.704565+0000","last_fullsized":"2024-09-10T09:54:29.704565+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:02:24.749291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,3],"acting":[6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.29","version":"96'8","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.696836+0000","last_change":"2024-09-10T09:52:22.313538+0000","last_active":"2024-09-10T09:54:29.696836+0000","last_peered":"2024-09-10T09:54:29.696836+0000","last_clean":"2024-09-10T09:54:29.696836+0000","last_became_active":"2024-09-10T09:52:22.313196+0000","last_became_peered":"2024-09-10T09:52:22.313196+0000","last_unstale":"2024-09-10T09:54:29.696836+0000","last_undegraded":"2024-09-10T09:54:29.696836+0000","last_fullsized":"2024-09-10T09:54:29.696836+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:53:23.387167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"96'10","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.696632+0000","last_change":"2024-09-10T09:52:22.316518+0000","last_active":"2024-09-10T09:54:29.696632+0000","last_peered":"2024-09-10T09:54:29.696632+0000","last_clean":"2024-09-10T09:54:29.696632+0000","last_became_active":"2024-09-10T09:52:22.316193+0000","last_became_peered":"2024-09-10T09:52:22.316193+0000","last_unstale":"2024-09-10T09:54:29.696632+0000","last_undegraded":"2024-09-10T09:54:29.696632+0000","last_fullsized":"2024-09-10T09:54:29.696632+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:49:55.903928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"96'11","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.706245+0000","last_change":"2024-09-10T09:52:22.329163+0000","last_active":"2024-09-10T09:54:29.706245+0000","last_peered":"2024-09-10T09:54:29.706245+0000","last_clean":"2024-09-10T09:54:29.706245+0000","last_became_active":"2024-09-10T09:52:22.328453+0000","last_became_peered":"2024-09-10T09:52:22.328453+0000","last_unstale":"2024-09-10T09:54:29.706245+0000","last_undegraded":"2024-09-10T09:54:29.706245+0000","last_fullsized":"2024-09-10T09:54:29.706245+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:53:33.286055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"96'8","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.681766+0000","last_change":"2024-09-10T09:52:22.314302+0000","last_active":"2024-09-10T09:54:29.681766+0000","last_peered":"2024-09-10T09:54:29.681766+0000","last_clean":"2024-09-10T09:54:29.681766+0000","last_became_active":"2024-09-10T09:52:22.313992+0000","last_became_peered":"2024-09-10T09:52:22.313992+0000","last_unstale":"2024-09-10T09:54:29.681766+0000","last_undegraded":"2024-09-10T09:54:29.681766+0000","last_fullsized":"2024-09-10T09:54:29.681766+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:07:35.764015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"96'7","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.688705+0000","last_change":"2024-09-10T09:52:22.316400+0000","last_active":"2024-09-10T09:54:29.688705+0000","last_peered":"2024-09-10T09:54:29.688705+0000","last_clean":"2024-09-10T09:54:29.688705+0000","last_became_active":"2024-09-10T09:52:22.316110+0000","last_became_peered":"2024-09-10T09:52:22.316110+0000","last_unstale":"2024-09-10T09:54:29.688705+0000","last_undegraded":"2024-09-10T09:54:29.688705+0000","last_fullsized":"2024-09-10T09:54:29.688705+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:17:26.528899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"96'10","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.704074+0000","last_change":"2024-09-10T09:52:22.317914+0000","last_active":"2024-09-10T09:54:29.704074+0000","last_peered":"2024-09-10T09:54:29.704074+0000","last_clean":"2024-09-10T09:54:29.704074+0000","last_became_active":"2024-09-10T09:52:22.317575+0000","last_became_peered":"2024-09-10T09:52:22.317575+0000","last_unstale":"2024-09-10T09:54:29.704074+0000","last_undegraded":"2024-09-10T09:54:29.704074+0000","last_fullsized":"2024-09-10T09:54:29.704074+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:17:51.446233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"96'8","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.697848+0000","last_change":"2024-09-10T09:52:22.327749+0000","last_active":"2024-09-10T09:54:29.697848+0000","last_peered":"2024-09-10T09:54:29.697848+0000","last_clean":"2024-09-10T09:54:29.697848+0000","last_became_active":"2024-09-10T09:52:22.327461+0000","last_became_peered":"2024-09-10T09:52:22.327461+0000","last_unstale":"2024-09-10T09:54:29.697848+0000","last_undegraded":"2024-09-10T09:54:29.697848+0000","last_fullsized":"2024-09-10T09:54:29.697848+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:54.208064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"96'9","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.696695+0000","last_change":"2024-09-10T09:52:22.326982+0000","last_active":"2024-09-10T09:54:29.696695+0000","last_peered":"2024-09-10T09:54:29.696695+0000","last_clean":"2024-09-10T09:54:29.696695+0000","last_became_active":"2024-09-10T09:52:22.326635+0000","last_became_peered":"2024-09-10T09:52:22.326635+0000","last_unstale":"2024-09-10T09:54:29.696695+0000","last_undegraded":"2024-09-10T09:54:29.696695+0000","last_fullsized":"2024-09-10T09:54:29.696695+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:50:54.835899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.695757+0000","last_change":"2024-09-10T09:52:22.323410+0000","last_active":"2024-09-10T09:54:29.695757+0000","last_peered":"2024-09-10T09:54:29.695757+0000","last_clean":"2024-09-10T09:54:29.695757+0000","last_became_active":"2024-09-10T09:52:22.323116+0000","last_became_peered":"2024-09-10T09:52:22.323116+0000","last_unstale":"2024-09-10T09:54:29.695757+0000","last_undegraded":"2024-09-10T09:54:29.695757+0000","last_fullsized":"2024-09-10T09:54:29.695757+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:09:15.115162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.680224+0000","last_change":"2024-09-10T09:52:22.324908+0000","last_active":"2024-09-10T09:54:29.680224+0000","last_peered":"2024-09-10T09:54:29.680224+0000","last_clean":"2024-09-10T09:54:29.680224+0000","last_became_active":"2024-09-10T09:52:22.324449+0000","last_became_peered":"2024-09-10T09:52:22.324449+0000","last_unstale":"2024-09-10T09:54:29.680224+0000","last_undegraded":"2024-09-10T09:54:29.680224+0000","last_fullsized":"2024-09-10T09:54:29.680224+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:43:24.535721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"96'10","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.700039+0000","last_change":"2024-09-10T09:52:22.322157+0000","last_active":"2024-09-10T09:54:29.700039+0000","last_peered":"2024-09-10T09:54:29.700039+0000","last_clean":"2024-09-10T09:54:29.700039+0000","last_became_active":"2024-09-10T09:52:22.321856+0000","last_became_peered":"2024-09-10T09:52:22.321856+0000","last_unstale":"2024-09-10T09:54:29.700039+0000","last_undegraded":"2024-09-10T09:54:29.700039+0000","last_fullsized":"2024-09-10T09:54:29.700039+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:36:38.395370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"96'7","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.703464+0000","last_change":"2024-09-10T09:53:00.800337+0000","last_active":"2024-09-10T09:54:29.703464+0000","last_peered":"2024-09-10T09:54:29.703464+0000","last_clean":"2024-09-10T09:54:29.703464+0000","last_became_active":"2024-09-10T09:53:00.799863+0000","last_became_peered":"2024-09-10T09:53:00.799863+0000","last_unstale":"2024-09-10T09:54:29.703464+0000","last_undegraded":"2024-09-10T09:54:29.703464+0000","last_fullsized":"2024-09-10T09:54:29.703464+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:18.017672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.35","version":"96'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.693580+0000","last_change":"2024-09-10T09:52:22.310735+0000","last_active":"2024-09-10T09:54:29.693580+0000","last_peered":"2024-09-10T09:54:29.693580+0000","last_clean":"2024-09-10T09:54:29.693580+0000","last_became_active":"2024-09-10T09:52:22.310376+0000","last_became_peered":"2024-09-10T09:52:22.310376+0000","last_unstale":"2024-09-10T09:54:29.693580+0000","last_undegraded":"2024-09-10T09:54:29.693580+0000","last_fullsized":"2024-09-10T09:54:29.693580+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:38:08.637588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"96'16","reported_seq":55,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.685458+0000","last_change":"2024-09-10T09:52:22.321113+0000","last_active":"2024-09-10T09:54:29.685458+0000","last_peered":"2024-09-10T09:54:29.685458+0000","last_clean":"2024-09-10T09:54:29.685458+0000","last_became_active":"2024-09-10T09:52:22.320767+0000","last_became_peered":"2024-09-10T09:52:22.320767+0000","last_unstale":"2024-09-10T09:54:29.685458+0000","last_undegraded":"2024-09-10T09:54:29.685458+0000","last_fullsized":"2024-09-10T09:54:29.685458+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:37.549322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"96'11","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.687162+0000","last_change":"2024-09-10T09:52:22.318634+0000","last_active":"2024-09-10T09:54:29.687162+0000","last_peered":"2024-09-10T09:54:29.687162+0000","last_clean":"2024-09-10T09:54:29.687162+0000","last_became_active":"2024-09-10T09:52:22.318294+0000","last_became_peered":"2024-09-10T09:52:22.318294+0000","last_unstale":"2024-09-10T09:54:29.687162+0000","last_undegraded":"2024-09-10T09:54:29.687162+0000","last_fullsized":"2024-09-10T09:54:29.687162+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:19:31.262055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"96'9","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.698725+0000","last_change":"2024-09-10T09:52:22.324567+0000","last_active":"2024-09-10T09:54:29.698725+0000","last_peered":"2024-09-10T09:54:29.698725+0000","last_clean":"2024-09-10T09:54:29.698725+0000","last_became_active":"2024-09-10T09:52:22.321803+0000","last_became_peered":"2024-09-10T09:52:22.321803+0000","last_unstale":"2024-09-10T09:54:29.698725+0000","last_undegraded":"2024-09-10T09:54:29.698725+0000","last_fullsized":"2024-09-10T09:54:29.698725+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:08:11.099428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"96'5","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.699735+0000","last_change":"2024-09-10T09:52:22.318163+0000","last_active":"2024-09-10T09:54:29.699735+0000","last_peered":"2024-09-10T09:54:29.699735+0000","last_clean":"2024-09-10T09:54:29.699735+0000","last_became_active":"2024-09-10T09:52:22.317808+0000","last_became_peered":"2024-09-10T09:52:22.317808+0000","last_unstale":"2024-09-10T09:54:29.699735+0000","last_undegraded":"2024-09-10T09:54:29.699735+0000","last_fullsized":"2024-09-10T09:54:29.699735+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:19.214291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"96'6","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.704960+0000","last_change":"2024-09-10T09:52:22.323035+0000","last_active":"2024-09-10T09:54:29.704960+0000","last_peered":"2024-09-10T09:54:29.704960+0000","last_clean":"2024-09-10T09:54:29.704960+0000","last_became_active":"2024-09-10T09:52:22.322679+0000","last_became_peered":"2024-09-10T09:52:22.322679+0000","last_unstale":"2024-09-10T09:54:29.704960+0000","last_undegraded":"2024-09-10T09:54:29.704960+0000","last_fullsized":"2024-09-10T09:54:29.704960+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:23:13.762612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"96'4","reported_seq":40,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.700111+0000","last_change":"2024-09-10T09:52:22.315887+0000","last_active":"2024-09-10T09:54:29.700111+0000","last_peered":"2024-09-10T09:54:29.700111+0000","last_clean":"2024-09-10T09:54:29.700111+0000","last_became_active":"2024-09-10T09:52:22.315569+0000","last_became_peered":"2024-09-10T09:52:22.315569+0000","last_unstale":"2024-09-10T09:54:29.700111+0000","last_undegraded":"2024-09-10T09:54:29.700111+0000","last_fullsized":"2024-09-10T09:54:29.700111+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:46:57.094320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"96'8","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.676312+0000","last_change":"2024-09-10T09:52:22.314060+0000","last_active":"2024-09-10T09:54:29.676312+0000","last_peered":"2024-09-10T09:54:29.676312+0000","last_clean":"2024-09-10T09:54:29.676312+0000","last_became_active":"2024-09-10T09:52:22.313572+0000","last_became_peered":"2024-09-10T09:52:22.313572+0000","last_unstale":"2024-09-10T09:54:29.676312+0000","last_undegraded":"2024-09-10T09:54:29.676312+0000","last_fullsized":"2024-09-10T09:54:29.676312+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:11:17.158625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.692163+0000","last_change":"2024-09-10T09:52:22.327233+0000","last_active":"2024-09-10T09:54:29.692163+0000","last_peered":"2024-09-10T09:54:29.692163+0000","last_clean":"2024-09-10T09:54:29.692163+0000","last_became_active":"2024-09-10T09:52:22.326945+0000","last_became_peered":"2024-09-10T09:52:22.326945+0000","last_unstale":"2024-09-10T09:54:29.692163+0000","last_undegraded":"2024-09-10T09:54:29.692163+0000","last_fullsized":"2024-09-10T09:54:29.692163+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:31:13.490147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"96'4","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.681664+0000","last_change":"2024-09-10T09:52:22.330912+0000","last_active":"2024-09-10T09:54:29.681664+0000","last_peered":"2024-09-10T09:54:29.681664+0000","last_clean":"2024-09-10T09:54:29.681664+0000","last_became_active":"2024-09-10T09:52:22.330149+0000","last_became_peered":"2024-09-10T09:52:22.330149+0000","last_unstale":"2024-09-10T09:54:29.681664+0000","last_undegraded":"2024-09-10T09:54:29.681664+0000","last_fullsized":"2024-09-10T09:54:29.681664+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:10:48.842499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"96'8","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.697891+0000","last_change":"2024-09-10T09:52:22.327035+0000","last_active":"2024-09-10T09:54:29.697891+0000","last_peered":"2024-09-10T09:54:29.697891+0000","last_clean":"2024-09-10T09:54:29.697891+0000","last_became_active":"2024-09-10T09:52:22.326741+0000","last_became_peered":"2024-09-10T09:52:22.326741+0000","last_unstale":"2024-09-10T09:54:29.697891+0000","last_undegraded":"2024-09-10T09:54:29.697891+0000","last_fullsized":"2024-09-10T09:54:29.697891+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:21:05.026614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70741,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1211,"num_read_kb":685,"num_write":259,"num_write_kb":150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":4426,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":213467,"data_compressed":5661,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":223,"ondisk_log_size":223,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282980,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449137,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1073152,"data_stored":554558,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":70,"seq":300647710772,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32372,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712780,"statfs":{"total":95995035648,"available":95961886720,"internally_reserved":0,"allocated":1216512,"data_stored":1204571,"data_compressed":70462,"data_compressed_allocated":94208,"data_compressed_original":688128,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":65,"seq":279172874300,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32256,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712896,"statfs":{"total":95995035648,"available":95962005504,"internally_reserved":0,"allocated":1097728,"data_stored":557996,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135938,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32244,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712908,"statfs":{"total":95995035648,"available":95962017792,"internally_reserved":0,"allocated":1085440,"data_stored":557702,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364870,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28136,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717016,"statfs":{"total":95995035648,"available":95966224384,"internally_reserved":0,"allocated":1138688,"data_stored":1150864,"data_compressed":68575,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528396,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1069056,"data_stored":549715,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790034,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28144,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717008,"statfs":{"total":95995035648,"available":95966216192,"internally_reserved":0,"allocated":1146880,"data_stored":1142404,"data_compressed":68575,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953557,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28144,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717008,"statfs":{"total":95995035648,"available":95966216192,"internally_reserved":0,"allocated":1146880,"data_stored":617574,"data_compressed":34470,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182491,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32240,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712912,"statfs":{"total":95995035648,"available":95962021888,"internally_reserved":0,"allocated":1081344,"data_stored":549112,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411425,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36404,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708748,"statfs":{"total":95995035648,"available":95957757952,"internally_reserved":0,"allocated":1085440,"data_stored":551188,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574951,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28140,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717012,"statfs":{"total":95995035648,"available":95966220288,"internally_reserved":0,"allocated":1142784,"data_stored":614979,"data_compressed":34470,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738476,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1093632,"data_stored":550860,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34340,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34340,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34340,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35992,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35992,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35992,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35992,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35992,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":66297,"data_compressed":1887,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":64796,"data_compressed":1887,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":65521,"data_compressed":1887,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T09:55:18.170 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T09:55:18.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:55:17 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/2339218315' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:55:18.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.362+0000 7fc8b1000640 1 -- 172.21.15.77:0/2706054108 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc8ac103d30 msgr2=0x7fc8ac104130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:18.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.362+0000 7fc8b1000640 1 --2- 172.21.15.77:0/2706054108 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc8ac103d30 0x7fc8ac104130 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc89c009860 tx=0x7fc89c02a340 comp rx=0 tx=0).stop 2024-09-10T09:55:18.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.362+0000 7fc8b1000640 1 -- 172.21.15.77:0/2706054108 shutdown_connections 2024-09-10T09:55:18.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.362+0000 7fc8b1000640 1 --2- 172.21.15.77:0/2706054108 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc8ac1058e0 0x7fc8ac10c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:18.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.362+0000 7fc8b1000640 1 --2- 172.21.15.77:0/2706054108 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8ac104f20 0x7fc8ac1053a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:18.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.362+0000 7fc8b1000640 1 --2- 172.21.15.77:0/2706054108 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc8ac103d30 0x7fc8ac104130 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:18.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.362+0000 7fc8b1000640 1 -- 172.21.15.77:0/2706054108 >> 172.21.15.77:0/2706054108 conn(0x7fc8ac0ff4b0 msgr2=0x7fc8ac1018d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:18.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.362+0000 7fc8b1000640 1 -- 172.21.15.77:0/2706054108 shutdown_connections 2024-09-10T09:55:18.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.362+0000 7fc8b1000640 1 -- 172.21.15.77:0/2706054108 wait complete. 2024-09-10T09:55:18.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.363+0000 7fc8b1000640 1 Processor -- start 2024-09-10T09:55:18.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.363+0000 7fc8b1000640 1 -- start start 2024-09-10T09:55:18.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.363+0000 7fc8b1000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc8ac103d30 0x7fc8ac19a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:18.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.363+0000 7fc8b1000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8ac104f20 0x7fc8ac19a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:18.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.363+0000 7fc8b1000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc8ac1058e0 0x7fc8ac1a1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:18.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.363+0000 7fc8b1000640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc8ac10f9b0 con 0x7fc8ac1058e0 2024-09-10T09:55:18.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.363+0000 7fc8b1000640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc8ac10f830 con 0x7fc8ac103d30 2024-09-10T09:55:18.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.363+0000 7fc8b1000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc8ac10fb30 con 0x7fc8ac104f20 2024-09-10T09:55:18.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.363+0000 7fc8aa800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc8ac1058e0 0x7fc8ac1a1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:18.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.363+0000 7fc8aa800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc8ac1058e0 0x7fc8ac1a1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52730/0 (socket says 172.21.15.77:52730) 2024-09-10T09:55:18.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.363+0000 7fc8a9e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc8ac103d30 0x7fc8ac19a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:18.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.363+0000 7fc8aa800640 1 -- 172.21.15.77:0/4049547015 learned_addr learned my addr 172.21.15.77:0/4049547015 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:55:18.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.363+0000 7fc8a1400640 1 --2- 172.21.15.77:0/4049547015 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8ac104f20 0x7fc8ac19a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:18.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.364+0000 7fc8aa800640 1 -- 172.21.15.77:0/4049547015 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8ac104f20 msgr2=0x7fc8ac19a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:18.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.364+0000 7fc8aa800640 1 --2- 172.21.15.77:0/4049547015 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8ac104f20 0x7fc8ac19a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:18.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.364+0000 7fc8aa800640 1 -- 172.21.15.77:0/4049547015 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc8ac103d30 msgr2=0x7fc8ac19a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:18.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.364+0000 7fc8aa800640 1 --2- 172.21.15.77:0/4049547015 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc8ac103d30 0x7fc8ac19a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:18.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.364+0000 7fc8aa800640 1 -- 172.21.15.77:0/4049547015 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8ac1a1cc0 con 0x7fc8ac1058e0 2024-09-10T09:55:18.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.364+0000 7fc8a9e00640 1 --2- 172.21.15.77:0/4049547015 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc8ac103d30 0x7fc8ac19a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:55:18.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.364+0000 7fc8aa800640 1 --2- 172.21.15.77:0/4049547015 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc8ac1058e0 0x7fc8ac1a1650 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fc8a400efc0 tx=0x7fc8a400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:18.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.364+0000 7fc8a3400640 1 -- 172.21.15.77:0/4049547015 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8a4003970 con 0x7fc8ac1058e0 2024-09-10T09:55:18.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.364+0000 7fc8a3400640 1 -- 172.21.15.77:0/4049547015 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8a4004020 con 0x7fc8ac1058e0 2024-09-10T09:55:18.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.364+0000 7fc8b1000640 1 -- 172.21.15.77:0/4049547015 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8ac1a1fb0 con 0x7fc8ac1058e0 2024-09-10T09:55:18.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.365+0000 7fc8a3400640 1 -- 172.21.15.77:0/4049547015 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8a400b9f0 con 0x7fc8ac1058e0 2024-09-10T09:55:18.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.365+0000 7fc8b1000640 1 -- 172.21.15.77:0/4049547015 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8ac1a2420 con 0x7fc8ac1058e0 2024-09-10T09:55:18.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.366+0000 7fc8b1000640 1 -- 172.21.15.77:0/4049547015 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8ac10b120 con 0x7fc8ac1058e0 2024-09-10T09:55:18.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.368+0000 7fc8a3400640 1 -- 172.21.15.77:0/4049547015 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc8a401d020 con 0x7fc8ac1058e0 2024-09-10T09:55:18.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.369+0000 7fc8a3400640 1 --2- 172.21.15.77:0/4049547015 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fc8900a2b90 0x7fc8900a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:18.370 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.369+0000 7fc8a3400640 1 -- 172.21.15.77:0/4049547015 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fc8a40cf910 con 0x7fc8ac1058e0 2024-09-10T09:55:18.370 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.369+0000 7fc8a9e00640 1 --2- 172.21.15.77:0/4049547015 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fc8900a2b90 0x7fc8900a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:18.370 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.370+0000 7fc8a9e00640 1 --2- 172.21.15.77:0/4049547015 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fc8900a2b90 0x7fc8900a5050 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc89c030410 tx=0x7fc89c030550 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:18.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.371+0000 7fc8a3400640 1 -- 172.21.15.77:0/4049547015 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc8a409c420 con 0x7fc8ac1058e0 2024-09-10T09:55:18.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:55:17 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/2339218315' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:55:18.543 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.542+0000 7fc8b1000640 1 -- 172.21.15.77:0/4049547015 --> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc8ac062630 con 0x7fc8900a2b90 2024-09-10T09:55:18.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.550+0000 7fc8a3400640 1 -- 172.21.15.77:0/4049547015 <== mgr.14150 v2:172.21.15.77:6800/4261324650 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340852 (secure 0 0 0) 0x7fc8ac062630 con 0x7fc8900a2b90 2024-09-10T09:55:18.555 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:55:18.557 INFO:teuthology.orchestra.run.smithi077.stderr:dumped all 2024-09-10T09:55:18.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.555+0000 7fc8b1000640 1 -- 172.21.15.77:0/4049547015 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fc8900a2b90 msgr2=0x7fc8900a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:18.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.555+0000 7fc8b1000640 1 --2- 172.21.15.77:0/4049547015 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fc8900a2b90 0x7fc8900a5050 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc89c030410 tx=0x7fc89c030550 comp rx=0 tx=0).stop 2024-09-10T09:55:18.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.555+0000 7fc8b1000640 1 -- 172.21.15.77:0/4049547015 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc8ac1058e0 msgr2=0x7fc8ac1a1650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:18.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.555+0000 7fc8b1000640 1 --2- 172.21.15.77:0/4049547015 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc8ac1058e0 0x7fc8ac1a1650 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fc8a400efc0 tx=0x7fc8a400c800 comp rx=0 tx=0).stop 2024-09-10T09:55:18.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.555+0000 7fc8b1000640 1 -- 172.21.15.77:0/4049547015 shutdown_connections 2024-09-10T09:55:18.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.555+0000 7fc8b1000640 1 --2- 172.21.15.77:0/4049547015 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fc8900a2b90 0x7fc8900a5050 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:18.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.555+0000 7fc8b1000640 1 --2- 172.21.15.77:0/4049547015 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc8ac1058e0 0x7fc8ac1a1650 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:18.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.555+0000 7fc8b1000640 1 --2- 172.21.15.77:0/4049547015 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8ac104f20 0x7fc8ac19a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:18.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.555+0000 7fc8b1000640 1 --2- 172.21.15.77:0/4049547015 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc8ac103d30 0x7fc8ac19a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:18.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.555+0000 7fc8b1000640 1 -- 172.21.15.77:0/4049547015 >> 172.21.15.77:0/4049547015 conn(0x7fc8ac0ff4b0 msgr2=0x7fc8ac1082f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:18.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.555+0000 7fc8b1000640 1 -- 172.21.15.77:0/4049547015 shutdown_connections 2024-09-10T09:55:18.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.555+0000 7fc8b1000640 1 -- 172.21.15.77:0/4049547015 wait complete. 2024-09-10T09:55:18.572 INFO:teuthology.orchestra.run.smithi077.stdout:{"pg_ready":true,"pg_map":{"version":395,"stamp":"2024-09-10T09:55:17.719147+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":661343,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1337,"num_read_kb":794,"num_write":442,"num_write_kb":3090,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1185162,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":862,"ondisk_log_size":862,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":366472,"kb_used_data":13064,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124575352,"statfs":{"total":1151940427776,"available":1151565160448,"internally_reserved":0,"allocated":13377536,"data_stored":8601523,"data_compressed":504633,"data_compressed_allocated":688128,"data_compressed_original":2875392,"omap_allocated":19143,"internal_metadata":361674041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":105,"num_read_kb":55,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003408"},"pg_stats":[{"pgid":"2.1f","version":"96'14","reported_seq":188,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:55:15.056536+0000","last_change":"2024-09-10T09:53:00.803590+0000","last_active":"2024-09-10T09:55:15.056536+0000","last_peered":"2024-09-10T09:55:15.056536+0000","last_clean":"2024-09-10T09:55:15.056536+0000","last_became_active":"2024-09-10T09:53:00.798189+0000","last_became_peered":"2024-09-10T09:53:00.798189+0000","last_unstale":"2024-09-10T09:55:15.056536+0000","last_undegraded":"2024-09-10T09:55:15.056536+0000","last_fullsized":"2024-09-10T09:55:15.056536+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:18:48.808836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":121,"num_read_kb":121,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"96'5","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.682459+0000","last_change":"2024-09-10T09:52:22.318195+0000","last_active":"2024-09-10T09:54:29.682459+0000","last_peered":"2024-09-10T09:54:29.682459+0000","last_clean":"2024-09-10T09:54:29.682459+0000","last_became_active":"2024-09-10T09:52:22.315905+0000","last_became_peered":"2024-09-10T09:52:22.315905+0000","last_unstale":"2024-09-10T09:54:29.682459+0000","last_undegraded":"2024-09-10T09:54:29.682459+0000","last_fullsized":"2024-09-10T09:54:29.682459+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:53:55.515984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.843988+0000","last_change":"2024-09-10T09:52:31.369780+0000","last_active":"2024-09-10T09:54:00.843988+0000","last_peered":"2024-09-10T09:54:00.843988+0000","last_clean":"2024-09-10T09:54:00.843988+0000","last_became_active":"2024-09-10T09:52:31.369238+0000","last_became_peered":"2024-09-10T09:52:31.369238+0000","last_unstale":"2024-09-10T09:54:00.843988+0000","last_undegraded":"2024-09-10T09:54:00.843988+0000","last_fullsized":"2024-09-10T09:54:00.843988+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:39:05.318493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"94'2","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.847203+0000","last_change":"2024-09-10T09:52:19.912276+0000","last_active":"2024-09-10T09:54:00.847203+0000","last_peered":"2024-09-10T09:54:00.847203+0000","last_clean":"2024-09-10T09:54:00.847203+0000","last_became_active":"2024-09-10T09:52:19.911452+0000","last_became_peered":"2024-09-10T09:52:19.911452+0000","last_unstale":"2024-09-10T09:54:00.847203+0000","last_undegraded":"2024-09-10T09:54:00.847203+0000","last_fullsized":"2024-09-10T09:54:00.847203+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:24:31.262207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"96'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.705105+0000","last_change":"2024-09-10T09:52:22.327641+0000","last_active":"2024-09-10T09:54:29.705105+0000","last_peered":"2024-09-10T09:54:29.705105+0000","last_clean":"2024-09-10T09:54:29.705105+0000","last_became_active":"2024-09-10T09:52:22.326969+0000","last_became_peered":"2024-09-10T09:52:22.326969+0000","last_unstale":"2024-09-10T09:54:29.705105+0000","last_undegraded":"2024-09-10T09:54:29.705105+0000","last_fullsized":"2024-09-10T09:54:29.705105+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:35:27.825534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.951014+0000","last_change":"2024-09-10T09:52:31.372360+0000","last_active":"2024-09-10T09:54:00.951014+0000","last_peered":"2024-09-10T09:54:00.951014+0000","last_clean":"2024-09-10T09:54:00.951014+0000","last_became_active":"2024-09-10T09:52:31.371957+0000","last_became_peered":"2024-09-10T09:52:31.371957+0000","last_unstale":"2024-09-10T09:54:00.951014+0000","last_undegraded":"2024-09-10T09:54:00.951014+0000","last_fullsized":"2024-09-10T09:54:00.951014+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:49:34.776185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"96'13","reported_seq":55,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:51.750986+0000","last_change":"2024-09-10T09:52:19.843532+0000","last_active":"2024-09-10T09:54:51.750986+0000","last_peered":"2024-09-10T09:54:51.750986+0000","last_clean":"2024-09-10T09:54:51.750986+0000","last_became_active":"2024-09-10T09:52:19.843070+0000","last_became_peered":"2024-09-10T09:52:19.843070+0000","last_unstale":"2024-09-10T09:54:51.750986+0000","last_undegraded":"2024-09-10T09:54:51.750986+0000","last_fullsized":"2024-09-10T09:54:51.750986+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:08:02.223373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"96'9","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.692662+0000","last_change":"2024-09-10T09:52:22.316981+0000","last_active":"2024-09-10T09:54:29.692662+0000","last_peered":"2024-09-10T09:54:29.692662+0000","last_clean":"2024-09-10T09:54:29.692662+0000","last_became_active":"2024-09-10T09:52:22.316648+0000","last_became_peered":"2024-09-10T09:52:22.316648+0000","last_unstale":"2024-09-10T09:54:29.692662+0000","last_undegraded":"2024-09-10T09:54:29.692662+0000","last_fullsized":"2024-09-10T09:54:29.692662+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:28:56.742496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.951485+0000","last_change":"2024-09-10T09:52:31.376811+0000","last_active":"2024-09-10T09:54:00.951485+0000","last_peered":"2024-09-10T09:54:00.951485+0000","last_clean":"2024-09-10T09:54:00.951485+0000","last_became_active":"2024-09-10T09:52:31.376007+0000","last_became_peered":"2024-09-10T09:52:31.376007+0000","last_unstale":"2024-09-10T09:54:00.951485+0000","last_undegraded":"2024-09-10T09:54:00.951485+0000","last_fullsized":"2024-09-10T09:54:00.951485+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:29:26.213281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"94'5","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.149772+0000","last_change":"2024-09-10T09:52:19.837848+0000","last_active":"2024-09-10T09:54:01.149772+0000","last_peered":"2024-09-10T09:54:01.149772+0000","last_clean":"2024-09-10T09:54:01.149772+0000","last_became_active":"2024-09-10T09:52:19.837383+0000","last_became_peered":"2024-09-10T09:52:19.837383+0000","last_unstale":"2024-09-10T09:54:01.149772+0000","last_undegraded":"2024-09-10T09:54:01.149772+0000","last_fullsized":"2024-09-10T09:54:01.149772+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:50:42.813665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"96'15","reported_seq":60,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.685844+0000","last_change":"2024-09-10T09:54:01.060407+0000","last_active":"2024-09-10T09:54:29.685844+0000","last_peered":"2024-09-10T09:54:29.685844+0000","last_clean":"2024-09-10T09:54:29.685844+0000","last_became_active":"2024-09-10T09:54:01.060126+0000","last_became_peered":"2024-09-10T09:54:01.060126+0000","last_unstale":"2024-09-10T09:54:29.685844+0000","last_undegraded":"2024-09-10T09:54:29.685844+0000","last_fullsized":"2024-09-10T09:54:29.685844+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:59:29.765375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.951119+0000","last_change":"2024-09-10T09:52:31.370910+0000","last_active":"2024-09-10T09:54:00.951119+0000","last_peered":"2024-09-10T09:54:00.951119+0000","last_clean":"2024-09-10T09:54:00.951119+0000","last_became_active":"2024-09-10T09:52:31.370467+0000","last_became_peered":"2024-09-10T09:52:31.370467+0000","last_unstale":"2024-09-10T09:54:00.951119+0000","last_undegraded":"2024-09-10T09:54:00.951119+0000","last_fullsized":"2024-09-10T09:54:00.951119+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:49:25.037820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"96'5","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.702762+0000","last_change":"2024-09-10T09:52:22.310708+0000","last_active":"2024-09-10T09:54:29.702762+0000","last_peered":"2024-09-10T09:54:29.702762+0000","last_clean":"2024-09-10T09:54:29.702762+0000","last_became_active":"2024-09-10T09:52:22.310387+0000","last_became_peered":"2024-09-10T09:52:22.310387+0000","last_unstale":"2024-09-10T09:54:29.702762+0000","last_undegraded":"2024-09-10T09:54:29.702762+0000","last_fullsized":"2024-09-10T09:54:29.702762+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:35:03.115408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"94'4","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.149453+0000","last_change":"2024-09-10T09:52:19.841417+0000","last_active":"2024-09-10T09:54:01.149453+0000","last_peered":"2024-09-10T09:54:01.149453+0000","last_clean":"2024-09-10T09:54:01.149453+0000","last_became_active":"2024-09-10T09:52:19.841083+0000","last_became_peered":"2024-09-10T09:52:19.841083+0000","last_unstale":"2024-09-10T09:54:01.149453+0000","last_undegraded":"2024-09-10T09:54:01.149453+0000","last_fullsized":"2024-09-10T09:54:01.149453+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:01:19.008345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.951294+0000","last_change":"2024-09-10T09:52:31.367406+0000","last_active":"2024-09-10T09:54:00.951294+0000","last_peered":"2024-09-10T09:54:00.951294+0000","last_clean":"2024-09-10T09:54:00.951294+0000","last_became_active":"2024-09-10T09:52:31.366961+0000","last_became_peered":"2024-09-10T09:52:31.366961+0000","last_unstale":"2024-09-10T09:54:00.951294+0000","last_undegraded":"2024-09-10T09:54:00.951294+0000","last_fullsized":"2024-09-10T09:54:00.951294+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:04:51.714465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"96'8","reported_seq":19,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.705252+0000","last_change":"2024-09-10T09:54:01.249212+0000","last_active":"2024-09-10T09:54:29.705252+0000","last_peered":"2024-09-10T09:54:29.705252+0000","last_clean":"2024-09-10T09:54:29.705252+0000","last_became_active":"2024-09-10T09:54:01.248840+0000","last_became_peered":"2024-09-10T09:54:01.248840+0000","last_unstale":"2024-09-10T09:54:29.705252+0000","last_undegraded":"2024-09-10T09:54:29.705252+0000","last_fullsized":"2024-09-10T09:54:29.705252+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:39:13.167435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,10],"acting":[9,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.a","version":"94'3","reported_seq":19,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.954964+0000","last_change":"2024-09-10T09:53:00.800421+0000","last_active":"2024-09-10T09:54:00.954964+0000","last_peered":"2024-09-10T09:54:00.954964+0000","last_clean":"2024-09-10T09:54:00.954964+0000","last_became_active":"2024-09-10T09:53:00.800026+0000","last_became_peered":"2024-09-10T09:53:00.800026+0000","last_unstale":"2024-09-10T09:54:00.954964+0000","last_undegraded":"2024-09-10T09:54:00.954964+0000","last_fullsized":"2024-09-10T09:54:00.954964+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:57:45.252420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"94'1","reported_seq":30,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.951388+0000","last_change":"2024-09-10T09:52:31.376605+0000","last_active":"2024-09-10T09:54:00.951388+0000","last_peered":"2024-09-10T09:54:00.951388+0000","last_clean":"2024-09-10T09:54:00.951388+0000","last_became_active":"2024-09-10T09:52:31.376083+0000","last_became_peered":"2024-09-10T09:52:31.376083+0000","last_unstale":"2024-09-10T09:54:00.951388+0000","last_undegraded":"2024-09-10T09:54:00.951388+0000","last_fullsized":"2024-09-10T09:54:00.951388+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:05:10.030673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"96'9","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.697029+0000","last_change":"2024-09-10T09:52:22.327303+0000","last_active":"2024-09-10T09:54:29.697029+0000","last_peered":"2024-09-10T09:54:29.697029+0000","last_clean":"2024-09-10T09:54:29.697029+0000","last_became_active":"2024-09-10T09:52:22.326753+0000","last_became_peered":"2024-09-10T09:52:22.326753+0000","last_unstale":"2024-09-10T09:54:29.697029+0000","last_undegraded":"2024-09-10T09:54:29.697029+0000","last_fullsized":"2024-09-10T09:54:29.697029+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:27:26.668151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"94'7","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.845956+0000","last_change":"2024-09-10T09:52:19.836484+0000","last_active":"2024-09-10T09:54:00.845956+0000","last_peered":"2024-09-10T09:54:00.845956+0000","last_clean":"2024-09-10T09:54:00.845956+0000","last_became_active":"2024-09-10T09:52:19.836130+0000","last_became_peered":"2024-09-10T09:52:19.836130+0000","last_unstale":"2024-09-10T09:54:00.845956+0000","last_undegraded":"2024-09-10T09:54:00.845956+0000","last_fullsized":"2024-09-10T09:54:00.845956+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:31:58.731981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.844293+0000","last_change":"2024-09-10T09:52:31.375279+0000","last_active":"2024-09-10T09:54:00.844293+0000","last_peered":"2024-09-10T09:54:00.844293+0000","last_clean":"2024-09-10T09:54:00.844293+0000","last_became_active":"2024-09-10T09:52:31.374181+0000","last_became_peered":"2024-09-10T09:52:31.374181+0000","last_unstale":"2024-09-10T09:54:00.844293+0000","last_undegraded":"2024-09-10T09:54:00.844293+0000","last_fullsized":"2024-09-10T09:54:00.844293+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:54:51.299224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"96'2","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.676489+0000","last_change":"2024-09-10T09:52:22.314817+0000","last_active":"2024-09-10T09:54:29.676489+0000","last_peered":"2024-09-10T09:54:29.676489+0000","last_clean":"2024-09-10T09:54:29.676489+0000","last_became_active":"2024-09-10T09:52:22.314491+0000","last_became_peered":"2024-09-10T09:52:22.314491+0000","last_unstale":"2024-09-10T09:54:29.676489+0000","last_undegraded":"2024-09-10T09:54:29.676489+0000","last_fullsized":"2024-09-10T09:54:29.676489+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:49:41.941667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"94'2","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.988558+0000","last_change":"2024-09-10T09:52:19.911730+0000","last_active":"2024-09-10T09:54:00.988558+0000","last_peered":"2024-09-10T09:54:00.988558+0000","last_clean":"2024-09-10T09:54:00.988558+0000","last_became_active":"2024-09-10T09:52:19.911393+0000","last_became_peered":"2024-09-10T09:52:19.911393+0000","last_unstale":"2024-09-10T09:54:00.988558+0000","last_undegraded":"2024-09-10T09:54:00.988558+0000","last_fullsized":"2024-09-10T09:54:00.988558+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:46:17.943129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.951301+0000","last_change":"2024-09-10T09:52:31.376498+0000","last_active":"2024-09-10T09:54:00.951301+0000","last_peered":"2024-09-10T09:54:00.951301+0000","last_clean":"2024-09-10T09:54:00.951301+0000","last_became_active":"2024-09-10T09:52:31.375966+0000","last_became_peered":"2024-09-10T09:52:31.375966+0000","last_unstale":"2024-09-10T09:54:00.951301+0000","last_undegraded":"2024-09-10T09:54:00.951301+0000","last_fullsized":"2024-09-10T09:54:00.951301+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:08:30.649399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.705573+0000","last_change":"2024-09-10T09:52:22.331611+0000","last_active":"2024-09-10T09:54:29.705573+0000","last_peered":"2024-09-10T09:54:29.705573+0000","last_clean":"2024-09-10T09:54:29.705573+0000","last_became_active":"2024-09-10T09:52:22.331268+0000","last_became_peered":"2024-09-10T09:52:22.331268+0000","last_unstale":"2024-09-10T09:54:29.705573+0000","last_undegraded":"2024-09-10T09:54:29.705573+0000","last_fullsized":"2024-09-10T09:54:29.705573+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:01:05.433994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"94'7","reported_seq":50,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.951927+0000","last_change":"2024-09-10T09:52:19.838387+0000","last_active":"2024-09-10T09:54:00.951927+0000","last_peered":"2024-09-10T09:54:00.951927+0000","last_clean":"2024-09-10T09:54:00.951927+0000","last_became_active":"2024-09-10T09:52:19.837889+0000","last_became_peered":"2024-09-10T09:52:19.837889+0000","last_unstale":"2024-09-10T09:54:00.951927+0000","last_undegraded":"2024-09-10T09:54:00.951927+0000","last_fullsized":"2024-09-10T09:54:00.951927+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:50:00.187243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.844187+0000","last_change":"2024-09-10T09:52:31.363262+0000","last_active":"2024-09-10T09:54:00.844187+0000","last_peered":"2024-09-10T09:54:00.844187+0000","last_clean":"2024-09-10T09:54:00.844187+0000","last_became_active":"2024-09-10T09:52:31.362762+0000","last_became_peered":"2024-09-10T09:52:31.362762+0000","last_unstale":"2024-09-10T09:54:00.844187+0000","last_undegraded":"2024-09-10T09:54:00.844187+0000","last_fullsized":"2024-09-10T09:54:00.844187+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:48:28.275041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"96'5","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.702169+0000","last_change":"2024-09-10T09:52:22.312472+0000","last_active":"2024-09-10T09:54:29.702169+0000","last_peered":"2024-09-10T09:54:29.702169+0000","last_clean":"2024-09-10T09:54:29.702169+0000","last_became_active":"2024-09-10T09:52:22.312117+0000","last_became_peered":"2024-09-10T09:52:22.312117+0000","last_unstale":"2024-09-10T09:54:29.702169+0000","last_undegraded":"2024-09-10T09:54:29.702169+0000","last_fullsized":"2024-09-10T09:54:29.702169+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:39:10.742137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"96'16","reported_seq":57,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:31.212993+0000","last_change":"2024-09-10T09:52:19.835543+0000","last_active":"2024-09-10T09:54:31.212993+0000","last_peered":"2024-09-10T09:54:31.212993+0000","last_clean":"2024-09-10T09:54:31.212993+0000","last_became_active":"2024-09-10T09:52:19.835170+0000","last_became_peered":"2024-09-10T09:52:19.835170+0000","last_unstale":"2024-09-10T09:54:31.212993+0000","last_undegraded":"2024-09-10T09:54:31.212993+0000","last_fullsized":"2024-09-10T09:54:31.212993+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:37.969277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.845447+0000","last_change":"2024-09-10T09:52:31.372398+0000","last_active":"2024-09-10T09:54:00.845447+0000","last_peered":"2024-09-10T09:54:00.845447+0000","last_clean":"2024-09-10T09:54:00.845447+0000","last_became_active":"2024-09-10T09:52:31.371874+0000","last_became_peered":"2024-09-10T09:52:31.371874+0000","last_unstale":"2024-09-10T09:54:00.845447+0000","last_undegraded":"2024-09-10T09:54:00.845447+0000","last_fullsized":"2024-09-10T09:54:00.845447+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:09:45.374320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"96'9","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.700381+0000","last_change":"2024-09-10T09:52:22.319592+0000","last_active":"2024-09-10T09:54:29.700381+0000","last_peered":"2024-09-10T09:54:29.700381+0000","last_clean":"2024-09-10T09:54:29.700381+0000","last_became_active":"2024-09-10T09:52:22.319298+0000","last_became_peered":"2024-09-10T09:52:22.319298+0000","last_unstale":"2024-09-10T09:54:29.700381+0000","last_undegraded":"2024-09-10T09:54:29.700381+0000","last_fullsized":"2024-09-10T09:54:29.700381+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:56:59.787638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"94'4","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.150011+0000","last_change":"2024-09-10T09:52:19.912923+0000","last_active":"2024-09-10T09:54:01.150011+0000","last_peered":"2024-09-10T09:54:01.150011+0000","last_clean":"2024-09-10T09:54:01.150011+0000","last_became_active":"2024-09-10T09:52:19.912268+0000","last_became_peered":"2024-09-10T09:52:19.912268+0000","last_unstale":"2024-09-10T09:54:01.150011+0000","last_undegraded":"2024-09-10T09:54:01.150011+0000","last_fullsized":"2024-09-10T09:54:01.150011+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:06:45.421004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.955091+0000","last_change":"2024-09-10T09:52:31.372761+0000","last_active":"2024-09-10T09:54:00.955091+0000","last_peered":"2024-09-10T09:54:00.955091+0000","last_clean":"2024-09-10T09:54:00.955091+0000","last_became_active":"2024-09-10T09:52:31.372293+0000","last_became_peered":"2024-09-10T09:52:31.372293+0000","last_unstale":"2024-09-10T09:54:00.955091+0000","last_undegraded":"2024-09-10T09:54:00.955091+0000","last_fullsized":"2024-09-10T09:54:00.955091+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:36:41.446810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"96'4","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.688817+0000","last_change":"2024-09-10T09:52:22.326291+0000","last_active":"2024-09-10T09:54:29.688817+0000","last_peered":"2024-09-10T09:54:29.688817+0000","last_clean":"2024-09-10T09:54:29.688817+0000","last_became_active":"2024-09-10T09:52:22.325985+0000","last_became_peered":"2024-09-10T09:52:22.325985+0000","last_unstale":"2024-09-10T09:54:29.688817+0000","last_undegraded":"2024-09-10T09:54:29.688817+0000","last_fullsized":"2024-09-10T09:54:29.688817+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:06:21.871502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"94'1","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.844068+0000","last_change":"2024-09-10T09:52:19.912851+0000","last_active":"2024-09-10T09:54:00.844068+0000","last_peered":"2024-09-10T09:54:00.844068+0000","last_clean":"2024-09-10T09:54:00.844068+0000","last_became_active":"2024-09-10T09:52:19.912200+0000","last_became_peered":"2024-09-10T09:52:19.912200+0000","last_unstale":"2024-09-10T09:54:00.844068+0000","last_undegraded":"2024-09-10T09:54:00.844068+0000","last_fullsized":"2024-09-10T09:54:00.844068+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:00:47.331755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.149311+0000","last_change":"2024-09-10T09:52:31.371329+0000","last_active":"2024-09-10T09:54:01.149311+0000","last_peered":"2024-09-10T09:54:01.149311+0000","last_clean":"2024-09-10T09:54:01.149311+0000","last_became_active":"2024-09-10T09:52:31.370971+0000","last_became_peered":"2024-09-10T09:52:31.370971+0000","last_unstale":"2024-09-10T09:54:01.149311+0000","last_undegraded":"2024-09-10T09:54:01.149311+0000","last_fullsized":"2024-09-10T09:54:01.149311+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:07:12.928421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"96'7","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.686209+0000","last_change":"2024-09-10T09:52:22.328039+0000","last_active":"2024-09-10T09:54:29.686209+0000","last_peered":"2024-09-10T09:54:29.686209+0000","last_clean":"2024-09-10T09:54:29.686209+0000","last_became_active":"2024-09-10T09:52:22.327588+0000","last_became_peered":"2024-09-10T09:52:22.327588+0000","last_unstale":"2024-09-10T09:54:29.686209+0000","last_undegraded":"2024-09-10T09:54:29.686209+0000","last_fullsized":"2024-09-10T09:54:29.686209+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:35:39.219544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"94'5","reported_seq":138,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:55:11.050835+0000","last_change":"2024-09-10T09:53:00.797651+0000","last_active":"2024-09-10T09:55:11.050835+0000","last_peered":"2024-09-10T09:55:11.050835+0000","last_clean":"2024-09-10T09:55:11.050835+0000","last_became_active":"2024-09-10T09:53:00.797285+0000","last_became_peered":"2024-09-10T09:53:00.797285+0000","last_unstale":"2024-09-10T09:55:11.050835+0000","last_undegraded":"2024-09-10T09:55:11.050835+0000","last_fullsized":"2024-09-10T09:55:11.050835+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:18:02.320032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,6,1],"acting":[11,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.841787+0000","last_change":"2024-09-10T09:52:31.379215+0000","last_active":"2024-09-10T09:54:00.841787+0000","last_peered":"2024-09-10T09:54:00.841787+0000","last_clean":"2024-09-10T09:54:00.841787+0000","last_became_active":"2024-09-10T09:52:31.377107+0000","last_became_peered":"2024-09-10T09:52:31.377107+0000","last_unstale":"2024-09-10T09:54:00.841787+0000","last_undegraded":"2024-09-10T09:54:00.841787+0000","last_fullsized":"2024-09-10T09:54:00.841787+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:09:39.474077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"96'8","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.673518+0000","last_change":"2024-09-10T09:52:22.316474+0000","last_active":"2024-09-10T09:54:29.673518+0000","last_peered":"2024-09-10T09:54:29.673518+0000","last_clean":"2024-09-10T09:54:29.673518+0000","last_became_active":"2024-09-10T09:52:22.316054+0000","last_became_peered":"2024-09-10T09:52:22.316054+0000","last_unstale":"2024-09-10T09:54:29.673518+0000","last_undegraded":"2024-09-10T09:54:29.673518+0000","last_fullsized":"2024-09-10T09:54:29.673518+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:11:31.334780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"96'6","reported_seq":166,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:55:12.082651+0000","last_change":"2024-09-10T09:52:19.911601+0000","last_active":"2024-09-10T09:55:12.082651+0000","last_peered":"2024-09-10T09:55:12.082651+0000","last_clean":"2024-09-10T09:55:12.082651+0000","last_became_active":"2024-09-10T09:52:19.911268+0000","last_became_peered":"2024-09-10T09:52:19.911268+0000","last_unstale":"2024-09-10T09:55:12.082651+0000","last_undegraded":"2024-09-10T09:55:12.082651+0000","last_fullsized":"2024-09-10T09:55:12.082651+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:46:38.158784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":118,"num_read_kb":118,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.841796+0000","last_change":"2024-09-10T09:52:31.375938+0000","last_active":"2024-09-10T09:54:00.841796+0000","last_peered":"2024-09-10T09:54:00.841796+0000","last_clean":"2024-09-10T09:54:00.841796+0000","last_became_active":"2024-09-10T09:52:31.375536+0000","last_became_peered":"2024-09-10T09:52:31.375536+0000","last_unstale":"2024-09-10T09:54:00.841796+0000","last_undegraded":"2024-09-10T09:54:00.841796+0000","last_fullsized":"2024-09-10T09:54:00.841796+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:55:17.303809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"96'5","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.688934+0000","last_change":"2024-09-10T09:52:22.315912+0000","last_active":"2024-09-10T09:54:29.688934+0000","last_peered":"2024-09-10T09:54:29.688934+0000","last_clean":"2024-09-10T09:54:29.688934+0000","last_became_active":"2024-09-10T09:52:22.315433+0000","last_became_peered":"2024-09-10T09:52:22.315433+0000","last_unstale":"2024-09-10T09:54:29.688934+0000","last_undegraded":"2024-09-10T09:54:29.688934+0000","last_fullsized":"2024-09-10T09:54:29.688934+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:57:10.454716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"94'4","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.059138+0000","last_change":"2024-09-10T09:52:19.838966+0000","last_active":"2024-09-10T09:54:01.059138+0000","last_peered":"2024-09-10T09:54:01.059138+0000","last_clean":"2024-09-10T09:54:01.059138+0000","last_became_active":"2024-09-10T09:52:19.838545+0000","last_became_peered":"2024-09-10T09:52:19.838545+0000","last_unstale":"2024-09-10T09:54:01.059138+0000","last_undegraded":"2024-09-10T09:54:01.059138+0000","last_fullsized":"2024-09-10T09:54:01.059138+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:07:00.930146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.847601+0000","last_change":"2024-09-10T09:53:00.802039+0000","last_active":"2024-09-10T09:54:00.847601+0000","last_peered":"2024-09-10T09:54:00.847601+0000","last_clean":"2024-09-10T09:54:00.847601+0000","last_became_active":"2024-09-10T09:53:00.800223+0000","last_became_peered":"2024-09-10T09:53:00.800223+0000","last_unstale":"2024-09-10T09:54:00.847601+0000","last_undegraded":"2024-09-10T09:54:00.847601+0000","last_fullsized":"2024-09-10T09:54:00.847601+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:50:16.026631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,6,10],"acting":[5,8,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"96'8","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.698863+0000","last_change":"2024-09-10T09:52:22.313819+0000","last_active":"2024-09-10T09:54:29.698863+0000","last_peered":"2024-09-10T09:54:29.698863+0000","last_clean":"2024-09-10T09:54:29.698863+0000","last_became_active":"2024-09-10T09:52:22.313473+0000","last_became_peered":"2024-09-10T09:52:22.313473+0000","last_unstale":"2024-09-10T09:54:29.698863+0000","last_undegraded":"2024-09-10T09:54:29.698863+0000","last_fullsized":"2024-09-10T09:54:29.698863+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:44:10.003187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"94'6","reported_seq":51,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.951818+0000","last_change":"2024-09-10T09:52:19.837376+0000","last_active":"2024-09-10T09:54:00.951818+0000","last_peered":"2024-09-10T09:54:00.951818+0000","last_clean":"2024-09-10T09:54:00.951818+0000","last_became_active":"2024-09-10T09:52:19.836880+0000","last_became_peered":"2024-09-10T09:52:19.836880+0000","last_unstale":"2024-09-10T09:54:00.951818+0000","last_undegraded":"2024-09-10T09:54:00.951818+0000","last_fullsized":"2024-09-10T09:54:00.951818+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:48:58.198188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":361,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.059392+0000","last_change":"2024-09-10T09:51:05.871529+0000","last_active":"2024-09-10T09:54:01.059392+0000","last_peered":"2024-09-10T09:54:01.059392+0000","last_clean":"2024-09-10T09:54:01.059392+0000","last_became_active":"2024-09-10T09:51:05.871115+0000","last_became_peered":"2024-09-10T09:51:05.871115+0000","last_unstale":"2024-09-10T09:54:01.059392+0000","last_undegraded":"2024-09-10T09:54:01.059392+0000","last_fullsized":"2024-09-10T09:54:01.059392+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:47:24.175358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:47:24.175358+0000","last_clean_scrub_stamp":"2024-09-10T09:47:24.175358+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:58:30.247077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"94'1","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.059534+0000","last_change":"2024-09-10T09:53:00.800474+0000","last_active":"2024-09-10T09:54:01.059534+0000","last_peered":"2024-09-10T09:54:01.059534+0000","last_clean":"2024-09-10T09:54:01.059534+0000","last_became_active":"2024-09-10T09:53:00.799932+0000","last_became_peered":"2024-09-10T09:53:00.799932+0000","last_unstale":"2024-09-10T09:54:01.059534+0000","last_undegraded":"2024-09-10T09:54:01.059534+0000","last_fullsized":"2024-09-10T09:54:01.059534+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:14:50.998952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,11,9],"acting":[7,8,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.693609+0000","last_change":"2024-09-10T09:52:22.312521+0000","last_active":"2024-09-10T09:54:29.693609+0000","last_peered":"2024-09-10T09:54:29.693609+0000","last_clean":"2024-09-10T09:54:29.693609+0000","last_became_active":"2024-09-10T09:52:22.312107+0000","last_became_peered":"2024-09-10T09:52:22.312107+0000","last_unstale":"2024-09-10T09:54:29.693609+0000","last_undegraded":"2024-09-10T09:54:29.693609+0000","last_fullsized":"2024-09-10T09:54:29.693609+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:14:19.799259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"94'5","reported_seq":166,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:55:12.052387+0000","last_change":"2024-09-10T09:52:19.911454+0000","last_active":"2024-09-10T09:55:12.052387+0000","last_peered":"2024-09-10T09:55:12.052387+0000","last_clean":"2024-09-10T09:55:12.052387+0000","last_became_active":"2024-09-10T09:52:19.911079+0000","last_became_peered":"2024-09-10T09:52:19.911079+0000","last_unstale":"2024-09-10T09:55:12.052387+0000","last_undegraded":"2024-09-10T09:55:12.052387+0000","last_fullsized":"2024-09-10T09:55:12.052387+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:43:38.787689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":119,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.988531+0000","last_change":"2024-09-10T09:52:31.381877+0000","last_active":"2024-09-10T09:54:00.988531+0000","last_peered":"2024-09-10T09:54:00.988531+0000","last_clean":"2024-09-10T09:54:00.988531+0000","last_became_active":"2024-09-10T09:52:31.381472+0000","last_became_peered":"2024-09-10T09:52:31.381472+0000","last_unstale":"2024-09-10T09:54:00.988531+0000","last_undegraded":"2024-09-10T09:54:00.988531+0000","last_fullsized":"2024-09-10T09:54:00.988531+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:41:24.686325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"96'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.702544+0000","last_change":"2024-09-10T09:52:22.313902+0000","last_active":"2024-09-10T09:54:29.702544+0000","last_peered":"2024-09-10T09:54:29.702544+0000","last_clean":"2024-09-10T09:54:29.702544+0000","last_became_active":"2024-09-10T09:52:22.313515+0000","last_became_peered":"2024-09-10T09:52:22.313515+0000","last_unstale":"2024-09-10T09:54:29.702544+0000","last_undegraded":"2024-09-10T09:54:29.702544+0000","last_fullsized":"2024-09-10T09:54:29.702544+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:25:00.238307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"94'3","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.847873+0000","last_change":"2024-09-10T09:52:19.837736+0000","last_active":"2024-09-10T09:54:00.847873+0000","last_peered":"2024-09-10T09:54:00.847873+0000","last_clean":"2024-09-10T09:54:00.847873+0000","last_became_active":"2024-09-10T09:52:19.837370+0000","last_became_peered":"2024-09-10T09:52:19.837370+0000","last_unstale":"2024-09-10T09:54:00.847873+0000","last_undegraded":"2024-09-10T09:54:00.847873+0000","last_fullsized":"2024-09-10T09:54:00.847873+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:54:35.753027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.248213+0000","last_change":"2024-09-10T09:52:31.378945+0000","last_active":"2024-09-10T09:54:01.248213+0000","last_peered":"2024-09-10T09:54:01.248213+0000","last_clean":"2024-09-10T09:54:01.248213+0000","last_became_active":"2024-09-10T09:52:31.378144+0000","last_became_peered":"2024-09-10T09:52:31.378144+0000","last_unstale":"2024-09-10T09:54:01.248213+0000","last_undegraded":"2024-09-10T09:54:01.248213+0000","last_fullsized":"2024-09-10T09:54:01.248213+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:36:06.288293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"96'9","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.692039+0000","last_change":"2024-09-10T09:52:22.316400+0000","last_active":"2024-09-10T09:54:29.692039+0000","last_peered":"2024-09-10T09:54:29.692039+0000","last_clean":"2024-09-10T09:54:29.692039+0000","last_became_active":"2024-09-10T09:52:22.315974+0000","last_became_peered":"2024-09-10T09:52:22.315974+0000","last_unstale":"2024-09-10T09:54:29.692039+0000","last_undegraded":"2024-09-10T09:54:29.692039+0000","last_fullsized":"2024-09-10T09:54:29.692039+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:31:05.538237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"94'5","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.841976+0000","last_change":"2024-09-10T09:52:19.837942+0000","last_active":"2024-09-10T09:54:00.841976+0000","last_peered":"2024-09-10T09:54:00.841976+0000","last_clean":"2024-09-10T09:54:00.841976+0000","last_became_active":"2024-09-10T09:52:19.837629+0000","last_became_peered":"2024-09-10T09:52:19.837629+0000","last_unstale":"2024-09-10T09:54:00.841976+0000","last_undegraded":"2024-09-10T09:54:00.841976+0000","last_fullsized":"2024-09-10T09:54:00.841976+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:34:13.207770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.847019+0000","last_change":"2024-09-10T09:52:31.364231+0000","last_active":"2024-09-10T09:54:00.847019+0000","last_peered":"2024-09-10T09:54:00.847019+0000","last_clean":"2024-09-10T09:54:00.847019+0000","last_became_active":"2024-09-10T09:52:31.363833+0000","last_became_peered":"2024-09-10T09:52:31.363833+0000","last_unstale":"2024-09-10T09:54:00.847019+0000","last_undegraded":"2024-09-10T09:54:00.847019+0000","last_fullsized":"2024-09-10T09:54:00.847019+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:53:51.435700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"96'5","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.680562+0000","last_change":"2024-09-10T09:52:22.319530+0000","last_active":"2024-09-10T09:54:29.680562+0000","last_peered":"2024-09-10T09:54:29.680562+0000","last_clean":"2024-09-10T09:54:29.680562+0000","last_became_active":"2024-09-10T09:52:22.318742+0000","last_became_peered":"2024-09-10T09:52:22.318742+0000","last_unstale":"2024-09-10T09:54:29.680562+0000","last_undegraded":"2024-09-10T09:54:29.680562+0000","last_fullsized":"2024-09-10T09:54:29.680562+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:44:35.050454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"94'2","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.150048+0000","last_change":"2024-09-10T09:52:19.836398+0000","last_active":"2024-09-10T09:54:01.150048+0000","last_peered":"2024-09-10T09:54:01.150048+0000","last_clean":"2024-09-10T09:54:01.150048+0000","last_became_active":"2024-09-10T09:52:19.836055+0000","last_became_peered":"2024-09-10T09:52:19.836055+0000","last_unstale":"2024-09-10T09:54:01.150048+0000","last_undegraded":"2024-09-10T09:54:01.150048+0000","last_fullsized":"2024-09-10T09:54:01.150048+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:49:26.923367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.847215+0000","last_change":"2024-09-10T09:52:31.371109+0000","last_active":"2024-09-10T09:54:00.847215+0000","last_peered":"2024-09-10T09:54:00.847215+0000","last_clean":"2024-09-10T09:54:00.847215+0000","last_became_active":"2024-09-10T09:52:31.370592+0000","last_became_peered":"2024-09-10T09:52:31.370592+0000","last_unstale":"2024-09-10T09:54:00.847215+0000","last_undegraded":"2024-09-10T09:54:00.847215+0000","last_fullsized":"2024-09-10T09:54:00.847215+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:20:23.585187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"96'6","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.685236+0000","last_change":"2024-09-10T09:52:22.319365+0000","last_active":"2024-09-10T09:54:29.685236+0000","last_peered":"2024-09-10T09:54:29.685236+0000","last_clean":"2024-09-10T09:54:29.685236+0000","last_became_active":"2024-09-10T09:52:22.319064+0000","last_became_peered":"2024-09-10T09:52:22.319064+0000","last_unstale":"2024-09-10T09:54:29.685236+0000","last_undegraded":"2024-09-10T09:54:29.685236+0000","last_fullsized":"2024-09-10T09:54:29.685236+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:14:50.089566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"94'17","reported_seq":59,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.149324+0000","last_change":"2024-09-10T09:52:19.911698+0000","last_active":"2024-09-10T09:54:01.149324+0000","last_peered":"2024-09-10T09:54:01.149324+0000","last_clean":"2024-09-10T09:54:01.149324+0000","last_became_active":"2024-09-10T09:52:19.911369+0000","last_became_peered":"2024-09-10T09:52:19.911369+0000","last_unstale":"2024-09-10T09:54:01.149324+0000","last_undegraded":"2024-09-10T09:54:01.149324+0000","last_fullsized":"2024-09-10T09:54:01.149324+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:40:39.045797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.841947+0000","last_change":"2024-09-10T09:52:31.380463+0000","last_active":"2024-09-10T09:54:00.841947+0000","last_peered":"2024-09-10T09:54:00.841947+0000","last_clean":"2024-09-10T09:54:00.841947+0000","last_became_active":"2024-09-10T09:52:31.380114+0000","last_became_peered":"2024-09-10T09:52:31.380114+0000","last_unstale":"2024-09-10T09:54:00.841947+0000","last_undegraded":"2024-09-10T09:54:00.841947+0000","last_fullsized":"2024-09-10T09:54:00.841947+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:19:50.869160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"96'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.682831+0000","last_change":"2024-09-10T09:52:22.313928+0000","last_active":"2024-09-10T09:54:29.682831+0000","last_peered":"2024-09-10T09:54:29.682831+0000","last_clean":"2024-09-10T09:54:29.682831+0000","last_became_active":"2024-09-10T09:52:22.313510+0000","last_became_peered":"2024-09-10T09:52:22.313510+0000","last_unstale":"2024-09-10T09:54:29.682831+0000","last_undegraded":"2024-09-10T09:54:29.682831+0000","last_fullsized":"2024-09-10T09:54:29.682831+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:58:19.031178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"96'7","reported_seq":170,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:55:12.083520+0000","last_change":"2024-09-10T09:52:19.839451+0000","last_active":"2024-09-10T09:55:12.083520+0000","last_peered":"2024-09-10T09:55:12.083520+0000","last_clean":"2024-09-10T09:55:12.083520+0000","last_became_active":"2024-09-10T09:52:19.839124+0000","last_became_peered":"2024-09-10T09:52:19.839124+0000","last_unstale":"2024-09-10T09:55:12.083520+0000","last_undegraded":"2024-09-10T09:55:12.083520+0000","last_fullsized":"2024-09-10T09:55:12.083520+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:51:33.489830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":122,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.844344+0000","last_change":"2024-09-10T09:52:31.376386+0000","last_active":"2024-09-10T09:54:00.844344+0000","last_peered":"2024-09-10T09:54:00.844344+0000","last_clean":"2024-09-10T09:54:00.844344+0000","last_became_active":"2024-09-10T09:52:31.376066+0000","last_became_peered":"2024-09-10T09:52:31.376066+0000","last_unstale":"2024-09-10T09:54:00.844344+0000","last_undegraded":"2024-09-10T09:54:00.844344+0000","last_fullsized":"2024-09-10T09:54:00.844344+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:23:07.191842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"96'15","reported_seq":52,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.702901+0000","last_change":"2024-09-10T09:52:22.323590+0000","last_active":"2024-09-10T09:54:29.702901+0000","last_peered":"2024-09-10T09:54:29.702901+0000","last_clean":"2024-09-10T09:54:29.702901+0000","last_became_active":"2024-09-10T09:52:22.323264+0000","last_became_peered":"2024-09-10T09:52:22.323264+0000","last_unstale":"2024-09-10T09:54:29.702901+0000","last_undegraded":"2024-09-10T09:54:29.702901+0000","last_fullsized":"2024-09-10T09:54:29.702901+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:07:56.572337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"94'5","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.847940+0000","last_change":"2024-09-10T09:52:19.840620+0000","last_active":"2024-09-10T09:54:00.847940+0000","last_peered":"2024-09-10T09:54:00.847940+0000","last_clean":"2024-09-10T09:54:00.847940+0000","last_became_active":"2024-09-10T09:52:19.840217+0000","last_became_peered":"2024-09-10T09:52:19.840217+0000","last_unstale":"2024-09-10T09:54:00.847940+0000","last_undegraded":"2024-09-10T09:54:00.847940+0000","last_fullsized":"2024-09-10T09:54:00.847940+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:55:13.111993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.955071+0000","last_change":"2024-09-10T09:52:31.373895+0000","last_active":"2024-09-10T09:54:00.955071+0000","last_peered":"2024-09-10T09:54:00.955071+0000","last_clean":"2024-09-10T09:54:00.955071+0000","last_became_active":"2024-09-10T09:52:31.373474+0000","last_became_peered":"2024-09-10T09:52:31.373474+0000","last_unstale":"2024-09-10T09:54:00.955071+0000","last_undegraded":"2024-09-10T09:54:00.955071+0000","last_fullsized":"2024-09-10T09:54:00.955071+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:39:16.049859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"96'9","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.704377+0000","last_change":"2024-09-10T09:52:22.319294+0000","last_active":"2024-09-10T09:54:29.704377+0000","last_peered":"2024-09-10T09:54:29.704377+0000","last_clean":"2024-09-10T09:54:29.704377+0000","last_became_active":"2024-09-10T09:52:22.318918+0000","last_became_peered":"2024-09-10T09:52:22.318918+0000","last_unstale":"2024-09-10T09:54:29.704377+0000","last_undegraded":"2024-09-10T09:54:29.704377+0000","last_fullsized":"2024-09-10T09:54:29.704377+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:52:46.434182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"94'9","reported_seq":53,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.844244+0000","last_change":"2024-09-10T09:52:19.842286+0000","last_active":"2024-09-10T09:54:00.844244+0000","last_peered":"2024-09-10T09:54:00.844244+0000","last_clean":"2024-09-10T09:54:00.844244+0000","last_became_active":"2024-09-10T09:52:19.841930+0000","last_became_peered":"2024-09-10T09:52:19.841930+0000","last_unstale":"2024-09-10T09:54:00.844244+0000","last_undegraded":"2024-09-10T09:54:00.844244+0000","last_fullsized":"2024-09-10T09:54:00.844244+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:38:46.358389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.847531+0000","last_change":"2024-09-10T09:52:31.370961+0000","last_active":"2024-09-10T09:54:00.847531+0000","last_peered":"2024-09-10T09:54:00.847531+0000","last_clean":"2024-09-10T09:54:00.847531+0000","last_became_active":"2024-09-10T09:52:31.370545+0000","last_became_peered":"2024-09-10T09:52:31.370545+0000","last_unstale":"2024-09-10T09:54:00.847531+0000","last_undegraded":"2024-09-10T09:54:00.847531+0000","last_fullsized":"2024-09-10T09:54:00.847531+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:17.461447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"96'6","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.685895+0000","last_change":"2024-09-10T09:52:22.313215+0000","last_active":"2024-09-10T09:54:29.685895+0000","last_peered":"2024-09-10T09:54:29.685895+0000","last_clean":"2024-09-10T09:54:29.685895+0000","last_became_active":"2024-09-10T09:52:22.312918+0000","last_became_peered":"2024-09-10T09:52:22.312918+0000","last_unstale":"2024-09-10T09:54:29.685895+0000","last_undegraded":"2024-09-10T09:54:29.685895+0000","last_fullsized":"2024-09-10T09:54:29.685895+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:48:09.568004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"96'37","reported_seq":337,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:55:12.051648+0000","last_change":"2024-09-10T09:53:01.226146+0000","last_active":"2024-09-10T09:55:12.051648+0000","last_peered":"2024-09-10T09:55:12.051648+0000","last_clean":"2024-09-10T09:55:12.051648+0000","last_became_active":"2024-09-10T09:53:01.218622+0000","last_became_peered":"2024-09-10T09:53:01.218622+0000","last_unstale":"2024-09-10T09:55:12.051648+0000","last_undegraded":"2024-09-10T09:55:12.051648+0000","last_fullsized":"2024-09-10T09:55:12.051648+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:43:11.927029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63251,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":241,"num_read_kb":182,"num_write":39,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":3844,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":16,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.841392+0000","last_change":"2024-09-10T09:53:00.800650+0000","last_active":"2024-09-10T09:54:00.841392+0000","last_peered":"2024-09-10T09:54:00.841392+0000","last_clean":"2024-09-10T09:54:00.841392+0000","last_became_active":"2024-09-10T09:53:00.800104+0000","last_became_peered":"2024-09-10T09:53:00.800104+0000","last_unstale":"2024-09-10T09:54:00.841392+0000","last_undegraded":"2024-09-10T09:54:00.841392+0000","last_fullsized":"2024-09-10T09:54:00.841392+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:41:16.022884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.14","version":"96'11","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.690121+0000","last_change":"2024-09-10T09:52:22.317174+0000","last_active":"2024-09-10T09:54:29.690121+0000","last_peered":"2024-09-10T09:54:29.690121+0000","last_clean":"2024-09-10T09:54:29.690121+0000","last_became_active":"2024-09-10T09:52:22.316851+0000","last_became_peered":"2024-09-10T09:52:22.316851+0000","last_unstale":"2024-09-10T09:54:29.690121+0000","last_undegraded":"2024-09-10T09:54:29.690121+0000","last_fullsized":"2024-09-10T09:54:29.690121+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:55:18.742393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"94'5","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.847731+0000","last_change":"2024-09-10T09:52:19.912093+0000","last_active":"2024-09-10T09:54:00.847731+0000","last_peered":"2024-09-10T09:54:00.847731+0000","last_clean":"2024-09-10T09:54:00.847731+0000","last_became_active":"2024-09-10T09:52:19.911611+0000","last_became_peered":"2024-09-10T09:52:19.911611+0000","last_unstale":"2024-09-10T09:54:00.847731+0000","last_undegraded":"2024-09-10T09:54:00.847731+0000","last_fullsized":"2024-09-10T09:54:00.847731+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:01:10.859310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.951658+0000","last_change":"2024-09-10T09:52:31.362884+0000","last_active":"2024-09-10T09:54:00.951658+0000","last_peered":"2024-09-10T09:54:00.951658+0000","last_clean":"2024-09-10T09:54:00.951658+0000","last_became_active":"2024-09-10T09:52:31.362489+0000","last_became_peered":"2024-09-10T09:52:31.362489+0000","last_unstale":"2024-09-10T09:54:00.951658+0000","last_undegraded":"2024-09-10T09:54:00.951658+0000","last_fullsized":"2024-09-10T09:54:00.951658+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:02:33.084795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"96'9","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.692830+0000","last_change":"2024-09-10T09:52:22.325871+0000","last_active":"2024-09-10T09:54:29.692830+0000","last_peered":"2024-09-10T09:54:29.692830+0000","last_clean":"2024-09-10T09:54:29.692830+0000","last_became_active":"2024-09-10T09:52:22.324980+0000","last_became_peered":"2024-09-10T09:52:22.324980+0000","last_unstale":"2024-09-10T09:54:29.692830+0000","last_undegraded":"2024-09-10T09:54:29.692830+0000","last_fullsized":"2024-09-10T09:54:29.692830+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:31:55.713444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"94'4","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.059691+0000","last_change":"2024-09-10T09:52:19.839491+0000","last_active":"2024-09-10T09:54:01.059691+0000","last_peered":"2024-09-10T09:54:01.059691+0000","last_clean":"2024-09-10T09:54:01.059691+0000","last_became_active":"2024-09-10T09:52:19.839092+0000","last_became_peered":"2024-09-10T09:52:19.839092+0000","last_unstale":"2024-09-10T09:54:01.059691+0000","last_undegraded":"2024-09-10T09:54:01.059691+0000","last_fullsized":"2024-09-10T09:54:01.059691+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:42:13.453281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.843285+0000","last_change":"2024-09-10T09:52:31.371346+0000","last_active":"2024-09-10T09:54:00.843285+0000","last_peered":"2024-09-10T09:54:00.843285+0000","last_clean":"2024-09-10T09:54:00.843285+0000","last_became_active":"2024-09-10T09:52:31.370929+0000","last_became_peered":"2024-09-10T09:52:31.370929+0000","last_unstale":"2024-09-10T09:54:00.843285+0000","last_undegraded":"2024-09-10T09:54:00.843285+0000","last_fullsized":"2024-09-10T09:54:00.843285+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:44:22.925082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"96'10","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.694062+0000","last_change":"2024-09-10T09:52:22.317453+0000","last_active":"2024-09-10T09:54:29.694062+0000","last_peered":"2024-09-10T09:54:29.694062+0000","last_clean":"2024-09-10T09:54:29.694062+0000","last_became_active":"2024-09-10T09:52:22.315556+0000","last_became_peered":"2024-09-10T09:52:22.315556+0000","last_unstale":"2024-09-10T09:54:29.694062+0000","last_undegraded":"2024-09-10T09:54:29.694062+0000","last_fullsized":"2024-09-10T09:54:29.694062+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:56:05.860275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"96'5","reported_seq":280,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:55:12.051699+0000","last_change":"2024-09-10T09:52:19.835985+0000","last_active":"2024-09-10T09:55:12.051699+0000","last_peered":"2024-09-10T09:55:12.051699+0000","last_clean":"2024-09-10T09:55:12.051699+0000","last_became_active":"2024-09-10T09:52:19.834835+0000","last_became_peered":"2024-09-10T09:52:19.834835+0000","last_unstale":"2024-09-10T09:55:12.051699+0000","last_undegraded":"2024-09-10T09:55:12.051699+0000","last_fullsized":"2024-09-10T09:55:12.051699+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:09:11.590443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":232,"num_read_kb":232,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.845724+0000","last_change":"2024-09-10T09:52:31.368663+0000","last_active":"2024-09-10T09:54:00.845724+0000","last_peered":"2024-09-10T09:54:00.845724+0000","last_clean":"2024-09-10T09:54:00.845724+0000","last_became_active":"2024-09-10T09:52:31.368004+0000","last_became_peered":"2024-09-10T09:52:31.368004+0000","last_unstale":"2024-09-10T09:54:00.845724+0000","last_undegraded":"2024-09-10T09:54:00.845724+0000","last_fullsized":"2024-09-10T09:54:00.845724+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:56:15.100515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.247917+0000","last_change":"2024-09-10T09:52:31.370992+0000","last_active":"2024-09-10T09:54:01.247917+0000","last_peered":"2024-09-10T09:54:01.247917+0000","last_clean":"2024-09-10T09:54:01.247917+0000","last_became_active":"2024-09-10T09:52:31.370357+0000","last_became_peered":"2024-09-10T09:52:31.370357+0000","last_unstale":"2024-09-10T09:54:01.247917+0000","last_undegraded":"2024-09-10T09:54:01.247917+0000","last_fullsized":"2024-09-10T09:54:01.247917+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:02:49.482532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"96'11","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.706426+0000","last_change":"2024-09-10T09:52:22.319363+0000","last_active":"2024-09-10T09:54:29.706426+0000","last_peered":"2024-09-10T09:54:29.706426+0000","last_clean":"2024-09-10T09:54:29.706426+0000","last_became_active":"2024-09-10T09:52:22.319044+0000","last_became_peered":"2024-09-10T09:52:22.319044+0000","last_unstale":"2024-09-10T09:54:29.706426+0000","last_undegraded":"2024-09-10T09:54:29.706426+0000","last_fullsized":"2024-09-10T09:54:29.706426+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:30:51.328413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"94'3","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.847831+0000","last_change":"2024-09-10T09:52:19.841745+0000","last_active":"2024-09-10T09:54:00.847831+0000","last_peered":"2024-09-10T09:54:00.847831+0000","last_clean":"2024-09-10T09:54:00.847831+0000","last_became_active":"2024-09-10T09:52:19.841404+0000","last_became_peered":"2024-09-10T09:52:19.841404+0000","last_unstale":"2024-09-10T09:54:00.847831+0000","last_undegraded":"2024-09-10T09:54:00.847831+0000","last_fullsized":"2024-09-10T09:54:00.847831+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:44:36.346785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.955521+0000","last_change":"2024-09-10T09:53:01.218412+0000","last_active":"2024-09-10T09:54:00.955521+0000","last_peered":"2024-09-10T09:54:00.955521+0000","last_clean":"2024-09-10T09:54:00.955521+0000","last_became_active":"2024-09-10T09:53:01.217966+0000","last_became_peered":"2024-09-10T09:53:01.217966+0000","last_unstale":"2024-09-10T09:54:00.955521+0000","last_undegraded":"2024-09-10T09:54:00.955521+0000","last_fullsized":"2024-09-10T09:54:00.955521+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:53:25.743913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"96'8","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.703441+0000","last_change":"2024-09-10T09:52:22.316120+0000","last_active":"2024-09-10T09:54:29.703441+0000","last_peered":"2024-09-10T09:54:29.703441+0000","last_clean":"2024-09-10T09:54:29.703441+0000","last_became_active":"2024-09-10T09:52:22.315769+0000","last_became_peered":"2024-09-10T09:52:22.315769+0000","last_unstale":"2024-09-10T09:54:29.703441+0000","last_undegraded":"2024-09-10T09:54:29.703441+0000","last_fullsized":"2024-09-10T09:54:29.703441+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:16:41.866587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"94'3","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.841046+0000","last_change":"2024-09-10T09:52:19.835038+0000","last_active":"2024-09-10T09:54:00.841046+0000","last_peered":"2024-09-10T09:54:00.841046+0000","last_clean":"2024-09-10T09:54:00.841046+0000","last_became_active":"2024-09-10T09:52:19.834329+0000","last_became_peered":"2024-09-10T09:52:19.834329+0000","last_unstale":"2024-09-10T09:54:00.841046+0000","last_undegraded":"2024-09-10T09:54:00.841046+0000","last_fullsized":"2024-09-10T09:54:00.841046+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:58:24.084373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"94'7","reported_seq":56,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.841764+0000","last_change":"2024-09-10T09:53:01.219122+0000","last_active":"2024-09-10T09:54:00.841764+0000","last_peered":"2024-09-10T09:54:00.841764+0000","last_clean":"2024-09-10T09:54:00.841764+0000","last_became_active":"2024-09-10T09:53:01.218599+0000","last_became_peered":"2024-09-10T09:53:01.218599+0000","last_unstale":"2024-09-10T09:54:00.841764+0000","last_undegraded":"2024-09-10T09:54:00.841764+0000","last_fullsized":"2024-09-10T09:54:00.841764+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:31:10.092264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"96'8","reported_seq":19,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.706393+0000","last_change":"2024-09-10T09:54:00.952617+0000","last_active":"2024-09-10T09:54:29.706393+0000","last_peered":"2024-09-10T09:54:29.706393+0000","last_clean":"2024-09-10T09:54:29.706393+0000","last_became_active":"2024-09-10T09:54:00.952202+0000","last_became_peered":"2024-09-10T09:54:00.952202+0000","last_unstale":"2024-09-10T09:54:29.706393+0000","last_undegraded":"2024-09-10T09:54:29.706393+0000","last_fullsized":"2024-09-10T09:54:29.706393+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:35:40.179591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.844169+0000","last_change":"2024-09-10T09:52:31.371612+0000","last_active":"2024-09-10T09:54:00.844169+0000","last_peered":"2024-09-10T09:54:00.844169+0000","last_clean":"2024-09-10T09:54:00.844169+0000","last_became_active":"2024-09-10T09:52:31.371163+0000","last_became_peered":"2024-09-10T09:52:31.371163+0000","last_unstale":"2024-09-10T09:54:00.844169+0000","last_undegraded":"2024-09-10T09:54:00.844169+0000","last_fullsized":"2024-09-10T09:54:00.844169+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:11:43.681524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"96'7","reported_seq":171,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:55:12.052584+0000","last_change":"2024-09-10T09:52:19.841873+0000","last_active":"2024-09-10T09:55:12.052584+0000","last_peered":"2024-09-10T09:55:12.052584+0000","last_clean":"2024-09-10T09:55:12.052584+0000","last_became_active":"2024-09-10T09:52:19.841513+0000","last_became_peered":"2024-09-10T09:52:19.841513+0000","last_unstale":"2024-09-10T09:55:12.052584+0000","last_undegraded":"2024-09-10T09:55:12.052584+0000","last_fullsized":"2024-09-10T09:55:12.052584+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:53:31.854185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1156,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":121,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"96'7","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.697738+0000","last_change":"2024-09-10T09:52:22.314781+0000","last_active":"2024-09-10T09:54:29.697738+0000","last_peered":"2024-09-10T09:54:29.697738+0000","last_clean":"2024-09-10T09:54:29.697738+0000","last_became_active":"2024-09-10T09:52:22.313346+0000","last_became_peered":"2024-09-10T09:52:22.313346+0000","last_unstale":"2024-09-10T09:54:29.697738+0000","last_undegraded":"2024-09-10T09:54:29.697738+0000","last_fullsized":"2024-09-10T09:54:29.697738+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:15:16.921963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.844401+0000","last_change":"2024-09-10T09:52:31.365544+0000","last_active":"2024-09-10T09:54:00.844401+0000","last_peered":"2024-09-10T09:54:00.844401+0000","last_clean":"2024-09-10T09:54:00.844401+0000","last_became_active":"2024-09-10T09:52:31.365151+0000","last_became_peered":"2024-09-10T09:52:31.365151+0000","last_unstale":"2024-09-10T09:54:00.844401+0000","last_undegraded":"2024-09-10T09:54:00.844401+0000","last_fullsized":"2024-09-10T09:54:00.844401+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:36:55.247890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"96'9","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.683641+0000","last_change":"2024-09-10T09:52:22.314117+0000","last_active":"2024-09-10T09:54:29.683641+0000","last_peered":"2024-09-10T09:54:29.683641+0000","last_clean":"2024-09-10T09:54:29.683641+0000","last_became_active":"2024-09-10T09:52:22.313772+0000","last_became_peered":"2024-09-10T09:52:22.313772+0000","last_unstale":"2024-09-10T09:54:29.683641+0000","last_undegraded":"2024-09-10T09:54:29.683641+0000","last_fullsized":"2024-09-10T09:54:29.683641+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:19:37.373598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"96'12","reported_seq":50,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.682477+0000","last_change":"2024-09-10T09:52:22.314362+0000","last_active":"2024-09-10T09:54:29.682477+0000","last_peered":"2024-09-10T09:54:29.682477+0000","last_clean":"2024-09-10T09:54:29.682477+0000","last_became_active":"2024-09-10T09:52:22.314050+0000","last_became_peered":"2024-09-10T09:52:22.314050+0000","last_unstale":"2024-09-10T09:54:29.682477+0000","last_undegraded":"2024-09-10T09:54:29.682477+0000","last_fullsized":"2024-09-10T09:54:29.682477+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:37:07.218656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"96'8","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.699844+0000","last_change":"2024-09-10T09:52:22.322363+0000","last_active":"2024-09-10T09:54:29.699844+0000","last_peered":"2024-09-10T09:54:29.699844+0000","last_clean":"2024-09-10T09:54:29.699844+0000","last_became_active":"2024-09-10T09:52:22.322045+0000","last_became_peered":"2024-09-10T09:52:22.322045+0000","last_unstale":"2024-09-10T09:54:29.699844+0000","last_undegraded":"2024-09-10T09:54:29.699844+0000","last_fullsized":"2024-09-10T09:54:29.699844+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:10:27.918769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.700329+0000","last_change":"2024-09-10T09:52:22.331594+0000","last_active":"2024-09-10T09:54:29.700329+0000","last_peered":"2024-09-10T09:54:29.700329+0000","last_clean":"2024-09-10T09:54:29.700329+0000","last_became_active":"2024-09-10T09:52:22.330585+0000","last_became_peered":"2024-09-10T09:52:22.330585+0000","last_unstale":"2024-09-10T09:54:29.700329+0000","last_undegraded":"2024-09-10T09:54:29.700329+0000","last_fullsized":"2024-09-10T09:54:29.700329+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:45:56.412313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"96'4","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.705719+0000","last_change":"2024-09-10T09:52:22.325753+0000","last_active":"2024-09-10T09:54:29.705719+0000","last_peered":"2024-09-10T09:54:29.705719+0000","last_clean":"2024-09-10T09:54:29.705719+0000","last_became_active":"2024-09-10T09:52:22.325385+0000","last_became_peered":"2024-09-10T09:52:22.325385+0000","last_unstale":"2024-09-10T09:54:29.705719+0000","last_undegraded":"2024-09-10T09:54:29.705719+0000","last_fullsized":"2024-09-10T09:54:29.705719+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:18:28.776789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"96'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.681378+0000","last_change":"2024-09-10T09:52:22.319069+0000","last_active":"2024-09-10T09:54:29.681378+0000","last_peered":"2024-09-10T09:54:29.681378+0000","last_clean":"2024-09-10T09:54:29.681378+0000","last_became_active":"2024-09-10T09:52:22.318667+0000","last_became_peered":"2024-09-10T09:52:22.318667+0000","last_unstale":"2024-09-10T09:54:29.681378+0000","last_undegraded":"2024-09-10T09:54:29.681378+0000","last_fullsized":"2024-09-10T09:54:29.681378+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:21:32.028235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"96'15","reported_seq":53,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.681487+0000","last_change":"2024-09-10T09:52:22.312890+0000","last_active":"2024-09-10T09:54:29.681487+0000","last_peered":"2024-09-10T09:54:29.681487+0000","last_clean":"2024-09-10T09:54:29.681487+0000","last_became_active":"2024-09-10T09:52:22.312608+0000","last_became_peered":"2024-09-10T09:52:22.312608+0000","last_unstale":"2024-09-10T09:54:29.681487+0000","last_undegraded":"2024-09-10T09:54:29.681487+0000","last_fullsized":"2024-09-10T09:54:29.681487+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:07:46.046915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"96'2","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.689884+0000","last_change":"2024-09-10T09:54:01.151385+0000","last_active":"2024-09-10T09:54:29.689884+0000","last_peered":"2024-09-10T09:54:29.689884+0000","last_clean":"2024-09-10T09:54:29.689884+0000","last_became_active":"2024-09-10T09:54:01.150971+0000","last_became_peered":"2024-09-10T09:54:01.150971+0000","last_unstale":"2024-09-10T09:54:29.689884+0000","last_undegraded":"2024-09-10T09:54:29.689884+0000","last_fullsized":"2024-09-10T09:54:29.689884+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:27:10.492071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"96'6","reported_seq":17,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.704565+0000","last_change":"2024-09-10T09:54:00.848273+0000","last_active":"2024-09-10T09:54:29.704565+0000","last_peered":"2024-09-10T09:54:29.704565+0000","last_clean":"2024-09-10T09:54:29.704565+0000","last_became_active":"2024-09-10T09:54:00.847492+0000","last_became_peered":"2024-09-10T09:54:00.847492+0000","last_unstale":"2024-09-10T09:54:29.704565+0000","last_undegraded":"2024-09-10T09:54:29.704565+0000","last_fullsized":"2024-09-10T09:54:29.704565+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:02:24.749291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,3],"acting":[6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.29","version":"96'8","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.696836+0000","last_change":"2024-09-10T09:52:22.313538+0000","last_active":"2024-09-10T09:54:29.696836+0000","last_peered":"2024-09-10T09:54:29.696836+0000","last_clean":"2024-09-10T09:54:29.696836+0000","last_became_active":"2024-09-10T09:52:22.313196+0000","last_became_peered":"2024-09-10T09:52:22.313196+0000","last_unstale":"2024-09-10T09:54:29.696836+0000","last_undegraded":"2024-09-10T09:54:29.696836+0000","last_fullsized":"2024-09-10T09:54:29.696836+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:53:23.387167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"96'10","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.696632+0000","last_change":"2024-09-10T09:52:22.316518+0000","last_active":"2024-09-10T09:54:29.696632+0000","last_peered":"2024-09-10T09:54:29.696632+0000","last_clean":"2024-09-10T09:54:29.696632+0000","last_became_active":"2024-09-10T09:52:22.316193+0000","last_became_peered":"2024-09-10T09:52:22.316193+0000","last_unstale":"2024-09-10T09:54:29.696632+0000","last_undegraded":"2024-09-10T09:54:29.696632+0000","last_fullsized":"2024-09-10T09:54:29.696632+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:49:55.903928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"96'11","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.706245+0000","last_change":"2024-09-10T09:52:22.329163+0000","last_active":"2024-09-10T09:54:29.706245+0000","last_peered":"2024-09-10T09:54:29.706245+0000","last_clean":"2024-09-10T09:54:29.706245+0000","last_became_active":"2024-09-10T09:52:22.328453+0000","last_became_peered":"2024-09-10T09:52:22.328453+0000","last_unstale":"2024-09-10T09:54:29.706245+0000","last_undegraded":"2024-09-10T09:54:29.706245+0000","last_fullsized":"2024-09-10T09:54:29.706245+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:53:33.286055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"96'8","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.681766+0000","last_change":"2024-09-10T09:52:22.314302+0000","last_active":"2024-09-10T09:54:29.681766+0000","last_peered":"2024-09-10T09:54:29.681766+0000","last_clean":"2024-09-10T09:54:29.681766+0000","last_became_active":"2024-09-10T09:52:22.313992+0000","last_became_peered":"2024-09-10T09:52:22.313992+0000","last_unstale":"2024-09-10T09:54:29.681766+0000","last_undegraded":"2024-09-10T09:54:29.681766+0000","last_fullsized":"2024-09-10T09:54:29.681766+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:07:35.764015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"96'7","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.688705+0000","last_change":"2024-09-10T09:52:22.316400+0000","last_active":"2024-09-10T09:54:29.688705+0000","last_peered":"2024-09-10T09:54:29.688705+0000","last_clean":"2024-09-10T09:54:29.688705+0000","last_became_active":"2024-09-10T09:52:22.316110+0000","last_became_peered":"2024-09-10T09:52:22.316110+0000","last_unstale":"2024-09-10T09:54:29.688705+0000","last_undegraded":"2024-09-10T09:54:29.688705+0000","last_fullsized":"2024-09-10T09:54:29.688705+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:17:26.528899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"96'10","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.704074+0000","last_change":"2024-09-10T09:52:22.317914+0000","last_active":"2024-09-10T09:54:29.704074+0000","last_peered":"2024-09-10T09:54:29.704074+0000","last_clean":"2024-09-10T09:54:29.704074+0000","last_became_active":"2024-09-10T09:52:22.317575+0000","last_became_peered":"2024-09-10T09:52:22.317575+0000","last_unstale":"2024-09-10T09:54:29.704074+0000","last_undegraded":"2024-09-10T09:54:29.704074+0000","last_fullsized":"2024-09-10T09:54:29.704074+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:17:51.446233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"96'8","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.697848+0000","last_change":"2024-09-10T09:52:22.327749+0000","last_active":"2024-09-10T09:54:29.697848+0000","last_peered":"2024-09-10T09:54:29.697848+0000","last_clean":"2024-09-10T09:54:29.697848+0000","last_became_active":"2024-09-10T09:52:22.327461+0000","last_became_peered":"2024-09-10T09:52:22.327461+0000","last_unstale":"2024-09-10T09:54:29.697848+0000","last_undegraded":"2024-09-10T09:54:29.697848+0000","last_fullsized":"2024-09-10T09:54:29.697848+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:54.208064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"96'9","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.696695+0000","last_change":"2024-09-10T09:52:22.326982+0000","last_active":"2024-09-10T09:54:29.696695+0000","last_peered":"2024-09-10T09:54:29.696695+0000","last_clean":"2024-09-10T09:54:29.696695+0000","last_became_active":"2024-09-10T09:52:22.326635+0000","last_became_peered":"2024-09-10T09:52:22.326635+0000","last_unstale":"2024-09-10T09:54:29.696695+0000","last_undegraded":"2024-09-10T09:54:29.696695+0000","last_fullsized":"2024-09-10T09:54:29.696695+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:50:54.835899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.695757+0000","last_change":"2024-09-10T09:52:22.323410+0000","last_active":"2024-09-10T09:54:29.695757+0000","last_peered":"2024-09-10T09:54:29.695757+0000","last_clean":"2024-09-10T09:54:29.695757+0000","last_became_active":"2024-09-10T09:52:22.323116+0000","last_became_peered":"2024-09-10T09:52:22.323116+0000","last_unstale":"2024-09-10T09:54:29.695757+0000","last_undegraded":"2024-09-10T09:54:29.695757+0000","last_fullsized":"2024-09-10T09:54:29.695757+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:09:15.115162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.680224+0000","last_change":"2024-09-10T09:52:22.324908+0000","last_active":"2024-09-10T09:54:29.680224+0000","last_peered":"2024-09-10T09:54:29.680224+0000","last_clean":"2024-09-10T09:54:29.680224+0000","last_became_active":"2024-09-10T09:52:22.324449+0000","last_became_peered":"2024-09-10T09:52:22.324449+0000","last_unstale":"2024-09-10T09:54:29.680224+0000","last_undegraded":"2024-09-10T09:54:29.680224+0000","last_fullsized":"2024-09-10T09:54:29.680224+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:43:24.535721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"96'10","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.700039+0000","last_change":"2024-09-10T09:52:22.322157+0000","last_active":"2024-09-10T09:54:29.700039+0000","last_peered":"2024-09-10T09:54:29.700039+0000","last_clean":"2024-09-10T09:54:29.700039+0000","last_became_active":"2024-09-10T09:52:22.321856+0000","last_became_peered":"2024-09-10T09:52:22.321856+0000","last_unstale":"2024-09-10T09:54:29.700039+0000","last_undegraded":"2024-09-10T09:54:29.700039+0000","last_fullsized":"2024-09-10T09:54:29.700039+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:36:38.395370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"96'7","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.703464+0000","last_change":"2024-09-10T09:53:00.800337+0000","last_active":"2024-09-10T09:54:29.703464+0000","last_peered":"2024-09-10T09:54:29.703464+0000","last_clean":"2024-09-10T09:54:29.703464+0000","last_became_active":"2024-09-10T09:53:00.799863+0000","last_became_peered":"2024-09-10T09:53:00.799863+0000","last_unstale":"2024-09-10T09:54:29.703464+0000","last_undegraded":"2024-09-10T09:54:29.703464+0000","last_fullsized":"2024-09-10T09:54:29.703464+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:18.017672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.35","version":"96'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.693580+0000","last_change":"2024-09-10T09:52:22.310735+0000","last_active":"2024-09-10T09:54:29.693580+0000","last_peered":"2024-09-10T09:54:29.693580+0000","last_clean":"2024-09-10T09:54:29.693580+0000","last_became_active":"2024-09-10T09:52:22.310376+0000","last_became_peered":"2024-09-10T09:52:22.310376+0000","last_unstale":"2024-09-10T09:54:29.693580+0000","last_undegraded":"2024-09-10T09:54:29.693580+0000","last_fullsized":"2024-09-10T09:54:29.693580+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:38:08.637588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"96'16","reported_seq":55,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.685458+0000","last_change":"2024-09-10T09:52:22.321113+0000","last_active":"2024-09-10T09:54:29.685458+0000","last_peered":"2024-09-10T09:54:29.685458+0000","last_clean":"2024-09-10T09:54:29.685458+0000","last_became_active":"2024-09-10T09:52:22.320767+0000","last_became_peered":"2024-09-10T09:52:22.320767+0000","last_unstale":"2024-09-10T09:54:29.685458+0000","last_undegraded":"2024-09-10T09:54:29.685458+0000","last_fullsized":"2024-09-10T09:54:29.685458+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:37.549322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"96'11","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.687162+0000","last_change":"2024-09-10T09:52:22.318634+0000","last_active":"2024-09-10T09:54:29.687162+0000","last_peered":"2024-09-10T09:54:29.687162+0000","last_clean":"2024-09-10T09:54:29.687162+0000","last_became_active":"2024-09-10T09:52:22.318294+0000","last_became_peered":"2024-09-10T09:52:22.318294+0000","last_unstale":"2024-09-10T09:54:29.687162+0000","last_undegraded":"2024-09-10T09:54:29.687162+0000","last_fullsized":"2024-09-10T09:54:29.687162+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:19:31.262055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"96'9","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.698725+0000","last_change":"2024-09-10T09:52:22.324567+0000","last_active":"2024-09-10T09:54:29.698725+0000","last_peered":"2024-09-10T09:54:29.698725+0000","last_clean":"2024-09-10T09:54:29.698725+0000","last_became_active":"2024-09-10T09:52:22.321803+0000","last_became_peered":"2024-09-10T09:52:22.321803+0000","last_unstale":"2024-09-10T09:54:29.698725+0000","last_undegraded":"2024-09-10T09:54:29.698725+0000","last_fullsized":"2024-09-10T09:54:29.698725+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:08:11.099428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"96'5","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.699735+0000","last_change":"2024-09-10T09:52:22.318163+0000","last_active":"2024-09-10T09:54:29.699735+0000","last_peered":"2024-09-10T09:54:29.699735+0000","last_clean":"2024-09-10T09:54:29.699735+0000","last_became_active":"2024-09-10T09:52:22.317808+0000","last_became_peered":"2024-09-10T09:52:22.317808+0000","last_unstale":"2024-09-10T09:54:29.699735+0000","last_undegraded":"2024-09-10T09:54:29.699735+0000","last_fullsized":"2024-09-10T09:54:29.699735+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:19.214291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"96'6","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.704960+0000","last_change":"2024-09-10T09:52:22.323035+0000","last_active":"2024-09-10T09:54:29.704960+0000","last_peered":"2024-09-10T09:54:29.704960+0000","last_clean":"2024-09-10T09:54:29.704960+0000","last_became_active":"2024-09-10T09:52:22.322679+0000","last_became_peered":"2024-09-10T09:52:22.322679+0000","last_unstale":"2024-09-10T09:54:29.704960+0000","last_undegraded":"2024-09-10T09:54:29.704960+0000","last_fullsized":"2024-09-10T09:54:29.704960+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:23:13.762612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"96'4","reported_seq":40,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.700111+0000","last_change":"2024-09-10T09:52:22.315887+0000","last_active":"2024-09-10T09:54:29.700111+0000","last_peered":"2024-09-10T09:54:29.700111+0000","last_clean":"2024-09-10T09:54:29.700111+0000","last_became_active":"2024-09-10T09:52:22.315569+0000","last_became_peered":"2024-09-10T09:52:22.315569+0000","last_unstale":"2024-09-10T09:54:29.700111+0000","last_undegraded":"2024-09-10T09:54:29.700111+0000","last_fullsized":"2024-09-10T09:54:29.700111+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:46:57.094320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"96'8","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.676312+0000","last_change":"2024-09-10T09:52:22.314060+0000","last_active":"2024-09-10T09:54:29.676312+0000","last_peered":"2024-09-10T09:54:29.676312+0000","last_clean":"2024-09-10T09:54:29.676312+0000","last_became_active":"2024-09-10T09:52:22.313572+0000","last_became_peered":"2024-09-10T09:52:22.313572+0000","last_unstale":"2024-09-10T09:54:29.676312+0000","last_undegraded":"2024-09-10T09:54:29.676312+0000","last_fullsized":"2024-09-10T09:54:29.676312+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:11:17.158625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.692163+0000","last_change":"2024-09-10T09:52:22.327233+0000","last_active":"2024-09-10T09:54:29.692163+0000","last_peered":"2024-09-10T09:54:29.692163+0000","last_clean":"2024-09-10T09:54:29.692163+0000","last_became_active":"2024-09-10T09:52:22.326945+0000","last_became_peered":"2024-09-10T09:52:22.326945+0000","last_unstale":"2024-09-10T09:54:29.692163+0000","last_undegraded":"2024-09-10T09:54:29.692163+0000","last_fullsized":"2024-09-10T09:54:29.692163+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:31:13.490147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"96'4","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.681664+0000","last_change":"2024-09-10T09:52:22.330912+0000","last_active":"2024-09-10T09:54:29.681664+0000","last_peered":"2024-09-10T09:54:29.681664+0000","last_clean":"2024-09-10T09:54:29.681664+0000","last_became_active":"2024-09-10T09:52:22.330149+0000","last_became_peered":"2024-09-10T09:52:22.330149+0000","last_unstale":"2024-09-10T09:54:29.681664+0000","last_undegraded":"2024-09-10T09:54:29.681664+0000","last_fullsized":"2024-09-10T09:54:29.681664+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:10:48.842499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"96'8","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.697891+0000","last_change":"2024-09-10T09:52:22.327035+0000","last_active":"2024-09-10T09:54:29.697891+0000","last_peered":"2024-09-10T09:54:29.697891+0000","last_clean":"2024-09-10T09:54:29.697891+0000","last_became_active":"2024-09-10T09:52:22.326741+0000","last_became_peered":"2024-09-10T09:52:22.326741+0000","last_unstale":"2024-09-10T09:54:29.697891+0000","last_undegraded":"2024-09-10T09:54:29.697891+0000","last_fullsized":"2024-09-10T09:54:29.697891+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:21:05.026614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70741,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1211,"num_read_kb":685,"num_write":259,"num_write_kb":150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":4426,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":213467,"data_compressed":5661,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":223,"ondisk_log_size":223,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282980,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449137,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1073152,"data_stored":554558,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":70,"seq":300647710772,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32372,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712780,"statfs":{"total":95995035648,"available":95961886720,"internally_reserved":0,"allocated":1216512,"data_stored":1204571,"data_compressed":70462,"data_compressed_allocated":94208,"data_compressed_original":688128,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":65,"seq":279172874300,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32256,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712896,"statfs":{"total":95995035648,"available":95962005504,"internally_reserved":0,"allocated":1097728,"data_stored":557996,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135938,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32244,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712908,"statfs":{"total":95995035648,"available":95962017792,"internally_reserved":0,"allocated":1085440,"data_stored":557702,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364870,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28136,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717016,"statfs":{"total":95995035648,"available":95966224384,"internally_reserved":0,"allocated":1138688,"data_stored":1150864,"data_compressed":68575,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528396,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1069056,"data_stored":549715,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790034,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28144,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717008,"statfs":{"total":95995035648,"available":95966216192,"internally_reserved":0,"allocated":1146880,"data_stored":1142404,"data_compressed":68575,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953557,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28144,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717008,"statfs":{"total":95995035648,"available":95966216192,"internally_reserved":0,"allocated":1146880,"data_stored":617574,"data_compressed":34470,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182491,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32240,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712912,"statfs":{"total":95995035648,"available":95962021888,"internally_reserved":0,"allocated":1081344,"data_stored":549112,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411425,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36404,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708748,"statfs":{"total":95995035648,"available":95957757952,"internally_reserved":0,"allocated":1085440,"data_stored":551188,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574951,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28140,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717012,"statfs":{"total":95995035648,"available":95966220288,"internally_reserved":0,"allocated":1142784,"data_stored":614979,"data_compressed":34470,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738476,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1093632,"data_stored":550860,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34340,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34340,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34340,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35992,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35992,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35992,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35992,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35992,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":66297,"data_compressed":1887,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":64796,"data_compressed":1887,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":65521,"data_compressed":1887,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T09:55:18.577 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-09-10T09:55:18.578 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T09:55:18.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.767+0000 7f810d200640 1 -- 172.21.15.77:0/1744850978 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8108104d60 msgr2=0x7f81081051e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:18.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.767+0000 7f810d200640 1 --2- 172.21.15.77:0/1744850978 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8108104d60 0x7f81081051e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f80f4009900 tx=0x7f80f402a3f0 comp rx=0 tx=0).stop 2024-09-10T09:55:18.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.767+0000 7f810d200640 1 -- 172.21.15.77:0/1744850978 shutdown_connections 2024-09-10T09:55:18.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.767+0000 7f810d200640 1 --2- 172.21.15.77:0/1744850978 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8108105720 0x7f810810bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:18.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.767+0000 7f810d200640 1 --2- 172.21.15.77:0/1744850978 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8108104d60 0x7f81081051e0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:18.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.767+0000 7f810d200640 1 --2- 172.21.15.77:0/1744850978 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8108103b70 0x7f8108103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:18.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.767+0000 7f810d200640 1 -- 172.21.15.77:0/1744850978 >> 172.21.15.77:0/1744850978 conn(0x7f81080ff2f0 msgr2=0x7f8108101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:18.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.767+0000 7f810d200640 1 -- 172.21.15.77:0/1744850978 shutdown_connections 2024-09-10T09:55:18.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.767+0000 7f810d200640 1 -- 172.21.15.77:0/1744850978 wait complete. 2024-09-10T09:55:18.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.768+0000 7f810d200640 1 Processor -- start 2024-09-10T09:55:18.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.768+0000 7f810d200640 1 -- start start 2024-09-10T09:55:18.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.768+0000 7f810d200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8108103b70 0x7f8108199eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:18.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.768+0000 7f810d200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8108104d60 0x7f810819a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:18.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.768+0000 7f810d200640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8108105720 0x7f81081a1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:18.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.768+0000 7f810d200640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f810810f7d0 con 0x7f8108104d60 2024-09-10T09:55:18.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.768+0000 7f810d200640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f810810f650 con 0x7f8108105720 2024-09-10T09:55:18.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.768+0000 7f810d200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f810810f950 con 0x7f8108103b70 2024-09-10T09:55:18.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.769+0000 7f8106a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8108104d60 0x7f810819a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:18.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.769+0000 7f8106a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8108104d60 0x7f810819a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52748/0 (socket says 172.21.15.77:52748) 2024-09-10T09:55:18.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.769+0000 7f8106a00640 1 -- 172.21.15.77:0/4038078658 learned_addr learned my addr 172.21.15.77:0/4038078658 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:55:18.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.769+0000 7f8107e00640 1 --2- 172.21.15.77:0/4038078658 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8108105720 0x7f81081a1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:18.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.769+0000 7f8107400640 1 --2- 172.21.15.77:0/4038078658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8108103b70 0x7f8108199eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:18.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.769+0000 7f8106a00640 1 -- 172.21.15.77:0/4038078658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8108103b70 msgr2=0x7f8108199eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:18.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.769+0000 7f8106a00640 1 --2- 172.21.15.77:0/4038078658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8108103b70 0x7f8108199eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:18.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.769+0000 7f8106a00640 1 -- 172.21.15.77:0/4038078658 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8108105720 msgr2=0x7f81081a1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:18.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.769+0000 7f8106a00640 1 --2- 172.21.15.77:0/4038078658 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8108105720 0x7f81081a1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:18.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.769+0000 7f8106a00640 1 -- 172.21.15.77:0/4038078658 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81081a1ad0 con 0x7f8108104d60 2024-09-10T09:55:18.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.769+0000 7f8106a00640 1 --2- 172.21.15.77:0/4038078658 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8108104d60 0x7f810819a3f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f80f4009230 tx=0x7f80f4031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:18.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.770+0000 7f80ffe00640 1 -- 172.21.15.77:0/4038078658 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f403f030 con 0x7f8108104d60 2024-09-10T09:55:18.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.770+0000 7f80ffe00640 1 -- 172.21.15.77:0/4038078658 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80f4003ad0 con 0x7f8108104d60 2024-09-10T09:55:18.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.770+0000 7f810d200640 1 -- 172.21.15.77:0/4038078658 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81081a1d60 con 0x7f8108104d60 2024-09-10T09:55:18.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.770+0000 7f80ffe00640 1 -- 172.21.15.77:0/4038078658 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f4035070 con 0x7f8108104d60 2024-09-10T09:55:18.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.770+0000 7f810d200640 1 -- 172.21.15.77:0/4038078658 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81081a2240 con 0x7f8108104d60 2024-09-10T09:55:18.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.771+0000 7f810d200640 1 -- 172.21.15.77:0/4038078658 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f810810af50 con 0x7f8108104d60 2024-09-10T09:55:18.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.773+0000 7f80ffe00640 1 -- 172.21.15.77:0/4038078658 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f80f4042030 con 0x7f8108104d60 2024-09-10T09:55:18.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.774+0000 7f80ffe00640 1 --2- 172.21.15.77:0/4038078658 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f80ec0a2c10 0x7f80ec0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:18.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.774+0000 7f80ffe00640 1 -- 172.21.15.77:0/4038078658 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f80f40ef910 con 0x7f8108104d60 2024-09-10T09:55:18.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.774+0000 7f8107400640 1 --2- 172.21.15.77:0/4038078658 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f80ec0a2c10 0x7f80ec0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:18.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.775+0000 7f8107400640 1 --2- 172.21.15.77:0/4038078658 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f80ec0a2c10 0x7f80ec0a50d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f80f80095f0 tx=0x7f80f8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:18.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.778+0000 7f80ffe00640 1 -- 172.21.15.77:0/4038078658 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f80f40bc480 con 0x7f8108104d60 2024-09-10T09:55:18.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.956+0000 7f810d200640 1 -- 172.21.15.77:0/4038078658 --> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8108062630 con 0x7f80ec0a2c10 2024-09-10T09:55:18.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.964+0000 7f80ffe00640 1 -- 172.21.15.77:0/4038078658 <== mgr.14150 v2:172.21.15.77:6800/4261324650 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340852 (secure 0 0 0) 0x7f8108062630 con 0x7f80ec0a2c10 2024-09-10T09:55:18.968 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:55:18.970 INFO:teuthology.orchestra.run.smithi077.stderr:dumped all 2024-09-10T09:55:18.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.968+0000 7f810d200640 1 -- 172.21.15.77:0/4038078658 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f80ec0a2c10 msgr2=0x7f80ec0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:18.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.968+0000 7f810d200640 1 --2- 172.21.15.77:0/4038078658 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f80ec0a2c10 0x7f80ec0a50d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f80f80095f0 tx=0x7f80f8005f90 comp rx=0 tx=0).stop 2024-09-10T09:55:18.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.968+0000 7f810d200640 1 -- 172.21.15.77:0/4038078658 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8108104d60 msgr2=0x7f810819a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:18.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.968+0000 7f810d200640 1 --2- 172.21.15.77:0/4038078658 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8108104d60 0x7f810819a3f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f80f4009230 tx=0x7f80f4031b60 comp rx=0 tx=0).stop 2024-09-10T09:55:18.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.968+0000 7f810d200640 1 -- 172.21.15.77:0/4038078658 shutdown_connections 2024-09-10T09:55:18.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.968+0000 7f810d200640 1 --2- 172.21.15.77:0/4038078658 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f80ec0a2c10 0x7f80ec0a50d0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:18.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.968+0000 7f810d200640 1 --2- 172.21.15.77:0/4038078658 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8108105720 0x7f81081a1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:18.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.968+0000 7f810d200640 1 --2- 172.21.15.77:0/4038078658 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8108104d60 0x7f810819a3f0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:18.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.968+0000 7f810d200640 1 --2- 172.21.15.77:0/4038078658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8108103b70 0x7f8108199eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:18.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.968+0000 7f810d200640 1 -- 172.21.15.77:0/4038078658 >> 172.21.15.77:0/4038078658 conn(0x7f81080ff2f0 msgr2=0x7f8108108120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:18.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.969+0000 7f810d200640 1 -- 172.21.15.77:0/4038078658 shutdown_connections 2024-09-10T09:55:18.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:18.969+0000 7f810d200640 1 -- 172.21.15.77:0/4038078658 wait complete. 2024-09-10T09:55:18.986 INFO:teuthology.orchestra.run.smithi077.stdout:{"pg_ready":true,"pg_map":{"version":395,"stamp":"2024-09-10T09:55:17.719147+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":661343,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1337,"num_read_kb":794,"num_write":442,"num_write_kb":3090,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1185162,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":862,"ondisk_log_size":862,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":366472,"kb_used_data":13064,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124575352,"statfs":{"total":1151940427776,"available":1151565160448,"internally_reserved":0,"allocated":13377536,"data_stored":8601523,"data_compressed":504633,"data_compressed_allocated":688128,"data_compressed_original":2875392,"omap_allocated":19143,"internal_metadata":361674041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":105,"num_read_kb":55,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003408"},"pg_stats":[{"pgid":"2.1f","version":"96'14","reported_seq":188,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:55:15.056536+0000","last_change":"2024-09-10T09:53:00.803590+0000","last_active":"2024-09-10T09:55:15.056536+0000","last_peered":"2024-09-10T09:55:15.056536+0000","last_clean":"2024-09-10T09:55:15.056536+0000","last_became_active":"2024-09-10T09:53:00.798189+0000","last_became_peered":"2024-09-10T09:53:00.798189+0000","last_unstale":"2024-09-10T09:55:15.056536+0000","last_undegraded":"2024-09-10T09:55:15.056536+0000","last_fullsized":"2024-09-10T09:55:15.056536+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:18:48.808836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":121,"num_read_kb":121,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"96'5","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.682459+0000","last_change":"2024-09-10T09:52:22.318195+0000","last_active":"2024-09-10T09:54:29.682459+0000","last_peered":"2024-09-10T09:54:29.682459+0000","last_clean":"2024-09-10T09:54:29.682459+0000","last_became_active":"2024-09-10T09:52:22.315905+0000","last_became_peered":"2024-09-10T09:52:22.315905+0000","last_unstale":"2024-09-10T09:54:29.682459+0000","last_undegraded":"2024-09-10T09:54:29.682459+0000","last_fullsized":"2024-09-10T09:54:29.682459+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:53:55.515984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.843988+0000","last_change":"2024-09-10T09:52:31.369780+0000","last_active":"2024-09-10T09:54:00.843988+0000","last_peered":"2024-09-10T09:54:00.843988+0000","last_clean":"2024-09-10T09:54:00.843988+0000","last_became_active":"2024-09-10T09:52:31.369238+0000","last_became_peered":"2024-09-10T09:52:31.369238+0000","last_unstale":"2024-09-10T09:54:00.843988+0000","last_undegraded":"2024-09-10T09:54:00.843988+0000","last_fullsized":"2024-09-10T09:54:00.843988+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:39:05.318493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"94'2","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.847203+0000","last_change":"2024-09-10T09:52:19.912276+0000","last_active":"2024-09-10T09:54:00.847203+0000","last_peered":"2024-09-10T09:54:00.847203+0000","last_clean":"2024-09-10T09:54:00.847203+0000","last_became_active":"2024-09-10T09:52:19.911452+0000","last_became_peered":"2024-09-10T09:52:19.911452+0000","last_unstale":"2024-09-10T09:54:00.847203+0000","last_undegraded":"2024-09-10T09:54:00.847203+0000","last_fullsized":"2024-09-10T09:54:00.847203+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:24:31.262207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"96'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.705105+0000","last_change":"2024-09-10T09:52:22.327641+0000","last_active":"2024-09-10T09:54:29.705105+0000","last_peered":"2024-09-10T09:54:29.705105+0000","last_clean":"2024-09-10T09:54:29.705105+0000","last_became_active":"2024-09-10T09:52:22.326969+0000","last_became_peered":"2024-09-10T09:52:22.326969+0000","last_unstale":"2024-09-10T09:54:29.705105+0000","last_undegraded":"2024-09-10T09:54:29.705105+0000","last_fullsized":"2024-09-10T09:54:29.705105+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:35:27.825534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.951014+0000","last_change":"2024-09-10T09:52:31.372360+0000","last_active":"2024-09-10T09:54:00.951014+0000","last_peered":"2024-09-10T09:54:00.951014+0000","last_clean":"2024-09-10T09:54:00.951014+0000","last_became_active":"2024-09-10T09:52:31.371957+0000","last_became_peered":"2024-09-10T09:52:31.371957+0000","last_unstale":"2024-09-10T09:54:00.951014+0000","last_undegraded":"2024-09-10T09:54:00.951014+0000","last_fullsized":"2024-09-10T09:54:00.951014+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:49:34.776185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"96'13","reported_seq":55,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:51.750986+0000","last_change":"2024-09-10T09:52:19.843532+0000","last_active":"2024-09-10T09:54:51.750986+0000","last_peered":"2024-09-10T09:54:51.750986+0000","last_clean":"2024-09-10T09:54:51.750986+0000","last_became_active":"2024-09-10T09:52:19.843070+0000","last_became_peered":"2024-09-10T09:52:19.843070+0000","last_unstale":"2024-09-10T09:54:51.750986+0000","last_undegraded":"2024-09-10T09:54:51.750986+0000","last_fullsized":"2024-09-10T09:54:51.750986+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:08:02.223373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"96'9","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.692662+0000","last_change":"2024-09-10T09:52:22.316981+0000","last_active":"2024-09-10T09:54:29.692662+0000","last_peered":"2024-09-10T09:54:29.692662+0000","last_clean":"2024-09-10T09:54:29.692662+0000","last_became_active":"2024-09-10T09:52:22.316648+0000","last_became_peered":"2024-09-10T09:52:22.316648+0000","last_unstale":"2024-09-10T09:54:29.692662+0000","last_undegraded":"2024-09-10T09:54:29.692662+0000","last_fullsized":"2024-09-10T09:54:29.692662+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:28:56.742496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.951485+0000","last_change":"2024-09-10T09:52:31.376811+0000","last_active":"2024-09-10T09:54:00.951485+0000","last_peered":"2024-09-10T09:54:00.951485+0000","last_clean":"2024-09-10T09:54:00.951485+0000","last_became_active":"2024-09-10T09:52:31.376007+0000","last_became_peered":"2024-09-10T09:52:31.376007+0000","last_unstale":"2024-09-10T09:54:00.951485+0000","last_undegraded":"2024-09-10T09:54:00.951485+0000","last_fullsized":"2024-09-10T09:54:00.951485+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:29:26.213281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"94'5","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.149772+0000","last_change":"2024-09-10T09:52:19.837848+0000","last_active":"2024-09-10T09:54:01.149772+0000","last_peered":"2024-09-10T09:54:01.149772+0000","last_clean":"2024-09-10T09:54:01.149772+0000","last_became_active":"2024-09-10T09:52:19.837383+0000","last_became_peered":"2024-09-10T09:52:19.837383+0000","last_unstale":"2024-09-10T09:54:01.149772+0000","last_undegraded":"2024-09-10T09:54:01.149772+0000","last_fullsized":"2024-09-10T09:54:01.149772+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:50:42.813665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"96'15","reported_seq":60,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.685844+0000","last_change":"2024-09-10T09:54:01.060407+0000","last_active":"2024-09-10T09:54:29.685844+0000","last_peered":"2024-09-10T09:54:29.685844+0000","last_clean":"2024-09-10T09:54:29.685844+0000","last_became_active":"2024-09-10T09:54:01.060126+0000","last_became_peered":"2024-09-10T09:54:01.060126+0000","last_unstale":"2024-09-10T09:54:29.685844+0000","last_undegraded":"2024-09-10T09:54:29.685844+0000","last_fullsized":"2024-09-10T09:54:29.685844+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:59:29.765375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.951119+0000","last_change":"2024-09-10T09:52:31.370910+0000","last_active":"2024-09-10T09:54:00.951119+0000","last_peered":"2024-09-10T09:54:00.951119+0000","last_clean":"2024-09-10T09:54:00.951119+0000","last_became_active":"2024-09-10T09:52:31.370467+0000","last_became_peered":"2024-09-10T09:52:31.370467+0000","last_unstale":"2024-09-10T09:54:00.951119+0000","last_undegraded":"2024-09-10T09:54:00.951119+0000","last_fullsized":"2024-09-10T09:54:00.951119+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:49:25.037820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"96'5","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.702762+0000","last_change":"2024-09-10T09:52:22.310708+0000","last_active":"2024-09-10T09:54:29.702762+0000","last_peered":"2024-09-10T09:54:29.702762+0000","last_clean":"2024-09-10T09:54:29.702762+0000","last_became_active":"2024-09-10T09:52:22.310387+0000","last_became_peered":"2024-09-10T09:52:22.310387+0000","last_unstale":"2024-09-10T09:54:29.702762+0000","last_undegraded":"2024-09-10T09:54:29.702762+0000","last_fullsized":"2024-09-10T09:54:29.702762+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:35:03.115408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"94'4","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.149453+0000","last_change":"2024-09-10T09:52:19.841417+0000","last_active":"2024-09-10T09:54:01.149453+0000","last_peered":"2024-09-10T09:54:01.149453+0000","last_clean":"2024-09-10T09:54:01.149453+0000","last_became_active":"2024-09-10T09:52:19.841083+0000","last_became_peered":"2024-09-10T09:52:19.841083+0000","last_unstale":"2024-09-10T09:54:01.149453+0000","last_undegraded":"2024-09-10T09:54:01.149453+0000","last_fullsized":"2024-09-10T09:54:01.149453+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:01:19.008345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.951294+0000","last_change":"2024-09-10T09:52:31.367406+0000","last_active":"2024-09-10T09:54:00.951294+0000","last_peered":"2024-09-10T09:54:00.951294+0000","last_clean":"2024-09-10T09:54:00.951294+0000","last_became_active":"2024-09-10T09:52:31.366961+0000","last_became_peered":"2024-09-10T09:52:31.366961+0000","last_unstale":"2024-09-10T09:54:00.951294+0000","last_undegraded":"2024-09-10T09:54:00.951294+0000","last_fullsized":"2024-09-10T09:54:00.951294+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:04:51.714465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"96'8","reported_seq":19,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.705252+0000","last_change":"2024-09-10T09:54:01.249212+0000","last_active":"2024-09-10T09:54:29.705252+0000","last_peered":"2024-09-10T09:54:29.705252+0000","last_clean":"2024-09-10T09:54:29.705252+0000","last_became_active":"2024-09-10T09:54:01.248840+0000","last_became_peered":"2024-09-10T09:54:01.248840+0000","last_unstale":"2024-09-10T09:54:29.705252+0000","last_undegraded":"2024-09-10T09:54:29.705252+0000","last_fullsized":"2024-09-10T09:54:29.705252+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:39:13.167435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,10],"acting":[9,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.a","version":"94'3","reported_seq":19,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.954964+0000","last_change":"2024-09-10T09:53:00.800421+0000","last_active":"2024-09-10T09:54:00.954964+0000","last_peered":"2024-09-10T09:54:00.954964+0000","last_clean":"2024-09-10T09:54:00.954964+0000","last_became_active":"2024-09-10T09:53:00.800026+0000","last_became_peered":"2024-09-10T09:53:00.800026+0000","last_unstale":"2024-09-10T09:54:00.954964+0000","last_undegraded":"2024-09-10T09:54:00.954964+0000","last_fullsized":"2024-09-10T09:54:00.954964+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:57:45.252420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"94'1","reported_seq":30,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.951388+0000","last_change":"2024-09-10T09:52:31.376605+0000","last_active":"2024-09-10T09:54:00.951388+0000","last_peered":"2024-09-10T09:54:00.951388+0000","last_clean":"2024-09-10T09:54:00.951388+0000","last_became_active":"2024-09-10T09:52:31.376083+0000","last_became_peered":"2024-09-10T09:52:31.376083+0000","last_unstale":"2024-09-10T09:54:00.951388+0000","last_undegraded":"2024-09-10T09:54:00.951388+0000","last_fullsized":"2024-09-10T09:54:00.951388+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:05:10.030673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"96'9","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.697029+0000","last_change":"2024-09-10T09:52:22.327303+0000","last_active":"2024-09-10T09:54:29.697029+0000","last_peered":"2024-09-10T09:54:29.697029+0000","last_clean":"2024-09-10T09:54:29.697029+0000","last_became_active":"2024-09-10T09:52:22.326753+0000","last_became_peered":"2024-09-10T09:52:22.326753+0000","last_unstale":"2024-09-10T09:54:29.697029+0000","last_undegraded":"2024-09-10T09:54:29.697029+0000","last_fullsized":"2024-09-10T09:54:29.697029+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:27:26.668151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"94'7","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.845956+0000","last_change":"2024-09-10T09:52:19.836484+0000","last_active":"2024-09-10T09:54:00.845956+0000","last_peered":"2024-09-10T09:54:00.845956+0000","last_clean":"2024-09-10T09:54:00.845956+0000","last_became_active":"2024-09-10T09:52:19.836130+0000","last_became_peered":"2024-09-10T09:52:19.836130+0000","last_unstale":"2024-09-10T09:54:00.845956+0000","last_undegraded":"2024-09-10T09:54:00.845956+0000","last_fullsized":"2024-09-10T09:54:00.845956+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:31:58.731981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.844293+0000","last_change":"2024-09-10T09:52:31.375279+0000","last_active":"2024-09-10T09:54:00.844293+0000","last_peered":"2024-09-10T09:54:00.844293+0000","last_clean":"2024-09-10T09:54:00.844293+0000","last_became_active":"2024-09-10T09:52:31.374181+0000","last_became_peered":"2024-09-10T09:52:31.374181+0000","last_unstale":"2024-09-10T09:54:00.844293+0000","last_undegraded":"2024-09-10T09:54:00.844293+0000","last_fullsized":"2024-09-10T09:54:00.844293+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:54:51.299224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"96'2","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.676489+0000","last_change":"2024-09-10T09:52:22.314817+0000","last_active":"2024-09-10T09:54:29.676489+0000","last_peered":"2024-09-10T09:54:29.676489+0000","last_clean":"2024-09-10T09:54:29.676489+0000","last_became_active":"2024-09-10T09:52:22.314491+0000","last_became_peered":"2024-09-10T09:52:22.314491+0000","last_unstale":"2024-09-10T09:54:29.676489+0000","last_undegraded":"2024-09-10T09:54:29.676489+0000","last_fullsized":"2024-09-10T09:54:29.676489+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:49:41.941667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"94'2","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.988558+0000","last_change":"2024-09-10T09:52:19.911730+0000","last_active":"2024-09-10T09:54:00.988558+0000","last_peered":"2024-09-10T09:54:00.988558+0000","last_clean":"2024-09-10T09:54:00.988558+0000","last_became_active":"2024-09-10T09:52:19.911393+0000","last_became_peered":"2024-09-10T09:52:19.911393+0000","last_unstale":"2024-09-10T09:54:00.988558+0000","last_undegraded":"2024-09-10T09:54:00.988558+0000","last_fullsized":"2024-09-10T09:54:00.988558+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:46:17.943129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.951301+0000","last_change":"2024-09-10T09:52:31.376498+0000","last_active":"2024-09-10T09:54:00.951301+0000","last_peered":"2024-09-10T09:54:00.951301+0000","last_clean":"2024-09-10T09:54:00.951301+0000","last_became_active":"2024-09-10T09:52:31.375966+0000","last_became_peered":"2024-09-10T09:52:31.375966+0000","last_unstale":"2024-09-10T09:54:00.951301+0000","last_undegraded":"2024-09-10T09:54:00.951301+0000","last_fullsized":"2024-09-10T09:54:00.951301+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:08:30.649399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.705573+0000","last_change":"2024-09-10T09:52:22.331611+0000","last_active":"2024-09-10T09:54:29.705573+0000","last_peered":"2024-09-10T09:54:29.705573+0000","last_clean":"2024-09-10T09:54:29.705573+0000","last_became_active":"2024-09-10T09:52:22.331268+0000","last_became_peered":"2024-09-10T09:52:22.331268+0000","last_unstale":"2024-09-10T09:54:29.705573+0000","last_undegraded":"2024-09-10T09:54:29.705573+0000","last_fullsized":"2024-09-10T09:54:29.705573+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:01:05.433994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"94'7","reported_seq":50,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.951927+0000","last_change":"2024-09-10T09:52:19.838387+0000","last_active":"2024-09-10T09:54:00.951927+0000","last_peered":"2024-09-10T09:54:00.951927+0000","last_clean":"2024-09-10T09:54:00.951927+0000","last_became_active":"2024-09-10T09:52:19.837889+0000","last_became_peered":"2024-09-10T09:52:19.837889+0000","last_unstale":"2024-09-10T09:54:00.951927+0000","last_undegraded":"2024-09-10T09:54:00.951927+0000","last_fullsized":"2024-09-10T09:54:00.951927+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:50:00.187243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.844187+0000","last_change":"2024-09-10T09:52:31.363262+0000","last_active":"2024-09-10T09:54:00.844187+0000","last_peered":"2024-09-10T09:54:00.844187+0000","last_clean":"2024-09-10T09:54:00.844187+0000","last_became_active":"2024-09-10T09:52:31.362762+0000","last_became_peered":"2024-09-10T09:52:31.362762+0000","last_unstale":"2024-09-10T09:54:00.844187+0000","last_undegraded":"2024-09-10T09:54:00.844187+0000","last_fullsized":"2024-09-10T09:54:00.844187+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:48:28.275041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"96'5","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.702169+0000","last_change":"2024-09-10T09:52:22.312472+0000","last_active":"2024-09-10T09:54:29.702169+0000","last_peered":"2024-09-10T09:54:29.702169+0000","last_clean":"2024-09-10T09:54:29.702169+0000","last_became_active":"2024-09-10T09:52:22.312117+0000","last_became_peered":"2024-09-10T09:52:22.312117+0000","last_unstale":"2024-09-10T09:54:29.702169+0000","last_undegraded":"2024-09-10T09:54:29.702169+0000","last_fullsized":"2024-09-10T09:54:29.702169+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:39:10.742137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"96'16","reported_seq":57,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:31.212993+0000","last_change":"2024-09-10T09:52:19.835543+0000","last_active":"2024-09-10T09:54:31.212993+0000","last_peered":"2024-09-10T09:54:31.212993+0000","last_clean":"2024-09-10T09:54:31.212993+0000","last_became_active":"2024-09-10T09:52:19.835170+0000","last_became_peered":"2024-09-10T09:52:19.835170+0000","last_unstale":"2024-09-10T09:54:31.212993+0000","last_undegraded":"2024-09-10T09:54:31.212993+0000","last_fullsized":"2024-09-10T09:54:31.212993+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:37.969277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.845447+0000","last_change":"2024-09-10T09:52:31.372398+0000","last_active":"2024-09-10T09:54:00.845447+0000","last_peered":"2024-09-10T09:54:00.845447+0000","last_clean":"2024-09-10T09:54:00.845447+0000","last_became_active":"2024-09-10T09:52:31.371874+0000","last_became_peered":"2024-09-10T09:52:31.371874+0000","last_unstale":"2024-09-10T09:54:00.845447+0000","last_undegraded":"2024-09-10T09:54:00.845447+0000","last_fullsized":"2024-09-10T09:54:00.845447+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:09:45.374320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"96'9","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.700381+0000","last_change":"2024-09-10T09:52:22.319592+0000","last_active":"2024-09-10T09:54:29.700381+0000","last_peered":"2024-09-10T09:54:29.700381+0000","last_clean":"2024-09-10T09:54:29.700381+0000","last_became_active":"2024-09-10T09:52:22.319298+0000","last_became_peered":"2024-09-10T09:52:22.319298+0000","last_unstale":"2024-09-10T09:54:29.700381+0000","last_undegraded":"2024-09-10T09:54:29.700381+0000","last_fullsized":"2024-09-10T09:54:29.700381+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:56:59.787638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"94'4","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.150011+0000","last_change":"2024-09-10T09:52:19.912923+0000","last_active":"2024-09-10T09:54:01.150011+0000","last_peered":"2024-09-10T09:54:01.150011+0000","last_clean":"2024-09-10T09:54:01.150011+0000","last_became_active":"2024-09-10T09:52:19.912268+0000","last_became_peered":"2024-09-10T09:52:19.912268+0000","last_unstale":"2024-09-10T09:54:01.150011+0000","last_undegraded":"2024-09-10T09:54:01.150011+0000","last_fullsized":"2024-09-10T09:54:01.150011+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:06:45.421004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.955091+0000","last_change":"2024-09-10T09:52:31.372761+0000","last_active":"2024-09-10T09:54:00.955091+0000","last_peered":"2024-09-10T09:54:00.955091+0000","last_clean":"2024-09-10T09:54:00.955091+0000","last_became_active":"2024-09-10T09:52:31.372293+0000","last_became_peered":"2024-09-10T09:52:31.372293+0000","last_unstale":"2024-09-10T09:54:00.955091+0000","last_undegraded":"2024-09-10T09:54:00.955091+0000","last_fullsized":"2024-09-10T09:54:00.955091+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:36:41.446810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"96'4","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.688817+0000","last_change":"2024-09-10T09:52:22.326291+0000","last_active":"2024-09-10T09:54:29.688817+0000","last_peered":"2024-09-10T09:54:29.688817+0000","last_clean":"2024-09-10T09:54:29.688817+0000","last_became_active":"2024-09-10T09:52:22.325985+0000","last_became_peered":"2024-09-10T09:52:22.325985+0000","last_unstale":"2024-09-10T09:54:29.688817+0000","last_undegraded":"2024-09-10T09:54:29.688817+0000","last_fullsized":"2024-09-10T09:54:29.688817+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:06:21.871502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"94'1","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.844068+0000","last_change":"2024-09-10T09:52:19.912851+0000","last_active":"2024-09-10T09:54:00.844068+0000","last_peered":"2024-09-10T09:54:00.844068+0000","last_clean":"2024-09-10T09:54:00.844068+0000","last_became_active":"2024-09-10T09:52:19.912200+0000","last_became_peered":"2024-09-10T09:52:19.912200+0000","last_unstale":"2024-09-10T09:54:00.844068+0000","last_undegraded":"2024-09-10T09:54:00.844068+0000","last_fullsized":"2024-09-10T09:54:00.844068+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:00:47.331755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.149311+0000","last_change":"2024-09-10T09:52:31.371329+0000","last_active":"2024-09-10T09:54:01.149311+0000","last_peered":"2024-09-10T09:54:01.149311+0000","last_clean":"2024-09-10T09:54:01.149311+0000","last_became_active":"2024-09-10T09:52:31.370971+0000","last_became_peered":"2024-09-10T09:52:31.370971+0000","last_unstale":"2024-09-10T09:54:01.149311+0000","last_undegraded":"2024-09-10T09:54:01.149311+0000","last_fullsized":"2024-09-10T09:54:01.149311+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:07:12.928421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"96'7","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.686209+0000","last_change":"2024-09-10T09:52:22.328039+0000","last_active":"2024-09-10T09:54:29.686209+0000","last_peered":"2024-09-10T09:54:29.686209+0000","last_clean":"2024-09-10T09:54:29.686209+0000","last_became_active":"2024-09-10T09:52:22.327588+0000","last_became_peered":"2024-09-10T09:52:22.327588+0000","last_unstale":"2024-09-10T09:54:29.686209+0000","last_undegraded":"2024-09-10T09:54:29.686209+0000","last_fullsized":"2024-09-10T09:54:29.686209+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:35:39.219544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"94'5","reported_seq":138,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:55:11.050835+0000","last_change":"2024-09-10T09:53:00.797651+0000","last_active":"2024-09-10T09:55:11.050835+0000","last_peered":"2024-09-10T09:55:11.050835+0000","last_clean":"2024-09-10T09:55:11.050835+0000","last_became_active":"2024-09-10T09:53:00.797285+0000","last_became_peered":"2024-09-10T09:53:00.797285+0000","last_unstale":"2024-09-10T09:55:11.050835+0000","last_undegraded":"2024-09-10T09:55:11.050835+0000","last_fullsized":"2024-09-10T09:55:11.050835+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:18:02.320032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,6,1],"acting":[11,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.841787+0000","last_change":"2024-09-10T09:52:31.379215+0000","last_active":"2024-09-10T09:54:00.841787+0000","last_peered":"2024-09-10T09:54:00.841787+0000","last_clean":"2024-09-10T09:54:00.841787+0000","last_became_active":"2024-09-10T09:52:31.377107+0000","last_became_peered":"2024-09-10T09:52:31.377107+0000","last_unstale":"2024-09-10T09:54:00.841787+0000","last_undegraded":"2024-09-10T09:54:00.841787+0000","last_fullsized":"2024-09-10T09:54:00.841787+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:09:39.474077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"96'8","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.673518+0000","last_change":"2024-09-10T09:52:22.316474+0000","last_active":"2024-09-10T09:54:29.673518+0000","last_peered":"2024-09-10T09:54:29.673518+0000","last_clean":"2024-09-10T09:54:29.673518+0000","last_became_active":"2024-09-10T09:52:22.316054+0000","last_became_peered":"2024-09-10T09:52:22.316054+0000","last_unstale":"2024-09-10T09:54:29.673518+0000","last_undegraded":"2024-09-10T09:54:29.673518+0000","last_fullsized":"2024-09-10T09:54:29.673518+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:11:31.334780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"96'6","reported_seq":166,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:55:12.082651+0000","last_change":"2024-09-10T09:52:19.911601+0000","last_active":"2024-09-10T09:55:12.082651+0000","last_peered":"2024-09-10T09:55:12.082651+0000","last_clean":"2024-09-10T09:55:12.082651+0000","last_became_active":"2024-09-10T09:52:19.911268+0000","last_became_peered":"2024-09-10T09:52:19.911268+0000","last_unstale":"2024-09-10T09:55:12.082651+0000","last_undegraded":"2024-09-10T09:55:12.082651+0000","last_fullsized":"2024-09-10T09:55:12.082651+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:46:38.158784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":118,"num_read_kb":118,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.841796+0000","last_change":"2024-09-10T09:52:31.375938+0000","last_active":"2024-09-10T09:54:00.841796+0000","last_peered":"2024-09-10T09:54:00.841796+0000","last_clean":"2024-09-10T09:54:00.841796+0000","last_became_active":"2024-09-10T09:52:31.375536+0000","last_became_peered":"2024-09-10T09:52:31.375536+0000","last_unstale":"2024-09-10T09:54:00.841796+0000","last_undegraded":"2024-09-10T09:54:00.841796+0000","last_fullsized":"2024-09-10T09:54:00.841796+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:55:17.303809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"96'5","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.688934+0000","last_change":"2024-09-10T09:52:22.315912+0000","last_active":"2024-09-10T09:54:29.688934+0000","last_peered":"2024-09-10T09:54:29.688934+0000","last_clean":"2024-09-10T09:54:29.688934+0000","last_became_active":"2024-09-10T09:52:22.315433+0000","last_became_peered":"2024-09-10T09:52:22.315433+0000","last_unstale":"2024-09-10T09:54:29.688934+0000","last_undegraded":"2024-09-10T09:54:29.688934+0000","last_fullsized":"2024-09-10T09:54:29.688934+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:57:10.454716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"94'4","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.059138+0000","last_change":"2024-09-10T09:52:19.838966+0000","last_active":"2024-09-10T09:54:01.059138+0000","last_peered":"2024-09-10T09:54:01.059138+0000","last_clean":"2024-09-10T09:54:01.059138+0000","last_became_active":"2024-09-10T09:52:19.838545+0000","last_became_peered":"2024-09-10T09:52:19.838545+0000","last_unstale":"2024-09-10T09:54:01.059138+0000","last_undegraded":"2024-09-10T09:54:01.059138+0000","last_fullsized":"2024-09-10T09:54:01.059138+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:07:00.930146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.847601+0000","last_change":"2024-09-10T09:53:00.802039+0000","last_active":"2024-09-10T09:54:00.847601+0000","last_peered":"2024-09-10T09:54:00.847601+0000","last_clean":"2024-09-10T09:54:00.847601+0000","last_became_active":"2024-09-10T09:53:00.800223+0000","last_became_peered":"2024-09-10T09:53:00.800223+0000","last_unstale":"2024-09-10T09:54:00.847601+0000","last_undegraded":"2024-09-10T09:54:00.847601+0000","last_fullsized":"2024-09-10T09:54:00.847601+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:50:16.026631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,6,10],"acting":[5,8,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"96'8","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.698863+0000","last_change":"2024-09-10T09:52:22.313819+0000","last_active":"2024-09-10T09:54:29.698863+0000","last_peered":"2024-09-10T09:54:29.698863+0000","last_clean":"2024-09-10T09:54:29.698863+0000","last_became_active":"2024-09-10T09:52:22.313473+0000","last_became_peered":"2024-09-10T09:52:22.313473+0000","last_unstale":"2024-09-10T09:54:29.698863+0000","last_undegraded":"2024-09-10T09:54:29.698863+0000","last_fullsized":"2024-09-10T09:54:29.698863+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:44:10.003187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"94'6","reported_seq":51,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.951818+0000","last_change":"2024-09-10T09:52:19.837376+0000","last_active":"2024-09-10T09:54:00.951818+0000","last_peered":"2024-09-10T09:54:00.951818+0000","last_clean":"2024-09-10T09:54:00.951818+0000","last_became_active":"2024-09-10T09:52:19.836880+0000","last_became_peered":"2024-09-10T09:52:19.836880+0000","last_unstale":"2024-09-10T09:54:00.951818+0000","last_undegraded":"2024-09-10T09:54:00.951818+0000","last_fullsized":"2024-09-10T09:54:00.951818+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:48:58.198188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":361,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.059392+0000","last_change":"2024-09-10T09:51:05.871529+0000","last_active":"2024-09-10T09:54:01.059392+0000","last_peered":"2024-09-10T09:54:01.059392+0000","last_clean":"2024-09-10T09:54:01.059392+0000","last_became_active":"2024-09-10T09:51:05.871115+0000","last_became_peered":"2024-09-10T09:51:05.871115+0000","last_unstale":"2024-09-10T09:54:01.059392+0000","last_undegraded":"2024-09-10T09:54:01.059392+0000","last_fullsized":"2024-09-10T09:54:01.059392+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:47:24.175358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:47:24.175358+0000","last_clean_scrub_stamp":"2024-09-10T09:47:24.175358+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:58:30.247077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"94'1","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.059534+0000","last_change":"2024-09-10T09:53:00.800474+0000","last_active":"2024-09-10T09:54:01.059534+0000","last_peered":"2024-09-10T09:54:01.059534+0000","last_clean":"2024-09-10T09:54:01.059534+0000","last_became_active":"2024-09-10T09:53:00.799932+0000","last_became_peered":"2024-09-10T09:53:00.799932+0000","last_unstale":"2024-09-10T09:54:01.059534+0000","last_undegraded":"2024-09-10T09:54:01.059534+0000","last_fullsized":"2024-09-10T09:54:01.059534+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:14:50.998952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,11,9],"acting":[7,8,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.693609+0000","last_change":"2024-09-10T09:52:22.312521+0000","last_active":"2024-09-10T09:54:29.693609+0000","last_peered":"2024-09-10T09:54:29.693609+0000","last_clean":"2024-09-10T09:54:29.693609+0000","last_became_active":"2024-09-10T09:52:22.312107+0000","last_became_peered":"2024-09-10T09:52:22.312107+0000","last_unstale":"2024-09-10T09:54:29.693609+0000","last_undegraded":"2024-09-10T09:54:29.693609+0000","last_fullsized":"2024-09-10T09:54:29.693609+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:14:19.799259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"94'5","reported_seq":166,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:55:12.052387+0000","last_change":"2024-09-10T09:52:19.911454+0000","last_active":"2024-09-10T09:55:12.052387+0000","last_peered":"2024-09-10T09:55:12.052387+0000","last_clean":"2024-09-10T09:55:12.052387+0000","last_became_active":"2024-09-10T09:52:19.911079+0000","last_became_peered":"2024-09-10T09:52:19.911079+0000","last_unstale":"2024-09-10T09:55:12.052387+0000","last_undegraded":"2024-09-10T09:55:12.052387+0000","last_fullsized":"2024-09-10T09:55:12.052387+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:43:38.787689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":119,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.988531+0000","last_change":"2024-09-10T09:52:31.381877+0000","last_active":"2024-09-10T09:54:00.988531+0000","last_peered":"2024-09-10T09:54:00.988531+0000","last_clean":"2024-09-10T09:54:00.988531+0000","last_became_active":"2024-09-10T09:52:31.381472+0000","last_became_peered":"2024-09-10T09:52:31.381472+0000","last_unstale":"2024-09-10T09:54:00.988531+0000","last_undegraded":"2024-09-10T09:54:00.988531+0000","last_fullsized":"2024-09-10T09:54:00.988531+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:41:24.686325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"96'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.702544+0000","last_change":"2024-09-10T09:52:22.313902+0000","last_active":"2024-09-10T09:54:29.702544+0000","last_peered":"2024-09-10T09:54:29.702544+0000","last_clean":"2024-09-10T09:54:29.702544+0000","last_became_active":"2024-09-10T09:52:22.313515+0000","last_became_peered":"2024-09-10T09:52:22.313515+0000","last_unstale":"2024-09-10T09:54:29.702544+0000","last_undegraded":"2024-09-10T09:54:29.702544+0000","last_fullsized":"2024-09-10T09:54:29.702544+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:25:00.238307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"94'3","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.847873+0000","last_change":"2024-09-10T09:52:19.837736+0000","last_active":"2024-09-10T09:54:00.847873+0000","last_peered":"2024-09-10T09:54:00.847873+0000","last_clean":"2024-09-10T09:54:00.847873+0000","last_became_active":"2024-09-10T09:52:19.837370+0000","last_became_peered":"2024-09-10T09:52:19.837370+0000","last_unstale":"2024-09-10T09:54:00.847873+0000","last_undegraded":"2024-09-10T09:54:00.847873+0000","last_fullsized":"2024-09-10T09:54:00.847873+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:54:35.753027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.248213+0000","last_change":"2024-09-10T09:52:31.378945+0000","last_active":"2024-09-10T09:54:01.248213+0000","last_peered":"2024-09-10T09:54:01.248213+0000","last_clean":"2024-09-10T09:54:01.248213+0000","last_became_active":"2024-09-10T09:52:31.378144+0000","last_became_peered":"2024-09-10T09:52:31.378144+0000","last_unstale":"2024-09-10T09:54:01.248213+0000","last_undegraded":"2024-09-10T09:54:01.248213+0000","last_fullsized":"2024-09-10T09:54:01.248213+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:36:06.288293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"96'9","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.692039+0000","last_change":"2024-09-10T09:52:22.316400+0000","last_active":"2024-09-10T09:54:29.692039+0000","last_peered":"2024-09-10T09:54:29.692039+0000","last_clean":"2024-09-10T09:54:29.692039+0000","last_became_active":"2024-09-10T09:52:22.315974+0000","last_became_peered":"2024-09-10T09:52:22.315974+0000","last_unstale":"2024-09-10T09:54:29.692039+0000","last_undegraded":"2024-09-10T09:54:29.692039+0000","last_fullsized":"2024-09-10T09:54:29.692039+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:31:05.538237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"94'5","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.841976+0000","last_change":"2024-09-10T09:52:19.837942+0000","last_active":"2024-09-10T09:54:00.841976+0000","last_peered":"2024-09-10T09:54:00.841976+0000","last_clean":"2024-09-10T09:54:00.841976+0000","last_became_active":"2024-09-10T09:52:19.837629+0000","last_became_peered":"2024-09-10T09:52:19.837629+0000","last_unstale":"2024-09-10T09:54:00.841976+0000","last_undegraded":"2024-09-10T09:54:00.841976+0000","last_fullsized":"2024-09-10T09:54:00.841976+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:34:13.207770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.847019+0000","last_change":"2024-09-10T09:52:31.364231+0000","last_active":"2024-09-10T09:54:00.847019+0000","last_peered":"2024-09-10T09:54:00.847019+0000","last_clean":"2024-09-10T09:54:00.847019+0000","last_became_active":"2024-09-10T09:52:31.363833+0000","last_became_peered":"2024-09-10T09:52:31.363833+0000","last_unstale":"2024-09-10T09:54:00.847019+0000","last_undegraded":"2024-09-10T09:54:00.847019+0000","last_fullsized":"2024-09-10T09:54:00.847019+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:53:51.435700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"96'5","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.680562+0000","last_change":"2024-09-10T09:52:22.319530+0000","last_active":"2024-09-10T09:54:29.680562+0000","last_peered":"2024-09-10T09:54:29.680562+0000","last_clean":"2024-09-10T09:54:29.680562+0000","last_became_active":"2024-09-10T09:52:22.318742+0000","last_became_peered":"2024-09-10T09:52:22.318742+0000","last_unstale":"2024-09-10T09:54:29.680562+0000","last_undegraded":"2024-09-10T09:54:29.680562+0000","last_fullsized":"2024-09-10T09:54:29.680562+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:44:35.050454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"94'2","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.150048+0000","last_change":"2024-09-10T09:52:19.836398+0000","last_active":"2024-09-10T09:54:01.150048+0000","last_peered":"2024-09-10T09:54:01.150048+0000","last_clean":"2024-09-10T09:54:01.150048+0000","last_became_active":"2024-09-10T09:52:19.836055+0000","last_became_peered":"2024-09-10T09:52:19.836055+0000","last_unstale":"2024-09-10T09:54:01.150048+0000","last_undegraded":"2024-09-10T09:54:01.150048+0000","last_fullsized":"2024-09-10T09:54:01.150048+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:49:26.923367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.847215+0000","last_change":"2024-09-10T09:52:31.371109+0000","last_active":"2024-09-10T09:54:00.847215+0000","last_peered":"2024-09-10T09:54:00.847215+0000","last_clean":"2024-09-10T09:54:00.847215+0000","last_became_active":"2024-09-10T09:52:31.370592+0000","last_became_peered":"2024-09-10T09:52:31.370592+0000","last_unstale":"2024-09-10T09:54:00.847215+0000","last_undegraded":"2024-09-10T09:54:00.847215+0000","last_fullsized":"2024-09-10T09:54:00.847215+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:20:23.585187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"96'6","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.685236+0000","last_change":"2024-09-10T09:52:22.319365+0000","last_active":"2024-09-10T09:54:29.685236+0000","last_peered":"2024-09-10T09:54:29.685236+0000","last_clean":"2024-09-10T09:54:29.685236+0000","last_became_active":"2024-09-10T09:52:22.319064+0000","last_became_peered":"2024-09-10T09:52:22.319064+0000","last_unstale":"2024-09-10T09:54:29.685236+0000","last_undegraded":"2024-09-10T09:54:29.685236+0000","last_fullsized":"2024-09-10T09:54:29.685236+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:14:50.089566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"94'17","reported_seq":59,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.149324+0000","last_change":"2024-09-10T09:52:19.911698+0000","last_active":"2024-09-10T09:54:01.149324+0000","last_peered":"2024-09-10T09:54:01.149324+0000","last_clean":"2024-09-10T09:54:01.149324+0000","last_became_active":"2024-09-10T09:52:19.911369+0000","last_became_peered":"2024-09-10T09:52:19.911369+0000","last_unstale":"2024-09-10T09:54:01.149324+0000","last_undegraded":"2024-09-10T09:54:01.149324+0000","last_fullsized":"2024-09-10T09:54:01.149324+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:40:39.045797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.841947+0000","last_change":"2024-09-10T09:52:31.380463+0000","last_active":"2024-09-10T09:54:00.841947+0000","last_peered":"2024-09-10T09:54:00.841947+0000","last_clean":"2024-09-10T09:54:00.841947+0000","last_became_active":"2024-09-10T09:52:31.380114+0000","last_became_peered":"2024-09-10T09:52:31.380114+0000","last_unstale":"2024-09-10T09:54:00.841947+0000","last_undegraded":"2024-09-10T09:54:00.841947+0000","last_fullsized":"2024-09-10T09:54:00.841947+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:19:50.869160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"96'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.682831+0000","last_change":"2024-09-10T09:52:22.313928+0000","last_active":"2024-09-10T09:54:29.682831+0000","last_peered":"2024-09-10T09:54:29.682831+0000","last_clean":"2024-09-10T09:54:29.682831+0000","last_became_active":"2024-09-10T09:52:22.313510+0000","last_became_peered":"2024-09-10T09:52:22.313510+0000","last_unstale":"2024-09-10T09:54:29.682831+0000","last_undegraded":"2024-09-10T09:54:29.682831+0000","last_fullsized":"2024-09-10T09:54:29.682831+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:58:19.031178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"96'7","reported_seq":170,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:55:12.083520+0000","last_change":"2024-09-10T09:52:19.839451+0000","last_active":"2024-09-10T09:55:12.083520+0000","last_peered":"2024-09-10T09:55:12.083520+0000","last_clean":"2024-09-10T09:55:12.083520+0000","last_became_active":"2024-09-10T09:52:19.839124+0000","last_became_peered":"2024-09-10T09:52:19.839124+0000","last_unstale":"2024-09-10T09:55:12.083520+0000","last_undegraded":"2024-09-10T09:55:12.083520+0000","last_fullsized":"2024-09-10T09:55:12.083520+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:51:33.489830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":122,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.844344+0000","last_change":"2024-09-10T09:52:31.376386+0000","last_active":"2024-09-10T09:54:00.844344+0000","last_peered":"2024-09-10T09:54:00.844344+0000","last_clean":"2024-09-10T09:54:00.844344+0000","last_became_active":"2024-09-10T09:52:31.376066+0000","last_became_peered":"2024-09-10T09:52:31.376066+0000","last_unstale":"2024-09-10T09:54:00.844344+0000","last_undegraded":"2024-09-10T09:54:00.844344+0000","last_fullsized":"2024-09-10T09:54:00.844344+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:23:07.191842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"96'15","reported_seq":52,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.702901+0000","last_change":"2024-09-10T09:52:22.323590+0000","last_active":"2024-09-10T09:54:29.702901+0000","last_peered":"2024-09-10T09:54:29.702901+0000","last_clean":"2024-09-10T09:54:29.702901+0000","last_became_active":"2024-09-10T09:52:22.323264+0000","last_became_peered":"2024-09-10T09:52:22.323264+0000","last_unstale":"2024-09-10T09:54:29.702901+0000","last_undegraded":"2024-09-10T09:54:29.702901+0000","last_fullsized":"2024-09-10T09:54:29.702901+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:07:56.572337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"94'5","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.847940+0000","last_change":"2024-09-10T09:52:19.840620+0000","last_active":"2024-09-10T09:54:00.847940+0000","last_peered":"2024-09-10T09:54:00.847940+0000","last_clean":"2024-09-10T09:54:00.847940+0000","last_became_active":"2024-09-10T09:52:19.840217+0000","last_became_peered":"2024-09-10T09:52:19.840217+0000","last_unstale":"2024-09-10T09:54:00.847940+0000","last_undegraded":"2024-09-10T09:54:00.847940+0000","last_fullsized":"2024-09-10T09:54:00.847940+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:55:13.111993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.955071+0000","last_change":"2024-09-10T09:52:31.373895+0000","last_active":"2024-09-10T09:54:00.955071+0000","last_peered":"2024-09-10T09:54:00.955071+0000","last_clean":"2024-09-10T09:54:00.955071+0000","last_became_active":"2024-09-10T09:52:31.373474+0000","last_became_peered":"2024-09-10T09:52:31.373474+0000","last_unstale":"2024-09-10T09:54:00.955071+0000","last_undegraded":"2024-09-10T09:54:00.955071+0000","last_fullsized":"2024-09-10T09:54:00.955071+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:39:16.049859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"96'9","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.704377+0000","last_change":"2024-09-10T09:52:22.319294+0000","last_active":"2024-09-10T09:54:29.704377+0000","last_peered":"2024-09-10T09:54:29.704377+0000","last_clean":"2024-09-10T09:54:29.704377+0000","last_became_active":"2024-09-10T09:52:22.318918+0000","last_became_peered":"2024-09-10T09:52:22.318918+0000","last_unstale":"2024-09-10T09:54:29.704377+0000","last_undegraded":"2024-09-10T09:54:29.704377+0000","last_fullsized":"2024-09-10T09:54:29.704377+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:52:46.434182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"94'9","reported_seq":53,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.844244+0000","last_change":"2024-09-10T09:52:19.842286+0000","last_active":"2024-09-10T09:54:00.844244+0000","last_peered":"2024-09-10T09:54:00.844244+0000","last_clean":"2024-09-10T09:54:00.844244+0000","last_became_active":"2024-09-10T09:52:19.841930+0000","last_became_peered":"2024-09-10T09:52:19.841930+0000","last_unstale":"2024-09-10T09:54:00.844244+0000","last_undegraded":"2024-09-10T09:54:00.844244+0000","last_fullsized":"2024-09-10T09:54:00.844244+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:38:46.358389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.847531+0000","last_change":"2024-09-10T09:52:31.370961+0000","last_active":"2024-09-10T09:54:00.847531+0000","last_peered":"2024-09-10T09:54:00.847531+0000","last_clean":"2024-09-10T09:54:00.847531+0000","last_became_active":"2024-09-10T09:52:31.370545+0000","last_became_peered":"2024-09-10T09:52:31.370545+0000","last_unstale":"2024-09-10T09:54:00.847531+0000","last_undegraded":"2024-09-10T09:54:00.847531+0000","last_fullsized":"2024-09-10T09:54:00.847531+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:17.461447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"96'6","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.685895+0000","last_change":"2024-09-10T09:52:22.313215+0000","last_active":"2024-09-10T09:54:29.685895+0000","last_peered":"2024-09-10T09:54:29.685895+0000","last_clean":"2024-09-10T09:54:29.685895+0000","last_became_active":"2024-09-10T09:52:22.312918+0000","last_became_peered":"2024-09-10T09:52:22.312918+0000","last_unstale":"2024-09-10T09:54:29.685895+0000","last_undegraded":"2024-09-10T09:54:29.685895+0000","last_fullsized":"2024-09-10T09:54:29.685895+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:48:09.568004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"96'37","reported_seq":337,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:55:12.051648+0000","last_change":"2024-09-10T09:53:01.226146+0000","last_active":"2024-09-10T09:55:12.051648+0000","last_peered":"2024-09-10T09:55:12.051648+0000","last_clean":"2024-09-10T09:55:12.051648+0000","last_became_active":"2024-09-10T09:53:01.218622+0000","last_became_peered":"2024-09-10T09:53:01.218622+0000","last_unstale":"2024-09-10T09:55:12.051648+0000","last_undegraded":"2024-09-10T09:55:12.051648+0000","last_fullsized":"2024-09-10T09:55:12.051648+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:43:11.927029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63251,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":241,"num_read_kb":182,"num_write":39,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":3844,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":16,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.841392+0000","last_change":"2024-09-10T09:53:00.800650+0000","last_active":"2024-09-10T09:54:00.841392+0000","last_peered":"2024-09-10T09:54:00.841392+0000","last_clean":"2024-09-10T09:54:00.841392+0000","last_became_active":"2024-09-10T09:53:00.800104+0000","last_became_peered":"2024-09-10T09:53:00.800104+0000","last_unstale":"2024-09-10T09:54:00.841392+0000","last_undegraded":"2024-09-10T09:54:00.841392+0000","last_fullsized":"2024-09-10T09:54:00.841392+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:41:16.022884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.14","version":"96'11","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.690121+0000","last_change":"2024-09-10T09:52:22.317174+0000","last_active":"2024-09-10T09:54:29.690121+0000","last_peered":"2024-09-10T09:54:29.690121+0000","last_clean":"2024-09-10T09:54:29.690121+0000","last_became_active":"2024-09-10T09:52:22.316851+0000","last_became_peered":"2024-09-10T09:52:22.316851+0000","last_unstale":"2024-09-10T09:54:29.690121+0000","last_undegraded":"2024-09-10T09:54:29.690121+0000","last_fullsized":"2024-09-10T09:54:29.690121+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:55:18.742393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"94'5","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.847731+0000","last_change":"2024-09-10T09:52:19.912093+0000","last_active":"2024-09-10T09:54:00.847731+0000","last_peered":"2024-09-10T09:54:00.847731+0000","last_clean":"2024-09-10T09:54:00.847731+0000","last_became_active":"2024-09-10T09:52:19.911611+0000","last_became_peered":"2024-09-10T09:52:19.911611+0000","last_unstale":"2024-09-10T09:54:00.847731+0000","last_undegraded":"2024-09-10T09:54:00.847731+0000","last_fullsized":"2024-09-10T09:54:00.847731+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:01:10.859310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.951658+0000","last_change":"2024-09-10T09:52:31.362884+0000","last_active":"2024-09-10T09:54:00.951658+0000","last_peered":"2024-09-10T09:54:00.951658+0000","last_clean":"2024-09-10T09:54:00.951658+0000","last_became_active":"2024-09-10T09:52:31.362489+0000","last_became_peered":"2024-09-10T09:52:31.362489+0000","last_unstale":"2024-09-10T09:54:00.951658+0000","last_undegraded":"2024-09-10T09:54:00.951658+0000","last_fullsized":"2024-09-10T09:54:00.951658+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:02:33.084795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"96'9","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.692830+0000","last_change":"2024-09-10T09:52:22.325871+0000","last_active":"2024-09-10T09:54:29.692830+0000","last_peered":"2024-09-10T09:54:29.692830+0000","last_clean":"2024-09-10T09:54:29.692830+0000","last_became_active":"2024-09-10T09:52:22.324980+0000","last_became_peered":"2024-09-10T09:52:22.324980+0000","last_unstale":"2024-09-10T09:54:29.692830+0000","last_undegraded":"2024-09-10T09:54:29.692830+0000","last_fullsized":"2024-09-10T09:54:29.692830+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:31:55.713444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"94'4","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.059691+0000","last_change":"2024-09-10T09:52:19.839491+0000","last_active":"2024-09-10T09:54:01.059691+0000","last_peered":"2024-09-10T09:54:01.059691+0000","last_clean":"2024-09-10T09:54:01.059691+0000","last_became_active":"2024-09-10T09:52:19.839092+0000","last_became_peered":"2024-09-10T09:52:19.839092+0000","last_unstale":"2024-09-10T09:54:01.059691+0000","last_undegraded":"2024-09-10T09:54:01.059691+0000","last_fullsized":"2024-09-10T09:54:01.059691+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:42:13.453281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.843285+0000","last_change":"2024-09-10T09:52:31.371346+0000","last_active":"2024-09-10T09:54:00.843285+0000","last_peered":"2024-09-10T09:54:00.843285+0000","last_clean":"2024-09-10T09:54:00.843285+0000","last_became_active":"2024-09-10T09:52:31.370929+0000","last_became_peered":"2024-09-10T09:52:31.370929+0000","last_unstale":"2024-09-10T09:54:00.843285+0000","last_undegraded":"2024-09-10T09:54:00.843285+0000","last_fullsized":"2024-09-10T09:54:00.843285+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:44:22.925082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"96'10","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.694062+0000","last_change":"2024-09-10T09:52:22.317453+0000","last_active":"2024-09-10T09:54:29.694062+0000","last_peered":"2024-09-10T09:54:29.694062+0000","last_clean":"2024-09-10T09:54:29.694062+0000","last_became_active":"2024-09-10T09:52:22.315556+0000","last_became_peered":"2024-09-10T09:52:22.315556+0000","last_unstale":"2024-09-10T09:54:29.694062+0000","last_undegraded":"2024-09-10T09:54:29.694062+0000","last_fullsized":"2024-09-10T09:54:29.694062+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:56:05.860275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"96'5","reported_seq":280,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:55:12.051699+0000","last_change":"2024-09-10T09:52:19.835985+0000","last_active":"2024-09-10T09:55:12.051699+0000","last_peered":"2024-09-10T09:55:12.051699+0000","last_clean":"2024-09-10T09:55:12.051699+0000","last_became_active":"2024-09-10T09:52:19.834835+0000","last_became_peered":"2024-09-10T09:52:19.834835+0000","last_unstale":"2024-09-10T09:55:12.051699+0000","last_undegraded":"2024-09-10T09:55:12.051699+0000","last_fullsized":"2024-09-10T09:55:12.051699+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:09:11.590443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":232,"num_read_kb":232,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.845724+0000","last_change":"2024-09-10T09:52:31.368663+0000","last_active":"2024-09-10T09:54:00.845724+0000","last_peered":"2024-09-10T09:54:00.845724+0000","last_clean":"2024-09-10T09:54:00.845724+0000","last_became_active":"2024-09-10T09:52:31.368004+0000","last_became_peered":"2024-09-10T09:52:31.368004+0000","last_unstale":"2024-09-10T09:54:00.845724+0000","last_undegraded":"2024-09-10T09:54:00.845724+0000","last_fullsized":"2024-09-10T09:54:00.845724+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:56:15.100515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.247917+0000","last_change":"2024-09-10T09:52:31.370992+0000","last_active":"2024-09-10T09:54:01.247917+0000","last_peered":"2024-09-10T09:54:01.247917+0000","last_clean":"2024-09-10T09:54:01.247917+0000","last_became_active":"2024-09-10T09:52:31.370357+0000","last_became_peered":"2024-09-10T09:52:31.370357+0000","last_unstale":"2024-09-10T09:54:01.247917+0000","last_undegraded":"2024-09-10T09:54:01.247917+0000","last_fullsized":"2024-09-10T09:54:01.247917+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:02:49.482532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"96'11","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.706426+0000","last_change":"2024-09-10T09:52:22.319363+0000","last_active":"2024-09-10T09:54:29.706426+0000","last_peered":"2024-09-10T09:54:29.706426+0000","last_clean":"2024-09-10T09:54:29.706426+0000","last_became_active":"2024-09-10T09:52:22.319044+0000","last_became_peered":"2024-09-10T09:52:22.319044+0000","last_unstale":"2024-09-10T09:54:29.706426+0000","last_undegraded":"2024-09-10T09:54:29.706426+0000","last_fullsized":"2024-09-10T09:54:29.706426+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:30:51.328413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"94'3","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.847831+0000","last_change":"2024-09-10T09:52:19.841745+0000","last_active":"2024-09-10T09:54:00.847831+0000","last_peered":"2024-09-10T09:54:00.847831+0000","last_clean":"2024-09-10T09:54:00.847831+0000","last_became_active":"2024-09-10T09:52:19.841404+0000","last_became_peered":"2024-09-10T09:52:19.841404+0000","last_unstale":"2024-09-10T09:54:00.847831+0000","last_undegraded":"2024-09-10T09:54:00.847831+0000","last_fullsized":"2024-09-10T09:54:00.847831+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:44:36.346785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.955521+0000","last_change":"2024-09-10T09:53:01.218412+0000","last_active":"2024-09-10T09:54:00.955521+0000","last_peered":"2024-09-10T09:54:00.955521+0000","last_clean":"2024-09-10T09:54:00.955521+0000","last_became_active":"2024-09-10T09:53:01.217966+0000","last_became_peered":"2024-09-10T09:53:01.217966+0000","last_unstale":"2024-09-10T09:54:00.955521+0000","last_undegraded":"2024-09-10T09:54:00.955521+0000","last_fullsized":"2024-09-10T09:54:00.955521+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:53:25.743913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"96'8","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.703441+0000","last_change":"2024-09-10T09:52:22.316120+0000","last_active":"2024-09-10T09:54:29.703441+0000","last_peered":"2024-09-10T09:54:29.703441+0000","last_clean":"2024-09-10T09:54:29.703441+0000","last_became_active":"2024-09-10T09:52:22.315769+0000","last_became_peered":"2024-09-10T09:52:22.315769+0000","last_unstale":"2024-09-10T09:54:29.703441+0000","last_undegraded":"2024-09-10T09:54:29.703441+0000","last_fullsized":"2024-09-10T09:54:29.703441+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:16:41.866587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"94'3","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.841046+0000","last_change":"2024-09-10T09:52:19.835038+0000","last_active":"2024-09-10T09:54:00.841046+0000","last_peered":"2024-09-10T09:54:00.841046+0000","last_clean":"2024-09-10T09:54:00.841046+0000","last_became_active":"2024-09-10T09:52:19.834329+0000","last_became_peered":"2024-09-10T09:52:19.834329+0000","last_unstale":"2024-09-10T09:54:00.841046+0000","last_undegraded":"2024-09-10T09:54:00.841046+0000","last_fullsized":"2024-09-10T09:54:00.841046+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:58:24.084373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"94'7","reported_seq":56,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.841764+0000","last_change":"2024-09-10T09:53:01.219122+0000","last_active":"2024-09-10T09:54:00.841764+0000","last_peered":"2024-09-10T09:54:00.841764+0000","last_clean":"2024-09-10T09:54:00.841764+0000","last_became_active":"2024-09-10T09:53:01.218599+0000","last_became_peered":"2024-09-10T09:53:01.218599+0000","last_unstale":"2024-09-10T09:54:00.841764+0000","last_undegraded":"2024-09-10T09:54:00.841764+0000","last_fullsized":"2024-09-10T09:54:00.841764+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:31:10.092264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"96'8","reported_seq":19,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.706393+0000","last_change":"2024-09-10T09:54:00.952617+0000","last_active":"2024-09-10T09:54:29.706393+0000","last_peered":"2024-09-10T09:54:29.706393+0000","last_clean":"2024-09-10T09:54:29.706393+0000","last_became_active":"2024-09-10T09:54:00.952202+0000","last_became_peered":"2024-09-10T09:54:00.952202+0000","last_unstale":"2024-09-10T09:54:29.706393+0000","last_undegraded":"2024-09-10T09:54:29.706393+0000","last_fullsized":"2024-09-10T09:54:29.706393+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:35:40.179591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.844169+0000","last_change":"2024-09-10T09:52:31.371612+0000","last_active":"2024-09-10T09:54:00.844169+0000","last_peered":"2024-09-10T09:54:00.844169+0000","last_clean":"2024-09-10T09:54:00.844169+0000","last_became_active":"2024-09-10T09:52:31.371163+0000","last_became_peered":"2024-09-10T09:52:31.371163+0000","last_unstale":"2024-09-10T09:54:00.844169+0000","last_undegraded":"2024-09-10T09:54:00.844169+0000","last_fullsized":"2024-09-10T09:54:00.844169+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:11:43.681524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"96'7","reported_seq":171,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:55:12.052584+0000","last_change":"2024-09-10T09:52:19.841873+0000","last_active":"2024-09-10T09:55:12.052584+0000","last_peered":"2024-09-10T09:55:12.052584+0000","last_clean":"2024-09-10T09:55:12.052584+0000","last_became_active":"2024-09-10T09:52:19.841513+0000","last_became_peered":"2024-09-10T09:52:19.841513+0000","last_unstale":"2024-09-10T09:55:12.052584+0000","last_undegraded":"2024-09-10T09:55:12.052584+0000","last_fullsized":"2024-09-10T09:55:12.052584+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:53:31.854185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1156,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":121,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"96'7","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.697738+0000","last_change":"2024-09-10T09:52:22.314781+0000","last_active":"2024-09-10T09:54:29.697738+0000","last_peered":"2024-09-10T09:54:29.697738+0000","last_clean":"2024-09-10T09:54:29.697738+0000","last_became_active":"2024-09-10T09:52:22.313346+0000","last_became_peered":"2024-09-10T09:52:22.313346+0000","last_unstale":"2024-09-10T09:54:29.697738+0000","last_undegraded":"2024-09-10T09:54:29.697738+0000","last_fullsized":"2024-09-10T09:54:29.697738+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:15:16.921963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.844401+0000","last_change":"2024-09-10T09:52:31.365544+0000","last_active":"2024-09-10T09:54:00.844401+0000","last_peered":"2024-09-10T09:54:00.844401+0000","last_clean":"2024-09-10T09:54:00.844401+0000","last_became_active":"2024-09-10T09:52:31.365151+0000","last_became_peered":"2024-09-10T09:52:31.365151+0000","last_unstale":"2024-09-10T09:54:00.844401+0000","last_undegraded":"2024-09-10T09:54:00.844401+0000","last_fullsized":"2024-09-10T09:54:00.844401+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:36:55.247890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"96'9","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.683641+0000","last_change":"2024-09-10T09:52:22.314117+0000","last_active":"2024-09-10T09:54:29.683641+0000","last_peered":"2024-09-10T09:54:29.683641+0000","last_clean":"2024-09-10T09:54:29.683641+0000","last_became_active":"2024-09-10T09:52:22.313772+0000","last_became_peered":"2024-09-10T09:52:22.313772+0000","last_unstale":"2024-09-10T09:54:29.683641+0000","last_undegraded":"2024-09-10T09:54:29.683641+0000","last_fullsized":"2024-09-10T09:54:29.683641+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:19:37.373598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"96'12","reported_seq":50,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.682477+0000","last_change":"2024-09-10T09:52:22.314362+0000","last_active":"2024-09-10T09:54:29.682477+0000","last_peered":"2024-09-10T09:54:29.682477+0000","last_clean":"2024-09-10T09:54:29.682477+0000","last_became_active":"2024-09-10T09:52:22.314050+0000","last_became_peered":"2024-09-10T09:52:22.314050+0000","last_unstale":"2024-09-10T09:54:29.682477+0000","last_undegraded":"2024-09-10T09:54:29.682477+0000","last_fullsized":"2024-09-10T09:54:29.682477+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:37:07.218656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"96'8","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.699844+0000","last_change":"2024-09-10T09:52:22.322363+0000","last_active":"2024-09-10T09:54:29.699844+0000","last_peered":"2024-09-10T09:54:29.699844+0000","last_clean":"2024-09-10T09:54:29.699844+0000","last_became_active":"2024-09-10T09:52:22.322045+0000","last_became_peered":"2024-09-10T09:52:22.322045+0000","last_unstale":"2024-09-10T09:54:29.699844+0000","last_undegraded":"2024-09-10T09:54:29.699844+0000","last_fullsized":"2024-09-10T09:54:29.699844+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:10:27.918769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.700329+0000","last_change":"2024-09-10T09:52:22.331594+0000","last_active":"2024-09-10T09:54:29.700329+0000","last_peered":"2024-09-10T09:54:29.700329+0000","last_clean":"2024-09-10T09:54:29.700329+0000","last_became_active":"2024-09-10T09:52:22.330585+0000","last_became_peered":"2024-09-10T09:52:22.330585+0000","last_unstale":"2024-09-10T09:54:29.700329+0000","last_undegraded":"2024-09-10T09:54:29.700329+0000","last_fullsized":"2024-09-10T09:54:29.700329+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:45:56.412313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"96'4","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.705719+0000","last_change":"2024-09-10T09:52:22.325753+0000","last_active":"2024-09-10T09:54:29.705719+0000","last_peered":"2024-09-10T09:54:29.705719+0000","last_clean":"2024-09-10T09:54:29.705719+0000","last_became_active":"2024-09-10T09:52:22.325385+0000","last_became_peered":"2024-09-10T09:52:22.325385+0000","last_unstale":"2024-09-10T09:54:29.705719+0000","last_undegraded":"2024-09-10T09:54:29.705719+0000","last_fullsized":"2024-09-10T09:54:29.705719+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:18:28.776789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"96'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.681378+0000","last_change":"2024-09-10T09:52:22.319069+0000","last_active":"2024-09-10T09:54:29.681378+0000","last_peered":"2024-09-10T09:54:29.681378+0000","last_clean":"2024-09-10T09:54:29.681378+0000","last_became_active":"2024-09-10T09:52:22.318667+0000","last_became_peered":"2024-09-10T09:52:22.318667+0000","last_unstale":"2024-09-10T09:54:29.681378+0000","last_undegraded":"2024-09-10T09:54:29.681378+0000","last_fullsized":"2024-09-10T09:54:29.681378+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:21:32.028235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"96'15","reported_seq":53,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.681487+0000","last_change":"2024-09-10T09:52:22.312890+0000","last_active":"2024-09-10T09:54:29.681487+0000","last_peered":"2024-09-10T09:54:29.681487+0000","last_clean":"2024-09-10T09:54:29.681487+0000","last_became_active":"2024-09-10T09:52:22.312608+0000","last_became_peered":"2024-09-10T09:52:22.312608+0000","last_unstale":"2024-09-10T09:54:29.681487+0000","last_undegraded":"2024-09-10T09:54:29.681487+0000","last_fullsized":"2024-09-10T09:54:29.681487+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:07:46.046915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"96'2","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.689884+0000","last_change":"2024-09-10T09:54:01.151385+0000","last_active":"2024-09-10T09:54:29.689884+0000","last_peered":"2024-09-10T09:54:29.689884+0000","last_clean":"2024-09-10T09:54:29.689884+0000","last_became_active":"2024-09-10T09:54:01.150971+0000","last_became_peered":"2024-09-10T09:54:01.150971+0000","last_unstale":"2024-09-10T09:54:29.689884+0000","last_undegraded":"2024-09-10T09:54:29.689884+0000","last_fullsized":"2024-09-10T09:54:29.689884+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:27:10.492071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"96'6","reported_seq":17,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.704565+0000","last_change":"2024-09-10T09:54:00.848273+0000","last_active":"2024-09-10T09:54:29.704565+0000","last_peered":"2024-09-10T09:54:29.704565+0000","last_clean":"2024-09-10T09:54:29.704565+0000","last_became_active":"2024-09-10T09:54:00.847492+0000","last_became_peered":"2024-09-10T09:54:00.847492+0000","last_unstale":"2024-09-10T09:54:29.704565+0000","last_undegraded":"2024-09-10T09:54:29.704565+0000","last_fullsized":"2024-09-10T09:54:29.704565+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:02:24.749291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,3],"acting":[6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.29","version":"96'8","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.696836+0000","last_change":"2024-09-10T09:52:22.313538+0000","last_active":"2024-09-10T09:54:29.696836+0000","last_peered":"2024-09-10T09:54:29.696836+0000","last_clean":"2024-09-10T09:54:29.696836+0000","last_became_active":"2024-09-10T09:52:22.313196+0000","last_became_peered":"2024-09-10T09:52:22.313196+0000","last_unstale":"2024-09-10T09:54:29.696836+0000","last_undegraded":"2024-09-10T09:54:29.696836+0000","last_fullsized":"2024-09-10T09:54:29.696836+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:53:23.387167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"96'10","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.696632+0000","last_change":"2024-09-10T09:52:22.316518+0000","last_active":"2024-09-10T09:54:29.696632+0000","last_peered":"2024-09-10T09:54:29.696632+0000","last_clean":"2024-09-10T09:54:29.696632+0000","last_became_active":"2024-09-10T09:52:22.316193+0000","last_became_peered":"2024-09-10T09:52:22.316193+0000","last_unstale":"2024-09-10T09:54:29.696632+0000","last_undegraded":"2024-09-10T09:54:29.696632+0000","last_fullsized":"2024-09-10T09:54:29.696632+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:49:55.903928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"96'11","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.706245+0000","last_change":"2024-09-10T09:52:22.329163+0000","last_active":"2024-09-10T09:54:29.706245+0000","last_peered":"2024-09-10T09:54:29.706245+0000","last_clean":"2024-09-10T09:54:29.706245+0000","last_became_active":"2024-09-10T09:52:22.328453+0000","last_became_peered":"2024-09-10T09:52:22.328453+0000","last_unstale":"2024-09-10T09:54:29.706245+0000","last_undegraded":"2024-09-10T09:54:29.706245+0000","last_fullsized":"2024-09-10T09:54:29.706245+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:53:33.286055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"96'8","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.681766+0000","last_change":"2024-09-10T09:52:22.314302+0000","last_active":"2024-09-10T09:54:29.681766+0000","last_peered":"2024-09-10T09:54:29.681766+0000","last_clean":"2024-09-10T09:54:29.681766+0000","last_became_active":"2024-09-10T09:52:22.313992+0000","last_became_peered":"2024-09-10T09:52:22.313992+0000","last_unstale":"2024-09-10T09:54:29.681766+0000","last_undegraded":"2024-09-10T09:54:29.681766+0000","last_fullsized":"2024-09-10T09:54:29.681766+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:07:35.764015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"96'7","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.688705+0000","last_change":"2024-09-10T09:52:22.316400+0000","last_active":"2024-09-10T09:54:29.688705+0000","last_peered":"2024-09-10T09:54:29.688705+0000","last_clean":"2024-09-10T09:54:29.688705+0000","last_became_active":"2024-09-10T09:52:22.316110+0000","last_became_peered":"2024-09-10T09:52:22.316110+0000","last_unstale":"2024-09-10T09:54:29.688705+0000","last_undegraded":"2024-09-10T09:54:29.688705+0000","last_fullsized":"2024-09-10T09:54:29.688705+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:17:26.528899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"96'10","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.704074+0000","last_change":"2024-09-10T09:52:22.317914+0000","last_active":"2024-09-10T09:54:29.704074+0000","last_peered":"2024-09-10T09:54:29.704074+0000","last_clean":"2024-09-10T09:54:29.704074+0000","last_became_active":"2024-09-10T09:52:22.317575+0000","last_became_peered":"2024-09-10T09:52:22.317575+0000","last_unstale":"2024-09-10T09:54:29.704074+0000","last_undegraded":"2024-09-10T09:54:29.704074+0000","last_fullsized":"2024-09-10T09:54:29.704074+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:17:51.446233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"96'8","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.697848+0000","last_change":"2024-09-10T09:52:22.327749+0000","last_active":"2024-09-10T09:54:29.697848+0000","last_peered":"2024-09-10T09:54:29.697848+0000","last_clean":"2024-09-10T09:54:29.697848+0000","last_became_active":"2024-09-10T09:52:22.327461+0000","last_became_peered":"2024-09-10T09:52:22.327461+0000","last_unstale":"2024-09-10T09:54:29.697848+0000","last_undegraded":"2024-09-10T09:54:29.697848+0000","last_fullsized":"2024-09-10T09:54:29.697848+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:54.208064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"96'9","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.696695+0000","last_change":"2024-09-10T09:52:22.326982+0000","last_active":"2024-09-10T09:54:29.696695+0000","last_peered":"2024-09-10T09:54:29.696695+0000","last_clean":"2024-09-10T09:54:29.696695+0000","last_became_active":"2024-09-10T09:52:22.326635+0000","last_became_peered":"2024-09-10T09:52:22.326635+0000","last_unstale":"2024-09-10T09:54:29.696695+0000","last_undegraded":"2024-09-10T09:54:29.696695+0000","last_fullsized":"2024-09-10T09:54:29.696695+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:50:54.835899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.695757+0000","last_change":"2024-09-10T09:52:22.323410+0000","last_active":"2024-09-10T09:54:29.695757+0000","last_peered":"2024-09-10T09:54:29.695757+0000","last_clean":"2024-09-10T09:54:29.695757+0000","last_became_active":"2024-09-10T09:52:22.323116+0000","last_became_peered":"2024-09-10T09:52:22.323116+0000","last_unstale":"2024-09-10T09:54:29.695757+0000","last_undegraded":"2024-09-10T09:54:29.695757+0000","last_fullsized":"2024-09-10T09:54:29.695757+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:09:15.115162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.680224+0000","last_change":"2024-09-10T09:52:22.324908+0000","last_active":"2024-09-10T09:54:29.680224+0000","last_peered":"2024-09-10T09:54:29.680224+0000","last_clean":"2024-09-10T09:54:29.680224+0000","last_became_active":"2024-09-10T09:52:22.324449+0000","last_became_peered":"2024-09-10T09:52:22.324449+0000","last_unstale":"2024-09-10T09:54:29.680224+0000","last_undegraded":"2024-09-10T09:54:29.680224+0000","last_fullsized":"2024-09-10T09:54:29.680224+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:43:24.535721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"96'10","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.700039+0000","last_change":"2024-09-10T09:52:22.322157+0000","last_active":"2024-09-10T09:54:29.700039+0000","last_peered":"2024-09-10T09:54:29.700039+0000","last_clean":"2024-09-10T09:54:29.700039+0000","last_became_active":"2024-09-10T09:52:22.321856+0000","last_became_peered":"2024-09-10T09:52:22.321856+0000","last_unstale":"2024-09-10T09:54:29.700039+0000","last_undegraded":"2024-09-10T09:54:29.700039+0000","last_fullsized":"2024-09-10T09:54:29.700039+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:36:38.395370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"96'7","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.703464+0000","last_change":"2024-09-10T09:53:00.800337+0000","last_active":"2024-09-10T09:54:29.703464+0000","last_peered":"2024-09-10T09:54:29.703464+0000","last_clean":"2024-09-10T09:54:29.703464+0000","last_became_active":"2024-09-10T09:53:00.799863+0000","last_became_peered":"2024-09-10T09:53:00.799863+0000","last_unstale":"2024-09-10T09:54:29.703464+0000","last_undegraded":"2024-09-10T09:54:29.703464+0000","last_fullsized":"2024-09-10T09:54:29.703464+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:18.017672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.35","version":"96'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.693580+0000","last_change":"2024-09-10T09:52:22.310735+0000","last_active":"2024-09-10T09:54:29.693580+0000","last_peered":"2024-09-10T09:54:29.693580+0000","last_clean":"2024-09-10T09:54:29.693580+0000","last_became_active":"2024-09-10T09:52:22.310376+0000","last_became_peered":"2024-09-10T09:52:22.310376+0000","last_unstale":"2024-09-10T09:54:29.693580+0000","last_undegraded":"2024-09-10T09:54:29.693580+0000","last_fullsized":"2024-09-10T09:54:29.693580+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:38:08.637588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"96'16","reported_seq":55,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.685458+0000","last_change":"2024-09-10T09:52:22.321113+0000","last_active":"2024-09-10T09:54:29.685458+0000","last_peered":"2024-09-10T09:54:29.685458+0000","last_clean":"2024-09-10T09:54:29.685458+0000","last_became_active":"2024-09-10T09:52:22.320767+0000","last_became_peered":"2024-09-10T09:52:22.320767+0000","last_unstale":"2024-09-10T09:54:29.685458+0000","last_undegraded":"2024-09-10T09:54:29.685458+0000","last_fullsized":"2024-09-10T09:54:29.685458+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:37.549322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"96'11","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.687162+0000","last_change":"2024-09-10T09:52:22.318634+0000","last_active":"2024-09-10T09:54:29.687162+0000","last_peered":"2024-09-10T09:54:29.687162+0000","last_clean":"2024-09-10T09:54:29.687162+0000","last_became_active":"2024-09-10T09:52:22.318294+0000","last_became_peered":"2024-09-10T09:52:22.318294+0000","last_unstale":"2024-09-10T09:54:29.687162+0000","last_undegraded":"2024-09-10T09:54:29.687162+0000","last_fullsized":"2024-09-10T09:54:29.687162+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:19:31.262055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"96'9","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.698725+0000","last_change":"2024-09-10T09:52:22.324567+0000","last_active":"2024-09-10T09:54:29.698725+0000","last_peered":"2024-09-10T09:54:29.698725+0000","last_clean":"2024-09-10T09:54:29.698725+0000","last_became_active":"2024-09-10T09:52:22.321803+0000","last_became_peered":"2024-09-10T09:52:22.321803+0000","last_unstale":"2024-09-10T09:54:29.698725+0000","last_undegraded":"2024-09-10T09:54:29.698725+0000","last_fullsized":"2024-09-10T09:54:29.698725+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:08:11.099428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"96'5","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.699735+0000","last_change":"2024-09-10T09:52:22.318163+0000","last_active":"2024-09-10T09:54:29.699735+0000","last_peered":"2024-09-10T09:54:29.699735+0000","last_clean":"2024-09-10T09:54:29.699735+0000","last_became_active":"2024-09-10T09:52:22.317808+0000","last_became_peered":"2024-09-10T09:52:22.317808+0000","last_unstale":"2024-09-10T09:54:29.699735+0000","last_undegraded":"2024-09-10T09:54:29.699735+0000","last_fullsized":"2024-09-10T09:54:29.699735+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:19.214291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"96'6","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.704960+0000","last_change":"2024-09-10T09:52:22.323035+0000","last_active":"2024-09-10T09:54:29.704960+0000","last_peered":"2024-09-10T09:54:29.704960+0000","last_clean":"2024-09-10T09:54:29.704960+0000","last_became_active":"2024-09-10T09:52:22.322679+0000","last_became_peered":"2024-09-10T09:52:22.322679+0000","last_unstale":"2024-09-10T09:54:29.704960+0000","last_undegraded":"2024-09-10T09:54:29.704960+0000","last_fullsized":"2024-09-10T09:54:29.704960+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:23:13.762612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"96'4","reported_seq":40,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.700111+0000","last_change":"2024-09-10T09:52:22.315887+0000","last_active":"2024-09-10T09:54:29.700111+0000","last_peered":"2024-09-10T09:54:29.700111+0000","last_clean":"2024-09-10T09:54:29.700111+0000","last_became_active":"2024-09-10T09:52:22.315569+0000","last_became_peered":"2024-09-10T09:52:22.315569+0000","last_unstale":"2024-09-10T09:54:29.700111+0000","last_undegraded":"2024-09-10T09:54:29.700111+0000","last_fullsized":"2024-09-10T09:54:29.700111+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:46:57.094320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"96'8","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.676312+0000","last_change":"2024-09-10T09:52:22.314060+0000","last_active":"2024-09-10T09:54:29.676312+0000","last_peered":"2024-09-10T09:54:29.676312+0000","last_clean":"2024-09-10T09:54:29.676312+0000","last_became_active":"2024-09-10T09:52:22.313572+0000","last_became_peered":"2024-09-10T09:52:22.313572+0000","last_unstale":"2024-09-10T09:54:29.676312+0000","last_undegraded":"2024-09-10T09:54:29.676312+0000","last_fullsized":"2024-09-10T09:54:29.676312+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:11:17.158625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.692163+0000","last_change":"2024-09-10T09:52:22.327233+0000","last_active":"2024-09-10T09:54:29.692163+0000","last_peered":"2024-09-10T09:54:29.692163+0000","last_clean":"2024-09-10T09:54:29.692163+0000","last_became_active":"2024-09-10T09:52:22.326945+0000","last_became_peered":"2024-09-10T09:52:22.326945+0000","last_unstale":"2024-09-10T09:54:29.692163+0000","last_undegraded":"2024-09-10T09:54:29.692163+0000","last_fullsized":"2024-09-10T09:54:29.692163+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:31:13.490147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"96'4","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.681664+0000","last_change":"2024-09-10T09:52:22.330912+0000","last_active":"2024-09-10T09:54:29.681664+0000","last_peered":"2024-09-10T09:54:29.681664+0000","last_clean":"2024-09-10T09:54:29.681664+0000","last_became_active":"2024-09-10T09:52:22.330149+0000","last_became_peered":"2024-09-10T09:52:22.330149+0000","last_unstale":"2024-09-10T09:54:29.681664+0000","last_undegraded":"2024-09-10T09:54:29.681664+0000","last_fullsized":"2024-09-10T09:54:29.681664+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:10:48.842499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"96'8","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.697891+0000","last_change":"2024-09-10T09:52:22.327035+0000","last_active":"2024-09-10T09:54:29.697891+0000","last_peered":"2024-09-10T09:54:29.697891+0000","last_clean":"2024-09-10T09:54:29.697891+0000","last_became_active":"2024-09-10T09:52:22.326741+0000","last_became_peered":"2024-09-10T09:52:22.326741+0000","last_unstale":"2024-09-10T09:54:29.697891+0000","last_undegraded":"2024-09-10T09:54:29.697891+0000","last_fullsized":"2024-09-10T09:54:29.697891+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:21:05.026614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70741,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1211,"num_read_kb":685,"num_write":259,"num_write_kb":150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":4426,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":213467,"data_compressed":5661,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":223,"ondisk_log_size":223,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282980,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449137,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1073152,"data_stored":554558,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":70,"seq":300647710772,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32372,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712780,"statfs":{"total":95995035648,"available":95961886720,"internally_reserved":0,"allocated":1216512,"data_stored":1204571,"data_compressed":70462,"data_compressed_allocated":94208,"data_compressed_original":688128,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":65,"seq":279172874300,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32256,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712896,"statfs":{"total":95995035648,"available":95962005504,"internally_reserved":0,"allocated":1097728,"data_stored":557996,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135938,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32244,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712908,"statfs":{"total":95995035648,"available":95962017792,"internally_reserved":0,"allocated":1085440,"data_stored":557702,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364870,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28136,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717016,"statfs":{"total":95995035648,"available":95966224384,"internally_reserved":0,"allocated":1138688,"data_stored":1150864,"data_compressed":68575,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528396,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1069056,"data_stored":549715,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790034,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28144,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717008,"statfs":{"total":95995035648,"available":95966216192,"internally_reserved":0,"allocated":1146880,"data_stored":1142404,"data_compressed":68575,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953557,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28144,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717008,"statfs":{"total":95995035648,"available":95966216192,"internally_reserved":0,"allocated":1146880,"data_stored":617574,"data_compressed":34470,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182491,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32240,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712912,"statfs":{"total":95995035648,"available":95962021888,"internally_reserved":0,"allocated":1081344,"data_stored":549112,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411425,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36404,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708748,"statfs":{"total":95995035648,"available":95957757952,"internally_reserved":0,"allocated":1085440,"data_stored":551188,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574951,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28140,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717012,"statfs":{"total":95995035648,"available":95966220288,"internally_reserved":0,"allocated":1142784,"data_stored":614979,"data_compressed":34470,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738476,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1093632,"data_stored":550860,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34340,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34340,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34340,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35992,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35992,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35992,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35992,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35992,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":66297,"data_compressed":1887,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":64796,"data_compressed":1887,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":65521,"data_compressed":1887,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T09:55:18.991 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:55:19.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:55:18 smithi120 ceph-mon[34770]: pgmap v395: 129 pgs: 129 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-10T09:55:19.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:55:18 smithi077 ceph-mon[30319]: pgmap v395: 129 pgs: 129 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-10T09:55:19.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:55:18 smithi134 ceph-mon[35674]: pgmap v395: 129 pgs: 129 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-10T09:55:19.393 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:55:19.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:19.936+0000 7f1320200640 1 -- 172.21.15.77:0/1204142175 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1318106f10 msgr2=0x7f131810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:19.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:19.936+0000 7f1320200640 1 --2- 172.21.15.77:0/1204142175 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1318106f10 0x7f131810ea90 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f1310009900 tx=0x7f131002a3f0 comp rx=0 tx=0).stop 2024-09-10T09:55:19.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:19.936+0000 7f1320200640 1 -- 172.21.15.77:0/1204142175 shutdown_connections 2024-09-10T09:55:19.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:19.936+0000 7f1320200640 1 --2- 172.21.15.77:0/1204142175 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1318106f10 0x7f131810ea90 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:19.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:19.936+0000 7f1320200640 1 --2- 172.21.15.77:0/1204142175 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1318102790 0x7f13181067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:19.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:19.936+0000 7f1320200640 1 --2- 172.21.15.77:0/1204142175 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1318101de0 0x7f13181021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:19.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:19.936+0000 7f1320200640 1 -- 172.21.15.77:0/1204142175 >> 172.21.15.77:0/1204142175 conn(0x7f13180fd8c0 msgr2=0x7f13180ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:19.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:19.936+0000 7f1320200640 1 -- 172.21.15.77:0/1204142175 shutdown_connections 2024-09-10T09:55:19.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:19.937+0000 7f1320200640 1 -- 172.21.15.77:0/1204142175 wait complete. 2024-09-10T09:55:19.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:19.937+0000 7f1320200640 1 Processor -- start 2024-09-10T09:55:19.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:19.938+0000 7f1320200640 1 -- start start 2024-09-10T09:55:19.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:19.938+0000 7f1320200640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1318101de0 0x7f1318198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:19.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:19.938+0000 7f1320200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1318102790 0x7f13181989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:19.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:19.938+0000 7f1320200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1318106f10 0x7f131819cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:19.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:19.938+0000 7f1320200640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1318111fc0 con 0x7f1318102790 2024-09-10T09:55:19.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:19.938+0000 7f1320200640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1318111e40 con 0x7f1318101de0 2024-09-10T09:55:19.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:19.938+0000 7f1320200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1318112140 con 0x7f1318106f10 2024-09-10T09:55:19.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:19.938+0000 7f131ce00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1318102790 0x7f13181989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:19.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:19.938+0000 7f131ce00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1318102790 0x7f13181989c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52770/0 (socket says 172.21.15.77:52770) 2024-09-10T09:55:19.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:19.938+0000 7f131ce00640 1 -- 172.21.15.77:0/3432306881 learned_addr learned my addr 172.21.15.77:0/3432306881 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:55:19.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:19.938+0000 7f131e200640 1 --2- 172.21.15.77:0/3432306881 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1318106f10 0x7f131819cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:19.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:19.938+0000 7f131d800640 1 --2- 172.21.15.77:0/3432306881 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1318101de0 0x7f1318198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:19.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:19.939+0000 7f131ce00640 1 -- 172.21.15.77:0/3432306881 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1318106f10 msgr2=0x7f131819cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:19.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:19.939+0000 7f131ce00640 1 --2- 172.21.15.77:0/3432306881 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1318106f10 0x7f131819cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:19.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:19.939+0000 7f131ce00640 1 -- 172.21.15.77:0/3432306881 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1318101de0 msgr2=0x7f1318198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:19.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:19.939+0000 7f131ce00640 1 --2- 172.21.15.77:0/3432306881 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1318101de0 0x7f1318198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:19.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:19.939+0000 7f131ce00640 1 -- 172.21.15.77:0/3432306881 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f131819d430 con 0x7f1318102790 2024-09-10T09:55:19.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:19.939+0000 7f131ce00640 1 --2- 172.21.15.77:0/3432306881 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1318102790 0x7f13181989c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f130400e9f0 tx=0x7f130400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:19.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:19.939+0000 7f1316000640 1 -- 172.21.15.77:0/3432306881 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f130400cd80 con 0x7f1318102790 2024-09-10T09:55:19.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:19.939+0000 7f1320200640 1 -- 172.21.15.77:0/3432306881 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f131819d720 con 0x7f1318102790 2024-09-10T09:55:19.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:19.939+0000 7f1316000640 1 -- 172.21.15.77:0/3432306881 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1304010910 con 0x7f1318102790 2024-09-10T09:55:19.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:19.939+0000 7f1316000640 1 -- 172.21.15.77:0/3432306881 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13040188f0 con 0x7f1318102790 2024-09-10T09:55:19.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:19.939+0000 7f1320200640 1 -- 172.21.15.77:0/3432306881 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13181a5000 con 0x7f1318102790 2024-09-10T09:55:19.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:19.941+0000 7f1320200640 1 -- 172.21.15.77:0/3432306881 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13181021c0 con 0x7f1318102790 2024-09-10T09:55:19.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:19.943+0000 7f1316000640 1 -- 172.21.15.77:0/3432306881 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1304010ac0 con 0x7f1318102790 2024-09-10T09:55:19.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:19.944+0000 7f1316000640 1 --2- 172.21.15.77:0/3432306881 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f12fc0a2c00 0x7f12fc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:19.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:19.945+0000 7f1316000640 1 -- 172.21.15.77:0/3432306881 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f13040cf710 con 0x7f1318102790 2024-09-10T09:55:19.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:19.945+0000 7f131d800640 1 --2- 172.21.15.77:0/3432306881 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f12fc0a2c00 0x7f12fc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:19.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:19.945+0000 7f131d800640 1 --2- 172.21.15.77:0/3432306881 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f12fc0a2c00 0x7f12fc0a50c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1308005fd0 tx=0x7f1308009680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:19.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:19.949+0000 7f1316000640 1 -- 172.21.15.77:0/3432306881 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f130409bbd0 con 0x7f1318102790 2024-09-10T09:55:20.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:20.159+0000 7f1320200640 1 -- 172.21.15.77:0/3432306881 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f131810a0f0 con 0x7f1318102790 2024-09-10T09:55:20.160 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:55:19 smithi077 ceph-mon[30319]: from='client.15231 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T09:55:20.160 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:55:19 smithi077 ceph-mon[30319]: from='client.15237 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T09:55:20.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:20.160+0000 7f1316000640 1 -- 172.21.15.77:0/3432306881 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11989 (secure 0 0 0) 0x7f130409fed0 con 0x7f1318102790 2024-09-10T09:55:20.161 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:55:20.161 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":29,"btime":"2024-09-10T09:55:13:863715+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3159289675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3159289675},{"type":"v1","addr":"172.21.15.77:6835","nonce":3159289675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1951962341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1951962341},{"type":"v1","addr":"172.21.15.77:6837","nonce":1951962341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:52:23.368252+0000","modified":"2024-09-10T09:55:13.786975+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24563,"mds_1":24524,"mds_2":24520,"mds_3":24502,"mds_4":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14487":{"gid":14487,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/4291121888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":4291121888},{"type":"v1","addr":"172.21.15.77:6839","nonce":4291121888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14499":{"gid":14499,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6841/445284194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":445284194},{"type":"v1","addr":"172.21.15.77:6841","nonce":445284194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1005054423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1005054423},{"type":"v1","addr":"172.21.15.134:6833","nonce":1005054423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6833/3716307744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":3716307744},{"type":"v1","addr":"172.21.15.120:6833","nonce":3716307744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/1319892544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1319892544},{"type":"v1","addr":"172.21.15.120:6835","nonce":1319892544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.134:6835/3293933699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3293933699},{"type":"v1","addr":"172.21.15.134:6835","nonce":3293933699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.134:6837/2461497091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2461497091},{"type":"v1","addr":"172.21.15.134:6837","nonce":2461497091}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":20,"addr":"172.21.15.134:6839/1965117680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1965117680},{"type":"v1","addr":"172.21.15.134:6839","nonce":1965117680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":20,"addr":"172.21.15.120:6837/3033280709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3033280709},{"type":"v1","addr":"172.21.15.120:6837","nonce":3033280709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.120:6839/1777710157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1777710157},{"type":"v1","addr":"172.21.15.120:6839","nonce":1777710157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24563,"qdb_cluster":[24496,24502,24520,24524,24563]},"id":1}]} 2024-09-10T09:55:20.161 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 29 2024-09-10T09:55:20.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:20.162+0000 7f1320200640 1 -- 172.21.15.77:0/3432306881 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f12fc0a2c00 msgr2=0x7f12fc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:20.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:20.162+0000 7f1320200640 1 --2- 172.21.15.77:0/3432306881 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f12fc0a2c00 0x7f12fc0a50c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1308005fd0 tx=0x7f1308009680 comp rx=0 tx=0).stop 2024-09-10T09:55:20.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:20.162+0000 7f1320200640 1 -- 172.21.15.77:0/3432306881 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1318102790 msgr2=0x7f13181989c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:20.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:20.162+0000 7f1320200640 1 --2- 172.21.15.77:0/3432306881 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1318102790 0x7f13181989c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f130400e9f0 tx=0x7f130400eeb0 comp rx=0 tx=0).stop 2024-09-10T09:55:20.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:20.162+0000 7f1320200640 1 -- 172.21.15.77:0/3432306881 shutdown_connections 2024-09-10T09:55:20.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:20.162+0000 7f1320200640 1 --2- 172.21.15.77:0/3432306881 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f12fc0a2c00 0x7f12fc0a50c0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:20.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:20.162+0000 7f1320200640 1 --2- 172.21.15.77:0/3432306881 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1318106f10 0x7f131819cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:20.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:20.162+0000 7f1320200640 1 --2- 172.21.15.77:0/3432306881 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1318102790 0x7f13181989c0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:20.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:20.162+0000 7f1320200640 1 --2- 172.21.15.77:0/3432306881 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1318101de0 0x7f1318198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:20.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:20.162+0000 7f1320200640 1 -- 172.21.15.77:0/3432306881 >> 172.21.15.77:0/3432306881 conn(0x7f13180fd8c0 msgr2=0x7f13180febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:20.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:20.163+0000 7f1320200640 1 -- 172.21.15.77:0/3432306881 shutdown_connections 2024-09-10T09:55:20.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:20.163+0000 7f1320200640 1 -- 172.21.15.77:0/3432306881 wait complete. 2024-09-10T09:55:20.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:55:19 smithi120 ceph-mon[34770]: from='client.15231 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T09:55:20.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:55:19 smithi120 ceph-mon[34770]: from='client.15237 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T09:55:20.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:55:19 smithi134 ceph-mon[35674]: from='client.15231 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T09:55:20.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:55:19 smithi134 ceph-mon[35674]: from='client.15237 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T09:55:21.045 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T09:55:21.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:55:20 smithi120 ceph-mon[34770]: from='client.15243 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T09:55:21.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:55:20 smithi120 ceph-mon[34770]: pgmap v396: 129 pgs: 129 active+clean; 651 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 938 B/s wr, 13 op/s 2024-09-10T09:55:21.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:55:20 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/3432306881' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:55:21.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:55:20 smithi077 ceph-mon[30319]: from='client.15243 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T09:55:21.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:55:20 smithi077 ceph-mon[30319]: pgmap v396: 129 pgs: 129 active+clean; 651 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 938 B/s wr, 13 op/s 2024-09-10T09:55:21.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:55:20 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/3432306881' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:55:21.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:55:20 smithi134 ceph-mon[35674]: from='client.15243 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T09:55:21.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:55:20 smithi134 ceph-mon[35674]: pgmap v396: 129 pgs: 129 active+clean; 651 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 938 B/s wr, 13 op/s 2024-09-10T09:55:21.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:55:20 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/3432306881' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:55:21.411 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:55:21.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:21.953+0000 7fafd1c00640 1 -- 172.21.15.77:0/1508341901 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fafcc103e30 msgr2=0x7fafcc104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:21.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:21.953+0000 7fafd1c00640 1 --2- 172.21.15.77:0/1508341901 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fafcc103e30 0x7fafcc104210 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fafbc009900 tx=0x7fafbc02a400 comp rx=0 tx=0).stop 2024-09-10T09:55:21.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:21.953+0000 7fafd1c00640 1 -- 172.21.15.77:0/1508341901 shutdown_connections 2024-09-10T09:55:21.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:21.953+0000 7fafd1c00640 1 --2- 172.21.15.77:0/1508341901 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fafcc108f60 0x7fafcc110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:21.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:21.953+0000 7fafd1c00640 1 --2- 172.21.15.77:0/1508341901 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fafcc1047e0 0x7fafcc108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:21.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:21.953+0000 7fafd1c00640 1 --2- 172.21.15.77:0/1508341901 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fafcc103e30 0x7fafcc104210 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:21.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:21.953+0000 7fafd1c00640 1 -- 172.21.15.77:0/1508341901 >> 172.21.15.77:0/1508341901 conn(0x7fafcc0ff910 msgr2=0x7fafcc101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:21.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:21.953+0000 7fafd1c00640 1 -- 172.21.15.77:0/1508341901 shutdown_connections 2024-09-10T09:55:21.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:21.954+0000 7fafd1c00640 1 -- 172.21.15.77:0/1508341901 wait complete. 2024-09-10T09:55:21.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:21.954+0000 7fafd1c00640 1 Processor -- start 2024-09-10T09:55:21.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:21.955+0000 7fafd1c00640 1 -- start start 2024-09-10T09:55:21.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:21.955+0000 7fafd1c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fafcc103e30 0x7fafcc19a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:21.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:21.955+0000 7fafd1c00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fafcc1047e0 0x7fafcc19a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:21.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:21.955+0000 7fafd1c00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fafcc108f60 0x7fafcc19ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:21.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:21.955+0000 7fafd1c00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fafcc113fc0 con 0x7fafcc1047e0 2024-09-10T09:55:21.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:21.955+0000 7fafd1c00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fafcc113e40 con 0x7fafcc108f60 2024-09-10T09:55:21.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:21.955+0000 7fafd1c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fafcc114140 con 0x7fafcc103e30 2024-09-10T09:55:21.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:21.955+0000 7fafcbe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fafcc103e30 0x7fafcc19a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:21.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:21.955+0000 7fafcb400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fafcc1047e0 0x7fafcc19a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:21.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:21.955+0000 7fafcbe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fafcc103e30 0x7fafcc19a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:51646/0 (socket says 172.21.15.77:51646) 2024-09-10T09:55:21.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:21.955+0000 7fafcbe00640 1 -- 172.21.15.77:0/563724608 learned_addr learned my addr 172.21.15.77:0/563724608 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:55:21.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:21.955+0000 7fafd1200640 1 --2- 172.21.15.77:0/563724608 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fafcc108f60 0x7fafcc19ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:21.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:21.956+0000 7fafcbe00640 1 -- 172.21.15.77:0/563724608 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fafcc108f60 msgr2=0x7fafcc19ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:21.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:21.956+0000 7fafcbe00640 1 --2- 172.21.15.77:0/563724608 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fafcc108f60 0x7fafcc19ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:21.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:21.956+0000 7fafcbe00640 1 -- 172.21.15.77:0/563724608 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fafcc1047e0 msgr2=0x7fafcc19a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:21.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:21.956+0000 7fafcbe00640 1 --2- 172.21.15.77:0/563724608 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fafcc1047e0 0x7fafcc19a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:21.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:21.956+0000 7fafcbe00640 1 -- 172.21.15.77:0/563724608 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafcc19f430 con 0x7fafcc103e30 2024-09-10T09:55:21.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:21.956+0000 7fafcb400640 1 --2- 172.21.15.77:0/563724608 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fafcc1047e0 0x7fafcc19a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:55:21.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:21.956+0000 7fafcbe00640 1 --2- 172.21.15.77:0/563724608 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fafcc103e30 0x7fafcc19a480 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fafbc006010 tx=0x7fafbc0042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:21.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:21.956+0000 7fafc8c00640 1 -- 172.21.15.77:0/563724608 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafbc03f030 con 0x7fafcc103e30 2024-09-10T09:55:21.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:21.956+0000 7fafc8c00640 1 -- 172.21.15.77:0/563724608 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafbc003ce0 con 0x7fafcc103e30 2024-09-10T09:55:21.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:21.956+0000 7fafd1c00640 1 -- 172.21.15.77:0/563724608 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafcc19f6c0 con 0x7fafcc103e30 2024-09-10T09:55:21.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:21.957+0000 7fafc8c00640 1 -- 172.21.15.77:0/563724608 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafbc035070 con 0x7fafcc103e30 2024-09-10T09:55:21.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:21.957+0000 7fafd1c00640 1 -- 172.21.15.77:0/563724608 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafcc1a7000 con 0x7fafcc103e30 2024-09-10T09:55:21.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:21.958+0000 7fafd1c00640 1 -- 172.21.15.77:0/563724608 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafcc105310 con 0x7fafcc103e30 2024-09-10T09:55:21.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:21.959+0000 7fafc8c00640 1 -- 172.21.15.77:0/563724608 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fafbc042030 con 0x7fafcc103e30 2024-09-10T09:55:21.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:21.961+0000 7fafc8c00640 1 --2- 172.21.15.77:0/563724608 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fafb00a2b80 0x7fafb00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:21.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:21.961+0000 7fafc8c00640 1 -- 172.21.15.77:0/563724608 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fafbc0ef500 con 0x7fafcc103e30 2024-09-10T09:55:21.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:21.961+0000 7fafcb400640 1 --2- 172.21.15.77:0/563724608 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fafb00a2b80 0x7fafb00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:21.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:21.961+0000 7fafcb400640 1 --2- 172.21.15.77:0/563724608 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fafb00a2b80 0x7fafb00a5040 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fafcc101e70 tx=0x7fafb800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:21.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:21.965+0000 7fafc8c00640 1 -- 172.21.15.77:0/563724608 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fafbc0bba80 con 0x7fafcc103e30 2024-09-10T09:55:22.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:22.141+0000 7fafd1c00640 1 -- 172.21.15.77:0/563724608 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fafcc10c140 con 0x7fafcc103e30 2024-09-10T09:55:22.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:22.142+0000 7fafc8c00640 1 -- 172.21.15.77:0/563724608 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+24941 (secure 0 0 0) 0x7fafbc0bfd80 con 0x7fafcc103e30 2024-09-10T09:55:22.144 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:55:22.144 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":96,"fsid":"14a55226-6f59-11ef-bcea-c7b262605968","created":"2024-09-10T09:44:07.359542+0000","modified":"2024-09-10T09:54:00.837074+0000","last_up_change":"2024-09-10T09:51:26.731971+0000","last_in_change":"2024-09-10T09:51:03.857873+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:47:24.174660+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T09:52:18.766738+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T09:52:21.284745+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T09:52:29.846594+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8d46f634-114c-4926-afdc-f8fa37e40336","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":1509649079},{"type":"v1","addr":"172.21.15.77:6803","nonce":1509649079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":1509649079},{"type":"v1","addr":"172.21.15.77:6805","nonce":1509649079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":1509649079},{"type":"v1","addr":"172.21.15.77:6809","nonce":1509649079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":1509649079},{"type":"v1","addr":"172.21.15.77:6807","nonce":1509649079}]},"public_addr":"172.21.15.77:6803/1509649079","cluster_addr":"172.21.15.77:6805/1509649079","heartbeat_back_addr":"172.21.15.77:6809/1509649079","heartbeat_front_addr":"172.21.15.77:6807/1509649079","state":["exists","up"]},{"osd":1,"uuid":"b6570764-736a-49e4-877a-1965e5812d1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":4243920698},{"type":"v1","addr":"172.21.15.120:6801","nonce":4243920698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":4243920698},{"type":"v1","addr":"172.21.15.120:6803","nonce":4243920698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":4243920698},{"type":"v1","addr":"172.21.15.120:6807","nonce":4243920698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":4243920698},{"type":"v1","addr":"172.21.15.120:6805","nonce":4243920698}]},"public_addr":"172.21.15.120:6801/4243920698","cluster_addr":"172.21.15.120:6803/4243920698","heartbeat_back_addr":"172.21.15.120:6807/4243920698","heartbeat_front_addr":"172.21.15.120:6805/4243920698","state":["exists","up"]},{"osd":2,"uuid":"5e83fe13-d1ee-483e-a575-e8041ec836f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":3280466823},{"type":"v1","addr":"172.21.15.134:6801","nonce":3280466823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":3280466823},{"type":"v1","addr":"172.21.15.134:6803","nonce":3280466823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":3280466823},{"type":"v1","addr":"172.21.15.134:6807","nonce":3280466823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":3280466823},{"type":"v1","addr":"172.21.15.134:6805","nonce":3280466823}]},"public_addr":"172.21.15.134:6801/3280466823","cluster_addr":"172.21.15.134:6803/3280466823","heartbeat_back_addr":"172.21.15.134:6807/3280466823","heartbeat_front_addr":"172.21.15.134:6805/3280466823","state":["exists","up"]},{"osd":3,"uuid":"5affc6a7-1cae-48fe-b7db-befc2fcafecb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":1629416786},{"type":"v1","addr":"172.21.15.77:6811","nonce":1629416786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":1629416786},{"type":"v1","addr":"172.21.15.77:6813","nonce":1629416786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":1629416786},{"type":"v1","addr":"172.21.15.77:6817","nonce":1629416786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":1629416786},{"type":"v1","addr":"172.21.15.77:6815","nonce":1629416786}]},"public_addr":"172.21.15.77:6811/1629416786","cluster_addr":"172.21.15.77:6813/1629416786","heartbeat_back_addr":"172.21.15.77:6817/1629416786","heartbeat_front_addr":"172.21.15.77:6815/1629416786","state":["exists","up"]},{"osd":4,"uuid":"c826978f-40b0-43ae-adbb-5c93b32a68ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":3162658038},{"type":"v1","addr":"172.21.15.120:6809","nonce":3162658038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":3162658038},{"type":"v1","addr":"172.21.15.120:6811","nonce":3162658038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":3162658038},{"type":"v1","addr":"172.21.15.120:6815","nonce":3162658038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":3162658038},{"type":"v1","addr":"172.21.15.120:6813","nonce":3162658038}]},"public_addr":"172.21.15.120:6809/3162658038","cluster_addr":"172.21.15.120:6811/3162658038","heartbeat_back_addr":"172.21.15.120:6815/3162658038","heartbeat_front_addr":"172.21.15.120:6813/3162658038","state":["exists","up"]},{"osd":5,"uuid":"be51c7fb-2765-46a3-9861-c950f63de148","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":4081444621},{"type":"v1","addr":"172.21.15.134:6809","nonce":4081444621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":4081444621},{"type":"v1","addr":"172.21.15.134:6811","nonce":4081444621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":4081444621},{"type":"v1","addr":"172.21.15.134:6815","nonce":4081444621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":4081444621},{"type":"v1","addr":"172.21.15.134:6813","nonce":4081444621}]},"public_addr":"172.21.15.134:6809/4081444621","cluster_addr":"172.21.15.134:6811/4081444621","heartbeat_back_addr":"172.21.15.134:6815/4081444621","heartbeat_front_addr":"172.21.15.134:6813/4081444621","state":["exists","up"]},{"osd":6,"uuid":"4eb403ab-473b-41cc-803d-18c908f768a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":879565639},{"type":"v1","addr":"172.21.15.77:6819","nonce":879565639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":879565639},{"type":"v1","addr":"172.21.15.77:6821","nonce":879565639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":879565639},{"type":"v1","addr":"172.21.15.77:6825","nonce":879565639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":879565639},{"type":"v1","addr":"172.21.15.77:6823","nonce":879565639}]},"public_addr":"172.21.15.77:6819/879565639","cluster_addr":"172.21.15.77:6821/879565639","heartbeat_back_addr":"172.21.15.77:6825/879565639","heartbeat_front_addr":"172.21.15.77:6823/879565639","state":["exists","up"]},{"osd":7,"uuid":"ba982bd0-c236-456c-9889-61087d79c1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":2688667400},{"type":"v1","addr":"172.21.15.120:6817","nonce":2688667400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":2688667400},{"type":"v1","addr":"172.21.15.120:6819","nonce":2688667400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":2688667400},{"type":"v1","addr":"172.21.15.120:6823","nonce":2688667400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2688667400},{"type":"v1","addr":"172.21.15.120:6821","nonce":2688667400}]},"public_addr":"172.21.15.120:6817/2688667400","cluster_addr":"172.21.15.120:6819/2688667400","heartbeat_back_addr":"172.21.15.120:6823/2688667400","heartbeat_front_addr":"172.21.15.120:6821/2688667400","state":["exists","up"]},{"osd":8,"uuid":"f2e8ca43-57c9-4933-b5f7-73f4d1b3e08b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2081686544},{"type":"v1","addr":"172.21.15.134:6817","nonce":2081686544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2081686544},{"type":"v1","addr":"172.21.15.134:6819","nonce":2081686544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2081686544},{"type":"v1","addr":"172.21.15.134:6823","nonce":2081686544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2081686544},{"type":"v1","addr":"172.21.15.134:6821","nonce":2081686544}]},"public_addr":"172.21.15.134:6817/2081686544","cluster_addr":"172.21.15.134:6819/2081686544","heartbeat_back_addr":"172.21.15.134:6823/2081686544","heartbeat_front_addr":"172.21.15.134:6821/2081686544","state":["exists","up"]},{"osd":9,"uuid":"9486f8b5-0614-4208-87ec-117741f6b622","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":1210619387},{"type":"v1","addr":"172.21.15.77:6827","nonce":1210619387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":1210619387},{"type":"v1","addr":"172.21.15.77:6829","nonce":1210619387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":1210619387},{"type":"v1","addr":"172.21.15.77:6833","nonce":1210619387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":1210619387},{"type":"v1","addr":"172.21.15.77:6831","nonce":1210619387}]},"public_addr":"172.21.15.77:6827/1210619387","cluster_addr":"172.21.15.77:6829/1210619387","heartbeat_back_addr":"172.21.15.77:6833/1210619387","heartbeat_front_addr":"172.21.15.77:6831/1210619387","state":["exists","up"]},{"osd":10,"uuid":"8e8192b0-1398-4d4d-9756-762b2f874d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":754733885},{"type":"v1","addr":"172.21.15.120:6825","nonce":754733885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":754733885},{"type":"v1","addr":"172.21.15.120:6827","nonce":754733885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":754733885},{"type":"v1","addr":"172.21.15.120:6831","nonce":754733885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":754733885},{"type":"v1","addr":"172.21.15.120:6829","nonce":754733885}]},"public_addr":"172.21.15.120:6825/754733885","cluster_addr":"172.21.15.120:6827/754733885","heartbeat_back_addr":"172.21.15.120:6831/754733885","heartbeat_front_addr":"172.21.15.120:6829/754733885","state":["exists","up"]},{"osd":11,"uuid":"85c1ccd4-5147-4995-896e-fe977f73f749","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1602213143},{"type":"v1","addr":"172.21.15.134:6825","nonce":1602213143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1602213143},{"type":"v1","addr":"172.21.15.134:6827","nonce":1602213143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1602213143},{"type":"v1","addr":"172.21.15.134:6831","nonce":1602213143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1602213143},{"type":"v1","addr":"172.21.15.134:6829","nonce":1602213143}]},"public_addr":"172.21.15.134:6825/1602213143","cluster_addr":"172.21.15.134:6827/1602213143","heartbeat_back_addr":"172.21.15.134:6831/1602213143","heartbeat_front_addr":"172.21.15.134:6829/1602213143","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:46:30.293451+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:46:56.554801+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:47:21.476225+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:47:49.737332+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:48:16.092181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:48:41.240815+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:49:11.683950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:49:38.859255+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:50:04.105401+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:50:33.847810+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:50:59.901797+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:51:25.241507+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.2","mappings":[{"from":7,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":7}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":5,"to":8}]},{"pgid":"4.6","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.77:0/1697982655":"2024-09-11T09:44:59.575663+0000","172.21.15.77:6801/1527070442":"2024-09-11T09:44:59.575663+0000","172.21.15.77:0/3146688196":"2024-09-11T09:44:32.185342+0000","172.21.15.77:0/371773177":"2024-09-11T09:44:59.575663+0000","172.21.15.77:0/1925456910":"2024-09-11T09:44:59.575663+0000","172.21.15.77:0/138545832":"2024-09-11T09:44:32.185342+0000","172.21.15.77:6801/3235900503":"2024-09-11T09:44:32.185342+0000","172.21.15.77:6800/1527070442":"2024-09-11T09:44:59.575663+0000","172.21.15.77:6800/3235900503":"2024-09-11T09:44:32.185342+0000","172.21.15.77:0/1647092548":"2024-09-11T09:44:32.185342+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T09:55:22.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:22.145+0000 7fafd1c00640 1 -- 172.21.15.77:0/563724608 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fafb00a2b80 msgr2=0x7fafb00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:22.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:22.145+0000 7fafd1c00640 1 --2- 172.21.15.77:0/563724608 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fafb00a2b80 0x7fafb00a5040 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fafcc101e70 tx=0x7fafb800b390 comp rx=0 tx=0).stop 2024-09-10T09:55:22.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:22.145+0000 7fafd1c00640 1 -- 172.21.15.77:0/563724608 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fafcc103e30 msgr2=0x7fafcc19a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:22.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:22.145+0000 7fafd1c00640 1 --2- 172.21.15.77:0/563724608 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fafcc103e30 0x7fafcc19a480 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fafbc006010 tx=0x7fafbc0042f0 comp rx=0 tx=0).stop 2024-09-10T09:55:22.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:22.145+0000 7fafd1c00640 1 -- 172.21.15.77:0/563724608 shutdown_connections 2024-09-10T09:55:22.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:22.145+0000 7fafd1c00640 1 --2- 172.21.15.77:0/563724608 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fafb00a2b80 0x7fafb00a5040 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:22.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:22.145+0000 7fafd1c00640 1 --2- 172.21.15.77:0/563724608 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fafcc108f60 0x7fafcc19ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:22.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:22.145+0000 7fafd1c00640 1 --2- 172.21.15.77:0/563724608 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fafcc1047e0 0x7fafcc19a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:22.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:22.145+0000 7fafd1c00640 1 --2- 172.21.15.77:0/563724608 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fafcc103e30 0x7fafcc19a480 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:22.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:22.145+0000 7fafd1c00640 1 -- 172.21.15.77:0/563724608 >> 172.21.15.77:0/563724608 conn(0x7fafcc0ff910 msgr2=0x7fafcc100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:22.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:22.146+0000 7fafd1c00640 1 -- 172.21.15.77:0/563724608 shutdown_connections 2024-09-10T09:55:22.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:22.146+0000 7fafd1c00640 1 -- 172.21.15.77:0/563724608 wait complete. 2024-09-10T09:55:22.980 DEBUG:tasks.fwd_scrub:Yielding 2024-09-10T09:55:22.980 INFO:teuthology.run_tasks:Running task quiescer... 2024-09-10T09:55:22.994 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-10T09:55:22.994 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T09:55:22.994 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-09-10T09:55:22.994 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T09:55:22.995 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T09:55:23.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:55:22 smithi120 ceph-mon[34770]: pgmap v397: 129 pgs: 129 active+clean; 651 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-10T09:55:23.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:55:22 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/563724608' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T09:55:23.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:55:22 smithi077 ceph-mon[30319]: pgmap v397: 129 pgs: 129 active+clean; 651 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-10T09:55:23.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:55:22 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/563724608' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T09:55:23.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:55:22 smithi134 ceph-mon[35674]: pgmap v397: 129 pgs: 129 active+clean; 651 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-10T09:55:23.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:55:22 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/563724608' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T09:55:23.385 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:55:23.389 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:55:23.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:23.995+0000 7f1b75600640 1 -- 172.21.15.77:0/3769275732 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1b70106f10 msgr2=0x7f1b7010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:23.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:23.995+0000 7f1b75600640 1 --2- 172.21.15.77:0/3769275732 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1b70106f10 0x7f1b7010ea90 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f1b68009900 tx=0x7f1b6802a3f0 comp rx=0 tx=0).stop 2024-09-10T09:55:23.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:23.995+0000 7f1b75600640 1 -- 172.21.15.77:0/3769275732 shutdown_connections 2024-09-10T09:55:23.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:23.995+0000 7f1b75600640 1 --2- 172.21.15.77:0/3769275732 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1b70106f10 0x7f1b7010ea90 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:23.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:23.995+0000 7f1b75600640 1 --2- 172.21.15.77:0/3769275732 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1b70102790 0x7f1b701067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:23.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:23.995+0000 7f1b75600640 1 --2- 172.21.15.77:0/3769275732 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b70101de0 0x7f1b701021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:23.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:23.995+0000 7f1b75600640 1 -- 172.21.15.77:0/3769275732 >> 172.21.15.77:0/3769275732 conn(0x7f1b700fd8c0 msgr2=0x7f1b700ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:23.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:23.995+0000 7f1b75600640 1 -- 172.21.15.77:0/3769275732 shutdown_connections 2024-09-10T09:55:23.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:23.995+0000 7f1b75600640 1 -- 172.21.15.77:0/3769275732 wait complete. 2024-09-10T09:55:23.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:23.996+0000 7f1b75600640 1 Processor -- start 2024-09-10T09:55:23.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:23.996+0000 7f1b75600640 1 -- start start 2024-09-10T09:55:23.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:23.996+0000 7f1b75600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b70101de0 0x7f1b7019a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:23.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:23.996+0000 7f1b75600640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1b70102790 0x7f1b7019abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:23.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:23.996+0000 7f1b75600640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1b70106f10 0x7f1b7019ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:23.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:23.996+0000 7f1b75600640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1b70111f70 con 0x7f1b70106f10 2024-09-10T09:55:23.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:23.996+0000 7f1b75600640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1b70111df0 con 0x7f1b70102790 2024-09-10T09:55:23.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:23.996+0000 7f1b75600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1b701120f0 con 0x7f1b70101de0 2024-09-10T09:55:23.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:23.997+0000 7f1b66000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1b70102790 0x7f1b7019abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:23.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:23.997+0000 7f1b6ea00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b70101de0 0x7f1b7019a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:23.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:23.997+0000 7f1b6f400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1b70106f10 0x7f1b7019ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:23.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:23.997+0000 7f1b66000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1b70102790 0x7f1b7019abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:35326/0 (socket says 172.21.15.77:35326) 2024-09-10T09:55:23.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:23.997+0000 7f1b66000640 1 -- 172.21.15.77:0/1402228980 learned_addr learned my addr 172.21.15.77:0/1402228980 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:55:23.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:23.997+0000 7f1b6ea00640 1 -- 172.21.15.77:0/1402228980 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1b70102790 msgr2=0x7f1b7019abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:23.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:23.997+0000 7f1b6ea00640 1 --2- 172.21.15.77:0/1402228980 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1b70102790 0x7f1b7019abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:23.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:23.997+0000 7f1b6ea00640 1 -- 172.21.15.77:0/1402228980 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1b70106f10 msgr2=0x7f1b7019ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:23.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:23.997+0000 7f1b6ea00640 1 --2- 172.21.15.77:0/1402228980 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1b70106f10 0x7f1b7019ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:23.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:23.997+0000 7f1b6ea00640 1 -- 172.21.15.77:0/1402228980 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b7019f620 con 0x7f1b70101de0 2024-09-10T09:55:23.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:23.998+0000 7f1b6ea00640 1 --2- 172.21.15.77:0/1402228980 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b70101de0 0x7f1b7019a670 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1b6000e9f0 tx=0x7f1b6000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:24.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:23.998+0000 7f1b67e00640 1 -- 172.21.15.77:0/1402228980 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b6000ccc0 con 0x7f1b70101de0 2024-09-10T09:55:24.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:23.998+0000 7f1b75600640 1 -- 172.21.15.77:0/1402228980 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b7019f910 con 0x7f1b70101de0 2024-09-10T09:55:24.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:23.998+0000 7f1b67e00640 1 -- 172.21.15.77:0/1402228980 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b60010910 con 0x7f1b70101de0 2024-09-10T09:55:24.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:23.998+0000 7f1b67e00640 1 -- 172.21.15.77:0/1402228980 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b60018960 con 0x7f1b70101de0 2024-09-10T09:55:24.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:23.998+0000 7f1b75600640 1 -- 172.21.15.77:0/1402228980 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b701a71f0 con 0x7f1b70101de0 2024-09-10T09:55:24.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.001+0000 7f1b78231740 1 Processor -- start 2024-09-10T09:55:24.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.001+0000 7f1b67e00640 1 -- 172.21.15.77:0/1402228980 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1b60010ac0 con 0x7f1b70101de0 2024-09-10T09:55:24.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.001+0000 7f1b78231740 1 -- start start 2024-09-10T09:55:24.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.001+0000 7f1b78231740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584542adac0 0x5584542adea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:24.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.001+0000 7f1b78231740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584542ae3e0 0x5584542b8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:24.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.001+0000 7f1b78231740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584542b8ca0 0x5584542bb090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:24.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.001+0000 7f1b78231740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558454214290 con 0x5584542ae3e0 2024-09-10T09:55:24.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.001+0000 7f1b78231740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5584541bbb40 con 0x5584542adac0 2024-09-10T09:55:24.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.001+0000 7f1b78231740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558454210250 con 0x5584542b8ca0 2024-09-10T09:55:24.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.002+0000 7f1b6f400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584542adac0 0x5584542adea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:24.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.002+0000 7f1b6ea00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584542ae3e0 0x5584542b8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:24.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.002+0000 7f1b6f400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584542adac0 0x5584542adea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:35340/0 (socket says 172.21.15.77:35340) 2024-09-10T09:55:24.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.002+0000 7f1b6f400640 1 -- 172.21.15.77:0/2636407793 learned_addr learned my addr 172.21.15.77:0/2636407793 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:55:24.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.002+0000 7f1b66000640 1 --2- 172.21.15.77:0/2636407793 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584542b8ca0 0x5584542bb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:24.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.002+0000 7f1b6f400640 1 -- 172.21.15.77:0/2636407793 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584542b8ca0 msgr2=0x5584542bb090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:24.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.002+0000 7f1b6f400640 1 --2- 172.21.15.77:0/2636407793 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584542b8ca0 0x5584542bb090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:24.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.002+0000 7f1b6f400640 1 -- 172.21.15.77:0/2636407793 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584542ae3e0 msgr2=0x5584542b8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:24.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.002+0000 7f1b6f400640 1 --2- 172.21.15.77:0/2636407793 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584542ae3e0 0x5584542b8760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:24.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.002+0000 7f1b6f400640 1 -- 172.21.15.77:0/2636407793 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558453f856a0 con 0x5584542adac0 2024-09-10T09:55:24.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.002+0000 7f1b6ea00640 1 --2- 172.21.15.77:0/2636407793 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584542ae3e0 0x5584542b8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:55:24.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.002+0000 7f1b6f400640 1 --2- 172.21.15.77:0/2636407793 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584542adac0 0x5584542adea0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1b68033000 tx=0x7f1b68005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4e91d3355573645 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:24.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.003+0000 7f1b67e00640 1 --2- 172.21.15.77:0/1402228980 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f1b540a2b80 0x7f1b540a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:24.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.003+0000 7f1b5f400640 1 -- 172.21.15.77:0/2636407793 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b68040070 con 0x5584542adac0 2024-09-10T09:55:24.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.003+0000 7f1b5f400640 1 -- 172.21.15.77:0/2636407793 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b680317b0 con 0x5584542adac0 2024-09-10T09:55:24.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.003+0000 7f1b5f400640 1 -- 172.21.15.77:0/2636407793 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b68035040 con 0x5584542adac0 2024-09-10T09:55:24.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.003+0000 7f1b67e00640 1 -- 172.21.15.77:0/1402228980 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f1b600cf820 con 0x7f1b70101de0 2024-09-10T09:55:24.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.003+0000 7f1b6ea00640 1 --2- 172.21.15.77:0/1402228980 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f1b540a2b80 0x7f1b540a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:24.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.003+0000 7f1b78231740 1 -- 172.21.15.77:0/2636407793 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584542adac0 msgr2=0x5584542adea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:24.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.003+0000 7f1b78231740 1 --2- 172.21.15.77:0/2636407793 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584542adac0 0x5584542adea0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1b68033000 tx=0x7f1b68005d50 comp rx=0 tx=0).stop 2024-09-10T09:55:24.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.004+0000 7f1b78231740 1 -- 172.21.15.77:0/2636407793 shutdown_connections 2024-09-10T09:55:24.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.004+0000 7f1b78231740 1 --2- 172.21.15.77:0/2636407793 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584542b8ca0 0x5584542bb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:24.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.004+0000 7f1b78231740 1 --2- 172.21.15.77:0/2636407793 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584542ae3e0 0x5584542b8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:24.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.004+0000 7f1b78231740 1 --2- 172.21.15.77:0/2636407793 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584542adac0 0x5584542adea0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:24.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.004+0000 7f1b78231740 1 -- 172.21.15.77:0/2636407793 >> 172.21.15.77:0/2636407793 conn(0x5584542aa050 msgr2=0x5584542aa430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:24.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.004+0000 7f1b78231740 1 -- 172.21.15.77:0/2636407793 shutdown_connections 2024-09-10T09:55:24.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.004+0000 7f1b6ea00640 1 --2- 172.21.15.77:0/1402228980 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f1b540a2b80 0x7f1b540a5040 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1b60098140 tx=0x7f1b60009700 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:24.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.004+0000 7f1b78231740 1 -- 172.21.15.77:0/2636407793 wait complete. 2024-09-10T09:55:24.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.004+0000 7f1b78231740 1 -- start start 2024-09-10T09:55:24.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.005+0000 7f1b78231740 1 Processor -- start 2024-09-10T09:55:24.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.005+0000 7f1b78231740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584542adac0 0x5584542b85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:24.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.005+0000 7f1b78231740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584542ae3e0 0x5584542b5850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:24.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.005+0000 7f1b78231740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584542b8ca0 0x5584542b5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:24.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.005+0000 7f1b78231740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5584542bb910 con 0x5584542ae3e0 2024-09-10T09:55:24.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.005+0000 7f1b78231740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5584541cefa0 con 0x5584542b8ca0 2024-09-10T09:55:24.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.005+0000 7f1b78231740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5584542bba90 con 0x5584542adac0 2024-09-10T09:55:24.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.005+0000 7f1b78231740 10 client.0 _ll_register_callbacks cb 0x558453f22ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:55:24.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.005+0000 7f1b66000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584542ae3e0 0x5584542b5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:24.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.005+0000 7f1b66000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584542ae3e0 0x5584542b5850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56882/0 (socket says 172.21.15.77:56882) 2024-09-10T09:55:24.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.005+0000 7f1b66000640 1 -- 172.21.15.77:0/3292482497 learned_addr learned my addr 172.21.15.77:0/3292482497 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:55:24.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.005+0000 7f1b6f400640 1 --2- 172.21.15.77:0/3292482497 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584542adac0 0x5584542b85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:24.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.006+0000 7f1b6f400640 1 --2- 172.21.15.77:0/3292482497 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584542b8ca0 0x5584542b5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:24.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.006+0000 7f1b66000640 1 -- 172.21.15.77:0/3292482497 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584542adac0 msgr2=0x5584542b85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:24.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.006+0000 7f1b66000640 1 --2- 172.21.15.77:0/3292482497 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584542adac0 0x5584542b85a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:24.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.006+0000 7f1b66000640 1 -- 172.21.15.77:0/3292482497 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584542b8ca0 msgr2=0x5584542b5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:24.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.006+0000 7f1b66000640 1 --2- 172.21.15.77:0/3292482497 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584542b8ca0 0x5584542b5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:24.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.006+0000 7f1b66000640 1 -- 172.21.15.77:0/3292482497 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584542b65e0 con 0x5584542ae3e0 2024-09-10T09:55:24.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.006+0000 7f1b6f400640 1 --2- 172.21.15.77:0/3292482497 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584542adac0 0x5584542b85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:55:24.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.006+0000 7f1b66000640 1 --2- 172.21.15.77:0/3292482497 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584542ae3e0 0x5584542b5850 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f1b58011d10 tx=0x7f1b5800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8353fc2f705544d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:24.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.006+0000 7f1b53e00640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-10T09:55:24.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.006+0000 7f1b53e00640 1 -- 172.21.15.77:0/3292482497 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b5801f060 con 0x5584542ae3e0 2024-09-10T09:55:24.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.006+0000 7f1b53e00640 1 -- 172.21.15.77:0/3292482497 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f1b58017070 con 0x5584542ae3e0 2024-09-10T09:55:24.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.006+0000 7f1b78231740 1 -- 172.21.15.77:0/3292482497 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b680317b0 con 0x5584542ae3e0 2024-09-10T09:55:24.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.007+0000 7f1b53e00640 1 -- 172.21.15.77:0/3292482497 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1b58026050 con 0x5584542ae3e0 2024-09-10T09:55:24.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.007+0000 7f1b78231740 10 client.15267 fetch_fsmap learned FSMap version 29 2024-09-10T09:55:24.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.007+0000 7f1b78231740 1 -- 172.21.15.77:0/3292482497 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5584542c7850 con 0x5584542ae3e0 2024-09-10T09:55:24.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.007+0000 7f1b53e00640 1 -- 172.21.15.77:0/3292482497 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b680317b0 con 0x5584542ae3e0 2024-09-10T09:55:24.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.007+0000 7f1b53e00640 1 -- 172.21.15.77:0/3292482497 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f1b58019370 con 0x5584542ae3e0 2024-09-10T09:55:24.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.007+0000 7f1b78231740 10 client.15267 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T09:55:24.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.007+0000 7f1b78231740 10 client.15267 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:55:24.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.007+0000 7f1b78231740 20 client.15267 populate_metadata read hostname 'smithi077' 2024-09-10T09:55:24.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.007+0000 7f1b78231740 1 --2- 172.21.15.77:0/3292482497 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x5584542c8d10 0x5584542cb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:24.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.007+0000 7f1b78231740 4 client.15267 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:55:24.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.007+0000 7f1b78231740 1 -- 172.21.15.77:0/3292482497 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558454221800 con 0x5584542c8d10 2024-09-10T09:55:24.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.008+0000 7f1b6f400640 1 --2- 172.21.15.77:0/3292482497 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x5584542c8d10 0x5584542cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:24.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.008+0000 7f1b6f400640 1 --2- 172.21.15.77:0/3292482497 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x5584542c8d10 0x5584542cb130 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d7f9597b06f1a62 server_cookie=3591b2048a367d0c in_seq=0 out_seq=0 2024-09-10T09:55:24.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.008+0000 7f1b53e00640 10 client.15267 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T09:55:24.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.009+0000 7f1b53e00640 1 -- 172.21.15.77:0/3292482497 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558454221800 con 0x5584542c8d10 2024-09-10T09:55:24.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.009+0000 7f1b53e00640 10 client.15267 handle_command_reply: tid=0 2024-09-10T09:55:24.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.009+0000 7f1b78231740 1 client.15267 shutdown 2024-09-10T09:55:24.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.009+0000 7f1b78231740 1 -- 172.21.15.77:0/3292482497 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584542ae3e0 msgr2=0x5584542b5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:24.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.009+0000 7f1b78231740 1 --2- 172.21.15.77:0/3292482497 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584542ae3e0 0x5584542b5850 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f1b58011d10 tx=0x7f1b5800f790 comp rx=0 tx=0).stop 2024-09-10T09:55:24.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.009+0000 7f1b78231740 1 -- 172.21.15.77:0/3292482497 shutdown_connections 2024-09-10T09:55:24.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.009+0000 7f1b78231740 1 --2- 172.21.15.77:0/3292482497 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x5584542c8d10 0x5584542cb130 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:24.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.009+0000 7f1b78231740 1 --2- 172.21.15.77:0/3292482497 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584542b8ca0 0x5584542b5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:24.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.009+0000 7f1b78231740 1 --2- 172.21.15.77:0/3292482497 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584542ae3e0 0x5584542b5850 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:24.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.010+0000 7f1b78231740 1 --2- 172.21.15.77:0/3292482497 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584542adac0 0x5584542b85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:24.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.010+0000 7f1b78231740 1 -- 172.21.15.77:0/3292482497 >> 172.21.15.77:0/3292482497 conn(0x5584542aa050 msgr2=0x5584542a91c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:24.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.010+0000 7f1b53e00640 0 client.15267 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T09:55:24.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.010+0000 7f1b78231740 1 -- 172.21.15.77:0/3292482497 shutdown_connections 2024-09-10T09:55:24.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.010+0000 7f1b78231740 1 -- 172.21.15.77:0/3292482497 wait complete. 2024-09-10T09:55:24.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.010+0000 7f1b78231740 20 client.15267 trim_cache size 0 max 16384 2024-09-10T09:55:24.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.029+0000 7f1b78231740 1 Processor -- start 2024-09-10T09:55:24.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.029+0000 7f1b78231740 1 -- start start 2024-09-10T09:55:24.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.030+0000 7f1b78231740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584542adac0 0x5584542b0d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:24.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.030+0000 7f1b78231740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584542ae3e0 0x5584542d09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:24.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.030+0000 7f1b78231740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584542b8ca0 0x5584542d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:24.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.030+0000 7f1b78231740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1b701b1060 con 0x5584542b8ca0 2024-09-10T09:55:24.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.030+0000 7f1b78231740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1b701b2ba0 con 0x5584542adac0 2024-09-10T09:55:24.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.030+0000 7f1b78231740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1b701b1730 con 0x5584542ae3e0 2024-09-10T09:55:24.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.030+0000 7f1b6f400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584542adac0 0x5584542b0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:24.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.030+0000 7f1b6f400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584542adac0 0x5584542b0d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:35356/0 (socket says 172.21.15.77:35356) 2024-09-10T09:55:24.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.030+0000 7f1b6f400640 1 -- 172.21.15.77:0/4080065727 learned_addr learned my addr 172.21.15.77:0/4080065727 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:55:24.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.030+0000 7f1b6f400640 1 --2- 172.21.15.77:0/4080065727 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584542b8ca0 0x5584542d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:24.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.030+0000 7f1b66000640 1 --2- 172.21.15.77:0/4080065727 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584542ae3e0 0x5584542d09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:24.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.030+0000 7f1b6f400640 1 -- 172.21.15.77:0/4080065727 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584542ae3e0 msgr2=0x5584542d09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:24.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.030+0000 7f1b6f400640 1 --2- 172.21.15.77:0/4080065727 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584542ae3e0 0x5584542d09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:24.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.030+0000 7f1b6f400640 1 -- 172.21.15.77:0/4080065727 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584542b8ca0 msgr2=0x5584542d0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:24.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.030+0000 7f1b6f400640 1 --2- 172.21.15.77:0/4080065727 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584542b8ca0 0x5584542d0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:24.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.030+0000 7f1b6f400640 1 -- 172.21.15.77:0/4080065727 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584542a9e80 con 0x5584542adac0 2024-09-10T09:55:24.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.031+0000 7f1b6f400640 1 --2- 172.21.15.77:0/4080065727 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584542adac0 0x5584542b0d70 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1b68033000 tx=0x7f1b6802a3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8da2a612ac01d5ac server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:24.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.031+0000 7f1b5ea00640 1 -- 172.21.15.77:0/4080065727 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b68057070 con 0x5584542adac0 2024-09-10T09:55:24.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.031+0000 7f1b5ea00640 1 -- 172.21.15.77:0/4080065727 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b680318f0 con 0x5584542adac0 2024-09-10T09:55:24.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.031+0000 7f1b5ea00640 1 -- 172.21.15.77:0/4080065727 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b68035040 con 0x5584542adac0 2024-09-10T09:55:24.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.032+0000 7f1b78231740 1 -- 172.21.15.77:0/4080065727 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584542adac0 msgr2=0x5584542b0d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:24.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.032+0000 7f1b78231740 1 --2- 172.21.15.77:0/4080065727 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584542adac0 0x5584542b0d70 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1b68033000 tx=0x7f1b6802a3f0 comp rx=0 tx=0).stop 2024-09-10T09:55:24.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.032+0000 7f1b78231740 1 -- 172.21.15.77:0/4080065727 shutdown_connections 2024-09-10T09:55:24.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.032+0000 7f1b78231740 1 --2- 172.21.15.77:0/4080065727 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584542b8ca0 0x5584542d0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:24.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.032+0000 7f1b78231740 1 --2- 172.21.15.77:0/4080065727 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584542ae3e0 0x5584542d09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:24.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.032+0000 7f1b78231740 1 --2- 172.21.15.77:0/4080065727 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584542adac0 0x5584542b0d70 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:24.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.032+0000 7f1b78231740 1 -- 172.21.15.77:0/4080065727 >> 172.21.15.77:0/4080065727 conn(0x5584542aa050 msgr2=0x5584542b78d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:24.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.032+0000 7f1b78231740 1 -- 172.21.15.77:0/4080065727 shutdown_connections 2024-09-10T09:55:24.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.032+0000 7f1b78231740 1 -- 172.21.15.77:0/4080065727 wait complete. 2024-09-10T09:55:24.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.032+0000 7f1b78231740 1 -- start start 2024-09-10T09:55:24.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.033+0000 7f1b78231740 1 Processor -- start 2024-09-10T09:55:24.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.033+0000 7f1b78231740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584542adac0 0x5584542c0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:24.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.033+0000 7f1b78231740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584542ae3e0 0x5584542c1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:24.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.033+0000 7f1b78231740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584542b8ca0 0x5584542bdcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:24.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.033+0000 7f1b78231740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1b701b24d0 con 0x5584542b8ca0 2024-09-10T09:55:24.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.033+0000 7f1b78231740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1b701b1e00 con 0x5584542adac0 2024-09-10T09:55:24.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.033+0000 7f1b78231740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1b701af040 con 0x5584542ae3e0 2024-09-10T09:55:24.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.033+0000 7f1b78231740 10 client.0 _ll_register_callbacks cb 0x5584542b67b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:55:24.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.033+0000 7f1b6f400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584542b8ca0 0x5584542bdcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:24.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.034+0000 7f1b66000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584542ae3e0 0x5584542c1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:24.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.034+0000 7f1b6f400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584542b8ca0 0x5584542bdcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56894/0 (socket says 172.21.15.77:56894) 2024-09-10T09:55:24.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.034+0000 7f1b66000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584542ae3e0 0x5584542c1220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:60492/0 (socket says 172.21.15.77:60492) 2024-09-10T09:55:24.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.034+0000 7f1b6f400640 1 -- 172.21.15.77:0/4112622187 learned_addr learned my addr 172.21.15.77:0/4112622187 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:55:24.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.034+0000 7f1b6f400640 1 --2- 172.21.15.77:0/4112622187 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584542adac0 0x5584542c0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:24.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.034+0000 7f1b66000640 1 -- 172.21.15.77:0/4112622187 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584542adac0 msgr2=0x5584542c0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:24.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.034+0000 7f1b66000640 1 --2- 172.21.15.77:0/4112622187 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584542adac0 0x5584542c0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:24.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.034+0000 7f1b66000640 1 -- 172.21.15.77:0/4112622187 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584542b8ca0 msgr2=0x5584542bdcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:24.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.034+0000 7f1b66000640 1 --2- 172.21.15.77:0/4112622187 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584542b8ca0 0x5584542bdcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:24.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.034+0000 7f1b66000640 1 -- 172.21.15.77:0/4112622187 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584542aac00 con 0x5584542ae3e0 2024-09-10T09:55:24.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.034+0000 7f1b66000640 1 --2- 172.21.15.77:0/4112622187 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584542ae3e0 0x5584542c1220 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x5584542b6e00 tx=0x7f1b580095e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea1aef48743f06ba server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:24.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.034+0000 7f1b53e00640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-10T09:55:24.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.034+0000 7f1b53e00640 1 -- 172.21.15.77:0/4112622187 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b5802f060 con 0x5584542ae3e0 2024-09-10T09:55:24.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.034+0000 7f1b78231740 1 -- 172.21.15.77:0/4112622187 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b680318f0 con 0x5584542ae3e0 2024-09-10T09:55:24.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.035+0000 7f1b53e00640 1 -- 172.21.15.77:0/4112622187 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f1b58011e00 con 0x5584542ae3e0 2024-09-10T09:55:24.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.035+0000 7f1b53e00640 1 -- 172.21.15.77:0/4112622187 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1b58036050 con 0x5584542ae3e0 2024-09-10T09:55:24.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.035+0000 7f1b78231740 10 client.25300 fetch_fsmap learned FSMap version 29 2024-09-10T09:55:24.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.035+0000 7f1b78231740 1 -- 172.21.15.77:0/4112622187 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558454334160 con 0x5584542ae3e0 2024-09-10T09:55:24.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.035+0000 7f1b53e00640 1 -- 172.21.15.77:0/4112622187 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b680318f0 con 0x5584542ae3e0 2024-09-10T09:55:24.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.035+0000 7f1b53e00640 1 -- 172.21.15.77:0/4112622187 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f1b580050d0 con 0x5584542ae3e0 2024-09-10T09:55:24.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.035+0000 7f1b78231740 10 client.25300 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T09:55:24.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.035+0000 7f1b78231740 10 client.25300 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:55:24.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.035+0000 7f1b78231740 20 client.25300 populate_metadata read hostname 'smithi077' 2024-09-10T09:55:24.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.035+0000 7f1b78231740 1 --2- 172.21.15.77:0/4112622187 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x558454334880 0x558454336ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:24.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.035+0000 7f1b78231740 4 client.25300 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T09:55:24.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.035+0000 7f1b78231740 1 -- 172.21.15.77:0/4112622187 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55845421bd10 con 0x558454334880 2024-09-10T09:55:24.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.036+0000 7f1b6f400640 1 --2- 172.21.15.77:0/4112622187 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x558454334880 0x558454336ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:24.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.036+0000 7f1b6f400640 1 --2- 172.21.15.77:0/4112622187 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x558454334880 0x558454336ca0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d64f171a6bb80c2 server_cookie=86843ccc8d588e23 in_seq=0 out_seq=0 2024-09-10T09:55:24.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.036+0000 7f1b53e00640 10 client.25300 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T09:55:24.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.037+0000 7f1b53e00640 1 -- 172.21.15.77:0/4112622187 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55845421bd10 con 0x558454334880 2024-09-10T09:55:24.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.037+0000 7f1b53e00640 10 client.25300 handle_command_reply: tid=0 2024-09-10T09:55:24.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.037+0000 7f1b78231740 1 client.25300 shutdown 2024-09-10T09:55:24.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.037+0000 7f1b78231740 1 -- 172.21.15.77:0/4112622187 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584542ae3e0 msgr2=0x5584542c1220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:24.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.037+0000 7f1b78231740 1 --2- 172.21.15.77:0/4112622187 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584542ae3e0 0x5584542c1220 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x5584542b6e00 tx=0x7f1b580095e0 comp rx=0 tx=0).stop 2024-09-10T09:55:24.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.038+0000 7f1b78231740 1 -- 172.21.15.77:0/4112622187 shutdown_connections 2024-09-10T09:55:24.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.038+0000 7f1b78231740 1 --2- 172.21.15.77:0/4112622187 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x558454334880 0x558454336ca0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:24.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.038+0000 7f1b78231740 1 --2- 172.21.15.77:0/4112622187 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584542b8ca0 0x5584542bdcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:24.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.038+0000 7f1b78231740 1 --2- 172.21.15.77:0/4112622187 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584542ae3e0 0x5584542c1220 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:24.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.038+0000 7f1b78231740 1 --2- 172.21.15.77:0/4112622187 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584542adac0 0x5584542c0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:24.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.038+0000 7f1b53e00640 0 client.25300 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T09:55:24.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.038+0000 7f1b78231740 1 -- 172.21.15.77:0/4112622187 >> 172.21.15.77:0/4112622187 conn(0x5584542aa050 msgr2=0x5584542af4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:24.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.038+0000 7f1b78231740 1 -- 172.21.15.77:0/4112622187 shutdown_connections 2024-09-10T09:55:24.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.038+0000 7f1b78231740 1 -- 172.21.15.77:0/4112622187 wait complete. 2024-09-10T09:55:24.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.038+0000 7f1b78231740 20 client.25300 trim_cache size 0 max 16384 2024-09-10T09:55:24.042 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-10T09:55:24.042 INFO:teuthology.orchestra.run.smithi077.stdout: "return_code": 0, 2024-09-10T09:55:24.042 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_tag": "0f675642-7920-4b2e-b937-aa35f6a02f29", 2024-09-10T09:55:24.042 INFO:teuthology.orchestra.run.smithi077.stdout: "mode": "asynchronous" 2024-09-10T09:55:24.042 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-10T09:55:24.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.039+0000 7f1b75600640 1 -- 172.21.15.77:0/1402228980 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f1b540a2b80 msgr2=0x7f1b540a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:24.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.039+0000 7f1b75600640 1 --2- 172.21.15.77:0/1402228980 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f1b540a2b80 0x7f1b540a5040 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1b60098140 tx=0x7f1b60009700 comp rx=0 tx=0).stop 2024-09-10T09:55:24.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.039+0000 7f1b75600640 1 -- 172.21.15.77:0/1402228980 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b70101de0 msgr2=0x7f1b7019a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:24.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.039+0000 7f1b75600640 1 --2- 172.21.15.77:0/1402228980 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b70101de0 0x7f1b7019a670 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1b6000e9f0 tx=0x7f1b6000eeb0 comp rx=0 tx=0).stop 2024-09-10T09:55:24.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.039+0000 7f1b75600640 1 -- 172.21.15.77:0/1402228980 shutdown_connections 2024-09-10T09:55:24.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.039+0000 7f1b75600640 1 --2- 172.21.15.77:0/1402228980 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f1b540a2b80 0x7f1b540a5040 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:24.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.039+0000 7f1b75600640 1 --2- 172.21.15.77:0/1402228980 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1b70106f10 0x7f1b7019ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:24.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.039+0000 7f1b75600640 1 --2- 172.21.15.77:0/1402228980 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1b70102790 0x7f1b7019abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:24.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.039+0000 7f1b75600640 1 --2- 172.21.15.77:0/1402228980 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b70101de0 0x7f1b7019a670 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:24.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.039+0000 7f1b75600640 1 -- 172.21.15.77:0/1402228980 >> 172.21.15.77:0/1402228980 conn(0x7f1b700fd8c0 msgr2=0x7f1b700ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:24.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.039+0000 7f1b75600640 1 -- 172.21.15.77:0/1402228980 shutdown_connections 2024-09-10T09:55:24.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.039+0000 7f1b75600640 1 -- 172.21.15.77:0/1402228980 wait complete. 2024-09-10T09:55:25.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:55:24 smithi120 ceph-mon[34770]: pgmap v398: 129 pgs: 129 active+clean; 651 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-10T09:55:25.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:55:24 smithi120 ceph-mon[34770]: scrub queued for path: / 2024-09-10T09:55:25.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:55:24 smithi120 ceph-mon[34770]: scrub summary: idle+waiting paths [/] 2024-09-10T09:55:25.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:55:24 smithi120 ceph-mon[34770]: scrub summary: active paths [/] 2024-09-10T09:55:25.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:55:24 smithi120 ceph-mon[34770]: scrub summary: idle+waiting paths [/] 2024-09-10T09:55:25.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:55:24 smithi077 ceph-mon[30319]: pgmap v398: 129 pgs: 129 active+clean; 651 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-10T09:55:25.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:55:24 smithi077 ceph-mon[30319]: scrub queued for path: / 2024-09-10T09:55:25.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:55:24 smithi077 ceph-mon[30319]: scrub summary: idle+waiting paths [/] 2024-09-10T09:55:25.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:55:24 smithi077 ceph-mon[30319]: scrub summary: active paths [/] 2024-09-10T09:55:25.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:55:24 smithi077 ceph-mon[30319]: scrub summary: idle+waiting paths [/] 2024-09-10T09:55:25.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.886+0000 7f4293e00640 1 -- 172.21.15.77:0/208298819 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4294103e50 msgr2=0x7f4294104230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:25.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.886+0000 7f4293e00640 1 --2- 172.21.15.77:0/208298819 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4294103e50 0x7f4294104230 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f4284009900 tx=0x7f428402a410 comp rx=0 tx=0).stop 2024-09-10T09:55:25.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.886+0000 7f4293e00640 1 -- 172.21.15.77:0/208298819 shutdown_connections 2024-09-10T09:55:25.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.886+0000 7f4293e00640 1 --2- 172.21.15.77:0/208298819 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4294109090 0x7f4294110c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:25.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.886+0000 7f4293e00640 1 --2- 172.21.15.77:0/208298819 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4294104800 0x7f4294108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:25.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.886+0000 7f4293e00640 1 --2- 172.21.15.77:0/208298819 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4294103e50 0x7f4294104230 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:25.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.886+0000 7f4293e00640 1 -- 172.21.15.77:0/208298819 >> 172.21.15.77:0/208298819 conn(0x7f42940ff8b0 msgr2=0x7f4294101cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:25.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.886+0000 7f4293e00640 1 -- 172.21.15.77:0/208298819 shutdown_connections 2024-09-10T09:55:25.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.886+0000 7f4293e00640 1 -- 172.21.15.77:0/208298819 wait complete. 2024-09-10T09:55:25.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.887+0000 7f4293e00640 1 Processor -- start 2024-09-10T09:55:25.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.887+0000 7f4293e00640 1 -- start start 2024-09-10T09:55:25.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.888+0000 7f4293e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4294103e50 0x7f429419a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:25.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.888+0000 7f4293e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4294104800 0x7f429419aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:25.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.888+0000 7f4293e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4294109090 0x7f429419ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:25.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.888+0000 7f4293e00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f42941140b0 con 0x7f4294109090 2024-09-10T09:55:25.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.888+0000 7f4293e00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4294113f30 con 0x7f4294103e50 2024-09-10T09:55:25.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.888+0000 7f4293e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4294114230 con 0x7f4294104800 2024-09-10T09:55:25.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.888+0000 7f4293400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4294109090 0x7f429419ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:25.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.888+0000 7f4292000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4294104800 0x7f429419aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:25.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.888+0000 7f4293400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4294109090 0x7f429419ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56910/0 (socket says 172.21.15.77:56910) 2024-09-10T09:55:25.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.888+0000 7f4293400640 1 -- 172.21.15.77:0/902953998 learned_addr learned my addr 172.21.15.77:0/902953998 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:55:25.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.888+0000 7f4292a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4294103e50 0x7f429419a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:25.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.888+0000 7f4293400640 1 -- 172.21.15.77:0/902953998 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4294104800 msgr2=0x7f429419aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:25.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.888+0000 7f4293400640 1 --2- 172.21.15.77:0/902953998 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4294104800 0x7f429419aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:25.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.888+0000 7f4293400640 1 -- 172.21.15.77:0/902953998 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4294103e50 msgr2=0x7f429419a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:25.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.888+0000 7f4293400640 1 --2- 172.21.15.77:0/902953998 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4294103e50 0x7f429419a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:25.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.888+0000 7f4293400640 1 -- 172.21.15.77:0/902953998 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f429419f520 con 0x7f4294109090 2024-09-10T09:55:25.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.889+0000 7f4293400640 1 --2- 172.21.15.77:0/902953998 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4294109090 0x7f429419ee40 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f428c00eaa0 tx=0x7f428c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:25.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.889+0000 7f428b400640 1 -- 172.21.15.77:0/902953998 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f428c00cdb0 con 0x7f4294109090 2024-09-10T09:55:25.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.889+0000 7f4293e00640 1 -- 172.21.15.77:0/902953998 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f429419f810 con 0x7f4294109090 2024-09-10T09:55:25.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.889+0000 7f428b400640 1 -- 172.21.15.77:0/902953998 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f428c005020 con 0x7f4294109090 2024-09-10T09:55:25.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.889+0000 7f428b400640 1 -- 172.21.15.77:0/902953998 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f428c0116d0 con 0x7f4294109090 2024-09-10T09:55:25.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.889+0000 7f4293e00640 1 -- 172.21.15.77:0/902953998 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42941a70f0 con 0x7f4294109090 2024-09-10T09:55:25.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.890+0000 7f4293e00640 1 -- 172.21.15.77:0/902953998 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4294105330 con 0x7f4294109090 2024-09-10T09:55:25.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.892+0000 7f428b400640 1 -- 172.21.15.77:0/902953998 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f428c0054d0 con 0x7f4294109090 2024-09-10T09:55:25.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.893+0000 7f428b400640 1 --2- 172.21.15.77:0/902953998 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f42780a2b80 0x7f42780a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:25.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.893+0000 7f428b400640 1 -- 172.21.15.77:0/902953998 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f428c0cf5e0 con 0x7f4294109090 2024-09-10T09:55:25.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.893+0000 7f4292a00640 1 --2- 172.21.15.77:0/902953998 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f42780a2b80 0x7f42780a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:25.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.894+0000 7f4292a00640 1 --2- 172.21.15.77:0/902953998 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f42780a2b80 0x7f42780a5040 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f4284009230 tx=0x7f42840058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:25.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:24.898+0000 7f428b400640 1 -- 172.21.15.77:0/902953998 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f428c016030 con 0x7f4294109090 2024-09-10T09:55:25.311 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:55:25.311 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:25.075+0000 7f4293e00640 1 -- 172.21.15.77:0/902953998 --> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f429410c2e0 con 0x7f42780a2b80 2024-09-10T09:55:25.311 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:25.083+0000 7f428b400640 1 -- 172.21.15.77:0/902953998 <== mgr.14150 v2:172.21.15.77:6800/4261324650 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340856 (secure 0 0 0) 0x7f4284031630 con 0x7f42780a2b80 2024-09-10T09:55:25.315 INFO:teuthology.orchestra.run.smithi077.stderr:dumped all 2024-09-10T09:55:25.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:25.312+0000 7f4293e00640 1 -- 172.21.15.77:0/902953998 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f42780a2b80 msgr2=0x7f42780a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:25.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:25.312+0000 7f4293e00640 1 --2- 172.21.15.77:0/902953998 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f42780a2b80 0x7f42780a5040 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f4284009230 tx=0x7f42840058d0 comp rx=0 tx=0).stop 2024-09-10T09:55:25.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:25.312+0000 7f4293e00640 1 -- 172.21.15.77:0/902953998 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4294109090 msgr2=0x7f429419ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:25.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:25.312+0000 7f4293e00640 1 --2- 172.21.15.77:0/902953998 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4294109090 0x7f429419ee40 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f428c00eaa0 tx=0x7f428c00ef60 comp rx=0 tx=0).stop 2024-09-10T09:55:25.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:25.312+0000 7f4293e00640 1 -- 172.21.15.77:0/902953998 shutdown_connections 2024-09-10T09:55:25.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:25.312+0000 7f4293e00640 1 --2- 172.21.15.77:0/902953998 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f42780a2b80 0x7f42780a5040 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:25.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:25.312+0000 7f4293e00640 1 --2- 172.21.15.77:0/902953998 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4294109090 0x7f429419ee40 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:25.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:25.312+0000 7f4293e00640 1 --2- 172.21.15.77:0/902953998 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4294104800 0x7f429419aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:25.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:25.312+0000 7f4293e00640 1 --2- 172.21.15.77:0/902953998 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4294103e50 0x7f429419a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:25.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:25.312+0000 7f4293e00640 1 -- 172.21.15.77:0/902953998 >> 172.21.15.77:0/902953998 conn(0x7f42940ff8b0 msgr2=0x7f4294100b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:25.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:25.312+0000 7f4293e00640 1 -- 172.21.15.77:0/902953998 shutdown_connections 2024-09-10T09:55:25.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:25.312+0000 7f4293e00640 1 -- 172.21.15.77:0/902953998 wait complete. 2024-09-10T09:55:25.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:55:24 smithi134 ceph-mon[35674]: pgmap v398: 129 pgs: 129 active+clean; 651 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-10T09:55:25.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:55:24 smithi134 ceph-mon[35674]: scrub queued for path: / 2024-09-10T09:55:25.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:55:24 smithi134 ceph-mon[35674]: scrub summary: idle+waiting paths [/] 2024-09-10T09:55:25.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:55:24 smithi134 ceph-mon[35674]: scrub summary: active paths [/] 2024-09-10T09:55:25.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:55:24 smithi134 ceph-mon[35674]: scrub summary: idle+waiting paths [/] 2024-09-10T09:55:25.582 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T09:55:26.219 INFO:teuthology.orchestra.run.smithi077.stdout:{"pg_ready":true,"pg_map":{"version":398,"stamp":"2024-09-10T09:55:23.720885+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":666279,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1425,"num_read_kb":841,"num_write":446,"num_write_kb":3098,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1185162,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":866,"ondisk_log_size":866,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":366484,"kb_used_data":13076,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124575340,"statfs":{"total":1151940427776,"available":1151565148160,"internally_reserved":0,"allocated":13389824,"data_stored":8615087,"data_compressed":504633,"data_compressed_allocated":688128,"data_compressed_original":2875392,"omap_allocated":19143,"internal_metadata":361674041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4936,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":138,"num_read_kb":72,"num_write":9,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003357"},"pg_stats":[{"pgid":"2.1f","version":"96'14","reported_seq":193,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:55:20.065800+0000","last_change":"2024-09-10T09:53:00.803590+0000","last_active":"2024-09-10T09:55:20.065800+0000","last_peered":"2024-09-10T09:55:20.065800+0000","last_clean":"2024-09-10T09:55:20.065800+0000","last_became_active":"2024-09-10T09:53:00.798189+0000","last_became_peered":"2024-09-10T09:53:00.798189+0000","last_unstale":"2024-09-10T09:55:20.065800+0000","last_undegraded":"2024-09-10T09:55:20.065800+0000","last_fullsized":"2024-09-10T09:55:20.065800+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:18:48.808836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":126,"num_read_kb":126,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"96'5","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.682459+0000","last_change":"2024-09-10T09:52:22.318195+0000","last_active":"2024-09-10T09:54:29.682459+0000","last_peered":"2024-09-10T09:54:29.682459+0000","last_clean":"2024-09-10T09:54:29.682459+0000","last_became_active":"2024-09-10T09:52:22.315905+0000","last_became_peered":"2024-09-10T09:52:22.315905+0000","last_unstale":"2024-09-10T09:54:29.682459+0000","last_undegraded":"2024-09-10T09:54:29.682459+0000","last_fullsized":"2024-09-10T09:54:29.682459+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:53:55.515984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.843988+0000","last_change":"2024-09-10T09:52:31.369780+0000","last_active":"2024-09-10T09:54:00.843988+0000","last_peered":"2024-09-10T09:54:00.843988+0000","last_clean":"2024-09-10T09:54:00.843988+0000","last_became_active":"2024-09-10T09:52:31.369238+0000","last_became_peered":"2024-09-10T09:52:31.369238+0000","last_unstale":"2024-09-10T09:54:00.843988+0000","last_undegraded":"2024-09-10T09:54:00.843988+0000","last_fullsized":"2024-09-10T09:54:00.843988+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:39:05.318493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"94'2","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.847203+0000","last_change":"2024-09-10T09:52:19.912276+0000","last_active":"2024-09-10T09:54:00.847203+0000","last_peered":"2024-09-10T09:54:00.847203+0000","last_clean":"2024-09-10T09:54:00.847203+0000","last_became_active":"2024-09-10T09:52:19.911452+0000","last_became_peered":"2024-09-10T09:52:19.911452+0000","last_unstale":"2024-09-10T09:54:00.847203+0000","last_undegraded":"2024-09-10T09:54:00.847203+0000","last_fullsized":"2024-09-10T09:54:00.847203+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:24:31.262207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"96'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.705105+0000","last_change":"2024-09-10T09:52:22.327641+0000","last_active":"2024-09-10T09:54:29.705105+0000","last_peered":"2024-09-10T09:54:29.705105+0000","last_clean":"2024-09-10T09:54:29.705105+0000","last_became_active":"2024-09-10T09:52:22.326969+0000","last_became_peered":"2024-09-10T09:52:22.326969+0000","last_unstale":"2024-09-10T09:54:29.705105+0000","last_undegraded":"2024-09-10T09:54:29.705105+0000","last_fullsized":"2024-09-10T09:54:29.705105+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:35:27.825534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.951014+0000","last_change":"2024-09-10T09:52:31.372360+0000","last_active":"2024-09-10T09:54:00.951014+0000","last_peered":"2024-09-10T09:54:00.951014+0000","last_clean":"2024-09-10T09:54:00.951014+0000","last_became_active":"2024-09-10T09:52:31.371957+0000","last_became_peered":"2024-09-10T09:52:31.371957+0000","last_unstale":"2024-09-10T09:54:00.951014+0000","last_undegraded":"2024-09-10T09:54:00.951014+0000","last_fullsized":"2024-09-10T09:54:00.951014+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:49:34.776185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"96'13","reported_seq":55,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:51.750986+0000","last_change":"2024-09-10T09:52:19.843532+0000","last_active":"2024-09-10T09:54:51.750986+0000","last_peered":"2024-09-10T09:54:51.750986+0000","last_clean":"2024-09-10T09:54:51.750986+0000","last_became_active":"2024-09-10T09:52:19.843070+0000","last_became_peered":"2024-09-10T09:52:19.843070+0000","last_unstale":"2024-09-10T09:54:51.750986+0000","last_undegraded":"2024-09-10T09:54:51.750986+0000","last_fullsized":"2024-09-10T09:54:51.750986+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:08:02.223373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"96'9","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.692662+0000","last_change":"2024-09-10T09:52:22.316981+0000","last_active":"2024-09-10T09:54:29.692662+0000","last_peered":"2024-09-10T09:54:29.692662+0000","last_clean":"2024-09-10T09:54:29.692662+0000","last_became_active":"2024-09-10T09:52:22.316648+0000","last_became_peered":"2024-09-10T09:52:22.316648+0000","last_unstale":"2024-09-10T09:54:29.692662+0000","last_undegraded":"2024-09-10T09:54:29.692662+0000","last_fullsized":"2024-09-10T09:54:29.692662+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:28:56.742496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.951485+0000","last_change":"2024-09-10T09:52:31.376811+0000","last_active":"2024-09-10T09:54:00.951485+0000","last_peered":"2024-09-10T09:54:00.951485+0000","last_clean":"2024-09-10T09:54:00.951485+0000","last_became_active":"2024-09-10T09:52:31.376007+0000","last_became_peered":"2024-09-10T09:52:31.376007+0000","last_unstale":"2024-09-10T09:54:00.951485+0000","last_undegraded":"2024-09-10T09:54:00.951485+0000","last_fullsized":"2024-09-10T09:54:00.951485+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:29:26.213281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"94'5","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.149772+0000","last_change":"2024-09-10T09:52:19.837848+0000","last_active":"2024-09-10T09:54:01.149772+0000","last_peered":"2024-09-10T09:54:01.149772+0000","last_clean":"2024-09-10T09:54:01.149772+0000","last_became_active":"2024-09-10T09:52:19.837383+0000","last_became_peered":"2024-09-10T09:52:19.837383+0000","last_unstale":"2024-09-10T09:54:01.149772+0000","last_undegraded":"2024-09-10T09:54:01.149772+0000","last_fullsized":"2024-09-10T09:54:01.149772+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:50:42.813665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"96'15","reported_seq":60,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.685844+0000","last_change":"2024-09-10T09:54:01.060407+0000","last_active":"2024-09-10T09:54:29.685844+0000","last_peered":"2024-09-10T09:54:29.685844+0000","last_clean":"2024-09-10T09:54:29.685844+0000","last_became_active":"2024-09-10T09:54:01.060126+0000","last_became_peered":"2024-09-10T09:54:01.060126+0000","last_unstale":"2024-09-10T09:54:29.685844+0000","last_undegraded":"2024-09-10T09:54:29.685844+0000","last_fullsized":"2024-09-10T09:54:29.685844+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:59:29.765375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.951119+0000","last_change":"2024-09-10T09:52:31.370910+0000","last_active":"2024-09-10T09:54:00.951119+0000","last_peered":"2024-09-10T09:54:00.951119+0000","last_clean":"2024-09-10T09:54:00.951119+0000","last_became_active":"2024-09-10T09:52:31.370467+0000","last_became_peered":"2024-09-10T09:52:31.370467+0000","last_unstale":"2024-09-10T09:54:00.951119+0000","last_undegraded":"2024-09-10T09:54:00.951119+0000","last_fullsized":"2024-09-10T09:54:00.951119+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:49:25.037820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"96'5","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.702762+0000","last_change":"2024-09-10T09:52:22.310708+0000","last_active":"2024-09-10T09:54:29.702762+0000","last_peered":"2024-09-10T09:54:29.702762+0000","last_clean":"2024-09-10T09:54:29.702762+0000","last_became_active":"2024-09-10T09:52:22.310387+0000","last_became_peered":"2024-09-10T09:52:22.310387+0000","last_unstale":"2024-09-10T09:54:29.702762+0000","last_undegraded":"2024-09-10T09:54:29.702762+0000","last_fullsized":"2024-09-10T09:54:29.702762+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:35:03.115408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"94'4","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.149453+0000","last_change":"2024-09-10T09:52:19.841417+0000","last_active":"2024-09-10T09:54:01.149453+0000","last_peered":"2024-09-10T09:54:01.149453+0000","last_clean":"2024-09-10T09:54:01.149453+0000","last_became_active":"2024-09-10T09:52:19.841083+0000","last_became_peered":"2024-09-10T09:52:19.841083+0000","last_unstale":"2024-09-10T09:54:01.149453+0000","last_undegraded":"2024-09-10T09:54:01.149453+0000","last_fullsized":"2024-09-10T09:54:01.149453+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:01:19.008345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.951294+0000","last_change":"2024-09-10T09:52:31.367406+0000","last_active":"2024-09-10T09:54:00.951294+0000","last_peered":"2024-09-10T09:54:00.951294+0000","last_clean":"2024-09-10T09:54:00.951294+0000","last_became_active":"2024-09-10T09:52:31.366961+0000","last_became_peered":"2024-09-10T09:52:31.366961+0000","last_unstale":"2024-09-10T09:54:00.951294+0000","last_undegraded":"2024-09-10T09:54:00.951294+0000","last_fullsized":"2024-09-10T09:54:00.951294+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:04:51.714465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"96'8","reported_seq":19,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.705252+0000","last_change":"2024-09-10T09:54:01.249212+0000","last_active":"2024-09-10T09:54:29.705252+0000","last_peered":"2024-09-10T09:54:29.705252+0000","last_clean":"2024-09-10T09:54:29.705252+0000","last_became_active":"2024-09-10T09:54:01.248840+0000","last_became_peered":"2024-09-10T09:54:01.248840+0000","last_unstale":"2024-09-10T09:54:29.705252+0000","last_undegraded":"2024-09-10T09:54:29.705252+0000","last_fullsized":"2024-09-10T09:54:29.705252+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:39:13.167435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,10],"acting":[9,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.a","version":"94'3","reported_seq":19,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.954964+0000","last_change":"2024-09-10T09:53:00.800421+0000","last_active":"2024-09-10T09:54:00.954964+0000","last_peered":"2024-09-10T09:54:00.954964+0000","last_clean":"2024-09-10T09:54:00.954964+0000","last_became_active":"2024-09-10T09:53:00.800026+0000","last_became_peered":"2024-09-10T09:53:00.800026+0000","last_unstale":"2024-09-10T09:54:00.954964+0000","last_undegraded":"2024-09-10T09:54:00.954964+0000","last_fullsized":"2024-09-10T09:54:00.954964+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:57:45.252420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"94'1","reported_seq":30,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.951388+0000","last_change":"2024-09-10T09:52:31.376605+0000","last_active":"2024-09-10T09:54:00.951388+0000","last_peered":"2024-09-10T09:54:00.951388+0000","last_clean":"2024-09-10T09:54:00.951388+0000","last_became_active":"2024-09-10T09:52:31.376083+0000","last_became_peered":"2024-09-10T09:52:31.376083+0000","last_unstale":"2024-09-10T09:54:00.951388+0000","last_undegraded":"2024-09-10T09:54:00.951388+0000","last_fullsized":"2024-09-10T09:54:00.951388+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:05:10.030673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"96'9","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.697029+0000","last_change":"2024-09-10T09:52:22.327303+0000","last_active":"2024-09-10T09:54:29.697029+0000","last_peered":"2024-09-10T09:54:29.697029+0000","last_clean":"2024-09-10T09:54:29.697029+0000","last_became_active":"2024-09-10T09:52:22.326753+0000","last_became_peered":"2024-09-10T09:52:22.326753+0000","last_unstale":"2024-09-10T09:54:29.697029+0000","last_undegraded":"2024-09-10T09:54:29.697029+0000","last_fullsized":"2024-09-10T09:54:29.697029+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:27:26.668151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"94'7","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.845956+0000","last_change":"2024-09-10T09:52:19.836484+0000","last_active":"2024-09-10T09:54:00.845956+0000","last_peered":"2024-09-10T09:54:00.845956+0000","last_clean":"2024-09-10T09:54:00.845956+0000","last_became_active":"2024-09-10T09:52:19.836130+0000","last_became_peered":"2024-09-10T09:52:19.836130+0000","last_unstale":"2024-09-10T09:54:00.845956+0000","last_undegraded":"2024-09-10T09:54:00.845956+0000","last_fullsized":"2024-09-10T09:54:00.845956+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:31:58.731981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.844293+0000","last_change":"2024-09-10T09:52:31.375279+0000","last_active":"2024-09-10T09:54:00.844293+0000","last_peered":"2024-09-10T09:54:00.844293+0000","last_clean":"2024-09-10T09:54:00.844293+0000","last_became_active":"2024-09-10T09:52:31.374181+0000","last_became_peered":"2024-09-10T09:52:31.374181+0000","last_unstale":"2024-09-10T09:54:00.844293+0000","last_undegraded":"2024-09-10T09:54:00.844293+0000","last_fullsized":"2024-09-10T09:54:00.844293+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:54:51.299224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"96'2","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.676489+0000","last_change":"2024-09-10T09:52:22.314817+0000","last_active":"2024-09-10T09:54:29.676489+0000","last_peered":"2024-09-10T09:54:29.676489+0000","last_clean":"2024-09-10T09:54:29.676489+0000","last_became_active":"2024-09-10T09:52:22.314491+0000","last_became_peered":"2024-09-10T09:52:22.314491+0000","last_unstale":"2024-09-10T09:54:29.676489+0000","last_undegraded":"2024-09-10T09:54:29.676489+0000","last_fullsized":"2024-09-10T09:54:29.676489+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:49:41.941667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"94'2","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.988558+0000","last_change":"2024-09-10T09:52:19.911730+0000","last_active":"2024-09-10T09:54:00.988558+0000","last_peered":"2024-09-10T09:54:00.988558+0000","last_clean":"2024-09-10T09:54:00.988558+0000","last_became_active":"2024-09-10T09:52:19.911393+0000","last_became_peered":"2024-09-10T09:52:19.911393+0000","last_unstale":"2024-09-10T09:54:00.988558+0000","last_undegraded":"2024-09-10T09:54:00.988558+0000","last_fullsized":"2024-09-10T09:54:00.988558+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:46:17.943129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.951301+0000","last_change":"2024-09-10T09:52:31.376498+0000","last_active":"2024-09-10T09:54:00.951301+0000","last_peered":"2024-09-10T09:54:00.951301+0000","last_clean":"2024-09-10T09:54:00.951301+0000","last_became_active":"2024-09-10T09:52:31.375966+0000","last_became_peered":"2024-09-10T09:52:31.375966+0000","last_unstale":"2024-09-10T09:54:00.951301+0000","last_undegraded":"2024-09-10T09:54:00.951301+0000","last_fullsized":"2024-09-10T09:54:00.951301+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:08:30.649399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.705573+0000","last_change":"2024-09-10T09:52:22.331611+0000","last_active":"2024-09-10T09:54:29.705573+0000","last_peered":"2024-09-10T09:54:29.705573+0000","last_clean":"2024-09-10T09:54:29.705573+0000","last_became_active":"2024-09-10T09:52:22.331268+0000","last_became_peered":"2024-09-10T09:52:22.331268+0000","last_unstale":"2024-09-10T09:54:29.705573+0000","last_undegraded":"2024-09-10T09:54:29.705573+0000","last_fullsized":"2024-09-10T09:54:29.705573+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:01:05.433994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"94'7","reported_seq":50,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.951927+0000","last_change":"2024-09-10T09:52:19.838387+0000","last_active":"2024-09-10T09:54:00.951927+0000","last_peered":"2024-09-10T09:54:00.951927+0000","last_clean":"2024-09-10T09:54:00.951927+0000","last_became_active":"2024-09-10T09:52:19.837889+0000","last_became_peered":"2024-09-10T09:52:19.837889+0000","last_unstale":"2024-09-10T09:54:00.951927+0000","last_undegraded":"2024-09-10T09:54:00.951927+0000","last_fullsized":"2024-09-10T09:54:00.951927+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:50:00.187243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.844187+0000","last_change":"2024-09-10T09:52:31.363262+0000","last_active":"2024-09-10T09:54:00.844187+0000","last_peered":"2024-09-10T09:54:00.844187+0000","last_clean":"2024-09-10T09:54:00.844187+0000","last_became_active":"2024-09-10T09:52:31.362762+0000","last_became_peered":"2024-09-10T09:52:31.362762+0000","last_unstale":"2024-09-10T09:54:00.844187+0000","last_undegraded":"2024-09-10T09:54:00.844187+0000","last_fullsized":"2024-09-10T09:54:00.844187+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:48:28.275041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"96'5","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.702169+0000","last_change":"2024-09-10T09:52:22.312472+0000","last_active":"2024-09-10T09:54:29.702169+0000","last_peered":"2024-09-10T09:54:29.702169+0000","last_clean":"2024-09-10T09:54:29.702169+0000","last_became_active":"2024-09-10T09:52:22.312117+0000","last_became_peered":"2024-09-10T09:52:22.312117+0000","last_unstale":"2024-09-10T09:54:29.702169+0000","last_undegraded":"2024-09-10T09:54:29.702169+0000","last_fullsized":"2024-09-10T09:54:29.702169+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:39:10.742137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"96'16","reported_seq":57,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:31.212993+0000","last_change":"2024-09-10T09:52:19.835543+0000","last_active":"2024-09-10T09:54:31.212993+0000","last_peered":"2024-09-10T09:54:31.212993+0000","last_clean":"2024-09-10T09:54:31.212993+0000","last_became_active":"2024-09-10T09:52:19.835170+0000","last_became_peered":"2024-09-10T09:52:19.835170+0000","last_unstale":"2024-09-10T09:54:31.212993+0000","last_undegraded":"2024-09-10T09:54:31.212993+0000","last_fullsized":"2024-09-10T09:54:31.212993+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:37.969277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.845447+0000","last_change":"2024-09-10T09:52:31.372398+0000","last_active":"2024-09-10T09:54:00.845447+0000","last_peered":"2024-09-10T09:54:00.845447+0000","last_clean":"2024-09-10T09:54:00.845447+0000","last_became_active":"2024-09-10T09:52:31.371874+0000","last_became_peered":"2024-09-10T09:52:31.371874+0000","last_unstale":"2024-09-10T09:54:00.845447+0000","last_undegraded":"2024-09-10T09:54:00.845447+0000","last_fullsized":"2024-09-10T09:54:00.845447+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:09:45.374320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"96'9","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.700381+0000","last_change":"2024-09-10T09:52:22.319592+0000","last_active":"2024-09-10T09:54:29.700381+0000","last_peered":"2024-09-10T09:54:29.700381+0000","last_clean":"2024-09-10T09:54:29.700381+0000","last_became_active":"2024-09-10T09:52:22.319298+0000","last_became_peered":"2024-09-10T09:52:22.319298+0000","last_unstale":"2024-09-10T09:54:29.700381+0000","last_undegraded":"2024-09-10T09:54:29.700381+0000","last_fullsized":"2024-09-10T09:54:29.700381+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:56:59.787638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"94'4","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.150011+0000","last_change":"2024-09-10T09:52:19.912923+0000","last_active":"2024-09-10T09:54:01.150011+0000","last_peered":"2024-09-10T09:54:01.150011+0000","last_clean":"2024-09-10T09:54:01.150011+0000","last_became_active":"2024-09-10T09:52:19.912268+0000","last_became_peered":"2024-09-10T09:52:19.912268+0000","last_unstale":"2024-09-10T09:54:01.150011+0000","last_undegraded":"2024-09-10T09:54:01.150011+0000","last_fullsized":"2024-09-10T09:54:01.150011+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:06:45.421004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.955091+0000","last_change":"2024-09-10T09:52:31.372761+0000","last_active":"2024-09-10T09:54:00.955091+0000","last_peered":"2024-09-10T09:54:00.955091+0000","last_clean":"2024-09-10T09:54:00.955091+0000","last_became_active":"2024-09-10T09:52:31.372293+0000","last_became_peered":"2024-09-10T09:52:31.372293+0000","last_unstale":"2024-09-10T09:54:00.955091+0000","last_undegraded":"2024-09-10T09:54:00.955091+0000","last_fullsized":"2024-09-10T09:54:00.955091+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:36:41.446810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"96'4","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.688817+0000","last_change":"2024-09-10T09:52:22.326291+0000","last_active":"2024-09-10T09:54:29.688817+0000","last_peered":"2024-09-10T09:54:29.688817+0000","last_clean":"2024-09-10T09:54:29.688817+0000","last_became_active":"2024-09-10T09:52:22.325985+0000","last_became_peered":"2024-09-10T09:52:22.325985+0000","last_unstale":"2024-09-10T09:54:29.688817+0000","last_undegraded":"2024-09-10T09:54:29.688817+0000","last_fullsized":"2024-09-10T09:54:29.688817+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:06:21.871502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"94'1","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.844068+0000","last_change":"2024-09-10T09:52:19.912851+0000","last_active":"2024-09-10T09:54:00.844068+0000","last_peered":"2024-09-10T09:54:00.844068+0000","last_clean":"2024-09-10T09:54:00.844068+0000","last_became_active":"2024-09-10T09:52:19.912200+0000","last_became_peered":"2024-09-10T09:52:19.912200+0000","last_unstale":"2024-09-10T09:54:00.844068+0000","last_undegraded":"2024-09-10T09:54:00.844068+0000","last_fullsized":"2024-09-10T09:54:00.844068+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:00:47.331755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.149311+0000","last_change":"2024-09-10T09:52:31.371329+0000","last_active":"2024-09-10T09:54:01.149311+0000","last_peered":"2024-09-10T09:54:01.149311+0000","last_clean":"2024-09-10T09:54:01.149311+0000","last_became_active":"2024-09-10T09:52:31.370971+0000","last_became_peered":"2024-09-10T09:52:31.370971+0000","last_unstale":"2024-09-10T09:54:01.149311+0000","last_undegraded":"2024-09-10T09:54:01.149311+0000","last_fullsized":"2024-09-10T09:54:01.149311+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:07:12.928421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"96'7","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.686209+0000","last_change":"2024-09-10T09:52:22.328039+0000","last_active":"2024-09-10T09:54:29.686209+0000","last_peered":"2024-09-10T09:54:29.686209+0000","last_clean":"2024-09-10T09:54:29.686209+0000","last_became_active":"2024-09-10T09:52:22.327588+0000","last_became_peered":"2024-09-10T09:52:22.327588+0000","last_unstale":"2024-09-10T09:54:29.686209+0000","last_undegraded":"2024-09-10T09:54:29.686209+0000","last_fullsized":"2024-09-10T09:54:29.686209+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:35:39.219544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"96'6","reported_seq":150,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:55:21.067336+0000","last_change":"2024-09-10T09:53:00.797651+0000","last_active":"2024-09-10T09:55:21.067336+0000","last_peered":"2024-09-10T09:55:21.067336+0000","last_clean":"2024-09-10T09:55:21.067336+0000","last_became_active":"2024-09-10T09:53:00.797285+0000","last_became_peered":"2024-09-10T09:53:00.797285+0000","last_unstale":"2024-09-10T09:55:21.067336+0000","last_undegraded":"2024-09-10T09:55:21.067336+0000","last_fullsized":"2024-09-10T09:55:21.067336+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:18:02.320032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":899,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":128,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,6,1],"acting":[11,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.841787+0000","last_change":"2024-09-10T09:52:31.379215+0000","last_active":"2024-09-10T09:54:00.841787+0000","last_peered":"2024-09-10T09:54:00.841787+0000","last_clean":"2024-09-10T09:54:00.841787+0000","last_became_active":"2024-09-10T09:52:31.377107+0000","last_became_peered":"2024-09-10T09:52:31.377107+0000","last_unstale":"2024-09-10T09:54:00.841787+0000","last_undegraded":"2024-09-10T09:54:00.841787+0000","last_fullsized":"2024-09-10T09:54:00.841787+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:09:39.474077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"96'8","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.673518+0000","last_change":"2024-09-10T09:52:22.316474+0000","last_active":"2024-09-10T09:54:29.673518+0000","last_peered":"2024-09-10T09:54:29.673518+0000","last_clean":"2024-09-10T09:54:29.673518+0000","last_became_active":"2024-09-10T09:52:22.316054+0000","last_became_peered":"2024-09-10T09:52:22.316054+0000","last_unstale":"2024-09-10T09:54:29.673518+0000","last_undegraded":"2024-09-10T09:54:29.673518+0000","last_fullsized":"2024-09-10T09:54:29.673518+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:11:31.334780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"96'6","reported_seq":171,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:55:17.091036+0000","last_change":"2024-09-10T09:52:19.911601+0000","last_active":"2024-09-10T09:55:17.091036+0000","last_peered":"2024-09-10T09:55:17.091036+0000","last_clean":"2024-09-10T09:55:17.091036+0000","last_became_active":"2024-09-10T09:52:19.911268+0000","last_became_peered":"2024-09-10T09:52:19.911268+0000","last_unstale":"2024-09-10T09:55:17.091036+0000","last_undegraded":"2024-09-10T09:55:17.091036+0000","last_fullsized":"2024-09-10T09:55:17.091036+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:46:38.158784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":123,"num_read_kb":123,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.841796+0000","last_change":"2024-09-10T09:52:31.375938+0000","last_active":"2024-09-10T09:54:00.841796+0000","last_peered":"2024-09-10T09:54:00.841796+0000","last_clean":"2024-09-10T09:54:00.841796+0000","last_became_active":"2024-09-10T09:52:31.375536+0000","last_became_peered":"2024-09-10T09:52:31.375536+0000","last_unstale":"2024-09-10T09:54:00.841796+0000","last_undegraded":"2024-09-10T09:54:00.841796+0000","last_fullsized":"2024-09-10T09:54:00.841796+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:55:17.303809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"96'5","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.688934+0000","last_change":"2024-09-10T09:52:22.315912+0000","last_active":"2024-09-10T09:54:29.688934+0000","last_peered":"2024-09-10T09:54:29.688934+0000","last_clean":"2024-09-10T09:54:29.688934+0000","last_became_active":"2024-09-10T09:52:22.315433+0000","last_became_peered":"2024-09-10T09:52:22.315433+0000","last_unstale":"2024-09-10T09:54:29.688934+0000","last_undegraded":"2024-09-10T09:54:29.688934+0000","last_fullsized":"2024-09-10T09:54:29.688934+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:57:10.454716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"94'4","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.059138+0000","last_change":"2024-09-10T09:52:19.838966+0000","last_active":"2024-09-10T09:54:01.059138+0000","last_peered":"2024-09-10T09:54:01.059138+0000","last_clean":"2024-09-10T09:54:01.059138+0000","last_became_active":"2024-09-10T09:52:19.838545+0000","last_became_peered":"2024-09-10T09:52:19.838545+0000","last_unstale":"2024-09-10T09:54:01.059138+0000","last_undegraded":"2024-09-10T09:54:01.059138+0000","last_fullsized":"2024-09-10T09:54:01.059138+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:07:00.930146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.847601+0000","last_change":"2024-09-10T09:53:00.802039+0000","last_active":"2024-09-10T09:54:00.847601+0000","last_peered":"2024-09-10T09:54:00.847601+0000","last_clean":"2024-09-10T09:54:00.847601+0000","last_became_active":"2024-09-10T09:53:00.800223+0000","last_became_peered":"2024-09-10T09:53:00.800223+0000","last_unstale":"2024-09-10T09:54:00.847601+0000","last_undegraded":"2024-09-10T09:54:00.847601+0000","last_fullsized":"2024-09-10T09:54:00.847601+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:50:16.026631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,6,10],"acting":[5,8,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"96'8","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.698863+0000","last_change":"2024-09-10T09:52:22.313819+0000","last_active":"2024-09-10T09:54:29.698863+0000","last_peered":"2024-09-10T09:54:29.698863+0000","last_clean":"2024-09-10T09:54:29.698863+0000","last_became_active":"2024-09-10T09:52:22.313473+0000","last_became_peered":"2024-09-10T09:52:22.313473+0000","last_unstale":"2024-09-10T09:54:29.698863+0000","last_undegraded":"2024-09-10T09:54:29.698863+0000","last_fullsized":"2024-09-10T09:54:29.698863+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:44:10.003187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"94'6","reported_seq":51,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.951818+0000","last_change":"2024-09-10T09:52:19.837376+0000","last_active":"2024-09-10T09:54:00.951818+0000","last_peered":"2024-09-10T09:54:00.951818+0000","last_clean":"2024-09-10T09:54:00.951818+0000","last_became_active":"2024-09-10T09:52:19.836880+0000","last_became_peered":"2024-09-10T09:52:19.836880+0000","last_unstale":"2024-09-10T09:54:00.951818+0000","last_undegraded":"2024-09-10T09:54:00.951818+0000","last_fullsized":"2024-09-10T09:54:00.951818+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:48:58.198188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":361,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.059392+0000","last_change":"2024-09-10T09:51:05.871529+0000","last_active":"2024-09-10T09:54:01.059392+0000","last_peered":"2024-09-10T09:54:01.059392+0000","last_clean":"2024-09-10T09:54:01.059392+0000","last_became_active":"2024-09-10T09:51:05.871115+0000","last_became_peered":"2024-09-10T09:51:05.871115+0000","last_unstale":"2024-09-10T09:54:01.059392+0000","last_undegraded":"2024-09-10T09:54:01.059392+0000","last_fullsized":"2024-09-10T09:54:01.059392+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:47:24.175358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:47:24.175358+0000","last_clean_scrub_stamp":"2024-09-10T09:47:24.175358+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:58:30.247077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"94'1","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.059534+0000","last_change":"2024-09-10T09:53:00.800474+0000","last_active":"2024-09-10T09:54:01.059534+0000","last_peered":"2024-09-10T09:54:01.059534+0000","last_clean":"2024-09-10T09:54:01.059534+0000","last_became_active":"2024-09-10T09:53:00.799932+0000","last_became_peered":"2024-09-10T09:53:00.799932+0000","last_unstale":"2024-09-10T09:54:01.059534+0000","last_undegraded":"2024-09-10T09:54:01.059534+0000","last_fullsized":"2024-09-10T09:54:01.059534+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:14:50.998952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,11,9],"acting":[7,8,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.693609+0000","last_change":"2024-09-10T09:52:22.312521+0000","last_active":"2024-09-10T09:54:29.693609+0000","last_peered":"2024-09-10T09:54:29.693609+0000","last_clean":"2024-09-10T09:54:29.693609+0000","last_became_active":"2024-09-10T09:52:22.312107+0000","last_became_peered":"2024-09-10T09:52:22.312107+0000","last_unstale":"2024-09-10T09:54:29.693609+0000","last_undegraded":"2024-09-10T09:54:29.693609+0000","last_fullsized":"2024-09-10T09:54:29.693609+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:14:19.799259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"94'5","reported_seq":176,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:55:22.067694+0000","last_change":"2024-09-10T09:52:19.911454+0000","last_active":"2024-09-10T09:55:22.067694+0000","last_peered":"2024-09-10T09:55:22.067694+0000","last_clean":"2024-09-10T09:55:22.067694+0000","last_became_active":"2024-09-10T09:52:19.911079+0000","last_became_peered":"2024-09-10T09:52:19.911079+0000","last_unstale":"2024-09-10T09:55:22.067694+0000","last_undegraded":"2024-09-10T09:55:22.067694+0000","last_fullsized":"2024-09-10T09:55:22.067694+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:43:38.787689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":129,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.988531+0000","last_change":"2024-09-10T09:52:31.381877+0000","last_active":"2024-09-10T09:54:00.988531+0000","last_peered":"2024-09-10T09:54:00.988531+0000","last_clean":"2024-09-10T09:54:00.988531+0000","last_became_active":"2024-09-10T09:52:31.381472+0000","last_became_peered":"2024-09-10T09:52:31.381472+0000","last_unstale":"2024-09-10T09:54:00.988531+0000","last_undegraded":"2024-09-10T09:54:00.988531+0000","last_fullsized":"2024-09-10T09:54:00.988531+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:41:24.686325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"96'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.702544+0000","last_change":"2024-09-10T09:52:22.313902+0000","last_active":"2024-09-10T09:54:29.702544+0000","last_peered":"2024-09-10T09:54:29.702544+0000","last_clean":"2024-09-10T09:54:29.702544+0000","last_became_active":"2024-09-10T09:52:22.313515+0000","last_became_peered":"2024-09-10T09:52:22.313515+0000","last_unstale":"2024-09-10T09:54:29.702544+0000","last_undegraded":"2024-09-10T09:54:29.702544+0000","last_fullsized":"2024-09-10T09:54:29.702544+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:25:00.238307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"94'3","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.847873+0000","last_change":"2024-09-10T09:52:19.837736+0000","last_active":"2024-09-10T09:54:00.847873+0000","last_peered":"2024-09-10T09:54:00.847873+0000","last_clean":"2024-09-10T09:54:00.847873+0000","last_became_active":"2024-09-10T09:52:19.837370+0000","last_became_peered":"2024-09-10T09:52:19.837370+0000","last_unstale":"2024-09-10T09:54:00.847873+0000","last_undegraded":"2024-09-10T09:54:00.847873+0000","last_fullsized":"2024-09-10T09:54:00.847873+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:54:35.753027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.248213+0000","last_change":"2024-09-10T09:52:31.378945+0000","last_active":"2024-09-10T09:54:01.248213+0000","last_peered":"2024-09-10T09:54:01.248213+0000","last_clean":"2024-09-10T09:54:01.248213+0000","last_became_active":"2024-09-10T09:52:31.378144+0000","last_became_peered":"2024-09-10T09:52:31.378144+0000","last_unstale":"2024-09-10T09:54:01.248213+0000","last_undegraded":"2024-09-10T09:54:01.248213+0000","last_fullsized":"2024-09-10T09:54:01.248213+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:36:06.288293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"96'9","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.692039+0000","last_change":"2024-09-10T09:52:22.316400+0000","last_active":"2024-09-10T09:54:29.692039+0000","last_peered":"2024-09-10T09:54:29.692039+0000","last_clean":"2024-09-10T09:54:29.692039+0000","last_became_active":"2024-09-10T09:52:22.315974+0000","last_became_peered":"2024-09-10T09:52:22.315974+0000","last_unstale":"2024-09-10T09:54:29.692039+0000","last_undegraded":"2024-09-10T09:54:29.692039+0000","last_fullsized":"2024-09-10T09:54:29.692039+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:31:05.538237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"94'5","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.841976+0000","last_change":"2024-09-10T09:52:19.837942+0000","last_active":"2024-09-10T09:54:00.841976+0000","last_peered":"2024-09-10T09:54:00.841976+0000","last_clean":"2024-09-10T09:54:00.841976+0000","last_became_active":"2024-09-10T09:52:19.837629+0000","last_became_peered":"2024-09-10T09:52:19.837629+0000","last_unstale":"2024-09-10T09:54:00.841976+0000","last_undegraded":"2024-09-10T09:54:00.841976+0000","last_fullsized":"2024-09-10T09:54:00.841976+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:34:13.207770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.847019+0000","last_change":"2024-09-10T09:52:31.364231+0000","last_active":"2024-09-10T09:54:00.847019+0000","last_peered":"2024-09-10T09:54:00.847019+0000","last_clean":"2024-09-10T09:54:00.847019+0000","last_became_active":"2024-09-10T09:52:31.363833+0000","last_became_peered":"2024-09-10T09:52:31.363833+0000","last_unstale":"2024-09-10T09:54:00.847019+0000","last_undegraded":"2024-09-10T09:54:00.847019+0000","last_fullsized":"2024-09-10T09:54:00.847019+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:53:51.435700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"96'5","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.680562+0000","last_change":"2024-09-10T09:52:22.319530+0000","last_active":"2024-09-10T09:54:29.680562+0000","last_peered":"2024-09-10T09:54:29.680562+0000","last_clean":"2024-09-10T09:54:29.680562+0000","last_became_active":"2024-09-10T09:52:22.318742+0000","last_became_peered":"2024-09-10T09:52:22.318742+0000","last_unstale":"2024-09-10T09:54:29.680562+0000","last_undegraded":"2024-09-10T09:54:29.680562+0000","last_fullsized":"2024-09-10T09:54:29.680562+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:44:35.050454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"94'2","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.150048+0000","last_change":"2024-09-10T09:52:19.836398+0000","last_active":"2024-09-10T09:54:01.150048+0000","last_peered":"2024-09-10T09:54:01.150048+0000","last_clean":"2024-09-10T09:54:01.150048+0000","last_became_active":"2024-09-10T09:52:19.836055+0000","last_became_peered":"2024-09-10T09:52:19.836055+0000","last_unstale":"2024-09-10T09:54:01.150048+0000","last_undegraded":"2024-09-10T09:54:01.150048+0000","last_fullsized":"2024-09-10T09:54:01.150048+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:49:26.923367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.847215+0000","last_change":"2024-09-10T09:52:31.371109+0000","last_active":"2024-09-10T09:54:00.847215+0000","last_peered":"2024-09-10T09:54:00.847215+0000","last_clean":"2024-09-10T09:54:00.847215+0000","last_became_active":"2024-09-10T09:52:31.370592+0000","last_became_peered":"2024-09-10T09:52:31.370592+0000","last_unstale":"2024-09-10T09:54:00.847215+0000","last_undegraded":"2024-09-10T09:54:00.847215+0000","last_fullsized":"2024-09-10T09:54:00.847215+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:20:23.585187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"96'6","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.685236+0000","last_change":"2024-09-10T09:52:22.319365+0000","last_active":"2024-09-10T09:54:29.685236+0000","last_peered":"2024-09-10T09:54:29.685236+0000","last_clean":"2024-09-10T09:54:29.685236+0000","last_became_active":"2024-09-10T09:52:22.319064+0000","last_became_peered":"2024-09-10T09:52:22.319064+0000","last_unstale":"2024-09-10T09:54:29.685236+0000","last_undegraded":"2024-09-10T09:54:29.685236+0000","last_fullsized":"2024-09-10T09:54:29.685236+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:14:50.089566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"94'17","reported_seq":59,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.149324+0000","last_change":"2024-09-10T09:52:19.911698+0000","last_active":"2024-09-10T09:54:01.149324+0000","last_peered":"2024-09-10T09:54:01.149324+0000","last_clean":"2024-09-10T09:54:01.149324+0000","last_became_active":"2024-09-10T09:52:19.911369+0000","last_became_peered":"2024-09-10T09:52:19.911369+0000","last_unstale":"2024-09-10T09:54:01.149324+0000","last_undegraded":"2024-09-10T09:54:01.149324+0000","last_fullsized":"2024-09-10T09:54:01.149324+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:40:39.045797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.841947+0000","last_change":"2024-09-10T09:52:31.380463+0000","last_active":"2024-09-10T09:54:00.841947+0000","last_peered":"2024-09-10T09:54:00.841947+0000","last_clean":"2024-09-10T09:54:00.841947+0000","last_became_active":"2024-09-10T09:52:31.380114+0000","last_became_peered":"2024-09-10T09:52:31.380114+0000","last_unstale":"2024-09-10T09:54:00.841947+0000","last_undegraded":"2024-09-10T09:54:00.841947+0000","last_fullsized":"2024-09-10T09:54:00.841947+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:19:50.869160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"96'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.682831+0000","last_change":"2024-09-10T09:52:22.313928+0000","last_active":"2024-09-10T09:54:29.682831+0000","last_peered":"2024-09-10T09:54:29.682831+0000","last_clean":"2024-09-10T09:54:29.682831+0000","last_became_active":"2024-09-10T09:52:22.313510+0000","last_became_peered":"2024-09-10T09:52:22.313510+0000","last_unstale":"2024-09-10T09:54:29.682831+0000","last_undegraded":"2024-09-10T09:54:29.682831+0000","last_fullsized":"2024-09-10T09:54:29.682831+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:58:19.031178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"96'7","reported_seq":175,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:55:17.091890+0000","last_change":"2024-09-10T09:52:19.839451+0000","last_active":"2024-09-10T09:55:17.091890+0000","last_peered":"2024-09-10T09:55:17.091890+0000","last_clean":"2024-09-10T09:55:17.091890+0000","last_became_active":"2024-09-10T09:52:19.839124+0000","last_became_peered":"2024-09-10T09:52:19.839124+0000","last_unstale":"2024-09-10T09:55:17.091890+0000","last_undegraded":"2024-09-10T09:55:17.091890+0000","last_fullsized":"2024-09-10T09:55:17.091890+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:51:33.489830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":127,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.844344+0000","last_change":"2024-09-10T09:52:31.376386+0000","last_active":"2024-09-10T09:54:00.844344+0000","last_peered":"2024-09-10T09:54:00.844344+0000","last_clean":"2024-09-10T09:54:00.844344+0000","last_became_active":"2024-09-10T09:52:31.376066+0000","last_became_peered":"2024-09-10T09:52:31.376066+0000","last_unstale":"2024-09-10T09:54:00.844344+0000","last_undegraded":"2024-09-10T09:54:00.844344+0000","last_fullsized":"2024-09-10T09:54:00.844344+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:23:07.191842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"96'15","reported_seq":52,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.702901+0000","last_change":"2024-09-10T09:52:22.323590+0000","last_active":"2024-09-10T09:54:29.702901+0000","last_peered":"2024-09-10T09:54:29.702901+0000","last_clean":"2024-09-10T09:54:29.702901+0000","last_became_active":"2024-09-10T09:52:22.323264+0000","last_became_peered":"2024-09-10T09:52:22.323264+0000","last_unstale":"2024-09-10T09:54:29.702901+0000","last_undegraded":"2024-09-10T09:54:29.702901+0000","last_fullsized":"2024-09-10T09:54:29.702901+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:07:56.572337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"94'5","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.847940+0000","last_change":"2024-09-10T09:52:19.840620+0000","last_active":"2024-09-10T09:54:00.847940+0000","last_peered":"2024-09-10T09:54:00.847940+0000","last_clean":"2024-09-10T09:54:00.847940+0000","last_became_active":"2024-09-10T09:52:19.840217+0000","last_became_peered":"2024-09-10T09:52:19.840217+0000","last_unstale":"2024-09-10T09:54:00.847940+0000","last_undegraded":"2024-09-10T09:54:00.847940+0000","last_fullsized":"2024-09-10T09:54:00.847940+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:55:13.111993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.955071+0000","last_change":"2024-09-10T09:52:31.373895+0000","last_active":"2024-09-10T09:54:00.955071+0000","last_peered":"2024-09-10T09:54:00.955071+0000","last_clean":"2024-09-10T09:54:00.955071+0000","last_became_active":"2024-09-10T09:52:31.373474+0000","last_became_peered":"2024-09-10T09:52:31.373474+0000","last_unstale":"2024-09-10T09:54:00.955071+0000","last_undegraded":"2024-09-10T09:54:00.955071+0000","last_fullsized":"2024-09-10T09:54:00.955071+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:39:16.049859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"96'9","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.704377+0000","last_change":"2024-09-10T09:52:22.319294+0000","last_active":"2024-09-10T09:54:29.704377+0000","last_peered":"2024-09-10T09:54:29.704377+0000","last_clean":"2024-09-10T09:54:29.704377+0000","last_became_active":"2024-09-10T09:52:22.318918+0000","last_became_peered":"2024-09-10T09:52:22.318918+0000","last_unstale":"2024-09-10T09:54:29.704377+0000","last_undegraded":"2024-09-10T09:54:29.704377+0000","last_fullsized":"2024-09-10T09:54:29.704377+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:52:46.434182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"94'9","reported_seq":53,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.844244+0000","last_change":"2024-09-10T09:52:19.842286+0000","last_active":"2024-09-10T09:54:00.844244+0000","last_peered":"2024-09-10T09:54:00.844244+0000","last_clean":"2024-09-10T09:54:00.844244+0000","last_became_active":"2024-09-10T09:52:19.841930+0000","last_became_peered":"2024-09-10T09:52:19.841930+0000","last_unstale":"2024-09-10T09:54:00.844244+0000","last_undegraded":"2024-09-10T09:54:00.844244+0000","last_fullsized":"2024-09-10T09:54:00.844244+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:38:46.358389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.847531+0000","last_change":"2024-09-10T09:52:31.370961+0000","last_active":"2024-09-10T09:54:00.847531+0000","last_peered":"2024-09-10T09:54:00.847531+0000","last_clean":"2024-09-10T09:54:00.847531+0000","last_became_active":"2024-09-10T09:52:31.370545+0000","last_became_peered":"2024-09-10T09:52:31.370545+0000","last_unstale":"2024-09-10T09:54:00.847531+0000","last_undegraded":"2024-09-10T09:54:00.847531+0000","last_fullsized":"2024-09-10T09:54:00.847531+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:17.461447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"96'6","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.685895+0000","last_change":"2024-09-10T09:52:22.313215+0000","last_active":"2024-09-10T09:54:29.685895+0000","last_peered":"2024-09-10T09:54:29.685895+0000","last_clean":"2024-09-10T09:54:29.685895+0000","last_became_active":"2024-09-10T09:52:22.312918+0000","last_became_peered":"2024-09-10T09:52:22.312918+0000","last_unstale":"2024-09-10T09:54:29.685895+0000","last_undegraded":"2024-09-10T09:54:29.685895+0000","last_fullsized":"2024-09-10T09:54:29.685895+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:48:09.568004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"96'40","reported_seq":362,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:55:22.066770+0000","last_change":"2024-09-10T09:53:01.226146+0000","last_active":"2024-09-10T09:55:22.066770+0000","last_peered":"2024-09-10T09:55:22.066770+0000","last_clean":"2024-09-10T09:55:22.066770+0000","last_became_active":"2024-09-10T09:53:01.218622+0000","last_became_peered":"2024-09-10T09:53:01.218622+0000","last_unstale":"2024-09-10T09:55:22.066770+0000","last_undegraded":"2024-09-10T09:55:22.066770+0000","last_fullsized":"2024-09-10T09:55:22.066770+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:43:11.927029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67876,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":263,"num_read_kb":198,"num_write":42,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":3844,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.841475+0000","last_change":"2024-09-10T09:53:00.800650+0000","last_active":"2024-09-10T09:54:00.841475+0000","last_peered":"2024-09-10T09:54:00.841475+0000","last_clean":"2024-09-10T09:54:00.841475+0000","last_became_active":"2024-09-10T09:53:00.800104+0000","last_became_peered":"2024-09-10T09:53:00.800104+0000","last_unstale":"2024-09-10T09:54:00.841475+0000","last_undegraded":"2024-09-10T09:54:00.841475+0000","last_fullsized":"2024-09-10T09:54:00.841475+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:41:16.022884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.14","version":"96'11","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.690121+0000","last_change":"2024-09-10T09:52:22.317174+0000","last_active":"2024-09-10T09:54:29.690121+0000","last_peered":"2024-09-10T09:54:29.690121+0000","last_clean":"2024-09-10T09:54:29.690121+0000","last_became_active":"2024-09-10T09:52:22.316851+0000","last_became_peered":"2024-09-10T09:52:22.316851+0000","last_unstale":"2024-09-10T09:54:29.690121+0000","last_undegraded":"2024-09-10T09:54:29.690121+0000","last_fullsized":"2024-09-10T09:54:29.690121+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:55:18.742393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"94'5","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.847731+0000","last_change":"2024-09-10T09:52:19.912093+0000","last_active":"2024-09-10T09:54:00.847731+0000","last_peered":"2024-09-10T09:54:00.847731+0000","last_clean":"2024-09-10T09:54:00.847731+0000","last_became_active":"2024-09-10T09:52:19.911611+0000","last_became_peered":"2024-09-10T09:52:19.911611+0000","last_unstale":"2024-09-10T09:54:00.847731+0000","last_undegraded":"2024-09-10T09:54:00.847731+0000","last_fullsized":"2024-09-10T09:54:00.847731+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:01:10.859310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.951658+0000","last_change":"2024-09-10T09:52:31.362884+0000","last_active":"2024-09-10T09:54:00.951658+0000","last_peered":"2024-09-10T09:54:00.951658+0000","last_clean":"2024-09-10T09:54:00.951658+0000","last_became_active":"2024-09-10T09:52:31.362489+0000","last_became_peered":"2024-09-10T09:52:31.362489+0000","last_unstale":"2024-09-10T09:54:00.951658+0000","last_undegraded":"2024-09-10T09:54:00.951658+0000","last_fullsized":"2024-09-10T09:54:00.951658+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:02:33.084795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"96'9","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.692830+0000","last_change":"2024-09-10T09:52:22.325871+0000","last_active":"2024-09-10T09:54:29.692830+0000","last_peered":"2024-09-10T09:54:29.692830+0000","last_clean":"2024-09-10T09:54:29.692830+0000","last_became_active":"2024-09-10T09:52:22.324980+0000","last_became_peered":"2024-09-10T09:52:22.324980+0000","last_unstale":"2024-09-10T09:54:29.692830+0000","last_undegraded":"2024-09-10T09:54:29.692830+0000","last_fullsized":"2024-09-10T09:54:29.692830+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:31:55.713444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"94'4","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.059691+0000","last_change":"2024-09-10T09:52:19.839491+0000","last_active":"2024-09-10T09:54:01.059691+0000","last_peered":"2024-09-10T09:54:01.059691+0000","last_clean":"2024-09-10T09:54:01.059691+0000","last_became_active":"2024-09-10T09:52:19.839092+0000","last_became_peered":"2024-09-10T09:52:19.839092+0000","last_unstale":"2024-09-10T09:54:01.059691+0000","last_undegraded":"2024-09-10T09:54:01.059691+0000","last_fullsized":"2024-09-10T09:54:01.059691+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:42:13.453281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.843285+0000","last_change":"2024-09-10T09:52:31.371346+0000","last_active":"2024-09-10T09:54:00.843285+0000","last_peered":"2024-09-10T09:54:00.843285+0000","last_clean":"2024-09-10T09:54:00.843285+0000","last_became_active":"2024-09-10T09:52:31.370929+0000","last_became_peered":"2024-09-10T09:52:31.370929+0000","last_unstale":"2024-09-10T09:54:00.843285+0000","last_undegraded":"2024-09-10T09:54:00.843285+0000","last_fullsized":"2024-09-10T09:54:00.843285+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:44:22.925082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"96'10","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.694062+0000","last_change":"2024-09-10T09:52:22.317453+0000","last_active":"2024-09-10T09:54:29.694062+0000","last_peered":"2024-09-10T09:54:29.694062+0000","last_clean":"2024-09-10T09:54:29.694062+0000","last_became_active":"2024-09-10T09:52:22.315556+0000","last_became_peered":"2024-09-10T09:52:22.315556+0000","last_unstale":"2024-09-10T09:54:29.694062+0000","last_undegraded":"2024-09-10T09:54:29.694062+0000","last_fullsized":"2024-09-10T09:54:29.694062+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:56:05.860275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"96'5","reported_seq":300,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:55:22.066812+0000","last_change":"2024-09-10T09:52:19.835985+0000","last_active":"2024-09-10T09:55:22.066812+0000","last_peered":"2024-09-10T09:55:22.066812+0000","last_clean":"2024-09-10T09:55:22.066812+0000","last_became_active":"2024-09-10T09:52:19.834835+0000","last_became_peered":"2024-09-10T09:52:19.834835+0000","last_unstale":"2024-09-10T09:55:22.066812+0000","last_undegraded":"2024-09-10T09:55:22.066812+0000","last_fullsized":"2024-09-10T09:55:22.066812+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:09:11.590443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":252,"num_read_kb":252,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.845724+0000","last_change":"2024-09-10T09:52:31.368663+0000","last_active":"2024-09-10T09:54:00.845724+0000","last_peered":"2024-09-10T09:54:00.845724+0000","last_clean":"2024-09-10T09:54:00.845724+0000","last_became_active":"2024-09-10T09:52:31.368004+0000","last_became_peered":"2024-09-10T09:52:31.368004+0000","last_unstale":"2024-09-10T09:54:00.845724+0000","last_undegraded":"2024-09-10T09:54:00.845724+0000","last_fullsized":"2024-09-10T09:54:00.845724+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:56:15.100515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.247917+0000","last_change":"2024-09-10T09:52:31.370992+0000","last_active":"2024-09-10T09:54:01.247917+0000","last_peered":"2024-09-10T09:54:01.247917+0000","last_clean":"2024-09-10T09:54:01.247917+0000","last_became_active":"2024-09-10T09:52:31.370357+0000","last_became_peered":"2024-09-10T09:52:31.370357+0000","last_unstale":"2024-09-10T09:54:01.247917+0000","last_undegraded":"2024-09-10T09:54:01.247917+0000","last_fullsized":"2024-09-10T09:54:01.247917+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:02:49.482532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"96'11","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.706426+0000","last_change":"2024-09-10T09:52:22.319363+0000","last_active":"2024-09-10T09:54:29.706426+0000","last_peered":"2024-09-10T09:54:29.706426+0000","last_clean":"2024-09-10T09:54:29.706426+0000","last_became_active":"2024-09-10T09:52:22.319044+0000","last_became_peered":"2024-09-10T09:52:22.319044+0000","last_unstale":"2024-09-10T09:54:29.706426+0000","last_undegraded":"2024-09-10T09:54:29.706426+0000","last_fullsized":"2024-09-10T09:54:29.706426+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:30:51.328413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"94'3","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.847831+0000","last_change":"2024-09-10T09:52:19.841745+0000","last_active":"2024-09-10T09:54:00.847831+0000","last_peered":"2024-09-10T09:54:00.847831+0000","last_clean":"2024-09-10T09:54:00.847831+0000","last_became_active":"2024-09-10T09:52:19.841404+0000","last_became_peered":"2024-09-10T09:52:19.841404+0000","last_unstale":"2024-09-10T09:54:00.847831+0000","last_undegraded":"2024-09-10T09:54:00.847831+0000","last_fullsized":"2024-09-10T09:54:00.847831+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:44:36.346785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.955521+0000","last_change":"2024-09-10T09:53:01.218412+0000","last_active":"2024-09-10T09:54:00.955521+0000","last_peered":"2024-09-10T09:54:00.955521+0000","last_clean":"2024-09-10T09:54:00.955521+0000","last_became_active":"2024-09-10T09:53:01.217966+0000","last_became_peered":"2024-09-10T09:53:01.217966+0000","last_unstale":"2024-09-10T09:54:00.955521+0000","last_undegraded":"2024-09-10T09:54:00.955521+0000","last_fullsized":"2024-09-10T09:54:00.955521+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:53:25.743913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"96'8","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.703441+0000","last_change":"2024-09-10T09:52:22.316120+0000","last_active":"2024-09-10T09:54:29.703441+0000","last_peered":"2024-09-10T09:54:29.703441+0000","last_clean":"2024-09-10T09:54:29.703441+0000","last_became_active":"2024-09-10T09:52:22.315769+0000","last_became_peered":"2024-09-10T09:52:22.315769+0000","last_unstale":"2024-09-10T09:54:29.703441+0000","last_undegraded":"2024-09-10T09:54:29.703441+0000","last_fullsized":"2024-09-10T09:54:29.703441+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:16:41.866587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"94'3","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.841046+0000","last_change":"2024-09-10T09:52:19.835038+0000","last_active":"2024-09-10T09:54:00.841046+0000","last_peered":"2024-09-10T09:54:00.841046+0000","last_clean":"2024-09-10T09:54:00.841046+0000","last_became_active":"2024-09-10T09:52:19.834329+0000","last_became_peered":"2024-09-10T09:52:19.834329+0000","last_unstale":"2024-09-10T09:54:00.841046+0000","last_undegraded":"2024-09-10T09:54:00.841046+0000","last_fullsized":"2024-09-10T09:54:00.841046+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:58:24.084373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"94'7","reported_seq":56,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.841764+0000","last_change":"2024-09-10T09:53:01.219122+0000","last_active":"2024-09-10T09:54:00.841764+0000","last_peered":"2024-09-10T09:54:00.841764+0000","last_clean":"2024-09-10T09:54:00.841764+0000","last_became_active":"2024-09-10T09:53:01.218599+0000","last_became_peered":"2024-09-10T09:53:01.218599+0000","last_unstale":"2024-09-10T09:54:00.841764+0000","last_undegraded":"2024-09-10T09:54:00.841764+0000","last_fullsized":"2024-09-10T09:54:00.841764+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:31:10.092264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"96'8","reported_seq":19,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.706393+0000","last_change":"2024-09-10T09:54:00.952617+0000","last_active":"2024-09-10T09:54:29.706393+0000","last_peered":"2024-09-10T09:54:29.706393+0000","last_clean":"2024-09-10T09:54:29.706393+0000","last_became_active":"2024-09-10T09:54:00.952202+0000","last_became_peered":"2024-09-10T09:54:00.952202+0000","last_unstale":"2024-09-10T09:54:29.706393+0000","last_undegraded":"2024-09-10T09:54:29.706393+0000","last_fullsized":"2024-09-10T09:54:29.706393+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:35:40.179591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.844169+0000","last_change":"2024-09-10T09:52:31.371612+0000","last_active":"2024-09-10T09:54:00.844169+0000","last_peered":"2024-09-10T09:54:00.844169+0000","last_clean":"2024-09-10T09:54:00.844169+0000","last_became_active":"2024-09-10T09:52:31.371163+0000","last_became_peered":"2024-09-10T09:52:31.371163+0000","last_unstale":"2024-09-10T09:54:00.844169+0000","last_undegraded":"2024-09-10T09:54:00.844169+0000","last_fullsized":"2024-09-10T09:54:00.844169+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:11:43.681524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"96'7","reported_seq":181,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:55:22.067476+0000","last_change":"2024-09-10T09:52:19.841873+0000","last_active":"2024-09-10T09:55:22.067476+0000","last_peered":"2024-09-10T09:55:22.067476+0000","last_clean":"2024-09-10T09:55:22.067476+0000","last_became_active":"2024-09-10T09:52:19.841513+0000","last_became_peered":"2024-09-10T09:52:19.841513+0000","last_unstale":"2024-09-10T09:55:22.067476+0000","last_undegraded":"2024-09-10T09:55:22.067476+0000","last_fullsized":"2024-09-10T09:55:22.067476+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:53:31.854185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1156,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":131,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"96'7","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.697738+0000","last_change":"2024-09-10T09:52:22.314781+0000","last_active":"2024-09-10T09:54:29.697738+0000","last_peered":"2024-09-10T09:54:29.697738+0000","last_clean":"2024-09-10T09:54:29.697738+0000","last_became_active":"2024-09-10T09:52:22.313346+0000","last_became_peered":"2024-09-10T09:52:22.313346+0000","last_unstale":"2024-09-10T09:54:29.697738+0000","last_undegraded":"2024-09-10T09:54:29.697738+0000","last_fullsized":"2024-09-10T09:54:29.697738+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:15:16.921963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.844401+0000","last_change":"2024-09-10T09:52:31.365544+0000","last_active":"2024-09-10T09:54:00.844401+0000","last_peered":"2024-09-10T09:54:00.844401+0000","last_clean":"2024-09-10T09:54:00.844401+0000","last_became_active":"2024-09-10T09:52:31.365151+0000","last_became_peered":"2024-09-10T09:52:31.365151+0000","last_unstale":"2024-09-10T09:54:00.844401+0000","last_undegraded":"2024-09-10T09:54:00.844401+0000","last_fullsized":"2024-09-10T09:54:00.844401+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:36:55.247890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"96'9","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.683641+0000","last_change":"2024-09-10T09:52:22.314117+0000","last_active":"2024-09-10T09:54:29.683641+0000","last_peered":"2024-09-10T09:54:29.683641+0000","last_clean":"2024-09-10T09:54:29.683641+0000","last_became_active":"2024-09-10T09:52:22.313772+0000","last_became_peered":"2024-09-10T09:52:22.313772+0000","last_unstale":"2024-09-10T09:54:29.683641+0000","last_undegraded":"2024-09-10T09:54:29.683641+0000","last_fullsized":"2024-09-10T09:54:29.683641+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:19:37.373598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"96'12","reported_seq":50,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.682477+0000","last_change":"2024-09-10T09:52:22.314362+0000","last_active":"2024-09-10T09:54:29.682477+0000","last_peered":"2024-09-10T09:54:29.682477+0000","last_clean":"2024-09-10T09:54:29.682477+0000","last_became_active":"2024-09-10T09:52:22.314050+0000","last_became_peered":"2024-09-10T09:52:22.314050+0000","last_unstale":"2024-09-10T09:54:29.682477+0000","last_undegraded":"2024-09-10T09:54:29.682477+0000","last_fullsized":"2024-09-10T09:54:29.682477+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:37:07.218656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"96'8","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.699844+0000","last_change":"2024-09-10T09:52:22.322363+0000","last_active":"2024-09-10T09:54:29.699844+0000","last_peered":"2024-09-10T09:54:29.699844+0000","last_clean":"2024-09-10T09:54:29.699844+0000","last_became_active":"2024-09-10T09:52:22.322045+0000","last_became_peered":"2024-09-10T09:52:22.322045+0000","last_unstale":"2024-09-10T09:54:29.699844+0000","last_undegraded":"2024-09-10T09:54:29.699844+0000","last_fullsized":"2024-09-10T09:54:29.699844+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:10:27.918769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.700329+0000","last_change":"2024-09-10T09:52:22.331594+0000","last_active":"2024-09-10T09:54:29.700329+0000","last_peered":"2024-09-10T09:54:29.700329+0000","last_clean":"2024-09-10T09:54:29.700329+0000","last_became_active":"2024-09-10T09:52:22.330585+0000","last_became_peered":"2024-09-10T09:52:22.330585+0000","last_unstale":"2024-09-10T09:54:29.700329+0000","last_undegraded":"2024-09-10T09:54:29.700329+0000","last_fullsized":"2024-09-10T09:54:29.700329+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:45:56.412313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"96'4","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.705719+0000","last_change":"2024-09-10T09:52:22.325753+0000","last_active":"2024-09-10T09:54:29.705719+0000","last_peered":"2024-09-10T09:54:29.705719+0000","last_clean":"2024-09-10T09:54:29.705719+0000","last_became_active":"2024-09-10T09:52:22.325385+0000","last_became_peered":"2024-09-10T09:52:22.325385+0000","last_unstale":"2024-09-10T09:54:29.705719+0000","last_undegraded":"2024-09-10T09:54:29.705719+0000","last_fullsized":"2024-09-10T09:54:29.705719+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:18:28.776789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"96'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.681378+0000","last_change":"2024-09-10T09:52:22.319069+0000","last_active":"2024-09-10T09:54:29.681378+0000","last_peered":"2024-09-10T09:54:29.681378+0000","last_clean":"2024-09-10T09:54:29.681378+0000","last_became_active":"2024-09-10T09:52:22.318667+0000","last_became_peered":"2024-09-10T09:52:22.318667+0000","last_unstale":"2024-09-10T09:54:29.681378+0000","last_undegraded":"2024-09-10T09:54:29.681378+0000","last_fullsized":"2024-09-10T09:54:29.681378+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:21:32.028235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"96'15","reported_seq":53,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.681487+0000","last_change":"2024-09-10T09:52:22.312890+0000","last_active":"2024-09-10T09:54:29.681487+0000","last_peered":"2024-09-10T09:54:29.681487+0000","last_clean":"2024-09-10T09:54:29.681487+0000","last_became_active":"2024-09-10T09:52:22.312608+0000","last_became_peered":"2024-09-10T09:52:22.312608+0000","last_unstale":"2024-09-10T09:54:29.681487+0000","last_undegraded":"2024-09-10T09:54:29.681487+0000","last_fullsized":"2024-09-10T09:54:29.681487+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:07:46.046915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"96'2","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.689884+0000","last_change":"2024-09-10T09:54:01.151385+0000","last_active":"2024-09-10T09:54:29.689884+0000","last_peered":"2024-09-10T09:54:29.689884+0000","last_clean":"2024-09-10T09:54:29.689884+0000","last_became_active":"2024-09-10T09:54:01.150971+0000","last_became_peered":"2024-09-10T09:54:01.150971+0000","last_unstale":"2024-09-10T09:54:29.689884+0000","last_undegraded":"2024-09-10T09:54:29.689884+0000","last_fullsized":"2024-09-10T09:54:29.689884+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:27:10.492071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"96'6","reported_seq":17,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.704565+0000","last_change":"2024-09-10T09:54:00.848273+0000","last_active":"2024-09-10T09:54:29.704565+0000","last_peered":"2024-09-10T09:54:29.704565+0000","last_clean":"2024-09-10T09:54:29.704565+0000","last_became_active":"2024-09-10T09:54:00.847492+0000","last_became_peered":"2024-09-10T09:54:00.847492+0000","last_unstale":"2024-09-10T09:54:29.704565+0000","last_undegraded":"2024-09-10T09:54:29.704565+0000","last_fullsized":"2024-09-10T09:54:29.704565+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:02:24.749291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,3],"acting":[6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.29","version":"96'8","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.696836+0000","last_change":"2024-09-10T09:52:22.313538+0000","last_active":"2024-09-10T09:54:29.696836+0000","last_peered":"2024-09-10T09:54:29.696836+0000","last_clean":"2024-09-10T09:54:29.696836+0000","last_became_active":"2024-09-10T09:52:22.313196+0000","last_became_peered":"2024-09-10T09:52:22.313196+0000","last_unstale":"2024-09-10T09:54:29.696836+0000","last_undegraded":"2024-09-10T09:54:29.696836+0000","last_fullsized":"2024-09-10T09:54:29.696836+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:53:23.387167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"96'10","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.696632+0000","last_change":"2024-09-10T09:52:22.316518+0000","last_active":"2024-09-10T09:54:29.696632+0000","last_peered":"2024-09-10T09:54:29.696632+0000","last_clean":"2024-09-10T09:54:29.696632+0000","last_became_active":"2024-09-10T09:52:22.316193+0000","last_became_peered":"2024-09-10T09:52:22.316193+0000","last_unstale":"2024-09-10T09:54:29.696632+0000","last_undegraded":"2024-09-10T09:54:29.696632+0000","last_fullsized":"2024-09-10T09:54:29.696632+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:49:55.903928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"96'11","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.706245+0000","last_change":"2024-09-10T09:52:22.329163+0000","last_active":"2024-09-10T09:54:29.706245+0000","last_peered":"2024-09-10T09:54:29.706245+0000","last_clean":"2024-09-10T09:54:29.706245+0000","last_became_active":"2024-09-10T09:52:22.328453+0000","last_became_peered":"2024-09-10T09:52:22.328453+0000","last_unstale":"2024-09-10T09:54:29.706245+0000","last_undegraded":"2024-09-10T09:54:29.706245+0000","last_fullsized":"2024-09-10T09:54:29.706245+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:53:33.286055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"96'8","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.681766+0000","last_change":"2024-09-10T09:52:22.314302+0000","last_active":"2024-09-10T09:54:29.681766+0000","last_peered":"2024-09-10T09:54:29.681766+0000","last_clean":"2024-09-10T09:54:29.681766+0000","last_became_active":"2024-09-10T09:52:22.313992+0000","last_became_peered":"2024-09-10T09:52:22.313992+0000","last_unstale":"2024-09-10T09:54:29.681766+0000","last_undegraded":"2024-09-10T09:54:29.681766+0000","last_fullsized":"2024-09-10T09:54:29.681766+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:07:35.764015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"96'7","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.688705+0000","last_change":"2024-09-10T09:52:22.316400+0000","last_active":"2024-09-10T09:54:29.688705+0000","last_peered":"2024-09-10T09:54:29.688705+0000","last_clean":"2024-09-10T09:54:29.688705+0000","last_became_active":"2024-09-10T09:52:22.316110+0000","last_became_peered":"2024-09-10T09:52:22.316110+0000","last_unstale":"2024-09-10T09:54:29.688705+0000","last_undegraded":"2024-09-10T09:54:29.688705+0000","last_fullsized":"2024-09-10T09:54:29.688705+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:17:26.528899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"96'10","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.704074+0000","last_change":"2024-09-10T09:52:22.317914+0000","last_active":"2024-09-10T09:54:29.704074+0000","last_peered":"2024-09-10T09:54:29.704074+0000","last_clean":"2024-09-10T09:54:29.704074+0000","last_became_active":"2024-09-10T09:52:22.317575+0000","last_became_peered":"2024-09-10T09:52:22.317575+0000","last_unstale":"2024-09-10T09:54:29.704074+0000","last_undegraded":"2024-09-10T09:54:29.704074+0000","last_fullsized":"2024-09-10T09:54:29.704074+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:17:51.446233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"96'8","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.697848+0000","last_change":"2024-09-10T09:52:22.327749+0000","last_active":"2024-09-10T09:54:29.697848+0000","last_peered":"2024-09-10T09:54:29.697848+0000","last_clean":"2024-09-10T09:54:29.697848+0000","last_became_active":"2024-09-10T09:52:22.327461+0000","last_became_peered":"2024-09-10T09:52:22.327461+0000","last_unstale":"2024-09-10T09:54:29.697848+0000","last_undegraded":"2024-09-10T09:54:29.697848+0000","last_fullsized":"2024-09-10T09:54:29.697848+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:54.208064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"96'9","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.696695+0000","last_change":"2024-09-10T09:52:22.326982+0000","last_active":"2024-09-10T09:54:29.696695+0000","last_peered":"2024-09-10T09:54:29.696695+0000","last_clean":"2024-09-10T09:54:29.696695+0000","last_became_active":"2024-09-10T09:52:22.326635+0000","last_became_peered":"2024-09-10T09:52:22.326635+0000","last_unstale":"2024-09-10T09:54:29.696695+0000","last_undegraded":"2024-09-10T09:54:29.696695+0000","last_fullsized":"2024-09-10T09:54:29.696695+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:50:54.835899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.695757+0000","last_change":"2024-09-10T09:52:22.323410+0000","last_active":"2024-09-10T09:54:29.695757+0000","last_peered":"2024-09-10T09:54:29.695757+0000","last_clean":"2024-09-10T09:54:29.695757+0000","last_became_active":"2024-09-10T09:52:22.323116+0000","last_became_peered":"2024-09-10T09:52:22.323116+0000","last_unstale":"2024-09-10T09:54:29.695757+0000","last_undegraded":"2024-09-10T09:54:29.695757+0000","last_fullsized":"2024-09-10T09:54:29.695757+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:09:15.115162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.680224+0000","last_change":"2024-09-10T09:52:22.324908+0000","last_active":"2024-09-10T09:54:29.680224+0000","last_peered":"2024-09-10T09:54:29.680224+0000","last_clean":"2024-09-10T09:54:29.680224+0000","last_became_active":"2024-09-10T09:52:22.324449+0000","last_became_peered":"2024-09-10T09:52:22.324449+0000","last_unstale":"2024-09-10T09:54:29.680224+0000","last_undegraded":"2024-09-10T09:54:29.680224+0000","last_fullsized":"2024-09-10T09:54:29.680224+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:43:24.535721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"96'10","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.700039+0000","last_change":"2024-09-10T09:52:22.322157+0000","last_active":"2024-09-10T09:54:29.700039+0000","last_peered":"2024-09-10T09:54:29.700039+0000","last_clean":"2024-09-10T09:54:29.700039+0000","last_became_active":"2024-09-10T09:52:22.321856+0000","last_became_peered":"2024-09-10T09:52:22.321856+0000","last_unstale":"2024-09-10T09:54:29.700039+0000","last_undegraded":"2024-09-10T09:54:29.700039+0000","last_fullsized":"2024-09-10T09:54:29.700039+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:36:38.395370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"96'7","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.703464+0000","last_change":"2024-09-10T09:53:00.800337+0000","last_active":"2024-09-10T09:54:29.703464+0000","last_peered":"2024-09-10T09:54:29.703464+0000","last_clean":"2024-09-10T09:54:29.703464+0000","last_became_active":"2024-09-10T09:53:00.799863+0000","last_became_peered":"2024-09-10T09:53:00.799863+0000","last_unstale":"2024-09-10T09:54:29.703464+0000","last_undegraded":"2024-09-10T09:54:29.703464+0000","last_fullsized":"2024-09-10T09:54:29.703464+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:18.017672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.35","version":"96'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.693580+0000","last_change":"2024-09-10T09:52:22.310735+0000","last_active":"2024-09-10T09:54:29.693580+0000","last_peered":"2024-09-10T09:54:29.693580+0000","last_clean":"2024-09-10T09:54:29.693580+0000","last_became_active":"2024-09-10T09:52:22.310376+0000","last_became_peered":"2024-09-10T09:52:22.310376+0000","last_unstale":"2024-09-10T09:54:29.693580+0000","last_undegraded":"2024-09-10T09:54:29.693580+0000","last_fullsized":"2024-09-10T09:54:29.693580+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:38:08.637588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"96'16","reported_seq":55,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.685458+0000","last_change":"2024-09-10T09:52:22.321113+0000","last_active":"2024-09-10T09:54:29.685458+0000","last_peered":"2024-09-10T09:54:29.685458+0000","last_clean":"2024-09-10T09:54:29.685458+0000","last_became_active":"2024-09-10T09:52:22.320767+0000","last_became_peered":"2024-09-10T09:52:22.320767+0000","last_unstale":"2024-09-10T09:54:29.685458+0000","last_undegraded":"2024-09-10T09:54:29.685458+0000","last_fullsized":"2024-09-10T09:54:29.685458+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:37.549322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"96'11","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.687162+0000","last_change":"2024-09-10T09:52:22.318634+0000","last_active":"2024-09-10T09:54:29.687162+0000","last_peered":"2024-09-10T09:54:29.687162+0000","last_clean":"2024-09-10T09:54:29.687162+0000","last_became_active":"2024-09-10T09:52:22.318294+0000","last_became_peered":"2024-09-10T09:52:22.318294+0000","last_unstale":"2024-09-10T09:54:29.687162+0000","last_undegraded":"2024-09-10T09:54:29.687162+0000","last_fullsized":"2024-09-10T09:54:29.687162+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:19:31.262055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"96'9","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.698725+0000","last_change":"2024-09-10T09:52:22.324567+0000","last_active":"2024-09-10T09:54:29.698725+0000","last_peered":"2024-09-10T09:54:29.698725+0000","last_clean":"2024-09-10T09:54:29.698725+0000","last_became_active":"2024-09-10T09:52:22.321803+0000","last_became_peered":"2024-09-10T09:52:22.321803+0000","last_unstale":"2024-09-10T09:54:29.698725+0000","last_undegraded":"2024-09-10T09:54:29.698725+0000","last_fullsized":"2024-09-10T09:54:29.698725+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:08:11.099428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"96'5","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.699735+0000","last_change":"2024-09-10T09:52:22.318163+0000","last_active":"2024-09-10T09:54:29.699735+0000","last_peered":"2024-09-10T09:54:29.699735+0000","last_clean":"2024-09-10T09:54:29.699735+0000","last_became_active":"2024-09-10T09:52:22.317808+0000","last_became_peered":"2024-09-10T09:52:22.317808+0000","last_unstale":"2024-09-10T09:54:29.699735+0000","last_undegraded":"2024-09-10T09:54:29.699735+0000","last_fullsized":"2024-09-10T09:54:29.699735+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:19.214291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"96'6","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.704960+0000","last_change":"2024-09-10T09:52:22.323035+0000","last_active":"2024-09-10T09:54:29.704960+0000","last_peered":"2024-09-10T09:54:29.704960+0000","last_clean":"2024-09-10T09:54:29.704960+0000","last_became_active":"2024-09-10T09:52:22.322679+0000","last_became_peered":"2024-09-10T09:52:22.322679+0000","last_unstale":"2024-09-10T09:54:29.704960+0000","last_undegraded":"2024-09-10T09:54:29.704960+0000","last_fullsized":"2024-09-10T09:54:29.704960+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:23:13.762612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"96'4","reported_seq":40,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.700111+0000","last_change":"2024-09-10T09:52:22.315887+0000","last_active":"2024-09-10T09:54:29.700111+0000","last_peered":"2024-09-10T09:54:29.700111+0000","last_clean":"2024-09-10T09:54:29.700111+0000","last_became_active":"2024-09-10T09:52:22.315569+0000","last_became_peered":"2024-09-10T09:52:22.315569+0000","last_unstale":"2024-09-10T09:54:29.700111+0000","last_undegraded":"2024-09-10T09:54:29.700111+0000","last_fullsized":"2024-09-10T09:54:29.700111+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:46:57.094320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"96'8","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.676312+0000","last_change":"2024-09-10T09:52:22.314060+0000","last_active":"2024-09-10T09:54:29.676312+0000","last_peered":"2024-09-10T09:54:29.676312+0000","last_clean":"2024-09-10T09:54:29.676312+0000","last_became_active":"2024-09-10T09:52:22.313572+0000","last_became_peered":"2024-09-10T09:52:22.313572+0000","last_unstale":"2024-09-10T09:54:29.676312+0000","last_undegraded":"2024-09-10T09:54:29.676312+0000","last_fullsized":"2024-09-10T09:54:29.676312+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:11:17.158625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.692163+0000","last_change":"2024-09-10T09:52:22.327233+0000","last_active":"2024-09-10T09:54:29.692163+0000","last_peered":"2024-09-10T09:54:29.692163+0000","last_clean":"2024-09-10T09:54:29.692163+0000","last_became_active":"2024-09-10T09:52:22.326945+0000","last_became_peered":"2024-09-10T09:52:22.326945+0000","last_unstale":"2024-09-10T09:54:29.692163+0000","last_undegraded":"2024-09-10T09:54:29.692163+0000","last_fullsized":"2024-09-10T09:54:29.692163+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:31:13.490147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"96'4","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.681664+0000","last_change":"2024-09-10T09:52:22.330912+0000","last_active":"2024-09-10T09:54:29.681664+0000","last_peered":"2024-09-10T09:54:29.681664+0000","last_clean":"2024-09-10T09:54:29.681664+0000","last_became_active":"2024-09-10T09:52:22.330149+0000","last_became_peered":"2024-09-10T09:52:22.330149+0000","last_unstale":"2024-09-10T09:54:29.681664+0000","last_undegraded":"2024-09-10T09:54:29.681664+0000","last_fullsized":"2024-09-10T09:54:29.681664+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:10:48.842499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"96'8","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.697891+0000","last_change":"2024-09-10T09:52:22.327035+0000","last_active":"2024-09-10T09:54:29.697891+0000","last_peered":"2024-09-10T09:54:29.697891+0000","last_clean":"2024-09-10T09:54:29.697891+0000","last_became_active":"2024-09-10T09:52:22.326741+0000","last_became_peered":"2024-09-10T09:52:22.326741+0000","last_unstale":"2024-09-10T09:54:29.697891+0000","last_undegraded":"2024-09-10T09:54:29.697891+0000","last_fullsized":"2024-09-10T09:54:29.697891+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:21:05.026614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":75677,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1299,"num_read_kb":732,"num_write":263,"num_write_kb":158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":4426,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":227031,"data_compressed":5661,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":227,"ondisk_log_size":227,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282980,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449139,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1073152,"data_stored":554869,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":70,"seq":300647710774,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32376,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712776,"statfs":{"total":95995035648,"available":95961882624,"internally_reserved":0,"allocated":1220608,"data_stored":1209196,"data_compressed":70462,"data_compressed_allocated":94208,"data_compressed_original":688128,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":65,"seq":279172874301,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32256,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712896,"statfs":{"total":95995035648,"available":95962005504,"internally_reserved":0,"allocated":1097728,"data_stored":557996,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135939,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32244,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712908,"statfs":{"total":95995035648,"available":95962017792,"internally_reserved":0,"allocated":1085440,"data_stored":557702,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364871,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28136,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717016,"statfs":{"total":95995035648,"available":95966224384,"internally_reserved":0,"allocated":1138688,"data_stored":1150864,"data_compressed":68575,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528397,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1069056,"data_stored":549715,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790036,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28144,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717008,"statfs":{"total":95995035648,"available":95966216192,"internally_reserved":0,"allocated":1146880,"data_stored":1142404,"data_compressed":68575,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953558,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28148,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717004,"statfs":{"total":95995035648,"available":95966212096,"internally_reserved":0,"allocated":1150976,"data_stored":621888,"data_compressed":34470,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182492,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32240,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712912,"statfs":{"total":95995035648,"available":95962021888,"internally_reserved":0,"allocated":1081344,"data_stored":549112,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411427,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36404,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708748,"statfs":{"total":95995035648,"available":95957757952,"internally_reserved":0,"allocated":1085440,"data_stored":551188,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574952,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28144,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717008,"statfs":{"total":95995035648,"available":95966216192,"internally_reserved":0,"allocated":1146880,"data_stored":619293,"data_compressed":34470,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738478,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1093632,"data_stored":550860,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34340,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34340,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34340,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35992,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35992,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35992,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35992,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35992,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":70611,"data_compressed":1887,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":69110,"data_compressed":1887,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":70146,"data_compressed":1887,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T09:55:26.225 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T09:55:26.408 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:55:26.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.951+0000 7f8804000640 1 -- 172.21.15.77:0/1886396725 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87fc104010 msgr2=0x7f87fc1043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:26.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.951+0000 7f8804000640 1 --2- 172.21.15.77:0/1886396725 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87fc104010 0x7f87fc1043f0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f87f0009900 tx=0x7f87f002a3f0 comp rx=0 tx=0).stop 2024-09-10T09:55:26.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.951+0000 7f8804000640 1 -- 172.21.15.77:0/1886396725 shutdown_connections 2024-09-10T09:55:26.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.951+0000 7f8804000640 1 --2- 172.21.15.77:0/1886396725 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f87fc109140 0x7f87fc110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:26.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.951+0000 7f8804000640 1 --2- 172.21.15.77:0/1886396725 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f87fc1049c0 0x7f87fc108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:26.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.951+0000 7f8804000640 1 --2- 172.21.15.77:0/1886396725 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87fc104010 0x7f87fc1043f0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:26.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.951+0000 7f8804000640 1 -- 172.21.15.77:0/1886396725 >> 172.21.15.77:0/1886396725 conn(0x7f87fc0ffa50 msgr2=0x7f87fc101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:26.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.951+0000 7f8804000640 1 -- 172.21.15.77:0/1886396725 shutdown_connections 2024-09-10T09:55:26.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.951+0000 7f8804000640 1 -- 172.21.15.77:0/1886396725 wait complete. 2024-09-10T09:55:26.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.952+0000 7f8804000640 1 Processor -- start 2024-09-10T09:55:26.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.952+0000 7f8804000640 1 -- start start 2024-09-10T09:55:26.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.953+0000 7f8804000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f87fc104010 0x7f87fc19a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:26.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.953+0000 7f8804000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f87fc1049c0 0x7f87fc19abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:26.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.953+0000 7f8804000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87fc109140 0x7f87fc19ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:26.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.953+0000 7f8804000640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f87fc114190 con 0x7f87fc109140 2024-09-10T09:55:26.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.953+0000 7f8804000640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f87fc114010 con 0x7f87fc104010 2024-09-10T09:55:26.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.953+0000 7f8804000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f87fc114310 con 0x7f87fc1049c0 2024-09-10T09:55:26.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.953+0000 7f8801600640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f87fc104010 0x7f87fc19a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:26.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.953+0000 7f8801600640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f87fc104010 0x7f87fc19a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:35416/0 (socket says 172.21.15.77:35416) 2024-09-10T09:55:26.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.953+0000 7f8801600640 1 -- 172.21.15.77:0/3441999647 learned_addr learned my addr 172.21.15.77:0/3441999647 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:55:26.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.953+0000 7f8800c00640 1 --2- 172.21.15.77:0/3441999647 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f87fc1049c0 0x7f87fc19abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:26.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.953+0000 7f8802000640 1 --2- 172.21.15.77:0/3441999647 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87fc109140 0x7f87fc19ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:26.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.954+0000 7f8801600640 1 -- 172.21.15.77:0/3441999647 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f87fc1049c0 msgr2=0x7f87fc19abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:26.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.954+0000 7f8801600640 1 --2- 172.21.15.77:0/3441999647 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f87fc1049c0 0x7f87fc19abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:26.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.954+0000 7f8801600640 1 -- 172.21.15.77:0/3441999647 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87fc109140 msgr2=0x7f87fc19ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:26.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.954+0000 7f8801600640 1 --2- 172.21.15.77:0/3441999647 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87fc109140 0x7f87fc19ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:26.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.954+0000 7f8801600640 1 -- 172.21.15.77:0/3441999647 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87fc19f660 con 0x7f87fc104010 2024-09-10T09:55:26.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.954+0000 7f8802000640 1 --2- 172.21.15.77:0/3441999647 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87fc109140 0x7f87fc19ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:55:26.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.954+0000 7f8801600640 1 --2- 172.21.15.77:0/3441999647 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f87fc104010 0x7f87fc19a6b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f87f0004770 tx=0x7f87f0003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:26.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.954+0000 7f87fa000640 1 -- 172.21.15.77:0/3441999647 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f003f070 con 0x7f87fc104010 2024-09-10T09:55:26.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.954+0000 7f87fa000640 1 -- 172.21.15.77:0/3441999647 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87f0031ca0 con 0x7f87fc104010 2024-09-10T09:55:26.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.954+0000 7f8804000640 1 -- 172.21.15.77:0/3441999647 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87fc19f8f0 con 0x7f87fc104010 2024-09-10T09:55:26.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.954+0000 7f87fa000640 1 -- 172.21.15.77:0/3441999647 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f0035030 con 0x7f87fc104010 2024-09-10T09:55:26.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.955+0000 7f8804000640 1 -- 172.21.15.77:0/3441999647 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87fc1a7230 con 0x7f87fc104010 2024-09-10T09:55:26.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.957+0000 7f87fa000640 1 -- 172.21.15.77:0/3441999647 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f87f0042070 con 0x7f87fc104010 2024-09-10T09:55:26.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.958+0000 7f8806a39740 1 Processor -- start 2024-09-10T09:55:26.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.958+0000 7f8806a39740 1 -- start start 2024-09-10T09:55:26.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.958+0000 7f8806a39740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c612b69b90 0x55c612b69f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:26.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.958+0000 7f8806a39740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c612b6a4b0 0x55c612b74830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:26.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.958+0000 7f8806a39740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c612b74d70 0x55c612b77160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:26.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.958+0000 7f8806a39740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c612ad03e0 con 0x55c612b6a4b0 2024-09-10T09:55:26.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.958+0000 7f8806a39740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c612a77d80 con 0x55c612b74d70 2024-09-10T09:55:26.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.958+0000 7f8806a39740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c612acc3a0 con 0x55c612b69b90 2024-09-10T09:55:26.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.959+0000 7f8802000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c612b69b90 0x55c612b69f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:26.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.959+0000 7f8802000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c612b69b90 0x55c612b69f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:60532/0 (socket says 172.21.15.77:60532) 2024-09-10T09:55:26.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.959+0000 7f8802000640 1 -- 172.21.15.77:0/807362702 learned_addr learned my addr 172.21.15.77:0/807362702 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:55:26.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.959+0000 7f8800c00640 1 --2- 172.21.15.77:0/807362702 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c612b74d70 0x55c612b77160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:26.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.959+0000 7f8801600640 1 --2- 172.21.15.77:0/807362702 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c612b6a4b0 0x55c612b74830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:26.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.959+0000 7f8800c00640 1 -- 172.21.15.77:0/807362702 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c612b69b90 msgr2=0x55c612b69f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:26.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.959+0000 7f8800c00640 1 --2- 172.21.15.77:0/807362702 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c612b69b90 0x55c612b69f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:26.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.959+0000 7f8800c00640 1 -- 172.21.15.77:0/807362702 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c612b6a4b0 msgr2=0x55c612b74830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:26.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.959+0000 7f8800c00640 1 --2- 172.21.15.77:0/807362702 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c612b6a4b0 0x55c612b74830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:26.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.959+0000 7f8800c00640 1 -- 172.21.15.77:0/807362702 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c612a96ae0 con 0x55c612b74d70 2024-09-10T09:55:26.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.959+0000 7f8801600640 1 --2- 172.21.15.77:0/807362702 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c612b6a4b0 0x55c612b74830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:55:26.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.959+0000 7f87fa000640 1 --2- 172.21.15.77:0/3441999647 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f87e40a2c00 0x7f87e40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:26.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.959+0000 7f8800c00640 1 --2- 172.21.15.77:0/807362702 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c612b74d70 0x55c612b77160 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f87e8009bf0 tx=0x7f87e8011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66aca1f404c85e18 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:26.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.960+0000 7f87fa000640 1 -- 172.21.15.77:0/3441999647 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f87f00efa80 con 0x7f87fc104010 2024-09-10T09:55:26.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.960+0000 7f87ee600640 1 -- 172.21.15.77:0/807362702 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e8004050 con 0x55c612b74d70 2024-09-10T09:55:26.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.960+0000 7f87ee600640 1 -- 172.21.15.77:0/807362702 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87e8005500 con 0x55c612b74d70 2024-09-10T09:55:26.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.960+0000 7f87ee600640 1 -- 172.21.15.77:0/807362702 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e80035f0 con 0x55c612b74d70 2024-09-10T09:55:26.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.960+0000 7f8802000640 1 --2- 172.21.15.77:0/3441999647 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f87e40a2c00 0x7f87e40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:26.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.961+0000 7f8806a39740 1 -- 172.21.15.77:0/807362702 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c612b74d70 msgr2=0x55c612b77160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:26.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.961+0000 7f8802000640 1 --2- 172.21.15.77:0/3441999647 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f87e40a2c00 0x7f87e40a50c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f87f4009fd0 tx=0x7f87f4009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:26.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.961+0000 7f8806a39740 1 --2- 172.21.15.77:0/807362702 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c612b74d70 0x55c612b77160 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f87e8009bf0 tx=0x7f87e8011810 comp rx=0 tx=0).stop 2024-09-10T09:55:26.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.961+0000 7f8806a39740 1 -- 172.21.15.77:0/807362702 shutdown_connections 2024-09-10T09:55:26.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.961+0000 7f8806a39740 1 --2- 172.21.15.77:0/807362702 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c612b74d70 0x55c612b77160 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:26.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.961+0000 7f8806a39740 1 --2- 172.21.15.77:0/807362702 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c612b6a4b0 0x55c612b74830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:26.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.961+0000 7f8806a39740 1 --2- 172.21.15.77:0/807362702 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c612b69b90 0x55c612b69f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:26.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.961+0000 7f8806a39740 1 -- 172.21.15.77:0/807362702 >> 172.21.15.77:0/807362702 conn(0x55c612b66120 msgr2=0x55c612b66500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:26.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.961+0000 7f8806a39740 1 -- 172.21.15.77:0/807362702 shutdown_connections 2024-09-10T09:55:26.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.961+0000 7f8806a39740 1 -- 172.21.15.77:0/807362702 wait complete. 2024-09-10T09:55:26.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.962+0000 7f8806a39740 1 -- start start 2024-09-10T09:55:26.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.962+0000 7f8806a39740 1 Processor -- start 2024-09-10T09:55:26.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.963+0000 7f8806a39740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c612b69b90 0x55c612b71c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:26.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.963+0000 7f8806a39740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c612b6a4b0 0x55c612b721d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:26.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.963+0000 7f8806a39740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c612b74d70 0x55c612b72710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:26.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.963+0000 7f8806a39740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c612b779e0 con 0x55c612b69b90 2024-09-10T09:55:26.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.963+0000 7f8806a39740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c612a8b0f0 con 0x55c612b74d70 2024-09-10T09:55:26.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.963+0000 7f8806a39740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c612b77b60 con 0x55c612b6a4b0 2024-09-10T09:55:26.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.963+0000 7f8806a39740 10 client.0 _ll_register_callbacks cb 0x55c612a7c9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:55:26.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.963+0000 7f8800c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c612b6a4b0 0x55c612b721d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:26.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.963+0000 7f8801600640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c612b69b90 0x55c612b71c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:26.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.963+0000 7f8800c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c612b6a4b0 0x55c612b721d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:60538/0 (socket says 172.21.15.77:60538) 2024-09-10T09:55:26.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.963+0000 7f8800c00640 1 -- 172.21.15.77:0/2231945612 learned_addr learned my addr 172.21.15.77:0/2231945612 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:55:26.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.963+0000 7f8802000640 1 --2- 172.21.15.77:0/2231945612 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c612b74d70 0x55c612b72710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:26.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.964+0000 7f8800c00640 1 -- 172.21.15.77:0/2231945612 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c612b74d70 msgr2=0x55c612b72710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:26.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.964+0000 7f8800c00640 1 --2- 172.21.15.77:0/2231945612 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c612b74d70 0x55c612b72710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:26.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.964+0000 7f8800c00640 1 -- 172.21.15.77:0/2231945612 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c612b69b90 msgr2=0x55c612b71c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:26.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.964+0000 7f8800c00640 1 --2- 172.21.15.77:0/2231945612 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c612b69b90 0x55c612b71c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:26.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.964+0000 7f8800c00640 1 -- 172.21.15.77:0/2231945612 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6128413c0 con 0x55c612b6a4b0 2024-09-10T09:55:26.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.964+0000 7f8801600640 1 --2- 172.21.15.77:0/2231945612 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c612b69b90 0x55c612b71c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:55:26.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.964+0000 7f8800c00640 1 --2- 172.21.15.77:0/2231945612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c612b6a4b0 0x55c612b721d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f87e8009fd0 tx=0x7f87e8005750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=672e92bfa8901bf0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:26.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.964+0000 7f87e2a00640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-10T09:55:26.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.964+0000 7f87e2a00640 1 -- 172.21.15.77:0/2231945612 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e8029070 con 0x55c612b6a4b0 2024-09-10T09:55:26.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.964+0000 7f8806a39740 1 -- 172.21.15.77:0/2231945612 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87e8005500 con 0x55c612b6a4b0 2024-09-10T09:55:26.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.964+0000 7f87e2a00640 1 -- 172.21.15.77:0/2231945612 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f87e8005030 con 0x55c612b6a4b0 2024-09-10T09:55:26.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.965+0000 7f87e2a00640 1 -- 172.21.15.77:0/2231945612 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f87e8033050 con 0x55c612b6a4b0 2024-09-10T09:55:26.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.965+0000 7f8806a39740 10 client.25318 fetch_fsmap learned FSMap version 29 2024-09-10T09:55:26.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.965+0000 7f8806a39740 1 -- 172.21.15.77:0/2231945612 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c612b836c0 con 0x55c612b6a4b0 2024-09-10T09:55:26.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.965+0000 7f87e2a00640 1 -- 172.21.15.77:0/2231945612 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87e8005500 con 0x55c612b6a4b0 2024-09-10T09:55:26.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.966+0000 7f87e2a00640 1 -- 172.21.15.77:0/2231945612 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f87e802e440 con 0x55c612b6a4b0 2024-09-10T09:55:26.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.966+0000 7f8806a39740 10 client.25318 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T09:55:26.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.966+0000 7f8806a39740 10 client.25318 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:55:26.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.966+0000 7f8806a39740 20 client.25318 populate_metadata read hostname 'smithi077' 2024-09-10T09:55:26.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.966+0000 7f8806a39740 1 --2- 172.21.15.77:0/2231945612 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55c612b84a10 0x55c612b86e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:26.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.966+0000 7f8806a39740 4 client.25318 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:55:26.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.966+0000 7f8806a39740 1 -- 172.21.15.77:0/2231945612 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c612add950 con 0x55c612b84a10 2024-09-10T09:55:26.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.966+0000 7f8801600640 1 --2- 172.21.15.77:0/2231945612 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55c612b84a10 0x55c612b86e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:26.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.967+0000 7f8801600640 1 --2- 172.21.15.77:0/2231945612 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55c612b84a10 0x55c612b86e00 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3175234b3d1a737 server_cookie=821a1163f2714241 in_seq=0 out_seq=0 2024-09-10T09:55:26.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.967+0000 7f87e2a00640 10 client.25318 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T09:55:26.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.967+0000 7f87e2a00640 1 -- 172.21.15.77:0/2231945612 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c612add950 con 0x55c612b84a10 2024-09-10T09:55:26.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.967+0000 7f87e2a00640 10 client.25318 handle_command_reply: tid=0 2024-09-10T09:55:26.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.968+0000 7f8806a39740 1 client.25318 shutdown 2024-09-10T09:55:26.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.968+0000 7f8806a39740 1 -- 172.21.15.77:0/2231945612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c612b6a4b0 msgr2=0x55c612b721d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:26.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.968+0000 7f8806a39740 1 --2- 172.21.15.77:0/2231945612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c612b6a4b0 0x55c612b721d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f87e8009fd0 tx=0x7f87e8005750 comp rx=0 tx=0).stop 2024-09-10T09:55:26.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.968+0000 7f8806a39740 1 -- 172.21.15.77:0/2231945612 shutdown_connections 2024-09-10T09:55:26.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.968+0000 7f8806a39740 1 --2- 172.21.15.77:0/2231945612 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55c612b84a10 0x55c612b86e00 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:26.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.968+0000 7f8806a39740 1 --2- 172.21.15.77:0/2231945612 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c612b74d70 0x55c612b72710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:26.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.968+0000 7f8806a39740 1 --2- 172.21.15.77:0/2231945612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c612b6a4b0 0x55c612b721d0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:26.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.968+0000 7f8806a39740 1 --2- 172.21.15.77:0/2231945612 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c612b69b90 0x55c612b71c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:26.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.968+0000 7f87e2a00640 0 client.25318 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T09:55:26.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.968+0000 7f8806a39740 1 -- 172.21.15.77:0/2231945612 >> 172.21.15.77:0/2231945612 conn(0x55c612b66120 msgr2=0x55c612b6be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:26.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.968+0000 7f8806a39740 1 -- 172.21.15.77:0/2231945612 shutdown_connections 2024-09-10T09:55:26.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.968+0000 7f8806a39740 1 -- 172.21.15.77:0/2231945612 wait complete. 2024-09-10T09:55:26.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.968+0000 7f8806a39740 20 client.25318 trim_cache size 0 max 16384 2024-09-10T09:55:26.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.995+0000 7f8806a39740 1 Processor -- start 2024-09-10T09:55:26.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.995+0000 7f8806a39740 1 -- start start 2024-09-10T09:55:26.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.995+0000 7f8806a39740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c612b69b90 0x55c612b73c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:26.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.996+0000 7f8806a39740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c612b6a4b0 0x55c612b8c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:26.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.996+0000 7f8806a39740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c612b74d70 0x55c612b8cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:26.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.996+0000 7f8806a39740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f87fc1b1090 con 0x55c612b69b90 2024-09-10T09:55:26.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.996+0000 7f8806a39740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f87fc1b2bd0 con 0x55c612b74d70 2024-09-10T09:55:26.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.996+0000 7f8806a39740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f87fc1b1760 con 0x55c612b6a4b0 2024-09-10T09:55:26.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.996+0000 7f8801600640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c612b69b90 0x55c612b73c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:26.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.996+0000 7f8801600640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c612b69b90 0x55c612b73c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56944/0 (socket says 172.21.15.77:56944) 2024-09-10T09:55:26.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.996+0000 7f8801600640 1 -- 172.21.15.77:0/2212085681 learned_addr learned my addr 172.21.15.77:0/2212085681 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:55:26.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.997+0000 7f8802000640 1 --2- 172.21.15.77:0/2212085681 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c612b74d70 0x55c612b8cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:26.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.997+0000 7f8801600640 1 -- 172.21.15.77:0/2212085681 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c612b6a4b0 msgr2=0x55c612b8c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:26.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.997+0000 7f8801600640 1 --2- 172.21.15.77:0/2212085681 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c612b6a4b0 0x55c612b8c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:26.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.997+0000 7f8801600640 1 -- 172.21.15.77:0/2212085681 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c612b74d70 msgr2=0x55c612b8cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:26.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.997+0000 7f8801600640 1 --2- 172.21.15.77:0/2212085681 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c612b74d70 0x55c612b8cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:26.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.997+0000 7f8801600640 1 -- 172.21.15.77:0/2212085681 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c612b701e0 con 0x55c612b69b90 2024-09-10T09:55:26.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.997+0000 7f8801600640 1 --2- 172.21.15.77:0/2212085681 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c612b69b90 0x55c612b73c00 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f87f0031550 tx=0x7f87f003b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96c977b32e7fda9a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:26.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.998+0000 7f87edc00640 1 -- 172.21.15.77:0/2212085681 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f0049040 con 0x55c612b69b90 2024-09-10T09:55:26.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.998+0000 7f87edc00640 1 -- 172.21.15.77:0/2212085681 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87f00bc650 con 0x55c612b69b90 2024-09-10T09:55:26.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.998+0000 7f87edc00640 1 -- 172.21.15.77:0/2212085681 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f00ed240 con 0x55c612b69b90 2024-09-10T09:55:27.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.999+0000 7f8806a39740 1 -- 172.21.15.77:0/2212085681 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c612b69b90 msgr2=0x55c612b73c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:27.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.999+0000 7f8806a39740 1 --2- 172.21.15.77:0/2212085681 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c612b69b90 0x55c612b73c00 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f87f0031550 tx=0x7f87f003b890 comp rx=0 tx=0).stop 2024-09-10T09:55:27.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.999+0000 7f8806a39740 1 -- 172.21.15.77:0/2212085681 shutdown_connections 2024-09-10T09:55:27.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.999+0000 7f8806a39740 1 --2- 172.21.15.77:0/2212085681 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c612b74d70 0x55c612b8cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:27.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.999+0000 7f8806a39740 1 --2- 172.21.15.77:0/2212085681 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c612b6a4b0 0x55c612b8c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:27.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.999+0000 7f8806a39740 1 --2- 172.21.15.77:0/2212085681 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c612b69b90 0x55c612b73c00 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:27.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.999+0000 7f8806a39740 1 -- 172.21.15.77:0/2212085681 >> 172.21.15.77:0/2212085681 conn(0x55c612b66120 msgr2=0x55c612b734f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:27.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:26.999+0000 7f8806a39740 1 -- 172.21.15.77:0/2212085681 shutdown_connections 2024-09-10T09:55:27.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:27.000+0000 7f8806a39740 1 -- 172.21.15.77:0/2212085681 wait complete. 2024-09-10T09:55:27.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:27.000+0000 7f8806a39740 1 -- start start 2024-09-10T09:55:27.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:27.001+0000 7f8806a39740 1 Processor -- start 2024-09-10T09:55:27.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:27.001+0000 7f8806a39740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c612b69b90 0x55c612b78d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:27.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:27.001+0000 7f8806a39740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c612b6a4b0 0x55c612b79270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:27.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:27.001+0000 7f8806a39740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c612b74d70 0x55c612b797b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:27.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:27.001+0000 7f8806a39740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f87fc1b2500 con 0x55c612b6a4b0 2024-09-10T09:55:27.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:27.001+0000 7f8806a39740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f87fc1b1e30 con 0x55c612b69b90 2024-09-10T09:55:27.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:27.001+0000 7f8806a39740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f87fc1af070 con 0x55c612b74d70 2024-09-10T09:55:27.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:27.001+0000 7f8806a39740 10 client.0 _ll_register_callbacks cb 0x7f87e80043f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:55:27.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:27.001+0000 7f8801600640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c612b69b90 0x55c612b78d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:27.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:27.002+0000 7f8801600640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c612b69b90 0x55c612b78d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:35442/0 (socket says 172.21.15.77:35442) 2024-09-10T09:55:27.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:27.002+0000 7f8801600640 1 -- 172.21.15.77:0/437686912 learned_addr learned my addr 172.21.15.77:0/437686912 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:55:27.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:27.002+0000 7f8800c00640 1 --2- 172.21.15.77:0/437686912 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c612b6a4b0 0x55c612b79270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:27.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:27.002+0000 7f8801600640 1 -- 172.21.15.77:0/437686912 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c612b74d70 msgr2=0x55c612b797b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T09:55:27.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:27.002+0000 7f8801600640 1 --2- 172.21.15.77:0/437686912 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c612b74d70 0x55c612b797b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:27.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:27.002+0000 7f8801600640 1 -- 172.21.15.77:0/437686912 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c612b6a4b0 msgr2=0x55c612b79270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:27.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:27.002+0000 7f8801600640 1 --2- 172.21.15.77:0/437686912 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c612b6a4b0 0x55c612b79270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:27.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:27.002+0000 7f8801600640 1 -- 172.21.15.77:0/437686912 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c612bee740 con 0x55c612b69b90 2024-09-10T09:55:27.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:27.002+0000 7f8800c00640 1 --2- 172.21.15.77:0/437686912 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c612b6a4b0 0x55c612b79270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:55:27.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:27.003+0000 7f8801600640 1 --2- 172.21.15.77:0/437686912 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c612b69b90 0x55c612b78d30 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f87f0000c00 tx=0x7f87f002a3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0a5824e0188fb0e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:27.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:27.003+0000 7f87e2a00640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T09:55:27.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:27.003+0000 7f87e2a00640 1 -- 172.21.15.77:0/437686912 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f0049070 con 0x55c612b69b90 2024-09-10T09:55:27.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:27.003+0000 7f87e2a00640 1 -- 172.21.15.77:0/437686912 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f87f00ed1b0 con 0x55c612b69b90 2024-09-10T09:55:27.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:27.003+0000 7f8806a39740 1 -- 172.21.15.77:0/437686912 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87f00bc650 con 0x55c612b69b90 2024-09-10T09:55:27.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:27.003+0000 7f87e2a00640 1 -- 172.21.15.77:0/437686912 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f87f00bbf50 con 0x55c612b69b90 2024-09-10T09:55:27.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:27.003+0000 7f8806a39740 10 client.25346 fetch_fsmap learned FSMap version 29 2024-09-10T09:55:27.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:27.003+0000 7f8806a39740 1 -- 172.21.15.77:0/437686912 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c612bef9f0 con 0x55c612b69b90 2024-09-10T09:55:27.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:27.004+0000 7f87e2a00640 1 -- 172.21.15.77:0/437686912 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87f00bc650 con 0x55c612b69b90 2024-09-10T09:55:27.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:27.004+0000 7f87e2a00640 1 -- 172.21.15.77:0/437686912 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f87f00c4730 con 0x55c612b69b90 2024-09-10T09:55:27.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:27.004+0000 7f8806a39740 10 client.25346 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T09:55:27.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:27.004+0000 7f8806a39740 10 client.25346 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:55:27.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:27.004+0000 7f8806a39740 20 client.25346 populate_metadata read hostname 'smithi077' 2024-09-10T09:55:27.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:27.004+0000 7f8806a39740 1 --2- 172.21.15.77:0/437686912 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55c612bf0010 0x55c612bf2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:27.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:27.004+0000 7f8806a39740 4 client.25346 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T09:55:27.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:27.004+0000 7f8806a39740 1 -- 172.21.15.77:0/437686912 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c612ad7e60 con 0x55c612bf0010 2024-09-10T09:55:27.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:27.005+0000 7f8800c00640 1 --2- 172.21.15.77:0/437686912 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55c612bf0010 0x55c612bf2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:27.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:27.005+0000 7f8800c00640 1 --2- 172.21.15.77:0/437686912 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55c612bf0010 0x55c612bf2400 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47f923594b1ea3b server_cookie=feb93249e6483ecf in_seq=0 out_seq=0 2024-09-10T09:55:27.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:27.005+0000 7f87e2a00640 10 client.25346 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T09:55:27.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:27.005+0000 7f87e2a00640 1 -- 172.21.15.77:0/437686912 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55c612ad7e60 con 0x55c612bf0010 2024-09-10T09:55:27.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:27.005+0000 7f87e2a00640 10 client.25346 handle_command_reply: tid=0 2024-09-10T09:55:27.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:27.005+0000 7f8806a39740 1 client.25346 shutdown 2024-09-10T09:55:27.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:27.006+0000 7f8806a39740 1 -- 172.21.15.77:0/437686912 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c612b69b90 msgr2=0x55c612b78d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:27.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:27.006+0000 7f8806a39740 1 --2- 172.21.15.77:0/437686912 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c612b69b90 0x55c612b78d30 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f87f0000c00 tx=0x7f87f002a3f0 comp rx=0 tx=0).stop 2024-09-10T09:55:27.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:27.006+0000 7f8806a39740 1 -- 172.21.15.77:0/437686912 shutdown_connections 2024-09-10T09:55:27.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:27.006+0000 7f8806a39740 1 --2- 172.21.15.77:0/437686912 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55c612bf0010 0x55c612bf2400 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:27.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:27.006+0000 7f8806a39740 1 --2- 172.21.15.77:0/437686912 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c612b74d70 0x55c612b797b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:27.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:27.006+0000 7f8806a39740 1 --2- 172.21.15.77:0/437686912 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c612b6a4b0 0x55c612b79270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:27.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:27.006+0000 7f8806a39740 1 --2- 172.21.15.77:0/437686912 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c612b69b90 0x55c612b78d30 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:27.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:27.006+0000 7f8806a39740 1 -- 172.21.15.77:0/437686912 >> 172.21.15.77:0/437686912 conn(0x55c612b66120 msgr2=0x55c612b651c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:27.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:27.006+0000 7f87e2a00640 0 client.25346 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T09:55:27.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:27.006+0000 7f8806a39740 1 -- 172.21.15.77:0/437686912 shutdown_connections 2024-09-10T09:55:27.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:27.006+0000 7f8806a39740 1 -- 172.21.15.77:0/437686912 wait complete. 2024-09-10T09:55:27.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:27.006+0000 7f8806a39740 20 client.25346 trim_cache size 0 max 16384 2024-09-10T09:55:27.009 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-10T09:55:27.009 INFO:teuthology.orchestra.run.smithi077.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T09:55:27.009 INFO:teuthology.orchestra.run.smithi077.stdout: "scrubs": { 2024-09-10T09:55:27.009 INFO:teuthology.orchestra.run.smithi077.stdout: "0f675642-7920-4b2e-b937-aa35f6a02f29": { 2024-09-10T09:55:27.009 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/", 2024-09-10T09:55:27.009 INFO:teuthology.orchestra.run.smithi077.stdout: "tag": "0f675642-7920-4b2e-b937-aa35f6a02f29", 2024-09-10T09:55:27.010 INFO:teuthology.orchestra.run.smithi077.stdout: "options": "recursive,force" 2024-09-10T09:55:27.010 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T09:55:27.010 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T09:55:27.010 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-10T09:55:27.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:27.007+0000 7f8804000640 1 -- 172.21.15.77:0/3441999647 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f87e40a2c00 msgr2=0x7f87e40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:27.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:27.007+0000 7f8804000640 1 --2- 172.21.15.77:0/3441999647 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f87e40a2c00 0x7f87e40a50c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f87f4009fd0 tx=0x7f87f4009500 comp rx=0 tx=0).stop 2024-09-10T09:55:27.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:27.007+0000 7f8804000640 1 -- 172.21.15.77:0/3441999647 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f87fc104010 msgr2=0x7f87fc19a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:27.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:27.007+0000 7f8804000640 1 --2- 172.21.15.77:0/3441999647 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f87fc104010 0x7f87fc19a6b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f87f0004770 tx=0x7f87f0003c10 comp rx=0 tx=0).stop 2024-09-10T09:55:27.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:27.007+0000 7f8804000640 1 -- 172.21.15.77:0/3441999647 shutdown_connections 2024-09-10T09:55:27.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:27.007+0000 7f8804000640 1 --2- 172.21.15.77:0/3441999647 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f87e40a2c00 0x7f87e40a50c0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:27.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:27.007+0000 7f8804000640 1 --2- 172.21.15.77:0/3441999647 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87fc109140 0x7f87fc19ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:27.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:27.007+0000 7f8804000640 1 --2- 172.21.15.77:0/3441999647 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f87fc1049c0 0x7f87fc19abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:27.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:27.007+0000 7f8804000640 1 --2- 172.21.15.77:0/3441999647 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f87fc104010 0x7f87fc19a6b0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:27.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:27.007+0000 7f8804000640 1 -- 172.21.15.77:0/3441999647 >> 172.21.15.77:0/3441999647 conn(0x7f87fc0ffa50 msgr2=0x7f87fc100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:27.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:27.007+0000 7f8804000640 1 -- 172.21.15.77:0/3441999647 shutdown_connections 2024-09-10T09:55:27.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:27.008+0000 7f8804000640 1 -- 172.21.15.77:0/3441999647 wait complete. 2024-09-10T09:55:27.021 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:55:27.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:55:26 smithi120 ceph-mon[34770]: from='client.15279 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T09:55:27.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:55:26 smithi120 ceph-mon[34770]: pgmap v399: 129 pgs: 129 active+clean; 651 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-10T09:55:27.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:55:26 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:55:27.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:55:26 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:55:27.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:55:26 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:55:27.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:55:26 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:55:27.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:55:26 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:55:27.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:55:26 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:55:27.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:55:26 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:55:27.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:55:26 smithi077 ceph-mon[30319]: from='client.15279 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T09:55:27.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:55:26 smithi077 ceph-mon[30319]: pgmap v399: 129 pgs: 129 active+clean; 651 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-10T09:55:27.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:55:26 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:55:27.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:55:26 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:55:27.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:55:26 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:55:27.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:55:26 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:55:27.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:55:26 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:55:27.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:55:26 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:55:27.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:55:26 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:55:27.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:55:26 smithi134 ceph-mon[35674]: from='client.15279 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T09:55:27.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:55:26 smithi134 ceph-mon[35674]: pgmap v399: 129 pgs: 129 active+clean; 651 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-10T09:55:27.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:55:26 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:55:27.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:55:26 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:55:27.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:55:26 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:55:27.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:55:26 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:55:27.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:55:26 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:55:27.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:55:26 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:55:27.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:55:26 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:55:27.914 INFO:tasks.cephfs.filesystem:scrub status for tag:0f675642-7920-4b2e-b937-aa35f6a02f29 - {'path': '/', 'tag': '0f675642-7920-4b2e-b937-aa35f6a02f29', 'options': 'recursive,force'} 2024-09-10T09:55:27.991 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:55:27 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:55:27.991 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:55:27 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:55:27.991 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:55:27 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:55:28.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:55:27 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:55:28.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:55:27 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:55:28.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:55:27 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:55:28.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:55:27 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:55:28.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:55:27 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:55:28.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:55:27 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:55:28.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:28.420+0000 7f51ff000640 1 -- 172.21.15.77:0/3278433560 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f51f8106f10 msgr2=0x7f51f810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:28.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:28.420+0000 7f51ff000640 1 --2- 172.21.15.77:0/3278433560 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f51f8106f10 0x7f51f810ea90 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f51f0009ca0 tx=0x7f51f002a8c0 comp rx=0 tx=0).stop 2024-09-10T09:55:28.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:28.420+0000 7f51ff000640 1 -- 172.21.15.77:0/3278433560 shutdown_connections 2024-09-10T09:55:28.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:28.420+0000 7f51ff000640 1 --2- 172.21.15.77:0/3278433560 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f51f8106f10 0x7f51f810ea90 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:28.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:28.420+0000 7f51ff000640 1 --2- 172.21.15.77:0/3278433560 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51f8102790 0x7f51f81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:28.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:28.420+0000 7f51ff000640 1 --2- 172.21.15.77:0/3278433560 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f51f8101de0 0x7f51f81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:28.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:28.420+0000 7f51ff000640 1 -- 172.21.15.77:0/3278433560 >> 172.21.15.77:0/3278433560 conn(0x7f51f80fd8c0 msgr2=0x7f51f80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:28.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:28.420+0000 7f51ff000640 1 -- 172.21.15.77:0/3278433560 shutdown_connections 2024-09-10T09:55:28.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:28.420+0000 7f51ff000640 1 -- 172.21.15.77:0/3278433560 wait complete. 2024-09-10T09:55:28.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:28.421+0000 7f51ff000640 1 Processor -- start 2024-09-10T09:55:28.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:28.421+0000 7f51ff000640 1 -- start start 2024-09-10T09:55:28.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:28.422+0000 7f51ff000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f51f8101de0 0x7f51f819a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:28.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:28.422+0000 7f51ff000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f51f8102790 0x7f51f819ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:28.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:28.422+0000 7f51ff000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51f8106f10 0x7f51f819ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:28.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:28.422+0000 7f51ff000640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f51f8111fc0 con 0x7f51f8102790 2024-09-10T09:55:28.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:28.422+0000 7f51ff000640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f51f8111e40 con 0x7f51f8101de0 2024-09-10T09:55:28.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:28.422+0000 7f51ff000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f51f8112140 con 0x7f51f8106f10 2024-09-10T09:55:28.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:28.422+0000 7f51f7400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f51f8102790 0x7f51f819ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:28.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:28.422+0000 7f51f7400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f51f8102790 0x7f51f819ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56968/0 (socket says 172.21.15.77:56968) 2024-09-10T09:55:28.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:28.422+0000 7f51f7400640 1 -- 172.21.15.77:0/187155896 learned_addr learned my addr 172.21.15.77:0/187155896 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:55:28.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:28.422+0000 7f51f7e00640 1 --2- 172.21.15.77:0/187155896 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f51f8101de0 0x7f51f819a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:28.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:28.422+0000 7f51fd000640 1 --2- 172.21.15.77:0/187155896 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51f8106f10 0x7f51f819ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:28.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:28.423+0000 7f51f7400640 1 -- 172.21.15.77:0/187155896 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51f8106f10 msgr2=0x7f51f819ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:28.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:28.423+0000 7f51f7400640 1 --2- 172.21.15.77:0/187155896 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51f8106f10 0x7f51f819ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:28.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:28.423+0000 7f51f7400640 1 -- 172.21.15.77:0/187155896 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f51f8101de0 msgr2=0x7f51f819a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:28.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:28.423+0000 7f51f7400640 1 --2- 172.21.15.77:0/187155896 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f51f8101de0 0x7f51f819a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:28.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:28.423+0000 7f51f7400640 1 -- 172.21.15.77:0/187155896 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51f819f5f0 con 0x7f51f8102790 2024-09-10T09:55:28.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:28.423+0000 7f51f7e00640 1 --2- 172.21.15.77:0/187155896 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f51f8101de0 0x7f51f819a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:55:28.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:28.423+0000 7f51f7400640 1 --2- 172.21.15.77:0/187155896 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f51f8102790 0x7f51f819ab80 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f51e400e9f0 tx=0x7f51e400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:28.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:28.423+0000 7f51f4c00640 1 -- 172.21.15.77:0/187155896 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51e400cd80 con 0x7f51f8102790 2024-09-10T09:55:28.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:28.423+0000 7f51ff000640 1 -- 172.21.15.77:0/187155896 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51f819f8e0 con 0x7f51f8102790 2024-09-10T09:55:28.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:28.423+0000 7f51f4c00640 1 -- 172.21.15.77:0/187155896 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51e4010910 con 0x7f51f8102790 2024-09-10T09:55:28.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:28.423+0000 7f51f4c00640 1 -- 172.21.15.77:0/187155896 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51e40188f0 con 0x7f51f8102790 2024-09-10T09:55:28.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:28.423+0000 7f51ff000640 1 -- 172.21.15.77:0/187155896 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51f81a71c0 con 0x7f51f8102790 2024-09-10T09:55:28.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:28.425+0000 7f51ff000640 1 -- 172.21.15.77:0/187155896 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51f81021c0 con 0x7f51f8102790 2024-09-10T09:55:28.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:28.427+0000 7f51f4c00640 1 -- 172.21.15.77:0/187155896 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f51e4010ac0 con 0x7f51f8102790 2024-09-10T09:55:28.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:28.428+0000 7f51f4c00640 1 --2- 172.21.15.77:0/187155896 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f51dc0a2b80 0x7f51dc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:28.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:28.428+0000 7f51f4c00640 1 -- 172.21.15.77:0/187155896 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f51e40cf7a0 con 0x7f51f8102790 2024-09-10T09:55:28.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:28.428+0000 7f51f7e00640 1 --2- 172.21.15.77:0/187155896 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f51dc0a2b80 0x7f51dc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:28.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:28.429+0000 7f51f7e00640 1 --2- 172.21.15.77:0/187155896 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f51dc0a2b80 0x7f51dc0a5040 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f51e8009fc0 tx=0x7f51e8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:28.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:28.432+0000 7f51f4c00640 1 -- 172.21.15.77:0/187155896 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f51e409bc60 con 0x7f51f8102790 2024-09-10T09:55:28.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:28.602+0000 7f51ff000640 1 -- 172.21.15.77:0/187155896 --> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f51f8075430 con 0x7f51dc0a2b80 2024-09-10T09:55:28.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:28.612+0000 7f51f4c00640 1 -- 172.21.15.77:0/187155896 <== mgr.14150 v2:172.21.15.77:6800/4261324650 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340856 (secure 0 0 0) 0x7f51f8075430 con 0x7f51dc0a2b80 2024-09-10T09:55:28.616 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:55:28.618 INFO:teuthology.orchestra.run.smithi077.stderr:dumped all 2024-09-10T09:55:28.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:28.617+0000 7f51ff000640 1 -- 172.21.15.77:0/187155896 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f51dc0a2b80 msgr2=0x7f51dc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:28.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:28.617+0000 7f51ff000640 1 --2- 172.21.15.77:0/187155896 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f51dc0a2b80 0x7f51dc0a5040 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f51e8009fc0 tx=0x7f51e8005f90 comp rx=0 tx=0).stop 2024-09-10T09:55:28.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:28.617+0000 7f51ff000640 1 -- 172.21.15.77:0/187155896 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f51f8102790 msgr2=0x7f51f819ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:28.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:28.617+0000 7f51ff000640 1 --2- 172.21.15.77:0/187155896 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f51f8102790 0x7f51f819ab80 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f51e400e9f0 tx=0x7f51e400eeb0 comp rx=0 tx=0).stop 2024-09-10T09:55:28.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:28.617+0000 7f51ff000640 1 -- 172.21.15.77:0/187155896 shutdown_connections 2024-09-10T09:55:28.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:28.617+0000 7f51ff000640 1 --2- 172.21.15.77:0/187155896 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f51dc0a2b80 0x7f51dc0a5040 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:28.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:28.617+0000 7f51ff000640 1 --2- 172.21.15.77:0/187155896 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51f8106f10 0x7f51f819ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:28.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:28.617+0000 7f51ff000640 1 --2- 172.21.15.77:0/187155896 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f51f8102790 0x7f51f819ab80 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:28.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:28.617+0000 7f51ff000640 1 --2- 172.21.15.77:0/187155896 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f51f8101de0 0x7f51f819a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:28.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:28.617+0000 7f51ff000640 1 -- 172.21.15.77:0/187155896 >> 172.21.15.77:0/187155896 conn(0x7f51f80fd8c0 msgr2=0x7f51f80fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:28.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:28.617+0000 7f51ff000640 1 -- 172.21.15.77:0/187155896 shutdown_connections 2024-09-10T09:55:28.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:28.617+0000 7f51ff000640 1 -- 172.21.15.77:0/187155896 wait complete. 2024-09-10T09:55:29.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:55:28 smithi120 ceph-mon[34770]: pgmap v400: 129 pgs: 129 active+clean; 651 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-10T09:55:29.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:55:28 smithi077 ceph-mon[30319]: pgmap v400: 129 pgs: 129 active+clean; 651 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-10T09:55:29.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:55:28 smithi134 ceph-mon[35674]: pgmap v400: 129 pgs: 129 active+clean; 651 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-10T09:55:29.397 INFO:teuthology.orchestra.run.smithi077.stdout:{"pg_ready":true,"pg_map":{"version":400,"stamp":"2024-09-10T09:55:27.722033+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":666279,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1441,"num_read_kb":852,"num_write":446,"num_write_kb":3098,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1185162,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":866,"ondisk_log_size":866,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":366484,"kb_used_data":13076,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124575340,"statfs":{"total":1151940427776,"available":1151565148160,"internally_reserved":0,"allocated":13389824,"data_stored":8615087,"data_compressed":504633,"data_compressed_allocated":688128,"data_compressed_original":2875392,"omap_allocated":19143,"internal_metadata":361674041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4936,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":109,"num_read_kb":63,"num_write":9,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003451"},"pg_stats":[{"pgid":"2.1f","version":"96'14","reported_seq":199,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:55:25.071673+0000","last_change":"2024-09-10T09:53:00.803590+0000","last_active":"2024-09-10T09:55:25.071673+0000","last_peered":"2024-09-10T09:55:25.071673+0000","last_clean":"2024-09-10T09:55:25.071673+0000","last_became_active":"2024-09-10T09:53:00.798189+0000","last_became_peered":"2024-09-10T09:53:00.798189+0000","last_unstale":"2024-09-10T09:55:25.071673+0000","last_undegraded":"2024-09-10T09:55:25.071673+0000","last_fullsized":"2024-09-10T09:55:25.071673+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:18:48.808836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":132,"num_read_kb":132,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"96'5","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.682459+0000","last_change":"2024-09-10T09:52:22.318195+0000","last_active":"2024-09-10T09:54:29.682459+0000","last_peered":"2024-09-10T09:54:29.682459+0000","last_clean":"2024-09-10T09:54:29.682459+0000","last_became_active":"2024-09-10T09:52:22.315905+0000","last_became_peered":"2024-09-10T09:52:22.315905+0000","last_unstale":"2024-09-10T09:54:29.682459+0000","last_undegraded":"2024-09-10T09:54:29.682459+0000","last_fullsized":"2024-09-10T09:54:29.682459+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:53:55.515984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.843988+0000","last_change":"2024-09-10T09:52:31.369780+0000","last_active":"2024-09-10T09:54:00.843988+0000","last_peered":"2024-09-10T09:54:00.843988+0000","last_clean":"2024-09-10T09:54:00.843988+0000","last_became_active":"2024-09-10T09:52:31.369238+0000","last_became_peered":"2024-09-10T09:52:31.369238+0000","last_unstale":"2024-09-10T09:54:00.843988+0000","last_undegraded":"2024-09-10T09:54:00.843988+0000","last_fullsized":"2024-09-10T09:54:00.843988+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:39:05.318493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"94'2","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.847203+0000","last_change":"2024-09-10T09:52:19.912276+0000","last_active":"2024-09-10T09:54:00.847203+0000","last_peered":"2024-09-10T09:54:00.847203+0000","last_clean":"2024-09-10T09:54:00.847203+0000","last_became_active":"2024-09-10T09:52:19.911452+0000","last_became_peered":"2024-09-10T09:52:19.911452+0000","last_unstale":"2024-09-10T09:54:00.847203+0000","last_undegraded":"2024-09-10T09:54:00.847203+0000","last_fullsized":"2024-09-10T09:54:00.847203+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:24:31.262207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"96'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.705105+0000","last_change":"2024-09-10T09:52:22.327641+0000","last_active":"2024-09-10T09:54:29.705105+0000","last_peered":"2024-09-10T09:54:29.705105+0000","last_clean":"2024-09-10T09:54:29.705105+0000","last_became_active":"2024-09-10T09:52:22.326969+0000","last_became_peered":"2024-09-10T09:52:22.326969+0000","last_unstale":"2024-09-10T09:54:29.705105+0000","last_undegraded":"2024-09-10T09:54:29.705105+0000","last_fullsized":"2024-09-10T09:54:29.705105+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:35:27.825534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.951014+0000","last_change":"2024-09-10T09:52:31.372360+0000","last_active":"2024-09-10T09:54:00.951014+0000","last_peered":"2024-09-10T09:54:00.951014+0000","last_clean":"2024-09-10T09:54:00.951014+0000","last_became_active":"2024-09-10T09:52:31.371957+0000","last_became_peered":"2024-09-10T09:52:31.371957+0000","last_unstale":"2024-09-10T09:54:00.951014+0000","last_undegraded":"2024-09-10T09:54:00.951014+0000","last_fullsized":"2024-09-10T09:54:00.951014+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:49:34.776185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"96'13","reported_seq":55,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:51.750986+0000","last_change":"2024-09-10T09:52:19.843532+0000","last_active":"2024-09-10T09:54:51.750986+0000","last_peered":"2024-09-10T09:54:51.750986+0000","last_clean":"2024-09-10T09:54:51.750986+0000","last_became_active":"2024-09-10T09:52:19.843070+0000","last_became_peered":"2024-09-10T09:52:19.843070+0000","last_unstale":"2024-09-10T09:54:51.750986+0000","last_undegraded":"2024-09-10T09:54:51.750986+0000","last_fullsized":"2024-09-10T09:54:51.750986+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:08:02.223373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"96'9","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.692662+0000","last_change":"2024-09-10T09:52:22.316981+0000","last_active":"2024-09-10T09:54:29.692662+0000","last_peered":"2024-09-10T09:54:29.692662+0000","last_clean":"2024-09-10T09:54:29.692662+0000","last_became_active":"2024-09-10T09:52:22.316648+0000","last_became_peered":"2024-09-10T09:52:22.316648+0000","last_unstale":"2024-09-10T09:54:29.692662+0000","last_undegraded":"2024-09-10T09:54:29.692662+0000","last_fullsized":"2024-09-10T09:54:29.692662+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:28:56.742496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.951485+0000","last_change":"2024-09-10T09:52:31.376811+0000","last_active":"2024-09-10T09:54:00.951485+0000","last_peered":"2024-09-10T09:54:00.951485+0000","last_clean":"2024-09-10T09:54:00.951485+0000","last_became_active":"2024-09-10T09:52:31.376007+0000","last_became_peered":"2024-09-10T09:52:31.376007+0000","last_unstale":"2024-09-10T09:54:00.951485+0000","last_undegraded":"2024-09-10T09:54:00.951485+0000","last_fullsized":"2024-09-10T09:54:00.951485+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:29:26.213281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"94'5","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.149772+0000","last_change":"2024-09-10T09:52:19.837848+0000","last_active":"2024-09-10T09:54:01.149772+0000","last_peered":"2024-09-10T09:54:01.149772+0000","last_clean":"2024-09-10T09:54:01.149772+0000","last_became_active":"2024-09-10T09:52:19.837383+0000","last_became_peered":"2024-09-10T09:52:19.837383+0000","last_unstale":"2024-09-10T09:54:01.149772+0000","last_undegraded":"2024-09-10T09:54:01.149772+0000","last_fullsized":"2024-09-10T09:54:01.149772+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:50:42.813665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"96'15","reported_seq":60,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.685844+0000","last_change":"2024-09-10T09:54:01.060407+0000","last_active":"2024-09-10T09:54:29.685844+0000","last_peered":"2024-09-10T09:54:29.685844+0000","last_clean":"2024-09-10T09:54:29.685844+0000","last_became_active":"2024-09-10T09:54:01.060126+0000","last_became_peered":"2024-09-10T09:54:01.060126+0000","last_unstale":"2024-09-10T09:54:29.685844+0000","last_undegraded":"2024-09-10T09:54:29.685844+0000","last_fullsized":"2024-09-10T09:54:29.685844+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:59:29.765375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.951119+0000","last_change":"2024-09-10T09:52:31.370910+0000","last_active":"2024-09-10T09:54:00.951119+0000","last_peered":"2024-09-10T09:54:00.951119+0000","last_clean":"2024-09-10T09:54:00.951119+0000","last_became_active":"2024-09-10T09:52:31.370467+0000","last_became_peered":"2024-09-10T09:52:31.370467+0000","last_unstale":"2024-09-10T09:54:00.951119+0000","last_undegraded":"2024-09-10T09:54:00.951119+0000","last_fullsized":"2024-09-10T09:54:00.951119+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:49:25.037820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"96'5","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.702762+0000","last_change":"2024-09-10T09:52:22.310708+0000","last_active":"2024-09-10T09:54:29.702762+0000","last_peered":"2024-09-10T09:54:29.702762+0000","last_clean":"2024-09-10T09:54:29.702762+0000","last_became_active":"2024-09-10T09:52:22.310387+0000","last_became_peered":"2024-09-10T09:52:22.310387+0000","last_unstale":"2024-09-10T09:54:29.702762+0000","last_undegraded":"2024-09-10T09:54:29.702762+0000","last_fullsized":"2024-09-10T09:54:29.702762+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:35:03.115408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"94'4","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.149453+0000","last_change":"2024-09-10T09:52:19.841417+0000","last_active":"2024-09-10T09:54:01.149453+0000","last_peered":"2024-09-10T09:54:01.149453+0000","last_clean":"2024-09-10T09:54:01.149453+0000","last_became_active":"2024-09-10T09:52:19.841083+0000","last_became_peered":"2024-09-10T09:52:19.841083+0000","last_unstale":"2024-09-10T09:54:01.149453+0000","last_undegraded":"2024-09-10T09:54:01.149453+0000","last_fullsized":"2024-09-10T09:54:01.149453+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:01:19.008345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.951294+0000","last_change":"2024-09-10T09:52:31.367406+0000","last_active":"2024-09-10T09:54:00.951294+0000","last_peered":"2024-09-10T09:54:00.951294+0000","last_clean":"2024-09-10T09:54:00.951294+0000","last_became_active":"2024-09-10T09:52:31.366961+0000","last_became_peered":"2024-09-10T09:52:31.366961+0000","last_unstale":"2024-09-10T09:54:00.951294+0000","last_undegraded":"2024-09-10T09:54:00.951294+0000","last_fullsized":"2024-09-10T09:54:00.951294+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:04:51.714465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"96'8","reported_seq":19,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.705252+0000","last_change":"2024-09-10T09:54:01.249212+0000","last_active":"2024-09-10T09:54:29.705252+0000","last_peered":"2024-09-10T09:54:29.705252+0000","last_clean":"2024-09-10T09:54:29.705252+0000","last_became_active":"2024-09-10T09:54:01.248840+0000","last_became_peered":"2024-09-10T09:54:01.248840+0000","last_unstale":"2024-09-10T09:54:29.705252+0000","last_undegraded":"2024-09-10T09:54:29.705252+0000","last_fullsized":"2024-09-10T09:54:29.705252+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:39:13.167435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,10],"acting":[9,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.a","version":"94'3","reported_seq":19,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.954964+0000","last_change":"2024-09-10T09:53:00.800421+0000","last_active":"2024-09-10T09:54:00.954964+0000","last_peered":"2024-09-10T09:54:00.954964+0000","last_clean":"2024-09-10T09:54:00.954964+0000","last_became_active":"2024-09-10T09:53:00.800026+0000","last_became_peered":"2024-09-10T09:53:00.800026+0000","last_unstale":"2024-09-10T09:54:00.954964+0000","last_undegraded":"2024-09-10T09:54:00.954964+0000","last_fullsized":"2024-09-10T09:54:00.954964+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:57:45.252420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"94'1","reported_seq":30,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.951388+0000","last_change":"2024-09-10T09:52:31.376605+0000","last_active":"2024-09-10T09:54:00.951388+0000","last_peered":"2024-09-10T09:54:00.951388+0000","last_clean":"2024-09-10T09:54:00.951388+0000","last_became_active":"2024-09-10T09:52:31.376083+0000","last_became_peered":"2024-09-10T09:52:31.376083+0000","last_unstale":"2024-09-10T09:54:00.951388+0000","last_undegraded":"2024-09-10T09:54:00.951388+0000","last_fullsized":"2024-09-10T09:54:00.951388+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:05:10.030673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"96'9","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.697029+0000","last_change":"2024-09-10T09:52:22.327303+0000","last_active":"2024-09-10T09:54:29.697029+0000","last_peered":"2024-09-10T09:54:29.697029+0000","last_clean":"2024-09-10T09:54:29.697029+0000","last_became_active":"2024-09-10T09:52:22.326753+0000","last_became_peered":"2024-09-10T09:52:22.326753+0000","last_unstale":"2024-09-10T09:54:29.697029+0000","last_undegraded":"2024-09-10T09:54:29.697029+0000","last_fullsized":"2024-09-10T09:54:29.697029+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:27:26.668151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"94'7","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.845956+0000","last_change":"2024-09-10T09:52:19.836484+0000","last_active":"2024-09-10T09:54:00.845956+0000","last_peered":"2024-09-10T09:54:00.845956+0000","last_clean":"2024-09-10T09:54:00.845956+0000","last_became_active":"2024-09-10T09:52:19.836130+0000","last_became_peered":"2024-09-10T09:52:19.836130+0000","last_unstale":"2024-09-10T09:54:00.845956+0000","last_undegraded":"2024-09-10T09:54:00.845956+0000","last_fullsized":"2024-09-10T09:54:00.845956+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:31:58.731981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.844293+0000","last_change":"2024-09-10T09:52:31.375279+0000","last_active":"2024-09-10T09:54:00.844293+0000","last_peered":"2024-09-10T09:54:00.844293+0000","last_clean":"2024-09-10T09:54:00.844293+0000","last_became_active":"2024-09-10T09:52:31.374181+0000","last_became_peered":"2024-09-10T09:52:31.374181+0000","last_unstale":"2024-09-10T09:54:00.844293+0000","last_undegraded":"2024-09-10T09:54:00.844293+0000","last_fullsized":"2024-09-10T09:54:00.844293+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:54:51.299224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"96'2","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.676489+0000","last_change":"2024-09-10T09:52:22.314817+0000","last_active":"2024-09-10T09:54:29.676489+0000","last_peered":"2024-09-10T09:54:29.676489+0000","last_clean":"2024-09-10T09:54:29.676489+0000","last_became_active":"2024-09-10T09:52:22.314491+0000","last_became_peered":"2024-09-10T09:52:22.314491+0000","last_unstale":"2024-09-10T09:54:29.676489+0000","last_undegraded":"2024-09-10T09:54:29.676489+0000","last_fullsized":"2024-09-10T09:54:29.676489+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:49:41.941667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"94'2","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.988558+0000","last_change":"2024-09-10T09:52:19.911730+0000","last_active":"2024-09-10T09:54:00.988558+0000","last_peered":"2024-09-10T09:54:00.988558+0000","last_clean":"2024-09-10T09:54:00.988558+0000","last_became_active":"2024-09-10T09:52:19.911393+0000","last_became_peered":"2024-09-10T09:52:19.911393+0000","last_unstale":"2024-09-10T09:54:00.988558+0000","last_undegraded":"2024-09-10T09:54:00.988558+0000","last_fullsized":"2024-09-10T09:54:00.988558+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:46:17.943129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.951301+0000","last_change":"2024-09-10T09:52:31.376498+0000","last_active":"2024-09-10T09:54:00.951301+0000","last_peered":"2024-09-10T09:54:00.951301+0000","last_clean":"2024-09-10T09:54:00.951301+0000","last_became_active":"2024-09-10T09:52:31.375966+0000","last_became_peered":"2024-09-10T09:52:31.375966+0000","last_unstale":"2024-09-10T09:54:00.951301+0000","last_undegraded":"2024-09-10T09:54:00.951301+0000","last_fullsized":"2024-09-10T09:54:00.951301+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:08:30.649399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.705573+0000","last_change":"2024-09-10T09:52:22.331611+0000","last_active":"2024-09-10T09:54:29.705573+0000","last_peered":"2024-09-10T09:54:29.705573+0000","last_clean":"2024-09-10T09:54:29.705573+0000","last_became_active":"2024-09-10T09:52:22.331268+0000","last_became_peered":"2024-09-10T09:52:22.331268+0000","last_unstale":"2024-09-10T09:54:29.705573+0000","last_undegraded":"2024-09-10T09:54:29.705573+0000","last_fullsized":"2024-09-10T09:54:29.705573+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:01:05.433994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"94'7","reported_seq":50,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.951927+0000","last_change":"2024-09-10T09:52:19.838387+0000","last_active":"2024-09-10T09:54:00.951927+0000","last_peered":"2024-09-10T09:54:00.951927+0000","last_clean":"2024-09-10T09:54:00.951927+0000","last_became_active":"2024-09-10T09:52:19.837889+0000","last_became_peered":"2024-09-10T09:52:19.837889+0000","last_unstale":"2024-09-10T09:54:00.951927+0000","last_undegraded":"2024-09-10T09:54:00.951927+0000","last_fullsized":"2024-09-10T09:54:00.951927+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:50:00.187243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.844187+0000","last_change":"2024-09-10T09:52:31.363262+0000","last_active":"2024-09-10T09:54:00.844187+0000","last_peered":"2024-09-10T09:54:00.844187+0000","last_clean":"2024-09-10T09:54:00.844187+0000","last_became_active":"2024-09-10T09:52:31.362762+0000","last_became_peered":"2024-09-10T09:52:31.362762+0000","last_unstale":"2024-09-10T09:54:00.844187+0000","last_undegraded":"2024-09-10T09:54:00.844187+0000","last_fullsized":"2024-09-10T09:54:00.844187+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:48:28.275041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"96'5","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.702169+0000","last_change":"2024-09-10T09:52:22.312472+0000","last_active":"2024-09-10T09:54:29.702169+0000","last_peered":"2024-09-10T09:54:29.702169+0000","last_clean":"2024-09-10T09:54:29.702169+0000","last_became_active":"2024-09-10T09:52:22.312117+0000","last_became_peered":"2024-09-10T09:52:22.312117+0000","last_unstale":"2024-09-10T09:54:29.702169+0000","last_undegraded":"2024-09-10T09:54:29.702169+0000","last_fullsized":"2024-09-10T09:54:29.702169+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:39:10.742137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"96'16","reported_seq":57,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:31.212993+0000","last_change":"2024-09-10T09:52:19.835543+0000","last_active":"2024-09-10T09:54:31.212993+0000","last_peered":"2024-09-10T09:54:31.212993+0000","last_clean":"2024-09-10T09:54:31.212993+0000","last_became_active":"2024-09-10T09:52:19.835170+0000","last_became_peered":"2024-09-10T09:52:19.835170+0000","last_unstale":"2024-09-10T09:54:31.212993+0000","last_undegraded":"2024-09-10T09:54:31.212993+0000","last_fullsized":"2024-09-10T09:54:31.212993+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:37.969277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.845447+0000","last_change":"2024-09-10T09:52:31.372398+0000","last_active":"2024-09-10T09:54:00.845447+0000","last_peered":"2024-09-10T09:54:00.845447+0000","last_clean":"2024-09-10T09:54:00.845447+0000","last_became_active":"2024-09-10T09:52:31.371874+0000","last_became_peered":"2024-09-10T09:52:31.371874+0000","last_unstale":"2024-09-10T09:54:00.845447+0000","last_undegraded":"2024-09-10T09:54:00.845447+0000","last_fullsized":"2024-09-10T09:54:00.845447+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:09:45.374320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"96'9","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.700381+0000","last_change":"2024-09-10T09:52:22.319592+0000","last_active":"2024-09-10T09:54:29.700381+0000","last_peered":"2024-09-10T09:54:29.700381+0000","last_clean":"2024-09-10T09:54:29.700381+0000","last_became_active":"2024-09-10T09:52:22.319298+0000","last_became_peered":"2024-09-10T09:52:22.319298+0000","last_unstale":"2024-09-10T09:54:29.700381+0000","last_undegraded":"2024-09-10T09:54:29.700381+0000","last_fullsized":"2024-09-10T09:54:29.700381+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:56:59.787638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"94'4","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.150011+0000","last_change":"2024-09-10T09:52:19.912923+0000","last_active":"2024-09-10T09:54:01.150011+0000","last_peered":"2024-09-10T09:54:01.150011+0000","last_clean":"2024-09-10T09:54:01.150011+0000","last_became_active":"2024-09-10T09:52:19.912268+0000","last_became_peered":"2024-09-10T09:52:19.912268+0000","last_unstale":"2024-09-10T09:54:01.150011+0000","last_undegraded":"2024-09-10T09:54:01.150011+0000","last_fullsized":"2024-09-10T09:54:01.150011+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:06:45.421004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.955091+0000","last_change":"2024-09-10T09:52:31.372761+0000","last_active":"2024-09-10T09:54:00.955091+0000","last_peered":"2024-09-10T09:54:00.955091+0000","last_clean":"2024-09-10T09:54:00.955091+0000","last_became_active":"2024-09-10T09:52:31.372293+0000","last_became_peered":"2024-09-10T09:52:31.372293+0000","last_unstale":"2024-09-10T09:54:00.955091+0000","last_undegraded":"2024-09-10T09:54:00.955091+0000","last_fullsized":"2024-09-10T09:54:00.955091+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:36:41.446810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"96'4","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.688817+0000","last_change":"2024-09-10T09:52:22.326291+0000","last_active":"2024-09-10T09:54:29.688817+0000","last_peered":"2024-09-10T09:54:29.688817+0000","last_clean":"2024-09-10T09:54:29.688817+0000","last_became_active":"2024-09-10T09:52:22.325985+0000","last_became_peered":"2024-09-10T09:52:22.325985+0000","last_unstale":"2024-09-10T09:54:29.688817+0000","last_undegraded":"2024-09-10T09:54:29.688817+0000","last_fullsized":"2024-09-10T09:54:29.688817+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:06:21.871502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"94'1","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.844068+0000","last_change":"2024-09-10T09:52:19.912851+0000","last_active":"2024-09-10T09:54:00.844068+0000","last_peered":"2024-09-10T09:54:00.844068+0000","last_clean":"2024-09-10T09:54:00.844068+0000","last_became_active":"2024-09-10T09:52:19.912200+0000","last_became_peered":"2024-09-10T09:52:19.912200+0000","last_unstale":"2024-09-10T09:54:00.844068+0000","last_undegraded":"2024-09-10T09:54:00.844068+0000","last_fullsized":"2024-09-10T09:54:00.844068+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:00:47.331755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.149311+0000","last_change":"2024-09-10T09:52:31.371329+0000","last_active":"2024-09-10T09:54:01.149311+0000","last_peered":"2024-09-10T09:54:01.149311+0000","last_clean":"2024-09-10T09:54:01.149311+0000","last_became_active":"2024-09-10T09:52:31.370971+0000","last_became_peered":"2024-09-10T09:52:31.370971+0000","last_unstale":"2024-09-10T09:54:01.149311+0000","last_undegraded":"2024-09-10T09:54:01.149311+0000","last_fullsized":"2024-09-10T09:54:01.149311+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:07:12.928421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"96'7","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.686209+0000","last_change":"2024-09-10T09:52:22.328039+0000","last_active":"2024-09-10T09:54:29.686209+0000","last_peered":"2024-09-10T09:54:29.686209+0000","last_clean":"2024-09-10T09:54:29.686209+0000","last_became_active":"2024-09-10T09:52:22.327588+0000","last_became_peered":"2024-09-10T09:52:22.327588+0000","last_unstale":"2024-09-10T09:54:29.686209+0000","last_undegraded":"2024-09-10T09:54:29.686209+0000","last_fullsized":"2024-09-10T09:54:29.686209+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:35:39.219544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"96'6","reported_seq":150,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:55:21.067336+0000","last_change":"2024-09-10T09:53:00.797651+0000","last_active":"2024-09-10T09:55:21.067336+0000","last_peered":"2024-09-10T09:55:21.067336+0000","last_clean":"2024-09-10T09:55:21.067336+0000","last_became_active":"2024-09-10T09:53:00.797285+0000","last_became_peered":"2024-09-10T09:53:00.797285+0000","last_unstale":"2024-09-10T09:55:21.067336+0000","last_undegraded":"2024-09-10T09:55:21.067336+0000","last_fullsized":"2024-09-10T09:55:21.067336+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:18:02.320032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":899,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":128,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,6,1],"acting":[11,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.841787+0000","last_change":"2024-09-10T09:52:31.379215+0000","last_active":"2024-09-10T09:54:00.841787+0000","last_peered":"2024-09-10T09:54:00.841787+0000","last_clean":"2024-09-10T09:54:00.841787+0000","last_became_active":"2024-09-10T09:52:31.377107+0000","last_became_peered":"2024-09-10T09:52:31.377107+0000","last_unstale":"2024-09-10T09:54:00.841787+0000","last_undegraded":"2024-09-10T09:54:00.841787+0000","last_fullsized":"2024-09-10T09:54:00.841787+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:09:39.474077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"96'8","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.673518+0000","last_change":"2024-09-10T09:52:22.316474+0000","last_active":"2024-09-10T09:54:29.673518+0000","last_peered":"2024-09-10T09:54:29.673518+0000","last_clean":"2024-09-10T09:54:29.673518+0000","last_became_active":"2024-09-10T09:52:22.316054+0000","last_became_peered":"2024-09-10T09:52:22.316054+0000","last_unstale":"2024-09-10T09:54:29.673518+0000","last_undegraded":"2024-09-10T09:54:29.673518+0000","last_fullsized":"2024-09-10T09:54:29.673518+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:11:31.334780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"96'6","reported_seq":176,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:55:22.099423+0000","last_change":"2024-09-10T09:52:19.911601+0000","last_active":"2024-09-10T09:55:22.099423+0000","last_peered":"2024-09-10T09:55:22.099423+0000","last_clean":"2024-09-10T09:55:22.099423+0000","last_became_active":"2024-09-10T09:52:19.911268+0000","last_became_peered":"2024-09-10T09:52:19.911268+0000","last_unstale":"2024-09-10T09:55:22.099423+0000","last_undegraded":"2024-09-10T09:55:22.099423+0000","last_fullsized":"2024-09-10T09:55:22.099423+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:46:38.158784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":128,"num_read_kb":128,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.841796+0000","last_change":"2024-09-10T09:52:31.375938+0000","last_active":"2024-09-10T09:54:00.841796+0000","last_peered":"2024-09-10T09:54:00.841796+0000","last_clean":"2024-09-10T09:54:00.841796+0000","last_became_active":"2024-09-10T09:52:31.375536+0000","last_became_peered":"2024-09-10T09:52:31.375536+0000","last_unstale":"2024-09-10T09:54:00.841796+0000","last_undegraded":"2024-09-10T09:54:00.841796+0000","last_fullsized":"2024-09-10T09:54:00.841796+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:55:17.303809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"96'5","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.688934+0000","last_change":"2024-09-10T09:52:22.315912+0000","last_active":"2024-09-10T09:54:29.688934+0000","last_peered":"2024-09-10T09:54:29.688934+0000","last_clean":"2024-09-10T09:54:29.688934+0000","last_became_active":"2024-09-10T09:52:22.315433+0000","last_became_peered":"2024-09-10T09:52:22.315433+0000","last_unstale":"2024-09-10T09:54:29.688934+0000","last_undegraded":"2024-09-10T09:54:29.688934+0000","last_fullsized":"2024-09-10T09:54:29.688934+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:57:10.454716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"94'4","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.059138+0000","last_change":"2024-09-10T09:52:19.838966+0000","last_active":"2024-09-10T09:54:01.059138+0000","last_peered":"2024-09-10T09:54:01.059138+0000","last_clean":"2024-09-10T09:54:01.059138+0000","last_became_active":"2024-09-10T09:52:19.838545+0000","last_became_peered":"2024-09-10T09:52:19.838545+0000","last_unstale":"2024-09-10T09:54:01.059138+0000","last_undegraded":"2024-09-10T09:54:01.059138+0000","last_fullsized":"2024-09-10T09:54:01.059138+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:07:00.930146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.847601+0000","last_change":"2024-09-10T09:53:00.802039+0000","last_active":"2024-09-10T09:54:00.847601+0000","last_peered":"2024-09-10T09:54:00.847601+0000","last_clean":"2024-09-10T09:54:00.847601+0000","last_became_active":"2024-09-10T09:53:00.800223+0000","last_became_peered":"2024-09-10T09:53:00.800223+0000","last_unstale":"2024-09-10T09:54:00.847601+0000","last_undegraded":"2024-09-10T09:54:00.847601+0000","last_fullsized":"2024-09-10T09:54:00.847601+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:50:16.026631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,6,10],"acting":[5,8,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"96'8","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.698863+0000","last_change":"2024-09-10T09:52:22.313819+0000","last_active":"2024-09-10T09:54:29.698863+0000","last_peered":"2024-09-10T09:54:29.698863+0000","last_clean":"2024-09-10T09:54:29.698863+0000","last_became_active":"2024-09-10T09:52:22.313473+0000","last_became_peered":"2024-09-10T09:52:22.313473+0000","last_unstale":"2024-09-10T09:54:29.698863+0000","last_undegraded":"2024-09-10T09:54:29.698863+0000","last_fullsized":"2024-09-10T09:54:29.698863+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:44:10.003187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"94'6","reported_seq":51,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.951818+0000","last_change":"2024-09-10T09:52:19.837376+0000","last_active":"2024-09-10T09:54:00.951818+0000","last_peered":"2024-09-10T09:54:00.951818+0000","last_clean":"2024-09-10T09:54:00.951818+0000","last_became_active":"2024-09-10T09:52:19.836880+0000","last_became_peered":"2024-09-10T09:52:19.836880+0000","last_unstale":"2024-09-10T09:54:00.951818+0000","last_undegraded":"2024-09-10T09:54:00.951818+0000","last_fullsized":"2024-09-10T09:54:00.951818+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:48:58.198188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":361,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.059392+0000","last_change":"2024-09-10T09:51:05.871529+0000","last_active":"2024-09-10T09:54:01.059392+0000","last_peered":"2024-09-10T09:54:01.059392+0000","last_clean":"2024-09-10T09:54:01.059392+0000","last_became_active":"2024-09-10T09:51:05.871115+0000","last_became_peered":"2024-09-10T09:51:05.871115+0000","last_unstale":"2024-09-10T09:54:01.059392+0000","last_undegraded":"2024-09-10T09:54:01.059392+0000","last_fullsized":"2024-09-10T09:54:01.059392+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:47:24.175358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:47:24.175358+0000","last_clean_scrub_stamp":"2024-09-10T09:47:24.175358+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:58:30.247077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"94'1","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.059534+0000","last_change":"2024-09-10T09:53:00.800474+0000","last_active":"2024-09-10T09:54:01.059534+0000","last_peered":"2024-09-10T09:54:01.059534+0000","last_clean":"2024-09-10T09:54:01.059534+0000","last_became_active":"2024-09-10T09:53:00.799932+0000","last_became_peered":"2024-09-10T09:53:00.799932+0000","last_unstale":"2024-09-10T09:54:01.059534+0000","last_undegraded":"2024-09-10T09:54:01.059534+0000","last_fullsized":"2024-09-10T09:54:01.059534+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:14:50.998952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,11,9],"acting":[7,8,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.693609+0000","last_change":"2024-09-10T09:52:22.312521+0000","last_active":"2024-09-10T09:54:29.693609+0000","last_peered":"2024-09-10T09:54:29.693609+0000","last_clean":"2024-09-10T09:54:29.693609+0000","last_became_active":"2024-09-10T09:52:22.312107+0000","last_became_peered":"2024-09-10T09:52:22.312107+0000","last_unstale":"2024-09-10T09:54:29.693609+0000","last_undegraded":"2024-09-10T09:54:29.693609+0000","last_fullsized":"2024-09-10T09:54:29.693609+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:14:19.799259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"94'5","reported_seq":176,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:55:22.067694+0000","last_change":"2024-09-10T09:52:19.911454+0000","last_active":"2024-09-10T09:55:22.067694+0000","last_peered":"2024-09-10T09:55:22.067694+0000","last_clean":"2024-09-10T09:55:22.067694+0000","last_became_active":"2024-09-10T09:52:19.911079+0000","last_became_peered":"2024-09-10T09:52:19.911079+0000","last_unstale":"2024-09-10T09:55:22.067694+0000","last_undegraded":"2024-09-10T09:55:22.067694+0000","last_fullsized":"2024-09-10T09:55:22.067694+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:43:38.787689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":129,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.988531+0000","last_change":"2024-09-10T09:52:31.381877+0000","last_active":"2024-09-10T09:54:00.988531+0000","last_peered":"2024-09-10T09:54:00.988531+0000","last_clean":"2024-09-10T09:54:00.988531+0000","last_became_active":"2024-09-10T09:52:31.381472+0000","last_became_peered":"2024-09-10T09:52:31.381472+0000","last_unstale":"2024-09-10T09:54:00.988531+0000","last_undegraded":"2024-09-10T09:54:00.988531+0000","last_fullsized":"2024-09-10T09:54:00.988531+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:41:24.686325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"96'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.702544+0000","last_change":"2024-09-10T09:52:22.313902+0000","last_active":"2024-09-10T09:54:29.702544+0000","last_peered":"2024-09-10T09:54:29.702544+0000","last_clean":"2024-09-10T09:54:29.702544+0000","last_became_active":"2024-09-10T09:52:22.313515+0000","last_became_peered":"2024-09-10T09:52:22.313515+0000","last_unstale":"2024-09-10T09:54:29.702544+0000","last_undegraded":"2024-09-10T09:54:29.702544+0000","last_fullsized":"2024-09-10T09:54:29.702544+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:25:00.238307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"94'3","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.847873+0000","last_change":"2024-09-10T09:52:19.837736+0000","last_active":"2024-09-10T09:54:00.847873+0000","last_peered":"2024-09-10T09:54:00.847873+0000","last_clean":"2024-09-10T09:54:00.847873+0000","last_became_active":"2024-09-10T09:52:19.837370+0000","last_became_peered":"2024-09-10T09:52:19.837370+0000","last_unstale":"2024-09-10T09:54:00.847873+0000","last_undegraded":"2024-09-10T09:54:00.847873+0000","last_fullsized":"2024-09-10T09:54:00.847873+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:54:35.753027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.248213+0000","last_change":"2024-09-10T09:52:31.378945+0000","last_active":"2024-09-10T09:54:01.248213+0000","last_peered":"2024-09-10T09:54:01.248213+0000","last_clean":"2024-09-10T09:54:01.248213+0000","last_became_active":"2024-09-10T09:52:31.378144+0000","last_became_peered":"2024-09-10T09:52:31.378144+0000","last_unstale":"2024-09-10T09:54:01.248213+0000","last_undegraded":"2024-09-10T09:54:01.248213+0000","last_fullsized":"2024-09-10T09:54:01.248213+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:36:06.288293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"96'9","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.692039+0000","last_change":"2024-09-10T09:52:22.316400+0000","last_active":"2024-09-10T09:54:29.692039+0000","last_peered":"2024-09-10T09:54:29.692039+0000","last_clean":"2024-09-10T09:54:29.692039+0000","last_became_active":"2024-09-10T09:52:22.315974+0000","last_became_peered":"2024-09-10T09:52:22.315974+0000","last_unstale":"2024-09-10T09:54:29.692039+0000","last_undegraded":"2024-09-10T09:54:29.692039+0000","last_fullsized":"2024-09-10T09:54:29.692039+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:31:05.538237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"94'5","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.841976+0000","last_change":"2024-09-10T09:52:19.837942+0000","last_active":"2024-09-10T09:54:00.841976+0000","last_peered":"2024-09-10T09:54:00.841976+0000","last_clean":"2024-09-10T09:54:00.841976+0000","last_became_active":"2024-09-10T09:52:19.837629+0000","last_became_peered":"2024-09-10T09:52:19.837629+0000","last_unstale":"2024-09-10T09:54:00.841976+0000","last_undegraded":"2024-09-10T09:54:00.841976+0000","last_fullsized":"2024-09-10T09:54:00.841976+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:34:13.207770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.847019+0000","last_change":"2024-09-10T09:52:31.364231+0000","last_active":"2024-09-10T09:54:00.847019+0000","last_peered":"2024-09-10T09:54:00.847019+0000","last_clean":"2024-09-10T09:54:00.847019+0000","last_became_active":"2024-09-10T09:52:31.363833+0000","last_became_peered":"2024-09-10T09:52:31.363833+0000","last_unstale":"2024-09-10T09:54:00.847019+0000","last_undegraded":"2024-09-10T09:54:00.847019+0000","last_fullsized":"2024-09-10T09:54:00.847019+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:53:51.435700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"96'5","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.680562+0000","last_change":"2024-09-10T09:52:22.319530+0000","last_active":"2024-09-10T09:54:29.680562+0000","last_peered":"2024-09-10T09:54:29.680562+0000","last_clean":"2024-09-10T09:54:29.680562+0000","last_became_active":"2024-09-10T09:52:22.318742+0000","last_became_peered":"2024-09-10T09:52:22.318742+0000","last_unstale":"2024-09-10T09:54:29.680562+0000","last_undegraded":"2024-09-10T09:54:29.680562+0000","last_fullsized":"2024-09-10T09:54:29.680562+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:44:35.050454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"94'2","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.150048+0000","last_change":"2024-09-10T09:52:19.836398+0000","last_active":"2024-09-10T09:54:01.150048+0000","last_peered":"2024-09-10T09:54:01.150048+0000","last_clean":"2024-09-10T09:54:01.150048+0000","last_became_active":"2024-09-10T09:52:19.836055+0000","last_became_peered":"2024-09-10T09:52:19.836055+0000","last_unstale":"2024-09-10T09:54:01.150048+0000","last_undegraded":"2024-09-10T09:54:01.150048+0000","last_fullsized":"2024-09-10T09:54:01.150048+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:49:26.923367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.847215+0000","last_change":"2024-09-10T09:52:31.371109+0000","last_active":"2024-09-10T09:54:00.847215+0000","last_peered":"2024-09-10T09:54:00.847215+0000","last_clean":"2024-09-10T09:54:00.847215+0000","last_became_active":"2024-09-10T09:52:31.370592+0000","last_became_peered":"2024-09-10T09:52:31.370592+0000","last_unstale":"2024-09-10T09:54:00.847215+0000","last_undegraded":"2024-09-10T09:54:00.847215+0000","last_fullsized":"2024-09-10T09:54:00.847215+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:20:23.585187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"96'6","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.685236+0000","last_change":"2024-09-10T09:52:22.319365+0000","last_active":"2024-09-10T09:54:29.685236+0000","last_peered":"2024-09-10T09:54:29.685236+0000","last_clean":"2024-09-10T09:54:29.685236+0000","last_became_active":"2024-09-10T09:52:22.319064+0000","last_became_peered":"2024-09-10T09:52:22.319064+0000","last_unstale":"2024-09-10T09:54:29.685236+0000","last_undegraded":"2024-09-10T09:54:29.685236+0000","last_fullsized":"2024-09-10T09:54:29.685236+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:14:50.089566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"94'17","reported_seq":59,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.149324+0000","last_change":"2024-09-10T09:52:19.911698+0000","last_active":"2024-09-10T09:54:01.149324+0000","last_peered":"2024-09-10T09:54:01.149324+0000","last_clean":"2024-09-10T09:54:01.149324+0000","last_became_active":"2024-09-10T09:52:19.911369+0000","last_became_peered":"2024-09-10T09:52:19.911369+0000","last_unstale":"2024-09-10T09:54:01.149324+0000","last_undegraded":"2024-09-10T09:54:01.149324+0000","last_fullsized":"2024-09-10T09:54:01.149324+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:40:39.045797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.841947+0000","last_change":"2024-09-10T09:52:31.380463+0000","last_active":"2024-09-10T09:54:00.841947+0000","last_peered":"2024-09-10T09:54:00.841947+0000","last_clean":"2024-09-10T09:54:00.841947+0000","last_became_active":"2024-09-10T09:52:31.380114+0000","last_became_peered":"2024-09-10T09:52:31.380114+0000","last_unstale":"2024-09-10T09:54:00.841947+0000","last_undegraded":"2024-09-10T09:54:00.841947+0000","last_fullsized":"2024-09-10T09:54:00.841947+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:19:50.869160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"96'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.682831+0000","last_change":"2024-09-10T09:52:22.313928+0000","last_active":"2024-09-10T09:54:29.682831+0000","last_peered":"2024-09-10T09:54:29.682831+0000","last_clean":"2024-09-10T09:54:29.682831+0000","last_became_active":"2024-09-10T09:52:22.313510+0000","last_became_peered":"2024-09-10T09:52:22.313510+0000","last_unstale":"2024-09-10T09:54:29.682831+0000","last_undegraded":"2024-09-10T09:54:29.682831+0000","last_fullsized":"2024-09-10T09:54:29.682831+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:58:19.031178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"96'7","reported_seq":180,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:55:22.100266+0000","last_change":"2024-09-10T09:52:19.839451+0000","last_active":"2024-09-10T09:55:22.100266+0000","last_peered":"2024-09-10T09:55:22.100266+0000","last_clean":"2024-09-10T09:55:22.100266+0000","last_became_active":"2024-09-10T09:52:19.839124+0000","last_became_peered":"2024-09-10T09:52:19.839124+0000","last_unstale":"2024-09-10T09:55:22.100266+0000","last_undegraded":"2024-09-10T09:55:22.100266+0000","last_fullsized":"2024-09-10T09:55:22.100266+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:51:33.489830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":132,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.844344+0000","last_change":"2024-09-10T09:52:31.376386+0000","last_active":"2024-09-10T09:54:00.844344+0000","last_peered":"2024-09-10T09:54:00.844344+0000","last_clean":"2024-09-10T09:54:00.844344+0000","last_became_active":"2024-09-10T09:52:31.376066+0000","last_became_peered":"2024-09-10T09:52:31.376066+0000","last_unstale":"2024-09-10T09:54:00.844344+0000","last_undegraded":"2024-09-10T09:54:00.844344+0000","last_fullsized":"2024-09-10T09:54:00.844344+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:23:07.191842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"96'15","reported_seq":52,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.702901+0000","last_change":"2024-09-10T09:52:22.323590+0000","last_active":"2024-09-10T09:54:29.702901+0000","last_peered":"2024-09-10T09:54:29.702901+0000","last_clean":"2024-09-10T09:54:29.702901+0000","last_became_active":"2024-09-10T09:52:22.323264+0000","last_became_peered":"2024-09-10T09:52:22.323264+0000","last_unstale":"2024-09-10T09:54:29.702901+0000","last_undegraded":"2024-09-10T09:54:29.702901+0000","last_fullsized":"2024-09-10T09:54:29.702901+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:07:56.572337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"94'5","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.847940+0000","last_change":"2024-09-10T09:52:19.840620+0000","last_active":"2024-09-10T09:54:00.847940+0000","last_peered":"2024-09-10T09:54:00.847940+0000","last_clean":"2024-09-10T09:54:00.847940+0000","last_became_active":"2024-09-10T09:52:19.840217+0000","last_became_peered":"2024-09-10T09:52:19.840217+0000","last_unstale":"2024-09-10T09:54:00.847940+0000","last_undegraded":"2024-09-10T09:54:00.847940+0000","last_fullsized":"2024-09-10T09:54:00.847940+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:55:13.111993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.955071+0000","last_change":"2024-09-10T09:52:31.373895+0000","last_active":"2024-09-10T09:54:00.955071+0000","last_peered":"2024-09-10T09:54:00.955071+0000","last_clean":"2024-09-10T09:54:00.955071+0000","last_became_active":"2024-09-10T09:52:31.373474+0000","last_became_peered":"2024-09-10T09:52:31.373474+0000","last_unstale":"2024-09-10T09:54:00.955071+0000","last_undegraded":"2024-09-10T09:54:00.955071+0000","last_fullsized":"2024-09-10T09:54:00.955071+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:39:16.049859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"96'9","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.704377+0000","last_change":"2024-09-10T09:52:22.319294+0000","last_active":"2024-09-10T09:54:29.704377+0000","last_peered":"2024-09-10T09:54:29.704377+0000","last_clean":"2024-09-10T09:54:29.704377+0000","last_became_active":"2024-09-10T09:52:22.318918+0000","last_became_peered":"2024-09-10T09:52:22.318918+0000","last_unstale":"2024-09-10T09:54:29.704377+0000","last_undegraded":"2024-09-10T09:54:29.704377+0000","last_fullsized":"2024-09-10T09:54:29.704377+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:52:46.434182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"94'9","reported_seq":53,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.844244+0000","last_change":"2024-09-10T09:52:19.842286+0000","last_active":"2024-09-10T09:54:00.844244+0000","last_peered":"2024-09-10T09:54:00.844244+0000","last_clean":"2024-09-10T09:54:00.844244+0000","last_became_active":"2024-09-10T09:52:19.841930+0000","last_became_peered":"2024-09-10T09:52:19.841930+0000","last_unstale":"2024-09-10T09:54:00.844244+0000","last_undegraded":"2024-09-10T09:54:00.844244+0000","last_fullsized":"2024-09-10T09:54:00.844244+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:38:46.358389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.847531+0000","last_change":"2024-09-10T09:52:31.370961+0000","last_active":"2024-09-10T09:54:00.847531+0000","last_peered":"2024-09-10T09:54:00.847531+0000","last_clean":"2024-09-10T09:54:00.847531+0000","last_became_active":"2024-09-10T09:52:31.370545+0000","last_became_peered":"2024-09-10T09:52:31.370545+0000","last_unstale":"2024-09-10T09:54:00.847531+0000","last_undegraded":"2024-09-10T09:54:00.847531+0000","last_fullsized":"2024-09-10T09:54:00.847531+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:17.461447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"96'6","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.685895+0000","last_change":"2024-09-10T09:52:22.313215+0000","last_active":"2024-09-10T09:54:29.685895+0000","last_peered":"2024-09-10T09:54:29.685895+0000","last_clean":"2024-09-10T09:54:29.685895+0000","last_became_active":"2024-09-10T09:52:22.312918+0000","last_became_peered":"2024-09-10T09:52:22.312918+0000","last_unstale":"2024-09-10T09:54:29.685895+0000","last_undegraded":"2024-09-10T09:54:29.685895+0000","last_fullsized":"2024-09-10T09:54:29.685895+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:48:09.568004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"96'40","reported_seq":362,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:55:22.066770+0000","last_change":"2024-09-10T09:53:01.226146+0000","last_active":"2024-09-10T09:55:22.066770+0000","last_peered":"2024-09-10T09:55:22.066770+0000","last_clean":"2024-09-10T09:55:22.066770+0000","last_became_active":"2024-09-10T09:53:01.218622+0000","last_became_peered":"2024-09-10T09:53:01.218622+0000","last_unstale":"2024-09-10T09:55:22.066770+0000","last_undegraded":"2024-09-10T09:55:22.066770+0000","last_fullsized":"2024-09-10T09:55:22.066770+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:43:11.927029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67876,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":263,"num_read_kb":198,"num_write":42,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":3844,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.841475+0000","last_change":"2024-09-10T09:53:00.800650+0000","last_active":"2024-09-10T09:54:00.841475+0000","last_peered":"2024-09-10T09:54:00.841475+0000","last_clean":"2024-09-10T09:54:00.841475+0000","last_became_active":"2024-09-10T09:53:00.800104+0000","last_became_peered":"2024-09-10T09:53:00.800104+0000","last_unstale":"2024-09-10T09:54:00.841475+0000","last_undegraded":"2024-09-10T09:54:00.841475+0000","last_fullsized":"2024-09-10T09:54:00.841475+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:41:16.022884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.14","version":"96'11","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.690121+0000","last_change":"2024-09-10T09:52:22.317174+0000","last_active":"2024-09-10T09:54:29.690121+0000","last_peered":"2024-09-10T09:54:29.690121+0000","last_clean":"2024-09-10T09:54:29.690121+0000","last_became_active":"2024-09-10T09:52:22.316851+0000","last_became_peered":"2024-09-10T09:52:22.316851+0000","last_unstale":"2024-09-10T09:54:29.690121+0000","last_undegraded":"2024-09-10T09:54:29.690121+0000","last_fullsized":"2024-09-10T09:54:29.690121+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:55:18.742393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"94'5","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.847731+0000","last_change":"2024-09-10T09:52:19.912093+0000","last_active":"2024-09-10T09:54:00.847731+0000","last_peered":"2024-09-10T09:54:00.847731+0000","last_clean":"2024-09-10T09:54:00.847731+0000","last_became_active":"2024-09-10T09:52:19.911611+0000","last_became_peered":"2024-09-10T09:52:19.911611+0000","last_unstale":"2024-09-10T09:54:00.847731+0000","last_undegraded":"2024-09-10T09:54:00.847731+0000","last_fullsized":"2024-09-10T09:54:00.847731+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:01:10.859310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.951658+0000","last_change":"2024-09-10T09:52:31.362884+0000","last_active":"2024-09-10T09:54:00.951658+0000","last_peered":"2024-09-10T09:54:00.951658+0000","last_clean":"2024-09-10T09:54:00.951658+0000","last_became_active":"2024-09-10T09:52:31.362489+0000","last_became_peered":"2024-09-10T09:52:31.362489+0000","last_unstale":"2024-09-10T09:54:00.951658+0000","last_undegraded":"2024-09-10T09:54:00.951658+0000","last_fullsized":"2024-09-10T09:54:00.951658+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:02:33.084795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"96'9","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.692830+0000","last_change":"2024-09-10T09:52:22.325871+0000","last_active":"2024-09-10T09:54:29.692830+0000","last_peered":"2024-09-10T09:54:29.692830+0000","last_clean":"2024-09-10T09:54:29.692830+0000","last_became_active":"2024-09-10T09:52:22.324980+0000","last_became_peered":"2024-09-10T09:52:22.324980+0000","last_unstale":"2024-09-10T09:54:29.692830+0000","last_undegraded":"2024-09-10T09:54:29.692830+0000","last_fullsized":"2024-09-10T09:54:29.692830+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:31:55.713444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"94'4","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.059691+0000","last_change":"2024-09-10T09:52:19.839491+0000","last_active":"2024-09-10T09:54:01.059691+0000","last_peered":"2024-09-10T09:54:01.059691+0000","last_clean":"2024-09-10T09:54:01.059691+0000","last_became_active":"2024-09-10T09:52:19.839092+0000","last_became_peered":"2024-09-10T09:52:19.839092+0000","last_unstale":"2024-09-10T09:54:01.059691+0000","last_undegraded":"2024-09-10T09:54:01.059691+0000","last_fullsized":"2024-09-10T09:54:01.059691+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:42:13.453281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.843285+0000","last_change":"2024-09-10T09:52:31.371346+0000","last_active":"2024-09-10T09:54:00.843285+0000","last_peered":"2024-09-10T09:54:00.843285+0000","last_clean":"2024-09-10T09:54:00.843285+0000","last_became_active":"2024-09-10T09:52:31.370929+0000","last_became_peered":"2024-09-10T09:52:31.370929+0000","last_unstale":"2024-09-10T09:54:00.843285+0000","last_undegraded":"2024-09-10T09:54:00.843285+0000","last_fullsized":"2024-09-10T09:54:00.843285+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:44:22.925082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"96'10","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.694062+0000","last_change":"2024-09-10T09:52:22.317453+0000","last_active":"2024-09-10T09:54:29.694062+0000","last_peered":"2024-09-10T09:54:29.694062+0000","last_clean":"2024-09-10T09:54:29.694062+0000","last_became_active":"2024-09-10T09:52:22.315556+0000","last_became_peered":"2024-09-10T09:52:22.315556+0000","last_unstale":"2024-09-10T09:54:29.694062+0000","last_undegraded":"2024-09-10T09:54:29.694062+0000","last_fullsized":"2024-09-10T09:54:29.694062+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:56:05.860275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"96'5","reported_seq":300,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:55:22.066812+0000","last_change":"2024-09-10T09:52:19.835985+0000","last_active":"2024-09-10T09:55:22.066812+0000","last_peered":"2024-09-10T09:55:22.066812+0000","last_clean":"2024-09-10T09:55:22.066812+0000","last_became_active":"2024-09-10T09:52:19.834835+0000","last_became_peered":"2024-09-10T09:52:19.834835+0000","last_unstale":"2024-09-10T09:55:22.066812+0000","last_undegraded":"2024-09-10T09:55:22.066812+0000","last_fullsized":"2024-09-10T09:55:22.066812+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:09:11.590443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":252,"num_read_kb":252,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.845724+0000","last_change":"2024-09-10T09:52:31.368663+0000","last_active":"2024-09-10T09:54:00.845724+0000","last_peered":"2024-09-10T09:54:00.845724+0000","last_clean":"2024-09-10T09:54:00.845724+0000","last_became_active":"2024-09-10T09:52:31.368004+0000","last_became_peered":"2024-09-10T09:52:31.368004+0000","last_unstale":"2024-09-10T09:54:00.845724+0000","last_undegraded":"2024-09-10T09:54:00.845724+0000","last_fullsized":"2024-09-10T09:54:00.845724+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:56:15.100515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.247917+0000","last_change":"2024-09-10T09:52:31.370992+0000","last_active":"2024-09-10T09:54:01.247917+0000","last_peered":"2024-09-10T09:54:01.247917+0000","last_clean":"2024-09-10T09:54:01.247917+0000","last_became_active":"2024-09-10T09:52:31.370357+0000","last_became_peered":"2024-09-10T09:52:31.370357+0000","last_unstale":"2024-09-10T09:54:01.247917+0000","last_undegraded":"2024-09-10T09:54:01.247917+0000","last_fullsized":"2024-09-10T09:54:01.247917+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:02:49.482532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"96'11","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.706426+0000","last_change":"2024-09-10T09:52:22.319363+0000","last_active":"2024-09-10T09:54:29.706426+0000","last_peered":"2024-09-10T09:54:29.706426+0000","last_clean":"2024-09-10T09:54:29.706426+0000","last_became_active":"2024-09-10T09:52:22.319044+0000","last_became_peered":"2024-09-10T09:52:22.319044+0000","last_unstale":"2024-09-10T09:54:29.706426+0000","last_undegraded":"2024-09-10T09:54:29.706426+0000","last_fullsized":"2024-09-10T09:54:29.706426+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:30:51.328413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"94'3","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.847831+0000","last_change":"2024-09-10T09:52:19.841745+0000","last_active":"2024-09-10T09:54:00.847831+0000","last_peered":"2024-09-10T09:54:00.847831+0000","last_clean":"2024-09-10T09:54:00.847831+0000","last_became_active":"2024-09-10T09:52:19.841404+0000","last_became_peered":"2024-09-10T09:52:19.841404+0000","last_unstale":"2024-09-10T09:54:00.847831+0000","last_undegraded":"2024-09-10T09:54:00.847831+0000","last_fullsized":"2024-09-10T09:54:00.847831+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:44:36.346785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.955521+0000","last_change":"2024-09-10T09:53:01.218412+0000","last_active":"2024-09-10T09:54:00.955521+0000","last_peered":"2024-09-10T09:54:00.955521+0000","last_clean":"2024-09-10T09:54:00.955521+0000","last_became_active":"2024-09-10T09:53:01.217966+0000","last_became_peered":"2024-09-10T09:53:01.217966+0000","last_unstale":"2024-09-10T09:54:00.955521+0000","last_undegraded":"2024-09-10T09:54:00.955521+0000","last_fullsized":"2024-09-10T09:54:00.955521+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:53:25.743913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"96'8","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.703441+0000","last_change":"2024-09-10T09:52:22.316120+0000","last_active":"2024-09-10T09:54:29.703441+0000","last_peered":"2024-09-10T09:54:29.703441+0000","last_clean":"2024-09-10T09:54:29.703441+0000","last_became_active":"2024-09-10T09:52:22.315769+0000","last_became_peered":"2024-09-10T09:52:22.315769+0000","last_unstale":"2024-09-10T09:54:29.703441+0000","last_undegraded":"2024-09-10T09:54:29.703441+0000","last_fullsized":"2024-09-10T09:54:29.703441+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:16:41.866587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"94'3","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.841046+0000","last_change":"2024-09-10T09:52:19.835038+0000","last_active":"2024-09-10T09:54:00.841046+0000","last_peered":"2024-09-10T09:54:00.841046+0000","last_clean":"2024-09-10T09:54:00.841046+0000","last_became_active":"2024-09-10T09:52:19.834329+0000","last_became_peered":"2024-09-10T09:52:19.834329+0000","last_unstale":"2024-09-10T09:54:00.841046+0000","last_undegraded":"2024-09-10T09:54:00.841046+0000","last_fullsized":"2024-09-10T09:54:00.841046+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:58:24.084373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"94'7","reported_seq":56,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.841764+0000","last_change":"2024-09-10T09:53:01.219122+0000","last_active":"2024-09-10T09:54:00.841764+0000","last_peered":"2024-09-10T09:54:00.841764+0000","last_clean":"2024-09-10T09:54:00.841764+0000","last_became_active":"2024-09-10T09:53:01.218599+0000","last_became_peered":"2024-09-10T09:53:01.218599+0000","last_unstale":"2024-09-10T09:54:00.841764+0000","last_undegraded":"2024-09-10T09:54:00.841764+0000","last_fullsized":"2024-09-10T09:54:00.841764+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:31:10.092264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"96'8","reported_seq":19,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.706393+0000","last_change":"2024-09-10T09:54:00.952617+0000","last_active":"2024-09-10T09:54:29.706393+0000","last_peered":"2024-09-10T09:54:29.706393+0000","last_clean":"2024-09-10T09:54:29.706393+0000","last_became_active":"2024-09-10T09:54:00.952202+0000","last_became_peered":"2024-09-10T09:54:00.952202+0000","last_unstale":"2024-09-10T09:54:29.706393+0000","last_undegraded":"2024-09-10T09:54:29.706393+0000","last_fullsized":"2024-09-10T09:54:29.706393+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:35:40.179591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.844169+0000","last_change":"2024-09-10T09:52:31.371612+0000","last_active":"2024-09-10T09:54:00.844169+0000","last_peered":"2024-09-10T09:54:00.844169+0000","last_clean":"2024-09-10T09:54:00.844169+0000","last_became_active":"2024-09-10T09:52:31.371163+0000","last_became_peered":"2024-09-10T09:52:31.371163+0000","last_unstale":"2024-09-10T09:54:00.844169+0000","last_undegraded":"2024-09-10T09:54:00.844169+0000","last_fullsized":"2024-09-10T09:54:00.844169+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:11:43.681524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"96'7","reported_seq":181,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:55:22.067476+0000","last_change":"2024-09-10T09:52:19.841873+0000","last_active":"2024-09-10T09:55:22.067476+0000","last_peered":"2024-09-10T09:55:22.067476+0000","last_clean":"2024-09-10T09:55:22.067476+0000","last_became_active":"2024-09-10T09:52:19.841513+0000","last_became_peered":"2024-09-10T09:52:19.841513+0000","last_unstale":"2024-09-10T09:55:22.067476+0000","last_undegraded":"2024-09-10T09:55:22.067476+0000","last_fullsized":"2024-09-10T09:55:22.067476+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:53:31.854185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1156,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":131,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"96'7","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.697738+0000","last_change":"2024-09-10T09:52:22.314781+0000","last_active":"2024-09-10T09:54:29.697738+0000","last_peered":"2024-09-10T09:54:29.697738+0000","last_clean":"2024-09-10T09:54:29.697738+0000","last_became_active":"2024-09-10T09:52:22.313346+0000","last_became_peered":"2024-09-10T09:52:22.313346+0000","last_unstale":"2024-09-10T09:54:29.697738+0000","last_undegraded":"2024-09-10T09:54:29.697738+0000","last_fullsized":"2024-09-10T09:54:29.697738+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:15:16.921963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.844401+0000","last_change":"2024-09-10T09:52:31.365544+0000","last_active":"2024-09-10T09:54:00.844401+0000","last_peered":"2024-09-10T09:54:00.844401+0000","last_clean":"2024-09-10T09:54:00.844401+0000","last_became_active":"2024-09-10T09:52:31.365151+0000","last_became_peered":"2024-09-10T09:52:31.365151+0000","last_unstale":"2024-09-10T09:54:00.844401+0000","last_undegraded":"2024-09-10T09:54:00.844401+0000","last_fullsized":"2024-09-10T09:54:00.844401+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:36:55.247890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"96'9","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.683641+0000","last_change":"2024-09-10T09:52:22.314117+0000","last_active":"2024-09-10T09:54:29.683641+0000","last_peered":"2024-09-10T09:54:29.683641+0000","last_clean":"2024-09-10T09:54:29.683641+0000","last_became_active":"2024-09-10T09:52:22.313772+0000","last_became_peered":"2024-09-10T09:52:22.313772+0000","last_unstale":"2024-09-10T09:54:29.683641+0000","last_undegraded":"2024-09-10T09:54:29.683641+0000","last_fullsized":"2024-09-10T09:54:29.683641+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:19:37.373598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"96'12","reported_seq":50,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.682477+0000","last_change":"2024-09-10T09:52:22.314362+0000","last_active":"2024-09-10T09:54:29.682477+0000","last_peered":"2024-09-10T09:54:29.682477+0000","last_clean":"2024-09-10T09:54:29.682477+0000","last_became_active":"2024-09-10T09:52:22.314050+0000","last_became_peered":"2024-09-10T09:52:22.314050+0000","last_unstale":"2024-09-10T09:54:29.682477+0000","last_undegraded":"2024-09-10T09:54:29.682477+0000","last_fullsized":"2024-09-10T09:54:29.682477+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:37:07.218656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"96'8","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.699844+0000","last_change":"2024-09-10T09:52:22.322363+0000","last_active":"2024-09-10T09:54:29.699844+0000","last_peered":"2024-09-10T09:54:29.699844+0000","last_clean":"2024-09-10T09:54:29.699844+0000","last_became_active":"2024-09-10T09:52:22.322045+0000","last_became_peered":"2024-09-10T09:52:22.322045+0000","last_unstale":"2024-09-10T09:54:29.699844+0000","last_undegraded":"2024-09-10T09:54:29.699844+0000","last_fullsized":"2024-09-10T09:54:29.699844+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:10:27.918769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.700329+0000","last_change":"2024-09-10T09:52:22.331594+0000","last_active":"2024-09-10T09:54:29.700329+0000","last_peered":"2024-09-10T09:54:29.700329+0000","last_clean":"2024-09-10T09:54:29.700329+0000","last_became_active":"2024-09-10T09:52:22.330585+0000","last_became_peered":"2024-09-10T09:52:22.330585+0000","last_unstale":"2024-09-10T09:54:29.700329+0000","last_undegraded":"2024-09-10T09:54:29.700329+0000","last_fullsized":"2024-09-10T09:54:29.700329+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:45:56.412313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"96'4","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.705719+0000","last_change":"2024-09-10T09:52:22.325753+0000","last_active":"2024-09-10T09:54:29.705719+0000","last_peered":"2024-09-10T09:54:29.705719+0000","last_clean":"2024-09-10T09:54:29.705719+0000","last_became_active":"2024-09-10T09:52:22.325385+0000","last_became_peered":"2024-09-10T09:52:22.325385+0000","last_unstale":"2024-09-10T09:54:29.705719+0000","last_undegraded":"2024-09-10T09:54:29.705719+0000","last_fullsized":"2024-09-10T09:54:29.705719+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:18:28.776789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"96'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.681378+0000","last_change":"2024-09-10T09:52:22.319069+0000","last_active":"2024-09-10T09:54:29.681378+0000","last_peered":"2024-09-10T09:54:29.681378+0000","last_clean":"2024-09-10T09:54:29.681378+0000","last_became_active":"2024-09-10T09:52:22.318667+0000","last_became_peered":"2024-09-10T09:52:22.318667+0000","last_unstale":"2024-09-10T09:54:29.681378+0000","last_undegraded":"2024-09-10T09:54:29.681378+0000","last_fullsized":"2024-09-10T09:54:29.681378+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:21:32.028235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"96'15","reported_seq":53,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.681487+0000","last_change":"2024-09-10T09:52:22.312890+0000","last_active":"2024-09-10T09:54:29.681487+0000","last_peered":"2024-09-10T09:54:29.681487+0000","last_clean":"2024-09-10T09:54:29.681487+0000","last_became_active":"2024-09-10T09:52:22.312608+0000","last_became_peered":"2024-09-10T09:52:22.312608+0000","last_unstale":"2024-09-10T09:54:29.681487+0000","last_undegraded":"2024-09-10T09:54:29.681487+0000","last_fullsized":"2024-09-10T09:54:29.681487+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:07:46.046915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"96'2","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.689884+0000","last_change":"2024-09-10T09:54:01.151385+0000","last_active":"2024-09-10T09:54:29.689884+0000","last_peered":"2024-09-10T09:54:29.689884+0000","last_clean":"2024-09-10T09:54:29.689884+0000","last_became_active":"2024-09-10T09:54:01.150971+0000","last_became_peered":"2024-09-10T09:54:01.150971+0000","last_unstale":"2024-09-10T09:54:29.689884+0000","last_undegraded":"2024-09-10T09:54:29.689884+0000","last_fullsized":"2024-09-10T09:54:29.689884+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:27:10.492071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"96'6","reported_seq":17,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.704565+0000","last_change":"2024-09-10T09:54:00.848273+0000","last_active":"2024-09-10T09:54:29.704565+0000","last_peered":"2024-09-10T09:54:29.704565+0000","last_clean":"2024-09-10T09:54:29.704565+0000","last_became_active":"2024-09-10T09:54:00.847492+0000","last_became_peered":"2024-09-10T09:54:00.847492+0000","last_unstale":"2024-09-10T09:54:29.704565+0000","last_undegraded":"2024-09-10T09:54:29.704565+0000","last_fullsized":"2024-09-10T09:54:29.704565+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:02:24.749291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,3],"acting":[6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.29","version":"96'8","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.696836+0000","last_change":"2024-09-10T09:52:22.313538+0000","last_active":"2024-09-10T09:54:29.696836+0000","last_peered":"2024-09-10T09:54:29.696836+0000","last_clean":"2024-09-10T09:54:29.696836+0000","last_became_active":"2024-09-10T09:52:22.313196+0000","last_became_peered":"2024-09-10T09:52:22.313196+0000","last_unstale":"2024-09-10T09:54:29.696836+0000","last_undegraded":"2024-09-10T09:54:29.696836+0000","last_fullsized":"2024-09-10T09:54:29.696836+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:53:23.387167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"96'10","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.696632+0000","last_change":"2024-09-10T09:52:22.316518+0000","last_active":"2024-09-10T09:54:29.696632+0000","last_peered":"2024-09-10T09:54:29.696632+0000","last_clean":"2024-09-10T09:54:29.696632+0000","last_became_active":"2024-09-10T09:52:22.316193+0000","last_became_peered":"2024-09-10T09:52:22.316193+0000","last_unstale":"2024-09-10T09:54:29.696632+0000","last_undegraded":"2024-09-10T09:54:29.696632+0000","last_fullsized":"2024-09-10T09:54:29.696632+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:49:55.903928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"96'11","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.706245+0000","last_change":"2024-09-10T09:52:22.329163+0000","last_active":"2024-09-10T09:54:29.706245+0000","last_peered":"2024-09-10T09:54:29.706245+0000","last_clean":"2024-09-10T09:54:29.706245+0000","last_became_active":"2024-09-10T09:52:22.328453+0000","last_became_peered":"2024-09-10T09:52:22.328453+0000","last_unstale":"2024-09-10T09:54:29.706245+0000","last_undegraded":"2024-09-10T09:54:29.706245+0000","last_fullsized":"2024-09-10T09:54:29.706245+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:53:33.286055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"96'8","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.681766+0000","last_change":"2024-09-10T09:52:22.314302+0000","last_active":"2024-09-10T09:54:29.681766+0000","last_peered":"2024-09-10T09:54:29.681766+0000","last_clean":"2024-09-10T09:54:29.681766+0000","last_became_active":"2024-09-10T09:52:22.313992+0000","last_became_peered":"2024-09-10T09:52:22.313992+0000","last_unstale":"2024-09-10T09:54:29.681766+0000","last_undegraded":"2024-09-10T09:54:29.681766+0000","last_fullsized":"2024-09-10T09:54:29.681766+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:07:35.764015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"96'7","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.688705+0000","last_change":"2024-09-10T09:52:22.316400+0000","last_active":"2024-09-10T09:54:29.688705+0000","last_peered":"2024-09-10T09:54:29.688705+0000","last_clean":"2024-09-10T09:54:29.688705+0000","last_became_active":"2024-09-10T09:52:22.316110+0000","last_became_peered":"2024-09-10T09:52:22.316110+0000","last_unstale":"2024-09-10T09:54:29.688705+0000","last_undegraded":"2024-09-10T09:54:29.688705+0000","last_fullsized":"2024-09-10T09:54:29.688705+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:17:26.528899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"96'10","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.704074+0000","last_change":"2024-09-10T09:52:22.317914+0000","last_active":"2024-09-10T09:54:29.704074+0000","last_peered":"2024-09-10T09:54:29.704074+0000","last_clean":"2024-09-10T09:54:29.704074+0000","last_became_active":"2024-09-10T09:52:22.317575+0000","last_became_peered":"2024-09-10T09:52:22.317575+0000","last_unstale":"2024-09-10T09:54:29.704074+0000","last_undegraded":"2024-09-10T09:54:29.704074+0000","last_fullsized":"2024-09-10T09:54:29.704074+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:17:51.446233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"96'8","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.697848+0000","last_change":"2024-09-10T09:52:22.327749+0000","last_active":"2024-09-10T09:54:29.697848+0000","last_peered":"2024-09-10T09:54:29.697848+0000","last_clean":"2024-09-10T09:54:29.697848+0000","last_became_active":"2024-09-10T09:52:22.327461+0000","last_became_peered":"2024-09-10T09:52:22.327461+0000","last_unstale":"2024-09-10T09:54:29.697848+0000","last_undegraded":"2024-09-10T09:54:29.697848+0000","last_fullsized":"2024-09-10T09:54:29.697848+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:54.208064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"96'9","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.696695+0000","last_change":"2024-09-10T09:52:22.326982+0000","last_active":"2024-09-10T09:54:29.696695+0000","last_peered":"2024-09-10T09:54:29.696695+0000","last_clean":"2024-09-10T09:54:29.696695+0000","last_became_active":"2024-09-10T09:52:22.326635+0000","last_became_peered":"2024-09-10T09:52:22.326635+0000","last_unstale":"2024-09-10T09:54:29.696695+0000","last_undegraded":"2024-09-10T09:54:29.696695+0000","last_fullsized":"2024-09-10T09:54:29.696695+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:50:54.835899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.695757+0000","last_change":"2024-09-10T09:52:22.323410+0000","last_active":"2024-09-10T09:54:29.695757+0000","last_peered":"2024-09-10T09:54:29.695757+0000","last_clean":"2024-09-10T09:54:29.695757+0000","last_became_active":"2024-09-10T09:52:22.323116+0000","last_became_peered":"2024-09-10T09:52:22.323116+0000","last_unstale":"2024-09-10T09:54:29.695757+0000","last_undegraded":"2024-09-10T09:54:29.695757+0000","last_fullsized":"2024-09-10T09:54:29.695757+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:09:15.115162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.680224+0000","last_change":"2024-09-10T09:52:22.324908+0000","last_active":"2024-09-10T09:54:29.680224+0000","last_peered":"2024-09-10T09:54:29.680224+0000","last_clean":"2024-09-10T09:54:29.680224+0000","last_became_active":"2024-09-10T09:52:22.324449+0000","last_became_peered":"2024-09-10T09:52:22.324449+0000","last_unstale":"2024-09-10T09:54:29.680224+0000","last_undegraded":"2024-09-10T09:54:29.680224+0000","last_fullsized":"2024-09-10T09:54:29.680224+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:43:24.535721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"96'10","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.700039+0000","last_change":"2024-09-10T09:52:22.322157+0000","last_active":"2024-09-10T09:54:29.700039+0000","last_peered":"2024-09-10T09:54:29.700039+0000","last_clean":"2024-09-10T09:54:29.700039+0000","last_became_active":"2024-09-10T09:52:22.321856+0000","last_became_peered":"2024-09-10T09:52:22.321856+0000","last_unstale":"2024-09-10T09:54:29.700039+0000","last_undegraded":"2024-09-10T09:54:29.700039+0000","last_fullsized":"2024-09-10T09:54:29.700039+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:36:38.395370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"96'7","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.703907+0000","last_change":"2024-09-10T09:53:00.800337+0000","last_active":"2024-09-10T09:54:29.703907+0000","last_peered":"2024-09-10T09:54:29.703907+0000","last_clean":"2024-09-10T09:54:29.703907+0000","last_became_active":"2024-09-10T09:53:00.799863+0000","last_became_peered":"2024-09-10T09:53:00.799863+0000","last_unstale":"2024-09-10T09:54:29.703907+0000","last_undegraded":"2024-09-10T09:54:29.703907+0000","last_fullsized":"2024-09-10T09:54:29.703907+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:18.017672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.35","version":"96'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.693580+0000","last_change":"2024-09-10T09:52:22.310735+0000","last_active":"2024-09-10T09:54:29.693580+0000","last_peered":"2024-09-10T09:54:29.693580+0000","last_clean":"2024-09-10T09:54:29.693580+0000","last_became_active":"2024-09-10T09:52:22.310376+0000","last_became_peered":"2024-09-10T09:52:22.310376+0000","last_unstale":"2024-09-10T09:54:29.693580+0000","last_undegraded":"2024-09-10T09:54:29.693580+0000","last_fullsized":"2024-09-10T09:54:29.693580+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:38:08.637588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"96'16","reported_seq":55,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.685458+0000","last_change":"2024-09-10T09:52:22.321113+0000","last_active":"2024-09-10T09:54:29.685458+0000","last_peered":"2024-09-10T09:54:29.685458+0000","last_clean":"2024-09-10T09:54:29.685458+0000","last_became_active":"2024-09-10T09:52:22.320767+0000","last_became_peered":"2024-09-10T09:52:22.320767+0000","last_unstale":"2024-09-10T09:54:29.685458+0000","last_undegraded":"2024-09-10T09:54:29.685458+0000","last_fullsized":"2024-09-10T09:54:29.685458+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:37.549322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"96'11","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.687162+0000","last_change":"2024-09-10T09:52:22.318634+0000","last_active":"2024-09-10T09:54:29.687162+0000","last_peered":"2024-09-10T09:54:29.687162+0000","last_clean":"2024-09-10T09:54:29.687162+0000","last_became_active":"2024-09-10T09:52:22.318294+0000","last_became_peered":"2024-09-10T09:52:22.318294+0000","last_unstale":"2024-09-10T09:54:29.687162+0000","last_undegraded":"2024-09-10T09:54:29.687162+0000","last_fullsized":"2024-09-10T09:54:29.687162+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:19:31.262055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"96'9","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.698725+0000","last_change":"2024-09-10T09:52:22.324567+0000","last_active":"2024-09-10T09:54:29.698725+0000","last_peered":"2024-09-10T09:54:29.698725+0000","last_clean":"2024-09-10T09:54:29.698725+0000","last_became_active":"2024-09-10T09:52:22.321803+0000","last_became_peered":"2024-09-10T09:52:22.321803+0000","last_unstale":"2024-09-10T09:54:29.698725+0000","last_undegraded":"2024-09-10T09:54:29.698725+0000","last_fullsized":"2024-09-10T09:54:29.698725+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:08:11.099428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"96'5","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.699735+0000","last_change":"2024-09-10T09:52:22.318163+0000","last_active":"2024-09-10T09:54:29.699735+0000","last_peered":"2024-09-10T09:54:29.699735+0000","last_clean":"2024-09-10T09:54:29.699735+0000","last_became_active":"2024-09-10T09:52:22.317808+0000","last_became_peered":"2024-09-10T09:52:22.317808+0000","last_unstale":"2024-09-10T09:54:29.699735+0000","last_undegraded":"2024-09-10T09:54:29.699735+0000","last_fullsized":"2024-09-10T09:54:29.699735+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:19.214291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"96'6","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.704960+0000","last_change":"2024-09-10T09:52:22.323035+0000","last_active":"2024-09-10T09:54:29.704960+0000","last_peered":"2024-09-10T09:54:29.704960+0000","last_clean":"2024-09-10T09:54:29.704960+0000","last_became_active":"2024-09-10T09:52:22.322679+0000","last_became_peered":"2024-09-10T09:52:22.322679+0000","last_unstale":"2024-09-10T09:54:29.704960+0000","last_undegraded":"2024-09-10T09:54:29.704960+0000","last_fullsized":"2024-09-10T09:54:29.704960+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:23:13.762612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"96'4","reported_seq":40,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.700111+0000","last_change":"2024-09-10T09:52:22.315887+0000","last_active":"2024-09-10T09:54:29.700111+0000","last_peered":"2024-09-10T09:54:29.700111+0000","last_clean":"2024-09-10T09:54:29.700111+0000","last_became_active":"2024-09-10T09:52:22.315569+0000","last_became_peered":"2024-09-10T09:52:22.315569+0000","last_unstale":"2024-09-10T09:54:29.700111+0000","last_undegraded":"2024-09-10T09:54:29.700111+0000","last_fullsized":"2024-09-10T09:54:29.700111+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:46:57.094320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"96'8","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.676312+0000","last_change":"2024-09-10T09:52:22.314060+0000","last_active":"2024-09-10T09:54:29.676312+0000","last_peered":"2024-09-10T09:54:29.676312+0000","last_clean":"2024-09-10T09:54:29.676312+0000","last_became_active":"2024-09-10T09:52:22.313572+0000","last_became_peered":"2024-09-10T09:52:22.313572+0000","last_unstale":"2024-09-10T09:54:29.676312+0000","last_undegraded":"2024-09-10T09:54:29.676312+0000","last_fullsized":"2024-09-10T09:54:29.676312+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:11:17.158625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.692163+0000","last_change":"2024-09-10T09:52:22.327233+0000","last_active":"2024-09-10T09:54:29.692163+0000","last_peered":"2024-09-10T09:54:29.692163+0000","last_clean":"2024-09-10T09:54:29.692163+0000","last_became_active":"2024-09-10T09:52:22.326945+0000","last_became_peered":"2024-09-10T09:52:22.326945+0000","last_unstale":"2024-09-10T09:54:29.692163+0000","last_undegraded":"2024-09-10T09:54:29.692163+0000","last_fullsized":"2024-09-10T09:54:29.692163+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:31:13.490147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"96'4","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.681664+0000","last_change":"2024-09-10T09:52:22.330912+0000","last_active":"2024-09-10T09:54:29.681664+0000","last_peered":"2024-09-10T09:54:29.681664+0000","last_clean":"2024-09-10T09:54:29.681664+0000","last_became_active":"2024-09-10T09:52:22.330149+0000","last_became_peered":"2024-09-10T09:52:22.330149+0000","last_unstale":"2024-09-10T09:54:29.681664+0000","last_undegraded":"2024-09-10T09:54:29.681664+0000","last_fullsized":"2024-09-10T09:54:29.681664+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:10:48.842499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"96'8","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.697891+0000","last_change":"2024-09-10T09:52:22.327035+0000","last_active":"2024-09-10T09:54:29.697891+0000","last_peered":"2024-09-10T09:54:29.697891+0000","last_clean":"2024-09-10T09:54:29.697891+0000","last_became_active":"2024-09-10T09:52:22.326741+0000","last_became_peered":"2024-09-10T09:52:22.326741+0000","last_unstale":"2024-09-10T09:54:29.697891+0000","last_undegraded":"2024-09-10T09:54:29.697891+0000","last_fullsized":"2024-09-10T09:54:29.697891+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:21:05.026614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":75677,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1315,"num_read_kb":743,"num_write":263,"num_write_kb":158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":4426,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":227031,"data_compressed":5661,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":227,"ondisk_log_size":227,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282980,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449139,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1073152,"data_stored":554869,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":70,"seq":300647710774,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32376,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712776,"statfs":{"total":95995035648,"available":95961882624,"internally_reserved":0,"allocated":1220608,"data_stored":1209196,"data_compressed":70462,"data_compressed_allocated":94208,"data_compressed_original":688128,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":65,"seq":279172874302,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32256,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712896,"statfs":{"total":95995035648,"available":95962005504,"internally_reserved":0,"allocated":1097728,"data_stored":557996,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135940,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32244,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712908,"statfs":{"total":95995035648,"available":95962017792,"internally_reserved":0,"allocated":1085440,"data_stored":557702,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364872,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28136,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717016,"statfs":{"total":95995035648,"available":95966224384,"internally_reserved":0,"allocated":1138688,"data_stored":1150864,"data_compressed":68575,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528398,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1069056,"data_stored":549715,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790036,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28144,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717008,"statfs":{"total":95995035648,"available":95966216192,"internally_reserved":0,"allocated":1146880,"data_stored":1142404,"data_compressed":68575,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953559,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28148,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717004,"statfs":{"total":95995035648,"available":95966212096,"internally_reserved":0,"allocated":1150976,"data_stored":621888,"data_compressed":34470,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182493,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32240,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712912,"statfs":{"total":95995035648,"available":95962021888,"internally_reserved":0,"allocated":1081344,"data_stored":549112,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411427,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36404,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708748,"statfs":{"total":95995035648,"available":95957757952,"internally_reserved":0,"allocated":1085440,"data_stored":551188,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574953,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28144,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717008,"statfs":{"total":95995035648,"available":95966216192,"internally_reserved":0,"allocated":1146880,"data_stored":619293,"data_compressed":34470,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738478,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1093632,"data_stored":550860,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34340,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34340,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34340,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35992,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35992,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35992,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35992,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35992,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":70611,"data_compressed":1887,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":69110,"data_compressed":1887,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":70146,"data_compressed":1887,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T09:55:29.402 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-10T09:55:29.402 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T09:55:29.772 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:55:30.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:55:29 smithi120 ceph-mon[34770]: from='client.15303 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T09:55:30.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:55:29 smithi077 ceph-mon[30319]: from='client.15303 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T09:55:30.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:30.371+0000 7f1aab000640 1 -- 172.21.15.77:0/1614515355 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1aa4102360 msgr2=0x7f1aa410e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:30.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:30.371+0000 7f1aab000640 1 --2- 172.21.15.77:0/1614515355 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1aa4102360 0x7f1aa410e890 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c009cd0 tx=0x7f1a9c02a8e0 comp rx=0 tx=0).stop 2024-09-10T09:55:30.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:30.371+0000 7f1aab000640 1 -- 172.21.15.77:0/1614515355 shutdown_connections 2024-09-10T09:55:30.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:30.371+0000 7f1aab000640 1 --2- 172.21.15.77:0/1614515355 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1aa4102360 0x7f1aa410e890 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:30.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:30.371+0000 7f1aab000640 1 --2- 172.21.15.77:0/1614515355 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1aa41019c0 0x7f1aa4101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:30.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:30.371+0000 7f1aab000640 1 --2- 172.21.15.77:0/1614515355 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1aa41079c0 0x7f1aa4107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:30.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:30.371+0000 7f1aab000640 1 -- 172.21.15.77:0/1614515355 >> 172.21.15.77:0/1614515355 conn(0x7f1aa40fd740 msgr2=0x7f1aa40ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:30.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:30.371+0000 7f1aab000640 1 -- 172.21.15.77:0/1614515355 shutdown_connections 2024-09-10T09:55:30.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:30.371+0000 7f1aab000640 1 -- 172.21.15.77:0/1614515355 wait complete. 2024-09-10T09:55:30.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:30.372+0000 7f1aab000640 1 Processor -- start 2024-09-10T09:55:30.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:30.372+0000 7f1aab000640 1 -- start start 2024-09-10T09:55:30.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:30.373+0000 7f1aab000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1aa41019c0 0x7f1aa40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:30.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:30.373+0000 7f1aab000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1aa4102360 0x7f1aa4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:30.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:30.373+0000 7f1aab000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1aa41079c0 0x7f1aa4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:30.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:30.373+0000 7f1aab000640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa4111dc0 con 0x7f1aa41079c0 2024-09-10T09:55:30.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:30.373+0000 7f1aab000640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa4111c40 con 0x7f1aa41019c0 2024-09-10T09:55:30.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:30.373+0000 7f1aab000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa4111f40 con 0x7f1aa4102360 2024-09-10T09:55:30.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:30.373+0000 7f1aa9200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1aa4102360 0x7f1aa4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:30.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:30.373+0000 7f1aa9c00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1aa41019c0 0x7f1aa40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:30.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:30.373+0000 7f1aa9c00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1aa41019c0 0x7f1aa40744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:35482/0 (socket says 172.21.15.77:35482) 2024-09-10T09:55:30.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:30.373+0000 7f1aa9c00640 1 -- 172.21.15.77:0/1534089727 learned_addr learned my addr 172.21.15.77:0/1534089727 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:55:30.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:30.373+0000 7f1aaa600640 1 --2- 172.21.15.77:0/1534089727 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1aa41079c0 0x7f1aa4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:30.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:30.374+0000 7f1aaa600640 1 -- 172.21.15.77:0/1534089727 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1aa4102360 msgr2=0x7f1aa4074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:30.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:30.374+0000 7f1aaa600640 1 --2- 172.21.15.77:0/1534089727 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1aa4102360 0x7f1aa4074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:30.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:30.374+0000 7f1aaa600640 1 -- 172.21.15.77:0/1534089727 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1aa41019c0 msgr2=0x7f1aa40744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:30.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:30.374+0000 7f1aaa600640 1 --2- 172.21.15.77:0/1534089727 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1aa41019c0 0x7f1aa40744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:30.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:30.374+0000 7f1aaa600640 1 -- 172.21.15.77:0/1534089727 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1aa41a6740 con 0x7f1aa41079c0 2024-09-10T09:55:30.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:30.375+0000 7f1aaa600640 1 --2- 172.21.15.77:0/1534089727 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1aa41079c0 0x7f1aa4074f70 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c02ae00 tx=0x7f1a9c031df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:30.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:30.375+0000 7f1aa2000640 1 -- 172.21.15.77:0/1534089727 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a9c003d80 con 0x7f1aa41079c0 2024-09-10T09:55:30.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:30.375+0000 7f1aa2000640 1 -- 172.21.15.77:0/1534089727 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a9c004520 con 0x7f1aa41079c0 2024-09-10T09:55:30.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:30.375+0000 7f1aab000640 1 -- 172.21.15.77:0/1534089727 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1aa41a69d0 con 0x7f1aa41079c0 2024-09-10T09:55:30.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:30.375+0000 7f1aa2000640 1 -- 172.21.15.77:0/1534089727 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a9c030d00 con 0x7f1aa41079c0 2024-09-10T09:55:30.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:30.375+0000 7f1aab000640 1 -- 172.21.15.77:0/1534089727 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1aa41a6f60 con 0x7f1aa41079c0 2024-09-10T09:55:30.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:30.376+0000 7f1aab000640 1 -- 172.21.15.77:0/1534089727 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1aa4101e20 con 0x7f1aa41079c0 2024-09-10T09:55:30.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:30.378+0000 7f1aa2000640 1 -- 172.21.15.77:0/1534089727 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1a9c0046d0 con 0x7f1aa41079c0 2024-09-10T09:55:30.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:55:29 smithi134 ceph-mon[35674]: from='client.15303 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T09:55:30.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:30.380+0000 7f1aa2000640 1 --2- 172.21.15.77:0/1534089727 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f1a8c0a2c00 0x7f1a8c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:30.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:30.380+0000 7f1aa2000640 1 -- 172.21.15.77:0/1534089727 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f1a9c02e030 con 0x7f1aa41079c0 2024-09-10T09:55:30.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:30.380+0000 7f1aa9c00640 1 --2- 172.21.15.77:0/1534089727 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f1a8c0a2c00 0x7f1a8c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:30.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:30.381+0000 7f1aa9c00640 1 --2- 172.21.15.77:0/1534089727 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f1a8c0a2c00 0x7f1a8c0a50c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1a98009c50 tx=0x7f1a98005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:30.384 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:30.383+0000 7f1aa2000640 1 -- 172.21.15.77:0/1534089727 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1a9c04b060 con 0x7f1aa41079c0 2024-09-10T09:55:30.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:30.563+0000 7f1aab000640 1 -- 172.21.15.77:0/1534089727 --> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1aa4109fa0 con 0x7f1a8c0a2c00 2024-09-10T09:55:30.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:30.573+0000 7f1aa2000640 1 -- 172.21.15.77:0/1534089727 <== mgr.14150 v2:172.21.15.77:6800/4261324650 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340856 (secure 0 0 0) 0x7f1a98009d40 con 0x7f1a8c0a2c00 2024-09-10T09:55:30.577 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:55:30.579 INFO:teuthology.orchestra.run.smithi077.stderr:dumped all 2024-09-10T09:55:30.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:30.577+0000 7f1aab000640 1 -- 172.21.15.77:0/1534089727 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f1a8c0a2c00 msgr2=0x7f1a8c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:30.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:30.577+0000 7f1aab000640 1 --2- 172.21.15.77:0/1534089727 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f1a8c0a2c00 0x7f1a8c0a50c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1a98009c50 tx=0x7f1a98005f90 comp rx=0 tx=0).stop 2024-09-10T09:55:30.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:30.577+0000 7f1aab000640 1 -- 172.21.15.77:0/1534089727 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1aa41079c0 msgr2=0x7f1aa4074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:30.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:30.577+0000 7f1aab000640 1 --2- 172.21.15.77:0/1534089727 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1aa41079c0 0x7f1aa4074f70 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c02ae00 tx=0x7f1a9c031df0 comp rx=0 tx=0).stop 2024-09-10T09:55:30.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:30.577+0000 7f1aab000640 1 -- 172.21.15.77:0/1534089727 shutdown_connections 2024-09-10T09:55:30.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:30.577+0000 7f1aab000640 1 --2- 172.21.15.77:0/1534089727 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f1a8c0a2c00 0x7f1a8c0a50c0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:30.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:30.578+0000 7f1aab000640 1 --2- 172.21.15.77:0/1534089727 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1aa41079c0 0x7f1aa4074f70 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:30.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:30.578+0000 7f1aab000640 1 --2- 172.21.15.77:0/1534089727 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1aa4102360 0x7f1aa4074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:30.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:30.578+0000 7f1aab000640 1 --2- 172.21.15.77:0/1534089727 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1aa41019c0 0x7f1aa40744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:30.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:30.578+0000 7f1aab000640 1 -- 172.21.15.77:0/1534089727 >> 172.21.15.77:0/1534089727 conn(0x7f1aa40fd740 msgr2=0x7f1aa40fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:30.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:30.578+0000 7f1aab000640 1 -- 172.21.15.77:0/1534089727 shutdown_connections 2024-09-10T09:55:30.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:30.578+0000 7f1aab000640 1 -- 172.21.15.77:0/1534089727 wait complete. 2024-09-10T09:55:31.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:55:30 smithi077 ceph-mon[30319]: pgmap v401: 129 pgs: 129 active+clean; 652 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 853 B/s wr, 12 op/s 2024-09-10T09:55:31.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:55:30 smithi134 ceph-mon[35674]: pgmap v401: 129 pgs: 129 active+clean; 652 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 853 B/s wr, 12 op/s 2024-09-10T09:55:31.384 INFO:teuthology.orchestra.run.smithi077.stdout:{"pg_ready":true,"pg_map":{"version":401,"stamp":"2024-09-10T09:55:29.722575+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":667417,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1488,"num_read_kb":875,"num_write":447,"num_write_kb":3100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1185162,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":867,"ondisk_log_size":867,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":366484,"kb_used_data":13076,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124575340,"statfs":{"total":1151940427776,"available":1151565148160,"internally_reserved":0,"allocated":13389824,"data_stored":8618501,"data_compressed":504633,"data_compressed_allocated":688128,"data_compressed_original":2875392,"omap_allocated":19143,"internal_metadata":361674041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6074,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":151,"num_read_kb":81,"num_write":5,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003428"},"pg_stats":[{"pgid":"2.1f","version":"96'14","reported_seq":199,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:55:25.071673+0000","last_change":"2024-09-10T09:53:00.803590+0000","last_active":"2024-09-10T09:55:25.071673+0000","last_peered":"2024-09-10T09:55:25.071673+0000","last_clean":"2024-09-10T09:55:25.071673+0000","last_became_active":"2024-09-10T09:53:00.798189+0000","last_became_peered":"2024-09-10T09:53:00.798189+0000","last_unstale":"2024-09-10T09:55:25.071673+0000","last_undegraded":"2024-09-10T09:55:25.071673+0000","last_fullsized":"2024-09-10T09:55:25.071673+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:18:48.808836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":132,"num_read_kb":132,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"96'5","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.682459+0000","last_change":"2024-09-10T09:52:22.318195+0000","last_active":"2024-09-10T09:54:29.682459+0000","last_peered":"2024-09-10T09:54:29.682459+0000","last_clean":"2024-09-10T09:54:29.682459+0000","last_became_active":"2024-09-10T09:52:22.315905+0000","last_became_peered":"2024-09-10T09:52:22.315905+0000","last_unstale":"2024-09-10T09:54:29.682459+0000","last_undegraded":"2024-09-10T09:54:29.682459+0000","last_fullsized":"2024-09-10T09:54:29.682459+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:53:55.515984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.843988+0000","last_change":"2024-09-10T09:52:31.369780+0000","last_active":"2024-09-10T09:54:00.843988+0000","last_peered":"2024-09-10T09:54:00.843988+0000","last_clean":"2024-09-10T09:54:00.843988+0000","last_became_active":"2024-09-10T09:52:31.369238+0000","last_became_peered":"2024-09-10T09:52:31.369238+0000","last_unstale":"2024-09-10T09:54:00.843988+0000","last_undegraded":"2024-09-10T09:54:00.843988+0000","last_fullsized":"2024-09-10T09:54:00.843988+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:39:05.318493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"94'2","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.847203+0000","last_change":"2024-09-10T09:52:19.912276+0000","last_active":"2024-09-10T09:54:00.847203+0000","last_peered":"2024-09-10T09:54:00.847203+0000","last_clean":"2024-09-10T09:54:00.847203+0000","last_became_active":"2024-09-10T09:52:19.911452+0000","last_became_peered":"2024-09-10T09:52:19.911452+0000","last_unstale":"2024-09-10T09:54:00.847203+0000","last_undegraded":"2024-09-10T09:54:00.847203+0000","last_fullsized":"2024-09-10T09:54:00.847203+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:24:31.262207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"96'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.705105+0000","last_change":"2024-09-10T09:52:22.327641+0000","last_active":"2024-09-10T09:54:29.705105+0000","last_peered":"2024-09-10T09:54:29.705105+0000","last_clean":"2024-09-10T09:54:29.705105+0000","last_became_active":"2024-09-10T09:52:22.326969+0000","last_became_peered":"2024-09-10T09:52:22.326969+0000","last_unstale":"2024-09-10T09:54:29.705105+0000","last_undegraded":"2024-09-10T09:54:29.705105+0000","last_fullsized":"2024-09-10T09:54:29.705105+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:35:27.825534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.951014+0000","last_change":"2024-09-10T09:52:31.372360+0000","last_active":"2024-09-10T09:54:00.951014+0000","last_peered":"2024-09-10T09:54:00.951014+0000","last_clean":"2024-09-10T09:54:00.951014+0000","last_became_active":"2024-09-10T09:52:31.371957+0000","last_became_peered":"2024-09-10T09:52:31.371957+0000","last_unstale":"2024-09-10T09:54:00.951014+0000","last_undegraded":"2024-09-10T09:54:00.951014+0000","last_fullsized":"2024-09-10T09:54:00.951014+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:49:34.776185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"96'13","reported_seq":55,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:51.750986+0000","last_change":"2024-09-10T09:52:19.843532+0000","last_active":"2024-09-10T09:54:51.750986+0000","last_peered":"2024-09-10T09:54:51.750986+0000","last_clean":"2024-09-10T09:54:51.750986+0000","last_became_active":"2024-09-10T09:52:19.843070+0000","last_became_peered":"2024-09-10T09:52:19.843070+0000","last_unstale":"2024-09-10T09:54:51.750986+0000","last_undegraded":"2024-09-10T09:54:51.750986+0000","last_fullsized":"2024-09-10T09:54:51.750986+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:08:02.223373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"96'9","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.692662+0000","last_change":"2024-09-10T09:52:22.316981+0000","last_active":"2024-09-10T09:54:29.692662+0000","last_peered":"2024-09-10T09:54:29.692662+0000","last_clean":"2024-09-10T09:54:29.692662+0000","last_became_active":"2024-09-10T09:52:22.316648+0000","last_became_peered":"2024-09-10T09:52:22.316648+0000","last_unstale":"2024-09-10T09:54:29.692662+0000","last_undegraded":"2024-09-10T09:54:29.692662+0000","last_fullsized":"2024-09-10T09:54:29.692662+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:28:56.742496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.951485+0000","last_change":"2024-09-10T09:52:31.376811+0000","last_active":"2024-09-10T09:54:00.951485+0000","last_peered":"2024-09-10T09:54:00.951485+0000","last_clean":"2024-09-10T09:54:00.951485+0000","last_became_active":"2024-09-10T09:52:31.376007+0000","last_became_peered":"2024-09-10T09:52:31.376007+0000","last_unstale":"2024-09-10T09:54:00.951485+0000","last_undegraded":"2024-09-10T09:54:00.951485+0000","last_fullsized":"2024-09-10T09:54:00.951485+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:29:26.213281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"94'5","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.149772+0000","last_change":"2024-09-10T09:52:19.837848+0000","last_active":"2024-09-10T09:54:01.149772+0000","last_peered":"2024-09-10T09:54:01.149772+0000","last_clean":"2024-09-10T09:54:01.149772+0000","last_became_active":"2024-09-10T09:52:19.837383+0000","last_became_peered":"2024-09-10T09:52:19.837383+0000","last_unstale":"2024-09-10T09:54:01.149772+0000","last_undegraded":"2024-09-10T09:54:01.149772+0000","last_fullsized":"2024-09-10T09:54:01.149772+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:50:42.813665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"96'15","reported_seq":60,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.685844+0000","last_change":"2024-09-10T09:54:01.060407+0000","last_active":"2024-09-10T09:54:29.685844+0000","last_peered":"2024-09-10T09:54:29.685844+0000","last_clean":"2024-09-10T09:54:29.685844+0000","last_became_active":"2024-09-10T09:54:01.060126+0000","last_became_peered":"2024-09-10T09:54:01.060126+0000","last_unstale":"2024-09-10T09:54:29.685844+0000","last_undegraded":"2024-09-10T09:54:29.685844+0000","last_fullsized":"2024-09-10T09:54:29.685844+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:59:29.765375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.951119+0000","last_change":"2024-09-10T09:52:31.370910+0000","last_active":"2024-09-10T09:54:00.951119+0000","last_peered":"2024-09-10T09:54:00.951119+0000","last_clean":"2024-09-10T09:54:00.951119+0000","last_became_active":"2024-09-10T09:52:31.370467+0000","last_became_peered":"2024-09-10T09:52:31.370467+0000","last_unstale":"2024-09-10T09:54:00.951119+0000","last_undegraded":"2024-09-10T09:54:00.951119+0000","last_fullsized":"2024-09-10T09:54:00.951119+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:49:25.037820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"96'5","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.702762+0000","last_change":"2024-09-10T09:52:22.310708+0000","last_active":"2024-09-10T09:54:29.702762+0000","last_peered":"2024-09-10T09:54:29.702762+0000","last_clean":"2024-09-10T09:54:29.702762+0000","last_became_active":"2024-09-10T09:52:22.310387+0000","last_became_peered":"2024-09-10T09:52:22.310387+0000","last_unstale":"2024-09-10T09:54:29.702762+0000","last_undegraded":"2024-09-10T09:54:29.702762+0000","last_fullsized":"2024-09-10T09:54:29.702762+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:35:03.115408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"94'4","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.149453+0000","last_change":"2024-09-10T09:52:19.841417+0000","last_active":"2024-09-10T09:54:01.149453+0000","last_peered":"2024-09-10T09:54:01.149453+0000","last_clean":"2024-09-10T09:54:01.149453+0000","last_became_active":"2024-09-10T09:52:19.841083+0000","last_became_peered":"2024-09-10T09:52:19.841083+0000","last_unstale":"2024-09-10T09:54:01.149453+0000","last_undegraded":"2024-09-10T09:54:01.149453+0000","last_fullsized":"2024-09-10T09:54:01.149453+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:01:19.008345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.951294+0000","last_change":"2024-09-10T09:52:31.367406+0000","last_active":"2024-09-10T09:54:00.951294+0000","last_peered":"2024-09-10T09:54:00.951294+0000","last_clean":"2024-09-10T09:54:00.951294+0000","last_became_active":"2024-09-10T09:52:31.366961+0000","last_became_peered":"2024-09-10T09:52:31.366961+0000","last_unstale":"2024-09-10T09:54:00.951294+0000","last_undegraded":"2024-09-10T09:54:00.951294+0000","last_fullsized":"2024-09-10T09:54:00.951294+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:04:51.714465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"96'8","reported_seq":19,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.705252+0000","last_change":"2024-09-10T09:54:01.249212+0000","last_active":"2024-09-10T09:54:29.705252+0000","last_peered":"2024-09-10T09:54:29.705252+0000","last_clean":"2024-09-10T09:54:29.705252+0000","last_became_active":"2024-09-10T09:54:01.248840+0000","last_became_peered":"2024-09-10T09:54:01.248840+0000","last_unstale":"2024-09-10T09:54:29.705252+0000","last_undegraded":"2024-09-10T09:54:29.705252+0000","last_fullsized":"2024-09-10T09:54:29.705252+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:39:13.167435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,10],"acting":[9,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.a","version":"94'3","reported_seq":19,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.954964+0000","last_change":"2024-09-10T09:53:00.800421+0000","last_active":"2024-09-10T09:54:00.954964+0000","last_peered":"2024-09-10T09:54:00.954964+0000","last_clean":"2024-09-10T09:54:00.954964+0000","last_became_active":"2024-09-10T09:53:00.800026+0000","last_became_peered":"2024-09-10T09:53:00.800026+0000","last_unstale":"2024-09-10T09:54:00.954964+0000","last_undegraded":"2024-09-10T09:54:00.954964+0000","last_fullsized":"2024-09-10T09:54:00.954964+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:57:45.252420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"94'1","reported_seq":30,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.951388+0000","last_change":"2024-09-10T09:52:31.376605+0000","last_active":"2024-09-10T09:54:00.951388+0000","last_peered":"2024-09-10T09:54:00.951388+0000","last_clean":"2024-09-10T09:54:00.951388+0000","last_became_active":"2024-09-10T09:52:31.376083+0000","last_became_peered":"2024-09-10T09:52:31.376083+0000","last_unstale":"2024-09-10T09:54:00.951388+0000","last_undegraded":"2024-09-10T09:54:00.951388+0000","last_fullsized":"2024-09-10T09:54:00.951388+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:05:10.030673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"96'9","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.697029+0000","last_change":"2024-09-10T09:52:22.327303+0000","last_active":"2024-09-10T09:54:29.697029+0000","last_peered":"2024-09-10T09:54:29.697029+0000","last_clean":"2024-09-10T09:54:29.697029+0000","last_became_active":"2024-09-10T09:52:22.326753+0000","last_became_peered":"2024-09-10T09:52:22.326753+0000","last_unstale":"2024-09-10T09:54:29.697029+0000","last_undegraded":"2024-09-10T09:54:29.697029+0000","last_fullsized":"2024-09-10T09:54:29.697029+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:27:26.668151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"94'7","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.845956+0000","last_change":"2024-09-10T09:52:19.836484+0000","last_active":"2024-09-10T09:54:00.845956+0000","last_peered":"2024-09-10T09:54:00.845956+0000","last_clean":"2024-09-10T09:54:00.845956+0000","last_became_active":"2024-09-10T09:52:19.836130+0000","last_became_peered":"2024-09-10T09:52:19.836130+0000","last_unstale":"2024-09-10T09:54:00.845956+0000","last_undegraded":"2024-09-10T09:54:00.845956+0000","last_fullsized":"2024-09-10T09:54:00.845956+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:31:58.731981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.844293+0000","last_change":"2024-09-10T09:52:31.375279+0000","last_active":"2024-09-10T09:54:00.844293+0000","last_peered":"2024-09-10T09:54:00.844293+0000","last_clean":"2024-09-10T09:54:00.844293+0000","last_became_active":"2024-09-10T09:52:31.374181+0000","last_became_peered":"2024-09-10T09:52:31.374181+0000","last_unstale":"2024-09-10T09:54:00.844293+0000","last_undegraded":"2024-09-10T09:54:00.844293+0000","last_fullsized":"2024-09-10T09:54:00.844293+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:54:51.299224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"96'2","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.676489+0000","last_change":"2024-09-10T09:52:22.314817+0000","last_active":"2024-09-10T09:54:29.676489+0000","last_peered":"2024-09-10T09:54:29.676489+0000","last_clean":"2024-09-10T09:54:29.676489+0000","last_became_active":"2024-09-10T09:52:22.314491+0000","last_became_peered":"2024-09-10T09:52:22.314491+0000","last_unstale":"2024-09-10T09:54:29.676489+0000","last_undegraded":"2024-09-10T09:54:29.676489+0000","last_fullsized":"2024-09-10T09:54:29.676489+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:49:41.941667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"94'2","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.988558+0000","last_change":"2024-09-10T09:52:19.911730+0000","last_active":"2024-09-10T09:54:00.988558+0000","last_peered":"2024-09-10T09:54:00.988558+0000","last_clean":"2024-09-10T09:54:00.988558+0000","last_became_active":"2024-09-10T09:52:19.911393+0000","last_became_peered":"2024-09-10T09:52:19.911393+0000","last_unstale":"2024-09-10T09:54:00.988558+0000","last_undegraded":"2024-09-10T09:54:00.988558+0000","last_fullsized":"2024-09-10T09:54:00.988558+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:46:17.943129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.951301+0000","last_change":"2024-09-10T09:52:31.376498+0000","last_active":"2024-09-10T09:54:00.951301+0000","last_peered":"2024-09-10T09:54:00.951301+0000","last_clean":"2024-09-10T09:54:00.951301+0000","last_became_active":"2024-09-10T09:52:31.375966+0000","last_became_peered":"2024-09-10T09:52:31.375966+0000","last_unstale":"2024-09-10T09:54:00.951301+0000","last_undegraded":"2024-09-10T09:54:00.951301+0000","last_fullsized":"2024-09-10T09:54:00.951301+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:08:30.649399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.705573+0000","last_change":"2024-09-10T09:52:22.331611+0000","last_active":"2024-09-10T09:54:29.705573+0000","last_peered":"2024-09-10T09:54:29.705573+0000","last_clean":"2024-09-10T09:54:29.705573+0000","last_became_active":"2024-09-10T09:52:22.331268+0000","last_became_peered":"2024-09-10T09:52:22.331268+0000","last_unstale":"2024-09-10T09:54:29.705573+0000","last_undegraded":"2024-09-10T09:54:29.705573+0000","last_fullsized":"2024-09-10T09:54:29.705573+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:01:05.433994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"94'7","reported_seq":50,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.951927+0000","last_change":"2024-09-10T09:52:19.838387+0000","last_active":"2024-09-10T09:54:00.951927+0000","last_peered":"2024-09-10T09:54:00.951927+0000","last_clean":"2024-09-10T09:54:00.951927+0000","last_became_active":"2024-09-10T09:52:19.837889+0000","last_became_peered":"2024-09-10T09:52:19.837889+0000","last_unstale":"2024-09-10T09:54:00.951927+0000","last_undegraded":"2024-09-10T09:54:00.951927+0000","last_fullsized":"2024-09-10T09:54:00.951927+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:50:00.187243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.844187+0000","last_change":"2024-09-10T09:52:31.363262+0000","last_active":"2024-09-10T09:54:00.844187+0000","last_peered":"2024-09-10T09:54:00.844187+0000","last_clean":"2024-09-10T09:54:00.844187+0000","last_became_active":"2024-09-10T09:52:31.362762+0000","last_became_peered":"2024-09-10T09:52:31.362762+0000","last_unstale":"2024-09-10T09:54:00.844187+0000","last_undegraded":"2024-09-10T09:54:00.844187+0000","last_fullsized":"2024-09-10T09:54:00.844187+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:48:28.275041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"96'5","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.702169+0000","last_change":"2024-09-10T09:52:22.312472+0000","last_active":"2024-09-10T09:54:29.702169+0000","last_peered":"2024-09-10T09:54:29.702169+0000","last_clean":"2024-09-10T09:54:29.702169+0000","last_became_active":"2024-09-10T09:52:22.312117+0000","last_became_peered":"2024-09-10T09:52:22.312117+0000","last_unstale":"2024-09-10T09:54:29.702169+0000","last_undegraded":"2024-09-10T09:54:29.702169+0000","last_fullsized":"2024-09-10T09:54:29.702169+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:39:10.742137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"96'16","reported_seq":57,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:31.212993+0000","last_change":"2024-09-10T09:52:19.835543+0000","last_active":"2024-09-10T09:54:31.212993+0000","last_peered":"2024-09-10T09:54:31.212993+0000","last_clean":"2024-09-10T09:54:31.212993+0000","last_became_active":"2024-09-10T09:52:19.835170+0000","last_became_peered":"2024-09-10T09:52:19.835170+0000","last_unstale":"2024-09-10T09:54:31.212993+0000","last_undegraded":"2024-09-10T09:54:31.212993+0000","last_fullsized":"2024-09-10T09:54:31.212993+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:37.969277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.845447+0000","last_change":"2024-09-10T09:52:31.372398+0000","last_active":"2024-09-10T09:54:00.845447+0000","last_peered":"2024-09-10T09:54:00.845447+0000","last_clean":"2024-09-10T09:54:00.845447+0000","last_became_active":"2024-09-10T09:52:31.371874+0000","last_became_peered":"2024-09-10T09:52:31.371874+0000","last_unstale":"2024-09-10T09:54:00.845447+0000","last_undegraded":"2024-09-10T09:54:00.845447+0000","last_fullsized":"2024-09-10T09:54:00.845447+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:09:45.374320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"96'9","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.700381+0000","last_change":"2024-09-10T09:52:22.319592+0000","last_active":"2024-09-10T09:54:29.700381+0000","last_peered":"2024-09-10T09:54:29.700381+0000","last_clean":"2024-09-10T09:54:29.700381+0000","last_became_active":"2024-09-10T09:52:22.319298+0000","last_became_peered":"2024-09-10T09:52:22.319298+0000","last_unstale":"2024-09-10T09:54:29.700381+0000","last_undegraded":"2024-09-10T09:54:29.700381+0000","last_fullsized":"2024-09-10T09:54:29.700381+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:56:59.787638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"94'4","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.150011+0000","last_change":"2024-09-10T09:52:19.912923+0000","last_active":"2024-09-10T09:54:01.150011+0000","last_peered":"2024-09-10T09:54:01.150011+0000","last_clean":"2024-09-10T09:54:01.150011+0000","last_became_active":"2024-09-10T09:52:19.912268+0000","last_became_peered":"2024-09-10T09:52:19.912268+0000","last_unstale":"2024-09-10T09:54:01.150011+0000","last_undegraded":"2024-09-10T09:54:01.150011+0000","last_fullsized":"2024-09-10T09:54:01.150011+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:06:45.421004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.955091+0000","last_change":"2024-09-10T09:52:31.372761+0000","last_active":"2024-09-10T09:54:00.955091+0000","last_peered":"2024-09-10T09:54:00.955091+0000","last_clean":"2024-09-10T09:54:00.955091+0000","last_became_active":"2024-09-10T09:52:31.372293+0000","last_became_peered":"2024-09-10T09:52:31.372293+0000","last_unstale":"2024-09-10T09:54:00.955091+0000","last_undegraded":"2024-09-10T09:54:00.955091+0000","last_fullsized":"2024-09-10T09:54:00.955091+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:36:41.446810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"96'4","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.688817+0000","last_change":"2024-09-10T09:52:22.326291+0000","last_active":"2024-09-10T09:54:29.688817+0000","last_peered":"2024-09-10T09:54:29.688817+0000","last_clean":"2024-09-10T09:54:29.688817+0000","last_became_active":"2024-09-10T09:52:22.325985+0000","last_became_peered":"2024-09-10T09:52:22.325985+0000","last_unstale":"2024-09-10T09:54:29.688817+0000","last_undegraded":"2024-09-10T09:54:29.688817+0000","last_fullsized":"2024-09-10T09:54:29.688817+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:06:21.871502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"94'1","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.844068+0000","last_change":"2024-09-10T09:52:19.912851+0000","last_active":"2024-09-10T09:54:00.844068+0000","last_peered":"2024-09-10T09:54:00.844068+0000","last_clean":"2024-09-10T09:54:00.844068+0000","last_became_active":"2024-09-10T09:52:19.912200+0000","last_became_peered":"2024-09-10T09:52:19.912200+0000","last_unstale":"2024-09-10T09:54:00.844068+0000","last_undegraded":"2024-09-10T09:54:00.844068+0000","last_fullsized":"2024-09-10T09:54:00.844068+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:00:47.331755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.149311+0000","last_change":"2024-09-10T09:52:31.371329+0000","last_active":"2024-09-10T09:54:01.149311+0000","last_peered":"2024-09-10T09:54:01.149311+0000","last_clean":"2024-09-10T09:54:01.149311+0000","last_became_active":"2024-09-10T09:52:31.370971+0000","last_became_peered":"2024-09-10T09:52:31.370971+0000","last_unstale":"2024-09-10T09:54:01.149311+0000","last_undegraded":"2024-09-10T09:54:01.149311+0000","last_fullsized":"2024-09-10T09:54:01.149311+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:07:12.928421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"96'7","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.686209+0000","last_change":"2024-09-10T09:52:22.328039+0000","last_active":"2024-09-10T09:54:29.686209+0000","last_peered":"2024-09-10T09:54:29.686209+0000","last_clean":"2024-09-10T09:54:29.686209+0000","last_became_active":"2024-09-10T09:52:22.327588+0000","last_became_peered":"2024-09-10T09:52:22.327588+0000","last_unstale":"2024-09-10T09:54:29.686209+0000","last_undegraded":"2024-09-10T09:54:29.686209+0000","last_fullsized":"2024-09-10T09:54:29.686209+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:35:39.219544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"96'6","reported_seq":155,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:55:26.073494+0000","last_change":"2024-09-10T09:53:00.797651+0000","last_active":"2024-09-10T09:55:26.073494+0000","last_peered":"2024-09-10T09:55:26.073494+0000","last_clean":"2024-09-10T09:55:26.073494+0000","last_became_active":"2024-09-10T09:53:00.797285+0000","last_became_peered":"2024-09-10T09:53:00.797285+0000","last_unstale":"2024-09-10T09:55:26.073494+0000","last_undegraded":"2024-09-10T09:55:26.073494+0000","last_fullsized":"2024-09-10T09:55:26.073494+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:18:02.320032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":899,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":133,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,6,1],"acting":[11,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.841787+0000","last_change":"2024-09-10T09:52:31.379215+0000","last_active":"2024-09-10T09:54:00.841787+0000","last_peered":"2024-09-10T09:54:00.841787+0000","last_clean":"2024-09-10T09:54:00.841787+0000","last_became_active":"2024-09-10T09:52:31.377107+0000","last_became_peered":"2024-09-10T09:52:31.377107+0000","last_unstale":"2024-09-10T09:54:00.841787+0000","last_undegraded":"2024-09-10T09:54:00.841787+0000","last_fullsized":"2024-09-10T09:54:00.841787+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:09:39.474077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"96'8","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.673518+0000","last_change":"2024-09-10T09:52:22.316474+0000","last_active":"2024-09-10T09:54:29.673518+0000","last_peered":"2024-09-10T09:54:29.673518+0000","last_clean":"2024-09-10T09:54:29.673518+0000","last_became_active":"2024-09-10T09:52:22.316054+0000","last_became_peered":"2024-09-10T09:52:22.316054+0000","last_unstale":"2024-09-10T09:54:29.673518+0000","last_undegraded":"2024-09-10T09:54:29.673518+0000","last_fullsized":"2024-09-10T09:54:29.673518+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:11:31.334780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"96'6","reported_seq":181,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:55:27.107862+0000","last_change":"2024-09-10T09:52:19.911601+0000","last_active":"2024-09-10T09:55:27.107862+0000","last_peered":"2024-09-10T09:55:27.107862+0000","last_clean":"2024-09-10T09:55:27.107862+0000","last_became_active":"2024-09-10T09:52:19.911268+0000","last_became_peered":"2024-09-10T09:52:19.911268+0000","last_unstale":"2024-09-10T09:55:27.107862+0000","last_undegraded":"2024-09-10T09:55:27.107862+0000","last_fullsized":"2024-09-10T09:55:27.107862+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:46:38.158784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":133,"num_read_kb":133,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.841796+0000","last_change":"2024-09-10T09:52:31.375938+0000","last_active":"2024-09-10T09:54:00.841796+0000","last_peered":"2024-09-10T09:54:00.841796+0000","last_clean":"2024-09-10T09:54:00.841796+0000","last_became_active":"2024-09-10T09:52:31.375536+0000","last_became_peered":"2024-09-10T09:52:31.375536+0000","last_unstale":"2024-09-10T09:54:00.841796+0000","last_undegraded":"2024-09-10T09:54:00.841796+0000","last_fullsized":"2024-09-10T09:54:00.841796+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:55:17.303809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"96'5","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.688934+0000","last_change":"2024-09-10T09:52:22.315912+0000","last_active":"2024-09-10T09:54:29.688934+0000","last_peered":"2024-09-10T09:54:29.688934+0000","last_clean":"2024-09-10T09:54:29.688934+0000","last_became_active":"2024-09-10T09:52:22.315433+0000","last_became_peered":"2024-09-10T09:52:22.315433+0000","last_unstale":"2024-09-10T09:54:29.688934+0000","last_undegraded":"2024-09-10T09:54:29.688934+0000","last_fullsized":"2024-09-10T09:54:29.688934+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:57:10.454716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"94'4","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.059138+0000","last_change":"2024-09-10T09:52:19.838966+0000","last_active":"2024-09-10T09:54:01.059138+0000","last_peered":"2024-09-10T09:54:01.059138+0000","last_clean":"2024-09-10T09:54:01.059138+0000","last_became_active":"2024-09-10T09:52:19.838545+0000","last_became_peered":"2024-09-10T09:52:19.838545+0000","last_unstale":"2024-09-10T09:54:01.059138+0000","last_undegraded":"2024-09-10T09:54:01.059138+0000","last_fullsized":"2024-09-10T09:54:01.059138+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:07:00.930146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.847601+0000","last_change":"2024-09-10T09:53:00.802039+0000","last_active":"2024-09-10T09:54:00.847601+0000","last_peered":"2024-09-10T09:54:00.847601+0000","last_clean":"2024-09-10T09:54:00.847601+0000","last_became_active":"2024-09-10T09:53:00.800223+0000","last_became_peered":"2024-09-10T09:53:00.800223+0000","last_unstale":"2024-09-10T09:54:00.847601+0000","last_undegraded":"2024-09-10T09:54:00.847601+0000","last_fullsized":"2024-09-10T09:54:00.847601+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:50:16.026631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,6,10],"acting":[5,8,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"96'8","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.698863+0000","last_change":"2024-09-10T09:52:22.313819+0000","last_active":"2024-09-10T09:54:29.698863+0000","last_peered":"2024-09-10T09:54:29.698863+0000","last_clean":"2024-09-10T09:54:29.698863+0000","last_became_active":"2024-09-10T09:52:22.313473+0000","last_became_peered":"2024-09-10T09:52:22.313473+0000","last_unstale":"2024-09-10T09:54:29.698863+0000","last_undegraded":"2024-09-10T09:54:29.698863+0000","last_fullsized":"2024-09-10T09:54:29.698863+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:44:10.003187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"94'6","reported_seq":51,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.951818+0000","last_change":"2024-09-10T09:52:19.837376+0000","last_active":"2024-09-10T09:54:00.951818+0000","last_peered":"2024-09-10T09:54:00.951818+0000","last_clean":"2024-09-10T09:54:00.951818+0000","last_became_active":"2024-09-10T09:52:19.836880+0000","last_became_peered":"2024-09-10T09:52:19.836880+0000","last_unstale":"2024-09-10T09:54:00.951818+0000","last_undegraded":"2024-09-10T09:54:00.951818+0000","last_fullsized":"2024-09-10T09:54:00.951818+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:48:58.198188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":361,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.059392+0000","last_change":"2024-09-10T09:51:05.871529+0000","last_active":"2024-09-10T09:54:01.059392+0000","last_peered":"2024-09-10T09:54:01.059392+0000","last_clean":"2024-09-10T09:54:01.059392+0000","last_became_active":"2024-09-10T09:51:05.871115+0000","last_became_peered":"2024-09-10T09:51:05.871115+0000","last_unstale":"2024-09-10T09:54:01.059392+0000","last_undegraded":"2024-09-10T09:54:01.059392+0000","last_fullsized":"2024-09-10T09:54:01.059392+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:47:24.175358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:47:24.175358+0000","last_clean_scrub_stamp":"2024-09-10T09:47:24.175358+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:58:30.247077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"94'1","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.059534+0000","last_change":"2024-09-10T09:53:00.800474+0000","last_active":"2024-09-10T09:54:01.059534+0000","last_peered":"2024-09-10T09:54:01.059534+0000","last_clean":"2024-09-10T09:54:01.059534+0000","last_became_active":"2024-09-10T09:53:00.799932+0000","last_became_peered":"2024-09-10T09:53:00.799932+0000","last_unstale":"2024-09-10T09:54:01.059534+0000","last_undegraded":"2024-09-10T09:54:01.059534+0000","last_fullsized":"2024-09-10T09:54:01.059534+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:14:50.998952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,11,9],"acting":[7,8,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.693609+0000","last_change":"2024-09-10T09:52:22.312521+0000","last_active":"2024-09-10T09:54:29.693609+0000","last_peered":"2024-09-10T09:54:29.693609+0000","last_clean":"2024-09-10T09:54:29.693609+0000","last_became_active":"2024-09-10T09:52:22.312107+0000","last_became_peered":"2024-09-10T09:52:22.312107+0000","last_unstale":"2024-09-10T09:54:29.693609+0000","last_undegraded":"2024-09-10T09:54:29.693609+0000","last_fullsized":"2024-09-10T09:54:29.693609+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:14:19.799259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"94'5","reported_seq":181,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:55:27.075089+0000","last_change":"2024-09-10T09:52:19.911454+0000","last_active":"2024-09-10T09:55:27.075089+0000","last_peered":"2024-09-10T09:55:27.075089+0000","last_clean":"2024-09-10T09:55:27.075089+0000","last_became_active":"2024-09-10T09:52:19.911079+0000","last_became_peered":"2024-09-10T09:52:19.911079+0000","last_unstale":"2024-09-10T09:55:27.075089+0000","last_undegraded":"2024-09-10T09:55:27.075089+0000","last_fullsized":"2024-09-10T09:55:27.075089+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:43:38.787689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":134,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.988531+0000","last_change":"2024-09-10T09:52:31.381877+0000","last_active":"2024-09-10T09:54:00.988531+0000","last_peered":"2024-09-10T09:54:00.988531+0000","last_clean":"2024-09-10T09:54:00.988531+0000","last_became_active":"2024-09-10T09:52:31.381472+0000","last_became_peered":"2024-09-10T09:52:31.381472+0000","last_unstale":"2024-09-10T09:54:00.988531+0000","last_undegraded":"2024-09-10T09:54:00.988531+0000","last_fullsized":"2024-09-10T09:54:00.988531+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:41:24.686325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"96'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.702544+0000","last_change":"2024-09-10T09:52:22.313902+0000","last_active":"2024-09-10T09:54:29.702544+0000","last_peered":"2024-09-10T09:54:29.702544+0000","last_clean":"2024-09-10T09:54:29.702544+0000","last_became_active":"2024-09-10T09:52:22.313515+0000","last_became_peered":"2024-09-10T09:52:22.313515+0000","last_unstale":"2024-09-10T09:54:29.702544+0000","last_undegraded":"2024-09-10T09:54:29.702544+0000","last_fullsized":"2024-09-10T09:54:29.702544+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:25:00.238307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"94'3","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.847873+0000","last_change":"2024-09-10T09:52:19.837736+0000","last_active":"2024-09-10T09:54:00.847873+0000","last_peered":"2024-09-10T09:54:00.847873+0000","last_clean":"2024-09-10T09:54:00.847873+0000","last_became_active":"2024-09-10T09:52:19.837370+0000","last_became_peered":"2024-09-10T09:52:19.837370+0000","last_unstale":"2024-09-10T09:54:00.847873+0000","last_undegraded":"2024-09-10T09:54:00.847873+0000","last_fullsized":"2024-09-10T09:54:00.847873+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:54:35.753027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.248213+0000","last_change":"2024-09-10T09:52:31.378945+0000","last_active":"2024-09-10T09:54:01.248213+0000","last_peered":"2024-09-10T09:54:01.248213+0000","last_clean":"2024-09-10T09:54:01.248213+0000","last_became_active":"2024-09-10T09:52:31.378144+0000","last_became_peered":"2024-09-10T09:52:31.378144+0000","last_unstale":"2024-09-10T09:54:01.248213+0000","last_undegraded":"2024-09-10T09:54:01.248213+0000","last_fullsized":"2024-09-10T09:54:01.248213+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:36:06.288293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"96'9","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.692039+0000","last_change":"2024-09-10T09:52:22.316400+0000","last_active":"2024-09-10T09:54:29.692039+0000","last_peered":"2024-09-10T09:54:29.692039+0000","last_clean":"2024-09-10T09:54:29.692039+0000","last_became_active":"2024-09-10T09:52:22.315974+0000","last_became_peered":"2024-09-10T09:52:22.315974+0000","last_unstale":"2024-09-10T09:54:29.692039+0000","last_undegraded":"2024-09-10T09:54:29.692039+0000","last_fullsized":"2024-09-10T09:54:29.692039+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:31:05.538237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"94'5","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.841976+0000","last_change":"2024-09-10T09:52:19.837942+0000","last_active":"2024-09-10T09:54:00.841976+0000","last_peered":"2024-09-10T09:54:00.841976+0000","last_clean":"2024-09-10T09:54:00.841976+0000","last_became_active":"2024-09-10T09:52:19.837629+0000","last_became_peered":"2024-09-10T09:52:19.837629+0000","last_unstale":"2024-09-10T09:54:00.841976+0000","last_undegraded":"2024-09-10T09:54:00.841976+0000","last_fullsized":"2024-09-10T09:54:00.841976+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:34:13.207770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.847019+0000","last_change":"2024-09-10T09:52:31.364231+0000","last_active":"2024-09-10T09:54:00.847019+0000","last_peered":"2024-09-10T09:54:00.847019+0000","last_clean":"2024-09-10T09:54:00.847019+0000","last_became_active":"2024-09-10T09:52:31.363833+0000","last_became_peered":"2024-09-10T09:52:31.363833+0000","last_unstale":"2024-09-10T09:54:00.847019+0000","last_undegraded":"2024-09-10T09:54:00.847019+0000","last_fullsized":"2024-09-10T09:54:00.847019+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:53:51.435700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"96'5","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.680562+0000","last_change":"2024-09-10T09:52:22.319530+0000","last_active":"2024-09-10T09:54:29.680562+0000","last_peered":"2024-09-10T09:54:29.680562+0000","last_clean":"2024-09-10T09:54:29.680562+0000","last_became_active":"2024-09-10T09:52:22.318742+0000","last_became_peered":"2024-09-10T09:52:22.318742+0000","last_unstale":"2024-09-10T09:54:29.680562+0000","last_undegraded":"2024-09-10T09:54:29.680562+0000","last_fullsized":"2024-09-10T09:54:29.680562+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:44:35.050454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"94'2","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.150048+0000","last_change":"2024-09-10T09:52:19.836398+0000","last_active":"2024-09-10T09:54:01.150048+0000","last_peered":"2024-09-10T09:54:01.150048+0000","last_clean":"2024-09-10T09:54:01.150048+0000","last_became_active":"2024-09-10T09:52:19.836055+0000","last_became_peered":"2024-09-10T09:52:19.836055+0000","last_unstale":"2024-09-10T09:54:01.150048+0000","last_undegraded":"2024-09-10T09:54:01.150048+0000","last_fullsized":"2024-09-10T09:54:01.150048+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:49:26.923367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.847215+0000","last_change":"2024-09-10T09:52:31.371109+0000","last_active":"2024-09-10T09:54:00.847215+0000","last_peered":"2024-09-10T09:54:00.847215+0000","last_clean":"2024-09-10T09:54:00.847215+0000","last_became_active":"2024-09-10T09:52:31.370592+0000","last_became_peered":"2024-09-10T09:52:31.370592+0000","last_unstale":"2024-09-10T09:54:00.847215+0000","last_undegraded":"2024-09-10T09:54:00.847215+0000","last_fullsized":"2024-09-10T09:54:00.847215+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:20:23.585187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"96'6","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.685236+0000","last_change":"2024-09-10T09:52:22.319365+0000","last_active":"2024-09-10T09:54:29.685236+0000","last_peered":"2024-09-10T09:54:29.685236+0000","last_clean":"2024-09-10T09:54:29.685236+0000","last_became_active":"2024-09-10T09:52:22.319064+0000","last_became_peered":"2024-09-10T09:52:22.319064+0000","last_unstale":"2024-09-10T09:54:29.685236+0000","last_undegraded":"2024-09-10T09:54:29.685236+0000","last_fullsized":"2024-09-10T09:54:29.685236+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:14:50.089566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"94'17","reported_seq":59,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.149324+0000","last_change":"2024-09-10T09:52:19.911698+0000","last_active":"2024-09-10T09:54:01.149324+0000","last_peered":"2024-09-10T09:54:01.149324+0000","last_clean":"2024-09-10T09:54:01.149324+0000","last_became_active":"2024-09-10T09:52:19.911369+0000","last_became_peered":"2024-09-10T09:52:19.911369+0000","last_unstale":"2024-09-10T09:54:01.149324+0000","last_undegraded":"2024-09-10T09:54:01.149324+0000","last_fullsized":"2024-09-10T09:54:01.149324+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:40:39.045797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.841947+0000","last_change":"2024-09-10T09:52:31.380463+0000","last_active":"2024-09-10T09:54:00.841947+0000","last_peered":"2024-09-10T09:54:00.841947+0000","last_clean":"2024-09-10T09:54:00.841947+0000","last_became_active":"2024-09-10T09:52:31.380114+0000","last_became_peered":"2024-09-10T09:52:31.380114+0000","last_unstale":"2024-09-10T09:54:00.841947+0000","last_undegraded":"2024-09-10T09:54:00.841947+0000","last_fullsized":"2024-09-10T09:54:00.841947+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:19:50.869160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"96'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.682831+0000","last_change":"2024-09-10T09:52:22.313928+0000","last_active":"2024-09-10T09:54:29.682831+0000","last_peered":"2024-09-10T09:54:29.682831+0000","last_clean":"2024-09-10T09:54:29.682831+0000","last_became_active":"2024-09-10T09:52:22.313510+0000","last_became_peered":"2024-09-10T09:52:22.313510+0000","last_unstale":"2024-09-10T09:54:29.682831+0000","last_undegraded":"2024-09-10T09:54:29.682831+0000","last_fullsized":"2024-09-10T09:54:29.682831+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:58:19.031178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"96'7","reported_seq":185,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:55:27.108827+0000","last_change":"2024-09-10T09:52:19.839451+0000","last_active":"2024-09-10T09:55:27.108827+0000","last_peered":"2024-09-10T09:55:27.108827+0000","last_clean":"2024-09-10T09:55:27.108827+0000","last_became_active":"2024-09-10T09:52:19.839124+0000","last_became_peered":"2024-09-10T09:52:19.839124+0000","last_unstale":"2024-09-10T09:55:27.108827+0000","last_undegraded":"2024-09-10T09:55:27.108827+0000","last_fullsized":"2024-09-10T09:55:27.108827+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:51:33.489830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":137,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.844344+0000","last_change":"2024-09-10T09:52:31.376386+0000","last_active":"2024-09-10T09:54:00.844344+0000","last_peered":"2024-09-10T09:54:00.844344+0000","last_clean":"2024-09-10T09:54:00.844344+0000","last_became_active":"2024-09-10T09:52:31.376066+0000","last_became_peered":"2024-09-10T09:52:31.376066+0000","last_unstale":"2024-09-10T09:54:00.844344+0000","last_undegraded":"2024-09-10T09:54:00.844344+0000","last_fullsized":"2024-09-10T09:54:00.844344+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:23:07.191842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"96'15","reported_seq":52,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.702901+0000","last_change":"2024-09-10T09:52:22.323590+0000","last_active":"2024-09-10T09:54:29.702901+0000","last_peered":"2024-09-10T09:54:29.702901+0000","last_clean":"2024-09-10T09:54:29.702901+0000","last_became_active":"2024-09-10T09:52:22.323264+0000","last_became_peered":"2024-09-10T09:52:22.323264+0000","last_unstale":"2024-09-10T09:54:29.702901+0000","last_undegraded":"2024-09-10T09:54:29.702901+0000","last_fullsized":"2024-09-10T09:54:29.702901+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:07:56.572337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"94'5","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.847940+0000","last_change":"2024-09-10T09:52:19.840620+0000","last_active":"2024-09-10T09:54:00.847940+0000","last_peered":"2024-09-10T09:54:00.847940+0000","last_clean":"2024-09-10T09:54:00.847940+0000","last_became_active":"2024-09-10T09:52:19.840217+0000","last_became_peered":"2024-09-10T09:52:19.840217+0000","last_unstale":"2024-09-10T09:54:00.847940+0000","last_undegraded":"2024-09-10T09:54:00.847940+0000","last_fullsized":"2024-09-10T09:54:00.847940+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:55:13.111993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.955071+0000","last_change":"2024-09-10T09:52:31.373895+0000","last_active":"2024-09-10T09:54:00.955071+0000","last_peered":"2024-09-10T09:54:00.955071+0000","last_clean":"2024-09-10T09:54:00.955071+0000","last_became_active":"2024-09-10T09:52:31.373474+0000","last_became_peered":"2024-09-10T09:52:31.373474+0000","last_unstale":"2024-09-10T09:54:00.955071+0000","last_undegraded":"2024-09-10T09:54:00.955071+0000","last_fullsized":"2024-09-10T09:54:00.955071+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:39:16.049859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"96'9","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.704377+0000","last_change":"2024-09-10T09:52:22.319294+0000","last_active":"2024-09-10T09:54:29.704377+0000","last_peered":"2024-09-10T09:54:29.704377+0000","last_clean":"2024-09-10T09:54:29.704377+0000","last_became_active":"2024-09-10T09:52:22.318918+0000","last_became_peered":"2024-09-10T09:52:22.318918+0000","last_unstale":"2024-09-10T09:54:29.704377+0000","last_undegraded":"2024-09-10T09:54:29.704377+0000","last_fullsized":"2024-09-10T09:54:29.704377+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:52:46.434182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"94'9","reported_seq":53,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.844244+0000","last_change":"2024-09-10T09:52:19.842286+0000","last_active":"2024-09-10T09:54:00.844244+0000","last_peered":"2024-09-10T09:54:00.844244+0000","last_clean":"2024-09-10T09:54:00.844244+0000","last_became_active":"2024-09-10T09:52:19.841930+0000","last_became_peered":"2024-09-10T09:52:19.841930+0000","last_unstale":"2024-09-10T09:54:00.844244+0000","last_undegraded":"2024-09-10T09:54:00.844244+0000","last_fullsized":"2024-09-10T09:54:00.844244+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:38:46.358389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.847531+0000","last_change":"2024-09-10T09:52:31.370961+0000","last_active":"2024-09-10T09:54:00.847531+0000","last_peered":"2024-09-10T09:54:00.847531+0000","last_clean":"2024-09-10T09:54:00.847531+0000","last_became_active":"2024-09-10T09:52:31.370545+0000","last_became_peered":"2024-09-10T09:52:31.370545+0000","last_unstale":"2024-09-10T09:54:00.847531+0000","last_undegraded":"2024-09-10T09:54:00.847531+0000","last_fullsized":"2024-09-10T09:54:00.847531+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:17.461447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"96'6","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.685895+0000","last_change":"2024-09-10T09:52:22.313215+0000","last_active":"2024-09-10T09:54:29.685895+0000","last_peered":"2024-09-10T09:54:29.685895+0000","last_clean":"2024-09-10T09:54:29.685895+0000","last_became_active":"2024-09-10T09:52:22.312918+0000","last_became_peered":"2024-09-10T09:52:22.312918+0000","last_unstale":"2024-09-10T09:54:29.685895+0000","last_undegraded":"2024-09-10T09:54:29.685895+0000","last_fullsized":"2024-09-10T09:54:29.685895+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:48:09.568004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"96'41","reported_seq":375,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:55:27.074266+0000","last_change":"2024-09-10T09:53:01.226146+0000","last_active":"2024-09-10T09:55:27.074266+0000","last_peered":"2024-09-10T09:55:27.074266+0000","last_clean":"2024-09-10T09:55:27.074266+0000","last_became_active":"2024-09-10T09:53:01.218622+0000","last_became_peered":"2024-09-10T09:53:01.218622+0000","last_unstale":"2024-09-10T09:55:27.074266+0000","last_undegraded":"2024-09-10T09:55:27.074266+0000","last_fullsized":"2024-09-10T09:55:27.074266+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:43:11.927029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69014,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":275,"num_read_kb":206,"num_write":43,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":3844,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.841475+0000","last_change":"2024-09-10T09:53:00.800650+0000","last_active":"2024-09-10T09:54:00.841475+0000","last_peered":"2024-09-10T09:54:00.841475+0000","last_clean":"2024-09-10T09:54:00.841475+0000","last_became_active":"2024-09-10T09:53:00.800104+0000","last_became_peered":"2024-09-10T09:53:00.800104+0000","last_unstale":"2024-09-10T09:54:00.841475+0000","last_undegraded":"2024-09-10T09:54:00.841475+0000","last_fullsized":"2024-09-10T09:54:00.841475+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:41:16.022884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.14","version":"96'11","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.690121+0000","last_change":"2024-09-10T09:52:22.317174+0000","last_active":"2024-09-10T09:54:29.690121+0000","last_peered":"2024-09-10T09:54:29.690121+0000","last_clean":"2024-09-10T09:54:29.690121+0000","last_became_active":"2024-09-10T09:52:22.316851+0000","last_became_peered":"2024-09-10T09:52:22.316851+0000","last_unstale":"2024-09-10T09:54:29.690121+0000","last_undegraded":"2024-09-10T09:54:29.690121+0000","last_fullsized":"2024-09-10T09:54:29.690121+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:55:18.742393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"94'5","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.847731+0000","last_change":"2024-09-10T09:52:19.912093+0000","last_active":"2024-09-10T09:54:00.847731+0000","last_peered":"2024-09-10T09:54:00.847731+0000","last_clean":"2024-09-10T09:54:00.847731+0000","last_became_active":"2024-09-10T09:52:19.911611+0000","last_became_peered":"2024-09-10T09:52:19.911611+0000","last_unstale":"2024-09-10T09:54:00.847731+0000","last_undegraded":"2024-09-10T09:54:00.847731+0000","last_fullsized":"2024-09-10T09:54:00.847731+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:01:10.859310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.951658+0000","last_change":"2024-09-10T09:52:31.362884+0000","last_active":"2024-09-10T09:54:00.951658+0000","last_peered":"2024-09-10T09:54:00.951658+0000","last_clean":"2024-09-10T09:54:00.951658+0000","last_became_active":"2024-09-10T09:52:31.362489+0000","last_became_peered":"2024-09-10T09:52:31.362489+0000","last_unstale":"2024-09-10T09:54:00.951658+0000","last_undegraded":"2024-09-10T09:54:00.951658+0000","last_fullsized":"2024-09-10T09:54:00.951658+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:02:33.084795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"96'9","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.692830+0000","last_change":"2024-09-10T09:52:22.325871+0000","last_active":"2024-09-10T09:54:29.692830+0000","last_peered":"2024-09-10T09:54:29.692830+0000","last_clean":"2024-09-10T09:54:29.692830+0000","last_became_active":"2024-09-10T09:52:22.324980+0000","last_became_peered":"2024-09-10T09:52:22.324980+0000","last_unstale":"2024-09-10T09:54:29.692830+0000","last_undegraded":"2024-09-10T09:54:29.692830+0000","last_fullsized":"2024-09-10T09:54:29.692830+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:31:55.713444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"94'4","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.059691+0000","last_change":"2024-09-10T09:52:19.839491+0000","last_active":"2024-09-10T09:54:01.059691+0000","last_peered":"2024-09-10T09:54:01.059691+0000","last_clean":"2024-09-10T09:54:01.059691+0000","last_became_active":"2024-09-10T09:52:19.839092+0000","last_became_peered":"2024-09-10T09:52:19.839092+0000","last_unstale":"2024-09-10T09:54:01.059691+0000","last_undegraded":"2024-09-10T09:54:01.059691+0000","last_fullsized":"2024-09-10T09:54:01.059691+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:42:13.453281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.843285+0000","last_change":"2024-09-10T09:52:31.371346+0000","last_active":"2024-09-10T09:54:00.843285+0000","last_peered":"2024-09-10T09:54:00.843285+0000","last_clean":"2024-09-10T09:54:00.843285+0000","last_became_active":"2024-09-10T09:52:31.370929+0000","last_became_peered":"2024-09-10T09:52:31.370929+0000","last_unstale":"2024-09-10T09:54:00.843285+0000","last_undegraded":"2024-09-10T09:54:00.843285+0000","last_fullsized":"2024-09-10T09:54:00.843285+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:44:22.925082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"96'10","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.694062+0000","last_change":"2024-09-10T09:52:22.317453+0000","last_active":"2024-09-10T09:54:29.694062+0000","last_peered":"2024-09-10T09:54:29.694062+0000","last_clean":"2024-09-10T09:54:29.694062+0000","last_became_active":"2024-09-10T09:52:22.315556+0000","last_became_peered":"2024-09-10T09:52:22.315556+0000","last_unstale":"2024-09-10T09:54:29.694062+0000","last_undegraded":"2024-09-10T09:54:29.694062+0000","last_fullsized":"2024-09-10T09:54:29.694062+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:56:05.860275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"96'5","reported_seq":310,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:55:27.074266+0000","last_change":"2024-09-10T09:52:19.835985+0000","last_active":"2024-09-10T09:55:27.074266+0000","last_peered":"2024-09-10T09:55:27.074266+0000","last_clean":"2024-09-10T09:55:27.074266+0000","last_became_active":"2024-09-10T09:52:19.834835+0000","last_became_peered":"2024-09-10T09:52:19.834835+0000","last_unstale":"2024-09-10T09:55:27.074266+0000","last_undegraded":"2024-09-10T09:55:27.074266+0000","last_fullsized":"2024-09-10T09:55:27.074266+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:09:11.590443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":262,"num_read_kb":262,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.845724+0000","last_change":"2024-09-10T09:52:31.368663+0000","last_active":"2024-09-10T09:54:00.845724+0000","last_peered":"2024-09-10T09:54:00.845724+0000","last_clean":"2024-09-10T09:54:00.845724+0000","last_became_active":"2024-09-10T09:52:31.368004+0000","last_became_peered":"2024-09-10T09:52:31.368004+0000","last_unstale":"2024-09-10T09:54:00.845724+0000","last_undegraded":"2024-09-10T09:54:00.845724+0000","last_fullsized":"2024-09-10T09:54:00.845724+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:56:15.100515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:01.247917+0000","last_change":"2024-09-10T09:52:31.370992+0000","last_active":"2024-09-10T09:54:01.247917+0000","last_peered":"2024-09-10T09:54:01.247917+0000","last_clean":"2024-09-10T09:54:01.247917+0000","last_became_active":"2024-09-10T09:52:31.370357+0000","last_became_peered":"2024-09-10T09:52:31.370357+0000","last_unstale":"2024-09-10T09:54:01.247917+0000","last_undegraded":"2024-09-10T09:54:01.247917+0000","last_fullsized":"2024-09-10T09:54:01.247917+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:02:49.482532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"96'11","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.706426+0000","last_change":"2024-09-10T09:52:22.319363+0000","last_active":"2024-09-10T09:54:29.706426+0000","last_peered":"2024-09-10T09:54:29.706426+0000","last_clean":"2024-09-10T09:54:29.706426+0000","last_became_active":"2024-09-10T09:52:22.319044+0000","last_became_peered":"2024-09-10T09:52:22.319044+0000","last_unstale":"2024-09-10T09:54:29.706426+0000","last_undegraded":"2024-09-10T09:54:29.706426+0000","last_fullsized":"2024-09-10T09:54:29.706426+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:30:51.328413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"94'3","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.847831+0000","last_change":"2024-09-10T09:52:19.841745+0000","last_active":"2024-09-10T09:54:00.847831+0000","last_peered":"2024-09-10T09:54:00.847831+0000","last_clean":"2024-09-10T09:54:00.847831+0000","last_became_active":"2024-09-10T09:52:19.841404+0000","last_became_peered":"2024-09-10T09:52:19.841404+0000","last_unstale":"2024-09-10T09:54:00.847831+0000","last_undegraded":"2024-09-10T09:54:00.847831+0000","last_fullsized":"2024-09-10T09:54:00.847831+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:44:36.346785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.955521+0000","last_change":"2024-09-10T09:53:01.218412+0000","last_active":"2024-09-10T09:54:00.955521+0000","last_peered":"2024-09-10T09:54:00.955521+0000","last_clean":"2024-09-10T09:54:00.955521+0000","last_became_active":"2024-09-10T09:53:01.217966+0000","last_became_peered":"2024-09-10T09:53:01.217966+0000","last_unstale":"2024-09-10T09:54:00.955521+0000","last_undegraded":"2024-09-10T09:54:00.955521+0000","last_fullsized":"2024-09-10T09:54:00.955521+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:53:25.743913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"96'8","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.703441+0000","last_change":"2024-09-10T09:52:22.316120+0000","last_active":"2024-09-10T09:54:29.703441+0000","last_peered":"2024-09-10T09:54:29.703441+0000","last_clean":"2024-09-10T09:54:29.703441+0000","last_became_active":"2024-09-10T09:52:22.315769+0000","last_became_peered":"2024-09-10T09:52:22.315769+0000","last_unstale":"2024-09-10T09:54:29.703441+0000","last_undegraded":"2024-09-10T09:54:29.703441+0000","last_fullsized":"2024-09-10T09:54:29.703441+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:16:41.866587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"94'3","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.841046+0000","last_change":"2024-09-10T09:52:19.835038+0000","last_active":"2024-09-10T09:54:00.841046+0000","last_peered":"2024-09-10T09:54:00.841046+0000","last_clean":"2024-09-10T09:54:00.841046+0000","last_became_active":"2024-09-10T09:52:19.834329+0000","last_became_peered":"2024-09-10T09:52:19.834329+0000","last_unstale":"2024-09-10T09:54:00.841046+0000","last_undegraded":"2024-09-10T09:54:00.841046+0000","last_fullsized":"2024-09-10T09:54:00.841046+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:58:24.084373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"94'7","reported_seq":56,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.841764+0000","last_change":"2024-09-10T09:53:01.219122+0000","last_active":"2024-09-10T09:54:00.841764+0000","last_peered":"2024-09-10T09:54:00.841764+0000","last_clean":"2024-09-10T09:54:00.841764+0000","last_became_active":"2024-09-10T09:53:01.218599+0000","last_became_peered":"2024-09-10T09:53:01.218599+0000","last_unstale":"2024-09-10T09:54:00.841764+0000","last_undegraded":"2024-09-10T09:54:00.841764+0000","last_fullsized":"2024-09-10T09:54:00.841764+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:31:10.092264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"96'8","reported_seq":19,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.706393+0000","last_change":"2024-09-10T09:54:00.952617+0000","last_active":"2024-09-10T09:54:29.706393+0000","last_peered":"2024-09-10T09:54:29.706393+0000","last_clean":"2024-09-10T09:54:29.706393+0000","last_became_active":"2024-09-10T09:54:00.952202+0000","last_became_peered":"2024-09-10T09:54:00.952202+0000","last_unstale":"2024-09-10T09:54:29.706393+0000","last_undegraded":"2024-09-10T09:54:29.706393+0000","last_fullsized":"2024-09-10T09:54:29.706393+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:35:40.179591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.844169+0000","last_change":"2024-09-10T09:52:31.371612+0000","last_active":"2024-09-10T09:54:00.844169+0000","last_peered":"2024-09-10T09:54:00.844169+0000","last_clean":"2024-09-10T09:54:00.844169+0000","last_became_active":"2024-09-10T09:52:31.371163+0000","last_became_peered":"2024-09-10T09:52:31.371163+0000","last_unstale":"2024-09-10T09:54:00.844169+0000","last_undegraded":"2024-09-10T09:54:00.844169+0000","last_fullsized":"2024-09-10T09:54:00.844169+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:11:43.681524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"96'7","reported_seq":186,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:55:27.075107+0000","last_change":"2024-09-10T09:52:19.841873+0000","last_active":"2024-09-10T09:55:27.075107+0000","last_peered":"2024-09-10T09:55:27.075107+0000","last_clean":"2024-09-10T09:55:27.075107+0000","last_became_active":"2024-09-10T09:52:19.841513+0000","last_became_peered":"2024-09-10T09:52:19.841513+0000","last_unstale":"2024-09-10T09:55:27.075107+0000","last_undegraded":"2024-09-10T09:55:27.075107+0000","last_fullsized":"2024-09-10T09:55:27.075107+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:18.816893+0000","last_clean_scrub_stamp":"2024-09-10T09:52:18.816893+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:53:31.854185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1156,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"96'7","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.697738+0000","last_change":"2024-09-10T09:52:22.314781+0000","last_active":"2024-09-10T09:54:29.697738+0000","last_peered":"2024-09-10T09:54:29.697738+0000","last_clean":"2024-09-10T09:54:29.697738+0000","last_became_active":"2024-09-10T09:52:22.313346+0000","last_became_peered":"2024-09-10T09:52:22.313346+0000","last_unstale":"2024-09-10T09:54:29.697738+0000","last_undegraded":"2024-09-10T09:54:29.697738+0000","last_fullsized":"2024-09-10T09:54:29.697738+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:15:16.921963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:00.844401+0000","last_change":"2024-09-10T09:52:31.365544+0000","last_active":"2024-09-10T09:54:00.844401+0000","last_peered":"2024-09-10T09:54:00.844401+0000","last_clean":"2024-09-10T09:54:00.844401+0000","last_became_active":"2024-09-10T09:52:31.365151+0000","last_became_peered":"2024-09-10T09:52:31.365151+0000","last_unstale":"2024-09-10T09:54:00.844401+0000","last_undegraded":"2024-09-10T09:54:00.844401+0000","last_fullsized":"2024-09-10T09:54:00.844401+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:30.339352+0000","last_clean_scrub_stamp":"2024-09-10T09:52:30.339352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:36:55.247890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"96'9","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.683641+0000","last_change":"2024-09-10T09:52:22.314117+0000","last_active":"2024-09-10T09:54:29.683641+0000","last_peered":"2024-09-10T09:54:29.683641+0000","last_clean":"2024-09-10T09:54:29.683641+0000","last_became_active":"2024-09-10T09:52:22.313772+0000","last_became_peered":"2024-09-10T09:52:22.313772+0000","last_unstale":"2024-09-10T09:54:29.683641+0000","last_undegraded":"2024-09-10T09:54:29.683641+0000","last_fullsized":"2024-09-10T09:54:29.683641+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:19:37.373598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"96'12","reported_seq":50,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.682477+0000","last_change":"2024-09-10T09:52:22.314362+0000","last_active":"2024-09-10T09:54:29.682477+0000","last_peered":"2024-09-10T09:54:29.682477+0000","last_clean":"2024-09-10T09:54:29.682477+0000","last_became_active":"2024-09-10T09:52:22.314050+0000","last_became_peered":"2024-09-10T09:52:22.314050+0000","last_unstale":"2024-09-10T09:54:29.682477+0000","last_undegraded":"2024-09-10T09:54:29.682477+0000","last_fullsized":"2024-09-10T09:54:29.682477+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:37:07.218656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"96'8","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.699844+0000","last_change":"2024-09-10T09:52:22.322363+0000","last_active":"2024-09-10T09:54:29.699844+0000","last_peered":"2024-09-10T09:54:29.699844+0000","last_clean":"2024-09-10T09:54:29.699844+0000","last_became_active":"2024-09-10T09:52:22.322045+0000","last_became_peered":"2024-09-10T09:52:22.322045+0000","last_unstale":"2024-09-10T09:54:29.699844+0000","last_undegraded":"2024-09-10T09:54:29.699844+0000","last_fullsized":"2024-09-10T09:54:29.699844+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:10:27.918769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.700329+0000","last_change":"2024-09-10T09:52:22.331594+0000","last_active":"2024-09-10T09:54:29.700329+0000","last_peered":"2024-09-10T09:54:29.700329+0000","last_clean":"2024-09-10T09:54:29.700329+0000","last_became_active":"2024-09-10T09:52:22.330585+0000","last_became_peered":"2024-09-10T09:52:22.330585+0000","last_unstale":"2024-09-10T09:54:29.700329+0000","last_undegraded":"2024-09-10T09:54:29.700329+0000","last_fullsized":"2024-09-10T09:54:29.700329+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:45:56.412313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"96'4","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.705719+0000","last_change":"2024-09-10T09:52:22.325753+0000","last_active":"2024-09-10T09:54:29.705719+0000","last_peered":"2024-09-10T09:54:29.705719+0000","last_clean":"2024-09-10T09:54:29.705719+0000","last_became_active":"2024-09-10T09:52:22.325385+0000","last_became_peered":"2024-09-10T09:52:22.325385+0000","last_unstale":"2024-09-10T09:54:29.705719+0000","last_undegraded":"2024-09-10T09:54:29.705719+0000","last_fullsized":"2024-09-10T09:54:29.705719+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:18:28.776789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"96'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.681378+0000","last_change":"2024-09-10T09:52:22.319069+0000","last_active":"2024-09-10T09:54:29.681378+0000","last_peered":"2024-09-10T09:54:29.681378+0000","last_clean":"2024-09-10T09:54:29.681378+0000","last_became_active":"2024-09-10T09:52:22.318667+0000","last_became_peered":"2024-09-10T09:52:22.318667+0000","last_unstale":"2024-09-10T09:54:29.681378+0000","last_undegraded":"2024-09-10T09:54:29.681378+0000","last_fullsized":"2024-09-10T09:54:29.681378+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:21:32.028235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"96'15","reported_seq":53,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.681487+0000","last_change":"2024-09-10T09:52:22.312890+0000","last_active":"2024-09-10T09:54:29.681487+0000","last_peered":"2024-09-10T09:54:29.681487+0000","last_clean":"2024-09-10T09:54:29.681487+0000","last_became_active":"2024-09-10T09:52:22.312608+0000","last_became_peered":"2024-09-10T09:52:22.312608+0000","last_unstale":"2024-09-10T09:54:29.681487+0000","last_undegraded":"2024-09-10T09:54:29.681487+0000","last_fullsized":"2024-09-10T09:54:29.681487+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:07:46.046915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"96'2","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.689884+0000","last_change":"2024-09-10T09:54:01.151385+0000","last_active":"2024-09-10T09:54:29.689884+0000","last_peered":"2024-09-10T09:54:29.689884+0000","last_clean":"2024-09-10T09:54:29.689884+0000","last_became_active":"2024-09-10T09:54:01.150971+0000","last_became_peered":"2024-09-10T09:54:01.150971+0000","last_unstale":"2024-09-10T09:54:29.689884+0000","last_undegraded":"2024-09-10T09:54:29.689884+0000","last_fullsized":"2024-09-10T09:54:29.689884+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:27:10.492071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"96'6","reported_seq":17,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.704565+0000","last_change":"2024-09-10T09:54:00.848273+0000","last_active":"2024-09-10T09:54:29.704565+0000","last_peered":"2024-09-10T09:54:29.704565+0000","last_clean":"2024-09-10T09:54:29.704565+0000","last_became_active":"2024-09-10T09:54:00.847492+0000","last_became_peered":"2024-09-10T09:54:00.847492+0000","last_unstale":"2024-09-10T09:54:29.704565+0000","last_undegraded":"2024-09-10T09:54:29.704565+0000","last_fullsized":"2024-09-10T09:54:29.704565+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:02:24.749291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,3],"acting":[6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.29","version":"96'8","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.696836+0000","last_change":"2024-09-10T09:52:22.313538+0000","last_active":"2024-09-10T09:54:29.696836+0000","last_peered":"2024-09-10T09:54:29.696836+0000","last_clean":"2024-09-10T09:54:29.696836+0000","last_became_active":"2024-09-10T09:52:22.313196+0000","last_became_peered":"2024-09-10T09:52:22.313196+0000","last_unstale":"2024-09-10T09:54:29.696836+0000","last_undegraded":"2024-09-10T09:54:29.696836+0000","last_fullsized":"2024-09-10T09:54:29.696836+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:53:23.387167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"96'10","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.696632+0000","last_change":"2024-09-10T09:52:22.316518+0000","last_active":"2024-09-10T09:54:29.696632+0000","last_peered":"2024-09-10T09:54:29.696632+0000","last_clean":"2024-09-10T09:54:29.696632+0000","last_became_active":"2024-09-10T09:52:22.316193+0000","last_became_peered":"2024-09-10T09:52:22.316193+0000","last_unstale":"2024-09-10T09:54:29.696632+0000","last_undegraded":"2024-09-10T09:54:29.696632+0000","last_fullsized":"2024-09-10T09:54:29.696632+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:49:55.903928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"96'11","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.706245+0000","last_change":"2024-09-10T09:52:22.329163+0000","last_active":"2024-09-10T09:54:29.706245+0000","last_peered":"2024-09-10T09:54:29.706245+0000","last_clean":"2024-09-10T09:54:29.706245+0000","last_became_active":"2024-09-10T09:52:22.328453+0000","last_became_peered":"2024-09-10T09:52:22.328453+0000","last_unstale":"2024-09-10T09:54:29.706245+0000","last_undegraded":"2024-09-10T09:54:29.706245+0000","last_fullsized":"2024-09-10T09:54:29.706245+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:53:33.286055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"96'8","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.681766+0000","last_change":"2024-09-10T09:52:22.314302+0000","last_active":"2024-09-10T09:54:29.681766+0000","last_peered":"2024-09-10T09:54:29.681766+0000","last_clean":"2024-09-10T09:54:29.681766+0000","last_became_active":"2024-09-10T09:52:22.313992+0000","last_became_peered":"2024-09-10T09:52:22.313992+0000","last_unstale":"2024-09-10T09:54:29.681766+0000","last_undegraded":"2024-09-10T09:54:29.681766+0000","last_fullsized":"2024-09-10T09:54:29.681766+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:07:35.764015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"96'7","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.688705+0000","last_change":"2024-09-10T09:52:22.316400+0000","last_active":"2024-09-10T09:54:29.688705+0000","last_peered":"2024-09-10T09:54:29.688705+0000","last_clean":"2024-09-10T09:54:29.688705+0000","last_became_active":"2024-09-10T09:52:22.316110+0000","last_became_peered":"2024-09-10T09:52:22.316110+0000","last_unstale":"2024-09-10T09:54:29.688705+0000","last_undegraded":"2024-09-10T09:54:29.688705+0000","last_fullsized":"2024-09-10T09:54:29.688705+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:17:26.528899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"96'10","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.704074+0000","last_change":"2024-09-10T09:52:22.317914+0000","last_active":"2024-09-10T09:54:29.704074+0000","last_peered":"2024-09-10T09:54:29.704074+0000","last_clean":"2024-09-10T09:54:29.704074+0000","last_became_active":"2024-09-10T09:52:22.317575+0000","last_became_peered":"2024-09-10T09:52:22.317575+0000","last_unstale":"2024-09-10T09:54:29.704074+0000","last_undegraded":"2024-09-10T09:54:29.704074+0000","last_fullsized":"2024-09-10T09:54:29.704074+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:17:51.446233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"96'8","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.697848+0000","last_change":"2024-09-10T09:52:22.327749+0000","last_active":"2024-09-10T09:54:29.697848+0000","last_peered":"2024-09-10T09:54:29.697848+0000","last_clean":"2024-09-10T09:54:29.697848+0000","last_became_active":"2024-09-10T09:52:22.327461+0000","last_became_peered":"2024-09-10T09:52:22.327461+0000","last_unstale":"2024-09-10T09:54:29.697848+0000","last_undegraded":"2024-09-10T09:54:29.697848+0000","last_fullsized":"2024-09-10T09:54:29.697848+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:54.208064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"96'9","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.696695+0000","last_change":"2024-09-10T09:52:22.326982+0000","last_active":"2024-09-10T09:54:29.696695+0000","last_peered":"2024-09-10T09:54:29.696695+0000","last_clean":"2024-09-10T09:54:29.696695+0000","last_became_active":"2024-09-10T09:52:22.326635+0000","last_became_peered":"2024-09-10T09:52:22.326635+0000","last_unstale":"2024-09-10T09:54:29.696695+0000","last_undegraded":"2024-09-10T09:54:29.696695+0000","last_fullsized":"2024-09-10T09:54:29.696695+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:50:54.835899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.695757+0000","last_change":"2024-09-10T09:52:22.323410+0000","last_active":"2024-09-10T09:54:29.695757+0000","last_peered":"2024-09-10T09:54:29.695757+0000","last_clean":"2024-09-10T09:54:29.695757+0000","last_became_active":"2024-09-10T09:52:22.323116+0000","last_became_peered":"2024-09-10T09:52:22.323116+0000","last_unstale":"2024-09-10T09:54:29.695757+0000","last_undegraded":"2024-09-10T09:54:29.695757+0000","last_fullsized":"2024-09-10T09:54:29.695757+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:09:15.115162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.680224+0000","last_change":"2024-09-10T09:52:22.324908+0000","last_active":"2024-09-10T09:54:29.680224+0000","last_peered":"2024-09-10T09:54:29.680224+0000","last_clean":"2024-09-10T09:54:29.680224+0000","last_became_active":"2024-09-10T09:52:22.324449+0000","last_became_peered":"2024-09-10T09:52:22.324449+0000","last_unstale":"2024-09-10T09:54:29.680224+0000","last_undegraded":"2024-09-10T09:54:29.680224+0000","last_fullsized":"2024-09-10T09:54:29.680224+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:43:24.535721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"96'10","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.700039+0000","last_change":"2024-09-10T09:52:22.322157+0000","last_active":"2024-09-10T09:54:29.700039+0000","last_peered":"2024-09-10T09:54:29.700039+0000","last_clean":"2024-09-10T09:54:29.700039+0000","last_became_active":"2024-09-10T09:52:22.321856+0000","last_became_peered":"2024-09-10T09:52:22.321856+0000","last_unstale":"2024-09-10T09:54:29.700039+0000","last_undegraded":"2024-09-10T09:54:29.700039+0000","last_fullsized":"2024-09-10T09:54:29.700039+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:36:38.395370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"96'7","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.703907+0000","last_change":"2024-09-10T09:53:00.800337+0000","last_active":"2024-09-10T09:54:29.703907+0000","last_peered":"2024-09-10T09:54:29.703907+0000","last_clean":"2024-09-10T09:54:29.703907+0000","last_became_active":"2024-09-10T09:53:00.799863+0000","last_became_peered":"2024-09-10T09:53:00.799863+0000","last_unstale":"2024-09-10T09:54:29.703907+0000","last_undegraded":"2024-09-10T09:54:29.703907+0000","last_fullsized":"2024-09-10T09:54:29.703907+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:18.017672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.35","version":"96'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.693580+0000","last_change":"2024-09-10T09:52:22.310735+0000","last_active":"2024-09-10T09:54:29.693580+0000","last_peered":"2024-09-10T09:54:29.693580+0000","last_clean":"2024-09-10T09:54:29.693580+0000","last_became_active":"2024-09-10T09:52:22.310376+0000","last_became_peered":"2024-09-10T09:52:22.310376+0000","last_unstale":"2024-09-10T09:54:29.693580+0000","last_undegraded":"2024-09-10T09:54:29.693580+0000","last_fullsized":"2024-09-10T09:54:29.693580+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:38:08.637588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"96'16","reported_seq":55,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.685458+0000","last_change":"2024-09-10T09:52:22.321113+0000","last_active":"2024-09-10T09:54:29.685458+0000","last_peered":"2024-09-10T09:54:29.685458+0000","last_clean":"2024-09-10T09:54:29.685458+0000","last_became_active":"2024-09-10T09:52:22.320767+0000","last_became_peered":"2024-09-10T09:52:22.320767+0000","last_unstale":"2024-09-10T09:54:29.685458+0000","last_undegraded":"2024-09-10T09:54:29.685458+0000","last_fullsized":"2024-09-10T09:54:29.685458+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:37.549322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"96'11","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.687162+0000","last_change":"2024-09-10T09:52:22.318634+0000","last_active":"2024-09-10T09:54:29.687162+0000","last_peered":"2024-09-10T09:54:29.687162+0000","last_clean":"2024-09-10T09:54:29.687162+0000","last_became_active":"2024-09-10T09:52:22.318294+0000","last_became_peered":"2024-09-10T09:52:22.318294+0000","last_unstale":"2024-09-10T09:54:29.687162+0000","last_undegraded":"2024-09-10T09:54:29.687162+0000","last_fullsized":"2024-09-10T09:54:29.687162+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:19:31.262055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"96'9","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.698725+0000","last_change":"2024-09-10T09:52:22.324567+0000","last_active":"2024-09-10T09:54:29.698725+0000","last_peered":"2024-09-10T09:54:29.698725+0000","last_clean":"2024-09-10T09:54:29.698725+0000","last_became_active":"2024-09-10T09:52:22.321803+0000","last_became_peered":"2024-09-10T09:52:22.321803+0000","last_unstale":"2024-09-10T09:54:29.698725+0000","last_undegraded":"2024-09-10T09:54:29.698725+0000","last_fullsized":"2024-09-10T09:54:29.698725+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:08:11.099428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"96'5","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.699735+0000","last_change":"2024-09-10T09:52:22.318163+0000","last_active":"2024-09-10T09:54:29.699735+0000","last_peered":"2024-09-10T09:54:29.699735+0000","last_clean":"2024-09-10T09:54:29.699735+0000","last_became_active":"2024-09-10T09:52:22.317808+0000","last_became_peered":"2024-09-10T09:52:22.317808+0000","last_unstale":"2024-09-10T09:54:29.699735+0000","last_undegraded":"2024-09-10T09:54:29.699735+0000","last_fullsized":"2024-09-10T09:54:29.699735+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:19.214291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"96'6","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.704960+0000","last_change":"2024-09-10T09:52:22.323035+0000","last_active":"2024-09-10T09:54:29.704960+0000","last_peered":"2024-09-10T09:54:29.704960+0000","last_clean":"2024-09-10T09:54:29.704960+0000","last_became_active":"2024-09-10T09:52:22.322679+0000","last_became_peered":"2024-09-10T09:52:22.322679+0000","last_unstale":"2024-09-10T09:54:29.704960+0000","last_undegraded":"2024-09-10T09:54:29.704960+0000","last_fullsized":"2024-09-10T09:54:29.704960+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:23:13.762612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"96'4","reported_seq":40,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.700111+0000","last_change":"2024-09-10T09:52:22.315887+0000","last_active":"2024-09-10T09:54:29.700111+0000","last_peered":"2024-09-10T09:54:29.700111+0000","last_clean":"2024-09-10T09:54:29.700111+0000","last_became_active":"2024-09-10T09:52:22.315569+0000","last_became_peered":"2024-09-10T09:52:22.315569+0000","last_unstale":"2024-09-10T09:54:29.700111+0000","last_undegraded":"2024-09-10T09:54:29.700111+0000","last_fullsized":"2024-09-10T09:54:29.700111+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:46:57.094320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"96'8","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.676312+0000","last_change":"2024-09-10T09:52:22.314060+0000","last_active":"2024-09-10T09:54:29.676312+0000","last_peered":"2024-09-10T09:54:29.676312+0000","last_clean":"2024-09-10T09:54:29.676312+0000","last_became_active":"2024-09-10T09:52:22.313572+0000","last_became_peered":"2024-09-10T09:52:22.313572+0000","last_unstale":"2024-09-10T09:54:29.676312+0000","last_undegraded":"2024-09-10T09:54:29.676312+0000","last_fullsized":"2024-09-10T09:54:29.676312+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:11:17.158625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.692163+0000","last_change":"2024-09-10T09:52:22.327233+0000","last_active":"2024-09-10T09:54:29.692163+0000","last_peered":"2024-09-10T09:54:29.692163+0000","last_clean":"2024-09-10T09:54:29.692163+0000","last_became_active":"2024-09-10T09:52:22.326945+0000","last_became_peered":"2024-09-10T09:52:22.326945+0000","last_unstale":"2024-09-10T09:54:29.692163+0000","last_undegraded":"2024-09-10T09:54:29.692163+0000","last_fullsized":"2024-09-10T09:54:29.692163+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:31:13.490147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"96'4","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.681664+0000","last_change":"2024-09-10T09:52:22.330912+0000","last_active":"2024-09-10T09:54:29.681664+0000","last_peered":"2024-09-10T09:54:29.681664+0000","last_clean":"2024-09-10T09:54:29.681664+0000","last_became_active":"2024-09-10T09:52:22.330149+0000","last_became_peered":"2024-09-10T09:52:22.330149+0000","last_unstale":"2024-09-10T09:54:29.681664+0000","last_undegraded":"2024-09-10T09:54:29.681664+0000","last_fullsized":"2024-09-10T09:54:29.681664+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:10:48.842499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"96'8","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:54:29.697891+0000","last_change":"2024-09-10T09:52:22.327035+0000","last_active":"2024-09-10T09:54:29.697891+0000","last_peered":"2024-09-10T09:54:29.697891+0000","last_clean":"2024-09-10T09:54:29.697891+0000","last_became_active":"2024-09-10T09:52:22.326741+0000","last_became_peered":"2024-09-10T09:52:22.326741+0000","last_unstale":"2024-09-10T09:54:29.697891+0000","last_undegraded":"2024-09-10T09:54:29.697891+0000","last_fullsized":"2024-09-10T09:54:29.697891+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:52:21.285432+0000","last_clean_scrub_stamp":"2024-09-10T09:52:21.285432+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:21:05.026614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":76815,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1362,"num_read_kb":766,"num_write":264,"num_write_kb":160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":4426,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":230445,"data_compressed":5661,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":228,"ondisk_log_size":228,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282980,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449140,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1073152,"data_stored":554869,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":70,"seq":300647710775,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32376,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712776,"statfs":{"total":95995035648,"available":95961882624,"internally_reserved":0,"allocated":1220608,"data_stored":1210334,"data_compressed":70462,"data_compressed_allocated":94208,"data_compressed_original":688128,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":65,"seq":279172874302,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32256,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712896,"statfs":{"total":95995035648,"available":95962005504,"internally_reserved":0,"allocated":1097728,"data_stored":557996,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135940,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32244,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712908,"statfs":{"total":95995035648,"available":95962017792,"internally_reserved":0,"allocated":1085440,"data_stored":557702,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364872,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28136,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717016,"statfs":{"total":95995035648,"available":95966224384,"internally_reserved":0,"allocated":1138688,"data_stored":1150864,"data_compressed":68575,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528399,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1069056,"data_stored":549715,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790037,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28144,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717008,"statfs":{"total":95995035648,"available":95966216192,"internally_reserved":0,"allocated":1146880,"data_stored":1142404,"data_compressed":68575,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953560,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28148,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717004,"statfs":{"total":95995035648,"available":95966212096,"internally_reserved":0,"allocated":1150976,"data_stored":623026,"data_compressed":34470,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182493,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32240,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712912,"statfs":{"total":95995035648,"available":95962021888,"internally_reserved":0,"allocated":1081344,"data_stored":549112,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411428,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36404,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708748,"statfs":{"total":95995035648,"available":95957757952,"internally_reserved":0,"allocated":1085440,"data_stored":551188,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574954,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28144,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717008,"statfs":{"total":95995035648,"available":95966216192,"internally_reserved":0,"allocated":1146880,"data_stored":620431,"data_compressed":34470,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738479,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1093632,"data_stored":550860,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34340,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34340,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34340,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35992,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35992,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35992,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35992,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35992,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":71749,"data_compressed":1887,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":70248,"data_compressed":1887,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":71284,"data_compressed":1887,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T09:55:31.429 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:55:31.430 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:55:30 smithi120 ceph-mon[34770]: pgmap v401: 129 pgs: 129 active+clean; 652 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 853 B/s wr, 12 op/s 2024-09-10T09:55:31.806 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:55:32.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:55:31 smithi077 ceph-mon[30319]: from='client.15309 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T09:55:32.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:55:31 smithi077 ceph-mon[30319]: pgmap v402: 129 pgs: 129 active+clean; 652 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-10T09:55:32.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:55:31 smithi134 ceph-mon[35674]: from='client.15309 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T09:55:32.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:55:31 smithi134 ceph-mon[35674]: pgmap v402: 129 pgs: 129 active+clean; 652 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-10T09:55:32.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:32.383+0000 7faccca00640 1 -- 172.21.15.77:0/93844268 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7facc8108f60 msgr2=0x7facc8110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:32.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:32.383+0000 7faccca00640 1 --2- 172.21.15.77:0/93844268 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7facc8108f60 0x7facc8110ae0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7facc0009ca0 tx=0x7facc002a8c0 comp rx=0 tx=0).stop 2024-09-10T09:55:32.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:32.383+0000 7faccca00640 1 -- 172.21.15.77:0/93844268 shutdown_connections 2024-09-10T09:55:32.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:32.383+0000 7faccca00640 1 --2- 172.21.15.77:0/93844268 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7facc8108f60 0x7facc8110ae0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:32.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:32.383+0000 7faccca00640 1 --2- 172.21.15.77:0/93844268 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7facc81047e0 0x7facc8108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:32.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:32.383+0000 7faccca00640 1 --2- 172.21.15.77:0/93844268 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7facc8103e30 0x7facc8104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:32.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:32.383+0000 7faccca00640 1 -- 172.21.15.77:0/93844268 >> 172.21.15.77:0/93844268 conn(0x7facc80ff910 msgr2=0x7facc8101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:32.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:32.383+0000 7faccca00640 1 -- 172.21.15.77:0/93844268 shutdown_connections 2024-09-10T09:55:32.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:32.383+0000 7faccca00640 1 -- 172.21.15.77:0/93844268 wait complete. 2024-09-10T09:55:32.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:32.384+0000 7faccca00640 1 Processor -- start 2024-09-10T09:55:32.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:32.385+0000 7faccca00640 1 -- start start 2024-09-10T09:55:32.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:32.385+0000 7faccca00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7facc8103e30 0x7facc819a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:32.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:32.385+0000 7faccca00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7facc81047e0 0x7facc819a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:32.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:32.385+0000 7faccca00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7facc8108f60 0x7facc819ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:32.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:32.385+0000 7faccca00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7facc8113fc0 con 0x7facc8103e30 2024-09-10T09:55:32.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:32.385+0000 7faccca00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7facc8113e40 con 0x7facc81047e0 2024-09-10T09:55:32.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:32.385+0000 7faccca00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7facc8114140 con 0x7facc8108f60 2024-09-10T09:55:32.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:32.385+0000 7facc6800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7facc8103e30 0x7facc819a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:32.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:32.385+0000 7facc6800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7facc8103e30 0x7facc819a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56996/0 (socket says 172.21.15.77:56996) 2024-09-10T09:55:32.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:32.385+0000 7facc6800640 1 -- 172.21.15.77:0/3441754284 learned_addr learned my addr 172.21.15.77:0/3441754284 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:55:32.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:32.386+0000 7facc7200640 1 --2- 172.21.15.77:0/3441754284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7facc8108f60 0x7facc819ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:32.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:32.386+0000 7facbfe00640 1 --2- 172.21.15.77:0/3441754284 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7facc81047e0 0x7facc819a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:32.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:32.386+0000 7facc6800640 1 -- 172.21.15.77:0/3441754284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7facc8108f60 msgr2=0x7facc819ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:32.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:32.386+0000 7facc6800640 1 --2- 172.21.15.77:0/3441754284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7facc8108f60 0x7facc819ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:32.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:32.386+0000 7facc6800640 1 -- 172.21.15.77:0/3441754284 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7facc81047e0 msgr2=0x7facc819a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:32.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:32.386+0000 7facc6800640 1 --2- 172.21.15.77:0/3441754284 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7facc81047e0 0x7facc819a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:32.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:32.386+0000 7facc6800640 1 -- 172.21.15.77:0/3441754284 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facc819f3b0 con 0x7facc8103e30 2024-09-10T09:55:32.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:32.386+0000 7facc6800640 1 --2- 172.21.15.77:0/3441754284 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7facc8103e30 0x7facc819a400 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7facb800e9f0 tx=0x7facb800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:32.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:32.387+0000 7facbf400640 1 -- 172.21.15.77:0/3441754284 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb800cd80 con 0x7facc8103e30 2024-09-10T09:55:32.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:32.387+0000 7facbf400640 1 -- 172.21.15.77:0/3441754284 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facb8010910 con 0x7facc8103e30 2024-09-10T09:55:32.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:32.387+0000 7faccca00640 1 -- 172.21.15.77:0/3441754284 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facc819f6a0 con 0x7facc8103e30 2024-09-10T09:55:32.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:32.387+0000 7facbf400640 1 -- 172.21.15.77:0/3441754284 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb8018990 con 0x7facc8103e30 2024-09-10T09:55:32.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:32.387+0000 7faccca00640 1 -- 172.21.15.77:0/3441754284 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facc81a6f80 con 0x7facc8103e30 2024-09-10T09:55:32.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:32.388+0000 7faccca00640 1 -- 172.21.15.77:0/3441754284 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facc8105310 con 0x7facc8103e30 2024-09-10T09:55:32.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:32.389+0000 7facbf400640 1 -- 172.21.15.77:0/3441754284 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7facb8010ac0 con 0x7facc8103e30 2024-09-10T09:55:32.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:32.391+0000 7facbf400640 1 --2- 172.21.15.77:0/3441754284 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7facac0a2c00 0x7facac0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:32.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:32.391+0000 7facbf400640 1 -- 172.21.15.77:0/3441754284 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7facb80cf700 con 0x7facc8103e30 2024-09-10T09:55:32.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:32.391+0000 7facbfe00640 1 --2- 172.21.15.77:0/3441754284 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7facac0a2c00 0x7facac0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:32.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:32.392+0000 7facbfe00640 1 --2- 172.21.15.77:0/3441754284 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7facac0a2c00 0x7facac0a50c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7facc80ffd30 tx=0x7facb0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:32.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:32.395+0000 7facbf400640 1 -- 172.21.15.77:0/3441754284 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7facb809bbc0 con 0x7facc8103e30 2024-09-10T09:55:32.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:55:31 smithi120 ceph-mon[34770]: from='client.15309 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T09:55:32.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:55:31 smithi120 ceph-mon[34770]: pgmap v402: 129 pgs: 129 active+clean; 652 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-10T09:55:32.602 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:32.601+0000 7faccca00640 1 -- 172.21.15.77:0/3441754284 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7facc8104210 con 0x7facc8103e30 2024-09-10T09:55:32.602 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:32.602+0000 7facbf400640 1 -- 172.21.15.77:0/3441754284 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11989 (secure 0 0 0) 0x7facb809fec0 con 0x7facc8103e30 2024-09-10T09:55:32.603 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:55:32.603 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":29,"btime":"2024-09-10T09:55:13:863715+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3159289675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3159289675},{"type":"v1","addr":"172.21.15.77:6835","nonce":3159289675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1951962341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1951962341},{"type":"v1","addr":"172.21.15.77:6837","nonce":1951962341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:52:23.368252+0000","modified":"2024-09-10T09:55:13.786975+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24563,"mds_1":24524,"mds_2":24520,"mds_3":24502,"mds_4":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14487":{"gid":14487,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/4291121888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":4291121888},{"type":"v1","addr":"172.21.15.77:6839","nonce":4291121888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14499":{"gid":14499,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6841/445284194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":445284194},{"type":"v1","addr":"172.21.15.77:6841","nonce":445284194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1005054423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1005054423},{"type":"v1","addr":"172.21.15.134:6833","nonce":1005054423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6833/3716307744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":3716307744},{"type":"v1","addr":"172.21.15.120:6833","nonce":3716307744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/1319892544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1319892544},{"type":"v1","addr":"172.21.15.120:6835","nonce":1319892544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.134:6835/3293933699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3293933699},{"type":"v1","addr":"172.21.15.134:6835","nonce":3293933699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.134:6837/2461497091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2461497091},{"type":"v1","addr":"172.21.15.134:6837","nonce":2461497091}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":20,"addr":"172.21.15.134:6839/1965117680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1965117680},{"type":"v1","addr":"172.21.15.134:6839","nonce":1965117680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":20,"addr":"172.21.15.120:6837/3033280709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3033280709},{"type":"v1","addr":"172.21.15.120:6837","nonce":3033280709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.120:6839/1777710157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1777710157},{"type":"v1","addr":"172.21.15.120:6839","nonce":1777710157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24563,"qdb_cluster":[24496,24502,24520,24524,24563]},"id":1}]} 2024-09-10T09:55:32.603 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 29 2024-09-10T09:55:32.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:32.604+0000 7faccca00640 1 -- 172.21.15.77:0/3441754284 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7facac0a2c00 msgr2=0x7facac0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:32.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:32.604+0000 7faccca00640 1 --2- 172.21.15.77:0/3441754284 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7facac0a2c00 0x7facac0a50c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7facc80ffd30 tx=0x7facb0005f90 comp rx=0 tx=0).stop 2024-09-10T09:55:32.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:32.604+0000 7faccca00640 1 -- 172.21.15.77:0/3441754284 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7facc8103e30 msgr2=0x7facc819a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:32.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:32.604+0000 7faccca00640 1 --2- 172.21.15.77:0/3441754284 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7facc8103e30 0x7facc819a400 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7facb800e9f0 tx=0x7facb800eeb0 comp rx=0 tx=0).stop 2024-09-10T09:55:32.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:32.604+0000 7faccca00640 1 -- 172.21.15.77:0/3441754284 shutdown_connections 2024-09-10T09:55:32.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:32.604+0000 7faccca00640 1 --2- 172.21.15.77:0/3441754284 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7facac0a2c00 0x7facac0a50c0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:32.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:32.604+0000 7faccca00640 1 --2- 172.21.15.77:0/3441754284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7facc8108f60 0x7facc819ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:32.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:32.604+0000 7faccca00640 1 --2- 172.21.15.77:0/3441754284 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7facc81047e0 0x7facc819a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:32.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:32.604+0000 7faccca00640 1 --2- 172.21.15.77:0/3441754284 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7facc8103e30 0x7facc819a400 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:32.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:32.604+0000 7faccca00640 1 -- 172.21.15.77:0/3441754284 >> 172.21.15.77:0/3441754284 conn(0x7facc80ff910 msgr2=0x7facc8100c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:32.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:32.605+0000 7faccca00640 1 -- 172.21.15.77:0/3441754284 shutdown_connections 2024-09-10T09:55:32.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:32.605+0000 7faccca00640 1 -- 172.21.15.77:0/3441754284 wait complete. 2024-09-10T09:55:33.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:55:32 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/3441754284' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:55:33.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:55:32 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/3441754284' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:55:33.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:55:32 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/3441754284' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:55:33.468 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:55:33.836 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:55:34.207 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:55:33 smithi077 ceph-mon[30319]: pgmap v403: 129 pgs: 129 active+clean; 652 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-10T09:55:34.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:55:33 smithi134 ceph-mon[35674]: pgmap v403: 129 pgs: 129 active+clean; 652 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-10T09:55:34.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:55:33 smithi120 ceph-mon[34770]: pgmap v403: 129 pgs: 129 active+clean; 652 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-10T09:55:34.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:34.452+0000 7f8d28200640 1 -- 172.21.15.77:0/3718909887 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8d201027a0 msgr2=0x7f8d201067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:34.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:34.452+0000 7f8d28200640 1 --2- 172.21.15.77:0/3718909887 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8d201027a0 0x7f8d201067f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c009900 tx=0x7f8d0c02a410 comp rx=0 tx=0).stop 2024-09-10T09:55:34.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:34.452+0000 7f8d28200640 1 -- 172.21.15.77:0/3718909887 shutdown_connections 2024-09-10T09:55:34.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:34.452+0000 7f8d28200640 1 --2- 172.21.15.77:0/3718909887 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d20106f20 0x7f8d2010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:34.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:34.452+0000 7f8d28200640 1 --2- 172.21.15.77:0/3718909887 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8d201027a0 0x7f8d201067f0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:34.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:34.452+0000 7f8d28200640 1 --2- 172.21.15.77:0/3718909887 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d20101df0 0x7f8d201021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:34.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:34.452+0000 7f8d28200640 1 -- 172.21.15.77:0/3718909887 >> 172.21.15.77:0/3718909887 conn(0x7f8d200fd8b0 msgr2=0x7f8d200ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:34.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:34.452+0000 7f8d28200640 1 -- 172.21.15.77:0/3718909887 shutdown_connections 2024-09-10T09:55:34.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:34.452+0000 7f8d28200640 1 -- 172.21.15.77:0/3718909887 wait complete. 2024-09-10T09:55:34.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:34.453+0000 7f8d28200640 1 Processor -- start 2024-09-10T09:55:34.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:34.453+0000 7f8d28200640 1 -- start start 2024-09-10T09:55:34.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:34.454+0000 7f8d28200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8d20101df0 0x7f8d20198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:34.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:34.454+0000 7f8d28200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d201027a0 0x7f8d20198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:34.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:34.454+0000 7f8d28200640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d20106f20 0x7f8d2019cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:34.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:34.454+0000 7f8d28200640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8d20111fd0 con 0x7f8d20101df0 2024-09-10T09:55:34.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:34.454+0000 7f8d28200640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8d20111e50 con 0x7f8d20106f20 2024-09-10T09:55:34.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:34.454+0000 7f8d28200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8d20112150 con 0x7f8d201027a0 2024-09-10T09:55:34.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:34.454+0000 7f8d25800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8d20101df0 0x7f8d20198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:34.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:34.454+0000 7f8d25800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8d20101df0 0x7f8d20198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:44898/0 (socket says 172.21.15.77:44898) 2024-09-10T09:55:34.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:34.454+0000 7f8d25800640 1 -- 172.21.15.77:0/2296242133 learned_addr learned my addr 172.21.15.77:0/2296242133 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:55:34.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:34.454+0000 7f8d26200640 1 --2- 172.21.15.77:0/2296242133 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d20106f20 0x7f8d2019cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:34.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:34.454+0000 7f8d24e00640 1 --2- 172.21.15.77:0/2296242133 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d201027a0 0x7f8d20198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:34.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:34.454+0000 7f8d25800640 1 -- 172.21.15.77:0/2296242133 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d201027a0 msgr2=0x7f8d20198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:34.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:34.454+0000 7f8d25800640 1 --2- 172.21.15.77:0/2296242133 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d201027a0 0x7f8d20198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:34.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:34.454+0000 7f8d25800640 1 -- 172.21.15.77:0/2296242133 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d20106f20 msgr2=0x7f8d2019cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:34.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:34.454+0000 7f8d25800640 1 --2- 172.21.15.77:0/2296242133 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d20106f20 0x7f8d2019cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:34.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:34.454+0000 7f8d25800640 1 -- 172.21.15.77:0/2296242133 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d2019d400 con 0x7f8d20101df0 2024-09-10T09:55:34.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:34.455+0000 7f8d25800640 1 --2- 172.21.15.77:0/2296242133 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8d20101df0 0x7f8d20198450 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f8d1400d950 tx=0x7f8d1400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:34.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:34.455+0000 7f8d1e000640 1 -- 172.21.15.77:0/2296242133 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d14017070 con 0x7f8d20101df0 2024-09-10T09:55:34.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:34.455+0000 7f8d1e000640 1 -- 172.21.15.77:0/2296242133 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d14009c30 con 0x7f8d20101df0 2024-09-10T09:55:34.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:34.455+0000 7f8d28200640 1 -- 172.21.15.77:0/2296242133 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d2019d6f0 con 0x7f8d20101df0 2024-09-10T09:55:34.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:34.455+0000 7f8d1e000640 1 -- 172.21.15.77:0/2296242133 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d1400f6b0 con 0x7f8d20101df0 2024-09-10T09:55:34.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:34.455+0000 7f8d28200640 1 -- 172.21.15.77:0/2296242133 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d201a4fd0 con 0x7f8d20101df0 2024-09-10T09:55:34.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:34.456+0000 7f8d28200640 1 -- 172.21.15.77:0/2296242133 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d201021d0 con 0x7f8d20101df0 2024-09-10T09:55:34.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:34.458+0000 7f8d1e000640 1 -- 172.21.15.77:0/2296242133 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8d140109d0 con 0x7f8d20101df0 2024-09-10T09:55:34.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:34.460+0000 7f8d1e000640 1 --2- 172.21.15.77:0/2296242133 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f8d080a2b80 0x7f8d080a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:34.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:34.460+0000 7f8d1e000640 1 -- 172.21.15.77:0/2296242133 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f8d140ceff0 con 0x7f8d20101df0 2024-09-10T09:55:34.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:34.461+0000 7f8d24e00640 1 --2- 172.21.15.77:0/2296242133 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f8d080a2b80 0x7f8d080a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:34.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:34.461+0000 7f8d24e00640 1 --2- 172.21.15.77:0/2296242133 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f8d080a2b80 0x7f8d080a5040 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c000c00 tx=0x7f8d0c005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:34.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:34.465+0000 7f8d1e000640 1 -- 172.21.15.77:0/2296242133 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8d14009de0 con 0x7f8d20101df0 2024-09-10T09:55:34.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:34.674+0000 7f8d28200640 1 -- 172.21.15.77:0/2296242133 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8d2010a160 con 0x7f8d20101df0 2024-09-10T09:55:34.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:34.675+0000 7f8d1e000640 1 -- 172.21.15.77:0/2296242133 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11989 (secure 0 0 0) 0x7f8d1409b4f0 con 0x7f8d20101df0 2024-09-10T09:55:34.676 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:55:34.677 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":29,"btime":"2024-09-10T09:55:13:863715+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3159289675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3159289675},{"type":"v1","addr":"172.21.15.77:6835","nonce":3159289675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1951962341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1951962341},{"type":"v1","addr":"172.21.15.77:6837","nonce":1951962341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:52:23.368252+0000","modified":"2024-09-10T09:55:13.786975+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24563,"mds_1":24524,"mds_2":24520,"mds_3":24502,"mds_4":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14487":{"gid":14487,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/4291121888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":4291121888},{"type":"v1","addr":"172.21.15.77:6839","nonce":4291121888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14499":{"gid":14499,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6841/445284194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":445284194},{"type":"v1","addr":"172.21.15.77:6841","nonce":445284194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1005054423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1005054423},{"type":"v1","addr":"172.21.15.134:6833","nonce":1005054423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6833/3716307744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":3716307744},{"type":"v1","addr":"172.21.15.120:6833","nonce":3716307744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/1319892544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1319892544},{"type":"v1","addr":"172.21.15.120:6835","nonce":1319892544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.134:6835/3293933699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3293933699},{"type":"v1","addr":"172.21.15.134:6835","nonce":3293933699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.134:6837/2461497091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2461497091},{"type":"v1","addr":"172.21.15.134:6837","nonce":2461497091}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":20,"addr":"172.21.15.134:6839/1965117680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1965117680},{"type":"v1","addr":"172.21.15.134:6839","nonce":1965117680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":20,"addr":"172.21.15.120:6837/3033280709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3033280709},{"type":"v1","addr":"172.21.15.120:6837","nonce":3033280709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.120:6839/1777710157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1777710157},{"type":"v1","addr":"172.21.15.120:6839","nonce":1777710157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24563,"qdb_cluster":[24496,24502,24520,24524,24563]},"id":1}]} 2024-09-10T09:55:34.677 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 29 2024-09-10T09:55:34.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:34.677+0000 7f8d28200640 1 -- 172.21.15.77:0/2296242133 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f8d080a2b80 msgr2=0x7f8d080a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:34.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:34.677+0000 7f8d28200640 1 --2- 172.21.15.77:0/2296242133 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f8d080a2b80 0x7f8d080a5040 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c000c00 tx=0x7f8d0c005b50 comp rx=0 tx=0).stop 2024-09-10T09:55:34.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:34.677+0000 7f8d28200640 1 -- 172.21.15.77:0/2296242133 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8d20101df0 msgr2=0x7f8d20198450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:34.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:34.677+0000 7f8d28200640 1 --2- 172.21.15.77:0/2296242133 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8d20101df0 0x7f8d20198450 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f8d1400d950 tx=0x7f8d1400de10 comp rx=0 tx=0).stop 2024-09-10T09:55:34.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:34.677+0000 7f8d28200640 1 -- 172.21.15.77:0/2296242133 shutdown_connections 2024-09-10T09:55:34.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:34.677+0000 7f8d28200640 1 --2- 172.21.15.77:0/2296242133 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f8d080a2b80 0x7f8d080a5040 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:34.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:34.677+0000 7f8d28200640 1 --2- 172.21.15.77:0/2296242133 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d20106f20 0x7f8d2019cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:34.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:34.677+0000 7f8d28200640 1 --2- 172.21.15.77:0/2296242133 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d201027a0 0x7f8d20198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:34.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:34.677+0000 7f8d28200640 1 --2- 172.21.15.77:0/2296242133 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8d20101df0 0x7f8d20198450 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:34.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:34.677+0000 7f8d28200640 1 -- 172.21.15.77:0/2296242133 >> 172.21.15.77:0/2296242133 conn(0x7f8d200fd8b0 msgr2=0x7f8d200feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:34.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:34.678+0000 7f8d28200640 1 -- 172.21.15.77:0/2296242133 shutdown_connections 2024-09-10T09:55:34.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:34.678+0000 7f8d28200640 1 -- 172.21.15.77:0/2296242133 wait complete. 2024-09-10T09:55:35.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:55:34 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/2296242133' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:55:35.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:55:34 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/2296242133' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:55:35.440 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:55:34 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/2296242133' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:55:35.663 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T09:55:36.033 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:55:36.325 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:55:35 smithi077 ceph-mon[30319]: scrub summary: idle 2024-09-10T09:55:36.325 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:55:35 smithi077 ceph-mon[30319]: pgmap v404: 129 pgs: 129 active+clean; 652 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-10T09:55:36.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:55:35 smithi134 ceph-mon[35674]: scrub summary: idle 2024-09-10T09:55:36.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:55:35 smithi134 ceph-mon[35674]: pgmap v404: 129 pgs: 129 active+clean; 652 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-10T09:55:36.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:55:35 smithi120 ceph-mon[34770]: scrub summary: idle 2024-09-10T09:55:36.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:55:35 smithi120 ceph-mon[34770]: pgmap v404: 129 pgs: 129 active+clean; 652 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-10T09:55:36.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:36.609+0000 7f19bb000640 1 -- 172.21.15.77:0/2496377349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19b4101de0 msgr2=0x7f19b41021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:36.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:36.609+0000 7f19bb000640 1 --2- 172.21.15.77:0/2496377349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19b4101de0 0x7f19b41021c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f19a4009860 tx=0x7f19a402a340 comp rx=0 tx=0).stop 2024-09-10T09:55:36.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:36.609+0000 7f19bb000640 1 -- 172.21.15.77:0/2496377349 shutdown_connections 2024-09-10T09:55:36.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:36.609+0000 7f19bb000640 1 --2- 172.21.15.77:0/2496377349 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f19b4106f10 0x7f19b410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:36.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:36.609+0000 7f19bb000640 1 --2- 172.21.15.77:0/2496377349 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f19b4102790 0x7f19b41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:36.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:36.609+0000 7f19bb000640 1 --2- 172.21.15.77:0/2496377349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19b4101de0 0x7f19b41021c0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:36.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:36.609+0000 7f19bb000640 1 -- 172.21.15.77:0/2496377349 >> 172.21.15.77:0/2496377349 conn(0x7f19b40fd8c0 msgr2=0x7f19b40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:36.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:36.609+0000 7f19bb000640 1 -- 172.21.15.77:0/2496377349 shutdown_connections 2024-09-10T09:55:36.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:36.609+0000 7f19bb000640 1 -- 172.21.15.77:0/2496377349 wait complete. 2024-09-10T09:55:36.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:36.610+0000 7f19bb000640 1 Processor -- start 2024-09-10T09:55:36.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:36.610+0000 7f19bb000640 1 -- start start 2024-09-10T09:55:36.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:36.611+0000 7f19bb000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f19b4101de0 0x7f19b4198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:36.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:36.611+0000 7f19bb000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f19b4102790 0x7f19b41989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:36.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:36.611+0000 7f19bb000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19b4106f10 0x7f19b419cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:36.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:36.611+0000 7f19bb000640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f19b4111f70 con 0x7f19b4101de0 2024-09-10T09:55:36.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:36.611+0000 7f19bb000640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f19b4111df0 con 0x7f19b4102790 2024-09-10T09:55:36.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:36.611+0000 7f19bb000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f19b41120f0 con 0x7f19b4106f10 2024-09-10T09:55:36.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:36.611+0000 7f19b3e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f19b4101de0 0x7f19b4198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:36.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:36.611+0000 7f19ab400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f19b4102790 0x7f19b41989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:36.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:36.611+0000 7f19b3e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f19b4101de0 0x7f19b4198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:44914/0 (socket says 172.21.15.77:44914) 2024-09-10T09:55:36.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:36.611+0000 7f19b3e00640 1 -- 172.21.15.77:0/3817040679 learned_addr learned my addr 172.21.15.77:0/3817040679 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:55:36.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:36.611+0000 7f19b9000640 1 --2- 172.21.15.77:0/3817040679 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19b4106f10 0x7f19b419cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:36.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:36.611+0000 7f19b3e00640 1 -- 172.21.15.77:0/3817040679 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19b4106f10 msgr2=0x7f19b419cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:36.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:36.611+0000 7f19b3e00640 1 --2- 172.21.15.77:0/3817040679 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19b4106f10 0x7f19b419cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:36.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:36.611+0000 7f19b3e00640 1 -- 172.21.15.77:0/3817040679 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f19b4102790 msgr2=0x7f19b41989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:36.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:36.611+0000 7f19b3e00640 1 --2- 172.21.15.77:0/3817040679 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f19b4102790 0x7f19b41989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:36.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:36.611+0000 7f19b3e00640 1 -- 172.21.15.77:0/3817040679 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19b419d440 con 0x7f19b4101de0 2024-09-10T09:55:36.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:36.611+0000 7f19ab400640 1 --2- 172.21.15.77:0/3817040679 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f19b4102790 0x7f19b41989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:55:36.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:36.612+0000 7f19b3e00640 1 --2- 172.21.15.77:0/3817040679 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f19b4101de0 0x7f19b4198490 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f19a4035040 tx=0x7f19a4033b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:36.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:36.612+0000 7f19b1600640 1 -- 172.21.15.77:0/3817040679 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a4037040 con 0x7f19b4101de0 2024-09-10T09:55:36.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:36.612+0000 7f19b1600640 1 -- 172.21.15.77:0/3817040679 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19a4040070 con 0x7f19b4101de0 2024-09-10T09:55:36.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:36.612+0000 7f19b1600640 1 -- 172.21.15.77:0/3817040679 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a40039b0 con 0x7f19b4101de0 2024-09-10T09:55:36.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:36.612+0000 7f19bb000640 1 -- 172.21.15.77:0/3817040679 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19b419d6d0 con 0x7f19b4101de0 2024-09-10T09:55:36.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:36.612+0000 7f19bb000640 1 -- 172.21.15.77:0/3817040679 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19b419db60 con 0x7f19b4101de0 2024-09-10T09:55:36.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:36.613+0000 7f19bb000640 1 -- 172.21.15.77:0/3817040679 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19b41021c0 con 0x7f19b4101de0 2024-09-10T09:55:36.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:36.615+0000 7f19b1600640 1 -- 172.21.15.77:0/3817040679 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f19a40305e0 con 0x7f19b4101de0 2024-09-10T09:55:36.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:36.616+0000 7f19b1600640 1 --2- 172.21.15.77:0/3817040679 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f19980a2c00 0x7f19980a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:36.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:36.617+0000 7f19b1600640 1 -- 172.21.15.77:0/3817040679 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f19a40ef3a0 con 0x7f19b4101de0 2024-09-10T09:55:36.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:36.617+0000 7f19ab400640 1 --2- 172.21.15.77:0/3817040679 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f19980a2c00 0x7f19980a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:36.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:36.618+0000 7f19ab400640 1 --2- 172.21.15.77:0/3817040679 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f19980a2c00 0x7f19980a50c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f199c0096c0 tx=0x7f199c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:36.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:36.621+0000 7f19b1600640 1 -- 172.21.15.77:0/3817040679 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f19a40bbeb0 con 0x7f19b4101de0 2024-09-10T09:55:36.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:36.799+0000 7f19bb000640 1 -- 172.21.15.77:0/3817040679 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f19b4109b60 con 0x7f19b4101de0 2024-09-10T09:55:36.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:36.800+0000 7f19b1600640 1 -- 172.21.15.77:0/3817040679 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+24941 (secure 0 0 0) 0x7f19a40bb7b0 con 0x7f19b4101de0 2024-09-10T09:55:36.801 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:55:36.802 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":96,"fsid":"14a55226-6f59-11ef-bcea-c7b262605968","created":"2024-09-10T09:44:07.359542+0000","modified":"2024-09-10T09:54:00.837074+0000","last_up_change":"2024-09-10T09:51:26.731971+0000","last_in_change":"2024-09-10T09:51:03.857873+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:47:24.174660+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T09:52:18.766738+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T09:52:21.284745+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T09:52:29.846594+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8d46f634-114c-4926-afdc-f8fa37e40336","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":1509649079},{"type":"v1","addr":"172.21.15.77:6803","nonce":1509649079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":1509649079},{"type":"v1","addr":"172.21.15.77:6805","nonce":1509649079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":1509649079},{"type":"v1","addr":"172.21.15.77:6809","nonce":1509649079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":1509649079},{"type":"v1","addr":"172.21.15.77:6807","nonce":1509649079}]},"public_addr":"172.21.15.77:6803/1509649079","cluster_addr":"172.21.15.77:6805/1509649079","heartbeat_back_addr":"172.21.15.77:6809/1509649079","heartbeat_front_addr":"172.21.15.77:6807/1509649079","state":["exists","up"]},{"osd":1,"uuid":"b6570764-736a-49e4-877a-1965e5812d1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":4243920698},{"type":"v1","addr":"172.21.15.120:6801","nonce":4243920698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":4243920698},{"type":"v1","addr":"172.21.15.120:6803","nonce":4243920698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":4243920698},{"type":"v1","addr":"172.21.15.120:6807","nonce":4243920698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":4243920698},{"type":"v1","addr":"172.21.15.120:6805","nonce":4243920698}]},"public_addr":"172.21.15.120:6801/4243920698","cluster_addr":"172.21.15.120:6803/4243920698","heartbeat_back_addr":"172.21.15.120:6807/4243920698","heartbeat_front_addr":"172.21.15.120:6805/4243920698","state":["exists","up"]},{"osd":2,"uuid":"5e83fe13-d1ee-483e-a575-e8041ec836f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":3280466823},{"type":"v1","addr":"172.21.15.134:6801","nonce":3280466823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":3280466823},{"type":"v1","addr":"172.21.15.134:6803","nonce":3280466823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":3280466823},{"type":"v1","addr":"172.21.15.134:6807","nonce":3280466823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":3280466823},{"type":"v1","addr":"172.21.15.134:6805","nonce":3280466823}]},"public_addr":"172.21.15.134:6801/3280466823","cluster_addr":"172.21.15.134:6803/3280466823","heartbeat_back_addr":"172.21.15.134:6807/3280466823","heartbeat_front_addr":"172.21.15.134:6805/3280466823","state":["exists","up"]},{"osd":3,"uuid":"5affc6a7-1cae-48fe-b7db-befc2fcafecb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":1629416786},{"type":"v1","addr":"172.21.15.77:6811","nonce":1629416786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":1629416786},{"type":"v1","addr":"172.21.15.77:6813","nonce":1629416786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":1629416786},{"type":"v1","addr":"172.21.15.77:6817","nonce":1629416786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":1629416786},{"type":"v1","addr":"172.21.15.77:6815","nonce":1629416786}]},"public_addr":"172.21.15.77:6811/1629416786","cluster_addr":"172.21.15.77:6813/1629416786","heartbeat_back_addr":"172.21.15.77:6817/1629416786","heartbeat_front_addr":"172.21.15.77:6815/1629416786","state":["exists","up"]},{"osd":4,"uuid":"c826978f-40b0-43ae-adbb-5c93b32a68ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":3162658038},{"type":"v1","addr":"172.21.15.120:6809","nonce":3162658038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":3162658038},{"type":"v1","addr":"172.21.15.120:6811","nonce":3162658038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":3162658038},{"type":"v1","addr":"172.21.15.120:6815","nonce":3162658038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":3162658038},{"type":"v1","addr":"172.21.15.120:6813","nonce":3162658038}]},"public_addr":"172.21.15.120:6809/3162658038","cluster_addr":"172.21.15.120:6811/3162658038","heartbeat_back_addr":"172.21.15.120:6815/3162658038","heartbeat_front_addr":"172.21.15.120:6813/3162658038","state":["exists","up"]},{"osd":5,"uuid":"be51c7fb-2765-46a3-9861-c950f63de148","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":4081444621},{"type":"v1","addr":"172.21.15.134:6809","nonce":4081444621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":4081444621},{"type":"v1","addr":"172.21.15.134:6811","nonce":4081444621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":4081444621},{"type":"v1","addr":"172.21.15.134:6815","nonce":4081444621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":4081444621},{"type":"v1","addr":"172.21.15.134:6813","nonce":4081444621}]},"public_addr":"172.21.15.134:6809/4081444621","cluster_addr":"172.21.15.134:6811/4081444621","heartbeat_back_addr":"172.21.15.134:6815/4081444621","heartbeat_front_addr":"172.21.15.134:6813/4081444621","state":["exists","up"]},{"osd":6,"uuid":"4eb403ab-473b-41cc-803d-18c908f768a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":879565639},{"type":"v1","addr":"172.21.15.77:6819","nonce":879565639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":879565639},{"type":"v1","addr":"172.21.15.77:6821","nonce":879565639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":879565639},{"type":"v1","addr":"172.21.15.77:6825","nonce":879565639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":879565639},{"type":"v1","addr":"172.21.15.77:6823","nonce":879565639}]},"public_addr":"172.21.15.77:6819/879565639","cluster_addr":"172.21.15.77:6821/879565639","heartbeat_back_addr":"172.21.15.77:6825/879565639","heartbeat_front_addr":"172.21.15.77:6823/879565639","state":["exists","up"]},{"osd":7,"uuid":"ba982bd0-c236-456c-9889-61087d79c1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":2688667400},{"type":"v1","addr":"172.21.15.120:6817","nonce":2688667400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":2688667400},{"type":"v1","addr":"172.21.15.120:6819","nonce":2688667400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":2688667400},{"type":"v1","addr":"172.21.15.120:6823","nonce":2688667400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2688667400},{"type":"v1","addr":"172.21.15.120:6821","nonce":2688667400}]},"public_addr":"172.21.15.120:6817/2688667400","cluster_addr":"172.21.15.120:6819/2688667400","heartbeat_back_addr":"172.21.15.120:6823/2688667400","heartbeat_front_addr":"172.21.15.120:6821/2688667400","state":["exists","up"]},{"osd":8,"uuid":"f2e8ca43-57c9-4933-b5f7-73f4d1b3e08b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2081686544},{"type":"v1","addr":"172.21.15.134:6817","nonce":2081686544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2081686544},{"type":"v1","addr":"172.21.15.134:6819","nonce":2081686544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2081686544},{"type":"v1","addr":"172.21.15.134:6823","nonce":2081686544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2081686544},{"type":"v1","addr":"172.21.15.134:6821","nonce":2081686544}]},"public_addr":"172.21.15.134:6817/2081686544","cluster_addr":"172.21.15.134:6819/2081686544","heartbeat_back_addr":"172.21.15.134:6823/2081686544","heartbeat_front_addr":"172.21.15.134:6821/2081686544","state":["exists","up"]},{"osd":9,"uuid":"9486f8b5-0614-4208-87ec-117741f6b622","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":1210619387},{"type":"v1","addr":"172.21.15.77:6827","nonce":1210619387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":1210619387},{"type":"v1","addr":"172.21.15.77:6829","nonce":1210619387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":1210619387},{"type":"v1","addr":"172.21.15.77:6833","nonce":1210619387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":1210619387},{"type":"v1","addr":"172.21.15.77:6831","nonce":1210619387}]},"public_addr":"172.21.15.77:6827/1210619387","cluster_addr":"172.21.15.77:6829/1210619387","heartbeat_back_addr":"172.21.15.77:6833/1210619387","heartbeat_front_addr":"172.21.15.77:6831/1210619387","state":["exists","up"]},{"osd":10,"uuid":"8e8192b0-1398-4d4d-9756-762b2f874d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":754733885},{"type":"v1","addr":"172.21.15.120:6825","nonce":754733885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":754733885},{"type":"v1","addr":"172.21.15.120:6827","nonce":754733885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":754733885},{"type":"v1","addr":"172.21.15.120:6831","nonce":754733885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":754733885},{"type":"v1","addr":"172.21.15.120:6829","nonce":754733885}]},"public_addr":"172.21.15.120:6825/754733885","cluster_addr":"172.21.15.120:6827/754733885","heartbeat_back_addr":"172.21.15.120:6831/754733885","heartbeat_front_addr":"172.21.15.120:6829/754733885","state":["exists","up"]},{"osd":11,"uuid":"85c1ccd4-5147-4995-896e-fe977f73f749","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1602213143},{"type":"v1","addr":"172.21.15.134:6825","nonce":1602213143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1602213143},{"type":"v1","addr":"172.21.15.134:6827","nonce":1602213143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1602213143},{"type":"v1","addr":"172.21.15.134:6831","nonce":1602213143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1602213143},{"type":"v1","addr":"172.21.15.134:6829","nonce":1602213143}]},"public_addr":"172.21.15.134:6825/1602213143","cluster_addr":"172.21.15.134:6827/1602213143","heartbeat_back_addr":"172.21.15.134:6831/1602213143","heartbeat_front_addr":"172.21.15.134:6829/1602213143","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:46:30.293451+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:46:56.554801+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:47:21.476225+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:47:49.737332+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:48:16.092181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:48:41.240815+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:49:11.683950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:49:38.859255+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:50:04.105401+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:50:33.847810+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:50:59.901797+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:51:25.241507+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.2","mappings":[{"from":7,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":7}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":5,"to":8}]},{"pgid":"4.6","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.77:0/1697982655":"2024-09-11T09:44:59.575663+0000","172.21.15.77:6801/1527070442":"2024-09-11T09:44:59.575663+0000","172.21.15.77:0/3146688196":"2024-09-11T09:44:32.185342+0000","172.21.15.77:0/371773177":"2024-09-11T09:44:59.575663+0000","172.21.15.77:0/1925456910":"2024-09-11T09:44:59.575663+0000","172.21.15.77:0/138545832":"2024-09-11T09:44:32.185342+0000","172.21.15.77:6801/3235900503":"2024-09-11T09:44:32.185342+0000","172.21.15.77:6800/1527070442":"2024-09-11T09:44:59.575663+0000","172.21.15.77:6800/3235900503":"2024-09-11T09:44:32.185342+0000","172.21.15.77:0/1647092548":"2024-09-11T09:44:32.185342+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T09:55:36.804 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:36.803+0000 7f19bb000640 1 -- 172.21.15.77:0/3817040679 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f19980a2c00 msgr2=0x7f19980a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:36.804 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:36.803+0000 7f19bb000640 1 --2- 172.21.15.77:0/3817040679 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f19980a2c00 0x7f19980a50c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f199c0096c0 tx=0x7f199c009260 comp rx=0 tx=0).stop 2024-09-10T09:55:36.804 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:36.803+0000 7f19bb000640 1 -- 172.21.15.77:0/3817040679 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f19b4101de0 msgr2=0x7f19b4198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:36.804 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:36.803+0000 7f19bb000640 1 --2- 172.21.15.77:0/3817040679 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f19b4101de0 0x7f19b4198490 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f19a4035040 tx=0x7f19a4033b60 comp rx=0 tx=0).stop 2024-09-10T09:55:36.804 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:36.803+0000 7f19bb000640 1 -- 172.21.15.77:0/3817040679 shutdown_connections 2024-09-10T09:55:36.804 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:36.803+0000 7f19bb000640 1 --2- 172.21.15.77:0/3817040679 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f19980a2c00 0x7f19980a50c0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:36.804 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:36.803+0000 7f19bb000640 1 --2- 172.21.15.77:0/3817040679 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19b4106f10 0x7f19b419cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:36.804 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:36.803+0000 7f19bb000640 1 --2- 172.21.15.77:0/3817040679 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f19b4102790 0x7f19b41989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:36.805 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:36.803+0000 7f19bb000640 1 --2- 172.21.15.77:0/3817040679 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f19b4101de0 0x7f19b4198490 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:36.805 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:36.803+0000 7f19bb000640 1 -- 172.21.15.77:0/3817040679 >> 172.21.15.77:0/3817040679 conn(0x7f19b40fd8c0 msgr2=0x7f19b40febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:36.805 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:36.803+0000 7f19bb000640 1 -- 172.21.15.77:0/3817040679 shutdown_connections 2024-09-10T09:55:36.805 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:36.803+0000 7f19bb000640 1 -- 172.21.15.77:0/3817040679 wait complete. 2024-09-10T09:55:37.206 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:55:36 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/3817040679' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T09:55:37.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:55:36 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/3817040679' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T09:55:37.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:55:36 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/3817040679' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T09:55:37.592 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-09-10T09:55:37.966 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:55:38.252 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:55:37 smithi077 ceph-mon[30319]: pgmap v405: 129 pgs: 129 active+clean; 652 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-10T09:55:38.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:55:37 smithi134 ceph-mon[35674]: pgmap v405: 129 pgs: 129 active+clean; 652 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-10T09:55:38.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:55:37 smithi120 ceph-mon[34770]: pgmap v405: 129 pgs: 129 active+clean; 652 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-10T09:55:39.249 INFO:teuthology.orchestra.run.smithi077.stdout:{"run_dir":"/var/run/ceph"} 2024-09-10T09:55:39.249 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-09-10T09:55:39.249 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 9893 2024-09-10T09:55:39.250 DEBUG:tasks.quiescer:Yielding 2024-09-10T09:55:39.250 INFO:teuthology.run_tasks:Running task workunit... 2024-09-10T09:55:39.266 INFO:tasks.workunit:Pulling workunits from ref 1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T09:55:39.266 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-10T09:55:39.266 INFO:tasks.workunit:timeout=3h 2024-09-10T09:55:39.266 INFO:tasks.workunit:cleanup=True 2024-09-10T09:55:39.267 DEBUG:teuthology.orchestra.run.smithi077:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-10T09:55:39.267 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:55:39.303 INFO:teuthology.orchestra.run.smithi077.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-10T09:55:39.303 INFO:teuthology.orchestra.run.smithi077.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-09-10T09:55:39.303 INFO:teuthology.orchestra.run.smithi077.stdout:Device: 78h/120d Inode: 1099511628283 Links: 2 2024-09-10T09:55:39.303 INFO:teuthology.orchestra.run.smithi077.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T09:55:39.303 INFO:teuthology.orchestra.run.smithi077.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-09-10T09:55:39.303 INFO:teuthology.orchestra.run.smithi077.stdout:Access: 2024-09-10 09:53:25.478436919 +0000 2024-09-10T09:55:39.304 INFO:teuthology.orchestra.run.smithi077.stdout:Modify: 2024-09-10 09:53:25.478436919 +0000 2024-09-10T09:55:39.304 INFO:teuthology.orchestra.run.smithi077.stdout:Change: 2024-09-10 09:55:12.926583746 +0000 2024-09-10T09:55:39.304 INFO:teuthology.orchestra.run.smithi077.stdout: Birth: 2024-09-10 09:53:25.478436919 +0000 2024-09-10T09:55:39.304 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-10T09:55:39.304 DEBUG:teuthology.orchestra.run.smithi077:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-10T09:55:39.413 DEBUG:teuthology.orchestra.run.smithi077:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T09:55:39.451 INFO:tasks.workunit.client.0.smithi077.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-10T09:55:39.656 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:55:40.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:40.259+0000 7f7edc000640 1 -- 172.21.15.77:0/1203227946 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7ed41049c0 msgr2=0x7f7ed4108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:40.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:40.259+0000 7f7edc000640 1 --2- 172.21.15.77:0/1203227946 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7ed41049c0 0x7f7ed4108a10 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f7ec0009900 tx=0x7f7ec002a410 comp rx=0 tx=0).stop 2024-09-10T09:55:40.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:40.259+0000 7f7edc000640 1 -- 172.21.15.77:0/1203227946 shutdown_connections 2024-09-10T09:55:40.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:40.259+0000 7f7edc000640 1 --2- 172.21.15.77:0/1203227946 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ed4109140 0x7f7ed4110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:40.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:40.259+0000 7f7edc000640 1 --2- 172.21.15.77:0/1203227946 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7ed41049c0 0x7f7ed4108a10 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:40.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:40.259+0000 7f7edc000640 1 --2- 172.21.15.77:0/1203227946 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ed4104010 0x7f7ed41043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:40.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:40.259+0000 7f7edc000640 1 -- 172.21.15.77:0/1203227946 >> 172.21.15.77:0/1203227946 conn(0x7f7ed40ffa50 msgr2=0x7f7ed4101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:40.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:40.259+0000 7f7edc000640 1 -- 172.21.15.77:0/1203227946 shutdown_connections 2024-09-10T09:55:40.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:40.259+0000 7f7edc000640 1 -- 172.21.15.77:0/1203227946 wait complete. 2024-09-10T09:55:40.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:40.260+0000 7f7edc000640 1 Processor -- start 2024-09-10T09:55:40.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:40.260+0000 7f7edc000640 1 -- start start 2024-09-10T09:55:40.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:40.261+0000 7f7edc000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ed4104010 0x7f7ed419a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:40.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:40.261+0000 7f7edc000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ed41049c0 0x7f7ed419aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:40.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:40.261+0000 7f7edc000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7ed4109140 0x7f7ed419ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:40.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:40.261+0000 7f7edc000640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed4114160 con 0x7f7ed4109140 2024-09-10T09:55:40.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:40.261+0000 7f7edc000640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed4113fe0 con 0x7f7ed41049c0 2024-09-10T09:55:40.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:40.261+0000 7f7edc000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed41142e0 con 0x7f7ed4104010 2024-09-10T09:55:40.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:40.261+0000 7f7eda000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7ed4109140 0x7f7ed419ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:40.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:40.261+0000 7f7eda000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7ed4109140 0x7f7ed419ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:44938/0 (socket says 172.21.15.77:44938) 2024-09-10T09:55:40.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:40.261+0000 7f7eda000640 1 -- 172.21.15.77:0/3373557878 learned_addr learned my addr 172.21.15.77:0/3373557878 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:55:40.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:40.261+0000 7f7ed8c00640 1 --2- 172.21.15.77:0/3373557878 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ed41049c0 0x7f7ed419aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:40.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:40.261+0000 7f7ed9600640 1 --2- 172.21.15.77:0/3373557878 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ed4104010 0x7f7ed419a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:40.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:40.261+0000 7f7eda000640 1 -- 172.21.15.77:0/3373557878 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ed4104010 msgr2=0x7f7ed419a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:40.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:40.261+0000 7f7eda000640 1 --2- 172.21.15.77:0/3373557878 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ed4104010 0x7f7ed419a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:40.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:40.261+0000 7f7eda000640 1 -- 172.21.15.77:0/3373557878 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ed41049c0 msgr2=0x7f7ed419aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:40.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:40.261+0000 7f7eda000640 1 --2- 172.21.15.77:0/3373557878 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ed41049c0 0x7f7ed419aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:40.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:40.261+0000 7f7eda000640 1 -- 172.21.15.77:0/3373557878 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ed419f610 con 0x7f7ed4109140 2024-09-10T09:55:40.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:40.262+0000 7f7eda000640 1 --2- 172.21.15.77:0/3373557878 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7ed4109140 0x7f7ed419ef30 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc00eb10 tx=0x7f7ecc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:40.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:40.262+0000 7f7ed2000640 1 -- 172.21.15.77:0/3373557878 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ecc004b20 con 0x7f7ed4109140 2024-09-10T09:55:40.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:40.262+0000 7f7edc000640 1 -- 172.21.15.77:0/3373557878 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ed419f900 con 0x7f7ed4109140 2024-09-10T09:55:40.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:40.262+0000 7f7ed2000640 1 -- 172.21.15.77:0/3373557878 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ecc004cc0 con 0x7f7ed4109140 2024-09-10T09:55:40.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:40.262+0000 7f7ed2000640 1 -- 172.21.15.77:0/3373557878 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ecc010760 con 0x7f7ed4109140 2024-09-10T09:55:40.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:40.262+0000 7f7edc000640 1 -- 172.21.15.77:0/3373557878 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ed41a71e0 con 0x7f7ed4109140 2024-09-10T09:55:40.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:40.263+0000 7f7edc000640 1 -- 172.21.15.77:0/3373557878 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ed41043f0 con 0x7f7ed4109140 2024-09-10T09:55:40.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:40.265+0000 7f7ed2000640 1 -- 172.21.15.77:0/3373557878 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7ecc005020 con 0x7f7ed4109140 2024-09-10T09:55:40.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:40.266+0000 7f7ed2000640 1 --2- 172.21.15.77:0/3373557878 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f7ebc0a2b80 0x7f7ebc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:40.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:40.267+0000 7f7ed2000640 1 -- 172.21.15.77:0/3373557878 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f7ecc0cfad0 con 0x7f7ed4109140 2024-09-10T09:55:40.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:40.267+0000 7f7ed9600640 1 --2- 172.21.15.77:0/3373557878 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f7ebc0a2b80 0x7f7ebc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:40.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:40.267+0000 7f7ed9600640 1 --2- 172.21.15.77:0/3373557878 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f7ebc0a2b80 0x7f7ebc0a5040 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7ec8009bd0 tx=0x7f7ec8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:40.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:40.271+0000 7f7ed2000640 1 -- 172.21.15.77:0/3373557878 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7ecc09c5e0 con 0x7f7ed4109140 2024-09-10T09:55:40.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:40.483+0000 7f7edc000640 1 -- 172.21.15.77:0/3373557878 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7ed402d7c0 con 0x7f7ed4109140 2024-09-10T09:55:40.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:40.484+0000 7f7ed2000640 1 -- 172.21.15.77:0/3373557878 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11989 (secure 0 0 0) 0x7f7ecc09bee0 con 0x7f7ed4109140 2024-09-10T09:55:40.485 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:55:40.485 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":29,"btime":"2024-09-10T09:55:13:863715+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3159289675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3159289675},{"type":"v1","addr":"172.21.15.77:6835","nonce":3159289675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1951962341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1951962341},{"type":"v1","addr":"172.21.15.77:6837","nonce":1951962341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:52:23.368252+0000","modified":"2024-09-10T09:55:13.786975+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24563,"mds_1":24524,"mds_2":24520,"mds_3":24502,"mds_4":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14487":{"gid":14487,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/4291121888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":4291121888},{"type":"v1","addr":"172.21.15.77:6839","nonce":4291121888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14499":{"gid":14499,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6841/445284194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":445284194},{"type":"v1","addr":"172.21.15.77:6841","nonce":445284194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1005054423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1005054423},{"type":"v1","addr":"172.21.15.134:6833","nonce":1005054423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6833/3716307744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":3716307744},{"type":"v1","addr":"172.21.15.120:6833","nonce":3716307744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/1319892544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1319892544},{"type":"v1","addr":"172.21.15.120:6835","nonce":1319892544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.134:6835/3293933699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3293933699},{"type":"v1","addr":"172.21.15.134:6835","nonce":3293933699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.134:6837/2461497091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2461497091},{"type":"v1","addr":"172.21.15.134:6837","nonce":2461497091}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":20,"addr":"172.21.15.134:6839/1965117680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1965117680},{"type":"v1","addr":"172.21.15.134:6839","nonce":1965117680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":20,"addr":"172.21.15.120:6837/3033280709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3033280709},{"type":"v1","addr":"172.21.15.120:6837","nonce":3033280709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.120:6839/1777710157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1777710157},{"type":"v1","addr":"172.21.15.120:6839","nonce":1777710157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24563,"qdb_cluster":[24496,24502,24520,24524,24563]},"id":1}]} 2024-09-10T09:55:40.485 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 29 2024-09-10T09:55:40.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:40.486+0000 7f7edc000640 1 -- 172.21.15.77:0/3373557878 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f7ebc0a2b80 msgr2=0x7f7ebc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:40.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:40.486+0000 7f7edc000640 1 --2- 172.21.15.77:0/3373557878 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f7ebc0a2b80 0x7f7ebc0a5040 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7ec8009bd0 tx=0x7f7ec8005f90 comp rx=0 tx=0).stop 2024-09-10T09:55:40.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:40.486+0000 7f7edc000640 1 -- 172.21.15.77:0/3373557878 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7ed4109140 msgr2=0x7f7ed419ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:40.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:40.486+0000 7f7edc000640 1 --2- 172.21.15.77:0/3373557878 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7ed4109140 0x7f7ed419ef30 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc00eb10 tx=0x7f7ecc00c2d0 comp rx=0 tx=0).stop 2024-09-10T09:55:40.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:40.486+0000 7f7edc000640 1 -- 172.21.15.77:0/3373557878 shutdown_connections 2024-09-10T09:55:40.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:40.486+0000 7f7edc000640 1 --2- 172.21.15.77:0/3373557878 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f7ebc0a2b80 0x7f7ebc0a5040 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:40.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:40.486+0000 7f7edc000640 1 --2- 172.21.15.77:0/3373557878 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7ed4109140 0x7f7ed419ef30 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:40.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:40.486+0000 7f7edc000640 1 --2- 172.21.15.77:0/3373557878 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ed41049c0 0x7f7ed419aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:40.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:40.486+0000 7f7edc000640 1 --2- 172.21.15.77:0/3373557878 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ed4104010 0x7f7ed419a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:40.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:40.486+0000 7f7edc000640 1 -- 172.21.15.77:0/3373557878 >> 172.21.15.77:0/3373557878 conn(0x7f7ed40ffa50 msgr2=0x7f7ed4100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:40.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:40.486+0000 7f7edc000640 1 -- 172.21.15.77:0/3373557878 shutdown_connections 2024-09-10T09:55:40.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:40.486+0000 7f7edc000640 1 -- 172.21.15.77:0/3373557878 wait complete. 2024-09-10T09:55:41.089 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:55:40 smithi077 ceph-mon[30319]: pgmap v406: 129 pgs: 129 active+clean; 652 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 170 B/s wr, 12 op/s 2024-09-10T09:55:41.090 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:55:40 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/3373557878' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:55:41.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:55:40 smithi134 ceph-mon[35674]: pgmap v406: 129 pgs: 129 active+clean; 652 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 170 B/s wr, 12 op/s 2024-09-10T09:55:41.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:55:40 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/3373557878' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:55:41.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:55:40 smithi120 ceph-mon[34770]: pgmap v406: 129 pgs: 129 active+clean; 652 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 170 B/s wr, 12 op/s 2024-09-10T09:55:41.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:55:40 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/3373557878' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:55:41.386 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 29, 'flags': 306, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': True}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-10T09:52:23.368252+0000', 'modified': '2024-09-10T09:55:13.786975+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24563, 'mds_1': 24524, 'mds_2': 24520, 'mds_3': 24502, 'mds_4': 24496}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14487': {'gid': 14487, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.77:6839/4291121888', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.77:6838', 'nonce': 4291121888}, {'type': 'v1', 'addr': '172.21.15.77:6839', 'nonce': 4291121888}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14499': {'gid': 14499, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.77:6841/445284194', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.77:6840', 'nonce': 445284194}, {'type': 'v1', 'addr': '172.21.15.77:6841', 'nonce': 445284194}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24448': {'gid': 24448, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.134:6833/1005054423', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6832', 'nonce': 1005054423}, {'type': 'v1', 'addr': '172.21.15.134:6833', 'nonce': 1005054423}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24458': {'gid': 24458, 'name': 'k', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.120:6833/3716307744', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6832', 'nonce': 3716307744}, {'type': 'v1', 'addr': '172.21.15.120:6833', 'nonce': 3716307744}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24494': {'gid': 24494, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.120:6835/1319892544', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6834', 'nonce': 1319892544}, {'type': 'v1', 'addr': '172.21.15.120:6835', 'nonce': 1319892544}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24496': {'gid': 24496, 'name': 'f', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 22, 'addr': '172.21.15.134:6835/3293933699', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6834', 'nonce': 3293933699}, {'type': 'v1', 'addr': '172.21.15.134:6835', 'nonce': 3293933699}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24502': {'gid': 24502, 'name': 'c', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 21, 'addr': '172.21.15.134:6837/2461497091', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6836', 'nonce': 2461497091}, {'type': 'v1', 'addr': '172.21.15.134:6837', 'nonce': 2461497091}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24520': {'gid': 24520, 'name': 'i', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 20, 'addr': '172.21.15.134:6839/1965117680', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6838', 'nonce': 1965117680}, {'type': 'v1', 'addr': '172.21.15.134:6839', 'nonce': 1965117680}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24524': {'gid': 24524, 'name': 'h', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 20, 'addr': '172.21.15.120:6837/3033280709', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6836', 'nonce': 3033280709}, {'type': 'v1', 'addr': '172.21.15.120:6837', 'nonce': 3033280709}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24563': {'gid': 24563, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.120:6839/1777710157', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6838', 'nonce': 1777710157}, {'type': 'v1', 'addr': '172.21.15.120:6839', 'nonce': 1777710157}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24563, 'qdb_cluster': [24496, 24502, 24520, 24524, 24563]} 2024-09-10T09:55:41.386 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-10T09:55:41.386 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.f status 2024-09-10T09:55:41.807 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:55:42.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.380+0000 7fa5d7400640 1 -- 172.21.15.77:0/3287556687 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa5d0074f40 msgr2=0x7fa5d010de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:42.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.380+0000 7fa5d7400640 1 --2- 172.21.15.77:0/3287556687 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa5d0074f40 0x7fa5d010de80 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fa5b8009900 tx=0x7fa5b802a480 comp rx=0 tx=0).stop 2024-09-10T09:55:42.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.380+0000 7fa5d7400640 1 -- 172.21.15.77:0/3287556687 shutdown_connections 2024-09-10T09:55:42.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.380+0000 7fa5d7400640 1 --2- 172.21.15.77:0/3287556687 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5d010e3c0 0x7fa5d0110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:42.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.380+0000 7fa5d7400640 1 --2- 172.21.15.77:0/3287556687 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa5d0074f40 0x7fa5d010de80 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:42.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.380+0000 7fa5d7400640 1 --2- 172.21.15.77:0/3287556687 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa5d0074620 0x7fa5d0074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:42.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.380+0000 7fa5d7400640 1 -- 172.21.15.77:0/3287556687 >> 172.21.15.77:0/3287556687 conn(0x7fa5d00fd8c0 msgr2=0x7fa5d00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:42.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.380+0000 7fa5d7400640 1 -- 172.21.15.77:0/3287556687 shutdown_connections 2024-09-10T09:55:42.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.381+0000 7fa5d7400640 1 -- 172.21.15.77:0/3287556687 wait complete. 2024-09-10T09:55:42.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.381+0000 7fa5d7400640 1 Processor -- start 2024-09-10T09:55:42.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.382+0000 7fa5d7400640 1 -- start start 2024-09-10T09:55:42.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.382+0000 7fa5d7400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5d0074620 0x7fa5d019a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:42.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.382+0000 7fa5d7400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa5d0074f40 0x7fa5d019abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:42.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.382+0000 7fa5d7400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa5d010e3c0 0x7fa5d019ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:42.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.382+0000 7fa5d7400640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa5d0114160 con 0x7fa5d010e3c0 2024-09-10T09:55:42.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.382+0000 7fa5d7400640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa5d0113fe0 con 0x7fa5d0074f40 2024-09-10T09:55:42.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.382+0000 7fa5d7400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa5d01142e0 con 0x7fa5d0074620 2024-09-10T09:55:42.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.382+0000 7fa5d4a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5d0074620 0x7fa5d019a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:42.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.382+0000 7fa5d5400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa5d010e3c0 0x7fa5d019ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:42.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.382+0000 7fa5d4a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5d0074620 0x7fa5d019a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:40890/0 (socket says 172.21.15.77:40890) 2024-09-10T09:55:42.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.382+0000 7fa5d4a00640 1 -- 172.21.15.77:0/3586788243 learned_addr learned my addr 172.21.15.77:0/3586788243 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:55:42.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.383+0000 7fa5d4a00640 1 -- 172.21.15.77:0/3586788243 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa5d0074f40 msgr2=0x7fa5d019abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:42.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.383+0000 7fa5d4a00640 1 --2- 172.21.15.77:0/3586788243 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa5d0074f40 0x7fa5d019abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:42.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.383+0000 7fa5d4a00640 1 -- 172.21.15.77:0/3586788243 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa5d010e3c0 msgr2=0x7fa5d019ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:42.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.383+0000 7fa5d4a00640 1 --2- 172.21.15.77:0/3586788243 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa5d010e3c0 0x7fa5d019ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:42.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.383+0000 7fa5d4a00640 1 -- 172.21.15.77:0/3586788243 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5d019f650 con 0x7fa5d0074620 2024-09-10T09:55:42.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.383+0000 7fa5d4a00640 1 --2- 172.21.15.77:0/3586788243 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5d0074620 0x7fa5d019a6a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa5c000e9f0 tx=0x7fa5c000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:42.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.383+0000 7fa5ce000640 1 -- 172.21.15.77:0/3586788243 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c000cd80 con 0x7fa5d0074620 2024-09-10T09:55:42.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.383+0000 7fa5d7400640 1 -- 172.21.15.77:0/3586788243 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5d019f940 con 0x7fa5d0074620 2024-09-10T09:55:42.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.383+0000 7fa5ce000640 1 -- 172.21.15.77:0/3586788243 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5c0010910 con 0x7fa5d0074620 2024-09-10T09:55:42.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.383+0000 7fa5ce000640 1 -- 172.21.15.77:0/3586788243 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c00188f0 con 0x7fa5d0074620 2024-09-10T09:55:42.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.383+0000 7fa5d7400640 1 -- 172.21.15.77:0/3586788243 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5d01a7220 con 0x7fa5d0074620 2024-09-10T09:55:42.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.386+0000 7fa5d9df8740 1 Processor -- start 2024-09-10T09:55:42.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.387+0000 7fa5d9df8740 1 -- start start 2024-09-10T09:55:42.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.387+0000 7fa5ce000640 1 -- 172.21.15.77:0/3586788243 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa5c0010ac0 con 0x7fa5d0074620 2024-09-10T09:55:42.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.387+0000 7fa5d9df8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5650f792a330 0x5650f792a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:42.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.387+0000 7fa5d9df8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5650f792ac50 0x5650f7934fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:42.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.387+0000 7fa5d9df8740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650f7935510 0x5650f7937900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:42.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.387+0000 7fa5d9df8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5650f78900b0 con 0x5650f792a330 2024-09-10T09:55:42.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.387+0000 7fa5d9df8740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5650f7837ab0 con 0x5650f7935510 2024-09-10T09:55:42.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.387+0000 7fa5d9df8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5650f788c070 con 0x5650f792ac50 2024-09-10T09:55:42.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.387+0000 7fa5d5400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5650f792a330 0x5650f792a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:42.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.387+0000 7fa5d5400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5650f792a330 0x5650f792a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:44954/0 (socket says 172.21.15.77:44954) 2024-09-10T09:55:42.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.387+0000 7fa5d5400640 1 -- 172.21.15.77:0/4239514960 learned_addr learned my addr 172.21.15.77:0/4239514960 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:55:42.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.387+0000 7fa5d4a00640 1 --2- 172.21.15.77:0/4239514960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5650f792ac50 0x5650f7934fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:42.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.388+0000 7fa5c7e00640 1 --2- 172.21.15.77:0/4239514960 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650f7935510 0x5650f7937900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:42.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.388+0000 7fa5d5400640 1 -- 172.21.15.77:0/4239514960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5650f792ac50 msgr2=0x5650f7934fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:42.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.388+0000 7fa5d5400640 1 --2- 172.21.15.77:0/4239514960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5650f792ac50 0x5650f7934fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:42.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.388+0000 7fa5d5400640 1 -- 172.21.15.77:0/4239514960 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650f7935510 msgr2=0x5650f7937900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:42.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.388+0000 7fa5d5400640 1 --2- 172.21.15.77:0/4239514960 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650f7935510 0x5650f7937900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:42.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.388+0000 7fa5d5400640 1 -- 172.21.15.77:0/4239514960 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650f78444a0 con 0x5650f792a330 2024-09-10T09:55:42.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.388+0000 7fa5d5400640 1 --2- 172.21.15.77:0/4239514960 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5650f792a330 0x5650f792a710 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa5c8009f50 tx=0x7fa5c80118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84eaf291260ca4cc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:42.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.388+0000 7fa5c5c00640 1 -- 172.21.15.77:0/4239514960 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c8004d20 con 0x5650f792a330 2024-09-10T09:55:42.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.389+0000 7fa5c5c00640 1 -- 172.21.15.77:0/4239514960 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5c8004050 con 0x5650f792a330 2024-09-10T09:55:42.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.389+0000 7fa5c5c00640 1 -- 172.21.15.77:0/4239514960 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c8005240 con 0x5650f792a330 2024-09-10T09:55:42.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.389+0000 7fa5ce000640 1 --2- 172.21.15.77:0/3586788243 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fa5b40a2c00 0x7fa5b40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:42.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.389+0000 7fa5ce000640 1 -- 172.21.15.77:0/3586788243 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fa5c00cf880 con 0x7fa5d0074620 2024-09-10T09:55:42.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.389+0000 7fa5d4a00640 1 --2- 172.21.15.77:0/3586788243 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fa5b40a2c00 0x7fa5b40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:42.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.389+0000 7fa5d9df8740 1 -- 172.21.15.77:0/4239514960 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5650f792a330 msgr2=0x5650f792a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:42.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.389+0000 7fa5d9df8740 1 --2- 172.21.15.77:0/4239514960 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5650f792a330 0x5650f792a710 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa5c8009f50 tx=0x7fa5c80118d0 comp rx=0 tx=0).stop 2024-09-10T09:55:42.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.389+0000 7fa5d9df8740 1 -- 172.21.15.77:0/4239514960 shutdown_connections 2024-09-10T09:55:42.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.389+0000 7fa5d9df8740 1 --2- 172.21.15.77:0/4239514960 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650f7935510 0x5650f7937900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:42.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.389+0000 7fa5d9df8740 1 --2- 172.21.15.77:0/4239514960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5650f792ac50 0x5650f7934fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:42.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.389+0000 7fa5d9df8740 1 --2- 172.21.15.77:0/4239514960 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5650f792a330 0x5650f792a710 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:42.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.389+0000 7fa5d9df8740 1 -- 172.21.15.77:0/4239514960 >> 172.21.15.77:0/4239514960 conn(0x5650f79268c0 msgr2=0x5650f7926ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:42.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.389+0000 7fa5d9df8740 1 -- 172.21.15.77:0/4239514960 shutdown_connections 2024-09-10T09:55:42.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.390+0000 7fa5d4a00640 1 --2- 172.21.15.77:0/3586788243 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fa5b40a2c00 0x7fa5b40a50c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa5c009bc20 tx=0x7fa5c00d5530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:42.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.390+0000 7fa5d9df8740 1 -- 172.21.15.77:0/4239514960 wait complete. 2024-09-10T09:55:42.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.390+0000 7fa5d9df8740 1 -- start start 2024-09-10T09:55:42.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.390+0000 7fa5d9df8740 1 Processor -- start 2024-09-10T09:55:42.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.391+0000 7fa5d9df8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5650f792a330 0x5650f79325c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:42.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.391+0000 7fa5d9df8740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650f792ac50 0x5650f7932b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:42.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.391+0000 7fa5d9df8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5650f7935510 0x5650f7933040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:42.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.391+0000 7fa5d9df8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5650f7938060 con 0x5650f792a330 2024-09-10T09:55:42.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.391+0000 7fa5d9df8740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5650f7818920 con 0x5650f792ac50 2024-09-10T09:55:42.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.391+0000 7fa5d9df8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5650f79381e0 con 0x5650f7935510 2024-09-10T09:55:42.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.391+0000 7fa5d9df8740 10 client.0 _ll_register_callbacks cb 0x5650f783c8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:55:42.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.391+0000 7fa5d5400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5650f792a330 0x5650f79325c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:42.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.391+0000 7fa5d5400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5650f792a330 0x5650f79325c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:44956/0 (socket says 172.21.15.77:44956) 2024-09-10T09:55:42.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.391+0000 7fa5d5400640 1 -- 172.21.15.77:0/2310035593 learned_addr learned my addr 172.21.15.77:0/2310035593 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:55:42.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.391+0000 7fa5c7e00640 1 --2- 172.21.15.77:0/2310035593 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650f792ac50 0x5650f7932b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:42.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.391+0000 7fa5d5400640 1 --2- 172.21.15.77:0/2310035593 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5650f7935510 0x5650f7933040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:42.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.392+0000 7fa5c7e00640 1 -- 172.21.15.77:0/2310035593 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5650f7935510 msgr2=0x5650f7933040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:42.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.392+0000 7fa5c7e00640 1 --2- 172.21.15.77:0/2310035593 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5650f7935510 0x5650f7933040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:42.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.392+0000 7fa5c7e00640 1 -- 172.21.15.77:0/2310035593 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5650f792a330 msgr2=0x5650f79325c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:42.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.392+0000 7fa5c7e00640 1 --2- 172.21.15.77:0/2310035593 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5650f792a330 0x5650f79325c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:42.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.392+0000 7fa5c7e00640 1 -- 172.21.15.77:0/2310035593 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650f76012b0 con 0x5650f792ac50 2024-09-10T09:55:42.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.392+0000 7fa5c7e00640 1 --2- 172.21.15.77:0/2310035593 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650f792ac50 0x5650f7932b00 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x5650f7927c40 tx=0x7fa5b8032e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf1446d71c1f91fc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:42.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.392+0000 7fa5be000640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T09:55:42.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.392+0000 7fa5be000640 1 -- 172.21.15.77:0/2310035593 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b8040070 con 0x5650f792ac50 2024-09-10T09:55:42.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.392+0000 7fa5d9df8740 1 -- 172.21.15.77:0/2310035593 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5c8004050 con 0x5650f792ac50 2024-09-10T09:55:42.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.392+0000 7fa5be000640 1 -- 172.21.15.77:0/2310035593 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fa5b80319c0 con 0x5650f792ac50 2024-09-10T09:55:42.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.393+0000 7fa5be000640 1 -- 172.21.15.77:0/2310035593 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa5b8035040 con 0x5650f792ac50 2024-09-10T09:55:42.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.393+0000 7fa5d9df8740 10 client.25391 fetch_fsmap learned FSMap version 29 2024-09-10T09:55:42.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.393+0000 7fa5d9df8740 1 -- 172.21.15.77:0/2310035593 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5650f7943e20 con 0x5650f792ac50 2024-09-10T09:55:42.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.393+0000 7fa5be000640 1 -- 172.21.15.77:0/2310035593 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5c8004050 con 0x5650f792ac50 2024-09-10T09:55:42.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.393+0000 7fa5be000640 1 -- 172.21.15.77:0/2310035593 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fa5b80046c0 con 0x5650f792ac50 2024-09-10T09:55:42.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.393+0000 7fa5d9df8740 10 client.25391 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T09:55:42.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.393+0000 7fa5d9df8740 10 client.25391 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-10T09:55:42.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.393+0000 7fa5d9df8740 20 client.25391 populate_metadata read hostname 'smithi077' 2024-09-10T09:55:42.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.393+0000 7fa5d9df8740 1 --2- 172.21.15.77:0/2310035593 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x5650f79463e0 0x5650f7947850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:42.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.393+0000 7fa5d9df8740 4 client.25391 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:55:42.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.393+0000 7fa5d9df8740 1 -- 172.21.15.77:0/2310035593 --> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5650f789d620 con 0x5650f79463e0 2024-09-10T09:55:42.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.394+0000 7fa5d5400640 1 --2- 172.21.15.77:0/2310035593 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x5650f79463e0 0x5650f7947850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:42.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.394+0000 7fa5d5400640 1 --2- 172.21.15.77:0/2310035593 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x5650f79463e0 0x5650f7947850 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=80d580a07e8b9f97 server_cookie=4d0822e16e371e3 in_seq=0 out_seq=0 2024-09-10T09:55:42.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.394+0000 7fa5be000640 10 client.25391 ms_handle_connect on v2:172.21.15.134:6834/3293933699 2024-09-10T09:55:42.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.395+0000 7fa5be000640 1 -- 172.21.15.77:0/2310035593 <== mds.4 v2:172.21.15.134:6834/3293933699 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5650f789d620 con 0x5650f79463e0 2024-09-10T09:55:42.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.395+0000 7fa5be000640 10 client.25391 handle_command_reply: tid=0 2024-09-10T09:55:42.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.395+0000 7fa5d9df8740 1 client.25391 shutdown 2024-09-10T09:55:42.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.396+0000 7fa5d9df8740 1 -- 172.21.15.77:0/2310035593 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650f792ac50 msgr2=0x5650f7932b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:42.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.396+0000 7fa5d9df8740 1 --2- 172.21.15.77:0/2310035593 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650f792ac50 0x5650f7932b00 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x5650f7927c40 tx=0x7fa5b8032e60 comp rx=0 tx=0).stop 2024-09-10T09:55:42.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.396+0000 7fa5d9df8740 1 -- 172.21.15.77:0/2310035593 shutdown_connections 2024-09-10T09:55:42.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.396+0000 7fa5d9df8740 1 --2- 172.21.15.77:0/2310035593 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x5650f79463e0 0x5650f7947850 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:42.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.396+0000 7fa5d9df8740 1 --2- 172.21.15.77:0/2310035593 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5650f7935510 0x5650f7933040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:42.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.396+0000 7fa5d9df8740 1 --2- 172.21.15.77:0/2310035593 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650f792ac50 0x5650f7932b00 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:42.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.396+0000 7fa5be000640 0 client.25391 ms_handle_reset on v2:172.21.15.134:6834/3293933699 2024-09-10T09:55:42.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.396+0000 7fa5d9df8740 1 --2- 172.21.15.77:0/2310035593 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5650f792a330 0x5650f79325c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:42.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.396+0000 7fa5d9df8740 1 -- 172.21.15.77:0/2310035593 >> 172.21.15.77:0/2310035593 conn(0x5650f79268c0 msgr2=0x5650f792c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:42.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.396+0000 7fa5d9df8740 1 -- 172.21.15.77:0/2310035593 shutdown_connections 2024-09-10T09:55:42.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.396+0000 7fa5d9df8740 1 -- 172.21.15.77:0/2310035593 wait complete. 2024-09-10T09:55:42.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.396+0000 7fa5d9df8740 20 client.25391 trim_cache size 0 max 16384 2024-09-10T09:55:42.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.414+0000 7fa5d9df8740 1 Processor -- start 2024-09-10T09:55:42.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.414+0000 7fa5d9df8740 1 -- start start 2024-09-10T09:55:42.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.414+0000 7fa5d9df8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5650f792a330 0x5650f792d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:42.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.414+0000 7fa5d9df8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5650f792ac50 0x5650f7932e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:42.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.414+0000 7fa5d9df8740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650f7935510 0x5650f79315c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:42.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.414+0000 7fa5d9df8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5650f7945b90 con 0x5650f792ac50 2024-09-10T09:55:42.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.414+0000 7fa5d9df8740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5650f7946260 con 0x5650f7935510 2024-09-10T09:55:42.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.414+0000 7fa5d9df8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5650f7946930 con 0x5650f792a330 2024-09-10T09:55:42.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.415+0000 7fa5c7e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5650f792ac50 0x5650f7932e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:42.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.415+0000 7fa5c7e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5650f792ac50 0x5650f7932e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:44966/0 (socket says 172.21.15.77:44966) 2024-09-10T09:55:42.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.415+0000 7fa5c7e00640 1 -- 172.21.15.77:0/566814730 learned_addr learned my addr 172.21.15.77:0/566814730 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:55:42.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.415+0000 7fa5d5400640 1 --2- 172.21.15.77:0/566814730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5650f792a330 0x5650f792d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:42.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.415+0000 7fa5d5400640 1 --2- 172.21.15.77:0/566814730 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650f7935510 0x5650f79315c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:42.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.415+0000 7fa5c7e00640 1 -- 172.21.15.77:0/566814730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5650f792a330 msgr2=0x5650f792d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:42.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.415+0000 7fa5c7e00640 1 --2- 172.21.15.77:0/566814730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5650f792a330 0x5650f792d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:42.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.415+0000 7fa5c7e00640 1 -- 172.21.15.77:0/566814730 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650f7935510 msgr2=0x5650f79315c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:42.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.415+0000 7fa5c7e00640 1 --2- 172.21.15.77:0/566814730 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650f7935510 0x5650f79315c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:42.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.415+0000 7fa5c7e00640 1 -- 172.21.15.77:0/566814730 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650f792aa80 con 0x5650f792ac50 2024-09-10T09:55:42.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.415+0000 7fa5d5400640 1 --2- 172.21.15.77:0/566814730 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650f7935510 0x5650f79315c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:55:42.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.416+0000 7fa5c7e00640 1 --2- 172.21.15.77:0/566814730 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5650f792ac50 0x5650f7932e90 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fa5b8004a90 tx=0x7fa5b8001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2129f1878cb8ccfc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:42.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.416+0000 7fa5c5200640 1 -- 172.21.15.77:0/566814730 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b8040070 con 0x5650f792ac50 2024-09-10T09:55:42.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.416+0000 7fa5c5200640 1 -- 172.21.15.77:0/566814730 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5b8031ca0 con 0x5650f792ac50 2024-09-10T09:55:42.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.416+0000 7fa5c5200640 1 -- 172.21.15.77:0/566814730 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b80053a0 con 0x5650f792ac50 2024-09-10T09:55:42.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.416+0000 7fa5d9df8740 1 -- 172.21.15.77:0/566814730 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5650f792ac50 msgr2=0x5650f7932e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:42.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.416+0000 7fa5d9df8740 1 --2- 172.21.15.77:0/566814730 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5650f792ac50 0x5650f7932e90 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fa5b8004a90 tx=0x7fa5b8001e70 comp rx=0 tx=0).stop 2024-09-10T09:55:42.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.417+0000 7fa5d9df8740 1 -- 172.21.15.77:0/566814730 shutdown_connections 2024-09-10T09:55:42.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.417+0000 7fa5d9df8740 1 --2- 172.21.15.77:0/566814730 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650f7935510 0x5650f79315c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:42.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.417+0000 7fa5d9df8740 1 --2- 172.21.15.77:0/566814730 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5650f792ac50 0x5650f7932e90 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:42.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.417+0000 7fa5d9df8740 1 --2- 172.21.15.77:0/566814730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5650f792a330 0x5650f792d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:42.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.417+0000 7fa5d9df8740 1 -- 172.21.15.77:0/566814730 >> 172.21.15.77:0/566814730 conn(0x5650f79268c0 msgr2=0x5650f7930eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:42.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.417+0000 7fa5d9df8740 1 -- 172.21.15.77:0/566814730 shutdown_connections 2024-09-10T09:55:42.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.417+0000 7fa5d9df8740 1 -- 172.21.15.77:0/566814730 wait complete. 2024-09-10T09:55:42.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.417+0000 7fa5d9df8740 1 -- start start 2024-09-10T09:55:42.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.417+0000 7fa5d9df8740 1 Processor -- start 2024-09-10T09:55:42.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.418+0000 7fa5d9df8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5650f792a330 0x5650f7949ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:42.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.418+0000 7fa5d9df8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5650f792ac50 0x5650f794a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:42.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.418+0000 7fa5d9df8740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650f7935510 0x5650f794a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:42.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.418+0000 7fa5d9df8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5650f79476d0 con 0x5650f792a330 2024-09-10T09:55:42.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.418+0000 7fa5d9df8740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5650f7947000 con 0x5650f7935510 2024-09-10T09:55:42.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.418+0000 7fa5d9df8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5650f7947da0 con 0x5650f792ac50 2024-09-10T09:55:42.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.418+0000 7fa5d9df8740 10 client.0 _ll_register_callbacks cb 0x7fa5b8009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:55:42.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.418+0000 7fa5d5400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5650f792a330 0x5650f7949ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:42.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.418+0000 7fa5d5400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5650f792a330 0x5650f7949ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:44970/0 (socket says 172.21.15.77:44970) 2024-09-10T09:55:42.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.418+0000 7fa5d5400640 1 -- 172.21.15.77:0/2524002222 learned_addr learned my addr 172.21.15.77:0/2524002222 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:55:42.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.418+0000 7fa5c7e00640 1 --2- 172.21.15.77:0/2524002222 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5650f792ac50 0x5650f794a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:42.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.418+0000 7fa5d5400640 1 --2- 172.21.15.77:0/2524002222 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650f7935510 0x5650f794a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:42.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.418+0000 7fa5c7e00640 1 -- 172.21.15.77:0/2524002222 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650f7935510 msgr2=0x5650f794a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:42.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.418+0000 7fa5c7e00640 1 --2- 172.21.15.77:0/2524002222 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650f7935510 0x5650f794a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:42.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.418+0000 7fa5c7e00640 1 -- 172.21.15.77:0/2524002222 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5650f792a330 msgr2=0x5650f7949ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:42.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.418+0000 7fa5c7e00640 1 --2- 172.21.15.77:0/2524002222 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5650f792a330 0x5650f7949ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:42.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.418+0000 7fa5c7e00640 1 -- 172.21.15.77:0/2524002222 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650f7935310 con 0x5650f792ac50 2024-09-10T09:55:42.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.418+0000 7fa5d5400640 1 --2- 172.21.15.77:0/2524002222 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5650f792a330 0x5650f7949ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:55:42.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.419+0000 7fa5c7e00640 1 --2- 172.21.15.77:0/2524002222 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5650f792ac50 0x5650f794a0e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x5650f792b0e0 tx=0x7fa5b8032e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bee8e31adb8b47d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:42.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.419+0000 7fa5be000640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-10T09:55:42.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.419+0000 7fa5be000640 1 -- 172.21.15.77:0/2524002222 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b8040040 con 0x5650f792ac50 2024-09-10T09:55:42.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.419+0000 7fa5d9df8740 1 -- 172.21.15.77:0/2524002222 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5b8031ca0 con 0x5650f792ac50 2024-09-10T09:55:42.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.419+0000 7fa5be000640 1 -- 172.21.15.77:0/2524002222 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fa5b80054f0 con 0x5650f792ac50 2024-09-10T09:55:42.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.419+0000 7fa5be000640 1 -- 172.21.15.77:0/2524002222 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa5b8005d50 con 0x5650f792ac50 2024-09-10T09:55:42.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.419+0000 7fa5d9df8740 10 client.25372 fetch_fsmap learned FSMap version 29 2024-09-10T09:55:42.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.419+0000 7fa5d9df8740 1 -- 172.21.15.77:0/2524002222 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5650f79b0ed0 con 0x5650f792ac50 2024-09-10T09:55:42.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.420+0000 7fa5be000640 1 -- 172.21.15.77:0/2524002222 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5b8031ca0 con 0x5650f792ac50 2024-09-10T09:55:42.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.420+0000 7fa5be000640 1 -- 172.21.15.77:0/2524002222 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fa5b8051420 con 0x5650f792ac50 2024-09-10T09:55:42.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.420+0000 7fa5d9df8740 10 client.25372 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T09:55:42.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.420+0000 7fa5d9df8740 10 client.25372 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-10T09:55:42.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.420+0000 7fa5d9df8740 20 client.25372 populate_metadata read hostname 'smithi077' 2024-09-10T09:55:42.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.420+0000 7fa5d9df8740 1 --2- 172.21.15.77:0/2524002222 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x5650f79b5660 0x5650f79b4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:42.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.420+0000 7fa5d9df8740 4 client.25372 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T09:55:42.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.420+0000 7fa5d9df8740 1 -- 172.21.15.77:0/2524002222 --> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] -- command(tid 0: {"prefix": "status"}) -- 0x5650f7897b30 con 0x5650f79b5660 2024-09-10T09:55:42.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.420+0000 7fa5d5400640 1 --2- 172.21.15.77:0/2524002222 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x5650f79b5660 0x5650f79b4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:42.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.421+0000 7fa5d5400640 1 --2- 172.21.15.77:0/2524002222 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x5650f79b5660 0x5650f79b4370 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c6705dfcdc96bc86 server_cookie=2f1661a69071213a in_seq=0 out_seq=0 2024-09-10T09:55:42.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.421+0000 7fa5be000640 10 client.25372 ms_handle_connect on v2:172.21.15.134:6834/3293933699 2024-09-10T09:55:42.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.421+0000 7fa5be000640 1 -- 172.21.15.77:0/2524002222 <== mds.4 v2:172.21.15.134:6834/3293933699 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5650f7897b30 con 0x5650f79b5660 2024-09-10T09:55:42.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.421+0000 7fa5be000640 10 client.25372 handle_command_reply: tid=0 2024-09-10T09:55:42.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.421+0000 7fa5d9df8740 1 client.25372 shutdown 2024-09-10T09:55:42.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.421+0000 7fa5d9df8740 1 -- 172.21.15.77:0/2524002222 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5650f792ac50 msgr2=0x5650f794a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:42.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.421+0000 7fa5d9df8740 1 --2- 172.21.15.77:0/2524002222 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5650f792ac50 0x5650f794a0e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x5650f792b0e0 tx=0x7fa5b8032e60 comp rx=0 tx=0).stop 2024-09-10T09:55:42.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.421+0000 7fa5d9df8740 1 -- 172.21.15.77:0/2524002222 shutdown_connections 2024-09-10T09:55:42.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.421+0000 7fa5d9df8740 1 --2- 172.21.15.77:0/2524002222 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x5650f79b5660 0x5650f79b4370 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:42.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.421+0000 7fa5d9df8740 1 --2- 172.21.15.77:0/2524002222 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650f7935510 0x5650f794a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:42.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.421+0000 7fa5d9df8740 1 --2- 172.21.15.77:0/2524002222 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5650f792ac50 0x5650f794a0e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:42.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.421+0000 7fa5d9df8740 1 --2- 172.21.15.77:0/2524002222 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5650f792a330 0x5650f7949ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:42.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.421+0000 7fa5be000640 0 client.25372 ms_handle_reset on v2:172.21.15.134:6834/3293933699 2024-09-10T09:55:42.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.421+0000 7fa5d9df8740 1 -- 172.21.15.77:0/2524002222 >> 172.21.15.77:0/2524002222 conn(0x5650f79268c0 msgr2=0x5650f7932ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:42.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.421+0000 7fa5d9df8740 1 -- 172.21.15.77:0/2524002222 shutdown_connections 2024-09-10T09:55:42.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.421+0000 7fa5d9df8740 1 -- 172.21.15.77:0/2524002222 wait complete. 2024-09-10T09:55:42.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.422+0000 7fa5d9df8740 20 client.25372 trim_cache size 0 max 16384 2024-09-10T09:55:42.427 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-10T09:55:42.427 INFO:teuthology.orchestra.run.smithi077.stdout: "cluster_fsid": "14a55226-6f59-11ef-bcea-c7b262605968", 2024-09-10T09:55:42.427 INFO:teuthology.orchestra.run.smithi077.stdout: "whoami": 4, 2024-09-10T09:55:42.427 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 24496, 2024-09-10T09:55:42.427 INFO:teuthology.orchestra.run.smithi077.stdout: "want_state": "up:active", 2024-09-10T09:55:42.428 INFO:teuthology.orchestra.run.smithi077.stdout: "state": "up:active", 2024-09-10T09:55:42.428 INFO:teuthology.orchestra.run.smithi077.stdout: "fs_name": "cephfs", 2024-09-10T09:55:42.428 INFO:teuthology.orchestra.run.smithi077.stdout: "rank_uptime": 146.55669947999999, 2024-09-10T09:55:42.428 INFO:teuthology.orchestra.run.smithi077.stdout: "mdsmap_epoch": 29, 2024-09-10T09:55:42.428 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch": 96, 2024-09-10T09:55:42.428 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch_barrier": 96, 2024-09-10T09:55:42.428 INFO:teuthology.orchestra.run.smithi077.stdout: "uptime": 230.031021385 2024-09-10T09:55:42.428 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-10T09:55:42.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.422+0000 7fa5d7400640 1 -- 172.21.15.77:0/3586788243 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fa5b40a2c00 msgr2=0x7fa5b40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:42.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.422+0000 7fa5d7400640 1 --2- 172.21.15.77:0/3586788243 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fa5b40a2c00 0x7fa5b40a50c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa5c009bc20 tx=0x7fa5c00d5530 comp rx=0 tx=0).stop 2024-09-10T09:55:42.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.422+0000 7fa5d7400640 1 -- 172.21.15.77:0/3586788243 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5d0074620 msgr2=0x7fa5d019a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:42.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.422+0000 7fa5d7400640 1 --2- 172.21.15.77:0/3586788243 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5d0074620 0x7fa5d019a6a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa5c000e9f0 tx=0x7fa5c000eeb0 comp rx=0 tx=0).stop 2024-09-10T09:55:42.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.422+0000 7fa5d7400640 1 -- 172.21.15.77:0/3586788243 shutdown_connections 2024-09-10T09:55:42.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.422+0000 7fa5d7400640 1 --2- 172.21.15.77:0/3586788243 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fa5b40a2c00 0x7fa5b40a50c0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:42.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.422+0000 7fa5d7400640 1 --2- 172.21.15.77:0/3586788243 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa5d010e3c0 0x7fa5d019ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:42.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.422+0000 7fa5d7400640 1 --2- 172.21.15.77:0/3586788243 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa5d0074f40 0x7fa5d019abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:42.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.422+0000 7fa5d7400640 1 --2- 172.21.15.77:0/3586788243 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5d0074620 0x7fa5d019a6a0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:42.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.422+0000 7fa5d7400640 1 -- 172.21.15.77:0/3586788243 >> 172.21.15.77:0/3586788243 conn(0x7fa5d00fd8c0 msgr2=0x7fa5d00fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:42.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.422+0000 7fa5d7400640 1 -- 172.21.15.77:0/3586788243 shutdown_connections 2024-09-10T09:55:42.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:42.422+0000 7fa5d7400640 1 -- 172.21.15.77:0/3586788243 wait complete. 2024-09-10T09:55:43.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:55:42 smithi134 ceph-mon[35674]: pgmap v407: 129 pgs: 129 active+clean; 652 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-10T09:55:43.164 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:55:42 smithi077 ceph-mon[30319]: pgmap v407: 129 pgs: 129 active+clean; 652 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-10T09:55:43.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:55:42 smithi120 ceph-mon[34770]: pgmap v407: 129 pgs: 129 active+clean; 652 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-10T09:55:43.205 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.c status 2024-09-10T09:55:43.574 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:55:44.130 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.127+0000 7f5ccd600640 1 -- 172.21.15.77:0/3803356742 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5cc8074620 msgr2=0x7f5cc8074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:44.130 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.127+0000 7f5ccd600640 1 --2- 172.21.15.77:0/3803356742 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5cc8074620 0x7f5cc8074a00 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f5cb8009900 tx=0x7f5cb802a480 comp rx=0 tx=0).stop 2024-09-10T09:55:44.130 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.127+0000 7f5ccd600640 1 -- 172.21.15.77:0/3803356742 shutdown_connections 2024-09-10T09:55:44.130 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.127+0000 7f5ccd600640 1 --2- 172.21.15.77:0/3803356742 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5cc810e3c0 0x7f5cc8110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:44.130 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.127+0000 7f5ccd600640 1 --2- 172.21.15.77:0/3803356742 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5cc8074f40 0x7f5cc810de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:44.130 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.127+0000 7f5ccd600640 1 --2- 172.21.15.77:0/3803356742 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5cc8074620 0x7f5cc8074a00 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:44.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.127+0000 7f5ccd600640 1 -- 172.21.15.77:0/3803356742 >> 172.21.15.77:0/3803356742 conn(0x7f5cc80fd8c0 msgr2=0x7f5cc80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:44.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.127+0000 7f5ccd600640 1 -- 172.21.15.77:0/3803356742 shutdown_connections 2024-09-10T09:55:44.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.127+0000 7f5ccd600640 1 -- 172.21.15.77:0/3803356742 wait complete. 2024-09-10T09:55:44.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.128+0000 7f5ccd600640 1 Processor -- start 2024-09-10T09:55:44.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.129+0000 7f5ccd600640 1 -- start start 2024-09-10T09:55:44.132 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.129+0000 7f5ccd600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5cc8074620 0x7f5cc819a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:44.132 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.129+0000 7f5ccd600640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5cc8074f40 0x7f5cc819abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:44.132 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.129+0000 7f5ccd600640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5cc810e3c0 0x7f5cc819ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:44.132 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.129+0000 7f5ccd600640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc8114170 con 0x7f5cc810e3c0 2024-09-10T09:55:44.133 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.129+0000 7f5ccd600640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc8113ff0 con 0x7f5cc8074f40 2024-09-10T09:55:44.133 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.129+0000 7f5ccd600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc81142f0 con 0x7f5cc8074620 2024-09-10T09:55:44.133 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.129+0000 7f5cc6a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5cc8074620 0x7f5cc819a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:44.133 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.129+0000 7f5cc6a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5cc8074620 0x7f5cc819a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:39220/0 (socket says 172.21.15.77:39220) 2024-09-10T09:55:44.133 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.129+0000 7f5cc6a00640 1 -- 172.21.15.77:0/3715118214 learned_addr learned my addr 172.21.15.77:0/3715118214 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:55:44.133 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.129+0000 7f5cc7400640 1 --2- 172.21.15.77:0/3715118214 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5cc810e3c0 0x7f5cc819ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:44.133 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.129+0000 7f5cc6000640 1 --2- 172.21.15.77:0/3715118214 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5cc8074f40 0x7f5cc819abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:44.133 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.130+0000 7f5cc6a00640 1 -- 172.21.15.77:0/3715118214 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5cc8074f40 msgr2=0x7f5cc819abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:44.133 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.130+0000 7f5cc6a00640 1 --2- 172.21.15.77:0/3715118214 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5cc8074f40 0x7f5cc819abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:44.133 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.130+0000 7f5cc6a00640 1 -- 172.21.15.77:0/3715118214 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5cc810e3c0 msgr2=0x7f5cc819ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:44.133 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.130+0000 7f5cc6a00640 1 --2- 172.21.15.77:0/3715118214 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5cc810e3c0 0x7f5cc819ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:44.133 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.130+0000 7f5cc6a00640 1 -- 172.21.15.77:0/3715118214 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cc819f630 con 0x7f5cc8074620 2024-09-10T09:55:44.133 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.130+0000 7f5cc6a00640 1 --2- 172.21.15.77:0/3715118214 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5cc8074620 0x7f5cc819a680 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5cb8006010 tx=0x7f5cb80042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:44.133 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.130+0000 7f5cbf400640 1 -- 172.21.15.77:0/3715118214 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb803f030 con 0x7f5cc8074620 2024-09-10T09:55:44.134 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.130+0000 7f5ccd600640 1 -- 172.21.15.77:0/3715118214 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cc819f8c0 con 0x7f5cc8074620 2024-09-10T09:55:44.134 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.130+0000 7f5cbf400640 1 -- 172.21.15.77:0/3715118214 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cb8003ce0 con 0x7f5cc8074620 2024-09-10T09:55:44.134 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.130+0000 7f5cbf400640 1 -- 172.21.15.77:0/3715118214 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb8035070 con 0x7f5cc8074620 2024-09-10T09:55:44.134 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.131+0000 7f5ccd600640 1 -- 172.21.15.77:0/3715118214 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cc81a7200 con 0x7f5cc8074620 2024-09-10T09:55:44.135 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.133+0000 7f5cd0257740 1 Processor -- start 2024-09-10T09:55:44.135 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.134+0000 7f5cd0257740 1 -- start start 2024-09-10T09:55:44.136 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.134+0000 7f5cbf400640 1 -- 172.21.15.77:0/3715118214 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5cb8042030 con 0x7f5cc8074620 2024-09-10T09:55:44.136 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.134+0000 7f5cd0257740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558720a64330 0x558720a64710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:44.136 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.134+0000 7f5cd0257740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558720a64c50 0x558720a6efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:44.136 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.134+0000 7f5cd0257740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558720a6f510 0x558720a71900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:44.136 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.134+0000 7f5cd0257740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5587209ca0b0 con 0x558720a64c50 2024-09-10T09:55:44.136 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.134+0000 7f5cd0257740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558720971ab0 con 0x558720a6f510 2024-09-10T09:55:44.137 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.134+0000 7f5cd0257740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5587209c6070 con 0x558720a64330 2024-09-10T09:55:44.137 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.134+0000 7f5cc6a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558720a64c50 0x558720a6efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:44.137 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.134+0000 7f5cc6a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558720a64c50 0x558720a6efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33140/0 (socket says 172.21.15.77:33140) 2024-09-10T09:55:44.137 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.134+0000 7f5cc6a00640 1 -- 172.21.15.77:0/589963774 learned_addr learned my addr 172.21.15.77:0/589963774 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:55:44.137 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.134+0000 7f5cc7400640 1 --2- 172.21.15.77:0/589963774 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558720a64330 0x558720a64710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:44.137 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.134+0000 7f5cc6000640 1 --2- 172.21.15.77:0/589963774 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558720a6f510 0x558720a71900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:44.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.135+0000 7f5cc6a00640 1 -- 172.21.15.77:0/589963774 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558720a64330 msgr2=0x558720a64710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:44.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.135+0000 7f5cc6a00640 1 --2- 172.21.15.77:0/589963774 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558720a64330 0x558720a64710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:44.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.135+0000 7f5cc6a00640 1 -- 172.21.15.77:0/589963774 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558720a6f510 msgr2=0x558720a71900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:44.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.135+0000 7f5cc6a00640 1 --2- 172.21.15.77:0/589963774 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558720a6f510 0x558720a71900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:44.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.135+0000 7f5cc6a00640 1 -- 172.21.15.77:0/589963774 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55872097e4a0 con 0x558720a64c50 2024-09-10T09:55:44.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.135+0000 7f5cc6a00640 1 --2- 172.21.15.77:0/589963774 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558720a64c50 0x558720a6efd0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f5cb80f5c20 tx=0x7f5cb80f5c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe20a6032a679e52 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:44.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.135+0000 7f5cbf400640 1 --2- 172.21.15.77:0/3715118214 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f5cac0a2c00 0x7f5cac0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:44.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.136+0000 7f5cb3400640 1 -- 172.21.15.77:0/589963774 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb80fe070 con 0x558720a64c50 2024-09-10T09:55:44.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.136+0000 7f5cb3400640 1 -- 172.21.15.77:0/589963774 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cb8039b50 con 0x558720a64c50 2024-09-10T09:55:44.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.136+0000 7f5cb3400640 1 -- 172.21.15.77:0/589963774 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb80bb820 con 0x558720a64c50 2024-09-10T09:55:44.139 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.136+0000 7f5cbf400640 1 -- 172.21.15.77:0/3715118214 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f5cb80ef940 con 0x7f5cc8074620 2024-09-10T09:55:44.139 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.136+0000 7f5cc7400640 1 --2- 172.21.15.77:0/3715118214 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f5cac0a2c00 0x7f5cac0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:44.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.136+0000 7f5cd0257740 1 -- 172.21.15.77:0/589963774 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558720a64c50 msgr2=0x558720a6efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:44.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.136+0000 7f5cd0257740 1 --2- 172.21.15.77:0/589963774 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558720a64c50 0x558720a6efd0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f5cb80f5c20 tx=0x7f5cb80f5c50 comp rx=0 tx=0).stop 2024-09-10T09:55:44.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.137+0000 7f5cc7400640 1 --2- 172.21.15.77:0/3715118214 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f5cac0a2c00 0x7f5cac0a50c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f5cc0005fd0 tx=0x7f5cc0009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:44.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.137+0000 7f5cd0257740 1 -- 172.21.15.77:0/589963774 shutdown_connections 2024-09-10T09:55:44.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.137+0000 7f5cd0257740 1 --2- 172.21.15.77:0/589963774 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558720a6f510 0x558720a71900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:44.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.137+0000 7f5cd0257740 1 --2- 172.21.15.77:0/589963774 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558720a64c50 0x558720a6efd0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:44.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.137+0000 7f5cd0257740 1 --2- 172.21.15.77:0/589963774 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558720a64330 0x558720a64710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:44.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.137+0000 7f5cd0257740 1 -- 172.21.15.77:0/589963774 >> 172.21.15.77:0/589963774 conn(0x558720a608c0 msgr2=0x558720a60ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:44.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.137+0000 7f5cd0257740 1 -- 172.21.15.77:0/589963774 shutdown_connections 2024-09-10T09:55:44.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.137+0000 7f5cd0257740 1 -- 172.21.15.77:0/589963774 wait complete. 2024-09-10T09:55:44.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.137+0000 7f5cd0257740 1 -- start start 2024-09-10T09:55:44.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.137+0000 7f5cd0257740 1 Processor -- start 2024-09-10T09:55:44.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.138+0000 7f5cd0257740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558720a64330 0x558720a6c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:44.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.138+0000 7f5cd0257740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558720a64c50 0x558720a6ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:44.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.138+0000 7f5cd0257740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558720a6f510 0x558720a6cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:44.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.138+0000 7f5cd0257740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558720a71ea0 con 0x558720a64330 2024-09-10T09:55:44.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.138+0000 7f5cd0257740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558720952920 con 0x558720a6f510 2024-09-10T09:55:44.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.138+0000 7f5cd0257740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558720a72020 con 0x558720a64c50 2024-09-10T09:55:44.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.138+0000 7f5cd0257740 10 client.0 _ll_register_callbacks cb 0x5587209768a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:55:44.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.138+0000 7f5cc6a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558720a64330 0x558720a6c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:44.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.138+0000 7f5cc6a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558720a64330 0x558720a6c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33144/0 (socket says 172.21.15.77:33144) 2024-09-10T09:55:44.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.138+0000 7f5cc6a00640 1 -- 172.21.15.77:0/1104816162 learned_addr learned my addr 172.21.15.77:0/1104816162 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:55:44.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.138+0000 7f5cc7400640 1 --2- 172.21.15.77:0/1104816162 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558720a6f510 0x558720a6cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:44.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.138+0000 7f5cc6000640 1 --2- 172.21.15.77:0/1104816162 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558720a64c50 0x558720a6ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:44.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.139+0000 7f5cc6a00640 1 -- 172.21.15.77:0/1104816162 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558720a64c50 msgr2=0x558720a6ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:44.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.139+0000 7f5cc6a00640 1 --2- 172.21.15.77:0/1104816162 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558720a64c50 0x558720a6ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:44.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.139+0000 7f5cc6a00640 1 -- 172.21.15.77:0/1104816162 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558720a6f510 msgr2=0x558720a6cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:44.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.139+0000 7f5cc6a00640 1 --2- 172.21.15.77:0/1104816162 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558720a6f510 0x558720a6cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:44.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.139+0000 7f5cc6a00640 1 -- 172.21.15.77:0/1104816162 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55872073b2b0 con 0x558720a64330 2024-09-10T09:55:44.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.139+0000 7f5cc7400640 1 --2- 172.21.15.77:0/1104816162 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558720a6f510 0x558720a6cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:55:44.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.139+0000 7f5cc6a00640 1 --2- 172.21.15.77:0/1104816162 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558720a64330 0x558720a6c4d0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f5cb80b8890 tx=0x7f5cb803b5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ec0213b548cc78c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:44.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.139+0000 7f5ca7e00640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-10T09:55:44.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.139+0000 7f5ca7e00640 1 -- 172.21.15.77:0/1104816162 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb802d3c0 con 0x558720a64330 2024-09-10T09:55:44.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.139+0000 7f5ca7e00640 1 -- 172.21.15.77:0/1104816162 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f5cb80ed100 con 0x558720a64330 2024-09-10T09:55:44.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.139+0000 7f5cd0257740 1 -- 172.21.15.77:0/1104816162 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5cb8039b50 con 0x558720a64330 2024-09-10T09:55:44.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.140+0000 7f5ca7e00640 1 -- 172.21.15.77:0/1104816162 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5cb8049070 con 0x558720a64330 2024-09-10T09:55:44.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.140+0000 7f5cd0257740 10 client.15363 fetch_fsmap learned FSMap version 29 2024-09-10T09:55:44.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.140+0000 7f5cd0257740 1 -- 172.21.15.77:0/1104816162 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558720a71ea0 con 0x558720a64330 2024-09-10T09:55:44.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.140+0000 7f5ca7e00640 1 -- 172.21.15.77:0/1104816162 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cb8039b50 con 0x558720a64330 2024-09-10T09:55:44.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.140+0000 7f5ca7e00640 1 -- 172.21.15.77:0/1104816162 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f5cb80b8530 con 0x558720a64330 2024-09-10T09:55:44.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.140+0000 7f5cd0257740 10 client.15363 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T09:55:44.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.140+0000 7f5cd0257740 10 client.15363 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T09:55:44.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.140+0000 7f5cd0257740 20 client.15363 populate_metadata read hostname 'smithi077' 2024-09-10T09:55:44.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.140+0000 7f5cd0257740 1 --2- 172.21.15.77:0/1104816162 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x558720a7ff10 0x558720a81380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:44.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.140+0000 7f5cd0257740 4 client.15363 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:55:44.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.140+0000 7f5cd0257740 1 -- 172.21.15.77:0/1104816162 --> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5587209d7620 con 0x558720a7ff10 2024-09-10T09:55:44.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.141+0000 7f5cc6000640 1 --2- 172.21.15.77:0/1104816162 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x558720a7ff10 0x558720a81380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:44.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.141+0000 7f5cc6000640 1 --2- 172.21.15.77:0/1104816162 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x558720a7ff10 0x558720a81380 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b6ff9c60b2eb9ff1 server_cookie=23886e8515fea931 in_seq=0 out_seq=0 2024-09-10T09:55:44.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.141+0000 7f5ca7e00640 10 client.15363 ms_handle_connect on v2:172.21.15.134:6836/2461497091 2024-09-10T09:55:44.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.142+0000 7f5ca7e00640 1 -- 172.21.15.77:0/1104816162 <== mds.3 v2:172.21.15.134:6836/2461497091 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5587209d7620 con 0x558720a7ff10 2024-09-10T09:55:44.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.142+0000 7f5ca7e00640 10 client.15363 handle_command_reply: tid=0 2024-09-10T09:55:44.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.142+0000 7f5cd0257740 1 client.15363 shutdown 2024-09-10T09:55:44.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.142+0000 7f5cd0257740 1 -- 172.21.15.77:0/1104816162 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558720a64330 msgr2=0x558720a6c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:44.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.142+0000 7f5cd0257740 1 --2- 172.21.15.77:0/1104816162 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558720a64330 0x558720a6c4d0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f5cb80b8890 tx=0x7f5cb803b5d0 comp rx=0 tx=0).stop 2024-09-10T09:55:44.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.142+0000 7f5cd0257740 1 -- 172.21.15.77:0/1104816162 shutdown_connections 2024-09-10T09:55:44.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.142+0000 7f5cd0257740 1 --2- 172.21.15.77:0/1104816162 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558720a6f510 0x558720a6cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:44.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.142+0000 7f5cd0257740 1 --2- 172.21.15.77:0/1104816162 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558720a64c50 0x558720a6ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:44.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.142+0000 7f5cd0257740 1 --2- 172.21.15.77:0/1104816162 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x558720a7ff10 0x558720a81380 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:44.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.143+0000 7f5cd0257740 1 --2- 172.21.15.77:0/1104816162 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558720a64330 0x558720a6c4d0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:44.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.143+0000 7f5ca7e00640 0 client.15363 ms_handle_reset on v2:172.21.15.134:6836/2461497091 2024-09-10T09:55:44.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.143+0000 7f5cd0257740 1 -- 172.21.15.77:0/1104816162 >> 172.21.15.77:0/1104816162 conn(0x558720a608c0 msgr2=0x558720a665a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:44.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.143+0000 7f5cd0257740 1 -- 172.21.15.77:0/1104816162 shutdown_connections 2024-09-10T09:55:44.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.143+0000 7f5cd0257740 1 -- 172.21.15.77:0/1104816162 wait complete. 2024-09-10T09:55:44.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.143+0000 7f5cd0257740 20 client.15363 trim_cache size 0 max 16384 2024-09-10T09:55:44.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.161+0000 7f5cd0257740 1 Processor -- start 2024-09-10T09:55:44.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.161+0000 7f5cd0257740 1 -- start start 2024-09-10T09:55:44.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.161+0000 7f5cd0257740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558720a64330 0x558720a6cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:44.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.161+0000 7f5cd0257740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558720a64c50 0x558720a697b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:44.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.161+0000 7f5cd0257740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558720a6f510 0x558720a69cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:44.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.161+0000 7f5cd0257740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558720a7f6c0 con 0x558720a6f510 2024-09-10T09:55:44.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.161+0000 7f5cd0257740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558720a7fd90 con 0x558720a64330 2024-09-10T09:55:44.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.161+0000 7f5cd0257740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558720a80460 con 0x558720a64c50 2024-09-10T09:55:44.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.161+0000 7f5cc7400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558720a6f510 0x558720a69cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:44.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.161+0000 7f5cc7400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558720a6f510 0x558720a69cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33146/0 (socket says 172.21.15.77:33146) 2024-09-10T09:55:44.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.161+0000 7f5cc7400640 1 -- 172.21.15.77:0/1042704358 learned_addr learned my addr 172.21.15.77:0/1042704358 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:55:44.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.162+0000 7f5cc6000640 1 --2- 172.21.15.77:0/1042704358 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558720a64c50 0x558720a697b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:44.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.162+0000 7f5cc6a00640 1 --2- 172.21.15.77:0/1042704358 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558720a64330 0x558720a6cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:44.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.162+0000 7f5cc6000640 1 -- 172.21.15.77:0/1042704358 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558720a64330 msgr2=0x558720a6cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:44.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.162+0000 7f5cc6000640 1 --2- 172.21.15.77:0/1042704358 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558720a64330 0x558720a6cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:44.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.162+0000 7f5cc6000640 1 -- 172.21.15.77:0/1042704358 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558720a6f510 msgr2=0x558720a69cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:44.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.162+0000 7f5cc6000640 1 --2- 172.21.15.77:0/1042704358 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558720a6f510 0x558720a69cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:44.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.162+0000 7f5cc6000640 1 -- 172.21.15.77:0/1042704358 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558720a61a50 con 0x558720a64c50 2024-09-10T09:55:44.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.162+0000 7f5cc6000640 1 --2- 172.21.15.77:0/1042704358 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558720a64c50 0x558720a697b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x558720a6bd80 tx=0x7f5cb400b410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14fe7e2c4c231f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:44.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.163+0000 7f5cb2a00640 1 -- 172.21.15.77:0/1042704358 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb4004220 con 0x558720a64c50 2024-09-10T09:55:44.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.163+0000 7f5cb2a00640 1 -- 172.21.15.77:0/1042704358 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cb40043c0 con 0x558720a64c50 2024-09-10T09:55:44.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.163+0000 7f5cb2a00640 1 -- 172.21.15.77:0/1042704358 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb40035f0 con 0x558720a64c50 2024-09-10T09:55:44.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.163+0000 7f5cd0257740 1 -- 172.21.15.77:0/1042704358 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558720a64c50 msgr2=0x558720a697b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:44.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.163+0000 7f5cd0257740 1 --2- 172.21.15.77:0/1042704358 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558720a64c50 0x558720a697b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x558720a6bd80 tx=0x7f5cb400b410 comp rx=0 tx=0).stop 2024-09-10T09:55:44.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.163+0000 7f5cd0257740 1 -- 172.21.15.77:0/1042704358 shutdown_connections 2024-09-10T09:55:44.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.163+0000 7f5cd0257740 1 --2- 172.21.15.77:0/1042704358 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558720a6f510 0x558720a69cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:44.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.163+0000 7f5cd0257740 1 --2- 172.21.15.77:0/1042704358 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558720a64c50 0x558720a697b0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:44.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.163+0000 7f5cd0257740 1 --2- 172.21.15.77:0/1042704358 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558720a64330 0x558720a6cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:44.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.163+0000 7f5cd0257740 1 -- 172.21.15.77:0/1042704358 >> 172.21.15.77:0/1042704358 conn(0x558720a608c0 msgr2=0x558720a6b620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:44.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.164+0000 7f5cd0257740 1 -- 172.21.15.77:0/1042704358 shutdown_connections 2024-09-10T09:55:44.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.164+0000 7f5cd0257740 1 -- 172.21.15.77:0/1042704358 wait complete. 2024-09-10T09:55:44.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.164+0000 7f5cd0257740 1 -- start start 2024-09-10T09:55:44.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.164+0000 7f5cd0257740 1 Processor -- start 2024-09-10T09:55:44.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.165+0000 7f5cd0257740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558720a64330 0x558720a895e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:44.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.165+0000 7f5cd0257740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558720a64c50 0x558720a89b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:44.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.165+0000 7f5cd0257740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558720a6f510 0x558720a84590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:44.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.165+0000 7f5cd0257740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558720a81200 con 0x558720a6f510 2024-09-10T09:55:44.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.165+0000 7f5cd0257740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558720a80b30 con 0x558720a64330 2024-09-10T09:55:44.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.165+0000 7f5cd0257740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558720a818d0 con 0x558720a64c50 2024-09-10T09:55:44.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.165+0000 7f5cd0257740 10 client.0 _ll_register_callbacks cb 0x7f5cb80f5a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:55:44.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.165+0000 7f5cc6a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558720a64330 0x558720a895e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:44.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.165+0000 7f5cc7400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558720a6f510 0x558720a84590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:44.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.165+0000 7f5cc6a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558720a64330 0x558720a895e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:52124/0 (socket says 172.21.15.77:52124) 2024-09-10T09:55:44.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.165+0000 7f5cc6a00640 1 -- 172.21.15.77:0/1771034169 learned_addr learned my addr 172.21.15.77:0/1771034169 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:55:44.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.165+0000 7f5cc6000640 1 --2- 172.21.15.77:0/1771034169 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558720a64c50 0x558720a89b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:44.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.165+0000 7f5cc6a00640 1 -- 172.21.15.77:0/1771034169 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558720a64c50 msgr2=0x558720a89b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:44.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.165+0000 7f5cc6a00640 1 --2- 172.21.15.77:0/1771034169 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558720a64c50 0x558720a89b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:44.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.165+0000 7f5cc6a00640 1 -- 172.21.15.77:0/1771034169 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558720a6f510 msgr2=0x558720a84590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:44.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.165+0000 7f5cc6a00640 1 --2- 172.21.15.77:0/1771034169 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558720a6f510 0x558720a84590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:44.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.165+0000 7f5cc6a00640 1 -- 172.21.15.77:0/1771034169 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558720a84ad0 con 0x558720a64330 2024-09-10T09:55:44.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.166+0000 7f5cc6a00640 1 --2- 172.21.15.77:0/1771034169 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558720a64330 0x558720a895e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5cb80b8320 tx=0x7f5cb80bbae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=799da693d3d1a7c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:44.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.166+0000 7f5ca7e00640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T09:55:44.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.166+0000 7f5ca7e00640 1 -- 172.21.15.77:0/1771034169 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb80fe040 con 0x558720a64330 2024-09-10T09:55:44.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.166+0000 7f5cd0257740 1 -- 172.21.15.77:0/1771034169 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5cb40043c0 con 0x558720a64330 2024-09-10T09:55:44.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.166+0000 7f5ca7e00640 1 -- 172.21.15.77:0/1771034169 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f5cb80ed070 con 0x558720a64330 2024-09-10T09:55:44.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.166+0000 7f5ca7e00640 1 -- 172.21.15.77:0/1771034169 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5cb802d600 con 0x558720a64330 2024-09-10T09:55:44.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.166+0000 7f5cd0257740 10 client.25415 fetch_fsmap learned FSMap version 29 2024-09-10T09:55:44.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.166+0000 7f5cd0257740 1 -- 172.21.15.77:0/1771034169 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558720a84ad0 con 0x558720a64330 2024-09-10T09:55:44.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.166+0000 7f5ca7e00640 1 -- 172.21.15.77:0/1771034169 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cb40043c0 con 0x558720a64330 2024-09-10T09:55:44.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.167+0000 7f5ca7e00640 1 -- 172.21.15.77:0/1771034169 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f5cb80c45f0 con 0x558720a64330 2024-09-10T09:55:44.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.167+0000 7f5cd0257740 10 client.25415 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T09:55:44.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.167+0000 7f5cd0257740 10 client.25415 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T09:55:44.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.167+0000 7f5cd0257740 20 client.25415 populate_metadata read hostname 'smithi077' 2024-09-10T09:55:44.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.167+0000 7f5cd0257740 1 --2- 172.21.15.77:0/1771034169 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x558720aef5a0 0x558720aee2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:44.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.167+0000 7f5cd0257740 4 client.25415 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T09:55:44.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.167+0000 7f5cd0257740 1 -- 172.21.15.77:0/1771034169 --> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] -- command(tid 0: {"prefix": "status"}) -- 0x5587209d1b30 con 0x558720aef5a0 2024-09-10T09:55:44.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.167+0000 7f5cc6000640 1 --2- 172.21.15.77:0/1771034169 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x558720aef5a0 0x558720aee2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:44.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.168+0000 7f5cc6000640 1 --2- 172.21.15.77:0/1771034169 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x558720aef5a0 0x558720aee2b0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d621715102aba73 server_cookie=a58b077eff08deb5 in_seq=0 out_seq=0 2024-09-10T09:55:44.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.168+0000 7f5ca7e00640 10 client.25415 ms_handle_connect on v2:172.21.15.134:6836/2461497091 2024-09-10T09:55:44.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.168+0000 7f5ca7e00640 1 -- 172.21.15.77:0/1771034169 <== mds.3 v2:172.21.15.134:6836/2461497091 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5587209d1b30 con 0x558720aef5a0 2024-09-10T09:55:44.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.168+0000 7f5ca7e00640 10 client.25415 handle_command_reply: tid=0 2024-09-10T09:55:44.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.168+0000 7f5cd0257740 1 client.25415 shutdown 2024-09-10T09:55:44.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.168+0000 7f5cd0257740 1 -- 172.21.15.77:0/1771034169 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558720a64330 msgr2=0x558720a895e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:44.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.168+0000 7f5cd0257740 1 --2- 172.21.15.77:0/1771034169 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558720a64330 0x558720a895e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5cb80b8320 tx=0x7f5cb80bbae0 comp rx=0 tx=0).stop 2024-09-10T09:55:44.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.168+0000 7f5cd0257740 1 -- 172.21.15.77:0/1771034169 shutdown_connections 2024-09-10T09:55:44.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.168+0000 7f5cd0257740 1 --2- 172.21.15.77:0/1771034169 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x558720aef5a0 0x558720aee2b0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:44.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.168+0000 7f5cd0257740 1 --2- 172.21.15.77:0/1771034169 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558720a6f510 0x558720a84590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:44.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.168+0000 7f5cd0257740 1 --2- 172.21.15.77:0/1771034169 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558720a64c50 0x558720a89b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:44.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.168+0000 7f5ca7e00640 0 client.25415 ms_handle_reset on v2:172.21.15.134:6836/2461497091 2024-09-10T09:55:44.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.168+0000 7f5cd0257740 1 --2- 172.21.15.77:0/1771034169 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558720a64330 0x558720a895e0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:44.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.168+0000 7f5cd0257740 1 -- 172.21.15.77:0/1771034169 >> 172.21.15.77:0/1771034169 conn(0x558720a608c0 msgr2=0x558720a7a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:44.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.168+0000 7f5cd0257740 1 -- 172.21.15.77:0/1771034169 shutdown_connections 2024-09-10T09:55:44.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.168+0000 7f5cd0257740 1 -- 172.21.15.77:0/1771034169 wait complete. 2024-09-10T09:55:44.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.169+0000 7f5cd0257740 20 client.25415 trim_cache size 0 max 16384 2024-09-10T09:55:44.174 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-10T09:55:44.174 INFO:teuthology.orchestra.run.smithi077.stdout: "cluster_fsid": "14a55226-6f59-11ef-bcea-c7b262605968", 2024-09-10T09:55:44.174 INFO:teuthology.orchestra.run.smithi077.stdout: "whoami": 3, 2024-09-10T09:55:44.174 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 24502, 2024-09-10T09:55:44.174 INFO:teuthology.orchestra.run.smithi077.stdout: "want_state": "up:active", 2024-09-10T09:55:44.174 INFO:teuthology.orchestra.run.smithi077.stdout: "state": "up:active", 2024-09-10T09:55:44.174 INFO:teuthology.orchestra.run.smithi077.stdout: "fs_name": "cephfs", 2024-09-10T09:55:44.174 INFO:teuthology.orchestra.run.smithi077.stdout: "rank_uptime": 149.31339226899999, 2024-09-10T09:55:44.174 INFO:teuthology.orchestra.run.smithi077.stdout: "mdsmap_epoch": 29, 2024-09-10T09:55:44.174 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch": 96, 2024-09-10T09:55:44.174 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch_barrier": 96, 2024-09-10T09:55:44.175 INFO:teuthology.orchestra.run.smithi077.stdout: "uptime": 229.17790766799999 2024-09-10T09:55:44.175 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-10T09:55:44.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.169+0000 7f5ccd600640 1 -- 172.21.15.77:0/3715118214 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f5cac0a2c00 msgr2=0x7f5cac0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:44.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.169+0000 7f5ccd600640 1 --2- 172.21.15.77:0/3715118214 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f5cac0a2c00 0x7f5cac0a50c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f5cc0005fd0 tx=0x7f5cc0009500 comp rx=0 tx=0).stop 2024-09-10T09:55:44.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.169+0000 7f5ccd600640 1 -- 172.21.15.77:0/3715118214 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5cc8074620 msgr2=0x7f5cc819a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:44.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.169+0000 7f5ccd600640 1 --2- 172.21.15.77:0/3715118214 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5cc8074620 0x7f5cc819a680 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5cb8006010 tx=0x7f5cb80042f0 comp rx=0 tx=0).stop 2024-09-10T09:55:44.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.169+0000 7f5ccd600640 1 -- 172.21.15.77:0/3715118214 shutdown_connections 2024-09-10T09:55:44.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.169+0000 7f5ccd600640 1 --2- 172.21.15.77:0/3715118214 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f5cac0a2c00 0x7f5cac0a50c0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:44.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.169+0000 7f5ccd600640 1 --2- 172.21.15.77:0/3715118214 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5cc810e3c0 0x7f5cc819ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:44.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.169+0000 7f5ccd600640 1 --2- 172.21.15.77:0/3715118214 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5cc8074f40 0x7f5cc819abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:44.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.169+0000 7f5ccd600640 1 --2- 172.21.15.77:0/3715118214 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5cc8074620 0x7f5cc819a680 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:44.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.169+0000 7f5ccd600640 1 -- 172.21.15.77:0/3715118214 >> 172.21.15.77:0/3715118214 conn(0x7f5cc80fd8c0 msgr2=0x7f5cc80ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:44.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.170+0000 7f5ccd600640 1 -- 172.21.15.77:0/3715118214 shutdown_connections 2024-09-10T09:55:44.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:44.170+0000 7f5ccd600640 1 -- 172.21.15.77:0/3715118214 wait complete. 2024-09-10T09:55:44.955 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.i status 2024-09-10T09:55:45.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:55:44 smithi134 ceph-mon[35674]: pgmap v408: 129 pgs: 129 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-10T09:55:45.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:55:44 smithi120 ceph-mon[34770]: pgmap v408: 129 pgs: 129 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-10T09:55:45.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:55:44 smithi077 ceph-mon[30319]: pgmap v408: 129 pgs: 129 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-10T09:55:45.337 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:55:45.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.938+0000 7fcf1b800640 1 -- 172.21.15.77:0/668984806 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcf14074620 msgr2=0x7fcf14074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:45.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.938+0000 7fcf1b800640 1 --2- 172.21.15.77:0/668984806 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcf14074620 0x7fcf14074a00 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fcf04009900 tx=0x7fcf0402a480 comp rx=0 tx=0).stop 2024-09-10T09:55:45.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.938+0000 7fcf1b800640 1 -- 172.21.15.77:0/668984806 shutdown_connections 2024-09-10T09:55:45.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.938+0000 7fcf1b800640 1 --2- 172.21.15.77:0/668984806 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf1410e3c0 0x7fcf14110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:45.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.938+0000 7fcf1b800640 1 --2- 172.21.15.77:0/668984806 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcf14074f40 0x7fcf1410de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:45.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.938+0000 7fcf1b800640 1 --2- 172.21.15.77:0/668984806 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcf14074620 0x7fcf14074a00 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:45.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.938+0000 7fcf1b800640 1 -- 172.21.15.77:0/668984806 >> 172.21.15.77:0/668984806 conn(0x7fcf140fd8c0 msgr2=0x7fcf140ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:45.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.938+0000 7fcf1b800640 1 -- 172.21.15.77:0/668984806 shutdown_connections 2024-09-10T09:55:45.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.938+0000 7fcf1b800640 1 -- 172.21.15.77:0/668984806 wait complete. 2024-09-10T09:55:45.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.939+0000 7fcf1b800640 1 Processor -- start 2024-09-10T09:55:45.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.939+0000 7fcf1b800640 1 -- start start 2024-09-10T09:55:45.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.939+0000 7fcf1b800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcf14074620 0x7fcf1419a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:45.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.939+0000 7fcf1b800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcf14074f40 0x7fcf1419abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:45.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.939+0000 7fcf1b800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf1410e3c0 0x7fcf1419ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:45.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.939+0000 7fcf1b800640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fcf141141a0 con 0x7fcf14074620 2024-09-10T09:55:45.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.939+0000 7fcf1b800640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fcf14114020 con 0x7fcf14074f40 2024-09-10T09:55:45.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.939+0000 7fcf1b800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcf14114320 con 0x7fcf1410e3c0 2024-09-10T09:55:45.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.939+0000 7fcf18e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcf14074620 0x7fcf1419a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:45.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.939+0000 7fcf18e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcf14074620 0x7fcf1419a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33172/0 (socket says 172.21.15.77:33172) 2024-09-10T09:55:45.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.939+0000 7fcf18e00640 1 -- 172.21.15.77:0/746422086 learned_addr learned my addr 172.21.15.77:0/746422086 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:55:45.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.940+0000 7fcf13e00640 1 --2- 172.21.15.77:0/746422086 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcf14074f40 0x7fcf1419abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:45.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.940+0000 7fcf19800640 1 --2- 172.21.15.77:0/746422086 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf1410e3c0 0x7fcf1419ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:45.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.940+0000 7fcf18e00640 1 -- 172.21.15.77:0/746422086 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf1410e3c0 msgr2=0x7fcf1419ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:45.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.940+0000 7fcf18e00640 1 --2- 172.21.15.77:0/746422086 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf1410e3c0 0x7fcf1419ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:45.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.940+0000 7fcf18e00640 1 -- 172.21.15.77:0/746422086 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcf14074f40 msgr2=0x7fcf1419abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:45.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.940+0000 7fcf18e00640 1 --2- 172.21.15.77:0/746422086 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcf14074f40 0x7fcf1419abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:45.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.940+0000 7fcf18e00640 1 -- 172.21.15.77:0/746422086 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf1419f620 con 0x7fcf14074620 2024-09-10T09:55:45.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.940+0000 7fcf18e00640 1 --2- 172.21.15.77:0/746422086 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcf14074620 0x7fcf1419a670 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fcf04004770 tx=0x7fcf04003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:45.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.941+0000 7fcf11600640 1 -- 172.21.15.77:0/746422086 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf0403f070 con 0x7fcf14074620 2024-09-10T09:55:45.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.941+0000 7fcf1b800640 1 -- 172.21.15.77:0/746422086 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf1419f8b0 con 0x7fcf14074620 2024-09-10T09:55:45.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.941+0000 7fcf11600640 1 -- 172.21.15.77:0/746422086 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf04031ca0 con 0x7fcf14074620 2024-09-10T09:55:45.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.941+0000 7fcf11600640 1 -- 172.21.15.77:0/746422086 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf04035030 con 0x7fcf14074620 2024-09-10T09:55:45.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.941+0000 7fcf1b800640 1 -- 172.21.15.77:0/746422086 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf141a71f0 con 0x7fcf14074620 2024-09-10T09:55:45.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.944+0000 7fcf11600640 1 -- 172.21.15.77:0/746422086 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fcf04042070 con 0x7fcf14074620 2024-09-10T09:55:45.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.944+0000 7fcf1e270740 1 Processor -- start 2024-09-10T09:55:45.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.944+0000 7fcf1e270740 1 -- start start 2024-09-10T09:55:45.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.944+0000 7fcf1e270740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576c45c9b40 0x5576c45c9f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:45.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.944+0000 7fcf1e270740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576c45ca460 0x5576c45d47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:45.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.944+0000 7fcf1e270740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576c45d4d20 0x5576c45d7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:45.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.944+0000 7fcf1e270740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5576c45270b0 con 0x5576c45ca460 2024-09-10T09:55:45.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.944+0000 7fcf1e270740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5576c44ceab0 con 0x5576c45d4d20 2024-09-10T09:55:45.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.944+0000 7fcf1e270740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5576c4523070 con 0x5576c45c9b40 2024-09-10T09:55:45.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.945+0000 7fcf18e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576c45ca460 0x5576c45d47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:45.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.945+0000 7fcf19800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576c45c9b40 0x5576c45c9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:45.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.945+0000 7fcf13e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576c45d4d20 0x5576c45d7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:45.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.945+0000 7fcf18e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576c45ca460 0x5576c45d47e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33182/0 (socket says 172.21.15.77:33182) 2024-09-10T09:55:45.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.945+0000 7fcf18e00640 1 -- 172.21.15.77:0/290130562 learned_addr learned my addr 172.21.15.77:0/290130562 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:55:45.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.945+0000 7fcf19800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576c45c9b40 0x5576c45c9f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:39280/0 (socket says 172.21.15.77:39280) 2024-09-10T09:55:45.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.945+0000 7fcf19800640 1 -- 172.21.15.77:0/290130562 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576c45d4d20 msgr2=0x5576c45d7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:45.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.945+0000 7fcf19800640 1 --2- 172.21.15.77:0/290130562 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576c45d4d20 0x5576c45d7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:45.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.945+0000 7fcf19800640 1 -- 172.21.15.77:0/290130562 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576c45ca460 msgr2=0x5576c45d47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:45.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.945+0000 7fcf19800640 1 --2- 172.21.15.77:0/290130562 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576c45ca460 0x5576c45d47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:45.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.945+0000 7fcf19800640 1 -- 172.21.15.77:0/290130562 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576c44db4a0 con 0x5576c45c9b40 2024-09-10T09:55:45.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.945+0000 7fcf13e00640 1 --2- 172.21.15.77:0/290130562 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576c45d4d20 0x5576c45d7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:55:45.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.945+0000 7fcf11600640 1 --2- 172.21.15.77:0/746422086 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fcef80a2b80 0x7fcef80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:45.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.946+0000 7fcf19800640 1 --2- 172.21.15.77:0/290130562 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576c45c9b40 0x5576c45c9f20 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c009da0 tx=0x7fcf0c010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43da332ec544e323 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:45.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.946+0000 7fcf18e00640 1 --2- 172.21.15.77:0/746422086 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fcef80a2b80 0x7fcef80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:45.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.946+0000 7fcf11600640 1 -- 172.21.15.77:0/746422086 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fcf040efbe0 con 0x7fcf14074620 2024-09-10T09:55:45.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.946+0000 7fcf0a000640 1 -- 172.21.15.77:0/290130562 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c016070 con 0x5576c45c9b40 2024-09-10T09:55:45.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.946+0000 7fcf0a000640 1 -- 172.21.15.77:0/290130562 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf0c010e50 con 0x5576c45c9b40 2024-09-10T09:55:45.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.946+0000 7fcf0a000640 1 -- 172.21.15.77:0/290130562 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c012740 con 0x5576c45c9b40 2024-09-10T09:55:45.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.947+0000 7fcf1e270740 1 -- 172.21.15.77:0/290130562 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576c45c9b40 msgr2=0x5576c45c9f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:45.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.947+0000 7fcf18e00640 1 --2- 172.21.15.77:0/746422086 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fcef80a2b80 0x7fcef80a5040 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fcf040efe80 tx=0x7fcf040f3620 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:45.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.947+0000 7fcf1e270740 1 --2- 172.21.15.77:0/290130562 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576c45c9b40 0x5576c45c9f20 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c009da0 tx=0x7fcf0c010740 comp rx=0 tx=0).stop 2024-09-10T09:55:45.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.947+0000 7fcf1e270740 1 -- 172.21.15.77:0/290130562 shutdown_connections 2024-09-10T09:55:45.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.947+0000 7fcf1e270740 1 --2- 172.21.15.77:0/290130562 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576c45d4d20 0x5576c45d7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:45.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.947+0000 7fcf1e270740 1 --2- 172.21.15.77:0/290130562 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576c45ca460 0x5576c45d47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:45.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.947+0000 7fcf1e270740 1 --2- 172.21.15.77:0/290130562 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576c45c9b40 0x5576c45c9f20 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:45.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.947+0000 7fcf1e270740 1 -- 172.21.15.77:0/290130562 >> 172.21.15.77:0/290130562 conn(0x5576c45c60d0 msgr2=0x5576c45c64b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:45.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.947+0000 7fcf1e270740 1 -- 172.21.15.77:0/290130562 shutdown_connections 2024-09-10T09:55:45.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.947+0000 7fcf1e270740 1 -- 172.21.15.77:0/290130562 wait complete. 2024-09-10T09:55:45.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.947+0000 7fcf1e270740 1 -- start start 2024-09-10T09:55:45.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.948+0000 7fcf1e270740 1 Processor -- start 2024-09-10T09:55:45.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.948+0000 7fcf1e270740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576c45c9b40 0x5576c45d1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:45.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.948+0000 7fcf1e270740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576c45ca460 0x5576c45d2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:45.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.948+0000 7fcf1e270740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576c45d4d20 0x5576c45d2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:45.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.948+0000 7fcf1e270740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5576c45d7990 con 0x5576c45ca460 2024-09-10T09:55:45.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.948+0000 7fcf1e270740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5576c44af920 con 0x5576c45c9b40 2024-09-10T09:55:45.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.948+0000 7fcf1e270740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5576c45d7b10 con 0x5576c45d4d20 2024-09-10T09:55:45.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.948+0000 7fcf1e270740 10 client.0 _ll_register_callbacks cb 0x5576c44d38a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:55:45.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.948+0000 7fcf13e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576c45ca460 0x5576c45d2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:45.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.948+0000 7fcf13e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576c45ca460 0x5576c45d2140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33192/0 (socket says 172.21.15.77:33192) 2024-09-10T09:55:45.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.948+0000 7fcf13e00640 1 -- 172.21.15.77:0/2148792451 learned_addr learned my addr 172.21.15.77:0/2148792451 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:55:45.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.949+0000 7fcf19800640 1 --2- 172.21.15.77:0/2148792451 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576c45c9b40 0x5576c45d1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:45.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.949+0000 7fcf19800640 1 --2- 172.21.15.77:0/2148792451 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576c45d4d20 0x5576c45d2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:45.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.949+0000 7fcf13e00640 1 -- 172.21.15.77:0/2148792451 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576c45d4d20 msgr2=0x5576c45d2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:45.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.949+0000 7fcf13e00640 1 --2- 172.21.15.77:0/2148792451 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576c45d4d20 0x5576c45d2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:45.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.949+0000 7fcf13e00640 1 -- 172.21.15.77:0/2148792451 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576c45c9b40 msgr2=0x5576c45d1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:45.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.949+0000 7fcf13e00640 1 --2- 172.21.15.77:0/2148792451 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576c45c9b40 0x5576c45d1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:45.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.949+0000 7fcf13e00640 1 -- 172.21.15.77:0/2148792451 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576c42982b0 con 0x5576c45ca460 2024-09-10T09:55:45.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.949+0000 7fcf19800640 1 --2- 172.21.15.77:0/2148792451 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576c45c9b40 0x5576c45d1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:55:45.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.949+0000 7fcf13e00640 1 --2- 172.21.15.77:0/2148792451 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576c45ca460 0x5576c45d2140 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fcf00012730 tx=0x7fcf00012bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e2b960ae6635d8e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:45.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.949+0000 7fcefea00640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-10T09:55:45.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.949+0000 7fcefea00640 1 -- 172.21.15.77:0/2148792451 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf00004030 con 0x5576c45ca460 2024-09-10T09:55:45.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.950+0000 7fcefea00640 1 -- 172.21.15.77:0/2148792451 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fcf000041d0 con 0x5576c45ca460 2024-09-10T09:55:45.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.950+0000 7fcf1e270740 1 -- 172.21.15.77:0/2148792451 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf0c010e50 con 0x5576c45ca460 2024-09-10T09:55:45.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.950+0000 7fcefea00640 1 -- 172.21.15.77:0/2148792451 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcf00025050 con 0x5576c45ca460 2024-09-10T09:55:45.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.950+0000 7fcf1e270740 10 client.15381 fetch_fsmap learned FSMap version 29 2024-09-10T09:55:45.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.950+0000 7fcf1e270740 1 -- 172.21.15.77:0/2148792451 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5576c45e36a0 con 0x5576c45ca460 2024-09-10T09:55:45.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.950+0000 7fcefea00640 1 -- 172.21.15.77:0/2148792451 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf0c010e50 con 0x5576c45ca460 2024-09-10T09:55:45.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.950+0000 7fcefea00640 1 -- 172.21.15.77:0/2148792451 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fcf00003a90 con 0x5576c45ca460 2024-09-10T09:55:45.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.950+0000 7fcf1e270740 10 client.15381 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T09:55:45.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.950+0000 7fcf1e270740 10 client.15381 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-10T09:55:45.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.950+0000 7fcf1e270740 20 client.15381 populate_metadata read hostname 'smithi077' 2024-09-10T09:55:45.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.950+0000 7fcf1e270740 1 --2- 172.21.15.77:0/2148792451 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x5576c45e5d00 0x5576c45e7170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:45.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.950+0000 7fcf1e270740 4 client.15381 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:55:45.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.950+0000 7fcf1e270740 1 -- 172.21.15.77:0/2148792451 --> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5576c4534620 con 0x5576c45e5d00 2024-09-10T09:55:45.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.951+0000 7fcf19800640 1 --2- 172.21.15.77:0/2148792451 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x5576c45e5d00 0x5576c45e7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:45.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.951+0000 7fcf19800640 1 --2- 172.21.15.77:0/2148792451 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x5576c45e5d00 0x5576c45e7170 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=21a250e0484ea68c server_cookie=4acf4a1b32d89023 in_seq=0 out_seq=0 2024-09-10T09:55:45.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.951+0000 7fcefea00640 10 client.15381 ms_handle_connect on v2:172.21.15.134:6838/1965117680 2024-09-10T09:55:45.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.952+0000 7fcefea00640 1 -- 172.21.15.77:0/2148792451 <== mds.2 v2:172.21.15.134:6838/1965117680 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5576c4534620 con 0x5576c45e5d00 2024-09-10T09:55:45.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.952+0000 7fcefea00640 10 client.15381 handle_command_reply: tid=0 2024-09-10T09:55:45.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.952+0000 7fcf1e270740 1 client.15381 shutdown 2024-09-10T09:55:45.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.952+0000 7fcf1e270740 1 -- 172.21.15.77:0/2148792451 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576c45ca460 msgr2=0x5576c45d2140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:45.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.952+0000 7fcf1e270740 1 --2- 172.21.15.77:0/2148792451 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576c45ca460 0x5576c45d2140 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fcf00012730 tx=0x7fcf00012bf0 comp rx=0 tx=0).stop 2024-09-10T09:55:45.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.952+0000 7fcf1e270740 1 -- 172.21.15.77:0/2148792451 shutdown_connections 2024-09-10T09:55:45.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.952+0000 7fcf1e270740 1 --2- 172.21.15.77:0/2148792451 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x5576c45e5d00 0x5576c45e7170 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:45.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.952+0000 7fcf1e270740 1 --2- 172.21.15.77:0/2148792451 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576c45d4d20 0x5576c45d2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:45.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.952+0000 7fcf1e270740 1 --2- 172.21.15.77:0/2148792451 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576c45ca460 0x5576c45d2140 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:45.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.952+0000 7fcf1e270740 1 --2- 172.21.15.77:0/2148792451 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576c45c9b40 0x5576c45d1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:45.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.952+0000 7fcefea00640 0 client.15381 ms_handle_reset on v2:172.21.15.134:6838/1965117680 2024-09-10T09:55:45.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.952+0000 7fcf1e270740 1 -- 172.21.15.77:0/2148792451 >> 172.21.15.77:0/2148792451 conn(0x5576c45c60d0 msgr2=0x5576c45bca20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:45.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.953+0000 7fcf1e270740 1 -- 172.21.15.77:0/2148792451 shutdown_connections 2024-09-10T09:55:45.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.953+0000 7fcf1e270740 1 -- 172.21.15.77:0/2148792451 wait complete. 2024-09-10T09:55:45.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.953+0000 7fcf1e270740 20 client.15381 trim_cache size 0 max 16384 2024-09-10T09:55:45.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.971+0000 7fcf1e270740 1 Processor -- start 2024-09-10T09:55:45.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.971+0000 7fcf1e270740 1 -- start start 2024-09-10T09:55:45.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.971+0000 7fcf1e270740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576c45c9b40 0x5576c45cc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:45.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.972+0000 7fcf1e270740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576c45ca460 0x5576c45d04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:45.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.972+0000 7fcf1e270740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576c45d4d20 0x5576c45d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:45.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.972+0000 7fcf1e270740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5576c45e54b0 con 0x5576c45d4d20 2024-09-10T09:55:45.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.972+0000 7fcf1e270740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5576c45e5b80 con 0x5576c45ca460 2024-09-10T09:55:45.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.972+0000 7fcf1e270740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5576c45e6250 con 0x5576c45c9b40 2024-09-10T09:55:45.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.972+0000 7fcf19800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576c45d4d20 0x5576c45d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:45.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.972+0000 7fcf13e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576c45ca460 0x5576c45d04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:45.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.972+0000 7fcf19800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576c45c9b40 0x5576c45cc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:45.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.972+0000 7fcf13e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576c45ca460 0x5576c45d04d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:52168/0 (socket says 172.21.15.77:52168) 2024-09-10T09:55:45.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.972+0000 7fcf13e00640 1 -- 172.21.15.77:0/1480981264 learned_addr learned my addr 172.21.15.77:0/1480981264 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:55:45.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.972+0000 7fcf13e00640 1 -- 172.21.15.77:0/1480981264 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576c45c9b40 msgr2=0x5576c45cc5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:45.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.972+0000 7fcf13e00640 1 --2- 172.21.15.77:0/1480981264 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576c45c9b40 0x5576c45cc5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:45.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.972+0000 7fcf13e00640 1 -- 172.21.15.77:0/1480981264 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576c45d4d20 msgr2=0x5576c45d24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:45.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.972+0000 7fcf13e00640 1 --2- 172.21.15.77:0/1480981264 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576c45d4d20 0x5576c45d24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:45.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.972+0000 7fcf13e00640 1 -- 172.21.15.77:0/1480981264 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576c45d0bb0 con 0x5576c45ca460 2024-09-10T09:55:45.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.972+0000 7fcf19800640 1 --2- 172.21.15.77:0/1480981264 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576c45c9b40 0x5576c45cc5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:55:45.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.973+0000 7fcf13e00640 1 --2- 172.21.15.77:0/1480981264 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576c45ca460 0x5576c45d04d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fcf00012420 tx=0x7fcf00005840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d20192c282915ba0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:45.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.973+0000 7fcf09600640 1 -- 172.21.15.77:0/1480981264 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf0002e070 con 0x5576c45ca460 2024-09-10T09:55:45.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.973+0000 7fcf09600640 1 -- 172.21.15.77:0/1480981264 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf00010500 con 0x5576c45ca460 2024-09-10T09:55:45.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.973+0000 7fcf09600640 1 -- 172.21.15.77:0/1480981264 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf00022040 con 0x5576c45ca460 2024-09-10T09:55:45.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.974+0000 7fcf1e270740 1 -- 172.21.15.77:0/1480981264 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576c45ca460 msgr2=0x5576c45d04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:45.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.974+0000 7fcf1e270740 1 --2- 172.21.15.77:0/1480981264 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576c45ca460 0x5576c45d04d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fcf00012420 tx=0x7fcf00005840 comp rx=0 tx=0).stop 2024-09-10T09:55:45.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.974+0000 7fcf1e270740 1 -- 172.21.15.77:0/1480981264 shutdown_connections 2024-09-10T09:55:45.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.974+0000 7fcf1e270740 1 --2- 172.21.15.77:0/1480981264 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576c45d4d20 0x5576c45d24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:45.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.974+0000 7fcf1e270740 1 --2- 172.21.15.77:0/1480981264 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576c45ca460 0x5576c45d04d0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:45.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.974+0000 7fcf1e270740 1 --2- 172.21.15.77:0/1480981264 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576c45c9b40 0x5576c45cc5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:45.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.974+0000 7fcf1e270740 1 -- 172.21.15.77:0/1480981264 >> 172.21.15.77:0/1480981264 conn(0x5576c45c60d0 msgr2=0x5576c45dfd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:45.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.974+0000 7fcf1e270740 1 -- 172.21.15.77:0/1480981264 shutdown_connections 2024-09-10T09:55:45.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.974+0000 7fcf1e270740 1 -- 172.21.15.77:0/1480981264 wait complete. 2024-09-10T09:55:45.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.974+0000 7fcf1e270740 1 -- start start 2024-09-10T09:55:45.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.974+0000 7fcf1e270740 1 Processor -- start 2024-09-10T09:55:45.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.975+0000 7fcf1e270740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576c45c9b40 0x5576c45efcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:45.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.975+0000 7fcf1e270740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576c45ca460 0x5576c45e9000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:45.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.975+0000 7fcf1e270740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576c45d4d20 0x5576c45e9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:45.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.975+0000 7fcf1e270740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5576c45e6ff0 con 0x5576c45d4d20 2024-09-10T09:55:45.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.975+0000 7fcf1e270740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5576c45e6920 con 0x5576c45c9b40 2024-09-10T09:55:45.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.975+0000 7fcf1e270740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5576c45e76c0 con 0x5576c45ca460 2024-09-10T09:55:45.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.975+0000 7fcf1e270740 10 client.0 _ll_register_callbacks cb 0x7fcf00009ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:55:45.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.975+0000 7fcf19800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576c45d4d20 0x5576c45e9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:45.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.975+0000 7fcf13e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576c45ca460 0x5576c45e9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:45.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.975+0000 7fcf19800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576c45d4d20 0x5576c45e9740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33220/0 (socket says 172.21.15.77:33220) 2024-09-10T09:55:45.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.975+0000 7fcf19800640 1 -- 172.21.15.77:0/1274476458 learned_addr learned my addr 172.21.15.77:0/1274476458 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:55:45.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.975+0000 7fcf13e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576c45ca460 0x5576c45e9000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:39302/0 (socket says 172.21.15.77:39302) 2024-09-10T09:55:45.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.975+0000 7fcf19800640 1 --2- 172.21.15.77:0/1274476458 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576c45c9b40 0x5576c45efcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:45.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.976+0000 7fcf13e00640 1 -- 172.21.15.77:0/1274476458 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576c45c9b40 msgr2=0x5576c45efcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:45.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.976+0000 7fcf13e00640 1 --2- 172.21.15.77:0/1274476458 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576c45c9b40 0x5576c45efcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:45.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.976+0000 7fcf13e00640 1 -- 172.21.15.77:0/1274476458 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576c45d4d20 msgr2=0x5576c45e9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:45.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.976+0000 7fcf13e00640 1 --2- 172.21.15.77:0/1274476458 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576c45d4d20 0x5576c45e9740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:45.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.976+0000 7fcf13e00640 1 -- 172.21.15.77:0/1274476458 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576c45ed7f0 con 0x5576c45ca460 2024-09-10T09:55:45.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.976+0000 7fcf19800640 1 --2- 172.21.15.77:0/1274476458 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576c45d4d20 0x5576c45e9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:55:45.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.976+0000 7fcf13e00640 1 --2- 172.21.15.77:0/1274476458 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576c45ca460 0x5576c45e9000 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x5576c45e1ad0 tx=0x7fcf00012bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18e125fce785cd4e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:45.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.976+0000 7fcefea00640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-10T09:55:45.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.976+0000 7fcefea00640 1 -- 172.21.15.77:0/1274476458 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf0002e070 con 0x5576c45ca460 2024-09-10T09:55:45.982 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.976+0000 7fcf1e270740 1 -- 172.21.15.77:0/1274476458 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf00010500 con 0x5576c45ca460 2024-09-10T09:55:45.982 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.976+0000 7fcefea00640 1 -- 172.21.15.77:0/1274476458 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fcf000050d0 con 0x5576c45ca460 2024-09-10T09:55:45.982 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.977+0000 7fcefea00640 1 -- 172.21.15.77:0/1274476458 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcf000052f0 con 0x5576c45ca460 2024-09-10T09:55:45.982 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.977+0000 7fcf1e270740 10 client.25408 fetch_fsmap learned FSMap version 29 2024-09-10T09:55:45.982 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.977+0000 7fcf1e270740 1 -- 172.21.15.77:0/1274476458 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5576c46505d0 con 0x5576c45ca460 2024-09-10T09:55:45.982 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.977+0000 7fcefea00640 1 -- 172.21.15.77:0/1274476458 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf00010500 con 0x5576c45ca460 2024-09-10T09:55:45.982 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.977+0000 7fcefea00640 1 -- 172.21.15.77:0/1274476458 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fcf0000f9c0 con 0x5576c45ca460 2024-09-10T09:55:45.982 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.977+0000 7fcf1e270740 10 client.25408 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T09:55:45.982 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.977+0000 7fcf1e270740 10 client.25408 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-10T09:55:45.982 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.977+0000 7fcf1e270740 20 client.25408 populate_metadata read hostname 'smithi077' 2024-09-10T09:55:45.982 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.978+0000 7fcf1e270740 1 --2- 172.21.15.77:0/1274476458 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x5576c46526a0 0x5576c4653b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:45.982 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.978+0000 7fcf1e270740 4 client.25408 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T09:55:45.982 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.978+0000 7fcf1e270740 1 -- 172.21.15.77:0/1274476458 --> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] -- command(tid 0: {"prefix": "status"}) -- 0x5576c452eb30 con 0x5576c46526a0 2024-09-10T09:55:45.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.978+0000 7fcf19800640 1 --2- 172.21.15.77:0/1274476458 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x5576c46526a0 0x5576c4653b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:45.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.978+0000 7fcf19800640 1 --2- 172.21.15.77:0/1274476458 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x5576c46526a0 0x5576c4653b10 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e43c339e47d481eb server_cookie=c2c31f86258c0700 in_seq=0 out_seq=0 2024-09-10T09:55:45.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.978+0000 7fcefea00640 10 client.25408 ms_handle_connect on v2:172.21.15.134:6838/1965117680 2024-09-10T09:55:45.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.979+0000 7fcefea00640 1 -- 172.21.15.77:0/1274476458 <== mds.2 v2:172.21.15.134:6838/1965117680 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x7fcf0c030160 con 0x5576c46526a0 2024-09-10T09:55:45.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.979+0000 7fcefea00640 10 client.25408 handle_command_reply: tid=0 2024-09-10T09:55:45.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.979+0000 7fcf1e270740 1 client.25408 shutdown 2024-09-10T09:55:45.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.979+0000 7fcf1e270740 1 -- 172.21.15.77:0/1274476458 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576c45ca460 msgr2=0x5576c45e9000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:45.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.979+0000 7fcf1e270740 1 --2- 172.21.15.77:0/1274476458 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576c45ca460 0x5576c45e9000 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x5576c45e1ad0 tx=0x7fcf00012bf0 comp rx=0 tx=0).stop 2024-09-10T09:55:45.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.979+0000 7fcf1e270740 1 -- 172.21.15.77:0/1274476458 shutdown_connections 2024-09-10T09:55:45.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.979+0000 7fcf1e270740 1 --2- 172.21.15.77:0/1274476458 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x5576c46526a0 0x5576c4653b10 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:45.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.979+0000 7fcf1e270740 1 --2- 172.21.15.77:0/1274476458 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576c45d4d20 0x5576c45e9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:45.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.979+0000 7fcf1e270740 1 --2- 172.21.15.77:0/1274476458 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576c45ca460 0x5576c45e9000 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:45.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.979+0000 7fcefea00640 0 client.25408 ms_handle_reset on v2:172.21.15.134:6838/1965117680 2024-09-10T09:55:45.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.979+0000 7fcf1e270740 1 --2- 172.21.15.77:0/1274476458 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576c45c9b40 0x5576c45efcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:45.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.979+0000 7fcf1e270740 1 -- 172.21.15.77:0/1274476458 >> 172.21.15.77:0/1274476458 conn(0x5576c45c60d0 msgr2=0x5576c45bc200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:45.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.979+0000 7fcf1e270740 1 -- 172.21.15.77:0/1274476458 shutdown_connections 2024-09-10T09:55:45.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.979+0000 7fcf1e270740 1 -- 172.21.15.77:0/1274476458 wait complete. 2024-09-10T09:55:45.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.979+0000 7fcf1e270740 20 client.25408 trim_cache size 0 max 16384 2024-09-10T09:55:45.984 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-10T09:55:45.984 INFO:teuthology.orchestra.run.smithi077.stdout: "cluster_fsid": "14a55226-6f59-11ef-bcea-c7b262605968", 2024-09-10T09:55:45.984 INFO:teuthology.orchestra.run.smithi077.stdout: "whoami": 2, 2024-09-10T09:55:45.984 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 24520, 2024-09-10T09:55:45.984 INFO:teuthology.orchestra.run.smithi077.stdout: "want_state": "up:active", 2024-09-10T09:55:45.984 INFO:teuthology.orchestra.run.smithi077.stdout: "state": "up:active", 2024-09-10T09:55:45.985 INFO:teuthology.orchestra.run.smithi077.stdout: "fs_name": "cephfs", 2024-09-10T09:55:45.985 INFO:teuthology.orchestra.run.smithi077.stdout: "rank_uptime": 152.13017673300001, 2024-09-10T09:55:45.985 INFO:teuthology.orchestra.run.smithi077.stdout: "mdsmap_epoch": 29, 2024-09-10T09:55:45.985 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch": 96, 2024-09-10T09:55:45.985 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch_barrier": 96, 2024-09-10T09:55:45.985 INFO:teuthology.orchestra.run.smithi077.stdout: "uptime": 225.58176982800001 2024-09-10T09:55:45.985 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-10T09:55:45.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.980+0000 7fcf1b800640 1 -- 172.21.15.77:0/746422086 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fcef80a2b80 msgr2=0x7fcef80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:45.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.980+0000 7fcf1b800640 1 --2- 172.21.15.77:0/746422086 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fcef80a2b80 0x7fcef80a5040 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fcf040efe80 tx=0x7fcf040f3620 comp rx=0 tx=0).stop 2024-09-10T09:55:45.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.980+0000 7fcf1b800640 1 -- 172.21.15.77:0/746422086 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcf14074620 msgr2=0x7fcf1419a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:45.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.980+0000 7fcf1b800640 1 --2- 172.21.15.77:0/746422086 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcf14074620 0x7fcf1419a670 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fcf04004770 tx=0x7fcf04003c10 comp rx=0 tx=0).stop 2024-09-10T09:55:45.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.980+0000 7fcf1b800640 1 -- 172.21.15.77:0/746422086 shutdown_connections 2024-09-10T09:55:45.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.980+0000 7fcf1b800640 1 --2- 172.21.15.77:0/746422086 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fcef80a2b80 0x7fcef80a5040 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:45.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.980+0000 7fcf1b800640 1 --2- 172.21.15.77:0/746422086 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf1410e3c0 0x7fcf1419ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:45.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.980+0000 7fcf1b800640 1 --2- 172.21.15.77:0/746422086 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcf14074f40 0x7fcf1419abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:45.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.980+0000 7fcf1b800640 1 --2- 172.21.15.77:0/746422086 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcf14074620 0x7fcf1419a670 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:45.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.980+0000 7fcf1b800640 1 -- 172.21.15.77:0/746422086 >> 172.21.15.77:0/746422086 conn(0x7fcf140fd8c0 msgr2=0x7fcf140fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:45.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.980+0000 7fcf1b800640 1 -- 172.21.15.77:0/746422086 shutdown_connections 2024-09-10T09:55:45.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:45.980+0000 7fcf1b800640 1 -- 172.21.15.77:0/746422086 wait complete. 2024-09-10T09:55:46.875 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:55:46 smithi077 ceph-mon[30319]: pgmap v409: 129 pgs: 129 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-10T09:55:46.917 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.h status 2024-09-10T09:55:47.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:55:46 smithi134 ceph-mon[35674]: pgmap v409: 129 pgs: 129 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-10T09:55:47.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:55:46 smithi120 ceph-mon[34770]: pgmap v409: 129 pgs: 129 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-10T09:55:47.292 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:55:47.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.824+0000 7f570c400640 1 -- 172.21.15.77:0/2735847328 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5704106f10 msgr2=0x7f570410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:47.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.824+0000 7f570c400640 1 --2- 172.21.15.77:0/2735847328 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5704106f10 0x7f570410ea90 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f56fc009ca0 tx=0x7f56fc02a8c0 comp rx=0 tx=0).stop 2024-09-10T09:55:47.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.824+0000 7f570c400640 1 -- 172.21.15.77:0/2735847328 shutdown_connections 2024-09-10T09:55:47.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.824+0000 7f570c400640 1 --2- 172.21.15.77:0/2735847328 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5704106f10 0x7f570410ea90 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:47.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.824+0000 7f570c400640 1 --2- 172.21.15.77:0/2735847328 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5704102790 0x7f57041067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:47.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.824+0000 7f570c400640 1 --2- 172.21.15.77:0/2735847328 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5704101de0 0x7f57041021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:47.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.824+0000 7f570c400640 1 -- 172.21.15.77:0/2735847328 >> 172.21.15.77:0/2735847328 conn(0x7f57040fd8c0 msgr2=0x7f57040ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:47.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.824+0000 7f570c400640 1 -- 172.21.15.77:0/2735847328 shutdown_connections 2024-09-10T09:55:47.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.824+0000 7f570c400640 1 -- 172.21.15.77:0/2735847328 wait complete. 2024-09-10T09:55:47.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.825+0000 7f570c400640 1 Processor -- start 2024-09-10T09:55:47.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.825+0000 7f570c400640 1 -- start start 2024-09-10T09:55:47.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.826+0000 7f570c400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5704101de0 0x7f570419a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:47.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.826+0000 7f570c400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5704102790 0x7f570419ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:47.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.826+0000 7f570c400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5704106f10 0x7f570419eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:47.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.826+0000 7f570c400640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5704111fc0 con 0x7f5704106f10 2024-09-10T09:55:47.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.826+0000 7f570c400640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5704111e40 con 0x7f5704101de0 2024-09-10T09:55:47.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.826+0000 7f570c400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5704112140 con 0x7f5704102790 2024-09-10T09:55:47.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.826+0000 7f570a400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5704106f10 0x7f570419eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:47.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.826+0000 7f5709000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5704102790 0x7f570419ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:47.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.826+0000 7f570a400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5704106f10 0x7f570419eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33238/0 (socket says 172.21.15.77:33238) 2024-09-10T09:55:47.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.826+0000 7f570a400640 1 -- 172.21.15.77:0/1037330783 learned_addr learned my addr 172.21.15.77:0/1037330783 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:55:47.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.826+0000 7f5709a00640 1 --2- 172.21.15.77:0/1037330783 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5704101de0 0x7f570419a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:47.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.827+0000 7f570a400640 1 -- 172.21.15.77:0/1037330783 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5704102790 msgr2=0x7f570419ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:47.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.827+0000 7f570a400640 1 --2- 172.21.15.77:0/1037330783 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5704102790 0x7f570419ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:47.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.827+0000 7f570a400640 1 -- 172.21.15.77:0/1037330783 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5704101de0 msgr2=0x7f570419a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:47.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.827+0000 7f570a400640 1 --2- 172.21.15.77:0/1037330783 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5704101de0 0x7f570419a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:47.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.827+0000 7f570a400640 1 -- 172.21.15.77:0/1037330783 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f570419f590 con 0x7f5704106f10 2024-09-10T09:55:47.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.827+0000 7f5709000640 1 --2- 172.21.15.77:0/1037330783 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5704102790 0x7f570419ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:55:47.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.827+0000 7f570a400640 1 --2- 172.21.15.77:0/1037330783 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5704106f10 0x7f570419eeb0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f56fc003f10 tx=0x7f56fc003f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:47.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.827+0000 7f5702000640 1 -- 172.21.15.77:0/1037330783 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56fc031710 con 0x7f5704106f10 2024-09-10T09:55:47.842 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.827+0000 7f5702000640 1 -- 172.21.15.77:0/1037330783 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56fc0318b0 con 0x7f5704106f10 2024-09-10T09:55:47.842 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.827+0000 7f5702000640 1 -- 172.21.15.77:0/1037330783 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56fc02fb90 con 0x7f5704106f10 2024-09-10T09:55:47.842 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.827+0000 7f570c400640 1 -- 172.21.15.77:0/1037330783 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f570419f820 con 0x7f5704106f10 2024-09-10T09:55:47.842 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.827+0000 7f570c400640 1 -- 172.21.15.77:0/1037330783 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57041a7160 con 0x7f5704106f10 2024-09-10T09:55:47.842 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.830+0000 7f5702000640 1 -- 172.21.15.77:0/1037330783 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f56fc02fd30 con 0x7f5704106f10 2024-09-10T09:55:47.842 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.830+0000 7f570f044740 1 Processor -- start 2024-09-10T09:55:47.842 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.830+0000 7f570f044740 1 -- start start 2024-09-10T09:55:47.842 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.831+0000 7f570f044740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f8cf3b2330 0x55f8cf3b2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:47.842 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.831+0000 7f570f044740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f8cf3b2c50 0x55f8cf3bcfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:47.842 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.831+0000 7f570f044740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8cf3bd510 0x55f8cf3bf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:47.842 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.831+0000 7f570f044740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f8cf3180b0 con 0x55f8cf3bd510 2024-09-10T09:55:47.842 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.831+0000 7f570f044740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f8cf2bfab0 con 0x55f8cf3b2330 2024-09-10T09:55:47.842 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.831+0000 7f570f044740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f8cf314070 con 0x55f8cf3b2c50 2024-09-10T09:55:47.842 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.831+0000 7f5709a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f8cf3b2c50 0x55f8cf3bcfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:47.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.831+0000 7f5709a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f8cf3b2c50 0x55f8cf3bcfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:39318/0 (socket says 172.21.15.77:39318) 2024-09-10T09:55:47.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.831+0000 7f5709000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8cf3bd510 0x55f8cf3bf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:47.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.831+0000 7f5709a00640 1 -- 172.21.15.77:0/2921938896 learned_addr learned my addr 172.21.15.77:0/2921938896 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:55:47.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.831+0000 7f5709000640 1 --2- 172.21.15.77:0/2921938896 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f8cf3b2330 0x55f8cf3b2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:47.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.832+0000 7f5709a00640 1 -- 172.21.15.77:0/2921938896 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f8cf3b2330 msgr2=0x55f8cf3b2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:47.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.832+0000 7f5709a00640 1 --2- 172.21.15.77:0/2921938896 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f8cf3b2330 0x55f8cf3b2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:47.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.832+0000 7f5709a00640 1 -- 172.21.15.77:0/2921938896 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8cf3bd510 msgr2=0x55f8cf3bf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:47.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.832+0000 7f5709a00640 1 --2- 172.21.15.77:0/2921938896 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8cf3bd510 0x55f8cf3bf900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:47.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.832+0000 7f5709a00640 1 -- 172.21.15.77:0/2921938896 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8cf2cc4a0 con 0x55f8cf3b2c50 2024-09-10T09:55:47.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.832+0000 7f5702000640 1 --2- 172.21.15.77:0/1037330783 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f56ec0a2c00 0x7f56ec0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:47.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.832+0000 7f5702000640 1 -- 172.21.15.77:0/1037330783 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f56fc0f08e0 con 0x7f5704106f10 2024-09-10T09:55:47.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.832+0000 7f5709a00640 1 --2- 172.21.15.77:0/2921938896 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f8cf3b2c50 0x55f8cf3bcfd0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f56f800bd10 tx=0x7f56f80107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85ec6a4e30bef976 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:47.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.832+0000 7f5709000640 1 --2- 172.21.15.77:0/1037330783 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f56ec0a2c00 0x7f56ec0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:47.844 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.832+0000 7f56f6a00640 1 -- 172.21.15.77:0/2921938896 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f8004030 con 0x55f8cf3b2c50 2024-09-10T09:55:47.844 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.832+0000 7f56f6a00640 1 -- 172.21.15.77:0/2921938896 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56f80041d0 con 0x55f8cf3b2c50 2024-09-10T09:55:47.844 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.832+0000 7f56f6a00640 1 -- 172.21.15.77:0/2921938896 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f8011840 con 0x55f8cf3b2c50 2024-09-10T09:55:47.844 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.833+0000 7f570f044740 1 -- 172.21.15.77:0/2921938896 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f8cf3b2c50 msgr2=0x55f8cf3bcfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:47.844 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.833+0000 7f570f044740 1 --2- 172.21.15.77:0/2921938896 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f8cf3b2c50 0x55f8cf3bcfd0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f56f800bd10 tx=0x7f56f80107a0 comp rx=0 tx=0).stop 2024-09-10T09:55:47.844 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.833+0000 7f5709000640 1 --2- 172.21.15.77:0/1037330783 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f56ec0a2c00 0x7f56ec0a50c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x55f8cf3afc40 tx=0x7f56f000ba90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:47.844 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.833+0000 7f570f044740 1 -- 172.21.15.77:0/2921938896 shutdown_connections 2024-09-10T09:55:47.844 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.833+0000 7f570f044740 1 --2- 172.21.15.77:0/2921938896 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8cf3bd510 0x55f8cf3bf900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:47.844 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.833+0000 7f570f044740 1 --2- 172.21.15.77:0/2921938896 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f8cf3b2c50 0x55f8cf3bcfd0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:47.844 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.833+0000 7f570f044740 1 --2- 172.21.15.77:0/2921938896 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f8cf3b2330 0x55f8cf3b2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:47.844 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.833+0000 7f570f044740 1 -- 172.21.15.77:0/2921938896 >> 172.21.15.77:0/2921938896 conn(0x55f8cf3ae8c0 msgr2=0x55f8cf3aeca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:47.844 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.833+0000 7f570f044740 1 -- 172.21.15.77:0/2921938896 shutdown_connections 2024-09-10T09:55:47.844 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.833+0000 7f570f044740 1 -- 172.21.15.77:0/2921938896 wait complete. 2024-09-10T09:55:47.844 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.834+0000 7f570f044740 1 -- start start 2024-09-10T09:55:47.845 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.834+0000 7f570f044740 1 Processor -- start 2024-09-10T09:55:47.845 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.834+0000 7f570f044740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8cf3b2330 0x55f8cf3ba550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:47.845 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.834+0000 7f570f044740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f8cf3b2c50 0x55f8cf3baa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:47.845 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.834+0000 7f570f044740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f8cf3bd510 0x55f8cf3bafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:47.845 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.834+0000 7f570f044740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f8cf3c0060 con 0x55f8cf3b2330 2024-09-10T09:55:47.845 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.834+0000 7f570f044740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f8cf2a0920 con 0x55f8cf3bd510 2024-09-10T09:55:47.845 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.834+0000 7f570f044740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f8cf3c01e0 con 0x55f8cf3b2c50 2024-09-10T09:55:47.845 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.835+0000 7f570f044740 10 client.0 _ll_register_callbacks cb 0x55f8cf2c48a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:55:47.845 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.835+0000 7f5709a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8cf3b2330 0x55f8cf3ba550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:47.845 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.835+0000 7f570a400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f8cf3bd510 0x55f8cf3bafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:47.845 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.835+0000 7f570a400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f8cf3bd510 0x55f8cf3bafd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:52204/0 (socket says 172.21.15.77:52204) 2024-09-10T09:55:47.845 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.835+0000 7f570a400640 1 -- 172.21.15.77:0/4083305104 learned_addr learned my addr 172.21.15.77:0/4083305104 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:55:47.845 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.835+0000 7f5709000640 1 --2- 172.21.15.77:0/4083305104 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f8cf3b2c50 0x55f8cf3baa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:47.846 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.835+0000 7f570a400640 1 -- 172.21.15.77:0/4083305104 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f8cf3b2c50 msgr2=0x55f8cf3baa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:47.846 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.835+0000 7f570a400640 1 --2- 172.21.15.77:0/4083305104 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f8cf3b2c50 0x55f8cf3baa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:47.846 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.835+0000 7f570a400640 1 -- 172.21.15.77:0/4083305104 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8cf3b2330 msgr2=0x55f8cf3ba550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:47.846 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.835+0000 7f570a400640 1 --2- 172.21.15.77:0/4083305104 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8cf3b2330 0x55f8cf3ba550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:47.846 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.835+0000 7f570a400640 1 -- 172.21.15.77:0/4083305104 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8cf0892b0 con 0x55f8cf3bd510 2024-09-10T09:55:47.846 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.836+0000 7f570a400640 1 --2- 172.21.15.77:0/4083305104 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f8cf3bd510 0x55f8cf3bafd0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f56fc030040 tx=0x7f56fc0bc650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad61d5432e6c30c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:47.846 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.836+0000 7f56eb400640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T09:55:47.846 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.836+0000 7f56eb400640 1 -- 172.21.15.77:0/4083305104 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56fc0c10a0 con 0x55f8cf3bd510 2024-09-10T09:55:47.846 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.836+0000 7f570f044740 1 -- 172.21.15.77:0/4083305104 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56f80041d0 con 0x55f8cf3bd510 2024-09-10T09:55:47.846 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.836+0000 7f56eb400640 1 -- 172.21.15.77:0/4083305104 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f56fc0bcf50 con 0x55f8cf3bd510 2024-09-10T09:55:47.846 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.836+0000 7f56eb400640 1 -- 172.21.15.77:0/4083305104 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f56fc0c5960 con 0x55f8cf3bd510 2024-09-10T09:55:47.846 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.836+0000 7f570f044740 10 client.25445 fetch_fsmap learned FSMap version 29 2024-09-10T09:55:47.846 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.836+0000 7f570f044740 1 -- 172.21.15.77:0/4083305104 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f8cf3c0060 con 0x55f8cf3bd510 2024-09-10T09:55:47.846 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.836+0000 7f56eb400640 1 -- 172.21.15.77:0/4083305104 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56f80041d0 con 0x55f8cf3bd510 2024-09-10T09:55:47.847 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.836+0000 7f56eb400640 1 -- 172.21.15.77:0/4083305104 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f56fc004bd0 con 0x55f8cf3bd510 2024-09-10T09:55:47.847 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.837+0000 7f570f044740 10 client.25445 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T09:55:47.847 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.837+0000 7f570f044740 10 client.25445 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-10T09:55:47.847 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.837+0000 7f570f044740 20 client.25445 populate_metadata read hostname 'smithi077' 2024-09-10T09:55:47.847 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.837+0000 7f570f044740 1 --2- 172.21.15.77:0/4083305104 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x55f8cf3ce4f0 0x55f8cf3cf960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:47.847 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.837+0000 7f570f044740 4 client.25445 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:55:47.847 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.837+0000 7f570f044740 1 -- 172.21.15.77:0/4083305104 --> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f8cf325620 con 0x55f8cf3ce4f0 2024-09-10T09:55:47.847 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.837+0000 7f5709a00640 1 --2- 172.21.15.77:0/4083305104 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x55f8cf3ce4f0 0x55f8cf3cf960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:47.847 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.838+0000 7f5709a00640 1 --2- 172.21.15.77:0/4083305104 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x55f8cf3ce4f0 0x55f8cf3cf960 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=387859e968cd3738 server_cookie=1c8d0d546e9876dd in_seq=0 out_seq=0 2024-09-10T09:55:47.847 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.838+0000 7f56eb400640 10 client.25445 ms_handle_connect on v2:172.21.15.120:6836/3033280709 2024-09-10T09:55:47.848 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.838+0000 7f56eb400640 1 -- 172.21.15.77:0/4083305104 <== mds.1 v2:172.21.15.120:6836/3033280709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f8cf325620 con 0x55f8cf3ce4f0 2024-09-10T09:55:47.848 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.838+0000 7f56eb400640 10 client.25445 handle_command_reply: tid=0 2024-09-10T09:55:47.848 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.838+0000 7f570f044740 1 client.25445 shutdown 2024-09-10T09:55:47.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.839+0000 7f570f044740 1 -- 172.21.15.77:0/4083305104 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f8cf3bd510 msgr2=0x55f8cf3bafd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:47.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.839+0000 7f570f044740 1 --2- 172.21.15.77:0/4083305104 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f8cf3bd510 0x55f8cf3bafd0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f56fc030040 tx=0x7f56fc0bc650 comp rx=0 tx=0).stop 2024-09-10T09:55:47.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.839+0000 7f570f044740 1 -- 172.21.15.77:0/4083305104 shutdown_connections 2024-09-10T09:55:47.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.839+0000 7f570f044740 1 --2- 172.21.15.77:0/4083305104 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x55f8cf3ce4f0 0x55f8cf3cf960 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:47.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.839+0000 7f570f044740 1 --2- 172.21.15.77:0/4083305104 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f8cf3bd510 0x55f8cf3bafd0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:47.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.839+0000 7f570f044740 1 --2- 172.21.15.77:0/4083305104 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f8cf3b2c50 0x55f8cf3baa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:47.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.839+0000 7f570f044740 1 --2- 172.21.15.77:0/4083305104 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8cf3b2330 0x55f8cf3ba550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:47.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.839+0000 7f56eb400640 0 client.25445 ms_handle_reset on v2:172.21.15.120:6836/3033280709 2024-09-10T09:55:47.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.839+0000 7f570f044740 1 -- 172.21.15.77:0/4083305104 >> 172.21.15.77:0/4083305104 conn(0x55f8cf3ae8c0 msgr2=0x55f8cf3b45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:47.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.839+0000 7f570f044740 1 -- 172.21.15.77:0/4083305104 shutdown_connections 2024-09-10T09:55:47.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.839+0000 7f570f044740 1 -- 172.21.15.77:0/4083305104 wait complete. 2024-09-10T09:55:47.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.839+0000 7f570f044740 20 client.25445 trim_cache size 0 max 16384 2024-09-10T09:55:47.860 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.858+0000 7f570f044740 1 Processor -- start 2024-09-10T09:55:47.860 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.858+0000 7f570f044740 1 -- start start 2024-09-10T09:55:47.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.858+0000 7f570f044740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8cf3b2330 0x55f8cf3b9e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:47.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.858+0000 7f570f044740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f8cf3b2c50 0x55f8cf3b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:47.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.858+0000 7f570f044740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f8cf3bd510 0x55f8cf3ccf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:47.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.858+0000 7f570f044740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f8cf3cdca0 con 0x55f8cf3b2330 2024-09-10T09:55:47.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.858+0000 7f570f044740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f8cf3ce370 con 0x55f8cf3b2c50 2024-09-10T09:55:47.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.858+0000 7f570f044740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f8cf3cea40 con 0x55f8cf3bd510 2024-09-10T09:55:47.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.859+0000 7f5709a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8cf3b2330 0x55f8cf3b9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:47.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.859+0000 7f5709a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8cf3b2330 0x55f8cf3b9e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33268/0 (socket says 172.21.15.77:33268) 2024-09-10T09:55:47.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.859+0000 7f5709a00640 1 -- 172.21.15.77:0/895253979 learned_addr learned my addr 172.21.15.77:0/895253979 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:55:47.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.859+0000 7f5709000640 1 --2- 172.21.15.77:0/895253979 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f8cf3b2c50 0x55f8cf3b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:47.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.859+0000 7f570a400640 1 --2- 172.21.15.77:0/895253979 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f8cf3bd510 0x55f8cf3ccf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:47.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.859+0000 7f5709a00640 1 -- 172.21.15.77:0/895253979 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f8cf3bd510 msgr2=0x55f8cf3ccf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:47.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.859+0000 7f5709a00640 1 --2- 172.21.15.77:0/895253979 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f8cf3bd510 0x55f8cf3ccf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:47.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.859+0000 7f5709a00640 1 -- 172.21.15.77:0/895253979 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f8cf3b2c50 msgr2=0x55f8cf3b4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:47.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.859+0000 7f5709a00640 1 --2- 172.21.15.77:0/895253979 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f8cf3b2c50 0x55f8cf3b4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:47.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.859+0000 7f5709a00640 1 -- 172.21.15.77:0/895253979 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8cf3af550 con 0x55f8cf3b2330 2024-09-10T09:55:47.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.859+0000 7f5709a00640 1 --2- 172.21.15.77:0/895253979 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8cf3b2330 0x55f8cf3b9e10 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f56f8005d00 tx=0x7f56f800be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68744a05b37fe97a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:47.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.860+0000 7f56f6000640 1 -- 172.21.15.77:0/895253979 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f8005240 con 0x55f8cf3b2330 2024-09-10T09:55:47.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.860+0000 7f56f6000640 1 -- 172.21.15.77:0/895253979 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56f80053e0 con 0x55f8cf3b2330 2024-09-10T09:55:47.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.860+0000 7f56f6000640 1 -- 172.21.15.77:0/895253979 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f8011800 con 0x55f8cf3b2330 2024-09-10T09:55:47.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.860+0000 7f570f044740 1 -- 172.21.15.77:0/895253979 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8cf3b2330 msgr2=0x55f8cf3b9e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:47.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.860+0000 7f570f044740 1 --2- 172.21.15.77:0/895253979 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8cf3b2330 0x55f8cf3b9e10 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f56f8005d00 tx=0x7f56f800be00 comp rx=0 tx=0).stop 2024-09-10T09:55:47.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.860+0000 7f570f044740 1 -- 172.21.15.77:0/895253979 shutdown_connections 2024-09-10T09:55:47.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.860+0000 7f570f044740 1 --2- 172.21.15.77:0/895253979 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f8cf3bd510 0x55f8cf3ccf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:47.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.860+0000 7f570f044740 1 --2- 172.21.15.77:0/895253979 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f8cf3b2c50 0x55f8cf3b4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:47.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.860+0000 7f570f044740 1 --2- 172.21.15.77:0/895253979 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8cf3b2330 0x55f8cf3b9e10 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:47.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.860+0000 7f570f044740 1 -- 172.21.15.77:0/895253979 >> 172.21.15.77:0/895253979 conn(0x55f8cf3ae8c0 msgr2=0x55f8cf3b8e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:47.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.861+0000 7f570f044740 1 -- 172.21.15.77:0/895253979 shutdown_connections 2024-09-10T09:55:47.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.861+0000 7f570f044740 1 -- 172.21.15.77:0/895253979 wait complete. 2024-09-10T09:55:47.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.861+0000 7f570f044740 1 -- start start 2024-09-10T09:55:47.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.861+0000 7f570f044740 1 Processor -- start 2024-09-10T09:55:47.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.862+0000 7f570f044740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f8cf3b2330 0x55f8cf3d7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:47.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.862+0000 7f570f044740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8cf3b2c50 0x55f8cf3d83e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:47.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.862+0000 7f570f044740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f8cf3bd510 0x55f8cf3b6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:47.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.862+0000 7f570f044740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f8cf3cf7e0 con 0x55f8cf3b2c50 2024-09-10T09:55:47.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.862+0000 7f570f044740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f8cf3cf110 con 0x55f8cf3b2330 2024-09-10T09:55:47.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.862+0000 7f570f044740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f8cf3cfeb0 con 0x55f8cf3bd510 2024-09-10T09:55:47.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.862+0000 7f570f044740 10 client.0 _ll_register_callbacks cb 0x7f56fc0bcac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:55:47.868 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.862+0000 7f5709000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8cf3b2c50 0x55f8cf3d83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:47.868 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.862+0000 7f5709a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f8cf3b2330 0x55f8cf3d7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:47.868 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.862+0000 7f5709000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8cf3b2c50 0x55f8cf3d83e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33284/0 (socket says 172.21.15.77:33284) 2024-09-10T09:55:47.868 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.862+0000 7f5709000640 1 -- 172.21.15.77:0/1487405779 learned_addr learned my addr 172.21.15.77:0/1487405779 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:55:47.868 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.862+0000 7f570a400640 1 --2- 172.21.15.77:0/1487405779 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f8cf3bd510 0x55f8cf3b6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:47.868 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.862+0000 7f5709000640 1 -- 172.21.15.77:0/1487405779 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f8cf3bd510 msgr2=0x55f8cf3b6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:47.868 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.862+0000 7f5709000640 1 --2- 172.21.15.77:0/1487405779 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f8cf3bd510 0x55f8cf3b6fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:47.868 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.862+0000 7f5709000640 1 -- 172.21.15.77:0/1487405779 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f8cf3b2330 msgr2=0x55f8cf3d7ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:47.868 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.862+0000 7f5709000640 1 --2- 172.21.15.77:0/1487405779 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f8cf3b2330 0x55f8cf3d7ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:47.868 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.862+0000 7f5709000640 1 -- 172.21.15.77:0/1487405779 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8cf3b77b0 con 0x55f8cf3b2c50 2024-09-10T09:55:47.868 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.863+0000 7f5709000640 1 --2- 172.21.15.77:0/1487405779 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8cf3b2c50 0x55f8cf3d83e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f5704076480 tx=0x7f56f0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e31821d5fe4e04f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:47.868 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.863+0000 7f56eb400640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-10T09:55:47.868 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.863+0000 7f56eb400640 1 -- 172.21.15.77:0/1487405779 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f0023c70 con 0x55f8cf3b2c50 2024-09-10T09:55:47.868 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.863+0000 7f56eb400640 1 -- 172.21.15.77:0/1487405779 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f56f0004020 con 0x55f8cf3b2c50 2024-09-10T09:55:47.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.863+0000 7f570f044740 1 -- 172.21.15.77:0/1487405779 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56f80053e0 con 0x55f8cf3b2c50 2024-09-10T09:55:47.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.863+0000 7f56eb400640 1 -- 172.21.15.77:0/1487405779 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f56f0037050 con 0x55f8cf3b2c50 2024-09-10T09:55:47.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.863+0000 7f570f044740 10 client.15405 fetch_fsmap learned FSMap version 29 2024-09-10T09:55:47.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.863+0000 7f570f044740 1 -- 172.21.15.77:0/1487405779 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f8cf3d7830 con 0x55f8cf3b2c50 2024-09-10T09:55:47.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.864+0000 7f56eb400640 1 -- 172.21.15.77:0/1487405779 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56f80053e0 con 0x55f8cf3b2c50 2024-09-10T09:55:47.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.864+0000 7f56eb400640 1 -- 172.21.15.77:0/1487405779 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f56f001e050 con 0x55f8cf3b2c50 2024-09-10T09:55:47.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.864+0000 7f570f044740 10 client.15405 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T09:55:47.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.864+0000 7f570f044740 10 client.15405 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-10T09:55:47.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.864+0000 7f570f044740 20 client.15405 populate_metadata read hostname 'smithi077' 2024-09-10T09:55:47.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.864+0000 7f570f044740 1 --2- 172.21.15.77:0/1487405779 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x55f8cf43d4d0 0x55f8cf43c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:47.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.864+0000 7f570f044740 4 client.15405 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T09:55:47.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.864+0000 7f570f044740 1 -- 172.21.15.77:0/1487405779 --> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] -- command(tid 0: {"prefix": "status"}) -- 0x55f8cf31fb30 con 0x55f8cf43d4d0 2024-09-10T09:55:47.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.864+0000 7f5709a00640 1 --2- 172.21.15.77:0/1487405779 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x55f8cf43d4d0 0x55f8cf43c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:47.871 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.865+0000 7f5709a00640 1 --2- 172.21.15.77:0/1487405779 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x55f8cf43d4d0 0x55f8cf43c1e0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d4f3ae67c0b30cfd server_cookie=75db2b40387e4348 in_seq=0 out_seq=0 2024-09-10T09:55:47.871 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.865+0000 7f56eb400640 10 client.15405 ms_handle_connect on v2:172.21.15.120:6836/3033280709 2024-09-10T09:55:47.871 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.865+0000 7f56eb400640 1 -- 172.21.15.77:0/1487405779 <== mds.1 v2:172.21.15.120:6836/3033280709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55f8cf31fb30 con 0x55f8cf43d4d0 2024-09-10T09:55:47.871 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.865+0000 7f56eb400640 10 client.15405 handle_command_reply: tid=0 2024-09-10T09:55:47.871 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.865+0000 7f570f044740 1 client.15405 shutdown 2024-09-10T09:55:47.871 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.865+0000 7f570f044740 1 -- 172.21.15.77:0/1487405779 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8cf3b2c50 msgr2=0x55f8cf3d83e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:47.871 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.865+0000 7f570f044740 1 --2- 172.21.15.77:0/1487405779 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8cf3b2c50 0x55f8cf3d83e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f5704076480 tx=0x7f56f0004730 comp rx=0 tx=0).stop 2024-09-10T09:55:47.871 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.865+0000 7f570f044740 1 -- 172.21.15.77:0/1487405779 shutdown_connections 2024-09-10T09:55:47.871 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.865+0000 7f570f044740 1 --2- 172.21.15.77:0/1487405779 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x55f8cf43d4d0 0x55f8cf43c1e0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:47.871 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.865+0000 7f570f044740 1 --2- 172.21.15.77:0/1487405779 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f8cf3bd510 0x55f8cf3b6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:47.871 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.865+0000 7f56eb400640 0 client.15405 ms_handle_reset on v2:172.21.15.120:6836/3033280709 2024-09-10T09:55:47.871 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.865+0000 7f570f044740 1 --2- 172.21.15.77:0/1487405779 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8cf3b2c50 0x55f8cf3d83e0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:47.871 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.865+0000 7f570f044740 1 --2- 172.21.15.77:0/1487405779 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f8cf3b2330 0x55f8cf3d7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:47.872 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.865+0000 7f570f044740 1 -- 172.21.15.77:0/1487405779 >> 172.21.15.77:0/1487405779 conn(0x55f8cf3ae8c0 msgr2=0x55f8cf3b8a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:47.872 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.866+0000 7f570f044740 1 -- 172.21.15.77:0/1487405779 shutdown_connections 2024-09-10T09:55:47.872 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.866+0000 7f570f044740 1 -- 172.21.15.77:0/1487405779 wait complete. 2024-09-10T09:55:47.872 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.866+0000 7f570f044740 20 client.15405 trim_cache size 0 max 16384 2024-09-10T09:55:47.872 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-10T09:55:47.872 INFO:teuthology.orchestra.run.smithi077.stdout: "cluster_fsid": "14a55226-6f59-11ef-bcea-c7b262605968", 2024-09-10T09:55:47.872 INFO:teuthology.orchestra.run.smithi077.stdout: "whoami": 1, 2024-09-10T09:55:47.872 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 24524, 2024-09-10T09:55:47.872 INFO:teuthology.orchestra.run.smithi077.stdout: "want_state": "up:active", 2024-09-10T09:55:47.872 INFO:teuthology.orchestra.run.smithi077.stdout: "state": "up:active", 2024-09-10T09:55:47.872 INFO:teuthology.orchestra.run.smithi077.stdout: "fs_name": "cephfs", 2024-09-10T09:55:47.872 INFO:teuthology.orchestra.run.smithi077.stdout: "rank_uptime": 155.028350448, 2024-09-10T09:55:47.873 INFO:teuthology.orchestra.run.smithi077.stdout: "mdsmap_epoch": 29, 2024-09-10T09:55:47.873 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch": 96, 2024-09-10T09:55:47.873 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch_barrier": 96, 2024-09-10T09:55:47.873 INFO:teuthology.orchestra.run.smithi077.stdout: "uptime": 230.159509407 2024-09-10T09:55:47.873 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-10T09:55:47.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.866+0000 7f570c400640 1 -- 172.21.15.77:0/1037330783 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f56ec0a2c00 msgr2=0x7f56ec0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:47.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.866+0000 7f570c400640 1 --2- 172.21.15.77:0/1037330783 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f56ec0a2c00 0x7f56ec0a50c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x55f8cf3afc40 tx=0x7f56f000ba90 comp rx=0 tx=0).stop 2024-09-10T09:55:47.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.866+0000 7f570c400640 1 -- 172.21.15.77:0/1037330783 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5704106f10 msgr2=0x7f570419eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:47.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.866+0000 7f570c400640 1 --2- 172.21.15.77:0/1037330783 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5704106f10 0x7f570419eeb0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f56fc003f10 tx=0x7f56fc003f40 comp rx=0 tx=0).stop 2024-09-10T09:55:47.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.867+0000 7f570c400640 1 -- 172.21.15.77:0/1037330783 shutdown_connections 2024-09-10T09:55:47.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.867+0000 7f570c400640 1 --2- 172.21.15.77:0/1037330783 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f56ec0a2c00 0x7f56ec0a50c0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:47.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.867+0000 7f570c400640 1 --2- 172.21.15.77:0/1037330783 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5704106f10 0x7f570419eeb0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:47.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.867+0000 7f570c400640 1 --2- 172.21.15.77:0/1037330783 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5704102790 0x7f570419ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:47.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.867+0000 7f570c400640 1 --2- 172.21.15.77:0/1037330783 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5704101de0 0x7f570419a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:47.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.867+0000 7f570c400640 1 -- 172.21.15.77:0/1037330783 >> 172.21.15.77:0/1037330783 conn(0x7f57040fd8c0 msgr2=0x7f57040fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:47.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.867+0000 7f570c400640 1 -- 172.21.15.77:0/1037330783 shutdown_connections 2024-09-10T09:55:47.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:47.867+0000 7f570c400640 1 -- 172.21.15.77:0/1037330783 wait complete. 2024-09-10T09:55:48.636 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-10T09:55:48.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:55:48 smithi077 ceph-mon[30319]: pgmap v410: 129 pgs: 129 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-10T09:55:49.005 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:55:49.133 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:55:48 smithi134 ceph-mon[35674]: pgmap v410: 129 pgs: 129 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-10T09:55:49.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:55:48 smithi120 ceph-mon[34770]: pgmap v410: 129 pgs: 129 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-10T09:55:49.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.554+0000 7f8455c00640 1 -- 172.21.15.77:0/3794544166 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84501047e0 msgr2=0x7f8450108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:49.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.554+0000 7f8455c00640 1 --2- 172.21.15.77:0/3794544166 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84501047e0 0x7f8450108830 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f843c009900 tx=0x7f843c02a3f0 comp rx=0 tx=0).stop 2024-09-10T09:55:49.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.554+0000 7f8455c00640 1 -- 172.21.15.77:0/3794544166 shutdown_connections 2024-09-10T09:55:49.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.554+0000 7f8455c00640 1 --2- 172.21.15.77:0/3794544166 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8450108f60 0x7f8450110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:49.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.554+0000 7f8455c00640 1 --2- 172.21.15.77:0/3794544166 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84501047e0 0x7f8450108830 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:49.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.554+0000 7f8455c00640 1 --2- 172.21.15.77:0/3794544166 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8450103e30 0x7f8450104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:49.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.554+0000 7f8455c00640 1 -- 172.21.15.77:0/3794544166 >> 172.21.15.77:0/3794544166 conn(0x7f84500ff8d0 msgr2=0x7f8450101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:49.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.555+0000 7f8455c00640 1 -- 172.21.15.77:0/3794544166 shutdown_connections 2024-09-10T09:55:49.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.555+0000 7f8455c00640 1 -- 172.21.15.77:0/3794544166 wait complete. 2024-09-10T09:55:49.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.556+0000 7f8455c00640 1 Processor -- start 2024-09-10T09:55:49.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.556+0000 7f8455c00640 1 -- start start 2024-09-10T09:55:49.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.556+0000 7f8455c00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8450103e30 0x7f845019a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:49.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.556+0000 7f8455c00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84501047e0 0x7f845019a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:49.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.556+0000 7f8455c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8450108f60 0x7f845019ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:49.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.556+0000 7f8455c00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8450113fc0 con 0x7f84501047e0 2024-09-10T09:55:49.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.556+0000 7f8455c00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8450113e40 con 0x7f8450103e30 2024-09-10T09:55:49.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.556+0000 7f8455c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8450114140 con 0x7f8450108f60 2024-09-10T09:55:49.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.556+0000 7f844f400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84501047e0 0x7f845019a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:49.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.556+0000 7f844f400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84501047e0 0x7f845019a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33296/0 (socket says 172.21.15.77:33296) 2024-09-10T09:55:49.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.556+0000 7f844fe00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8450103e30 0x7f845019a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:49.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.557+0000 7f844f400640 1 -- 172.21.15.77:0/2386482167 learned_addr learned my addr 172.21.15.77:0/2386482167 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:55:49.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.557+0000 7f8455200640 1 --2- 172.21.15.77:0/2386482167 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8450108f60 0x7f845019ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:49.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.557+0000 7f844fe00640 1 -- 172.21.15.77:0/2386482167 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8450108f60 msgr2=0x7f845019ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:49.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.557+0000 7f844fe00640 1 --2- 172.21.15.77:0/2386482167 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8450108f60 0x7f845019ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:49.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.557+0000 7f844fe00640 1 -- 172.21.15.77:0/2386482167 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84501047e0 msgr2=0x7f845019a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:49.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.557+0000 7f844fe00640 1 --2- 172.21.15.77:0/2386482167 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84501047e0 0x7f845019a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:49.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.557+0000 7f844fe00640 1 -- 172.21.15.77:0/2386482167 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f845019f410 con 0x7f8450103e30 2024-09-10T09:55:49.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.557+0000 7f8455200640 1 --2- 172.21.15.77:0/2386482167 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8450108f60 0x7f845019ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:55:49.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.557+0000 7f844fe00640 1 --2- 172.21.15.77:0/2386482167 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8450103e30 0x7f845019a460 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f844000e9f0 tx=0x7f844000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:49.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.557+0000 7f844cc00640 1 -- 172.21.15.77:0/2386482167 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f844000ccc0 con 0x7f8450103e30 2024-09-10T09:55:49.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.558+0000 7f8455c00640 1 -- 172.21.15.77:0/2386482167 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f845019f700 con 0x7f8450103e30 2024-09-10T09:55:49.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.558+0000 7f844cc00640 1 -- 172.21.15.77:0/2386482167 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8440010910 con 0x7f8450103e30 2024-09-10T09:55:49.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.558+0000 7f844cc00640 1 -- 172.21.15.77:0/2386482167 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8440018960 con 0x7f8450103e30 2024-09-10T09:55:49.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.558+0000 7f8455c00640 1 -- 172.21.15.77:0/2386482167 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84501a6fe0 con 0x7f8450103e30 2024-09-10T09:55:49.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.561+0000 7f8459c45740 1 Processor -- start 2024-09-10T09:55:49.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.561+0000 7f844cc00640 1 -- 172.21.15.77:0/2386482167 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8440010ac0 con 0x7f8450103e30 2024-09-10T09:55:49.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.561+0000 7f8459c45740 1 -- start start 2024-09-10T09:55:49.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.561+0000 7f8459c45740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56331f1784e0 0x56331f1788c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:49.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.561+0000 7f8459c45740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56331f178e00 0x56331f183180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:49.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.561+0000 7f8459c45740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56331f1836c0 0x56331f185ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:49.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.561+0000 7f8459c45740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56331f0de0b0 con 0x56331f1784e0 2024-09-10T09:55:49.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.561+0000 7f8459c45740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56331f085ab0 con 0x56331f178e00 2024-09-10T09:55:49.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.561+0000 7f8459c45740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56331f0da070 con 0x56331f1836c0 2024-09-10T09:55:49.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.562+0000 7f8455200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56331f1784e0 0x56331f1788c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:49.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.562+0000 7f8455200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56331f1784e0 0x56331f1788c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33298/0 (socket says 172.21.15.77:33298) 2024-09-10T09:55:49.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.562+0000 7f8455200640 1 -- 172.21.15.77:0/3740223729 learned_addr learned my addr 172.21.15.77:0/3740223729 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:55:49.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.562+0000 7f844fe00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56331f178e00 0x56331f183180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:49.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.562+0000 7f844f400640 1 --2- 172.21.15.77:0/3740223729 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56331f1836c0 0x56331f185ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:49.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.562+0000 7f8455200640 1 -- 172.21.15.77:0/3740223729 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56331f1836c0 msgr2=0x56331f185ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:49.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.562+0000 7f8455200640 1 --2- 172.21.15.77:0/3740223729 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56331f1836c0 0x56331f185ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:49.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.562+0000 7f8455200640 1 -- 172.21.15.77:0/3740223729 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56331f178e00 msgr2=0x56331f183180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:49.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.562+0000 7f8455200640 1 --2- 172.21.15.77:0/3740223729 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56331f178e00 0x56331f183180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:49.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.562+0000 7f8455200640 1 -- 172.21.15.77:0/3740223729 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56331f0924a0 con 0x56331f1784e0 2024-09-10T09:55:49.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.562+0000 7f8455200640 1 --2- 172.21.15.77:0/3740223729 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56331f1784e0 0x56331f1788c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f8448011790 tx=0x7f8448011c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7543dcf11faef131 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:49.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.563+0000 7f844cc00640 1 --2- 172.21.15.77:0/2386482167 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f84340a2b80 0x7f84340a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:49.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.563+0000 7f8445600640 1 -- 172.21.15.77:0/3740223729 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8448003c30 con 0x56331f1784e0 2024-09-10T09:55:49.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.563+0000 7f8445600640 1 -- 172.21.15.77:0/3740223729 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f844800fe80 con 0x56331f1784e0 2024-09-10T09:55:49.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.563+0000 7f8445600640 1 -- 172.21.15.77:0/3740223729 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84480193e0 con 0x56331f1784e0 2024-09-10T09:55:49.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.563+0000 7f844cc00640 1 -- 172.21.15.77:0/2386482167 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f84400cf7e0 con 0x7f8450103e30 2024-09-10T09:55:49.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.563+0000 7f844fe00640 1 --2- 172.21.15.77:0/2386482167 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f84340a2b80 0x7f84340a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:49.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.563+0000 7f8459c45740 1 -- 172.21.15.77:0/3740223729 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56331f1784e0 msgr2=0x56331f1788c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:49.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.563+0000 7f8459c45740 1 --2- 172.21.15.77:0/3740223729 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56331f1784e0 0x56331f1788c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f8448011790 tx=0x7f8448011c50 comp rx=0 tx=0).stop 2024-09-10T09:55:49.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.564+0000 7f844fe00640 1 --2- 172.21.15.77:0/2386482167 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f84340a2b80 0x7f84340a5040 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f84400cced0 tx=0x7f84400d5530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:49.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.564+0000 7f8459c45740 1 -- 172.21.15.77:0/3740223729 shutdown_connections 2024-09-10T09:55:49.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.564+0000 7f8459c45740 1 --2- 172.21.15.77:0/3740223729 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56331f1836c0 0x56331f185ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:49.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.564+0000 7f8459c45740 1 --2- 172.21.15.77:0/3740223729 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56331f178e00 0x56331f183180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:49.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.564+0000 7f8459c45740 1 --2- 172.21.15.77:0/3740223729 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56331f1784e0 0x56331f1788c0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:49.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.564+0000 7f8459c45740 1 -- 172.21.15.77:0/3740223729 >> 172.21.15.77:0/3740223729 conn(0x56331f174a70 msgr2=0x56331f174e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:49.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.564+0000 7f8459c45740 1 -- 172.21.15.77:0/3740223729 shutdown_connections 2024-09-10T09:55:49.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.564+0000 7f8459c45740 1 -- 172.21.15.77:0/3740223729 wait complete. 2024-09-10T09:55:49.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.564+0000 7f8459c45740 1 -- start start 2024-09-10T09:55:49.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.565+0000 7f8459c45740 1 Processor -- start 2024-09-10T09:55:49.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.565+0000 7f8459c45740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56331f1784e0 0x56331f1805d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:49.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.565+0000 7f8459c45740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56331f178e00 0x56331f180b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:49.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.565+0000 7f8459c45740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56331f1836c0 0x56331f181050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:49.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.565+0000 7f8459c45740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56331f186330 con 0x56331f1784e0 2024-09-10T09:55:49.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.565+0000 7f8459c45740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56331f066920 con 0x56331f178e00 2024-09-10T09:55:49.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.565+0000 7f8459c45740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56331f1864b0 con 0x56331f1836c0 2024-09-10T09:55:49.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.565+0000 7f8459c45740 10 client.0 _ll_register_callbacks cb 0x56331f08a8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:55:49.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.565+0000 7f8455200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56331f1784e0 0x56331f1805d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:49.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.565+0000 7f8455200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56331f1784e0 0x56331f1805d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33312/0 (socket says 172.21.15.77:33312) 2024-09-10T09:55:49.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.565+0000 7f8455200640 1 -- 172.21.15.77:0/3790930941 learned_addr learned my addr 172.21.15.77:0/3790930941 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:55:49.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.566+0000 7f844f400640 1 --2- 172.21.15.77:0/3790930941 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56331f178e00 0x56331f180b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:49.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.566+0000 7f8455200640 1 --2- 172.21.15.77:0/3790930941 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56331f1836c0 0x56331f181050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:49.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.566+0000 7f844f400640 1 -- 172.21.15.77:0/3790930941 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56331f1836c0 msgr2=0x56331f181050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:49.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.566+0000 7f844f400640 1 --2- 172.21.15.77:0/3790930941 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56331f1836c0 0x56331f181050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:49.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.566+0000 7f844f400640 1 -- 172.21.15.77:0/3790930941 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56331f1784e0 msgr2=0x56331f1805d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:49.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.566+0000 7f844f400640 1 --2- 172.21.15.77:0/3790930941 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56331f1784e0 0x56331f1805d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:49.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.566+0000 7f844f400640 1 -- 172.21.15.77:0/3790930941 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56331ee4f2b0 con 0x56331f178e00 2024-09-10T09:55:49.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.566+0000 7f8455200640 1 --2- 172.21.15.77:0/3790930941 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56331f1784e0 0x56331f1805d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:55:49.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.566+0000 7f844f400640 1 --2- 172.21.15.77:0/3790930941 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56331f178e00 0x56331f180b10 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f843c005ab0 tx=0x7f843c032ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c5d99f558d8a7b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:49.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.566+0000 7f843a000640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T09:55:49.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.567+0000 7f843a000640 1 -- 172.21.15.77:0/3790930941 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f843c035040 con 0x56331f178e00 2024-09-10T09:55:49.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.567+0000 7f843a000640 1 -- 172.21.15.77:0/3790930941 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f843c004050 con 0x56331f178e00 2024-09-10T09:55:49.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.567+0000 7f8459c45740 1 -- 172.21.15.77:0/3790930941 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56331f0eb620 con 0x56331f178e00 2024-09-10T09:55:49.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.567+0000 7f843a000640 1 -- 172.21.15.77:0/3790930941 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f843c004250 con 0x56331f178e00 2024-09-10T09:55:49.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.567+0000 7f8459c45740 10 client.25463 fetch_fsmap learned FSMap version 29 2024-09-10T09:55:49.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.567+0000 7f8459c45740 1 -- 172.21.15.77:0/3790930941 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56331f192010 con 0x56331f178e00 2024-09-10T09:55:49.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.567+0000 7f843a000640 1 -- 172.21.15.77:0/3790930941 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x56331f0eb620 con 0x56331f178e00 2024-09-10T09:55:49.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.568+0000 7f843a000640 1 -- 172.21.15.77:0/3790930941 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f843c004a10 con 0x56331f178e00 2024-09-10T09:55:49.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.568+0000 7f8459c45740 10 client.25463 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T09:55:49.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.568+0000 7f8459c45740 10 client.25463 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T09:55:49.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.568+0000 7f8459c45740 20 client.25463 populate_metadata read hostname 'smithi077' 2024-09-10T09:55:49.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.568+0000 7f8459c45740 1 --2- 172.21.15.77:0/3790930941 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x56331f1945e0 0x56331f195a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:49.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.568+0000 7f8459c45740 4 client.25463 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:55:49.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.568+0000 7f8459c45740 1 -- 172.21.15.77:0/3790930941 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56331f0e5b30 con 0x56331f1945e0 2024-09-10T09:55:49.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.568+0000 7f8455200640 1 --2- 172.21.15.77:0/3790930941 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x56331f1945e0 0x56331f195a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:49.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.569+0000 7f8455200640 1 --2- 172.21.15.77:0/3790930941 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x56331f1945e0 0x56331f195a50 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be86b6188221fd34 server_cookie=e015f9d80537786e in_seq=0 out_seq=0 2024-09-10T09:55:49.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.569+0000 7f843a000640 10 client.25463 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T09:55:49.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.569+0000 7f843a000640 1 -- 172.21.15.77:0/3790930941 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56331f0e5b30 con 0x56331f1945e0 2024-09-10T09:55:49.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.569+0000 7f843a000640 10 client.25463 handle_command_reply: tid=0 2024-09-10T09:55:49.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.570+0000 7f8459c45740 1 client.25463 shutdown 2024-09-10T09:55:49.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.570+0000 7f8459c45740 1 -- 172.21.15.77:0/3790930941 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56331f178e00 msgr2=0x56331f180b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:49.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.570+0000 7f8459c45740 1 --2- 172.21.15.77:0/3790930941 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56331f178e00 0x56331f180b10 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f843c005ab0 tx=0x7f843c032ed0 comp rx=0 tx=0).stop 2024-09-10T09:55:49.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.570+0000 7f8459c45740 1 -- 172.21.15.77:0/3790930941 shutdown_connections 2024-09-10T09:55:49.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.570+0000 7f8459c45740 1 --2- 172.21.15.77:0/3790930941 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x56331f1945e0 0x56331f195a50 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:49.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.570+0000 7f8459c45740 1 --2- 172.21.15.77:0/3790930941 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56331f1836c0 0x56331f181050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:49.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.570+0000 7f8459c45740 1 --2- 172.21.15.77:0/3790930941 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56331f178e00 0x56331f180b10 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:49.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.570+0000 7f843a000640 0 client.25463 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T09:55:49.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.570+0000 7f8459c45740 1 --2- 172.21.15.77:0/3790930941 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56331f1784e0 0x56331f1805d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:49.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.570+0000 7f8459c45740 1 -- 172.21.15.77:0/3790930941 >> 172.21.15.77:0/3790930941 conn(0x56331f174a70 msgr2=0x56331f173bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:49.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.570+0000 7f8459c45740 1 -- 172.21.15.77:0/3790930941 shutdown_connections 2024-09-10T09:55:49.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.570+0000 7f8459c45740 1 -- 172.21.15.77:0/3790930941 wait complete. 2024-09-10T09:55:49.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.570+0000 7f8459c45740 20 client.25463 trim_cache size 0 max 16384 2024-09-10T09:55:49.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.588+0000 7f8459c45740 1 Processor -- start 2024-09-10T09:55:49.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.589+0000 7f8459c45740 1 -- start start 2024-09-10T09:55:49.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.589+0000 7f8459c45740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56331f1784e0 0x56331f195a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:49.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.589+0000 7f8459c45740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56331f178e00 0x56331f17b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:49.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.589+0000 7f8459c45740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56331f1836c0 0x56331f17b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:49.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.589+0000 7f8459c45740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56331f193d90 con 0x56331f178e00 2024-09-10T09:55:49.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.589+0000 7f8459c45740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56331f194460 con 0x56331f1836c0 2024-09-10T09:55:49.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.589+0000 7f8459c45740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56331f194b30 con 0x56331f1784e0 2024-09-10T09:55:49.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.589+0000 7f844f400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56331f178e00 0x56331f17b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:49.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.589+0000 7f844f400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56331f178e00 0x56331f17b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33318/0 (socket says 172.21.15.77:33318) 2024-09-10T09:55:49.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.589+0000 7f844f400640 1 -- 172.21.15.77:0/1630682411 learned_addr learned my addr 172.21.15.77:0/1630682411 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:55:49.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.589+0000 7f8455200640 1 --2- 172.21.15.77:0/1630682411 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56331f1784e0 0x56331f195a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:49.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.589+0000 7f8455200640 1 --2- 172.21.15.77:0/1630682411 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56331f1836c0 0x56331f17b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:49.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.590+0000 7f844f400640 1 -- 172.21.15.77:0/1630682411 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56331f1784e0 msgr2=0x56331f195a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:49.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.590+0000 7f844f400640 1 --2- 172.21.15.77:0/1630682411 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56331f1784e0 0x56331f195a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:49.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.590+0000 7f844f400640 1 -- 172.21.15.77:0/1630682411 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56331f1836c0 msgr2=0x56331f17b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:49.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.590+0000 7f844f400640 1 --2- 172.21.15.77:0/1630682411 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56331f1836c0 0x56331f17b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:49.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.590+0000 7f844f400640 1 -- 172.21.15.77:0/1630682411 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56331f190330 con 0x56331f178e00 2024-09-10T09:55:49.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.590+0000 7f8455200640 1 --2- 172.21.15.77:0/1630682411 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56331f1836c0 0x56331f17b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:55:49.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.590+0000 7f844f400640 1 --2- 172.21.15.77:0/1630682411 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56331f178e00 0x56331f17b000 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f843c001eb0 tx=0x7f843c02a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba7452ea6c10c3bb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:49.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.590+0000 7f8444c00640 1 -- 172.21.15.77:0/1630682411 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f843c035030 con 0x56331f178e00 2024-09-10T09:55:49.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.590+0000 7f8444c00640 1 -- 172.21.15.77:0/1630682411 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f843c041070 con 0x56331f178e00 2024-09-10T09:55:49.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.590+0000 7f8444c00640 1 -- 172.21.15.77:0/1630682411 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f843c031c30 con 0x56331f178e00 2024-09-10T09:55:49.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.591+0000 7f8459c45740 1 -- 172.21.15.77:0/1630682411 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56331f178e00 msgr2=0x56331f17b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:49.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.591+0000 7f8459c45740 1 --2- 172.21.15.77:0/1630682411 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56331f178e00 0x56331f17b000 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f843c001eb0 tx=0x7f843c02a3f0 comp rx=0 tx=0).stop 2024-09-10T09:55:49.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.591+0000 7f8459c45740 1 -- 172.21.15.77:0/1630682411 shutdown_connections 2024-09-10T09:55:49.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.591+0000 7f8459c45740 1 --2- 172.21.15.77:0/1630682411 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56331f1836c0 0x56331f17b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:49.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.591+0000 7f8459c45740 1 --2- 172.21.15.77:0/1630682411 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56331f178e00 0x56331f17b000 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:49.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.591+0000 7f8459c45740 1 --2- 172.21.15.77:0/1630682411 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56331f1784e0 0x56331f195a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:49.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.591+0000 7f8459c45740 1 -- 172.21.15.77:0/1630682411 >> 172.21.15.77:0/1630682411 conn(0x56331f174a70 msgr2=0x56331f16f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:49.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.591+0000 7f8459c45740 1 -- 172.21.15.77:0/1630682411 shutdown_connections 2024-09-10T09:55:49.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.591+0000 7f8459c45740 1 -- 172.21.15.77:0/1630682411 wait complete. 2024-09-10T09:55:49.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.591+0000 7f8459c45740 1 -- start start 2024-09-10T09:55:49.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.592+0000 7f8459c45740 1 Processor -- start 2024-09-10T09:55:49.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.592+0000 7f8459c45740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56331f1784e0 0x56331f19dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:49.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.592+0000 7f8459c45740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56331f178e00 0x56331f19e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:49.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.592+0000 7f8459c45740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56331f1836c0 0x56331f1971f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:49.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.592+0000 7f8459c45740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56331f1958d0 con 0x56331f178e00 2024-09-10T09:55:49.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.592+0000 7f8459c45740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56331f195200 con 0x56331f1836c0 2024-09-10T09:55:49.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.592+0000 7f8459c45740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56331f195fa0 con 0x56331f1784e0 2024-09-10T09:55:49.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.592+0000 7f8459c45740 10 client.0 _ll_register_callbacks cb 0x7f843c005dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:55:49.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.592+0000 7f844f400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56331f178e00 0x56331f19e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:49.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.592+0000 7f8455200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56331f1784e0 0x56331f19dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:49.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.592+0000 7f844f400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56331f178e00 0x56331f19e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33330/0 (socket says 172.21.15.77:33330) 2024-09-10T09:55:49.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.592+0000 7f844f400640 1 -- 172.21.15.77:0/2110877836 learned_addr learned my addr 172.21.15.77:0/2110877836 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:55:49.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.593+0000 7f8455200640 1 --2- 172.21.15.77:0/2110877836 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56331f1836c0 0x56331f1971f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:49.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.593+0000 7f844f400640 1 -- 172.21.15.77:0/2110877836 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56331f1784e0 msgr2=0x56331f19dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:49.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.593+0000 7f844f400640 1 --2- 172.21.15.77:0/2110877836 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56331f1784e0 0x56331f19dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:49.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.593+0000 7f844f400640 1 -- 172.21.15.77:0/2110877836 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56331f1836c0 msgr2=0x56331f1971f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:49.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.593+0000 7f844f400640 1 --2- 172.21.15.77:0/2110877836 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56331f1836c0 0x56331f1971f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:49.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.593+0000 7f844f400640 1 -- 172.21.15.77:0/2110877836 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56331f197a20 con 0x56331f178e00 2024-09-10T09:55:49.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.593+0000 7f844f400640 1 --2- 172.21.15.77:0/2110877836 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56331f178e00 0x56331f19e400 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x56331f170400 tx=0x7f843c003940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6340f583ee6af090 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:49.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.593+0000 7f843a000640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-10T09:55:49.600 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.593+0000 7f843a000640 1 -- 172.21.15.77:0/2110877836 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f843c031930 con 0x56331f178e00 2024-09-10T09:55:49.600 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.593+0000 7f843a000640 1 -- 172.21.15.77:0/2110877836 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f843c031ad0 con 0x56331f178e00 2024-09-10T09:55:49.600 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.593+0000 7f8459c45740 1 -- 172.21.15.77:0/2110877836 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f843c041070 con 0x56331f178e00 2024-09-10T09:55:49.600 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.594+0000 7f843a000640 1 -- 172.21.15.77:0/2110877836 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f843c031cd0 con 0x56331f178e00 2024-09-10T09:55:49.600 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.594+0000 7f8459c45740 10 client.15423 fetch_fsmap learned FSMap version 29 2024-09-10T09:55:49.600 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.594+0000 7f8459c45740 1 -- 172.21.15.77:0/2110877836 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56331f19c440 con 0x56331f178e00 2024-09-10T09:55:49.600 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.594+0000 7f843a000640 1 -- 172.21.15.77:0/2110877836 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f843c041070 con 0x56331f178e00 2024-09-10T09:55:49.600 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.594+0000 7f843a000640 1 -- 172.21.15.77:0/2110877836 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f843c005270 con 0x56331f178e00 2024-09-10T09:55:49.600 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.594+0000 7f8459c45740 10 client.15423 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T09:55:49.600 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.594+0000 7f8459c45740 10 client.15423 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T09:55:49.600 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.594+0000 7f8459c45740 20 client.15423 populate_metadata read hostname 'smithi077' 2024-09-10T09:55:49.600 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.594+0000 7f8459c45740 1 --2- 172.21.15.77:0/2110877836 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x56331f202f90 0x56331f201ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:49.600 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.594+0000 7f8459c45740 4 client.15423 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T09:55:49.601 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.594+0000 7f8459c45740 1 -- 172.21.15.77:0/2110877836 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "status"}) -- 0x56331f061f80 con 0x56331f202f90 2024-09-10T09:55:49.601 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.595+0000 7f8455200640 1 --2- 172.21.15.77:0/2110877836 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x56331f202f90 0x56331f201ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:49.601 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.595+0000 7f8455200640 1 --2- 172.21.15.77:0/2110877836 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x56331f202f90 0x56331f201ca0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be5e53404be0e29b server_cookie=d2a282ce0bef32a in_seq=0 out_seq=0 2024-09-10T09:55:49.601 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.595+0000 7f843a000640 10 client.15423 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T09:55:49.601 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.595+0000 7f843a000640 1 -- 172.21.15.77:0/2110877836 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x56331f061f80 con 0x56331f202f90 2024-09-10T09:55:49.601 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.595+0000 7f843a000640 10 client.15423 handle_command_reply: tid=0 2024-09-10T09:55:49.601 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.595+0000 7f8459c45740 1 client.15423 shutdown 2024-09-10T09:55:49.601 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.596+0000 7f8459c45740 1 -- 172.21.15.77:0/2110877836 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56331f178e00 msgr2=0x56331f19e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:49.601 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.596+0000 7f8459c45740 1 --2- 172.21.15.77:0/2110877836 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56331f178e00 0x56331f19e400 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x56331f170400 tx=0x7f843c003940 comp rx=0 tx=0).stop 2024-09-10T09:55:49.601 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.596+0000 7f8459c45740 1 -- 172.21.15.77:0/2110877836 shutdown_connections 2024-09-10T09:55:49.601 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.596+0000 7f8459c45740 1 --2- 172.21.15.77:0/2110877836 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x56331f202f90 0x56331f201ca0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:49.601 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.596+0000 7f8459c45740 1 --2- 172.21.15.77:0/2110877836 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56331f1836c0 0x56331f1971f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:49.601 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.596+0000 7f8459c45740 1 --2- 172.21.15.77:0/2110877836 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56331f178e00 0x56331f19e400 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:49.601 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.596+0000 7f8459c45740 1 --2- 172.21.15.77:0/2110877836 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56331f1784e0 0x56331f19dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:49.602 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.596+0000 7f843a000640 0 client.15423 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T09:55:49.602 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.596+0000 7f8459c45740 1 -- 172.21.15.77:0/2110877836 >> 172.21.15.77:0/2110877836 conn(0x56331f174a70 msgr2=0x56331f17b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:49.602 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.596+0000 7f8459c45740 1 -- 172.21.15.77:0/2110877836 shutdown_connections 2024-09-10T09:55:49.602 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.596+0000 7f8459c45740 1 -- 172.21.15.77:0/2110877836 wait complete. 2024-09-10T09:55:49.602 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.596+0000 7f8459c45740 20 client.15423 trim_cache size 0 max 16384 2024-09-10T09:55:49.602 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-10T09:55:49.602 INFO:teuthology.orchestra.run.smithi077.stdout: "cluster_fsid": "14a55226-6f59-11ef-bcea-c7b262605968", 2024-09-10T09:55:49.602 INFO:teuthology.orchestra.run.smithi077.stdout: "whoami": 0, 2024-09-10T09:55:49.602 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 24563, 2024-09-10T09:55:49.602 INFO:teuthology.orchestra.run.smithi077.stdout: "want_state": "up:active", 2024-09-10T09:55:49.602 INFO:teuthology.orchestra.run.smithi077.stdout: "state": "up:active", 2024-09-10T09:55:49.602 INFO:teuthology.orchestra.run.smithi077.stdout: "fs_name": "cephfs", 2024-09-10T09:55:49.602 INFO:teuthology.orchestra.run.smithi077.stdout: "rank_uptime": 205.283436008, 2024-09-10T09:55:49.603 INFO:teuthology.orchestra.run.smithi077.stdout: "mdsmap_epoch": 29, 2024-09-10T09:55:49.603 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch": 96, 2024-09-10T09:55:49.603 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch_barrier": 96, 2024-09-10T09:55:49.603 INFO:teuthology.orchestra.run.smithi077.stdout: "uptime": 218.913739448 2024-09-10T09:55:49.603 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-10T09:55:49.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.597+0000 7f8455c00640 1 -- 172.21.15.77:0/2386482167 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f84340a2b80 msgr2=0x7f84340a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:49.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.597+0000 7f8455c00640 1 --2- 172.21.15.77:0/2386482167 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f84340a2b80 0x7f84340a5040 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f84400cced0 tx=0x7f84400d5530 comp rx=0 tx=0).stop 2024-09-10T09:55:49.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.597+0000 7f8455c00640 1 -- 172.21.15.77:0/2386482167 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8450103e30 msgr2=0x7f845019a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:49.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.597+0000 7f8455c00640 1 --2- 172.21.15.77:0/2386482167 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8450103e30 0x7f845019a460 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f844000e9f0 tx=0x7f844000eeb0 comp rx=0 tx=0).stop 2024-09-10T09:55:49.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.597+0000 7f8455c00640 1 -- 172.21.15.77:0/2386482167 shutdown_connections 2024-09-10T09:55:49.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.597+0000 7f8455c00640 1 --2- 172.21.15.77:0/2386482167 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f84340a2b80 0x7f84340a5040 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:49.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.597+0000 7f8455c00640 1 --2- 172.21.15.77:0/2386482167 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8450108f60 0x7f845019ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:49.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.597+0000 7f8455c00640 1 --2- 172.21.15.77:0/2386482167 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84501047e0 0x7f845019a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:49.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.597+0000 7f8455c00640 1 --2- 172.21.15.77:0/2386482167 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8450103e30 0x7f845019a460 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:49.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.597+0000 7f8455c00640 1 -- 172.21.15.77:0/2386482167 >> 172.21.15.77:0/2386482167 conn(0x7f84500ff8d0 msgr2=0x7f8450100bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:49.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.597+0000 7f8455c00640 1 -- 172.21.15.77:0/2386482167 shutdown_connections 2024-09-10T09:55:49.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:49.597+0000 7f8455c00640 1 -- 172.21.15.77:0/2386482167 wait complete. 2024-09-10T09:55:50.349 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-09-10T09:55:51.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:55:50 smithi134 ceph-mon[35674]: pgmap v411: 129 pgs: 129 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-10T09:55:51.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:55:50 smithi120 ceph-mon[34770]: pgmap v411: 129 pgs: 129 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-10T09:55:51.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:55:50 smithi077 ceph-mon[30319]: pgmap v411: 129 pgs: 129 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-10T09:55:53.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:55:52 smithi134 ceph-mon[35674]: pgmap v412: 129 pgs: 129 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-10T09:55:53.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:55:52 smithi120 ceph-mon[34770]: pgmap v412: 129 pgs: 129 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-10T09:55:53.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:55:52 smithi077 ceph-mon[30319]: pgmap v412: 129 pgs: 129 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-10T09:55:55.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:55:54 smithi134 ceph-mon[35674]: pgmap v413: 129 pgs: 129 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 853 B/s wr, 11 op/s 2024-09-10T09:55:55.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:55:54 smithi120 ceph-mon[34770]: pgmap v413: 129 pgs: 129 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 853 B/s wr, 11 op/s 2024-09-10T09:55:55.176 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:55:54 smithi077 ceph-mon[30319]: pgmap v413: 129 pgs: 129 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 853 B/s wr, 11 op/s 2024-09-10T09:55:57.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:55:56 smithi134 ceph-mon[35674]: pgmap v414: 129 pgs: 129 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T09:55:57.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:55:56 smithi120 ceph-mon[34770]: pgmap v414: 129 pgs: 129 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T09:55:57.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:55:56 smithi077 ceph-mon[30319]: pgmap v414: 129 pgs: 129 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T09:55:57.915 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T09:55:58.295 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:55:58.878 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.876+0000 7fd488800640 1 -- 172.21.15.77:0/438685403 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd480074620 msgr2=0x7fd480074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:58.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.876+0000 7fd488800640 1 --2- 172.21.15.77:0/438685403 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd480074620 0x7fd480074a00 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd474009900 tx=0x7fd47402a480 comp rx=0 tx=0).stop 2024-09-10T09:55:58.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.876+0000 7fd488800640 1 -- 172.21.15.77:0/438685403 shutdown_connections 2024-09-10T09:55:58.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.876+0000 7fd488800640 1 --2- 172.21.15.77:0/438685403 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd48010e3c0 0x7fd480110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:58.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.876+0000 7fd488800640 1 --2- 172.21.15.77:0/438685403 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd480074f40 0x7fd48010de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:58.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.876+0000 7fd488800640 1 --2- 172.21.15.77:0/438685403 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd480074620 0x7fd480074a00 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:58.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.876+0000 7fd488800640 1 -- 172.21.15.77:0/438685403 >> 172.21.15.77:0/438685403 conn(0x7fd4800fd8c0 msgr2=0x7fd4800ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:58.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.876+0000 7fd488800640 1 -- 172.21.15.77:0/438685403 shutdown_connections 2024-09-10T09:55:58.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.876+0000 7fd488800640 1 -- 172.21.15.77:0/438685403 wait complete. 2024-09-10T09:55:58.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.877+0000 7fd488800640 1 Processor -- start 2024-09-10T09:55:58.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.878+0000 7fd488800640 1 -- start start 2024-09-10T09:55:58.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.878+0000 7fd488800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd480074620 0x7fd48019a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:58.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.878+0000 7fd488800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd480074f40 0x7fd48019aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:58.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.878+0000 7fd488800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd48010e3c0 0x7fd48019df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:58.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.878+0000 7fd488800640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd4801141a0 con 0x7fd480074620 2024-09-10T09:55:58.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.878+0000 7fd488800640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd480114020 con 0x7fd480074f40 2024-09-10T09:55:58.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.878+0000 7fd488800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd480114320 con 0x7fd48010e3c0 2024-09-10T09:55:58.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.879+0000 7fd486800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd48010e3c0 0x7fd48019df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:58.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.879+0000 7fd486800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd48010e3c0 0x7fd48019df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:46082/0 (socket says 172.21.15.77:46082) 2024-09-10T09:55:58.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.879+0000 7fd486800640 1 -- 172.21.15.77:0/1069546468 learned_addr learned my addr 172.21.15.77:0/1069546468 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:55:58.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.879+0000 7fd485e00640 1 --2- 172.21.15.77:0/1069546468 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd480074620 0x7fd48019a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:58.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.879+0000 7fd485400640 1 --2- 172.21.15.77:0/1069546468 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd480074f40 0x7fd48019aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:58.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.879+0000 7fd486800640 1 -- 172.21.15.77:0/1069546468 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd480074f40 msgr2=0x7fd48019aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:58.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.879+0000 7fd486800640 1 --2- 172.21.15.77:0/1069546468 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd480074f40 0x7fd48019aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:58.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.879+0000 7fd486800640 1 -- 172.21.15.77:0/1069546468 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd480074620 msgr2=0x7fd48019a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:58.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.879+0000 7fd486800640 1 --2- 172.21.15.77:0/1069546468 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd480074620 0x7fd48019a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:58.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.879+0000 7fd486800640 1 -- 172.21.15.77:0/1069546468 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd48019e6b0 con 0x7fd48010e3c0 2024-09-10T09:55:58.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.880+0000 7fd486800640 1 --2- 172.21.15.77:0/1069546468 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd48010e3c0 0x7fd48019df80 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd47c00eb10 tx=0x7fd47c00c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:58.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.880+0000 7fd47aa00640 1 -- 172.21.15.77:0/1069546468 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd47c004b20 con 0x7fd48010e3c0 2024-09-10T09:55:58.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.880+0000 7fd488800640 1 -- 172.21.15.77:0/1069546468 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd48019e9a0 con 0x7fd48010e3c0 2024-09-10T09:55:58.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.880+0000 7fd47aa00640 1 -- 172.21.15.77:0/1069546468 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd47c004cc0 con 0x7fd48010e3c0 2024-09-10T09:55:58.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.880+0000 7fd488800640 1 -- 172.21.15.77:0/1069546468 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4801a7310 con 0x7fd48010e3c0 2024-09-10T09:55:58.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.880+0000 7fd47aa00640 1 -- 172.21.15.77:0/1069546468 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd47c010690 con 0x7fd48010e3c0 2024-09-10T09:55:58.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.883+0000 7fd48b299740 1 Processor -- start 2024-09-10T09:55:58.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.883+0000 7fd47aa00640 1 -- 172.21.15.77:0/1069546468 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd47c010830 con 0x7fd48010e3c0 2024-09-10T09:55:58.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.884+0000 7fd48b299740 1 -- start start 2024-09-10T09:55:58.886 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.884+0000 7fd48b299740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b5ffd9ab90 0x55b5ffd9af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:58.886 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.884+0000 7fd48b299740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b5ffd9b4b0 0x55b5ffda5830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:58.886 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.884+0000 7fd48b299740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b5ffda5d70 0x55b5ffda8160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:58.886 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.884+0000 7fd48b299740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55b5ffd013e0 con 0x55b5ffd9ab90 2024-09-10T09:55:58.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.884+0000 7fd48b299740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b5ffca8d80 con 0x55b5ffd9b4b0 2024-09-10T09:55:58.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.884+0000 7fd48b299740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b5ffcfd3a0 con 0x55b5ffda5d70 2024-09-10T09:55:58.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.884+0000 7fd485400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b5ffd9ab90 0x55b5ffd9af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:58.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.884+0000 7fd485400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b5ffd9ab90 0x55b5ffd9af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56056/0 (socket says 172.21.15.77:56056) 2024-09-10T09:55:58.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.884+0000 7fd485400640 1 -- 172.21.15.77:0/182847958 learned_addr learned my addr 172.21.15.77:0/182847958 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:55:58.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.885+0000 7fd485400640 1 --2- 172.21.15.77:0/182847958 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b5ffda5d70 0x55b5ffda8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:58.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.885+0000 7fd485e00640 1 --2- 172.21.15.77:0/182847958 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b5ffd9b4b0 0x55b5ffda5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:58.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.885+0000 7fd485400640 1 -- 172.21.15.77:0/182847958 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b5ffda5d70 msgr2=0x55b5ffda8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:58.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.885+0000 7fd485400640 1 --2- 172.21.15.77:0/182847958 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b5ffda5d70 0x55b5ffda8160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:58.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.885+0000 7fd485400640 1 -- 172.21.15.77:0/182847958 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b5ffd9b4b0 msgr2=0x55b5ffda5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:58.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.885+0000 7fd485400640 1 --2- 172.21.15.77:0/182847958 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b5ffd9b4b0 0x55b5ffda5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:58.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.885+0000 7fd485400640 1 -- 172.21.15.77:0/182847958 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5ffcc7ae0 con 0x55b5ffd9ab90 2024-09-10T09:55:58.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.885+0000 7fd485e00640 1 --2- 172.21.15.77:0/182847958 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b5ffd9b4b0 0x55b5ffda5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:55:58.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.885+0000 7fd47aa00640 1 --2- 172.21.15.77:0/1069546468 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fd4680a2c00 0x7fd4680a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:58.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.886+0000 7fd485400640 1 --2- 172.21.15.77:0/182847958 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b5ffd9ab90 0x55b5ffd9af70 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fd4700137c0 tx=0x7fd470009570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1241834f4d21f7dc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:58.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.886+0000 7fd47aa00640 1 -- 172.21.15.77:0/1069546468 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fd47c0cf720 con 0x7fd48010e3c0 2024-09-10T09:55:58.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.886+0000 7fd46f000640 1 -- 172.21.15.77:0/182847958 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd470013d40 con 0x55b5ffd9ab90 2024-09-10T09:55:58.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.886+0000 7fd46f000640 1 -- 172.21.15.77:0/182847958 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd470013120 con 0x55b5ffd9ab90 2024-09-10T09:55:58.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.886+0000 7fd46f000640 1 -- 172.21.15.77:0/182847958 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4700035f0 con 0x55b5ffd9ab90 2024-09-10T09:55:58.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.886+0000 7fd485e00640 1 --2- 172.21.15.77:0/1069546468 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fd4680a2c00 0x7fd4680a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:58.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.887+0000 7fd48b299740 1 -- 172.21.15.77:0/182847958 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b5ffd9ab90 msgr2=0x55b5ffd9af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:58.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.887+0000 7fd48b299740 1 --2- 172.21.15.77:0/182847958 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b5ffd9ab90 0x55b5ffd9af70 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fd4700137c0 tx=0x7fd470009570 comp rx=0 tx=0).stop 2024-09-10T09:55:58.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.887+0000 7fd48b299740 1 -- 172.21.15.77:0/182847958 shutdown_connections 2024-09-10T09:55:58.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.887+0000 7fd48b299740 1 --2- 172.21.15.77:0/182847958 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b5ffda5d70 0x55b5ffda8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:58.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.887+0000 7fd48b299740 1 --2- 172.21.15.77:0/182847958 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b5ffd9b4b0 0x55b5ffda5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:58.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.887+0000 7fd48b299740 1 --2- 172.21.15.77:0/182847958 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b5ffd9ab90 0x55b5ffd9af70 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:58.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.887+0000 7fd48b299740 1 -- 172.21.15.77:0/182847958 >> 172.21.15.77:0/182847958 conn(0x55b5ffd97120 msgr2=0x55b5ffd97500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:58.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.887+0000 7fd48b299740 1 -- 172.21.15.77:0/182847958 shutdown_connections 2024-09-10T09:55:58.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.887+0000 7fd48b299740 1 -- 172.21.15.77:0/182847958 wait complete. 2024-09-10T09:55:58.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.887+0000 7fd485e00640 1 --2- 172.21.15.77:0/1069546468 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fd4680a2c00 0x7fd4680a50c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd474000c00 tx=0x7fd474005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:58.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.888+0000 7fd48b299740 1 -- start start 2024-09-10T09:55:58.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.888+0000 7fd48b299740 1 Processor -- start 2024-09-10T09:55:58.892 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.889+0000 7fd48b299740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b5ffd9ab90 0x55b5ffda2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:58.892 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.889+0000 7fd48b299740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b5ffd9b4b0 0x55b5ffda30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:58.892 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.889+0000 7fd48b299740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b5ffda5d70 0x55b5ffda3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:58.892 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.889+0000 7fd48b299740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55b5ffda89e0 con 0x55b5ffda5d70 2024-09-10T09:55:58.892 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.889+0000 7fd48b299740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b5ffcbc0f0 con 0x55b5ffd9ab90 2024-09-10T09:55:58.892 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.889+0000 7fd48b299740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b5ffda8b60 con 0x55b5ffd9b4b0 2024-09-10T09:55:58.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.889+0000 7fd48b299740 10 client.0 _ll_register_callbacks cb 0x55b5ffcad9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:55:58.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.889+0000 7fd485e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b5ffd9ab90 0x55b5ffda2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:58.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.889+0000 7fd486800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b5ffda5d70 0x55b5ffda3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:58.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.889+0000 7fd485400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b5ffd9b4b0 0x55b5ffda30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:58.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.889+0000 7fd486800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b5ffda5d70 0x55b5ffda3610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56060/0 (socket says 172.21.15.77:56060) 2024-09-10T09:55:58.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.889+0000 7fd486800640 1 -- 172.21.15.77:0/3663785911 learned_addr learned my addr 172.21.15.77:0/3663785911 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:55:58.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.889+0000 7fd485e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b5ffd9ab90 0x55b5ffda2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:60764/0 (socket says 172.21.15.77:60764) 2024-09-10T09:55:58.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.889+0000 7fd486800640 1 -- 172.21.15.77:0/3663785911 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b5ffd9b4b0 msgr2=0x55b5ffda30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:58.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.889+0000 7fd486800640 1 --2- 172.21.15.77:0/3663785911 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b5ffd9b4b0 0x55b5ffda30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:58.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.889+0000 7fd486800640 1 -- 172.21.15.77:0/3663785911 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b5ffd9ab90 msgr2=0x55b5ffda2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:58.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.889+0000 7fd486800640 1 --2- 172.21.15.77:0/3663785911 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b5ffd9ab90 0x55b5ffda2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:58.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.889+0000 7fd486800640 1 -- 172.21.15.77:0/3663785911 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5ffa723c0 con 0x55b5ffda5d70 2024-09-10T09:55:58.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.889+0000 7fd485400640 1 --2- 172.21.15.77:0/3663785911 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b5ffd9b4b0 0x55b5ffda30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:55:58.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.890+0000 7fd486800640 1 --2- 172.21.15.77:0/3663785911 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b5ffda5d70 0x55b5ffda3610 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fd47c0a42d0 tx=0x7fd47c09c290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b76438eecc1d8b64 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:58.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.890+0000 7fd463400640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-10T09:55:58.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.890+0000 7fd463400640 1 -- 172.21.15.77:0/3663785911 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd47c09ff40 con 0x55b5ffda5d70 2024-09-10T09:55:58.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.890+0000 7fd463400640 1 -- 172.21.15.77:0/3663785911 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fd47c0a4370 con 0x55b5ffda5d70 2024-09-10T09:55:58.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.890+0000 7fd48b299740 1 -- 172.21.15.77:0/3663785911 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd470013120 con 0x55b5ffda5d70 2024-09-10T09:55:58.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.890+0000 7fd463400640 1 -- 172.21.15.77:0/3663785911 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd47c0a4800 con 0x55b5ffda5d70 2024-09-10T09:55:58.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.890+0000 7fd48b299740 10 client.15435 fetch_fsmap learned FSMap version 29 2024-09-10T09:55:58.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.890+0000 7fd48b299740 1 -- 172.21.15.77:0/3663785911 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b5ffdb4680 con 0x55b5ffda5d70 2024-09-10T09:55:58.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.891+0000 7fd463400640 1 -- 172.21.15.77:0/3663785911 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd470013120 con 0x55b5ffda5d70 2024-09-10T09:55:58.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.891+0000 7fd463400640 1 -- 172.21.15.77:0/3663785911 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fd47c0ce040 con 0x55b5ffda5d70 2024-09-10T09:55:58.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.891+0000 7fd48b299740 10 client.15435 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T09:55:58.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.891+0000 7fd48b299740 10 client.15435 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:55:58.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.891+0000 7fd48b299740 20 client.15435 populate_metadata read hostname 'smithi077' 2024-09-10T09:55:58.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.891+0000 7fd48b299740 1 --2- 172.21.15.77:0/3663785911 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55b5ffdb5a90 0x55b5ffdb7f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:58.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.891+0000 7fd48b299740 4 client.15435 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:55:58.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.891+0000 7fd48b299740 1 -- 172.21.15.77:0/3663785911 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b5ffd0e950 con 0x55b5ffdb5a90 2024-09-10T09:55:58.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.892+0000 7fd485e00640 1 --2- 172.21.15.77:0/3663785911 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55b5ffdb5a90 0x55b5ffdb7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:58.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.892+0000 7fd485e00640 1 --2- 172.21.15.77:0/3663785911 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55b5ffdb5a90 0x55b5ffdb7f50 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d403fe2f5445255 server_cookie=8e33a2bb87355a1b in_seq=0 out_seq=0 2024-09-10T09:55:58.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.892+0000 7fd463400640 10 client.15435 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T09:55:58.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.893+0000 7fd463400640 1 -- 172.21.15.77:0/3663785911 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b5ffd0e950 con 0x55b5ffdb5a90 2024-09-10T09:55:58.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.893+0000 7fd463400640 10 client.15435 handle_command_reply: tid=0 2024-09-10T09:55:58.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.893+0000 7fd48b299740 1 client.15435 shutdown 2024-09-10T09:55:58.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.893+0000 7fd48b299740 1 -- 172.21.15.77:0/3663785911 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b5ffda5d70 msgr2=0x55b5ffda3610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:58.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.893+0000 7fd48b299740 1 --2- 172.21.15.77:0/3663785911 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b5ffda5d70 0x55b5ffda3610 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fd47c0a42d0 tx=0x7fd47c09c290 comp rx=0 tx=0).stop 2024-09-10T09:55:58.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.893+0000 7fd48b299740 1 -- 172.21.15.77:0/3663785911 shutdown_connections 2024-09-10T09:55:58.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.893+0000 7fd48b299740 1 --2- 172.21.15.77:0/3663785911 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55b5ffdb5a90 0x55b5ffdb7f50 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:58.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.893+0000 7fd48b299740 1 --2- 172.21.15.77:0/3663785911 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b5ffda5d70 0x55b5ffda3610 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:58.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.893+0000 7fd48b299740 1 --2- 172.21.15.77:0/3663785911 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b5ffd9b4b0 0x55b5ffda30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:58.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.893+0000 7fd48b299740 1 --2- 172.21.15.77:0/3663785911 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b5ffd9ab90 0x55b5ffda2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:58.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.893+0000 7fd463400640 0 client.15435 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T09:55:58.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.893+0000 7fd48b299740 1 -- 172.21.15.77:0/3663785911 >> 172.21.15.77:0/3663785911 conn(0x55b5ffd97120 msgr2=0x55b5ffd9ce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:58.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.894+0000 7fd48b299740 1 -- 172.21.15.77:0/3663785911 shutdown_connections 2024-09-10T09:55:58.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.894+0000 7fd48b299740 1 -- 172.21.15.77:0/3663785911 wait complete. 2024-09-10T09:55:58.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.894+0000 7fd48b299740 20 client.15435 trim_cache size 0 max 16384 2024-09-10T09:55:58.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.912+0000 7fd48b299740 1 Processor -- start 2024-09-10T09:55:58.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.912+0000 7fd48b299740 1 -- start start 2024-09-10T09:55:58.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.912+0000 7fd48b299740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b5ffd9ab90 0x55b5ffda1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:58.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.912+0000 7fd48b299740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b5ffd9b4b0 0x55b5ffda4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:58.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.912+0000 7fd48b299740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b5ffda5d70 0x55b5ffdbd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:58.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.912+0000 7fd48b299740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd46400fc80 con 0x55b5ffd9ab90 2024-09-10T09:55:58.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.912+0000 7fd48b299740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd4640117c0 con 0x55b5ffd9b4b0 2024-09-10T09:55:58.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.912+0000 7fd48b299740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd464010350 con 0x55b5ffda5d70 2024-09-10T09:55:58.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.913+0000 7fd485e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b5ffd9ab90 0x55b5ffda1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:58.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.913+0000 7fd485e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b5ffd9ab90 0x55b5ffda1460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56072/0 (socket says 172.21.15.77:56072) 2024-09-10T09:55:58.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.913+0000 7fd485e00640 1 -- 172.21.15.77:0/3580960054 learned_addr learned my addr 172.21.15.77:0/3580960054 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:55:58.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.913+0000 7fd486800640 1 --2- 172.21.15.77:0/3580960054 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b5ffda5d70 0x55b5ffdbd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:58.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.913+0000 7fd485400640 1 --2- 172.21.15.77:0/3580960054 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b5ffd9b4b0 0x55b5ffda4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:58.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.913+0000 7fd485e00640 1 -- 172.21.15.77:0/3580960054 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b5ffda5d70 msgr2=0x55b5ffdbd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:58.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.913+0000 7fd485e00640 1 --2- 172.21.15.77:0/3580960054 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b5ffda5d70 0x55b5ffdbd820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:58.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.913+0000 7fd485e00640 1 -- 172.21.15.77:0/3580960054 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b5ffd9b4b0 msgr2=0x55b5ffda4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:58.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.913+0000 7fd485e00640 1 --2- 172.21.15.77:0/3580960054 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b5ffd9b4b0 0x55b5ffda4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:58.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.913+0000 7fd485e00640 1 -- 172.21.15.77:0/3580960054 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5ffd96f50 con 0x55b5ffd9ab90 2024-09-10T09:55:58.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.913+0000 7fd485e00640 1 --2- 172.21.15.77:0/3580960054 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b5ffd9ab90 0x55b5ffda1460 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fd474005c40 tx=0x7fd47403e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9da8c74af0c82ef server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:58.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.914+0000 7fd46e600640 1 -- 172.21.15.77:0/3580960054 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd474035040 con 0x55b5ffd9ab90 2024-09-10T09:55:58.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.914+0000 7fd46e600640 1 -- 172.21.15.77:0/3580960054 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4740325c0 con 0x55b5ffd9ab90 2024-09-10T09:55:58.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.914+0000 7fd46e600640 1 -- 172.21.15.77:0/3580960054 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd474003830 con 0x55b5ffd9ab90 2024-09-10T09:55:58.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.915+0000 7fd48b299740 1 -- 172.21.15.77:0/3580960054 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b5ffd9ab90 msgr2=0x55b5ffda1460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:58.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.915+0000 7fd48b299740 1 --2- 172.21.15.77:0/3580960054 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b5ffd9ab90 0x55b5ffda1460 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fd474005c40 tx=0x7fd47403e000 comp rx=0 tx=0).stop 2024-09-10T09:55:58.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.915+0000 7fd48b299740 1 -- 172.21.15.77:0/3580960054 shutdown_connections 2024-09-10T09:55:58.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.915+0000 7fd48b299740 1 --2- 172.21.15.77:0/3580960054 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b5ffda5d70 0x55b5ffdbd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:58.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.915+0000 7fd48b299740 1 --2- 172.21.15.77:0/3580960054 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b5ffd9b4b0 0x55b5ffda4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:58.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.915+0000 7fd48b299740 1 --2- 172.21.15.77:0/3580960054 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b5ffd9ab90 0x55b5ffda1460 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:58.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.915+0000 7fd48b299740 1 -- 172.21.15.77:0/3580960054 >> 172.21.15.77:0/3580960054 conn(0x55b5ffd97120 msgr2=0x55b5ffda43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:58.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.915+0000 7fd48b299740 1 -- 172.21.15.77:0/3580960054 shutdown_connections 2024-09-10T09:55:58.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.915+0000 7fd48b299740 1 -- 172.21.15.77:0/3580960054 wait complete. 2024-09-10T09:55:58.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.915+0000 7fd48b299740 1 -- start start 2024-09-10T09:55:58.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.916+0000 7fd48b299740 1 Processor -- start 2024-09-10T09:55:58.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.916+0000 7fd48b299740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b5ffd9ab90 0x55b5ffdaa1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:58.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.916+0000 7fd48b299740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b5ffd9b4b0 0x55b5ffdaa700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:58.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.916+0000 7fd48b299740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b5ffda5d70 0x55b5ffe1f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:58.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.916+0000 7fd48b299740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd4640110f0 con 0x55b5ffd9b4b0 2024-09-10T09:55:58.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.916+0000 7fd48b299740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd464010a20 con 0x55b5ffda5d70 2024-09-10T09:55:58.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.916+0000 7fd48b299740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd46400dc60 con 0x55b5ffd9ab90 2024-09-10T09:55:58.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.916+0000 7fd48b299740 10 client.0 _ll_register_callbacks cb 0x7fd47c09b6d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:55:58.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.916+0000 7fd485400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b5ffd9b4b0 0x55b5ffdaa700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:58.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.917+0000 7fd485400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b5ffd9b4b0 0x55b5ffdaa700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56084/0 (socket says 172.21.15.77:56084) 2024-09-10T09:55:58.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.917+0000 7fd485400640 1 -- 172.21.15.77:0/2846212564 learned_addr learned my addr 172.21.15.77:0/2846212564 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:55:58.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.917+0000 7fd485e00640 1 --2- 172.21.15.77:0/2846212564 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b5ffd9ab90 0x55b5ffdaa1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:58.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.917+0000 7fd486800640 1 --2- 172.21.15.77:0/2846212564 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b5ffda5d70 0x55b5ffe1f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:58.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.917+0000 7fd485e00640 1 -- 172.21.15.77:0/2846212564 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b5ffda5d70 msgr2=0x55b5ffe1f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:58.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.917+0000 7fd485e00640 1 --2- 172.21.15.77:0/2846212564 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b5ffda5d70 0x55b5ffe1f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:58.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.917+0000 7fd485e00640 1 -- 172.21.15.77:0/2846212564 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b5ffd9b4b0 msgr2=0x55b5ffdaa700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:58.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.917+0000 7fd485e00640 1 --2- 172.21.15.77:0/2846212564 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b5ffd9b4b0 0x55b5ffdaa700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:58.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.917+0000 7fd485e00640 1 -- 172.21.15.77:0/2846212564 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5ffe1fd30 con 0x55b5ffd9ab90 2024-09-10T09:55:58.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.917+0000 7fd485e00640 1 --2- 172.21.15.77:0/2846212564 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b5ffd9ab90 0x55b5ffdaa1c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd47402aa70 tx=0x7fd4740050c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f20f1377af5d42f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:58.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.917+0000 7fd463400640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-10T09:55:58.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.918+0000 7fd463400640 1 -- 172.21.15.77:0/2846212564 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd474062070 con 0x55b5ffd9ab90 2024-09-10T09:55:58.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.918+0000 7fd463400640 1 -- 172.21.15.77:0/2846212564 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fd474032cc0 con 0x55b5ffd9ab90 2024-09-10T09:55:58.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.918+0000 7fd48b299740 1 -- 172.21.15.77:0/2846212564 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4740325c0 con 0x55b5ffd9ab90 2024-09-10T09:55:58.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.918+0000 7fd463400640 1 -- 172.21.15.77:0/2846212564 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd47406e050 con 0x55b5ffd9ab90 2024-09-10T09:55:58.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.918+0000 7fd48b299740 10 client.25462 fetch_fsmap learned FSMap version 29 2024-09-10T09:55:58.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.918+0000 7fd48b299740 1 -- 172.21.15.77:0/2846212564 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b5ffe20f50 con 0x55b5ffd9ab90 2024-09-10T09:55:58.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.919+0000 7fd463400640 1 -- 172.21.15.77:0/2846212564 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4740325c0 con 0x55b5ffd9ab90 2024-09-10T09:55:58.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.919+0000 7fd463400640 1 -- 172.21.15.77:0/2846212564 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fd474003b20 con 0x55b5ffd9ab90 2024-09-10T09:55:58.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.919+0000 7fd48b299740 10 client.25462 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T09:55:58.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.919+0000 7fd48b299740 10 client.25462 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:55:58.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.919+0000 7fd48b299740 20 client.25462 populate_metadata read hostname 'smithi077' 2024-09-10T09:55:58.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.919+0000 7fd48b299740 1 --2- 172.21.15.77:0/2846212564 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55b5ffe21610 0x55b5ffe23a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:58.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.919+0000 7fd48b299740 4 client.25462 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T09:55:58.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.919+0000 7fd48b299740 1 -- 172.21.15.77:0/2846212564 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b5ffd08e60 con 0x55b5ffe21610 2024-09-10T09:55:58.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.920+0000 7fd485400640 1 --2- 172.21.15.77:0/2846212564 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55b5ffe21610 0x55b5ffe23a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:58.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.920+0000 7fd485400640 1 --2- 172.21.15.77:0/2846212564 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55b5ffe21610 0x55b5ffe23a00 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24e0057ecb91fb30 server_cookie=9d190fcb3c562003 in_seq=0 out_seq=0 2024-09-10T09:55:58.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.920+0000 7fd463400640 10 client.25462 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T09:55:58.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.920+0000 7fd463400640 1 -- 172.21.15.77:0/2846212564 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b5ffd08e60 con 0x55b5ffe21610 2024-09-10T09:55:58.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.920+0000 7fd463400640 10 client.25462 handle_command_reply: tid=0 2024-09-10T09:55:58.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.920+0000 7fd48b299740 1 client.25462 shutdown 2024-09-10T09:55:58.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.921+0000 7fd48b299740 1 -- 172.21.15.77:0/2846212564 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b5ffd9ab90 msgr2=0x55b5ffdaa1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:58.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.921+0000 7fd48b299740 1 --2- 172.21.15.77:0/2846212564 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b5ffd9ab90 0x55b5ffdaa1c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd47402aa70 tx=0x7fd4740050c0 comp rx=0 tx=0).stop 2024-09-10T09:55:58.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.921+0000 7fd48b299740 1 -- 172.21.15.77:0/2846212564 shutdown_connections 2024-09-10T09:55:58.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.921+0000 7fd48b299740 1 --2- 172.21.15.77:0/2846212564 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55b5ffe21610 0x55b5ffe23a00 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:58.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.921+0000 7fd48b299740 1 --2- 172.21.15.77:0/2846212564 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b5ffda5d70 0x55b5ffe1f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:58.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.921+0000 7fd48b299740 1 --2- 172.21.15.77:0/2846212564 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b5ffd9b4b0 0x55b5ffdaa700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:58.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.921+0000 7fd48b299740 1 --2- 172.21.15.77:0/2846212564 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b5ffd9ab90 0x55b5ffdaa1c0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:58.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.921+0000 7fd48b299740 1 -- 172.21.15.77:0/2846212564 >> 172.21.15.77:0/2846212564 conn(0x55b5ffd97120 msgr2=0x55b5ffda4c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:58.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.921+0000 7fd463400640 0 client.25462 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T09:55:58.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.921+0000 7fd48b299740 1 -- 172.21.15.77:0/2846212564 shutdown_connections 2024-09-10T09:55:58.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.921+0000 7fd48b299740 1 -- 172.21.15.77:0/2846212564 wait complete. 2024-09-10T09:55:58.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.921+0000 7fd48b299740 20 client.25462 trim_cache size 0 max 16384 2024-09-10T09:55:58.926 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-10T09:55:58.926 INFO:teuthology.orchestra.run.smithi077.stdout: "status": "no active scrubs running", 2024-09-10T09:55:58.927 INFO:teuthology.orchestra.run.smithi077.stdout: "scrubs": {} 2024-09-10T09:55:58.927 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-10T09:55:58.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.922+0000 7fd488800640 1 -- 172.21.15.77:0/1069546468 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fd4680a2c00 msgr2=0x7fd4680a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:58.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.922+0000 7fd488800640 1 --2- 172.21.15.77:0/1069546468 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fd4680a2c00 0x7fd4680a50c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd474000c00 tx=0x7fd474005b50 comp rx=0 tx=0).stop 2024-09-10T09:55:58.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.922+0000 7fd488800640 1 -- 172.21.15.77:0/1069546468 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd48010e3c0 msgr2=0x7fd48019df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:58.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.922+0000 7fd488800640 1 --2- 172.21.15.77:0/1069546468 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd48010e3c0 0x7fd48019df80 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd47c00eb10 tx=0x7fd47c00c2d0 comp rx=0 tx=0).stop 2024-09-10T09:55:58.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.922+0000 7fd488800640 1 -- 172.21.15.77:0/1069546468 shutdown_connections 2024-09-10T09:55:58.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.922+0000 7fd488800640 1 --2- 172.21.15.77:0/1069546468 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fd4680a2c00 0x7fd4680a50c0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:58.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.922+0000 7fd488800640 1 --2- 172.21.15.77:0/1069546468 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd48010e3c0 0x7fd48019df80 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:58.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.922+0000 7fd488800640 1 --2- 172.21.15.77:0/1069546468 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd480074f40 0x7fd48019aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:58.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.922+0000 7fd488800640 1 --2- 172.21.15.77:0/1069546468 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd480074620 0x7fd48019a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:58.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.922+0000 7fd488800640 1 -- 172.21.15.77:0/1069546468 >> 172.21.15.77:0/1069546468 conn(0x7fd4800fd8c0 msgr2=0x7fd4800fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:58.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.922+0000 7fd488800640 1 -- 172.21.15.77:0/1069546468 shutdown_connections 2024-09-10T09:55:58.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:55:58.922+0000 7fd488800640 1 -- 172.21.15.77:0/1069546468 wait complete. 2024-09-10T09:55:59.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:55:58 smithi134 ceph-mon[35674]: pgmap v415: 129 pgs: 129 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-10T09:55:59.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:55:58 smithi120 ceph-mon[34770]: pgmap v415: 129 pgs: 129 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-10T09:55:59.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:55:58 smithi077 ceph-mon[30319]: pgmap v415: 129 pgs: 129 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-10T09:55:59.717 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T09:55:59.718 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:56:00.105 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:56:00.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:00.805+0000 7f42bfe00640 1 -- 172.21.15.77:0/2917053595 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42c01079c0 msgr2=0x7f42c0107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:00.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:00.805+0000 7f42bfe00640 1 --2- 172.21.15.77:0/2917053595 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42c01079c0 0x7f42c0107da0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f42b0009930 tx=0x7f42b002a420 comp rx=0 tx=0).stop 2024-09-10T09:56:00.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:00.806+0000 7f42bfe00640 1 -- 172.21.15.77:0/2917053595 shutdown_connections 2024-09-10T09:56:00.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:00.806+0000 7f42bfe00640 1 --2- 172.21.15.77:0/2917053595 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f42c0102360 0x7f42c010e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:00.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:00.806+0000 7f42bfe00640 1 --2- 172.21.15.77:0/2917053595 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f42c01019c0 0x7f42c0101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:00.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:00.806+0000 7f42bfe00640 1 --2- 172.21.15.77:0/2917053595 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42c01079c0 0x7f42c0107da0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:00.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:00.806+0000 7f42bfe00640 1 -- 172.21.15.77:0/2917053595 >> 172.21.15.77:0/2917053595 conn(0x7f42c00fd740 msgr2=0x7f42c00ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:00.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:00.806+0000 7f42bfe00640 1 -- 172.21.15.77:0/2917053595 shutdown_connections 2024-09-10T09:56:00.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:00.806+0000 7f42bfe00640 1 -- 172.21.15.77:0/2917053595 wait complete. 2024-09-10T09:56:00.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:00.807+0000 7f42bfe00640 1 Processor -- start 2024-09-10T09:56:00.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:00.807+0000 7f42bfe00640 1 -- start start 2024-09-10T09:56:00.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:00.807+0000 7f42bfe00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f42c01019c0 0x7f42c019a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:00.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:00.807+0000 7f42bfe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42c0102360 0x7f42c019aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:00.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:00.807+0000 7f42bfe00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f42c01079c0 0x7f42c019edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:00.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:00.807+0000 7f42bfe00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f42c0111dd0 con 0x7f42c01019c0 2024-09-10T09:56:00.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:00.807+0000 7f42bfe00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f42c0111c50 con 0x7f42c01079c0 2024-09-10T09:56:00.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:00.807+0000 7f42bfe00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f42c0111f50 con 0x7f42c0102360 2024-09-10T09:56:00.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:00.808+0000 7f42bea00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f42c01019c0 0x7f42c019a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:00.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:00.808+0000 7f42be000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42c0102360 0x7f42c019aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:00.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:00.808+0000 7f42bea00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f42c01019c0 0x7f42c019a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56112/0 (socket says 172.21.15.77:56112) 2024-09-10T09:56:00.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:00.808+0000 7f42bea00640 1 -- 172.21.15.77:0/2145216610 learned_addr learned my addr 172.21.15.77:0/2145216610 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:00.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:00.808+0000 7f42be000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42c0102360 0x7f42c019aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:46138/0 (socket says 172.21.15.77:46138) 2024-09-10T09:56:00.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:00.808+0000 7f42bf400640 1 --2- 172.21.15.77:0/2145216610 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f42c01079c0 0x7f42c019edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:00.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:00.808+0000 7f42be000640 1 -- 172.21.15.77:0/2145216610 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f42c01079c0 msgr2=0x7f42c019edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:00.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:00.808+0000 7f42be000640 1 --2- 172.21.15.77:0/2145216610 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f42c01079c0 0x7f42c019edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:00.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:00.808+0000 7f42be000640 1 -- 172.21.15.77:0/2145216610 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f42c01019c0 msgr2=0x7f42c019a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:00.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:00.808+0000 7f42be000640 1 --2- 172.21.15.77:0/2145216610 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f42c01019c0 0x7f42c019a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:00.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:00.808+0000 7f42be000640 1 -- 172.21.15.77:0/2145216610 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42c019f490 con 0x7f42c0102360 2024-09-10T09:56:00.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:00.808+0000 7f42bea00640 1 --2- 172.21.15.77:0/2145216610 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f42c01019c0 0x7f42c019a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:56:00.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:00.809+0000 7f42be000640 1 --2- 172.21.15.77:0/2145216610 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42c0102360 0x7f42c019aa20 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f42ac00e9f0 tx=0x7f42ac00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:00.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:00.809+0000 7f42b7400640 1 -- 172.21.15.77:0/2145216610 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42ac00ccc0 con 0x7f42c0102360 2024-09-10T09:56:00.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:00.809+0000 7f42b7400640 1 -- 172.21.15.77:0/2145216610 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42ac010910 con 0x7f42c0102360 2024-09-10T09:56:00.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:00.809+0000 7f42bfe00640 1 -- 172.21.15.77:0/2145216610 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42c019f780 con 0x7f42c0102360 2024-09-10T09:56:00.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:00.809+0000 7f42b7400640 1 -- 172.21.15.77:0/2145216610 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42ac018960 con 0x7f42c0102360 2024-09-10T09:56:00.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:00.809+0000 7f42bfe00640 1 -- 172.21.15.77:0/2145216610 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42c01a7060 con 0x7f42c0102360 2024-09-10T09:56:00.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:00.810+0000 7f42bfe00640 1 -- 172.21.15.77:0/2145216610 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42c0101e20 con 0x7f42c0102360 2024-09-10T09:56:00.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:00.813+0000 7f42b7400640 1 -- 172.21.15.77:0/2145216610 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f42ac010ac0 con 0x7f42c0102360 2024-09-10T09:56:00.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:00.815+0000 7f42b7400640 1 --2- 172.21.15.77:0/2145216610 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f42a40a2b80 0x7f42a40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:00.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:00.815+0000 7f42bea00640 1 --2- 172.21.15.77:0/2145216610 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f42a40a2b80 0x7f42a40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:00.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:00.816+0000 7f42bea00640 1 --2- 172.21.15.77:0/2145216610 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f42a40a2b80 0x7f42a40a5040 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f42b0000c00 tx=0x7f42b0031510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:00.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:00.816+0000 7f42b7400640 1 -- 172.21.15.77:0/2145216610 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f42ac09a240 con 0x7f42c0102360 2024-09-10T09:56:00.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:00.821+0000 7f42b7400640 1 -- 172.21.15.77:0/2145216610 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f42ac09c1a0 con 0x7f42c0102360 2024-09-10T09:56:00.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:56:00 smithi077 ceph-mon[30319]: pgmap v416: 129 pgs: 129 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-09-10T09:56:01.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:01.073+0000 7f42bfe00640 1 -- 172.21.15.77:0/2145216610 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f42c019b840 con 0x7f42c0102360 2024-09-10T09:56:01.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:01.074+0000 7f42b7400640 1 -- 172.21.15.77:0/2145216610 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11989 (secure 0 0 0) 0x7f42c019b840 con 0x7f42c0102360 2024-09-10T09:56:01.076 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:56:01.076 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":29,"btime":"2024-09-10T09:55:13:863715+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3159289675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3159289675},{"type":"v1","addr":"172.21.15.77:6835","nonce":3159289675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1951962341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1951962341},{"type":"v1","addr":"172.21.15.77:6837","nonce":1951962341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:52:23.368252+0000","modified":"2024-09-10T09:55:13.786975+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24563,"mds_1":24524,"mds_2":24520,"mds_3":24502,"mds_4":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14487":{"gid":14487,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/4291121888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":4291121888},{"type":"v1","addr":"172.21.15.77:6839","nonce":4291121888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14499":{"gid":14499,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6841/445284194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":445284194},{"type":"v1","addr":"172.21.15.77:6841","nonce":445284194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1005054423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1005054423},{"type":"v1","addr":"172.21.15.134:6833","nonce":1005054423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6833/3716307744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":3716307744},{"type":"v1","addr":"172.21.15.120:6833","nonce":3716307744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/1319892544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1319892544},{"type":"v1","addr":"172.21.15.120:6835","nonce":1319892544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.134:6835/3293933699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3293933699},{"type":"v1","addr":"172.21.15.134:6835","nonce":3293933699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.134:6837/2461497091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2461497091},{"type":"v1","addr":"172.21.15.134:6837","nonce":2461497091}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":20,"addr":"172.21.15.134:6839/1965117680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1965117680},{"type":"v1","addr":"172.21.15.134:6839","nonce":1965117680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":20,"addr":"172.21.15.120:6837/3033280709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3033280709},{"type":"v1","addr":"172.21.15.120:6837","nonce":3033280709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.120:6839/1777710157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1777710157},{"type":"v1","addr":"172.21.15.120:6839","nonce":1777710157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24563,"qdb_cluster":[24496,24502,24520,24524,24563]},"id":1}]} 2024-09-10T09:56:01.076 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 29 2024-09-10T09:56:01.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:01.077+0000 7f42bfe00640 1 -- 172.21.15.77:0/2145216610 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f42a40a2b80 msgr2=0x7f42a40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:01.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:01.077+0000 7f42bfe00640 1 --2- 172.21.15.77:0/2145216610 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f42a40a2b80 0x7f42a40a5040 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f42b0000c00 tx=0x7f42b0031510 comp rx=0 tx=0).stop 2024-09-10T09:56:01.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:01.077+0000 7f42bfe00640 1 -- 172.21.15.77:0/2145216610 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42c0102360 msgr2=0x7f42c019aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:01.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:01.077+0000 7f42bfe00640 1 --2- 172.21.15.77:0/2145216610 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42c0102360 0x7f42c019aa20 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f42ac00e9f0 tx=0x7f42ac00eeb0 comp rx=0 tx=0).stop 2024-09-10T09:56:01.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:01.077+0000 7f42bfe00640 1 -- 172.21.15.77:0/2145216610 shutdown_connections 2024-09-10T09:56:01.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:01.077+0000 7f42bfe00640 1 --2- 172.21.15.77:0/2145216610 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f42a40a2b80 0x7f42a40a5040 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:01.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:01.077+0000 7f42bfe00640 1 --2- 172.21.15.77:0/2145216610 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f42c01079c0 0x7f42c019edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:01.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:01.077+0000 7f42bfe00640 1 --2- 172.21.15.77:0/2145216610 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42c0102360 0x7f42c019aa20 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:01.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:01.077+0000 7f42bfe00640 1 --2- 172.21.15.77:0/2145216610 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f42c01019c0 0x7f42c019a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:01.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:01.077+0000 7f42bfe00640 1 -- 172.21.15.77:0/2145216610 >> 172.21.15.77:0/2145216610 conn(0x7f42c00fd740 msgr2=0x7f42c010cc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:01.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:01.077+0000 7f42bfe00640 1 -- 172.21.15.77:0/2145216610 shutdown_connections 2024-09-10T09:56:01.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:01.078+0000 7f42bfe00640 1 -- 172.21.15.77:0/2145216610 wait complete. 2024-09-10T09:56:01.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:56:00 smithi134 ceph-mon[35674]: pgmap v416: 129 pgs: 129 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-09-10T09:56:01.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:56:00 smithi120 ceph-mon[34770]: pgmap v416: 129 pgs: 129 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-09-10T09:56:01.805 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T09:56:02.020 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:56:01 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/2145216610' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:56:02.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:56:01 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/2145216610' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:56:02.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:56:01 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/2145216610' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:56:02.211 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:56:02.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.834+0000 7fb511200640 1 -- 172.21.15.77:0/2843463921 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb50c10e280 msgr2=0x7fb50c110670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:02.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.834+0000 7fb511200640 1 --2- 172.21.15.77:0/2843463921 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb50c10e280 0x7fb50c110670 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb504009ca0 tx=0x7fb50402a950 comp rx=0 tx=0).stop 2024-09-10T09:56:02.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.834+0000 7fb511200640 1 -- 172.21.15.77:0/2843463921 shutdown_connections 2024-09-10T09:56:02.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.834+0000 7fb511200640 1 --2- 172.21.15.77:0/2843463921 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb50c10e280 0x7fb50c110670 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:02.852 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.834+0000 7fb511200640 1 --2- 172.21.15.77:0/2843463921 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb50c074e30 0x7fb50c10dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:02.852 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.834+0000 7fb511200640 1 --2- 172.21.15.77:0/2843463921 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb50c074510 0x7fb50c0748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:02.852 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.834+0000 7fb511200640 1 -- 172.21.15.77:0/2843463921 >> 172.21.15.77:0/2843463921 conn(0x7fb50c0fd740 msgr2=0x7fb50c0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:02.852 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.834+0000 7fb511200640 1 -- 172.21.15.77:0/2843463921 shutdown_connections 2024-09-10T09:56:02.852 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.834+0000 7fb511200640 1 -- 172.21.15.77:0/2843463921 wait complete. 2024-09-10T09:56:02.852 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.835+0000 7fb511200640 1 Processor -- start 2024-09-10T09:56:02.852 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.835+0000 7fb511200640 1 -- start start 2024-09-10T09:56:02.852 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.836+0000 7fb511200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb50c074510 0x7fb50c19a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:02.852 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.836+0000 7fb511200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb50c074e30 0x7fb50c19a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:02.852 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.836+0000 7fb511200640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb50c10e280 0x7fb50c19ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:02.852 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.836+0000 7fb511200640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb50c113fb0 con 0x7fb50c074510 2024-09-10T09:56:02.852 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.836+0000 7fb511200640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb50c113e30 con 0x7fb50c10e280 2024-09-10T09:56:02.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.836+0000 7fb511200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb50c114130 con 0x7fb50c074e30 2024-09-10T09:56:02.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.836+0000 7fb50b400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb50c074510 0x7fb50c19a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:02.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.836+0000 7fb50b400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb50c074510 0x7fb50c19a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33876/0 (socket says 172.21.15.77:33876) 2024-09-10T09:56:02.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.836+0000 7fb50b400640 1 -- 172.21.15.77:0/235994798 learned_addr learned my addr 172.21.15.77:0/235994798 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:02.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.836+0000 7fb50aa00640 1 --2- 172.21.15.77:0/235994798 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb50c074e30 0x7fb50c19a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:02.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.836+0000 7fb50be00640 1 --2- 172.21.15.77:0/235994798 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb50c10e280 0x7fb50c19ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:02.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.837+0000 7fb50aa00640 1 -- 172.21.15.77:0/235994798 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb50c10e280 msgr2=0x7fb50c19ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:02.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.837+0000 7fb50aa00640 1 --2- 172.21.15.77:0/235994798 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb50c10e280 0x7fb50c19ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:02.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.837+0000 7fb50aa00640 1 -- 172.21.15.77:0/235994798 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb50c074510 msgr2=0x7fb50c19a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:02.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.837+0000 7fb50aa00640 1 --2- 172.21.15.77:0/235994798 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb50c074510 0x7fb50c19a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:02.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.837+0000 7fb50aa00640 1 -- 172.21.15.77:0/235994798 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb50c19f440 con 0x7fb50c074e30 2024-09-10T09:56:02.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.837+0000 7fb50b400640 1 --2- 172.21.15.77:0/235994798 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb50c074510 0x7fb50c19a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:56:02.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.837+0000 7fb50aa00640 1 --2- 172.21.15.77:0/235994798 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb50c074e30 0x7fb50c19a9d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb4f800e9f0 tx=0x7fb4f800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:02.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.837+0000 7fb503e00640 1 -- 172.21.15.77:0/235994798 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f800ccc0 con 0x7fb50c074e30 2024-09-10T09:56:02.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.837+0000 7fb511200640 1 -- 172.21.15.77:0/235994798 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb50c19f730 con 0x7fb50c074e30 2024-09-10T09:56:02.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.837+0000 7fb503e00640 1 -- 172.21.15.77:0/235994798 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4f8010910 con 0x7fb50c074e30 2024-09-10T09:56:02.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.837+0000 7fb503e00640 1 -- 172.21.15.77:0/235994798 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f8018960 con 0x7fb50c074e30 2024-09-10T09:56:02.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.838+0000 7fb511200640 1 -- 172.21.15.77:0/235994798 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb50c1a7010 con 0x7fb50c074e30 2024-09-10T09:56:02.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.842+0000 7fb515430740 1 Processor -- start 2024-09-10T09:56:02.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.842+0000 7fb515430740 1 -- start start 2024-09-10T09:56:02.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.842+0000 7fb515430740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559603a47e40 0x559603a48220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:02.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.842+0000 7fb515430740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559603a48760 0x559603a52ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:02.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.842+0000 7fb515430740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559603a53020 0x559603a55410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:02.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.842+0000 7fb515430740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5596039ae3e0 con 0x559603a53020 2024-09-10T09:56:02.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.842+0000 7fb515430740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x559603955d80 con 0x559603a47e40 2024-09-10T09:56:02.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.842+0000 7fb515430740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5596039aa3a0 con 0x559603a48760 2024-09-10T09:56:02.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.843+0000 7fb50b400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559603a48760 0x559603a52ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:02.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.843+0000 7fb503e00640 1 -- 172.21.15.77:0/235994798 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb4f8010ac0 con 0x7fb50c074e30 2024-09-10T09:56:02.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.843+0000 7fb50b400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559603a48760 0x559603a52ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:53392/0 (socket says 172.21.15.77:53392) 2024-09-10T09:56:02.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.843+0000 7fb50be00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559603a47e40 0x559603a48220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:02.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.843+0000 7fb50b400640 1 -- 172.21.15.77:0/2669499727 learned_addr learned my addr 172.21.15.77:0/2669499727 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:02.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.843+0000 7fb50aa00640 1 --2- 172.21.15.77:0/2669499727 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559603a53020 0x559603a55410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:02.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.843+0000 7fb50b400640 1 -- 172.21.15.77:0/2669499727 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559603a47e40 msgr2=0x559603a48220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:02.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.843+0000 7fb50b400640 1 --2- 172.21.15.77:0/2669499727 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559603a47e40 0x559603a48220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:02.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.843+0000 7fb50b400640 1 -- 172.21.15.77:0/2669499727 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559603a53020 msgr2=0x559603a55410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:02.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.843+0000 7fb50b400640 1 --2- 172.21.15.77:0/2669499727 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559603a53020 0x559603a55410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:02.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.843+0000 7fb50b400640 1 -- 172.21.15.77:0/2669499727 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559603974ae0 con 0x559603a48760 2024-09-10T09:56:02.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.843+0000 7fb50aa00640 1 --2- 172.21.15.77:0/2669499727 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559603a53020 0x559603a55410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:56:02.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.844+0000 7fb50b400640 1 --2- 172.21.15.77:0/2669499727 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559603a48760 0x559603a52ae0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc011630 tx=0x7fb4fc011af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8bbd4ddb4391ed2f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:02.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.844+0000 7fb500c00640 1 -- 172.21.15.77:0/2669499727 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc01e070 con 0x559603a48760 2024-09-10T09:56:02.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.844+0000 7fb500c00640 1 -- 172.21.15.77:0/2669499727 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4fc00fbd0 con 0x559603a48760 2024-09-10T09:56:02.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.844+0000 7fb500c00640 1 -- 172.21.15.77:0/2669499727 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc019ca0 con 0x559603a48760 2024-09-10T09:56:02.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.844+0000 7fb503e00640 1 --2- 172.21.15.77:0/235994798 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fb4f00a2b80 0x7fb4f00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:02.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.844+0000 7fb503e00640 1 -- 172.21.15.77:0/235994798 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fb4f80cf750 con 0x7fb50c074e30 2024-09-10T09:56:02.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.845+0000 7fb515430740 1 -- 172.21.15.77:0/2669499727 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559603a48760 msgr2=0x559603a52ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:02.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.845+0000 7fb515430740 1 --2- 172.21.15.77:0/2669499727 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559603a48760 0x559603a52ae0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc011630 tx=0x7fb4fc011af0 comp rx=0 tx=0).stop 2024-09-10T09:56:02.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.845+0000 7fb50be00640 1 --2- 172.21.15.77:0/235994798 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fb4f00a2b80 0x7fb4f00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:02.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.845+0000 7fb515430740 1 -- 172.21.15.77:0/2669499727 shutdown_connections 2024-09-10T09:56:02.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.845+0000 7fb515430740 1 --2- 172.21.15.77:0/2669499727 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559603a53020 0x559603a55410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:02.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.845+0000 7fb515430740 1 --2- 172.21.15.77:0/2669499727 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559603a48760 0x559603a52ae0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:02.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.845+0000 7fb515430740 1 --2- 172.21.15.77:0/2669499727 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559603a47e40 0x559603a48220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:02.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.845+0000 7fb515430740 1 -- 172.21.15.77:0/2669499727 >> 172.21.15.77:0/2669499727 conn(0x559603a443d0 msgr2=0x559603a447b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:02.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.845+0000 7fb515430740 1 -- 172.21.15.77:0/2669499727 shutdown_connections 2024-09-10T09:56:02.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.845+0000 7fb515430740 1 -- 172.21.15.77:0/2669499727 wait complete. 2024-09-10T09:56:02.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.846+0000 7fb515430740 1 -- start start 2024-09-10T09:56:02.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.846+0000 7fb50be00640 1 --2- 172.21.15.77:0/235994798 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fb4f00a2b80 0x7fb4f00a5040 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb504005e30 tx=0x7fb504005c50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:02.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.846+0000 7fb515430740 1 Processor -- start 2024-09-10T09:56:02.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.846+0000 7fb515430740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559603a47e40 0x559603a4ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:02.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.846+0000 7fb515430740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559603a48760 0x559603a50490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:02.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.846+0000 7fb515430740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559603a53020 0x559603a509d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:02.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.846+0000 7fb515430740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x559603a55bf0 con 0x559603a47e40 2024-09-10T09:56:02.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.846+0000 7fb515430740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5596039690f0 con 0x559603a48760 2024-09-10T09:56:02.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.846+0000 7fb515430740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559603a55d70 con 0x559603a53020 2024-09-10T09:56:02.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.846+0000 7fb515430740 10 client.0 _ll_register_callbacks cb 0x559603950bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:56:02.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.847+0000 7fb50aa00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559603a48760 0x559603a50490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:02.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.847+0000 7fb50b400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559603a47e40 0x559603a4ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:02.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.847+0000 7fb50aa00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559603a48760 0x559603a50490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:50872/0 (socket says 172.21.15.77:50872) 2024-09-10T09:56:02.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.847+0000 7fb50aa00640 1 -- 172.21.15.77:0/1639249477 learned_addr learned my addr 172.21.15.77:0/1639249477 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:02.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.847+0000 7fb50aa00640 1 -- 172.21.15.77:0/1639249477 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559603a53020 msgr2=0x559603a509d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:02.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.847+0000 7fb50aa00640 1 --2- 172.21.15.77:0/1639249477 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559603a53020 0x559603a509d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:02.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.847+0000 7fb50aa00640 1 -- 172.21.15.77:0/1639249477 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559603a47e40 msgr2=0x559603a4ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:02.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.847+0000 7fb50aa00640 1 --2- 172.21.15.77:0/1639249477 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559603a47e40 0x559603a4ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:02.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.847+0000 7fb50aa00640 1 -- 172.21.15.77:0/1639249477 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55960371f3c0 con 0x559603a48760 2024-09-10T09:56:02.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.847+0000 7fb50aa00640 1 --2- 172.21.15.77:0/1639249477 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559603a48760 0x559603a50490 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb4f800c6a0 tx=0x7fb4f809b690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6625a0994810683f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:02.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.847+0000 7fb4f5600640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T09:56:02.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.848+0000 7fb4f5600640 1 -- 172.21.15.77:0/1639249477 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f80a4830 con 0x559603a48760 2024-09-10T09:56:02.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.848+0000 7fb515430740 1 -- 172.21.15.77:0/1639249477 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4fc00fbd0 con 0x559603a48760 2024-09-10T09:56:02.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.848+0000 7fb4f5600640 1 -- 172.21.15.77:0/1639249477 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fb4f809b0c0 con 0x559603a48760 2024-09-10T09:56:02.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.848+0000 7fb4f5600640 1 -- 172.21.15.77:0/1639249477 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb4f80b9050 con 0x559603a48760 2024-09-10T09:56:02.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.848+0000 7fb515430740 10 client.25505 fetch_fsmap learned FSMap version 29 2024-09-10T09:56:02.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.848+0000 7fb515430740 1 -- 172.21.15.77:0/1639249477 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559603a61bf0 con 0x559603a48760 2024-09-10T09:56:02.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.848+0000 7fb4f5600640 1 -- 172.21.15.77:0/1639249477 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4fc00fbd0 con 0x559603a48760 2024-09-10T09:56:02.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.848+0000 7fb4f5600640 1 -- 172.21.15.77:0/1639249477 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fb4f807aae0 con 0x559603a48760 2024-09-10T09:56:02.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.848+0000 7fb515430740 10 client.25505 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T09:56:02.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.848+0000 7fb515430740 10 client.25505 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-10T09:56:02.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.849+0000 7fb515430740 20 client.25505 populate_metadata read hostname 'smithi077' 2024-09-10T09:56:02.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.849+0000 7fb515430740 1 --2- 172.21.15.77:0/1639249477 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x559603a63000 0x559603a654c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:02.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.849+0000 7fb515430740 4 client.25505 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:56:02.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.849+0000 7fb515430740 1 -- 172.21.15.77:0/1639249477 --> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5596039bb950 con 0x559603a63000 2024-09-10T09:56:02.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.849+0000 7fb50b400640 1 --2- 172.21.15.77:0/1639249477 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x559603a63000 0x559603a654c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:02.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.850+0000 7fb50b400640 1 --2- 172.21.15.77:0/1639249477 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x559603a63000 0x559603a654c0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=68043392379c5124 server_cookie=35e62bcd5cf38c7d in_seq=0 out_seq=0 2024-09-10T09:56:02.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.850+0000 7fb4f5600640 10 client.25505 ms_handle_connect on v2:172.21.15.134:6834/3293933699 2024-09-10T09:56:02.860 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.850+0000 7fb4f5600640 1 -- 172.21.15.77:0/1639249477 <== mds.4 v2:172.21.15.134:6834/3293933699 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5596039bb950 con 0x559603a63000 2024-09-10T09:56:02.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.850+0000 7fb4f5600640 10 client.25505 handle_command_reply: tid=0 2024-09-10T09:56:02.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.851+0000 7fb515430740 1 client.25505 shutdown 2024-09-10T09:56:02.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.851+0000 7fb515430740 1 -- 172.21.15.77:0/1639249477 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559603a48760 msgr2=0x559603a50490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:02.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.851+0000 7fb515430740 1 --2- 172.21.15.77:0/1639249477 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559603a48760 0x559603a50490 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb4f800c6a0 tx=0x7fb4f809b690 comp rx=0 tx=0).stop 2024-09-10T09:56:02.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.851+0000 7fb515430740 1 -- 172.21.15.77:0/1639249477 shutdown_connections 2024-09-10T09:56:02.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.851+0000 7fb515430740 1 --2- 172.21.15.77:0/1639249477 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x559603a63000 0x559603a654c0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:02.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.851+0000 7fb515430740 1 --2- 172.21.15.77:0/1639249477 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559603a53020 0x559603a509d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:02.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.851+0000 7fb515430740 1 --2- 172.21.15.77:0/1639249477 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559603a48760 0x559603a50490 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:02.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.851+0000 7fb515430740 1 --2- 172.21.15.77:0/1639249477 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559603a47e40 0x559603a4ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:02.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.851+0000 7fb4f5600640 0 client.25505 ms_handle_reset on v2:172.21.15.134:6834/3293933699 2024-09-10T09:56:02.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.851+0000 7fb515430740 1 -- 172.21.15.77:0/1639249477 >> 172.21.15.77:0/1639249477 conn(0x559603a443d0 msgr2=0x559603a4a0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:02.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.851+0000 7fb515430740 1 -- 172.21.15.77:0/1639249477 shutdown_connections 2024-09-10T09:56:02.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.851+0000 7fb515430740 1 -- 172.21.15.77:0/1639249477 wait complete. 2024-09-10T09:56:02.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.851+0000 7fb515430740 20 client.25505 trim_cache size 0 max 16384 2024-09-10T09:56:02.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.870+0000 7fb515430740 1 Processor -- start 2024-09-10T09:56:02.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.870+0000 7fb515430740 1 -- start start 2024-09-10T09:56:02.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.870+0000 7fb515430740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559603a47e40 0x559603a6ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:02.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.870+0000 7fb515430740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559603a48760 0x559603a6b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:02.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.870+0000 7fb515430740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559603a53020 0x559603a49890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:02.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.870+0000 7fb515430740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb50c1b0eb0 con 0x559603a48760 2024-09-10T09:56:02.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.870+0000 7fb515430740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb50c1b29f0 con 0x559603a53020 2024-09-10T09:56:02.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.871+0000 7fb515430740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb50c1b1580 con 0x559603a47e40 2024-09-10T09:56:02.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.871+0000 7fb50aa00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559603a48760 0x559603a6b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:02.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.871+0000 7fb50aa00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559603a48760 0x559603a6b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33906/0 (socket says 172.21.15.77:33906) 2024-09-10T09:56:02.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.871+0000 7fb50aa00640 1 -- 172.21.15.77:0/1205302189 learned_addr learned my addr 172.21.15.77:0/1205302189 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:02.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.871+0000 7fb50b400640 1 --2- 172.21.15.77:0/1205302189 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559603a47e40 0x559603a6ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:02.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.871+0000 7fb50be00640 1 --2- 172.21.15.77:0/1205302189 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559603a53020 0x559603a49890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:02.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.871+0000 7fb50aa00640 1 -- 172.21.15.77:0/1205302189 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559603a47e40 msgr2=0x559603a6ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:02.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.871+0000 7fb50aa00640 1 --2- 172.21.15.77:0/1205302189 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559603a47e40 0x559603a6ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:02.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.871+0000 7fb50aa00640 1 -- 172.21.15.77:0/1205302189 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559603a53020 msgr2=0x559603a49890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:02.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.871+0000 7fb50aa00640 1 --2- 172.21.15.77:0/1205302189 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559603a53020 0x559603a49890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:02.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.871+0000 7fb50aa00640 1 -- 172.21.15.77:0/1205302189 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559603a44200 con 0x559603a48760 2024-09-10T09:56:02.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.871+0000 7fb50be00640 1 --2- 172.21.15.77:0/1205302189 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559603a53020 0x559603a49890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:56:02.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.872+0000 7fb50aa00640 1 --2- 172.21.15.77:0/1205302189 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559603a48760 0x559603a6b2d0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb4f8000c00 tx=0x7fb4f8009700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50bedc0181c1b65b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:02.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.872+0000 7fb4f7e00640 1 -- 172.21.15.77:0/1205302189 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f807aae0 con 0x559603a48760 2024-09-10T09:56:02.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.872+0000 7fb4f7e00640 1 -- 172.21.15.77:0/1205302189 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4f8097b50 con 0x559603a48760 2024-09-10T09:56:02.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.872+0000 7fb4f7e00640 1 -- 172.21.15.77:0/1205302189 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f8018b00 con 0x559603a48760 2024-09-10T09:56:02.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.873+0000 7fb515430740 1 -- 172.21.15.77:0/1205302189 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559603a48760 msgr2=0x559603a6b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:02.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.873+0000 7fb515430740 1 --2- 172.21.15.77:0/1205302189 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559603a48760 0x559603a6b2d0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb4f8000c00 tx=0x7fb4f8009700 comp rx=0 tx=0).stop 2024-09-10T09:56:02.877 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.873+0000 7fb515430740 1 -- 172.21.15.77:0/1205302189 shutdown_connections 2024-09-10T09:56:02.877 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.873+0000 7fb515430740 1 --2- 172.21.15.77:0/1205302189 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559603a53020 0x559603a49890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:02.877 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.873+0000 7fb515430740 1 --2- 172.21.15.77:0/1205302189 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559603a48760 0x559603a6b2d0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:02.877 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.873+0000 7fb515430740 1 --2- 172.21.15.77:0/1205302189 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559603a47e40 0x559603a6ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:02.877 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.873+0000 7fb515430740 1 -- 172.21.15.77:0/1205302189 >> 172.21.15.77:0/1205302189 conn(0x559603a443d0 msgr2=0x559603a4f040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:02.877 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.873+0000 7fb515430740 1 -- 172.21.15.77:0/1205302189 shutdown_connections 2024-09-10T09:56:02.877 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.873+0000 7fb515430740 1 -- 172.21.15.77:0/1205302189 wait complete. 2024-09-10T09:56:02.877 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.874+0000 7fb515430740 1 -- start start 2024-09-10T09:56:02.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.874+0000 7fb515430740 1 Processor -- start 2024-09-10T09:56:02.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.875+0000 7fb515430740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559603a47e40 0x559603a5b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:02.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.875+0000 7fb515430740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559603a48760 0x559603a56940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:02.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.875+0000 7fb515430740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559603a53020 0x559603a56e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:02.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.875+0000 7fb515430740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb50c1b2320 con 0x559603a53020 2024-09-10T09:56:02.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.875+0000 7fb515430740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb50c1b1c50 con 0x559603a48760 2024-09-10T09:56:02.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.875+0000 7fb515430740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb50c1aee90 con 0x559603a47e40 2024-09-10T09:56:02.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.875+0000 7fb515430740 10 client.0 _ll_register_callbacks cb 0x7fb4f800e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:56:02.886 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.875+0000 7fb50b400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559603a47e40 0x559603a5b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:02.886 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.875+0000 7fb50b400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559603a47e40 0x559603a5b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:53416/0 (socket says 172.21.15.77:53416) 2024-09-10T09:56:02.886 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.875+0000 7fb50b400640 1 -- 172.21.15.77:0/2852055653 learned_addr learned my addr 172.21.15.77:0/2852055653 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:02.886 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.875+0000 7fb50aa00640 1 --2- 172.21.15.77:0/2852055653 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559603a48760 0x559603a56940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:02.886 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.875+0000 7fb50be00640 1 --2- 172.21.15.77:0/2852055653 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559603a53020 0x559603a56e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:02.886 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.875+0000 7fb50aa00640 1 -- 172.21.15.77:0/2852055653 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559603a47e40 msgr2=0x559603a5b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:02.886 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.875+0000 7fb50aa00640 1 --2- 172.21.15.77:0/2852055653 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559603a47e40 0x559603a5b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:02.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.875+0000 7fb50aa00640 1 -- 172.21.15.77:0/2852055653 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559603a53020 msgr2=0x559603a56e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:02.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.875+0000 7fb50aa00640 1 --2- 172.21.15.77:0/2852055653 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559603a53020 0x559603a56e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:02.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.875+0000 7fb50aa00640 1 -- 172.21.15.77:0/2852055653 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559603a576b0 con 0x559603a48760 2024-09-10T09:56:02.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.876+0000 7fb50aa00640 1 --2- 172.21.15.77:0/2852055653 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559603a48760 0x559603a56940 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb4f8098640 tx=0x7fb4f80ccee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b839f0cce6fb4c6b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:02.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.876+0000 7fb4f5600640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T09:56:02.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.876+0000 7fb4f5600640 1 -- 172.21.15.77:0/2852055653 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f807f060 con 0x559603a48760 2024-09-10T09:56:02.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.876+0000 7fb4f5600640 1 -- 172.21.15.77:0/2852055653 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fb4f8010630 con 0x559603a48760 2024-09-10T09:56:02.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.876+0000 7fb515430740 1 -- 172.21.15.77:0/2852055653 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4f8097b50 con 0x559603a48760 2024-09-10T09:56:02.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.876+0000 7fb4f5600640 1 -- 172.21.15.77:0/2852055653 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb4f8097d00 con 0x559603a48760 2024-09-10T09:56:02.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.876+0000 7fb515430740 10 client.25511 fetch_fsmap learned FSMap version 29 2024-09-10T09:56:02.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.876+0000 7fb515430740 1 -- 172.21.15.77:0/2852055653 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559603acdc60 con 0x559603a48760 2024-09-10T09:56:02.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.877+0000 7fb4f5600640 1 -- 172.21.15.77:0/2852055653 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4f8097b50 con 0x559603a48760 2024-09-10T09:56:02.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.877+0000 7fb4f5600640 1 -- 172.21.15.77:0/2852055653 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fb4f8082080 con 0x559603a48760 2024-09-10T09:56:02.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.877+0000 7fb515430740 10 client.25511 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T09:56:02.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.877+0000 7fb515430740 10 client.25511 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-10T09:56:02.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.877+0000 7fb515430740 20 client.25511 populate_metadata read hostname 'smithi077' 2024-09-10T09:56:02.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.877+0000 7fb515430740 1 --2- 172.21.15.77:0/2852055653 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x559603ace180 0x559603ad0570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:02.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.877+0000 7fb515430740 4 client.25511 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:56:02.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.877+0000 7fb515430740 1 -- 172.21.15.77:0/2852055653 --> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5596039b5e60 con 0x559603ace180 2024-09-10T09:56:02.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.877+0000 7fb50b400640 1 --2- 172.21.15.77:0/2852055653 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x559603ace180 0x559603ad0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:02.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.878+0000 7fb4f5600640 10 client.25511 ms_handle_connect on v2:172.21.15.134:6834/3293933699 2024-09-10T09:56:02.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.878+0000 7fb50b400640 1 --2- 172.21.15.77:0/2852055653 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x559603ace180 0x559603ad0570 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=de2fb460a31b3b70 server_cookie=fee158b97d8e69b3 in_seq=0 out_seq=0 2024-09-10T09:56:02.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.878+0000 7fb4f5600640 1 -- 172.21.15.77:0/2852055653 <== mds.4 v2:172.21.15.134:6834/3293933699 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5596039b5e60 con 0x559603ace180 2024-09-10T09:56:02.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.878+0000 7fb4f5600640 10 client.25511 handle_command_reply: tid=0 2024-09-10T09:56:02.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.878+0000 7fb515430740 1 client.25511 shutdown 2024-09-10T09:56:02.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.878+0000 7fb515430740 1 -- 172.21.15.77:0/2852055653 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559603a48760 msgr2=0x559603a56940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:02.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.879+0000 7fb515430740 1 --2- 172.21.15.77:0/2852055653 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559603a48760 0x559603a56940 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb4f8098640 tx=0x7fb4f80ccee0 comp rx=0 tx=0).stop 2024-09-10T09:56:02.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.879+0000 7fb515430740 1 -- 172.21.15.77:0/2852055653 shutdown_connections 2024-09-10T09:56:02.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.879+0000 7fb515430740 1 --2- 172.21.15.77:0/2852055653 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x559603ace180 0x559603ad0570 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:02.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.879+0000 7fb515430740 1 --2- 172.21.15.77:0/2852055653 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559603a53020 0x559603a56e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:02.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.879+0000 7fb515430740 1 --2- 172.21.15.77:0/2852055653 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559603a48760 0x559603a56940 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:02.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.879+0000 7fb515430740 1 --2- 172.21.15.77:0/2852055653 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559603a47e40 0x559603a5b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:02.892 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.879+0000 7fb515430740 1 -- 172.21.15.77:0/2852055653 >> 172.21.15.77:0/2852055653 conn(0x559603a443d0 msgr2=0x559603a517b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:02.892 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.879+0000 7fb4f5600640 0 client.25511 ms_handle_reset on v2:172.21.15.134:6834/3293933699 2024-09-10T09:56:02.892 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.879+0000 7fb515430740 1 -- 172.21.15.77:0/2852055653 shutdown_connections 2024-09-10T09:56:02.892 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.879+0000 7fb515430740 1 -- 172.21.15.77:0/2852055653 wait complete. 2024-09-10T09:56:02.892 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.879+0000 7fb515430740 20 client.25511 trim_cache size 0 max 16384 2024-09-10T09:56:02.893 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-10T09:56:02.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.880+0000 7fb511200640 1 -- 172.21.15.77:0/235994798 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fb4f00a2b80 msgr2=0x7fb4f00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:02.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.880+0000 7fb511200640 1 --2- 172.21.15.77:0/235994798 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fb4f00a2b80 0x7fb4f00a5040 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb504005e30 tx=0x7fb504005c50 comp rx=0 tx=0).stop 2024-09-10T09:56:02.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.880+0000 7fb511200640 1 -- 172.21.15.77:0/235994798 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb50c074e30 msgr2=0x7fb50c19a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:02.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.880+0000 7fb511200640 1 --2- 172.21.15.77:0/235994798 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb50c074e30 0x7fb50c19a9d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb4f800e9f0 tx=0x7fb4f800eeb0 comp rx=0 tx=0).stop 2024-09-10T09:56:02.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.880+0000 7fb511200640 1 -- 172.21.15.77:0/235994798 shutdown_connections 2024-09-10T09:56:02.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.880+0000 7fb511200640 1 --2- 172.21.15.77:0/235994798 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fb4f00a2b80 0x7fb4f00a5040 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:02.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.880+0000 7fb511200640 1 --2- 172.21.15.77:0/235994798 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb50c10e280 0x7fb50c19ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:02.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.880+0000 7fb511200640 1 --2- 172.21.15.77:0/235994798 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb50c074e30 0x7fb50c19a9d0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:02.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.880+0000 7fb511200640 1 --2- 172.21.15.77:0/235994798 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb50c074510 0x7fb50c19a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:02.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.880+0000 7fb511200640 1 -- 172.21.15.77:0/235994798 >> 172.21.15.77:0/235994798 conn(0x7fb50c0fd740 msgr2=0x7fb50c0fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:02.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.880+0000 7fb511200640 1 -- 172.21.15.77:0/235994798 shutdown_connections 2024-09-10T09:56:02.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:02.880+0000 7fb511200640 1 -- 172.21.15.77:0/235994798 wait complete. 2024-09-10T09:56:03.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:56:02 smithi134 ceph-mon[35674]: pgmap v417: 129 pgs: 129 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-10T09:56:03.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:56:02 smithi120 ceph-mon[34770]: pgmap v417: 129 pgs: 129 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-10T09:56:03.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:56:02 smithi077 ceph-mon[30319]: pgmap v417: 129 pgs: 129 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-10T09:56:03.983 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T09:56:04.373 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:56:04.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.954+0000 7ff311a00640 1 -- 172.21.15.77:0/3995797973 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff30c10e350 msgr2=0x7ff30c110740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:04.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.954+0000 7ff311a00640 1 --2- 172.21.15.77:0/3995797973 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff30c10e350 0x7ff30c110740 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ff304009ca0 tx=0x7ff30402a8c0 comp rx=0 tx=0).stop 2024-09-10T09:56:04.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.954+0000 7ff311a00640 1 -- 172.21.15.77:0/3995797973 shutdown_connections 2024-09-10T09:56:04.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.954+0000 7ff311a00640 1 --2- 172.21.15.77:0/3995797973 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff30c10e350 0x7ff30c110740 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:04.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.954+0000 7ff311a00640 1 --2- 172.21.15.77:0/3995797973 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff30c101150 0x7ff30c10de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:04.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.954+0000 7ff311a00640 1 --2- 172.21.15.77:0/3995797973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff30c1007a0 0x7ff30c100b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:04.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.954+0000 7ff311a00640 1 -- 172.21.15.77:0/3995797973 >> 172.21.15.77:0/3995797973 conn(0x7ff30c0fc610 msgr2=0x7ff30c0fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:04.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.954+0000 7ff311a00640 1 -- 172.21.15.77:0/3995797973 shutdown_connections 2024-09-10T09:56:04.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.954+0000 7ff311a00640 1 -- 172.21.15.77:0/3995797973 wait complete. 2024-09-10T09:56:04.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.955+0000 7ff311a00640 1 Processor -- start 2024-09-10T09:56:04.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.955+0000 7ff311a00640 1 -- start start 2024-09-10T09:56:04.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.956+0000 7ff311a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff30c1007a0 0x7ff30c19a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:04.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.956+0000 7ff311a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff30c101150 0x7ff30c19a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:04.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.956+0000 7ff311a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff30c10e350 0x7ff30c19ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:04.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.956+0000 7ff311a00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff30c113fd0 con 0x7ff30c101150 2024-09-10T09:56:04.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.956+0000 7ff311a00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff30c113e50 con 0x7ff30c1007a0 2024-09-10T09:56:04.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.956+0000 7ff311a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff30c114150 con 0x7ff30c10e350 2024-09-10T09:56:04.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.956+0000 7ff30be00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff30c1007a0 0x7ff30c19a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:04.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.956+0000 7ff30be00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff30c1007a0 0x7ff30c19a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:50922/0 (socket says 172.21.15.77:50922) 2024-09-10T09:56:04.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.956+0000 7ff30be00640 1 -- 172.21.15.77:0/150791239 learned_addr learned my addr 172.21.15.77:0/150791239 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:04.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.956+0000 7ff303400640 1 --2- 172.21.15.77:0/150791239 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff30c101150 0x7ff30c19a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:04.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.956+0000 7ff311000640 1 --2- 172.21.15.77:0/150791239 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff30c10e350 0x7ff30c19ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:04.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.957+0000 7ff30be00640 1 -- 172.21.15.77:0/150791239 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff30c10e350 msgr2=0x7ff30c19ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:04.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.957+0000 7ff30be00640 1 --2- 172.21.15.77:0/150791239 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff30c10e350 0x7ff30c19ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:04.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.957+0000 7ff30be00640 1 -- 172.21.15.77:0/150791239 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff30c101150 msgr2=0x7ff30c19a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:04.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.957+0000 7ff30be00640 1 --2- 172.21.15.77:0/150791239 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff30c101150 0x7ff30c19a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:04.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.957+0000 7ff30be00640 1 -- 172.21.15.77:0/150791239 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff30c19f440 con 0x7ff30c1007a0 2024-09-10T09:56:04.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.957+0000 7ff303400640 1 --2- 172.21.15.77:0/150791239 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff30c101150 0x7ff30c19a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:56:04.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.957+0000 7ff30be00640 1 --2- 172.21.15.77:0/150791239 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff30c1007a0 0x7ff30c19a490 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc00e9f0 tx=0x7ff2fc00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:04.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.957+0000 7ff309600640 1 -- 172.21.15.77:0/150791239 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2fc00cd80 con 0x7ff30c1007a0 2024-09-10T09:56:04.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.957+0000 7ff309600640 1 -- 172.21.15.77:0/150791239 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2fc005500 con 0x7ff30c1007a0 2024-09-10T09:56:04.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.957+0000 7ff311a00640 1 -- 172.21.15.77:0/150791239 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff30c19f730 con 0x7ff30c1007a0 2024-09-10T09:56:04.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.958+0000 7ff309600640 1 -- 172.21.15.77:0/150791239 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2fc010690 con 0x7ff30c1007a0 2024-09-10T09:56:04.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.958+0000 7ff311a00640 1 -- 172.21.15.77:0/150791239 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff30c1a7010 con 0x7ff30c1007a0 2024-09-10T09:56:04.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.961+0000 7ff309600640 1 -- 172.21.15.77:0/150791239 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff2fc0046c0 con 0x7ff30c1007a0 2024-09-10T09:56:04.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.961+0000 7ff315bcd740 1 Processor -- start 2024-09-10T09:56:04.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.962+0000 7ff315bcd740 1 -- start start 2024-09-10T09:56:04.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.962+0000 7ff309600640 1 --2- 172.21.15.77:0/150791239 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ff2f00a2c00 0x7ff2f00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:04.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.962+0000 7ff315bcd740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fd4ab2e40 0x560fd4ab3220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:04.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.962+0000 7ff315bcd740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560fd4ab3760 0x560fd4abdae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:04.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.962+0000 7ff315bcd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fd4abe020 0x560fd4ac0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:04.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.962+0000 7ff315bcd740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560fd4a193e0 con 0x560fd4ab2e40 2024-09-10T09:56:04.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.962+0000 7ff315bcd740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x560fd49c0d80 con 0x560fd4ab3760 2024-09-10T09:56:04.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.962+0000 7ff315bcd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560fd4a153a0 con 0x560fd4abe020 2024-09-10T09:56:04.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.962+0000 7ff309600640 1 -- 172.21.15.77:0/150791239 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7ff2fc0cf790 con 0x7ff30c1007a0 2024-09-10T09:56:04.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.962+0000 7ff311000640 1 --2- 172.21.15.77:0/150791239 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ff2f00a2c00 0x7ff2f00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:04.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.962+0000 7ff303400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560fd4ab3760 0x560fd4abdae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:04.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.963+0000 7ff303400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560fd4ab3760 0x560fd4abdae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:50938/0 (socket says 172.21.15.77:50938) 2024-09-10T09:56:04.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.963+0000 7ff303400640 1 -- 172.21.15.77:0/2424935730 learned_addr learned my addr 172.21.15.77:0/2424935730 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:04.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.963+0000 7ff311000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fd4abe020 0x560fd4ac0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:04.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.963+0000 7ff30be00640 1 --2- 172.21.15.77:0/2424935730 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fd4ab2e40 0x560fd4ab3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:04.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.963+0000 7ff303400640 1 -- 172.21.15.77:0/2424935730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fd4abe020 msgr2=0x560fd4ac0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:04.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.963+0000 7ff303400640 1 --2- 172.21.15.77:0/2424935730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fd4abe020 0x560fd4ac0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:04.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.963+0000 7ff303400640 1 -- 172.21.15.77:0/2424935730 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fd4ab2e40 msgr2=0x560fd4ab3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:04.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.963+0000 7ff303400640 1 --2- 172.21.15.77:0/2424935730 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fd4ab2e40 0x560fd4ab3220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:04.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.963+0000 7ff303400640 1 -- 172.21.15.77:0/2424935730 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fd49dfae0 con 0x560fd4ab3760 2024-09-10T09:56:04.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.963+0000 7ff311000640 1 --2- 172.21.15.77:0/2424935730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fd4abe020 0x560fd4ac0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:56:04.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.964+0000 7ff303400640 1 --2- 172.21.15.77:0/2424935730 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560fd4ab3760 0x560fd4abdae0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4011470 tx=0x7ff2f4011930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10cd6a2e78b6cfd4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:04.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.964+0000 7ff311000640 1 --2- 172.21.15.77:0/150791239 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ff2f00a2c00 0x7ff2f00a50c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff304006010 tx=0x7ff30402a8c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:04.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.964+0000 7ff301600640 1 -- 172.21.15.77:0/2424935730 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f401e070 con 0x560fd4ab3760 2024-09-10T09:56:04.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.964+0000 7ff301600640 1 -- 172.21.15.77:0/2424935730 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2f40035f0 con 0x560fd4ab3760 2024-09-10T09:56:04.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.964+0000 7ff301600640 1 -- 172.21.15.77:0/2424935730 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f400f830 con 0x560fd4ab3760 2024-09-10T09:56:04.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.965+0000 7ff315bcd740 1 -- 172.21.15.77:0/2424935730 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560fd4ab3760 msgr2=0x560fd4abdae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:04.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.965+0000 7ff315bcd740 1 --2- 172.21.15.77:0/2424935730 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560fd4ab3760 0x560fd4abdae0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4011470 tx=0x7ff2f4011930 comp rx=0 tx=0).stop 2024-09-10T09:56:04.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.965+0000 7ff315bcd740 1 -- 172.21.15.77:0/2424935730 shutdown_connections 2024-09-10T09:56:04.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.965+0000 7ff315bcd740 1 --2- 172.21.15.77:0/2424935730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fd4abe020 0x560fd4ac0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:04.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.965+0000 7ff315bcd740 1 --2- 172.21.15.77:0/2424935730 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560fd4ab3760 0x560fd4abdae0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:04.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.965+0000 7ff315bcd740 1 --2- 172.21.15.77:0/2424935730 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fd4ab2e40 0x560fd4ab3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:04.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.965+0000 7ff315bcd740 1 -- 172.21.15.77:0/2424935730 >> 172.21.15.77:0/2424935730 conn(0x560fd4aaf3d0 msgr2=0x560fd4aaf7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:04.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.965+0000 7ff315bcd740 1 -- 172.21.15.77:0/2424935730 shutdown_connections 2024-09-10T09:56:04.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.965+0000 7ff315bcd740 1 -- 172.21.15.77:0/2424935730 wait complete. 2024-09-10T09:56:04.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.966+0000 7ff315bcd740 1 -- start start 2024-09-10T09:56:04.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.966+0000 7ff315bcd740 1 Processor -- start 2024-09-10T09:56:04.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.966+0000 7ff315bcd740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560fd4ab2e40 0x560fd4abaf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:04.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.966+0000 7ff315bcd740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fd4ab3760 0x560fd4abb470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:04.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.966+0000 7ff315bcd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fd4abe020 0x560fd4abb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:04.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.966+0000 7ff315bcd740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560fd4ac0bf0 con 0x560fd4ab3760 2024-09-10T09:56:04.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.966+0000 7ff315bcd740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x560fd49d40f0 con 0x560fd4ab2e40 2024-09-10T09:56:04.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.966+0000 7ff315bcd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560fd4ac0d70 con 0x560fd4abe020 2024-09-10T09:56:04.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.966+0000 7ff315bcd740 10 client.0 _ll_register_callbacks cb 0x560fd49bbbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:56:04.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.967+0000 7ff303400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fd4ab3760 0x560fd4abb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:04.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.967+0000 7ff303400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fd4ab3760 0x560fd4abb470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33938/0 (socket says 172.21.15.77:33938) 2024-09-10T09:56:04.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.967+0000 7ff303400640 1 -- 172.21.15.77:0/3176585627 learned_addr learned my addr 172.21.15.77:0/3176585627 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:04.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.967+0000 7ff30be00640 1 --2- 172.21.15.77:0/3176585627 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560fd4ab2e40 0x560fd4abaf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:04.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.967+0000 7ff311000640 1 --2- 172.21.15.77:0/3176585627 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fd4abe020 0x560fd4abb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:04.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.967+0000 7ff303400640 1 -- 172.21.15.77:0/3176585627 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fd4abe020 msgr2=0x560fd4abb9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:04.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.967+0000 7ff303400640 1 --2- 172.21.15.77:0/3176585627 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fd4abe020 0x560fd4abb9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:04.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.967+0000 7ff303400640 1 -- 172.21.15.77:0/3176585627 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560fd4ab2e40 msgr2=0x560fd4abaf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:04.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.967+0000 7ff303400640 1 --2- 172.21.15.77:0/3176585627 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560fd4ab2e40 0x560fd4abaf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:04.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.967+0000 7ff303400640 1 -- 172.21.15.77:0/3176585627 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fd478a3c0 con 0x560fd4ab3760 2024-09-10T09:56:04.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.968+0000 7ff303400640 1 --2- 172.21.15.77:0/3176585627 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fd4ab3760 0x560fd4abb470 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7ff2f40096e0 tx=0x7ff2f4005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35fe15c820985e7e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:04.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.968+0000 7ff2fa000640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-10T09:56:04.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.968+0000 7ff2fa000640 1 -- 172.21.15.77:0/3176585627 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f401e040 con 0x560fd4ab3760 2024-09-10T09:56:04.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.968+0000 7ff2fa000640 1 -- 172.21.15.77:0/3176585627 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7ff2f4005460 con 0x560fd4ab3760 2024-09-10T09:56:04.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.968+0000 7ff315bcd740 1 -- 172.21.15.77:0/3176585627 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2f40035f0 con 0x560fd4ab3760 2024-09-10T09:56:04.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.968+0000 7ff2fa000640 1 -- 172.21.15.77:0/3176585627 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff2f4030050 con 0x560fd4ab3760 2024-09-10T09:56:04.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.968+0000 7ff315bcd740 10 client.15474 fetch_fsmap learned FSMap version 29 2024-09-10T09:56:04.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.968+0000 7ff315bcd740 1 -- 172.21.15.77:0/3176585627 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560fd4accc60 con 0x560fd4ab3760 2024-09-10T09:56:04.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.969+0000 7ff2fa000640 1 -- 172.21.15.77:0/3176585627 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2f40035f0 con 0x560fd4ab3760 2024-09-10T09:56:04.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.969+0000 7ff2fa000640 1 -- 172.21.15.77:0/3176585627 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7ff2f4021070 con 0x560fd4ab3760 2024-09-10T09:56:04.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.969+0000 7ff315bcd740 10 client.15474 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T09:56:04.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.969+0000 7ff315bcd740 10 client.15474 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T09:56:04.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.969+0000 7ff315bcd740 20 client.15474 populate_metadata read hostname 'smithi077' 2024-09-10T09:56:04.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.969+0000 7ff315bcd740 1 --2- 172.21.15.77:0/3176585627 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x560fd4ace010 0x560fd4ad0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:04.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.969+0000 7ff315bcd740 4 client.15474 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:56:04.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.969+0000 7ff315bcd740 1 -- 172.21.15.77:0/3176585627 --> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560fd4a26950 con 0x560fd4ace010 2024-09-10T09:56:04.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.970+0000 7ff30be00640 1 --2- 172.21.15.77:0/3176585627 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x560fd4ace010 0x560fd4ad0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:04.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.970+0000 7ff30be00640 1 --2- 172.21.15.77:0/3176585627 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x560fd4ace010 0x560fd4ad0400 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f641890a3815cb45 server_cookie=b7a033f90eea3a85 in_seq=0 out_seq=0 2024-09-10T09:56:04.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.970+0000 7ff2fa000640 10 client.15474 ms_handle_connect on v2:172.21.15.134:6836/2461497091 2024-09-10T09:56:04.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.971+0000 7ff2fa000640 1 -- 172.21.15.77:0/3176585627 <== mds.3 v2:172.21.15.134:6836/2461497091 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560fd4a26950 con 0x560fd4ace010 2024-09-10T09:56:04.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.971+0000 7ff2fa000640 10 client.15474 handle_command_reply: tid=0 2024-09-10T09:56:04.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.971+0000 7ff315bcd740 1 client.15474 shutdown 2024-09-10T09:56:04.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.971+0000 7ff315bcd740 1 -- 172.21.15.77:0/3176585627 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fd4ab3760 msgr2=0x560fd4abb470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:04.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.971+0000 7ff315bcd740 1 --2- 172.21.15.77:0/3176585627 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fd4ab3760 0x560fd4abb470 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7ff2f40096e0 tx=0x7ff2f4005270 comp rx=0 tx=0).stop 2024-09-10T09:56:04.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.971+0000 7ff315bcd740 1 -- 172.21.15.77:0/3176585627 shutdown_connections 2024-09-10T09:56:04.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.971+0000 7ff315bcd740 1 --2- 172.21.15.77:0/3176585627 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fd4abe020 0x560fd4abb9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:04.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.971+0000 7ff315bcd740 1 --2- 172.21.15.77:0/3176585627 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x560fd4ace010 0x560fd4ad0400 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:04.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.971+0000 7ff315bcd740 1 --2- 172.21.15.77:0/3176585627 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fd4ab3760 0x560fd4abb470 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:04.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.971+0000 7ff315bcd740 1 --2- 172.21.15.77:0/3176585627 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560fd4ab2e40 0x560fd4abaf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:04.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.971+0000 7ff315bcd740 1 -- 172.21.15.77:0/3176585627 >> 172.21.15.77:0/3176585627 conn(0x560fd4aaf3d0 msgr2=0x560fd4ab5100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:04.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.971+0000 7ff2fa000640 0 client.15474 ms_handle_reset on v2:172.21.15.134:6836/2461497091 2024-09-10T09:56:04.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.972+0000 7ff315bcd740 1 -- 172.21.15.77:0/3176585627 shutdown_connections 2024-09-10T09:56:04.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.972+0000 7ff315bcd740 1 -- 172.21.15.77:0/3176585627 wait complete. 2024-09-10T09:56:04.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.972+0000 7ff315bcd740 20 client.15474 trim_cache size 0 max 16384 2024-09-10T09:56:04.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.996+0000 7ff315bcd740 1 Processor -- start 2024-09-10T09:56:04.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.996+0000 7ff315bcd740 1 -- start start 2024-09-10T09:56:04.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.997+0000 7ff315bcd740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560fd4ab2e40 0x560fd4ad5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:04.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.997+0000 7ff315bcd740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fd4ab3760 0x560fd4ad60b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:04.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.997+0000 7ff315bcd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fd4abe020 0x560fd4abaef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:04.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.997+0000 7ff315bcd740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff30c1b0ed0 con 0x560fd4ab3760 2024-09-10T09:56:04.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.997+0000 7ff315bcd740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff30c1b2a10 con 0x560fd4ab2e40 2024-09-10T09:56:04.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.997+0000 7ff315bcd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff30c1b15a0 con 0x560fd4abe020 2024-09-10T09:56:04.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.997+0000 7ff303400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fd4ab3760 0x560fd4ad60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:04.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.997+0000 7ff30be00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560fd4ab2e40 0x560fd4ad5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:04.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.997+0000 7ff303400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fd4ab3760 0x560fd4ad60b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33944/0 (socket says 172.21.15.77:33944) 2024-09-10T09:56:04.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.997+0000 7ff303400640 1 -- 172.21.15.77:0/1950956386 learned_addr learned my addr 172.21.15.77:0/1950956386 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:04.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.997+0000 7ff30be00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560fd4ab2e40 0x560fd4ad5b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:50962/0 (socket says 172.21.15.77:50962) 2024-09-10T09:56:04.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.997+0000 7ff311000640 1 --2- 172.21.15.77:0/1950956386 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fd4abe020 0x560fd4abaef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:05.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.998+0000 7ff303400640 1 -- 172.21.15.77:0/1950956386 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fd4abe020 msgr2=0x560fd4abaef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:05.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.998+0000 7ff303400640 1 --2- 172.21.15.77:0/1950956386 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fd4abe020 0x560fd4abaef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:05.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.998+0000 7ff303400640 1 -- 172.21.15.77:0/1950956386 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560fd4ab2e40 msgr2=0x560fd4ad5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:05.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.998+0000 7ff303400640 1 --2- 172.21.15.77:0/1950956386 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560fd4ab2e40 0x560fd4ad5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:05.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.998+0000 7ff303400640 1 -- 172.21.15.77:0/1950956386 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fd4aaec80 con 0x560fd4ab3760 2024-09-10T09:56:05.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.998+0000 7ff30be00640 1 --2- 172.21.15.77:0/1950956386 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560fd4ab2e40 0x560fd4ad5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:56:05.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.998+0000 7ff303400640 1 --2- 172.21.15.77:0/1950956386 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fd4ab3760 0x560fd4ad60b0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4004af0 tx=0x7ff2f40044d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a85201d61ea7c3b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:05.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.998+0000 7ff300c00640 1 -- 172.21.15.77:0/1950956386 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f4016030 con 0x560fd4ab3760 2024-09-10T09:56:05.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.998+0000 7ff300c00640 1 -- 172.21.15.77:0/1950956386 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2f4027070 con 0x560fd4ab3760 2024-09-10T09:56:05.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.998+0000 7ff300c00640 1 -- 172.21.15.77:0/1950956386 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f401e040 con 0x560fd4ab3760 2024-09-10T09:56:05.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.999+0000 7ff315bcd740 1 -- 172.21.15.77:0/1950956386 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fd4ab3760 msgr2=0x560fd4ad60b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:05.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.999+0000 7ff315bcd740 1 --2- 172.21.15.77:0/1950956386 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fd4ab3760 0x560fd4ad60b0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4004af0 tx=0x7ff2f40044d0 comp rx=0 tx=0).stop 2024-09-10T09:56:05.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.999+0000 7ff315bcd740 1 -- 172.21.15.77:0/1950956386 shutdown_connections 2024-09-10T09:56:05.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.999+0000 7ff315bcd740 1 --2- 172.21.15.77:0/1950956386 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fd4abe020 0x560fd4abaef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:05.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.999+0000 7ff315bcd740 1 --2- 172.21.15.77:0/1950956386 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fd4ab3760 0x560fd4ad60b0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:05.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.999+0000 7ff315bcd740 1 --2- 172.21.15.77:0/1950956386 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560fd4ab2e40 0x560fd4ad5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:05.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.999+0000 7ff315bcd740 1 -- 172.21.15.77:0/1950956386 >> 172.21.15.77:0/1950956386 conn(0x560fd4aaf3d0 msgr2=0x560fd4ab6200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:05.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.999+0000 7ff315bcd740 1 -- 172.21.15.77:0/1950956386 shutdown_connections 2024-09-10T09:56:05.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:04.999+0000 7ff315bcd740 1 -- 172.21.15.77:0/1950956386 wait complete. 2024-09-10T09:56:05.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:05.000+0000 7ff315bcd740 1 -- start start 2024-09-10T09:56:05.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:05.000+0000 7ff315bcd740 1 Processor -- start 2024-09-10T09:56:05.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:05.000+0000 7ff315bcd740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560fd4ab2e40 0x560fd4ac0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:05.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:05.001+0000 7ff315bcd740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fd4ab3760 0x560fd4ac14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:05.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:05.001+0000 7ff315bcd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fd4abe020 0x560fd4ac1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:05.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:05.001+0000 7ff315bcd740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff30c1b2340 con 0x560fd4ab3760 2024-09-10T09:56:05.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:05.001+0000 7ff315bcd740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff30c1b1c70 con 0x560fd4ab2e40 2024-09-10T09:56:05.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:05.001+0000 7ff315bcd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff30c1aeeb0 con 0x560fd4abe020 2024-09-10T09:56:05.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:05.001+0000 7ff315bcd740 10 client.0 _ll_register_callbacks cb 0x7ff2f401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:56:05.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:05.001+0000 7ff30be00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560fd4ab2e40 0x560fd4ac0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:05.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:05.001+0000 7ff30be00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560fd4ab2e40 0x560fd4ac0f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:50970/0 (socket says 172.21.15.77:50970) 2024-09-10T09:56:05.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:05.001+0000 7ff30be00640 1 -- 172.21.15.77:0/2358700870 learned_addr learned my addr 172.21.15.77:0/2358700870 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:05.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:05.001+0000 7ff303400640 1 --2- 172.21.15.77:0/2358700870 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fd4ab3760 0x560fd4ac14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:05.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:05.001+0000 7ff311000640 1 --2- 172.21.15.77:0/2358700870 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fd4abe020 0x560fd4ac1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:05.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:05.001+0000 7ff30be00640 1 -- 172.21.15.77:0/2358700870 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fd4abe020 msgr2=0x560fd4ac1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:05.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:05.001+0000 7ff30be00640 1 --2- 172.21.15.77:0/2358700870 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fd4abe020 0x560fd4ac1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:05.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:05.001+0000 7ff30be00640 1 -- 172.21.15.77:0/2358700870 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fd4ab3760 msgr2=0x560fd4ac14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:05.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:05.001+0000 7ff30be00640 1 --2- 172.21.15.77:0/2358700870 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fd4ab3760 0x560fd4ac14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:05.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:05.001+0000 7ff30be00640 1 -- 172.21.15.77:0/2358700870 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fd4b36d80 con 0x560fd4ab2e40 2024-09-10T09:56:05.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:05.001+0000 7ff303400640 1 --2- 172.21.15.77:0/2358700870 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fd4ab3760 0x560fd4ac14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:56:05.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:05.002+0000 7ff30be00640 1 --2- 172.21.15.77:0/2358700870 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560fd4ab2e40 0x560fd4ac0f80 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc098000 tx=0x7ff2fc009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1cc8b29e856b1c7e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:05.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:05.002+0000 7ff2fa000640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T09:56:05.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:05.002+0000 7ff2fa000640 1 -- 172.21.15.77:0/2358700870 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2fc07f060 con 0x560fd4ab2e40 2024-09-10T09:56:05.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:05.002+0000 7ff2fa000640 1 -- 172.21.15.77:0/2358700870 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7ff2fc098210 con 0x560fd4ab2e40 2024-09-10T09:56:05.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:05.002+0000 7ff315bcd740 1 -- 172.21.15.77:0/2358700870 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2f4027070 con 0x560fd4ab2e40 2024-09-10T09:56:05.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:05.002+0000 7ff2fa000640 1 -- 172.21.15.77:0/2358700870 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff2fc09ff50 con 0x560fd4ab2e40 2024-09-10T09:56:05.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:05.002+0000 7ff315bcd740 10 client.25529 fetch_fsmap learned FSMap version 29 2024-09-10T09:56:05.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:05.002+0000 7ff315bcd740 1 -- 172.21.15.77:0/2358700870 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560fd4b380a0 con 0x560fd4ab2e40 2024-09-10T09:56:05.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:05.003+0000 7ff2fa000640 1 -- 172.21.15.77:0/2358700870 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2f4027070 con 0x560fd4ab2e40 2024-09-10T09:56:05.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:05.003+0000 7ff2fa000640 1 -- 172.21.15.77:0/2358700870 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7ff2fc0a4480 con 0x560fd4ab2e40 2024-09-10T09:56:05.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:05.003+0000 7ff315bcd740 10 client.25529 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T09:56:05.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:05.003+0000 7ff315bcd740 10 client.25529 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T09:56:05.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:05.003+0000 7ff315bcd740 20 client.25529 populate_metadata read hostname 'smithi077' 2024-09-10T09:56:05.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:05.003+0000 7ff315bcd740 1 --2- 172.21.15.77:0/2358700870 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x560fd4b38770 0x560fd4b3ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:05.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:05.003+0000 7ff315bcd740 4 client.25529 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:56:05.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:05.003+0000 7ff315bcd740 1 -- 172.21.15.77:0/2358700870 --> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560fd4a20e60 con 0x560fd4b38770 2024-09-10T09:56:05.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:05.003+0000 7ff303400640 1 --2- 172.21.15.77:0/2358700870 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x560fd4b38770 0x560fd4b3ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:05.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:05.004+0000 7ff303400640 1 --2- 172.21.15.77:0/2358700870 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x560fd4b38770 0x560fd4b3ab60 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f9b4c95eb151a4bc server_cookie=1e54b3739c509c4c in_seq=0 out_seq=0 2024-09-10T09:56:05.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:05.004+0000 7ff2fa000640 10 client.25529 ms_handle_connect on v2:172.21.15.134:6836/2461497091 2024-09-10T09:56:05.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:05.004+0000 7ff2fa000640 1 -- 172.21.15.77:0/2358700870 <== mds.3 v2:172.21.15.134:6836/2461497091 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560fd4a20e60 con 0x560fd4b38770 2024-09-10T09:56:05.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:05.004+0000 7ff2fa000640 10 client.25529 handle_command_reply: tid=0 2024-09-10T09:56:05.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:05.004+0000 7ff315bcd740 1 client.25529 shutdown 2024-09-10T09:56:05.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:05.005+0000 7ff315bcd740 1 -- 172.21.15.77:0/2358700870 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560fd4ab2e40 msgr2=0x560fd4ac0f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:05.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:05.005+0000 7ff315bcd740 1 --2- 172.21.15.77:0/2358700870 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560fd4ab2e40 0x560fd4ac0f80 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc098000 tx=0x7ff2fc009310 comp rx=0 tx=0).stop 2024-09-10T09:56:05.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:05.005+0000 7ff315bcd740 1 -- 172.21.15.77:0/2358700870 shutdown_connections 2024-09-10T09:56:05.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:05.005+0000 7ff315bcd740 1 --2- 172.21.15.77:0/2358700870 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fd4abe020 0x560fd4ac1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:05.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:05.005+0000 7ff315bcd740 1 --2- 172.21.15.77:0/2358700870 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x560fd4b38770 0x560fd4b3ab60 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:05.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:05.005+0000 7ff315bcd740 1 --2- 172.21.15.77:0/2358700870 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fd4ab3760 0x560fd4ac14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:05.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:05.005+0000 7ff315bcd740 1 --2- 172.21.15.77:0/2358700870 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560fd4ab2e40 0x560fd4ac0f80 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:05.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:05.005+0000 7ff2fa000640 0 client.25529 ms_handle_reset on v2:172.21.15.134:6836/2461497091 2024-09-10T09:56:05.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:05.005+0000 7ff315bcd740 1 -- 172.21.15.77:0/2358700870 >> 172.21.15.77:0/2358700870 conn(0x560fd4aaf3d0 msgr2=0x560fd4aadc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:05.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:05.005+0000 7ff315bcd740 1 -- 172.21.15.77:0/2358700870 shutdown_connections 2024-09-10T09:56:05.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:05.005+0000 7ff315bcd740 1 -- 172.21.15.77:0/2358700870 wait complete. 2024-09-10T09:56:05.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:05.005+0000 7ff315bcd740 20 client.25529 trim_cache size 0 max 16384 2024-09-10T09:56:05.010 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-10T09:56:05.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:05.006+0000 7ff311a00640 1 -- 172.21.15.77:0/150791239 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ff2f00a2c00 msgr2=0x7ff2f00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:05.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:05.006+0000 7ff311a00640 1 --2- 172.21.15.77:0/150791239 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ff2f00a2c00 0x7ff2f00a50c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff304006010 tx=0x7ff30402a8c0 comp rx=0 tx=0).stop 2024-09-10T09:56:05.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:05.006+0000 7ff311a00640 1 -- 172.21.15.77:0/150791239 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff30c1007a0 msgr2=0x7ff30c19a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:05.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:05.006+0000 7ff311a00640 1 --2- 172.21.15.77:0/150791239 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff30c1007a0 0x7ff30c19a490 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc00e9f0 tx=0x7ff2fc00edf0 comp rx=0 tx=0).stop 2024-09-10T09:56:05.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:05.006+0000 7ff311a00640 1 -- 172.21.15.77:0/150791239 shutdown_connections 2024-09-10T09:56:05.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:05.006+0000 7ff311a00640 1 --2- 172.21.15.77:0/150791239 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ff2f00a2c00 0x7ff2f00a50c0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:05.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:05.006+0000 7ff311a00640 1 --2- 172.21.15.77:0/150791239 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff30c10e350 0x7ff30c19ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:05.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:05.006+0000 7ff311a00640 1 --2- 172.21.15.77:0/150791239 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff30c101150 0x7ff30c19a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:05.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:05.006+0000 7ff311a00640 1 --2- 172.21.15.77:0/150791239 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff30c1007a0 0x7ff30c19a490 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:05.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:05.006+0000 7ff311a00640 1 -- 172.21.15.77:0/150791239 >> 172.21.15.77:0/150791239 conn(0x7ff30c0fc610 msgr2=0x7ff30c0fea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:05.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:05.006+0000 7ff311a00640 1 -- 172.21.15.77:0/150791239 shutdown_connections 2024-09-10T09:56:05.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:05.006+0000 7ff311a00640 1 -- 172.21.15.77:0/150791239 wait complete. 2024-09-10T09:56:05.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:56:04 smithi134 ceph-mon[35674]: pgmap v418: 129 pgs: 129 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-10T09:56:05.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:56:04 smithi120 ceph-mon[34770]: pgmap v418: 129 pgs: 129 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-10T09:56:05.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:56:04 smithi077 ceph-mon[30319]: pgmap v418: 129 pgs: 129 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-10T09:56:05.350 INFO:tasks.quiescer.fs.[cephfs]:Total duration (60) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-10T09:56:05.350 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-09-10T09:56:05.351 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:56:05.786 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T09:56:05.879 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:56:06.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:06.648+0000 7fd468e00640 1 -- 172.21.15.77:0/2520311822 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd4641027a0 msgr2=0x7fd4641067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:06.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:06.648+0000 7fd468e00640 1 --2- 172.21.15.77:0/2520311822 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd4641027a0 0x7fd4641067f0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fd450009900 tx=0x7fd45002a3f0 comp rx=0 tx=0).stop 2024-09-10T09:56:06.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:06.648+0000 7fd468e00640 1 -- 172.21.15.77:0/2520311822 shutdown_connections 2024-09-10T09:56:06.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:06.648+0000 7fd468e00640 1 --2- 172.21.15.77:0/2520311822 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd464106f20 0x7fd46410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:06.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:06.648+0000 7fd468e00640 1 --2- 172.21.15.77:0/2520311822 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd4641027a0 0x7fd4641067f0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:06.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:06.648+0000 7fd468e00640 1 --2- 172.21.15.77:0/2520311822 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd464101df0 0x7fd4641021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:06.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:06.648+0000 7fd468e00640 1 -- 172.21.15.77:0/2520311822 >> 172.21.15.77:0/2520311822 conn(0x7fd4640fd8b0 msgr2=0x7fd4640ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:06.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:06.649+0000 7fd468e00640 1 -- 172.21.15.77:0/2520311822 shutdown_connections 2024-09-10T09:56:06.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:06.649+0000 7fd468e00640 1 -- 172.21.15.77:0/2520311822 wait complete. 2024-09-10T09:56:06.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:06.650+0000 7fd468e00640 1 Processor -- start 2024-09-10T09:56:06.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:06.650+0000 7fd468e00640 1 -- start start 2024-09-10T09:56:06.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:06.651+0000 7fd468e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd464101df0 0x7fd46410b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:06.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:06.651+0000 7fd468e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4641027a0 0x7fd46410b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:06.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:06.651+0000 7fd468e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd464106f20 0x7fd4641aa650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:06.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:06.651+0000 7fd468e00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd4640756e0 con 0x7fd464101df0 2024-09-10T09:56:06.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:06.651+0000 7fd468e00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd464075560 con 0x7fd464106f20 2024-09-10T09:56:06.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:06.651+0000 7fd468e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd464075860 con 0x7fd4641027a0 2024-09-10T09:56:06.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:06.651+0000 7fd461e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd464101df0 0x7fd46410b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:06.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:06.651+0000 7fd461e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd464101df0 0x7fd46410b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33982/0 (socket says 172.21.15.77:33982) 2024-09-10T09:56:06.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:06.651+0000 7fd461e00640 1 -- 172.21.15.77:0/3684623937 learned_addr learned my addr 172.21.15.77:0/3684623937 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:06.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:06.651+0000 7fd462800640 1 --2- 172.21.15.77:0/3684623937 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd464106f20 0x7fd4641aa650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:06.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:06.651+0000 7fd461400640 1 --2- 172.21.15.77:0/3684623937 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4641027a0 0x7fd46410b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:06.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:06.652+0000 7fd462800640 1 -- 172.21.15.77:0/3684623937 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4641027a0 msgr2=0x7fd46410b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:06.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:06.652+0000 7fd462800640 1 --2- 172.21.15.77:0/3684623937 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4641027a0 0x7fd46410b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:06.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:06.652+0000 7fd462800640 1 -- 172.21.15.77:0/3684623937 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd464101df0 msgr2=0x7fd46410b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:06.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:06.652+0000 7fd462800640 1 --2- 172.21.15.77:0/3684623937 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd464101df0 0x7fd46410b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:06.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:06.652+0000 7fd462800640 1 -- 172.21.15.77:0/3684623937 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4641aaca0 con 0x7fd464106f20 2024-09-10T09:56:06.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:06.652+0000 7fd462800640 1 --2- 172.21.15.77:0/3684623937 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd464106f20 0x7fd4641aa650 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd45c009fd0 tx=0x7fd45c00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:06.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:06.652+0000 7fd45aa00640 1 -- 172.21.15.77:0/3684623937 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd45c00ce20 con 0x7fd464106f20 2024-09-10T09:56:06.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:06.652+0000 7fd45aa00640 1 -- 172.21.15.77:0/3684623937 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd45c005020 con 0x7fd464106f20 2024-09-10T09:56:06.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:06.652+0000 7fd468e00640 1 -- 172.21.15.77:0/3684623937 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4641aaf90 con 0x7fd464106f20 2024-09-10T09:56:06.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:06.652+0000 7fd45aa00640 1 -- 172.21.15.77:0/3684623937 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd45c010720 con 0x7fd464106f20 2024-09-10T09:56:06.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:06.652+0000 7fd468e00640 1 -- 172.21.15.77:0/3684623937 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4641ab450 con 0x7fd464106f20 2024-09-10T09:56:06.656 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:06.655+0000 7fd468e00640 1 -- 172.21.15.77:0/3684623937 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd444005290 con 0x7fd464106f20 2024-09-10T09:56:06.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:06.656+0000 7fd45aa00640 1 -- 172.21.15.77:0/3684623937 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd45c004a40 con 0x7fd464106f20 2024-09-10T09:56:06.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:06.658+0000 7fd45aa00640 1 --2- 172.21.15.77:0/3684623937 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fd4480a2c00 0x7fd4480a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:06.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:06.658+0000 7fd45aa00640 1 -- 172.21.15.77:0/3684623937 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fd45c0cf080 con 0x7fd464106f20 2024-09-10T09:56:06.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:06.658+0000 7fd461e00640 1 --2- 172.21.15.77:0/3684623937 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fd4480a2c00 0x7fd4480a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:06.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:06.659+0000 7fd461e00640 1 --2- 172.21.15.77:0/3684623937 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fd4480a2c00 0x7fd4480a50c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd4540095f0 tx=0x7fd454005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:06.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:06.662+0000 7fd45aa00640 1 -- 172.21.15.77:0/3684623937 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd45c09bbf0 con 0x7fd464106f20 2024-09-10T09:56:06.701 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:56:06.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:06.894+0000 7fd468e00640 1 -- 172.21.15.77:0/3684623937 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd444005580 con 0x7fd464106f20 2024-09-10T09:56:06.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:06.895+0000 7fd45aa00640 1 -- 172.21.15.77:0/3684623937 <== mon.2 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11989 (secure 0 0 0) 0x7fd45c09b4f0 con 0x7fd464106f20 2024-09-10T09:56:06.896 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:56:06.896 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":29,"btime":"2024-09-10T09:55:13:863715+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3159289675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3159289675},{"type":"v1","addr":"172.21.15.77:6835","nonce":3159289675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1951962341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1951962341},{"type":"v1","addr":"172.21.15.77:6837","nonce":1951962341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:52:23.368252+0000","modified":"2024-09-10T09:55:13.786975+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24563,"mds_1":24524,"mds_2":24520,"mds_3":24502,"mds_4":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14487":{"gid":14487,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/4291121888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":4291121888},{"type":"v1","addr":"172.21.15.77:6839","nonce":4291121888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14499":{"gid":14499,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6841/445284194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":445284194},{"type":"v1","addr":"172.21.15.77:6841","nonce":445284194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1005054423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1005054423},{"type":"v1","addr":"172.21.15.134:6833","nonce":1005054423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6833/3716307744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":3716307744},{"type":"v1","addr":"172.21.15.120:6833","nonce":3716307744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/1319892544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1319892544},{"type":"v1","addr":"172.21.15.120:6835","nonce":1319892544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.134:6835/3293933699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3293933699},{"type":"v1","addr":"172.21.15.134:6835","nonce":3293933699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.134:6837/2461497091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2461497091},{"type":"v1","addr":"172.21.15.134:6837","nonce":2461497091}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":20,"addr":"172.21.15.134:6839/1965117680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1965117680},{"type":"v1","addr":"172.21.15.134:6839","nonce":1965117680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":20,"addr":"172.21.15.120:6837/3033280709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3033280709},{"type":"v1","addr":"172.21.15.120:6837","nonce":3033280709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.120:6839/1777710157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1777710157},{"type":"v1","addr":"172.21.15.120:6839","nonce":1777710157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24563,"qdb_cluster":[24496,24502,24520,24524,24563]},"id":1}]} 2024-09-10T09:56:06.897 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 29 2024-09-10T09:56:06.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:06.897+0000 7fd468e00640 1 -- 172.21.15.77:0/3684623937 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fd4480a2c00 msgr2=0x7fd4480a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:06.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:06.897+0000 7fd468e00640 1 --2- 172.21.15.77:0/3684623937 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fd4480a2c00 0x7fd4480a50c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd4540095f0 tx=0x7fd454005f90 comp rx=0 tx=0).stop 2024-09-10T09:56:06.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:06.898+0000 7fd468e00640 1 -- 172.21.15.77:0/3684623937 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd464106f20 msgr2=0x7fd4641aa650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:06.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:06.898+0000 7fd468e00640 1 --2- 172.21.15.77:0/3684623937 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd464106f20 0x7fd4641aa650 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd45c009fd0 tx=0x7fd45c00ef10 comp rx=0 tx=0).stop 2024-09-10T09:56:06.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:06.898+0000 7fd468e00640 1 -- 172.21.15.77:0/3684623937 shutdown_connections 2024-09-10T09:56:06.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:06.898+0000 7fd468e00640 1 --2- 172.21.15.77:0/3684623937 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fd4480a2c00 0x7fd4480a50c0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:06.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:06.898+0000 7fd468e00640 1 --2- 172.21.15.77:0/3684623937 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd464106f20 0x7fd4641aa650 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:06.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:06.898+0000 7fd468e00640 1 --2- 172.21.15.77:0/3684623937 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4641027a0 0x7fd46410b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:06.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:06.898+0000 7fd468e00640 1 --2- 172.21.15.77:0/3684623937 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd464101df0 0x7fd46410b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:06.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:06.898+0000 7fd468e00640 1 -- 172.21.15.77:0/3684623937 >> 172.21.15.77:0/3684623937 conn(0x7fd4640fd8b0 msgr2=0x7fd4640fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:06.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:06.899+0000 7fd468e00640 1 -- 172.21.15.77:0/3684623937 shutdown_connections 2024-09-10T09:56:06.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:06.899+0000 7fd468e00640 1 -- 172.21.15.77:0/3684623937 wait complete. 2024-09-10T09:56:06.902 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:56:06 smithi077 ceph-mon[30319]: pgmap v419: 129 pgs: 129 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T09:56:07.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:56:06 smithi134 ceph-mon[35674]: pgmap v419: 129 pgs: 129 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T09:56:07.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:56:06 smithi120 ceph-mon[34770]: pgmap v419: 129 pgs: 129 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T09:56:07.758 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24563 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-09-10T09:56:07.758 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24563 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-09-10T09:56:07.981 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:56:07 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/3684623937' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:56:08.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.007+0000 7f6f51800640 1 -- 172.21.15.77:0/3875895312 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6f4c0745c0 msgr2=0x7f6f4c0749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:08.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.007+0000 7f6f51800640 1 --2- 172.21.15.77:0/3875895312 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6f4c0745c0 0x7f6f4c0749a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c009be0 tx=0x7f6f3c02a750 comp rx=0 tx=0).stop 2024-09-10T09:56:08.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.007+0000 7f6f51800640 1 -- 172.21.15.77:0/3875895312 shutdown_connections 2024-09-10T09:56:08.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.007+0000 7f6f51800640 1 --2- 172.21.15.77:0/3875895312 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6f4c10e430 0x7f6f4c110820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:08.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.007+0000 7f6f51800640 1 --2- 172.21.15.77:0/3875895312 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6f4c074ee0 0x7f6f4c10def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:08.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.007+0000 7f6f51800640 1 --2- 172.21.15.77:0/3875895312 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6f4c0745c0 0x7f6f4c0749a0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:08.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.007+0000 7f6f51800640 1 -- 172.21.15.77:0/3875895312 >> 172.21.15.77:0/3875895312 conn(0x7f6f4c0fd8b0 msgr2=0x7f6f4c0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:08.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.008+0000 7f6f51800640 1 -- 172.21.15.77:0/3875895312 shutdown_connections 2024-09-10T09:56:08.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.008+0000 7f6f51800640 1 -- 172.21.15.77:0/3875895312 wait complete. 2024-09-10T09:56:08.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.009+0000 7f6f51800640 1 Processor -- start 2024-09-10T09:56:08.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.009+0000 7f6f51800640 1 -- start start 2024-09-10T09:56:08.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.009+0000 7f6f51800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6f4c0745c0 0x7f6f4c19a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:08.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.009+0000 7f6f51800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6f4c074ee0 0x7f6f4c19abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:08.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.010+0000 7f6f51800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6f4c10e430 0x7f6f4c19ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:08.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.010+0000 7f6f51800640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6f4c114190 con 0x7f6f4c0745c0 2024-09-10T09:56:08.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.010+0000 7f6f51800640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6f4c114010 con 0x7f6f4c10e430 2024-09-10T09:56:08.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.010+0000 7f6f51800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6f4c114310 con 0x7f6f4c074ee0 2024-09-10T09:56:08.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.010+0000 7f6f4aa00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6f4c0745c0 0x7f6f4c19a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:08.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.010+0000 7f6f4aa00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6f4c0745c0 0x7f6f4c19a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33998/0 (socket says 172.21.15.77:33998) 2024-09-10T09:56:08.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.010+0000 7f6f4aa00640 1 -- 172.21.15.77:0/1088276404 learned_addr learned my addr 172.21.15.77:0/1088276404 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:08.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.010+0000 7f6f4a000640 1 --2- 172.21.15.77:0/1088276404 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6f4c074ee0 0x7f6f4c19abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:08.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.010+0000 7f6f4b400640 1 --2- 172.21.15.77:0/1088276404 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6f4c10e430 0x7f6f4c19ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:08.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.010+0000 7f6f4aa00640 1 -- 172.21.15.77:0/1088276404 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6f4c074ee0 msgr2=0x7f6f4c19abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:08.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.010+0000 7f6f4aa00640 1 --2- 172.21.15.77:0/1088276404 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6f4c074ee0 0x7f6f4c19abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:08.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.010+0000 7f6f4aa00640 1 -- 172.21.15.77:0/1088276404 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6f4c10e430 msgr2=0x7f6f4c19ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:08.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.010+0000 7f6f4aa00640 1 --2- 172.21.15.77:0/1088276404 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6f4c10e430 0x7f6f4c19ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:08.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.011+0000 7f6f4aa00640 1 -- 172.21.15.77:0/1088276404 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f4c19f620 con 0x7f6f4c0745c0 2024-09-10T09:56:08.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.011+0000 7f6f4aa00640 1 --2- 172.21.15.77:0/1088276404 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6f4c0745c0 0x7f6f4c19a670 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c034040 tx=0x7f6f3c0305d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:08.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.011+0000 7f6f43400640 1 -- 172.21.15.77:0/1088276404 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f3c041020 con 0x7f6f4c0745c0 2024-09-10T09:56:08.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.012+0000 7f6f43400640 1 -- 172.21.15.77:0/1088276404 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f3c003e50 con 0x7f6f4c0745c0 2024-09-10T09:56:08.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.012+0000 7f6f43400640 1 -- 172.21.15.77:0/1088276404 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f3c036040 con 0x7f6f4c0745c0 2024-09-10T09:56:08.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.012+0000 7f6f51800640 1 -- 172.21.15.77:0/1088276404 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f4c19f8b0 con 0x7f6f4c0745c0 2024-09-10T09:56:08.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.012+0000 7f6f51800640 1 -- 172.21.15.77:0/1088276404 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f4c19fd40 con 0x7f6f4c0745c0 2024-09-10T09:56:08.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.015+0000 7f6f43400640 1 -- 172.21.15.77:0/1088276404 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6f3c044020 con 0x7f6f4c0745c0 2024-09-10T09:56:08.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.016+0000 7f6f54418740 1 Processor -- start 2024-09-10T09:56:08.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.016+0000 7f6f54418740 1 -- start start 2024-09-10T09:56:08.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.016+0000 7f6f54418740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b578862480 0x55b578862860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:08.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.016+0000 7f6f54418740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b578862da0 0x55b57886d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:08.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.016+0000 7f6f54418740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b57886d660 0x55b57886fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:08.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.016+0000 7f6f54418740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55b5787c03e0 con 0x55b578862480 2024-09-10T09:56:08.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.016+0000 7f6f54418740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b578767d80 con 0x55b578862da0 2024-09-10T09:56:08.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.016+0000 7f6f54418740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b5787bc3a0 con 0x55b57886d660 2024-09-10T09:56:08.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.017+0000 7f6f4b400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b578862480 0x55b578862860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:08.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.017+0000 7f6f4b400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b578862480 0x55b578862860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34014/0 (socket says 172.21.15.77:34014) 2024-09-10T09:56:08.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.017+0000 7f6f4b400640 1 -- 172.21.15.77:0/3788135371 learned_addr learned my addr 172.21.15.77:0/3788135371 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:08.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.017+0000 7f6f4aa00640 1 --2- 172.21.15.77:0/3788135371 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b578862da0 0x55b57886d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:08.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.017+0000 7f6f4a000640 1 --2- 172.21.15.77:0/3788135371 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b57886d660 0x55b57886fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:08.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.017+0000 7f6f43400640 1 --2- 172.21.15.77:0/1088276404 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f6f300a2c00 0x7f6f300a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:08.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.018+0000 7f6f43400640 1 -- 172.21.15.77:0/1088276404 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f6f3c0f0010 con 0x7f6f4c0745c0 2024-09-10T09:56:08.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.018+0000 7f6f4b400640 1 -- 172.21.15.77:0/3788135371 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b57886d660 msgr2=0x55b57886fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:08.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.018+0000 7f6f4b400640 1 --2- 172.21.15.77:0/3788135371 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b57886d660 0x55b57886fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:08.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.018+0000 7f6f4b400640 1 -- 172.21.15.77:0/3788135371 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b578862da0 msgr2=0x55b57886d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:08.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.018+0000 7f6f4b400640 1 --2- 172.21.15.77:0/3788135371 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b578862da0 0x55b57886d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:08.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.018+0000 7f6f4b400640 1 -- 172.21.15.77:0/3788135371 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b578786ae0 con 0x55b578862480 2024-09-10T09:56:08.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.018+0000 7f6f4a000640 1 --2- 172.21.15.77:0/3788135371 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b57886d660 0x55b57886fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:56:08.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.019+0000 7f6f4b400640 1 --2- 172.21.15.77:0/3788135371 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b578862480 0x55b578862860 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f6f44009ba0 tx=0x7f6f44011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc6d01d43f751365 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:08.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.019+0000 7f6f4b400640 1 --2- 172.21.15.77:0/1088276404 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f6f300a2c00 0x7f6f300a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:08.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.019+0000 7f6f3be00640 1 -- 172.21.15.77:0/3788135371 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f44004030 con 0x55b578862480 2024-09-10T09:56:08.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.019+0000 7f6f3be00640 1 -- 172.21.15.77:0/3788135371 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f440041d0 con 0x55b578862480 2024-09-10T09:56:08.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.019+0000 7f6f3be00640 1 -- 172.21.15.77:0/3788135371 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f4401a860 con 0x55b578862480 2024-09-10T09:56:08.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.020+0000 7f6f54418740 1 -- 172.21.15.77:0/3788135371 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b578862480 msgr2=0x55b578862860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:08.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.020+0000 7f6f4b400640 1 --2- 172.21.15.77:0/1088276404 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f6f300a2c00 0x7f6f300a50c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f6f4401c040 tx=0x7f6f4401ab00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:08.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.020+0000 7f6f54418740 1 --2- 172.21.15.77:0/3788135371 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b578862480 0x55b578862860 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f6f44009ba0 tx=0x7f6f44011810 comp rx=0 tx=0).stop 2024-09-10T09:56:08.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.020+0000 7f6f54418740 1 -- 172.21.15.77:0/3788135371 shutdown_connections 2024-09-10T09:56:08.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.020+0000 7f6f54418740 1 --2- 172.21.15.77:0/3788135371 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b57886d660 0x55b57886fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:08.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.020+0000 7f6f54418740 1 --2- 172.21.15.77:0/3788135371 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b578862da0 0x55b57886d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:08.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.020+0000 7f6f54418740 1 --2- 172.21.15.77:0/3788135371 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b578862480 0x55b578862860 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:08.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.020+0000 7f6f54418740 1 -- 172.21.15.77:0/3788135371 >> 172.21.15.77:0/3788135371 conn(0x55b57885ea10 msgr2=0x55b57885edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:08.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.021+0000 7f6f54418740 1 -- 172.21.15.77:0/3788135371 shutdown_connections 2024-09-10T09:56:08.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.021+0000 7f6f54418740 1 -- 172.21.15.77:0/3788135371 wait complete. 2024-09-10T09:56:08.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.021+0000 7f6f54418740 1 -- start start 2024-09-10T09:56:08.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.022+0000 7f6f54418740 1 Processor -- start 2024-09-10T09:56:08.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.022+0000 7f6f54418740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b578862480 0x55b57886a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:08.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.022+0000 7f6f54418740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b578862da0 0x55b57886aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:08.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.022+0000 7f6f54418740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b57886d660 0x55b57886af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:08.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.022+0000 7f6f54418740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55b578870230 con 0x55b578862da0 2024-09-10T09:56:08.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.022+0000 7f6f54418740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b57877b0f0 con 0x55b57886d660 2024-09-10T09:56:08.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.022+0000 7f6f54418740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b5788703b0 con 0x55b578862480 2024-09-10T09:56:08.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.023+0000 7f6f54418740 10 client.0 _ll_register_callbacks cb 0x55b578762bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:56:08.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.023+0000 7f6f4aa00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b578862480 0x55b57886a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:08.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.023+0000 7f6f4b400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b57886d660 0x55b57886af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:08.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.023+0000 7f6f4aa00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b578862480 0x55b57886a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:53516/0 (socket says 172.21.15.77:53516) 2024-09-10T09:56:08.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.023+0000 7f6f4aa00640 1 -- 172.21.15.77:0/453544649 learned_addr learned my addr 172.21.15.77:0/453544649 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:08.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.023+0000 7f6f4a000640 1 --2- 172.21.15.77:0/453544649 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b578862da0 0x55b57886aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:08.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.023+0000 7f6f4b400640 1 -- 172.21.15.77:0/453544649 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b578862480 msgr2=0x55b57886a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:08.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.023+0000 7f6f4b400640 1 --2- 172.21.15.77:0/453544649 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b578862480 0x55b57886a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:08.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.023+0000 7f6f4b400640 1 -- 172.21.15.77:0/453544649 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b578862da0 msgr2=0x55b57886aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:08.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.024+0000 7f6f4b400640 1 --2- 172.21.15.77:0/453544649 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b578862da0 0x55b57886aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:08.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.024+0000 7f6f4b400640 1 -- 172.21.15.77:0/453544649 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5785313c0 con 0x55b57886d660 2024-09-10T09:56:08.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.024+0000 7f6f4b400640 1 --2- 172.21.15.77:0/453544649 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b57886d660 0x55b57886af80 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f6f44011d10 tx=0x7f6f44018000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab10bdb283ebe9c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:08.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.024+0000 7f6f38c00640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T09:56:08.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.024+0000 7f6f38c00640 1 -- 172.21.15.77:0/453544649 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f44037060 con 0x55b57886d660 2024-09-10T09:56:08.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.024+0000 7f6f54418740 1 -- 172.21.15.77:0/453544649 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f440041d0 con 0x55b57886d660 2024-09-10T09:56:08.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.024+0000 7f6f38c00640 1 -- 172.21.15.77:0/453544649 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f6f44004380 con 0x55b57886d660 2024-09-10T09:56:08.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.025+0000 7f6f38c00640 1 -- 172.21.15.77:0/453544649 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6f4403e050 con 0x55b57886d660 2024-09-10T09:56:08.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.025+0000 7f6f54418740 10 client.25547 fetch_fsmap learned FSMap version 29 2024-09-10T09:56:08.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.025+0000 7f6f54418740 1 -- 172.21.15.77:0/453544649 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b57887c210 con 0x55b57886d660 2024-09-10T09:56:08.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.025+0000 7f6f38c00640 1 -- 172.21.15.77:0/453544649 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f440041d0 con 0x55b57886d660 2024-09-10T09:56:08.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.025+0000 7f6f38c00640 1 -- 172.21.15.77:0/453544649 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f6f44018070 con 0x55b57886d660 2024-09-10T09:56:08.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.025+0000 7f6f54418740 10 client.25547 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T09:56:08.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.025+0000 7f6f54418740 10 client.25547 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T09:56:08.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.025+0000 7f6f54418740 20 client.25547 populate_metadata read hostname 'smithi077' 2024-09-10T09:56:08.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.025+0000 7f6f54418740 1 --2- 172.21.15.77:0/453544649 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x55b57887d5c0 0x55b57887f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:08.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.025+0000 7f6f54418740 4 client.25547 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:56:08.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.025+0000 7f6f54418740 1 -- 172.21.15.77:0/453544649 --> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b5787cd950 con 0x55b57887d5c0 2024-09-10T09:56:08.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.026+0000 7f6f4aa00640 1 --2- 172.21.15.77:0/453544649 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x55b57887d5c0 0x55b57887f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:08.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.027+0000 7f6f38c00640 10 client.25547 ms_handle_connect on v2:172.21.15.134:6838/1965117680 2024-09-10T09:56:08.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.027+0000 7f6f4aa00640 1 --2- 172.21.15.77:0/453544649 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x55b57887d5c0 0x55b57887f9b0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=52987816e587d6f8 server_cookie=f90251b32a7e49f2 in_seq=0 out_seq=0 2024-09-10T09:56:08.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.027+0000 7f6f38c00640 1 -- 172.21.15.77:0/453544649 <== mds.2 v2:172.21.15.134:6838/1965117680 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b5787cd950 con 0x55b57887d5c0 2024-09-10T09:56:08.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.027+0000 7f6f38c00640 10 client.25547 handle_command_reply: tid=0 2024-09-10T09:56:08.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.027+0000 7f6f54418740 1 client.25547 shutdown 2024-09-10T09:56:08.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.028+0000 7f6f54418740 1 -- 172.21.15.77:0/453544649 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b57886d660 msgr2=0x55b57886af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:08.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.028+0000 7f6f54418740 1 --2- 172.21.15.77:0/453544649 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b57886d660 0x55b57886af80 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f6f44011d10 tx=0x7f6f44018000 comp rx=0 tx=0).stop 2024-09-10T09:56:08.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.028+0000 7f6f54418740 1 -- 172.21.15.77:0/453544649 shutdown_connections 2024-09-10T09:56:08.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.028+0000 7f6f54418740 1 --2- 172.21.15.77:0/453544649 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x55b57887d5c0 0x55b57887f9b0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:08.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.028+0000 7f6f54418740 1 --2- 172.21.15.77:0/453544649 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b57886d660 0x55b57886af80 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:08.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.028+0000 7f6f38c00640 0 client.25547 ms_handle_reset on v2:172.21.15.134:6838/1965117680 2024-09-10T09:56:08.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.028+0000 7f6f54418740 1 --2- 172.21.15.77:0/453544649 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b578862da0 0x55b57886aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:08.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.028+0000 7f6f54418740 1 --2- 172.21.15.77:0/453544649 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b578862480 0x55b57886a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:08.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.028+0000 7f6f54418740 1 -- 172.21.15.77:0/453544649 >> 172.21.15.77:0/453544649 conn(0x55b57885ea10 msgr2=0x55b578855380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:08.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.028+0000 7f6f54418740 1 -- 172.21.15.77:0/453544649 shutdown_connections 2024-09-10T09:56:08.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.028+0000 7f6f54418740 1 -- 172.21.15.77:0/453544649 wait complete. 2024-09-10T09:56:08.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.028+0000 7f6f54418740 20 client.25547 trim_cache size 0 max 16384 2024-09-10T09:56:08.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.042+0000 7f6f54418740 1 Processor -- start 2024-09-10T09:56:08.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.042+0000 7f6f54418740 1 -- start start 2024-09-10T09:56:08.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.043+0000 7f6f54418740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b578862480 0x55b578864f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:08.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.043+0000 7f6f54418740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b578862da0 0x55b5788851a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:08.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.043+0000 7f6f54418740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b57886d660 0x55b5788856e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:08.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.043+0000 7f6f54418740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6f4c1acd20 con 0x55b578862480 2024-09-10T09:56:08.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.043+0000 7f6f54418740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6f4c1ae860 con 0x55b57886d660 2024-09-10T09:56:08.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.043+0000 7f6f54418740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6f4c1ad3f0 con 0x55b578862da0 2024-09-10T09:56:08.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.043+0000 7f6f4b400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b57886d660 0x55b5788856e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:08.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.043+0000 7f6f4a000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b578862da0 0x55b5788851a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:08.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.043+0000 7f6f4b400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b57886d660 0x55b5788856e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:51022/0 (socket says 172.21.15.77:51022) 2024-09-10T09:56:08.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.043+0000 7f6f4b400640 1 -- 172.21.15.77:0/271157633 learned_addr learned my addr 172.21.15.77:0/271157633 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:08.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.043+0000 7f6f4b400640 1 -- 172.21.15.77:0/271157633 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b578862da0 msgr2=0x55b5788851a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:08.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.043+0000 7f6f4b400640 1 --2- 172.21.15.77:0/271157633 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b578862da0 0x55b5788851a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:08.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.043+0000 7f6f4b400640 1 -- 172.21.15.77:0/271157633 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b578862480 msgr2=0x55b578864f30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T09:56:08.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.044+0000 7f6f4aa00640 1 --2- 172.21.15.77:0/271157633 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b578862480 0x55b578864f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:08.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.044+0000 7f6f4b400640 1 --2- 172.21.15.77:0/271157633 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b578862480 0x55b578864f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:08.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.044+0000 7f6f4b400640 1 -- 172.21.15.77:0/271157633 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b578862bd0 con 0x55b57886d660 2024-09-10T09:56:08.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.044+0000 7f6f4a000640 1 --2- 172.21.15.77:0/271157633 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b578862da0 0x55b5788851a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:56:08.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.044+0000 7f6f4aa00640 1 --2- 172.21.15.77:0/271157633 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b578862480 0x55b578864f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:56:08.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.044+0000 7f6f4b400640 1 --2- 172.21.15.77:0/271157633 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b57886d660 0x55b5788856e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6f4401b000 tx=0x7f6f44009b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88a97e543ca5d801 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:08.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.044+0000 7f6f3b400640 1 -- 172.21.15.77:0/271157633 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f44037040 con 0x55b57886d660 2024-09-10T09:56:08.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.045+0000 7f6f3b400640 1 -- 172.21.15.77:0/271157633 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f44047070 con 0x55b57886d660 2024-09-10T09:56:08.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.045+0000 7f6f3b400640 1 -- 172.21.15.77:0/271157633 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f44005160 con 0x55b57886d660 2024-09-10T09:56:08.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.045+0000 7f6f54418740 1 -- 172.21.15.77:0/271157633 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b57886d660 msgr2=0x55b5788856e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:08.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.045+0000 7f6f54418740 1 --2- 172.21.15.77:0/271157633 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b57886d660 0x55b5788856e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6f4401b000 tx=0x7f6f44009b20 comp rx=0 tx=0).stop 2024-09-10T09:56:08.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.046+0000 7f6f54418740 1 -- 172.21.15.77:0/271157633 shutdown_connections 2024-09-10T09:56:08.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.046+0000 7f6f54418740 1 --2- 172.21.15.77:0/271157633 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b57886d660 0x55b5788856e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:08.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.046+0000 7f6f54418740 1 --2- 172.21.15.77:0/271157633 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b578862da0 0x55b5788851a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:08.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.046+0000 7f6f54418740 1 --2- 172.21.15.77:0/271157633 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b578862480 0x55b578864f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:08.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.046+0000 7f6f54418740 1 -- 172.21.15.77:0/271157633 >> 172.21.15.77:0/271157633 conn(0x55b57885ea10 msgr2=0x55b57886c580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:08.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.046+0000 7f6f54418740 1 -- 172.21.15.77:0/271157633 shutdown_connections 2024-09-10T09:56:08.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.046+0000 7f6f54418740 1 -- 172.21.15.77:0/271157633 wait complete. 2024-09-10T09:56:08.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.046+0000 7f6f54418740 1 -- start start 2024-09-10T09:56:08.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.047+0000 7f6f54418740 1 Processor -- start 2024-09-10T09:56:08.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.047+0000 7f6f54418740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b578862480 0x55b578875ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:08.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.047+0000 7f6f54418740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b578862da0 0x55b578871010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:08.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.047+0000 7f6f54418740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b57886d660 0x55b578871550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:08.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.047+0000 7f6f54418740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6f4c1ae190 con 0x55b578862da0 2024-09-10T09:56:08.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.047+0000 7f6f54418740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6f4c1adac0 con 0x55b578862480 2024-09-10T09:56:08.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.047+0000 7f6f54418740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6f4c1aad00 con 0x55b57886d660 2024-09-10T09:56:08.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.047+0000 7f6f54418740 10 client.0 _ll_register_callbacks cb 0x7f6f44009a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:56:08.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.048+0000 7f6f4aa00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b578862480 0x55b578875ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:08.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.048+0000 7f6f4b400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b57886d660 0x55b578871550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:08.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.048+0000 7f6f4b400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b57886d660 0x55b578871550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:53530/0 (socket says 172.21.15.77:53530) 2024-09-10T09:56:08.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.048+0000 7f6f4b400640 1 -- 172.21.15.77:0/1869558397 learned_addr learned my addr 172.21.15.77:0/1869558397 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:08.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.048+0000 7f6f4a000640 1 --2- 172.21.15.77:0/1869558397 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b578862da0 0x55b578871010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:08.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.048+0000 7f6f4b400640 1 -- 172.21.15.77:0/1869558397 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b578862480 msgr2=0x55b578875ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:08.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.048+0000 7f6f4b400640 1 --2- 172.21.15.77:0/1869558397 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b578862480 0x55b578875ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:08.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.048+0000 7f6f4b400640 1 -- 172.21.15.77:0/1869558397 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b578862da0 msgr2=0x55b578871010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:08.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.048+0000 7f6f4b400640 1 --2- 172.21.15.77:0/1869558397 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b578862da0 0x55b578871010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:08.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.048+0000 7f6f4b400640 1 -- 172.21.15.77:0/1869558397 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b578871d50 con 0x55b57886d660 2024-09-10T09:56:08.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.048+0000 7f6f4a000640 1 --2- 172.21.15.77:0/1869558397 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b578862da0 0x55b578871010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:56:08.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.049+0000 7f6f4b400640 1 --2- 172.21.15.77:0/1869558397 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b57886d660 0x55b578871550 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6f4401af80 tx=0x7f6f4401a8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93a85916d442e8a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:08.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.049+0000 7f6f38c00640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-10T09:56:08.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.049+0000 7f6f38c00640 1 -- 172.21.15.77:0/1869558397 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f44019a90 con 0x55b57886d660 2024-09-10T09:56:08.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.049+0000 7f6f38c00640 1 -- 172.21.15.77:0/1869558397 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f6f44019c30 con 0x55b57886d660 2024-09-10T09:56:08.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.049+0000 7f6f54418740 1 -- 172.21.15.77:0/1869558397 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f44047070 con 0x55b57886d660 2024-09-10T09:56:08.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.049+0000 7f6f38c00640 1 -- 172.21.15.77:0/1869558397 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6f4405a050 con 0x55b57886d660 2024-09-10T09:56:08.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.049+0000 7f6f54418740 10 client.25522 fetch_fsmap learned FSMap version 29 2024-09-10T09:56:08.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.049+0000 7f6f54418740 1 -- 172.21.15.77:0/1869558397 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b5788e8050 con 0x55b57886d660 2024-09-10T09:56:08.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.049+0000 7f6f38c00640 1 -- 172.21.15.77:0/1869558397 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f44047070 con 0x55b57886d660 2024-09-10T09:56:08.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.050+0000 7f6f38c00640 1 -- 172.21.15.77:0/1869558397 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f6f44018040 con 0x55b57886d660 2024-09-10T09:56:08.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.050+0000 7f6f54418740 10 client.25522 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T09:56:08.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.050+0000 7f6f54418740 10 client.25522 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T09:56:08.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.050+0000 7f6f54418740 20 client.25522 populate_metadata read hostname 'smithi077' 2024-09-10T09:56:08.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.050+0000 7f6f54418740 1 --2- 172.21.15.77:0/1869558397 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x55b5788e86a0 0x55b5788eaac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:08.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.050+0000 7f6f54418740 4 client.25522 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:56:08.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.050+0000 7f6f54418740 1 -- 172.21.15.77:0/1869558397 --> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b5787c7e60 con 0x55b5788e86a0 2024-09-10T09:56:08.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.051+0000 7f6f4aa00640 1 --2- 172.21.15.77:0/1869558397 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x55b5788e86a0 0x55b5788eaac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:08.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.051+0000 7f6f4aa00640 1 --2- 172.21.15.77:0/1869558397 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x55b5788e86a0 0x55b5788eaac0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=31c884db7ff37050 server_cookie=e945a0a80f5a29fa in_seq=0 out_seq=0 2024-09-10T09:56:08.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.051+0000 7f6f38c00640 10 client.25522 ms_handle_connect on v2:172.21.15.134:6838/1965117680 2024-09-10T09:56:08.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.051+0000 7f6f38c00640 1 -- 172.21.15.77:0/1869558397 <== mds.2 v2:172.21.15.134:6838/1965117680 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b5787c7e60 con 0x55b5788e86a0 2024-09-10T09:56:08.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.051+0000 7f6f38c00640 10 client.25522 handle_command_reply: tid=0 2024-09-10T09:56:08.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.051+0000 7f6f54418740 1 client.25522 shutdown 2024-09-10T09:56:08.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.052+0000 7f6f54418740 1 -- 172.21.15.77:0/1869558397 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b57886d660 msgr2=0x55b578871550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:08.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.052+0000 7f6f54418740 1 --2- 172.21.15.77:0/1869558397 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b57886d660 0x55b578871550 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6f4401af80 tx=0x7f6f4401a8b0 comp rx=0 tx=0).stop 2024-09-10T09:56:08.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.052+0000 7f6f54418740 1 -- 172.21.15.77:0/1869558397 shutdown_connections 2024-09-10T09:56:08.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.052+0000 7f6f54418740 1 --2- 172.21.15.77:0/1869558397 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x55b5788e86a0 0x55b5788eaac0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:08.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.052+0000 7f6f54418740 1 --2- 172.21.15.77:0/1869558397 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b57886d660 0x55b578871550 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:08.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.052+0000 7f6f54418740 1 --2- 172.21.15.77:0/1869558397 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b578862da0 0x55b578871010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:08.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.052+0000 7f6f54418740 1 --2- 172.21.15.77:0/1869558397 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b578862480 0x55b578875ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:08.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.052+0000 7f6f54418740 1 -- 172.21.15.77:0/1869558397 >> 172.21.15.77:0/1869558397 conn(0x55b57885ea10 msgr2=0x55b57886bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:08.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.052+0000 7f6f38c00640 0 client.25522 ms_handle_reset on v2:172.21.15.134:6838/1965117680 2024-09-10T09:56:08.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.052+0000 7f6f54418740 1 -- 172.21.15.77:0/1869558397 shutdown_connections 2024-09-10T09:56:08.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.052+0000 7f6f54418740 1 -- 172.21.15.77:0/1869558397 wait complete. 2024-09-10T09:56:08.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.052+0000 7f6f54418740 20 client.25522 trim_cache size 0 max 16384 2024-09-10T09:56:08.056 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-10T09:56:08.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.053+0000 7f6f51800640 1 -- 172.21.15.77:0/1088276404 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f6f300a2c00 msgr2=0x7f6f300a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:08.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.053+0000 7f6f51800640 1 --2- 172.21.15.77:0/1088276404 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f6f300a2c00 0x7f6f300a50c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f6f4401c040 tx=0x7f6f4401ab00 comp rx=0 tx=0).stop 2024-09-10T09:56:08.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.053+0000 7f6f51800640 1 -- 172.21.15.77:0/1088276404 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6f4c0745c0 msgr2=0x7f6f4c19a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:08.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.053+0000 7f6f51800640 1 --2- 172.21.15.77:0/1088276404 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6f4c0745c0 0x7f6f4c19a670 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c034040 tx=0x7f6f3c0305d0 comp rx=0 tx=0).stop 2024-09-10T09:56:08.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.053+0000 7f6f51800640 1 -- 172.21.15.77:0/1088276404 shutdown_connections 2024-09-10T09:56:08.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.053+0000 7f6f51800640 1 --2- 172.21.15.77:0/1088276404 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f6f300a2c00 0x7f6f300a50c0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:08.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.053+0000 7f6f51800640 1 --2- 172.21.15.77:0/1088276404 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6f4c10e430 0x7f6f4c19ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:08.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.053+0000 7f6f51800640 1 --2- 172.21.15.77:0/1088276404 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6f4c074ee0 0x7f6f4c19abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:08.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.053+0000 7f6f51800640 1 --2- 172.21.15.77:0/1088276404 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6f4c0745c0 0x7f6f4c19a670 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:08.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.053+0000 7f6f51800640 1 -- 172.21.15.77:0/1088276404 >> 172.21.15.77:0/1088276404 conn(0x7f6f4c0fd8b0 msgr2=0x7f6f4c0fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:08.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.054+0000 7f6f51800640 1 -- 172.21.15.77:0/1088276404 shutdown_connections 2024-09-10T09:56:08.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:08.054+0000 7f6f51800640 1 -- 172.21.15.77:0/1088276404 wait complete. 2024-09-10T09:56:08.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:56:07 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/3684623937' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:56:08.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:56:07 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/3684623937' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:56:08.373 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:56:08.915 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T09:56:09.096 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:56:08 smithi077 ceph-mon[30319]: pgmap v420: 129 pgs: 129 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-10T09:56:09.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:56:08 smithi134 ceph-mon[35674]: pgmap v420: 129 pgs: 129 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-10T09:56:09.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:56:08 smithi120 ceph-mon[34770]: pgmap v420: 129 pgs: 129 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-10T09:56:09.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.547+0000 7fd09c000640 1 -- 172.21.15.77:0/1179588825 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd094104010 msgr2=0x7fd0941043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:09.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.547+0000 7fd09c000640 1 --2- 172.21.15.77:0/1179588825 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd094104010 0x7fd0941043f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd088009860 tx=0x7fd08802a340 comp rx=0 tx=0).stop 2024-09-10T09:56:09.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.547+0000 7fd09c000640 1 -- 172.21.15.77:0/1179588825 shutdown_connections 2024-09-10T09:56:09.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.547+0000 7fd09c000640 1 --2- 172.21.15.77:0/1179588825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd094109140 0x7fd094110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:09.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.547+0000 7fd09c000640 1 --2- 172.21.15.77:0/1179588825 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd0941049c0 0x7fd094108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:09.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.547+0000 7fd09c000640 1 --2- 172.21.15.77:0/1179588825 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd094104010 0x7fd0941043f0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:09.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.547+0000 7fd09c000640 1 -- 172.21.15.77:0/1179588825 >> 172.21.15.77:0/1179588825 conn(0x7fd0940ffa50 msgr2=0x7fd094101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:09.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.547+0000 7fd09c000640 1 -- 172.21.15.77:0/1179588825 shutdown_connections 2024-09-10T09:56:09.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.547+0000 7fd09c000640 1 -- 172.21.15.77:0/1179588825 wait complete. 2024-09-10T09:56:09.550 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.548+0000 7fd09c000640 1 Processor -- start 2024-09-10T09:56:09.550 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.548+0000 7fd09c000640 1 -- start start 2024-09-10T09:56:09.550 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.549+0000 7fd09c000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd094104010 0x7fd09419a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:09.550 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.549+0000 7fd09c000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0941049c0 0x7fd09419ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:09.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.549+0000 7fd09c000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd094109140 0x7fd09419ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:09.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.549+0000 7fd09c000640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd094114170 con 0x7fd094104010 2024-09-10T09:56:09.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.549+0000 7fd09c000640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd094113ff0 con 0x7fd094109140 2024-09-10T09:56:09.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.549+0000 7fd09c000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd0941142f0 con 0x7fd0941049c0 2024-09-10T09:56:09.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.549+0000 7fd098c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0941049c0 0x7fd09419ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:09.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.549+0000 7fd099600640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd094104010 0x7fd09419a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:09.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.549+0000 7fd098c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0941049c0 0x7fd09419ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:53538/0 (socket says 172.21.15.77:53538) 2024-09-10T09:56:09.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.549+0000 7fd098c00640 1 -- 172.21.15.77:0/797441792 learned_addr learned my addr 172.21.15.77:0/797441792 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:09.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.549+0000 7fd09a000640 1 --2- 172.21.15.77:0/797441792 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd094109140 0x7fd09419ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:09.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.550+0000 7fd099600640 1 -- 172.21.15.77:0/797441792 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0941049c0 msgr2=0x7fd09419ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:09.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.550+0000 7fd099600640 1 --2- 172.21.15.77:0/797441792 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0941049c0 0x7fd09419ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:09.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.550+0000 7fd099600640 1 -- 172.21.15.77:0/797441792 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd094109140 msgr2=0x7fd09419ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:09.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.550+0000 7fd099600640 1 --2- 172.21.15.77:0/797441792 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd094109140 0x7fd09419ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:09.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.550+0000 7fd099600640 1 -- 172.21.15.77:0/797441792 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd09419f5e0 con 0x7fd094104010 2024-09-10T09:56:09.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.550+0000 7fd098c00640 1 --2- 172.21.15.77:0/797441792 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0941049c0 0x7fd09419ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:56:09.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.550+0000 7fd09a000640 1 --2- 172.21.15.77:0/797441792 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd094109140 0x7fd09419ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:56:09.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.550+0000 7fd099600640 1 --2- 172.21.15.77:0/797441792 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd094104010 0x7fd09419a630 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fd088035040 tx=0x7fd088033b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:09.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.551+0000 7fd092000640 1 -- 172.21.15.77:0/797441792 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd08803f020 con 0x7fd094104010 2024-09-10T09:56:09.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.551+0000 7fd092000640 1 -- 172.21.15.77:0/797441792 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd088030550 con 0x7fd094104010 2024-09-10T09:56:09.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.551+0000 7fd09c000640 1 -- 172.21.15.77:0/797441792 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd09419f870 con 0x7fd094104010 2024-09-10T09:56:09.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.551+0000 7fd09c000640 1 -- 172.21.15.77:0/797441792 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0941a71b0 con 0x7fd094104010 2024-09-10T09:56:09.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.551+0000 7fd092000640 1 -- 172.21.15.77:0/797441792 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd088037040 con 0x7fd094104010 2024-09-10T09:56:09.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.554+0000 7fd092000640 1 -- 172.21.15.77:0/797441792 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd088042020 con 0x7fd094104010 2024-09-10T09:56:09.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.555+0000 7fd09ebda740 1 Processor -- start 2024-09-10T09:56:09.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.555+0000 7fd09ebda740 1 -- start start 2024-09-10T09:56:09.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.556+0000 7fd09ebda740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563105e622d0 0x563105e626b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:09.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.556+0000 7fd09ebda740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563105e62bf0 0x563105e75860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:09.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.556+0000 7fd09ebda740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563105e75da0 0x563105e78190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:09.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.556+0000 7fd09ebda740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x563105dc8530 con 0x563105e62bf0 2024-09-10T09:56:09.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.556+0000 7fd09ebda740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x563105d6f860 con 0x563105e75da0 2024-09-10T09:56:09.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.556+0000 7fd09ebda740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x563105dc44f0 con 0x563105e622d0 2024-09-10T09:56:09.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.556+0000 7fd092000640 1 --2- 172.21.15.77:0/797441792 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fd07c0a2b80 0x7fd07c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:09.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.556+0000 7fd099600640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563105e62bf0 0x563105e75860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:09.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.557+0000 7fd092000640 1 -- 172.21.15.77:0/797441792 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fd0880ef400 con 0x7fd094104010 2024-09-10T09:56:09.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.557+0000 7fd098c00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563105e75da0 0x563105e78190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:09.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.557+0000 7fd099600640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563105e62bf0 0x563105e75860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34066/0 (socket says 172.21.15.77:34066) 2024-09-10T09:56:09.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.557+0000 7fd099600640 1 -- 172.21.15.77:0/1897283295 learned_addr learned my addr 172.21.15.77:0/1897283295 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:09.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.557+0000 7fd098c00640 1 -- 172.21.15.77:0/1897283295 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563105e622d0 msgr2=0x563105e626b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:09.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.557+0000 7fd098c00640 1 --2- 172.21.15.77:0/1897283295 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563105e622d0 0x563105e626b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:09.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.557+0000 7fd098c00640 1 -- 172.21.15.77:0/1897283295 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563105e62bf0 msgr2=0x563105e75860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:09.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.557+0000 7fd09a000640 1 --2- 172.21.15.77:0/797441792 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fd07c0a2b80 0x7fd07c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:09.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.557+0000 7fd098c00640 1 --2- 172.21.15.77:0/1897283295 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563105e62bf0 0x563105e75860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:09.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.557+0000 7fd098c00640 1 -- 172.21.15.77:0/1897283295 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563105b39b90 con 0x563105e75da0 2024-09-10T09:56:09.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.558+0000 7fd098c00640 1 --2- 172.21.15.77:0/1897283295 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563105e75da0 0x563105e78190 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd080005c80 tx=0x7fd0800119a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55aace0baad111f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:09.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.558+0000 7fd086600640 1 -- 172.21.15.77:0/1897283295 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd08001e070 con 0x563105e75da0 2024-09-10T09:56:09.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.558+0000 7fd086600640 1 -- 172.21.15.77:0/1897283295 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd08000faa0 con 0x563105e75da0 2024-09-10T09:56:09.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.558+0000 7fd086600640 1 -- 172.21.15.77:0/1897283295 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd080019bc0 con 0x563105e75da0 2024-09-10T09:56:09.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.559+0000 7fd09ebda740 1 -- 172.21.15.77:0/1897283295 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563105e75da0 msgr2=0x563105e78190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:09.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.559+0000 7fd09ebda740 1 --2- 172.21.15.77:0/1897283295 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563105e75da0 0x563105e78190 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd080005c80 tx=0x7fd0800119a0 comp rx=0 tx=0).stop 2024-09-10T09:56:09.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.559+0000 7fd09a000640 1 --2- 172.21.15.77:0/797441792 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fd07c0a2b80 0x7fd07c0a5040 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd08c005fd0 tx=0x7fd08c012040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:09.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.560+0000 7fd09ebda740 1 -- 172.21.15.77:0/1897283295 shutdown_connections 2024-09-10T09:56:09.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.560+0000 7fd09ebda740 1 --2- 172.21.15.77:0/1897283295 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563105e75da0 0x563105e78190 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:09.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.560+0000 7fd09ebda740 1 --2- 172.21.15.77:0/1897283295 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563105e62bf0 0x563105e75860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:09.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.560+0000 7fd09ebda740 1 --2- 172.21.15.77:0/1897283295 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563105e622d0 0x563105e626b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:09.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.560+0000 7fd09ebda740 1 -- 172.21.15.77:0/1897283295 >> 172.21.15.77:0/1897283295 conn(0x563105e5e940 msgr2=0x563105e5ed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:09.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.560+0000 7fd09ebda740 1 -- 172.21.15.77:0/1897283295 shutdown_connections 2024-09-10T09:56:09.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.560+0000 7fd09ebda740 1 -- 172.21.15.77:0/1897283295 wait complete. 2024-09-10T09:56:09.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.560+0000 7fd09ebda740 1 -- start start 2024-09-10T09:56:09.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.561+0000 7fd09ebda740 1 Processor -- start 2024-09-10T09:56:09.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.561+0000 7fd09ebda740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563105e622d0 0x563105e72d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:09.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.561+0000 7fd09ebda740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563105e62bf0 0x563105e73260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:09.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.561+0000 7fd09ebda740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563105e75da0 0x563105e737a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:09.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.561+0000 7fd09ebda740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x563105e78910 con 0x563105e622d0 2024-09-10T09:56:09.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.561+0000 7fd09ebda740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x563105d82e60 con 0x563105e75da0 2024-09-10T09:56:09.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.561+0000 7fd09ebda740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x563105e78a90 con 0x563105e62bf0 2024-09-10T09:56:09.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.561+0000 7fd09ebda740 10 client.0 _ll_register_callbacks cb 0x563105d87cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:56:09.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.562+0000 7fd09a000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563105e75da0 0x563105e737a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:09.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.562+0000 7fd09a000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563105e75da0 0x563105e737a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:51074/0 (socket says 172.21.15.77:51074) 2024-09-10T09:56:09.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.562+0000 7fd09a000640 1 -- 172.21.15.77:0/2318529521 learned_addr learned my addr 172.21.15.77:0/2318529521 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:09.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.562+0000 7fd098c00640 1 --2- 172.21.15.77:0/2318529521 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563105e62bf0 0x563105e73260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:09.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.562+0000 7fd099600640 1 --2- 172.21.15.77:0/2318529521 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563105e622d0 0x563105e72d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:09.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.562+0000 7fd09a000640 1 -- 172.21.15.77:0/2318529521 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563105e62bf0 msgr2=0x563105e73260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:09.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.562+0000 7fd09a000640 1 --2- 172.21.15.77:0/2318529521 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563105e62bf0 0x563105e73260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:09.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.562+0000 7fd09a000640 1 -- 172.21.15.77:0/2318529521 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563105e622d0 msgr2=0x563105e72d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:09.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.562+0000 7fd09a000640 1 --2- 172.21.15.77:0/2318529521 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563105e622d0 0x563105e72d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:09.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.562+0000 7fd09a000640 1 -- 172.21.15.77:0/2318529521 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563105d8e990 con 0x563105e75da0 2024-09-10T09:56:09.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.562+0000 7fd099600640 1 --2- 172.21.15.77:0/2318529521 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563105e622d0 0x563105e72d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:56:09.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.563+0000 7fd09a000640 1 --2- 172.21.15.77:0/2318529521 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563105e75da0 0x563105e737a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd08c01d6e0 tx=0x7fd08c004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d150b2d8cced96 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:09.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.563+0000 7fd07aa00640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T09:56:09.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.563+0000 7fd07aa00640 1 -- 172.21.15.77:0/2318529521 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd08c003e40 con 0x563105e75da0 2024-09-10T09:56:09.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.563+0000 7fd07aa00640 1 -- 172.21.15.77:0/2318529521 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fd08c004050 con 0x563105e75da0 2024-09-10T09:56:09.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.563+0000 7fd09ebda740 1 -- 172.21.15.77:0/2318529521 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd08000faa0 con 0x563105e75da0 2024-09-10T09:56:09.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.563+0000 7fd07aa00640 1 -- 172.21.15.77:0/2318529521 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd08c004250 con 0x563105e75da0 2024-09-10T09:56:09.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.563+0000 7fd09ebda740 10 client.25565 fetch_fsmap learned FSMap version 29 2024-09-10T09:56:09.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.563+0000 7fd09ebda740 1 -- 172.21.15.77:0/2318529521 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563105e846f0 con 0x563105e75da0 2024-09-10T09:56:09.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.564+0000 7fd07aa00640 1 -- 172.21.15.77:0/2318529521 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd08000faa0 con 0x563105e75da0 2024-09-10T09:56:09.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.564+0000 7fd07aa00640 1 -- 172.21.15.77:0/2318529521 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fd08c00ec70 con 0x563105e75da0 2024-09-10T09:56:09.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.564+0000 7fd09ebda740 10 client.25565 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T09:56:09.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.564+0000 7fd09ebda740 10 client.25565 resolve_mds: validated gid 24563 aka daemon mds.b 2024-09-10T09:56:09.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.564+0000 7fd09ebda740 20 client.25565 populate_metadata read hostname 'smithi077' 2024-09-10T09:56:09.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.564+0000 7fd09ebda740 1 --2- 172.21.15.77:0/2318529521 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x563105e85b50 0x563105e87fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:09.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.564+0000 7fd09ebda740 4 client.25565 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:56:09.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.564+0000 7fd09ebda740 1 -- 172.21.15.77:0/2318529521 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563105d6b310 con 0x563105e85b50 2024-09-10T09:56:09.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.565+0000 7fd099600640 1 --2- 172.21.15.77:0/2318529521 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x563105e85b50 0x563105e87fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:09.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.565+0000 7fd07aa00640 10 client.25565 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T09:56:09.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.565+0000 7fd099600640 1 --2- 172.21.15.77:0/2318529521 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x563105e85b50 0x563105e87fb0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97026bb263d1df8a server_cookie=648ab4488a5ae5d5 in_seq=0 out_seq=0 2024-09-10T09:56:09.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.566+0000 7fd07aa00640 1 -- 172.21.15.77:0/2318529521 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563105d6b310 con 0x563105e85b50 2024-09-10T09:56:09.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.566+0000 7fd07aa00640 10 client.25565 handle_command_reply: tid=0 2024-09-10T09:56:09.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.566+0000 7fd09ebda740 1 client.25565 shutdown 2024-09-10T09:56:09.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.567+0000 7fd09ebda740 1 -- 172.21.15.77:0/2318529521 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563105e75da0 msgr2=0x563105e737a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:09.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.567+0000 7fd09ebda740 1 --2- 172.21.15.77:0/2318529521 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563105e75da0 0x563105e737a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd08c01d6e0 tx=0x7fd08c004ab0 comp rx=0 tx=0).stop 2024-09-10T09:56:09.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.567+0000 7fd09ebda740 1 -- 172.21.15.77:0/2318529521 shutdown_connections 2024-09-10T09:56:09.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.567+0000 7fd09ebda740 1 --2- 172.21.15.77:0/2318529521 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x563105e85b50 0x563105e87fb0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:09.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.567+0000 7fd09ebda740 1 --2- 172.21.15.77:0/2318529521 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563105e75da0 0x563105e737a0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:09.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.567+0000 7fd09ebda740 1 --2- 172.21.15.77:0/2318529521 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563105e62bf0 0x563105e73260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:09.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.567+0000 7fd07aa00640 0 client.25565 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T09:56:09.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.567+0000 7fd09ebda740 1 --2- 172.21.15.77:0/2318529521 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563105e622d0 0x563105e72d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:09.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.567+0000 7fd09ebda740 1 -- 172.21.15.77:0/2318529521 >> 172.21.15.77:0/2318529521 conn(0x563105e5e940 msgr2=0x563105e5dc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:09.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.567+0000 7fd09ebda740 1 -- 172.21.15.77:0/2318529521 shutdown_connections 2024-09-10T09:56:09.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.567+0000 7fd09ebda740 1 -- 172.21.15.77:0/2318529521 wait complete. 2024-09-10T09:56:09.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.568+0000 7fd09ebda740 20 client.25565 trim_cache size 0 max 16384 2024-09-10T09:56:09.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.587+0000 7fd09ebda740 1 Processor -- start 2024-09-10T09:56:09.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.587+0000 7fd09ebda740 1 -- start start 2024-09-10T09:56:09.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.588+0000 7fd09ebda740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563105e622d0 0x563105e5da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:09.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.588+0000 7fd09ebda740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563105e62bf0 0x563105e8d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:09.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.588+0000 7fd09ebda740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563105e75da0 0x563105e8ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:09.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.588+0000 7fd09ebda740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd07400fd00 con 0x563105e62bf0 2024-09-10T09:56:09.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.588+0000 7fd09ebda740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd074011840 con 0x563105e622d0 2024-09-10T09:56:09.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.588+0000 7fd09ebda740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd0740103d0 con 0x563105e75da0 2024-09-10T09:56:09.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.588+0000 7fd098c00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563105e62bf0 0x563105e8d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:09.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.588+0000 7fd099600640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563105e622d0 0x563105e5da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:09.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.588+0000 7fd098c00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563105e62bf0 0x563105e8d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34086/0 (socket says 172.21.15.77:34086) 2024-09-10T09:56:09.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.588+0000 7fd098c00640 1 -- 172.21.15.77:0/2625707668 learned_addr learned my addr 172.21.15.77:0/2625707668 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:09.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.588+0000 7fd09a000640 1 --2- 172.21.15.77:0/2625707668 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563105e75da0 0x563105e8ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:09.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.589+0000 7fd09a000640 1 -- 172.21.15.77:0/2625707668 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563105e622d0 msgr2=0x563105e5da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:09.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.589+0000 7fd09a000640 1 --2- 172.21.15.77:0/2625707668 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563105e622d0 0x563105e5da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:09.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.589+0000 7fd09a000640 1 -- 172.21.15.77:0/2625707668 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563105e62bf0 msgr2=0x563105e8d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:09.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.589+0000 7fd09a000640 1 --2- 172.21.15.77:0/2625707668 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563105e62bf0 0x563105e8d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:09.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.589+0000 7fd09a000640 1 -- 172.21.15.77:0/2625707668 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563105e63850 con 0x563105e75da0 2024-09-10T09:56:09.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.589+0000 7fd09a000640 1 --2- 172.21.15.77:0/2625707668 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563105e75da0 0x563105e8ddc0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd08c01d6b0 tx=0x7fd08c01a7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a9f4eb3b75338fe server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:09.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.590+0000 7fd085c00640 1 -- 172.21.15.77:0/2625707668 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd08c00fea0 con 0x563105e75da0 2024-09-10T09:56:09.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.590+0000 7fd085c00640 1 -- 172.21.15.77:0/2625707668 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd08c01a990 con 0x563105e75da0 2024-09-10T09:56:09.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.590+0000 7fd085c00640 1 -- 172.21.15.77:0/2625707668 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd08c00e4a0 con 0x563105e75da0 2024-09-10T09:56:09.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.591+0000 7fd09ebda740 1 -- 172.21.15.77:0/2625707668 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563105e75da0 msgr2=0x563105e8ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:09.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.591+0000 7fd09ebda740 1 --2- 172.21.15.77:0/2625707668 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563105e75da0 0x563105e8ddc0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd08c01d6b0 tx=0x7fd08c01a7a0 comp rx=0 tx=0).stop 2024-09-10T09:56:09.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.591+0000 7fd09ebda740 1 -- 172.21.15.77:0/2625707668 shutdown_connections 2024-09-10T09:56:09.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.591+0000 7fd09ebda740 1 --2- 172.21.15.77:0/2625707668 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563105e75da0 0x563105e8ddc0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:09.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.591+0000 7fd09ebda740 1 --2- 172.21.15.77:0/2625707668 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563105e62bf0 0x563105e8d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:09.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.591+0000 7fd09ebda740 1 --2- 172.21.15.77:0/2625707668 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563105e622d0 0x563105e5da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:09.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.591+0000 7fd09ebda740 1 -- 172.21.15.77:0/2625707668 >> 172.21.15.77:0/2625707668 conn(0x563105e5e940 msgr2=0x563105e64590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:09.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.591+0000 7fd09ebda740 1 -- 172.21.15.77:0/2625707668 shutdown_connections 2024-09-10T09:56:09.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.591+0000 7fd09ebda740 1 -- 172.21.15.77:0/2625707668 wait complete. 2024-09-10T09:56:09.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.592+0000 7fd09ebda740 1 -- start start 2024-09-10T09:56:09.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.592+0000 7fd09ebda740 1 Processor -- start 2024-09-10T09:56:09.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.592+0000 7fd09ebda740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563105e622d0 0x563105e7ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:09.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.592+0000 7fd09ebda740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563105e62bf0 0x563105e7b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:09.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.593+0000 7fd09ebda740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563105e75da0 0x563105e7b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:09.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.593+0000 7fd09ebda740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd074011170 con 0x563105e75da0 2024-09-10T09:56:09.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.593+0000 7fd09ebda740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd074010aa0 con 0x563105e62bf0 2024-09-10T09:56:09.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.593+0000 7fd09ebda740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd07400dce0 con 0x563105e622d0 2024-09-10T09:56:09.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.593+0000 7fd09ebda740 10 client.0 _ll_register_callbacks cb 0x7fd08c009e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:56:09.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.593+0000 7fd09a000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563105e75da0 0x563105e7b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:09.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.593+0000 7fd09a000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563105e75da0 0x563105e7b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34098/0 (socket says 172.21.15.77:34098) 2024-09-10T09:56:09.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.593+0000 7fd09a000640 1 -- 172.21.15.77:0/4204555430 learned_addr learned my addr 172.21.15.77:0/4204555430 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:09.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.593+0000 7fd099600640 1 --2- 172.21.15.77:0/4204555430 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563105e622d0 0x563105e7ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:09.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.593+0000 7fd098c00640 1 --2- 172.21.15.77:0/4204555430 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563105e62bf0 0x563105e7b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:09.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.593+0000 7fd099600640 1 -- 172.21.15.77:0/4204555430 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563105e62bf0 msgr2=0x563105e7b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:09.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.593+0000 7fd099600640 1 --2- 172.21.15.77:0/4204555430 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563105e62bf0 0x563105e7b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:09.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.593+0000 7fd099600640 1 -- 172.21.15.77:0/4204555430 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563105e75da0 msgr2=0x563105e7b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:09.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.593+0000 7fd099600640 1 --2- 172.21.15.77:0/4204555430 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563105e75da0 0x563105e7b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:09.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.593+0000 7fd099600640 1 -- 172.21.15.77:0/4204555430 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563105ef0860 con 0x563105e622d0 2024-09-10T09:56:09.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.594+0000 7fd09a000640 1 --2- 172.21.15.77:0/4204555430 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563105e75da0 0x563105e7b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:56:09.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.594+0000 7fd099600640 1 --2- 172.21.15.77:0/4204555430 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563105e622d0 0x563105e7ab20 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd088031000 tx=0x7fd08802fef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aaf199b78b821214 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:09.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.594+0000 7fd07aa00640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-10T09:56:09.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.594+0000 7fd07aa00640 1 -- 172.21.15.77:0/4204555430 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0880d4070 con 0x563105e622d0 2024-09-10T09:56:09.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.594+0000 7fd09ebda740 1 -- 172.21.15.77:0/4204555430 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd08c01a990 con 0x563105e622d0 2024-09-10T09:56:09.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.594+0000 7fd07aa00640 1 -- 172.21.15.77:0/4204555430 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fd0880ecbc0 con 0x563105e622d0 2024-09-10T09:56:09.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.595+0000 7fd07aa00640 1 -- 172.21.15.77:0/4204555430 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd0880bb810 con 0x563105e622d0 2024-09-10T09:56:09.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.595+0000 7fd09ebda740 10 client.25537 fetch_fsmap learned FSMap version 29 2024-09-10T09:56:09.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.595+0000 7fd09ebda740 1 -- 172.21.15.77:0/4204555430 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563105ef1b80 con 0x563105e622d0 2024-09-10T09:56:09.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.595+0000 7fd07aa00640 1 -- 172.21.15.77:0/4204555430 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd08c01a990 con 0x563105e622d0 2024-09-10T09:56:09.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.595+0000 7fd07aa00640 1 -- 172.21.15.77:0/4204555430 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fd08803a030 con 0x563105e622d0 2024-09-10T09:56:09.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.595+0000 7fd09ebda740 10 client.25537 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T09:56:09.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.595+0000 7fd09ebda740 10 client.25537 resolve_mds: validated gid 24563 aka daemon mds.b 2024-09-10T09:56:09.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.595+0000 7fd09ebda740 20 client.25537 populate_metadata read hostname 'smithi077' 2024-09-10T09:56:09.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.595+0000 7fd09ebda740 1 --2- 172.21.15.77:0/4204555430 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x563105ef22c0 0x563105ef46e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:09.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.595+0000 7fd09ebda740 4 client.25537 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-09-10T09:56:09.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.595+0000 7fd09ebda740 1 -- 172.21.15.77:0/4204555430 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x563105dd5c60 con 0x563105ef22c0 2024-09-10T09:56:09.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.596+0000 7fd098c00640 1 --2- 172.21.15.77:0/4204555430 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x563105ef22c0 0x563105ef46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:09.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.596+0000 7fd098c00640 1 --2- 172.21.15.77:0/4204555430 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x563105ef22c0 0x563105ef46e0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0403ffd1357701 server_cookie=384a84dce7883bec in_seq=0 out_seq=0 2024-09-10T09:56:09.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.596+0000 7fd07aa00640 10 client.25537 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T09:56:09.600 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:56:09.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.608+0000 7fd07aa00640 1 -- 172.21.15.77:0/4204555430 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x563105dd5c60 con 0x563105ef22c0 2024-09-10T09:56:09.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.608+0000 7fd07aa00640 10 client.25537 handle_command_reply: tid=0 2024-09-10T09:56:09.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.608+0000 7fd09ebda740 1 client.25537 shutdown 2024-09-10T09:56:09.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.608+0000 7fd09ebda740 1 -- 172.21.15.77:0/4204555430 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563105e622d0 msgr2=0x563105e7ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:09.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.608+0000 7fd09ebda740 1 --2- 172.21.15.77:0/4204555430 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563105e622d0 0x563105e7ab20 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd088031000 tx=0x7fd08802fef0 comp rx=0 tx=0).stop 2024-09-10T09:56:09.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.608+0000 7fd09ebda740 1 -- 172.21.15.77:0/4204555430 shutdown_connections 2024-09-10T09:56:09.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.608+0000 7fd09ebda740 1 --2- 172.21.15.77:0/4204555430 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x563105ef22c0 0x563105ef46e0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:09.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.608+0000 7fd09ebda740 1 --2- 172.21.15.77:0/4204555430 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563105e75da0 0x563105e7b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:09.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.608+0000 7fd09ebda740 1 --2- 172.21.15.77:0/4204555430 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563105e62bf0 0x563105e7b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:09.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.608+0000 7fd09ebda740 1 --2- 172.21.15.77:0/4204555430 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563105e622d0 0x563105e7ab20 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:09.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.608+0000 7fd09ebda740 1 -- 172.21.15.77:0/4204555430 >> 172.21.15.77:0/4204555430 conn(0x563105e5e940 msgr2=0x563105e72ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:09.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.608+0000 7fd07aa00640 0 client.25537 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T09:56:09.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.609+0000 7fd09ebda740 1 -- 172.21.15.77:0/4204555430 shutdown_connections 2024-09-10T09:56:09.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.609+0000 7fd09ebda740 1 -- 172.21.15.77:0/4204555430 wait complete. 2024-09-10T09:56:09.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.609+0000 7fd09ebda740 20 client.25537 trim_cache size 0 max 16384 2024-09-10T09:56:09.611 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-10T09:56:09.611 INFO:teuthology.orchestra.run.smithi077.stdout: "epoch": 29, 2024-09-10T09:56:09.611 INFO:teuthology.orchestra.run.smithi077.stdout: "leader": 24563, 2024-09-10T09:56:09.611 INFO:teuthology.orchestra.run.smithi077.stdout: "set_version": 2, 2024-09-10T09:56:09.612 INFO:teuthology.orchestra.run.smithi077.stdout: "sets": { 2024-09-10T09:56:09.612 INFO:teuthology.orchestra.run.smithi077.stdout: "c4ed4a5a": { 2024-09-10T09:56:09.612 INFO:teuthology.orchestra.run.smithi077.stdout: "version": 2, 2024-09-10T09:56:09.612 INFO:teuthology.orchestra.run.smithi077.stdout: "age_ref": 0.0, 2024-09-10T09:56:09.612 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-10T09:56:09.612 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "QUIESCED", 2024-09-10T09:56:09.612 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-10T09:56:09.612 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T09:56:09.612 INFO:teuthology.orchestra.run.smithi077.stdout: "timeout": 90.0, 2024-09-10T09:56:09.612 INFO:teuthology.orchestra.run.smithi077.stdout: "expiration": 150.0, 2024-09-10T09:56:09.612 INFO:teuthology.orchestra.run.smithi077.stdout: "members": { 2024-09-10T09:56:09.612 INFO:teuthology.orchestra.run.smithi077.stdout: "file:/": { 2024-09-10T09:56:09.612 INFO:teuthology.orchestra.run.smithi077.stdout: "excluded": false, 2024-09-10T09:56:09.613 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-10T09:56:09.613 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "QUIESCED", 2024-09-10T09:56:09.613 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-10T09:56:09.613 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T09:56:09.613 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T09:56:09.613 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T09:56:09.613 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T09:56:09.613 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T09:56:09.613 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-10T09:56:09.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.610+0000 7fd09c000640 1 -- 172.21.15.77:0/797441792 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fd07c0a2b80 msgr2=0x7fd07c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:09.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.610+0000 7fd09c000640 1 --2- 172.21.15.77:0/797441792 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fd07c0a2b80 0x7fd07c0a5040 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd08c005fd0 tx=0x7fd08c012040 comp rx=0 tx=0).stop 2024-09-10T09:56:09.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.610+0000 7fd09c000640 1 -- 172.21.15.77:0/797441792 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd094104010 msgr2=0x7fd09419a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:09.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.610+0000 7fd09c000640 1 --2- 172.21.15.77:0/797441792 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd094104010 0x7fd09419a630 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fd088035040 tx=0x7fd088033b60 comp rx=0 tx=0).stop 2024-09-10T09:56:09.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.610+0000 7fd09c000640 1 -- 172.21.15.77:0/797441792 shutdown_connections 2024-09-10T09:56:09.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.610+0000 7fd09c000640 1 --2- 172.21.15.77:0/797441792 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fd07c0a2b80 0x7fd07c0a5040 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:09.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.610+0000 7fd09c000640 1 --2- 172.21.15.77:0/797441792 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd094109140 0x7fd09419ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:09.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.610+0000 7fd09c000640 1 --2- 172.21.15.77:0/797441792 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0941049c0 0x7fd09419ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:09.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.610+0000 7fd09c000640 1 --2- 172.21.15.77:0/797441792 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd094104010 0x7fd09419a630 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:09.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.610+0000 7fd09c000640 1 -- 172.21.15.77:0/797441792 >> 172.21.15.77:0/797441792 conn(0x7fd0940ffa50 msgr2=0x7fd094100e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:09.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.610+0000 7fd09c000640 1 -- 172.21.15.77:0/797441792 shutdown_connections 2024-09-10T09:56:09.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:09.610+0000 7fd09c000640 1 -- 172.21.15.77:0/797441792 wait complete. 2024-09-10T09:56:10.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.488+0000 7fcca9000640 1 -- 172.21.15.77:0/9795309 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcca41047e0 msgr2=0x7fcca4108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:10.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.488+0000 7fcca9000640 1 --2- 172.21.15.77:0/9795309 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcca41047e0 0x7fcca4108830 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fcc90009900 tx=0x7fcc9002a3f0 comp rx=0 tx=0).stop 2024-09-10T09:56:10.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.488+0000 7fcca9000640 1 -- 172.21.15.77:0/9795309 shutdown_connections 2024-09-10T09:56:10.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.488+0000 7fcca9000640 1 --2- 172.21.15.77:0/9795309 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcca4108f60 0x7fcca4110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:10.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.488+0000 7fcca9000640 1 --2- 172.21.15.77:0/9795309 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcca41047e0 0x7fcca4108830 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:10.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.488+0000 7fcca9000640 1 --2- 172.21.15.77:0/9795309 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcca4103e30 0x7fcca4104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:10.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.488+0000 7fcca9000640 1 -- 172.21.15.77:0/9795309 >> 172.21.15.77:0/9795309 conn(0x7fcca40ff910 msgr2=0x7fcca4101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:10.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.488+0000 7fcca9000640 1 -- 172.21.15.77:0/9795309 shutdown_connections 2024-09-10T09:56:10.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.488+0000 7fcca9000640 1 -- 172.21.15.77:0/9795309 wait complete. 2024-09-10T09:56:10.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.489+0000 7fcca9000640 1 Processor -- start 2024-09-10T09:56:10.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.489+0000 7fcca9000640 1 -- start start 2024-09-10T09:56:10.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.489+0000 7fcca9000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcca4103e30 0x7fcca419a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:10.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.489+0000 7fcca9000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcca41047e0 0x7fcca419aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:10.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.490+0000 7fcca9000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcca4108f60 0x7fcca419ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:10.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.490+0000 7fcca9000640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fcca4113fc0 con 0x7fcca4108f60 2024-09-10T09:56:10.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.490+0000 7fcca9000640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fcca4113e40 con 0x7fcca4103e30 2024-09-10T09:56:10.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.490+0000 7fcca9000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcca4114140 con 0x7fcca41047e0 2024-09-10T09:56:10.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.490+0000 7fcca3400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcca4103e30 0x7fcca419a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:10.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.490+0000 7fcca3400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcca4103e30 0x7fcca419a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:51102/0 (socket says 172.21.15.77:51102) 2024-09-10T09:56:10.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.490+0000 7fcca3400640 1 -- 172.21.15.77:0/1841122749 learned_addr learned my addr 172.21.15.77:0/1841122749 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:10.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.490+0000 7fcca3e00640 1 --2- 172.21.15.77:0/1841122749 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcca4108f60 0x7fcca419ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:10.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.490+0000 7fcca2a00640 1 --2- 172.21.15.77:0/1841122749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcca41047e0 0x7fcca419aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:10.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.490+0000 7fcca3e00640 1 -- 172.21.15.77:0/1841122749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcca41047e0 msgr2=0x7fcca419aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:10.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.490+0000 7fcca3e00640 1 --2- 172.21.15.77:0/1841122749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcca41047e0 0x7fcca419aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:10.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.490+0000 7fcca3e00640 1 -- 172.21.15.77:0/1841122749 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcca4103e30 msgr2=0x7fcca419a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:10.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.490+0000 7fcca3e00640 1 --2- 172.21.15.77:0/1841122749 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcca4103e30 0x7fcca419a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:10.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.491+0000 7fcca3e00640 1 -- 172.21.15.77:0/1841122749 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcca419f470 con 0x7fcca4108f60 2024-09-10T09:56:10.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.491+0000 7fcca3e00640 1 --2- 172.21.15.77:0/1841122749 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcca4108f60 0x7fcca419ed90 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c009fd0 tx=0x7fcc9c00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:10.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.491+0000 7fcc9be00640 1 -- 172.21.15.77:0/1841122749 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9c00ce20 con 0x7fcca4108f60 2024-09-10T09:56:10.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.491+0000 7fcc9be00640 1 -- 172.21.15.77:0/1841122749 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc9c005020 con 0x7fcca4108f60 2024-09-10T09:56:10.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.491+0000 7fcca9000640 1 -- 172.21.15.77:0/1841122749 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcca419f760 con 0x7fcca4108f60 2024-09-10T09:56:10.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.491+0000 7fcc9be00640 1 -- 172.21.15.77:0/1841122749 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9c010720 con 0x7fcca4108f60 2024-09-10T09:56:10.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.491+0000 7fcca9000640 1 -- 172.21.15.77:0/1841122749 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcca41a7040 con 0x7fcca4108f60 2024-09-10T09:56:10.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.495+0000 7fccad06c740 1 Processor -- start 2024-09-10T09:56:10.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.496+0000 7fcc9be00640 1 -- 172.21.15.77:0/1841122749 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fcc9c005540 con 0x7fcca4108f60 2024-09-10T09:56:10.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.496+0000 7fccad06c740 1 -- start start 2024-09-10T09:56:10.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.496+0000 7fccad06c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555df41b4e40 0x555df41b5220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:10.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.496+0000 7fccad06c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555df41b5760 0x555df41bfae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:10.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.496+0000 7fccad06c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555df41c0020 0x555df41c2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:10.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.496+0000 7fccad06c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x555df411b3e0 con 0x555df41c0020 2024-09-10T09:56:10.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.496+0000 7fccad06c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x555df40c2d80 con 0x555df41b5760 2024-09-10T09:56:10.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.496+0000 7fccad06c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x555df41173a0 con 0x555df41b4e40 2024-09-10T09:56:10.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.497+0000 7fcca3400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555df41b5760 0x555df41bfae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:10.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.497+0000 7fcca2a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555df41c0020 0x555df41c2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:10.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.497+0000 7fcca3400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555df41b5760 0x555df41bfae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:51108/0 (socket says 172.21.15.77:51108) 2024-09-10T09:56:10.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.497+0000 7fcca3400640 1 -- 172.21.15.77:0/2535720464 learned_addr learned my addr 172.21.15.77:0/2535720464 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:10.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.497+0000 7fcca2a00640 1 --2- 172.21.15.77:0/2535720464 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555df41b4e40 0x555df41b5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:10.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.497+0000 7fcca2a00640 1 -- 172.21.15.77:0/2535720464 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555df41b4e40 msgr2=0x555df41b5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:10.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.497+0000 7fcca2a00640 1 --2- 172.21.15.77:0/2535720464 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555df41b4e40 0x555df41b5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:10.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.497+0000 7fcca2a00640 1 -- 172.21.15.77:0/2535720464 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555df41b5760 msgr2=0x555df41bfae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:10.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.497+0000 7fcca2a00640 1 --2- 172.21.15.77:0/2535720464 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555df41b5760 0x555df41bfae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:10.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.497+0000 7fcca2a00640 1 -- 172.21.15.77:0/2535720464 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555df40e1ae0 con 0x555df41c0020 2024-09-10T09:56:10.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.497+0000 7fcc9be00640 1 --2- 172.21.15.77:0/1841122749 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fcc880a2b80 0x7fcc880a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:10.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.498+0000 7fcc9be00640 1 -- 172.21.15.77:0/1841122749 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fcc9c0cfab0 con 0x7fcca4108f60 2024-09-10T09:56:10.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.498+0000 7fcca2a00640 1 --2- 172.21.15.77:0/2535720464 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555df41c0020 0x555df41c2410 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fcc90005940 tx=0x7fcc900314e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae0a8d7d35769cd8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:10.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.498+0000 7fcca3400640 1 --2- 172.21.15.77:0/1841122749 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fcc880a2b80 0x7fcc880a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:10.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.498+0000 7fcc98c00640 1 -- 172.21.15.77:0/2535720464 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc90035070 con 0x555df41c0020 2024-09-10T09:56:10.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.498+0000 7fcc98c00640 1 -- 172.21.15.77:0/2535720464 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc900319d0 con 0x555df41c0020 2024-09-10T09:56:10.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.498+0000 7fcc98c00640 1 -- 172.21.15.77:0/2535720464 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc90004770 con 0x555df41c0020 2024-09-10T09:56:10.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.499+0000 7fccad06c740 1 -- 172.21.15.77:0/2535720464 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555df41c0020 msgr2=0x555df41c2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:10.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.499+0000 7fcca3400640 1 --2- 172.21.15.77:0/1841122749 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fcc880a2b80 0x7fcc880a5040 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x555df41b2230 tx=0x7fcc9400b4a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:10.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.499+0000 7fccad06c740 1 --2- 172.21.15.77:0/2535720464 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555df41c0020 0x555df41c2410 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fcc90005940 tx=0x7fcc900314e0 comp rx=0 tx=0).stop 2024-09-10T09:56:10.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.499+0000 7fccad06c740 1 -- 172.21.15.77:0/2535720464 shutdown_connections 2024-09-10T09:56:10.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.499+0000 7fccad06c740 1 --2- 172.21.15.77:0/2535720464 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555df41c0020 0x555df41c2410 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:10.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.499+0000 7fccad06c740 1 --2- 172.21.15.77:0/2535720464 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555df41b5760 0x555df41bfae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:10.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.499+0000 7fccad06c740 1 --2- 172.21.15.77:0/2535720464 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555df41b4e40 0x555df41b5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:10.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.499+0000 7fccad06c740 1 -- 172.21.15.77:0/2535720464 >> 172.21.15.77:0/2535720464 conn(0x555df41b13d0 msgr2=0x555df41b17b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:10.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.499+0000 7fccad06c740 1 -- 172.21.15.77:0/2535720464 shutdown_connections 2024-09-10T09:56:10.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.499+0000 7fccad06c740 1 -- 172.21.15.77:0/2535720464 wait complete. 2024-09-10T09:56:10.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.500+0000 7fccad06c740 1 -- start start 2024-09-10T09:56:10.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.500+0000 7fccad06c740 1 Processor -- start 2024-09-10T09:56:10.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.501+0000 7fccad06c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555df41b4e40 0x555df41bcfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:10.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.501+0000 7fccad06c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555df41b5760 0x555df41bd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:10.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.501+0000 7fccad06c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555df41c0020 0x555df41bda30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:10.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.501+0000 7fccad06c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x555df41c2b90 con 0x555df41b4e40 2024-09-10T09:56:10.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.501+0000 7fccad06c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x555df40d60f0 con 0x555df41c0020 2024-09-10T09:56:10.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.501+0000 7fccad06c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x555df41c2d10 con 0x555df41b5760 2024-09-10T09:56:10.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.501+0000 7fccad06c740 10 client.0 _ll_register_callbacks cb 0x555df40bdbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:56:10.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.501+0000 7fcca2a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555df41b5760 0x555df41bd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:10.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.501+0000 7fcca3400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555df41b4e40 0x555df41bcfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:10.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.501+0000 7fcca2a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555df41b5760 0x555df41bd4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:53620/0 (socket says 172.21.15.77:53620) 2024-09-10T09:56:10.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.501+0000 7fcca3e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555df41c0020 0x555df41bda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:10.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.501+0000 7fcca2a00640 1 -- 172.21.15.77:0/1473277968 learned_addr learned my addr 172.21.15.77:0/1473277968 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:10.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.502+0000 7fcca3e00640 1 -- 172.21.15.77:0/1473277968 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555df41b5760 msgr2=0x555df41bd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:10.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.502+0000 7fcca3e00640 1 --2- 172.21.15.77:0/1473277968 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555df41b5760 0x555df41bd4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:10.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.502+0000 7fcca3e00640 1 -- 172.21.15.77:0/1473277968 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555df41b4e40 msgr2=0x555df41bcfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:10.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.502+0000 7fcca3e00640 1 --2- 172.21.15.77:0/1473277968 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555df41b4e40 0x555df41bcfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:10.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.502+0000 7fcca3e00640 1 -- 172.21.15.77:0/1473277968 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555df3e8c3c0 con 0x555df41c0020 2024-09-10T09:56:10.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.502+0000 7fcca2a00640 1 --2- 172.21.15.77:0/1473277968 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555df41b5760 0x555df41bd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:56:10.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.502+0000 7fcca3e00640 1 --2- 172.21.15.77:0/1473277968 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555df41c0020 0x555df41bda30 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c0cd1c0 tx=0x7fcc9c09c760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40a77d74e59e2a17 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:10.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.502+0000 7fcc8d600640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T09:56:10.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.502+0000 7fcc8d600640 1 -- 172.21.15.77:0/1473277968 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9c0a4b90 con 0x555df41c0020 2024-09-10T09:56:10.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.502+0000 7fcc8d600640 1 -- 172.21.15.77:0/1473277968 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fcc9c0a4700 con 0x555df41c0020 2024-09-10T09:56:10.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.502+0000 7fccad06c740 1 -- 172.21.15.77:0/1473277968 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc900319d0 con 0x555df41c0020 2024-09-10T09:56:10.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.502+0000 7fcc8d600640 1 -- 172.21.15.77:0/1473277968 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcc9c0a4900 con 0x555df41c0020 2024-09-10T09:56:10.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.502+0000 7fccad06c740 10 client.25583 fetch_fsmap learned FSMap version 29 2024-09-10T09:56:10.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.502+0000 7fccad06c740 1 -- 172.21.15.77:0/1473277968 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555df41cecd0 con 0x555df41c0020 2024-09-10T09:56:10.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.503+0000 7fcc8d600640 1 -- 172.21.15.77:0/1473277968 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc900319d0 con 0x555df41c0020 2024-09-10T09:56:10.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.503+0000 7fcc8d600640 1 -- 172.21.15.77:0/1473277968 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fcc9c09f070 con 0x555df41c0020 2024-09-10T09:56:10.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.503+0000 7fccad06c740 10 client.25583 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T09:56:10.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.503+0000 7fccad06c740 10 client.25583 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-10T09:56:10.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.503+0000 7fccad06c740 20 client.25583 populate_metadata read hostname 'smithi077' 2024-09-10T09:56:10.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.503+0000 7fccad06c740 1 --2- 172.21.15.77:0/1473277968 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x555df41d0130 0x555df41d2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:10.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.503+0000 7fccad06c740 4 client.25583 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:56:10.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.503+0000 7fccad06c740 1 -- 172.21.15.77:0/1473277968 --> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555df4128950 con 0x555df41d0130 2024-09-10T09:56:10.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.504+0000 7fcca3400640 1 --2- 172.21.15.77:0/1473277968 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x555df41d0130 0x555df41d2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:10.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.504+0000 7fcca3400640 1 --2- 172.21.15.77:0/1473277968 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x555df41d0130 0x555df41d2540 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=26fecdfd2910ad6e server_cookie=dcb2ebb75a5c480 in_seq=0 out_seq=0 2024-09-10T09:56:10.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.504+0000 7fcc8d600640 10 client.25583 ms_handle_connect on v2:172.21.15.120:6836/3033280709 2024-09-10T09:56:10.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.505+0000 7fcc8d600640 1 -- 172.21.15.77:0/1473277968 <== mds.1 v2:172.21.15.120:6836/3033280709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555df4128950 con 0x555df41d0130 2024-09-10T09:56:10.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.505+0000 7fcc8d600640 10 client.25583 handle_command_reply: tid=0 2024-09-10T09:56:10.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.505+0000 7fccad06c740 1 client.25583 shutdown 2024-09-10T09:56:10.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.505+0000 7fccad06c740 1 -- 172.21.15.77:0/1473277968 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555df41c0020 msgr2=0x555df41bda30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:10.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.505+0000 7fccad06c740 1 --2- 172.21.15.77:0/1473277968 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555df41c0020 0x555df41bda30 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c0cd1c0 tx=0x7fcc9c09c760 comp rx=0 tx=0).stop 2024-09-10T09:56:10.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.505+0000 7fccad06c740 1 -- 172.21.15.77:0/1473277968 shutdown_connections 2024-09-10T09:56:10.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.505+0000 7fccad06c740 1 --2- 172.21.15.77:0/1473277968 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x555df41d0130 0x555df41d2540 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:10.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.505+0000 7fccad06c740 1 --2- 172.21.15.77:0/1473277968 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555df41c0020 0x555df41bda30 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:10.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.505+0000 7fccad06c740 1 --2- 172.21.15.77:0/1473277968 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555df41b5760 0x555df41bd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:10.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.505+0000 7fccad06c740 1 --2- 172.21.15.77:0/1473277968 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555df41b4e40 0x555df41bcfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:10.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.505+0000 7fccad06c740 1 -- 172.21.15.77:0/1473277968 >> 172.21.15.77:0/1473277968 conn(0x555df41b13d0 msgr2=0x555df41b70b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:10.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.506+0000 7fcc8d600640 0 client.25583 ms_handle_reset on v2:172.21.15.120:6836/3033280709 2024-09-10T09:56:10.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.506+0000 7fccad06c740 1 -- 172.21.15.77:0/1473277968 shutdown_connections 2024-09-10T09:56:10.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.506+0000 7fccad06c740 1 -- 172.21.15.77:0/1473277968 wait complete. 2024-09-10T09:56:10.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.506+0000 7fccad06c740 20 client.25583 trim_cache size 0 max 16384 2024-09-10T09:56:10.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.525+0000 7fccad06c740 1 Processor -- start 2024-09-10T09:56:10.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.525+0000 7fccad06c740 1 -- start start 2024-09-10T09:56:10.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.525+0000 7fccad06c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555df41b4e40 0x555df41bef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:10.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.525+0000 7fccad06c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555df41b5760 0x555df41d7e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:10.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.525+0000 7fccad06c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555df41c0020 0x555df41d8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:10.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.525+0000 7fccad06c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fcca41b0f50 con 0x555df41b5760 2024-09-10T09:56:10.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.525+0000 7fccad06c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fcca41b2a90 con 0x555df41c0020 2024-09-10T09:56:10.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.525+0000 7fccad06c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcca41b1620 con 0x555df41b4e40 2024-09-10T09:56:10.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.526+0000 7fcca2a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555df41b5760 0x555df41d7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:10.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.526+0000 7fcca3400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555df41b4e40 0x555df41bef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:10.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.526+0000 7fcca2a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555df41b5760 0x555df41d7e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34146/0 (socket says 172.21.15.77:34146) 2024-09-10T09:56:10.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.526+0000 7fcca2a00640 1 -- 172.21.15.77:0/246980691 learned_addr learned my addr 172.21.15.77:0/246980691 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:10.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.526+0000 7fcca3e00640 1 --2- 172.21.15.77:0/246980691 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555df41c0020 0x555df41d8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:10.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.526+0000 7fcca3e00640 1 -- 172.21.15.77:0/246980691 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555df41b4e40 msgr2=0x555df41bef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:10.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.526+0000 7fcca3e00640 1 --2- 172.21.15.77:0/246980691 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555df41b4e40 0x555df41bef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:10.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.526+0000 7fcca3e00640 1 -- 172.21.15.77:0/246980691 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555df41b5760 msgr2=0x555df41d7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:10.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.526+0000 7fcca3e00640 1 --2- 172.21.15.77:0/246980691 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555df41b5760 0x555df41d7e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:10.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.526+0000 7fcca3e00640 1 -- 172.21.15.77:0/246980691 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555df41b4c70 con 0x555df41c0020 2024-09-10T09:56:10.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.526+0000 7fcca2a00640 1 --2- 172.21.15.77:0/246980691 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555df41b5760 0x555df41d7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:56:10.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.527+0000 7fcca3e00640 1 --2- 172.21.15.77:0/246980691 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555df41c0020 0x555df41d8350 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c09beb0 tx=0x7fcc9c09a430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8a0075b820961a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:10.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.527+0000 7fcc8fe00640 1 -- 172.21.15.77:0/246980691 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9c0b7030 con 0x555df41c0020 2024-09-10T09:56:10.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.527+0000 7fcc8fe00640 1 -- 172.21.15.77:0/246980691 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc9c07f070 con 0x555df41c0020 2024-09-10T09:56:10.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.527+0000 7fcc8fe00640 1 -- 172.21.15.77:0/246980691 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9c09b370 con 0x555df41c0020 2024-09-10T09:56:10.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.528+0000 7fccad06c740 1 -- 172.21.15.77:0/246980691 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555df41c0020 msgr2=0x555df41d8350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:10.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.528+0000 7fccad06c740 1 --2- 172.21.15.77:0/246980691 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555df41c0020 0x555df41d8350 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c09beb0 tx=0x7fcc9c09a430 comp rx=0 tx=0).stop 2024-09-10T09:56:10.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.528+0000 7fccad06c740 1 -- 172.21.15.77:0/246980691 shutdown_connections 2024-09-10T09:56:10.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.528+0000 7fccad06c740 1 --2- 172.21.15.77:0/246980691 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555df41c0020 0x555df41d8350 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:10.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.528+0000 7fccad06c740 1 --2- 172.21.15.77:0/246980691 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555df41b5760 0x555df41d7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:10.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.528+0000 7fccad06c740 1 --2- 172.21.15.77:0/246980691 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555df41b4e40 0x555df41bef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:10.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.528+0000 7fccad06c740 1 -- 172.21.15.77:0/246980691 >> 172.21.15.77:0/246980691 conn(0x555df41b13d0 msgr2=0x555df41be810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:10.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.528+0000 7fccad06c740 1 -- 172.21.15.77:0/246980691 shutdown_connections 2024-09-10T09:56:10.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.528+0000 7fccad06c740 1 -- 172.21.15.77:0/246980691 wait complete. 2024-09-10T09:56:10.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.529+0000 7fccad06c740 1 -- start start 2024-09-10T09:56:10.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.529+0000 7fccad06c740 1 Processor -- start 2024-09-10T09:56:10.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.530+0000 7fccad06c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555df41b4e40 0x555df41c8520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:10.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.530+0000 7fccad06c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555df41b5760 0x555df41b9b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:10.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.530+0000 7fccad06c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555df41c0020 0x555df41ba0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:10.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.530+0000 7fccad06c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fcca41b23c0 con 0x555df41b4e40 2024-09-10T09:56:10.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.530+0000 7fccad06c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fcca41b1cf0 con 0x555df41c0020 2024-09-10T09:56:10.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.530+0000 7fccad06c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcca41aef30 con 0x555df41b5760 2024-09-10T09:56:10.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.530+0000 7fccad06c740 10 client.0 _ll_register_callbacks cb 0x7fcc9c005830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:56:10.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.530+0000 7fcca3400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555df41b4e40 0x555df41c8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:10.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.530+0000 7fcca3400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555df41b4e40 0x555df41c8520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34158/0 (socket says 172.21.15.77:34158) 2024-09-10T09:56:10.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.530+0000 7fcca3400640 1 -- 172.21.15.77:0/751699658 learned_addr learned my addr 172.21.15.77:0/751699658 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:10.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.530+0000 7fcca3e00640 1 --2- 172.21.15.77:0/751699658 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555df41c0020 0x555df41ba0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:10.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.530+0000 7fcca2a00640 1 --2- 172.21.15.77:0/751699658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555df41b5760 0x555df41b9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:10.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.530+0000 7fcca3400640 1 -- 172.21.15.77:0/751699658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555df41b5760 msgr2=0x555df41b9b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:10.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.530+0000 7fcca3400640 1 --2- 172.21.15.77:0/751699658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555df41b5760 0x555df41b9b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:10.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.530+0000 7fcca3400640 1 -- 172.21.15.77:0/751699658 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555df41c0020 msgr2=0x555df41ba0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:10.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.530+0000 7fcca3400640 1 --2- 172.21.15.77:0/751699658 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555df41c0020 0x555df41ba0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:10.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.530+0000 7fcca3400640 1 -- 172.21.15.77:0/751699658 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555df41b0c80 con 0x555df41b4e40 2024-09-10T09:56:10.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.531+0000 7fcca3400640 1 --2- 172.21.15.77:0/751699658 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555df41b4e40 0x555df41c8520 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fcc940039c0 tx=0x7fcc940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7318d3fe79aa415f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:10.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.531+0000 7fcc8d600640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-10T09:56:10.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.531+0000 7fcc8d600640 1 -- 172.21.15.77:0/751699658 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9402aa80 con 0x555df41b4e40 2024-09-10T09:56:10.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.531+0000 7fcc8d600640 1 -- 172.21.15.77:0/751699658 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fcc9402ac20 con 0x555df41b4e40 2024-09-10T09:56:10.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.531+0000 7fccad06c740 1 -- 172.21.15.77:0/751699658 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc9c07f070 con 0x555df41b4e40 2024-09-10T09:56:10.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.531+0000 7fcc8d600640 1 -- 172.21.15.77:0/751699658 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcc94037580 con 0x555df41b4e40 2024-09-10T09:56:10.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.532+0000 7fccad06c740 10 client.15534 fetch_fsmap learned FSMap version 29 2024-09-10T09:56:10.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.532+0000 7fccad06c740 1 -- 172.21.15.77:0/751699658 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555df42432e0 con 0x555df41b4e40 2024-09-10T09:56:10.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.532+0000 7fcc8d600640 1 -- 172.21.15.77:0/751699658 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc9c07f070 con 0x555df41b4e40 2024-09-10T09:56:10.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.532+0000 7fcc8d600640 1 -- 172.21.15.77:0/751699658 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fcc9401e5e0 con 0x555df41b4e40 2024-09-10T09:56:10.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.532+0000 7fccad06c740 10 client.15534 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T09:56:10.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.532+0000 7fccad06c740 10 client.15534 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-10T09:56:10.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.532+0000 7fccad06c740 20 client.15534 populate_metadata read hostname 'smithi077' 2024-09-10T09:56:10.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.532+0000 7fccad06c740 1 --2- 172.21.15.77:0/751699658 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x555df4243a60 0x555df4245e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:10.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.532+0000 7fccad06c740 4 client.15534 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:56:10.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.532+0000 7fccad06c740 1 -- 172.21.15.77:0/751699658 --> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555df4122e60 con 0x555df4243a60 2024-09-10T09:56:10.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.533+0000 7fcca2a00640 1 --2- 172.21.15.77:0/751699658 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x555df4243a60 0x555df4245e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:10.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.533+0000 7fcca2a00640 1 --2- 172.21.15.77:0/751699658 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x555df4243a60 0x555df4245e80 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=da6a771b1dbcf0fb server_cookie=67711b8347c7af37 in_seq=0 out_seq=0 2024-09-10T09:56:10.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.533+0000 7fcc8d600640 10 client.15534 ms_handle_connect on v2:172.21.15.120:6836/3033280709 2024-09-10T09:56:10.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.534+0000 7fcc8d600640 1 -- 172.21.15.77:0/751699658 <== mds.1 v2:172.21.15.120:6836/3033280709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555df4122e60 con 0x555df4243a60 2024-09-10T09:56:10.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.534+0000 7fcc8d600640 10 client.15534 handle_command_reply: tid=0 2024-09-10T09:56:10.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.534+0000 7fccad06c740 1 client.15534 shutdown 2024-09-10T09:56:10.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.534+0000 7fccad06c740 1 -- 172.21.15.77:0/751699658 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555df41b4e40 msgr2=0x555df41c8520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:10.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.534+0000 7fccad06c740 1 --2- 172.21.15.77:0/751699658 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555df41b4e40 0x555df41c8520 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fcc940039c0 tx=0x7fcc940046c0 comp rx=0 tx=0).stop 2024-09-10T09:56:10.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.534+0000 7fccad06c740 1 -- 172.21.15.77:0/751699658 shutdown_connections 2024-09-10T09:56:10.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.534+0000 7fccad06c740 1 --2- 172.21.15.77:0/751699658 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x555df4243a60 0x555df4245e80 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:10.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.534+0000 7fccad06c740 1 --2- 172.21.15.77:0/751699658 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555df41c0020 0x555df41ba0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:10.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.534+0000 7fccad06c740 1 --2- 172.21.15.77:0/751699658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555df41b5760 0x555df41b9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:10.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.534+0000 7fccad06c740 1 --2- 172.21.15.77:0/751699658 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555df41b4e40 0x555df41c8520 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:10.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.534+0000 7fccad06c740 1 -- 172.21.15.77:0/751699658 >> 172.21.15.77:0/751699658 conn(0x555df41b13d0 msgr2=0x555df41b0490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:10.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.534+0000 7fcc8d600640 0 client.15534 ms_handle_reset on v2:172.21.15.120:6836/3033280709 2024-09-10T09:56:10.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.534+0000 7fccad06c740 1 -- 172.21.15.77:0/751699658 shutdown_connections 2024-09-10T09:56:10.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.535+0000 7fccad06c740 1 -- 172.21.15.77:0/751699658 wait complete. 2024-09-10T09:56:10.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.535+0000 7fccad06c740 20 client.15534 trim_cache size 0 max 16384 2024-09-10T09:56:10.538 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-10T09:56:10.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.535+0000 7fcca9000640 1 -- 172.21.15.77:0/1841122749 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fcc880a2b80 msgr2=0x7fcc880a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:10.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.535+0000 7fcca9000640 1 --2- 172.21.15.77:0/1841122749 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fcc880a2b80 0x7fcc880a5040 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x555df41b2230 tx=0x7fcc9400b4a0 comp rx=0 tx=0).stop 2024-09-10T09:56:10.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.536+0000 7fcca9000640 1 -- 172.21.15.77:0/1841122749 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcca4108f60 msgr2=0x7fcca419ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:10.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.536+0000 7fcca9000640 1 --2- 172.21.15.77:0/1841122749 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcca4108f60 0x7fcca419ed90 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c009fd0 tx=0x7fcc9c00ef10 comp rx=0 tx=0).stop 2024-09-10T09:56:10.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.536+0000 7fcca9000640 1 -- 172.21.15.77:0/1841122749 shutdown_connections 2024-09-10T09:56:10.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.536+0000 7fcca9000640 1 --2- 172.21.15.77:0/1841122749 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fcc880a2b80 0x7fcc880a5040 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:10.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.536+0000 7fcca9000640 1 --2- 172.21.15.77:0/1841122749 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcca4108f60 0x7fcca419ed90 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:10.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.536+0000 7fcca9000640 1 --2- 172.21.15.77:0/1841122749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcca41047e0 0x7fcca419aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:10.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.536+0000 7fcca9000640 1 --2- 172.21.15.77:0/1841122749 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcca4103e30 0x7fcca419a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:10.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.536+0000 7fcca9000640 1 -- 172.21.15.77:0/1841122749 >> 172.21.15.77:0/1841122749 conn(0x7fcca40ff910 msgr2=0x7fcca4100c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:10.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.536+0000 7fcca9000640 1 -- 172.21.15.77:0/1841122749 shutdown_connections 2024-09-10T09:56:10.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:10.536+0000 7fcca9000640 1 -- 172.21.15.77:0/1841122749 wait complete. 2024-09-10T09:56:10.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:56:10 smithi077 ceph-mon[30319]: pgmap v421: 129 pgs: 129 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-09-10T09:56:11.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:56:10 smithi134 ceph-mon[35674]: pgmap v421: 129 pgs: 129 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-09-10T09:56:11.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:56:10 smithi120 ceph-mon[34770]: pgmap v421: 129 pgs: 129 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-09-10T09:56:11.397 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T09:56:11.893 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'c4ed4a5a', quiesce took 6.5 seconds. Will release after: 23.5 2024-09-10T09:56:12.008 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:56:12.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.618+0000 7fb846400640 1 -- 172.21.15.77:0/668409703 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb84010e4e0 msgr2=0x7fb8401108d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:12.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.618+0000 7fb846400640 1 --2- 172.21.15.77:0/668409703 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb84010e4e0 0x7fb8401108d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb838009be0 tx=0x7fb83802a6c0 comp rx=0 tx=0).stop 2024-09-10T09:56:12.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.618+0000 7fb846400640 1 -- 172.21.15.77:0/668409703 shutdown_connections 2024-09-10T09:56:12.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.618+0000 7fb846400640 1 --2- 172.21.15.77:0/668409703 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb84010e4e0 0x7fb8401108d0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:12.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.618+0000 7fb846400640 1 --2- 172.21.15.77:0/668409703 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb8401012e0 0x7fb84010dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:12.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.618+0000 7fb846400640 1 --2- 172.21.15.77:0/668409703 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb840100930 0x7fb840100d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:12.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.618+0000 7fb846400640 1 -- 172.21.15.77:0/668409703 >> 172.21.15.77:0/668409703 conn(0x7fb8400fc7a0 msgr2=0x7fb8400febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:12.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.619+0000 7fb846400640 1 -- 172.21.15.77:0/668409703 shutdown_connections 2024-09-10T09:56:12.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.619+0000 7fb846400640 1 -- 172.21.15.77:0/668409703 wait complete. 2024-09-10T09:56:12.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.620+0000 7fb846400640 1 Processor -- start 2024-09-10T09:56:12.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.620+0000 7fb846400640 1 -- start start 2024-09-10T09:56:12.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.621+0000 7fb846400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb840100930 0x7fb84019a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:12.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.621+0000 7fb846400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb8401012e0 0x7fb84019ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:12.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.621+0000 7fb846400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb84010e4e0 0x7fb84019eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:12.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.621+0000 7fb846400640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb840114150 con 0x7fb84010e4e0 2024-09-10T09:56:12.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.621+0000 7fb846400640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb840113fd0 con 0x7fb8401012e0 2024-09-10T09:56:12.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.621+0000 7fb846400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb8401142d0 con 0x7fb840100930 2024-09-10T09:56:12.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.621+0000 7fb83ea00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb8401012e0 0x7fb84019ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:12.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.621+0000 7fb83ea00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb8401012e0 0x7fb84019ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:51140/0 (socket says 172.21.15.77:51140) 2024-09-10T09:56:12.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.621+0000 7fb83ea00640 1 -- 172.21.15.77:0/3183410065 learned_addr learned my addr 172.21.15.77:0/3183410065 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:12.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.621+0000 7fb83f400640 1 --2- 172.21.15.77:0/3183410065 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb840100930 0x7fb84019a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:12.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.621+0000 7fb83fe00640 1 --2- 172.21.15.77:0/3183410065 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb84010e4e0 0x7fb84019eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:12.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.622+0000 7fb83ea00640 1 -- 172.21.15.77:0/3183410065 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb840100930 msgr2=0x7fb84019a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:12.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.622+0000 7fb83ea00640 1 --2- 172.21.15.77:0/3183410065 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb840100930 0x7fb84019a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:12.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.622+0000 7fb83ea00640 1 -- 172.21.15.77:0/3183410065 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb84010e4e0 msgr2=0x7fb84019eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:12.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.622+0000 7fb83ea00640 1 --2- 172.21.15.77:0/3183410065 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb84010e4e0 0x7fb84019eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:12.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.622+0000 7fb83ea00640 1 -- 172.21.15.77:0/3183410065 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb84019f580 con 0x7fb8401012e0 2024-09-10T09:56:12.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.622+0000 7fb83fe00640 1 --2- 172.21.15.77:0/3183410065 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb84010e4e0 0x7fb84019eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:56:12.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.622+0000 7fb83ea00640 1 --2- 172.21.15.77:0/3183410065 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb8401012e0 0x7fb84019ab10 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb82c00e9f0 tx=0x7fb82c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:12.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.623+0000 7fb837e00640 1 -- 172.21.15.77:0/3183410065 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb82c00ccc0 con 0x7fb8401012e0 2024-09-10T09:56:12.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.623+0000 7fb837e00640 1 -- 172.21.15.77:0/3183410065 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb82c010910 con 0x7fb8401012e0 2024-09-10T09:56:12.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.623+0000 7fb846400640 1 -- 172.21.15.77:0/3183410065 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb84019f870 con 0x7fb8401012e0 2024-09-10T09:56:12.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.623+0000 7fb837e00640 1 -- 172.21.15.77:0/3183410065 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb82c0188a0 con 0x7fb8401012e0 2024-09-10T09:56:12.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.623+0000 7fb846400640 1 -- 172.21.15.77:0/3183410065 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8401a7150 con 0x7fb8401012e0 2024-09-10T09:56:12.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.627+0000 7fb848dd0740 1 Processor -- start 2024-09-10T09:56:12.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.628+0000 7fb848dd0740 1 -- start start 2024-09-10T09:56:12.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.628+0000 7fb848dd0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a350bc6c70 0x55a350bc7050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:12.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.628+0000 7fb848dd0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a350bc7590 0x55a350bd1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:12.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.628+0000 7fb848dd0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a350bd1e50 0x55a350bd4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:12.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.628+0000 7fb848dd0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55a350b2d3e0 con 0x55a350bc6c70 2024-09-10T09:56:12.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.628+0000 7fb848dd0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a350ad4d80 con 0x55a350bd1e50 2024-09-10T09:56:12.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.628+0000 7fb848dd0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a350b293a0 con 0x55a350bc7590 2024-09-10T09:56:12.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.629+0000 7fb837e00640 1 -- 172.21.15.77:0/3183410065 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb82c010ac0 con 0x7fb8401012e0 2024-09-10T09:56:12.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.629+0000 7fb83f400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a350bc7590 0x55a350bd1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:12.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.629+0000 7fb83fe00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a350bc6c70 0x55a350bc7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:12.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.629+0000 7fb83fe00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a350bc6c70 0x55a350bc7050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34186/0 (socket says 172.21.15.77:34186) 2024-09-10T09:56:12.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.629+0000 7fb83fe00640 1 -- 172.21.15.77:0/713136045 learned_addr learned my addr 172.21.15.77:0/713136045 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:12.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.629+0000 7fb83ea00640 1 --2- 172.21.15.77:0/713136045 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a350bd1e50 0x55a350bd4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:12.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.629+0000 7fb83ea00640 1 -- 172.21.15.77:0/713136045 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a350bc7590 msgr2=0x55a350bd1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:12.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.629+0000 7fb83ea00640 1 --2- 172.21.15.77:0/713136045 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a350bc7590 0x55a350bd1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:12.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.630+0000 7fb83ea00640 1 -- 172.21.15.77:0/713136045 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a350bc6c70 msgr2=0x55a350bc7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:12.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.630+0000 7fb83ea00640 1 --2- 172.21.15.77:0/713136045 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a350bc6c70 0x55a350bc7050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:12.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.630+0000 7fb83ea00640 1 -- 172.21.15.77:0/713136045 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a350af3ae0 con 0x55a350bd1e50 2024-09-10T09:56:12.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.630+0000 7fb83fe00640 1 --2- 172.21.15.77:0/713136045 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a350bc6c70 0x55a350bc7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:56:12.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.630+0000 7fb83ea00640 1 --2- 172.21.15.77:0/713136045 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a350bd1e50 0x55a350bd4240 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb82c0cd040 tx=0x7fb82c0d7670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff9a3b3bcee8abaa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:12.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.630+0000 7fb834c00640 1 -- 172.21.15.77:0/713136045 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb82c0de070 con 0x55a350bd1e50 2024-09-10T09:56:12.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.630+0000 7fb834c00640 1 -- 172.21.15.77:0/713136045 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb82c0dab00 con 0x55a350bd1e50 2024-09-10T09:56:12.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.631+0000 7fb837e00640 1 --2- 172.21.15.77:0/3183410065 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fb8240a2c80 0x7fb8240a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:12.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.631+0000 7fb837e00640 1 -- 172.21.15.77:0/3183410065 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fb82c0cf9f0 con 0x7fb8401012e0 2024-09-10T09:56:12.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.631+0000 7fb848dd0740 1 -- 172.21.15.77:0/713136045 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a350bd1e50 msgr2=0x55a350bd4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:12.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.631+0000 7fb848dd0740 1 --2- 172.21.15.77:0/713136045 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a350bd1e50 0x55a350bd4240 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb82c0cd040 tx=0x7fb82c0d7670 comp rx=0 tx=0).stop 2024-09-10T09:56:12.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.632+0000 7fb848dd0740 1 -- 172.21.15.77:0/713136045 shutdown_connections 2024-09-10T09:56:12.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.632+0000 7fb848dd0740 1 --2- 172.21.15.77:0/713136045 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a350bd1e50 0x55a350bd4240 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:12.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.632+0000 7fb848dd0740 1 --2- 172.21.15.77:0/713136045 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a350bc7590 0x55a350bd1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:12.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.632+0000 7fb848dd0740 1 --2- 172.21.15.77:0/713136045 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a350bc6c70 0x55a350bc7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:12.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.632+0000 7fb848dd0740 1 -- 172.21.15.77:0/713136045 >> 172.21.15.77:0/713136045 conn(0x55a350bc3200 msgr2=0x55a350bc35e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:12.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.632+0000 7fb83fe00640 1 --2- 172.21.15.77:0/3183410065 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fb8240a2c80 0x7fb8240a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:12.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.632+0000 7fb848dd0740 1 -- 172.21.15.77:0/713136045 shutdown_connections 2024-09-10T09:56:12.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.632+0000 7fb848dd0740 1 -- 172.21.15.77:0/713136045 wait complete. 2024-09-10T09:56:12.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.633+0000 7fb848dd0740 1 -- start start 2024-09-10T09:56:12.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.633+0000 7fb848dd0740 1 Processor -- start 2024-09-10T09:56:12.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.633+0000 7fb83fe00640 1 --2- 172.21.15.77:0/3183410065 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fb8240a2c80 0x7fb8240a5140 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb838005f00 tx=0x7fb838009840 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:12.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.633+0000 7fb848dd0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a350bc6c70 0x55a350bd18d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:12.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.634+0000 7fb848dd0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a350bc7590 0x55a350bceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:12.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.634+0000 7fb848dd0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a350bd1e50 0x55a350bcf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:12.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.634+0000 7fb848dd0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55a350bd4a70 con 0x55a350bc7590 2024-09-10T09:56:12.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.634+0000 7fb848dd0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a350ae80f0 con 0x55a350bc6c70 2024-09-10T09:56:12.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.634+0000 7fb848dd0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a350bd4bf0 con 0x55a350bd1e50 2024-09-10T09:56:12.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.634+0000 7fb848dd0740 10 client.0 _ll_register_callbacks cb 0x55a350acfbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:56:12.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.634+0000 7fb83f400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a350bc6c70 0x55a350bd18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:12.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.634+0000 7fb83f400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a350bc6c70 0x55a350bd18d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:51152/0 (socket says 172.21.15.77:51152) 2024-09-10T09:56:12.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.634+0000 7fb83f400640 1 -- 172.21.15.77:0/3306923114 learned_addr learned my addr 172.21.15.77:0/3306923114 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:12.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.634+0000 7fb83ea00640 1 --2- 172.21.15.77:0/3306923114 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a350bc7590 0x55a350bceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:12.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.634+0000 7fb83f400640 1 -- 172.21.15.77:0/3306923114 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a350bd1e50 msgr2=0x55a350bcf2c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T09:56:12.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.634+0000 7fb83f400640 1 --2- 172.21.15.77:0/3306923114 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a350bd1e50 0x55a350bcf2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:12.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.634+0000 7fb83f400640 1 -- 172.21.15.77:0/3306923114 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a350bc7590 msgr2=0x55a350bceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:12.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.634+0000 7fb83f400640 1 --2- 172.21.15.77:0/3306923114 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a350bc7590 0x55a350bceb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:12.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.634+0000 7fb83f400640 1 -- 172.21.15.77:0/3306923114 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a35089e3c0 con 0x55a350bc6c70 2024-09-10T09:56:12.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.634+0000 7fb83ea00640 1 --2- 172.21.15.77:0/3306923114 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a350bc7590 0x55a350bceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:56:12.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.635+0000 7fb83f400640 1 --2- 172.21.15.77:0/3306923114 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a350bc6c70 0x55a350bd18d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb830012740 tx=0x7fb830012c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=429cac616d80c1f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:12.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.635+0000 7fb829600640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T09:56:12.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.635+0000 7fb829600640 1 -- 172.21.15.77:0/3306923114 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb83000f920 con 0x55a350bc6c70 2024-09-10T09:56:12.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.635+0000 7fb829600640 1 -- 172.21.15.77:0/3306923114 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fb83000fac0 con 0x55a350bc6c70 2024-09-10T09:56:12.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.635+0000 7fb848dd0740 1 -- 172.21.15.77:0/3306923114 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb82c0dab00 con 0x55a350bc6c70 2024-09-10T09:56:12.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.635+0000 7fb829600640 1 -- 172.21.15.77:0/3306923114 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb83000fcc0 con 0x55a350bc6c70 2024-09-10T09:56:12.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.635+0000 7fb848dd0740 10 client.25601 fetch_fsmap learned FSMap version 29 2024-09-10T09:56:12.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.635+0000 7fb848dd0740 1 -- 172.21.15.77:0/3306923114 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a350be9230 con 0x55a350bc6c70 2024-09-10T09:56:12.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.636+0000 7fb829600640 1 -- 172.21.15.77:0/3306923114 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb82c0dab00 con 0x55a350bc6c70 2024-09-10T09:56:12.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.636+0000 7fb829600640 1 -- 172.21.15.77:0/3306923114 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fb83001c420 con 0x55a350bc6c70 2024-09-10T09:56:12.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.636+0000 7fb848dd0740 10 client.25601 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T09:56:12.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.636+0000 7fb848dd0740 10 client.25601 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:56:12.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.636+0000 7fb848dd0740 20 client.25601 populate_metadata read hostname 'smithi077' 2024-09-10T09:56:12.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.636+0000 7fb848dd0740 1 --2- 172.21.15.77:0/3306923114 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55a350bea790 0x55a350becc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:12.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.636+0000 7fb848dd0740 4 client.25601 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:56:12.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.636+0000 7fb848dd0740 1 -- 172.21.15.77:0/3306923114 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a350b3a950 con 0x55a350bea790 2024-09-10T09:56:12.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.636+0000 7fb83ea00640 1 --2- 172.21.15.77:0/3306923114 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55a350bea790 0x55a350becc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:12.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.637+0000 7fb829600640 10 client.25601 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T09:56:12.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.637+0000 7fb83ea00640 1 --2- 172.21.15.77:0/3306923114 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55a350bea790 0x55a350becc30 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7681309ddd3aa979 server_cookie=6daa725eebcd6532 in_seq=0 out_seq=0 2024-09-10T09:56:12.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.638+0000 7fb829600640 1 -- 172.21.15.77:0/3306923114 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a350b3a950 con 0x55a350bea790 2024-09-10T09:56:12.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.638+0000 7fb829600640 10 client.25601 handle_command_reply: tid=0 2024-09-10T09:56:12.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.638+0000 7fb848dd0740 1 client.25601 shutdown 2024-09-10T09:56:12.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.638+0000 7fb848dd0740 1 -- 172.21.15.77:0/3306923114 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a350bc6c70 msgr2=0x55a350bd18d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:12.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.638+0000 7fb848dd0740 1 --2- 172.21.15.77:0/3306923114 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a350bc6c70 0x55a350bd18d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb830012740 tx=0x7fb830012c00 comp rx=0 tx=0).stop 2024-09-10T09:56:12.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.639+0000 7fb848dd0740 1 -- 172.21.15.77:0/3306923114 shutdown_connections 2024-09-10T09:56:12.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.639+0000 7fb848dd0740 1 --2- 172.21.15.77:0/3306923114 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55a350bea790 0x55a350becc30 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:12.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.639+0000 7fb848dd0740 1 --2- 172.21.15.77:0/3306923114 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a350bd1e50 0x55a350bcf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:12.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.639+0000 7fb848dd0740 1 --2- 172.21.15.77:0/3306923114 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a350bc7590 0x55a350bceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:12.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.639+0000 7fb848dd0740 1 --2- 172.21.15.77:0/3306923114 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a350bc6c70 0x55a350bd18d0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:12.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.639+0000 7fb829600640 0 client.25601 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T09:56:12.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.639+0000 7fb848dd0740 1 -- 172.21.15.77:0/3306923114 >> 172.21.15.77:0/3306923114 conn(0x55a350bc3200 msgr2=0x55a350bc8f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:12.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.639+0000 7fb848dd0740 1 -- 172.21.15.77:0/3306923114 shutdown_connections 2024-09-10T09:56:12.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.639+0000 7fb848dd0740 1 -- 172.21.15.77:0/3306923114 wait complete. 2024-09-10T09:56:12.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.640+0000 7fb848dd0740 20 client.25601 trim_cache size 0 max 16384 2024-09-10T09:56:12.656 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.655+0000 7fb848dd0740 1 Processor -- start 2024-09-10T09:56:12.656 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.655+0000 7fb848dd0740 1 -- start start 2024-09-10T09:56:12.656 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.655+0000 7fb848dd0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a350bc6c70 0x55a350bc2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:12.656 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.655+0000 7fb848dd0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a350bc7590 0x55a350bf2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:12.656 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.655+0000 7fb848dd0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a350bd1e50 0x55a350bf2a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:12.656 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.655+0000 7fb848dd0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb8401acc30 con 0x55a350bc7590 2024-09-10T09:56:12.656 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.655+0000 7fb848dd0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb8401ae770 con 0x55a350bc6c70 2024-09-10T09:56:12.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.655+0000 7fb848dd0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb8401ad300 con 0x55a350bd1e50 2024-09-10T09:56:12.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.656+0000 7fb83ea00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a350bc7590 0x55a350bf2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:12.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.656+0000 7fb83ea00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a350bc7590 0x55a350bf2500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34216/0 (socket says 172.21.15.77:34216) 2024-09-10T09:56:12.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.656+0000 7fb83ea00640 1 -- 172.21.15.77:0/1538702220 learned_addr learned my addr 172.21.15.77:0/1538702220 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:12.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.656+0000 7fb83fe00640 1 --2- 172.21.15.77:0/1538702220 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a350bd1e50 0x55a350bf2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:12.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.656+0000 7fb83ea00640 1 -- 172.21.15.77:0/1538702220 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a350bd1e50 msgr2=0x55a350bf2a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:12.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.656+0000 7fb83f400640 1 --2- 172.21.15.77:0/1538702220 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a350bc6c70 0x55a350bc2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:12.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.656+0000 7fb83ea00640 1 --2- 172.21.15.77:0/1538702220 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a350bd1e50 0x55a350bf2a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:12.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.656+0000 7fb83ea00640 1 -- 172.21.15.77:0/1538702220 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a350bc6c70 msgr2=0x55a350bc2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:12.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.657+0000 7fb83ea00640 1 --2- 172.21.15.77:0/1538702220 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a350bc6c70 0x55a350bc2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:12.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.657+0000 7fb83ea00640 1 -- 172.21.15.77:0/1538702220 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a350bc73c0 con 0x55a350bc7590 2024-09-10T09:56:12.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.657+0000 7fb83f400640 1 --2- 172.21.15.77:0/1538702220 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a350bc6c70 0x55a350bc2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:56:12.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.657+0000 7fb83ea00640 1 --2- 172.21.15.77:0/1538702220 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a350bc7590 0x55a350bf2500 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fb82c09bd90 tx=0x7fb82c0d7f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99e93cc7d788533b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:12.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.657+0000 7fb82be00640 1 -- 172.21.15.77:0/1538702220 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb82c0de040 con 0x55a350bc7590 2024-09-10T09:56:12.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.657+0000 7fb82be00640 1 -- 172.21.15.77:0/1538702220 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb82c0d9530 con 0x55a350bc7590 2024-09-10T09:56:12.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.657+0000 7fb82be00640 1 -- 172.21.15.77:0/1538702220 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb82c09b760 con 0x55a350bc7590 2024-09-10T09:56:12.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.658+0000 7fb848dd0740 1 -- 172.21.15.77:0/1538702220 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a350bc7590 msgr2=0x55a350bf2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:12.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.658+0000 7fb848dd0740 1 --2- 172.21.15.77:0/1538702220 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a350bc7590 0x55a350bf2500 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fb82c09bd90 tx=0x7fb82c0d7f10 comp rx=0 tx=0).stop 2024-09-10T09:56:12.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.658+0000 7fb848dd0740 1 -- 172.21.15.77:0/1538702220 shutdown_connections 2024-09-10T09:56:12.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.658+0000 7fb848dd0740 1 --2- 172.21.15.77:0/1538702220 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a350bd1e50 0x55a350bf2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:12.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.658+0000 7fb848dd0740 1 --2- 172.21.15.77:0/1538702220 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a350bc7590 0x55a350bf2500 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:12.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.658+0000 7fb848dd0740 1 --2- 172.21.15.77:0/1538702220 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a350bc6c70 0x55a350bc2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:12.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.658+0000 7fb848dd0740 1 -- 172.21.15.77:0/1538702220 >> 172.21.15.77:0/1538702220 conn(0x55a350bc3200 msgr2=0x55a350bc1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:12.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.659+0000 7fb848dd0740 1 -- 172.21.15.77:0/1538702220 shutdown_connections 2024-09-10T09:56:12.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.659+0000 7fb848dd0740 1 -- 172.21.15.77:0/1538702220 wait complete. 2024-09-10T09:56:12.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.659+0000 7fb848dd0740 1 -- start start 2024-09-10T09:56:12.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.660+0000 7fb848dd0740 1 Processor -- start 2024-09-10T09:56:12.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.660+0000 7fb848dd0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a350bc6c70 0x55a350be2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:12.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.660+0000 7fb848dd0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a350bc7590 0x55a350bde310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:12.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.660+0000 7fb848dd0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a350bd1e50 0x55a350bde920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:12.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.660+0000 7fb848dd0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb8401ae0a0 con 0x55a350bd1e50 2024-09-10T09:56:12.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.660+0000 7fb848dd0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb8401ad9d0 con 0x55a350bc7590 2024-09-10T09:56:12.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.660+0000 7fb848dd0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb8401aac10 con 0x55a350bc6c70 2024-09-10T09:56:12.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.660+0000 7fb848dd0740 10 client.0 _ll_register_callbacks cb 0x7fb830009890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:56:12.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.660+0000 7fb83f400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a350bc6c70 0x55a350be2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:12.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.661+0000 7fb83f400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a350bc6c70 0x55a350be2fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:53712/0 (socket says 172.21.15.77:53712) 2024-09-10T09:56:12.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.661+0000 7fb83f400640 1 -- 172.21.15.77:0/50246291 learned_addr learned my addr 172.21.15.77:0/50246291 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:12.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.661+0000 7fb83fe00640 1 --2- 172.21.15.77:0/50246291 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a350bd1e50 0x55a350bde920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:12.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.661+0000 7fb83ea00640 1 --2- 172.21.15.77:0/50246291 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a350bc7590 0x55a350bde310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:12.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.661+0000 7fb83f400640 1 -- 172.21.15.77:0/50246291 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a350bc7590 msgr2=0x55a350bde310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:12.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.661+0000 7fb83f400640 1 --2- 172.21.15.77:0/50246291 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a350bc7590 0x55a350bde310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:12.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.661+0000 7fb83f400640 1 -- 172.21.15.77:0/50246291 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a350bd1e50 msgr2=0x55a350bde920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:12.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.661+0000 7fb83f400640 1 --2- 172.21.15.77:0/50246291 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a350bd1e50 0x55a350bde920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:12.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.661+0000 7fb83f400640 1 -- 172.21.15.77:0/50246291 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a350bdf150 con 0x55a350bc6c70 2024-09-10T09:56:12.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.662+0000 7fb83f400640 1 --2- 172.21.15.77:0/50246291 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a350bc6c70 0x55a350be2fe0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb830000c00 tx=0x7fb83000fee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=221ad23f9557eb23 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:12.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.662+0000 7fb829600640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-10T09:56:12.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.662+0000 7fb829600640 1 -- 172.21.15.77:0/50246291 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb83001c420 con 0x55a350bc6c70 2024-09-10T09:56:12.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.662+0000 7fb829600640 1 -- 172.21.15.77:0/50246291 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fb83001c5c0 con 0x55a350bc6c70 2024-09-10T09:56:12.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.662+0000 7fb848dd0740 1 -- 172.21.15.77:0/50246291 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb82c0d9530 con 0x55a350bc6c70 2024-09-10T09:56:12.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.663+0000 7fb829600640 1 -- 172.21.15.77:0/50246291 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb8300050d0 con 0x55a350bc6c70 2024-09-10T09:56:12.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.663+0000 7fb848dd0740 10 client.25567 fetch_fsmap learned FSMap version 29 2024-09-10T09:56:12.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.663+0000 7fb848dd0740 1 -- 172.21.15.77:0/50246291 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a350c555a0 con 0x55a350bc6c70 2024-09-10T09:56:12.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.663+0000 7fb829600640 1 -- 172.21.15.77:0/50246291 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb82c0d9530 con 0x55a350bc6c70 2024-09-10T09:56:12.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.663+0000 7fb829600640 1 -- 172.21.15.77:0/50246291 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fb830003c70 con 0x55a350bc6c70 2024-09-10T09:56:12.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.663+0000 7fb848dd0740 10 client.25567 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T09:56:12.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.663+0000 7fb848dd0740 10 client.25567 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:56:12.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.663+0000 7fb848dd0740 20 client.25567 populate_metadata read hostname 'smithi077' 2024-09-10T09:56:12.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.663+0000 7fb848dd0740 1 --2- 172.21.15.77:0/50246291 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55a350c55da0 0x55a350c581c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:12.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.664+0000 7fb848dd0740 4 client.25567 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:56:12.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.664+0000 7fb848dd0740 1 -- 172.21.15.77:0/50246291 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a350b34e60 con 0x55a350c55da0 2024-09-10T09:56:12.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.664+0000 7fb83ea00640 1 --2- 172.21.15.77:0/50246291 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55a350c55da0 0x55a350c581c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:12.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.664+0000 7fb83ea00640 1 --2- 172.21.15.77:0/50246291 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55a350c55da0 0x55a350c581c0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5518b5f03e69d3a server_cookie=a0e42a30c38db5be in_seq=0 out_seq=0 2024-09-10T09:56:12.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.664+0000 7fb829600640 10 client.25567 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T09:56:12.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.665+0000 7fb829600640 1 -- 172.21.15.77:0/50246291 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a350b34e60 con 0x55a350c55da0 2024-09-10T09:56:12.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.665+0000 7fb829600640 10 client.25567 handle_command_reply: tid=0 2024-09-10T09:56:12.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.665+0000 7fb848dd0740 1 client.25567 shutdown 2024-09-10T09:56:12.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.665+0000 7fb848dd0740 1 -- 172.21.15.77:0/50246291 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a350bc6c70 msgr2=0x55a350be2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:12.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.665+0000 7fb848dd0740 1 --2- 172.21.15.77:0/50246291 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a350bc6c70 0x55a350be2fe0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb830000c00 tx=0x7fb83000fee0 comp rx=0 tx=0).stop 2024-09-10T09:56:12.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.665+0000 7fb848dd0740 1 -- 172.21.15.77:0/50246291 shutdown_connections 2024-09-10T09:56:12.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.665+0000 7fb848dd0740 1 --2- 172.21.15.77:0/50246291 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55a350c55da0 0x55a350c581c0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:12.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.665+0000 7fb848dd0740 1 --2- 172.21.15.77:0/50246291 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a350bd1e50 0x55a350bde920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:12.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.665+0000 7fb848dd0740 1 --2- 172.21.15.77:0/50246291 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a350bc7590 0x55a350bde310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:12.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.665+0000 7fb848dd0740 1 --2- 172.21.15.77:0/50246291 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a350bc6c70 0x55a350be2fe0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:12.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.665+0000 7fb829600640 0 client.25567 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T09:56:12.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.665+0000 7fb848dd0740 1 -- 172.21.15.77:0/50246291 >> 172.21.15.77:0/50246291 conn(0x55a350bc3200 msgr2=0x55a350be5df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:12.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.665+0000 7fb848dd0740 1 -- 172.21.15.77:0/50246291 shutdown_connections 2024-09-10T09:56:12.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.666+0000 7fb848dd0740 1 -- 172.21.15.77:0/50246291 wait complete. 2024-09-10T09:56:12.668 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-10T09:56:12.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.666+0000 7fb848dd0740 20 client.25567 trim_cache size 0 max 16384 2024-09-10T09:56:12.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.667+0000 7fb846400640 1 -- 172.21.15.77:0/3183410065 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fb8240a2c80 msgr2=0x7fb8240a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:12.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.667+0000 7fb846400640 1 --2- 172.21.15.77:0/3183410065 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fb8240a2c80 0x7fb8240a5140 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb838005f00 tx=0x7fb838009840 comp rx=0 tx=0).stop 2024-09-10T09:56:12.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.667+0000 7fb846400640 1 -- 172.21.15.77:0/3183410065 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb8401012e0 msgr2=0x7fb84019ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:12.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.667+0000 7fb846400640 1 --2- 172.21.15.77:0/3183410065 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb8401012e0 0x7fb84019ab10 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb82c00e9f0 tx=0x7fb82c00edf0 comp rx=0 tx=0).stop 2024-09-10T09:56:12.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.667+0000 7fb846400640 1 -- 172.21.15.77:0/3183410065 shutdown_connections 2024-09-10T09:56:12.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.667+0000 7fb846400640 1 --2- 172.21.15.77:0/3183410065 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fb8240a2c80 0x7fb8240a5140 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:12.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.667+0000 7fb846400640 1 --2- 172.21.15.77:0/3183410065 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb84010e4e0 0x7fb84019eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:12.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.667+0000 7fb846400640 1 --2- 172.21.15.77:0/3183410065 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb8401012e0 0x7fb84019ab10 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:12.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.667+0000 7fb846400640 1 --2- 172.21.15.77:0/3183410065 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb840100930 0x7fb84019a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:12.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.667+0000 7fb846400640 1 -- 172.21.15.77:0/3183410065 >> 172.21.15.77:0/3183410065 conn(0x7fb8400fc7a0 msgr2=0x7fb8400feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:12.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.667+0000 7fb846400640 1 -- 172.21.15.77:0/3183410065 shutdown_connections 2024-09-10T09:56:12.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:12.667+0000 7fb846400640 1 -- 172.21.15.77:0/3183410065 wait complete. 2024-09-10T09:56:13.103 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:56:12 smithi077 ceph-mon[30319]: pgmap v422: 129 pgs: 129 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-10T09:56:13.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:56:12 smithi134 ceph-mon[35674]: pgmap v422: 129 pgs: 129 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-10T09:56:13.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:56:12 smithi120 ceph-mon[34770]: pgmap v422: 129 pgs: 129 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-10T09:56:14.427 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T09:56:14.427 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T09:56:14.840 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:56:14.843 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:56:14 smithi077 ceph-mon[30319]: pgmap v423: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-10T09:56:15.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:56:14 smithi134 ceph-mon[35674]: pgmap v423: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-10T09:56:15.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:56:14 smithi120 ceph-mon[34770]: pgmap v423: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-10T09:56:15.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.519+0000 7f96df000640 1 -- 172.21.15.77:0/3775601211 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f96d8103ff0 msgr2=0x7f96d81043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:15.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.519+0000 7f96df000640 1 --2- 172.21.15.77:0/3775601211 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f96d8103ff0 0x7f96d81043d0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f96c8009900 tx=0x7f96c802a410 comp rx=0 tx=0).stop 2024-09-10T09:56:15.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.519+0000 7f96df000640 1 -- 172.21.15.77:0/3775601211 shutdown_connections 2024-09-10T09:56:15.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.519+0000 7f96df000640 1 --2- 172.21.15.77:0/3775601211 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f96d8109120 0x7f96d8110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:15.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.519+0000 7f96df000640 1 --2- 172.21.15.77:0/3775601211 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f96d81049a0 0x7f96d81089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:15.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.519+0000 7f96df000640 1 --2- 172.21.15.77:0/3775601211 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f96d8103ff0 0x7f96d81043d0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:15.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.519+0000 7f96df000640 1 -- 172.21.15.77:0/3775601211 >> 172.21.15.77:0/3775601211 conn(0x7f96d80ffa90 msgr2=0x7f96d8101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:15.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.519+0000 7f96df000640 1 -- 172.21.15.77:0/3775601211 shutdown_connections 2024-09-10T09:56:15.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.519+0000 7f96df000640 1 -- 172.21.15.77:0/3775601211 wait complete. 2024-09-10T09:56:15.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.520+0000 7f96df000640 1 Processor -- start 2024-09-10T09:56:15.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.521+0000 7f96df000640 1 -- start start 2024-09-10T09:56:15.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.521+0000 7f96df000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f96d8103ff0 0x7f96d819a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:15.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.521+0000 7f96df000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f96d81049a0 0x7f96d819abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:15.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.521+0000 7f96df000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f96d8109120 0x7f96d819ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:15.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.521+0000 7f96df000640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f96d8114170 con 0x7f96d8109120 2024-09-10T09:56:15.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.521+0000 7f96df000640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f96d8113ff0 con 0x7f96d81049a0 2024-09-10T09:56:15.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.521+0000 7f96df000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f96d81142f0 con 0x7f96d8103ff0 2024-09-10T09:56:15.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.521+0000 7f96d7e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f96d8103ff0 0x7f96d819a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:15.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.521+0000 7f96d7e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f96d8103ff0 0x7f96d819a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:54612/0 (socket says 172.21.15.77:54612) 2024-09-10T09:56:15.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.522+0000 7f96d7e00640 1 -- 172.21.15.77:0/13748166 learned_addr learned my addr 172.21.15.77:0/13748166 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:15.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.522+0000 7f96dd000640 1 --2- 172.21.15.77:0/13748166 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f96d8109120 0x7f96d819ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:15.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.522+0000 7f96d7400640 1 --2- 172.21.15.77:0/13748166 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f96d81049a0 0x7f96d819abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:15.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.522+0000 7f96d7e00640 1 -- 172.21.15.77:0/13748166 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f96d81049a0 msgr2=0x7f96d819abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:15.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.522+0000 7f96d7e00640 1 --2- 172.21.15.77:0/13748166 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f96d81049a0 0x7f96d819abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:15.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.522+0000 7f96d7e00640 1 -- 172.21.15.77:0/13748166 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f96d8109120 msgr2=0x7f96d819ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:15.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.522+0000 7f96d7e00640 1 --2- 172.21.15.77:0/13748166 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f96d8109120 0x7f96d819ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:15.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.522+0000 7f96d7e00640 1 -- 172.21.15.77:0/13748166 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96d819f640 con 0x7f96d8103ff0 2024-09-10T09:56:15.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.522+0000 7f96d7e00640 1 --2- 172.21.15.77:0/13748166 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f96d8103ff0 0x7f96d819a690 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f96c8004770 tx=0x7f96c8003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:15.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.523+0000 7f96d4c00640 1 -- 172.21.15.77:0/13748166 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c803f070 con 0x7f96d8103ff0 2024-09-10T09:56:15.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.523+0000 7f96df000640 1 -- 172.21.15.77:0/13748166 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96d819f8d0 con 0x7f96d8103ff0 2024-09-10T09:56:15.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.523+0000 7f96d4c00640 1 -- 172.21.15.77:0/13748166 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96c8031ca0 con 0x7f96d8103ff0 2024-09-10T09:56:15.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.523+0000 7f96d4c00640 1 -- 172.21.15.77:0/13748166 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c8035030 con 0x7f96d8103ff0 2024-09-10T09:56:15.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.523+0000 7f96df000640 1 -- 172.21.15.77:0/13748166 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96d81a7210 con 0x7f96d8103ff0 2024-09-10T09:56:15.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.528+0000 7f96e1a87740 1 Processor -- start 2024-09-10T09:56:15.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.528+0000 7f96e1a87740 1 -- start start 2024-09-10T09:56:15.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.529+0000 7f96e1a87740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637cc818ac0 0x5637cc818ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:15.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.529+0000 7f96e1a87740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5637cc8193e0 0x5637cc823760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:15.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.529+0000 7f96e1a87740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637cc823ca0 0x5637cc826090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:15.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.529+0000 7f96e1a87740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5637cc77f290 con 0x5637cc8193e0 2024-09-10T09:56:15.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.529+0000 7f96e1a87740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5637cc726b40 con 0x5637cc823ca0 2024-09-10T09:56:15.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.529+0000 7f96e1a87740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5637cc77b250 con 0x5637cc818ac0 2024-09-10T09:56:15.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.529+0000 7f96d4c00640 1 -- 172.21.15.77:0/13748166 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f96c8042070 con 0x7f96d8103ff0 2024-09-10T09:56:15.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.529+0000 7f96d7e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5637cc8193e0 0x5637cc823760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:15.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.529+0000 7f96dd000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637cc818ac0 0x5637cc818ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:15.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.529+0000 7f96dd000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637cc818ac0 0x5637cc818ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:54624/0 (socket says 172.21.15.77:54624) 2024-09-10T09:56:15.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.529+0000 7f96dd000640 1 -- 172.21.15.77:0/894225150 learned_addr learned my addr 172.21.15.77:0/894225150 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:15.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.530+0000 7f96dd000640 1 -- 172.21.15.77:0/894225150 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637cc823ca0 msgr2=0x5637cc826090 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T09:56:15.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.530+0000 7f96d7400640 1 --2- 172.21.15.77:0/894225150 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637cc823ca0 0x5637cc826090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:15.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.530+0000 7f96dd000640 1 --2- 172.21.15.77:0/894225150 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637cc823ca0 0x5637cc826090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:15.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.530+0000 7f96dd000640 1 -- 172.21.15.77:0/894225150 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5637cc8193e0 msgr2=0x5637cc823760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:15.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.530+0000 7f96dd000640 1 --2- 172.21.15.77:0/894225150 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5637cc8193e0 0x5637cc823760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:15.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.530+0000 7f96dd000640 1 -- 172.21.15.77:0/894225150 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637cc4f06a0 con 0x5637cc818ac0 2024-09-10T09:56:15.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.530+0000 7f96d7e00640 1 --2- 172.21.15.77:0/894225150 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5637cc8193e0 0x5637cc823760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:56:15.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.530+0000 7f96d7400640 1 --2- 172.21.15.77:0/894225150 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637cc823ca0 0x5637cc826090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:56:15.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.531+0000 7f96dd000640 1 --2- 172.21.15.77:0/894225150 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637cc818ac0 0x5637cc818ea0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f96d0009f90 tx=0x7f96d0010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=906e75e6c645a978 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:15.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.531+0000 7f96cd600640 1 -- 172.21.15.77:0/894225150 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d0011700 con 0x5637cc818ac0 2024-09-10T09:56:15.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.531+0000 7f96cd600640 1 -- 172.21.15.77:0/894225150 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96d0011d80 con 0x5637cc818ac0 2024-09-10T09:56:15.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.531+0000 7f96cd600640 1 -- 172.21.15.77:0/894225150 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d0018380 con 0x5637cc818ac0 2024-09-10T09:56:15.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.531+0000 7f96d4c00640 1 --2- 172.21.15.77:0/13748166 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f96bc0a2c00 0x7f96bc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:15.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.532+0000 7f96d4c00640 1 -- 172.21.15.77:0/13748166 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f96c80efb10 con 0x7f96d8103ff0 2024-09-10T09:56:15.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.532+0000 7f96e1a87740 1 -- 172.21.15.77:0/894225150 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637cc818ac0 msgr2=0x5637cc818ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:15.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.532+0000 7f96e1a87740 1 --2- 172.21.15.77:0/894225150 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637cc818ac0 0x5637cc818ea0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f96d0009f90 tx=0x7f96d0010860 comp rx=0 tx=0).stop 2024-09-10T09:56:15.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.532+0000 7f96d7e00640 1 --2- 172.21.15.77:0/13748166 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f96bc0a2c00 0x7f96bc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:15.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.532+0000 7f96e1a87740 1 -- 172.21.15.77:0/894225150 shutdown_connections 2024-09-10T09:56:15.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.532+0000 7f96e1a87740 1 --2- 172.21.15.77:0/894225150 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637cc823ca0 0x5637cc826090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:15.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.532+0000 7f96e1a87740 1 --2- 172.21.15.77:0/894225150 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5637cc8193e0 0x5637cc823760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:15.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.532+0000 7f96e1a87740 1 --2- 172.21.15.77:0/894225150 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637cc818ac0 0x5637cc818ea0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:15.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.532+0000 7f96e1a87740 1 -- 172.21.15.77:0/894225150 >> 172.21.15.77:0/894225150 conn(0x5637cc815050 msgr2=0x5637cc815430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:15.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.533+0000 7f96e1a87740 1 -- 172.21.15.77:0/894225150 shutdown_connections 2024-09-10T09:56:15.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.533+0000 7f96e1a87740 1 -- 172.21.15.77:0/894225150 wait complete. 2024-09-10T09:56:15.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.533+0000 7f96d7e00640 1 --2- 172.21.15.77:0/13748166 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f96bc0a2c00 0x7f96bc0a50c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f96c803b8b0 tx=0x7f96c80f3750 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:15.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.534+0000 7f96e1a87740 1 -- start start 2024-09-10T09:56:15.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.534+0000 7f96e1a87740 1 Processor -- start 2024-09-10T09:56:15.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.534+0000 7f96e1a87740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637cc818ac0 0x5637cc823620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:15.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.535+0000 7f96e1a87740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5637cc8193e0 0x5637cc8208d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:15.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.535+0000 7f96e1a87740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637cc823ca0 0x5637cc820e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:15.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.535+0000 7f96e1a87740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5637cc826910 con 0x5637cc8193e0 2024-09-10T09:56:15.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.535+0000 7f96e1a87740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5637cc739fa0 con 0x5637cc823ca0 2024-09-10T09:56:15.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.535+0000 7f96e1a87740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5637cc826a90 con 0x5637cc818ac0 2024-09-10T09:56:15.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.535+0000 7f96e1a87740 10 client.0 _ll_register_callbacks cb 0x5637cc48dba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:56:15.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.535+0000 7f96dd000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637cc818ac0 0x5637cc823620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:15.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.535+0000 7f96d7400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5637cc8193e0 0x5637cc8208d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:15.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.535+0000 7f96dd000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637cc818ac0 0x5637cc823620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:54634/0 (socket says 172.21.15.77:54634) 2024-09-10T09:56:15.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.535+0000 7f96dd000640 1 -- 172.21.15.77:0/3871108370 learned_addr learned my addr 172.21.15.77:0/3871108370 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:15.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.535+0000 7f96dd000640 1 --2- 172.21.15.77:0/3871108370 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637cc823ca0 0x5637cc820e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:15.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.536+0000 7f96d7400640 1 -- 172.21.15.77:0/3871108370 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637cc818ac0 msgr2=0x5637cc823620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:15.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.536+0000 7f96d7400640 1 --2- 172.21.15.77:0/3871108370 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637cc818ac0 0x5637cc823620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:15.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.536+0000 7f96d7400640 1 -- 172.21.15.77:0/3871108370 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637cc823ca0 msgr2=0x5637cc820e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:15.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.536+0000 7f96d7400640 1 --2- 172.21.15.77:0/3871108370 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637cc823ca0 0x5637cc820e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:15.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.536+0000 7f96d7400640 1 -- 172.21.15.77:0/3871108370 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637cc821680 con 0x5637cc8193e0 2024-09-10T09:56:15.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.536+0000 7f96dd000640 1 --2- 172.21.15.77:0/3871108370 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637cc823ca0 0x5637cc820e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:56:15.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.536+0000 7f96d7400640 1 --2- 172.21.15.77:0/3871108370 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5637cc8193e0 0x5637cc8208d0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f96d80fffb0 tx=0x7f96c4013b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d716319a87e6a60 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:15.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.536+0000 7f96c2000640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-10T09:56:15.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.537+0000 7f96c2000640 1 -- 172.21.15.77:0/3871108370 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c40119c0 con 0x5637cc8193e0 2024-09-10T09:56:15.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.537+0000 7f96c2000640 1 -- 172.21.15.77:0/3871108370 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f96c4011b60 con 0x5637cc8193e0 2024-09-10T09:56:15.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.537+0000 7f96e1a87740 1 -- 172.21.15.77:0/3871108370 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96d0011d80 con 0x5637cc8193e0 2024-09-10T09:56:15.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.537+0000 7f96c2000640 1 -- 172.21.15.77:0/3871108370 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f96c4011d60 con 0x5637cc8193e0 2024-09-10T09:56:15.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.537+0000 7f96e1a87740 10 client.15561 fetch_fsmap learned FSMap version 29 2024-09-10T09:56:15.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.537+0000 7f96e1a87740 1 -- 172.21.15.77:0/3871108370 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5637cc83b070 con 0x5637cc8193e0 2024-09-10T09:56:15.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.538+0000 7f96c2000640 1 -- 172.21.15.77:0/3871108370 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96d0011d80 con 0x5637cc8193e0 2024-09-10T09:56:15.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.538+0000 7f96c2000640 1 -- 172.21.15.77:0/3871108370 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f96c401a070 con 0x5637cc8193e0 2024-09-10T09:56:15.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.538+0000 7f96e1a87740 10 client.15561 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T09:56:15.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.538+0000 7f96e1a87740 10 client.15561 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:56:15.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.538+0000 7f96e1a87740 20 client.15561 populate_metadata read hostname 'smithi077' 2024-09-10T09:56:15.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.538+0000 7f96e1a87740 1 --2- 172.21.15.77:0/3871108370 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x5637cc83c4d0 0x5637cc83e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:15.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.538+0000 7f96e1a87740 4 client.15561 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:56:15.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.538+0000 7f96e1a87740 1 -- 172.21.15.77:0/3871108370 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5637cc78c800 con 0x5637cc83c4d0 2024-09-10T09:56:15.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.539+0000 7f96dd000640 1 --2- 172.21.15.77:0/3871108370 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x5637cc83c4d0 0x5637cc83e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:15.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.539+0000 7f96c2000640 10 client.15561 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T09:56:15.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.539+0000 7f96dd000640 1 --2- 172.21.15.77:0/3871108370 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x5637cc83c4d0 0x5637cc83e8f0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdcae583301c78fa server_cookie=42dc8b9e2f3380da in_seq=0 out_seq=0 2024-09-10T09:56:15.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.540+0000 7f96c2000640 1 -- 172.21.15.77:0/3871108370 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5637cc78c800 con 0x5637cc83c4d0 2024-09-10T09:56:15.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.540+0000 7f96c2000640 10 client.15561 handle_command_reply: tid=0 2024-09-10T09:56:15.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.540+0000 7f96e1a87740 1 client.15561 shutdown 2024-09-10T09:56:15.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.541+0000 7f96e1a87740 1 -- 172.21.15.77:0/3871108370 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5637cc8193e0 msgr2=0x5637cc8208d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:15.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.541+0000 7f96e1a87740 1 --2- 172.21.15.77:0/3871108370 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5637cc8193e0 0x5637cc8208d0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f96d80fffb0 tx=0x7f96c4013b70 comp rx=0 tx=0).stop 2024-09-10T09:56:15.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.541+0000 7f96e1a87740 1 -- 172.21.15.77:0/3871108370 shutdown_connections 2024-09-10T09:56:15.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.541+0000 7f96e1a87740 1 --2- 172.21.15.77:0/3871108370 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x5637cc83c4d0 0x5637cc83e8f0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:15.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.541+0000 7f96e1a87740 1 --2- 172.21.15.77:0/3871108370 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637cc823ca0 0x5637cc820e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:15.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.541+0000 7f96e1a87740 1 --2- 172.21.15.77:0/3871108370 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5637cc8193e0 0x5637cc8208d0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:15.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.541+0000 7f96e1a87740 1 --2- 172.21.15.77:0/3871108370 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637cc818ac0 0x5637cc823620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:15.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.541+0000 7f96e1a87740 1 -- 172.21.15.77:0/3871108370 >> 172.21.15.77:0/3871108370 conn(0x5637cc815050 msgr2=0x5637cc81ad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:15.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.541+0000 7f96c2000640 0 client.15561 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T09:56:15.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.541+0000 7f96e1a87740 1 -- 172.21.15.77:0/3871108370 shutdown_connections 2024-09-10T09:56:15.543 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.541+0000 7f96e1a87740 1 -- 172.21.15.77:0/3871108370 wait complete. 2024-09-10T09:56:15.543 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.542+0000 7f96e1a87740 20 client.15561 trim_cache size 0 max 16384 2024-09-10T09:56:15.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.563+0000 7f96e1a87740 1 Processor -- start 2024-09-10T09:56:15.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.563+0000 7f96e1a87740 1 -- start start 2024-09-10T09:56:15.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.563+0000 7f96e1a87740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637cc818ac0 0x5637cc81be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:15.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.563+0000 7f96e1a87740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637cc823ca0 0x5637cc844180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:15.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.563+0000 7f96e1a87740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5637cc81f620 0x5637cc8446c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:15.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.563+0000 7f96e1a87740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f96d81acd40 con 0x5637cc81f620 2024-09-10T09:56:15.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.563+0000 7f96e1a87740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f96d81ae880 con 0x5637cc823ca0 2024-09-10T09:56:15.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.563+0000 7f96e1a87740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f96d81ad410 con 0x5637cc818ac0 2024-09-10T09:56:15.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.564+0000 7f96dd000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637cc818ac0 0x5637cc81be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:15.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.564+0000 7f96dd000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637cc818ac0 0x5637cc81be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:54638/0 (socket says 172.21.15.77:54638) 2024-09-10T09:56:15.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.564+0000 7f96dd000640 1 -- 172.21.15.77:0/307774383 learned_addr learned my addr 172.21.15.77:0/307774383 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:15.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.564+0000 7f96dd000640 1 --2- 172.21.15.77:0/307774383 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5637cc81f620 0x5637cc8446c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:15.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.564+0000 7f96d7400640 1 --2- 172.21.15.77:0/307774383 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637cc823ca0 0x5637cc844180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:15.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.565+0000 7f96dd000640 1 -- 172.21.15.77:0/307774383 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637cc823ca0 msgr2=0x5637cc844180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:15.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.565+0000 7f96dd000640 1 --2- 172.21.15.77:0/307774383 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637cc823ca0 0x5637cc844180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:15.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.565+0000 7f96dd000640 1 -- 172.21.15.77:0/307774383 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5637cc81f620 msgr2=0x5637cc8446c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:15.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.565+0000 7f96dd000640 1 --2- 172.21.15.77:0/307774383 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5637cc81f620 0x5637cc8446c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:15.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.565+0000 7f96dd000640 1 -- 172.21.15.77:0/307774383 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637cc81ee40 con 0x5637cc818ac0 2024-09-10T09:56:15.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.565+0000 7f96dd000640 1 --2- 172.21.15.77:0/307774383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637cc818ac0 0x5637cc81be10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f96d0019aa0 tx=0x7f96d0005c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fcdc1a9d39421f89 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:15.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.565+0000 7f96ccc00640 1 -- 172.21.15.77:0/307774383 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d0019cd0 con 0x5637cc818ac0 2024-09-10T09:56:15.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.565+0000 7f96ccc00640 1 -- 172.21.15.77:0/307774383 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96d0020040 con 0x5637cc818ac0 2024-09-10T09:56:15.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.565+0000 7f96ccc00640 1 -- 172.21.15.77:0/307774383 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d0005800 con 0x5637cc818ac0 2024-09-10T09:56:15.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.566+0000 7f96e1a87740 1 -- 172.21.15.77:0/307774383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637cc818ac0 msgr2=0x5637cc81be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:15.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.566+0000 7f96e1a87740 1 --2- 172.21.15.77:0/307774383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637cc818ac0 0x5637cc81be10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f96d0019aa0 tx=0x7f96d0005c80 comp rx=0 tx=0).stop 2024-09-10T09:56:15.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.566+0000 7f96e1a87740 1 -- 172.21.15.77:0/307774383 shutdown_connections 2024-09-10T09:56:15.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.566+0000 7f96e1a87740 1 --2- 172.21.15.77:0/307774383 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5637cc81f620 0x5637cc8446c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:15.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.566+0000 7f96e1a87740 1 --2- 172.21.15.77:0/307774383 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637cc823ca0 0x5637cc844180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:15.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.566+0000 7f96e1a87740 1 --2- 172.21.15.77:0/307774383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637cc818ac0 0x5637cc81be10 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:15.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.566+0000 7f96e1a87740 1 -- 172.21.15.77:0/307774383 >> 172.21.15.77:0/307774383 conn(0x5637cc815050 msgr2=0x5637cc81b700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:15.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.567+0000 7f96e1a87740 1 -- 172.21.15.77:0/307774383 shutdown_connections 2024-09-10T09:56:15.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.567+0000 7f96e1a87740 1 -- 172.21.15.77:0/307774383 wait complete. 2024-09-10T09:56:15.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.567+0000 7f96e1a87740 1 -- start start 2024-09-10T09:56:15.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.567+0000 7f96e1a87740 1 Processor -- start 2024-09-10T09:56:15.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.568+0000 7f96e1a87740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637cc818ac0 0x5637cc82bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:15.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.568+0000 7f96e1a87740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5637cc81f620 0x5637cc82c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:15.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.568+0000 7f96e1a87740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637cc823ca0 0x5637cc828ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:15.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.568+0000 7f96e1a87740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f96d81ae1b0 con 0x5637cc81f620 2024-09-10T09:56:15.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.568+0000 7f96e1a87740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f96d81adae0 con 0x5637cc818ac0 2024-09-10T09:56:15.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.568+0000 7f96e1a87740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f96d81aad20 con 0x5637cc823ca0 2024-09-10T09:56:15.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.568+0000 7f96e1a87740 10 client.0 _ll_register_callbacks cb 0x7f96c40097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:56:15.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.568+0000 7f96dd000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637cc818ac0 0x5637cc82bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:15.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.568+0000 7f96dd000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637cc818ac0 0x5637cc82bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:58900/0 (socket says 172.21.15.77:58900) 2024-09-10T09:56:15.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.568+0000 7f96dd000640 1 -- 172.21.15.77:0/15512072 learned_addr learned my addr 172.21.15.77:0/15512072 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:15.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.568+0000 7f96dd000640 1 --2- 172.21.15.77:0/15512072 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637cc823ca0 0x5637cc828ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:15.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.568+0000 7f96d7400640 1 --2- 172.21.15.77:0/15512072 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5637cc81f620 0x5637cc82c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:15.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.569+0000 7f96dd000640 1 -- 172.21.15.77:0/15512072 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637cc818ac0 msgr2=0x5637cc82bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:15.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.569+0000 7f96dd000640 1 --2- 172.21.15.77:0/15512072 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637cc818ac0 0x5637cc82bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:15.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.569+0000 7f96dd000640 1 -- 172.21.15.77:0/15512072 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5637cc81f620 msgr2=0x5637cc82c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:15.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.569+0000 7f96dd000640 1 --2- 172.21.15.77:0/15512072 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5637cc81f620 0x5637cc82c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:15.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.569+0000 7f96dd000640 1 -- 172.21.15.77:0/15512072 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637cc829320 con 0x5637cc823ca0 2024-09-10T09:56:15.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.569+0000 7f96dd000640 1 --2- 172.21.15.77:0/15512072 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637cc823ca0 0x5637cc828ce0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f96d00197c0 tx=0x7f96d0000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=307b7e40354df387 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:15.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.569+0000 7f96c2000640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-10T09:56:15.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.569+0000 7f96c2000640 1 -- 172.21.15.77:0/15512072 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d0019cd0 con 0x5637cc823ca0 2024-09-10T09:56:15.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.569+0000 7f96c2000640 1 -- 172.21.15.77:0/15512072 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f96d0005580 con 0x5637cc823ca0 2024-09-10T09:56:15.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.569+0000 7f96e1a87740 1 -- 172.21.15.77:0/15512072 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96d0020040 con 0x5637cc823ca0 2024-09-10T09:56:15.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.570+0000 7f96c2000640 1 -- 172.21.15.77:0/15512072 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f96d0019e70 con 0x5637cc823ca0 2024-09-10T09:56:15.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.570+0000 7f96e1a87740 10 client.25585 fetch_fsmap learned FSMap version 29 2024-09-10T09:56:15.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.570+0000 7f96e1a87740 1 -- 172.21.15.77:0/15512072 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5637cc8a7ae0 con 0x5637cc823ca0 2024-09-10T09:56:15.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.570+0000 7f96c2000640 1 -- 172.21.15.77:0/15512072 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96d0020040 con 0x5637cc823ca0 2024-09-10T09:56:15.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.570+0000 7f96c2000640 1 -- 172.21.15.77:0/15512072 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f96d00117d0 con 0x5637cc823ca0 2024-09-10T09:56:15.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.570+0000 7f96e1a87740 10 client.25585 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T09:56:15.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.570+0000 7f96e1a87740 10 client.25585 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:56:15.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.570+0000 7f96e1a87740 20 client.25585 populate_metadata read hostname 'smithi077' 2024-09-10T09:56:15.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.571+0000 7f96e1a87740 1 --2- 172.21.15.77:0/15512072 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x5637cc8a7f00 0x5637cc8aa2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:15.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.571+0000 7f96e1a87740 4 client.25585 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T09:56:15.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.571+0000 7f96e1a87740 1 -- 172.21.15.77:0/15512072 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5637cc786d10 con 0x5637cc8a7f00 2024-09-10T09:56:15.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.571+0000 7f96d7400640 1 --2- 172.21.15.77:0/15512072 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x5637cc8a7f00 0x5637cc8aa2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:15.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.572+0000 7f96d7400640 1 --2- 172.21.15.77:0/15512072 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x5637cc8a7f00 0x5637cc8aa2f0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=999efee8c46b1f1 server_cookie=dca734ead86cffc1 in_seq=0 out_seq=0 2024-09-10T09:56:15.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.572+0000 7f96c2000640 10 client.25585 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T09:56:15.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.573+0000 7f96c2000640 1 -- 172.21.15.77:0/15512072 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5637cc786d10 con 0x5637cc8a7f00 2024-09-10T09:56:15.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.573+0000 7f96c2000640 10 client.25585 handle_command_reply: tid=0 2024-09-10T09:56:15.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.573+0000 7f96e1a87740 1 client.25585 shutdown 2024-09-10T09:56:15.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.574+0000 7f96e1a87740 1 -- 172.21.15.77:0/15512072 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637cc823ca0 msgr2=0x5637cc828ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:15.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.574+0000 7f96e1a87740 1 --2- 172.21.15.77:0/15512072 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637cc823ca0 0x5637cc828ce0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f96d00197c0 tx=0x7f96d0000f40 comp rx=0 tx=0).stop 2024-09-10T09:56:15.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.574+0000 7f96e1a87740 1 -- 172.21.15.77:0/15512072 shutdown_connections 2024-09-10T09:56:15.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.574+0000 7f96e1a87740 1 --2- 172.21.15.77:0/15512072 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x5637cc8a7f00 0x5637cc8aa2f0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:15.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.574+0000 7f96e1a87740 1 --2- 172.21.15.77:0/15512072 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637cc823ca0 0x5637cc828ce0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:15.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.574+0000 7f96c2000640 0 client.25585 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T09:56:15.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.574+0000 7f96e1a87740 1 --2- 172.21.15.77:0/15512072 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5637cc81f620 0x5637cc82c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:15.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.574+0000 7f96e1a87740 1 --2- 172.21.15.77:0/15512072 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637cc818ac0 0x5637cc82bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:15.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.574+0000 7f96e1a87740 1 -- 172.21.15.77:0/15512072 >> 172.21.15.77:0/15512072 conn(0x5637cc815050 msgr2=0x5637cc822950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:15.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.575+0000 7f96e1a87740 1 -- 172.21.15.77:0/15512072 shutdown_connections 2024-09-10T09:56:15.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.575+0000 7f96e1a87740 1 -- 172.21.15.77:0/15512072 wait complete. 2024-09-10T09:56:15.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.575+0000 7f96e1a87740 20 client.25585 trim_cache size 0 max 16384 2024-09-10T09:56:15.577 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-10T09:56:15.577 INFO:teuthology.orchestra.run.smithi077.stdout: "return_code": 0, 2024-09-10T09:56:15.577 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_tag": "b4447db7-82e6-4c0d-858e-dbefec4ead16", 2024-09-10T09:56:15.577 INFO:teuthology.orchestra.run.smithi077.stdout: "mode": "asynchronous" 2024-09-10T09:56:15.577 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-10T09:56:15.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.576+0000 7f96df000640 1 -- 172.21.15.77:0/13748166 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f96bc0a2c00 msgr2=0x7f96bc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:15.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.576+0000 7f96df000640 1 --2- 172.21.15.77:0/13748166 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f96bc0a2c00 0x7f96bc0a50c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f96c803b8b0 tx=0x7f96c80f3750 comp rx=0 tx=0).stop 2024-09-10T09:56:15.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.576+0000 7f96df000640 1 -- 172.21.15.77:0/13748166 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f96d8103ff0 msgr2=0x7f96d819a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:15.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.576+0000 7f96df000640 1 --2- 172.21.15.77:0/13748166 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f96d8103ff0 0x7f96d819a690 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f96c8004770 tx=0x7f96c8003c10 comp rx=0 tx=0).stop 2024-09-10T09:56:15.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.576+0000 7f96df000640 1 -- 172.21.15.77:0/13748166 shutdown_connections 2024-09-10T09:56:15.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.576+0000 7f96df000640 1 --2- 172.21.15.77:0/13748166 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f96bc0a2c00 0x7f96bc0a50c0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:15.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.576+0000 7f96df000640 1 --2- 172.21.15.77:0/13748166 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f96d8109120 0x7f96d819ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:15.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.576+0000 7f96df000640 1 --2- 172.21.15.77:0/13748166 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f96d81049a0 0x7f96d819abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:15.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.576+0000 7f96df000640 1 --2- 172.21.15.77:0/13748166 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f96d8103ff0 0x7f96d819a690 secure :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f96c8004770 tx=0x7f96c8003c10 comp rx=0 tx=0).stop 2024-09-10T09:56:15.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.576+0000 7f96df000640 1 -- 172.21.15.77:0/13748166 >> 172.21.15.77:0/13748166 conn(0x7f96d80ffa90 msgr2=0x7f96d8100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:15.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.577+0000 7f96df000640 1 -- 172.21.15.77:0/13748166 shutdown_connections 2024-09-10T09:56:15.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:15.577+0000 7f96df000640 1 -- 172.21.15.77:0/13748166 wait complete. 2024-09-10T09:56:15.853 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:56:15 smithi077 ceph-mon[30319]: scrub queued for path: / 2024-09-10T09:56:15.853 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:56:15 smithi077 ceph-mon[30319]: scrub summary: idle+waiting paths [/] 2024-09-10T09:56:15.853 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:56:15 smithi077 ceph-mon[30319]: scrub summary: active paths [/] 2024-09-10T09:56:15.853 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:56:15 smithi077 ceph-mon[30319]: scrub summary: idle+waiting paths [/] 2024-09-10T09:56:15.853 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:56:15 smithi077 ceph-mon[30319]: scrub summary: idle 2024-09-10T09:56:16.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:56:15 smithi134 ceph-mon[35674]: scrub queued for path: / 2024-09-10T09:56:16.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:56:15 smithi134 ceph-mon[35674]: scrub summary: idle+waiting paths [/] 2024-09-10T09:56:16.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:56:15 smithi134 ceph-mon[35674]: scrub summary: active paths [/] 2024-09-10T09:56:16.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:56:15 smithi134 ceph-mon[35674]: scrub summary: idle+waiting paths [/] 2024-09-10T09:56:16.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:56:15 smithi134 ceph-mon[35674]: scrub summary: idle 2024-09-10T09:56:16.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:56:15 smithi120 ceph-mon[34770]: scrub queued for path: / 2024-09-10T09:56:16.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:56:15 smithi120 ceph-mon[34770]: scrub summary: idle+waiting paths [/] 2024-09-10T09:56:16.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:56:15 smithi120 ceph-mon[34770]: scrub summary: active paths [/] 2024-09-10T09:56:16.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:56:15 smithi120 ceph-mon[34770]: scrub summary: idle+waiting paths [/] 2024-09-10T09:56:16.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:56:15 smithi120 ceph-mon[34770]: scrub summary: idle 2024-09-10T09:56:16.375 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T09:56:16.905 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:56:17.112 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:56:16 smithi077 ceph-mon[30319]: pgmap v424: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-10T09:56:17.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:56:16 smithi134 ceph-mon[35674]: pgmap v424: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-10T09:56:17.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:56:16 smithi120 ceph-mon[34770]: pgmap v424: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-10T09:56:17.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.616+0000 7fcb20c00640 1 -- 172.21.15.77:0/3695723140 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcb1c1047e0 msgr2=0x7fcb1c108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:17.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.616+0000 7fcb20c00640 1 --2- 172.21.15.77:0/3695723140 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcb1c1047e0 0x7fcb1c108830 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fcb08009900 tx=0x7fcb0802a410 comp rx=0 tx=0).stop 2024-09-10T09:56:17.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.617+0000 7fcb20c00640 1 -- 172.21.15.77:0/3695723140 shutdown_connections 2024-09-10T09:56:17.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.617+0000 7fcb20c00640 1 --2- 172.21.15.77:0/3695723140 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb1c108f60 0x7fcb1c110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:17.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.617+0000 7fcb20c00640 1 --2- 172.21.15.77:0/3695723140 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcb1c1047e0 0x7fcb1c108830 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:17.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.617+0000 7fcb20c00640 1 --2- 172.21.15.77:0/3695723140 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb1c103e30 0x7fcb1c104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:17.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.617+0000 7fcb20c00640 1 -- 172.21.15.77:0/3695723140 >> 172.21.15.77:0/3695723140 conn(0x7fcb1c0ff910 msgr2=0x7fcb1c101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:17.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.617+0000 7fcb20c00640 1 -- 172.21.15.77:0/3695723140 shutdown_connections 2024-09-10T09:56:17.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.617+0000 7fcb20c00640 1 -- 172.21.15.77:0/3695723140 wait complete. 2024-09-10T09:56:17.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.618+0000 7fcb20c00640 1 Processor -- start 2024-09-10T09:56:17.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.619+0000 7fcb20c00640 1 -- start start 2024-09-10T09:56:17.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.619+0000 7fcb20c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb1c103e30 0x7fcb1c19a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:17.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.619+0000 7fcb20c00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcb1c1047e0 0x7fcb1c19aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:17.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.619+0000 7fcb20c00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb1c108f60 0x7fcb1c19ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:17.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.619+0000 7fcb20c00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fcb1c114010 con 0x7fcb1c1047e0 2024-09-10T09:56:17.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.619+0000 7fcb20c00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fcb1c113e90 con 0x7fcb1c108f60 2024-09-10T09:56:17.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.619+0000 7fcb20c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcb1c114190 con 0x7fcb1c103e30 2024-09-10T09:56:17.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.620+0000 7fcb1b400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb1c103e30 0x7fcb1c19a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:17.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.620+0000 7fcb1aa00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcb1c1047e0 0x7fcb1c19aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:17.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.620+0000 7fcb1b400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb1c103e30 0x7fcb1c19a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:54656/0 (socket says 172.21.15.77:54656) 2024-09-10T09:56:17.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.620+0000 7fcb1b400640 1 -- 172.21.15.77:0/1452824070 learned_addr learned my addr 172.21.15.77:0/1452824070 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:17.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.620+0000 7fcb1be00640 1 --2- 172.21.15.77:0/1452824070 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb1c108f60 0x7fcb1c19ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:17.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.620+0000 7fcb1b400640 1 -- 172.21.15.77:0/1452824070 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb1c108f60 msgr2=0x7fcb1c19ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:17.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.620+0000 7fcb1b400640 1 --2- 172.21.15.77:0/1452824070 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb1c108f60 0x7fcb1c19ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:17.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.620+0000 7fcb1b400640 1 -- 172.21.15.77:0/1452824070 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcb1c1047e0 msgr2=0x7fcb1c19aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:17.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.620+0000 7fcb1b400640 1 --2- 172.21.15.77:0/1452824070 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcb1c1047e0 0x7fcb1c19aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:17.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.620+0000 7fcb1b400640 1 -- 172.21.15.77:0/1452824070 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb1c19f4e0 con 0x7fcb1c103e30 2024-09-10T09:56:17.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.620+0000 7fcb1aa00640 1 --2- 172.21.15.77:0/1452824070 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcb1c1047e0 0x7fcb1c19aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:56:17.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.621+0000 7fcb1b400640 1 --2- 172.21.15.77:0/1452824070 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb1c103e30 0x7fcb1c19a530 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c00e9f0 tx=0x7fcb0c00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:17.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.621+0000 7fcb13e00640 1 -- 172.21.15.77:0/1452824070 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb0c00cd80 con 0x7fcb1c103e30 2024-09-10T09:56:17.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.621+0000 7fcb20c00640 1 -- 172.21.15.77:0/1452824070 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb1c19f7d0 con 0x7fcb1c103e30 2024-09-10T09:56:17.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.621+0000 7fcb20c00640 1 -- 172.21.15.77:0/1452824070 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb1c076630 con 0x7fcb1c103e30 2024-09-10T09:56:17.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.621+0000 7fcb13e00640 1 -- 172.21.15.77:0/1452824070 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb0c005500 con 0x7fcb1c103e30 2024-09-10T09:56:17.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.621+0000 7fcb13e00640 1 -- 172.21.15.77:0/1452824070 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb0c010690 con 0x7fcb1c103e30 2024-09-10T09:56:17.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.626+0000 7fcb24c6a740 1 Processor -- start 2024-09-10T09:56:17.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.626+0000 7fcb24c6a740 1 -- start start 2024-09-10T09:56:17.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.626+0000 7fcb13e00640 1 -- 172.21.15.77:0/1452824070 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fcb0c004f20 con 0x7fcb1c103e30 2024-09-10T09:56:17.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.626+0000 7fcb24c6a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c5cc338d50 0x55c5cc339130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:17.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.626+0000 7fcb24c6a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c5cc339670 0x55c5cc3439f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:17.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.626+0000 7fcb24c6a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5cc343f30 0x55c5cc346320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:17.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.626+0000 7fcb24c6a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c5cc29f3e0 con 0x55c5cc338d50 2024-09-10T09:56:17.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.626+0000 7fcb24c6a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c5cc246d80 con 0x55c5cc339670 2024-09-10T09:56:17.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.626+0000 7fcb24c6a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c5cc29b3a0 con 0x55c5cc343f30 2024-09-10T09:56:17.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.627+0000 7fcb1be00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c5cc338d50 0x55c5cc339130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:17.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.627+0000 7fcb1be00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c5cc338d50 0x55c5cc339130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42100/0 (socket says 172.21.15.77:42100) 2024-09-10T09:56:17.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.627+0000 7fcb1be00640 1 -- 172.21.15.77:0/3909054167 learned_addr learned my addr 172.21.15.77:0/3909054167 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:17.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.627+0000 7fcb1aa00640 1 --2- 172.21.15.77:0/3909054167 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5cc343f30 0x55c5cc346320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:17.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.627+0000 7fcb1b400640 1 --2- 172.21.15.77:0/3909054167 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c5cc339670 0x55c5cc3439f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:17.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.627+0000 7fcb1be00640 1 -- 172.21.15.77:0/3909054167 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5cc343f30 msgr2=0x55c5cc346320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:17.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.627+0000 7fcb1be00640 1 --2- 172.21.15.77:0/3909054167 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5cc343f30 0x55c5cc346320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:17.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.627+0000 7fcb1be00640 1 -- 172.21.15.77:0/3909054167 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c5cc339670 msgr2=0x55c5cc3439f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:17.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.627+0000 7fcb1be00640 1 --2- 172.21.15.77:0/3909054167 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c5cc339670 0x55c5cc3439f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:17.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.627+0000 7fcb1be00640 1 -- 172.21.15.77:0/3909054167 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5cc265ae0 con 0x55c5cc338d50 2024-09-10T09:56:17.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.627+0000 7fcb1aa00640 1 --2- 172.21.15.77:0/3909054167 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5cc343f30 0x55c5cc346320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:56:17.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.628+0000 7fcb1be00640 1 --2- 172.21.15.77:0/3909054167 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c5cc338d50 0x55c5cc339130 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fcb14009aa0 tx=0x7fcb14009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37c73fd995457de2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:17.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.628+0000 7fcb13e00640 1 --2- 172.21.15.77:0/1452824070 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fcb000a2c00 0x7fcb000a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:17.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.628+0000 7fcb13e00640 1 -- 172.21.15.77:0/1452824070 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fcb0c0cf670 con 0x7fcb1c103e30 2024-09-10T09:56:17.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.628+0000 7fcb07e00640 1 -- 172.21.15.77:0/3909054167 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb14004b20 con 0x55c5cc338d50 2024-09-10T09:56:17.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.628+0000 7fcb07e00640 1 -- 172.21.15.77:0/3909054167 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb14004cc0 con 0x55c5cc338d50 2024-09-10T09:56:17.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.628+0000 7fcb07e00640 1 -- 172.21.15.77:0/3909054167 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb140042b0 con 0x55c5cc338d50 2024-09-10T09:56:17.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.629+0000 7fcb1b400640 1 --2- 172.21.15.77:0/1452824070 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fcb000a2c00 0x7fcb000a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:17.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.629+0000 7fcb24c6a740 1 -- 172.21.15.77:0/3909054167 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c5cc338d50 msgr2=0x55c5cc339130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:17.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.629+0000 7fcb24c6a740 1 --2- 172.21.15.77:0/3909054167 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c5cc338d50 0x55c5cc339130 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fcb14009aa0 tx=0x7fcb14009500 comp rx=0 tx=0).stop 2024-09-10T09:56:17.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.630+0000 7fcb24c6a740 1 -- 172.21.15.77:0/3909054167 shutdown_connections 2024-09-10T09:56:17.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.630+0000 7fcb24c6a740 1 --2- 172.21.15.77:0/3909054167 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5cc343f30 0x55c5cc346320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:17.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.630+0000 7fcb24c6a740 1 --2- 172.21.15.77:0/3909054167 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c5cc339670 0x55c5cc3439f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:17.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.630+0000 7fcb24c6a740 1 --2- 172.21.15.77:0/3909054167 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c5cc338d50 0x55c5cc339130 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:17.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.630+0000 7fcb24c6a740 1 -- 172.21.15.77:0/3909054167 >> 172.21.15.77:0/3909054167 conn(0x55c5cc3352e0 msgr2=0x55c5cc3356c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:17.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.630+0000 7fcb24c6a740 1 -- 172.21.15.77:0/3909054167 shutdown_connections 2024-09-10T09:56:17.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.630+0000 7fcb1b400640 1 --2- 172.21.15.77:0/1452824070 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fcb000a2c00 0x7fcb000a50c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c0cd0f0 tx=0x7fcb0c0095b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:17.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.630+0000 7fcb24c6a740 1 -- 172.21.15.77:0/3909054167 wait complete. 2024-09-10T09:56:17.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.631+0000 7fcb24c6a740 1 -- start start 2024-09-10T09:56:17.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.631+0000 7fcb24c6a740 1 Processor -- start 2024-09-10T09:56:17.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.631+0000 7fcb24c6a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c5cc338d50 0x55c5cc340e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:17.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.631+0000 7fcb24c6a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c5cc339670 0x55c5cc3413b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:17.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.632+0000 7fcb24c6a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5cc343f30 0x55c5cc3418f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:17.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.632+0000 7fcb24c6a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c5cc346ba0 con 0x55c5cc339670 2024-09-10T09:56:17.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.632+0000 7fcb24c6a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c5cc25a0f0 con 0x55c5cc338d50 2024-09-10T09:56:17.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.632+0000 7fcb24c6a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c5cc346d20 con 0x55c5cc343f30 2024-09-10T09:56:17.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.632+0000 7fcb24c6a740 10 client.0 _ll_register_callbacks cb 0x55c5cc24b9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:56:17.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.632+0000 7fcb1be00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c5cc338d50 0x55c5cc340e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:17.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.632+0000 7fcb1be00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c5cc338d50 0x55c5cc340e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:58926/0 (socket says 172.21.15.77:58926) 2024-09-10T09:56:17.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.632+0000 7fcb1be00640 1 -- 172.21.15.77:0/2768170099 learned_addr learned my addr 172.21.15.77:0/2768170099 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:17.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.632+0000 7fcb1aa00640 1 --2- 172.21.15.77:0/2768170099 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c5cc339670 0x55c5cc3413b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:17.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.632+0000 7fcb1be00640 1 --2- 172.21.15.77:0/2768170099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5cc343f30 0x55c5cc3418f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:17.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.633+0000 7fcb1be00640 1 -- 172.21.15.77:0/2768170099 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c5cc338d50 msgr2=0x55c5cc340e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:17.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.633+0000 7fcb1be00640 1 --2- 172.21.15.77:0/2768170099 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c5cc338d50 0x55c5cc340e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:17.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.633+0000 7fcb1be00640 1 -- 172.21.15.77:0/2768170099 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c5cc339670 msgr2=0x55c5cc3413b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:17.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.633+0000 7fcb1be00640 1 --2- 172.21.15.77:0/2768170099 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c5cc339670 0x55c5cc3413b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:17.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.633+0000 7fcb1be00640 1 -- 172.21.15.77:0/2768170099 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5cc0103c0 con 0x55c5cc343f30 2024-09-10T09:56:17.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.633+0000 7fcb1aa00640 1 --2- 172.21.15.77:0/2768170099 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c5cc339670 0x55c5cc3413b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:56:17.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.634+0000 7fcb1be00640 1 --2- 172.21.15.77:0/2768170099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5cc343f30 0x55c5cc3418f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fcb14015fd0 tx=0x7fcb14016040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d2e5835a68a87ca server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:17.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.634+0000 7fcb04c00640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-10T09:56:17.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.634+0000 7fcb04c00640 1 -- 172.21.15.77:0/2768170099 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb14004280 con 0x55c5cc343f30 2024-09-10T09:56:17.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.634+0000 7fcb04c00640 1 -- 172.21.15.77:0/2768170099 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fcb14004420 con 0x55c5cc343f30 2024-09-10T09:56:17.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.634+0000 7fcb24c6a740 1 -- 172.21.15.77:0/2768170099 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb14004cc0 con 0x55c5cc343f30 2024-09-10T09:56:17.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.635+0000 7fcb04c00640 1 -- 172.21.15.77:0/2768170099 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcb1403b050 con 0x55c5cc343f30 2024-09-10T09:56:17.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.635+0000 7fcb24c6a740 10 client.25597 fetch_fsmap learned FSMap version 29 2024-09-10T09:56:17.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.635+0000 7fcb24c6a740 1 -- 172.21.15.77:0/2768170099 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c5cc352900 con 0x55c5cc343f30 2024-09-10T09:56:17.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.635+0000 7fcb04c00640 1 -- 172.21.15.77:0/2768170099 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb14004cc0 con 0x55c5cc343f30 2024-09-10T09:56:17.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.636+0000 7fcb04c00640 1 -- 172.21.15.77:0/2768170099 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fcb1402cc10 con 0x55c5cc343f30 2024-09-10T09:56:17.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.636+0000 7fcb24c6a740 10 client.25597 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T09:56:17.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.636+0000 7fcb24c6a740 10 client.25597 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:56:17.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.636+0000 7fcb24c6a740 20 client.25597 populate_metadata read hostname 'smithi077' 2024-09-10T09:56:17.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.636+0000 7fcb24c6a740 1 --2- 172.21.15.77:0/2768170099 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55c5cc353d10 0x55c5cc356100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:17.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.636+0000 7fcb24c6a740 4 client.25597 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:56:17.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.636+0000 7fcb24c6a740 1 -- 172.21.15.77:0/2768170099 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c5cc2ac950 con 0x55c5cc353d10 2024-09-10T09:56:17.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.637+0000 7fcb1aa00640 1 --2- 172.21.15.77:0/2768170099 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55c5cc353d10 0x55c5cc356100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:17.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.637+0000 7fcb1aa00640 1 --2- 172.21.15.77:0/2768170099 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55c5cc353d10 0x55c5cc356100 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f96a1e91668f12 server_cookie=a82e7944657fd501 in_seq=0 out_seq=0 2024-09-10T09:56:17.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.637+0000 7fcb04c00640 10 client.25597 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T09:56:17.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.638+0000 7fcb04c00640 1 -- 172.21.15.77:0/2768170099 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c5cc2ac950 con 0x55c5cc353d10 2024-09-10T09:56:17.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.638+0000 7fcb04c00640 10 client.25597 handle_command_reply: tid=0 2024-09-10T09:56:17.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.638+0000 7fcb24c6a740 1 client.25597 shutdown 2024-09-10T09:56:17.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.639+0000 7fcb24c6a740 1 -- 172.21.15.77:0/2768170099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5cc343f30 msgr2=0x55c5cc3418f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:17.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.639+0000 7fcb24c6a740 1 --2- 172.21.15.77:0/2768170099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5cc343f30 0x55c5cc3418f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fcb14015fd0 tx=0x7fcb14016040 comp rx=0 tx=0).stop 2024-09-10T09:56:17.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.639+0000 7fcb24c6a740 1 -- 172.21.15.77:0/2768170099 shutdown_connections 2024-09-10T09:56:17.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.639+0000 7fcb24c6a740 1 --2- 172.21.15.77:0/2768170099 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55c5cc353d10 0x55c5cc356100 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:17.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.639+0000 7fcb24c6a740 1 --2- 172.21.15.77:0/2768170099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5cc343f30 0x55c5cc3418f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:17.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.639+0000 7fcb24c6a740 1 --2- 172.21.15.77:0/2768170099 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c5cc339670 0x55c5cc3413b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:17.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.639+0000 7fcb24c6a740 1 --2- 172.21.15.77:0/2768170099 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c5cc338d50 0x55c5cc340e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:17.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.639+0000 7fcb04c00640 0 client.25597 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T09:56:17.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.639+0000 7fcb24c6a740 1 -- 172.21.15.77:0/2768170099 >> 172.21.15.77:0/2768170099 conn(0x55c5cc3352e0 msgr2=0x55c5cc33b010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:17.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.639+0000 7fcb24c6a740 1 -- 172.21.15.77:0/2768170099 shutdown_connections 2024-09-10T09:56:17.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.639+0000 7fcb24c6a740 1 -- 172.21.15.77:0/2768170099 wait complete. 2024-09-10T09:56:17.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.640+0000 7fcb24c6a740 20 client.25597 trim_cache size 0 max 16384 2024-09-10T09:56:17.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.674+0000 7fcb24c6a740 1 Processor -- start 2024-09-10T09:56:17.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.674+0000 7fcb24c6a740 1 -- start start 2024-09-10T09:56:17.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.675+0000 7fcb24c6a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c5cc338d50 0x55c5cc341740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:17.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.675+0000 7fcb24c6a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c5cc339670 0x55c5cc35b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:17.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.675+0000 7fcb24c6a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5cc343f30 0x55c5cc35bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:17.677 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.675+0000 7fcb24c6a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fcafc00fd20 con 0x55c5cc339670 2024-09-10T09:56:17.677 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.675+0000 7fcb24c6a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fcafc011860 con 0x55c5cc338d50 2024-09-10T09:56:17.677 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.675+0000 7fcb24c6a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcafc0103f0 con 0x55c5cc343f30 2024-09-10T09:56:17.677 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.675+0000 7fcb1be00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c5cc338d50 0x55c5cc341740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:17.677 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.675+0000 7fcb1be00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c5cc338d50 0x55c5cc341740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:58940/0 (socket says 172.21.15.77:58940) 2024-09-10T09:56:17.677 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.675+0000 7fcb1be00640 1 -- 172.21.15.77:0/2076783156 learned_addr learned my addr 172.21.15.77:0/2076783156 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:17.677 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.675+0000 7fcb1aa00640 1 --2- 172.21.15.77:0/2076783156 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c5cc339670 0x55c5cc35b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:17.677 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.676+0000 7fcb1be00640 1 -- 172.21.15.77:0/2076783156 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5cc343f30 msgr2=0x55c5cc35bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:17.677 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.676+0000 7fcb1be00640 1 --2- 172.21.15.77:0/2076783156 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5cc343f30 0x55c5cc35bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:17.677 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.676+0000 7fcb1be00640 1 -- 172.21.15.77:0/2076783156 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c5cc339670 msgr2=0x55c5cc35b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:17.677 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.676+0000 7fcb1be00640 1 --2- 172.21.15.77:0/2076783156 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c5cc339670 0x55c5cc35b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:17.677 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.676+0000 7fcb1be00640 1 -- 172.21.15.77:0/2076783156 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5cc346d30 con 0x55c5cc338d50 2024-09-10T09:56:17.677 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.676+0000 7fcb1be00640 1 --2- 172.21.15.77:0/2076783156 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c5cc338d50 0x55c5cc341740 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fcb14002230 tx=0x7fcb140051f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c90bbc769ed87c77 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:17.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.676+0000 7fcb07400640 1 -- 172.21.15.77:0/2076783156 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb14044070 con 0x55c5cc338d50 2024-09-10T09:56:17.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.676+0000 7fcb07400640 1 -- 172.21.15.77:0/2076783156 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb14009500 con 0x55c5cc338d50 2024-09-10T09:56:17.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.676+0000 7fcb07400640 1 -- 172.21.15.77:0/2076783156 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb14004e30 con 0x55c5cc338d50 2024-09-10T09:56:17.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.678+0000 7fcb24c6a740 1 -- 172.21.15.77:0/2076783156 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c5cc338d50 msgr2=0x55c5cc341740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:17.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.678+0000 7fcb24c6a740 1 --2- 172.21.15.77:0/2076783156 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c5cc338d50 0x55c5cc341740 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fcb14002230 tx=0x7fcb140051f0 comp rx=0 tx=0).stop 2024-09-10T09:56:17.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.678+0000 7fcb24c6a740 1 -- 172.21.15.77:0/2076783156 shutdown_connections 2024-09-10T09:56:17.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.678+0000 7fcb24c6a740 1 --2- 172.21.15.77:0/2076783156 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5cc343f30 0x55c5cc35bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:17.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.678+0000 7fcb24c6a740 1 --2- 172.21.15.77:0/2076783156 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c5cc339670 0x55c5cc35b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:17.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.678+0000 7fcb24c6a740 1 --2- 172.21.15.77:0/2076783156 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c5cc338d50 0x55c5cc341740 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:17.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.678+0000 7fcb24c6a740 1 -- 172.21.15.77:0/2076783156 >> 172.21.15.77:0/2076783156 conn(0x55c5cc3352e0 msgr2=0x55c5cc342ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:17.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.678+0000 7fcb24c6a740 1 -- 172.21.15.77:0/2076783156 shutdown_connections 2024-09-10T09:56:17.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.678+0000 7fcb24c6a740 1 -- 172.21.15.77:0/2076783156 wait complete. 2024-09-10T09:56:17.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.678+0000 7fcb24c6a740 1 -- start start 2024-09-10T09:56:17.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.679+0000 7fcb24c6a740 1 Processor -- start 2024-09-10T09:56:17.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.679+0000 7fcb24c6a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c5cc338d50 0x55c5cc347730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:17.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.679+0000 7fcb24c6a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c5cc339670 0x55c5cc347c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:17.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.679+0000 7fcb24c6a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5cc343f30 0x55c5cc3b6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:17.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.679+0000 7fcb24c6a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fcafc011190 con 0x55c5cc338d50 2024-09-10T09:56:17.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.679+0000 7fcb24c6a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fcafc010ac0 con 0x55c5cc339670 2024-09-10T09:56:17.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.679+0000 7fcb24c6a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcafc00dd00 con 0x55c5cc343f30 2024-09-10T09:56:17.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.679+0000 7fcb24c6a740 10 client.0 _ll_register_callbacks cb 0x7fcb1400bba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:56:17.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.680+0000 7fcb1aa00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c5cc339670 0x55c5cc347c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:17.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.680+0000 7fcb1aa00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c5cc339670 0x55c5cc347c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:58950/0 (socket says 172.21.15.77:58950) 2024-09-10T09:56:17.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.680+0000 7fcb1aa00640 1 -- 172.21.15.77:0/3069528731 learned_addr learned my addr 172.21.15.77:0/3069528731 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:17.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.680+0000 7fcb1be00640 1 --2- 172.21.15.77:0/3069528731 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c5cc338d50 0x55c5cc347730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:17.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.680+0000 7fcb1be00640 1 --2- 172.21.15.77:0/3069528731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5cc343f30 0x55c5cc3b6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:17.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.680+0000 7fcb1aa00640 1 -- 172.21.15.77:0/3069528731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5cc343f30 msgr2=0x55c5cc3b6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:17.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.680+0000 7fcb1aa00640 1 --2- 172.21.15.77:0/3069528731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5cc343f30 0x55c5cc3b6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:17.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.680+0000 7fcb1aa00640 1 -- 172.21.15.77:0/3069528731 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c5cc338d50 msgr2=0x55c5cc347730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:17.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.681+0000 7fcb1aa00640 1 --2- 172.21.15.77:0/3069528731 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c5cc338d50 0x55c5cc347730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:17.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.681+0000 7fcb1aa00640 1 -- 172.21.15.77:0/3069528731 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5cc336390 con 0x55c5cc339670 2024-09-10T09:56:17.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.681+0000 7fcb1be00640 1 --2- 172.21.15.77:0/3069528731 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c5cc338d50 0x55c5cc347730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:56:17.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.681+0000 7fcb1aa00640 1 --2- 172.21.15.77:0/3069528731 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c5cc339670 0x55c5cc347c70 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fcb08009900 tx=0x7fcb08004820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40907dbd1a1d5875 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:17.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.681+0000 7fcb04c00640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T09:56:17.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.681+0000 7fcb04c00640 1 -- 172.21.15.77:0/3069528731 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb08057060 con 0x55c5cc339670 2024-09-10T09:56:17.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.681+0000 7fcb04c00640 1 -- 172.21.15.77:0/3069528731 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fcb08005410 con 0x55c5cc339670 2024-09-10T09:56:17.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.682+0000 7fcb24c6a740 1 -- 172.21.15.77:0/3069528731 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c5cc2a6e60 con 0x55c5cc339670 2024-09-10T09:56:17.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.682+0000 7fcb04c00640 1 -- 172.21.15.77:0/3069528731 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcb08004a50 con 0x55c5cc339670 2024-09-10T09:56:17.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.682+0000 7fcb24c6a740 10 client.25634 fetch_fsmap learned FSMap version 29 2024-09-10T09:56:17.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.682+0000 7fcb24c6a740 1 -- 172.21.15.77:0/3069528731 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c5cc3bdfc0 con 0x55c5cc339670 2024-09-10T09:56:17.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.683+0000 7fcb04c00640 1 -- 172.21.15.77:0/3069528731 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb08005610 con 0x55c5cc339670 2024-09-10T09:56:17.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.683+0000 7fcb04c00640 1 -- 172.21.15.77:0/3069528731 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fcb08052670 con 0x55c5cc339670 2024-09-10T09:56:17.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.683+0000 7fcb24c6a740 10 client.25634 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T09:56:17.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.683+0000 7fcb24c6a740 10 client.25634 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:56:17.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.683+0000 7fcb24c6a740 20 client.25634 populate_metadata read hostname 'smithi077' 2024-09-10T09:56:17.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.683+0000 7fcb24c6a740 1 --2- 172.21.15.77:0/3069528731 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55c5cc3be6d0 0x55c5cc3c0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:17.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.683+0000 7fcb24c6a740 4 client.25634 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T09:56:17.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.683+0000 7fcb24c6a740 1 -- 172.21.15.77:0/3069528731 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c5cc2a6e60 con 0x55c5cc3be6d0 2024-09-10T09:56:17.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.684+0000 7fcb1be00640 1 --2- 172.21.15.77:0/3069528731 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55c5cc3be6d0 0x55c5cc3c0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:17.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.685+0000 7fcb1be00640 1 --2- 172.21.15.77:0/3069528731 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55c5cc3be6d0 0x55c5cc3c0af0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2896c0e4e0499e21 server_cookie=aac934f3d444a542 in_seq=0 out_seq=0 2024-09-10T09:56:17.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.685+0000 7fcb04c00640 10 client.25634 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T09:56:17.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.685+0000 7fcb04c00640 1 -- 172.21.15.77:0/3069528731 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c5cc2a6e60 con 0x55c5cc3be6d0 2024-09-10T09:56:17.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.685+0000 7fcb04c00640 10 client.25634 handle_command_reply: tid=0 2024-09-10T09:56:17.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.685+0000 7fcb24c6a740 1 client.25634 shutdown 2024-09-10T09:56:17.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.685+0000 7fcb24c6a740 1 -- 172.21.15.77:0/3069528731 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c5cc339670 msgr2=0x55c5cc347c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:17.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.685+0000 7fcb24c6a740 1 --2- 172.21.15.77:0/3069528731 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c5cc339670 0x55c5cc347c70 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fcb08009900 tx=0x7fcb08004820 comp rx=0 tx=0).stop 2024-09-10T09:56:17.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.685+0000 7fcb24c6a740 1 -- 172.21.15.77:0/3069528731 shutdown_connections 2024-09-10T09:56:17.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.685+0000 7fcb24c6a740 1 --2- 172.21.15.77:0/3069528731 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55c5cc3be6d0 0x55c5cc3c0af0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:17.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.685+0000 7fcb24c6a740 1 --2- 172.21.15.77:0/3069528731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5cc343f30 0x55c5cc3b6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:17.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.685+0000 7fcb24c6a740 1 --2- 172.21.15.77:0/3069528731 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c5cc339670 0x55c5cc347c70 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:17.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.685+0000 7fcb04c00640 0 client.25634 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T09:56:17.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.685+0000 7fcb24c6a740 1 --2- 172.21.15.77:0/3069528731 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c5cc338d50 0x55c5cc347730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:17.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.685+0000 7fcb24c6a740 1 -- 172.21.15.77:0/3069528731 >> 172.21.15.77:0/3069528731 conn(0x55c5cc3352e0 msgr2=0x55c5cc3426d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:17.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.686+0000 7fcb24c6a740 1 -- 172.21.15.77:0/3069528731 shutdown_connections 2024-09-10T09:56:17.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.686+0000 7fcb24c6a740 1 -- 172.21.15.77:0/3069528731 wait complete. 2024-09-10T09:56:17.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.686+0000 7fcb24c6a740 20 client.25634 trim_cache size 0 max 16384 2024-09-10T09:56:17.688 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-10T09:56:17.688 INFO:teuthology.orchestra.run.smithi077.stdout: "status": "no active scrubs running", 2024-09-10T09:56:17.688 INFO:teuthology.orchestra.run.smithi077.stdout: "scrubs": {} 2024-09-10T09:56:17.688 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-10T09:56:17.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.687+0000 7fcb20c00640 1 -- 172.21.15.77:0/1452824070 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fcb000a2c00 msgr2=0x7fcb000a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:17.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.687+0000 7fcb20c00640 1 --2- 172.21.15.77:0/1452824070 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fcb000a2c00 0x7fcb000a50c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c0cd0f0 tx=0x7fcb0c0095b0 comp rx=0 tx=0).stop 2024-09-10T09:56:17.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.687+0000 7fcb20c00640 1 -- 172.21.15.77:0/1452824070 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb1c103e30 msgr2=0x7fcb1c19a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:17.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.687+0000 7fcb20c00640 1 --2- 172.21.15.77:0/1452824070 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb1c103e30 0x7fcb1c19a530 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c00e9f0 tx=0x7fcb0c00edf0 comp rx=0 tx=0).stop 2024-09-10T09:56:17.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.687+0000 7fcb20c00640 1 -- 172.21.15.77:0/1452824070 shutdown_connections 2024-09-10T09:56:17.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.687+0000 7fcb20c00640 1 --2- 172.21.15.77:0/1452824070 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fcb000a2c00 0x7fcb000a50c0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:17.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.687+0000 7fcb20c00640 1 --2- 172.21.15.77:0/1452824070 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb1c108f60 0x7fcb1c19ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:17.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.687+0000 7fcb20c00640 1 --2- 172.21.15.77:0/1452824070 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcb1c1047e0 0x7fcb1c19aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:17.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.687+0000 7fcb20c00640 1 --2- 172.21.15.77:0/1452824070 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb1c103e30 0x7fcb1c19a530 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:17.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.687+0000 7fcb20c00640 1 -- 172.21.15.77:0/1452824070 >> 172.21.15.77:0/1452824070 conn(0x7fcb1c0ff910 msgr2=0x7fcb1c100d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:17.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.688+0000 7fcb20c00640 1 -- 172.21.15.77:0/1452824070 shutdown_connections 2024-09-10T09:56:17.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:17.688+0000 7fcb20c00640 1 -- 172.21.15.77:0/1452824070 wait complete. 2024-09-10T09:56:18.579 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T09:56:18.579 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:56:19.079 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:56:19.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:56:18 smithi134 ceph-mon[35674]: pgmap v425: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-10T09:56:19.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:56:18 smithi120 ceph-mon[34770]: pgmap v425: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-10T09:56:19.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:56:18 smithi077 ceph-mon[30319]: pgmap v425: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-10T09:56:19.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:19.674+0000 7ff5e4c00640 1 -- 172.21.15.77:0/2469336946 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff5e0102360 msgr2=0x7ff5e010e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:19.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:19.674+0000 7ff5e4c00640 1 --2- 172.21.15.77:0/2469336946 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff5e0102360 0x7ff5e010e890 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7ff5d8009c10 tx=0x7ff5d802a6f0 comp rx=0 tx=0).stop 2024-09-10T09:56:19.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:19.674+0000 7ff5e4c00640 1 -- 172.21.15.77:0/2469336946 shutdown_connections 2024-09-10T09:56:19.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:19.674+0000 7ff5e4c00640 1 --2- 172.21.15.77:0/2469336946 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff5e0102360 0x7ff5e010e890 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:19.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:19.674+0000 7ff5e4c00640 1 --2- 172.21.15.77:0/2469336946 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff5e01019c0 0x7ff5e0101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:19.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:19.674+0000 7ff5e4c00640 1 --2- 172.21.15.77:0/2469336946 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5e01079c0 0x7ff5e0107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:19.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:19.674+0000 7ff5e4c00640 1 -- 172.21.15.77:0/2469336946 >> 172.21.15.77:0/2469336946 conn(0x7ff5e00fd740 msgr2=0x7ff5e00ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:19.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:19.674+0000 7ff5e4c00640 1 -- 172.21.15.77:0/2469336946 shutdown_connections 2024-09-10T09:56:19.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:19.674+0000 7ff5e4c00640 1 -- 172.21.15.77:0/2469336946 wait complete. 2024-09-10T09:56:19.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:19.675+0000 7ff5e4c00640 1 Processor -- start 2024-09-10T09:56:19.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:19.676+0000 7ff5e4c00640 1 -- start start 2024-09-10T09:56:19.677 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:19.676+0000 7ff5e4c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5e01019c0 0x7ff5e00744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:19.677 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:19.676+0000 7ff5e4c00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff5e0102360 0x7ff5e0074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:19.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:19.676+0000 7ff5e4c00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff5e01079c0 0x7ff5e0074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:19.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:19.676+0000 7ff5e4c00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e0111d70 con 0x7ff5e0102360 2024-09-10T09:56:19.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:19.676+0000 7ff5e4c00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e0111bf0 con 0x7ff5e01079c0 2024-09-10T09:56:19.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:19.676+0000 7ff5e4c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e0111ef0 con 0x7ff5e01019c0 2024-09-10T09:56:19.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:19.677+0000 7ff5df400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5e01019c0 0x7ff5e00744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:19.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:19.677+0000 7ff5dfe00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff5e01079c0 0x7ff5e0074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:19.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:19.677+0000 7ff5dfe00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff5e01079c0 0x7ff5e0074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:58964/0 (socket says 172.21.15.77:58964) 2024-09-10T09:56:19.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:19.677+0000 7ff5df400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5e01019c0 0x7ff5e00744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:54728/0 (socket says 172.21.15.77:54728) 2024-09-10T09:56:19.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:19.677+0000 7ff5dfe00640 1 -- 172.21.15.77:0/813404148 learned_addr learned my addr 172.21.15.77:0/813404148 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:19.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:19.677+0000 7ff5dea00640 1 --2- 172.21.15.77:0/813404148 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff5e0102360 0x7ff5e0074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:19.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:19.678+0000 7ff5df400640 1 -- 172.21.15.77:0/813404148 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff5e01079c0 msgr2=0x7ff5e0074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:19.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:19.678+0000 7ff5df400640 1 --2- 172.21.15.77:0/813404148 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff5e01079c0 0x7ff5e0074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:19.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:19.678+0000 7ff5df400640 1 -- 172.21.15.77:0/813404148 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff5e0102360 msgr2=0x7ff5e0074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:19.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:19.678+0000 7ff5df400640 1 --2- 172.21.15.77:0/813404148 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff5e0102360 0x7ff5e0074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:19.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:19.678+0000 7ff5df400640 1 -- 172.21.15.77:0/813404148 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5e01a6720 con 0x7ff5e01019c0 2024-09-10T09:56:19.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:19.678+0000 7ff5dea00640 1 --2- 172.21.15.77:0/813404148 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff5e0102360 0x7ff5e0074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:56:19.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:19.678+0000 7ff5df400640 1 --2- 172.21.15.77:0/813404148 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5e01019c0 0x7ff5e00744f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff5d0009fd0 tx=0x7ff5d000ec20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:19.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:19.679+0000 7ff5d7e00640 1 -- 172.21.15.77:0/813404148 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d000ccc0 con 0x7ff5e01019c0 2024-09-10T09:56:19.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:19.679+0000 7ff5e4c00640 1 -- 172.21.15.77:0/813404148 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5e01a6a10 con 0x7ff5e01019c0 2024-09-10T09:56:19.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:19.679+0000 7ff5e4c00640 1 -- 172.21.15.77:0/813404148 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5e01a6f50 con 0x7ff5e01019c0 2024-09-10T09:56:19.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:19.679+0000 7ff5d7e00640 1 -- 172.21.15.77:0/813404148 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5d000ee70 con 0x7ff5e01019c0 2024-09-10T09:56:19.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:19.679+0000 7ff5d7e00640 1 -- 172.21.15.77:0/813404148 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d0004a00 con 0x7ff5e01019c0 2024-09-10T09:56:19.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:19.680+0000 7ff5e4c00640 1 -- 172.21.15.77:0/813404148 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5c0005290 con 0x7ff5e01019c0 2024-09-10T09:56:19.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:19.683+0000 7ff5d7e00640 1 -- 172.21.15.77:0/813404148 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff5d0005020 con 0x7ff5e01019c0 2024-09-10T09:56:19.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:19.685+0000 7ff5d7e00640 1 --2- 172.21.15.77:0/813404148 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ff5c40a2c00 0x7ff5c40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:19.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:19.686+0000 7ff5d7e00640 1 -- 172.21.15.77:0/813404148 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7ff5d00cf200 con 0x7ff5e01019c0 2024-09-10T09:56:19.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:19.686+0000 7ff5dea00640 1 --2- 172.21.15.77:0/813404148 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ff5c40a2c00 0x7ff5c40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:19.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:19.687+0000 7ff5dea00640 1 --2- 172.21.15.77:0/813404148 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ff5c40a2c00 0x7ff5c40a50c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff5e0075f80 tx=0x7ff5cc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:19.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:19.689+0000 7ff5d7e00640 1 -- 172.21.15.77:0/813404148 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff5d009bd10 con 0x7ff5e01019c0 2024-09-10T09:56:19.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:19.911+0000 7ff5e4c00640 1 -- 172.21.15.77:0/813404148 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff5c0005580 con 0x7ff5e01019c0 2024-09-10T09:56:19.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:19.912+0000 7ff5d7e00640 1 -- 172.21.15.77:0/813404148 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11989 (secure 0 0 0) 0x7ff5d009b610 con 0x7ff5e01019c0 2024-09-10T09:56:19.914 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:56:19.914 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":29,"btime":"2024-09-10T09:55:13:863715+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3159289675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3159289675},{"type":"v1","addr":"172.21.15.77:6835","nonce":3159289675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1951962341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1951962341},{"type":"v1","addr":"172.21.15.77:6837","nonce":1951962341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:52:23.368252+0000","modified":"2024-09-10T09:55:13.786975+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24563,"mds_1":24524,"mds_2":24520,"mds_3":24502,"mds_4":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14487":{"gid":14487,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/4291121888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":4291121888},{"type":"v1","addr":"172.21.15.77:6839","nonce":4291121888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14499":{"gid":14499,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6841/445284194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":445284194},{"type":"v1","addr":"172.21.15.77:6841","nonce":445284194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1005054423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1005054423},{"type":"v1","addr":"172.21.15.134:6833","nonce":1005054423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6833/3716307744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":3716307744},{"type":"v1","addr":"172.21.15.120:6833","nonce":3716307744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/1319892544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1319892544},{"type":"v1","addr":"172.21.15.120:6835","nonce":1319892544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.134:6835/3293933699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3293933699},{"type":"v1","addr":"172.21.15.134:6835","nonce":3293933699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.134:6837/2461497091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2461497091},{"type":"v1","addr":"172.21.15.134:6837","nonce":2461497091}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":20,"addr":"172.21.15.134:6839/1965117680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1965117680},{"type":"v1","addr":"172.21.15.134:6839","nonce":1965117680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":20,"addr":"172.21.15.120:6837/3033280709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3033280709},{"type":"v1","addr":"172.21.15.120:6837","nonce":3033280709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.120:6839/1777710157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1777710157},{"type":"v1","addr":"172.21.15.120:6839","nonce":1777710157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24563,"qdb_cluster":[24496,24502,24520,24524,24563]},"id":1}]} 2024-09-10T09:56:19.914 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 29 2024-09-10T09:56:19.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:19.916+0000 7ff5e4c00640 1 -- 172.21.15.77:0/813404148 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ff5c40a2c00 msgr2=0x7ff5c40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:19.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:19.916+0000 7ff5e4c00640 1 --2- 172.21.15.77:0/813404148 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ff5c40a2c00 0x7ff5c40a50c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff5e0075f80 tx=0x7ff5cc005f90 comp rx=0 tx=0).stop 2024-09-10T09:56:19.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:19.916+0000 7ff5e4c00640 1 -- 172.21.15.77:0/813404148 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5e01019c0 msgr2=0x7ff5e00744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:19.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:19.916+0000 7ff5e4c00640 1 --2- 172.21.15.77:0/813404148 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5e01019c0 0x7ff5e00744f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff5d0009fd0 tx=0x7ff5d000ec20 comp rx=0 tx=0).stop 2024-09-10T09:56:19.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:19.917+0000 7ff5e4c00640 1 -- 172.21.15.77:0/813404148 shutdown_connections 2024-09-10T09:56:19.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:19.917+0000 7ff5e4c00640 1 --2- 172.21.15.77:0/813404148 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ff5c40a2c00 0x7ff5c40a50c0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:19.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:19.917+0000 7ff5e4c00640 1 --2- 172.21.15.77:0/813404148 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff5e01079c0 0x7ff5e0074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:19.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:19.917+0000 7ff5e4c00640 1 --2- 172.21.15.77:0/813404148 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff5e0102360 0x7ff5e0074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:19.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:19.917+0000 7ff5e4c00640 1 --2- 172.21.15.77:0/813404148 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5e01019c0 0x7ff5e00744f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:19.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:19.917+0000 7ff5e4c00640 1 -- 172.21.15.77:0/813404148 >> 172.21.15.77:0/813404148 conn(0x7ff5e00fd740 msgr2=0x7ff5e00fef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:19.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:19.917+0000 7ff5e4c00640 1 -- 172.21.15.77:0/813404148 shutdown_connections 2024-09-10T09:56:19.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:19.917+0000 7ff5e4c00640 1 -- 172.21.15.77:0/813404148 wait complete. 2024-09-10T09:56:20.629 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T09:56:20.861 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:56:20 smithi077 ceph-mon[30319]: pgmap v426: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 597 B/s wr, 12 op/s 2024-09-10T09:56:20.861 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:56:20 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/813404148' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:56:21.082 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:56:21.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:56:20 smithi134 ceph-mon[35674]: pgmap v426: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 597 B/s wr, 12 op/s 2024-09-10T09:56:21.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:56:20 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/813404148' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:56:21.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:56:20 smithi120 ceph-mon[34770]: pgmap v426: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 597 B/s wr, 12 op/s 2024-09-10T09:56:21.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:56:20 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/813404148' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:56:21.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.713+0000 7ff09e600640 1 -- 172.21.15.77:0/910700004 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff098108f60 msgr2=0x7ff098110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:21.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.713+0000 7ff09e600640 1 --2- 172.21.15.77:0/910700004 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff098108f60 0x7ff098110ae0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff090009ca0 tx=0x7ff09002a8e0 comp rx=0 tx=0).stop 2024-09-10T09:56:21.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.713+0000 7ff09e600640 1 -- 172.21.15.77:0/910700004 shutdown_connections 2024-09-10T09:56:21.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.713+0000 7ff09e600640 1 --2- 172.21.15.77:0/910700004 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff098108f60 0x7ff098110ae0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:21.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.713+0000 7ff09e600640 1 --2- 172.21.15.77:0/910700004 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff0981047e0 0x7ff098108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:21.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.713+0000 7ff09e600640 1 --2- 172.21.15.77:0/910700004 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff098103e30 0x7ff098104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:21.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.713+0000 7ff09e600640 1 -- 172.21.15.77:0/910700004 >> 172.21.15.77:0/910700004 conn(0x7ff0980ff910 msgr2=0x7ff098101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:21.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.713+0000 7ff09e600640 1 -- 172.21.15.77:0/910700004 shutdown_connections 2024-09-10T09:56:21.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.714+0000 7ff09e600640 1 -- 172.21.15.77:0/910700004 wait complete. 2024-09-10T09:56:21.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.715+0000 7ff09e600640 1 Processor -- start 2024-09-10T09:56:21.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.715+0000 7ff09e600640 1 -- start start 2024-09-10T09:56:21.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.716+0000 7ff09e600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff098103e30 0x7ff09819a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:21.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.716+0000 7ff09e600640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff0981047e0 0x7ff09819a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:21.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.716+0000 7ff09e600640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff098108f60 0x7ff09819ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:21.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.716+0000 7ff09e600640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff098113fc0 con 0x7ff098108f60 2024-09-10T09:56:21.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.716+0000 7ff09e600640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff098113e40 con 0x7ff0981047e0 2024-09-10T09:56:21.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.716+0000 7ff09e600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff098114140 con 0x7ff098103e30 2024-09-10T09:56:21.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.717+0000 7ff09d200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff098103e30 0x7ff09819a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:21.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.717+0000 7ff09d200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff098103e30 0x7ff09819a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:54746/0 (socket says 172.21.15.77:54746) 2024-09-10T09:56:21.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.717+0000 7ff09d200640 1 -- 172.21.15.77:0/3031830562 learned_addr learned my addr 172.21.15.77:0/3031830562 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:21.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.717+0000 7ff097e00640 1 --2- 172.21.15.77:0/3031830562 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff0981047e0 0x7ff09819a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:21.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.717+0000 7ff09dc00640 1 --2- 172.21.15.77:0/3031830562 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff098108f60 0x7ff09819ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:21.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.717+0000 7ff097e00640 1 -- 172.21.15.77:0/3031830562 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff098103e30 msgr2=0x7ff09819a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:21.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.717+0000 7ff097e00640 1 --2- 172.21.15.77:0/3031830562 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff098103e30 0x7ff09819a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:21.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.717+0000 7ff097e00640 1 -- 172.21.15.77:0/3031830562 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff098108f60 msgr2=0x7ff09819ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:21.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.717+0000 7ff097e00640 1 --2- 172.21.15.77:0/3031830562 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff098108f60 0x7ff09819ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:21.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.717+0000 7ff097e00640 1 -- 172.21.15.77:0/3031830562 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff09819f410 con 0x7ff0981047e0 2024-09-10T09:56:21.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.717+0000 7ff09dc00640 1 --2- 172.21.15.77:0/3031830562 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff098108f60 0x7ff09819ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:56:21.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.718+0000 7ff097e00640 1 --2- 172.21.15.77:0/3031830562 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff0981047e0 0x7ff09819a9a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff08400e9f0 tx=0x7ff08400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:21.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.718+0000 7ff095600640 1 -- 172.21.15.77:0/3031830562 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08400ccc0 con 0x7ff0981047e0 2024-09-10T09:56:21.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.718+0000 7ff095600640 1 -- 172.21.15.77:0/3031830562 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff084010910 con 0x7ff0981047e0 2024-09-10T09:56:21.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.718+0000 7ff095600640 1 -- 172.21.15.77:0/3031830562 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff084018960 con 0x7ff0981047e0 2024-09-10T09:56:21.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.718+0000 7ff09e600640 1 -- 172.21.15.77:0/3031830562 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff09819f700 con 0x7ff0981047e0 2024-09-10T09:56:21.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.718+0000 7ff09e600640 1 -- 172.21.15.77:0/3031830562 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0981a6fe0 con 0x7ff0981047e0 2024-09-10T09:56:21.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.723+0000 7ff0a2832740 1 Processor -- start 2024-09-10T09:56:21.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.723+0000 7ff0a2832740 1 -- start start 2024-09-10T09:56:21.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.723+0000 7ff095600640 1 -- 172.21.15.77:0/3031830562 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff084010ac0 con 0x7ff0981047e0 2024-09-10T09:56:21.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.724+0000 7ff0a2832740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556af05fbe40 0x556af05fc220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:21.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.724+0000 7ff0a2832740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556af05fc760 0x556af0606ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:21.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.724+0000 7ff0a2832740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556af0607020 0x556af0609410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:21.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.724+0000 7ff0a2832740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x556af05623e0 con 0x556af05fc760 2024-09-10T09:56:21.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.724+0000 7ff0a2832740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x556af0509d80 con 0x556af0607020 2024-09-10T09:56:21.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.724+0000 7ff0a2832740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556af055e3a0 con 0x556af05fbe40 2024-09-10T09:56:21.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.724+0000 7ff09dc00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556af05fbe40 0x556af05fc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:21.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.724+0000 7ff09dc00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556af05fbe40 0x556af05fc220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:54762/0 (socket says 172.21.15.77:54762) 2024-09-10T09:56:21.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.724+0000 7ff09dc00640 1 -- 172.21.15.77:0/3112144611 learned_addr learned my addr 172.21.15.77:0/3112144611 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:21.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.725+0000 7ff09d200640 1 --2- 172.21.15.77:0/3112144611 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556af05fc760 0x556af0606ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:21.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.725+0000 7ff097e00640 1 --2- 172.21.15.77:0/3112144611 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556af0607020 0x556af0609410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:21.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.725+0000 7ff097e00640 1 -- 172.21.15.77:0/3112144611 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556af05fbe40 msgr2=0x556af05fc220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:21.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.725+0000 7ff097e00640 1 --2- 172.21.15.77:0/3112144611 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556af05fbe40 0x556af05fc220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:21.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.725+0000 7ff097e00640 1 -- 172.21.15.77:0/3112144611 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556af05fc760 msgr2=0x556af0606ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:21.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.725+0000 7ff097e00640 1 --2- 172.21.15.77:0/3112144611 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556af05fc760 0x556af0606ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:21.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.725+0000 7ff097e00640 1 -- 172.21.15.77:0/3112144611 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556af0528ae0 con 0x556af0607020 2024-09-10T09:56:21.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.725+0000 7ff09d200640 1 --2- 172.21.15.77:0/3112144611 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556af05fc760 0x556af0606ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:56:21.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.726+0000 7ff097e00640 1 --2- 172.21.15.77:0/3112144611 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556af0607020 0x556af0609410 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff0840cceb0 tx=0x7ff0840d7670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2ff4ffa23bd752f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:21.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.726+0000 7ff095600640 1 --2- 172.21.15.77:0/3031830562 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ff0800a2b80 0x7ff0800a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:21.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.726+0000 7ff095600640 1 -- 172.21.15.77:0/3031830562 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7ff0840cf860 con 0x7ff0981047e0 2024-09-10T09:56:21.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.726+0000 7ff08a000640 1 -- 172.21.15.77:0/3112144611 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0840de070 con 0x556af0607020 2024-09-10T09:56:21.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.726+0000 7ff08a000640 1 -- 172.21.15.77:0/3112144611 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff08409c370 con 0x556af0607020 2024-09-10T09:56:21.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.726+0000 7ff08a000640 1 -- 172.21.15.77:0/3112144611 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08409ba40 con 0x556af0607020 2024-09-10T09:56:21.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.727+0000 7ff09dc00640 1 --2- 172.21.15.77:0/3031830562 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ff0800a2b80 0x7ff0800a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:21.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.727+0000 7ff0a2832740 1 -- 172.21.15.77:0/3112144611 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556af0607020 msgr2=0x556af0609410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:21.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.727+0000 7ff0a2832740 1 --2- 172.21.15.77:0/3112144611 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556af0607020 0x556af0609410 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff0840cceb0 tx=0x7ff0840d7670 comp rx=0 tx=0).stop 2024-09-10T09:56:21.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.728+0000 7ff09dc00640 1 --2- 172.21.15.77:0/3031830562 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ff0800a2b80 0x7ff0800a5040 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ff09002ae00 tx=0x7ff090009360 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:21.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.728+0000 7ff0a2832740 1 -- 172.21.15.77:0/3112144611 shutdown_connections 2024-09-10T09:56:21.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.728+0000 7ff0a2832740 1 --2- 172.21.15.77:0/3112144611 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556af0607020 0x556af0609410 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:21.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.728+0000 7ff0a2832740 1 --2- 172.21.15.77:0/3112144611 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556af05fc760 0x556af0606ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:21.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.728+0000 7ff0a2832740 1 --2- 172.21.15.77:0/3112144611 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556af05fbe40 0x556af05fc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:21.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.728+0000 7ff0a2832740 1 -- 172.21.15.77:0/3112144611 >> 172.21.15.77:0/3112144611 conn(0x556af05f83d0 msgr2=0x556af05f87b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:21.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.728+0000 7ff0a2832740 1 -- 172.21.15.77:0/3112144611 shutdown_connections 2024-09-10T09:56:21.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.728+0000 7ff0a2832740 1 -- 172.21.15.77:0/3112144611 wait complete. 2024-09-10T09:56:21.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.729+0000 7ff0a2832740 1 -- start start 2024-09-10T09:56:21.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.729+0000 7ff0a2832740 1 Processor -- start 2024-09-10T09:56:21.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.730+0000 7ff0a2832740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556af05fbe40 0x556af0603ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:21.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.730+0000 7ff0a2832740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556af05fc760 0x556af0604410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:21.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.730+0000 7ff0a2832740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556af0607020 0x556af0604950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:21.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.730+0000 7ff0a2832740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x556af0609bf0 con 0x556af05fbe40 2024-09-10T09:56:21.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.730+0000 7ff0a2832740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x556af051d0f0 con 0x556af05fc760 2024-09-10T09:56:21.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.730+0000 7ff0a2832740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556af0609d70 con 0x556af0607020 2024-09-10T09:56:21.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.730+0000 7ff0a2832740 10 client.0 _ll_register_callbacks cb 0x556af0504bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:56:21.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.730+0000 7ff097e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556af05fc760 0x556af0604410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:21.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.730+0000 7ff097e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556af05fc760 0x556af0604410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:59012/0 (socket says 172.21.15.77:59012) 2024-09-10T09:56:21.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.730+0000 7ff097e00640 1 -- 172.21.15.77:0/430640930 learned_addr learned my addr 172.21.15.77:0/430640930 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:21.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.730+0000 7ff09d200640 1 --2- 172.21.15.77:0/430640930 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556af05fbe40 0x556af0603ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:21.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.731+0000 7ff097e00640 1 -- 172.21.15.77:0/430640930 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556af0607020 msgr2=0x556af0604950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:21.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.731+0000 7ff097e00640 1 --2- 172.21.15.77:0/430640930 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556af0607020 0x556af0604950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:21.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.731+0000 7ff097e00640 1 -- 172.21.15.77:0/430640930 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556af05fbe40 msgr2=0x556af0603ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:21.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.731+0000 7ff097e00640 1 --2- 172.21.15.77:0/430640930 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556af05fbe40 0x556af0603ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:21.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.731+0000 7ff097e00640 1 -- 172.21.15.77:0/430640930 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556af02d33c0 con 0x556af05fc760 2024-09-10T09:56:21.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.731+0000 7ff09d200640 1 --2- 172.21.15.77:0/430640930 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556af05fbe40 0x556af0603ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:56:21.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.731+0000 7ff097e00640 1 --2- 172.21.15.77:0/430640930 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556af05fc760 0x556af0604410 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff0840cce80 tx=0x7ff0840cfae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3c2cb4f245af76a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:21.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.732+0000 7ff07ea00640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T09:56:21.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.732+0000 7ff07ea00640 1 -- 172.21.15.77:0/430640930 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0840de030 con 0x556af05fc760 2024-09-10T09:56:21.734 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.732+0000 7ff07ea00640 1 -- 172.21.15.77:0/430640930 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7ff0840d56d0 con 0x556af05fc760 2024-09-10T09:56:21.734 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.732+0000 7ff0a2832740 1 -- 172.21.15.77:0/430640930 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff08409c370 con 0x556af05fc760 2024-09-10T09:56:21.734 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.732+0000 7ff07ea00640 1 -- 172.21.15.77:0/430640930 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff08409b5d0 con 0x556af05fc760 2024-09-10T09:56:21.734 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.732+0000 7ff0a2832740 10 client.25652 fetch_fsmap learned FSMap version 29 2024-09-10T09:56:21.734 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.732+0000 7ff0a2832740 1 -- 172.21.15.77:0/430640930 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556af0615c30 con 0x556af05fc760 2024-09-10T09:56:21.734 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.733+0000 7ff07ea00640 1 -- 172.21.15.77:0/430640930 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff08409c370 con 0x556af05fc760 2024-09-10T09:56:21.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.733+0000 7ff07ea00640 1 -- 172.21.15.77:0/430640930 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7ff08409bed0 con 0x556af05fc760 2024-09-10T09:56:21.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.733+0000 7ff0a2832740 10 client.25652 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T09:56:21.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.733+0000 7ff0a2832740 10 client.25652 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-10T09:56:21.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.733+0000 7ff0a2832740 20 client.25652 populate_metadata read hostname 'smithi077' 2024-09-10T09:56:21.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.733+0000 7ff0a2832740 1 --2- 172.21.15.77:0/430640930 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x556af0616f80 0x556af0619370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:21.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.733+0000 7ff0a2832740 4 client.25652 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:56:21.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.733+0000 7ff0a2832740 1 -- 172.21.15.77:0/430640930 --> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556af056f950 con 0x556af0616f80 2024-09-10T09:56:21.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.734+0000 7ff09d200640 1 --2- 172.21.15.77:0/430640930 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x556af0616f80 0x556af0619370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:21.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.735+0000 7ff09d200640 1 --2- 172.21.15.77:0/430640930 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x556af0616f80 0x556af0619370 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=37e36a69073139fa server_cookie=5c44ed31f15098f in_seq=0 out_seq=0 2024-09-10T09:56:21.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.735+0000 7ff07ea00640 10 client.25652 ms_handle_connect on v2:172.21.15.134:6834/3293933699 2024-09-10T09:56:21.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.735+0000 7ff07ea00640 1 -- 172.21.15.77:0/430640930 <== mds.4 v2:172.21.15.134:6834/3293933699 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556af056f950 con 0x556af0616f80 2024-09-10T09:56:21.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.735+0000 7ff07ea00640 10 client.25652 handle_command_reply: tid=0 2024-09-10T09:56:21.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.736+0000 7ff0a2832740 1 client.25652 shutdown 2024-09-10T09:56:21.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.736+0000 7ff0a2832740 1 -- 172.21.15.77:0/430640930 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556af05fc760 msgr2=0x556af0604410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:21.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.736+0000 7ff0a2832740 1 --2- 172.21.15.77:0/430640930 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556af05fc760 0x556af0604410 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff0840cce80 tx=0x7ff0840cfae0 comp rx=0 tx=0).stop 2024-09-10T09:56:21.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.736+0000 7ff0a2832740 1 -- 172.21.15.77:0/430640930 shutdown_connections 2024-09-10T09:56:21.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.736+0000 7ff0a2832740 1 --2- 172.21.15.77:0/430640930 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x556af0616f80 0x556af0619370 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:21.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.736+0000 7ff0a2832740 1 --2- 172.21.15.77:0/430640930 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556af0607020 0x556af0604950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:21.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.736+0000 7ff0a2832740 1 --2- 172.21.15.77:0/430640930 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556af05fc760 0x556af0604410 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:21.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.736+0000 7ff0a2832740 1 --2- 172.21.15.77:0/430640930 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556af05fbe40 0x556af0603ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:21.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.736+0000 7ff0a2832740 1 -- 172.21.15.77:0/430640930 >> 172.21.15.77:0/430640930 conn(0x556af05f83d0 msgr2=0x556af05fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:21.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.736+0000 7ff07ea00640 0 client.25652 ms_handle_reset on v2:172.21.15.134:6834/3293933699 2024-09-10T09:56:21.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.738+0000 7ff0a2832740 1 -- 172.21.15.77:0/430640930 shutdown_connections 2024-09-10T09:56:21.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.738+0000 7ff0a2832740 1 -- 172.21.15.77:0/430640930 wait complete. 2024-09-10T09:56:21.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.738+0000 7ff0a2832740 20 client.25652 trim_cache size 0 max 16384 2024-09-10T09:56:21.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.763+0000 7ff0a2832740 1 Processor -- start 2024-09-10T09:56:21.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.764+0000 7ff0a2832740 1 -- start start 2024-09-10T09:56:21.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.764+0000 7ff0a2832740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556af05fbe40 0x556af05f7380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:21.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.764+0000 7ff0a2832740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556af05fc760 0x556af061c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:21.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.764+0000 7ff0a2832740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556af0607020 0x556af061c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:21.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.764+0000 7ff0a2832740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff0981b1030 con 0x556af05fbe40 2024-09-10T09:56:21.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.764+0000 7ff0a2832740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff0981b2b70 con 0x556af05fc760 2024-09-10T09:56:21.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.764+0000 7ff0a2832740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff0981b1700 con 0x556af0607020 2024-09-10T09:56:21.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.765+0000 7ff09d200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556af05fbe40 0x556af05f7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:21.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.765+0000 7ff097e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556af05fc760 0x556af061c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:21.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.765+0000 7ff097e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556af05fc760 0x556af061c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:59028/0 (socket says 172.21.15.77:59028) 2024-09-10T09:56:21.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.765+0000 7ff09d200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556af05fbe40 0x556af05f7380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42176/0 (socket says 172.21.15.77:42176) 2024-09-10T09:56:21.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.765+0000 7ff097e00640 1 -- 172.21.15.77:0/2902023958 learned_addr learned my addr 172.21.15.77:0/2902023958 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:21.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.765+0000 7ff09dc00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556af0607020 0x556af061c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:21.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.765+0000 7ff09dc00640 1 -- 172.21.15.77:0/2902023958 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556af05fc760 msgr2=0x556af061c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:21.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.765+0000 7ff09dc00640 1 --2- 172.21.15.77:0/2902023958 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556af05fc760 0x556af061c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:21.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.766+0000 7ff09dc00640 1 -- 172.21.15.77:0/2902023958 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556af05fbe40 msgr2=0x556af05f7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:21.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.766+0000 7ff09dc00640 1 --2- 172.21.15.77:0/2902023958 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556af05fbe40 0x556af05f7380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:21.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.766+0000 7ff09dc00640 1 -- 172.21.15.77:0/2902023958 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556af05fc590 con 0x556af0607020 2024-09-10T09:56:21.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.766+0000 7ff09dc00640 1 --2- 172.21.15.77:0/2902023958 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556af0607020 0x556af061c8d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff090009d90 tx=0x7ff09003e040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d79487975256f18 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:21.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.766+0000 7ff09d200640 1 --2- 172.21.15.77:0/2902023958 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556af05fbe40 0x556af05f7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:56:21.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.766+0000 7ff089600640 1 -- 172.21.15.77:0/2902023958 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff090004240 con 0x556af0607020 2024-09-10T09:56:21.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.766+0000 7ff089600640 1 -- 172.21.15.77:0/2902023958 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff090036040 con 0x556af0607020 2024-09-10T09:56:21.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.766+0000 7ff089600640 1 -- 172.21.15.77:0/2902023958 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff090050480 con 0x556af0607020 2024-09-10T09:56:21.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.767+0000 7ff0a2832740 1 -- 172.21.15.77:0/2902023958 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556af0607020 msgr2=0x556af061c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:21.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.767+0000 7ff0a2832740 1 --2- 172.21.15.77:0/2902023958 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556af0607020 0x556af061c8d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff090009d90 tx=0x7ff09003e040 comp rx=0 tx=0).stop 2024-09-10T09:56:21.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.768+0000 7ff0a2832740 1 -- 172.21.15.77:0/2902023958 shutdown_connections 2024-09-10T09:56:21.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.768+0000 7ff0a2832740 1 --2- 172.21.15.77:0/2902023958 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556af0607020 0x556af061c8d0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:21.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.768+0000 7ff0a2832740 1 --2- 172.21.15.77:0/2902023958 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556af05fc760 0x556af061c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:21.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.768+0000 7ff0a2832740 1 --2- 172.21.15.77:0/2902023958 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556af05fbe40 0x556af05f7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:21.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.768+0000 7ff0a2832740 1 -- 172.21.15.77:0/2902023958 >> 172.21.15.77:0/2902023958 conn(0x556af05f83d0 msgr2=0x556af05f6c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:21.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.768+0000 7ff0a2832740 1 -- 172.21.15.77:0/2902023958 shutdown_connections 2024-09-10T09:56:21.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.768+0000 7ff0a2832740 1 -- 172.21.15.77:0/2902023958 wait complete. 2024-09-10T09:56:21.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.769+0000 7ff0a2832740 1 -- start start 2024-09-10T09:56:21.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.769+0000 7ff0a2832740 1 Processor -- start 2024-09-10T09:56:21.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.770+0000 7ff0a2832740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556af05fbe40 0x556af061a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:21.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.770+0000 7ff0a2832740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556af05fc760 0x556af061a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:21.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.770+0000 7ff0a2832740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556af0607020 0x556af061ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:21.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.770+0000 7ff0a2832740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff0981b24a0 con 0x556af05fbe40 2024-09-10T09:56:21.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.770+0000 7ff0a2832740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff0981b1dd0 con 0x556af05fc760 2024-09-10T09:56:21.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.770+0000 7ff0a2832740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff0981af010 con 0x556af0607020 2024-09-10T09:56:21.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.770+0000 7ff0a2832740 10 client.0 _ll_register_callbacks cb 0x7ff08c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:56:21.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.770+0000 7ff09dc00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556af0607020 0x556af061ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:21.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.770+0000 7ff09dc00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556af0607020 0x556af061ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:54784/0 (socket says 172.21.15.77:54784) 2024-09-10T09:56:21.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.770+0000 7ff09dc00640 1 -- 172.21.15.77:0/2628135724 learned_addr learned my addr 172.21.15.77:0/2628135724 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:21.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.770+0000 7ff09d200640 1 --2- 172.21.15.77:0/2628135724 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556af05fbe40 0x556af061a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:21.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.771+0000 7ff09dc00640 1 -- 172.21.15.77:0/2628135724 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556af05fc760 msgr2=0x556af061a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:21.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.771+0000 7ff09dc00640 1 --2- 172.21.15.77:0/2628135724 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556af05fc760 0x556af061a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:21.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.771+0000 7ff09dc00640 1 -- 172.21.15.77:0/2628135724 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556af05fbe40 msgr2=0x556af061a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:21.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.771+0000 7ff09dc00640 1 --2- 172.21.15.77:0/2628135724 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556af05fbe40 0x556af061a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:21.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.771+0000 7ff09dc00640 1 -- 172.21.15.77:0/2628135724 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556af0689240 con 0x556af0607020 2024-09-10T09:56:21.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.771+0000 7ff09d200640 1 --2- 172.21.15.77:0/2628135724 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556af05fbe40 0x556af061a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:56:21.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.771+0000 7ff09dc00640 1 --2- 172.21.15.77:0/2628135724 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556af0607020 0x556af061ad00 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff0900307f0 tx=0x7ff090030820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c47c3dee5fb20b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:21.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.771+0000 7ff07ea00640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-10T09:56:21.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.772+0000 7ff07ea00640 1 -- 172.21.15.77:0/2628135724 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff09005e060 con 0x556af0607020 2024-09-10T09:56:21.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.772+0000 7ff07ea00640 1 -- 172.21.15.77:0/2628135724 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7ff090004250 con 0x556af0607020 2024-09-10T09:56:21.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.772+0000 7ff0a2832740 1 -- 172.21.15.77:0/2628135724 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff090036040 con 0x556af0607020 2024-09-10T09:56:21.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.772+0000 7ff07ea00640 1 -- 172.21.15.77:0/2628135724 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff0900309c0 con 0x556af0607020 2024-09-10T09:56:21.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.772+0000 7ff0a2832740 10 client.25621 fetch_fsmap learned FSMap version 29 2024-09-10T09:56:21.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.772+0000 7ff0a2832740 1 -- 172.21.15.77:0/2628135724 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556af068a500 con 0x556af0607020 2024-09-10T09:56:21.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.773+0000 7ff07ea00640 1 -- 172.21.15.77:0/2628135724 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff090050480 con 0x556af0607020 2024-09-10T09:56:21.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.773+0000 7ff07ea00640 1 -- 172.21.15.77:0/2628135724 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7ff090043040 con 0x556af0607020 2024-09-10T09:56:21.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.773+0000 7ff0a2832740 10 client.25621 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T09:56:21.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.773+0000 7ff0a2832740 10 client.25621 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-10T09:56:21.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.773+0000 7ff0a2832740 20 client.25621 populate_metadata read hostname 'smithi077' 2024-09-10T09:56:21.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.773+0000 7ff0a2832740 1 --2- 172.21.15.77:0/2628135724 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x556af068ab70 0x556af068cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:21.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.773+0000 7ff0a2832740 4 client.25621 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:56:21.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.773+0000 7ff0a2832740 1 -- 172.21.15.77:0/2628135724 --> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7ff090036040 con 0x556af068ab70 2024-09-10T09:56:21.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.774+0000 7ff09d200640 1 --2- 172.21.15.77:0/2628135724 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x556af068ab70 0x556af068cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:21.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.774+0000 7ff09d200640 1 --2- 172.21.15.77:0/2628135724 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x556af068ab70 0x556af068cf90 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a5241bdfb6b623b0 server_cookie=35e8ecd6c68735bf in_seq=0 out_seq=0 2024-09-10T09:56:21.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.774+0000 7ff07ea00640 10 client.25621 ms_handle_connect on v2:172.21.15.134:6834/3293933699 2024-09-10T09:56:21.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.774+0000 7ff07ea00640 1 -- 172.21.15.77:0/2628135724 <== mds.4 v2:172.21.15.134:6834/3293933699 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7ff090036040 con 0x556af068ab70 2024-09-10T09:56:21.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.774+0000 7ff07ea00640 10 client.25621 handle_command_reply: tid=0 2024-09-10T09:56:21.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.774+0000 7ff0a2832740 1 client.25621 shutdown 2024-09-10T09:56:21.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.775+0000 7ff0a2832740 1 -- 172.21.15.77:0/2628135724 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556af0607020 msgr2=0x556af061ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:21.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.775+0000 7ff0a2832740 1 --2- 172.21.15.77:0/2628135724 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556af0607020 0x556af061ad00 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff0900307f0 tx=0x7ff090030820 comp rx=0 tx=0).stop 2024-09-10T09:56:21.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.775+0000 7ff0a2832740 1 -- 172.21.15.77:0/2628135724 shutdown_connections 2024-09-10T09:56:21.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.775+0000 7ff0a2832740 1 --2- 172.21.15.77:0/2628135724 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x556af068ab70 0x556af068cf90 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:21.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.775+0000 7ff0a2832740 1 --2- 172.21.15.77:0/2628135724 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556af0607020 0x556af061ad00 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:21.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.775+0000 7ff0a2832740 1 --2- 172.21.15.77:0/2628135724 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556af05fc760 0x556af061a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:21.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.775+0000 7ff0a2832740 1 --2- 172.21.15.77:0/2628135724 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556af05fbe40 0x556af061a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:21.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.775+0000 7ff0a2832740 1 -- 172.21.15.77:0/2628135724 >> 172.21.15.77:0/2628135724 conn(0x556af05f83d0 msgr2=0x556af05f6c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:21.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.775+0000 7ff07ea00640 0 client.25621 ms_handle_reset on v2:172.21.15.134:6834/3293933699 2024-09-10T09:56:21.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.775+0000 7ff0a2832740 1 -- 172.21.15.77:0/2628135724 shutdown_connections 2024-09-10T09:56:21.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.775+0000 7ff0a2832740 1 -- 172.21.15.77:0/2628135724 wait complete. 2024-09-10T09:56:21.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.775+0000 7ff0a2832740 20 client.25621 trim_cache size 0 max 16384 2024-09-10T09:56:21.777 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-10T09:56:21.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.776+0000 7ff09e600640 1 -- 172.21.15.77:0/3031830562 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ff0800a2b80 msgr2=0x7ff0800a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:21.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.776+0000 7ff09e600640 1 --2- 172.21.15.77:0/3031830562 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ff0800a2b80 0x7ff0800a5040 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ff09002ae00 tx=0x7ff090009360 comp rx=0 tx=0).stop 2024-09-10T09:56:21.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.776+0000 7ff09e600640 1 -- 172.21.15.77:0/3031830562 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff0981047e0 msgr2=0x7ff09819a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:21.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.776+0000 7ff09e600640 1 --2- 172.21.15.77:0/3031830562 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff0981047e0 0x7ff09819a9a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff08400e9f0 tx=0x7ff08400eeb0 comp rx=0 tx=0).stop 2024-09-10T09:56:21.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.776+0000 7ff09e600640 1 -- 172.21.15.77:0/3031830562 shutdown_connections 2024-09-10T09:56:21.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.776+0000 7ff09e600640 1 --2- 172.21.15.77:0/3031830562 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ff0800a2b80 0x7ff0800a5040 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:21.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.776+0000 7ff09e600640 1 --2- 172.21.15.77:0/3031830562 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff098108f60 0x7ff09819ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:21.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.776+0000 7ff09e600640 1 --2- 172.21.15.77:0/3031830562 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff0981047e0 0x7ff09819a9a0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:21.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.776+0000 7ff09e600640 1 --2- 172.21.15.77:0/3031830562 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff098103e30 0x7ff09819a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:21.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.776+0000 7ff09e600640 1 -- 172.21.15.77:0/3031830562 >> 172.21.15.77:0/3031830562 conn(0x7ff0980ff910 msgr2=0x7ff098100be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:21.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.777+0000 7ff09e600640 1 -- 172.21.15.77:0/3031830562 shutdown_connections 2024-09-10T09:56:21.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:21.777+0000 7ff09e600640 1 -- 172.21.15.77:0/3031830562 wait complete. 2024-09-10T09:56:22.710 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T09:56:22.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:56:22 smithi077 ceph-mon[30319]: pgmap v427: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-10T09:56:23.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:56:22 smithi134 ceph-mon[35674]: pgmap v427: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-10T09:56:23.132 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:56:23.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:56:22 smithi120 ceph-mon[34770]: pgmap v427: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-10T09:56:23.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.734+0000 7f5c41a00640 1 -- 172.21.15.77:0/1319626336 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5c3c1049a0 msgr2=0x7f5c3c1089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:23.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.734+0000 7f5c41a00640 1 --2- 172.21.15.77:0/1319626336 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5c3c1049a0 0x7f5c3c1089f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5c28009900 tx=0x7f5c2802a3f0 comp rx=0 tx=0).stop 2024-09-10T09:56:23.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.734+0000 7f5c41a00640 1 -- 172.21.15.77:0/1319626336 shutdown_connections 2024-09-10T09:56:23.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.734+0000 7f5c41a00640 1 --2- 172.21.15.77:0/1319626336 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5c3c109120 0x7f5c3c110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:23.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.734+0000 7f5c41a00640 1 --2- 172.21.15.77:0/1319626336 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5c3c1049a0 0x7f5c3c1089f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:23.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.734+0000 7f5c41a00640 1 --2- 172.21.15.77:0/1319626336 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5c3c103ff0 0x7f5c3c1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:23.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.735+0000 7f5c41a00640 1 -- 172.21.15.77:0/1319626336 >> 172.21.15.77:0/1319626336 conn(0x7f5c3c0ffa90 msgr2=0x7f5c3c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:23.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.735+0000 7f5c41a00640 1 -- 172.21.15.77:0/1319626336 shutdown_connections 2024-09-10T09:56:23.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.735+0000 7f5c41a00640 1 -- 172.21.15.77:0/1319626336 wait complete. 2024-09-10T09:56:23.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.736+0000 7f5c41a00640 1 Processor -- start 2024-09-10T09:56:23.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.736+0000 7f5c41a00640 1 -- start start 2024-09-10T09:56:23.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.737+0000 7f5c41a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5c3c103ff0 0x7f5c3c19a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:23.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.737+0000 7f5c41a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5c3c1049a0 0x7f5c3c19ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:23.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.737+0000 7f5c41a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5c3c109120 0x7f5c3c19ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:23.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.737+0000 7f5c41a00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5c3c114170 con 0x7f5c3c103ff0 2024-09-10T09:56:23.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.737+0000 7f5c41a00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5c3c113ff0 con 0x7f5c3c1049a0 2024-09-10T09:56:23.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.737+0000 7f5c41a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5c3c1142f0 con 0x7f5c3c109120 2024-09-10T09:56:23.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.737+0000 7f5c3b400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5c3c109120 0x7f5c3c19ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:23.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.737+0000 7f5c3a000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5c3c1049a0 0x7f5c3c19ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:23.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.737+0000 7f5c3a000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5c3c1049a0 0x7f5c3c19ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:56666/0 (socket says 172.21.15.77:56666) 2024-09-10T09:56:23.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.737+0000 7f5c3a000640 1 -- 172.21.15.77:0/454958729 learned_addr learned my addr 172.21.15.77:0/454958729 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:23.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.738+0000 7f5c3a000640 1 -- 172.21.15.77:0/454958729 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5c3c109120 msgr2=0x7f5c3c19ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:23.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.738+0000 7f5c3a000640 1 --2- 172.21.15.77:0/454958729 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5c3c109120 0x7f5c3c19ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:23.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.738+0000 7f5c3a000640 1 -- 172.21.15.77:0/454958729 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5c3c103ff0 msgr2=0x7f5c3c19a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:23.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.738+0000 7f5c3a000640 1 --2- 172.21.15.77:0/454958729 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5c3c103ff0 0x7f5c3c19a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:23.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.738+0000 7f5c3a000640 1 -- 172.21.15.77:0/454958729 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c3c19f600 con 0x7f5c3c1049a0 2024-09-10T09:56:23.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.738+0000 7f5c3a000640 1 --2- 172.21.15.77:0/454958729 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5c3c1049a0 0x7f5c3c19ab90 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5c28004770 tx=0x7f5c28003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:23.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.739+0000 7f5c33400640 1 -- 172.21.15.77:0/454958729 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c2803f070 con 0x7f5c3c1049a0 2024-09-10T09:56:23.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.739+0000 7f5c33400640 1 -- 172.21.15.77:0/454958729 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c28031ca0 con 0x7f5c3c1049a0 2024-09-10T09:56:23.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.739+0000 7f5c41a00640 1 -- 172.21.15.77:0/454958729 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c3c19f890 con 0x7f5c3c1049a0 2024-09-10T09:56:23.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.739+0000 7f5c33400640 1 -- 172.21.15.77:0/454958729 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c28035030 con 0x7f5c3c1049a0 2024-09-10T09:56:23.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.739+0000 7f5c41a00640 1 -- 172.21.15.77:0/454958729 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c3c19fd60 con 0x7f5c3c1049a0 2024-09-10T09:56:23.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.743+0000 7f5c33400640 1 -- 172.21.15.77:0/454958729 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5c28042070 con 0x7f5c3c1049a0 2024-09-10T09:56:23.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.744+0000 7f5c443e0740 1 Processor -- start 2024-09-10T09:56:23.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.745+0000 7f5c443e0740 1 -- start start 2024-09-10T09:56:23.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.745+0000 7f5c443e0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3597bc480 0x55a3597bc860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:23.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.745+0000 7f5c443e0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a3597bcda0 0x55a3597c7120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:23.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.745+0000 7f5c443e0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3597c7660 0x55a3597c9a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:23.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.745+0000 7f5c443e0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55a35971a3e0 con 0x55a3597bc480 2024-09-10T09:56:23.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.745+0000 7f5c443e0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a3596c1d80 con 0x55a3597c7660 2024-09-10T09:56:23.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.745+0000 7f5c443e0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a3597163a0 con 0x55a3597bcda0 2024-09-10T09:56:23.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.745+0000 7f5c3a000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3597c7660 0x55a3597c9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:23.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.745+0000 7f5c3a000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3597c7660 0x55a3597c9a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:56668/0 (socket says 172.21.15.77:56668) 2024-09-10T09:56:23.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.745+0000 7f5c3a000640 1 -- 172.21.15.77:0/4166227031 learned_addr learned my addr 172.21.15.77:0/4166227031 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:23.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.745+0000 7f5c3aa00640 1 --2- 172.21.15.77:0/4166227031 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a3597bcda0 0x55a3597c7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:23.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.746+0000 7f5c3a000640 1 -- 172.21.15.77:0/4166227031 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a3597bcda0 msgr2=0x55a3597c7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:23.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.746+0000 7f5c3a000640 1 --2- 172.21.15.77:0/4166227031 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a3597bcda0 0x55a3597c7120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:23.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.746+0000 7f5c3a000640 1 -- 172.21.15.77:0/4166227031 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3597bc480 msgr2=0x55a3597bc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:23.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.746+0000 7f5c3a000640 1 --2- 172.21.15.77:0/4166227031 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3597bc480 0x55a3597bc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:23.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.746+0000 7f5c3a000640 1 -- 172.21.15.77:0/4166227031 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3596e0ae0 con 0x55a3597c7660 2024-09-10T09:56:23.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.746+0000 7f5c3a000640 1 --2- 172.21.15.77:0/4166227031 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3597c7660 0x55a3597c9a50 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5c280f5c20 tx=0x7f5c280f5d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80a0786ecc05aaec server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:23.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.746+0000 7f5c27e00640 1 -- 172.21.15.77:0/4166227031 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c280fe070 con 0x55a3597c7660 2024-09-10T09:56:23.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.746+0000 7f5c33400640 1 --2- 172.21.15.77:0/454958729 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f5c200a2c00 0x7f5c200a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:23.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.746+0000 7f5c27e00640 1 -- 172.21.15.77:0/4166227031 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c280bc220 con 0x55a3597c7660 2024-09-10T09:56:23.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.746+0000 7f5c27e00640 1 -- 172.21.15.77:0/4166227031 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c280bbb20 con 0x55a3597c7660 2024-09-10T09:56:23.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.747+0000 7f5c33400640 1 -- 172.21.15.77:0/454958729 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f5c280ef710 con 0x7f5c3c1049a0 2024-09-10T09:56:23.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.747+0000 7f5c3b400640 1 --2- 172.21.15.77:0/454958729 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f5c200a2c00 0x7f5c200a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:23.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.747+0000 7f5c443e0740 1 -- 172.21.15.77:0/4166227031 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3597c7660 msgr2=0x55a3597c9a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:23.751 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.747+0000 7f5c443e0740 1 --2- 172.21.15.77:0/4166227031 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3597c7660 0x55a3597c9a50 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5c280f5c20 tx=0x7f5c280f5d10 comp rx=0 tx=0).stop 2024-09-10T09:56:23.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.749+0000 7f5c3b400640 1 --2- 172.21.15.77:0/454958729 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f5c200a2c00 0x7f5c200a50c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f5c34005fd0 tx=0x7f5c34009880 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:23.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.749+0000 7f5c443e0740 1 -- 172.21.15.77:0/4166227031 shutdown_connections 2024-09-10T09:56:23.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.749+0000 7f5c443e0740 1 --2- 172.21.15.77:0/4166227031 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3597c7660 0x55a3597c9a50 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:23.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.749+0000 7f5c443e0740 1 --2- 172.21.15.77:0/4166227031 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a3597bcda0 0x55a3597c7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:23.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.749+0000 7f5c443e0740 1 --2- 172.21.15.77:0/4166227031 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3597bc480 0x55a3597bc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:23.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.749+0000 7f5c443e0740 1 -- 172.21.15.77:0/4166227031 >> 172.21.15.77:0/4166227031 conn(0x55a3597b8a10 msgr2=0x55a3597b8df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:23.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.749+0000 7f5c443e0740 1 -- 172.21.15.77:0/4166227031 shutdown_connections 2024-09-10T09:56:23.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.749+0000 7f5c443e0740 1 -- 172.21.15.77:0/4166227031 wait complete. 2024-09-10T09:56:23.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.750+0000 7f5c443e0740 1 -- start start 2024-09-10T09:56:23.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.750+0000 7f5c443e0740 1 Processor -- start 2024-09-10T09:56:23.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.750+0000 7f5c443e0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a3597bc480 0x55a3597b5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:23.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.750+0000 7f5c443e0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3597bcda0 0x55a3597b5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:23.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.750+0000 7f5c443e0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3597c7660 0x55a3597b5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:23.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.750+0000 7f5c443e0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55a3597afbd0 con 0x55a3597bcda0 2024-09-10T09:56:23.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.750+0000 7f5c443e0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a3596d50f0 con 0x55a3597c7660 2024-09-10T09:56:23.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.750+0000 7f5c443e0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a3597afd50 con 0x55a3597bc480 2024-09-10T09:56:23.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.750+0000 7f5c443e0740 10 client.0 _ll_register_callbacks cb 0x55a3596bcbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:56:23.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.751+0000 7f5c3aa00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a3597bc480 0x55a3597b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:23.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.751+0000 7f5c3a000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3597bcda0 0x55a3597b5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:23.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.751+0000 7f5c3b400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3597c7660 0x55a3597b5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:23.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.751+0000 7f5c3a000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3597bcda0 0x55a3597b5970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:53332/0 (socket says 172.21.15.77:53332) 2024-09-10T09:56:23.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.751+0000 7f5c3a000640 1 -- 172.21.15.77:0/110743820 learned_addr learned my addr 172.21.15.77:0/110743820 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:23.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.752+0000 7f5c3b400640 1 -- 172.21.15.77:0/110743820 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a3597bc480 msgr2=0x55a3597b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:23.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.752+0000 7f5c3b400640 1 --2- 172.21.15.77:0/110743820 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a3597bc480 0x55a3597b5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:23.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.752+0000 7f5c3b400640 1 -- 172.21.15.77:0/110743820 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3597bcda0 msgr2=0x55a3597b5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:23.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.752+0000 7f5c3b400640 1 --2- 172.21.15.77:0/110743820 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3597bcda0 0x55a3597b5970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:23.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.752+0000 7f5c3b400640 1 -- 172.21.15.77:0/110743820 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a35948b3c0 con 0x55a3597c7660 2024-09-10T09:56:23.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.752+0000 7f5c3a000640 1 --2- 172.21.15.77:0/110743820 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3597bcda0 0x55a3597b5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:56:23.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.752+0000 7f5c3aa00640 1 --2- 172.21.15.77:0/110743820 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a3597bc480 0x55a3597b5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:56:23.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.752+0000 7f5c3b400640 1 --2- 172.21.15.77:0/110743820 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3597c7660 0x55a3597b5eb0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5c34011ca0 tx=0x7f5c34009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8a1127d2b08094b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:23.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.752+0000 7f5c24c00640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T09:56:23.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.753+0000 7f5c24c00640 1 -- 172.21.15.77:0/110743820 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c3400fa40 con 0x55a3597c7660 2024-09-10T09:56:23.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.753+0000 7f5c24c00640 1 -- 172.21.15.77:0/110743820 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f5c3400fbe0 con 0x55a3597c7660 2024-09-10T09:56:23.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.753+0000 7f5c443e0740 1 -- 172.21.15.77:0/110743820 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c280bc220 con 0x55a3597c7660 2024-09-10T09:56:23.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.753+0000 7f5c24c00640 1 -- 172.21.15.77:0/110743820 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5c3400fde0 con 0x55a3597c7660 2024-09-10T09:56:23.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.753+0000 7f5c443e0740 10 client.25667 fetch_fsmap learned FSMap version 29 2024-09-10T09:56:23.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.753+0000 7f5c443e0740 1 -- 172.21.15.77:0/110743820 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a3597cd7a0 con 0x55a3597c7660 2024-09-10T09:56:23.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.754+0000 7f5c24c00640 1 -- 172.21.15.77:0/110743820 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c280bc220 con 0x55a3597c7660 2024-09-10T09:56:23.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.755+0000 7f5c24c00640 1 -- 172.21.15.77:0/110743820 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f5c3401d6e0 con 0x55a3597c7660 2024-09-10T09:56:23.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.755+0000 7f5c443e0740 10 client.25667 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T09:56:23.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.755+0000 7f5c443e0740 10 client.25667 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T09:56:23.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.755+0000 7f5c443e0740 20 client.25667 populate_metadata read hostname 'smithi077' 2024-09-10T09:56:23.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.755+0000 7f5c443e0740 1 --2- 172.21.15.77:0/110743820 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x55a3597cec60 0x55a3597d1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:23.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.755+0000 7f5c443e0740 4 client.25667 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:56:23.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.755+0000 7f5c443e0740 1 -- 172.21.15.77:0/110743820 --> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a359727950 con 0x55a3597cec60 2024-09-10T09:56:23.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.755+0000 7f5c3aa00640 1 --2- 172.21.15.77:0/110743820 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x55a3597cec60 0x55a3597d1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:23.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.756+0000 7f5c3aa00640 1 --2- 172.21.15.77:0/110743820 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x55a3597cec60 0x55a3597d1050 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=cb51f189344c4da5 server_cookie=6d90f06256e99c9b in_seq=0 out_seq=0 2024-09-10T09:56:23.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.756+0000 7f5c24c00640 10 client.25667 ms_handle_connect on v2:172.21.15.134:6836/2461497091 2024-09-10T09:56:23.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.757+0000 7f5c24c00640 1 -- 172.21.15.77:0/110743820 <== mds.3 v2:172.21.15.134:6836/2461497091 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a359727950 con 0x55a3597cec60 2024-09-10T09:56:23.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.757+0000 7f5c24c00640 10 client.25667 handle_command_reply: tid=0 2024-09-10T09:56:23.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.757+0000 7f5c443e0740 1 client.25667 shutdown 2024-09-10T09:56:23.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.757+0000 7f5c443e0740 1 -- 172.21.15.77:0/110743820 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3597c7660 msgr2=0x55a3597b5eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:23.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.757+0000 7f5c443e0740 1 --2- 172.21.15.77:0/110743820 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3597c7660 0x55a3597b5eb0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5c34011ca0 tx=0x7f5c34009500 comp rx=0 tx=0).stop 2024-09-10T09:56:23.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.758+0000 7f5c443e0740 1 -- 172.21.15.77:0/110743820 shutdown_connections 2024-09-10T09:56:23.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.758+0000 7f5c443e0740 1 --2- 172.21.15.77:0/110743820 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3597c7660 0x55a3597b5eb0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:23.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.758+0000 7f5c443e0740 1 --2- 172.21.15.77:0/110743820 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x55a3597cec60 0x55a3597d1050 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:23.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.758+0000 7f5c443e0740 1 --2- 172.21.15.77:0/110743820 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3597bcda0 0x55a3597b5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:23.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.758+0000 7f5c443e0740 1 --2- 172.21.15.77:0/110743820 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a3597bc480 0x55a3597b5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:23.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.758+0000 7f5c24c00640 0 client.25667 ms_handle_reset on v2:172.21.15.134:6836/2461497091 2024-09-10T09:56:23.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.758+0000 7f5c443e0740 1 -- 172.21.15.77:0/110743820 >> 172.21.15.77:0/110743820 conn(0x55a3597b8a10 msgr2=0x55a3597af380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:23.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.758+0000 7f5c443e0740 1 -- 172.21.15.77:0/110743820 shutdown_connections 2024-09-10T09:56:23.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.758+0000 7f5c443e0740 1 -- 172.21.15.77:0/110743820 wait complete. 2024-09-10T09:56:23.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.758+0000 7f5c443e0740 20 client.25667 trim_cache size 0 max 16384 2024-09-10T09:56:23.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.772+0000 7f5c443e0740 1 Processor -- start 2024-09-10T09:56:23.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.772+0000 7f5c443e0740 1 -- start start 2024-09-10T09:56:23.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.773+0000 7f5c443e0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a3597bc480 0x55a3597b5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:23.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.773+0000 7f5c443e0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3597bcda0 0x55a3597d68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:23.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.773+0000 7f5c443e0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3597c7660 0x55a3597d6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:23.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.773+0000 7f5c443e0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5c3c1acc50 con 0x55a3597c7660 2024-09-10T09:56:23.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.773+0000 7f5c443e0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5c3c1ae790 con 0x55a3597bcda0 2024-09-10T09:56:23.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.773+0000 7f5c443e0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5c3c1ad320 con 0x55a3597bc480 2024-09-10T09:56:23.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.773+0000 7f5c3aa00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a3597bc480 0x55a3597b5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:23.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.773+0000 7f5c3aa00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a3597bc480 0x55a3597b5d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:56368/0 (socket says 172.21.15.77:56368) 2024-09-10T09:56:23.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.773+0000 7f5c3aa00640 1 -- 172.21.15.77:0/2177939593 learned_addr learned my addr 172.21.15.77:0/2177939593 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:23.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.773+0000 7f5c3a000640 1 --2- 172.21.15.77:0/2177939593 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3597bcda0 0x55a3597d68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:23.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.773+0000 7f5c3b400640 1 --2- 172.21.15.77:0/2177939593 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3597c7660 0x55a3597d6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:23.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.774+0000 7f5c3aa00640 1 -- 172.21.15.77:0/2177939593 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3597bcda0 msgr2=0x55a3597d68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:23.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.774+0000 7f5c3aa00640 1 --2- 172.21.15.77:0/2177939593 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3597bcda0 0x55a3597d68d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:23.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.774+0000 7f5c3aa00640 1 -- 172.21.15.77:0/2177939593 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3597c7660 msgr2=0x55a3597d6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:23.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.774+0000 7f5c3aa00640 1 --2- 172.21.15.77:0/2177939593 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3597c7660 0x55a3597d6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:23.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.774+0000 7f5c3aa00640 1 -- 172.21.15.77:0/2177939593 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3597bcbd0 con 0x55a3597bc480 2024-09-10T09:56:23.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.774+0000 7f5c3b400640 1 --2- 172.21.15.77:0/2177939593 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3597c7660 0x55a3597d6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:56:23.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.775+0000 7f5c3aa00640 1 --2- 172.21.15.77:0/2177939593 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a3597bc480 0x55a3597b5d00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x55a3597d1870 tx=0x7f5c2c004410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ed887a658032393 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:23.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.775+0000 7f5c27400640 1 -- 172.21.15.77:0/2177939593 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c2c003d60 con 0x55a3597bc480 2024-09-10T09:56:23.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.775+0000 7f5c27400640 1 -- 172.21.15.77:0/2177939593 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c2c02e900 con 0x55a3597bc480 2024-09-10T09:56:23.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.775+0000 7f5c27400640 1 -- 172.21.15.77:0/2177939593 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c2c021a90 con 0x55a3597bc480 2024-09-10T09:56:23.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.776+0000 7f5c443e0740 1 -- 172.21.15.77:0/2177939593 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a3597bc480 msgr2=0x55a3597b5d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:23.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.776+0000 7f5c443e0740 1 --2- 172.21.15.77:0/2177939593 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a3597bc480 0x55a3597b5d00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x55a3597d1870 tx=0x7f5c2c004410 comp rx=0 tx=0).stop 2024-09-10T09:56:23.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.776+0000 7f5c443e0740 1 -- 172.21.15.77:0/2177939593 shutdown_connections 2024-09-10T09:56:23.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.776+0000 7f5c443e0740 1 --2- 172.21.15.77:0/2177939593 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3597c7660 0x55a3597d6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:23.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.776+0000 7f5c443e0740 1 --2- 172.21.15.77:0/2177939593 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3597bcda0 0x55a3597d68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:23.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.776+0000 7f5c443e0740 1 --2- 172.21.15.77:0/2177939593 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a3597bc480 0x55a3597b5d00 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:23.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.776+0000 7f5c443e0740 1 -- 172.21.15.77:0/2177939593 >> 172.21.15.77:0/2177939593 conn(0x55a3597b8a10 msgr2=0x55a3597b74b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:23.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.777+0000 7f5c443e0740 1 -- 172.21.15.77:0/2177939593 shutdown_connections 2024-09-10T09:56:23.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.777+0000 7f5c443e0740 1 -- 172.21.15.77:0/2177939593 wait complete. 2024-09-10T09:56:23.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.777+0000 7f5c443e0740 1 -- start start 2024-09-10T09:56:23.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.777+0000 7f5c443e0740 1 Processor -- start 2024-09-10T09:56:23.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.778+0000 7f5c443e0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3597bc480 0x55a3597c5900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:23.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.778+0000 7f5c443e0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3597bcda0 0x55a3597c1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:23.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.778+0000 7f5c443e0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a3597c7660 0x55a3597c1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:23.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.778+0000 7f5c443e0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5c3c1ae0c0 con 0x55a3597bcda0 2024-09-10T09:56:23.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.778+0000 7f5c443e0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5c3c1ad9f0 con 0x55a3597bc480 2024-09-10T09:56:23.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.778+0000 7f5c443e0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5c3c1aac30 con 0x55a3597c7660 2024-09-10T09:56:23.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.778+0000 7f5c443e0740 10 client.0 _ll_register_callbacks cb 0x7f5c34009d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:56:23.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.779+0000 7f5c3b400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a3597c7660 0x55a3597c1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:23.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.779+0000 7f5c3b400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a3597c7660 0x55a3597c1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:56370/0 (socket says 172.21.15.77:56370) 2024-09-10T09:56:23.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.779+0000 7f5c3b400640 1 -- 172.21.15.77:0/1338459070 learned_addr learned my addr 172.21.15.77:0/1338459070 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:23.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.779+0000 7f5c3aa00640 1 --2- 172.21.15.77:0/1338459070 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3597bc480 0x55a3597c5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:23.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.779+0000 7f5c3b400640 1 -- 172.21.15.77:0/1338459070 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3597bc480 msgr2=0x55a3597c5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:23.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.779+0000 7f5c3b400640 1 --2- 172.21.15.77:0/1338459070 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3597bc480 0x55a3597c5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:23.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.779+0000 7f5c3b400640 1 -- 172.21.15.77:0/1338459070 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3597bcda0 msgr2=0x55a3597c1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:23.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.779+0000 7f5c3a000640 1 --2- 172.21.15.77:0/1338459070 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3597bcda0 0x55a3597c1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:23.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.780+0000 7f5c3b400640 1 --2- 172.21.15.77:0/1338459070 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3597bcda0 0x55a3597c1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:23.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.780+0000 7f5c3b400640 1 -- 172.21.15.77:0/1338459070 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3597c1bc0 con 0x55a3597c7660 2024-09-10T09:56:23.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.780+0000 7f5c3a000640 1 --2- 172.21.15.77:0/1338459070 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3597bcda0 0x55a3597c1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:56:23.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.780+0000 7f5c3b400640 1 --2- 172.21.15.77:0/1338459070 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a3597c7660 0x55a3597c1580 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5c34011d40 tx=0x7f5c3401f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d8d1f3ca1c85322 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:23.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.780+0000 7f5c24c00640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-10T09:56:23.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.780+0000 7f5c24c00640 1 -- 172.21.15.77:0/1338459070 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c3401c740 con 0x55a3597c7660 2024-09-10T09:56:23.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.781+0000 7f5c24c00640 1 -- 172.21.15.77:0/1338459070 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f5c34003b70 con 0x55a3597c7660 2024-09-10T09:56:23.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.781+0000 7f5c443e0740 1 -- 172.21.15.77:0/1338459070 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c2c02e900 con 0x55a3597c7660 2024-09-10T09:56:23.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.781+0000 7f5c24c00640 1 -- 172.21.15.77:0/1338459070 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5c34005bd0 con 0x55a3597c7660 2024-09-10T09:56:23.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.781+0000 7f5c443e0740 10 client.25639 fetch_fsmap learned FSMap version 29 2024-09-10T09:56:23.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.781+0000 7f5c443e0740 1 -- 172.21.15.77:0/1338459070 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a359839710 con 0x55a3597c7660 2024-09-10T09:56:23.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.782+0000 7f5c24c00640 1 -- 172.21.15.77:0/1338459070 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c2c02e900 con 0x55a3597c7660 2024-09-10T09:56:23.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.782+0000 7f5c24c00640 1 -- 172.21.15.77:0/1338459070 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f5c3401dce0 con 0x55a3597c7660 2024-09-10T09:56:23.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.782+0000 7f5c443e0740 10 client.25639 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T09:56:23.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.782+0000 7f5c443e0740 10 client.25639 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T09:56:23.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.782+0000 7f5c443e0740 20 client.25639 populate_metadata read hostname 'smithi077' 2024-09-10T09:56:23.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.782+0000 7f5c443e0740 1 --2- 172.21.15.77:0/1338459070 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x55a359839e60 0x55a35983c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:23.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.782+0000 7f5c443e0740 4 client.25639 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:56:23.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.782+0000 7f5c443e0740 1 -- 172.21.15.77:0/1338459070 --> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a359721e60 con 0x55a359839e60 2024-09-10T09:56:23.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.782+0000 7f5c3aa00640 1 --2- 172.21.15.77:0/1338459070 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x55a359839e60 0x55a35983c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:23.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.786+0000 7f5c3aa00640 1 --2- 172.21.15.77:0/1338459070 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x55a359839e60 0x55a35983c2a0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=de40ff3a81682ae0 server_cookie=4c5c8f12eff9696b in_seq=0 out_seq=0 2024-09-10T09:56:23.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.786+0000 7f5c24c00640 10 client.25639 ms_handle_connect on v2:172.21.15.134:6836/2461497091 2024-09-10T09:56:23.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.786+0000 7f5c24c00640 1 -- 172.21.15.77:0/1338459070 <== mds.3 v2:172.21.15.134:6836/2461497091 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a359721e60 con 0x55a359839e60 2024-09-10T09:56:23.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.786+0000 7f5c24c00640 10 client.25639 handle_command_reply: tid=0 2024-09-10T09:56:23.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.786+0000 7f5c443e0740 1 client.25639 shutdown 2024-09-10T09:56:23.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.786+0000 7f5c443e0740 1 -- 172.21.15.77:0/1338459070 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a3597c7660 msgr2=0x55a3597c1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:23.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.786+0000 7f5c443e0740 1 --2- 172.21.15.77:0/1338459070 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a3597c7660 0x55a3597c1580 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5c34011d40 tx=0x7f5c3401f000 comp rx=0 tx=0).stop 2024-09-10T09:56:23.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.787+0000 7f5c443e0740 1 -- 172.21.15.77:0/1338459070 shutdown_connections 2024-09-10T09:56:23.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.787+0000 7f5c443e0740 1 --2- 172.21.15.77:0/1338459070 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a3597c7660 0x55a3597c1580 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:23.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.787+0000 7f5c443e0740 1 --2- 172.21.15.77:0/1338459070 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x55a359839e60 0x55a35983c2a0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:23.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.787+0000 7f5c443e0740 1 --2- 172.21.15.77:0/1338459070 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3597bcda0 0x55a3597c1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:23.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.787+0000 7f5c24c00640 0 client.25639 ms_handle_reset on v2:172.21.15.134:6836/2461497091 2024-09-10T09:56:23.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.787+0000 7f5c443e0740 1 --2- 172.21.15.77:0/1338459070 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3597bc480 0x55a3597c5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:23.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.787+0000 7f5c443e0740 1 -- 172.21.15.77:0/1338459070 >> 172.21.15.77:0/1338459070 conn(0x55a3597b8a10 msgr2=0x55a3597b6c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:23.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.787+0000 7f5c443e0740 1 -- 172.21.15.77:0/1338459070 shutdown_connections 2024-09-10T09:56:23.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.787+0000 7f5c443e0740 1 -- 172.21.15.77:0/1338459070 wait complete. 2024-09-10T09:56:23.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.787+0000 7f5c443e0740 20 client.25639 trim_cache size 0 max 16384 2024-09-10T09:56:23.789 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-10T09:56:23.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.788+0000 7f5c41a00640 1 -- 172.21.15.77:0/454958729 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f5c200a2c00 msgr2=0x7f5c200a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:23.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.788+0000 7f5c41a00640 1 --2- 172.21.15.77:0/454958729 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f5c200a2c00 0x7f5c200a50c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f5c34005fd0 tx=0x7f5c34009880 comp rx=0 tx=0).stop 2024-09-10T09:56:23.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.788+0000 7f5c41a00640 1 -- 172.21.15.77:0/454958729 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5c3c1049a0 msgr2=0x7f5c3c19ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:23.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.788+0000 7f5c41a00640 1 --2- 172.21.15.77:0/454958729 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5c3c1049a0 0x7f5c3c19ab90 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5c28004770 tx=0x7f5c28003c10 comp rx=0 tx=0).stop 2024-09-10T09:56:23.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.789+0000 7f5c41a00640 1 -- 172.21.15.77:0/454958729 shutdown_connections 2024-09-10T09:56:23.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.789+0000 7f5c41a00640 1 --2- 172.21.15.77:0/454958729 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f5c200a2c00 0x7f5c200a50c0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:23.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.789+0000 7f5c41a00640 1 --2- 172.21.15.77:0/454958729 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5c3c109120 0x7f5c3c19ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:23.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.789+0000 7f5c41a00640 1 --2- 172.21.15.77:0/454958729 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5c3c1049a0 0x7f5c3c19ab90 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:23.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.789+0000 7f5c41a00640 1 --2- 172.21.15.77:0/454958729 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5c3c103ff0 0x7f5c3c19a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:23.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.789+0000 7f5c41a00640 1 -- 172.21.15.77:0/454958729 >> 172.21.15.77:0/454958729 conn(0x7f5c3c0ffa90 msgr2=0x7f5c3c100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:23.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.789+0000 7f5c41a00640 1 -- 172.21.15.77:0/454958729 shutdown_connections 2024-09-10T09:56:23.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:23.789+0000 7f5c41a00640 1 -- 172.21.15.77:0/454958729 wait complete. 2024-09-10T09:56:24.714 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T09:56:24.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:56:24 smithi077 ceph-mon[30319]: pgmap v428: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-10T09:56:25.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:56:24 smithi134 ceph-mon[35674]: pgmap v428: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-10T09:56:25.147 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:56:25.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:56:24 smithi120 ceph-mon[34770]: pgmap v428: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-10T09:56:25.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.882+0000 7fed0f200640 1 -- 172.21.15.77:0/3945923367 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fed10074510 msgr2=0x7fed100748f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:25.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.882+0000 7fed0f200640 1 --2- 172.21.15.77:0/3945923367 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fed10074510 0x7fed100748f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fed00009860 tx=0x7fed0002a3d0 comp rx=0 tx=0).stop 2024-09-10T09:56:25.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.882+0000 7fed0f200640 1 -- 172.21.15.77:0/3945923367 shutdown_connections 2024-09-10T09:56:25.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.882+0000 7fed0f200640 1 --2- 172.21.15.77:0/3945923367 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed1010e280 0x7fed10110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:25.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.882+0000 7fed0f200640 1 --2- 172.21.15.77:0/3945923367 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fed10074e30 0x7fed1010dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:25.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.882+0000 7fed0f200640 1 --2- 172.21.15.77:0/3945923367 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fed10074510 0x7fed100748f0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:25.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.882+0000 7fed0f200640 1 -- 172.21.15.77:0/3945923367 >> 172.21.15.77:0/3945923367 conn(0x7fed100fd740 msgr2=0x7fed100ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:25.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.882+0000 7fed0f200640 1 -- 172.21.15.77:0/3945923367 shutdown_connections 2024-09-10T09:56:25.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.883+0000 7fed0f200640 1 -- 172.21.15.77:0/3945923367 wait complete. 2024-09-10T09:56:25.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.884+0000 7fed0f200640 1 Processor -- start 2024-09-10T09:56:25.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.884+0000 7fed0f200640 1 -- start start 2024-09-10T09:56:25.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.884+0000 7fed0f200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fed10074510 0x7fed1019a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:25.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.884+0000 7fed0f200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed10074e30 0x7fed1019aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:25.886 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.885+0000 7fed0f200640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fed1010e280 0x7fed1019eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:25.886 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.885+0000 7fed0f200640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fed10113fb0 con 0x7fed10074510 2024-09-10T09:56:25.886 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.885+0000 7fed0f200640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fed10113e30 con 0x7fed1010e280 2024-09-10T09:56:25.886 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.885+0000 7fed0f200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fed10114130 con 0x7fed10074e30 2024-09-10T09:56:25.886 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.885+0000 7fed0de00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fed10074510 0x7fed1019a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:25.886 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.885+0000 7fed0de00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fed10074510 0x7fed1019a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:53366/0 (socket says 172.21.15.77:53366) 2024-09-10T09:56:25.886 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.885+0000 7fed0de00640 1 -- 172.21.15.77:0/3604365400 learned_addr learned my addr 172.21.15.77:0/3604365400 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:25.886 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.885+0000 7fed0e800640 1 --2- 172.21.15.77:0/3604365400 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fed1010e280 0x7fed1019eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:25.886 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.885+0000 7fed0d400640 1 --2- 172.21.15.77:0/3604365400 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed10074e30 0x7fed1019aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:25.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.886+0000 7fed0de00640 1 -- 172.21.15.77:0/3604365400 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed10074e30 msgr2=0x7fed1019aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:25.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.886+0000 7fed0de00640 1 --2- 172.21.15.77:0/3604365400 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed10074e30 0x7fed1019aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:25.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.886+0000 7fed0de00640 1 -- 172.21.15.77:0/3604365400 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fed1010e280 msgr2=0x7fed1019eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:25.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.886+0000 7fed0de00640 1 --2- 172.21.15.77:0/3604365400 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fed1010e280 0x7fed1019eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:25.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.886+0000 7fed0de00640 1 -- 172.21.15.77:0/3604365400 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed1019f480 con 0x7fed10074510 2024-09-10T09:56:25.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.886+0000 7fed0de00640 1 --2- 172.21.15.77:0/3604365400 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fed10074510 0x7fed1019a4d0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fed0002a8f0 tx=0x7fed00030650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:25.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.887+0000 7fed06a00640 1 -- 172.21.15.77:0/3604365400 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed00040020 con 0x7fed10074510 2024-09-10T09:56:25.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.887+0000 7fed0f200640 1 -- 172.21.15.77:0/3604365400 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed1019f710 con 0x7fed10074510 2024-09-10T09:56:25.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.887+0000 7fed06a00640 1 -- 172.21.15.77:0/3604365400 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed00004460 con 0x7fed10074510 2024-09-10T09:56:25.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.887+0000 7fed06a00640 1 -- 172.21.15.77:0/3604365400 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed000046c0 con 0x7fed10074510 2024-09-10T09:56:25.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.887+0000 7fed0f200640 1 -- 172.21.15.77:0/3604365400 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed101a7050 con 0x7fed10074510 2024-09-10T09:56:25.892 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.891+0000 7fed174a4740 1 Processor -- start 2024-09-10T09:56:25.892 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.892+0000 7fed174a4740 1 -- start start 2024-09-10T09:56:25.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.892+0000 7fed06a00640 1 -- 172.21.15.77:0/3604365400 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fed00043020 con 0x7fed10074510 2024-09-10T09:56:25.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.893+0000 7fed174a4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b906827650 0x55b906827a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:25.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.893+0000 7fed174a4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b906827f70 0x55b9068322f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:25.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.893+0000 7fed174a4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b906832830 0x55b906834c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:25.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.893+0000 7fed174a4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55b9067853e0 con 0x55b906832830 2024-09-10T09:56:25.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.893+0000 7fed174a4740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b90672cd80 con 0x55b906827650 2024-09-10T09:56:25.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.893+0000 7fed174a4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b9067813a0 con 0x55b906827f70 2024-09-10T09:56:25.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.893+0000 7fed0de00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b906827f70 0x55b9068322f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:25.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.893+0000 7fed0e800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b906827650 0x55b906827a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:25.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.893+0000 7fed0e800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b906827650 0x55b906827a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:56716/0 (socket says 172.21.15.77:56716) 2024-09-10T09:56:25.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.893+0000 7fed0e800640 1 -- 172.21.15.77:0/4084391665 learned_addr learned my addr 172.21.15.77:0/4084391665 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:25.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.894+0000 7fed0e800640 1 -- 172.21.15.77:0/4084391665 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b906827f70 msgr2=0x55b9068322f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:25.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.894+0000 7fed0e800640 1 --2- 172.21.15.77:0/4084391665 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b906827f70 0x55b9068322f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:25.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.894+0000 7fed0e800640 1 -- 172.21.15.77:0/4084391665 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b906832830 msgr2=0x55b906834c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:25.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.895+0000 7fed0e800640 1 --2- 172.21.15.77:0/4084391665 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b906832830 0x55b906834c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:25.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.895+0000 7fed0e800640 1 -- 172.21.15.77:0/4084391665 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b90674bae0 con 0x55b906827650 2024-09-10T09:56:25.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.895+0000 7fed06a00640 1 --2- 172.21.15.77:0/3604365400 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fecf40a2cd0 0x7fecf40a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:25.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.895+0000 7fed06a00640 1 -- 172.21.15.77:0/3604365400 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fed000efd30 con 0x7fed10074510 2024-09-10T09:56:25.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.895+0000 7fed0de00640 1 --2- 172.21.15.77:0/3604365400 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fecf40a2cd0 0x7fecf40a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:25.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.895+0000 7fed0e800640 1 --2- 172.21.15.77:0/4084391665 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b906827650 0x55b906827a30 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fed08009f30 tx=0x7fed080118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c231630867f70ffe server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:25.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.896+0000 7fecfb000640 1 -- 172.21.15.77:0/4084391665 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed0801e070 con 0x55b906827650 2024-09-10T09:56:25.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.896+0000 7fecfb000640 1 -- 172.21.15.77:0/4084391665 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed08003c30 con 0x55b906827650 2024-09-10T09:56:25.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.896+0000 7fecfb000640 1 -- 172.21.15.77:0/4084391665 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed0800f980 con 0x55b906827650 2024-09-10T09:56:25.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.897+0000 7fed174a4740 1 -- 172.21.15.77:0/4084391665 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b906827650 msgr2=0x55b906827a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:25.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.897+0000 7fed174a4740 1 --2- 172.21.15.77:0/4084391665 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b906827650 0x55b906827a30 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fed08009f30 tx=0x7fed080118d0 comp rx=0 tx=0).stop 2024-09-10T09:56:25.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.897+0000 7fed0de00640 1 --2- 172.21.15.77:0/3604365400 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fecf40a2cd0 0x7fecf40a5190 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fed000ed7b0 tx=0x7fed00005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:25.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.897+0000 7fed174a4740 1 -- 172.21.15.77:0/4084391665 shutdown_connections 2024-09-10T09:56:25.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.897+0000 7fed174a4740 1 --2- 172.21.15.77:0/4084391665 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b906832830 0x55b906834c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:25.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.897+0000 7fed174a4740 1 --2- 172.21.15.77:0/4084391665 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b906827f70 0x55b9068322f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:25.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.897+0000 7fed174a4740 1 --2- 172.21.15.77:0/4084391665 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b906827650 0x55b906827a30 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:25.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.897+0000 7fed174a4740 1 -- 172.21.15.77:0/4084391665 >> 172.21.15.77:0/4084391665 conn(0x55b906823be0 msgr2=0x55b906823fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:25.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.897+0000 7fed174a4740 1 -- 172.21.15.77:0/4084391665 shutdown_connections 2024-09-10T09:56:25.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.897+0000 7fed174a4740 1 -- 172.21.15.77:0/4084391665 wait complete. 2024-09-10T09:56:25.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.898+0000 7fed174a4740 1 -- start start 2024-09-10T09:56:25.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.898+0000 7fed174a4740 1 Processor -- start 2024-09-10T09:56:25.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.899+0000 7fed174a4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b906827650 0x55b90682f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:25.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.899+0000 7fed174a4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b906827f70 0x55b90682fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:25.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.899+0000 7fed174a4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b906832830 0x55b906830210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:25.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.899+0000 7fed174a4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55b906835400 con 0x55b906827650 2024-09-10T09:56:25.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.899+0000 7fed174a4740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b9067400f0 con 0x55b906827f70 2024-09-10T09:56:25.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.899+0000 7fed174a4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b906835580 con 0x55b906832830 2024-09-10T09:56:25.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.899+0000 7fed174a4740 10 client.0 _ll_register_callbacks cb 0x55b906727bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:56:25.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.899+0000 7fed0e800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b906827650 0x55b90682f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:25.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.900+0000 7fed0d400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b906827f70 0x55b90682fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:25.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.900+0000 7fed0e800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b906827650 0x55b90682f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:53372/0 (socket says 172.21.15.77:53372) 2024-09-10T09:56:25.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.900+0000 7fed0d400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b906827f70 0x55b90682fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:56720/0 (socket says 172.21.15.77:56720) 2024-09-10T09:56:25.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.900+0000 7fed0e800640 1 -- 172.21.15.77:0/2105848087 learned_addr learned my addr 172.21.15.77:0/2105848087 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:25.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.900+0000 7fed0e800640 1 --2- 172.21.15.77:0/2105848087 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b906832830 0x55b906830210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:25.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.900+0000 7fed0d400640 1 -- 172.21.15.77:0/2105848087 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b906832830 msgr2=0x55b906830210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:25.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.900+0000 7fed0d400640 1 --2- 172.21.15.77:0/2105848087 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b906832830 0x55b906830210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:25.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.900+0000 7fed0d400640 1 -- 172.21.15.77:0/2105848087 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b906827650 msgr2=0x55b90682f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:25.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.900+0000 7fed0d400640 1 --2- 172.21.15.77:0/2105848087 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b906827650 0x55b90682f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:25.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.900+0000 7fed0d400640 1 -- 172.21.15.77:0/2105848087 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9064f63c0 con 0x55b906827f70 2024-09-10T09:56:25.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.900+0000 7fed0e800640 1 --2- 172.21.15.77:0/2105848087 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b906827650 0x55b90682f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:56:25.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.901+0000 7fed0d400640 1 --2- 172.21.15.77:0/2105848087 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b906827f70 0x55b90682fcd0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fed1010f710 tx=0x7fecfc011b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18081dbc9990684a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:25.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.901+0000 7fecef400640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T09:56:25.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.901+0000 7fecef400640 1 -- 172.21.15.77:0/2105848087 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecfc01e060 con 0x55b906827f70 2024-09-10T09:56:25.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.901+0000 7fecef400640 1 -- 172.21.15.77:0/2105848087 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fecfc00fcc0 con 0x55b906827f70 2024-09-10T09:56:25.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.901+0000 7fed174a4740 1 -- 172.21.15.77:0/2105848087 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed08003c30 con 0x55b906827f70 2024-09-10T09:56:25.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.902+0000 7fecef400640 1 -- 172.21.15.77:0/2105848087 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fecfc025050 con 0x55b906827f70 2024-09-10T09:56:25.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.902+0000 7fed174a4740 10 client.25685 fetch_fsmap learned FSMap version 29 2024-09-10T09:56:25.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.902+0000 7fed174a4740 1 -- 172.21.15.77:0/2105848087 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b906841520 con 0x55b906827f70 2024-09-10T09:56:25.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.902+0000 7fecef400640 1 -- 172.21.15.77:0/2105848087 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed08003c30 con 0x55b906827f70 2024-09-10T09:56:25.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.902+0000 7fecef400640 1 -- 172.21.15.77:0/2105848087 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fecfc00e4d0 con 0x55b906827f70 2024-09-10T09:56:25.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.902+0000 7fed174a4740 10 client.25685 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T09:56:25.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.902+0000 7fed174a4740 10 client.25685 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T09:56:25.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.902+0000 7fed174a4740 20 client.25685 populate_metadata read hostname 'smithi077' 2024-09-10T09:56:25.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.902+0000 7fed174a4740 1 --2- 172.21.15.77:0/2105848087 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x55b9068429e0 0x55b906844dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:25.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.902+0000 7fed174a4740 4 client.25685 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:56:25.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.902+0000 7fed174a4740 1 -- 172.21.15.77:0/2105848087 --> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b906792950 con 0x55b9068429e0 2024-09-10T09:56:25.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.903+0000 7fed0e800640 1 --2- 172.21.15.77:0/2105848087 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x55b9068429e0 0x55b906844dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:25.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.904+0000 7fed0e800640 1 --2- 172.21.15.77:0/2105848087 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x55b9068429e0 0x55b906844dd0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3bd54187ec2e49fc server_cookie=e0433a1aba2e1355 in_seq=0 out_seq=0 2024-09-10T09:56:25.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.904+0000 7fecef400640 10 client.25685 ms_handle_connect on v2:172.21.15.134:6838/1965117680 2024-09-10T09:56:25.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.904+0000 7fecef400640 1 -- 172.21.15.77:0/2105848087 <== mds.2 v2:172.21.15.134:6838/1965117680 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b906792950 con 0x55b9068429e0 2024-09-10T09:56:25.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.904+0000 7fecef400640 10 client.25685 handle_command_reply: tid=0 2024-09-10T09:56:25.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.905+0000 7fed174a4740 1 client.25685 shutdown 2024-09-10T09:56:25.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.905+0000 7fed174a4740 1 -- 172.21.15.77:0/2105848087 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b906827f70 msgr2=0x55b90682fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:25.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.905+0000 7fed174a4740 1 --2- 172.21.15.77:0/2105848087 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b906827f70 0x55b90682fcd0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fed1010f710 tx=0x7fecfc011b50 comp rx=0 tx=0).stop 2024-09-10T09:56:25.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.905+0000 7fed174a4740 1 -- 172.21.15.77:0/2105848087 shutdown_connections 2024-09-10T09:56:25.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.905+0000 7fed174a4740 1 --2- 172.21.15.77:0/2105848087 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x55b9068429e0 0x55b906844dd0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:25.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.905+0000 7fed174a4740 1 --2- 172.21.15.77:0/2105848087 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b906832830 0x55b906830210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:25.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.905+0000 7fed174a4740 1 --2- 172.21.15.77:0/2105848087 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b906827f70 0x55b90682fcd0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:25.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.905+0000 7fed174a4740 1 --2- 172.21.15.77:0/2105848087 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b906827650 0x55b90682f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:25.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.905+0000 7fed174a4740 1 -- 172.21.15.77:0/2105848087 >> 172.21.15.77:0/2105848087 conn(0x55b906823be0 msgr2=0x55b9068298c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:25.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.905+0000 7fecef400640 0 client.25685 ms_handle_reset on v2:172.21.15.134:6838/1965117680 2024-09-10T09:56:25.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.905+0000 7fed174a4740 1 -- 172.21.15.77:0/2105848087 shutdown_connections 2024-09-10T09:56:25.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.905+0000 7fed174a4740 1 -- 172.21.15.77:0/2105848087 wait complete. 2024-09-10T09:56:25.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.906+0000 7fed174a4740 20 client.25685 trim_cache size 0 max 16384 2024-09-10T09:56:25.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.931+0000 7fed174a4740 1 Processor -- start 2024-09-10T09:56:25.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.931+0000 7fed174a4740 1 -- start start 2024-09-10T09:56:25.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.931+0000 7fed174a4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b906827650 0x55b906830060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:25.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.931+0000 7fed174a4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b906827f70 0x55b90684a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:25.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.931+0000 7fed174a4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b906832830 0x55b90684ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:25.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.931+0000 7fed174a4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fed101acb50 con 0x55b906827650 2024-09-10T09:56:25.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.931+0000 7fed174a4740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fed101ae690 con 0x55b906832830 2024-09-10T09:56:25.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.931+0000 7fed174a4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fed101ad220 con 0x55b906827f70 2024-09-10T09:56:25.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.932+0000 7fed0e800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b906827650 0x55b906830060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:25.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.932+0000 7fed0e800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b906827650 0x55b906830060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:53386/0 (socket says 172.21.15.77:53386) 2024-09-10T09:56:25.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.932+0000 7fed0e800640 1 -- 172.21.15.77:0/3731891576 learned_addr learned my addr 172.21.15.77:0/3731891576 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:25.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.932+0000 7fed0e800640 1 --2- 172.21.15.77:0/3731891576 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b906832830 0x55b90684ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:25.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.932+0000 7fed0d400640 1 --2- 172.21.15.77:0/3731891576 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b906827f70 0x55b90684a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:25.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.932+0000 7fed0d400640 1 -- 172.21.15.77:0/3731891576 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b906832830 msgr2=0x55b90684ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:25.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.932+0000 7fed0d400640 1 --2- 172.21.15.77:0/3731891576 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b906832830 0x55b90684ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:25.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.932+0000 7fed0d400640 1 -- 172.21.15.77:0/3731891576 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b906827650 msgr2=0x55b906830060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:25.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.932+0000 7fed0d400640 1 --2- 172.21.15.77:0/3731891576 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b906827650 0x55b906830060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:25.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.932+0000 7fed0d400640 1 -- 172.21.15.77:0/3731891576 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b906845400 con 0x55b906827f70 2024-09-10T09:56:25.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.933+0000 7fed0d400640 1 --2- 172.21.15.77:0/3731891576 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b906827f70 0x55b90684a630 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x55b906830d40 tx=0x7fecfc0097d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97a9ae5fab4c3dc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:25.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.933+0000 7fecfa600640 1 -- 172.21.15.77:0/3731891576 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecfc000e60 con 0x55b906827f70 2024-09-10T09:56:25.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.933+0000 7fecfa600640 1 -- 172.21.15.77:0/3731891576 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fecfc00ec40 con 0x55b906827f70 2024-09-10T09:56:25.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.933+0000 7fecfa600640 1 -- 172.21.15.77:0/3731891576 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecfc004420 con 0x55b906827f70 2024-09-10T09:56:25.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.934+0000 7fed174a4740 1 -- 172.21.15.77:0/3731891576 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b906827f70 msgr2=0x55b90684a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:25.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.934+0000 7fed174a4740 1 --2- 172.21.15.77:0/3731891576 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b906827f70 0x55b90684a630 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x55b906830d40 tx=0x7fecfc0097d0 comp rx=0 tx=0).stop 2024-09-10T09:56:25.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.935+0000 7fed174a4740 1 -- 172.21.15.77:0/3731891576 shutdown_connections 2024-09-10T09:56:25.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.935+0000 7fed174a4740 1 --2- 172.21.15.77:0/3731891576 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b906832830 0x55b90684ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:25.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.935+0000 7fed174a4740 1 --2- 172.21.15.77:0/3731891576 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b906827f70 0x55b90684a630 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:25.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.935+0000 7fed174a4740 1 --2- 172.21.15.77:0/3731891576 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b906827650 0x55b906830060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:25.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.935+0000 7fed174a4740 1 -- 172.21.15.77:0/3731891576 >> 172.21.15.77:0/3731891576 conn(0x55b906823be0 msgr2=0x55b906831810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:25.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.935+0000 7fed174a4740 1 -- 172.21.15.77:0/3731891576 shutdown_connections 2024-09-10T09:56:25.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.935+0000 7fed174a4740 1 -- 172.21.15.77:0/3731891576 wait complete. 2024-09-10T09:56:25.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.935+0000 7fed174a4740 1 -- start start 2024-09-10T09:56:25.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.936+0000 7fed174a4740 1 Processor -- start 2024-09-10T09:56:25.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.936+0000 7fed174a4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b906827650 0x55b90683a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:25.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.936+0000 7fed174a4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b906827f70 0x55b90683ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:25.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.936+0000 7fed174a4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b906832830 0x55b9068377a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:25.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.936+0000 7fed174a4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fed101adfc0 con 0x55b906832830 2024-09-10T09:56:25.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.936+0000 7fed174a4740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fed101ad8f0 con 0x55b906827650 2024-09-10T09:56:25.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.936+0000 7fed174a4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fed101aab30 con 0x55b906827f70 2024-09-10T09:56:25.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.936+0000 7fed174a4740 10 client.0 _ll_register_callbacks cb 0x7fecfc009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:56:25.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.937+0000 7fed0e800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b906827650 0x55b90683a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:25.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.937+0000 7fed0e800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b906832830 0x55b9068377a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:25.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.937+0000 7fed0e800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b906827650 0x55b90683a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:56724/0 (socket says 172.21.15.77:56724) 2024-09-10T09:56:25.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.937+0000 7fed0e800640 1 -- 172.21.15.77:0/743739083 learned_addr learned my addr 172.21.15.77:0/743739083 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:25.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.937+0000 7fed0d400640 1 --2- 172.21.15.77:0/743739083 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b906827f70 0x55b90683ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:25.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.937+0000 7fed0e800640 1 -- 172.21.15.77:0/743739083 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b906827f70 msgr2=0x55b90683ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:25.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.937+0000 7fed0e800640 1 --2- 172.21.15.77:0/743739083 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b906827f70 0x55b90683ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:25.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.937+0000 7fed0e800640 1 -- 172.21.15.77:0/743739083 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b906827650 msgr2=0x55b90683a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:25.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.937+0000 7fed0e800640 1 --2- 172.21.15.77:0/743739083 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b906827650 0x55b90683a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:25.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.937+0000 7fed0e800640 1 -- 172.21.15.77:0/743739083 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b90683b380 con 0x55b906832830 2024-09-10T09:56:25.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.938+0000 7fed0e800640 1 --2- 172.21.15.77:0/743739083 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b906832830 0x55b9068377a0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fed08011dd0 tx=0x7fecfc009d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73a4297aeeafad6b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:25.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.938+0000 7fecef400640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-10T09:56:25.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.938+0000 7fecef400640 1 -- 172.21.15.77:0/743739083 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed0801e040 con 0x55b906832830 2024-09-10T09:56:25.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.938+0000 7fecef400640 1 -- 172.21.15.77:0/743739083 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fed080044e0 con 0x55b906832830 2024-09-10T09:56:25.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.938+0000 7fed174a4740 1 -- 172.21.15.77:0/743739083 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fecfc00ec40 con 0x55b906832830 2024-09-10T09:56:25.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.939+0000 7fecef400640 1 -- 172.21.15.77:0/743739083 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fed0801e040 con 0x55b906832830 2024-09-10T09:56:25.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.939+0000 7fed174a4740 10 client.15630 fetch_fsmap learned FSMap version 29 2024-09-10T09:56:25.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.939+0000 7fed174a4740 1 -- 172.21.15.77:0/743739083 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b9068adb60 con 0x55b906832830 2024-09-10T09:56:25.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.939+0000 7fecef400640 1 -- 172.21.15.77:0/743739083 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fecfc00ec40 con 0x55b906832830 2024-09-10T09:56:25.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.940+0000 7fecef400640 1 -- 172.21.15.77:0/743739083 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fed080194c0 con 0x55b906832830 2024-09-10T09:56:25.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.940+0000 7fed174a4740 10 client.15630 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T09:56:25.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.940+0000 7fed174a4740 10 client.15630 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T09:56:25.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.940+0000 7fed174a4740 20 client.15630 populate_metadata read hostname 'smithi077' 2024-09-10T09:56:25.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.940+0000 7fed174a4740 1 --2- 172.21.15.77:0/743739083 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x55b9068adff0 0x55b9068b03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:25.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.940+0000 7fed174a4740 4 client.15630 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:56:25.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.940+0000 7fed174a4740 1 -- 172.21.15.77:0/743739083 --> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b90678ce60 con 0x55b9068adff0 2024-09-10T09:56:25.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.940+0000 7fed0d400640 1 --2- 172.21.15.77:0/743739083 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x55b9068adff0 0x55b9068b03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:25.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.941+0000 7fed0d400640 1 --2- 172.21.15.77:0/743739083 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x55b9068adff0 0x55b9068b03e0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a92ba0e439f3e89f server_cookie=7a5dffa979016942 in_seq=0 out_seq=0 2024-09-10T09:56:25.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.941+0000 7fecef400640 10 client.15630 ms_handle_connect on v2:172.21.15.134:6838/1965117680 2024-09-10T09:56:25.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.941+0000 7fecef400640 1 -- 172.21.15.77:0/743739083 <== mds.2 v2:172.21.15.134:6838/1965117680 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b90678ce60 con 0x55b9068adff0 2024-09-10T09:56:25.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.941+0000 7fecef400640 10 client.15630 handle_command_reply: tid=0 2024-09-10T09:56:25.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.941+0000 7fed174a4740 1 client.15630 shutdown 2024-09-10T09:56:25.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.941+0000 7fed174a4740 1 -- 172.21.15.77:0/743739083 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b906832830 msgr2=0x55b9068377a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:25.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.941+0000 7fed174a4740 1 --2- 172.21.15.77:0/743739083 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b906832830 0x55b9068377a0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fed08011dd0 tx=0x7fecfc009d30 comp rx=0 tx=0).stop 2024-09-10T09:56:25.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.942+0000 7fed174a4740 1 -- 172.21.15.77:0/743739083 shutdown_connections 2024-09-10T09:56:25.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.942+0000 7fed174a4740 1 --2- 172.21.15.77:0/743739083 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x55b9068adff0 0x55b9068b03e0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:25.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.942+0000 7fed174a4740 1 --2- 172.21.15.77:0/743739083 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b906832830 0x55b9068377a0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:25.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.942+0000 7fed174a4740 1 --2- 172.21.15.77:0/743739083 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b906827f70 0x55b90683ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:25.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.942+0000 7fed174a4740 1 --2- 172.21.15.77:0/743739083 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b906827650 0x55b90683a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:25.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.942+0000 7fed174a4740 1 -- 172.21.15.77:0/743739083 >> 172.21.15.77:0/743739083 conn(0x55b906823be0 msgr2=0x55b906819c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:25.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.942+0000 7fecef400640 0 client.15630 ms_handle_reset on v2:172.21.15.134:6838/1965117680 2024-09-10T09:56:25.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.942+0000 7fed174a4740 1 -- 172.21.15.77:0/743739083 shutdown_connections 2024-09-10T09:56:25.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.942+0000 7fed174a4740 1 -- 172.21.15.77:0/743739083 wait complete. 2024-09-10T09:56:25.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.942+0000 7fed174a4740 20 client.15630 trim_cache size 0 max 16384 2024-09-10T09:56:25.944 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-10T09:56:25.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.943+0000 7fed0f200640 1 -- 172.21.15.77:0/3604365400 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fecf40a2cd0 msgr2=0x7fecf40a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:25.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.943+0000 7fed0f200640 1 --2- 172.21.15.77:0/3604365400 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fecf40a2cd0 0x7fecf40a5190 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fed000ed7b0 tx=0x7fed00005f90 comp rx=0 tx=0).stop 2024-09-10T09:56:25.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.943+0000 7fed0f200640 1 -- 172.21.15.77:0/3604365400 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fed10074510 msgr2=0x7fed1019a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:25.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.943+0000 7fed0f200640 1 --2- 172.21.15.77:0/3604365400 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fed10074510 0x7fed1019a4d0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fed0002a8f0 tx=0x7fed00030650 comp rx=0 tx=0).stop 2024-09-10T09:56:25.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.943+0000 7fed0f200640 1 -- 172.21.15.77:0/3604365400 shutdown_connections 2024-09-10T09:56:25.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.943+0000 7fed0f200640 1 --2- 172.21.15.77:0/3604365400 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fecf40a2cd0 0x7fecf40a5190 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:25.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.943+0000 7fed0f200640 1 --2- 172.21.15.77:0/3604365400 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fed1010e280 0x7fed1019eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:25.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.943+0000 7fed0f200640 1 --2- 172.21.15.77:0/3604365400 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed10074e30 0x7fed1019aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:25.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.943+0000 7fed0f200640 1 --2- 172.21.15.77:0/3604365400 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fed10074510 0x7fed1019a4d0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:25.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.943+0000 7fed0f200640 1 -- 172.21.15.77:0/3604365400 >> 172.21.15.77:0/3604365400 conn(0x7fed100fd740 msgr2=0x7fed100fe440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:25.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.944+0000 7fed0f200640 1 -- 172.21.15.77:0/3604365400 shutdown_connections 2024-09-10T09:56:25.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:25.944+0000 7fed0f200640 1 -- 172.21.15.77:0/3604365400 wait complete. 2024-09-10T09:56:26.766 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T09:56:27.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:56:26 smithi134 ceph-mon[35674]: pgmap v429: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-10T09:56:27.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:56:26 smithi120 ceph-mon[34770]: pgmap v429: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-10T09:56:27.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:56:26 smithi077 ceph-mon[30319]: pgmap v429: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-10T09:56:27.345 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:56:28.087 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:56:27 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:56:28.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:56:27 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:56:28.151 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.149+0000 7f3c4e600640 1 -- 172.21.15.77:0/649553574 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3c48108f60 msgr2=0x7f3c48110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:28.151 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.149+0000 7f3c4e600640 1 --2- 172.21.15.77:0/649553574 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3c48108f60 0x7f3c48110ae0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3c40009ca0 tx=0x7f3c4002a8c0 comp rx=0 tx=0).stop 2024-09-10T09:56:28.151 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.149+0000 7f3c4e600640 1 -- 172.21.15.77:0/649553574 shutdown_connections 2024-09-10T09:56:28.151 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.149+0000 7f3c4e600640 1 --2- 172.21.15.77:0/649553574 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3c48108f60 0x7f3c48110ae0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:28.151 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.149+0000 7f3c4e600640 1 --2- 172.21.15.77:0/649553574 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c481047e0 0x7f3c48108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:28.151 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.149+0000 7f3c4e600640 1 --2- 172.21.15.77:0/649553574 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3c48103e30 0x7f3c48104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:28.151 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.149+0000 7f3c4e600640 1 -- 172.21.15.77:0/649553574 >> 172.21.15.77:0/649553574 conn(0x7f3c480ff910 msgr2=0x7f3c48101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:28.151 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.149+0000 7f3c4e600640 1 -- 172.21.15.77:0/649553574 shutdown_connections 2024-09-10T09:56:28.151 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.149+0000 7f3c4e600640 1 -- 172.21.15.77:0/649553574 wait complete. 2024-09-10T09:56:28.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.151+0000 7f3c4e600640 1 Processor -- start 2024-09-10T09:56:28.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.151+0000 7f3c4e600640 1 -- start start 2024-09-10T09:56:28.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.151+0000 7f3c4e600640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3c48103e30 0x7f3c48110910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:28.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.152+0000 7f3c4e600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c481047e0 0x7f3c4810d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:28.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.152+0000 7f3c4e600640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3c48108f60 0x7f3c4810df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:28.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.152+0000 7f3c4e600640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3c48114010 con 0x7f3c48108f60 2024-09-10T09:56:28.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.152+0000 7f3c4e600640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3c48113e90 con 0x7f3c48103e30 2024-09-10T09:56:28.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.152+0000 7f3c4e600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3c48114190 con 0x7f3c481047e0 2024-09-10T09:56:28.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.152+0000 7f3c4dc00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3c48108f60 0x7f3c4810df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:28.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.152+0000 7f3c4dc00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3c48108f60 0x7f3c4810df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:53426/0 (socket says 172.21.15.77:53426) 2024-09-10T09:56:28.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.152+0000 7f3c4dc00640 1 -- 172.21.15.77:0/385843695 learned_addr learned my addr 172.21.15.77:0/385843695 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:28.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.153+0000 7f3c4dc00640 1 -- 172.21.15.77:0/385843695 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c481047e0 msgr2=0x7f3c4810d9c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T09:56:28.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.153+0000 7f3c4dc00640 1 --2- 172.21.15.77:0/385843695 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c481047e0 0x7f3c4810d9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:28.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.153+0000 7f3c4dc00640 1 -- 172.21.15.77:0/385843695 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3c48103e30 msgr2=0x7f3c48110910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:28.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.153+0000 7f3c4dc00640 1 --2- 172.21.15.77:0/385843695 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3c48103e30 0x7f3c48110910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:28.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.153+0000 7f3c4dc00640 1 -- 172.21.15.77:0/385843695 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c4810e7c0 con 0x7f3c48108f60 2024-09-10T09:56:28.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.154+0000 7f3c4dc00640 1 --2- 172.21.15.77:0/385843695 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3c48108f60 0x7f3c4810df00 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f3c40005e30 tx=0x7f3c40003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:28.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.154+0000 7f3c45600640 1 -- 172.21.15.77:0/385843695 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c40031650 con 0x7f3c48108f60 2024-09-10T09:56:28.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.154+0000 7f3c45600640 1 -- 172.21.15.77:0/385843695 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c400317f0 con 0x7f3c48108f60 2024-09-10T09:56:28.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.154+0000 7f3c45600640 1 -- 172.21.15.77:0/385843695 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c4002f9d0 con 0x7f3c48108f60 2024-09-10T09:56:28.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.154+0000 7f3c4e600640 1 -- 172.21.15.77:0/385843695 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c481aadb0 con 0x7f3c48108f60 2024-09-10T09:56:28.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.155+0000 7f3c4e600640 1 -- 172.21.15.77:0/385843695 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c481ab210 con 0x7f3c48108f60 2024-09-10T09:56:28.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.160+0000 7f3c527cf740 1 Processor -- start 2024-09-10T09:56:28.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.160+0000 7f3c45600640 1 -- 172.21.15.77:0/385843695 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3c4002fb70 con 0x7f3c48108f60 2024-09-10T09:56:28.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.161+0000 7f3c527cf740 1 -- start start 2024-09-10T09:56:28.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.162+0000 7f3c527cf740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e5ea274e40 0x55e5ea275220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:28.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.162+0000 7f3c527cf740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5ea275760 0x55e5ea27fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:28.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.162+0000 7f3c527cf740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5ea280020 0x55e5ea282410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:28.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.162+0000 7f3c527cf740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e5ea1db3e0 con 0x55e5ea274e40 2024-09-10T09:56:28.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.162+0000 7f3c527cf740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e5ea182d80 con 0x55e5ea280020 2024-09-10T09:56:28.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.162+0000 7f3c527cf740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e5ea1d73a0 con 0x55e5ea275760 2024-09-10T09:56:28.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.163+0000 7f3c47e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e5ea274e40 0x55e5ea275220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:28.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.163+0000 7f3c45600640 1 --2- 172.21.15.77:0/385843695 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f3c300a2c00 0x7f3c300a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:28.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.163+0000 7f3c4d200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5ea275760 0x55e5ea27fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:28.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.163+0000 7f3c4d200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5ea275760 0x55e5ea27fae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:56450/0 (socket says 172.21.15.77:56450) 2024-09-10T09:56:28.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.163+0000 7f3c4d200640 1 -- 172.21.15.77:0/1670397872 learned_addr learned my addr 172.21.15.77:0/1670397872 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:28.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.163+0000 7f3c47e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e5ea274e40 0x55e5ea275220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:53434/0 (socket says 172.21.15.77:53434) 2024-09-10T09:56:28.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.163+0000 7f3c45600640 1 -- 172.21.15.77:0/385843695 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f3c400f0a00 con 0x7f3c48108f60 2024-09-10T09:56:28.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.164+0000 7f3c4dc00640 1 --2- 172.21.15.77:0/385843695 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f3c300a2c00 0x7f3c300a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:28.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.165+0000 7f3c47e00640 1 -- 172.21.15.77:0/1670397872 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5ea275760 msgr2=0x55e5ea27fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:28.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.165+0000 7f3c47e00640 1 --2- 172.21.15.77:0/1670397872 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5ea275760 0x55e5ea27fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:28.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.165+0000 7f3c47e00640 1 -- 172.21.15.77:0/1670397872 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5ea280020 msgr2=0x55e5ea282410 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T09:56:28.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.165+0000 7f3c47e00640 1 --2- 172.21.15.77:0/1670397872 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5ea280020 0x55e5ea282410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:28.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.165+0000 7f3c47e00640 1 -- 172.21.15.77:0/1670397872 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5ea1a1ae0 con 0x55e5ea274e40 2024-09-10T09:56:28.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.165+0000 7f3c47e00640 1 --2- 172.21.15.77:0/1670397872 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e5ea274e40 0x55e5ea275220 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f3c4810f180 tx=0x7f3c34009f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f29218b7e0e51824 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:28.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.167+0000 7f3c3a000640 1 -- 172.21.15.77:0/1670397872 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c3400ec00 con 0x55e5ea274e40 2024-09-10T09:56:28.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.167+0000 7f3c3a000640 1 -- 172.21.15.77:0/1670397872 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c3400eda0 con 0x55e5ea274e40 2024-09-10T09:56:28.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.167+0000 7f3c3a000640 1 -- 172.21.15.77:0/1670397872 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c34010430 con 0x55e5ea274e40 2024-09-10T09:56:28.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.168+0000 7f3c4dc00640 1 --2- 172.21.15.77:0/385843695 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f3c300a2c00 0x7f3c300a50c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f3c400bcb70 tx=0x7f3c4003e040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:28.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.168+0000 7f3c527cf740 1 -- 172.21.15.77:0/1670397872 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e5ea274e40 msgr2=0x55e5ea275220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:28.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.168+0000 7f3c527cf740 1 --2- 172.21.15.77:0/1670397872 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e5ea274e40 0x55e5ea275220 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f3c4810f180 tx=0x7f3c34009f00 comp rx=0 tx=0).stop 2024-09-10T09:56:28.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.168+0000 7f3c527cf740 1 -- 172.21.15.77:0/1670397872 shutdown_connections 2024-09-10T09:56:28.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.168+0000 7f3c527cf740 1 --2- 172.21.15.77:0/1670397872 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5ea280020 0x55e5ea282410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:28.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.168+0000 7f3c527cf740 1 --2- 172.21.15.77:0/1670397872 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5ea275760 0x55e5ea27fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:28.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.168+0000 7f3c527cf740 1 --2- 172.21.15.77:0/1670397872 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e5ea274e40 0x55e5ea275220 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:28.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.168+0000 7f3c527cf740 1 -- 172.21.15.77:0/1670397872 >> 172.21.15.77:0/1670397872 conn(0x55e5ea2713d0 msgr2=0x55e5ea2717b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:28.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.169+0000 7f3c527cf740 1 -- 172.21.15.77:0/1670397872 shutdown_connections 2024-09-10T09:56:28.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.169+0000 7f3c527cf740 1 -- 172.21.15.77:0/1670397872 wait complete. 2024-09-10T09:56:28.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.170+0000 7f3c527cf740 1 -- start start 2024-09-10T09:56:28.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.170+0000 7f3c527cf740 1 Processor -- start 2024-09-10T09:56:28.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.171+0000 7f3c527cf740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5ea275760 0x55e5ea27d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:28.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.171+0000 7f3c527cf740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e5ea280020 0x55e5ea27d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:28.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.171+0000 7f3c527cf740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5ea27dcb0 0x55e5ea28c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:28.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:56:27 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:56:28.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.171+0000 7f3c527cf740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e5ea282bf0 con 0x55e5ea280020 2024-09-10T09:56:28.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.171+0000 7f3c527cf740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e5ea1960f0 con 0x55e5ea275760 2024-09-10T09:56:28.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.171+0000 7f3c527cf740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e5ea282d70 con 0x55e5ea27dcb0 2024-09-10T09:56:28.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.171+0000 7f3c527cf740 10 client.0 _ll_register_callbacks cb 0x55e5ea17dbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:56:28.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.172+0000 7f3c47e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5ea275760 0x55e5ea27d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:28.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.172+0000 7f3c47e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5ea275760 0x55e5ea27d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:56748/0 (socket says 172.21.15.77:56748) 2024-09-10T09:56:28.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.172+0000 7f3c4d200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e5ea280020 0x55e5ea27d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:28.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.172+0000 7f3c4d200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e5ea280020 0x55e5ea27d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:53440/0 (socket says 172.21.15.77:53440) 2024-09-10T09:56:28.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.172+0000 7f3c4d200640 1 -- 172.21.15.77:0/1573731733 learned_addr learned my addr 172.21.15.77:0/1573731733 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:28.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.172+0000 7f3c47e00640 1 --2- 172.21.15.77:0/1573731733 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5ea27dcb0 0x55e5ea28c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:28.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.173+0000 7f3c4d200640 1 -- 172.21.15.77:0/1573731733 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5ea27dcb0 msgr2=0x55e5ea28c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:28.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.173+0000 7f3c4d200640 1 --2- 172.21.15.77:0/1573731733 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5ea27dcb0 0x55e5ea28c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:28.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.173+0000 7f3c4d200640 1 -- 172.21.15.77:0/1573731733 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5ea275760 msgr2=0x55e5ea27d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:28.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.173+0000 7f3c4d200640 1 --2- 172.21.15.77:0/1573731733 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5ea275760 0x55e5ea27d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:28.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.173+0000 7f3c4d200640 1 -- 172.21.15.77:0/1573731733 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5e9f4c3c0 con 0x55e5ea280020 2024-09-10T09:56:28.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.173+0000 7f3c4d200640 1 --2- 172.21.15.77:0/1573731733 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e5ea280020 0x55e5ea27d720 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c0117d0 tx=0x7f3c3c011c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dd62586551c654a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:28.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.173+0000 7f3c2ea00640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-10T09:56:28.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.174+0000 7f3c2ea00640 1 -- 172.21.15.77:0/1573731733 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c3c004230 con 0x55e5ea280020 2024-09-10T09:56:28.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.174+0000 7f3c2ea00640 1 -- 172.21.15.77:0/1573731733 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f3c3c0043d0 con 0x55e5ea280020 2024-09-10T09:56:28.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.174+0000 7f3c527cf740 1 -- 172.21.15.77:0/1573731733 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c3400eda0 con 0x55e5ea280020 2024-09-10T09:56:28.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.175+0000 7f3c2ea00640 1 -- 172.21.15.77:0/1573731733 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3c3c025050 con 0x55e5ea280020 2024-09-10T09:56:28.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.175+0000 7f3c527cf740 10 client.15642 fetch_fsmap learned FSMap version 29 2024-09-10T09:56:28.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.175+0000 7f3c527cf740 1 -- 172.21.15.77:0/1573731733 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e5ea28f390 con 0x55e5ea280020 2024-09-10T09:56:28.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.175+0000 7f3c2ea00640 1 -- 172.21.15.77:0/1573731733 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c3400eda0 con 0x55e5ea280020 2024-09-10T09:56:28.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.175+0000 7f3c2ea00640 1 -- 172.21.15.77:0/1573731733 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f3c3c003ae0 con 0x55e5ea280020 2024-09-10T09:56:28.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.176+0000 7f3c527cf740 10 client.15642 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T09:56:28.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.176+0000 7f3c527cf740 10 client.15642 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-10T09:56:28.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.176+0000 7f3c527cf740 20 client.15642 populate_metadata read hostname 'smithi077' 2024-09-10T09:56:28.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.176+0000 7f3c527cf740 1 --2- 172.21.15.77:0/1573731733 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x55e5ea2908f0 0x55e5ea292d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:28.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.176+0000 7f3c527cf740 4 client.15642 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:56:28.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.176+0000 7f3c527cf740 1 -- 172.21.15.77:0/1573731733 --> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e5ea1e8950 con 0x55e5ea2908f0 2024-09-10T09:56:28.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.176+0000 7f3c47e00640 1 --2- 172.21.15.77:0/1573731733 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x55e5ea2908f0 0x55e5ea292d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:28.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.178+0000 7f3c47e00640 1 --2- 172.21.15.77:0/1573731733 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x55e5ea2908f0 0x55e5ea292d30 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=483ef4449932fa01 server_cookie=f7f5ca2b7384838e in_seq=0 out_seq=0 2024-09-10T09:56:28.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.178+0000 7f3c2ea00640 10 client.15642 ms_handle_connect on v2:172.21.15.120:6836/3033280709 2024-09-10T09:56:28.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.178+0000 7f3c2ea00640 1 -- 172.21.15.77:0/1573731733 <== mds.1 v2:172.21.15.120:6836/3033280709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e5ea1e8950 con 0x55e5ea2908f0 2024-09-10T09:56:28.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.178+0000 7f3c2ea00640 10 client.15642 handle_command_reply: tid=0 2024-09-10T09:56:28.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.178+0000 7f3c527cf740 1 client.15642 shutdown 2024-09-10T09:56:28.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.179+0000 7f3c527cf740 1 -- 172.21.15.77:0/1573731733 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e5ea280020 msgr2=0x55e5ea27d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:28.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.179+0000 7f3c527cf740 1 --2- 172.21.15.77:0/1573731733 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e5ea280020 0x55e5ea27d720 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c0117d0 tx=0x7f3c3c011c90 comp rx=0 tx=0).stop 2024-09-10T09:56:28.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.179+0000 7f3c527cf740 1 -- 172.21.15.77:0/1573731733 shutdown_connections 2024-09-10T09:56:28.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.179+0000 7f3c527cf740 1 --2- 172.21.15.77:0/1573731733 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x55e5ea2908f0 0x55e5ea292d30 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:28.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.179+0000 7f3c527cf740 1 --2- 172.21.15.77:0/1573731733 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5ea27dcb0 0x55e5ea28c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:28.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.179+0000 7f3c527cf740 1 --2- 172.21.15.77:0/1573731733 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e5ea280020 0x55e5ea27d720 secure :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c0117d0 tx=0x7f3c3c011c90 comp rx=0 tx=0).stop 2024-09-10T09:56:28.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.179+0000 7f3c527cf740 1 --2- 172.21.15.77:0/1573731733 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5ea275760 0x55e5ea27d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:28.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.179+0000 7f3c2ea00640 0 client.15642 ms_handle_reset on v2:172.21.15.120:6836/3033280709 2024-09-10T09:56:28.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.179+0000 7f3c527cf740 1 -- 172.21.15.77:0/1573731733 >> 172.21.15.77:0/1573731733 conn(0x55e5ea2713d0 msgr2=0x55e5ea277100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:28.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.180+0000 7f3c527cf740 1 -- 172.21.15.77:0/1573731733 shutdown_connections 2024-09-10T09:56:28.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.180+0000 7f3c527cf740 1 -- 172.21.15.77:0/1573731733 wait complete. 2024-09-10T09:56:28.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.181+0000 7f3c527cf740 20 client.15642 trim_cache size 0 max 16384 2024-09-10T09:56:28.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.207+0000 7f3c527cf740 1 Processor -- start 2024-09-10T09:56:28.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.207+0000 7f3c527cf740 1 -- start start 2024-09-10T09:56:28.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.207+0000 7f3c527cf740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5ea275760 0x55e5ea2717b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:28.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.208+0000 7f3c527cf740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5ea27dcb0 0x55e5ea276fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:28.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.208+0000 7f3c527cf740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e5ea28be50 0x55e5ea298640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:28.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.208+0000 7f3c527cf740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3c481b52c0 con 0x55e5ea28be50 2024-09-10T09:56:28.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.208+0000 7f3c527cf740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3c481b6e00 con 0x55e5ea27dcb0 2024-09-10T09:56:28.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.208+0000 7f3c527cf740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3c481b5990 con 0x55e5ea275760 2024-09-10T09:56:28.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.208+0000 7f3c4d200640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5ea27dcb0 0x55e5ea276fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:28.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.208+0000 7f3c4d200640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5ea27dcb0 0x55e5ea276fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:56760/0 (socket says 172.21.15.77:56760) 2024-09-10T09:56:28.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.208+0000 7f3c4d200640 1 -- 172.21.15.77:0/1253139388 learned_addr learned my addr 172.21.15.77:0/1253139388 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:28.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.208+0000 7f3c47e00640 1 --2- 172.21.15.77:0/1253139388 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5ea275760 0x55e5ea2717b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:28.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.209+0000 7f3c47e00640 1 --2- 172.21.15.77:0/1253139388 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e5ea28be50 0x55e5ea298640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:28.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.209+0000 7f3c4d200640 1 -- 172.21.15.77:0/1253139388 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5ea275760 msgr2=0x55e5ea2717b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:28.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.209+0000 7f3c4d200640 1 --2- 172.21.15.77:0/1253139388 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5ea275760 0x55e5ea2717b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:28.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.209+0000 7f3c4d200640 1 -- 172.21.15.77:0/1253139388 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e5ea28be50 msgr2=0x55e5ea298640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:28.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.209+0000 7f3c4d200640 1 --2- 172.21.15.77:0/1253139388 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e5ea28be50 0x55e5ea298640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:28.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.209+0000 7f3c4d200640 1 -- 172.21.15.77:0/1253139388 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5ea27dab0 con 0x55e5ea27dcb0 2024-09-10T09:56:28.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.209+0000 7f3c47e00640 1 --2- 172.21.15.77:0/1253139388 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e5ea28be50 0x55e5ea298640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:56:28.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.210+0000 7f3c4d200640 1 --2- 172.21.15.77:0/1253139388 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5ea27dcb0 0x55e5ea276fa0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c0124a0 tx=0x7f3c3c011440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e086b6fd56981493 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:28.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.210+0000 7f3c39600640 1 -- 172.21.15.77:0/1253139388 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c3c012720 con 0x55e5ea27dcb0 2024-09-10T09:56:28.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.210+0000 7f3c39600640 1 -- 172.21.15.77:0/1253139388 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c3c02d070 con 0x55e5ea27dcb0 2024-09-10T09:56:28.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.210+0000 7f3c39600640 1 -- 172.21.15.77:0/1253139388 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c3c00fd60 con 0x55e5ea27dcb0 2024-09-10T09:56:28.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.211+0000 7f3c527cf740 1 -- 172.21.15.77:0/1253139388 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5ea27dcb0 msgr2=0x55e5ea276fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:28.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.211+0000 7f3c527cf740 1 --2- 172.21.15.77:0/1253139388 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5ea27dcb0 0x55e5ea276fa0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c0124a0 tx=0x7f3c3c011440 comp rx=0 tx=0).stop 2024-09-10T09:56:28.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.212+0000 7f3c527cf740 1 -- 172.21.15.77:0/1253139388 shutdown_connections 2024-09-10T09:56:28.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.212+0000 7f3c527cf740 1 --2- 172.21.15.77:0/1253139388 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e5ea28be50 0x55e5ea298640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:28.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.212+0000 7f3c527cf740 1 --2- 172.21.15.77:0/1253139388 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5ea27dcb0 0x55e5ea276fa0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:28.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.212+0000 7f3c527cf740 1 --2- 172.21.15.77:0/1253139388 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5ea275760 0x55e5ea2717b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:28.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.212+0000 7f3c527cf740 1 -- 172.21.15.77:0/1253139388 >> 172.21.15.77:0/1253139388 conn(0x55e5ea2713d0 msgr2=0x55e5ea276890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:28.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.212+0000 7f3c527cf740 1 -- 172.21.15.77:0/1253139388 shutdown_connections 2024-09-10T09:56:28.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.213+0000 7f3c527cf740 1 -- 172.21.15.77:0/1253139388 wait complete. 2024-09-10T09:56:28.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.214+0000 7f3c527cf740 1 -- start start 2024-09-10T09:56:28.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.214+0000 7f3c527cf740 1 Processor -- start 2024-09-10T09:56:28.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.215+0000 7f3c527cf740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5ea275760 0x55e5ea284450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:28.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.215+0000 7f3c527cf740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e5ea27dcb0 0x55e5ea284990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:28.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.215+0000 7f3c527cf740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5ea28be50 0x55e5ea2fa610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:28.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.215+0000 7f3c527cf740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3c481b6730 con 0x55e5ea27dcb0 2024-09-10T09:56:28.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.215+0000 7f3c527cf740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3c481b6060 con 0x55e5ea28be50 2024-09-10T09:56:28.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.215+0000 7f3c527cf740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3c481b32a0 con 0x55e5ea275760 2024-09-10T09:56:28.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.215+0000 7f3c527cf740 10 client.0 _ll_register_callbacks cb 0x7f3c3c0096c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:56:28.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.216+0000 7f3c47e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5ea28be50 0x55e5ea2fa610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:28.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.216+0000 7f3c47e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5ea28be50 0x55e5ea2fa610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:56770/0 (socket says 172.21.15.77:56770) 2024-09-10T09:56:28.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.216+0000 7f3c47e00640 1 -- 172.21.15.77:0/1483797231 learned_addr learned my addr 172.21.15.77:0/1483797231 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:28.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.216+0000 7f3c4d200640 1 --2- 172.21.15.77:0/1483797231 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e5ea27dcb0 0x55e5ea284990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:28.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.216+0000 7f3c47e00640 1 --2- 172.21.15.77:0/1483797231 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5ea275760 0x55e5ea284450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:28.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.217+0000 7f3c4d200640 1 -- 172.21.15.77:0/1483797231 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5ea275760 msgr2=0x55e5ea284450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:28.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.217+0000 7f3c4d200640 1 --2- 172.21.15.77:0/1483797231 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5ea275760 0x55e5ea284450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:28.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.217+0000 7f3c4d200640 1 -- 172.21.15.77:0/1483797231 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5ea28be50 msgr2=0x55e5ea2fa610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:28.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.217+0000 7f3c4d200640 1 --2- 172.21.15.77:0/1483797231 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5ea28be50 0x55e5ea2fa610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:28.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.217+0000 7f3c4d200640 1 -- 172.21.15.77:0/1483797231 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5ea2fab50 con 0x55e5ea27dcb0 2024-09-10T09:56:28.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.217+0000 7f3c47e00640 1 --2- 172.21.15.77:0/1483797231 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5ea28be50 0x55e5ea2fa610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:56:28.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.218+0000 7f3c2ea00640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-10T09:56:28.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.218+0000 7f3c4d200640 1 --2- 172.21.15.77:0/1483797231 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e5ea27dcb0 0x55e5ea284990 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x55e5ea27e670 tx=0x7f3c3c012750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66c17079307a1f9f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:28.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.218+0000 7f3c2ea00640 1 -- 172.21.15.77:0/1483797231 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c3c012dd0 con 0x55e5ea27dcb0 2024-09-10T09:56:28.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.218+0000 7f3c2ea00640 1 -- 172.21.15.77:0/1483797231 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f3c3c00fa00 con 0x55e5ea27dcb0 2024-09-10T09:56:28.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.218+0000 7f3c527cf740 1 -- 172.21.15.77:0/1483797231 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c3c02d070 con 0x55e5ea27dcb0 2024-09-10T09:56:28.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.219+0000 7f3c2ea00640 1 -- 172.21.15.77:0/1483797231 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3c3c040050 con 0x55e5ea27dcb0 2024-09-10T09:56:28.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.219+0000 7f3c527cf740 10 client.15648 fetch_fsmap learned FSMap version 29 2024-09-10T09:56:28.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.219+0000 7f3c527cf740 1 -- 172.21.15.77:0/1483797231 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e5ea2fbe50 con 0x55e5ea27dcb0 2024-09-10T09:56:28.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.220+0000 7f3c2ea00640 1 -- 172.21.15.77:0/1483797231 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c3c02d070 con 0x55e5ea27dcb0 2024-09-10T09:56:28.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.220+0000 7f3c2ea00640 1 -- 172.21.15.77:0/1483797231 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f3c3c005020 con 0x55e5ea27dcb0 2024-09-10T09:56:28.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.220+0000 7f3c527cf740 10 client.15648 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T09:56:28.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.220+0000 7f3c527cf740 10 client.15648 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-10T09:56:28.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.220+0000 7f3c527cf740 20 client.15648 populate_metadata read hostname 'smithi077' 2024-09-10T09:56:28.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.220+0000 7f3c527cf740 1 --2- 172.21.15.77:0/1483797231 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x55e5ea2fc400 0x55e5ea2fe820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:28.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.220+0000 7f3c527cf740 4 client.15648 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:56:28.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.220+0000 7f3c527cf740 1 -- 172.21.15.77:0/1483797231 --> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e5ea1e2e60 con 0x55e5ea2fc400 2024-09-10T09:56:28.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.220+0000 7f3c47e00640 1 --2- 172.21.15.77:0/1483797231 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x55e5ea2fc400 0x55e5ea2fe820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:28.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.221+0000 7f3c2ea00640 10 client.15648 ms_handle_connect on v2:172.21.15.120:6836/3033280709 2024-09-10T09:56:28.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.221+0000 7f3c47e00640 1 --2- 172.21.15.77:0/1483797231 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x55e5ea2fc400 0x55e5ea2fe820 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a2d43662292e928a server_cookie=779477698a0743e6 in_seq=0 out_seq=0 2024-09-10T09:56:28.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.222+0000 7f3c2ea00640 1 -- 172.21.15.77:0/1483797231 <== mds.1 v2:172.21.15.120:6836/3033280709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e5ea1e2e60 con 0x55e5ea2fc400 2024-09-10T09:56:28.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.222+0000 7f3c2ea00640 10 client.15648 handle_command_reply: tid=0 2024-09-10T09:56:28.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.222+0000 7f3c527cf740 1 client.15648 shutdown 2024-09-10T09:56:28.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.222+0000 7f3c527cf740 1 -- 172.21.15.77:0/1483797231 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e5ea27dcb0 msgr2=0x55e5ea284990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:28.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.222+0000 7f3c527cf740 1 --2- 172.21.15.77:0/1483797231 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e5ea27dcb0 0x55e5ea284990 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x55e5ea27e670 tx=0x7f3c3c012750 comp rx=0 tx=0).stop 2024-09-10T09:56:28.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.222+0000 7f3c527cf740 1 -- 172.21.15.77:0/1483797231 shutdown_connections 2024-09-10T09:56:28.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.222+0000 7f3c527cf740 1 --2- 172.21.15.77:0/1483797231 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x55e5ea2fc400 0x55e5ea2fe820 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:28.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.222+0000 7f3c527cf740 1 --2- 172.21.15.77:0/1483797231 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5ea28be50 0x55e5ea2fa610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:28.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.222+0000 7f3c527cf740 1 --2- 172.21.15.77:0/1483797231 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e5ea27dcb0 0x55e5ea284990 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:28.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.222+0000 7f3c527cf740 1 --2- 172.21.15.77:0/1483797231 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5ea275760 0x55e5ea284450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:28.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.222+0000 7f3c2ea00640 0 client.15648 ms_handle_reset on v2:172.21.15.120:6836/3033280709 2024-09-10T09:56:28.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.222+0000 7f3c527cf740 1 -- 172.21.15.77:0/1483797231 >> 172.21.15.77:0/1483797231 conn(0x55e5ea2713d0 msgr2=0x55e5ea2770b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:28.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.223+0000 7f3c527cf740 1 -- 172.21.15.77:0/1483797231 shutdown_connections 2024-09-10T09:56:28.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.223+0000 7f3c527cf740 1 -- 172.21.15.77:0/1483797231 wait complete. 2024-09-10T09:56:28.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.223+0000 7f3c527cf740 20 client.15648 trim_cache size 0 max 16384 2024-09-10T09:56:28.225 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-10T09:56:28.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.224+0000 7f3c4e600640 1 -- 172.21.15.77:0/385843695 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f3c300a2c00 msgr2=0x7f3c300a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:28.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.224+0000 7f3c4e600640 1 --2- 172.21.15.77:0/385843695 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f3c300a2c00 0x7f3c300a50c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f3c400bcb70 tx=0x7f3c4003e040 comp rx=0 tx=0).stop 2024-09-10T09:56:28.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.224+0000 7f3c4e600640 1 -- 172.21.15.77:0/385843695 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3c48108f60 msgr2=0x7f3c4810df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:28.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.224+0000 7f3c4e600640 1 --2- 172.21.15.77:0/385843695 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3c48108f60 0x7f3c4810df00 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f3c40005e30 tx=0x7f3c40003d00 comp rx=0 tx=0).stop 2024-09-10T09:56:28.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.225+0000 7f3c4e600640 1 -- 172.21.15.77:0/385843695 shutdown_connections 2024-09-10T09:56:28.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.225+0000 7f3c4e600640 1 --2- 172.21.15.77:0/385843695 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f3c300a2c00 0x7f3c300a50c0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:28.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.225+0000 7f3c4e600640 1 --2- 172.21.15.77:0/385843695 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3c48108f60 0x7f3c4810df00 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:28.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.225+0000 7f3c4e600640 1 --2- 172.21.15.77:0/385843695 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c481047e0 0x7f3c4810d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:28.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.225+0000 7f3c4e600640 1 --2- 172.21.15.77:0/385843695 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3c48103e30 0x7f3c48110910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:28.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.225+0000 7f3c4e600640 1 -- 172.21.15.77:0/385843695 >> 172.21.15.77:0/385843695 conn(0x7f3c480ff910 msgr2=0x7f3c48100be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:28.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.225+0000 7f3c4e600640 1 -- 172.21.15.77:0/385843695 shutdown_connections 2024-09-10T09:56:28.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:28.225+0000 7f3c4e600640 1 -- 172.21.15.77:0/385843695 wait complete. 2024-09-10T09:56:29.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:56:28 smithi134 ceph-mon[35674]: pgmap v430: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-10T09:56:29.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:56:28 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:56:29.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:56:28 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:56:29.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:56:28 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:56:29.137 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:56:28 smithi077 ceph-mon[30319]: pgmap v430: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-10T09:56:29.137 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:56:28 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:56:29.137 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:56:28 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:56:29.137 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:56:28 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:56:29.139 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T09:56:29.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:56:28 smithi120 ceph-mon[34770]: pgmap v430: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-10T09:56:29.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:56:28 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:56:29.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:56:28 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:56:29.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:56:28 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:56:29.715 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:56:30.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.423+0000 7faf0d800640 1 -- 172.21.15.77:0/3380348514 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faf08074f40 msgr2=0x7faf0810dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:30.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.423+0000 7faf0d800640 1 --2- 172.21.15.77:0/3380348514 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faf08074f40 0x7faf0810dcd0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7faef0009860 tx=0x7faef002a3d0 comp rx=0 tx=0).stop 2024-09-10T09:56:30.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.423+0000 7faf0d800640 1 -- 172.21.15.77:0/3380348514 shutdown_connections 2024-09-10T09:56:30.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.423+0000 7faf0d800640 1 --2- 172.21.15.77:0/3380348514 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faf0810e210 0x7faf08110660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:30.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.423+0000 7faf0d800640 1 --2- 172.21.15.77:0/3380348514 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faf08074f40 0x7faf0810dcd0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:30.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.423+0000 7faf0d800640 1 --2- 172.21.15.77:0/3380348514 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faf08074620 0x7faf08074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:30.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.423+0000 7faf0d800640 1 -- 172.21.15.77:0/3380348514 >> 172.21.15.77:0/3380348514 conn(0x7faf080fd710 msgr2=0x7faf080ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:30.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.423+0000 7faf0d800640 1 -- 172.21.15.77:0/3380348514 shutdown_connections 2024-09-10T09:56:30.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.425+0000 7faf0d800640 1 -- 172.21.15.77:0/3380348514 wait complete. 2024-09-10T09:56:30.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.426+0000 7faf0d800640 1 Processor -- start 2024-09-10T09:56:30.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.427+0000 7faf0d800640 1 -- start start 2024-09-10T09:56:30.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.427+0000 7faf0d800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faf08074620 0x7faf08101930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:30.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.427+0000 7faf0d800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faf0810e210 0x7faf08101e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:30.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.427+0000 7faf0d800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faf081023b0 0x7faf081aac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:30.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.427+0000 7faf0d800640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7faf08113fd0 con 0x7faf081023b0 2024-09-10T09:56:30.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.427+0000 7faf0d800640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7faf08113e50 con 0x7faf08074620 2024-09-10T09:56:30.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.427+0000 7faf0d800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7faf08114150 con 0x7faf0810e210 2024-09-10T09:56:30.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.427+0000 7faf07e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faf08074620 0x7faf08101930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:30.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.427+0000 7faeffe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faf0810e210 0x7faf08101e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:30.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.428+0000 7faeffe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faf0810e210 0x7faf08101e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:56504/0 (socket says 172.21.15.77:56504) 2024-09-10T09:56:30.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.428+0000 7faeffe00640 1 -- 172.21.15.77:0/3690218634 learned_addr learned my addr 172.21.15.77:0/3690218634 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:30.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.428+0000 7faf0ce00640 1 --2- 172.21.15.77:0/3690218634 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faf081023b0 0x7faf081aac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:30.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.428+0000 7faeffe00640 1 -- 172.21.15.77:0/3690218634 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faf08074620 msgr2=0x7faf08101930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:30.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.428+0000 7faeffe00640 1 --2- 172.21.15.77:0/3690218634 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faf08074620 0x7faf08101930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:30.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.428+0000 7faeffe00640 1 -- 172.21.15.77:0/3690218634 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faf081023b0 msgr2=0x7faf081aac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:30.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.428+0000 7faeffe00640 1 --2- 172.21.15.77:0/3690218634 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faf081023b0 0x7faf081aac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:30.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.428+0000 7faeffe00640 1 -- 172.21.15.77:0/3690218634 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf081ab1a0 con 0x7faf0810e210 2024-09-10T09:56:30.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.428+0000 7faf0ce00640 1 --2- 172.21.15.77:0/3690218634 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faf081023b0 0x7faf081aac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:56:30.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.429+0000 7faeffe00640 1 --2- 172.21.15.77:0/3690218634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faf0810e210 0x7faf08101e70 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7faef0034000 tx=0x7faef00309a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:30.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.429+0000 7faf05600640 1 -- 172.21.15.77:0/3690218634 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef0040020 con 0x7faf0810e210 2024-09-10T09:56:30.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.429+0000 7faf05600640 1 -- 172.21.15.77:0/3690218634 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faef0004440 con 0x7faf0810e210 2024-09-10T09:56:30.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.429+0000 7faf05600640 1 -- 172.21.15.77:0/3690218634 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef0004920 con 0x7faf0810e210 2024-09-10T09:56:30.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.429+0000 7faf0d800640 1 -- 172.21.15.77:0/3690218634 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf081ab430 con 0x7faf0810e210 2024-09-10T09:56:30.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.429+0000 7faf0d800640 1 -- 172.21.15.77:0/3690218634 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf081ab890 con 0x7faf0810e210 2024-09-10T09:56:30.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.434+0000 7faf117e2740 1 Processor -- start 2024-09-10T09:56:30.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.435+0000 7faf117e2740 1 -- start start 2024-09-10T09:56:30.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.435+0000 7faf117e2740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55590e8c8e40 0x55590e8c9220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:30.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.435+0000 7faf117e2740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55590e8c9760 0x55590e8d3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:30.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.435+0000 7faf117e2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55590e8d4020 0x55590e8d6410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:30.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.435+0000 7faf117e2740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55590e82f3e0 con 0x55590e8c9760 2024-09-10T09:56:30.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.435+0000 7faf117e2740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55590e7d6d80 con 0x55590e8c8e40 2024-09-10T09:56:30.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.435+0000 7faf117e2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55590e82b3a0 con 0x55590e8d4020 2024-09-10T09:56:30.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.436+0000 7faf05600640 1 -- 172.21.15.77:0/3690218634 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7faef0043020 con 0x7faf0810e210 2024-09-10T09:56:30.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.436+0000 7faf0ce00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55590e8c8e40 0x55590e8c9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:30.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.436+0000 7faf0ce00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55590e8c8e40 0x55590e8c9220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:56796/0 (socket says 172.21.15.77:56796) 2024-09-10T09:56:30.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.436+0000 7faf0ce00640 1 -- 172.21.15.77:0/3363348320 learned_addr learned my addr 172.21.15.77:0/3363348320 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:30.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.436+0000 7faeffe00640 1 --2- 172.21.15.77:0/3363348320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55590e8d4020 0x55590e8d6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:30.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.436+0000 7faf0ce00640 1 -- 172.21.15.77:0/3363348320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55590e8d4020 msgr2=0x55590e8d6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:30.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.436+0000 7faf0ce00640 1 --2- 172.21.15.77:0/3363348320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55590e8d4020 0x55590e8d6410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:30.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.436+0000 7faf0ce00640 1 -- 172.21.15.77:0/3363348320 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55590e8c9760 msgr2=0x55590e8d3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:30.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.436+0000 7faf07e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55590e8c9760 0x55590e8d3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:30.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.436+0000 7faf0ce00640 1 --2- 172.21.15.77:0/3363348320 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55590e8c9760 0x55590e8d3ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:30.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.436+0000 7faf0ce00640 1 -- 172.21.15.77:0/3363348320 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55590e7f5ae0 con 0x55590e8c8e40 2024-09-10T09:56:30.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.437+0000 7faf07e00640 1 --2- 172.21.15.77:0/3363348320 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55590e8c9760 0x55590e8d3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:56:30.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.437+0000 7faf0ce00640 1 --2- 172.21.15.77:0/3363348320 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55590e8c8e40 0x55590e8c9220 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7faf0000e870 tx=0x7faf000095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6811953914a7927 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:30.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.438+0000 7faefd600640 1 -- 172.21.15.77:0/3363348320 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0000ddd0 con 0x55590e8c8e40 2024-09-10T09:56:30.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.438+0000 7faefd600640 1 -- 172.21.15.77:0/3363348320 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf0001c070 con 0x55590e8c8e40 2024-09-10T09:56:30.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.438+0000 7faefd600640 1 -- 172.21.15.77:0/3363348320 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf00016700 con 0x55590e8c8e40 2024-09-10T09:56:30.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.438+0000 7faf05600640 1 --2- 172.21.15.77:0/3690218634 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7faeec0a2c80 0x7faeec0a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:30.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.438+0000 7faf05600640 1 -- 172.21.15.77:0/3690218634 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7faef00f0150 con 0x7faf0810e210 2024-09-10T09:56:30.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.439+0000 7faf117e2740 1 -- 172.21.15.77:0/3363348320 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55590e8c8e40 msgr2=0x55590e8c9220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:30.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.439+0000 7faf117e2740 1 --2- 172.21.15.77:0/3363348320 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55590e8c8e40 0x55590e8c9220 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7faf0000e870 tx=0x7faf000095c0 comp rx=0 tx=0).stop 2024-09-10T09:56:30.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.439+0000 7faf117e2740 1 -- 172.21.15.77:0/3363348320 shutdown_connections 2024-09-10T09:56:30.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.439+0000 7faf117e2740 1 --2- 172.21.15.77:0/3363348320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55590e8d4020 0x55590e8d6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:30.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.439+0000 7faf117e2740 1 --2- 172.21.15.77:0/3363348320 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55590e8c9760 0x55590e8d3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:30.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.439+0000 7faf117e2740 1 --2- 172.21.15.77:0/3363348320 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55590e8c8e40 0x55590e8c9220 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:30.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.439+0000 7faf117e2740 1 -- 172.21.15.77:0/3363348320 >> 172.21.15.77:0/3363348320 conn(0x55590e8c53d0 msgr2=0x55590e8c57b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:30.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.439+0000 7faf117e2740 1 -- 172.21.15.77:0/3363348320 shutdown_connections 2024-09-10T09:56:30.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.439+0000 7faf117e2740 1 -- 172.21.15.77:0/3363348320 wait complete. 2024-09-10T09:56:30.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.440+0000 7faf117e2740 1 -- start start 2024-09-10T09:56:30.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.440+0000 7faf117e2740 1 Processor -- start 2024-09-10T09:56:30.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.441+0000 7faf117e2740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55590e8c8e40 0x55590e8d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:30.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.441+0000 7faf117e2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55590e8c9760 0x55590e8d1500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:30.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.441+0000 7faf117e2740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55590e8d4020 0x55590e8d1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:30.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.441+0000 7faf117e2740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55590e8d6bf0 con 0x55590e8d4020 2024-09-10T09:56:30.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.441+0000 7faf117e2740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55590e7ea0f0 con 0x55590e8c8e40 2024-09-10T09:56:30.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.441+0000 7faf117e2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55590e8d6d70 con 0x55590e8c9760 2024-09-10T09:56:30.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.441+0000 7faf117e2740 10 client.0 _ll_register_callbacks cb 0x55590e7d1bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:56:30.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.441+0000 7faeffe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55590e8c9760 0x55590e8d1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:30.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.441+0000 7faf07e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55590e8c8e40 0x55590e8d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:30.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.441+0000 7faeffe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55590e8c9760 0x55590e8d1500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:56522/0 (socket says 172.21.15.77:56522) 2024-09-10T09:56:30.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.441+0000 7faeffe00640 1 -- 172.21.15.77:0/3695621255 learned_addr learned my addr 172.21.15.77:0/3695621255 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:30.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.441+0000 7faf0ce00640 1 --2- 172.21.15.77:0/3695621255 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55590e8d4020 0x55590e8d1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:30.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.442+0000 7faeffe00640 1 -- 172.21.15.77:0/3695621255 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55590e8c8e40 msgr2=0x55590e8d0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:30.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.442+0000 7faeffe00640 1 --2- 172.21.15.77:0/3695621255 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55590e8c8e40 0x55590e8d0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:30.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.442+0000 7faeffe00640 1 -- 172.21.15.77:0/3695621255 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55590e8d4020 msgr2=0x55590e8d1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:30.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.442+0000 7faeffe00640 1 --2- 172.21.15.77:0/3695621255 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55590e8d4020 0x55590e8d1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:30.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.442+0000 7faeffe00640 1 -- 172.21.15.77:0/3695621255 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55590e5a03c0 con 0x55590e8c9760 2024-09-10T09:56:30.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.442+0000 7faf07e00640 1 --2- 172.21.15.77:0/3695621255 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55590e8c8e40 0x55590e8d0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:56:30.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.442+0000 7faf0ce00640 1 --2- 172.21.15.77:0/3695621255 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55590e8d4020 0x55590e8d1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:56:30.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.442+0000 7faeffe00640 1 --2- 172.21.15.77:0/3695621255 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55590e8c9760 0x55590e8d1500 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7faef00c4ca0 tx=0x7faef0031400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4e2c51e546e5d0d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:30.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.442+0000 7faef6000640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-10T09:56:30.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.443+0000 7faef6000640 1 -- 172.21.15.77:0/3695621255 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef00b8800 con 0x55590e8c9760 2024-09-10T09:56:30.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.443+0000 7faf117e2740 1 -- 172.21.15.77:0/3695621255 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf0001c070 con 0x55590e8c9760 2024-09-10T09:56:30.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.443+0000 7faf07e00640 1 --2- 172.21.15.77:0/3690218634 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7faeec0a2c80 0x7faeec0a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:30.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.443+0000 7faef6000640 1 -- 172.21.15.77:0/3695621255 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7faef00b8bd0 con 0x55590e8c9760 2024-09-10T09:56:30.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.443+0000 7faef6000640 1 -- 172.21.15.77:0/3695621255 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7faef00bbec0 con 0x55590e8c9760 2024-09-10T09:56:30.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.443+0000 7faf117e2740 10 client.25684 fetch_fsmap learned FSMap version 29 2024-09-10T09:56:30.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.443+0000 7faf117e2740 1 -- 172.21.15.77:0/3695621255 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55590e8d6ca0 con 0x55590e8c9760 2024-09-10T09:56:30.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.443+0000 7faef6000640 1 -- 172.21.15.77:0/3695621255 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf0001c070 con 0x55590e8c9760 2024-09-10T09:56:30.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.444+0000 7faf07e00640 1 --2- 172.21.15.77:0/3690218634 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7faeec0a2c80 0x7faeec0a5140 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7faef8009400 tx=0x7faef8014040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:30.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.444+0000 7faef6000640 1 -- 172.21.15.77:0/3695621255 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7faef0031490 con 0x55590e8c9760 2024-09-10T09:56:30.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.444+0000 7faf117e2740 10 client.25684 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T09:56:30.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.444+0000 7faf117e2740 10 client.25684 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:56:30.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.444+0000 7faf117e2740 20 client.25684 populate_metadata read hostname 'smithi077' 2024-09-10T09:56:30.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.444+0000 7faf117e2740 1 --2- 172.21.15.77:0/3695621255 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55590e8f4eb0 0x55590e8f72d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:30.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.444+0000 7faf117e2740 4 client.25684 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:56:30.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.444+0000 7faf117e2740 1 -- 172.21.15.77:0/3695621255 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55590e83c950 con 0x55590e8f4eb0 2024-09-10T09:56:30.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.445+0000 7faf07e00640 1 --2- 172.21.15.77:0/3695621255 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55590e8f4eb0 0x55590e8f72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:30.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.445+0000 7faf07e00640 1 --2- 172.21.15.77:0/3695621255 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55590e8f4eb0 0x55590e8f72d0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=522a64d271da373a server_cookie=e21ca9c8872e6787 in_seq=0 out_seq=0 2024-09-10T09:56:30.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.445+0000 7faef6000640 10 client.25684 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T09:56:30.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.446+0000 7faef6000640 1 -- 172.21.15.77:0/3695621255 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55590e83c950 con 0x55590e8f4eb0 2024-09-10T09:56:30.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.446+0000 7faef6000640 10 client.25684 handle_command_reply: tid=0 2024-09-10T09:56:30.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.446+0000 7faf117e2740 1 client.25684 shutdown 2024-09-10T09:56:30.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.447+0000 7faf117e2740 1 -- 172.21.15.77:0/3695621255 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55590e8c9760 msgr2=0x55590e8d1500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:30.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.447+0000 7faf117e2740 1 --2- 172.21.15.77:0/3695621255 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55590e8c9760 0x55590e8d1500 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7faef00c4ca0 tx=0x7faef0031400 comp rx=0 tx=0).stop 2024-09-10T09:56:30.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.447+0000 7faf117e2740 1 -- 172.21.15.77:0/3695621255 shutdown_connections 2024-09-10T09:56:30.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.447+0000 7faf117e2740 1 --2- 172.21.15.77:0/3695621255 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55590e8f4eb0 0x55590e8f72d0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:30.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.447+0000 7faf117e2740 1 --2- 172.21.15.77:0/3695621255 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55590e8d4020 0x55590e8d1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:30.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.447+0000 7faf117e2740 1 --2- 172.21.15.77:0/3695621255 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55590e8c9760 0x55590e8d1500 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:30.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.447+0000 7faef6000640 0 client.25684 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T09:56:30.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.447+0000 7faf117e2740 1 --2- 172.21.15.77:0/3695621255 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55590e8c8e40 0x55590e8d0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:30.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.447+0000 7faf117e2740 1 -- 172.21.15.77:0/3695621255 >> 172.21.15.77:0/3695621255 conn(0x55590e8c53d0 msgr2=0x55590e8cb150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:30.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.447+0000 7faf117e2740 1 -- 172.21.15.77:0/3695621255 shutdown_connections 2024-09-10T09:56:30.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.447+0000 7faf117e2740 1 -- 172.21.15.77:0/3695621255 wait complete. 2024-09-10T09:56:30.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.447+0000 7faf117e2740 20 client.25684 trim_cache size 0 max 16384 2024-09-10T09:56:30.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.472+0000 7faf117e2740 1 Processor -- start 2024-09-10T09:56:30.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.472+0000 7faf117e2740 1 -- start start 2024-09-10T09:56:30.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.473+0000 7faf117e2740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55590e8c8e40 0x55590e8cf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:30.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.473+0000 7faf117e2740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55590e8c9760 0x55590e8d1890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:30.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.473+0000 7faf117e2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55590e8d4020 0x55590e8fcb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:30.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.473+0000 7faf117e2740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7faf0806d060 con 0x55590e8c8e40 2024-09-10T09:56:30.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.473+0000 7faf117e2740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7faf0806eba0 con 0x55590e8c9760 2024-09-10T09:56:30.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.473+0000 7faf117e2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7faf0806d730 con 0x55590e8d4020 2024-09-10T09:56:30.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.473+0000 7faeffe00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55590e8c9760 0x55590e8d1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:30.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.473+0000 7faeffe00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55590e8c9760 0x55590e8d1890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:56802/0 (socket says 172.21.15.77:56802) 2024-09-10T09:56:30.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.473+0000 7faeffe00640 1 -- 172.21.15.77:0/1778700757 learned_addr learned my addr 172.21.15.77:0/1778700757 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:30.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.473+0000 7faf07e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55590e8c8e40 0x55590e8cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:30.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.474+0000 7faf0ce00640 1 --2- 172.21.15.77:0/1778700757 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55590e8d4020 0x55590e8fcb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:30.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.474+0000 7faeffe00640 1 -- 172.21.15.77:0/1778700757 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55590e8d4020 msgr2=0x55590e8fcb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:30.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.474+0000 7faeffe00640 1 --2- 172.21.15.77:0/1778700757 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55590e8d4020 0x55590e8fcb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:30.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.474+0000 7faeffe00640 1 -- 172.21.15.77:0/1778700757 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55590e8c8e40 msgr2=0x55590e8cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:30.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.474+0000 7faeffe00640 1 --2- 172.21.15.77:0/1778700757 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55590e8c8e40 0x55590e8cf890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:30.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.474+0000 7faeffe00640 1 -- 172.21.15.77:0/1778700757 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55590e8c5200 con 0x55590e8c9760 2024-09-10T09:56:30.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.474+0000 7faf07e00640 1 --2- 172.21.15.77:0/1778700757 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55590e8c8e40 0x55590e8cf890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:56:30.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.475+0000 7faeffe00640 1 --2- 172.21.15.77:0/1778700757 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55590e8c9760 0x55590e8d1890 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7faef00bc260 tx=0x7faef00baa70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b9656fde7fb8a8f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:30.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.475+0000 7faefcc00640 1 -- 172.21.15.77:0/1778700757 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef00bc0a0 con 0x55590e8c9760 2024-09-10T09:56:30.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.475+0000 7faefcc00640 1 -- 172.21.15.77:0/1778700757 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faef00bcc60 con 0x55590e8c9760 2024-09-10T09:56:30.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.475+0000 7faefcc00640 1 -- 172.21.15.77:0/1778700757 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef00bc330 con 0x55590e8c9760 2024-09-10T09:56:30.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.476+0000 7faf117e2740 1 -- 172.21.15.77:0/1778700757 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55590e8c9760 msgr2=0x55590e8d1890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:30.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.476+0000 7faf117e2740 1 --2- 172.21.15.77:0/1778700757 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55590e8c9760 0x55590e8d1890 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7faef00bc260 tx=0x7faef00baa70 comp rx=0 tx=0).stop 2024-09-10T09:56:30.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.476+0000 7faf117e2740 1 -- 172.21.15.77:0/1778700757 shutdown_connections 2024-09-10T09:56:30.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.476+0000 7faf117e2740 1 --2- 172.21.15.77:0/1778700757 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55590e8d4020 0x55590e8fcb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:30.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.476+0000 7faf117e2740 1 --2- 172.21.15.77:0/1778700757 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55590e8c9760 0x55590e8d1890 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:30.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.476+0000 7faf117e2740 1 --2- 172.21.15.77:0/1778700757 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55590e8c8e40 0x55590e8cf890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:30.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.476+0000 7faf117e2740 1 -- 172.21.15.77:0/1778700757 >> 172.21.15.77:0/1778700757 conn(0x55590e8c53d0 msgr2=0x55590e8d3040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:30.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.476+0000 7faf117e2740 1 -- 172.21.15.77:0/1778700757 shutdown_connections 2024-09-10T09:56:30.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.477+0000 7faf117e2740 1 -- 172.21.15.77:0/1778700757 wait complete. 2024-09-10T09:56:30.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.477+0000 7faf117e2740 1 -- start start 2024-09-10T09:56:30.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.478+0000 7faf117e2740 1 Processor -- start 2024-09-10T09:56:30.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.478+0000 7faf117e2740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55590e8c8e40 0x55590e8ed560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:30.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.478+0000 7faf117e2740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55590e8c9760 0x55590e8e8890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:30.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.478+0000 7faf117e2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55590e8d4020 0x55590e8e8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:30.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.478+0000 7faf117e2740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7faf0806e4d0 con 0x55590e8c9760 2024-09-10T09:56:30.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.478+0000 7faf117e2740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7faf0806de00 con 0x55590e8c8e40 2024-09-10T09:56:30.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.478+0000 7faf117e2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7faf081a2030 con 0x55590e8d4020 2024-09-10T09:56:30.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.478+0000 7faf117e2740 10 client.0 _ll_register_callbacks cb 0x7faef0005f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:56:30.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.479+0000 7faeffe00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55590e8c9760 0x55590e8e8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:30.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.479+0000 7faf0ce00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55590e8d4020 0x55590e8e8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:30.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.479+0000 7faf0ce00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55590e8d4020 0x55590e8e8dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:56532/0 (socket says 172.21.15.77:56532) 2024-09-10T09:56:30.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.479+0000 7faf0ce00640 1 -- 172.21.15.77:0/675852911 learned_addr learned my addr 172.21.15.77:0/675852911 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:30.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.479+0000 7faf07e00640 1 --2- 172.21.15.77:0/675852911 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55590e8c8e40 0x55590e8ed560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:30.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.480+0000 7faeffe00640 1 -- 172.21.15.77:0/675852911 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55590e8d4020 msgr2=0x55590e8e8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:30.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.480+0000 7faeffe00640 1 --2- 172.21.15.77:0/675852911 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55590e8d4020 0x55590e8e8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:30.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.480+0000 7faeffe00640 1 -- 172.21.15.77:0/675852911 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55590e8c8e40 msgr2=0x55590e8ed560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:30.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.480+0000 7faeffe00640 1 --2- 172.21.15.77:0/675852911 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55590e8c8e40 0x55590e8ed560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:30.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.480+0000 7faeffe00640 1 -- 172.21.15.77:0/675852911 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55590e8c6480 con 0x55590e8c9760 2024-09-10T09:56:30.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.480+0000 7faf07e00640 1 --2- 172.21.15.77:0/675852911 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55590e8c8e40 0x55590e8ed560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:56:30.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.480+0000 7faeffe00640 1 --2- 172.21.15.77:0/675852911 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55590e8c9760 0x55590e8e8890 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x55590e8b9c50 tx=0x7faef002a3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9a5782ab65536fd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:30.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.480+0000 7faef6000640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-10T09:56:30.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.481+0000 7faef6000640 1 -- 172.21.15.77:0/675852911 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef00bc330 con 0x55590e8c9760 2024-09-10T09:56:30.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.481+0000 7faf117e2740 1 -- 172.21.15.77:0/675852911 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faef00bcc60 con 0x55590e8c9760 2024-09-10T09:56:30.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.481+0000 7faef6000640 1 -- 172.21.15.77:0/675852911 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7faef00a5af0 con 0x55590e8c9760 2024-09-10T09:56:30.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.481+0000 7faef6000640 1 -- 172.21.15.77:0/675852911 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7faef00c0910 con 0x55590e8c9760 2024-09-10T09:56:30.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.481+0000 7faf117e2740 10 client.15663 fetch_fsmap learned FSMap version 29 2024-09-10T09:56:30.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.481+0000 7faf117e2740 1 -- 172.21.15.77:0/675852911 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55590e8c6480 con 0x55590e8c9760 2024-09-10T09:56:30.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.482+0000 7faef6000640 1 -- 172.21.15.77:0/675852911 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faef00bcc60 con 0x55590e8c9760 2024-09-10T09:56:30.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.482+0000 7faef6000640 1 -- 172.21.15.77:0/675852911 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7faef00c1aa0 con 0x55590e8c9760 2024-09-10T09:56:30.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.482+0000 7faf117e2740 10 client.15663 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T09:56:30.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.482+0000 7faf117e2740 10 client.15663 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:56:30.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.482+0000 7faf117e2740 20 client.15663 populate_metadata read hostname 'smithi077' 2024-09-10T09:56:30.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.482+0000 7faf117e2740 1 --2- 172.21.15.77:0/675852911 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55590e95f990 0x55590e961d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:30.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.482+0000 7faf117e2740 4 client.15663 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:56:30.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.482+0000 7faf117e2740 1 -- 172.21.15.77:0/675852911 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55590e836e60 con 0x55590e95f990 2024-09-10T09:56:30.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.483+0000 7faf07e00640 1 --2- 172.21.15.77:0/675852911 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55590e95f990 0x55590e961d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:30.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.483+0000 7faf07e00640 1 --2- 172.21.15.77:0/675852911 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55590e95f990 0x55590e961d80 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7f467783d9c62cf server_cookie=397e4d48c00f7060 in_seq=0 out_seq=0 2024-09-10T09:56:30.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.483+0000 7faef6000640 10 client.15663 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T09:56:30.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.484+0000 7faef6000640 1 -- 172.21.15.77:0/675852911 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55590e836e60 con 0x55590e95f990 2024-09-10T09:56:30.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.484+0000 7faef6000640 10 client.15663 handle_command_reply: tid=0 2024-09-10T09:56:30.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.484+0000 7faf117e2740 1 client.15663 shutdown 2024-09-10T09:56:30.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.485+0000 7faf117e2740 1 -- 172.21.15.77:0/675852911 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55590e8c9760 msgr2=0x55590e8e8890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:30.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.485+0000 7faf117e2740 1 --2- 172.21.15.77:0/675852911 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55590e8c9760 0x55590e8e8890 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x55590e8b9c50 tx=0x7faef002a3d0 comp rx=0 tx=0).stop 2024-09-10T09:56:30.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.485+0000 7faf117e2740 1 -- 172.21.15.77:0/675852911 shutdown_connections 2024-09-10T09:56:30.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.485+0000 7faf117e2740 1 --2- 172.21.15.77:0/675852911 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55590e95f990 0x55590e961d80 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:30.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.485+0000 7faf117e2740 1 --2- 172.21.15.77:0/675852911 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55590e8d4020 0x55590e8e8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:30.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.485+0000 7faf117e2740 1 --2- 172.21.15.77:0/675852911 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55590e8c9760 0x55590e8e8890 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:30.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.485+0000 7faf117e2740 1 --2- 172.21.15.77:0/675852911 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55590e8c8e40 0x55590e8ed560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:30.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.485+0000 7faf117e2740 1 -- 172.21.15.77:0/675852911 >> 172.21.15.77:0/675852911 conn(0x55590e8c53d0 msgr2=0x55590e8d2820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:30.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.485+0000 7faef6000640 0 client.15663 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T09:56:30.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.486+0000 7faf117e2740 1 -- 172.21.15.77:0/675852911 shutdown_connections 2024-09-10T09:56:30.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.486+0000 7faf117e2740 1 -- 172.21.15.77:0/675852911 wait complete. 2024-09-10T09:56:30.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.486+0000 7faf117e2740 20 client.15663 trim_cache size 0 max 16384 2024-09-10T09:56:30.488 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-10T09:56:30.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.487+0000 7faf0d800640 1 -- 172.21.15.77:0/3690218634 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7faeec0a2c80 msgr2=0x7faeec0a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:30.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.487+0000 7faf0d800640 1 --2- 172.21.15.77:0/3690218634 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7faeec0a2c80 0x7faeec0a5140 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7faef8009400 tx=0x7faef8014040 comp rx=0 tx=0).stop 2024-09-10T09:56:30.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.487+0000 7faf0d800640 1 -- 172.21.15.77:0/3690218634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faf0810e210 msgr2=0x7faf08101e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:30.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.487+0000 7faf0d800640 1 --2- 172.21.15.77:0/3690218634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faf0810e210 0x7faf08101e70 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7faef0034000 tx=0x7faef00309a0 comp rx=0 tx=0).stop 2024-09-10T09:56:30.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.488+0000 7faf0d800640 1 -- 172.21.15.77:0/3690218634 shutdown_connections 2024-09-10T09:56:30.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.488+0000 7faf0d800640 1 --2- 172.21.15.77:0/3690218634 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7faeec0a2c80 0x7faeec0a5140 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:30.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.488+0000 7faf0d800640 1 --2- 172.21.15.77:0/3690218634 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faf081023b0 0x7faf081aac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:30.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.488+0000 7faf0d800640 1 --2- 172.21.15.77:0/3690218634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faf0810e210 0x7faf08101e70 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:30.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.488+0000 7faf0d800640 1 --2- 172.21.15.77:0/3690218634 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faf08074620 0x7faf08101930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:30.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.488+0000 7faf0d800640 1 -- 172.21.15.77:0/3690218634 >> 172.21.15.77:0/3690218634 conn(0x7faf080fd710 msgr2=0x7faf080fefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:30.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.488+0000 7faf0d800640 1 -- 172.21.15.77:0/3690218634 shutdown_connections 2024-09-10T09:56:30.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:30.488+0000 7faf0d800640 1 -- 172.21.15.77:0/3690218634 wait complete. 2024-09-10T09:56:31.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:56:30 smithi120 ceph-mon[34770]: pgmap v431: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 85 B/s wr, 12 op/s 2024-09-10T09:56:31.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:56:30 smithi077 ceph-mon[30319]: pgmap v431: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 85 B/s wr, 12 op/s 2024-09-10T09:56:31.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:56:30 smithi134 ceph-mon[35674]: pgmap v431: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 85 B/s wr, 12 op/s 2024-09-10T09:56:32.383 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T09:56:32.383 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T09:56:32.948 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:56:33.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:56:32 smithi120 ceph-mon[34770]: pgmap v432: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-10T09:56:33.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:56:32 smithi077 ceph-mon[30319]: pgmap v432: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-10T09:56:33.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:56:32 smithi134 ceph-mon[35674]: pgmap v432: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-10T09:56:33.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.605+0000 7f0cf7200640 1 -- 172.21.15.77:0/3412864526 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0cf8112870 msgr2=0x7f0cf8114c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:33.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.605+0000 7f0cf7200640 1 --2- 172.21.15.77:0/3412864526 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0cf8112870 0x7f0cf8114c60 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0cf0009be0 tx=0x7f0cf002a6d0 comp rx=0 tx=0).stop 2024-09-10T09:56:33.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.605+0000 7f0cf7200640 1 -- 172.21.15.77:0/3412864526 shutdown_connections 2024-09-10T09:56:33.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.605+0000 7f0cf7200640 1 --2- 172.21.15.77:0/3412864526 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0cf8112870 0x7f0cf8114c60 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:33.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.605+0000 7f0cf7200640 1 --2- 172.21.15.77:0/3412864526 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0cf8077040 0x7f0cf8112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:33.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.605+0000 7f0cf7200640 1 --2- 172.21.15.77:0/3412864526 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0cf8076720 0x7f0cf8076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:33.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.605+0000 7f0cf7200640 1 -- 172.21.15.77:0/3412864526 >> 172.21.15.77:0/3412864526 conn(0x7f0cf80ff910 msgr2=0x7f0cf8101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:33.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.605+0000 7f0cf7200640 1 -- 172.21.15.77:0/3412864526 shutdown_connections 2024-09-10T09:56:33.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.605+0000 7f0cf7200640 1 -- 172.21.15.77:0/3412864526 wait complete. 2024-09-10T09:56:33.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.607+0000 7f0cf7200640 1 Processor -- start 2024-09-10T09:56:33.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.607+0000 7f0cf7200640 1 -- start start 2024-09-10T09:56:33.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.607+0000 7f0cf7200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0cf8076720 0x7f0cf81a45d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:33.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.607+0000 7f0cf7200640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0cf8077040 0x7f0cf81a4b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:33.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.607+0000 7f0cf7200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0cf8112870 0x7f0cf819e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:33.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.608+0000 7f0cf7200640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf81183b0 con 0x7f0cf8112870 2024-09-10T09:56:33.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.608+0000 7f0cf7200640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf8118230 con 0x7f0cf8077040 2024-09-10T09:56:33.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.608+0000 7f0cf7200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf8118530 con 0x7f0cf8076720 2024-09-10T09:56:33.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.608+0000 7f0cf5400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0cf8077040 0x7f0cf81a4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:33.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.608+0000 7f0cf5400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0cf8077040 0x7f0cf81a4b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:49534/0 (socket says 172.21.15.77:49534) 2024-09-10T09:56:33.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.608+0000 7f0cf5400640 1 -- 172.21.15.77:0/3621826095 learned_addr learned my addr 172.21.15.77:0/3621826095 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:33.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.608+0000 7f0cf6800640 1 --2- 172.21.15.77:0/3621826095 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0cf8112870 0x7f0cf819e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:33.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.608+0000 7f0cf5e00640 1 --2- 172.21.15.77:0/3621826095 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0cf8076720 0x7f0cf81a45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:33.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.608+0000 7f0cf5400640 1 -- 172.21.15.77:0/3621826095 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0cf8076720 msgr2=0x7f0cf81a45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:33.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.608+0000 7f0cf5400640 1 --2- 172.21.15.77:0/3621826095 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0cf8076720 0x7f0cf81a45d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:33.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.608+0000 7f0cf5400640 1 -- 172.21.15.77:0/3621826095 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0cf8112870 msgr2=0x7f0cf819e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:33.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.608+0000 7f0cf5400640 1 --2- 172.21.15.77:0/3621826095 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0cf8112870 0x7f0cf819e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:33.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.608+0000 7f0cf5400640 1 -- 172.21.15.77:0/3621826095 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cf819ef10 con 0x7f0cf8077040 2024-09-10T09:56:33.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.608+0000 7f0cf6800640 1 --2- 172.21.15.77:0/3621826095 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0cf8112870 0x7f0cf819e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:56:33.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.609+0000 7f0cf5400640 1 --2- 172.21.15.77:0/3621826095 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0cf8077040 0x7f0cf81a4b10 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0ce400e9c0 tx=0x7f0ce400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:33.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.609+0000 7f0ceea00640 1 -- 172.21.15.77:0/3621826095 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce400cde0 con 0x7f0cf8077040 2024-09-10T09:56:33.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.609+0000 7f0cf7200640 1 -- 172.21.15.77:0/3621826095 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cf819f200 con 0x7f0cf8077040 2024-09-10T09:56:33.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.609+0000 7f0ceea00640 1 -- 172.21.15.77:0/3621826095 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ce40035f0 con 0x7f0cf8077040 2024-09-10T09:56:33.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.609+0000 7f0ceea00640 1 -- 172.21.15.77:0/3621826095 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce4010650 con 0x7f0cf8077040 2024-09-10T09:56:33.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.609+0000 7f0cf7200640 1 -- 172.21.15.77:0/3621826095 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cf81ab550 con 0x7f0cf8077040 2024-09-10T09:56:33.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.614+0000 7f0cff425740 1 Processor -- start 2024-09-10T09:56:33.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.614+0000 7f0cff425740 1 -- start start 2024-09-10T09:56:33.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.614+0000 7f0cff425740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562146df2cb0 0x562146df3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:33.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.615+0000 7f0cff425740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562146df35d0 0x562146dfd950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:33.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.615+0000 7f0cff425740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562146dfde90 0x562146e00280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:33.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.615+0000 7f0cff425740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x562146d59290 con 0x562146df35d0 2024-09-10T09:56:33.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.615+0000 7f0cff425740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x562146d00b40 con 0x562146df2cb0 2024-09-10T09:56:33.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.615+0000 7f0cff425740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562146d55250 con 0x562146dfde90 2024-09-10T09:56:33.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.615+0000 7f0cf6800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562146df2cb0 0x562146df3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:33.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.615+0000 7f0cf6800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562146df2cb0 0x562146df3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:49542/0 (socket says 172.21.15.77:49542) 2024-09-10T09:56:33.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.615+0000 7f0cf6800640 1 -- 172.21.15.77:0/1303744756 learned_addr learned my addr 172.21.15.77:0/1303744756 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:33.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.615+0000 7f0cf5e00640 1 --2- 172.21.15.77:0/1303744756 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562146df35d0 0x562146dfd950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:33.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.616+0000 7f0cf5400640 1 --2- 172.21.15.77:0/1303744756 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562146dfde90 0x562146e00280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:33.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.616+0000 7f0cf6800640 1 -- 172.21.15.77:0/1303744756 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562146dfde90 msgr2=0x562146e00280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:33.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.616+0000 7f0cf6800640 1 --2- 172.21.15.77:0/1303744756 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562146dfde90 0x562146e00280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:33.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.616+0000 7f0cf6800640 1 -- 172.21.15.77:0/1303744756 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562146df35d0 msgr2=0x562146dfd950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:33.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.616+0000 7f0cf6800640 1 --2- 172.21.15.77:0/1303744756 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562146df35d0 0x562146dfd950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:33.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.616+0000 7f0cf6800640 1 -- 172.21.15.77:0/1303744756 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562146aca6a0 con 0x562146df2cb0 2024-09-10T09:56:33.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.616+0000 7f0cf5e00640 1 --2- 172.21.15.77:0/1303744756 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562146df35d0 0x562146dfd950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:56:33.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.617+0000 7f0cf6800640 1 --2- 172.21.15.77:0/1303744756 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562146df2cb0 0x562146df3090 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0cf0006010 tx=0x7f0cf0030b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=187f265b256243bb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:33.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.616+0000 7f0ceea00640 1 -- 172.21.15.77:0/3621826095 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0ce40037a0 con 0x7f0cf8077040 2024-09-10T09:56:33.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.617+0000 7f0ce3000640 1 -- 172.21.15.77:0/1303744756 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf0040030 con 0x562146df2cb0 2024-09-10T09:56:33.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.617+0000 7f0ce3000640 1 -- 172.21.15.77:0/1303744756 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cf0030d20 con 0x562146df2cb0 2024-09-10T09:56:33.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.617+0000 7f0ce3000640 1 -- 172.21.15.77:0/1303744756 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf0036040 con 0x562146df2cb0 2024-09-10T09:56:33.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.618+0000 7f0cff425740 1 -- 172.21.15.77:0/1303744756 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562146df2cb0 msgr2=0x562146df3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:33.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.618+0000 7f0cff425740 1 --2- 172.21.15.77:0/1303744756 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562146df2cb0 0x562146df3090 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0cf0006010 tx=0x7f0cf0030b30 comp rx=0 tx=0).stop 2024-09-10T09:56:33.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.618+0000 7f0cff425740 1 -- 172.21.15.77:0/1303744756 shutdown_connections 2024-09-10T09:56:33.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.618+0000 7f0cff425740 1 --2- 172.21.15.77:0/1303744756 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562146dfde90 0x562146e00280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:33.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.618+0000 7f0cff425740 1 --2- 172.21.15.77:0/1303744756 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562146df35d0 0x562146dfd950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:33.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.618+0000 7f0cff425740 1 --2- 172.21.15.77:0/1303744756 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562146df2cb0 0x562146df3090 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:33.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.618+0000 7f0cff425740 1 -- 172.21.15.77:0/1303744756 >> 172.21.15.77:0/1303744756 conn(0x562146def240 msgr2=0x562146def620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:33.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.619+0000 7f0ceea00640 1 --2- 172.21.15.77:0/3621826095 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f0cdc0a2c80 0x7f0cdc0a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:33.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.619+0000 7f0ceea00640 1 -- 172.21.15.77:0/3621826095 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f0ce40cf830 con 0x7f0cf8077040 2024-09-10T09:56:33.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.619+0000 7f0cf5e00640 1 --2- 172.21.15.77:0/3621826095 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f0cdc0a2c80 0x7f0cdc0a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:33.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.620+0000 7f0cff425740 1 -- 172.21.15.77:0/1303744756 shutdown_connections 2024-09-10T09:56:33.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.620+0000 7f0cff425740 1 -- 172.21.15.77:0/1303744756 wait complete. 2024-09-10T09:56:33.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.620+0000 7f0cf5e00640 1 --2- 172.21.15.77:0/3621826095 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f0cdc0a2c80 0x7f0cdc0a5140 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f0ce80095f0 tx=0x7f0ce8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:33.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.620+0000 7f0cff425740 1 -- start start 2024-09-10T09:56:33.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.621+0000 7f0cff425740 1 Processor -- start 2024-09-10T09:56:33.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.621+0000 7f0cff425740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562146df2cb0 0x562146dfd730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:33.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.621+0000 7f0cff425740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562146df35d0 0x562146dfa9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:33.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.621+0000 7f0cff425740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562146dfde90 0x562146dfaf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:33.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.622+0000 7f0cff425740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x562146e00b00 con 0x562146df2cb0 2024-09-10T09:56:33.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.622+0000 7f0cff425740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x562146d13fa0 con 0x562146dfde90 2024-09-10T09:56:33.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.622+0000 7f0cff425740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562146e00c80 con 0x562146df35d0 2024-09-10T09:56:33.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.622+0000 7f0cff425740 10 client.0 _ll_register_callbacks cb 0x562146a67ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:56:33.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.622+0000 7f0cf5e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562146df2cb0 0x562146dfd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:33.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.622+0000 7f0cf5e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562146df2cb0 0x562146dfd730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:48402/0 (socket says 172.21.15.77:48402) 2024-09-10T09:56:33.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.622+0000 7f0cf5e00640 1 -- 172.21.15.77:0/1946166371 learned_addr learned my addr 172.21.15.77:0/1946166371 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:33.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.622+0000 7f0cf5400640 1 --2- 172.21.15.77:0/1946166371 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562146df35d0 0x562146dfa9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:33.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.622+0000 7f0cf6800640 1 --2- 172.21.15.77:0/1946166371 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562146dfde90 0x562146dfaf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:33.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.623+0000 7f0cf5e00640 1 -- 172.21.15.77:0/1946166371 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562146df35d0 msgr2=0x562146dfa9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:33.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.623+0000 7f0cf5e00640 1 --2- 172.21.15.77:0/1946166371 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562146df35d0 0x562146dfa9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:33.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.623+0000 7f0cf5e00640 1 -- 172.21.15.77:0/1946166371 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562146dfde90 msgr2=0x562146dfaf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:33.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.623+0000 7f0cf5e00640 1 --2- 172.21.15.77:0/1946166371 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562146dfde90 0x562146dfaf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:33.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.623+0000 7f0cf5e00640 1 -- 172.21.15.77:0/1946166371 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562146dfb720 con 0x562146df2cb0 2024-09-10T09:56:33.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.623+0000 7f0cf6800640 1 --2- 172.21.15.77:0/1946166371 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562146dfde90 0x562146dfaf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:56:33.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.623+0000 7f0cf5400640 1 --2- 172.21.15.77:0/1946166371 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562146df35d0 0x562146dfa9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:56:33.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.623+0000 7f0cf5e00640 1 --2- 172.21.15.77:0/1946166371 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562146df2cb0 0x562146dfd730 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f0ce801d6b0 tx=0x7f0ce80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f663619913af5018 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:33.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.623+0000 7f0cd7400640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-10T09:56:33.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.623+0000 7f0cd7400640 1 -- 172.21.15.77:0/1946166371 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce802a060 con 0x562146df2cb0 2024-09-10T09:56:33.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.624+0000 7f0cff425740 1 -- 172.21.15.77:0/1946166371 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0cf0030d20 con 0x562146df2cb0 2024-09-10T09:56:33.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.624+0000 7f0cd7400640 1 -- 172.21.15.77:0/1946166371 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f0ce80107d0 con 0x562146df2cb0 2024-09-10T09:56:33.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.624+0000 7f0cd7400640 1 -- 172.21.15.77:0/1946166371 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0ce80109d0 con 0x562146df2cb0 2024-09-10T09:56:33.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.624+0000 7f0cff425740 10 client.15675 fetch_fsmap learned FSMap version 29 2024-09-10T09:56:33.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.624+0000 7f0cff425740 1 -- 172.21.15.77:0/1946166371 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562146e0ca60 con 0x562146df2cb0 2024-09-10T09:56:33.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.625+0000 7f0cd7400640 1 -- 172.21.15.77:0/1946166371 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cf0030d20 con 0x562146df2cb0 2024-09-10T09:56:33.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.625+0000 7f0cd7400640 1 -- 172.21.15.77:0/1946166371 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f0ce8019040 con 0x562146df2cb0 2024-09-10T09:56:33.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.625+0000 7f0cff425740 10 client.15675 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T09:56:33.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.625+0000 7f0cff425740 10 client.15675 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:56:33.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.625+0000 7f0cff425740 20 client.15675 populate_metadata read hostname 'smithi077' 2024-09-10T09:56:33.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.625+0000 7f0cff425740 1 --2- 172.21.15.77:0/1946166371 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x562146e0de70 0x562146e102e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:33.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.625+0000 7f0cff425740 4 client.15675 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:56:33.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.625+0000 7f0cff425740 1 -- 172.21.15.77:0/1946166371 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562146d66800 con 0x562146e0de70 2024-09-10T09:56:33.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.626+0000 7f0cf5400640 1 --2- 172.21.15.77:0/1946166371 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x562146e0de70 0x562146e102e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:33.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.627+0000 7f0cf5400640 1 --2- 172.21.15.77:0/1946166371 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x562146e0de70 0x562146e102e0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4027240292bef8a0 server_cookie=35475e72ebf8ed7 in_seq=0 out_seq=0 2024-09-10T09:56:33.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.627+0000 7f0cd7400640 10 client.15675 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T09:56:33.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.628+0000 7f0cd7400640 1 -- 172.21.15.77:0/1946166371 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562146d66800 con 0x562146e0de70 2024-09-10T09:56:33.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.628+0000 7f0cd7400640 10 client.15675 handle_command_reply: tid=0 2024-09-10T09:56:33.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.628+0000 7f0cff425740 1 client.15675 shutdown 2024-09-10T09:56:33.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.628+0000 7f0cff425740 1 -- 172.21.15.77:0/1946166371 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562146df2cb0 msgr2=0x562146dfd730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:33.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.628+0000 7f0cff425740 1 --2- 172.21.15.77:0/1946166371 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562146df2cb0 0x562146dfd730 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f0ce801d6b0 tx=0x7f0ce80046c0 comp rx=0 tx=0).stop 2024-09-10T09:56:33.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.628+0000 7f0cff425740 1 -- 172.21.15.77:0/1946166371 shutdown_connections 2024-09-10T09:56:33.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.628+0000 7f0cff425740 1 --2- 172.21.15.77:0/1946166371 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x562146e0de70 0x562146e102e0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:33.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.629+0000 7f0cff425740 1 --2- 172.21.15.77:0/1946166371 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562146dfde90 0x562146dfaf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:33.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.629+0000 7f0cd7400640 0 client.15675 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T09:56:33.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.629+0000 7f0cff425740 1 --2- 172.21.15.77:0/1946166371 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562146df35d0 0x562146dfa9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:33.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.629+0000 7f0cff425740 1 --2- 172.21.15.77:0/1946166371 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562146df2cb0 0x562146dfd730 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:33.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.629+0000 7f0cff425740 1 -- 172.21.15.77:0/1946166371 >> 172.21.15.77:0/1946166371 conn(0x562146def240 msgr2=0x562146df4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:33.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.629+0000 7f0cff425740 1 -- 172.21.15.77:0/1946166371 shutdown_connections 2024-09-10T09:56:33.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.629+0000 7f0cff425740 1 -- 172.21.15.77:0/1946166371 wait complete. 2024-09-10T09:56:33.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.630+0000 7f0cff425740 20 client.15675 trim_cache size 0 max 16384 2024-09-10T09:56:33.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.664+0000 7f0cff425740 1 Processor -- start 2024-09-10T09:56:33.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.664+0000 7f0cff425740 1 -- start start 2024-09-10T09:56:33.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.665+0000 7f0cff425740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562146df2cb0 0x562146e15bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:33.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.665+0000 7f0cff425740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562146df35d0 0x562146e160f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:33.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.665+0000 7f0cff425740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562146dfde90 0x562146df7c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:33.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.665+0000 7f0cff425740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf81b5370 con 0x562146df35d0 2024-09-10T09:56:33.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.665+0000 7f0cff425740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf81b6eb0 con 0x562146df2cb0 2024-09-10T09:56:33.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.665+0000 7f0cff425740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf81b5a40 con 0x562146dfde90 2024-09-10T09:56:33.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.665+0000 7f0cf5e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562146df2cb0 0x562146e15bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:33.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.665+0000 7f0cf5e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562146df2cb0 0x562146e15bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:49566/0 (socket says 172.21.15.77:49566) 2024-09-10T09:56:33.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.665+0000 7f0cf5e00640 1 -- 172.21.15.77:0/628710511 learned_addr learned my addr 172.21.15.77:0/628710511 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:33.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.666+0000 7f0cf6800640 1 --2- 172.21.15.77:0/628710511 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562146dfde90 0x562146df7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:33.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.666+0000 7f0cf5400640 1 --2- 172.21.15.77:0/628710511 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562146df35d0 0x562146e160f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:33.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.666+0000 7f0cf5e00640 1 -- 172.21.15.77:0/628710511 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562146dfde90 msgr2=0x562146df7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:33.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.666+0000 7f0cf5e00640 1 --2- 172.21.15.77:0/628710511 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562146dfde90 0x562146df7c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:33.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.666+0000 7f0cf5e00640 1 -- 172.21.15.77:0/628710511 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562146df35d0 msgr2=0x562146e160f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:33.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.666+0000 7f0cf5e00640 1 --2- 172.21.15.77:0/628710511 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562146df35d0 0x562146e160f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:33.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.666+0000 7f0cf5e00640 1 -- 172.21.15.77:0/628710511 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562146e005c0 con 0x562146df2cb0 2024-09-10T09:56:33.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.666+0000 7f0cf5400640 1 --2- 172.21.15.77:0/628710511 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562146df35d0 0x562146e160f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:56:33.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.666+0000 7f0cf5e00640 1 --2- 172.21.15.77:0/628710511 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562146df2cb0 0x562146e15bb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0ce80253e0 tx=0x7f0ce8010960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee15945bdd2bbcf2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:33.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.667+0000 7f0ce2600640 1 -- 172.21.15.77:0/628710511 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce8010d70 con 0x562146df2cb0 2024-09-10T09:56:33.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.667+0000 7f0ce2600640 1 -- 172.21.15.77:0/628710511 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ce8025c40 con 0x562146df2cb0 2024-09-10T09:56:33.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.667+0000 7f0ce2600640 1 -- 172.21.15.77:0/628710511 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce801bbe0 con 0x562146df2cb0 2024-09-10T09:56:33.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.668+0000 7f0cff425740 1 -- 172.21.15.77:0/628710511 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562146df2cb0 msgr2=0x562146e15bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:33.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.668+0000 7f0cff425740 1 --2- 172.21.15.77:0/628710511 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562146df2cb0 0x562146e15bb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0ce80253e0 tx=0x7f0ce8010960 comp rx=0 tx=0).stop 2024-09-10T09:56:33.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.668+0000 7f0cff425740 1 -- 172.21.15.77:0/628710511 shutdown_connections 2024-09-10T09:56:33.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.668+0000 7f0cff425740 1 --2- 172.21.15.77:0/628710511 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562146dfde90 0x562146df7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:33.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.668+0000 7f0cff425740 1 --2- 172.21.15.77:0/628710511 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562146df35d0 0x562146e160f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:33.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.668+0000 7f0cff425740 1 --2- 172.21.15.77:0/628710511 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562146df2cb0 0x562146e15bb0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:33.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.668+0000 7f0cff425740 1 -- 172.21.15.77:0/628710511 >> 172.21.15.77:0/628710511 conn(0x562146def240 msgr2=0x562146dee290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:33.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.668+0000 7f0cff425740 1 -- 172.21.15.77:0/628710511 shutdown_connections 2024-09-10T09:56:33.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.668+0000 7f0cff425740 1 -- 172.21.15.77:0/628710511 wait complete. 2024-09-10T09:56:33.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.669+0000 7f0cff425740 1 -- start start 2024-09-10T09:56:33.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.669+0000 7f0cff425740 1 Processor -- start 2024-09-10T09:56:33.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.670+0000 7f0cff425740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562146df2cb0 0x562146e03c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:33.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.670+0000 7f0cff425740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562146df35d0 0x562146e041c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:33.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.670+0000 7f0cff425740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562146dfde90 0x562146e04700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:33.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.670+0000 7f0cff425740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf81b67e0 con 0x562146df2cb0 2024-09-10T09:56:33.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.670+0000 7f0cff425740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf81b6110 con 0x562146dfde90 2024-09-10T09:56:33.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.670+0000 7f0cff425740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf81b3350 con 0x562146df35d0 2024-09-10T09:56:33.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.670+0000 7f0cff425740 10 client.0 _ll_register_callbacks cb 0x7f0ce8012ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:56:33.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.670+0000 7f0cf5e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562146df2cb0 0x562146e03c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:33.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.670+0000 7f0cf5400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562146df35d0 0x562146e041c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:33.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.670+0000 7f0cf5400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562146df35d0 0x562146e041c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:37050/0 (socket says 172.21.15.77:37050) 2024-09-10T09:56:33.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.670+0000 7f0cf5400640 1 -- 172.21.15.77:0/1450637576 learned_addr learned my addr 172.21.15.77:0/1450637576 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:33.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.671+0000 7f0cf6800640 1 --2- 172.21.15.77:0/1450637576 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562146dfde90 0x562146e04700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:33.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.671+0000 7f0cf5400640 1 -- 172.21.15.77:0/1450637576 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562146dfde90 msgr2=0x562146e04700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:33.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.671+0000 7f0cf5400640 1 --2- 172.21.15.77:0/1450637576 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562146dfde90 0x562146e04700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:33.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.671+0000 7f0cf5400640 1 -- 172.21.15.77:0/1450637576 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562146df2cb0 msgr2=0x562146e03c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:33.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.671+0000 7f0cf5400640 1 --2- 172.21.15.77:0/1450637576 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562146df2cb0 0x562146e03c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:33.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.671+0000 7f0cf5400640 1 -- 172.21.15.77:0/1450637576 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562146e77b90 con 0x562146df35d0 2024-09-10T09:56:33.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.671+0000 7f0cf5400640 1 --2- 172.21.15.77:0/1450637576 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562146df35d0 0x562146e041c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0ce40980a0 tx=0x7f0ce409bd00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1766f7e0f49c23b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:33.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.671+0000 7f0cd7400640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-10T09:56:33.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.672+0000 7f0cd7400640 1 -- 172.21.15.77:0/1450637576 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce40ccff0 con 0x562146df35d0 2024-09-10T09:56:33.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.672+0000 7f0cd7400640 1 -- 172.21.15.77:0/1450637576 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f0ce4003b10 con 0x562146df35d0 2024-09-10T09:56:33.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.672+0000 7f0cff425740 1 -- 172.21.15.77:0/1450637576 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ce8025c40 con 0x562146df35d0 2024-09-10T09:56:33.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.672+0000 7f0cd7400640 1 -- 172.21.15.77:0/1450637576 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0ce40cf420 con 0x562146df35d0 2024-09-10T09:56:33.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.672+0000 7f0cff425740 10 client.25702 fetch_fsmap learned FSMap version 29 2024-09-10T09:56:33.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.672+0000 7f0cff425740 1 -- 172.21.15.77:0/1450637576 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562146e78f10 con 0x562146df35d0 2024-09-10T09:56:33.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.672+0000 7f0cd7400640 1 -- 172.21.15.77:0/1450637576 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ce8025c40 con 0x562146df35d0 2024-09-10T09:56:33.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.672+0000 7f0cd7400640 1 -- 172.21.15.77:0/1450637576 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f0ce409b090 con 0x562146df35d0 2024-09-10T09:56:33.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.673+0000 7f0cff425740 10 client.25702 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T09:56:33.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.673+0000 7f0cff425740 10 client.25702 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:56:33.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.673+0000 7f0cff425740 20 client.25702 populate_metadata read hostname 'smithi077' 2024-09-10T09:56:33.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.673+0000 7f0cff425740 1 --2- 172.21.15.77:0/1450637576 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x562146e0de70 0x562146e7b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:33.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.673+0000 7f0cff425740 4 client.25702 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T09:56:33.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.673+0000 7f0cff425740 1 -- 172.21.15.77:0/1450637576 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562146dfb2b0 con 0x562146e0de70 2024-09-10T09:56:33.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.673+0000 7f0cf5e00640 1 --2- 172.21.15.77:0/1450637576 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x562146e0de70 0x562146e7b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:33.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.674+0000 7f0cf5e00640 1 --2- 172.21.15.77:0/1450637576 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x562146e0de70 0x562146e7b6b0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9a16b8fc2b6933a server_cookie=6cd01ae571eceeb3 in_seq=0 out_seq=0 2024-09-10T09:56:33.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.674+0000 7f0cd7400640 10 client.25702 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T09:56:33.677 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.675+0000 7f0cd7400640 1 -- 172.21.15.77:0/1450637576 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562146dfb2b0 con 0x562146e0de70 2024-09-10T09:56:33.677 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.675+0000 7f0cd7400640 10 client.25702 handle_command_reply: tid=0 2024-09-10T09:56:33.677 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.675+0000 7f0cff425740 1 client.25702 shutdown 2024-09-10T09:56:33.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.676+0000 7f0cff425740 1 -- 172.21.15.77:0/1450637576 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562146df35d0 msgr2=0x562146e041c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:33.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.676+0000 7f0cff425740 1 --2- 172.21.15.77:0/1450637576 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562146df35d0 0x562146e041c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0ce40980a0 tx=0x7f0ce409bd00 comp rx=0 tx=0).stop 2024-09-10T09:56:33.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.676+0000 7f0cff425740 1 -- 172.21.15.77:0/1450637576 shutdown_connections 2024-09-10T09:56:33.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.676+0000 7f0cff425740 1 --2- 172.21.15.77:0/1450637576 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x562146e0de70 0x562146e7b6b0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:33.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.676+0000 7f0cd7400640 0 client.25702 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T09:56:33.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.676+0000 7f0cff425740 1 --2- 172.21.15.77:0/1450637576 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562146dfde90 0x562146e04700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:33.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.676+0000 7f0cff425740 1 --2- 172.21.15.77:0/1450637576 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562146df35d0 0x562146e041c0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:33.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.676+0000 7f0cff425740 1 --2- 172.21.15.77:0/1450637576 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562146df2cb0 0x562146e03c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:33.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.676+0000 7f0cff425740 1 -- 172.21.15.77:0/1450637576 >> 172.21.15.77:0/1450637576 conn(0x562146def240 msgr2=0x562146e08d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:33.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.677+0000 7f0cff425740 1 -- 172.21.15.77:0/1450637576 shutdown_connections 2024-09-10T09:56:33.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.677+0000 7f0cff425740 1 -- 172.21.15.77:0/1450637576 wait complete. 2024-09-10T09:56:33.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.677+0000 7f0cff425740 20 client.25702 trim_cache size 0 max 16384 2024-09-10T09:56:33.680 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-10T09:56:33.680 INFO:teuthology.orchestra.run.smithi077.stdout: "return_code": 0, 2024-09-10T09:56:33.680 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_tag": "5c0ab740-8cff-42b9-bf77-b1f7e284f3f3", 2024-09-10T09:56:33.680 INFO:teuthology.orchestra.run.smithi077.stdout: "mode": "asynchronous" 2024-09-10T09:56:33.680 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-10T09:56:33.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.678+0000 7f0cf7200640 1 -- 172.21.15.77:0/3621826095 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f0cdc0a2c80 msgr2=0x7f0cdc0a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:33.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.678+0000 7f0cf7200640 1 --2- 172.21.15.77:0/3621826095 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f0cdc0a2c80 0x7f0cdc0a5140 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f0ce80095f0 tx=0x7f0ce8005f90 comp rx=0 tx=0).stop 2024-09-10T09:56:33.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.678+0000 7f0cf7200640 1 -- 172.21.15.77:0/3621826095 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0cf8077040 msgr2=0x7f0cf81a4b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:33.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.678+0000 7f0cf7200640 1 --2- 172.21.15.77:0/3621826095 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0cf8077040 0x7f0cf81a4b10 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0ce400e9c0 tx=0x7f0ce400ee80 comp rx=0 tx=0).stop 2024-09-10T09:56:33.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.678+0000 7f0cf7200640 1 -- 172.21.15.77:0/3621826095 shutdown_connections 2024-09-10T09:56:33.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.678+0000 7f0cf7200640 1 --2- 172.21.15.77:0/3621826095 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f0cdc0a2c80 0x7f0cdc0a5140 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:33.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.678+0000 7f0cf7200640 1 --2- 172.21.15.77:0/3621826095 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0cf8112870 0x7f0cf819e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:33.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.678+0000 7f0cf7200640 1 --2- 172.21.15.77:0/3621826095 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0cf8077040 0x7f0cf81a4b10 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:33.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.678+0000 7f0cf7200640 1 --2- 172.21.15.77:0/3621826095 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0cf8076720 0x7f0cf81a45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:33.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.678+0000 7f0cf7200640 1 -- 172.21.15.77:0/3621826095 >> 172.21.15.77:0/3621826095 conn(0x7f0cf80ff910 msgr2=0x7f0cf8101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:33.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.679+0000 7f0cf7200640 1 -- 172.21.15.77:0/3621826095 shutdown_connections 2024-09-10T09:56:33.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:33.679+0000 7f0cf7200640 1 -- 172.21.15.77:0/3621826095 wait complete. 2024-09-10T09:56:34.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:56:33 smithi120 ceph-mon[34770]: scrub queued for path: / 2024-09-10T09:56:34.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:56:33 smithi120 ceph-mon[34770]: scrub summary: idle+waiting paths [/] 2024-09-10T09:56:34.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:56:33 smithi120 ceph-mon[34770]: scrub summary: active paths [/] 2024-09-10T09:56:34.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:56:33 smithi120 ceph-mon[34770]: scrub summary: idle+waiting paths [/] 2024-09-10T09:56:34.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:56:33 smithi077 ceph-mon[30319]: scrub queued for path: / 2024-09-10T09:56:34.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:56:33 smithi077 ceph-mon[30319]: scrub summary: idle+waiting paths [/] 2024-09-10T09:56:34.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:56:33 smithi077 ceph-mon[30319]: scrub summary: active paths [/] 2024-09-10T09:56:34.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:56:33 smithi077 ceph-mon[30319]: scrub summary: idle+waiting paths [/] 2024-09-10T09:56:34.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:56:33 smithi134 ceph-mon[35674]: scrub queued for path: / 2024-09-10T09:56:34.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:56:33 smithi134 ceph-mon[35674]: scrub summary: idle+waiting paths [/] 2024-09-10T09:56:34.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:56:33 smithi134 ceph-mon[35674]: scrub summary: active paths [/] 2024-09-10T09:56:34.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:56:33 smithi134 ceph-mon[35674]: scrub summary: idle+waiting paths [/] 2024-09-10T09:56:34.756 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T09:56:35.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:56:34 smithi120 ceph-mon[34770]: pgmap v433: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-10T09:56:35.186 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:56:35.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:56:34 smithi077 ceph-mon[30319]: pgmap v433: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-10T09:56:35.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:56:34 smithi134 ceph-mon[35674]: pgmap v433: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-10T09:56:35.394 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:56:35.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.779+0000 7f7f54e00640 1 -- 172.21.15.77:0/4196110877 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f50074e30 msgr2=0x7f7f5010dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:35.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.779+0000 7f7f54e00640 1 --2- 172.21.15.77:0/4196110877 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f50074e30 0x7f7f5010dd40 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c009900 tx=0x7f7f3c02a480 comp rx=0 tx=0).stop 2024-09-10T09:56:35.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.779+0000 7f7f54e00640 1 -- 172.21.15.77:0/4196110877 shutdown_connections 2024-09-10T09:56:35.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.779+0000 7f7f54e00640 1 --2- 172.21.15.77:0/4196110877 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f5010e280 0x7f7f50110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:35.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.779+0000 7f7f54e00640 1 --2- 172.21.15.77:0/4196110877 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f50074e30 0x7f7f5010dd40 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:35.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.779+0000 7f7f54e00640 1 --2- 172.21.15.77:0/4196110877 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7f50074510 0x7f7f500748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:35.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.779+0000 7f7f54e00640 1 -- 172.21.15.77:0/4196110877 >> 172.21.15.77:0/4196110877 conn(0x7f7f500fd740 msgr2=0x7f7f500ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:35.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.780+0000 7f7f54e00640 1 -- 172.21.15.77:0/4196110877 shutdown_connections 2024-09-10T09:56:35.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.780+0000 7f7f54e00640 1 -- 172.21.15.77:0/4196110877 wait complete. 2024-09-10T09:56:35.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.781+0000 7f7f54e00640 1 Processor -- start 2024-09-10T09:56:35.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.781+0000 7f7f54e00640 1 -- start start 2024-09-10T09:56:35.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.782+0000 7f7f54e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7f50074510 0x7f7f5010dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:35.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.782+0000 7f7f54e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f50074e30 0x7f7f5010acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:35.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.782+0000 7f7f54e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f5010e280 0x7f7f5010b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:35.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.782+0000 7f7f54e00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7f50114000 con 0x7f7f5010e280 2024-09-10T09:56:35.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.782+0000 7f7f54e00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7f50113e80 con 0x7f7f50074510 2024-09-10T09:56:35.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.782+0000 7f7f54e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7f50114180 con 0x7f7f50074e30 2024-09-10T09:56:35.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.782+0000 7f7f4fe00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f5010e280 0x7f7f5010b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:35.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.782+0000 7f7f4fe00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f5010e280 0x7f7f5010b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:48430/0 (socket says 172.21.15.77:48430) 2024-09-10T09:56:35.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.782+0000 7f7f4fe00640 1 -- 172.21.15.77:0/1964630736 learned_addr learned my addr 172.21.15.77:0/1964630736 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:35.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.782+0000 7f7f4f400640 1 --2- 172.21.15.77:0/1964630736 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7f50074510 0x7f7f5010dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:35.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.782+0000 7f7f4ea00640 1 --2- 172.21.15.77:0/1964630736 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f50074e30 0x7f7f5010acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:35.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.783+0000 7f7f4f400640 1 -- 172.21.15.77:0/1964630736 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f50074e30 msgr2=0x7f7f5010acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:35.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.783+0000 7f7f4f400640 1 --2- 172.21.15.77:0/1964630736 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f50074e30 0x7f7f5010acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:35.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.783+0000 7f7f4f400640 1 -- 172.21.15.77:0/1964630736 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f5010e280 msgr2=0x7f7f5010b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:35.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.783+0000 7f7f4f400640 1 --2- 172.21.15.77:0/1964630736 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f5010e280 0x7f7f5010b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:35.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.783+0000 7f7f4f400640 1 -- 172.21.15.77:0/1964630736 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f5010bb10 con 0x7f7f50074510 2024-09-10T09:56:35.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.783+0000 7f7f4fe00640 1 --2- 172.21.15.77:0/1964630736 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f5010e280 0x7f7f5010b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:56:35.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.783+0000 7f7f4f400640 1 --2- 172.21.15.77:0/1964630736 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7f50074510 0x7f7f5010dc20 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7f4000e9f0 tx=0x7f7f4000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:35.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.783+0000 7f7f47e00640 1 -- 172.21.15.77:0/1964630736 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f4000ccc0 con 0x7f7f50074510 2024-09-10T09:56:35.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.783+0000 7f7f47e00640 1 -- 172.21.15.77:0/1964630736 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f40010910 con 0x7f7f50074510 2024-09-10T09:56:35.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.783+0000 7f7f54e00640 1 -- 172.21.15.77:0/1964630736 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f501aae40 con 0x7f7f50074510 2024-09-10T09:56:35.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.784+0000 7f7f54e00640 1 -- 172.21.15.77:0/1964630736 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f501ab300 con 0x7f7f50074510 2024-09-10T09:56:35.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.784+0000 7f7f47e00640 1 -- 172.21.15.77:0/1964630736 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f40018960 con 0x7f7f50074510 2024-09-10T09:56:35.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.787+0000 7f7f47e00640 1 -- 172.21.15.77:0/1964630736 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7f40018b00 con 0x7f7f50074510 2024-09-10T09:56:35.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.788+0000 7f7f59016740 1 Processor -- start 2024-09-10T09:56:35.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.788+0000 7f7f59016740 1 -- start start 2024-09-10T09:56:35.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.789+0000 7f7f59016740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc4ec30d50 0x55fc4ec31130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:35.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.789+0000 7f7f59016740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fc4ec31670 0x55fc4ec3b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:35.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.789+0000 7f7f59016740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fc4ec3bf30 0x55fc4ec3e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:35.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.789+0000 7f7f59016740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55fc4eb973e0 con 0x55fc4ec3bf30 2024-09-10T09:56:35.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.789+0000 7f7f59016740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55fc4eb3ed80 con 0x55fc4ec30d50 2024-09-10T09:56:35.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.789+0000 7f7f59016740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55fc4eb933a0 con 0x55fc4ec31670 2024-09-10T09:56:35.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.789+0000 7f7f47e00640 1 --2- 172.21.15.77:0/1964630736 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f7f340a2c80 0x7f7f340a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:35.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.789+0000 7f7f47e00640 1 -- 172.21.15.77:0/1964630736 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f7f400cf750 con 0x7f7f50074510 2024-09-10T09:56:35.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.789+0000 7f7f4fe00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc4ec30d50 0x55fc4ec31130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:35.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.789+0000 7f7f4fe00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc4ec30d50 0x55fc4ec31130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:49606/0 (socket says 172.21.15.77:49606) 2024-09-10T09:56:35.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.789+0000 7f7f4fe00640 1 -- 172.21.15.77:0/3881533760 learned_addr learned my addr 172.21.15.77:0/3881533760 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:35.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.790+0000 7f7f4ea00640 1 --2- 172.21.15.77:0/3881533760 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fc4ec3bf30 0x55fc4ec3e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:35.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.790+0000 7f7f4fe00640 1 -- 172.21.15.77:0/3881533760 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fc4ec31670 msgr2=0x55fc4ec3b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:35.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.790+0000 7f7f4f400640 1 --2- 172.21.15.77:0/3881533760 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fc4ec31670 0x55fc4ec3b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:35.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.790+0000 7f7f4fe00640 1 --2- 172.21.15.77:0/3881533760 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fc4ec31670 0x55fc4ec3b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:35.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.790+0000 7f7f4fe00640 1 -- 172.21.15.77:0/3881533760 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fc4ec3bf30 msgr2=0x55fc4ec3e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:35.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.790+0000 7f7f4fe00640 1 --2- 172.21.15.77:0/3881533760 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fc4ec3bf30 0x55fc4ec3e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:35.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.790+0000 7f7f4fe00640 1 -- 172.21.15.77:0/3881533760 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc4eb5dae0 con 0x55fc4ec30d50 2024-09-10T09:56:35.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.790+0000 7f7f4ea00640 1 --2- 172.21.15.77:0/3881533760 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fc4ec3bf30 0x55fc4ec3e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:56:35.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.790+0000 7f7f4f400640 1 --2- 172.21.15.77:0/3881533760 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fc4ec31670 0x55fc4ec3b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:56:35.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.790+0000 7f7f4fe00640 1 --2- 172.21.15.77:0/1964630736 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f7f340a2c80 0x7f7f340a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:35.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.790+0000 7f7f4fe00640 1 --2- 172.21.15.77:0/3881533760 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc4ec30d50 0x55fc4ec31130 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7f48011940 tx=0x7f7f48011e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54b4794f56401cf7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:35.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.791+0000 7f7f44c00640 1 -- 172.21.15.77:0/3881533760 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f4801c440 con 0x55fc4ec30d50 2024-09-10T09:56:35.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.791+0000 7f7f4fe00640 1 --2- 172.21.15.77:0/1964630736 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f7f340a2c80 0x7f7f340a5140 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7f48021000 tx=0x7f7f4801e000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:35.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.791+0000 7f7f44c00640 1 -- 172.21.15.77:0/3881533760 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f4801cac0 con 0x55fc4ec30d50 2024-09-10T09:56:35.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.791+0000 7f7f44c00640 1 -- 172.21.15.77:0/3881533760 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f4800e6a0 con 0x55fc4ec30d50 2024-09-10T09:56:35.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.792+0000 7f7f59016740 1 -- 172.21.15.77:0/3881533760 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc4ec30d50 msgr2=0x55fc4ec31130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:35.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.792+0000 7f7f59016740 1 --2- 172.21.15.77:0/3881533760 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc4ec30d50 0x55fc4ec31130 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7f48011940 tx=0x7f7f48011e00 comp rx=0 tx=0).stop 2024-09-10T09:56:35.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.792+0000 7f7f59016740 1 -- 172.21.15.77:0/3881533760 shutdown_connections 2024-09-10T09:56:35.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.792+0000 7f7f59016740 1 --2- 172.21.15.77:0/3881533760 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fc4ec3bf30 0x55fc4ec3e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:35.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.792+0000 7f7f59016740 1 --2- 172.21.15.77:0/3881533760 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fc4ec31670 0x55fc4ec3b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:35.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.792+0000 7f7f59016740 1 --2- 172.21.15.77:0/3881533760 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc4ec30d50 0x55fc4ec31130 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:35.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.792+0000 7f7f59016740 1 -- 172.21.15.77:0/3881533760 >> 172.21.15.77:0/3881533760 conn(0x55fc4ec2d2e0 msgr2=0x55fc4ec2d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:35.804 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.792+0000 7f7f59016740 1 -- 172.21.15.77:0/3881533760 shutdown_connections 2024-09-10T09:56:35.804 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.793+0000 7f7f59016740 1 -- 172.21.15.77:0/3881533760 wait complete. 2024-09-10T09:56:35.804 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.793+0000 7f7f59016740 1 -- start start 2024-09-10T09:56:35.804 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.793+0000 7f7f59016740 1 Processor -- start 2024-09-10T09:56:35.804 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.794+0000 7f7f59016740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc4ec30d50 0x55fc4ec38d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:35.804 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.794+0000 7f7f59016740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fc4ec31670 0x55fc4ec392d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:35.804 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.794+0000 7f7f59016740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fc4ec3bf30 0x55fc4ec39810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:35.804 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.794+0000 7f7f59016740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55fc4ec3eba0 con 0x55fc4ec3bf30 2024-09-10T09:56:35.804 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.794+0000 7f7f59016740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55fc4eb520f0 con 0x55fc4ec30d50 2024-09-10T09:56:35.804 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.794+0000 7f7f59016740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55fc4ec3ed20 con 0x55fc4ec31670 2024-09-10T09:56:35.804 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.794+0000 7f7f59016740 10 client.0 _ll_register_callbacks cb 0x55fc4eb439e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:56:35.804 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.794+0000 7f7f4f400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc4ec30d50 0x55fc4ec38d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:35.804 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.794+0000 7f7f4f400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc4ec30d50 0x55fc4ec38d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:49612/0 (socket says 172.21.15.77:49612) 2024-09-10T09:56:35.804 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.794+0000 7f7f4f400640 1 -- 172.21.15.77:0/3476658 learned_addr learned my addr 172.21.15.77:0/3476658 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:35.805 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.794+0000 7f7f4ea00640 1 --2- 172.21.15.77:0/3476658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fc4ec31670 0x55fc4ec392d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:35.805 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.795+0000 7f7f4f400640 1 -- 172.21.15.77:0/3476658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fc4ec31670 msgr2=0x55fc4ec392d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:35.805 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.795+0000 7f7f4f400640 1 --2- 172.21.15.77:0/3476658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fc4ec31670 0x55fc4ec392d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:35.805 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.795+0000 7f7f4f400640 1 -- 172.21.15.77:0/3476658 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fc4ec3bf30 msgr2=0x55fc4ec39810 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T09:56:35.805 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.795+0000 7f7f4fe00640 1 --2- 172.21.15.77:0/3476658 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fc4ec3bf30 0x55fc4ec39810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:35.805 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.795+0000 7f7f4f400640 1 --2- 172.21.15.77:0/3476658 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fc4ec3bf30 0x55fc4ec39810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:35.805 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.795+0000 7f7f4f400640 1 -- 172.21.15.77:0/3476658 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc4e9083c0 con 0x55fc4ec30d50 2024-09-10T09:56:35.805 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.796+0000 7f7f4fe00640 1 --2- 172.21.15.77:0/3476658 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fc4ec3bf30 0x55fc4ec39810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:56:35.805 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.796+0000 7f7f4f400640 1 --2- 172.21.15.77:0/3476658 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc4ec30d50 0x55fc4ec38d90 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7f400a42a0 tx=0x7f7f4009c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee141c3c9c46e555 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:35.805 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.796+0000 7f7f39600640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T09:56:35.805 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.796+0000 7f7f39600640 1 -- 172.21.15.77:0/3476658 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f400ccf10 con 0x55fc4ec30d50 2024-09-10T09:56:35.805 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.796+0000 7f7f59016740 1 -- 172.21.15.77:0/3476658 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f4801cac0 con 0x55fc4ec30d50 2024-09-10T09:56:35.805 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.796+0000 7f7f39600640 1 -- 172.21.15.77:0/3476658 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f7f400cf3e0 con 0x55fc4ec30d50 2024-09-10T09:56:35.805 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.796+0000 7f7f39600640 1 -- 172.21.15.77:0/3476658 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7f400ba2e0 con 0x55fc4ec30d50 2024-09-10T09:56:35.806 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.796+0000 7f7f59016740 10 client.25748 fetch_fsmap learned FSMap version 29 2024-09-10T09:56:35.806 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.796+0000 7f7f59016740 1 -- 172.21.15.77:0/3476658 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55fc4ec4a7a0 con 0x55fc4ec30d50 2024-09-10T09:56:35.806 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.797+0000 7f7f39600640 1 -- 172.21.15.77:0/3476658 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f4801cac0 con 0x55fc4ec30d50 2024-09-10T09:56:35.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.797+0000 7f7f39600640 1 -- 172.21.15.77:0/3476658 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f7f40010c50 con 0x55fc4ec30d50 2024-09-10T09:56:35.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.797+0000 7f7f59016740 10 client.25748 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T09:56:35.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.797+0000 7f7f59016740 10 client.25748 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:56:35.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.797+0000 7f7f59016740 20 client.25748 populate_metadata read hostname 'smithi077' 2024-09-10T09:56:35.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.797+0000 7f7f59016740 1 --2- 172.21.15.77:0/3476658 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55fc4ec4bba0 0x55fc4ec4e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:35.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.797+0000 7f7f59016740 4 client.25748 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:56:35.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.797+0000 7f7f59016740 1 -- 172.21.15.77:0/3476658 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fc4eba4950 con 0x55fc4ec4bba0 2024-09-10T09:56:35.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.799+0000 7f7f4ea00640 1 --2- 172.21.15.77:0/3476658 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55fc4ec4bba0 0x55fc4ec4e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:35.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.799+0000 7f7f4ea00640 1 --2- 172.21.15.77:0/3476658 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55fc4ec4bba0 0x55fc4ec4e040 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5bafd9cc7d49ff42 server_cookie=9c8ae74854836093 in_seq=0 out_seq=0 2024-09-10T09:56:35.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.799+0000 7f7f39600640 10 client.25748 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T09:56:35.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.800+0000 7f7f39600640 1 -- 172.21.15.77:0/3476658 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fc4eba4950 con 0x55fc4ec4bba0 2024-09-10T09:56:35.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.800+0000 7f7f39600640 10 client.25748 handle_command_reply: tid=0 2024-09-10T09:56:35.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.800+0000 7f7f59016740 1 client.25748 shutdown 2024-09-10T09:56:35.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.801+0000 7f7f59016740 1 -- 172.21.15.77:0/3476658 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc4ec30d50 msgr2=0x55fc4ec38d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:35.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.801+0000 7f7f59016740 1 --2- 172.21.15.77:0/3476658 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc4ec30d50 0x55fc4ec38d90 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7f400a42a0 tx=0x7f7f4009c3a0 comp rx=0 tx=0).stop 2024-09-10T09:56:35.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.801+0000 7f7f59016740 1 -- 172.21.15.77:0/3476658 shutdown_connections 2024-09-10T09:56:35.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.801+0000 7f7f59016740 1 --2- 172.21.15.77:0/3476658 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55fc4ec4bba0 0x55fc4ec4e040 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:35.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.801+0000 7f7f59016740 1 --2- 172.21.15.77:0/3476658 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fc4ec3bf30 0x55fc4ec39810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:35.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.801+0000 7f7f59016740 1 --2- 172.21.15.77:0/3476658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fc4ec31670 0x55fc4ec392d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:35.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.801+0000 7f7f59016740 1 --2- 172.21.15.77:0/3476658 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc4ec30d50 0x55fc4ec38d90 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:35.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.801+0000 7f7f59016740 1 -- 172.21.15.77:0/3476658 >> 172.21.15.77:0/3476658 conn(0x55fc4ec2d2e0 msgr2=0x55fc4ec32fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:35.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.801+0000 7f7f39600640 0 client.25748 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T09:56:35.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.801+0000 7f7f59016740 1 -- 172.21.15.77:0/3476658 shutdown_connections 2024-09-10T09:56:35.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.801+0000 7f7f59016740 1 -- 172.21.15.77:0/3476658 wait complete. 2024-09-10T09:56:35.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.802+0000 7f7f59016740 20 client.25748 trim_cache size 0 max 16384 2024-09-10T09:56:35.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.838+0000 7f7f59016740 1 Processor -- start 2024-09-10T09:56:35.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.838+0000 7f7f59016740 1 -- start start 2024-09-10T09:56:35.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.838+0000 7f7f59016740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc4ec30d50 0x55fc4ec337b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:35.842 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.838+0000 7f7f59016740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fc4ec31670 0x55fc4ec53910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:35.842 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.838+0000 7f7f59016740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fc4ec3bf30 0x55fc4ec53e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:35.842 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.838+0000 7f7f59016740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7f3000fd20 con 0x55fc4ec3bf30 2024-09-10T09:56:35.842 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.838+0000 7f7f59016740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7f30011860 con 0x55fc4ec30d50 2024-09-10T09:56:35.842 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.838+0000 7f7f59016740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7f300103f0 con 0x55fc4ec31670 2024-09-10T09:56:35.842 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.839+0000 7f7f4f400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc4ec30d50 0x55fc4ec337b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:35.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.839+0000 7f7f4f400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc4ec30d50 0x55fc4ec337b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:49622/0 (socket says 172.21.15.77:49622) 2024-09-10T09:56:35.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.839+0000 7f7f4f400640 1 -- 172.21.15.77:0/1748590138 learned_addr learned my addr 172.21.15.77:0/1748590138 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:35.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.839+0000 7f7f4fe00640 1 --2- 172.21.15.77:0/1748590138 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fc4ec3bf30 0x55fc4ec53e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:35.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.839+0000 7f7f4f400640 1 -- 172.21.15.77:0/1748590138 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fc4ec31670 msgr2=0x55fc4ec53910 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T09:56:35.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.839+0000 7f7f4f400640 1 --2- 172.21.15.77:0/1748590138 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fc4ec31670 0x55fc4ec53910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:35.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.839+0000 7f7f4f400640 1 -- 172.21.15.77:0/1748590138 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fc4ec3bf30 msgr2=0x55fc4ec53e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:35.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.839+0000 7f7f4f400640 1 --2- 172.21.15.77:0/1748590138 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fc4ec3bf30 0x55fc4ec53e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:35.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.839+0000 7f7f4f400640 1 -- 172.21.15.77:0/1748590138 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc4ec3b700 con 0x55fc4ec30d50 2024-09-10T09:56:35.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.840+0000 7f7f4fe00640 1 --2- 172.21.15.77:0/1748590138 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fc4ec3bf30 0x55fc4ec53e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:56:35.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.840+0000 7f7f4f400640 1 --2- 172.21.15.77:0/1748590138 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc4ec30d50 0x55fc4ec337b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7f40099e10 tx=0x7f7f400a4950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c19be103bb372c98 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:35.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.840+0000 7f7f3be00640 1 -- 172.21.15.77:0/1748590138 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f4009ff10 con 0x55fc4ec30d50 2024-09-10T09:56:35.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.841+0000 7f7f3be00640 1 -- 172.21.15.77:0/1748590138 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f400b3020 con 0x55fc4ec30d50 2024-09-10T09:56:35.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.841+0000 7f7f3be00640 1 -- 172.21.15.77:0/1748590138 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f40097a40 con 0x55fc4ec30d50 2024-09-10T09:56:35.844 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.841+0000 7f7f59016740 1 -- 172.21.15.77:0/1748590138 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc4ec30d50 msgr2=0x55fc4ec337b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:35.845 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.841+0000 7f7f59016740 1 --2- 172.21.15.77:0/1748590138 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc4ec30d50 0x55fc4ec337b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7f40099e10 tx=0x7f7f400a4950 comp rx=0 tx=0).stop 2024-09-10T09:56:35.845 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.842+0000 7f7f59016740 1 -- 172.21.15.77:0/1748590138 shutdown_connections 2024-09-10T09:56:35.845 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.842+0000 7f7f59016740 1 --2- 172.21.15.77:0/1748590138 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fc4ec3bf30 0x55fc4ec53e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:35.845 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.842+0000 7f7f59016740 1 --2- 172.21.15.77:0/1748590138 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fc4ec31670 0x55fc4ec53910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:35.845 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.842+0000 7f7f59016740 1 --2- 172.21.15.77:0/1748590138 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc4ec30d50 0x55fc4ec337b0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:35.845 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.842+0000 7f7f59016740 1 -- 172.21.15.77:0/1748590138 >> 172.21.15.77:0/1748590138 conn(0x55fc4ec2d2e0 msgr2=0x55fc4ec37680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:35.845 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.842+0000 7f7f59016740 1 -- 172.21.15.77:0/1748590138 shutdown_connections 2024-09-10T09:56:35.845 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.842+0000 7f7f59016740 1 -- 172.21.15.77:0/1748590138 wait complete. 2024-09-10T09:56:35.846 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.843+0000 7f7f59016740 1 -- start start 2024-09-10T09:56:35.846 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.843+0000 7f7f59016740 1 Processor -- start 2024-09-10T09:56:35.846 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.844+0000 7f7f59016740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc4ec30d50 0x55fc4ec43fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:35.847 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.844+0000 7f7f59016740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fc4ec31670 0x55fc4ec44510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:35.847 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.844+0000 7f7f59016740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fc4ec3bf30 0x55fc4ec40fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:35.847 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.844+0000 7f7f59016740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7f30011190 con 0x55fc4ec31670 2024-09-10T09:56:35.847 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.844+0000 7f7f59016740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7f30010ac0 con 0x55fc4ec30d50 2024-09-10T09:56:35.847 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.844+0000 7f7f59016740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7f3000dd00 con 0x55fc4ec3bf30 2024-09-10T09:56:35.847 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.845+0000 7f7f59016740 10 client.0 _ll_register_callbacks cb 0x7f7f4009b5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:56:35.848 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.845+0000 7f7f4f400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc4ec30d50 0x55fc4ec43fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:35.848 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.845+0000 7f7f4fe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fc4ec3bf30 0x55fc4ec40fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:35.848 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.845+0000 7f7f4f400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc4ec30d50 0x55fc4ec43fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:49624/0 (socket says 172.21.15.77:49624) 2024-09-10T09:56:35.848 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.845+0000 7f7f4f400640 1 -- 172.21.15.77:0/117025828 learned_addr learned my addr 172.21.15.77:0/117025828 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:35.848 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.846+0000 7f7f4ea00640 1 --2- 172.21.15.77:0/117025828 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fc4ec31670 0x55fc4ec44510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:35.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.846+0000 7f7f4f400640 1 -- 172.21.15.77:0/117025828 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fc4ec3bf30 msgr2=0x55fc4ec40fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:35.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.846+0000 7f7f4f400640 1 --2- 172.21.15.77:0/117025828 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fc4ec3bf30 0x55fc4ec40fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:35.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.846+0000 7f7f4f400640 1 -- 172.21.15.77:0/117025828 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fc4ec31670 msgr2=0x55fc4ec44510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:35.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.846+0000 7f7f4f400640 1 --2- 172.21.15.77:0/117025828 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fc4ec31670 0x55fc4ec44510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:35.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.846+0000 7f7f4f400640 1 -- 172.21.15.77:0/117025828 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc4ec44c00 con 0x55fc4ec30d50 2024-09-10T09:56:35.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.846+0000 7f7f4ea00640 1 --2- 172.21.15.77:0/117025828 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fc4ec31670 0x55fc4ec44510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:56:35.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.847+0000 7f7f4f400640 1 --2- 172.21.15.77:0/117025828 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc4ec30d50 0x55fc4ec43fd0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7f40018490 tx=0x7f7f4009a070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d4fd2fd920bb99d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:35.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.847+0000 7f7f39600640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T09:56:35.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.847+0000 7f7f39600640 1 -- 172.21.15.77:0/117025828 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f4009b170 con 0x55fc4ec30d50 2024-09-10T09:56:35.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.847+0000 7f7f39600640 1 -- 172.21.15.77:0/117025828 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f7f400979f0 con 0x55fc4ec30d50 2024-09-10T09:56:35.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.847+0000 7f7f59016740 1 -- 172.21.15.77:0/117025828 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f400b3020 con 0x55fc4ec30d50 2024-09-10T09:56:35.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.848+0000 7f7f39600640 1 -- 172.21.15.77:0/117025828 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7f40097bf0 con 0x55fc4ec30d50 2024-09-10T09:56:35.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.848+0000 7f7f59016740 10 client.25754 fetch_fsmap learned FSMap version 29 2024-09-10T09:56:35.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.848+0000 7f7f59016740 1 -- 172.21.15.77:0/117025828 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55fc4ec44c00 con 0x55fc4ec30d50 2024-09-10T09:56:35.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.848+0000 7f7f39600640 1 -- 172.21.15.77:0/117025828 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f400b3020 con 0x55fc4ec30d50 2024-09-10T09:56:35.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.849+0000 7f7f39600640 1 -- 172.21.15.77:0/117025828 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f7f40083030 con 0x55fc4ec30d50 2024-09-10T09:56:35.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.849+0000 7f7f59016740 10 client.25754 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T09:56:35.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.849+0000 7f7f59016740 10 client.25754 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:56:35.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.849+0000 7f7f59016740 20 client.25754 populate_metadata read hostname 'smithi077' 2024-09-10T09:56:35.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.849+0000 7f7f59016740 1 --2- 172.21.15.77:0/117025828 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55fc4ecc8190 0x55fc4ecca5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:35.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.849+0000 7f7f59016740 4 client.25754 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T09:56:35.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.849+0000 7f7f59016740 1 -- 172.21.15.77:0/117025828 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fc4eb9ee60 con 0x55fc4ecc8190 2024-09-10T09:56:35.852 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.850+0000 7f7f4ea00640 1 --2- 172.21.15.77:0/117025828 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55fc4ecc8190 0x55fc4ecca5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:35.852 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.851+0000 7f7f4ea00640 1 --2- 172.21.15.77:0/117025828 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55fc4ecc8190 0x55fc4ecca5b0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c64b5298dd0d4578 server_cookie=12c1eee0294c01a4 in_seq=0 out_seq=0 2024-09-10T09:56:35.852 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.851+0000 7f7f39600640 10 client.25754 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T09:56:35.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.851+0000 7f7f39600640 1 -- 172.21.15.77:0/117025828 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55fc4eb9ee60 con 0x55fc4ecc8190 2024-09-10T09:56:35.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.851+0000 7f7f39600640 10 client.25754 handle_command_reply: tid=0 2024-09-10T09:56:35.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.851+0000 7f7f59016740 1 client.25754 shutdown 2024-09-10T09:56:35.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.852+0000 7f7f59016740 1 -- 172.21.15.77:0/117025828 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc4ec30d50 msgr2=0x55fc4ec43fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:35.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.852+0000 7f7f59016740 1 --2- 172.21.15.77:0/117025828 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc4ec30d50 0x55fc4ec43fd0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7f40018490 tx=0x7f7f4009a070 comp rx=0 tx=0).stop 2024-09-10T09:56:35.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.852+0000 7f7f59016740 1 -- 172.21.15.77:0/117025828 shutdown_connections 2024-09-10T09:56:35.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.852+0000 7f7f59016740 1 --2- 172.21.15.77:0/117025828 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55fc4ecc8190 0x55fc4ecca5b0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:35.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.852+0000 7f7f59016740 1 --2- 172.21.15.77:0/117025828 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fc4ec3bf30 0x55fc4ec40fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:35.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.852+0000 7f7f59016740 1 --2- 172.21.15.77:0/117025828 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fc4ec31670 0x55fc4ec44510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:35.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.852+0000 7f7f59016740 1 --2- 172.21.15.77:0/117025828 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc4ec30d50 0x55fc4ec43fd0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:35.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.852+0000 7f7f59016740 1 -- 172.21.15.77:0/117025828 >> 172.21.15.77:0/117025828 conn(0x55fc4ec2d2e0 msgr2=0x55fc4ec46e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:35.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.852+0000 7f7f39600640 0 client.25754 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T09:56:35.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.853+0000 7f7f59016740 1 -- 172.21.15.77:0/117025828 shutdown_connections 2024-09-10T09:56:35.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.853+0000 7f7f59016740 1 -- 172.21.15.77:0/117025828 wait complete. 2024-09-10T09:56:35.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.853+0000 7f7f59016740 20 client.25754 trim_cache size 0 max 16384 2024-09-10T09:56:35.855 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-10T09:56:35.855 INFO:teuthology.orchestra.run.smithi077.stdout: "status": "no active scrubs running", 2024-09-10T09:56:35.855 INFO:teuthology.orchestra.run.smithi077.stdout: "scrubs": {} 2024-09-10T09:56:35.855 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-10T09:56:35.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.854+0000 7f7f54e00640 1 -- 172.21.15.77:0/1964630736 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f7f340a2c80 msgr2=0x7f7f340a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:35.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.854+0000 7f7f54e00640 1 --2- 172.21.15.77:0/1964630736 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f7f340a2c80 0x7f7f340a5140 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7f48021000 tx=0x7f7f4801e000 comp rx=0 tx=0).stop 2024-09-10T09:56:35.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.854+0000 7f7f54e00640 1 -- 172.21.15.77:0/1964630736 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7f50074510 msgr2=0x7f7f5010dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:35.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.854+0000 7f7f54e00640 1 --2- 172.21.15.77:0/1964630736 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7f50074510 0x7f7f5010dc20 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7f4000e9f0 tx=0x7f7f4000eeb0 comp rx=0 tx=0).stop 2024-09-10T09:56:35.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.854+0000 7f7f54e00640 1 -- 172.21.15.77:0/1964630736 shutdown_connections 2024-09-10T09:56:35.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.854+0000 7f7f54e00640 1 --2- 172.21.15.77:0/1964630736 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f7f340a2c80 0x7f7f340a5140 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:35.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.854+0000 7f7f54e00640 1 --2- 172.21.15.77:0/1964630736 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f5010e280 0x7f7f5010b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:35.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.854+0000 7f7f54e00640 1 --2- 172.21.15.77:0/1964630736 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f50074e30 0x7f7f5010acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:35.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.854+0000 7f7f54e00640 1 --2- 172.21.15.77:0/1964630736 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7f50074510 0x7f7f5010dc20 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:35.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.854+0000 7f7f54e00640 1 -- 172.21.15.77:0/1964630736 >> 172.21.15.77:0/1964630736 conn(0x7f7f500fd740 msgr2=0x7f7f500fe440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:35.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.855+0000 7f7f54e00640 1 -- 172.21.15.77:0/1964630736 shutdown_connections 2024-09-10T09:56:35.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:35.855+0000 7f7f54e00640 1 -- 172.21.15.77:0/1964630736 wait complete. 2024-09-10T09:56:36.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:56:35 smithi120 ceph-mon[34770]: scrub summary: idle 2024-09-10T09:56:36.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:56:35 smithi077 ceph-mon[30319]: scrub summary: idle 2024-09-10T09:56:36.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:56:35 smithi134 ceph-mon[35674]: scrub summary: idle 2024-09-10T09:56:36.810 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T09:56:36.811 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:56:36.849 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:56:37.039 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:56:36 smithi077 ceph-mon[30319]: pgmap v434: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T09:56:37.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:56:36 smithi120 ceph-mon[34770]: pgmap v434: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T09:56:37.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:56:36 smithi134 ceph-mon[35674]: pgmap v434: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T09:56:37.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:37.501+0000 7f0258c00640 1 -- 172.21.15.77:0/822546988 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f02541079c0 msgr2=0x7f0254107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:37.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:37.501+0000 7f0258c00640 1 --2- 172.21.15.77:0/822546988 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f02541079c0 0x7f0254107da0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0244009930 tx=0x7f024402a420 comp rx=0 tx=0).stop 2024-09-10T09:56:37.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:37.502+0000 7f0258c00640 1 -- 172.21.15.77:0/822546988 shutdown_connections 2024-09-10T09:56:37.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:37.502+0000 7f0258c00640 1 --2- 172.21.15.77:0/822546988 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0254102360 0x7f025410e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:37.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:37.502+0000 7f0258c00640 1 --2- 172.21.15.77:0/822546988 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f02541019c0 0x7f0254101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:37.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:37.502+0000 7f0258c00640 1 --2- 172.21.15.77:0/822546988 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f02541079c0 0x7f0254107da0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:37.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:37.502+0000 7f0258c00640 1 -- 172.21.15.77:0/822546988 >> 172.21.15.77:0/822546988 conn(0x7f02540fd740 msgr2=0x7f02540ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:37.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:37.503+0000 7f0258c00640 1 -- 172.21.15.77:0/822546988 shutdown_connections 2024-09-10T09:56:37.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:37.503+0000 7f0258c00640 1 -- 172.21.15.77:0/822546988 wait complete. 2024-09-10T09:56:37.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:37.504+0000 7f0258c00640 1 Processor -- start 2024-09-10T09:56:37.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:37.504+0000 7f0258c00640 1 -- start start 2024-09-10T09:56:37.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:37.505+0000 7f0258c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f02541019c0 0x7f02541960e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:37.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:37.505+0000 7f0258c00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0254102360 0x7f0254196620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:37.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:37.505+0000 7f0258c00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f02541079c0 0x7f025419a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:37.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:37.505+0000 7f0258c00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f025410fc00 con 0x7f02541079c0 2024-09-10T09:56:37.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:37.505+0000 7f0258c00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f025410fa80 con 0x7f0254102360 2024-09-10T09:56:37.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:37.505+0000 7f0258c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f025410fd80 con 0x7f02541019c0 2024-09-10T09:56:37.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:37.505+0000 7f0253400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f02541019c0 0x7f02541960e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:37.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:37.505+0000 7f0253400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f02541019c0 0x7f02541960e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:37102/0 (socket says 172.21.15.77:37102) 2024-09-10T09:56:37.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:37.505+0000 7f0253400640 1 -- 172.21.15.77:0/284947053 learned_addr learned my addr 172.21.15.77:0/284947053 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:37.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:37.506+0000 7f0253e00640 1 --2- 172.21.15.77:0/284947053 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f02541079c0 0x7f025419a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:37.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:37.506+0000 7f0253400640 1 -- 172.21.15.77:0/284947053 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0254102360 msgr2=0x7f0254196620 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T09:56:37.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:37.506+0000 7f0253400640 1 --2- 172.21.15.77:0/284947053 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0254102360 0x7f0254196620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:37.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:37.506+0000 7f0253400640 1 -- 172.21.15.77:0/284947053 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f02541079c0 msgr2=0x7f025419a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:37.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:37.506+0000 7f0253400640 1 --2- 172.21.15.77:0/284947053 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f02541079c0 0x7f025419a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:37.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:37.506+0000 7f0253400640 1 -- 172.21.15.77:0/284947053 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f025419b090 con 0x7f02541019c0 2024-09-10T09:56:37.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:37.507+0000 7f0253400640 1 --2- 172.21.15.77:0/284947053 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f02541019c0 0x7f02541960e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f024402a940 tx=0x7f0244003b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:37.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:37.508+0000 7f024be00640 1 -- 172.21.15.77:0/284947053 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f024403f070 con 0x7f02541019c0 2024-09-10T09:56:37.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:37.508+0000 7f024be00640 1 -- 172.21.15.77:0/284947053 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0244031c90 con 0x7f02541019c0 2024-09-10T09:56:37.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:37.508+0000 7f0258c00640 1 -- 172.21.15.77:0/284947053 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f025419b320 con 0x7f02541019c0 2024-09-10T09:56:37.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:37.508+0000 7f0258c00640 1 -- 172.21.15.77:0/284947053 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02541a2c60 con 0x7f02541019c0 2024-09-10T09:56:37.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:37.508+0000 7f024be00640 1 -- 172.21.15.77:0/284947053 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0244035030 con 0x7f02541019c0 2024-09-10T09:56:37.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:37.510+0000 7f0258c00640 1 -- 172.21.15.77:0/284947053 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0254101e20 con 0x7f02541019c0 2024-09-10T09:56:37.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:37.514+0000 7f024be00640 1 -- 172.21.15.77:0/284947053 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0244042070 con 0x7f02541019c0 2024-09-10T09:56:37.519 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:37.517+0000 7f024be00640 1 --2- 172.21.15.77:0/284947053 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f02380a70e0 0x7f02380a95a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:37.519 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:37.517+0000 7f024be00640 1 -- 172.21.15.77:0/284947053 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f02440f0210 con 0x7f02541019c0 2024-09-10T09:56:37.519 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:37.517+0000 7f024be00640 1 -- 172.21.15.77:0/284947053 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f02440f0730 con 0x7f02541019c0 2024-09-10T09:56:37.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:37.518+0000 7f0252a00640 1 --2- 172.21.15.77:0/284947053 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f02380a70e0 0x7f02380a95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:37.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:37.519+0000 7f0252a00640 1 --2- 172.21.15.77:0/284947053 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f02380a70e0 0x7f02380a95a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f02540ffd50 tx=0x7f0240005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:37.586 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:56:37.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:37.811+0000 7f0258c00640 1 -- 172.21.15.77:0/284947053 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0254109f30 con 0x7f02541019c0 2024-09-10T09:56:37.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:37.812+0000 7f024be00640 1 -- 172.21.15.77:0/284947053 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11989 (secure 0 0 0) 0x7f02440bce00 con 0x7f02541019c0 2024-09-10T09:56:37.814 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:56:37.814 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":29,"btime":"2024-09-10T09:55:13:863715+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3159289675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3159289675},{"type":"v1","addr":"172.21.15.77:6835","nonce":3159289675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1951962341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1951962341},{"type":"v1","addr":"172.21.15.77:6837","nonce":1951962341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:52:23.368252+0000","modified":"2024-09-10T09:55:13.786975+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24563,"mds_1":24524,"mds_2":24520,"mds_3":24502,"mds_4":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14487":{"gid":14487,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/4291121888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":4291121888},{"type":"v1","addr":"172.21.15.77:6839","nonce":4291121888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14499":{"gid":14499,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6841/445284194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":445284194},{"type":"v1","addr":"172.21.15.77:6841","nonce":445284194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1005054423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1005054423},{"type":"v1","addr":"172.21.15.134:6833","nonce":1005054423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6833/3716307744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":3716307744},{"type":"v1","addr":"172.21.15.120:6833","nonce":3716307744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/1319892544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1319892544},{"type":"v1","addr":"172.21.15.120:6835","nonce":1319892544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.134:6835/3293933699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3293933699},{"type":"v1","addr":"172.21.15.134:6835","nonce":3293933699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.134:6837/2461497091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2461497091},{"type":"v1","addr":"172.21.15.134:6837","nonce":2461497091}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":20,"addr":"172.21.15.134:6839/1965117680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1965117680},{"type":"v1","addr":"172.21.15.134:6839","nonce":1965117680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":20,"addr":"172.21.15.120:6837/3033280709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3033280709},{"type":"v1","addr":"172.21.15.120:6837","nonce":3033280709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.120:6839/1777710157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1777710157},{"type":"v1","addr":"172.21.15.120:6839","nonce":1777710157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24563,"qdb_cluster":[24496,24502,24520,24524,24563]},"id":1}]} 2024-09-10T09:56:37.814 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 29 2024-09-10T09:56:37.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:37.815+0000 7f0258c00640 1 -- 172.21.15.77:0/284947053 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f02380a70e0 msgr2=0x7f02380a95a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:37.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:37.815+0000 7f0258c00640 1 --2- 172.21.15.77:0/284947053 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f02380a70e0 0x7f02380a95a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f02540ffd50 tx=0x7f0240005f90 comp rx=0 tx=0).stop 2024-09-10T09:56:37.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:37.815+0000 7f0258c00640 1 -- 172.21.15.77:0/284947053 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f02541019c0 msgr2=0x7f02541960e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:37.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:37.815+0000 7f0258c00640 1 --2- 172.21.15.77:0/284947053 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f02541019c0 0x7f02541960e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f024402a940 tx=0x7f0244003b20 comp rx=0 tx=0).stop 2024-09-10T09:56:37.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:37.815+0000 7f0258c00640 1 -- 172.21.15.77:0/284947053 shutdown_connections 2024-09-10T09:56:37.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:37.815+0000 7f0258c00640 1 --2- 172.21.15.77:0/284947053 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f02380a70e0 0x7f02380a95a0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:37.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:37.815+0000 7f0258c00640 1 --2- 172.21.15.77:0/284947053 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f02541079c0 0x7f025419a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:37.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:37.815+0000 7f0258c00640 1 --2- 172.21.15.77:0/284947053 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0254102360 0x7f0254196620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:37.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:37.815+0000 7f0258c00640 1 --2- 172.21.15.77:0/284947053 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f02541019c0 0x7f02541960e0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:37.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:37.815+0000 7f0258c00640 1 -- 172.21.15.77:0/284947053 >> 172.21.15.77:0/284947053 conn(0x7f02540fd740 msgr2=0x7f02540fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:37.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:37.816+0000 7f0258c00640 1 -- 172.21.15.77:0/284947053 shutdown_connections 2024-09-10T09:56:37.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:37.816+0000 7f0258c00640 1 -- 172.21.15.77:0/284947053 wait complete. 2024-09-10T09:56:38.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:56:37 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/284947053' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:56:38.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:56:37 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/284947053' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:56:38.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:56:37 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/284947053' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:56:38.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:38.374+0000 7f4186000640 1 -- 172.21.15.77:0/2847517118 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4180077040 msgr2=0x7f41801124f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:38.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:38.374+0000 7f4186000640 1 --2- 172.21.15.77:0/2847517118 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4180077040 0x7f41801124f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f416c009860 tx=0x7f416c02a340 comp rx=0 tx=0).stop 2024-09-10T09:56:38.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:38.374+0000 7f4186000640 1 -- 172.21.15.77:0/2847517118 shutdown_connections 2024-09-10T09:56:38.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:38.374+0000 7f4186000640 1 --2- 172.21.15.77:0/2847517118 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4180112a30 0x7f4180114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:38.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:38.374+0000 7f4186000640 1 --2- 172.21.15.77:0/2847517118 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4180077040 0x7f41801124f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:38.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:38.374+0000 7f4186000640 1 --2- 172.21.15.77:0/2847517118 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4180076720 0x7f4180076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:38.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:38.374+0000 7f4186000640 1 -- 172.21.15.77:0/2847517118 >> 172.21.15.77:0/2847517118 conn(0x7f41800ffa90 msgr2=0x7f4180101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:38.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:38.374+0000 7f4186000640 1 -- 172.21.15.77:0/2847517118 shutdown_connections 2024-09-10T09:56:38.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:38.374+0000 7f4186000640 1 -- 172.21.15.77:0/2847517118 wait complete. 2024-09-10T09:56:38.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:38.375+0000 7f4186000640 1 Processor -- start 2024-09-10T09:56:38.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:38.375+0000 7f4186000640 1 -- start start 2024-09-10T09:56:38.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:38.376+0000 7f4186000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4180076720 0x7f41801a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:38.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:38.376+0000 7f4186000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4180077040 0x7f41801a4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:38.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:38.376+0000 7f4186000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4180112a30 0x7f418019e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:38.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:38.376+0000 7f4186000640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f41801185a0 con 0x7f4180112a30 2024-09-10T09:56:38.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:38.376+0000 7f4186000640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4180118420 con 0x7f4180076720 2024-09-10T09:56:38.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:38.376+0000 7f4186000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4180118720 con 0x7f4180077040 2024-09-10T09:56:38.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:38.376+0000 7f417f400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4180076720 0x7f41801a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:38.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:38.376+0000 7f417f400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4180076720 0x7f41801a46f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:49650/0 (socket says 172.21.15.77:49650) 2024-09-10T09:56:38.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:38.376+0000 7f417f400640 1 -- 172.21.15.77:0/290869116 learned_addr learned my addr 172.21.15.77:0/290869116 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:38.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:38.377+0000 7f417fe00640 1 --2- 172.21.15.77:0/290869116 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4180112a30 0x7f418019e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:38.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:38.377+0000 7f417f400640 1 -- 172.21.15.77:0/290869116 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4180077040 msgr2=0x7f41801a4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:38.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:38.377+0000 7f417f400640 1 --2- 172.21.15.77:0/290869116 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4180077040 0x7f41801a4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:38.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:38.377+0000 7f417f400640 1 -- 172.21.15.77:0/290869116 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4180112a30 msgr2=0x7f418019e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:38.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:38.377+0000 7f417f400640 1 --2- 172.21.15.77:0/290869116 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4180112a30 0x7f418019e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:38.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:38.377+0000 7f417f400640 1 -- 172.21.15.77:0/290869116 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f418019efe0 con 0x7f4180076720 2024-09-10T09:56:38.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:38.377+0000 7f417fe00640 1 --2- 172.21.15.77:0/290869116 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4180112a30 0x7f418019e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:56:38.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:38.377+0000 7f417f400640 1 --2- 172.21.15.77:0/290869116 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4180076720 0x7f41801a46f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f417000e9c0 tx=0x7f417000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:38.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:38.378+0000 7f4177e00640 1 -- 172.21.15.77:0/290869116 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f417000cde0 con 0x7f4180076720 2024-09-10T09:56:38.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:38.378+0000 7f4177e00640 1 -- 172.21.15.77:0/290869116 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41700035f0 con 0x7f4180076720 2024-09-10T09:56:38.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:38.378+0000 7f4177e00640 1 -- 172.21.15.77:0/290869116 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4170010690 con 0x7f4180076720 2024-09-10T09:56:38.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:38.378+0000 7f4186000640 1 -- 172.21.15.77:0/290869116 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f418019f2d0 con 0x7f4180076720 2024-09-10T09:56:38.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:38.378+0000 7f4186000640 1 -- 172.21.15.77:0/290869116 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41801ab5c0 con 0x7f4180076720 2024-09-10T09:56:38.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:38.380+0000 7f4186000640 1 -- 172.21.15.77:0/290869116 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4180076b00 con 0x7f4180076720 2024-09-10T09:56:38.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:38.383+0000 7f4177e00640 1 -- 172.21.15.77:0/290869116 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f41700037a0 con 0x7f4180076720 2024-09-10T09:56:38.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:38.385+0000 7f4177e00640 1 --2- 172.21.15.77:0/290869116 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f41640a2c80 0x7f41640a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:38.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:38.385+0000 7f4177e00640 1 -- 172.21.15.77:0/290869116 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f41700cf840 con 0x7f4180076720 2024-09-10T09:56:38.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:38.385+0000 7f417ea00640 1 --2- 172.21.15.77:0/290869116 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f41640a2c80 0x7f41640a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:38.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:38.386+0000 7f417ea00640 1 --2- 172.21.15.77:0/290869116 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f41640a2c80 0x7f41640a5140 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f416c009780 tx=0x7f416c030460 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:38.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:38.388+0000 7f4177e00640 1 -- 172.21.15.77:0/290869116 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f417009c350 con 0x7f4180076720 2024-09-10T09:56:38.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:38.612+0000 7f4186000640 1 -- 172.21.15.77:0/290869116 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f418019fde0 con 0x7f4180076720 2024-09-10T09:56:38.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:38.613+0000 7f4177e00640 1 -- 172.21.15.77:0/290869116 <== mon.2 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11989 (secure 0 0 0) 0x7f418019fde0 con 0x7f4180076720 2024-09-10T09:56:38.616 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:56:38.616 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":29,"btime":"2024-09-10T09:55:13:863715+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3159289675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3159289675},{"type":"v1","addr":"172.21.15.77:6835","nonce":3159289675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1951962341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1951962341},{"type":"v1","addr":"172.21.15.77:6837","nonce":1951962341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:52:23.368252+0000","modified":"2024-09-10T09:55:13.786975+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24563,"mds_1":24524,"mds_2":24520,"mds_3":24502,"mds_4":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14487":{"gid":14487,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/4291121888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":4291121888},{"type":"v1","addr":"172.21.15.77:6839","nonce":4291121888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14499":{"gid":14499,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6841/445284194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":445284194},{"type":"v1","addr":"172.21.15.77:6841","nonce":445284194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1005054423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1005054423},{"type":"v1","addr":"172.21.15.134:6833","nonce":1005054423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6833/3716307744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":3716307744},{"type":"v1","addr":"172.21.15.120:6833","nonce":3716307744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/1319892544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1319892544},{"type":"v1","addr":"172.21.15.120:6835","nonce":1319892544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.134:6835/3293933699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3293933699},{"type":"v1","addr":"172.21.15.134:6835","nonce":3293933699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.134:6837/2461497091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2461497091},{"type":"v1","addr":"172.21.15.134:6837","nonce":2461497091}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":20,"addr":"172.21.15.134:6839/1965117680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1965117680},{"type":"v1","addr":"172.21.15.134:6839","nonce":1965117680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":20,"addr":"172.21.15.120:6837/3033280709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3033280709},{"type":"v1","addr":"172.21.15.120:6837","nonce":3033280709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.120:6839/1777710157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1777710157},{"type":"v1","addr":"172.21.15.120:6839","nonce":1777710157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24563,"qdb_cluster":[24496,24502,24520,24524,24563]},"id":1}]} 2024-09-10T09:56:38.617 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 29 2024-09-10T09:56:38.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:38.616+0000 7f4186000640 1 -- 172.21.15.77:0/290869116 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f41640a2c80 msgr2=0x7f41640a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:38.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:38.616+0000 7f4186000640 1 --2- 172.21.15.77:0/290869116 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f41640a2c80 0x7f41640a5140 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f416c009780 tx=0x7f416c030460 comp rx=0 tx=0).stop 2024-09-10T09:56:38.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:38.616+0000 7f4186000640 1 -- 172.21.15.77:0/290869116 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4180076720 msgr2=0x7f41801a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:38.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:38.616+0000 7f4186000640 1 --2- 172.21.15.77:0/290869116 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4180076720 0x7f41801a46f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f417000e9c0 tx=0x7f417000ee80 comp rx=0 tx=0).stop 2024-09-10T09:56:38.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:38.617+0000 7f4186000640 1 -- 172.21.15.77:0/290869116 shutdown_connections 2024-09-10T09:56:38.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:38.617+0000 7f4186000640 1 --2- 172.21.15.77:0/290869116 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f41640a2c80 0x7f41640a5140 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:38.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:38.617+0000 7f4186000640 1 --2- 172.21.15.77:0/290869116 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4180112a30 0x7f418019e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:38.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:38.617+0000 7f4186000640 1 --2- 172.21.15.77:0/290869116 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4180077040 0x7f41801a4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:38.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:38.617+0000 7f4186000640 1 --2- 172.21.15.77:0/290869116 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4180076720 0x7f41801a46f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:38.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:38.617+0000 7f4186000640 1 -- 172.21.15.77:0/290869116 >> 172.21.15.77:0/290869116 conn(0x7f41800ffa90 msgr2=0x7f4180101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:38.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:38.617+0000 7f4186000640 1 -- 172.21.15.77:0/290869116 shutdown_connections 2024-09-10T09:56:38.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:38.617+0000 7f4186000640 1 -- 172.21.15.77:0/290869116 wait complete. 2024-09-10T09:56:38.917 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:56:38 smithi077 ceph-mon[30319]: pgmap v435: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-09-10T09:56:38.918 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:56:38 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/290869116' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:56:39.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:56:38 smithi120 ceph-mon[34770]: pgmap v435: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-09-10T09:56:39.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:56:38 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/290869116' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:56:39.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:56:38 smithi134 ceph-mon[35674]: pgmap v435: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-09-10T09:56:39.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:56:38 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/290869116' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:56:41.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:56:40 smithi120 ceph-mon[34770]: pgmap v436: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 12 op/s 2024-09-10T09:56:41.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:56:40 smithi077 ceph-mon[30319]: pgmap v436: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 12 op/s 2024-09-10T09:56:41.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:56:40 smithi134 ceph-mon[35674]: pgmap v436: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 12 op/s 2024-09-10T09:56:42.334 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T09:56:42.826 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24563 quiesce db --set-id c4ed4a5a --release --await' 2024-09-10T09:56:42.826 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24563 quiesce db --set-id c4ed4a5a --release --await 2024-09-10T09:56:42.950 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:56:43.096 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:56:42 smithi077 ceph-mon[30319]: pgmap v437: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-09-10T09:56:43.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:56:42 smithi134 ceph-mon[35674]: pgmap v437: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-09-10T09:56:43.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:56:42 smithi120 ceph-mon[34770]: pgmap v437: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-09-10T09:56:43.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.804+0000 7fe0c6a00640 1 -- 172.21.15.77:0/1997768834 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe0c0102d20 msgr2=0x7fe0c0103100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:43.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.804+0000 7fe0c6a00640 1 --2- 172.21.15.77:0/1997768834 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe0c0102d20 0x7fe0c0103100 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fe0b0009900 tx=0x7fe0b002a480 comp rx=0 tx=0).stop 2024-09-10T09:56:43.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.805+0000 7fe0c6a00640 1 -- 172.21.15.77:0/1997768834 shutdown_connections 2024-09-10T09:56:43.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.805+0000 7fe0c6a00640 1 --2- 172.21.15.77:0/1997768834 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe0c010c150 0x7fe0c010e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:43.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.805+0000 7fe0c6a00640 1 --2- 172.21.15.77:0/1997768834 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe0c0103640 0x7fe0c010bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:43.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.805+0000 7fe0c6a00640 1 --2- 172.21.15.77:0/1997768834 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe0c0102d20 0x7fe0c0103100 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:43.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.805+0000 7fe0c6a00640 1 -- 172.21.15.77:0/1997768834 >> 172.21.15.77:0/1997768834 conn(0x7fe0c00fc780 msgr2=0x7fe0c00feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:43.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.805+0000 7fe0c6a00640 1 -- 172.21.15.77:0/1997768834 shutdown_connections 2024-09-10T09:56:43.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.805+0000 7fe0c6a00640 1 -- 172.21.15.77:0/1997768834 wait complete. 2024-09-10T09:56:43.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.806+0000 7fe0c6a00640 1 Processor -- start 2024-09-10T09:56:43.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.806+0000 7fe0c6a00640 1 -- start start 2024-09-10T09:56:43.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.807+0000 7fe0c6a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe0c0102d20 0x7fe0c01a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:43.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.808+0000 7fe0c6a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe0c0103640 0x7fe0c01a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:43.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.808+0000 7fe0c6a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe0c010c150 0x7fe0c010ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:43.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.808+0000 7fe0c6a00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c0111f50 con 0x7fe0c0102d20 2024-09-10T09:56:43.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.808+0000 7fe0c6a00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c0111dd0 con 0x7fe0c0103640 2024-09-10T09:56:43.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.808+0000 7fe0c6a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c01120d0 con 0x7fe0c010c150 2024-09-10T09:56:43.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.808+0000 7fe0bf400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe0c0103640 0x7fe0c01a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:43.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.808+0000 7fe0bf400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe0c0103640 0x7fe0c01a4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:42764/0 (socket says 172.21.15.77:42764) 2024-09-10T09:56:43.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.809+0000 7fe0bf400640 1 -- 172.21.15.77:0/2900473259 learned_addr learned my addr 172.21.15.77:0/2900473259 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:43.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.809+0000 7fe0c4a00640 1 --2- 172.21.15.77:0/2900473259 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe0c010c150 0x7fe0c010ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:43.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.809+0000 7fe0bfe00640 1 --2- 172.21.15.77:0/2900473259 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe0c0102d20 0x7fe0c01a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:43.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.809+0000 7fe0bf400640 1 -- 172.21.15.77:0/2900473259 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe0c010c150 msgr2=0x7fe0c010ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:43.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.809+0000 7fe0bf400640 1 --2- 172.21.15.77:0/2900473259 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe0c010c150 0x7fe0c010ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:43.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.809+0000 7fe0bf400640 1 -- 172.21.15.77:0/2900473259 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe0c0102d20 msgr2=0x7fe0c01a4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:43.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.809+0000 7fe0bf400640 1 --2- 172.21.15.77:0/2900473259 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe0c0102d20 0x7fe0c01a4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:43.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.809+0000 7fe0bf400640 1 -- 172.21.15.77:0/2900473259 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0c010b5b0 con 0x7fe0c0103640 2024-09-10T09:56:43.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.809+0000 7fe0bfe00640 1 --2- 172.21.15.77:0/2900473259 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe0c0102d20 0x7fe0c01a4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:56:43.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.810+0000 7fe0bf400640 1 --2- 172.21.15.77:0/2900473259 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe0c0103640 0x7fe0c01a4d50 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac00e9f0 tx=0x7fe0ac00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:43.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.810+0000 7fe0bcc00640 1 -- 172.21.15.77:0/2900473259 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0ac00cab0 con 0x7fe0c0103640 2024-09-10T09:56:43.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.810+0000 7fe0c6a00640 1 -- 172.21.15.77:0/2900473259 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0c010b8a0 con 0x7fe0c0103640 2024-09-10T09:56:43.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.811+0000 7fe0c6a00640 1 -- 172.21.15.77:0/2900473259 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0c01ab740 con 0x7fe0c0103640 2024-09-10T09:56:43.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.811+0000 7fe0bcc00640 1 -- 172.21.15.77:0/2900473259 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0ac00cc50 con 0x7fe0c0103640 2024-09-10T09:56:43.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.812+0000 7fe0bcc00640 1 -- 172.21.15.77:0/2900473259 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0ac0187e0 con 0x7fe0c0103640 2024-09-10T09:56:43.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.815+0000 7fe0c9639740 1 Processor -- start 2024-09-10T09:56:43.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.816+0000 7fe0c9639740 1 -- start start 2024-09-10T09:56:43.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.816+0000 7fe0c9639740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560940564c70 0x560940565050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:43.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.816+0000 7fe0c9639740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560940565590 0x56094056f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:43.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.817+0000 7fe0c9639740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56094056fe50 0x560940572240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:43.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.817+0000 7fe0c9639740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5609404cb3e0 con 0x560940564c70 2024-09-10T09:56:43.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.817+0000 7fe0c9639740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x560940472d80 con 0x560940565590 2024-09-10T09:56:43.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.817+0000 7fe0c9639740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5609404c73a0 con 0x56094056fe50 2024-09-10T09:56:43.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.817+0000 7fe0bcc00640 1 -- 172.21.15.77:0/2900473259 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe0ac018980 con 0x7fe0c0103640 2024-09-10T09:56:43.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.818+0000 7fe0c4a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560940564c70 0x560940565050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:43.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.818+0000 7fe0bfe00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560940565590 0x56094056f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:43.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.818+0000 7fe0c4a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560940564c70 0x560940565050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:55428/0 (socket says 172.21.15.77:55428) 2024-09-10T09:56:43.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.818+0000 7fe0c4a00640 1 -- 172.21.15.77:0/1665991050 learned_addr learned my addr 172.21.15.77:0/1665991050 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:43.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.818+0000 7fe0bf400640 1 --2- 172.21.15.77:0/1665991050 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56094056fe50 0x560940572240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:43.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.819+0000 7fe0bf400640 1 -- 172.21.15.77:0/1665991050 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560940565590 msgr2=0x56094056f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:43.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.819+0000 7fe0bf400640 1 --2- 172.21.15.77:0/1665991050 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560940565590 0x56094056f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:43.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.819+0000 7fe0bf400640 1 -- 172.21.15.77:0/1665991050 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560940564c70 msgr2=0x560940565050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:43.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.819+0000 7fe0bf400640 1 --2- 172.21.15.77:0/1665991050 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560940564c70 0x560940565050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:43.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.819+0000 7fe0bf400640 1 -- 172.21.15.77:0/1665991050 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560940491ae0 con 0x56094056fe50 2024-09-10T09:56:43.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.819+0000 7fe0bf400640 1 --2- 172.21.15.77:0/1665991050 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56094056fe50 0x560940572240 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac0cccb0 tx=0x7fe0ac0d7670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4691178424faa723 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:43.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.819+0000 7fe0b5200640 1 -- 172.21.15.77:0/1665991050 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0ac0de070 con 0x56094056fe50 2024-09-10T09:56:43.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.820+0000 7fe0b5200640 1 -- 172.21.15.77:0/1665991050 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0ac0d17b0 con 0x56094056fe50 2024-09-10T09:56:43.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.820+0000 7fe0bcc00640 1 --2- 172.21.15.77:0/2900473259 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fe0a40a2c00 0x7fe0a40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:43.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.820+0000 7fe0bcc00640 1 -- 172.21.15.77:0/2900473259 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fe0ac0cf660 con 0x7fe0c0103640 2024-09-10T09:56:43.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.820+0000 7fe0c9639740 1 -- 172.21.15.77:0/1665991050 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56094056fe50 msgr2=0x560940572240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:43.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.820+0000 7fe0c9639740 1 --2- 172.21.15.77:0/1665991050 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56094056fe50 0x560940572240 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac0cccb0 tx=0x7fe0ac0d7670 comp rx=0 tx=0).stop 2024-09-10T09:56:43.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.821+0000 7fe0c9639740 1 -- 172.21.15.77:0/1665991050 shutdown_connections 2024-09-10T09:56:43.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.821+0000 7fe0c9639740 1 --2- 172.21.15.77:0/1665991050 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56094056fe50 0x560940572240 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:43.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.821+0000 7fe0c9639740 1 --2- 172.21.15.77:0/1665991050 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560940565590 0x56094056f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:43.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.821+0000 7fe0c9639740 1 --2- 172.21.15.77:0/1665991050 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560940564c70 0x560940565050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:43.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.822+0000 7fe0c9639740 1 -- 172.21.15.77:0/1665991050 >> 172.21.15.77:0/1665991050 conn(0x560940561200 msgr2=0x5609405615e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:43.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.822+0000 7fe0c9639740 1 -- 172.21.15.77:0/1665991050 shutdown_connections 2024-09-10T09:56:43.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.822+0000 7fe0c4a00640 1 --2- 172.21.15.77:0/2900473259 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fe0a40a2c00 0x7fe0a40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:43.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.822+0000 7fe0c9639740 1 -- 172.21.15.77:0/1665991050 wait complete. 2024-09-10T09:56:43.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.822+0000 7fe0c9639740 1 -- start start 2024-09-10T09:56:43.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.823+0000 7fe0c9639740 1 Processor -- start 2024-09-10T09:56:43.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.823+0000 7fe0c4a00640 1 --2- 172.21.15.77:0/2900473259 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fe0a40a2c00 0x7fe0a40a50c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe0b8005fd0 tx=0x7fe0b8009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:43.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.824+0000 7fe0c9639740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560940564c70 0x56094056f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:43.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.824+0000 7fe0c9639740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560940565590 0x56094056cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:43.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.824+0000 7fe0c9639740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56094056fe50 0x56094056d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:43.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.824+0000 7fe0bf400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560940565590 0x56094056cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:43.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.824+0000 7fe0bf400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560940565590 0x56094056cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:42788/0 (socket says 172.21.15.77:42788) 2024-09-10T09:56:43.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.824+0000 7fe0c4a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56094056fe50 0x56094056d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:43.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.825+0000 7fe0c4a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56094056fe50 0x56094056d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:55434/0 (socket says 172.21.15.77:55434) 2024-09-10T09:56:43.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.825+0000 7fe0bf400640 1 -- 172.21.15.77:0/3431852963 learned_addr learned my addr 172.21.15.77:0/3431852963 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:43.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.825+0000 7fe0c9639740 1 -- 172.21.15.77:0/3431852963 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560940572ac0 con 0x56094056fe50 2024-09-10T09:56:43.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.825+0000 7fe0c9639740 1 -- 172.21.15.77:0/3431852963 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5609404860f0 con 0x560940565590 2024-09-10T09:56:43.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.825+0000 7fe0c9639740 1 -- 172.21.15.77:0/3431852963 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560940572c40 con 0x560940564c70 2024-09-10T09:56:43.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.825+0000 7fe0bfe00640 1 --2- 172.21.15.77:0/3431852963 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560940564c70 0x56094056f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:43.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.825+0000 7fe0c9639740 10 client.0 _ll_register_callbacks cb 0x56094046dbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:56:43.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.827+0000 7fe0bf400640 1 -- 172.21.15.77:0/3431852963 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560940564c70 msgr2=0x56094056f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:43.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.827+0000 7fe0bf400640 1 --2- 172.21.15.77:0/3431852963 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560940564c70 0x56094056f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:43.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.827+0000 7fe0bf400640 1 -- 172.21.15.77:0/3431852963 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56094056fe50 msgr2=0x56094056d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:43.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.827+0000 7fe0bf400640 1 --2- 172.21.15.77:0/3431852963 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56094056fe50 0x56094056d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:43.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.827+0000 7fe0bf400640 1 -- 172.21.15.77:0/3431852963 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56094023c3c0 con 0x560940565590 2024-09-10T09:56:43.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.827+0000 7fe0bfe00640 1 --2- 172.21.15.77:0/3431852963 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560940564c70 0x56094056f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:56:43.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.828+0000 7fe0c4a00640 1 --2- 172.21.15.77:0/3431852963 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56094056fe50 0x56094056d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:56:43.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.828+0000 7fe0bf400640 1 --2- 172.21.15.77:0/3431852963 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560940565590 0x56094056cb50 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac097ed0 tx=0x7fe0ac0da7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33fa013e74e8102e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:43.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.828+0000 7fe0a9600640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T09:56:43.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.829+0000 7fe0a9600640 1 -- 172.21.15.77:0/3431852963 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0ac0de070 con 0x560940565590 2024-09-10T09:56:43.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.829+0000 7fe0a9600640 1 -- 172.21.15.77:0/3431852963 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fe0ac0dae00 con 0x560940565590 2024-09-10T09:56:43.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.829+0000 7fe0c9639740 1 -- 172.21.15.77:0/3431852963 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0ac0d17b0 con 0x560940565590 2024-09-10T09:56:43.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.829+0000 7fe0a9600640 1 -- 172.21.15.77:0/3431852963 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe0ac097d10 con 0x560940565590 2024-09-10T09:56:43.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.829+0000 7fe0c9639740 10 client.25775 fetch_fsmap learned FSMap version 29 2024-09-10T09:56:43.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.829+0000 7fe0c9639740 1 -- 172.21.15.77:0/3431852963 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56094057e990 con 0x560940565590 2024-09-10T09:56:43.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.830+0000 7fe0a9600640 1 -- 172.21.15.77:0/3431852963 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0ac0d17b0 con 0x560940565590 2024-09-10T09:56:43.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.830+0000 7fe0a9600640 1 -- 172.21.15.77:0/3431852963 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fe0ac0cf2d0 con 0x560940565590 2024-09-10T09:56:43.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.830+0000 7fe0c9639740 10 client.25775 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T09:56:43.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.830+0000 7fe0c9639740 10 client.25775 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-10T09:56:43.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.830+0000 7fe0c9639740 20 client.25775 populate_metadata read hostname 'smithi077' 2024-09-10T09:56:43.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.830+0000 7fe0c9639740 1 --2- 172.21.15.77:0/3431852963 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x56094057fce0 0x5609405820d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:43.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.830+0000 7fe0c9639740 4 client.25775 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:56:43.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.830+0000 7fe0c9639740 1 -- 172.21.15.77:0/3431852963 --> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5609404d8950 con 0x56094057fce0 2024-09-10T09:56:43.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.831+0000 7fe0bfe00640 1 --2- 172.21.15.77:0/3431852963 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x56094057fce0 0x5609405820d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:43.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.832+0000 7fe0bfe00640 1 --2- 172.21.15.77:0/3431852963 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x56094057fce0 0x5609405820d0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a79538d1cd39a27 server_cookie=28b5f087deadcc5f in_seq=0 out_seq=0 2024-09-10T09:56:43.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.832+0000 7fe0a9600640 10 client.25775 ms_handle_connect on v2:172.21.15.134:6834/3293933699 2024-09-10T09:56:43.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.833+0000 7fe0a9600640 1 -- 172.21.15.77:0/3431852963 <== mds.4 v2:172.21.15.134:6834/3293933699 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5609404d8950 con 0x56094057fce0 2024-09-10T09:56:43.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.833+0000 7fe0a9600640 10 client.25775 handle_command_reply: tid=0 2024-09-10T09:56:43.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.833+0000 7fe0c9639740 1 client.25775 shutdown 2024-09-10T09:56:43.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.834+0000 7fe0c9639740 1 -- 172.21.15.77:0/3431852963 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560940565590 msgr2=0x56094056cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:43.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.834+0000 7fe0c9639740 1 --2- 172.21.15.77:0/3431852963 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560940565590 0x56094056cb50 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac097ed0 tx=0x7fe0ac0da7a0 comp rx=0 tx=0).stop 2024-09-10T09:56:43.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.834+0000 7fe0c9639740 1 -- 172.21.15.77:0/3431852963 shutdown_connections 2024-09-10T09:56:43.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.834+0000 7fe0c9639740 1 --2- 172.21.15.77:0/3431852963 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x56094057fce0 0x5609405820d0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:43.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.834+0000 7fe0c9639740 1 --2- 172.21.15.77:0/3431852963 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56094056fe50 0x56094056d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:43.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.834+0000 7fe0c9639740 1 --2- 172.21.15.77:0/3431852963 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560940565590 0x56094056cb50 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:43.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.834+0000 7fe0c9639740 1 --2- 172.21.15.77:0/3431852963 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560940564c70 0x56094056f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:43.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.834+0000 7fe0c9639740 1 -- 172.21.15.77:0/3431852963 >> 172.21.15.77:0/3431852963 conn(0x560940561200 msgr2=0x560940566f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:43.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.835+0000 7fe0a9600640 0 client.25775 ms_handle_reset on v2:172.21.15.134:6834/3293933699 2024-09-10T09:56:43.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.835+0000 7fe0c9639740 1 -- 172.21.15.77:0/3431852963 shutdown_connections 2024-09-10T09:56:43.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.835+0000 7fe0c9639740 1 -- 172.21.15.77:0/3431852963 wait complete. 2024-09-10T09:56:43.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.835+0000 7fe0c9639740 20 client.25775 trim_cache size 0 max 16384 2024-09-10T09:56:43.853 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:56:43.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.855+0000 7fe0c9639740 1 Processor -- start 2024-09-10T09:56:43.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.855+0000 7fe0c9639740 1 -- start start 2024-09-10T09:56:43.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.855+0000 7fe0c9639740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560940564c70 0x56094056c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:43.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.855+0000 7fe0c9639740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560940565590 0x56094056c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:43.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.855+0000 7fe0c9639740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56094056fe50 0x560940569df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:43.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.855+0000 7fe0c9639740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe0a000fd60 con 0x560940564c70 2024-09-10T09:56:43.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.855+0000 7fe0c9639740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe0a00118a0 con 0x560940565590 2024-09-10T09:56:43.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.855+0000 7fe0c9639740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe0a0010430 con 0x56094056fe50 2024-09-10T09:56:43.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.856+0000 7fe0bf400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560940565590 0x56094056c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:43.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.856+0000 7fe0bf400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560940565590 0x56094056c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:42794/0 (socket says 172.21.15.77:42794) 2024-09-10T09:56:43.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.856+0000 7fe0bf400640 1 -- 172.21.15.77:0/644164597 learned_addr learned my addr 172.21.15.77:0/644164597 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:43.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.857+0000 7fe0c4a00640 1 --2- 172.21.15.77:0/644164597 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56094056fe50 0x560940569df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:43.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.857+0000 7fe0bfe00640 1 --2- 172.21.15.77:0/644164597 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560940564c70 0x56094056c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:43.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.857+0000 7fe0bf400640 1 -- 172.21.15.77:0/644164597 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56094056fe50 msgr2=0x560940569df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:43.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.857+0000 7fe0bf400640 1 --2- 172.21.15.77:0/644164597 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56094056fe50 0x560940569df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:43.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.857+0000 7fe0bf400640 1 -- 172.21.15.77:0/644164597 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560940564c70 msgr2=0x56094056c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:43.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.857+0000 7fe0bf400640 1 --2- 172.21.15.77:0/644164597 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560940564c70 0x56094056c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:43.860 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.857+0000 7fe0bf400640 1 -- 172.21.15.77:0/644164597 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56094056d650 con 0x560940565590 2024-09-10T09:56:43.860 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.858+0000 7fe0bf400640 1 --2- 172.21.15.77:0/644164597 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560940565590 0x56094056c970 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac099d20 tx=0x7fe0ac0d7670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=453f856e85a56b12 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:43.860 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.858+0000 7fe0abe00640 1 -- 172.21.15.77:0/644164597 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0ac0de070 con 0x560940565590 2024-09-10T09:56:43.860 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.859+0000 7fe0abe00640 1 -- 172.21.15.77:0/644164597 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0ac0da490 con 0x560940565590 2024-09-10T09:56:43.860 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.859+0000 7fe0abe00640 1 -- 172.21.15.77:0/644164597 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0ac09b3d0 con 0x560940565590 2024-09-10T09:56:43.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.859+0000 7fe0c9639740 1 -- 172.21.15.77:0/644164597 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560940565590 msgr2=0x56094056c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:43.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.859+0000 7fe0c9639740 1 --2- 172.21.15.77:0/644164597 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560940565590 0x56094056c970 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac099d20 tx=0x7fe0ac0d7670 comp rx=0 tx=0).stop 2024-09-10T09:56:43.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.860+0000 7fe0c9639740 1 -- 172.21.15.77:0/644164597 shutdown_connections 2024-09-10T09:56:43.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.860+0000 7fe0c9639740 1 --2- 172.21.15.77:0/644164597 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56094056fe50 0x560940569df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:43.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.860+0000 7fe0c9639740 1 --2- 172.21.15.77:0/644164597 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560940565590 0x56094056c970 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:43.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.860+0000 7fe0c9639740 1 --2- 172.21.15.77:0/644164597 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560940564c70 0x56094056c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:43.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.860+0000 7fe0c9639740 1 -- 172.21.15.77:0/644164597 >> 172.21.15.77:0/644164597 conn(0x560940561200 msgr2=0x5609405615e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:43.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.860+0000 7fe0c9639740 1 -- 172.21.15.77:0/644164597 shutdown_connections 2024-09-10T09:56:43.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.860+0000 7fe0c9639740 1 -- 172.21.15.77:0/644164597 wait complete. 2024-09-10T09:56:43.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.860+0000 7fe0c9639740 1 -- start start 2024-09-10T09:56:43.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.861+0000 7fe0c9639740 1 Processor -- start 2024-09-10T09:56:43.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.861+0000 7fe0c9639740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560940564c70 0x560940569b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:43.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.862+0000 7fe0c9639740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560940565590 0x56094056a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:43.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.862+0000 7fe0c9639740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56094056fe50 0x56094056a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:43.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.862+0000 7fe0c9639740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe0a00111d0 con 0x56094056fe50 2024-09-10T09:56:43.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.862+0000 7fe0c9639740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe0a0010b00 con 0x560940565590 2024-09-10T09:56:43.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.862+0000 7fe0c9639740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe0a000dd40 con 0x560940564c70 2024-09-10T09:56:43.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.862+0000 7fe0c9639740 10 client.0 _ll_register_callbacks cb 0x7fe0b8009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:56:43.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.862+0000 7fe0bfe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560940564c70 0x560940569b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:43.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.862+0000 7fe0bfe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560940564c70 0x560940569b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:59422/0 (socket says 172.21.15.77:59422) 2024-09-10T09:56:43.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.862+0000 7fe0c4a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56094056fe50 0x56094056a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:43.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.863+0000 7fe0bf400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560940565590 0x56094056a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:43.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.863+0000 7fe0bfe00640 1 -- 172.21.15.77:0/133940361 learned_addr learned my addr 172.21.15.77:0/133940361 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:43.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.863+0000 7fe0bf400640 1 -- 172.21.15.77:0/133940361 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560940564c70 msgr2=0x560940569b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:43.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.863+0000 7fe0bf400640 1 --2- 172.21.15.77:0/133940361 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560940564c70 0x560940569b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:43.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.863+0000 7fe0bf400640 1 -- 172.21.15.77:0/133940361 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56094056fe50 msgr2=0x56094056a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:43.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.863+0000 7fe0bf400640 1 --2- 172.21.15.77:0/133940361 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56094056fe50 0x56094056a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:43.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.864+0000 7fe0bf400640 1 -- 172.21.15.77:0/133940361 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609405653c0 con 0x560940565590 2024-09-10T09:56:43.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.864+0000 7fe0c4a00640 1 --2- 172.21.15.77:0/133940361 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56094056fe50 0x56094056a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:56:43.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.864+0000 7fe0bf400640 1 --2- 172.21.15.77:0/133940361 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560940565590 0x56094056a0c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x560940560da0 tx=0x7fe0ac09ff70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af0ea76619e8563d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:43.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.864+0000 7fe0bfe00640 1 --2- 172.21.15.77:0/133940361 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560940564c70 0x560940569b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:56:43.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.865+0000 7fe0a9600640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T09:56:43.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.865+0000 7fe0a9600640 1 -- 172.21.15.77:0/133940361 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0ac0b8060 con 0x560940565590 2024-09-10T09:56:43.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.865+0000 7fe0a9600640 1 -- 172.21.15.77:0/133940361 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fe0ac0d5040 con 0x560940565590 2024-09-10T09:56:43.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.865+0000 7fe0c9639740 1 -- 172.21.15.77:0/133940361 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0ac0da490 con 0x560940565590 2024-09-10T09:56:43.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.865+0000 7fe0a9600640 1 -- 172.21.15.77:0/133940361 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe0ac0de070 con 0x560940565590 2024-09-10T09:56:43.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.865+0000 7fe0c9639740 10 client.25781 fetch_fsmap learned FSMap version 29 2024-09-10T09:56:43.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.865+0000 7fe0c9639740 1 -- 172.21.15.77:0/133940361 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5609405653c0 con 0x560940565590 2024-09-10T09:56:43.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.865+0000 7fe0a9600640 1 -- 172.21.15.77:0/133940361 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0ac0da490 con 0x560940565590 2024-09-10T09:56:43.870 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.868+0000 7fe0a9600640 1 -- 172.21.15.77:0/133940361 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fe0ac0ba300 con 0x560940565590 2024-09-10T09:56:43.870 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.868+0000 7fe0c9639740 10 client.25781 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T09:56:43.871 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.868+0000 7fe0c9639740 10 client.25781 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-10T09:56:43.871 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.868+0000 7fe0c9639740 20 client.25781 populate_metadata read hostname 'smithi077' 2024-09-10T09:56:43.871 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.868+0000 7fe0c9639740 1 --2- 172.21.15.77:0/133940361 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x5609405eaa70 0x5609405ece90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:43.871 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.868+0000 7fe0c9639740 4 client.25781 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:56:43.871 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.868+0000 7fe0c9639740 1 -- 172.21.15.77:0/133940361 --> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5609404d2e60 con 0x5609405eaa70 2024-09-10T09:56:43.871 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.869+0000 7fe0bfe00640 1 --2- 172.21.15.77:0/133940361 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x5609405eaa70 0x5609405ece90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:43.871 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.870+0000 7fe0a9600640 10 client.25781 ms_handle_connect on v2:172.21.15.134:6834/3293933699 2024-09-10T09:56:43.871 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.870+0000 7fe0bfe00640 1 --2- 172.21.15.77:0/133940361 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x5609405eaa70 0x5609405ece90 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8952eb52a06595c0 server_cookie=bf3f8c58cd32ee17 in_seq=0 out_seq=0 2024-09-10T09:56:43.872 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.870+0000 7fe0a9600640 1 -- 172.21.15.77:0/133940361 <== mds.4 v2:172.21.15.134:6834/3293933699 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5609404d2e60 con 0x5609405eaa70 2024-09-10T09:56:43.872 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.870+0000 7fe0a9600640 10 client.25781 handle_command_reply: tid=0 2024-09-10T09:56:43.872 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.870+0000 7fe0c9639740 1 client.25781 shutdown 2024-09-10T09:56:43.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.870+0000 7fe0c9639740 1 -- 172.21.15.77:0/133940361 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560940565590 msgr2=0x56094056a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:43.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.870+0000 7fe0c9639740 1 --2- 172.21.15.77:0/133940361 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560940565590 0x56094056a0c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x560940560da0 tx=0x7fe0ac09ff70 comp rx=0 tx=0).stop 2024-09-10T09:56:43.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.871+0000 7fe0c9639740 1 -- 172.21.15.77:0/133940361 shutdown_connections 2024-09-10T09:56:43.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.871+0000 7fe0c9639740 1 --2- 172.21.15.77:0/133940361 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x5609405eaa70 0x5609405ece90 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:43.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.871+0000 7fe0c9639740 1 --2- 172.21.15.77:0/133940361 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56094056fe50 0x56094056a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:43.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.871+0000 7fe0c9639740 1 --2- 172.21.15.77:0/133940361 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560940565590 0x56094056a0c0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:43.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.871+0000 7fe0c9639740 1 --2- 172.21.15.77:0/133940361 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560940564c70 0x560940569b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:43.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.871+0000 7fe0c9639740 1 -- 172.21.15.77:0/133940361 >> 172.21.15.77:0/133940361 conn(0x560940561200 msgr2=0x56094056c380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:43.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.871+0000 7fe0a9600640 0 client.25781 ms_handle_reset on v2:172.21.15.134:6834/3293933699 2024-09-10T09:56:43.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.872+0000 7fe0c9639740 1 -- 172.21.15.77:0/133940361 shutdown_connections 2024-09-10T09:56:43.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.872+0000 7fe0c9639740 1 -- 172.21.15.77:0/133940361 wait complete. 2024-09-10T09:56:43.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.872+0000 7fe0c9639740 20 client.25781 trim_cache size 0 max 16384 2024-09-10T09:56:43.874 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-10T09:56:43.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.873+0000 7fe0c6a00640 1 -- 172.21.15.77:0/2900473259 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fe0a40a2c00 msgr2=0x7fe0a40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:43.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.873+0000 7fe0c6a00640 1 --2- 172.21.15.77:0/2900473259 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fe0a40a2c00 0x7fe0a40a50c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe0b8005fd0 tx=0x7fe0b8009500 comp rx=0 tx=0).stop 2024-09-10T09:56:43.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.873+0000 7fe0c6a00640 1 -- 172.21.15.77:0/2900473259 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe0c0103640 msgr2=0x7fe0c01a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:43.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.873+0000 7fe0c6a00640 1 --2- 172.21.15.77:0/2900473259 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe0c0103640 0x7fe0c01a4d50 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac00e9f0 tx=0x7fe0ac00edf0 comp rx=0 tx=0).stop 2024-09-10T09:56:43.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.873+0000 7fe0c6a00640 1 -- 172.21.15.77:0/2900473259 shutdown_connections 2024-09-10T09:56:43.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.873+0000 7fe0c6a00640 1 --2- 172.21.15.77:0/2900473259 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fe0a40a2c00 0x7fe0a40a50c0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:43.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.873+0000 7fe0c6a00640 1 --2- 172.21.15.77:0/2900473259 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe0c010c150 0x7fe0c010ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:43.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.873+0000 7fe0c6a00640 1 --2- 172.21.15.77:0/2900473259 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe0c0103640 0x7fe0c01a4d50 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:43.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.873+0000 7fe0c6a00640 1 --2- 172.21.15.77:0/2900473259 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe0c0102d20 0x7fe0c01a4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:43.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.873+0000 7fe0c6a00640 1 -- 172.21.15.77:0/2900473259 >> 172.21.15.77:0/2900473259 conn(0x7fe0c00fc780 msgr2=0x7fe0c01008b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:43.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.874+0000 7fe0c6a00640 1 -- 172.21.15.77:0/2900473259 shutdown_connections 2024-09-10T09:56:43.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:43.874+0000 7fe0c6a00640 1 -- 172.21.15.77:0/2900473259 wait complete. 2024-09-10T09:56:44.860 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T09:56:44.964 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:56:44 smithi077 ceph-mon[30319]: pgmap v438: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-09-10T09:56:45.183 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:56:44 smithi134 ceph-mon[35674]: pgmap v438: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-09-10T09:56:45.333 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.330+0000 7efc07200640 1 -- 172.21.15.77:0/1081435378 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efc08108f60 msgr2=0x7efc08110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:45.333 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.330+0000 7efc07200640 1 --2- 172.21.15.77:0/1081435378 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efc08108f60 0x7efc08110ae0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7efc00009ca0 tx=0x7efc0002a8b0 comp rx=0 tx=0).stop 2024-09-10T09:56:45.333 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.331+0000 7efc07200640 1 -- 172.21.15.77:0/1081435378 shutdown_connections 2024-09-10T09:56:45.333 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.331+0000 7efc07200640 1 --2- 172.21.15.77:0/1081435378 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efc08108f60 0x7efc08110ae0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:45.333 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.331+0000 7efc07200640 1 --2- 172.21.15.77:0/1081435378 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efc081047e0 0x7efc08108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:45.333 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.331+0000 7efc07200640 1 --2- 172.21.15.77:0/1081435378 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc08103e30 0x7efc08104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:45.333 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.331+0000 7efc07200640 1 -- 172.21.15.77:0/1081435378 >> 172.21.15.77:0/1081435378 conn(0x7efc080ff910 msgr2=0x7efc08101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:45.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.333+0000 7efc07200640 1 -- 172.21.15.77:0/1081435378 shutdown_connections 2024-09-10T09:56:45.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.333+0000 7efc07200640 1 -- 172.21.15.77:0/1081435378 wait complete. 2024-09-10T09:56:45.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.334+0000 7efc07200640 1 Processor -- start 2024-09-10T09:56:45.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.334+0000 7efc07200640 1 -- start start 2024-09-10T09:56:45.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.335+0000 7efc07200640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efc08103e30 0x7efc0819a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:45.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.335+0000 7efc07200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc081047e0 0x7efc0819a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:45.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.335+0000 7efc07200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efc08108f60 0x7efc0819ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:45.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.335+0000 7efc07200640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7efc08114030 con 0x7efc08108f60 2024-09-10T09:56:45.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.335+0000 7efc07200640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7efc08113eb0 con 0x7efc08103e30 2024-09-10T09:56:45.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.335+0000 7efc07200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7efc081141b0 con 0x7efc081047e0 2024-09-10T09:56:45.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.335+0000 7efc05e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efc08103e30 0x7efc0819a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:45.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.335+0000 7efc05e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efc08103e30 0x7efc0819a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:42820/0 (socket says 172.21.15.77:42820) 2024-09-10T09:56:45.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.335+0000 7efc05e00640 1 -- 172.21.15.77:0/1965246055 learned_addr learned my addr 172.21.15.77:0/1965246055 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:45.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.335+0000 7efc05400640 1 --2- 172.21.15.77:0/1965246055 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc081047e0 0x7efc0819a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:45.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.336+0000 7efc05e00640 1 -- 172.21.15.77:0/1965246055 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc081047e0 msgr2=0x7efc0819a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:45.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.336+0000 7efc05e00640 1 --2- 172.21.15.77:0/1965246055 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc081047e0 0x7efc0819a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:45.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.336+0000 7efc05e00640 1 -- 172.21.15.77:0/1965246055 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efc08108f60 msgr2=0x7efc0819ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:45.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.336+0000 7efc05e00640 1 --2- 172.21.15.77:0/1965246055 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efc08108f60 0x7efc0819ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:45.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.336+0000 7efc05e00640 1 -- 172.21.15.77:0/1965246055 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc0819f440 con 0x7efc08103e30 2024-09-10T09:56:45.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.336+0000 7efc05e00640 1 --2- 172.21.15.77:0/1965246055 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efc08103e30 0x7efc0819a490 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7efbf000d950 tx=0x7efbf000dd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:45.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.336+0000 7efbfea00640 1 -- 172.21.15.77:0/1965246055 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf0017070 con 0x7efc08103e30 2024-09-10T09:56:45.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.336+0000 7efc07200640 1 -- 172.21.15.77:0/1965246055 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc0819f730 con 0x7efc08103e30 2024-09-10T09:56:45.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.336+0000 7efbfea00640 1 -- 172.21.15.77:0/1965246055 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efbf000de80 con 0x7efc08103e30 2024-09-10T09:56:45.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.336+0000 7efc07200640 1 -- 172.21.15.77:0/1965246055 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc081a7010 con 0x7efc08103e30 2024-09-10T09:56:45.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.337+0000 7efbfea00640 1 -- 172.21.15.77:0/1965246055 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf000f620 con 0x7efc08103e30 2024-09-10T09:56:45.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.340+0000 7efc0f849740 1 Processor -- start 2024-09-10T09:56:45.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.341+0000 7efc0f849740 1 -- start start 2024-09-10T09:56:45.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.341+0000 7efc0f849740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633f655d520 0x5633f655d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:45.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.341+0000 7efc0f849740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5633f655de40 0x5633f65681c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:45.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.341+0000 7efc0f849740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5633f6568700 0x5633f656aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:45.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.341+0000 7efc0f849740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5633f64c3650 con 0x5633f655de40 2024-09-10T09:56:45.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.341+0000 7efc0f849740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5633f646af10 con 0x5633f6568700 2024-09-10T09:56:45.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.341+0000 7efc0f849740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5633f64bf610 con 0x5633f655d520 2024-09-10T09:56:45.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.341+0000 7efc06800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633f655d520 0x5633f655d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:45.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.341+0000 7efc06800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633f655d520 0x5633f655d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:59456/0 (socket says 172.21.15.77:59456) 2024-09-10T09:56:45.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.341+0000 7efc06800640 1 -- 172.21.15.77:0/312554471 learned_addr learned my addr 172.21.15.77:0/312554471 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:45.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.342+0000 7efbfea00640 1 -- 172.21.15.77:0/1965246055 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7efbf000f7c0 con 0x7efc08103e30 2024-09-10T09:56:45.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.342+0000 7efc05400640 1 --2- 172.21.15.77:0/312554471 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5633f6568700 0x5633f656aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:45.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.342+0000 7efc06800640 1 -- 172.21.15.77:0/312554471 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5633f6568700 msgr2=0x5633f656aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:45.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.342+0000 7efc06800640 1 --2- 172.21.15.77:0/312554471 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5633f6568700 0x5633f656aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:45.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.342+0000 7efc06800640 1 -- 172.21.15.77:0/312554471 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5633f655de40 msgr2=0x5633f65681c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:45.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.342+0000 7efc06800640 1 --2- 172.21.15.77:0/312554471 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5633f655de40 0x5633f65681c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:45.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.342+0000 7efc06800640 1 -- 172.21.15.77:0/312554471 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633f61d2580 con 0x5633f655d520 2024-09-10T09:56:45.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.342+0000 7efc05400640 1 --2- 172.21.15.77:0/312554471 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5633f6568700 0x5633f656aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:56:45.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.343+0000 7efc06800640 1 --2- 172.21.15.77:0/312554471 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633f655d520 0x5633f655d900 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7efc00009a60 tx=0x7efc000303a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f7781897fd592a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:45.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.344+0000 7efbfea00640 1 --2- 172.21.15.77:0/1965246055 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7efbec0a2c00 0x7efbec0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:45.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.344+0000 7efbf7000640 1 -- 172.21.15.77:0/312554471 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc00030740 con 0x5633f655d520 2024-09-10T09:56:45.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.344+0000 7efbf7000640 1 -- 172.21.15.77:0/312554471 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc00047070 con 0x5633f655d520 2024-09-10T09:56:45.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.344+0000 7efbf7000640 1 -- 172.21.15.77:0/312554471 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc000043c0 con 0x5633f655d520 2024-09-10T09:56:45.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.344+0000 7efbfea00640 1 -- 172.21.15.77:0/1965246055 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7efbf00cefe0 con 0x7efc08103e30 2024-09-10T09:56:45.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.345+0000 7efc05e00640 1 --2- 172.21.15.77:0/1965246055 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7efbec0a2c00 0x7efbec0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:45.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.345+0000 7efc0f849740 1 -- 172.21.15.77:0/312554471 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633f655d520 msgr2=0x5633f655d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:45.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.345+0000 7efc0f849740 1 --2- 172.21.15.77:0/312554471 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633f655d520 0x5633f655d900 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7efc00009a60 tx=0x7efc000303a0 comp rx=0 tx=0).stop 2024-09-10T09:56:45.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.346+0000 7efc0f849740 1 -- 172.21.15.77:0/312554471 shutdown_connections 2024-09-10T09:56:45.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.346+0000 7efc0f849740 1 --2- 172.21.15.77:0/312554471 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5633f6568700 0x5633f656aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:45.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.346+0000 7efc0f849740 1 --2- 172.21.15.77:0/312554471 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5633f655de40 0x5633f65681c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:45.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.346+0000 7efc0f849740 1 --2- 172.21.15.77:0/312554471 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633f655d520 0x5633f655d900 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:45.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.346+0000 7efc0f849740 1 -- 172.21.15.77:0/312554471 >> 172.21.15.77:0/312554471 conn(0x5633f6559ab0 msgr2=0x5633f6559e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:45.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.346+0000 7efc0f849740 1 -- 172.21.15.77:0/312554471 shutdown_connections 2024-09-10T09:56:45.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.346+0000 7efc0f849740 1 -- 172.21.15.77:0/312554471 wait complete. 2024-09-10T09:56:45.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.346+0000 7efc05e00640 1 --2- 172.21.15.77:0/1965246055 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7efbec0a2c00 0x7efbec0a50c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7efbf0097850 tx=0x7efbf0005bb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:45.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.347+0000 7efc0f849740 1 -- start start 2024-09-10T09:56:45.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.347+0000 7efc0f849740 1 Processor -- start 2024-09-10T09:56:45.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.348+0000 7efc0f849740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5633f655d520 0x5633f65655d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:45.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.348+0000 7efc0f849740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5633f655de40 0x5633f6565b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:45.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.348+0000 7efc0f849740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633f6568700 0x5633f6566050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:45.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.348+0000 7efc0f849740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5633f656b2d0 con 0x5633f655de40 2024-09-10T09:56:45.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.348+0000 7efc0f849740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5633f647e350 con 0x5633f655d520 2024-09-10T09:56:45.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.348+0000 7efc0f849740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5633f656b450 con 0x5633f6568700 2024-09-10T09:56:45.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.348+0000 7efc0f849740 10 client.0 _ll_register_callbacks cb 0x5633f64a1290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:56:45.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.349+0000 7efc06800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5633f655d520 0x5633f65655d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:45.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.349+0000 7efc06800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633f6568700 0x5633f6566050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:45.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.349+0000 7efc06800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633f6568700 0x5633f6566050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:59464/0 (socket says 172.21.15.77:59464) 2024-09-10T09:56:45.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.349+0000 7efc06800640 1 -- 172.21.15.77:0/1122542804 learned_addr learned my addr 172.21.15.77:0/1122542804 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:45.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.349+0000 7efc05400640 1 --2- 172.21.15.77:0/1122542804 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5633f655de40 0x5633f6565b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:45.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.349+0000 7efc06800640 1 -- 172.21.15.77:0/1122542804 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5633f655d520 msgr2=0x5633f65655d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:45.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.349+0000 7efc06800640 1 --2- 172.21.15.77:0/1122542804 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5633f655d520 0x5633f65655d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:45.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.349+0000 7efc06800640 1 -- 172.21.15.77:0/1122542804 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5633f655de40 msgr2=0x5633f6565b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:45.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.349+0000 7efc06800640 1 --2- 172.21.15.77:0/1122542804 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5633f655de40 0x5633f6565b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:45.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.350+0000 7efc06800640 1 -- 172.21.15.77:0/1122542804 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633f629c960 con 0x5633f6568700 2024-09-10T09:56:45.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.350+0000 7efc05400640 1 --2- 172.21.15.77:0/1122542804 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5633f655de40 0x5633f6565b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:56:45.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.350+0000 7efc06800640 1 --2- 172.21.15.77:0/1122542804 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633f6568700 0x5633f6566050 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7efc00004520 tx=0x7efc00004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=609d1a03d1bafadb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:45.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.350+0000 7efbeb400640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-10T09:56:45.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.351+0000 7efbeb400640 1 -- 172.21.15.77:0/1122542804 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc00050030 con 0x5633f6568700 2024-09-10T09:56:45.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.351+0000 7efbeb400640 1 -- 172.21.15.77:0/1122542804 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7efc00004560 con 0x5633f6568700 2024-09-10T09:56:45.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.351+0000 7efc0f849740 1 -- 172.21.15.77:0/1122542804 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc00047070 con 0x5633f6568700 2024-09-10T09:56:45.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.352+0000 7efbeb400640 1 -- 172.21.15.77:0/1122542804 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7efc00065050 con 0x5633f6568700 2024-09-10T09:56:45.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.352+0000 7efc0f849740 10 client.25753 fetch_fsmap learned FSMap version 29 2024-09-10T09:56:45.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.352+0000 7efc0f849740 1 -- 172.21.15.77:0/1122542804 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5633f656b2d0 con 0x5633f6568700 2024-09-10T09:56:45.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.353+0000 7efbeb400640 1 -- 172.21.15.77:0/1122542804 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc00047070 con 0x5633f6568700 2024-09-10T09:56:45.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.353+0000 7efbeb400640 1 -- 172.21.15.77:0/1122542804 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7efc0004c8b0 con 0x5633f6568700 2024-09-10T09:56:45.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.353+0000 7efc0f849740 10 client.25753 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T09:56:45.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.353+0000 7efc0f849740 10 client.25753 resolve_mds: validated gid 24563 aka daemon mds.b 2024-09-10T09:56:45.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.353+0000 7efc0f849740 20 client.25753 populate_metadata read hostname 'smithi077' 2024-09-10T09:56:45.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.353+0000 7efc0f849740 1 --2- 172.21.15.77:0/1122542804 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x5633f65780e0 0x5633f657a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:45.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.353+0000 7efc0f849740 4 client.25753 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:56:45.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.353+0000 7efc0f849740 1 -- 172.21.15.77:0/1122542804 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5633f64d0bc0 con 0x5633f65780e0 2024-09-10T09:56:45.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.354+0000 7efc05400640 1 --2- 172.21.15.77:0/1122542804 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x5633f65780e0 0x5633f657a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:45.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.354+0000 7efc05400640 1 --2- 172.21.15.77:0/1122542804 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x5633f65780e0 0x5633f657a4d0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a085c4885a3f1928 server_cookie=738a30099c887e01 in_seq=0 out_seq=0 2024-09-10T09:56:45.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.354+0000 7efbeb400640 10 client.25753 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T09:56:45.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.355+0000 7efbeb400640 1 -- 172.21.15.77:0/1122542804 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5633f64d0bc0 con 0x5633f65780e0 2024-09-10T09:56:45.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.355+0000 7efbeb400640 10 client.25753 handle_command_reply: tid=0 2024-09-10T09:56:45.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.355+0000 7efc0f849740 1 client.25753 shutdown 2024-09-10T09:56:45.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.356+0000 7efc0f849740 1 -- 172.21.15.77:0/1122542804 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633f6568700 msgr2=0x5633f6566050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:45.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.356+0000 7efc0f849740 1 --2- 172.21.15.77:0/1122542804 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633f6568700 0x5633f6566050 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7efc00004520 tx=0x7efc00004a40 comp rx=0 tx=0).stop 2024-09-10T09:56:45.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.356+0000 7efc0f849740 1 -- 172.21.15.77:0/1122542804 shutdown_connections 2024-09-10T09:56:45.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.356+0000 7efc0f849740 1 --2- 172.21.15.77:0/1122542804 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x5633f65780e0 0x5633f657a4d0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:45.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.356+0000 7efc0f849740 1 --2- 172.21.15.77:0/1122542804 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633f6568700 0x5633f6566050 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:45.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.356+0000 7efc0f849740 1 --2- 172.21.15.77:0/1122542804 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5633f655de40 0x5633f6565b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:45.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.356+0000 7efc0f849740 1 --2- 172.21.15.77:0/1122542804 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5633f655d520 0x5633f65655d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:45.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.356+0000 7efc0f849740 1 -- 172.21.15.77:0/1122542804 >> 172.21.15.77:0/1122542804 conn(0x5633f6559ab0 msgr2=0x5633f6558cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:45.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.356+0000 7efbeb400640 0 client.25753 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T09:56:45.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.356+0000 7efc0f849740 1 -- 172.21.15.77:0/1122542804 shutdown_connections 2024-09-10T09:56:45.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.357+0000 7efc0f849740 1 -- 172.21.15.77:0/1122542804 wait complete. 2024-09-10T09:56:45.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.357+0000 7efc0f849740 20 client.25753 trim_cache size 0 max 16384 2024-09-10T09:56:45.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.375+0000 7efc0f849740 1 Processor -- start 2024-09-10T09:56:45.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.375+0000 7efc0f849740 1 -- start start 2024-09-10T09:56:45.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.375+0000 7efc0f849740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5633f655d520 0x5633f6567430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:45.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.375+0000 7efc0f849740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633f655de40 0x5633f657fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:45.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.375+0000 7efc0f849740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5633f6568700 0x5633f6580100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:45.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.375+0000 7efc0f849740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7efbe400fdf0 con 0x5633f655d520 2024-09-10T09:56:45.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.375+0000 7efc0f849740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7efbe4011930 con 0x5633f6568700 2024-09-10T09:56:45.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.375+0000 7efc0f849740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7efbe40104c0 con 0x5633f655de40 2024-09-10T09:56:45.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.376+0000 7efc06800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5633f655d520 0x5633f6567430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:45.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.376+0000 7efc06800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5633f655d520 0x5633f6567430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:55524/0 (socket says 172.21.15.77:55524) 2024-09-10T09:56:45.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.376+0000 7efc06800640 1 -- 172.21.15.77:0/3827841270 learned_addr learned my addr 172.21.15.77:0/3827841270 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:45.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.376+0000 7efc06800640 1 --2- 172.21.15.77:0/3827841270 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5633f6568700 0x5633f6580100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:45.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.376+0000 7efc06800640 1 -- 172.21.15.77:0/3827841270 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633f655de40 msgr2=0x5633f657fbc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T09:56:45.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.376+0000 7efc06800640 1 --2- 172.21.15.77:0/3827841270 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633f655de40 0x5633f657fbc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:45.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.376+0000 7efc06800640 1 -- 172.21.15.77:0/3827841270 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5633f655d520 msgr2=0x5633f6567430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:45.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.376+0000 7efc06800640 1 --2- 172.21.15.77:0/3827841270 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5633f655d520 0x5633f6567430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:45.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.376+0000 7efc06800640 1 -- 172.21.15.77:0/3827841270 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633f65598e0 con 0x5633f6568700 2024-09-10T09:56:45.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.377+0000 7efc06800640 1 --2- 172.21.15.77:0/3827841270 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5633f6568700 0x5633f6580100 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7efc00009ba0 tx=0x7efc00030f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c4c35b14d82cf7e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:45.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.377+0000 7efbf6600640 1 -- 172.21.15.77:0/3827841270 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc00050040 con 0x5633f6568700 2024-09-10T09:56:45.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.377+0000 7efbf6600640 1 -- 172.21.15.77:0/3827841270 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc00047070 con 0x5633f6568700 2024-09-10T09:56:45.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.377+0000 7efbf6600640 1 -- 172.21.15.77:0/3827841270 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc00054050 con 0x5633f6568700 2024-09-10T09:56:45.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.378+0000 7efc0f849740 1 -- 172.21.15.77:0/3827841270 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5633f6568700 msgr2=0x5633f6580100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:45.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.378+0000 7efc0f849740 1 --2- 172.21.15.77:0/3827841270 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5633f6568700 0x5633f6580100 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7efc00009ba0 tx=0x7efc00030f70 comp rx=0 tx=0).stop 2024-09-10T09:56:45.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.378+0000 7efc0f849740 1 -- 172.21.15.77:0/3827841270 shutdown_connections 2024-09-10T09:56:45.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.378+0000 7efc0f849740 1 --2- 172.21.15.77:0/3827841270 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5633f6568700 0x5633f6580100 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:45.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.378+0000 7efc0f849740 1 --2- 172.21.15.77:0/3827841270 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633f655de40 0x5633f657fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:45.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.378+0000 7efc0f849740 1 --2- 172.21.15.77:0/3827841270 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5633f655d520 0x5633f6567430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:45.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.378+0000 7efc0f849740 1 -- 172.21.15.77:0/3827841270 >> 172.21.15.77:0/3827841270 conn(0x5633f6559ab0 msgr2=0x5633f6566d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:45.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.378+0000 7efc0f849740 1 -- 172.21.15.77:0/3827841270 shutdown_connections 2024-09-10T09:56:45.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.379+0000 7efc0f849740 1 -- 172.21.15.77:0/3827841270 wait complete. 2024-09-10T09:56:45.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.379+0000 7efc0f849740 1 -- start start 2024-09-10T09:56:45.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.379+0000 7efc0f849740 1 Processor -- start 2024-09-10T09:56:45.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.380+0000 7efc0f849740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5633f655d520 0x5633f6570b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:45.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.380+0000 7efc0f849740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5633f655de40 0x5633f656cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:45.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.380+0000 7efc0f849740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633f6568700 0x5633f656d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:45.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.380+0000 7efc0f849740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7efbe4011260 con 0x5633f655de40 2024-09-10T09:56:45.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.380+0000 7efc0f849740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7efbe4010b90 con 0x5633f655d520 2024-09-10T09:56:45.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.380+0000 7efc0f849740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7efbe400ddd0 con 0x5633f6568700 2024-09-10T09:56:45.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.380+0000 7efc0f849740 10 client.0 _ll_register_callbacks cb 0x7efc00009990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:56:45.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.380+0000 7efc05400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5633f655de40 0x5633f656cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:45.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.380+0000 7efc05400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5633f655de40 0x5633f656cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:55530/0 (socket says 172.21.15.77:55530) 2024-09-10T09:56:45.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.380+0000 7efc05400640 1 -- 172.21.15.77:0/3379585125 learned_addr learned my addr 172.21.15.77:0/3379585125 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:45.384 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.380+0000 7efc06800640 1 --2- 172.21.15.77:0/3379585125 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5633f655d520 0x5633f6570b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:45.384 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.381+0000 7efc06800640 1 --2- 172.21.15.77:0/3379585125 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633f6568700 0x5633f656d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:45.384 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.381+0000 7efc05400640 1 -- 172.21.15.77:0/3379585125 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633f6568700 msgr2=0x5633f656d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:45.384 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.381+0000 7efc05400640 1 --2- 172.21.15.77:0/3379585125 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633f6568700 0x5633f656d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:45.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.381+0000 7efc05400640 1 -- 172.21.15.77:0/3379585125 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5633f655d520 msgr2=0x5633f6570b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:45.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.381+0000 7efc05400640 1 --2- 172.21.15.77:0/3379585125 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5633f655d520 0x5633f6570b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:45.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.381+0000 7efc05400640 1 -- 172.21.15.77:0/3379585125 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633f655ab60 con 0x5633f655de40 2024-09-10T09:56:45.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.381+0000 7efc05400640 1 --2- 172.21.15.77:0/3379585125 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5633f655de40 0x5633f656cea0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x5633f6564e60 tx=0x7efbf8016800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=204ea101931ef35f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:45.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.381+0000 7efbeb400640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-10T09:56:45.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.382+0000 7efbeb400640 1 -- 172.21.15.77:0/3379585125 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf801c520 con 0x5633f655de40 2024-09-10T09:56:45.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.382+0000 7efbeb400640 1 -- 172.21.15.77:0/3379585125 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7efbf801c6c0 con 0x5633f655de40 2024-09-10T09:56:45.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.382+0000 7efc0f849740 1 -- 172.21.15.77:0/3379585125 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc00047070 con 0x5633f655de40 2024-09-10T09:56:45.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.382+0000 7efbeb400640 1 -- 172.21.15.77:0/3379585125 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7efbf802c050 con 0x5633f655de40 2024-09-10T09:56:45.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.383+0000 7efc0f849740 10 client.15723 fetch_fsmap learned FSMap version 29 2024-09-10T09:56:45.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.383+0000 7efc0f849740 1 -- 172.21.15.77:0/3379585125 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5633f65e3700 con 0x5633f655de40 2024-09-10T09:56:45.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.383+0000 7efbeb400640 1 -- 172.21.15.77:0/3379585125 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc00047070 con 0x5633f655de40 2024-09-10T09:56:45.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.383+0000 7efbeb400640 1 -- 172.21.15.77:0/3379585125 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7efbf8024c10 con 0x5633f655de40 2024-09-10T09:56:45.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.384+0000 7efc0f849740 10 client.15723 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T09:56:45.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.384+0000 7efc0f849740 10 client.15723 resolve_mds: validated gid 24563 aka daemon mds.b 2024-09-10T09:56:45.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.384+0000 7efc0f849740 20 client.15723 populate_metadata read hostname 'smithi077' 2024-09-10T09:56:45.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.384+0000 7efc0f849740 1 --2- 172.21.15.77:0/3379585125 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x5633f65e3f50 0x5633f65e6370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:45.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.384+0000 7efc0f849740 4 client.15723 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "c4ed4a5a", "release": true, "await": true}] 2024-09-10T09:56:45.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.384+0000 7efc0f849740 1 -- 172.21.15.77:0/3379585125 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "quiesce db", "set_id": "c4ed4a5a", "release": true, "await": true}) -- 0x5633f64cb0d0 con 0x5633f65e3f50 2024-09-10T09:56:45.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.385+0000 7efc06800640 1 --2- 172.21.15.77:0/3379585125 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x5633f65e3f50 0x5633f65e6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:45.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.387+0000 7efc06800640 1 --2- 172.21.15.77:0/3379585125 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x5633f65e3f50 0x5633f65e6370 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25c17738e9ef4b81 server_cookie=784c39c8479c1119 in_seq=0 out_seq=0 2024-09-10T09:56:45.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.387+0000 7efbeb400640 10 client.15723 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T09:56:45.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.388+0000 7efbeb400640 1 -- 172.21.15.77:0/3379585125 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x5633f64cb0d0 con 0x5633f65e3f50 2024-09-10T09:56:45.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.388+0000 7efbeb400640 10 client.15723 handle_command_reply: tid=0 2024-09-10T09:56:45.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.388+0000 7efc0f849740 1 client.15723 shutdown 2024-09-10T09:56:45.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.389+0000 7efc0f849740 1 -- 172.21.15.77:0/3379585125 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5633f655de40 msgr2=0x5633f656cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:45.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.389+0000 7efc0f849740 1 --2- 172.21.15.77:0/3379585125 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5633f655de40 0x5633f656cea0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x5633f6564e60 tx=0x7efbf8016800 comp rx=0 tx=0).stop 2024-09-10T09:56:45.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.389+0000 7efc0f849740 1 -- 172.21.15.77:0/3379585125 shutdown_connections 2024-09-10T09:56:45.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.389+0000 7efc0f849740 1 --2- 172.21.15.77:0/3379585125 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x5633f65e3f50 0x5633f65e6370 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:45.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.389+0000 7efc0f849740 1 --2- 172.21.15.77:0/3379585125 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633f6568700 0x5633f656d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:45.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.389+0000 7efc0f849740 1 --2- 172.21.15.77:0/3379585125 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5633f655de40 0x5633f656cea0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:45.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.389+0000 7efc0f849740 1 --2- 172.21.15.77:0/3379585125 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5633f655d520 0x5633f6570b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:45.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.389+0000 7efc0f849740 1 -- 172.21.15.77:0/3379585125 >> 172.21.15.77:0/3379585125 conn(0x5633f6559ab0 msgr2=0x5633f6558cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:45.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.389+0000 7efbeb400640 0 client.15723 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T09:56:45.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.390+0000 7efc0f849740 1 -- 172.21.15.77:0/3379585125 shutdown_connections 2024-09-10T09:56:45.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.390+0000 7efc0f849740 1 -- 172.21.15.77:0/3379585125 wait complete. 2024-09-10T09:56:45.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.391+0000 7efc0f849740 20 client.15723 trim_cache size 0 max 16384 2024-09-10T09:56:45.393 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-10T09:56:45.393 INFO:teuthology.orchestra.run.smithi077.stdout: "epoch": 29, 2024-09-10T09:56:45.393 INFO:teuthology.orchestra.run.smithi077.stdout: "leader": 24563, 2024-09-10T09:56:45.394 INFO:teuthology.orchestra.run.smithi077.stdout: "set_version": 4, 2024-09-10T09:56:45.394 INFO:teuthology.orchestra.run.smithi077.stdout: "sets": { 2024-09-10T09:56:45.394 INFO:teuthology.orchestra.run.smithi077.stdout: "c4ed4a5a": { 2024-09-10T09:56:45.394 INFO:teuthology.orchestra.run.smithi077.stdout: "version": 4, 2024-09-10T09:56:45.394 INFO:teuthology.orchestra.run.smithi077.stdout: "age_ref": 0.0, 2024-09-10T09:56:45.394 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-10T09:56:45.394 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RELEASED", 2024-09-10T09:56:45.394 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-10T09:56:45.394 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T09:56:45.394 INFO:teuthology.orchestra.run.smithi077.stdout: "timeout": 90.0, 2024-09-10T09:56:45.394 INFO:teuthology.orchestra.run.smithi077.stdout: "expiration": 150.0, 2024-09-10T09:56:45.394 INFO:teuthology.orchestra.run.smithi077.stdout: "members": { 2024-09-10T09:56:45.394 INFO:teuthology.orchestra.run.smithi077.stdout: "file:/": { 2024-09-10T09:56:45.394 INFO:teuthology.orchestra.run.smithi077.stdout: "excluded": false, 2024-09-10T09:56:45.394 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-10T09:56:45.395 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RELEASED", 2024-09-10T09:56:45.395 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-10T09:56:45.395 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T09:56:45.395 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T09:56:45.395 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T09:56:45.395 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T09:56:45.395 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T09:56:45.395 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-10T09:56:45.395 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.392+0000 7efc07200640 1 -- 172.21.15.77:0/1965246055 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7efbec0a2c00 msgr2=0x7efbec0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:45.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.392+0000 7efc07200640 1 --2- 172.21.15.77:0/1965246055 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7efbec0a2c00 0x7efbec0a50c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7efbf0097850 tx=0x7efbf0005bb0 comp rx=0 tx=0).stop 2024-09-10T09:56:45.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.392+0000 7efc07200640 1 -- 172.21.15.77:0/1965246055 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efc08103e30 msgr2=0x7efc0819a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:45.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.392+0000 7efc07200640 1 --2- 172.21.15.77:0/1965246055 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efc08103e30 0x7efc0819a490 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7efbf000d950 tx=0x7efbf000dd50 comp rx=0 tx=0).stop 2024-09-10T09:56:45.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.392+0000 7efc07200640 1 -- 172.21.15.77:0/1965246055 shutdown_connections 2024-09-10T09:56:45.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.392+0000 7efc07200640 1 --2- 172.21.15.77:0/1965246055 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7efbec0a2c00 0x7efbec0a50c0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:45.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.392+0000 7efc07200640 1 --2- 172.21.15.77:0/1965246055 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efc08108f60 0x7efc0819ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:45.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.392+0000 7efc07200640 1 --2- 172.21.15.77:0/1965246055 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc081047e0 0x7efc0819a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:45.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.392+0000 7efc07200640 1 --2- 172.21.15.77:0/1965246055 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efc08103e30 0x7efc0819a490 secure :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7efbf000d950 tx=0x7efbf000dd50 comp rx=0 tx=0).stop 2024-09-10T09:56:45.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.393+0000 7efc07200640 1 -- 172.21.15.77:0/1965246055 >> 172.21.15.77:0/1965246055 conn(0x7efc080ff910 msgr2=0x7efc08100be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:45.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.395+0000 7efc07200640 1 -- 172.21.15.77:0/1965246055 shutdown_connections 2024-09-10T09:56:45.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:45.395+0000 7efc07200640 1 -- 172.21.15.77:0/1965246055 wait complete. 2024-09-10T09:56:45.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:56:44 smithi120 ceph-mon[34770]: pgmap v438: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-09-10T09:56:45.625 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:56:46.216 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'c4ed4a5a', total seconds elapsed: 40.9 2024-09-10T09:56:46.216 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-09-10T09:56:46.216 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:56:46.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.753+0000 7ff671200640 1 -- 172.21.15.77:0/870776861 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff66c104010 msgr2=0x7ff66c1043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:46.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.753+0000 7ff671200640 1 --2- 172.21.15.77:0/870776861 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff66c104010 0x7ff66c1043f0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7ff65c009900 tx=0x7ff65c02a3f0 comp rx=0 tx=0).stop 2024-09-10T09:56:46.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.753+0000 7ff671200640 1 -- 172.21.15.77:0/870776861 shutdown_connections 2024-09-10T09:56:46.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.753+0000 7ff671200640 1 --2- 172.21.15.77:0/870776861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff66c109140 0x7ff66c110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:46.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.753+0000 7ff671200640 1 --2- 172.21.15.77:0/870776861 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff66c1049c0 0x7ff66c108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:46.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.753+0000 7ff671200640 1 --2- 172.21.15.77:0/870776861 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff66c104010 0x7ff66c1043f0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:46.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.753+0000 7ff671200640 1 -- 172.21.15.77:0/870776861 >> 172.21.15.77:0/870776861 conn(0x7ff66c0ffa50 msgr2=0x7ff66c101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:46.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.753+0000 7ff671200640 1 -- 172.21.15.77:0/870776861 shutdown_connections 2024-09-10T09:56:46.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.753+0000 7ff671200640 1 -- 172.21.15.77:0/870776861 wait complete. 2024-09-10T09:56:46.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.754+0000 7ff671200640 1 Processor -- start 2024-09-10T09:56:46.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.754+0000 7ff671200640 1 -- start start 2024-09-10T09:56:46.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.754+0000 7ff671200640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff66c104010 0x7ff66c19a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:46.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.754+0000 7ff671200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff66c1049c0 0x7ff66c19ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:46.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.754+0000 7ff671200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff66c109140 0x7ff66c19ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:46.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.754+0000 7ff671200640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff66c1141c0 con 0x7ff66c109140 2024-09-10T09:56:46.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.754+0000 7ff671200640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff66c114040 con 0x7ff66c104010 2024-09-10T09:56:46.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.754+0000 7ff671200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff66c114340 con 0x7ff66c1049c0 2024-09-10T09:56:46.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.755+0000 7ff66a800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff66c109140 0x7ff66c19ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:46.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.755+0000 7ff669e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff66c104010 0x7ff66c19a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:46.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.755+0000 7ff669400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff66c1049c0 0x7ff66c19ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:46.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.755+0000 7ff669e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff66c104010 0x7ff66c19a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:42864/0 (socket says 172.21.15.77:42864) 2024-09-10T09:56:46.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.755+0000 7ff66a800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff66c109140 0x7ff66c19ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:55548/0 (socket says 172.21.15.77:55548) 2024-09-10T09:56:46.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.755+0000 7ff669e00640 1 -- 172.21.15.77:0/653320621 learned_addr learned my addr 172.21.15.77:0/653320621 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:46.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.755+0000 7ff669400640 1 -- 172.21.15.77:0/653320621 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff66c104010 msgr2=0x7ff66c19a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:46.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.755+0000 7ff669400640 1 --2- 172.21.15.77:0/653320621 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff66c104010 0x7ff66c19a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:46.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.755+0000 7ff669400640 1 -- 172.21.15.77:0/653320621 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff66c109140 msgr2=0x7ff66c19ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:46.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.755+0000 7ff669400640 1 --2- 172.21.15.77:0/653320621 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff66c109140 0x7ff66c19ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:46.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.755+0000 7ff669400640 1 -- 172.21.15.77:0/653320621 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff66c19f5f0 con 0x7ff66c1049c0 2024-09-10T09:56:46.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.756+0000 7ff669400640 1 --2- 172.21.15.77:0/653320621 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff66c1049c0 0x7ff66c19ab80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff65800e9f0 tx=0x7ff65800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:46.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.756+0000 7ff662a00640 1 -- 172.21.15.77:0/653320621 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff65800ccc0 con 0x7ff66c1049c0 2024-09-10T09:56:46.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.756+0000 7ff671200640 1 -- 172.21.15.77:0/653320621 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff66c19f8e0 con 0x7ff66c1049c0 2024-09-10T09:56:46.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.756+0000 7ff662a00640 1 -- 172.21.15.77:0/653320621 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff658010910 con 0x7ff66c1049c0 2024-09-10T09:56:46.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.756+0000 7ff662a00640 1 -- 172.21.15.77:0/653320621 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff658018960 con 0x7ff66c1049c0 2024-09-10T09:56:46.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.756+0000 7ff671200640 1 -- 172.21.15.77:0/653320621 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff66c1a71c0 con 0x7ff66c1049c0 2024-09-10T09:56:46.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.760+0000 7ff673e4c740 1 Processor -- start 2024-09-10T09:56:46.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.760+0000 7ff673e4c740 1 -- start start 2024-09-10T09:56:46.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.761+0000 7ff673e4c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55962d1e1c70 0x55962d1e2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:46.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.761+0000 7ff673e4c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55962d1e2590 0x55962d1ec910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:46.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.761+0000 7ff673e4c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55962d1ece50 0x55962d1ef240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:46.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.761+0000 7ff673e4c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55962d1483e0 con 0x55962d1ece50 2024-09-10T09:56:46.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.761+0000 7ff673e4c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55962d0efd80 con 0x55962d1e2590 2024-09-10T09:56:46.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.761+0000 7ff673e4c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55962d1443a0 con 0x55962d1e1c70 2024-09-10T09:56:46.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.761+0000 7ff662a00640 1 -- 172.21.15.77:0/653320621 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff658010ac0 con 0x7ff66c1049c0 2024-09-10T09:56:46.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.761+0000 7ff669400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55962d1ece50 0x55962d1ef240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:46.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.761+0000 7ff66a800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55962d1e1c70 0x55962d1e2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:46.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.761+0000 7ff66a800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55962d1e1c70 0x55962d1e2050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:59510/0 (socket says 172.21.15.77:59510) 2024-09-10T09:56:46.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.761+0000 7ff66a800640 1 -- 172.21.15.77:0/4027909275 learned_addr learned my addr 172.21.15.77:0/4027909275 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:46.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.762+0000 7ff669e00640 1 --2- 172.21.15.77:0/4027909275 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55962d1e2590 0x55962d1ec910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:46.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.762+0000 7ff66a800640 1 -- 172.21.15.77:0/4027909275 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55962d1e2590 msgr2=0x55962d1ec910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:46.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.762+0000 7ff66a800640 1 --2- 172.21.15.77:0/4027909275 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55962d1e2590 0x55962d1ec910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:46.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.762+0000 7ff66a800640 1 -- 172.21.15.77:0/4027909275 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55962d1ece50 msgr2=0x55962d1ef240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:46.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.762+0000 7ff66a800640 1 --2- 172.21.15.77:0/4027909275 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55962d1ece50 0x55962d1ef240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:46.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.762+0000 7ff66a800640 1 -- 172.21.15.77:0/4027909275 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55962d10eae0 con 0x55962d1e1c70 2024-09-10T09:56:46.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.763+0000 7ff66a800640 1 --2- 172.21.15.77:0/4027909275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55962d1e1c70 0x55962d1e2050 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff664009f70 tx=0x7ff664010800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d144f79ac8a74b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:46.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.763+0000 7ff657400640 1 -- 172.21.15.77:0/4027909275 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff664016070 con 0x55962d1e1c70 2024-09-10T09:56:46.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.763+0000 7ff657400640 1 -- 172.21.15.77:0/4027909275 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff664004d20 con 0x55962d1e1c70 2024-09-10T09:56:46.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.763+0000 7ff657400640 1 -- 172.21.15.77:0/4027909275 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff664012950 con 0x55962d1e1c70 2024-09-10T09:56:46.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.763+0000 7ff662a00640 1 --2- 172.21.15.77:0/653320621 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ff6500a2c00 0x7ff6500a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:46.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.763+0000 7ff662a00640 1 -- 172.21.15.77:0/653320621 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7ff6580cf820 con 0x7ff66c1049c0 2024-09-10T09:56:46.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.764+0000 7ff669e00640 1 --2- 172.21.15.77:0/653320621 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ff6500a2c00 0x7ff6500a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:46.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.764+0000 7ff673e4c740 1 -- 172.21.15.77:0/4027909275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55962d1e1c70 msgr2=0x55962d1e2050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:46.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.764+0000 7ff673e4c740 1 --2- 172.21.15.77:0/4027909275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55962d1e1c70 0x55962d1e2050 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff664009f70 tx=0x7ff664010800 comp rx=0 tx=0).stop 2024-09-10T09:56:46.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.764+0000 7ff673e4c740 1 -- 172.21.15.77:0/4027909275 shutdown_connections 2024-09-10T09:56:46.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.764+0000 7ff673e4c740 1 --2- 172.21.15.77:0/4027909275 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55962d1ece50 0x55962d1ef240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:46.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.764+0000 7ff673e4c740 1 --2- 172.21.15.77:0/4027909275 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55962d1e2590 0x55962d1ec910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:46.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.764+0000 7ff673e4c740 1 --2- 172.21.15.77:0/4027909275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55962d1e1c70 0x55962d1e2050 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:46.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.764+0000 7ff673e4c740 1 -- 172.21.15.77:0/4027909275 >> 172.21.15.77:0/4027909275 conn(0x55962d1de200 msgr2=0x55962d1de5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:46.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.765+0000 7ff669e00640 1 --2- 172.21.15.77:0/653320621 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ff6500a2c00 0x7ff6500a50c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ff65c0096c0 tx=0x7ff65c0046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:46.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.765+0000 7ff673e4c740 1 -- 172.21.15.77:0/4027909275 shutdown_connections 2024-09-10T09:56:46.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.765+0000 7ff673e4c740 1 -- 172.21.15.77:0/4027909275 wait complete. 2024-09-10T09:56:46.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.765+0000 7ff673e4c740 1 -- start start 2024-09-10T09:56:46.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.766+0000 7ff673e4c740 1 Processor -- start 2024-09-10T09:56:46.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.766+0000 7ff673e4c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55962d1e1c70 0x55962d1e9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:46.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.766+0000 7ff673e4c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55962d1e2590 0x55962d1ea2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:46.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.766+0000 7ff673e4c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55962d1ece50 0x55962d1ea830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:46.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.766+0000 7ff673e4c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55962d1f8230 con 0x55962d1e2590 2024-09-10T09:56:46.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.766+0000 7ff673e4c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55962d1030f0 con 0x55962d1ece50 2024-09-10T09:56:46.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.766+0000 7ff673e4c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55962d1f83b0 con 0x55962d1e1c70 2024-09-10T09:56:46.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.766+0000 7ff673e4c740 10 client.0 _ll_register_callbacks cb 0x55962d0eabd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:56:46.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.767+0000 7ff669e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55962d1e1c70 0x55962d1e9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:46.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.767+0000 7ff669e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55962d1e1c70 0x55962d1e9db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:59520/0 (socket says 172.21.15.77:59520) 2024-09-10T09:56:46.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.767+0000 7ff669e00640 1 -- 172.21.15.77:0/2756241187 learned_addr learned my addr 172.21.15.77:0/2756241187 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:46.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.767+0000 7ff66a800640 1 --2- 172.21.15.77:0/2756241187 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55962d1ece50 0x55962d1ea830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:46.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.767+0000 7ff669400640 1 --2- 172.21.15.77:0/2756241187 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55962d1e2590 0x55962d1ea2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:46.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.767+0000 7ff66a800640 1 -- 172.21.15.77:0/2756241187 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55962d1e1c70 msgr2=0x55962d1e9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:46.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.767+0000 7ff66a800640 1 --2- 172.21.15.77:0/2756241187 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55962d1e1c70 0x55962d1e9db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:46.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.767+0000 7ff66a800640 1 -- 172.21.15.77:0/2756241187 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55962d1e2590 msgr2=0x55962d1ea2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:46.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.767+0000 7ff66a800640 1 --2- 172.21.15.77:0/2756241187 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55962d1e2590 0x55962d1ea2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:46.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.767+0000 7ff66a800640 1 -- 172.21.15.77:0/2756241187 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55962ceb93c0 con 0x55962d1ece50 2024-09-10T09:56:46.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.768+0000 7ff66a800640 1 --2- 172.21.15.77:0/2756241187 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55962d1ece50 0x55962d1ea830 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff664019000 tx=0x7ff664004c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1442265334bc2512 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:46.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.768+0000 7ff64be00640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T09:56:46.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.768+0000 7ff64be00640 1 -- 172.21.15.77:0/2756241187 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff664016070 con 0x55962d1ece50 2024-09-10T09:56:46.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.768+0000 7ff64be00640 1 -- 172.21.15.77:0/2756241187 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7ff6640050e0 con 0x55962d1ece50 2024-09-10T09:56:46.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.768+0000 7ff673e4c740 1 -- 172.21.15.77:0/2756241187 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff664004d20 con 0x55962d1ece50 2024-09-10T09:56:46.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.769+0000 7ff64be00640 1 -- 172.21.15.77:0/2756241187 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff664009b70 con 0x55962d1ece50 2024-09-10T09:56:46.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.769+0000 7ff673e4c740 10 client.25808 fetch_fsmap learned FSMap version 29 2024-09-10T09:56:46.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.769+0000 7ff673e4c740 1 -- 172.21.15.77:0/2756241187 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55962d204320 con 0x55962d1ece50 2024-09-10T09:56:46.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.769+0000 7ff64be00640 1 -- 172.21.15.77:0/2756241187 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff664004d20 con 0x55962d1ece50 2024-09-10T09:56:46.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.769+0000 7ff64be00640 1 -- 172.21.15.77:0/2756241187 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7ff66400f040 con 0x55962d1ece50 2024-09-10T09:56:46.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.769+0000 7ff673e4c740 10 client.25808 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T09:56:46.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.769+0000 7ff673e4c740 10 client.25808 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T09:56:46.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.769+0000 7ff673e4c740 20 client.25808 populate_metadata read hostname 'smithi077' 2024-09-10T09:56:46.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.770+0000 7ff673e4c740 1 --2- 172.21.15.77:0/2756241187 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x55962d205730 0x55962d207b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:46.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.770+0000 7ff673e4c740 4 client.25808 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:56:46.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.770+0000 7ff673e4c740 1 -- 172.21.15.77:0/2756241187 --> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55962d155950 con 0x55962d205730 2024-09-10T09:56:46.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.770+0000 7ff669e00640 1 --2- 172.21.15.77:0/2756241187 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x55962d205730 0x55962d207b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:46.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.771+0000 7ff669e00640 1 --2- 172.21.15.77:0/2756241187 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x55962d205730 0x55962d207b20 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=15758dc9db89b20e server_cookie=4c4d638d2851429f in_seq=0 out_seq=0 2024-09-10T09:56:46.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.771+0000 7ff64be00640 10 client.25808 ms_handle_connect on v2:172.21.15.134:6836/2461497091 2024-09-10T09:56:46.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.772+0000 7ff64be00640 1 -- 172.21.15.77:0/2756241187 <== mds.3 v2:172.21.15.134:6836/2461497091 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55962d155950 con 0x55962d205730 2024-09-10T09:56:46.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.772+0000 7ff64be00640 10 client.25808 handle_command_reply: tid=0 2024-09-10T09:56:46.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.772+0000 7ff673e4c740 1 client.25808 shutdown 2024-09-10T09:56:46.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.772+0000 7ff673e4c740 1 -- 172.21.15.77:0/2756241187 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55962d1ece50 msgr2=0x55962d1ea830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:46.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.772+0000 7ff673e4c740 1 --2- 172.21.15.77:0/2756241187 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55962d1ece50 0x55962d1ea830 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff664019000 tx=0x7ff664004c70 comp rx=0 tx=0).stop 2024-09-10T09:56:46.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.772+0000 7ff673e4c740 1 -- 172.21.15.77:0/2756241187 shutdown_connections 2024-09-10T09:56:46.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.772+0000 7ff673e4c740 1 --2- 172.21.15.77:0/2756241187 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55962d1ece50 0x55962d1ea830 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:46.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.772+0000 7ff673e4c740 1 --2- 172.21.15.77:0/2756241187 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x55962d205730 0x55962d207b20 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:46.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.772+0000 7ff673e4c740 1 --2- 172.21.15.77:0/2756241187 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55962d1e2590 0x55962d1ea2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:46.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.772+0000 7ff673e4c740 1 --2- 172.21.15.77:0/2756241187 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55962d1e1c70 0x55962d1e9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:46.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.772+0000 7ff673e4c740 1 -- 172.21.15.77:0/2756241187 >> 172.21.15.77:0/2756241187 conn(0x55962d1de200 msgr2=0x55962d1dd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:46.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.772+0000 7ff64be00640 0 client.25808 ms_handle_reset on v2:172.21.15.134:6836/2461497091 2024-09-10T09:56:46.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.772+0000 7ff673e4c740 1 -- 172.21.15.77:0/2756241187 shutdown_connections 2024-09-10T09:56:46.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.772+0000 7ff673e4c740 1 -- 172.21.15.77:0/2756241187 wait complete. 2024-09-10T09:56:46.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.773+0000 7ff673e4c740 20 client.25808 trim_cache size 0 max 16384 2024-09-10T09:56:46.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.787+0000 7ff673e4c740 1 Processor -- start 2024-09-10T09:56:46.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.787+0000 7ff673e4c740 1 -- start start 2024-09-10T09:56:46.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.788+0000 7ff673e4c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55962d1e1c70 0x55962d1e8680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:46.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.788+0000 7ff673e4c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55962d1e2590 0x55962d1ea680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:46.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.788+0000 7ff673e4c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55962d1ece50 0x55962d20d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:46.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.788+0000 7ff673e4c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff66c1acc40 con 0x55962d1e2590 2024-09-10T09:56:46.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.788+0000 7ff673e4c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff66c1ae780 con 0x55962d1e1c70 2024-09-10T09:56:46.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.788+0000 7ff673e4c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff66c1ad310 con 0x55962d1ece50 2024-09-10T09:56:46.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.788+0000 7ff669400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55962d1e2590 0x55962d1ea680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:46.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.788+0000 7ff669400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55962d1e2590 0x55962d1ea680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:55572/0 (socket says 172.21.15.77:55572) 2024-09-10T09:56:46.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.788+0000 7ff669400640 1 -- 172.21.15.77:0/3900423000 learned_addr learned my addr 172.21.15.77:0/3900423000 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:46.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.788+0000 7ff669e00640 1 --2- 172.21.15.77:0/3900423000 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55962d1e1c70 0x55962d1e8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:46.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.788+0000 7ff66a800640 1 --2- 172.21.15.77:0/3900423000 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55962d1ece50 0x55962d20d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:46.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.789+0000 7ff669e00640 1 -- 172.21.15.77:0/3900423000 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55962d1ece50 msgr2=0x55962d20d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:46.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.789+0000 7ff669e00640 1 --2- 172.21.15.77:0/3900423000 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55962d1ece50 0x55962d20d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:46.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.789+0000 7ff669e00640 1 -- 172.21.15.77:0/3900423000 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55962d1e2590 msgr2=0x55962d1ea680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:46.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.789+0000 7ff669e00640 1 --2- 172.21.15.77:0/3900423000 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55962d1e2590 0x55962d1ea680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:46.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.789+0000 7ff669e00640 1 -- 172.21.15.77:0/3900423000 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55962d1d9770 con 0x55962d1e1c70 2024-09-10T09:56:46.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.789+0000 7ff66a800640 1 --2- 172.21.15.77:0/3900423000 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55962d1ece50 0x55962d20d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:56:46.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.790+0000 7ff669e00640 1 --2- 172.21.15.77:0/3900423000 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55962d1e1c70 0x55962d1e8680 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x55962d208150 tx=0x7ff65c03f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc071461b97d7c83 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:46.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.790+0000 7ff656a00640 1 -- 172.21.15.77:0/3900423000 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff65c035040 con 0x55962d1e1c70 2024-09-10T09:56:46.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.790+0000 7ff656a00640 1 -- 172.21.15.77:0/3900423000 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff65c05a070 con 0x55962d1e1c70 2024-09-10T09:56:46.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.790+0000 7ff656a00640 1 -- 172.21.15.77:0/3900423000 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff65c054980 con 0x55962d1e1c70 2024-09-10T09:56:46.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.791+0000 7ff673e4c740 1 -- 172.21.15.77:0/3900423000 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55962d1e1c70 msgr2=0x55962d1e8680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:46.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.791+0000 7ff673e4c740 1 --2- 172.21.15.77:0/3900423000 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55962d1e1c70 0x55962d1e8680 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x55962d208150 tx=0x7ff65c03f000 comp rx=0 tx=0).stop 2024-09-10T09:56:46.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.791+0000 7ff673e4c740 1 -- 172.21.15.77:0/3900423000 shutdown_connections 2024-09-10T09:56:46.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.791+0000 7ff673e4c740 1 --2- 172.21.15.77:0/3900423000 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55962d1ece50 0x55962d20d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:46.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.791+0000 7ff673e4c740 1 --2- 172.21.15.77:0/3900423000 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55962d1e2590 0x55962d1ea680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:46.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.791+0000 7ff673e4c740 1 --2- 172.21.15.77:0/3900423000 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55962d1e1c70 0x55962d1e8680 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:46.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.791+0000 7ff673e4c740 1 -- 172.21.15.77:0/3900423000 >> 172.21.15.77:0/3900423000 conn(0x55962d1de200 msgr2=0x55962d1ebe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:46.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.791+0000 7ff673e4c740 1 -- 172.21.15.77:0/3900423000 shutdown_connections 2024-09-10T09:56:46.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.791+0000 7ff673e4c740 1 -- 172.21.15.77:0/3900423000 wait complete. 2024-09-10T09:56:46.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.792+0000 7ff673e4c740 1 -- start start 2024-09-10T09:56:46.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.792+0000 7ff673e4c740 1 Processor -- start 2024-09-10T09:56:46.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.793+0000 7ff673e4c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55962d1e1c70 0x55962d1fdd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:46.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.793+0000 7ff673e4c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55962d1e2590 0x55962d1f9080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:46.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.793+0000 7ff673e4c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55962d1ece50 0x55962d1f95c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:46.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.793+0000 7ff673e4c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff66c1ae0b0 con 0x55962d1ece50 2024-09-10T09:56:46.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.793+0000 7ff673e4c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff66c1ad9e0 con 0x55962d1e1c70 2024-09-10T09:56:46.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.793+0000 7ff673e4c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff66c1aac20 con 0x55962d1e2590 2024-09-10T09:56:46.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.793+0000 7ff673e4c740 10 client.0 _ll_register_callbacks cb 0x7ff6580d5400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:56:46.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.793+0000 7ff669e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55962d1e1c70 0x55962d1fdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:46.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.793+0000 7ff669e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55962d1e1c70 0x55962d1fdd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:42902/0 (socket says 172.21.15.77:42902) 2024-09-10T09:56:46.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.793+0000 7ff669e00640 1 -- 172.21.15.77:0/741101167 learned_addr learned my addr 172.21.15.77:0/741101167 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:46.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.793+0000 7ff669400640 1 --2- 172.21.15.77:0/741101167 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55962d1e2590 0x55962d1f9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:46.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.793+0000 7ff66a800640 1 --2- 172.21.15.77:0/741101167 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55962d1ece50 0x55962d1f95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:46.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.794+0000 7ff669e00640 1 -- 172.21.15.77:0/741101167 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55962d1e2590 msgr2=0x55962d1f9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:46.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.794+0000 7ff669e00640 1 --2- 172.21.15.77:0/741101167 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55962d1e2590 0x55962d1f9080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:46.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.794+0000 7ff669e00640 1 -- 172.21.15.77:0/741101167 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55962d1ece50 msgr2=0x55962d1f95c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:46.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.794+0000 7ff669e00640 1 --2- 172.21.15.77:0/741101167 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55962d1ece50 0x55962d1f95c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:46.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.794+0000 7ff669e00640 1 -- 172.21.15.77:0/741101167 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55962d1f9dc0 con 0x55962d1e1c70 2024-09-10T09:56:46.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.794+0000 7ff66a800640 1 --2- 172.21.15.77:0/741101167 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55962d1ece50 0x55962d1f95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:56:46.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.794+0000 7ff669e00640 1 --2- 172.21.15.77:0/741101167 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55962d1e1c70 0x55962d1fdd50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff65c057040 tx=0x7ff65c031f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b7efb56ed528a01 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:46.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.794+0000 7ff64be00640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T09:56:46.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.795+0000 7ff64be00640 1 -- 172.21.15.77:0/741101167 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff65c054a10 con 0x55962d1e1c70 2024-09-10T09:56:46.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.795+0000 7ff673e4c740 1 -- 172.21.15.77:0/741101167 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff65c05a070 con 0x55962d1e1c70 2024-09-10T09:56:46.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.795+0000 7ff64be00640 1 -- 172.21.15.77:0/741101167 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7ff65c054bb0 con 0x55962d1e1c70 2024-09-10T09:56:46.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.795+0000 7ff64be00640 1 -- 172.21.15.77:0/741101167 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff65c0035f0 con 0x55962d1e1c70 2024-09-10T09:56:46.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.795+0000 7ff673e4c740 10 client.25814 fetch_fsmap learned FSMap version 29 2024-09-10T09:56:46.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.795+0000 7ff673e4c740 1 -- 172.21.15.77:0/741101167 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55962d2701b0 con 0x55962d1e1c70 2024-09-10T09:56:46.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.795+0000 7ff64be00640 1 -- 172.21.15.77:0/741101167 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff65c05a070 con 0x55962d1e1c70 2024-09-10T09:56:46.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.796+0000 7ff64be00640 1 -- 172.21.15.77:0/741101167 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7ff65c03d850 con 0x55962d1e1c70 2024-09-10T09:56:46.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.796+0000 7ff673e4c740 10 client.25814 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T09:56:46.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.796+0000 7ff673e4c740 10 client.25814 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T09:56:46.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.796+0000 7ff673e4c740 20 client.25814 populate_metadata read hostname 'smithi077' 2024-09-10T09:56:46.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.796+0000 7ff673e4c740 1 --2- 172.21.15.77:0/741101167 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x55962d270870 0x55962d272c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:46.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.796+0000 7ff673e4c740 4 client.25814 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:56:46.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.796+0000 7ff673e4c740 1 -- 172.21.15.77:0/741101167 --> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55962d14fe60 con 0x55962d270870 2024-09-10T09:56:46.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.796+0000 7ff669400640 1 --2- 172.21.15.77:0/741101167 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x55962d270870 0x55962d272c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:46.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.797+0000 7ff669400640 1 --2- 172.21.15.77:0/741101167 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x55962d270870 0x55962d272c90 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=55532796b74d14c5 server_cookie=715e008cdf760c4a in_seq=0 out_seq=0 2024-09-10T09:56:46.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.797+0000 7ff64be00640 10 client.25814 ms_handle_connect on v2:172.21.15.134:6836/2461497091 2024-09-10T09:56:46.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.797+0000 7ff64be00640 1 -- 172.21.15.77:0/741101167 <== mds.3 v2:172.21.15.134:6836/2461497091 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55962d14fe60 con 0x55962d270870 2024-09-10T09:56:46.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.797+0000 7ff64be00640 10 client.25814 handle_command_reply: tid=0 2024-09-10T09:56:46.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.797+0000 7ff673e4c740 1 client.25814 shutdown 2024-09-10T09:56:46.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.797+0000 7ff673e4c740 1 -- 172.21.15.77:0/741101167 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55962d1e1c70 msgr2=0x55962d1fdd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:46.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.797+0000 7ff673e4c740 1 --2- 172.21.15.77:0/741101167 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55962d1e1c70 0x55962d1fdd50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff65c057040 tx=0x7ff65c031f20 comp rx=0 tx=0).stop 2024-09-10T09:56:46.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.797+0000 7ff673e4c740 1 -- 172.21.15.77:0/741101167 shutdown_connections 2024-09-10T09:56:46.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.797+0000 7ff673e4c740 1 --2- 172.21.15.77:0/741101167 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x55962d270870 0x55962d272c90 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:46.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.797+0000 7ff673e4c740 1 --2- 172.21.15.77:0/741101167 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55962d1ece50 0x55962d1f95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:46.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.797+0000 7ff673e4c740 1 --2- 172.21.15.77:0/741101167 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55962d1e2590 0x55962d1f9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:46.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.797+0000 7ff673e4c740 1 --2- 172.21.15.77:0/741101167 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55962d1e1c70 0x55962d1fdd50 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:46.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.797+0000 7ff673e4c740 1 -- 172.21.15.77:0/741101167 >> 172.21.15.77:0/741101167 conn(0x55962d1de200 msgr2=0x55962d1dcb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:46.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.797+0000 7ff64be00640 0 client.25814 ms_handle_reset on v2:172.21.15.134:6836/2461497091 2024-09-10T09:56:46.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.798+0000 7ff673e4c740 1 -- 172.21.15.77:0/741101167 shutdown_connections 2024-09-10T09:56:46.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.800+0000 7ff673e4c740 1 -- 172.21.15.77:0/741101167 wait complete. 2024-09-10T09:56:46.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.800+0000 7ff673e4c740 20 client.25814 trim_cache size 0 max 16384 2024-09-10T09:56:46.802 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-10T09:56:46.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.801+0000 7ff671200640 1 -- 172.21.15.77:0/653320621 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ff6500a2c00 msgr2=0x7ff6500a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:46.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.801+0000 7ff671200640 1 --2- 172.21.15.77:0/653320621 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ff6500a2c00 0x7ff6500a50c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ff65c0096c0 tx=0x7ff65c0046c0 comp rx=0 tx=0).stop 2024-09-10T09:56:46.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.801+0000 7ff671200640 1 -- 172.21.15.77:0/653320621 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff66c1049c0 msgr2=0x7ff66c19ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:46.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.801+0000 7ff671200640 1 --2- 172.21.15.77:0/653320621 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff66c1049c0 0x7ff66c19ab80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff65800e9f0 tx=0x7ff65800eeb0 comp rx=0 tx=0).stop 2024-09-10T09:56:46.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.801+0000 7ff671200640 1 -- 172.21.15.77:0/653320621 shutdown_connections 2024-09-10T09:56:46.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.801+0000 7ff671200640 1 --2- 172.21.15.77:0/653320621 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ff6500a2c00 0x7ff6500a50c0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:46.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.801+0000 7ff671200640 1 --2- 172.21.15.77:0/653320621 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff66c109140 0x7ff66c19ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:46.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.801+0000 7ff671200640 1 --2- 172.21.15.77:0/653320621 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff66c1049c0 0x7ff66c19ab80 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:46.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.801+0000 7ff671200640 1 --2- 172.21.15.77:0/653320621 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff66c104010 0x7ff66c19a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:46.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.801+0000 7ff671200640 1 -- 172.21.15.77:0/653320621 >> 172.21.15.77:0/653320621 conn(0x7ff66c0ffa50 msgr2=0x7ff66c100dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:46.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.801+0000 7ff671200640 1 -- 172.21.15.77:0/653320621 shutdown_connections 2024-09-10T09:56:46.804 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:46.801+0000 7ff671200640 1 -- 172.21.15.77:0/653320621 wait complete. 2024-09-10T09:56:46.805 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:56:46.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:56:46 smithi077 ceph-mon[30319]: pgmap v439: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-10T09:56:47.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:56:46 smithi134 ceph-mon[35674]: pgmap v439: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-10T09:56:47.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:56:46 smithi120 ceph-mon[34770]: pgmap v439: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-10T09:56:48.286 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T09:56:48.414 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:56:48.414 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":29,"btime":"2024-09-10T09:55:13:863715+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3159289675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3159289675},{"type":"v1","addr":"172.21.15.77:6835","nonce":3159289675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1951962341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1951962341},{"type":"v1","addr":"172.21.15.77:6837","nonce":1951962341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:52:23.368252+0000","modified":"2024-09-10T09:55:13.786975+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24563,"mds_1":24524,"mds_2":24520,"mds_3":24502,"mds_4":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14487":{"gid":14487,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/4291121888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":4291121888},{"type":"v1","addr":"172.21.15.77:6839","nonce":4291121888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14499":{"gid":14499,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6841/445284194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":445284194},{"type":"v1","addr":"172.21.15.77:6841","nonce":445284194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1005054423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1005054423},{"type":"v1","addr":"172.21.15.134:6833","nonce":1005054423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6833/3716307744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":3716307744},{"type":"v1","addr":"172.21.15.120:6833","nonce":3716307744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/1319892544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1319892544},{"type":"v1","addr":"172.21.15.120:6835","nonce":1319892544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.134:6835/3293933699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3293933699},{"type":"v1","addr":"172.21.15.134:6835","nonce":3293933699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.134:6837/2461497091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2461497091},{"type":"v1","addr":"172.21.15.134:6837","nonce":2461497091}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":20,"addr":"172.21.15.134:6839/1965117680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1965117680},{"type":"v1","addr":"172.21.15.134:6839","nonce":1965117680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":20,"addr":"172.21.15.120:6837/3033280709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3033280709},{"type":"v1","addr":"172.21.15.120:6837","nonce":3033280709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.120:6839/1777710157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1777710157},{"type":"v1","addr":"172.21.15.120:6839","nonce":1777710157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24563,"qdb_cluster":[24496,24502,24520,24524,24563]},"id":1}]} 2024-09-10T09:56:48.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:47.695+0000 7f0c4d400640 1 -- 172.21.15.77:0/3128721200 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c481129e0 msgr2=0x7f0c48114dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:48.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:47.695+0000 7f0c4d400640 1 --2- 172.21.15.77:0/3128721200 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c481129e0 0x7f0c48114dd0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0c40009be0 tx=0x7f0c4002a750 comp rx=0 tx=0).stop 2024-09-10T09:56:48.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:47.695+0000 7f0c4d400640 1 -- 172.21.15.77:0/3128721200 shutdown_connections 2024-09-10T09:56:48.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:47.695+0000 7f0c4d400640 1 --2- 172.21.15.77:0/3128721200 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c481129e0 0x7f0c48114dd0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:48.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:47.695+0000 7f0c4d400640 1 --2- 172.21.15.77:0/3128721200 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0c48074eb0 0x7f0c481124a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:48.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:47.695+0000 7f0c4d400640 1 --2- 172.21.15.77:0/3128721200 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0c48074590 0x7f0c48074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:48.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:47.695+0000 7f0c4d400640 1 -- 172.21.15.77:0/3128721200 >> 172.21.15.77:0/3128721200 conn(0x7f0c480fd710 msgr2=0x7f0c480ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:48.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:47.695+0000 7f0c4d400640 1 -- 172.21.15.77:0/3128721200 shutdown_connections 2024-09-10T09:56:48.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:47.695+0000 7f0c4d400640 1 -- 172.21.15.77:0/3128721200 wait complete. 2024-09-10T09:56:48.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:47.696+0000 7f0c4d400640 1 Processor -- start 2024-09-10T09:56:48.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:47.696+0000 7f0c4d400640 1 -- start start 2024-09-10T09:56:48.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:47.696+0000 7f0c4d400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0c48074590 0x7f0c481a4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:48.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:47.696+0000 7f0c4d400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c48074eb0 0x7f0c481a4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:48.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:47.696+0000 7f0c4d400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0c481129e0 0x7f0c4819e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:48.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:47.697+0000 7f0c4d400640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0c48118350 con 0x7f0c481129e0 2024-09-10T09:56:48.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:47.697+0000 7f0c4d400640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0c481181d0 con 0x7f0c48074590 2024-09-10T09:56:48.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:47.697+0000 7f0c4d400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0c481184d0 con 0x7f0c48074eb0 2024-09-10T09:56:48.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:47.697+0000 7f0c4ca00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0c481129e0 0x7f0c4819e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:48.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:47.697+0000 7f0c47400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c48074eb0 0x7f0c481a4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:48.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:47.697+0000 7f0c47400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c48074eb0 0x7f0c481a4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:59544/0 (socket says 172.21.15.77:59544) 2024-09-10T09:56:48.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:47.697+0000 7f0c47400640 1 -- 172.21.15.77:0/1827077070 learned_addr learned my addr 172.21.15.77:0/1827077070 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:48.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:47.697+0000 7f0c47e00640 1 --2- 172.21.15.77:0/1827077070 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0c48074590 0x7f0c481a4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:48.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:47.697+0000 7f0c47400640 1 -- 172.21.15.77:0/1827077070 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0c48074590 msgr2=0x7f0c481a4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:48.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:47.697+0000 7f0c47400640 1 --2- 172.21.15.77:0/1827077070 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0c48074590 0x7f0c481a4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:48.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:47.697+0000 7f0c47400640 1 -- 172.21.15.77:0/1827077070 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0c481129e0 msgr2=0x7f0c4819e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:48.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:47.697+0000 7f0c47400640 1 --2- 172.21.15.77:0/1827077070 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0c481129e0 0x7f0c4819e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:48.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:47.697+0000 7f0c47400640 1 -- 172.21.15.77:0/1827077070 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c4819efb0 con 0x7f0c48074eb0 2024-09-10T09:56:48.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:47.697+0000 7f0c47e00640 1 --2- 172.21.15.77:0/1827077070 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0c48074590 0x7f0c481a4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:56:48.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:47.698+0000 7f0c47400640 1 --2- 172.21.15.77:0/1827077070 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c48074eb0 0x7f0c481a4bb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0c3400e9c0 tx=0x7f0c3400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:48.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:47.698+0000 7f0c44c00640 1 -- 172.21.15.77:0/1827077070 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c3400cde0 con 0x7f0c48074eb0 2024-09-10T09:56:48.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:47.698+0000 7f0c4d400640 1 -- 172.21.15.77:0/1827077070 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c4819f2a0 con 0x7f0c48074eb0 2024-09-10T09:56:48.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:47.698+0000 7f0c44c00640 1 -- 172.21.15.77:0/1827077070 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c340035f0 con 0x7f0c48074eb0 2024-09-10T09:56:48.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:47.698+0000 7f0c44c00640 1 -- 172.21.15.77:0/1827077070 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c34010650 con 0x7f0c48074eb0 2024-09-10T09:56:48.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:47.698+0000 7f0c4d400640 1 -- 172.21.15.77:0/1827077070 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c481ab5f0 con 0x7f0c48074eb0 2024-09-10T09:56:48.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:47.699+0000 7f0c4d400640 1 -- 172.21.15.77:0/1827077070 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c48075cd0 con 0x7f0c48074eb0 2024-09-10T09:56:48.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:47.702+0000 7f0c44c00640 1 -- 172.21.15.77:0/1827077070 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0c340037a0 con 0x7f0c48074eb0 2024-09-10T09:56:48.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:47.703+0000 7f0c44c00640 1 --2- 172.21.15.77:0/1827077070 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f0c2c0a2b80 0x7f0c2c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:48.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:47.703+0000 7f0c44c00640 1 -- 172.21.15.77:0/1827077070 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f0c340cf730 con 0x7f0c48074eb0 2024-09-10T09:56:48.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:47.704+0000 7f0c47e00640 1 --2- 172.21.15.77:0/1827077070 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f0c2c0a2b80 0x7f0c2c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:48.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:47.704+0000 7f0c47e00640 1 --2- 172.21.15.77:0/1827077070 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f0c2c0a2b80 0x7f0c2c0a5040 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f0c380095f0 tx=0x7f0c38005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:48.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:47.708+0000 7f0c44c00640 1 -- 172.21.15.77:0/1827077070 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0c3409c240 con 0x7f0c48074eb0 2024-09-10T09:56:48.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:47.930+0000 7f0c4d400640 1 -- 172.21.15.77:0/1827077070 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0c48104e20 con 0x7f0c48074eb0 2024-09-10T09:56:48.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:47.932+0000 7f0c44c00640 1 -- 172.21.15.77:0/1827077070 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11989 (secure 0 0 0) 0x7f0c3409bb40 con 0x7f0c48074eb0 2024-09-10T09:56:48.417 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 29 2024-09-10T09:56:48.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:47.935+0000 7f0c4d400640 1 -- 172.21.15.77:0/1827077070 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f0c2c0a2b80 msgr2=0x7f0c2c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:48.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:47.935+0000 7f0c4d400640 1 --2- 172.21.15.77:0/1827077070 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f0c2c0a2b80 0x7f0c2c0a5040 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f0c380095f0 tx=0x7f0c38005f90 comp rx=0 tx=0).stop 2024-09-10T09:56:48.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:47.935+0000 7f0c4d400640 1 -- 172.21.15.77:0/1827077070 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c48074eb0 msgr2=0x7f0c481a4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:48.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:47.935+0000 7f0c4d400640 1 --2- 172.21.15.77:0/1827077070 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c48074eb0 0x7f0c481a4bb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0c3400e9c0 tx=0x7f0c3400ee80 comp rx=0 tx=0).stop 2024-09-10T09:56:48.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:47.935+0000 7f0c4d400640 1 -- 172.21.15.77:0/1827077070 shutdown_connections 2024-09-10T09:56:48.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:47.935+0000 7f0c4d400640 1 --2- 172.21.15.77:0/1827077070 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f0c2c0a2b80 0x7f0c2c0a5040 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:48.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:47.935+0000 7f0c4d400640 1 --2- 172.21.15.77:0/1827077070 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0c481129e0 0x7f0c4819e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:48.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:47.935+0000 7f0c4d400640 1 --2- 172.21.15.77:0/1827077070 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c48074eb0 0x7f0c481a4bb0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:48.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:47.935+0000 7f0c4d400640 1 --2- 172.21.15.77:0/1827077070 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0c48074590 0x7f0c481a4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:48.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:47.935+0000 7f0c4d400640 1 -- 172.21.15.77:0/1827077070 >> 172.21.15.77:0/1827077070 conn(0x7f0c480fd710 msgr2=0x7f0c480ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:48.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:47.935+0000 7f0c4d400640 1 -- 172.21.15.77:0/1827077070 shutdown_connections 2024-09-10T09:56:48.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:47.935+0000 7f0c4d400640 1 -- 172.21.15.77:0/1827077070 wait complete. 2024-09-10T09:56:49.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:56:48 smithi077 ceph-mon[30319]: pgmap v440: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-09-10T09:56:49.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:56:48 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/1827077070' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:56:49.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:56:48 smithi134 ceph-mon[35674]: pgmap v440: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-09-10T09:56:49.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:56:48 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/1827077070' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:56:49.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:56:48 smithi120 ceph-mon[34770]: pgmap v440: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-09-10T09:56:49.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:56:48 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/1827077070' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:56:49.434 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24563 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-09-10T09:56:49.434 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24563 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-09-10T09:56:49.473 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:56:50.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.080+0000 7f1be7200640 1 -- 172.21.15.77:0/2450540773 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1be8074e30 msgr2=0x7f1be810dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:50.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.080+0000 7f1be7200640 1 --2- 172.21.15.77:0/2450540773 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1be8074e30 0x7f1be810dd40 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f1bd0009900 tx=0x7f1bd002a480 comp rx=0 tx=0).stop 2024-09-10T09:56:50.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.081+0000 7f1be7200640 1 -- 172.21.15.77:0/2450540773 shutdown_connections 2024-09-10T09:56:50.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.081+0000 7f1be7200640 1 --2- 172.21.15.77:0/2450540773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1be810e280 0x7f1be8110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:50.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.081+0000 7f1be7200640 1 --2- 172.21.15.77:0/2450540773 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1be8074e30 0x7f1be810dd40 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:50.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.081+0000 7f1be7200640 1 --2- 172.21.15.77:0/2450540773 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1be8074510 0x7f1be80748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:50.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.081+0000 7f1be7200640 1 -- 172.21.15.77:0/2450540773 >> 172.21.15.77:0/2450540773 conn(0x7f1be80fd740 msgr2=0x7f1be80ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:50.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.081+0000 7f1be7200640 1 -- 172.21.15.77:0/2450540773 shutdown_connections 2024-09-10T09:56:50.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.081+0000 7f1be7200640 1 -- 172.21.15.77:0/2450540773 wait complete. 2024-09-10T09:56:50.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.081+0000 7f1be7200640 1 Processor -- start 2024-09-10T09:56:50.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.082+0000 7f1be7200640 1 -- start start 2024-09-10T09:56:50.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.082+0000 7f1be7200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1be8074510 0x7f1be819a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:50.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.082+0000 7f1be7200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1be8074e30 0x7f1be819aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:50.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.082+0000 7f1be7200640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1be810e280 0x7f1be819ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:50.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.082+0000 7f1be7200640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1be8113f70 con 0x7f1be8074e30 2024-09-10T09:56:50.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.082+0000 7f1be7200640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1be8113df0 con 0x7f1be810e280 2024-09-10T09:56:50.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.082+0000 7f1be7200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1be81140f0 con 0x7f1be8074510 2024-09-10T09:56:50.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.082+0000 7f1bdfe00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1be8074e30 0x7f1be819aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:50.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.082+0000 7f1bdfe00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1be8074e30 0x7f1be819aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:55614/0 (socket says 172.21.15.77:55614) 2024-09-10T09:56:50.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.082+0000 7f1bdfe00640 1 -- 172.21.15.77:0/3915979689 learned_addr learned my addr 172.21.15.77:0/3915979689 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:50.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.082+0000 7f1be5e00640 1 --2- 172.21.15.77:0/3915979689 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1be8074510 0x7f1be819a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:50.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.082+0000 7f1be6800640 1 --2- 172.21.15.77:0/3915979689 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1be810e280 0x7f1be819ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:50.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.083+0000 7f1be5e00640 1 -- 172.21.15.77:0/3915979689 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1be810e280 msgr2=0x7f1be819ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:50.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.083+0000 7f1be5e00640 1 --2- 172.21.15.77:0/3915979689 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1be810e280 0x7f1be819ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:50.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.083+0000 7f1be5e00640 1 -- 172.21.15.77:0/3915979689 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1be8074e30 msgr2=0x7f1be819aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:50.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.083+0000 7f1be5e00640 1 --2- 172.21.15.77:0/3915979689 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1be8074e30 0x7f1be819aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:50.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.083+0000 7f1be5e00640 1 -- 172.21.15.77:0/3915979689 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1be819f500 con 0x7f1be8074510 2024-09-10T09:56:50.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.083+0000 7f1bdfe00640 1 --2- 172.21.15.77:0/3915979689 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1be8074e30 0x7f1be819aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:56:50.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.083+0000 7f1be5e00640 1 --2- 172.21.15.77:0/3915979689 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1be8074510 0x7f1be819a550 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1bd8005fd0 tx=0x7f1bd800dde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:50.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.083+0000 7f1bdea00640 1 -- 172.21.15.77:0/3915979689 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd8009c30 con 0x7f1be8074510 2024-09-10T09:56:50.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.084+0000 7f1be7200640 1 -- 172.21.15.77:0/3915979689 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1be819f7f0 con 0x7f1be8074510 2024-09-10T09:56:50.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.084+0000 7f1bdea00640 1 -- 172.21.15.77:0/3915979689 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bd8009dd0 con 0x7f1be8074510 2024-09-10T09:56:50.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.084+0000 7f1bdea00640 1 -- 172.21.15.77:0/3915979689 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd8010460 con 0x7f1be8074510 2024-09-10T09:56:50.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.084+0000 7f1be7200640 1 -- 172.21.15.77:0/3915979689 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1be81a70d0 con 0x7f1be8074510 2024-09-10T09:56:50.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.087+0000 7f1bef9d9740 1 Processor -- start 2024-09-10T09:56:50.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.087+0000 7f1bdea00640 1 -- 172.21.15.77:0/3915979689 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1bd8003ef0 con 0x7f1be8074510 2024-09-10T09:56:50.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.087+0000 7f1bef9d9740 1 -- start start 2024-09-10T09:56:50.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.087+0000 7f1bef9d9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5622f0b91e40 0x5622f0b92220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:50.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.087+0000 7f1bef9d9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622f0b92760 0x5622f0b9cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:50.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.087+0000 7f1bef9d9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5622f0b9d020 0x5622f0b9f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:50.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.087+0000 7f1bef9d9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5622f0af83e0 con 0x5622f0b9d020 2024-09-10T09:56:50.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.087+0000 7f1bef9d9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5622f0a9fd80 con 0x5622f0b92760 2024-09-10T09:56:50.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.087+0000 7f1bef9d9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5622f0af43a0 con 0x5622f0b91e40 2024-09-10T09:56:50.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.088+0000 7f1be6800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5622f0b91e40 0x5622f0b92220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:50.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.088+0000 7f1be6800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5622f0b91e40 0x5622f0b92220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:59558/0 (socket says 172.21.15.77:59558) 2024-09-10T09:56:50.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.088+0000 7f1be6800640 1 -- 172.21.15.77:0/1418473720 learned_addr learned my addr 172.21.15.77:0/1418473720 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:50.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.088+0000 7f1bdfe00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5622f0b9d020 0x5622f0b9f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:50.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.088+0000 7f1be5e00640 1 --2- 172.21.15.77:0/1418473720 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622f0b92760 0x5622f0b9cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:50.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.088+0000 7f1be6800640 1 -- 172.21.15.77:0/1418473720 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622f0b92760 msgr2=0x5622f0b9cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:50.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.089+0000 7f1bdea00640 1 --2- 172.21.15.77:0/3915979689 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f1bcc0a2c00 0x7f1bcc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:50.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.089+0000 7f1be6800640 1 --2- 172.21.15.77:0/1418473720 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622f0b92760 0x5622f0b9cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:50.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.089+0000 7f1be6800640 1 -- 172.21.15.77:0/1418473720 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5622f0b9d020 msgr2=0x5622f0b9f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:50.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.089+0000 7f1be6800640 1 --2- 172.21.15.77:0/1418473720 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5622f0b9d020 0x5622f0b9f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:50.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.089+0000 7f1be6800640 1 -- 172.21.15.77:0/1418473720 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622f0abeae0 con 0x5622f0b91e40 2024-09-10T09:56:50.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.089+0000 7f1bdea00640 1 -- 172.21.15.77:0/3915979689 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f1bd80ceec0 con 0x7f1be8074510 2024-09-10T09:56:50.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.089+0000 7f1be5e00640 1 --2- 172.21.15.77:0/1418473720 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622f0b92760 0x5622f0b9cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:56:50.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.089+0000 7f1bdfe00640 1 --2- 172.21.15.77:0/1418473720 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5622f0b9d020 0x5622f0b9f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:56:50.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.089+0000 7f1be6800640 1 --2- 172.21.15.77:0/3915979689 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f1bcc0a2c00 0x7f1bcc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:50.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.089+0000 7f1be6800640 1 --2- 172.21.15.77:0/1418473720 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5622f0b91e40 0x5622f0b92220 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1be0009f40 tx=0x7f1be0010800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51073f88219c4f06 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:50.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.090+0000 7f1bd7400640 1 -- 172.21.15.77:0/1418473720 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1be0022070 con 0x5622f0b91e40 2024-09-10T09:56:50.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.090+0000 7f1bd7400640 1 -- 172.21.15.77:0/1418473720 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1be00113e0 con 0x5622f0b91e40 2024-09-10T09:56:50.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.090+0000 7f1bd7400640 1 -- 172.21.15.77:0/1418473720 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1be0016070 con 0x5622f0b91e40 2024-09-10T09:56:50.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.091+0000 7f1bef9d9740 1 -- 172.21.15.77:0/1418473720 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5622f0b91e40 msgr2=0x5622f0b92220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:50.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.091+0000 7f1be6800640 1 --2- 172.21.15.77:0/3915979689 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f1bcc0a2c00 0x7f1bcc0a50c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f1be0009fd0 tx=0x7f1be0022000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:50.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.091+0000 7f1bef9d9740 1 --2- 172.21.15.77:0/1418473720 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5622f0b91e40 0x5622f0b92220 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1be0009f40 tx=0x7f1be0010800 comp rx=0 tx=0).stop 2024-09-10T09:56:50.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.091+0000 7f1bef9d9740 1 -- 172.21.15.77:0/1418473720 shutdown_connections 2024-09-10T09:56:50.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.091+0000 7f1bef9d9740 1 --2- 172.21.15.77:0/1418473720 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5622f0b9d020 0x5622f0b9f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:50.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.091+0000 7f1bef9d9740 1 --2- 172.21.15.77:0/1418473720 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622f0b92760 0x5622f0b9cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:50.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.091+0000 7f1bef9d9740 1 --2- 172.21.15.77:0/1418473720 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5622f0b91e40 0x5622f0b92220 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:50.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.091+0000 7f1bef9d9740 1 -- 172.21.15.77:0/1418473720 >> 172.21.15.77:0/1418473720 conn(0x5622f0b8e3d0 msgr2=0x5622f0b8e7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:50.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.091+0000 7f1bef9d9740 1 -- 172.21.15.77:0/1418473720 shutdown_connections 2024-09-10T09:56:50.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.091+0000 7f1bef9d9740 1 -- 172.21.15.77:0/1418473720 wait complete. 2024-09-10T09:56:50.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.092+0000 7f1bef9d9740 1 -- start start 2024-09-10T09:56:50.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.092+0000 7f1bef9d9740 1 Processor -- start 2024-09-10T09:56:50.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.092+0000 7f1bef9d9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622f0b91e40 0x5622f0b99fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:50.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.093+0000 7f1bef9d9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5622f0b92760 0x5622f0b9a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:50.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.093+0000 7f1bef9d9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5622f0b9d020 0x5622f0b9aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:50.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.093+0000 7f1bef9d9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5622f0b9fbf0 con 0x5622f0b92760 2024-09-10T09:56:50.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.093+0000 7f1bef9d9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5622f0ab30f0 con 0x5622f0b91e40 2024-09-10T09:56:50.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.093+0000 7f1bef9d9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5622f0b9fd70 con 0x5622f0b9d020 2024-09-10T09:56:50.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.093+0000 7f1bef9d9740 10 client.0 _ll_register_callbacks cb 0x5622f0a9abd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:56:50.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.093+0000 7f1be5e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622f0b91e40 0x5622f0b99fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:50.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.093+0000 7f1be5e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622f0b91e40 0x5622f0b99fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:42978/0 (socket says 172.21.15.77:42978) 2024-09-10T09:56:50.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.093+0000 7f1be5e00640 1 -- 172.21.15.77:0/1438243084 learned_addr learned my addr 172.21.15.77:0/1438243084 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:50.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.093+0000 7f1bdfe00640 1 --2- 172.21.15.77:0/1438243084 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5622f0b92760 0x5622f0b9a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:50.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.093+0000 7f1be5e00640 1 -- 172.21.15.77:0/1438243084 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5622f0b9d020 msgr2=0x5622f0b9aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:50.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.093+0000 7f1be6800640 1 --2- 172.21.15.77:0/1438243084 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5622f0b9d020 0x5622f0b9aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:50.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.093+0000 7f1be5e00640 1 --2- 172.21.15.77:0/1438243084 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5622f0b9d020 0x5622f0b9aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:50.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.093+0000 7f1be5e00640 1 -- 172.21.15.77:0/1438243084 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5622f0b92760 msgr2=0x5622f0b9a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:50.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.093+0000 7f1be5e00640 1 --2- 172.21.15.77:0/1438243084 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5622f0b92760 0x5622f0b9a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:50.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.093+0000 7f1be5e00640 1 -- 172.21.15.77:0/1438243084 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622f08693c0 con 0x5622f0b91e40 2024-09-10T09:56:50.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.093+0000 7f1be6800640 1 --2- 172.21.15.77:0/1438243084 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5622f0b9d020 0x5622f0b9aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:56:50.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.094+0000 7f1be5e00640 1 --2- 172.21.15.77:0/1438243084 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622f0b91e40 0x5622f0b99fa0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1bd8009610 tx=0x7f1bd809af20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2bd1860b62295212 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:50.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.094+0000 7f1bcbe00640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T09:56:50.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.094+0000 7f1bcbe00640 1 -- 172.21.15.77:0/1438243084 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd8020050 con 0x5622f0b91e40 2024-09-10T09:56:50.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.094+0000 7f1bcbe00640 1 -- 172.21.15.77:0/1438243084 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f1bd8097a50 con 0x5622f0b91e40 2024-09-10T09:56:50.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.094+0000 7f1bef9d9740 1 -- 172.21.15.77:0/1438243084 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1be00113e0 con 0x5622f0b91e40 2024-09-10T09:56:50.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.094+0000 7f1bcbe00640 1 -- 172.21.15.77:0/1438243084 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1bd809f6e0 con 0x5622f0b91e40 2024-09-10T09:56:50.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.094+0000 7f1bef9d9740 10 client.25829 fetch_fsmap learned FSMap version 29 2024-09-10T09:56:50.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.094+0000 7f1bef9d9740 1 -- 172.21.15.77:0/1438243084 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5622f0babc80 con 0x5622f0b91e40 2024-09-10T09:56:50.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.095+0000 7f1bcbe00640 1 -- 172.21.15.77:0/1438243084 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1be00113e0 con 0x5622f0b91e40 2024-09-10T09:56:50.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.095+0000 7f1bcbe00640 1 -- 172.21.15.77:0/1438243084 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f1bd809a830 con 0x5622f0b91e40 2024-09-10T09:56:50.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.095+0000 7f1bef9d9740 10 client.25829 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T09:56:50.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.095+0000 7f1bef9d9740 10 client.25829 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T09:56:50.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.095+0000 7f1bef9d9740 20 client.25829 populate_metadata read hostname 'smithi077' 2024-09-10T09:56:50.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.095+0000 7f1bef9d9740 1 --2- 172.21.15.77:0/1438243084 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x5622f0bacfd0 0x5622f0baf400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:50.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.095+0000 7f1bef9d9740 4 client.25829 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:56:50.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.095+0000 7f1bef9d9740 1 -- 172.21.15.77:0/1438243084 --> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5622f0b05950 con 0x5622f0bacfd0 2024-09-10T09:56:50.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.096+0000 7f1bdfe00640 1 --2- 172.21.15.77:0/1438243084 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x5622f0bacfd0 0x5622f0baf400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:50.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.096+0000 7f1bdfe00640 1 --2- 172.21.15.77:0/1438243084 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x5622f0bacfd0 0x5622f0baf400 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7fb66328bf787019 server_cookie=1df602ed8c9dc83 in_seq=0 out_seq=0 2024-09-10T09:56:50.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.096+0000 7f1bcbe00640 10 client.25829 ms_handle_connect on v2:172.21.15.134:6838/1965117680 2024-09-10T09:56:50.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.097+0000 7f1bcbe00640 1 -- 172.21.15.77:0/1438243084 <== mds.2 v2:172.21.15.134:6838/1965117680 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5622f0b05950 con 0x5622f0bacfd0 2024-09-10T09:56:50.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.097+0000 7f1bcbe00640 10 client.25829 handle_command_reply: tid=0 2024-09-10T09:56:50.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.097+0000 7f1bef9d9740 1 client.25829 shutdown 2024-09-10T09:56:50.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.097+0000 7f1bef9d9740 1 -- 172.21.15.77:0/1438243084 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622f0b91e40 msgr2=0x5622f0b99fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:50.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.097+0000 7f1bef9d9740 1 --2- 172.21.15.77:0/1438243084 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622f0b91e40 0x5622f0b99fa0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1bd8009610 tx=0x7f1bd809af20 comp rx=0 tx=0).stop 2024-09-10T09:56:50.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.097+0000 7f1bef9d9740 1 -- 172.21.15.77:0/1438243084 shutdown_connections 2024-09-10T09:56:50.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.097+0000 7f1bef9d9740 1 --2- 172.21.15.77:0/1438243084 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x5622f0bacfd0 0x5622f0baf400 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:50.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.097+0000 7f1bef9d9740 1 --2- 172.21.15.77:0/1438243084 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5622f0b9d020 0x5622f0b9aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:50.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.097+0000 7f1bef9d9740 1 --2- 172.21.15.77:0/1438243084 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5622f0b92760 0x5622f0b9a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:50.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.097+0000 7f1bef9d9740 1 --2- 172.21.15.77:0/1438243084 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622f0b91e40 0x5622f0b99fa0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:50.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.097+0000 7f1bef9d9740 1 -- 172.21.15.77:0/1438243084 >> 172.21.15.77:0/1438243084 conn(0x5622f0b8e3d0 msgr2=0x5622f0b94100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:50.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.097+0000 7f1bcbe00640 0 client.25829 ms_handle_reset on v2:172.21.15.134:6838/1965117680 2024-09-10T09:56:50.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.098+0000 7f1bef9d9740 1 -- 172.21.15.77:0/1438243084 shutdown_connections 2024-09-10T09:56:50.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.098+0000 7f1bef9d9740 1 -- 172.21.15.77:0/1438243084 wait complete. 2024-09-10T09:56:50.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.098+0000 7f1bef9d9740 20 client.25829 trim_cache size 0 max 16384 2024-09-10T09:56:50.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.112+0000 7f1bef9d9740 1 Processor -- start 2024-09-10T09:56:50.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.112+0000 7f1bef9d9740 1 -- start start 2024-09-10T09:56:50.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.112+0000 7f1bef9d9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5622f0b91e40 0x5622f0b98fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:50.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.112+0000 7f1bef9d9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622f0b92760 0x5622f0bb4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:50.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.112+0000 7f1bef9d9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5622f0b9d020 0x5622f0bb51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:50.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.112+0000 7f1bef9d9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1be81b0ff0 con 0x5622f0b9d020 2024-09-10T09:56:50.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.112+0000 7f1bef9d9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1be81b2b30 con 0x5622f0b92760 2024-09-10T09:56:50.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.112+0000 7f1bef9d9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1be81b16c0 con 0x5622f0b91e40 2024-09-10T09:56:50.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.113+0000 7f1be6800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5622f0b9d020 0x5622f0bb51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:50.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.113+0000 7f1be6800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5622f0b9d020 0x5622f0bb51e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:55638/0 (socket says 172.21.15.77:55638) 2024-09-10T09:56:50.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.113+0000 7f1be6800640 1 -- 172.21.15.77:0/793253830 learned_addr learned my addr 172.21.15.77:0/793253830 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:50.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.113+0000 7f1be5e00640 1 --2- 172.21.15.77:0/793253830 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5622f0b91e40 0x5622f0b98fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:50.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.113+0000 7f1bdfe00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622f0b92760 0x5622f0bb4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:50.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.113+0000 7f1be5e00640 1 -- 172.21.15.77:0/793253830 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622f0b92760 msgr2=0x5622f0bb4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:50.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.113+0000 7f1be5e00640 1 --2- 172.21.15.77:0/793253830 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622f0b92760 0x5622f0bb4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:50.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.113+0000 7f1be5e00640 1 -- 172.21.15.77:0/793253830 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5622f0b9d020 msgr2=0x5622f0bb51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:50.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.113+0000 7f1be5e00640 1 --2- 172.21.15.77:0/793253830 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5622f0b9d020 0x5622f0bb51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:50.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.113+0000 7f1be5e00640 1 -- 172.21.15.77:0/793253830 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622f0b9c910 con 0x5622f0b91e40 2024-09-10T09:56:50.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.113+0000 7f1be6800640 1 --2- 172.21.15.77:0/793253830 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5622f0b9d020 0x5622f0bb51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:56:50.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.114+0000 7f1be5e00640 1 --2- 172.21.15.77:0/793253830 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5622f0b91e40 0x5622f0b98fc0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1bd809b370 tx=0x7f1bd800df90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73ba8ccae9fa52a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:50.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.114+0000 7f1bd6a00640 1 -- 172.21.15.77:0/793253830 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd8020020 con 0x5622f0b91e40 2024-09-10T09:56:50.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.114+0000 7f1bd6a00640 1 -- 172.21.15.77:0/793253830 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bd8097a50 con 0x5622f0b91e40 2024-09-10T09:56:50.117 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.114+0000 7f1bd6a00640 1 -- 172.21.15.77:0/793253830 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd8097200 con 0x5622f0b91e40 2024-09-10T09:56:50.117 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.115+0000 7f1bef9d9740 1 -- 172.21.15.77:0/793253830 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5622f0b91e40 msgr2=0x5622f0b98fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:50.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.115+0000 7f1bef9d9740 1 --2- 172.21.15.77:0/793253830 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5622f0b91e40 0x5622f0b98fc0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1bd809b370 tx=0x7f1bd800df90 comp rx=0 tx=0).stop 2024-09-10T09:56:50.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.115+0000 7f1bef9d9740 1 -- 172.21.15.77:0/793253830 shutdown_connections 2024-09-10T09:56:50.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.115+0000 7f1bef9d9740 1 --2- 172.21.15.77:0/793253830 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5622f0b9d020 0x5622f0bb51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:50.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.115+0000 7f1bef9d9740 1 --2- 172.21.15.77:0/793253830 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622f0b92760 0x5622f0bb4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:50.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.115+0000 7f1bef9d9740 1 --2- 172.21.15.77:0/793253830 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5622f0b91e40 0x5622f0b98fc0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:50.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.115+0000 7f1bef9d9740 1 -- 172.21.15.77:0/793253830 >> 172.21.15.77:0/793253830 conn(0x5622f0b8e3d0 msgr2=0x5622f0b988b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:50.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.115+0000 7f1bef9d9740 1 -- 172.21.15.77:0/793253830 shutdown_connections 2024-09-10T09:56:50.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.115+0000 7f1bef9d9740 1 -- 172.21.15.77:0/793253830 wait complete. 2024-09-10T09:56:50.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.116+0000 7f1bef9d9740 1 -- start start 2024-09-10T09:56:50.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.116+0000 7f1bef9d9740 1 Processor -- start 2024-09-10T09:56:50.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.116+0000 7f1bef9d9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5622f0b91e40 0x5622f0ba51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:50.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.116+0000 7f1bef9d9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5622f0b92760 0x5622f0ba56e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:50.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.116+0000 7f1bef9d9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622f0b9d020 0x5622f0b967f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:50.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.116+0000 7f1bef9d9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1be81b2460 con 0x5622f0b92760 2024-09-10T09:56:50.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.116+0000 7f1bef9d9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1be81b1d90 con 0x5622f0b9d020 2024-09-10T09:56:50.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.116+0000 7f1bef9d9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1be81aefd0 con 0x5622f0b91e40 2024-09-10T09:56:50.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.116+0000 7f1bef9d9740 10 client.0 _ll_register_callbacks cb 0x7f1bd8095f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:56:50.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.117+0000 7f1be5e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5622f0b91e40 0x5622f0ba51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:50.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.117+0000 7f1be5e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5622f0b91e40 0x5622f0ba51a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:59602/0 (socket says 172.21.15.77:59602) 2024-09-10T09:56:50.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.117+0000 7f1be5e00640 1 -- 172.21.15.77:0/2097500809 learned_addr learned my addr 172.21.15.77:0/2097500809 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:50.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.117+0000 7f1be6800640 1 --2- 172.21.15.77:0/2097500809 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622f0b9d020 0x5622f0b967f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:50.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.117+0000 7f1bdfe00640 1 --2- 172.21.15.77:0/2097500809 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5622f0b92760 0x5622f0ba56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:50.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.117+0000 7f1be5e00640 1 -- 172.21.15.77:0/2097500809 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622f0b9d020 msgr2=0x5622f0b967f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:50.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.117+0000 7f1be5e00640 1 --2- 172.21.15.77:0/2097500809 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622f0b9d020 0x5622f0b967f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:50.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.117+0000 7f1be5e00640 1 -- 172.21.15.77:0/2097500809 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5622f0b92760 msgr2=0x5622f0ba56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:50.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.117+0000 7f1be5e00640 1 --2- 172.21.15.77:0/2097500809 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5622f0b92760 0x5622f0ba56e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:50.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.117+0000 7f1be5e00640 1 -- 172.21.15.77:0/2097500809 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622f0b97020 con 0x5622f0b91e40 2024-09-10T09:56:50.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.117+0000 7f1be5e00640 1 --2- 172.21.15.77:0/2097500809 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5622f0b91e40 0x5622f0ba51a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1bd8097760 tx=0x7f1bd809b050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5cd00e154b81b4c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:50.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.117+0000 7f1bcbe00640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-10T09:56:50.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.118+0000 7f1bcbe00640 1 -- 172.21.15.77:0/2097500809 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd8020030 con 0x5622f0b91e40 2024-09-10T09:56:50.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.118+0000 7f1bcbe00640 1 -- 172.21.15.77:0/2097500809 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f1bd80bb0e0 con 0x5622f0b91e40 2024-09-10T09:56:50.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.118+0000 7f1bef9d9740 1 -- 172.21.15.77:0/2097500809 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1bd8097a50 con 0x5622f0b91e40 2024-09-10T09:56:50.123 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.118+0000 7f1bcbe00640 1 -- 172.21.15.77:0/2097500809 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1bd80a3fa0 con 0x5622f0b91e40 2024-09-10T09:56:50.123 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.118+0000 7f1bef9d9740 10 client.25795 fetch_fsmap learned FSMap version 29 2024-09-10T09:56:50.123 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.118+0000 7f1bef9d9740 1 -- 172.21.15.77:0/2097500809 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5622f0c1ff60 con 0x5622f0b91e40 2024-09-10T09:56:50.123 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.119+0000 7f1bcbe00640 1 -- 172.21.15.77:0/2097500809 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bd8097a50 con 0x5622f0b91e40 2024-09-10T09:56:50.123 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.119+0000 7f1bcbe00640 1 -- 172.21.15.77:0/2097500809 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f1bd809a2a0 con 0x5622f0b91e40 2024-09-10T09:56:50.123 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.119+0000 7f1bef9d9740 10 client.25795 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T09:56:50.123 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.119+0000 7f1bef9d9740 10 client.25795 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T09:56:50.123 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.119+0000 7f1bef9d9740 20 client.25795 populate_metadata read hostname 'smithi077' 2024-09-10T09:56:50.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.119+0000 7f1bef9d9740 1 --2- 172.21.15.77:0/2097500809 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x5622f0c203e0 0x5622f0c227d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:50.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.119+0000 7f1bef9d9740 4 client.25795 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:56:50.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.119+0000 7f1bef9d9740 1 -- 172.21.15.77:0/2097500809 --> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5622f0affe60 con 0x5622f0c203e0 2024-09-10T09:56:50.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.119+0000 7f1bdfe00640 1 --2- 172.21.15.77:0/2097500809 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x5622f0c203e0 0x5622f0c227d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:50.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.120+0000 7f1bdfe00640 1 --2- 172.21.15.77:0/2097500809 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x5622f0c203e0 0x5622f0c227d0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6c03c2a0a5b01aef server_cookie=c72ed6228b20ea46 in_seq=0 out_seq=0 2024-09-10T09:56:50.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.120+0000 7f1bcbe00640 10 client.25795 ms_handle_connect on v2:172.21.15.134:6838/1965117680 2024-09-10T09:56:50.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.120+0000 7f1bcbe00640 1 -- 172.21.15.77:0/2097500809 <== mds.2 v2:172.21.15.134:6838/1965117680 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5622f0affe60 con 0x5622f0c203e0 2024-09-10T09:56:50.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.120+0000 7f1bcbe00640 10 client.25795 handle_command_reply: tid=0 2024-09-10T09:56:50.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.120+0000 7f1bef9d9740 1 client.25795 shutdown 2024-09-10T09:56:50.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.120+0000 7f1bef9d9740 1 -- 172.21.15.77:0/2097500809 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5622f0b91e40 msgr2=0x5622f0ba51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:50.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.120+0000 7f1bef9d9740 1 --2- 172.21.15.77:0/2097500809 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5622f0b91e40 0x5622f0ba51a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1bd8097760 tx=0x7f1bd809b050 comp rx=0 tx=0).stop 2024-09-10T09:56:50.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.120+0000 7f1bef9d9740 1 -- 172.21.15.77:0/2097500809 shutdown_connections 2024-09-10T09:56:50.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.120+0000 7f1bef9d9740 1 --2- 172.21.15.77:0/2097500809 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x5622f0c203e0 0x5622f0c227d0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:50.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.120+0000 7f1bef9d9740 1 --2- 172.21.15.77:0/2097500809 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622f0b9d020 0x5622f0b967f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:50.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.121+0000 7f1bef9d9740 1 --2- 172.21.15.77:0/2097500809 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5622f0b92760 0x5622f0ba56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:50.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.121+0000 7f1bef9d9740 1 --2- 172.21.15.77:0/2097500809 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5622f0b91e40 0x5622f0ba51a0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:50.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.121+0000 7f1bef9d9740 1 -- 172.21.15.77:0/2097500809 >> 172.21.15.77:0/2097500809 conn(0x5622f0b8e3d0 msgr2=0x5622f0b990d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:50.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.121+0000 7f1bcbe00640 0 client.25795 ms_handle_reset on v2:172.21.15.134:6838/1965117680 2024-09-10T09:56:50.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.121+0000 7f1bef9d9740 1 -- 172.21.15.77:0/2097500809 shutdown_connections 2024-09-10T09:56:50.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.121+0000 7f1bef9d9740 1 -- 172.21.15.77:0/2097500809 wait complete. 2024-09-10T09:56:50.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.121+0000 7f1bef9d9740 20 client.25795 trim_cache size 0 max 16384 2024-09-10T09:56:50.125 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-10T09:56:50.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.122+0000 7f1be7200640 1 -- 172.21.15.77:0/3915979689 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f1bcc0a2c00 msgr2=0x7f1bcc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:50.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.122+0000 7f1be7200640 1 --2- 172.21.15.77:0/3915979689 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f1bcc0a2c00 0x7f1bcc0a50c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f1be0009fd0 tx=0x7f1be0022000 comp rx=0 tx=0).stop 2024-09-10T09:56:50.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.122+0000 7f1be7200640 1 -- 172.21.15.77:0/3915979689 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1be8074510 msgr2=0x7f1be819a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:50.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.122+0000 7f1be7200640 1 --2- 172.21.15.77:0/3915979689 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1be8074510 0x7f1be819a550 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1bd8005fd0 tx=0x7f1bd800dde0 comp rx=0 tx=0).stop 2024-09-10T09:56:50.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.122+0000 7f1be7200640 1 -- 172.21.15.77:0/3915979689 shutdown_connections 2024-09-10T09:56:50.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.122+0000 7f1be7200640 1 --2- 172.21.15.77:0/3915979689 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f1bcc0a2c00 0x7f1bcc0a50c0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:50.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.122+0000 7f1be7200640 1 --2- 172.21.15.77:0/3915979689 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1be810e280 0x7f1be819ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:50.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.122+0000 7f1be7200640 1 --2- 172.21.15.77:0/3915979689 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1be8074e30 0x7f1be819aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:50.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.122+0000 7f1be7200640 1 --2- 172.21.15.77:0/3915979689 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1be8074510 0x7f1be819a550 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:50.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.122+0000 7f1be7200640 1 -- 172.21.15.77:0/3915979689 >> 172.21.15.77:0/3915979689 conn(0x7f1be80fd740 msgr2=0x7f1be80fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:50.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.122+0000 7f1be7200640 1 -- 172.21.15.77:0/3915979689 shutdown_connections 2024-09-10T09:56:50.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:50.122+0000 7f1be7200640 1 -- 172.21.15.77:0/3915979689 wait complete. 2024-09-10T09:56:50.911 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:56:50.940 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T09:56:51.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:56:50 smithi077 ceph-mon[30319]: pgmap v441: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 12 op/s 2024-09-10T09:56:51.248 INFO:tasks.workunit.client.0.smithi077.stderr:Updating files: 58% (7401/12697) Updating files: 59% (7492/12697) Updating files: 60% (7619/12697) Updating files: 61% (7746/12697) Updating files: 62% (7873/12697) Updating files: 63% (8000/12697) Updating files: 64% (8127/12697) Updating files: 65% (8254/12697) Updating files: 66% (8381/12697) Updating files: 67% (8507/12697) Updating files: 68% (8634/12697) Updating files: 69% (8761/12697) Updating files: 70% (8888/12697) Updating files: 71% (9015/12697) Updating files: 72% (9142/12697) Updating files: 73% (9269/12697) Updating files: 74% (9396/12697) Updating files: 75% (9523/12697) Updating files: 76% (9650/12697) Updating files: 77% (9777/12697) Updating files: 78% (9904/12697) Updating files: 79% (10031/12697) Updating files: 80% (10158/12697) Updating files: 81% (10285/12697) Updating files: 82% (10412/12697) Updating files: 83% (10539/12697) Updating files: 84% (10666/12697) Updating files: 85% (10793/12697) Updating files: 86% (10920/12697) Updating files: 87% (11047/12697) Updating files: 88% (11174/12697) Updating files: 89% (11301/12697) Updating files: 90% (11428/12697) Updating files: 91% (11555/12697) Updating files: 92% (11682/12697) Updating files: 93% (11809/12697) Updating files: 94% (11936/12697) Updating files: 95% (12063/12697) Updating files: 96% (12190/12697) Updating files: 96% (12302/12697) Updating files: 97% (12317/12697) Updating files: 98% (12444/12697) Updating files: 99% (12571/12697) Updating files: 100% (12697/12697) Updating files: 100% (12697/12697), done. 2024-09-10T09:56:51.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:56:50 smithi134 ceph-mon[35674]: pgmap v441: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 12 op/s 2024-09-10T09:56:51.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:56:50 smithi120 ceph-mon[34770]: pgmap v441: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 12 op/s 2024-09-10T09:56:51.544 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.542+0000 7f47d2000640 1 -- 172.21.15.77:0/825585723 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f47cc103e30 msgr2=0x7f47cc104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:51.545 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.542+0000 7f47d2000640 1 --2- 172.21.15.77:0/825585723 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f47cc103e30 0x7f47cc104210 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f47c0009900 tx=0x7f47c002a410 comp rx=0 tx=0).stop 2024-09-10T09:56:51.545 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.542+0000 7f47d2000640 1 -- 172.21.15.77:0/825585723 shutdown_connections 2024-09-10T09:56:51.545 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.542+0000 7f47d2000640 1 --2- 172.21.15.77:0/825585723 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47cc108f60 0x7f47cc110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:51.545 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.542+0000 7f47d2000640 1 --2- 172.21.15.77:0/825585723 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f47cc1047e0 0x7f47cc108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:51.545 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.542+0000 7f47d2000640 1 --2- 172.21.15.77:0/825585723 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f47cc103e30 0x7f47cc104210 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:51.545 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.542+0000 7f47d2000640 1 -- 172.21.15.77:0/825585723 >> 172.21.15.77:0/825585723 conn(0x7f47cc0ff910 msgr2=0x7f47cc101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:51.545 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.542+0000 7f47d2000640 1 -- 172.21.15.77:0/825585723 shutdown_connections 2024-09-10T09:56:51.545 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.542+0000 7f47d2000640 1 -- 172.21.15.77:0/825585723 wait complete. 2024-09-10T09:56:51.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.543+0000 7f47d2000640 1 Processor -- start 2024-09-10T09:56:51.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.543+0000 7f47d2000640 1 -- start start 2024-09-10T09:56:51.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.544+0000 7f47d2000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47cc103e30 0x7f47cc19a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:51.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.544+0000 7f47d2000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f47cc1047e0 0x7f47cc19aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:51.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.544+0000 7f47d2000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f47cc108f60 0x7f47cc19eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:51.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.544+0000 7f47d2000640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f47cc114010 con 0x7f47cc1047e0 2024-09-10T09:56:51.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.544+0000 7f47d2000640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f47cc113e90 con 0x7f47cc108f60 2024-09-10T09:56:51.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.544+0000 7f47d2000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f47cc114190 con 0x7f47cc103e30 2024-09-10T09:56:51.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.544+0000 7f47d0c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47cc103e30 0x7f47cc19a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:51.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.544+0000 7f47d0c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47cc103e30 0x7f47cc19a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:59618/0 (socket says 172.21.15.77:59618) 2024-09-10T09:56:51.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.544+0000 7f47d0c00640 1 -- 172.21.15.77:0/867938300 learned_addr learned my addr 172.21.15.77:0/867938300 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:51.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.544+0000 7f47cbe00640 1 --2- 172.21.15.77:0/867938300 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f47cc1047e0 0x7f47cc19aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:51.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.544+0000 7f47d1600640 1 --2- 172.21.15.77:0/867938300 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f47cc108f60 0x7f47cc19eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:51.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.545+0000 7f47d0c00640 1 -- 172.21.15.77:0/867938300 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f47cc108f60 msgr2=0x7f47cc19eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:51.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.545+0000 7f47d0c00640 1 --2- 172.21.15.77:0/867938300 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f47cc108f60 0x7f47cc19eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:51.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.545+0000 7f47d0c00640 1 -- 172.21.15.77:0/867938300 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f47cc1047e0 msgr2=0x7f47cc19aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:51.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.545+0000 7f47d0c00640 1 --2- 172.21.15.77:0/867938300 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f47cc1047e0 0x7f47cc19aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:51.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.545+0000 7f47d0c00640 1 -- 172.21.15.77:0/867938300 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47cc19f480 con 0x7f47cc103e30 2024-09-10T09:56:51.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.545+0000 7f47d0c00640 1 --2- 172.21.15.77:0/867938300 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47cc103e30 0x7f47cc19a4d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f47c0004770 tx=0x7f47c0003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:51.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.545+0000 7f47c9600640 1 -- 172.21.15.77:0/867938300 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c003f070 con 0x7f47cc103e30 2024-09-10T09:56:51.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.545+0000 7f47c9600640 1 -- 172.21.15.77:0/867938300 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47c0031ca0 con 0x7f47cc103e30 2024-09-10T09:56:51.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.545+0000 7f47c9600640 1 -- 172.21.15.77:0/867938300 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c0035030 con 0x7f47cc103e30 2024-09-10T09:56:51.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.545+0000 7f47d2000640 1 -- 172.21.15.77:0/867938300 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47cc19f710 con 0x7f47cc103e30 2024-09-10T09:56:51.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.546+0000 7f47d2000640 1 -- 172.21.15.77:0/867938300 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47cc1a7050 con 0x7f47cc103e30 2024-09-10T09:56:51.550 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.549+0000 7f47d6046740 1 Processor -- start 2024-09-10T09:56:51.550 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.549+0000 7f47d6046740 1 -- start start 2024-09-10T09:56:51.550 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.549+0000 7f47c9600640 1 -- 172.21.15.77:0/867938300 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f47c0042070 con 0x7f47cc103e30 2024-09-10T09:56:51.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.549+0000 7f47d6046740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b88159560 0x559b88159940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:51.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.549+0000 7f47d6046740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b88159e80 0x559b88164200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:51.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.549+0000 7f47d6046740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b88164740 0x559b88166b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:51.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.549+0000 7f47d6046740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x559b880bf530 con 0x559b88164740 2024-09-10T09:56:51.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.549+0000 7f47d6046740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x559b88066860 con 0x559b88159e80 2024-09-10T09:56:51.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.549+0000 7f47d6046740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559b880bb4f0 con 0x559b88159560 2024-09-10T09:56:51.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.550+0000 7f47d0c00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b88159e80 0x559b88164200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:51.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.550+0000 7f47d1600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b88159560 0x559b88159940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:51.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.550+0000 7f47d1600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b88159560 0x559b88159940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:59628/0 (socket says 172.21.15.77:59628) 2024-09-10T09:56:51.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.550+0000 7f47d1600640 1 -- 172.21.15.77:0/1276228330 learned_addr learned my addr 172.21.15.77:0/1276228330 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:51.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.550+0000 7f47cbe00640 1 --2- 172.21.15.77:0/1276228330 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b88164740 0x559b88166b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:51.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.550+0000 7f47d1600640 1 -- 172.21.15.77:0/1276228330 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b88159e80 msgr2=0x559b88164200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:51.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.550+0000 7f47d1600640 1 --2- 172.21.15.77:0/1276228330 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b88159e80 0x559b88164200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:51.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.550+0000 7f47d1600640 1 -- 172.21.15.77:0/1276228330 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b88164740 msgr2=0x559b88166b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:51.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.550+0000 7f47d1600640 1 --2- 172.21.15.77:0/1276228330 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b88164740 0x559b88166b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:51.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.550+0000 7f47d1600640 1 -- 172.21.15.77:0/1276228330 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b87e30b90 con 0x559b88159560 2024-09-10T09:56:51.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.550+0000 7f47d1600640 1 --2- 172.21.15.77:0/1276228330 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b88159560 0x559b88159940 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f47c4009f50 tx=0x7f47c40118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a75c461298a6e8b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:51.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.551+0000 7f47c9600640 1 --2- 172.21.15.77:0/867938300 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f47b40a2b80 0x7f47b40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:51.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.551+0000 7f47bdc00640 1 -- 172.21.15.77:0/1276228330 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c401e070 con 0x559b88159560 2024-09-10T09:56:51.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.551+0000 7f47bdc00640 1 -- 172.21.15.77:0/1276228330 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47c4003c30 con 0x559b88159560 2024-09-10T09:56:51.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.551+0000 7f47bdc00640 1 -- 172.21.15.77:0/1276228330 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c400f9a0 con 0x559b88159560 2024-09-10T09:56:51.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.551+0000 7f47c9600640 1 -- 172.21.15.77:0/867938300 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f47c00efa50 con 0x7f47cc103e30 2024-09-10T09:56:51.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.551+0000 7f47d0c00640 1 --2- 172.21.15.77:0/867938300 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f47b40a2b80 0x7f47b40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:51.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.552+0000 7f47d6046740 1 -- 172.21.15.77:0/1276228330 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b88159560 msgr2=0x559b88159940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:51.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.552+0000 7f47d6046740 1 --2- 172.21.15.77:0/1276228330 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b88159560 0x559b88159940 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f47c4009f50 tx=0x7f47c40118d0 comp rx=0 tx=0).stop 2024-09-10T09:56:51.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.552+0000 7f47d0c00640 1 --2- 172.21.15.77:0/867938300 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f47b40a2b80 0x7f47b40a5040 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f47c00b8430 tx=0x7f47c00f5530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:51.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.552+0000 7f47d6046740 1 -- 172.21.15.77:0/1276228330 shutdown_connections 2024-09-10T09:56:51.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.552+0000 7f47d6046740 1 --2- 172.21.15.77:0/1276228330 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b88164740 0x559b88166b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:51.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.552+0000 7f47d6046740 1 --2- 172.21.15.77:0/1276228330 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b88159e80 0x559b88164200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:51.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.552+0000 7f47d6046740 1 --2- 172.21.15.77:0/1276228330 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b88159560 0x559b88159940 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:51.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.552+0000 7f47d6046740 1 -- 172.21.15.77:0/1276228330 >> 172.21.15.77:0/1276228330 conn(0x559b88155af0 msgr2=0x559b88155ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:51.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.552+0000 7f47d6046740 1 -- 172.21.15.77:0/1276228330 shutdown_connections 2024-09-10T09:56:51.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.552+0000 7f47d6046740 1 -- 172.21.15.77:0/1276228330 wait complete. 2024-09-10T09:56:51.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.552+0000 7f47d6046740 1 -- start start 2024-09-10T09:56:51.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.553+0000 7f47d6046740 1 Processor -- start 2024-09-10T09:56:51.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.553+0000 7f47d6046740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b88159560 0x559b88161620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:51.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.553+0000 7f47d6046740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b88159e80 0x559b88161b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:51.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.553+0000 7f47d6046740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b88164740 0x559b881620a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:51.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.553+0000 7f47d6046740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x559b88167310 con 0x559b88159e80 2024-09-10T09:56:51.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.553+0000 7f47d6046740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x559b88079e60 con 0x559b88159560 2024-09-10T09:56:51.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.553+0000 7f47d6046740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559b88167490 con 0x559b88164740 2024-09-10T09:56:51.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.553+0000 7f47d6046740 10 client.0 _ll_register_callbacks cb 0x559b8807ecc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:56:51.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.553+0000 7f47d1600640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b88159560 0x559b88161620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:51.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.553+0000 7f47cbe00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b88159e80 0x559b88161b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:51.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.553+0000 7f47d1600640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b88159560 0x559b88161620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:43030/0 (socket says 172.21.15.77:43030) 2024-09-10T09:56:51.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.553+0000 7f47d1600640 1 -- 172.21.15.77:0/466569225 learned_addr learned my addr 172.21.15.77:0/466569225 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:51.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.554+0000 7f47d1600640 1 --2- 172.21.15.77:0/466569225 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b88164740 0x559b881620a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:51.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.554+0000 7f47cbe00640 1 -- 172.21.15.77:0/466569225 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b88164740 msgr2=0x559b881620a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:51.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.554+0000 7f47cbe00640 1 --2- 172.21.15.77:0/466569225 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b88164740 0x559b881620a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:51.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.554+0000 7f47cbe00640 1 -- 172.21.15.77:0/466569225 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b88159560 msgr2=0x559b88161620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:51.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.554+0000 7f47cbe00640 1 --2- 172.21.15.77:0/466569225 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b88159560 0x559b88161620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:51.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.554+0000 7f47cbe00640 1 -- 172.21.15.77:0/466569225 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b88085990 con 0x559b88159e80 2024-09-10T09:56:51.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.554+0000 7f47cbe00640 1 --2- 172.21.15.77:0/466569225 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b88159e80 0x559b88161b60 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f47cc10a820 tx=0x7f47b8012b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ed0f0be7398cc7b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:51.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.554+0000 7f47b2000640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-10T09:56:51.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.555+0000 7f47b2000640 1 -- 172.21.15.77:0/466569225 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b800f920 con 0x559b88159e80 2024-09-10T09:56:51.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.555+0000 7f47b2000640 1 -- 172.21.15.77:0/466569225 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f47b800fac0 con 0x559b88159e80 2024-09-10T09:56:51.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.555+0000 7f47d6046740 1 -- 172.21.15.77:0/466569225 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47c4003c30 con 0x559b88159e80 2024-09-10T09:56:51.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.555+0000 7f47b2000640 1 -- 172.21.15.77:0/466569225 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f47b800fcc0 con 0x559b88159e80 2024-09-10T09:56:51.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.555+0000 7f47d6046740 10 client.15777 fetch_fsmap learned FSMap version 29 2024-09-10T09:56:51.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.555+0000 7f47d6046740 1 -- 172.21.15.77:0/466569225 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559b88172f70 con 0x559b88159e80 2024-09-10T09:56:51.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.555+0000 7f47b2000640 1 -- 172.21.15.77:0/466569225 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47c4003c30 con 0x559b88159e80 2024-09-10T09:56:51.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.555+0000 7f47b2000640 1 -- 172.21.15.77:0/466569225 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f47b801c420 con 0x559b88159e80 2024-09-10T09:56:51.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.555+0000 7f47d6046740 10 client.15777 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T09:56:51.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.556+0000 7f47d6046740 10 client.15777 resolve_mds: validated gid 24563 aka daemon mds.b 2024-09-10T09:56:51.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.556+0000 7f47d6046740 20 client.15777 populate_metadata read hostname 'smithi077' 2024-09-10T09:56:51.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.556+0000 7f47d6046740 1 --2- 172.21.15.77:0/466569225 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x559b88174430 0x559b88176860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:51.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.556+0000 7f47d6046740 4 client.15777 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:56:51.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.556+0000 7f47d6046740 1 -- 172.21.15.77:0/466569225 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559b88062310 con 0x559b88174430 2024-09-10T09:56:51.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.556+0000 7f47d1600640 1 --2- 172.21.15.77:0/466569225 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x559b88174430 0x559b88176860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:51.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.557+0000 7f47d1600640 1 --2- 172.21.15.77:0/466569225 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x559b88174430 0x559b88176860 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d616aec2fdffb975 server_cookie=7a742bd55804b211 in_seq=0 out_seq=0 2024-09-10T09:56:51.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.557+0000 7f47b2000640 10 client.15777 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T09:56:51.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.557+0000 7f47b2000640 1 -- 172.21.15.77:0/466569225 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559b88062310 con 0x559b88174430 2024-09-10T09:56:51.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.557+0000 7f47b2000640 10 client.15777 handle_command_reply: tid=0 2024-09-10T09:56:51.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.557+0000 7f47d6046740 1 client.15777 shutdown 2024-09-10T09:56:51.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.558+0000 7f47d6046740 1 -- 172.21.15.77:0/466569225 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b88159e80 msgr2=0x559b88161b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:51.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.558+0000 7f47d6046740 1 --2- 172.21.15.77:0/466569225 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b88159e80 0x559b88161b60 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f47cc10a820 tx=0x7f47b8012b60 comp rx=0 tx=0).stop 2024-09-10T09:56:51.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.558+0000 7f47d6046740 1 -- 172.21.15.77:0/466569225 shutdown_connections 2024-09-10T09:56:51.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.558+0000 7f47d6046740 1 --2- 172.21.15.77:0/466569225 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x559b88174430 0x559b88176860 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:51.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.558+0000 7f47d6046740 1 --2- 172.21.15.77:0/466569225 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b88164740 0x559b881620a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:51.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.558+0000 7f47d6046740 1 --2- 172.21.15.77:0/466569225 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b88159e80 0x559b88161b60 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:51.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.558+0000 7f47b2000640 0 client.15777 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T09:56:51.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.558+0000 7f47d6046740 1 --2- 172.21.15.77:0/466569225 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b88159560 0x559b88161620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:51.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.558+0000 7f47d6046740 1 -- 172.21.15.77:0/466569225 >> 172.21.15.77:0/466569225 conn(0x559b88155af0 msgr2=0x559b8815b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:51.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.558+0000 7f47d6046740 1 -- 172.21.15.77:0/466569225 shutdown_connections 2024-09-10T09:56:51.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.558+0000 7f47d6046740 1 -- 172.21.15.77:0/466569225 wait complete. 2024-09-10T09:56:51.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.558+0000 7f47d6046740 20 client.15777 trim_cache size 0 max 16384 2024-09-10T09:56:51.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.576+0000 7f47d6046740 1 Processor -- start 2024-09-10T09:56:51.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.576+0000 7f47d6046740 1 -- start start 2024-09-10T09:56:51.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.576+0000 7f47d6046740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b88159560 0x559b88160560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:51.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.576+0000 7f47d6046740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b88159e80 0x559b8817c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:51.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.576+0000 7f47d6046740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b88164740 0x559b8817c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:51.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.576+0000 7f47d6046740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f47cc1b0e10 con 0x559b88159560 2024-09-10T09:56:51.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.576+0000 7f47d6046740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f47cc1b2950 con 0x559b88164740 2024-09-10T09:56:51.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.576+0000 7f47d6046740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f47cc1b14e0 con 0x559b88159e80 2024-09-10T09:56:51.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.577+0000 7f47d1600640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b88159560 0x559b88160560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:51.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.577+0000 7f47cbe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b88159e80 0x559b8817c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:51.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.577+0000 7f47cbe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b88159e80 0x559b8817c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:59648/0 (socket says 172.21.15.77:59648) 2024-09-10T09:56:51.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.577+0000 7f47cbe00640 1 -- 172.21.15.77:0/104754298 learned_addr learned my addr 172.21.15.77:0/104754298 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:51.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.577+0000 7f47d1600640 1 --2- 172.21.15.77:0/104754298 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b88164740 0x559b8817c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:51.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.577+0000 7f47d1600640 1 -- 172.21.15.77:0/104754298 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b88159e80 msgr2=0x559b8817c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:51.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.577+0000 7f47d1600640 1 --2- 172.21.15.77:0/104754298 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b88159e80 0x559b8817c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:51.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.577+0000 7f47d1600640 1 -- 172.21.15.77:0/104754298 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b88164740 msgr2=0x559b8817c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:51.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.577+0000 7f47d1600640 1 --2- 172.21.15.77:0/104754298 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b88164740 0x559b8817c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:51.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.577+0000 7f47d1600640 1 -- 172.21.15.77:0/104754298 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b88176ee0 con 0x559b88159560 2024-09-10T09:56:51.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.577+0000 7f47cbe00640 1 --2- 172.21.15.77:0/104754298 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b88159e80 0x559b8817c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:56:51.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.577+0000 7f47d1600640 1 --2- 172.21.15.77:0/104754298 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b88159560 0x559b88160560 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x559b881770e0 tx=0x7f47c4009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90e4c525f305cf27 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:51.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.577+0000 7f47bd200640 1 -- 172.21.15.77:0/104754298 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c401e040 con 0x559b88159560 2024-09-10T09:56:51.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.578+0000 7f47bd200640 1 -- 172.21.15.77:0/104754298 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47c4004730 con 0x559b88159560 2024-09-10T09:56:51.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.578+0000 7f47bd200640 1 -- 172.21.15.77:0/104754298 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c4019430 con 0x559b88159560 2024-09-10T09:56:51.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.579+0000 7f47d6046740 1 -- 172.21.15.77:0/104754298 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b88159560 msgr2=0x559b88160560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:51.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.579+0000 7f47d6046740 1 --2- 172.21.15.77:0/104754298 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b88159560 0x559b88160560 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x559b881770e0 tx=0x7f47c4009500 comp rx=0 tx=0).stop 2024-09-10T09:56:51.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.579+0000 7f47d6046740 1 -- 172.21.15.77:0/104754298 shutdown_connections 2024-09-10T09:56:51.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.579+0000 7f47d6046740 1 --2- 172.21.15.77:0/104754298 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b88164740 0x559b8817c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:51.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.579+0000 7f47d6046740 1 --2- 172.21.15.77:0/104754298 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b88159e80 0x559b8817c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:51.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.579+0000 7f47d6046740 1 --2- 172.21.15.77:0/104754298 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b88159560 0x559b88160560 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:51.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.579+0000 7f47d6046740 1 -- 172.21.15.77:0/104754298 >> 172.21.15.77:0/104754298 conn(0x559b88155af0 msgr2=0x559b8815b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:51.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.579+0000 7f47d6046740 1 -- 172.21.15.77:0/104754298 shutdown_connections 2024-09-10T09:56:51.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.579+0000 7f47d6046740 1 -- 172.21.15.77:0/104754298 wait complete. 2024-09-10T09:56:51.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.579+0000 7f47d6046740 1 -- start start 2024-09-10T09:56:51.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.579+0000 7f47d6046740 1 Processor -- start 2024-09-10T09:56:51.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.580+0000 7f47d6046740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b88159560 0x559b8816cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:51.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.580+0000 7f47d6046740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b88159e80 0x559b88169680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:51.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.580+0000 7f47d6046740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b88164740 0x559b88169bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:51.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.580+0000 7f47d6046740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f47cc1b2280 con 0x559b88159560 2024-09-10T09:56:51.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.580+0000 7f47d6046740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f47cc1b1bb0 con 0x559b88164740 2024-09-10T09:56:51.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.580+0000 7f47d6046740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f47cc1aedf0 con 0x559b88159e80 2024-09-10T09:56:51.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.580+0000 7f47d6046740 10 client.0 _ll_register_callbacks cb 0x7f47b8009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:56:51.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.580+0000 7f47d1600640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b88159560 0x559b8816cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:51.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.580+0000 7f47cbe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b88159e80 0x559b88169680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:51.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.580+0000 7f47d1600640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b88159560 0x559b8816cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:55722/0 (socket says 172.21.15.77:55722) 2024-09-10T09:56:51.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.580+0000 7f47cbe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b88159e80 0x559b88169680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:59660/0 (socket says 172.21.15.77:59660) 2024-09-10T09:56:51.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.580+0000 7f47d1600640 1 -- 172.21.15.77:0/3345720665 learned_addr learned my addr 172.21.15.77:0/3345720665 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:51.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.580+0000 7f47d1600640 1 --2- 172.21.15.77:0/3345720665 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b88164740 0x559b88169bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:51.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.581+0000 7f47cbe00640 1 -- 172.21.15.77:0/3345720665 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b88164740 msgr2=0x559b88169bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:51.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.581+0000 7f47cbe00640 1 --2- 172.21.15.77:0/3345720665 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b88164740 0x559b88169bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:51.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.581+0000 7f47cbe00640 1 -- 172.21.15.77:0/3345720665 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b88159560 msgr2=0x559b8816cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:51.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.581+0000 7f47cbe00640 1 --2- 172.21.15.77:0/3345720665 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b88159560 0x559b8816cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:51.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.581+0000 7f47cbe00640 1 -- 172.21.15.77:0/3345720665 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b88155920 con 0x559b88159e80 2024-09-10T09:56:51.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.581+0000 7f47d1600640 1 --2- 172.21.15.77:0/3345720665 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b88159560 0x559b8816cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:56:51.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.581+0000 7f47cbe00640 1 --2- 172.21.15.77:0/3345720665 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b88159e80 0x559b88169680 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f47b801c890 tx=0x7f47b801cdb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e71716c01d5492ad server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:51.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.581+0000 7f47b2000640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-10T09:56:51.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.581+0000 7f47b2000640 1 -- 172.21.15.77:0/3345720665 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b802f060 con 0x559b88159e80 2024-09-10T09:56:51.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.581+0000 7f47d6046740 1 -- 172.21.15.77:0/3345720665 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47c4004730 con 0x559b88159e80 2024-09-10T09:56:51.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.581+0000 7f47b2000640 1 -- 172.21.15.77:0/3345720665 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f47b801c630 con 0x559b88159e80 2024-09-10T09:56:51.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.582+0000 7f47b2000640 1 -- 172.21.15.77:0/3345720665 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f47b800fe60 con 0x559b88159e80 2024-09-10T09:56:51.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.582+0000 7f47d6046740 10 client.25813 fetch_fsmap learned FSMap version 29 2024-09-10T09:56:51.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.582+0000 7f47d6046740 1 -- 172.21.15.77:0/3345720665 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559b881dfc50 con 0x559b88159e80 2024-09-10T09:56:51.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.582+0000 7f47b2000640 1 -- 172.21.15.77:0/3345720665 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47c4004730 con 0x559b88159e80 2024-09-10T09:56:51.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.582+0000 7f47b2000640 1 -- 172.21.15.77:0/3345720665 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f47b8003aa0 con 0x559b88159e80 2024-09-10T09:56:51.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.582+0000 7f47d6046740 10 client.25813 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T09:56:51.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.582+0000 7f47d6046740 10 client.25813 resolve_mds: validated gid 24563 aka daemon mds.b 2024-09-10T09:56:51.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.582+0000 7f47d6046740 20 client.25813 populate_metadata read hostname 'smithi077' 2024-09-10T09:56:51.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.582+0000 7f47d6046740 1 --2- 172.21.15.77:0/3345720665 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x559b881e03e0 0x559b881e2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:51.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.582+0000 7f47d6046740 4 client.25813 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-09-10T09:56:51.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.582+0000 7f47d6046740 1 -- 172.21.15.77:0/3345720665 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x559b880ccc60 con 0x559b881e03e0 2024-09-10T09:56:51.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.583+0000 7f47d1600640 1 --2- 172.21.15.77:0/3345720665 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x559b881e03e0 0x559b881e2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:51.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.583+0000 7f47d1600640 1 --2- 172.21.15.77:0/3345720665 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x559b881e03e0 0x559b881e2800 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3787dc2a92f0200b server_cookie=4ddd226aaa89c8c0 in_seq=0 out_seq=0 2024-09-10T09:56:51.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.583+0000 7f47b2000640 10 client.25813 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T09:56:51.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.592+0000 7f47b2000640 1 -- 172.21.15.77:0/3345720665 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x559b880ccc60 con 0x559b881e03e0 2024-09-10T09:56:51.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.592+0000 7f47b2000640 10 client.25813 handle_command_reply: tid=0 2024-09-10T09:56:51.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.592+0000 7f47d6046740 1 client.25813 shutdown 2024-09-10T09:56:51.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.592+0000 7f47d6046740 1 -- 172.21.15.77:0/3345720665 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b88159e80 msgr2=0x559b88169680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:51.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.592+0000 7f47d6046740 1 --2- 172.21.15.77:0/3345720665 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b88159e80 0x559b88169680 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f47b801c890 tx=0x7f47b801cdb0 comp rx=0 tx=0).stop 2024-09-10T09:56:51.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.592+0000 7f47d6046740 1 -- 172.21.15.77:0/3345720665 shutdown_connections 2024-09-10T09:56:51.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.592+0000 7f47d6046740 1 --2- 172.21.15.77:0/3345720665 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x559b881e03e0 0x559b881e2800 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:51.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.592+0000 7f47d6046740 1 --2- 172.21.15.77:0/3345720665 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b88164740 0x559b88169bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:51.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.592+0000 7f47d6046740 1 --2- 172.21.15.77:0/3345720665 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b88159e80 0x559b88169680 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:51.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.592+0000 7f47b2000640 0 client.25813 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T09:56:51.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.592+0000 7f47d6046740 1 --2- 172.21.15.77:0/3345720665 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b88159560 0x559b8816cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:51.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.592+0000 7f47d6046740 1 -- 172.21.15.77:0/3345720665 >> 172.21.15.77:0/3345720665 conn(0x559b88155af0 msgr2=0x559b8815afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:51.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.593+0000 7f47d6046740 1 -- 172.21.15.77:0/3345720665 shutdown_connections 2024-09-10T09:56:51.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.593+0000 7f47d6046740 1 -- 172.21.15.77:0/3345720665 wait complete. 2024-09-10T09:56:51.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.593+0000 7f47d6046740 20 client.25813 trim_cache size 0 max 16384 2024-09-10T09:56:51.596 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-10T09:56:51.596 INFO:teuthology.orchestra.run.smithi077.stdout: "epoch": 29, 2024-09-10T09:56:51.596 INFO:teuthology.orchestra.run.smithi077.stdout: "leader": 24563, 2024-09-10T09:56:51.596 INFO:teuthology.orchestra.run.smithi077.stdout: "set_version": 6, 2024-09-10T09:56:51.596 INFO:teuthology.orchestra.run.smithi077.stdout: "sets": { 2024-09-10T09:56:51.596 INFO:teuthology.orchestra.run.smithi077.stdout: "b59524e9": { 2024-09-10T09:56:51.596 INFO:teuthology.orchestra.run.smithi077.stdout: "version": 6, 2024-09-10T09:56:51.596 INFO:teuthology.orchestra.run.smithi077.stdout: "age_ref": 0.0, 2024-09-10T09:56:51.596 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-10T09:56:51.596 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "QUIESCED", 2024-09-10T09:56:51.597 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-10T09:56:51.597 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T09:56:51.597 INFO:teuthology.orchestra.run.smithi077.stdout: "timeout": 90.0, 2024-09-10T09:56:51.597 INFO:teuthology.orchestra.run.smithi077.stdout: "expiration": 150.0, 2024-09-10T09:56:51.597 INFO:teuthology.orchestra.run.smithi077.stdout: "members": { 2024-09-10T09:56:51.597 INFO:teuthology.orchestra.run.smithi077.stdout: "file:/": { 2024-09-10T09:56:51.597 INFO:teuthology.orchestra.run.smithi077.stdout: "excluded": false, 2024-09-10T09:56:51.597 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-10T09:56:51.597 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "QUIESCED", 2024-09-10T09:56:51.597 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-10T09:56:51.597 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T09:56:51.597 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T09:56:51.597 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T09:56:51.598 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T09:56:51.598 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T09:56:51.598 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-10T09:56:51.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.593+0000 7f47d2000640 1 -- 172.21.15.77:0/867938300 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f47b40a2b80 msgr2=0x7f47b40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:51.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.593+0000 7f47d2000640 1 --2- 172.21.15.77:0/867938300 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f47b40a2b80 0x7f47b40a5040 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f47c00b8430 tx=0x7f47c00f5530 comp rx=0 tx=0).stop 2024-09-10T09:56:51.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.593+0000 7f47d2000640 1 -- 172.21.15.77:0/867938300 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47cc103e30 msgr2=0x7f47cc19a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:51.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.593+0000 7f47d2000640 1 --2- 172.21.15.77:0/867938300 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47cc103e30 0x7f47cc19a4d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f47c0004770 tx=0x7f47c0003c10 comp rx=0 tx=0).stop 2024-09-10T09:56:51.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.593+0000 7f47d2000640 1 -- 172.21.15.77:0/867938300 shutdown_connections 2024-09-10T09:56:51.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.593+0000 7f47d2000640 1 --2- 172.21.15.77:0/867938300 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f47b40a2b80 0x7f47b40a5040 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:51.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.593+0000 7f47d2000640 1 --2- 172.21.15.77:0/867938300 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f47cc108f60 0x7f47cc19eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:51.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.593+0000 7f47d2000640 1 --2- 172.21.15.77:0/867938300 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f47cc1047e0 0x7f47cc19aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:51.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.593+0000 7f47d2000640 1 --2- 172.21.15.77:0/867938300 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47cc103e30 0x7f47cc19a4d0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:51.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.593+0000 7f47d2000640 1 -- 172.21.15.77:0/867938300 >> 172.21.15.77:0/867938300 conn(0x7f47cc0ff910 msgr2=0x7f47cc100bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:51.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.594+0000 7f47d2000640 1 -- 172.21.15.77:0/867938300 shutdown_connections 2024-09-10T09:56:51.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:51.594+0000 7f47d2000640 1 -- 172.21.15.77:0/867938300 wait complete. 2024-09-10T09:56:51.934 INFO:tasks.workunit.client.0.smithi077.stderr:Note: switching to '1cf819b7212a6048e92fc78c4eff963344f6d56b'. 2024-09-10T09:56:51.935 INFO:tasks.workunit.client.0.smithi077.stderr: 2024-09-10T09:56:51.935 INFO:tasks.workunit.client.0.smithi077.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-10T09:56:51.935 INFO:tasks.workunit.client.0.smithi077.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-10T09:56:51.935 INFO:tasks.workunit.client.0.smithi077.stderr:state without impacting any branches by switching back to a branch. 2024-09-10T09:56:51.935 INFO:tasks.workunit.client.0.smithi077.stderr: 2024-09-10T09:56:51.935 INFO:tasks.workunit.client.0.smithi077.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-10T09:56:51.935 INFO:tasks.workunit.client.0.smithi077.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-10T09:56:51.935 INFO:tasks.workunit.client.0.smithi077.stderr: 2024-09-10T09:56:51.935 INFO:tasks.workunit.client.0.smithi077.stderr: git switch -c 2024-09-10T09:56:51.935 INFO:tasks.workunit.client.0.smithi077.stderr: 2024-09-10T09:56:51.935 INFO:tasks.workunit.client.0.smithi077.stderr:Or undo this operation with: 2024-09-10T09:56:51.935 INFO:tasks.workunit.client.0.smithi077.stderr: 2024-09-10T09:56:51.935 INFO:tasks.workunit.client.0.smithi077.stderr: git switch - 2024-09-10T09:56:51.936 INFO:tasks.workunit.client.0.smithi077.stderr: 2024-09-10T09:56:51.936 INFO:tasks.workunit.client.0.smithi077.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-10T09:56:51.936 INFO:tasks.workunit.client.0.smithi077.stderr: 2024-09-10T09:56:51.936 INFO:tasks.workunit.client.0.smithi077.stderr:HEAD is now at 1cf819b7212 Merge PR #59672 into wip-rishabh-testing-20240909.165402-squid 2024-09-10T09:56:51.952 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:56:51.953 DEBUG:teuthology.orchestra.run.smithi077:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-10T09:56:52.064 INFO:tasks.workunit.client.0.smithi077.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-10T09:56:52.069 INFO:tasks.workunit.client.0.smithi077.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-10T09:56:52.069 INFO:tasks.workunit.client.0.smithi077.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-10T09:56:52.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:56:51 smithi134 ceph-mon[35674]: pgmap v442: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-10T09:56:52.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:56:51 smithi120 ceph-mon[34770]: pgmap v442: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-10T09:56:52.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:56:51 smithi077 ceph-mon[30319]: pgmap v442: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-10T09:56:52.733 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'b59524e9', quiesce took 6.5 seconds. Will release after: 23.5 2024-09-10T09:56:53.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.437+0000 7f8ed3e00640 1 -- 172.21.15.77:0/4283916706 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ed4102b90 msgr2=0x7f8ed4102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:53.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.437+0000 7f8ed3e00640 1 --2- 172.21.15.77:0/4283916706 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ed4102b90 0x7f8ed4102f70 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f8ec4009900 tx=0x7f8ec402a480 comp rx=0 tx=0).stop 2024-09-10T09:56:53.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.437+0000 7f8ed3e00640 1 -- 172.21.15.77:0/4283916706 shutdown_connections 2024-09-10T09:56:53.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.437+0000 7f8ed3e00640 1 --2- 172.21.15.77:0/4283916706 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ed410bfc0 0x7f8ed410e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:53.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.437+0000 7f8ed3e00640 1 --2- 172.21.15.77:0/4283916706 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8ed41034b0 0x7f8ed410b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:53.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.437+0000 7f8ed3e00640 1 --2- 172.21.15.77:0/4283916706 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ed4102b90 0x7f8ed4102f70 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:53.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.437+0000 7f8ed3e00640 1 -- 172.21.15.77:0/4283916706 >> 172.21.15.77:0/4283916706 conn(0x7f8ed40fc610 msgr2=0x7f8ed40fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:53.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.438+0000 7f8ed3e00640 1 -- 172.21.15.77:0/4283916706 shutdown_connections 2024-09-10T09:56:53.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.438+0000 7f8ed3e00640 1 -- 172.21.15.77:0/4283916706 wait complete. 2024-09-10T09:56:53.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.439+0000 7f8ed3e00640 1 Processor -- start 2024-09-10T09:56:53.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.439+0000 7f8ed3e00640 1 -- start start 2024-09-10T09:56:53.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.439+0000 7f8ed3e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ed4102b90 0x7f8ed419a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:53.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.439+0000 7f8ed3e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8ed41034b0 0x7f8ed419aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:53.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.439+0000 7f8ed3e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ed410bfc0 0x7f8ed419ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:53.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.439+0000 7f8ed3e00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed4111e10 con 0x7f8ed410bfc0 2024-09-10T09:56:53.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.439+0000 7f8ed3e00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed4111c90 con 0x7f8ed41034b0 2024-09-10T09:56:53.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.439+0000 7f8ed3e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed4111f90 con 0x7f8ed4102b90 2024-09-10T09:56:53.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.439+0000 7f8ed2a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ed4102b90 0x7f8ed419a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:53.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.439+0000 7f8ed2a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ed4102b90 0x7f8ed419a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:44364/0 (socket says 172.21.15.77:44364) 2024-09-10T09:56:53.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.439+0000 7f8ed2a00640 1 -- 172.21.15.77:0/3874340279 learned_addr learned my addr 172.21.15.77:0/3874340279 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:53.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.439+0000 7f8ed3400640 1 --2- 172.21.15.77:0/3874340279 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ed410bfc0 0x7f8ed419ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:53.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.439+0000 7f8ed2000640 1 --2- 172.21.15.77:0/3874340279 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8ed41034b0 0x7f8ed419aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:53.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.440+0000 7f8ed2a00640 1 -- 172.21.15.77:0/3874340279 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8ed41034b0 msgr2=0x7f8ed419aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:53.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.440+0000 7f8ed2a00640 1 --2- 172.21.15.77:0/3874340279 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8ed41034b0 0x7f8ed419aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:53.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.440+0000 7f8ed2a00640 1 -- 172.21.15.77:0/3874340279 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ed410bfc0 msgr2=0x7f8ed419ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:53.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.440+0000 7f8ed2a00640 1 --2- 172.21.15.77:0/3874340279 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ed410bfc0 0x7f8ed419ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:53.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.440+0000 7f8ed2a00640 1 -- 172.21.15.77:0/3874340279 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ed419f4c0 con 0x7f8ed4102b90 2024-09-10T09:56:53.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.440+0000 7f8ed3400640 1 --2- 172.21.15.77:0/3874340279 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ed410bfc0 0x7f8ed419ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:56:53.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.440+0000 7f8ed2000640 1 --2- 172.21.15.77:0/3874340279 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8ed41034b0 0x7f8ed419aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:56:53.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.440+0000 7f8ed2a00640 1 --2- 172.21.15.77:0/3874340279 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ed4102b90 0x7f8ed419a510 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8ec4004770 tx=0x7f8ec4003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:53.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.440+0000 7f8ecb400640 1 -- 172.21.15.77:0/3874340279 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec403f070 con 0x7f8ed4102b90 2024-09-10T09:56:53.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.441+0000 7f8ed3e00640 1 -- 172.21.15.77:0/3874340279 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ed419f750 con 0x7f8ed4102b90 2024-09-10T09:56:53.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.441+0000 7f8ecb400640 1 -- 172.21.15.77:0/3874340279 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ec4031ca0 con 0x7f8ed4102b90 2024-09-10T09:56:53.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.441+0000 7f8ecb400640 1 -- 172.21.15.77:0/3874340279 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec4035030 con 0x7f8ed4102b90 2024-09-10T09:56:53.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.441+0000 7f8ed3e00640 1 -- 172.21.15.77:0/3874340279 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ed41a7090 con 0x7f8ed4102b90 2024-09-10T09:56:53.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.443+0000 7f8edc486740 1 Processor -- start 2024-09-10T09:56:53.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.444+0000 7f8edc486740 1 -- start start 2024-09-10T09:56:53.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.444+0000 7f8ecb400640 1 -- 172.21.15.77:0/3874340279 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8ec4042070 con 0x7f8ed4102b90 2024-09-10T09:56:53.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.444+0000 7f8edc486740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5608923e2e40 0x5608923e3220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:53.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.444+0000 7f8edc486740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5608923e3760 0x5608923edae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:53.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.444+0000 7f8edc486740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608923ee020 0x5608923f0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:53.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.444+0000 7f8edc486740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5608923493e0 con 0x5608923e3760 2024-09-10T09:56:53.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.444+0000 7f8edc486740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5608922f0d80 con 0x5608923e2e40 2024-09-10T09:56:53.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.444+0000 7f8edc486740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5608923453a0 con 0x5608923ee020 2024-09-10T09:56:53.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.444+0000 7f8ed2a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5608923e3760 0x5608923edae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:53.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.444+0000 7f8ed2a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5608923e3760 0x5608923edae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:41832/0 (socket says 172.21.15.77:41832) 2024-09-10T09:56:53.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.444+0000 7f8ed2a00640 1 -- 172.21.15.77:0/3232454331 learned_addr learned my addr 172.21.15.77:0/3232454331 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:53.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.444+0000 7f8ed3400640 1 --2- 172.21.15.77:0/3232454331 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5608923e2e40 0x5608923e3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:53.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.444+0000 7f8ed2000640 1 --2- 172.21.15.77:0/3232454331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608923ee020 0x5608923f0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:53.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.445+0000 7f8ed2a00640 1 -- 172.21.15.77:0/3232454331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608923ee020 msgr2=0x5608923f0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:53.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.445+0000 7f8ed2a00640 1 --2- 172.21.15.77:0/3232454331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608923ee020 0x5608923f0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:53.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.445+0000 7f8ed2a00640 1 -- 172.21.15.77:0/3232454331 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5608923e2e40 msgr2=0x5608923e3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:53.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.445+0000 7f8ed2a00640 1 --2- 172.21.15.77:0/3232454331 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5608923e2e40 0x5608923e3220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:53.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.445+0000 7f8ed2a00640 1 -- 172.21.15.77:0/3232454331 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56089230fae0 con 0x5608923e3760 2024-09-10T09:56:53.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.445+0000 7f8ed3400640 1 --2- 172.21.15.77:0/3232454331 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5608923e2e40 0x5608923e3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:56:53.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.445+0000 7f8ed2a00640 1 --2- 172.21.15.77:0/3232454331 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5608923e3760 0x5608923edae0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f8ec40f5bf0 tx=0x7f8ec40f5ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=845a4370729e709 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:53.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.445+0000 7f8ebfe00640 1 -- 172.21.15.77:0/3232454331 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec40fe070 con 0x5608923e3760 2024-09-10T09:56:53.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.445+0000 7f8ebfe00640 1 -- 172.21.15.77:0/3232454331 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ec40efe50 con 0x5608923e3760 2024-09-10T09:56:53.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.445+0000 7f8ebfe00640 1 -- 172.21.15.77:0/3232454331 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec40f95e0 con 0x5608923e3760 2024-09-10T09:56:53.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.446+0000 7f8ecb400640 1 --2- 172.21.15.77:0/3874340279 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f8eb80a2c00 0x7f8eb80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:53.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.446+0000 7f8ecb400640 1 -- 172.21.15.77:0/3874340279 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f8ec40efa60 con 0x7f8ed4102b90 2024-09-10T09:56:53.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.446+0000 7f8edc486740 1 -- 172.21.15.77:0/3232454331 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5608923e3760 msgr2=0x5608923edae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:53.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.446+0000 7f8edc486740 1 --2- 172.21.15.77:0/3232454331 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5608923e3760 0x5608923edae0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f8ec40f5bf0 tx=0x7f8ec40f5ce0 comp rx=0 tx=0).stop 2024-09-10T09:56:53.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.446+0000 7f8ed3400640 1 --2- 172.21.15.77:0/3874340279 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f8eb80a2c00 0x7f8eb80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:53.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.446+0000 7f8edc486740 1 -- 172.21.15.77:0/3232454331 shutdown_connections 2024-09-10T09:56:53.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.446+0000 7f8edc486740 1 --2- 172.21.15.77:0/3232454331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608923ee020 0x5608923f0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:53.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.446+0000 7f8edc486740 1 --2- 172.21.15.77:0/3232454331 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5608923e3760 0x5608923edae0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:53.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.446+0000 7f8edc486740 1 --2- 172.21.15.77:0/3232454331 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5608923e2e40 0x5608923e3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:53.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.446+0000 7f8edc486740 1 -- 172.21.15.77:0/3232454331 >> 172.21.15.77:0/3232454331 conn(0x5608923df3d0 msgr2=0x5608923df7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:53.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.446+0000 7f8edc486740 1 -- 172.21.15.77:0/3232454331 shutdown_connections 2024-09-10T09:56:53.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.447+0000 7f8edc486740 1 -- 172.21.15.77:0/3232454331 wait complete. 2024-09-10T09:56:53.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.447+0000 7f8ed3400640 1 --2- 172.21.15.77:0/3874340279 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f8eb80a2c00 0x7f8eb80a50c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc005fd0 tx=0x7f8ecc009600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:53.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.447+0000 7f8edc486740 1 -- start start 2024-09-10T09:56:53.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.447+0000 7f8edc486740 1 Processor -- start 2024-09-10T09:56:53.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.448+0000 7f8edc486740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608923e2e40 0x5608923eaea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:53.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.448+0000 7f8edc486740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5608923e3760 0x5608923eb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:53.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.448+0000 7f8edc486740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5608923ee020 0x5608923eb920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:53.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.448+0000 7f8edc486740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5608923f09b0 con 0x5608923e3760 2024-09-10T09:56:53.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.448+0000 7f8edc486740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5608923040f0 con 0x5608923ee020 2024-09-10T09:56:53.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.448+0000 7f8edc486740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5608923f0b30 con 0x5608923e2e40 2024-09-10T09:56:53.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.448+0000 7f8edc486740 10 client.0 _ll_register_callbacks cb 0x5608922ebbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:56:53.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.448+0000 7f8ed2000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5608923e3760 0x5608923eb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:53.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.448+0000 7f8ed2000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5608923e3760 0x5608923eb3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:41848/0 (socket says 172.21.15.77:41848) 2024-09-10T09:56:53.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.448+0000 7f8ed2000640 1 -- 172.21.15.77:0/4223559884 learned_addr learned my addr 172.21.15.77:0/4223559884 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:53.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.448+0000 7f8ed3400640 1 --2- 172.21.15.77:0/4223559884 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5608923ee020 0x5608923eb920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:53.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.448+0000 7f8ed2a00640 1 --2- 172.21.15.77:0/4223559884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608923e2e40 0x5608923eaea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:53.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.448+0000 7f8ed2000640 1 -- 172.21.15.77:0/4223559884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608923e2e40 msgr2=0x5608923eaea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:53.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.448+0000 7f8ed2000640 1 --2- 172.21.15.77:0/4223559884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608923e2e40 0x5608923eaea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:53.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.448+0000 7f8ed2000640 1 -- 172.21.15.77:0/4223559884 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5608923ee020 msgr2=0x5608923eb920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:53.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.448+0000 7f8ed2000640 1 --2- 172.21.15.77:0/4223559884 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5608923ee020 0x5608923eb920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:53.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.448+0000 7f8ed2000640 1 -- 172.21.15.77:0/4223559884 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608920ba3c0 con 0x5608923e3760 2024-09-10T09:56:53.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.449+0000 7f8ed2000640 1 --2- 172.21.15.77:0/4223559884 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5608923e3760 0x5608923eb3e0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f8ec0012d50 tx=0x7f8ec000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=660576ac4950029c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:53.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.449+0000 7f8ebcc00640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-10T09:56:53.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.449+0000 7f8ebcc00640 1 -- 172.21.15.77:0/4223559884 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec0012110 con 0x5608923e3760 2024-09-10T09:56:53.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.449+0000 7f8edc486740 1 -- 172.21.15.77:0/4223559884 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ec40efe50 con 0x5608923e3760 2024-09-10T09:56:53.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.449+0000 7f8ebcc00640 1 -- 172.21.15.77:0/4223559884 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f8ec0004050 con 0x5608923e3760 2024-09-10T09:56:53.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.449+0000 7f8ebcc00640 1 -- 172.21.15.77:0/4223559884 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8ec0025050 con 0x5608923e3760 2024-09-10T09:56:53.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.449+0000 7f8edc486740 10 client.15795 fetch_fsmap learned FSMap version 29 2024-09-10T09:56:53.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.449+0000 7f8edc486740 1 -- 172.21.15.77:0/4223559884 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5608923fc950 con 0x5608923e3760 2024-09-10T09:56:53.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.450+0000 7f8ebcc00640 1 -- 172.21.15.77:0/4223559884 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ec40efe50 con 0x5608923e3760 2024-09-10T09:56:53.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.450+0000 7f8ebcc00640 1 -- 172.21.15.77:0/4223559884 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f8ec0010430 con 0x5608923e3760 2024-09-10T09:56:53.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.450+0000 7f8edc486740 10 client.15795 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T09:56:53.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.450+0000 7f8edc486740 10 client.15795 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-10T09:56:53.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.450+0000 7f8edc486740 20 client.15795 populate_metadata read hostname 'smithi077' 2024-09-10T09:56:53.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.450+0000 7f8edc486740 1 --2- 172.21.15.77:0/4223559884 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x5608923fde60 0x560892400250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:53.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.450+0000 7f8edc486740 4 client.15795 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:56:53.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.450+0000 7f8edc486740 1 -- 172.21.15.77:0/4223559884 --> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560892356950 con 0x5608923fde60 2024-09-10T09:56:53.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.450+0000 7f8ed2a00640 1 --2- 172.21.15.77:0/4223559884 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x5608923fde60 0x560892400250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:53.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.451+0000 7f8ebcc00640 10 client.15795 ms_handle_connect on v2:172.21.15.120:6836/3033280709 2024-09-10T09:56:53.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.451+0000 7f8ed2a00640 1 --2- 172.21.15.77:0/4223559884 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x5608923fde60 0x560892400250 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f15c3ba7f7340046 server_cookie=95a4bbd60dbf32e6 in_seq=0 out_seq=0 2024-09-10T09:56:53.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.452+0000 7f8ebcc00640 1 -- 172.21.15.77:0/4223559884 <== mds.1 v2:172.21.15.120:6836/3033280709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560892356950 con 0x5608923fde60 2024-09-10T09:56:53.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.452+0000 7f8ebcc00640 10 client.15795 handle_command_reply: tid=0 2024-09-10T09:56:53.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.452+0000 7f8edc486740 1 client.15795 shutdown 2024-09-10T09:56:53.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.452+0000 7f8edc486740 1 -- 172.21.15.77:0/4223559884 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5608923e3760 msgr2=0x5608923eb3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:53.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.452+0000 7f8edc486740 1 --2- 172.21.15.77:0/4223559884 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5608923e3760 0x5608923eb3e0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f8ec0012d50 tx=0x7f8ec000f790 comp rx=0 tx=0).stop 2024-09-10T09:56:53.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.452+0000 7f8edc486740 1 -- 172.21.15.77:0/4223559884 shutdown_connections 2024-09-10T09:56:53.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.452+0000 7f8edc486740 1 --2- 172.21.15.77:0/4223559884 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x5608923fde60 0x560892400250 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:53.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.452+0000 7f8edc486740 1 --2- 172.21.15.77:0/4223559884 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5608923ee020 0x5608923eb920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:53.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.452+0000 7f8edc486740 1 --2- 172.21.15.77:0/4223559884 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5608923e3760 0x5608923eb3e0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:53.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.452+0000 7f8ebcc00640 0 client.15795 ms_handle_reset on v2:172.21.15.120:6836/3033280709 2024-09-10T09:56:53.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.452+0000 7f8edc486740 1 --2- 172.21.15.77:0/4223559884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608923e2e40 0x5608923eaea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:53.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.452+0000 7f8edc486740 1 -- 172.21.15.77:0/4223559884 >> 172.21.15.77:0/4223559884 conn(0x5608923df3d0 msgr2=0x5608923e5100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:53.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.452+0000 7f8edc486740 1 -- 172.21.15.77:0/4223559884 shutdown_connections 2024-09-10T09:56:53.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.452+0000 7f8edc486740 1 -- 172.21.15.77:0/4223559884 wait complete. 2024-09-10T09:56:53.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.452+0000 7f8edc486740 20 client.15795 trim_cache size 0 max 16384 2024-09-10T09:56:53.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.466+0000 7f8edc486740 1 Processor -- start 2024-09-10T09:56:53.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.466+0000 7f8edc486740 1 -- start start 2024-09-10T09:56:53.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.466+0000 7f8edc486740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5608923e2e40 0x5608923ea760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:53.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.466+0000 7f8edc486740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608923e3760 0x5608923ece10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:53.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.466+0000 7f8edc486740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5608923ee020 0x560892405b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:53.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.466+0000 7f8edc486740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed41b0eb0 con 0x5608923ee020 2024-09-10T09:56:53.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.466+0000 7f8edc486740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed41b29f0 con 0x5608923e2e40 2024-09-10T09:56:53.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.466+0000 7f8edc486740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed41b1580 con 0x5608923e3760 2024-09-10T09:56:53.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.467+0000 7f8ed3400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5608923ee020 0x560892405b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:53.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.467+0000 7f8ed2000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608923e3760 0x5608923ece10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:53.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.467+0000 7f8ed2a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5608923e2e40 0x5608923ea760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:53.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.467+0000 7f8ed2000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608923e3760 0x5608923ece10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:44400/0 (socket says 172.21.15.77:44400) 2024-09-10T09:56:53.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.467+0000 7f8ed2000640 1 -- 172.21.15.77:0/1343803637 learned_addr learned my addr 172.21.15.77:0/1343803637 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:53.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.467+0000 7f8ed2a00640 1 -- 172.21.15.77:0/1343803637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608923e3760 msgr2=0x5608923ece10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:53.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.467+0000 7f8ed2a00640 1 --2- 172.21.15.77:0/1343803637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608923e3760 0x5608923ece10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:53.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.467+0000 7f8ed2a00640 1 -- 172.21.15.77:0/1343803637 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5608923ee020 msgr2=0x560892405b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:53.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.467+0000 7f8ed2a00640 1 --2- 172.21.15.77:0/1343803637 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5608923ee020 0x560892405b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:53.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.467+0000 7f8ed2a00640 1 -- 172.21.15.77:0/1343803637 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608923e9e50 con 0x5608923e2e40 2024-09-10T09:56:53.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.467+0000 7f8ed2a00640 1 --2- 172.21.15.77:0/1343803637 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5608923e2e40 0x5608923ea760 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8ec4000c00 tx=0x7f8ec403b890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46a9a996bed45815 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:53.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.468+0000 7f8ebf400640 1 -- 172.21.15.77:0/1343803637 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec4049070 con 0x5608923e2e40 2024-09-10T09:56:53.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.468+0000 7f8ebf400640 1 -- 172.21.15.77:0/1343803637 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ec40efe50 con 0x5608923e2e40 2024-09-10T09:56:53.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.468+0000 7f8ebf400640 1 -- 172.21.15.77:0/1343803637 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec40fe070 con 0x5608923e2e40 2024-09-10T09:56:53.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.468+0000 7f8edc486740 1 -- 172.21.15.77:0/1343803637 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5608923e2e40 msgr2=0x5608923ea760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:53.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.468+0000 7f8edc486740 1 --2- 172.21.15.77:0/1343803637 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5608923e2e40 0x5608923ea760 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8ec4000c00 tx=0x7f8ec403b890 comp rx=0 tx=0).stop 2024-09-10T09:56:53.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.468+0000 7f8edc486740 1 -- 172.21.15.77:0/1343803637 shutdown_connections 2024-09-10T09:56:53.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.468+0000 7f8edc486740 1 --2- 172.21.15.77:0/1343803637 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5608923ee020 0x560892405b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:53.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.468+0000 7f8edc486740 1 --2- 172.21.15.77:0/1343803637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608923e3760 0x5608923ece10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:53.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.468+0000 7f8edc486740 1 --2- 172.21.15.77:0/1343803637 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5608923e2e40 0x5608923ea760 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:53.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.468+0000 7f8edc486740 1 -- 172.21.15.77:0/1343803637 >> 172.21.15.77:0/1343803637 conn(0x5608923df3d0 msgr2=0x5608923ec700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:53.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.469+0000 7f8edc486740 1 -- 172.21.15.77:0/1343803637 shutdown_connections 2024-09-10T09:56:53.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.469+0000 7f8edc486740 1 -- 172.21.15.77:0/1343803637 wait complete. 2024-09-10T09:56:53.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.469+0000 7f8edc486740 1 -- start start 2024-09-10T09:56:53.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.469+0000 7f8edc486740 1 Processor -- start 2024-09-10T09:56:53.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.469+0000 7f8edc486740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5608923e2e40 0x5608923f5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:53.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.470+0000 7f8edc486740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5608923e3760 0x5608923f64d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:53.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.470+0000 7f8edc486740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608923ee020 0x5608923f1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:53.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.470+0000 7f8edc486740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed41b2320 con 0x5608923e2e40 2024-09-10T09:56:53.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.470+0000 7f8edc486740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed41b1c50 con 0x5608923e3760 2024-09-10T09:56:53.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.470+0000 7f8edc486740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed41aee90 con 0x5608923ee020 2024-09-10T09:56:53.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.470+0000 7f8edc486740 10 client.0 _ll_register_callbacks cb 0x7f8ec0009cd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:56:53.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.470+0000 7f8ed2000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5608923e3760 0x5608923f64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:53.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.470+0000 7f8ed2a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5608923e2e40 0x5608923f5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:53.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.470+0000 7f8ed2000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5608923e3760 0x5608923f64d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:43832/0 (socket says 172.21.15.77:43832) 2024-09-10T09:56:53.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.470+0000 7f8ed2000640 1 -- 172.21.15.77:0/696248274 learned_addr learned my addr 172.21.15.77:0/696248274 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:53.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.470+0000 7f8ed3400640 1 --2- 172.21.15.77:0/696248274 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608923ee020 0x5608923f1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:53.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.470+0000 7f8ed2000640 1 -- 172.21.15.77:0/696248274 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608923ee020 msgr2=0x5608923f1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:53.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.470+0000 7f8ed2000640 1 --2- 172.21.15.77:0/696248274 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608923ee020 0x5608923f1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:53.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.470+0000 7f8ed2000640 1 -- 172.21.15.77:0/696248274 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5608923e2e40 msgr2=0x5608923f5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:53.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.470+0000 7f8ed2000640 1 --2- 172.21.15.77:0/696248274 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5608923e2e40 0x5608923f5f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:53.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.470+0000 7f8ed2000640 1 -- 172.21.15.77:0/696248274 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608923f1b80 con 0x5608923e3760 2024-09-10T09:56:53.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.470+0000 7f8ed2000640 1 --2- 172.21.15.77:0/696248274 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5608923e3760 0x5608923f64d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8ec0000c00 tx=0x7f8ec0009c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12552d9a62f5929e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:53.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.471+0000 7f8ebcc00640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T09:56:53.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.471+0000 7f8ebcc00640 1 -- 172.21.15.77:0/696248274 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec0022040 con 0x5608923e3760 2024-09-10T09:56:53.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.471+0000 7f8edc486740 1 -- 172.21.15.77:0/696248274 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ec40efe50 con 0x5608923e3760 2024-09-10T09:56:53.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.471+0000 7f8ebcc00640 1 -- 172.21.15.77:0/696248274 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f8ec0010a70 con 0x5608923e3760 2024-09-10T09:56:53.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.471+0000 7f8ebcc00640 1 -- 172.21.15.77:0/696248274 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8ec0039050 con 0x5608923e3760 2024-09-10T09:56:53.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.471+0000 7f8edc486740 10 client.25871 fetch_fsmap learned FSMap version 29 2024-09-10T09:56:53.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.471+0000 7f8edc486740 1 -- 172.21.15.77:0/696248274 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5608924686d0 con 0x5608923e3760 2024-09-10T09:56:53.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.471+0000 7f8ebcc00640 1 -- 172.21.15.77:0/696248274 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ec40efe50 con 0x5608923e3760 2024-09-10T09:56:53.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.471+0000 7f8ebcc00640 1 -- 172.21.15.77:0/696248274 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f8ec00035f0 con 0x5608923e3760 2024-09-10T09:56:53.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.471+0000 7f8edc486740 10 client.25871 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T09:56:53.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.471+0000 7f8edc486740 10 client.25871 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-10T09:56:53.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.471+0000 7f8edc486740 20 client.25871 populate_metadata read hostname 'smithi077' 2024-09-10T09:56:53.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.472+0000 7f8edc486740 1 --2- 172.21.15.77:0/696248274 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x560892468e20 0x56089246b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:53.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.472+0000 7f8edc486740 4 client.25871 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:56:53.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.472+0000 7f8edc486740 1 -- 172.21.15.77:0/696248274 --> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560892350e60 con 0x560892468e20 2024-09-10T09:56:53.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.472+0000 7f8ed2a00640 1 --2- 172.21.15.77:0/696248274 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x560892468e20 0x56089246b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:53.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.472+0000 7f8ed2a00640 1 --2- 172.21.15.77:0/696248274 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x560892468e20 0x56089246b240 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=260441f7aafd6256 server_cookie=906625d482e50c96 in_seq=0 out_seq=0 2024-09-10T09:56:53.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.472+0000 7f8ebcc00640 10 client.25871 ms_handle_connect on v2:172.21.15.120:6836/3033280709 2024-09-10T09:56:53.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.472+0000 7f8ebcc00640 1 -- 172.21.15.77:0/696248274 <== mds.1 v2:172.21.15.120:6836/3033280709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560892350e60 con 0x560892468e20 2024-09-10T09:56:53.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.473+0000 7f8ebcc00640 10 client.25871 handle_command_reply: tid=0 2024-09-10T09:56:53.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.473+0000 7f8edc486740 1 client.25871 shutdown 2024-09-10T09:56:53.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.473+0000 7f8edc486740 1 -- 172.21.15.77:0/696248274 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5608923e3760 msgr2=0x5608923f64d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:53.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.473+0000 7f8edc486740 1 --2- 172.21.15.77:0/696248274 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5608923e3760 0x5608923f64d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8ec0000c00 tx=0x7f8ec0009c60 comp rx=0 tx=0).stop 2024-09-10T09:56:53.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.473+0000 7f8edc486740 1 -- 172.21.15.77:0/696248274 shutdown_connections 2024-09-10T09:56:53.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.473+0000 7f8edc486740 1 --2- 172.21.15.77:0/696248274 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x560892468e20 0x56089246b240 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:53.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.473+0000 7f8edc486740 1 --2- 172.21.15.77:0/696248274 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608923ee020 0x5608923f1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:53.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.473+0000 7f8ebcc00640 0 client.25871 ms_handle_reset on v2:172.21.15.120:6836/3033280709 2024-09-10T09:56:53.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.473+0000 7f8edc486740 1 --2- 172.21.15.77:0/696248274 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5608923e3760 0x5608923f64d0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:53.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.473+0000 7f8edc486740 1 --2- 172.21.15.77:0/696248274 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5608923e2e40 0x5608923f5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:53.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.473+0000 7f8edc486740 1 -- 172.21.15.77:0/696248274 >> 172.21.15.77:0/696248274 conn(0x5608923df3d0 msgr2=0x5608923ecf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:53.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.473+0000 7f8edc486740 1 -- 172.21.15.77:0/696248274 shutdown_connections 2024-09-10T09:56:53.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.473+0000 7f8edc486740 1 -- 172.21.15.77:0/696248274 wait complete. 2024-09-10T09:56:53.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.473+0000 7f8edc486740 20 client.25871 trim_cache size 0 max 16384 2024-09-10T09:56:53.477 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-10T09:56:53.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.474+0000 7f8ed3e00640 1 -- 172.21.15.77:0/3874340279 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f8eb80a2c00 msgr2=0x7f8eb80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:53.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.474+0000 7f8ed3e00640 1 --2- 172.21.15.77:0/3874340279 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f8eb80a2c00 0x7f8eb80a50c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc005fd0 tx=0x7f8ecc009600 comp rx=0 tx=0).stop 2024-09-10T09:56:53.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.474+0000 7f8ed3e00640 1 -- 172.21.15.77:0/3874340279 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ed4102b90 msgr2=0x7f8ed419a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:53.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.474+0000 7f8ed3e00640 1 --2- 172.21.15.77:0/3874340279 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ed4102b90 0x7f8ed419a510 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8ec4004770 tx=0x7f8ec4003c10 comp rx=0 tx=0).stop 2024-09-10T09:56:53.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.474+0000 7f8ed3e00640 1 -- 172.21.15.77:0/3874340279 shutdown_connections 2024-09-10T09:56:53.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.474+0000 7f8ed3e00640 1 --2- 172.21.15.77:0/3874340279 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f8eb80a2c00 0x7f8eb80a50c0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:53.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.474+0000 7f8ed3e00640 1 --2- 172.21.15.77:0/3874340279 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ed410bfc0 0x7f8ed419ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:53.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.474+0000 7f8ed3e00640 1 --2- 172.21.15.77:0/3874340279 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8ed41034b0 0x7f8ed419aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:53.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.474+0000 7f8ed3e00640 1 --2- 172.21.15.77:0/3874340279 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ed4102b90 0x7f8ed419a510 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:53.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.474+0000 7f8ed3e00640 1 -- 172.21.15.77:0/3874340279 >> 172.21.15.77:0/3874340279 conn(0x7f8ed40fc610 msgr2=0x7f8ed40fd210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:53.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.474+0000 7f8ed3e00640 1 -- 172.21.15.77:0/3874340279 shutdown_connections 2024-09-10T09:56:53.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:53.474+0000 7f8ed3e00640 1 -- 172.21.15.77:0/3874340279 wait complete. 2024-09-10T09:56:54.430 INFO:tasks.workunit.client.0.smithi077.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-10T09:56:54.543 INFO:tasks.workunit.client.0.smithi077.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-10T09:56:54.609 INFO:tasks.workunit.client.0.smithi077.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-10T09:56:54.613 INFO:tasks.workunit.client.0.smithi077.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-10T09:56:54.613 INFO:tasks.workunit.client.0.smithi077.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-10T09:56:54.658 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T09:56:54.682 INFO:tasks.workunit.client.0.smithi077.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-10T09:56:54.689 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-09-10T09:56:54.689 DEBUG:teuthology.orchestra.run.smithi077:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-10T09:56:54.766 INFO:tasks.workunit:Running workunits matching suites/fsx.sh on client.0... 2024-09-10T09:56:54.767 INFO:tasks.workunit:Running workunit suites/fsx.sh... 2024-09-10T09:56:54.768 DEBUG:teuthology.orchestra.run.smithi077:workunit test suites/fsx.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1cf819b7212a6048e92fc78c4eff963344f6d56b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh 2024-09-10T09:56:54.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:56:54 smithi077 ceph-mon[30319]: pgmap v443: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-10T09:56:55.038 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:56:55.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:56:54 smithi134 ceph-mon[35674]: pgmap v443: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-10T09:56:55.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:56:54 smithi120 ceph-mon[34770]: pgmap v443: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-10T09:56:55.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.620+0000 7f8623e00640 1 -- 172.21.15.77:0/2102449615 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86241047e0 msgr2=0x7f8624108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:55.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.620+0000 7f8623e00640 1 --2- 172.21.15.77:0/2102449615 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86241047e0 0x7f8624108830 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f8610009900 tx=0x7f861002a410 comp rx=0 tx=0).stop 2024-09-10T09:56:55.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.620+0000 7f8623e00640 1 -- 172.21.15.77:0/2102449615 shutdown_connections 2024-09-10T09:56:55.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.620+0000 7f8623e00640 1 --2- 172.21.15.77:0/2102449615 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8624108f60 0x7f8624110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:55.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.620+0000 7f8623e00640 1 --2- 172.21.15.77:0/2102449615 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86241047e0 0x7f8624108830 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:55.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.620+0000 7f8623e00640 1 --2- 172.21.15.77:0/2102449615 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8624103e30 0x7f8624104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:55.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.620+0000 7f8623e00640 1 -- 172.21.15.77:0/2102449615 >> 172.21.15.77:0/2102449615 conn(0x7f86240ff910 msgr2=0x7f8624101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:55.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.620+0000 7f8623e00640 1 -- 172.21.15.77:0/2102449615 shutdown_connections 2024-09-10T09:56:55.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.620+0000 7f8623e00640 1 -- 172.21.15.77:0/2102449615 wait complete. 2024-09-10T09:56:55.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.621+0000 7f8623e00640 1 Processor -- start 2024-09-10T09:56:55.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.621+0000 7f8623e00640 1 -- start start 2024-09-10T09:56:55.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.622+0000 7f8623e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8624103e30 0x7f862419a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:55.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.622+0000 7f8623e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86241047e0 0x7f862419aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:55.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.622+0000 7f8623e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8624108f60 0x7f862419edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:55.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.622+0000 7f8623e00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8624113fc0 con 0x7f86241047e0 2024-09-10T09:56:55.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.622+0000 7f8623e00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8624113e40 con 0x7f8624108f60 2024-09-10T09:56:55.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.622+0000 7f8623e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8624114140 con 0x7f8624103e30 2024-09-10T09:56:55.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.622+0000 7f8622000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86241047e0 0x7f862419aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:55.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.622+0000 7f8622000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86241047e0 0x7f862419aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:41888/0 (socket says 172.21.15.77:41888) 2024-09-10T09:56:55.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.622+0000 7f8622000640 1 -- 172.21.15.77:0/2640734704 learned_addr learned my addr 172.21.15.77:0/2640734704 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:55.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.622+0000 7f8622a00640 1 --2- 172.21.15.77:0/2640734704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8624103e30 0x7f862419a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:55.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.622+0000 7f8623400640 1 --2- 172.21.15.77:0/2640734704 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8624108f60 0x7f862419edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:55.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.622+0000 7f8622000640 1 -- 172.21.15.77:0/2640734704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8624103e30 msgr2=0x7f862419a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:55.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.622+0000 7f8622000640 1 --2- 172.21.15.77:0/2640734704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8624103e30 0x7f862419a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:55.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.622+0000 7f8622000640 1 -- 172.21.15.77:0/2640734704 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8624108f60 msgr2=0x7f862419edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:55.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.622+0000 7f8622000640 1 --2- 172.21.15.77:0/2640734704 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8624108f60 0x7f862419edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:55.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.622+0000 7f8622000640 1 -- 172.21.15.77:0/2640734704 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f862419f4d0 con 0x7f86241047e0 2024-09-10T09:56:55.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.623+0000 7f8622000640 1 --2- 172.21.15.77:0/2640734704 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86241047e0 0x7f862419aa60 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f8610006010 tx=0x7f8610004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:55.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.623+0000 7f861b400640 1 -- 172.21.15.77:0/2640734704 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f861003f030 con 0x7f86241047e0 2024-09-10T09:56:55.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.623+0000 7f861b400640 1 -- 172.21.15.77:0/2640734704 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8610004440 con 0x7f86241047e0 2024-09-10T09:56:55.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.623+0000 7f8623e00640 1 -- 172.21.15.77:0/2640734704 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f862419f760 con 0x7f86241047e0 2024-09-10T09:56:55.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.623+0000 7f8623e00640 1 -- 172.21.15.77:0/2640734704 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86241a70a0 con 0x7f86241047e0 2024-09-10T09:56:55.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.624+0000 7f861b400640 1 -- 172.21.15.77:0/2640734704 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8610049060 con 0x7f86241047e0 2024-09-10T09:56:55.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.626+0000 7f862c7ca740 1 Processor -- start 2024-09-10T09:56:55.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.627+0000 7f862c7ca740 1 -- start start 2024-09-10T09:56:55.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.627+0000 7f862c7ca740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55908b8cbe40 0x55908b8cc220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:55.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.627+0000 7f862c7ca740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55908b8cc760 0x55908b8d6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:55.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.627+0000 7f862c7ca740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55908b8d7020 0x55908b8d9410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:55.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.627+0000 7f861b400640 1 -- 172.21.15.77:0/2640734704 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8610042030 con 0x7f86241047e0 2024-09-10T09:56:55.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.627+0000 7f862c7ca740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55908b8323e0 con 0x55908b8cbe40 2024-09-10T09:56:55.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.627+0000 7f862c7ca740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55908b7d9d80 con 0x55908b8d7020 2024-09-10T09:56:55.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.627+0000 7f862c7ca740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55908b82e3a0 con 0x55908b8cc760 2024-09-10T09:56:55.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.627+0000 7f8622a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55908b8cc760 0x55908b8d6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:55.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.627+0000 7f8623400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55908b8cbe40 0x55908b8cc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:55.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.627+0000 7f8622a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55908b8cc760 0x55908b8d6ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:44440/0 (socket says 172.21.15.77:44440) 2024-09-10T09:56:55.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.627+0000 7f8622a00640 1 -- 172.21.15.77:0/1812906674 learned_addr learned my addr 172.21.15.77:0/1812906674 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:55.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.627+0000 7f8622000640 1 --2- 172.21.15.77:0/1812906674 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55908b8d7020 0x55908b8d9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:55.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.628+0000 7f8622a00640 1 -- 172.21.15.77:0/1812906674 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55908b8d7020 msgr2=0x55908b8d9410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:55.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.628+0000 7f8622a00640 1 --2- 172.21.15.77:0/1812906674 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55908b8d7020 0x55908b8d9410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:55.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.628+0000 7f8622a00640 1 -- 172.21.15.77:0/1812906674 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55908b8cbe40 msgr2=0x55908b8cc220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:55.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.628+0000 7f8622a00640 1 --2- 172.21.15.77:0/1812906674 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55908b8cbe40 0x55908b8cc220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:55.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.628+0000 7f8622a00640 1 -- 172.21.15.77:0/1812906674 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55908b7f8ae0 con 0x55908b8cc760 2024-09-10T09:56:55.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.628+0000 7f8622a00640 1 --2- 172.21.15.77:0/1812906674 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55908b8cc760 0x55908b8d6ae0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8614009cf0 tx=0x7f8614011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c3cca1504fcf0c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:55.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.628+0000 7f860fe00640 1 -- 172.21.15.77:0/1812906674 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8614004230 con 0x55908b8cc760 2024-09-10T09:56:55.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.628+0000 7f861b400640 1 --2- 172.21.15.77:0/2640734704 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f86080a2b80 0x7f86080a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:55.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.628+0000 7f860fe00640 1 -- 172.21.15.77:0/1812906674 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86140043d0 con 0x55908b8cc760 2024-09-10T09:56:55.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.628+0000 7f860fe00640 1 -- 172.21.15.77:0/1812906674 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86140198c0 con 0x55908b8cc760 2024-09-10T09:56:55.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.629+0000 7f861b400640 1 -- 172.21.15.77:0/2640734704 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f86100efa90 con 0x7f86241047e0 2024-09-10T09:56:55.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.629+0000 7f8623400640 1 --2- 172.21.15.77:0/2640734704 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f86080a2b80 0x7f86080a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:55.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.629+0000 7f862c7ca740 1 -- 172.21.15.77:0/1812906674 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55908b8cc760 msgr2=0x55908b8d6ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:55.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.629+0000 7f862c7ca740 1 --2- 172.21.15.77:0/1812906674 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55908b8cc760 0x55908b8d6ae0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8614009cf0 tx=0x7f8614011810 comp rx=0 tx=0).stop 2024-09-10T09:56:55.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.629+0000 7f8623400640 1 --2- 172.21.15.77:0/2640734704 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f86080a2b80 0x7f86080a5040 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f861c005fd0 tx=0x7f861c009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:55.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.629+0000 7f862c7ca740 1 -- 172.21.15.77:0/1812906674 shutdown_connections 2024-09-10T09:56:55.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.629+0000 7f862c7ca740 1 --2- 172.21.15.77:0/1812906674 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55908b8d7020 0x55908b8d9410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:55.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.629+0000 7f862c7ca740 1 --2- 172.21.15.77:0/1812906674 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55908b8cc760 0x55908b8d6ae0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:55.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.629+0000 7f862c7ca740 1 --2- 172.21.15.77:0/1812906674 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55908b8cbe40 0x55908b8cc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:55.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.629+0000 7f862c7ca740 1 -- 172.21.15.77:0/1812906674 >> 172.21.15.77:0/1812906674 conn(0x55908b8c83d0 msgr2=0x55908b8c87b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:55.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.630+0000 7f862c7ca740 1 -- 172.21.15.77:0/1812906674 shutdown_connections 2024-09-10T09:56:55.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.630+0000 7f862c7ca740 1 -- 172.21.15.77:0/1812906674 wait complete. 2024-09-10T09:56:55.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.630+0000 7f862c7ca740 1 -- start start 2024-09-10T09:56:55.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.630+0000 7f862c7ca740 1 Processor -- start 2024-09-10T09:56:55.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.631+0000 7f862c7ca740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55908b8cbe40 0x55908b8d3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:55.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.631+0000 7f862c7ca740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55908b8cc760 0x55908b8d4420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:55.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.631+0000 7f862c7ca740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55908b8d7020 0x55908b8d4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:55.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.631+0000 7f862c7ca740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55908b8d9bf0 con 0x55908b8cc760 2024-09-10T09:56:55.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.631+0000 7f862c7ca740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55908b7ed0f0 con 0x55908b8d7020 2024-09-10T09:56:55.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.631+0000 7f862c7ca740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55908b8d9d70 con 0x55908b8cbe40 2024-09-10T09:56:55.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.631+0000 7f862c7ca740 10 client.0 _ll_register_callbacks cb 0x55908b7d4bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:56:55.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.631+0000 7f8622000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55908b8cc760 0x55908b8d4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:55.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.631+0000 7f8622000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55908b8cc760 0x55908b8d4420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:41908/0 (socket says 172.21.15.77:41908) 2024-09-10T09:56:55.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.631+0000 7f8622a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55908b8cbe40 0x55908b8d3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:55.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.631+0000 7f8622000640 1 -- 172.21.15.77:0/703364453 learned_addr learned my addr 172.21.15.77:0/703364453 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:55.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.631+0000 7f8623400640 1 --2- 172.21.15.77:0/703364453 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55908b8d7020 0x55908b8d4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:55.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.632+0000 7f8622a00640 1 -- 172.21.15.77:0/703364453 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55908b8d7020 msgr2=0x55908b8d4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:55.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.632+0000 7f8622a00640 1 --2- 172.21.15.77:0/703364453 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55908b8d7020 0x55908b8d4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:55.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.632+0000 7f8622a00640 1 -- 172.21.15.77:0/703364453 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55908b8cc760 msgr2=0x55908b8d4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:55.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.632+0000 7f8622a00640 1 --2- 172.21.15.77:0/703364453 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55908b8cc760 0x55908b8d4420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:55.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.632+0000 7f8622a00640 1 -- 172.21.15.77:0/703364453 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55908b5a33c0 con 0x55908b8cbe40 2024-09-10T09:56:55.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.632+0000 7f8622000640 1 --2- 172.21.15.77:0/703364453 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55908b8cc760 0x55908b8d4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:56:55.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.632+0000 7f8622a00640 1 --2- 172.21.15.77:0/703364453 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55908b8cbe40 0x55908b8d3ee0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8614019460 tx=0x7f8614009f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67c7fd0ad64d373b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:55.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.632+0000 7f860cc00640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-10T09:56:55.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.632+0000 7f860cc00640 1 -- 172.21.15.77:0/703364453 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8614027070 con 0x55908b8cbe40 2024-09-10T09:56:55.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.632+0000 7f862c7ca740 1 -- 172.21.15.77:0/703364453 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86140043d0 con 0x55908b8cbe40 2024-09-10T09:56:55.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.632+0000 7f860cc00640 1 -- 172.21.15.77:0/703364453 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f8614005240 con 0x55908b8cbe40 2024-09-10T09:56:55.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.633+0000 7f860cc00640 1 -- 172.21.15.77:0/703364453 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8614033050 con 0x55908b8cbe40 2024-09-10T09:56:55.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.633+0000 7f862c7ca740 10 client.25843 fetch_fsmap learned FSMap version 29 2024-09-10T09:56:55.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.633+0000 7f862c7ca740 1 -- 172.21.15.77:0/703364453 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55908b8e5c00 con 0x55908b8cbe40 2024-09-10T09:56:55.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.633+0000 7f860cc00640 1 -- 172.21.15.77:0/703364453 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86140043d0 con 0x55908b8cbe40 2024-09-10T09:56:55.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.633+0000 7f860cc00640 1 -- 172.21.15.77:0/703364453 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f8614004b40 con 0x55908b8cbe40 2024-09-10T09:56:55.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.633+0000 7f862c7ca740 10 client.25843 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T09:56:55.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.633+0000 7f862c7ca740 10 client.25843 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:56:55.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.633+0000 7f862c7ca740 20 client.25843 populate_metadata read hostname 'smithi077' 2024-09-10T09:56:55.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.633+0000 7f862c7ca740 1 --2- 172.21.15.77:0/703364453 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55908b8e7010 0x55908b8e9400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:55.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.633+0000 7f862c7ca740 4 client.25843 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:56:55.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.633+0000 7f862c7ca740 1 -- 172.21.15.77:0/703364453 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55908b83f950 con 0x55908b8e7010 2024-09-10T09:56:55.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.634+0000 7f8622000640 1 --2- 172.21.15.77:0/703364453 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55908b8e7010 0x55908b8e9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:55.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.634+0000 7f8622000640 1 --2- 172.21.15.77:0/703364453 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55908b8e7010 0x55908b8e9400 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95824d03e515b40f server_cookie=967579ac9b3ad73d in_seq=0 out_seq=0 2024-09-10T09:56:55.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.634+0000 7f860cc00640 10 client.25843 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T09:56:55.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.635+0000 7f860cc00640 1 -- 172.21.15.77:0/703364453 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55908b83f950 con 0x55908b8e7010 2024-09-10T09:56:55.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.635+0000 7f860cc00640 10 client.25843 handle_command_reply: tid=0 2024-09-10T09:56:55.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.635+0000 7f862c7ca740 1 client.25843 shutdown 2024-09-10T09:56:55.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.635+0000 7f862c7ca740 1 -- 172.21.15.77:0/703364453 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55908b8cbe40 msgr2=0x55908b8d3ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:55.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.635+0000 7f862c7ca740 1 --2- 172.21.15.77:0/703364453 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55908b8cbe40 0x55908b8d3ee0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8614019460 tx=0x7f8614009f10 comp rx=0 tx=0).stop 2024-09-10T09:56:55.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.635+0000 7f862c7ca740 1 -- 172.21.15.77:0/703364453 shutdown_connections 2024-09-10T09:56:55.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.635+0000 7f862c7ca740 1 --2- 172.21.15.77:0/703364453 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55908b8e7010 0x55908b8e9400 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:55.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.635+0000 7f862c7ca740 1 --2- 172.21.15.77:0/703364453 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55908b8d7020 0x55908b8d4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:55.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.635+0000 7f862c7ca740 1 --2- 172.21.15.77:0/703364453 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55908b8cc760 0x55908b8d4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:55.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.635+0000 7f860cc00640 0 client.25843 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T09:56:55.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.635+0000 7f862c7ca740 1 --2- 172.21.15.77:0/703364453 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55908b8cbe40 0x55908b8d3ee0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:55.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.635+0000 7f862c7ca740 1 -- 172.21.15.77:0/703364453 >> 172.21.15.77:0/703364453 conn(0x55908b8c83d0 msgr2=0x55908b8ce100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:55.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.635+0000 7f862c7ca740 1 -- 172.21.15.77:0/703364453 shutdown_connections 2024-09-10T09:56:55.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.635+0000 7f862c7ca740 1 -- 172.21.15.77:0/703364453 wait complete. 2024-09-10T09:56:55.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.635+0000 7f862c7ca740 20 client.25843 trim_cache size 0 max 16384 2024-09-10T09:56:55.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.649+0000 7f862c7ca740 1 Processor -- start 2024-09-10T09:56:55.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.649+0000 7f862c7ca740 1 -- start start 2024-09-10T09:56:55.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.649+0000 7f862c7ca740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55908b8cbe40 0x55908b8c3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:55.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.649+0000 7f862c7ca740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55908b8cc760 0x55908b8eeb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:55.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.649+0000 7f862c7ca740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55908b8d7020 0x55908b8ef0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:55.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.649+0000 7f862c7ca740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f86241b0e70 con 0x55908b8d7020 2024-09-10T09:56:55.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.649+0000 7f862c7ca740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f86241b29b0 con 0x55908b8cbe40 2024-09-10T09:56:55.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.649+0000 7f862c7ca740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f86241b1540 con 0x55908b8cc760 2024-09-10T09:56:55.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.649+0000 7f8623400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55908b8d7020 0x55908b8ef0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:55.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.650+0000 7f8622a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55908b8cbe40 0x55908b8c3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:55.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.650+0000 7f8623400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55908b8d7020 0x55908b8ef0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:41918/0 (socket says 172.21.15.77:41918) 2024-09-10T09:56:55.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.650+0000 7f8622000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55908b8cc760 0x55908b8eeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:55.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.650+0000 7f8623400640 1 -- 172.21.15.77:0/1279828053 learned_addr learned my addr 172.21.15.77:0/1279828053 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:55.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.650+0000 7f8622a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55908b8cbe40 0x55908b8c3300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:43884/0 (socket says 172.21.15.77:43884) 2024-09-10T09:56:55.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.650+0000 7f8622a00640 1 -- 172.21.15.77:0/1279828053 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55908b8cc760 msgr2=0x55908b8eeb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:55.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.650+0000 7f8622a00640 1 --2- 172.21.15.77:0/1279828053 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55908b8cc760 0x55908b8eeb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:55.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.650+0000 7f8622a00640 1 -- 172.21.15.77:0/1279828053 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55908b8d7020 msgr2=0x55908b8ef0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:55.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.650+0000 7f8622a00640 1 --2- 172.21.15.77:0/1279828053 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55908b8d7020 0x55908b8ef0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:55.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.650+0000 7f8622a00640 1 -- 172.21.15.77:0/1279828053 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55908b8d9750 con 0x55908b8cbe40 2024-09-10T09:56:55.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.650+0000 7f8622a00640 1 --2- 172.21.15.77:0/1279828053 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55908b8cbe40 0x55908b8c3300 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8614019660 tx=0x7f86140199c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12c21244bb72ebd7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:55.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.650+0000 7f860f400640 1 -- 172.21.15.77:0/1279828053 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8614030070 con 0x55908b8cbe40 2024-09-10T09:56:55.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.650+0000 7f860f400640 1 -- 172.21.15.77:0/1279828053 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8614009910 con 0x55908b8cbe40 2024-09-10T09:56:55.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.650+0000 7f860f400640 1 -- 172.21.15.77:0/1279828053 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8614027030 con 0x55908b8cbe40 2024-09-10T09:56:55.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.651+0000 7f862c7ca740 1 -- 172.21.15.77:0/1279828053 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55908b8cbe40 msgr2=0x55908b8c3300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:55.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.651+0000 7f862c7ca740 1 --2- 172.21.15.77:0/1279828053 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55908b8cbe40 0x55908b8c3300 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8614019660 tx=0x7f86140199c0 comp rx=0 tx=0).stop 2024-09-10T09:56:55.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.651+0000 7f862c7ca740 1 -- 172.21.15.77:0/1279828053 shutdown_connections 2024-09-10T09:56:55.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.651+0000 7f862c7ca740 1 --2- 172.21.15.77:0/1279828053 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55908b8d7020 0x55908b8ef0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:55.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.651+0000 7f862c7ca740 1 --2- 172.21.15.77:0/1279828053 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55908b8cc760 0x55908b8eeb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:55.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.651+0000 7f862c7ca740 1 --2- 172.21.15.77:0/1279828053 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55908b8cbe40 0x55908b8c3300 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:55.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.651+0000 7f862c7ca740 1 -- 172.21.15.77:0/1279828053 >> 172.21.15.77:0/1279828053 conn(0x55908b8c83d0 msgr2=0x55908b8ce0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:55.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.651+0000 7f862c7ca740 1 -- 172.21.15.77:0/1279828053 shutdown_connections 2024-09-10T09:56:55.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.651+0000 7f862c7ca740 1 -- 172.21.15.77:0/1279828053 wait complete. 2024-09-10T09:56:55.656 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.652+0000 7f862c7ca740 1 -- start start 2024-09-10T09:56:55.656 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.652+0000 7f862c7ca740 1 Processor -- start 2024-09-10T09:56:55.656 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.652+0000 7f862c7ca740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55908b8cbe40 0x55908b8df560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:55.656 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.652+0000 7f862c7ca740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55908b8cc760 0x55908b8d0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:55.656 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.652+0000 7f862c7ca740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55908b8d7020 0x55908b8d1020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:55.656 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.652+0000 7f862c7ca740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f86241b22e0 con 0x55908b8cc760 2024-09-10T09:56:55.656 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.652+0000 7f862c7ca740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f86241b1c10 con 0x55908b8d7020 2024-09-10T09:56:55.656 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.652+0000 7f862c7ca740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f86241aee50 con 0x55908b8cbe40 2024-09-10T09:56:55.656 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.652+0000 7f862c7ca740 10 client.0 _ll_register_callbacks cb 0x7f861c009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:56:55.656 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.652+0000 7f8622000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55908b8cc760 0x55908b8d0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:55.656 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.652+0000 7f8622000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55908b8cc760 0x55908b8d0ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:41926/0 (socket says 172.21.15.77:41926) 2024-09-10T09:56:55.656 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.653+0000 7f8622000640 1 -- 172.21.15.77:0/2163004012 learned_addr learned my addr 172.21.15.77:0/2163004012 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:55.656 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.653+0000 7f8622a00640 1 --2- 172.21.15.77:0/2163004012 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55908b8cbe40 0x55908b8df560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:55.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.653+0000 7f8623400640 1 --2- 172.21.15.77:0/2163004012 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55908b8d7020 0x55908b8d1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:55.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.653+0000 7f8622000640 1 -- 172.21.15.77:0/2163004012 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55908b8cbe40 msgr2=0x55908b8df560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:55.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.653+0000 7f8622000640 1 --2- 172.21.15.77:0/2163004012 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55908b8cbe40 0x55908b8df560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:55.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.653+0000 7f8622000640 1 -- 172.21.15.77:0/2163004012 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55908b8d7020 msgr2=0x55908b8d1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:55.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.653+0000 7f8622000640 1 --2- 172.21.15.77:0/2163004012 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55908b8d7020 0x55908b8d1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:55.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.653+0000 7f8622000640 1 -- 172.21.15.77:0/2163004012 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55908b8d1690 con 0x55908b8cc760 2024-09-10T09:56:55.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.653+0000 7f8622000640 1 --2- 172.21.15.77:0/2163004012 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55908b8cc760 0x55908b8d0ae0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f86100bbba0 tx=0x7f86100041b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa95768dcae16991 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:55.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.653+0000 7f860cc00640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-10T09:56:55.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.653+0000 7f860cc00640 1 -- 172.21.15.77:0/2163004012 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8610035040 con 0x55908b8cc760 2024-09-10T09:56:55.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.653+0000 7f860cc00640 1 -- 172.21.15.77:0/2163004012 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f86100ed250 con 0x55908b8cc760 2024-09-10T09:56:55.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.653+0000 7f862c7ca740 1 -- 172.21.15.77:0/2163004012 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8614009910 con 0x55908b8cc760 2024-09-10T09:56:55.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.654+0000 7f860cc00640 1 -- 172.21.15.77:0/2163004012 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f86100c0250 con 0x55908b8cc760 2024-09-10T09:56:55.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.654+0000 7f862c7ca740 10 client.15819 fetch_fsmap learned FSMap version 29 2024-09-10T09:56:55.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.654+0000 7f862c7ca740 1 -- 172.21.15.77:0/2163004012 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55908b951a60 con 0x55908b8cc760 2024-09-10T09:56:55.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.654+0000 7f860cc00640 1 -- 172.21.15.77:0/2163004012 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8614009910 con 0x55908b8cc760 2024-09-10T09:56:55.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.654+0000 7f860cc00640 1 -- 172.21.15.77:0/2163004012 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f86100bc100 con 0x55908b8cc760 2024-09-10T09:56:55.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.654+0000 7f862c7ca740 10 client.15819 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T09:56:55.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.654+0000 7f862c7ca740 10 client.15819 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:56:55.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.654+0000 7f862c7ca740 20 client.15819 populate_metadata read hostname 'smithi077' 2024-09-10T09:56:55.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.654+0000 7f862c7ca740 1 --2- 172.21.15.77:0/2163004012 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55908b952190 0x55908b9545b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:55.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.654+0000 7f862c7ca740 4 client.15819 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:56:55.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.654+0000 7f862c7ca740 1 -- 172.21.15.77:0/2163004012 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55908b839e60 con 0x55908b952190 2024-09-10T09:56:55.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.655+0000 7f8622a00640 1 --2- 172.21.15.77:0/2163004012 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55908b952190 0x55908b9545b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:55.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.655+0000 7f8622a00640 1 --2- 172.21.15.77:0/2163004012 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55908b952190 0x55908b9545b0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36e9ea7292971839 server_cookie=65d66892a77ee28d in_seq=0 out_seq=0 2024-09-10T09:56:55.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.655+0000 7f860cc00640 10 client.15819 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T09:56:55.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.656+0000 7f860cc00640 1 -- 172.21.15.77:0/2163004012 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55908b839e60 con 0x55908b952190 2024-09-10T09:56:55.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.656+0000 7f860cc00640 10 client.15819 handle_command_reply: tid=0 2024-09-10T09:56:55.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.656+0000 7f862c7ca740 1 client.15819 shutdown 2024-09-10T09:56:55.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.656+0000 7f862c7ca740 1 -- 172.21.15.77:0/2163004012 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55908b8cc760 msgr2=0x55908b8d0ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:55.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.656+0000 7f862c7ca740 1 --2- 172.21.15.77:0/2163004012 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55908b8cc760 0x55908b8d0ae0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f86100bbba0 tx=0x7f86100041b0 comp rx=0 tx=0).stop 2024-09-10T09:56:55.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.656+0000 7f862c7ca740 1 -- 172.21.15.77:0/2163004012 shutdown_connections 2024-09-10T09:56:55.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.656+0000 7f862c7ca740 1 --2- 172.21.15.77:0/2163004012 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55908b952190 0x55908b9545b0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:55.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.656+0000 7f862c7ca740 1 --2- 172.21.15.77:0/2163004012 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55908b8d7020 0x55908b8d1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:55.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.656+0000 7f862c7ca740 1 --2- 172.21.15.77:0/2163004012 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55908b8cc760 0x55908b8d0ae0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:55.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.656+0000 7f862c7ca740 1 --2- 172.21.15.77:0/2163004012 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55908b8cbe40 0x55908b8df560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:55.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.656+0000 7f860cc00640 0 client.15819 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T09:56:55.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.656+0000 7f862c7ca740 1 -- 172.21.15.77:0/2163004012 >> 172.21.15.77:0/2163004012 conn(0x55908b8c83d0 msgr2=0x55908b8c6c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:55.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.656+0000 7f862c7ca740 1 -- 172.21.15.77:0/2163004012 shutdown_connections 2024-09-10T09:56:55.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.656+0000 7f862c7ca740 1 -- 172.21.15.77:0/2163004012 wait complete. 2024-09-10T09:56:55.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.656+0000 7f862c7ca740 20 client.15819 trim_cache size 0 max 16384 2024-09-10T09:56:55.661 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-10T09:56:55.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.657+0000 7f8623e00640 1 -- 172.21.15.77:0/2640734704 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f86080a2b80 msgr2=0x7f86080a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:55.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.657+0000 7f8623e00640 1 --2- 172.21.15.77:0/2640734704 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f86080a2b80 0x7f86080a5040 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f861c005fd0 tx=0x7f861c009500 comp rx=0 tx=0).stop 2024-09-10T09:56:55.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.657+0000 7f8623e00640 1 -- 172.21.15.77:0/2640734704 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86241047e0 msgr2=0x7f862419aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:55.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.657+0000 7f8623e00640 1 --2- 172.21.15.77:0/2640734704 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86241047e0 0x7f862419aa60 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f8610006010 tx=0x7f8610004040 comp rx=0 tx=0).stop 2024-09-10T09:56:55.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.657+0000 7f8623e00640 1 -- 172.21.15.77:0/2640734704 shutdown_connections 2024-09-10T09:56:55.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.657+0000 7f8623e00640 1 --2- 172.21.15.77:0/2640734704 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f86080a2b80 0x7f86080a5040 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:55.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.657+0000 7f8623e00640 1 --2- 172.21.15.77:0/2640734704 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8624108f60 0x7f862419edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:55.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.657+0000 7f8623e00640 1 --2- 172.21.15.77:0/2640734704 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86241047e0 0x7f862419aa60 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:55.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.657+0000 7f8623e00640 1 --2- 172.21.15.77:0/2640734704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8624103e30 0x7f862419a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:55.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.657+0000 7f8623e00640 1 -- 172.21.15.77:0/2640734704 >> 172.21.15.77:0/2640734704 conn(0x7f86240ff910 msgr2=0x7f8624100d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:55.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.657+0000 7f8623e00640 1 -- 172.21.15.77:0/2640734704 shutdown_connections 2024-09-10T09:56:55.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:55.657+0000 7f8623e00640 1 -- 172.21.15.77:0/2640734704 wait complete. 2024-09-10T09:56:57.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:56:56 smithi134 ceph-mon[35674]: pgmap v444: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T09:56:57.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:56:56 smithi120 ceph-mon[34770]: pgmap v444: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T09:56:57.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:56:56 smithi077 ceph-mon[30319]: pgmap v444: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T09:56:57.512 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T09:56:57.513 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T09:56:57.889 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:56:58.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.464+0000 7f0727800640 1 -- 172.21.15.77:0/1257335219 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0720102790 msgr2=0x7f07201067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:58.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.464+0000 7f0727800640 1 --2- 172.21.15.77:0/1257335219 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0720102790 0x7f07201067e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f0708009900 tx=0x7f070802a3f0 comp rx=0 tx=0).stop 2024-09-10T09:56:58.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.464+0000 7f0727800640 1 -- 172.21.15.77:0/1257335219 shutdown_connections 2024-09-10T09:56:58.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.464+0000 7f0727800640 1 --2- 172.21.15.77:0/1257335219 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0720106f10 0x7f072010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:58.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.464+0000 7f0727800640 1 --2- 172.21.15.77:0/1257335219 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0720102790 0x7f07201067e0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:58.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.464+0000 7f0727800640 1 --2- 172.21.15.77:0/1257335219 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0720101de0 0x7f07201021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:58.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.464+0000 7f0727800640 1 -- 172.21.15.77:0/1257335219 >> 172.21.15.77:0/1257335219 conn(0x7f07200fd8c0 msgr2=0x7f07200ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:58.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.464+0000 7f0727800640 1 -- 172.21.15.77:0/1257335219 shutdown_connections 2024-09-10T09:56:58.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.464+0000 7f0727800640 1 -- 172.21.15.77:0/1257335219 wait complete. 2024-09-10T09:56:58.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.465+0000 7f0727800640 1 Processor -- start 2024-09-10T09:56:58.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.465+0000 7f0727800640 1 -- start start 2024-09-10T09:56:58.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.465+0000 7f0727800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0720101de0 0x7f0720198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:58.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.465+0000 7f0727800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0720102790 0x7f07201989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:58.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.465+0000 7f0727800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0720106f10 0x7f072019cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:58.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.465+0000 7f0727800640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0720111fc0 con 0x7f0720101de0 2024-09-10T09:56:58.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.465+0000 7f0727800640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0720111e40 con 0x7f0720106f10 2024-09-10T09:56:58.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.465+0000 7f0727800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0720112140 con 0x7f0720102790 2024-09-10T09:56:58.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.466+0000 7f0724e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0720101de0 0x7f0720198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:58.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.466+0000 7f0724e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0720101de0 0x7f0720198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:41936/0 (socket says 172.21.15.77:41936) 2024-09-10T09:56:58.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.466+0000 7f0724e00640 1 -- 172.21.15.77:0/1510979039 learned_addr learned my addr 172.21.15.77:0/1510979039 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:58.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.466+0000 7f0717e00640 1 --2- 172.21.15.77:0/1510979039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0720102790 0x7f07201989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:58.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.466+0000 7f0725800640 1 --2- 172.21.15.77:0/1510979039 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0720106f10 0x7f072019cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:58.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.466+0000 7f0724e00640 1 -- 172.21.15.77:0/1510979039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0720102790 msgr2=0x7f07201989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:58.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.466+0000 7f0724e00640 1 --2- 172.21.15.77:0/1510979039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0720102790 0x7f07201989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:58.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.466+0000 7f0724e00640 1 -- 172.21.15.77:0/1510979039 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0720106f10 msgr2=0x7f072019cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:58.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.466+0000 7f0724e00640 1 --2- 172.21.15.77:0/1510979039 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0720106f10 0x7f072019cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:58.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.466+0000 7f0724e00640 1 -- 172.21.15.77:0/1510979039 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f072019d410 con 0x7f0720101de0 2024-09-10T09:56:58.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.467+0000 7f0724e00640 1 --2- 172.21.15.77:0/1510979039 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0720101de0 0x7f0720198460 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f071000e9f0 tx=0x7f071000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:58.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.467+0000 7f071e000640 1 -- 172.21.15.77:0/1510979039 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f071000cd80 con 0x7f0720101de0 2024-09-10T09:56:58.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.467+0000 7f071e000640 1 -- 172.21.15.77:0/1510979039 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0710005500 con 0x7f0720101de0 2024-09-10T09:56:58.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.467+0000 7f0727800640 1 -- 172.21.15.77:0/1510979039 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f072019d700 con 0x7f0720101de0 2024-09-10T09:56:58.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.467+0000 7f071e000640 1 -- 172.21.15.77:0/1510979039 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0710010690 con 0x7f0720101de0 2024-09-10T09:56:58.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.467+0000 7f0727800640 1 -- 172.21.15.77:0/1510979039 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07201a4fe0 con 0x7f0720101de0 2024-09-10T09:56:58.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.470+0000 7f071e000640 1 -- 172.21.15.77:0/1510979039 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f07100046c0 con 0x7f0720101de0 2024-09-10T09:56:58.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.470+0000 7f072a3ca740 1 Processor -- start 2024-09-10T09:56:58.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.470+0000 7f072a3ca740 1 -- start start 2024-09-10T09:56:58.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.471+0000 7f072a3ca740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da06e85ed0 0x55da06e862b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:58.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.471+0000 7f072a3ca740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da06e867f0 0x55da06e90b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:58.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.471+0000 7f072a3ca740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da06e910b0 0x55da06e934a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:58.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.471+0000 7f072a3ca740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55da06de8290 con 0x55da06e85ed0 2024-09-10T09:56:58.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.471+0000 7f072a3ca740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55da06d8fb40 con 0x55da06e910b0 2024-09-10T09:56:58.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.471+0000 7f072a3ca740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55da06de4250 con 0x55da06e867f0 2024-09-10T09:56:58.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.471+0000 7f0725800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da06e85ed0 0x55da06e862b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:58.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.471+0000 7f0725800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da06e85ed0 0x55da06e862b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:41942/0 (socket says 172.21.15.77:41942) 2024-09-10T09:56:58.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.471+0000 7f0725800640 1 -- 172.21.15.77:0/2005889768 learned_addr learned my addr 172.21.15.77:0/2005889768 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:58.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.471+0000 7f0724e00640 1 --2- 172.21.15.77:0/2005889768 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da06e867f0 0x55da06e90b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:58.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.471+0000 7f0717e00640 1 --2- 172.21.15.77:0/2005889768 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da06e910b0 0x55da06e934a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:58.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.471+0000 7f0725800640 1 -- 172.21.15.77:0/2005889768 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da06e867f0 msgr2=0x55da06e90b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:58.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.471+0000 7f0725800640 1 --2- 172.21.15.77:0/2005889768 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da06e867f0 0x55da06e90b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:58.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.471+0000 7f0725800640 1 -- 172.21.15.77:0/2005889768 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da06e910b0 msgr2=0x55da06e934a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:58.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.471+0000 7f0725800640 1 --2- 172.21.15.77:0/2005889768 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da06e910b0 0x55da06e934a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:58.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.471+0000 7f0725800640 1 -- 172.21.15.77:0/2005889768 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da06b596a0 con 0x55da06e85ed0 2024-09-10T09:56:58.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.472+0000 7f071e000640 1 --2- 172.21.15.77:0/1510979039 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f07040a2b80 0x7f07040a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:58.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.472+0000 7f0725800640 1 --2- 172.21.15.77:0/2005889768 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da06e85ed0 0x55da06e862b0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f0718009f50 tx=0x7f07180118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11aba0cc6ee57464 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:58.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.472+0000 7f071e000640 1 -- 172.21.15.77:0/1510979039 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f07100cf8b0 con 0x7f0720101de0 2024-09-10T09:56:58.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.472+0000 7f0724e00640 1 --2- 172.21.15.77:0/1510979039 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f07040a2b80 0x7f07040a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:58.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.472+0000 7f0716000640 1 -- 172.21.15.77:0/2005889768 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0718004d20 con 0x55da06e85ed0 2024-09-10T09:56:58.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.472+0000 7f0716000640 1 -- 172.21.15.77:0/2005889768 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0718004050 con 0x55da06e85ed0 2024-09-10T09:56:58.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.472+0000 7f0716000640 1 -- 172.21.15.77:0/2005889768 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0718005280 con 0x55da06e85ed0 2024-09-10T09:56:58.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.473+0000 7f072a3ca740 1 -- 172.21.15.77:0/2005889768 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da06e85ed0 msgr2=0x55da06e862b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:58.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.473+0000 7f0724e00640 1 --2- 172.21.15.77:0/1510979039 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f07040a2b80 0x7f07040a5040 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f0710098690 tx=0x7f07100d5530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:58.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.473+0000 7f072a3ca740 1 --2- 172.21.15.77:0/2005889768 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da06e85ed0 0x55da06e862b0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f0718009f50 tx=0x7f07180118d0 comp rx=0 tx=0).stop 2024-09-10T09:56:58.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.473+0000 7f072a3ca740 1 -- 172.21.15.77:0/2005889768 shutdown_connections 2024-09-10T09:56:58.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.473+0000 7f072a3ca740 1 --2- 172.21.15.77:0/2005889768 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da06e910b0 0x55da06e934a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:58.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.473+0000 7f072a3ca740 1 --2- 172.21.15.77:0/2005889768 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da06e867f0 0x55da06e90b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:58.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.473+0000 7f072a3ca740 1 --2- 172.21.15.77:0/2005889768 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da06e85ed0 0x55da06e862b0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:58.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.473+0000 7f072a3ca740 1 -- 172.21.15.77:0/2005889768 >> 172.21.15.77:0/2005889768 conn(0x55da06e82460 msgr2=0x55da06e82840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:58.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.473+0000 7f072a3ca740 1 -- 172.21.15.77:0/2005889768 shutdown_connections 2024-09-10T09:56:58.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.473+0000 7f072a3ca740 1 -- 172.21.15.77:0/2005889768 wait complete. 2024-09-10T09:56:58.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.474+0000 7f072a3ca740 1 -- start start 2024-09-10T09:56:58.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.474+0000 7f072a3ca740 1 Processor -- start 2024-09-10T09:56:58.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.474+0000 7f072a3ca740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da06e85ed0 0x55da06e90a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:58.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.474+0000 7f072a3ca740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da06e867f0 0x55da06e8dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:58.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.474+0000 7f072a3ca740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da06e910b0 0x55da06e8e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:58.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.474+0000 7f072a3ca740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55da06e93d20 con 0x55da06e910b0 2024-09-10T09:56:58.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.474+0000 7f072a3ca740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55da06da2fa0 con 0x55da06e867f0 2024-09-10T09:56:58.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.474+0000 7f072a3ca740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55da06e93ea0 con 0x55da06e85ed0 2024-09-10T09:56:58.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.475+0000 7f072a3ca740 10 client.0 _ll_register_callbacks cb 0x55da06af6ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:56:58.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.475+0000 7f0725800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da06e910b0 0x55da06e8e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:58.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.475+0000 7f0717e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da06e867f0 0x55da06e8dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:58.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.475+0000 7f0717e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da06e867f0 0x55da06e8dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:43926/0 (socket says 172.21.15.77:43926) 2024-09-10T09:56:58.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.475+0000 7f0717e00640 1 -- 172.21.15.77:0/366241504 learned_addr learned my addr 172.21.15.77:0/366241504 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:58.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.475+0000 7f0725800640 1 --2- 172.21.15.77:0/366241504 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da06e85ed0 0x55da06e90a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:58.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.475+0000 7f0717e00640 1 -- 172.21.15.77:0/366241504 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da06e85ed0 msgr2=0x55da06e90a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:58.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.475+0000 7f0717e00640 1 --2- 172.21.15.77:0/366241504 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da06e85ed0 0x55da06e90a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:58.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.475+0000 7f0717e00640 1 -- 172.21.15.77:0/366241504 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da06e910b0 msgr2=0x55da06e8e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:58.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.475+0000 7f0717e00640 1 --2- 172.21.15.77:0/366241504 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da06e910b0 0x55da06e8e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:58.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.475+0000 7f0717e00640 1 -- 172.21.15.77:0/366241504 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da06e8eaa0 con 0x55da06e867f0 2024-09-10T09:56:58.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.476+0000 7f0717e00640 1 --2- 172.21.15.77:0/366241504 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da06e867f0 0x55da06e8dcf0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0708009900 tx=0x7f0708004390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c853ca40c8d2af7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:58.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.476+0000 7f070ea00640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T09:56:58.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.476+0000 7f070ea00640 1 -- 172.21.15.77:0/366241504 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0708035040 con 0x55da06e867f0 2024-09-10T09:56:58.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.476+0000 7f072a3ca740 1 -- 172.21.15.77:0/366241504 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0718004050 con 0x55da06e867f0 2024-09-10T09:56:58.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.476+0000 7f070ea00640 1 -- 172.21.15.77:0/366241504 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f0708005450 con 0x55da06e867f0 2024-09-10T09:56:58.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.476+0000 7f070ea00640 1 -- 172.21.15.77:0/366241504 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f070804f050 con 0x55da06e867f0 2024-09-10T09:56:58.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.476+0000 7f072a3ca740 10 client.25901 fetch_fsmap learned FSMap version 29 2024-09-10T09:56:58.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.476+0000 7f072a3ca740 1 -- 172.21.15.77:0/366241504 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55da06e9fcc0 con 0x55da06e867f0 2024-09-10T09:56:58.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.477+0000 7f070ea00640 1 -- 172.21.15.77:0/366241504 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0718004050 con 0x55da06e867f0 2024-09-10T09:56:58.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.477+0000 7f070ea00640 1 -- 172.21.15.77:0/366241504 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f07080039c0 con 0x55da06e867f0 2024-09-10T09:56:58.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.477+0000 7f072a3ca740 10 client.25901 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T09:56:58.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.477+0000 7f072a3ca740 10 client.25901 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:56:58.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.477+0000 7f072a3ca740 20 client.25901 populate_metadata read hostname 'smithi077' 2024-09-10T09:56:58.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.477+0000 7f072a3ca740 1 --2- 172.21.15.77:0/366241504 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55da06ea10d0 0x55da06ea34f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:58.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.477+0000 7f072a3ca740 4 client.25901 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:56:58.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.477+0000 7f072a3ca740 1 -- 172.21.15.77:0/366241504 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55da06df5800 con 0x55da06ea10d0 2024-09-10T09:56:58.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.477+0000 7f0725800640 1 --2- 172.21.15.77:0/366241504 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55da06ea10d0 0x55da06ea34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:58.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.478+0000 7f0725800640 1 --2- 172.21.15.77:0/366241504 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55da06ea10d0 0x55da06ea34f0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9b0af8cf6eba79a server_cookie=8a6a14494b0d919f in_seq=0 out_seq=0 2024-09-10T09:56:58.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.478+0000 7f070ea00640 10 client.25901 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T09:56:58.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.478+0000 7f070ea00640 1 -- 172.21.15.77:0/366241504 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55da06df5800 con 0x55da06ea10d0 2024-09-10T09:56:58.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.478+0000 7f070ea00640 10 client.25901 handle_command_reply: tid=0 2024-09-10T09:56:58.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.479+0000 7f072a3ca740 1 client.25901 shutdown 2024-09-10T09:56:58.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.479+0000 7f072a3ca740 1 -- 172.21.15.77:0/366241504 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da06e867f0 msgr2=0x55da06e8dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:58.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.479+0000 7f072a3ca740 1 --2- 172.21.15.77:0/366241504 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da06e867f0 0x55da06e8dcf0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0708009900 tx=0x7f0708004390 comp rx=0 tx=0).stop 2024-09-10T09:56:58.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.479+0000 7f072a3ca740 1 -- 172.21.15.77:0/366241504 shutdown_connections 2024-09-10T09:56:58.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.479+0000 7f072a3ca740 1 --2- 172.21.15.77:0/366241504 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55da06ea10d0 0x55da06ea34f0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:58.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.479+0000 7f072a3ca740 1 --2- 172.21.15.77:0/366241504 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da06e910b0 0x55da06e8e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:58.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.479+0000 7f070ea00640 0 client.25901 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T09:56:58.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.479+0000 7f072a3ca740 1 --2- 172.21.15.77:0/366241504 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da06e867f0 0x55da06e8dcf0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:58.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.479+0000 7f072a3ca740 1 --2- 172.21.15.77:0/366241504 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da06e85ed0 0x55da06e90a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:58.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.479+0000 7f072a3ca740 1 -- 172.21.15.77:0/366241504 >> 172.21.15.77:0/366241504 conn(0x55da06e82460 msgr2=0x55da06e7d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:58.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.479+0000 7f072a3ca740 1 -- 172.21.15.77:0/366241504 shutdown_connections 2024-09-10T09:56:58.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.479+0000 7f072a3ca740 1 -- 172.21.15.77:0/366241504 wait complete. 2024-09-10T09:56:58.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.479+0000 7f072a3ca740 20 client.25901 trim_cache size 0 max 16384 2024-09-10T09:56:58.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.497+0000 7f072a3ca740 1 Processor -- start 2024-09-10T09:56:58.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.497+0000 7f072a3ca740 1 -- start start 2024-09-10T09:56:58.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.497+0000 7f072a3ca740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da06e85ed0 0x55da06e8ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:58.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.497+0000 7f072a3ca740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da06e867f0 0x55da06ea8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:58.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.497+0000 7f072a3ca740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da06e910b0 0x55da06ea9300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:58.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.497+0000 7f072a3ca740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f07201accb0 con 0x55da06e85ed0 2024-09-10T09:56:58.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.497+0000 7f072a3ca740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f07201ae7f0 con 0x55da06e910b0 2024-09-10T09:56:58.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.497+0000 7f072a3ca740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f07201ad380 con 0x55da06e867f0 2024-09-10T09:56:58.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.498+0000 7f0725800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da06e85ed0 0x55da06e8ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:58.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.498+0000 7f0725800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da06e85ed0 0x55da06e8ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:41956/0 (socket says 172.21.15.77:41956) 2024-09-10T09:56:58.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.498+0000 7f0725800640 1 -- 172.21.15.77:0/2976232335 learned_addr learned my addr 172.21.15.77:0/2976232335 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:58.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.498+0000 7f0717e00640 1 --2- 172.21.15.77:0/2976232335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da06e867f0 0x55da06ea8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:58.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.498+0000 7f0725800640 1 --2- 172.21.15.77:0/2976232335 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da06e910b0 0x55da06ea9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:58.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.498+0000 7f0725800640 1 -- 172.21.15.77:0/2976232335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da06e867f0 msgr2=0x55da06ea8dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:58.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.498+0000 7f0725800640 1 --2- 172.21.15.77:0/2976232335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da06e867f0 0x55da06ea8dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:58.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.498+0000 7f0725800640 1 -- 172.21.15.77:0/2976232335 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da06e910b0 msgr2=0x55da06ea9300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:58.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.498+0000 7f0725800640 1 --2- 172.21.15.77:0/2976232335 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da06e910b0 0x55da06ea9300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:58.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.498+0000 7f0725800640 1 -- 172.21.15.77:0/2976232335 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da06e86620 con 0x55da06e85ed0 2024-09-10T09:56:58.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.498+0000 7f0717e00640 1 --2- 172.21.15.77:0/2976232335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da06e867f0 0x55da06ea8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:56:58.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.498+0000 7f0725800640 1 --2- 172.21.15.77:0/2976232335 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da06e85ed0 0x55da06e8ccd0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f0718012fd0 tx=0x7f0718009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=772bb82751e13467 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:58.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.499+0000 7f0715600640 1 -- 172.21.15.77:0/2976232335 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0718004c80 con 0x55da06e85ed0 2024-09-10T09:56:58.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.499+0000 7f0715600640 1 -- 172.21.15.77:0/2976232335 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0718004e20 con 0x55da06e85ed0 2024-09-10T09:56:58.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.499+0000 7f0715600640 1 -- 172.21.15.77:0/2976232335 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07180054d0 con 0x55da06e85ed0 2024-09-10T09:56:58.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.499+0000 7f072a3ca740 1 -- 172.21.15.77:0/2976232335 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da06e85ed0 msgr2=0x55da06e8ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:58.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.499+0000 7f072a3ca740 1 --2- 172.21.15.77:0/2976232335 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da06e85ed0 0x55da06e8ccd0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f0718012fd0 tx=0x7f0718009500 comp rx=0 tx=0).stop 2024-09-10T09:56:58.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.499+0000 7f072a3ca740 1 -- 172.21.15.77:0/2976232335 shutdown_connections 2024-09-10T09:56:58.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.499+0000 7f072a3ca740 1 --2- 172.21.15.77:0/2976232335 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da06e910b0 0x55da06ea9300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:58.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.499+0000 7f072a3ca740 1 --2- 172.21.15.77:0/2976232335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da06e867f0 0x55da06ea8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:58.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.499+0000 7f072a3ca740 1 --2- 172.21.15.77:0/2976232335 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da06e85ed0 0x55da06e8ccd0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:58.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.499+0000 7f072a3ca740 1 -- 172.21.15.77:0/2976232335 >> 172.21.15.77:0/2976232335 conn(0x55da06e82460 msgr2=0x55da06e8c5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:58.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.499+0000 7f072a3ca740 1 -- 172.21.15.77:0/2976232335 shutdown_connections 2024-09-10T09:56:58.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.499+0000 7f072a3ca740 1 -- 172.21.15.77:0/2976232335 wait complete. 2024-09-10T09:56:58.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.500+0000 7f072a3ca740 1 -- start start 2024-09-10T09:56:58.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.500+0000 7f072a3ca740 1 Processor -- start 2024-09-10T09:56:58.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.500+0000 7f072a3ca740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da06e85ed0 0x55da06e99a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:58.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.500+0000 7f072a3ca740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da06e867f0 0x55da06e94d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:58.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.500+0000 7f072a3ca740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da06e910b0 0x55da06e954c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:58.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.501+0000 7f072a3ca740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f07201ae120 con 0x55da06e85ed0 2024-09-10T09:56:58.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.501+0000 7f072a3ca740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f07201ada50 con 0x55da06e867f0 2024-09-10T09:56:58.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.501+0000 7f072a3ca740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f07201aac90 con 0x55da06e910b0 2024-09-10T09:56:58.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.501+0000 7f072a3ca740 10 client.0 _ll_register_callbacks cb 0x7f0708009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:56:58.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.501+0000 7f0725800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da06e85ed0 0x55da06e99a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:58.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.501+0000 7f0717e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da06e867f0 0x55da06e94d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:58.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.501+0000 7f0717e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da06e867f0 0x55da06e94d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:43946/0 (socket says 172.21.15.77:43946) 2024-09-10T09:56:58.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.501+0000 7f0725800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da06e85ed0 0x55da06e99a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:41964/0 (socket says 172.21.15.77:41964) 2024-09-10T09:56:58.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.501+0000 7f0717e00640 1 -- 172.21.15.77:0/3943803385 learned_addr learned my addr 172.21.15.77:0/3943803385 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:56:58.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.501+0000 7f0725800640 1 --2- 172.21.15.77:0/3943803385 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da06e910b0 0x55da06e954c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:58.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.501+0000 7f0717e00640 1 -- 172.21.15.77:0/3943803385 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da06e910b0 msgr2=0x55da06e954c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:58.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.501+0000 7f0717e00640 1 --2- 172.21.15.77:0/3943803385 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da06e910b0 0x55da06e954c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:58.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.501+0000 7f0717e00640 1 -- 172.21.15.77:0/3943803385 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da06e85ed0 msgr2=0x55da06e99a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:58.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.501+0000 7f0717e00640 1 --2- 172.21.15.77:0/3943803385 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da06e85ed0 0x55da06e99a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:58.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.501+0000 7f0717e00640 1 -- 172.21.15.77:0/3943803385 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da06f0ad90 con 0x55da06e867f0 2024-09-10T09:56:58.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.501+0000 7f0725800640 1 --2- 172.21.15.77:0/3943803385 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da06e85ed0 0x55da06e99a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:56:58.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.502+0000 7f0717e00640 1 --2- 172.21.15.77:0/3943803385 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da06e867f0 0x55da06e94d30 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0708031550 tx=0x7f0708005900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67c4a2d39e2afe52 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:58.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.502+0000 7f070ea00640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T09:56:58.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.502+0000 7f070ea00640 1 -- 172.21.15.77:0/3943803385 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0708059060 con 0x55da06e867f0 2024-09-10T09:56:58.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.502+0000 7f072a3ca740 1 -- 172.21.15.77:0/3943803385 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0718004e20 con 0x55da06e867f0 2024-09-10T09:56:58.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.502+0000 7f070ea00640 1 -- 172.21.15.77:0/3943803385 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f07080315b0 con 0x55da06e867f0 2024-09-10T09:56:58.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.502+0000 7f070ea00640 1 -- 172.21.15.77:0/3943803385 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0708035070 con 0x55da06e867f0 2024-09-10T09:56:58.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.502+0000 7f072a3ca740 10 client.25907 fetch_fsmap learned FSMap version 29 2024-09-10T09:56:58.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.502+0000 7f072a3ca740 1 -- 172.21.15.77:0/3943803385 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55da06f0bfc0 con 0x55da06e867f0 2024-09-10T09:56:58.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.503+0000 7f070ea00640 1 -- 172.21.15.77:0/3943803385 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0718004e20 con 0x55da06e867f0 2024-09-10T09:56:58.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.503+0000 7f070ea00640 1 -- 172.21.15.77:0/3943803385 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f0708031a90 con 0x55da06e867f0 2024-09-10T09:56:58.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.503+0000 7f072a3ca740 10 client.25907 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T09:56:58.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.503+0000 7f072a3ca740 10 client.25907 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:56:58.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.503+0000 7f072a3ca740 20 client.25907 populate_metadata read hostname 'smithi077' 2024-09-10T09:56:58.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.503+0000 7f072a3ca740 1 --2- 172.21.15.77:0/3943803385 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55da06f0c740 0x55da06f0eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:58.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.503+0000 7f072a3ca740 4 client.25907 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T09:56:58.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.503+0000 7f072a3ca740 1 -- 172.21.15.77:0/3943803385 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55da06defd10 con 0x55da06f0c740 2024-09-10T09:56:58.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.504+0000 7f0725800640 1 --2- 172.21.15.77:0/3943803385 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55da06f0c740 0x55da06f0eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:58.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.504+0000 7f0725800640 1 --2- 172.21.15.77:0/3943803385 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55da06f0c740 0x55da06f0eb60 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f152d0e6a2da1fcd server_cookie=f4d4489d6689634c in_seq=0 out_seq=0 2024-09-10T09:56:58.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.504+0000 7f070ea00640 10 client.25907 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T09:56:58.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.505+0000 7f070ea00640 1 -- 172.21.15.77:0/3943803385 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55da06defd10 con 0x55da06f0c740 2024-09-10T09:56:58.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.505+0000 7f070ea00640 10 client.25907 handle_command_reply: tid=0 2024-09-10T09:56:58.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.505+0000 7f072a3ca740 1 client.25907 shutdown 2024-09-10T09:56:58.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.505+0000 7f072a3ca740 1 -- 172.21.15.77:0/3943803385 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da06e867f0 msgr2=0x55da06e94d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:58.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.505+0000 7f072a3ca740 1 --2- 172.21.15.77:0/3943803385 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da06e867f0 0x55da06e94d30 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0708031550 tx=0x7f0708005900 comp rx=0 tx=0).stop 2024-09-10T09:56:58.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.505+0000 7f072a3ca740 1 -- 172.21.15.77:0/3943803385 shutdown_connections 2024-09-10T09:56:58.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.505+0000 7f072a3ca740 1 --2- 172.21.15.77:0/3943803385 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55da06f0c740 0x55da06f0eb60 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:58.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.505+0000 7f072a3ca740 1 --2- 172.21.15.77:0/3943803385 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da06e910b0 0x55da06e954c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:58.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.505+0000 7f072a3ca740 1 --2- 172.21.15.77:0/3943803385 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da06e867f0 0x55da06e94d30 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:58.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.505+0000 7f072a3ca740 1 --2- 172.21.15.77:0/3943803385 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da06e85ed0 0x55da06e99a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:58.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.505+0000 7f072a3ca740 1 -- 172.21.15.77:0/3943803385 >> 172.21.15.77:0/3943803385 conn(0x55da06e82460 msgr2=0x55da06e8cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:58.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.505+0000 7f070ea00640 0 client.25907 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T09:56:58.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.506+0000 7f072a3ca740 1 -- 172.21.15.77:0/3943803385 shutdown_connections 2024-09-10T09:56:58.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.506+0000 7f072a3ca740 1 -- 172.21.15.77:0/3943803385 wait complete. 2024-09-10T09:56:58.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.506+0000 7f072a3ca740 20 client.25907 trim_cache size 0 max 16384 2024-09-10T09:56:58.510 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-10T09:56:58.510 INFO:teuthology.orchestra.run.smithi077.stdout: "return_code": 0, 2024-09-10T09:56:58.510 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_tag": "39564bc9-b545-45f5-95eb-bbdde588ce78", 2024-09-10T09:56:58.510 INFO:teuthology.orchestra.run.smithi077.stdout: "mode": "asynchronous" 2024-09-10T09:56:58.510 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-10T09:56:58.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.506+0000 7f0727800640 1 -- 172.21.15.77:0/1510979039 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f07040a2b80 msgr2=0x7f07040a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:58.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.506+0000 7f0727800640 1 --2- 172.21.15.77:0/1510979039 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f07040a2b80 0x7f07040a5040 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f0710098690 tx=0x7f07100d5530 comp rx=0 tx=0).stop 2024-09-10T09:56:58.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.506+0000 7f0727800640 1 -- 172.21.15.77:0/1510979039 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0720101de0 msgr2=0x7f0720198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:58.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.506+0000 7f0727800640 1 --2- 172.21.15.77:0/1510979039 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0720101de0 0x7f0720198460 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f071000e9f0 tx=0x7f071000edf0 comp rx=0 tx=0).stop 2024-09-10T09:56:58.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.506+0000 7f0727800640 1 -- 172.21.15.77:0/1510979039 shutdown_connections 2024-09-10T09:56:58.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.506+0000 7f0727800640 1 --2- 172.21.15.77:0/1510979039 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f07040a2b80 0x7f07040a5040 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:58.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.506+0000 7f0727800640 1 --2- 172.21.15.77:0/1510979039 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0720106f10 0x7f072019cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:58.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.506+0000 7f0727800640 1 --2- 172.21.15.77:0/1510979039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0720102790 0x7f07201989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:58.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.506+0000 7f0727800640 1 --2- 172.21.15.77:0/1510979039 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0720101de0 0x7f0720198460 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:58.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.506+0000 7f0727800640 1 -- 172.21.15.77:0/1510979039 >> 172.21.15.77:0/1510979039 conn(0x7f07200fd8c0 msgr2=0x7f07200fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:58.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.507+0000 7f0727800640 1 -- 172.21.15.77:0/1510979039 shutdown_connections 2024-09-10T09:56:58.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:56:58.507+0000 7f0727800640 1 -- 172.21.15.77:0/1510979039 wait complete. 2024-09-10T09:56:59.060 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:56:58 smithi077 ceph-mon[30319]: pgmap v445: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-10T09:56:59.061 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:56:58 smithi077 ceph-mon[30319]: scrub queued for path: / 2024-09-10T09:56:59.061 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:56:58 smithi077 ceph-mon[30319]: scrub summary: idle+waiting paths [/] 2024-09-10T09:56:59.061 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:56:58 smithi077 ceph-mon[30319]: scrub summary: active paths [/] 2024-09-10T09:56:59.061 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:56:58 smithi077 ceph-mon[30319]: scrub summary: idle+waiting paths [/] 2024-09-10T09:56:59.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:56:58 smithi134 ceph-mon[35674]: pgmap v445: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-10T09:56:59.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:56:58 smithi134 ceph-mon[35674]: scrub queued for path: / 2024-09-10T09:56:59.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:56:58 smithi134 ceph-mon[35674]: scrub summary: idle+waiting paths [/] 2024-09-10T09:56:59.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:56:58 smithi134 ceph-mon[35674]: scrub summary: active paths [/] 2024-09-10T09:56:59.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:56:58 smithi134 ceph-mon[35674]: scrub summary: idle+waiting paths [/] 2024-09-10T09:56:59.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:56:58 smithi120 ceph-mon[34770]: pgmap v445: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-10T09:56:59.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:56:58 smithi120 ceph-mon[34770]: scrub queued for path: / 2024-09-10T09:56:59.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:56:58 smithi120 ceph-mon[34770]: scrub summary: idle+waiting paths [/] 2024-09-10T09:56:59.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:56:58 smithi120 ceph-mon[34770]: scrub summary: active paths [/] 2024-09-10T09:56:59.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:56:58 smithi120 ceph-mon[34770]: scrub summary: idle+waiting paths [/] 2024-09-10T09:56:59.243 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T09:56:59.611 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:57:00.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.196+0000 7f52bee00640 1 -- 172.21.15.77:0/3617571576 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f52b810bfc0 msgr2=0x7f52b810e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:00.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.196+0000 7f52bee00640 1 --2- 172.21.15.77:0/3617571576 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f52b810bfc0 0x7f52b810e460 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f52b0009ca0 tx=0x7f52b002a950 comp rx=0 tx=0).stop 2024-09-10T09:57:00.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.196+0000 7f52bee00640 1 -- 172.21.15.77:0/3617571576 shutdown_connections 2024-09-10T09:57:00.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.196+0000 7f52bee00640 1 --2- 172.21.15.77:0/3617571576 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f52b810bfc0 0x7f52b810e460 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:00.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.196+0000 7f52bee00640 1 --2- 172.21.15.77:0/3617571576 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f52b81034b0 0x7f52b810b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:00.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.196+0000 7f52bee00640 1 --2- 172.21.15.77:0/3617571576 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52b8102b90 0x7f52b8102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:00.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.196+0000 7f52bee00640 1 -- 172.21.15.77:0/3617571576 >> 172.21.15.77:0/3617571576 conn(0x7f52b80fc610 msgr2=0x7f52b80fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:00.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.196+0000 7f52bee00640 1 -- 172.21.15.77:0/3617571576 shutdown_connections 2024-09-10T09:57:00.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.196+0000 7f52bee00640 1 -- 172.21.15.77:0/3617571576 wait complete. 2024-09-10T09:57:00.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.197+0000 7f52bee00640 1 Processor -- start 2024-09-10T09:57:00.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.197+0000 7f52bee00640 1 -- start start 2024-09-10T09:57:00.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.198+0000 7f52bee00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f52b8102b90 0x7f52b819a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:00.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.198+0000 7f52bee00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52b81034b0 0x7f52b819a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:00.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.198+0000 7f52bee00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f52b810bfc0 0x7f52b819ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:00.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.198+0000 7f52bee00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f52b8111e10 con 0x7f52b8102b90 2024-09-10T09:57:00.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.198+0000 7f52bee00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f52b8111c90 con 0x7f52b810bfc0 2024-09-10T09:57:00.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.198+0000 7f52bee00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f52b8111f90 con 0x7f52b81034b0 2024-09-10T09:57:00.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.198+0000 7f52bda00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f52b8102b90 0x7f52b819a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:00.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.198+0000 7f52bda00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f52b8102b90 0x7f52b819a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:41994/0 (socket says 172.21.15.77:41994) 2024-09-10T09:57:00.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.198+0000 7f52bda00640 1 -- 172.21.15.77:0/910896584 learned_addr learned my addr 172.21.15.77:0/910896584 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:57:00.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.198+0000 7f52bd000640 1 --2- 172.21.15.77:0/910896584 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52b81034b0 0x7f52b819a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:00.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.198+0000 7f52be400640 1 --2- 172.21.15.77:0/910896584 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f52b810bfc0 0x7f52b819ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:00.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.198+0000 7f52bda00640 1 -- 172.21.15.77:0/910896584 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52b81034b0 msgr2=0x7f52b819a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:00.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.198+0000 7f52bda00640 1 --2- 172.21.15.77:0/910896584 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52b81034b0 0x7f52b819a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:00.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.198+0000 7f52bda00640 1 -- 172.21.15.77:0/910896584 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f52b810bfc0 msgr2=0x7f52b819ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:00.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.198+0000 7f52bda00640 1 --2- 172.21.15.77:0/910896584 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f52b810bfc0 0x7f52b819ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:00.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.198+0000 7f52bda00640 1 -- 172.21.15.77:0/910896584 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52b819f460 con 0x7f52b8102b90 2024-09-10T09:57:00.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.199+0000 7f52bda00640 1 --2- 172.21.15.77:0/910896584 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f52b8102b90 0x7f52b819a4b0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f52ac00e9f0 tx=0x7f52ac00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:00.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.199+0000 7f52b6000640 1 -- 172.21.15.77:0/910896584 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52ac00cd80 con 0x7f52b8102b90 2024-09-10T09:57:00.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.199+0000 7f52b6000640 1 -- 172.21.15.77:0/910896584 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52ac005500 con 0x7f52b8102b90 2024-09-10T09:57:00.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.199+0000 7f52bee00640 1 -- 172.21.15.77:0/910896584 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52b819f750 con 0x7f52b8102b90 2024-09-10T09:57:00.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.199+0000 7f52b6000640 1 -- 172.21.15.77:0/910896584 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52ac010690 con 0x7f52b8102b90 2024-09-10T09:57:00.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.199+0000 7f52bee00640 1 -- 172.21.15.77:0/910896584 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52b81a7030 con 0x7f52b8102b90 2024-09-10T09:57:00.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.202+0000 7f52c2ecc740 1 Processor -- start 2024-09-10T09:57:00.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.202+0000 7f52c2ecc740 1 -- start start 2024-09-10T09:57:00.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.202+0000 7f52b6000640 1 -- 172.21.15.77:0/910896584 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f52ac0046c0 con 0x7f52b8102b90 2024-09-10T09:57:00.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.203+0000 7f52c2ecc740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c6bf0a9d50 0x55c6bf0aa130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:00.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.203+0000 7f52c2ecc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6bf0aa670 0x55c6bf0b49f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:00.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.203+0000 7f52c2ecc740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c6bf0b4f30 0x55c6bf0b7320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:00.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.203+0000 7f52c2ecc740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c6bf0103e0 con 0x55c6bf0a9d50 2024-09-10T09:57:00.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.203+0000 7f52c2ecc740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c6befb7d80 con 0x55c6bf0b4f30 2024-09-10T09:57:00.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.203+0000 7f52c2ecc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c6bf00c3a0 con 0x55c6bf0aa670 2024-09-10T09:57:00.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.203+0000 7f52be400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c6bf0a9d50 0x55c6bf0aa130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:00.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.203+0000 7f52be400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c6bf0a9d50 0x55c6bf0aa130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42006/0 (socket says 172.21.15.77:42006) 2024-09-10T09:57:00.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.203+0000 7f52be400640 1 -- 172.21.15.77:0/2101152851 learned_addr learned my addr 172.21.15.77:0/2101152851 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:57:00.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.203+0000 7f52bd000640 1 --2- 172.21.15.77:0/2101152851 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c6bf0b4f30 0x55c6bf0b7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:00.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.203+0000 7f52bda00640 1 --2- 172.21.15.77:0/2101152851 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6bf0aa670 0x55c6bf0b49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:00.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.204+0000 7f52bd000640 1 -- 172.21.15.77:0/2101152851 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6bf0aa670 msgr2=0x55c6bf0b49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:00.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.204+0000 7f52bd000640 1 --2- 172.21.15.77:0/2101152851 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6bf0aa670 0x55c6bf0b49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:00.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.204+0000 7f52bd000640 1 -- 172.21.15.77:0/2101152851 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c6bf0a9d50 msgr2=0x55c6bf0aa130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:00.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.204+0000 7f52bd000640 1 --2- 172.21.15.77:0/2101152851 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c6bf0a9d50 0x55c6bf0aa130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:00.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.204+0000 7f52bd000640 1 -- 172.21.15.77:0/2101152851 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6befd6ae0 con 0x55c6bf0b4f30 2024-09-10T09:57:00.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.204+0000 7f52be400640 1 --2- 172.21.15.77:0/2101152851 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c6bf0a9d50 0x55c6bf0aa130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:57:00.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.204+0000 7f52b6000640 1 --2- 172.21.15.77:0/910896584 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f52a00a2b80 0x7f52a00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:00.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.204+0000 7f52bd000640 1 --2- 172.21.15.77:0/2101152851 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c6bf0b4f30 0x55c6bf0b7320 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f52a4009be0 tx=0x7f52a4011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14e3e8cc2a02060c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:00.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.204+0000 7f52b6000640 1 -- 172.21.15.77:0/910896584 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f52ac0cf8c0 con 0x7f52b8102b90 2024-09-10T09:57:00.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.204+0000 7f52aaa00640 1 -- 172.21.15.77:0/2101152851 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a4004050 con 0x55c6bf0b4f30 2024-09-10T09:57:00.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.204+0000 7f52aaa00640 1 -- 172.21.15.77:0/2101152851 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52a4005500 con 0x55c6bf0b4f30 2024-09-10T09:57:00.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.204+0000 7f52aaa00640 1 -- 172.21.15.77:0/2101152851 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a4004d70 con 0x55c6bf0b4f30 2024-09-10T09:57:00.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.204+0000 7f52be400640 1 --2- 172.21.15.77:0/910896584 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f52a00a2b80 0x7f52a00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:00.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.205+0000 7f52c2ecc740 1 -- 172.21.15.77:0/2101152851 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c6bf0b4f30 msgr2=0x55c6bf0b7320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:00.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.205+0000 7f52c2ecc740 1 --2- 172.21.15.77:0/2101152851 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c6bf0b4f30 0x55c6bf0b7320 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f52a4009be0 tx=0x7f52a4011810 comp rx=0 tx=0).stop 2024-09-10T09:57:00.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.205+0000 7f52be400640 1 --2- 172.21.15.77:0/910896584 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f52a00a2b80 0x7f52a00a5040 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f52b0005fa0 tx=0x7f52b0004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:00.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.205+0000 7f52c2ecc740 1 -- 172.21.15.77:0/2101152851 shutdown_connections 2024-09-10T09:57:00.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.205+0000 7f52c2ecc740 1 --2- 172.21.15.77:0/2101152851 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c6bf0b4f30 0x55c6bf0b7320 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:00.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.205+0000 7f52c2ecc740 1 --2- 172.21.15.77:0/2101152851 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6bf0aa670 0x55c6bf0b49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:00.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.205+0000 7f52c2ecc740 1 --2- 172.21.15.77:0/2101152851 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c6bf0a9d50 0x55c6bf0aa130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:00.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.205+0000 7f52c2ecc740 1 -- 172.21.15.77:0/2101152851 >> 172.21.15.77:0/2101152851 conn(0x55c6bf0a62e0 msgr2=0x55c6bf0a66c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:00.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.205+0000 7f52c2ecc740 1 -- 172.21.15.77:0/2101152851 shutdown_connections 2024-09-10T09:57:00.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.205+0000 7f52c2ecc740 1 -- 172.21.15.77:0/2101152851 wait complete. 2024-09-10T09:57:00.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.206+0000 7f52c2ecc740 1 -- start start 2024-09-10T09:57:00.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.206+0000 7f52c2ecc740 1 Processor -- start 2024-09-10T09:57:00.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.206+0000 7f52c2ecc740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c6bf0a9d50 0x55c6bf0b1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:00.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.206+0000 7f52c2ecc740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c6bf0aa670 0x55c6bf0b2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:00.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.206+0000 7f52c2ecc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6bf0b4f30 0x55c6bf0b2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:00.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.207+0000 7f52c2ecc740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c6bf0b7ba0 con 0x55c6bf0a9d50 2024-09-10T09:57:00.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.207+0000 7f52c2ecc740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c6befcb0f0 con 0x55c6bf0aa670 2024-09-10T09:57:00.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.207+0000 7f52c2ecc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c6bf0b7d20 con 0x55c6bf0b4f30 2024-09-10T09:57:00.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.207+0000 7f52c2ecc740 10 client.0 _ll_register_callbacks cb 0x55c6befbc9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:57:00.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.207+0000 7f52bda00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c6bf0a9d50 0x55c6bf0b1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:00.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.207+0000 7f52bda00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c6bf0a9d50 0x55c6bf0b1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42022/0 (socket says 172.21.15.77:42022) 2024-09-10T09:57:00.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.207+0000 7f52bda00640 1 -- 172.21.15.77:0/3637948281 learned_addr learned my addr 172.21.15.77:0/3637948281 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:57:00.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.207+0000 7f52bd000640 1 --2- 172.21.15.77:0/3637948281 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c6bf0aa670 0x55c6bf0b2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:00.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.207+0000 7f52be400640 1 --2- 172.21.15.77:0/3637948281 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6bf0b4f30 0x55c6bf0b2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:00.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.207+0000 7f52bda00640 1 -- 172.21.15.77:0/3637948281 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6bf0b4f30 msgr2=0x55c6bf0b2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:00.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.207+0000 7f52bda00640 1 --2- 172.21.15.77:0/3637948281 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6bf0b4f30 0x55c6bf0b2860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:00.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.207+0000 7f52bda00640 1 -- 172.21.15.77:0/3637948281 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c6bf0aa670 msgr2=0x55c6bf0b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:00.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.207+0000 7f52bda00640 1 --2- 172.21.15.77:0/3637948281 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c6bf0aa670 0x55c6bf0b2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:00.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.207+0000 7f52bda00640 1 -- 172.21.15.77:0/3637948281 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6bed813c0 con 0x55c6bf0a9d50 2024-09-10T09:57:00.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.207+0000 7f52be400640 1 --2- 172.21.15.77:0/3637948281 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6bf0b4f30 0x55c6bf0b2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:57:00.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.208+0000 7f52bda00640 1 --2- 172.21.15.77:0/3637948281 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c6bf0a9d50 0x55c6bf0b1de0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f52ac00e9c0 tx=0x7f52ac004ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd1a8c1fbf63b06a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:00.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.208+0000 7f529f400640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-10T09:57:00.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.208+0000 7f529f400640 1 -- 172.21.15.77:0/3637948281 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52ac09b690 con 0x55c6bf0a9d50 2024-09-10T09:57:00.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.208+0000 7f529f400640 1 -- 172.21.15.77:0/3637948281 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f52ac0a4710 con 0x55c6bf0a9d50 2024-09-10T09:57:00.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.208+0000 7f52c2ecc740 1 -- 172.21.15.77:0/3637948281 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52a4005500 con 0x55c6bf0a9d50 2024-09-10T09:57:00.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.208+0000 7f529f400640 1 -- 172.21.15.77:0/3637948281 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f52ac09c430 con 0x55c6bf0a9d50 2024-09-10T09:57:00.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.208+0000 7f52c2ecc740 10 client.15849 fetch_fsmap learned FSMap version 29 2024-09-10T09:57:00.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.208+0000 7f52c2ecc740 1 -- 172.21.15.77:0/3637948281 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c6bf0c3880 con 0x55c6bf0a9d50 2024-09-10T09:57:00.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.209+0000 7f529f400640 1 -- 172.21.15.77:0/3637948281 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52a4005500 con 0x55c6bf0a9d50 2024-09-10T09:57:00.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.209+0000 7f529f400640 1 -- 172.21.15.77:0/3637948281 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f52ac09abf0 con 0x55c6bf0a9d50 2024-09-10T09:57:00.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.209+0000 7f52c2ecc740 10 client.15849 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T09:57:00.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.209+0000 7f52c2ecc740 10 client.15849 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:57:00.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.209+0000 7f52c2ecc740 20 client.15849 populate_metadata read hostname 'smithi077' 2024-09-10T09:57:00.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.209+0000 7f52c2ecc740 1 --2- 172.21.15.77:0/3637948281 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55c6bf0c4c90 0x55c6bf0c7100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:00.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.209+0000 7f52c2ecc740 4 client.15849 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:57:00.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.209+0000 7f52c2ecc740 1 -- 172.21.15.77:0/3637948281 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c6bf01d950 con 0x55c6bf0c4c90 2024-09-10T09:57:00.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.209+0000 7f52bd000640 1 --2- 172.21.15.77:0/3637948281 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55c6bf0c4c90 0x55c6bf0c7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:00.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.210+0000 7f52bd000640 1 --2- 172.21.15.77:0/3637948281 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55c6bf0c4c90 0x55c6bf0c7100 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd6d2d2a9faedeed server_cookie=ae429ebd767f9b37 in_seq=0 out_seq=0 2024-09-10T09:57:00.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.210+0000 7f529f400640 10 client.15849 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T09:57:00.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.211+0000 7f529f400640 1 -- 172.21.15.77:0/3637948281 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c6bf01d950 con 0x55c6bf0c4c90 2024-09-10T09:57:00.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.211+0000 7f529f400640 10 client.15849 handle_command_reply: tid=0 2024-09-10T09:57:00.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.211+0000 7f52c2ecc740 1 client.15849 shutdown 2024-09-10T09:57:00.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.211+0000 7f52c2ecc740 1 -- 172.21.15.77:0/3637948281 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c6bf0a9d50 msgr2=0x55c6bf0b1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:00.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.211+0000 7f52c2ecc740 1 --2- 172.21.15.77:0/3637948281 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c6bf0a9d50 0x55c6bf0b1de0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f52ac00e9c0 tx=0x7f52ac004ba0 comp rx=0 tx=0).stop 2024-09-10T09:57:00.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.211+0000 7f52c2ecc740 1 -- 172.21.15.77:0/3637948281 shutdown_connections 2024-09-10T09:57:00.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.211+0000 7f52c2ecc740 1 --2- 172.21.15.77:0/3637948281 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55c6bf0c4c90 0x55c6bf0c7100 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:00.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.211+0000 7f52c2ecc740 1 --2- 172.21.15.77:0/3637948281 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6bf0b4f30 0x55c6bf0b2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:00.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.211+0000 7f52c2ecc740 1 --2- 172.21.15.77:0/3637948281 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c6bf0aa670 0x55c6bf0b2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:00.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.211+0000 7f52c2ecc740 1 --2- 172.21.15.77:0/3637948281 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c6bf0a9d50 0x55c6bf0b1de0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:00.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.211+0000 7f52c2ecc740 1 -- 172.21.15.77:0/3637948281 >> 172.21.15.77:0/3637948281 conn(0x55c6bf0a62e0 msgr2=0x55c6bf0ac010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:00.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.211+0000 7f529f400640 0 client.15849 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T09:57:00.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.211+0000 7f52c2ecc740 1 -- 172.21.15.77:0/3637948281 shutdown_connections 2024-09-10T09:57:00.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.211+0000 7f52c2ecc740 1 -- 172.21.15.77:0/3637948281 wait complete. 2024-09-10T09:57:00.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.212+0000 7f52c2ecc740 20 client.15849 trim_cache size 0 max 16384 2024-09-10T09:57:00.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.230+0000 7f52c2ecc740 1 Processor -- start 2024-09-10T09:57:00.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.230+0000 7f52c2ecc740 1 -- start start 2024-09-10T09:57:00.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.230+0000 7f52c2ecc740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c6bf0a9d50 0x55c6bf0ac7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:00.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.230+0000 7f52c2ecc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6bf0aa670 0x55c6bf0b3d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:00.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.230+0000 7f52c2ecc740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c6bf0b4f30 0x55c6bf0cca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:00.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.230+0000 7f52c2ecc740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f52b81b0f80 con 0x55c6bf0a9d50 2024-09-10T09:57:00.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.230+0000 7f52c2ecc740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f52b81b2ac0 con 0x55c6bf0b4f30 2024-09-10T09:57:00.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.230+0000 7f52c2ecc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f52b81b1650 con 0x55c6bf0aa670 2024-09-10T09:57:00.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.230+0000 7f52bda00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c6bf0a9d50 0x55c6bf0ac7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:00.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.230+0000 7f52bda00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c6bf0a9d50 0x55c6bf0ac7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42026/0 (socket says 172.21.15.77:42026) 2024-09-10T09:57:00.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.230+0000 7f52bda00640 1 -- 172.21.15.77:0/2038025759 learned_addr learned my addr 172.21.15.77:0/2038025759 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:57:00.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.231+0000 7f52be400640 1 --2- 172.21.15.77:0/2038025759 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c6bf0b4f30 0x55c6bf0cca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:00.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.231+0000 7f52bda00640 1 -- 172.21.15.77:0/2038025759 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6bf0aa670 msgr2=0x55c6bf0b3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:00.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.231+0000 7f52bda00640 1 --2- 172.21.15.77:0/2038025759 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6bf0aa670 0x55c6bf0b3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:00.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.231+0000 7f52bda00640 1 -- 172.21.15.77:0/2038025759 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c6bf0b4f30 msgr2=0x55c6bf0cca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:00.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.231+0000 7f52bda00640 1 --2- 172.21.15.77:0/2038025759 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c6bf0b4f30 0x55c6bf0cca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:00.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.231+0000 7f52bda00640 1 -- 172.21.15.77:0/2038025759 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6bf0a7470 con 0x55c6bf0a9d50 2024-09-10T09:57:00.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.231+0000 7f52bda00640 1 --2- 172.21.15.77:0/2038025759 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c6bf0a9d50 0x55c6bf0ac7b0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f52ac00c6a0 tx=0x7f52ac09a240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a54133de227090d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:00.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.231+0000 7f52aa000640 1 -- 172.21.15.77:0/2038025759 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52ac09bb00 con 0x55c6bf0a9d50 2024-09-10T09:57:00.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.231+0000 7f52aa000640 1 -- 172.21.15.77:0/2038025759 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52ac0a49a0 con 0x55c6bf0a9d50 2024-09-10T09:57:00.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.231+0000 7f52aa000640 1 -- 172.21.15.77:0/2038025759 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52ac09b180 con 0x55c6bf0a9d50 2024-09-10T09:57:00.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.232+0000 7f52c2ecc740 1 -- 172.21.15.77:0/2038025759 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c6bf0a9d50 msgr2=0x55c6bf0ac7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:00.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.232+0000 7f52c2ecc740 1 --2- 172.21.15.77:0/2038025759 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c6bf0a9d50 0x55c6bf0ac7b0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f52ac00c6a0 tx=0x7f52ac09a240 comp rx=0 tx=0).stop 2024-09-10T09:57:00.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.232+0000 7f52c2ecc740 1 -- 172.21.15.77:0/2038025759 shutdown_connections 2024-09-10T09:57:00.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.232+0000 7f52c2ecc740 1 --2- 172.21.15.77:0/2038025759 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c6bf0b4f30 0x55c6bf0cca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:00.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.232+0000 7f52c2ecc740 1 --2- 172.21.15.77:0/2038025759 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6bf0aa670 0x55c6bf0b3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:00.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.232+0000 7f52c2ecc740 1 --2- 172.21.15.77:0/2038025759 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c6bf0a9d50 0x55c6bf0ac7b0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:00.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.232+0000 7f52c2ecc740 1 -- 172.21.15.77:0/2038025759 >> 172.21.15.77:0/2038025759 conn(0x55c6bf0a62e0 msgr2=0x55c6bf0b3640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:00.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.232+0000 7f52c2ecc740 1 -- 172.21.15.77:0/2038025759 shutdown_connections 2024-09-10T09:57:00.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.232+0000 7f52c2ecc740 1 -- 172.21.15.77:0/2038025759 wait complete. 2024-09-10T09:57:00.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.233+0000 7f52c2ecc740 1 -- start start 2024-09-10T09:57:00.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.233+0000 7f52c2ecc740 1 Processor -- start 2024-09-10T09:57:00.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.233+0000 7f52c2ecc740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c6bf0a9d50 0x55c6bf0bd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:00.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.233+0000 7f52c2ecc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6bf0aa670 0x55c6bf0ae910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:00.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.233+0000 7f52c2ecc740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c6bf0b4f30 0x55c6bf0aee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:00.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.233+0000 7f52c2ecc740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f52b81b23f0 con 0x55c6bf0b4f30 2024-09-10T09:57:00.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.233+0000 7f52c2ecc740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f52b81b1d20 con 0x55c6bf0a9d50 2024-09-10T09:57:00.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.233+0000 7f52c2ecc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f52b81aef60 con 0x55c6bf0aa670 2024-09-10T09:57:00.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.234+0000 7f52c2ecc740 10 client.0 _ll_register_callbacks cb 0x7f52ac00cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:57:00.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.234+0000 7f52be400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c6bf0b4f30 0x55c6bf0aee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:00.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.234+0000 7f52bda00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c6bf0a9d50 0x55c6bf0bd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:00.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.234+0000 7f52be400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c6bf0b4f30 0x55c6bf0aee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42036/0 (socket says 172.21.15.77:42036) 2024-09-10T09:57:00.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.234+0000 7f52be400640 1 -- 172.21.15.77:0/1631575395 learned_addr learned my addr 172.21.15.77:0/1631575395 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:57:00.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.234+0000 7f52bd000640 1 --2- 172.21.15.77:0/1631575395 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6bf0aa670 0x55c6bf0ae910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:00.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.234+0000 7f52bda00640 1 -- 172.21.15.77:0/1631575395 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6bf0aa670 msgr2=0x55c6bf0ae910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:00.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.234+0000 7f52bda00640 1 --2- 172.21.15.77:0/1631575395 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6bf0aa670 0x55c6bf0ae910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:00.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.234+0000 7f52bda00640 1 -- 172.21.15.77:0/1631575395 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c6bf0b4f30 msgr2=0x55c6bf0aee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:00.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.234+0000 7f52bda00640 1 --2- 172.21.15.77:0/1631575395 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c6bf0b4f30 0x55c6bf0aee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:00.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.234+0000 7f52bda00640 1 -- 172.21.15.77:0/1631575395 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6bf0af440 con 0x55c6bf0a9d50 2024-09-10T09:57:00.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.234+0000 7f52bda00640 1 --2- 172.21.15.77:0/1631575395 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c6bf0a9d50 0x55c6bf0bd4b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f52ac0a4b60 tx=0x7f52ac09ae80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e523310b3e4849c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:00.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.234+0000 7f529f400640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T09:57:00.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.235+0000 7f529f400640 1 -- 172.21.15.77:0/1631575395 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52ac0d5040 con 0x55c6bf0a9d50 2024-09-10T09:57:00.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.235+0000 7f529f400640 1 -- 172.21.15.77:0/1631575395 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f52ac09b180 con 0x55c6bf0a9d50 2024-09-10T09:57:00.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.235+0000 7f52c2ecc740 1 -- 172.21.15.77:0/1631575395 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c6bf017e60 con 0x55c6bf0a9d50 2024-09-10T09:57:00.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.235+0000 7f529f400640 1 -- 172.21.15.77:0/1631575395 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f52ac0a00e0 con 0x55c6bf0a9d50 2024-09-10T09:57:00.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.235+0000 7f52c2ecc740 10 client.25925 fetch_fsmap learned FSMap version 29 2024-09-10T09:57:00.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.235+0000 7f52c2ecc740 1 -- 172.21.15.77:0/1631575395 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c6bf0af440 con 0x55c6bf0a9d50 2024-09-10T09:57:00.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.235+0000 7f529f400640 1 -- 172.21.15.77:0/1631575395 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55c6bf017e60 con 0x55c6bf0a9d50 2024-09-10T09:57:00.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.235+0000 7f529f400640 1 -- 172.21.15.77:0/1631575395 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f52ac086360 con 0x55c6bf0a9d50 2024-09-10T09:57:00.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.235+0000 7f52c2ecc740 10 client.25925 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T09:57:00.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.235+0000 7f52c2ecc740 10 client.25925 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:57:00.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.235+0000 7f52c2ecc740 20 client.25925 populate_metadata read hostname 'smithi077' 2024-09-10T09:57:00.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.235+0000 7f52c2ecc740 1 --2- 172.21.15.77:0/1631575395 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55c6bf12fb00 0x55c6bf131ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:00.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.235+0000 7f52c2ecc740 4 client.25925 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T09:57:00.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.235+0000 7f52c2ecc740 1 -- 172.21.15.77:0/1631575395 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c6bef940c0 con 0x55c6bf12fb00 2024-09-10T09:57:00.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.236+0000 7f52bd000640 1 --2- 172.21.15.77:0/1631575395 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55c6bf12fb00 0x55c6bf131ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:00.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.236+0000 7f52bd000640 1 --2- 172.21.15.77:0/1631575395 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55c6bf12fb00 0x55c6bf131ef0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1259597e069face5 server_cookie=92119a713b22a4a in_seq=0 out_seq=0 2024-09-10T09:57:00.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.236+0000 7f529f400640 10 client.25925 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T09:57:00.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.236+0000 7f529f400640 1 -- 172.21.15.77:0/1631575395 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55c6bef940c0 con 0x55c6bf12fb00 2024-09-10T09:57:00.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.236+0000 7f529f400640 10 client.25925 handle_command_reply: tid=0 2024-09-10T09:57:00.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.236+0000 7f52c2ecc740 1 client.25925 shutdown 2024-09-10T09:57:00.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.237+0000 7f52c2ecc740 1 -- 172.21.15.77:0/1631575395 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c6bf0a9d50 msgr2=0x55c6bf0bd4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:00.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.237+0000 7f52c2ecc740 1 --2- 172.21.15.77:0/1631575395 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c6bf0a9d50 0x55c6bf0bd4b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f52ac0a4b60 tx=0x7f52ac09ae80 comp rx=0 tx=0).stop 2024-09-10T09:57:00.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.237+0000 7f52c2ecc740 1 -- 172.21.15.77:0/1631575395 shutdown_connections 2024-09-10T09:57:00.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.237+0000 7f52c2ecc740 1 --2- 172.21.15.77:0/1631575395 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55c6bf12fb00 0x55c6bf131ef0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:00.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.237+0000 7f52c2ecc740 1 --2- 172.21.15.77:0/1631575395 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c6bf0b4f30 0x55c6bf0aee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:00.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.237+0000 7f52c2ecc740 1 --2- 172.21.15.77:0/1631575395 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6bf0aa670 0x55c6bf0ae910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:00.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.237+0000 7f529f400640 0 client.25925 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T09:57:00.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.237+0000 7f52c2ecc740 1 --2- 172.21.15.77:0/1631575395 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c6bf0a9d50 0x55c6bf0bd4b0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:00.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.237+0000 7f52c2ecc740 1 -- 172.21.15.77:0/1631575395 >> 172.21.15.77:0/1631575395 conn(0x55c6bf0a62e0 msgr2=0x55c6bf0b25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:00.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.237+0000 7f52c2ecc740 1 -- 172.21.15.77:0/1631575395 shutdown_connections 2024-09-10T09:57:00.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.237+0000 7f52c2ecc740 1 -- 172.21.15.77:0/1631575395 wait complete. 2024-09-10T09:57:00.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.237+0000 7f52c2ecc740 20 client.25925 trim_cache size 0 max 16384 2024-09-10T09:57:00.241 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-10T09:57:00.242 INFO:teuthology.orchestra.run.smithi077.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T09:57:00.242 INFO:teuthology.orchestra.run.smithi077.stdout: "scrubs": { 2024-09-10T09:57:00.242 INFO:teuthology.orchestra.run.smithi077.stdout: "39564bc9-b545-45f5-95eb-bbdde588ce78": { 2024-09-10T09:57:00.242 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/", 2024-09-10T09:57:00.242 INFO:teuthology.orchestra.run.smithi077.stdout: "tag": "39564bc9-b545-45f5-95eb-bbdde588ce78", 2024-09-10T09:57:00.242 INFO:teuthology.orchestra.run.smithi077.stdout: "options": "recursive,force" 2024-09-10T09:57:00.242 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T09:57:00.242 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T09:57:00.242 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-10T09:57:00.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.238+0000 7f52bee00640 1 -- 172.21.15.77:0/910896584 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f52a00a2b80 msgr2=0x7f52a00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:00.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.238+0000 7f52bee00640 1 --2- 172.21.15.77:0/910896584 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f52a00a2b80 0x7f52a00a5040 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f52b0005fa0 tx=0x7f52b0004a40 comp rx=0 tx=0).stop 2024-09-10T09:57:00.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.238+0000 7f52bee00640 1 -- 172.21.15.77:0/910896584 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f52b8102b90 msgr2=0x7f52b819a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:00.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.238+0000 7f52bee00640 1 --2- 172.21.15.77:0/910896584 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f52b8102b90 0x7f52b819a4b0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f52ac00e9f0 tx=0x7f52ac00edf0 comp rx=0 tx=0).stop 2024-09-10T09:57:00.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.238+0000 7f52bee00640 1 -- 172.21.15.77:0/910896584 shutdown_connections 2024-09-10T09:57:00.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.238+0000 7f52bee00640 1 --2- 172.21.15.77:0/910896584 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f52a00a2b80 0x7f52a00a5040 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:00.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.238+0000 7f52bee00640 1 --2- 172.21.15.77:0/910896584 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f52b810bfc0 0x7f52b819ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:00.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.238+0000 7f52bee00640 1 --2- 172.21.15.77:0/910896584 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52b81034b0 0x7f52b819a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:00.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.238+0000 7f52bee00640 1 --2- 172.21.15.77:0/910896584 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f52b8102b90 0x7f52b819a4b0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:00.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.238+0000 7f52bee00640 1 -- 172.21.15.77:0/910896584 >> 172.21.15.77:0/910896584 conn(0x7f52b80fc610 msgr2=0x7f52b80fd210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:00.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.238+0000 7f52bee00640 1 -- 172.21.15.77:0/910896584 shutdown_connections 2024-09-10T09:57:00.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:00.238+0000 7f52bee00640 1 -- 172.21.15.77:0/910896584 wait complete. 2024-09-10T09:57:00.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:57:00 smithi077 ceph-mon[30319]: pgmap v446: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 85 B/s wr, 12 op/s 2024-09-10T09:57:00.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:57:00 smithi077 ceph-mon[30319]: scrub summary: idle 2024-09-10T09:57:01.104 INFO:tasks.cephfs.filesystem:scrub status for tag:39564bc9-b545-45f5-95eb-bbdde588ce78 - {'path': '/', 'tag': '39564bc9-b545-45f5-95eb-bbdde588ce78', 'options': 'recursive,force'} 2024-09-10T09:57:01.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:57:00 smithi134 ceph-mon[35674]: pgmap v446: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 85 B/s wr, 12 op/s 2024-09-10T09:57:01.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:57:00 smithi134 ceph-mon[35674]: scrub summary: idle 2024-09-10T09:57:01.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:57:00 smithi120 ceph-mon[34770]: pgmap v446: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 85 B/s wr, 12 op/s 2024-09-10T09:57:01.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:57:00 smithi120 ceph-mon[34770]: scrub summary: idle 2024-09-10T09:57:03.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:57:02 smithi134 ceph-mon[35674]: pgmap v447: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-10T09:57:03.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:57:02 smithi120 ceph-mon[34770]: pgmap v447: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-10T09:57:03.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:57:02 smithi077 ceph-mon[30319]: pgmap v447: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-10T09:57:05.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:57:04 smithi134 ceph-mon[35674]: pgmap v448: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 85 B/s wr, 11 op/s 2024-09-10T09:57:05.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:57:04 smithi120 ceph-mon[34770]: pgmap v448: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 85 B/s wr, 11 op/s 2024-09-10T09:57:05.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:57:04 smithi077 ceph-mon[30319]: pgmap v448: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 85 B/s wr, 11 op/s 2024-09-10T09:57:07.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:57:06 smithi134 ceph-mon[35674]: pgmap v449: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-10T09:57:07.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:57:06 smithi120 ceph-mon[34770]: pgmap v449: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-10T09:57:07.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:57:06 smithi077 ceph-mon[30319]: pgmap v449: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-10T09:57:09.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:57:08 smithi134 ceph-mon[35674]: pgmap v450: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-10T09:57:09.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:57:08 smithi120 ceph-mon[34770]: pgmap v450: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-10T09:57:09.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:57:08 smithi077 ceph-mon[30319]: pgmap v450: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-10T09:57:11.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:57:10 smithi134 ceph-mon[35674]: pgmap v451: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 12 op/s 2024-09-10T09:57:11.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:57:10 smithi120 ceph-mon[34770]: pgmap v451: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 12 op/s 2024-09-10T09:57:11.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:57:10 smithi077 ceph-mon[30319]: pgmap v451: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 12 op/s 2024-09-10T09:57:13.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:57:12 smithi134 ceph-mon[35674]: pgmap v452: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-09-10T09:57:13.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:57:12 smithi120 ceph-mon[34770]: pgmap v452: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-09-10T09:57:13.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:57:12 smithi077 ceph-mon[30319]: pgmap v452: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-09-10T09:57:15.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:57:14 smithi134 ceph-mon[35674]: pgmap v453: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-10T09:57:15.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:57:14 smithi120 ceph-mon[34770]: pgmap v453: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-10T09:57:15.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:57:14 smithi077 ceph-mon[30319]: pgmap v453: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-10T09:57:16.234 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:57:16.625 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:57:16.928 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:57:16 smithi077 ceph-mon[30319]: pgmap v454: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-09-10T09:57:17.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:57:16 smithi134 ceph-mon[35674]: pgmap v454: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-09-10T09:57:17.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:57:16 smithi120 ceph-mon[34770]: pgmap v454: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-09-10T09:57:17.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:17.191+0000 7fe8c4c00640 1 -- 172.21.15.77:0/1202847101 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8c010e280 msgr2=0x7fe8c0110670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:17.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:17.191+0000 7fe8c4c00640 1 --2- 172.21.15.77:0/1202847101 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8c010e280 0x7fe8c0110670 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8009ca0 tx=0x7fe8b802a950 comp rx=0 tx=0).stop 2024-09-10T09:57:17.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:17.191+0000 7fe8c4c00640 1 -- 172.21.15.77:0/1202847101 shutdown_connections 2024-09-10T09:57:17.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:17.191+0000 7fe8c4c00640 1 --2- 172.21.15.77:0/1202847101 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8c010e280 0x7fe8c0110670 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:17.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:17.191+0000 7fe8c4c00640 1 --2- 172.21.15.77:0/1202847101 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe8c0074e30 0x7fe8c010dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:17.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:17.191+0000 7fe8c4c00640 1 --2- 172.21.15.77:0/1202847101 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe8c0074510 0x7fe8c00748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:17.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:17.191+0000 7fe8c4c00640 1 -- 172.21.15.77:0/1202847101 >> 172.21.15.77:0/1202847101 conn(0x7fe8c00fd740 msgr2=0x7fe8c00ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:17.194 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:17.191+0000 7fe8c4c00640 1 -- 172.21.15.77:0/1202847101 shutdown_connections 2024-09-10T09:57:17.194 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:17.191+0000 7fe8c4c00640 1 -- 172.21.15.77:0/1202847101 wait complete. 2024-09-10T09:57:17.194 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:17.192+0000 7fe8c4c00640 1 Processor -- start 2024-09-10T09:57:17.194 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:17.192+0000 7fe8c4c00640 1 -- start start 2024-09-10T09:57:17.195 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:17.193+0000 7fe8c4c00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe8c0074510 0x7fe8c019a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:17.195 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:17.193+0000 7fe8c4c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe8c0074e30 0x7fe8c019a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:17.195 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:17.193+0000 7fe8c4c00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8c010e280 0x7fe8c019ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:17.195 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:17.193+0000 7fe8c4c00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c0113fc0 con 0x7fe8c010e280 2024-09-10T09:57:17.195 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:17.193+0000 7fe8c4c00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c0113e40 con 0x7fe8c0074510 2024-09-10T09:57:17.195 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:17.193+0000 7fe8c4c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c0114140 con 0x7fe8c0074e30 2024-09-10T09:57:17.196 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:17.193+0000 7fe8bf400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe8c0074510 0x7fe8c019a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:17.196 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:17.193+0000 7fe8bfe00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8c010e280 0x7fe8c019ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:17.196 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:17.193+0000 7fe8bf400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe8c0074510 0x7fe8c019a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:45906/0 (socket says 172.21.15.77:45906) 2024-09-10T09:57:17.196 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:17.193+0000 7fe8bf400640 1 -- 172.21.15.77:0/1651060043 learned_addr learned my addr 172.21.15.77:0/1651060043 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:57:17.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:17.194+0000 7fe8bea00640 1 --2- 172.21.15.77:0/1651060043 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe8c0074e30 0x7fe8c019a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:17.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:17.194+0000 7fe8bf400640 1 -- 172.21.15.77:0/1651060043 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe8c0074e30 msgr2=0x7fe8c019a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:17.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:17.194+0000 7fe8bf400640 1 --2- 172.21.15.77:0/1651060043 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe8c0074e30 0x7fe8c019a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:17.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:17.194+0000 7fe8bf400640 1 -- 172.21.15.77:0/1651060043 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8c010e280 msgr2=0x7fe8c019ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:17.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:17.194+0000 7fe8bf400640 1 --2- 172.21.15.77:0/1651060043 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8c010e280 0x7fe8c019ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:17.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:17.194+0000 7fe8bf400640 1 -- 172.21.15.77:0/1651060043 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8c019f430 con 0x7fe8c0074510 2024-09-10T09:57:17.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:17.194+0000 7fe8bf400640 1 --2- 172.21.15.77:0/1651060043 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe8c0074510 0x7fe8c019a480 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe8b000d950 tx=0x7fe8b000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:17.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:17.194+0000 7fe8b7e00640 1 -- 172.21.15.77:0/1651060043 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b0017070 con 0x7fe8c0074510 2024-09-10T09:57:17.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:17.194+0000 7fe8c4c00640 1 -- 172.21.15.77:0/1651060043 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8c019f720 con 0x7fe8c0074510 2024-09-10T09:57:17.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:17.195+0000 7fe8b7e00640 1 -- 172.21.15.77:0/1651060043 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8b0009c30 con 0x7fe8c0074510 2024-09-10T09:57:17.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:17.195+0000 7fe8b7e00640 1 -- 172.21.15.77:0/1651060043 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b000f6b0 con 0x7fe8c0074510 2024-09-10T09:57:17.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:17.195+0000 7fe8c4c00640 1 -- 172.21.15.77:0/1651060043 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8c01a7000 con 0x7fe8c0074510 2024-09-10T09:57:17.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:17.196+0000 7fe8c4c00640 1 -- 172.21.15.77:0/1651060043 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8c00748f0 con 0x7fe8c0074510 2024-09-10T09:57:17.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:17.198+0000 7fe8b7e00640 1 -- 172.21.15.77:0/1651060043 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe8b00109d0 con 0x7fe8c0074510 2024-09-10T09:57:17.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:17.199+0000 7fe8b7e00640 1 --2- 172.21.15.77:0/1651060043 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fe8a40a2b80 0x7fe8a40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:17.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:17.199+0000 7fe8b7e00640 1 -- 172.21.15.77:0/1651060043 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fe8b00cefe0 con 0x7fe8c0074510 2024-09-10T09:57:17.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:17.199+0000 7fe8bea00640 1 --2- 172.21.15.77:0/1651060043 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fe8a40a2b80 0x7fe8a40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:17.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:17.200+0000 7fe8bea00640 1 --2- 172.21.15.77:0/1651060043 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fe8a40a2b80 0x7fe8a40a5040 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe8c010f610 tx=0x7fe8ac005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:17.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:17.201+0000 7fe8b7e00640 1 -- 172.21.15.77:0/1651060043 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe8b0009de0 con 0x7fe8c0074510 2024-09-10T09:57:17.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:17.418+0000 7fe8c4c00640 1 -- 172.21.15.77:0/1651060043 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe8c002d7c0 con 0x7fe8c0074510 2024-09-10T09:57:17.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:17.419+0000 7fe8b7e00640 1 -- 172.21.15.77:0/1651060043 <== mon.2 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11989 (secure 0 0 0) 0x7fe8b009b500 con 0x7fe8c0074510 2024-09-10T09:57:17.421 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:57:17.421 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":29,"btime":"2024-09-10T09:55:13:863715+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3159289675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3159289675},{"type":"v1","addr":"172.21.15.77:6835","nonce":3159289675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1951962341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1951962341},{"type":"v1","addr":"172.21.15.77:6837","nonce":1951962341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:52:23.368252+0000","modified":"2024-09-10T09:55:13.786975+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24563,"mds_1":24524,"mds_2":24520,"mds_3":24502,"mds_4":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14487":{"gid":14487,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/4291121888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":4291121888},{"type":"v1","addr":"172.21.15.77:6839","nonce":4291121888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14499":{"gid":14499,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6841/445284194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":445284194},{"type":"v1","addr":"172.21.15.77:6841","nonce":445284194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1005054423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1005054423},{"type":"v1","addr":"172.21.15.134:6833","nonce":1005054423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6833/3716307744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":3716307744},{"type":"v1","addr":"172.21.15.120:6833","nonce":3716307744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/1319892544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1319892544},{"type":"v1","addr":"172.21.15.120:6835","nonce":1319892544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.134:6835/3293933699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3293933699},{"type":"v1","addr":"172.21.15.134:6835","nonce":3293933699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.134:6837/2461497091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2461497091},{"type":"v1","addr":"172.21.15.134:6837","nonce":2461497091}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":20,"addr":"172.21.15.134:6839/1965117680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1965117680},{"type":"v1","addr":"172.21.15.134:6839","nonce":1965117680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":20,"addr":"172.21.15.120:6837/3033280709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3033280709},{"type":"v1","addr":"172.21.15.120:6837","nonce":3033280709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.120:6839/1777710157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1777710157},{"type":"v1","addr":"172.21.15.120:6839","nonce":1777710157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24563,"qdb_cluster":[24496,24502,24520,24524,24563]},"id":1}]} 2024-09-10T09:57:17.422 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 29 2024-09-10T09:57:17.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:17.421+0000 7fe8c4c00640 1 -- 172.21.15.77:0/1651060043 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fe8a40a2b80 msgr2=0x7fe8a40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:17.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:17.421+0000 7fe8c4c00640 1 --2- 172.21.15.77:0/1651060043 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fe8a40a2b80 0x7fe8a40a5040 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe8c010f610 tx=0x7fe8ac005f90 comp rx=0 tx=0).stop 2024-09-10T09:57:17.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:17.421+0000 7fe8c4c00640 1 -- 172.21.15.77:0/1651060043 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe8c0074510 msgr2=0x7fe8c019a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:17.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:17.421+0000 7fe8c4c00640 1 --2- 172.21.15.77:0/1651060043 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe8c0074510 0x7fe8c019a480 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe8b000d950 tx=0x7fe8b000de10 comp rx=0 tx=0).stop 2024-09-10T09:57:17.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:17.421+0000 7fe8c4c00640 1 -- 172.21.15.77:0/1651060043 shutdown_connections 2024-09-10T09:57:17.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:17.421+0000 7fe8c4c00640 1 --2- 172.21.15.77:0/1651060043 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fe8a40a2b80 0x7fe8a40a5040 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:17.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:17.421+0000 7fe8c4c00640 1 --2- 172.21.15.77:0/1651060043 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8c010e280 0x7fe8c019ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:17.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:17.421+0000 7fe8c4c00640 1 --2- 172.21.15.77:0/1651060043 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe8c0074e30 0x7fe8c019a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:17.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:17.421+0000 7fe8c4c00640 1 --2- 172.21.15.77:0/1651060043 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe8c0074510 0x7fe8c019a480 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:17.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:17.421+0000 7fe8c4c00640 1 -- 172.21.15.77:0/1651060043 >> 172.21.15.77:0/1651060043 conn(0x7fe8c00fd740 msgr2=0x7fe8c00fe440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:17.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:17.422+0000 7fe8c4c00640 1 -- 172.21.15.77:0/1651060043 shutdown_connections 2024-09-10T09:57:17.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:17.422+0000 7fe8c4c00640 1 -- 172.21.15.77:0/1651060043 wait complete. 2024-09-10T09:57:18.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:57:17 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/1651060043' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:57:18.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:57:17 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/1651060043' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:57:18.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:57:17 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/1651060043' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:57:18.290 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24563 quiesce db --set-id b59524e9 --release --await' 2024-09-10T09:57:18.290 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24563 quiesce db --set-id b59524e9 --release --await 2024-09-10T09:57:18.659 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:57:19.123 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:57:18 smithi077 ceph-mon[30319]: pgmap v455: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-09-10T09:57:19.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:57:18 smithi134 ceph-mon[35674]: pgmap v455: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-09-10T09:57:19.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:57:18 smithi120 ceph-mon[34770]: pgmap v455: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-09-10T09:57:19.247 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.245+0000 7f20a9600640 1 -- 172.21.15.77:0/2713331199 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f20a4101df0 msgr2=0x7f20a41021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:19.247 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.245+0000 7f20a9600640 1 --2- 172.21.15.77:0/2713331199 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f20a4101df0 0x7f20a41021d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2094009900 tx=0x7f209402a3f0 comp rx=0 tx=0).stop 2024-09-10T09:57:19.247 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.245+0000 7f20a9600640 1 -- 172.21.15.77:0/2713331199 shutdown_connections 2024-09-10T09:57:19.247 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.245+0000 7f20a9600640 1 --2- 172.21.15.77:0/2713331199 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f20a4106f20 0x7f20a410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:19.247 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.245+0000 7f20a9600640 1 --2- 172.21.15.77:0/2713331199 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f20a41027a0 0x7f20a41067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:19.248 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.245+0000 7f20a9600640 1 --2- 172.21.15.77:0/2713331199 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f20a4101df0 0x7f20a41021d0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:19.248 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.245+0000 7f20a9600640 1 -- 172.21.15.77:0/2713331199 >> 172.21.15.77:0/2713331199 conn(0x7f20a40fd8b0 msgr2=0x7f20a40ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:19.248 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.245+0000 7f20a9600640 1 -- 172.21.15.77:0/2713331199 shutdown_connections 2024-09-10T09:57:19.248 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.245+0000 7f20a9600640 1 -- 172.21.15.77:0/2713331199 wait complete. 2024-09-10T09:57:19.249 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.246+0000 7f20a9600640 1 Processor -- start 2024-09-10T09:57:19.250 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.246+0000 7f20a9600640 1 -- start start 2024-09-10T09:57:19.250 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.246+0000 7f20a9600640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f20a4101df0 0x7f20a40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:19.250 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.246+0000 7f20a9600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f20a41027a0 0x7f20a4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:19.250 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.246+0000 7f20a9600640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f20a4106f20 0x7f20a4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:19.250 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.247+0000 7f20a9600640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f20a4111fd0 con 0x7f20a4101df0 2024-09-10T09:57:19.250 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.247+0000 7f20a9600640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f20a4111e50 con 0x7f20a4106f20 2024-09-10T09:57:19.250 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.247+0000 7f20a9600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f20a4112150 con 0x7f20a41027a0 2024-09-10T09:57:19.250 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.247+0000 7f20a2000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f20a41027a0 0x7f20a4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:19.250 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.247+0000 7f20a2000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f20a41027a0 0x7f20a4074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:56470/0 (socket says 172.21.15.77:56470) 2024-09-10T09:57:19.250 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.247+0000 7f20a2000640 1 -- 172.21.15.77:0/2858344067 learned_addr learned my addr 172.21.15.77:0/2858344067 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:57:19.250 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.247+0000 7f20a2a00640 1 --2- 172.21.15.77:0/2858344067 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f20a4101df0 0x7f20a40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:19.250 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.247+0000 7f20a3400640 1 --2- 172.21.15.77:0/2858344067 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f20a4106f20 0x7f20a4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:19.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.247+0000 7f20a3400640 1 -- 172.21.15.77:0/2858344067 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f20a41027a0 msgr2=0x7f20a4074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:19.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.247+0000 7f20a3400640 1 --2- 172.21.15.77:0/2858344067 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f20a41027a0 0x7f20a4074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:19.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.247+0000 7f20a3400640 1 -- 172.21.15.77:0/2858344067 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f20a4101df0 msgr2=0x7f20a40744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:19.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.247+0000 7f20a3400640 1 --2- 172.21.15.77:0/2858344067 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f20a4101df0 0x7f20a40744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:19.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.247+0000 7f20a3400640 1 -- 172.21.15.77:0/2858344067 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20a41a6970 con 0x7f20a4106f20 2024-09-10T09:57:19.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.247+0000 7f20a2a00640 1 --2- 172.21.15.77:0/2858344067 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f20a4101df0 0x7f20a40744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:57:19.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.248+0000 7f20a3400640 1 --2- 172.21.15.77:0/2858344067 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f20a4106f20 0x7f20a4074f70 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f209c00ec50 tx=0x7f209c00c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:19.252 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.248+0000 7f209b400640 1 -- 172.21.15.77:0/2858344067 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f209c003b70 con 0x7f20a4106f20 2024-09-10T09:57:19.252 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.248+0000 7f209b400640 1 -- 172.21.15.77:0/2858344067 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f209c003d10 con 0x7f20a4106f20 2024-09-10T09:57:19.252 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.248+0000 7f209b400640 1 -- 172.21.15.77:0/2858344067 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f209c010890 con 0x7f20a4106f20 2024-09-10T09:57:19.252 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.248+0000 7f20a9600640 1 -- 172.21.15.77:0/2858344067 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20a41a6c60 con 0x7f20a4106f20 2024-09-10T09:57:19.252 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.249+0000 7f20a9600640 1 -- 172.21.15.77:0/2858344067 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20a41a7150 con 0x7f20a4106f20 2024-09-10T09:57:19.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.252+0000 7f20ac1e9740 1 Processor -- start 2024-09-10T09:57:19.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.253+0000 7f209b400640 1 -- 172.21.15.77:0/2858344067 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f209c003fe0 con 0x7f20a4106f20 2024-09-10T09:57:19.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.253+0000 7f20ac1e9740 1 -- start start 2024-09-10T09:57:19.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.253+0000 7f20ac1e9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56032e6db310 0x56032e6db6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:19.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.254+0000 7f20ac1e9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56032e6dbc30 0x56032e6e5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:19.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.254+0000 7f20ac1e9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56032e6e64f0 0x56032e6e88e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:19.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.254+0000 7f20ac1e9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56032e641650 con 0x56032e6db310 2024-09-10T09:57:19.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.254+0000 7f20ac1e9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56032e5e8f10 con 0x56032e6e64f0 2024-09-10T09:57:19.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.254+0000 7f20ac1e9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56032e63d610 con 0x56032e6dbc30 2024-09-10T09:57:19.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.254+0000 7f20a2000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56032e6db310 0x56032e6db6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:19.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.254+0000 7f20a2a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56032e6dbc30 0x56032e6e5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:19.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.254+0000 7f20a2a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56032e6dbc30 0x56032e6e5fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:56476/0 (socket says 172.21.15.77:56476) 2024-09-10T09:57:19.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.254+0000 7f20a2a00640 1 -- 172.21.15.77:0/1069889297 learned_addr learned my addr 172.21.15.77:0/1069889297 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:57:19.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.254+0000 7f20a2000640 1 --2- 172.21.15.77:0/1069889297 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56032e6e64f0 0x56032e6e88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:19.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.254+0000 7f20a2a00640 1 -- 172.21.15.77:0/1069889297 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56032e6e64f0 msgr2=0x56032e6e88e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:19.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.254+0000 7f20a2a00640 1 --2- 172.21.15.77:0/1069889297 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56032e6e64f0 0x56032e6e88e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:19.258 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.254+0000 7f20a2a00640 1 -- 172.21.15.77:0/1069889297 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56032e6db310 msgr2=0x56032e6db6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:19.258 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.254+0000 7f20a2a00640 1 --2- 172.21.15.77:0/1069889297 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56032e6db310 0x56032e6db6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:19.258 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.254+0000 7f20a2a00640 1 -- 172.21.15.77:0/1069889297 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56032e350580 con 0x56032e6dbc30 2024-09-10T09:57:19.258 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.255+0000 7f20a2a00640 1 --2- 172.21.15.77:0/1069889297 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56032e6dbc30 0x56032e6e5fb0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2094009900 tx=0x7f2094005de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=afcf57796bb53726 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:19.258 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.255+0000 7f209b400640 1 --2- 172.21.15.77:0/2858344067 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f20880a2c00 0x7f20880a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:19.258 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.255+0000 7f209b400640 1 -- 172.21.15.77:0/2858344067 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f209c0cf920 con 0x7f20a4106f20 2024-09-10T09:57:19.258 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.255+0000 7f208f400640 1 -- 172.21.15.77:0/1069889297 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2094035040 con 0x56032e6dbc30 2024-09-10T09:57:19.258 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.255+0000 7f20a2000640 1 --2- 172.21.15.77:0/2858344067 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f20880a2c00 0x7f20880a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:19.258 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.255+0000 7f208f400640 1 -- 172.21.15.77:0/1069889297 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2094004760 con 0x56032e6dbc30 2024-09-10T09:57:19.258 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.255+0000 7f208f400640 1 -- 172.21.15.77:0/1069889297 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20940038e0 con 0x56032e6dbc30 2024-09-10T09:57:19.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.256+0000 7f20ac1e9740 1 -- 172.21.15.77:0/1069889297 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56032e6dbc30 msgr2=0x56032e6e5fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:19.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.256+0000 7f20a2000640 1 --2- 172.21.15.77:0/2858344067 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f20880a2c00 0x7f20880a50c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f2090009810 tx=0x7f2090009880 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:19.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.256+0000 7f20ac1e9740 1 --2- 172.21.15.77:0/1069889297 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56032e6dbc30 0x56032e6e5fb0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2094009900 tx=0x7f2094005de0 comp rx=0 tx=0).stop 2024-09-10T09:57:19.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.256+0000 7f20ac1e9740 1 -- 172.21.15.77:0/1069889297 shutdown_connections 2024-09-10T09:57:19.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.256+0000 7f20ac1e9740 1 --2- 172.21.15.77:0/1069889297 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56032e6e64f0 0x56032e6e88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:19.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.256+0000 7f20ac1e9740 1 --2- 172.21.15.77:0/1069889297 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56032e6dbc30 0x56032e6e5fb0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:19.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.256+0000 7f20ac1e9740 1 --2- 172.21.15.77:0/1069889297 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56032e6db310 0x56032e6db6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:19.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.256+0000 7f20ac1e9740 1 -- 172.21.15.77:0/1069889297 >> 172.21.15.77:0/1069889297 conn(0x56032e6d78a0 msgr2=0x56032e6d7c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:19.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.257+0000 7f20ac1e9740 1 -- 172.21.15.77:0/1069889297 shutdown_connections 2024-09-10T09:57:19.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.257+0000 7f20ac1e9740 1 -- 172.21.15.77:0/1069889297 wait complete. 2024-09-10T09:57:19.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.257+0000 7f20ac1e9740 1 -- start start 2024-09-10T09:57:19.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.257+0000 7f20ac1e9740 1 Processor -- start 2024-09-10T09:57:19.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.258+0000 7f20ac1e9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56032e6db310 0x56032e6e3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:19.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.258+0000 7f20ac1e9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56032e6dbc30 0x56032e6e38c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:19.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.258+0000 7f20ac1e9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56032e6e64f0 0x56032e6e3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:19.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.258+0000 7f20ac1e9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56032e6e90c0 con 0x56032e6e64f0 2024-09-10T09:57:19.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.258+0000 7f20ac1e9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56032e5fc350 con 0x56032e6dbc30 2024-09-10T09:57:19.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.258+0000 7f20ac1e9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56032e6e9240 con 0x56032e6db310 2024-09-10T09:57:19.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.258+0000 7f20ac1e9740 10 client.0 _ll_register_callbacks cb 0x56032e61f290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:57:19.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.258+0000 7f20a3400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56032e6e64f0 0x56032e6e3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:19.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.258+0000 7f20a2a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56032e6db310 0x56032e6e3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:19.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.258+0000 7f20a2000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56032e6dbc30 0x56032e6e38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:19.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.258+0000 7f20a2a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56032e6db310 0x56032e6e3380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:56486/0 (socket says 172.21.15.77:56486) 2024-09-10T09:57:19.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.258+0000 7f20a2a00640 1 -- 172.21.15.77:0/894747725 learned_addr learned my addr 172.21.15.77:0/894747725 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:57:19.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.259+0000 7f20a2a00640 1 -- 172.21.15.77:0/894747725 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56032e6dbc30 msgr2=0x56032e6e38c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:19.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.259+0000 7f20a2a00640 1 --2- 172.21.15.77:0/894747725 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56032e6dbc30 0x56032e6e38c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:19.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.259+0000 7f20a2a00640 1 -- 172.21.15.77:0/894747725 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56032e6e64f0 msgr2=0x56032e6e3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:19.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.259+0000 7f20a2a00640 1 --2- 172.21.15.77:0/894747725 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56032e6e64f0 0x56032e6e3e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:19.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.259+0000 7f20a2a00640 1 -- 172.21.15.77:0/894747725 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56032e41a960 con 0x56032e6db310 2024-09-10T09:57:19.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.259+0000 7f20a2000640 1 --2- 172.21.15.77:0/894747725 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56032e6dbc30 0x56032e6e38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:57:19.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.259+0000 7f20a3400640 1 --2- 172.21.15.77:0/894747725 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56032e6e64f0 0x56032e6e3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:57:19.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.259+0000 7f20a2a00640 1 --2- 172.21.15.77:0/894747725 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56032e6db310 0x56032e6e3380 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2094004b40 tx=0x7f2094004b70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7ecfe60b0b09904 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:19.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.259+0000 7f2083e00640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-10T09:57:19.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.259+0000 7f2083e00640 1 -- 172.21.15.77:0/894747725 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2094052060 con 0x56032e6db310 2024-09-10T09:57:19.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.259+0000 7f20ac1e9740 1 -- 172.21.15.77:0/894747725 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2094004760 con 0x56032e6db310 2024-09-10T09:57:19.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.259+0000 7f2083e00640 1 -- 172.21.15.77:0/894747725 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f2094005130 con 0x56032e6db310 2024-09-10T09:57:19.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.260+0000 7f2083e00640 1 -- 172.21.15.77:0/894747725 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2094059050 con 0x56032e6db310 2024-09-10T09:57:19.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.260+0000 7f20ac1e9740 10 client.25900 fetch_fsmap learned FSMap version 29 2024-09-10T09:57:19.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.260+0000 7f20ac1e9740 1 -- 172.21.15.77:0/894747725 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56032e6e90c0 con 0x56032e6db310 2024-09-10T09:57:19.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.260+0000 7f2083e00640 1 -- 172.21.15.77:0/894747725 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2094004760 con 0x56032e6db310 2024-09-10T09:57:19.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.260+0000 7f2083e00640 1 -- 172.21.15.77:0/894747725 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f20940315b0 con 0x56032e6db310 2024-09-10T09:57:19.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.261+0000 7f20ac1e9740 10 client.25900 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T09:57:19.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.261+0000 7f20ac1e9740 10 client.25900 resolve_mds: validated gid 24563 aka daemon mds.b 2024-09-10T09:57:19.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.261+0000 7f20ac1e9740 20 client.25900 populate_metadata read hostname 'smithi077' 2024-09-10T09:57:19.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.261+0000 7f20ac1e9740 1 --2- 172.21.15.77:0/894747725 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x56032e6f5e50 0x56032e6f8240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:19.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.261+0000 7f20ac1e9740 4 client.25900 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:57:19.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.261+0000 7f20ac1e9740 1 -- 172.21.15.77:0/894747725 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56032e64ebc0 con 0x56032e6f5e50 2024-09-10T09:57:19.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.261+0000 7f20a2000640 1 --2- 172.21.15.77:0/894747725 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x56032e6f5e50 0x56032e6f8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:19.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.262+0000 7f20a2000640 1 --2- 172.21.15.77:0/894747725 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x56032e6f5e50 0x56032e6f8240 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbdaae61b4755c36 server_cookie=27edb5deaec81a12 in_seq=0 out_seq=0 2024-09-10T09:57:19.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.262+0000 7f2083e00640 10 client.25900 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T09:57:19.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.263+0000 7f2083e00640 1 -- 172.21.15.77:0/894747725 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56032e64ebc0 con 0x56032e6f5e50 2024-09-10T09:57:19.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.263+0000 7f2083e00640 10 client.25900 handle_command_reply: tid=0 2024-09-10T09:57:19.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.263+0000 7f20ac1e9740 1 client.25900 shutdown 2024-09-10T09:57:19.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.263+0000 7f20ac1e9740 1 -- 172.21.15.77:0/894747725 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56032e6db310 msgr2=0x56032e6e3380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:19.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.263+0000 7f20ac1e9740 1 --2- 172.21.15.77:0/894747725 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56032e6db310 0x56032e6e3380 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2094004b40 tx=0x7f2094004b70 comp rx=0 tx=0).stop 2024-09-10T09:57:19.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.263+0000 7f20ac1e9740 1 -- 172.21.15.77:0/894747725 shutdown_connections 2024-09-10T09:57:19.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.263+0000 7f20ac1e9740 1 --2- 172.21.15.77:0/894747725 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x56032e6f5e50 0x56032e6f8240 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:19.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.263+0000 7f20ac1e9740 1 --2- 172.21.15.77:0/894747725 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56032e6e64f0 0x56032e6e3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:19.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.263+0000 7f20ac1e9740 1 --2- 172.21.15.77:0/894747725 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56032e6dbc30 0x56032e6e38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:19.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.263+0000 7f2083e00640 0 client.25900 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T09:57:19.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.263+0000 7f20ac1e9740 1 --2- 172.21.15.77:0/894747725 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56032e6db310 0x56032e6e3380 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:19.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.263+0000 7f20ac1e9740 1 -- 172.21.15.77:0/894747725 >> 172.21.15.77:0/894747725 conn(0x56032e6d78a0 msgr2=0x56032e6dd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:19.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.263+0000 7f20ac1e9740 1 -- 172.21.15.77:0/894747725 shutdown_connections 2024-09-10T09:57:19.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.263+0000 7f20ac1e9740 1 -- 172.21.15.77:0/894747725 wait complete. 2024-09-10T09:57:19.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.263+0000 7f20ac1e9740 20 client.25900 trim_cache size 0 max 16384 2024-09-10T09:57:19.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.281+0000 7f20ac1e9740 1 Processor -- start 2024-09-10T09:57:19.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.281+0000 7f20ac1e9740 1 -- start start 2024-09-10T09:57:19.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.281+0000 7f20ac1e9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56032e6db310 0x56032e6e22c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:19.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.281+0000 7f20ac1e9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56032e6dbc30 0x56032e6ddd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:19.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.281+0000 7f20ac1e9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56032e6e64f0 0x56032e6fd900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:19.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.281+0000 7f20ac1e9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f20a41b1000 con 0x56032e6db310 2024-09-10T09:57:19.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.281+0000 7f20ac1e9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f20a41b2b40 con 0x56032e6e64f0 2024-09-10T09:57:19.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.281+0000 7f20ac1e9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f20a41b16d0 con 0x56032e6dbc30 2024-09-10T09:57:19.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.282+0000 7f20a2a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56032e6db310 0x56032e6e22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:19.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.282+0000 7f20a2a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56032e6db310 0x56032e6e22c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34838/0 (socket says 172.21.15.77:34838) 2024-09-10T09:57:19.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.282+0000 7f20a2a00640 1 -- 172.21.15.77:0/3742335437 learned_addr learned my addr 172.21.15.77:0/3742335437 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:57:19.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.282+0000 7f20a2000640 1 --2- 172.21.15.77:0/3742335437 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56032e6dbc30 0x56032e6ddd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:19.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.282+0000 7f20a3400640 1 --2- 172.21.15.77:0/3742335437 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56032e6e64f0 0x56032e6fd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:19.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.282+0000 7f20a2a00640 1 -- 172.21.15.77:0/3742335437 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56032e6dbc30 msgr2=0x56032e6ddd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:19.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.282+0000 7f20a2a00640 1 --2- 172.21.15.77:0/3742335437 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56032e6dbc30 0x56032e6ddd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:19.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.282+0000 7f20a2a00640 1 -- 172.21.15.77:0/3742335437 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56032e6e64f0 msgr2=0x56032e6fd900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:19.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.282+0000 7f20a2a00640 1 --2- 172.21.15.77:0/3742335437 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56032e6e64f0 0x56032e6fd900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:19.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.282+0000 7f20a2a00640 1 -- 172.21.15.77:0/3742335437 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56032e6d76d0 con 0x56032e6db310 2024-09-10T09:57:19.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.282+0000 7f20a2000640 1 --2- 172.21.15.77:0/3742335437 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56032e6dbc30 0x56032e6ddd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:57:19.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.282+0000 7f20a2a00640 1 --2- 172.21.15.77:0/3742335437 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56032e6db310 0x56032e6e22c0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f2094004e00 tx=0x7f2094031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f750382d89f5e60 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:19.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.282+0000 7f208ea00640 1 -- 172.21.15.77:0/3742335437 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2094052070 con 0x56032e6db310 2024-09-10T09:57:19.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.283+0000 7f208ea00640 1 -- 172.21.15.77:0/3742335437 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2094004760 con 0x56032e6db310 2024-09-10T09:57:19.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.283+0000 7f208ea00640 1 -- 172.21.15.77:0/3742335437 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2094035070 con 0x56032e6db310 2024-09-10T09:57:19.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.283+0000 7f20ac1e9740 1 -- 172.21.15.77:0/3742335437 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56032e6db310 msgr2=0x56032e6e22c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:19.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.283+0000 7f20ac1e9740 1 --2- 172.21.15.77:0/3742335437 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56032e6db310 0x56032e6e22c0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f2094004e00 tx=0x7f2094031400 comp rx=0 tx=0).stop 2024-09-10T09:57:19.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.283+0000 7f20ac1e9740 1 -- 172.21.15.77:0/3742335437 shutdown_connections 2024-09-10T09:57:19.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.283+0000 7f20ac1e9740 1 --2- 172.21.15.77:0/3742335437 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56032e6e64f0 0x56032e6fd900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:19.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.283+0000 7f20ac1e9740 1 --2- 172.21.15.77:0/3742335437 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56032e6dbc30 0x56032e6ddd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:19.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.283+0000 7f20ac1e9740 1 --2- 172.21.15.77:0/3742335437 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56032e6db310 0x56032e6e22c0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:19.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.283+0000 7f20ac1e9740 1 -- 172.21.15.77:0/3742335437 >> 172.21.15.77:0/3742335437 conn(0x56032e6d78a0 msgr2=0x56032e6e52f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:19.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.283+0000 7f20ac1e9740 1 -- 172.21.15.77:0/3742335437 shutdown_connections 2024-09-10T09:57:19.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.283+0000 7f20ac1e9740 1 -- 172.21.15.77:0/3742335437 wait complete. 2024-09-10T09:57:19.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.284+0000 7f20ac1e9740 1 -- start start 2024-09-10T09:57:19.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.284+0000 7f20ac1e9740 1 Processor -- start 2024-09-10T09:57:19.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.284+0000 7f20ac1e9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56032e6db310 0x56032e6e9a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:19.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.284+0000 7f20ac1e9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56032e6dbc30 0x56032e6e9f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:19.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.284+0000 7f20ac1e9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56032e6e64f0 0x56032e75e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:19.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.284+0000 7f20ac1e9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f20a41b2470 con 0x56032e6e64f0 2024-09-10T09:57:19.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.284+0000 7f20ac1e9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f20a41b1da0 con 0x56032e6dbc30 2024-09-10T09:57:19.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.284+0000 7f20ac1e9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f20a41aefe0 con 0x56032e6db310 2024-09-10T09:57:19.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.285+0000 7f20ac1e9740 10 client.0 _ll_register_callbacks cb 0x7f2090009550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:57:19.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.285+0000 7f20a3400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56032e6e64f0 0x56032e75e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:19.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.285+0000 7f20a2000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56032e6dbc30 0x56032e6e9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:19.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.285+0000 7f20a3400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56032e6e64f0 0x56032e75e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34848/0 (socket says 172.21.15.77:34848) 2024-09-10T09:57:19.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.285+0000 7f20a3400640 1 -- 172.21.15.77:0/648960198 learned_addr learned my addr 172.21.15.77:0/648960198 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:57:19.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.285+0000 7f20a2a00640 1 --2- 172.21.15.77:0/648960198 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56032e6db310 0x56032e6e9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:19.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.285+0000 7f20a3400640 1 -- 172.21.15.77:0/648960198 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56032e6db310 msgr2=0x56032e6e9a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:19.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.285+0000 7f20a3400640 1 --2- 172.21.15.77:0/648960198 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56032e6db310 0x56032e6e9a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:19.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.285+0000 7f20a3400640 1 -- 172.21.15.77:0/648960198 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56032e6dbc30 msgr2=0x56032e6e9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:19.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.285+0000 7f20a3400640 1 --2- 172.21.15.77:0/648960198 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56032e6dbc30 0x56032e6e9f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:19.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.285+0000 7f20a3400640 1 -- 172.21.15.77:0/648960198 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56032e75ef00 con 0x56032e6e64f0 2024-09-10T09:57:19.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.286+0000 7f20a3400640 1 --2- 172.21.15.77:0/648960198 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56032e6e64f0 0x56032e75e8c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f209c00cba0 tx=0x7f209c09b690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=700edced7f5b2057 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:19.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.286+0000 7f2083e00640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-10T09:57:19.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.286+0000 7f2083e00640 1 -- 172.21.15.77:0/648960198 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f209c07ada0 con 0x56032e6e64f0 2024-09-10T09:57:19.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.286+0000 7f2083e00640 1 -- 172.21.15.77:0/648960198 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f209c004b40 con 0x56032e6e64f0 2024-09-10T09:57:19.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.286+0000 7f20ac1e9740 1 -- 172.21.15.77:0/648960198 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2094004760 con 0x56032e6e64f0 2024-09-10T09:57:19.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.286+0000 7f2083e00640 1 -- 172.21.15.77:0/648960198 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f209c083050 con 0x56032e6e64f0 2024-09-10T09:57:19.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.286+0000 7f20ac1e9740 10 client.15879 fetch_fsmap learned FSMap version 29 2024-09-10T09:57:19.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.286+0000 7f20ac1e9740 1 -- 172.21.15.77:0/648960198 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56032e760240 con 0x56032e6e64f0 2024-09-10T09:57:19.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.286+0000 7f2083e00640 1 -- 172.21.15.77:0/648960198 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2094004760 con 0x56032e6e64f0 2024-09-10T09:57:19.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.287+0000 7f2083e00640 1 -- 172.21.15.77:0/648960198 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f209c09b180 con 0x56032e6e64f0 2024-09-10T09:57:19.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.287+0000 7f20ac1e9740 10 client.15879 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T09:57:19.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.287+0000 7f20ac1e9740 10 client.15879 resolve_mds: validated gid 24563 aka daemon mds.b 2024-09-10T09:57:19.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.287+0000 7f20ac1e9740 20 client.15879 populate_metadata read hostname 'smithi077' 2024-09-10T09:57:19.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.287+0000 7f20ac1e9740 1 --2- 172.21.15.77:0/648960198 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x56032e760910 0x56032e762d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:19.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.287+0000 7f20ac1e9740 4 client.15879 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "b59524e9", "release": true, "await": true}] 2024-09-10T09:57:19.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.287+0000 7f20ac1e9740 1 -- 172.21.15.77:0/648960198 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "quiesce db", "set_id": "b59524e9", "release": true, "await": true}) -- 0x56032e6490d0 con 0x56032e760910 2024-09-10T09:57:19.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.287+0000 7f20a2a00640 1 --2- 172.21.15.77:0/648960198 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x56032e760910 0x56032e762d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:19.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.288+0000 7f20a2a00640 1 --2- 172.21.15.77:0/648960198 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x56032e760910 0x56032e762d30 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f58e2ddebcb8cc1 server_cookie=97b1cdedd5c8b7f1 in_seq=0 out_seq=0 2024-09-10T09:57:19.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.288+0000 7f2083e00640 10 client.15879 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T09:57:19.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.290+0000 7f2083e00640 1 -- 172.21.15.77:0/648960198 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x56032e6490d0 con 0x56032e760910 2024-09-10T09:57:19.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.290+0000 7f2083e00640 10 client.15879 handle_command_reply: tid=0 2024-09-10T09:57:19.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.290+0000 7f20ac1e9740 1 client.15879 shutdown 2024-09-10T09:57:19.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.291+0000 7f20ac1e9740 1 -- 172.21.15.77:0/648960198 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56032e6e64f0 msgr2=0x56032e75e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:19.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.291+0000 7f20ac1e9740 1 --2- 172.21.15.77:0/648960198 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56032e6e64f0 0x56032e75e8c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f209c00cba0 tx=0x7f209c09b690 comp rx=0 tx=0).stop 2024-09-10T09:57:19.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.291+0000 7f20ac1e9740 1 -- 172.21.15.77:0/648960198 shutdown_connections 2024-09-10T09:57:19.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.291+0000 7f20ac1e9740 1 --2- 172.21.15.77:0/648960198 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x56032e760910 0x56032e762d30 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:19.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.291+0000 7f20ac1e9740 1 --2- 172.21.15.77:0/648960198 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56032e6e64f0 0x56032e75e8c0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:19.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.291+0000 7f20ac1e9740 1 --2- 172.21.15.77:0/648960198 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56032e6dbc30 0x56032e6e9f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:19.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.291+0000 7f20ac1e9740 1 --2- 172.21.15.77:0/648960198 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56032e6db310 0x56032e6e9a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:19.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.291+0000 7f20ac1e9740 1 -- 172.21.15.77:0/648960198 >> 172.21.15.77:0/648960198 conn(0x56032e6d78a0 msgr2=0x56032e6f1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:19.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.291+0000 7f2083e00640 0 client.15879 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T09:57:19.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.291+0000 7f20ac1e9740 1 -- 172.21.15.77:0/648960198 shutdown_connections 2024-09-10T09:57:19.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.291+0000 7f20ac1e9740 1 -- 172.21.15.77:0/648960198 wait complete. 2024-09-10T09:57:19.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.291+0000 7f20ac1e9740 20 client.15879 trim_cache size 0 max 16384 2024-09-10T09:57:19.294 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-10T09:57:19.294 INFO:teuthology.orchestra.run.smithi077.stdout: "epoch": 29, 2024-09-10T09:57:19.294 INFO:teuthology.orchestra.run.smithi077.stdout: "leader": 24563, 2024-09-10T09:57:19.295 INFO:teuthology.orchestra.run.smithi077.stdout: "set_version": 8, 2024-09-10T09:57:19.295 INFO:teuthology.orchestra.run.smithi077.stdout: "sets": { 2024-09-10T09:57:19.295 INFO:teuthology.orchestra.run.smithi077.stdout: "b59524e9": { 2024-09-10T09:57:19.295 INFO:teuthology.orchestra.run.smithi077.stdout: "version": 8, 2024-09-10T09:57:19.295 INFO:teuthology.orchestra.run.smithi077.stdout: "age_ref": 0.0, 2024-09-10T09:57:19.295 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-10T09:57:19.295 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RELEASED", 2024-09-10T09:57:19.295 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-10T09:57:19.295 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T09:57:19.295 INFO:teuthology.orchestra.run.smithi077.stdout: "timeout": 90.0, 2024-09-10T09:57:19.295 INFO:teuthology.orchestra.run.smithi077.stdout: "expiration": 150.0, 2024-09-10T09:57:19.295 INFO:teuthology.orchestra.run.smithi077.stdout: "members": { 2024-09-10T09:57:19.295 INFO:teuthology.orchestra.run.smithi077.stdout: "file:/": { 2024-09-10T09:57:19.296 INFO:teuthology.orchestra.run.smithi077.stdout: "excluded": false, 2024-09-10T09:57:19.296 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-10T09:57:19.296 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RELEASED", 2024-09-10T09:57:19.296 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-10T09:57:19.296 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T09:57:19.296 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T09:57:19.296 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T09:57:19.296 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T09:57:19.296 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T09:57:19.296 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-10T09:57:19.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.292+0000 7f20a9600640 1 -- 172.21.15.77:0/2858344067 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f20880a2c00 msgr2=0x7f20880a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:19.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.292+0000 7f20a9600640 1 --2- 172.21.15.77:0/2858344067 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f20880a2c00 0x7f20880a50c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f2090009810 tx=0x7f2090009880 comp rx=0 tx=0).stop 2024-09-10T09:57:19.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.292+0000 7f20a9600640 1 -- 172.21.15.77:0/2858344067 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f20a4106f20 msgr2=0x7f20a4074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:19.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.292+0000 7f20a9600640 1 --2- 172.21.15.77:0/2858344067 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f20a4106f20 0x7f20a4074f70 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f209c00ec50 tx=0x7f209c00c6a0 comp rx=0 tx=0).stop 2024-09-10T09:57:19.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.292+0000 7f20a9600640 1 -- 172.21.15.77:0/2858344067 shutdown_connections 2024-09-10T09:57:19.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.292+0000 7f20a9600640 1 --2- 172.21.15.77:0/2858344067 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f20880a2c00 0x7f20880a50c0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:19.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.292+0000 7f20a9600640 1 --2- 172.21.15.77:0/2858344067 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f20a4106f20 0x7f20a4074f70 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:19.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.292+0000 7f20a9600640 1 --2- 172.21.15.77:0/2858344067 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f20a41027a0 0x7f20a4074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:19.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.292+0000 7f20a9600640 1 --2- 172.21.15.77:0/2858344067 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f20a4101df0 0x7f20a40744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:19.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.292+0000 7f20a9600640 1 -- 172.21.15.77:0/2858344067 >> 172.21.15.77:0/2858344067 conn(0x7f20a40fd8b0 msgr2=0x7f20a40feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:19.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.292+0000 7f20a9600640 1 -- 172.21.15.77:0/2858344067 shutdown_connections 2024-09-10T09:57:19.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:19.292+0000 7f20a9600640 1 -- 172.21.15.77:0/2858344067 wait complete. 2024-09-10T09:57:19.304 INFO:tasks.workunit.client.0.smithi077.stderr:+ set -e 2024-09-10T09:57:19.304 INFO:tasks.workunit.client.0.smithi077.stderr:+ git clone https://git.ceph.com/xfstests-dev.git 2024-09-10T09:57:19.309 INFO:tasks.workunit.client.0.smithi077.stderr:Cloning into 'xfstests-dev'... 2024-09-10T09:57:20.353 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'b59524e9', total seconds elapsed: 34.1 2024-09-10T09:57:21.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:57:20 smithi134 ceph-mon[35674]: pgmap v456: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-09-10T09:57:21.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:57:20 smithi120 ceph-mon[34770]: pgmap v456: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-09-10T09:57:21.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:57:20 smithi077 ceph-mon[30319]: pgmap v456: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-09-10T09:57:23.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:57:22 smithi134 ceph-mon[35674]: pgmap v457: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-10T09:57:23.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:57:22 smithi120 ceph-mon[34770]: pgmap v457: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-10T09:57:23.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:57:22 smithi077 ceph-mon[30319]: pgmap v457: 129 pgs: 129 active+clean; 663 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-10T09:57:25.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:57:24 smithi134 ceph-mon[35674]: pgmap v458: 129 pgs: 129 active+clean; 6.5 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 504 KiB/s wr, 14 op/s 2024-09-10T09:57:25.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:57:24 smithi120 ceph-mon[34770]: pgmap v458: 129 pgs: 129 active+clean; 6.5 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 504 KiB/s wr, 14 op/s 2024-09-10T09:57:25.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:57:24 smithi077 ceph-mon[30319]: pgmap v458: 129 pgs: 129 active+clean; 6.5 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 504 KiB/s wr, 14 op/s 2024-09-10T09:57:27.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:57:26 smithi134 ceph-mon[35674]: pgmap v459: 129 pgs: 129 active+clean; 11 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 845 KiB/s wr, 12 op/s 2024-09-10T09:57:27.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:57:26 smithi134 ceph-mon[35674]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T09:57:27.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:57:26 smithi120 ceph-mon[34770]: pgmap v459: 129 pgs: 129 active+clean; 11 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 845 KiB/s wr, 12 op/s 2024-09-10T09:57:27.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:57:26 smithi120 ceph-mon[34770]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T09:57:27.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:57:26 smithi077 ceph-mon[30319]: pgmap v459: 129 pgs: 129 active+clean; 11 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 845 KiB/s wr, 12 op/s 2024-09-10T09:57:27.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:57:26 smithi077 ceph-mon[30319]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T09:57:29.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:57:28 smithi134 ceph-mon[35674]: pgmap v460: 129 pgs: 129 active+clean; 15 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 1.2 MiB/s wr, 13 op/s 2024-09-10T09:57:29.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:57:28 smithi120 ceph-mon[34770]: pgmap v460: 129 pgs: 129 active+clean; 15 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 1.2 MiB/s wr, 13 op/s 2024-09-10T09:57:29.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:57:28 smithi077 ceph-mon[30319]: pgmap v460: 129 pgs: 129 active+clean; 15 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 1.2 MiB/s wr, 13 op/s 2024-09-10T09:57:30.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:57:29 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:57:30.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:57:29 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:57:30.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:57:29 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:57:30.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:57:29 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:57:30.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:57:29 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:57:30.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:57:29 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:57:30.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:57:29 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:57:30.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:57:29 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:57:30.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:57:29 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:57:30.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:57:29 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:57:30.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:57:29 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:57:30.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:57:29 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:57:31.104 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T09:57:31.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:57:30 smithi134 ceph-mon[35674]: pgmap v461: 129 pgs: 129 active+clean; 16 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 1.3 MiB/s wr, 22 op/s 2024-09-10T09:57:31.156 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:57:30 smithi077 ceph-mon[30319]: pgmap v461: 129 pgs: 129 active+clean; 16 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 1.3 MiB/s wr, 22 op/s 2024-09-10T09:57:31.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:57:30 smithi120 ceph-mon[34770]: pgmap v461: 129 pgs: 129 active+clean; 16 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 1.3 MiB/s wr, 22 op/s 2024-09-10T09:57:31.484 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:57:32.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.050+0000 7fefc5c00640 1 -- 172.21.15.77:0/380172412 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fefc010e1e0 msgr2=0x7fefc0110630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:32.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.050+0000 7fefc5c00640 1 --2- 172.21.15.77:0/380172412 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fefc010e1e0 0x7fefc0110630 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fefb8009ca0 tx=0x7fefb802a950 comp rx=0 tx=0).stop 2024-09-10T09:57:32.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.050+0000 7fefc5c00640 1 -- 172.21.15.77:0/380172412 shutdown_connections 2024-09-10T09:57:32.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.050+0000 7fefc5c00640 1 --2- 172.21.15.77:0/380172412 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fefc010e1e0 0x7fefc0110630 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:32.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.050+0000 7fefc5c00640 1 --2- 172.21.15.77:0/380172412 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fefc0074f40 0x7fefc010dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:32.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.050+0000 7fefc5c00640 1 --2- 172.21.15.77:0/380172412 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fefc0074620 0x7fefc0074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:32.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.050+0000 7fefc5c00640 1 -- 172.21.15.77:0/380172412 >> 172.21.15.77:0/380172412 conn(0x7fefc00fd700 msgr2=0x7fefc00ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:32.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.050+0000 7fefc5c00640 1 -- 172.21.15.77:0/380172412 shutdown_connections 2024-09-10T09:57:32.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.050+0000 7fefc5c00640 1 -- 172.21.15.77:0/380172412 wait complete. 2024-09-10T09:57:32.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.051+0000 7fefc5c00640 1 Processor -- start 2024-09-10T09:57:32.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.051+0000 7fefc5c00640 1 -- start start 2024-09-10T09:57:32.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.052+0000 7fefc5c00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fefc0074620 0x7fefc019a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:32.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.052+0000 7fefc5c00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fefc0074f40 0x7fefc019aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:32.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.052+0000 7fefc5c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fefc010e1e0 0x7fefc019edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:32.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.052+0000 7fefc5c00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fefc0113f70 con 0x7fefc0074f40 2024-09-10T09:57:32.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.052+0000 7fefc5c00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fefc0113df0 con 0x7fefc0074620 2024-09-10T09:57:32.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.052+0000 7fefc5c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fefc01140f0 con 0x7fefc010e1e0 2024-09-10T09:57:32.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.052+0000 7fefbfe00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fefc0074620 0x7fefc019a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:32.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.052+0000 7fefbfe00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fefc0074620 0x7fefc019a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:43516/0 (socket says 172.21.15.77:43516) 2024-09-10T09:57:32.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.052+0000 7fefbfe00640 1 -- 172.21.15.77:0/2224908448 learned_addr learned my addr 172.21.15.77:0/2224908448 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:57:32.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.052+0000 7fefb7e00640 1 --2- 172.21.15.77:0/2224908448 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fefc0074f40 0x7fefc019aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:32.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.052+0000 7fefc5200640 1 --2- 172.21.15.77:0/2224908448 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fefc010e1e0 0x7fefc019edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:32.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.052+0000 7fefbfe00640 1 -- 172.21.15.77:0/2224908448 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fefc010e1e0 msgr2=0x7fefc019edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:32.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.052+0000 7fefbfe00640 1 --2- 172.21.15.77:0/2224908448 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fefc010e1e0 0x7fefc019edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:32.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.052+0000 7fefbfe00640 1 -- 172.21.15.77:0/2224908448 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fefc0074f40 msgr2=0x7fefc019aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:32.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.052+0000 7fefbfe00640 1 --2- 172.21.15.77:0/2224908448 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fefc0074f40 0x7fefc019aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:32.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.052+0000 7fefbfe00640 1 -- 172.21.15.77:0/2224908448 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefc019f4d0 con 0x7fefc0074620 2024-09-10T09:57:32.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.053+0000 7fefbfe00640 1 --2- 172.21.15.77:0/2224908448 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fefc0074620 0x7fefc019a520 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fefb000ea60 tx=0x7fefb000ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:32.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.053+0000 7fefbd600640 1 -- 172.21.15.77:0/2224908448 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb000cde0 con 0x7fefc0074620 2024-09-10T09:57:32.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.053+0000 7fefbd600640 1 -- 172.21.15.77:0/2224908448 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefb00035f0 con 0x7fefc0074620 2024-09-10T09:57:32.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.053+0000 7fefbd600640 1 -- 172.21.15.77:0/2224908448 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb0010670 con 0x7fefc0074620 2024-09-10T09:57:32.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.053+0000 7fefc5c00640 1 -- 172.21.15.77:0/2224908448 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefc019f7c0 con 0x7fefc0074620 2024-09-10T09:57:32.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.053+0000 7fefc5c00640 1 -- 172.21.15.77:0/2224908448 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefc01a70a0 con 0x7fefc0074620 2024-09-10T09:57:32.058 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.056+0000 7fefc9c50740 1 Processor -- start 2024-09-10T09:57:32.058 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.056+0000 7fefc9c50740 1 -- start start 2024-09-10T09:57:32.058 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.056+0000 7fefbd600640 1 -- 172.21.15.77:0/2224908448 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fefb00037a0 con 0x7fefc0074620 2024-09-10T09:57:32.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.057+0000 7fefc9c50740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d5a915d50 0x558d5a916130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:32.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.057+0000 7fefc9c50740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d5a916670 0x558d5a9209f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:32.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.057+0000 7fefc9c50740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d5a920f30 0x558d5a923320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:32.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.057+0000 7fefc9c50740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558d5a87c3e0 con 0x558d5a915d50 2024-09-10T09:57:32.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.057+0000 7fefc9c50740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558d5a823d80 con 0x558d5a916670 2024-09-10T09:57:32.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.057+0000 7fefc9c50740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558d5a8783a0 con 0x558d5a920f30 2024-09-10T09:57:32.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.057+0000 7fefbfe00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d5a916670 0x558d5a9209f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:32.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.057+0000 7fefbfe00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d5a916670 0x558d5a9209f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:43520/0 (socket says 172.21.15.77:43520) 2024-09-10T09:57:32.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.057+0000 7fefbfe00640 1 -- 172.21.15.77:0/3548673009 learned_addr learned my addr 172.21.15.77:0/3548673009 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:57:32.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.057+0000 7fefc5200640 1 --2- 172.21.15.77:0/3548673009 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d5a915d50 0x558d5a916130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:32.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.057+0000 7fefb7e00640 1 --2- 172.21.15.77:0/3548673009 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d5a920f30 0x558d5a923320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:32.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.058+0000 7fefbfe00640 1 -- 172.21.15.77:0/3548673009 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d5a920f30 msgr2=0x558d5a923320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:32.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.058+0000 7fefbfe00640 1 --2- 172.21.15.77:0/3548673009 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d5a920f30 0x558d5a923320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:32.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.058+0000 7fefbfe00640 1 -- 172.21.15.77:0/3548673009 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d5a915d50 msgr2=0x558d5a916130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:32.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.058+0000 7fefbfe00640 1 --2- 172.21.15.77:0/3548673009 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d5a915d50 0x558d5a916130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:32.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.058+0000 7fefbfe00640 1 -- 172.21.15.77:0/3548673009 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d5a842ae0 con 0x558d5a916670 2024-09-10T09:57:32.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.058+0000 7fefc5200640 1 --2- 172.21.15.77:0/3548673009 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d5a915d50 0x558d5a916130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:57:32.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.058+0000 7fefbfe00640 1 --2- 172.21.15.77:0/3548673009 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d5a916670 0x558d5a9209f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fefb00d5a90 tx=0x7fefb00d5f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=939dd89771788541 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:32.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.058+0000 7fefb5600640 1 -- 172.21.15.77:0/3548673009 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb00de070 con 0x558d5a916670 2024-09-10T09:57:32.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.058+0000 7fefb5600640 1 -- 172.21.15.77:0/3548673009 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefb00d10d0 con 0x558d5a916670 2024-09-10T09:57:32.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.058+0000 7fefb5600640 1 -- 172.21.15.77:0/3548673009 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb00da680 con 0x558d5a916670 2024-09-10T09:57:32.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.059+0000 7fefbd600640 1 --2- 172.21.15.77:0/2224908448 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fefa40a2c00 0x7fefa40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:32.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.059+0000 7fefbd600640 1 -- 172.21.15.77:0/2224908448 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fefb00cf6e0 con 0x7fefc0074620 2024-09-10T09:57:32.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.059+0000 7fefc5200640 1 --2- 172.21.15.77:0/2224908448 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fefa40a2c00 0x7fefa40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:32.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.059+0000 7fefc9c50740 1 -- 172.21.15.77:0/3548673009 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d5a916670 msgr2=0x558d5a9209f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:32.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.059+0000 7fefc9c50740 1 --2- 172.21.15.77:0/3548673009 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d5a916670 0x558d5a9209f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fefb00d5a90 tx=0x7fefb00d5f50 comp rx=0 tx=0).stop 2024-09-10T09:57:32.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.059+0000 7fefc9c50740 1 -- 172.21.15.77:0/3548673009 shutdown_connections 2024-09-10T09:57:32.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.059+0000 7fefc9c50740 1 --2- 172.21.15.77:0/3548673009 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d5a920f30 0x558d5a923320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:32.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.059+0000 7fefc9c50740 1 --2- 172.21.15.77:0/3548673009 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d5a916670 0x558d5a9209f0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:32.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.059+0000 7fefc9c50740 1 --2- 172.21.15.77:0/3548673009 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d5a915d50 0x558d5a916130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:32.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.059+0000 7fefc9c50740 1 -- 172.21.15.77:0/3548673009 >> 172.21.15.77:0/3548673009 conn(0x558d5a9122e0 msgr2=0x558d5a9126c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:32.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.060+0000 7fefc5200640 1 --2- 172.21.15.77:0/2224908448 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fefa40a2c00 0x7fefa40a50c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fefb8030040 tx=0x7fefb8005c50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:32.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.060+0000 7fefc9c50740 1 -- 172.21.15.77:0/3548673009 shutdown_connections 2024-09-10T09:57:32.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.060+0000 7fefc9c50740 1 -- 172.21.15.77:0/3548673009 wait complete. 2024-09-10T09:57:32.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.060+0000 7fefc9c50740 1 -- start start 2024-09-10T09:57:32.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.060+0000 7fefc9c50740 1 Processor -- start 2024-09-10T09:57:32.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.061+0000 7fefc9c50740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d5a915d50 0x558d5a91de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:32.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.061+0000 7fefc9c50740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d5a916670 0x558d5a91e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:32.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.061+0000 7fefc9c50740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d5a920f30 0x558d5a91e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:32.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.061+0000 7fefc9c50740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558d5a923ba0 con 0x558d5a915d50 2024-09-10T09:57:32.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.061+0000 7fefc9c50740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558d5a8370f0 con 0x558d5a920f30 2024-09-10T09:57:32.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.061+0000 7fefc9c50740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558d5a923d20 con 0x558d5a916670 2024-09-10T09:57:32.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.061+0000 7fefc9c50740 10 client.0 _ll_register_callbacks cb 0x558d5a8289e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:57:32.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.061+0000 7fefbfe00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d5a915d50 0x558d5a91de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:32.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.061+0000 7fefc5200640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d5a920f30 0x558d5a91e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:32.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.061+0000 7fefb7e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d5a916670 0x558d5a91e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:32.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.061+0000 7fefbfe00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d5a915d50 0x558d5a91de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46268/0 (socket says 172.21.15.77:46268) 2024-09-10T09:57:32.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.061+0000 7fefb7e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d5a916670 0x558d5a91e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:50376/0 (socket says 172.21.15.77:50376) 2024-09-10T09:57:32.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.061+0000 7fefbfe00640 1 -- 172.21.15.77:0/460796905 learned_addr learned my addr 172.21.15.77:0/460796905 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:57:32.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.062+0000 7fefb7e00640 1 -- 172.21.15.77:0/460796905 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d5a920f30 msgr2=0x558d5a91e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:32.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.062+0000 7fefb7e00640 1 --2- 172.21.15.77:0/460796905 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d5a920f30 0x558d5a91e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:32.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.062+0000 7fefb7e00640 1 -- 172.21.15.77:0/460796905 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d5a915d50 msgr2=0x558d5a91de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:32.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.062+0000 7fefb7e00640 1 --2- 172.21.15.77:0/460796905 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d5a915d50 0x558d5a91de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:32.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.062+0000 7fefb7e00640 1 -- 172.21.15.77:0/460796905 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d5a5ed3c0 con 0x558d5a916670 2024-09-10T09:57:32.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.062+0000 7fefbfe00640 1 --2- 172.21.15.77:0/460796905 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d5a915d50 0x558d5a91de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:57:32.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.062+0000 7fefb7e00640 1 --2- 172.21.15.77:0/460796905 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d5a916670 0x558d5a91e380 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fefa80126d0 tx=0x7fefa8012b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9bc0dcf736d4f15a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:32.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.062+0000 7fefae000640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-10T09:57:32.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.062+0000 7fefae000640 1 -- 172.21.15.77:0/460796905 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa8004030 con 0x558d5a916670 2024-09-10T09:57:32.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.062+0000 7fefc9c50740 1 -- 172.21.15.77:0/460796905 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fefb00d10d0 con 0x558d5a916670 2024-09-10T09:57:32.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.062+0000 7fefae000640 1 -- 172.21.15.77:0/460796905 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fefa80041d0 con 0x558d5a916670 2024-09-10T09:57:32.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.063+0000 7fefae000640 1 -- 172.21.15.77:0/460796905 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fefa8025050 con 0x558d5a916670 2024-09-10T09:57:32.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.063+0000 7fefc9c50740 10 client.25918 fetch_fsmap learned FSMap version 30 2024-09-10T09:57:32.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.063+0000 7fefc9c50740 1 -- 172.21.15.77:0/460796905 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558d5a92f960 con 0x558d5a916670 2024-09-10T09:57:32.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.063+0000 7fefae000640 1 -- 172.21.15.77:0/460796905 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefb00d10d0 con 0x558d5a916670 2024-09-10T09:57:32.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.063+0000 7fefae000640 1 -- 172.21.15.77:0/460796905 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7fefa8003a10 con 0x558d5a916670 2024-09-10T09:57:32.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.063+0000 7fefc9c50740 10 client.25918 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T09:57:32.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.063+0000 7fefc9c50740 10 client.25918 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:57:32.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.063+0000 7fefc9c50740 20 client.25918 populate_metadata read hostname 'smithi077' 2024-09-10T09:57:32.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.063+0000 7fefc9c50740 1 --2- 172.21.15.77:0/460796905 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x558d5a930dc0 0x558d5a933230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:32.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.063+0000 7fefc9c50740 4 client.25918 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:57:32.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.063+0000 7fefc9c50740 1 -- 172.21.15.77:0/460796905 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558d5a889950 con 0x558d5a930dc0 2024-09-10T09:57:32.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.064+0000 7fefbfe00640 1 --2- 172.21.15.77:0/460796905 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x558d5a930dc0 0x558d5a933230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:32.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.064+0000 7fefbfe00640 1 --2- 172.21.15.77:0/460796905 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x558d5a930dc0 0x558d5a933230 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f822792bddfc5b61 server_cookie=c6575b3188ee0cbf in_seq=0 out_seq=0 2024-09-10T09:57:32.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.064+0000 7fefae000640 10 client.25918 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T09:57:32.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.066+0000 7fefae000640 1 -- 172.21.15.77:0/460796905 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558d5a889950 con 0x558d5a930dc0 2024-09-10T09:57:32.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.066+0000 7fefae000640 10 client.25918 handle_command_reply: tid=0 2024-09-10T09:57:32.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.067+0000 7fefc9c50740 1 client.25918 shutdown 2024-09-10T09:57:32.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.067+0000 7fefc9c50740 1 -- 172.21.15.77:0/460796905 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d5a916670 msgr2=0x558d5a91e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:32.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.067+0000 7fefc9c50740 1 --2- 172.21.15.77:0/460796905 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d5a916670 0x558d5a91e380 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fefa80126d0 tx=0x7fefa8012b90 comp rx=0 tx=0).stop 2024-09-10T09:57:32.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.067+0000 7fefc9c50740 1 -- 172.21.15.77:0/460796905 shutdown_connections 2024-09-10T09:57:32.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.067+0000 7fefc9c50740 1 --2- 172.21.15.77:0/460796905 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x558d5a930dc0 0x558d5a933230 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:32.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.067+0000 7fefc9c50740 1 --2- 172.21.15.77:0/460796905 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d5a920f30 0x558d5a91e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:32.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.067+0000 7fefc9c50740 1 --2- 172.21.15.77:0/460796905 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d5a916670 0x558d5a91e380 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:32.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.067+0000 7fefae000640 0 client.25918 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T09:57:32.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.067+0000 7fefc9c50740 1 --2- 172.21.15.77:0/460796905 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d5a915d50 0x558d5a91de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:32.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.067+0000 7fefc9c50740 1 -- 172.21.15.77:0/460796905 >> 172.21.15.77:0/460796905 conn(0x558d5a9122e0 msgr2=0x558d5a918010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:32.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.067+0000 7fefc9c50740 1 -- 172.21.15.77:0/460796905 shutdown_connections 2024-09-10T09:57:32.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.067+0000 7fefc9c50740 1 -- 172.21.15.77:0/460796905 wait complete. 2024-09-10T09:57:32.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.067+0000 7fefc9c50740 20 client.25918 trim_cache size 0 max 16384 2024-09-10T09:57:32.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.090+0000 7fefc9c50740 1 Processor -- start 2024-09-10T09:57:32.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.090+0000 7fefc9c50740 1 -- start start 2024-09-10T09:57:32.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.090+0000 7fefc9c50740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d5a915d50 0x558d5a91d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:32.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.090+0000 7fefc9c50740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d5a916670 0x558d5a917eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:32.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.091+0000 7fefc9c50740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d5a920f30 0x558d5a938b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:32.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.091+0000 7fefc9c50740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fefc01b0ff0 con 0x558d5a915d50 2024-09-10T09:57:32.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.091+0000 7fefc9c50740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fefc01b2b90 con 0x558d5a920f30 2024-09-10T09:57:32.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.091+0000 7fefc9c50740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fefc01b16c0 con 0x558d5a916670 2024-09-10T09:57:32.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.091+0000 7fefbfe00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d5a915d50 0x558d5a91d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:32.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.091+0000 7fefb7e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d5a916670 0x558d5a917eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:32.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.091+0000 7fefbfe00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d5a915d50 0x558d5a91d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46282/0 (socket says 172.21.15.77:46282) 2024-09-10T09:57:32.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.091+0000 7fefbfe00640 1 -- 172.21.15.77:0/2819529847 learned_addr learned my addr 172.21.15.77:0/2819529847 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:57:32.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.091+0000 7fefc5200640 1 --2- 172.21.15.77:0/2819529847 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d5a920f30 0x558d5a938b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:32.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.091+0000 7fefb7e00640 1 -- 172.21.15.77:0/2819529847 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d5a920f30 msgr2=0x558d5a938b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:32.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.091+0000 7fefb7e00640 1 --2- 172.21.15.77:0/2819529847 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d5a920f30 0x558d5a938b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:32.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.091+0000 7fefb7e00640 1 -- 172.21.15.77:0/2819529847 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d5a915d50 msgr2=0x558d5a91d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:32.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.091+0000 7fefb7e00640 1 --2- 172.21.15.77:0/2819529847 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d5a915d50 0x558d5a91d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:32.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.091+0000 7fefb7e00640 1 -- 172.21.15.77:0/2819529847 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d5a91cdf0 con 0x558d5a916670 2024-09-10T09:57:32.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.092+0000 7fefb7e00640 1 --2- 172.21.15.77:0/2819529847 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d5a916670 0x558d5a917eb0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fefa8000c00 tx=0x7fefa800f810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ed6e9b003b55038 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:32.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.092+0000 7fefb4c00640 1 -- 172.21.15.77:0/2819529847 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa801c4b0 con 0x558d5a916670 2024-09-10T09:57:32.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.092+0000 7fefb4c00640 1 -- 172.21.15.77:0/2819529847 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefa801c650 con 0x558d5a916670 2024-09-10T09:57:32.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.092+0000 7fefb4c00640 1 -- 172.21.15.77:0/2819529847 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa8005110 con 0x558d5a916670 2024-09-10T09:57:32.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.093+0000 7fefc9c50740 1 -- 172.21.15.77:0/2819529847 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d5a916670 msgr2=0x558d5a917eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:32.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.093+0000 7fefc9c50740 1 --2- 172.21.15.77:0/2819529847 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d5a916670 0x558d5a917eb0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fefa8000c00 tx=0x7fefa800f810 comp rx=0 tx=0).stop 2024-09-10T09:57:32.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.093+0000 7fefc9c50740 1 -- 172.21.15.77:0/2819529847 shutdown_connections 2024-09-10T09:57:32.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.093+0000 7fefc9c50740 1 --2- 172.21.15.77:0/2819529847 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d5a920f30 0x558d5a938b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:32.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.093+0000 7fefc9c50740 1 --2- 172.21.15.77:0/2819529847 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d5a916670 0x558d5a917eb0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:32.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.093+0000 7fefc9c50740 1 --2- 172.21.15.77:0/2819529847 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d5a915d50 0x558d5a91d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:32.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.093+0000 7fefc9c50740 1 -- 172.21.15.77:0/2819529847 >> 172.21.15.77:0/2819529847 conn(0x558d5a9122e0 msgr2=0x558d5a9177a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:32.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.093+0000 7fefc9c50740 1 -- 172.21.15.77:0/2819529847 shutdown_connections 2024-09-10T09:57:32.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.093+0000 7fefc9c50740 1 -- 172.21.15.77:0/2819529847 wait complete. 2024-09-10T09:57:32.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.093+0000 7fefc9c50740 1 -- start start 2024-09-10T09:57:32.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.094+0000 7fefc9c50740 1 Processor -- start 2024-09-10T09:57:32.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.094+0000 7fefc9c50740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d5a915d50 0x558d5a9297f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:32.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.094+0000 7fefc9c50740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d5a916670 0x558d5a924b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:32.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.094+0000 7fefc9c50740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d5a920f30 0x558d5a9250f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:32.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.094+0000 7fefc9c50740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fefc01b2460 con 0x558d5a920f30 2024-09-10T09:57:32.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.094+0000 7fefc9c50740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fefc01b1d90 con 0x558d5a916670 2024-09-10T09:57:32.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.094+0000 7fefc9c50740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fefc01aefd0 con 0x558d5a915d50 2024-09-10T09:57:32.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.094+0000 7fefc9c50740 10 client.0 _ll_register_callbacks cb 0x7fefa8009de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:57:32.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.094+0000 7fefb7e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d5a916670 0x558d5a924b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:32.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.094+0000 7fefb7e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d5a916670 0x558d5a924b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:43546/0 (socket says 172.21.15.77:43546) 2024-09-10T09:57:32.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.095+0000 7fefbfe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d5a915d50 0x558d5a9297f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:32.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.095+0000 7fefb7e00640 1 -- 172.21.15.77:0/2559178787 learned_addr learned my addr 172.21.15.77:0/2559178787 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:57:32.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.095+0000 7fefc5200640 1 --2- 172.21.15.77:0/2559178787 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d5a920f30 0x558d5a9250f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:32.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.095+0000 7fefb7e00640 1 -- 172.21.15.77:0/2559178787 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d5a915d50 msgr2=0x558d5a9297f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:32.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.095+0000 7fefb7e00640 1 --2- 172.21.15.77:0/2559178787 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d5a915d50 0x558d5a9297f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:32.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.095+0000 7fefb7e00640 1 -- 172.21.15.77:0/2559178787 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d5a920f30 msgr2=0x558d5a9250f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:32.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.095+0000 7fefb7e00640 1 --2- 172.21.15.77:0/2559178787 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d5a920f30 0x558d5a9250f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:32.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.095+0000 7fefb7e00640 1 -- 172.21.15.77:0/2559178787 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d5a9164a0 con 0x558d5a916670 2024-09-10T09:57:32.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.095+0000 7fefbfe00640 1 --2- 172.21.15.77:0/2559178787 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d5a915d50 0x558d5a9297f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:57:32.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.095+0000 7fefb7e00640 1 --2- 172.21.15.77:0/2559178787 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d5a916670 0x558d5a924b20 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fefa80092e0 tx=0x7fefa8003820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a76e0697b3544fb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:32.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.095+0000 7fefae000640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T09:57:32.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.095+0000 7fefae000640 1 -- 172.21.15.77:0/2559178787 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa8003bd0 con 0x558d5a916670 2024-09-10T09:57:32.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.095+0000 7fefc9c50740 1 -- 172.21.15.77:0/2559178787 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fefa801c650 con 0x558d5a916670 2024-09-10T09:57:32.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.095+0000 7fefae000640 1 -- 172.21.15.77:0/2559178787 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fefa800fb30 con 0x558d5a916670 2024-09-10T09:57:32.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.096+0000 7fefae000640 1 -- 172.21.15.77:0/2559178787 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fefa800fd50 con 0x558d5a916670 2024-09-10T09:57:32.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.096+0000 7fefc9c50740 10 client.25967 fetch_fsmap learned FSMap version 30 2024-09-10T09:57:32.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.096+0000 7fefc9c50740 1 -- 172.21.15.77:0/2559178787 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558d5a99ba40 con 0x558d5a916670 2024-09-10T09:57:32.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.096+0000 7fefae000640 1 -- 172.21.15.77:0/2559178787 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefa801c650 con 0x558d5a916670 2024-09-10T09:57:32.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.096+0000 7fefae000640 1 -- 172.21.15.77:0/2559178787 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7fefa8029870 con 0x558d5a916670 2024-09-10T09:57:32.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.096+0000 7fefc9c50740 10 client.25967 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T09:57:32.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.096+0000 7fefc9c50740 10 client.25967 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:57:32.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.096+0000 7fefc9c50740 20 client.25967 populate_metadata read hostname 'smithi077' 2024-09-10T09:57:32.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.096+0000 7fefc9c50740 1 --2- 172.21.15.77:0/2559178787 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x558d5a99c180 0x558d5a99e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:32.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.096+0000 7fefc9c50740 4 client.25967 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T09:57:32.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.096+0000 7fefc9c50740 1 -- 172.21.15.77:0/2559178787 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558d5a883e60 con 0x558d5a99c180 2024-09-10T09:57:32.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.096+0000 7fefbfe00640 1 --2- 172.21.15.77:0/2559178787 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x558d5a99c180 0x558d5a99e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:32.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.097+0000 7fefbfe00640 1 --2- 172.21.15.77:0/2559178787 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x558d5a99c180 0x558d5a99e5a0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98d0e4ef118ae261 server_cookie=b289055fb9607f5 in_seq=0 out_seq=0 2024-09-10T09:57:32.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.097+0000 7fefae000640 10 client.25967 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T09:57:32.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.098+0000 7fefae000640 1 -- 172.21.15.77:0/2559178787 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x558d5a883e60 con 0x558d5a99c180 2024-09-10T09:57:32.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.098+0000 7fefae000640 10 client.25967 handle_command_reply: tid=0 2024-09-10T09:57:32.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.098+0000 7fefc9c50740 1 client.25967 shutdown 2024-09-10T09:57:32.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.098+0000 7fefc9c50740 1 -- 172.21.15.77:0/2559178787 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d5a916670 msgr2=0x558d5a924b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:32.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.098+0000 7fefc9c50740 1 --2- 172.21.15.77:0/2559178787 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d5a916670 0x558d5a924b20 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fefa80092e0 tx=0x7fefa8003820 comp rx=0 tx=0).stop 2024-09-10T09:57:32.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.098+0000 7fefc9c50740 1 -- 172.21.15.77:0/2559178787 shutdown_connections 2024-09-10T09:57:32.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.098+0000 7fefc9c50740 1 --2- 172.21.15.77:0/2559178787 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x558d5a99c180 0x558d5a99e5a0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:32.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.098+0000 7fefc9c50740 1 --2- 172.21.15.77:0/2559178787 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d5a920f30 0x558d5a9250f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:32.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.098+0000 7fefae000640 0 client.25967 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T09:57:32.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.098+0000 7fefc9c50740 1 --2- 172.21.15.77:0/2559178787 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d5a916670 0x558d5a924b20 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:32.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.098+0000 7fefc9c50740 1 --2- 172.21.15.77:0/2559178787 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d5a915d50 0x558d5a9297f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:32.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.098+0000 7fefc9c50740 1 -- 172.21.15.77:0/2559178787 >> 172.21.15.77:0/2559178787 conn(0x558d5a9122e0 msgr2=0x558d5a917fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:32.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.098+0000 7fefc9c50740 1 -- 172.21.15.77:0/2559178787 shutdown_connections 2024-09-10T09:57:32.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.098+0000 7fefc9c50740 1 -- 172.21.15.77:0/2559178787 wait complete. 2024-09-10T09:57:32.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.098+0000 7fefc9c50740 20 client.25967 trim_cache size 0 max 16384 2024-09-10T09:57:32.103 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-10T09:57:32.103 INFO:teuthology.orchestra.run.smithi077.stdout: "status": "no active scrubs running", 2024-09-10T09:57:32.103 INFO:teuthology.orchestra.run.smithi077.stdout: "scrubs": {} 2024-09-10T09:57:32.103 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-10T09:57:32.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.099+0000 7fefc5c00640 1 -- 172.21.15.77:0/2224908448 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fefa40a2c00 msgr2=0x7fefa40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:32.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.099+0000 7fefc5c00640 1 --2- 172.21.15.77:0/2224908448 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fefa40a2c00 0x7fefa40a50c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fefb8030040 tx=0x7fefb8005c50 comp rx=0 tx=0).stop 2024-09-10T09:57:32.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.099+0000 7fefc5c00640 1 -- 172.21.15.77:0/2224908448 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fefc0074620 msgr2=0x7fefc019a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:32.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.099+0000 7fefc5c00640 1 --2- 172.21.15.77:0/2224908448 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fefc0074620 0x7fefc019a520 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fefb000ea60 tx=0x7fefb000ef20 comp rx=0 tx=0).stop 2024-09-10T09:57:32.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.099+0000 7fefc5c00640 1 -- 172.21.15.77:0/2224908448 shutdown_connections 2024-09-10T09:57:32.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.099+0000 7fefc5c00640 1 --2- 172.21.15.77:0/2224908448 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fefa40a2c00 0x7fefa40a50c0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:32.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.099+0000 7fefc5c00640 1 --2- 172.21.15.77:0/2224908448 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fefc010e1e0 0x7fefc019edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:32.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.099+0000 7fefc5c00640 1 --2- 172.21.15.77:0/2224908448 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fefc0074f40 0x7fefc019aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:32.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.099+0000 7fefc5c00640 1 --2- 172.21.15.77:0/2224908448 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fefc0074620 0x7fefc019a520 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:32.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.099+0000 7fefc5c00640 1 -- 172.21.15.77:0/2224908448 >> 172.21.15.77:0/2224908448 conn(0x7fefc00fd700 msgr2=0x7fefc00fef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:32.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.100+0000 7fefc5c00640 1 -- 172.21.15.77:0/2224908448 shutdown_connections 2024-09-10T09:57:32.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:32.100+0000 7fefc5c00640 1 -- 172.21.15.77:0/2224908448 wait complete. 2024-09-10T09:57:33.101 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T09:57:33.102 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:57:33.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:57:32 smithi134 ceph-mon[35674]: pgmap v462: 129 pgs: 129 active+clean; 16 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 1.3 MiB/s wr, 19 op/s 2024-09-10T09:57:33.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:57:32 smithi120 ceph-mon[34770]: pgmap v462: 129 pgs: 129 active+clean; 16 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 1.3 MiB/s wr, 19 op/s 2024-09-10T09:57:33.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:57:32 smithi077 ceph-mon[30319]: pgmap v462: 129 pgs: 129 active+clean; 16 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 1.3 MiB/s wr, 19 op/s 2024-09-10T09:57:33.508 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:57:34.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:34.083+0000 7f886da00640 1 -- 172.21.15.77:0/839383057 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8868076720 msgr2=0x7f8868076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:34.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:34.083+0000 7f886da00640 1 --2- 172.21.15.77:0/839383057 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8868076720 0x7f8868076b00 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f8858009900 tx=0x7f885802a410 comp rx=0 tx=0).stop 2024-09-10T09:57:34.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:34.083+0000 7f886da00640 1 -- 172.21.15.77:0/839383057 shutdown_connections 2024-09-10T09:57:34.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:34.083+0000 7f886da00640 1 --2- 172.21.15.77:0/839383057 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8868112a50 0x7f8868114e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:34.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:34.083+0000 7f886da00640 1 --2- 172.21.15.77:0/839383057 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8868077040 0x7f8868112510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:34.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:34.083+0000 7f886da00640 1 --2- 172.21.15.77:0/839383057 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8868076720 0x7f8868076b00 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:34.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:34.083+0000 7f886da00640 1 -- 172.21.15.77:0/839383057 >> 172.21.15.77:0/839383057 conn(0x7f88680ffa50 msgr2=0x7f8868101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:34.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:34.083+0000 7f886da00640 1 -- 172.21.15.77:0/839383057 shutdown_connections 2024-09-10T09:57:34.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:34.083+0000 7f886da00640 1 -- 172.21.15.77:0/839383057 wait complete. 2024-09-10T09:57:34.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:34.084+0000 7f886da00640 1 Processor -- start 2024-09-10T09:57:34.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:34.084+0000 7f886da00640 1 -- start start 2024-09-10T09:57:34.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:34.085+0000 7f886da00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8868076720 0x7f88681a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:34.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:34.085+0000 7f886da00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8868077040 0x7f88681a4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:34.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:34.085+0000 7f886da00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8868112a50 0x7f886819e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:34.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:34.085+0000 7f886da00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f88681185c0 con 0x7f8868112a50 2024-09-10T09:57:34.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:34.085+0000 7f886da00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8868118440 con 0x7f8868076720 2024-09-10T09:57:34.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:34.085+0000 7f886da00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8868118740 con 0x7f8868077040 2024-09-10T09:57:34.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:34.085+0000 7f8867400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8868112a50 0x7f886819e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:34.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:34.085+0000 7f8867400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8868112a50 0x7f886819e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:36360/0 (socket says 172.21.15.77:36360) 2024-09-10T09:57:34.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:34.085+0000 7f8867400640 1 -- 172.21.15.77:0/3545629580 learned_addr learned my addr 172.21.15.77:0/3545629580 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:57:34.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:34.085+0000 7f8866000640 1 --2- 172.21.15.77:0/3545629580 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8868077040 0x7f88681a4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:34.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:34.085+0000 7f8866a00640 1 --2- 172.21.15.77:0/3545629580 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8868076720 0x7f88681a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:34.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:34.085+0000 7f8867400640 1 -- 172.21.15.77:0/3545629580 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8868077040 msgr2=0x7f88681a4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:34.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:34.085+0000 7f8867400640 1 --2- 172.21.15.77:0/3545629580 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8868077040 0x7f88681a4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:34.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:34.085+0000 7f8867400640 1 -- 172.21.15.77:0/3545629580 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8868076720 msgr2=0x7f88681a46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:34.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:34.085+0000 7f8867400640 1 --2- 172.21.15.77:0/3545629580 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8868076720 0x7f88681a46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:34.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:34.085+0000 7f8867400640 1 -- 172.21.15.77:0/3545629580 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f886819efe0 con 0x7f8868112a50 2024-09-10T09:57:34.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:34.086+0000 7f8867400640 1 --2- 172.21.15.77:0/3545629580 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8868112a50 0x7f886819e8b0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f886000eb10 tx=0x7f886000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:34.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:34.086+0000 7f885f400640 1 -- 172.21.15.77:0/3545629580 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8860004b20 con 0x7f8868112a50 2024-09-10T09:57:34.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:34.086+0000 7f885f400640 1 -- 172.21.15.77:0/3545629580 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8860004cc0 con 0x7f8868112a50 2024-09-10T09:57:34.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:34.086+0000 7f886da00640 1 -- 172.21.15.77:0/3545629580 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f886819f2d0 con 0x7f8868112a50 2024-09-10T09:57:34.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:34.086+0000 7f885f400640 1 -- 172.21.15.77:0/3545629580 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8860010790 con 0x7f8868112a50 2024-09-10T09:57:34.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:34.086+0000 7f886da00640 1 -- 172.21.15.77:0/3545629580 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88681ab5c0 con 0x7f8868112a50 2024-09-10T09:57:34.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:34.087+0000 7f886da00640 1 -- 172.21.15.77:0/3545629580 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8868076b00 con 0x7f8868112a50 2024-09-10T09:57:34.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:34.089+0000 7f885f400640 1 -- 172.21.15.77:0/3545629580 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8860005020 con 0x7f8868112a50 2024-09-10T09:57:34.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:34.090+0000 7f885f400640 1 --2- 172.21.15.77:0/3545629580 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f884c0a2c00 0x7f884c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:34.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:34.091+0000 7f885f400640 1 -- 172.21.15.77:0/3545629580 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f88600cfb10 con 0x7f8868112a50 2024-09-10T09:57:34.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:34.091+0000 7f8866a00640 1 --2- 172.21.15.77:0/3545629580 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f884c0a2c00 0x7f884c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:34.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:34.091+0000 7f8866a00640 1 --2- 172.21.15.77:0/3545629580 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f884c0a2c00 0x7f884c0a50c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f886819f9f0 tx=0x7f88580058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:34.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:34.095+0000 7f885f400640 1 -- 172.21.15.77:0/3545629580 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f886009c620 con 0x7f8868112a50 2024-09-10T09:57:34.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:34.303+0000 7f886da00640 1 -- 172.21.15.77:0/3545629580 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f88681ab910 con 0x7f8868112a50 2024-09-10T09:57:34.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:34.304+0000 7f885f400640 1 -- 172.21.15.77:0/3545629580 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11992 (secure 0 0 0) 0x7f886009bf20 con 0x7f8868112a50 2024-09-10T09:57:34.306 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:57:34.306 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":30,"btime":"2024-09-10T09:57:25:857100+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3159289675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3159289675},{"type":"v1","addr":"172.21.15.77:6835","nonce":3159289675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1951962341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1951962341},{"type":"v1","addr":"172.21.15.77:6837","nonce":1951962341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:52:23.368252+0000","modified":"2024-09-10T09:57:25.694561+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24563,"mds_1":24524,"mds_2":24520,"mds_3":24502,"mds_4":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14487":{"gid":14487,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/4291121888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":4291121888},{"type":"v1","addr":"172.21.15.77:6839","nonce":4291121888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14499":{"gid":14499,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6841/445284194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":445284194},{"type":"v1","addr":"172.21.15.77:6841","nonce":445284194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1005054423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1005054423},{"type":"v1","addr":"172.21.15.134:6833","nonce":1005054423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6833/3716307744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":3716307744},{"type":"v1","addr":"172.21.15.120:6833","nonce":3716307744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/1319892544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1319892544},{"type":"v1","addr":"172.21.15.120:6835","nonce":1319892544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.134:6835/3293933699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3293933699},{"type":"v1","addr":"172.21.15.134:6835","nonce":3293933699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.134:6837/2461497091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2461497091},{"type":"v1","addr":"172.21.15.134:6837","nonce":2461497091}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":20,"addr":"172.21.15.134:6839/1965117680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1965117680},{"type":"v1","addr":"172.21.15.134:6839","nonce":1965117680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":20,"addr":"172.21.15.120:6837/3033280709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3033280709},{"type":"v1","addr":"172.21.15.120:6837","nonce":3033280709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.120:6839/1777710157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1777710157},{"type":"v1","addr":"172.21.15.120:6839","nonce":1777710157}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24563,"qdb_cluster":[24563,24524,24520,24502,24496]},"id":1}]} 2024-09-10T09:57:34.306 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 30 2024-09-10T09:57:34.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:34.306+0000 7f886da00640 1 -- 172.21.15.77:0/3545629580 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f884c0a2c00 msgr2=0x7f884c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:34.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:34.306+0000 7f886da00640 1 --2- 172.21.15.77:0/3545629580 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f884c0a2c00 0x7f884c0a50c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f886819f9f0 tx=0x7f88580058d0 comp rx=0 tx=0).stop 2024-09-10T09:57:34.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:34.306+0000 7f886da00640 1 -- 172.21.15.77:0/3545629580 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8868112a50 msgr2=0x7f886819e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:34.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:34.306+0000 7f886da00640 1 --2- 172.21.15.77:0/3545629580 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8868112a50 0x7f886819e8b0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f886000eb10 tx=0x7f886000c2d0 comp rx=0 tx=0).stop 2024-09-10T09:57:34.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:34.306+0000 7f886da00640 1 -- 172.21.15.77:0/3545629580 shutdown_connections 2024-09-10T09:57:34.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:34.306+0000 7f886da00640 1 --2- 172.21.15.77:0/3545629580 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f884c0a2c00 0x7f884c0a50c0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:34.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:34.306+0000 7f886da00640 1 --2- 172.21.15.77:0/3545629580 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8868112a50 0x7f886819e8b0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:34.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:34.306+0000 7f886da00640 1 --2- 172.21.15.77:0/3545629580 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8868077040 0x7f88681a4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:34.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:34.306+0000 7f886da00640 1 --2- 172.21.15.77:0/3545629580 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8868076720 0x7f88681a46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:34.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:34.306+0000 7f886da00640 1 -- 172.21.15.77:0/3545629580 >> 172.21.15.77:0/3545629580 conn(0x7f88680ffa50 msgr2=0x7f8868101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:34.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:34.306+0000 7f886da00640 1 -- 172.21.15.77:0/3545629580 shutdown_connections 2024-09-10T09:57:34.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:34.306+0000 7f886da00640 1 -- 172.21.15.77:0/3545629580 wait complete. 2024-09-10T09:57:34.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:57:34 smithi077 ceph-mon[30319]: pgmap v463: 129 pgs: 129 active+clean; 20 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 1.7 MiB/s wr, 32 op/s 2024-09-10T09:57:34.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:57:34 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/3545629580' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:57:35.105 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T09:57:35.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:57:34 smithi134 ceph-mon[35674]: pgmap v463: 129 pgs: 129 active+clean; 20 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 1.7 MiB/s wr, 32 op/s 2024-09-10T09:57:35.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:57:34 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/3545629580' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:57:35.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:57:34 smithi120 ceph-mon[34770]: pgmap v463: 129 pgs: 129 active+clean; 20 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 1.7 MiB/s wr, 32 op/s 2024-09-10T09:57:35.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:57:34 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/3545629580' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:57:35.475 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:57:36.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.044+0000 7f2398e00640 1 -- 172.21.15.77:0/2778815854 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2394100720 msgr2=0x7f2394100b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:36.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.044+0000 7f2398e00640 1 --2- 172.21.15.77:0/2778815854 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2394100720 0x7f2394100b00 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f2384009900 tx=0x7f238402a410 comp rx=0 tx=0).stop 2024-09-10T09:57:36.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.044+0000 7f2398e00640 1 -- 172.21.15.77:0/2778815854 shutdown_connections 2024-09-10T09:57:36.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.044+0000 7f2398e00640 1 --2- 172.21.15.77:0/2778815854 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f239410e2d0 0x7f23941106c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:36.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.044+0000 7f2398e00640 1 --2- 172.21.15.77:0/2778815854 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f23941010d0 0x7f239410dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:36.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.044+0000 7f2398e00640 1 --2- 172.21.15.77:0/2778815854 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2394100720 0x7f2394100b00 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:36.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.044+0000 7f2398e00640 1 -- 172.21.15.77:0/2778815854 >> 172.21.15.77:0/2778815854 conn(0x7f23940fc610 msgr2=0x7f23940fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:36.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.044+0000 7f2398e00640 1 -- 172.21.15.77:0/2778815854 shutdown_connections 2024-09-10T09:57:36.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.044+0000 7f2398e00640 1 -- 172.21.15.77:0/2778815854 wait complete. 2024-09-10T09:57:36.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.045+0000 7f2398e00640 1 Processor -- start 2024-09-10T09:57:36.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.046+0000 7f2398e00640 1 -- start start 2024-09-10T09:57:36.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.046+0000 7f2398e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2394100720 0x7f239419a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:36.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.046+0000 7f2398e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f23941010d0 0x7f239419aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:36.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.046+0000 7f2398e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f239410e2d0 0x7f239419ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:36.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.046+0000 7f2398e00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2394114040 con 0x7f2394100720 2024-09-10T09:57:36.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.046+0000 7f2398e00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2394113ec0 con 0x7f239410e2d0 2024-09-10T09:57:36.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.046+0000 7f2398e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f23941141c0 con 0x7f23941010d0 2024-09-10T09:57:36.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.046+0000 7f2393400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2394100720 0x7f239419a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:36.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.046+0000 7f2392a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f23941010d0 0x7f239419aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:36.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.046+0000 7f2393e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f239410e2d0 0x7f239419ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:36.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.046+0000 7f2393400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2394100720 0x7f239419a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:36374/0 (socket says 172.21.15.77:36374) 2024-09-10T09:57:36.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.046+0000 7f2393400640 1 -- 172.21.15.77:0/844174020 learned_addr learned my addr 172.21.15.77:0/844174020 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:57:36.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.046+0000 7f2393e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f239410e2d0 0x7f239419ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:53842/0 (socket says 172.21.15.77:53842) 2024-09-10T09:57:36.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.047+0000 7f2392a00640 1 -- 172.21.15.77:0/844174020 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f239410e2d0 msgr2=0x7f239419ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:36.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.047+0000 7f2392a00640 1 --2- 172.21.15.77:0/844174020 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f239410e2d0 0x7f239419ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:36.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.047+0000 7f2392a00640 1 -- 172.21.15.77:0/844174020 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2394100720 msgr2=0x7f239419a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:36.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.047+0000 7f2392a00640 1 --2- 172.21.15.77:0/844174020 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2394100720 0x7f239419a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:36.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.047+0000 7f2392a00640 1 -- 172.21.15.77:0/844174020 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f239419f4e0 con 0x7f23941010d0 2024-09-10T09:57:36.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.047+0000 7f2392a00640 1 --2- 172.21.15.77:0/844174020 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f23941010d0 0x7f239419aa70 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f238000e9f0 tx=0x7f238000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:36.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.047+0000 7f238be00640 1 -- 172.21.15.77:0/844174020 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f238000ccc0 con 0x7f23941010d0 2024-09-10T09:57:36.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.047+0000 7f2398e00640 1 -- 172.21.15.77:0/844174020 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f239419f7d0 con 0x7f23941010d0 2024-09-10T09:57:36.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.047+0000 7f238be00640 1 -- 172.21.15.77:0/844174020 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2380005020 con 0x7f23941010d0 2024-09-10T09:57:36.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.047+0000 7f238be00640 1 -- 172.21.15.77:0/844174020 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2380004b20 con 0x7f23941010d0 2024-09-10T09:57:36.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.047+0000 7f2398e00640 1 -- 172.21.15.77:0/844174020 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23941a70b0 con 0x7f23941010d0 2024-09-10T09:57:36.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.050+0000 7f239cdda740 1 Processor -- start 2024-09-10T09:57:36.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.051+0000 7f239cdda740 1 -- start start 2024-09-10T09:57:36.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.051+0000 7f239cdda740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c584b0be40 0x55c584b0c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:36.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.051+0000 7f239cdda740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c584b0c760 0x55c584b16ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:36.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.051+0000 7f239cdda740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c584b17020 0x55c584b19410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:36.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.051+0000 7f239cdda740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c584a723e0 con 0x55c584b17020 2024-09-10T09:57:36.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.051+0000 7f239cdda740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c584a19d80 con 0x55c584b0be40 2024-09-10T09:57:36.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.051+0000 7f239cdda740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c584a6e3a0 con 0x55c584b0c760 2024-09-10T09:57:36.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.051+0000 7f2393e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c584b0be40 0x55c584b0c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:36.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.051+0000 7f2393e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c584b0be40 0x55c584b0c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:53856/0 (socket says 172.21.15.77:53856) 2024-09-10T09:57:36.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.051+0000 7f2393e00640 1 -- 172.21.15.77:0/3477183177 learned_addr learned my addr 172.21.15.77:0/3477183177 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:57:36.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.051+0000 7f238be00640 1 -- 172.21.15.77:0/844174020 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2380004cc0 con 0x7f23941010d0 2024-09-10T09:57:36.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.051+0000 7f2392a00640 1 --2- 172.21.15.77:0/3477183177 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c584b17020 0x55c584b19410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:36.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.052+0000 7f2393400640 1 --2- 172.21.15.77:0/3477183177 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c584b0c760 0x55c584b16ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:36.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.052+0000 7f2393e00640 1 -- 172.21.15.77:0/3477183177 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c584b0c760 msgr2=0x55c584b16ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:36.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.052+0000 7f2393e00640 1 --2- 172.21.15.77:0/3477183177 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c584b0c760 0x55c584b16ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:36.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.052+0000 7f2393e00640 1 -- 172.21.15.77:0/3477183177 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c584b17020 msgr2=0x55c584b19410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:36.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.052+0000 7f2393e00640 1 --2- 172.21.15.77:0/3477183177 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c584b17020 0x55c584b19410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:36.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.052+0000 7f2393e00640 1 -- 172.21.15.77:0/3477183177 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c584a38ae0 con 0x55c584b0be40 2024-09-10T09:57:36.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.052+0000 7f2392a00640 1 --2- 172.21.15.77:0/3477183177 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c584b17020 0x55c584b19410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:57:36.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.052+0000 7f2393e00640 1 --2- 172.21.15.77:0/3477183177 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c584b0be40 0x55c584b0c220 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f238c009f30 tx=0x7f238c0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6c84dcad5b8265c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:36.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.052+0000 7f2388c00640 1 -- 172.21.15.77:0/3477183177 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f238c004d20 con 0x55c584b0be40 2024-09-10T09:57:36.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.052+0000 7f2388c00640 1 -- 172.21.15.77:0/3477183177 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f238c004050 con 0x55c584b0be40 2024-09-10T09:57:36.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.052+0000 7f2388c00640 1 -- 172.21.15.77:0/3477183177 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f238c005220 con 0x55c584b0be40 2024-09-10T09:57:36.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.053+0000 7f239cdda740 1 -- 172.21.15.77:0/3477183177 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c584b0be40 msgr2=0x55c584b0c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:36.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.053+0000 7f239cdda740 1 --2- 172.21.15.77:0/3477183177 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c584b0be40 0x55c584b0c220 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f238c009f30 tx=0x7f238c0118d0 comp rx=0 tx=0).stop 2024-09-10T09:57:36.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.053+0000 7f239cdda740 1 -- 172.21.15.77:0/3477183177 shutdown_connections 2024-09-10T09:57:36.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.053+0000 7f239cdda740 1 --2- 172.21.15.77:0/3477183177 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c584b17020 0x55c584b19410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:36.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.053+0000 7f239cdda740 1 --2- 172.21.15.77:0/3477183177 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c584b0c760 0x55c584b16ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:36.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.053+0000 7f239cdda740 1 --2- 172.21.15.77:0/3477183177 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c584b0be40 0x55c584b0c220 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:36.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.053+0000 7f239cdda740 1 -- 172.21.15.77:0/3477183177 >> 172.21.15.77:0/3477183177 conn(0x55c584b083d0 msgr2=0x55c584b087b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:36.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.053+0000 7f239cdda740 1 -- 172.21.15.77:0/3477183177 shutdown_connections 2024-09-10T09:57:36.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.053+0000 7f239cdda740 1 -- 172.21.15.77:0/3477183177 wait complete. 2024-09-10T09:57:36.058 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.053+0000 7f238be00640 1 --2- 172.21.15.77:0/844174020 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f23780a2b10 0x7f23780a4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:36.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.053+0000 7f238be00640 1 -- 172.21.15.77:0/844174020 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f23800cf410 con 0x7f23941010d0 2024-09-10T09:57:36.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.054+0000 7f239cdda740 1 -- start start 2024-09-10T09:57:36.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.054+0000 7f2393400640 1 --2- 172.21.15.77:0/844174020 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f23780a2b10 0x7f23780a4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:36.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.054+0000 7f239cdda740 1 Processor -- start 2024-09-10T09:57:36.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.054+0000 7f239cdda740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c584b0be40 0x55c584b13e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:36.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.054+0000 7f239cdda740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c584b0c760 0x55c584b143a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:36.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.054+0000 7f239cdda740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c584b17020 0x55c584b148e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:36.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.054+0000 7f239cdda740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c584b19bf0 con 0x55c584b0be40 2024-09-10T09:57:36.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.055+0000 7f239cdda740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c584a2d0f0 con 0x55c584b17020 2024-09-10T09:57:36.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.055+0000 7f239cdda740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c584b19d70 con 0x55c584b0c760 2024-09-10T09:57:36.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.055+0000 7f239cdda740 10 client.0 _ll_register_callbacks cb 0x55c584a14bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:57:36.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.055+0000 7f2393400640 1 --2- 172.21.15.77:0/844174020 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f23780a2b10 0x7f23780a4fd0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x55c584b09230 tx=0x7f23840046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:36.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.055+0000 7f2393400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c584b0be40 0x55c584b13e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:36.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.055+0000 7f2393e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c584b17020 0x55c584b148e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:36.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.055+0000 7f2393e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c584b17020 0x55c584b148e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:53864/0 (socket says 172.21.15.77:53864) 2024-09-10T09:57:36.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.055+0000 7f2393e00640 1 -- 172.21.15.77:0/3695755763 learned_addr learned my addr 172.21.15.77:0/3695755763 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:57:36.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.055+0000 7f2393400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c584b0be40 0x55c584b13e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:36388/0 (socket says 172.21.15.77:36388) 2024-09-10T09:57:36.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.055+0000 7f2392a00640 1 --2- 172.21.15.77:0/3695755763 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c584b0c760 0x55c584b143a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:36.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.055+0000 7f2393e00640 1 -- 172.21.15.77:0/3695755763 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c584b0c760 msgr2=0x55c584b143a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:36.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.055+0000 7f2393e00640 1 --2- 172.21.15.77:0/3695755763 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c584b0c760 0x55c584b143a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:36.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.055+0000 7f2393e00640 1 -- 172.21.15.77:0/3695755763 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c584b0be40 msgr2=0x55c584b13e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:36.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.055+0000 7f2393e00640 1 --2- 172.21.15.77:0/3695755763 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c584b0be40 0x55c584b13e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:36.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.055+0000 7f2393e00640 1 -- 172.21.15.77:0/3695755763 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5847e33c0 con 0x55c584b17020 2024-09-10T09:57:36.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.055+0000 7f2392a00640 1 --2- 172.21.15.77:0/3695755763 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c584b0c760 0x55c584b143a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:57:36.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.056+0000 7f2393e00640 1 --2- 172.21.15.77:0/3695755763 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c584b17020 0x55c584b148e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f238c01b040 tx=0x7f238c0044b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d9eaa8a8122ee0a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:36.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.056+0000 7f237d600640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T09:57:36.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.056+0000 7f237d600640 1 -- 172.21.15.77:0/3695755763 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f238c028020 con 0x55c584b17020 2024-09-10T09:57:36.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.056+0000 7f237d600640 1 -- 172.21.15.77:0/3695755763 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f238c004c50 con 0x55c584b17020 2024-09-10T09:57:36.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.056+0000 7f239cdda740 1 -- 172.21.15.77:0/3695755763 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f238c004050 con 0x55c584b17020 2024-09-10T09:57:36.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.056+0000 7f237d600640 1 -- 172.21.15.77:0/3695755763 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f238c004e50 con 0x55c584b17020 2024-09-10T09:57:36.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.056+0000 7f239cdda740 10 client.25985 fetch_fsmap learned FSMap version 31 2024-09-10T09:57:36.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.056+0000 7f239cdda740 1 -- 172.21.15.77:0/3695755763 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c584b25b80 con 0x55c584b17020 2024-09-10T09:57:36.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.057+0000 7f237d600640 1 -- 172.21.15.77:0/3695755763 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f238c004050 con 0x55c584b17020 2024-09-10T09:57:36.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.057+0000 7f237d600640 1 -- 172.21.15.77:0/3695755763 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f238c023690 con 0x55c584b17020 2024-09-10T09:57:36.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.057+0000 7f239cdda740 10 client.25985 fetch_fsmap finished waiting for FSMap version 31 2024-09-10T09:57:36.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.057+0000 7f239cdda740 10 client.25985 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-10T09:57:36.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.057+0000 7f239cdda740 20 client.25985 populate_metadata read hostname 'smithi077' 2024-09-10T09:57:36.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.057+0000 7f239cdda740 1 --2- 172.21.15.77:0/3695755763 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x55c584b26fe0 0x55c584b293d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:36.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.057+0000 7f239cdda740 4 client.25985 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:57:36.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.057+0000 7f239cdda740 1 -- 172.21.15.77:0/3695755763 --> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c584a7f950 con 0x55c584b26fe0 2024-09-10T09:57:36.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.057+0000 7f2393400640 1 --2- 172.21.15.77:0/3695755763 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x55c584b26fe0 0x55c584b293d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:36.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.058+0000 7f2393400640 1 --2- 172.21.15.77:0/3695755763 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x55c584b26fe0 0x55c584b293d0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ba48e5d026aeec0b server_cookie=d6298f312c4d24fa in_seq=0 out_seq=0 2024-09-10T09:57:36.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.058+0000 7f237d600640 10 client.25985 ms_handle_connect on v2:172.21.15.134:6834/3293933699 2024-09-10T09:57:36.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.059+0000 7f237d600640 1 -- 172.21.15.77:0/3695755763 <== mds.4 v2:172.21.15.134:6834/3293933699 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c584a7f950 con 0x55c584b26fe0 2024-09-10T09:57:36.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.059+0000 7f237d600640 10 client.25985 handle_command_reply: tid=0 2024-09-10T09:57:36.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.059+0000 7f239cdda740 1 client.25985 shutdown 2024-09-10T09:57:36.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.059+0000 7f239cdda740 1 -- 172.21.15.77:0/3695755763 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c584b17020 msgr2=0x55c584b148e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:36.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.059+0000 7f239cdda740 1 --2- 172.21.15.77:0/3695755763 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c584b17020 0x55c584b148e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f238c01b040 tx=0x7f238c0044b0 comp rx=0 tx=0).stop 2024-09-10T09:57:36.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.059+0000 7f239cdda740 1 -- 172.21.15.77:0/3695755763 shutdown_connections 2024-09-10T09:57:36.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.059+0000 7f239cdda740 1 --2- 172.21.15.77:0/3695755763 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x55c584b26fe0 0x55c584b293d0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:36.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.059+0000 7f239cdda740 1 --2- 172.21.15.77:0/3695755763 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c584b17020 0x55c584b148e0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:36.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.059+0000 7f239cdda740 1 --2- 172.21.15.77:0/3695755763 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c584b0c760 0x55c584b143a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:36.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.059+0000 7f239cdda740 1 --2- 172.21.15.77:0/3695755763 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c584b0be40 0x55c584b13e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:36.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.059+0000 7f237d600640 0 client.25985 ms_handle_reset on v2:172.21.15.134:6834/3293933699 2024-09-10T09:57:36.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.059+0000 7f239cdda740 1 -- 172.21.15.77:0/3695755763 >> 172.21.15.77:0/3695755763 conn(0x55c584b083d0 msgr2=0x55c584b074b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:36.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.059+0000 7f239cdda740 1 -- 172.21.15.77:0/3695755763 shutdown_connections 2024-09-10T09:57:36.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.059+0000 7f239cdda740 1 -- 172.21.15.77:0/3695755763 wait complete. 2024-09-10T09:57:36.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.060+0000 7f239cdda740 20 client.25985 trim_cache size 0 max 16384 2024-09-10T09:57:36.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.081+0000 7f239cdda740 1 Processor -- start 2024-09-10T09:57:36.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.081+0000 7f239cdda740 1 -- start start 2024-09-10T09:57:36.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.081+0000 7f239cdda740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c584b0be40 0x55c584b0e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:36.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.081+0000 7f239cdda740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c584b0c760 0x55c584b13720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:36.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.081+0000 7f239cdda740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c584b17020 0x55c584b15dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:36.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.081+0000 7f239cdda740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f23941b0f10 con 0x55c584b0be40 2024-09-10T09:57:36.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.081+0000 7f239cdda740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f23941b2ae0 con 0x55c584b0c760 2024-09-10T09:57:36.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.081+0000 7f239cdda740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f23941b15e0 con 0x55c584b17020 2024-09-10T09:57:36.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.081+0000 7f2393400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c584b0be40 0x55c584b0e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:36.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.081+0000 7f2393e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c584b17020 0x55c584b15dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:36.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.081+0000 7f2392a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c584b0c760 0x55c584b13720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:36.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.081+0000 7f2393400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c584b0be40 0x55c584b0e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:36402/0 (socket says 172.21.15.77:36402) 2024-09-10T09:57:36.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.081+0000 7f2393400640 1 -- 172.21.15.77:0/3268757719 learned_addr learned my addr 172.21.15.77:0/3268757719 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:57:36.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.082+0000 7f2392a00640 1 -- 172.21.15.77:0/3268757719 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c584b17020 msgr2=0x55c584b15dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:36.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.082+0000 7f2392a00640 1 --2- 172.21.15.77:0/3268757719 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c584b17020 0x55c584b15dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:36.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.082+0000 7f2392a00640 1 -- 172.21.15.77:0/3268757719 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c584b0be40 msgr2=0x55c584b0e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:36.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.082+0000 7f2392a00640 1 --2- 172.21.15.77:0/3268757719 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c584b0be40 0x55c584b0e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:36.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.082+0000 7f2392a00640 1 -- 172.21.15.77:0/3268757719 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c584b09060 con 0x55c584b0c760 2024-09-10T09:57:36.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.082+0000 7f2392a00640 1 --2- 172.21.15.77:0/3268757719 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c584b0c760 0x55c584b13720 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x55c584b09750 tx=0x7f238009b700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ddb9b4801f4f3e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:36.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.082+0000 7f237fe00640 1 -- 172.21.15.77:0/3268757719 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f238001c040 con 0x55c584b0c760 2024-09-10T09:57:36.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.082+0000 7f237fe00640 1 -- 172.21.15.77:0/3268757719 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f238009b820 con 0x55c584b0c760 2024-09-10T09:57:36.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.082+0000 7f237fe00640 1 -- 172.21.15.77:0/3268757719 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2380099bb0 con 0x55c584b0c760 2024-09-10T09:57:36.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.083+0000 7f239cdda740 1 -- 172.21.15.77:0/3268757719 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c584b0c760 msgr2=0x55c584b13720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:36.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.083+0000 7f239cdda740 1 --2- 172.21.15.77:0/3268757719 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c584b0c760 0x55c584b13720 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x55c584b09750 tx=0x7f238009b700 comp rx=0 tx=0).stop 2024-09-10T09:57:36.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.083+0000 7f239cdda740 1 -- 172.21.15.77:0/3268757719 shutdown_connections 2024-09-10T09:57:36.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.083+0000 7f239cdda740 1 --2- 172.21.15.77:0/3268757719 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c584b17020 0x55c584b15dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:36.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.083+0000 7f239cdda740 1 --2- 172.21.15.77:0/3268757719 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c584b0c760 0x55c584b13720 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:36.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.083+0000 7f239cdda740 1 --2- 172.21.15.77:0/3268757719 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c584b0be40 0x55c584b0e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:36.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.083+0000 7f239cdda740 1 -- 172.21.15.77:0/3268757719 >> 172.21.15.77:0/3268757719 conn(0x55c584b083d0 msgr2=0x55c584b156c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:36.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.083+0000 7f239cdda740 1 -- 172.21.15.77:0/3268757719 shutdown_connections 2024-09-10T09:57:36.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.083+0000 7f239cdda740 1 -- 172.21.15.77:0/3268757719 wait complete. 2024-09-10T09:57:36.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.084+0000 7f239cdda740 1 -- start start 2024-09-10T09:57:36.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.084+0000 7f239cdda740 1 Processor -- start 2024-09-10T09:57:36.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.084+0000 7f239cdda740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c584b0be40 0x55c584b1f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:36.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.084+0000 7f239cdda740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c584b0c760 0x55c584b10840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:36.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.084+0000 7f239cdda740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c584b17020 0x55c584b10d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:36.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.084+0000 7f239cdda740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f23941b2380 con 0x55c584b17020 2024-09-10T09:57:36.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.084+0000 7f239cdda740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f23941b1cb0 con 0x55c584b0be40 2024-09-10T09:57:36.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.084+0000 7f239cdda740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f23941aeef0 con 0x55c584b0c760 2024-09-10T09:57:36.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.085+0000 7f239cdda740 10 client.0 _ll_register_callbacks cb 0x7f238c009e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:57:36.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.085+0000 7f2393400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c584b0be40 0x55c584b1f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:36.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.085+0000 7f2393400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c584b0be40 0x55c584b1f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:53888/0 (socket says 172.21.15.77:53888) 2024-09-10T09:57:36.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.085+0000 7f2393400640 1 -- 172.21.15.77:0/3049589127 learned_addr learned my addr 172.21.15.77:0/3049589127 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:57:36.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.085+0000 7f2392a00640 1 --2- 172.21.15.77:0/3049589127 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c584b0c760 0x55c584b10840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:36.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.085+0000 7f2393e00640 1 --2- 172.21.15.77:0/3049589127 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c584b17020 0x55c584b10d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:36.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.085+0000 7f2393400640 1 -- 172.21.15.77:0/3049589127 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c584b0c760 msgr2=0x55c584b10840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:36.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.085+0000 7f2393400640 1 --2- 172.21.15.77:0/3049589127 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c584b0c760 0x55c584b10840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:36.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.085+0000 7f2393400640 1 -- 172.21.15.77:0/3049589127 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c584b17020 msgr2=0x55c584b10d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:36.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.085+0000 7f2393400640 1 --2- 172.21.15.77:0/3049589127 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c584b17020 0x55c584b10d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:36.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.085+0000 7f2393400640 1 -- 172.21.15.77:0/3049589127 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c584b112c0 con 0x55c584b0be40 2024-09-10T09:57:36.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.085+0000 7f2393e00640 1 --2- 172.21.15.77:0/3049589127 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c584b17020 0x55c584b10d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:57:36.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.085+0000 7f2393400640 1 --2- 172.21.15.77:0/3049589127 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c584b0be40 0x55c584b1f3c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2384000c00 tx=0x7f2384009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad868fb0846f99b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:36.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.085+0000 7f237d600640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T09:57:36.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.086+0000 7f237d600640 1 -- 172.21.15.77:0/3049589127 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2384035040 con 0x55c584b0be40 2024-09-10T09:57:36.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.086+0000 7f239cdda740 1 -- 172.21.15.77:0/3049589127 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f238009b820 con 0x55c584b0be40 2024-09-10T09:57:36.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.086+0000 7f237d600640 1 -- 172.21.15.77:0/3049589127 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f2384031df0 con 0x55c584b0be40 2024-09-10T09:57:36.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.086+0000 7f237d600640 1 -- 172.21.15.77:0/3049589127 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f2384064050 con 0x55c584b0be40 2024-09-10T09:57:36.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.086+0000 7f239cdda740 10 client.25991 fetch_fsmap learned FSMap version 31 2024-09-10T09:57:36.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.086+0000 7f239cdda740 1 -- 172.21.15.77:0/3049589127 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c584b91740 con 0x55c584b0be40 2024-09-10T09:57:36.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.086+0000 7f237d600640 1 -- 172.21.15.77:0/3049589127 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f238009b820 con 0x55c584b0be40 2024-09-10T09:57:36.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.086+0000 7f237d600640 1 -- 172.21.15.77:0/3049589127 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f238403c670 con 0x55c584b0be40 2024-09-10T09:57:36.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.086+0000 7f239cdda740 10 client.25991 fetch_fsmap finished waiting for FSMap version 31 2024-09-10T09:57:36.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.086+0000 7f239cdda740 10 client.25991 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-10T09:57:36.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.086+0000 7f239cdda740 20 client.25991 populate_metadata read hostname 'smithi077' 2024-09-10T09:57:36.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.086+0000 7f239cdda740 1 --2- 172.21.15.77:0/3049589127 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x55c584b91ea0 0x55c584b942c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:36.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.086+0000 7f239cdda740 4 client.25991 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:57:36.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.086+0000 7f239cdda740 1 -- 172.21.15.77:0/3049589127 --> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c584a79e60 con 0x55c584b91ea0 2024-09-10T09:57:36.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.087+0000 7f2392a00640 1 --2- 172.21.15.77:0/3049589127 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x55c584b91ea0 0x55c584b942c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:36.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.087+0000 7f2392a00640 1 --2- 172.21.15.77:0/3049589127 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x55c584b91ea0 0x55c584b942c0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6c03d199ad5dd9e9 server_cookie=273aff3907a89d04 in_seq=0 out_seq=0 2024-09-10T09:57:36.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.087+0000 7f237d600640 10 client.25991 ms_handle_connect on v2:172.21.15.134:6834/3293933699 2024-09-10T09:57:36.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.087+0000 7f237d600640 1 -- 172.21.15.77:0/3049589127 <== mds.4 v2:172.21.15.134:6834/3293933699 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c584a79e60 con 0x55c584b91ea0 2024-09-10T09:57:36.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.087+0000 7f237d600640 10 client.25991 handle_command_reply: tid=0 2024-09-10T09:57:36.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.087+0000 7f239cdda740 1 client.25991 shutdown 2024-09-10T09:57:36.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.088+0000 7f239cdda740 1 -- 172.21.15.77:0/3049589127 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c584b0be40 msgr2=0x55c584b1f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:36.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.088+0000 7f239cdda740 1 --2- 172.21.15.77:0/3049589127 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c584b0be40 0x55c584b1f3c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2384000c00 tx=0x7f2384009260 comp rx=0 tx=0).stop 2024-09-10T09:57:36.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.088+0000 7f239cdda740 1 -- 172.21.15.77:0/3049589127 shutdown_connections 2024-09-10T09:57:36.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.088+0000 7f239cdda740 1 --2- 172.21.15.77:0/3049589127 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x55c584b91ea0 0x55c584b942c0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:36.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.088+0000 7f239cdda740 1 --2- 172.21.15.77:0/3049589127 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c584b17020 0x55c584b10d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:36.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.088+0000 7f239cdda740 1 --2- 172.21.15.77:0/3049589127 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c584b0c760 0x55c584b10840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:36.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.088+0000 7f239cdda740 1 --2- 172.21.15.77:0/3049589127 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c584b0be40 0x55c584b1f3c0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:36.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.088+0000 7f237d600640 0 client.25991 ms_handle_reset on v2:172.21.15.134:6834/3293933699 2024-09-10T09:57:36.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.088+0000 7f239cdda740 1 -- 172.21.15.77:0/3049589127 >> 172.21.15.77:0/3049589127 conn(0x55c584b083d0 msgr2=0x55c584b15ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:36.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.088+0000 7f239cdda740 1 -- 172.21.15.77:0/3049589127 shutdown_connections 2024-09-10T09:57:36.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.088+0000 7f239cdda740 1 -- 172.21.15.77:0/3049589127 wait complete. 2024-09-10T09:57:36.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.088+0000 7f239cdda740 20 client.25991 trim_cache size 0 max 16384 2024-09-10T09:57:36.094 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-10T09:57:36.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.089+0000 7f2398e00640 1 -- 172.21.15.77:0/844174020 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f23780a2b10 msgr2=0x7f23780a4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:36.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.089+0000 7f2398e00640 1 --2- 172.21.15.77:0/844174020 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f23780a2b10 0x7f23780a4fd0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x55c584b09230 tx=0x7f23840046c0 comp rx=0 tx=0).stop 2024-09-10T09:57:36.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.089+0000 7f2398e00640 1 -- 172.21.15.77:0/844174020 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f23941010d0 msgr2=0x7f239419aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:36.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.089+0000 7f2398e00640 1 --2- 172.21.15.77:0/844174020 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f23941010d0 0x7f239419aa70 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f238000e9f0 tx=0x7f238000eeb0 comp rx=0 tx=0).stop 2024-09-10T09:57:36.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.089+0000 7f2398e00640 1 -- 172.21.15.77:0/844174020 shutdown_connections 2024-09-10T09:57:36.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.089+0000 7f2398e00640 1 --2- 172.21.15.77:0/844174020 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f23780a2b10 0x7f23780a4fd0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:36.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.089+0000 7f2398e00640 1 --2- 172.21.15.77:0/844174020 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f239410e2d0 0x7f239419ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:36.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.089+0000 7f2398e00640 1 --2- 172.21.15.77:0/844174020 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f23941010d0 0x7f239419aa70 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:36.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.089+0000 7f2398e00640 1 --2- 172.21.15.77:0/844174020 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2394100720 0x7f239419a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:36.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.089+0000 7f2398e00640 1 -- 172.21.15.77:0/844174020 >> 172.21.15.77:0/844174020 conn(0x7f23940fc610 msgr2=0x7f23940fea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:36.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.089+0000 7f2398e00640 1 -- 172.21.15.77:0/844174020 shutdown_connections 2024-09-10T09:57:36.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:36.089+0000 7f2398e00640 1 -- 172.21.15.77:0/844174020 wait complete. 2024-09-10T09:57:36.915 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T09:57:37.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:57:36 smithi120 ceph-mon[34770]: pgmap v464: 129 pgs: 129 active+clean; 20 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 1.2 MiB/s wr, 30 op/s 2024-09-10T09:57:37.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:57:36 smithi120 ceph-mon[34770]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T09:57:37.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:57:36 smithi077 ceph-mon[30319]: pgmap v464: 129 pgs: 129 active+clean; 20 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 1.2 MiB/s wr, 30 op/s 2024-09-10T09:57:37.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:57:36 smithi077 ceph-mon[30319]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T09:57:37.304 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:57:37.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:57:36 smithi134 ceph-mon[35674]: pgmap v464: 129 pgs: 129 active+clean; 20 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 1.2 MiB/s wr, 30 op/s 2024-09-10T09:57:37.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:57:36 smithi134 ceph-mon[35674]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T09:57:37.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.887+0000 7f3888c00640 1 -- 172.21.15.77:0/1457799344 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f388410bfc0 msgr2=0x7f388410e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:37.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.887+0000 7f3888c00640 1 --2- 172.21.15.77:0/1457799344 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f388410bfc0 0x7f388410e460 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f387c009be0 tx=0x7f387c02a750 comp rx=0 tx=0).stop 2024-09-10T09:57:37.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.887+0000 7f3888c00640 1 -- 172.21.15.77:0/1457799344 shutdown_connections 2024-09-10T09:57:37.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.887+0000 7f3888c00640 1 --2- 172.21.15.77:0/1457799344 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f388410bfc0 0x7f388410e460 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:37.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.887+0000 7f3888c00640 1 --2- 172.21.15.77:0/1457799344 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f38841034b0 0x7f388410b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:37.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.887+0000 7f3888c00640 1 --2- 172.21.15.77:0/1457799344 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3884102b90 0x7f3884102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:37.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.887+0000 7f3888c00640 1 -- 172.21.15.77:0/1457799344 >> 172.21.15.77:0/1457799344 conn(0x7f38840fc610 msgr2=0x7f38840fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:37.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.888+0000 7f3888c00640 1 -- 172.21.15.77:0/1457799344 shutdown_connections 2024-09-10T09:57:37.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.888+0000 7f3888c00640 1 -- 172.21.15.77:0/1457799344 wait complete. 2024-09-10T09:57:37.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.889+0000 7f3888c00640 1 Processor -- start 2024-09-10T09:57:37.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.889+0000 7f3888c00640 1 -- start start 2024-09-10T09:57:37.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.889+0000 7f3888c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3884102b90 0x7f3884198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:37.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.889+0000 7f3888c00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f38841034b0 0x7f3884198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:37.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.889+0000 7f3888c00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f388410bfc0 0x7f388419cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:37.892 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.889+0000 7f3888c00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3884111dc0 con 0x7f388410bfc0 2024-09-10T09:57:37.892 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.889+0000 7f3888c00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3884111c40 con 0x7f38841034b0 2024-09-10T09:57:37.892 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.889+0000 7f3888c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3884111f40 con 0x7f3884102b90 2024-09-10T09:57:37.892 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.889+0000 7f3883400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3884102b90 0x7f3884198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:37.892 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.889+0000 7f387be00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f38841034b0 0x7f3884198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:37.892 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.890+0000 7f3883e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f388410bfc0 0x7f388419cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:37.892 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.890+0000 7f3883400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3884102b90 0x7f3884198250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:37446/0 (socket says 172.21.15.77:37446) 2024-09-10T09:57:37.892 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.890+0000 7f3883400640 1 -- 172.21.15.77:0/1090100140 learned_addr learned my addr 172.21.15.77:0/1090100140 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:57:37.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.890+0000 7f387be00640 1 -- 172.21.15.77:0/1090100140 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3884102b90 msgr2=0x7f3884198250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:37.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.890+0000 7f387be00640 1 --2- 172.21.15.77:0/1090100140 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3884102b90 0x7f3884198250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:37.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.890+0000 7f387be00640 1 -- 172.21.15.77:0/1090100140 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f388410bfc0 msgr2=0x7f388419cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:37.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.890+0000 7f387be00640 1 --2- 172.21.15.77:0/1090100140 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f388410bfc0 0x7f388419cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:37.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.890+0000 7f387be00640 1 -- 172.21.15.77:0/1090100140 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f388419d200 con 0x7f38841034b0 2024-09-10T09:57:37.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.890+0000 7f387be00640 1 --2- 172.21.15.77:0/1090100140 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f38841034b0 0x7f3884198790 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f386c00e6c0 tx=0x7f386c00eb80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:37.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.890+0000 7f3880c00640 1 -- 172.21.15.77:0/1090100140 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f386c00cab0 con 0x7f38841034b0 2024-09-10T09:57:37.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.891+0000 7f3880c00640 1 -- 172.21.15.77:0/1090100140 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f386c00cc50 con 0x7f38841034b0 2024-09-10T09:57:37.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.891+0000 7f3880c00640 1 -- 172.21.15.77:0/1090100140 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f386c018840 con 0x7f38841034b0 2024-09-10T09:57:37.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.891+0000 7f3888c00640 1 -- 172.21.15.77:0/1090100140 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f388419d4f0 con 0x7f38841034b0 2024-09-10T09:57:37.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.891+0000 7f3888c00640 1 -- 172.21.15.77:0/1090100140 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38841a4dd0 con 0x7f38841034b0 2024-09-10T09:57:37.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.894+0000 7f388ce6a740 1 Processor -- start 2024-09-10T09:57:37.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.894+0000 7f3880c00640 1 -- 172.21.15.77:0/1090100140 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f386c0189e0 con 0x7f38841034b0 2024-09-10T09:57:37.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.894+0000 7f388ce6a740 1 -- start start 2024-09-10T09:57:37.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.894+0000 7f388ce6a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2f2bb0e40 0x55a2f2bb1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:37.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.894+0000 7f388ce6a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a2f2bb1760 0x55a2f2bbbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:37.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.894+0000 7f388ce6a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2f2bbc020 0x55a2f2bbe410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:37.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.894+0000 7f388ce6a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55a2f2b173e0 con 0x55a2f2bb1760 2024-09-10T09:57:37.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.894+0000 7f388ce6a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a2f2abed80 con 0x55a2f2bbc020 2024-09-10T09:57:37.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.894+0000 7f388ce6a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a2f2b133a0 con 0x55a2f2bb0e40 2024-09-10T09:57:37.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.895+0000 7f3883400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a2f2bb1760 0x55a2f2bbbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:37.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.895+0000 7f3883400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a2f2bb1760 0x55a2f2bbbae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:36430/0 (socket says 172.21.15.77:36430) 2024-09-10T09:57:37.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.895+0000 7f3883400640 1 -- 172.21.15.77:0/1408732065 learned_addr learned my addr 172.21.15.77:0/1408732065 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:57:37.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.895+0000 7f387be00640 1 --2- 172.21.15.77:0/1408732065 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2f2bbc020 0x55a2f2bbe410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:37.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.895+0000 7f3883e00640 1 --2- 172.21.15.77:0/1408732065 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2f2bb0e40 0x55a2f2bb1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:37.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.895+0000 7f3883400640 1 -- 172.21.15.77:0/1408732065 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2f2bb0e40 msgr2=0x55a2f2bb1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:37.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.895+0000 7f3883400640 1 --2- 172.21.15.77:0/1408732065 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2f2bb0e40 0x55a2f2bb1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:37.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.895+0000 7f3883400640 1 -- 172.21.15.77:0/1408732065 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2f2bbc020 msgr2=0x55a2f2bbe410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:37.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.895+0000 7f3883400640 1 --2- 172.21.15.77:0/1408732065 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2f2bbc020 0x55a2f2bbe410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:37.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.895+0000 7f3883400640 1 -- 172.21.15.77:0/1408732065 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2f2addae0 con 0x55a2f2bb1760 2024-09-10T09:57:37.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.895+0000 7f3883400640 1 --2- 172.21.15.77:0/1408732065 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a2f2bb1760 0x55a2f2bbbae0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f3874009e10 tx=0x7f3874011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a208a5c743719c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:37.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.896+0000 7f3880c00640 1 --2- 172.21.15.77:0/1090100140 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f38680a2b80 0x7f38680a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:37.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.896+0000 7f3880c00640 1 -- 172.21.15.77:0/1090100140 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f386c0cf500 con 0x7f38841034b0 2024-09-10T09:57:37.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.896+0000 7f3873e00640 1 -- 172.21.15.77:0/1408732065 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f387401e070 con 0x55a2f2bb1760 2024-09-10T09:57:37.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.896+0000 7f3873e00640 1 -- 172.21.15.77:0/1408732065 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3874019b10 con 0x55a2f2bb1760 2024-09-10T09:57:37.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.896+0000 7f3873e00640 1 -- 172.21.15.77:0/1408732065 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f387400fc00 con 0x55a2f2bb1760 2024-09-10T09:57:37.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.896+0000 7f3883e00640 1 --2- 172.21.15.77:0/1090100140 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f38680a2b80 0x7f38680a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:37.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.897+0000 7f388ce6a740 1 -- 172.21.15.77:0/1408732065 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a2f2bb1760 msgr2=0x55a2f2bbbae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:37.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.897+0000 7f388ce6a740 1 --2- 172.21.15.77:0/1408732065 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a2f2bb1760 0x55a2f2bbbae0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f3874009e10 tx=0x7f3874011810 comp rx=0 tx=0).stop 2024-09-10T09:57:37.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.897+0000 7f3883e00640 1 --2- 172.21.15.77:0/1090100140 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f38680a2b80 0x7f38680a5040 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f387c006010 tx=0x7f387c0098c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:37.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.897+0000 7f388ce6a740 1 -- 172.21.15.77:0/1408732065 shutdown_connections 2024-09-10T09:57:37.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.897+0000 7f388ce6a740 1 --2- 172.21.15.77:0/1408732065 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2f2bbc020 0x55a2f2bbe410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:37.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.897+0000 7f388ce6a740 1 --2- 172.21.15.77:0/1408732065 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a2f2bb1760 0x55a2f2bbbae0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:37.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.897+0000 7f388ce6a740 1 --2- 172.21.15.77:0/1408732065 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2f2bb0e40 0x55a2f2bb1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:37.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.897+0000 7f388ce6a740 1 -- 172.21.15.77:0/1408732065 >> 172.21.15.77:0/1408732065 conn(0x55a2f2bad3d0 msgr2=0x55a2f2bad7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:37.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.897+0000 7f388ce6a740 1 -- 172.21.15.77:0/1408732065 shutdown_connections 2024-09-10T09:57:37.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.897+0000 7f388ce6a740 1 -- 172.21.15.77:0/1408732065 wait complete. 2024-09-10T09:57:37.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.898+0000 7f388ce6a740 1 -- start start 2024-09-10T09:57:37.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.898+0000 7f388ce6a740 1 Processor -- start 2024-09-10T09:57:37.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.898+0000 7f388ce6a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2f2bb0e40 0x55a2f2bb8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:37.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.898+0000 7f388ce6a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a2f2bb1760 0x55a2f2bb94b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:37.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.898+0000 7f388ce6a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2f2bbc020 0x55a2f2bb99f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:37.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.899+0000 7f388ce6a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55a2f2bbebf0 con 0x55a2f2bb1760 2024-09-10T09:57:37.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.899+0000 7f388ce6a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a2f2ad20f0 con 0x55a2f2bb0e40 2024-09-10T09:57:37.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.899+0000 7f388ce6a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a2f2bbed70 con 0x55a2f2bbc020 2024-09-10T09:57:37.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.899+0000 7f388ce6a740 10 client.0 _ll_register_callbacks cb 0x55a2f2ab9bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:57:37.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.899+0000 7f3883400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2f2bb0e40 0x55a2f2bb8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:37.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.899+0000 7f3883400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2f2bb0e40 0x55a2f2bb8f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:53928/0 (socket says 172.21.15.77:53928) 2024-09-10T09:57:37.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.899+0000 7f3883400640 1 -- 172.21.15.77:0/1478200821 learned_addr learned my addr 172.21.15.77:0/1478200821 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:57:37.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.899+0000 7f3883e00640 1 --2- 172.21.15.77:0/1478200821 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2f2bbc020 0x55a2f2bb99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:37.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.899+0000 7f3883400640 1 -- 172.21.15.77:0/1478200821 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2f2bbc020 msgr2=0x55a2f2bb99f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:37.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.899+0000 7f3883400640 1 --2- 172.21.15.77:0/1478200821 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2f2bbc020 0x55a2f2bb99f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:37.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.899+0000 7f3883400640 1 -- 172.21.15.77:0/1478200821 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a2f2bb1760 msgr2=0x55a2f2bb94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:37.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.899+0000 7f3883400640 1 --2- 172.21.15.77:0/1478200821 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a2f2bb1760 0x55a2f2bb94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:37.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.899+0000 7f3883400640 1 -- 172.21.15.77:0/1478200821 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2f28883c0 con 0x55a2f2bb0e40 2024-09-10T09:57:37.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.899+0000 7f3883400640 1 --2- 172.21.15.77:0/1478200821 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2f2bb0e40 0x55a2f2bb8f70 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3874009fa0 tx=0x7f38740198a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c38a8c0a6bbe964 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:37.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.899+0000 7f3870c00640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T09:57:37.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.900+0000 7f3870c00640 1 -- 172.21.15.77:0/1478200821 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3874026020 con 0x55a2f2bb0e40 2024-09-10T09:57:37.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.900+0000 7f388ce6a740 1 -- 172.21.15.77:0/1478200821 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3874019b10 con 0x55a2f2bb0e40 2024-09-10T09:57:37.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.900+0000 7f3870c00640 1 -- 172.21.15.77:0/1478200821 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f387400f900 con 0x55a2f2bb0e40 2024-09-10T09:57:37.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.900+0000 7f3870c00640 1 -- 172.21.15.77:0/1478200821 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f3874031050 con 0x55a2f2bb0e40 2024-09-10T09:57:37.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.900+0000 7f388ce6a740 10 client.26003 fetch_fsmap learned FSMap version 31 2024-09-10T09:57:37.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.900+0000 7f388ce6a740 1 -- 172.21.15.77:0/1478200821 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55a2f2bcf100 con 0x55a2f2bb0e40 2024-09-10T09:57:37.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.900+0000 7f3870c00640 1 -- 172.21.15.77:0/1478200821 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3874019b10 con 0x55a2f2bb0e40 2024-09-10T09:57:37.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.900+0000 7f3870c00640 1 -- 172.21.15.77:0/1478200821 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f38740046c0 con 0x55a2f2bb0e40 2024-09-10T09:57:37.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.901+0000 7f388ce6a740 10 client.26003 fetch_fsmap finished waiting for FSMap version 31 2024-09-10T09:57:37.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.901+0000 7f388ce6a740 10 client.26003 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T09:57:37.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.901+0000 7f388ce6a740 20 client.26003 populate_metadata read hostname 'smithi077' 2024-09-10T09:57:37.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.901+0000 7f388ce6a740 1 --2- 172.21.15.77:0/1478200821 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x55a2f2bd0510 0x55a2f2bd2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:37.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.901+0000 7f388ce6a740 4 client.26003 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:57:37.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.901+0000 7f388ce6a740 1 -- 172.21.15.77:0/1478200821 --> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a2f2b24950 con 0x55a2f2bd0510 2024-09-10T09:57:37.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.901+0000 7f387be00640 1 --2- 172.21.15.77:0/1478200821 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x55a2f2bd0510 0x55a2f2bd2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:37.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.902+0000 7f3870c00640 10 client.26003 ms_handle_connect on v2:172.21.15.134:6836/2461497091 2024-09-10T09:57:37.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.902+0000 7f387be00640 1 --2- 172.21.15.77:0/1478200821 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x55a2f2bd0510 0x55a2f2bd2900 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d8b9e84bc79f78c4 server_cookie=d86929c45ec849e in_seq=0 out_seq=0 2024-09-10T09:57:37.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.903+0000 7f3870c00640 1 -- 172.21.15.77:0/1478200821 <== mds.3 v2:172.21.15.134:6836/2461497091 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a2f2b24950 con 0x55a2f2bd0510 2024-09-10T09:57:37.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.903+0000 7f3870c00640 10 client.26003 handle_command_reply: tid=0 2024-09-10T09:57:37.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.903+0000 7f388ce6a740 1 client.26003 shutdown 2024-09-10T09:57:37.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.903+0000 7f388ce6a740 1 -- 172.21.15.77:0/1478200821 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2f2bb0e40 msgr2=0x55a2f2bb8f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:37.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.903+0000 7f388ce6a740 1 --2- 172.21.15.77:0/1478200821 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2f2bb0e40 0x55a2f2bb8f70 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3874009fa0 tx=0x7f38740198a0 comp rx=0 tx=0).stop 2024-09-10T09:57:37.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.903+0000 7f388ce6a740 1 -- 172.21.15.77:0/1478200821 shutdown_connections 2024-09-10T09:57:37.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.903+0000 7f388ce6a740 1 --2- 172.21.15.77:0/1478200821 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2f2bbc020 0x55a2f2bb99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:37.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.903+0000 7f388ce6a740 1 --2- 172.21.15.77:0/1478200821 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x55a2f2bd0510 0x55a2f2bd2900 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:37.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.903+0000 7f388ce6a740 1 --2- 172.21.15.77:0/1478200821 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a2f2bb1760 0x55a2f2bb94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:37.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.903+0000 7f388ce6a740 1 --2- 172.21.15.77:0/1478200821 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2f2bb0e40 0x55a2f2bb8f70 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:37.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.903+0000 7f3870c00640 0 client.26003 ms_handle_reset on v2:172.21.15.134:6836/2461497091 2024-09-10T09:57:37.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.903+0000 7f388ce6a740 1 -- 172.21.15.77:0/1478200821 >> 172.21.15.77:0/1478200821 conn(0x55a2f2bad3d0 msgr2=0x55a2f2bb30b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:37.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.903+0000 7f388ce6a740 1 -- 172.21.15.77:0/1478200821 shutdown_connections 2024-09-10T09:57:37.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.903+0000 7f388ce6a740 1 -- 172.21.15.77:0/1478200821 wait complete. 2024-09-10T09:57:37.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.903+0000 7f388ce6a740 20 client.26003 trim_cache size 0 max 16384 2024-09-10T09:57:37.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.918+0000 7f388ce6a740 1 Processor -- start 2024-09-10T09:57:37.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.918+0000 7f388ce6a740 1 -- start start 2024-09-10T09:57:37.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.918+0000 7f388ce6a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2f2bb0e40 0x55a2f2bb84d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:37.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.918+0000 7f388ce6a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2f2bb1760 0x55a2f2bd8110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:37.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.919+0000 7f388ce6a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a2f2bbc020 0x55a2f2bd8650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:37.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.919+0000 7f388ce6a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f38841acb20 con 0x55a2f2bbc020 2024-09-10T09:57:37.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.919+0000 7f388ce6a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f38841ae6f0 con 0x55a2f2bb0e40 2024-09-10T09:57:37.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.919+0000 7f388ce6a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f38841ad1f0 con 0x55a2f2bb1760 2024-09-10T09:57:37.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.919+0000 7f3883400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2f2bb0e40 0x55a2f2bb84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:37.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.919+0000 7f387be00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2f2bb1760 0x55a2f2bd8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:37.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.919+0000 7f3883400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2f2bb0e40 0x55a2f2bb84d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:53934/0 (socket says 172.21.15.77:53934) 2024-09-10T09:57:37.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.919+0000 7f3883400640 1 -- 172.21.15.77:0/314109744 learned_addr learned my addr 172.21.15.77:0/314109744 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:57:37.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.919+0000 7f3883e00640 1 --2- 172.21.15.77:0/314109744 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a2f2bbc020 0x55a2f2bd8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:37.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.919+0000 7f3883400640 1 -- 172.21.15.77:0/314109744 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2f2bb1760 msgr2=0x55a2f2bd8110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:37.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.919+0000 7f3883400640 1 --2- 172.21.15.77:0/314109744 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2f2bb1760 0x55a2f2bd8110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:37.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.919+0000 7f3883400640 1 -- 172.21.15.77:0/314109744 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a2f2bbc020 msgr2=0x55a2f2bd8650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:37.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.919+0000 7f3883400640 1 --2- 172.21.15.77:0/314109744 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a2f2bbc020 0x55a2f2bd8650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:37.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.919+0000 7f3883400640 1 -- 172.21.15.77:0/314109744 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2f2bb0bc0 con 0x55a2f2bb0e40 2024-09-10T09:57:37.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.919+0000 7f387be00640 1 --2- 172.21.15.77:0/314109744 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2f2bb1760 0x55a2f2bd8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:57:37.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.919+0000 7f3883e00640 1 --2- 172.21.15.77:0/314109744 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a2f2bbc020 0x55a2f2bd8650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:57:37.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.920+0000 7f3883400640 1 --2- 172.21.15.77:0/314109744 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2f2bb0e40 0x55a2f2bb84d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f38740059f0 tx=0x7f387400f780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70af85f7644c1be0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:37.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.920+0000 7f3873400640 1 -- 172.21.15.77:0/314109744 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3874026070 con 0x55a2f2bb0e40 2024-09-10T09:57:37.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.920+0000 7f3873400640 1 -- 172.21.15.77:0/314109744 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3874004d00 con 0x55a2f2bb0e40 2024-09-10T09:57:37.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.920+0000 7f3873400640 1 -- 172.21.15.77:0/314109744 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f387401e040 con 0x55a2f2bb0e40 2024-09-10T09:57:37.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.921+0000 7f388ce6a740 1 -- 172.21.15.77:0/314109744 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2f2bb0e40 msgr2=0x55a2f2bb84d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:37.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.921+0000 7f388ce6a740 1 --2- 172.21.15.77:0/314109744 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2f2bb0e40 0x55a2f2bb84d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f38740059f0 tx=0x7f387400f780 comp rx=0 tx=0).stop 2024-09-10T09:57:37.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.921+0000 7f388ce6a740 1 -- 172.21.15.77:0/314109744 shutdown_connections 2024-09-10T09:57:37.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.921+0000 7f388ce6a740 1 --2- 172.21.15.77:0/314109744 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a2f2bbc020 0x55a2f2bd8650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:37.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.921+0000 7f388ce6a740 1 --2- 172.21.15.77:0/314109744 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2f2bb1760 0x55a2f2bd8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:37.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.921+0000 7f388ce6a740 1 --2- 172.21.15.77:0/314109744 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2f2bb0e40 0x55a2f2bb84d0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:37.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.921+0000 7f388ce6a740 1 -- 172.21.15.77:0/314109744 >> 172.21.15.77:0/314109744 conn(0x55a2f2bad3d0 msgr2=0x55a2f2bbaff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:37.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.921+0000 7f388ce6a740 1 -- 172.21.15.77:0/314109744 shutdown_connections 2024-09-10T09:57:37.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.921+0000 7f388ce6a740 1 -- 172.21.15.77:0/314109744 wait complete. 2024-09-10T09:57:37.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.921+0000 7f388ce6a740 1 -- start start 2024-09-10T09:57:37.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.922+0000 7f388ce6a740 1 Processor -- start 2024-09-10T09:57:37.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.922+0000 7f388ce6a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2f2bb0e40 0x55a2f2bc8be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:37.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.922+0000 7f388ce6a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2f2bb1760 0x55a2f2bc3f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:37.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.922+0000 7f388ce6a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a2f2bbc020 0x55a2f2bc4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:37.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.922+0000 7f388ce6a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f38841adf90 con 0x55a2f2bbc020 2024-09-10T09:57:37.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.922+0000 7f388ce6a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f38841ad8c0 con 0x55a2f2bb0e40 2024-09-10T09:57:37.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.922+0000 7f388ce6a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f38841aab00 con 0x55a2f2bb1760 2024-09-10T09:57:37.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.922+0000 7f388ce6a740 10 client.0 _ll_register_callbacks cb 0x7f387401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:57:37.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.922+0000 7f3883400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2f2bb0e40 0x55a2f2bc8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:37.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.922+0000 7f3883400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2f2bb0e40 0x55a2f2bc8be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:53950/0 (socket says 172.21.15.77:53950) 2024-09-10T09:57:37.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.922+0000 7f3883400640 1 -- 172.21.15.77:0/3835593650 learned_addr learned my addr 172.21.15.77:0/3835593650 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:57:37.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.922+0000 7f387be00640 1 --2- 172.21.15.77:0/3835593650 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2f2bb1760 0x55a2f2bc3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:37.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.922+0000 7f3883e00640 1 --2- 172.21.15.77:0/3835593650 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a2f2bbc020 0x55a2f2bc4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:37.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.923+0000 7f3883400640 1 -- 172.21.15.77:0/3835593650 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2f2bb1760 msgr2=0x55a2f2bc3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:37.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.923+0000 7f3883400640 1 --2- 172.21.15.77:0/3835593650 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2f2bb1760 0x55a2f2bc3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:37.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.923+0000 7f3883400640 1 -- 172.21.15.77:0/3835593650 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a2f2bbc020 msgr2=0x55a2f2bc4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:37.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.923+0000 7f3883400640 1 --2- 172.21.15.77:0/3835593650 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a2f2bbc020 0x55a2f2bc4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:37.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.923+0000 7f3883400640 1 -- 172.21.15.77:0/3835593650 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2f2bc4c50 con 0x55a2f2bb0e40 2024-09-10T09:57:37.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.923+0000 7f3883e00640 1 --2- 172.21.15.77:0/3835593650 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a2f2bbc020 0x55a2f2bc4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:57:37.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.923+0000 7f3883400640 1 --2- 172.21.15.77:0/3835593650 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2f2bb0e40 0x55a2f2bc8be0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3874005b90 tx=0x7f3874004680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dafe48db6cccd312 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:37.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.923+0000 7f3870c00640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T09:57:37.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.923+0000 7f3870c00640 1 -- 172.21.15.77:0/3835593650 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3874039070 con 0x55a2f2bb0e40 2024-09-10T09:57:37.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.923+0000 7f3870c00640 1 -- 172.21.15.77:0/3835593650 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f387400f800 con 0x55a2f2bb0e40 2024-09-10T09:57:37.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.923+0000 7f388ce6a740 1 -- 172.21.15.77:0/3835593650 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3874004d00 con 0x55a2f2bb0e40 2024-09-10T09:57:37.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.923+0000 7f3870c00640 1 -- 172.21.15.77:0/3835593650 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f3874041050 con 0x55a2f2bb0e40 2024-09-10T09:57:37.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.923+0000 7f388ce6a740 10 client.26009 fetch_fsmap learned FSMap version 31 2024-09-10T09:57:37.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.923+0000 7f388ce6a740 1 -- 172.21.15.77:0/3835593650 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55a2f2c3b110 con 0x55a2f2bb0e40 2024-09-10T09:57:37.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.924+0000 7f3870c00640 1 -- 172.21.15.77:0/3835593650 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3874004d00 con 0x55a2f2bb0e40 2024-09-10T09:57:37.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.924+0000 7f3870c00640 1 -- 172.21.15.77:0/3835593650 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f387403c030 con 0x55a2f2bb0e40 2024-09-10T09:57:37.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.924+0000 7f388ce6a740 10 client.26009 fetch_fsmap finished waiting for FSMap version 31 2024-09-10T09:57:37.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.924+0000 7f388ce6a740 10 client.26009 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T09:57:37.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.924+0000 7f388ce6a740 20 client.26009 populate_metadata read hostname 'smithi077' 2024-09-10T09:57:37.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.924+0000 7f388ce6a740 1 --2- 172.21.15.77:0/3835593650 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x55a2f2c3b770 0x55a2f2c3db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:37.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.924+0000 7f388ce6a740 4 client.26009 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:57:37.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.924+0000 7f388ce6a740 1 -- 172.21.15.77:0/3835593650 --> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a2f2b1ee60 con 0x55a2f2c3b770 2024-09-10T09:57:37.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.924+0000 7f387be00640 1 --2- 172.21.15.77:0/3835593650 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x55a2f2c3b770 0x55a2f2c3db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:37.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.925+0000 7f387be00640 1 --2- 172.21.15.77:0/3835593650 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x55a2f2c3b770 0x55a2f2c3db60 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e86a5538185c0ab9 server_cookie=69ab43442eeef24a in_seq=0 out_seq=0 2024-09-10T09:57:37.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.925+0000 7f3870c00640 10 client.26009 ms_handle_connect on v2:172.21.15.134:6836/2461497091 2024-09-10T09:57:37.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.925+0000 7f3870c00640 1 -- 172.21.15.77:0/3835593650 <== mds.3 v2:172.21.15.134:6836/2461497091 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a2f2b1ee60 con 0x55a2f2c3b770 2024-09-10T09:57:37.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.925+0000 7f3870c00640 10 client.26009 handle_command_reply: tid=0 2024-09-10T09:57:37.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.925+0000 7f388ce6a740 1 client.26009 shutdown 2024-09-10T09:57:37.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.925+0000 7f388ce6a740 1 -- 172.21.15.77:0/3835593650 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2f2bb0e40 msgr2=0x55a2f2bc8be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:37.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.925+0000 7f388ce6a740 1 --2- 172.21.15.77:0/3835593650 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2f2bb0e40 0x55a2f2bc8be0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3874005b90 tx=0x7f3874004680 comp rx=0 tx=0).stop 2024-09-10T09:57:37.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.925+0000 7f388ce6a740 1 -- 172.21.15.77:0/3835593650 shutdown_connections 2024-09-10T09:57:37.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.925+0000 7f388ce6a740 1 --2- 172.21.15.77:0/3835593650 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x55a2f2c3b770 0x55a2f2c3db60 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:37.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.925+0000 7f388ce6a740 1 --2- 172.21.15.77:0/3835593650 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a2f2bbc020 0x55a2f2bc4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:37.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.925+0000 7f388ce6a740 1 --2- 172.21.15.77:0/3835593650 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2f2bb1760 0x55a2f2bc3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:37.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.925+0000 7f3870c00640 0 client.26009 ms_handle_reset on v2:172.21.15.134:6836/2461497091 2024-09-10T09:57:37.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.925+0000 7f388ce6a740 1 --2- 172.21.15.77:0/3835593650 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2f2bb0e40 0x55a2f2bc8be0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:37.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.925+0000 7f388ce6a740 1 -- 172.21.15.77:0/3835593650 >> 172.21.15.77:0/3835593650 conn(0x55a2f2bad3d0 msgr2=0x55a2f2bb2800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:37.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.926+0000 7f388ce6a740 1 -- 172.21.15.77:0/3835593650 shutdown_connections 2024-09-10T09:57:37.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.926+0000 7f388ce6a740 1 -- 172.21.15.77:0/3835593650 wait complete. 2024-09-10T09:57:37.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.926+0000 7f388ce6a740 20 client.26009 trim_cache size 0 max 16384 2024-09-10T09:57:37.931 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-10T09:57:37.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.926+0000 7f3888c00640 1 -- 172.21.15.77:0/1090100140 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f38680a2b80 msgr2=0x7f38680a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:37.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.926+0000 7f3888c00640 1 --2- 172.21.15.77:0/1090100140 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f38680a2b80 0x7f38680a5040 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f387c006010 tx=0x7f387c0098c0 comp rx=0 tx=0).stop 2024-09-10T09:57:37.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.926+0000 7f3888c00640 1 -- 172.21.15.77:0/1090100140 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f38841034b0 msgr2=0x7f3884198790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:37.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.926+0000 7f3888c00640 1 --2- 172.21.15.77:0/1090100140 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f38841034b0 0x7f3884198790 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f386c00e6c0 tx=0x7f386c00eb80 comp rx=0 tx=0).stop 2024-09-10T09:57:37.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.926+0000 7f3888c00640 1 -- 172.21.15.77:0/1090100140 shutdown_connections 2024-09-10T09:57:37.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.926+0000 7f3888c00640 1 --2- 172.21.15.77:0/1090100140 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f38680a2b80 0x7f38680a5040 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:37.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.926+0000 7f3888c00640 1 --2- 172.21.15.77:0/1090100140 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f388410bfc0 0x7f388419cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:37.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.926+0000 7f3888c00640 1 --2- 172.21.15.77:0/1090100140 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f38841034b0 0x7f3884198790 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:37.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.926+0000 7f3888c00640 1 --2- 172.21.15.77:0/1090100140 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3884102b90 0x7f3884198250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:37.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.926+0000 7f3888c00640 1 -- 172.21.15.77:0/1090100140 >> 172.21.15.77:0/1090100140 conn(0x7f38840fc610 msgr2=0x7f38840fd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:37.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.927+0000 7f3888c00640 1 -- 172.21.15.77:0/1090100140 shutdown_connections 2024-09-10T09:57:37.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:37.927+0000 7f3888c00640 1 -- 172.21.15.77:0/1090100140 wait complete. 2024-09-10T09:57:38.063 INFO:tasks.workunit.client.0.smithi077.stderr:Updating files: 7% (322/4351) Updating files: 8% (349/4351) Updating files: 9% (392/4351) Updating files: 10% (436/4351) Updating files: 11% (479/4351) Updating files: 12% (523/4351) Updating files: 13% (566/4351) Updating files: 14% (610/4351) Updating files: 15% (653/4351) Updating files: 16% (697/4351) Updating files: 17% (740/4351) Updating files: 18% (784/4351) Updating files: 19% (827/4351) Updating files: 20% (871/4351) Updating files: 21% (914/4351) Updating files: 22% (958/4351) Updating files: 23% (1001/4351) Updating files: 24% (1045/4351) Updating files: 25% (1088/4351) Updating files: 25% (1109/4351) Updating files: 26% (1132/4351) Updating files: 27% (1175/4351) Updating files: 28% (1219/4351) Updating files: 29% (1262/4351) Updating files: 29% (1303/4351) Updating files: 30% (1306/4351) Updating files: 31% (1349/4351) Updating files: 32% (1393/4351) Updating files: 33% (1436/4351) Updating files: 34% (1480/4351) Updating files: 35% (1523/4351) Updating files: 36% (1567/4351) Updating files: 37% (1610/4351) Updating files: 38% (1654/4351) Updating files: 39% (1697/4351) Updating files: 40% (1741/4351) Updating files: 41% (1784/4351) Updating files: 42% (1828/4351) Updating files: 43% (1871/4351) Updating files: 44% (1915/4351) Updating files: 45% (1958/4351) Updating files: 45% (1974/4351) Updating files: 46% (2002/4351) Updating files: 47% (2045/4351) Updating files: 48% (2089/4351) Updating files: 49% (2132/4351) Updating files: 50% (2176/4351) Updating files: 51% (2220/4351) Updating files: 52% (2263/4351) Updating files: 53% (2307/4351) Updating files: 54% (2350/4351) Updating files: 55% (2394/4351) Updating files: 56% (2437/4351) Updating files: 57% (2481/4351) Updating files: 58% (2524/4351) Updating files: 58% (2565/4351) Updating files: 59% (2568/4351) Updating files: 60% (2611/4351) Updating files: 61% (2655/4351) Updating files: 62% (2698/4351) Updating files: 63% (2742/4351) Updating files: 64% (2785/4351) Updating files: 64% (2809/4351) Updating files: 64% (2820/4351) Updating files: 65% (2829/4351) Updating files: 66% (2872/4351) Updating files: 67% (2916/4351) Updating files: 68% (2959/4351) Updating files: 69% (3003/4351) Updating files: 70% (3046/4351) Updating files: 71% (3090/4351) Updating files: 72% (3133/4351) Updating files: 73% (3177/4351) Updating files: 74% (3220/4351) Updating files: 75% (3264/4351) Updating files: 76% (3307/4351) Updating files: 77% (3351/4351) Updating files: 78% (3394/4351) Updating files: 79% (3438/4351) Updating files: 80% (3481/4351) Updating files: 81% (3525/4351) Updating files: 82% (3568/4351) Updating files: 82% (3604/4351) Updating files: 83% (3612/4351) Updating files: 84% (3655/4351) Updating files: 85% (3699/4351) Updating files: 86% (3742/4351) Updating files: 87% (3786/4351) Updating files: 88% (3829/4351) Updating files: 89% (3873/4351) Updating files: 90% (3916/4351) Updating files: 91% (3960/4351) Updating files: 92% (4003/4351) Updating files: 93% (4047/4351) Updating files: 94% (4090/4351) Updating files: 95% (4134/4351) Updating files: 96% (4177/4351) Updating files: 96% (4189/4351) Updating files: 97% (4221/4351) Updating files: 98% (4264/4351) Updating files: 99% (4308/4351) Updating files: 100% (4351/4351) Updating files: 100% (4351/4351), done. 2024-09-10T09:57:38.077 INFO:tasks.workunit.client.0.smithi077.stderr:+ cd xfstests-dev 2024-09-10T09:57:38.077 INFO:tasks.workunit.client.0.smithi077.stderr:+ git checkout 0e5c12dfd008efc2848c98108c9237487e91ef35 2024-09-10T09:57:38.749 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T09:57:39.126 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:57:39.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:57:38 smithi120 ceph-mon[34770]: pgmap v465: 129 pgs: 129 active+clean; 20 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 965 KiB/s wr, 37 op/s 2024-09-10T09:57:39.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:57:38 smithi077 ceph-mon[30319]: pgmap v465: 129 pgs: 129 active+clean; 20 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 965 KiB/s wr, 37 op/s 2024-09-10T09:57:39.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:57:38 smithi134 ceph-mon[35674]: pgmap v465: 129 pgs: 129 active+clean; 20 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 965 KiB/s wr, 37 op/s 2024-09-10T09:57:39.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.721+0000 7ff1a1200640 1 -- 172.21.15.77:0/2266251953 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff19c076720 msgr2=0x7ff19c076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:39.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.721+0000 7ff1a1200640 1 --2- 172.21.15.77:0/2266251953 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff19c076720 0x7ff19c076b00 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff18c009900 tx=0x7ff18c02a400 comp rx=0 tx=0).stop 2024-09-10T09:57:39.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.721+0000 7ff1a1200640 1 -- 172.21.15.77:0/2266251953 shutdown_connections 2024-09-10T09:57:39.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.721+0000 7ff1a1200640 1 --2- 172.21.15.77:0/2266251953 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff19c112870 0x7ff19c114c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:39.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.721+0000 7ff1a1200640 1 --2- 172.21.15.77:0/2266251953 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff19c077040 0x7ff19c112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:39.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.721+0000 7ff1a1200640 1 --2- 172.21.15.77:0/2266251953 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff19c076720 0x7ff19c076b00 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:39.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.721+0000 7ff1a1200640 1 -- 172.21.15.77:0/2266251953 >> 172.21.15.77:0/2266251953 conn(0x7ff19c0ff910 msgr2=0x7ff19c101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:39.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.721+0000 7ff1a1200640 1 -- 172.21.15.77:0/2266251953 shutdown_connections 2024-09-10T09:57:39.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.721+0000 7ff1a1200640 1 -- 172.21.15.77:0/2266251953 wait complete. 2024-09-10T09:57:39.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.722+0000 7ff1a1200640 1 Processor -- start 2024-09-10T09:57:39.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.722+0000 7ff1a1200640 1 -- start start 2024-09-10T09:57:39.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.722+0000 7ff1a1200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff19c076720 0x7ff19c1a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:39.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.722+0000 7ff1a1200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff19c077040 0x7ff19c1a4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:39.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.722+0000 7ff1a1200640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff19c112870 0x7ff19c19e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:39.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.722+0000 7ff1a1200640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff19c118450 con 0x7ff19c077040 2024-09-10T09:57:39.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.722+0000 7ff1a1200640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff19c1182d0 con 0x7ff19c112870 2024-09-10T09:57:39.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.722+0000 7ff1a1200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff19c1185d0 con 0x7ff19c076720 2024-09-10T09:57:39.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.723+0000 7ff19be00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff19c112870 0x7ff19c19e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:39.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.723+0000 7ff19b400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff19c076720 0x7ff19c1a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:39.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.723+0000 7ff19be00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff19c112870 0x7ff19c19e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:53962/0 (socket says 172.21.15.77:53962) 2024-09-10T09:57:39.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.723+0000 7ff19be00640 1 -- 172.21.15.77:0/3935158655 learned_addr learned my addr 172.21.15.77:0/3935158655 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:57:39.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.723+0000 7ff19aa00640 1 --2- 172.21.15.77:0/3935158655 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff19c077040 0x7ff19c1a4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:39.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.723+0000 7ff19be00640 1 -- 172.21.15.77:0/3935158655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff19c076720 msgr2=0x7ff19c1a4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:39.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.723+0000 7ff19be00640 1 --2- 172.21.15.77:0/3935158655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff19c076720 0x7ff19c1a4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:39.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.723+0000 7ff19be00640 1 -- 172.21.15.77:0/3935158655 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff19c077040 msgr2=0x7ff19c1a4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:39.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.723+0000 7ff19be00640 1 --2- 172.21.15.77:0/3935158655 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff19c077040 0x7ff19c1a4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:39.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.723+0000 7ff19be00640 1 -- 172.21.15.77:0/3935158655 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff19c19ef20 con 0x7ff19c112870 2024-09-10T09:57:39.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.723+0000 7ff19be00640 1 --2- 172.21.15.77:0/3935158655 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff19c112870 0x7ff19c19e780 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff19400eb10 tx=0x7ff19400c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:39.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.723+0000 7ff193e00640 1 -- 172.21.15.77:0/3935158655 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff194004b20 con 0x7ff19c112870 2024-09-10T09:57:39.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.723+0000 7ff1a1200640 1 -- 172.21.15.77:0/3935158655 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff19c19f210 con 0x7ff19c112870 2024-09-10T09:57:39.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.724+0000 7ff193e00640 1 -- 172.21.15.77:0/3935158655 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff194004cc0 con 0x7ff19c112870 2024-09-10T09:57:39.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.724+0000 7ff193e00640 1 -- 172.21.15.77:0/3935158655 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff194010790 con 0x7ff19c112870 2024-09-10T09:57:39.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.724+0000 7ff1a1200640 1 -- 172.21.15.77:0/3935158655 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff19c1ab460 con 0x7ff19c112870 2024-09-10T09:57:39.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.726+0000 7ff1a52b8740 1 Processor -- start 2024-09-10T09:57:39.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.727+0000 7ff193e00640 1 -- 172.21.15.77:0/3935158655 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff194005020 con 0x7ff19c112870 2024-09-10T09:57:39.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.727+0000 7ff1a52b8740 1 -- start start 2024-09-10T09:57:39.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.727+0000 7ff1a52b8740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da856bde40 0x55da856be220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:39.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.727+0000 7ff1a52b8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da856be760 0x55da856c8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:39.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.727+0000 7ff1a52b8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da856c9020 0x55da856cb410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:39.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.727+0000 7ff1a52b8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55da856243e0 con 0x55da856c9020 2024-09-10T09:57:39.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.727+0000 7ff1a52b8740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55da855cbd80 con 0x55da856bde40 2024-09-10T09:57:39.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.727+0000 7ff1a52b8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55da856203a0 con 0x55da856be760 2024-09-10T09:57:39.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.727+0000 7ff19b400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da856be760 0x55da856c8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:39.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.727+0000 7ff19aa00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da856bde40 0x55da856be220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:39.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.727+0000 7ff19b400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da856be760 0x55da856c8ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:37494/0 (socket says 172.21.15.77:37494) 2024-09-10T09:57:39.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.727+0000 7ff19b400640 1 -- 172.21.15.77:0/3584772353 learned_addr learned my addr 172.21.15.77:0/3584772353 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:57:39.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.728+0000 7ff19aa00640 1 --2- 172.21.15.77:0/3584772353 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da856c9020 0x55da856cb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:39.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.728+0000 7ff19b400640 1 -- 172.21.15.77:0/3584772353 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da856bde40 msgr2=0x55da856be220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:39.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.728+0000 7ff19b400640 1 --2- 172.21.15.77:0/3584772353 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da856bde40 0x55da856be220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:39.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.728+0000 7ff19b400640 1 -- 172.21.15.77:0/3584772353 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da856c9020 msgr2=0x55da856cb410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:39.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.728+0000 7ff19b400640 1 --2- 172.21.15.77:0/3584772353 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da856c9020 0x55da856cb410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:39.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.728+0000 7ff19b400640 1 -- 172.21.15.77:0/3584772353 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da855eaae0 con 0x55da856be760 2024-09-10T09:57:39.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.728+0000 7ff19aa00640 1 --2- 172.21.15.77:0/3584772353 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da856c9020 0x55da856cb410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:57:39.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.728+0000 7ff19b400640 1 --2- 172.21.15.77:0/3584772353 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da856be760 0x55da856c8ae0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff18c009900 tx=0x7ff18c005de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14d952b93c2a410 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:39.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.728+0000 7ff190c00640 1 -- 172.21.15.77:0/3584772353 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff18c035040 con 0x55da856be760 2024-09-10T09:57:39.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.728+0000 7ff190c00640 1 -- 172.21.15.77:0/3584772353 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff18c004400 con 0x55da856be760 2024-09-10T09:57:39.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.728+0000 7ff190c00640 1 -- 172.21.15.77:0/3584772353 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff18c0035f0 con 0x55da856be760 2024-09-10T09:57:39.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.729+0000 7ff193e00640 1 --2- 172.21.15.77:0/3935158655 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ff1800a2b80 0x7ff1800a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:39.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.729+0000 7ff193e00640 1 -- 172.21.15.77:0/3935158655 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7ff1940cfb40 con 0x7ff19c112870 2024-09-10T09:57:39.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.729+0000 7ff1a52b8740 1 -- 172.21.15.77:0/3584772353 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da856be760 msgr2=0x55da856c8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:39.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.729+0000 7ff1a52b8740 1 --2- 172.21.15.77:0/3584772353 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da856be760 0x55da856c8ae0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff18c009900 tx=0x7ff18c005de0 comp rx=0 tx=0).stop 2024-09-10T09:57:39.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.729+0000 7ff19aa00640 1 --2- 172.21.15.77:0/3935158655 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ff1800a2b80 0x7ff1800a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:39.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.729+0000 7ff1a52b8740 1 -- 172.21.15.77:0/3584772353 shutdown_connections 2024-09-10T09:57:39.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.729+0000 7ff1a52b8740 1 --2- 172.21.15.77:0/3584772353 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da856c9020 0x55da856cb410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:39.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.729+0000 7ff1a52b8740 1 --2- 172.21.15.77:0/3584772353 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da856be760 0x55da856c8ae0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:39.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.729+0000 7ff1a52b8740 1 --2- 172.21.15.77:0/3584772353 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da856bde40 0x55da856be220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:39.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.729+0000 7ff1a52b8740 1 -- 172.21.15.77:0/3584772353 >> 172.21.15.77:0/3584772353 conn(0x55da856ba3d0 msgr2=0x55da856ba7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:39.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.729+0000 7ff1a52b8740 1 -- 172.21.15.77:0/3584772353 shutdown_connections 2024-09-10T09:57:39.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.730+0000 7ff1a52b8740 1 -- 172.21.15.77:0/3584772353 wait complete. 2024-09-10T09:57:39.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.730+0000 7ff19aa00640 1 --2- 172.21.15.77:0/3935158655 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ff1800a2b80 0x7ff1800a5040 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x55da856bb750 tx=0x7ff188013040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:39.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.730+0000 7ff1a52b8740 1 -- start start 2024-09-10T09:57:39.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.730+0000 7ff1a52b8740 1 Processor -- start 2024-09-10T09:57:39.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.731+0000 7ff1a52b8740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da856bde40 0x55da856c5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:39.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.731+0000 7ff1a52b8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da856be760 0x55da856c6400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:39.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.731+0000 7ff1a52b8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da856c9020 0x55da856c6940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:39.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.731+0000 7ff1a52b8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55da856cbbf0 con 0x55da856c9020 2024-09-10T09:57:39.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.731+0000 7ff1a52b8740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55da855df0f0 con 0x55da856bde40 2024-09-10T09:57:39.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.731+0000 7ff1a52b8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55da856cbd70 con 0x55da856be760 2024-09-10T09:57:39.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.731+0000 7ff1a52b8740 10 client.0 _ll_register_callbacks cb 0x55da855c6bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:57:39.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.731+0000 7ff19b400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da856bde40 0x55da856c5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:39.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.731+0000 7ff19aa00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da856be760 0x55da856c6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:39.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.731+0000 7ff19b400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da856bde40 0x55da856c5ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:53976/0 (socket says 172.21.15.77:53976) 2024-09-10T09:57:39.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.731+0000 7ff19aa00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da856be760 0x55da856c6400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:37510/0 (socket says 172.21.15.77:37510) 2024-09-10T09:57:39.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.731+0000 7ff19b400640 1 -- 172.21.15.77:0/3639399019 learned_addr learned my addr 172.21.15.77:0/3639399019 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:57:39.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.731+0000 7ff19be00640 1 --2- 172.21.15.77:0/3639399019 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da856c9020 0x55da856c6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:39.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.731+0000 7ff19b400640 1 -- 172.21.15.77:0/3639399019 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da856be760 msgr2=0x55da856c6400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:39.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.731+0000 7ff19b400640 1 --2- 172.21.15.77:0/3639399019 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da856be760 0x55da856c6400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:39.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.731+0000 7ff19b400640 1 -- 172.21.15.77:0/3639399019 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da856c9020 msgr2=0x55da856c6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:39.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.731+0000 7ff19b400640 1 --2- 172.21.15.77:0/3639399019 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da856c9020 0x55da856c6940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:39.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.731+0000 7ff19b400640 1 -- 172.21.15.77:0/3639399019 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da853953c0 con 0x55da856bde40 2024-09-10T09:57:39.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.732+0000 7ff19b400640 1 --2- 172.21.15.77:0/3639399019 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da856bde40 0x55da856c5ec0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff18c005c40 tx=0x7ff18c004b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4f50cd9946bb9c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:39.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.732+0000 7ff185600640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T09:57:39.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.732+0000 7ff185600640 1 -- 172.21.15.77:0/3639399019 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff18c052060 con 0x55da856bde40 2024-09-10T09:57:39.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.732+0000 7ff1a52b8740 1 -- 172.21.15.77:0/3639399019 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff18c004400 con 0x55da856bde40 2024-09-10T09:57:39.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.732+0000 7ff185600640 1 -- 172.21.15.77:0/3639399019 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7ff18c004200 con 0x55da856bde40 2024-09-10T09:57:39.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.732+0000 7ff185600640 1 -- 172.21.15.77:0/3639399019 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff18c035070 con 0x55da856bde40 2024-09-10T09:57:39.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.732+0000 7ff1a52b8740 10 client.26021 fetch_fsmap learned FSMap version 31 2024-09-10T09:57:39.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.732+0000 7ff1a52b8740 1 -- 172.21.15.77:0/3639399019 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55da856d7bf0 con 0x55da856bde40 2024-09-10T09:57:39.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.733+0000 7ff185600640 1 -- 172.21.15.77:0/3639399019 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff18c004400 con 0x55da856bde40 2024-09-10T09:57:39.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.733+0000 7ff185600640 1 -- 172.21.15.77:0/3639399019 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7ff18c031b50 con 0x55da856bde40 2024-09-10T09:57:39.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.733+0000 7ff1a52b8740 10 client.26021 fetch_fsmap finished waiting for FSMap version 31 2024-09-10T09:57:39.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.733+0000 7ff1a52b8740 10 client.26021 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T09:57:39.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.733+0000 7ff1a52b8740 20 client.26021 populate_metadata read hostname 'smithi077' 2024-09-10T09:57:39.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.733+0000 7ff1a52b8740 1 --2- 172.21.15.77:0/3639399019 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x55da856d8f40 0x55da856db330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:39.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.733+0000 7ff1a52b8740 4 client.26021 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:57:39.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.733+0000 7ff1a52b8740 1 -- 172.21.15.77:0/3639399019 --> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55da85631950 con 0x55da856d8f40 2024-09-10T09:57:39.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.733+0000 7ff19aa00640 1 --2- 172.21.15.77:0/3639399019 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x55da856d8f40 0x55da856db330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:39.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.734+0000 7ff19aa00640 1 --2- 172.21.15.77:0/3639399019 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x55da856d8f40 0x55da856db330 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=59c4ff79c1b0b73 server_cookie=784969acd64dd6a3 in_seq=0 out_seq=0 2024-09-10T09:57:39.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.734+0000 7ff185600640 10 client.26021 ms_handle_connect on v2:172.21.15.134:6838/1965117680 2024-09-10T09:57:39.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.735+0000 7ff185600640 1 -- 172.21.15.77:0/3639399019 <== mds.2 v2:172.21.15.134:6838/1965117680 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55da85631950 con 0x55da856d8f40 2024-09-10T09:57:39.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.735+0000 7ff185600640 10 client.26021 handle_command_reply: tid=0 2024-09-10T09:57:39.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.735+0000 7ff1a52b8740 1 client.26021 shutdown 2024-09-10T09:57:39.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.735+0000 7ff1a52b8740 1 -- 172.21.15.77:0/3639399019 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da856bde40 msgr2=0x55da856c5ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:39.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.735+0000 7ff1a52b8740 1 --2- 172.21.15.77:0/3639399019 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da856bde40 0x55da856c5ec0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff18c005c40 tx=0x7ff18c004b20 comp rx=0 tx=0).stop 2024-09-10T09:57:39.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.735+0000 7ff1a52b8740 1 -- 172.21.15.77:0/3639399019 shutdown_connections 2024-09-10T09:57:39.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.735+0000 7ff1a52b8740 1 --2- 172.21.15.77:0/3639399019 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x55da856d8f40 0x55da856db330 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:39.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.735+0000 7ff1a52b8740 1 --2- 172.21.15.77:0/3639399019 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da856c9020 0x55da856c6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:39.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.735+0000 7ff185600640 0 client.26021 ms_handle_reset on v2:172.21.15.134:6838/1965117680 2024-09-10T09:57:39.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.735+0000 7ff1a52b8740 1 --2- 172.21.15.77:0/3639399019 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da856be760 0x55da856c6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:39.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.735+0000 7ff1a52b8740 1 --2- 172.21.15.77:0/3639399019 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da856bde40 0x55da856c5ec0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:39.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.736+0000 7ff1a52b8740 1 -- 172.21.15.77:0/3639399019 >> 172.21.15.77:0/3639399019 conn(0x55da856ba3d0 msgr2=0x55da856c00b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:39.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.736+0000 7ff1a52b8740 1 -- 172.21.15.77:0/3639399019 shutdown_connections 2024-09-10T09:57:39.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.736+0000 7ff1a52b8740 1 -- 172.21.15.77:0/3639399019 wait complete. 2024-09-10T09:57:39.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.736+0000 7ff1a52b8740 20 client.26021 trim_cache size 0 max 16384 2024-09-10T09:57:39.751 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.750+0000 7ff1a52b8740 1 Processor -- start 2024-09-10T09:57:39.751 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.750+0000 7ff1a52b8740 1 -- start start 2024-09-10T09:57:39.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.750+0000 7ff1a52b8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da856bde40 0x55da856c08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:39.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.750+0000 7ff1a52b8740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da856be760 0x55da856b9380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:39.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.750+0000 7ff1a52b8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da856c9020 0x55da856e0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:39.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.750+0000 7ff1a52b8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff19c1b53a0 con 0x55da856c9020 2024-09-10T09:57:39.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.750+0000 7ff1a52b8740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff19c1b6f70 con 0x55da856be760 2024-09-10T09:57:39.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.750+0000 7ff1a52b8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff19c1b5a70 con 0x55da856bde40 2024-09-10T09:57:39.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.750+0000 7ff19b400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da856bde40 0x55da856c08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:39.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.750+0000 7ff19b400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da856bde40 0x55da856c08a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:37522/0 (socket says 172.21.15.77:37522) 2024-09-10T09:57:39.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.750+0000 7ff19b400640 1 -- 172.21.15.77:0/2321358727 learned_addr learned my addr 172.21.15.77:0/2321358727 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:57:39.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.750+0000 7ff19be00640 1 --2- 172.21.15.77:0/2321358727 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da856c9020 0x55da856e0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:39.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.750+0000 7ff19aa00640 1 --2- 172.21.15.77:0/2321358727 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da856be760 0x55da856b9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:39.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.750+0000 7ff19b400640 1 -- 172.21.15.77:0/2321358727 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da856be760 msgr2=0x55da856b9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:39.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.750+0000 7ff19b400640 1 --2- 172.21.15.77:0/2321358727 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da856be760 0x55da856b9380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:39.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.750+0000 7ff19b400640 1 -- 172.21.15.77:0/2321358727 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da856c9020 msgr2=0x55da856e0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:39.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.750+0000 7ff19b400640 1 --2- 172.21.15.77:0/2321358727 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da856c9020 0x55da856e0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:39.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.750+0000 7ff19b400640 1 -- 172.21.15.77:0/2321358727 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da856cbd50 con 0x55da856bde40 2024-09-10T09:57:39.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.751+0000 7ff19b400640 1 --2- 172.21.15.77:0/2321358727 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da856bde40 0x55da856c08a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff18c004af0 tx=0x7ff18c005de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63b5dd3162c3b1af server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:39.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.751+0000 7ff187e00640 1 -- 172.21.15.77:0/2321358727 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff18c052050 con 0x55da856bde40 2024-09-10T09:57:39.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.751+0000 7ff187e00640 1 -- 172.21.15.77:0/2321358727 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff18c004200 con 0x55da856bde40 2024-09-10T09:57:39.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.751+0000 7ff187e00640 1 -- 172.21.15.77:0/2321358727 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff18c005350 con 0x55da856bde40 2024-09-10T09:57:39.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.752+0000 7ff1a52b8740 1 -- 172.21.15.77:0/2321358727 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da856bde40 msgr2=0x55da856c08a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:39.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.752+0000 7ff1a52b8740 1 --2- 172.21.15.77:0/2321358727 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da856bde40 0x55da856c08a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff18c004af0 tx=0x7ff18c005de0 comp rx=0 tx=0).stop 2024-09-10T09:57:39.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.752+0000 7ff1a52b8740 1 -- 172.21.15.77:0/2321358727 shutdown_connections 2024-09-10T09:57:39.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.752+0000 7ff1a52b8740 1 --2- 172.21.15.77:0/2321358727 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da856c9020 0x55da856e0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:39.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.752+0000 7ff1a52b8740 1 --2- 172.21.15.77:0/2321358727 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da856be760 0x55da856b9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:39.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.752+0000 7ff1a52b8740 1 --2- 172.21.15.77:0/2321358727 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da856bde40 0x55da856c08a0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:39.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.752+0000 7ff1a52b8740 1 -- 172.21.15.77:0/2321358727 >> 172.21.15.77:0/2321358727 conn(0x55da856ba3d0 msgr2=0x55da856b8c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:39.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.752+0000 7ff1a52b8740 1 -- 172.21.15.77:0/2321358727 shutdown_connections 2024-09-10T09:57:39.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.752+0000 7ff1a52b8740 1 -- 172.21.15.77:0/2321358727 wait complete. 2024-09-10T09:57:39.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.752+0000 7ff1a52b8740 1 -- start start 2024-09-10T09:57:39.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.753+0000 7ff1a52b8740 1 Processor -- start 2024-09-10T09:57:39.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.753+0000 7ff1a52b8740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da856bde40 0x55da856d1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:39.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.753+0000 7ff1a52b8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da856be760 0x55da856c2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:39.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.753+0000 7ff1a52b8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da856c9020 0x55da856c3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:39.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.753+0000 7ff1a52b8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff19c1b6810 con 0x55da856c9020 2024-09-10T09:57:39.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.753+0000 7ff1a52b8740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff19c1b6140 con 0x55da856bde40 2024-09-10T09:57:39.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.753+0000 7ff1a52b8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff19c1b3380 con 0x55da856be760 2024-09-10T09:57:39.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.753+0000 7ff1a52b8740 10 client.0 _ll_register_callbacks cb 0x7ff188009a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:57:39.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.753+0000 7ff19aa00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da856be760 0x55da856c2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:39.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.753+0000 7ff19b400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da856bde40 0x55da856d1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:39.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.753+0000 7ff19be00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da856c9020 0x55da856c3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:39.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.753+0000 7ff19aa00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da856be760 0x55da856c2d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:37536/0 (socket says 172.21.15.77:37536) 2024-09-10T09:57:39.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.753+0000 7ff19aa00640 1 -- 172.21.15.77:0/2720029996 learned_addr learned my addr 172.21.15.77:0/2720029996 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:57:39.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.754+0000 7ff19aa00640 1 -- 172.21.15.77:0/2720029996 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da856bde40 msgr2=0x55da856d1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:39.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.754+0000 7ff19aa00640 1 --2- 172.21.15.77:0/2720029996 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da856bde40 0x55da856d1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:39.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.754+0000 7ff19aa00640 1 -- 172.21.15.77:0/2720029996 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da856c9020 msgr2=0x55da856c3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:39.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.754+0000 7ff19aa00640 1 --2- 172.21.15.77:0/2720029996 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da856c9020 0x55da856c3360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:39.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.754+0000 7ff19aa00640 1 -- 172.21.15.77:0/2720029996 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da856ba200 con 0x55da856be760 2024-09-10T09:57:39.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.754+0000 7ff19b400640 1 --2- 172.21.15.77:0/2720029996 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da856bde40 0x55da856d1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:57:39.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.754+0000 7ff19aa00640 1 --2- 172.21.15.77:0/2720029996 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da856be760 0x55da856c2d50 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff188009900 tx=0x7ff188005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6cf246db7884f0a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:39.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.754+0000 7ff185600640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-10T09:57:39.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.754+0000 7ff185600640 1 -- 172.21.15.77:0/2720029996 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff188005020 con 0x55da856be760 2024-09-10T09:57:39.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.754+0000 7ff1a52b8740 1 -- 172.21.15.77:0/2720029996 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff18c004200 con 0x55da856be760 2024-09-10T09:57:39.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.754+0000 7ff185600640 1 -- 172.21.15.77:0/2720029996 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7ff188009040 con 0x55da856be760 2024-09-10T09:57:39.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.754+0000 7ff185600640 1 -- 172.21.15.77:0/2720029996 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff1880051c0 con 0x55da856be760 2024-09-10T09:57:39.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.754+0000 7ff1a52b8740 10 client.25978 fetch_fsmap learned FSMap version 31 2024-09-10T09:57:39.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.754+0000 7ff1a52b8740 1 -- 172.21.15.77:0/2720029996 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55da857438a0 con 0x55da856be760 2024-09-10T09:57:39.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.755+0000 7ff185600640 1 -- 172.21.15.77:0/2720029996 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff18c004200 con 0x55da856be760 2024-09-10T09:57:39.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.755+0000 7ff185600640 1 -- 172.21.15.77:0/2720029996 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7ff18801e670 con 0x55da856be760 2024-09-10T09:57:39.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.755+0000 7ff1a52b8740 10 client.25978 fetch_fsmap finished waiting for FSMap version 31 2024-09-10T09:57:39.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.755+0000 7ff1a52b8740 10 client.25978 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T09:57:39.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.755+0000 7ff1a52b8740 20 client.25978 populate_metadata read hostname 'smithi077' 2024-09-10T09:57:39.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.755+0000 7ff1a52b8740 1 --2- 172.21.15.77:0/2720029996 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x55da85744050 0x55da85746470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:39.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.755+0000 7ff1a52b8740 4 client.25978 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:57:39.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.755+0000 7ff1a52b8740 1 -- 172.21.15.77:0/2720029996 --> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55da8562be60 con 0x55da85744050 2024-09-10T09:57:39.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.755+0000 7ff19b400640 1 --2- 172.21.15.77:0/2720029996 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x55da85744050 0x55da85746470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:39.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.756+0000 7ff19b400640 1 --2- 172.21.15.77:0/2720029996 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x55da85744050 0x55da85746470 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ab7cca8dedbbf424 server_cookie=a881584ce595a74b in_seq=0 out_seq=0 2024-09-10T09:57:39.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.756+0000 7ff185600640 10 client.25978 ms_handle_connect on v2:172.21.15.134:6838/1965117680 2024-09-10T09:57:39.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.756+0000 7ff185600640 1 -- 172.21.15.77:0/2720029996 <== mds.2 v2:172.21.15.134:6838/1965117680 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55da8562be60 con 0x55da85744050 2024-09-10T09:57:39.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.756+0000 7ff185600640 10 client.25978 handle_command_reply: tid=0 2024-09-10T09:57:39.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.756+0000 7ff1a52b8740 1 client.25978 shutdown 2024-09-10T09:57:39.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.756+0000 7ff1a52b8740 1 -- 172.21.15.77:0/2720029996 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da856be760 msgr2=0x55da856c2d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:39.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.756+0000 7ff1a52b8740 1 --2- 172.21.15.77:0/2720029996 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da856be760 0x55da856c2d50 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff188009900 tx=0x7ff188005f90 comp rx=0 tx=0).stop 2024-09-10T09:57:39.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.756+0000 7ff1a52b8740 1 -- 172.21.15.77:0/2720029996 shutdown_connections 2024-09-10T09:57:39.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.756+0000 7ff1a52b8740 1 --2- 172.21.15.77:0/2720029996 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x55da85744050 0x55da85746470 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:39.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.756+0000 7ff1a52b8740 1 --2- 172.21.15.77:0/2720029996 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da856c9020 0x55da856c3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:39.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.756+0000 7ff1a52b8740 1 --2- 172.21.15.77:0/2720029996 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da856be760 0x55da856c2d50 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:39.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.756+0000 7ff1a52b8740 1 --2- 172.21.15.77:0/2720029996 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da856bde40 0x55da856d1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:39.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.756+0000 7ff1a52b8740 1 -- 172.21.15.77:0/2720029996 >> 172.21.15.77:0/2720029996 conn(0x55da856ba3d0 msgr2=0x55da856b9490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:39.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.756+0000 7ff185600640 0 client.25978 ms_handle_reset on v2:172.21.15.134:6838/1965117680 2024-09-10T09:57:39.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.757+0000 7ff1a52b8740 1 -- 172.21.15.77:0/2720029996 shutdown_connections 2024-09-10T09:57:39.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.757+0000 7ff1a52b8740 1 -- 172.21.15.77:0/2720029996 wait complete. 2024-09-10T09:57:39.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.757+0000 7ff1a52b8740 20 client.25978 trim_cache size 0 max 16384 2024-09-10T09:57:39.763 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-10T09:57:39.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.757+0000 7ff1a1200640 1 -- 172.21.15.77:0/3935158655 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ff1800a2b80 msgr2=0x7ff1800a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:39.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.757+0000 7ff1a1200640 1 --2- 172.21.15.77:0/3935158655 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ff1800a2b80 0x7ff1800a5040 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x55da856bb750 tx=0x7ff188013040 comp rx=0 tx=0).stop 2024-09-10T09:57:39.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.757+0000 7ff1a1200640 1 -- 172.21.15.77:0/3935158655 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff19c112870 msgr2=0x7ff19c19e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:39.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.757+0000 7ff1a1200640 1 --2- 172.21.15.77:0/3935158655 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff19c112870 0x7ff19c19e780 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff19400eb10 tx=0x7ff19400c2d0 comp rx=0 tx=0).stop 2024-09-10T09:57:39.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.757+0000 7ff1a1200640 1 -- 172.21.15.77:0/3935158655 shutdown_connections 2024-09-10T09:57:39.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.757+0000 7ff1a1200640 1 --2- 172.21.15.77:0/3935158655 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ff1800a2b80 0x7ff1800a5040 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:39.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.757+0000 7ff1a1200640 1 --2- 172.21.15.77:0/3935158655 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff19c112870 0x7ff19c19e780 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:39.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.757+0000 7ff1a1200640 1 --2- 172.21.15.77:0/3935158655 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff19c077040 0x7ff19c1a4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:39.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.757+0000 7ff1a1200640 1 --2- 172.21.15.77:0/3935158655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff19c076720 0x7ff19c1a4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:39.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.757+0000 7ff1a1200640 1 -- 172.21.15.77:0/3935158655 >> 172.21.15.77:0/3935158655 conn(0x7ff19c0ff910 msgr2=0x7ff19c101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:39.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.758+0000 7ff1a1200640 1 -- 172.21.15.77:0/3935158655 shutdown_connections 2024-09-10T09:57:39.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:39.758+0000 7ff1a1200640 1 -- 172.21.15.77:0/3935158655 wait complete. 2024-09-10T09:57:40.642 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T09:57:41.020 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:57:41.156 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:57:40 smithi077 ceph-mon[30319]: pgmap v466: 129 pgs: 129 active+clean; 23 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 894 KiB/s wr, 51 op/s 2024-09-10T09:57:41.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:57:40 smithi120 ceph-mon[34770]: pgmap v466: 129 pgs: 129 active+clean; 23 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 894 KiB/s wr, 51 op/s 2024-09-10T09:57:41.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:57:40 smithi134 ceph-mon[35674]: pgmap v466: 129 pgs: 129 active+clean; 23 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 894 KiB/s wr, 51 op/s 2024-09-10T09:57:41.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.555+0000 7f0727200640 1 -- 172.21.15.77:0/401314977 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0720103e30 msgr2=0x7f0720104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:41.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.555+0000 7f0727200640 1 --2- 172.21.15.77:0/401314977 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0720103e30 0x7f0720104210 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f0714009900 tx=0x7f071402a3f0 comp rx=0 tx=0).stop 2024-09-10T09:57:41.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.555+0000 7f0727200640 1 -- 172.21.15.77:0/401314977 shutdown_connections 2024-09-10T09:57:41.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.555+0000 7f0727200640 1 --2- 172.21.15.77:0/401314977 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0720108f60 0x7f0720110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:41.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.555+0000 7f0727200640 1 --2- 172.21.15.77:0/401314977 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f07201047e0 0x7f0720108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:41.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.555+0000 7f0727200640 1 --2- 172.21.15.77:0/401314977 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0720103e30 0x7f0720104210 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:41.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.555+0000 7f0727200640 1 -- 172.21.15.77:0/401314977 >> 172.21.15.77:0/401314977 conn(0x7f07200ff8d0 msgr2=0x7f0720101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:41.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.555+0000 7f0727200640 1 -- 172.21.15.77:0/401314977 shutdown_connections 2024-09-10T09:57:41.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.555+0000 7f0727200640 1 -- 172.21.15.77:0/401314977 wait complete. 2024-09-10T09:57:41.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.556+0000 7f0727200640 1 Processor -- start 2024-09-10T09:57:41.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.556+0000 7f0727200640 1 -- start start 2024-09-10T09:57:41.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.556+0000 7f0727200640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0720103e30 0x7f072019a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:41.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.556+0000 7f0727200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f07201047e0 0x7f072019a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:41.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.556+0000 7f0727200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0720108f60 0x7f072019ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:41.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.556+0000 7f0727200640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0720113f80 con 0x7f0720108f60 2024-09-10T09:57:41.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.556+0000 7f0727200640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0720113e00 con 0x7f0720103e30 2024-09-10T09:57:41.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.556+0000 7f0727200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0720114100 con 0x7f07201047e0 2024-09-10T09:57:41.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.557+0000 7f0725e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0720103e30 0x7f072019a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:41.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.557+0000 7f0725e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0720103e30 0x7f072019a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:54012/0 (socket says 172.21.15.77:54012) 2024-09-10T09:57:41.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.557+0000 7f0725e00640 1 -- 172.21.15.77:0/3644987239 learned_addr learned my addr 172.21.15.77:0/3644987239 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:57:41.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.557+0000 7f0725400640 1 --2- 172.21.15.77:0/3644987239 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f07201047e0 0x7f072019a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:41.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.557+0000 7f0726800640 1 --2- 172.21.15.77:0/3644987239 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0720108f60 0x7f072019ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:41.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.557+0000 7f0725e00640 1 -- 172.21.15.77:0/3644987239 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f07201047e0 msgr2=0x7f072019a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:41.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.557+0000 7f0725e00640 1 --2- 172.21.15.77:0/3644987239 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f07201047e0 0x7f072019a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:41.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.557+0000 7f0725e00640 1 -- 172.21.15.77:0/3644987239 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0720108f60 msgr2=0x7f072019ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:41.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.557+0000 7f0725e00640 1 --2- 172.21.15.77:0/3644987239 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0720108f60 0x7f072019ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:41.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.557+0000 7f0725e00640 1 -- 172.21.15.77:0/3644987239 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f072019f430 con 0x7f0720103e30 2024-09-10T09:57:41.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.558+0000 7f0725e00640 1 --2- 172.21.15.77:0/3644987239 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0720103e30 0x7f072019a480 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0714006010 tx=0x7f07140042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:41.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.558+0000 7f071aa00640 1 -- 172.21.15.77:0/3644987239 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f071403f030 con 0x7f0720103e30 2024-09-10T09:57:41.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.558+0000 7f0727200640 1 -- 172.21.15.77:0/3644987239 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f072019f6c0 con 0x7f0720103e30 2024-09-10T09:57:41.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.558+0000 7f071aa00640 1 -- 172.21.15.77:0/3644987239 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0714003ce0 con 0x7f0720103e30 2024-09-10T09:57:41.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.558+0000 7f071aa00640 1 -- 172.21.15.77:0/3644987239 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0714035070 con 0x7f0720103e30 2024-09-10T09:57:41.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.558+0000 7f0727200640 1 -- 172.21.15.77:0/3644987239 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07201a7000 con 0x7f0720103e30 2024-09-10T09:57:41.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.561+0000 7f072b249740 1 Processor -- start 2024-09-10T09:57:41.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.561+0000 7f072b249740 1 -- start start 2024-09-10T09:57:41.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.561+0000 7f071aa00640 1 -- 172.21.15.77:0/3644987239 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0714042030 con 0x7f0720103e30 2024-09-10T09:57:41.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.561+0000 7f072b249740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b8b29a3e40 0x55b8b29a4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:41.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.561+0000 7f072b249740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b8b29a4760 0x55b8b29aeae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:41.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.561+0000 7f072b249740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b8b29af020 0x55b8b29b1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:41.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.561+0000 7f072b249740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55b8b290a3e0 con 0x55b8b29a4760 2024-09-10T09:57:41.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.561+0000 7f072b249740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b8b28b1d80 con 0x55b8b29a3e40 2024-09-10T09:57:41.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.561+0000 7f072b249740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b8b29063a0 con 0x55b8b29af020 2024-09-10T09:57:41.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.562+0000 7f0726800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b8b29a3e40 0x55b8b29a4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:41.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.562+0000 7f0726800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b8b29a3e40 0x55b8b29a4220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:54026/0 (socket says 172.21.15.77:54026) 2024-09-10T09:57:41.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.562+0000 7f0726800640 1 -- 172.21.15.77:0/4194747218 learned_addr learned my addr 172.21.15.77:0/4194747218 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:57:41.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.562+0000 7f0725e00640 1 --2- 172.21.15.77:0/4194747218 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b8b29a4760 0x55b8b29aeae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:41.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.562+0000 7f0725400640 1 --2- 172.21.15.77:0/4194747218 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b8b29af020 0x55b8b29b1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:41.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.562+0000 7f0726800640 1 -- 172.21.15.77:0/4194747218 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b8b29af020 msgr2=0x55b8b29b1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:41.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.562+0000 7f0726800640 1 --2- 172.21.15.77:0/4194747218 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b8b29af020 0x55b8b29b1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:41.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.562+0000 7f0726800640 1 -- 172.21.15.77:0/4194747218 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b8b29a4760 msgr2=0x55b8b29aeae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:41.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.562+0000 7f0726800640 1 --2- 172.21.15.77:0/4194747218 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b8b29a4760 0x55b8b29aeae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:41.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.562+0000 7f0726800640 1 -- 172.21.15.77:0/4194747218 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8b28d0ae0 con 0x55b8b29a3e40 2024-09-10T09:57:41.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.562+0000 7f0725400640 1 --2- 172.21.15.77:0/4194747218 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b8b29af020 0x55b8b29b1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:57:41.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.563+0000 7f0726800640 1 --2- 172.21.15.77:0/4194747218 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b8b29a3e40 0x55b8b29a4220 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f071c009e60 tx=0x7f071c010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93e4b9014acba22e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:41.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.563+0000 7f070f000640 1 -- 172.21.15.77:0/4194747218 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f071c016070 con 0x55b8b29a3e40 2024-09-10T09:57:41.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.563+0000 7f071aa00640 1 --2- 172.21.15.77:0/3644987239 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f07080a2b80 0x7f07080a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:41.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.563+0000 7f070f000640 1 -- 172.21.15.77:0/4194747218 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f071c004d20 con 0x55b8b29a3e40 2024-09-10T09:57:41.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.563+0000 7f070f000640 1 -- 172.21.15.77:0/4194747218 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f071c012830 con 0x55b8b29a3e40 2024-09-10T09:57:41.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.563+0000 7f071aa00640 1 -- 172.21.15.77:0/3644987239 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f07140efa70 con 0x7f0720103e30 2024-09-10T09:57:41.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.563+0000 7f0725e00640 1 --2- 172.21.15.77:0/3644987239 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f07080a2b80 0x7f07080a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:41.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.564+0000 7f072b249740 1 -- 172.21.15.77:0/4194747218 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b8b29a3e40 msgr2=0x55b8b29a4220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:41.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.564+0000 7f072b249740 1 --2- 172.21.15.77:0/4194747218 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b8b29a3e40 0x55b8b29a4220 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f071c009e60 tx=0x7f071c010740 comp rx=0 tx=0).stop 2024-09-10T09:57:41.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.564+0000 7f072b249740 1 -- 172.21.15.77:0/4194747218 shutdown_connections 2024-09-10T09:57:41.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.564+0000 7f072b249740 1 --2- 172.21.15.77:0/4194747218 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b8b29af020 0x55b8b29b1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:41.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.564+0000 7f072b249740 1 --2- 172.21.15.77:0/4194747218 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b8b29a4760 0x55b8b29aeae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:41.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.564+0000 7f072b249740 1 --2- 172.21.15.77:0/4194747218 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b8b29a3e40 0x55b8b29a4220 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:41.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.564+0000 7f072b249740 1 -- 172.21.15.77:0/4194747218 >> 172.21.15.77:0/4194747218 conn(0x55b8b29a03d0 msgr2=0x55b8b29a07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:41.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.564+0000 7f0725e00640 1 --2- 172.21.15.77:0/3644987239 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f07080a2b80 0x7f07080a5040 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0714005ab0 tx=0x7f07140f5510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:41.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.564+0000 7f072b249740 1 -- 172.21.15.77:0/4194747218 shutdown_connections 2024-09-10T09:57:41.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.564+0000 7f072b249740 1 -- 172.21.15.77:0/4194747218 wait complete. 2024-09-10T09:57:41.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.564+0000 7f072b249740 1 -- start start 2024-09-10T09:57:41.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.565+0000 7f072b249740 1 Processor -- start 2024-09-10T09:57:41.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.565+0000 7f072b249740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b8b29a3e40 0x55b8b29abef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:41.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.565+0000 7f072b249740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b8b29a4760 0x55b8b29ac430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:41.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.565+0000 7f072b249740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b8b29af020 0x55b8b29ac970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:41.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.565+0000 7f072b249740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55b8b29b1bf0 con 0x55b8b29af020 2024-09-10T09:57:41.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.565+0000 7f072b249740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b8b28c50f0 con 0x55b8b29a4760 2024-09-10T09:57:41.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.565+0000 7f072b249740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b8b29b1d70 con 0x55b8b29a3e40 2024-09-10T09:57:41.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.565+0000 7f072b249740 10 client.0 _ll_register_callbacks cb 0x55b8b28acbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:57:41.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.565+0000 7f0725400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b8b29a4760 0x55b8b29ac430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:41.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.565+0000 7f0725400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b8b29a4760 0x55b8b29ac430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:54038/0 (socket says 172.21.15.77:54038) 2024-09-10T09:57:41.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.565+0000 7f0725400640 1 -- 172.21.15.77:0/4019983611 learned_addr learned my addr 172.21.15.77:0/4019983611 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:57:41.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.565+0000 7f0726800640 1 --2- 172.21.15.77:0/4019983611 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b8b29a3e40 0x55b8b29abef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:41.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.566+0000 7f0726800640 1 --2- 172.21.15.77:0/4019983611 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b8b29af020 0x55b8b29ac970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:41.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.566+0000 7f0725400640 1 -- 172.21.15.77:0/4019983611 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b8b29a3e40 msgr2=0x55b8b29abef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:41.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.566+0000 7f0725400640 1 --2- 172.21.15.77:0/4019983611 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b8b29a3e40 0x55b8b29abef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:41.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.566+0000 7f0725400640 1 -- 172.21.15.77:0/4019983611 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b8b29af020 msgr2=0x55b8b29ac970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:41.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.566+0000 7f0725400640 1 --2- 172.21.15.77:0/4019983611 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b8b29af020 0x55b8b29ac970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:41.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.566+0000 7f0725400640 1 -- 172.21.15.77:0/4019983611 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8b267b3c0 con 0x55b8b29a4760 2024-09-10T09:57:41.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.566+0000 7f0726800640 1 --2- 172.21.15.77:0/4019983611 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b8b29af020 0x55b8b29ac970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:57:41.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.566+0000 7f0725400640 1 --2- 172.21.15.77:0/4019983611 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b8b29a4760 0x55b8b29ac430 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0710012700 tx=0x7f0710012b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e89b0e9e9551160c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:41.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.566+0000 7f0703400640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T09:57:41.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.566+0000 7f0703400640 1 -- 172.21.15.77:0/4019983611 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0710004050 con 0x55b8b29a4760 2024-09-10T09:57:41.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.566+0000 7f0703400640 1 -- 172.21.15.77:0/4019983611 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f07100041f0 con 0x55b8b29a4760 2024-09-10T09:57:41.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.566+0000 7f072b249740 1 -- 172.21.15.77:0/4019983611 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f071c004d20 con 0x55b8b29a4760 2024-09-10T09:57:41.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.567+0000 7f0703400640 1 -- 172.21.15.77:0/4019983611 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0710025050 con 0x55b8b29a4760 2024-09-10T09:57:41.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.567+0000 7f072b249740 10 client.26039 fetch_fsmap learned FSMap version 32 2024-09-10T09:57:41.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.567+0000 7f072b249740 1 -- 172.21.15.77:0/4019983611 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b8b29bdc70 con 0x55b8b29a4760 2024-09-10T09:57:41.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.567+0000 7f0703400640 1 -- 172.21.15.77:0/4019983611 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f071c004d20 con 0x55b8b29a4760 2024-09-10T09:57:41.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.567+0000 7f0703400640 1 -- 172.21.15.77:0/4019983611 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f0710003790 con 0x55b8b29a4760 2024-09-10T09:57:41.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.567+0000 7f072b249740 10 client.26039 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T09:57:41.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.567+0000 7f072b249740 10 client.26039 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-10T09:57:41.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.567+0000 7f072b249740 20 client.26039 populate_metadata read hostname 'smithi077' 2024-09-10T09:57:41.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.567+0000 7f072b249740 1 --2- 172.21.15.77:0/4019983611 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x55b8b29bf180 0x55b8b29c15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:41.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.567+0000 7f072b249740 4 client.26039 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:57:41.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.567+0000 7f072b249740 1 -- 172.21.15.77:0/4019983611 --> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b8b2917950 con 0x55b8b29bf180 2024-09-10T09:57:41.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.568+0000 7f0726800640 1 --2- 172.21.15.77:0/4019983611 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x55b8b29bf180 0x55b8b29c15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:41.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.568+0000 7f0726800640 1 --2- 172.21.15.77:0/4019983611 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x55b8b29bf180 0x55b8b29c15a0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9b82f4b377150172 server_cookie=2870582d2d9a377 in_seq=0 out_seq=0 2024-09-10T09:57:41.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.568+0000 7f0703400640 10 client.26039 ms_handle_connect on v2:172.21.15.120:6836/3033280709 2024-09-10T09:57:41.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.569+0000 7f0703400640 1 -- 172.21.15.77:0/4019983611 <== mds.1 v2:172.21.15.120:6836/3033280709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b8b2917950 con 0x55b8b29bf180 2024-09-10T09:57:41.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.569+0000 7f0703400640 10 client.26039 handle_command_reply: tid=0 2024-09-10T09:57:41.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.570+0000 7f072b249740 1 client.26039 shutdown 2024-09-10T09:57:41.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.570+0000 7f072b249740 1 -- 172.21.15.77:0/4019983611 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b8b29a4760 msgr2=0x55b8b29ac430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:41.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.570+0000 7f072b249740 1 --2- 172.21.15.77:0/4019983611 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b8b29a4760 0x55b8b29ac430 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0710012700 tx=0x7f0710012b00 comp rx=0 tx=0).stop 2024-09-10T09:57:41.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.570+0000 7f072b249740 1 -- 172.21.15.77:0/4019983611 shutdown_connections 2024-09-10T09:57:41.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.570+0000 7f072b249740 1 --2- 172.21.15.77:0/4019983611 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x55b8b29bf180 0x55b8b29c15a0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:41.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.570+0000 7f072b249740 1 --2- 172.21.15.77:0/4019983611 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b8b29af020 0x55b8b29ac970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:41.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.570+0000 7f072b249740 1 --2- 172.21.15.77:0/4019983611 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b8b29a4760 0x55b8b29ac430 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:41.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.570+0000 7f072b249740 1 --2- 172.21.15.77:0/4019983611 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b8b29a3e40 0x55b8b29abef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:41.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.570+0000 7f072b249740 1 -- 172.21.15.77:0/4019983611 >> 172.21.15.77:0/4019983611 conn(0x55b8b29a03d0 msgr2=0x55b8b29a6100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:41.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.570+0000 7f0703400640 0 client.26039 ms_handle_reset on v2:172.21.15.120:6836/3033280709 2024-09-10T09:57:41.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.570+0000 7f072b249740 1 -- 172.21.15.77:0/4019983611 shutdown_connections 2024-09-10T09:57:41.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.570+0000 7f072b249740 1 -- 172.21.15.77:0/4019983611 wait complete. 2024-09-10T09:57:41.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.570+0000 7f072b249740 20 client.26039 trim_cache size 0 max 16384 2024-09-10T09:57:41.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.585+0000 7f072b249740 1 Processor -- start 2024-09-10T09:57:41.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.585+0000 7f072b249740 1 -- start start 2024-09-10T09:57:41.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.585+0000 7f072b249740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b8b29a3e40 0x55b8b29a7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:41.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.585+0000 7f072b249740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b8b29a4760 0x55b8b29ac7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:41.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.585+0000 7f072b249740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b8b29af020 0x55b8b29ade60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:41.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.585+0000 7f072b249740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f07201b0ec0 con 0x55b8b29a4760 2024-09-10T09:57:41.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.585+0000 7f072b249740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f07201b2ac0 con 0x55b8b29a3e40 2024-09-10T09:57:41.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.585+0000 7f072b249740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f07201b1590 con 0x55b8b29af020 2024-09-10T09:57:41.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.586+0000 7f0726800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b8b29a3e40 0x55b8b29a7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:41.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.586+0000 7f0726800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b8b29a3e40 0x55b8b29a7660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:54054/0 (socket says 172.21.15.77:54054) 2024-09-10T09:57:41.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.586+0000 7f0726800640 1 -- 172.21.15.77:0/2784748513 learned_addr learned my addr 172.21.15.77:0/2784748513 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:57:41.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.586+0000 7f0725400640 1 --2- 172.21.15.77:0/2784748513 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b8b29a4760 0x55b8b29ac7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:41.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.586+0000 7f0726800640 1 --2- 172.21.15.77:0/2784748513 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b8b29af020 0x55b8b29ade60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:41.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.586+0000 7f0726800640 1 -- 172.21.15.77:0/2784748513 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b8b29af020 msgr2=0x55b8b29ade60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:41.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.586+0000 7f0726800640 1 --2- 172.21.15.77:0/2784748513 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b8b29af020 0x55b8b29ade60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:41.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.586+0000 7f0726800640 1 -- 172.21.15.77:0/2784748513 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b8b29a4760 msgr2=0x55b8b29ac7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:41.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.586+0000 7f0726800640 1 --2- 172.21.15.77:0/2784748513 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b8b29a4760 0x55b8b29ac7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:41.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.586+0000 7f0726800640 1 -- 172.21.15.77:0/2784748513 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8b29a0200 con 0x55b8b29a3e40 2024-09-10T09:57:41.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.586+0000 7f0725400640 1 --2- 172.21.15.77:0/2784748513 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b8b29a4760 0x55b8b29ac7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:57:41.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.586+0000 7f0726800640 1 --2- 172.21.15.77:0/2784748513 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b8b29a3e40 0x55b8b29a7660 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f071c010780 tx=0x7f071c000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a509fe6cd217c757 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:41.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.587+0000 7f070e600640 1 -- 172.21.15.77:0/2784748513 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f071c024040 con 0x55b8b29a3e40 2024-09-10T09:57:41.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.587+0000 7f070e600640 1 -- 172.21.15.77:0/2784748513 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f071c012c40 con 0x55b8b29a3e40 2024-09-10T09:57:41.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.587+0000 7f070e600640 1 -- 172.21.15.77:0/2784748513 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f071c016040 con 0x55b8b29a3e40 2024-09-10T09:57:41.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.587+0000 7f072b249740 1 -- 172.21.15.77:0/2784748513 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b8b29a3e40 msgr2=0x55b8b29a7660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:41.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.587+0000 7f072b249740 1 --2- 172.21.15.77:0/2784748513 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b8b29a3e40 0x55b8b29a7660 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f071c010780 tx=0x7f071c000f40 comp rx=0 tx=0).stop 2024-09-10T09:57:41.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.587+0000 7f072b249740 1 -- 172.21.15.77:0/2784748513 shutdown_connections 2024-09-10T09:57:41.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.587+0000 7f072b249740 1 --2- 172.21.15.77:0/2784748513 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b8b29af020 0x55b8b29ade60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:41.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.587+0000 7f072b249740 1 --2- 172.21.15.77:0/2784748513 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b8b29a4760 0x55b8b29ac7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:41.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.587+0000 7f072b249740 1 --2- 172.21.15.77:0/2784748513 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b8b29a3e40 0x55b8b29a7660 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:41.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.587+0000 7f072b249740 1 -- 172.21.15.77:0/2784748513 >> 172.21.15.77:0/2784748513 conn(0x55b8b29a03d0 msgr2=0x55b8b29ad750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:41.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.587+0000 7f072b249740 1 -- 172.21.15.77:0/2784748513 shutdown_connections 2024-09-10T09:57:41.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.587+0000 7f072b249740 1 -- 172.21.15.77:0/2784748513 wait complete. 2024-09-10T09:57:41.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.588+0000 7f072b249740 1 -- start start 2024-09-10T09:57:41.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.588+0000 7f072b249740 1 Processor -- start 2024-09-10T09:57:41.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.588+0000 7f072b249740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b8b29a3e40 0x55b8b29b7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:41.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.588+0000 7f072b249740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b8b29a4760 0x55b8b29b2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:41.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.588+0000 7f072b249740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b8b29af020 0x55b8b29b2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:41.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.588+0000 7f072b249740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f07201b2330 con 0x55b8b29a4760 2024-09-10T09:57:41.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.588+0000 7f072b249740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f07201b1c60 con 0x55b8b29af020 2024-09-10T09:57:41.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.588+0000 7f072b249740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f07201aeea0 con 0x55b8b29a3e40 2024-09-10T09:57:41.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.589+0000 7f072b249740 10 client.0 _ll_register_callbacks cb 0x7f0710009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:57:41.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.589+0000 7f0726800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b8b29af020 0x55b8b29b2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:41.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.589+0000 7f0726800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b8b29af020 0x55b8b29b2dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:54058/0 (socket says 172.21.15.77:54058) 2024-09-10T09:57:41.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.589+0000 7f0726800640 1 -- 172.21.15.77:0/232502525 learned_addr learned my addr 172.21.15.77:0/232502525 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:57:41.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.589+0000 7f0726800640 1 --2- 172.21.15.77:0/232502525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b8b29a3e40 0x55b8b29b7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:41.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.589+0000 7f0725400640 1 --2- 172.21.15.77:0/232502525 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b8b29a4760 0x55b8b29b2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:41.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.589+0000 7f0726800640 1 -- 172.21.15.77:0/232502525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b8b29a3e40 msgr2=0x55b8b29b7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:41.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.589+0000 7f0726800640 1 --2- 172.21.15.77:0/232502525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b8b29a3e40 0x55b8b29b7560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:41.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.589+0000 7f0726800640 1 -- 172.21.15.77:0/232502525 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b8b29a4760 msgr2=0x55b8b29b2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:41.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.589+0000 7f0726800640 1 --2- 172.21.15.77:0/232502525 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b8b29a4760 0x55b8b29b2890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:41.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.589+0000 7f0726800640 1 -- 172.21.15.77:0/232502525 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8b29b3410 con 0x55b8b29af020 2024-09-10T09:57:41.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.589+0000 7f0726800640 1 --2- 172.21.15.77:0/232502525 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b8b29af020 0x55b8b29b2dd0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x55b8b29a36d0 tx=0x7f071c028550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c73a03c8c6b80e4d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:41.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.590+0000 7f0703400640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T09:57:41.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.590+0000 7f0703400640 1 -- 172.21.15.77:0/232502525 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f071c03a060 con 0x55b8b29af020 2024-09-10T09:57:41.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.590+0000 7f072b249740 1 -- 172.21.15.77:0/232502525 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f071c012c40 con 0x55b8b29af020 2024-09-10T09:57:41.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.590+0000 7f0703400640 1 -- 172.21.15.77:0/232502525 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f071c012650 con 0x55b8b29af020 2024-09-10T09:57:41.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.590+0000 7f0703400640 1 -- 172.21.15.77:0/232502525 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f071c016040 con 0x55b8b29af020 2024-09-10T09:57:41.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.590+0000 7f072b249740 10 client.26045 fetch_fsmap learned FSMap version 32 2024-09-10T09:57:41.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.590+0000 7f072b249740 1 -- 172.21.15.77:0/232502525 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b8b2a29c70 con 0x55b8b29af020 2024-09-10T09:57:41.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.590+0000 7f0703400640 1 -- 172.21.15.77:0/232502525 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f071c012c40 con 0x55b8b29af020 2024-09-10T09:57:41.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.590+0000 7f0703400640 1 -- 172.21.15.77:0/232502525 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f071c031070 con 0x55b8b29af020 2024-09-10T09:57:41.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.591+0000 7f072b249740 10 client.26045 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T09:57:41.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.591+0000 7f072b249740 10 client.26045 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-10T09:57:41.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.591+0000 7f072b249740 20 client.26045 populate_metadata read hostname 'smithi077' 2024-09-10T09:57:41.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.591+0000 7f072b249740 1 --2- 172.21.15.77:0/232502525 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x55b8b2a2a0f0 0x55b8b2a2c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:41.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.591+0000 7f072b249740 4 client.26045 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:57:41.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.591+0000 7f072b249740 1 -- 172.21.15.77:0/232502525 --> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b8b2911e60 con 0x55b8b2a2a0f0 2024-09-10T09:57:41.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.591+0000 7f0725400640 1 --2- 172.21.15.77:0/232502525 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x55b8b2a2a0f0 0x55b8b2a2c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:41.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.591+0000 7f0725400640 1 --2- 172.21.15.77:0/232502525 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x55b8b2a2a0f0 0x55b8b2a2c4e0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cbf8b019b8035595 server_cookie=2eb5949f6b8b47d1 in_seq=0 out_seq=0 2024-09-10T09:57:41.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.591+0000 7f0703400640 10 client.26045 ms_handle_connect on v2:172.21.15.120:6836/3033280709 2024-09-10T09:57:41.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.592+0000 7f0703400640 1 -- 172.21.15.77:0/232502525 <== mds.1 v2:172.21.15.120:6836/3033280709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b8b2911e60 con 0x55b8b2a2a0f0 2024-09-10T09:57:41.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.592+0000 7f0703400640 10 client.26045 handle_command_reply: tid=0 2024-09-10T09:57:41.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.592+0000 7f072b249740 1 client.26045 shutdown 2024-09-10T09:57:41.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.592+0000 7f072b249740 1 -- 172.21.15.77:0/232502525 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b8b29af020 msgr2=0x55b8b29b2dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:41.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.592+0000 7f072b249740 1 --2- 172.21.15.77:0/232502525 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b8b29af020 0x55b8b29b2dd0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x55b8b29a36d0 tx=0x7f071c028550 comp rx=0 tx=0).stop 2024-09-10T09:57:41.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.592+0000 7f072b249740 1 -- 172.21.15.77:0/232502525 shutdown_connections 2024-09-10T09:57:41.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.592+0000 7f072b249740 1 --2- 172.21.15.77:0/232502525 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x55b8b2a2a0f0 0x55b8b2a2c4e0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:41.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.592+0000 7f072b249740 1 --2- 172.21.15.77:0/232502525 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b8b29af020 0x55b8b29b2dd0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:41.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.592+0000 7f072b249740 1 --2- 172.21.15.77:0/232502525 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b8b29a4760 0x55b8b29b2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:41.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.592+0000 7f072b249740 1 --2- 172.21.15.77:0/232502525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b8b29a3e40 0x55b8b29b7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:41.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.592+0000 7f0703400640 0 client.26045 ms_handle_reset on v2:172.21.15.120:6836/3033280709 2024-09-10T09:57:41.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.592+0000 7f072b249740 1 -- 172.21.15.77:0/232502525 >> 172.21.15.77:0/232502525 conn(0x55b8b29a03d0 msgr2=0x55b8b29a60b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:41.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.592+0000 7f072b249740 1 -- 172.21.15.77:0/232502525 shutdown_connections 2024-09-10T09:57:41.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.592+0000 7f072b249740 1 -- 172.21.15.77:0/232502525 wait complete. 2024-09-10T09:57:41.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.592+0000 7f072b249740 20 client.26045 trim_cache size 0 max 16384 2024-09-10T09:57:41.600 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-10T09:57:41.600 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.593+0000 7f0727200640 1 -- 172.21.15.77:0/3644987239 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f07080a2b80 msgr2=0x7f07080a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:41.601 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.593+0000 7f0727200640 1 --2- 172.21.15.77:0/3644987239 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f07080a2b80 0x7f07080a5040 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0714005ab0 tx=0x7f07140f5510 comp rx=0 tx=0).stop 2024-09-10T09:57:41.601 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.593+0000 7f0727200640 1 -- 172.21.15.77:0/3644987239 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0720103e30 msgr2=0x7f072019a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:41.601 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.593+0000 7f0727200640 1 --2- 172.21.15.77:0/3644987239 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0720103e30 0x7f072019a480 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0714006010 tx=0x7f07140042f0 comp rx=0 tx=0).stop 2024-09-10T09:57:41.601 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.593+0000 7f0727200640 1 -- 172.21.15.77:0/3644987239 shutdown_connections 2024-09-10T09:57:41.601 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.593+0000 7f0727200640 1 --2- 172.21.15.77:0/3644987239 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f07080a2b80 0x7f07080a5040 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:41.601 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.593+0000 7f0727200640 1 --2- 172.21.15.77:0/3644987239 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0720108f60 0x7f072019ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:41.601 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.593+0000 7f0727200640 1 --2- 172.21.15.77:0/3644987239 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f07201047e0 0x7f072019a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:41.601 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.593+0000 7f0727200640 1 --2- 172.21.15.77:0/3644987239 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0720103e30 0x7f072019a480 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:41.601 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.593+0000 7f0727200640 1 -- 172.21.15.77:0/3644987239 >> 172.21.15.77:0/3644987239 conn(0x7f07200ff8d0 msgr2=0x7f0720100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:41.601 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.593+0000 7f0727200640 1 -- 172.21.15.77:0/3644987239 shutdown_connections 2024-09-10T09:57:41.601 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:41.593+0000 7f0727200640 1 -- 172.21.15.77:0/3644987239 wait complete. 2024-09-10T09:57:42.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:57:41 smithi120 ceph-mon[34770]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T09:57:42.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:57:41 smithi077 ceph-mon[30319]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T09:57:42.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:57:41 smithi134 ceph-mon[35674]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T09:57:42.503 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T09:57:42.890 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:57:43.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:57:42 smithi120 ceph-mon[34770]: pgmap v467: 129 pgs: 129 active+clean; 24 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 611 KiB/s rd, 854 KiB/s wr, 45 op/s 2024-09-10T09:57:43.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:57:42 smithi077 ceph-mon[30319]: pgmap v467: 129 pgs: 129 active+clean; 24 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 611 KiB/s rd, 854 KiB/s wr, 45 op/s 2024-09-10T09:57:43.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:57:42 smithi134 ceph-mon[35674]: pgmap v467: 129 pgs: 129 active+clean; 24 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 611 KiB/s rd, 854 KiB/s wr, 45 op/s 2024-09-10T09:57:43.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.465+0000 7fa9dd000640 1 -- 172.21.15.77:0/264242519 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa9d8101de0 msgr2=0x7fa9d81021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:43.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.465+0000 7fa9dd000640 1 --2- 172.21.15.77:0/264242519 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa9d8101de0 0x7fa9d81021c0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fa9c8009ca0 tx=0x7fa9c802a8c0 comp rx=0 tx=0).stop 2024-09-10T09:57:43.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.465+0000 7fa9dd000640 1 -- 172.21.15.77:0/264242519 shutdown_connections 2024-09-10T09:57:43.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.465+0000 7fa9dd000640 1 --2- 172.21.15.77:0/264242519 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa9d8106f10 0x7fa9d810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:43.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.465+0000 7fa9dd000640 1 --2- 172.21.15.77:0/264242519 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa9d8102790 0x7fa9d81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:43.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.465+0000 7fa9dd000640 1 --2- 172.21.15.77:0/264242519 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa9d8101de0 0x7fa9d81021c0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:43.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.465+0000 7fa9dd000640 1 -- 172.21.15.77:0/264242519 >> 172.21.15.77:0/264242519 conn(0x7fa9d80fd8c0 msgr2=0x7fa9d80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:43.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.465+0000 7fa9dd000640 1 -- 172.21.15.77:0/264242519 shutdown_connections 2024-09-10T09:57:43.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.465+0000 7fa9dd000640 1 -- 172.21.15.77:0/264242519 wait complete. 2024-09-10T09:57:43.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.466+0000 7fa9dd000640 1 Processor -- start 2024-09-10T09:57:43.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.466+0000 7fa9dd000640 1 -- start start 2024-09-10T09:57:43.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.467+0000 7fa9dd000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa9d8101de0 0x7fa9d80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:43.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.467+0000 7fa9dd000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa9d8102790 0x7fa9d8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:43.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.467+0000 7fa9dd000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa9d8106f10 0x7fa9d8074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:43.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.467+0000 7fa9dd000640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d8111f70 con 0x7fa9d8102790 2024-09-10T09:57:43.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.467+0000 7fa9dd000640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d8111df0 con 0x7fa9d8101de0 2024-09-10T09:57:43.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.467+0000 7fa9dd000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d81120f0 con 0x7fa9d8106f10 2024-09-10T09:57:43.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.467+0000 7fa9d5e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa9d8101de0 0x7fa9d80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:43.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.467+0000 7fa9d5e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa9d8101de0 0x7fa9d80744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:33288/0 (socket says 172.21.15.77:33288) 2024-09-10T09:57:43.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.467+0000 7fa9d5400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa9d8102790 0x7fa9d8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:43.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.467+0000 7fa9d5e00640 1 -- 172.21.15.77:0/848177970 learned_addr learned my addr 172.21.15.77:0/848177970 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:57:43.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.467+0000 7fa9d6800640 1 --2- 172.21.15.77:0/848177970 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa9d8106f10 0x7fa9d8074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:43.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.467+0000 7fa9d5e00640 1 -- 172.21.15.77:0/848177970 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa9d8106f10 msgr2=0x7fa9d8074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:43.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.467+0000 7fa9d5e00640 1 --2- 172.21.15.77:0/848177970 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa9d8106f10 0x7fa9d8074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:43.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.467+0000 7fa9d5e00640 1 -- 172.21.15.77:0/848177970 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa9d8102790 msgr2=0x7fa9d8074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:43.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.467+0000 7fa9d5e00640 1 --2- 172.21.15.77:0/848177970 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa9d8102790 0x7fa9d8074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:43.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.467+0000 7fa9d5e00640 1 -- 172.21.15.77:0/848177970 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9d81a68b0 con 0x7fa9d8101de0 2024-09-10T09:57:43.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.467+0000 7fa9d6800640 1 --2- 172.21.15.77:0/848177970 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa9d8106f10 0x7fa9d8074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:57:43.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.468+0000 7fa9d5e00640 1 --2- 172.21.15.77:0/848177970 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa9d8101de0 0x7fa9d80744f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa9c802ade0 tx=0x7fa9c80319f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:43.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.468+0000 7fa9cea00640 1 -- 172.21.15.77:0/848177970 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c802fa20 con 0x7fa9d8101de0 2024-09-10T09:57:43.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.468+0000 7fa9dd000640 1 -- 172.21.15.77:0/848177970 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9d81a6b40 con 0x7fa9d8101de0 2024-09-10T09:57:43.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.468+0000 7fa9cea00640 1 -- 172.21.15.77:0/848177970 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9c802fbc0 con 0x7fa9d8101de0 2024-09-10T09:57:43.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.468+0000 7fa9cea00640 1 -- 172.21.15.77:0/848177970 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c80466e0 con 0x7fa9d8101de0 2024-09-10T09:57:43.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.468+0000 7fa9dd000640 1 -- 172.21.15.77:0/848177970 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9d81a70d0 con 0x7fa9d8101de0 2024-09-10T09:57:43.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.471+0000 7fa9cea00640 1 -- 172.21.15.77:0/848177970 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa9c8046940 con 0x7fa9d8101de0 2024-09-10T09:57:43.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.471+0000 7fa9dfc01740 1 Processor -- start 2024-09-10T09:57:43.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.472+0000 7fa9dfc01740 1 -- start start 2024-09-10T09:57:43.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.472+0000 7fa9dfc01740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5558329efc70 0x5558329f0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:43.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.472+0000 7fa9dfc01740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5558329f0590 0x5558329fa910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:43.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.472+0000 7fa9dfc01740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5558329fae50 0x5558329fd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:43.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.472+0000 7fa9dfc01740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5558329563e0 con 0x5558329efc70 2024-09-10T09:57:43.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.472+0000 7fa9dfc01740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5558328fdd80 con 0x5558329f0590 2024-09-10T09:57:43.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.472+0000 7fa9dfc01740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5558329523a0 con 0x5558329fae50 2024-09-10T09:57:43.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.472+0000 7fa9d6800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5558329efc70 0x5558329f0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:43.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.472+0000 7fa9d5e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5558329f0590 0x5558329fa910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:43.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.472+0000 7fa9d6800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5558329efc70 0x5558329f0050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52440/0 (socket says 172.21.15.77:52440) 2024-09-10T09:57:43.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.472+0000 7fa9d6800640 1 -- 172.21.15.77:0/2553249618 learned_addr learned my addr 172.21.15.77:0/2553249618 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:57:43.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.473+0000 7fa9d5400640 1 --2- 172.21.15.77:0/2553249618 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5558329fae50 0x5558329fd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:43.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.473+0000 7fa9d5e00640 1 -- 172.21.15.77:0/2553249618 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5558329fae50 msgr2=0x5558329fd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:43.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.473+0000 7fa9d5e00640 1 --2- 172.21.15.77:0/2553249618 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5558329fae50 0x5558329fd240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:43.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.473+0000 7fa9d5e00640 1 -- 172.21.15.77:0/2553249618 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5558329efc70 msgr2=0x5558329f0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:43.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.473+0000 7fa9d5e00640 1 --2- 172.21.15.77:0/2553249618 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5558329efc70 0x5558329f0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:43.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.473+0000 7fa9d5e00640 1 -- 172.21.15.77:0/2553249618 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55583291cae0 con 0x5558329f0590 2024-09-10T09:57:43.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.473+0000 7fa9cea00640 1 --2- 172.21.15.77:0/848177970 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fa9bc0a2b80 0x7fa9bc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:43.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.473+0000 7fa9d5e00640 1 --2- 172.21.15.77:0/2553249618 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5558329f0590 0x5558329fa910 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa9c80edf20 tx=0x7fa9c80f8670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=807d05769df18718 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:43.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.473+0000 7fa9c3400640 1 -- 172.21.15.77:0/2553249618 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c80ff070 con 0x5558329f0590 2024-09-10T09:57:43.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.473+0000 7fa9c3400640 1 -- 172.21.15.77:0/2553249618 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9c80bd3e0 con 0x5558329f0590 2024-09-10T09:57:43.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.473+0000 7fa9c3400640 1 -- 172.21.15.77:0/2553249618 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c80a76a0 con 0x5558329f0590 2024-09-10T09:57:43.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.474+0000 7fa9d6800640 1 --2- 172.21.15.77:0/848177970 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fa9bc0a2b80 0x7fa9bc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:43.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.474+0000 7fa9cea00640 1 -- 172.21.15.77:0/848177970 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fa9c80f08d0 con 0x7fa9d8101de0 2024-09-10T09:57:43.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.474+0000 7fa9dfc01740 1 -- 172.21.15.77:0/2553249618 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5558329f0590 msgr2=0x5558329fa910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:43.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.474+0000 7fa9dfc01740 1 --2- 172.21.15.77:0/2553249618 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5558329f0590 0x5558329fa910 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa9c80edf20 tx=0x7fa9c80f8670 comp rx=0 tx=0).stop 2024-09-10T09:57:43.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.474+0000 7fa9d6800640 1 --2- 172.21.15.77:0/848177970 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fa9bc0a2b80 0x7fa9bc0a5040 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa9d0005fd0 tx=0x7fa9d000b5c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:43.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.474+0000 7fa9dfc01740 1 -- 172.21.15.77:0/2553249618 shutdown_connections 2024-09-10T09:57:43.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.474+0000 7fa9dfc01740 1 --2- 172.21.15.77:0/2553249618 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5558329fae50 0x5558329fd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:43.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.474+0000 7fa9dfc01740 1 --2- 172.21.15.77:0/2553249618 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5558329f0590 0x5558329fa910 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:43.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.474+0000 7fa9dfc01740 1 --2- 172.21.15.77:0/2553249618 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5558329efc70 0x5558329f0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:43.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.474+0000 7fa9dfc01740 1 -- 172.21.15.77:0/2553249618 >> 172.21.15.77:0/2553249618 conn(0x5558329ec200 msgr2=0x5558329ec5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:43.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.474+0000 7fa9dfc01740 1 -- 172.21.15.77:0/2553249618 shutdown_connections 2024-09-10T09:57:43.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.475+0000 7fa9dfc01740 1 -- 172.21.15.77:0/2553249618 wait complete. 2024-09-10T09:57:43.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.475+0000 7fa9dfc01740 1 -- start start 2024-09-10T09:57:43.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.475+0000 7fa9dfc01740 1 Processor -- start 2024-09-10T09:57:43.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.476+0000 7fa9dfc01740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5558329efc70 0x5558329f7c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:43.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.476+0000 7fa9dfc01740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5558329f0590 0x5558329f81a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:43.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.476+0000 7fa9dfc01740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5558329fae50 0x5558329f86e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:43.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.476+0000 7fa9dfc01740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5558328fdd80 con 0x5558329fae50 2024-09-10T09:57:43.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.476+0000 7fa9dfc01740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5558329110f0 con 0x5558329f0590 2024-09-10T09:57:43.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.476+0000 7fa9dfc01740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5558329fd7e0 con 0x5558329efc70 2024-09-10T09:57:43.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.476+0000 7fa9dfc01740 10 client.0 _ll_register_callbacks cb 0x5558328f8bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:57:43.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.476+0000 7fa9d5400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5558329f0590 0x5558329f81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:43.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.476+0000 7fa9d5400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5558329f0590 0x5558329f81a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:33306/0 (socket says 172.21.15.77:33306) 2024-09-10T09:57:43.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.476+0000 7fa9d5400640 1 -- 172.21.15.77:0/2022021135 learned_addr learned my addr 172.21.15.77:0/2022021135 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:57:43.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.476+0000 7fa9d6800640 1 --2- 172.21.15.77:0/2022021135 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5558329fae50 0x5558329f86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:43.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.476+0000 7fa9d5e00640 1 --2- 172.21.15.77:0/2022021135 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5558329efc70 0x5558329f7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:43.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.476+0000 7fa9d5400640 1 -- 172.21.15.77:0/2022021135 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5558329efc70 msgr2=0x5558329f7c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:43.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.476+0000 7fa9d5400640 1 --2- 172.21.15.77:0/2022021135 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5558329efc70 0x5558329f7c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:43.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.476+0000 7fa9d5400640 1 -- 172.21.15.77:0/2022021135 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5558329fae50 msgr2=0x5558329f86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:43.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.476+0000 7fa9d5400640 1 --2- 172.21.15.77:0/2022021135 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5558329fae50 0x5558329f86e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:43.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.476+0000 7fa9d5400640 1 -- 172.21.15.77:0/2022021135 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558326c73c0 con 0x5558329f0590 2024-09-10T09:57:43.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.476+0000 7fa9d5e00640 1 --2- 172.21.15.77:0/2022021135 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5558329efc70 0x5558329f7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:57:43.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.477+0000 7fa9d5400640 1 --2- 172.21.15.77:0/2022021135 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5558329f0590 0x5558329f81a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa9c4012700 tx=0x7fa9c4012bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=361b422396ba038f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:43.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.477+0000 7fa9b7e00640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T09:57:43.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.477+0000 7fa9b7e00640 1 -- 172.21.15.77:0/2022021135 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c4004050 con 0x5558329f0590 2024-09-10T09:57:43.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.477+0000 7fa9b7e00640 1 -- 172.21.15.77:0/2022021135 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fa9c40041f0 con 0x5558329f0590 2024-09-10T09:57:43.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.477+0000 7fa9dfc01740 1 -- 172.21.15.77:0/2022021135 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9c80bd3e0 con 0x5558329f0590 2024-09-10T09:57:43.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.477+0000 7fa9b7e00640 1 -- 172.21.15.77:0/2022021135 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa9c4025050 con 0x5558329f0590 2024-09-10T09:57:43.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.477+0000 7fa9dfc01740 10 client.26057 fetch_fsmap learned FSMap version 32 2024-09-10T09:57:43.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.477+0000 7fa9dfc01740 1 -- 172.21.15.77:0/2022021135 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x555832a09590 con 0x5558329f0590 2024-09-10T09:57:43.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.477+0000 7fa9b7e00640 1 -- 172.21.15.77:0/2022021135 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9c80bd3e0 con 0x5558329f0590 2024-09-10T09:57:43.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.478+0000 7fa9b7e00640 1 -- 172.21.15.77:0/2022021135 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fa9c4003a60 con 0x5558329f0590 2024-09-10T09:57:43.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.478+0000 7fa9dfc01740 10 client.26057 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T09:57:43.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.478+0000 7fa9dfc01740 10 client.26057 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:57:43.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.478+0000 7fa9dfc01740 20 client.26057 populate_metadata read hostname 'smithi077' 2024-09-10T09:57:43.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.478+0000 7fa9dfc01740 1 --2- 172.21.15.77:0/2022021135 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x555832a0ab40 0x555832a0cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:43.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.478+0000 7fa9dfc01740 4 client.26057 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:57:43.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.478+0000 7fa9dfc01740 1 -- 172.21.15.77:0/2022021135 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555832963950 con 0x555832a0ab40 2024-09-10T09:57:43.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.478+0000 7fa9d5e00640 1 --2- 172.21.15.77:0/2022021135 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x555832a0ab40 0x555832a0cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:43.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.479+0000 7fa9d5e00640 1 --2- 172.21.15.77:0/2022021135 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x555832a0ab40 0x555832a0cfa0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74f3c0c28a1affee server_cookie=1ca066a25ae3317d in_seq=0 out_seq=0 2024-09-10T09:57:43.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.479+0000 7fa9b7e00640 10 client.26057 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T09:57:43.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.479+0000 7fa9b7e00640 1 -- 172.21.15.77:0/2022021135 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555832963950 con 0x555832a0ab40 2024-09-10T09:57:43.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.479+0000 7fa9b7e00640 10 client.26057 handle_command_reply: tid=0 2024-09-10T09:57:43.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.479+0000 7fa9dfc01740 1 client.26057 shutdown 2024-09-10T09:57:43.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.480+0000 7fa9dfc01740 1 -- 172.21.15.77:0/2022021135 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5558329f0590 msgr2=0x5558329f81a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:43.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.480+0000 7fa9dfc01740 1 --2- 172.21.15.77:0/2022021135 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5558329f0590 0x5558329f81a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa9c4012700 tx=0x7fa9c4012bc0 comp rx=0 tx=0).stop 2024-09-10T09:57:43.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.480+0000 7fa9dfc01740 1 -- 172.21.15.77:0/2022021135 shutdown_connections 2024-09-10T09:57:43.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.480+0000 7fa9dfc01740 1 --2- 172.21.15.77:0/2022021135 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x555832a0ab40 0x555832a0cfa0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:43.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.480+0000 7fa9dfc01740 1 --2- 172.21.15.77:0/2022021135 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5558329fae50 0x5558329f86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:43.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.480+0000 7fa9b7e00640 0 client.26057 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T09:57:43.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.480+0000 7fa9dfc01740 1 --2- 172.21.15.77:0/2022021135 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5558329f0590 0x5558329f81a0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:43.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.480+0000 7fa9dfc01740 1 --2- 172.21.15.77:0/2022021135 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5558329efc70 0x5558329f7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:43.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.480+0000 7fa9dfc01740 1 -- 172.21.15.77:0/2022021135 >> 172.21.15.77:0/2022021135 conn(0x5558329ec200 msgr2=0x5558329f1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:43.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.480+0000 7fa9dfc01740 1 -- 172.21.15.77:0/2022021135 shutdown_connections 2024-09-10T09:57:43.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.480+0000 7fa9dfc01740 1 -- 172.21.15.77:0/2022021135 wait complete. 2024-09-10T09:57:43.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.480+0000 7fa9dfc01740 20 client.26057 trim_cache size 0 max 16384 2024-09-10T09:57:43.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.494+0000 7fa9dfc01740 1 Processor -- start 2024-09-10T09:57:43.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.494+0000 7fa9dfc01740 1 -- start start 2024-09-10T09:57:43.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.494+0000 7fa9dfc01740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5558329efc70 0x5558329f9f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:43.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.494+0000 7fa9dfc01740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5558329f0590 0x5558329f26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:43.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.494+0000 7fa9dfc01740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5558329fae50 0x555832a12870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:43.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.494+0000 7fa9dfc01740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d81b0fd0 con 0x5558329efc70 2024-09-10T09:57:43.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.494+0000 7fa9dfc01740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d81b2bd0 con 0x5558329fae50 2024-09-10T09:57:43.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.494+0000 7fa9dfc01740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d81b16a0 con 0x5558329f0590 2024-09-10T09:57:43.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.494+0000 7fa9d5e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5558329efc70 0x5558329f9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:43.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.494+0000 7fa9d5e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5558329efc70 0x5558329f9f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52456/0 (socket says 172.21.15.77:52456) 2024-09-10T09:57:43.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.494+0000 7fa9d5e00640 1 -- 172.21.15.77:0/3329032668 learned_addr learned my addr 172.21.15.77:0/3329032668 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:57:43.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.494+0000 7fa9d5400640 1 --2- 172.21.15.77:0/3329032668 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5558329f0590 0x5558329f26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:43.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.494+0000 7fa9d6800640 1 --2- 172.21.15.77:0/3329032668 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5558329fae50 0x555832a12870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:43.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.495+0000 7fa9d5e00640 1 -- 172.21.15.77:0/3329032668 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5558329f0590 msgr2=0x5558329f26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:43.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.495+0000 7fa9d5e00640 1 --2- 172.21.15.77:0/3329032668 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5558329f0590 0x5558329f26d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:43.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.495+0000 7fa9d5e00640 1 -- 172.21.15.77:0/3329032668 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5558329fae50 msgr2=0x555832a12870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:43.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.495+0000 7fa9d5e00640 1 --2- 172.21.15.77:0/3329032668 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5558329fae50 0x555832a12870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:43.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.495+0000 7fa9d5e00640 1 -- 172.21.15.77:0/3329032668 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558329ec030 con 0x5558329efc70 2024-09-10T09:57:43.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.495+0000 7fa9d5e00640 1 --2- 172.21.15.77:0/3329032668 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5558329efc70 0x5558329f9f90 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fa9c8006010 tx=0x7fa9c802a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9acff16128a31507 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:43.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.495+0000 7fa9c2a00640 1 -- 172.21.15.77:0/3329032668 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c80ff070 con 0x5558329efc70 2024-09-10T09:57:43.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.495+0000 7fa9c2a00640 1 -- 172.21.15.77:0/3329032668 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9c8036040 con 0x5558329efc70 2024-09-10T09:57:43.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.496+0000 7fa9c2a00640 1 -- 172.21.15.77:0/3329032668 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c80bcce0 con 0x5558329efc70 2024-09-10T09:57:43.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.496+0000 7fa9dfc01740 1 -- 172.21.15.77:0/3329032668 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5558329efc70 msgr2=0x5558329f9f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:43.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.496+0000 7fa9dfc01740 1 --2- 172.21.15.77:0/3329032668 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5558329efc70 0x5558329f9f90 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fa9c8006010 tx=0x7fa9c802a8c0 comp rx=0 tx=0).stop 2024-09-10T09:57:43.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.496+0000 7fa9dfc01740 1 -- 172.21.15.77:0/3329032668 shutdown_connections 2024-09-10T09:57:43.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.496+0000 7fa9dfc01740 1 --2- 172.21.15.77:0/3329032668 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5558329fae50 0x555832a12870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:43.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.496+0000 7fa9dfc01740 1 --2- 172.21.15.77:0/3329032668 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5558329f0590 0x5558329f26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:43.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.496+0000 7fa9dfc01740 1 --2- 172.21.15.77:0/3329032668 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5558329efc70 0x5558329f9f90 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:43.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.496+0000 7fa9dfc01740 1 -- 172.21.15.77:0/3329032668 >> 172.21.15.77:0/3329032668 conn(0x5558329ec200 msgr2=0x5558329f6c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:43.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.496+0000 7fa9dfc01740 1 -- 172.21.15.77:0/3329032668 shutdown_connections 2024-09-10T09:57:43.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.496+0000 7fa9dfc01740 1 -- 172.21.15.77:0/3329032668 wait complete. 2024-09-10T09:57:43.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.497+0000 7fa9dfc01740 1 -- start start 2024-09-10T09:57:43.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.497+0000 7fa9dfc01740 1 Processor -- start 2024-09-10T09:57:43.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.497+0000 7fa9dfc01740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5558329efc70 0x5558329ff000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:43.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.497+0000 7fa9dfc01740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5558329f0590 0x5558329ff540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:43.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.497+0000 7fa9dfc01740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5558329fae50 0x555832a6e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:43.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.497+0000 7fa9dfc01740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d81b2440 con 0x5558329fae50 2024-09-10T09:57:43.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.497+0000 7fa9dfc01740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d81b1d70 con 0x5558329efc70 2024-09-10T09:57:43.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.497+0000 7fa9dfc01740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d81aefb0 con 0x5558329f0590 2024-09-10T09:57:43.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.497+0000 7fa9dfc01740 10 client.0 _ll_register_callbacks cb 0x7fa9c4009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:57:43.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.498+0000 7fa9d5e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5558329efc70 0x5558329ff000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:43.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.498+0000 7fa9d5e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5558329efc70 0x5558329ff000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:33320/0 (socket says 172.21.15.77:33320) 2024-09-10T09:57:43.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.498+0000 7fa9d5e00640 1 -- 172.21.15.77:0/841848534 learned_addr learned my addr 172.21.15.77:0/841848534 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:57:43.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.498+0000 7fa9d6800640 1 --2- 172.21.15.77:0/841848534 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5558329fae50 0x555832a6e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:43.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.498+0000 7fa9d5400640 1 --2- 172.21.15.77:0/841848534 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5558329f0590 0x5558329ff540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:43.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.498+0000 7fa9d5e00640 1 -- 172.21.15.77:0/841848534 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5558329f0590 msgr2=0x5558329ff540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:43.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.498+0000 7fa9d5e00640 1 --2- 172.21.15.77:0/841848534 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5558329f0590 0x5558329ff540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:43.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.498+0000 7fa9d5e00640 1 -- 172.21.15.77:0/841848534 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5558329fae50 msgr2=0x555832a6e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:43.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.498+0000 7fa9d5e00640 1 --2- 172.21.15.77:0/841848534 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5558329fae50 0x555832a6e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:43.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.498+0000 7fa9d5e00640 1 -- 172.21.15.77:0/841848534 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558329ece90 con 0x5558329efc70 2024-09-10T09:57:43.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.498+0000 7fa9d5e00640 1 --2- 172.21.15.77:0/841848534 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5558329efc70 0x5558329ff000 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa9c80bc680 tx=0x7fa9c8005ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c48600cb614d69c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:43.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.498+0000 7fa9b7e00640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T09:57:43.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.498+0000 7fa9b7e00640 1 -- 172.21.15.77:0/841848534 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c80ff050 con 0x5558329efc70 2024-09-10T09:57:43.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.498+0000 7fa9b7e00640 1 -- 172.21.15.77:0/841848534 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fa9c802fd70 con 0x5558329efc70 2024-09-10T09:57:43.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.498+0000 7fa9dfc01740 1 -- 172.21.15.77:0/841848534 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9c8036040 con 0x5558329efc70 2024-09-10T09:57:43.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.499+0000 7fa9b7e00640 1 -- 172.21.15.77:0/841848534 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa9c80bcab0 con 0x5558329efc70 2024-09-10T09:57:43.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.499+0000 7fa9dfc01740 10 client.26063 fetch_fsmap learned FSMap version 32 2024-09-10T09:57:43.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.499+0000 7fa9dfc01740 1 -- 172.21.15.77:0/841848534 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5558329e7770 con 0x5558329efc70 2024-09-10T09:57:43.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.499+0000 7fa9b7e00640 1 -- 172.21.15.77:0/841848534 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9c8036040 con 0x5558329efc70 2024-09-10T09:57:43.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.499+0000 7fa9b7e00640 1 -- 172.21.15.77:0/841848534 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fa9c804e030 con 0x5558329efc70 2024-09-10T09:57:43.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.499+0000 7fa9dfc01740 10 client.26063 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T09:57:43.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.499+0000 7fa9dfc01740 10 client.26063 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:57:43.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.499+0000 7fa9dfc01740 20 client.26063 populate_metadata read hostname 'smithi077' 2024-09-10T09:57:43.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.499+0000 7fa9dfc01740 1 --2- 172.21.15.77:0/841848534 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x555832a0ab40 0x555832a78170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:43.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.499+0000 7fa9dfc01740 4 client.26063 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:57:43.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.499+0000 7fa9dfc01740 1 -- 172.21.15.77:0/841848534 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55583295de60 con 0x555832a0ab40 2024-09-10T09:57:43.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.500+0000 7fa9d5400640 1 --2- 172.21.15.77:0/841848534 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x555832a0ab40 0x555832a78170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:43.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.500+0000 7fa9d5400640 1 --2- 172.21.15.77:0/841848534 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x555832a0ab40 0x555832a78170 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6c08a1dda5915ea server_cookie=1d12e51f12b5b915 in_seq=0 out_seq=0 2024-09-10T09:57:43.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.500+0000 7fa9b7e00640 10 client.26063 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T09:57:43.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.500+0000 7fa9b7e00640 1 -- 172.21.15.77:0/841848534 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55583295de60 con 0x555832a0ab40 2024-09-10T09:57:43.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.500+0000 7fa9b7e00640 10 client.26063 handle_command_reply: tid=0 2024-09-10T09:57:43.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.500+0000 7fa9dfc01740 1 client.26063 shutdown 2024-09-10T09:57:43.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.501+0000 7fa9dfc01740 1 -- 172.21.15.77:0/841848534 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5558329efc70 msgr2=0x5558329ff000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:43.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.501+0000 7fa9dfc01740 1 --2- 172.21.15.77:0/841848534 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5558329efc70 0x5558329ff000 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa9c80bc680 tx=0x7fa9c8005ef0 comp rx=0 tx=0).stop 2024-09-10T09:57:43.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.501+0000 7fa9dfc01740 1 -- 172.21.15.77:0/841848534 shutdown_connections 2024-09-10T09:57:43.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.501+0000 7fa9dfc01740 1 --2- 172.21.15.77:0/841848534 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x555832a0ab40 0x555832a78170 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:43.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.501+0000 7fa9dfc01740 1 --2- 172.21.15.77:0/841848534 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5558329fae50 0x555832a6e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:43.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.501+0000 7fa9dfc01740 1 --2- 172.21.15.77:0/841848534 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5558329f0590 0x5558329ff540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:43.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.501+0000 7fa9dfc01740 1 --2- 172.21.15.77:0/841848534 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5558329efc70 0x5558329ff000 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:43.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.501+0000 7fa9dfc01740 1 -- 172.21.15.77:0/841848534 >> 172.21.15.77:0/841848534 conn(0x5558329ec200 msgr2=0x5558329f7bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:43.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.501+0000 7fa9b7e00640 0 client.26063 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T09:57:43.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.501+0000 7fa9dfc01740 1 -- 172.21.15.77:0/841848534 shutdown_connections 2024-09-10T09:57:43.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.501+0000 7fa9dfc01740 1 -- 172.21.15.77:0/841848534 wait complete. 2024-09-10T09:57:43.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.501+0000 7fa9dfc01740 20 client.26063 trim_cache size 0 max 16384 2024-09-10T09:57:43.506 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-10T09:57:43.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.501+0000 7fa9dd000640 1 -- 172.21.15.77:0/848177970 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fa9bc0a2b80 msgr2=0x7fa9bc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:43.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.501+0000 7fa9dd000640 1 --2- 172.21.15.77:0/848177970 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fa9bc0a2b80 0x7fa9bc0a5040 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa9d0005fd0 tx=0x7fa9d000b5c0 comp rx=0 tx=0).stop 2024-09-10T09:57:43.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.502+0000 7fa9dd000640 1 -- 172.21.15.77:0/848177970 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa9d8101de0 msgr2=0x7fa9d80744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:43.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.502+0000 7fa9dd000640 1 --2- 172.21.15.77:0/848177970 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa9d8101de0 0x7fa9d80744f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa9c802ade0 tx=0x7fa9c80319f0 comp rx=0 tx=0).stop 2024-09-10T09:57:43.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.502+0000 7fa9dd000640 1 -- 172.21.15.77:0/848177970 shutdown_connections 2024-09-10T09:57:43.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.502+0000 7fa9dd000640 1 --2- 172.21.15.77:0/848177970 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fa9bc0a2b80 0x7fa9bc0a5040 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:43.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.502+0000 7fa9dd000640 1 --2- 172.21.15.77:0/848177970 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa9d8106f10 0x7fa9d8074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:43.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.502+0000 7fa9dd000640 1 --2- 172.21.15.77:0/848177970 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa9d8102790 0x7fa9d8074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:43.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.502+0000 7fa9dd000640 1 --2- 172.21.15.77:0/848177970 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa9d8101de0 0x7fa9d80744f0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:43.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.502+0000 7fa9dd000640 1 -- 172.21.15.77:0/848177970 >> 172.21.15.77:0/848177970 conn(0x7fa9d80fd8c0 msgr2=0x7fa9d80fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:43.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.502+0000 7fa9dd000640 1 -- 172.21.15.77:0/848177970 shutdown_connections 2024-09-10T09:57:43.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:43.502+0000 7fa9dd000640 1 -- 172.21.15.77:0/848177970 wait complete. 2024-09-10T09:57:45.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:57:44 smithi120 ceph-mon[34770]: pgmap v468: 129 pgs: 129 active+clean; 29 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 56 op/s 2024-09-10T09:57:45.177 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:57:44 smithi077 ceph-mon[30319]: pgmap v468: 129 pgs: 129 active+clean; 29 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 56 op/s 2024-09-10T09:57:45.365 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T09:57:45.365 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T09:57:45.380 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:57:44 smithi134 ceph-mon[35674]: pgmap v468: 129 pgs: 129 active+clean; 29 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 56 op/s 2024-09-10T09:57:46.221 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:57:46.428 INFO:tasks.workunit.client.0.smithi077.stderr:Note: switching to '0e5c12dfd008efc2848c98108c9237487e91ef35'. 2024-09-10T09:57:46.428 INFO:tasks.workunit.client.0.smithi077.stderr: 2024-09-10T09:57:46.428 INFO:tasks.workunit.client.0.smithi077.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-10T09:57:46.428 INFO:tasks.workunit.client.0.smithi077.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-10T09:57:46.428 INFO:tasks.workunit.client.0.smithi077.stderr:state without impacting any branches by switching back to a branch. 2024-09-10T09:57:46.428 INFO:tasks.workunit.client.0.smithi077.stderr: 2024-09-10T09:57:46.428 INFO:tasks.workunit.client.0.smithi077.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-10T09:57:46.428 INFO:tasks.workunit.client.0.smithi077.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-10T09:57:46.428 INFO:tasks.workunit.client.0.smithi077.stderr: 2024-09-10T09:57:46.429 INFO:tasks.workunit.client.0.smithi077.stderr: git switch -c 2024-09-10T09:57:46.429 INFO:tasks.workunit.client.0.smithi077.stderr: 2024-09-10T09:57:46.429 INFO:tasks.workunit.client.0.smithi077.stderr:Or undo this operation with: 2024-09-10T09:57:46.429 INFO:tasks.workunit.client.0.smithi077.stderr: 2024-09-10T09:57:46.429 INFO:tasks.workunit.client.0.smithi077.stderr: git switch - 2024-09-10T09:57:46.429 INFO:tasks.workunit.client.0.smithi077.stderr: 2024-09-10T09:57:46.429 INFO:tasks.workunit.client.0.smithi077.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-10T09:57:46.429 INFO:tasks.workunit.client.0.smithi077.stderr: 2024-09-10T09:57:46.429 INFO:tasks.workunit.client.0.smithi077.stderr:HEAD is now at 0e5c12df xfs/522: use reflink instead of crc as test feature 2024-09-10T09:57:46.471 INFO:tasks.workunit.client.0.smithi077.stderr:+ make -j4 2024-09-10T09:57:46.494 INFO:tasks.workunit.client.0.smithi077.stdout:libtoolize -cfi 2024-09-10T09:57:46.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.887+0000 7fae2da00640 1 -- 172.21.15.77:0/180341353 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fae28103e30 msgr2=0x7fae28104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:46.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.887+0000 7fae2da00640 1 --2- 172.21.15.77:0/180341353 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fae28103e30 0x7fae28104210 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fae18009900 tx=0x7fae1802a400 comp rx=0 tx=0).stop 2024-09-10T09:57:46.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.887+0000 7fae2da00640 1 -- 172.21.15.77:0/180341353 shutdown_connections 2024-09-10T09:57:46.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.887+0000 7fae2da00640 1 --2- 172.21.15.77:0/180341353 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae28108f60 0x7fae28110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:46.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.887+0000 7fae2da00640 1 --2- 172.21.15.77:0/180341353 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fae281047e0 0x7fae28108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:46.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.887+0000 7fae2da00640 1 --2- 172.21.15.77:0/180341353 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fae28103e30 0x7fae28104210 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:46.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.887+0000 7fae2da00640 1 -- 172.21.15.77:0/180341353 >> 172.21.15.77:0/180341353 conn(0x7fae280ff910 msgr2=0x7fae28101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:46.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.887+0000 7fae2da00640 1 -- 172.21.15.77:0/180341353 shutdown_connections 2024-09-10T09:57:46.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.887+0000 7fae2da00640 1 -- 172.21.15.77:0/180341353 wait complete. 2024-09-10T09:57:46.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.888+0000 7fae2da00640 1 Processor -- start 2024-09-10T09:57:46.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.888+0000 7fae2da00640 1 -- start start 2024-09-10T09:57:46.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.889+0000 7fae2da00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae28103e30 0x7fae2819a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:46.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.889+0000 7fae2da00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fae281047e0 0x7fae2819a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:46.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.889+0000 7fae2da00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fae28108f60 0x7fae2819ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:46.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.889+0000 7fae2da00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fae28113fc0 con 0x7fae281047e0 2024-09-10T09:57:46.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.889+0000 7fae2da00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fae28113e40 con 0x7fae28103e30 2024-09-10T09:57:46.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.889+0000 7fae2da00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fae28114140 con 0x7fae28108f60 2024-09-10T09:57:46.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.889+0000 7fae27e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae28103e30 0x7fae2819a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:46.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.889+0000 7fae2d000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fae28108f60 0x7fae2819ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:46.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.889+0000 7fae27400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fae281047e0 0x7fae2819a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:46.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.889+0000 7fae27e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae28103e30 0x7fae2819a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:33330/0 (socket says 172.21.15.77:33330) 2024-09-10T09:57:46.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.889+0000 7fae27e00640 1 -- 172.21.15.77:0/3398607628 learned_addr learned my addr 172.21.15.77:0/3398607628 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:57:46.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.890+0000 7fae2d000640 1 -- 172.21.15.77:0/3398607628 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae28103e30 msgr2=0x7fae2819a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:46.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.890+0000 7fae2d000640 1 --2- 172.21.15.77:0/3398607628 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae28103e30 0x7fae2819a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:46.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.890+0000 7fae2d000640 1 -- 172.21.15.77:0/3398607628 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fae281047e0 msgr2=0x7fae2819a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:46.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.890+0000 7fae2d000640 1 --2- 172.21.15.77:0/3398607628 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fae281047e0 0x7fae2819a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:46.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.890+0000 7fae2d000640 1 -- 172.21.15.77:0/3398607628 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae2819f410 con 0x7fae28108f60 2024-09-10T09:57:46.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.890+0000 7fae27400640 1 --2- 172.21.15.77:0/3398607628 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fae281047e0 0x7fae2819a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:57:46.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.890+0000 7fae2d000640 1 --2- 172.21.15.77:0/3398607628 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fae28108f60 0x7fae2819ed30 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fae20009fd0 tx=0x7fae2000ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:46.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.890+0000 7fae24c00640 1 -- 172.21.15.77:0/3398607628 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae2000ce20 con 0x7fae28108f60 2024-09-10T09:57:46.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.890+0000 7fae2da00640 1 -- 172.21.15.77:0/3398607628 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae2819f700 con 0x7fae28108f60 2024-09-10T09:57:46.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.890+0000 7fae24c00640 1 -- 172.21.15.77:0/3398607628 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae20005020 con 0x7fae28108f60 2024-09-10T09:57:46.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.890+0000 7fae24c00640 1 -- 172.21.15.77:0/3398607628 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae20010720 con 0x7fae28108f60 2024-09-10T09:57:46.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.891+0000 7fae2da00640 1 -- 172.21.15.77:0/3398607628 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae281a6fe0 con 0x7fae28108f60 2024-09-10T09:57:46.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.894+0000 7fae31a77740 1 Processor -- start 2024-09-10T09:57:46.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.895+0000 7fae24c00640 1 -- 172.21.15.77:0/3398607628 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fae20005540 con 0x7fae28108f60 2024-09-10T09:57:46.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.895+0000 7fae31a77740 1 -- start start 2024-09-10T09:57:46.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.895+0000 7fae31a77740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654bad53cb0 0x5654bad54090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:46.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.895+0000 7fae31a77740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654bad545d0 0x5654bad5e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:46.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.895+0000 7fae31a77740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5654bad5ee90 0x5654bad61280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:46.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.895+0000 7fae31a77740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5654bacba290 con 0x5654bad5ee90 2024-09-10T09:57:46.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.895+0000 7fae31a77740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5654bac61b40 con 0x5654bad53cb0 2024-09-10T09:57:46.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.895+0000 7fae31a77740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5654bacb6250 con 0x5654bad545d0 2024-09-10T09:57:46.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.895+0000 7fae27e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654bad545d0 0x5654bad5e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:46.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.895+0000 7fae27e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654bad545d0 0x5654bad5e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:45008/0 (socket says 172.21.15.77:45008) 2024-09-10T09:57:46.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.895+0000 7fae27e00640 1 -- 172.21.15.77:0/3780009890 learned_addr learned my addr 172.21.15.77:0/3780009890 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:57:46.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.896+0000 7fae27400640 1 --2- 172.21.15.77:0/3780009890 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654bad53cb0 0x5654bad54090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:46.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.896+0000 7fae27400640 1 --2- 172.21.15.77:0/3780009890 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5654bad5ee90 0x5654bad61280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:46.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.896+0000 7fae27e00640 1 -- 172.21.15.77:0/3780009890 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654bad53cb0 msgr2=0x5654bad54090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:46.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.896+0000 7fae27e00640 1 --2- 172.21.15.77:0/3780009890 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654bad53cb0 0x5654bad54090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:46.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.896+0000 7fae27e00640 1 -- 172.21.15.77:0/3780009890 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5654bad5ee90 msgr2=0x5654bad61280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:46.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.896+0000 7fae27e00640 1 --2- 172.21.15.77:0/3780009890 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5654bad5ee90 0x5654bad61280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:46.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.896+0000 7fae27e00640 1 -- 172.21.15.77:0/3780009890 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654baa2b6a0 con 0x5654bad545d0 2024-09-10T09:57:46.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.896+0000 7fae24c00640 1 --2- 172.21.15.77:0/3398607628 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fae0c0a2c00 0x7fae0c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:46.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.896+0000 7fae24c00640 1 -- 172.21.15.77:0/3398607628 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fae200cfa40 con 0x7fae28108f60 2024-09-10T09:57:46.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.896+0000 7fae27e00640 1 --2- 172.21.15.77:0/3780009890 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654bad545d0 0x5654bad5e950 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fae18009a20 tx=0x7fae18005dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=736eea361feb1442 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:46.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.896+0000 7fae27400640 1 --2- 172.21.15.77:0/3398607628 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fae0c0a2c00 0x7fae0c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:46.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.897+0000 7fae1d600640 1 -- 172.21.15.77:0/3780009890 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae18040070 con 0x5654bad545d0 2024-09-10T09:57:46.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.897+0000 7fae1d600640 1 -- 172.21.15.77:0/3780009890 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae18004030 con 0x5654bad545d0 2024-09-10T09:57:46.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.897+0000 7fae1d600640 1 -- 172.21.15.77:0/3780009890 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae18035040 con 0x5654bad545d0 2024-09-10T09:57:46.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.898+0000 7fae31a77740 1 -- 172.21.15.77:0/3780009890 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654bad545d0 msgr2=0x5654bad5e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:46.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.898+0000 7fae27400640 1 --2- 172.21.15.77:0/3398607628 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fae0c0a2c00 0x7fae0c0a50c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x5654bad515c0 tx=0x7fae14005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:46.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.898+0000 7fae31a77740 1 --2- 172.21.15.77:0/3780009890 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654bad545d0 0x5654bad5e950 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fae18009a20 tx=0x7fae18005dd0 comp rx=0 tx=0).stop 2024-09-10T09:57:46.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.898+0000 7fae31a77740 1 -- 172.21.15.77:0/3780009890 shutdown_connections 2024-09-10T09:57:46.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.898+0000 7fae31a77740 1 --2- 172.21.15.77:0/3780009890 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5654bad5ee90 0x5654bad61280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:46.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.898+0000 7fae31a77740 1 --2- 172.21.15.77:0/3780009890 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654bad545d0 0x5654bad5e950 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:46.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.898+0000 7fae31a77740 1 --2- 172.21.15.77:0/3780009890 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654bad53cb0 0x5654bad54090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:46.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.898+0000 7fae31a77740 1 -- 172.21.15.77:0/3780009890 >> 172.21.15.77:0/3780009890 conn(0x5654bad50240 msgr2=0x5654bad50620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:46.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.898+0000 7fae31a77740 1 -- 172.21.15.77:0/3780009890 shutdown_connections 2024-09-10T09:57:46.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.898+0000 7fae31a77740 1 -- 172.21.15.77:0/3780009890 wait complete. 2024-09-10T09:57:46.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.899+0000 7fae31a77740 1 -- start start 2024-09-10T09:57:46.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.899+0000 7fae31a77740 1 Processor -- start 2024-09-10T09:57:46.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.899+0000 7fae31a77740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654bad53cb0 0x5654bad5e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:46.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.899+0000 7fae31a77740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654bad545d0 0x5654bad5b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:46.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.899+0000 7fae31a77740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5654bad5ee90 0x5654bad5bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:46.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.899+0000 7fae31a77740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5654bad61b00 con 0x5654bad5ee90 2024-09-10T09:57:46.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.899+0000 7fae31a77740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5654bac74fa0 con 0x5654bad545d0 2024-09-10T09:57:46.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.899+0000 7fae31a77740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5654bad61c80 con 0x5654bad53cb0 2024-09-10T09:57:46.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.899+0000 7fae31a77740 10 client.0 _ll_register_callbacks cb 0x5654ba9c8ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:57:46.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.899+0000 7fae27e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654bad53cb0 0x5654bad5e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:46.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.899+0000 7fae27400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654bad545d0 0x5654bad5b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:46.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.899+0000 7fae27e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654bad53cb0 0x5654bad5e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:45024/0 (socket says 172.21.15.77:45024) 2024-09-10T09:57:46.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.899+0000 7fae27e00640 1 -- 172.21.15.77:0/644739400 learned_addr learned my addr 172.21.15.77:0/644739400 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:57:46.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.900+0000 7fae2d000640 1 --2- 172.21.15.77:0/644739400 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5654bad5ee90 0x5654bad5bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:46.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.900+0000 7fae27e00640 1 -- 172.21.15.77:0/644739400 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654bad545d0 msgr2=0x5654bad5b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:46.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.900+0000 7fae27e00640 1 --2- 172.21.15.77:0/644739400 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654bad545d0 0x5654bad5b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:46.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.900+0000 7fae27e00640 1 -- 172.21.15.77:0/644739400 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5654bad5ee90 msgr2=0x5654bad5bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:46.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.900+0000 7fae27e00640 1 --2- 172.21.15.77:0/644739400 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5654bad5ee90 0x5654bad5bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:46.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.900+0000 7fae27e00640 1 -- 172.21.15.77:0/644739400 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654bad5c6e0 con 0x5654bad53cb0 2024-09-10T09:57:46.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.900+0000 7fae27400640 1 --2- 172.21.15.77:0/644739400 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654bad545d0 0x5654bad5b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:57:46.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.900+0000 7fae2d000640 1 --2- 172.21.15.77:0/644739400 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5654bad5ee90 0x5654bad5bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:57:46.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.900+0000 7fae27e00640 1 --2- 172.21.15.77:0/644739400 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654bad53cb0 0x5654bad5e6e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fae1802a440 tx=0x7fae18009680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c2ff35bef663439 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:46.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.900+0000 7fae12000640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-10T09:57:46.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.900+0000 7fae12000640 1 -- 172.21.15.77:0/644739400 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae18040070 con 0x5654bad53cb0 2024-09-10T09:57:46.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.900+0000 7fae31a77740 1 -- 172.21.15.77:0/644739400 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae18004030 con 0x5654bad53cb0 2024-09-10T09:57:46.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.901+0000 7fae12000640 1 -- 172.21.15.77:0/644739400 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fae180315b0 con 0x5654bad53cb0 2024-09-10T09:57:46.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.901+0000 7fae12000640 1 -- 172.21.15.77:0/644739400 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fae18057050 con 0x5654bad53cb0 2024-09-10T09:57:46.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.901+0000 7fae31a77740 10 client.26023 fetch_fsmap learned FSMap version 32 2024-09-10T09:57:46.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.901+0000 7fae31a77740 1 -- 172.21.15.77:0/644739400 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5654bad61b50 con 0x5654bad53cb0 2024-09-10T09:57:46.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.901+0000 7fae12000640 1 -- 172.21.15.77:0/644739400 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae18004030 con 0x5654bad53cb0 2024-09-10T09:57:46.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.901+0000 7fae12000640 1 -- 172.21.15.77:0/644739400 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fae1804c020 con 0x5654bad53cb0 2024-09-10T09:57:46.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.901+0000 7fae31a77740 10 client.26023 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T09:57:46.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.901+0000 7fae31a77740 10 client.26023 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:57:46.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.901+0000 7fae31a77740 20 client.26023 populate_metadata read hostname 'smithi077' 2024-09-10T09:57:46.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.901+0000 7fae31a77740 1 --2- 172.21.15.77:0/644739400 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x5654bad6ead0 0x5654bad70ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:46.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.901+0000 7fae31a77740 4 client.26023 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:57:46.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.901+0000 7fae31a77740 1 -- 172.21.15.77:0/644739400 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5654bacc7800 con 0x5654bad6ead0 2024-09-10T09:57:46.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.902+0000 7fae27400640 1 --2- 172.21.15.77:0/644739400 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x5654bad6ead0 0x5654bad70ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:46.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.902+0000 7fae27400640 1 --2- 172.21.15.77:0/644739400 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x5654bad6ead0 0x5654bad70ec0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4d5039f6670c359 server_cookie=fe0738e4a03777e7 in_seq=0 out_seq=0 2024-09-10T09:57:46.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.902+0000 7fae12000640 10 client.26023 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T09:57:46.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.903+0000 7fae12000640 1 -- 172.21.15.77:0/644739400 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5654bacc7800 con 0x5654bad6ead0 2024-09-10T09:57:46.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.903+0000 7fae12000640 10 client.26023 handle_command_reply: tid=0 2024-09-10T09:57:46.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.903+0000 7fae31a77740 1 client.26023 shutdown 2024-09-10T09:57:46.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.903+0000 7fae31a77740 1 -- 172.21.15.77:0/644739400 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654bad53cb0 msgr2=0x5654bad5e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:46.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.903+0000 7fae31a77740 1 --2- 172.21.15.77:0/644739400 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654bad53cb0 0x5654bad5e6e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fae1802a440 tx=0x7fae18009680 comp rx=0 tx=0).stop 2024-09-10T09:57:46.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.903+0000 7fae31a77740 1 -- 172.21.15.77:0/644739400 shutdown_connections 2024-09-10T09:57:46.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.903+0000 7fae31a77740 1 --2- 172.21.15.77:0/644739400 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x5654bad6ead0 0x5654bad70ec0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:46.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.903+0000 7fae31a77740 1 --2- 172.21.15.77:0/644739400 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5654bad5ee90 0x5654bad5bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:46.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.903+0000 7fae31a77740 1 --2- 172.21.15.77:0/644739400 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654bad545d0 0x5654bad5b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:46.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.903+0000 7fae12000640 0 client.26023 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T09:57:46.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.903+0000 7fae31a77740 1 --2- 172.21.15.77:0/644739400 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654bad53cb0 0x5654bad5e6e0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:46.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.903+0000 7fae31a77740 1 -- 172.21.15.77:0/644739400 >> 172.21.15.77:0/644739400 conn(0x5654bad50240 msgr2=0x5654bad55f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:46.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.903+0000 7fae31a77740 1 -- 172.21.15.77:0/644739400 shutdown_connections 2024-09-10T09:57:46.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.903+0000 7fae31a77740 1 -- 172.21.15.77:0/644739400 wait complete. 2024-09-10T09:57:46.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.904+0000 7fae31a77740 20 client.26023 trim_cache size 0 max 16384 2024-09-10T09:57:46.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.938+0000 7fae31a77740 1 Processor -- start 2024-09-10T09:57:46.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.938+0000 7fae31a77740 1 -- start start 2024-09-10T09:57:46.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.939+0000 7fae31a77740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5654bad53cb0 0x5654bad5d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:46.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.939+0000 7fae31a77740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654bad545d0 0x5654bad76750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:46.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.939+0000 7fae31a77740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654bad5ee90 0x5654bad76c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:46.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.939+0000 7fae31a77740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fae281b0e30 con 0x5654bad53cb0 2024-09-10T09:57:46.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.939+0000 7fae31a77740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fae281b2a30 con 0x5654bad5ee90 2024-09-10T09:57:46.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.939+0000 7fae31a77740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fae281b1500 con 0x5654bad545d0 2024-09-10T09:57:46.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.939+0000 7fae27e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5654bad53cb0 0x5654bad5d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:46.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.939+0000 7fae27e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5654bad53cb0 0x5654bad5d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52512/0 (socket says 172.21.15.77:52512) 2024-09-10T09:57:46.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.939+0000 7fae27e00640 1 -- 172.21.15.77:0/3248668562 learned_addr learned my addr 172.21.15.77:0/3248668562 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:57:46.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.939+0000 7fae27400640 1 --2- 172.21.15.77:0/3248668562 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654bad545d0 0x5654bad76750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:46.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.939+0000 7fae2d000640 1 --2- 172.21.15.77:0/3248668562 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654bad5ee90 0x5654bad76c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:46.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.940+0000 7fae27e00640 1 -- 172.21.15.77:0/3248668562 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654bad545d0 msgr2=0x5654bad76750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:46.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.940+0000 7fae27e00640 1 --2- 172.21.15.77:0/3248668562 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654bad545d0 0x5654bad76750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:46.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.940+0000 7fae27e00640 1 -- 172.21.15.77:0/3248668562 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654bad5ee90 msgr2=0x5654bad76c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:46.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.940+0000 7fae27e00640 1 --2- 172.21.15.77:0/3248668562 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654bad5ee90 0x5654bad76c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:46.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.940+0000 7fae27e00640 1 -- 172.21.15.77:0/3248668562 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654bad5a940 con 0x5654bad53cb0 2024-09-10T09:57:46.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.940+0000 7fae2d000640 1 --2- 172.21.15.77:0/3248668562 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654bad5ee90 0x5654bad76c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:57:46.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.940+0000 7fae27e00640 1 --2- 172.21.15.77:0/3248668562 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5654bad53cb0 0x5654bad5d900 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fae18006010 tx=0x7fae18009290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cbe93bdd4d2e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:46.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.940+0000 7fae1cc00640 1 -- 172.21.15.77:0/3248668562 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae18040070 con 0x5654bad53cb0 2024-09-10T09:57:46.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.940+0000 7fae1cc00640 1 -- 172.21.15.77:0/3248668562 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae18003710 con 0x5654bad53cb0 2024-09-10T09:57:46.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.940+0000 7fae1cc00640 1 -- 172.21.15.77:0/3248668562 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae18035040 con 0x5654bad53cb0 2024-09-10T09:57:46.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.941+0000 7fae31a77740 1 -- 172.21.15.77:0/3248668562 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5654bad53cb0 msgr2=0x5654bad5d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:46.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.941+0000 7fae31a77740 1 --2- 172.21.15.77:0/3248668562 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5654bad53cb0 0x5654bad5d900 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fae18006010 tx=0x7fae18009290 comp rx=0 tx=0).stop 2024-09-10T09:57:46.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.941+0000 7fae31a77740 1 -- 172.21.15.77:0/3248668562 shutdown_connections 2024-09-10T09:57:46.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.941+0000 7fae31a77740 1 --2- 172.21.15.77:0/3248668562 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654bad5ee90 0x5654bad76c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:46.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.941+0000 7fae31a77740 1 --2- 172.21.15.77:0/3248668562 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654bad545d0 0x5654bad76750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:46.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.941+0000 7fae31a77740 1 --2- 172.21.15.77:0/3248668562 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5654bad53cb0 0x5654bad5d900 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:46.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.941+0000 7fae31a77740 1 -- 172.21.15.77:0/3248668562 >> 172.21.15.77:0/3248668562 conn(0x5654bad50240 msgr2=0x5654bad5d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:46.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.942+0000 7fae31a77740 1 -- 172.21.15.77:0/3248668562 shutdown_connections 2024-09-10T09:57:46.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.942+0000 7fae31a77740 1 -- 172.21.15.77:0/3248668562 wait complete. 2024-09-10T09:57:46.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.942+0000 7fae31a77740 1 -- start start 2024-09-10T09:57:46.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.942+0000 7fae31a77740 1 Processor -- start 2024-09-10T09:57:46.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.943+0000 7fae31a77740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654bad53cb0 0x5654bad65430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:46.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.943+0000 7fae31a77740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5654bad545d0 0x5654bad65970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:46.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.943+0000 7fae31a77740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654bad5ee90 0x5654badd8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:46.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.943+0000 7fae31a77740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fae281b22a0 con 0x5654bad545d0 2024-09-10T09:57:46.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.943+0000 7fae31a77740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fae281b1bd0 con 0x5654bad5ee90 2024-09-10T09:57:46.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.943+0000 7fae31a77740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fae281aee10 con 0x5654bad53cb0 2024-09-10T09:57:46.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.943+0000 7fae31a77740 10 client.0 _ll_register_callbacks cb 0x7fae18009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:57:46.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.943+0000 7fae27e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654bad53cb0 0x5654bad65430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:46.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.943+0000 7fae27e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654bad53cb0 0x5654bad65430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:45050/0 (socket says 172.21.15.77:45050) 2024-09-10T09:57:46.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.943+0000 7fae27400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5654bad545d0 0x5654bad65970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:46.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.943+0000 7fae27e00640 1 -- 172.21.15.77:0/958212950 learned_addr learned my addr 172.21.15.77:0/958212950 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:57:46.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.944+0000 7fae2d000640 1 --2- 172.21.15.77:0/958212950 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654bad5ee90 0x5654badd8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:46.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.944+0000 7fae27400640 1 -- 172.21.15.77:0/958212950 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654bad53cb0 msgr2=0x5654bad65430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:46.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.944+0000 7fae27400640 1 --2- 172.21.15.77:0/958212950 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654bad53cb0 0x5654bad65430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:46.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.944+0000 7fae27400640 1 -- 172.21.15.77:0/958212950 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654bad5ee90 msgr2=0x5654badd8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:46.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.944+0000 7fae27400640 1 --2- 172.21.15.77:0/958212950 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654bad5ee90 0x5654badd8730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:46.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.944+0000 7fae27400640 1 -- 172.21.15.77:0/958212950 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654bad54400 con 0x5654bad545d0 2024-09-10T09:57:46.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.944+0000 7fae27e00640 1 --2- 172.21.15.77:0/958212950 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654bad53cb0 0x5654bad65430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:57:46.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.944+0000 7fae27400640 1 --2- 172.21.15.77:0/958212950 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5654bad545d0 0x5654bad65970 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x5654bad5b220 tx=0x7fae14005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5837d40611a3bcd2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:46.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.944+0000 7fae12000640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-10T09:57:46.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.945+0000 7fae12000640 1 -- 172.21.15.77:0/958212950 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae14031070 con 0x5654bad545d0 2024-09-10T09:57:46.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.945+0000 7fae31a77740 1 -- 172.21.15.77:0/958212950 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae18003710 con 0x5654bad545d0 2024-09-10T09:57:46.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.945+0000 7fae12000640 1 -- 172.21.15.77:0/958212950 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fae1400fcb0 con 0x5654bad545d0 2024-09-10T09:57:46.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.945+0000 7fae12000640 1 -- 172.21.15.77:0/958212950 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fae1403d050 con 0x5654bad545d0 2024-09-10T09:57:46.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.945+0000 7fae31a77740 10 client.16005 fetch_fsmap learned FSMap version 32 2024-09-10T09:57:46.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.945+0000 7fae31a77740 1 -- 172.21.15.77:0/958212950 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5654badd9eb0 con 0x5654bad545d0 2024-09-10T09:57:46.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.945+0000 7fae12000640 1 -- 172.21.15.77:0/958212950 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae18003710 con 0x5654bad545d0 2024-09-10T09:57:46.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.945+0000 7fae12000640 1 -- 172.21.15.77:0/958212950 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fae14020560 con 0x5654bad545d0 2024-09-10T09:57:46.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.946+0000 7fae31a77740 10 client.16005 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T09:57:46.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.946+0000 7fae31a77740 10 client.16005 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:57:46.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.946+0000 7fae31a77740 20 client.16005 populate_metadata read hostname 'smithi077' 2024-09-10T09:57:46.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.946+0000 7fae31a77740 1 --2- 172.21.15.77:0/958212950 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x5654badda6a0 0x5654baddcac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:46.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.946+0000 7fae31a77740 4 client.16005 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T09:57:46.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.946+0000 7fae31a77740 1 -- 172.21.15.77:0/958212950 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5654bad50df0 con 0x5654badda6a0 2024-09-10T09:57:46.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.947+0000 7fae27e00640 1 --2- 172.21.15.77:0/958212950 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x5654badda6a0 0x5654baddcac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:46.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.947+0000 7fae27e00640 1 --2- 172.21.15.77:0/958212950 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x5654badda6a0 0x5654baddcac0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b83d36d514c3a2be server_cookie=8df03ccef691ddf9 in_seq=0 out_seq=0 2024-09-10T09:57:46.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.947+0000 7fae12000640 10 client.16005 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T09:57:46.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.948+0000 7fae12000640 1 -- 172.21.15.77:0/958212950 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5654bad50df0 con 0x5654badda6a0 2024-09-10T09:57:46.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.948+0000 7fae12000640 10 client.16005 handle_command_reply: tid=0 2024-09-10T09:57:46.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.948+0000 7fae31a77740 1 client.16005 shutdown 2024-09-10T09:57:46.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.948+0000 7fae31a77740 1 -- 172.21.15.77:0/958212950 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5654bad545d0 msgr2=0x5654bad65970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:46.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.948+0000 7fae31a77740 1 --2- 172.21.15.77:0/958212950 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5654bad545d0 0x5654bad65970 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x5654bad5b220 tx=0x7fae14005850 comp rx=0 tx=0).stop 2024-09-10T09:57:46.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.948+0000 7fae31a77740 1 -- 172.21.15.77:0/958212950 shutdown_connections 2024-09-10T09:57:46.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.948+0000 7fae31a77740 1 --2- 172.21.15.77:0/958212950 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x5654badda6a0 0x5654baddcac0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:46.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.949+0000 7fae31a77740 1 --2- 172.21.15.77:0/958212950 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654bad5ee90 0x5654badd8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:46.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.949+0000 7fae31a77740 1 --2- 172.21.15.77:0/958212950 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5654bad545d0 0x5654bad65970 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:46.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.949+0000 7fae31a77740 1 --2- 172.21.15.77:0/958212950 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654bad53cb0 0x5654bad65430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:46.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.949+0000 7fae31a77740 1 -- 172.21.15.77:0/958212950 >> 172.21.15.77:0/958212950 conn(0x5654bad50240 msgr2=0x5654bad5da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:46.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.949+0000 7fae12000640 0 client.16005 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T09:57:46.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.949+0000 7fae31a77740 1 -- 172.21.15.77:0/958212950 shutdown_connections 2024-09-10T09:57:46.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.949+0000 7fae31a77740 1 -- 172.21.15.77:0/958212950 wait complete. 2024-09-10T09:57:46.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.949+0000 7fae31a77740 20 client.16005 trim_cache size 0 max 16384 2024-09-10T09:57:46.952 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-10T09:57:46.952 INFO:teuthology.orchestra.run.smithi077.stdout: "return_code": 0, 2024-09-10T09:57:46.952 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_tag": "542d64d4-46f0-4926-8168-90b562009115", 2024-09-10T09:57:46.952 INFO:teuthology.orchestra.run.smithi077.stdout: "mode": "asynchronous" 2024-09-10T09:57:46.952 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-10T09:57:46.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.950+0000 7fae2da00640 1 -- 172.21.15.77:0/3398607628 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fae0c0a2c00 msgr2=0x7fae0c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:46.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.950+0000 7fae2da00640 1 --2- 172.21.15.77:0/3398607628 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fae0c0a2c00 0x7fae0c0a50c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x5654bad515c0 tx=0x7fae14005f90 comp rx=0 tx=0).stop 2024-09-10T09:57:46.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.950+0000 7fae2da00640 1 -- 172.21.15.77:0/3398607628 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fae28108f60 msgr2=0x7fae2819ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:46.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.950+0000 7fae2da00640 1 --2- 172.21.15.77:0/3398607628 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fae28108f60 0x7fae2819ed30 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fae20009fd0 tx=0x7fae2000ef10 comp rx=0 tx=0).stop 2024-09-10T09:57:46.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.950+0000 7fae2da00640 1 -- 172.21.15.77:0/3398607628 shutdown_connections 2024-09-10T09:57:46.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.950+0000 7fae2da00640 1 --2- 172.21.15.77:0/3398607628 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fae0c0a2c00 0x7fae0c0a50c0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:46.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.950+0000 7fae2da00640 1 --2- 172.21.15.77:0/3398607628 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fae28108f60 0x7fae2819ed30 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:46.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.950+0000 7fae2da00640 1 --2- 172.21.15.77:0/3398607628 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fae281047e0 0x7fae2819a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:46.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.950+0000 7fae2da00640 1 --2- 172.21.15.77:0/3398607628 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae28103e30 0x7fae2819a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:46.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.950+0000 7fae2da00640 1 -- 172.21.15.77:0/3398607628 >> 172.21.15.77:0/3398607628 conn(0x7fae280ff910 msgr2=0x7fae28100d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:46.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.950+0000 7fae2da00640 1 -- 172.21.15.77:0/3398607628 shutdown_connections 2024-09-10T09:57:46.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:46.950+0000 7fae2da00640 1 -- 172.21.15.77:0/3398607628 wait complete. 2024-09-10T09:57:47.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:57:46 smithi120 ceph-mon[34770]: pgmap v469: 129 pgs: 129 active+clean; 29 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 735 KiB/s rd, 960 KiB/s wr, 44 op/s 2024-09-10T09:57:47.182 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:57:46 smithi134 ceph-mon[35674]: pgmap v469: 129 pgs: 129 active+clean; 29 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 735 KiB/s rd, 960 KiB/s wr, 44 op/s 2024-09-10T09:57:47.192 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:57:46 smithi077 ceph-mon[30319]: pgmap v469: 129 pgs: 129 active+clean; 29 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 735 KiB/s rd, 960 KiB/s wr, 44 op/s 2024-09-10T09:57:47.471 INFO:tasks.workunit.client.0.smithi077.stdout:libtoolize: putting auxiliary files in '.'. 2024-09-10T09:57:47.471 INFO:tasks.workunit.client.0.smithi077.stdout:libtoolize: copying file './config.guess' 2024-09-10T09:57:47.482 INFO:tasks.workunit.client.0.smithi077.stdout:libtoolize: copying file './config.sub' 2024-09-10T09:57:47.493 INFO:tasks.workunit.client.0.smithi077.stdout:libtoolize: copying file './install-sh' 2024-09-10T09:57:47.505 INFO:tasks.workunit.client.0.smithi077.stdout:libtoolize: copying file './ltmain.sh' 2024-09-10T09:57:47.566 INFO:tasks.workunit.client.0.smithi077.stdout:libtoolize: putting macros in AC_CONFIG_MACRO_DIRS, 'm4'. 2024-09-10T09:57:47.566 INFO:tasks.workunit.client.0.smithi077.stdout:libtoolize: copying file 'm4/libtool.m4' 2024-09-10T09:57:47.580 INFO:tasks.workunit.client.0.smithi077.stdout:libtoolize: copying file 'm4/ltoptions.m4' 2024-09-10T09:57:47.593 INFO:tasks.workunit.client.0.smithi077.stdout:libtoolize: copying file 'm4/ltsugar.m4' 2024-09-10T09:57:47.606 INFO:tasks.workunit.client.0.smithi077.stdout:libtoolize: copying file 'm4/ltversion.m4' 2024-09-10T09:57:47.620 INFO:tasks.workunit.client.0.smithi077.stdout:libtoolize: copying file 'm4/lt~obsolete.m4' 2024-09-10T09:57:47.637 INFO:tasks.workunit.client.0.smithi077.stdout:libtoolize: Consider adding '-I m4' to ACLOCAL_AMFLAGS in Makefile.am. 2024-09-10T09:57:47.640 INFO:tasks.workunit.client.0.smithi077.stdout:cp include/install-sh . 2024-09-10T09:57:47.651 INFO:tasks.workunit.client.0.smithi077.stdout:aclocal -I m4 2024-09-10T09:57:48.140 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:57:47 smithi077 ceph-mon[30319]: scrub queued for path: / 2024-09-10T09:57:48.141 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:57:47 smithi077 ceph-mon[30319]: scrub summary: idle+waiting paths [/] 2024-09-10T09:57:48.141 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:57:47 smithi077 ceph-mon[30319]: scrub summary: active paths [/] 2024-09-10T09:57:48.141 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:57:47 smithi077 ceph-mon[30319]: scrub summary: active paths [/] 2024-09-10T09:57:48.141 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:57:47 smithi077 ceph-mon[30319]: scrub summary: active paths [/] 2024-09-10T09:57:48.141 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:57:47 smithi077 ceph-mon[30319]: scrub summary: active paths [/] 2024-09-10T09:57:48.141 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:57:47 smithi077 ceph-mon[30319]: scrub summary: idle+waiting paths [/] 2024-09-10T09:57:48.141 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:57:47 smithi077 ceph-mon[30319]: scrub summary: active paths [/] 2024-09-10T09:57:48.141 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:57:47 smithi077 ceph-mon[30319]: scrub summary: idle+waiting paths [/] 2024-09-10T09:57:48.141 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:57:47 smithi077 ceph-mon[30319]: scrub summary: idle+waiting paths [/] 2024-09-10T09:57:48.141 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:57:47 smithi077 ceph-mon[30319]: scrub summary: idle+waiting paths [/] 2024-09-10T09:57:48.141 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:57:47 smithi077 ceph-mon[30319]: scrub summary: active paths [/] 2024-09-10T09:57:48.141 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:57:47 smithi077 ceph-mon[30319]: scrub summary: active paths [/] 2024-09-10T09:57:48.141 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:57:47 smithi077 ceph-mon[30319]: scrub summary: active paths [/] 2024-09-10T09:57:48.142 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:57:47 smithi077 ceph-mon[30319]: scrub summary: active paths [/] 2024-09-10T09:57:48.142 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:57:47 smithi077 ceph-mon[30319]: scrub summary: idle+waiting paths [/] 2024-09-10T09:57:48.142 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:57:47 smithi077 ceph-mon[30319]: scrub summary: active paths [/] 2024-09-10T09:57:48.142 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:57:47 smithi077 ceph-mon[30319]: scrub summary: idle+waiting paths [/] 2024-09-10T09:57:48.142 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:57:47 smithi077 ceph-mon[30319]: scrub summary: idle+waiting paths [/] 2024-09-10T09:57:48.142 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:57:47 smithi077 ceph-mon[30319]: scrub summary: idle+waiting paths [/] 2024-09-10T09:57:48.142 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:57:47 smithi077 ceph-mon[30319]: scrub summary: idle+waiting paths [/] 2024-09-10T09:57:48.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:57:47 smithi120 ceph-mon[34770]: scrub queued for path: / 2024-09-10T09:57:48.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:57:47 smithi120 ceph-mon[34770]: scrub summary: idle+waiting paths [/] 2024-09-10T09:57:48.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:57:47 smithi120 ceph-mon[34770]: scrub summary: active paths [/] 2024-09-10T09:57:48.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:57:47 smithi120 ceph-mon[34770]: scrub summary: active paths [/] 2024-09-10T09:57:48.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:57:47 smithi120 ceph-mon[34770]: scrub summary: active paths [/] 2024-09-10T09:57:48.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:57:47 smithi120 ceph-mon[34770]: scrub summary: active paths [/] 2024-09-10T09:57:48.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:57:47 smithi120 ceph-mon[34770]: scrub summary: idle+waiting paths [/] 2024-09-10T09:57:48.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:57:47 smithi120 ceph-mon[34770]: scrub summary: active paths [/] 2024-09-10T09:57:48.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:57:47 smithi120 ceph-mon[34770]: scrub summary: idle+waiting paths [/] 2024-09-10T09:57:48.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:57:47 smithi120 ceph-mon[34770]: scrub summary: idle+waiting paths [/] 2024-09-10T09:57:48.175 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:57:47 smithi120 ceph-mon[34770]: scrub summary: idle+waiting paths [/] 2024-09-10T09:57:48.175 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:57:47 smithi120 ceph-mon[34770]: scrub summary: active paths [/] 2024-09-10T09:57:48.175 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:57:47 smithi120 ceph-mon[34770]: scrub summary: active paths [/] 2024-09-10T09:57:48.175 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:57:47 smithi120 ceph-mon[34770]: scrub summary: active paths [/] 2024-09-10T09:57:48.175 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:57:47 smithi120 ceph-mon[34770]: scrub summary: active paths [/] 2024-09-10T09:57:48.175 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:57:47 smithi120 ceph-mon[34770]: scrub summary: idle+waiting paths [/] 2024-09-10T09:57:48.175 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:57:47 smithi120 ceph-mon[34770]: scrub summary: active paths [/] 2024-09-10T09:57:48.175 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:57:47 smithi120 ceph-mon[34770]: scrub summary: idle+waiting paths [/] 2024-09-10T09:57:48.175 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:57:47 smithi120 ceph-mon[34770]: scrub summary: idle+waiting paths [/] 2024-09-10T09:57:48.175 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:57:47 smithi120 ceph-mon[34770]: scrub summary: idle+waiting paths [/] 2024-09-10T09:57:48.175 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:57:47 smithi120 ceph-mon[34770]: scrub summary: idle+waiting paths [/] 2024-09-10T09:57:48.184 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T09:57:48.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:57:47 smithi134 ceph-mon[35674]: scrub queued for path: / 2024-09-10T09:57:48.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:57:47 smithi134 ceph-mon[35674]: scrub summary: idle+waiting paths [/] 2024-09-10T09:57:48.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:57:47 smithi134 ceph-mon[35674]: scrub summary: active paths [/] 2024-09-10T09:57:48.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:57:47 smithi134 ceph-mon[35674]: scrub summary: active paths [/] 2024-09-10T09:57:48.382 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:57:47 smithi134 ceph-mon[35674]: scrub summary: active paths [/] 2024-09-10T09:57:48.382 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:57:47 smithi134 ceph-mon[35674]: scrub summary: active paths [/] 2024-09-10T09:57:48.382 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:57:47 smithi134 ceph-mon[35674]: scrub summary: idle+waiting paths [/] 2024-09-10T09:57:48.382 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:57:47 smithi134 ceph-mon[35674]: scrub summary: active paths [/] 2024-09-10T09:57:48.382 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:57:47 smithi134 ceph-mon[35674]: scrub summary: idle+waiting paths [/] 2024-09-10T09:57:48.382 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:57:47 smithi134 ceph-mon[35674]: scrub summary: idle+waiting paths [/] 2024-09-10T09:57:48.382 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:57:47 smithi134 ceph-mon[35674]: scrub summary: idle+waiting paths [/] 2024-09-10T09:57:48.382 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:57:47 smithi134 ceph-mon[35674]: scrub summary: active paths [/] 2024-09-10T09:57:48.382 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:57:47 smithi134 ceph-mon[35674]: scrub summary: active paths [/] 2024-09-10T09:57:48.382 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:57:47 smithi134 ceph-mon[35674]: scrub summary: active paths [/] 2024-09-10T09:57:48.382 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:57:47 smithi134 ceph-mon[35674]: scrub summary: active paths [/] 2024-09-10T09:57:48.382 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:57:47 smithi134 ceph-mon[35674]: scrub summary: idle+waiting paths [/] 2024-09-10T09:57:48.382 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:57:47 smithi134 ceph-mon[35674]: scrub summary: active paths [/] 2024-09-10T09:57:48.382 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:57:47 smithi134 ceph-mon[35674]: scrub summary: idle+waiting paths [/] 2024-09-10T09:57:48.382 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:57:47 smithi134 ceph-mon[35674]: scrub summary: idle+waiting paths [/] 2024-09-10T09:57:48.382 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:57:47 smithi134 ceph-mon[35674]: scrub summary: idle+waiting paths [/] 2024-09-10T09:57:48.383 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:57:47 smithi134 ceph-mon[35674]: scrub summary: idle+waiting paths [/] 2024-09-10T09:57:48.580 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:57:49.062 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:57:48 smithi077 ceph-mon[30319]: scrub summary: idle+waiting paths [/] 2024-09-10T09:57:49.062 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:57:48 smithi077 ceph-mon[30319]: pgmap v470: 129 pgs: 129 active+clean; 38 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 87 op/s 2024-09-10T09:57:49.071 INFO:tasks.workunit.client.0.smithi077.stdout:autoheader 2024-09-10T09:57:49.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:57:48 smithi120 ceph-mon[34770]: scrub summary: idle+waiting paths [/] 2024-09-10T09:57:49.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:57:48 smithi120 ceph-mon[34770]: pgmap v470: 129 pgs: 129 active+clean; 38 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 87 op/s 2024-09-10T09:57:49.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.176+0000 7f7f32200640 1 -- 172.21.15.77:0/541368545 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f2c109140 msgr2=0x7f7f2c110cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:49.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.176+0000 7f7f32200640 1 --2- 172.21.15.77:0/541368545 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f2c109140 0x7f7f2c110cc0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f7f24009ca0 tx=0x7f7f2402a8c0 comp rx=0 tx=0).stop 2024-09-10T09:57:49.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.176+0000 7f7f32200640 1 -- 172.21.15.77:0/541368545 shutdown_connections 2024-09-10T09:57:49.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.176+0000 7f7f32200640 1 --2- 172.21.15.77:0/541368545 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f2c109140 0x7f7f2c110cc0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:49.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.176+0000 7f7f32200640 1 --2- 172.21.15.77:0/541368545 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f2c1049c0 0x7f7f2c108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:49.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.176+0000 7f7f32200640 1 --2- 172.21.15.77:0/541368545 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7f2c104010 0x7f7f2c1043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:49.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.176+0000 7f7f32200640 1 -- 172.21.15.77:0/541368545 >> 172.21.15.77:0/541368545 conn(0x7f7f2c0ffa50 msgr2=0x7f7f2c101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:49.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.176+0000 7f7f32200640 1 -- 172.21.15.77:0/541368545 shutdown_connections 2024-09-10T09:57:49.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.176+0000 7f7f32200640 1 -- 172.21.15.77:0/541368545 wait complete. 2024-09-10T09:57:49.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.177+0000 7f7f32200640 1 Processor -- start 2024-09-10T09:57:49.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.177+0000 7f7f32200640 1 -- start start 2024-09-10T09:57:49.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.178+0000 7f7f32200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f2c104010 0x7f7f2c19a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:49.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.178+0000 7f7f32200640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7f2c1049c0 0x7f7f2c19ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:49.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.178+0000 7f7f32200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f2c109140 0x7f7f2c19ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:49.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.178+0000 7f7f32200640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7f2c1141c0 con 0x7f7f2c104010 2024-09-10T09:57:49.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.178+0000 7f7f32200640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7f2c114040 con 0x7f7f2c1049c0 2024-09-10T09:57:49.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.178+0000 7f7f32200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7f2c114340 con 0x7f7f2c109140 2024-09-10T09:57:49.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.178+0000 7f7f2b400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f2c104010 0x7f7f2c19a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:49.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.178+0000 7f7f2b400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f2c104010 0x7f7f2c19a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52546/0 (socket says 172.21.15.77:52546) 2024-09-10T09:57:49.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.178+0000 7f7f2b400640 1 -- 172.21.15.77:0/2920085348 learned_addr learned my addr 172.21.15.77:0/2920085348 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:57:49.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.178+0000 7f7f22a00640 1 --2- 172.21.15.77:0/2920085348 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7f2c1049c0 0x7f7f2c19ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:49.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.178+0000 7f7f2be00640 1 --2- 172.21.15.77:0/2920085348 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f2c109140 0x7f7f2c19ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:49.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.179+0000 7f7f2b400640 1 -- 172.21.15.77:0/2920085348 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f2c109140 msgr2=0x7f7f2c19ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:49.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.179+0000 7f7f2b400640 1 --2- 172.21.15.77:0/2920085348 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f2c109140 0x7f7f2c19ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:49.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.179+0000 7f7f2b400640 1 -- 172.21.15.77:0/2920085348 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7f2c1049c0 msgr2=0x7f7f2c19ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:49.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.179+0000 7f7f2b400640 1 --2- 172.21.15.77:0/2920085348 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7f2c1049c0 0x7f7f2c19ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:49.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.179+0000 7f7f2b400640 1 -- 172.21.15.77:0/2920085348 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f2c19f670 con 0x7f7f2c104010 2024-09-10T09:57:49.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.179+0000 7f7f22a00640 1 --2- 172.21.15.77:0/2920085348 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7f2c1049c0 0x7f7f2c19ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:57:49.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.179+0000 7f7f2b400640 1 --2- 172.21.15.77:0/2920085348 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f2c104010 0x7f7f2c19a6c0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c00e9f0 tx=0x7f7f1c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:49.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.179+0000 7f7f28c00640 1 -- 172.21.15.77:0/2920085348 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f1c00cd80 con 0x7f7f2c104010 2024-09-10T09:57:49.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.179+0000 7f7f28c00640 1 -- 172.21.15.77:0/2920085348 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f1c010910 con 0x7f7f2c104010 2024-09-10T09:57:49.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.179+0000 7f7f28c00640 1 -- 172.21.15.77:0/2920085348 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f1c018990 con 0x7f7f2c104010 2024-09-10T09:57:49.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.179+0000 7f7f32200640 1 -- 172.21.15.77:0/2920085348 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f2c19f960 con 0x7f7f2c104010 2024-09-10T09:57:49.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.179+0000 7f7f32200640 1 -- 172.21.15.77:0/2920085348 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f2c1a7240 con 0x7f7f2c104010 2024-09-10T09:57:49.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.183+0000 7f7f28c00640 1 -- 172.21.15.77:0/2920085348 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7f1c010ac0 con 0x7f7f2c104010 2024-09-10T09:57:49.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.184+0000 7f7f28c00640 1 --2- 172.21.15.77:0/2920085348 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f7f100a2b80 0x7f7f100a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:49.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.184+0000 7f7f34ca3740 1 Processor -- start 2024-09-10T09:57:49.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.184+0000 7f7f28c00640 1 -- 172.21.15.77:0/2920085348 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f7f1c0cf790 con 0x7f7f2c104010 2024-09-10T09:57:49.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.185+0000 7f7f34ca3740 1 -- start start 2024-09-10T09:57:49.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.185+0000 7f7f22a00640 1 --2- 172.21.15.77:0/2920085348 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f7f100a2b80 0x7f7f100a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:49.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.185+0000 7f7f34ca3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d72316b90 0x560d72316f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:49.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.185+0000 7f7f34ca3740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560d723174b0 0x560d72321830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:49.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.185+0000 7f7f34ca3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d72321d70 0x560d72324160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:49.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.185+0000 7f7f34ca3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560d7227d3e0 con 0x560d72321d70 2024-09-10T09:57:49.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.185+0000 7f7f34ca3740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x560d72224d80 con 0x560d723174b0 2024-09-10T09:57:49.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.185+0000 7f7f34ca3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560d722793a0 con 0x560d72316b90 2024-09-10T09:57:49.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.185+0000 7f7f22a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560d723174b0 0x560d72321830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:49.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.185+0000 7f7f2b400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d72316b90 0x560d72316f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:49.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.186+0000 7f7f22a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560d723174b0 0x560d72321830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:33386/0 (socket says 172.21.15.77:33386) 2024-09-10T09:57:49.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.186+0000 7f7f22a00640 1 -- 172.21.15.77:0/2384518636 learned_addr learned my addr 172.21.15.77:0/2384518636 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:57:49.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.186+0000 7f7f22a00640 1 --2- 172.21.15.77:0/2920085348 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f7f100a2b80 0x7f7f100a5040 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c10ab10 tx=0x7f7f14005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:49.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.186+0000 7f7f2b400640 1 -- 172.21.15.77:0/2384518636 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560d723174b0 msgr2=0x560d72321830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:49.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.186+0000 7f7f2b400640 1 --2- 172.21.15.77:0/2384518636 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560d723174b0 0x560d72321830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:49.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.186+0000 7f7f2b400640 1 -- 172.21.15.77:0/2384518636 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d72321d70 msgr2=0x560d72324160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:49.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.186+0000 7f7f2b400640 1 --2- 172.21.15.77:0/2384518636 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d72321d70 0x560d72324160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:49.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.186+0000 7f7f2b400640 1 -- 172.21.15.77:0/2384518636 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d72243ae0 con 0x560d72316b90 2024-09-10T09:57:49.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.186+0000 7f7f22a00640 1 --2- 172.21.15.77:0/2384518636 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560d723174b0 0x560d72321830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:57:49.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.186+0000 7f7f2b400640 1 --2- 172.21.15.77:0/2384518636 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d72316b90 0x560d72316f70 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c00e9c0 tx=0x7f7f1c09bd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af0680a56bb8a110 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:49.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.187+0000 7f7f1be00640 1 -- 172.21.15.77:0/2384518636 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f1c099cf0 con 0x560d72316b90 2024-09-10T09:57:49.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.187+0000 7f7f1be00640 1 -- 172.21.15.77:0/2384518636 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f1c099f50 con 0x560d72316b90 2024-09-10T09:57:49.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.187+0000 7f7f1be00640 1 -- 172.21.15.77:0/2384518636 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f1c018e20 con 0x560d72316b90 2024-09-10T09:57:49.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.187+0000 7f7f34ca3740 1 -- 172.21.15.77:0/2384518636 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d72316b90 msgr2=0x560d72316f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:49.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.187+0000 7f7f34ca3740 1 --2- 172.21.15.77:0/2384518636 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d72316b90 0x560d72316f70 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c00e9c0 tx=0x7f7f1c09bd10 comp rx=0 tx=0).stop 2024-09-10T09:57:49.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.188+0000 7f7f34ca3740 1 -- 172.21.15.77:0/2384518636 shutdown_connections 2024-09-10T09:57:49.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.188+0000 7f7f34ca3740 1 --2- 172.21.15.77:0/2384518636 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d72321d70 0x560d72324160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:49.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.188+0000 7f7f34ca3740 1 --2- 172.21.15.77:0/2384518636 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560d723174b0 0x560d72321830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:49.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.188+0000 7f7f34ca3740 1 --2- 172.21.15.77:0/2384518636 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d72316b90 0x560d72316f70 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:49.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.188+0000 7f7f34ca3740 1 -- 172.21.15.77:0/2384518636 >> 172.21.15.77:0/2384518636 conn(0x560d72313120 msgr2=0x560d72313500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:49.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.188+0000 7f7f34ca3740 1 -- 172.21.15.77:0/2384518636 shutdown_connections 2024-09-10T09:57:49.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.188+0000 7f7f34ca3740 1 -- 172.21.15.77:0/2384518636 wait complete. 2024-09-10T09:57:49.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.188+0000 7f7f34ca3740 1 -- start start 2024-09-10T09:57:49.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.188+0000 7f7f34ca3740 1 Processor -- start 2024-09-10T09:57:49.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.189+0000 7f7f34ca3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d72316b90 0x560d7231ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:49.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.189+0000 7f7f34ca3740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560d723174b0 0x560d7231f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:49.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.189+0000 7f7f34ca3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d72321d70 0x560d7231f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:49.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.189+0000 7f7f34ca3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560d723249e0 con 0x560d72321d70 2024-09-10T09:57:49.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.189+0000 7f7f34ca3740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x560d722380f0 con 0x560d723174b0 2024-09-10T09:57:49.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.189+0000 7f7f34ca3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560d72324b60 con 0x560d72316b90 2024-09-10T09:57:49.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.189+0000 7f7f34ca3740 10 client.0 _ll_register_callbacks cb 0x560d722299e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:57:49.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.189+0000 7f7f22a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560d723174b0 0x560d7231f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:49.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.189+0000 7f7f22a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560d723174b0 0x560d7231f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:33400/0 (socket says 172.21.15.77:33400) 2024-09-10T09:57:49.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.189+0000 7f7f22a00640 1 -- 172.21.15.77:0/4226301928 learned_addr learned my addr 172.21.15.77:0/4226301928 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:57:49.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.189+0000 7f7f2b400640 1 --2- 172.21.15.77:0/4226301928 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d72316b90 0x560d7231ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:49.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.190+0000 7f7f2be00640 1 --2- 172.21.15.77:0/4226301928 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d72321d70 0x560d7231f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:49.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.190+0000 7f7f22a00640 1 -- 172.21.15.77:0/4226301928 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d72316b90 msgr2=0x560d7231ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:49.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.190+0000 7f7f22a00640 1 --2- 172.21.15.77:0/4226301928 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d72316b90 0x560d7231ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:49.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.190+0000 7f7f22a00640 1 -- 172.21.15.77:0/4226301928 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d72321d70 msgr2=0x560d7231f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:49.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.190+0000 7f7f22a00640 1 --2- 172.21.15.77:0/4226301928 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d72321d70 0x560d7231f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:49.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.190+0000 7f7f22a00640 1 -- 172.21.15.77:0/4226301928 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d71fee3c0 con 0x560d723174b0 2024-09-10T09:57:49.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.190+0000 7f7f2b400640 1 --2- 172.21.15.77:0/4226301928 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d72316b90 0x560d7231ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:57:49.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.190+0000 7f7f22a00640 1 --2- 172.21.15.77:0/4226301928 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560d723174b0 0x560d7231f220 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7f1400fad0 tx=0x7f7f14004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bef4223fa01b7790 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:49.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.190+0000 7f7f18c00640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T09:57:49.194 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.191+0000 7f7f18c00640 1 -- 172.21.15.77:0/4226301928 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f1401cd10 con 0x560d723174b0 2024-09-10T09:57:49.194 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.191+0000 7f7f34ca3740 1 -- 172.21.15.77:0/4226301928 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f1c099f50 con 0x560d723174b0 2024-09-10T09:57:49.194 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.191+0000 7f7f18c00640 1 -- 172.21.15.77:0/4226301928 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f7f14022040 con 0x560d723174b0 2024-09-10T09:57:49.194 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.191+0000 7f7f18c00640 1 -- 172.21.15.77:0/4226301928 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7f14032050 con 0x560d723174b0 2024-09-10T09:57:49.194 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.191+0000 7f7f34ca3740 10 client.26093 fetch_fsmap learned FSMap version 32 2024-09-10T09:57:49.194 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.191+0000 7f7f34ca3740 1 -- 172.21.15.77:0/4226301928 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560d723307c0 con 0x560d723174b0 2024-09-10T09:57:49.194 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.191+0000 7f7f18c00640 1 -- 172.21.15.77:0/4226301928 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f1c099f50 con 0x560d723174b0 2024-09-10T09:57:49.194 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.191+0000 7f7f18c00640 1 -- 172.21.15.77:0/4226301928 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f7f1401a050 con 0x560d723174b0 2024-09-10T09:57:49.195 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.192+0000 7f7f34ca3740 10 client.26093 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T09:57:49.195 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.192+0000 7f7f34ca3740 10 client.26093 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:57:49.195 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.192+0000 7f7f34ca3740 20 client.26093 populate_metadata read hostname 'smithi077' 2024-09-10T09:57:49.195 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.192+0000 7f7f34ca3740 1 --2- 172.21.15.77:0/4226301928 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x560d72331c90 0x560d72334100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:49.195 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.192+0000 7f7f34ca3740 4 client.26093 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:57:49.195 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.192+0000 7f7f34ca3740 1 -- 172.21.15.77:0/4226301928 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d7228a950 con 0x560d72331c90 2024-09-10T09:57:49.195 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.192+0000 7f7f2b400640 1 --2- 172.21.15.77:0/4226301928 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x560d72331c90 0x560d72334100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:49.195 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.193+0000 7f7f2b400640 1 --2- 172.21.15.77:0/4226301928 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x560d72331c90 0x560d72334100 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=891c30430eda948d server_cookie=8f3a572b417a91e3 in_seq=0 out_seq=0 2024-09-10T09:57:49.195 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.193+0000 7f7f18c00640 10 client.26093 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T09:57:49.195 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.193+0000 7f7f18c00640 1 -- 172.21.15.77:0/4226301928 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560d7228a950 con 0x560d72331c90 2024-09-10T09:57:49.195 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.193+0000 7f7f18c00640 10 client.26093 handle_command_reply: tid=0 2024-09-10T09:57:49.195 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.193+0000 7f7f34ca3740 1 client.26093 shutdown 2024-09-10T09:57:49.196 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.194+0000 7f7f34ca3740 1 -- 172.21.15.77:0/4226301928 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560d723174b0 msgr2=0x560d7231f220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:49.196 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.194+0000 7f7f34ca3740 1 --2- 172.21.15.77:0/4226301928 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560d723174b0 0x560d7231f220 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7f1400fad0 tx=0x7f7f14004730 comp rx=0 tx=0).stop 2024-09-10T09:57:49.196 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.194+0000 7f7f34ca3740 1 -- 172.21.15.77:0/4226301928 shutdown_connections 2024-09-10T09:57:49.196 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.194+0000 7f7f34ca3740 1 --2- 172.21.15.77:0/4226301928 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x560d72331c90 0x560d72334100 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:49.196 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.194+0000 7f7f34ca3740 1 --2- 172.21.15.77:0/4226301928 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d72321d70 0x560d7231f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:49.196 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.194+0000 7f7f34ca3740 1 --2- 172.21.15.77:0/4226301928 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560d723174b0 0x560d7231f220 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:49.196 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.194+0000 7f7f34ca3740 1 --2- 172.21.15.77:0/4226301928 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d72316b90 0x560d7231ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:49.196 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.194+0000 7f7f34ca3740 1 -- 172.21.15.77:0/4226301928 >> 172.21.15.77:0/4226301928 conn(0x560d72313120 msgr2=0x560d72318e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:49.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.194+0000 7f7f18c00640 0 client.26093 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T09:57:49.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.194+0000 7f7f34ca3740 1 -- 172.21.15.77:0/4226301928 shutdown_connections 2024-09-10T09:57:49.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.194+0000 7f7f34ca3740 1 -- 172.21.15.77:0/4226301928 wait complete. 2024-09-10T09:57:49.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.194+0000 7f7f34ca3740 20 client.26093 trim_cache size 0 max 16384 2024-09-10T09:57:49.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.212+0000 7f7f34ca3740 1 Processor -- start 2024-09-10T09:57:49.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.212+0000 7f7f34ca3740 1 -- start start 2024-09-10T09:57:49.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.212+0000 7f7f34ca3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d72316b90 0x560d72318cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:49.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.212+0000 7f7f34ca3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d723174b0 0x560d72339a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:49.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.212+0000 7f7f34ca3740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560d72321d70 0x560d72339f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:49.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.212+0000 7f7f34ca3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7f2c1b11f0 con 0x560d72316b90 2024-09-10T09:57:49.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.212+0000 7f7f34ca3740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7f2c1b2df0 con 0x560d72321d70 2024-09-10T09:57:49.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.212+0000 7f7f34ca3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7f2c1b18c0 con 0x560d723174b0 2024-09-10T09:57:49.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.213+0000 7f7f2b400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d72316b90 0x560d72318cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:49.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.213+0000 7f7f2b400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d72316b90 0x560d72318cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52580/0 (socket says 172.21.15.77:52580) 2024-09-10T09:57:49.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.213+0000 7f7f2b400640 1 -- 172.21.15.77:0/3577386264 learned_addr learned my addr 172.21.15.77:0/3577386264 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:57:49.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.213+0000 7f7f2be00640 1 --2- 172.21.15.77:0/3577386264 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560d72321d70 0x560d72339f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:49.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.213+0000 7f7f22a00640 1 --2- 172.21.15.77:0/3577386264 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d723174b0 0x560d72339a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:49.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.213+0000 7f7f2be00640 1 -- 172.21.15.77:0/3577386264 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d723174b0 msgr2=0x560d72339a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:49.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.213+0000 7f7f2be00640 1 --2- 172.21.15.77:0/3577386264 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d723174b0 0x560d72339a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:49.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.213+0000 7f7f2be00640 1 -- 172.21.15.77:0/3577386264 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d72316b90 msgr2=0x560d72318cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:49.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.213+0000 7f7f2be00640 1 --2- 172.21.15.77:0/3577386264 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d72316b90 0x560d72318cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:49.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.213+0000 7f7f2be00640 1 -- 172.21.15.77:0/3577386264 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d72334470 con 0x560d72321d70 2024-09-10T09:57:49.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.213+0000 7f7f22a00640 1 --2- 172.21.15.77:0/3577386264 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d723174b0 0x560d72339a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:57:49.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.214+0000 7f7f2b400640 1 --2- 172.21.15.77:0/3577386264 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d72316b90 0x560d72318cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:57:49.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.214+0000 7f7f2be00640 1 --2- 172.21.15.77:0/3577386264 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560d72321d70 0x560d72339f50 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7f24005e30 tx=0x7f7f2402f3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=501e1f3292ad34e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:49.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.214+0000 7f7f1b400640 1 -- 172.21.15.77:0/3577386264 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f24031bb0 con 0x560d72321d70 2024-09-10T09:57:49.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.214+0000 7f7f1b400640 1 -- 172.21.15.77:0/3577386264 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f24036050 con 0x560d72321d70 2024-09-10T09:57:49.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.214+0000 7f7f1b400640 1 -- 172.21.15.77:0/3577386264 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f240042a0 con 0x560d72321d70 2024-09-10T09:57:49.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.215+0000 7f7f34ca3740 1 -- 172.21.15.77:0/3577386264 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560d72321d70 msgr2=0x560d72339f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:49.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.215+0000 7f7f34ca3740 1 --2- 172.21.15.77:0/3577386264 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560d72321d70 0x560d72339f50 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7f24005e30 tx=0x7f7f2402f3c0 comp rx=0 tx=0).stop 2024-09-10T09:57:49.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.215+0000 7f7f34ca3740 1 -- 172.21.15.77:0/3577386264 shutdown_connections 2024-09-10T09:57:49.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.215+0000 7f7f34ca3740 1 --2- 172.21.15.77:0/3577386264 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560d72321d70 0x560d72339f50 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:49.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.215+0000 7f7f34ca3740 1 --2- 172.21.15.77:0/3577386264 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d723174b0 0x560d72339a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:49.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.215+0000 7f7f34ca3740 1 --2- 172.21.15.77:0/3577386264 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d72316b90 0x560d72318cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:49.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.215+0000 7f7f34ca3740 1 -- 172.21.15.77:0/3577386264 >> 172.21.15.77:0/3577386264 conn(0x560d72313120 msgr2=0x560d723185e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:49.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.215+0000 7f7f34ca3740 1 -- 172.21.15.77:0/3577386264 shutdown_connections 2024-09-10T09:57:49.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.215+0000 7f7f34ca3740 1 -- 172.21.15.77:0/3577386264 wait complete. 2024-09-10T09:57:49.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.216+0000 7f7f34ca3740 1 -- start start 2024-09-10T09:57:49.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.216+0000 7f7f34ca3740 1 Processor -- start 2024-09-10T09:57:49.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.216+0000 7f7f34ca3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d72316b90 0x560d7231bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:49.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.216+0000 7f7f34ca3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d723174b0 0x560d7231c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:49.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.216+0000 7f7f34ca3740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560d72321d70 0x560d7231c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:49.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.216+0000 7f7f34ca3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7f2c1b2660 con 0x560d72316b90 2024-09-10T09:57:49.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.216+0000 7f7f34ca3740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7f2c1b1f90 con 0x560d72321d70 2024-09-10T09:57:49.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.216+0000 7f7f34ca3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7f2c1af1d0 con 0x560d723174b0 2024-09-10T09:57:49.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.217+0000 7f7f34ca3740 10 client.0 _ll_register_callbacks cb 0x7f7f1400f9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:57:49.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.217+0000 7f7f2b400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d72316b90 0x560d7231bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:49.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.217+0000 7f7f2b400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d72316b90 0x560d7231bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52584/0 (socket says 172.21.15.77:52584) 2024-09-10T09:57:49.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.217+0000 7f7f2b400640 1 -- 172.21.15.77:0/2249127682 learned_addr learned my addr 172.21.15.77:0/2249127682 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:57:49.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.217+0000 7f7f22a00640 1 --2- 172.21.15.77:0/2249127682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d723174b0 0x560d7231c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:49.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.217+0000 7f7f2be00640 1 --2- 172.21.15.77:0/2249127682 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560d72321d70 0x560d7231c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:49.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.217+0000 7f7f22a00640 1 -- 172.21.15.77:0/2249127682 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560d72321d70 msgr2=0x560d7231c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:49.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.217+0000 7f7f22a00640 1 --2- 172.21.15.77:0/2249127682 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560d72321d70 0x560d7231c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:49.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.217+0000 7f7f22a00640 1 -- 172.21.15.77:0/2249127682 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d72316b90 msgr2=0x560d7231bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:49.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.217+0000 7f7f22a00640 1 --2- 172.21.15.77:0/2249127682 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d72316b90 0x560d7231bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:49.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.217+0000 7f7f22a00640 1 -- 172.21.15.77:0/2249127682 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d7239bbe0 con 0x560d723174b0 2024-09-10T09:57:49.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.217+0000 7f7f2b400640 1 --2- 172.21.15.77:0/2249127682 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d72316b90 0x560d7231bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:57:49.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.218+0000 7f7f22a00640 1 --2- 172.21.15.77:0/2249127682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d723174b0 0x560d7231c410 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7f14003ba0 tx=0x7f7f140038a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c7244cbd23cf7dc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:49.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.218+0000 7f7f18c00640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-10T09:57:49.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.218+0000 7f7f18c00640 1 -- 172.21.15.77:0/2249127682 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f1401c7d0 con 0x560d723174b0 2024-09-10T09:57:49.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.218+0000 7f7f34ca3740 1 -- 172.21.15.77:0/2249127682 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f24036050 con 0x560d723174b0 2024-09-10T09:57:49.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.218+0000 7f7f18c00640 1 -- 172.21.15.77:0/2249127682 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f7f14022040 con 0x560d723174b0 2024-09-10T09:57:49.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.218+0000 7f7f18c00640 1 -- 172.21.15.77:0/2249127682 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7f1403d050 con 0x560d723174b0 2024-09-10T09:57:49.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.218+0000 7f7f34ca3740 10 client.26047 fetch_fsmap learned FSMap version 32 2024-09-10T09:57:49.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.218+0000 7f7f34ca3740 1 -- 172.21.15.77:0/2249127682 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560d7239ce90 con 0x560d723174b0 2024-09-10T09:57:49.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.219+0000 7f7f18c00640 1 -- 172.21.15.77:0/2249127682 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f24036050 con 0x560d723174b0 2024-09-10T09:57:49.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.219+0000 7f7f18c00640 1 -- 172.21.15.77:0/2249127682 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f7f1401a040 con 0x560d723174b0 2024-09-10T09:57:49.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.219+0000 7f7f34ca3740 10 client.26047 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T09:57:49.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.219+0000 7f7f34ca3740 10 client.26047 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:57:49.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.219+0000 7f7f34ca3740 20 client.26047 populate_metadata read hostname 'smithi077' 2024-09-10T09:57:49.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.219+0000 7f7f34ca3740 1 --2- 172.21.15.77:0/2249127682 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x560d7239d6d0 0x560d7239faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:49.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.219+0000 7f7f34ca3740 4 client.26047 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T09:57:49.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.219+0000 7f7f34ca3740 1 -- 172.21.15.77:0/2249127682 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560d72284e60 con 0x560d7239d6d0 2024-09-10T09:57:49.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.220+0000 7f7f2b400640 1 --2- 172.21.15.77:0/2249127682 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x560d7239d6d0 0x560d7239faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:49.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.220+0000 7f7f2b400640 1 --2- 172.21.15.77:0/2249127682 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x560d7239d6d0 0x560d7239faf0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e4c828608e8ff86 server_cookie=c97dec99ae7c500a in_seq=0 out_seq=0 2024-09-10T09:57:49.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.220+0000 7f7f18c00640 10 client.26047 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T09:57:49.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.220+0000 7f7f18c00640 1 -- 172.21.15.77:0/2249127682 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x560d72284e60 con 0x560d7239d6d0 2024-09-10T09:57:49.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.220+0000 7f7f18c00640 10 client.26047 handle_command_reply: tid=0 2024-09-10T09:57:49.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.220+0000 7f7f34ca3740 1 client.26047 shutdown 2024-09-10T09:57:49.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.221+0000 7f7f34ca3740 1 -- 172.21.15.77:0/2249127682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d723174b0 msgr2=0x560d7231c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:49.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.221+0000 7f7f34ca3740 1 --2- 172.21.15.77:0/2249127682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d723174b0 0x560d7231c410 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7f14003ba0 tx=0x7f7f140038a0 comp rx=0 tx=0).stop 2024-09-10T09:57:49.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.221+0000 7f7f34ca3740 1 -- 172.21.15.77:0/2249127682 shutdown_connections 2024-09-10T09:57:49.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.221+0000 7f7f34ca3740 1 --2- 172.21.15.77:0/2249127682 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x560d7239d6d0 0x560d7239faf0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:49.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.221+0000 7f7f34ca3740 1 --2- 172.21.15.77:0/2249127682 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560d72321d70 0x560d7231c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:49.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.221+0000 7f7f34ca3740 1 --2- 172.21.15.77:0/2249127682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d723174b0 0x560d7231c410 secure :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7f14003ba0 tx=0x7f7f140038a0 comp rx=0 tx=0).stop 2024-09-10T09:57:49.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.221+0000 7f7f34ca3740 1 --2- 172.21.15.77:0/2249127682 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d72316b90 0x560d7231bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:49.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.221+0000 7f7f34ca3740 1 -- 172.21.15.77:0/2249127682 >> 172.21.15.77:0/2249127682 conn(0x560d72313120 msgr2=0x560d723121c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:49.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.221+0000 7f7f18c00640 0 client.26047 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T09:57:49.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.222+0000 7f7f34ca3740 1 -- 172.21.15.77:0/2249127682 shutdown_connections 2024-09-10T09:57:49.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.222+0000 7f7f34ca3740 1 -- 172.21.15.77:0/2249127682 wait complete. 2024-09-10T09:57:49.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.222+0000 7f7f34ca3740 20 client.26047 trim_cache size 0 max 16384 2024-09-10T09:57:49.224 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-10T09:57:49.224 INFO:teuthology.orchestra.run.smithi077.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T09:57:49.224 INFO:teuthology.orchestra.run.smithi077.stdout: "scrubs": { 2024-09-10T09:57:49.224 INFO:teuthology.orchestra.run.smithi077.stdout: "542d64d4-46f0-4926-8168-90b562009115": { 2024-09-10T09:57:49.224 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/", 2024-09-10T09:57:49.224 INFO:teuthology.orchestra.run.smithi077.stdout: "tag": "542d64d4-46f0-4926-8168-90b562009115", 2024-09-10T09:57:49.224 INFO:teuthology.orchestra.run.smithi077.stdout: "options": "recursive,force" 2024-09-10T09:57:49.225 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T09:57:49.225 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T09:57:49.225 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-10T09:57:49.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.222+0000 7f7f32200640 1 -- 172.21.15.77:0/2920085348 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f7f100a2b80 msgr2=0x7f7f100a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:49.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.222+0000 7f7f32200640 1 --2- 172.21.15.77:0/2920085348 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f7f100a2b80 0x7f7f100a5040 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c10ab10 tx=0x7f7f14005f90 comp rx=0 tx=0).stop 2024-09-10T09:57:49.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.222+0000 7f7f32200640 1 -- 172.21.15.77:0/2920085348 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f2c104010 msgr2=0x7f7f2c19a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:49.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.222+0000 7f7f32200640 1 --2- 172.21.15.77:0/2920085348 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f2c104010 0x7f7f2c19a6c0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c00e9f0 tx=0x7f7f1c00eeb0 comp rx=0 tx=0).stop 2024-09-10T09:57:49.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.223+0000 7f7f32200640 1 -- 172.21.15.77:0/2920085348 shutdown_connections 2024-09-10T09:57:49.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.223+0000 7f7f32200640 1 --2- 172.21.15.77:0/2920085348 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f7f100a2b80 0x7f7f100a5040 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:49.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.223+0000 7f7f32200640 1 --2- 172.21.15.77:0/2920085348 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f2c109140 0x7f7f2c19ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:49.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.223+0000 7f7f32200640 1 --2- 172.21.15.77:0/2920085348 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7f2c1049c0 0x7f7f2c19ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:49.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.223+0000 7f7f32200640 1 --2- 172.21.15.77:0/2920085348 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f2c104010 0x7f7f2c19a6c0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:49.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.223+0000 7f7f32200640 1 -- 172.21.15.77:0/2920085348 >> 172.21.15.77:0/2920085348 conn(0x7f7f2c0ffa50 msgr2=0x7f7f2c100db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:49.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.223+0000 7f7f32200640 1 -- 172.21.15.77:0/2920085348 shutdown_connections 2024-09-10T09:57:49.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:57:49.223+0000 7f7f32200640 1 -- 172.21.15.77:0/2920085348 wait complete. 2024-09-10T09:57:49.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:57:48 smithi134 ceph-mon[35674]: scrub summary: idle+waiting paths [/] 2024-09-10T09:57:49.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:57:48 smithi134 ceph-mon[35674]: pgmap v470: 129 pgs: 129 active+clean; 38 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 87 op/s 2024-09-10T09:57:49.475 INFO:tasks.workunit.client.0.smithi077.stdout:autoconf 2024-09-10T09:57:49.717 INFO:tasks.workunit.client.0.smithi077.stdout:./configure \ 2024-09-10T09:57:49.717 INFO:tasks.workunit.client.0.smithi077.stdout: --libexecdir=/usr/lib \ 2024-09-10T09:57:49.717 INFO:tasks.workunit.client.0.smithi077.stdout: --exec_prefix=/var/lib 2024-09-10T09:57:50.219 INFO:tasks.workunit.client.0.smithi077.stdout:checking build system type... x86_64-pc-linux-gnu 2024-09-10T09:57:50.219 INFO:tasks.workunit.client.0.smithi077.stdout:checking host system type... x86_64-pc-linux-gnu 2024-09-10T09:57:50.230 INFO:tasks.workunit.client.0.smithi077.stdout:checking how to print strings... printf 2024-09-10T09:57:50.230 INFO:tasks.workunit.client.0.smithi077.stdout:checking for gcc... gcc 2024-09-10T09:57:50.350 INFO:tasks.cephfs.filesystem:scrub status for tag:542d64d4-46f0-4926-8168-90b562009115 - {'path': '/', 'tag': '542d64d4-46f0-4926-8168-90b562009115', 'options': 'recursive,force'} 2024-09-10T09:57:51.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:57:50 smithi077 ceph-mon[30319]: pgmap v471: 129 pgs: 129 active+clean; 47 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 235 op/s 2024-09-10T09:57:51.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:57:50 smithi134 ceph-mon[35674]: pgmap v471: 129 pgs: 129 active+clean; 47 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 235 op/s 2024-09-10T09:57:51.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:57:50 smithi120 ceph-mon[34770]: pgmap v471: 129 pgs: 129 active+clean; 47 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 235 op/s 2024-09-10T09:57:52.259 INFO:tasks.workunit.client.0.smithi077.stdout:checking whether the C compiler works... yes 2024-09-10T09:57:52.259 INFO:tasks.workunit.client.0.smithi077.stdout:checking for C compiler default output file name... a.out 2024-09-10T09:57:52.320 INFO:tasks.workunit.client.0.smithi077.stdout:checking for suffix of executables... 2024-09-10T09:57:52.383 INFO:tasks.workunit.client.0.smithi077.stdout:checking whether we are cross compiling... no 2024-09-10T09:57:52.441 INFO:tasks.workunit.client.0.smithi077.stdout:checking for suffix of object files... o 2024-09-10T09:57:52.483 INFO:tasks.workunit.client.0.smithi077.stdout:checking whether we are using the GNU C compiler... yes 2024-09-10T09:57:52.530 INFO:tasks.workunit.client.0.smithi077.stdout:checking whether gcc accepts -g... yes 2024-09-10T09:57:52.654 INFO:tasks.workunit.client.0.smithi077.stdout:checking for gcc option to accept ISO C89... none needed 2024-09-10T09:57:52.665 INFO:tasks.workunit.client.0.smithi077.stdout:checking for a sed that does not truncate output... /usr/bin/sed 2024-09-10T09:57:52.676 INFO:tasks.workunit.client.0.smithi077.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2024-09-10T09:57:52.681 INFO:tasks.workunit.client.0.smithi077.stdout:checking for egrep... /usr/bin/grep -E 2024-09-10T09:57:52.684 INFO:tasks.workunit.client.0.smithi077.stdout:checking for fgrep... /usr/bin/grep -F 2024-09-10T09:57:52.697 INFO:tasks.workunit.client.0.smithi077.stdout:checking for ld used by gcc... /usr/bin/ld 2024-09-10T09:57:52.704 INFO:tasks.workunit.client.0.smithi077.stdout:checking if the linker (/usr/bin/ld) is GNU ld... yes 2024-09-10T09:57:52.722 INFO:tasks.workunit.client.0.smithi077.stdout:checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B 2024-09-10T09:57:53.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:57:52 smithi077 ceph-mon[30319]: pgmap v472: 129 pgs: 129 active+clean; 48 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 273 op/s 2024-09-10T09:57:53.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:57:52 smithi134 ceph-mon[35674]: pgmap v472: 129 pgs: 129 active+clean; 48 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 273 op/s 2024-09-10T09:57:53.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:57:52 smithi120 ceph-mon[34770]: pgmap v472: 129 pgs: 129 active+clean; 48 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 273 op/s 2024-09-10T09:57:53.760 INFO:tasks.workunit.client.0.smithi077.stdout:checking the name lister (/usr/bin/nm -B) interface... BSD nm 2024-09-10T09:57:53.761 INFO:tasks.workunit.client.0.smithi077.stdout:checking whether ln -s works... yes 2024-09-10T09:57:53.781 INFO:tasks.workunit.client.0.smithi077.stdout:checking the maximum length of command line arguments... 1572864 2024-09-10T09:57:53.788 INFO:tasks.workunit.client.0.smithi077.stdout:checking how to convert x86_64-pc-linux-gnu file names to x86_64-pc-linux-gnu format... func_convert_file_noop 2024-09-10T09:57:53.788 INFO:tasks.workunit.client.0.smithi077.stdout:checking how to convert x86_64-pc-linux-gnu file names to toolchain format... func_convert_file_noop 2024-09-10T09:57:53.788 INFO:tasks.workunit.client.0.smithi077.stdout:checking for /usr/bin/ld option to reload object files... -r 2024-09-10T09:57:53.788 INFO:tasks.workunit.client.0.smithi077.stdout:checking for objdump... objdump 2024-09-10T09:57:53.788 INFO:tasks.workunit.client.0.smithi077.stdout:checking how to recognize dependent libraries... pass_all 2024-09-10T09:57:53.789 INFO:tasks.workunit.client.0.smithi077.stdout:checking for dlltool... no 2024-09-10T09:57:53.789 INFO:tasks.workunit.client.0.smithi077.stdout:checking how to associate runtime and link libraries... printf %s\n 2024-09-10T09:57:53.790 INFO:tasks.workunit.client.0.smithi077.stdout:checking for ar... ar 2024-09-10T09:57:53.901 INFO:tasks.workunit.client.0.smithi077.stdout:checking for archiver @FILE support... @ 2024-09-10T09:57:53.901 INFO:tasks.workunit.client.0.smithi077.stdout:checking for strip... strip 2024-09-10T09:57:53.901 INFO:tasks.workunit.client.0.smithi077.stdout:checking for ranlib... ranlib 2024-09-10T09:57:53.902 INFO:tasks.workunit.client.0.smithi077.stdout:checking for gawk... gawk 2024-09-10T09:57:54.063 INFO:tasks.workunit.client.0.smithi077.stdout:checking command to parse /usr/bin/nm -B output from gcc object... ok 2024-09-10T09:57:54.066 INFO:tasks.workunit.client.0.smithi077.stdout:checking for sysroot... no 2024-09-10T09:57:54.081 INFO:tasks.workunit.client.0.smithi077.stdout:checking for a working dd... /usr/bin/dd 2024-09-10T09:57:54.095 INFO:tasks.workunit.client.0.smithi077.stdout:checking how to truncate binary pipes... /usr/bin/dd bs=4096 count=1 2024-09-10T09:57:54.148 INFO:tasks.workunit.client.0.smithi077.stdout:checking for mt... no 2024-09-10T09:57:54.166 INFO:tasks.workunit.client.0.smithi077.stdout:checking if : is a manifest tool... no 2024-09-10T09:57:54.282 INFO:tasks.workunit.client.0.smithi077.stdout:checking how to run the C preprocessor... gcc -E 2024-09-10T09:57:54.582 INFO:tasks.workunit.client.0.smithi077.stdout:checking for ANSI C header files... yes 2024-09-10T09:57:54.637 INFO:tasks.workunit.client.0.smithi077.stdout:checking for sys/types.h... yes 2024-09-10T09:57:54.701 INFO:tasks.workunit.client.0.smithi077.stdout:checking for sys/stat.h... yes 2024-09-10T09:57:54.767 INFO:tasks.workunit.client.0.smithi077.stdout:checking for stdlib.h... yes 2024-09-10T09:57:54.833 INFO:tasks.workunit.client.0.smithi077.stdout:checking for string.h... yes 2024-09-10T09:57:54.899 INFO:tasks.workunit.client.0.smithi077.stdout:checking for memory.h... yes 2024-09-10T09:57:54.966 INFO:tasks.workunit.client.0.smithi077.stdout:checking for strings.h... yes 2024-09-10T09:57:55.036 INFO:tasks.workunit.client.0.smithi077.stdout:checking for inttypes.h... yes 2024-09-10T09:57:55.103 INFO:tasks.workunit.client.0.smithi077.stdout:checking for stdint.h... yes 2024-09-10T09:57:55.171 INFO:tasks.workunit.client.0.smithi077.stdout:checking for unistd.h... yes 2024-09-10T09:57:55.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:57:54 smithi077 ceph-mon[30319]: pgmap v473: 129 pgs: 129 active+clean; 57 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.3 MiB/s wr, 539 op/s 2024-09-10T09:57:55.241 INFO:tasks.workunit.client.0.smithi077.stdout:checking for dlfcn.h... yes 2024-09-10T09:57:55.251 INFO:tasks.workunit.client.0.smithi077.stdout:checking for objdir... .libs 2024-09-10T09:57:55.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:57:54 smithi134 ceph-mon[35674]: pgmap v473: 129 pgs: 129 active+clean; 57 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.3 MiB/s wr, 539 op/s 2024-09-10T09:57:55.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:57:54 smithi120 ceph-mon[34770]: pgmap v473: 129 pgs: 129 active+clean; 57 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.3 MiB/s wr, 539 op/s 2024-09-10T09:57:55.434 INFO:tasks.workunit.client.0.smithi077.stdout:checking if gcc supports -fno-rtti -fno-exceptions... no 2024-09-10T09:57:55.434 INFO:tasks.workunit.client.0.smithi077.stdout:checking for gcc option to produce PIC... -fPIC -DPIC 2024-09-10T09:57:55.496 INFO:tasks.workunit.client.0.smithi077.stdout:checking if gcc PIC flag -fPIC -DPIC works... yes 2024-09-10T09:57:55.564 INFO:tasks.workunit.client.0.smithi077.stdout:checking if gcc static flag -static works... no 2024-09-10T09:57:55.656 INFO:tasks.workunit.client.0.smithi077.stdout:checking if gcc supports -c -o file.o... yes 2024-09-10T09:57:55.657 INFO:tasks.workunit.client.0.smithi077.stdout:checking if gcc supports -c -o file.o... (cached) yes 2024-09-10T09:57:55.672 INFO:tasks.workunit.client.0.smithi077.stdout:checking whether the gcc linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes 2024-09-10T09:57:55.919 INFO:tasks.workunit.client.0.smithi077.stdout:checking whether -lc should be explicitly linked in... no 2024-09-10T09:57:56.033 INFO:tasks.workunit.client.0.smithi077.stdout:checking dynamic linker characteristics... GNU/Linux ld.so 2024-09-10T09:57:56.033 INFO:tasks.workunit.client.0.smithi077.stdout:checking how to hardcode library paths into programs... immediate 2024-09-10T09:57:56.036 INFO:tasks.workunit.client.0.smithi077.stdout:checking whether stripping libraries is possible... yes 2024-09-10T09:57:56.036 INFO:tasks.workunit.client.0.smithi077.stdout:checking if libtool supports shared libraries... yes 2024-09-10T09:57:56.036 INFO:tasks.workunit.client.0.smithi077.stdout:checking whether to build shared libraries... yes 2024-09-10T09:57:56.036 INFO:tasks.workunit.client.0.smithi077.stdout:checking whether to build static libraries... yes 2024-09-10T09:57:56.051 INFO:tasks.workunit.client.0.smithi077.stdout:checking for gcc... (cached) gcc 2024-09-10T09:57:56.106 INFO:tasks.workunit.client.0.smithi077.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-09-10T09:57:56.106 INFO:tasks.workunit.client.0.smithi077.stdout:checking whether gcc accepts -g... (cached) yes 2024-09-10T09:57:56.106 INFO:tasks.workunit.client.0.smithi077.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-09-10T09:57:56.106 INFO:tasks.workunit.client.0.smithi077.stdout:checking for gmake... /usr/bin/gmake 2024-09-10T09:57:56.107 INFO:tasks.workunit.client.0.smithi077.stderr:./configure: line 12082: test: $(SHELL): binary operator expected 2024-09-10T09:57:56.107 INFO:tasks.workunit.client.0.smithi077.stdout:checking for tar... /bin/tar 2024-09-10T09:57:56.107 INFO:tasks.workunit.client.0.smithi077.stdout:checking for gzip... /bin/gzip 2024-09-10T09:57:56.138 INFO:tasks.workunit.client.0.smithi077.stdout:checking whether gcc -MM is supported... yes 2024-09-10T09:57:56.138 INFO:tasks.workunit.client.0.smithi077.stdout:checking for sort... /bin/sort 2024-09-10T09:57:56.138 INFO:tasks.workunit.client.0.smithi077.stdout:checking whether ln -s works... yes 2024-09-10T09:57:56.139 INFO:tasks.workunit.client.0.smithi077.stdout:checking for rpm... /bin/rpm 2024-09-10T09:57:56.146 INFO:tasks.workunit.client.0.smithi077.stdout:checking for rpmbuild... no 2024-09-10T09:57:56.217 INFO:tasks.workunit.client.0.smithi077.stdout:checking assert.h usability... yes 2024-09-10T09:57:56.245 INFO:tasks.workunit.client.0.smithi077.stdout:checking assert.h presence... yes 2024-09-10T09:57:56.245 INFO:tasks.workunit.client.0.smithi077.stdout:checking for assert.h... yes 2024-09-10T09:57:56.320 INFO:tasks.workunit.client.0.smithi077.stdout:checking bstring.h usability... no 2024-09-10T09:57:56.366 INFO:tasks.workunit.client.0.smithi077.stdout:checking bstring.h presence... no 2024-09-10T09:57:56.366 INFO:tasks.workunit.client.0.smithi077.stdout:checking for bstring.h... no 2024-09-10T09:57:56.429 INFO:tasks.workunit.client.0.smithi077.stdout:checking libgen.h usability... yes 2024-09-10T09:57:56.454 INFO:tasks.workunit.client.0.smithi077.stdout:checking libgen.h presence... yes 2024-09-10T09:57:56.455 INFO:tasks.workunit.client.0.smithi077.stdout:checking for libgen.h... yes 2024-09-10T09:57:56.545 INFO:tasks.workunit.client.0.smithi077.stdout:checking dirent.h usability... yes 2024-09-10T09:57:56.573 INFO:tasks.workunit.client.0.smithi077.stdout:checking dirent.h presence... yes 2024-09-10T09:57:56.573 INFO:tasks.workunit.client.0.smithi077.stdout:checking for dirent.h... yes 2024-09-10T09:57:56.646 INFO:tasks.workunit.client.0.smithi077.stdout:checking errno.h usability... yes 2024-09-10T09:57:56.672 INFO:tasks.workunit.client.0.smithi077.stdout:checking errno.h presence... yes 2024-09-10T09:57:56.672 INFO:tasks.workunit.client.0.smithi077.stdout:checking for errno.h... yes 2024-09-10T09:57:56.748 INFO:tasks.workunit.client.0.smithi077.stdout:checking malloc.h usability... yes 2024-09-10T09:57:56.781 INFO:tasks.workunit.client.0.smithi077.stdout:checking malloc.h presence... yes 2024-09-10T09:57:56.781 INFO:tasks.workunit.client.0.smithi077.stdout:checking for malloc.h... yes 2024-09-10T09:57:56.862 INFO:tasks.workunit.client.0.smithi077.stdout:checking uuid.h usability... no 2024-09-10T09:57:56.906 INFO:tasks.workunit.client.0.smithi077.stdout:checking uuid.h presence... no 2024-09-10T09:57:56.906 INFO:tasks.workunit.client.0.smithi077.stdout:checking for uuid.h... no 2024-09-10T09:57:56.977 INFO:tasks.workunit.client.0.smithi077.stdout:checking uuid/uuid.h usability... yes 2024-09-10T09:57:57.006 INFO:tasks.workunit.client.0.smithi077.stdout:checking uuid/uuid.h presence... yes 2024-09-10T09:57:57.006 INFO:tasks.workunit.client.0.smithi077.stdout:checking for uuid/uuid.h... yes 2024-09-10T09:57:57.083 INFO:tasks.workunit.client.0.smithi077.stdout:checking sys/uuid.h usability... no 2024-09-10T09:57:57.126 INFO:tasks.workunit.client.0.smithi077.stdout:checking sys/uuid.h presence... no 2024-09-10T09:57:57.126 INFO:tasks.workunit.client.0.smithi077.stdout:checking for sys/uuid.h... no 2024-09-10T09:57:57.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:57:56 smithi077 ceph-mon[30319]: pgmap v474: 129 pgs: 129 active+clean; 59 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 713 op/s 2024-09-10T09:57:57.190 INFO:tasks.workunit.client.0.smithi077.stdout:checking sys/file.h usability... yes 2024-09-10T09:57:57.218 INFO:tasks.workunit.client.0.smithi077.stdout:checking sys/file.h presence... yes 2024-09-10T09:57:57.218 INFO:tasks.workunit.client.0.smithi077.stdout:checking for sys/file.h... yes 2024-09-10T09:57:57.300 INFO:tasks.workunit.client.0.smithi077.stdout:checking sys/fcntl.h usability... yes 2024-09-10T09:57:57.327 INFO:tasks.workunit.client.0.smithi077.stdout:checking sys/fcntl.h presence... yes 2024-09-10T09:57:57.327 INFO:tasks.workunit.client.0.smithi077.stdout:checking for sys/fcntl.h... yes 2024-09-10T09:57:57.380 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:57:56 smithi134 ceph-mon[35674]: pgmap v474: 129 pgs: 129 active+clean; 59 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 713 op/s 2024-09-10T09:57:57.406 INFO:tasks.workunit.client.0.smithi077.stdout:checking sys/syssgi.h usability... no 2024-09-10T09:57:57.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:57:56 smithi120 ceph-mon[34770]: pgmap v474: 129 pgs: 129 active+clean; 59 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 713 op/s 2024-09-10T09:57:57.446 INFO:tasks.workunit.client.0.smithi077.stdout:checking sys/syssgi.h presence... no 2024-09-10T09:57:57.446 INFO:tasks.workunit.client.0.smithi077.stdout:checking for sys/syssgi.h... no 2024-09-10T09:57:57.569 INFO:tasks.workunit.client.0.smithi077.stdout:checking sys/param.h usability... yes 2024-09-10T09:57:57.600 INFO:tasks.workunit.client.0.smithi077.stdout:checking sys/param.h presence... yes 2024-09-10T09:57:57.601 INFO:tasks.workunit.client.0.smithi077.stdout:checking for sys/param.h... yes 2024-09-10T09:57:57.622 INFO:tasks.workunit.client.0.smithi077.stdout:checking for sys/stat.h... (cached) yes 2024-09-10T09:57:57.696 INFO:tasks.workunit.client.0.smithi077.stdout:checking sys/statvfs.h usability... yes 2024-09-10T09:57:57.723 INFO:tasks.workunit.client.0.smithi077.stdout:checking sys/statvfs.h presence... yes 2024-09-10T09:57:57.723 INFO:tasks.workunit.client.0.smithi077.stdout:checking for sys/statvfs.h... yes 2024-09-10T09:57:57.794 INFO:tasks.workunit.client.0.smithi077.stdout:checking sys/time.h usability... yes 2024-09-10T09:57:57.821 INFO:tasks.workunit.client.0.smithi077.stdout:checking sys/time.h presence... yes 2024-09-10T09:57:57.821 INFO:tasks.workunit.client.0.smithi077.stdout:checking for sys/time.h... yes 2024-09-10T09:57:57.895 INFO:tasks.workunit.client.0.smithi077.stdout:checking sys/ioctl.h usability... yes 2024-09-10T09:57:57.921 INFO:tasks.workunit.client.0.smithi077.stdout:checking sys/ioctl.h presence... yes 2024-09-10T09:57:57.921 INFO:tasks.workunit.client.0.smithi077.stdout:checking for sys/ioctl.h... yes 2024-09-10T09:57:58.013 INFO:tasks.workunit.client.0.smithi077.stdout:checking sys/wait.h usability... yes 2024-09-10T09:57:58.043 INFO:tasks.workunit.client.0.smithi077.stdout:checking sys/wait.h presence... yes 2024-09-10T09:57:58.043 INFO:tasks.workunit.client.0.smithi077.stdout:checking for sys/wait.h... yes 2024-09-10T09:57:58.068 INFO:tasks.workunit.client.0.smithi077.stdout:checking for sys/types.h... (cached) yes 2024-09-10T09:57:58.080 INFO:tasks.workunit.client.0.smithi077.stdout:checking for strings.h... (cached) yes 2024-09-10T09:57:58.145 INFO:tasks.workunit.client.0.smithi077.stdout:checking err.h usability... yes 2024-09-10T09:57:58.172 INFO:tasks.workunit.client.0.smithi077.stdout:checking err.h presence... yes 2024-09-10T09:57:58.172 INFO:tasks.workunit.client.0.smithi077.stdout:checking for err.h... yes 2024-09-10T09:57:58.250 INFO:tasks.workunit.client.0.smithi077.stdout:checking linux/falloc.h usability... yes 2024-09-10T09:57:58.274 INFO:tasks.workunit.client.0.smithi077.stdout:checking linux/falloc.h presence... yes 2024-09-10T09:57:58.274 INFO:tasks.workunit.client.0.smithi077.stdout:checking for linux/falloc.h... yes 2024-09-10T09:57:58.357 INFO:tasks.workunit.client.0.smithi077.stdout:checking sys/fs/xfs_fsops.h usability... no 2024-09-10T09:57:58.398 INFO:tasks.workunit.client.0.smithi077.stdout:checking sys/fs/xfs_fsops.h presence... no 2024-09-10T09:57:58.398 INFO:tasks.workunit.client.0.smithi077.stdout:checking for sys/fs/xfs_fsops.h... no 2024-09-10T09:57:58.465 INFO:tasks.workunit.client.0.smithi077.stdout:checking sys/fs/xfs_itable.h usability... no 2024-09-10T09:57:58.507 INFO:tasks.workunit.client.0.smithi077.stdout:checking sys/fs/xfs_itable.h presence... no 2024-09-10T09:57:58.507 INFO:tasks.workunit.client.0.smithi077.stdout:checking for sys/fs/xfs_itable.h... no 2024-09-10T09:57:58.581 INFO:tasks.workunit.client.0.smithi077.stdout:checking xfs/platform_defs.h usability... no 2024-09-10T09:57:58.625 INFO:tasks.workunit.client.0.smithi077.stdout:checking xfs/platform_defs.h presence... no 2024-09-10T09:57:58.625 INFO:tasks.workunit.client.0.smithi077.stdout:checking for xfs/platform_defs.h... no 2024-09-10T09:57:58.691 INFO:tasks.workunit.client.0.smithi077.stdout:checking btrfs/ioctl.h usability... no 2024-09-10T09:57:58.732 INFO:tasks.workunit.client.0.smithi077.stdout:checking btrfs/ioctl.h presence... no 2024-09-10T09:57:58.732 INFO:tasks.workunit.client.0.smithi077.stdout:checking for btrfs/ioctl.h... no 2024-09-10T09:57:58.798 INFO:tasks.workunit.client.0.smithi077.stdout:checking cifs/ioctl.h usability... no 2024-09-10T09:57:58.843 INFO:tasks.workunit.client.0.smithi077.stdout:checking cifs/ioctl.h presence... no 2024-09-10T09:57:58.843 INFO:tasks.workunit.client.0.smithi077.stdout:checking for cifs/ioctl.h... no 2024-09-10T09:57:58.919 INFO:tasks.workunit.client.0.smithi077.stdout:checking sys/mman.h usability... yes 2024-09-10T09:57:58.947 INFO:tasks.workunit.client.0.smithi077.stdout:checking sys/mman.h presence... yes 2024-09-10T09:57:58.947 INFO:tasks.workunit.client.0.smithi077.stdout:checking for sys/mman.h... yes 2024-09-10T09:57:59.037 INFO:tasks.workunit.client.0.smithi077.stdout:checking linux/ext4.h usability... no 2024-09-10T09:57:59.079 INFO:tasks.workunit.client.0.smithi077.stdout:checking linux/ext4.h presence... no 2024-09-10T09:57:59.079 INFO:tasks.workunit.client.0.smithi077.stdout:checking for linux/ext4.h... no 2024-09-10T09:57:59.121 INFO:tasks.workunit.client.0.smithi077.stdout:checking for xfs/xfs_log_format.h... no 2024-09-10T09:57:59.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:57:58 smithi077 ceph-mon[30319]: pgmap v475: 129 pgs: 129 active+clean; 63 MiB data, 573 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.6 MiB/s wr, 750 op/s 2024-09-10T09:57:59.206 INFO:tasks.workunit.client.0.smithi077.stdout:checking for uuid_compare... no 2024-09-10T09:57:59.272 INFO:tasks.workunit.client.0.smithi077.stdout:checking for uuid_compare in -luuid... yes 2024-09-10T09:57:59.366 INFO:tasks.workunit.client.0.smithi077.stdout:checking for xfs/xfs.h... yes 2024-09-10T09:57:59.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:57:58 smithi134 ceph-mon[35674]: pgmap v475: 129 pgs: 129 active+clean; 63 MiB data, 573 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.6 MiB/s wr, 750 op/s 2024-09-10T09:57:59.417 INFO:tasks.workunit.client.0.smithi077.stdout:checking for xfs/libxfs.h... no 2024-09-10T09:57:59.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:57:58 smithi120 ceph-mon[34770]: pgmap v475: 129 pgs: 129 active+clean; 63 MiB data, 573 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.6 MiB/s wr, 750 op/s 2024-09-10T09:57:59.469 INFO:tasks.workunit.client.0.smithi077.stdout:checking whether xlog_assign_lsn is declared... no 2024-09-10T09:57:59.532 INFO:tasks.workunit.client.0.smithi077.stdout:checking for xfs/xqm.h... yes 2024-09-10T09:57:59.621 INFO:tasks.workunit.client.0.smithi077.stdout:checking xfsctl from xfs/xfs.h... ok 2024-09-10T09:57:59.678 INFO:tasks.workunit.client.0.smithi077.stdout:checking xfs/handle.h usability... yes 2024-09-10T09:57:59.702 INFO:tasks.workunit.client.0.smithi077.stdout:checking xfs/handle.h presence... yes 2024-09-10T09:57:59.703 INFO:tasks.workunit.client.0.smithi077.stdout:checking for xfs/handle.h... yes 2024-09-10T09:57:59.784 INFO:tasks.workunit.client.0.smithi077.stdout:checking for attr_list_by_handle in -lhandle... yes 2024-09-10T09:57:59.844 INFO:tasks.workunit.client.0.smithi077.stdout:checking sys/acl.h usability... yes 2024-09-10T09:57:59.873 INFO:tasks.workunit.client.0.smithi077.stdout:checking sys/acl.h presence... yes 2024-09-10T09:57:59.873 INFO:tasks.workunit.client.0.smithi077.stdout:checking for sys/acl.h... yes 2024-09-10T09:57:59.936 INFO:tasks.workunit.client.0.smithi077.stdout:checking acl/libacl.h usability... yes 2024-09-10T09:57:59.966 INFO:tasks.workunit.client.0.smithi077.stdout:checking acl/libacl.h presence... yes 2024-09-10T09:57:59.966 INFO:tasks.workunit.client.0.smithi077.stdout:checking for acl/libacl.h... yes 2024-09-10T09:58:00.034 INFO:tasks.workunit.client.0.smithi077.stdout:checking for acl_init in -lacl... yes 2024-09-10T09:58:00.122 INFO:tasks.workunit.client.0.smithi077.stdout:checking gdbm-ndbm.h usability... no 2024-09-10T09:58:00.165 INFO:tasks.workunit.client.0.smithi077.stdout:checking gdbm-ndbm.h presence... no 2024-09-10T09:58:00.165 INFO:tasks.workunit.client.0.smithi077.stdout:checking for gdbm-ndbm.h... no 2024-09-10T09:58:00.255 INFO:tasks.workunit.client.0.smithi077.stdout:checking gdbm/ndbm.h usability... no 2024-09-10T09:58:00.297 INFO:tasks.workunit.client.0.smithi077.stdout:checking gdbm/ndbm.h presence... no 2024-09-10T09:58:00.297 INFO:tasks.workunit.client.0.smithi077.stdout:checking for gdbm/ndbm.h... no 2024-09-10T09:58:00.712 INFO:tasks.workunit.client.0.smithi077.stdout:checking ndbm.h usability... no 2024-09-10T09:58:00.755 INFO:tasks.workunit.client.0.smithi077.stdout:checking ndbm.h presence... no 2024-09-10T09:58:00.755 INFO:tasks.workunit.client.0.smithi077.stdout:checking for ndbm.h... no 2024-09-10T09:58:01.215 INFO:tasks.workunit.client.0.smithi077.stdout:checking libaio.h usability... yes 2024-09-10T09:58:01.250 INFO:tasks.workunit.client.0.smithi077.stdout:checking libaio.h presence... yes 2024-09-10T09:58:01.250 INFO:tasks.workunit.client.0.smithi077.stdout:checking for libaio.h... yes 2024-09-10T09:58:01.323 INFO:tasks.workunit.client.0.smithi077.stdout:checking liburing.h usability... no 2024-09-10T09:58:01.367 INFO:tasks.workunit.client.0.smithi077.stdout:checking liburing.h presence... no 2024-09-10T09:58:01.367 INFO:tasks.workunit.client.0.smithi077.stdout:checking for liburing.h... no 2024-09-10T09:58:01.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:58:00 smithi134 ceph-mon[35674]: pgmap v476: 129 pgs: 129 active+clean; 64 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.0 MiB/s wr, 724 op/s 2024-09-10T09:58:01.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:58:00 smithi134 ceph-mon[35674]: scrub summary: idle 2024-09-10T09:58:01.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:58:00 smithi134 ceph-mon[35674]: scrub summary: idle 2024-09-10T09:58:01.407 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:58:00 smithi077 ceph-mon[30319]: pgmap v476: 129 pgs: 129 active+clean; 64 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.0 MiB/s wr, 724 op/s 2024-09-10T09:58:01.408 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:58:00 smithi077 ceph-mon[30319]: scrub summary: idle 2024-09-10T09:58:01.408 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:58:00 smithi077 ceph-mon[30319]: scrub summary: idle 2024-09-10T09:58:01.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:58:00 smithi120 ceph-mon[34770]: pgmap v476: 129 pgs: 129 active+clean; 64 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.0 MiB/s wr, 724 op/s 2024-09-10T09:58:01.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:58:00 smithi120 ceph-mon[34770]: scrub summary: idle 2024-09-10T09:58:01.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:58:00 smithi120 ceph-mon[34770]: scrub summary: idle 2024-09-10T09:58:01.431 INFO:tasks.workunit.client.0.smithi077.stdout:checking sys/capability.h usability... no 2024-09-10T09:58:01.475 INFO:tasks.workunit.client.0.smithi077.stdout:checking sys/capability.h presence... no 2024-09-10T09:58:01.475 INFO:tasks.workunit.client.0.smithi077.stdout:checking for sys/capability.h... no 2024-09-10T09:58:01.528 INFO:tasks.workunit.client.0.smithi077.stdout:checking linux/fiemap.h usability... yes 2024-09-10T09:58:01.555 INFO:tasks.workunit.client.0.smithi077.stdout:checking linux/fiemap.h presence... yes 2024-09-10T09:58:01.555 INFO:tasks.workunit.client.0.smithi077.stdout:checking for linux/fiemap.h... yes 2024-09-10T09:58:01.621 INFO:tasks.workunit.client.0.smithi077.stdout:checking for fallocate... yes 2024-09-10T09:58:01.696 INFO:tasks.workunit.client.0.smithi077.stdout:checking for open_by_handle_at... yes 2024-09-10T09:58:01.755 INFO:tasks.workunit.client.0.smithi077.stdout:checking linux/fs.h usability... yes 2024-09-10T09:58:01.781 INFO:tasks.workunit.client.0.smithi077.stdout:checking linux/fs.h presence... yes 2024-09-10T09:58:01.781 INFO:tasks.workunit.client.0.smithi077.stdout:checking for linux/fs.h... yes 2024-09-10T09:58:01.857 INFO:tasks.workunit.client.0.smithi077.stdout:checking btrfsutil.h usability... no 2024-09-10T09:58:01.899 INFO:tasks.workunit.client.0.smithi077.stdout:checking btrfsutil.h presence... no 2024-09-10T09:58:01.899 INFO:tasks.workunit.client.0.smithi077.stdout:checking for btrfsutil.h... no 2024-09-10T09:58:01.984 INFO:tasks.workunit.client.0.smithi077.stdout:checking for copy_file_range... yes 2024-09-10T09:58:02.057 INFO:tasks.workunit.client.0.smithi077.stdout:checking for SEEK_DATA... yes 2024-09-10T09:58:02.150 INFO:tasks.workunit.client.0.smithi077.stdout:checking for BMV_OF_SHARED... yes 2024-09-10T09:58:02.224 INFO:tasks.workunit.client.0.smithi077.stdout:checking for nftw... yes 2024-09-10T09:58:02.308 INFO:tasks.workunit.client.0.smithi077.stdout:checking for RLIMIT_NOFILE... yes 2024-09-10T09:58:02.377 INFO:tasks.workunit.client.0.smithi077.stdout:checking for XFS_IOC_EXCHANGE_RANGE... no 2024-09-10T09:58:02.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:58:01 smithi134 ceph-mon[35674]: scrub summary: idle 2024-09-10T09:58:02.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:58:01 smithi134 ceph-mon[35674]: scrub summary: idle 2024-09-10T09:58:02.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:58:01 smithi134 ceph-mon[35674]: scrub summary: idle 2024-09-10T09:58:02.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:58:01 smithi120 ceph-mon[34770]: scrub summary: idle 2024-09-10T09:58:02.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:58:01 smithi120 ceph-mon[34770]: scrub summary: idle 2024-09-10T09:58:02.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:58:01 smithi120 ceph-mon[34770]: scrub summary: idle 2024-09-10T09:58:02.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:58:01 smithi077 ceph-mon[30319]: scrub summary: idle 2024-09-10T09:58:02.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:58:01 smithi077 ceph-mon[30319]: scrub summary: idle 2024-09-10T09:58:02.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:58:01 smithi077 ceph-mon[30319]: scrub summary: idle 2024-09-10T09:58:02.480 INFO:tasks.workunit.client.0.smithi077.stdout:checking for FICLONE... yes 2024-09-10T09:58:02.554 INFO:tasks.workunit.client.0.smithi077.stdout:checking for renameat2... yes 2024-09-10T09:58:02.623 INFO:tasks.workunit.client.0.smithi077.stdout:checking for reallocarray... yes 2024-09-10T09:58:02.730 INFO:tasks.workunit.client.0.smithi077.stdout:checking for struct mount_attr... yes 2024-09-10T09:58:02.838 INFO:tasks.workunit.client.0.smithi077.stdout:checking for struct btrfs_qgroup_limit... yes 2024-09-10T09:58:02.931 INFO:tasks.workunit.client.0.smithi077.stdout:checking for struct btrfs_qgroup_inherit... yes 2024-09-10T09:58:03.028 INFO:tasks.workunit.client.0.smithi077.stdout:checking for struct btrfs_ioctl_vol_args... yes 2024-09-10T09:58:03.124 INFO:tasks.workunit.client.0.smithi077.stdout:checking for struct btrfs_ioctl_vol_args_v2... yes 2024-09-10T09:58:03.219 INFO:tasks.workunit.client.0.smithi077.stdout:checking for struct btrfs_ioctl_ino_lookup_args... yes 2024-09-10T09:58:03.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:58:02 smithi134 ceph-mon[35674]: pgmap v477: 129 pgs: 129 active+clean; 66 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 577 op/s 2024-09-10T09:58:03.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:58:02 smithi120 ceph-mon[34770]: pgmap v477: 129 pgs: 129 active+clean; 66 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 577 op/s 2024-09-10T09:58:03.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:58:02 smithi077 ceph-mon[30319]: pgmap v477: 129 pgs: 129 active+clean; 66 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 577 op/s 2024-09-10T09:58:03.455 INFO:tasks.workunit.client.0.smithi077.stdout:checking for struct btrfs_ioctl_ino_lookup_user_args... yes 2024-09-10T09:58:03.553 INFO:tasks.workunit.client.0.smithi077.stdout:checking for struct btrfs_ioctl_get_subvol_rootref_args... yes 2024-09-10T09:58:03.621 INFO:tasks.workunit.client.0.smithi077.stdout:checking linux/btrfs.h usability... yes 2024-09-10T09:58:03.651 INFO:tasks.workunit.client.0.smithi077.stdout:checking linux/btrfs.h presence... yes 2024-09-10T09:58:03.651 INFO:tasks.workunit.client.0.smithi077.stdout:checking for linux/btrfs.h... yes 2024-09-10T09:58:03.727 INFO:tasks.workunit.client.0.smithi077.stdout:checking linux/btrfs_tree.h usability... yes 2024-09-10T09:58:03.754 INFO:tasks.workunit.client.0.smithi077.stdout:checking linux/btrfs_tree.h presence... yes 2024-09-10T09:58:03.754 INFO:tasks.workunit.client.0.smithi077.stdout:checking for linux/btrfs_tree.h... yes 2024-09-10T09:58:03.815 INFO:tasks.workunit.client.0.smithi077.stdout:checking for struct btrfs_ioctl_vol_args_v2.subvolid... yes 2024-09-10T09:58:03.870 INFO:tasks.workunit.client.0.smithi077.stdout:checking whether BTRFS_IOC_SNAP_DESTROY_V2 is declared... yes 2024-09-10T09:58:03.903 INFO:tasks.workunit.client.0.smithi077.stdout:configure: creating ./config.status 2024-09-10T09:58:04.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:58:04 smithi134 ceph-mon[35674]: pgmap v478: 129 pgs: 129 active+clean; 66 MiB data, 573 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 526 op/s 2024-09-10T09:58:04.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:58:04 smithi120 ceph-mon[34770]: pgmap v478: 129 pgs: 129 active+clean; 66 MiB data, 573 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 526 op/s 2024-09-10T09:58:04.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:58:04 smithi077 ceph-mon[30319]: pgmap v478: 129 pgs: 129 active+clean; 66 MiB data, 573 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 526 op/s 2024-09-10T09:58:04.776 INFO:tasks.workunit.client.0.smithi077.stdout:config.status: creating include/builddefs 2024-09-10T09:58:04.808 INFO:tasks.workunit.client.0.smithi077.stdout:config.status: creating include/config.h 2024-09-10T09:58:04.832 INFO:tasks.workunit.client.0.smithi077.stdout:config.status: executing libtool commands 2024-09-10T09:58:04.953 INFO:tasks.workunit.client.0.smithi077.stdout:Building include 2024-09-10T09:58:04.954 INFO:tasks.workunit.client.0.smithi077.stdout:Building lib 2024-09-10T09:58:04.954 INFO:tasks.workunit.client.0.smithi077.stdout:Building tests 2024-09-10T09:58:04.962 INFO:tasks.workunit.client.0.smithi077.stdout: [LTDEP] 2024-09-10T09:58:04.978 INFO:tasks.workunit.client.0.smithi077.stdout:Building btrfs 2024-09-10T09:58:04.978 INFO:tasks.workunit.client.0.smithi077.stdout:Building ceph 2024-09-10T09:58:04.978 INFO:tasks.workunit.client.0.smithi077.stdout:Building cifs 2024-09-10T09:58:04.991 INFO:tasks.workunit.client.0.smithi077.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/cifs/group.list 2024-09-10T09:58:04.991 INFO:tasks.workunit.client.0.smithi077.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/ceph/group.list 2024-09-10T09:58:05.016 INFO:tasks.workunit.client.0.smithi077.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/btrfs/group.list 2024-09-10T09:58:05.035 INFO:tasks.workunit.client.0.smithi077.stdout:Building ext4 2024-09-10T09:58:05.038 INFO:tasks.workunit.client.0.smithi077.stdout:Building f2fs 2024-09-10T09:58:05.049 INFO:tasks.workunit.client.0.smithi077.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/f2fs/group.list 2024-09-10T09:58:05.055 INFO:tasks.workunit.client.0.smithi077.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/ext4/group.list 2024-09-10T09:58:05.109 INFO:tasks.workunit.client.0.smithi077.stdout:Building generic 2024-09-10T09:58:05.178 INFO:tasks.workunit.client.0.smithi077.stdout:Building nfs 2024-09-10T09:58:05.191 INFO:tasks.workunit.client.0.smithi077.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/nfs/group.list 2024-09-10T09:58:05.237 INFO:tasks.workunit.client.0.smithi077.stdout:Building ocfs2 2024-09-10T09:58:05.250 INFO:tasks.workunit.client.0.smithi077.stdout:Building overlay 2024-09-10T09:58:05.251 INFO:tasks.workunit.client.0.smithi077.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/ocfs2/group.list 2024-09-10T09:58:05.269 INFO:tasks.workunit.client.0.smithi077.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/generic/group.list 2024-09-10T09:58:05.279 INFO:tasks.workunit.client.0.smithi077.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/overlay/group.list 2024-09-10T09:58:05.296 INFO:tasks.workunit.client.0.smithi077.stdout:Building perf 2024-09-10T09:58:05.296 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] dataascii.lo 2024-09-10T09:58:05.310 INFO:tasks.workunit.client.0.smithi077.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/perf/group.list 2024-09-10T09:58:05.351 INFO:tasks.workunit.client.0.smithi077.stdout:Building selftest 2024-09-10T09:58:05.364 INFO:tasks.workunit.client.0.smithi077.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/selftest/group.list 2024-09-10T09:58:05.405 INFO:tasks.workunit.client.0.smithi077.stdout:Building shared 2024-09-10T09:58:05.418 INFO:tasks.workunit.client.0.smithi077.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/shared/group.list 2024-09-10T09:58:05.429 INFO:tasks.workunit.client.0.smithi077.stdout:Building tmpfs 2024-09-10T09:58:05.442 INFO:tasks.workunit.client.0.smithi077.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/tmpfs/group.list 2024-09-10T09:58:05.476 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] databin.lo 2024-09-10T09:58:05.488 INFO:tasks.workunit.client.0.smithi077.stdout:Building udf 2024-09-10T09:58:05.502 INFO:tasks.workunit.client.0.smithi077.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/udf/group.list 2024-09-10T09:58:05.540 INFO:tasks.workunit.client.0.smithi077.stdout:Building xfs 2024-09-10T09:58:05.542 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] datapid.lo 2024-09-10T09:58:05.645 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] file_lock.lo 2024-09-10T09:58:05.693 INFO:tasks.workunit.client.0.smithi077.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/xfs/group.list 2024-09-10T09:58:05.743 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] forker.lo 2024-09-10T09:58:05.746 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] pattern.lo 2024-09-10T09:58:05.928 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] open_flags.lo 2024-09-10T09:58:05.975 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] random_range.lo 2024-09-10T09:58:05.993 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] string_to_tokens.lo 2024-09-10T09:58:06.064 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] str_to_bytes.lo 2024-09-10T09:58:06.180 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] tlibio.lo 2024-09-10T09:58:06.203 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] write_log.lo 2024-09-10T09:58:06.318 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] random.lo 2024-09-10T09:58:06.570 INFO:tasks.workunit.client.0.smithi077.stdout: [LD] 2024-09-10T09:58:06.752 INFO:tasks.workunit.client.0.smithi077.stdout:Building ltp 2024-09-10T09:58:06.752 INFO:tasks.workunit.client.0.smithi077.stdout:Building src 2024-09-10T09:58:06.752 INFO:tasks.workunit.client.0.smithi077.stdout:Building m4 2024-09-10T09:58:06.753 INFO:tasks.workunit.client.0.smithi077.stdout:Building common 2024-09-10T09:58:06.761 INFO:tasks.workunit.client.0.smithi077.stdout: [DEP] 2024-09-10T09:58:06.781 INFO:tasks.workunit.client.0.smithi077.stdout: [DEP] 2024-09-10T09:58:07.110 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] doio 2024-09-10T09:58:07.110 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] fsstress 2024-09-10T09:58:07.110 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] fsx 2024-09-10T09:58:07.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:58:06 smithi134 ceph-mon[35674]: pgmap v479: 129 pgs: 129 active+clean; 66 MiB data, 573 MiB used, 1.0 TiB / 1.0 TiB avail; 941 KiB/s rd, 1.1 MiB/s wr, 259 op/s 2024-09-10T09:58:07.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:58:06 smithi120 ceph-mon[34770]: pgmap v479: 129 pgs: 129 active+clean; 66 MiB data, 573 MiB used, 1.0 TiB / 1.0 TiB avail; 941 KiB/s rd, 1.1 MiB/s wr, 259 op/s 2024-09-10T09:58:07.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:58:06 smithi077 ceph-mon[30319]: pgmap v479: 129 pgs: 129 active+clean; 66 MiB data, 573 MiB used, 1.0 TiB / 1.0 TiB avail; 941 KiB/s rd, 1.1 MiB/s wr, 259 op/s 2024-09-10T09:58:07.953 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] growfiles 2024-09-10T09:58:08.016 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] iogen 2024-09-10T09:58:08.527 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] aio-stress 2024-09-10T09:58:09.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:58:08 smithi134 ceph-mon[35674]: pgmap v480: 129 pgs: 129 active+clean; 68 MiB data, 573 MiB used, 1.0 TiB / 1.0 TiB avail; 1020 KiB/s rd, 835 KiB/s wr, 80 op/s 2024-09-10T09:58:09.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:58:08 smithi120 ceph-mon[34770]: pgmap v480: 129 pgs: 129 active+clean; 68 MiB data, 573 MiB used, 1.0 TiB / 1.0 TiB avail; 1020 KiB/s rd, 835 KiB/s wr, 80 op/s 2024-09-10T09:58:09.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:58:08 smithi077 ceph-mon[30319]: pgmap v480: 129 pgs: 129 active+clean; 68 MiB data, 573 MiB used, 1.0 TiB / 1.0 TiB avail; 1020 KiB/s rd, 835 KiB/s wr, 80 op/s 2024-09-10T09:58:10.101 INFO:tasks.workunit.client.0.smithi077.stdout: [DEP] 2024-09-10T09:58:10.101 INFO:tasks.workunit.client.0.smithi077.stdout: [DEP] 2024-09-10T09:58:10.102 INFO:tasks.workunit.client.0.smithi077.stdout: [DEP] 2024-09-10T09:58:10.104 INFO:tasks.workunit.client.0.smithi077.stdout: [DEP] 2024-09-10T09:58:10.508 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] dirstress 2024-09-10T09:58:10.509 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] fill 2024-09-10T09:58:10.509 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] fill2 2024-09-10T09:58:10.510 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] getpagesize 2024-09-10T09:58:10.827 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] holes 2024-09-10T09:58:10.841 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] lstat64 2024-09-10T09:58:11.092 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:58:10 smithi077 ceph-mon[30319]: pgmap v481: 129 pgs: 129 active+clean; 68 MiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 529 KiB/s rd, 473 KiB/s wr, 47 op/s 2024-09-10T09:58:11.130 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:58:10 smithi134 ceph-mon[35674]: pgmap v481: 129 pgs: 129 active+clean; 68 MiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 529 KiB/s rd, 473 KiB/s wr, 47 op/s 2024-09-10T09:58:11.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:58:10 smithi120 ceph-mon[34770]: pgmap v481: 129 pgs: 129 active+clean; 68 MiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 529 KiB/s rd, 473 KiB/s wr, 47 op/s 2024-09-10T09:58:11.237 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] nametest 2024-09-10T09:58:13.086 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:58:12 smithi077 ceph-mon[30319]: pgmap v482: 129 pgs: 129 active+clean; 68 MiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 347 KiB/s wr, 35 op/s 2024-09-10T09:58:13.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:58:12 smithi134 ceph-mon[35674]: pgmap v482: 129 pgs: 129 active+clean; 68 MiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 347 KiB/s wr, 35 op/s 2024-09-10T09:58:13.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:58:12 smithi120 ceph-mon[34770]: pgmap v482: 129 pgs: 129 active+clean; 68 MiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 347 KiB/s wr, 35 op/s 2024-09-10T09:58:15.045 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:58:14 smithi077 ceph-mon[30319]: pgmap v483: 129 pgs: 129 active+clean; 71 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 567 KiB/s wr, 80 op/s 2024-09-10T09:58:15.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:58:14 smithi134 ceph-mon[35674]: pgmap v483: 129 pgs: 129 active+clean; 71 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 567 KiB/s wr, 80 op/s 2024-09-10T09:58:15.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:58:14 smithi120 ceph-mon[34770]: pgmap v483: 129 pgs: 129 active+clean; 71 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 567 KiB/s wr, 80 op/s 2024-09-10T09:58:16.166 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] permname 2024-09-10T09:58:16.468 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] randholes 2024-09-10T09:58:16.472 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] runas 2024-09-10T09:58:16.559 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] truncfile 2024-09-10T09:58:16.621 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] usemem 2024-09-10T09:58:16.915 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] mmapcat 2024-09-10T09:58:16.929 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] append_reader 2024-09-10T09:58:16.931 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] append_writer 2024-09-10T09:58:17.073 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] dirperf 2024-09-10T09:58:17.130 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:58:16 smithi134 ceph-mon[35674]: pgmap v484: 129 pgs: 129 active+clean; 74 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 795 KiB/s wr, 100 op/s 2024-09-10T09:58:17.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:58:16 smithi120 ceph-mon[34770]: pgmap v484: 129 pgs: 129 active+clean; 74 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 795 KiB/s wr, 100 op/s 2024-09-10T09:58:17.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:58:16 smithi077 ceph-mon[30319]: pgmap v484: 129 pgs: 129 active+clean; 74 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 795 KiB/s wr, 100 op/s 2024-09-10T09:58:17.200 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] metaperf 2024-09-10T09:58:17.209 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] devzero 2024-09-10T09:58:17.213 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] feature 2024-09-10T09:58:17.479 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] alloc 2024-09-10T09:58:17.532 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] fault 2024-09-10T09:58:17.562 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] fstest 2024-09-10T09:58:17.609 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] t_access_root 2024-09-10T09:58:17.847 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] resvtest 2024-09-10T09:58:17.847 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] godown 2024-09-10T09:58:17.906 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] writemod 2024-09-10T09:58:17.942 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] writev_on_pagefault 2024-09-10T09:58:18.165 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] itrash 2024-09-10T09:58:18.166 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] makeextents 2024-09-10T09:58:18.194 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] rename 2024-09-10T09:58:18.225 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] multi_open_unlink 2024-09-10T09:58:18.459 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] unwritten_sync 2024-09-10T09:58:18.469 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] genhashnames 2024-09-10T09:58:18.486 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] t_holes 2024-09-10T09:58:18.540 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] t_mmap_writev 2024-09-10T09:58:18.800 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] t_truncate_cmtime 2024-09-10T09:58:18.801 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] t_rename_overwrite 2024-09-10T09:58:18.801 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] dirhash_collide 2024-09-10T09:58:18.807 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] holetest 2024-09-10T09:58:19.080 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] t_truncate_self 2024-09-10T09:58:19.081 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] af_unix 2024-09-10T09:58:19.125 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] t_mmap_stale_pmd 2024-09-10T09:58:19.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:58:18 smithi134 ceph-mon[35674]: pgmap v485: 129 pgs: 129 active+clean; 75 MiB data, 608 MiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 856 KiB/s wr, 105 op/s 2024-09-10T09:58:19.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:58:18 smithi120 ceph-mon[34770]: pgmap v485: 129 pgs: 129 active+clean; 75 MiB data, 608 MiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 856 KiB/s wr, 105 op/s 2024-09-10T09:58:19.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:58:18 smithi077 ceph-mon[30319]: pgmap v485: 129 pgs: 129 active+clean; 75 MiB data, 608 MiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 856 KiB/s wr, 105 op/s 2024-09-10T09:58:19.206 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] t_mmap_cow_race 2024-09-10T09:58:19.360 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] t_mmap_fallocate 2024-09-10T09:58:19.389 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] fsync-err 2024-09-10T09:58:19.428 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] t_mmap_write_ro 2024-09-10T09:58:19.525 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] t_ext4_dax_journal_corruption 2024-09-10T09:58:19.650 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] t_ext4_dax_inline_corruption 2024-09-10T09:58:19.730 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] t_mmap_collision 2024-09-10T09:58:19.730 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] t_ofd_locks 2024-09-10T09:58:19.843 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] mmap-write-concurrent 2024-09-10T09:58:19.941 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] t_get_file_time 2024-09-10T09:58:20.089 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] t_create_long_dirs 2024-09-10T09:58:20.089 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] t_create_short_dirs 2024-09-10T09:58:20.143 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] t_enospc 2024-09-10T09:58:20.239 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] t_mmap_writev_overlap 2024-09-10T09:58:20.351 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T09:58:20.422 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] checkpoint_journal 2024-09-10T09:58:20.428 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] mmap-rw-fault 2024-09-10T09:58:20.477 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] allocstale 2024-09-10T09:58:20.565 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] t_mmap_cow_memory_failure 2024-09-10T09:58:20.757 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] fake-dump-rootino 2024-09-10T09:58:20.801 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] dio-buf-fault 2024-09-10T09:58:20.842 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] rewinddir-test 2024-09-10T09:58:20.869 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:58:20.935 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] readdir-while-renames 2024-09-10T09:58:21.078 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] xfsctl 2024-09-10T09:58:21.113 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] bstat 2024-09-10T09:58:21.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:58:20 smithi134 ceph-mon[35674]: pgmap v486: 129 pgs: 129 active+clean; 75 MiB data, 595 MiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 814 KiB/s wr, 116 op/s 2024-09-10T09:58:21.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:58:20 smithi120 ceph-mon[34770]: pgmap v486: 129 pgs: 129 active+clean; 75 MiB data, 595 MiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 814 KiB/s wr, 116 op/s 2024-09-10T09:58:21.186 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] t_mtab 2024-09-10T09:58:21.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:58:20 smithi077 ceph-mon[30319]: pgmap v486: 129 pgs: 129 active+clean; 75 MiB data, 595 MiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 814 KiB/s wr, 116 op/s 2024-09-10T09:58:21.260 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] getdevicesize 2024-09-10T09:58:21.441 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] preallo_rw_pattern_reader 2024-09-10T09:58:21.514 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] preallo_rw_pattern_writer 2024-09-10T09:58:21.543 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] ftrunc 2024-09-10T09:58:21.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.499+0000 7fda70c00640 1 -- 172.21.15.77:0/2972525776 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fda6c1082a0 msgr2=0x7fda6c10ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:21.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.499+0000 7fda70c00640 1 --2- 172.21.15.77:0/2972525776 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fda6c1082a0 0x7fda6c10ec40 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fda64009cd0 tx=0x7fda6402a8e0 comp rx=0 tx=0).stop 2024-09-10T09:58:21.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.500+0000 7fda70c00640 1 -- 172.21.15.77:0/2972525776 shutdown_connections 2024-09-10T09:58:21.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.500+0000 7fda70c00640 1 --2- 172.21.15.77:0/2972525776 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fda6c1082a0 0x7fda6c10ec40 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:21.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.500+0000 7fda70c00640 1 --2- 172.21.15.77:0/2972525776 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda6c1036e0 0x7fda6c107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:21.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.500+0000 7fda70c00640 1 --2- 172.21.15.77:0/2972525776 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fda6c102d30 0x7fda6c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:21.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.500+0000 7fda70c00640 1 -- 172.21.15.77:0/2972525776 >> 172.21.15.77:0/2972525776 conn(0x7fda6c0fc7d0 msgr2=0x7fda6c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:21.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.500+0000 7fda70c00640 1 -- 172.21.15.77:0/2972525776 shutdown_connections 2024-09-10T09:58:21.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.500+0000 7fda70c00640 1 -- 172.21.15.77:0/2972525776 wait complete. 2024-09-10T09:58:21.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.501+0000 7fda70c00640 1 Processor -- start 2024-09-10T09:58:21.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.501+0000 7fda70c00640 1 -- start start 2024-09-10T09:58:21.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.502+0000 7fda70c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda6c102d30 0x7fda6c103c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:21.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.502+0000 7fda70c00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fda6c1036e0 0x7fda6c104160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:21.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.502+0000 7fda70c00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fda6c1082a0 0x7fda6c1046a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:21.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.502+0000 7fda70c00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fda6c1120d0 con 0x7fda6c1036e0 2024-09-10T09:58:21.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.502+0000 7fda70c00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fda6c111f50 con 0x7fda6c1082a0 2024-09-10T09:58:21.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.502+0000 7fda70c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fda6c112250 con 0x7fda6c102d30 2024-09-10T09:58:21.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.502+0000 7fda69400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fda6c1036e0 0x7fda6c104160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:21.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.503+0000 7fda69400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fda6c1036e0 0x7fda6c104160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:48998/0 (socket says 172.21.15.77:48998) 2024-09-10T09:58:21.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.503+0000 7fda69400640 1 -- 172.21.15.77:0/850108523 learned_addr learned my addr 172.21.15.77:0/850108523 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:58:21.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.503+0000 7fda6a800640 1 --2- 172.21.15.77:0/850108523 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fda6c1082a0 0x7fda6c1046a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:21.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.503+0000 7fda69e00640 1 --2- 172.21.15.77:0/850108523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda6c102d30 0x7fda6c103c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:21.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.503+0000 7fda69400640 1 -- 172.21.15.77:0/850108523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda6c102d30 msgr2=0x7fda6c103c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:21.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.503+0000 7fda69400640 1 --2- 172.21.15.77:0/850108523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda6c102d30 0x7fda6c103c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:21.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.503+0000 7fda69400640 1 -- 172.21.15.77:0/850108523 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fda6c1082a0 msgr2=0x7fda6c1046a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:21.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.503+0000 7fda69400640 1 --2- 172.21.15.77:0/850108523 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fda6c1082a0 0x7fda6c1046a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:21.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.503+0000 7fda69400640 1 -- 172.21.15.77:0/850108523 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda6c1aae70 con 0x7fda6c1036e0 2024-09-10T09:58:21.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.504+0000 7fda69400640 1 --2- 172.21.15.77:0/850108523 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fda6c1036e0 0x7fda6c104160 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fda5800e9f0 tx=0x7fda5800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:21.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.504+0000 7fda62a00640 1 -- 172.21.15.77:0/850108523 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda5800ccc0 con 0x7fda6c1036e0 2024-09-10T09:58:21.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.504+0000 7fda62a00640 1 -- 172.21.15.77:0/850108523 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda58010910 con 0x7fda6c1036e0 2024-09-10T09:58:21.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.504+0000 7fda70c00640 1 -- 172.21.15.77:0/850108523 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda6c1ab160 con 0x7fda6c1036e0 2024-09-10T09:58:21.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.504+0000 7fda62a00640 1 -- 172.21.15.77:0/850108523 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda58018920 con 0x7fda6c1036e0 2024-09-10T09:58:21.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.504+0000 7fda70c00640 1 -- 172.21.15.77:0/850108523 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda6c1ab6f0 con 0x7fda6c1036e0 2024-09-10T09:58:21.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.508+0000 7fda7387a740 1 Processor -- start 2024-09-10T09:58:21.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.508+0000 7fda7387a740 1 -- start start 2024-09-10T09:58:21.550 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.508+0000 7fda62a00640 1 -- 172.21.15.77:0/850108523 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fda58018ac0 con 0x7fda6c1036e0 2024-09-10T09:58:21.550 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.508+0000 7fda7387a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b0924e8a50 0x55b0924e8e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:21.550 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.508+0000 7fda7387a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0924e9370 0x55b092504840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:21.550 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.509+0000 7fda7387a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b092504d80 0x55b092507170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:21.550 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.509+0000 7fda7387a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55b09244f3e0 con 0x55b092504d80 2024-09-10T09:58:21.550 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.509+0000 7fda7387a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b0923f6d80 con 0x55b0924e9370 2024-09-10T09:58:21.550 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.509+0000 7fda7387a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b09244b3a0 con 0x55b0924e8a50 2024-09-10T09:58:21.550 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.509+0000 7fda6a800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b0924e8a50 0x55b0924e8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:21.550 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.509+0000 7fda6a800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b0924e8a50 0x55b0924e8e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:48392/0 (socket says 172.21.15.77:48392) 2024-09-10T09:58:21.550 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.509+0000 7fda6a800640 1 -- 172.21.15.77:0/470212403 learned_addr learned my addr 172.21.15.77:0/470212403 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:58:21.550 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.509+0000 7fda69400640 1 --2- 172.21.15.77:0/470212403 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b092504d80 0x55b092507170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:21.550 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.509+0000 7fda6a800640 1 -- 172.21.15.77:0/470212403 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0924e9370 msgr2=0x55b092504840 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T09:58:21.550 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.509+0000 7fda6a800640 1 --2- 172.21.15.77:0/470212403 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0924e9370 0x55b092504840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:21.550 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.509+0000 7fda6a800640 1 -- 172.21.15.77:0/470212403 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b092504d80 msgr2=0x55b092507170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:21.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.509+0000 7fda6a800640 1 --2- 172.21.15.77:0/470212403 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b092504d80 0x55b092507170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:21.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.509+0000 7fda6a800640 1 -- 172.21.15.77:0/470212403 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b092415ae0 con 0x55b0924e8a50 2024-09-10T09:58:21.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.510+0000 7fda6a800640 1 --2- 172.21.15.77:0/470212403 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b0924e8a50 0x55b0924e8e30 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fda64009cd0 tx=0x7fda64009a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4107ba2c40819e05 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:21.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.510+0000 7fda62a00640 1 --2- 172.21.15.77:0/850108523 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fda500a2c80 0x7fda500a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:21.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.510+0000 7fda62a00640 1 -- 172.21.15.77:0/850108523 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fda580cf5f0 con 0x7fda6c1036e0 2024-09-10T09:58:21.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.510+0000 7fda57000640 1 -- 172.21.15.77:0/470212403 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda64049070 con 0x55b0924e8a50 2024-09-10T09:58:21.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.510+0000 7fda57000640 1 -- 172.21.15.77:0/470212403 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda64036040 con 0x55b0924e8a50 2024-09-10T09:58:21.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.510+0000 7fda57000640 1 -- 172.21.15.77:0/470212403 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda64004040 con 0x55b0924e8a50 2024-09-10T09:58:21.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.511+0000 7fda69e00640 1 --2- 172.21.15.77:0/850108523 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fda500a2c80 0x7fda500a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:21.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.511+0000 7fda7387a740 1 -- 172.21.15.77:0/470212403 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b0924e8a50 msgr2=0x55b0924e8e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:21.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.511+0000 7fda7387a740 1 --2- 172.21.15.77:0/470212403 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b0924e8a50 0x55b0924e8e30 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fda64009cd0 tx=0x7fda64009a20 comp rx=0 tx=0).stop 2024-09-10T09:58:21.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.511+0000 7fda7387a740 1 -- 172.21.15.77:0/470212403 shutdown_connections 2024-09-10T09:58:21.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.511+0000 7fda7387a740 1 --2- 172.21.15.77:0/470212403 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b092504d80 0x55b092507170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:21.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.511+0000 7fda7387a740 1 --2- 172.21.15.77:0/470212403 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0924e9370 0x55b092504840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:21.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.511+0000 7fda7387a740 1 --2- 172.21.15.77:0/470212403 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b0924e8a50 0x55b0924e8e30 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:21.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.511+0000 7fda7387a740 1 -- 172.21.15.77:0/470212403 >> 172.21.15.77:0/470212403 conn(0x55b0924e5120 msgr2=0x55b0924e5500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:21.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.511+0000 7fda7387a740 1 -- 172.21.15.77:0/470212403 shutdown_connections 2024-09-10T09:58:21.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.512+0000 7fda7387a740 1 -- 172.21.15.77:0/470212403 wait complete. 2024-09-10T09:58:21.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.512+0000 7fda7387a740 1 -- start start 2024-09-10T09:58:21.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.512+0000 7fda7387a740 1 Processor -- start 2024-09-10T09:58:21.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.512+0000 7fda69e00640 1 --2- 172.21.15.77:0/850108523 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fda500a2c80 0x7fda500a5140 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fda5c009a80 tx=0x7fda5c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:21.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.513+0000 7fda7387a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b0924e8a50 0x55b092501cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:21.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.513+0000 7fda7387a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0924e9370 0x55b092502230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:21.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.513+0000 7fda7387a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b092504d80 0x55b092502770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:21.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.513+0000 7fda7387a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55b092507990 con 0x55b092504d80 2024-09-10T09:58:21.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.513+0000 7fda7387a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b09240a0f0 con 0x55b0924e9370 2024-09-10T09:58:21.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.513+0000 7fda7387a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b092507b10 con 0x55b0924e8a50 2024-09-10T09:58:21.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.513+0000 7fda7387a740 10 client.0 _ll_register_callbacks cb 0x55b0923fb9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:58:21.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.513+0000 7fda69e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b0924e8a50 0x55b092501cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:21.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.513+0000 7fda69e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b0924e8a50 0x55b092501cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:48398/0 (socket says 172.21.15.77:48398) 2024-09-10T09:58:21.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.513+0000 7fda69e00640 1 -- 172.21.15.77:0/3774643107 learned_addr learned my addr 172.21.15.77:0/3774643107 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:58:21.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.513+0000 7fda69400640 1 --2- 172.21.15.77:0/3774643107 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0924e9370 0x55b092502230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:21.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.514+0000 7fda6a800640 1 --2- 172.21.15.77:0/3774643107 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b092504d80 0x55b092502770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:21.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.514+0000 7fda69e00640 1 -- 172.21.15.77:0/3774643107 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0924e9370 msgr2=0x55b092502230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:21.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.514+0000 7fda69e00640 1 --2- 172.21.15.77:0/3774643107 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0924e9370 0x55b092502230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:21.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.514+0000 7fda69e00640 1 -- 172.21.15.77:0/3774643107 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b092504d80 msgr2=0x55b092502770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:21.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.514+0000 7fda69e00640 1 --2- 172.21.15.77:0/3774643107 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b092504d80 0x55b092502770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:21.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.514+0000 7fda69e00640 1 -- 172.21.15.77:0/3774643107 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0921c03c0 con 0x55b0924e8a50 2024-09-10T09:58:21.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.514+0000 7fda6a800640 1 --2- 172.21.15.77:0/3774643107 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b092504d80 0x55b092502770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:58:21.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.514+0000 7fda69e00640 1 --2- 172.21.15.77:0/3774643107 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b0924e8a50 0x55b092501cf0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fda5c010ec0 tx=0x7fda5c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75656eb0d6f852d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:21.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.514+0000 7fda4b400640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-10T09:58:21.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.514+0000 7fda4b400640 1 -- 172.21.15.77:0/3774643107 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda5c023070 con 0x55b0924e8a50 2024-09-10T09:58:21.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.515+0000 7fda7387a740 1 -- 172.21.15.77:0/3774643107 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda64036040 con 0x55b0924e8a50 2024-09-10T09:58:21.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.515+0000 7fda4b400640 1 -- 172.21.15.77:0/3774643107 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fda5c005220 con 0x55b0924e8a50 2024-09-10T09:58:21.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.515+0000 7fda4b400640 1 -- 172.21.15.77:0/3774643107 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fda5c02e050 con 0x55b0924e8a50 2024-09-10T09:58:21.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.515+0000 7fda7387a740 10 client.26056 fetch_fsmap learned FSMap version 32 2024-09-10T09:58:21.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.515+0000 7fda7387a740 1 -- 172.21.15.77:0/3774643107 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b092513870 con 0x55b0924e8a50 2024-09-10T09:58:21.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.514+0000 7fda69400640 1 --2- 172.21.15.77:0/3774643107 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0924e9370 0x55b092502230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:58:21.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.515+0000 7fda4b400640 1 -- 172.21.15.77:0/3774643107 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda64036040 con 0x55b0924e8a50 2024-09-10T09:58:21.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.515+0000 7fda4b400640 1 -- 172.21.15.77:0/3774643107 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fda5c00f040 con 0x55b0924e8a50 2024-09-10T09:58:21.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.515+0000 7fda7387a740 10 client.26056 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T09:58:21.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.515+0000 7fda7387a740 10 client.26056 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:58:21.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.516+0000 7fda7387a740 20 client.26056 populate_metadata read hostname 'smithi077' 2024-09-10T09:58:21.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.516+0000 7fda7387a740 1 --2- 172.21.15.77:0/3774643107 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55b092514cd0 0x55b092517190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:21.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.516+0000 7fda7387a740 4 client.26056 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:58:21.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.516+0000 7fda7387a740 1 -- 172.21.15.77:0/3774643107 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b09245c950 con 0x55b092514cd0 2024-09-10T09:58:21.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.517+0000 7fda69400640 1 --2- 172.21.15.77:0/3774643107 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55b092514cd0 0x55b092517190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:21.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.517+0000 7fda69400640 1 --2- 172.21.15.77:0/3774643107 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55b092514cd0 0x55b092517190 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d90a31aff462dbfe server_cookie=144523f0bd610d1c in_seq=0 out_seq=0 2024-09-10T09:58:21.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.517+0000 7fda4b400640 10 client.26056 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T09:58:21.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.518+0000 7fda4b400640 1 -- 172.21.15.77:0/3774643107 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b09245c950 con 0x55b092514cd0 2024-09-10T09:58:21.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.518+0000 7fda4b400640 10 client.26056 handle_command_reply: tid=0 2024-09-10T09:58:21.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.518+0000 7fda7387a740 1 client.26056 shutdown 2024-09-10T09:58:21.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.518+0000 7fda7387a740 1 -- 172.21.15.77:0/3774643107 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b0924e8a50 msgr2=0x55b092501cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:21.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.518+0000 7fda7387a740 1 --2- 172.21.15.77:0/3774643107 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b0924e8a50 0x55b092501cf0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fda5c010ec0 tx=0x7fda5c0046c0 comp rx=0 tx=0).stop 2024-09-10T09:58:21.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.518+0000 7fda7387a740 1 -- 172.21.15.77:0/3774643107 shutdown_connections 2024-09-10T09:58:21.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.518+0000 7fda7387a740 1 --2- 172.21.15.77:0/3774643107 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55b092514cd0 0x55b092517190 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:21.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.518+0000 7fda4b400640 0 client.26056 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T09:58:21.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.518+0000 7fda7387a740 1 --2- 172.21.15.77:0/3774643107 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b092504d80 0x55b092502770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:21.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.518+0000 7fda7387a740 1 --2- 172.21.15.77:0/3774643107 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0924e9370 0x55b092502230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:21.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.518+0000 7fda7387a740 1 --2- 172.21.15.77:0/3774643107 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b0924e8a50 0x55b092501cf0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:21.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.519+0000 7fda7387a740 1 -- 172.21.15.77:0/3774643107 >> 172.21.15.77:0/3774643107 conn(0x55b0924e5120 msgr2=0x55b0924eae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:21.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.519+0000 7fda7387a740 1 -- 172.21.15.77:0/3774643107 shutdown_connections 2024-09-10T09:58:21.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.519+0000 7fda7387a740 1 -- 172.21.15.77:0/3774643107 wait complete. 2024-09-10T09:58:21.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.519+0000 7fda7387a740 20 client.26056 trim_cache size 0 max 16384 2024-09-10T09:58:21.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.547+0000 7fda7387a740 1 Processor -- start 2024-09-10T09:58:21.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.547+0000 7fda7387a740 1 -- start start 2024-09-10T09:58:21.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.547+0000 7fda7387a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b0924e8a50 0x55b09251ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:21.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.547+0000 7fda7387a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0924e9370 0x55b09251cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:21.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.547+0000 7fda7387a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b092504d80 0x55b0924ea4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:21.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.547+0000 7fda7387a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fda4c00fd20 con 0x55b092504d80 2024-09-10T09:58:21.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.547+0000 7fda7387a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fda4c011920 con 0x55b0924e9370 2024-09-10T09:58:21.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.547+0000 7fda7387a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fda4c0103f0 con 0x55b0924e8a50 2024-09-10T09:58:21.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.548+0000 7fda69e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b0924e8a50 0x55b09251ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:21.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.548+0000 7fda6a800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b092504d80 0x55b0924ea4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:21.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.548+0000 7fda69e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b0924e8a50 0x55b09251ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:48404/0 (socket says 172.21.15.77:48404) 2024-09-10T09:58:21.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.548+0000 7fda69e00640 1 -- 172.21.15.77:0/3702987093 learned_addr learned my addr 172.21.15.77:0/3702987093 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:58:21.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.549+0000 7fda6a800640 1 -- 172.21.15.77:0/3702987093 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b0924e8a50 msgr2=0x55b09251ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:21.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.549+0000 7fda6a800640 1 --2- 172.21.15.77:0/3702987093 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b0924e8a50 0x55b09251ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:21.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.549+0000 7fda6a800640 1 -- 172.21.15.77:0/3702987093 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0924e9370 msgr2=0x55b09251cfa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T09:58:21.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.549+0000 7fda6a800640 1 --2- 172.21.15.77:0/3702987093 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0924e9370 0x55b09251cfa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:21.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.549+0000 7fda6a800640 1 -- 172.21.15.77:0/3702987093 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0924e9c40 con 0x55b092504d80 2024-09-10T09:58:21.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.549+0000 7fda6a800640 1 --2- 172.21.15.77:0/3702987093 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b092504d80 0x55b0924ea4f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fda6402ae00 tx=0x7fda64004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15e5cc2cebe8641f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:21.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.550+0000 7fda56600640 1 -- 172.21.15.77:0/3702987093 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda64049070 con 0x55b092504d80 2024-09-10T09:58:21.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.550+0000 7fda56600640 1 -- 172.21.15.77:0/3702987093 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda64036040 con 0x55b092504d80 2024-09-10T09:58:21.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.550+0000 7fda56600640 1 -- 172.21.15.77:0/3702987093 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda640042a0 con 0x55b092504d80 2024-09-10T09:58:21.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.551+0000 7fda7387a740 1 -- 172.21.15.77:0/3702987093 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b092504d80 msgr2=0x55b0924ea4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:21.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.551+0000 7fda7387a740 1 --2- 172.21.15.77:0/3702987093 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b092504d80 0x55b0924ea4f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fda6402ae00 tx=0x7fda64004a40 comp rx=0 tx=0).stop 2024-09-10T09:58:21.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.551+0000 7fda7387a740 1 -- 172.21.15.77:0/3702987093 shutdown_connections 2024-09-10T09:58:21.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.551+0000 7fda7387a740 1 --2- 172.21.15.77:0/3702987093 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b092504d80 0x55b0924ea4f0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:21.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.551+0000 7fda7387a740 1 --2- 172.21.15.77:0/3702987093 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0924e9370 0x55b09251cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:21.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.551+0000 7fda7387a740 1 --2- 172.21.15.77:0/3702987093 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b0924e8a50 0x55b09251ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:21.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.551+0000 7fda7387a740 1 -- 172.21.15.77:0/3702987093 >> 172.21.15.77:0/3702987093 conn(0x55b0924e5120 msgr2=0x55b092500de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:21.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.552+0000 7fda7387a740 1 -- 172.21.15.77:0/3702987093 shutdown_connections 2024-09-10T09:58:21.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.552+0000 7fda7387a740 1 -- 172.21.15.77:0/3702987093 wait complete. 2024-09-10T09:58:21.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.552+0000 7fda7387a740 1 -- start start 2024-09-10T09:58:21.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.553+0000 7fda7387a740 1 Processor -- start 2024-09-10T09:58:21.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.553+0000 7fda7387a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0924e8a50 0x55b092578170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:21.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.553+0000 7fda7387a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b0924e9370 0x55b0925786b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:21.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.553+0000 7fda7387a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b092504d80 0x55b09250be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:21.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.553+0000 7fda7387a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fda4c011190 con 0x55b0924e9370 2024-09-10T09:58:21.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.554+0000 7fda7387a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fda4c010ac0 con 0x55b0924e8a50 2024-09-10T09:58:21.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.554+0000 7fda7387a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fda4c00dd00 con 0x55b092504d80 2024-09-10T09:58:21.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.554+0000 7fda7387a740 10 client.0 _ll_register_callbacks cb 0x7fda5c010b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:58:21.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.554+0000 7fda69e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0924e8a50 0x55b092578170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:21.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.554+0000 7fda69e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0924e8a50 0x55b092578170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:34610/0 (socket says 172.21.15.77:34610) 2024-09-10T09:58:21.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.554+0000 7fda69e00640 1 -- 172.21.15.77:0/1385115942 learned_addr learned my addr 172.21.15.77:0/1385115942 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:58:21.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.554+0000 7fda6a800640 1 --2- 172.21.15.77:0/1385115942 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b092504d80 0x55b09250be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:21.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.555+0000 7fda6a800640 1 -- 172.21.15.77:0/1385115942 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0924e8a50 msgr2=0x55b092578170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:21.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.555+0000 7fda6a800640 1 --2- 172.21.15.77:0/1385115942 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0924e8a50 0x55b092578170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:21.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.555+0000 7fda6a800640 1 -- 172.21.15.77:0/1385115942 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b0924e9370 msgr2=0x55b0925786b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T09:58:21.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.555+0000 7fda6a800640 1 --2- 172.21.15.77:0/1385115942 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b0924e9370 0x55b0925786b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:21.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.555+0000 7fda6a800640 1 -- 172.21.15.77:0/1385115942 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b09250c360 con 0x55b092504d80 2024-09-10T09:58:21.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.556+0000 7fda6a800640 1 --2- 172.21.15.77:0/1385115942 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b092504d80 0x55b09250be20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fda64009cd0 tx=0x7fda64031820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=abf42a7fa65893d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:21.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.556+0000 7fda4b400640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-10T09:58:21.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.556+0000 7fda4b400640 1 -- 172.21.15.77:0/1385115942 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda6405c060 con 0x55b092504d80 2024-09-10T09:58:21.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.557+0000 7fda7387a740 1 -- 172.21.15.77:0/1385115942 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda64036040 con 0x55b092504d80 2024-09-10T09:58:21.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.557+0000 7fda4b400640 1 -- 172.21.15.77:0/1385115942 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fda64031ad0 con 0x55b092504d80 2024-09-10T09:58:21.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.557+0000 7fda4b400640 1 -- 172.21.15.77:0/1385115942 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fda64063050 con 0x55b092504d80 2024-09-10T09:58:21.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.557+0000 7fda7387a740 10 client.26062 fetch_fsmap learned FSMap version 32 2024-09-10T09:58:21.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.557+0000 7fda7387a740 1 -- 172.21.15.77:0/1385115942 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b09250d670 con 0x55b092504d80 2024-09-10T09:58:21.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.558+0000 7fda4b400640 1 -- 172.21.15.77:0/1385115942 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda64036040 con 0x55b092504d80 2024-09-10T09:58:21.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.558+0000 7fda4b400640 1 -- 172.21.15.77:0/1385115942 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fda64030070 con 0x55b092504d80 2024-09-10T09:58:21.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.558+0000 7fda7387a740 10 client.26062 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T09:58:21.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.558+0000 7fda7387a740 10 client.26062 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:58:21.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.558+0000 7fda7387a740 20 client.26062 populate_metadata read hostname 'smithi077' 2024-09-10T09:58:21.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.558+0000 7fda7387a740 1 --2- 172.21.15.77:0/1385115942 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55b092514cd0 0x55b092592a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:21.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.558+0000 7fda7387a740 4 client.26062 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T09:58:21.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.558+0000 7fda7387a740 1 -- 172.21.15.77:0/1385115942 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b092456e60 con 0x55b092514cd0 2024-09-10T09:58:21.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.559+0000 7fda69e00640 1 --2- 172.21.15.77:0/1385115942 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55b092514cd0 0x55b092592a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:21.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.559+0000 7fda69e00640 1 --2- 172.21.15.77:0/1385115942 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55b092514cd0 0x55b092592a50 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e75d4cfb9f6656f server_cookie=b3b52ade2750c3b2 in_seq=0 out_seq=0 2024-09-10T09:58:21.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.559+0000 7fda4b400640 10 client.26062 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T09:58:21.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.560+0000 7fda4b400640 1 -- 172.21.15.77:0/1385115942 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b092456e60 con 0x55b092514cd0 2024-09-10T09:58:21.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.560+0000 7fda4b400640 10 client.26062 handle_command_reply: tid=0 2024-09-10T09:58:21.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.560+0000 7fda7387a740 1 client.26062 shutdown 2024-09-10T09:58:21.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.560+0000 7fda7387a740 1 -- 172.21.15.77:0/1385115942 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b092504d80 msgr2=0x55b09250be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:21.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.560+0000 7fda7387a740 1 --2- 172.21.15.77:0/1385115942 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b092504d80 0x55b09250be20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fda64009cd0 tx=0x7fda64031820 comp rx=0 tx=0).stop 2024-09-10T09:58:21.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.560+0000 7fda7387a740 1 -- 172.21.15.77:0/1385115942 shutdown_connections 2024-09-10T09:58:21.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.560+0000 7fda7387a740 1 --2- 172.21.15.77:0/1385115942 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55b092514cd0 0x55b092592a50 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:21.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.560+0000 7fda7387a740 1 --2- 172.21.15.77:0/1385115942 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b092504d80 0x55b09250be20 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:21.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.560+0000 7fda4b400640 0 client.26062 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T09:58:21.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.560+0000 7fda7387a740 1 --2- 172.21.15.77:0/1385115942 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b0924e9370 0x55b0925786b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:21.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.560+0000 7fda7387a740 1 --2- 172.21.15.77:0/1385115942 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0924e8a50 0x55b092578170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:21.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.560+0000 7fda7387a740 1 -- 172.21.15.77:0/1385115942 >> 172.21.15.77:0/1385115942 conn(0x55b0924e5120 msgr2=0x55b0924e39c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:21.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.562+0000 7fda7387a740 1 -- 172.21.15.77:0/1385115942 shutdown_connections 2024-09-10T09:58:21.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.562+0000 7fda7387a740 1 -- 172.21.15.77:0/1385115942 wait complete. 2024-09-10T09:58:21.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.562+0000 7fda7387a740 20 client.26062 trim_cache size 0 max 16384 2024-09-10T09:58:21.567 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-10T09:58:21.567 INFO:teuthology.orchestra.run.smithi077.stdout: "status": "no active scrubs running", 2024-09-10T09:58:21.567 INFO:teuthology.orchestra.run.smithi077.stdout: "scrubs": {} 2024-09-10T09:58:21.567 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-10T09:58:21.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.563+0000 7fda70c00640 1 -- 172.21.15.77:0/850108523 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fda500a2c80 msgr2=0x7fda500a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:21.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.563+0000 7fda70c00640 1 --2- 172.21.15.77:0/850108523 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fda500a2c80 0x7fda500a5140 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fda5c009a80 tx=0x7fda5c005f90 comp rx=0 tx=0).stop 2024-09-10T09:58:21.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.563+0000 7fda70c00640 1 -- 172.21.15.77:0/850108523 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fda6c1036e0 msgr2=0x7fda6c104160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:21.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.563+0000 7fda70c00640 1 --2- 172.21.15.77:0/850108523 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fda6c1036e0 0x7fda6c104160 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fda5800e9f0 tx=0x7fda5800eeb0 comp rx=0 tx=0).stop 2024-09-10T09:58:21.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.564+0000 7fda70c00640 1 -- 172.21.15.77:0/850108523 shutdown_connections 2024-09-10T09:58:21.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.564+0000 7fda70c00640 1 --2- 172.21.15.77:0/850108523 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fda500a2c80 0x7fda500a5140 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:21.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.564+0000 7fda70c00640 1 --2- 172.21.15.77:0/850108523 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fda6c1082a0 0x7fda6c1046a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:21.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.564+0000 7fda70c00640 1 --2- 172.21.15.77:0/850108523 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fda6c1036e0 0x7fda6c104160 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:21.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.564+0000 7fda70c00640 1 --2- 172.21.15.77:0/850108523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda6c102d30 0x7fda6c103c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:21.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.564+0000 7fda70c00640 1 -- 172.21.15.77:0/850108523 >> 172.21.15.77:0/850108523 conn(0x7fda6c0fc7d0 msgr2=0x7fda6c101370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:21.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.566+0000 7fda70c00640 1 -- 172.21.15.77:0/850108523 shutdown_connections 2024-09-10T09:58:21.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:21.566+0000 7fda70c00640 1 -- 172.21.15.77:0/850108523 wait complete. 2024-09-10T09:58:21.594 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] trunc 2024-09-10T09:58:21.787 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] fs_perms 2024-09-10T09:58:21.857 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] looptest 2024-09-10T09:58:21.857 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] testx 2024-09-10T09:58:21.901 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] locktest 2024-09-10T09:58:22.120 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] unwritten_mmap 2024-09-10T09:58:22.123 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] bulkstat_unlink_test 2024-09-10T09:58:22.198 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] deduperace 2024-09-10T09:58:22.422 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] bulkstat_unlink_test_modified 2024-09-10T09:58:22.439 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] t_dir_offset 2024-09-10T09:58:22.476 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] t_futimens 2024-09-10T09:58:22.563 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] t_immutable 2024-09-10T09:58:22.618 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T09:58:22.618 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:58:22.746 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] stale_handle 2024-09-10T09:58:22.799 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] pwrite_mmap_blocked 2024-09-10T09:58:22.815 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] t_dir_offset2 2024-09-10T09:58:23.120 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:58:23.127 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] seek_sanity_test 2024-09-10T09:58:23.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:58:22 smithi134 ceph-mon[35674]: pgmap v487: 129 pgs: 129 active+clean; 75 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 808 KiB/s wr, 116 op/s 2024-09-10T09:58:23.134 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] seek_copy_test 2024-09-10T09:58:23.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:58:22 smithi120 ceph-mon[34770]: pgmap v487: 129 pgs: 129 active+clean; 75 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 808 KiB/s wr, 116 op/s 2024-09-10T09:58:23.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:58:22 smithi077 ceph-mon[30319]: pgmap v487: 129 pgs: 129 active+clean; 75 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 808 KiB/s wr, 116 op/s 2024-09-10T09:58:23.200 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] t_readdir_1 2024-09-10T09:58:23.490 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] t_readdir_2 2024-09-10T09:58:23.506 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] fsync-tester 2024-09-10T09:58:23.538 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] nsexec 2024-09-10T09:58:23.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:23.786+0000 7f2348e00640 1 -- 172.21.15.77:0/917168352 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f23441049a0 msgr2=0x7f23441089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:23.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:23.786+0000 7f2348e00640 1 --2- 172.21.15.77:0/917168352 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f23441049a0 0x7f23441089f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2330009900 tx=0x7f233002a400 comp rx=0 tx=0).stop 2024-09-10T09:58:23.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:23.786+0000 7f2348e00640 1 -- 172.21.15.77:0/917168352 shutdown_connections 2024-09-10T09:58:23.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:23.786+0000 7f2348e00640 1 --2- 172.21.15.77:0/917168352 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2344109120 0x7f2344110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:23.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:23.786+0000 7f2348e00640 1 --2- 172.21.15.77:0/917168352 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f23441049a0 0x7f23441089f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:23.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:23.786+0000 7f2348e00640 1 --2- 172.21.15.77:0/917168352 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2344103ff0 0x7f23441043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:23.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:23.786+0000 7f2348e00640 1 -- 172.21.15.77:0/917168352 >> 172.21.15.77:0/917168352 conn(0x7f23440ffa90 msgr2=0x7f2344101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:23.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:23.787+0000 7f2348e00640 1 -- 172.21.15.77:0/917168352 shutdown_connections 2024-09-10T09:58:23.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:23.787+0000 7f2348e00640 1 -- 172.21.15.77:0/917168352 wait complete. 2024-09-10T09:58:23.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:23.788+0000 7f2348e00640 1 Processor -- start 2024-09-10T09:58:23.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:23.788+0000 7f2348e00640 1 -- start start 2024-09-10T09:58:23.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:23.789+0000 7f2348e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2344103ff0 0x7f23440737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:23.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:23.789+0000 7f2348e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f23441049a0 0x7f2344073d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:23.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:23.789+0000 7f2348e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2344109120 0x7f234406c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:23.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:23.789+0000 7f2348e00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2344114170 con 0x7f2344103ff0 2024-09-10T09:58:23.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:23.789+0000 7f2348e00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2344113ff0 con 0x7f23441049a0 2024-09-10T09:58:23.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:23.789+0000 7f2348e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f23441142f0 con 0x7f2344109120 2024-09-10T09:58:23.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:23.789+0000 7f2341400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f23441049a0 0x7f2344073d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:23.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:23.789+0000 7f2341400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f23441049a0 0x7f2344073d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:48474/0 (socket says 172.21.15.77:48474) 2024-09-10T09:58:23.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:23.789+0000 7f2341400640 1 -- 172.21.15.77:0/1687853039 learned_addr learned my addr 172.21.15.77:0/1687853039 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:58:23.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:23.790+0000 7f2341400640 1 -- 172.21.15.77:0/1687853039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2344109120 msgr2=0x7f234406c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:23.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:23.790+0000 7f2342800640 1 --2- 172.21.15.77:0/1687853039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2344109120 0x7f234406c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:23.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:23.790+0000 7f2341400640 1 --2- 172.21.15.77:0/1687853039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2344109120 0x7f234406c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:23.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:23.790+0000 7f2341400640 1 -- 172.21.15.77:0/1687853039 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2344103ff0 msgr2=0x7f23440737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:23.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:23.790+0000 7f2341e00640 1 --2- 172.21.15.77:0/1687853039 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2344103ff0 0x7f23440737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:23.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:23.790+0000 7f2341400640 1 --2- 172.21.15.77:0/1687853039 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2344103ff0 0x7f23440737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:23.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:23.790+0000 7f2341400640 1 -- 172.21.15.77:0/1687853039 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f234406d130 con 0x7f23441049a0 2024-09-10T09:58:23.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:23.791+0000 7f2341e00640 1 --2- 172.21.15.77:0/1687853039 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2344103ff0 0x7f23440737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:58:23.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:23.791+0000 7f2341400640 1 --2- 172.21.15.77:0/1687853039 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f23441049a0 0x7f2344073d20 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2330009850 tx=0x7f23300309c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:23.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:23.791+0000 7f233aa00640 1 -- 172.21.15.77:0/1687853039 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f233003f070 con 0x7f23441049a0 2024-09-10T09:58:23.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:23.792+0000 7f233aa00640 1 -- 172.21.15.77:0/1687853039 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23300035f0 con 0x7f23441049a0 2024-09-10T09:58:23.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:23.792+0000 7f233aa00640 1 -- 172.21.15.77:0/1687853039 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2330035030 con 0x7f23441049a0 2024-09-10T09:58:23.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:23.792+0000 7f2348e00640 1 -- 172.21.15.77:0/1687853039 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f234406d3c0 con 0x7f23441049a0 2024-09-10T09:58:23.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:23.792+0000 7f2348e00640 1 -- 172.21.15.77:0/1687853039 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2344071310 con 0x7f23441049a0 2024-09-10T09:58:23.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:23.793+0000 7f2348e00640 1 -- 172.21.15.77:0/1687853039 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23441054d0 con 0x7f23441049a0 2024-09-10T09:58:23.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:23.796+0000 7f233aa00640 1 -- 172.21.15.77:0/1687853039 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2330042070 con 0x7f23441049a0 2024-09-10T09:58:23.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:23.798+0000 7f233aa00640 1 --2- 172.21.15.77:0/1687853039 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f23280a2c80 0x7f23280a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:23.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:23.798+0000 7f233aa00640 1 -- 172.21.15.77:0/1687853039 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f23300f03e0 con 0x7f23441049a0 2024-09-10T09:58:23.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:23.799+0000 7f2341e00640 1 --2- 172.21.15.77:0/1687853039 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f23280a2c80 0x7f23280a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:23.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:23.800+0000 7f233aa00640 1 -- 172.21.15.77:0/1687853039 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2330049070 con 0x7f23441049a0 2024-09-10T09:58:23.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:23.801+0000 7f2341e00640 1 --2- 172.21.15.77:0/1687853039 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f23280a2c80 0x7f23280a5140 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f233400baa0 tx=0x7f2334005d90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:23.851 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] cloner 2024-09-10T09:58:23.916 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] renameat2 2024-09-10T09:58:23.956 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] t_getcwd 2024-09-10T09:58:24.010 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] e4compact 2024-09-10T09:58:24.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:24.140+0000 7f2348e00640 1 -- 172.21.15.77:0/1687853039 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2344071770 con 0x7f23441049a0 2024-09-10T09:58:24.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:24.141+0000 7f233aa00640 1 -- 172.21.15.77:0/1687853039 <== mon.2 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11996 (secure 0 0 0) 0x7f23300bcef0 con 0x7f23441049a0 2024-09-10T09:58:24.143 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:58:24.144 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":32,"btime":"2024-09-10T09:57:40:888996+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3159289675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3159289675},{"type":"v1","addr":"172.21.15.77:6835","nonce":3159289675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1951962341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1951962341},{"type":"v1","addr":"172.21.15.77:6837","nonce":1951962341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:52:23.368252+0000","modified":"2024-09-10T09:57:40.695191+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24563,"mds_1":24524,"mds_2":24520,"mds_3":24502,"mds_4":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14487":{"gid":14487,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/4291121888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":4291121888},{"type":"v1","addr":"172.21.15.77:6839","nonce":4291121888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14499":{"gid":14499,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6841/445284194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":445284194},{"type":"v1","addr":"172.21.15.77:6841","nonce":445284194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1005054423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1005054423},{"type":"v1","addr":"172.21.15.134:6833","nonce":1005054423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6833/3716307744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":3716307744},{"type":"v1","addr":"172.21.15.120:6833","nonce":3716307744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/1319892544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1319892544},{"type":"v1","addr":"172.21.15.120:6835","nonce":1319892544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.134:6835/3293933699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3293933699},{"type":"v1","addr":"172.21.15.134:6835","nonce":3293933699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.134:6837/2461497091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2461497091},{"type":"v1","addr":"172.21.15.134:6837","nonce":2461497091}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":20,"addr":"172.21.15.134:6839/1965117680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1965117680},{"type":"v1","addr":"172.21.15.134:6839","nonce":1965117680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":20,"addr":"172.21.15.120:6837/3033280709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3033280709},{"type":"v1","addr":"172.21.15.120:6837","nonce":3033280709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.120:6839/1777710157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1777710157},{"type":"v1","addr":"172.21.15.120:6839","nonce":1777710157}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24563,"qdb_cluster":[24563,24524,24520,24502,24496]},"id":1}]} 2024-09-10T09:58:24.145 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 32 2024-09-10T09:58:24.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:24.144+0000 7f2348e00640 1 -- 172.21.15.77:0/1687853039 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f23280a2c80 msgr2=0x7f23280a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:24.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:24.144+0000 7f2348e00640 1 --2- 172.21.15.77:0/1687853039 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f23280a2c80 0x7f23280a5140 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f233400baa0 tx=0x7f2334005d90 comp rx=0 tx=0).stop 2024-09-10T09:58:24.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:24.144+0000 7f2348e00640 1 -- 172.21.15.77:0/1687853039 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f23441049a0 msgr2=0x7f2344073d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:24.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:24.144+0000 7f2348e00640 1 --2- 172.21.15.77:0/1687853039 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f23441049a0 0x7f2344073d20 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2330009850 tx=0x7f23300309c0 comp rx=0 tx=0).stop 2024-09-10T09:58:24.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:24.144+0000 7f2348e00640 1 -- 172.21.15.77:0/1687853039 shutdown_connections 2024-09-10T09:58:24.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:24.144+0000 7f2348e00640 1 --2- 172.21.15.77:0/1687853039 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f23280a2c80 0x7f23280a5140 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:24.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:24.144+0000 7f2348e00640 1 --2- 172.21.15.77:0/1687853039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2344109120 0x7f234406c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:24.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:24.144+0000 7f2348e00640 1 --2- 172.21.15.77:0/1687853039 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f23441049a0 0x7f2344073d20 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:24.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:24.144+0000 7f2348e00640 1 --2- 172.21.15.77:0/1687853039 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2344103ff0 0x7f23440737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:24.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:24.144+0000 7f2348e00640 1 -- 172.21.15.77:0/1687853039 >> 172.21.15.77:0/1687853039 conn(0x7f23440ffa90 msgr2=0x7f23441002d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:24.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:24.145+0000 7f2348e00640 1 -- 172.21.15.77:0/1687853039 shutdown_connections 2024-09-10T09:58:24.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:24.145+0000 7f2348e00640 1 -- 172.21.15.77:0/1687853039 wait complete. 2024-09-10T09:58:24.256 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] test-nextquota 2024-09-10T09:58:24.316 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] attr-list-by-handle-cursor-test 2024-09-10T09:58:24.316 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] punch-alternating 2024-09-10T09:58:24.440 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] listxattr 2024-09-10T09:58:24.642 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] dio-interleaved 2024-09-10T09:58:24.669 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] t_dir_type 2024-09-10T09:58:24.685 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] dio-invalidate-cache 2024-09-10T09:58:24.783 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] stat_test 2024-09-10T09:58:24.853 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:58:24 smithi077 ceph-mon[30319]: pgmap v488: 129 pgs: 129 active+clean; 75 MiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 774 KiB/s wr, 122 op/s 2024-09-10T09:58:24.853 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:58:24 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/1687853039' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:58:24.991 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] t_encrypted_d_revalidate 2024-09-10T09:58:25.009 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] attr_replace_test 2024-09-10T09:58:25.072 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] swapon 2024-09-10T09:58:25.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:58:24 smithi134 ceph-mon[35674]: pgmap v488: 129 pgs: 129 active+clean; 75 MiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 774 KiB/s wr, 122 op/s 2024-09-10T09:58:25.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:58:24 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/1687853039' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:58:25.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:58:24 smithi120 ceph-mon[34770]: pgmap v488: 129 pgs: 129 active+clean; 75 MiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 774 KiB/s wr, 122 op/s 2024-09-10T09:58:25.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:58:24 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/1687853039' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:58:25.190 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T09:58:25.367 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] mkswap 2024-09-10T09:58:25.373 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] t_attr_corruption 2024-09-10T09:58:25.377 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] t_open_tmpfiles 2024-09-10T09:58:25.449 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] fscrypt-crypt-util 2024-09-10T09:58:25.717 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] bulkstat_null_ocount 2024-09-10T09:58:25.726 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] splice-test 2024-09-10T09:58:25.787 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:58:25.797 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] chprojid_fail 2024-09-10T09:58:26.029 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] detached_mounts_propagation 2024-09-10T09:58:26.095 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] ext4_resize 2024-09-10T09:58:26.112 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] t_readdir_3 2024-09-10T09:58:26.601 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] uuid_ioctl 2024-09-10T09:58:26.601 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] fiemap-fault 2024-09-10T09:58:26.601 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] t_snapshot_deleted_subvolume 2024-09-10T09:58:26.602 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] splice2pipe 2024-09-10T09:58:26.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.806+0000 7ff7d3a00640 1 -- 172.21.15.77:0/1747417178 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7cc10c150 msgr2=0x7ff7cc10e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:26.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.806+0000 7ff7d3a00640 1 --2- 172.21.15.77:0/1747417178 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7cc10c150 0x7ff7cc10e5f0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7ff7c4009ca0 tx=0x7ff7c402a960 comp rx=0 tx=0).stop 2024-09-10T09:58:26.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.807+0000 7ff7d3a00640 1 -- 172.21.15.77:0/1747417178 shutdown_connections 2024-09-10T09:58:26.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.807+0000 7ff7d3a00640 1 --2- 172.21.15.77:0/1747417178 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7cc10c150 0x7ff7cc10e5f0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:26.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.807+0000 7ff7d3a00640 1 --2- 172.21.15.77:0/1747417178 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff7cc103640 0x7ff7cc10bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:26.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.807+0000 7ff7d3a00640 1 --2- 172.21.15.77:0/1747417178 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff7cc102d20 0x7ff7cc103100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:26.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.807+0000 7ff7d3a00640 1 -- 172.21.15.77:0/1747417178 >> 172.21.15.77:0/1747417178 conn(0x7ff7cc0fc780 msgr2=0x7ff7cc0feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:26.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.807+0000 7ff7d3a00640 1 -- 172.21.15.77:0/1747417178 shutdown_connections 2024-09-10T09:58:26.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.807+0000 7ff7d3a00640 1 -- 172.21.15.77:0/1747417178 wait complete. 2024-09-10T09:58:26.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.808+0000 7ff7d3a00640 1 Processor -- start 2024-09-10T09:58:26.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.808+0000 7ff7d3a00640 1 -- start start 2024-09-10T09:58:26.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.809+0000 7ff7d3a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7cc102d20 0x7ff7cc19a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:26.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.809+0000 7ff7d3a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff7cc103640 0x7ff7cc19ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:26.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.809+0000 7ff7d3a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff7cc10c150 0x7ff7cc19ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:26.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.809+0000 7ff7d3a00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff7cc111f60 con 0x7ff7cc102d20 2024-09-10T09:58:26.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.809+0000 7ff7d3a00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff7cc111de0 con 0x7ff7cc10c150 2024-09-10T09:58:26.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.809+0000 7ff7d3a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff7cc1120e0 con 0x7ff7cc103640 2024-09-10T09:58:26.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.809+0000 7ff7d1000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7cc102d20 0x7ff7cc19a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:26.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.809+0000 7ff7d1000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7cc102d20 0x7ff7cc19a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33164/0 (socket says 172.21.15.77:33164) 2024-09-10T09:58:26.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.809+0000 7ff7d1000640 1 -- 172.21.15.77:0/2653801250 learned_addr learned my addr 172.21.15.77:0/2653801250 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:58:26.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.810+0000 7ff7cbe00640 1 --2- 172.21.15.77:0/2653801250 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff7cc103640 0x7ff7cc19ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:26.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.810+0000 7ff7d1a00640 1 --2- 172.21.15.77:0/2653801250 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff7cc10c150 0x7ff7cc19ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:26.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.810+0000 7ff7d1a00640 1 -- 172.21.15.77:0/2653801250 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff7cc103640 msgr2=0x7ff7cc19ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:26.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.810+0000 7ff7d1a00640 1 --2- 172.21.15.77:0/2653801250 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff7cc103640 0x7ff7cc19ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:26.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.810+0000 7ff7d1a00640 1 -- 172.21.15.77:0/2653801250 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7cc102d20 msgr2=0x7ff7cc19a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:26.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.810+0000 7ff7d1a00640 1 --2- 172.21.15.77:0/2653801250 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7cc102d20 0x7ff7cc19a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:26.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.810+0000 7ff7d1a00640 1 -- 172.21.15.77:0/2653801250 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7cc19f5e0 con 0x7ff7cc10c150 2024-09-10T09:58:26.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.810+0000 7ff7d1000640 1 --2- 172.21.15.77:0/2653801250 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7cc102d20 0x7ff7cc19a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:58:26.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.811+0000 7ff7d1a00640 1 --2- 172.21.15.77:0/2653801250 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff7cc10c150 0x7ff7cc19ef00 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff7c4005e30 tx=0x7ff7c4003f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:26.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.811+0000 7ff7c9600640 1 -- 172.21.15.77:0/2653801250 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7c4034030 con 0x7ff7cc10c150 2024-09-10T09:58:26.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.811+0000 7ff7c9600640 1 -- 172.21.15.77:0/2653801250 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7c402fdc0 con 0x7ff7cc10c150 2024-09-10T09:58:26.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.811+0000 7ff7c9600640 1 -- 172.21.15.77:0/2653801250 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7c40046a0 con 0x7ff7cc10c150 2024-09-10T09:58:26.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.811+0000 7ff7d3a00640 1 -- 172.21.15.77:0/2653801250 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7cc19f870 con 0x7ff7cc10c150 2024-09-10T09:58:26.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.811+0000 7ff7d3a00640 1 -- 172.21.15.77:0/2653801250 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7cc19fd00 con 0x7ff7cc10c150 2024-09-10T09:58:26.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.815+0000 7ff7c9600640 1 -- 172.21.15.77:0/2653801250 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff7c4037020 con 0x7ff7cc10c150 2024-09-10T09:58:26.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.816+0000 7ff7d64c4740 1 Processor -- start 2024-09-10T09:58:26.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.816+0000 7ff7d64c4740 1 -- start start 2024-09-10T09:58:26.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.816+0000 7ff7d64c4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a8ec092320 0x55a8ec092700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:26.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.816+0000 7ff7d64c4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8ec092c40 0x55a8ec08dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:26.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.816+0000 7ff7d64c4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a8ec0a60d0 0x55a8ec0a84c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:26.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.816+0000 7ff7d64c4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55a8ebff03e0 con 0x55a8ec092c40 2024-09-10T09:58:26.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.816+0000 7ff7d64c4740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a8ebf97d80 con 0x55a8ec0a60d0 2024-09-10T09:58:26.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.816+0000 7ff7d64c4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a8ebfec3a0 con 0x55a8ec092320 2024-09-10T09:58:26.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.817+0000 7ff7cbe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a8ec092320 0x55a8ec092700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:26.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.817+0000 7ff7cbe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a8ec092320 0x55a8ec092700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:42340/0 (socket says 172.21.15.77:42340) 2024-09-10T09:58:26.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.817+0000 7ff7cbe00640 1 -- 172.21.15.77:0/330664796 learned_addr learned my addr 172.21.15.77:0/330664796 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:58:26.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.817+0000 7ff7cbe00640 1 --2- 172.21.15.77:0/330664796 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a8ec0a60d0 0x55a8ec0a84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:26.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.817+0000 7ff7d1000640 1 --2- 172.21.15.77:0/330664796 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8ec092c40 0x55a8ec08dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:26.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.818+0000 7ff7c9600640 1 --2- 172.21.15.77:0/2653801250 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ff7b40a2c00 0x7ff7b40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:26.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.818+0000 7ff7cbe00640 1 -- 172.21.15.77:0/330664796 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a8ec0a60d0 msgr2=0x55a8ec0a84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:26.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.818+0000 7ff7cbe00640 1 --2- 172.21.15.77:0/330664796 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a8ec0a60d0 0x55a8ec0a84c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:26.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.818+0000 7ff7cbe00640 1 -- 172.21.15.77:0/330664796 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8ec092c40 msgr2=0x55a8ec08dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:26.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.818+0000 7ff7c9600640 1 -- 172.21.15.77:0/2653801250 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7ff7c40effe0 con 0x7ff7cc10c150 2024-09-10T09:58:26.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.818+0000 7ff7cbe00640 1 --2- 172.21.15.77:0/330664796 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8ec092c40 0x55a8ec08dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:26.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.818+0000 7ff7cbe00640 1 -- 172.21.15.77:0/330664796 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8ebfb6ae0 con 0x55a8ec092320 2024-09-10T09:58:26.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.818+0000 7ff7d1000640 1 --2- 172.21.15.77:0/330664796 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8ec092c40 0x55a8ec08dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:58:26.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.818+0000 7ff7d1a00640 1 --2- 172.21.15.77:0/2653801250 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ff7b40a2c00 0x7ff7b40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:26.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.819+0000 7ff7cbe00640 1 --2- 172.21.15.77:0/330664796 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a8ec092320 0x55a8ec092700 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff7b80147d0 tx=0x7ff7b80095b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6889a47e26f7b9e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:26.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.819+0000 7ff7be000640 1 -- 172.21.15.77:0/330664796 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b8014e10 con 0x55a8ec092320 2024-09-10T09:58:26.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.819+0000 7ff7be000640 1 -- 172.21.15.77:0/330664796 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7b8005500 con 0x55a8ec092320 2024-09-10T09:58:26.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.819+0000 7ff7be000640 1 -- 172.21.15.77:0/330664796 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b8004e50 con 0x55a8ec092320 2024-09-10T09:58:26.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.820+0000 7ff7d64c4740 1 -- 172.21.15.77:0/330664796 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a8ec092320 msgr2=0x55a8ec092700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:26.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.820+0000 7ff7d64c4740 1 --2- 172.21.15.77:0/330664796 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a8ec092320 0x55a8ec092700 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff7b80147d0 tx=0x7ff7b80095b0 comp rx=0 tx=0).stop 2024-09-10T09:58:26.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.820+0000 7ff7d1a00640 1 --2- 172.21.15.77:0/2653801250 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ff7b40a2c00 0x7ff7b40a50c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff7c40b8850 tx=0x7ff7c4004140 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:26.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.820+0000 7ff7d64c4740 1 -- 172.21.15.77:0/330664796 shutdown_connections 2024-09-10T09:58:26.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.820+0000 7ff7d64c4740 1 --2- 172.21.15.77:0/330664796 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a8ec0a60d0 0x55a8ec0a84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:26.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.820+0000 7ff7d64c4740 1 --2- 172.21.15.77:0/330664796 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8ec092c40 0x55a8ec08dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:26.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.820+0000 7ff7d64c4740 1 --2- 172.21.15.77:0/330664796 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a8ec092320 0x55a8ec092700 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:26.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.820+0000 7ff7d64c4740 1 -- 172.21.15.77:0/330664796 >> 172.21.15.77:0/330664796 conn(0x55a8ec08ea10 msgr2=0x55a8ec08edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:26.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.820+0000 7ff7d64c4740 1 -- 172.21.15.77:0/330664796 shutdown_connections 2024-09-10T09:58:26.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.821+0000 7ff7d64c4740 1 -- 172.21.15.77:0/330664796 wait complete. 2024-09-10T09:58:26.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.821+0000 7ff7d64c4740 1 -- start start 2024-09-10T09:58:26.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.821+0000 7ff7d64c4740 1 Processor -- start 2024-09-10T09:58:26.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.822+0000 7ff7d64c4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a8ec092320 0x55a8ec08b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:26.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.822+0000 7ff7d64c4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a8ec092c40 0x55a8ec08b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:26.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.822+0000 7ff7d64c4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8ec0a60d0 0x55a8ec08bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:26.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.822+0000 7ff7d64c4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55a8ec0a8a00 con 0x55a8ec0a60d0 2024-09-10T09:58:26.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.822+0000 7ff7d64c4740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a8ebfab0f0 con 0x55a8ec092c40 2024-09-10T09:58:26.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.822+0000 7ff7d64c4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a8ec0a8b80 con 0x55a8ec092320 2024-09-10T09:58:26.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.822+0000 7ff7d64c4740 10 client.0 _ll_register_callbacks cb 0x55a8ebf92bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:58:26.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.823+0000 7ff7cbe00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8ec0a60d0 0x55a8ec08bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:26.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.823+0000 7ff7cbe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a8ec092320 0x55a8ec08b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:26.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.823+0000 7ff7cbe00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8ec0a60d0 0x55a8ec08bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33184/0 (socket says 172.21.15.77:33184) 2024-09-10T09:58:26.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.823+0000 7ff7cbe00640 1 -- 172.21.15.77:0/352362809 learned_addr learned my addr 172.21.15.77:0/352362809 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:58:26.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.823+0000 7ff7d1000640 1 --2- 172.21.15.77:0/352362809 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a8ec092c40 0x55a8ec08b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:26.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.824+0000 7ff7cbe00640 1 -- 172.21.15.77:0/352362809 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a8ec092c40 msgr2=0x55a8ec08b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:26.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.824+0000 7ff7cbe00640 1 --2- 172.21.15.77:0/352362809 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a8ec092c40 0x55a8ec08b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:26.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.824+0000 7ff7cbe00640 1 -- 172.21.15.77:0/352362809 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8ec0a60d0 msgr2=0x55a8ec08bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:26.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.824+0000 7ff7cbe00640 1 --2- 172.21.15.77:0/352362809 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8ec0a60d0 0x55a8ec08bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:26.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.824+0000 7ff7cbe00640 1 -- 172.21.15.77:0/352362809 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8ebd613c0 con 0x55a8ec092320 2024-09-10T09:58:26.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.824+0000 7ff7cbe00640 1 --2- 172.21.15.77:0/352362809 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a8ec092320 0x55a8ec08b160 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff7b8014570 tx=0x7ff7b8009a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9363b884a27e0b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:26.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.824+0000 7ff7b2a00640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-10T09:58:26.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.824+0000 7ff7b2a00640 1 -- 172.21.15.77:0/352362809 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b8014e10 con 0x55a8ec092320 2024-09-10T09:58:26.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.825+0000 7ff7d64c4740 1 -- 172.21.15.77:0/352362809 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7b8005500 con 0x55a8ec092320 2024-09-10T09:58:26.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.825+0000 7ff7b2a00640 1 -- 172.21.15.77:0/352362809 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7ff7b8014110 con 0x55a8ec092320 2024-09-10T09:58:26.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.825+0000 7ff7b2a00640 1 -- 172.21.15.77:0/352362809 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff7b8004a30 con 0x55a8ec092320 2024-09-10T09:58:26.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.825+0000 7ff7d64c4740 10 client.26071 fetch_fsmap learned FSMap version 32 2024-09-10T09:58:26.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.825+0000 7ff7d64c4740 1 -- 172.21.15.77:0/352362809 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a8ec0b4b80 con 0x55a8ec092320 2024-09-10T09:58:26.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.826+0000 7ff7b2a00640 1 -- 172.21.15.77:0/352362809 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7b8005500 con 0x55a8ec092320 2024-09-10T09:58:26.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.826+0000 7ff7b2a00640 1 -- 172.21.15.77:0/352362809 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7ff7b802b020 con 0x55a8ec092320 2024-09-10T09:58:26.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.826+0000 7ff7d64c4740 10 client.26071 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T09:58:26.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.826+0000 7ff7d64c4740 10 client.26071 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-10T09:58:26.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.826+0000 7ff7d64c4740 20 client.26071 populate_metadata read hostname 'smithi077' 2024-09-10T09:58:26.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.826+0000 7ff7d64c4740 1 --2- 172.21.15.77:0/352362809 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x55a8ec0b5f90 0x55a8ec0b8380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:26.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.826+0000 7ff7d64c4740 4 client.26071 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:58:26.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.826+0000 7ff7d64c4740 1 -- 172.21.15.77:0/352362809 --> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a8ebffd950 con 0x55a8ec0b5f90 2024-09-10T09:58:26.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.827+0000 7ff7d1000640 1 --2- 172.21.15.77:0/352362809 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x55a8ec0b5f90 0x55a8ec0b8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:26.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.828+0000 7ff7d1000640 1 --2- 172.21.15.77:0/352362809 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x55a8ec0b5f90 0x55a8ec0b8380 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7c42807d72fb247b server_cookie=3497dae6e8ea3505 in_seq=0 out_seq=0 2024-09-10T09:58:26.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.828+0000 7ff7b2a00640 10 client.26071 ms_handle_connect on v2:172.21.15.134:6834/3293933699 2024-09-10T09:58:26.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.829+0000 7ff7b2a00640 1 -- 172.21.15.77:0/352362809 <== mds.4 v2:172.21.15.134:6834/3293933699 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a8ebffd950 con 0x55a8ec0b5f90 2024-09-10T09:58:26.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.829+0000 7ff7b2a00640 10 client.26071 handle_command_reply: tid=0 2024-09-10T09:58:26.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.829+0000 7ff7d64c4740 1 client.26071 shutdown 2024-09-10T09:58:26.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.829+0000 7ff7d64c4740 1 -- 172.21.15.77:0/352362809 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a8ec092320 msgr2=0x55a8ec08b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:26.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.829+0000 7ff7d64c4740 1 --2- 172.21.15.77:0/352362809 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a8ec092320 0x55a8ec08b160 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff7b8014570 tx=0x7ff7b8009a20 comp rx=0 tx=0).stop 2024-09-10T09:58:26.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.829+0000 7ff7d64c4740 1 -- 172.21.15.77:0/352362809 shutdown_connections 2024-09-10T09:58:26.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.829+0000 7ff7d64c4740 1 --2- 172.21.15.77:0/352362809 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x55a8ec0b5f90 0x55a8ec0b8380 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:26.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.829+0000 7ff7d64c4740 1 --2- 172.21.15.77:0/352362809 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8ec0a60d0 0x55a8ec08bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:26.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.830+0000 7ff7d64c4740 1 --2- 172.21.15.77:0/352362809 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a8ec092c40 0x55a8ec08b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:26.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.830+0000 7ff7d64c4740 1 --2- 172.21.15.77:0/352362809 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a8ec092320 0x55a8ec08b160 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:26.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.830+0000 7ff7d64c4740 1 -- 172.21.15.77:0/352362809 >> 172.21.15.77:0/352362809 conn(0x55a8ec08ea10 msgr2=0x55a8ec094770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:26.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.830+0000 7ff7b2a00640 0 client.26071 ms_handle_reset on v2:172.21.15.134:6834/3293933699 2024-09-10T09:58:26.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.830+0000 7ff7d64c4740 1 -- 172.21.15.77:0/352362809 shutdown_connections 2024-09-10T09:58:26.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.830+0000 7ff7d64c4740 1 -- 172.21.15.77:0/352362809 wait complete. 2024-09-10T09:58:26.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.830+0000 7ff7d64c4740 20 client.26071 trim_cache size 0 max 16384 2024-09-10T09:58:26.847 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.845+0000 7ff7d64c4740 1 Processor -- start 2024-09-10T09:58:26.847 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.845+0000 7ff7d64c4740 1 -- start start 2024-09-10T09:58:26.847 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.845+0000 7ff7d64c4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a8ec092320 0x55a8ec08a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:26.848 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.845+0000 7ff7d64c4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a8ec092c40 0x55a8ec0bdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:26.848 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.845+0000 7ff7d64c4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8ec0a60d0 0x55a8ec0bdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:26.848 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.845+0000 7ff7d64c4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff7cc1acc30 con 0x55a8ec0a60d0 2024-09-10T09:58:26.848 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.845+0000 7ff7d64c4740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff7cc1ae830 con 0x55a8ec092320 2024-09-10T09:58:26.848 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.845+0000 7ff7d64c4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff7cc1ad300 con 0x55a8ec092c40 2024-09-10T09:58:26.848 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.846+0000 7ff7cbe00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a8ec092320 0x55a8ec08a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:26.848 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.846+0000 7ff7cbe00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8ec0a60d0 0x55a8ec0bdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:26.848 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.846+0000 7ff7d1000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a8ec092c40 0x55a8ec0bdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:26.848 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.846+0000 7ff7d1000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a8ec092c40 0x55a8ec0bdaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:42356/0 (socket says 172.21.15.77:42356) 2024-09-10T09:58:26.848 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.846+0000 7ff7cbe00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8ec0a60d0 0x55a8ec0bdfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33196/0 (socket says 172.21.15.77:33196) 2024-09-10T09:58:26.848 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.846+0000 7ff7d1000640 1 -- 172.21.15.77:0/2848021325 learned_addr learned my addr 172.21.15.77:0/2848021325 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:58:26.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.846+0000 7ff7d1000640 1 -- 172.21.15.77:0/2848021325 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a8ec092320 msgr2=0x55a8ec08a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:26.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.846+0000 7ff7d1000640 1 --2- 172.21.15.77:0/2848021325 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a8ec092320 0x55a8ec08a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:26.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.846+0000 7ff7d1000640 1 -- 172.21.15.77:0/2848021325 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8ec0a60d0 msgr2=0x55a8ec0bdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:26.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.846+0000 7ff7d1000640 1 --2- 172.21.15.77:0/2848021325 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8ec0a60d0 0x55a8ec0bdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:26.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.846+0000 7ff7d1000640 1 -- 172.21.15.77:0/2848021325 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8ec081770 con 0x55a8ec092c40 2024-09-10T09:58:26.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.847+0000 7ff7d1000640 1 --2- 172.21.15.77:0/2848021325 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a8ec092c40 0x55a8ec0bdaa0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff7c0004700 tx=0x7ff7c0005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=275b06027463afe4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:26.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.847+0000 7ff7bd600640 1 -- 172.21.15.77:0/2848021325 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7c0004050 con 0x55a8ec092c40 2024-09-10T09:58:26.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.847+0000 7ff7bd600640 1 -- 172.21.15.77:0/2848021325 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7c000fd70 con 0x55a8ec092c40 2024-09-10T09:58:26.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.847+0000 7ff7bd600640 1 -- 172.21.15.77:0/2848021325 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7c00054b0 con 0x55a8ec092c40 2024-09-10T09:58:26.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.848+0000 7ff7d64c4740 1 -- 172.21.15.77:0/2848021325 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a8ec092c40 msgr2=0x55a8ec0bdaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:26.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.848+0000 7ff7d64c4740 1 --2- 172.21.15.77:0/2848021325 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a8ec092c40 0x55a8ec0bdaa0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff7c0004700 tx=0x7ff7c0005f90 comp rx=0 tx=0).stop 2024-09-10T09:58:26.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.848+0000 7ff7d64c4740 1 -- 172.21.15.77:0/2848021325 shutdown_connections 2024-09-10T09:58:26.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.848+0000 7ff7d64c4740 1 --2- 172.21.15.77:0/2848021325 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8ec0a60d0 0x55a8ec0bdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:26.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.848+0000 7ff7d64c4740 1 --2- 172.21.15.77:0/2848021325 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a8ec092c40 0x55a8ec0bdaa0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:26.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.848+0000 7ff7d64c4740 1 --2- 172.21.15.77:0/2848021325 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a8ec092320 0x55a8ec08a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:26.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.848+0000 7ff7d64c4740 1 -- 172.21.15.77:0/2848021325 >> 172.21.15.77:0/2848021325 conn(0x55a8ec08ea10 msgr2=0x55a8ec089a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:26.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.848+0000 7ff7d64c4740 1 -- 172.21.15.77:0/2848021325 shutdown_connections 2024-09-10T09:58:26.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.848+0000 7ff7d64c4740 1 -- 172.21.15.77:0/2848021325 wait complete. 2024-09-10T09:58:26.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.849+0000 7ff7d64c4740 1 -- start start 2024-09-10T09:58:26.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.849+0000 7ff7d64c4740 1 Processor -- start 2024-09-10T09:58:26.852 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.849+0000 7ff7d64c4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a8ec092320 0x55a8ec0ae5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:26.852 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.849+0000 7ff7d64c4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8ec092c40 0x55a8ec0a98d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:26.852 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.849+0000 7ff7d64c4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a8ec0a60d0 0x55a8ec0a9e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:26.852 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.849+0000 7ff7d64c4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff7cc1ae0a0 con 0x55a8ec092c40 2024-09-10T09:58:26.852 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.849+0000 7ff7d64c4740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff7cc1ad9d0 con 0x55a8ec0a60d0 2024-09-10T09:58:26.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.849+0000 7ff7d64c4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff7cc1aac70 con 0x55a8ec092320 2024-09-10T09:58:26.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.849+0000 7ff7d64c4740 10 client.0 _ll_register_callbacks cb 0x7ff7c0009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:58:26.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.850+0000 7ff7d1000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8ec092c40 0x55a8ec0a98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:26.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.850+0000 7ff7cbe00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a8ec0a60d0 0x55a8ec0a9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:26.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.850+0000 7ff7d1000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8ec092c40 0x55a8ec0a98d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33210/0 (socket says 172.21.15.77:33210) 2024-09-10T09:58:26.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.850+0000 7ff7d1000640 1 -- 172.21.15.77:0/2971558594 learned_addr learned my addr 172.21.15.77:0/2971558594 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:58:26.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.850+0000 7ff7cbe00640 1 --2- 172.21.15.77:0/2971558594 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a8ec092320 0x55a8ec0ae5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:26.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.850+0000 7ff7cbe00640 1 -- 172.21.15.77:0/2971558594 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a8ec092320 msgr2=0x55a8ec0ae5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:26.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.850+0000 7ff7cbe00640 1 --2- 172.21.15.77:0/2971558594 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a8ec092320 0x55a8ec0ae5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:26.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.850+0000 7ff7cbe00640 1 -- 172.21.15.77:0/2971558594 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8ec092c40 msgr2=0x55a8ec0a98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:26.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.850+0000 7ff7cbe00640 1 --2- 172.21.15.77:0/2971558594 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8ec092c40 0x55a8ec0a98d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:26.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.850+0000 7ff7cbe00640 1 -- 172.21.15.77:0/2971558594 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8ec0aa640 con 0x55a8ec0a60d0 2024-09-10T09:58:26.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.850+0000 7ff7d1000640 1 --2- 172.21.15.77:0/2971558594 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8ec092c40 0x55a8ec0a98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:58:26.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.851+0000 7ff7cbe00640 1 --2- 172.21.15.77:0/2971558594 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a8ec0a60d0 0x55a8ec0a9e10 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x55a8ec086a90 tx=0x7ff7b8009a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd141cbcf287d4c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:26.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.851+0000 7ff7b2a00640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T09:58:26.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.851+0000 7ff7b2a00640 1 -- 172.21.15.77:0/2971558594 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b803d070 con 0x55a8ec0a60d0 2024-09-10T09:58:26.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.851+0000 7ff7d64c4740 1 -- 172.21.15.77:0/2971558594 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7c000fd70 con 0x55a8ec0a60d0 2024-09-10T09:58:26.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.851+0000 7ff7b2a00640 1 -- 172.21.15.77:0/2971558594 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7ff7b80140c0 con 0x55a8ec0a60d0 2024-09-10T09:58:26.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.851+0000 7ff7b2a00640 1 -- 172.21.15.77:0/2971558594 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff7b804c050 con 0x55a8ec0a60d0 2024-09-10T09:58:26.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.851+0000 7ff7d64c4740 10 client.26135 fetch_fsmap learned FSMap version 32 2024-09-10T09:58:26.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.851+0000 7ff7d64c4740 1 -- 172.21.15.77:0/2971558594 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a8ec120a60 con 0x55a8ec0a60d0 2024-09-10T09:58:26.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.852+0000 7ff7b2a00640 1 -- 172.21.15.77:0/2971558594 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7c000fd70 con 0x55a8ec0a60d0 2024-09-10T09:58:26.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.852+0000 7ff7b2a00640 1 -- 172.21.15.77:0/2971558594 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7ff7b8043070 con 0x55a8ec0a60d0 2024-09-10T09:58:26.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.852+0000 7ff7d64c4740 10 client.26135 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T09:58:26.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.852+0000 7ff7d64c4740 10 client.26135 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-10T09:58:26.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.852+0000 7ff7d64c4740 20 client.26135 populate_metadata read hostname 'smithi077' 2024-09-10T09:58:26.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.852+0000 7ff7d64c4740 1 --2- 172.21.15.77:0/2971558594 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x55a8ec121130 0x55a8ec123550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:26.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.852+0000 7ff7d64c4740 4 client.26135 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:58:26.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.852+0000 7ff7d64c4740 1 -- 172.21.15.77:0/2971558594 --> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a8ebff7e60 con 0x55a8ec121130 2024-09-10T09:58:26.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.852+0000 7ff7d1000640 1 --2- 172.21.15.77:0/2971558594 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x55a8ec121130 0x55a8ec123550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:26.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.853+0000 7ff7d1000640 1 --2- 172.21.15.77:0/2971558594 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x55a8ec121130 0x55a8ec123550 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e85f2ae7de7a9de8 server_cookie=ff61caeedd20ce2e in_seq=0 out_seq=0 2024-09-10T09:58:26.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.853+0000 7ff7b2a00640 10 client.26135 ms_handle_connect on v2:172.21.15.134:6834/3293933699 2024-09-10T09:58:26.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.853+0000 7ff7b2a00640 1 -- 172.21.15.77:0/2971558594 <== mds.4 v2:172.21.15.134:6834/3293933699 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a8ebff7e60 con 0x55a8ec121130 2024-09-10T09:58:26.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.853+0000 7ff7b2a00640 10 client.26135 handle_command_reply: tid=0 2024-09-10T09:58:26.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.853+0000 7ff7d64c4740 1 client.26135 shutdown 2024-09-10T09:58:26.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.854+0000 7ff7d64c4740 1 -- 172.21.15.77:0/2971558594 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a8ec0a60d0 msgr2=0x55a8ec0a9e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:26.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.854+0000 7ff7d64c4740 1 --2- 172.21.15.77:0/2971558594 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a8ec0a60d0 0x55a8ec0a9e10 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x55a8ec086a90 tx=0x7ff7b8009a90 comp rx=0 tx=0).stop 2024-09-10T09:58:26.860 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.854+0000 7ff7d64c4740 1 -- 172.21.15.77:0/2971558594 shutdown_connections 2024-09-10T09:58:26.860 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.854+0000 7ff7d64c4740 1 --2- 172.21.15.77:0/2971558594 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x55a8ec121130 0x55a8ec123550 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:26.860 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.854+0000 7ff7d64c4740 1 --2- 172.21.15.77:0/2971558594 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a8ec0a60d0 0x55a8ec0a9e10 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:26.860 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.854+0000 7ff7b2a00640 0 client.26135 ms_handle_reset on v2:172.21.15.134:6834/3293933699 2024-09-10T09:58:26.860 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.854+0000 7ff7d64c4740 1 --2- 172.21.15.77:0/2971558594 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8ec092c40 0x55a8ec0a98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:26.860 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.854+0000 7ff7d64c4740 1 --2- 172.21.15.77:0/2971558594 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a8ec092320 0x55a8ec0ae5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:26.860 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.854+0000 7ff7d64c4740 1 -- 172.21.15.77:0/2971558594 >> 172.21.15.77:0/2971558594 conn(0x55a8ec08ea10 msgr2=0x55a8ec08a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:26.860 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.855+0000 7ff7d64c4740 1 -- 172.21.15.77:0/2971558594 shutdown_connections 2024-09-10T09:58:26.860 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.856+0000 7ff7d64c4740 1 -- 172.21.15.77:0/2971558594 wait complete. 2024-09-10T09:58:26.860 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.856+0000 7ff7d64c4740 20 client.26135 trim_cache size 0 max 16384 2024-09-10T09:58:26.860 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-10T09:58:26.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.857+0000 7ff7d3a00640 1 -- 172.21.15.77:0/2653801250 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ff7b40a2c00 msgr2=0x7ff7b40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:26.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.857+0000 7ff7d3a00640 1 --2- 172.21.15.77:0/2653801250 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ff7b40a2c00 0x7ff7b40a50c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff7c40b8850 tx=0x7ff7c4004140 comp rx=0 tx=0).stop 2024-09-10T09:58:26.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.857+0000 7ff7d3a00640 1 -- 172.21.15.77:0/2653801250 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff7cc10c150 msgr2=0x7ff7cc19ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:26.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.857+0000 7ff7d3a00640 1 --2- 172.21.15.77:0/2653801250 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff7cc10c150 0x7ff7cc19ef00 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff7c4005e30 tx=0x7ff7c4003f10 comp rx=0 tx=0).stop 2024-09-10T09:58:26.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.858+0000 7ff7d3a00640 1 -- 172.21.15.77:0/2653801250 shutdown_connections 2024-09-10T09:58:26.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.858+0000 7ff7d3a00640 1 --2- 172.21.15.77:0/2653801250 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ff7b40a2c00 0x7ff7b40a50c0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:26.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.858+0000 7ff7d3a00640 1 --2- 172.21.15.77:0/2653801250 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff7cc10c150 0x7ff7cc19ef00 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:26.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.858+0000 7ff7d3a00640 1 --2- 172.21.15.77:0/2653801250 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff7cc103640 0x7ff7cc19ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:26.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.858+0000 7ff7d3a00640 1 --2- 172.21.15.77:0/2653801250 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7cc102d20 0x7ff7cc19a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:26.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.858+0000 7ff7d3a00640 1 -- 172.21.15.77:0/2653801250 >> 172.21.15.77:0/2653801250 conn(0x7ff7cc0fc780 msgr2=0x7ff7cc1008b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:26.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.858+0000 7ff7d3a00640 1 -- 172.21.15.77:0/2653801250 shutdown_connections 2024-09-10T09:58:26.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:26.859+0000 7ff7d3a00640 1 -- 172.21.15.77:0/2653801250 wait complete. 2024-09-10T09:58:26.936 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] fiemap-tester 2024-09-10T09:58:26.937 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] t_stripealign 2024-09-10T09:58:26.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:58:26 smithi077 ceph-mon[30319]: pgmap v489: 129 pgs: 129 active+clean; 75 MiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 389 KiB/s wr, 70 op/s 2024-09-10T09:58:26.966 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] open_by_handle 2024-09-10T09:58:26.976 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] t_mmap_dio 2024-09-10T09:58:27.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:58:26 smithi134 ceph-mon[35674]: pgmap v489: 129 pgs: 129 active+clean; 75 MiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 389 KiB/s wr, 70 op/s 2024-09-10T09:58:27.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:58:26 smithi120 ceph-mon[34770]: pgmap v489: 129 pgs: 129 active+clean; 75 MiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 389 KiB/s wr, 70 op/s 2024-09-10T09:58:27.245 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] xfsfind 2024-09-10T09:58:27.275 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] t_reflink_read_race 2024-09-10T09:58:27.371 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] fssum 2024-09-10T09:58:27.371 INFO:tasks.workunit.client.0.smithi077.stdout:Building log-writes 2024-09-10T09:58:27.404 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] replay-log 2024-09-10T09:58:27.620 INFO:tasks.workunit.client.0.smithi077.stdout:Building aio-dio-regress 2024-09-10T09:58:27.620 INFO:tasks.workunit.client.0.smithi077.stdout:Building perf 2024-09-10T09:58:27.633 INFO:tasks.workunit.client.0.smithi077.stdout:Building vfs 2024-09-10T09:58:27.637 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] aiocp 2024-09-10T09:58:27.650 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] vfstest 2024-09-10T09:58:27.941 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] mount-idmapped 2024-09-10T09:58:28.016 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T09:58:28.022 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] aio-dio-append-write-fallocate-race 2024-09-10T09:58:28.034 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] aio-dio-append-write-read-race 2024-09-10T09:58:28.356 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] aio-dio-cow-race 2024-09-10T09:58:28.413 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] aio-dio-cycle-write 2024-09-10T09:58:28.508 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:58:28.647 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] aio-dio-eof-race 2024-09-10T09:58:28.675 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] aio-dio-extend-stat 2024-09-10T09:58:28.750 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] aio-dio-fcntl-race 2024-09-10T09:58:28.929 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:58:28 smithi077 ceph-mon[30319]: pgmap v490: 129 pgs: 129 active+clean; 75 MiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 162 KiB/s wr, 49 op/s 2024-09-10T09:58:28.972 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] aio-dio-hole-filling-race 2024-09-10T09:58:28.993 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] aio-dio-invalidate-failure 2024-09-10T09:58:29.042 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] aio-dio-invalidate-readahead 2024-09-10T09:58:29.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:58:28 smithi134 ceph-mon[35674]: pgmap v490: 129 pgs: 129 active+clean; 75 MiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 162 KiB/s wr, 49 op/s 2024-09-10T09:58:29.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:58:28 smithi120 ceph-mon[34770]: pgmap v490: 129 pgs: 129 active+clean; 75 MiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 162 KiB/s wr, 49 op/s 2024-09-10T09:58:29.265 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] aiodio_sparse2 2024-09-10T09:58:29.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.279+0000 7f46c0400640 1 -- 172.21.15.77:0/3993163041 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46b8109140 msgr2=0x7f46b8110cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:29.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.279+0000 7f46c0400640 1 --2- 172.21.15.77:0/3993163041 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46b8109140 0x7f46b8110cc0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f46b0009ca0 tx=0x7f46b002a8b0 comp rx=0 tx=0).stop 2024-09-10T09:58:29.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.279+0000 7f46c0400640 1 -- 172.21.15.77:0/3993163041 shutdown_connections 2024-09-10T09:58:29.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.279+0000 7f46c0400640 1 --2- 172.21.15.77:0/3993163041 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46b8109140 0x7f46b8110cc0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:29.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.279+0000 7f46c0400640 1 --2- 172.21.15.77:0/3993163041 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f46b81049c0 0x7f46b8108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:29.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.279+0000 7f46c0400640 1 --2- 172.21.15.77:0/3993163041 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46b8104010 0x7f46b81043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:29.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.279+0000 7f46c0400640 1 -- 172.21.15.77:0/3993163041 >> 172.21.15.77:0/3993163041 conn(0x7f46b80ffa50 msgr2=0x7f46b8101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:29.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.280+0000 7f46c0400640 1 -- 172.21.15.77:0/3993163041 shutdown_connections 2024-09-10T09:58:29.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.280+0000 7f46c0400640 1 -- 172.21.15.77:0/3993163041 wait complete. 2024-09-10T09:58:29.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.280+0000 7f46c0400640 1 Processor -- start 2024-09-10T09:58:29.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.281+0000 7f46c0400640 1 -- start start 2024-09-10T09:58:29.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.281+0000 7f46c0400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f46b8104010 0x7f46b819a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:29.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.281+0000 7f46c0400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46b81049c0 0x7f46b819ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:29.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.281+0000 7f46c0400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46b8109140 0x7f46b819ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:29.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.281+0000 7f46c0400640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f46b81141c0 con 0x7f46b8109140 2024-09-10T09:58:29.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.281+0000 7f46c0400640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f46b8114040 con 0x7f46b8104010 2024-09-10T09:58:29.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.281+0000 7f46c0400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f46b8114340 con 0x7f46b81049c0 2024-09-10T09:58:29.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.282+0000 7f46bda00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f46b8104010 0x7f46b819a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:29.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.282+0000 7f46bda00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f46b8104010 0x7f46b819a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:48550/0 (socket says 172.21.15.77:48550) 2024-09-10T09:58:29.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.282+0000 7f46bda00640 1 -- 172.21.15.77:0/2372625431 learned_addr learned my addr 172.21.15.77:0/2372625431 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:58:29.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.282+0000 7f46bd000640 1 --2- 172.21.15.77:0/2372625431 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46b81049c0 0x7f46b819ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:29.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.282+0000 7f46be400640 1 --2- 172.21.15.77:0/2372625431 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46b8109140 0x7f46b819ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:29.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.282+0000 7f46bda00640 1 -- 172.21.15.77:0/2372625431 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46b81049c0 msgr2=0x7f46b819ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:29.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.282+0000 7f46bda00640 1 --2- 172.21.15.77:0/2372625431 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46b81049c0 0x7f46b819ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:29.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.282+0000 7f46bda00640 1 -- 172.21.15.77:0/2372625431 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46b8109140 msgr2=0x7f46b819ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:29.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.282+0000 7f46bda00640 1 --2- 172.21.15.77:0/2372625431 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46b8109140 0x7f46b819ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:29.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.282+0000 7f46bda00640 1 -- 172.21.15.77:0/2372625431 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46b819f670 con 0x7f46b8104010 2024-09-10T09:58:29.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.283+0000 7f46bda00640 1 --2- 172.21.15.77:0/2372625431 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f46b8104010 0x7f46b819a6c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f46ac00e9f0 tx=0x7f46ac00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:29.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.283+0000 7f46b6000640 1 -- 172.21.15.77:0/2372625431 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ac00cd80 con 0x7f46b8104010 2024-09-10T09:58:29.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.283+0000 7f46b6000640 1 -- 172.21.15.77:0/2372625431 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46ac010910 con 0x7f46b8104010 2024-09-10T09:58:29.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.283+0000 7f46c0400640 1 -- 172.21.15.77:0/2372625431 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46b819f960 con 0x7f46b8104010 2024-09-10T09:58:29.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.283+0000 7f46b6000640 1 -- 172.21.15.77:0/2372625431 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ac018990 con 0x7f46b8104010 2024-09-10T09:58:29.285 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] aio-dio-subblock-eof-read 2024-09-10T09:58:29.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.284+0000 7f46c0400640 1 -- 172.21.15.77:0/2372625431 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46b81a7240 con 0x7f46b8104010 2024-09-10T09:58:29.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.287+0000 7f46b6000640 1 -- 172.21.15.77:0/2372625431 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f46ac010ac0 con 0x7f46b8104010 2024-09-10T09:58:29.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.288+0000 7f46c2fe3740 1 Processor -- start 2024-09-10T09:58:29.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.288+0000 7f46b6000640 1 --2- 172.21.15.77:0/2372625431 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f46a00a2b80 0x7f46a00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:29.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.289+0000 7f46b6000640 1 -- 172.21.15.77:0/2372625431 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f46ac0cf850 con 0x7f46b8104010 2024-09-10T09:58:29.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.289+0000 7f46c2fe3740 1 -- start start 2024-09-10T09:58:29.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.289+0000 7f46c2fe3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a804d20480 0x55a804d20860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:29.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.289+0000 7f46c2fe3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a804d20da0 0x55a804d2b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:29.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.289+0000 7f46c2fe3740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a804d2b660 0x55a804d2da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:29.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.289+0000 7f46c2fe3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55a804c7e3e0 con 0x55a804d20da0 2024-09-10T09:58:29.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.289+0000 7f46c2fe3740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a804c25d80 con 0x55a804d2b660 2024-09-10T09:58:29.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.290+0000 7f46c2fe3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a804c7a3a0 con 0x55a804d20480 2024-09-10T09:58:29.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.290+0000 7f46be400640 1 --2- 172.21.15.77:0/2372625431 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f46a00a2b80 0x7f46a00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:29.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.290+0000 7f46bda00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a804d20480 0x55a804d20860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:29.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.290+0000 7f46bda00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a804d20480 0x55a804d20860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:42392/0 (socket says 172.21.15.77:42392) 2024-09-10T09:58:29.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.290+0000 7f46bda00640 1 -- 172.21.15.77:0/1549314715 learned_addr learned my addr 172.21.15.77:0/1549314715 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:58:29.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.290+0000 7f46bd000640 1 --2- 172.21.15.77:0/1549314715 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a804d20da0 0x55a804d2b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:29.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.290+0000 7f46be400640 1 --2- 172.21.15.77:0/1549314715 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a804d2b660 0x55a804d2da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:29.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.290+0000 7f46bda00640 1 -- 172.21.15.77:0/1549314715 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a804d2b660 msgr2=0x55a804d2da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:29.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.290+0000 7f46bda00640 1 --2- 172.21.15.77:0/1549314715 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a804d2b660 0x55a804d2da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:29.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.290+0000 7f46bda00640 1 -- 172.21.15.77:0/1549314715 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a804d20da0 msgr2=0x55a804d2b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:29.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.291+0000 7f46bda00640 1 --2- 172.21.15.77:0/1549314715 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a804d20da0 0x55a804d2b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:29.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.291+0000 7f46bda00640 1 -- 172.21.15.77:0/1549314715 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a804c44ae0 con 0x55a804d20480 2024-09-10T09:58:29.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.291+0000 7f46be400640 1 --2- 172.21.15.77:0/1549314715 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a804d2b660 0x55a804d2da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:58:29.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.291+0000 7f46bd000640 1 --2- 172.21.15.77:0/1549314715 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a804d20da0 0x55a804d2b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:58:29.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.291+0000 7f46bda00640 1 --2- 172.21.15.77:0/1549314715 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a804d20480 0x55a804d20860 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f46ac0a43a0 tx=0x7f46ac09c420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9567788d229b648d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:29.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.291+0000 7f46aaa00640 1 -- 172.21.15.77:0/1549314715 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ac0a48d0 con 0x55a804d20480 2024-09-10T09:58:29.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.291+0000 7f46aaa00640 1 -- 172.21.15.77:0/1549314715 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46ac0b1030 con 0x55a804d20480 2024-09-10T09:58:29.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.291+0000 7f46aaa00640 1 -- 172.21.15.77:0/1549314715 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ac09b2b0 con 0x55a804d20480 2024-09-10T09:58:29.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.292+0000 7f46c2fe3740 1 -- 172.21.15.77:0/1549314715 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a804d20480 msgr2=0x55a804d20860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:29.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.292+0000 7f46c2fe3740 1 --2- 172.21.15.77:0/1549314715 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a804d20480 0x55a804d20860 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f46ac0a43a0 tx=0x7f46ac09c420 comp rx=0 tx=0).stop 2024-09-10T09:58:29.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.292+0000 7f46c2fe3740 1 -- 172.21.15.77:0/1549314715 shutdown_connections 2024-09-10T09:58:29.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.292+0000 7f46c2fe3740 1 --2- 172.21.15.77:0/1549314715 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a804d2b660 0x55a804d2da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:29.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.292+0000 7f46c2fe3740 1 --2- 172.21.15.77:0/1549314715 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a804d20da0 0x55a804d2b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:29.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.292+0000 7f46c2fe3740 1 --2- 172.21.15.77:0/1549314715 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a804d20480 0x55a804d20860 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:29.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.292+0000 7f46c2fe3740 1 -- 172.21.15.77:0/1549314715 >> 172.21.15.77:0/1549314715 conn(0x55a804d1ca10 msgr2=0x55a804d1cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:29.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.293+0000 7f46be400640 1 --2- 172.21.15.77:0/2372625431 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f46a00a2b80 0x7f46a00a5040 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x55a804d1dd90 tx=0x7f46b00303a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:29.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.293+0000 7f46c2fe3740 1 -- 172.21.15.77:0/1549314715 shutdown_connections 2024-09-10T09:58:29.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.293+0000 7f46c2fe3740 1 -- 172.21.15.77:0/1549314715 wait complete. 2024-09-10T09:58:29.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.293+0000 7f46c2fe3740 1 -- start start 2024-09-10T09:58:29.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.294+0000 7f46c2fe3740 1 Processor -- start 2024-09-10T09:58:29.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.294+0000 7f46c2fe3740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a804d20480 0x55a804d19930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:29.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.294+0000 7f46c2fe3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a804d20da0 0x55a804d19e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:29.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.294+0000 7f46c2fe3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a804d2b660 0x55a804d2a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:29.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.294+0000 7f46c2fe3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55a804d13e70 con 0x55a804d20da0 2024-09-10T09:58:29.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.294+0000 7f46c2fe3740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a804c390f0 con 0x55a804d20480 2024-09-10T09:58:29.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.295+0000 7f46c2fe3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a804d13ff0 con 0x55a804d2b660 2024-09-10T09:58:29.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.295+0000 7f46c2fe3740 10 client.0 _ll_register_callbacks cb 0x55a804c20bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:58:29.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.295+0000 7f46bda00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a804d20480 0x55a804d19930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:29.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.295+0000 7f46bda00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a804d20480 0x55a804d19930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:48568/0 (socket says 172.21.15.77:48568) 2024-09-10T09:58:29.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.295+0000 7f46bda00640 1 -- 172.21.15.77:0/1355179159 learned_addr learned my addr 172.21.15.77:0/1355179159 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:58:29.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.295+0000 7f46bda00640 1 -- 172.21.15.77:0/1355179159 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a804d2b660 msgr2=0x55a804d2a110 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T09:58:29.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.295+0000 7f46bda00640 1 --2- 172.21.15.77:0/1355179159 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a804d2b660 0x55a804d2a110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:29.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.295+0000 7f46bda00640 1 -- 172.21.15.77:0/1355179159 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a804d20da0 msgr2=0x55a804d19e70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T09:58:29.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.296+0000 7f46bd000640 1 --2- 172.21.15.77:0/1355179159 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a804d20da0 0x55a804d19e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:29.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.297+0000 7f46bda00640 1 --2- 172.21.15.77:0/1355179159 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a804d20da0 0x55a804d19e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:29.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.297+0000 7f46bda00640 1 -- 172.21.15.77:0/1355179159 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8049ef3c0 con 0x55a804d20480 2024-09-10T09:58:29.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.297+0000 7f46bd000640 1 --2- 172.21.15.77:0/1355179159 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a804d20da0 0x55a804d19e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:58:29.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.298+0000 7f46bda00640 1 --2- 172.21.15.77:0/1355179159 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a804d20480 0x55a804d19930 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f46ac09bbf0 tx=0x7f46ac099d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=903542c97094ca36 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:29.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.298+0000 7f469f400640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T09:58:29.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.298+0000 7f469f400640 1 -- 172.21.15.77:0/1355179159 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ac0b4e60 con 0x55a804d20480 2024-09-10T09:58:29.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.298+0000 7f469f400640 1 -- 172.21.15.77:0/1355179159 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f46ac09b2b0 con 0x55a804d20480 2024-09-10T09:58:29.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.298+0000 7f46c2fe3740 1 -- 172.21.15.77:0/1355179159 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46ac0b1030 con 0x55a804d20480 2024-09-10T09:58:29.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.298+0000 7f469f400640 1 -- 172.21.15.77:0/1355179159 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f46ac018bb0 con 0x55a804d20480 2024-09-10T09:58:29.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.299+0000 7f46c2fe3740 10 client.26144 fetch_fsmap learned FSMap version 32 2024-09-10T09:58:29.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.299+0000 7f46c2fe3740 1 -- 172.21.15.77:0/1355179159 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a804d42970 con 0x55a804d20480 2024-09-10T09:58:29.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.299+0000 7f469f400640 1 -- 172.21.15.77:0/1355179159 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46ac0b1030 con 0x55a804d20480 2024-09-10T09:58:29.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.301+0000 7f469f400640 1 -- 172.21.15.77:0/1355179159 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f46ac0ac030 con 0x55a804d20480 2024-09-10T09:58:29.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.301+0000 7f46c2fe3740 10 client.26144 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T09:58:29.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.301+0000 7f46c2fe3740 10 client.26144 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T09:58:29.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.301+0000 7f46c2fe3740 20 client.26144 populate_metadata read hostname 'smithi077' 2024-09-10T09:58:29.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.301+0000 7f46c2fe3740 1 --2- 172.21.15.77:0/1355179159 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x55a804d43cc0 0x55a804d460b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:29.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.301+0000 7f46c2fe3740 4 client.26144 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:58:29.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.301+0000 7f46c2fe3740 1 -- 172.21.15.77:0/1355179159 --> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a804c8b950 con 0x55a804d43cc0 2024-09-10T09:58:29.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.301+0000 7f46bd000640 1 --2- 172.21.15.77:0/1355179159 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x55a804d43cc0 0x55a804d460b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:29.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.302+0000 7f46bd000640 1 --2- 172.21.15.77:0/1355179159 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x55a804d43cc0 0x55a804d460b0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8a1e7f2987e557b7 server_cookie=68da161568591683 in_seq=0 out_seq=0 2024-09-10T09:58:29.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.303+0000 7f469f400640 10 client.26144 ms_handle_connect on v2:172.21.15.134:6836/2461497091 2024-09-10T09:58:29.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.303+0000 7f469f400640 1 -- 172.21.15.77:0/1355179159 <== mds.3 v2:172.21.15.134:6836/2461497091 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a804c8b950 con 0x55a804d43cc0 2024-09-10T09:58:29.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.303+0000 7f469f400640 10 client.26144 handle_command_reply: tid=0 2024-09-10T09:58:29.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.303+0000 7f46c2fe3740 1 client.26144 shutdown 2024-09-10T09:58:29.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.304+0000 7f46c2fe3740 1 -- 172.21.15.77:0/1355179159 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a804d20480 msgr2=0x55a804d19930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:29.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.304+0000 7f46c2fe3740 1 --2- 172.21.15.77:0/1355179159 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a804d20480 0x55a804d19930 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f46ac09bbf0 tx=0x7f46ac099d00 comp rx=0 tx=0).stop 2024-09-10T09:58:29.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.304+0000 7f46c2fe3740 1 -- 172.21.15.77:0/1355179159 shutdown_connections 2024-09-10T09:58:29.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.304+0000 7f46c2fe3740 1 --2- 172.21.15.77:0/1355179159 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a804d2b660 0x55a804d2a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:29.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.304+0000 7f46c2fe3740 1 --2- 172.21.15.77:0/1355179159 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x55a804d43cc0 0x55a804d460b0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:29.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.304+0000 7f46c2fe3740 1 --2- 172.21.15.77:0/1355179159 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a804d20da0 0x55a804d19e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:29.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.304+0000 7f46c2fe3740 1 --2- 172.21.15.77:0/1355179159 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a804d20480 0x55a804d19930 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:29.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.304+0000 7f469f400640 0 client.26144 ms_handle_reset on v2:172.21.15.134:6836/2461497091 2024-09-10T09:58:29.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.304+0000 7f46c2fe3740 1 -- 172.21.15.77:0/1355179159 >> 172.21.15.77:0/1355179159 conn(0x55a804d1ca10 msgr2=0x55a804d13380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:29.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.304+0000 7f46c2fe3740 1 -- 172.21.15.77:0/1355179159 shutdown_connections 2024-09-10T09:58:29.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.304+0000 7f46c2fe3740 1 -- 172.21.15.77:0/1355179159 wait complete. 2024-09-10T09:58:29.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.305+0000 7f46c2fe3740 20 client.26144 trim_cache size 0 max 16384 2024-09-10T09:58:29.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.333+0000 7f46c2fe3740 1 Processor -- start 2024-09-10T09:58:29.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.333+0000 7f46c2fe3740 1 -- start start 2024-09-10T09:58:29.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.333+0000 7f46c2fe3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a804d20480 0x55a804d1cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:29.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.333+0000 7f46c2fe3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a804d20da0 0x55a804d1b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:29.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.333+0000 7f46c2fe3740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a804d2b660 0x55a804d4b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:29.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.333+0000 7f46c2fe3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f46b81acd20 con 0x55a804d20da0 2024-09-10T09:58:29.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.333+0000 7f46c2fe3740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f46b81ae920 con 0x55a804d2b660 2024-09-10T09:58:29.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.333+0000 7f46c2fe3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f46b81ad3f0 con 0x55a804d20480 2024-09-10T09:58:29.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.334+0000 7f46bda00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a804d20480 0x55a804d1cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:29.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.334+0000 7f46bda00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a804d20480 0x55a804d1cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:42400/0 (socket says 172.21.15.77:42400) 2024-09-10T09:58:29.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.334+0000 7f46bda00640 1 -- 172.21.15.77:0/2722453646 learned_addr learned my addr 172.21.15.77:0/2722453646 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:58:29.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.334+0000 7f46bd000640 1 --2- 172.21.15.77:0/2722453646 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a804d20da0 0x55a804d1b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:29.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.334+0000 7f46bda00640 1 -- 172.21.15.77:0/2722453646 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a804d2b660 msgr2=0x55a804d4b710 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T09:58:29.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.334+0000 7f46bda00640 1 --2- 172.21.15.77:0/2722453646 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a804d2b660 0x55a804d4b710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:29.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.334+0000 7f46bda00640 1 -- 172.21.15.77:0/2722453646 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a804d20da0 msgr2=0x55a804d1b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:29.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.334+0000 7f46bda00640 1 --2- 172.21.15.77:0/2722453646 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a804d20da0 0x55a804d1b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:29.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.334+0000 7f46bda00640 1 -- 172.21.15.77:0/2722453646 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a804d1c840 con 0x55a804d20480 2024-09-10T09:58:29.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.335+0000 7f46bda00640 1 --2- 172.21.15.77:0/2722453646 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a804d20480 0x55a804d1cdf0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f46ac0a43a0 tx=0x7f46ac000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60e021bd320f233f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:29.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.335+0000 7f46aa000640 1 -- 172.21.15.77:0/2722453646 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ac0b4e60 con 0x55a804d20480 2024-09-10T09:58:29.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.335+0000 7f46aa000640 1 -- 172.21.15.77:0/2722453646 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46ac0b1070 con 0x55a804d20480 2024-09-10T09:58:29.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.335+0000 7f46aa000640 1 -- 172.21.15.77:0/2722453646 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ac097c10 con 0x55a804d20480 2024-09-10T09:58:29.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.336+0000 7f46c2fe3740 1 -- 172.21.15.77:0/2722453646 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a804d20480 msgr2=0x55a804d1cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:29.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.336+0000 7f46c2fe3740 1 --2- 172.21.15.77:0/2722453646 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a804d20480 0x55a804d1cdf0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f46ac0a43a0 tx=0x7f46ac000bc0 comp rx=0 tx=0).stop 2024-09-10T09:58:29.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.336+0000 7f46c2fe3740 1 -- 172.21.15.77:0/2722453646 shutdown_connections 2024-09-10T09:58:29.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.336+0000 7f46c2fe3740 1 --2- 172.21.15.77:0/2722453646 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a804d2b660 0x55a804d4b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:29.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.336+0000 7f46c2fe3740 1 --2- 172.21.15.77:0/2722453646 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a804d20da0 0x55a804d1b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:29.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.336+0000 7f46c2fe3740 1 --2- 172.21.15.77:0/2722453646 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a804d20480 0x55a804d1cdf0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:29.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.337+0000 7f46c2fe3740 1 -- 172.21.15.77:0/2722453646 >> 172.21.15.77:0/2722453646 conn(0x55a804d1ca10 msgr2=0x55a804d1afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:29.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.337+0000 7f46c2fe3740 1 -- 172.21.15.77:0/2722453646 shutdown_connections 2024-09-10T09:58:29.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.337+0000 7f46c2fe3740 1 -- 172.21.15.77:0/2722453646 wait complete. 2024-09-10T09:58:29.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.337+0000 7f46c2fe3740 1 -- start start 2024-09-10T09:58:29.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.338+0000 7f46c2fe3740 1 Processor -- start 2024-09-10T09:58:29.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.338+0000 7f46c2fe3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a804d20480 0x55a804d24dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:29.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.338+0000 7f46c2fe3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a804d20da0 0x55a804d25310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:29.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.338+0000 7f46c2fe3740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a804d2b660 0x55a804d25850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:29.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.339+0000 7f46c2fe3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f46b81ae190 con 0x55a804d20da0 2024-09-10T09:58:29.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.339+0000 7f46c2fe3740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f46b81adac0 con 0x55a804d2b660 2024-09-10T09:58:29.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.339+0000 7f46c2fe3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f46b81aad00 con 0x55a804d20480 2024-09-10T09:58:29.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.339+0000 7f46c2fe3740 10 client.0 _ll_register_callbacks cb 0x7f46ac09b5c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:58:29.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.339+0000 7f46bda00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a804d20480 0x55a804d24dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:29.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.339+0000 7f46bda00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a804d20480 0x55a804d24dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:42414/0 (socket says 172.21.15.77:42414) 2024-09-10T09:58:29.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.339+0000 7f46bda00640 1 -- 172.21.15.77:0/2174956621 learned_addr learned my addr 172.21.15.77:0/2174956621 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:58:29.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.339+0000 7f46be400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a804d2b660 0x55a804d25850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:29.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.339+0000 7f46bda00640 1 -- 172.21.15.77:0/2174956621 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a804d2b660 msgr2=0x55a804d25850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:29.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.339+0000 7f46bda00640 1 --2- 172.21.15.77:0/2174956621 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a804d2b660 0x55a804d25850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:29.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.339+0000 7f46bda00640 1 -- 172.21.15.77:0/2174956621 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a804d20da0 msgr2=0x55a804d25310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:29.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.339+0000 7f46bd000640 1 --2- 172.21.15.77:0/2174956621 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a804d20da0 0x55a804d25310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:29.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.340+0000 7f46bda00640 1 --2- 172.21.15.77:0/2174956621 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a804d20da0 0x55a804d25310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:29.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.340+0000 7f46bda00640 1 -- 172.21.15.77:0/2174956621 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a804d0f770 con 0x55a804d20480 2024-09-10T09:58:29.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.340+0000 7f46be400640 1 --2- 172.21.15.77:0/2174956621 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a804d2b660 0x55a804d25850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:58:29.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.340+0000 7f46bd000640 1 --2- 172.21.15.77:0/2174956621 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a804d20da0 0x55a804d25310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:58:29.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.340+0000 7f46bda00640 1 --2- 172.21.15.77:0/2174956621 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a804d20480 0x55a804d24dd0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f46ac0985e0 tx=0x7f46ac010660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1aba66191c4afb5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:29.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.340+0000 7f469f400640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-10T09:58:29.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.340+0000 7f469f400640 1 -- 172.21.15.77:0/2174956621 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ac0b4e60 con 0x55a804d20480 2024-09-10T09:58:29.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.340+0000 7f469f400640 1 -- 172.21.15.77:0/2174956621 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f46ac0979b0 con 0x55a804d20480 2024-09-10T09:58:29.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.340+0000 7f46c2fe3740 1 -- 172.21.15.77:0/2174956621 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46ac0b1070 con 0x55a804d20480 2024-09-10T09:58:29.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.341+0000 7f469f400640 1 -- 172.21.15.77:0/2174956621 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f46ac097bb0 con 0x55a804d20480 2024-09-10T09:58:29.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.341+0000 7f46c2fe3740 10 client.26092 fetch_fsmap learned FSMap version 32 2024-09-10T09:58:29.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.341+0000 7f46c2fe3740 1 -- 172.21.15.77:0/2174956621 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a804d1a6f0 con 0x55a804d20480 2024-09-10T09:58:29.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.342+0000 7f469f400640 1 -- 172.21.15.77:0/2174956621 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46ac0b1070 con 0x55a804d20480 2024-09-10T09:58:29.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.342+0000 7f469f400640 1 -- 172.21.15.77:0/2174956621 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f46ac0ac030 con 0x55a804d20480 2024-09-10T09:58:29.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.342+0000 7f46c2fe3740 10 client.26092 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T09:58:29.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.342+0000 7f46c2fe3740 10 client.26092 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T09:58:29.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.342+0000 7f46c2fe3740 20 client.26092 populate_metadata read hostname 'smithi077' 2024-09-10T09:58:29.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.342+0000 7f46c2fe3740 1 --2- 172.21.15.77:0/2174956621 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x55a804daea50 0x55a804db0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:29.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.342+0000 7f46c2fe3740 4 client.26092 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:58:29.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.342+0000 7f46c2fe3740 1 -- 172.21.15.77:0/2174956621 --> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a804d1d5c0 con 0x55a804daea50 2024-09-10T09:58:29.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.343+0000 7f46bd000640 1 --2- 172.21.15.77:0/2174956621 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x55a804daea50 0x55a804db0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:29.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.343+0000 7f46bd000640 1 --2- 172.21.15.77:0/2174956621 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x55a804daea50 0x55a804db0e40 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=256bbe7a739e6898 server_cookie=8216e048cf643f70 in_seq=0 out_seq=0 2024-09-10T09:58:29.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.343+0000 7f469f400640 10 client.26092 ms_handle_connect on v2:172.21.15.134:6836/2461497091 2024-09-10T09:58:29.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.344+0000 7f469f400640 1 -- 172.21.15.77:0/2174956621 <== mds.3 v2:172.21.15.134:6836/2461497091 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a804d1d5c0 con 0x55a804daea50 2024-09-10T09:58:29.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.344+0000 7f469f400640 10 client.26092 handle_command_reply: tid=0 2024-09-10T09:58:29.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.344+0000 7f46c2fe3740 1 client.26092 shutdown 2024-09-10T09:58:29.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.344+0000 7f46c2fe3740 1 -- 172.21.15.77:0/2174956621 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a804d20480 msgr2=0x55a804d24dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:29.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.344+0000 7f46c2fe3740 1 --2- 172.21.15.77:0/2174956621 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a804d20480 0x55a804d24dd0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f46ac0985e0 tx=0x7f46ac010660 comp rx=0 tx=0).stop 2024-09-10T09:58:29.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.344+0000 7f46c2fe3740 1 -- 172.21.15.77:0/2174956621 shutdown_connections 2024-09-10T09:58:29.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.344+0000 7f46c2fe3740 1 --2- 172.21.15.77:0/2174956621 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a804d2b660 0x55a804d25850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:29.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.344+0000 7f46c2fe3740 1 --2- 172.21.15.77:0/2174956621 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x55a804daea50 0x55a804db0e40 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:29.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.344+0000 7f46c2fe3740 1 --2- 172.21.15.77:0/2174956621 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a804d20da0 0x55a804d25310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:29.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.344+0000 7f46c2fe3740 1 --2- 172.21.15.77:0/2174956621 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a804d20480 0x55a804d24dd0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:29.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.344+0000 7f46c2fe3740 1 -- 172.21.15.77:0/2174956621 >> 172.21.15.77:0/2174956621 conn(0x55a804d1ca10 msgr2=0x55a804d18710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:29.347 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-10T09:58:29.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.344+0000 7f469f400640 0 client.26092 ms_handle_reset on v2:172.21.15.134:6836/2461497091 2024-09-10T09:58:29.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.344+0000 7f46c2fe3740 1 -- 172.21.15.77:0/2174956621 shutdown_connections 2024-09-10T09:58:29.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.344+0000 7f46c2fe3740 1 -- 172.21.15.77:0/2174956621 wait complete. 2024-09-10T09:58:29.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.345+0000 7f46c2fe3740 20 client.26092 trim_cache size 0 max 16384 2024-09-10T09:58:29.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.346+0000 7f46c0400640 1 -- 172.21.15.77:0/2372625431 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f46a00a2b80 msgr2=0x7f46a00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:29.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.346+0000 7f46c0400640 1 --2- 172.21.15.77:0/2372625431 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f46a00a2b80 0x7f46a00a5040 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x55a804d1dd90 tx=0x7f46b00303a0 comp rx=0 tx=0).stop 2024-09-10T09:58:29.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.346+0000 7f46c0400640 1 -- 172.21.15.77:0/2372625431 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f46b8104010 msgr2=0x7f46b819a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:29.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.346+0000 7f46c0400640 1 --2- 172.21.15.77:0/2372625431 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f46b8104010 0x7f46b819a6c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f46ac00e9f0 tx=0x7f46ac00eeb0 comp rx=0 tx=0).stop 2024-09-10T09:58:29.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.346+0000 7f46c0400640 1 -- 172.21.15.77:0/2372625431 shutdown_connections 2024-09-10T09:58:29.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.346+0000 7f46c0400640 1 --2- 172.21.15.77:0/2372625431 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f46a00a2b80 0x7f46a00a5040 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:29.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.346+0000 7f46c0400640 1 --2- 172.21.15.77:0/2372625431 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46b8109140 0x7f46b819ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:29.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.346+0000 7f46c0400640 1 --2- 172.21.15.77:0/2372625431 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46b81049c0 0x7f46b819ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:29.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.346+0000 7f46c0400640 1 --2- 172.21.15.77:0/2372625431 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f46b8104010 0x7f46b819a6c0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:29.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.346+0000 7f46c0400640 1 -- 172.21.15.77:0/2372625431 >> 172.21.15.77:0/2372625431 conn(0x7f46b80ffa50 msgr2=0x7f46b8100db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:29.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.347+0000 7f46c0400640 1 -- 172.21.15.77:0/2372625431 shutdown_connections 2024-09-10T09:58:29.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:29.347+0000 7f46c0400640 1 -- 172.21.15.77:0/2372625431 wait complete. 2024-09-10T09:58:29.357 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] aio-dio-write-verify 2024-09-10T09:58:29.581 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] aio-free-ring-with-bogus-nr-pages 2024-09-10T09:58:29.642 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] aio-io-setup-with-nonwritable-context-pointer 2024-09-10T09:58:29.706 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] aio-last-ref-held-by-io 2024-09-10T09:58:30.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:58:29 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:58:30.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:58:29 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:58:30.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:58:29 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:58:30.333 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T09:58:30.455 INFO:tasks.workunit.client.0.smithi077.stderr:+ cd .. 2024-09-10T09:58:30.455 INFO:tasks.workunit.client.0.smithi077.stderr:+ cp xfstests-dev/ltp/fsx . 2024-09-10T09:58:30.460 INFO:tasks.workunit.client.0.smithi077.stderr:+ OPTIONS=-z 2024-09-10T09:58:30.460 INFO:tasks.workunit.client.0.smithi077.stderr:+ ./fsx -z 1MB -N 50000 -p 10000 -l 1048576 2024-09-10T09:58:30.461 INFO:tasks.workunit.client.0.smithi077.stdout:Seed set to 1 2024-09-10T09:58:30.486 INFO:tasks.workunit.client.0.smithi077.stderr:main: filesystem does not support fallocate mode 0, disabling! 2024-09-10T09:58:30.486 INFO:tasks.workunit.client.0.smithi077.stderr:main: filesystem does not support fallocate mode FALLOC_FL_KEEP_SIZE, disabling! 2024-09-10T09:58:30.488 INFO:tasks.workunit.client.0.smithi077.stderr:main: filesystem does not support fallocate mode FALLOC_FL_COLLAPSE_RANGE, disabling! 2024-09-10T09:58:30.489 INFO:tasks.workunit.client.0.smithi077.stderr:main: filesystem does not support fallocate mode FALLOC_FL_INSERT_RANGE, disabling! 2024-09-10T09:58:30.489 INFO:tasks.workunit.client.0.smithi077.stderr:main: filesystem does not support clone range, disabling! 2024-09-10T09:58:30.491 INFO:tasks.workunit.client.0.smithi077.stderr:main: filesystem does not support dedupe range, disabling! 2024-09-10T09:58:30.493 INFO:tasks.workunit.client.0.smithi077.stderr:main: filesystem does not support exchange range, disabling! 2024-09-10T09:58:30.494 INFO:tasks.workunit.client.0.smithi077.stdout:skipping zero size read 2024-09-10T09:58:30.494 INFO:tasks.workunit.client.0.smithi077.stdout:skipping zero size read 2024-09-10T09:58:30.494 INFO:tasks.workunit.client.0.smithi077.stdout:truncating to largest ever: 0x81d54 2024-09-10T09:58:30.499 INFO:tasks.workunit.client.0.smithi077.stdout:copying to largest ever: 0xb86d7 2024-09-10T09:58:30.508 INFO:tasks.workunit.client.0.smithi077.stdout:copying to largest ever: 0xe627e 2024-09-10T09:58:30.551 INFO:tasks.workunit.client.0.smithi077.stdout:truncating to largest ever: 0xf3194 2024-09-10T09:58:30.582 INFO:tasks.workunit.client.0.smithi077.stdout:copying to largest ever: 0xf3ceb 2024-09-10T09:58:30.589 INFO:tasks.workunit.client.0.smithi077.stdout:truncating to largest ever: 0xf47e4 2024-09-10T09:58:30.725 INFO:tasks.workunit.client.0.smithi077.stdout:copying to largest ever: 0xfaccc 2024-09-10T09:58:30.783 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:58:31.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:58:30 smithi134 ceph-mon[35674]: pgmap v491: 129 pgs: 129 active+clean; 75 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 127 KiB/s wr, 57 op/s 2024-09-10T09:58:31.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:58:30 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:58:31.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:58:30 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:58:31.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:58:30 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:58:31.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:58:30 smithi120 ceph-mon[34770]: pgmap v491: 129 pgs: 129 active+clean; 75 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 127 KiB/s wr, 57 op/s 2024-09-10T09:58:31.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:58:30 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:58:31.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:58:30 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:58:31.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:58:30 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:58:31.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:58:30 smithi077 ceph-mon[30319]: pgmap v491: 129 pgs: 129 active+clean; 75 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 127 KiB/s wr, 57 op/s 2024-09-10T09:58:31.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:58:30 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:58:31.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:58:30 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:58:31.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:58:30 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:58:31.292 INFO:tasks.workunit.client.0.smithi077.stdout:copying to largest ever: 0xfe423 2024-09-10T09:58:31.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.419+0000 7f68b9c00640 1 -- 172.21.15.77:0/4052404187 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f68b4076720 msgr2=0x7f68b4076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:31.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.419+0000 7f68b9c00640 1 --2- 172.21.15.77:0/4052404187 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f68b4076720 0x7f68b4076b00 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f68a4009ca0 tx=0x7f68a402a8c0 comp rx=0 tx=0).stop 2024-09-10T09:58:31.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.419+0000 7f68b9c00640 1 -- 172.21.15.77:0/4052404187 shutdown_connections 2024-09-10T09:58:31.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.420+0000 7f68b9c00640 1 --2- 172.21.15.77:0/4052404187 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f68b4112a30 0x7f68b4114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:31.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.420+0000 7f68b9c00640 1 --2- 172.21.15.77:0/4052404187 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68b4077040 0x7f68b41124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:31.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.420+0000 7f68b9c00640 1 --2- 172.21.15.77:0/4052404187 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f68b4076720 0x7f68b4076b00 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:31.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.420+0000 7f68b9c00640 1 -- 172.21.15.77:0/4052404187 >> 172.21.15.77:0/4052404187 conn(0x7f68b40ffa90 msgr2=0x7f68b4101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:31.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.420+0000 7f68b9c00640 1 -- 172.21.15.77:0/4052404187 shutdown_connections 2024-09-10T09:58:31.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.420+0000 7f68b9c00640 1 -- 172.21.15.77:0/4052404187 wait complete. 2024-09-10T09:58:31.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.421+0000 7f68b9c00640 1 Processor -- start 2024-09-10T09:58:31.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.421+0000 7f68b9c00640 1 -- start start 2024-09-10T09:58:31.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.422+0000 7f68b9c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68b4076720 0x7f68b41a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:31.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.422+0000 7f68b9c00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f68b4077040 0x7f68b41a4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:31.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.422+0000 7f68b9c00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f68b4112a30 0x7f68b419e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:31.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.422+0000 7f68b9c00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f68b41185c0 con 0x7f68b4077040 2024-09-10T09:58:31.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.422+0000 7f68b9c00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f68b4118440 con 0x7f68b4112a30 2024-09-10T09:58:31.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.422+0000 7f68b9c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f68b4118740 con 0x7f68b4076720 2024-09-10T09:58:31.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.422+0000 7f68aa000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f68b4077040 0x7f68b41a4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:31.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.422+0000 7f68aa000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f68b4077040 0x7f68b41a4c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33268/0 (socket says 172.21.15.77:33268) 2024-09-10T09:58:31.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.422+0000 7f68aa000640 1 -- 172.21.15.77:0/3423065390 learned_addr learned my addr 172.21.15.77:0/3423065390 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:58:31.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.422+0000 7f68b2a00640 1 --2- 172.21.15.77:0/3423065390 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68b4076720 0x7f68b41a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:31.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.423+0000 7f68aa000640 1 -- 172.21.15.77:0/3423065390 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68b4076720 msgr2=0x7f68b41a46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:31.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.423+0000 7f68aa000640 1 --2- 172.21.15.77:0/3423065390 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68b4076720 0x7f68b41a46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:31.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.423+0000 7f68aa000640 1 -- 172.21.15.77:0/3423065390 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f68b4112a30 msgr2=0x7f68b419e8b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T09:58:31.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.423+0000 7f68aa000640 1 --2- 172.21.15.77:0/3423065390 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f68b4112a30 0x7f68b419e8b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:31.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.423+0000 7f68aa000640 1 -- 172.21.15.77:0/3423065390 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68b419efe0 con 0x7f68b4077040 2024-09-10T09:58:31.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.423+0000 7f68aa000640 1 --2- 172.21.15.77:0/3423065390 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f68b4077040 0x7f68b41a4c30 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f689c00e9f0 tx=0x7f689c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:31.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.423+0000 7f68abe00640 1 -- 172.21.15.77:0/3423065390 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f689c00cd80 con 0x7f68b4077040 2024-09-10T09:58:31.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.423+0000 7f68abe00640 1 -- 172.21.15.77:0/3423065390 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f689c005500 con 0x7f68b4077040 2024-09-10T09:58:31.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.423+0000 7f68abe00640 1 -- 172.21.15.77:0/3423065390 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f689c010690 con 0x7f68b4077040 2024-09-10T09:58:31.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.423+0000 7f68b9c00640 1 -- 172.21.15.77:0/3423065390 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68b419f2d0 con 0x7f68b4077040 2024-09-10T09:58:31.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.424+0000 7f68b9c00640 1 -- 172.21.15.77:0/3423065390 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68b41ab5c0 con 0x7f68b4077040 2024-09-10T09:58:31.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.427+0000 7f68bc615740 1 Processor -- start 2024-09-10T09:58:31.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.427+0000 7f68abe00640 1 -- 172.21.15.77:0/3423065390 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f689c0046c0 con 0x7f68b4077040 2024-09-10T09:58:31.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.428+0000 7f68bc615740 1 -- start start 2024-09-10T09:58:31.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.428+0000 7f68bc615740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562f5c7b0c70 0x562f5c7b1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:31.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.428+0000 7f68bc615740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562f5c7b1590 0x562f5c7bb910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:31.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.428+0000 7f68bc615740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f5c7bbe50 0x562f5c7be240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:31.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.428+0000 7f68bc615740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x562f5c7173e0 con 0x562f5c7b1590 2024-09-10T09:58:31.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.428+0000 7f68bc615740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x562f5c6bed80 con 0x562f5c7bbe50 2024-09-10T09:58:31.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.428+0000 7f68bc615740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562f5c7133a0 con 0x562f5c7b0c70 2024-09-10T09:58:31.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.428+0000 7f68b3400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562f5c7b0c70 0x562f5c7b1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:31.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.428+0000 7f68b3400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562f5c7b0c70 0x562f5c7b1050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:42436/0 (socket says 172.21.15.77:42436) 2024-09-10T09:58:31.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.428+0000 7f68b3400640 1 -- 172.21.15.77:0/4148565359 learned_addr learned my addr 172.21.15.77:0/4148565359 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:58:31.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.428+0000 7f68aa000640 1 --2- 172.21.15.77:0/4148565359 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f5c7bbe50 0x562f5c7be240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:31.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.429+0000 7f68b2a00640 1 --2- 172.21.15.77:0/4148565359 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562f5c7b1590 0x562f5c7bb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:31.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.429+0000 7f68abe00640 1 --2- 172.21.15.77:0/3423065390 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f68980a2c00 0x7f68980a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:31.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.429+0000 7f68aa000640 1 -- 172.21.15.77:0/4148565359 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562f5c7b0c70 msgr2=0x562f5c7b1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:31.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.429+0000 7f68aa000640 1 --2- 172.21.15.77:0/4148565359 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562f5c7b0c70 0x562f5c7b1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:31.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.429+0000 7f68aa000640 1 -- 172.21.15.77:0/4148565359 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562f5c7b1590 msgr2=0x562f5c7bb910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:31.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.429+0000 7f68aa000640 1 --2- 172.21.15.77:0/4148565359 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562f5c7b1590 0x562f5c7bb910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:31.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.429+0000 7f68aa000640 1 -- 172.21.15.77:0/4148565359 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f5c6ddae0 con 0x562f5c7bbe50 2024-09-10T09:58:31.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.429+0000 7f68abe00640 1 -- 172.21.15.77:0/3423065390 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f689c0cf8c0 con 0x7f68b4077040 2024-09-10T09:58:31.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.429+0000 7f68b3400640 1 --2- 172.21.15.77:0/3423065390 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f68980a2c00 0x7f68980a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:31.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.430+0000 7f68aa000640 1 --2- 172.21.15.77:0/4148565359 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f5c7bbe50 0x562f5c7be240 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f689c0ccf10 tx=0x7f689c066960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2277c83842152ed7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:31.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.430+0000 7f68a3e00640 1 -- 172.21.15.77:0/4148565359 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f689c09b810 con 0x562f5c7bbe50 2024-09-10T09:58:31.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.430+0000 7f68a3e00640 1 -- 172.21.15.77:0/4148565359 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f689c066e70 con 0x562f5c7bbe50 2024-09-10T09:58:31.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.430+0000 7f68a3e00640 1 -- 172.21.15.77:0/4148565359 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f689c0bc090 con 0x562f5c7bbe50 2024-09-10T09:58:31.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.431+0000 7f68b3400640 1 --2- 172.21.15.77:0/3423065390 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f68980a2c00 0x7f68980a50c0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f68ac004650 tx=0x7f68ac004730 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:31.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.431+0000 7f68bc615740 1 -- 172.21.15.77:0/4148565359 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f5c7bbe50 msgr2=0x562f5c7be240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:31.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.431+0000 7f68bc615740 1 --2- 172.21.15.77:0/4148565359 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f5c7bbe50 0x562f5c7be240 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f689c0ccf10 tx=0x7f689c066960 comp rx=0 tx=0).stop 2024-09-10T09:58:31.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.431+0000 7f68bc615740 1 -- 172.21.15.77:0/4148565359 shutdown_connections 2024-09-10T09:58:31.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.431+0000 7f68bc615740 1 --2- 172.21.15.77:0/4148565359 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f5c7bbe50 0x562f5c7be240 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:31.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.431+0000 7f68bc615740 1 --2- 172.21.15.77:0/4148565359 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562f5c7b1590 0x562f5c7bb910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:31.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.431+0000 7f68bc615740 1 --2- 172.21.15.77:0/4148565359 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562f5c7b0c70 0x562f5c7b1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:31.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.431+0000 7f68bc615740 1 -- 172.21.15.77:0/4148565359 >> 172.21.15.77:0/4148565359 conn(0x562f5c7ad200 msgr2=0x562f5c7ad5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:31.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.431+0000 7f68bc615740 1 -- 172.21.15.77:0/4148565359 shutdown_connections 2024-09-10T09:58:31.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.431+0000 7f68bc615740 1 -- 172.21.15.77:0/4148565359 wait complete. 2024-09-10T09:58:31.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.432+0000 7f68bc615740 1 -- start start 2024-09-10T09:58:31.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.432+0000 7f68bc615740 1 Processor -- start 2024-09-10T09:58:31.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.433+0000 7f68bc615740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562f5c7b0c70 0x562f5c7b8c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:31.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.433+0000 7f68bc615740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562f5c7b1590 0x562f5c7b91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:31.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.433+0000 7f68bc615740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f5c7bbe50 0x562f5c7b96e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:31.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.433+0000 7f68bc615740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x562f5c7bea20 con 0x562f5c7b0c70 2024-09-10T09:58:31.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.433+0000 7f68bc615740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x562f5c6d20f0 con 0x562f5c7bbe50 2024-09-10T09:58:31.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.433+0000 7f68bc615740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562f5c7beba0 con 0x562f5c7b1590 2024-09-10T09:58:31.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.433+0000 7f68bc615740 10 client.0 _ll_register_callbacks cb 0x562f5c6b9bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:58:31.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.433+0000 7f68aa000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562f5c7b1590 0x562f5c7b91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:31.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.433+0000 7f68b3400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f5c7bbe50 0x562f5c7b96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:31.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.433+0000 7f68aa000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562f5c7b1590 0x562f5c7b91a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:42442/0 (socket says 172.21.15.77:42442) 2024-09-10T09:58:31.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.433+0000 7f68aa000640 1 -- 172.21.15.77:0/3676336107 learned_addr learned my addr 172.21.15.77:0/3676336107 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:58:31.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.433+0000 7f68b2a00640 1 --2- 172.21.15.77:0/3676336107 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562f5c7b0c70 0x562f5c7b8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:31.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.434+0000 7f68aa000640 1 -- 172.21.15.77:0/3676336107 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f5c7bbe50 msgr2=0x562f5c7b96e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:31.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.434+0000 7f68aa000640 1 --2- 172.21.15.77:0/3676336107 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f5c7bbe50 0x562f5c7b96e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:31.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.434+0000 7f68aa000640 1 -- 172.21.15.77:0/3676336107 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562f5c7b0c70 msgr2=0x562f5c7b8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:31.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.434+0000 7f68aa000640 1 --2- 172.21.15.77:0/3676336107 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562f5c7b0c70 0x562f5c7b8c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:31.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.434+0000 7f68aa000640 1 -- 172.21.15.77:0/3676336107 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f5c4883c0 con 0x562f5c7b1590 2024-09-10T09:58:31.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.434+0000 7f68aa000640 1 --2- 172.21.15.77:0/3676336107 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562f5c7b1590 0x562f5c7b91a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f689c09b9d0 tx=0x7f689c06d110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2f702d481dd49df server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:31.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.434+0000 7f68a0c00640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-10T09:58:31.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.434+0000 7f68a0c00640 1 -- 172.21.15.77:0/3676336107 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f689c09b630 con 0x562f5c7b1590 2024-09-10T09:58:31.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.434+0000 7f68a0c00640 1 -- 172.21.15.77:0/3676336107 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f689c0d5150 con 0x562f5c7b1590 2024-09-10T09:58:31.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.434+0000 7f68bc615740 1 -- 172.21.15.77:0/3676336107 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f689c066e70 con 0x562f5c7b1590 2024-09-10T09:58:31.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.434+0000 7f68a0c00640 1 -- 172.21.15.77:0/3676336107 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f689c09b810 con 0x562f5c7b1590 2024-09-10T09:58:31.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.434+0000 7f68bc615740 10 client.26104 fetch_fsmap learned FSMap version 32 2024-09-10T09:58:31.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.434+0000 7f68bc615740 1 -- 172.21.15.77:0/3676336107 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562f5c7bea20 con 0x562f5c7b1590 2024-09-10T09:58:31.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.435+0000 7f68a0c00640 1 -- 172.21.15.77:0/3676336107 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f689c066e70 con 0x562f5c7b1590 2024-09-10T09:58:31.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.435+0000 7f68a0c00640 1 -- 172.21.15.77:0/3676336107 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f689c0ccff0 con 0x562f5c7b1590 2024-09-10T09:58:31.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.435+0000 7f68bc615740 10 client.26104 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T09:58:31.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.435+0000 7f68bc615740 10 client.26104 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T09:58:31.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.435+0000 7f68bc615740 20 client.26104 populate_metadata read hostname 'smithi077' 2024-09-10T09:58:31.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.435+0000 7f68bc615740 1 --2- 172.21.15.77:0/3676336107 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x562f5c7cbb50 0x562f5c7cdf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:31.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.435+0000 7f68bc615740 4 client.26104 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:58:31.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.435+0000 7f68bc615740 1 -- 172.21.15.77:0/3676336107 --> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562f5c724950 con 0x562f5c7cbb50 2024-09-10T09:58:31.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.435+0000 7f68b2a00640 1 --2- 172.21.15.77:0/3676336107 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x562f5c7cbb50 0x562f5c7cdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:31.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.436+0000 7f68b2a00640 1 --2- 172.21.15.77:0/3676336107 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x562f5c7cbb50 0x562f5c7cdf40 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e2694e8d55a9685 server_cookie=9217e3247425b78 in_seq=0 out_seq=0 2024-09-10T09:58:31.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.436+0000 7f68a0c00640 10 client.26104 ms_handle_connect on v2:172.21.15.134:6838/1965117680 2024-09-10T09:58:31.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.437+0000 7f68a0c00640 1 -- 172.21.15.77:0/3676336107 <== mds.2 v2:172.21.15.134:6838/1965117680 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562f5c724950 con 0x562f5c7cbb50 2024-09-10T09:58:31.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.437+0000 7f68a0c00640 10 client.26104 handle_command_reply: tid=0 2024-09-10T09:58:31.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.437+0000 7f68bc615740 1 client.26104 shutdown 2024-09-10T09:58:31.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.437+0000 7f68bc615740 1 -- 172.21.15.77:0/3676336107 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562f5c7b1590 msgr2=0x562f5c7b91a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:31.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.437+0000 7f68bc615740 1 --2- 172.21.15.77:0/3676336107 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562f5c7b1590 0x562f5c7b91a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f689c09b9d0 tx=0x7f689c06d110 comp rx=0 tx=0).stop 2024-09-10T09:58:31.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.437+0000 7f68bc615740 1 -- 172.21.15.77:0/3676336107 shutdown_connections 2024-09-10T09:58:31.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.437+0000 7f68bc615740 1 --2- 172.21.15.77:0/3676336107 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x562f5c7cbb50 0x562f5c7cdf40 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:31.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.437+0000 7f68bc615740 1 --2- 172.21.15.77:0/3676336107 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f5c7bbe50 0x562f5c7b96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:31.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.437+0000 7f68bc615740 1 --2- 172.21.15.77:0/3676336107 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562f5c7b1590 0x562f5c7b91a0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:31.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.437+0000 7f68bc615740 1 --2- 172.21.15.77:0/3676336107 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562f5c7b0c70 0x562f5c7b8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:31.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.437+0000 7f68a0c00640 0 client.26104 ms_handle_reset on v2:172.21.15.134:6838/1965117680 2024-09-10T09:58:31.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.437+0000 7f68bc615740 1 -- 172.21.15.77:0/3676336107 >> 172.21.15.77:0/3676336107 conn(0x562f5c7ad200 msgr2=0x562f5c7b2f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:31.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.437+0000 7f68bc615740 1 -- 172.21.15.77:0/3676336107 shutdown_connections 2024-09-10T09:58:31.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.437+0000 7f68bc615740 1 -- 172.21.15.77:0/3676336107 wait complete. 2024-09-10T09:58:31.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.438+0000 7f68bc615740 20 client.26104 trim_cache size 0 max 16384 2024-09-10T09:58:31.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.452+0000 7f68bc615740 1 Processor -- start 2024-09-10T09:58:31.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.452+0000 7f68bc615740 1 -- start start 2024-09-10T09:58:31.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.452+0000 7f68bc615740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562f5c7b0c70 0x562f5c7b7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:31.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.452+0000 7f68bc615740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562f5c7b1590 0x562f5c7b9530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:31.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.452+0000 7f68bc615740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f5c7bbe50 0x562f5c7d36b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:31.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.452+0000 7f68bc615740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f68b41b54b0 con 0x562f5c7b1590 2024-09-10T09:58:31.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.452+0000 7f68bc615740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f68b41b70b0 con 0x562f5c7bbe50 2024-09-10T09:58:31.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.452+0000 7f68bc615740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f68b41b5b80 con 0x562f5c7b0c70 2024-09-10T09:58:31.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.452+0000 7f68b3400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f5c7bbe50 0x562f5c7d36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:31.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.452+0000 7f68aa000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562f5c7b1590 0x562f5c7b9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:31.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.452+0000 7f68b2a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562f5c7b0c70 0x562f5c7b7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:31.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.452+0000 7f68b3400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f5c7bbe50 0x562f5c7d36b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:48612/0 (socket says 172.21.15.77:48612) 2024-09-10T09:58:31.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.452+0000 7f68b3400640 1 -- 172.21.15.77:0/3349441592 learned_addr learned my addr 172.21.15.77:0/3349441592 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:58:31.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.453+0000 7f68b3400640 1 -- 172.21.15.77:0/3349441592 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562f5c7b0c70 msgr2=0x562f5c7b7ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:31.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.453+0000 7f68b3400640 1 --2- 172.21.15.77:0/3349441592 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562f5c7b0c70 0x562f5c7b7ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:31.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.453+0000 7f68b3400640 1 -- 172.21.15.77:0/3349441592 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562f5c7b1590 msgr2=0x562f5c7b9530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:31.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.453+0000 7f68b3400640 1 --2- 172.21.15.77:0/3349441592 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562f5c7b1590 0x562f5c7b9530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:31.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.453+0000 7f68b3400640 1 -- 172.21.15.77:0/3349441592 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f5c7ad030 con 0x562f5c7bbe50 2024-09-10T09:58:31.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.453+0000 7f68b3400640 1 --2- 172.21.15.77:0/3349441592 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f5c7bbe50 0x562f5c7d36b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f68ac009fa0 tx=0x7f68ac016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4123e2373122d250 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:31.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.453+0000 7f68a3400640 1 -- 172.21.15.77:0/3349441592 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ac028680 con 0x562f5c7bbe50 2024-09-10T09:58:31.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.453+0000 7f68a3400640 1 -- 172.21.15.77:0/3349441592 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68ac01b8f0 con 0x562f5c7bbe50 2024-09-10T09:58:31.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.453+0000 7f68a3400640 1 -- 172.21.15.77:0/3349441592 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ac00f940 con 0x562f5c7bbe50 2024-09-10T09:58:31.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.454+0000 7f68bc615740 1 -- 172.21.15.77:0/3349441592 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f5c7bbe50 msgr2=0x562f5c7d36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:31.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.454+0000 7f68bc615740 1 --2- 172.21.15.77:0/3349441592 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f5c7bbe50 0x562f5c7d36b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f68ac009fa0 tx=0x7f68ac016000 comp rx=0 tx=0).stop 2024-09-10T09:58:31.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.454+0000 7f68bc615740 1 -- 172.21.15.77:0/3349441592 shutdown_connections 2024-09-10T09:58:31.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.454+0000 7f68bc615740 1 --2- 172.21.15.77:0/3349441592 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f5c7bbe50 0x562f5c7d36b0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:31.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.454+0000 7f68bc615740 1 --2- 172.21.15.77:0/3349441592 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562f5c7b1590 0x562f5c7b9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:31.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.454+0000 7f68bc615740 1 --2- 172.21.15.77:0/3349441592 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562f5c7b0c70 0x562f5c7b7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:31.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.454+0000 7f68bc615740 1 -- 172.21.15.77:0/3349441592 >> 172.21.15.77:0/3349441592 conn(0x562f5c7ad200 msgr2=0x562f5c7ac380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:31.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.454+0000 7f68bc615740 1 -- 172.21.15.77:0/3349441592 shutdown_connections 2024-09-10T09:58:31.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.454+0000 7f68bc615740 1 -- 172.21.15.77:0/3349441592 wait complete. 2024-09-10T09:58:31.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.455+0000 7f68bc615740 1 -- start start 2024-09-10T09:58:31.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.455+0000 7f68bc615740 1 Processor -- start 2024-09-10T09:58:31.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.455+0000 7f68bc615740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f5c7b0c70 0x562f5c7c3df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:31.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.455+0000 7f68bc615740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562f5c7b1590 0x562f5c7c4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:31.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.455+0000 7f68bc615740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562f5c7bbe50 0x562f5c7c0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:31.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.455+0000 7f68bc615740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f68b41b6920 con 0x562f5c7b1590 2024-09-10T09:58:31.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.455+0000 7f68bc615740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f68b41b6250 con 0x562f5c7b0c70 2024-09-10T09:58:31.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.455+0000 7f68bc615740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f68b41b3490 con 0x562f5c7bbe50 2024-09-10T09:58:31.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.456+0000 7f68bc615740 10 client.0 _ll_register_callbacks cb 0x7f689c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:58:31.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.456+0000 7f68b3400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562f5c7bbe50 0x562f5c7c0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:31.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.456+0000 7f68b2a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f5c7b0c70 0x562f5c7c3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:31.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.456+0000 7f68b3400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562f5c7bbe50 0x562f5c7c0dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:42468/0 (socket says 172.21.15.77:42468) 2024-09-10T09:58:31.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.456+0000 7f68b3400640 1 -- 172.21.15.77:0/2211455174 learned_addr learned my addr 172.21.15.77:0/2211455174 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:58:31.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.456+0000 7f68b2a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f5c7b0c70 0x562f5c7c3df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:48628/0 (socket says 172.21.15.77:48628) 2024-09-10T09:58:31.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.456+0000 7f68b3400640 1 -- 172.21.15.77:0/2211455174 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f5c7b0c70 msgr2=0x562f5c7c3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:31.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.456+0000 7f68b3400640 1 --2- 172.21.15.77:0/2211455174 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f5c7b0c70 0x562f5c7c3df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:31.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.456+0000 7f68b3400640 1 -- 172.21.15.77:0/2211455174 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562f5c7b1590 msgr2=0x562f5c7c4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:31.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.456+0000 7f68b3400640 1 --2- 172.21.15.77:0/2211455174 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562f5c7b1590 0x562f5c7c4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:31.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.456+0000 7f68b3400640 1 -- 172.21.15.77:0/2211455174 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f5c7c1300 con 0x562f5c7bbe50 2024-09-10T09:58:31.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.456+0000 7f68b2a00640 1 --2- 172.21.15.77:0/2211455174 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f5c7b0c70 0x562f5c7c3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:58:31.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.457+0000 7f68a0c00640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-10T09:58:31.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.457+0000 7f68b3400640 1 --2- 172.21.15.77:0/2211455174 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562f5c7bbe50 0x562f5c7c0dc0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f68ac020010 tx=0x7f68ac00f4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a469d079ac58d74 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:31.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.457+0000 7f68a0c00640 1 -- 172.21.15.77:0/2211455174 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ac036060 con 0x562f5c7bbe50 2024-09-10T09:58:31.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.457+0000 7f68bc615740 1 -- 172.21.15.77:0/2211455174 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68ac01b8f0 con 0x562f5c7bbe50 2024-09-10T09:58:31.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.457+0000 7f68a0c00640 1 -- 172.21.15.77:0/2211455174 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f68ac00f5e0 con 0x562f5c7bbe50 2024-09-10T09:58:31.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.457+0000 7f68a0c00640 1 -- 172.21.15.77:0/2211455174 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f68ac03d050 con 0x562f5c7bbe50 2024-09-10T09:58:31.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.457+0000 7f68bc615740 10 client.26110 fetch_fsmap learned FSMap version 32 2024-09-10T09:58:31.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.457+0000 7f68bc615740 1 -- 172.21.15.77:0/2211455174 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562f5c836d00 con 0x562f5c7bbe50 2024-09-10T09:58:31.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.458+0000 7f68a0c00640 1 -- 172.21.15.77:0/2211455174 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68ac01b8f0 con 0x562f5c7bbe50 2024-09-10T09:58:31.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.458+0000 7f68a0c00640 1 -- 172.21.15.77:0/2211455174 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f68ac0284f0 con 0x562f5c7bbe50 2024-09-10T09:58:31.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.458+0000 7f68bc615740 10 client.26110 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T09:58:31.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.458+0000 7f68bc615740 10 client.26110 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T09:58:31.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.458+0000 7f68bc615740 20 client.26110 populate_metadata read hostname 'smithi077' 2024-09-10T09:58:31.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.458+0000 7f68bc615740 1 --2- 172.21.15.77:0/2211455174 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x562f5c837320 0x562f5c839710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:31.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.458+0000 7f68bc615740 4 client.26110 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:58:31.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.458+0000 7f68bc615740 1 -- 172.21.15.77:0/2211455174 --> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562f5c71ee60 con 0x562f5c837320 2024-09-10T09:58:31.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.458+0000 7f68b2a00640 1 --2- 172.21.15.77:0/2211455174 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x562f5c837320 0x562f5c839710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:31.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.459+0000 7f68b2a00640 1 --2- 172.21.15.77:0/2211455174 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x562f5c837320 0x562f5c839710 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5db1261ebbb848aa server_cookie=6b2d9fa7d3e4d4bf in_seq=0 out_seq=0 2024-09-10T09:58:31.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.459+0000 7f68a0c00640 10 client.26110 ms_handle_connect on v2:172.21.15.134:6838/1965117680 2024-09-10T09:58:31.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.459+0000 7f68a0c00640 1 -- 172.21.15.77:0/2211455174 <== mds.2 v2:172.21.15.134:6838/1965117680 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562f5c71ee60 con 0x562f5c837320 2024-09-10T09:58:31.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.459+0000 7f68a0c00640 10 client.26110 handle_command_reply: tid=0 2024-09-10T09:58:31.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.459+0000 7f68bc615740 1 client.26110 shutdown 2024-09-10T09:58:31.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.459+0000 7f68bc615740 1 -- 172.21.15.77:0/2211455174 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562f5c7bbe50 msgr2=0x562f5c7c0dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:31.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.459+0000 7f68bc615740 1 --2- 172.21.15.77:0/2211455174 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562f5c7bbe50 0x562f5c7c0dc0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f68ac020010 tx=0x7f68ac00f4a0 comp rx=0 tx=0).stop 2024-09-10T09:58:31.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.459+0000 7f68bc615740 1 -- 172.21.15.77:0/2211455174 shutdown_connections 2024-09-10T09:58:31.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.459+0000 7f68bc615740 1 --2- 172.21.15.77:0/2211455174 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x562f5c837320 0x562f5c839710 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:31.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.459+0000 7f68bc615740 1 --2- 172.21.15.77:0/2211455174 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562f5c7bbe50 0x562f5c7c0dc0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:31.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.459+0000 7f68bc615740 1 --2- 172.21.15.77:0/2211455174 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562f5c7b1590 0x562f5c7c4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:31.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.459+0000 7f68a0c00640 0 client.26110 ms_handle_reset on v2:172.21.15.134:6838/1965117680 2024-09-10T09:58:31.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.459+0000 7f68bc615740 1 --2- 172.21.15.77:0/2211455174 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f5c7b0c70 0x562f5c7c3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:31.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.459+0000 7f68bc615740 1 -- 172.21.15.77:0/2211455174 >> 172.21.15.77:0/2211455174 conn(0x562f5c7ad200 msgr2=0x562f5c7abb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:31.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.459+0000 7f68bc615740 1 -- 172.21.15.77:0/2211455174 shutdown_connections 2024-09-10T09:58:31.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.459+0000 7f68bc615740 1 -- 172.21.15.77:0/2211455174 wait complete. 2024-09-10T09:58:31.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.460+0000 7f68bc615740 20 client.26110 trim_cache size 0 max 16384 2024-09-10T09:58:31.463 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-10T09:58:31.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.460+0000 7f68b9c00640 1 -- 172.21.15.77:0/3423065390 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f68980a2c00 msgr2=0x7f68980a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:31.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.460+0000 7f68b9c00640 1 --2- 172.21.15.77:0/3423065390 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f68980a2c00 0x7f68980a50c0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f68ac004650 tx=0x7f68ac004730 comp rx=0 tx=0).stop 2024-09-10T09:58:31.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.460+0000 7f68b9c00640 1 -- 172.21.15.77:0/3423065390 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f68b4077040 msgr2=0x7f68b41a4c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:31.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.460+0000 7f68b9c00640 1 --2- 172.21.15.77:0/3423065390 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f68b4077040 0x7f68b41a4c30 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f689c00e9f0 tx=0x7f689c00edf0 comp rx=0 tx=0).stop 2024-09-10T09:58:31.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.461+0000 7f68b9c00640 1 -- 172.21.15.77:0/3423065390 shutdown_connections 2024-09-10T09:58:31.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.461+0000 7f68b9c00640 1 --2- 172.21.15.77:0/3423065390 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f68980a2c00 0x7f68980a50c0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:31.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.461+0000 7f68b9c00640 1 --2- 172.21.15.77:0/3423065390 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f68b4112a30 0x7f68b419e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:31.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.461+0000 7f68b9c00640 1 --2- 172.21.15.77:0/3423065390 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f68b4077040 0x7f68b41a4c30 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:31.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.461+0000 7f68b9c00640 1 --2- 172.21.15.77:0/3423065390 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68b4076720 0x7f68b41a46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:31.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.461+0000 7f68b9c00640 1 -- 172.21.15.77:0/3423065390 >> 172.21.15.77:0/3423065390 conn(0x7f68b40ffa90 msgr2=0x7f68b4101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:31.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.461+0000 7f68b9c00640 1 -- 172.21.15.77:0/3423065390 shutdown_connections 2024-09-10T09:58:31.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:31.461+0000 7f68b9c00640 1 -- 172.21.15.77:0/3423065390 wait complete. 2024-09-10T09:58:32.478 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T09:58:32.872 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:58:33.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:58:32 smithi134 ceph-mon[35674]: pgmap v492: 129 pgs: 129 active+clean; 75 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 68 KiB/s wr, 40 op/s 2024-09-10T09:58:33.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:58:32 smithi120 ceph-mon[34770]: pgmap v492: 129 pgs: 129 active+clean; 75 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 68 KiB/s wr, 40 op/s 2024-09-10T09:58:33.175 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:58:32 smithi077 ceph-mon[30319]: pgmap v492: 129 pgs: 129 active+clean; 75 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 68 KiB/s wr, 40 op/s 2024-09-10T09:58:33.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.470+0000 7ff93ec00640 1 -- 172.21.15.77:0/2379991081 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff93810e210 msgr2=0x7ff938110660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.470+0000 7ff93ec00640 1 --2- 172.21.15.77:0/2379991081 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff93810e210 0x7ff938110660 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7ff930009ca0 tx=0x7ff93002a950 comp rx=0 tx=0).stop 2024-09-10T09:58:33.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.470+0000 7ff93ec00640 1 -- 172.21.15.77:0/2379991081 shutdown_connections 2024-09-10T09:58:33.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.470+0000 7ff93ec00640 1 --2- 172.21.15.77:0/2379991081 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff93810e210 0x7ff938110660 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.470+0000 7ff93ec00640 1 --2- 172.21.15.77:0/2379991081 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff938074f40 0x7ff93810dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.470+0000 7ff93ec00640 1 --2- 172.21.15.77:0/2379991081 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff938074620 0x7ff938074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.470+0000 7ff93ec00640 1 -- 172.21.15.77:0/2379991081 >> 172.21.15.77:0/2379991081 conn(0x7ff9380fd710 msgr2=0x7ff9380ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:33.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.470+0000 7ff93ec00640 1 -- 172.21.15.77:0/2379991081 shutdown_connections 2024-09-10T09:58:33.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.470+0000 7ff93ec00640 1 -- 172.21.15.77:0/2379991081 wait complete. 2024-09-10T09:58:33.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.471+0000 7ff93ec00640 1 Processor -- start 2024-09-10T09:58:33.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.471+0000 7ff93ec00640 1 -- start start 2024-09-10T09:58:33.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.472+0000 7ff93ec00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff938074620 0x7ff93819a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.472+0000 7ff93ec00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff938074f40 0x7ff93819a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.472+0000 7ff93ec00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff93810e210 0x7ff93819ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.472+0000 7ff93ec00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff938113fd0 con 0x7ff938074620 2024-09-10T09:58:33.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.472+0000 7ff93ec00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff938113e50 con 0x7ff938074f40 2024-09-10T09:58:33.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.472+0000 7ff93ec00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff938114150 con 0x7ff93810e210 2024-09-10T09:58:33.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.472+0000 7ff93d800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff938074620 0x7ff93819a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.472+0000 7ff93ce00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff938074f40 0x7ff93819a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.472+0000 7ff93e200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff93810e210 0x7ff93819ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.472+0000 7ff93ce00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff938074f40 0x7ff93819a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:40300/0 (socket says 172.21.15.77:40300) 2024-09-10T09:58:33.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.472+0000 7ff93ce00640 1 -- 172.21.15.77:0/3069898006 learned_addr learned my addr 172.21.15.77:0/3069898006 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:58:33.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.472+0000 7ff93ce00640 1 -- 172.21.15.77:0/3069898006 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff93810e210 msgr2=0x7ff93819ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.472+0000 7ff93ce00640 1 --2- 172.21.15.77:0/3069898006 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff93810e210 0x7ff93819ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.472+0000 7ff93ce00640 1 -- 172.21.15.77:0/3069898006 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff938074620 msgr2=0x7ff93819a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.472+0000 7ff93ce00640 1 --2- 172.21.15.77:0/3069898006 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff938074620 0x7ff93819a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.472+0000 7ff93ce00640 1 -- 172.21.15.77:0/3069898006 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff93819f400 con 0x7ff938074f40 2024-09-10T09:58:33.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.472+0000 7ff93d800640 1 --2- 172.21.15.77:0/3069898006 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff938074620 0x7ff93819a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:58:33.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.473+0000 7ff93ce00640 1 --2- 172.21.15.77:0/3069898006 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff938074f40 0x7ff93819a990 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff92400d950 tx=0x7ff92400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:33.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.473+0000 7ff936000640 1 -- 172.21.15.77:0/3069898006 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff924017070 con 0x7ff938074f40 2024-09-10T09:58:33.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.473+0000 7ff936000640 1 -- 172.21.15.77:0/3069898006 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff924009c30 con 0x7ff938074f40 2024-09-10T09:58:33.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.473+0000 7ff93ec00640 1 -- 172.21.15.77:0/3069898006 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff93819f6f0 con 0x7ff938074f40 2024-09-10T09:58:33.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.473+0000 7ff936000640 1 -- 172.21.15.77:0/3069898006 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff92400f6b0 con 0x7ff938074f40 2024-09-10T09:58:33.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.473+0000 7ff93ec00640 1 -- 172.21.15.77:0/3069898006 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9381a6fd0 con 0x7ff938074f40 2024-09-10T09:58:33.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.476+0000 7ff936000640 1 -- 172.21.15.77:0/3069898006 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff9240109d0 con 0x7ff938074f40 2024-09-10T09:58:33.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.476+0000 7ff942cb9740 1 Processor -- start 2024-09-10T09:58:33.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.477+0000 7ff942cb9740 1 -- start start 2024-09-10T09:58:33.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.477+0000 7ff942cb9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55938e535e40 0x55938e536220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.477+0000 7ff942cb9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55938e536760 0x55938e540ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.477+0000 7ff942cb9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55938e541020 0x55938e543410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.477+0000 7ff942cb9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55938e49c3e0 con 0x55938e541020 2024-09-10T09:58:33.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.477+0000 7ff942cb9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55938e443d80 con 0x55938e535e40 2024-09-10T09:58:33.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.477+0000 7ff942cb9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55938e4983a0 con 0x55938e536760 2024-09-10T09:58:33.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.477+0000 7ff93d800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55938e536760 0x55938e540ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.477+0000 7ff93e200640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55938e535e40 0x55938e536220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.477+0000 7ff93d800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55938e536760 0x55938e540ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:49034/0 (socket says 172.21.15.77:49034) 2024-09-10T09:58:33.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.477+0000 7ff93d800640 1 -- 172.21.15.77:0/309720802 learned_addr learned my addr 172.21.15.77:0/309720802 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:58:33.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.477+0000 7ff93ce00640 1 --2- 172.21.15.77:0/309720802 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55938e541020 0x55938e543410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.478+0000 7ff93d800640 1 -- 172.21.15.77:0/309720802 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55938e535e40 msgr2=0x55938e536220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:33.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.478+0000 7ff93d800640 1 --2- 172.21.15.77:0/309720802 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55938e535e40 0x55938e536220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.478+0000 7ff93d800640 1 -- 172.21.15.77:0/309720802 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55938e541020 msgr2=0x55938e543410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:33.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.478+0000 7ff93d800640 1 --2- 172.21.15.77:0/309720802 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55938e541020 0x55938e543410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.478+0000 7ff93d800640 1 -- 172.21.15.77:0/309720802 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55938e462ae0 con 0x55938e536760 2024-09-10T09:58:33.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.478+0000 7ff93ce00640 1 --2- 172.21.15.77:0/309720802 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55938e541020 0x55938e543410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:58:33.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.478+0000 7ff93d800640 1 --2- 172.21.15.77:0/309720802 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55938e536760 0x55938e540ae0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff92c0115f0 tx=0x7ff92c011ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2c1c10171c8a58c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:33.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.478+0000 7ff936000640 1 --2- 172.21.15.77:0/3069898006 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ff9200a2b80 0x7ff9200a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.478+0000 7ff936000640 1 -- 172.21.15.77:0/3069898006 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7ff9240cefe0 con 0x7ff938074f40 2024-09-10T09:58:33.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.478+0000 7ff92aa00640 1 -- 172.21.15.77:0/309720802 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff92c01e070 con 0x55938e536760 2024-09-10T09:58:33.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.479+0000 7ff92aa00640 1 -- 172.21.15.77:0/309720802 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff92c00fb20 con 0x55938e536760 2024-09-10T09:58:33.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.479+0000 7ff92aa00640 1 -- 172.21.15.77:0/309720802 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff92c019c40 con 0x55938e536760 2024-09-10T09:58:33.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.479+0000 7ff93e200640 1 --2- 172.21.15.77:0/3069898006 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ff9200a2b80 0x7ff9200a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.479+0000 7ff942cb9740 1 -- 172.21.15.77:0/309720802 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55938e536760 msgr2=0x55938e540ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.479+0000 7ff942cb9740 1 --2- 172.21.15.77:0/309720802 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55938e536760 0x55938e540ae0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff92c0115f0 tx=0x7ff92c011ab0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.480+0000 7ff93e200640 1 --2- 172.21.15.77:0/3069898006 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ff9200a2b80 0x7ff9200a5040 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ff930006010 tx=0x7ff930005d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:33.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.480+0000 7ff942cb9740 1 -- 172.21.15.77:0/309720802 shutdown_connections 2024-09-10T09:58:33.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.480+0000 7ff942cb9740 1 --2- 172.21.15.77:0/309720802 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55938e541020 0x55938e543410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.480+0000 7ff942cb9740 1 --2- 172.21.15.77:0/309720802 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55938e536760 0x55938e540ae0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.480+0000 7ff942cb9740 1 --2- 172.21.15.77:0/309720802 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55938e535e40 0x55938e536220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.480+0000 7ff942cb9740 1 -- 172.21.15.77:0/309720802 >> 172.21.15.77:0/309720802 conn(0x55938e5323d0 msgr2=0x55938e5327b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:33.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.480+0000 7ff942cb9740 1 -- 172.21.15.77:0/309720802 shutdown_connections 2024-09-10T09:58:33.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.480+0000 7ff942cb9740 1 -- 172.21.15.77:0/309720802 wait complete. 2024-09-10T09:58:33.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.480+0000 7ff942cb9740 1 -- start start 2024-09-10T09:58:33.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.481+0000 7ff942cb9740 1 Processor -- start 2024-09-10T09:58:33.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.481+0000 7ff942cb9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55938e535e40 0x55938e53dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.481+0000 7ff942cb9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55938e536760 0x55938e53e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.481+0000 7ff942cb9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55938e541020 0x55938e53e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.481+0000 7ff942cb9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55938e543bf0 con 0x55938e541020 2024-09-10T09:58:33.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.481+0000 7ff942cb9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55938e4570f0 con 0x55938e536760 2024-09-10T09:58:33.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.481+0000 7ff942cb9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55938e543d70 con 0x55938e535e40 2024-09-10T09:58:33.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.481+0000 7ff942cb9740 10 client.0 _ll_register_callbacks cb 0x55938e43ebd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:58:33.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.481+0000 7ff93d800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55938e535e40 0x55938e53dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.481+0000 7ff93d800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55938e535e40 0x55938e53dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:49046/0 (socket says 172.21.15.77:49046) 2024-09-10T09:58:33.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.481+0000 7ff93d800640 1 -- 172.21.15.77:0/1353148287 learned_addr learned my addr 172.21.15.77:0/1353148287 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:58:33.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.481+0000 7ff93ce00640 1 --2- 172.21.15.77:0/1353148287 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55938e536760 0x55938e53e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.482+0000 7ff93e200640 1 --2- 172.21.15.77:0/1353148287 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55938e541020 0x55938e53e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.482+0000 7ff93d800640 1 -- 172.21.15.77:0/1353148287 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55938e536760 msgr2=0x55938e53e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:33.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.482+0000 7ff93d800640 1 --2- 172.21.15.77:0/1353148287 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55938e536760 0x55938e53e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.482+0000 7ff93d800640 1 -- 172.21.15.77:0/1353148287 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55938e541020 msgr2=0x55938e53e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:33.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.482+0000 7ff93d800640 1 --2- 172.21.15.77:0/1353148287 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55938e541020 0x55938e53e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.482+0000 7ff93d800640 1 -- 172.21.15.77:0/1353148287 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55938e20d3c0 con 0x55938e535e40 2024-09-10T09:58:33.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.482+0000 7ff93d800640 1 --2- 172.21.15.77:0/1353148287 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55938e535e40 0x55938e53dec0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff92c01c040 tx=0x7ff92c00f850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e86b8dd838c928e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:33.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.482+0000 7ff91f400640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-10T09:58:33.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.482+0000 7ff91f400640 1 -- 172.21.15.77:0/1353148287 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff92c01e040 con 0x55938e535e40 2024-09-10T09:58:33.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.482+0000 7ff91f400640 1 -- 172.21.15.77:0/1353148287 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7ff92c0199c0 con 0x55938e535e40 2024-09-10T09:58:33.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.482+0000 7ff942cb9740 1 -- 172.21.15.77:0/1353148287 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff92c00fb20 con 0x55938e535e40 2024-09-10T09:58:33.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.483+0000 7ff91f400640 1 -- 172.21.15.77:0/1353148287 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff92c02f050 con 0x55938e535e40 2024-09-10T09:58:33.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.483+0000 7ff942cb9740 10 client.26122 fetch_fsmap learned FSMap version 32 2024-09-10T09:58:33.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.483+0000 7ff942cb9740 1 -- 172.21.15.77:0/1353148287 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55938e54fc10 con 0x55938e535e40 2024-09-10T09:58:33.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.483+0000 7ff91f400640 1 -- 172.21.15.77:0/1353148287 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff92c00fb20 con 0x55938e535e40 2024-09-10T09:58:33.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.483+0000 7ff91f400640 1 -- 172.21.15.77:0/1353148287 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7ff92c0035f0 con 0x55938e535e40 2024-09-10T09:58:33.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.483+0000 7ff942cb9740 10 client.26122 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T09:58:33.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.483+0000 7ff942cb9740 10 client.26122 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-10T09:58:33.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.483+0000 7ff942cb9740 20 client.26122 populate_metadata read hostname 'smithi077' 2024-09-10T09:58:33.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.483+0000 7ff942cb9740 1 --2- 172.21.15.77:0/1353148287 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x55938e551070 0x55938e553460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.483+0000 7ff942cb9740 4 client.26122 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:58:33.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.483+0000 7ff942cb9740 1 -- 172.21.15.77:0/1353148287 --> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55938e4a9950 con 0x55938e551070 2024-09-10T09:58:33.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.484+0000 7ff93ce00640 1 --2- 172.21.15.77:0/1353148287 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x55938e551070 0x55938e553460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.484+0000 7ff93ce00640 1 --2- 172.21.15.77:0/1353148287 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x55938e551070 0x55938e553460 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b83112075a6bd6cb server_cookie=5b59cb4c5afe4dea in_seq=0 out_seq=0 2024-09-10T09:58:33.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.484+0000 7ff91f400640 10 client.26122 ms_handle_connect on v2:172.21.15.120:6836/3033280709 2024-09-10T09:58:33.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.485+0000 7ff91f400640 1 -- 172.21.15.77:0/1353148287 <== mds.1 v2:172.21.15.120:6836/3033280709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55938e4a9950 con 0x55938e551070 2024-09-10T09:58:33.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.485+0000 7ff91f400640 10 client.26122 handle_command_reply: tid=0 2024-09-10T09:58:33.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.485+0000 7ff942cb9740 1 client.26122 shutdown 2024-09-10T09:58:33.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.485+0000 7ff942cb9740 1 -- 172.21.15.77:0/1353148287 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55938e535e40 msgr2=0x55938e53dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.485+0000 7ff942cb9740 1 --2- 172.21.15.77:0/1353148287 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55938e535e40 0x55938e53dec0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff92c01c040 tx=0x7ff92c00f850 comp rx=0 tx=0).stop 2024-09-10T09:58:33.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.485+0000 7ff942cb9740 1 -- 172.21.15.77:0/1353148287 shutdown_connections 2024-09-10T09:58:33.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.485+0000 7ff942cb9740 1 --2- 172.21.15.77:0/1353148287 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x55938e551070 0x55938e553460 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.485+0000 7ff942cb9740 1 --2- 172.21.15.77:0/1353148287 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55938e541020 0x55938e53e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.485+0000 7ff942cb9740 1 --2- 172.21.15.77:0/1353148287 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55938e536760 0x55938e53e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.485+0000 7ff91f400640 0 client.26122 ms_handle_reset on v2:172.21.15.120:6836/3033280709 2024-09-10T09:58:33.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.485+0000 7ff942cb9740 1 --2- 172.21.15.77:0/1353148287 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55938e535e40 0x55938e53dec0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.485+0000 7ff942cb9740 1 -- 172.21.15.77:0/1353148287 >> 172.21.15.77:0/1353148287 conn(0x55938e5323d0 msgr2=0x55938e5380b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:33.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.486+0000 7ff942cb9740 1 -- 172.21.15.77:0/1353148287 shutdown_connections 2024-09-10T09:58:33.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.486+0000 7ff942cb9740 1 -- 172.21.15.77:0/1353148287 wait complete. 2024-09-10T09:58:33.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.486+0000 7ff942cb9740 20 client.26122 trim_cache size 0 max 16384 2024-09-10T09:58:33.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.500+0000 7ff942cb9740 1 Processor -- start 2024-09-10T09:58:33.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.500+0000 7ff942cb9740 1 -- start start 2024-09-10T09:58:33.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.501+0000 7ff942cb9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55938e535e40 0x55938e5388a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.501+0000 7ff942cb9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55938e536760 0x55938e53c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.501+0000 7ff942cb9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55938e541020 0x55938e558bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.501+0000 7ff942cb9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff9381b0f60 con 0x55938e541020 2024-09-10T09:58:33.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.501+0000 7ff942cb9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff9381b2b60 con 0x55938e535e40 2024-09-10T09:58:33.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.501+0000 7ff942cb9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff9381b1630 con 0x55938e536760 2024-09-10T09:58:33.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.501+0000 7ff93d800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55938e535e40 0x55938e5388a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.501+0000 7ff93ce00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55938e536760 0x55938e53c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.501+0000 7ff93d800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55938e535e40 0x55938e5388a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:40328/0 (socket says 172.21.15.77:40328) 2024-09-10T09:58:33.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.501+0000 7ff93d800640 1 -- 172.21.15.77:0/1236616424 learned_addr learned my addr 172.21.15.77:0/1236616424 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:58:33.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.501+0000 7ff93e200640 1 --2- 172.21.15.77:0/1236616424 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55938e541020 0x55938e558bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.501+0000 7ff93d800640 1 -- 172.21.15.77:0/1236616424 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55938e536760 msgr2=0x55938e53c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:33.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.501+0000 7ff93d800640 1 --2- 172.21.15.77:0/1236616424 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55938e536760 0x55938e53c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.501+0000 7ff93d800640 1 -- 172.21.15.77:0/1236616424 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55938e541020 msgr2=0x55938e558bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:33.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.501+0000 7ff93d800640 1 --2- 172.21.15.77:0/1236616424 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55938e541020 0x55938e558bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.501+0000 7ff93d800640 1 -- 172.21.15.77:0/1236616424 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55938e532200 con 0x55938e535e40 2024-09-10T09:58:33.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.501+0000 7ff93ce00640 1 --2- 172.21.15.77:0/1236616424 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55938e536760 0x55938e53c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:58:33.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.502+0000 7ff93d800640 1 --2- 172.21.15.77:0/1236616424 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55938e535e40 0x55938e5388a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff92c000c00 tx=0x7ff92c0059b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc703958e5cbed35 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:33.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.502+0000 7ff92a000640 1 -- 172.21.15.77:0/1236616424 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff92c0194d0 con 0x55938e535e40 2024-09-10T09:58:33.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.502+0000 7ff92a000640 1 -- 172.21.15.77:0/1236616424 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff92c015020 con 0x55938e535e40 2024-09-10T09:58:33.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.502+0000 7ff92a000640 1 -- 172.21.15.77:0/1236616424 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff92c0052a0 con 0x55938e535e40 2024-09-10T09:58:33.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.503+0000 7ff942cb9740 1 -- 172.21.15.77:0/1236616424 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55938e535e40 msgr2=0x55938e5388a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.503+0000 7ff942cb9740 1 --2- 172.21.15.77:0/1236616424 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55938e535e40 0x55938e5388a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff92c000c00 tx=0x7ff92c0059b0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.503+0000 7ff942cb9740 1 -- 172.21.15.77:0/1236616424 shutdown_connections 2024-09-10T09:58:33.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.503+0000 7ff942cb9740 1 --2- 172.21.15.77:0/1236616424 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55938e541020 0x55938e558bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.503+0000 7ff942cb9740 1 --2- 172.21.15.77:0/1236616424 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55938e536760 0x55938e53c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.503+0000 7ff942cb9740 1 --2- 172.21.15.77:0/1236616424 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55938e535e40 0x55938e5388a0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.503+0000 7ff942cb9740 1 -- 172.21.15.77:0/1236616424 >> 172.21.15.77:0/1236616424 conn(0x55938e5323d0 msgr2=0x55938e53ff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:33.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.503+0000 7ff942cb9740 1 -- 172.21.15.77:0/1236616424 shutdown_connections 2024-09-10T09:58:33.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.503+0000 7ff942cb9740 1 -- 172.21.15.77:0/1236616424 wait complete. 2024-09-10T09:58:33.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.503+0000 7ff942cb9740 1 -- start start 2024-09-10T09:58:33.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.504+0000 7ff942cb9740 1 Processor -- start 2024-09-10T09:58:33.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.504+0000 7ff942cb9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55938e535e40 0x55938e544dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.504+0000 7ff942cb9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55938e536760 0x55938e545310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.504+0000 7ff942cb9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55938e541020 0x55938e545850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.504+0000 7ff942cb9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff9381b23d0 con 0x55938e536760 2024-09-10T09:58:33.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.504+0000 7ff942cb9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff9381b1d00 con 0x55938e535e40 2024-09-10T09:58:33.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.504+0000 7ff942cb9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff9381aef40 con 0x55938e541020 2024-09-10T09:58:33.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.504+0000 7ff942cb9740 10 client.0 _ll_register_callbacks cb 0x7ff92c01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:58:33.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.504+0000 7ff93d800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55938e535e40 0x55938e544dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.504+0000 7ff93d800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55938e535e40 0x55938e544dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:40332/0 (socket says 172.21.15.77:40332) 2024-09-10T09:58:33.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.504+0000 7ff93d800640 1 -- 172.21.15.77:0/4149231763 learned_addr learned my addr 172.21.15.77:0/4149231763 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:58:33.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.504+0000 7ff93e200640 1 --2- 172.21.15.77:0/4149231763 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55938e541020 0x55938e545850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.505+0000 7ff93ce00640 1 --2- 172.21.15.77:0/4149231763 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55938e536760 0x55938e545310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.505+0000 7ff93d800640 1 -- 172.21.15.77:0/4149231763 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55938e541020 msgr2=0x55938e545850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:33.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.505+0000 7ff93d800640 1 --2- 172.21.15.77:0/4149231763 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55938e541020 0x55938e545850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.505+0000 7ff93d800640 1 -- 172.21.15.77:0/4149231763 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55938e536760 msgr2=0x55938e545310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:33.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.505+0000 7ff93d800640 1 --2- 172.21.15.77:0/4149231763 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55938e536760 0x55938e545310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.505+0000 7ff93d800640 1 -- 172.21.15.77:0/4149231763 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55938e5bac20 con 0x55938e535e40 2024-09-10T09:58:33.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.505+0000 7ff93d800640 1 --2- 172.21.15.77:0/4149231763 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55938e535e40 0x55938e544dd0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff92c006010 tx=0x7ff92c00fc10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=777ac54a5c73e9d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:33.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.505+0000 7ff91f400640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T09:58:33.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.505+0000 7ff91f400640 1 -- 172.21.15.77:0/4149231763 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff92c0035f0 con 0x55938e535e40 2024-09-10T09:58:33.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.505+0000 7ff942cb9740 1 -- 172.21.15.77:0/4149231763 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff92c015020 con 0x55938e535e40 2024-09-10T09:58:33.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.505+0000 7ff91f400640 1 -- 172.21.15.77:0/4149231763 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7ff92c016070 con 0x55938e535e40 2024-09-10T09:58:33.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.506+0000 7ff91f400640 1 -- 172.21.15.77:0/4149231763 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff92c042050 con 0x55938e535e40 2024-09-10T09:58:33.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.506+0000 7ff942cb9740 10 client.26177 fetch_fsmap learned FSMap version 32 2024-09-10T09:58:33.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.506+0000 7ff942cb9740 1 -- 172.21.15.77:0/4149231763 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55938e5bbf00 con 0x55938e535e40 2024-09-10T09:58:33.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.506+0000 7ff91f400640 1 -- 172.21.15.77:0/4149231763 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff92c015020 con 0x55938e535e40 2024-09-10T09:58:33.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.506+0000 7ff91f400640 1 -- 172.21.15.77:0/4149231763 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7ff92c005100 con 0x55938e535e40 2024-09-10T09:58:33.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.506+0000 7ff942cb9740 10 client.26177 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T09:58:33.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.506+0000 7ff942cb9740 10 client.26177 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-10T09:58:33.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.506+0000 7ff942cb9740 20 client.26177 populate_metadata read hostname 'smithi077' 2024-09-10T09:58:33.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.506+0000 7ff942cb9740 1 --2- 172.21.15.77:0/4149231763 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x55938e5bc500 0x55938e5be8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.506+0000 7ff942cb9740 4 client.26177 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:58:33.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.506+0000 7ff942cb9740 1 -- 172.21.15.77:0/4149231763 --> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55938e4a3e60 con 0x55938e5bc500 2024-09-10T09:58:33.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.506+0000 7ff93ce00640 1 --2- 172.21.15.77:0/4149231763 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x55938e5bc500 0x55938e5be8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.507+0000 7ff91f400640 10 client.26177 ms_handle_connect on v2:172.21.15.120:6836/3033280709 2024-09-10T09:58:33.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.507+0000 7ff93ce00640 1 --2- 172.21.15.77:0/4149231763 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x55938e5bc500 0x55938e5be8f0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=64d7d0e7964d9cf server_cookie=88a00ab163be437f in_seq=0 out_seq=0 2024-09-10T09:58:33.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.507+0000 7ff91f400640 1 -- 172.21.15.77:0/4149231763 <== mds.1 v2:172.21.15.120:6836/3033280709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55938e4a3e60 con 0x55938e5bc500 2024-09-10T09:58:33.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.507+0000 7ff91f400640 10 client.26177 handle_command_reply: tid=0 2024-09-10T09:58:33.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.507+0000 7ff942cb9740 1 client.26177 shutdown 2024-09-10T09:58:33.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.508+0000 7ff942cb9740 1 -- 172.21.15.77:0/4149231763 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55938e535e40 msgr2=0x55938e544dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.508+0000 7ff942cb9740 1 --2- 172.21.15.77:0/4149231763 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55938e535e40 0x55938e544dd0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff92c006010 tx=0x7ff92c00fc10 comp rx=0 tx=0).stop 2024-09-10T09:58:33.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.508+0000 7ff942cb9740 1 -- 172.21.15.77:0/4149231763 shutdown_connections 2024-09-10T09:58:33.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.508+0000 7ff942cb9740 1 --2- 172.21.15.77:0/4149231763 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x55938e5bc500 0x55938e5be8f0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.508+0000 7ff942cb9740 1 --2- 172.21.15.77:0/4149231763 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55938e541020 0x55938e545850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.508+0000 7ff942cb9740 1 --2- 172.21.15.77:0/4149231763 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55938e536760 0x55938e545310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.508+0000 7ff91f400640 0 client.26177 ms_handle_reset on v2:172.21.15.120:6836/3033280709 2024-09-10T09:58:33.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.508+0000 7ff942cb9740 1 --2- 172.21.15.77:0/4149231763 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55938e535e40 0x55938e544dd0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.508+0000 7ff942cb9740 1 -- 172.21.15.77:0/4149231763 >> 172.21.15.77:0/4149231763 conn(0x55938e5323d0 msgr2=0x55938e530c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:33.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.508+0000 7ff942cb9740 1 -- 172.21.15.77:0/4149231763 shutdown_connections 2024-09-10T09:58:33.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.508+0000 7ff942cb9740 1 -- 172.21.15.77:0/4149231763 wait complete. 2024-09-10T09:58:33.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.508+0000 7ff942cb9740 20 client.26177 trim_cache size 0 max 16384 2024-09-10T09:58:33.512 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-10T09:58:33.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.509+0000 7ff93ec00640 1 -- 172.21.15.77:0/3069898006 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ff9200a2b80 msgr2=0x7ff9200a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.509+0000 7ff93ec00640 1 --2- 172.21.15.77:0/3069898006 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ff9200a2b80 0x7ff9200a5040 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ff930006010 tx=0x7ff930005d80 comp rx=0 tx=0).stop 2024-09-10T09:58:33.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.509+0000 7ff93ec00640 1 -- 172.21.15.77:0/3069898006 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff938074f40 msgr2=0x7ff93819a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.509+0000 7ff93ec00640 1 --2- 172.21.15.77:0/3069898006 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff938074f40 0x7ff93819a990 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff92400d950 tx=0x7ff92400de10 comp rx=0 tx=0).stop 2024-09-10T09:58:33.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.509+0000 7ff93ec00640 1 -- 172.21.15.77:0/3069898006 shutdown_connections 2024-09-10T09:58:33.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.509+0000 7ff93ec00640 1 --2- 172.21.15.77:0/3069898006 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ff9200a2b80 0x7ff9200a5040 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.509+0000 7ff93ec00640 1 --2- 172.21.15.77:0/3069898006 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff93810e210 0x7ff93819ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.509+0000 7ff93ec00640 1 --2- 172.21.15.77:0/3069898006 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff938074f40 0x7ff93819a990 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.509+0000 7ff93ec00640 1 --2- 172.21.15.77:0/3069898006 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff938074620 0x7ff93819a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.509+0000 7ff93ec00640 1 -- 172.21.15.77:0/3069898006 >> 172.21.15.77:0/3069898006 conn(0x7ff9380fd710 msgr2=0x7ff9380fe370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:33.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.509+0000 7ff93ec00640 1 -- 172.21.15.77:0/3069898006 shutdown_connections 2024-09-10T09:58:33.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:33.509+0000 7ff93ec00640 1 -- 172.21.15.77:0/3069898006 wait complete. 2024-09-10T09:58:34.419 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T09:58:34.445 INFO:tasks.workunit.client.0.smithi077.stdout:copying to largest ever: 0xfe492 2024-09-10T09:58:34.813 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:58:35.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:58:34 smithi134 ceph-mon[35674]: pgmap v493: 129 pgs: 129 active+clean; 77 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 797 KiB/s rd, 1.1 MiB/s wr, 197 op/s 2024-09-10T09:58:35.132 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:58:34 smithi077 ceph-mon[30319]: pgmap v493: 129 pgs: 129 active+clean; 77 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 797 KiB/s rd, 1.1 MiB/s wr, 197 op/s 2024-09-10T09:58:35.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:58:34 smithi120 ceph-mon[34770]: pgmap v493: 129 pgs: 129 active+clean; 77 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 797 KiB/s rd, 1.1 MiB/s wr, 197 op/s 2024-09-10T09:58:35.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.368+0000 7fc763000640 1 -- 172.21.15.77:0/3755193367 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc75c074510 msgr2=0x7fc75c0748f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:35.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.368+0000 7fc763000640 1 --2- 172.21.15.77:0/3755193367 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc75c074510 0x7fc75c0748f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc74c009900 tx=0x7fc74c02a480 comp rx=0 tx=0).stop 2024-09-10T09:58:35.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.368+0000 7fc763000640 1 -- 172.21.15.77:0/3755193367 shutdown_connections 2024-09-10T09:58:35.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.368+0000 7fc763000640 1 --2- 172.21.15.77:0/3755193367 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc75c10e450 0x7fc75c110840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:35.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.368+0000 7fc763000640 1 --2- 172.21.15.77:0/3755193367 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc75c074e30 0x7fc75c10df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:35.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.368+0000 7fc763000640 1 --2- 172.21.15.77:0/3755193367 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc75c074510 0x7fc75c0748f0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:35.370 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.368+0000 7fc763000640 1 -- 172.21.15.77:0/3755193367 >> 172.21.15.77:0/3755193367 conn(0x7fc75c0fd8b0 msgr2=0x7fc75c0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:35.370 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.368+0000 7fc763000640 1 -- 172.21.15.77:0/3755193367 shutdown_connections 2024-09-10T09:58:35.370 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.369+0000 7fc763000640 1 -- 172.21.15.77:0/3755193367 wait complete. 2024-09-10T09:58:35.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.370+0000 7fc763000640 1 Processor -- start 2024-09-10T09:58:35.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.370+0000 7fc763000640 1 -- start start 2024-09-10T09:58:35.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.371+0000 7fc763000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc75c074510 0x7fc75c19a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:35.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.371+0000 7fc763000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc75c074e30 0x7fc75c19ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:35.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.371+0000 7fc763000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc75c10e450 0x7fc75c19efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:35.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.371+0000 7fc763000640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc75c1141b0 con 0x7fc75c10e450 2024-09-10T09:58:35.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.371+0000 7fc763000640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc75c114030 con 0x7fc75c074e30 2024-09-10T09:58:35.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.371+0000 7fc763000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc75c114330 con 0x7fc75c074510 2024-09-10T09:58:35.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.371+0000 7fc75be00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc75c074510 0x7fc75c19a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:35.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.371+0000 7fc75be00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc75c074510 0x7fc75c19a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:49096/0 (socket says 172.21.15.77:49096) 2024-09-10T09:58:35.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.371+0000 7fc75be00640 1 -- 172.21.15.77:0/3623769995 learned_addr learned my addr 172.21.15.77:0/3623769995 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:58:35.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.371+0000 7fc75b400640 1 --2- 172.21.15.77:0/3623769995 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc75c074e30 0x7fc75c19ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:35.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.371+0000 7fc75be00640 1 -- 172.21.15.77:0/3623769995 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc75c074e30 msgr2=0x7fc75c19ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:35.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.371+0000 7fc75be00640 1 --2- 172.21.15.77:0/3623769995 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc75c074e30 0x7fc75c19ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:35.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.371+0000 7fc75be00640 1 -- 172.21.15.77:0/3623769995 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc75c10e450 msgr2=0x7fc75c19efe0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T09:58:35.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.372+0000 7fc75be00640 1 --2- 172.21.15.77:0/3623769995 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc75c10e450 0x7fc75c19efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:35.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.372+0000 7fc75be00640 1 -- 172.21.15.77:0/3623769995 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc75c19f6c0 con 0x7fc75c074510 2024-09-10T09:58:35.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.372+0000 7fc75be00640 1 --2- 172.21.15.77:0/3623769995 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc75c074510 0x7fc75c19a710 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc74c004770 tx=0x7fc74c003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:35.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.372+0000 7fc758c00640 1 -- 172.21.15.77:0/3623769995 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc74c03f070 con 0x7fc75c074510 2024-09-10T09:58:35.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.373+0000 7fc763000640 1 -- 172.21.15.77:0/3623769995 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc75c19f950 con 0x7fc75c074510 2024-09-10T09:58:35.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.373+0000 7fc758c00640 1 -- 172.21.15.77:0/3623769995 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc74c031ca0 con 0x7fc75c074510 2024-09-10T09:58:35.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.373+0000 7fc758c00640 1 -- 172.21.15.77:0/3623769995 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc74c035030 con 0x7fc75c074510 2024-09-10T09:58:35.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.373+0000 7fc763000640 1 -- 172.21.15.77:0/3623769995 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc75c1a7290 con 0x7fc75c074510 2024-09-10T09:58:35.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.376+0000 7fc7659c1740 1 Processor -- start 2024-09-10T09:58:35.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.376+0000 7fc758c00640 1 -- 172.21.15.77:0/3623769995 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc74c042070 con 0x7fc75c074510 2024-09-10T09:58:35.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.376+0000 7fc7659c1740 1 -- start start 2024-09-10T09:58:35.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.376+0000 7fc7659c1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557616ac7c70 0x557616ac8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:35.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.376+0000 7fc7659c1740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557616ac8590 0x557616ad2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:35.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.376+0000 7fc7659c1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557616ad2e50 0x557616ad5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:35.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.376+0000 7fc7659c1740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557616a2e3e0 con 0x557616ac8590 2024-09-10T09:58:35.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.376+0000 7fc7659c1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5576169d5d80 con 0x557616ac7c70 2024-09-10T09:58:35.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.376+0000 7fc7659c1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557616a2a3a0 con 0x557616ad2e50 2024-09-10T09:58:35.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.376+0000 7fc75b400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557616ad2e50 0x557616ad5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:35.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.377+0000 7fc75b400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557616ad2e50 0x557616ad5240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:49098/0 (socket says 172.21.15.77:49098) 2024-09-10T09:58:35.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.377+0000 7fc75b400640 1 -- 172.21.15.77:0/1196065543 learned_addr learned my addr 172.21.15.77:0/1196065543 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:58:35.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.377+0000 7fc75be00640 1 --2- 172.21.15.77:0/1196065543 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557616ac8590 0x557616ad2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:35.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.377+0000 7fc761000640 1 --2- 172.21.15.77:0/1196065543 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557616ac7c70 0x557616ac8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:35.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.377+0000 7fc75b400640 1 -- 172.21.15.77:0/1196065543 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557616ac7c70 msgr2=0x557616ac8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:35.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.377+0000 7fc75b400640 1 --2- 172.21.15.77:0/1196065543 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557616ac7c70 0x557616ac8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:35.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.377+0000 7fc75b400640 1 -- 172.21.15.77:0/1196065543 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557616ac8590 msgr2=0x557616ad2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:35.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.377+0000 7fc75b400640 1 --2- 172.21.15.77:0/1196065543 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557616ac8590 0x557616ad2910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:35.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.377+0000 7fc75b400640 1 -- 172.21.15.77:0/1196065543 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576169f4ae0 con 0x557616ad2e50 2024-09-10T09:58:35.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.377+0000 7fc761000640 1 --2- 172.21.15.77:0/1196065543 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557616ac7c70 0x557616ac8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:58:35.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.377+0000 7fc75be00640 1 --2- 172.21.15.77:0/1196065543 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557616ac8590 0x557616ad2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:58:35.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.377+0000 7fc75b400640 1 --2- 172.21.15.77:0/1196065543 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557616ad2e50 0x557616ad5240 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc748009c90 tx=0x7fc748011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=213c701288d2823d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:35.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.377+0000 7fc758c00640 1 --2- 172.21.15.77:0/3623769995 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fc7400a2c00 0x7fc7400a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:35.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.378+0000 7fc751600640 1 -- 172.21.15.77:0/1196065543 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc74801e070 con 0x557616ad2e50 2024-09-10T09:58:35.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.378+0000 7fc758c00640 1 -- 172.21.15.77:0/3623769995 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fc74c0efa80 con 0x7fc75c074510 2024-09-10T09:58:35.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.378+0000 7fc751600640 1 -- 172.21.15.77:0/1196065543 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc748019b10 con 0x557616ad2e50 2024-09-10T09:58:35.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.378+0000 7fc751600640 1 -- 172.21.15.77:0/1196065543 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc74800eac0 con 0x557616ad2e50 2024-09-10T09:58:35.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.378+0000 7fc761000640 1 --2- 172.21.15.77:0/3623769995 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fc7400a2c00 0x7fc7400a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:35.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.378+0000 7fc7659c1740 1 -- 172.21.15.77:0/1196065543 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557616ad2e50 msgr2=0x557616ad5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:35.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.378+0000 7fc7659c1740 1 --2- 172.21.15.77:0/1196065543 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557616ad2e50 0x557616ad5240 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc748009c90 tx=0x7fc748011810 comp rx=0 tx=0).stop 2024-09-10T09:58:35.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.379+0000 7fc7659c1740 1 -- 172.21.15.77:0/1196065543 shutdown_connections 2024-09-10T09:58:35.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.379+0000 7fc7659c1740 1 --2- 172.21.15.77:0/1196065543 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557616ad2e50 0x557616ad5240 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:35.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.379+0000 7fc7659c1740 1 --2- 172.21.15.77:0/1196065543 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557616ac8590 0x557616ad2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:35.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.379+0000 7fc7659c1740 1 --2- 172.21.15.77:0/1196065543 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557616ac7c70 0x557616ac8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:35.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.379+0000 7fc7659c1740 1 -- 172.21.15.77:0/1196065543 >> 172.21.15.77:0/1196065543 conn(0x557616ac4200 msgr2=0x557616ac45e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:35.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.379+0000 7fc761000640 1 --2- 172.21.15.77:0/3623769995 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fc7400a2c00 0x7fc7400a50c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fc75c0754f0 tx=0x7fc754009b00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:35.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.379+0000 7fc7659c1740 1 -- 172.21.15.77:0/1196065543 shutdown_connections 2024-09-10T09:58:35.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.379+0000 7fc7659c1740 1 -- 172.21.15.77:0/1196065543 wait complete. 2024-09-10T09:58:35.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.379+0000 7fc7659c1740 1 -- start start 2024-09-10T09:58:35.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.380+0000 7fc7659c1740 1 Processor -- start 2024-09-10T09:58:35.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.380+0000 7fc7659c1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557616ac7c70 0x557616acfd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:35.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.380+0000 7fc7659c1740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557616ac8590 0x557616ad0250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:35.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.380+0000 7fc7659c1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557616ad2e50 0x557616ad0790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:35.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.380+0000 7fc7659c1740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557616ad5a20 con 0x557616ac8590 2024-09-10T09:58:35.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.380+0000 7fc7659c1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5576169e90f0 con 0x557616ad2e50 2024-09-10T09:58:35.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.380+0000 7fc7659c1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557616ad5ba0 con 0x557616ac7c70 2024-09-10T09:58:35.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.380+0000 7fc7659c1740 10 client.0 _ll_register_callbacks cb 0x5576169d0bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:58:35.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.380+0000 7fc75be00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557616ac7c70 0x557616acfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:35.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.380+0000 7fc75be00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557616ac7c70 0x557616acfd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:49112/0 (socket says 172.21.15.77:49112) 2024-09-10T09:58:35.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.380+0000 7fc75be00640 1 -- 172.21.15.77:0/91185638 learned_addr learned my addr 172.21.15.77:0/91185638 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:58:35.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.380+0000 7fc75b400640 1 --2- 172.21.15.77:0/91185638 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557616ac8590 0x557616ad0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:35.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.380+0000 7fc761000640 1 --2- 172.21.15.77:0/91185638 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557616ad2e50 0x557616ad0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:35.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.381+0000 7fc75be00640 1 -- 172.21.15.77:0/91185638 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557616ad2e50 msgr2=0x557616ad0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:35.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.381+0000 7fc75be00640 1 --2- 172.21.15.77:0/91185638 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557616ad2e50 0x557616ad0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:35.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.381+0000 7fc75be00640 1 -- 172.21.15.77:0/91185638 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557616ac8590 msgr2=0x557616ad0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:35.384 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.381+0000 7fc75be00640 1 --2- 172.21.15.77:0/91185638 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557616ac8590 0x557616ad0250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:35.384 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.381+0000 7fc75be00640 1 -- 172.21.15.77:0/91185638 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55761679f3c0 con 0x557616ac7c70 2024-09-10T09:58:35.384 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.381+0000 7fc75be00640 1 --2- 172.21.15.77:0/91185638 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557616ac7c70 0x557616acfd10 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc74c031550 tx=0x7fc74c02a480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7472b0f359b58094 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:35.384 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.381+0000 7fc746000640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-10T09:58:35.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.381+0000 7fc746000640 1 -- 172.21.15.77:0/91185638 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc74c049050 con 0x557616ac7c70 2024-09-10T09:58:35.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.381+0000 7fc746000640 1 -- 172.21.15.77:0/91185638 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fc74c0b85c0 con 0x557616ac7c70 2024-09-10T09:58:35.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.381+0000 7fc7659c1740 1 -- 172.21.15.77:0/91185638 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc748019b10 con 0x557616ac7c70 2024-09-10T09:58:35.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.382+0000 7fc746000640 1 -- 172.21.15.77:0/91185638 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc74c0bb8b0 con 0x557616ac7c70 2024-09-10T09:58:35.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.382+0000 7fc7659c1740 10 client.26140 fetch_fsmap learned FSMap version 32 2024-09-10T09:58:35.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.382+0000 7fc7659c1740 1 -- 172.21.15.77:0/91185638 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x557616ad5a80 con 0x557616ac7c70 2024-09-10T09:58:35.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.382+0000 7fc746000640 1 -- 172.21.15.77:0/91185638 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc748019b10 con 0x557616ac7c70 2024-09-10T09:58:35.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.382+0000 7fc746000640 1 -- 172.21.15.77:0/91185638 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fc74c03b4f0 con 0x557616ac7c70 2024-09-10T09:58:35.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.382+0000 7fc7659c1740 10 client.26140 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T09:58:35.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.382+0000 7fc7659c1740 10 client.26140 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:58:35.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.382+0000 7fc7659c1740 20 client.26140 populate_metadata read hostname 'smithi077' 2024-09-10T09:58:35.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.382+0000 7fc7659c1740 1 --2- 172.21.15.77:0/91185638 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x557616ae2f30 0x557616ae53b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:35.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.382+0000 7fc7659c1740 4 client.26140 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:58:35.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.382+0000 7fc7659c1740 1 -- 172.21.15.77:0/91185638 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557616a3b950 con 0x557616ae2f30 2024-09-10T09:58:35.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.383+0000 7fc75b400640 1 --2- 172.21.15.77:0/91185638 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x557616ae2f30 0x557616ae53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:35.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.383+0000 7fc75b400640 1 --2- 172.21.15.77:0/91185638 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x557616ae2f30 0x557616ae53b0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38642a630ea5a6c2 server_cookie=3cb8f16a310009df in_seq=0 out_seq=0 2024-09-10T09:58:35.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.383+0000 7fc746000640 10 client.26140 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T09:58:35.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.384+0000 7fc746000640 1 -- 172.21.15.77:0/91185638 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557616a3b950 con 0x557616ae2f30 2024-09-10T09:58:35.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.384+0000 7fc746000640 10 client.26140 handle_command_reply: tid=0 2024-09-10T09:58:35.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.384+0000 7fc7659c1740 1 client.26140 shutdown 2024-09-10T09:58:35.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.384+0000 7fc7659c1740 1 -- 172.21.15.77:0/91185638 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557616ac7c70 msgr2=0x557616acfd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:35.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.384+0000 7fc7659c1740 1 --2- 172.21.15.77:0/91185638 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557616ac7c70 0x557616acfd10 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc74c031550 tx=0x7fc74c02a480 comp rx=0 tx=0).stop 2024-09-10T09:58:35.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.384+0000 7fc7659c1740 1 -- 172.21.15.77:0/91185638 shutdown_connections 2024-09-10T09:58:35.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.384+0000 7fc7659c1740 1 --2- 172.21.15.77:0/91185638 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x557616ae2f30 0x557616ae53b0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:35.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.384+0000 7fc7659c1740 1 --2- 172.21.15.77:0/91185638 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557616ad2e50 0x557616ad0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:35.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.384+0000 7fc7659c1740 1 --2- 172.21.15.77:0/91185638 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557616ac8590 0x557616ad0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:35.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.384+0000 7fc7659c1740 1 --2- 172.21.15.77:0/91185638 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557616ac7c70 0x557616acfd10 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:35.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.384+0000 7fc7659c1740 1 -- 172.21.15.77:0/91185638 >> 172.21.15.77:0/91185638 conn(0x557616ac4200 msgr2=0x557616ac9f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:35.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.384+0000 7fc746000640 0 client.26140 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T09:58:35.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.387+0000 7fc7659c1740 1 -- 172.21.15.77:0/91185638 shutdown_connections 2024-09-10T09:58:35.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.387+0000 7fc7659c1740 1 -- 172.21.15.77:0/91185638 wait complete. 2024-09-10T09:58:35.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.387+0000 7fc7659c1740 20 client.26140 trim_cache size 0 max 16384 2024-09-10T09:58:35.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.401+0000 7fc7659c1740 1 Processor -- start 2024-09-10T09:58:35.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.401+0000 7fc7659c1740 1 -- start start 2024-09-10T09:58:35.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.402+0000 7fc7659c1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557616ac7c70 0x557616ac9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:35.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.402+0000 7fc7659c1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557616ac8590 0x557616aeac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:35.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.402+0000 7fc7659c1740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557616ad2e50 0x557616aeb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:35.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.402+0000 7fc7659c1740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc75c1b10d0 con 0x557616ad2e50 2024-09-10T09:58:35.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.402+0000 7fc7659c1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc75c1b2cd0 con 0x557616ac8590 2024-09-10T09:58:35.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.402+0000 7fc7659c1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc75c1b17a0 con 0x557616ac7c70 2024-09-10T09:58:35.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.402+0000 7fc75be00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557616ac7c70 0x557616ac9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:35.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.402+0000 7fc75be00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557616ac7c70 0x557616ac9e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:49114/0 (socket says 172.21.15.77:49114) 2024-09-10T09:58:35.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.402+0000 7fc75be00640 1 -- 172.21.15.77:0/1818741995 learned_addr learned my addr 172.21.15.77:0/1818741995 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:58:35.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.402+0000 7fc75b400640 1 --2- 172.21.15.77:0/1818741995 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557616ac8590 0x557616aeac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:35.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.402+0000 7fc761000640 1 --2- 172.21.15.77:0/1818741995 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557616ad2e50 0x557616aeb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:35.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.402+0000 7fc75be00640 1 -- 172.21.15.77:0/1818741995 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557616ac8590 msgr2=0x557616aeac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:35.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.402+0000 7fc75be00640 1 --2- 172.21.15.77:0/1818741995 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557616ac8590 0x557616aeac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:35.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.402+0000 7fc75be00640 1 -- 172.21.15.77:0/1818741995 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557616ad2e50 msgr2=0x557616aeb1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:35.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.403+0000 7fc75be00640 1 --2- 172.21.15.77:0/1818741995 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557616ad2e50 0x557616aeb1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:35.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.403+0000 7fc75be00640 1 -- 172.21.15.77:0/1818741995 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557616ac4030 con 0x557616ac7c70 2024-09-10T09:58:35.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.403+0000 7fc75be00640 1 --2- 172.21.15.77:0/1818741995 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557616ac7c70 0x557616ac9e20 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc74c0efe40 tx=0x7fc74c0ba460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=911fa73d25bcef09 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:35.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.403+0000 7fc750c00640 1 -- 172.21.15.77:0/1818741995 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc74c049030 con 0x557616ac7c70 2024-09-10T09:58:35.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.403+0000 7fc750c00640 1 -- 172.21.15.77:0/1818741995 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc74c0bc650 con 0x557616ac7c70 2024-09-10T09:58:35.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.403+0000 7fc750c00640 1 -- 172.21.15.77:0/1818741995 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc74c0d8030 con 0x557616ac7c70 2024-09-10T09:58:35.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.404+0000 7fc7659c1740 1 -- 172.21.15.77:0/1818741995 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557616ac7c70 msgr2=0x557616ac9e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:35.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.404+0000 7fc7659c1740 1 --2- 172.21.15.77:0/1818741995 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557616ac7c70 0x557616ac9e20 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc74c0efe40 tx=0x7fc74c0ba460 comp rx=0 tx=0).stop 2024-09-10T09:58:35.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.404+0000 7fc7659c1740 1 -- 172.21.15.77:0/1818741995 shutdown_connections 2024-09-10T09:58:35.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.404+0000 7fc7659c1740 1 --2- 172.21.15.77:0/1818741995 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557616ad2e50 0x557616aeb1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:35.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.404+0000 7fc7659c1740 1 --2- 172.21.15.77:0/1818741995 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557616ac8590 0x557616aeac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:35.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.404+0000 7fc7659c1740 1 --2- 172.21.15.77:0/1818741995 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557616ac7c70 0x557616ac9e20 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:35.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.404+0000 7fc7659c1740 1 -- 172.21.15.77:0/1818741995 >> 172.21.15.77:0/1818741995 conn(0x557616ac4200 msgr2=0x557616ac9710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:35.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.404+0000 7fc7659c1740 1 -- 172.21.15.77:0/1818741995 shutdown_connections 2024-09-10T09:58:35.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.404+0000 7fc7659c1740 1 -- 172.21.15.77:0/1818741995 wait complete. 2024-09-10T09:58:35.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.404+0000 7fc7659c1740 1 -- start start 2024-09-10T09:58:35.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.405+0000 7fc7659c1740 1 Processor -- start 2024-09-10T09:58:35.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.405+0000 7fc7659c1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557616ac7c70 0x557616acce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:35.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.405+0000 7fc7659c1740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557616ac8590 0x557616acd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:35.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.405+0000 7fc7659c1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557616ad2e50 0x557616acd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:35.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.405+0000 7fc7659c1740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc75c1b2540 con 0x557616ac8590 2024-09-10T09:58:35.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.405+0000 7fc7659c1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc75c1b1e70 con 0x557616ac7c70 2024-09-10T09:58:35.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.405+0000 7fc7659c1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc75c1af0b0 con 0x557616ad2e50 2024-09-10T09:58:35.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.405+0000 7fc7659c1740 10 client.0 _ll_register_callbacks cb 0x7fc74c0f7040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:58:35.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.405+0000 7fc75be00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557616ac7c70 0x557616acce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:35.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.405+0000 7fc75be00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557616ac7c70 0x557616acce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:40378/0 (socket says 172.21.15.77:40378) 2024-09-10T09:58:35.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.405+0000 7fc761000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557616ad2e50 0x557616acd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:35.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.405+0000 7fc75be00640 1 -- 172.21.15.77:0/1691213557 learned_addr learned my addr 172.21.15.77:0/1691213557 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:58:35.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.406+0000 7fc75b400640 1 --2- 172.21.15.77:0/1691213557 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557616ac8590 0x557616acd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:35.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.406+0000 7fc75be00640 1 -- 172.21.15.77:0/1691213557 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557616ad2e50 msgr2=0x557616acd8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:35.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.406+0000 7fc75be00640 1 --2- 172.21.15.77:0/1691213557 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557616ad2e50 0x557616acd8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:35.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.406+0000 7fc75be00640 1 -- 172.21.15.77:0/1691213557 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557616ac8590 msgr2=0x557616acd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:35.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.406+0000 7fc75be00640 1 --2- 172.21.15.77:0/1691213557 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557616ac8590 0x557616acd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:35.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.406+0000 7fc75be00640 1 -- 172.21.15.77:0/1691213557 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557616ac5390 con 0x557616ac7c70 2024-09-10T09:58:35.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.406+0000 7fc75b400640 1 --2- 172.21.15.77:0/1691213557 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557616ac8590 0x557616acd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:58:35.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.406+0000 7fc75be00640 1 --2- 172.21.15.77:0/1691213557 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557616ac7c70 0x557616acce50 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc74c000c00 tx=0x7fc74c0efee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4480a8d223958f91 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:35.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.406+0000 7fc746000640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T09:58:35.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.406+0000 7fc746000640 1 -- 172.21.15.77:0/1691213557 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc74c0d8020 con 0x557616ac7c70 2024-09-10T09:58:35.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.406+0000 7fc746000640 1 -- 172.21.15.77:0/1691213557 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fc74c0bb420 con 0x557616ac7c70 2024-09-10T09:58:35.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.406+0000 7fc7659c1740 1 -- 172.21.15.77:0/1691213557 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc74c0bc650 con 0x557616ac7c70 2024-09-10T09:58:35.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.407+0000 7fc746000640 1 -- 172.21.15.77:0/1691213557 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc74c0b4050 con 0x557616ac7c70 2024-09-10T09:58:35.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.407+0000 7fc7659c1740 10 client.26192 fetch_fsmap learned FSMap version 32 2024-09-10T09:58:35.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.407+0000 7fc7659c1740 1 -- 172.21.15.77:0/1691213557 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x557616abf770 con 0x557616ac7c70 2024-09-10T09:58:35.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.407+0000 7fc746000640 1 -- 172.21.15.77:0/1691213557 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc74c0bc650 con 0x557616ac7c70 2024-09-10T09:58:35.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.407+0000 7fc746000640 1 -- 172.21.15.77:0/1691213557 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fc74c0b7ca0 con 0x557616ac7c70 2024-09-10T09:58:35.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.407+0000 7fc7659c1740 10 client.26192 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T09:58:35.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.407+0000 7fc7659c1740 10 client.26192 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:58:35.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.407+0000 7fc7659c1740 20 client.26192 populate_metadata read hostname 'smithi077' 2024-09-10T09:58:35.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.407+0000 7fc7659c1740 1 --2- 172.21.15.77:0/1691213557 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x557616b4dfe0 0x557616b50400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:35.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.407+0000 7fc7659c1740 4 client.26192 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:58:35.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.407+0000 7fc7659c1740 1 -- 172.21.15.77:0/1691213557 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557616a35e60 con 0x557616b4dfe0 2024-09-10T09:58:35.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.407+0000 7fc75b400640 1 --2- 172.21.15.77:0/1691213557 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x557616b4dfe0 0x557616b50400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:35.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.408+0000 7fc75b400640 1 --2- 172.21.15.77:0/1691213557 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x557616b4dfe0 0x557616b50400 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=629bc74b3804aa12 server_cookie=8cb33594603ad64b in_seq=0 out_seq=0 2024-09-10T09:58:35.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.408+0000 7fc746000640 10 client.26192 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T09:58:35.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.408+0000 7fc746000640 1 -- 172.21.15.77:0/1691213557 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557616a35e60 con 0x557616b4dfe0 2024-09-10T09:58:35.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.408+0000 7fc746000640 10 client.26192 handle_command_reply: tid=0 2024-09-10T09:58:35.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.408+0000 7fc7659c1740 1 client.26192 shutdown 2024-09-10T09:58:35.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.408+0000 7fc7659c1740 1 -- 172.21.15.77:0/1691213557 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557616ac7c70 msgr2=0x557616acce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:35.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.408+0000 7fc7659c1740 1 --2- 172.21.15.77:0/1691213557 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557616ac7c70 0x557616acce50 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc74c000c00 tx=0x7fc74c0efee0 comp rx=0 tx=0).stop 2024-09-10T09:58:35.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.408+0000 7fc7659c1740 1 -- 172.21.15.77:0/1691213557 shutdown_connections 2024-09-10T09:58:35.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.408+0000 7fc7659c1740 1 --2- 172.21.15.77:0/1691213557 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x557616b4dfe0 0x557616b50400 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:35.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.408+0000 7fc7659c1740 1 --2- 172.21.15.77:0/1691213557 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557616ad2e50 0x557616acd8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:35.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.408+0000 7fc7659c1740 1 --2- 172.21.15.77:0/1691213557 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557616ac8590 0x557616acd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:35.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.408+0000 7fc7659c1740 1 --2- 172.21.15.77:0/1691213557 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557616ac7c70 0x557616acce50 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:35.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.408+0000 7fc7659c1740 1 -- 172.21.15.77:0/1691213557 >> 172.21.15.77:0/1691213557 conn(0x557616ac4200 msgr2=0x557616ac3380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:35.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.408+0000 7fc746000640 0 client.26192 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T09:58:35.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.409+0000 7fc7659c1740 1 -- 172.21.15.77:0/1691213557 shutdown_connections 2024-09-10T09:58:35.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.409+0000 7fc7659c1740 1 -- 172.21.15.77:0/1691213557 wait complete. 2024-09-10T09:58:35.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.409+0000 7fc7659c1740 20 client.26192 trim_cache size 0 max 16384 2024-09-10T09:58:35.414 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-10T09:58:35.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.409+0000 7fc763000640 1 -- 172.21.15.77:0/3623769995 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fc7400a2c00 msgr2=0x7fc7400a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:35.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.409+0000 7fc763000640 1 --2- 172.21.15.77:0/3623769995 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fc7400a2c00 0x7fc7400a50c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fc75c0754f0 tx=0x7fc754009b00 comp rx=0 tx=0).stop 2024-09-10T09:58:35.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.409+0000 7fc763000640 1 -- 172.21.15.77:0/3623769995 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc75c074510 msgr2=0x7fc75c19a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:35.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.409+0000 7fc763000640 1 --2- 172.21.15.77:0/3623769995 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc75c074510 0x7fc75c19a710 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc74c004770 tx=0x7fc74c003c10 comp rx=0 tx=0).stop 2024-09-10T09:58:35.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.410+0000 7fc763000640 1 -- 172.21.15.77:0/3623769995 shutdown_connections 2024-09-10T09:58:35.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.410+0000 7fc763000640 1 --2- 172.21.15.77:0/3623769995 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fc7400a2c00 0x7fc7400a50c0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:35.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.410+0000 7fc763000640 1 --2- 172.21.15.77:0/3623769995 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc75c10e450 0x7fc75c19efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:35.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.410+0000 7fc763000640 1 --2- 172.21.15.77:0/3623769995 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc75c074e30 0x7fc75c19ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:35.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.410+0000 7fc763000640 1 --2- 172.21.15.77:0/3623769995 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc75c074510 0x7fc75c19a710 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:35.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.410+0000 7fc763000640 1 -- 172.21.15.77:0/3623769995 >> 172.21.15.77:0/3623769995 conn(0x7fc75c0fd8b0 msgr2=0x7fc75c0fe550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:35.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.410+0000 7fc763000640 1 -- 172.21.15.77:0/3623769995 shutdown_connections 2024-09-10T09:58:35.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:35.410+0000 7fc763000640 1 -- 172.21.15.77:0/3623769995 wait complete. 2024-09-10T09:58:37.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:58:36 smithi134 ceph-mon[35674]: pgmap v494: 129 pgs: 129 active+clean; 77 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 763 KiB/s rd, 1.0 MiB/s wr, 183 op/s 2024-09-10T09:58:37.168 INFO:tasks.workunit.client.0.smithi077.stdout:copying to largest ever: 0xff038 2024-09-10T09:58:37.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:58:36 smithi120 ceph-mon[34770]: pgmap v494: 129 pgs: 129 active+clean; 77 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 763 KiB/s rd, 1.0 MiB/s wr, 183 op/s 2024-09-10T09:58:37.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:58:36 smithi077 ceph-mon[30319]: pgmap v494: 129 pgs: 129 active+clean; 77 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 763 KiB/s rd, 1.0 MiB/s wr, 183 op/s 2024-09-10T09:58:37.274 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T09:58:37.274 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T09:58:37.666 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:58:38.250 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.249+0000 7f5d08800640 1 -- 172.21.15.77:0/50101946 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d00104010 msgr2=0x7f5d001043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:38.250 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.249+0000 7f5d08800640 1 --2- 172.21.15.77:0/50101946 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d00104010 0x7f5d001043f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f5cf4009900 tx=0x7f5cf402a3f0 comp rx=0 tx=0).stop 2024-09-10T09:58:38.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.249+0000 7f5d08800640 1 -- 172.21.15.77:0/50101946 shutdown_connections 2024-09-10T09:58:38.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.249+0000 7f5d08800640 1 --2- 172.21.15.77:0/50101946 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d00109140 0x7f5d00110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:38.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.249+0000 7f5d08800640 1 --2- 172.21.15.77:0/50101946 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d001049c0 0x7f5d00108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:38.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.249+0000 7f5d08800640 1 --2- 172.21.15.77:0/50101946 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d00104010 0x7f5d001043f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:38.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.249+0000 7f5d08800640 1 -- 172.21.15.77:0/50101946 >> 172.21.15.77:0/50101946 conn(0x7f5d000ffa50 msgr2=0x7f5d00101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:38.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.249+0000 7f5d08800640 1 -- 172.21.15.77:0/50101946 shutdown_connections 2024-09-10T09:58:38.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.249+0000 7f5d08800640 1 -- 172.21.15.77:0/50101946 wait complete. 2024-09-10T09:58:38.253 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.250+0000 7f5d08800640 1 Processor -- start 2024-09-10T09:58:38.253 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.250+0000 7f5d08800640 1 -- start start 2024-09-10T09:58:38.253 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.251+0000 7f5d08800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d00104010 0x7f5d0019a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:38.253 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.251+0000 7f5d08800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d001049c0 0x7f5d0019ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:38.253 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.251+0000 7f5d08800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d00109140 0x7f5d0019ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:38.253 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.251+0000 7f5d08800640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5d00114140 con 0x7f5d001049c0 2024-09-10T09:58:38.253 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.251+0000 7f5d08800640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5d00113fc0 con 0x7f5d00104010 2024-09-10T09:58:38.253 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.251+0000 7f5d08800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5d001142c0 con 0x7f5d00109140 2024-09-10T09:58:38.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.251+0000 7f5d05e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d00104010 0x7f5d0019a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:38.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.251+0000 7f5d05e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d00104010 0x7f5d0019a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:40382/0 (socket says 172.21.15.77:40382) 2024-09-10T09:58:38.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.251+0000 7f5d05e00640 1 -- 172.21.15.77:0/3256221358 learned_addr learned my addr 172.21.15.77:0/3256221358 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:58:38.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.251+0000 7f5d06800640 1 --2- 172.21.15.77:0/3256221358 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d00109140 0x7f5d0019ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:38.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.251+0000 7f5d05400640 1 --2- 172.21.15.77:0/3256221358 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d001049c0 0x7f5d0019ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:38.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.251+0000 7f5d05e00640 1 -- 172.21.15.77:0/3256221358 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d00109140 msgr2=0x7f5d0019ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:38.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.251+0000 7f5d05e00640 1 --2- 172.21.15.77:0/3256221358 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d00109140 0x7f5d0019ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:38.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.252+0000 7f5d05e00640 1 -- 172.21.15.77:0/3256221358 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d001049c0 msgr2=0x7f5d0019ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:38.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.252+0000 7f5d05e00640 1 --2- 172.21.15.77:0/3256221358 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d001049c0 0x7f5d0019ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:38.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.252+0000 7f5d05e00640 1 -- 172.21.15.77:0/3256221358 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d0019f5f0 con 0x7f5d00104010 2024-09-10T09:58:38.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.252+0000 7f5d05e00640 1 --2- 172.21.15.77:0/3256221358 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d00104010 0x7f5d0019a640 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5cf4006010 tx=0x7f5cf4004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:38.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.252+0000 7f5cfaa00640 1 -- 172.21.15.77:0/3256221358 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf403f030 con 0x7f5d00104010 2024-09-10T09:58:38.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.252+0000 7f5cfaa00640 1 -- 172.21.15.77:0/3256221358 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cf4004440 con 0x7f5d00104010 2024-09-10T09:58:38.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.252+0000 7f5cfaa00640 1 -- 172.21.15.77:0/3256221358 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf4035070 con 0x7f5d00104010 2024-09-10T09:58:38.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.252+0000 7f5d08800640 1 -- 172.21.15.77:0/3256221358 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d0019f880 con 0x7f5d00104010 2024-09-10T09:58:38.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.252+0000 7f5d08800640 1 -- 172.21.15.77:0/3256221358 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d0019fd10 con 0x7f5d00104010 2024-09-10T09:58:38.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.255+0000 7f5d0b444740 1 Processor -- start 2024-09-10T09:58:38.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.255+0000 7f5cfaa00640 1 -- 172.21.15.77:0/3256221358 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5cf4042030 con 0x7f5d00104010 2024-09-10T09:58:38.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.255+0000 7f5d0b444740 1 -- start start 2024-09-10T09:58:38.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.256+0000 7f5d0b444740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559081fc2ac0 0x559081fc2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:38.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.256+0000 7f5d0b444740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559081fc33e0 0x559081fcd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:38.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.256+0000 7f5d0b444740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559081fcdca0 0x559081fd0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:38.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.256+0000 7f5d0b444740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x559081f29290 con 0x559081fcdca0 2024-09-10T09:58:38.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.256+0000 7f5d0b444740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x559081ed0b40 con 0x559081fc33e0 2024-09-10T09:58:38.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.256+0000 7f5d0b444740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559081f25250 con 0x559081fc2ac0 2024-09-10T09:58:38.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.256+0000 7f5d05e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559081fc33e0 0x559081fcd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:38.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.256+0000 7f5d06800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559081fc2ac0 0x559081fc2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:38.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.256+0000 7f5d06800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559081fc2ac0 0x559081fc2ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:49134/0 (socket says 172.21.15.77:49134) 2024-09-10T09:58:38.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.256+0000 7f5d06800640 1 -- 172.21.15.77:0/3628216205 learned_addr learned my addr 172.21.15.77:0/3628216205 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:58:38.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.257+0000 7f5d05400640 1 --2- 172.21.15.77:0/3628216205 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559081fcdca0 0x559081fd0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:38.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.257+0000 7f5d05e00640 1 -- 172.21.15.77:0/3628216205 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559081fc2ac0 msgr2=0x559081fc2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:38.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.257+0000 7f5d05e00640 1 --2- 172.21.15.77:0/3628216205 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559081fc2ac0 0x559081fc2ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:38.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.257+0000 7f5d05e00640 1 -- 172.21.15.77:0/3628216205 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559081fcdca0 msgr2=0x559081fd0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:38.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.257+0000 7f5d05e00640 1 --2- 172.21.15.77:0/3628216205 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559081fcdca0 0x559081fd0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:38.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.257+0000 7f5d05e00640 1 -- 172.21.15.77:0/3628216205 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559081c9a6a0 con 0x559081fc33e0 2024-09-10T09:58:38.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.257+0000 7f5d06800640 1 --2- 172.21.15.77:0/3628216205 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559081fc2ac0 0x559081fc2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:58:38.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.257+0000 7f5d05400640 1 --2- 172.21.15.77:0/3628216205 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559081fcdca0 0x559081fd0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:58:38.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.257+0000 7f5d05e00640 1 --2- 172.21.15.77:0/3628216205 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559081fc33e0 0x559081fcd760 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5cf40ecda0 tx=0x7f5cf40f5cb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eec1f1d6c2d4728b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:38.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.257+0000 7f5cfaa00640 1 --2- 172.21.15.77:0/3256221358 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f5ce80a2c00 0x7f5ce80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:38.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.257+0000 7f5cef000640 1 -- 172.21.15.77:0/3628216205 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf40fe070 con 0x559081fc33e0 2024-09-10T09:58:38.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.257+0000 7f5cef000640 1 -- 172.21.15.77:0/3628216205 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cf40bbc10 con 0x559081fc33e0 2024-09-10T09:58:38.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.257+0000 7f5cfaa00640 1 -- 172.21.15.77:0/3256221358 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f5cf40ef750 con 0x7f5d00104010 2024-09-10T09:58:38.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.257+0000 7f5cef000640 1 -- 172.21.15.77:0/3628216205 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf40bb9e0 con 0x559081fc33e0 2024-09-10T09:58:38.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.258+0000 7f5d06800640 1 --2- 172.21.15.77:0/3256221358 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f5ce80a2c00 0x7f5ce80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:38.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.258+0000 7f5d0b444740 1 -- 172.21.15.77:0/3628216205 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559081fc33e0 msgr2=0x559081fcd760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:38.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.258+0000 7f5d0b444740 1 --2- 172.21.15.77:0/3628216205 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559081fc33e0 0x559081fcd760 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5cf40ecda0 tx=0x7f5cf40f5cb0 comp rx=0 tx=0).stop 2024-09-10T09:58:38.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.258+0000 7f5d0b444740 1 -- 172.21.15.77:0/3628216205 shutdown_connections 2024-09-10T09:58:38.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.258+0000 7f5d0b444740 1 --2- 172.21.15.77:0/3628216205 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559081fcdca0 0x559081fd0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:38.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.258+0000 7f5d0b444740 1 --2- 172.21.15.77:0/3628216205 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559081fc33e0 0x559081fcd760 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:38.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.258+0000 7f5d0b444740 1 --2- 172.21.15.77:0/3628216205 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559081fc2ac0 0x559081fc2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:38.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.258+0000 7f5d06800640 1 --2- 172.21.15.77:0/3256221358 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f5ce80a2c00 0x7f5ce80a50c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc009fd0 tx=0x7f5cfc009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:38.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.258+0000 7f5d0b444740 1 -- 172.21.15.77:0/3628216205 >> 172.21.15.77:0/3628216205 conn(0x559081fbf050 msgr2=0x559081fbf430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:38.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.258+0000 7f5d0b444740 1 -- 172.21.15.77:0/3628216205 shutdown_connections 2024-09-10T09:58:38.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.258+0000 7f5d0b444740 1 -- 172.21.15.77:0/3628216205 wait complete. 2024-09-10T09:58:38.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.259+0000 7f5d0b444740 1 -- start start 2024-09-10T09:58:38.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.259+0000 7f5d0b444740 1 Processor -- start 2024-09-10T09:58:38.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.260+0000 7f5d0b444740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559081fc2ac0 0x559081fcd510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:38.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.260+0000 7f5d0b444740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559081fc33e0 0x559081fca7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:38.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.260+0000 7f5d0b444740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559081fcdca0 0x559081fcad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:38.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.260+0000 7f5d0b444740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x559081fd0630 con 0x559081fc33e0 2024-09-10T09:58:38.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.260+0000 7f5d0b444740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x559081ee3fa0 con 0x559081fcdca0 2024-09-10T09:58:38.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.260+0000 7f5d0b444740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559081fd07b0 con 0x559081fc2ac0 2024-09-10T09:58:38.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.260+0000 7f5d0b444740 10 client.0 _ll_register_callbacks cb 0x559081c37ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:58:38.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.260+0000 7f5d05e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559081fc2ac0 0x559081fcd510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:38.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.260+0000 7f5d05e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559081fc2ac0 0x559081fcd510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:49150/0 (socket says 172.21.15.77:49150) 2024-09-10T09:58:38.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.260+0000 7f5d05e00640 1 -- 172.21.15.77:0/2803432011 learned_addr learned my addr 172.21.15.77:0/2803432011 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:58:38.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.260+0000 7f5d05400640 1 --2- 172.21.15.77:0/2803432011 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559081fc33e0 0x559081fca7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:38.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.260+0000 7f5d06800640 1 --2- 172.21.15.77:0/2803432011 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559081fcdca0 0x559081fcad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:38.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.260+0000 7f5d06800640 1 -- 172.21.15.77:0/2803432011 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559081fc2ac0 msgr2=0x559081fcd510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:38.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.260+0000 7f5d06800640 1 --2- 172.21.15.77:0/2803432011 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559081fc2ac0 0x559081fcd510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:38.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.260+0000 7f5d06800640 1 -- 172.21.15.77:0/2803432011 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559081fc33e0 msgr2=0x559081fca7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:38.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.260+0000 7f5d06800640 1 --2- 172.21.15.77:0/2803432011 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559081fc33e0 0x559081fca7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:38.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.260+0000 7f5d06800640 1 -- 172.21.15.77:0/2803432011 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559081fcb530 con 0x559081fcdca0 2024-09-10T09:58:38.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.260+0000 7f5d05400640 1 --2- 172.21.15.77:0/2803432011 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559081fc33e0 0x559081fca7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:58:38.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.261+0000 7f5d06800640 1 --2- 172.21.15.77:0/2803432011 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559081fcdca0 0x559081fcad00 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc009fa0 tx=0x7f5cfc004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c326bec73cd65330 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:38.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.261+0000 7f5ce3400640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T09:58:38.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.261+0000 7f5ce3400640 1 -- 172.21.15.77:0/2803432011 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cfc004b20 con 0x559081fcdca0 2024-09-10T09:58:38.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.261+0000 7f5d0b444740 1 -- 172.21.15.77:0/2803432011 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5cf40bbc10 con 0x559081fcdca0 2024-09-10T09:58:38.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.261+0000 7f5ce3400640 1 -- 172.21.15.77:0/2803432011 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f5cfc017030 con 0x559081fcdca0 2024-09-10T09:58:38.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.261+0000 7f5ce3400640 1 -- 172.21.15.77:0/2803432011 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f5cfc004cc0 con 0x559081fcdca0 2024-09-10T09:58:38.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.261+0000 7f5d0b444740 10 client.26204 fetch_fsmap learned FSMap version 32 2024-09-10T09:58:38.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.261+0000 7f5d0b444740 1 -- 172.21.15.77:0/2803432011 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559081fdc4f0 con 0x559081fcdca0 2024-09-10T09:58:38.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.262+0000 7f5d05e00640 1 --2- 172.21.15.77:0/2803432011 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559081fc2ac0 0x559081fcd510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:58:38.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.262+0000 7f5ce3400640 1 -- 172.21.15.77:0/2803432011 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cf40bbc10 con 0x559081fcdca0 2024-09-10T09:58:38.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.262+0000 7f5ce3400640 1 -- 172.21.15.77:0/2803432011 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f5cfc01d5a0 con 0x559081fcdca0 2024-09-10T09:58:38.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.262+0000 7f5d0b444740 10 client.26204 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T09:58:38.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.262+0000 7f5d0b444740 10 client.26204 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:58:38.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.262+0000 7f5d0b444740 20 client.26204 populate_metadata read hostname 'smithi077' 2024-09-10T09:58:38.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.262+0000 7f5d0b444740 1 --2- 172.21.15.77:0/2803432011 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x559081fdd9d0 0x559081fdfe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:38.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.262+0000 7f5d0b444740 4 client.26204 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:58:38.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.262+0000 7f5d0b444740 1 -- 172.21.15.77:0/2803432011 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559081f36800 con 0x559081fdd9d0 2024-09-10T09:58:38.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.262+0000 7f5d05e00640 1 --2- 172.21.15.77:0/2803432011 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x559081fdd9d0 0x559081fdfe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:38.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.263+0000 7f5d05e00640 1 --2- 172.21.15.77:0/2803432011 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x559081fdd9d0 0x559081fdfe50 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ee981f26d011625 server_cookie=e5e2dc398a48d310 in_seq=0 out_seq=0 2024-09-10T09:58:38.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.263+0000 7f5ce3400640 10 client.26204 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T09:58:38.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.264+0000 7f5ce3400640 1 -- 172.21.15.77:0/2803432011 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559081f36800 con 0x559081fdd9d0 2024-09-10T09:58:38.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.264+0000 7f5ce3400640 10 client.26204 handle_command_reply: tid=0 2024-09-10T09:58:38.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.264+0000 7f5d0b444740 1 client.26204 shutdown 2024-09-10T09:58:38.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.264+0000 7f5d0b444740 1 -- 172.21.15.77:0/2803432011 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559081fcdca0 msgr2=0x559081fcad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:38.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.264+0000 7f5d0b444740 1 --2- 172.21.15.77:0/2803432011 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559081fcdca0 0x559081fcad00 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc009fa0 tx=0x7f5cfc004ab0 comp rx=0 tx=0).stop 2024-09-10T09:58:38.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.264+0000 7f5d0b444740 1 -- 172.21.15.77:0/2803432011 shutdown_connections 2024-09-10T09:58:38.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.264+0000 7f5d0b444740 1 --2- 172.21.15.77:0/2803432011 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x559081fdd9d0 0x559081fdfe50 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:38.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.264+0000 7f5d0b444740 1 --2- 172.21.15.77:0/2803432011 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559081fcdca0 0x559081fcad00 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:38.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.264+0000 7f5d0b444740 1 --2- 172.21.15.77:0/2803432011 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559081fc33e0 0x559081fca7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:38.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.264+0000 7f5ce3400640 0 client.26204 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T09:58:38.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.264+0000 7f5d0b444740 1 --2- 172.21.15.77:0/2803432011 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559081fc2ac0 0x559081fcd510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:38.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.264+0000 7f5d0b444740 1 -- 172.21.15.77:0/2803432011 >> 172.21.15.77:0/2803432011 conn(0x559081fbf050 msgr2=0x559081fc4d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:38.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.265+0000 7f5d0b444740 1 -- 172.21.15.77:0/2803432011 shutdown_connections 2024-09-10T09:58:38.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.265+0000 7f5d0b444740 1 -- 172.21.15.77:0/2803432011 wait complete. 2024-09-10T09:58:38.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.265+0000 7f5d0b444740 20 client.26204 trim_cache size 0 max 16384 2024-09-10T09:58:38.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.283+0000 7f5d0b444740 1 Processor -- start 2024-09-10T09:58:38.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.283+0000 7f5d0b444740 1 -- start start 2024-09-10T09:58:38.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.283+0000 7f5d0b444740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559081fc2ac0 0x559081fc5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:38.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.283+0000 7f5d0b444740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559081fc33e0 0x559081fc97a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:38.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.283+0000 7f5d0b444740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559081fcdca0 0x559081fe5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:38.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.283+0000 7f5d0b444740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5d001b1090 con 0x559081fc2ac0 2024-09-10T09:58:38.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.283+0000 7f5d0b444740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5d001b2c90 con 0x559081fcdca0 2024-09-10T09:58:38.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.283+0000 7f5d0b444740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5d001b1760 con 0x559081fc33e0 2024-09-10T09:58:38.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.284+0000 7f5d05e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559081fc2ac0 0x559081fc5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:38.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.284+0000 7f5d06800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559081fcdca0 0x559081fe5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:38.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.284+0000 7f5d06800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559081fcdca0 0x559081fe5720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:40394/0 (socket says 172.21.15.77:40394) 2024-09-10T09:58:38.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.284+0000 7f5d06800640 1 -- 172.21.15.77:0/1519940659 learned_addr learned my addr 172.21.15.77:0/1519940659 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:58:38.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.284+0000 7f5d05400640 1 --2- 172.21.15.77:0/1519940659 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559081fc33e0 0x559081fc97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:38.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.284+0000 7f5d06800640 1 -- 172.21.15.77:0/1519940659 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559081fc33e0 msgr2=0x559081fc97a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:38.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.284+0000 7f5d06800640 1 --2- 172.21.15.77:0/1519940659 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559081fc33e0 0x559081fc97a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:38.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.284+0000 7f5d06800640 1 -- 172.21.15.77:0/1519940659 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559081fc2ac0 msgr2=0x559081fc5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:38.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.284+0000 7f5d06800640 1 --2- 172.21.15.77:0/1519940659 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559081fc2ac0 0x559081fc5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:38.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.284+0000 7f5d06800640 1 -- 172.21.15.77:0/1519940659 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559081fbee80 con 0x559081fcdca0 2024-09-10T09:58:38.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.284+0000 7f5d05e00640 1 --2- 172.21.15.77:0/1519940659 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559081fc2ac0 0x559081fc5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:58:38.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.284+0000 7f5d06800640 1 --2- 172.21.15.77:0/1519940659 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559081fcdca0 0x559081fe5720 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc00fa00 tx=0x7f5cfc005f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e841b475c25a0ad6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:38.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.285+0000 7f5cee600640 1 -- 172.21.15.77:0/1519940659 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cfc011e90 con 0x559081fcdca0 2024-09-10T09:58:38.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.285+0000 7f5cee600640 1 -- 172.21.15.77:0/1519940659 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cfc003e50 con 0x559081fcdca0 2024-09-10T09:58:38.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.285+0000 7f5cee600640 1 -- 172.21.15.77:0/1519940659 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cfc0044f0 con 0x559081fcdca0 2024-09-10T09:58:38.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.285+0000 7f5d0b444740 1 -- 172.21.15.77:0/1519940659 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559081fcdca0 msgr2=0x559081fe5720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:38.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.285+0000 7f5d0b444740 1 --2- 172.21.15.77:0/1519940659 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559081fcdca0 0x559081fe5720 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc00fa00 tx=0x7f5cfc005f70 comp rx=0 tx=0).stop 2024-09-10T09:58:38.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.286+0000 7f5d0b444740 1 -- 172.21.15.77:0/1519940659 shutdown_connections 2024-09-10T09:58:38.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.286+0000 7f5d0b444740 1 --2- 172.21.15.77:0/1519940659 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559081fcdca0 0x559081fe5720 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:38.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.286+0000 7f5d0b444740 1 --2- 172.21.15.77:0/1519940659 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559081fc33e0 0x559081fc97a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:38.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.286+0000 7f5d0b444740 1 --2- 172.21.15.77:0/1519940659 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559081fc2ac0 0x559081fc5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:38.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.286+0000 7f5d0b444740 1 -- 172.21.15.77:0/1519940659 >> 172.21.15.77:0/1519940659 conn(0x559081fbf050 msgr2=0x559081fc9090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:38.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.286+0000 7f5d0b444740 1 -- 172.21.15.77:0/1519940659 shutdown_connections 2024-09-10T09:58:38.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.286+0000 7f5d0b444740 1 -- 172.21.15.77:0/1519940659 wait complete. 2024-09-10T09:58:38.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.286+0000 7f5d0b444740 1 -- start start 2024-09-10T09:58:38.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.286+0000 7f5d0b444740 1 Processor -- start 2024-09-10T09:58:38.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.287+0000 7f5d0b444740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559081fc2ac0 0x559081fd6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:38.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.287+0000 7f5d0b444740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559081fc33e0 0x559081fce9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:38.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.287+0000 7f5d0b444740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559081fcdca0 0x559081fcef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:38.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.287+0000 7f5d0b444740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5d001b2500 con 0x559081fc33e0 2024-09-10T09:58:38.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.287+0000 7f5d0b444740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5d001b1e30 con 0x559081fc2ac0 2024-09-10T09:58:38.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.287+0000 7f5d0b444740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5d001af070 con 0x559081fcdca0 2024-09-10T09:58:38.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.287+0000 7f5d0b444740 10 client.0 _ll_register_callbacks cb 0x7f5cfc011de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:58:38.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.287+0000 7f5d05e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559081fc2ac0 0x559081fd6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:38.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.287+0000 7f5d05400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559081fc33e0 0x559081fce9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:38.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.287+0000 7f5d05e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559081fc2ac0 0x559081fd6010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:40410/0 (socket says 172.21.15.77:40410) 2024-09-10T09:58:38.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.287+0000 7f5d05e00640 1 -- 172.21.15.77:0/2253499741 learned_addr learned my addr 172.21.15.77:0/2253499741 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:58:38.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.287+0000 7f5d06800640 1 --2- 172.21.15.77:0/2253499741 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559081fcdca0 0x559081fcef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:38.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.287+0000 7f5d05e00640 1 -- 172.21.15.77:0/2253499741 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559081fcdca0 msgr2=0x559081fcef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:38.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.287+0000 7f5d05e00640 1 --2- 172.21.15.77:0/2253499741 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559081fcdca0 0x559081fcef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:38.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.287+0000 7f5d05e00640 1 -- 172.21.15.77:0/2253499741 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559081fc33e0 msgr2=0x559081fce9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:38.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.287+0000 7f5d05e00640 1 --2- 172.21.15.77:0/2253499741 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559081fc33e0 0x559081fce9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:38.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.287+0000 7f5d05e00640 1 -- 172.21.15.77:0/2253499741 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559081fcf610 con 0x559081fc2ac0 2024-09-10T09:58:38.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.287+0000 7f5d06800640 1 --2- 172.21.15.77:0/2253499741 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559081fcdca0 0x559081fcef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:58:38.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.288+0000 7f5d05e00640 1 --2- 172.21.15.77:0/2253499741 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559081fc2ac0 0x559081fd6010 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5cf4000c00 tx=0x7f5cf40ef9d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e26dfaa6bb116206 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:38.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.288+0000 7f5ce3400640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T09:58:38.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.288+0000 7f5ce3400640 1 -- 172.21.15.77:0/2253499741 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf4049040 con 0x559081fc2ac0 2024-09-10T09:58:38.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.288+0000 7f5d0b444740 1 -- 172.21.15.77:0/2253499741 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5cfc003e50 con 0x559081fc2ac0 2024-09-10T09:58:38.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.288+0000 7f5ce3400640 1 -- 172.21.15.77:0/2253499741 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f5cf40efd50 con 0x559081fc2ac0 2024-09-10T09:58:38.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.288+0000 7f5ce3400640 1 -- 172.21.15.77:0/2253499741 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f5cf40fe040 con 0x559081fc2ac0 2024-09-10T09:58:38.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.288+0000 7f5d0b444740 10 client.26210 fetch_fsmap learned FSMap version 32 2024-09-10T09:58:38.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.288+0000 7f5d0b444740 1 -- 172.21.15.77:0/2253499741 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559082050f00 con 0x559081fc2ac0 2024-09-10T09:58:38.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.288+0000 7f5ce3400640 1 -- 172.21.15.77:0/2253499741 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cfc003e50 con 0x559081fc2ac0 2024-09-10T09:58:38.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.288+0000 7f5ce3400640 1 -- 172.21.15.77:0/2253499741 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f5cf40ef3d0 con 0x559081fc2ac0 2024-09-10T09:58:38.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.289+0000 7f5d0b444740 10 client.26210 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T09:58:38.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.289+0000 7f5d0b444740 10 client.26210 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:58:38.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.289+0000 7f5d0b444740 20 client.26210 populate_metadata read hostname 'smithi077' 2024-09-10T09:58:38.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.289+0000 7f5d0b444740 1 --2- 172.21.15.77:0/2253499741 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x559082051240 0x559082053630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:38.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.289+0000 7f5d0b444740 4 client.26210 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T09:58:38.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.289+0000 7f5d0b444740 1 -- 172.21.15.77:0/2253499741 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559081f30d10 con 0x559082051240 2024-09-10T09:58:38.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.289+0000 7f5d05400640 1 --2- 172.21.15.77:0/2253499741 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x559082051240 0x559082053630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:38.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.289+0000 7f5d05400640 1 --2- 172.21.15.77:0/2253499741 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x559082051240 0x559082053630 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e86776339f6845e9 server_cookie=72db06bf47c8c3c in_seq=0 out_seq=0 2024-09-10T09:58:38.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.289+0000 7f5ce3400640 10 client.26210 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T09:58:38.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.290+0000 7f5ce3400640 1 -- 172.21.15.77:0/2253499741 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559081f30d10 con 0x559082051240 2024-09-10T09:58:38.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.290+0000 7f5ce3400640 10 client.26210 handle_command_reply: tid=0 2024-09-10T09:58:38.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.290+0000 7f5d0b444740 1 client.26210 shutdown 2024-09-10T09:58:38.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.291+0000 7f5d0b444740 1 -- 172.21.15.77:0/2253499741 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559081fc2ac0 msgr2=0x559081fd6010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:38.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.291+0000 7f5d0b444740 1 --2- 172.21.15.77:0/2253499741 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559081fc2ac0 0x559081fd6010 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5cf4000c00 tx=0x7f5cf40ef9d0 comp rx=0 tx=0).stop 2024-09-10T09:58:38.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.291+0000 7f5d0b444740 1 -- 172.21.15.77:0/2253499741 shutdown_connections 2024-09-10T09:58:38.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.291+0000 7f5d0b444740 1 --2- 172.21.15.77:0/2253499741 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x559082051240 0x559082053630 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:38.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.291+0000 7f5d0b444740 1 --2- 172.21.15.77:0/2253499741 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559081fcdca0 0x559081fcef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:38.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.291+0000 7f5d0b444740 1 --2- 172.21.15.77:0/2253499741 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559081fc33e0 0x559081fce9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:38.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.291+0000 7f5d0b444740 1 --2- 172.21.15.77:0/2253499741 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559081fc2ac0 0x559081fd6010 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:38.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.291+0000 7f5d0b444740 1 -- 172.21.15.77:0/2253499741 >> 172.21.15.77:0/2253499741 conn(0x559081fbf050 msgr2=0x559081fc98b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:38.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.291+0000 7f5ce3400640 0 client.26210 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T09:58:38.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.291+0000 7f5d0b444740 1 -- 172.21.15.77:0/2253499741 shutdown_connections 2024-09-10T09:58:38.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.291+0000 7f5d0b444740 1 -- 172.21.15.77:0/2253499741 wait complete. 2024-09-10T09:58:38.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.291+0000 7f5d0b444740 20 client.26210 trim_cache size 0 max 16384 2024-09-10T09:58:38.296 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-10T09:58:38.297 INFO:teuthology.orchestra.run.smithi077.stdout: "return_code": 0, 2024-09-10T09:58:38.297 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_tag": "33d5f7f6-c954-474b-9564-0de6b5cc9114", 2024-09-10T09:58:38.297 INFO:teuthology.orchestra.run.smithi077.stdout: "mode": "asynchronous" 2024-09-10T09:58:38.297 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-10T09:58:38.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.292+0000 7f5d08800640 1 -- 172.21.15.77:0/3256221358 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f5ce80a2c00 msgr2=0x7f5ce80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:38.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.292+0000 7f5d08800640 1 --2- 172.21.15.77:0/3256221358 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f5ce80a2c00 0x7f5ce80a50c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc009fd0 tx=0x7f5cfc009500 comp rx=0 tx=0).stop 2024-09-10T09:58:38.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.292+0000 7f5d08800640 1 -- 172.21.15.77:0/3256221358 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d00104010 msgr2=0x7f5d0019a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:38.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.292+0000 7f5d08800640 1 --2- 172.21.15.77:0/3256221358 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d00104010 0x7f5d0019a640 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5cf4006010 tx=0x7f5cf4004040 comp rx=0 tx=0).stop 2024-09-10T09:58:38.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.292+0000 7f5d08800640 1 -- 172.21.15.77:0/3256221358 shutdown_connections 2024-09-10T09:58:38.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.292+0000 7f5d08800640 1 --2- 172.21.15.77:0/3256221358 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f5ce80a2c00 0x7f5ce80a50c0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:38.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.292+0000 7f5d08800640 1 --2- 172.21.15.77:0/3256221358 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d00109140 0x7f5d0019ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:38.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.292+0000 7f5d08800640 1 --2- 172.21.15.77:0/3256221358 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d001049c0 0x7f5d0019ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:38.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.292+0000 7f5d08800640 1 --2- 172.21.15.77:0/3256221358 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d00104010 0x7f5d0019a640 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:38.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.292+0000 7f5d08800640 1 -- 172.21.15.77:0/3256221358 >> 172.21.15.77:0/3256221358 conn(0x7f5d000ffa50 msgr2=0x7f5d00100df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:38.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.292+0000 7f5d08800640 1 -- 172.21.15.77:0/3256221358 shutdown_connections 2024-09-10T09:58:38.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:38.292+0000 7f5d08800640 1 -- 172.21.15.77:0/3256221358 wait complete. 2024-09-10T09:58:38.552 INFO:tasks.workunit.client.0.smithi077.stdout:truncating to largest ever: 0xffcac 2024-09-10T09:58:38.724 INFO:tasks.workunit.client.0.smithi077.stdout:10000 copy from 0x2d237 to 0x30552, (0x331b bytes) at 0x82aed 2024-09-10T09:58:39.120 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:58:38 smithi077 ceph-mon[30319]: pgmap v495: 129 pgs: 129 active+clean; 77 MiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 759 KiB/s rd, 1.0 MiB/s wr, 183 op/s 2024-09-10T09:58:39.120 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:58:38 smithi077 ceph-mon[30319]: scrub queued for path: / 2024-09-10T09:58:39.120 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:58:38 smithi077 ceph-mon[30319]: scrub summary: idle+waiting paths [/] 2024-09-10T09:58:39.120 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:58:38 smithi077 ceph-mon[30319]: scrub summary: active paths [/] 2024-09-10T09:58:39.120 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:58:38 smithi077 ceph-mon[30319]: scrub summary: active paths [/] 2024-09-10T09:58:39.120 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:58:38 smithi077 ceph-mon[30319]: scrub summary: active paths [/] 2024-09-10T09:58:39.121 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:58:38 smithi077 ceph-mon[30319]: scrub summary: active paths [/] 2024-09-10T09:58:39.121 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:58:38 smithi077 ceph-mon[30319]: scrub summary: idle+waiting paths [/] 2024-09-10T09:58:39.121 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:58:38 smithi077 ceph-mon[30319]: scrub summary: idle+waiting paths [/] 2024-09-10T09:58:39.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:58:38 smithi134 ceph-mon[35674]: pgmap v495: 129 pgs: 129 active+clean; 77 MiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 759 KiB/s rd, 1.0 MiB/s wr, 183 op/s 2024-09-10T09:58:39.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:58:38 smithi134 ceph-mon[35674]: scrub queued for path: / 2024-09-10T09:58:39.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:58:38 smithi134 ceph-mon[35674]: scrub summary: idle+waiting paths [/] 2024-09-10T09:58:39.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:58:38 smithi134 ceph-mon[35674]: scrub summary: active paths [/] 2024-09-10T09:58:39.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:58:38 smithi134 ceph-mon[35674]: scrub summary: active paths [/] 2024-09-10T09:58:39.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:58:38 smithi134 ceph-mon[35674]: scrub summary: active paths [/] 2024-09-10T09:58:39.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:58:38 smithi134 ceph-mon[35674]: scrub summary: active paths [/] 2024-09-10T09:58:39.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:58:38 smithi134 ceph-mon[35674]: scrub summary: idle+waiting paths [/] 2024-09-10T09:58:39.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:58:38 smithi134 ceph-mon[35674]: scrub summary: idle+waiting paths [/] 2024-09-10T09:58:39.160 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T09:58:39.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:58:38 smithi120 ceph-mon[34770]: pgmap v495: 129 pgs: 129 active+clean; 77 MiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 759 KiB/s rd, 1.0 MiB/s wr, 183 op/s 2024-09-10T09:58:39.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:58:38 smithi120 ceph-mon[34770]: scrub queued for path: / 2024-09-10T09:58:39.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:58:38 smithi120 ceph-mon[34770]: scrub summary: idle+waiting paths [/] 2024-09-10T09:58:39.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:58:38 smithi120 ceph-mon[34770]: scrub summary: active paths [/] 2024-09-10T09:58:39.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:58:38 smithi120 ceph-mon[34770]: scrub summary: active paths [/] 2024-09-10T09:58:39.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:58:38 smithi120 ceph-mon[34770]: scrub summary: active paths [/] 2024-09-10T09:58:39.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:58:38 smithi120 ceph-mon[34770]: scrub summary: active paths [/] 2024-09-10T09:58:39.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:58:38 smithi120 ceph-mon[34770]: scrub summary: idle+waiting paths [/] 2024-09-10T09:58:39.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:58:38 smithi120 ceph-mon[34770]: scrub summary: idle+waiting paths [/] 2024-09-10T09:58:39.548 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:58:39.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:58:39 smithi077 ceph-mon[30319]: scrub summary: active paths [/] 2024-09-10T09:58:39.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:58:39 smithi077 ceph-mon[30319]: scrub summary: idle+waiting paths [/] 2024-09-10T09:58:39.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:58:39 smithi077 ceph-mon[30319]: scrub summary: idle+waiting paths [/] 2024-09-10T09:58:39.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:58:39 smithi077 ceph-mon[30319]: scrub summary: idle+waiting paths [/] 2024-09-10T09:58:40.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.171+0000 7f9269800640 1 -- 172.21.15.77:0/3262705866 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9264103ff0 msgr2=0x7f92641043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:40.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.171+0000 7f9269800640 1 --2- 172.21.15.77:0/3262705866 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9264103ff0 0x7f92641043d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9254009860 tx=0x7f925402a340 comp rx=0 tx=0).stop 2024-09-10T09:58:40.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.171+0000 7f9269800640 1 -- 172.21.15.77:0/3262705866 shutdown_connections 2024-09-10T09:58:40.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.171+0000 7f9269800640 1 --2- 172.21.15.77:0/3262705866 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9264109120 0x7f9264110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:40.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.171+0000 7f9269800640 1 --2- 172.21.15.77:0/3262705866 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f92641049a0 0x7f92641089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:40.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.171+0000 7f9269800640 1 --2- 172.21.15.77:0/3262705866 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9264103ff0 0x7f92641043d0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:40.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.171+0000 7f9269800640 1 -- 172.21.15.77:0/3262705866 >> 172.21.15.77:0/3262705866 conn(0x7f92640ffa90 msgr2=0x7f9264101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:40.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.171+0000 7f9269800640 1 -- 172.21.15.77:0/3262705866 shutdown_connections 2024-09-10T09:58:40.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.171+0000 7f9269800640 1 -- 172.21.15.77:0/3262705866 wait complete. 2024-09-10T09:58:40.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:58:39 smithi120 ceph-mon[34770]: scrub summary: active paths [/] 2024-09-10T09:58:40.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:58:39 smithi120 ceph-mon[34770]: scrub summary: idle+waiting paths [/] 2024-09-10T09:58:40.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:58:39 smithi120 ceph-mon[34770]: scrub summary: idle+waiting paths [/] 2024-09-10T09:58:40.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:58:39 smithi120 ceph-mon[34770]: scrub summary: idle+waiting paths [/] 2024-09-10T09:58:40.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.172+0000 7f9269800640 1 Processor -- start 2024-09-10T09:58:40.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.172+0000 7f9269800640 1 -- start start 2024-09-10T09:58:40.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.172+0000 7f9269800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9264103ff0 0x7f926419a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:40.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.172+0000 7f9269800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92641049a0 0x7f926419ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:40.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.172+0000 7f9269800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9264109120 0x7f926419eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:40.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.172+0000 7f9269800640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9264114120 con 0x7f9264109120 2024-09-10T09:58:40.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.172+0000 7f9269800640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9264113fa0 con 0x7f9264103ff0 2024-09-10T09:58:40.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.172+0000 7f9269800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f92641142a0 con 0x7f92641049a0 2024-09-10T09:58:40.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.173+0000 7f9262a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9264103ff0 0x7f926419a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:40.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.173+0000 7f9262a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9264103ff0 0x7f926419a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:40442/0 (socket says 172.21.15.77:40442) 2024-09-10T09:58:40.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.173+0000 7f9262a00640 1 -- 172.21.15.77:0/1118185618 learned_addr learned my addr 172.21.15.77:0/1118185618 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:58:40.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.173+0000 7f9263400640 1 --2- 172.21.15.77:0/1118185618 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9264109120 0x7f926419eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:40.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.173+0000 7f925be00640 1 --2- 172.21.15.77:0/1118185618 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92641049a0 0x7f926419ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:40.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.173+0000 7f9262a00640 1 -- 172.21.15.77:0/1118185618 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92641049a0 msgr2=0x7f926419ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:40.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.173+0000 7f9262a00640 1 --2- 172.21.15.77:0/1118185618 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92641049a0 0x7f926419ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:40.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.173+0000 7f9262a00640 1 -- 172.21.15.77:0/1118185618 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9264109120 msgr2=0x7f926419eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:40.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.173+0000 7f9262a00640 1 --2- 172.21.15.77:0/1118185618 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9264109120 0x7f926419eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:40.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.173+0000 7f9262a00640 1 -- 172.21.15.77:0/1118185618 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f926419f590 con 0x7f9264103ff0 2024-09-10T09:58:40.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.173+0000 7f9263400640 1 --2- 172.21.15.77:0/1118185618 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9264109120 0x7f926419eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:58:40.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.173+0000 7f925be00640 1 --2- 172.21.15.77:0/1118185618 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92641049a0 0x7f926419ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:58:40.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.173+0000 7f9262a00640 1 --2- 172.21.15.77:0/1118185618 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9264103ff0 0x7f926419a5e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f9254035040 tx=0x7f9254033d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:40.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.174+0000 7f925b400640 1 -- 172.21.15.77:0/1118185618 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f925403f020 con 0x7f9264103ff0 2024-09-10T09:58:40.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.174+0000 7f9269800640 1 -- 172.21.15.77:0/1118185618 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f926419f820 con 0x7f9264103ff0 2024-09-10T09:58:40.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.174+0000 7f925b400640 1 -- 172.21.15.77:0/1118185618 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9254032770 con 0x7f9264103ff0 2024-09-10T09:58:40.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.174+0000 7f925b400640 1 -- 172.21.15.77:0/1118185618 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9254037040 con 0x7f9264103ff0 2024-09-10T09:58:40.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.174+0000 7f9269800640 1 -- 172.21.15.77:0/1118185618 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92641a7160 con 0x7f9264103ff0 2024-09-10T09:58:40.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.177+0000 7f926c462740 1 Processor -- start 2024-09-10T09:58:40.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.177+0000 7f926c462740 1 -- start start 2024-09-10T09:58:40.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.177+0000 7f926c462740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634431dab90 0x5634431daf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:40.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.177+0000 7f925b400640 1 -- 172.21.15.77:0/1118185618 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9254042020 con 0x7f9264103ff0 2024-09-10T09:58:40.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.177+0000 7f926c462740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5634431db4b0 0x5634431e5830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:40.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.177+0000 7f926c462740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5634431e5d70 0x5634431e8160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:40.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.177+0000 7f926c462740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5634431413e0 con 0x5634431e5d70 2024-09-10T09:58:40.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.177+0000 7f926c462740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5634430e8d80 con 0x5634431dab90 2024-09-10T09:58:40.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.177+0000 7f926c462740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56344313d3a0 con 0x5634431db4b0 2024-09-10T09:58:40.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.178+0000 7f9263400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634431dab90 0x5634431daf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:40.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.178+0000 7f9263400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634431dab90 0x5634431daf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:40452/0 (socket says 172.21.15.77:40452) 2024-09-10T09:58:40.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.178+0000 7f9263400640 1 -- 172.21.15.77:0/180015346 learned_addr learned my addr 172.21.15.77:0/180015346 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:58:40.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.178+0000 7f9262a00640 1 --2- 172.21.15.77:0/180015346 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5634431db4b0 0x5634431e5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:40.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.178+0000 7f925be00640 1 --2- 172.21.15.77:0/180015346 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5634431e5d70 0x5634431e8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:40.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.178+0000 7f9262a00640 1 -- 172.21.15.77:0/180015346 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634431dab90 msgr2=0x5634431daf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:40.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.178+0000 7f9262a00640 1 --2- 172.21.15.77:0/180015346 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634431dab90 0x5634431daf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:40.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.178+0000 7f9262a00640 1 -- 172.21.15.77:0/180015346 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5634431e5d70 msgr2=0x5634431e8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:40.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.178+0000 7f9262a00640 1 --2- 172.21.15.77:0/180015346 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5634431e5d70 0x5634431e8160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:40.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.178+0000 7f9262a00640 1 -- 172.21.15.77:0/180015346 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563443107ae0 con 0x5634431db4b0 2024-09-10T09:58:40.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.178+0000 7f9262a00640 1 --2- 172.21.15.77:0/180015346 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5634431db4b0 0x5634431e5830 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f92540ecda0 tx=0x7f92540f5b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89ce97e78d75b851 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:40.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.179+0000 7f9253e00640 1 -- 172.21.15.77:0/180015346 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92540fe070 con 0x5634431db4b0 2024-09-10T09:58:40.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.179+0000 7f9253e00640 1 -- 172.21.15.77:0/180015346 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92540efd40 con 0x5634431db4b0 2024-09-10T09:58:40.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.179+0000 7f9253e00640 1 -- 172.21.15.77:0/180015346 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92540f18d0 con 0x5634431db4b0 2024-09-10T09:58:40.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.179+0000 7f925b400640 1 --2- 172.21.15.77:0/1118185618 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f92480a2b80 0x7f92480a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:40.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.179+0000 7f925b400640 1 -- 172.21.15.77:0/1118185618 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f92540ef750 con 0x7f9264103ff0 2024-09-10T09:58:40.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.179+0000 7f9263400640 1 --2- 172.21.15.77:0/1118185618 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f92480a2b80 0x7f92480a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:40.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.179+0000 7f926c462740 1 -- 172.21.15.77:0/180015346 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5634431db4b0 msgr2=0x5634431e5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:40.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.179+0000 7f926c462740 1 --2- 172.21.15.77:0/180015346 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5634431db4b0 0x5634431e5830 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f92540ecda0 tx=0x7f92540f5b20 comp rx=0 tx=0).stop 2024-09-10T09:58:40.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.180+0000 7f926c462740 1 -- 172.21.15.77:0/180015346 shutdown_connections 2024-09-10T09:58:40.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.180+0000 7f926c462740 1 --2- 172.21.15.77:0/180015346 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5634431e5d70 0x5634431e8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:40.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.180+0000 7f926c462740 1 --2- 172.21.15.77:0/180015346 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5634431db4b0 0x5634431e5830 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:40.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.180+0000 7f926c462740 1 --2- 172.21.15.77:0/180015346 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634431dab90 0x5634431daf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:40.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.180+0000 7f926c462740 1 -- 172.21.15.77:0/180015346 >> 172.21.15.77:0/180015346 conn(0x5634431d7120 msgr2=0x5634431d7500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:40.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.180+0000 7f926c462740 1 -- 172.21.15.77:0/180015346 shutdown_connections 2024-09-10T09:58:40.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.180+0000 7f926c462740 1 -- 172.21.15.77:0/180015346 wait complete. 2024-09-10T09:58:40.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.180+0000 7f9263400640 1 --2- 172.21.15.77:0/1118185618 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f92480a2b80 0x7f92480a5040 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f925c009fd0 tx=0x7f925c011040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:40.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.180+0000 7f926c462740 1 -- start start 2024-09-10T09:58:40.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.181+0000 7f926c462740 1 Processor -- start 2024-09-10T09:58:40.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.181+0000 7f926c462740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5634431dab90 0x5634431e2be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:40.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.181+0000 7f926c462740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5634431db4b0 0x5634431e3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:40.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.181+0000 7f926c462740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634431e5d70 0x5634431e3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:40.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.181+0000 7f926c462740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5634431f0f10 con 0x5634431db4b0 2024-09-10T09:58:40.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.181+0000 7f926c462740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5634430fc0f0 con 0x5634431e5d70 2024-09-10T09:58:40.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.181+0000 7f926c462740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5634431f1090 con 0x5634431dab90 2024-09-10T09:58:40.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.181+0000 7f926c462740 10 client.0 _ll_register_callbacks cb 0x5634430ed9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:58:40.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.181+0000 7f9262a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5634431dab90 0x5634431e2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:40.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.181+0000 7f9262a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5634431dab90 0x5634431e2be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:49200/0 (socket says 172.21.15.77:49200) 2024-09-10T09:58:40.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.181+0000 7f9262a00640 1 -- 172.21.15.77:0/2689816499 learned_addr learned my addr 172.21.15.77:0/2689816499 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:58:40.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.181+0000 7f925be00640 1 --2- 172.21.15.77:0/2689816499 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5634431db4b0 0x5634431e3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:40.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.182+0000 7f9263400640 1 --2- 172.21.15.77:0/2689816499 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634431e5d70 0x5634431e3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:40.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.182+0000 7f9262a00640 1 -- 172.21.15.77:0/2689816499 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634431e5d70 msgr2=0x5634431e3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:40.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.182+0000 7f9262a00640 1 --2- 172.21.15.77:0/2689816499 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634431e5d70 0x5634431e3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:40.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.182+0000 7f9262a00640 1 -- 172.21.15.77:0/2689816499 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5634431db4b0 msgr2=0x5634431e3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:40.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.182+0000 7f9262a00640 1 --2- 172.21.15.77:0/2689816499 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5634431db4b0 0x5634431e3120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:40.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.182+0000 7f9262a00640 1 -- 172.21.15.77:0/2689816499 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563442eb23c0 con 0x5634431dab90 2024-09-10T09:58:40.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.182+0000 7f9262a00640 1 --2- 172.21.15.77:0/2689816499 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5634431dab90 0x5634431e2be0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f92540b7fc0 tx=0x7f9254004240 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f87360065b80e65d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:40.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.182+0000 7f9250c00640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-10T09:58:40.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.182+0000 7f9250c00640 1 -- 172.21.15.77:0/2689816499 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92540fe030 con 0x5634431dab90 2024-09-10T09:58:40.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.182+0000 7f9250c00640 1 -- 172.21.15.77:0/2689816499 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f92540ece80 con 0x5634431dab90 2024-09-10T09:58:40.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.182+0000 7f926c462740 1 -- 172.21.15.77:0/2689816499 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92540efd40 con 0x5634431dab90 2024-09-10T09:58:40.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.182+0000 7f9250c00640 1 -- 172.21.15.77:0/2689816499 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f92540b7e00 con 0x5634431dab90 2024-09-10T09:58:40.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.183+0000 7f926c462740 10 client.26173 fetch_fsmap learned FSMap version 32 2024-09-10T09:58:40.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.183+0000 7f926c462740 1 -- 172.21.15.77:0/2689816499 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5634431f0f10 con 0x5634431dab90 2024-09-10T09:58:40.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.183+0000 7f9250c00640 1 -- 172.21.15.77:0/2689816499 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92540efd40 con 0x5634431dab90 2024-09-10T09:58:40.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.183+0000 7f9250c00640 1 -- 172.21.15.77:0/2689816499 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f925403a030 con 0x5634431dab90 2024-09-10T09:58:40.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.183+0000 7f926c462740 10 client.26173 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T09:58:40.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.183+0000 7f926c462740 10 client.26173 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:58:40.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.183+0000 7f926c462740 20 client.26173 populate_metadata read hostname 'smithi077' 2024-09-10T09:58:40.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.183+0000 7f926c462740 1 --2- 172.21.15.77:0/2689816499 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x5634431fde50 0x5634432002d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:40.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.183+0000 7f926c462740 4 client.26173 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:58:40.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.183+0000 7f926c462740 1 -- 172.21.15.77:0/2689816499 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56344314e950 con 0x5634431fde50 2024-09-10T09:58:40.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.183+0000 7f925be00640 1 --2- 172.21.15.77:0/2689816499 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x5634431fde50 0x5634432002d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:40.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.184+0000 7f9250c00640 10 client.26173 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T09:58:40.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.184+0000 7f925be00640 1 --2- 172.21.15.77:0/2689816499 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x5634431fde50 0x5634432002d0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15316360e621b46e server_cookie=9b3f8348e08cb8e4 in_seq=0 out_seq=0 2024-09-10T09:58:40.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.185+0000 7f9250c00640 1 -- 172.21.15.77:0/2689816499 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56344314e950 con 0x5634431fde50 2024-09-10T09:58:40.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.185+0000 7f9250c00640 10 client.26173 handle_command_reply: tid=0 2024-09-10T09:58:40.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.185+0000 7f926c462740 1 client.26173 shutdown 2024-09-10T09:58:40.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.185+0000 7f926c462740 1 -- 172.21.15.77:0/2689816499 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5634431dab90 msgr2=0x5634431e2be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:40.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.185+0000 7f926c462740 1 --2- 172.21.15.77:0/2689816499 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5634431dab90 0x5634431e2be0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f92540b7fc0 tx=0x7f9254004240 comp rx=0 tx=0).stop 2024-09-10T09:58:40.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.185+0000 7f926c462740 1 -- 172.21.15.77:0/2689816499 shutdown_connections 2024-09-10T09:58:40.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.185+0000 7f926c462740 1 --2- 172.21.15.77:0/2689816499 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x5634431fde50 0x5634432002d0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:40.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.185+0000 7f926c462740 1 --2- 172.21.15.77:0/2689816499 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634431e5d70 0x5634431e3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:40.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.185+0000 7f9250c00640 0 client.26173 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T09:58:40.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.185+0000 7f926c462740 1 --2- 172.21.15.77:0/2689816499 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5634431db4b0 0x5634431e3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:40.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.185+0000 7f926c462740 1 --2- 172.21.15.77:0/2689816499 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5634431dab90 0x5634431e2be0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:40.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.185+0000 7f926c462740 1 -- 172.21.15.77:0/2689816499 >> 172.21.15.77:0/2689816499 conn(0x5634431d7120 msgr2=0x5634431dce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:40.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.185+0000 7f926c462740 1 -- 172.21.15.77:0/2689816499 shutdown_connections 2024-09-10T09:58:40.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.185+0000 7f926c462740 1 -- 172.21.15.77:0/2689816499 wait complete. 2024-09-10T09:58:40.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.185+0000 7f926c462740 20 client.26173 trim_cache size 0 max 16384 2024-09-10T09:58:40.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.203+0000 7f926c462740 1 Processor -- start 2024-09-10T09:58:40.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.204+0000 7f926c462740 1 -- start start 2024-09-10T09:58:40.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.204+0000 7f926c462740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5634431dab90 0x563443205ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:40.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.204+0000 7f926c462740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5634431db4b0 0x5634432060e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:40.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.204+0000 7f926c462740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634431e5d70 0x5634431dfe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:40.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.204+0000 7f926c462740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f92641acc90 con 0x5634431db4b0 2024-09-10T09:58:40.206 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.204+0000 7f926c462740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f92641ae890 con 0x5634431e5d70 2024-09-10T09:58:40.206 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.204+0000 7f926c462740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f92641ad360 con 0x5634431dab90 2024-09-10T09:58:40.206 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.204+0000 7f9263400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634431e5d70 0x5634431dfe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:40.206 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.204+0000 7f925be00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5634431db4b0 0x5634432060e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:40.206 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.204+0000 7f9262a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5634431dab90 0x563443205ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:40.206 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.204+0000 7f9263400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634431e5d70 0x5634431dfe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:40458/0 (socket says 172.21.15.77:40458) 2024-09-10T09:58:40.206 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.204+0000 7f9263400640 1 -- 172.21.15.77:0/2374247850 learned_addr learned my addr 172.21.15.77:0/2374247850 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:58:40.206 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.204+0000 7f9262a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5634431dab90 0x563443205ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:49206/0 (socket says 172.21.15.77:49206) 2024-09-10T09:58:40.206 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.204+0000 7f925be00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5634431db4b0 0x5634432060e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:48590/0 (socket says 172.21.15.77:48590) 2024-09-10T09:58:40.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.205+0000 7f9263400640 1 -- 172.21.15.77:0/2374247850 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5634431dab90 msgr2=0x563443205ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:40.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.205+0000 7f9263400640 1 --2- 172.21.15.77:0/2374247850 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5634431dab90 0x563443205ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:40.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.205+0000 7f9263400640 1 -- 172.21.15.77:0/2374247850 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5634431db4b0 msgr2=0x5634432060e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:40.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.205+0000 7f9263400640 1 --2- 172.21.15.77:0/2374247850 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5634431db4b0 0x5634432060e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:40.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.205+0000 7f9263400640 1 -- 172.21.15.77:0/2374247850 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634431dd9f0 con 0x5634431e5d70 2024-09-10T09:58:40.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.205+0000 7f9263400640 1 --2- 172.21.15.77:0/2374247850 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634431e5d70 0x5634431dfe40 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f925c00b7b0 tx=0x7f925c017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b00dcd7bf42bbca server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:40.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.205+0000 7f9253400640 1 -- 172.21.15.77:0/2374247850 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f925c021040 con 0x5634431e5d70 2024-09-10T09:58:40.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.205+0000 7f9253400640 1 -- 172.21.15.77:0/2374247850 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f925c003e50 con 0x5634431e5d70 2024-09-10T09:58:40.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.205+0000 7f9253400640 1 -- 172.21.15.77:0/2374247850 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f925c004280 con 0x5634431e5d70 2024-09-10T09:58:40.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.206+0000 7f926c462740 1 -- 172.21.15.77:0/2374247850 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634431e5d70 msgr2=0x5634431dfe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:40.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.206+0000 7f926c462740 1 --2- 172.21.15.77:0/2374247850 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634431e5d70 0x5634431dfe40 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f925c00b7b0 tx=0x7f925c017000 comp rx=0 tx=0).stop 2024-09-10T09:58:40.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.206+0000 7f926c462740 1 -- 172.21.15.77:0/2374247850 shutdown_connections 2024-09-10T09:58:40.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.206+0000 7f926c462740 1 --2- 172.21.15.77:0/2374247850 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634431e5d70 0x5634431dfe40 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:40.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.206+0000 7f926c462740 1 --2- 172.21.15.77:0/2374247850 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5634431db4b0 0x5634432060e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:40.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.206+0000 7f926c462740 1 --2- 172.21.15.77:0/2374247850 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5634431dab90 0x563443205ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:40.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.206+0000 7f926c462740 1 -- 172.21.15.77:0/2374247850 >> 172.21.15.77:0/2374247850 conn(0x5634431d7120 msgr2=0x5634431e14b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:40.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.206+0000 7f926c462740 1 -- 172.21.15.77:0/2374247850 shutdown_connections 2024-09-10T09:58:40.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.207+0000 7f926c462740 1 -- 172.21.15.77:0/2374247850 wait complete. 2024-09-10T09:58:40.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.207+0000 7f926c462740 1 -- start start 2024-09-10T09:58:40.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.207+0000 7f926c462740 1 Processor -- start 2024-09-10T09:58:40.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.208+0000 7f926c462740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5634431dab90 0x5634431f2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:40.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.208+0000 7f926c462740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5634431db4b0 0x5634431f2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:40.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.208+0000 7f926c462740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634431e5d70 0x563443267b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:40.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.208+0000 7f926c462740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f92641ae100 con 0x5634431dab90 2024-09-10T09:58:40.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.208+0000 7f926c462740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f92641ada30 con 0x5634431e5d70 2024-09-10T09:58:40.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.208+0000 7f926c462740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f92641aac70 con 0x5634431db4b0 2024-09-10T09:58:40.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.208+0000 7f926c462740 10 client.0 _ll_register_callbacks cb 0x7f92540efef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:58:40.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.208+0000 7f925be00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5634431db4b0 0x5634431f2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:40.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.208+0000 7f925be00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5634431db4b0 0x5634431f2ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:49210/0 (socket says 172.21.15.77:49210) 2024-09-10T09:58:40.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.208+0000 7f925be00640 1 -- 172.21.15.77:0/721629868 learned_addr learned my addr 172.21.15.77:0/721629868 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:58:40.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.208+0000 7f925be00640 1 -- 172.21.15.77:0/721629868 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634431e5d70 msgr2=0x563443267b70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T09:58:40.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.208+0000 7f9262a00640 1 --2- 172.21.15.77:0/721629868 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5634431dab90 0x5634431f2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:40.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.208+0000 7f925be00640 1 --2- 172.21.15.77:0/721629868 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634431e5d70 0x563443267b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:40.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.208+0000 7f925be00640 1 -- 172.21.15.77:0/721629868 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5634431dab90 msgr2=0x5634431f2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:40.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.208+0000 7f925be00640 1 --2- 172.21.15.77:0/721629868 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5634431dab90 0x5634431f2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:40.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.208+0000 7f925be00640 1 -- 172.21.15.77:0/721629868 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634432680b0 con 0x5634431db4b0 2024-09-10T09:58:40.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.208+0000 7f9262a00640 1 --2- 172.21.15.77:0/721629868 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5634431dab90 0x5634431f2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:58:40.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.209+0000 7f925be00640 1 --2- 172.21.15.77:0/721629868 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5634431db4b0 0x5634431f2ca0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f924c009b30 tx=0x7f924c01e7c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db07038c66db5d45 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:40.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.209+0000 7f9250c00640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-10T09:58:40.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.209+0000 7f9250c00640 1 -- 172.21.15.77:0/721629868 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f924c004220 con 0x5634431db4b0 2024-09-10T09:58:40.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.209+0000 7f9250c00640 1 -- 172.21.15.77:0/721629868 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f924c0043c0 con 0x5634431db4b0 2024-09-10T09:58:40.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.209+0000 7f926c462740 1 -- 172.21.15.77:0/721629868 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f925c003e50 con 0x5634431db4b0 2024-09-10T09:58:40.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.210+0000 7f9250c00640 1 -- 172.21.15.77:0/721629868 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f924c0035f0 con 0x5634431db4b0 2024-09-10T09:58:40.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.210+0000 7f926c462740 10 client.26179 fetch_fsmap learned FSMap version 32 2024-09-10T09:58:40.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.210+0000 7f926c462740 1 -- 172.21.15.77:0/721629868 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5634432693c0 con 0x5634431db4b0 2024-09-10T09:58:40.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.210+0000 7f9250c00640 1 -- 172.21.15.77:0/721629868 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f925c003e50 con 0x5634431db4b0 2024-09-10T09:58:40.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.210+0000 7f9250c00640 1 -- 172.21.15.77:0/721629868 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f924c00f040 con 0x5634431db4b0 2024-09-10T09:58:40.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.210+0000 7f926c462740 10 client.26179 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T09:58:40.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.210+0000 7f926c462740 10 client.26179 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:58:40.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.210+0000 7f926c462740 20 client.26179 populate_metadata read hostname 'smithi077' 2024-09-10T09:58:40.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.210+0000 7f926c462740 1 --2- 172.21.15.77:0/721629868 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x563443269c20 0x56344326c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:40.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.210+0000 7f926c462740 4 client.26179 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T09:58:40.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.210+0000 7f926c462740 1 -- 172.21.15.77:0/721629868 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563443148e60 con 0x563443269c20 2024-09-10T09:58:40.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.210+0000 7f9262a00640 1 --2- 172.21.15.77:0/721629868 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x563443269c20 0x56344326c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:40.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.211+0000 7f9262a00640 1 --2- 172.21.15.77:0/721629868 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x563443269c20 0x56344326c040 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c3fce3cf0fc8bea server_cookie=c60b9abeaf8d9233 in_seq=0 out_seq=0 2024-09-10T09:58:40.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.211+0000 7f9250c00640 10 client.26179 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T09:58:40.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.211+0000 7f9250c00640 1 -- 172.21.15.77:0/721629868 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x563443148e60 con 0x563443269c20 2024-09-10T09:58:40.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.211+0000 7f9250c00640 10 client.26179 handle_command_reply: tid=0 2024-09-10T09:58:40.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.211+0000 7f926c462740 1 client.26179 shutdown 2024-09-10T09:58:40.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.211+0000 7f926c462740 1 -- 172.21.15.77:0/721629868 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5634431db4b0 msgr2=0x5634431f2ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:40.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.211+0000 7f926c462740 1 --2- 172.21.15.77:0/721629868 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5634431db4b0 0x5634431f2ca0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f924c009b30 tx=0x7f924c01e7c0 comp rx=0 tx=0).stop 2024-09-10T09:58:40.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.211+0000 7f926c462740 1 -- 172.21.15.77:0/721629868 shutdown_connections 2024-09-10T09:58:40.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.212+0000 7f926c462740 1 --2- 172.21.15.77:0/721629868 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x563443269c20 0x56344326c040 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:40.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.212+0000 7f926c462740 1 --2- 172.21.15.77:0/721629868 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634431e5d70 0x563443267b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:40.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.212+0000 7f926c462740 1 --2- 172.21.15.77:0/721629868 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5634431db4b0 0x5634431f2ca0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:40.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.212+0000 7f9250c00640 0 client.26179 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T09:58:40.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.212+0000 7f926c462740 1 --2- 172.21.15.77:0/721629868 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5634431dab90 0x5634431f2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:40.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.212+0000 7f926c462740 1 -- 172.21.15.77:0/721629868 >> 172.21.15.77:0/721629868 conn(0x5634431d7120 msgr2=0x5634431d59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:40.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.212+0000 7f926c462740 1 -- 172.21.15.77:0/721629868 shutdown_connections 2024-09-10T09:58:40.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.212+0000 7f926c462740 1 -- 172.21.15.77:0/721629868 wait complete. 2024-09-10T09:58:40.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.212+0000 7f926c462740 20 client.26179 trim_cache size 0 max 16384 2024-09-10T09:58:40.217 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-10T09:58:40.217 INFO:teuthology.orchestra.run.smithi077.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T09:58:40.218 INFO:teuthology.orchestra.run.smithi077.stdout: "scrubs": { 2024-09-10T09:58:40.218 INFO:teuthology.orchestra.run.smithi077.stdout: "33d5f7f6-c954-474b-9564-0de6b5cc9114": { 2024-09-10T09:58:40.218 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/", 2024-09-10T09:58:40.218 INFO:teuthology.orchestra.run.smithi077.stdout: "tag": "33d5f7f6-c954-474b-9564-0de6b5cc9114", 2024-09-10T09:58:40.218 INFO:teuthology.orchestra.run.smithi077.stdout: "options": "recursive,force" 2024-09-10T09:58:40.218 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T09:58:40.218 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T09:58:40.218 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-10T09:58:40.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.213+0000 7f9269800640 1 -- 172.21.15.77:0/1118185618 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f92480a2b80 msgr2=0x7f92480a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:40.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.213+0000 7f9269800640 1 --2- 172.21.15.77:0/1118185618 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f92480a2b80 0x7f92480a5040 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f925c009fd0 tx=0x7f925c011040 comp rx=0 tx=0).stop 2024-09-10T09:58:40.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.213+0000 7f9269800640 1 -- 172.21.15.77:0/1118185618 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9264103ff0 msgr2=0x7f926419a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:40.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.213+0000 7f9269800640 1 --2- 172.21.15.77:0/1118185618 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9264103ff0 0x7f926419a5e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f9254035040 tx=0x7f9254033d90 comp rx=0 tx=0).stop 2024-09-10T09:58:40.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.213+0000 7f9269800640 1 -- 172.21.15.77:0/1118185618 shutdown_connections 2024-09-10T09:58:40.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.213+0000 7f9269800640 1 --2- 172.21.15.77:0/1118185618 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f92480a2b80 0x7f92480a5040 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:40.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.213+0000 7f9269800640 1 --2- 172.21.15.77:0/1118185618 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9264109120 0x7f926419eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:40.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.213+0000 7f9269800640 1 --2- 172.21.15.77:0/1118185618 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92641049a0 0x7f926419ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:40.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.213+0000 7f9269800640 1 --2- 172.21.15.77:0/1118185618 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9264103ff0 0x7f926419a5e0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:40.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.213+0000 7f9269800640 1 -- 172.21.15.77:0/1118185618 >> 172.21.15.77:0/1118185618 conn(0x7f92640ffa90 msgr2=0x7f9264100e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:40.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.215+0000 7f9269800640 1 -- 172.21.15.77:0/1118185618 shutdown_connections 2024-09-10T09:58:40.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:58:40.215+0000 7f9269800640 1 -- 172.21.15.77:0/1118185618 wait complete. 2024-09-10T09:58:40.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:58:39 smithi134 ceph-mon[35674]: scrub summary: active paths [/] 2024-09-10T09:58:40.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:58:39 smithi134 ceph-mon[35674]: scrub summary: idle+waiting paths [/] 2024-09-10T09:58:40.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:58:39 smithi134 ceph-mon[35674]: scrub summary: idle+waiting paths [/] 2024-09-10T09:58:40.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:58:39 smithi134 ceph-mon[35674]: scrub summary: idle+waiting paths [/] 2024-09-10T09:58:41.072 INFO:tasks.cephfs.filesystem:scrub status for tag:33d5f7f6-c954-474b-9564-0de6b5cc9114 - {'path': '/', 'tag': '33d5f7f6-c954-474b-9564-0de6b5cc9114', 'options': 'recursive,force'} 2024-09-10T09:58:41.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:58:40 smithi120 ceph-mon[34770]: pgmap v496: 129 pgs: 129 active+clean; 79 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.5 MiB/s wr, 400 op/s 2024-09-10T09:58:41.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:58:40 smithi077 ceph-mon[30319]: pgmap v496: 129 pgs: 129 active+clean; 79 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.5 MiB/s wr, 400 op/s 2024-09-10T09:58:41.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:58:40 smithi134 ceph-mon[35674]: pgmap v496: 129 pgs: 129 active+clean; 79 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.5 MiB/s wr, 400 op/s 2024-09-10T09:58:43.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:58:42 smithi120 ceph-mon[34770]: pgmap v497: 129 pgs: 129 active+clean; 79 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.4 MiB/s wr, 387 op/s 2024-09-10T09:58:43.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:58:42 smithi077 ceph-mon[30319]: pgmap v497: 129 pgs: 129 active+clean; 79 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.4 MiB/s wr, 387 op/s 2024-09-10T09:58:43.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:58:42 smithi134 ceph-mon[35674]: pgmap v497: 129 pgs: 129 active+clean; 79 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.4 MiB/s wr, 387 op/s 2024-09-10T09:58:45.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:58:44 smithi120 ceph-mon[34770]: pgmap v498: 129 pgs: 129 active+clean; 81 MiB data, 681 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.8 MiB/s wr, 599 op/s 2024-09-10T09:58:45.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:58:44 smithi077 ceph-mon[30319]: pgmap v498: 129 pgs: 129 active+clean; 81 MiB data, 681 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.8 MiB/s wr, 599 op/s 2024-09-10T09:58:45.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:58:44 smithi134 ceph-mon[35674]: pgmap v498: 129 pgs: 129 active+clean; 81 MiB data, 681 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.8 MiB/s wr, 599 op/s 2024-09-10T09:58:47.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:58:46 smithi120 ceph-mon[34770]: pgmap v499: 129 pgs: 129 active+clean; 81 MiB data, 671 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 437 op/s 2024-09-10T09:58:47.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:58:46 smithi077 ceph-mon[30319]: pgmap v499: 129 pgs: 129 active+clean; 81 MiB data, 671 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 437 op/s 2024-09-10T09:58:47.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:58:46 smithi134 ceph-mon[35674]: pgmap v499: 129 pgs: 129 active+clean; 81 MiB data, 671 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 437 op/s 2024-09-10T09:58:48.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:58:48 smithi077 ceph-mon[30319]: pgmap v500: 129 pgs: 129 active+clean; 81 MiB data, 681 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 437 op/s 2024-09-10T09:58:49.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:58:48 smithi120 ceph-mon[34770]: pgmap v500: 129 pgs: 129 active+clean; 81 MiB data, 681 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 437 op/s 2024-09-10T09:58:49.182 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:58:48 smithi134 ceph-mon[35674]: pgmap v500: 129 pgs: 129 active+clean; 81 MiB data, 681 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 437 op/s 2024-09-10T09:58:51.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:58:50 smithi120 ceph-mon[34770]: pgmap v501: 129 pgs: 129 active+clean; 82 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.0 MiB/s wr, 635 op/s 2024-09-10T09:58:51.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:58:50 smithi120 ceph-mon[34770]: scrub summary: idle 2024-09-10T09:58:51.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:58:50 smithi120 ceph-mon[34770]: scrub summary: idle 2024-09-10T09:58:51.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:58:50 smithi077 ceph-mon[30319]: pgmap v501: 129 pgs: 129 active+clean; 82 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.0 MiB/s wr, 635 op/s 2024-09-10T09:58:51.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:58:50 smithi077 ceph-mon[30319]: scrub summary: idle 2024-09-10T09:58:51.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:58:50 smithi077 ceph-mon[30319]: scrub summary: idle 2024-09-10T09:58:51.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:58:50 smithi134 ceph-mon[35674]: pgmap v501: 129 pgs: 129 active+clean; 82 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.0 MiB/s wr, 635 op/s 2024-09-10T09:58:51.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:58:50 smithi134 ceph-mon[35674]: scrub summary: idle 2024-09-10T09:58:51.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:58:50 smithi134 ceph-mon[35674]: scrub summary: idle 2024-09-10T09:58:52.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:58:51 smithi120 ceph-mon[34770]: scrub summary: idle 2024-09-10T09:58:52.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:58:51 smithi120 ceph-mon[34770]: scrub summary: idle 2024-09-10T09:58:52.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:58:51 smithi120 ceph-mon[34770]: scrub summary: idle 2024-09-10T09:58:52.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:58:51 smithi077 ceph-mon[30319]: scrub summary: idle 2024-09-10T09:58:52.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:58:51 smithi077 ceph-mon[30319]: scrub summary: idle 2024-09-10T09:58:52.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:58:51 smithi077 ceph-mon[30319]: scrub summary: idle 2024-09-10T09:58:52.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:58:51 smithi134 ceph-mon[35674]: scrub summary: idle 2024-09-10T09:58:52.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:58:51 smithi134 ceph-mon[35674]: scrub summary: idle 2024-09-10T09:58:52.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:58:51 smithi134 ceph-mon[35674]: scrub summary: idle 2024-09-10T09:58:53.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:58:52 smithi120 ceph-mon[34770]: pgmap v502: 129 pgs: 129 active+clean; 83 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.6 MiB/s wr, 433 op/s 2024-09-10T09:58:53.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:58:52 smithi077 ceph-mon[30319]: pgmap v502: 129 pgs: 129 active+clean; 83 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.6 MiB/s wr, 433 op/s 2024-09-10T09:58:53.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:58:52 smithi134 ceph-mon[35674]: pgmap v502: 129 pgs: 129 active+clean; 83 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.6 MiB/s wr, 433 op/s 2024-09-10T09:58:55.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:58:54 smithi120 ceph-mon[34770]: pgmap v503: 129 pgs: 129 active+clean; 84 MiB data, 711 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.7 MiB/s wr, 584 op/s 2024-09-10T09:58:55.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:58:54 smithi077 ceph-mon[30319]: pgmap v503: 129 pgs: 129 active+clean; 84 MiB data, 711 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.7 MiB/s wr, 584 op/s 2024-09-10T09:58:55.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:58:54 smithi134 ceph-mon[35674]: pgmap v503: 129 pgs: 129 active+clean; 84 MiB data, 711 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.7 MiB/s wr, 584 op/s 2024-09-10T09:58:57.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:58:56 smithi120 ceph-mon[34770]: pgmap v504: 129 pgs: 129 active+clean; 85 MiB data, 712 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.4 MiB/s wr, 373 op/s 2024-09-10T09:58:57.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:58:56 smithi077 ceph-mon[30319]: pgmap v504: 129 pgs: 129 active+clean; 85 MiB data, 712 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.4 MiB/s wr, 373 op/s 2024-09-10T09:58:57.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:58:56 smithi134 ceph-mon[35674]: pgmap v504: 129 pgs: 129 active+clean; 85 MiB data, 712 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.4 MiB/s wr, 373 op/s 2024-09-10T09:58:59.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:58:58 smithi120 ceph-mon[34770]: pgmap v505: 129 pgs: 129 active+clean; 87 MiB data, 712 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.6 MiB/s wr, 434 op/s 2024-09-10T09:58:59.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:58:58 smithi077 ceph-mon[30319]: pgmap v505: 129 pgs: 129 active+clean; 87 MiB data, 712 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.6 MiB/s wr, 434 op/s 2024-09-10T09:58:59.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:58:58 smithi134 ceph-mon[35674]: pgmap v505: 129 pgs: 129 active+clean; 87 MiB data, 712 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.6 MiB/s wr, 434 op/s 2024-09-10T09:58:59.901 INFO:tasks.workunit.client.0.smithi077.stdout:30000 trunc from 0x7bb2f to 0xcdf83 2024-09-10T09:59:00.451 INFO:tasks.workunit.client.0.smithi077.stdout:copying to largest ever: 0xffcdd 2024-09-10T09:59:01.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:59:00 smithi120 ceph-mon[34770]: pgmap v506: 129 pgs: 129 active+clean; 88 MiB data, 772 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.6 MiB/s wr, 552 op/s 2024-09-10T09:59:01.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:59:00 smithi077 ceph-mon[30319]: pgmap v506: 129 pgs: 129 active+clean; 88 MiB data, 772 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.6 MiB/s wr, 552 op/s 2024-09-10T09:59:01.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:59:00 smithi134 ceph-mon[35674]: pgmap v506: 129 pgs: 129 active+clean; 88 MiB data, 772 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.6 MiB/s wr, 552 op/s 2024-09-10T09:59:03.117 INFO:tasks.workunit.client.0.smithi077.stdout:copying to largest ever: 0xffd97 2024-09-10T09:59:03.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:59:02 smithi120 ceph-mon[34770]: pgmap v507: 129 pgs: 129 active+clean; 90 MiB data, 810 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.6 MiB/s wr, 409 op/s 2024-09-10T09:59:03.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:59:02 smithi077 ceph-mon[30319]: pgmap v507: 129 pgs: 129 active+clean; 90 MiB data, 810 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.6 MiB/s wr, 409 op/s 2024-09-10T09:59:03.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:59:02 smithi134 ceph-mon[35674]: pgmap v507: 129 pgs: 129 active+clean; 90 MiB data, 810 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.6 MiB/s wr, 409 op/s 2024-09-10T09:59:05.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:59:04 smithi120 ceph-mon[34770]: pgmap v508: 129 pgs: 129 active+clean; 90 MiB data, 840 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.5 MiB/s wr, 495 op/s 2024-09-10T09:59:05.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:59:04 smithi077 ceph-mon[30319]: pgmap v508: 129 pgs: 129 active+clean; 90 MiB data, 840 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.5 MiB/s wr, 495 op/s 2024-09-10T09:59:05.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:59:04 smithi134 ceph-mon[35674]: pgmap v508: 129 pgs: 129 active+clean; 90 MiB data, 840 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.5 MiB/s wr, 495 op/s 2024-09-10T09:59:07.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:59:06 smithi120 ceph-mon[34770]: pgmap v509: 129 pgs: 129 active+clean; 90 MiB data, 840 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.4 MiB/s wr, 345 op/s 2024-09-10T09:59:07.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:59:06 smithi077 ceph-mon[30319]: pgmap v509: 129 pgs: 129 active+clean; 90 MiB data, 840 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.4 MiB/s wr, 345 op/s 2024-09-10T09:59:07.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:59:06 smithi134 ceph-mon[35674]: pgmap v509: 129 pgs: 129 active+clean; 90 MiB data, 840 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.4 MiB/s wr, 345 op/s 2024-09-10T09:59:09.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:59:08 smithi077 ceph-mon[30319]: pgmap v510: 129 pgs: 129 active+clean; 92 MiB data, 855 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 379 op/s 2024-09-10T09:59:09.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:59:08 smithi134 ceph-mon[35674]: pgmap v510: 129 pgs: 129 active+clean; 92 MiB data, 855 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 379 op/s 2024-09-10T09:59:09.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:59:08 smithi120 ceph-mon[34770]: pgmap v510: 129 pgs: 129 active+clean; 92 MiB data, 855 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 379 op/s 2024-09-10T09:59:11.074 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T09:59:11.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:59:10 smithi077 ceph-mon[30319]: pgmap v511: 129 pgs: 129 active+clean; 94 MiB data, 862 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.2 MiB/s wr, 462 op/s 2024-09-10T09:59:11.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:59:10 smithi134 ceph-mon[35674]: pgmap v511: 129 pgs: 129 active+clean; 94 MiB data, 862 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.2 MiB/s wr, 462 op/s 2024-09-10T09:59:11.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:59:10 smithi120 ceph-mon[34770]: pgmap v511: 129 pgs: 129 active+clean; 94 MiB data, 862 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.2 MiB/s wr, 462 op/s 2024-09-10T09:59:11.463 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:59:11.782 INFO:tasks.quiescer.fs.[cephfs]:Total duration (45.5) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-10T09:59:11.782 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.75 2024-09-10T09:59:11.783 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:59:12.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.064+0000 7fdacb800640 1 -- 172.21.15.77:0/3430999832 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdac4106f00 msgr2=0x7fdac410ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:12.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.064+0000 7fdacb800640 1 --2- 172.21.15.77:0/3430999832 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdac4106f00 0x7fdac410ea80 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fdabc009ca0 tx=0x7fdabc02a8c0 comp rx=0 tx=0).stop 2024-09-10T09:59:12.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.064+0000 7fdacb800640 1 -- 172.21.15.77:0/3430999832 shutdown_connections 2024-09-10T09:59:12.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.064+0000 7fdacb800640 1 --2- 172.21.15.77:0/3430999832 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdac4106f00 0x7fdac410ea80 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:12.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.064+0000 7fdacb800640 1 --2- 172.21.15.77:0/3430999832 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdac4102780 0x7fdac41067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:12.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.064+0000 7fdacb800640 1 --2- 172.21.15.77:0/3430999832 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdac4101dd0 0x7fdac41021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:12.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.064+0000 7fdacb800640 1 -- 172.21.15.77:0/3430999832 >> 172.21.15.77:0/3430999832 conn(0x7fdac40fd8b0 msgr2=0x7fdac40ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:12.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.064+0000 7fdacb800640 1 -- 172.21.15.77:0/3430999832 shutdown_connections 2024-09-10T09:59:12.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.064+0000 7fdacb800640 1 -- 172.21.15.77:0/3430999832 wait complete. 2024-09-10T09:59:12.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.065+0000 7fdacb800640 1 Processor -- start 2024-09-10T09:59:12.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.066+0000 7fdacb800640 1 -- start start 2024-09-10T09:59:12.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.066+0000 7fdacb800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdac4101dd0 0x7fdac41983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:12.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.066+0000 7fdacb800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdac4102780 0x7fdac41988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:12.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.066+0000 7fdacb800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdac4106f00 0x7fdac419cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:12.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.066+0000 7fdacb800640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fdac4111f60 con 0x7fdac4101dd0 2024-09-10T09:59:12.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.066+0000 7fdacb800640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdac4111de0 con 0x7fdac4106f00 2024-09-10T09:59:12.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.066+0000 7fdacb800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdac41120e0 con 0x7fdac4102780 2024-09-10T09:59:12.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.066+0000 7fdac8e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdac4101dd0 0x7fdac41983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:12.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.067+0000 7fdac8e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdac4101dd0 0x7fdac41983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34044/0 (socket says 172.21.15.77:34044) 2024-09-10T09:59:12.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.067+0000 7fdac8e00640 1 -- 172.21.15.77:0/376636516 learned_addr learned my addr 172.21.15.77:0/376636516 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:59:12.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.067+0000 7fdac9800640 1 --2- 172.21.15.77:0/376636516 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdac4106f00 0x7fdac419cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:12.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.067+0000 7fdac3e00640 1 --2- 172.21.15.77:0/376636516 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdac4102780 0x7fdac41988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:12.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.067+0000 7fdac9800640 1 -- 172.21.15.77:0/376636516 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdac4102780 msgr2=0x7fdac41988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:12.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.067+0000 7fdac9800640 1 --2- 172.21.15.77:0/376636516 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdac4102780 0x7fdac41988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:12.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.067+0000 7fdac9800640 1 -- 172.21.15.77:0/376636516 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdac4101dd0 msgr2=0x7fdac41983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:12.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.067+0000 7fdac9800640 1 --2- 172.21.15.77:0/376636516 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdac4101dd0 0x7fdac41983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:12.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.067+0000 7fdac9800640 1 -- 172.21.15.77:0/376636516 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdac419d360 con 0x7fdac4106f00 2024-09-10T09:59:12.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.067+0000 7fdac9800640 1 --2- 172.21.15.77:0/376636516 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdac4106f00 0x7fdac419cc80 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fdabc02ade0 tx=0x7fdabc0319f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:12.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.068+0000 7fdac1600640 1 -- 172.21.15.77:0/376636516 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdabc003b70 con 0x7fdac4106f00 2024-09-10T09:59:12.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.068+0000 7fdac1600640 1 -- 172.21.15.77:0/376636516 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdabc003d10 con 0x7fdac4106f00 2024-09-10T09:59:12.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.068+0000 7fdac1600640 1 -- 172.21.15.77:0/376636516 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdabc02fc40 con 0x7fdac4106f00 2024-09-10T09:59:12.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.068+0000 7fdacb800640 1 -- 172.21.15.77:0/376636516 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdac419d5f0 con 0x7fdac4106f00 2024-09-10T09:59:12.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.068+0000 7fdacb800640 1 -- 172.21.15.77:0/376636516 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdac41a4f30 con 0x7fdac4106f00 2024-09-10T09:59:12.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.071+0000 7fdac1600640 1 -- 172.21.15.77:0/376636516 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fdabc004050 con 0x7fdac4106f00 2024-09-10T09:59:12.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.072+0000 7fdace23b740 1 Processor -- start 2024-09-10T09:59:12.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.072+0000 7fdace23b740 1 -- start start 2024-09-10T09:59:12.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.072+0000 7fdace23b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fbeeabbb90 0x55fbeeabbf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:12.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.072+0000 7fdace23b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fbeeabc4b0 0x55fbeeac6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:12.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.072+0000 7fdace23b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fbeeac6d70 0x55fbeeac9160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:12.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.072+0000 7fdace23b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55fbeea223e0 con 0x55fbeeabc4b0 2024-09-10T09:59:12.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.072+0000 7fdace23b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55fbee9c9d80 con 0x55fbeeac6d70 2024-09-10T09:59:12.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.072+0000 7fdace23b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55fbeea1e3a0 con 0x55fbeeabbb90 2024-09-10T09:59:12.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.072+0000 7fdac3e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fbeeabbb90 0x55fbeeabbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:12.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.073+0000 7fdac3e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fbeeabbb90 0x55fbeeabbf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:45418/0 (socket says 172.21.15.77:45418) 2024-09-10T09:59:12.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.073+0000 7fdac3e00640 1 -- 172.21.15.77:0/1389534775 learned_addr learned my addr 172.21.15.77:0/1389534775 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:59:12.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.073+0000 7fdac8e00640 1 --2- 172.21.15.77:0/1389534775 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fbeeabc4b0 0x55fbeeac6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:12.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.073+0000 7fdac3e00640 1 --2- 172.21.15.77:0/1389534775 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fbeeac6d70 0x55fbeeac9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:12.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.073+0000 7fdac3e00640 1 -- 172.21.15.77:0/1389534775 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fbeeac6d70 msgr2=0x55fbeeac9160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:12.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.073+0000 7fdac3e00640 1 --2- 172.21.15.77:0/1389534775 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fbeeac6d70 0x55fbeeac9160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:12.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.073+0000 7fdac3e00640 1 -- 172.21.15.77:0/1389534775 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fbeeabc4b0 msgr2=0x55fbeeac6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:12.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.073+0000 7fdac3e00640 1 --2- 172.21.15.77:0/1389534775 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fbeeabc4b0 0x55fbeeac6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:12.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.073+0000 7fdac3e00640 1 -- 172.21.15.77:0/1389534775 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fbee9e8ae0 con 0x55fbeeabbb90 2024-09-10T09:59:12.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.073+0000 7fdac1600640 1 --2- 172.21.15.77:0/376636516 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fdaac0a2c00 0x7fdaac0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:12.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.073+0000 7fdac3e00640 1 --2- 172.21.15.77:0/1389534775 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fbeeabbb90 0x55fbeeabbf70 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fdab0012770 tx=0x7fdab0009660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c5421f19db2bfcb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:12.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.073+0000 7fdac1600640 1 -- 172.21.15.77:0/376636516 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fdabc0f03e0 con 0x7fdac4106f00 2024-09-10T09:59:12.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.073+0000 7fdac8e00640 1 --2- 172.21.15.77:0/376636516 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fdaac0a2c00 0x7fdaac0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:12.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.074+0000 7fdab6000640 1 -- 172.21.15.77:0/1389534775 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab0012cf0 con 0x55fbeeabbb90 2024-09-10T09:59:12.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.074+0000 7fdab6000640 1 -- 172.21.15.77:0/1389534775 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdab00107d0 con 0x55fbeeabbb90 2024-09-10T09:59:12.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.074+0000 7fdab6000640 1 -- 172.21.15.77:0/1389534775 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab0003850 con 0x55fbeeabbb90 2024-09-10T09:59:12.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.074+0000 7fdace23b740 1 -- 172.21.15.77:0/1389534775 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fbeeabbb90 msgr2=0x55fbeeabbf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:12.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.074+0000 7fdac8e00640 1 --2- 172.21.15.77:0/376636516 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fdaac0a2c00 0x7fdaac0a50c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x55fbeeab8f80 tx=0x7fdab8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:12.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.074+0000 7fdace23b740 1 --2- 172.21.15.77:0/1389534775 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fbeeabbb90 0x55fbeeabbf70 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fdab0012770 tx=0x7fdab0009660 comp rx=0 tx=0).stop 2024-09-10T09:59:12.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.075+0000 7fdace23b740 1 -- 172.21.15.77:0/1389534775 shutdown_connections 2024-09-10T09:59:12.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.075+0000 7fdace23b740 1 --2- 172.21.15.77:0/1389534775 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fbeeac6d70 0x55fbeeac9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:12.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.075+0000 7fdace23b740 1 --2- 172.21.15.77:0/1389534775 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fbeeabc4b0 0x55fbeeac6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:12.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.075+0000 7fdace23b740 1 --2- 172.21.15.77:0/1389534775 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fbeeabbb90 0x55fbeeabbf70 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:12.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.075+0000 7fdace23b740 1 -- 172.21.15.77:0/1389534775 >> 172.21.15.77:0/1389534775 conn(0x55fbeeab8120 msgr2=0x55fbeeab8500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:12.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.075+0000 7fdace23b740 1 -- 172.21.15.77:0/1389534775 shutdown_connections 2024-09-10T09:59:12.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.075+0000 7fdace23b740 1 -- 172.21.15.77:0/1389534775 wait complete. 2024-09-10T09:59:12.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.075+0000 7fdace23b740 1 -- start start 2024-09-10T09:59:12.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.075+0000 7fdace23b740 1 Processor -- start 2024-09-10T09:59:12.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.076+0000 7fdace23b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fbeeabbb90 0x55fbeeac3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:12.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.076+0000 7fdace23b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fbeeabc4b0 0x55fbeeac4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:12.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.076+0000 7fdace23b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fbeeac6d70 0x55fbeeac47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:12.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.076+0000 7fdace23b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55fbeeac99e0 con 0x55fbeeabbb90 2024-09-10T09:59:12.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.076+0000 7fdace23b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55fbee9dd0f0 con 0x55fbeeac6d70 2024-09-10T09:59:12.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.076+0000 7fdace23b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55fbeeac9b60 con 0x55fbeeabc4b0 2024-09-10T09:59:12.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.076+0000 7fdace23b740 10 client.0 _ll_register_callbacks cb 0x55fbee9ce9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:59:12.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.076+0000 7fdac3e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fbeeabc4b0 0x55fbeeac4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:12.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.076+0000 7fdac3e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fbeeabc4b0 0x55fbeeac4260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:45434/0 (socket says 172.21.15.77:45434) 2024-09-10T09:59:12.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.076+0000 7fdac3e00640 1 -- 172.21.15.77:0/2262201061 learned_addr learned my addr 172.21.15.77:0/2262201061 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:59:12.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.076+0000 7fdac9800640 1 --2- 172.21.15.77:0/2262201061 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fbeeac6d70 0x55fbeeac47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:12.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.076+0000 7fdac8e00640 1 --2- 172.21.15.77:0/2262201061 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fbeeabbb90 0x55fbeeac3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:12.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.077+0000 7fdac9800640 1 -- 172.21.15.77:0/2262201061 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fbeeabc4b0 msgr2=0x55fbeeac4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:12.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.077+0000 7fdac9800640 1 --2- 172.21.15.77:0/2262201061 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fbeeabc4b0 0x55fbeeac4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:12.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.077+0000 7fdac9800640 1 -- 172.21.15.77:0/2262201061 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fbeeabbb90 msgr2=0x55fbeeac3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:12.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.077+0000 7fdac9800640 1 --2- 172.21.15.77:0/2262201061 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fbeeabbb90 0x55fbeeac3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:12.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.077+0000 7fdac9800640 1 -- 172.21.15.77:0/2262201061 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fbee7933c0 con 0x55fbeeac6d70 2024-09-10T09:59:12.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.077+0000 7fdac3e00640 1 --2- 172.21.15.77:0/2262201061 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fbeeabc4b0 0x55fbeeac4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:59:12.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.077+0000 7fdac9800640 1 --2- 172.21.15.77:0/2262201061 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fbeeac6d70 0x55fbeeac47a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fdabc009430 tx=0x7fdabc0bc7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=312e6dd6f9a17ae5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:12.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.077+0000 7fdaaaa00640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T09:59:12.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.077+0000 7fdaaaa00640 1 -- 172.21.15.77:0/2262201061 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdabc0bc910 con 0x55fbeeac6d70 2024-09-10T09:59:12.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.077+0000 7fdace23b740 1 -- 172.21.15.77:0/2262201061 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdab00107d0 con 0x55fbeeac6d70 2024-09-10T09:59:12.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.077+0000 7fdaaaa00640 1 -- 172.21.15.77:0/2262201061 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fdabc02fde0 con 0x55fbeeac6d70 2024-09-10T09:59:12.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.078+0000 7fdaaaa00640 1 -- 172.21.15.77:0/2262201061 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fdabc0c0cc0 con 0x55fbeeac6d70 2024-09-10T09:59:12.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.078+0000 7fdace23b740 10 client.26237 fetch_fsmap learned FSMap version 32 2024-09-10T09:59:12.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.078+0000 7fdace23b740 1 -- 172.21.15.77:0/2262201061 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55fbeeac99e0 con 0x55fbeeac6d70 2024-09-10T09:59:12.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.078+0000 7fdaaaa00640 1 -- 172.21.15.77:0/2262201061 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdab00107d0 con 0x55fbeeac6d70 2024-09-10T09:59:12.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.078+0000 7fdaaaa00640 1 -- 172.21.15.77:0/2262201061 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fdabc0c50f0 con 0x55fbeeac6d70 2024-09-10T09:59:12.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.078+0000 7fdace23b740 10 client.26237 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T09:59:12.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.078+0000 7fdace23b740 10 client.26237 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:59:12.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.078+0000 7fdace23b740 20 client.26237 populate_metadata read hostname 'smithi077' 2024-09-10T09:59:12.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.078+0000 7fdace23b740 1 --2- 172.21.15.77:0/2262201061 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55fbeeadb090 0x55fbeeadd4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:12.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.078+0000 7fdace23b740 4 client.26237 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:59:12.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.078+0000 7fdace23b740 1 -- 172.21.15.77:0/2262201061 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fbeea2f950 con 0x55fbeeadb090 2024-09-10T09:59:12.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.079+0000 7fdac8e00640 1 --2- 172.21.15.77:0/2262201061 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55fbeeadb090 0x55fbeeadd4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:12.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.079+0000 7fdac8e00640 1 --2- 172.21.15.77:0/2262201061 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55fbeeadb090 0x55fbeeadd4d0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ed99817e3baf24b server_cookie=3336edd06993279a in_seq=0 out_seq=0 2024-09-10T09:59:12.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.079+0000 7fdaaaa00640 10 client.26237 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T09:59:12.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.080+0000 7fdaaaa00640 1 -- 172.21.15.77:0/2262201061 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fbeea2f950 con 0x55fbeeadb090 2024-09-10T09:59:12.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.080+0000 7fdaaaa00640 10 client.26237 handle_command_reply: tid=0 2024-09-10T09:59:12.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.080+0000 7fdace23b740 1 client.26237 shutdown 2024-09-10T09:59:12.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.080+0000 7fdace23b740 1 -- 172.21.15.77:0/2262201061 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fbeeac6d70 msgr2=0x55fbeeac47a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:12.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.080+0000 7fdace23b740 1 --2- 172.21.15.77:0/2262201061 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fbeeac6d70 0x55fbeeac47a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fdabc009430 tx=0x7fdabc0bc7a0 comp rx=0 tx=0).stop 2024-09-10T09:59:12.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.080+0000 7fdace23b740 1 -- 172.21.15.77:0/2262201061 shutdown_connections 2024-09-10T09:59:12.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.080+0000 7fdace23b740 1 --2- 172.21.15.77:0/2262201061 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55fbeeadb090 0x55fbeeadd4d0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:12.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.080+0000 7fdace23b740 1 --2- 172.21.15.77:0/2262201061 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fbeeac6d70 0x55fbeeac47a0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:12.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.080+0000 7fdace23b740 1 --2- 172.21.15.77:0/2262201061 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fbeeabc4b0 0x55fbeeac4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:12.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.080+0000 7fdace23b740 1 --2- 172.21.15.77:0/2262201061 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fbeeabbb90 0x55fbeeac3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:12.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.080+0000 7fdaaaa00640 0 client.26237 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T09:59:12.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.080+0000 7fdace23b740 1 -- 172.21.15.77:0/2262201061 >> 172.21.15.77:0/2262201061 conn(0x55fbeeab8120 msgr2=0x55fbeeabde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:12.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.080+0000 7fdace23b740 1 -- 172.21.15.77:0/2262201061 shutdown_connections 2024-09-10T09:59:12.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.080+0000 7fdace23b740 1 -- 172.21.15.77:0/2262201061 wait complete. 2024-09-10T09:59:12.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.081+0000 7fdace23b740 20 client.26237 trim_cache size 0 max 16384 2024-09-10T09:59:12.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.099+0000 7fdace23b740 1 Processor -- start 2024-09-10T09:59:12.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.099+0000 7fdace23b740 1 -- start start 2024-09-10T09:59:12.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.099+0000 7fdace23b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fbeeab8120 0x55fbeeabe640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:12.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.099+0000 7fdace23b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fbeeabbb90 0x55fbeeae2de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:12.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.099+0000 7fdace23b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fbeeabc4b0 0x55fbeeae3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:12.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.099+0000 7fdace23b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fdac41acba0 con 0x55fbeeabbb90 2024-09-10T09:59:12.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.099+0000 7fdace23b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdac41ae7a0 con 0x55fbeeabc4b0 2024-09-10T09:59:12.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.099+0000 7fdace23b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdac41ad270 con 0x55fbeeab8120 2024-09-10T09:59:12.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.100+0000 7fdac8e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fbeeab8120 0x55fbeeabe640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:12.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.100+0000 7fdac8e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fbeeab8120 0x55fbeeabe640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:45444/0 (socket says 172.21.15.77:45444) 2024-09-10T09:59:12.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.100+0000 7fdac8e00640 1 -- 172.21.15.77:0/2791804014 learned_addr learned my addr 172.21.15.77:0/2791804014 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:59:12.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.100+0000 7fdac3e00640 1 --2- 172.21.15.77:0/2791804014 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fbeeabbb90 0x55fbeeae2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:12.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.100+0000 7fdac9800640 1 --2- 172.21.15.77:0/2791804014 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fbeeabc4b0 0x55fbeeae3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:12.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.100+0000 7fdac8e00640 1 -- 172.21.15.77:0/2791804014 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fbeeabc4b0 msgr2=0x55fbeeae3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:12.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.100+0000 7fdac8e00640 1 --2- 172.21.15.77:0/2791804014 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fbeeabc4b0 0x55fbeeae3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:12.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.100+0000 7fdac8e00640 1 -- 172.21.15.77:0/2791804014 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fbeeabbb90 msgr2=0x55fbeeae2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:12.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.100+0000 7fdac8e00640 1 --2- 172.21.15.77:0/2791804014 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fbeeabbb90 0x55fbeeae2de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:12.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.100+0000 7fdac8e00640 1 -- 172.21.15.77:0/2791804014 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fbeeaddb50 con 0x55fbeeab8120 2024-09-10T09:59:12.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.100+0000 7fdac9800640 1 --2- 172.21.15.77:0/2791804014 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fbeeabc4b0 0x55fbeeae3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:59:12.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.100+0000 7fdac3e00640 1 --2- 172.21.15.77:0/2791804014 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fbeeabbb90 0x55fbeeae2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:59:12.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.100+0000 7fdac8e00640 1 --2- 172.21.15.77:0/2791804014 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fbeeab8120 0x55fbeeabe640 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x55fbeeaddd80 tx=0x7fdab801b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc5c390ca3ddf9c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:12.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.101+0000 7fdab5600640 1 -- 172.21.15.77:0/2791804014 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab801db50 con 0x55fbeeab8120 2024-09-10T09:59:12.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.101+0000 7fdab5600640 1 -- 172.21.15.77:0/2791804014 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdab8005500 con 0x55fbeeab8120 2024-09-10T09:59:12.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.101+0000 7fdab5600640 1 -- 172.21.15.77:0/2791804014 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab80035f0 con 0x55fbeeab8120 2024-09-10T09:59:12.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.101+0000 7fdace23b740 1 -- 172.21.15.77:0/2791804014 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fbeeab8120 msgr2=0x55fbeeabe640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:12.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.101+0000 7fdace23b740 1 --2- 172.21.15.77:0/2791804014 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fbeeab8120 0x55fbeeabe640 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x55fbeeaddd80 tx=0x7fdab801b000 comp rx=0 tx=0).stop 2024-09-10T09:59:12.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.102+0000 7fdace23b740 1 -- 172.21.15.77:0/2791804014 shutdown_connections 2024-09-10T09:59:12.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.102+0000 7fdace23b740 1 --2- 172.21.15.77:0/2791804014 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fbeeabc4b0 0x55fbeeae3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:12.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.102+0000 7fdace23b740 1 --2- 172.21.15.77:0/2791804014 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fbeeabbb90 0x55fbeeae2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:12.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.102+0000 7fdace23b740 1 --2- 172.21.15.77:0/2791804014 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fbeeab8120 0x55fbeeabe640 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:12.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.102+0000 7fdace23b740 1 -- 172.21.15.77:0/2791804014 >> 172.21.15.77:0/2791804014 conn(0x55fbeeac4900 msgr2=0x55fbeeac25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:12.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.102+0000 7fdace23b740 1 -- 172.21.15.77:0/2791804014 shutdown_connections 2024-09-10T09:59:12.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.102+0000 7fdace23b740 1 -- 172.21.15.77:0/2791804014 wait complete. 2024-09-10T09:59:12.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.102+0000 7fdace23b740 1 -- start start 2024-09-10T09:59:12.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.102+0000 7fdace23b740 1 Processor -- start 2024-09-10T09:59:12.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.103+0000 7fdace23b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fbeeab8120 0x55fbeeacdba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:12.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.103+0000 7fdace23b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fbeeabbb90 0x55fbeead35b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:12.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.103+0000 7fdace23b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fbeeabc4b0 0x55fbeead3b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:12.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.103+0000 7fdace23b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fdac41ae010 con 0x55fbeeabbb90 2024-09-10T09:59:12.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.103+0000 7fdace23b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdac41ad940 con 0x55fbeeab8120 2024-09-10T09:59:12.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.103+0000 7fdace23b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdac41aab80 con 0x55fbeeabc4b0 2024-09-10T09:59:12.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.103+0000 7fdace23b740 10 client.0 _ll_register_callbacks cb 0x7fdabc0b8bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:59:12.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.103+0000 7fdac8e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fbeeab8120 0x55fbeeacdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:12.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.103+0000 7fdac8e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fbeeab8120 0x55fbeeacdba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:52266/0 (socket says 172.21.15.77:52266) 2024-09-10T09:59:12.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.103+0000 7fdac8e00640 1 -- 172.21.15.77:0/2083542584 learned_addr learned my addr 172.21.15.77:0/2083542584 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:59:12.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.104+0000 7fdac8e00640 1 -- 172.21.15.77:0/2083542584 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fbeeabc4b0 msgr2=0x55fbeead3b10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T09:59:12.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.104+0000 7fdac8e00640 1 --2- 172.21.15.77:0/2083542584 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fbeeabc4b0 0x55fbeead3b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:12.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.104+0000 7fdac8e00640 1 -- 172.21.15.77:0/2083542584 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fbeeabbb90 msgr2=0x55fbeead35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:12.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.104+0000 7fdac8e00640 1 --2- 172.21.15.77:0/2083542584 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fbeeabbb90 0x55fbeead35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:12.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.104+0000 7fdac8e00640 1 -- 172.21.15.77:0/2083542584 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fbeeab79d0 con 0x55fbeeab8120 2024-09-10T09:59:12.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.104+0000 7fdac8e00640 1 --2- 172.21.15.77:0/2083542584 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fbeeab8120 0x55fbeeacdba0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fdab8013010 tx=0x7fdab8004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ddc90441f9531ff server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:12.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.104+0000 7fdaaaa00640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T09:59:12.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.104+0000 7fdaaaa00640 1 -- 172.21.15.77:0/2083542584 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab801db50 con 0x55fbeeab8120 2024-09-10T09:59:12.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.104+0000 7fdace23b740 1 -- 172.21.15.77:0/2083542584 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdab8005500 con 0x55fbeeab8120 2024-09-10T09:59:12.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.104+0000 7fdaaaa00640 1 -- 172.21.15.77:0/2083542584 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fdab801dcf0 con 0x55fbeeab8120 2024-09-10T09:59:12.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.104+0000 7fdaaaa00640 1 -- 172.21.15.77:0/2083542584 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fdab8045050 con 0x55fbeeab8120 2024-09-10T09:59:12.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.105+0000 7fdace23b740 10 client.26240 fetch_fsmap learned FSMap version 32 2024-09-10T09:59:12.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.105+0000 7fdace23b740 1 -- 172.21.15.77:0/2083542584 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55fbeeab8db0 con 0x55fbeeab8120 2024-09-10T09:59:12.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.105+0000 7fdaaaa00640 1 -- 172.21.15.77:0/2083542584 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdab8005500 con 0x55fbeeab8120 2024-09-10T09:59:12.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.105+0000 7fdaaaa00640 1 -- 172.21.15.77:0/2083542584 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fdab80336c0 con 0x55fbeeab8120 2024-09-10T09:59:12.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.105+0000 7fdace23b740 10 client.26240 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T09:59:12.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.105+0000 7fdace23b740 10 client.26240 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:59:12.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.105+0000 7fdace23b740 20 client.26240 populate_metadata read hostname 'smithi077' 2024-09-10T09:59:12.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.105+0000 7fdace23b740 1 --2- 172.21.15.77:0/2083542584 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55fbeeac6d70 0x55fbeeb4ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:12.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.105+0000 7fdace23b740 4 client.26240 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T09:59:12.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.105+0000 7fdace23b740 1 -- 172.21.15.77:0/2083542584 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fbeea29e60 con 0x55fbeeac6d70 2024-09-10T09:59:12.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.106+0000 7fdac3e00640 1 --2- 172.21.15.77:0/2083542584 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55fbeeac6d70 0x55fbeeb4ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:12.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.106+0000 7fdac3e00640 1 --2- 172.21.15.77:0/2083542584 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55fbeeac6d70 0x55fbeeb4ceb0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d1d14a00bf6d9e2 server_cookie=48cd88a1385532d0 in_seq=0 out_seq=0 2024-09-10T09:59:12.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.106+0000 7fdaaaa00640 10 client.26240 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T09:59:12.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.106+0000 7fdaaaa00640 1 -- 172.21.15.77:0/2083542584 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55fbeea29e60 con 0x55fbeeac6d70 2024-09-10T09:59:12.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.106+0000 7fdaaaa00640 10 client.26240 handle_command_reply: tid=0 2024-09-10T09:59:12.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.106+0000 7fdace23b740 1 client.26240 shutdown 2024-09-10T09:59:12.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.107+0000 7fdace23b740 1 -- 172.21.15.77:0/2083542584 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fbeeab8120 msgr2=0x55fbeeacdba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:12.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.107+0000 7fdace23b740 1 --2- 172.21.15.77:0/2083542584 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fbeeab8120 0x55fbeeacdba0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fdab8013010 tx=0x7fdab8004750 comp rx=0 tx=0).stop 2024-09-10T09:59:12.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.107+0000 7fdace23b740 1 -- 172.21.15.77:0/2083542584 shutdown_connections 2024-09-10T09:59:12.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.107+0000 7fdace23b740 1 --2- 172.21.15.77:0/2083542584 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55fbeeac6d70 0x55fbeeb4ceb0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:12.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.107+0000 7fdace23b740 1 --2- 172.21.15.77:0/2083542584 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fbeeabc4b0 0x55fbeead3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:12.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.107+0000 7fdaaaa00640 0 client.26240 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T09:59:12.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.107+0000 7fdace23b740 1 --2- 172.21.15.77:0/2083542584 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fbeeabbb90 0x55fbeead35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:12.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.107+0000 7fdace23b740 1 --2- 172.21.15.77:0/2083542584 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fbeeab8120 0x55fbeeacdba0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:12.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.107+0000 7fdace23b740 1 -- 172.21.15.77:0/2083542584 >> 172.21.15.77:0/2083542584 conn(0x55fbeeac4900 msgr2=0x55fbeeab69c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:12.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.107+0000 7fdace23b740 1 -- 172.21.15.77:0/2083542584 shutdown_connections 2024-09-10T09:59:12.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.107+0000 7fdace23b740 1 -- 172.21.15.77:0/2083542584 wait complete. 2024-09-10T09:59:12.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.107+0000 7fdace23b740 20 client.26240 trim_cache size 0 max 16384 2024-09-10T09:59:12.112 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-10T09:59:12.112 INFO:teuthology.orchestra.run.smithi077.stdout: "status": "no active scrubs running", 2024-09-10T09:59:12.112 INFO:teuthology.orchestra.run.smithi077.stdout: "scrubs": {} 2024-09-10T09:59:12.112 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-10T09:59:12.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.108+0000 7fdacb800640 1 -- 172.21.15.77:0/376636516 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fdaac0a2c00 msgr2=0x7fdaac0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:12.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.108+0000 7fdacb800640 1 --2- 172.21.15.77:0/376636516 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fdaac0a2c00 0x7fdaac0a50c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x55fbeeab8f80 tx=0x7fdab8005f90 comp rx=0 tx=0).stop 2024-09-10T09:59:12.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.108+0000 7fdacb800640 1 -- 172.21.15.77:0/376636516 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdac4106f00 msgr2=0x7fdac419cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:12.113 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.108+0000 7fdacb800640 1 --2- 172.21.15.77:0/376636516 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdac4106f00 0x7fdac419cc80 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fdabc02ade0 tx=0x7fdabc0319f0 comp rx=0 tx=0).stop 2024-09-10T09:59:12.113 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.108+0000 7fdacb800640 1 -- 172.21.15.77:0/376636516 shutdown_connections 2024-09-10T09:59:12.113 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.108+0000 7fdacb800640 1 --2- 172.21.15.77:0/376636516 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fdaac0a2c00 0x7fdaac0a50c0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:12.113 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.108+0000 7fdacb800640 1 --2- 172.21.15.77:0/376636516 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdac4106f00 0x7fdac419cc80 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:12.113 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.108+0000 7fdacb800640 1 --2- 172.21.15.77:0/376636516 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdac4102780 0x7fdac41988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:12.113 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.108+0000 7fdacb800640 1 --2- 172.21.15.77:0/376636516 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdac4101dd0 0x7fdac41983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:12.113 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.108+0000 7fdacb800640 1 -- 172.21.15.77:0/376636516 >> 172.21.15.77:0/376636516 conn(0x7fdac40fd8b0 msgr2=0x7fdac40fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:12.113 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.109+0000 7fdacb800640 1 -- 172.21.15.77:0/376636516 shutdown_connections 2024-09-10T09:59:12.113 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:12.109+0000 7fdacb800640 1 -- 172.21.15.77:0/376636516 wait complete. 2024-09-10T09:59:13.103 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:59:13.130 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T09:59:13.131 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:59:13.182 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:59:12 smithi077 ceph-mon[30319]: pgmap v512: 129 pgs: 129 active+clean; 94 MiB data, 866 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.2 MiB/s wr, 344 op/s 2024-09-10T09:59:13.225 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:59:12 smithi120 ceph-mon[34770]: pgmap v512: 129 pgs: 129 active+clean; 94 MiB data, 866 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.2 MiB/s wr, 344 op/s 2024-09-10T09:59:13.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:59:12 smithi134 ceph-mon[35674]: pgmap v512: 129 pgs: 129 active+clean; 94 MiB data, 866 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.2 MiB/s wr, 344 op/s 2024-09-10T09:59:13.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:13.686+0000 7fcc09a00640 1 -- 172.21.15.77:0/2229069414 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc04104010 msgr2=0x7fcc041043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:13.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:13.686+0000 7fcc09a00640 1 --2- 172.21.15.77:0/2229069414 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc04104010 0x7fcc041043f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fcbf4009900 tx=0x7fcbf402a3f0 comp rx=0 tx=0).stop 2024-09-10T09:59:13.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:13.686+0000 7fcc09a00640 1 -- 172.21.15.77:0/2229069414 shutdown_connections 2024-09-10T09:59:13.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:13.686+0000 7fcc09a00640 1 --2- 172.21.15.77:0/2229069414 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcc04109140 0x7fcc04110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:13.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:13.686+0000 7fcc09a00640 1 --2- 172.21.15.77:0/2229069414 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc041049c0 0x7fcc04108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:13.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:13.686+0000 7fcc09a00640 1 --2- 172.21.15.77:0/2229069414 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc04104010 0x7fcc041043f0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:13.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:13.686+0000 7fcc09a00640 1 -- 172.21.15.77:0/2229069414 >> 172.21.15.77:0/2229069414 conn(0x7fcc040ffa50 msgr2=0x7fcc04101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:13.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:13.687+0000 7fcc09a00640 1 -- 172.21.15.77:0/2229069414 shutdown_connections 2024-09-10T09:59:13.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:13.687+0000 7fcc09a00640 1 -- 172.21.15.77:0/2229069414 wait complete. 2024-09-10T09:59:13.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:13.688+0000 7fcc09a00640 1 Processor -- start 2024-09-10T09:59:13.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:13.688+0000 7fcc09a00640 1 -- start start 2024-09-10T09:59:13.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:13.688+0000 7fcc09a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcc04104010 0x7fcc0419a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:13.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:13.688+0000 7fcc09a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc041049c0 0x7fcc0419ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:13.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:13.688+0000 7fcc09a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc04109140 0x7fcc0419ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:13.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:13.688+0000 7fcc09a00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fcc041141c0 con 0x7fcc04109140 2024-09-10T09:59:13.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:13.688+0000 7fcc09a00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fcc04114040 con 0x7fcc04104010 2024-09-10T09:59:13.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:13.688+0000 7fcc09a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcc04114340 con 0x7fcc041049c0 2024-09-10T09:59:13.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:13.688+0000 7fcc02a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcc04104010 0x7fcc0419a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:13.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:13.688+0000 7fcc02a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcc04104010 0x7fcc0419a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:42004/0 (socket says 172.21.15.77:42004) 2024-09-10T09:59:13.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:13.688+0000 7fcc02a00640 1 -- 172.21.15.77:0/3063351233 learned_addr learned my addr 172.21.15.77:0/3063351233 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:59:13.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:13.689+0000 7fcc02000640 1 --2- 172.21.15.77:0/3063351233 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc041049c0 0x7fcc0419ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:13.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:13.689+0000 7fcc03400640 1 --2- 172.21.15.77:0/3063351233 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc04109140 0x7fcc0419ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:13.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:13.689+0000 7fcc02a00640 1 -- 172.21.15.77:0/3063351233 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc041049c0 msgr2=0x7fcc0419ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:13.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:13.689+0000 7fcc02a00640 1 --2- 172.21.15.77:0/3063351233 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc041049c0 0x7fcc0419ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:13.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:13.689+0000 7fcc02a00640 1 -- 172.21.15.77:0/3063351233 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc04109140 msgr2=0x7fcc0419ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:13.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:13.689+0000 7fcc02a00640 1 --2- 172.21.15.77:0/3063351233 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc04109140 0x7fcc0419ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:13.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:13.689+0000 7fcc02a00640 1 -- 172.21.15.77:0/3063351233 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc0419f5f0 con 0x7fcc04104010 2024-09-10T09:59:13.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:13.689+0000 7fcc02000640 1 --2- 172.21.15.77:0/3063351233 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc041049c0 0x7fcc0419ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:59:13.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:13.689+0000 7fcc02a00640 1 --2- 172.21.15.77:0/3063351233 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcc04104010 0x7fcc0419a640 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fcbf402a960 tx=0x7fcbf4003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:13.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:13.689+0000 7fcbfb400640 1 -- 172.21.15.77:0/3063351233 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf403f070 con 0x7fcc04104010 2024-09-10T09:59:13.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:13.689+0000 7fcbfb400640 1 -- 172.21.15.77:0/3063351233 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbf4031c90 con 0x7fcc04104010 2024-09-10T09:59:13.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:13.690+0000 7fcc09a00640 1 -- 172.21.15.77:0/3063351233 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc0419f880 con 0x7fcc04104010 2024-09-10T09:59:13.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:13.690+0000 7fcbfb400640 1 -- 172.21.15.77:0/3063351233 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf4035030 con 0x7fcc04104010 2024-09-10T09:59:13.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:13.690+0000 7fcc09a00640 1 -- 172.21.15.77:0/3063351233 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc041a71c0 con 0x7fcc04104010 2024-09-10T09:59:13.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:13.691+0000 7fcc09a00640 1 -- 172.21.15.77:0/3063351233 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc041043f0 con 0x7fcc04104010 2024-09-10T09:59:13.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:13.693+0000 7fcbfb400640 1 -- 172.21.15.77:0/3063351233 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fcbf4042070 con 0x7fcc04104010 2024-09-10T09:59:13.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:13.694+0000 7fcbfb400640 1 --2- 172.21.15.77:0/3063351233 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fcbe80a2c40 0x7fcbe80a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:13.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:13.694+0000 7fcbfb400640 1 -- 172.21.15.77:0/3063351233 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fcbf40f0480 con 0x7fcc04104010 2024-09-10T09:59:13.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:13.694+0000 7fcc02000640 1 --2- 172.21.15.77:0/3063351233 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fcbe80a2c40 0x7fcbe80a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:13.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:13.695+0000 7fcc02000640 1 --2- 172.21.15.77:0/3063351233 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fcbe80a2c40 0x7fcbe80a5100 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fcc040ffe70 tx=0x7fcbf0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:13.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:13.698+0000 7fcbfb400640 1 -- 172.21.15.77:0/3063351233 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcbf40bcf90 con 0x7fcc04104010 2024-09-10T09:59:13.747 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:59:13.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:13.945+0000 7fcc09a00640 1 -- 172.21.15.77:0/3063351233 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcc0419b9a0 con 0x7fcc04104010 2024-09-10T09:59:13.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:13.946+0000 7fcbfb400640 1 -- 172.21.15.77:0/3063351233 <== mon.2 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11996 (secure 0 0 0) 0x7fcbf40bc890 con 0x7fcc04104010 2024-09-10T09:59:13.948 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:59:13.948 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":32,"btime":"2024-09-10T09:57:40:888996+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3159289675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3159289675},{"type":"v1","addr":"172.21.15.77:6835","nonce":3159289675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1951962341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1951962341},{"type":"v1","addr":"172.21.15.77:6837","nonce":1951962341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:52:23.368252+0000","modified":"2024-09-10T09:57:40.695191+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24563,"mds_1":24524,"mds_2":24520,"mds_3":24502,"mds_4":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14487":{"gid":14487,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/4291121888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":4291121888},{"type":"v1","addr":"172.21.15.77:6839","nonce":4291121888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14499":{"gid":14499,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6841/445284194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":445284194},{"type":"v1","addr":"172.21.15.77:6841","nonce":445284194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1005054423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1005054423},{"type":"v1","addr":"172.21.15.134:6833","nonce":1005054423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6833/3716307744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":3716307744},{"type":"v1","addr":"172.21.15.120:6833","nonce":3716307744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/1319892544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1319892544},{"type":"v1","addr":"172.21.15.120:6835","nonce":1319892544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.134:6835/3293933699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3293933699},{"type":"v1","addr":"172.21.15.134:6835","nonce":3293933699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.134:6837/2461497091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2461497091},{"type":"v1","addr":"172.21.15.134:6837","nonce":2461497091}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":20,"addr":"172.21.15.134:6839/1965117680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1965117680},{"type":"v1","addr":"172.21.15.134:6839","nonce":1965117680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":20,"addr":"172.21.15.120:6837/3033280709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3033280709},{"type":"v1","addr":"172.21.15.120:6837","nonce":3033280709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.120:6839/1777710157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1777710157},{"type":"v1","addr":"172.21.15.120:6839","nonce":1777710157}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24563,"qdb_cluster":[24563,24524,24520,24502,24496]},"id":1}]} 2024-09-10T09:59:13.949 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 32 2024-09-10T09:59:13.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:13.949+0000 7fcc09a00640 1 -- 172.21.15.77:0/3063351233 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fcbe80a2c40 msgr2=0x7fcbe80a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:13.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:13.949+0000 7fcc09a00640 1 --2- 172.21.15.77:0/3063351233 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fcbe80a2c40 0x7fcbe80a5100 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fcc040ffe70 tx=0x7fcbf0005f90 comp rx=0 tx=0).stop 2024-09-10T09:59:13.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:13.949+0000 7fcc09a00640 1 -- 172.21.15.77:0/3063351233 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcc04104010 msgr2=0x7fcc0419a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:13.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:13.949+0000 7fcc09a00640 1 --2- 172.21.15.77:0/3063351233 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcc04104010 0x7fcc0419a640 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fcbf402a960 tx=0x7fcbf4003b90 comp rx=0 tx=0).stop 2024-09-10T09:59:13.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:13.949+0000 7fcc09a00640 1 -- 172.21.15.77:0/3063351233 shutdown_connections 2024-09-10T09:59:13.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:13.949+0000 7fcc09a00640 1 --2- 172.21.15.77:0/3063351233 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fcbe80a2c40 0x7fcbe80a5100 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:13.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:13.949+0000 7fcc09a00640 1 --2- 172.21.15.77:0/3063351233 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc04109140 0x7fcc0419ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:13.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:13.949+0000 7fcc09a00640 1 --2- 172.21.15.77:0/3063351233 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc041049c0 0x7fcc0419ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:13.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:13.949+0000 7fcc09a00640 1 --2- 172.21.15.77:0/3063351233 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcc04104010 0x7fcc0419a640 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:13.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:13.949+0000 7fcc09a00640 1 -- 172.21.15.77:0/3063351233 >> 172.21.15.77:0/3063351233 conn(0x7fcc040ffa50 msgr2=0x7fcc04100dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:13.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:13.950+0000 7fcc09a00640 1 -- 172.21.15.77:0/3063351233 shutdown_connections 2024-09-10T09:59:13.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:13.950+0000 7fcc09a00640 1 -- 172.21.15.77:0/3063351233 wait complete. 2024-09-10T09:59:14.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:14.493+0000 7f6a74800640 1 -- 172.21.15.77:0/2352826089 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a6c103ff0 msgr2=0x7f6a6c1043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:14.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:14.493+0000 7f6a74800640 1 --2- 172.21.15.77:0/2352826089 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a6c103ff0 0x7f6a6c1043d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6a60009900 tx=0x7f6a6002a400 comp rx=0 tx=0).stop 2024-09-10T09:59:14.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:14.493+0000 7f6a74800640 1 -- 172.21.15.77:0/2352826089 shutdown_connections 2024-09-10T09:59:14.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:14.493+0000 7f6a74800640 1 --2- 172.21.15.77:0/2352826089 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6a6c109120 0x7f6a6c110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:14.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:14.493+0000 7f6a74800640 1 --2- 172.21.15.77:0/2352826089 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a6c1049a0 0x7f6a6c1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:14.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:14.493+0000 7f6a74800640 1 --2- 172.21.15.77:0/2352826089 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a6c103ff0 0x7f6a6c1043d0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:14.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:14.493+0000 7f6a74800640 1 -- 172.21.15.77:0/2352826089 >> 172.21.15.77:0/2352826089 conn(0x7f6a6c0ffa90 msgr2=0x7f6a6c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:14.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:14.493+0000 7f6a74800640 1 -- 172.21.15.77:0/2352826089 shutdown_connections 2024-09-10T09:59:14.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:14.493+0000 7f6a74800640 1 -- 172.21.15.77:0/2352826089 wait complete. 2024-09-10T09:59:14.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:14.494+0000 7f6a74800640 1 Processor -- start 2024-09-10T09:59:14.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:14.494+0000 7f6a74800640 1 -- start start 2024-09-10T09:59:14.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:14.495+0000 7f6a74800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a6c103ff0 0x7f6a6c19a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:14.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:14.495+0000 7f6a74800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6a6c1049a0 0x7f6a6c19ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:14.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:14.495+0000 7f6a74800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a6c109120 0x7f6a6c19eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:14.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:14.495+0000 7f6a74800640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6a6c114150 con 0x7f6a6c103ff0 2024-09-10T09:59:14.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:14.495+0000 7f6a74800640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6a6c113fd0 con 0x7f6a6c1049a0 2024-09-10T09:59:14.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:14.495+0000 7f6a74800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6a6c1142d0 con 0x7f6a6c109120 2024-09-10T09:59:14.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:14.495+0000 7f6a72800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a6c109120 0x7f6a6c19eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:14.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:14.495+0000 7f6a71e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a6c103ff0 0x7f6a6c19a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:14.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:14.495+0000 7f6a71400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6a6c1049a0 0x7f6a6c19ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:14.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:14.495+0000 7f6a72800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a6c109120 0x7f6a6c19eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:53204/0 (socket says 172.21.15.77:53204) 2024-09-10T09:59:14.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:14.495+0000 7f6a72800640 1 -- 172.21.15.77:0/1152155645 learned_addr learned my addr 172.21.15.77:0/1152155645 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:59:14.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:14.496+0000 7f6a72800640 1 -- 172.21.15.77:0/1152155645 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6a6c1049a0 msgr2=0x7f6a6c19ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:14.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:14.496+0000 7f6a72800640 1 --2- 172.21.15.77:0/1152155645 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6a6c1049a0 0x7f6a6c19ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:14.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:14.496+0000 7f6a72800640 1 -- 172.21.15.77:0/1152155645 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a6c103ff0 msgr2=0x7f6a6c19a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:14.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:14.496+0000 7f6a72800640 1 --2- 172.21.15.77:0/1152155645 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a6c103ff0 0x7f6a6c19a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:14.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:14.496+0000 7f6a72800640 1 -- 172.21.15.77:0/1152155645 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a6c19f5a0 con 0x7f6a6c109120 2024-09-10T09:59:14.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:14.496+0000 7f6a72800640 1 --2- 172.21.15.77:0/1152155645 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a6c109120 0x7f6a6c19eec0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6a6800eb10 tx=0x7f6a6800c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:14.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:14.496+0000 7f6a66a00640 1 -- 172.21.15.77:0/1152155645 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a68004b20 con 0x7f6a6c109120 2024-09-10T09:59:14.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:14.497+0000 7f6a66a00640 1 -- 172.21.15.77:0/1152155645 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a68004cc0 con 0x7f6a6c109120 2024-09-10T09:59:14.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:14.497+0000 7f6a66a00640 1 -- 172.21.15.77:0/1152155645 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a68010760 con 0x7f6a6c109120 2024-09-10T09:59:14.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:14.497+0000 7f6a74800640 1 -- 172.21.15.77:0/1152155645 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a6c19f890 con 0x7f6a6c109120 2024-09-10T09:59:14.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:14.497+0000 7f6a74800640 1 -- 172.21.15.77:0/1152155645 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a6c1a7170 con 0x7f6a6c109120 2024-09-10T09:59:14.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:14.498+0000 7f6a74800640 1 -- 172.21.15.77:0/1152155645 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a6c1043d0 con 0x7f6a6c109120 2024-09-10T09:59:14.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:14.502+0000 7f6a66a00640 1 -- 172.21.15.77:0/1152155645 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6a68010900 con 0x7f6a6c109120 2024-09-10T09:59:14.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:14.504+0000 7f6a66a00640 1 --2- 172.21.15.77:0/1152155645 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f6a540a2c00 0x7f6a540a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:14.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:14.504+0000 7f6a66a00640 1 -- 172.21.15.77:0/1152155645 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f6a680cfd40 con 0x7f6a6c109120 2024-09-10T09:59:14.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:14.504+0000 7f6a66a00640 1 -- 172.21.15.77:0/1152155645 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6a6809c850 con 0x7f6a6c109120 2024-09-10T09:59:14.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:14.504+0000 7f6a71e00640 1 --2- 172.21.15.77:0/1152155645 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f6a540a2c00 0x7f6a540a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:14.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:14.505+0000 7f6a71e00640 1 --2- 172.21.15.77:0/1152155645 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f6a540a2c00 0x7f6a540a50c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6a60009230 tx=0x7f6a600058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:14.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:14.742+0000 7f6a74800640 1 -- 172.21.15.77:0/1152155645 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6a6c19b950 con 0x7f6a6c109120 2024-09-10T09:59:14.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:14.743+0000 7f6a66a00640 1 -- 172.21.15.77:0/1152155645 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11996 (secure 0 0 0) 0x7f6a6809c150 con 0x7f6a6c109120 2024-09-10T09:59:14.745 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:59:14.746 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":32,"btime":"2024-09-10T09:57:40:888996+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3159289675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3159289675},{"type":"v1","addr":"172.21.15.77:6835","nonce":3159289675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1951962341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1951962341},{"type":"v1","addr":"172.21.15.77:6837","nonce":1951962341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:52:23.368252+0000","modified":"2024-09-10T09:57:40.695191+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24563,"mds_1":24524,"mds_2":24520,"mds_3":24502,"mds_4":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14487":{"gid":14487,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/4291121888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":4291121888},{"type":"v1","addr":"172.21.15.77:6839","nonce":4291121888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14499":{"gid":14499,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6841/445284194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":445284194},{"type":"v1","addr":"172.21.15.77:6841","nonce":445284194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1005054423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1005054423},{"type":"v1","addr":"172.21.15.134:6833","nonce":1005054423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6833/3716307744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":3716307744},{"type":"v1","addr":"172.21.15.120:6833","nonce":3716307744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/1319892544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1319892544},{"type":"v1","addr":"172.21.15.120:6835","nonce":1319892544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.134:6835/3293933699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3293933699},{"type":"v1","addr":"172.21.15.134:6835","nonce":3293933699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.134:6837/2461497091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2461497091},{"type":"v1","addr":"172.21.15.134:6837","nonce":2461497091}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":20,"addr":"172.21.15.134:6839/1965117680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1965117680},{"type":"v1","addr":"172.21.15.134:6839","nonce":1965117680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":20,"addr":"172.21.15.120:6837/3033280709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3033280709},{"type":"v1","addr":"172.21.15.120:6837","nonce":3033280709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.120:6839/1777710157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1777710157},{"type":"v1","addr":"172.21.15.120:6839","nonce":1777710157}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24563,"qdb_cluster":[24563,24524,24520,24502,24496]},"id":1}]} 2024-09-10T09:59:14.746 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 32 2024-09-10T09:59:14.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:14.746+0000 7f6a74800640 1 -- 172.21.15.77:0/1152155645 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f6a540a2c00 msgr2=0x7f6a540a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:14.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:14.746+0000 7f6a74800640 1 --2- 172.21.15.77:0/1152155645 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f6a540a2c00 0x7f6a540a50c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6a60009230 tx=0x7f6a600058d0 comp rx=0 tx=0).stop 2024-09-10T09:59:14.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:14.746+0000 7f6a74800640 1 -- 172.21.15.77:0/1152155645 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a6c109120 msgr2=0x7f6a6c19eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:14.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:14.746+0000 7f6a74800640 1 --2- 172.21.15.77:0/1152155645 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a6c109120 0x7f6a6c19eec0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6a6800eb10 tx=0x7f6a6800c2d0 comp rx=0 tx=0).stop 2024-09-10T09:59:14.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:14.746+0000 7f6a74800640 1 -- 172.21.15.77:0/1152155645 shutdown_connections 2024-09-10T09:59:14.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:14.746+0000 7f6a74800640 1 --2- 172.21.15.77:0/1152155645 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f6a540a2c00 0x7f6a540a50c0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:14.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:14.746+0000 7f6a74800640 1 --2- 172.21.15.77:0/1152155645 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a6c109120 0x7f6a6c19eec0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:14.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:14.746+0000 7f6a74800640 1 --2- 172.21.15.77:0/1152155645 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6a6c1049a0 0x7f6a6c19ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:14.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:14.746+0000 7f6a74800640 1 --2- 172.21.15.77:0/1152155645 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a6c103ff0 0x7f6a6c19a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:14.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:14.746+0000 7f6a74800640 1 -- 172.21.15.77:0/1152155645 >> 172.21.15.77:0/1152155645 conn(0x7f6a6c0ffa90 msgr2=0x7f6a6c100e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:14.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:14.746+0000 7f6a74800640 1 -- 172.21.15.77:0/1152155645 shutdown_connections 2024-09-10T09:59:14.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:14.746+0000 7f6a74800640 1 -- 172.21.15.77:0/1152155645 wait complete. 2024-09-10T09:59:15.161 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24563 quiesce db / --timeout 90 --expiration 142.75 --await-for 45.0' 2024-09-10T09:59:15.161 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24563 quiesce db / --timeout 90 --expiration 142.75 --await-for 45.0 2024-09-10T09:59:15.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:59:14 smithi077 ceph-mon[30319]: pgmap v513: 129 pgs: 129 active+clean; 95 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.9 MiB/s wr, 429 op/s 2024-09-10T09:59:15.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:59:14 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/3063351233' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:59:15.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:59:14 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/1152155645' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:59:15.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:59:14 smithi134 ceph-mon[35674]: pgmap v513: 129 pgs: 129 active+clean; 95 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.9 MiB/s wr, 429 op/s 2024-09-10T09:59:15.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:59:14 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/3063351233' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:59:15.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:59:14 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/1152155645' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:59:15.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:59:14 smithi120 ceph-mon[34770]: pgmap v513: 129 pgs: 129 active+clean; 95 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.9 MiB/s wr, 429 op/s 2024-09-10T09:59:15.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:59:14 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/3063351233' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:59:15.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:59:14 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/1152155645' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:59:15.949 INFO:tasks.workunit.client.0.smithi077.stdout:copying to largest ever: 0xffdb8 2024-09-10T09:59:16.032 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T09:59:16.102 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:59:16.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.742+0000 7ff6f7a00640 1 -- 172.21.15.77:0/494962370 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6f0103650 msgr2=0x7ff6f010bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:16.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.742+0000 7ff6f7a00640 1 --2- 172.21.15.77:0/494962370 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6f0103650 0x7ff6f010bb40 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc009860 tx=0x7ff6dc02a3d0 comp rx=0 tx=0).stop 2024-09-10T09:59:16.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.742+0000 7ff6f7a00640 1 -- 172.21.15.77:0/494962370 shutdown_connections 2024-09-10T09:59:16.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.742+0000 7ff6f7a00640 1 --2- 172.21.15.77:0/494962370 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff6f010c160 0x7ff6f010e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:16.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.742+0000 7ff6f7a00640 1 --2- 172.21.15.77:0/494962370 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6f0103650 0x7ff6f010bb40 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:16.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.742+0000 7ff6f7a00640 1 --2- 172.21.15.77:0/494962370 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff6f0102d30 0x7ff6f0103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:16.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.742+0000 7ff6f7a00640 1 -- 172.21.15.77:0/494962370 >> 172.21.15.77:0/494962370 conn(0x7ff6f00fc790 msgr2=0x7ff6f00febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:16.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.742+0000 7ff6f7a00640 1 -- 172.21.15.77:0/494962370 shutdown_connections 2024-09-10T09:59:16.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.743+0000 7ff6f7a00640 1 -- 172.21.15.77:0/494962370 wait complete. 2024-09-10T09:59:16.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.744+0000 7ff6f7a00640 1 Processor -- start 2024-09-10T09:59:16.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.744+0000 7ff6f7a00640 1 -- start start 2024-09-10T09:59:16.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.744+0000 7ff6f7a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff6f0102d30 0x7ff6f019a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:16.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.744+0000 7ff6f7a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6f0103650 0x7ff6f019ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:16.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.744+0000 7ff6f7a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff6f010c160 0x7ff6f019ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:16.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.744+0000 7ff6f7a00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff6f0111f60 con 0x7ff6f0102d30 2024-09-10T09:59:16.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.744+0000 7ff6f7a00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff6f0111de0 con 0x7ff6f010c160 2024-09-10T09:59:16.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.744+0000 7ff6f7a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff6f01120e0 con 0x7ff6f0103650 2024-09-10T09:59:16.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.744+0000 7ff6f5000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff6f0102d30 0x7ff6f019a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:16.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.744+0000 7ff6f5000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff6f0102d30 0x7ff6f019a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:39622/0 (socket says 172.21.15.77:39622) 2024-09-10T09:59:16.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.744+0000 7ff6f5000640 1 -- 172.21.15.77:0/1078496411 learned_addr learned my addr 172.21.15.77:0/1078496411 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:59:16.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.744+0000 7ff6efe00640 1 --2- 172.21.15.77:0/1078496411 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6f0103650 0x7ff6f019ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:16.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.745+0000 7ff6f5a00640 1 --2- 172.21.15.77:0/1078496411 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff6f010c160 0x7ff6f019ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:16.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.745+0000 7ff6f5000640 1 -- 172.21.15.77:0/1078496411 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6f0103650 msgr2=0x7ff6f019ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:16.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.745+0000 7ff6f5000640 1 --2- 172.21.15.77:0/1078496411 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6f0103650 0x7ff6f019ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:16.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.745+0000 7ff6f5000640 1 -- 172.21.15.77:0/1078496411 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff6f010c160 msgr2=0x7ff6f019ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:16.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.745+0000 7ff6f5000640 1 --2- 172.21.15.77:0/1078496411 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff6f010c160 0x7ff6f019ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:16.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.745+0000 7ff6f5000640 1 -- 172.21.15.77:0/1078496411 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6f019f5e0 con 0x7ff6f0102d30 2024-09-10T09:59:16.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.745+0000 7ff6f5000640 1 --2- 172.21.15.77:0/1078496411 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff6f0102d30 0x7ff6f019a630 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7ff6e400e9c0 tx=0x7ff6e400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:16.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.746+0000 7ff6ed600640 1 -- 172.21.15.77:0/1078496411 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e400cde0 con 0x7ff6f0102d30 2024-09-10T09:59:16.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.746+0000 7ff6f7a00640 1 -- 172.21.15.77:0/1078496411 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6f019f8d0 con 0x7ff6f0102d30 2024-09-10T09:59:16.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.746+0000 7ff6ed600640 1 -- 172.21.15.77:0/1078496411 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6e40035f0 con 0x7ff6f0102d30 2024-09-10T09:59:16.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.746+0000 7ff6f7a00640 1 -- 172.21.15.77:0/1078496411 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6f01a71b0 con 0x7ff6f0102d30 2024-09-10T09:59:16.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.746+0000 7ff6ed600640 1 -- 172.21.15.77:0/1078496411 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e4010650 con 0x7ff6f0102d30 2024-09-10T09:59:16.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.749+0000 7ff6ed600640 1 -- 172.21.15.77:0/1078496411 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff6e4004000 con 0x7ff6f0102d30 2024-09-10T09:59:16.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.749+0000 7ff6fa663740 1 Processor -- start 2024-09-10T09:59:16.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.749+0000 7ff6fa663740 1 -- start start 2024-09-10T09:59:16.751 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.750+0000 7ff6fa663740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584f26e1400 0x5584f26e17e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:16.751 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.750+0000 7ff6fa663740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584f26e1d20 0x5584f26f4990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:16.751 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.750+0000 7ff6fa663740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584f26f4ed0 0x5584f26f72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:16.751 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.750+0000 7ff6fa663740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5584f2646260 con 0x5584f26e1d20 2024-09-10T09:59:16.751 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.750+0000 7ff6fa663740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5584f25ed500 con 0x5584f26f4ed0 2024-09-10T09:59:16.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.750+0000 7ff6fa663740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5584f2642220 con 0x5584f26e1400 2024-09-10T09:59:16.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.750+0000 7ff6f5000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584f26e1d20 0x5584f26f4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:16.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.750+0000 7ff6f5a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584f26e1400 0x5584f26e17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:16.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.750+0000 7ff6f5000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584f26e1d20 0x5584f26f4990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:39634/0 (socket says 172.21.15.77:39634) 2024-09-10T09:59:16.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.750+0000 7ff6f5000640 1 -- 172.21.15.77:0/3986180882 learned_addr learned my addr 172.21.15.77:0/3986180882 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:59:16.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.750+0000 7ff6f5a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584f26e1400 0x5584f26e17e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:53234/0 (socket says 172.21.15.77:53234) 2024-09-10T09:59:16.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.750+0000 7ff6efe00640 1 --2- 172.21.15.77:0/3986180882 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584f26f4ed0 0x5584f26f72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:16.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.751+0000 7ff6f5a00640 1 -- 172.21.15.77:0/3986180882 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584f26f4ed0 msgr2=0x5584f26f72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:16.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.751+0000 7ff6f5a00640 1 --2- 172.21.15.77:0/3986180882 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584f26f4ed0 0x5584f26f72c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:16.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.751+0000 7ff6f5a00640 1 -- 172.21.15.77:0/3986180882 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584f26e1d20 msgr2=0x5584f26f4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:16.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.751+0000 7ff6f5a00640 1 --2- 172.21.15.77:0/3986180882 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584f26e1d20 0x5584f26f4990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:16.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.751+0000 7ff6f5a00640 1 -- 172.21.15.77:0/3986180882 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584f23b7ba0 con 0x5584f26e1400 2024-09-10T09:59:16.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.751+0000 7ff6ed600640 1 --2- 172.21.15.77:0/1078496411 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ff6d80a2c00 0x7ff6d80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:16.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.751+0000 7ff6f5a00640 1 --2- 172.21.15.77:0/3986180882 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584f26e1400 0x5584f26e17e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff6e8009f00 tx=0x7ff6e8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0987ad4f2f1f770 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:16.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.751+0000 7ff6ed600640 1 -- 172.21.15.77:0/1078496411 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7ff6e40cf5d0 con 0x7ff6f0102d30 2024-09-10T09:59:16.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.751+0000 7ff6f5000640 1 --2- 172.21.15.77:0/1078496411 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ff6d80a2c00 0x7ff6d80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:16.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.751+0000 7ff6e2000640 1 -- 172.21.15.77:0/3986180882 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e8004d20 con 0x5584f26e1400 2024-09-10T09:59:16.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.752+0000 7ff6e2000640 1 -- 172.21.15.77:0/3986180882 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6e8004050 con 0x5584f26e1400 2024-09-10T09:59:16.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.752+0000 7ff6e2000640 1 -- 172.21.15.77:0/3986180882 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e8005260 con 0x5584f26e1400 2024-09-10T09:59:16.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.753+0000 7ff6fa663740 1 -- 172.21.15.77:0/3986180882 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584f26e1400 msgr2=0x5584f26e17e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:16.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.753+0000 7ff6f5000640 1 --2- 172.21.15.77:0/1078496411 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ff6d80a2c00 0x7ff6d80a50c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff6e40976c0 tx=0x7ff6e40d2f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:16.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.753+0000 7ff6fa663740 1 --2- 172.21.15.77:0/3986180882 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584f26e1400 0x5584f26e17e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff6e8009f00 tx=0x7ff6e8011810 comp rx=0 tx=0).stop 2024-09-10T09:59:16.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.753+0000 7ff6fa663740 1 -- 172.21.15.77:0/3986180882 shutdown_connections 2024-09-10T09:59:16.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.753+0000 7ff6fa663740 1 --2- 172.21.15.77:0/3986180882 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584f26f4ed0 0x5584f26f72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:16.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.753+0000 7ff6fa663740 1 --2- 172.21.15.77:0/3986180882 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584f26e1d20 0x5584f26f4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:16.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.753+0000 7ff6fa663740 1 --2- 172.21.15.77:0/3986180882 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584f26e1400 0x5584f26e17e0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:16.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.753+0000 7ff6fa663740 1 -- 172.21.15.77:0/3986180882 >> 172.21.15.77:0/3986180882 conn(0x5584f26dda70 msgr2=0x5584f26dde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:16.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.753+0000 7ff6fa663740 1 -- 172.21.15.77:0/3986180882 shutdown_connections 2024-09-10T09:59:16.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.753+0000 7ff6fa663740 1 -- 172.21.15.77:0/3986180882 wait complete. 2024-09-10T09:59:16.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.754+0000 7ff6fa663740 1 -- start start 2024-09-10T09:59:16.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.754+0000 7ff6fa663740 1 Processor -- start 2024-09-10T09:59:16.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.755+0000 7ff6fa663740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584f26e1400 0x5584f26f1e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:16.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.755+0000 7ff6fa663740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584f26e1d20 0x5584f26f23b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:16.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.755+0000 7ff6fa663740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584f26f4ed0 0x5584f26f28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:16.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.755+0000 7ff6fa663740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5584f26f7a40 con 0x5584f26e1400 2024-09-10T09:59:16.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.755+0000 7ff6fa663740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5584f2600bd0 con 0x5584f26e1d20 2024-09-10T09:59:16.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.755+0000 7ff6fa663740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5584f26f7bc0 con 0x5584f26f4ed0 2024-09-10T09:59:16.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.755+0000 7ff6fa663740 10 client.0 _ll_register_callbacks cb 0x5584f23550a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:59:16.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.755+0000 7ff6efe00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584f26e1d20 0x5584f26f23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:16.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.756+0000 7ff6efe00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584f26e1d20 0x5584f26f23b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:42052/0 (socket says 172.21.15.77:42052) 2024-09-10T09:59:16.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.756+0000 7ff6efe00640 1 -- 172.21.15.77:0/724157035 learned_addr learned my addr 172.21.15.77:0/724157035 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:59:16.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.756+0000 7ff6f5a00640 1 --2- 172.21.15.77:0/724157035 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584f26e1400 0x5584f26f1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:16.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.756+0000 7ff6efe00640 1 -- 172.21.15.77:0/724157035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584f26f4ed0 msgr2=0x5584f26f28f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T09:59:16.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.756+0000 7ff6f5a00640 1 --2- 172.21.15.77:0/724157035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584f26f4ed0 0x5584f26f28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:16.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.756+0000 7ff6efe00640 1 --2- 172.21.15.77:0/724157035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584f26f4ed0 0x5584f26f28f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:16.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.756+0000 7ff6efe00640 1 -- 172.21.15.77:0/724157035 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584f26e1400 msgr2=0x5584f26f1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:16.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.756+0000 7ff6efe00640 1 --2- 172.21.15.77:0/724157035 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584f26e1400 0x5584f26f1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:16.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.756+0000 7ff6efe00640 1 -- 172.21.15.77:0/724157035 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584f2701030 con 0x5584f26e1d20 2024-09-10T09:59:16.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.756+0000 7ff6f5a00640 1 --2- 172.21.15.77:0/724157035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584f26f4ed0 0x5584f26f28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:59:16.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.756+0000 7ff6efe00640 1 --2- 172.21.15.77:0/724157035 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584f26e1d20 0x5584f26f23b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc009860 tx=0x7ff6dc009490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=775545e3216b1f44 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:16.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.756+0000 7ff6d6a00640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T09:59:16.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.757+0000 7ff6d6a00640 1 -- 172.21.15.77:0/724157035 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6dc0307b0 con 0x5584f26e1d20 2024-09-10T09:59:16.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.757+0000 7ff6fa663740 1 -- 172.21.15.77:0/724157035 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6e8004050 con 0x5584f26e1d20 2024-09-10T09:59:16.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.757+0000 7ff6d6a00640 1 -- 172.21.15.77:0/724157035 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7ff6dc005050 con 0x5584f26e1d20 2024-09-10T09:59:16.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.757+0000 7ff6d6a00640 1 -- 172.21.15.77:0/724157035 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff6dc04e050 con 0x5584f26e1d20 2024-09-10T09:59:16.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.757+0000 7ff6fa663740 10 client.26264 fetch_fsmap learned FSMap version 32 2024-09-10T09:59:16.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.757+0000 7ff6fa663740 1 -- 172.21.15.77:0/724157035 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5584f2703a20 con 0x5584f26e1d20 2024-09-10T09:59:16.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.757+0000 7ff6d6a00640 1 -- 172.21.15.77:0/724157035 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6e8004050 con 0x5584f26e1d20 2024-09-10T09:59:16.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.758+0000 7ff6d6a00640 1 -- 172.21.15.77:0/724157035 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7ff6dc004c10 con 0x5584f26e1d20 2024-09-10T09:59:16.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.758+0000 7ff6fa663740 10 client.26264 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T09:59:16.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.758+0000 7ff6fa663740 10 client.26264 resolve_mds: validated gid 24563 aka daemon mds.b 2024-09-10T09:59:16.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.758+0000 7ff6fa663740 20 client.26264 populate_metadata read hostname 'smithi077' 2024-09-10T09:59:16.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.758+0000 7ff6fa663740 1 --2- 172.21.15.77:0/724157035 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x5584f2704ef0 0x5584f2707350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:16.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.758+0000 7ff6fa663740 4 client.26264 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:59:16.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.758+0000 7ff6fa663740 1 -- 172.21.15.77:0/724157035 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5584f2653990 con 0x5584f2704ef0 2024-09-10T09:59:16.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.759+0000 7ff6f5a00640 1 --2- 172.21.15.77:0/724157035 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x5584f2704ef0 0x5584f2707350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:16.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.759+0000 7ff6f5a00640 1 --2- 172.21.15.77:0/724157035 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x5584f2704ef0 0x5584f2707350 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf5540318e8147f8 server_cookie=118df4ce374e8134 in_seq=0 out_seq=0 2024-09-10T09:59:16.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.759+0000 7ff6d6a00640 10 client.26264 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T09:59:16.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.760+0000 7ff6d6a00640 1 -- 172.21.15.77:0/724157035 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5584f2653990 con 0x5584f2704ef0 2024-09-10T09:59:16.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.760+0000 7ff6d6a00640 10 client.26264 handle_command_reply: tid=0 2024-09-10T09:59:16.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.760+0000 7ff6fa663740 1 client.26264 shutdown 2024-09-10T09:59:16.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.760+0000 7ff6fa663740 1 -- 172.21.15.77:0/724157035 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584f26e1d20 msgr2=0x5584f26f23b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:16.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.760+0000 7ff6fa663740 1 --2- 172.21.15.77:0/724157035 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584f26e1d20 0x5584f26f23b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc009860 tx=0x7ff6dc009490 comp rx=0 tx=0).stop 2024-09-10T09:59:16.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.761+0000 7ff6fa663740 1 -- 172.21.15.77:0/724157035 shutdown_connections 2024-09-10T09:59:16.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.761+0000 7ff6fa663740 1 --2- 172.21.15.77:0/724157035 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x5584f2704ef0 0x5584f2707350 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:16.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.761+0000 7ff6fa663740 1 --2- 172.21.15.77:0/724157035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584f26f4ed0 0x5584f26f28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:16.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.761+0000 7ff6fa663740 1 --2- 172.21.15.77:0/724157035 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584f26e1d20 0x5584f26f23b0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:16.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.761+0000 7ff6fa663740 1 --2- 172.21.15.77:0/724157035 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584f26e1400 0x5584f26f1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:16.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.761+0000 7ff6fa663740 1 -- 172.21.15.77:0/724157035 >> 172.21.15.77:0/724157035 conn(0x5584f26dda70 msgr2=0x5584f26dcbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:16.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.761+0000 7ff6d6a00640 0 client.26264 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T09:59:16.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.761+0000 7ff6fa663740 1 -- 172.21.15.77:0/724157035 shutdown_connections 2024-09-10T09:59:16.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.761+0000 7ff6fa663740 1 -- 172.21.15.77:0/724157035 wait complete. 2024-09-10T09:59:16.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.761+0000 7ff6fa663740 20 client.26264 trim_cache size 0 max 16384 2024-09-10T09:59:16.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.782+0000 7ff6fa663740 1 Processor -- start 2024-09-10T09:59:16.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.782+0000 7ff6fa663740 1 -- start start 2024-09-10T09:59:16.784 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.783+0000 7ff6fa663740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584f26e1400 0x5584f26f3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:16.784 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.783+0000 7ff6fa663740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584f26f4ed0 0x5584f270cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:16.784 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.783+0000 7ff6fa663740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584f26f0f60 0x5584f270d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:16.784 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.783+0000 7ff6fa663740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d000fd20 con 0x5584f26e1400 2024-09-10T09:59:16.784 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.783+0000 7ff6fa663740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d0011920 con 0x5584f26f4ed0 2024-09-10T09:59:16.784 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.783+0000 7ff6fa663740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d00103f0 con 0x5584f26f0f60 2024-09-10T09:59:16.784 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:59:16.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.783+0000 7ff6f5a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584f26e1400 0x5584f26f3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:16.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.783+0000 7ff6efe00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584f26f4ed0 0x5584f270cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:16.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.783+0000 7ff6f5a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584f26e1400 0x5584f26f3cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:39664/0 (socket says 172.21.15.77:39664) 2024-09-10T09:59:16.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.783+0000 7ff6efe00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584f26f4ed0 0x5584f270cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:42060/0 (socket says 172.21.15.77:42060) 2024-09-10T09:59:16.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.783+0000 7ff6f5a00640 1 -- 172.21.15.77:0/418591237 learned_addr learned my addr 172.21.15.77:0/418591237 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:59:16.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.783+0000 7ff6f5a00640 1 --2- 172.21.15.77:0/418591237 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584f26f0f60 0x5584f270d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:16.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.784+0000 7ff6efe00640 1 -- 172.21.15.77:0/418591237 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584f26f0f60 msgr2=0x5584f270d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:16.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.784+0000 7ff6efe00640 1 --2- 172.21.15.77:0/418591237 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584f26f0f60 0x5584f270d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:16.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.784+0000 7ff6efe00640 1 -- 172.21.15.77:0/418591237 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584f26e1400 msgr2=0x5584f26f3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:16.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.784+0000 7ff6efe00640 1 --2- 172.21.15.77:0/418591237 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584f26e1400 0x5584f26f3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:16.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.784+0000 7ff6efe00640 1 -- 172.21.15.77:0/418591237 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584f26ecca0 con 0x5584f26f4ed0 2024-09-10T09:59:16.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.784+0000 7ff6efe00640 1 --2- 172.21.15.77:0/418591237 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584f26f4ed0 0x5584f270cc60 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x5584f26f3420 tx=0x7ff6dc032f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ecdcc42b705c2ce server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:16.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.784+0000 7ff6e1600640 1 -- 172.21.15.77:0/418591237 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6dc005c90 con 0x5584f26f4ed0 2024-09-10T09:59:16.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.784+0000 7ff6e1600640 1 -- 172.21.15.77:0/418591237 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6dc04b040 con 0x5584f26f4ed0 2024-09-10T09:59:16.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.784+0000 7ff6e1600640 1 -- 172.21.15.77:0/418591237 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6dc0057a0 con 0x5584f26f4ed0 2024-09-10T09:59:16.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.785+0000 7ff6fa663740 1 -- 172.21.15.77:0/418591237 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584f26f4ed0 msgr2=0x5584f270cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:16.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.785+0000 7ff6fa663740 1 --2- 172.21.15.77:0/418591237 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584f26f4ed0 0x5584f270cc60 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x5584f26f3420 tx=0x7ff6dc032f40 comp rx=0 tx=0).stop 2024-09-10T09:59:16.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.785+0000 7ff6fa663740 1 -- 172.21.15.77:0/418591237 shutdown_connections 2024-09-10T09:59:16.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.785+0000 7ff6fa663740 1 --2- 172.21.15.77:0/418591237 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584f26f0f60 0x5584f270d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:16.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.785+0000 7ff6fa663740 1 --2- 172.21.15.77:0/418591237 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584f26f4ed0 0x5584f270cc60 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:16.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.785+0000 7ff6fa663740 1 --2- 172.21.15.77:0/418591237 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584f26e1400 0x5584f26f3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:16.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.785+0000 7ff6fa663740 1 -- 172.21.15.77:0/418591237 >> 172.21.15.77:0/418591237 conn(0x5584f26dda70 msgr2=0x5584f26f35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:16.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.785+0000 7ff6fa663740 1 -- 172.21.15.77:0/418591237 shutdown_connections 2024-09-10T09:59:16.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.786+0000 7ff6fa663740 1 -- 172.21.15.77:0/418591237 wait complete. 2024-09-10T09:59:16.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.786+0000 7ff6fa663740 1 -- start start 2024-09-10T09:59:16.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.786+0000 7ff6fa663740 1 Processor -- start 2024-09-10T09:59:16.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.787+0000 7ff6fa663740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584f26e1400 0x5584f26f8be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:16.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.787+0000 7ff6fa663740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584f26f0f60 0x5584f26f9120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:16.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.787+0000 7ff6fa663740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584f26f4ed0 0x5584f26f9660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:16.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.787+0000 7ff6fa663740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d0011190 con 0x5584f26f4ed0 2024-09-10T09:59:16.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.787+0000 7ff6fa663740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d0010ac0 con 0x5584f26e1400 2024-09-10T09:59:16.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.787+0000 7ff6fa663740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d000dd00 con 0x5584f26f0f60 2024-09-10T09:59:16.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.787+0000 7ff6fa663740 10 client.0 _ll_register_callbacks cb 0x7ff6e4005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:59:16.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.787+0000 7ff6efe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584f26f0f60 0x5584f26f9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:16.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.787+0000 7ff6efe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584f26f0f60 0x5584f26f9120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:53264/0 (socket says 172.21.15.77:53264) 2024-09-10T09:59:16.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.787+0000 7ff6efe00640 1 -- 172.21.15.77:0/2078271509 learned_addr learned my addr 172.21.15.77:0/2078271509 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:59:16.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.787+0000 7ff6f5a00640 1 --2- 172.21.15.77:0/2078271509 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584f26e1400 0x5584f26f8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:16.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.787+0000 7ff6f5a00640 1 --2- 172.21.15.77:0/2078271509 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584f26f4ed0 0x5584f26f9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:16.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.787+0000 7ff6f5a00640 1 -- 172.21.15.77:0/2078271509 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584f26f0f60 msgr2=0x5584f26f9120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:16.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.787+0000 7ff6f5a00640 1 --2- 172.21.15.77:0/2078271509 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584f26f0f60 0x5584f26f9120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:16.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.787+0000 7ff6f5a00640 1 -- 172.21.15.77:0/2078271509 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584f26f4ed0 msgr2=0x5584f26f9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:16.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.787+0000 7ff6f5a00640 1 --2- 172.21.15.77:0/2078271509 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584f26f4ed0 0x5584f26f9660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:16.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.787+0000 7ff6f5a00640 1 -- 172.21.15.77:0/2078271509 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584f276ec30 con 0x5584f26e1400 2024-09-10T09:59:16.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.788+0000 7ff6f5a00640 1 --2- 172.21.15.77:0/2078271509 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584f26e1400 0x5584f26f8be0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff6e8009d40 tx=0x7ff6e802e040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef5a4daf65df0ae2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:16.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.788+0000 7ff6d6a00640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T09:59:16.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.788+0000 7ff6d6a00640 1 -- 172.21.15.77:0/2078271509 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e80045b0 con 0x5584f26e1400 2024-09-10T09:59:16.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.788+0000 7ff6d6a00640 1 -- 172.21.15.77:0/2078271509 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7ff6e8019430 con 0x5584f26e1400 2024-09-10T09:59:16.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.788+0000 7ff6fa663740 1 -- 172.21.15.77:0/2078271509 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6dc04b040 con 0x5584f26e1400 2024-09-10T09:59:16.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.788+0000 7ff6d6a00640 1 -- 172.21.15.77:0/2078271509 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff6e8019630 con 0x5584f26e1400 2024-09-10T09:59:16.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.788+0000 7ff6fa663740 10 client.26270 fetch_fsmap learned FSMap version 32 2024-09-10T09:59:16.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.788+0000 7ff6fa663740 1 -- 172.21.15.77:0/2078271509 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5584f276fe90 con 0x5584f26e1400 2024-09-10T09:59:16.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.789+0000 7ff6d6a00640 1 -- 172.21.15.77:0/2078271509 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6dc04b040 con 0x5584f26e1400 2024-09-10T09:59:16.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.789+0000 7ff6d6a00640 1 -- 172.21.15.77:0/2078271509 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7ff6e8026770 con 0x5584f26e1400 2024-09-10T09:59:16.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.789+0000 7ff6fa663740 10 client.26270 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T09:59:16.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.789+0000 7ff6fa663740 10 client.26270 resolve_mds: validated gid 24563 aka daemon mds.b 2024-09-10T09:59:16.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.789+0000 7ff6fa663740 20 client.26270 populate_metadata read hostname 'smithi077' 2024-09-10T09:59:16.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.789+0000 7ff6fa663740 1 --2- 172.21.15.77:0/2078271509 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x5584f27704a0 0x5584f27728c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:16.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.789+0000 7ff6fa663740 4 client.26270 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.75, "await_for": 45.0}] 2024-09-10T09:59:16.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.789+0000 7ff6fa663740 1 -- 172.21.15.77:0/2078271509 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.75, "await_for": 45.0}) -- 0x5584f264dce0 con 0x5584f27704a0 2024-09-10T09:59:16.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.789+0000 7ff6efe00640 1 --2- 172.21.15.77:0/2078271509 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x5584f27704a0 0x5584f27728c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:16.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.790+0000 7ff6efe00640 1 --2- 172.21.15.77:0/2078271509 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x5584f27704a0 0x5584f27728c0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f32825d054f86b53 server_cookie=e2f6a9e65441896d in_seq=0 out_seq=0 2024-09-10T09:59:16.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:16.790+0000 7ff6d6a00640 10 client.26270 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T09:59:17.156 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:59:16 smithi077 ceph-mon[30319]: pgmap v514: 129 pgs: 129 active+clean; 95 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 327 op/s 2024-09-10T09:59:17.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:59:16 smithi134 ceph-mon[35674]: pgmap v514: 129 pgs: 129 active+clean; 95 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 327 op/s 2024-09-10T09:59:17.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.417+0000 7ff6d6a00640 1 -- 172.21.15.77:0/2078271509 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5584f264dce0 con 0x5584f27704a0 2024-09-10T09:59:17.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.417+0000 7ff6d6a00640 10 client.26270 handle_command_reply: tid=0 2024-09-10T09:59:17.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.417+0000 7ff6fa663740 1 client.26270 shutdown 2024-09-10T09:59:17.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.418+0000 7ff6fa663740 1 -- 172.21.15.77:0/2078271509 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584f26e1400 msgr2=0x5584f26f8be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:17.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.418+0000 7ff6fa663740 1 --2- 172.21.15.77:0/2078271509 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584f26e1400 0x5584f26f8be0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff6e8009d40 tx=0x7ff6e802e040 comp rx=0 tx=0).stop 2024-09-10T09:59:17.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.418+0000 7ff6fa663740 1 -- 172.21.15.77:0/2078271509 shutdown_connections 2024-09-10T09:59:17.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.418+0000 7ff6fa663740 1 --2- 172.21.15.77:0/2078271509 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x5584f27704a0 0x5584f27728c0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:17.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.418+0000 7ff6fa663740 1 --2- 172.21.15.77:0/2078271509 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584f26f4ed0 0x5584f26f9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:17.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.418+0000 7ff6fa663740 1 --2- 172.21.15.77:0/2078271509 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584f26f0f60 0x5584f26f9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:17.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.418+0000 7ff6fa663740 1 --2- 172.21.15.77:0/2078271509 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584f26e1400 0x5584f26f8be0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:17.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.418+0000 7ff6d6a00640 0 client.26270 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T09:59:17.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.418+0000 7ff6fa663740 1 -- 172.21.15.77:0/2078271509 >> 172.21.15.77:0/2078271509 conn(0x5584f26dda70 msgr2=0x5584f26e3670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:17.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.418+0000 7ff6fa663740 1 -- 172.21.15.77:0/2078271509 shutdown_connections 2024-09-10T09:59:17.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.419+0000 7ff6fa663740 1 -- 172.21.15.77:0/2078271509 wait complete. 2024-09-10T09:59:17.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.419+0000 7ff6fa663740 20 client.26270 trim_cache size 0 max 16384 2024-09-10T09:59:17.421 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-10T09:59:17.421 INFO:teuthology.orchestra.run.smithi077.stdout: "epoch": 32, 2024-09-10T09:59:17.421 INFO:teuthology.orchestra.run.smithi077.stdout: "leader": 24563, 2024-09-10T09:59:17.421 INFO:teuthology.orchestra.run.smithi077.stdout: "set_version": 10, 2024-09-10T09:59:17.421 INFO:teuthology.orchestra.run.smithi077.stdout: "sets": { 2024-09-10T09:59:17.421 INFO:teuthology.orchestra.run.smithi077.stdout: "95a54fcc": { 2024-09-10T09:59:17.421 INFO:teuthology.orchestra.run.smithi077.stdout: "version": 10, 2024-09-10T09:59:17.422 INFO:teuthology.orchestra.run.smithi077.stdout: "age_ref": 0.0, 2024-09-10T09:59:17.422 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-10T09:59:17.422 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "QUIESCED", 2024-09-10T09:59:17.422 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-10T09:59:17.422 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T09:59:17.422 INFO:teuthology.orchestra.run.smithi077.stdout: "timeout": 90.0, 2024-09-10T09:59:17.422 INFO:teuthology.orchestra.run.smithi077.stdout: "expiration": 142.8, 2024-09-10T09:59:17.422 INFO:teuthology.orchestra.run.smithi077.stdout: "members": { 2024-09-10T09:59:17.422 INFO:teuthology.orchestra.run.smithi077.stdout: "file:/": { 2024-09-10T09:59:17.422 INFO:teuthology.orchestra.run.smithi077.stdout: "excluded": false, 2024-09-10T09:59:17.422 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-10T09:59:17.422 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "QUIESCED", 2024-09-10T09:59:17.423 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-10T09:59:17.423 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T09:59:17.423 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T09:59:17.423 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T09:59:17.423 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T09:59:17.423 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T09:59:17.423 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-10T09:59:17.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.420+0000 7ff6f7a00640 1 -- 172.21.15.77:0/1078496411 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ff6d80a2c00 msgr2=0x7ff6d80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:17.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.420+0000 7ff6f7a00640 1 --2- 172.21.15.77:0/1078496411 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ff6d80a2c00 0x7ff6d80a50c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff6e40976c0 tx=0x7ff6e40d2f90 comp rx=0 tx=0).stop 2024-09-10T09:59:17.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.420+0000 7ff6f7a00640 1 -- 172.21.15.77:0/1078496411 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff6f0102d30 msgr2=0x7ff6f019a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:17.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.420+0000 7ff6f7a00640 1 --2- 172.21.15.77:0/1078496411 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff6f0102d30 0x7ff6f019a630 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7ff6e400e9c0 tx=0x7ff6e400ee80 comp rx=0 tx=0).stop 2024-09-10T09:59:17.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.420+0000 7ff6f7a00640 1 -- 172.21.15.77:0/1078496411 shutdown_connections 2024-09-10T09:59:17.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.420+0000 7ff6f7a00640 1 --2- 172.21.15.77:0/1078496411 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ff6d80a2c00 0x7ff6d80a50c0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:17.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.420+0000 7ff6f7a00640 1 --2- 172.21.15.77:0/1078496411 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff6f010c160 0x7ff6f019ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:17.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.420+0000 7ff6f7a00640 1 --2- 172.21.15.77:0/1078496411 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6f0103650 0x7ff6f019ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:17.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.420+0000 7ff6f7a00640 1 --2- 172.21.15.77:0/1078496411 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff6f0102d30 0x7ff6f019a630 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:17.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.420+0000 7ff6f7a00640 1 -- 172.21.15.77:0/1078496411 >> 172.21.15.77:0/1078496411 conn(0x7ff6f00fc790 msgr2=0x7ff6f0100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:17.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.420+0000 7ff6f7a00640 1 -- 172.21.15.77:0/1078496411 shutdown_connections 2024-09-10T09:59:17.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.420+0000 7ff6f7a00640 1 -- 172.21.15.77:0/1078496411 wait complete. 2024-09-10T09:59:17.425 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:59:16 smithi120 ceph-mon[34770]: pgmap v514: 129 pgs: 129 active+clean; 95 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 327 op/s 2024-09-10T09:59:17.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.425+0000 7fe529000640 1 -- 172.21.15.77:0/797104972 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe524103510 msgr2=0x7fe52410ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:17.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.425+0000 7fe529000640 1 --2- 172.21.15.77:0/797104972 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe524103510 0x7fe52410ba00 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe510009900 tx=0x7fe51002a480 comp rx=0 tx=0).stop 2024-09-10T09:59:17.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.425+0000 7fe529000640 1 -- 172.21.15.77:0/797104972 shutdown_connections 2024-09-10T09:59:17.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.425+0000 7fe529000640 1 --2- 172.21.15.77:0/797104972 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe52410bfd0 0x7fe52410e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:17.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.426+0000 7fe529000640 1 --2- 172.21.15.77:0/797104972 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe524103510 0x7fe52410ba00 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:17.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.426+0000 7fe529000640 1 --2- 172.21.15.77:0/797104972 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe524102bf0 0x7fe524102fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:17.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.426+0000 7fe529000640 1 -- 172.21.15.77:0/797104972 >> 172.21.15.77:0/797104972 conn(0x7fe5240fc610 msgr2=0x7fe5240fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:17.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.426+0000 7fe529000640 1 -- 172.21.15.77:0/797104972 shutdown_connections 2024-09-10T09:59:17.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.426+0000 7fe529000640 1 -- 172.21.15.77:0/797104972 wait complete. 2024-09-10T09:59:17.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.427+0000 7fe529000640 1 Processor -- start 2024-09-10T09:59:17.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.427+0000 7fe529000640 1 -- start start 2024-09-10T09:59:17.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.428+0000 7fe529000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe524102bf0 0x7fe524198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:17.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.428+0000 7fe529000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe524103510 0x7fe5241987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:17.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.428+0000 7fe529000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe52410bfd0 0x7fe52419cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:17.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.428+0000 7fe529000640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe524111e00 con 0x7fe52410bfd0 2024-09-10T09:59:17.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.428+0000 7fe529000640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe524111c80 con 0x7fe524102bf0 2024-09-10T09:59:17.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.428+0000 7fe529000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe524111f80 con 0x7fe524103510 2024-09-10T09:59:17.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.428+0000 7fe523400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe524102bf0 0x7fe524198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:17.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.428+0000 7fe523400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe524102bf0 0x7fe524198260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:42090/0 (socket says 172.21.15.77:42090) 2024-09-10T09:59:17.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.428+0000 7fe522a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe524103510 0x7fe5241987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:17.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.428+0000 7fe523400640 1 -- 172.21.15.77:0/1316081979 learned_addr learned my addr 172.21.15.77:0/1316081979 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:59:17.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.428+0000 7fe523e00640 1 --2- 172.21.15.77:0/1316081979 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe52410bfd0 0x7fe52419cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:17.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.428+0000 7fe523400640 1 -- 172.21.15.77:0/1316081979 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe524103510 msgr2=0x7fe5241987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:17.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.428+0000 7fe523400640 1 --2- 172.21.15.77:0/1316081979 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe524103510 0x7fe5241987a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:17.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.428+0000 7fe523400640 1 -- 172.21.15.77:0/1316081979 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe52410bfd0 msgr2=0x7fe52419cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:17.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.428+0000 7fe523400640 1 --2- 172.21.15.77:0/1316081979 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe52410bfd0 0x7fe52419cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:17.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.428+0000 7fe523400640 1 -- 172.21.15.77:0/1316081979 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe52419d210 con 0x7fe524102bf0 2024-09-10T09:59:17.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.429+0000 7fe523400640 1 --2- 172.21.15.77:0/1316081979 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe524102bf0 0x7fe524198260 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe51400e9f0 tx=0x7fe51400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:17.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.429+0000 7fe51be00640 1 -- 172.21.15.77:0/1316081979 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe51400cd80 con 0x7fe524102bf0 2024-09-10T09:59:17.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.429+0000 7fe51be00640 1 -- 172.21.15.77:0/1316081979 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe514010910 con 0x7fe524102bf0 2024-09-10T09:59:17.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.429+0000 7fe51be00640 1 -- 172.21.15.77:0/1316081979 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe514018990 con 0x7fe524102bf0 2024-09-10T09:59:17.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.429+0000 7fe529000640 1 -- 172.21.15.77:0/1316081979 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe52419d500 con 0x7fe524102bf0 2024-09-10T09:59:17.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.429+0000 7fe529000640 1 -- 172.21.15.77:0/1316081979 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5241a4de0 con 0x7fe524102bf0 2024-09-10T09:59:17.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.432+0000 7fe51be00640 1 -- 172.21.15.77:0/1316081979 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe514010ac0 con 0x7fe524102bf0 2024-09-10T09:59:17.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.432+0000 7fe52d01c740 1 Processor -- start 2024-09-10T09:59:17.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.433+0000 7fe52d01c740 1 -- start start 2024-09-10T09:59:17.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.433+0000 7fe52d01c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555c0d9e8e40 0x555c0d9e9220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:17.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.433+0000 7fe52d01c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555c0d9e9760 0x555c0d9f3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:17.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.433+0000 7fe52d01c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c0d9f4020 0x555c0d9f6410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:17.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.433+0000 7fe52d01c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x555c0d94f3e0 con 0x555c0d9e9760 2024-09-10T09:59:17.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.433+0000 7fe52d01c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x555c0d8f6d80 con 0x555c0d9e8e40 2024-09-10T09:59:17.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.433+0000 7fe52d01c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x555c0d94b3a0 con 0x555c0d9f4020 2024-09-10T09:59:17.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.433+0000 7fe523e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555c0d9e8e40 0x555c0d9e9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:17.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.433+0000 7fe523e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555c0d9e8e40 0x555c0d9e9220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:42104/0 (socket says 172.21.15.77:42104) 2024-09-10T09:59:17.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.433+0000 7fe523e00640 1 -- 172.21.15.77:0/4035453184 learned_addr learned my addr 172.21.15.77:0/4035453184 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:59:17.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.433+0000 7fe523400640 1 --2- 172.21.15.77:0/4035453184 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555c0d9e9760 0x555c0d9f3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:17.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.433+0000 7fe522a00640 1 --2- 172.21.15.77:0/4035453184 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c0d9f4020 0x555c0d9f6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:17.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.434+0000 7fe523e00640 1 -- 172.21.15.77:0/4035453184 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c0d9f4020 msgr2=0x555c0d9f6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:17.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.434+0000 7fe523e00640 1 --2- 172.21.15.77:0/4035453184 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c0d9f4020 0x555c0d9f6410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:17.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.434+0000 7fe523e00640 1 -- 172.21.15.77:0/4035453184 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555c0d9e9760 msgr2=0x555c0d9f3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:17.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.434+0000 7fe523e00640 1 --2- 172.21.15.77:0/4035453184 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555c0d9e9760 0x555c0d9f3ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:17.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.434+0000 7fe523e00640 1 -- 172.21.15.77:0/4035453184 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c0d915ae0 con 0x555c0d9e8e40 2024-09-10T09:59:17.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.434+0000 7fe523400640 1 --2- 172.21.15.77:0/4035453184 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555c0d9e9760 0x555c0d9f3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:59:17.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.434+0000 7fe523e00640 1 --2- 172.21.15.77:0/4035453184 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555c0d9e8e40 0x555c0d9e9220 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe51c009f70 tx=0x7fe51c010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=886074c1c7a31bdd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:17.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.434+0000 7fe51be00640 1 --2- 172.21.15.77:0/1316081979 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fe5080a2c00 0x7fe5080a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:17.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.434+0000 7fe51be00640 1 -- 172.21.15.77:0/1316081979 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fe5140cf700 con 0x7fe524102bf0 2024-09-10T09:59:17.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.434+0000 7fe518c00640 1 -- 172.21.15.77:0/4035453184 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe51c016070 con 0x555c0d9e8e40 2024-09-10T09:59:17.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.435+0000 7fe523400640 1 --2- 172.21.15.77:0/1316081979 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fe5080a2c00 0x7fe5080a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:17.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.435+0000 7fe518c00640 1 -- 172.21.15.77:0/4035453184 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe51c004d20 con 0x555c0d9e8e40 2024-09-10T09:59:17.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.435+0000 7fe518c00640 1 -- 172.21.15.77:0/4035453184 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe51c012910 con 0x555c0d9e8e40 2024-09-10T09:59:17.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.435+0000 7fe52d01c740 1 -- 172.21.15.77:0/4035453184 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555c0d9e8e40 msgr2=0x555c0d9e9220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:17.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.435+0000 7fe523400640 1 --2- 172.21.15.77:0/1316081979 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fe5080a2c00 0x7fe5080a50c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fe51409bb50 tx=0x7fe514000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:17.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.435+0000 7fe52d01c740 1 --2- 172.21.15.77:0/4035453184 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555c0d9e8e40 0x555c0d9e9220 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe51c009f70 tx=0x7fe51c010800 comp rx=0 tx=0).stop 2024-09-10T09:59:17.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.436+0000 7fe52d01c740 1 -- 172.21.15.77:0/4035453184 shutdown_connections 2024-09-10T09:59:17.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.436+0000 7fe52d01c740 1 --2- 172.21.15.77:0/4035453184 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c0d9f4020 0x555c0d9f6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:17.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.436+0000 7fe52d01c740 1 --2- 172.21.15.77:0/4035453184 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555c0d9e9760 0x555c0d9f3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:17.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.436+0000 7fe52d01c740 1 --2- 172.21.15.77:0/4035453184 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555c0d9e8e40 0x555c0d9e9220 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:17.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.436+0000 7fe52d01c740 1 -- 172.21.15.77:0/4035453184 >> 172.21.15.77:0/4035453184 conn(0x555c0d9e53d0 msgr2=0x555c0d9e57b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:17.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.437+0000 7fe52d01c740 1 -- 172.21.15.77:0/4035453184 shutdown_connections 2024-09-10T09:59:17.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.437+0000 7fe52d01c740 1 -- 172.21.15.77:0/4035453184 wait complete. 2024-09-10T09:59:17.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.437+0000 7fe52d01c740 1 -- start start 2024-09-10T09:59:17.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.437+0000 7fe52d01c740 1 Processor -- start 2024-09-10T09:59:17.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.438+0000 7fe52d01c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555c0d9e8e40 0x555c0d9f8520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:17.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.438+0000 7fe52d01c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555c0d9e9760 0x555c0d9f8a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:17.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.438+0000 7fe52d01c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c0d9f4020 0x555c0d9fcbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:17.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.438+0000 7fe52d01c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x555c0d9f6bf0 con 0x555c0d9e9760 2024-09-10T09:59:17.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.438+0000 7fe52d01c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x555c0d90a0f0 con 0x555c0d9e8e40 2024-09-10T09:59:17.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.438+0000 7fe52d01c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x555c0d9f6d70 con 0x555c0d9f4020 2024-09-10T09:59:17.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.438+0000 7fe52d01c740 10 client.0 _ll_register_callbacks cb 0x555c0d8f1bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:59:17.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.438+0000 7fe522a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555c0d9e9760 0x555c0d9f8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:17.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.438+0000 7fe523e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555c0d9e8e40 0x555c0d9f8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:17.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.438+0000 7fe522a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555c0d9e9760 0x555c0d9f8a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:39710/0 (socket says 172.21.15.77:39710) 2024-09-10T09:59:17.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.438+0000 7fe522a00640 1 -- 172.21.15.77:0/69743275 learned_addr learned my addr 172.21.15.77:0/69743275 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:59:17.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.438+0000 7fe523e00640 1 --2- 172.21.15.77:0/69743275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c0d9f4020 0x555c0d9fcbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:17.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.439+0000 7fe522a00640 1 -- 172.21.15.77:0/69743275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c0d9f4020 msgr2=0x555c0d9fcbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:17.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.439+0000 7fe522a00640 1 --2- 172.21.15.77:0/69743275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c0d9f4020 0x555c0d9fcbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:17.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.439+0000 7fe522a00640 1 -- 172.21.15.77:0/69743275 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555c0d9e8e40 msgr2=0x555c0d9f8520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:17.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.439+0000 7fe522a00640 1 --2- 172.21.15.77:0/69743275 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555c0d9e8e40 0x555c0d9f8520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:17.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.439+0000 7fe522a00640 1 -- 172.21.15.77:0/69743275 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c0d6c03c0 con 0x555c0d9e9760 2024-09-10T09:59:17.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.439+0000 7fe523e00640 1 --2- 172.21.15.77:0/69743275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c0d9f4020 0x555c0d9fcbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:59:17.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.439+0000 7fe522a00640 1 --2- 172.21.15.77:0/69743275 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555c0d9e9760 0x555c0d9f8a60 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fe510031550 tx=0x7fe5100043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a70b6943204dc54 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:17.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.439+0000 7fe50d600640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-10T09:59:17.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.439+0000 7fe50d600640 1 -- 172.21.15.77:0/69743275 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe510035040 con 0x555c0d9e9760 2024-09-10T09:59:17.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.439+0000 7fe50d600640 1 -- 172.21.15.77:0/69743275 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fe510005450 con 0x555c0d9e9760 2024-09-10T09:59:17.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.439+0000 7fe52d01c740 1 -- 172.21.15.77:0/69743275 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe51c004d20 con 0x555c0d9e9760 2024-09-10T09:59:17.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.440+0000 7fe50d600640 1 -- 172.21.15.77:0/69743275 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe51004e050 con 0x555c0d9e9760 2024-09-10T09:59:17.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.440+0000 7fe52d01c740 10 client.16200 fetch_fsmap learned FSMap version 32 2024-09-10T09:59:17.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.440+0000 7fe52d01c740 1 -- 172.21.15.77:0/69743275 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x555c0da07070 con 0x555c0d9e9760 2024-09-10T09:59:17.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.440+0000 7fe50d600640 1 -- 172.21.15.77:0/69743275 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe51c004d20 con 0x555c0d9e9760 2024-09-10T09:59:17.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.440+0000 7fe50d600640 1 -- 172.21.15.77:0/69743275 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fe510003850 con 0x555c0d9e9760 2024-09-10T09:59:17.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.440+0000 7fe52d01c740 10 client.16200 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T09:59:17.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.440+0000 7fe52d01c740 10 client.16200 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-10T09:59:17.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.440+0000 7fe52d01c740 20 client.16200 populate_metadata read hostname 'smithi077' 2024-09-10T09:59:17.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.440+0000 7fe52d01c740 1 --2- 172.21.15.77:0/69743275 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x555c0da08530 0x555c0da0a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:17.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.440+0000 7fe52d01c740 4 client.16200 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:59:17.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.440+0000 7fe52d01c740 1 -- 172.21.15.77:0/69743275 --> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555c0d95c950 con 0x555c0da08530 2024-09-10T09:59:17.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.441+0000 7fe523e00640 1 --2- 172.21.15.77:0/69743275 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x555c0da08530 0x555c0da0a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:17.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.441+0000 7fe523e00640 1 --2- 172.21.15.77:0/69743275 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x555c0da08530 0x555c0da0a920 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bd9c046da563755 server_cookie=b4009e20591b1cb9 in_seq=0 out_seq=0 2024-09-10T09:59:17.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.441+0000 7fe50d600640 10 client.16200 ms_handle_connect on v2:172.21.15.134:6834/3293933699 2024-09-10T09:59:17.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.442+0000 7fe50d600640 1 -- 172.21.15.77:0/69743275 <== mds.4 v2:172.21.15.134:6834/3293933699 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555c0d95c950 con 0x555c0da08530 2024-09-10T09:59:17.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.442+0000 7fe50d600640 10 client.16200 handle_command_reply: tid=0 2024-09-10T09:59:17.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.442+0000 7fe52d01c740 1 client.16200 shutdown 2024-09-10T09:59:17.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.442+0000 7fe52d01c740 1 -- 172.21.15.77:0/69743275 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555c0d9e9760 msgr2=0x555c0d9f8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:17.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.442+0000 7fe52d01c740 1 --2- 172.21.15.77:0/69743275 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555c0d9e9760 0x555c0d9f8a60 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fe510031550 tx=0x7fe5100043b0 comp rx=0 tx=0).stop 2024-09-10T09:59:17.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.442+0000 7fe52d01c740 1 -- 172.21.15.77:0/69743275 shutdown_connections 2024-09-10T09:59:17.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.442+0000 7fe52d01c740 1 --2- 172.21.15.77:0/69743275 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x555c0da08530 0x555c0da0a920 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:17.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.442+0000 7fe52d01c740 1 --2- 172.21.15.77:0/69743275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c0d9f4020 0x555c0d9fcbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:17.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.442+0000 7fe52d01c740 1 --2- 172.21.15.77:0/69743275 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555c0d9e9760 0x555c0d9f8a60 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:17.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.442+0000 7fe52d01c740 1 --2- 172.21.15.77:0/69743275 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555c0d9e8e40 0x555c0d9f8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:17.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.442+0000 7fe50d600640 0 client.16200 ms_handle_reset on v2:172.21.15.134:6834/3293933699 2024-09-10T09:59:17.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.442+0000 7fe52d01c740 1 -- 172.21.15.77:0/69743275 >> 172.21.15.77:0/69743275 conn(0x555c0d9e53d0 msgr2=0x555c0d9eb100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:17.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.443+0000 7fe52d01c740 1 -- 172.21.15.77:0/69743275 shutdown_connections 2024-09-10T09:59:17.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.443+0000 7fe52d01c740 1 -- 172.21.15.77:0/69743275 wait complete. 2024-09-10T09:59:17.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.443+0000 7fe52d01c740 20 client.16200 trim_cache size 0 max 16384 2024-09-10T09:59:17.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.457+0000 7fe52d01c740 1 Processor -- start 2024-09-10T09:59:17.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.457+0000 7fe52d01c740 1 -- start start 2024-09-10T09:59:17.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.458+0000 7fe52d01c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c0d9e8e40 0x555c0d9fcbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:17.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.458+0000 7fe52d01c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555c0d9e9760 0x555c0d9fe6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:17.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.458+0000 7fe52d01c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555c0d9f4020 0x555c0da10210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:17.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.458+0000 7fe52d01c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe5241acb70 con 0x555c0d9e9760 2024-09-10T09:59:17.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.458+0000 7fe52d01c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe5241ae770 con 0x555c0d9f4020 2024-09-10T09:59:17.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.458+0000 7fe52d01c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe5241ad240 con 0x555c0d9e8e40 2024-09-10T09:59:17.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.458+0000 7fe522a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555c0d9e9760 0x555c0d9fe6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:17.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.458+0000 7fe523e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555c0d9f4020 0x555c0da10210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:17.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.458+0000 7fe523e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555c0d9f4020 0x555c0da10210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:42118/0 (socket says 172.21.15.77:42118) 2024-09-10T09:59:17.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.458+0000 7fe522a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555c0d9e9760 0x555c0d9fe6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:39720/0 (socket says 172.21.15.77:39720) 2024-09-10T09:59:17.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.458+0000 7fe523e00640 1 -- 172.21.15.77:0/2097799275 learned_addr learned my addr 172.21.15.77:0/2097799275 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:59:17.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.458+0000 7fe523e00640 1 --2- 172.21.15.77:0/2097799275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c0d9e8e40 0x555c0d9fcbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:17.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.458+0000 7fe523e00640 1 -- 172.21.15.77:0/2097799275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c0d9e8e40 msgr2=0x555c0d9fcbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:17.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.458+0000 7fe523e00640 1 --2- 172.21.15.77:0/2097799275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c0d9e8e40 0x555c0d9fcbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:17.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.458+0000 7fe523e00640 1 -- 172.21.15.77:0/2097799275 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555c0d9e9760 msgr2=0x555c0d9fe6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:17.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.458+0000 7fe523e00640 1 --2- 172.21.15.77:0/2097799275 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555c0d9e9760 0x555c0d9fe6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:17.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.458+0000 7fe523e00640 1 -- 172.21.15.77:0/2097799275 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c0d9f0f70 con 0x555c0d9f4020 2024-09-10T09:59:17.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.459+0000 7fe523e00640 1 --2- 172.21.15.77:0/2097799275 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555c0d9f4020 0x555c0da10210 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe51c0124c0 tx=0x7fe51c005c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=496c9f404ee51a69 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:17.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.459+0000 7fe50fe00640 1 -- 172.21.15.77:0/2097799275 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe51c016040 con 0x555c0d9f4020 2024-09-10T09:59:17.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.459+0000 7fe50fe00640 1 -- 172.21.15.77:0/2097799275 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe51c0056d0 con 0x555c0d9f4020 2024-09-10T09:59:17.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.459+0000 7fe50fe00640 1 -- 172.21.15.77:0/2097799275 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe51c023040 con 0x555c0d9f4020 2024-09-10T09:59:17.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.460+0000 7fe52d01c740 1 -- 172.21.15.77:0/2097799275 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555c0d9f4020 msgr2=0x555c0da10210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:17.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.460+0000 7fe52d01c740 1 --2- 172.21.15.77:0/2097799275 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555c0d9f4020 0x555c0da10210 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe51c0124c0 tx=0x7fe51c005c80 comp rx=0 tx=0).stop 2024-09-10T09:59:17.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.460+0000 7fe52d01c740 1 -- 172.21.15.77:0/2097799275 shutdown_connections 2024-09-10T09:59:17.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.460+0000 7fe52d01c740 1 --2- 172.21.15.77:0/2097799275 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555c0d9f4020 0x555c0da10210 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:17.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.460+0000 7fe52d01c740 1 --2- 172.21.15.77:0/2097799275 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555c0d9e9760 0x555c0d9fe6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:17.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.460+0000 7fe52d01c740 1 --2- 172.21.15.77:0/2097799275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c0d9e8e40 0x555c0d9fcbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:17.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.460+0000 7fe52d01c740 1 -- 172.21.15.77:0/2097799275 >> 172.21.15.77:0/2097799275 conn(0x555c0d9e53d0 msgr2=0x555c0d9f8520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:17.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.460+0000 7fe52d01c740 1 -- 172.21.15.77:0/2097799275 shutdown_connections 2024-09-10T09:59:17.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.460+0000 7fe52d01c740 1 -- 172.21.15.77:0/2097799275 wait complete. 2024-09-10T09:59:17.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.461+0000 7fe52d01c740 1 -- start start 2024-09-10T09:59:17.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.461+0000 7fe52d01c740 1 Processor -- start 2024-09-10T09:59:17.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.461+0000 7fe52d01c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555c0d9e8e40 0x555c0da04270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:17.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.461+0000 7fe52d01c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c0d9e9760 0x555c0da047b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:17.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.461+0000 7fe52d01c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555c0d9f4020 0x555c0d9f1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:17.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.461+0000 7fe52d01c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe5241adfe0 con 0x555c0d9f4020 2024-09-10T09:59:17.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.461+0000 7fe52d01c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe5241ad910 con 0x555c0d9e8e40 2024-09-10T09:59:17.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.461+0000 7fe52d01c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe5241aab50 con 0x555c0d9e9760 2024-09-10T09:59:17.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.461+0000 7fe52d01c740 10 client.0 _ll_register_callbacks cb 0x7fe510009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:59:17.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.462+0000 7fe523e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555c0d9e8e40 0x555c0da04270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:17.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.462+0000 7fe523e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555c0d9e8e40 0x555c0da04270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:42124/0 (socket says 172.21.15.77:42124) 2024-09-10T09:59:17.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.462+0000 7fe523e00640 1 -- 172.21.15.77:0/3812263593 learned_addr learned my addr 172.21.15.77:0/3812263593 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:59:17.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.462+0000 7fe523e00640 1 --2- 172.21.15.77:0/3812263593 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555c0d9f4020 0x555c0d9f1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:17.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.462+0000 7fe522a00640 1 --2- 172.21.15.77:0/3812263593 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c0d9e9760 0x555c0da047b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:17.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.462+0000 7fe523e00640 1 -- 172.21.15.77:0/3812263593 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c0d9e9760 msgr2=0x555c0da047b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:17.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.462+0000 7fe523e00640 1 --2- 172.21.15.77:0/3812263593 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c0d9e9760 0x555c0da047b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:17.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.462+0000 7fe523e00640 1 -- 172.21.15.77:0/3812263593 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555c0d9f4020 msgr2=0x555c0d9f1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:17.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.462+0000 7fe523e00640 1 --2- 172.21.15.77:0/3812263593 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555c0d9f4020 0x555c0d9f1dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:17.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.462+0000 7fe523e00640 1 -- 172.21.15.77:0/3812263593 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c0d9f25f0 con 0x555c0d9e8e40 2024-09-10T09:59:17.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.463+0000 7fe523e00640 1 --2- 172.21.15.77:0/3812263593 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555c0d9e8e40 0x555c0da04270 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe51c012800 tx=0x7fe51c009730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e24a5ab5761452 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:17.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.463+0000 7fe50d600640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T09:59:17.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.463+0000 7fe50d600640 1 -- 172.21.15.77:0/3812263593 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe51c016070 con 0x555c0d9e8e40 2024-09-10T09:59:17.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.463+0000 7fe52d01c740 1 -- 172.21.15.77:0/3812263593 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe51c0056d0 con 0x555c0d9e8e40 2024-09-10T09:59:17.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.463+0000 7fe50d600640 1 -- 172.21.15.77:0/3812263593 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fe51c0051f0 con 0x555c0d9e8e40 2024-09-10T09:59:17.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.463+0000 7fe50d600640 1 -- 172.21.15.77:0/3812263593 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe51c004f60 con 0x555c0d9e8e40 2024-09-10T09:59:17.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.463+0000 7fe52d01c740 10 client.26288 fetch_fsmap learned FSMap version 32 2024-09-10T09:59:17.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.463+0000 7fe52d01c740 1 -- 172.21.15.77:0/3812263593 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x555c0da72d70 con 0x555c0d9e8e40 2024-09-10T09:59:17.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.463+0000 7fe50d600640 1 -- 172.21.15.77:0/3812263593 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe51c0056d0 con 0x555c0d9e8e40 2024-09-10T09:59:17.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.464+0000 7fe50d600640 1 -- 172.21.15.77:0/3812263593 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fe51c00f040 con 0x555c0d9e8e40 2024-09-10T09:59:17.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.464+0000 7fe52d01c740 10 client.26288 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T09:59:17.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.464+0000 7fe52d01c740 10 client.26288 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-10T09:59:17.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.464+0000 7fe52d01c740 20 client.26288 populate_metadata read hostname 'smithi077' 2024-09-10T09:59:17.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.464+0000 7fe52d01c740 1 --2- 172.21.15.77:0/3812263593 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x555c0da73250 0x555c0da75640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:17.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.464+0000 7fe52d01c740 4 client.26288 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:59:17.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.464+0000 7fe52d01c740 1 -- 172.21.15.77:0/3812263593 --> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555c0d956e60 con 0x555c0da73250 2024-09-10T09:59:17.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.465+0000 7fe522a00640 1 --2- 172.21.15.77:0/3812263593 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x555c0da73250 0x555c0da75640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:17.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.466+0000 7fe522a00640 1 --2- 172.21.15.77:0/3812263593 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x555c0da73250 0x555c0da75640 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c1ded9e713ef8920 server_cookie=c2e0d57ae23ebd4b in_seq=0 out_seq=0 2024-09-10T09:59:17.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.466+0000 7fe50d600640 10 client.26288 ms_handle_connect on v2:172.21.15.134:6834/3293933699 2024-09-10T09:59:17.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.466+0000 7fe50d600640 1 -- 172.21.15.77:0/3812263593 <== mds.4 v2:172.21.15.134:6834/3293933699 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555c0d956e60 con 0x555c0da73250 2024-09-10T09:59:17.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.466+0000 7fe50d600640 10 client.26288 handle_command_reply: tid=0 2024-09-10T09:59:17.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.466+0000 7fe52d01c740 1 client.26288 shutdown 2024-09-10T09:59:17.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.466+0000 7fe52d01c740 1 -- 172.21.15.77:0/3812263593 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555c0d9e8e40 msgr2=0x555c0da04270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:17.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.466+0000 7fe52d01c740 1 --2- 172.21.15.77:0/3812263593 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555c0d9e8e40 0x555c0da04270 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe51c012800 tx=0x7fe51c009730 comp rx=0 tx=0).stop 2024-09-10T09:59:17.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.466+0000 7fe52d01c740 1 -- 172.21.15.77:0/3812263593 shutdown_connections 2024-09-10T09:59:17.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.466+0000 7fe52d01c740 1 --2- 172.21.15.77:0/3812263593 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x555c0da73250 0x555c0da75640 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:17.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.466+0000 7fe50d600640 0 client.26288 ms_handle_reset on v2:172.21.15.134:6834/3293933699 2024-09-10T09:59:17.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.466+0000 7fe52d01c740 1 --2- 172.21.15.77:0/3812263593 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555c0d9f4020 0x555c0d9f1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:17.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.466+0000 7fe52d01c740 1 --2- 172.21.15.77:0/3812263593 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c0d9e9760 0x555c0da047b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:17.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.466+0000 7fe52d01c740 1 --2- 172.21.15.77:0/3812263593 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555c0d9e8e40 0x555c0da04270 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:17.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.466+0000 7fe52d01c740 1 -- 172.21.15.77:0/3812263593 >> 172.21.15.77:0/3812263593 conn(0x555c0d9e53d0 msgr2=0x555c0d9e3c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:17.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.467+0000 7fe52d01c740 1 -- 172.21.15.77:0/3812263593 shutdown_connections 2024-09-10T09:59:17.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.467+0000 7fe52d01c740 1 -- 172.21.15.77:0/3812263593 wait complete. 2024-09-10T09:59:17.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.467+0000 7fe52d01c740 20 client.26288 trim_cache size 0 max 16384 2024-09-10T09:59:17.471 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-10T09:59:17.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.468+0000 7fe529000640 1 -- 172.21.15.77:0/1316081979 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fe5080a2c00 msgr2=0x7fe5080a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:17.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.468+0000 7fe529000640 1 --2- 172.21.15.77:0/1316081979 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fe5080a2c00 0x7fe5080a50c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fe51409bb50 tx=0x7fe514000bc0 comp rx=0 tx=0).stop 2024-09-10T09:59:17.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.468+0000 7fe529000640 1 -- 172.21.15.77:0/1316081979 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe524102bf0 msgr2=0x7fe524198260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:17.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.468+0000 7fe529000640 1 --2- 172.21.15.77:0/1316081979 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe524102bf0 0x7fe524198260 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe51400e9f0 tx=0x7fe51400eeb0 comp rx=0 tx=0).stop 2024-09-10T09:59:17.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.468+0000 7fe529000640 1 -- 172.21.15.77:0/1316081979 shutdown_connections 2024-09-10T09:59:17.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.468+0000 7fe529000640 1 --2- 172.21.15.77:0/1316081979 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fe5080a2c00 0x7fe5080a50c0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:17.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.468+0000 7fe529000640 1 --2- 172.21.15.77:0/1316081979 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe52410bfd0 0x7fe52419cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:17.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.468+0000 7fe529000640 1 --2- 172.21.15.77:0/1316081979 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe524103510 0x7fe5241987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:17.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.468+0000 7fe529000640 1 --2- 172.21.15.77:0/1316081979 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe524102bf0 0x7fe524198260 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:17.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.468+0000 7fe529000640 1 -- 172.21.15.77:0/1316081979 >> 172.21.15.77:0/1316081979 conn(0x7fe5240fc610 msgr2=0x7fe5240fd2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:17.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.468+0000 7fe529000640 1 -- 172.21.15.77:0/1316081979 shutdown_connections 2024-09-10T09:59:17.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:17.468+0000 7fe529000640 1 -- 172.21.15.77:0/1316081979 wait complete. 2024-09-10T09:59:18.670 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T09:59:19.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:59:18 smithi077 ceph-mon[30319]: pgmap v515: 129 pgs: 129 active+clean; 95 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 326 op/s 2024-09-10T09:59:19.193 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '95a54fcc', quiesce took 7.4 seconds. Will release after: 15.35 2024-09-10T09:59:19.241 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:59:19.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:59:18 smithi134 ceph-mon[35674]: pgmap v515: 129 pgs: 129 active+clean; 95 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 326 op/s 2024-09-10T09:59:19.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:59:18 smithi120 ceph-mon[34770]: pgmap v515: 129 pgs: 129 active+clean; 95 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 326 op/s 2024-09-10T09:59:19.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.776+0000 7ff024000640 1 -- 172.21.15.77:0/3868777822 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff01c101de0 msgr2=0x7ff01c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:19.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.776+0000 7ff024000640 1 --2- 172.21.15.77:0/3868777822 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff01c101de0 0x7ff01c1021c0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7ff010009900 tx=0x7ff01002a410 comp rx=0 tx=0).stop 2024-09-10T09:59:19.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.776+0000 7ff024000640 1 -- 172.21.15.77:0/3868777822 shutdown_connections 2024-09-10T09:59:19.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.776+0000 7ff024000640 1 --2- 172.21.15.77:0/3868777822 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff01c106f10 0x7ff01c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:19.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.776+0000 7ff024000640 1 --2- 172.21.15.77:0/3868777822 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff01c102790 0x7ff01c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:19.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.776+0000 7ff024000640 1 --2- 172.21.15.77:0/3868777822 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff01c101de0 0x7ff01c1021c0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:19.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.776+0000 7ff024000640 1 -- 172.21.15.77:0/3868777822 >> 172.21.15.77:0/3868777822 conn(0x7ff01c0fd8c0 msgr2=0x7ff01c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:19.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.776+0000 7ff024000640 1 -- 172.21.15.77:0/3868777822 shutdown_connections 2024-09-10T09:59:19.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.776+0000 7ff024000640 1 -- 172.21.15.77:0/3868777822 wait complete. 2024-09-10T09:59:19.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.777+0000 7ff024000640 1 Processor -- start 2024-09-10T09:59:19.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.777+0000 7ff024000640 1 -- start start 2024-09-10T09:59:19.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.777+0000 7ff024000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff01c101de0 0x7ff01c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:19.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.777+0000 7ff024000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff01c102790 0x7ff01c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:19.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.777+0000 7ff024000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff01c106f10 0x7ff01c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:19.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.777+0000 7ff024000640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff01c111fc0 con 0x7ff01c101de0 2024-09-10T09:59:19.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.777+0000 7ff024000640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff01c111e40 con 0x7ff01c102790 2024-09-10T09:59:19.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.777+0000 7ff024000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff01c112140 con 0x7ff01c106f10 2024-09-10T09:59:19.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.778+0000 7ff021600640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff01c101de0 0x7ff01c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:19.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.778+0000 7ff021600640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff01c101de0 0x7ff01c0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:39744/0 (socket says 172.21.15.77:39744) 2024-09-10T09:59:19.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.778+0000 7ff021600640 1 -- 172.21.15.77:0/2918922222 learned_addr learned my addr 172.21.15.77:0/2918922222 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:59:19.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.778+0000 7ff020c00640 1 --2- 172.21.15.77:0/2918922222 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff01c102790 0x7ff01c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:19.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.778+0000 7ff022000640 1 --2- 172.21.15.77:0/2918922222 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff01c106f10 0x7ff01c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:19.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.778+0000 7ff021600640 1 -- 172.21.15.77:0/2918922222 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff01c106f10 msgr2=0x7ff01c074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:19.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.778+0000 7ff021600640 1 --2- 172.21.15.77:0/2918922222 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff01c106f10 0x7ff01c074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:19.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.778+0000 7ff021600640 1 -- 172.21.15.77:0/2918922222 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff01c102790 msgr2=0x7ff01c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:19.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.778+0000 7ff021600640 1 --2- 172.21.15.77:0/2918922222 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff01c102790 0x7ff01c074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:19.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.778+0000 7ff021600640 1 -- 172.21.15.77:0/2918922222 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff01c1a6900 con 0x7ff01c101de0 2024-09-10T09:59:19.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.778+0000 7ff020c00640 1 --2- 172.21.15.77:0/2918922222 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff01c102790 0x7ff01c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:59:19.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.779+0000 7ff021600640 1 --2- 172.21.15.77:0/2918922222 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff01c101de0 0x7ff01c0744f0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7ff010004770 tx=0x7ff010003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:19.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.779+0000 7ff01a000640 1 -- 172.21.15.77:0/2918922222 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff01003f070 con 0x7ff01c101de0 2024-09-10T09:59:19.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.779+0000 7ff01a000640 1 -- 172.21.15.77:0/2918922222 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff010031ca0 con 0x7ff01c101de0 2024-09-10T09:59:19.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.779+0000 7ff024000640 1 -- 172.21.15.77:0/2918922222 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff01c1a6b90 con 0x7ff01c101de0 2024-09-10T09:59:19.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.779+0000 7ff01a000640 1 -- 172.21.15.77:0/2918922222 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff010035030 con 0x7ff01c101de0 2024-09-10T09:59:19.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.779+0000 7ff024000640 1 -- 172.21.15.77:0/2918922222 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff01c1a70c0 con 0x7ff01c101de0 2024-09-10T09:59:19.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.783+0000 7ff026c1a740 1 Processor -- start 2024-09-10T09:59:19.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.783+0000 7ff01a000640 1 -- 172.21.15.77:0/2918922222 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff010042070 con 0x7ff01c101de0 2024-09-10T09:59:19.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.783+0000 7ff026c1a740 1 -- start start 2024-09-10T09:59:19.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.784+0000 7ff026c1a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fe5d8ac70 0x560fe5d8b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:19.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.784+0000 7ff026c1a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fe5d8b590 0x560fe5d95910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:19.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.784+0000 7ff026c1a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560fe5d95e50 0x560fe5d98240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:19.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.784+0000 7ff026c1a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560fe5cf13e0 con 0x560fe5d8b590 2024-09-10T09:59:19.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.784+0000 7ff026c1a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x560fe5c98d80 con 0x560fe5d95e50 2024-09-10T09:59:19.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.784+0000 7ff026c1a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560fe5ced3a0 con 0x560fe5d8ac70 2024-09-10T09:59:19.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.784+0000 7ff021600640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fe5d8b590 0x560fe5d95910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:19.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.784+0000 7ff021600640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fe5d8b590 0x560fe5d95910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:39752/0 (socket says 172.21.15.77:39752) 2024-09-10T09:59:19.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.784+0000 7ff021600640 1 -- 172.21.15.77:0/466096813 learned_addr learned my addr 172.21.15.77:0/466096813 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:59:19.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.784+0000 7ff022000640 1 --2- 172.21.15.77:0/466096813 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fe5d8ac70 0x560fe5d8b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:19.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.784+0000 7ff020c00640 1 --2- 172.21.15.77:0/466096813 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560fe5d95e50 0x560fe5d98240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:19.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.784+0000 7ff021600640 1 -- 172.21.15.77:0/466096813 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fe5d8ac70 msgr2=0x560fe5d8b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:19.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.784+0000 7ff021600640 1 --2- 172.21.15.77:0/466096813 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fe5d8ac70 0x560fe5d8b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:19.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.784+0000 7ff021600640 1 -- 172.21.15.77:0/466096813 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560fe5d95e50 msgr2=0x560fe5d98240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:19.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.784+0000 7ff021600640 1 --2- 172.21.15.77:0/466096813 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560fe5d95e50 0x560fe5d98240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:19.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.784+0000 7ff021600640 1 -- 172.21.15.77:0/466096813 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fe5cb7ae0 con 0x560fe5d8b590 2024-09-10T09:59:19.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.785+0000 7ff021600640 1 --2- 172.21.15.77:0/466096813 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fe5d8b590 0x560fe5d95910 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7ff0100f5c10 tx=0x7ff0100f5d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f62568e24b9bb9a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:19.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.785+0000 7ff01a000640 1 --2- 172.21.15.77:0/2918922222 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ff0040a2c00 0x7ff0040a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:19.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.785+0000 7ff01a000640 1 -- 172.21.15.77:0/2918922222 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7ff0100ef940 con 0x7ff01c101de0 2024-09-10T09:59:19.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.785+0000 7ff022000640 1 --2- 172.21.15.77:0/2918922222 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ff0040a2c00 0x7ff0040a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:19.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.785+0000 7ff00e600640 1 -- 172.21.15.77:0/466096813 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0100fe070 con 0x560fe5d8b590 2024-09-10T09:59:19.797 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.785+0000 7ff00e600640 1 -- 172.21.15.77:0/466096813 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0100c4ae0 con 0x560fe5d8b590 2024-09-10T09:59:19.797 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.785+0000 7ff00e600640 1 -- 172.21.15.77:0/466096813 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0100bba60 con 0x560fe5d8b590 2024-09-10T09:59:19.797 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.786+0000 7ff026c1a740 1 -- 172.21.15.77:0/466096813 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fe5d8b590 msgr2=0x560fe5d95910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:19.797 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.786+0000 7ff022000640 1 --2- 172.21.15.77:0/2918922222 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ff0040a2c00 0x7ff0040a50c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff014005fd0 tx=0x7ff014009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:19.797 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.786+0000 7ff026c1a740 1 --2- 172.21.15.77:0/466096813 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fe5d8b590 0x560fe5d95910 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7ff0100f5c10 tx=0x7ff0100f5d00 comp rx=0 tx=0).stop 2024-09-10T09:59:19.797 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.786+0000 7ff026c1a740 1 -- 172.21.15.77:0/466096813 shutdown_connections 2024-09-10T09:59:19.797 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.786+0000 7ff026c1a740 1 --2- 172.21.15.77:0/466096813 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560fe5d95e50 0x560fe5d98240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:19.797 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.786+0000 7ff026c1a740 1 --2- 172.21.15.77:0/466096813 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fe5d8b590 0x560fe5d95910 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:19.797 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.786+0000 7ff026c1a740 1 --2- 172.21.15.77:0/466096813 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fe5d8ac70 0x560fe5d8b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:19.797 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.786+0000 7ff026c1a740 1 -- 172.21.15.77:0/466096813 >> 172.21.15.77:0/466096813 conn(0x560fe5d87200 msgr2=0x560fe5d875e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:19.797 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.786+0000 7ff026c1a740 1 -- 172.21.15.77:0/466096813 shutdown_connections 2024-09-10T09:59:19.797 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.786+0000 7ff026c1a740 1 -- 172.21.15.77:0/466096813 wait complete. 2024-09-10T09:59:19.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.786+0000 7ff026c1a740 1 -- start start 2024-09-10T09:59:19.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.787+0000 7ff026c1a740 1 Processor -- start 2024-09-10T09:59:19.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.787+0000 7ff026c1a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fe5d8ac70 0x560fe5d92d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:19.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.787+0000 7ff026c1a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fe5d8b590 0x560fe5d93290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:19.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.787+0000 7ff026c1a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560fe5d95e50 0x560fe5d937d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:19.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.787+0000 7ff026c1a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560fe5d987e0 con 0x560fe5d8ac70 2024-09-10T09:59:19.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.787+0000 7ff026c1a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x560fe5cac0f0 con 0x560fe5d95e50 2024-09-10T09:59:19.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.787+0000 7ff026c1a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560fe5d98960 con 0x560fe5d8b590 2024-09-10T09:59:19.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.787+0000 7ff026c1a740 10 client.0 _ll_register_callbacks cb 0x560fe5c93bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:59:19.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.788+0000 7ff021600640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fe5d8ac70 0x560fe5d92d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:19.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.788+0000 7ff020c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fe5d8b590 0x560fe5d93290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:19.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.788+0000 7ff022000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560fe5d95e50 0x560fe5d937d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:19.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.788+0000 7ff020c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fe5d8b590 0x560fe5d93290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:53342/0 (socket says 172.21.15.77:53342) 2024-09-10T09:59:19.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.788+0000 7ff020c00640 1 -- 172.21.15.77:0/2206134044 learned_addr learned my addr 172.21.15.77:0/2206134044 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:59:19.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.788+0000 7ff020c00640 1 -- 172.21.15.77:0/2206134044 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560fe5d95e50 msgr2=0x560fe5d937d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:19.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.788+0000 7ff020c00640 1 --2- 172.21.15.77:0/2206134044 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560fe5d95e50 0x560fe5d937d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:19.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.788+0000 7ff020c00640 1 -- 172.21.15.77:0/2206134044 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fe5d8ac70 msgr2=0x560fe5d92d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:19.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.788+0000 7ff020c00640 1 --2- 172.21.15.77:0/2206134044 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fe5d8ac70 0x560fe5d92d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:19.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.788+0000 7ff020c00640 1 -- 172.21.15.77:0/2206134044 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fe5a623c0 con 0x560fe5d8b590 2024-09-10T09:59:19.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.788+0000 7ff022000640 1 --2- 172.21.15.77:0/2206134044 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560fe5d95e50 0x560fe5d937d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:59:19.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.788+0000 7ff020c00640 1 --2- 172.21.15.77:0/2206134044 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fe5d8b590 0x560fe5d93290 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff008012d30 tx=0x7ff00800f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b10044d1cd4e1d74 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:19.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.788+0000 7ff002a00640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-10T09:59:19.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.789+0000 7ff002a00640 1 -- 172.21.15.77:0/2206134044 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff00800fe20 con 0x560fe5d8b590 2024-09-10T09:59:19.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.789+0000 7ff026c1a740 1 -- 172.21.15.77:0/2206134044 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0100c4ae0 con 0x560fe5d8b590 2024-09-10T09:59:19.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.789+0000 7ff002a00640 1 -- 172.21.15.77:0/2206134044 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7ff008012110 con 0x560fe5d8b590 2024-09-10T09:59:19.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.789+0000 7ff002a00640 1 -- 172.21.15.77:0/2206134044 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff008025050 con 0x560fe5d8b590 2024-09-10T09:59:19.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.789+0000 7ff026c1a740 10 client.26251 fetch_fsmap learned FSMap version 32 2024-09-10T09:59:19.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.789+0000 7ff026c1a740 1 -- 172.21.15.77:0/2206134044 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560fe5da4850 con 0x560fe5d8b590 2024-09-10T09:59:19.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.789+0000 7ff002a00640 1 -- 172.21.15.77:0/2206134044 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0100c4ae0 con 0x560fe5d8b590 2024-09-10T09:59:19.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.790+0000 7ff002a00640 1 -- 172.21.15.77:0/2206134044 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7ff008010430 con 0x560fe5d8b590 2024-09-10T09:59:19.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.790+0000 7ff026c1a740 10 client.26251 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T09:59:19.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.790+0000 7ff026c1a740 10 client.26251 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T09:59:19.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.790+0000 7ff026c1a740 20 client.26251 populate_metadata read hostname 'smithi077' 2024-09-10T09:59:19.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.790+0000 7ff026c1a740 1 --2- 172.21.15.77:0/2206134044 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x560fe5da5cb0 0x560fe5da80d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:19.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.790+0000 7ff026c1a740 4 client.26251 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:59:19.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.790+0000 7ff026c1a740 1 -- 172.21.15.77:0/2206134044 --> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560fe5cfe950 con 0x560fe5da5cb0 2024-09-10T09:59:19.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.790+0000 7ff021600640 1 --2- 172.21.15.77:0/2206134044 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x560fe5da5cb0 0x560fe5da80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:19.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.791+0000 7ff021600640 1 --2- 172.21.15.77:0/2206134044 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x560fe5da5cb0 0x560fe5da80d0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=41d5708c404b7698 server_cookie=235220b43c7a378f in_seq=0 out_seq=0 2024-09-10T09:59:19.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.791+0000 7ff002a00640 10 client.26251 ms_handle_connect on v2:172.21.15.134:6836/2461497091 2024-09-10T09:59:19.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.792+0000 7ff002a00640 1 -- 172.21.15.77:0/2206134044 <== mds.3 v2:172.21.15.134:6836/2461497091 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560fe5cfe950 con 0x560fe5da5cb0 2024-09-10T09:59:19.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.792+0000 7ff002a00640 10 client.26251 handle_command_reply: tid=0 2024-09-10T09:59:19.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.792+0000 7ff026c1a740 1 client.26251 shutdown 2024-09-10T09:59:19.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.792+0000 7ff026c1a740 1 -- 172.21.15.77:0/2206134044 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fe5d8b590 msgr2=0x560fe5d93290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:19.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.792+0000 7ff026c1a740 1 --2- 172.21.15.77:0/2206134044 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fe5d8b590 0x560fe5d93290 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff008012d30 tx=0x7ff00800f790 comp rx=0 tx=0).stop 2024-09-10T09:59:19.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.792+0000 7ff026c1a740 1 -- 172.21.15.77:0/2206134044 shutdown_connections 2024-09-10T09:59:19.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.792+0000 7ff026c1a740 1 --2- 172.21.15.77:0/2206134044 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560fe5d95e50 0x560fe5d937d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:19.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.792+0000 7ff026c1a740 1 --2- 172.21.15.77:0/2206134044 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fe5d8b590 0x560fe5d93290 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:19.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.792+0000 7ff026c1a740 1 --2- 172.21.15.77:0/2206134044 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x560fe5da5cb0 0x560fe5da80d0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:19.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.792+0000 7ff026c1a740 1 --2- 172.21.15.77:0/2206134044 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fe5d8ac70 0x560fe5d92d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:19.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.792+0000 7ff026c1a740 1 -- 172.21.15.77:0/2206134044 >> 172.21.15.77:0/2206134044 conn(0x560fe5d87200 msgr2=0x560fe5d8cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:19.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.792+0000 7ff002a00640 0 client.26251 ms_handle_reset on v2:172.21.15.134:6836/2461497091 2024-09-10T09:59:19.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.792+0000 7ff026c1a740 1 -- 172.21.15.77:0/2206134044 shutdown_connections 2024-09-10T09:59:19.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.792+0000 7ff026c1a740 1 -- 172.21.15.77:0/2206134044 wait complete. 2024-09-10T09:59:19.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.792+0000 7ff026c1a740 20 client.26251 trim_cache size 0 max 16384 2024-09-10T09:59:19.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.806+0000 7ff026c1a740 1 Processor -- start 2024-09-10T09:59:19.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.806+0000 7ff026c1a740 1 -- start start 2024-09-10T09:59:19.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.806+0000 7ff026c1a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fe5d8ac70 0x560fe5dad9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:19.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.806+0000 7ff026c1a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fe5d8b590 0x560fe5dadee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:19.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.806+0000 7ff026c1a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560fe5d95e50 0x560fe5d8ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:19.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.806+0000 7ff026c1a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff01c1b1080 con 0x560fe5d8ac70 2024-09-10T09:59:19.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.806+0000 7ff026c1a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff01c1b2c80 con 0x560fe5d95e50 2024-09-10T09:59:19.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.806+0000 7ff026c1a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff01c1b1750 con 0x560fe5d8b590 2024-09-10T09:59:19.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.806+0000 7ff021600640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fe5d8ac70 0x560fe5dad9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:19.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.806+0000 7ff021600640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fe5d8ac70 0x560fe5dad9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:39780/0 (socket says 172.21.15.77:39780) 2024-09-10T09:59:19.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.806+0000 7ff021600640 1 -- 172.21.15.77:0/3459254643 learned_addr learned my addr 172.21.15.77:0/3459254643 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:59:19.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.806+0000 7ff022000640 1 --2- 172.21.15.77:0/3459254643 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560fe5d95e50 0x560fe5d8ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:19.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.806+0000 7ff020c00640 1 --2- 172.21.15.77:0/3459254643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fe5d8b590 0x560fe5dadee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:19.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.807+0000 7ff021600640 1 -- 172.21.15.77:0/3459254643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fe5d8b590 msgr2=0x560fe5dadee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:19.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.807+0000 7ff021600640 1 --2- 172.21.15.77:0/3459254643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fe5d8b590 0x560fe5dadee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:19.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.807+0000 7ff021600640 1 -- 172.21.15.77:0/3459254643 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560fe5d95e50 msgr2=0x560fe5d8ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:19.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.807+0000 7ff021600640 1 --2- 172.21.15.77:0/3459254643 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560fe5d95e50 0x560fe5d8ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:19.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.807+0000 7ff021600640 1 -- 172.21.15.77:0/3459254643 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fe5d87030 con 0x560fe5d8ac70 2024-09-10T09:59:19.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.807+0000 7ff020c00640 1 --2- 172.21.15.77:0/3459254643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fe5d8b590 0x560fe5dadee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:59:19.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.807+0000 7ff021600640 1 --2- 172.21.15.77:0/3459254643 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fe5d8ac70 0x560fe5dad9a0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7ff0100f5e30 tx=0x7ff010009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea57120fcb6ce97d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:19.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.807+0000 7ff00dc00640 1 -- 172.21.15.77:0/3459254643 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff010049040 con 0x560fe5d8ac70 2024-09-10T09:59:19.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.807+0000 7ff00dc00640 1 -- 172.21.15.77:0/3459254643 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0100bbf20 con 0x560fe5d8ac70 2024-09-10T09:59:19.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.807+0000 7ff00dc00640 1 -- 172.21.15.77:0/3459254643 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0100fe040 con 0x560fe5d8ac70 2024-09-10T09:59:19.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.808+0000 7ff026c1a740 1 -- 172.21.15.77:0/3459254643 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fe5d8ac70 msgr2=0x560fe5dad9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:19.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.808+0000 7ff026c1a740 1 --2- 172.21.15.77:0/3459254643 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fe5d8ac70 0x560fe5dad9a0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7ff0100f5e30 tx=0x7ff010009260 comp rx=0 tx=0).stop 2024-09-10T09:59:19.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.808+0000 7ff026c1a740 1 -- 172.21.15.77:0/3459254643 shutdown_connections 2024-09-10T09:59:19.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.808+0000 7ff026c1a740 1 --2- 172.21.15.77:0/3459254643 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560fe5d95e50 0x560fe5d8ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:19.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.808+0000 7ff026c1a740 1 --2- 172.21.15.77:0/3459254643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fe5d8b590 0x560fe5dadee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:19.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.808+0000 7ff026c1a740 1 --2- 172.21.15.77:0/3459254643 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fe5d8ac70 0x560fe5dad9a0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:19.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.808+0000 7ff026c1a740 1 -- 172.21.15.77:0/3459254643 >> 172.21.15.77:0/3459254643 conn(0x560fe5d87200 msgr2=0x560fe5d94dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:19.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.808+0000 7ff026c1a740 1 -- 172.21.15.77:0/3459254643 shutdown_connections 2024-09-10T09:59:19.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.808+0000 7ff026c1a740 1 -- 172.21.15.77:0/3459254643 wait complete. 2024-09-10T09:59:19.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.808+0000 7ff026c1a740 1 -- start start 2024-09-10T09:59:19.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.809+0000 7ff026c1a740 1 Processor -- start 2024-09-10T09:59:19.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.809+0000 7ff026c1a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fe5d8ac70 0x560fe5d998f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:19.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.809+0000 7ff026c1a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560fe5d8b590 0x560fe5d99e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:19.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.809+0000 7ff026c1a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fe5d95e50 0x560fe5d9a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:19.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.809+0000 7ff026c1a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff01c1b24f0 con 0x560fe5d95e50 2024-09-10T09:59:19.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.809+0000 7ff026c1a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff01c1b1e20 con 0x560fe5d8b590 2024-09-10T09:59:19.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.809+0000 7ff026c1a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff01c1af060 con 0x560fe5d8ac70 2024-09-10T09:59:19.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.809+0000 7ff026c1a740 10 client.0 _ll_register_callbacks cb 0x7ff008009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:59:19.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.809+0000 7ff022000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fe5d95e50 0x560fe5d9a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:19.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.809+0000 7ff022000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fe5d95e50 0x560fe5d9a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:39782/0 (socket says 172.21.15.77:39782) 2024-09-10T09:59:19.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.809+0000 7ff022000640 1 -- 172.21.15.77:0/1647165280 learned_addr learned my addr 172.21.15.77:0/1647165280 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:59:19.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.809+0000 7ff021600640 1 --2- 172.21.15.77:0/1647165280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fe5d8ac70 0x560fe5d998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:19.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.810+0000 7ff020c00640 1 --2- 172.21.15.77:0/1647165280 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560fe5d8b590 0x560fe5d99e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:19.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.810+0000 7ff021600640 1 -- 172.21.15.77:0/1647165280 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560fe5d8b590 msgr2=0x560fe5d99e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:19.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.810+0000 7ff021600640 1 --2- 172.21.15.77:0/1647165280 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560fe5d8b590 0x560fe5d99e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:19.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.810+0000 7ff021600640 1 -- 172.21.15.77:0/1647165280 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fe5d95e50 msgr2=0x560fe5d9a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:19.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.810+0000 7ff021600640 1 --2- 172.21.15.77:0/1647165280 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fe5d95e50 0x560fe5d9a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:19.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.810+0000 7ff021600640 1 -- 172.21.15.77:0/1647165280 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fe5e0fad0 con 0x560fe5d8ac70 2024-09-10T09:59:19.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.810+0000 7ff020c00640 1 --2- 172.21.15.77:0/1647165280 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560fe5d8b590 0x560fe5d99e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:59:19.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.810+0000 7ff022000640 1 --2- 172.21.15.77:0/1647165280 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fe5d95e50 0x560fe5d9a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:59:19.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.810+0000 7ff021600640 1 --2- 172.21.15.77:0/1647165280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fe5d8ac70 0x560fe5d998f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff010000c00 tx=0x7ff01003b460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25399afb3cf5e709 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:19.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.810+0000 7ff002a00640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-10T09:59:19.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.810+0000 7ff002a00640 1 -- 172.21.15.77:0/1647165280 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff010049030 con 0x560fe5d8ac70 2024-09-10T09:59:19.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.810+0000 7ff002a00640 1 -- 172.21.15.77:0/1647165280 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7ff0100f5320 con 0x560fe5d8ac70 2024-09-10T09:59:19.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.810+0000 7ff026c1a740 1 -- 172.21.15.77:0/1647165280 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0100bbf20 con 0x560fe5d8ac70 2024-09-10T09:59:19.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.811+0000 7ff002a00640 1 -- 172.21.15.77:0/1647165280 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff0100fe070 con 0x560fe5d8ac70 2024-09-10T09:59:19.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.811+0000 7ff026c1a740 10 client.26257 fetch_fsmap learned FSMap version 32 2024-09-10T09:59:19.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.811+0000 7ff026c1a740 1 -- 172.21.15.77:0/1647165280 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560fe5e0fad0 con 0x560fe5d8ac70 2024-09-10T09:59:19.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.811+0000 7ff002a00640 1 -- 172.21.15.77:0/1647165280 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0100bbf20 con 0x560fe5d8ac70 2024-09-10T09:59:19.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.811+0000 7ff002a00640 1 -- 172.21.15.77:0/1647165280 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7ff0100b8590 con 0x560fe5d8ac70 2024-09-10T09:59:19.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.811+0000 7ff026c1a740 10 client.26257 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T09:59:19.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.811+0000 7ff026c1a740 10 client.26257 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T09:59:19.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.811+0000 7ff026c1a740 20 client.26257 populate_metadata read hostname 'smithi077' 2024-09-10T09:59:19.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.811+0000 7ff026c1a740 1 --2- 172.21.15.77:0/1647165280 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x560fe5e10e50 0x560fe5e13240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:19.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.811+0000 7ff026c1a740 4 client.26257 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:59:19.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.811+0000 7ff026c1a740 1 -- 172.21.15.77:0/1647165280 --> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560fe5cf8e60 con 0x560fe5e10e50 2024-09-10T09:59:19.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.812+0000 7ff020c00640 1 --2- 172.21.15.77:0/1647165280 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x560fe5e10e50 0x560fe5e13240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:19.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.812+0000 7ff020c00640 1 --2- 172.21.15.77:0/1647165280 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x560fe5e10e50 0x560fe5e13240 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9046b8dc650ca4c6 server_cookie=3aaa9d1043b197bf in_seq=0 out_seq=0 2024-09-10T09:59:19.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.812+0000 7ff002a00640 10 client.26257 ms_handle_connect on v2:172.21.15.134:6836/2461497091 2024-09-10T09:59:19.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.812+0000 7ff002a00640 1 -- 172.21.15.77:0/1647165280 <== mds.3 v2:172.21.15.134:6836/2461497091 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560fe5cf8e60 con 0x560fe5e10e50 2024-09-10T09:59:19.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.812+0000 7ff002a00640 10 client.26257 handle_command_reply: tid=0 2024-09-10T09:59:19.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.812+0000 7ff026c1a740 1 client.26257 shutdown 2024-09-10T09:59:19.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.813+0000 7ff026c1a740 1 -- 172.21.15.77:0/1647165280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fe5d8ac70 msgr2=0x560fe5d998f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:19.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.813+0000 7ff026c1a740 1 --2- 172.21.15.77:0/1647165280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fe5d8ac70 0x560fe5d998f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff010000c00 tx=0x7ff01003b460 comp rx=0 tx=0).stop 2024-09-10T09:59:19.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.813+0000 7ff026c1a740 1 -- 172.21.15.77:0/1647165280 shutdown_connections 2024-09-10T09:59:19.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.813+0000 7ff026c1a740 1 --2- 172.21.15.77:0/1647165280 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x560fe5e10e50 0x560fe5e13240 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:19.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.813+0000 7ff026c1a740 1 --2- 172.21.15.77:0/1647165280 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fe5d95e50 0x560fe5d9a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:19.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.813+0000 7ff026c1a740 1 --2- 172.21.15.77:0/1647165280 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560fe5d8b590 0x560fe5d99e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:19.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.813+0000 7ff002a00640 0 client.26257 ms_handle_reset on v2:172.21.15.134:6836/2461497091 2024-09-10T09:59:19.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.813+0000 7ff026c1a740 1 --2- 172.21.15.77:0/1647165280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fe5d8ac70 0x560fe5d998f0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:19.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.813+0000 7ff026c1a740 1 -- 172.21.15.77:0/1647165280 >> 172.21.15.77:0/1647165280 conn(0x560fe5d87200 msgr2=0x560fe5d85b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:19.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.813+0000 7ff026c1a740 1 -- 172.21.15.77:0/1647165280 shutdown_connections 2024-09-10T09:59:19.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.813+0000 7ff026c1a740 1 -- 172.21.15.77:0/1647165280 wait complete. 2024-09-10T09:59:19.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.813+0000 7ff026c1a740 20 client.26257 trim_cache size 0 max 16384 2024-09-10T09:59:19.820 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-10T09:59:19.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.814+0000 7ff024000640 1 -- 172.21.15.77:0/2918922222 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ff0040a2c00 msgr2=0x7ff0040a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:19.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.814+0000 7ff024000640 1 --2- 172.21.15.77:0/2918922222 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ff0040a2c00 0x7ff0040a50c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff014005fd0 tx=0x7ff014009500 comp rx=0 tx=0).stop 2024-09-10T09:59:19.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.814+0000 7ff024000640 1 -- 172.21.15.77:0/2918922222 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff01c101de0 msgr2=0x7ff01c0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:19.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.814+0000 7ff024000640 1 --2- 172.21.15.77:0/2918922222 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff01c101de0 0x7ff01c0744f0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7ff010004770 tx=0x7ff010003c10 comp rx=0 tx=0).stop 2024-09-10T09:59:19.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.814+0000 7ff024000640 1 -- 172.21.15.77:0/2918922222 shutdown_connections 2024-09-10T09:59:19.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.814+0000 7ff024000640 1 --2- 172.21.15.77:0/2918922222 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ff0040a2c00 0x7ff0040a50c0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:19.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.814+0000 7ff024000640 1 --2- 172.21.15.77:0/2918922222 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff01c106f10 0x7ff01c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:19.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.814+0000 7ff024000640 1 --2- 172.21.15.77:0/2918922222 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff01c102790 0x7ff01c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:19.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.814+0000 7ff024000640 1 --2- 172.21.15.77:0/2918922222 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff01c101de0 0x7ff01c0744f0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:19.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.814+0000 7ff024000640 1 -- 172.21.15.77:0/2918922222 >> 172.21.15.77:0/2918922222 conn(0x7ff01c0fd8c0 msgr2=0x7ff01c0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:19.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.814+0000 7ff024000640 1 -- 172.21.15.77:0/2918922222 shutdown_connections 2024-09-10T09:59:19.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:19.814+0000 7ff024000640 1 -- 172.21.15.77:0/2918922222 wait complete. 2024-09-10T09:59:20.701 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T09:59:21.073 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:59:21.212 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:59:20 smithi077 ceph-mon[30319]: pgmap v516: 129 pgs: 129 active+clean; 96 MiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.3 MiB/s wr, 394 op/s 2024-09-10T09:59:21.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:59:20 smithi134 ceph-mon[35674]: pgmap v516: 129 pgs: 129 active+clean; 96 MiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.3 MiB/s wr, 394 op/s 2024-09-10T09:59:21.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:59:20 smithi120 ceph-mon[34770]: pgmap v516: 129 pgs: 129 active+clean; 96 MiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.3 MiB/s wr, 394 op/s 2024-09-10T09:59:21.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.630+0000 7efcb1c00640 1 -- 172.21.15.77:0/1126117314 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efcac102790 msgr2=0x7efcac1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:21.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.630+0000 7efcb1c00640 1 --2- 172.21.15.77:0/1126117314 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efcac102790 0x7efcac1067e0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7efc98009900 tx=0x7efc9802a3f0 comp rx=0 tx=0).stop 2024-09-10T09:59:21.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.630+0000 7efcb1c00640 1 -- 172.21.15.77:0/1126117314 shutdown_connections 2024-09-10T09:59:21.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.630+0000 7efcb1c00640 1 --2- 172.21.15.77:0/1126117314 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efcac106f10 0x7efcac10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:21.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.630+0000 7efcb1c00640 1 --2- 172.21.15.77:0/1126117314 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efcac102790 0x7efcac1067e0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:21.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.630+0000 7efcb1c00640 1 --2- 172.21.15.77:0/1126117314 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efcac101de0 0x7efcac1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:21.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.630+0000 7efcb1c00640 1 -- 172.21.15.77:0/1126117314 >> 172.21.15.77:0/1126117314 conn(0x7efcac0fd8c0 msgr2=0x7efcac0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:21.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.630+0000 7efcb1c00640 1 -- 172.21.15.77:0/1126117314 shutdown_connections 2024-09-10T09:59:21.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.630+0000 7efcb1c00640 1 -- 172.21.15.77:0/1126117314 wait complete. 2024-09-10T09:59:21.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.631+0000 7efcb1c00640 1 Processor -- start 2024-09-10T09:59:21.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.631+0000 7efcb1c00640 1 -- start start 2024-09-10T09:59:21.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.632+0000 7efcb1c00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efcac101de0 0x7efcac198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:21.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.632+0000 7efcb1c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efcac102790 0x7efcac198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:21.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.632+0000 7efcb1c00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efcac106f10 0x7efcac19cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:21.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.632+0000 7efcb1c00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7efcac111f30 con 0x7efcac101de0 2024-09-10T09:59:21.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.632+0000 7efcb1c00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7efcac111db0 con 0x7efcac106f10 2024-09-10T09:59:21.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.632+0000 7efcb1c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7efcac1120b0 con 0x7efcac102790 2024-09-10T09:59:21.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.632+0000 7efcaaa00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efcac101de0 0x7efcac198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:21.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.632+0000 7efcaaa00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efcac101de0 0x7efcac198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:39798/0 (socket says 172.21.15.77:39798) 2024-09-10T09:59:21.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.632+0000 7efcaaa00640 1 -- 172.21.15.77:0/3452207602 learned_addr learned my addr 172.21.15.77:0/3452207602 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:59:21.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.632+0000 7efcaa000640 1 --2- 172.21.15.77:0/3452207602 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efcac102790 0x7efcac198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:21.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.632+0000 7efcab400640 1 --2- 172.21.15.77:0/3452207602 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efcac106f10 0x7efcac19cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:21.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.633+0000 7efcaaa00640 1 -- 172.21.15.77:0/3452207602 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efcac102790 msgr2=0x7efcac198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:21.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.633+0000 7efcaaa00640 1 --2- 172.21.15.77:0/3452207602 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efcac102790 0x7efcac198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:21.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.633+0000 7efcaaa00640 1 -- 172.21.15.77:0/3452207602 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efcac106f10 msgr2=0x7efcac19cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:21.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.633+0000 7efcaaa00640 1 --2- 172.21.15.77:0/3452207602 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efcac106f10 0x7efcac19cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:21.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.633+0000 7efcaaa00640 1 -- 172.21.15.77:0/3452207602 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcac19d3c0 con 0x7efcac101de0 2024-09-10T09:59:21.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.633+0000 7efcab400640 1 --2- 172.21.15.77:0/3452207602 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efcac106f10 0x7efcac19cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:59:21.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.633+0000 7efcaaa00640 1 --2- 172.21.15.77:0/3452207602 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efcac101de0 0x7efcac198410 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7efc9c00e9f0 tx=0x7efc9c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:21.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.633+0000 7efca3400640 1 -- 172.21.15.77:0/3452207602 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc9c00cd80 con 0x7efcac101de0 2024-09-10T09:59:21.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.633+0000 7efca3400640 1 -- 172.21.15.77:0/3452207602 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc9c010910 con 0x7efcac101de0 2024-09-10T09:59:21.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.633+0000 7efcb1c00640 1 -- 172.21.15.77:0/3452207602 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcac19d6b0 con 0x7efcac101de0 2024-09-10T09:59:21.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.633+0000 7efca3400640 1 -- 172.21.15.77:0/3452207602 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc9c018990 con 0x7efcac101de0 2024-09-10T09:59:21.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.633+0000 7efcb1c00640 1 -- 172.21.15.77:0/3452207602 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcac1a4f90 con 0x7efcac101de0 2024-09-10T09:59:21.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.637+0000 7efcb461b740 1 Processor -- start 2024-09-10T09:59:21.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.638+0000 7efcb461b740 1 -- start start 2024-09-10T09:59:21.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.638+0000 7efca3400640 1 -- 172.21.15.77:0/3452207602 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7efc9c010ac0 con 0x7efcac101de0 2024-09-10T09:59:21.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.638+0000 7efcb461b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5569aa18ec70 0x5569aa18f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:21.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.638+0000 7efcb461b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5569aa18f590 0x5569aa199910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:21.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.638+0000 7efcb461b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5569aa199e50 0x5569aa19c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:21.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.638+0000 7efcb461b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5569aa0f53e0 con 0x5569aa18f590 2024-09-10T09:59:21.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.638+0000 7efcb461b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5569aa09cd80 con 0x5569aa199e50 2024-09-10T09:59:21.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.638+0000 7efcb461b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5569aa0f13a0 con 0x5569aa18ec70 2024-09-10T09:59:21.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.638+0000 7efcab400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5569aa18ec70 0x5569aa18f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:21.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.638+0000 7efcaaa00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5569aa18f590 0x5569aa199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:21.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.638+0000 7efcab400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5569aa18ec70 0x5569aa18f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:53394/0 (socket says 172.21.15.77:53394) 2024-09-10T09:59:21.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.638+0000 7efcab400640 1 -- 172.21.15.77:0/3872625726 learned_addr learned my addr 172.21.15.77:0/3872625726 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:59:21.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.638+0000 7efcaa000640 1 --2- 172.21.15.77:0/3872625726 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5569aa199e50 0x5569aa19c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:21.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.639+0000 7efcab400640 1 -- 172.21.15.77:0/3872625726 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5569aa199e50 msgr2=0x5569aa19c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:21.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.639+0000 7efcab400640 1 --2- 172.21.15.77:0/3872625726 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5569aa199e50 0x5569aa19c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:21.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.639+0000 7efcab400640 1 -- 172.21.15.77:0/3872625726 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5569aa18f590 msgr2=0x5569aa199910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:21.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.639+0000 7efcab400640 1 --2- 172.21.15.77:0/3872625726 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5569aa18f590 0x5569aa199910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:21.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.639+0000 7efcab400640 1 -- 172.21.15.77:0/3872625726 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569aa0bbae0 con 0x5569aa18ec70 2024-09-10T09:59:21.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.639+0000 7efcab400640 1 --2- 172.21.15.77:0/3872625726 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5569aa18ec70 0x5569aa18f050 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7efca4009fd0 tx=0x7efca4010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65abaaa6bbf43829 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:21.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.639+0000 7efc97e00640 1 -- 172.21.15.77:0/3872625726 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca4011700 con 0x5569aa18ec70 2024-09-10T09:59:21.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.639+0000 7efc97e00640 1 -- 172.21.15.77:0/3872625726 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efca4011d80 con 0x5569aa18ec70 2024-09-10T09:59:21.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.639+0000 7efc97e00640 1 -- 172.21.15.77:0/3872625726 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca400e3c0 con 0x5569aa18ec70 2024-09-10T09:59:21.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.640+0000 7efca3400640 1 --2- 172.21.15.77:0/3452207602 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7efc900a2c00 0x7efc900a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:21.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.640+0000 7efca3400640 1 -- 172.21.15.77:0/3452207602 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7efc9c0cf7b0 con 0x7efcac101de0 2024-09-10T09:59:21.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.640+0000 7efcaaa00640 1 --2- 172.21.15.77:0/3452207602 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7efc900a2c00 0x7efc900a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:21.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.640+0000 7efcb461b740 1 -- 172.21.15.77:0/3872625726 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5569aa18ec70 msgr2=0x5569aa18f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:21.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.640+0000 7efcb461b740 1 --2- 172.21.15.77:0/3872625726 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5569aa18ec70 0x5569aa18f050 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7efca4009fd0 tx=0x7efca4010860 comp rx=0 tx=0).stop 2024-09-10T09:59:21.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.640+0000 7efcb461b740 1 -- 172.21.15.77:0/3872625726 shutdown_connections 2024-09-10T09:59:21.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.640+0000 7efcb461b740 1 --2- 172.21.15.77:0/3872625726 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5569aa199e50 0x5569aa19c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:21.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.640+0000 7efcb461b740 1 --2- 172.21.15.77:0/3872625726 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5569aa18f590 0x5569aa199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:21.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.640+0000 7efcb461b740 1 --2- 172.21.15.77:0/3872625726 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5569aa18ec70 0x5569aa18f050 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:21.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.640+0000 7efcb461b740 1 -- 172.21.15.77:0/3872625726 >> 172.21.15.77:0/3872625726 conn(0x5569aa18b200 msgr2=0x5569aa18b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:21.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.641+0000 7efcaaa00640 1 --2- 172.21.15.77:0/3452207602 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7efc900a2c00 0x7efc900a50c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7efc9c098590 tx=0x7efc9c0d5530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:21.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.641+0000 7efcb461b740 1 -- 172.21.15.77:0/3872625726 shutdown_connections 2024-09-10T09:59:21.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.641+0000 7efcb461b740 1 -- 172.21.15.77:0/3872625726 wait complete. 2024-09-10T09:59:21.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.641+0000 7efcb461b740 1 -- start start 2024-09-10T09:59:21.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.642+0000 7efcb461b740 1 Processor -- start 2024-09-10T09:59:21.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.642+0000 7efcb461b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5569aa18ec70 0x5569aa196ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:21.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.642+0000 7efcb461b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5569aa18f590 0x5569aa197220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:21.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.642+0000 7efcb461b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5569aa199e50 0x5569aa197760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:21.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.642+0000 7efcb461b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5569aa1a0e30 con 0x5569aa199e50 2024-09-10T09:59:21.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.642+0000 7efcb461b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5569aa0b00f0 con 0x5569aa18f590 2024-09-10T09:59:21.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.642+0000 7efcb461b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5569aa1a0fb0 con 0x5569aa18ec70 2024-09-10T09:59:21.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.642+0000 7efcb461b740 10 client.0 _ll_register_callbacks cb 0x5569aa097bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:59:21.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.642+0000 7efcab400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5569aa18ec70 0x5569aa196ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:21.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.642+0000 7efcaa000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5569aa18f590 0x5569aa197220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:21.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.642+0000 7efcab400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5569aa199e50 0x5569aa197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:21.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.642+0000 7efcaa000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5569aa18f590 0x5569aa197220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:42244/0 (socket says 172.21.15.77:42244) 2024-09-10T09:59:21.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.642+0000 7efcaa000640 1 -- 172.21.15.77:0/666102810 learned_addr learned my addr 172.21.15.77:0/666102810 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:59:21.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.643+0000 7efcab400640 1 -- 172.21.15.77:0/666102810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5569aa18ec70 msgr2=0x5569aa196ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:21.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.643+0000 7efcab400640 1 --2- 172.21.15.77:0/666102810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5569aa18ec70 0x5569aa196ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:21.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.643+0000 7efcab400640 1 -- 172.21.15.77:0/666102810 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5569aa18f590 msgr2=0x5569aa197220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:21.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.643+0000 7efcab400640 1 --2- 172.21.15.77:0/666102810 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5569aa18f590 0x5569aa197220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:21.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.643+0000 7efcab400640 1 -- 172.21.15.77:0/666102810 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569a9e663c0 con 0x5569aa199e50 2024-09-10T09:59:21.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.643+0000 7efcaa000640 1 --2- 172.21.15.77:0/666102810 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5569aa18f590 0x5569aa197220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:59:21.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.643+0000 7efcab400640 1 --2- 172.21.15.77:0/666102810 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5569aa199e50 0x5569aa197760 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7efca4005fd0 tx=0x7efca4010fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9be314fe438ec6e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:21.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.643+0000 7efc94c00640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-10T09:59:21.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.644+0000 7efc94c00640 1 -- 172.21.15.77:0/666102810 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca4024040 con 0x5569aa199e50 2024-09-10T09:59:21.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.644+0000 7efcb461b740 1 -- 172.21.15.77:0/666102810 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efca4011d80 con 0x5569aa199e50 2024-09-10T09:59:21.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.644+0000 7efc94c00640 1 -- 172.21.15.77:0/666102810 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7efca4011ac0 con 0x5569aa199e50 2024-09-10T09:59:21.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.644+0000 7efc94c00640 1 -- 172.21.15.77:0/666102810 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7efca402b430 con 0x5569aa199e50 2024-09-10T09:59:21.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.644+0000 7efcb461b740 10 client.16236 fetch_fsmap learned FSMap version 32 2024-09-10T09:59:21.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.644+0000 7efcb461b740 1 -- 172.21.15.77:0/666102810 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5569aa1ace00 con 0x5569aa199e50 2024-09-10T09:59:21.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.644+0000 7efc94c00640 1 -- 172.21.15.77:0/666102810 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efca4011d80 con 0x5569aa199e50 2024-09-10T09:59:21.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.644+0000 7efc94c00640 1 -- 172.21.15.77:0/666102810 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7efca402a590 con 0x5569aa199e50 2024-09-10T09:59:21.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.645+0000 7efcb461b740 10 client.16236 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T09:59:21.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.645+0000 7efcb461b740 10 client.16236 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T09:59:21.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.645+0000 7efcb461b740 20 client.16236 populate_metadata read hostname 'smithi077' 2024-09-10T09:59:21.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.645+0000 7efcb461b740 1 --2- 172.21.15.77:0/666102810 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x5569aa1ae4e0 0x5569aa1b08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:21.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.645+0000 7efcb461b740 4 client.16236 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:59:21.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.645+0000 7efcb461b740 1 -- 172.21.15.77:0/666102810 --> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5569aa102950 con 0x5569aa1ae4e0 2024-09-10T09:59:21.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.645+0000 7efcaa000640 1 --2- 172.21.15.77:0/666102810 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x5569aa1ae4e0 0x5569aa1b08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:21.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.646+0000 7efcaa000640 1 --2- 172.21.15.77:0/666102810 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x5569aa1ae4e0 0x5569aa1b08d0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7d8c3c630d732ef server_cookie=e4d069354a184529 in_seq=0 out_seq=0 2024-09-10T09:59:21.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.646+0000 7efc94c00640 10 client.16236 ms_handle_connect on v2:172.21.15.134:6838/1965117680 2024-09-10T09:59:21.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.647+0000 7efc94c00640 1 -- 172.21.15.77:0/666102810 <== mds.2 v2:172.21.15.134:6838/1965117680 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5569aa102950 con 0x5569aa1ae4e0 2024-09-10T09:59:21.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.647+0000 7efc94c00640 10 client.16236 handle_command_reply: tid=0 2024-09-10T09:59:21.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.647+0000 7efcb461b740 1 client.16236 shutdown 2024-09-10T09:59:21.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.647+0000 7efcb461b740 1 -- 172.21.15.77:0/666102810 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5569aa199e50 msgr2=0x5569aa197760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:21.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.647+0000 7efcb461b740 1 --2- 172.21.15.77:0/666102810 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5569aa199e50 0x5569aa197760 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7efca4005fd0 tx=0x7efca4010fd0 comp rx=0 tx=0).stop 2024-09-10T09:59:21.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.647+0000 7efcb461b740 1 -- 172.21.15.77:0/666102810 shutdown_connections 2024-09-10T09:59:21.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.647+0000 7efcb461b740 1 --2- 172.21.15.77:0/666102810 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x5569aa1ae4e0 0x5569aa1b08d0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:21.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.647+0000 7efcb461b740 1 --2- 172.21.15.77:0/666102810 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5569aa199e50 0x5569aa197760 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:21.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.647+0000 7efcb461b740 1 --2- 172.21.15.77:0/666102810 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5569aa18f590 0x5569aa197220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:21.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.647+0000 7efcb461b740 1 --2- 172.21.15.77:0/666102810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5569aa18ec70 0x5569aa196ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:21.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.647+0000 7efcb461b740 1 -- 172.21.15.77:0/666102810 >> 172.21.15.77:0/666102810 conn(0x5569aa18b200 msgr2=0x5569aa18a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:21.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.647+0000 7efc94c00640 0 client.16236 ms_handle_reset on v2:172.21.15.134:6838/1965117680 2024-09-10T09:59:21.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.647+0000 7efcb461b740 1 -- 172.21.15.77:0/666102810 shutdown_connections 2024-09-10T09:59:21.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.647+0000 7efcb461b740 1 -- 172.21.15.77:0/666102810 wait complete. 2024-09-10T09:59:21.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.647+0000 7efcb461b740 20 client.16236 trim_cache size 0 max 16384 2024-09-10T09:59:21.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.661+0000 7efcb461b740 1 Processor -- start 2024-09-10T09:59:21.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.661+0000 7efcb461b740 1 -- start start 2024-09-10T09:59:21.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.661+0000 7efcb461b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5569aa18ec70 0x5569aa193f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:21.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.661+0000 7efcb461b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5569aa18f590 0x5569aa194480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:21.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.661+0000 7efcb461b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5569aa199e50 0x5569aa198540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:21.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.661+0000 7efcb461b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7efcac1acd20 con 0x5569aa18f590 2024-09-10T09:59:21.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.661+0000 7efcb461b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7efcac1ae920 con 0x5569aa18ec70 2024-09-10T09:59:21.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.661+0000 7efcb461b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7efcac1ad3f0 con 0x5569aa199e50 2024-09-10T09:59:21.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.661+0000 7efcaa000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5569aa18f590 0x5569aa194480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:21.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.661+0000 7efcaa000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5569aa18f590 0x5569aa194480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:39838/0 (socket says 172.21.15.77:39838) 2024-09-10T09:59:21.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.661+0000 7efcaa000640 1 -- 172.21.15.77:0/1260659642 learned_addr learned my addr 172.21.15.77:0/1260659642 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:59:21.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.662+0000 7efcab400640 1 --2- 172.21.15.77:0/1260659642 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5569aa199e50 0x5569aa198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:21.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.662+0000 7efcab400640 1 --2- 172.21.15.77:0/1260659642 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5569aa18ec70 0x5569aa193f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:21.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.662+0000 7efcaa000640 1 -- 172.21.15.77:0/1260659642 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5569aa199e50 msgr2=0x5569aa198540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:21.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.662+0000 7efcaa000640 1 --2- 172.21.15.77:0/1260659642 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5569aa199e50 0x5569aa198540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:21.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.662+0000 7efcaa000640 1 -- 172.21.15.77:0/1260659642 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5569aa18ec70 msgr2=0x5569aa193f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:21.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.662+0000 7efcaa000640 1 --2- 172.21.15.77:0/1260659642 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5569aa18ec70 0x5569aa193f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:21.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.662+0000 7efcaa000640 1 -- 172.21.15.77:0/1260659642 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569aa18f3c0 con 0x5569aa18f590 2024-09-10T09:59:21.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.662+0000 7efcab400640 1 --2- 172.21.15.77:0/1260659642 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5569aa18ec70 0x5569aa193f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:59:21.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.662+0000 7efcaa000640 1 --2- 172.21.15.77:0/1260659642 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5569aa18f590 0x5569aa194480 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7efc9802a910 tx=0x7efc980318f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c66ed27391bd4beb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:21.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.662+0000 7efc97400640 1 -- 172.21.15.77:0/1260659642 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc98035040 con 0x5569aa18f590 2024-09-10T09:59:21.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.662+0000 7efc97400640 1 -- 172.21.15.77:0/1260659642 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc98047470 con 0x5569aa18f590 2024-09-10T09:59:21.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.662+0000 7efc97400640 1 -- 172.21.15.77:0/1260659642 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc98003850 con 0x5569aa18f590 2024-09-10T09:59:21.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.663+0000 7efcb461b740 1 -- 172.21.15.77:0/1260659642 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5569aa18f590 msgr2=0x5569aa194480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:21.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.663+0000 7efcb461b740 1 --2- 172.21.15.77:0/1260659642 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5569aa18f590 0x5569aa194480 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7efc9802a910 tx=0x7efc980318f0 comp rx=0 tx=0).stop 2024-09-10T09:59:21.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.663+0000 7efcb461b740 1 -- 172.21.15.77:0/1260659642 shutdown_connections 2024-09-10T09:59:21.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.663+0000 7efcb461b740 1 --2- 172.21.15.77:0/1260659642 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5569aa199e50 0x5569aa198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:21.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.663+0000 7efcb461b740 1 --2- 172.21.15.77:0/1260659642 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5569aa18f590 0x5569aa194480 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:21.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.663+0000 7efcb461b740 1 --2- 172.21.15.77:0/1260659642 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5569aa18ec70 0x5569aa193f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:21.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.663+0000 7efcb461b740 1 -- 172.21.15.77:0/1260659642 >> 172.21.15.77:0/1260659642 conn(0x5569aa18b200 msgr2=0x5569aa191f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:21.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.663+0000 7efcb461b740 1 -- 172.21.15.77:0/1260659642 shutdown_connections 2024-09-10T09:59:21.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.663+0000 7efcb461b740 1 -- 172.21.15.77:0/1260659642 wait complete. 2024-09-10T09:59:21.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.664+0000 7efcb461b740 1 -- start start 2024-09-10T09:59:21.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.664+0000 7efcb461b740 1 Processor -- start 2024-09-10T09:59:21.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.664+0000 7efcb461b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5569aa18ec70 0x5569aa1b08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:21.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.664+0000 7efcb461b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5569aa18f590 0x5569aa1b0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:21.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.664+0000 7efcb461b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5569aa199e50 0x5569aa216e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:21.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.664+0000 7efcb461b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7efcac1ae190 con 0x5569aa18f590 2024-09-10T09:59:21.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.664+0000 7efcb461b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7efcac1adac0 con 0x5569aa18ec70 2024-09-10T09:59:21.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.664+0000 7efcb461b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7efcac1aad00 con 0x5569aa199e50 2024-09-10T09:59:21.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.665+0000 7efcb461b740 10 client.0 _ll_register_callbacks cb 0x7efc9c0d5400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:59:21.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.665+0000 7efcaa000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5569aa18f590 0x5569aa1b0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:21.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.665+0000 7efcaa000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5569aa18f590 0x5569aa1b0e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:39846/0 (socket says 172.21.15.77:39846) 2024-09-10T09:59:21.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.665+0000 7efcaa000640 1 -- 172.21.15.77:0/415878745 learned_addr learned my addr 172.21.15.77:0/415878745 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:59:21.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.665+0000 7efcab400640 1 --2- 172.21.15.77:0/415878745 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5569aa18ec70 0x5569aa1b08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:21.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.665+0000 7efcab400640 1 --2- 172.21.15.77:0/415878745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5569aa199e50 0x5569aa216e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:21.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.665+0000 7efcaa000640 1 -- 172.21.15.77:0/415878745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5569aa199e50 msgr2=0x5569aa216e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:21.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.665+0000 7efcaa000640 1 --2- 172.21.15.77:0/415878745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5569aa199e50 0x5569aa216e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:21.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.665+0000 7efcaa000640 1 -- 172.21.15.77:0/415878745 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5569aa18ec70 msgr2=0x5569aa1b08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:21.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.665+0000 7efcaa000640 1 --2- 172.21.15.77:0/415878745 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5569aa18ec70 0x5569aa1b08f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:21.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.665+0000 7efcaa000640 1 -- 172.21.15.77:0/415878745 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569aa217400 con 0x5569aa18f590 2024-09-10T09:59:21.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.666+0000 7efcaa000640 1 --2- 172.21.15.77:0/415878745 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5569aa18f590 0x5569aa1b0e30 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7efc98031710 tx=0x7efc98031740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cca7c90b047fd2a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:21.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.666+0000 7efc94c00640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-10T09:59:21.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.666+0000 7efc94c00640 1 -- 172.21.15.77:0/415878745 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc98035030 con 0x5569aa18f590 2024-09-10T09:59:21.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.666+0000 7efc94c00640 1 -- 172.21.15.77:0/415878745 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7efc98004240 con 0x5569aa18f590 2024-09-10T09:59:21.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.666+0000 7efcb461b740 1 -- 172.21.15.77:0/415878745 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc98047470 con 0x5569aa18f590 2024-09-10T09:59:21.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.666+0000 7efc94c00640 1 -- 172.21.15.77:0/415878745 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7efc98063050 con 0x5569aa18f590 2024-09-10T09:59:21.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.666+0000 7efcb461b740 10 client.16242 fetch_fsmap learned FSMap version 32 2024-09-10T09:59:21.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.666+0000 7efcb461b740 1 -- 172.21.15.77:0/415878745 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5569aa2186c0 con 0x5569aa18f590 2024-09-10T09:59:21.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.666+0000 7efc94c00640 1 -- 172.21.15.77:0/415878745 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc98047470 con 0x5569aa18f590 2024-09-10T09:59:21.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.667+0000 7efc94c00640 1 -- 172.21.15.77:0/415878745 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7efc9804f070 con 0x5569aa18f590 2024-09-10T09:59:21.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.667+0000 7efcb461b740 10 client.16242 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T09:59:21.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.667+0000 7efcb461b740 10 client.16242 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T09:59:21.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.667+0000 7efcb461b740 20 client.16242 populate_metadata read hostname 'smithi077' 2024-09-10T09:59:21.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.667+0000 7efcb461b740 1 --2- 172.21.15.77:0/415878745 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x5569aa218da0 0x5569aa21b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:21.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.667+0000 7efcb461b740 4 client.16242 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:59:21.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.667+0000 7efcb461b740 1 -- 172.21.15.77:0/415878745 --> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5569aa0fce60 con 0x5569aa218da0 2024-09-10T09:59:21.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.667+0000 7efcab400640 1 --2- 172.21.15.77:0/415878745 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x5569aa218da0 0x5569aa21b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:21.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.668+0000 7efcab400640 1 --2- 172.21.15.77:0/415878745 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x5569aa218da0 0x5569aa21b1c0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=86866c032c231766 server_cookie=9112a6717a9291a9 in_seq=0 out_seq=0 2024-09-10T09:59:21.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.668+0000 7efc94c00640 10 client.16242 ms_handle_connect on v2:172.21.15.134:6838/1965117680 2024-09-10T09:59:21.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.668+0000 7efc94c00640 1 -- 172.21.15.77:0/415878745 <== mds.2 v2:172.21.15.134:6838/1965117680 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5569aa0fce60 con 0x5569aa218da0 2024-09-10T09:59:21.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.668+0000 7efc94c00640 10 client.16242 handle_command_reply: tid=0 2024-09-10T09:59:21.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.668+0000 7efcb461b740 1 client.16242 shutdown 2024-09-10T09:59:21.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.668+0000 7efcb461b740 1 -- 172.21.15.77:0/415878745 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5569aa18f590 msgr2=0x5569aa1b0e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:21.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.668+0000 7efcb461b740 1 --2- 172.21.15.77:0/415878745 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5569aa18f590 0x5569aa1b0e30 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7efc98031710 tx=0x7efc98031740 comp rx=0 tx=0).stop 2024-09-10T09:59:21.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.668+0000 7efcb461b740 1 -- 172.21.15.77:0/415878745 shutdown_connections 2024-09-10T09:59:21.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.668+0000 7efcb461b740 1 --2- 172.21.15.77:0/415878745 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x5569aa218da0 0x5569aa21b1c0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:21.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.668+0000 7efcb461b740 1 --2- 172.21.15.77:0/415878745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5569aa199e50 0x5569aa216e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:21.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.668+0000 7efcb461b740 1 --2- 172.21.15.77:0/415878745 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5569aa18f590 0x5569aa1b0e30 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:21.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.668+0000 7efcb461b740 1 --2- 172.21.15.77:0/415878745 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5569aa18ec70 0x5569aa1b08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:21.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.668+0000 7efcb461b740 1 -- 172.21.15.77:0/415878745 >> 172.21.15.77:0/415878745 conn(0x5569aa18b200 msgr2=0x5569aa1906c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:21.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.668+0000 7efc94c00640 0 client.16242 ms_handle_reset on v2:172.21.15.134:6838/1965117680 2024-09-10T09:59:21.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.669+0000 7efcb461b740 1 -- 172.21.15.77:0/415878745 shutdown_connections 2024-09-10T09:59:21.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.669+0000 7efcb461b740 1 -- 172.21.15.77:0/415878745 wait complete. 2024-09-10T09:59:21.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.669+0000 7efcb461b740 20 client.16242 trim_cache size 0 max 16384 2024-09-10T09:59:21.673 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-10T09:59:21.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.669+0000 7efcb1c00640 1 -- 172.21.15.77:0/3452207602 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7efc900a2c00 msgr2=0x7efc900a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:21.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.669+0000 7efcb1c00640 1 --2- 172.21.15.77:0/3452207602 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7efc900a2c00 0x7efc900a50c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7efc9c098590 tx=0x7efc9c0d5530 comp rx=0 tx=0).stop 2024-09-10T09:59:21.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.669+0000 7efcb1c00640 1 -- 172.21.15.77:0/3452207602 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efcac101de0 msgr2=0x7efcac198410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:21.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.669+0000 7efcb1c00640 1 --2- 172.21.15.77:0/3452207602 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efcac101de0 0x7efcac198410 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7efc9c00e9f0 tx=0x7efc9c00eeb0 comp rx=0 tx=0).stop 2024-09-10T09:59:21.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.669+0000 7efcb1c00640 1 -- 172.21.15.77:0/3452207602 shutdown_connections 2024-09-10T09:59:21.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.669+0000 7efcb1c00640 1 --2- 172.21.15.77:0/3452207602 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7efc900a2c00 0x7efc900a50c0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:21.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.669+0000 7efcb1c00640 1 --2- 172.21.15.77:0/3452207602 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efcac106f10 0x7efcac19cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:21.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.669+0000 7efcb1c00640 1 --2- 172.21.15.77:0/3452207602 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efcac102790 0x7efcac198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:21.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.669+0000 7efcb1c00640 1 --2- 172.21.15.77:0/3452207602 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efcac101de0 0x7efcac198410 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:21.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.670+0000 7efcb1c00640 1 -- 172.21.15.77:0/3452207602 >> 172.21.15.77:0/3452207602 conn(0x7efcac0fd8c0 msgr2=0x7efcac0fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:21.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.670+0000 7efcb1c00640 1 -- 172.21.15.77:0/3452207602 shutdown_connections 2024-09-10T09:59:21.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:21.670+0000 7efcb1c00640 1 -- 172.21.15.77:0/3452207602 wait complete. 2024-09-10T09:59:22.579 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T09:59:22.944 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:59:23.221 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:59:22 smithi077 ceph-mon[30319]: pgmap v517: 129 pgs: 129 active+clean; 96 MiB data, 869 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 249 op/s 2024-09-10T09:59:23.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:59:22 smithi134 ceph-mon[35674]: pgmap v517: 129 pgs: 129 active+clean; 96 MiB data, 869 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 249 op/s 2024-09-10T09:59:23.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:59:22 smithi120 ceph-mon[34770]: pgmap v517: 129 pgs: 129 active+clean; 96 MiB data, 869 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 249 op/s 2024-09-10T09:59:23.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.491+0000 7f2726a00640 1 -- 172.21.15.77:0/913098671 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27201047e0 msgr2=0x7f2720108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:23.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.491+0000 7f2726a00640 1 --2- 172.21.15.77:0/913098671 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27201047e0 0x7f2720108830 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f270c009900 tx=0x7f270c02a3f0 comp rx=0 tx=0).stop 2024-09-10T09:59:23.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.491+0000 7f2726a00640 1 -- 172.21.15.77:0/913098671 shutdown_connections 2024-09-10T09:59:23.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.491+0000 7f2726a00640 1 --2- 172.21.15.77:0/913098671 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2720108f60 0x7f2720110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:23.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.491+0000 7f2726a00640 1 --2- 172.21.15.77:0/913098671 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27201047e0 0x7f2720108830 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:23.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.491+0000 7f2726a00640 1 --2- 172.21.15.77:0/913098671 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2720103e30 0x7f2720104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:23.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.491+0000 7f2726a00640 1 -- 172.21.15.77:0/913098671 >> 172.21.15.77:0/913098671 conn(0x7f27200ff8d0 msgr2=0x7f2720101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:23.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.491+0000 7f2726a00640 1 -- 172.21.15.77:0/913098671 shutdown_connections 2024-09-10T09:59:23.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.491+0000 7f2726a00640 1 -- 172.21.15.77:0/913098671 wait complete. 2024-09-10T09:59:23.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.492+0000 7f2726a00640 1 Processor -- start 2024-09-10T09:59:23.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.492+0000 7f2726a00640 1 -- start start 2024-09-10T09:59:23.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.493+0000 7f2726a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2720103e30 0x7f272019a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:23.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.493+0000 7f2726a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f27201047e0 0x7f272019a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:23.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.493+0000 7f2726a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2720108f60 0x7f272019ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:23.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.493+0000 7f2726a00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2720114010 con 0x7f27201047e0 2024-09-10T09:59:23.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.493+0000 7f2726a00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2720113e90 con 0x7f2720108f60 2024-09-10T09:59:23.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.493+0000 7f2726a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2720114190 con 0x7f2720103e30 2024-09-10T09:59:23.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.493+0000 7f2724c00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f27201047e0 0x7f272019a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:23.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.493+0000 7f2724c00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f27201047e0 0x7f272019a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:32992/0 (socket says 172.21.15.77:32992) 2024-09-10T09:59:23.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.493+0000 7f2724c00640 1 -- 172.21.15.77:0/508502173 learned_addr learned my addr 172.21.15.77:0/508502173 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:59:23.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.493+0000 7f2725600640 1 --2- 172.21.15.77:0/508502173 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2720103e30 0x7f272019a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:23.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.493+0000 7f2726000640 1 --2- 172.21.15.77:0/508502173 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2720108f60 0x7f272019ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:23.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.493+0000 7f2724c00640 1 -- 172.21.15.77:0/508502173 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2720103e30 msgr2=0x7f272019a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:23.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.493+0000 7f2724c00640 1 --2- 172.21.15.77:0/508502173 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2720103e30 0x7f272019a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:23.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.493+0000 7f2724c00640 1 -- 172.21.15.77:0/508502173 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2720108f60 msgr2=0x7f272019ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:23.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.493+0000 7f2724c00640 1 --2- 172.21.15.77:0/508502173 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2720108f60 0x7f272019ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:23.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.493+0000 7f2724c00640 1 -- 172.21.15.77:0/508502173 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f272019f450 con 0x7f27201047e0 2024-09-10T09:59:23.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.494+0000 7f2724c00640 1 --2- 172.21.15.77:0/508502173 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f27201047e0 0x7f272019a9e0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f270c004770 tx=0x7f270c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:23.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.494+0000 7f271e000640 1 -- 172.21.15.77:0/508502173 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f270c03f070 con 0x7f27201047e0 2024-09-10T09:59:23.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.494+0000 7f271e000640 1 -- 172.21.15.77:0/508502173 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f270c031ca0 con 0x7f27201047e0 2024-09-10T09:59:23.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.494+0000 7f2726a00640 1 -- 172.21.15.77:0/508502173 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f272019f6e0 con 0x7f27201047e0 2024-09-10T09:59:23.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.494+0000 7f271e000640 1 -- 172.21.15.77:0/508502173 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f270c035030 con 0x7f27201047e0 2024-09-10T09:59:23.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.494+0000 7f2726a00640 1 -- 172.21.15.77:0/508502173 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27201a7020 con 0x7f27201047e0 2024-09-10T09:59:23.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.497+0000 7f272aa8f740 1 Processor -- start 2024-09-10T09:59:23.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.497+0000 7f271e000640 1 -- 172.21.15.77:0/508502173 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f270c042070 con 0x7f27201047e0 2024-09-10T09:59:23.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.498+0000 7f272aa8f740 1 -- start start 2024-09-10T09:59:23.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.498+0000 7f272aa8f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603a3b63e40 0x5603a3b64220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:23.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.498+0000 7f272aa8f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5603a3b64760 0x5603a3b6eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:23.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.498+0000 7f272aa8f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5603a3b6f020 0x5603a3b71410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:23.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.498+0000 7f272aa8f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5603a3aca3e0 con 0x5603a3b6f020 2024-09-10T09:59:23.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.498+0000 7f272aa8f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5603a3a71d80 con 0x5603a3b64760 2024-09-10T09:59:23.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.498+0000 7f272aa8f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5603a3ac63a0 con 0x5603a3b63e40 2024-09-10T09:59:23.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.498+0000 7f2726000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603a3b63e40 0x5603a3b64220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:23.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.498+0000 7f2726000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603a3b63e40 0x5603a3b64220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:56458/0 (socket says 172.21.15.77:56458) 2024-09-10T09:59:23.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.498+0000 7f2726000640 1 -- 172.21.15.77:0/3759682504 learned_addr learned my addr 172.21.15.77:0/3759682504 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:59:23.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.498+0000 7f2724c00640 1 --2- 172.21.15.77:0/3759682504 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5603a3b6f020 0x5603a3b71410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:23.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.498+0000 7f2725600640 1 --2- 172.21.15.77:0/3759682504 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5603a3b64760 0x5603a3b6eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:23.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.499+0000 7f2726000640 1 -- 172.21.15.77:0/3759682504 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5603a3b64760 msgr2=0x5603a3b6eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:23.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.499+0000 7f2726000640 1 --2- 172.21.15.77:0/3759682504 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5603a3b64760 0x5603a3b6eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:23.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.499+0000 7f2726000640 1 -- 172.21.15.77:0/3759682504 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5603a3b6f020 msgr2=0x5603a3b71410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:23.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.499+0000 7f2726000640 1 --2- 172.21.15.77:0/3759682504 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5603a3b6f020 0x5603a3b71410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:23.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.499+0000 7f2726000640 1 -- 172.21.15.77:0/3759682504 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603a3a90ae0 con 0x5603a3b63e40 2024-09-10T09:59:23.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.499+0000 7f2724c00640 1 --2- 172.21.15.77:0/3759682504 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5603a3b6f020 0x5603a3b71410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:59:23.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.499+0000 7f271e000640 1 --2- 172.21.15.77:0/508502173 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f27080a2b80 0x7f27080a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:23.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.499+0000 7f2726000640 1 --2- 172.21.15.77:0/3759682504 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603a3b63e40 0x5603a3b64220 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2718009f60 tx=0x7f2718009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6807767be402894 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:23.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.499+0000 7f271e000640 1 -- 172.21.15.77:0/508502173 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f270c0efa50 con 0x7f27201047e0 2024-09-10T09:59:23.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.499+0000 7f2712600640 1 -- 172.21.15.77:0/3759682504 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f271801e070 con 0x5603a3b63e40 2024-09-10T09:59:23.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.499+0000 7f2725600640 1 --2- 172.21.15.77:0/508502173 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f27080a2b80 0x7f27080a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:23.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.500+0000 7f2712600640 1 -- 172.21.15.77:0/3759682504 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2718003c30 con 0x5603a3b63e40 2024-09-10T09:59:23.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.500+0000 7f2712600640 1 -- 172.21.15.77:0/3759682504 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f271800f980 con 0x5603a3b63e40 2024-09-10T09:59:23.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.500+0000 7f272aa8f740 1 -- 172.21.15.77:0/3759682504 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603a3b63e40 msgr2=0x5603a3b64220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:23.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.500+0000 7f272aa8f740 1 --2- 172.21.15.77:0/3759682504 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603a3b63e40 0x5603a3b64220 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2718009f60 tx=0x7f2718009f90 comp rx=0 tx=0).stop 2024-09-10T09:59:23.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.500+0000 7f2725600640 1 --2- 172.21.15.77:0/508502173 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f27080a2b80 0x7f27080a5040 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x5603a3b61230 tx=0x7f2714005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:23.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.501+0000 7f272aa8f740 1 -- 172.21.15.77:0/3759682504 shutdown_connections 2024-09-10T09:59:23.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.501+0000 7f272aa8f740 1 --2- 172.21.15.77:0/3759682504 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5603a3b6f020 0x5603a3b71410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:23.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.501+0000 7f272aa8f740 1 --2- 172.21.15.77:0/3759682504 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5603a3b64760 0x5603a3b6eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:23.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.501+0000 7f272aa8f740 1 --2- 172.21.15.77:0/3759682504 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603a3b63e40 0x5603a3b64220 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:23.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.501+0000 7f272aa8f740 1 -- 172.21.15.77:0/3759682504 >> 172.21.15.77:0/3759682504 conn(0x5603a3b603d0 msgr2=0x5603a3b607b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:23.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.501+0000 7f272aa8f740 1 -- 172.21.15.77:0/3759682504 shutdown_connections 2024-09-10T09:59:23.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.501+0000 7f272aa8f740 1 -- 172.21.15.77:0/3759682504 wait complete. 2024-09-10T09:59:23.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.501+0000 7f272aa8f740 1 -- start start 2024-09-10T09:59:23.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.502+0000 7f272aa8f740 1 Processor -- start 2024-09-10T09:59:23.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.502+0000 7f272aa8f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603a3b63e40 0x5603a3b6bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:23.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.502+0000 7f272aa8f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5603a3b64760 0x5603a3b6c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:23.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.502+0000 7f272aa8f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5603a3b6f020 0x5603a3b6c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:23.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.502+0000 7f272aa8f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5603a3b71bf0 con 0x5603a3b64760 2024-09-10T09:59:23.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.502+0000 7f272aa8f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5603a3a850f0 con 0x5603a3b6f020 2024-09-10T09:59:23.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.502+0000 7f272aa8f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5603a3b71d70 con 0x5603a3b63e40 2024-09-10T09:59:23.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.502+0000 7f272aa8f740 10 client.0 _ll_register_callbacks cb 0x5603a3a6cbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:59:23.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.502+0000 7f2724c00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5603a3b64760 0x5603a3b6c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:23.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.502+0000 7f2725600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603a3b63e40 0x5603a3b6bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:23.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.502+0000 7f2725600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603a3b63e40 0x5603a3b6bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:56464/0 (socket says 172.21.15.77:56464) 2024-09-10T09:59:23.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.502+0000 7f2725600640 1 -- 172.21.15.77:0/970382297 learned_addr learned my addr 172.21.15.77:0/970382297 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:59:23.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.502+0000 7f2726000640 1 --2- 172.21.15.77:0/970382297 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5603a3b6f020 0x5603a3b6c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:23.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.503+0000 7f2725600640 1 -- 172.21.15.77:0/970382297 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5603a3b6f020 msgr2=0x5603a3b6c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:23.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.503+0000 7f2725600640 1 --2- 172.21.15.77:0/970382297 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5603a3b6f020 0x5603a3b6c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:23.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.503+0000 7f2725600640 1 -- 172.21.15.77:0/970382297 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5603a3b64760 msgr2=0x5603a3b6c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:23.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.503+0000 7f2725600640 1 --2- 172.21.15.77:0/970382297 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5603a3b64760 0x5603a3b6c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:23.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.503+0000 7f2725600640 1 -- 172.21.15.77:0/970382297 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603a383b3c0 con 0x5603a3b63e40 2024-09-10T09:59:23.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.503+0000 7f2726000640 1 --2- 172.21.15.77:0/970382297 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5603a3b6f020 0x5603a3b6c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:59:23.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.503+0000 7f2725600640 1 --2- 172.21.15.77:0/970382297 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603a3b63e40 0x5603a3b6bf30 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f27140045f0 tx=0x7f2714004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f955f8c1113ecc2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:23.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.503+0000 7f2706a00640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-10T09:59:23.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.503+0000 7f2706a00640 1 -- 172.21.15.77:0/970382297 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2714009e60 con 0x5603a3b63e40 2024-09-10T09:59:23.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.504+0000 7f272aa8f740 1 -- 172.21.15.77:0/970382297 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2718003c30 con 0x5603a3b63e40 2024-09-10T09:59:23.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.504+0000 7f2706a00640 1 -- 172.21.15.77:0/970382297 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f271401f640 con 0x5603a3b63e40 2024-09-10T09:59:23.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.504+0000 7f2706a00640 1 -- 172.21.15.77:0/970382297 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f271401f840 con 0x5603a3b63e40 2024-09-10T09:59:23.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.504+0000 7f272aa8f740 10 client.26287 fetch_fsmap learned FSMap version 32 2024-09-10T09:59:23.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.504+0000 7f272aa8f740 1 -- 172.21.15.77:0/970382297 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5603a3b7dd10 con 0x5603a3b63e40 2024-09-10T09:59:23.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.504+0000 7f2706a00640 1 -- 172.21.15.77:0/970382297 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2718003c30 con 0x5603a3b63e40 2024-09-10T09:59:23.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.504+0000 7f2706a00640 1 -- 172.21.15.77:0/970382297 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f2714027020 con 0x5603a3b63e40 2024-09-10T09:59:23.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.504+0000 7f272aa8f740 10 client.26287 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T09:59:23.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.504+0000 7f272aa8f740 10 client.26287 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-10T09:59:23.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.505+0000 7f272aa8f740 20 client.26287 populate_metadata read hostname 'smithi077' 2024-09-10T09:59:23.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.505+0000 7f272aa8f740 1 --2- 172.21.15.77:0/970382297 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x5603a3b7f120 0x5603a3b81530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:23.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.505+0000 7f272aa8f740 4 client.26287 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:59:23.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.505+0000 7f272aa8f740 1 -- 172.21.15.77:0/970382297 --> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5603a3ad7950 con 0x5603a3b7f120 2024-09-10T09:59:23.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.505+0000 7f2724c00640 1 --2- 172.21.15.77:0/970382297 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x5603a3b7f120 0x5603a3b81530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:23.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.505+0000 7f2724c00640 1 --2- 172.21.15.77:0/970382297 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x5603a3b7f120 0x5603a3b81530 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2d49bae151f21b70 server_cookie=c6efddeabfe881d5 in_seq=0 out_seq=0 2024-09-10T09:59:23.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.505+0000 7f2706a00640 10 client.26287 ms_handle_connect on v2:172.21.15.120:6836/3033280709 2024-09-10T09:59:23.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.506+0000 7f2706a00640 1 -- 172.21.15.77:0/970382297 <== mds.1 v2:172.21.15.120:6836/3033280709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5603a3ad7950 con 0x5603a3b7f120 2024-09-10T09:59:23.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.506+0000 7f2706a00640 10 client.26287 handle_command_reply: tid=0 2024-09-10T09:59:23.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.506+0000 7f272aa8f740 1 client.26287 shutdown 2024-09-10T09:59:23.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.506+0000 7f272aa8f740 1 -- 172.21.15.77:0/970382297 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603a3b63e40 msgr2=0x5603a3b6bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:23.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.506+0000 7f272aa8f740 1 --2- 172.21.15.77:0/970382297 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603a3b63e40 0x5603a3b6bf30 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f27140045f0 tx=0x7f2714004730 comp rx=0 tx=0).stop 2024-09-10T09:59:23.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.507+0000 7f272aa8f740 1 -- 172.21.15.77:0/970382297 shutdown_connections 2024-09-10T09:59:23.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.507+0000 7f272aa8f740 1 --2- 172.21.15.77:0/970382297 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x5603a3b7f120 0x5603a3b81530 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:23.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.507+0000 7f272aa8f740 1 --2- 172.21.15.77:0/970382297 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5603a3b6f020 0x5603a3b6c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:23.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.507+0000 7f272aa8f740 1 --2- 172.21.15.77:0/970382297 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5603a3b64760 0x5603a3b6c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:23.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.507+0000 7f2706a00640 0 client.26287 ms_handle_reset on v2:172.21.15.120:6836/3033280709 2024-09-10T09:59:23.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.507+0000 7f272aa8f740 1 --2- 172.21.15.77:0/970382297 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603a3b63e40 0x5603a3b6bf30 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:23.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.507+0000 7f272aa8f740 1 -- 172.21.15.77:0/970382297 >> 172.21.15.77:0/970382297 conn(0x5603a3b603d0 msgr2=0x5603a3b660b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:23.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.507+0000 7f272aa8f740 1 -- 172.21.15.77:0/970382297 shutdown_connections 2024-09-10T09:59:23.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.507+0000 7f272aa8f740 1 -- 172.21.15.77:0/970382297 wait complete. 2024-09-10T09:59:23.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.507+0000 7f272aa8f740 20 client.26287 trim_cache size 0 max 16384 2024-09-10T09:59:23.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.520+0000 7f272aa8f740 1 Processor -- start 2024-09-10T09:59:23.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.520+0000 7f272aa8f740 1 -- start start 2024-09-10T09:59:23.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.521+0000 7f272aa8f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5603a3b63e40 0x5603a3b668a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:23.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.521+0000 7f272aa8f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603a3b64760 0x5603a3b5f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:23.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.521+0000 7f272aa8f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5603a3b6f020 0x5603a3b5f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:23.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.521+0000 7f272aa8f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f27201b0f20 con 0x5603a3b6f020 2024-09-10T09:59:23.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.521+0000 7f272aa8f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f27201b2b20 con 0x5603a3b63e40 2024-09-10T09:59:23.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.521+0000 7f272aa8f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f27201b15f0 con 0x5603a3b64760 2024-09-10T09:59:23.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.521+0000 7f2726000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5603a3b6f020 0x5603a3b5f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:23.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.521+0000 7f2726000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5603a3b6f020 0x5603a3b5f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33036/0 (socket says 172.21.15.77:33036) 2024-09-10T09:59:23.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.521+0000 7f2726000640 1 -- 172.21.15.77:0/879956512 learned_addr learned my addr 172.21.15.77:0/879956512 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:59:23.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.521+0000 7f2725600640 1 --2- 172.21.15.77:0/879956512 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5603a3b63e40 0x5603a3b668a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:23.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.521+0000 7f2724c00640 1 --2- 172.21.15.77:0/879956512 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603a3b64760 0x5603a3b5f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:23.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.522+0000 7f2726000640 1 -- 172.21.15.77:0/879956512 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603a3b64760 msgr2=0x5603a3b5f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:23.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.522+0000 7f2726000640 1 --2- 172.21.15.77:0/879956512 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603a3b64760 0x5603a3b5f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:23.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.522+0000 7f2726000640 1 -- 172.21.15.77:0/879956512 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5603a3b63e40 msgr2=0x5603a3b668a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:23.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.522+0000 7f2726000640 1 --2- 172.21.15.77:0/879956512 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5603a3b63e40 0x5603a3b668a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:23.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.522+0000 7f2726000640 1 -- 172.21.15.77:0/879956512 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603a3b5fe70 con 0x5603a3b6f020 2024-09-10T09:59:23.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.522+0000 7f2726000640 1 --2- 172.21.15.77:0/879956512 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5603a3b6f020 0x5603a3b5f8e0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f2718009f00 tx=0x7f271800f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85b91bd760c365db server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:23.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.522+0000 7f2711c00640 1 -- 172.21.15.77:0/879956512 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2718028070 con 0x5603a3b6f020 2024-09-10T09:59:23.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.522+0000 7f2711c00640 1 -- 172.21.15.77:0/879956512 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f271801f040 con 0x5603a3b6f020 2024-09-10T09:59:23.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.522+0000 7f2711c00640 1 -- 172.21.15.77:0/879956512 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f271801e040 con 0x5603a3b6f020 2024-09-10T09:59:23.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.523+0000 7f272aa8f740 1 -- 172.21.15.77:0/879956512 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5603a3b6f020 msgr2=0x5603a3b5f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:23.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.523+0000 7f272aa8f740 1 --2- 172.21.15.77:0/879956512 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5603a3b6f020 0x5603a3b5f8e0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f2718009f00 tx=0x7f271800f520 comp rx=0 tx=0).stop 2024-09-10T09:59:23.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.523+0000 7f272aa8f740 1 -- 172.21.15.77:0/879956512 shutdown_connections 2024-09-10T09:59:23.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.523+0000 7f272aa8f740 1 --2- 172.21.15.77:0/879956512 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5603a3b6f020 0x5603a3b5f8e0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:23.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.523+0000 7f272aa8f740 1 --2- 172.21.15.77:0/879956512 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603a3b64760 0x5603a3b5f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:23.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.523+0000 7f272aa8f740 1 --2- 172.21.15.77:0/879956512 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5603a3b63e40 0x5603a3b668a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:23.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.523+0000 7f272aa8f740 1 -- 172.21.15.77:0/879956512 >> 172.21.15.77:0/879956512 conn(0x5603a3b603d0 msgr2=0x5603a3b5ec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:23.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.523+0000 7f272aa8f740 1 -- 172.21.15.77:0/879956512 shutdown_connections 2024-09-10T09:59:23.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.523+0000 7f272aa8f740 1 -- 172.21.15.77:0/879956512 wait complete. 2024-09-10T09:59:23.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.523+0000 7f272aa8f740 1 -- start start 2024-09-10T09:59:23.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.524+0000 7f272aa8f740 1 Processor -- start 2024-09-10T09:59:23.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.524+0000 7f272aa8f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603a3b63e40 0x5603a3b72e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:23.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.524+0000 7f272aa8f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5603a3b64760 0x5603a3b73350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:23.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.524+0000 7f272aa8f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5603a3b6f020 0x5603a3b73890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:23.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.524+0000 7f272aa8f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f27201b2390 con 0x5603a3b64760 2024-09-10T09:59:23.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.524+0000 7f272aa8f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f27201b1cc0 con 0x5603a3b6f020 2024-09-10T09:59:23.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.524+0000 7f272aa8f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f27201aef00 con 0x5603a3b63e40 2024-09-10T09:59:23.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.524+0000 7f272aa8f740 10 client.0 _ll_register_callbacks cb 0x7f2714009ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:59:23.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.524+0000 7f2725600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603a3b63e40 0x5603a3b72e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:23.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.524+0000 7f2724c00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5603a3b64760 0x5603a3b73350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:23.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.524+0000 7f2725600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603a3b63e40 0x5603a3b72e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:56490/0 (socket says 172.21.15.77:56490) 2024-09-10T09:59:23.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.524+0000 7f2725600640 1 -- 172.21.15.77:0/2649448700 learned_addr learned my addr 172.21.15.77:0/2649448700 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:59:23.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.524+0000 7f2726000640 1 --2- 172.21.15.77:0/2649448700 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5603a3b6f020 0x5603a3b73890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:23.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.525+0000 7f2724c00640 1 -- 172.21.15.77:0/2649448700 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603a3b63e40 msgr2=0x5603a3b72e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:23.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.525+0000 7f2724c00640 1 --2- 172.21.15.77:0/2649448700 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603a3b63e40 0x5603a3b72e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:23.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.525+0000 7f2724c00640 1 -- 172.21.15.77:0/2649448700 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5603a3b6f020 msgr2=0x5603a3b73890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:23.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.525+0000 7f2724c00640 1 --2- 172.21.15.77:0/2649448700 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5603a3b6f020 0x5603a3b73890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:23.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.525+0000 7f2724c00640 1 -- 172.21.15.77:0/2649448700 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603a3be8ed0 con 0x5603a3b64760 2024-09-10T09:59:23.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.525+0000 7f2725600640 1 --2- 172.21.15.77:0/2649448700 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603a3b63e40 0x5603a3b72e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:59:23.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.525+0000 7f2724c00640 1 --2- 172.21.15.77:0/2649448700 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5603a3b64760 0x5603a3b73350 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x5603a3b61750 tx=0x7f270c03b720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e612aaa90ebd8dfc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:23.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.525+0000 7f2706a00640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-10T09:59:23.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.525+0000 7f2706a00640 1 -- 172.21.15.77:0/2649448700 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f270c049040 con 0x5603a3b64760 2024-09-10T09:59:23.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.525+0000 7f2706a00640 1 -- 172.21.15.77:0/2649448700 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f270c03b460 con 0x5603a3b64760 2024-09-10T09:59:23.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.525+0000 7f272aa8f740 1 -- 172.21.15.77:0/2649448700 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f271801f040 con 0x5603a3b64760 2024-09-10T09:59:23.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.525+0000 7f2706a00640 1 -- 172.21.15.77:0/2649448700 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f270c0bbe60 con 0x5603a3b64760 2024-09-10T09:59:23.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.526+0000 7f272aa8f740 10 client.16260 fetch_fsmap learned FSMap version 32 2024-09-10T09:59:23.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.526+0000 7f272aa8f740 1 -- 172.21.15.77:0/2649448700 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5603a3bea210 con 0x5603a3b64760 2024-09-10T09:59:23.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.526+0000 7f2706a00640 1 -- 172.21.15.77:0/2649448700 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f271801f040 con 0x5603a3b64760 2024-09-10T09:59:23.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.526+0000 7f2706a00640 1 -- 172.21.15.77:0/2649448700 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f270c0c4640 con 0x5603a3b64760 2024-09-10T09:59:23.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.526+0000 7f272aa8f740 10 client.16260 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T09:59:23.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.526+0000 7f272aa8f740 10 client.16260 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-10T09:59:23.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.526+0000 7f272aa8f740 20 client.16260 populate_metadata read hostname 'smithi077' 2024-09-10T09:59:23.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.526+0000 7f272aa8f740 1 --2- 172.21.15.77:0/2649448700 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x5603a3bea970 0x5603a3becd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:23.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.526+0000 7f272aa8f740 4 client.16260 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:59:23.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.526+0000 7f272aa8f740 1 -- 172.21.15.77:0/2649448700 --> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5603a3ad1e60 con 0x5603a3bea970 2024-09-10T09:59:23.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.527+0000 7f2725600640 1 --2- 172.21.15.77:0/2649448700 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x5603a3bea970 0x5603a3becd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:23.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.527+0000 7f2725600640 1 --2- 172.21.15.77:0/2649448700 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x5603a3bea970 0x5603a3becd90 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c4d4a0d7b3c81f2e server_cookie=c0491b08fdfa7fae in_seq=0 out_seq=0 2024-09-10T09:59:23.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.527+0000 7f2706a00640 10 client.16260 ms_handle_connect on v2:172.21.15.120:6836/3033280709 2024-09-10T09:59:23.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.527+0000 7f2706a00640 1 -- 172.21.15.77:0/2649448700 <== mds.1 v2:172.21.15.120:6836/3033280709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5603a3ad1e60 con 0x5603a3bea970 2024-09-10T09:59:23.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.527+0000 7f2706a00640 10 client.16260 handle_command_reply: tid=0 2024-09-10T09:59:23.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.527+0000 7f272aa8f740 1 client.16260 shutdown 2024-09-10T09:59:23.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.527+0000 7f272aa8f740 1 -- 172.21.15.77:0/2649448700 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5603a3b64760 msgr2=0x5603a3b73350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:23.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.527+0000 7f272aa8f740 1 --2- 172.21.15.77:0/2649448700 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5603a3b64760 0x5603a3b73350 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x5603a3b61750 tx=0x7f270c03b720 comp rx=0 tx=0).stop 2024-09-10T09:59:23.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.527+0000 7f272aa8f740 1 -- 172.21.15.77:0/2649448700 shutdown_connections 2024-09-10T09:59:23.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.527+0000 7f272aa8f740 1 --2- 172.21.15.77:0/2649448700 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x5603a3bea970 0x5603a3becd90 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:23.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.528+0000 7f272aa8f740 1 --2- 172.21.15.77:0/2649448700 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5603a3b6f020 0x5603a3b73890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:23.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.528+0000 7f272aa8f740 1 --2- 172.21.15.77:0/2649448700 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5603a3b64760 0x5603a3b73350 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:23.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.528+0000 7f272aa8f740 1 --2- 172.21.15.77:0/2649448700 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603a3b63e40 0x5603a3b72e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:23.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.528+0000 7f272aa8f740 1 -- 172.21.15.77:0/2649448700 >> 172.21.15.77:0/2649448700 conn(0x5603a3b603d0 msgr2=0x5603a3b5f4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:23.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.528+0000 7f2706a00640 0 client.16260 ms_handle_reset on v2:172.21.15.120:6836/3033280709 2024-09-10T09:59:23.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.528+0000 7f272aa8f740 1 -- 172.21.15.77:0/2649448700 shutdown_connections 2024-09-10T09:59:23.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.528+0000 7f272aa8f740 1 -- 172.21.15.77:0/2649448700 wait complete. 2024-09-10T09:59:23.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.528+0000 7f272aa8f740 20 client.16260 trim_cache size 0 max 16384 2024-09-10T09:59:23.534 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-10T09:59:23.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.528+0000 7f2726a00640 1 -- 172.21.15.77:0/508502173 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f27080a2b80 msgr2=0x7f27080a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:23.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.528+0000 7f2726a00640 1 --2- 172.21.15.77:0/508502173 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f27080a2b80 0x7f27080a5040 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x5603a3b61230 tx=0x7f2714005f90 comp rx=0 tx=0).stop 2024-09-10T09:59:23.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.528+0000 7f2726a00640 1 -- 172.21.15.77:0/508502173 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f27201047e0 msgr2=0x7f272019a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:23.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.528+0000 7f2726a00640 1 --2- 172.21.15.77:0/508502173 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f27201047e0 0x7f272019a9e0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f270c004770 tx=0x7f270c003c10 comp rx=0 tx=0).stop 2024-09-10T09:59:23.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.529+0000 7f2726a00640 1 -- 172.21.15.77:0/508502173 shutdown_connections 2024-09-10T09:59:23.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.529+0000 7f2726a00640 1 --2- 172.21.15.77:0/508502173 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f27080a2b80 0x7f27080a5040 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:23.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.529+0000 7f2726a00640 1 --2- 172.21.15.77:0/508502173 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2720108f60 0x7f272019ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:23.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.529+0000 7f2726a00640 1 --2- 172.21.15.77:0/508502173 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f27201047e0 0x7f272019a9e0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:23.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.529+0000 7f2726a00640 1 --2- 172.21.15.77:0/508502173 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2720103e30 0x7f272019a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:23.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.529+0000 7f2726a00640 1 -- 172.21.15.77:0/508502173 >> 172.21.15.77:0/508502173 conn(0x7f27200ff8d0 msgr2=0x7f2720100c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:23.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.529+0000 7f2726a00640 1 -- 172.21.15.77:0/508502173 shutdown_connections 2024-09-10T09:59:23.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:23.529+0000 7f2726a00640 1 -- 172.21.15.77:0/508502173 wait complete. 2024-09-10T09:59:24.426 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T09:59:24.794 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:59:25.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:59:24 smithi077 ceph-mon[30319]: pgmap v518: 129 pgs: 129 active+clean; 96 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 253 op/s 2024-09-10T09:59:25.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.367+0000 7f1af6c00640 1 -- 172.21.15.77:0/1712985465 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1af0102360 msgr2=0x7f1af010e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:25.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.367+0000 7f1af6c00640 1 --2- 172.21.15.77:0/1712985465 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1af0102360 0x7f1af010e890 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f1ae8009cd0 tx=0x7f1ae802a8e0 comp rx=0 tx=0).stop 2024-09-10T09:59:25.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.367+0000 7f1af6c00640 1 -- 172.21.15.77:0/1712985465 shutdown_connections 2024-09-10T09:59:25.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.367+0000 7f1af6c00640 1 --2- 172.21.15.77:0/1712985465 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1af0102360 0x7f1af010e890 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:25.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.367+0000 7f1af6c00640 1 --2- 172.21.15.77:0/1712985465 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1af01019c0 0x7f1af0101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:25.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.367+0000 7f1af6c00640 1 --2- 172.21.15.77:0/1712985465 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1af01079c0 0x7f1af0107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:25.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.367+0000 7f1af6c00640 1 -- 172.21.15.77:0/1712985465 >> 172.21.15.77:0/1712985465 conn(0x7f1af00fd740 msgr2=0x7f1af00ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:25.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.367+0000 7f1af6c00640 1 -- 172.21.15.77:0/1712985465 shutdown_connections 2024-09-10T09:59:25.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.367+0000 7f1af6c00640 1 -- 172.21.15.77:0/1712985465 wait complete. 2024-09-10T09:59:25.370 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.368+0000 7f1af6c00640 1 Processor -- start 2024-09-10T09:59:25.370 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.368+0000 7f1af6c00640 1 -- start start 2024-09-10T09:59:25.370 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.368+0000 7f1af6c00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1af01019c0 0x7f1af00744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:25.370 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.368+0000 7f1af6c00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1af0102360 0x7f1af0074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:25.370 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.368+0000 7f1af6c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1af01079c0 0x7f1af0074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:25.370 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.368+0000 7f1af6c00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1af0111e10 con 0x7f1af0102360 2024-09-10T09:59:25.370 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.368+0000 7f1af6c00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1af0111c90 con 0x7f1af01019c0 2024-09-10T09:59:25.370 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.368+0000 7f1af6c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1af0111f90 con 0x7f1af01079c0 2024-09-10T09:59:25.370 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.369+0000 7f1af5800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1af01019c0 0x7f1af00744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:25.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.369+0000 7f1af5800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1af01019c0 0x7f1af00744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:59938/0 (socket says 172.21.15.77:59938) 2024-09-10T09:59:25.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.369+0000 7f1af4e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1af0102360 0x7f1af0074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:25.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.369+0000 7f1af5800640 1 -- 172.21.15.77:0/1327701744 learned_addr learned my addr 172.21.15.77:0/1327701744 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:59:25.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.369+0000 7f1af6200640 1 --2- 172.21.15.77:0/1327701744 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1af01079c0 0x7f1af0074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:25.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.369+0000 7f1af5800640 1 -- 172.21.15.77:0/1327701744 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1af01079c0 msgr2=0x7f1af0074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:25.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.369+0000 7f1af5800640 1 --2- 172.21.15.77:0/1327701744 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1af01079c0 0x7f1af0074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:25.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.369+0000 7f1af5800640 1 -- 172.21.15.77:0/1327701744 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1af0102360 msgr2=0x7f1af0074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:25.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.369+0000 7f1af5800640 1 --2- 172.21.15.77:0/1327701744 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1af0102360 0x7f1af0074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:25.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.369+0000 7f1af5800640 1 -- 172.21.15.77:0/1327701744 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1af01a6730 con 0x7f1af01019c0 2024-09-10T09:59:25.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.369+0000 7f1af6200640 1 --2- 172.21.15.77:0/1327701744 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1af01079c0 0x7f1af0074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:59:25.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.369+0000 7f1af5800640 1 --2- 172.21.15.77:0/1327701744 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1af01019c0 0x7f1af00744f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1ae400e9f0 tx=0x7f1ae400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:25.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.370+0000 7f1aee000640 1 -- 172.21.15.77:0/1327701744 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae400cd80 con 0x7f1af01019c0 2024-09-10T09:59:25.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.370+0000 7f1af6c00640 1 -- 172.21.15.77:0/1327701744 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1af01a6a20 con 0x7f1af01019c0 2024-09-10T09:59:25.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.370+0000 7f1aee000640 1 -- 172.21.15.77:0/1327701744 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ae4010910 con 0x7f1af01019c0 2024-09-10T09:59:25.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.370+0000 7f1aee000640 1 -- 172.21.15.77:0/1327701744 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae40188f0 con 0x7f1af01019c0 2024-09-10T09:59:25.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.370+0000 7f1af6c00640 1 -- 172.21.15.77:0/1327701744 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1af01a6fb0 con 0x7f1af01019c0 2024-09-10T09:59:25.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.373+0000 7f1afac2c740 1 Processor -- start 2024-09-10T09:59:25.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.373+0000 7f1afac2c740 1 -- start start 2024-09-10T09:59:25.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.373+0000 7f1afac2c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562399b16e40 0x562399b17220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:25.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.373+0000 7f1aee000640 1 -- 172.21.15.77:0/1327701744 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1ae4010ac0 con 0x7f1af01019c0 2024-09-10T09:59:25.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.373+0000 7f1afac2c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562399b17760 0x562399b21ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:25.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.373+0000 7f1afac2c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562399b22020 0x562399b24410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:25.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.373+0000 7f1afac2c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x562399a7d3e0 con 0x562399b22020 2024-09-10T09:59:25.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.373+0000 7f1afac2c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x562399a24d80 con 0x562399b16e40 2024-09-10T09:59:25.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.373+0000 7f1afac2c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562399a793a0 con 0x562399b17760 2024-09-10T09:59:25.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.374+0000 7f1af6200640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562399b16e40 0x562399b17220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:25.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.374+0000 7f1af6200640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562399b16e40 0x562399b17220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:59948/0 (socket says 172.21.15.77:59948) 2024-09-10T09:59:25.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.374+0000 7f1af6200640 1 -- 172.21.15.77:0/1685251256 learned_addr learned my addr 172.21.15.77:0/1685251256 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:59:25.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.374+0000 7f1af4e00640 1 --2- 172.21.15.77:0/1685251256 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562399b22020 0x562399b24410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:25.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.374+0000 7f1af5800640 1 --2- 172.21.15.77:0/1685251256 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562399b17760 0x562399b21ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:25.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.374+0000 7f1af6200640 1 -- 172.21.15.77:0/1685251256 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562399b17760 msgr2=0x562399b21ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:25.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.374+0000 7f1af6200640 1 --2- 172.21.15.77:0/1685251256 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562399b17760 0x562399b21ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:25.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.374+0000 7f1af6200640 1 -- 172.21.15.77:0/1685251256 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562399b22020 msgr2=0x562399b24410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:25.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.374+0000 7f1af6200640 1 --2- 172.21.15.77:0/1685251256 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562399b22020 0x562399b24410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:25.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.374+0000 7f1af6200640 1 -- 172.21.15.77:0/1685251256 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562399a43ae0 con 0x562399b16e40 2024-09-10T09:59:25.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.374+0000 7f1af4e00640 1 --2- 172.21.15.77:0/1685251256 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562399b22020 0x562399b24410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:59:25.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.374+0000 7f1af5800640 1 --2- 172.21.15.77:0/1685251256 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562399b17760 0x562399b21ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:59:25.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.375+0000 7f1af6200640 1 --2- 172.21.15.77:0/1685251256 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562399b16e40 0x562399b17220 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1ae8009cd0 tx=0x7f1ae8009a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ddcfe765818e419 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:25.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.375+0000 7f1ae2a00640 1 -- 172.21.15.77:0/1685251256 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae80318c0 con 0x562399b16e40 2024-09-10T09:59:25.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.375+0000 7f1aee000640 1 --2- 172.21.15.77:0/1327701744 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f1ad80a2b80 0x7f1ad80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:25.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.375+0000 7f1ae2a00640 1 -- 172.21.15.77:0/1685251256 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ae8036040 con 0x562399b16e40 2024-09-10T09:59:25.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.375+0000 7f1ae2a00640 1 -- 172.21.15.77:0/1685251256 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae80045a0 con 0x562399b16e40 2024-09-10T09:59:25.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.375+0000 7f1aee000640 1 -- 172.21.15.77:0/1327701744 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f1ae40cf7f0 con 0x7f1af01019c0 2024-09-10T09:59:25.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.375+0000 7f1af5800640 1 --2- 172.21.15.77:0/1327701744 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f1ad80a2b80 0x7f1ad80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:25.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.376+0000 7f1afac2c740 1 -- 172.21.15.77:0/1685251256 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562399b16e40 msgr2=0x562399b17220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:25.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.376+0000 7f1afac2c740 1 --2- 172.21.15.77:0/1685251256 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562399b16e40 0x562399b17220 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1ae8009cd0 tx=0x7f1ae8009a20 comp rx=0 tx=0).stop 2024-09-10T09:59:25.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.376+0000 7f1afac2c740 1 -- 172.21.15.77:0/1685251256 shutdown_connections 2024-09-10T09:59:25.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.376+0000 7f1afac2c740 1 --2- 172.21.15.77:0/1685251256 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562399b22020 0x562399b24410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:25.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.376+0000 7f1afac2c740 1 --2- 172.21.15.77:0/1685251256 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562399b17760 0x562399b21ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:25.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.376+0000 7f1afac2c740 1 --2- 172.21.15.77:0/1685251256 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562399b16e40 0x562399b17220 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:25.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.376+0000 7f1afac2c740 1 -- 172.21.15.77:0/1685251256 >> 172.21.15.77:0/1685251256 conn(0x562399b133d0 msgr2=0x562399b137b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:25.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.376+0000 7f1af5800640 1 --2- 172.21.15.77:0/1327701744 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f1ad80a2b80 0x7f1ad80a5040 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f1ae40cd270 tx=0x7f1ae4098010 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:25.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.376+0000 7f1afac2c740 1 -- 172.21.15.77:0/1685251256 shutdown_connections 2024-09-10T09:59:25.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.376+0000 7f1afac2c740 1 -- 172.21.15.77:0/1685251256 wait complete. 2024-09-10T09:59:25.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.377+0000 7f1afac2c740 1 -- start start 2024-09-10T09:59:25.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.377+0000 7f1afac2c740 1 Processor -- start 2024-09-10T09:59:25.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.377+0000 7f1afac2c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562399b16e40 0x562399b1ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:25.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.377+0000 7f1afac2c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562399b17760 0x562399b1f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:25.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.377+0000 7f1afac2c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562399b22020 0x562399b1f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:25.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.377+0000 7f1afac2c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x562399b24bf0 con 0x562399b16e40 2024-09-10T09:59:25.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.377+0000 7f1afac2c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x562399a380f0 con 0x562399b17760 2024-09-10T09:59:25.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.377+0000 7f1afac2c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562399b24d70 con 0x562399b22020 2024-09-10T09:59:25.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.377+0000 7f1afac2c740 10 client.0 _ll_register_callbacks cb 0x562399a1fbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:59:25.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.378+0000 7f1af6200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562399b16e40 0x562399b1ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:25.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.378+0000 7f1af6200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562399b16e40 0x562399b1ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33082/0 (socket says 172.21.15.77:33082) 2024-09-10T09:59:25.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.378+0000 7f1af4e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562399b17760 0x562399b1f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:25.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.378+0000 7f1af6200640 1 -- 172.21.15.77:0/1628337605 learned_addr learned my addr 172.21.15.77:0/1628337605 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:59:25.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.378+0000 7f1af6200640 1 --2- 172.21.15.77:0/1628337605 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562399b22020 0x562399b1f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:25.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.378+0000 7f1af4e00640 1 -- 172.21.15.77:0/1628337605 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562399b22020 msgr2=0x562399b1f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:25.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.378+0000 7f1af4e00640 1 --2- 172.21.15.77:0/1628337605 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562399b22020 0x562399b1f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:25.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.378+0000 7f1af4e00640 1 -- 172.21.15.77:0/1628337605 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562399b16e40 msgr2=0x562399b1ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:25.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.378+0000 7f1af4e00640 1 --2- 172.21.15.77:0/1628337605 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562399b16e40 0x562399b1ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:25.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.378+0000 7f1af4e00640 1 -- 172.21.15.77:0/1628337605 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623997ee3c0 con 0x562399b17760 2024-09-10T09:59:25.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.378+0000 7f1af6200640 1 --2- 172.21.15.77:0/1628337605 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562399b16e40 0x562399b1ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:59:25.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.378+0000 7f1af4e00640 1 --2- 172.21.15.77:0/1628337605 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562399b17760 0x562399b1f470 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1adc0126d0 tx=0x7f1adc012ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9b57adf6c589e60 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:25.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.378+0000 7f1ad7400640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T09:59:25.384 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.379+0000 7f1ad7400640 1 -- 172.21.15.77:0/1628337605 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1adc004050 con 0x562399b17760 2024-09-10T09:59:25.384 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.379+0000 7f1afac2c740 1 -- 172.21.15.77:0/1628337605 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ae8036040 con 0x562399b17760 2024-09-10T09:59:25.384 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.379+0000 7f1ad7400640 1 -- 172.21.15.77:0/1628337605 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f1adc0041f0 con 0x562399b17760 2024-09-10T09:59:25.384 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.379+0000 7f1ad7400640 1 -- 172.21.15.77:0/1628337605 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1adc025050 con 0x562399b17760 2024-09-10T09:59:25.384 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.379+0000 7f1afac2c740 10 client.26354 fetch_fsmap learned FSMap version 32 2024-09-10T09:59:25.384 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.379+0000 7f1afac2c740 1 -- 172.21.15.77:0/1628337605 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562399b30cc0 con 0x562399b17760 2024-09-10T09:59:25.384 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.379+0000 7f1ad7400640 1 -- 172.21.15.77:0/1628337605 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ae8036040 con 0x562399b17760 2024-09-10T09:59:25.384 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.379+0000 7f1ad7400640 1 -- 172.21.15.77:0/1628337605 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f1adc003830 con 0x562399b17760 2024-09-10T09:59:25.384 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.379+0000 7f1afac2c740 10 client.26354 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T09:59:25.384 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.379+0000 7f1afac2c740 10 client.26354 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:59:25.384 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.380+0000 7f1afac2c740 20 client.26354 populate_metadata read hostname 'smithi077' 2024-09-10T09:59:25.384 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.380+0000 7f1afac2c740 1 --2- 172.21.15.77:0/1628337605 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x562399b321d0 0x562399b34620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:25.384 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.380+0000 7f1afac2c740 4 client.26354 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:59:25.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.380+0000 7f1afac2c740 1 -- 172.21.15.77:0/1628337605 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562399a8a950 con 0x562399b321d0 2024-09-10T09:59:25.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.380+0000 7f1af6200640 1 --2- 172.21.15.77:0/1628337605 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x562399b321d0 0x562399b34620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:25.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.380+0000 7f1ad7400640 10 client.26354 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T09:59:25.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.380+0000 7f1af6200640 1 --2- 172.21.15.77:0/1628337605 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x562399b321d0 0x562399b34620 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb5ea12f95d8f543 server_cookie=d017712e25b4a73c in_seq=0 out_seq=0 2024-09-10T09:59:25.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.381+0000 7f1ad7400640 1 -- 172.21.15.77:0/1628337605 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562399a8a950 con 0x562399b321d0 2024-09-10T09:59:25.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.381+0000 7f1ad7400640 10 client.26354 handle_command_reply: tid=0 2024-09-10T09:59:25.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.381+0000 7f1afac2c740 1 client.26354 shutdown 2024-09-10T09:59:25.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.382+0000 7f1afac2c740 1 -- 172.21.15.77:0/1628337605 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562399b17760 msgr2=0x562399b1f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:25.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.382+0000 7f1afac2c740 1 --2- 172.21.15.77:0/1628337605 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562399b17760 0x562399b1f470 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1adc0126d0 tx=0x7f1adc012ad0 comp rx=0 tx=0).stop 2024-09-10T09:59:25.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.382+0000 7f1afac2c740 1 -- 172.21.15.77:0/1628337605 shutdown_connections 2024-09-10T09:59:25.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.382+0000 7f1afac2c740 1 --2- 172.21.15.77:0/1628337605 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x562399b321d0 0x562399b34620 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:25.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.382+0000 7f1afac2c740 1 --2- 172.21.15.77:0/1628337605 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562399b22020 0x562399b1f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:25.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.382+0000 7f1afac2c740 1 --2- 172.21.15.77:0/1628337605 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562399b17760 0x562399b1f470 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:25.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.382+0000 7f1ad7400640 0 client.26354 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T09:59:25.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.382+0000 7f1afac2c740 1 --2- 172.21.15.77:0/1628337605 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562399b16e40 0x562399b1ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:25.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.382+0000 7f1afac2c740 1 -- 172.21.15.77:0/1628337605 >> 172.21.15.77:0/1628337605 conn(0x562399b133d0 msgr2=0x562399b190b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:25.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.382+0000 7f1afac2c740 1 -- 172.21.15.77:0/1628337605 shutdown_connections 2024-09-10T09:59:25.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.382+0000 7f1afac2c740 1 -- 172.21.15.77:0/1628337605 wait complete. 2024-09-10T09:59:25.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.382+0000 7f1afac2c740 20 client.26354 trim_cache size 0 max 16384 2024-09-10T09:59:25.386 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:59:24 smithi134 ceph-mon[35674]: pgmap v518: 129 pgs: 129 active+clean; 96 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 253 op/s 2024-09-10T09:59:25.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.395+0000 7f1afac2c740 1 Processor -- start 2024-09-10T09:59:25.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.396+0000 7f1afac2c740 1 -- start start 2024-09-10T09:59:25.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.396+0000 7f1afac2c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562399b16e40 0x562399b198a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:25.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.396+0000 7f1afac2c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562399b17760 0x562399b20ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:25.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.396+0000 7f1afac2c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562399b22020 0x562399b39f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:25.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.396+0000 7f1afac2c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1af01b0f70 con 0x562399b17760 2024-09-10T09:59:25.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.396+0000 7f1afac2c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1af01b2b70 con 0x562399b22020 2024-09-10T09:59:25.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.396+0000 7f1afac2c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1af01b1640 con 0x562399b16e40 2024-09-10T09:59:25.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.396+0000 7f1af4e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562399b17760 0x562399b20ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:25.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.396+0000 7f1af4e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562399b17760 0x562399b20ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33092/0 (socket says 172.21.15.77:33092) 2024-09-10T09:59:25.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.396+0000 7f1af4e00640 1 -- 172.21.15.77:0/717761868 learned_addr learned my addr 172.21.15.77:0/717761868 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:59:25.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.396+0000 7f1af6200640 1 --2- 172.21.15.77:0/717761868 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562399b16e40 0x562399b198a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:25.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.396+0000 7f1af6200640 1 --2- 172.21.15.77:0/717761868 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562399b22020 0x562399b39f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:25.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.397+0000 7f1af4e00640 1 -- 172.21.15.77:0/717761868 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562399b16e40 msgr2=0x562399b198a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:25.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.397+0000 7f1af4e00640 1 --2- 172.21.15.77:0/717761868 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562399b16e40 0x562399b198a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:25.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.397+0000 7f1af4e00640 1 -- 172.21.15.77:0/717761868 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562399b22020 msgr2=0x562399b39f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:25.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.397+0000 7f1af4e00640 1 --2- 172.21.15.77:0/717761868 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562399b22020 0x562399b39f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:25.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.397+0000 7f1af4e00640 1 -- 172.21.15.77:0/717761868 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562399b14060 con 0x562399b17760 2024-09-10T09:59:25.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.397+0000 7f1af4e00640 1 --2- 172.21.15.77:0/717761868 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562399b17760 0x562399b20ea0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f1adc012420 tx=0x7f1adc0097a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=911907b3402f74c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:25.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.397+0000 7f1ae2000640 1 -- 172.21.15.77:0/717761868 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1adc02d030 con 0x562399b17760 2024-09-10T09:59:25.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.397+0000 7f1ae2000640 1 -- 172.21.15.77:0/717761868 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1adc009df0 con 0x562399b17760 2024-09-10T09:59:25.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.397+0000 7f1ae2000640 1 -- 172.21.15.77:0/717761868 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1adc010630 con 0x562399b17760 2024-09-10T09:59:25.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.398+0000 7f1afac2c740 1 -- 172.21.15.77:0/717761868 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562399b17760 msgr2=0x562399b20ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:25.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.398+0000 7f1afac2c740 1 --2- 172.21.15.77:0/717761868 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562399b17760 0x562399b20ea0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f1adc012420 tx=0x7f1adc0097a0 comp rx=0 tx=0).stop 2024-09-10T09:59:25.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.398+0000 7f1afac2c740 1 -- 172.21.15.77:0/717761868 shutdown_connections 2024-09-10T09:59:25.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.398+0000 7f1afac2c740 1 --2- 172.21.15.77:0/717761868 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562399b22020 0x562399b39f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:25.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.398+0000 7f1afac2c740 1 --2- 172.21.15.77:0/717761868 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562399b17760 0x562399b20ea0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:25.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.398+0000 7f1afac2c740 1 --2- 172.21.15.77:0/717761868 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562399b16e40 0x562399b198a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:25.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.398+0000 7f1afac2c740 1 -- 172.21.15.77:0/717761868 >> 172.21.15.77:0/717761868 conn(0x562399b133d0 msgr2=0x562399b20790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:25.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.398+0000 7f1afac2c740 1 -- 172.21.15.77:0/717761868 shutdown_connections 2024-09-10T09:59:25.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.398+0000 7f1afac2c740 1 -- 172.21.15.77:0/717761868 wait complete. 2024-09-10T09:59:25.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.398+0000 7f1afac2c740 1 -- start start 2024-09-10T09:59:25.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.399+0000 7f1afac2c740 1 Processor -- start 2024-09-10T09:59:25.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.399+0000 7f1afac2c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562399b16e40 0x562399b2a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:25.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.399+0000 7f1afac2c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562399b17760 0x562399b25ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:25.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.399+0000 7f1afac2c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562399b22020 0x562399b26010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:25.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.399+0000 7f1afac2c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1af01b23e0 con 0x562399b16e40 2024-09-10T09:59:25.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.399+0000 7f1afac2c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1af01b1d10 con 0x562399b22020 2024-09-10T09:59:25.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.399+0000 7f1afac2c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1af01aef50 con 0x562399b17760 2024-09-10T09:59:25.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.399+0000 7f1afac2c740 10 client.0 _ll_register_callbacks cb 0x7f1adc009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:59:25.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.399+0000 7f1af6200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562399b16e40 0x562399b2a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:25.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.399+0000 7f1af4e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562399b17760 0x562399b25ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:25.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.399+0000 7f1af6200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562399b16e40 0x562399b2a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33104/0 (socket says 172.21.15.77:33104) 2024-09-10T09:59:25.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.399+0000 7f1af6200640 1 -- 172.21.15.77:0/1661816459 learned_addr learned my addr 172.21.15.77:0/1661816459 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:59:25.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.399+0000 7f1af6200640 1 --2- 172.21.15.77:0/1661816459 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562399b22020 0x562399b26010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:25.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.400+0000 7f1af4e00640 1 -- 172.21.15.77:0/1661816459 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562399b22020 msgr2=0x562399b26010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:25.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.400+0000 7f1af4e00640 1 --2- 172.21.15.77:0/1661816459 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562399b22020 0x562399b26010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:25.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.400+0000 7f1af4e00640 1 -- 172.21.15.77:0/1661816459 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562399b16e40 msgr2=0x562399b2a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:25.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.400+0000 7f1af4e00640 1 --2- 172.21.15.77:0/1661816459 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562399b16e40 0x562399b2a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:25.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.400+0000 7f1af4e00640 1 -- 172.21.15.77:0/1661816459 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562399b26840 con 0x562399b17760 2024-09-10T09:59:25.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.400+0000 7f1af6200640 1 --2- 172.21.15.77:0/1661816459 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562399b16e40 0x562399b2a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:59:25.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.400+0000 7f1af4e00640 1 --2- 172.21.15.77:0/1661816459 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562399b17760 0x562399b25ab0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1adc003c80 tx=0x7f1adc005400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b4e89dc15400ce7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:25.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.400+0000 7f1ad7400640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-10T09:59:25.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.400+0000 7f1ad7400640 1 -- 172.21.15.77:0/1661816459 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1adc035020 con 0x562399b17760 2024-09-10T09:59:25.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.401+0000 7f1ad7400640 1 -- 172.21.15.77:0/1661816459 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f1adc022070 con 0x562399b17760 2024-09-10T09:59:25.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.401+0000 7f1afac2c740 1 -- 172.21.15.77:0/1661816459 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1adc009df0 con 0x562399b17760 2024-09-10T09:59:25.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.401+0000 7f1ad7400640 1 -- 172.21.15.77:0/1661816459 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1adc040050 con 0x562399b17760 2024-09-10T09:59:25.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.401+0000 7f1afac2c740 10 client.26311 fetch_fsmap learned FSMap version 32 2024-09-10T09:59:25.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.401+0000 7f1afac2c740 1 -- 172.21.15.77:0/1661816459 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562399b9cf90 con 0x562399b17760 2024-09-10T09:59:25.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.401+0000 7f1ad7400640 1 -- 172.21.15.77:0/1661816459 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1adc009df0 con 0x562399b17760 2024-09-10T09:59:25.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.401+0000 7f1ad7400640 1 -- 172.21.15.77:0/1661816459 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f1adc00fa20 con 0x562399b17760 2024-09-10T09:59:25.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.401+0000 7f1afac2c740 10 client.26311 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T09:59:25.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.401+0000 7f1afac2c740 10 client.26311 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:59:25.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.401+0000 7f1afac2c740 20 client.26311 populate_metadata read hostname 'smithi077' 2024-09-10T09:59:25.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.401+0000 7f1afac2c740 1 --2- 172.21.15.77:0/1661816459 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x562399b9d6f0 0x562399b9fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:25.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.401+0000 7f1afac2c740 4 client.26311 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:59:25.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.401+0000 7f1afac2c740 1 -- 172.21.15.77:0/1661816459 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562399a84e60 con 0x562399b9d6f0 2024-09-10T09:59:25.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.402+0000 7f1af6200640 1 --2- 172.21.15.77:0/1661816459 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x562399b9d6f0 0x562399b9fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:25.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.402+0000 7f1af6200640 1 --2- 172.21.15.77:0/1661816459 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x562399b9d6f0 0x562399b9fb10 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8045d5cb5098fccc server_cookie=8c9e67a234d4369c in_seq=0 out_seq=0 2024-09-10T09:59:25.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.402+0000 7f1ad7400640 10 client.26311 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T09:59:25.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.403+0000 7f1ad7400640 1 -- 172.21.15.77:0/1661816459 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562399a84e60 con 0x562399b9d6f0 2024-09-10T09:59:25.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.403+0000 7f1ad7400640 10 client.26311 handle_command_reply: tid=0 2024-09-10T09:59:25.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.403+0000 7f1afac2c740 1 client.26311 shutdown 2024-09-10T09:59:25.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.403+0000 7f1afac2c740 1 -- 172.21.15.77:0/1661816459 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562399b17760 msgr2=0x562399b25ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:25.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.403+0000 7f1afac2c740 1 --2- 172.21.15.77:0/1661816459 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562399b17760 0x562399b25ab0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1adc003c80 tx=0x7f1adc005400 comp rx=0 tx=0).stop 2024-09-10T09:59:25.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.403+0000 7f1afac2c740 1 -- 172.21.15.77:0/1661816459 shutdown_connections 2024-09-10T09:59:25.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.403+0000 7f1afac2c740 1 --2- 172.21.15.77:0/1661816459 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x562399b9d6f0 0x562399b9fb10 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:25.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.403+0000 7f1afac2c740 1 --2- 172.21.15.77:0/1661816459 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562399b22020 0x562399b26010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:25.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.403+0000 7f1afac2c740 1 --2- 172.21.15.77:0/1661816459 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562399b17760 0x562399b25ab0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:25.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.403+0000 7f1afac2c740 1 --2- 172.21.15.77:0/1661816459 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562399b16e40 0x562399b2a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:25.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.403+0000 7f1ad7400640 0 client.26311 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T09:59:25.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.403+0000 7f1afac2c740 1 -- 172.21.15.77:0/1661816459 >> 172.21.15.77:0/1661816459 conn(0x562399b133d0 msgr2=0x562399b1f6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:25.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.403+0000 7f1afac2c740 1 -- 172.21.15.77:0/1661816459 shutdown_connections 2024-09-10T09:59:25.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.403+0000 7f1afac2c740 1 -- 172.21.15.77:0/1661816459 wait complete. 2024-09-10T09:59:25.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.403+0000 7f1afac2c740 20 client.26311 trim_cache size 0 max 16384 2024-09-10T09:59:25.407 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-10T09:59:25.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.404+0000 7f1af6c00640 1 -- 172.21.15.77:0/1327701744 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f1ad80a2b80 msgr2=0x7f1ad80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:25.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.404+0000 7f1af6c00640 1 --2- 172.21.15.77:0/1327701744 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f1ad80a2b80 0x7f1ad80a5040 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f1ae40cd270 tx=0x7f1ae4098010 comp rx=0 tx=0).stop 2024-09-10T09:59:25.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.404+0000 7f1af6c00640 1 -- 172.21.15.77:0/1327701744 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1af01019c0 msgr2=0x7f1af00744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:25.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.404+0000 7f1af6c00640 1 --2- 172.21.15.77:0/1327701744 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1af01019c0 0x7f1af00744f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1ae400e9f0 tx=0x7f1ae400eeb0 comp rx=0 tx=0).stop 2024-09-10T09:59:25.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.404+0000 7f1af6c00640 1 -- 172.21.15.77:0/1327701744 shutdown_connections 2024-09-10T09:59:25.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.404+0000 7f1af6c00640 1 --2- 172.21.15.77:0/1327701744 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f1ad80a2b80 0x7f1ad80a5040 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:25.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.404+0000 7f1af6c00640 1 --2- 172.21.15.77:0/1327701744 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1af01079c0 0x7f1af0074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:25.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.404+0000 7f1af6c00640 1 --2- 172.21.15.77:0/1327701744 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1af0102360 0x7f1af0074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:25.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.404+0000 7f1af6c00640 1 --2- 172.21.15.77:0/1327701744 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1af01019c0 0x7f1af00744f0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:25.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.404+0000 7f1af6c00640 1 -- 172.21.15.77:0/1327701744 >> 172.21.15.77:0/1327701744 conn(0x7f1af00fd740 msgr2=0x7f1af00fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:25.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.404+0000 7f1af6c00640 1 -- 172.21.15.77:0/1327701744 shutdown_connections 2024-09-10T09:59:25.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:25.404+0000 7f1af6c00640 1 -- 172.21.15.77:0/1327701744 wait complete. 2024-09-10T09:59:25.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:59:24 smithi120 ceph-mon[34770]: pgmap v518: 129 pgs: 129 active+clean; 96 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 253 op/s 2024-09-10T09:59:27.166 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T09:59:27.167 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T09:59:27.215 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:59:26 smithi077 ceph-mon[30319]: pgmap v519: 129 pgs: 129 active+clean; 96 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 599 KiB/s wr, 112 op/s 2024-09-10T09:59:27.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:59:26 smithi134 ceph-mon[35674]: pgmap v519: 129 pgs: 129 active+clean; 96 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 599 KiB/s wr, 112 op/s 2024-09-10T09:59:27.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:59:26 smithi120 ceph-mon[34770]: pgmap v519: 129 pgs: 129 active+clean; 96 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 599 KiB/s wr, 112 op/s 2024-09-10T09:59:27.533 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:59:28.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.163+0000 7ff61cc00640 1 -- 172.21.15.77:0/2277314409 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff6181019c0 msgr2=0x7ff618101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:28.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.163+0000 7ff61cc00640 1 --2- 172.21.15.77:0/2277314409 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff6181019c0 0x7ff618101e20 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7ff604009930 tx=0x7ff60402a440 comp rx=0 tx=0).stop 2024-09-10T09:59:28.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.163+0000 7ff61cc00640 1 -- 172.21.15.77:0/2277314409 shutdown_connections 2024-09-10T09:59:28.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.163+0000 7ff61cc00640 1 --2- 172.21.15.77:0/2277314409 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff618102360 0x7ff61810e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:28.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.163+0000 7ff61cc00640 1 --2- 172.21.15.77:0/2277314409 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff6181019c0 0x7ff618101e20 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:28.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.163+0000 7ff61cc00640 1 --2- 172.21.15.77:0/2277314409 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6181079c0 0x7ff618107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:28.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.163+0000 7ff61cc00640 1 -- 172.21.15.77:0/2277314409 >> 172.21.15.77:0/2277314409 conn(0x7ff6180fd740 msgr2=0x7ff6180ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:28.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.163+0000 7ff61cc00640 1 -- 172.21.15.77:0/2277314409 shutdown_connections 2024-09-10T09:59:28.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.163+0000 7ff61cc00640 1 -- 172.21.15.77:0/2277314409 wait complete. 2024-09-10T09:59:28.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.164+0000 7ff61cc00640 1 Processor -- start 2024-09-10T09:59:28.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.164+0000 7ff61cc00640 1 -- start start 2024-09-10T09:59:28.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.165+0000 7ff61cc00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff6181019c0 0x7ff61819a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:28.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.165+0000 7ff61cc00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff618102360 0x7ff61819aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:28.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.165+0000 7ff61cc00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6181079c0 0x7ff61819ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:28.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.165+0000 7ff61cc00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff618111dc0 con 0x7ff618102360 2024-09-10T09:59:28.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.165+0000 7ff61cc00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff618111c40 con 0x7ff6181019c0 2024-09-10T09:59:28.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.165+0000 7ff61cc00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff618111f40 con 0x7ff6181079c0 2024-09-10T09:59:28.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.165+0000 7ff616a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff618102360 0x7ff61819aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:28.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.165+0000 7ff616a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff618102360 0x7ff61819aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33126/0 (socket says 172.21.15.77:33126) 2024-09-10T09:59:28.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.165+0000 7ff616a00640 1 -- 172.21.15.77:0/3189057692 learned_addr learned my addr 172.21.15.77:0/3189057692 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:59:28.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.165+0000 7ff617e00640 1 --2- 172.21.15.77:0/3189057692 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6181079c0 0x7ff61819ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:28.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.165+0000 7ff617400640 1 --2- 172.21.15.77:0/3189057692 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff6181019c0 0x7ff61819a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:28.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.166+0000 7ff616a00640 1 -- 172.21.15.77:0/3189057692 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6181079c0 msgr2=0x7ff61819ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:28.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.166+0000 7ff616a00640 1 --2- 172.21.15.77:0/3189057692 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6181079c0 0x7ff61819ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:28.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.166+0000 7ff616a00640 1 -- 172.21.15.77:0/3189057692 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff6181019c0 msgr2=0x7ff61819a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:28.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.166+0000 7ff616a00640 1 --2- 172.21.15.77:0/3189057692 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff6181019c0 0x7ff61819a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:28.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.166+0000 7ff616a00640 1 -- 172.21.15.77:0/3189057692 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff61819f4e0 con 0x7ff618102360 2024-09-10T09:59:28.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.166+0000 7ff617e00640 1 --2- 172.21.15.77:0/3189057692 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6181079c0 0x7ff61819ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:59:28.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.166+0000 7ff616a00640 1 --2- 172.21.15.77:0/3189057692 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff618102360 0x7ff61819aa70 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7ff604009230 tx=0x7ff604004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:28.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.166+0000 7ff60fe00640 1 -- 172.21.15.77:0/3189057692 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff60403f030 con 0x7ff618102360 2024-09-10T09:59:28.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.166+0000 7ff60fe00640 1 -- 172.21.15.77:0/3189057692 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff604003ad0 con 0x7ff618102360 2024-09-10T09:59:28.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.166+0000 7ff60fe00640 1 -- 172.21.15.77:0/3189057692 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff604035070 con 0x7ff618102360 2024-09-10T09:59:28.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.166+0000 7ff61cc00640 1 -- 172.21.15.77:0/3189057692 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff61819f770 con 0x7ff618102360 2024-09-10T09:59:28.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.167+0000 7ff61cc00640 1 -- 172.21.15.77:0/3189057692 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6181a70b0 con 0x7ff618102360 2024-09-10T09:59:28.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.169+0000 7ff620e05740 1 Processor -- start 2024-09-10T09:59:28.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.169+0000 7ff60fe00640 1 -- 172.21.15.77:0/3189057692 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff604042030 con 0x7ff618102360 2024-09-10T09:59:28.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.170+0000 7ff620e05740 1 -- start start 2024-09-10T09:59:28.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.170+0000 7ff620e05740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a8013fcb0 0x557a80140090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:28.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.170+0000 7ff620e05740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a801405d0 0x557a8014a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:28.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.170+0000 7ff620e05740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a8014ae90 0x557a8014d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:28.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.170+0000 7ff620e05740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557a800a6290 con 0x557a8013fcb0 2024-09-10T09:59:28.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.170+0000 7ff620e05740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x557a8004db40 con 0x557a8014ae90 2024-09-10T09:59:28.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.170+0000 7ff620e05740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557a800a2250 con 0x557a801405d0 2024-09-10T09:59:28.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.170+0000 7ff617e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a8013fcb0 0x557a80140090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:28.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.170+0000 7ff617400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a801405d0 0x557a8014a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:28.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.170+0000 7ff617e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a8013fcb0 0x557a80140090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33132/0 (socket says 172.21.15.77:33132) 2024-09-10T09:59:28.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.170+0000 7ff617e00640 1 -- 172.21.15.77:0/2967022132 learned_addr learned my addr 172.21.15.77:0/2967022132 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:59:28.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.170+0000 7ff616a00640 1 --2- 172.21.15.77:0/2967022132 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a8014ae90 0x557a8014d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:28.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.171+0000 7ff617e00640 1 -- 172.21.15.77:0/2967022132 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a801405d0 msgr2=0x557a8014a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:28.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.171+0000 7ff617e00640 1 --2- 172.21.15.77:0/2967022132 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a801405d0 0x557a8014a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:28.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.171+0000 7ff617e00640 1 -- 172.21.15.77:0/2967022132 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a8014ae90 msgr2=0x557a8014d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:28.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.171+0000 7ff617e00640 1 --2- 172.21.15.77:0/2967022132 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a8014ae90 0x557a8014d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:28.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.171+0000 7ff617e00640 1 -- 172.21.15.77:0/2967022132 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a7fe176a0 con 0x557a8013fcb0 2024-09-10T09:59:28.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.171+0000 7ff617400640 1 --2- 172.21.15.77:0/2967022132 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a801405d0 0x557a8014a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:59:28.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.171+0000 7ff60fe00640 1 --2- 172.21.15.77:0/3189057692 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ff5fc0a2b80 0x7ff5fc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:28.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.171+0000 7ff617e00640 1 --2- 172.21.15.77:0/2967022132 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a8013fcb0 0x557a80140090 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ff610010750 tx=0x7ff610010c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6cb589825225ab server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:28.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.171+0000 7ff60fe00640 1 -- 172.21.15.77:0/3189057692 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7ff6040ef8d0 con 0x7ff618102360 2024-09-10T09:59:28.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.171+0000 7ff603e00640 1 -- 172.21.15.77:0/2967022132 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff61000ead0 con 0x557a8013fcb0 2024-09-10T09:59:28.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.172+0000 7ff603e00640 1 -- 172.21.15.77:0/2967022132 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff610010e60 con 0x557a8013fcb0 2024-09-10T09:59:28.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.172+0000 7ff603e00640 1 -- 172.21.15.77:0/2967022132 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff610011640 con 0x557a8013fcb0 2024-09-10T09:59:28.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.172+0000 7ff617400640 1 --2- 172.21.15.77:0/3189057692 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ff5fc0a2b80 0x7ff5fc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:28.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.172+0000 7ff620e05740 1 -- 172.21.15.77:0/2967022132 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a8013fcb0 msgr2=0x557a80140090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:28.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.172+0000 7ff617400640 1 --2- 172.21.15.77:0/3189057692 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ff5fc0a2b80 0x7ff5fc0a5040 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x557a8013d0a0 tx=0x7ff60800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:28.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.172+0000 7ff620e05740 1 --2- 172.21.15.77:0/2967022132 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a8013fcb0 0x557a80140090 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ff610010750 tx=0x7ff610010c10 comp rx=0 tx=0).stop 2024-09-10T09:59:28.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.172+0000 7ff620e05740 1 -- 172.21.15.77:0/2967022132 shutdown_connections 2024-09-10T09:59:28.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.172+0000 7ff620e05740 1 --2- 172.21.15.77:0/2967022132 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a8014ae90 0x557a8014d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:28.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.172+0000 7ff620e05740 1 --2- 172.21.15.77:0/2967022132 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a801405d0 0x557a8014a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:28.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.172+0000 7ff620e05740 1 --2- 172.21.15.77:0/2967022132 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a8013fcb0 0x557a80140090 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:28.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.172+0000 7ff620e05740 1 -- 172.21.15.77:0/2967022132 >> 172.21.15.77:0/2967022132 conn(0x557a8013c240 msgr2=0x557a8013c620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:28.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.173+0000 7ff620e05740 1 -- 172.21.15.77:0/2967022132 shutdown_connections 2024-09-10T09:59:28.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.173+0000 7ff620e05740 1 -- 172.21.15.77:0/2967022132 wait complete. 2024-09-10T09:59:28.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.173+0000 7ff620e05740 1 -- start start 2024-09-10T09:59:28.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.173+0000 7ff620e05740 1 Processor -- start 2024-09-10T09:59:28.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.174+0000 7ff620e05740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a8013fcb0 0x557a8014a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:28.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.174+0000 7ff620e05740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a801405d0 0x557a80147980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:28.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.174+0000 7ff620e05740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a8014ae90 0x557a80147ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:28.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.174+0000 7ff620e05740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557a8014db00 con 0x557a801405d0 2024-09-10T09:59:28.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.174+0000 7ff620e05740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x557a80060fa0 con 0x557a8014ae90 2024-09-10T09:59:28.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.174+0000 7ff620e05740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557a8014dc80 con 0x557a8013fcb0 2024-09-10T09:59:28.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.174+0000 7ff620e05740 10 client.0 _ll_register_callbacks cb 0x557a7fdb4ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:59:28.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.174+0000 7ff616a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a801405d0 0x557a80147980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:28.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.174+0000 7ff616a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a801405d0 0x557a80147980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33148/0 (socket says 172.21.15.77:33148) 2024-09-10T09:59:28.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.174+0000 7ff616a00640 1 -- 172.21.15.77:0/2023768286 learned_addr learned my addr 172.21.15.77:0/2023768286 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:59:28.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.174+0000 7ff617400640 1 --2- 172.21.15.77:0/2023768286 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a8013fcb0 0x557a8014a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:28.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.174+0000 7ff617e00640 1 --2- 172.21.15.77:0/2023768286 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a8014ae90 0x557a80147ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:28.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.174+0000 7ff616a00640 1 -- 172.21.15.77:0/2023768286 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a8013fcb0 msgr2=0x557a8014a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:28.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.174+0000 7ff616a00640 1 --2- 172.21.15.77:0/2023768286 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a8013fcb0 0x557a8014a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:28.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.174+0000 7ff616a00640 1 -- 172.21.15.77:0/2023768286 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a8014ae90 msgr2=0x557a80147ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:28.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.174+0000 7ff616a00640 1 --2- 172.21.15.77:0/2023768286 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a8014ae90 0x557a80147ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:28.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.174+0000 7ff616a00640 1 -- 172.21.15.77:0/2023768286 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a801486c0 con 0x557a801405d0 2024-09-10T09:59:28.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.174+0000 7ff617e00640 1 --2- 172.21.15.77:0/2023768286 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a8014ae90 0x557a80147ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:59:28.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.175+0000 7ff616a00640 1 --2- 172.21.15.77:0/2023768286 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a801405d0 0x557a80147980 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7ff6040bbd20 tx=0x7ff604009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55697964299ca11d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:28.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.175+0000 7ff600c00640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-10T09:59:28.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.175+0000 7ff600c00640 1 -- 172.21.15.77:0/2023768286 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff604049070 con 0x557a801405d0 2024-09-10T09:59:28.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.175+0000 7ff600c00640 1 -- 172.21.15.77:0/2023768286 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7ff6040b8400 con 0x557a801405d0 2024-09-10T09:59:28.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.175+0000 7ff620e05740 1 -- 172.21.15.77:0/2023768286 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff610010e60 con 0x557a801405d0 2024-09-10T09:59:28.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.175+0000 7ff600c00640 1 -- 172.21.15.77:0/2023768286 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff6040bb8d0 con 0x557a801405d0 2024-09-10T09:59:28.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.175+0000 7ff620e05740 10 client.16290 fetch_fsmap learned FSMap version 32 2024-09-10T09:59:28.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.175+0000 7ff620e05740 1 -- 172.21.15.77:0/2023768286 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x557a80159920 con 0x557a801405d0 2024-09-10T09:59:28.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.176+0000 7ff600c00640 1 -- 172.21.15.77:0/2023768286 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff610010e60 con 0x557a801405d0 2024-09-10T09:59:28.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.176+0000 7ff600c00640 1 -- 172.21.15.77:0/2023768286 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7ff6040c44c0 con 0x557a801405d0 2024-09-10T09:59:28.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.176+0000 7ff620e05740 10 client.16290 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T09:59:28.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.176+0000 7ff620e05740 10 client.16290 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:59:28.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.176+0000 7ff620e05740 20 client.16290 populate_metadata read hostname 'smithi077' 2024-09-10T09:59:28.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.176+0000 7ff620e05740 1 --2- 172.21.15.77:0/2023768286 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x557a8015ae60 0x557a8015d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:28.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.176+0000 7ff620e05740 4 client.16290 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:59:28.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.176+0000 7ff620e05740 1 -- 172.21.15.77:0/2023768286 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557a800b3800 con 0x557a8015ae60 2024-09-10T09:59:28.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.176+0000 7ff617400640 1 --2- 172.21.15.77:0/2023768286 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x557a8015ae60 0x557a8015d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:28.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.177+0000 7ff617400640 1 --2- 172.21.15.77:0/2023768286 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x557a8015ae60 0x557a8015d280 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d48a7252c888ef7f server_cookie=7178235e46d55174 in_seq=0 out_seq=0 2024-09-10T09:59:28.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.177+0000 7ff600c00640 10 client.16290 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T09:59:28.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.178+0000 7ff600c00640 1 -- 172.21.15.77:0/2023768286 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557a800b3800 con 0x557a8015ae60 2024-09-10T09:59:28.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.178+0000 7ff600c00640 10 client.16290 handle_command_reply: tid=0 2024-09-10T09:59:28.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.178+0000 7ff620e05740 1 client.16290 shutdown 2024-09-10T09:59:28.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.178+0000 7ff620e05740 1 -- 172.21.15.77:0/2023768286 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a801405d0 msgr2=0x557a80147980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:28.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.178+0000 7ff620e05740 1 --2- 172.21.15.77:0/2023768286 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a801405d0 0x557a80147980 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7ff6040bbd20 tx=0x7ff604009620 comp rx=0 tx=0).stop 2024-09-10T09:59:28.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.178+0000 7ff620e05740 1 -- 172.21.15.77:0/2023768286 shutdown_connections 2024-09-10T09:59:28.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.178+0000 7ff620e05740 1 --2- 172.21.15.77:0/2023768286 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x557a8015ae60 0x557a8015d280 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:28.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.178+0000 7ff620e05740 1 --2- 172.21.15.77:0/2023768286 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a8014ae90 0x557a80147ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:28.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.178+0000 7ff620e05740 1 --2- 172.21.15.77:0/2023768286 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a801405d0 0x557a80147980 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:28.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.178+0000 7ff600c00640 0 client.16290 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T09:59:28.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.178+0000 7ff620e05740 1 --2- 172.21.15.77:0/2023768286 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a8013fcb0 0x557a8014a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:28.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.178+0000 7ff620e05740 1 -- 172.21.15.77:0/2023768286 >> 172.21.15.77:0/2023768286 conn(0x557a8013c240 msgr2=0x557a80141f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:28.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.178+0000 7ff620e05740 1 -- 172.21.15.77:0/2023768286 shutdown_connections 2024-09-10T09:59:28.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.178+0000 7ff620e05740 1 -- 172.21.15.77:0/2023768286 wait complete. 2024-09-10T09:59:28.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.179+0000 7ff620e05740 20 client.16290 trim_cache size 0 max 16384 2024-09-10T09:59:28.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.196+0000 7ff620e05740 1 Processor -- start 2024-09-10T09:59:28.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.196+0000 7ff620e05740 1 -- start start 2024-09-10T09:59:28.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.196+0000 7ff620e05740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a8013fcb0 0x557a80143000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:28.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.196+0000 7ff620e05740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a801405d0 0x557a80162b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:28.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.196+0000 7ff620e05740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a8014ae90 0x557a80163090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:28.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.196+0000 7ff620e05740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff6181b0e60 con 0x557a8013fcb0 2024-09-10T09:59:28.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.196+0000 7ff620e05740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff6181b2a60 con 0x557a8014ae90 2024-09-10T09:59:28.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.196+0000 7ff620e05740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff6181b1530 con 0x557a801405d0 2024-09-10T09:59:28.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.197+0000 7ff617400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a8013fcb0 0x557a80143000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:28.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.197+0000 7ff617e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a8014ae90 0x557a80163090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:28.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.197+0000 7ff617400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a8013fcb0 0x557a80143000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33158/0 (socket says 172.21.15.77:33158) 2024-09-10T09:59:28.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.197+0000 7ff617400640 1 -- 172.21.15.77:0/2604136699 learned_addr learned my addr 172.21.15.77:0/2604136699 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:59:28.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.197+0000 7ff616a00640 1 --2- 172.21.15.77:0/2604136699 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a801405d0 0x557a80162b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:28.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.197+0000 7ff617400640 1 -- 172.21.15.77:0/2604136699 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a801405d0 msgr2=0x557a80162b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:28.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.197+0000 7ff617400640 1 --2- 172.21.15.77:0/2604136699 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a801405d0 0x557a80162b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:28.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.197+0000 7ff617400640 1 -- 172.21.15.77:0/2604136699 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a8014ae90 msgr2=0x557a80163090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:28.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.197+0000 7ff617400640 1 --2- 172.21.15.77:0/2604136699 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a8014ae90 0x557a80163090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:28.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.197+0000 7ff617400640 1 -- 172.21.15.77:0/2604136699 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a80145ed0 con 0x557a8013fcb0 2024-09-10T09:59:28.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.197+0000 7ff617e00640 1 --2- 172.21.15.77:0/2604136699 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a8014ae90 0x557a80163090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:59:28.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.197+0000 7ff617400640 1 --2- 172.21.15.77:0/2604136699 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a8013fcb0 0x557a80143000 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x557a8015daf0 tx=0x7ff608019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae21ef83b3624651 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:28.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.198+0000 7ff603400640 1 -- 172.21.15.77:0/2604136699 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff60801eb30 con 0x557a8013fcb0 2024-09-10T09:59:28.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.198+0000 7ff603400640 1 -- 172.21.15.77:0/2604136699 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff608005500 con 0x557a8013fcb0 2024-09-10T09:59:28.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.198+0000 7ff603400640 1 -- 172.21.15.77:0/2604136699 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff608004e40 con 0x557a8013fcb0 2024-09-10T09:59:28.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.198+0000 7ff620e05740 1 -- 172.21.15.77:0/2604136699 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a8013fcb0 msgr2=0x557a80143000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:28.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.198+0000 7ff620e05740 1 --2- 172.21.15.77:0/2604136699 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a8013fcb0 0x557a80143000 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x557a8015daf0 tx=0x7ff608019000 comp rx=0 tx=0).stop 2024-09-10T09:59:28.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.198+0000 7ff620e05740 1 -- 172.21.15.77:0/2604136699 shutdown_connections 2024-09-10T09:59:28.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.198+0000 7ff620e05740 1 --2- 172.21.15.77:0/2604136699 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a8014ae90 0x557a80163090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:28.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.198+0000 7ff620e05740 1 --2- 172.21.15.77:0/2604136699 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a801405d0 0x557a80162b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:28.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.198+0000 7ff620e05740 1 --2- 172.21.15.77:0/2604136699 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a8013fcb0 0x557a80143000 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:28.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.198+0000 7ff620e05740 1 -- 172.21.15.77:0/2604136699 >> 172.21.15.77:0/2604136699 conn(0x557a8013c240 msgr2=0x557a801428f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:28.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.199+0000 7ff620e05740 1 -- 172.21.15.77:0/2604136699 shutdown_connections 2024-09-10T09:59:28.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.199+0000 7ff620e05740 1 -- 172.21.15.77:0/2604136699 wait complete. 2024-09-10T09:59:28.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.199+0000 7ff620e05740 1 -- start start 2024-09-10T09:59:28.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.199+0000 7ff620e05740 1 Processor -- start 2024-09-10T09:59:28.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.200+0000 7ff620e05740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a8013fcb0 0x557a80142470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:28.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.200+0000 7ff620e05740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a801405d0 0x557a801429b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:28.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.200+0000 7ff620e05740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a8014ae90 0x557a80142ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:28.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.200+0000 7ff620e05740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff6181b22d0 con 0x557a8013fcb0 2024-09-10T09:59:28.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.200+0000 7ff620e05740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff6181b1c00 con 0x557a8014ae90 2024-09-10T09:59:28.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.200+0000 7ff620e05740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff6181aee40 con 0x557a801405d0 2024-09-10T09:59:28.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.200+0000 7ff620e05740 10 client.0 _ll_register_callbacks cb 0x7ff604009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:59:28.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.200+0000 7ff617400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a8013fcb0 0x557a80142470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:28.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.200+0000 7ff617400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a8013fcb0 0x557a80142470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33168/0 (socket says 172.21.15.77:33168) 2024-09-10T09:59:28.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.200+0000 7ff617400640 1 -- 172.21.15.77:0/2477606496 learned_addr learned my addr 172.21.15.77:0/2477606496 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:59:28.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.200+0000 7ff617e00640 1 --2- 172.21.15.77:0/2477606496 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a8014ae90 0x557a80142ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:28.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.200+0000 7ff616a00640 1 --2- 172.21.15.77:0/2477606496 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a801405d0 0x557a801429b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:28.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.200+0000 7ff617400640 1 -- 172.21.15.77:0/2477606496 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a801405d0 msgr2=0x557a801429b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:28.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.200+0000 7ff617400640 1 --2- 172.21.15.77:0/2477606496 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a801405d0 0x557a801429b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:28.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.200+0000 7ff617400640 1 -- 172.21.15.77:0/2477606496 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a8014ae90 msgr2=0x557a80142ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:28.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.200+0000 7ff617400640 1 --2- 172.21.15.77:0/2477606496 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a8014ae90 0x557a80142ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:28.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.200+0000 7ff617400640 1 -- 172.21.15.77:0/2477606496 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a801513a0 con 0x557a8013fcb0 2024-09-10T09:59:28.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.201+0000 7ff617400640 1 --2- 172.21.15.77:0/2477606496 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a8013fcb0 0x557a80142470 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7ff608003e50 tx=0x7ff608003e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85ca4476ad82c288 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:28.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.201+0000 7ff600c00640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-10T09:59:28.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.201+0000 7ff600c00640 1 -- 172.21.15.77:0/2477606496 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6080048c0 con 0x557a8013fcb0 2024-09-10T09:59:28.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.201+0000 7ff600c00640 1 -- 172.21.15.77:0/2477606496 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7ff60801eb30 con 0x557a8013fcb0 2024-09-10T09:59:28.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.201+0000 7ff620e05740 1 -- 172.21.15.77:0/2477606496 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff608005500 con 0x557a8013fcb0 2024-09-10T09:59:28.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.201+0000 7ff600c00640 1 -- 172.21.15.77:0/2477606496 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff608047050 con 0x557a8013fcb0 2024-09-10T09:59:28.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.201+0000 7ff620e05740 10 client.16296 fetch_fsmap learned FSMap version 32 2024-09-10T09:59:28.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.201+0000 7ff620e05740 1 -- 172.21.15.77:0/2477606496 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x557a801c5dc0 con 0x557a8013fcb0 2024-09-10T09:59:28.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.202+0000 7ff600c00640 1 -- 172.21.15.77:0/2477606496 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff608005500 con 0x557a8013fcb0 2024-09-10T09:59:28.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.202+0000 7ff600c00640 1 -- 172.21.15.77:0/2477606496 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7ff60802a6f0 con 0x557a8013fcb0 2024-09-10T09:59:28.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.202+0000 7ff620e05740 10 client.16296 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T09:59:28.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.202+0000 7ff620e05740 10 client.16296 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:59:28.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.202+0000 7ff620e05740 20 client.16296 populate_metadata read hostname 'smithi077' 2024-09-10T09:59:28.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.202+0000 7ff620e05740 1 --2- 172.21.15.77:0/2477606496 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x557a801c64e0 0x557a801c8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:28.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.202+0000 7ff620e05740 4 client.16296 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T09:59:28.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.202+0000 7ff620e05740 1 -- 172.21.15.77:0/2477606496 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x557a800add10 con 0x557a801c64e0 2024-09-10T09:59:28.206 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.202+0000 7ff616a00640 1 --2- 172.21.15.77:0/2477606496 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x557a801c64e0 0x557a801c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:28.206 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.203+0000 7ff616a00640 1 --2- 172.21.15.77:0/2477606496 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x557a801c64e0 0x557a801c8900 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c3a18f983949be1 server_cookie=c65133b260f0635d in_seq=0 out_seq=0 2024-09-10T09:59:28.206 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.203+0000 7ff600c00640 10 client.16296 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T09:59:28.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.204+0000 7ff600c00640 1 -- 172.21.15.77:0/2477606496 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x557a800add10 con 0x557a801c64e0 2024-09-10T09:59:28.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.204+0000 7ff600c00640 10 client.16296 handle_command_reply: tid=0 2024-09-10T09:59:28.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.204+0000 7ff620e05740 1 client.16296 shutdown 2024-09-10T09:59:28.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.204+0000 7ff620e05740 1 -- 172.21.15.77:0/2477606496 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a8013fcb0 msgr2=0x557a80142470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:28.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.204+0000 7ff620e05740 1 --2- 172.21.15.77:0/2477606496 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a8013fcb0 0x557a80142470 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7ff608003e50 tx=0x7ff608003e80 comp rx=0 tx=0).stop 2024-09-10T09:59:28.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.204+0000 7ff620e05740 1 -- 172.21.15.77:0/2477606496 shutdown_connections 2024-09-10T09:59:28.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.204+0000 7ff620e05740 1 --2- 172.21.15.77:0/2477606496 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x557a801c64e0 0x557a801c8900 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:28.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.204+0000 7ff620e05740 1 --2- 172.21.15.77:0/2477606496 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a8014ae90 0x557a80142ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:28.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.204+0000 7ff620e05740 1 --2- 172.21.15.77:0/2477606496 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a801405d0 0x557a801429b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:28.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.204+0000 7ff620e05740 1 --2- 172.21.15.77:0/2477606496 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a8013fcb0 0x557a80142470 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:28.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.204+0000 7ff600c00640 0 client.16296 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T09:59:28.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.204+0000 7ff620e05740 1 -- 172.21.15.77:0/2477606496 >> 172.21.15.77:0/2477606496 conn(0x557a8013c240 msgr2=0x557a8013b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:28.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.205+0000 7ff620e05740 1 -- 172.21.15.77:0/2477606496 shutdown_connections 2024-09-10T09:59:28.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.205+0000 7ff620e05740 1 -- 172.21.15.77:0/2477606496 wait complete. 2024-09-10T09:59:28.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.205+0000 7ff620e05740 20 client.16296 trim_cache size 0 max 16384 2024-09-10T09:59:28.208 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-10T09:59:28.208 INFO:teuthology.orchestra.run.smithi077.stdout: "return_code": 0, 2024-09-10T09:59:28.208 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_tag": "3afbde74-9f32-4915-aa5e-8910302ad662", 2024-09-10T09:59:28.208 INFO:teuthology.orchestra.run.smithi077.stdout: "mode": "asynchronous" 2024-09-10T09:59:28.208 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-10T09:59:28.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.205+0000 7ff61cc00640 1 -- 172.21.15.77:0/3189057692 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ff5fc0a2b80 msgr2=0x7ff5fc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:28.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.205+0000 7ff61cc00640 1 --2- 172.21.15.77:0/3189057692 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ff5fc0a2b80 0x7ff5fc0a5040 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x557a8013d0a0 tx=0x7ff60800b390 comp rx=0 tx=0).stop 2024-09-10T09:59:28.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.205+0000 7ff61cc00640 1 -- 172.21.15.77:0/3189057692 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff618102360 msgr2=0x7ff61819aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:28.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.205+0000 7ff61cc00640 1 --2- 172.21.15.77:0/3189057692 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff618102360 0x7ff61819aa70 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7ff604009230 tx=0x7ff604004270 comp rx=0 tx=0).stop 2024-09-10T09:59:28.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.206+0000 7ff61cc00640 1 -- 172.21.15.77:0/3189057692 shutdown_connections 2024-09-10T09:59:28.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.206+0000 7ff61cc00640 1 --2- 172.21.15.77:0/3189057692 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ff5fc0a2b80 0x7ff5fc0a5040 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:28.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.206+0000 7ff61cc00640 1 --2- 172.21.15.77:0/3189057692 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6181079c0 0x7ff61819ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:28.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.206+0000 7ff61cc00640 1 --2- 172.21.15.77:0/3189057692 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff618102360 0x7ff61819aa70 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:28.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.206+0000 7ff61cc00640 1 --2- 172.21.15.77:0/3189057692 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff6181019c0 0x7ff61819a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:28.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.206+0000 7ff61cc00640 1 -- 172.21.15.77:0/3189057692 >> 172.21.15.77:0/3189057692 conn(0x7ff6180fd740 msgr2=0x7ff6180fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:28.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.206+0000 7ff61cc00640 1 -- 172.21.15.77:0/3189057692 shutdown_connections 2024-09-10T09:59:28.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:28.206+0000 7ff61cc00640 1 -- 172.21.15.77:0/3189057692 wait complete. 2024-09-10T09:59:28.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:59:28 smithi134 ceph-mon[35674]: pgmap v520: 129 pgs: 129 active+clean; 96 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 599 KiB/s wr, 112 op/s 2024-09-10T09:59:28.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:59:28 smithi120 ceph-mon[34770]: pgmap v520: 129 pgs: 129 active+clean; 96 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 599 KiB/s wr, 112 op/s 2024-09-10T09:59:28.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:59:28 smithi077 ceph-mon[30319]: pgmap v520: 129 pgs: 129 active+clean; 96 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 599 KiB/s wr, 112 op/s 2024-09-10T09:59:29.101 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T09:59:29.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:59:29 smithi134 ceph-mon[35674]: scrub queued for path: / 2024-09-10T09:59:29.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:59:29 smithi134 ceph-mon[35674]: scrub summary: idle+waiting paths [/] 2024-09-10T09:59:29.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:59:29 smithi134 ceph-mon[35674]: scrub summary: active paths [/] 2024-09-10T09:59:29.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:59:29 smithi134 ceph-mon[35674]: scrub summary: active paths [/] 2024-09-10T09:59:29.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:59:29 smithi134 ceph-mon[35674]: scrub summary: active paths [/] 2024-09-10T09:59:29.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:59:29 smithi134 ceph-mon[35674]: scrub summary: active paths [/] 2024-09-10T09:59:29.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:59:29 smithi134 ceph-mon[35674]: scrub summary: idle+waiting paths [/] 2024-09-10T09:59:29.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:59:29 smithi134 ceph-mon[35674]: scrub summary: idle+waiting paths [/] 2024-09-10T09:59:29.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:59:29 smithi120 ceph-mon[34770]: scrub queued for path: / 2024-09-10T09:59:29.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:59:29 smithi120 ceph-mon[34770]: scrub summary: idle+waiting paths [/] 2024-09-10T09:59:29.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:59:29 smithi120 ceph-mon[34770]: scrub summary: active paths [/] 2024-09-10T09:59:29.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:59:29 smithi120 ceph-mon[34770]: scrub summary: active paths [/] 2024-09-10T09:59:29.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:59:29 smithi120 ceph-mon[34770]: scrub summary: active paths [/] 2024-09-10T09:59:29.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:59:29 smithi120 ceph-mon[34770]: scrub summary: active paths [/] 2024-09-10T09:59:29.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:59:29 smithi120 ceph-mon[34770]: scrub summary: idle+waiting paths [/] 2024-09-10T09:59:29.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:59:29 smithi120 ceph-mon[34770]: scrub summary: idle+waiting paths [/] 2024-09-10T09:59:29.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:59:29 smithi077 ceph-mon[30319]: scrub queued for path: / 2024-09-10T09:59:29.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:59:29 smithi077 ceph-mon[30319]: scrub summary: idle+waiting paths [/] 2024-09-10T09:59:29.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:59:29 smithi077 ceph-mon[30319]: scrub summary: active paths [/] 2024-09-10T09:59:29.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:59:29 smithi077 ceph-mon[30319]: scrub summary: active paths [/] 2024-09-10T09:59:29.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:59:29 smithi077 ceph-mon[30319]: scrub summary: active paths [/] 2024-09-10T09:59:29.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:59:29 smithi077 ceph-mon[30319]: scrub summary: active paths [/] 2024-09-10T09:59:29.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:59:29 smithi077 ceph-mon[30319]: scrub summary: idle+waiting paths [/] 2024-09-10T09:59:29.440 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:59:29 smithi077 ceph-mon[30319]: scrub summary: idle+waiting paths [/] 2024-09-10T09:59:29.469 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:59:30.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.022+0000 7fb213200640 1 -- 172.21.15.77:0/3812557935 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb20c1019c0 msgr2=0x7fb20c101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:30.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.022+0000 7fb213200640 1 --2- 172.21.15.77:0/3812557935 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb20c1019c0 0x7fb20c101e20 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc009930 tx=0x7fb1fc02a440 comp rx=0 tx=0).stop 2024-09-10T09:59:30.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.022+0000 7fb213200640 1 -- 172.21.15.77:0/3812557935 shutdown_connections 2024-09-10T09:59:30.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.022+0000 7fb213200640 1 --2- 172.21.15.77:0/3812557935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb20c102360 0x7fb20c10e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:30.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.022+0000 7fb213200640 1 --2- 172.21.15.77:0/3812557935 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb20c1019c0 0x7fb20c101e20 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:30.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.022+0000 7fb213200640 1 --2- 172.21.15.77:0/3812557935 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb20c1079c0 0x7fb20c107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:30.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.022+0000 7fb213200640 1 -- 172.21.15.77:0/3812557935 >> 172.21.15.77:0/3812557935 conn(0x7fb20c0fd740 msgr2=0x7fb20c0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:30.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.022+0000 7fb213200640 1 -- 172.21.15.77:0/3812557935 shutdown_connections 2024-09-10T09:59:30.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.023+0000 7fb213200640 1 -- 172.21.15.77:0/3812557935 wait complete. 2024-09-10T09:59:30.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.024+0000 7fb213200640 1 Processor -- start 2024-09-10T09:59:30.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.024+0000 7fb213200640 1 -- start start 2024-09-10T09:59:30.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.024+0000 7fb213200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb20c1019c0 0x7fb20c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:30.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.024+0000 7fb213200640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb20c102360 0x7fb20c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:30.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.024+0000 7fb213200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb20c1079c0 0x7fb20c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:30.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.024+0000 7fb213200640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb20c111e10 con 0x7fb20c1019c0 2024-09-10T09:59:30.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.024+0000 7fb213200640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb20c111c90 con 0x7fb20c102360 2024-09-10T09:59:30.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.024+0000 7fb213200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb20c111f90 con 0x7fb20c1079c0 2024-09-10T09:59:30.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.025+0000 7fb211e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb20c1019c0 0x7fb20c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:30.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.025+0000 7fb211400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb20c102360 0x7fb20c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:30.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.025+0000 7fb211e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb20c1019c0 0x7fb20c0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33190/0 (socket says 172.21.15.77:33190) 2024-09-10T09:59:30.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.025+0000 7fb211e00640 1 -- 172.21.15.77:0/4092014886 learned_addr learned my addr 172.21.15.77:0/4092014886 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:59:30.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.025+0000 7fb212800640 1 --2- 172.21.15.77:0/4092014886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb20c1079c0 0x7fb20c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:30.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.025+0000 7fb211400640 1 -- 172.21.15.77:0/4092014886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb20c1079c0 msgr2=0x7fb20c074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:30.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.025+0000 7fb211400640 1 --2- 172.21.15.77:0/4092014886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb20c1079c0 0x7fb20c074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:30.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.025+0000 7fb211400640 1 -- 172.21.15.77:0/4092014886 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb20c1019c0 msgr2=0x7fb20c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:30.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.025+0000 7fb211400640 1 --2- 172.21.15.77:0/4092014886 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb20c1019c0 0x7fb20c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:30.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.025+0000 7fb211400640 1 -- 172.21.15.77:0/4092014886 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb20c1a67b0 con 0x7fb20c102360 2024-09-10T09:59:30.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.025+0000 7fb211e00640 1 --2- 172.21.15.77:0/4092014886 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb20c1019c0 0x7fb20c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:59:30.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.026+0000 7fb211400640 1 --2- 172.21.15.77:0/4092014886 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb20c102360 0x7fb20c074a30 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc02a960 tx=0x7fb1fc003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:30.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.026+0000 7fb206a00640 1 -- 172.21.15.77:0/4092014886 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1fc03f070 con 0x7fb20c102360 2024-09-10T09:59:30.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.026+0000 7fb206a00640 1 -- 172.21.15.77:0/4092014886 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1fc031c90 con 0x7fb20c102360 2024-09-10T09:59:30.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.026+0000 7fb213200640 1 -- 172.21.15.77:0/4092014886 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb20c1a6a40 con 0x7fb20c102360 2024-09-10T09:59:30.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.026+0000 7fb206a00640 1 -- 172.21.15.77:0/4092014886 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1fc035030 con 0x7fb20c102360 2024-09-10T09:59:30.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.026+0000 7fb213200640 1 -- 172.21.15.77:0/4092014886 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb20c1a6f70 con 0x7fb20c102360 2024-09-10T09:59:30.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.029+0000 7fb206a00640 1 -- 172.21.15.77:0/4092014886 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb1fc042070 con 0x7fb20c102360 2024-09-10T09:59:30.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.029+0000 7fb2172c4740 1 Processor -- start 2024-09-10T09:59:30.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.029+0000 7fb2172c4740 1 -- start start 2024-09-10T09:59:30.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.030+0000 7fb2172c4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5e72a4d50 0x55e5e72a5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:30.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.030+0000 7fb2172c4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5e72a5670 0x55e5e72af9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:30.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.030+0000 7fb2172c4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e5e72aff30 0x55e5e72b2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:30.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.030+0000 7fb2172c4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e5e720b3e0 con 0x55e5e72aff30 2024-09-10T09:59:30.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.030+0000 7fb2172c4740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e5e71b2d80 con 0x55e5e72a4d50 2024-09-10T09:59:30.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.030+0000 7fb2172c4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e5e72073a0 con 0x55e5e72a5670 2024-09-10T09:59:30.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.030+0000 7fb211400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e5e72aff30 0x55e5e72b2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:30.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.030+0000 7fb211e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5e72a5670 0x55e5e72af9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:30.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.030+0000 7fb212800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5e72a4d50 0x55e5e72a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:30.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.030+0000 7fb211400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e5e72aff30 0x55e5e72b2320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33194/0 (socket says 172.21.15.77:33194) 2024-09-10T09:59:30.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.030+0000 7fb211e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5e72a5670 0x55e5e72af9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:56606/0 (socket says 172.21.15.77:56606) 2024-09-10T09:59:30.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.030+0000 7fb211400640 1 -- 172.21.15.77:0/1314086604 learned_addr learned my addr 172.21.15.77:0/1314086604 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:59:30.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.031+0000 7fb211e00640 1 -- 172.21.15.77:0/1314086604 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5e72a4d50 msgr2=0x55e5e72a5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:30.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.031+0000 7fb211e00640 1 --2- 172.21.15.77:0/1314086604 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5e72a4d50 0x55e5e72a5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:30.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.031+0000 7fb211e00640 1 -- 172.21.15.77:0/1314086604 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e5e72aff30 msgr2=0x55e5e72b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:30.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.031+0000 7fb211e00640 1 --2- 172.21.15.77:0/1314086604 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e5e72aff30 0x55e5e72b2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:30.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.031+0000 7fb211e00640 1 -- 172.21.15.77:0/1314086604 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5e71d1ae0 con 0x55e5e72a5670 2024-09-10T09:59:30.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.031+0000 7fb206a00640 1 --2- 172.21.15.77:0/4092014886 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fb1f40a2c00 0x7fb1f40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:30.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.031+0000 7fb211e00640 1 --2- 172.21.15.77:0/1314086604 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5e72a5670 0x55e5e72af9f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb200005d00 tx=0x7fb200011950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45eabbd253f80f04 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:30.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.031+0000 7fb206a00640 1 -- 172.21.15.77:0/4092014886 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fb1fc0ef9f0 con 0x7fb20c102360 2024-09-10T09:59:30.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.031+0000 7fb212800640 1 --2- 172.21.15.77:0/4092014886 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fb1f40a2c00 0x7fb1f40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:30.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.031+0000 7fb1fb000640 1 -- 172.21.15.77:0/1314086604 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb200005020 con 0x55e5e72a5670 2024-09-10T09:59:30.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.031+0000 7fb1fb000640 1 -- 172.21.15.77:0/1314086604 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2000047a0 con 0x55e5e72a5670 2024-09-10T09:59:30.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.031+0000 7fb1fb000640 1 -- 172.21.15.77:0/1314086604 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb20000f9c0 con 0x55e5e72a5670 2024-09-10T09:59:30.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.032+0000 7fb2172c4740 1 -- 172.21.15.77:0/1314086604 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5e72a5670 msgr2=0x55e5e72af9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:30.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.032+0000 7fb212800640 1 --2- 172.21.15.77:0/4092014886 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fb1f40a2c00 0x7fb1f40a50c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fb208005fd0 tx=0x7fb208009600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:30.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.032+0000 7fb2172c4740 1 --2- 172.21.15.77:0/1314086604 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5e72a5670 0x55e5e72af9f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb200005d00 tx=0x7fb200011950 comp rx=0 tx=0).stop 2024-09-10T09:59:30.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.032+0000 7fb2172c4740 1 -- 172.21.15.77:0/1314086604 shutdown_connections 2024-09-10T09:59:30.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.032+0000 7fb2172c4740 1 --2- 172.21.15.77:0/1314086604 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e5e72aff30 0x55e5e72b2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:30.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.032+0000 7fb2172c4740 1 --2- 172.21.15.77:0/1314086604 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5e72a5670 0x55e5e72af9f0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:30.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.032+0000 7fb2172c4740 1 --2- 172.21.15.77:0/1314086604 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5e72a4d50 0x55e5e72a5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:30.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.032+0000 7fb2172c4740 1 -- 172.21.15.77:0/1314086604 >> 172.21.15.77:0/1314086604 conn(0x55e5e72a12e0 msgr2=0x55e5e72a16c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:30.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.032+0000 7fb2172c4740 1 -- 172.21.15.77:0/1314086604 shutdown_connections 2024-09-10T09:59:30.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.033+0000 7fb2172c4740 1 -- 172.21.15.77:0/1314086604 wait complete. 2024-09-10T09:59:30.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.033+0000 7fb2172c4740 1 -- start start 2024-09-10T09:59:30.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.033+0000 7fb2172c4740 1 Processor -- start 2024-09-10T09:59:30.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.034+0000 7fb2172c4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e5e72a4d50 0x55e5e72acdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:30.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.034+0000 7fb2172c4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5e72a5670 0x55e5e72ad310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:30.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.034+0000 7fb2172c4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5e72aff30 0x55e5e72ad850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:30.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.034+0000 7fb2172c4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e5e72b2ba0 con 0x55e5e72a4d50 2024-09-10T09:59:30.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.034+0000 7fb2172c4740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e5e71c60f0 con 0x55e5e72a5670 2024-09-10T09:59:30.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.034+0000 7fb2172c4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e5e72b2d20 con 0x55e5e72aff30 2024-09-10T09:59:30.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.034+0000 7fb2172c4740 10 client.0 _ll_register_callbacks cb 0x55e5e71b79e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:59:30.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.034+0000 7fb211e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e5e72a4d50 0x55e5e72acdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:30.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.034+0000 7fb211e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e5e72a4d50 0x55e5e72acdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33204/0 (socket says 172.21.15.77:33204) 2024-09-10T09:59:30.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.034+0000 7fb211e00640 1 -- 172.21.15.77:0/871334675 learned_addr learned my addr 172.21.15.77:0/871334675 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:59:30.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.034+0000 7fb212800640 1 --2- 172.21.15.77:0/871334675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5e72aff30 0x55e5e72ad850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:30.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.034+0000 7fb211400640 1 --2- 172.21.15.77:0/871334675 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5e72a5670 0x55e5e72ad310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:30.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.034+0000 7fb212800640 1 -- 172.21.15.77:0/871334675 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5e72a5670 msgr2=0x55e5e72ad310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:30.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.034+0000 7fb212800640 1 --2- 172.21.15.77:0/871334675 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5e72a5670 0x55e5e72ad310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:30.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.034+0000 7fb212800640 1 -- 172.21.15.77:0/871334675 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e5e72a4d50 msgr2=0x55e5e72acdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:30.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.034+0000 7fb212800640 1 --2- 172.21.15.77:0/871334675 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e5e72a4d50 0x55e5e72acdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:30.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.034+0000 7fb212800640 1 -- 172.21.15.77:0/871334675 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5e6f7c3c0 con 0x55e5e72aff30 2024-09-10T09:59:30.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.035+0000 7fb211400640 1 --2- 172.21.15.77:0/871334675 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5e72a5670 0x55e5e72ad310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:59:30.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.035+0000 7fb212800640 1 --2- 172.21.15.77:0/871334675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5e72aff30 0x55e5e72ad850 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb208011d10 tx=0x7fb208004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39d69b26896e4b41 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:30.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.035+0000 7fb1ef400640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-10T09:59:30.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.035+0000 7fb1ef400640 1 -- 172.21.15.77:0/871334675 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb208020620 con 0x55e5e72aff30 2024-09-10T09:59:30.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.035+0000 7fb2172c4740 1 -- 172.21.15.77:0/871334675 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2000047a0 con 0x55e5e72aff30 2024-09-10T09:59:30.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.035+0000 7fb1ef400640 1 -- 172.21.15.77:0/871334675 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fb2080207c0 con 0x55e5e72aff30 2024-09-10T09:59:30.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.035+0000 7fb1ef400640 1 -- 172.21.15.77:0/871334675 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb208004b20 con 0x55e5e72aff30 2024-09-10T09:59:30.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.035+0000 7fb2172c4740 10 client.26341 fetch_fsmap learned FSMap version 32 2024-09-10T09:59:30.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.036+0000 7fb2172c4740 1 -- 172.21.15.77:0/871334675 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e5e72be8e0 con 0x55e5e72aff30 2024-09-10T09:59:30.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.036+0000 7fb1ef400640 1 -- 172.21.15.77:0/871334675 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2000047a0 con 0x55e5e72aff30 2024-09-10T09:59:30.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.036+0000 7fb1ef400640 1 -- 172.21.15.77:0/871334675 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fb20800f040 con 0x55e5e72aff30 2024-09-10T09:59:30.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.036+0000 7fb2172c4740 10 client.26341 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T09:59:30.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.036+0000 7fb2172c4740 10 client.26341 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:59:30.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.036+0000 7fb2172c4740 20 client.26341 populate_metadata read hostname 'smithi077' 2024-09-10T09:59:30.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.036+0000 7fb2172c4740 1 --2- 172.21.15.77:0/871334675 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55e5e72bfdc0 0x55e5e72c2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:30.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.036+0000 7fb2172c4740 4 client.26341 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:59:30.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.036+0000 7fb2172c4740 1 -- 172.21.15.77:0/871334675 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e5e7218950 con 0x55e5e72bfdc0 2024-09-10T09:59:30.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.037+0000 7fb211e00640 1 --2- 172.21.15.77:0/871334675 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55e5e72bfdc0 0x55e5e72c2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:30.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.037+0000 7fb211e00640 1 --2- 172.21.15.77:0/871334675 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55e5e72bfdc0 0x55e5e72c2240 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b9ba1f17edcb876 server_cookie=973e44ca1535a712 in_seq=0 out_seq=0 2024-09-10T09:59:30.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.037+0000 7fb1ef400640 10 client.26341 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T09:59:30.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.038+0000 7fb1ef400640 1 -- 172.21.15.77:0/871334675 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e5e7218950 con 0x55e5e72bfdc0 2024-09-10T09:59:30.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.038+0000 7fb1ef400640 10 client.26341 handle_command_reply: tid=0 2024-09-10T09:59:30.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.039+0000 7fb2172c4740 1 client.26341 shutdown 2024-09-10T09:59:30.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.039+0000 7fb2172c4740 1 -- 172.21.15.77:0/871334675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5e72aff30 msgr2=0x55e5e72ad850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:30.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.039+0000 7fb2172c4740 1 --2- 172.21.15.77:0/871334675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5e72aff30 0x55e5e72ad850 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb208011d10 tx=0x7fb208004a40 comp rx=0 tx=0).stop 2024-09-10T09:59:30.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.039+0000 7fb2172c4740 1 -- 172.21.15.77:0/871334675 shutdown_connections 2024-09-10T09:59:30.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.039+0000 7fb2172c4740 1 --2- 172.21.15.77:0/871334675 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55e5e72bfdc0 0x55e5e72c2240 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:30.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.039+0000 7fb2172c4740 1 --2- 172.21.15.77:0/871334675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5e72aff30 0x55e5e72ad850 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:30.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.039+0000 7fb2172c4740 1 --2- 172.21.15.77:0/871334675 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5e72a5670 0x55e5e72ad310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:30.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.039+0000 7fb2172c4740 1 --2- 172.21.15.77:0/871334675 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e5e72a4d50 0x55e5e72acdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:30.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.039+0000 7fb1ef400640 0 client.26341 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T09:59:30.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.039+0000 7fb2172c4740 1 -- 172.21.15.77:0/871334675 >> 172.21.15.77:0/871334675 conn(0x55e5e72a12e0 msgr2=0x55e5e72a6fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:30.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.039+0000 7fb2172c4740 1 -- 172.21.15.77:0/871334675 shutdown_connections 2024-09-10T09:59:30.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.039+0000 7fb2172c4740 1 -- 172.21.15.77:0/871334675 wait complete. 2024-09-10T09:59:30.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.039+0000 7fb2172c4740 20 client.26341 trim_cache size 0 max 16384 2024-09-10T09:59:30.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.057+0000 7fb2172c4740 1 Processor -- start 2024-09-10T09:59:30.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.057+0000 7fb2172c4740 1 -- start start 2024-09-10T09:59:30.058 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.057+0000 7fb2172c4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e5e72a4d50 0x55e5e729c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:30.058 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.057+0000 7fb2172c4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5e72a5670 0x55e5e72a0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:30.058 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.057+0000 7fb2172c4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5e72aff30 0x55e5e72c7b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:30.058 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.057+0000 7fb2172c4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb20c1b0fa0 con 0x55e5e72a4d50 2024-09-10T09:59:30.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.057+0000 7fb2172c4740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb20c1b2ba0 con 0x55e5e72a5670 2024-09-10T09:59:30.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.057+0000 7fb2172c4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb20c1b1670 con 0x55e5e72aff30 2024-09-10T09:59:30.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.057+0000 7fb211e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e5e72a4d50 0x55e5e729c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:30.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.057+0000 7fb211e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e5e72a4d50 0x55e5e729c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33208/0 (socket says 172.21.15.77:33208) 2024-09-10T09:59:30.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.057+0000 7fb211e00640 1 -- 172.21.15.77:0/512422135 learned_addr learned my addr 172.21.15.77:0/512422135 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:59:30.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.058+0000 7fb211400640 1 --2- 172.21.15.77:0/512422135 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5e72a5670 0x55e5e72a0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:30.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.058+0000 7fb212800640 1 --2- 172.21.15.77:0/512422135 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5e72aff30 0x55e5e72c7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:30.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.058+0000 7fb211e00640 1 -- 172.21.15.77:0/512422135 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5e72aff30 msgr2=0x55e5e72c7b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:30.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.058+0000 7fb211e00640 1 --2- 172.21.15.77:0/512422135 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5e72aff30 0x55e5e72c7b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:30.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.058+0000 7fb211e00640 1 -- 172.21.15.77:0/512422135 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5e72a5670 msgr2=0x55e5e72a0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:30.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.058+0000 7fb211e00640 1 --2- 172.21.15.77:0/512422135 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5e72a5670 0x55e5e72a0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:30.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.058+0000 7fb211e00640 1 -- 172.21.15.77:0/512422135 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5e72abd80 con 0x55e5e72a4d50 2024-09-10T09:59:30.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.058+0000 7fb211400640 1 --2- 172.21.15.77:0/512422135 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5e72a5670 0x55e5e72a0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:59:30.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.058+0000 7fb211e00640 1 --2- 172.21.15.77:0/512422135 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e5e72a4d50 0x55e5e729c060 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fb2000098e0 tx=0x7fb20000f620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c419a3ebce531d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:30.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.058+0000 7fb1fa600640 1 -- 172.21.15.77:0/512422135 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb200004050 con 0x55e5e72a4d50 2024-09-10T09:59:30.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.058+0000 7fb1fa600640 1 -- 172.21.15.77:0/512422135 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2000046d0 con 0x55e5e72a4d50 2024-09-10T09:59:30.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.058+0000 7fb1fa600640 1 -- 172.21.15.77:0/512422135 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb200003850 con 0x55e5e72a4d50 2024-09-10T09:59:30.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.059+0000 7fb2172c4740 1 -- 172.21.15.77:0/512422135 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e5e72a4d50 msgr2=0x55e5e729c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:30.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.059+0000 7fb2172c4740 1 --2- 172.21.15.77:0/512422135 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e5e72a4d50 0x55e5e729c060 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fb2000098e0 tx=0x7fb20000f620 comp rx=0 tx=0).stop 2024-09-10T09:59:30.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.059+0000 7fb2172c4740 1 -- 172.21.15.77:0/512422135 shutdown_connections 2024-09-10T09:59:30.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.059+0000 7fb2172c4740 1 --2- 172.21.15.77:0/512422135 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5e72aff30 0x55e5e72c7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:30.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.059+0000 7fb2172c4740 1 --2- 172.21.15.77:0/512422135 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5e72a5670 0x55e5e72a0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:30.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.059+0000 7fb2172c4740 1 --2- 172.21.15.77:0/512422135 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e5e72a4d50 0x55e5e729c060 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:30.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.059+0000 7fb2172c4740 1 -- 172.21.15.77:0/512422135 >> 172.21.15.77:0/512422135 conn(0x55e5e72a12e0 msgr2=0x55e5e729fb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:30.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.059+0000 7fb2172c4740 1 -- 172.21.15.77:0/512422135 shutdown_connections 2024-09-10T09:59:30.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.059+0000 7fb2172c4740 1 -- 172.21.15.77:0/512422135 wait complete. 2024-09-10T09:59:30.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.060+0000 7fb2172c4740 1 -- start start 2024-09-10T09:59:30.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.060+0000 7fb2172c4740 1 Processor -- start 2024-09-10T09:59:30.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.060+0000 7fb2172c4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e5e72a4d50 0x55e5e72b85b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:30.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.060+0000 7fb2172c4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5e72a5670 0x55e5e72b38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:30.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.060+0000 7fb2172c4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5e72aff30 0x55e5e72b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:30.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.060+0000 7fb2172c4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb20c1b2410 con 0x55e5e72a4d50 2024-09-10T09:59:30.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.060+0000 7fb2172c4740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb20c1b1d40 con 0x55e5e72a5670 2024-09-10T09:59:30.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.060+0000 7fb2172c4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb20c1aef80 con 0x55e5e72aff30 2024-09-10T09:59:30.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.060+0000 7fb2172c4740 10 client.0 _ll_register_callbacks cb 0x7fb2080118f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:59:30.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.061+0000 7fb211400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5e72a5670 0x55e5e72b38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:30.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.061+0000 7fb211400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5e72a5670 0x55e5e72b38e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:60112/0 (socket says 172.21.15.77:60112) 2024-09-10T09:59:30.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.061+0000 7fb211400640 1 -- 172.21.15.77:0/4091667529 learned_addr learned my addr 172.21.15.77:0/4091667529 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:59:30.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.061+0000 7fb211e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e5e72a4d50 0x55e5e72b85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:30.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.061+0000 7fb212800640 1 --2- 172.21.15.77:0/4091667529 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5e72aff30 0x55e5e72b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:30.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.061+0000 7fb211400640 1 -- 172.21.15.77:0/4091667529 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5e72aff30 msgr2=0x55e5e72b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:30.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.061+0000 7fb211400640 1 --2- 172.21.15.77:0/4091667529 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5e72aff30 0x55e5e72b3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:30.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.061+0000 7fb211400640 1 -- 172.21.15.77:0/4091667529 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e5e72a4d50 msgr2=0x55e5e72b85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:30.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.061+0000 7fb211400640 1 --2- 172.21.15.77:0/4091667529 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e5e72a4d50 0x55e5e72b85b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:30.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.061+0000 7fb211400640 1 -- 172.21.15.77:0/4091667529 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5e72b4510 con 0x55e5e72a5670 2024-09-10T09:59:30.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.061+0000 7fb211e00640 1 --2- 172.21.15.77:0/4091667529 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e5e72a4d50 0x55e5e72b85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:59:30.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.061+0000 7fb211400640 1 --2- 172.21.15.77:0/4091667529 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5e72a5670 0x55e5e72b38e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc004650 tx=0x7fb1fc031490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bbd30b44c9c85ecc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:30.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.061+0000 7fb1ef400640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T09:59:30.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.062+0000 7fb1ef400640 1 -- 172.21.15.77:0/4091667529 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1fc049070 con 0x55e5e72a5670 2024-09-10T09:59:30.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.062+0000 7fb1ef400640 1 -- 172.21.15.77:0/4091667529 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fb1fc0b8520 con 0x55e5e72a5670 2024-09-10T09:59:30.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.062+0000 7fb2172c4740 1 -- 172.21.15.77:0/4091667529 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2000046d0 con 0x55e5e72a5670 2024-09-10T09:59:30.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.062+0000 7fb1ef400640 1 -- 172.21.15.77:0/4091667529 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb1fc0bb9f0 con 0x55e5e72a5670 2024-09-10T09:59:30.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.062+0000 7fb2172c4740 10 client.26396 fetch_fsmap learned FSMap version 32 2024-09-10T09:59:30.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.062+0000 7fb2172c4740 1 -- 172.21.15.77:0/4091667529 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e5e732aa10 con 0x55e5e72a5670 2024-09-10T09:59:30.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.062+0000 7fb1ef400640 1 -- 172.21.15.77:0/4091667529 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2000046d0 con 0x55e5e72a5670 2024-09-10T09:59:30.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.062+0000 7fb1ef400640 1 -- 172.21.15.77:0/4091667529 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fb1fc0c45e0 con 0x55e5e72a5670 2024-09-10T09:59:30.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.062+0000 7fb2172c4740 10 client.26396 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T09:59:30.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.062+0000 7fb2172c4740 10 client.26396 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:59:30.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.062+0000 7fb2172c4740 20 client.26396 populate_metadata read hostname 'smithi077' 2024-09-10T09:59:30.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.062+0000 7fb2172c4740 1 --2- 172.21.15.77:0/4091667529 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55e5e732b200 0x55e5e732d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:30.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.062+0000 7fb2172c4740 4 client.26396 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T09:59:30.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.062+0000 7fb2172c4740 1 -- 172.21.15.77:0/4091667529 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e5e7212e60 con 0x55e5e732b200 2024-09-10T09:59:30.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.063+0000 7fb211e00640 1 --2- 172.21.15.77:0/4091667529 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55e5e732b200 0x55e5e732d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:30.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.063+0000 7fb211e00640 1 --2- 172.21.15.77:0/4091667529 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55e5e732b200 0x55e5e732d620 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c64771b2dfd890f8 server_cookie=fb309f077d72a276 in_seq=0 out_seq=0 2024-09-10T09:59:30.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.063+0000 7fb1ef400640 10 client.26396 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T09:59:30.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.064+0000 7fb1ef400640 1 -- 172.21.15.77:0/4091667529 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55e5e7212e60 con 0x55e5e732b200 2024-09-10T09:59:30.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.064+0000 7fb1ef400640 10 client.26396 handle_command_reply: tid=0 2024-09-10T09:59:30.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.064+0000 7fb2172c4740 1 client.26396 shutdown 2024-09-10T09:59:30.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.064+0000 7fb2172c4740 1 -- 172.21.15.77:0/4091667529 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5e72a5670 msgr2=0x55e5e72b38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:30.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.064+0000 7fb2172c4740 1 --2- 172.21.15.77:0/4091667529 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5e72a5670 0x55e5e72b38e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc004650 tx=0x7fb1fc031490 comp rx=0 tx=0).stop 2024-09-10T09:59:30.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.064+0000 7fb2172c4740 1 -- 172.21.15.77:0/4091667529 shutdown_connections 2024-09-10T09:59:30.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.064+0000 7fb2172c4740 1 --2- 172.21.15.77:0/4091667529 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55e5e732b200 0x55e5e732d620 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:30.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.064+0000 7fb2172c4740 1 --2- 172.21.15.77:0/4091667529 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5e72aff30 0x55e5e72b3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:30.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.064+0000 7fb2172c4740 1 --2- 172.21.15.77:0/4091667529 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5e72a5670 0x55e5e72b38e0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:30.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.064+0000 7fb2172c4740 1 --2- 172.21.15.77:0/4091667529 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e5e72a4d50 0x55e5e72b85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:30.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.064+0000 7fb2172c4740 1 -- 172.21.15.77:0/4091667529 >> 172.21.15.77:0/4091667529 conn(0x55e5e72a12e0 msgr2=0x55e5e72aee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:30.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.064+0000 7fb1ef400640 0 client.26396 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T09:59:30.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.064+0000 7fb2172c4740 1 -- 172.21.15.77:0/4091667529 shutdown_connections 2024-09-10T09:59:30.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.064+0000 7fb2172c4740 1 -- 172.21.15.77:0/4091667529 wait complete. 2024-09-10T09:59:30.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.064+0000 7fb2172c4740 20 client.26396 trim_cache size 0 max 16384 2024-09-10T09:59:30.069 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-10T09:59:30.069 INFO:teuthology.orchestra.run.smithi077.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T09:59:30.069 INFO:teuthology.orchestra.run.smithi077.stdout: "scrubs": { 2024-09-10T09:59:30.069 INFO:teuthology.orchestra.run.smithi077.stdout: "3afbde74-9f32-4915-aa5e-8910302ad662": { 2024-09-10T09:59:30.069 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/", 2024-09-10T09:59:30.069 INFO:teuthology.orchestra.run.smithi077.stdout: "tag": "3afbde74-9f32-4915-aa5e-8910302ad662", 2024-09-10T09:59:30.069 INFO:teuthology.orchestra.run.smithi077.stdout: "options": "recursive,force" 2024-09-10T09:59:30.069 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T09:59:30.069 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T09:59:30.069 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-10T09:59:30.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.065+0000 7fb213200640 1 -- 172.21.15.77:0/4092014886 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fb1f40a2c00 msgr2=0x7fb1f40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:30.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.065+0000 7fb213200640 1 --2- 172.21.15.77:0/4092014886 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fb1f40a2c00 0x7fb1f40a50c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fb208005fd0 tx=0x7fb208009600 comp rx=0 tx=0).stop 2024-09-10T09:59:30.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.065+0000 7fb213200640 1 -- 172.21.15.77:0/4092014886 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb20c102360 msgr2=0x7fb20c074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:30.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.065+0000 7fb213200640 1 --2- 172.21.15.77:0/4092014886 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb20c102360 0x7fb20c074a30 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc02a960 tx=0x7fb1fc003b90 comp rx=0 tx=0).stop 2024-09-10T09:59:30.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.065+0000 7fb213200640 1 -- 172.21.15.77:0/4092014886 shutdown_connections 2024-09-10T09:59:30.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.065+0000 7fb213200640 1 --2- 172.21.15.77:0/4092014886 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fb1f40a2c00 0x7fb1f40a50c0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:30.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.065+0000 7fb213200640 1 --2- 172.21.15.77:0/4092014886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb20c1079c0 0x7fb20c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:30.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.065+0000 7fb213200640 1 --2- 172.21.15.77:0/4092014886 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb20c102360 0x7fb20c074a30 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:30.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.065+0000 7fb213200640 1 --2- 172.21.15.77:0/4092014886 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb20c1019c0 0x7fb20c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:30.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.065+0000 7fb213200640 1 -- 172.21.15.77:0/4092014886 >> 172.21.15.77:0/4092014886 conn(0x7fb20c0fd740 msgr2=0x7fb20c0fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:30.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.065+0000 7fb213200640 1 -- 172.21.15.77:0/4092014886 shutdown_connections 2024-09-10T09:59:30.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:30.065+0000 7fb213200640 1 -- 172.21.15.77:0/4092014886 wait complete. 2024-09-10T09:59:30.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:59:30 smithi077 ceph-mon[30319]: scrub summary: active paths [/] 2024-09-10T09:59:30.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:59:30 smithi077 ceph-mon[30319]: scrub summary: idle+waiting paths [/] 2024-09-10T09:59:30.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:59:30 smithi077 ceph-mon[30319]: scrub summary: idle+waiting paths [/] 2024-09-10T09:59:30.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:59:30 smithi077 ceph-mon[30319]: scrub summary: idle+waiting paths [/] 2024-09-10T09:59:30.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:59:30 smithi077 ceph-mon[30319]: pgmap v521: 129 pgs: 129 active+clean; 96 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 601 KiB/s wr, 116 op/s 2024-09-10T09:59:30.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:59:30 smithi134 ceph-mon[35674]: scrub summary: active paths [/] 2024-09-10T09:59:30.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:59:30 smithi134 ceph-mon[35674]: scrub summary: idle+waiting paths [/] 2024-09-10T09:59:30.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:59:30 smithi134 ceph-mon[35674]: scrub summary: idle+waiting paths [/] 2024-09-10T09:59:30.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:59:30 smithi134 ceph-mon[35674]: scrub summary: idle+waiting paths [/] 2024-09-10T09:59:30.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:59:30 smithi134 ceph-mon[35674]: pgmap v521: 129 pgs: 129 active+clean; 96 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 601 KiB/s wr, 116 op/s 2024-09-10T09:59:30.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:59:30 smithi120 ceph-mon[34770]: scrub summary: active paths [/] 2024-09-10T09:59:30.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:59:30 smithi120 ceph-mon[34770]: scrub summary: idle+waiting paths [/] 2024-09-10T09:59:30.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:59:30 smithi120 ceph-mon[34770]: scrub summary: idle+waiting paths [/] 2024-09-10T09:59:30.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:59:30 smithi120 ceph-mon[34770]: scrub summary: idle+waiting paths [/] 2024-09-10T09:59:30.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:59:30 smithi120 ceph-mon[34770]: pgmap v521: 129 pgs: 129 active+clean; 96 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 601 KiB/s wr, 116 op/s 2024-09-10T09:59:31.111 INFO:tasks.cephfs.filesystem:scrub status for tag:3afbde74-9f32-4915-aa5e-8910302ad662 - {'path': '/', 'tag': '3afbde74-9f32-4915-aa5e-8910302ad662', 'options': 'recursive,force'} 2024-09-10T09:59:31.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:59:31 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:59:31.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:59:31 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:59:31.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:59:31 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:59:32.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:59:32 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:59:32.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:59:32 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:59:32.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:59:32 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:59:32.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:59:32 smithi134 ceph-mon[35674]: pgmap v522: 129 pgs: 129 active+clean; 96 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 4.2 KiB/s wr, 12 op/s 2024-09-10T09:59:32.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:59:32 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:59:32.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:59:32 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:59:32.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:59:32 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:59:32.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:59:32 smithi120 ceph-mon[34770]: pgmap v522: 129 pgs: 129 active+clean; 96 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 4.2 KiB/s wr, 12 op/s 2024-09-10T09:59:32.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:59:32 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:59:32.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:59:32 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:59:32.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:59:32 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T09:59:32.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:59:32 smithi077 ceph-mon[30319]: pgmap v522: 129 pgs: 129 active+clean; 96 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 4.2 KiB/s wr, 12 op/s 2024-09-10T09:59:34.544 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:59:34.921 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:59:35.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:59:34 smithi134 ceph-mon[35674]: pgmap v523: 129 pgs: 129 active+clean; 96 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-10T09:59:35.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:59:34 smithi120 ceph-mon[34770]: pgmap v523: 129 pgs: 129 active+clean; 96 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-10T09:59:35.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:59:34 smithi077 ceph-mon[30319]: pgmap v523: 129 pgs: 129 active+clean; 96 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-10T09:59:35.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:35.434+0000 7fcb20e00640 1 -- 172.21.15.77:0/2852437157 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcb1c103ff0 msgr2=0x7fcb1c1043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:35.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:35.434+0000 7fcb20e00640 1 --2- 172.21.15.77:0/2852437157 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcb1c103ff0 0x7fcb1c1043d0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c009900 tx=0x7fcb0c02a410 comp rx=0 tx=0).stop 2024-09-10T09:59:35.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:35.434+0000 7fcb20e00640 1 -- 172.21.15.77:0/2852437157 shutdown_connections 2024-09-10T09:59:35.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:35.434+0000 7fcb20e00640 1 --2- 172.21.15.77:0/2852437157 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb1c109120 0x7fcb1c110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:35.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:35.434+0000 7fcb20e00640 1 --2- 172.21.15.77:0/2852437157 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb1c1049a0 0x7fcb1c1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:35.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:35.434+0000 7fcb20e00640 1 --2- 172.21.15.77:0/2852437157 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcb1c103ff0 0x7fcb1c1043d0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:35.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:35.434+0000 7fcb20e00640 1 -- 172.21.15.77:0/2852437157 >> 172.21.15.77:0/2852437157 conn(0x7fcb1c0ffa90 msgr2=0x7fcb1c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:35.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:35.434+0000 7fcb20e00640 1 -- 172.21.15.77:0/2852437157 shutdown_connections 2024-09-10T09:59:35.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:35.434+0000 7fcb20e00640 1 -- 172.21.15.77:0/2852437157 wait complete. 2024-09-10T09:59:35.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:35.435+0000 7fcb20e00640 1 Processor -- start 2024-09-10T09:59:35.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:35.435+0000 7fcb20e00640 1 -- start start 2024-09-10T09:59:35.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:35.435+0000 7fcb20e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb1c103ff0 0x7fcb1c19a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:35.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:35.435+0000 7fcb20e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcb1c1049a0 0x7fcb1c19ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:35.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:35.435+0000 7fcb20e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb1c109120 0x7fcb1c19eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:35.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:35.435+0000 7fcb20e00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fcb1c1141a0 con 0x7fcb1c1049a0 2024-09-10T09:59:35.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:35.435+0000 7fcb20e00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fcb1c114020 con 0x7fcb1c109120 2024-09-10T09:59:35.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:35.435+0000 7fcb20e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcb1c114320 con 0x7fcb1c103ff0 2024-09-10T09:59:35.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:35.435+0000 7fcb13e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcb1c1049a0 0x7fcb1c19ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:35.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:35.436+0000 7fcb13e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcb1c1049a0 0x7fcb1c19ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:40888/0 (socket says 172.21.15.77:40888) 2024-09-10T09:59:35.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:35.436+0000 7fcb13e00640 1 -- 172.21.15.77:0/1550461173 learned_addr learned my addr 172.21.15.77:0/1550461173 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:59:35.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:35.436+0000 7fcb19e00640 1 --2- 172.21.15.77:0/1550461173 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb1c103ff0 0x7fcb1c19a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:35.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:35.436+0000 7fcb1a800640 1 --2- 172.21.15.77:0/1550461173 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb1c109120 0x7fcb1c19eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:35.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:35.436+0000 7fcb13e00640 1 -- 172.21.15.77:0/1550461173 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb1c103ff0 msgr2=0x7fcb1c19a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:35.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:35.436+0000 7fcb13e00640 1 --2- 172.21.15.77:0/1550461173 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb1c103ff0 0x7fcb1c19a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:35.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:35.436+0000 7fcb13e00640 1 -- 172.21.15.77:0/1550461173 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb1c109120 msgr2=0x7fcb1c19eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:35.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:35.436+0000 7fcb13e00640 1 --2- 172.21.15.77:0/1550461173 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb1c109120 0x7fcb1c19eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:35.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:35.436+0000 7fcb13e00640 1 -- 172.21.15.77:0/1550461173 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb1c19f5d0 con 0x7fcb1c1049a0 2024-09-10T09:59:35.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:35.436+0000 7fcb1a800640 1 --2- 172.21.15.77:0/1550461173 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb1c109120 0x7fcb1c19eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:59:35.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:35.436+0000 7fcb13e00640 1 --2- 172.21.15.77:0/1550461173 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcb1c1049a0 0x7fcb1c19ab60 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fcb0400d950 tx=0x7fcb0400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:35.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:35.437+0000 7fcb12a00640 1 -- 172.21.15.77:0/1550461173 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb04017030 con 0x7fcb1c1049a0 2024-09-10T09:59:35.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:35.437+0000 7fcb12a00640 1 -- 172.21.15.77:0/1550461173 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb04004020 con 0x7fcb1c1049a0 2024-09-10T09:59:35.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:35.437+0000 7fcb12a00640 1 -- 172.21.15.77:0/1550461173 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb040047a0 con 0x7fcb1c1049a0 2024-09-10T09:59:35.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:35.437+0000 7fcb20e00640 1 -- 172.21.15.77:0/1550461173 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb1c19f8c0 con 0x7fcb1c1049a0 2024-09-10T09:59:35.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:35.437+0000 7fcb20e00640 1 -- 172.21.15.77:0/1550461173 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb1c1a71a0 con 0x7fcb1c1049a0 2024-09-10T09:59:35.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:35.438+0000 7fcb20e00640 1 -- 172.21.15.77:0/1550461173 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb1c1043d0 con 0x7fcb1c1049a0 2024-09-10T09:59:35.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:35.440+0000 7fcb12a00640 1 -- 172.21.15.77:0/1550461173 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fcb040041d0 con 0x7fcb1c1049a0 2024-09-10T09:59:35.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:35.441+0000 7fcb12a00640 1 --2- 172.21.15.77:0/1550461173 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fcb000a2b80 0x7fcb000a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:35.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:35.442+0000 7fcb12a00640 1 -- 172.21.15.77:0/1550461173 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fcb040cedd0 con 0x7fcb1c1049a0 2024-09-10T09:59:35.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:35.442+0000 7fcb19e00640 1 --2- 172.21.15.77:0/1550461173 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fcb000a2b80 0x7fcb000a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:35.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:35.443+0000 7fcb19e00640 1 --2- 172.21.15.77:0/1550461173 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fcb000a2b80 0x7fcb000a5040 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c000c00 tx=0x7fcb0c005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:35.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:35.445+0000 7fcb12a00640 1 -- 172.21.15.77:0/1550461173 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcb0409b8e0 con 0x7fcb1c1049a0 2024-09-10T09:59:35.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:35.653+0000 7fcb20e00640 1 -- 172.21.15.77:0/1550461173 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcb1c19b980 con 0x7fcb1c1049a0 2024-09-10T09:59:35.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:35.653+0000 7fcb12a00640 1 -- 172.21.15.77:0/1550461173 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11996 (secure 0 0 0) 0x7fcb040b9e20 con 0x7fcb1c1049a0 2024-09-10T09:59:35.655 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:59:35.655 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":32,"btime":"2024-09-10T09:57:40:888996+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3159289675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3159289675},{"type":"v1","addr":"172.21.15.77:6835","nonce":3159289675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1951962341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1951962341},{"type":"v1","addr":"172.21.15.77:6837","nonce":1951962341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:52:23.368252+0000","modified":"2024-09-10T09:57:40.695191+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24563,"mds_1":24524,"mds_2":24520,"mds_3":24502,"mds_4":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14487":{"gid":14487,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/4291121888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":4291121888},{"type":"v1","addr":"172.21.15.77:6839","nonce":4291121888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14499":{"gid":14499,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6841/445284194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":445284194},{"type":"v1","addr":"172.21.15.77:6841","nonce":445284194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1005054423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1005054423},{"type":"v1","addr":"172.21.15.134:6833","nonce":1005054423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6833/3716307744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":3716307744},{"type":"v1","addr":"172.21.15.120:6833","nonce":3716307744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/1319892544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1319892544},{"type":"v1","addr":"172.21.15.120:6835","nonce":1319892544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.134:6835/3293933699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3293933699},{"type":"v1","addr":"172.21.15.134:6835","nonce":3293933699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.134:6837/2461497091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2461497091},{"type":"v1","addr":"172.21.15.134:6837","nonce":2461497091}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":20,"addr":"172.21.15.134:6839/1965117680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1965117680},{"type":"v1","addr":"172.21.15.134:6839","nonce":1965117680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":20,"addr":"172.21.15.120:6837/3033280709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3033280709},{"type":"v1","addr":"172.21.15.120:6837","nonce":3033280709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.120:6839/1777710157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1777710157},{"type":"v1","addr":"172.21.15.120:6839","nonce":1777710157}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24563,"qdb_cluster":[24563,24524,24520,24502,24496]},"id":1}]} 2024-09-10T09:59:35.655 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 32 2024-09-10T09:59:35.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:35.656+0000 7fcb20e00640 1 -- 172.21.15.77:0/1550461173 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fcb000a2b80 msgr2=0x7fcb000a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:35.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:35.656+0000 7fcb20e00640 1 --2- 172.21.15.77:0/1550461173 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fcb000a2b80 0x7fcb000a5040 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c000c00 tx=0x7fcb0c005b50 comp rx=0 tx=0).stop 2024-09-10T09:59:35.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:35.656+0000 7fcb20e00640 1 -- 172.21.15.77:0/1550461173 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcb1c1049a0 msgr2=0x7fcb1c19ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:35.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:35.656+0000 7fcb20e00640 1 --2- 172.21.15.77:0/1550461173 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcb1c1049a0 0x7fcb1c19ab60 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fcb0400d950 tx=0x7fcb0400de10 comp rx=0 tx=0).stop 2024-09-10T09:59:35.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:35.656+0000 7fcb20e00640 1 -- 172.21.15.77:0/1550461173 shutdown_connections 2024-09-10T09:59:35.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:35.656+0000 7fcb20e00640 1 --2- 172.21.15.77:0/1550461173 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fcb000a2b80 0x7fcb000a5040 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:35.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:35.656+0000 7fcb20e00640 1 --2- 172.21.15.77:0/1550461173 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb1c109120 0x7fcb1c19eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:35.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:35.656+0000 7fcb20e00640 1 --2- 172.21.15.77:0/1550461173 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcb1c1049a0 0x7fcb1c19ab60 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:35.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:35.656+0000 7fcb20e00640 1 --2- 172.21.15.77:0/1550461173 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb1c103ff0 0x7fcb1c19a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:35.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:35.656+0000 7fcb20e00640 1 -- 172.21.15.77:0/1550461173 >> 172.21.15.77:0/1550461173 conn(0x7fcb1c0ffa90 msgr2=0x7fcb1c100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:35.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:35.656+0000 7fcb20e00640 1 -- 172.21.15.77:0/1550461173 shutdown_connections 2024-09-10T09:59:35.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:35.656+0000 7fcb20e00640 1 -- 172.21.15.77:0/1550461173 wait complete. 2024-09-10T09:59:36.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:59:35 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/1550461173' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:59:36.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:59:35 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/1550461173' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:59:36.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:59:35 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/1550461173' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:59:36.465 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24563 quiesce db --set-id 95a54fcc --release --await' 2024-09-10T09:59:36.465 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24563 quiesce db --set-id 95a54fcc --release --await 2024-09-10T09:59:36.832 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:59:37.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:59:36 smithi134 ceph-mon[35674]: pgmap v524: 129 pgs: 129 active+clean; 96 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-09-10T09:59:37.141 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:59:36 smithi077 ceph-mon[30319]: pgmap v524: 129 pgs: 129 active+clean; 96 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-09-10T09:59:37.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:59:36 smithi120 ceph-mon[34770]: pgmap v524: 129 pgs: 129 active+clean; 96 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-09-10T09:59:37.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.381+0000 7fbefa800640 1 -- 172.21.15.77:0/1003614116 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbef4108f60 msgr2=0x7fbef4110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:37.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.381+0000 7fbefa800640 1 --2- 172.21.15.77:0/1003614116 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbef4108f60 0x7fbef4110ae0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fbeec009ca0 tx=0x7fbeec02a8c0 comp rx=0 tx=0).stop 2024-09-10T09:59:37.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.381+0000 7fbefa800640 1 -- 172.21.15.77:0/1003614116 shutdown_connections 2024-09-10T09:59:37.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.381+0000 7fbefa800640 1 --2- 172.21.15.77:0/1003614116 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbef4108f60 0x7fbef4110ae0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:37.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.381+0000 7fbefa800640 1 --2- 172.21.15.77:0/1003614116 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbef41047e0 0x7fbef4108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:37.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.381+0000 7fbefa800640 1 --2- 172.21.15.77:0/1003614116 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbef4103e30 0x7fbef4104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:37.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.381+0000 7fbefa800640 1 -- 172.21.15.77:0/1003614116 >> 172.21.15.77:0/1003614116 conn(0x7fbef40ff8d0 msgr2=0x7fbef4101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:37.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.381+0000 7fbefa800640 1 -- 172.21.15.77:0/1003614116 shutdown_connections 2024-09-10T09:59:37.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.381+0000 7fbefa800640 1 -- 172.21.15.77:0/1003614116 wait complete. 2024-09-10T09:59:37.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.382+0000 7fbefa800640 1 Processor -- start 2024-09-10T09:59:37.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.382+0000 7fbefa800640 1 -- start start 2024-09-10T09:59:37.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.382+0000 7fbefa800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbef4103e30 0x7fbef419a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:37.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.382+0000 7fbefa800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbef41047e0 0x7fbef419a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:37.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.382+0000 7fbefa800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbef4108f60 0x7fbef419ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:37.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.382+0000 7fbefa800640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbef4114010 con 0x7fbef41047e0 2024-09-10T09:59:37.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.382+0000 7fbefa800640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbef4113e90 con 0x7fbef4103e30 2024-09-10T09:59:37.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.382+0000 7fbefa800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbef4114190 con 0x7fbef4108f60 2024-09-10T09:59:37.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.382+0000 7fbef8a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbef41047e0 0x7fbef419a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:37.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.383+0000 7fbef8a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbef41047e0 0x7fbef419a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:40906/0 (socket says 172.21.15.77:40906) 2024-09-10T09:59:37.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.383+0000 7fbef8a00640 1 -- 172.21.15.77:0/3702386652 learned_addr learned my addr 172.21.15.77:0/3702386652 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:59:37.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.383+0000 7fbef9400640 1 --2- 172.21.15.77:0/3702386652 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbef4103e30 0x7fbef419a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:37.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.383+0000 7fbef9e00640 1 --2- 172.21.15.77:0/3702386652 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbef4108f60 0x7fbef419ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:37.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.383+0000 7fbef8a00640 1 -- 172.21.15.77:0/3702386652 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbef4108f60 msgr2=0x7fbef419ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:37.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.383+0000 7fbef8a00640 1 --2- 172.21.15.77:0/3702386652 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbef4108f60 0x7fbef419ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:37.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.383+0000 7fbef8a00640 1 -- 172.21.15.77:0/3702386652 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbef4103e30 msgr2=0x7fbef419a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:37.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.383+0000 7fbef8a00640 1 --2- 172.21.15.77:0/3702386652 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbef4103e30 0x7fbef419a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:37.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.383+0000 7fbef8a00640 1 -- 172.21.15.77:0/3702386652 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbef419f3a0 con 0x7fbef41047e0 2024-09-10T09:59:37.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.383+0000 7fbef8a00640 1 --2- 172.21.15.77:0/3702386652 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbef41047e0 0x7fbef419a930 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fbee000d950 tx=0x7fbee000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:37.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.384+0000 7fbef2000640 1 -- 172.21.15.77:0/3702386652 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee0017030 con 0x7fbef41047e0 2024-09-10T09:59:37.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.384+0000 7fbefa800640 1 -- 172.21.15.77:0/3702386652 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbef419f690 con 0x7fbef41047e0 2024-09-10T09:59:37.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.384+0000 7fbef2000640 1 -- 172.21.15.77:0/3702386652 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbee0004020 con 0x7fbef41047e0 2024-09-10T09:59:37.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.384+0000 7fbef2000640 1 -- 172.21.15.77:0/3702386652 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee00047a0 con 0x7fbef41047e0 2024-09-10T09:59:37.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.384+0000 7fbefa800640 1 -- 172.21.15.77:0/3702386652 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbef41a6f70 con 0x7fbef41047e0 2024-09-10T09:59:37.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.387+0000 7fbefe840740 1 Processor -- start 2024-09-10T09:59:37.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.387+0000 7fbefe840740 1 -- start start 2024-09-10T09:59:37.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.387+0000 7fbefe840740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f52cbe520 0x560f52cbe900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:37.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.387+0000 7fbef2000640 1 -- 172.21.15.77:0/3702386652 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fbee00041d0 con 0x7fbef41047e0 2024-09-10T09:59:37.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.387+0000 7fbefe840740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560f52cbee40 0x560f52cc91c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:37.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.387+0000 7fbefe840740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560f52cc9700 0x560f52ccbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:37.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.387+0000 7fbefe840740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560f52c24650 con 0x560f52cbee40 2024-09-10T09:59:37.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.387+0000 7fbefe840740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x560f52bcbf10 con 0x560f52cc9700 2024-09-10T09:59:37.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.387+0000 7fbefe840740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560f52c20610 con 0x560f52cbe520 2024-09-10T09:59:37.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.387+0000 7fbef9400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560f52cbee40 0x560f52cc91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:37.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.388+0000 7fbef9e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f52cbe520 0x560f52cbe900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:37.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.388+0000 7fbef9400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560f52cbee40 0x560f52cc91c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:40912/0 (socket says 172.21.15.77:40912) 2024-09-10T09:59:37.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.388+0000 7fbef9400640 1 -- 172.21.15.77:0/1770313624 learned_addr learned my addr 172.21.15.77:0/1770313624 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:59:37.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.388+0000 7fbef8a00640 1 --2- 172.21.15.77:0/1770313624 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560f52cc9700 0x560f52ccbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:37.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.388+0000 7fbef9e00640 1 -- 172.21.15.77:0/1770313624 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560f52cc9700 msgr2=0x560f52ccbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:37.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.388+0000 7fbef9e00640 1 --2- 172.21.15.77:0/1770313624 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560f52cc9700 0x560f52ccbaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:37.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.388+0000 7fbef9e00640 1 -- 172.21.15.77:0/1770313624 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560f52cbee40 msgr2=0x560f52cc91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:37.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.388+0000 7fbef9e00640 1 --2- 172.21.15.77:0/1770313624 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560f52cbee40 0x560f52cc91c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:37.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.388+0000 7fbef9e00640 1 -- 172.21.15.77:0/1770313624 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f52933580 con 0x560f52cbe520 2024-09-10T09:59:37.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.388+0000 7fbef9400640 1 --2- 172.21.15.77:0/1770313624 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560f52cbee40 0x560f52cc91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:59:37.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.388+0000 7fbef9e00640 1 --2- 172.21.15.77:0/1770313624 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f52cbe520 0x560f52cbe900 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fbeec005e30 tx=0x7fbeec02f3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5805c486e53d44e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:37.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.389+0000 7fbef2000640 1 --2- 172.21.15.77:0/3702386652 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fbedc0a2b80 0x7fbedc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:37.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.389+0000 7fbee6600640 1 -- 172.21.15.77:0/1770313624 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeec003970 con 0x560f52cbe520 2024-09-10T09:59:37.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.389+0000 7fbee6600640 1 -- 172.21.15.77:0/1770313624 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbeec036030 con 0x560f52cbe520 2024-09-10T09:59:37.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.389+0000 7fbee6600640 1 -- 172.21.15.77:0/1770313624 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeec004330 con 0x560f52cbe520 2024-09-10T09:59:37.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.389+0000 7fbef2000640 1 -- 172.21.15.77:0/3702386652 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fbee00ced70 con 0x7fbef41047e0 2024-09-10T09:59:37.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.389+0000 7fbef9400640 1 --2- 172.21.15.77:0/3702386652 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fbedc0a2b80 0x7fbedc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:37.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.390+0000 7fbefe840740 1 -- 172.21.15.77:0/1770313624 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f52cbe520 msgr2=0x560f52cbe900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:37.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.390+0000 7fbefe840740 1 --2- 172.21.15.77:0/1770313624 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f52cbe520 0x560f52cbe900 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fbeec005e30 tx=0x7fbeec02f3c0 comp rx=0 tx=0).stop 2024-09-10T09:59:37.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.390+0000 7fbef9400640 1 --2- 172.21.15.77:0/3702386652 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fbedc0a2b80 0x7fbedc0a5040 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x560f52cbb910 tx=0x7fbee8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:37.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.390+0000 7fbefe840740 1 -- 172.21.15.77:0/1770313624 shutdown_connections 2024-09-10T09:59:37.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.390+0000 7fbefe840740 1 --2- 172.21.15.77:0/1770313624 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560f52cc9700 0x560f52ccbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:37.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.390+0000 7fbefe840740 1 --2- 172.21.15.77:0/1770313624 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560f52cbee40 0x560f52cc91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:37.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.390+0000 7fbefe840740 1 --2- 172.21.15.77:0/1770313624 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f52cbe520 0x560f52cbe900 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:37.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.390+0000 7fbefe840740 1 -- 172.21.15.77:0/1770313624 >> 172.21.15.77:0/1770313624 conn(0x560f52cbaab0 msgr2=0x560f52cbae90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:37.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.390+0000 7fbefe840740 1 -- 172.21.15.77:0/1770313624 shutdown_connections 2024-09-10T09:59:37.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.390+0000 7fbefe840740 1 -- 172.21.15.77:0/1770313624 wait complete. 2024-09-10T09:59:37.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.390+0000 7fbefe840740 1 -- start start 2024-09-10T09:59:37.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.390+0000 7fbefe840740 1 Processor -- start 2024-09-10T09:59:37.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.391+0000 7fbefe840740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560f52cbe520 0x560f52cc65b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:37.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.391+0000 7fbefe840740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f52cbee40 0x560f52cc6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:37.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.391+0000 7fbefe840740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560f52cc9700 0x560f52cc7030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:37.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.391+0000 7fbefe840740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560f52ccc150 con 0x560f52cbe520 2024-09-10T09:59:37.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.391+0000 7fbefe840740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x560f52bdf350 con 0x560f52cc9700 2024-09-10T09:59:37.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.391+0000 7fbefe840740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560f52ccc2d0 con 0x560f52cbee40 2024-09-10T09:59:37.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.391+0000 7fbefe840740 10 client.0 _ll_register_callbacks cb 0x560f52c02290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:59:37.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.391+0000 7fbef9400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560f52cbe520 0x560f52cc65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:37.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.391+0000 7fbef9400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560f52cbe520 0x560f52cc65b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:40916/0 (socket says 172.21.15.77:40916) 2024-09-10T09:59:37.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.391+0000 7fbef9400640 1 -- 172.21.15.77:0/2060595726 learned_addr learned my addr 172.21.15.77:0/2060595726 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:59:37.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.391+0000 7fbef8a00640 1 --2- 172.21.15.77:0/2060595726 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f52cbee40 0x560f52cc6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:37.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.391+0000 7fbef9e00640 1 --2- 172.21.15.77:0/2060595726 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560f52cc9700 0x560f52cc7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:37.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.392+0000 7fbef9400640 1 -- 172.21.15.77:0/2060595726 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f52cbee40 msgr2=0x560f52cc6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:37.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.392+0000 7fbef9400640 1 --2- 172.21.15.77:0/2060595726 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f52cbee40 0x560f52cc6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:37.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.392+0000 7fbef9400640 1 -- 172.21.15.77:0/2060595726 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560f52cc9700 msgr2=0x560f52cc7030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:37.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.392+0000 7fbef9400640 1 --2- 172.21.15.77:0/2060595726 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560f52cc9700 0x560f52cc7030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:37.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.392+0000 7fbef9400640 1 -- 172.21.15.77:0/2060595726 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f529fd960 con 0x560f52cbe520 2024-09-10T09:59:37.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.392+0000 7fbef9400640 1 --2- 172.21.15.77:0/2060595726 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560f52cbe520 0x560f52cc65b0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fbee800fa50 tx=0x7fbee8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f2dd8dad93b1eb7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:37.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.392+0000 7fbedaa00640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-10T09:59:37.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.392+0000 7fbedaa00640 1 -- 172.21.15.77:0/2060595726 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee802d060 con 0x560f52cbe520 2024-09-10T09:59:37.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.392+0000 7fbefe840740 1 -- 172.21.15.77:0/2060595726 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbeec036030 con 0x560f52cbe520 2024-09-10T09:59:37.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.392+0000 7fbedaa00640 1 -- 172.21.15.77:0/2060595726 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fbee8018030 con 0x560f52cbe520 2024-09-10T09:59:37.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.393+0000 7fbedaa00640 1 -- 172.21.15.77:0/2060595726 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fbee8034050 con 0x560f52cbe520 2024-09-10T09:59:37.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.393+0000 7fbefe840740 10 client.16332 fetch_fsmap learned FSMap version 32 2024-09-10T09:59:37.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.393+0000 7fbefe840740 1 -- 172.21.15.77:0/2060595726 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560f52cd7dd0 con 0x560f52cbe520 2024-09-10T09:59:37.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.393+0000 7fbedaa00640 1 -- 172.21.15.77:0/2060595726 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbeec036030 con 0x560f52cbe520 2024-09-10T09:59:37.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.393+0000 7fbedaa00640 1 -- 172.21.15.77:0/2060595726 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fbee801c600 con 0x560f52cbe520 2024-09-10T09:59:37.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.393+0000 7fbefe840740 10 client.16332 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T09:59:37.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.393+0000 7fbefe840740 10 client.16332 resolve_mds: validated gid 24563 aka daemon mds.b 2024-09-10T09:59:37.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.393+0000 7fbefe840740 20 client.16332 populate_metadata read hostname 'smithi077' 2024-09-10T09:59:37.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.393+0000 7fbefe840740 1 --2- 172.21.15.77:0/2060595726 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x560f52cd9290 0x560f52cdb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:37.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.393+0000 7fbefe840740 4 client.16332 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:59:37.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.393+0000 7fbefe840740 1 -- 172.21.15.77:0/2060595726 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560f52c31bc0 con 0x560f52cd9290 2024-09-10T09:59:37.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.394+0000 7fbef8a00640 1 --2- 172.21.15.77:0/2060595726 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x560f52cd9290 0x560f52cdb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:37.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.394+0000 7fbef8a00640 1 --2- 172.21.15.77:0/2060595726 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x560f52cd9290 0x560f52cdb6c0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6186118fcbbf2e42 server_cookie=a72b780190fc2d99 in_seq=0 out_seq=0 2024-09-10T09:59:37.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.394+0000 7fbedaa00640 10 client.16332 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T09:59:37.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.395+0000 7fbedaa00640 1 -- 172.21.15.77:0/2060595726 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560f52c31bc0 con 0x560f52cd9290 2024-09-10T09:59:37.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.395+0000 7fbedaa00640 10 client.16332 handle_command_reply: tid=0 2024-09-10T09:59:37.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.395+0000 7fbefe840740 1 client.16332 shutdown 2024-09-10T09:59:37.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.395+0000 7fbefe840740 1 -- 172.21.15.77:0/2060595726 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560f52cbe520 msgr2=0x560f52cc65b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:37.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.395+0000 7fbefe840740 1 --2- 172.21.15.77:0/2060595726 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560f52cbe520 0x560f52cc65b0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fbee800fa50 tx=0x7fbee8004730 comp rx=0 tx=0).stop 2024-09-10T09:59:37.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.395+0000 7fbefe840740 1 -- 172.21.15.77:0/2060595726 shutdown_connections 2024-09-10T09:59:37.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.395+0000 7fbefe840740 1 --2- 172.21.15.77:0/2060595726 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x560f52cd9290 0x560f52cdb6c0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:37.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.395+0000 7fbefe840740 1 --2- 172.21.15.77:0/2060595726 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560f52cc9700 0x560f52cc7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:37.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.395+0000 7fbefe840740 1 --2- 172.21.15.77:0/2060595726 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f52cbee40 0x560f52cc6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:37.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.395+0000 7fbedaa00640 0 client.16332 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T09:59:37.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.395+0000 7fbefe840740 1 --2- 172.21.15.77:0/2060595726 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560f52cbe520 0x560f52cc65b0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:37.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.395+0000 7fbefe840740 1 -- 172.21.15.77:0/2060595726 >> 172.21.15.77:0/2060595726 conn(0x560f52cbaab0 msgr2=0x560f52cb9cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:37.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.395+0000 7fbefe840740 1 -- 172.21.15.77:0/2060595726 shutdown_connections 2024-09-10T09:59:37.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.396+0000 7fbefe840740 1 -- 172.21.15.77:0/2060595726 wait complete. 2024-09-10T09:59:37.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.396+0000 7fbefe840740 20 client.16332 trim_cache size 0 max 16384 2024-09-10T09:59:37.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.414+0000 7fbefe840740 1 Processor -- start 2024-09-10T09:59:37.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.415+0000 7fbefe840740 1 -- start start 2024-09-10T09:59:37.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.415+0000 7fbefe840740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560f52cbe520 0x560f52cc54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:37.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.415+0000 7fbefe840740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560f52cbee40 0x560f52cc0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:37.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.415+0000 7fbefe840740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f52cc9700 0x560f52ce0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:37.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.415+0000 7fbefe840740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbef41b0db0 con 0x560f52cbe520 2024-09-10T09:59:37.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.415+0000 7fbefe840740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbef41b29b0 con 0x560f52cbee40 2024-09-10T09:59:37.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.415+0000 7fbefe840740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbef41b1480 con 0x560f52cc9700 2024-09-10T09:59:37.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.415+0000 7fbef9400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560f52cbe520 0x560f52cc54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:37.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.415+0000 7fbef9400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560f52cbe520 0x560f52cc54f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:40924/0 (socket says 172.21.15.77:40924) 2024-09-10T09:59:37.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.415+0000 7fbef9400640 1 -- 172.21.15.77:0/265984275 learned_addr learned my addr 172.21.15.77:0/265984275 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:59:37.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.415+0000 7fbef8a00640 1 --2- 172.21.15.77:0/265984275 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560f52cbee40 0x560f52cc0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:37.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.415+0000 7fbef9e00640 1 --2- 172.21.15.77:0/265984275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f52cc9700 0x560f52ce0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:37.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.416+0000 7fbef9400640 1 -- 172.21.15.77:0/265984275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f52cc9700 msgr2=0x560f52ce0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:37.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.416+0000 7fbef9400640 1 --2- 172.21.15.77:0/265984275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f52cc9700 0x560f52ce0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:37.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.416+0000 7fbef9400640 1 -- 172.21.15.77:0/265984275 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560f52cbee40 msgr2=0x560f52cc0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:37.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.416+0000 7fbef9400640 1 --2- 172.21.15.77:0/265984275 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560f52cbee40 0x560f52cc0f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:37.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.416+0000 7fbef9400640 1 -- 172.21.15.77:0/265984275 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f52cd7be0 con 0x560f52cbe520 2024-09-10T09:59:37.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.416+0000 7fbef9400640 1 --2- 172.21.15.77:0/265984275 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560f52cbe520 0x560f52cc54f0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fbee8005c20 tx=0x7fbee8012e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b10f991fa9e0246a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:37.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.416+0000 7fbee5c00640 1 -- 172.21.15.77:0/265984275 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee801cc40 con 0x560f52cbe520 2024-09-10T09:59:37.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.416+0000 7fbee5c00640 1 -- 172.21.15.77:0/265984275 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbee801cde0 con 0x560f52cbe520 2024-09-10T09:59:37.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.416+0000 7fbee5c00640 1 -- 172.21.15.77:0/265984275 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee8028470 con 0x560f52cbe520 2024-09-10T09:59:37.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.417+0000 7fbefe840740 1 -- 172.21.15.77:0/265984275 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560f52cbe520 msgr2=0x560f52cc54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:37.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.417+0000 7fbefe840740 1 --2- 172.21.15.77:0/265984275 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560f52cbe520 0x560f52cc54f0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fbee8005c20 tx=0x7fbee8012e90 comp rx=0 tx=0).stop 2024-09-10T09:59:37.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.417+0000 7fbefe840740 1 -- 172.21.15.77:0/265984275 shutdown_connections 2024-09-10T09:59:37.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.417+0000 7fbefe840740 1 --2- 172.21.15.77:0/265984275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f52cc9700 0x560f52ce0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:37.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.417+0000 7fbefe840740 1 --2- 172.21.15.77:0/265984275 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560f52cbee40 0x560f52cc0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:37.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.417+0000 7fbefe840740 1 --2- 172.21.15.77:0/265984275 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560f52cbe520 0x560f52cc54f0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:37.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.417+0000 7fbefe840740 1 -- 172.21.15.77:0/265984275 >> 172.21.15.77:0/265984275 conn(0x560f52cbaab0 msgr2=0x560f52cb9cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:37.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.417+0000 7fbefe840740 1 -- 172.21.15.77:0/265984275 shutdown_connections 2024-09-10T09:59:37.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.417+0000 7fbefe840740 1 -- 172.21.15.77:0/265984275 wait complete. 2024-09-10T09:59:37.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.418+0000 7fbefe840740 1 -- start start 2024-09-10T09:59:37.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.418+0000 7fbefe840740 1 Processor -- start 2024-09-10T09:59:37.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.418+0000 7fbefe840740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f52cbe520 0x560f52ccd260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:37.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.418+0000 7fbefe840740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560f52cbee40 0x560f52ccd7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:37.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.418+0000 7fbefe840740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560f52cc9700 0x560f52ccdce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:37.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.418+0000 7fbefe840740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbef41b2220 con 0x560f52cc9700 2024-09-10T09:59:37.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.418+0000 7fbefe840740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbef41b1b50 con 0x560f52cbee40 2024-09-10T09:59:37.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.418+0000 7fbefe840740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbef41aed90 con 0x560f52cbe520 2024-09-10T09:59:37.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.418+0000 7fbefe840740 10 client.0 _ll_register_callbacks cb 0x7fbee800f910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:59:37.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.419+0000 7fbef8a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560f52cbee40 0x560f52ccd7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:37.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.419+0000 7fbef8a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560f52cbee40 0x560f52ccd7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:42182/0 (socket says 172.21.15.77:42182) 2024-09-10T09:59:37.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.419+0000 7fbef8a00640 1 -- 172.21.15.77:0/2645083134 learned_addr learned my addr 172.21.15.77:0/2645083134 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:59:37.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.419+0000 7fbef9e00640 1 --2- 172.21.15.77:0/2645083134 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560f52cc9700 0x560f52ccdce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:37.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.419+0000 7fbef9400640 1 --2- 172.21.15.77:0/2645083134 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f52cbe520 0x560f52ccd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:37.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.419+0000 7fbef8a00640 1 -- 172.21.15.77:0/2645083134 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f52cbe520 msgr2=0x560f52ccd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:37.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.419+0000 7fbef8a00640 1 --2- 172.21.15.77:0/2645083134 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f52cbe520 0x560f52ccd260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:37.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.419+0000 7fbef8a00640 1 -- 172.21.15.77:0/2645083134 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560f52cc9700 msgr2=0x560f52ccdce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:37.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.419+0000 7fbef8a00640 1 --2- 172.21.15.77:0/2645083134 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560f52cc9700 0x560f52ccdce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:37.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.419+0000 7fbef8a00640 1 -- 172.21.15.77:0/2645083134 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f52d42f60 con 0x560f52cbee40 2024-09-10T09:59:37.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.419+0000 7fbef8a00640 1 --2- 172.21.15.77:0/2645083134 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560f52cbee40 0x560f52ccd7a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fbee00975e0 tx=0x7fbee00d27b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dbfba8b15e1509f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:37.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.419+0000 7fbedaa00640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T09:59:37.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.420+0000 7fbedaa00640 1 -- 172.21.15.77:0/2645083134 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee009acc0 con 0x560f52cbee40 2024-09-10T09:59:37.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.420+0000 7fbedaa00640 1 -- 172.21.15.77:0/2645083134 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fbee00977f0 con 0x560f52cbee40 2024-09-10T09:59:37.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.420+0000 7fbefe840740 1 -- 172.21.15.77:0/2645083134 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbee801cde0 con 0x560f52cbee40 2024-09-10T09:59:37.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.420+0000 7fbedaa00640 1 -- 172.21.15.77:0/2645083134 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fbee009aae0 con 0x560f52cbee40 2024-09-10T09:59:37.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.420+0000 7fbefe840740 10 client.26420 fetch_fsmap learned FSMap version 32 2024-09-10T09:59:37.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.420+0000 7fbefe840740 1 -- 172.21.15.77:0/2645083134 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560f52d442f0 con 0x560f52cbee40 2024-09-10T09:59:37.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.420+0000 7fbedaa00640 1 -- 172.21.15.77:0/2645083134 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbee801cde0 con 0x560f52cbee40 2024-09-10T09:59:37.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.421+0000 7fbedaa00640 1 -- 172.21.15.77:0/2645083134 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fbee009f170 con 0x560f52cbee40 2024-09-10T09:59:37.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.421+0000 7fbefe840740 10 client.26420 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T09:59:37.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.421+0000 7fbefe840740 10 client.26420 resolve_mds: validated gid 24563 aka daemon mds.b 2024-09-10T09:59:37.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.421+0000 7fbefe840740 20 client.26420 populate_metadata read hostname 'smithi077' 2024-09-10T09:59:37.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.421+0000 7fbefe840740 1 --2- 172.21.15.77:0/2645083134 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x560f52d44ae0 0x560f52d46f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:37.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.421+0000 7fbefe840740 4 client.26420 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "95a54fcc", "release": true, "await": true}] 2024-09-10T09:59:37.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.421+0000 7fbefe840740 1 -- 172.21.15.77:0/2645083134 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "quiesce db", "set_id": "95a54fcc", "release": true, "await": true}) -- 0x560f52c2c0d0 con 0x560f52d44ae0 2024-09-10T09:59:37.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.421+0000 7fbef9400640 1 --2- 172.21.15.77:0/2645083134 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x560f52d44ae0 0x560f52d46f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:37.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.422+0000 7fbef9400640 1 --2- 172.21.15.77:0/2645083134 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x560f52d44ae0 0x560f52d46f00 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc31c392dd7a9651 server_cookie=befc6756ddda353e in_seq=0 out_seq=0 2024-09-10T09:59:37.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.422+0000 7fbedaa00640 10 client.26420 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T09:59:37.519 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.518+0000 7fbedaa00640 1 -- 172.21.15.77:0/2645083134 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x560f52c2c0d0 con 0x560f52d44ae0 2024-09-10T09:59:37.519 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.518+0000 7fbedaa00640 10 client.26420 handle_command_reply: tid=0 2024-09-10T09:59:37.519 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.518+0000 7fbefe840740 1 client.26420 shutdown 2024-09-10T09:59:37.519 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.518+0000 7fbefe840740 1 -- 172.21.15.77:0/2645083134 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560f52cbee40 msgr2=0x560f52ccd7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:37.519 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.518+0000 7fbefe840740 1 --2- 172.21.15.77:0/2645083134 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560f52cbee40 0x560f52ccd7a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fbee00975e0 tx=0x7fbee00d27b0 comp rx=0 tx=0).stop 2024-09-10T09:59:37.519 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.518+0000 7fbefe840740 1 -- 172.21.15.77:0/2645083134 shutdown_connections 2024-09-10T09:59:37.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.518+0000 7fbefe840740 1 --2- 172.21.15.77:0/2645083134 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x560f52d44ae0 0x560f52d46f00 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:37.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.518+0000 7fbefe840740 1 --2- 172.21.15.77:0/2645083134 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560f52cc9700 0x560f52ccdce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:37.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.518+0000 7fbedaa00640 0 client.26420 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T09:59:37.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.518+0000 7fbefe840740 1 --2- 172.21.15.77:0/2645083134 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560f52cbee40 0x560f52ccd7a0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:37.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.518+0000 7fbefe840740 1 --2- 172.21.15.77:0/2645083134 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f52cbe520 0x560f52ccd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:37.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.518+0000 7fbefe840740 1 -- 172.21.15.77:0/2645083134 >> 172.21.15.77:0/2645083134 conn(0x560f52cbaab0 msgr2=0x560f52cc7b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:37.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.518+0000 7fbefe840740 1 -- 172.21.15.77:0/2645083134 shutdown_connections 2024-09-10T09:59:37.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.518+0000 7fbefe840740 1 -- 172.21.15.77:0/2645083134 wait complete. 2024-09-10T09:59:37.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.518+0000 7fbefe840740 20 client.26420 trim_cache size 0 max 16384 2024-09-10T09:59:37.521 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-10T09:59:37.521 INFO:teuthology.orchestra.run.smithi077.stdout: "epoch": 32, 2024-09-10T09:59:37.521 INFO:teuthology.orchestra.run.smithi077.stdout: "leader": 24563, 2024-09-10T09:59:37.521 INFO:teuthology.orchestra.run.smithi077.stdout: "set_version": 12, 2024-09-10T09:59:37.521 INFO:teuthology.orchestra.run.smithi077.stdout: "sets": { 2024-09-10T09:59:37.521 INFO:teuthology.orchestra.run.smithi077.stdout: "95a54fcc": { 2024-09-10T09:59:37.521 INFO:teuthology.orchestra.run.smithi077.stdout: "version": 12, 2024-09-10T09:59:37.521 INFO:teuthology.orchestra.run.smithi077.stdout: "age_ref": 0.0, 2024-09-10T09:59:37.521 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-10T09:59:37.521 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RELEASED", 2024-09-10T09:59:37.521 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-10T09:59:37.522 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T09:59:37.522 INFO:teuthology.orchestra.run.smithi077.stdout: "timeout": 90.0, 2024-09-10T09:59:37.522 INFO:teuthology.orchestra.run.smithi077.stdout: "expiration": 142.8, 2024-09-10T09:59:37.522 INFO:teuthology.orchestra.run.smithi077.stdout: "members": { 2024-09-10T09:59:37.522 INFO:teuthology.orchestra.run.smithi077.stdout: "file:/": { 2024-09-10T09:59:37.522 INFO:teuthology.orchestra.run.smithi077.stdout: "excluded": false, 2024-09-10T09:59:37.522 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-10T09:59:37.522 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RELEASED", 2024-09-10T09:59:37.522 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-10T09:59:37.522 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T09:59:37.522 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T09:59:37.522 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T09:59:37.522 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T09:59:37.522 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T09:59:37.522 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-10T09:59:37.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.519+0000 7fbefa800640 1 -- 172.21.15.77:0/3702386652 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fbedc0a2b80 msgr2=0x7fbedc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:37.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.519+0000 7fbefa800640 1 --2- 172.21.15.77:0/3702386652 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fbedc0a2b80 0x7fbedc0a5040 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x560f52cbb910 tx=0x7fbee8005f90 comp rx=0 tx=0).stop 2024-09-10T09:59:37.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.519+0000 7fbefa800640 1 -- 172.21.15.77:0/3702386652 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbef41047e0 msgr2=0x7fbef419a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:37.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.519+0000 7fbefa800640 1 --2- 172.21.15.77:0/3702386652 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbef41047e0 0x7fbef419a930 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fbee000d950 tx=0x7fbee000de10 comp rx=0 tx=0).stop 2024-09-10T09:59:37.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.519+0000 7fbefa800640 1 -- 172.21.15.77:0/3702386652 shutdown_connections 2024-09-10T09:59:37.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.519+0000 7fbefa800640 1 --2- 172.21.15.77:0/3702386652 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fbedc0a2b80 0x7fbedc0a5040 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:37.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.519+0000 7fbefa800640 1 --2- 172.21.15.77:0/3702386652 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbef4108f60 0x7fbef419ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:37.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.519+0000 7fbefa800640 1 --2- 172.21.15.77:0/3702386652 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbef41047e0 0x7fbef419a930 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:37.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.519+0000 7fbefa800640 1 --2- 172.21.15.77:0/3702386652 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbef4103e30 0x7fbef419a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:37.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.519+0000 7fbefa800640 1 -- 172.21.15.77:0/3702386652 >> 172.21.15.77:0/3702386652 conn(0x7fbef40ff8d0 msgr2=0x7fbef4101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:37.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.519+0000 7fbefa800640 1 -- 172.21.15.77:0/3702386652 shutdown_connections 2024-09-10T09:59:37.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:37.519+0000 7fbefa800640 1 -- 172.21.15.77:0/3702386652 wait complete. 2024-09-10T09:59:38.761 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '95a54fcc', total seconds elapsed: 27.0 2024-09-10T09:59:38.761 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.75 2024-09-10T09:59:38.761 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:59:39.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:59:38 smithi134 ceph-mon[35674]: pgmap v525: 129 pgs: 129 active+clean; 96 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-09-10T09:59:39.148 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:59:39.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:59:38 smithi120 ceph-mon[34770]: pgmap v525: 129 pgs: 129 active+clean; 96 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-09-10T09:59:39.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:59:38 smithi077 ceph-mon[30319]: pgmap v525: 129 pgs: 129 active+clean; 96 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-09-10T09:59:39.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:39.689+0000 7fc540600640 1 -- 172.21.15.77:0/562802064 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc5381049c0 msgr2=0x7fc538108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:39.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:39.689+0000 7fc540600640 1 --2- 172.21.15.77:0/562802064 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc5381049c0 0x7fc538108a10 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc524009900 tx=0x7fc52402a3f0 comp rx=0 tx=0).stop 2024-09-10T09:59:39.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:39.690+0000 7fc540600640 1 -- 172.21.15.77:0/562802064 shutdown_connections 2024-09-10T09:59:39.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:39.690+0000 7fc540600640 1 --2- 172.21.15.77:0/562802064 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc538109140 0x7fc538110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:39.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:39.690+0000 7fc540600640 1 --2- 172.21.15.77:0/562802064 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc5381049c0 0x7fc538108a10 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:39.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:39.690+0000 7fc540600640 1 --2- 172.21.15.77:0/562802064 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc538104010 0x7fc5381043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:39.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:39.690+0000 7fc540600640 1 -- 172.21.15.77:0/562802064 >> 172.21.15.77:0/562802064 conn(0x7fc5380ffa50 msgr2=0x7fc538101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:39.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:39.690+0000 7fc540600640 1 -- 172.21.15.77:0/562802064 shutdown_connections 2024-09-10T09:59:39.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:39.690+0000 7fc540600640 1 -- 172.21.15.77:0/562802064 wait complete. 2024-09-10T09:59:39.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:39.691+0000 7fc540600640 1 Processor -- start 2024-09-10T09:59:39.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:39.691+0000 7fc540600640 1 -- start start 2024-09-10T09:59:39.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:39.692+0000 7fc540600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc538104010 0x7fc5381a03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:39.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:39.692+0000 7fc540600640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc5381049c0 0x7fc5381a0920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:39.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:39.692+0000 7fc540600640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc538109140 0x7fc53819a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:39.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:39.692+0000 7fc540600640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc538114190 con 0x7fc538109140 2024-09-10T09:59:39.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:39.692+0000 7fc540600640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc538114010 con 0x7fc5381049c0 2024-09-10T09:59:39.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:39.692+0000 7fc540600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc538114310 con 0x7fc538104010 2024-09-10T09:59:39.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:39.692+0000 7fc53dc00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc538104010 0x7fc5381a03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:39.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:39.692+0000 7fc53dc00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc538104010 0x7fc5381a03e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:43686/0 (socket says 172.21.15.77:43686) 2024-09-10T09:59:39.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:39.692+0000 7fc53dc00640 1 -- 172.21.15.77:0/167247158 learned_addr learned my addr 172.21.15.77:0/167247158 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:59:39.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:39.692+0000 7fc53d200640 1 --2- 172.21.15.77:0/167247158 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc5381049c0 0x7fc5381a0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:39.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:39.692+0000 7fc53e600640 1 --2- 172.21.15.77:0/167247158 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc538109140 0x7fc53819a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:39.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:39.692+0000 7fc53dc00640 1 -- 172.21.15.77:0/167247158 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc5381049c0 msgr2=0x7fc5381a0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:39.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:39.692+0000 7fc53dc00640 1 --2- 172.21.15.77:0/167247158 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc5381049c0 0x7fc5381a0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:39.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:39.692+0000 7fc53dc00640 1 -- 172.21.15.77:0/167247158 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc538109140 msgr2=0x7fc53819a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:39.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:39.692+0000 7fc53dc00640 1 --2- 172.21.15.77:0/167247158 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc538109140 0x7fc53819a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:39.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:39.692+0000 7fc53dc00640 1 -- 172.21.15.77:0/167247158 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc53819acd0 con 0x7fc538104010 2024-09-10T09:59:39.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:39.693+0000 7fc53dc00640 1 --2- 172.21.15.77:0/167247158 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc538104010 0x7fc5381a03e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc52c00e9f0 tx=0x7fc52c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:39.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:39.693+0000 7fc536000640 1 -- 172.21.15.77:0/167247158 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc52c00cd80 con 0x7fc538104010 2024-09-10T09:59:39.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:39.693+0000 7fc540600640 1 -- 172.21.15.77:0/167247158 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc53819afc0 con 0x7fc538104010 2024-09-10T09:59:39.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:39.693+0000 7fc536000640 1 -- 172.21.15.77:0/167247158 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc52c010910 con 0x7fc538104010 2024-09-10T09:59:39.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:39.693+0000 7fc540600640 1 -- 172.21.15.77:0/167247158 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5381a72b0 con 0x7fc538104010 2024-09-10T09:59:39.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:39.693+0000 7fc536000640 1 -- 172.21.15.77:0/167247158 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc52c0188f0 con 0x7fc538104010 2024-09-10T09:59:39.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:39.694+0000 7fc540600640 1 -- 172.21.15.77:0/167247158 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5381043f0 con 0x7fc538104010 2024-09-10T09:59:39.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:39.696+0000 7fc536000640 1 -- 172.21.15.77:0/167247158 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc52c010ac0 con 0x7fc538104010 2024-09-10T09:59:39.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:39.697+0000 7fc536000640 1 --2- 172.21.15.77:0/167247158 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fc5200a2c00 0x7fc5200a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:39.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:39.698+0000 7fc536000640 1 -- 172.21.15.77:0/167247158 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fc52c0cfd80 con 0x7fc538104010 2024-09-10T09:59:39.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:39.698+0000 7fc53d200640 1 --2- 172.21.15.77:0/167247158 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fc5200a2c00 0x7fc5200a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:39.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:39.698+0000 7fc53d200640 1 --2- 172.21.15.77:0/167247158 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fc5200a2c00 0x7fc5200a50c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fc524000c00 tx=0x7fc524005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:39.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:39.702+0000 7fc536000640 1 -- 172.21.15.77:0/167247158 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc52c09c890 con 0x7fc538104010 2024-09-10T09:59:39.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:39.925+0000 7fc540600640 1 -- 172.21.15.77:0/167247158 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc53810c380 con 0x7fc538104010 2024-09-10T09:59:39.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:39.926+0000 7fc536000640 1 -- 172.21.15.77:0/167247158 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11996 (secure 0 0 0) 0x7fc52c09c190 con 0x7fc538104010 2024-09-10T09:59:39.928 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T09:59:39.928 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":32,"btime":"2024-09-10T09:57:40:888996+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3159289675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3159289675},{"type":"v1","addr":"172.21.15.77:6835","nonce":3159289675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1951962341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1951962341},{"type":"v1","addr":"172.21.15.77:6837","nonce":1951962341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:52:23.368252+0000","modified":"2024-09-10T09:57:40.695191+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24563,"mds_1":24524,"mds_2":24520,"mds_3":24502,"mds_4":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14487":{"gid":14487,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/4291121888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":4291121888},{"type":"v1","addr":"172.21.15.77:6839","nonce":4291121888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14499":{"gid":14499,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6841/445284194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":445284194},{"type":"v1","addr":"172.21.15.77:6841","nonce":445284194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1005054423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1005054423},{"type":"v1","addr":"172.21.15.134:6833","nonce":1005054423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6833/3716307744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":3716307744},{"type":"v1","addr":"172.21.15.120:6833","nonce":3716307744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/1319892544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1319892544},{"type":"v1","addr":"172.21.15.120:6835","nonce":1319892544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.134:6835/3293933699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3293933699},{"type":"v1","addr":"172.21.15.134:6835","nonce":3293933699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.134:6837/2461497091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2461497091},{"type":"v1","addr":"172.21.15.134:6837","nonce":2461497091}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":20,"addr":"172.21.15.134:6839/1965117680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1965117680},{"type":"v1","addr":"172.21.15.134:6839","nonce":1965117680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":20,"addr":"172.21.15.120:6837/3033280709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3033280709},{"type":"v1","addr":"172.21.15.120:6837","nonce":3033280709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.120:6839/1777710157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1777710157},{"type":"v1","addr":"172.21.15.120:6839","nonce":1777710157}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24563,"qdb_cluster":[24563,24524,24520,24502,24496]},"id":1}]} 2024-09-10T09:59:39.928 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 32 2024-09-10T09:59:39.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:39.928+0000 7fc540600640 1 -- 172.21.15.77:0/167247158 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fc5200a2c00 msgr2=0x7fc5200a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:39.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:39.928+0000 7fc540600640 1 --2- 172.21.15.77:0/167247158 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fc5200a2c00 0x7fc5200a50c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fc524000c00 tx=0x7fc524005b50 comp rx=0 tx=0).stop 2024-09-10T09:59:39.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:39.928+0000 7fc540600640 1 -- 172.21.15.77:0/167247158 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc538104010 msgr2=0x7fc5381a03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:39.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:39.928+0000 7fc540600640 1 --2- 172.21.15.77:0/167247158 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc538104010 0x7fc5381a03e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc52c00e9f0 tx=0x7fc52c00eeb0 comp rx=0 tx=0).stop 2024-09-10T09:59:39.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:39.929+0000 7fc540600640 1 -- 172.21.15.77:0/167247158 shutdown_connections 2024-09-10T09:59:39.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:39.929+0000 7fc540600640 1 --2- 172.21.15.77:0/167247158 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fc5200a2c00 0x7fc5200a50c0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:39.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:39.929+0000 7fc540600640 1 --2- 172.21.15.77:0/167247158 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc538109140 0x7fc53819a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:39.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:39.929+0000 7fc540600640 1 --2- 172.21.15.77:0/167247158 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc5381049c0 0x7fc5381a0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:39.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:39.929+0000 7fc540600640 1 --2- 172.21.15.77:0/167247158 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc538104010 0x7fc5381a03e0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:39.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:39.929+0000 7fc540600640 1 -- 172.21.15.77:0/167247158 >> 172.21.15.77:0/167247158 conn(0x7fc5380ffa50 msgr2=0x7fc538100e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:39.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:39.929+0000 7fc540600640 1 -- 172.21.15.77:0/167247158 shutdown_connections 2024-09-10T09:59:39.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:39.929+0000 7fc540600640 1 -- 172.21.15.77:0/167247158 wait complete. 2024-09-10T09:59:40.781 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24563 quiesce db / --timeout 90 --expiration 142.75 --await-for 45.0' 2024-09-10T09:59:40.781 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24563 quiesce db / --timeout 90 --expiration 142.75 --await-for 45.0 2024-09-10T09:59:41.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:59:40 smithi134 ceph-mon[35674]: pgmap v526: 129 pgs: 129 active+clean; 96 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 212 KiB/s wr, 49 op/s 2024-09-10T09:59:41.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:59:40 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/167247158' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:59:41.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:59:40 smithi134 ceph-mon[35674]: scrub summary: idle 2024-09-10T09:59:41.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:59:40 smithi134 ceph-mon[35674]: scrub summary: idle 2024-09-10T09:59:41.163 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:59:41.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:59:40 smithi120 ceph-mon[34770]: pgmap v526: 129 pgs: 129 active+clean; 96 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 212 KiB/s wr, 49 op/s 2024-09-10T09:59:41.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:59:40 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/167247158' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:59:41.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:59:40 smithi120 ceph-mon[34770]: scrub summary: idle 2024-09-10T09:59:41.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:59:40 smithi120 ceph-mon[34770]: scrub summary: idle 2024-09-10T09:59:41.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:59:40 smithi077 ceph-mon[30319]: pgmap v526: 129 pgs: 129 active+clean; 96 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 212 KiB/s wr, 49 op/s 2024-09-10T09:59:41.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:59:40 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/167247158' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:59:41.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:59:40 smithi077 ceph-mon[30319]: scrub summary: idle 2024-09-10T09:59:41.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:59:40 smithi077 ceph-mon[30319]: scrub summary: idle 2024-09-10T09:59:41.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.753+0000 7fd583e00640 1 -- 172.21.15.77:0/2676989754 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd584103e30 msgr2=0x7fd584104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:41.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.753+0000 7fd583e00640 1 --2- 172.21.15.77:0/2676989754 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd584103e30 0x7fd584104210 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd574009900 tx=0x7fd57402a3f0 comp rx=0 tx=0).stop 2024-09-10T09:59:41.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.753+0000 7fd583e00640 1 -- 172.21.15.77:0/2676989754 shutdown_connections 2024-09-10T09:59:41.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.753+0000 7fd583e00640 1 --2- 172.21.15.77:0/2676989754 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd584108f60 0x7fd584110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:41.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.753+0000 7fd583e00640 1 --2- 172.21.15.77:0/2676989754 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd5841047e0 0x7fd584108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:41.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.753+0000 7fd583e00640 1 --2- 172.21.15.77:0/2676989754 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd584103e30 0x7fd584104210 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:41.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.753+0000 7fd583e00640 1 -- 172.21.15.77:0/2676989754 >> 172.21.15.77:0/2676989754 conn(0x7fd5840ff8d0 msgr2=0x7fd584101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:41.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.753+0000 7fd583e00640 1 -- 172.21.15.77:0/2676989754 shutdown_connections 2024-09-10T09:59:41.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.753+0000 7fd583e00640 1 -- 172.21.15.77:0/2676989754 wait complete. 2024-09-10T09:59:41.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.754+0000 7fd583e00640 1 Processor -- start 2024-09-10T09:59:41.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.754+0000 7fd583e00640 1 -- start start 2024-09-10T09:59:41.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.754+0000 7fd583e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd584103e30 0x7fd58419a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:41.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.754+0000 7fd583e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd5841047e0 0x7fd58419a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:41.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.754+0000 7fd583e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd584108f60 0x7fd58419ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:41.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.754+0000 7fd583e00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd584113fc0 con 0x7fd584108f60 2024-09-10T09:59:41.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.754+0000 7fd583e00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd584113e40 con 0x7fd5841047e0 2024-09-10T09:59:41.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.754+0000 7fd583e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd584114140 con 0x7fd584103e30 2024-09-10T09:59:41.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.754+0000 7fd582a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd584103e30 0x7fd58419a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:41.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.754+0000 7fd57a000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd5841047e0 0x7fd58419a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:41.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.754+0000 7fd582a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd584103e30 0x7fd58419a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:43716/0 (socket says 172.21.15.77:43716) 2024-09-10T09:59:41.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.754+0000 7fd582a00640 1 -- 172.21.15.77:0/2828572641 learned_addr learned my addr 172.21.15.77:0/2828572641 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:59:41.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.755+0000 7fd583400640 1 --2- 172.21.15.77:0/2828572641 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd584108f60 0x7fd58419ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:41.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.755+0000 7fd582a00640 1 -- 172.21.15.77:0/2828572641 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd5841047e0 msgr2=0x7fd58419a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:41.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.755+0000 7fd582a00640 1 --2- 172.21.15.77:0/2828572641 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd5841047e0 0x7fd58419a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:41.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.755+0000 7fd582a00640 1 -- 172.21.15.77:0/2828572641 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd584108f60 msgr2=0x7fd58419ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:41.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.755+0000 7fd582a00640 1 --2- 172.21.15.77:0/2828572641 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd584108f60 0x7fd58419ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:41.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.755+0000 7fd582a00640 1 -- 172.21.15.77:0/2828572641 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd58419f430 con 0x7fd584103e30 2024-09-10T09:59:41.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.755+0000 7fd582a00640 1 --2- 172.21.15.77:0/2828572641 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd584103e30 0x7fd58419a480 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd574006010 tx=0x7fd574004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:41.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.755+0000 7fd57be00640 1 -- 172.21.15.77:0/2828572641 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd57403f030 con 0x7fd584103e30 2024-09-10T09:59:41.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.755+0000 7fd57be00640 1 -- 172.21.15.77:0/2828572641 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd574004440 con 0x7fd584103e30 2024-09-10T09:59:41.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.755+0000 7fd583e00640 1 -- 172.21.15.77:0/2828572641 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd58419f6c0 con 0x7fd584103e30 2024-09-10T09:59:41.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.756+0000 7fd57be00640 1 -- 172.21.15.77:0/2828572641 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd574035070 con 0x7fd584103e30 2024-09-10T09:59:41.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.756+0000 7fd583e00640 1 -- 172.21.15.77:0/2828572641 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5841a7000 con 0x7fd584103e30 2024-09-10T09:59:41.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.758+0000 7fd57be00640 1 -- 172.21.15.77:0/2828572641 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd574042030 con 0x7fd584103e30 2024-09-10T09:59:41.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.758+0000 7fd58c40d740 1 Processor -- start 2024-09-10T09:59:41.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.759+0000 7fd58c40d740 1 -- start start 2024-09-10T09:59:41.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.759+0000 7fd58c40d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562381f886c0 0x562381f88aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:41.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.759+0000 7fd58c40d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562381f88fe0 0x562381f93360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:41.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.759+0000 7fd58c40d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562381f938a0 0x562381f95c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:41.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.759+0000 7fd58c40d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x562381eed260 con 0x562381f938a0 2024-09-10T09:59:41.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.759+0000 7fd58c40d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x562381e94500 con 0x562381f88fe0 2024-09-10T09:59:41.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.759+0000 7fd58c40d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562381ee9220 con 0x562381f886c0 2024-09-10T09:59:41.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.759+0000 7fd583400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562381f886c0 0x562381f88aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:41.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.759+0000 7fd582a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562381f88fe0 0x562381f93360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:41.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.759+0000 7fd583400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562381f886c0 0x562381f88aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:43728/0 (socket says 172.21.15.77:43728) 2024-09-10T09:59:41.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.759+0000 7fd583400640 1 -- 172.21.15.77:0/2515783827 learned_addr learned my addr 172.21.15.77:0/2515783827 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:59:41.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.760+0000 7fd57a000640 1 --2- 172.21.15.77:0/2515783827 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562381f938a0 0x562381f95c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:41.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.760+0000 7fd583400640 1 -- 172.21.15.77:0/2515783827 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562381f88fe0 msgr2=0x562381f93360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:41.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.760+0000 7fd583400640 1 --2- 172.21.15.77:0/2515783827 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562381f88fe0 0x562381f93360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:41.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.760+0000 7fd583400640 1 -- 172.21.15.77:0/2515783827 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562381f938a0 msgr2=0x562381f95c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:41.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.760+0000 7fd583400640 1 --2- 172.21.15.77:0/2515783827 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562381f938a0 0x562381f95c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:41.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.760+0000 7fd583400640 1 -- 172.21.15.77:0/2515783827 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562381c5eba0 con 0x562381f886c0 2024-09-10T09:59:41.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.760+0000 7fd57a000640 1 --2- 172.21.15.77:0/2515783827 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562381f938a0 0x562381f95c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:59:41.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.760+0000 7fd583400640 1 --2- 172.21.15.77:0/2515783827 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562381f886c0 0x562381f88aa0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd57c009f60 tx=0x7fd57c009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=960f23fd9fcd853e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:41.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.760+0000 7fd582a00640 1 --2- 172.21.15.77:0/2515783827 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562381f88fe0 0x562381f93360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:59:41.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.760+0000 7fd57be00640 1 --2- 172.21.15.77:0/2828572641 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fd5680a2c00 0x7fd5680a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:41.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.760+0000 7fd573e00640 1 -- 172.21.15.77:0/2515783827 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd57c01e070 con 0x562381f886c0 2024-09-10T09:59:41.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.760+0000 7fd573e00640 1 -- 172.21.15.77:0/2515783827 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd57c00fa20 con 0x562381f886c0 2024-09-10T09:59:41.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.760+0000 7fd573e00640 1 -- 172.21.15.77:0/2515783827 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd57c019b90 con 0x562381f886c0 2024-09-10T09:59:41.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.761+0000 7fd57be00640 1 -- 172.21.15.77:0/2828572641 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fd5740ef690 con 0x7fd584103e30 2024-09-10T09:59:41.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.761+0000 7fd582a00640 1 --2- 172.21.15.77:0/2828572641 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fd5680a2c00 0x7fd5680a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:41.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.761+0000 7fd582a00640 1 --2- 172.21.15.77:0/2828572641 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fd5680a2c00 0x7fd5680a50c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd5740c41e0 tx=0x7fd5740f5530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:41.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.761+0000 7fd58c40d740 1 -- 172.21.15.77:0/2515783827 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562381f886c0 msgr2=0x562381f88aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:41.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.761+0000 7fd58c40d740 1 --2- 172.21.15.77:0/2515783827 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562381f886c0 0x562381f88aa0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd57c009f60 tx=0x7fd57c009f90 comp rx=0 tx=0).stop 2024-09-10T09:59:41.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.761+0000 7fd58c40d740 1 -- 172.21.15.77:0/2515783827 shutdown_connections 2024-09-10T09:59:41.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.761+0000 7fd58c40d740 1 --2- 172.21.15.77:0/2515783827 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562381f938a0 0x562381f95c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:41.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.761+0000 7fd58c40d740 1 --2- 172.21.15.77:0/2515783827 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562381f88fe0 0x562381f93360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:41.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.761+0000 7fd58c40d740 1 --2- 172.21.15.77:0/2515783827 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562381f886c0 0x562381f88aa0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:41.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.761+0000 7fd58c40d740 1 -- 172.21.15.77:0/2515783827 >> 172.21.15.77:0/2515783827 conn(0x562381f84c50 msgr2=0x562381f85030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:41.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.761+0000 7fd58c40d740 1 -- 172.21.15.77:0/2515783827 shutdown_connections 2024-09-10T09:59:41.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.762+0000 7fd58c40d740 1 -- 172.21.15.77:0/2515783827 wait complete. 2024-09-10T09:59:41.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.762+0000 7fd58c40d740 1 -- start start 2024-09-10T09:59:41.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.762+0000 7fd58c40d740 1 Processor -- start 2024-09-10T09:59:41.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.763+0000 7fd58c40d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562381f886c0 0x562381f907e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:41.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.763+0000 7fd58c40d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562381f88fe0 0x562381f90d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:41.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.763+0000 7fd58c40d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562381f938a0 0x562381f91260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:41.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.763+0000 7fd58c40d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x562381f96470 con 0x562381f886c0 2024-09-10T09:59:41.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.763+0000 7fd58c40d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x562381ea7bd0 con 0x562381f88fe0 2024-09-10T09:59:41.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.763+0000 7fd58c40d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562381f965f0 con 0x562381f938a0 2024-09-10T09:59:41.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.763+0000 7fd58c40d740 10 client.0 _ll_register_callbacks cb 0x562381bfc0a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:59:41.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.763+0000 7fd57a000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562381f88fe0 0x562381f90d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:41.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.763+0000 7fd57a000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562381f88fe0 0x562381f90d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:42244/0 (socket says 172.21.15.77:42244) 2024-09-10T09:59:41.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.763+0000 7fd57a000640 1 -- 172.21.15.77:0/4046184489 learned_addr learned my addr 172.21.15.77:0/4046184489 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:59:41.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.763+0000 7fd583400640 1 --2- 172.21.15.77:0/4046184489 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562381f886c0 0x562381f907e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:41.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.763+0000 7fd583400640 1 --2- 172.21.15.77:0/4046184489 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562381f938a0 0x562381f91260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:41.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.763+0000 7fd57a000640 1 -- 172.21.15.77:0/4046184489 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562381f938a0 msgr2=0x562381f91260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:41.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.763+0000 7fd57a000640 1 --2- 172.21.15.77:0/4046184489 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562381f938a0 0x562381f91260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:41.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.763+0000 7fd57a000640 1 -- 172.21.15.77:0/4046184489 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562381f886c0 msgr2=0x562381f907e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:41.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.763+0000 7fd57a000640 1 --2- 172.21.15.77:0/4046184489 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562381f886c0 0x562381f907e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:41.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.763+0000 7fd57a000640 1 -- 172.21.15.77:0/4046184489 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562381f9f930 con 0x562381f88fe0 2024-09-10T09:59:41.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.763+0000 7fd583400640 1 --2- 172.21.15.77:0/4046184489 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562381f886c0 0x562381f907e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:59:41.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.764+0000 7fd57a000640 1 --2- 172.21.15.77:0/4046184489 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562381f88fe0 0x562381f90d20 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd56c012730 tx=0x7fd56c012bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f14ee37d1f8109 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:41.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.764+0000 7fd570c00640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T09:59:41.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.764+0000 7fd570c00640 1 -- 172.21.15.77:0/4046184489 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd56c004050 con 0x562381f88fe0 2024-09-10T09:59:41.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.764+0000 7fd58c40d740 1 -- 172.21.15.77:0/4046184489 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd57c00fa20 con 0x562381f88fe0 2024-09-10T09:59:41.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.764+0000 7fd570c00640 1 -- 172.21.15.77:0/4046184489 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fd56c0041f0 con 0x562381f88fe0 2024-09-10T09:59:41.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.764+0000 7fd570c00640 1 -- 172.21.15.77:0/4046184489 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd56c025050 con 0x562381f88fe0 2024-09-10T09:59:41.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.764+0000 7fd58c40d740 10 client.26438 fetch_fsmap learned FSMap version 32 2024-09-10T09:59:41.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.764+0000 7fd58c40d740 1 -- 172.21.15.77:0/4046184489 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562381fa2300 con 0x562381f88fe0 2024-09-10T09:59:41.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.765+0000 7fd570c00640 1 -- 172.21.15.77:0/4046184489 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd57c00fa20 con 0x562381f88fe0 2024-09-10T09:59:41.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.765+0000 7fd570c00640 1 -- 172.21.15.77:0/4046184489 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fd56c003a80 con 0x562381f88fe0 2024-09-10T09:59:41.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.765+0000 7fd58c40d740 10 client.26438 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T09:59:41.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.765+0000 7fd58c40d740 10 client.26438 resolve_mds: validated gid 24563 aka daemon mds.b 2024-09-10T09:59:41.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.765+0000 7fd58c40d740 20 client.26438 populate_metadata read hostname 'smithi077' 2024-09-10T09:59:41.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.765+0000 7fd58c40d740 1 --2- 172.21.15.77:0/4046184489 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x562381fa3760 0x562381fa5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:41.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.765+0000 7fd58c40d740 4 client.26438 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:59:41.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.765+0000 7fd58c40d740 1 -- 172.21.15.77:0/4046184489 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562381efa990 con 0x562381fa3760 2024-09-10T09:59:41.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.765+0000 7fd583400640 1 --2- 172.21.15.77:0/4046184489 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x562381fa3760 0x562381fa5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:41.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.766+0000 7fd583400640 1 --2- 172.21.15.77:0/4046184489 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x562381fa3760 0x562381fa5bb0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed1fe01b5ae89fd1 server_cookie=cbc11c64b75d040a in_seq=0 out_seq=0 2024-09-10T09:59:41.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.766+0000 7fd570c00640 10 client.26438 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T09:59:41.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.766+0000 7fd570c00640 1 -- 172.21.15.77:0/4046184489 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562381efa990 con 0x562381fa3760 2024-09-10T09:59:41.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.766+0000 7fd570c00640 10 client.26438 handle_command_reply: tid=0 2024-09-10T09:59:41.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.767+0000 7fd58c40d740 1 client.26438 shutdown 2024-09-10T09:59:41.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.767+0000 7fd58c40d740 1 -- 172.21.15.77:0/4046184489 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562381f88fe0 msgr2=0x562381f90d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:41.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.767+0000 7fd58c40d740 1 --2- 172.21.15.77:0/4046184489 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562381f88fe0 0x562381f90d20 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd56c012730 tx=0x7fd56c012bf0 comp rx=0 tx=0).stop 2024-09-10T09:59:41.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.767+0000 7fd58c40d740 1 -- 172.21.15.77:0/4046184489 shutdown_connections 2024-09-10T09:59:41.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.767+0000 7fd58c40d740 1 --2- 172.21.15.77:0/4046184489 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x562381fa3760 0x562381fa5bb0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:41.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.767+0000 7fd58c40d740 1 --2- 172.21.15.77:0/4046184489 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562381f938a0 0x562381f91260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:41.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.767+0000 7fd58c40d740 1 --2- 172.21.15.77:0/4046184489 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562381f88fe0 0x562381f90d20 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:41.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.767+0000 7fd570c00640 0 client.26438 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T09:59:41.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.767+0000 7fd58c40d740 1 --2- 172.21.15.77:0/4046184489 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562381f886c0 0x562381f907e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:41.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.767+0000 7fd58c40d740 1 -- 172.21.15.77:0/4046184489 >> 172.21.15.77:0/4046184489 conn(0x562381f84c50 msgr2=0x562381f83e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:41.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.767+0000 7fd58c40d740 1 -- 172.21.15.77:0/4046184489 shutdown_connections 2024-09-10T09:59:41.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.767+0000 7fd58c40d740 1 -- 172.21.15.77:0/4046184489 wait complete. 2024-09-10T09:59:41.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.767+0000 7fd58c40d740 20 client.26438 trim_cache size 0 max 16384 2024-09-10T09:59:41.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.786+0000 7fd58c40d740 1 Processor -- start 2024-09-10T09:59:41.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.786+0000 7fd58c40d740 1 -- start start 2024-09-10T09:59:41.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.786+0000 7fd58c40d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562381f886c0 0x562381f83d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:41.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.786+0000 7fd58c40d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562381f88fe0 0x562381fab4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:41.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.786+0000 7fd58c40d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562381f938a0 0x562381faba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:41.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.786+0000 7fd58c40d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd5841b0eb0 con 0x562381f88fe0 2024-09-10T09:59:41.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.786+0000 7fd58c40d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd5841b2ab0 con 0x562381f938a0 2024-09-10T09:59:41.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.786+0000 7fd58c40d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd5841b1580 con 0x562381f886c0 2024-09-10T09:59:41.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.786+0000 7fd583400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562381f886c0 0x562381f83d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:41.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.786+0000 7fd583400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562381f886c0 0x562381f83d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:43734/0 (socket says 172.21.15.77:43734) 2024-09-10T09:59:41.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.786+0000 7fd583400640 1 -- 172.21.15.77:0/439321284 learned_addr learned my addr 172.21.15.77:0/439321284 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:59:41.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.786+0000 7fd57a000640 1 --2- 172.21.15.77:0/439321284 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562381f88fe0 0x562381fab4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:41.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.786+0000 7fd583400640 1 --2- 172.21.15.77:0/439321284 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562381f938a0 0x562381faba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:41.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.787+0000 7fd583400640 1 -- 172.21.15.77:0/439321284 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562381f938a0 msgr2=0x562381faba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:41.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.787+0000 7fd583400640 1 --2- 172.21.15.77:0/439321284 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562381f938a0 0x562381faba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:41.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.787+0000 7fd583400640 1 -- 172.21.15.77:0/439321284 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562381f88fe0 msgr2=0x562381fab4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:41.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.787+0000 7fd583400640 1 --2- 172.21.15.77:0/439321284 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562381f88fe0 0x562381fab4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:41.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.787+0000 7fd583400640 1 -- 172.21.15.77:0/439321284 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562381f846f0 con 0x562381f886c0 2024-09-10T09:59:41.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.787+0000 7fd57a000640 1 --2- 172.21.15.77:0/439321284 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562381f88fe0 0x562381fab4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:59:41.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.787+0000 7fd583400640 1 --2- 172.21.15.77:0/439321284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562381f886c0 0x562381f83d60 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd57c009f00 tx=0x7fd57c009d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6c6bb89e76c8c27 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:41.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.787+0000 7fd573400640 1 -- 172.21.15.77:0/439321284 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd57c01e040 con 0x562381f886c0 2024-09-10T09:59:41.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.787+0000 7fd573400640 1 -- 172.21.15.77:0/439321284 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd57c01f040 con 0x562381f886c0 2024-09-10T09:59:41.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.787+0000 7fd573400640 1 -- 172.21.15.77:0/439321284 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd57c0199a0 con 0x562381f886c0 2024-09-10T09:59:41.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.788+0000 7fd58c40d740 1 -- 172.21.15.77:0/439321284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562381f886c0 msgr2=0x562381f83d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:41.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.788+0000 7fd58c40d740 1 --2- 172.21.15.77:0/439321284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562381f886c0 0x562381f83d60 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd57c009f00 tx=0x7fd57c009d10 comp rx=0 tx=0).stop 2024-09-10T09:59:41.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.788+0000 7fd58c40d740 1 -- 172.21.15.77:0/439321284 shutdown_connections 2024-09-10T09:59:41.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.788+0000 7fd58c40d740 1 --2- 172.21.15.77:0/439321284 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562381f938a0 0x562381faba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:41.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.788+0000 7fd58c40d740 1 --2- 172.21.15.77:0/439321284 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562381f88fe0 0x562381fab4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:41.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.788+0000 7fd58c40d740 1 --2- 172.21.15.77:0/439321284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562381f886c0 0x562381f83d60 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:41.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.788+0000 7fd58c40d740 1 -- 172.21.15.77:0/439321284 >> 172.21.15.77:0/439321284 conn(0x562381f84c50 msgr2=0x562381f83650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:41.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.788+0000 7fd58c40d740 1 -- 172.21.15.77:0/439321284 shutdown_connections 2024-09-10T09:59:41.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.788+0000 7fd58c40d740 1 -- 172.21.15.77:0/439321284 wait complete. 2024-09-10T09:59:41.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.789+0000 7fd58c40d740 1 -- start start 2024-09-10T09:59:41.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.789+0000 7fd58c40d740 1 Processor -- start 2024-09-10T09:59:41.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.789+0000 7fd58c40d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562381f886c0 0x562381f97da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:41.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.789+0000 7fd58c40d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562381f88fe0 0x562381f982e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:41.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.789+0000 7fd58c40d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562381f938a0 0x56238200d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:41.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.789+0000 7fd58c40d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd5841b2320 con 0x562381f886c0 2024-09-10T09:59:41.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.789+0000 7fd58c40d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd5841b1c50 con 0x562381f88fe0 2024-09-10T09:59:41.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.789+0000 7fd58c40d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd5841aee90 con 0x562381f938a0 2024-09-10T09:59:41.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.789+0000 7fd58c40d740 10 client.0 _ll_register_callbacks cb 0x7fd56c009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:59:41.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.790+0000 7fd583400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562381f886c0 0x562381f97da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:41.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.790+0000 7fd57a000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562381f88fe0 0x562381f982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:41.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.790+0000 7fd57a000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562381f88fe0 0x562381f982e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:42264/0 (socket says 172.21.15.77:42264) 2024-09-10T09:59:41.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.790+0000 7fd57a000640 1 -- 172.21.15.77:0/1692007683 learned_addr learned my addr 172.21.15.77:0/1692007683 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T09:59:41.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.790+0000 7fd583400640 1 --2- 172.21.15.77:0/1692007683 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562381f938a0 0x56238200d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:41.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.790+0000 7fd57a000640 1 -- 172.21.15.77:0/1692007683 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562381f938a0 msgr2=0x56238200d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:41.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.790+0000 7fd57a000640 1 --2- 172.21.15.77:0/1692007683 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562381f938a0 0x56238200d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:41.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.790+0000 7fd57a000640 1 -- 172.21.15.77:0/1692007683 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562381f886c0 msgr2=0x562381f97da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:41.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.790+0000 7fd57a000640 1 --2- 172.21.15.77:0/1692007683 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562381f886c0 0x562381f97da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:41.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.790+0000 7fd57a000640 1 -- 172.21.15.77:0/1692007683 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56238200dad0 con 0x562381f88fe0 2024-09-10T09:59:41.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.790+0000 7fd583400640 1 --2- 172.21.15.77:0/1692007683 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562381f938a0 0x56238200d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:59:41.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.790+0000 7fd57a000640 1 --2- 172.21.15.77:0/1692007683 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562381f88fe0 0x562381f982e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x562381f90070 tx=0x7fd56c009390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d45fd14cebd05197 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:41.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.790+0000 7fd570c00640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T09:59:41.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.791+0000 7fd570c00640 1 -- 172.21.15.77:0/1692007683 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd56c010b60 con 0x562381f88fe0 2024-09-10T09:59:41.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.791+0000 7fd58c40d740 1 -- 172.21.15.77:0/1692007683 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd57c01f040 con 0x562381f88fe0 2024-09-10T09:59:41.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.791+0000 7fd570c00640 1 -- 172.21.15.77:0/1692007683 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fd56c018070 con 0x562381f88fe0 2024-09-10T09:59:41.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.791+0000 7fd570c00640 1 -- 172.21.15.77:0/1692007683 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd56c038050 con 0x562381f88fe0 2024-09-10T09:59:41.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.791+0000 7fd58c40d740 10 client.26444 fetch_fsmap learned FSMap version 32 2024-09-10T09:59:41.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.791+0000 7fd58c40d740 1 -- 172.21.15.77:0/1692007683 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56238200eda0 con 0x562381f88fe0 2024-09-10T09:59:41.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.791+0000 7fd570c00640 1 -- 172.21.15.77:0/1692007683 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd57c01f040 con 0x562381f88fe0 2024-09-10T09:59:41.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.791+0000 7fd570c00640 1 -- 172.21.15.77:0/1692007683 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fd56c00fa60 con 0x562381f88fe0 2024-09-10T09:59:41.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.791+0000 7fd58c40d740 10 client.26444 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T09:59:41.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.791+0000 7fd58c40d740 10 client.26444 resolve_mds: validated gid 24563 aka daemon mds.b 2024-09-10T09:59:41.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.791+0000 7fd58c40d740 20 client.26444 populate_metadata read hostname 'smithi077' 2024-09-10T09:59:41.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.792+0000 7fd58c40d740 1 --2- 172.21.15.77:0/1692007683 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x56238200f6d0 0x562382011af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:41.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.792+0000 7fd58c40d740 4 client.26444 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.75, "await_for": 45.0}] 2024-09-10T09:59:41.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.792+0000 7fd58c40d740 1 -- 172.21.15.77:0/1692007683 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.75, "await_for": 45.0}) -- 0x562381ef4ce0 con 0x56238200f6d0 2024-09-10T09:59:41.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.792+0000 7fd583400640 1 --2- 172.21.15.77:0/1692007683 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x56238200f6d0 0x562382011af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:41.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.792+0000 7fd583400640 1 --2- 172.21.15.77:0/1692007683 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x56238200f6d0 0x562382011af0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=498f184d7cce00f4 server_cookie=2f27adf7ef7ee55b in_seq=0 out_seq=0 2024-09-10T09:59:41.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:41.792+0000 7fd570c00640 10 client.26444 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T09:59:41.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:59:41 smithi077 ceph-mon[30319]: scrub summary: idle 2024-09-10T09:59:41.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:59:41 smithi077 ceph-mon[30319]: scrub summary: idle 2024-09-10T09:59:41.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:59:41 smithi077 ceph-mon[30319]: scrub summary: idle 2024-09-10T09:59:42.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:59:41 smithi134 ceph-mon[35674]: scrub summary: idle 2024-09-10T09:59:42.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:59:41 smithi134 ceph-mon[35674]: scrub summary: idle 2024-09-10T09:59:42.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:59:41 smithi134 ceph-mon[35674]: scrub summary: idle 2024-09-10T09:59:42.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:59:41 smithi120 ceph-mon[34770]: scrub summary: idle 2024-09-10T09:59:42.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:59:41 smithi120 ceph-mon[34770]: scrub summary: idle 2024-09-10T09:59:42.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:59:41 smithi120 ceph-mon[34770]: scrub summary: idle 2024-09-10T09:59:42.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:42.379+0000 7fd570c00640 1 -- 172.21.15.77:0/1692007683 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x562381ef4ce0 con 0x56238200f6d0 2024-09-10T09:59:42.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:42.379+0000 7fd570c00640 10 client.26444 handle_command_reply: tid=0 2024-09-10T09:59:42.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:42.379+0000 7fd58c40d740 1 client.26444 shutdown 2024-09-10T09:59:42.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:42.379+0000 7fd58c40d740 1 -- 172.21.15.77:0/1692007683 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562381f88fe0 msgr2=0x562381f982e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:42.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:42.379+0000 7fd58c40d740 1 --2- 172.21.15.77:0/1692007683 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562381f88fe0 0x562381f982e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x562381f90070 tx=0x7fd56c009390 comp rx=0 tx=0).stop 2024-09-10T09:59:42.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:42.379+0000 7fd58c40d740 1 -- 172.21.15.77:0/1692007683 shutdown_connections 2024-09-10T09:59:42.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:42.379+0000 7fd58c40d740 1 --2- 172.21.15.77:0/1692007683 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x56238200f6d0 0x562382011af0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:42.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:42.379+0000 7fd58c40d740 1 --2- 172.21.15.77:0/1692007683 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562381f938a0 0x56238200d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:42.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:42.379+0000 7fd58c40d740 1 --2- 172.21.15.77:0/1692007683 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562381f88fe0 0x562381f982e0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:42.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:42.379+0000 7fd570c00640 0 client.26444 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T09:59:42.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:42.379+0000 7fd58c40d740 1 --2- 172.21.15.77:0/1692007683 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562381f886c0 0x562381f97da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:42.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:42.379+0000 7fd58c40d740 1 -- 172.21.15.77:0/1692007683 >> 172.21.15.77:0/1692007683 conn(0x562381f84c50 msgr2=0x562381f83e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:42.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:42.379+0000 7fd58c40d740 1 -- 172.21.15.77:0/1692007683 shutdown_connections 2024-09-10T09:59:42.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:42.379+0000 7fd58c40d740 1 -- 172.21.15.77:0/1692007683 wait complete. 2024-09-10T09:59:42.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:42.379+0000 7fd58c40d740 20 client.26444 trim_cache size 0 max 16384 2024-09-10T09:59:42.383 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-10T09:59:42.383 INFO:teuthology.orchestra.run.smithi077.stdout: "epoch": 32, 2024-09-10T09:59:42.383 INFO:teuthology.orchestra.run.smithi077.stdout: "leader": 24563, 2024-09-10T09:59:42.383 INFO:teuthology.orchestra.run.smithi077.stdout: "set_version": 14, 2024-09-10T09:59:42.383 INFO:teuthology.orchestra.run.smithi077.stdout: "sets": { 2024-09-10T09:59:42.383 INFO:teuthology.orchestra.run.smithi077.stdout: "90af319d": { 2024-09-10T09:59:42.383 INFO:teuthology.orchestra.run.smithi077.stdout: "version": 14, 2024-09-10T09:59:42.383 INFO:teuthology.orchestra.run.smithi077.stdout: "age_ref": 0.0, 2024-09-10T09:59:42.383 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-10T09:59:42.383 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "QUIESCED", 2024-09-10T09:59:42.383 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-10T09:59:42.384 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T09:59:42.384 INFO:teuthology.orchestra.run.smithi077.stdout: "timeout": 90.0, 2024-09-10T09:59:42.384 INFO:teuthology.orchestra.run.smithi077.stdout: "expiration": 142.8, 2024-09-10T09:59:42.384 INFO:teuthology.orchestra.run.smithi077.stdout: "members": { 2024-09-10T09:59:42.384 INFO:teuthology.orchestra.run.smithi077.stdout: "file:/": { 2024-09-10T09:59:42.384 INFO:teuthology.orchestra.run.smithi077.stdout: "excluded": false, 2024-09-10T09:59:42.384 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-10T09:59:42.384 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "QUIESCED", 2024-09-10T09:59:42.384 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-10T09:59:42.384 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T09:59:42.384 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T09:59:42.384 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T09:59:42.384 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T09:59:42.384 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T09:59:42.384 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-10T09:59:42.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:42.380+0000 7fd583e00640 1 -- 172.21.15.77:0/2828572641 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fd5680a2c00 msgr2=0x7fd5680a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:42.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:42.380+0000 7fd583e00640 1 --2- 172.21.15.77:0/2828572641 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fd5680a2c00 0x7fd5680a50c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd5740c41e0 tx=0x7fd5740f5530 comp rx=0 tx=0).stop 2024-09-10T09:59:42.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:42.380+0000 7fd583e00640 1 -- 172.21.15.77:0/2828572641 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd584103e30 msgr2=0x7fd58419a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:42.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:42.380+0000 7fd583e00640 1 --2- 172.21.15.77:0/2828572641 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd584103e30 0x7fd58419a480 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd574006010 tx=0x7fd574004040 comp rx=0 tx=0).stop 2024-09-10T09:59:42.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:42.380+0000 7fd583e00640 1 -- 172.21.15.77:0/2828572641 shutdown_connections 2024-09-10T09:59:42.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:42.380+0000 7fd583e00640 1 --2- 172.21.15.77:0/2828572641 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fd5680a2c00 0x7fd5680a50c0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:42.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:42.380+0000 7fd583e00640 1 --2- 172.21.15.77:0/2828572641 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd584108f60 0x7fd58419ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:42.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:42.380+0000 7fd583e00640 1 --2- 172.21.15.77:0/2828572641 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd5841047e0 0x7fd58419a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:42.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:42.380+0000 7fd583e00640 1 --2- 172.21.15.77:0/2828572641 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd584103e30 0x7fd58419a480 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:42.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:42.380+0000 7fd583e00640 1 -- 172.21.15.77:0/2828572641 >> 172.21.15.77:0/2828572641 conn(0x7fd5840ff8d0 msgr2=0x7fd584100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:42.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:42.380+0000 7fd583e00640 1 -- 172.21.15.77:0/2828572641 shutdown_connections 2024-09-10T09:59:42.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T09:59:42.380+0000 7fd583e00640 1 -- 172.21.15.77:0/2828572641 wait complete. 2024-09-10T09:59:43.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:59:42 smithi134 ceph-mon[35674]: pgmap v527: 129 pgs: 129 active+clean; 96 MiB data, 878 MiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 210 KiB/s wr, 44 op/s 2024-09-10T09:59:43.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:59:42 smithi120 ceph-mon[34770]: pgmap v527: 129 pgs: 129 active+clean; 96 MiB data, 878 MiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 210 KiB/s wr, 44 op/s 2024-09-10T09:59:43.184 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:59:42 smithi077 ceph-mon[30319]: pgmap v527: 129 pgs: 129 active+clean; 96 MiB data, 878 MiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 210 KiB/s wr, 44 op/s 2024-09-10T09:59:43.222 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '90af319d', quiesce took 4.5 seconds. Will release after: 18.25 2024-09-10T09:59:45.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:59:44 smithi134 ceph-mon[35674]: pgmap v528: 129 pgs: 129 active+clean; 97 MiB data, 934 MiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 837 KiB/s wr, 151 op/s 2024-09-10T09:59:45.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:59:44 smithi120 ceph-mon[34770]: pgmap v528: 129 pgs: 129 active+clean; 97 MiB data, 934 MiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 837 KiB/s wr, 151 op/s 2024-09-10T09:59:45.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:59:44 smithi077 ceph-mon[30319]: pgmap v528: 129 pgs: 129 active+clean; 97 MiB data, 934 MiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 837 KiB/s wr, 151 op/s 2024-09-10T09:59:47.150 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:59:46 smithi134 ceph-mon[35674]: pgmap v529: 129 pgs: 129 active+clean; 97 MiB data, 934 MiB used, 1.0 TiB / 1.0 TiB avail; 545 KiB/s rd, 822 KiB/s wr, 147 op/s 2024-09-10T09:59:47.175 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:59:46 smithi120 ceph-mon[34770]: pgmap v529: 129 pgs: 129 active+clean; 97 MiB data, 934 MiB used, 1.0 TiB / 1.0 TiB avail; 545 KiB/s rd, 822 KiB/s wr, 147 op/s 2024-09-10T09:59:47.193 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:59:46 smithi077 ceph-mon[30319]: pgmap v529: 129 pgs: 129 active+clean; 97 MiB data, 934 MiB used, 1.0 TiB / 1.0 TiB avail; 545 KiB/s rd, 822 KiB/s wr, 147 op/s 2024-09-10T09:59:49.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:59:48 smithi120 ceph-mon[34770]: pgmap v530: 129 pgs: 129 active+clean; 97 MiB data, 935 MiB used, 1.0 TiB / 1.0 TiB avail; 545 KiB/s rd, 822 KiB/s wr, 147 op/s 2024-09-10T09:59:49.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:59:48 smithi077 ceph-mon[30319]: pgmap v530: 129 pgs: 129 active+clean; 97 MiB data, 935 MiB used, 1.0 TiB / 1.0 TiB avail; 545 KiB/s rd, 822 KiB/s wr, 147 op/s 2024-09-10T09:59:49.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:59:48 smithi134 ceph-mon[35674]: pgmap v530: 129 pgs: 129 active+clean; 97 MiB data, 935 MiB used, 1.0 TiB / 1.0 TiB avail; 545 KiB/s rd, 822 KiB/s wr, 147 op/s 2024-09-10T09:59:51.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:59:50 smithi120 ceph-mon[34770]: pgmap v531: 129 pgs: 129 active+clean; 97 MiB data, 935 MiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 822 KiB/s wr, 151 op/s 2024-09-10T09:59:51.183 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:59:50 smithi134 ceph-mon[35674]: pgmap v531: 129 pgs: 129 active+clean; 97 MiB data, 935 MiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 822 KiB/s wr, 151 op/s 2024-09-10T09:59:51.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:59:50 smithi077 ceph-mon[30319]: pgmap v531: 129 pgs: 129 active+clean; 97 MiB data, 935 MiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 822 KiB/s wr, 151 op/s 2024-09-10T09:59:53.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:59:52 smithi120 ceph-mon[34770]: pgmap v532: 129 pgs: 129 active+clean; 97 MiB data, 935 MiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 627 KiB/s wr, 114 op/s 2024-09-10T09:59:53.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:59:52 smithi077 ceph-mon[30319]: pgmap v532: 129 pgs: 129 active+clean; 97 MiB data, 935 MiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 627 KiB/s wr, 114 op/s 2024-09-10T09:59:53.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:59:52 smithi134 ceph-mon[35674]: pgmap v532: 129 pgs: 129 active+clean; 97 MiB data, 935 MiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 627 KiB/s wr, 114 op/s 2024-09-10T09:59:55.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:59:54 smithi120 ceph-mon[34770]: pgmap v533: 129 pgs: 129 active+clean; 97 MiB data, 935 MiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 628 KiB/s wr, 117 op/s 2024-09-10T09:59:55.174 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:59:54 smithi077 ceph-mon[30319]: pgmap v533: 129 pgs: 129 active+clean; 97 MiB data, 935 MiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 628 KiB/s wr, 117 op/s 2024-09-10T09:59:55.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:59:54 smithi134 ceph-mon[35674]: pgmap v533: 129 pgs: 129 active+clean; 97 MiB data, 935 MiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 628 KiB/s wr, 117 op/s 2024-09-10T09:59:57.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:59:56 smithi120 ceph-mon[34770]: pgmap v534: 129 pgs: 129 active+clean; 97 MiB data, 935 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 10 op/s 2024-09-10T09:59:57.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:59:56 smithi077 ceph-mon[30319]: pgmap v534: 129 pgs: 129 active+clean; 97 MiB data, 935 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 10 op/s 2024-09-10T09:59:57.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:59:56 smithi134 ceph-mon[35674]: pgmap v534: 129 pgs: 129 active+clean; 97 MiB data, 935 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 10 op/s 2024-09-10T09:59:59.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 09:59:58 smithi120 ceph-mon[34770]: pgmap v535: 129 pgs: 129 active+clean; 97 MiB data, 935 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 10 op/s 2024-09-10T09:59:59.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 09:59:58 smithi077 ceph-mon[30319]: pgmap v535: 129 pgs: 129 active+clean; 97 MiB data, 935 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 10 op/s 2024-09-10T09:59:59.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 09:59:58 smithi134 ceph-mon[35674]: pgmap v535: 129 pgs: 129 active+clean; 97 MiB data, 935 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 10 op/s 2024-09-10T10:00:01.111 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T10:00:01.168 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:00:00 smithi077 ceph-mon[30319]: pgmap v536: 129 pgs: 129 active+clean; 97 MiB data, 935 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-09-10T10:00:01.168 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:00:00 smithi077 ceph-mon[30319]: overall HEALTH_OK 2024-09-10T10:00:01.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:00:00 smithi120 ceph-mon[34770]: pgmap v536: 129 pgs: 129 active+clean; 97 MiB data, 935 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-09-10T10:00:01.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:00:00 smithi120 ceph-mon[34770]: overall HEALTH_OK 2024-09-10T10:00:01.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:00:00 smithi134 ceph-mon[35674]: pgmap v536: 129 pgs: 129 active+clean; 97 MiB data, 935 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-09-10T10:00:01.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:00:00 smithi134 ceph-mon[35674]: overall HEALTH_OK 2024-09-10T10:00:01.473 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:00:01.496 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:00:02.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.066+0000 7f476a800640 1 -- 172.21.15.77:0/278395627 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4764109120 msgr2=0x7f4764110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:02.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.066+0000 7f476a800640 1 --2- 172.21.15.77:0/278395627 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4764109120 0x7f4764110ca0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f475c009ca0 tx=0x7f475c02a8c0 comp rx=0 tx=0).stop 2024-09-10T10:00:02.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.066+0000 7f476a800640 1 -- 172.21.15.77:0/278395627 shutdown_connections 2024-09-10T10:00:02.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.066+0000 7f476a800640 1 --2- 172.21.15.77:0/278395627 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4764109120 0x7f4764110ca0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:02.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.066+0000 7f476a800640 1 --2- 172.21.15.77:0/278395627 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f47641049a0 0x7f47641089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:02.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.066+0000 7f476a800640 1 --2- 172.21.15.77:0/278395627 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4764103ff0 0x7f47641043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:02.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.066+0000 7f476a800640 1 -- 172.21.15.77:0/278395627 >> 172.21.15.77:0/278395627 conn(0x7f47640ffa90 msgr2=0x7f4764101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:02.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.066+0000 7f476a800640 1 -- 172.21.15.77:0/278395627 shutdown_connections 2024-09-10T10:00:02.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.066+0000 7f476a800640 1 -- 172.21.15.77:0/278395627 wait complete. 2024-09-10T10:00:02.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.067+0000 7f476a800640 1 Processor -- start 2024-09-10T10:00:02.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.068+0000 7f476a800640 1 -- start start 2024-09-10T10:00:02.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.068+0000 7f476a800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4764103ff0 0x7f4764110a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:02.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.068+0000 7f476a800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47641049a0 0x7f476410db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:02.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.068+0000 7f476a800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4764109120 0x7f476410e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:02.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.068+0000 7f476a800640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f47641141a0 con 0x7f4764103ff0 2024-09-10T10:00:02.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.068+0000 7f476a800640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4764114020 con 0x7f4764109120 2024-09-10T10:00:02.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.068+0000 7f476a800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4764114320 con 0x7f47641049a0 2024-09-10T10:00:02.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.068+0000 7f4763400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4764103ff0 0x7f4764110a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:02.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.069+0000 7f4763400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4764103ff0 0x7f4764110a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:37694/0 (socket says 172.21.15.77:37694) 2024-09-10T10:00:02.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.069+0000 7f4763400640 1 -- 172.21.15.77:0/4039352472 learned_addr learned my addr 172.21.15.77:0/4039352472 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:00:02.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.069+0000 7f4762a00640 1 --2- 172.21.15.77:0/4039352472 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47641049a0 0x7f476410db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:02.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.069+0000 7f4763e00640 1 --2- 172.21.15.77:0/4039352472 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4764109120 0x7f476410e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:02.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.069+0000 7f4762a00640 1 -- 172.21.15.77:0/4039352472 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4764109120 msgr2=0x7f476410e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:02.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.069+0000 7f4762a00640 1 --2- 172.21.15.77:0/4039352472 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4764109120 0x7f476410e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:02.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.069+0000 7f4762a00640 1 -- 172.21.15.77:0/4039352472 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4764103ff0 msgr2=0x7f4764110a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:02.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.069+0000 7f4762a00640 1 --2- 172.21.15.77:0/4039352472 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4764103ff0 0x7f4764110a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:02.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.069+0000 7f4762a00640 1 -- 172.21.15.77:0/4039352472 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f476410e8e0 con 0x7f47641049a0 2024-09-10T10:00:02.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.069+0000 7f4763400640 1 --2- 172.21.15.77:0/4039352472 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4764103ff0 0x7f4764110a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:00:02.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.069+0000 7f4762a00640 1 --2- 172.21.15.77:0/4039352472 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47641049a0 0x7f476410db30 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f475000e9f0 tx=0x7f475000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:02.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.070+0000 7f475be00640 1 -- 172.21.15.77:0/4039352472 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f475000ccc0 con 0x7f47641049a0 2024-09-10T10:00:02.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.070+0000 7f475be00640 1 -- 172.21.15.77:0/4039352472 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4750005020 con 0x7f47641049a0 2024-09-10T10:00:02.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.070+0000 7f475be00640 1 -- 172.21.15.77:0/4039352472 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47500049e0 con 0x7f47641049a0 2024-09-10T10:00:02.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.070+0000 7f476a800640 1 -- 172.21.15.77:0/4039352472 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47641aaf40 con 0x7f47641049a0 2024-09-10T10:00:02.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.070+0000 7f476a800640 1 -- 172.21.15.77:0/4039352472 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47641ab340 con 0x7f47641049a0 2024-09-10T10:00:02.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.073+0000 7f476d2a4740 1 Processor -- start 2024-09-10T10:00:02.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.073+0000 7f476d2a4740 1 -- start start 2024-09-10T10:00:02.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.073+0000 7f475be00640 1 -- 172.21.15.77:0/4039352472 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4750005560 con 0x7f47641049a0 2024-09-10T10:00:02.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.074+0000 7f476d2a4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5589ff9dbb90 0x5589ff9dbf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:02.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.074+0000 7f476d2a4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5589ff9dc4b0 0x5589ff9e6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:02.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.074+0000 7f476d2a4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5589ff9e6d70 0x5589ff9e9160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:02.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.074+0000 7f476d2a4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5589ff9423e0 con 0x5589ff9dc4b0 2024-09-10T10:00:02.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.074+0000 7f476d2a4740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5589ff8e9d80 con 0x5589ff9e6d70 2024-09-10T10:00:02.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.074+0000 7f476d2a4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5589ff93e3a0 con 0x5589ff9dbb90 2024-09-10T10:00:02.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.074+0000 7f4763e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5589ff9dbb90 0x5589ff9dbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:02.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.074+0000 7f4763400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5589ff9dc4b0 0x5589ff9e6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:02.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.074+0000 7f4763e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5589ff9dbb90 0x5589ff9dbf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:59136/0 (socket says 172.21.15.77:59136) 2024-09-10T10:00:02.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.074+0000 7f4763e00640 1 -- 172.21.15.77:0/3067220463 learned_addr learned my addr 172.21.15.77:0/3067220463 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:00:02.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.074+0000 7f4762a00640 1 --2- 172.21.15.77:0/3067220463 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5589ff9e6d70 0x5589ff9e9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:02.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.075+0000 7f4763e00640 1 -- 172.21.15.77:0/3067220463 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5589ff9e6d70 msgr2=0x5589ff9e9160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:02.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.075+0000 7f4763e00640 1 --2- 172.21.15.77:0/3067220463 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5589ff9e6d70 0x5589ff9e9160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:02.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.075+0000 7f4763e00640 1 -- 172.21.15.77:0/3067220463 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5589ff9dc4b0 msgr2=0x5589ff9e6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:02.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.075+0000 7f4763e00640 1 --2- 172.21.15.77:0/3067220463 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5589ff9dc4b0 0x5589ff9e6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:02.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.075+0000 7f4763e00640 1 -- 172.21.15.77:0/3067220463 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589ff908ae0 con 0x5589ff9dbb90 2024-09-10T10:00:02.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.075+0000 7f4763e00640 1 --2- 172.21.15.77:0/3067220463 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5589ff9dbb90 0x5589ff9dbf70 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f475c009ca0 tx=0x7f475c0093e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16d5d5c5863434aa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:02.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.075+0000 7f475be00640 1 --2- 172.21.15.77:0/4039352472 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f47480a2c00 0x7f47480a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:02.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.075+0000 7f4758c00640 1 -- 172.21.15.77:0/3067220463 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f475c031db0 con 0x5589ff9dbb90 2024-09-10T10:00:02.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.075+0000 7f4758c00640 1 -- 172.21.15.77:0/3067220463 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f475c036030 con 0x5589ff9dbb90 2024-09-10T10:00:02.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.075+0000 7f4758c00640 1 -- 172.21.15.77:0/3067220463 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f475c0042a0 con 0x5589ff9dbb90 2024-09-10T10:00:02.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.075+0000 7f475be00640 1 -- 172.21.15.77:0/4039352472 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f47500cf2e0 con 0x7f47641049a0 2024-09-10T10:00:02.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.076+0000 7f4763400640 1 --2- 172.21.15.77:0/4039352472 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f47480a2c00 0x7f47480a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:02.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.076+0000 7f476d2a4740 1 -- 172.21.15.77:0/3067220463 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5589ff9dbb90 msgr2=0x5589ff9dbf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:02.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.076+0000 7f476d2a4740 1 --2- 172.21.15.77:0/3067220463 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5589ff9dbb90 0x5589ff9dbf70 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f475c009ca0 tx=0x7f475c0093e0 comp rx=0 tx=0).stop 2024-09-10T10:00:02.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.076+0000 7f4763400640 1 --2- 172.21.15.77:0/4039352472 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f47480a2c00 0x7f47480a50c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x5589ff9d8f80 tx=0x7f4754005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:02.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.076+0000 7f476d2a4740 1 -- 172.21.15.77:0/3067220463 shutdown_connections 2024-09-10T10:00:02.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.076+0000 7f476d2a4740 1 --2- 172.21.15.77:0/3067220463 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5589ff9e6d70 0x5589ff9e9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:02.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.076+0000 7f476d2a4740 1 --2- 172.21.15.77:0/3067220463 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5589ff9dc4b0 0x5589ff9e6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:02.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.076+0000 7f476d2a4740 1 --2- 172.21.15.77:0/3067220463 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5589ff9dbb90 0x5589ff9dbf70 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:02.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.076+0000 7f476d2a4740 1 -- 172.21.15.77:0/3067220463 >> 172.21.15.77:0/3067220463 conn(0x5589ff9d8120 msgr2=0x5589ff9d8500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:02.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.077+0000 7f476d2a4740 1 -- 172.21.15.77:0/3067220463 shutdown_connections 2024-09-10T10:00:02.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.077+0000 7f476d2a4740 1 -- 172.21.15.77:0/3067220463 wait complete. 2024-09-10T10:00:02.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.077+0000 7f476d2a4740 1 -- start start 2024-09-10T10:00:02.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.077+0000 7f476d2a4740 1 Processor -- start 2024-09-10T10:00:02.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.078+0000 7f476d2a4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5589ff9dbb90 0x5589ff9e3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:02.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.078+0000 7f476d2a4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5589ff9dc4b0 0x5589ff9e41d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:02.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.078+0000 7f476d2a4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5589ff9e6d70 0x5589ff9e4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:02.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.078+0000 7f476d2a4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5589ff9e9860 con 0x5589ff9dc4b0 2024-09-10T10:00:02.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.078+0000 7f476d2a4740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5589ff8fd0f0 con 0x5589ff9e6d70 2024-09-10T10:00:02.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.078+0000 7f476d2a4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5589ff9e99e0 con 0x5589ff9dbb90 2024-09-10T10:00:02.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.078+0000 7f476d2a4740 10 client.0 _ll_register_callbacks cb 0x5589ff8ee9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:00:02.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.078+0000 7f4763400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5589ff9dbb90 0x5589ff9e3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:02.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.078+0000 7f4763400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5589ff9dbb90 0x5589ff9e3c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:59150/0 (socket says 172.21.15.77:59150) 2024-09-10T10:00:02.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.078+0000 7f4763400640 1 -- 172.21.15.77:0/3981937536 learned_addr learned my addr 172.21.15.77:0/3981937536 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:00:02.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.078+0000 7f4762a00640 1 --2- 172.21.15.77:0/3981937536 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5589ff9dc4b0 0x5589ff9e41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:02.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.079+0000 7f4763e00640 1 --2- 172.21.15.77:0/3981937536 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5589ff9e6d70 0x5589ff9e4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:02.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.079+0000 7f4763400640 1 -- 172.21.15.77:0/3981937536 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5589ff9e6d70 msgr2=0x5589ff9e4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:02.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.079+0000 7f4763400640 1 --2- 172.21.15.77:0/3981937536 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5589ff9e6d70 0x5589ff9e4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:02.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.079+0000 7f4763400640 1 -- 172.21.15.77:0/3981937536 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5589ff9dc4b0 msgr2=0x5589ff9e41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:02.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.079+0000 7f4763400640 1 --2- 172.21.15.77:0/3981937536 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5589ff9dc4b0 0x5589ff9e41d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:02.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.079+0000 7f4763400640 1 -- 172.21.15.77:0/3981937536 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589ff6b33c0 con 0x5589ff9dbb90 2024-09-10T10:00:02.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.079+0000 7f4763e00640 1 --2- 172.21.15.77:0/3981937536 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5589ff9e6d70 0x5589ff9e4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:00:02.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.079+0000 7f4762a00640 1 --2- 172.21.15.77:0/3981937536 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5589ff9dc4b0 0x5589ff9e41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:00:02.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.079+0000 7f4763400640 1 --2- 172.21.15.77:0/3981937536 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5589ff9dbb90 0x5589ff9e3c90 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f475400f8d0 tx=0x7f4754004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=734b9ae0041d2342 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:02.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.079+0000 7f474d600640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-10T10:00:02.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.080+0000 7f474d600640 1 -- 172.21.15.77:0/3981937536 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f475400fe50 con 0x5589ff9dbb90 2024-09-10T10:00:02.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.080+0000 7f476d2a4740 1 -- 172.21.15.77:0/3981937536 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f475c036030 con 0x5589ff9dbb90 2024-09-10T10:00:02.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.080+0000 7f474d600640 1 -- 172.21.15.77:0/3981937536 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f4754011b50 con 0x5589ff9dbb90 2024-09-10T10:00:02.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.080+0000 7f474d600640 1 -- 172.21.15.77:0/3981937536 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4754011d50 con 0x5589ff9dbb90 2024-09-10T10:00:02.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.080+0000 7f476d2a4740 10 client.26407 fetch_fsmap learned FSMap version 32 2024-09-10T10:00:02.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.080+0000 7f476d2a4740 1 -- 172.21.15.77:0/3981937536 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5589ff9f55f0 con 0x5589ff9dbb90 2024-09-10T10:00:02.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.080+0000 7f474d600640 1 -- 172.21.15.77:0/3981937536 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f475c036030 con 0x5589ff9dbb90 2024-09-10T10:00:02.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.081+0000 7f474d600640 1 -- 172.21.15.77:0/3981937536 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f4754027020 con 0x5589ff9dbb90 2024-09-10T10:00:02.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.081+0000 7f476d2a4740 10 client.26407 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T10:00:02.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.081+0000 7f476d2a4740 10 client.26407 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T10:00:02.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.081+0000 7f476d2a4740 20 client.26407 populate_metadata read hostname 'smithi077' 2024-09-10T10:00:02.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.081+0000 7f476d2a4740 1 --2- 172.21.15.77:0/3981937536 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x5589ff9f6b50 0x5589ff9f8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:02.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.081+0000 7f476d2a4740 4 client.26407 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:00:02.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.081+0000 7f476d2a4740 1 -- 172.21.15.77:0/3981937536 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5589ff94f950 con 0x5589ff9f6b50 2024-09-10T10:00:02.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.081+0000 7f4762a00640 1 --2- 172.21.15.77:0/3981937536 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x5589ff9f6b50 0x5589ff9f8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:02.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.082+0000 7f474d600640 10 client.26407 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T10:00:02.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.082+0000 7f4762a00640 1 --2- 172.21.15.77:0/3981937536 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x5589ff9f6b50 0x5589ff9f8ff0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b919c45730ee52da server_cookie=822630efaa1704c6 in_seq=0 out_seq=0 2024-09-10T10:00:02.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.083+0000 7f474d600640 1 -- 172.21.15.77:0/3981937536 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5589ff94f950 con 0x5589ff9f6b50 2024-09-10T10:00:02.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.083+0000 7f474d600640 10 client.26407 handle_command_reply: tid=0 2024-09-10T10:00:02.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.083+0000 7f476d2a4740 1 client.26407 shutdown 2024-09-10T10:00:02.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.083+0000 7f476d2a4740 1 -- 172.21.15.77:0/3981937536 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5589ff9dbb90 msgr2=0x5589ff9e3c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:02.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.083+0000 7f476d2a4740 1 --2- 172.21.15.77:0/3981937536 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5589ff9dbb90 0x5589ff9e3c90 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f475400f8d0 tx=0x7f4754004730 comp rx=0 tx=0).stop 2024-09-10T10:00:02.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.083+0000 7f476d2a4740 1 -- 172.21.15.77:0/3981937536 shutdown_connections 2024-09-10T10:00:02.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.083+0000 7f476d2a4740 1 --2- 172.21.15.77:0/3981937536 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x5589ff9f6b50 0x5589ff9f8ff0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:02.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.083+0000 7f476d2a4740 1 --2- 172.21.15.77:0/3981937536 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5589ff9e6d70 0x5589ff9e4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:02.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.083+0000 7f476d2a4740 1 --2- 172.21.15.77:0/3981937536 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5589ff9dc4b0 0x5589ff9e41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:02.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.083+0000 7f476d2a4740 1 --2- 172.21.15.77:0/3981937536 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5589ff9dbb90 0x5589ff9e3c90 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:02.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.083+0000 7f474d600640 0 client.26407 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T10:00:02.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.083+0000 7f476d2a4740 1 -- 172.21.15.77:0/3981937536 >> 172.21.15.77:0/3981937536 conn(0x5589ff9d8120 msgr2=0x5589ff9dde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:02.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.083+0000 7f476d2a4740 1 -- 172.21.15.77:0/3981937536 shutdown_connections 2024-09-10T10:00:02.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.083+0000 7f476d2a4740 1 -- 172.21.15.77:0/3981937536 wait complete. 2024-09-10T10:00:02.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.083+0000 7f476d2a4740 20 client.26407 trim_cache size 0 max 16384 2024-09-10T10:00:02.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.101+0000 7f476d2a4740 1 Processor -- start 2024-09-10T10:00:02.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.102+0000 7f476d2a4740 1 -- start start 2024-09-10T10:00:02.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.102+0000 7f476d2a4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5589ff9dbb90 0x5589ff9fe8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:02.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.102+0000 7f476d2a4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5589ff9dc4b0 0x5589ff9fee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:02.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.102+0000 7f476d2a4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5589ff9e6d70 0x5589ff9e0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:02.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.102+0000 7f476d2a4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f47641b5300 con 0x5589ff9dc4b0 2024-09-10T10:00:02.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.102+0000 7f476d2a4740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f47641b6f00 con 0x5589ff9dbb90 2024-09-10T10:00:02.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.102+0000 7f476d2a4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f47641b59d0 con 0x5589ff9e6d70 2024-09-10T10:00:02.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.102+0000 7f4762a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5589ff9dc4b0 0x5589ff9fee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:02.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.102+0000 7f4763e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5589ff9e6d70 0x5589ff9e0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:02.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.102+0000 7f4762a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5589ff9dc4b0 0x5589ff9fee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:37708/0 (socket says 172.21.15.77:37708) 2024-09-10T10:00:02.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.102+0000 7f4762a00640 1 -- 172.21.15.77:0/2156585575 learned_addr learned my addr 172.21.15.77:0/2156585575 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:00:02.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.102+0000 7f4763400640 1 --2- 172.21.15.77:0/2156585575 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5589ff9dbb90 0x5589ff9fe8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:02.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.103+0000 7f4763e00640 1 -- 172.21.15.77:0/2156585575 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5589ff9dbb90 msgr2=0x5589ff9fe8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:02.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.103+0000 7f4763e00640 1 --2- 172.21.15.77:0/2156585575 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5589ff9dbb90 0x5589ff9fe8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:02.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.103+0000 7f4763e00640 1 -- 172.21.15.77:0/2156585575 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5589ff9dc4b0 msgr2=0x5589ff9fee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:02.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.103+0000 7f4763e00640 1 --2- 172.21.15.77:0/2156585575 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5589ff9dc4b0 0x5589ff9fee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:02.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.103+0000 7f4763e00640 1 -- 172.21.15.77:0/2156585575 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589ff9d7df0 con 0x5589ff9e6d70 2024-09-10T10:00:02.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.103+0000 7f4762a00640 1 --2- 172.21.15.77:0/2156585575 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5589ff9dc4b0 0x5589ff9fee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:00:02.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.103+0000 7f4763e00640 1 --2- 172.21.15.77:0/2156585575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5589ff9e6d70 0x5589ff9e0f50 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f475c004af0 tx=0x7f475c031930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ebd9e806aebf3d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:02.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.104+0000 7f474fe00640 1 -- 172.21.15.77:0/2156585575 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f475c054070 con 0x5589ff9e6d70 2024-09-10T10:00:02.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.104+0000 7f474fe00640 1 -- 172.21.15.77:0/2156585575 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f475c036040 con 0x5589ff9e6d70 2024-09-10T10:00:02.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.104+0000 7f474fe00640 1 -- 172.21.15.77:0/2156585575 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f475c004040 con 0x5589ff9e6d70 2024-09-10T10:00:02.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.104+0000 7f476d2a4740 1 -- 172.21.15.77:0/2156585575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5589ff9e6d70 msgr2=0x5589ff9e0f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:02.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.104+0000 7f476d2a4740 1 --2- 172.21.15.77:0/2156585575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5589ff9e6d70 0x5589ff9e0f50 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f475c004af0 tx=0x7f475c031930 comp rx=0 tx=0).stop 2024-09-10T10:00:02.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.104+0000 7f476d2a4740 1 -- 172.21.15.77:0/2156585575 shutdown_connections 2024-09-10T10:00:02.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.105+0000 7f476d2a4740 1 --2- 172.21.15.77:0/2156585575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5589ff9e6d70 0x5589ff9e0f50 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:02.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.105+0000 7f476d2a4740 1 --2- 172.21.15.77:0/2156585575 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5589ff9dc4b0 0x5589ff9fee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:02.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.105+0000 7f476d2a4740 1 --2- 172.21.15.77:0/2156585575 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5589ff9dbb90 0x5589ff9fe8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:02.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.105+0000 7f476d2a4740 1 -- 172.21.15.77:0/2156585575 >> 172.21.15.77:0/2156585575 conn(0x5589ff9d8120 msgr2=0x5589ff9e25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:02.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.105+0000 7f476d2a4740 1 -- 172.21.15.77:0/2156585575 shutdown_connections 2024-09-10T10:00:02.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.105+0000 7f476d2a4740 1 -- 172.21.15.77:0/2156585575 wait complete. 2024-09-10T10:00:02.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.105+0000 7f476d2a4740 1 -- start start 2024-09-10T10:00:02.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.105+0000 7f476d2a4740 1 Processor -- start 2024-09-10T10:00:02.113 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.106+0000 7f476d2a4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5589ff9dbb90 0x5589ff9eec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:02.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.106+0000 7f476d2a4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5589ff9dc4b0 0x5589ff9ef170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:02.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.106+0000 7f476d2a4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5589ff9e6d70 0x5589ff9ebc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:02.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.106+0000 7f476d2a4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f47641b6770 con 0x5589ff9dbb90 2024-09-10T10:00:02.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.106+0000 7f476d2a4740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f47641b60a0 con 0x5589ff9dc4b0 2024-09-10T10:00:02.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.106+0000 7f476d2a4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f47641b32e0 con 0x5589ff9e6d70 2024-09-10T10:00:02.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.106+0000 7f476d2a4740 10 client.0 _ll_register_callbacks cb 0x7f475400f760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:00:02.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.106+0000 7f4763400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5589ff9dbb90 0x5589ff9eec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:02.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.106+0000 7f4762a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5589ff9dc4b0 0x5589ff9ef170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:02.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.106+0000 7f4763400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5589ff9dbb90 0x5589ff9eec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:37724/0 (socket says 172.21.15.77:37724) 2024-09-10T10:00:02.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.107+0000 7f4763400640 1 -- 172.21.15.77:0/1991527916 learned_addr learned my addr 172.21.15.77:0/1991527916 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:00:02.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.107+0000 7f4763e00640 1 --2- 172.21.15.77:0/1991527916 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5589ff9e6d70 0x5589ff9ebc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:02.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.107+0000 7f4762a00640 1 -- 172.21.15.77:0/1991527916 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5589ff9e6d70 msgr2=0x5589ff9ebc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:02.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.107+0000 7f4762a00640 1 --2- 172.21.15.77:0/1991527916 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5589ff9e6d70 0x5589ff9ebc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:02.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.107+0000 7f4762a00640 1 -- 172.21.15.77:0/1991527916 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5589ff9dbb90 msgr2=0x5589ff9eec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:02.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.107+0000 7f4762a00640 1 --2- 172.21.15.77:0/1991527916 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5589ff9dbb90 0x5589ff9eec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:02.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.107+0000 7f4762a00640 1 -- 172.21.15.77:0/1991527916 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589ff9ec140 con 0x5589ff9dc4b0 2024-09-10T10:00:02.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.107+0000 7f4763400640 1 --2- 172.21.15.77:0/1991527916 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5589ff9dbb90 0x5589ff9eec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:00:02.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.107+0000 7f4762a00640 1 --2- 172.21.15.77:0/1991527916 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5589ff9dc4b0 0x5589ff9ef170 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f4750098070 tx=0x7f47500d2d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca62ab9d7e5a941d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:02.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.107+0000 7f474d600640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T10:00:02.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.107+0000 7f474d600640 1 -- 172.21.15.77:0/1991527916 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f475001c040 con 0x5589ff9dc4b0 2024-09-10T10:00:02.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.108+0000 7f476d2a4740 1 -- 172.21.15.77:0/1991527916 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f475c036040 con 0x5589ff9dc4b0 2024-09-10T10:00:02.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.108+0000 7f474d600640 1 -- 172.21.15.77:0/1991527916 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f47500ccaa0 con 0x5589ff9dc4b0 2024-09-10T10:00:02.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.108+0000 7f474d600640 1 -- 172.21.15.77:0/1991527916 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f475009b050 con 0x5589ff9dc4b0 2024-09-10T10:00:02.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.108+0000 7f476d2a4740 10 client.26459 fetch_fsmap learned FSMap version 32 2024-09-10T10:00:02.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.108+0000 7f476d2a4740 1 -- 172.21.15.77:0/1991527916 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5589ffa61f10 con 0x5589ff9dc4b0 2024-09-10T10:00:02.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.108+0000 7f474d600640 1 -- 172.21.15.77:0/1991527916 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f475c036040 con 0x5589ff9dc4b0 2024-09-10T10:00:02.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.108+0000 7f474d600640 1 -- 172.21.15.77:0/1991527916 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f475009b950 con 0x5589ff9dc4b0 2024-09-10T10:00:02.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.108+0000 7f476d2a4740 10 client.26459 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T10:00:02.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.108+0000 7f476d2a4740 10 client.26459 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T10:00:02.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.108+0000 7f476d2a4740 20 client.26459 populate_metadata read hostname 'smithi077' 2024-09-10T10:00:02.117 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.108+0000 7f476d2a4740 1 --2- 172.21.15.77:0/1991527916 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x5589ffa62700 0x5589ffa64b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:02.117 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.108+0000 7f476d2a4740 4 client.26459 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T10:00:02.117 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.108+0000 7f476d2a4740 1 -- 172.21.15.77:0/1991527916 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5589ff949e60 con 0x5589ffa62700 2024-09-10T10:00:02.117 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.109+0000 7f4763400640 1 --2- 172.21.15.77:0/1991527916 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x5589ffa62700 0x5589ffa64b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:02.117 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.109+0000 7f4763400640 1 --2- 172.21.15.77:0/1991527916 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x5589ffa62700 0x5589ffa64b20 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f08be3711ddd471 server_cookie=7b4f290826e2fcb0 in_seq=0 out_seq=0 2024-09-10T10:00:02.117 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.109+0000 7f474d600640 10 client.26459 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T10:00:02.117 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.109+0000 7f474d600640 1 -- 172.21.15.77:0/1991527916 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5589ff949e60 con 0x5589ffa62700 2024-09-10T10:00:02.117 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.109+0000 7f474d600640 10 client.26459 handle_command_reply: tid=0 2024-09-10T10:00:02.117 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.109+0000 7f476d2a4740 1 client.26459 shutdown 2024-09-10T10:00:02.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.110+0000 7f476d2a4740 1 -- 172.21.15.77:0/1991527916 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5589ff9dc4b0 msgr2=0x5589ff9ef170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:02.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.110+0000 7f476d2a4740 1 --2- 172.21.15.77:0/1991527916 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5589ff9dc4b0 0x5589ff9ef170 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f4750098070 tx=0x7f47500d2d20 comp rx=0 tx=0).stop 2024-09-10T10:00:02.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.110+0000 7f476d2a4740 1 -- 172.21.15.77:0/1991527916 shutdown_connections 2024-09-10T10:00:02.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.110+0000 7f476d2a4740 1 --2- 172.21.15.77:0/1991527916 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x5589ffa62700 0x5589ffa64b20 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:02.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.110+0000 7f476d2a4740 1 --2- 172.21.15.77:0/1991527916 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5589ff9e6d70 0x5589ff9ebc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:02.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.110+0000 7f476d2a4740 1 --2- 172.21.15.77:0/1991527916 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5589ff9dc4b0 0x5589ff9ef170 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:02.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.110+0000 7f476d2a4740 1 --2- 172.21.15.77:0/1991527916 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5589ff9dbb90 0x5589ff9eec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:02.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.110+0000 7f476d2a4740 1 -- 172.21.15.77:0/1991527916 >> 172.21.15.77:0/1991527916 conn(0x5589ff9d8120 msgr2=0x5589ff9e54f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:02.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.110+0000 7f474d600640 0 client.26459 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T10:00:02.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.110+0000 7f476d2a4740 1 -- 172.21.15.77:0/1991527916 shutdown_connections 2024-09-10T10:00:02.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.110+0000 7f476d2a4740 1 -- 172.21.15.77:0/1991527916 wait complete. 2024-09-10T10:00:02.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.110+0000 7f476d2a4740 20 client.26459 trim_cache size 0 max 16384 2024-09-10T10:00:02.119 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-10T10:00:02.119 INFO:teuthology.orchestra.run.smithi077.stdout: "status": "no active scrubs running", 2024-09-10T10:00:02.119 INFO:teuthology.orchestra.run.smithi077.stdout: "scrubs": {} 2024-09-10T10:00:02.119 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-10T10:00:02.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.111+0000 7f476a800640 1 -- 172.21.15.77:0/4039352472 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f47480a2c00 msgr2=0x7f47480a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:02.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.111+0000 7f476a800640 1 --2- 172.21.15.77:0/4039352472 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f47480a2c00 0x7f47480a50c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x5589ff9d8f80 tx=0x7f4754005f90 comp rx=0 tx=0).stop 2024-09-10T10:00:02.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.111+0000 7f476a800640 1 -- 172.21.15.77:0/4039352472 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47641049a0 msgr2=0x7f476410db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:02.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.111+0000 7f476a800640 1 --2- 172.21.15.77:0/4039352472 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47641049a0 0x7f476410db30 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f475000e9f0 tx=0x7f475000eeb0 comp rx=0 tx=0).stop 2024-09-10T10:00:02.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.111+0000 7f476a800640 1 -- 172.21.15.77:0/4039352472 shutdown_connections 2024-09-10T10:00:02.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.111+0000 7f476a800640 1 --2- 172.21.15.77:0/4039352472 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f47480a2c00 0x7f47480a50c0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:02.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.111+0000 7f476a800640 1 --2- 172.21.15.77:0/4039352472 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4764109120 0x7f476410e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:02.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.111+0000 7f476a800640 1 --2- 172.21.15.77:0/4039352472 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47641049a0 0x7f476410db30 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:02.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.111+0000 7f476a800640 1 --2- 172.21.15.77:0/4039352472 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4764103ff0 0x7f4764110a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:02.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.111+0000 7f476a800640 1 -- 172.21.15.77:0/4039352472 >> 172.21.15.77:0/4039352472 conn(0x7f47640ffa90 msgr2=0x7f47641005b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:02.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.111+0000 7f476a800640 1 -- 172.21.15.77:0/4039352472 shutdown_connections 2024-09-10T10:00:02.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:02.111+0000 7f476a800640 1 -- 172.21.15.77:0/4039352472 wait complete. 2024-09-10T10:00:02.127 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:00:03.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:00:02 smithi120 ceph-mon[34770]: pgmap v537: 129 pgs: 129 active+clean; 97 MiB data, 935 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 KiB/s wr, 10 op/s 2024-09-10T10:00:03.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:00:02 smithi077 ceph-mon[30319]: pgmap v537: 129 pgs: 129 active+clean; 97 MiB data, 935 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 KiB/s wr, 10 op/s 2024-09-10T10:00:03.227 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T10:00:03.227 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:00:03.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:00:02 smithi134 ceph-mon[35674]: pgmap v537: 129 pgs: 129 active+clean; 97 MiB data, 935 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 KiB/s wr, 10 op/s 2024-09-10T10:00:03.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:03.408+0000 7fa27b200640 1 -- 172.21.15.77:0/356621049 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa27c0745c0 msgr2=0x7fa27c0749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:03.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:03.408+0000 7fa27b200640 1 --2- 172.21.15.77:0/356621049 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa27c0745c0 0x7fa27c0749a0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fa26c009900 tx=0x7fa26c02a480 comp rx=0 tx=0).stop 2024-09-10T10:00:03.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:03.408+0000 7fa27b200640 1 -- 172.21.15.77:0/356621049 shutdown_connections 2024-09-10T10:00:03.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:03.408+0000 7fa27b200640 1 --2- 172.21.15.77:0/356621049 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa27c10e260 0x7fa27c110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:03.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:03.408+0000 7fa27b200640 1 --2- 172.21.15.77:0/356621049 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa27c074ee0 0x7fa27c10dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:03.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:03.408+0000 7fa27b200640 1 --2- 172.21.15.77:0/356621049 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa27c0745c0 0x7fa27c0749a0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:03.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:03.408+0000 7fa27b200640 1 -- 172.21.15.77:0/356621049 >> 172.21.15.77:0/356621049 conn(0x7fa27c0fd740 msgr2=0x7fa27c0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:03.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:03.409+0000 7fa27b200640 1 -- 172.21.15.77:0/356621049 shutdown_connections 2024-09-10T10:00:03.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:03.409+0000 7fa27b200640 1 -- 172.21.15.77:0/356621049 wait complete. 2024-09-10T10:00:03.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:03.410+0000 7fa27b200640 1 Processor -- start 2024-09-10T10:00:03.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:03.410+0000 7fa27b200640 1 -- start start 2024-09-10T10:00:03.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:03.410+0000 7fa27b200640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa27c0745c0 0x7fa27c19a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:03.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:03.410+0000 7fa27b200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa27c074ee0 0x7fa27c19aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:03.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:03.410+0000 7fa27b200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa27c10e260 0x7fa27c19ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:03.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:03.410+0000 7fa27b200640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa27c113fb0 con 0x7fa27c074ee0 2024-09-10T10:00:03.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:03.410+0000 7fa27b200640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa27c113e30 con 0x7fa27c0745c0 2024-09-10T10:00:03.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:03.410+0000 7fa27b200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa27c114130 con 0x7fa27c10e260 2024-09-10T10:00:03.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:03.410+0000 7fa279400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa27c074ee0 0x7fa27c19aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:03.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:03.411+0000 7fa279400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa27c074ee0 0x7fa27c19aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:51314/0 (socket says 172.21.15.77:51314) 2024-09-10T10:00:03.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:03.411+0000 7fa279400640 1 -- 172.21.15.77:0/2405106725 learned_addr learned my addr 172.21.15.77:0/2405106725 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:00:03.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:03.411+0000 7fa279e00640 1 --2- 172.21.15.77:0/2405106725 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa27c0745c0 0x7fa27c19a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:03.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:03.411+0000 7fa27a800640 1 --2- 172.21.15.77:0/2405106725 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa27c10e260 0x7fa27c19ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:03.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:03.411+0000 7fa279400640 1 -- 172.21.15.77:0/2405106725 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa27c10e260 msgr2=0x7fa27c19ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:03.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:03.411+0000 7fa279400640 1 --2- 172.21.15.77:0/2405106725 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa27c10e260 0x7fa27c19ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:03.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:03.411+0000 7fa279400640 1 -- 172.21.15.77:0/2405106725 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa27c0745c0 msgr2=0x7fa27c19a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:03.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:03.411+0000 7fa279400640 1 --2- 172.21.15.77:0/2405106725 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa27c0745c0 0x7fa27c19a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:03.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:03.411+0000 7fa279400640 1 -- 172.21.15.77:0/2405106725 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa27c19f470 con 0x7fa27c074ee0 2024-09-10T10:00:03.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:03.411+0000 7fa279400640 1 --2- 172.21.15.77:0/2405106725 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa27c074ee0 0x7fa27c19aa00 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fa26800e9f0 tx=0x7fa26800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:03.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:03.412+0000 7fa272a00640 1 -- 172.21.15.77:0/2405106725 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa26800cd80 con 0x7fa27c074ee0 2024-09-10T10:00:03.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:03.412+0000 7fa272a00640 1 -- 172.21.15.77:0/2405106725 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa268005500 con 0x7fa27c074ee0 2024-09-10T10:00:03.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:03.412+0000 7fa27b200640 1 -- 172.21.15.77:0/2405106725 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa27c19f760 con 0x7fa27c074ee0 2024-09-10T10:00:03.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:03.412+0000 7fa272a00640 1 -- 172.21.15.77:0/2405106725 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa268010690 con 0x7fa27c074ee0 2024-09-10T10:00:03.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:03.412+0000 7fa27b200640 1 -- 172.21.15.77:0/2405106725 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa27c1a7040 con 0x7fa27c074ee0 2024-09-10T10:00:03.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:03.413+0000 7fa27b200640 1 -- 172.21.15.77:0/2405106725 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa27c0749a0 con 0x7fa27c074ee0 2024-09-10T10:00:03.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:03.415+0000 7fa272a00640 1 -- 172.21.15.77:0/2405106725 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa2680046c0 con 0x7fa27c074ee0 2024-09-10T10:00:03.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:03.416+0000 7fa272a00640 1 --2- 172.21.15.77:0/2405106725 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fa2600a2b80 0x7fa2600a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:03.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:03.417+0000 7fa272a00640 1 -- 172.21.15.77:0/2405106725 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fa2680cf870 con 0x7fa27c074ee0 2024-09-10T10:00:03.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:03.417+0000 7fa279e00640 1 --2- 172.21.15.77:0/2405106725 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fa2600a2b80 0x7fa2600a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:03.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:03.417+0000 7fa279e00640 1 --2- 172.21.15.77:0/2405106725 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fa2600a2b80 0x7fa2600a5040 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fa26c009230 tx=0x7fa26c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:03.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:03.421+0000 7fa272a00640 1 -- 172.21.15.77:0/2405106725 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa26809c380 con 0x7fa27c074ee0 2024-09-10T10:00:03.671 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:00:03.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:03.678+0000 7fa27b200640 1 -- 172.21.15.77:0/2405106725 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa27c02d7c0 con 0x7fa27c074ee0 2024-09-10T10:00:03.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:03.679+0000 7fa272a00640 1 -- 172.21.15.77:0/2405106725 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11996 (secure 0 0 0) 0x7fa26809bc80 con 0x7fa27c074ee0 2024-09-10T10:00:03.681 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T10:00:03.681 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":32,"btime":"2024-09-10T09:57:40:888996+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3159289675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3159289675},{"type":"v1","addr":"172.21.15.77:6835","nonce":3159289675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1951962341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1951962341},{"type":"v1","addr":"172.21.15.77:6837","nonce":1951962341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:52:23.368252+0000","modified":"2024-09-10T09:57:40.695191+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24563,"mds_1":24524,"mds_2":24520,"mds_3":24502,"mds_4":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14487":{"gid":14487,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/4291121888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":4291121888},{"type":"v1","addr":"172.21.15.77:6839","nonce":4291121888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14499":{"gid":14499,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6841/445284194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":445284194},{"type":"v1","addr":"172.21.15.77:6841","nonce":445284194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1005054423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1005054423},{"type":"v1","addr":"172.21.15.134:6833","nonce":1005054423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6833/3716307744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":3716307744},{"type":"v1","addr":"172.21.15.120:6833","nonce":3716307744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/1319892544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1319892544},{"type":"v1","addr":"172.21.15.120:6835","nonce":1319892544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.134:6835/3293933699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3293933699},{"type":"v1","addr":"172.21.15.134:6835","nonce":3293933699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.134:6837/2461497091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2461497091},{"type":"v1","addr":"172.21.15.134:6837","nonce":2461497091}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":20,"addr":"172.21.15.134:6839/1965117680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1965117680},{"type":"v1","addr":"172.21.15.134:6839","nonce":1965117680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":20,"addr":"172.21.15.120:6837/3033280709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3033280709},{"type":"v1","addr":"172.21.15.120:6837","nonce":3033280709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.120:6839/1777710157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1777710157},{"type":"v1","addr":"172.21.15.120:6839","nonce":1777710157}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24563,"qdb_cluster":[24563,24524,24520,24502,24496]},"id":1}]} 2024-09-10T10:00:03.682 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 32 2024-09-10T10:00:03.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:03.682+0000 7fa27b200640 1 -- 172.21.15.77:0/2405106725 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fa2600a2b80 msgr2=0x7fa2600a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:03.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:03.682+0000 7fa27b200640 1 --2- 172.21.15.77:0/2405106725 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fa2600a2b80 0x7fa2600a5040 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fa26c009230 tx=0x7fa26c0058d0 comp rx=0 tx=0).stop 2024-09-10T10:00:03.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:03.682+0000 7fa27b200640 1 -- 172.21.15.77:0/2405106725 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa27c074ee0 msgr2=0x7fa27c19aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:03.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:03.682+0000 7fa27b200640 1 --2- 172.21.15.77:0/2405106725 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa27c074ee0 0x7fa27c19aa00 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fa26800e9f0 tx=0x7fa26800edf0 comp rx=0 tx=0).stop 2024-09-10T10:00:03.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:03.682+0000 7fa27b200640 1 -- 172.21.15.77:0/2405106725 shutdown_connections 2024-09-10T10:00:03.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:03.682+0000 7fa27b200640 1 --2- 172.21.15.77:0/2405106725 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fa2600a2b80 0x7fa2600a5040 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:03.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:03.682+0000 7fa27b200640 1 --2- 172.21.15.77:0/2405106725 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa27c10e260 0x7fa27c19ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:03.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:03.682+0000 7fa27b200640 1 --2- 172.21.15.77:0/2405106725 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa27c074ee0 0x7fa27c19aa00 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:03.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:03.682+0000 7fa27b200640 1 --2- 172.21.15.77:0/2405106725 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa27c0745c0 0x7fa27c19a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:03.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:03.682+0000 7fa27b200640 1 -- 172.21.15.77:0/2405106725 >> 172.21.15.77:0/2405106725 conn(0x7fa27c0fd740 msgr2=0x7fa27c0feed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:03.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:03.683+0000 7fa27b200640 1 -- 172.21.15.77:0/2405106725 shutdown_connections 2024-09-10T10:00:03.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:03.683+0000 7fa27b200640 1 -- 172.21.15.77:0/2405106725 wait complete. 2024-09-10T10:00:03.939 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:00:03 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/2405106725' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:00:04.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:00:03 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/2405106725' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:00:04.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:00:03 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/2405106725' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:00:04.978 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24563 quiesce db --set-id 90af319d --release --await' 2024-09-10T10:00:04.979 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24563 quiesce db --set-id 90af319d --release --await 2024-09-10T10:00:05.117 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:04.596+0000 7f9bd0400640 1 -- 172.21.15.77:0/4184104471 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9bc8109140 msgr2=0x7f9bc8110cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:05.117 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:04.596+0000 7f9bd0400640 1 --2- 172.21.15.77:0/4184104471 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9bc8109140 0x7f9bc8110cc0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9bc0009ca0 tx=0x7f9bc002a8c0 comp rx=0 tx=0).stop 2024-09-10T10:00:05.117 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:04.596+0000 7f9bd0400640 1 -- 172.21.15.77:0/4184104471 shutdown_connections 2024-09-10T10:00:05.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:04.596+0000 7f9bd0400640 1 --2- 172.21.15.77:0/4184104471 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9bc8109140 0x7f9bc8110cc0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:05.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:04.596+0000 7f9bd0400640 1 --2- 172.21.15.77:0/4184104471 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9bc81049c0 0x7f9bc8108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:05.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:04.596+0000 7f9bd0400640 1 --2- 172.21.15.77:0/4184104471 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9bc8104010 0x7f9bc81043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:05.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:04.596+0000 7f9bd0400640 1 -- 172.21.15.77:0/4184104471 >> 172.21.15.77:0/4184104471 conn(0x7f9bc80ffa50 msgr2=0x7f9bc8101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:05.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:04.596+0000 7f9bd0400640 1 -- 172.21.15.77:0/4184104471 shutdown_connections 2024-09-10T10:00:05.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:04.596+0000 7f9bd0400640 1 -- 172.21.15.77:0/4184104471 wait complete. 2024-09-10T10:00:05.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:04.597+0000 7f9bd0400640 1 Processor -- start 2024-09-10T10:00:05.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:04.597+0000 7f9bd0400640 1 -- start start 2024-09-10T10:00:05.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:04.597+0000 7f9bd0400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9bc8104010 0x7f9bc819a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:05.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:04.597+0000 7f9bd0400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9bc81049c0 0x7f9bc819ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:05.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:04.597+0000 7f9bd0400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9bc8109140 0x7f9bc819ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:05.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:04.598+0000 7f9bd0400640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9bc81141c0 con 0x7f9bc8109140 2024-09-10T10:00:05.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:04.598+0000 7f9bd0400640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9bc8114040 con 0x7f9bc8104010 2024-09-10T10:00:05.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:04.598+0000 7f9bd0400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9bc8114340 con 0x7f9bc81049c0 2024-09-10T10:00:05.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:04.598+0000 7f9bcda00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9bc8104010 0x7f9bc819a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:05.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:04.598+0000 7f9bce400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9bc8109140 0x7f9bc819ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:05.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:04.598+0000 7f9bcda00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9bc8104010 0x7f9bc819a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:60846/0 (socket says 172.21.15.77:60846) 2024-09-10T10:00:05.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:04.598+0000 7f9bcda00640 1 -- 172.21.15.77:0/1231698333 learned_addr learned my addr 172.21.15.77:0/1231698333 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:00:05.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:04.598+0000 7f9bcd000640 1 --2- 172.21.15.77:0/1231698333 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9bc81049c0 0x7f9bc819ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:05.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:04.598+0000 7f9bcda00640 1 -- 172.21.15.77:0/1231698333 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9bc81049c0 msgr2=0x7f9bc819ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:05.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:04.598+0000 7f9bcda00640 1 --2- 172.21.15.77:0/1231698333 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9bc81049c0 0x7f9bc819ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:05.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:04.598+0000 7f9bcda00640 1 -- 172.21.15.77:0/1231698333 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9bc8109140 msgr2=0x7f9bc819ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:05.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:04.598+0000 7f9bcda00640 1 --2- 172.21.15.77:0/1231698333 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9bc8109140 0x7f9bc819ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:05.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:04.598+0000 7f9bcda00640 1 -- 172.21.15.77:0/1231698333 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bc819f670 con 0x7f9bc8104010 2024-09-10T10:00:05.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:04.598+0000 7f9bcda00640 1 --2- 172.21.15.77:0/1231698333 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9bc8104010 0x7f9bc819a6c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc00e9f0 tx=0x7f9bbc00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:05.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:04.599+0000 7f9bc6000640 1 -- 172.21.15.77:0/1231698333 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bbc00cd80 con 0x7f9bc8104010 2024-09-10T10:00:05.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:04.599+0000 7f9bd0400640 1 -- 172.21.15.77:0/1231698333 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bc819f960 con 0x7f9bc8104010 2024-09-10T10:00:05.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:04.599+0000 7f9bc6000640 1 -- 172.21.15.77:0/1231698333 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9bbc005500 con 0x7f9bc8104010 2024-09-10T10:00:05.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:04.599+0000 7f9bc6000640 1 -- 172.21.15.77:0/1231698333 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bbc010690 con 0x7f9bc8104010 2024-09-10T10:00:05.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:04.599+0000 7f9bd0400640 1 -- 172.21.15.77:0/1231698333 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bc81a7240 con 0x7f9bc8104010 2024-09-10T10:00:05.120 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T10:00:05.120 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":32,"btime":"2024-09-10T09:57:40:888996+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3159289675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3159289675},{"type":"v1","addr":"172.21.15.77:6835","nonce":3159289675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1951962341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1951962341},{"type":"v1","addr":"172.21.15.77:6837","nonce":1951962341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:52:23.368252+0000","modified":"2024-09-10T09:57:40.695191+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24563,"mds_1":24524,"mds_2":24520,"mds_3":24502,"mds_4":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14487":{"gid":14487,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/4291121888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":4291121888},{"type":"v1","addr":"172.21.15.77:6839","nonce":4291121888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14499":{"gid":14499,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6841/445284194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":445284194},{"type":"v1","addr":"172.21.15.77:6841","nonce":445284194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1005054423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1005054423},{"type":"v1","addr":"172.21.15.134:6833","nonce":1005054423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6833/3716307744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":3716307744},{"type":"v1","addr":"172.21.15.120:6833","nonce":3716307744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/1319892544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1319892544},{"type":"v1","addr":"172.21.15.120:6835","nonce":1319892544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.134:6835/3293933699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3293933699},{"type":"v1","addr":"172.21.15.134:6835","nonce":3293933699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.134:6837/2461497091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2461497091},{"type":"v1","addr":"172.21.15.134:6837","nonce":2461497091}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":20,"addr":"172.21.15.134:6839/1965117680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1965117680},{"type":"v1","addr":"172.21.15.134:6839","nonce":1965117680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":20,"addr":"172.21.15.120:6837/3033280709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3033280709},{"type":"v1","addr":"172.21.15.120:6837","nonce":3033280709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.120:6839/1777710157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1777710157},{"type":"v1","addr":"172.21.15.120:6839","nonce":1777710157}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24563,"qdb_cluster":[24563,24524,24520,24502,24496]},"id":1}]} 2024-09-10T10:00:05.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:04.600+0000 7f9bd0400640 1 -- 172.21.15.77:0/1231698333 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bc81043f0 con 0x7f9bc8104010 2024-09-10T10:00:05.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:04.602+0000 7f9bc6000640 1 -- 172.21.15.77:0/1231698333 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9bbc0046c0 con 0x7f9bc8104010 2024-09-10T10:00:05.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:04.603+0000 7f9bc6000640 1 --2- 172.21.15.77:0/1231698333 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f9bb00a2c00 0x7f9bb00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:05.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:04.603+0000 7f9bc6000640 1 -- 172.21.15.77:0/1231698333 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f9bbc0cfaa0 con 0x7f9bc8104010 2024-09-10T10:00:05.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:04.603+0000 7f9bcd000640 1 --2- 172.21.15.77:0/1231698333 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f9bb00a2c00 0x7f9bb00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:05.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:04.604+0000 7f9bcd000640 1 --2- 172.21.15.77:0/1231698333 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f9bb00a2c00 0x7f9bb00a50c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f9bc810ab10 tx=0x7f9bb4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:05.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:04.605+0000 7f9bc6000640 1 -- 172.21.15.77:0/1231698333 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9bbc0541b0 con 0x7f9bc8104010 2024-09-10T10:00:05.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:04.823+0000 7f9bd0400640 1 -- 172.21.15.77:0/1231698333 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9bc810c3b0 con 0x7f9bc8104010 2024-09-10T10:00:05.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:04.824+0000 7f9bc6000640 1 -- 172.21.15.77:0/1231698333 <== mon.2 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11996 (secure 0 0 0) 0x7f9bbc09c5b0 con 0x7f9bc8104010 2024-09-10T10:00:05.121 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 32 2024-09-10T10:00:05.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:04.827+0000 7f9bd0400640 1 -- 172.21.15.77:0/1231698333 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f9bb00a2c00 msgr2=0x7f9bb00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:05.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:04.827+0000 7f9bd0400640 1 --2- 172.21.15.77:0/1231698333 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f9bb00a2c00 0x7f9bb00a50c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f9bc810ab10 tx=0x7f9bb4005f90 comp rx=0 tx=0).stop 2024-09-10T10:00:05.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:04.827+0000 7f9bd0400640 1 -- 172.21.15.77:0/1231698333 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9bc8104010 msgr2=0x7f9bc819a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:05.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:04.827+0000 7f9bd0400640 1 --2- 172.21.15.77:0/1231698333 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9bc8104010 0x7f9bc819a6c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc00e9f0 tx=0x7f9bbc00edf0 comp rx=0 tx=0).stop 2024-09-10T10:00:05.122 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:04.827+0000 7f9bd0400640 1 -- 172.21.15.77:0/1231698333 shutdown_connections 2024-09-10T10:00:05.122 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:04.827+0000 7f9bd0400640 1 --2- 172.21.15.77:0/1231698333 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f9bb00a2c00 0x7f9bb00a50c0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:05.122 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:04.827+0000 7f9bd0400640 1 --2- 172.21.15.77:0/1231698333 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9bc8109140 0x7f9bc819ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:05.122 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:04.827+0000 7f9bd0400640 1 --2- 172.21.15.77:0/1231698333 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9bc81049c0 0x7f9bc819ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:05.122 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:04.827+0000 7f9bd0400640 1 --2- 172.21.15.77:0/1231698333 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9bc8104010 0x7f9bc819a6c0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:05.122 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:04.827+0000 7f9bd0400640 1 -- 172.21.15.77:0/1231698333 >> 172.21.15.77:0/1231698333 conn(0x7f9bc80ffa50 msgr2=0x7f9bc8100db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:05.122 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:04.827+0000 7f9bd0400640 1 -- 172.21.15.77:0/1231698333 shutdown_connections 2024-09-10T10:00:05.122 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:04.827+0000 7f9bd0400640 1 -- 172.21.15.77:0/1231698333 wait complete. 2024-09-10T10:00:05.122 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:00:04 smithi077 ceph-mon[30319]: pgmap v538: 129 pgs: 129 active+clean; 97 MiB data, 935 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-09-10T10:00:05.122 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:00:04 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/1231698333' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:00:05.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:00:04 smithi120 ceph-mon[34770]: pgmap v538: 129 pgs: 129 active+clean; 97 MiB data, 935 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-09-10T10:00:05.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:00:04 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/1231698333' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:00:05.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:00:04 smithi134 ceph-mon[35674]: pgmap v538: 129 pgs: 129 active+clean; 97 MiB data, 935 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-09-10T10:00:05.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:00:04 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/1231698333' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:00:05.898 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T10:00:05.924 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:00:06.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.495+0000 7feacf200640 1 -- 172.21.15.77:0/1627206195 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feac81019c0 msgr2=0x7feac8101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:06.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.495+0000 7feacf200640 1 --2- 172.21.15.77:0/1627206195 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feac81019c0 0x7feac8101e20 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7feab8009930 tx=0x7feab802a420 comp rx=0 tx=0).stop 2024-09-10T10:00:06.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.496+0000 7feacf200640 1 -- 172.21.15.77:0/1627206195 shutdown_connections 2024-09-10T10:00:06.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.496+0000 7feacf200640 1 --2- 172.21.15.77:0/1627206195 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feac8102360 0x7feac810e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:06.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.496+0000 7feacf200640 1 --2- 172.21.15.77:0/1627206195 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feac81019c0 0x7feac8101e20 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:06.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.496+0000 7feacf200640 1 --2- 172.21.15.77:0/1627206195 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feac81079c0 0x7feac8107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:06.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.496+0000 7feacf200640 1 -- 172.21.15.77:0/1627206195 >> 172.21.15.77:0/1627206195 conn(0x7feac80fd740 msgr2=0x7feac80ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:06.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.496+0000 7feacf200640 1 -- 172.21.15.77:0/1627206195 shutdown_connections 2024-09-10T10:00:06.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.496+0000 7feacf200640 1 -- 172.21.15.77:0/1627206195 wait complete. 2024-09-10T10:00:06.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.497+0000 7feacf200640 1 Processor -- start 2024-09-10T10:00:06.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.497+0000 7feacf200640 1 -- start start 2024-09-10T10:00:06.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.497+0000 7feacf200640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feac81019c0 0x7feac81088f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:06.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.497+0000 7feacf200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feac8102360 0x7feac8108e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:06.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.497+0000 7feacf200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feac81079c0 0x7feac8109370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:06.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.497+0000 7feacf200640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7feac8111dc0 con 0x7feac81079c0 2024-09-10T10:00:06.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.497+0000 7feacf200640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7feac8111c40 con 0x7feac81019c0 2024-09-10T10:00:06.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.497+0000 7feacf200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7feac8111f40 con 0x7feac8102360 2024-09-10T10:00:06.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.498+0000 7feacde00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feac81019c0 0x7feac81088f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:06.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.498+0000 7feacde00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feac81019c0 0x7feac81088f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:60868/0 (socket says 172.21.15.77:60868) 2024-09-10T10:00:06.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.498+0000 7feacde00640 1 -- 172.21.15.77:0/2016638590 learned_addr learned my addr 172.21.15.77:0/2016638590 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:00:06.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.498+0000 7feacd400640 1 --2- 172.21.15.77:0/2016638590 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feac8102360 0x7feac8108e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:06.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.498+0000 7feace800640 1 --2- 172.21.15.77:0/2016638590 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feac81079c0 0x7feac8109370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:06.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.498+0000 7feacde00640 1 -- 172.21.15.77:0/2016638590 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feac8102360 msgr2=0x7feac8108e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:06.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.498+0000 7feacde00640 1 --2- 172.21.15.77:0/2016638590 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feac8102360 0x7feac8108e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:06.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.498+0000 7feacde00640 1 -- 172.21.15.77:0/2016638590 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feac81079c0 msgr2=0x7feac8109370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:06.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.498+0000 7feacde00640 1 --2- 172.21.15.77:0/2016638590 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feac81079c0 0x7feac8109370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:06.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.498+0000 7feacde00640 1 -- 172.21.15.77:0/2016638590 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feac81aabc0 con 0x7feac81019c0 2024-09-10T10:00:06.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.498+0000 7feacd400640 1 --2- 172.21.15.77:0/2016638590 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feac8102360 0x7feac8108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:00:06.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.499+0000 7feacde00640 1 --2- 172.21.15.77:0/2016638590 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feac81019c0 0x7feac81088f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7feabc00ea50 tx=0x7feabc00ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:06.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.499+0000 7feac2a00640 1 -- 172.21.15.77:0/2016638590 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feabc00cd60 con 0x7feac81019c0 2024-09-10T10:00:06.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.499+0000 7feacf200640 1 -- 172.21.15.77:0/2016638590 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feac81aaeb0 con 0x7feac81019c0 2024-09-10T10:00:06.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.499+0000 7feac2a00640 1 -- 172.21.15.77:0/2016638590 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feabc005500 con 0x7feac81019c0 2024-09-10T10:00:06.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.499+0000 7feac2a00640 1 -- 172.21.15.77:0/2016638590 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feabc010690 con 0x7feac81019c0 2024-09-10T10:00:06.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.499+0000 7feacf200640 1 -- 172.21.15.77:0/2016638590 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feac81ab310 con 0x7feac81019c0 2024-09-10T10:00:06.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.502+0000 7fead3260740 1 Processor -- start 2024-09-10T10:00:06.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.502+0000 7fead3260740 1 -- start start 2024-09-10T10:00:06.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.503+0000 7fead3260740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e6bc01520 0x556e6bc01900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:06.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.503+0000 7fead3260740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e6bc01e40 0x556e6bc0c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:06.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.503+0000 7fead3260740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e6bc0c700 0x556e6bc0eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:06.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.503+0000 7fead3260740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x556e6bb67650 con 0x556e6bc01520 2024-09-10T10:00:06.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.503+0000 7fead3260740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x556e6bb0ef10 con 0x556e6bc0c700 2024-09-10T10:00:06.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.503+0000 7fead3260740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556e6bb63610 con 0x556e6bc01e40 2024-09-10T10:00:06.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.503+0000 7feace800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e6bc01520 0x556e6bc01900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:06.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.503+0000 7feac2a00640 1 -- 172.21.15.77:0/2016638590 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7feabc004f30 con 0x7feac81019c0 2024-09-10T10:00:06.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.503+0000 7feace800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e6bc01520 0x556e6bc01900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:51366/0 (socket says 172.21.15.77:51366) 2024-09-10T10:00:06.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.503+0000 7feace800640 1 -- 172.21.15.77:0/2850408377 learned_addr learned my addr 172.21.15.77:0/2850408377 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:00:06.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.503+0000 7feacd400640 1 --2- 172.21.15.77:0/2850408377 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e6bc0c700 0x556e6bc0eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:06.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.503+0000 7feacde00640 1 --2- 172.21.15.77:0/2850408377 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e6bc01e40 0x556e6bc0c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:06.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.503+0000 7feace800640 1 -- 172.21.15.77:0/2850408377 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e6bc01e40 msgr2=0x556e6bc0c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:06.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.503+0000 7feace800640 1 --2- 172.21.15.77:0/2850408377 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e6bc01e40 0x556e6bc0c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:06.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.503+0000 7feace800640 1 -- 172.21.15.77:0/2850408377 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e6bc0c700 msgr2=0x556e6bc0eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:06.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.503+0000 7feace800640 1 --2- 172.21.15.77:0/2850408377 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e6bc0c700 0x556e6bc0eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:06.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.504+0000 7feace800640 1 -- 172.21.15.77:0/2850408377 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e6b876580 con 0x556e6bc01520 2024-09-10T10:00:06.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.504+0000 7feacde00640 1 --2- 172.21.15.77:0/2850408377 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e6bc01e40 0x556e6bc0c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:00:06.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.504+0000 7feace800640 1 --2- 172.21.15.77:0/2850408377 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e6bc01520 0x556e6bc01900 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7feac40039f0 tx=0x7feac4003eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53b111d84d524e87 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:06.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.504+0000 7feab7000640 1 -- 172.21.15.77:0/2850408377 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feac401a470 con 0x556e6bc01520 2024-09-10T10:00:06.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.504+0000 7feab7000640 1 -- 172.21.15.77:0/2850408377 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feac4015070 con 0x556e6bc01520 2024-09-10T10:00:06.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.504+0000 7feab7000640 1 -- 172.21.15.77:0/2850408377 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feac4010900 con 0x556e6bc01520 2024-09-10T10:00:06.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.505+0000 7fead3260740 1 -- 172.21.15.77:0/2850408377 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e6bc01520 msgr2=0x556e6bc01900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:06.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.505+0000 7fead3260740 1 --2- 172.21.15.77:0/2850408377 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e6bc01520 0x556e6bc01900 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7feac40039f0 tx=0x7feac4003eb0 comp rx=0 tx=0).stop 2024-09-10T10:00:06.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.505+0000 7fead3260740 1 -- 172.21.15.77:0/2850408377 shutdown_connections 2024-09-10T10:00:06.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.505+0000 7fead3260740 1 --2- 172.21.15.77:0/2850408377 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e6bc0c700 0x556e6bc0eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:06.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.505+0000 7fead3260740 1 --2- 172.21.15.77:0/2850408377 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e6bc01e40 0x556e6bc0c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:06.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.505+0000 7fead3260740 1 --2- 172.21.15.77:0/2850408377 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e6bc01520 0x556e6bc01900 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:06.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.505+0000 7fead3260740 1 -- 172.21.15.77:0/2850408377 >> 172.21.15.77:0/2850408377 conn(0x556e6bbfdab0 msgr2=0x556e6bbfde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:06.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.505+0000 7fead3260740 1 -- 172.21.15.77:0/2850408377 shutdown_connections 2024-09-10T10:00:06.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.505+0000 7fead3260740 1 -- 172.21.15.77:0/2850408377 wait complete. 2024-09-10T10:00:06.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.506+0000 7feac2a00640 1 --2- 172.21.15.77:0/2016638590 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7feab00a2a40 0x7feab00a4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:06.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.506+0000 7feac2a00640 1 -- 172.21.15.77:0/2016638590 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7feabc0cf740 con 0x7feac81019c0 2024-09-10T10:00:06.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.506+0000 7fead3260740 1 -- start start 2024-09-10T10:00:06.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.506+0000 7fead3260740 1 Processor -- start 2024-09-10T10:00:06.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.507+0000 7feacd400640 1 --2- 172.21.15.77:0/2016638590 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7feab00a2a40 0x7feab00a4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:06.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.507+0000 7fead3260740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e6bc01520 0x556e6bc09630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:06.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.507+0000 7fead3260740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e6bc01e40 0x556e6bc09b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:06.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.507+0000 7fead3260740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e6bc0c700 0x556e6bc0a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:06.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.507+0000 7fead3260740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x556e6bc0f2d0 con 0x556e6bc0c700 2024-09-10T10:00:06.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.507+0000 7fead3260740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x556e6bb22350 con 0x556e6bc01e40 2024-09-10T10:00:06.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.507+0000 7fead3260740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556e6bc0f450 con 0x556e6bc01520 2024-09-10T10:00:06.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.507+0000 7fead3260740 10 client.0 _ll_register_callbacks cb 0x556e6bb45290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:00:06.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.507+0000 7feace800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e6bc0c700 0x556e6bc0a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:06.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.507+0000 7feacde00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e6bc01520 0x556e6bc09630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:06.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.507+0000 7feace800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e6bc0c700 0x556e6bc0a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:51378/0 (socket says 172.21.15.77:51378) 2024-09-10T10:00:06.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.507+0000 7feace800640 1 -- 172.21.15.77:0/1195879541 learned_addr learned my addr 172.21.15.77:0/1195879541 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:00:06.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.507+0000 7feacde00640 1 -- 172.21.15.77:0/1195879541 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e6bc01e40 msgr2=0x556e6bc09b70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T10:00:06.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.507+0000 7feacde00640 1 --2- 172.21.15.77:0/1195879541 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e6bc01e40 0x556e6bc09b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:06.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.507+0000 7feacde00640 1 -- 172.21.15.77:0/1195879541 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e6bc0c700 msgr2=0x556e6bc0a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:06.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.507+0000 7feacde00640 1 --2- 172.21.15.77:0/1195879541 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e6bc0c700 0x556e6bc0a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:06.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.508+0000 7feacde00640 1 -- 172.21.15.77:0/1195879541 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e6b940960 con 0x556e6bc01520 2024-09-10T10:00:06.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.508+0000 7feace800640 1 --2- 172.21.15.77:0/1195879541 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e6bc0c700 0x556e6bc0a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:00:06.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.508+0000 7feacde00640 1 --2- 172.21.15.77:0/1195879541 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e6bc01520 0x556e6bc09630 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x556e6bbfe910 tx=0x7feabc09b9e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ee4ca8941c8b2a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:06.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.508+0000 7feaab400640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-10T10:00:06.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.508+0000 7feacd400640 1 --2- 172.21.15.77:0/2016638590 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7feab00a2a40 0x7feab00a4f00 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7feab8009900 tx=0x7feab80058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:06.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.508+0000 7feaab400640 1 -- 172.21.15.77:0/1195879541 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feabc0b3020 con 0x556e6bc01520 2024-09-10T10:00:06.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.508+0000 7fead3260740 1 -- 172.21.15.77:0/1195879541 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feac4015070 con 0x556e6bc01520 2024-09-10T10:00:06.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.508+0000 7feaab400640 1 -- 172.21.15.77:0/1195879541 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7feabc005220 con 0x556e6bc01520 2024-09-10T10:00:06.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.508+0000 7feaab400640 1 -- 172.21.15.77:0/1195879541 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7feabc09bb50 con 0x556e6bc01520 2024-09-10T10:00:06.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.508+0000 7fead3260740 10 client.26434 fetch_fsmap learned FSMap version 32 2024-09-10T10:00:06.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.508+0000 7fead3260740 1 -- 172.21.15.77:0/1195879541 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556e6bc1af30 con 0x556e6bc01520 2024-09-10T10:00:06.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.509+0000 7feaab400640 1 -- 172.21.15.77:0/1195879541 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feac4015070 con 0x556e6bc01520 2024-09-10T10:00:06.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.509+0000 7feaab400640 1 -- 172.21.15.77:0/1195879541 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7feabc0a4330 con 0x556e6bc01520 2024-09-10T10:00:06.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.509+0000 7fead3260740 10 client.26434 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T10:00:06.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.509+0000 7fead3260740 10 client.26434 resolve_mds: validated gid 24563 aka daemon mds.b 2024-09-10T10:00:06.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.509+0000 7fead3260740 20 client.26434 populate_metadata read hostname 'smithi077' 2024-09-10T10:00:06.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.509+0000 7fead3260740 1 --2- 172.21.15.77:0/1195879541 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x556e6bc1c440 0x556e6bc1e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:06.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.509+0000 7fead3260740 4 client.26434 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:00:06.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.509+0000 7fead3260740 1 -- 172.21.15.77:0/1195879541 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e6bb74bc0 con 0x556e6bc1c440 2024-09-10T10:00:06.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.510+0000 7feacd400640 1 --2- 172.21.15.77:0/1195879541 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x556e6bc1c440 0x556e6bc1e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:06.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.510+0000 7feaab400640 10 client.26434 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T10:00:06.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.510+0000 7feacd400640 1 --2- 172.21.15.77:0/1195879541 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x556e6bc1c440 0x556e6bc1e8c0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91511e7b040e2128 server_cookie=49b67ec9b1a314f4 in_seq=0 out_seq=0 2024-09-10T10:00:06.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.511+0000 7feaab400640 1 -- 172.21.15.77:0/1195879541 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556e6bb74bc0 con 0x556e6bc1c440 2024-09-10T10:00:06.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.511+0000 7feaab400640 10 client.26434 handle_command_reply: tid=0 2024-09-10T10:00:06.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.511+0000 7fead3260740 1 client.26434 shutdown 2024-09-10T10:00:06.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.511+0000 7fead3260740 1 -- 172.21.15.77:0/1195879541 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e6bc01520 msgr2=0x556e6bc09630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:06.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.511+0000 7fead3260740 1 --2- 172.21.15.77:0/1195879541 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e6bc01520 0x556e6bc09630 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x556e6bbfe910 tx=0x7feabc09b9e0 comp rx=0 tx=0).stop 2024-09-10T10:00:06.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.511+0000 7fead3260740 1 -- 172.21.15.77:0/1195879541 shutdown_connections 2024-09-10T10:00:06.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.511+0000 7fead3260740 1 --2- 172.21.15.77:0/1195879541 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x556e6bc1c440 0x556e6bc1e8c0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:06.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.511+0000 7fead3260740 1 --2- 172.21.15.77:0/1195879541 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e6bc0c700 0x556e6bc0a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:06.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.511+0000 7fead3260740 1 --2- 172.21.15.77:0/1195879541 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e6bc01e40 0x556e6bc09b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:06.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.511+0000 7feaab400640 0 client.26434 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T10:00:06.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.511+0000 7fead3260740 1 --2- 172.21.15.77:0/1195879541 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e6bc01520 0x556e6bc09630 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:06.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.511+0000 7fead3260740 1 -- 172.21.15.77:0/1195879541 >> 172.21.15.77:0/1195879541 conn(0x556e6bbfdab0 msgr2=0x556e6bbfccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:06.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.512+0000 7fead3260740 1 -- 172.21.15.77:0/1195879541 shutdown_connections 2024-09-10T10:00:06.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.512+0000 7fead3260740 1 -- 172.21.15.77:0/1195879541 wait complete. 2024-09-10T10:00:06.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.512+0000 7fead3260740 20 client.26434 trim_cache size 0 max 16384 2024-09-10T10:00:06.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.530+0000 7fead3260740 1 Processor -- start 2024-09-10T10:00:06.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.531+0000 7fead3260740 1 -- start start 2024-09-10T10:00:06.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.531+0000 7fead3260740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e6bbfdab0 0x556e6bbfcb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:06.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.531+0000 7fead3260740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e6bc01520 0x556e6bc07f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:06.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.531+0000 7fead3260740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e6bc01e40 0x556e6bc03f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:06.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.531+0000 7fead3260740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7feac81b5260 con 0x556e6bbfdab0 2024-09-10T10:00:06.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.531+0000 7fead3260740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7feac81b6e60 con 0x556e6bc01520 2024-09-10T10:00:06.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.531+0000 7fead3260740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7feac81b5930 con 0x556e6bc01e40 2024-09-10T10:00:06.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.531+0000 7feacd400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e6bc01520 0x556e6bc07f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:06.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.531+0000 7feacde00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e6bbfdab0 0x556e6bbfcb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:06.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.531+0000 7feacd400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e6bc01520 0x556e6bc07f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:60894/0 (socket says 172.21.15.77:60894) 2024-09-10T10:00:06.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.531+0000 7feacd400640 1 -- 172.21.15.77:0/3472346810 learned_addr learned my addr 172.21.15.77:0/3472346810 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:00:06.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.531+0000 7feace800640 1 --2- 172.21.15.77:0/3472346810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e6bc01e40 0x556e6bc03f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:06.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.532+0000 7feacd400640 1 -- 172.21.15.77:0/3472346810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e6bc01e40 msgr2=0x556e6bc03f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:06.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.532+0000 7feacd400640 1 --2- 172.21.15.77:0/3472346810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e6bc01e40 0x556e6bc03f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:06.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.532+0000 7feacd400640 1 -- 172.21.15.77:0/3472346810 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e6bbfdab0 msgr2=0x556e6bbfcb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:06.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.532+0000 7feacd400640 1 --2- 172.21.15.77:0/3472346810 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e6bbfdab0 0x556e6bbfcb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:06.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.532+0000 7feacd400640 1 -- 172.21.15.77:0/3472346810 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e6bbfd8e0 con 0x556e6bc01520 2024-09-10T10:00:06.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.532+0000 7feace800640 1 --2- 172.21.15.77:0/3472346810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e6bc01e40 0x556e6bc03f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:00:06.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.532+0000 7feacd400640 1 --2- 172.21.15.77:0/3472346810 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e6bc01520 0x556e6bc07f00 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7feab8009930 tx=0x7feab803f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82012ae42d4ba48 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:06.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.532+0000 7feab6600640 1 -- 172.21.15.77:0/3472346810 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab8043070 con 0x556e6bc01520 2024-09-10T10:00:06.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.532+0000 7feab6600640 1 -- 172.21.15.77:0/3472346810 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feab8056070 con 0x556e6bc01520 2024-09-10T10:00:06.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.532+0000 7feab6600640 1 -- 172.21.15.77:0/3472346810 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab8035040 con 0x556e6bc01520 2024-09-10T10:00:06.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.533+0000 7fead3260740 1 -- 172.21.15.77:0/3472346810 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e6bc01520 msgr2=0x556e6bc07f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:06.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.533+0000 7fead3260740 1 --2- 172.21.15.77:0/3472346810 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e6bc01520 0x556e6bc07f00 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7feab8009930 tx=0x7feab803f000 comp rx=0 tx=0).stop 2024-09-10T10:00:06.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.533+0000 7fead3260740 1 -- 172.21.15.77:0/3472346810 shutdown_connections 2024-09-10T10:00:06.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.533+0000 7fead3260740 1 --2- 172.21.15.77:0/3472346810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e6bc01e40 0x556e6bc03f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:06.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.533+0000 7fead3260740 1 --2- 172.21.15.77:0/3472346810 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e6bc01520 0x556e6bc07f00 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:06.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.533+0000 7fead3260740 1 --2- 172.21.15.77:0/3472346810 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e6bbfdab0 0x556e6bbfcb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:06.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.533+0000 7fead3260740 1 -- 172.21.15.77:0/3472346810 >> 172.21.15.77:0/3472346810 conn(0x556e6bc05020 msgr2=0x556e6bc0b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:06.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.533+0000 7fead3260740 1 -- 172.21.15.77:0/3472346810 shutdown_connections 2024-09-10T10:00:06.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.533+0000 7fead3260740 1 -- 172.21.15.77:0/3472346810 wait complete. 2024-09-10T10:00:06.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.534+0000 7fead3260740 1 -- start start 2024-09-10T10:00:06.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.534+0000 7fead3260740 1 Processor -- start 2024-09-10T10:00:06.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.535+0000 7fead3260740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e6bbfdab0 0x556e6bc10600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:06.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.535+0000 7fead3260740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e6bc01520 0x556e6bc10b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:06.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.535+0000 7fead3260740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e6bc01e40 0x556e6bc11080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:06.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.535+0000 7fead3260740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7feac81b66d0 con 0x556e6bbfdab0 2024-09-10T10:00:06.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.535+0000 7fead3260740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7feac81b6000 con 0x556e6bc01e40 2024-09-10T10:00:06.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.535+0000 7fead3260740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7feac81b3240 con 0x556e6bc01520 2024-09-10T10:00:06.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.535+0000 7fead3260740 10 client.0 _ll_register_callbacks cb 0x7feabc00cf00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:00:06.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.535+0000 7feacde00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e6bbfdab0 0x556e6bc10600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:06.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.535+0000 7feacd400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e6bc01520 0x556e6bc10b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:06.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.535+0000 7feacde00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e6bbfdab0 0x556e6bc10600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:51408/0 (socket says 172.21.15.77:51408) 2024-09-10T10:00:06.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.535+0000 7feacd400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e6bc01520 0x556e6bc10b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:35714/0 (socket says 172.21.15.77:35714) 2024-09-10T10:00:06.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.535+0000 7feacde00640 1 -- 172.21.15.77:0/4236955338 learned_addr learned my addr 172.21.15.77:0/4236955338 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:00:06.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.535+0000 7feace800640 1 --2- 172.21.15.77:0/4236955338 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e6bc01e40 0x556e6bc11080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:06.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.535+0000 7feacd400640 1 -- 172.21.15.77:0/4236955338 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e6bc01e40 msgr2=0x556e6bc11080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:06.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.536+0000 7feacd400640 1 --2- 172.21.15.77:0/4236955338 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e6bc01e40 0x556e6bc11080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:06.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.536+0000 7feacd400640 1 -- 172.21.15.77:0/4236955338 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e6bbfdab0 msgr2=0x556e6bc10600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:06.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.536+0000 7feacd400640 1 --2- 172.21.15.77:0/4236955338 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e6bbfdab0 0x556e6bc10600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:06.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.536+0000 7feacd400640 1 -- 172.21.15.77:0/4236955338 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e6bc86360 con 0x556e6bc01520 2024-09-10T10:00:06.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.536+0000 7feacd400640 1 --2- 172.21.15.77:0/4236955338 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e6bc01520 0x556e6bc10b40 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x556e6bbfea80 tx=0x7feab80502d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b17422f9b99ca72 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:06.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.536+0000 7feaab400640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-10T10:00:06.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.536+0000 7feaab400640 1 -- 172.21.15.77:0/4236955338 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab8005260 con 0x556e6bc01520 2024-09-10T10:00:06.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.536+0000 7fead3260740 1 -- 172.21.15.77:0/4236955338 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feab8056070 con 0x556e6bc01520 2024-09-10T10:00:06.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.536+0000 7feaab400640 1 -- 172.21.15.77:0/4236955338 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7feab8005400 con 0x556e6bc01520 2024-09-10T10:00:06.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.536+0000 7feaab400640 1 -- 172.21.15.77:0/4236955338 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7feab8005600 con 0x556e6bc01520 2024-09-10T10:00:06.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.536+0000 7fead3260740 10 client.26437 fetch_fsmap learned FSMap version 32 2024-09-10T10:00:06.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.536+0000 7fead3260740 1 -- 172.21.15.77:0/4236955338 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556e6bc87590 con 0x556e6bc01520 2024-09-10T10:00:06.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.537+0000 7feaab400640 1 -- 172.21.15.77:0/4236955338 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feab8056070 con 0x556e6bc01520 2024-09-10T10:00:06.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.538+0000 7feaab400640 1 -- 172.21.15.77:0/4236955338 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7feab8050b60 con 0x556e6bc01520 2024-09-10T10:00:06.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.538+0000 7fead3260740 10 client.26437 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T10:00:06.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.538+0000 7fead3260740 10 client.26437 resolve_mds: validated gid 24563 aka daemon mds.b 2024-09-10T10:00:06.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.538+0000 7fead3260740 20 client.26437 populate_metadata read hostname 'smithi077' 2024-09-10T10:00:06.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.538+0000 7fead3260740 1 --2- 172.21.15.77:0/4236955338 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x556e6bc0c700 0x556e6bc89d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:06.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.538+0000 7fead3260740 4 client.26437 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "90af319d", "release": true, "await": true}] 2024-09-10T10:00:06.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.538+0000 7fead3260740 1 -- 172.21.15.77:0/4236955338 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "quiesce db", "set_id": "90af319d", "release": true, "await": true}) -- 0x556e6bb6f0d0 con 0x556e6bc0c700 2024-09-10T10:00:06.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.538+0000 7feacde00640 1 --2- 172.21.15.77:0/4236955338 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x556e6bc0c700 0x556e6bc89d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:06.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.539+0000 7feacde00640 1 --2- 172.21.15.77:0/4236955338 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x556e6bc0c700 0x556e6bc89d60 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc124084744de607 server_cookie=1d88f0bc9cc6ad95 in_seq=0 out_seq=0 2024-09-10T10:00:06.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.539+0000 7feaab400640 10 client.26437 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T10:00:06.547 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:00:06.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.637+0000 7feaab400640 1 -- 172.21.15.77:0/4236955338 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x556e6bb6f0d0 con 0x556e6bc0c700 2024-09-10T10:00:06.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.637+0000 7feaab400640 10 client.26437 handle_command_reply: tid=0 2024-09-10T10:00:06.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.637+0000 7fead3260740 1 client.26437 shutdown 2024-09-10T10:00:06.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.637+0000 7fead3260740 1 -- 172.21.15.77:0/4236955338 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e6bc01520 msgr2=0x556e6bc10b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:06.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.637+0000 7fead3260740 1 --2- 172.21.15.77:0/4236955338 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e6bc01520 0x556e6bc10b40 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x556e6bbfea80 tx=0x7feab80502d0 comp rx=0 tx=0).stop 2024-09-10T10:00:06.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.637+0000 7fead3260740 1 -- 172.21.15.77:0/4236955338 shutdown_connections 2024-09-10T10:00:06.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.637+0000 7fead3260740 1 --2- 172.21.15.77:0/4236955338 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x556e6bc0c700 0x556e6bc89d60 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:06.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.637+0000 7fead3260740 1 --2- 172.21.15.77:0/4236955338 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e6bc01e40 0x556e6bc11080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:06.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.638+0000 7fead3260740 1 --2- 172.21.15.77:0/4236955338 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e6bc01520 0x556e6bc10b40 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:06.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.638+0000 7feaab400640 0 client.26437 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T10:00:06.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.638+0000 7fead3260740 1 --2- 172.21.15.77:0/4236955338 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e6bbfdab0 0x556e6bc10600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:06.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.638+0000 7fead3260740 1 -- 172.21.15.77:0/4236955338 >> 172.21.15.77:0/4236955338 conn(0x556e6bc05020 msgr2=0x556e6bc17520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:06.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.638+0000 7fead3260740 1 -- 172.21.15.77:0/4236955338 shutdown_connections 2024-09-10T10:00:06.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.638+0000 7fead3260740 1 -- 172.21.15.77:0/4236955338 wait complete. 2024-09-10T10:00:06.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.638+0000 7fead3260740 20 client.26437 trim_cache size 0 max 16384 2024-09-10T10:00:06.640 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-10T10:00:06.640 INFO:teuthology.orchestra.run.smithi077.stdout: "epoch": 32, 2024-09-10T10:00:06.641 INFO:teuthology.orchestra.run.smithi077.stdout: "leader": 24563, 2024-09-10T10:00:06.641 INFO:teuthology.orchestra.run.smithi077.stdout: "set_version": 16, 2024-09-10T10:00:06.641 INFO:teuthology.orchestra.run.smithi077.stdout: "sets": { 2024-09-10T10:00:06.641 INFO:teuthology.orchestra.run.smithi077.stdout: "90af319d": { 2024-09-10T10:00:06.641 INFO:teuthology.orchestra.run.smithi077.stdout: "version": 16, 2024-09-10T10:00:06.641 INFO:teuthology.orchestra.run.smithi077.stdout: "age_ref": 0.0, 2024-09-10T10:00:06.641 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-10T10:00:06.641 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RELEASED", 2024-09-10T10:00:06.641 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-10T10:00:06.641 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:00:06.641 INFO:teuthology.orchestra.run.smithi077.stdout: "timeout": 90.0, 2024-09-10T10:00:06.641 INFO:teuthology.orchestra.run.smithi077.stdout: "expiration": 142.8, 2024-09-10T10:00:06.642 INFO:teuthology.orchestra.run.smithi077.stdout: "members": { 2024-09-10T10:00:06.642 INFO:teuthology.orchestra.run.smithi077.stdout: "file:/": { 2024-09-10T10:00:06.642 INFO:teuthology.orchestra.run.smithi077.stdout: "excluded": false, 2024-09-10T10:00:06.642 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-10T10:00:06.642 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RELEASED", 2024-09-10T10:00:06.642 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-10T10:00:06.642 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:00:06.642 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:00:06.642 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:00:06.642 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:00:06.642 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:00:06.642 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-10T10:00:06.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.638+0000 7feacf200640 1 -- 172.21.15.77:0/2016638590 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7feab00a2a40 msgr2=0x7feab00a4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:06.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.638+0000 7feacf200640 1 --2- 172.21.15.77:0/2016638590 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7feab00a2a40 0x7feab00a4f00 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7feab8009900 tx=0x7feab80058d0 comp rx=0 tx=0).stop 2024-09-10T10:00:06.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.638+0000 7feacf200640 1 -- 172.21.15.77:0/2016638590 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feac81019c0 msgr2=0x7feac81088f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:06.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.638+0000 7feacf200640 1 --2- 172.21.15.77:0/2016638590 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feac81019c0 0x7feac81088f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7feabc00ea50 tx=0x7feabc00ee50 comp rx=0 tx=0).stop 2024-09-10T10:00:06.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.639+0000 7feacf200640 1 -- 172.21.15.77:0/2016638590 shutdown_connections 2024-09-10T10:00:06.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.639+0000 7feacf200640 1 --2- 172.21.15.77:0/2016638590 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7feab00a2a40 0x7feab00a4f00 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:06.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.639+0000 7feacf200640 1 --2- 172.21.15.77:0/2016638590 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feac81079c0 0x7feac8109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:06.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.639+0000 7feacf200640 1 --2- 172.21.15.77:0/2016638590 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feac8102360 0x7feac8108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:06.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.639+0000 7feacf200640 1 --2- 172.21.15.77:0/2016638590 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feac81019c0 0x7feac81088f0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:06.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.639+0000 7feacf200640 1 -- 172.21.15.77:0/2016638590 >> 172.21.15.77:0/2016638590 conn(0x7feac80fd740 msgr2=0x7feac810cfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:06.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.639+0000 7feacf200640 1 -- 172.21.15.77:0/2016638590 shutdown_connections 2024-09-10T10:00:06.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:06.639+0000 7feacf200640 1 -- 172.21.15.77:0/2016638590 wait complete. 2024-09-10T10:00:07.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:00:06 smithi120 ceph-mon[34770]: pgmap v539: 129 pgs: 129 active+clean; 97 MiB data, 935 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9.8 KiB/s wr, 10 op/s 2024-09-10T10:00:07.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:00:06 smithi077 ceph-mon[30319]: pgmap v539: 129 pgs: 129 active+clean; 97 MiB data, 935 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9.8 KiB/s wr, 10 op/s 2024-09-10T10:00:07.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.305+0000 7f346a600640 1 -- 172.21.15.77:0/1092842695 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3464103e50 msgr2=0x7f3464104230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:07.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.305+0000 7f346a600640 1 --2- 172.21.15.77:0/1092842695 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3464103e50 0x7f3464104230 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f3454009900 tx=0x7f345402a3f0 comp rx=0 tx=0).stop 2024-09-10T10:00:07.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.305+0000 7f346a600640 1 -- 172.21.15.77:0/1092842695 shutdown_connections 2024-09-10T10:00:07.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.305+0000 7f346a600640 1 --2- 172.21.15.77:0/1092842695 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3464109090 0x7f3464110c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:07.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.305+0000 7f346a600640 1 --2- 172.21.15.77:0/1092842695 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3464104800 0x7f3464108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:07.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.305+0000 7f346a600640 1 --2- 172.21.15.77:0/1092842695 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3464103e50 0x7f3464104230 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:07.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.305+0000 7f346a600640 1 -- 172.21.15.77:0/1092842695 >> 172.21.15.77:0/1092842695 conn(0x7f34640ff8b0 msgr2=0x7f3464101cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:07.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.305+0000 7f346a600640 1 -- 172.21.15.77:0/1092842695 shutdown_connections 2024-09-10T10:00:07.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.305+0000 7f346a600640 1 -- 172.21.15.77:0/1092842695 wait complete. 2024-09-10T10:00:07.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.306+0000 7f346a600640 1 Processor -- start 2024-09-10T10:00:07.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.306+0000 7f346a600640 1 -- start start 2024-09-10T10:00:07.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.307+0000 7f346a600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3464103e50 0x7f346419a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:07.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.307+0000 7f346a600640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3464104800 0x7f346419aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:07.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.307+0000 7f346a600640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3464109090 0x7f346419ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:07.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.307+0000 7f346a600640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3464114110 con 0x7f3464109090 2024-09-10T10:00:07.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.307+0000 7f346a600640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3464113f90 con 0x7f3464104800 2024-09-10T10:00:07.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.307+0000 7f346a600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3464114290 con 0x7f3464103e50 2024-09-10T10:00:07.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.307+0000 7f345be00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3464104800 0x7f346419aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:07.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.307+0000 7f3469200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3464103e50 0x7f346419a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:07.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.307+0000 7f345be00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3464104800 0x7f346419aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:60916/0 (socket says 172.21.15.77:60916) 2024-09-10T10:00:07.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.307+0000 7f345be00640 1 -- 172.21.15.77:0/2379219060 learned_addr learned my addr 172.21.15.77:0/2379219060 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:00:07.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.307+0000 7f3469c00640 1 --2- 172.21.15.77:0/2379219060 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3464109090 0x7f346419ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:07.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.307+0000 7f345be00640 1 -- 172.21.15.77:0/2379219060 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3464103e50 msgr2=0x7f346419a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:07.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.307+0000 7f345be00640 1 --2- 172.21.15.77:0/2379219060 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3464103e50 0x7f346419a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:07.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.307+0000 7f345be00640 1 -- 172.21.15.77:0/2379219060 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3464109090 msgr2=0x7f346419ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:07.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.307+0000 7f345be00640 1 --2- 172.21.15.77:0/2379219060 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3464109090 0x7f346419ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:07.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.307+0000 7f345be00640 1 -- 172.21.15.77:0/2379219060 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f346419f540 con 0x7f3464104800 2024-09-10T10:00:07.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.308+0000 7f345be00640 1 --2- 172.21.15.77:0/2379219060 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3464104800 0x7f346419aad0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f344c00e9f0 tx=0x7f344c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:07.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.308+0000 7f3462000640 1 -- 172.21.15.77:0/2379219060 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f344c00cd80 con 0x7f3464104800 2024-09-10T10:00:07.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.308+0000 7f346a600640 1 -- 172.21.15.77:0/2379219060 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f346419f830 con 0x7f3464104800 2024-09-10T10:00:07.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.308+0000 7f3462000640 1 -- 172.21.15.77:0/2379219060 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f344c010910 con 0x7f3464104800 2024-09-10T10:00:07.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.308+0000 7f3462000640 1 -- 172.21.15.77:0/2379219060 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f344c018990 con 0x7f3464104800 2024-09-10T10:00:07.311 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.308+0000 7f346a600640 1 -- 172.21.15.77:0/2379219060 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34641a7110 con 0x7f3464104800 2024-09-10T10:00:07.313 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.311+0000 7f346e809740 1 Processor -- start 2024-09-10T10:00:07.313 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.311+0000 7f346e809740 1 -- start start 2024-09-10T10:00:07.313 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.311+0000 7f3462000640 1 -- 172.21.15.77:0/2379219060 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f344c010ac0 con 0x7f3464104800 2024-09-10T10:00:07.314 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.312+0000 7f346e809740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a09d8a6e40 0x55a09d8a7220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:07.314 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.312+0000 7f346e809740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a09d8a7760 0x55a09d8b1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:07.314 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.312+0000 7f346e809740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a09d8b2020 0x55a09d8b4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:07.314 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.312+0000 7f346e809740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55a09d80d3e0 con 0x55a09d8a6e40 2024-09-10T10:00:07.314 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.312+0000 7f346e809740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a09d7b4d80 con 0x55a09d8a7760 2024-09-10T10:00:07.314 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.312+0000 7f346e809740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a09d8093a0 con 0x55a09d8b2020 2024-09-10T10:00:07.314 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.312+0000 7f3469c00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a09d8a6e40 0x55a09d8a7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:07.314 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.312+0000 7f3469200640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a09d8a7760 0x55a09d8b1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:07.315 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.312+0000 7f3469c00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a09d8a6e40 0x55a09d8a7220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:51438/0 (socket says 172.21.15.77:51438) 2024-09-10T10:00:07.315 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.312+0000 7f3469c00640 1 -- 172.21.15.77:0/3791651956 learned_addr learned my addr 172.21.15.77:0/3791651956 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:00:07.315 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.312+0000 7f345be00640 1 --2- 172.21.15.77:0/3791651956 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a09d8b2020 0x55a09d8b4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:07.315 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.312+0000 7f3469c00640 1 -- 172.21.15.77:0/3791651956 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a09d8b2020 msgr2=0x55a09d8b4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:07.315 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.312+0000 7f3469c00640 1 --2- 172.21.15.77:0/3791651956 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a09d8b2020 0x55a09d8b4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:07.315 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.312+0000 7f3469c00640 1 -- 172.21.15.77:0/3791651956 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a09d8a7760 msgr2=0x55a09d8b1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:07.315 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.312+0000 7f3469c00640 1 --2- 172.21.15.77:0/3791651956 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a09d8a7760 0x55a09d8b1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:07.315 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.312+0000 7f3469c00640 1 -- 172.21.15.77:0/3791651956 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a09d7d3ae0 con 0x55a09d8a6e40 2024-09-10T10:00:07.315 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.312+0000 7f3469200640 1 --2- 172.21.15.77:0/3791651956 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a09d8a7760 0x55a09d8b1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:00:07.315 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.313+0000 7f3469c00640 1 --2- 172.21.15.77:0/3791651956 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a09d8a6e40 0x55a09d8a7220 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f345c009f50 tx=0x7f345c0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40336594dbd7a2e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:07.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.313+0000 7f3462000640 1 --2- 172.21.15.77:0/2379219060 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f34480a2b80 0x7f34480a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:07.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.313+0000 7f345a000640 1 -- 172.21.15.77:0/3791651956 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f345c004d20 con 0x55a09d8a6e40 2024-09-10T10:00:07.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.313+0000 7f345a000640 1 -- 172.21.15.77:0/3791651956 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f345c004050 con 0x55a09d8a6e40 2024-09-10T10:00:07.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.313+0000 7f345a000640 1 -- 172.21.15.77:0/3791651956 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f345c005260 con 0x55a09d8a6e40 2024-09-10T10:00:07.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.313+0000 7f3462000640 1 -- 172.21.15.77:0/2379219060 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f344c0cf7a0 con 0x7f3464104800 2024-09-10T10:00:07.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.313+0000 7f3469200640 1 --2- 172.21.15.77:0/2379219060 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f34480a2b80 0x7f34480a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:07.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.314+0000 7f346e809740 1 -- 172.21.15.77:0/3791651956 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a09d8a6e40 msgr2=0x55a09d8a7220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:07.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.314+0000 7f346e809740 1 --2- 172.21.15.77:0/3791651956 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a09d8a6e40 0x55a09d8a7220 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f345c009f50 tx=0x7f345c0118d0 comp rx=0 tx=0).stop 2024-09-10T10:00:07.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.314+0000 7f346e809740 1 -- 172.21.15.77:0/3791651956 shutdown_connections 2024-09-10T10:00:07.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.314+0000 7f346e809740 1 --2- 172.21.15.77:0/3791651956 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a09d8b2020 0x55a09d8b4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:07.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.314+0000 7f346e809740 1 --2- 172.21.15.77:0/3791651956 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a09d8a7760 0x55a09d8b1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:07.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.314+0000 7f346e809740 1 --2- 172.21.15.77:0/3791651956 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a09d8a6e40 0x55a09d8a7220 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:07.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.314+0000 7f346e809740 1 -- 172.21.15.77:0/3791651956 >> 172.21.15.77:0/3791651956 conn(0x55a09d8a33d0 msgr2=0x55a09d8a37b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:07.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.314+0000 7f346e809740 1 -- 172.21.15.77:0/3791651956 shutdown_connections 2024-09-10T10:00:07.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.314+0000 7f3469200640 1 --2- 172.21.15.77:0/2379219060 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f34480a2b80 0x7f34480a5040 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f3454000c00 tx=0x7f3454005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:07.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.314+0000 7f346e809740 1 -- 172.21.15.77:0/3791651956 wait complete. 2024-09-10T10:00:07.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.315+0000 7f346e809740 1 -- start start 2024-09-10T10:00:07.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.315+0000 7f346e809740 1 Processor -- start 2024-09-10T10:00:07.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.315+0000 7f346e809740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a09d8a6e40 0x55a09d8aeeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:07.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.315+0000 7f346e809740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a09d8a7760 0x55a09d8af3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:07.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.315+0000 7f346e809740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a09d8b2020 0x55a09d8af930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:07.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.315+0000 7f346e809740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55a09d8b4bf0 con 0x55a09d8a6e40 2024-09-10T10:00:07.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.315+0000 7f346e809740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a09d7c80f0 con 0x55a09d8a7760 2024-09-10T10:00:07.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.315+0000 7f346e809740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a09d8b4d70 con 0x55a09d8b2020 2024-09-10T10:00:07.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.315+0000 7f346e809740 10 client.0 _ll_register_callbacks cb 0x55a09d7afbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:00:07.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.316+0000 7f345be00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a09d8a7760 0x55a09d8af3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:07.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.316+0000 7f345be00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a09d8a7760 0x55a09d8af3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:60940/0 (socket says 172.21.15.77:60940) 2024-09-10T10:00:07.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.316+0000 7f3469c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a09d8b2020 0x55a09d8af930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:07.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.316+0000 7f3469200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a09d8a6e40 0x55a09d8aeeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:07.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.316+0000 7f345be00640 1 -- 172.21.15.77:0/1236028656 learned_addr learned my addr 172.21.15.77:0/1236028656 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:00:07.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.316+0000 7f345be00640 1 -- 172.21.15.77:0/1236028656 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a09d8b2020 msgr2=0x55a09d8af930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:07.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.316+0000 7f345be00640 1 --2- 172.21.15.77:0/1236028656 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a09d8b2020 0x55a09d8af930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:07.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.316+0000 7f345be00640 1 -- 172.21.15.77:0/1236028656 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a09d8a6e40 msgr2=0x55a09d8aeeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:07.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.316+0000 7f345be00640 1 --2- 172.21.15.77:0/1236028656 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a09d8a6e40 0x55a09d8aeeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:07.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.316+0000 7f345be00640 1 -- 172.21.15.77:0/1236028656 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a09d57e3c0 con 0x55a09d8a7760 2024-09-10T10:00:07.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.316+0000 7f345be00640 1 --2- 172.21.15.77:0/1236028656 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a09d8a7760 0x55a09d8af3f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f344c09b960 tx=0x7f344c09baf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4376f5f5e0410543 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:07.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.316+0000 7f3452a00640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T10:00:07.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.316+0000 7f3452a00640 1 -- 172.21.15.77:0/1236028656 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f344c09ff60 con 0x55a09d8a7760 2024-09-10T10:00:07.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.317+0000 7f346e809740 1 -- 172.21.15.77:0/1236028656 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f345c004050 con 0x55a09d8a7760 2024-09-10T10:00:07.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.317+0000 7f3452a00640 1 -- 172.21.15.77:0/1236028656 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f344c0cfb90 con 0x55a09d8a7760 2024-09-10T10:00:07.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.317+0000 7f3452a00640 1 -- 172.21.15.77:0/1236028656 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f344c0b2020 con 0x55a09d8a7760 2024-09-10T10:00:07.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.317+0000 7f346e809740 10 client.26498 fetch_fsmap learned FSMap version 32 2024-09-10T10:00:07.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.317+0000 7f346e809740 1 -- 172.21.15.77:0/1236028656 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a09d8c0bf0 con 0x55a09d8a7760 2024-09-10T10:00:07.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.317+0000 7f3452a00640 1 -- 172.21.15.77:0/1236028656 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f345c004050 con 0x55a09d8a7760 2024-09-10T10:00:07.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.317+0000 7f3452a00640 1 -- 172.21.15.77:0/1236028656 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f344c09b0b0 con 0x55a09d8a7760 2024-09-10T10:00:07.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.317+0000 7f346e809740 10 client.26498 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T10:00:07.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.317+0000 7f346e809740 10 client.26498 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-10T10:00:07.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.317+0000 7f346e809740 20 client.26498 populate_metadata read hostname 'smithi077' 2024-09-10T10:00:07.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.317+0000 7f346e809740 1 --2- 172.21.15.77:0/1236028656 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x55a09d8c20b0 0x55a09d8c44c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:07.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.317+0000 7f346e809740 4 client.26498 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:00:07.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.317+0000 7f346e809740 1 -- 172.21.15.77:0/1236028656 --> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a09d81a950 con 0x55a09d8c20b0 2024-09-10T10:00:07.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.318+0000 7f3469200640 1 --2- 172.21.15.77:0/1236028656 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x55a09d8c20b0 0x55a09d8c44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:07.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.318+0000 7f3469200640 1 --2- 172.21.15.77:0/1236028656 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x55a09d8c20b0 0x55a09d8c44c0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b5428d9f73b855b9 server_cookie=c16b67c7df2a6d0e in_seq=0 out_seq=0 2024-09-10T10:00:07.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.318+0000 7f3452a00640 10 client.26498 ms_handle_connect on v2:172.21.15.134:6834/3293933699 2024-09-10T10:00:07.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.319+0000 7f3452a00640 1 -- 172.21.15.77:0/1236028656 <== mds.4 v2:172.21.15.134:6834/3293933699 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a09d81a950 con 0x55a09d8c20b0 2024-09-10T10:00:07.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.319+0000 7f3452a00640 10 client.26498 handle_command_reply: tid=0 2024-09-10T10:00:07.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.319+0000 7f346e809740 1 client.26498 shutdown 2024-09-10T10:00:07.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.319+0000 7f346e809740 1 -- 172.21.15.77:0/1236028656 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a09d8a7760 msgr2=0x55a09d8af3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:07.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.319+0000 7f346e809740 1 --2- 172.21.15.77:0/1236028656 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a09d8a7760 0x55a09d8af3f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f344c09b960 tx=0x7f344c09baf0 comp rx=0 tx=0).stop 2024-09-10T10:00:07.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.320+0000 7f346e809740 1 -- 172.21.15.77:0/1236028656 shutdown_connections 2024-09-10T10:00:07.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.320+0000 7f346e809740 1 --2- 172.21.15.77:0/1236028656 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x55a09d8c20b0 0x55a09d8c44c0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:07.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.320+0000 7f346e809740 1 --2- 172.21.15.77:0/1236028656 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a09d8b2020 0x55a09d8af930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:07.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.320+0000 7f346e809740 1 --2- 172.21.15.77:0/1236028656 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a09d8a7760 0x55a09d8af3f0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:07.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.320+0000 7f346e809740 1 --2- 172.21.15.77:0/1236028656 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a09d8a6e40 0x55a09d8aeeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:07.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.320+0000 7f346e809740 1 -- 172.21.15.77:0/1236028656 >> 172.21.15.77:0/1236028656 conn(0x55a09d8a33d0 msgr2=0x55a09d8a9100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:07.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.320+0000 7f3452a00640 0 client.26498 ms_handle_reset on v2:172.21.15.134:6834/3293933699 2024-09-10T10:00:07.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.320+0000 7f346e809740 1 -- 172.21.15.77:0/1236028656 shutdown_connections 2024-09-10T10:00:07.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.320+0000 7f346e809740 1 -- 172.21.15.77:0/1236028656 wait complete. 2024-09-10T10:00:07.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.320+0000 7f346e809740 20 client.26498 trim_cache size 0 max 16384 2024-09-10T10:00:07.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.335+0000 7f346e809740 1 Processor -- start 2024-09-10T10:00:07.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.335+0000 7f346e809740 1 -- start start 2024-09-10T10:00:07.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.335+0000 7f346e809740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a09d8a6e40 0x55a09d8c9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:07.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.335+0000 7f346e809740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a09d8a7760 0x55a09d8ca2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:07.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.335+0000 7f346e809740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a09d8b2020 0x55a09d8ac110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:07.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.335+0000 7f346e809740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f34641b0e50 con 0x55a09d8b2020 2024-09-10T10:00:07.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.335+0000 7f346e809740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f34641b2a50 con 0x55a09d8a7760 2024-09-10T10:00:07.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.335+0000 7f346e809740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f34641b1520 con 0x55a09d8a6e40 2024-09-10T10:00:07.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.336+0000 7f3469200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a09d8a6e40 0x55a09d8c9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:07.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.336+0000 7f345be00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a09d8a7760 0x55a09d8ca2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:07.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.336+0000 7f3469200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a09d8a6e40 0x55a09d8c9d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:35756/0 (socket says 172.21.15.77:35756) 2024-09-10T10:00:07.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.336+0000 7f3469200640 1 -- 172.21.15.77:0/1272078955 learned_addr learned my addr 172.21.15.77:0/1272078955 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:00:07.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.336+0000 7f3469c00640 1 --2- 172.21.15.77:0/1272078955 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a09d8b2020 0x55a09d8ac110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:07.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.336+0000 7f345be00640 1 -- 172.21.15.77:0/1272078955 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a09d8a6e40 msgr2=0x55a09d8c9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:07.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.336+0000 7f345be00640 1 --2- 172.21.15.77:0/1272078955 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a09d8a6e40 0x55a09d8c9d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:07.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.336+0000 7f345be00640 1 -- 172.21.15.77:0/1272078955 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a09d8b2020 msgr2=0x55a09d8ac110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:07.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.336+0000 7f345be00640 1 --2- 172.21.15.77:0/1272078955 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a09d8b2020 0x55a09d8ac110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:07.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.336+0000 7f345be00640 1 -- 172.21.15.77:0/1272078955 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a09d8b1820 con 0x55a09d8a7760 2024-09-10T10:00:07.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.336+0000 7f345be00640 1 --2- 172.21.15.77:0/1272078955 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a09d8a7760 0x55a09d8ca2d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x55a09d8b0460 tx=0x7f344c000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26f78f0c71c36a62 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:07.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.336+0000 7f3459600640 1 -- 172.21.15.77:0/1272078955 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f344c0a4820 con 0x55a09d8a7760 2024-09-10T10:00:07.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.336+0000 7f3459600640 1 -- 172.21.15.77:0/1272078955 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f344c018b30 con 0x55a09d8a7760 2024-09-10T10:00:07.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.336+0000 7f3459600640 1 -- 172.21.15.77:0/1272078955 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f344c097d00 con 0x55a09d8a7760 2024-09-10T10:00:07.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.337+0000 7f346e809740 1 -- 172.21.15.77:0/1272078955 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a09d8a7760 msgr2=0x55a09d8ca2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:07.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.337+0000 7f346e809740 1 --2- 172.21.15.77:0/1272078955 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a09d8a7760 0x55a09d8ca2d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x55a09d8b0460 tx=0x7f344c000bc0 comp rx=0 tx=0).stop 2024-09-10T10:00:07.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.337+0000 7f346e809740 1 -- 172.21.15.77:0/1272078955 shutdown_connections 2024-09-10T10:00:07.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.337+0000 7f346e809740 1 --2- 172.21.15.77:0/1272078955 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a09d8b2020 0x55a09d8ac110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:07.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.337+0000 7f346e809740 1 --2- 172.21.15.77:0/1272078955 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a09d8a7760 0x55a09d8ca2d0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:07.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.337+0000 7f346e809740 1 --2- 172.21.15.77:0/1272078955 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a09d8a6e40 0x55a09d8c9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:07.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.337+0000 7f346e809740 1 -- 172.21.15.77:0/1272078955 >> 172.21.15.77:0/1272078955 conn(0x55a09d8a33d0 msgr2=0x55a09d8adfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:07.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.337+0000 7f346e809740 1 -- 172.21.15.77:0/1272078955 shutdown_connections 2024-09-10T10:00:07.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.337+0000 7f346e809740 1 -- 172.21.15.77:0/1272078955 wait complete. 2024-09-10T10:00:07.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.338+0000 7f346e809740 1 -- start start 2024-09-10T10:00:07.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.338+0000 7f346e809740 1 Processor -- start 2024-09-10T10:00:07.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.339+0000 7f346e809740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a09d8a6e40 0x55a09d8b64c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:07.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.339+0000 7f346e809740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a09d8a7760 0x55a09d8b6a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:07.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.339+0000 7f346e809740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a09d8b2020 0x55a09d934570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:07.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.339+0000 7f346e809740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f34641b22c0 con 0x55a09d8a7760 2024-09-10T10:00:07.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.339+0000 7f346e809740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f34641b1bf0 con 0x55a09d8b2020 2024-09-10T10:00:07.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.339+0000 7f346e809740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f34641aee30 con 0x55a09d8a6e40 2024-09-10T10:00:07.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.339+0000 7f346e809740 10 client.0 _ll_register_callbacks cb 0x7f344c0d5220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:00:07.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.339+0000 7f3469200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a09d8a6e40 0x55a09d8b64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:07.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.339+0000 7f3469c00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a09d8b2020 0x55a09d934570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:07.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.339+0000 7f345be00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a09d8a7760 0x55a09d8b6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:07.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.339+0000 7f3469200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a09d8a6e40 0x55a09d8b64c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:35768/0 (socket says 172.21.15.77:35768) 2024-09-10T10:00:07.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.339+0000 7f3469200640 1 -- 172.21.15.77:0/298683902 learned_addr learned my addr 172.21.15.77:0/298683902 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:00:07.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.339+0000 7f3469c00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a09d8b2020 0x55a09d934570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:60956/0 (socket says 172.21.15.77:60956) 2024-09-10T10:00:07.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.339+0000 7f3469200640 1 -- 172.21.15.77:0/298683902 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a09d8b2020 msgr2=0x55a09d934570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:07.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.339+0000 7f3469200640 1 --2- 172.21.15.77:0/298683902 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a09d8b2020 0x55a09d934570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:07.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.339+0000 7f3469200640 1 -- 172.21.15.77:0/298683902 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a09d8a7760 msgr2=0x55a09d8b6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:07.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.339+0000 7f3469200640 1 --2- 172.21.15.77:0/298683902 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a09d8a7760 0x55a09d8b6a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:07.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.339+0000 7f3469200640 1 -- 172.21.15.77:0/298683902 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a09d934b30 con 0x55a09d8a6e40 2024-09-10T10:00:07.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.340+0000 7f3469200640 1 --2- 172.21.15.77:0/298683902 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a09d8a6e40 0x55a09d8b64c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3454033000 tx=0x7f345402a3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2bfacbb60cbe386 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:07.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.340+0000 7f3452a00640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-10T10:00:07.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.340+0000 7f3452a00640 1 -- 172.21.15.77:0/298683902 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3454035040 con 0x55a09d8a6e40 2024-09-10T10:00:07.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.340+0000 7f3452a00640 1 -- 172.21.15.77:0/298683902 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f3454004b90 con 0x55a09d8a6e40 2024-09-10T10:00:07.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.340+0000 7f346e809740 1 -- 172.21.15.77:0/298683902 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f344c018b30 con 0x55a09d8a6e40 2024-09-10T10:00:07.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.340+0000 7f3452a00640 1 -- 172.21.15.77:0/298683902 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3454004d90 con 0x55a09d8a6e40 2024-09-10T10:00:07.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.340+0000 7f346e809740 10 client.26455 fetch_fsmap learned FSMap version 32 2024-09-10T10:00:07.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.340+0000 7f346e809740 1 -- 172.21.15.77:0/298683902 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a09d935e50 con 0x55a09d8a6e40 2024-09-10T10:00:07.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.340+0000 7f3452a00640 1 -- 172.21.15.77:0/298683902 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f344c018b30 con 0x55a09d8a6e40 2024-09-10T10:00:07.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.341+0000 7f3452a00640 1 -- 172.21.15.77:0/298683902 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f345405b070 con 0x55a09d8a6e40 2024-09-10T10:00:07.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.341+0000 7f346e809740 10 client.26455 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T10:00:07.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.341+0000 7f346e809740 10 client.26455 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-10T10:00:07.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.341+0000 7f346e809740 20 client.26455 populate_metadata read hostname 'smithi077' 2024-09-10T10:00:07.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.341+0000 7f346e809740 1 --2- 172.21.15.77:0/298683902 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x55a09d936550 0x55a09d938970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:07.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.341+0000 7f346e809740 4 client.26455 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:00:07.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.341+0000 7f346e809740 1 -- 172.21.15.77:0/298683902 --> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a09d814e60 con 0x55a09d936550 2024-09-10T10:00:07.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.341+0000 7f345be00640 1 --2- 172.21.15.77:0/298683902 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x55a09d936550 0x55a09d938970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:07.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.342+0000 7f3452a00640 10 client.26455 ms_handle_connect on v2:172.21.15.134:6834/3293933699 2024-09-10T10:00:07.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.342+0000 7f345be00640 1 --2- 172.21.15.77:0/298683902 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x55a09d936550 0x55a09d938970 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=782ef6ef45f1b341 server_cookie=531dfceedf1ac5f1 in_seq=0 out_seq=0 2024-09-10T10:00:07.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.342+0000 7f3452a00640 1 -- 172.21.15.77:0/298683902 <== mds.4 v2:172.21.15.134:6834/3293933699 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a09d814e60 con 0x55a09d936550 2024-09-10T10:00:07.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.342+0000 7f3452a00640 10 client.26455 handle_command_reply: tid=0 2024-09-10T10:00:07.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.342+0000 7f346e809740 1 client.26455 shutdown 2024-09-10T10:00:07.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.342+0000 7f346e809740 1 -- 172.21.15.77:0/298683902 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a09d8a6e40 msgr2=0x55a09d8b64c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:07.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.342+0000 7f346e809740 1 --2- 172.21.15.77:0/298683902 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a09d8a6e40 0x55a09d8b64c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3454033000 tx=0x7f345402a3f0 comp rx=0 tx=0).stop 2024-09-10T10:00:07.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.342+0000 7f346e809740 1 -- 172.21.15.77:0/298683902 shutdown_connections 2024-09-10T10:00:07.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.342+0000 7f346e809740 1 --2- 172.21.15.77:0/298683902 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x55a09d936550 0x55a09d938970 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:07.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.342+0000 7f346e809740 1 --2- 172.21.15.77:0/298683902 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a09d8b2020 0x55a09d934570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:07.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.342+0000 7f346e809740 1 --2- 172.21.15.77:0/298683902 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a09d8a7760 0x55a09d8b6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:07.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.342+0000 7f3452a00640 0 client.26455 ms_handle_reset on v2:172.21.15.134:6834/3293933699 2024-09-10T10:00:07.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.342+0000 7f346e809740 1 --2- 172.21.15.77:0/298683902 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a09d8a6e40 0x55a09d8b64c0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:07.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.342+0000 7f346e809740 1 -- 172.21.15.77:0/298683902 >> 172.21.15.77:0/298683902 conn(0x55a09d8a33d0 msgr2=0x55a09d8a1c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:07.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.342+0000 7f346e809740 1 -- 172.21.15.77:0/298683902 shutdown_connections 2024-09-10T10:00:07.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.342+0000 7f346e809740 1 -- 172.21.15.77:0/298683902 wait complete. 2024-09-10T10:00:07.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.342+0000 7f346e809740 20 client.26455 trim_cache size 0 max 16384 2024-09-10T10:00:07.348 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-10T10:00:07.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.343+0000 7f346a600640 1 -- 172.21.15.77:0/2379219060 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f34480a2b80 msgr2=0x7f34480a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:07.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.343+0000 7f346a600640 1 --2- 172.21.15.77:0/2379219060 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f34480a2b80 0x7f34480a5040 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f3454000c00 tx=0x7f3454005b50 comp rx=0 tx=0).stop 2024-09-10T10:00:07.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.343+0000 7f346a600640 1 -- 172.21.15.77:0/2379219060 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3464104800 msgr2=0x7f346419aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:07.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.343+0000 7f346a600640 1 --2- 172.21.15.77:0/2379219060 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3464104800 0x7f346419aad0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f344c00e9f0 tx=0x7f344c00eeb0 comp rx=0 tx=0).stop 2024-09-10T10:00:07.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.343+0000 7f346a600640 1 -- 172.21.15.77:0/2379219060 shutdown_connections 2024-09-10T10:00:07.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.343+0000 7f346a600640 1 --2- 172.21.15.77:0/2379219060 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f34480a2b80 0x7f34480a5040 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:07.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.343+0000 7f346a600640 1 --2- 172.21.15.77:0/2379219060 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3464109090 0x7f346419ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:07.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.343+0000 7f346a600640 1 --2- 172.21.15.77:0/2379219060 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3464104800 0x7f346419aad0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:07.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.343+0000 7f346a600640 1 --2- 172.21.15.77:0/2379219060 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3464103e50 0x7f346419a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:07.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.343+0000 7f346a600640 1 -- 172.21.15.77:0/2379219060 >> 172.21.15.77:0/2379219060 conn(0x7f34640ff8b0 msgr2=0x7f3464100bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:07.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.343+0000 7f346a600640 1 -- 172.21.15.77:0/2379219060 shutdown_connections 2024-09-10T10:00:07.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:07.343+0000 7f346a600640 1 -- 172.21.15.77:0/2379219060 wait complete. 2024-09-10T10:00:07.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:00:06 smithi134 ceph-mon[35674]: pgmap v539: 129 pgs: 129 active+clean; 97 MiB data, 935 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9.8 KiB/s wr, 10 op/s 2024-09-10T10:00:08.245 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T10:00:08.648 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '90af319d', total seconds elapsed: 29.9 2024-09-10T10:00:08.743 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:00:09.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:00:08 smithi120 ceph-mon[34770]: pgmap v540: 129 pgs: 129 active+clean; 97 MiB data, 935 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9.8 KiB/s wr, 10 op/s 2024-09-10T10:00:09.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:00:08 smithi077 ceph-mon[30319]: pgmap v540: 129 pgs: 129 active+clean; 97 MiB data, 935 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9.8 KiB/s wr, 10 op/s 2024-09-10T10:00:09.313 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.312+0000 7f7176c00640 1 -- 172.21.15.77:0/1955685998 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7170102b90 msgr2=0x7f7170102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:09.313 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.312+0000 7f7176c00640 1 --2- 172.21.15.77:0/1955685998 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7170102b90 0x7f7170102f70 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7164009900 tx=0x7f716402a480 comp rx=0 tx=0).stop 2024-09-10T10:00:09.313 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.312+0000 7f7176c00640 1 -- 172.21.15.77:0/1955685998 shutdown_connections 2024-09-10T10:00:09.313 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.312+0000 7f7176c00640 1 --2- 172.21.15.77:0/1955685998 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f717010bfc0 0x7f717010e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:09.313 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.312+0000 7f7176c00640 1 --2- 172.21.15.77:0/1955685998 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f71701034b0 0x7f717010b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:09.314 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.312+0000 7f7176c00640 1 --2- 172.21.15.77:0/1955685998 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7170102b90 0x7f7170102f70 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:09.314 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.312+0000 7f7176c00640 1 -- 172.21.15.77:0/1955685998 >> 172.21.15.77:0/1955685998 conn(0x7f71700fc610 msgr2=0x7f71700fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:09.314 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.312+0000 7f7176c00640 1 -- 172.21.15.77:0/1955685998 shutdown_connections 2024-09-10T10:00:09.314 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.312+0000 7f7176c00640 1 -- 172.21.15.77:0/1955685998 wait complete. 2024-09-10T10:00:09.314 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.314+0000 7f7176c00640 1 Processor -- start 2024-09-10T10:00:09.314 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.314+0000 7f7176c00640 1 -- start start 2024-09-10T10:00:09.315 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.314+0000 7f7176c00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7170102b90 0x7f717019a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:09.315 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.314+0000 7f7176c00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f71701034b0 0x7f717019aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:09.315 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.314+0000 7f7176c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f717010bfc0 0x7f717019eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:09.315 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.314+0000 7f7176c00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7170111dc0 con 0x7f71701034b0 2024-09-10T10:00:09.315 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.314+0000 7f7176c00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7170111c40 con 0x7f7170102b90 2024-09-10T10:00:09.315 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.314+0000 7f7176c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7170111f40 con 0x7f717010bfc0 2024-09-10T10:00:09.316 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.314+0000 7f7175800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7170102b90 0x7f717019a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:09.316 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.314+0000 7f7174e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f71701034b0 0x7f717019aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:09.316 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.314+0000 7f7175800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7170102b90 0x7f717019a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:60976/0 (socket says 172.21.15.77:60976) 2024-09-10T10:00:09.316 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.314+0000 7f7176200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f717010bfc0 0x7f717019eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:09.316 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.314+0000 7f7175800640 1 -- 172.21.15.77:0/161172711 learned_addr learned my addr 172.21.15.77:0/161172711 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:00:09.316 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.315+0000 7f7175800640 1 -- 172.21.15.77:0/161172711 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f717010bfc0 msgr2=0x7f717019eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:09.316 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.315+0000 7f7175800640 1 --2- 172.21.15.77:0/161172711 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f717010bfc0 0x7f717019eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:09.316 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.315+0000 7f7175800640 1 -- 172.21.15.77:0/161172711 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f71701034b0 msgr2=0x7f717019aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:09.316 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.315+0000 7f7175800640 1 --2- 172.21.15.77:0/161172711 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f71701034b0 0x7f717019aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:09.316 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.315+0000 7f7175800640 1 -- 172.21.15.77:0/161172711 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f717019f480 con 0x7f7170102b90 2024-09-10T10:00:09.316 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.315+0000 7f7175800640 1 --2- 172.21.15.77:0/161172711 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7170102b90 0x7f717019a4d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7164006010 tx=0x7f7164004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:09.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.315+0000 7f716e000640 1 -- 172.21.15.77:0/161172711 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f716403f030 con 0x7f7170102b90 2024-09-10T10:00:09.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.315+0000 7f7176c00640 1 -- 172.21.15.77:0/161172711 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f717019f710 con 0x7f7170102b90 2024-09-10T10:00:09.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.316+0000 7f716e000640 1 -- 172.21.15.77:0/161172711 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7164004440 con 0x7f7170102b90 2024-09-10T10:00:09.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.316+0000 7f716e000640 1 -- 172.21.15.77:0/161172711 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7164035070 con 0x7f7170102b90 2024-09-10T10:00:09.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.316+0000 7f7176c00640 1 -- 172.21.15.77:0/161172711 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71701a7050 con 0x7f7170102b90 2024-09-10T10:00:09.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.319+0000 7f717ac7d740 1 Processor -- start 2024-09-10T10:00:09.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.319+0000 7f716e000640 1 -- 172.21.15.77:0/161172711 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7164042030 con 0x7f7170102b90 2024-09-10T10:00:09.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.319+0000 7f717ac7d740 1 -- start start 2024-09-10T10:00:09.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.319+0000 7f717ac7d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55af5e40be40 0x55af5e40c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:09.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.319+0000 7f717ac7d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af5e40c760 0x55af5e416ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:09.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.319+0000 7f717ac7d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55af5e417020 0x55af5e419410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:09.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.319+0000 7f717ac7d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55af5e3723e0 con 0x55af5e40c760 2024-09-10T10:00:09.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.319+0000 7f717ac7d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55af5e319d80 con 0x55af5e40be40 2024-09-10T10:00:09.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.319+0000 7f717ac7d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55af5e36e3a0 con 0x55af5e417020 2024-09-10T10:00:09.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.320+0000 7f7176200640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55af5e40be40 0x55af5e40c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:09.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.320+0000 7f7176200640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55af5e40be40 0x55af5e40c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:60988/0 (socket says 172.21.15.77:60988) 2024-09-10T10:00:09.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.320+0000 7f7176200640 1 -- 172.21.15.77:0/798829278 learned_addr learned my addr 172.21.15.77:0/798829278 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:00:09.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.320+0000 7f7175800640 1 --2- 172.21.15.77:0/798829278 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af5e40c760 0x55af5e416ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:09.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.320+0000 7f7174e00640 1 --2- 172.21.15.77:0/798829278 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55af5e417020 0x55af5e419410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:09.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.320+0000 7f7176200640 1 -- 172.21.15.77:0/798829278 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55af5e417020 msgr2=0x55af5e419410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:09.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.320+0000 7f7176200640 1 --2- 172.21.15.77:0/798829278 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55af5e417020 0x55af5e419410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:09.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.320+0000 7f7176200640 1 -- 172.21.15.77:0/798829278 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af5e40c760 msgr2=0x55af5e416ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:09.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.320+0000 7f7176200640 1 --2- 172.21.15.77:0/798829278 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af5e40c760 0x55af5e416ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:09.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.320+0000 7f7176200640 1 -- 172.21.15.77:0/798829278 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af5e338ae0 con 0x55af5e40be40 2024-09-10T10:00:09.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.320+0000 7f7174e00640 1 --2- 172.21.15.77:0/798829278 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55af5e417020 0x55af5e419410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:00:09.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.320+0000 7f7176200640 1 --2- 172.21.15.77:0/798829278 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55af5e40be40 0x55af5e40c220 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f7168009f50 tx=0x7f71680118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72bb396d78836015 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:09.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.321+0000 7f7162a00640 1 -- 172.21.15.77:0/798829278 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7168004d20 con 0x55af5e40be40 2024-09-10T10:00:09.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.321+0000 7f7162a00640 1 -- 172.21.15.77:0/798829278 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7168004050 con 0x55af5e40be40 2024-09-10T10:00:09.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.321+0000 7f716e000640 1 --2- 172.21.15.77:0/161172711 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f71580a2c00 0x7f71580a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:09.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.321+0000 7f7162a00640 1 -- 172.21.15.77:0/798829278 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7168005260 con 0x55af5e40be40 2024-09-10T10:00:09.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.321+0000 7f716e000640 1 -- 172.21.15.77:0/161172711 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f71640ef760 con 0x7f7170102b90 2024-09-10T10:00:09.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.321+0000 7f7175800640 1 --2- 172.21.15.77:0/161172711 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f71580a2c00 0x7f71580a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:09.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.322+0000 7f717ac7d740 1 -- 172.21.15.77:0/798829278 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55af5e40be40 msgr2=0x55af5e40c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:09.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.322+0000 7f717ac7d740 1 --2- 172.21.15.77:0/798829278 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55af5e40be40 0x55af5e40c220 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f7168009f50 tx=0x7f71680118d0 comp rx=0 tx=0).stop 2024-09-10T10:00:09.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.322+0000 7f717ac7d740 1 -- 172.21.15.77:0/798829278 shutdown_connections 2024-09-10T10:00:09.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.322+0000 7f717ac7d740 1 --2- 172.21.15.77:0/798829278 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55af5e417020 0x55af5e419410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:09.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.322+0000 7f7175800640 1 --2- 172.21.15.77:0/161172711 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f71580a2c00 0x7f71580a50c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f7164005ab0 tx=0x7f71640f5530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:09.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.322+0000 7f717ac7d740 1 --2- 172.21.15.77:0/798829278 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af5e40c760 0x55af5e416ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:09.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.322+0000 7f717ac7d740 1 --2- 172.21.15.77:0/798829278 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55af5e40be40 0x55af5e40c220 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:09.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.322+0000 7f717ac7d740 1 -- 172.21.15.77:0/798829278 >> 172.21.15.77:0/798829278 conn(0x55af5e4083d0 msgr2=0x55af5e4087b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:09.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.322+0000 7f717ac7d740 1 -- 172.21.15.77:0/798829278 shutdown_connections 2024-09-10T10:00:09.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.322+0000 7f717ac7d740 1 -- 172.21.15.77:0/798829278 wait complete. 2024-09-10T10:00:09.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.323+0000 7f717ac7d740 1 -- start start 2024-09-10T10:00:09.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.323+0000 7f717ac7d740 1 Processor -- start 2024-09-10T10:00:09.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.323+0000 7f717ac7d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55af5e40be40 0x55af5e413ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:09.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.323+0000 7f717ac7d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af5e40c760 0x55af5e414430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:09.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.323+0000 7f717ac7d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55af5e417020 0x55af5e414970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:09.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.323+0000 7f717ac7d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55af5e419bf0 con 0x55af5e40c760 2024-09-10T10:00:09.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.323+0000 7f717ac7d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55af5e32d0f0 con 0x55af5e417020 2024-09-10T10:00:09.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.323+0000 7f717ac7d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55af5e419d70 con 0x55af5e40be40 2024-09-10T10:00:09.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.323+0000 7f717ac7d740 10 client.0 _ll_register_callbacks cb 0x55af5e314bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:00:09.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.323+0000 7f7176200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55af5e40be40 0x55af5e413ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:09.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.323+0000 7f7176200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55af5e40be40 0x55af5e413ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:35802/0 (socket says 172.21.15.77:35802) 2024-09-10T10:00:09.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.323+0000 7f7176200640 1 -- 172.21.15.77:0/3268765079 learned_addr learned my addr 172.21.15.77:0/3268765079 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:00:09.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.324+0000 7f7174e00640 1 --2- 172.21.15.77:0/3268765079 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af5e40c760 0x55af5e414430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:09.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.324+0000 7f7176200640 1 --2- 172.21.15.77:0/3268765079 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55af5e417020 0x55af5e414970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:09.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.324+0000 7f7176200640 1 -- 172.21.15.77:0/3268765079 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55af5e417020 msgr2=0x55af5e414970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:09.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.324+0000 7f7176200640 1 --2- 172.21.15.77:0/3268765079 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55af5e417020 0x55af5e414970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:09.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.324+0000 7f7176200640 1 -- 172.21.15.77:0/3268765079 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af5e40c760 msgr2=0x55af5e414430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:09.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.324+0000 7f7176200640 1 --2- 172.21.15.77:0/3268765079 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af5e40c760 0x55af5e414430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:09.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.324+0000 7f7176200640 1 -- 172.21.15.77:0/3268765079 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af5e0e33c0 con 0x55af5e40be40 2024-09-10T10:00:09.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.324+0000 7f7174e00640 1 --2- 172.21.15.77:0/3268765079 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af5e40c760 0x55af5e414430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:00:09.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.324+0000 7f7176200640 1 --2- 172.21.15.77:0/3268765079 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55af5e40be40 0x55af5e413ef0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f7168009250 tx=0x7f7168005bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c573c172cf76dd9f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:09.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.324+0000 7f7157400640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-10T10:00:09.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.324+0000 7f7157400640 1 -- 172.21.15.77:0/3268765079 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f716802b070 con 0x55af5e40be40 2024-09-10T10:00:09.328 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.325+0000 7f717ac7d740 1 -- 172.21.15.77:0/3268765079 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7168004050 con 0x55af5e40be40 2024-09-10T10:00:09.328 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.325+0000 7f7157400640 1 -- 172.21.15.77:0/3268765079 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f7168027640 con 0x55af5e40be40 2024-09-10T10:00:09.328 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.325+0000 7f7157400640 1 -- 172.21.15.77:0/3268765079 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7168037050 con 0x55af5e40be40 2024-09-10T10:00:09.328 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.325+0000 7f717ac7d740 10 client.26467 fetch_fsmap learned FSMap version 32 2024-09-10T10:00:09.328 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.325+0000 7f717ac7d740 1 -- 172.21.15.77:0/3268765079 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55af5e425c40 con 0x55af5e40be40 2024-09-10T10:00:09.329 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.325+0000 7f7157400640 1 -- 172.21.15.77:0/3268765079 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7168004050 con 0x55af5e40be40 2024-09-10T10:00:09.329 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.325+0000 7f7157400640 1 -- 172.21.15.77:0/3268765079 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f716800f980 con 0x55af5e40be40 2024-09-10T10:00:09.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.325+0000 7f717ac7d740 10 client.26467 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T10:00:09.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.325+0000 7f717ac7d740 10 client.26467 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T10:00:09.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.325+0000 7f717ac7d740 20 client.26467 populate_metadata read hostname 'smithi077' 2024-09-10T10:00:09.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.325+0000 7f717ac7d740 1 --2- 172.21.15.77:0/3268765079 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x55af5e4270e0 0x55af5e4294d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:09.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.325+0000 7f717ac7d740 4 client.26467 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:00:09.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.325+0000 7f717ac7d740 1 -- 172.21.15.77:0/3268765079 --> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55af5e37f950 con 0x55af5e4270e0 2024-09-10T10:00:09.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.326+0000 7f7174e00640 1 --2- 172.21.15.77:0/3268765079 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x55af5e4270e0 0x55af5e4294d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:09.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.326+0000 7f7174e00640 1 --2- 172.21.15.77:0/3268765079 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x55af5e4270e0 0x55af5e4294d0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c75f859782fc814a server_cookie=5ffd8ac57d4e96a6 in_seq=0 out_seq=0 2024-09-10T10:00:09.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.326+0000 7f7157400640 10 client.26467 ms_handle_connect on v2:172.21.15.134:6836/2461497091 2024-09-10T10:00:09.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.327+0000 7f7157400640 1 -- 172.21.15.77:0/3268765079 <== mds.3 v2:172.21.15.134:6836/2461497091 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55af5e37f950 con 0x55af5e4270e0 2024-09-10T10:00:09.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.327+0000 7f7157400640 10 client.26467 handle_command_reply: tid=0 2024-09-10T10:00:09.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.327+0000 7f717ac7d740 1 client.26467 shutdown 2024-09-10T10:00:09.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.328+0000 7f717ac7d740 1 -- 172.21.15.77:0/3268765079 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55af5e40be40 msgr2=0x55af5e413ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:09.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.328+0000 7f717ac7d740 1 --2- 172.21.15.77:0/3268765079 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55af5e40be40 0x55af5e413ef0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f7168009250 tx=0x7f7168005bd0 comp rx=0 tx=0).stop 2024-09-10T10:00:09.331 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.328+0000 7f717ac7d740 1 -- 172.21.15.77:0/3268765079 shutdown_connections 2024-09-10T10:00:09.331 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.328+0000 7f717ac7d740 1 --2- 172.21.15.77:0/3268765079 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55af5e417020 0x55af5e414970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:09.331 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.328+0000 7f717ac7d740 1 --2- 172.21.15.77:0/3268765079 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x55af5e4270e0 0x55af5e4294d0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:09.331 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.328+0000 7f717ac7d740 1 --2- 172.21.15.77:0/3268765079 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af5e40c760 0x55af5e414430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:09.331 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.328+0000 7f717ac7d740 1 --2- 172.21.15.77:0/3268765079 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55af5e40be40 0x55af5e413ef0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:09.331 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.328+0000 7f7157400640 0 client.26467 ms_handle_reset on v2:172.21.15.134:6836/2461497091 2024-09-10T10:00:09.331 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.328+0000 7f717ac7d740 1 -- 172.21.15.77:0/3268765079 >> 172.21.15.77:0/3268765079 conn(0x55af5e4083d0 msgr2=0x55af5e40e100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:09.331 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.328+0000 7f717ac7d740 1 -- 172.21.15.77:0/3268765079 shutdown_connections 2024-09-10T10:00:09.331 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.328+0000 7f717ac7d740 1 -- 172.21.15.77:0/3268765079 wait complete. 2024-09-10T10:00:09.331 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.328+0000 7f717ac7d740 20 client.26467 trim_cache size 0 max 16384 2024-09-10T10:00:09.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.342+0000 7f717ac7d740 1 Processor -- start 2024-09-10T10:00:09.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.342+0000 7f717ac7d740 1 -- start start 2024-09-10T10:00:09.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.342+0000 7f717ac7d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55af5e40be40 0x55af5e413180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:09.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.342+0000 7f717ac7d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af5e40c760 0x55af5e40e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:09.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.342+0000 7f717ac7d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55af5e417020 0x55af5e42eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:09.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.342+0000 7f717ac7d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f71701b0ef0 con 0x55af5e40c760 2024-09-10T10:00:09.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.342+0000 7f717ac7d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f71701b2af0 con 0x55af5e417020 2024-09-10T10:00:09.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.342+0000 7f717ac7d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f71701b15c0 con 0x55af5e40be40 2024-09-10T10:00:09.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.343+0000 7f7176200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55af5e40be40 0x55af5e413180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:09.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.343+0000 7f7174e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af5e40c760 0x55af5e40e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:09.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.343+0000 7f7176200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55af5e40be40 0x55af5e413180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:35816/0 (socket says 172.21.15.77:35816) 2024-09-10T10:00:09.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.343+0000 7f7176200640 1 -- 172.21.15.77:0/1305633740 learned_addr learned my addr 172.21.15.77:0/1305633740 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:00:09.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.343+0000 7f7176200640 1 --2- 172.21.15.77:0/1305633740 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55af5e417020 0x55af5e42eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:09.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.343+0000 7f7174e00640 1 -- 172.21.15.77:0/1305633740 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55af5e40be40 msgr2=0x55af5e413180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:09.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.343+0000 7f7174e00640 1 --2- 172.21.15.77:0/1305633740 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55af5e40be40 0x55af5e413180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:09.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.343+0000 7f7174e00640 1 -- 172.21.15.77:0/1305633740 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55af5e417020 msgr2=0x55af5e42eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:09.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.343+0000 7f7174e00640 1 --2- 172.21.15.77:0/1305633740 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55af5e417020 0x55af5e42eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:09.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.343+0000 7f7174e00640 1 -- 172.21.15.77:0/1305633740 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af5e407c80 con 0x55af5e40c760 2024-09-10T10:00:09.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.343+0000 7f7174e00640 1 --2- 172.21.15.77:0/1305633740 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af5e40c760 0x55af5e40e8a0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x55af5e429b20 tx=0x7f715c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45ba39bc7361741c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:09.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.344+0000 7f7162000640 1 -- 172.21.15.77:0/1305633740 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f715c005220 con 0x55af5e40c760 2024-09-10T10:00:09.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.344+0000 7f7162000640 1 -- 172.21.15.77:0/1305633740 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f715c004c80 con 0x55af5e40c760 2024-09-10T10:00:09.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.344+0000 7f7162000640 1 -- 172.21.15.77:0/1305633740 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f715c010990 con 0x55af5e40c760 2024-09-10T10:00:09.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.344+0000 7f717ac7d740 1 -- 172.21.15.77:0/1305633740 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af5e40c760 msgr2=0x55af5e40e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:09.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.344+0000 7f717ac7d740 1 --2- 172.21.15.77:0/1305633740 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af5e40c760 0x55af5e40e8a0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x55af5e429b20 tx=0x7f715c005f90 comp rx=0 tx=0).stop 2024-09-10T10:00:09.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.344+0000 7f717ac7d740 1 -- 172.21.15.77:0/1305633740 shutdown_connections 2024-09-10T10:00:09.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.344+0000 7f717ac7d740 1 --2- 172.21.15.77:0/1305633740 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55af5e417020 0x55af5e42eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:09.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.344+0000 7f717ac7d740 1 --2- 172.21.15.77:0/1305633740 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af5e40c760 0x55af5e40e8a0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:09.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.345+0000 7f717ac7d740 1 --2- 172.21.15.77:0/1305633740 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55af5e40be40 0x55af5e413180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:09.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.345+0000 7f717ac7d740 1 -- 172.21.15.77:0/1305633740 >> 172.21.15.77:0/1305633740 conn(0x55af5e4083d0 msgr2=0x55af5e4146d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:09.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.345+0000 7f717ac7d740 1 -- 172.21.15.77:0/1305633740 shutdown_connections 2024-09-10T10:00:09.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.345+0000 7f717ac7d740 1 -- 172.21.15.77:0/1305633740 wait complete. 2024-09-10T10:00:09.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.345+0000 7f717ac7d740 1 -- start start 2024-09-10T10:00:09.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.345+0000 7f717ac7d740 1 Processor -- start 2024-09-10T10:00:09.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.346+0000 7f717ac7d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55af5e40be40 0x55af5e41f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:09.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.346+0000 7f717ac7d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af5e40c760 0x55af5e41f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:09.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.346+0000 7f717ac7d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55af5e417020 0x55af5e410720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:09.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.346+0000 7f717ac7d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f71701b2360 con 0x55af5e40c760 2024-09-10T10:00:09.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.346+0000 7f717ac7d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f71701b1c90 con 0x55af5e417020 2024-09-10T10:00:09.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.346+0000 7f717ac7d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f71701aeed0 con 0x55af5e40be40 2024-09-10T10:00:09.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.346+0000 7f717ac7d740 10 client.0 _ll_register_callbacks cb 0x7f71640f5400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:00:09.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.346+0000 7f7176200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55af5e40be40 0x55af5e41f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:09.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.346+0000 7f7174e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af5e40c760 0x55af5e41f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:09.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.346+0000 7f7174e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af5e40c760 0x55af5e41f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:51504/0 (socket says 172.21.15.77:51504) 2024-09-10T10:00:09.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.346+0000 7f7174e00640 1 -- 172.21.15.77:0/2152954920 learned_addr learned my addr 172.21.15.77:0/2152954920 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:00:09.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.346+0000 7f7176200640 1 --2- 172.21.15.77:0/2152954920 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55af5e417020 0x55af5e410720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:09.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.347+0000 7f7174e00640 1 -- 172.21.15.77:0/2152954920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55af5e40be40 msgr2=0x55af5e41f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:09.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.347+0000 7f7174e00640 1 --2- 172.21.15.77:0/2152954920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55af5e40be40 0x55af5e41f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:09.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.347+0000 7f7174e00640 1 -- 172.21.15.77:0/2152954920 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55af5e417020 msgr2=0x55af5e410720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:09.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.347+0000 7f7174e00640 1 --2- 172.21.15.77:0/2152954920 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55af5e417020 0x55af5e410720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:09.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.347+0000 7f7174e00640 1 -- 172.21.15.77:0/2152954920 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af5e410f50 con 0x55af5e40c760 2024-09-10T10:00:09.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.347+0000 7f7176200640 1 --2- 172.21.15.77:0/2152954920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55af5e40be40 0x55af5e41f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:00:09.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.347+0000 7f7174e00640 1 --2- 172.21.15.77:0/2152954920 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af5e40c760 0x55af5e41f710 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f715c005890 tx=0x7f715c0120e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e64ce5c3ebe119a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:09.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.347+0000 7f7157400640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-10T10:00:09.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.347+0000 7f7157400640 1 -- 172.21.15.77:0/2152954920 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f715c036060 con 0x55af5e40c760 2024-09-10T10:00:09.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.347+0000 7f717ac7d740 1 -- 172.21.15.77:0/2152954920 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f715c004c80 con 0x55af5e40c760 2024-09-10T10:00:09.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.347+0000 7f7157400640 1 -- 172.21.15.77:0/2152954920 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f715c004950 con 0x55af5e40c760 2024-09-10T10:00:09.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.347+0000 7f7157400640 1 -- 172.21.15.77:0/2152954920 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f715c010710 con 0x55af5e40c760 2024-09-10T10:00:09.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.348+0000 7f717ac7d740 10 client.16446 fetch_fsmap learned FSMap version 32 2024-09-10T10:00:09.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.348+0000 7f717ac7d740 1 -- 172.21.15.77:0/2152954920 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55af5e4915b0 con 0x55af5e40c760 2024-09-10T10:00:09.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.348+0000 7f7157400640 1 -- 172.21.15.77:0/2152954920 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f715c004c80 con 0x55af5e40c760 2024-09-10T10:00:09.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.348+0000 7f7157400640 1 -- 172.21.15.77:0/2152954920 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f715c0053c0 con 0x55af5e40c760 2024-09-10T10:00:09.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.348+0000 7f717ac7d740 10 client.16446 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T10:00:09.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.348+0000 7f717ac7d740 10 client.16446 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T10:00:09.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.348+0000 7f717ac7d740 20 client.16446 populate_metadata read hostname 'smithi077' 2024-09-10T10:00:09.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.348+0000 7f717ac7d740 1 --2- 172.21.15.77:0/2152954920 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x55af5e491d00 0x55af5e494120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:09.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.348+0000 7f717ac7d740 4 client.16446 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:00:09.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.348+0000 7f717ac7d740 1 -- 172.21.15.77:0/2152954920 --> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55af5e379e60 con 0x55af5e491d00 2024-09-10T10:00:09.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.348+0000 7f7176200640 1 --2- 172.21.15.77:0/2152954920 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x55af5e491d00 0x55af5e494120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:09.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.349+0000 7f7176200640 1 --2- 172.21.15.77:0/2152954920 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x55af5e491d00 0x55af5e494120 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=813af88f883f9815 server_cookie=444d134b2f284156 in_seq=0 out_seq=0 2024-09-10T10:00:09.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.349+0000 7f7157400640 10 client.16446 ms_handle_connect on v2:172.21.15.134:6836/2461497091 2024-09-10T10:00:09.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.349+0000 7f7157400640 1 -- 172.21.15.77:0/2152954920 <== mds.3 v2:172.21.15.134:6836/2461497091 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55af5e379e60 con 0x55af5e491d00 2024-09-10T10:00:09.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.349+0000 7f7157400640 10 client.16446 handle_command_reply: tid=0 2024-09-10T10:00:09.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.349+0000 7f717ac7d740 1 client.16446 shutdown 2024-09-10T10:00:09.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.349+0000 7f717ac7d740 1 -- 172.21.15.77:0/2152954920 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af5e40c760 msgr2=0x55af5e41f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:09.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.349+0000 7f717ac7d740 1 --2- 172.21.15.77:0/2152954920 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af5e40c760 0x55af5e41f710 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f715c005890 tx=0x7f715c0120e0 comp rx=0 tx=0).stop 2024-09-10T10:00:09.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.349+0000 7f717ac7d740 1 -- 172.21.15.77:0/2152954920 shutdown_connections 2024-09-10T10:00:09.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.349+0000 7f717ac7d740 1 --2- 172.21.15.77:0/2152954920 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55af5e417020 0x55af5e410720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:09.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.349+0000 7f717ac7d740 1 --2- 172.21.15.77:0/2152954920 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x55af5e491d00 0x55af5e494120 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:09.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.349+0000 7f717ac7d740 1 --2- 172.21.15.77:0/2152954920 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af5e40c760 0x55af5e41f710 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:09.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.349+0000 7f717ac7d740 1 --2- 172.21.15.77:0/2152954920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55af5e40be40 0x55af5e41f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:09.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.349+0000 7f717ac7d740 1 -- 172.21.15.77:0/2152954920 >> 172.21.15.77:0/2152954920 conn(0x55af5e4083d0 msgr2=0x55af5e413e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:09.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.349+0000 7f7157400640 0 client.16446 ms_handle_reset on v2:172.21.15.134:6836/2461497091 2024-09-10T10:00:09.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.350+0000 7f717ac7d740 1 -- 172.21.15.77:0/2152954920 shutdown_connections 2024-09-10T10:00:09.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.350+0000 7f717ac7d740 1 -- 172.21.15.77:0/2152954920 wait complete. 2024-09-10T10:00:09.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.350+0000 7f717ac7d740 20 client.16446 trim_cache size 0 max 16384 2024-09-10T10:00:09.354 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-10T10:00:09.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.350+0000 7f7176c00640 1 -- 172.21.15.77:0/161172711 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f71580a2c00 msgr2=0x7f71580a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:09.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.350+0000 7f7176c00640 1 --2- 172.21.15.77:0/161172711 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f71580a2c00 0x7f71580a50c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f7164005ab0 tx=0x7f71640f5530 comp rx=0 tx=0).stop 2024-09-10T10:00:09.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.350+0000 7f7176c00640 1 -- 172.21.15.77:0/161172711 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7170102b90 msgr2=0x7f717019a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:09.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.350+0000 7f7176c00640 1 --2- 172.21.15.77:0/161172711 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7170102b90 0x7f717019a4d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7164006010 tx=0x7f7164004040 comp rx=0 tx=0).stop 2024-09-10T10:00:09.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.350+0000 7f7176c00640 1 -- 172.21.15.77:0/161172711 shutdown_connections 2024-09-10T10:00:09.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.350+0000 7f7176c00640 1 --2- 172.21.15.77:0/161172711 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f71580a2c00 0x7f71580a50c0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:09.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.350+0000 7f7176c00640 1 --2- 172.21.15.77:0/161172711 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f717010bfc0 0x7f717019eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:09.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.350+0000 7f7176c00640 1 --2- 172.21.15.77:0/161172711 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f71701034b0 0x7f717019aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:09.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.350+0000 7f7176c00640 1 --2- 172.21.15.77:0/161172711 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7170102b90 0x7f717019a4d0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:09.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.350+0000 7f7176c00640 1 -- 172.21.15.77:0/161172711 >> 172.21.15.77:0/161172711 conn(0x7f71700fc610 msgr2=0x7f717010d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:09.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.351+0000 7f7176c00640 1 -- 172.21.15.77:0/161172711 shutdown_connections 2024-09-10T10:00:09.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:09.351+0000 7f7176c00640 1 -- 172.21.15.77:0/161172711 wait complete. 2024-09-10T10:00:09.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:00:08 smithi134 ceph-mon[35674]: pgmap v540: 129 pgs: 129 active+clean; 97 MiB data, 935 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9.8 KiB/s wr, 10 op/s 2024-09-10T10:00:10.117 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T10:00:10.492 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:00:11.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.062+0000 7fc1bc400640 1 -- 172.21.15.77:0/915641049 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1b4102790 msgr2=0x7fc1b41067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:11.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.062+0000 7fc1bc400640 1 --2- 172.21.15.77:0/915641049 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1b4102790 0x7fc1b41067e0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fc1a0009900 tx=0x7fc1a002a3f0 comp rx=0 tx=0).stop 2024-09-10T10:00:11.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.063+0000 7fc1bc400640 1 -- 172.21.15.77:0/915641049 shutdown_connections 2024-09-10T10:00:11.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.063+0000 7fc1bc400640 1 --2- 172.21.15.77:0/915641049 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1b4106f10 0x7fc1b410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:11.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.063+0000 7fc1bc400640 1 --2- 172.21.15.77:0/915641049 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1b4102790 0x7fc1b41067e0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:11.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.063+0000 7fc1bc400640 1 --2- 172.21.15.77:0/915641049 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc1b4101de0 0x7fc1b41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:11.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.063+0000 7fc1bc400640 1 -- 172.21.15.77:0/915641049 >> 172.21.15.77:0/915641049 conn(0x7fc1b40fd8c0 msgr2=0x7fc1b40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:11.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.063+0000 7fc1bc400640 1 -- 172.21.15.77:0/915641049 shutdown_connections 2024-09-10T10:00:11.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.063+0000 7fc1bc400640 1 -- 172.21.15.77:0/915641049 wait complete. 2024-09-10T10:00:11.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.063+0000 7fc1bc400640 1 Processor -- start 2024-09-10T10:00:11.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.064+0000 7fc1bc400640 1 -- start start 2024-09-10T10:00:11.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.064+0000 7fc1bc400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1b4101de0 0x7fc1b4198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:11.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.064+0000 7fc1bc400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc1b4102790 0x7fc1b41989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:11.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.064+0000 7fc1bc400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1b4106f10 0x7fc1b419cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:11.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.064+0000 7fc1bc400640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b4111fc0 con 0x7fc1b4101de0 2024-09-10T10:00:11.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.064+0000 7fc1bc400640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b4111e40 con 0x7fc1b4102790 2024-09-10T10:00:11.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.064+0000 7fc1bc400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b4112140 con 0x7fc1b4106f10 2024-09-10T10:00:11.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.064+0000 7fc1b9a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1b4101de0 0x7fc1b4198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:11.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.064+0000 7fc1b9000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc1b4102790 0x7fc1b41989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:11.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.064+0000 7fc1b9a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1b4101de0 0x7fc1b4198480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:51524/0 (socket says 172.21.15.77:51524) 2024-09-10T10:00:11.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.064+0000 7fc1b9a00640 1 -- 172.21.15.77:0/3885309671 learned_addr learned my addr 172.21.15.77:0/3885309671 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:00:11.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.064+0000 7fc1ba400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1b4106f10 0x7fc1b419cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:11.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.065+0000 7fc1b9000640 1 -- 172.21.15.77:0/3885309671 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1b4106f10 msgr2=0x7fc1b419cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:11.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.065+0000 7fc1b9000640 1 --2- 172.21.15.77:0/3885309671 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1b4106f10 0x7fc1b419cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:11.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.065+0000 7fc1b9000640 1 -- 172.21.15.77:0/3885309671 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1b4101de0 msgr2=0x7fc1b4198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:11.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.065+0000 7fc1b9000640 1 --2- 172.21.15.77:0/3885309671 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1b4101de0 0x7fc1b4198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:11.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.065+0000 7fc1b9000640 1 -- 172.21.15.77:0/3885309671 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1b419d430 con 0x7fc1b4102790 2024-09-10T10:00:11.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.065+0000 7fc1b9a00640 1 --2- 172.21.15.77:0/3885309671 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1b4101de0 0x7fc1b4198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:00:11.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.065+0000 7fc1b9000640 1 --2- 172.21.15.77:0/3885309671 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc1b4102790 0x7fc1b41989c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc1a002a910 tx=0x7fc1a0003b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:11.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.065+0000 7fc1b2000640 1 -- 172.21.15.77:0/3885309671 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a003f070 con 0x7fc1b4102790 2024-09-10T10:00:11.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.065+0000 7fc1bc400640 1 -- 172.21.15.77:0/3885309671 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1b419d6c0 con 0x7fc1b4102790 2024-09-10T10:00:11.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.065+0000 7fc1bc400640 1 -- 172.21.15.77:0/3885309671 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1b41a5000 con 0x7fc1b4102790 2024-09-10T10:00:11.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.066+0000 7fc1b2000640 1 -- 172.21.15.77:0/3885309671 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1a0031c90 con 0x7fc1b4102790 2024-09-10T10:00:11.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.066+0000 7fc1b2000640 1 -- 172.21.15.77:0/3885309671 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a0035030 con 0x7fc1b4102790 2024-09-10T10:00:11.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.068+0000 7fc1bee03740 1 Processor -- start 2024-09-10T10:00:11.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.069+0000 7fc1bee03740 1 -- start start 2024-09-10T10:00:11.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.069+0000 7fc1b2000640 1 -- 172.21.15.77:0/3885309671 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc1a0042070 con 0x7fc1b4102790 2024-09-10T10:00:11.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.069+0000 7fc1bee03740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560df3882c70 0x560df3883050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:11.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.069+0000 7fc1bee03740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560df3883590 0x560df388d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:11.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.069+0000 7fc1bee03740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560df388de50 0x560df3890240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:11.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.069+0000 7fc1bee03740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560df37e93e0 con 0x560df388de50 2024-09-10T10:00:11.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.069+0000 7fc1bee03740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x560df3790d80 con 0x560df3882c70 2024-09-10T10:00:11.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.069+0000 7fc1bee03740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560df37e53a0 con 0x560df3883590 2024-09-10T10:00:11.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.069+0000 7fc1ba400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560df3882c70 0x560df3883050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:11.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.069+0000 7fc1ba400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560df3882c70 0x560df3883050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:32808/0 (socket says 172.21.15.77:32808) 2024-09-10T10:00:11.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.069+0000 7fc1ba400640 1 -- 172.21.15.77:0/4128288678 learned_addr learned my addr 172.21.15.77:0/4128288678 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:00:11.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.069+0000 7fc1b9000640 1 --2- 172.21.15.77:0/4128288678 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560df388de50 0x560df3890240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:11.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.069+0000 7fc1b9a00640 1 --2- 172.21.15.77:0/4128288678 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560df3883590 0x560df388d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:11.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.070+0000 7fc1ba400640 1 -- 172.21.15.77:0/4128288678 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560df3883590 msgr2=0x560df388d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:11.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.070+0000 7fc1ba400640 1 --2- 172.21.15.77:0/4128288678 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560df3883590 0x560df388d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:11.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.070+0000 7fc1ba400640 1 -- 172.21.15.77:0/4128288678 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560df388de50 msgr2=0x560df3890240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:11.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.070+0000 7fc1ba400640 1 --2- 172.21.15.77:0/4128288678 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560df388de50 0x560df3890240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:11.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.070+0000 7fc1ba400640 1 -- 172.21.15.77:0/4128288678 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560df37afae0 con 0x560df3882c70 2024-09-10T10:00:11.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.070+0000 7fc1b9a00640 1 --2- 172.21.15.77:0/4128288678 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560df3883590 0x560df388d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:00:11.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.070+0000 7fc1ba400640 1 --2- 172.21.15.77:0/4128288678 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560df3882c70 0x560df3883050 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac009f50 tx=0x7fc1ac0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29243972eddf21be server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:11.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.070+0000 7fc1a6a00640 1 -- 172.21.15.77:0/4128288678 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1ac004d20 con 0x560df3882c70 2024-09-10T10:00:11.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.070+0000 7fc1b2000640 1 --2- 172.21.15.77:0/3885309671 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fc19c0a2b80 0x7fc19c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:11.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.070+0000 7fc1a6a00640 1 -- 172.21.15.77:0/4128288678 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1ac004050 con 0x560df3882c70 2024-09-10T10:00:11.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.070+0000 7fc1a6a00640 1 -- 172.21.15.77:0/4128288678 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1ac005280 con 0x560df3882c70 2024-09-10T10:00:11.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.070+0000 7fc1b2000640 1 -- 172.21.15.77:0/3885309671 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fc1a00ef790 con 0x7fc1b4102790 2024-09-10T10:00:11.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.071+0000 7fc1b9a00640 1 --2- 172.21.15.77:0/3885309671 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fc19c0a2b80 0x7fc19c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:11.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.071+0000 7fc1bee03740 1 -- 172.21.15.77:0/4128288678 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560df3882c70 msgr2=0x560df3883050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:11.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.071+0000 7fc1bee03740 1 --2- 172.21.15.77:0/4128288678 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560df3882c70 0x560df3883050 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac009f50 tx=0x7fc1ac0118d0 comp rx=0 tx=0).stop 2024-09-10T10:00:11.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.071+0000 7fc1bee03740 1 -- 172.21.15.77:0/4128288678 shutdown_connections 2024-09-10T10:00:11.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.071+0000 7fc1bee03740 1 --2- 172.21.15.77:0/4128288678 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560df388de50 0x560df3890240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:11.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.071+0000 7fc1bee03740 1 --2- 172.21.15.77:0/4128288678 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560df3883590 0x560df388d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:11.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.071+0000 7fc1bee03740 1 --2- 172.21.15.77:0/4128288678 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560df3882c70 0x560df3883050 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:11.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.071+0000 7fc1bee03740 1 -- 172.21.15.77:0/4128288678 >> 172.21.15.77:0/4128288678 conn(0x560df387f200 msgr2=0x560df387f5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:11.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.071+0000 7fc1bee03740 1 -- 172.21.15.77:0/4128288678 shutdown_connections 2024-09-10T10:00:11.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.071+0000 7fc1bee03740 1 -- 172.21.15.77:0/4128288678 wait complete. 2024-09-10T10:00:11.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.071+0000 7fc1b9a00640 1 --2- 172.21.15.77:0/3885309671 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fc19c0a2b80 0x7fc19c0a5040 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x560df3880060 tx=0x7fc1a8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:11.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.072+0000 7fc1bee03740 1 -- start start 2024-09-10T10:00:11.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.072+0000 7fc1bee03740 1 Processor -- start 2024-09-10T10:00:11.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.072+0000 7fc1bee03740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560df3882c70 0x560df3892350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:11.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.072+0000 7fc1bee03740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560df3883590 0x560df3892890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:11.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.072+0000 7fc1bee03740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560df388de50 0x560df3896a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:11.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.072+0000 7fc1bee03740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560df3890a20 con 0x560df388de50 2024-09-10T10:00:11.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.072+0000 7fc1bee03740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x560df37a40f0 con 0x560df3882c70 2024-09-10T10:00:11.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.072+0000 7fc1bee03740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560df3890ba0 con 0x560df3883590 2024-09-10T10:00:11.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.072+0000 7fc1bee03740 10 client.0 _ll_register_callbacks cb 0x560df378bbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:00:11.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.073+0000 7fc1b9a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560df3882c70 0x560df3892350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:11.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.073+0000 7fc1b9000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560df3883590 0x560df3892890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:11.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.073+0000 7fc1b9a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560df3882c70 0x560df3892350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:32816/0 (socket says 172.21.15.77:32816) 2024-09-10T10:00:11.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.073+0000 7fc1b9a00640 1 -- 172.21.15.77:0/3196799030 learned_addr learned my addr 172.21.15.77:0/3196799030 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:00:11.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.073+0000 7fc1ba400640 1 --2- 172.21.15.77:0/3196799030 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560df388de50 0x560df3896a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:11.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.073+0000 7fc1b9000640 1 -- 172.21.15.77:0/3196799030 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560df3882c70 msgr2=0x560df3892350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:11.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.073+0000 7fc1b9000640 1 --2- 172.21.15.77:0/3196799030 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560df3882c70 0x560df3892350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:11.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.073+0000 7fc1b9000640 1 -- 172.21.15.77:0/3196799030 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560df388de50 msgr2=0x560df3896a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:11.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.073+0000 7fc1b9000640 1 --2- 172.21.15.77:0/3196799030 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560df388de50 0x560df3896a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:11.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.073+0000 7fc1b9000640 1 -- 172.21.15.77:0/3196799030 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560df355a3c0 con 0x560df3883590 2024-09-10T10:00:11.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.073+0000 7fc1b9a00640 1 --2- 172.21.15.77:0/3196799030 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560df3882c70 0x560df3892350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:00:11.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.073+0000 7fc1b9000640 1 --2- 172.21.15.77:0/3196799030 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560df3883590 0x560df3892890 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc1a00ecef0 tx=0x7fc1a0031490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=741230a8dc2a4b28 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:11.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.073+0000 7fc19b400640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-10T10:00:11.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.073+0000 7fc19b400640 1 -- 172.21.15.77:0/3196799030 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a00bb740 con 0x560df3883590 2024-09-10T10:00:11.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.073+0000 7fc19b400640 1 -- 172.21.15.77:0/3196799030 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fc1a00b8270 con 0x560df3883590 2024-09-10T10:00:11.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.073+0000 7fc1bee03740 1 -- 172.21.15.77:0/3196799030 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1ac004050 con 0x560df3883590 2024-09-10T10:00:11.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.074+0000 7fc19b400640 1 -- 172.21.15.77:0/3196799030 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc1a0005d50 con 0x560df3883590 2024-09-10T10:00:11.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.074+0000 7fc1bee03740 10 client.26485 fetch_fsmap learned FSMap version 32 2024-09-10T10:00:11.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.074+0000 7fc1bee03740 1 -- 172.21.15.77:0/3196799030 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560df3890a80 con 0x560df3883590 2024-09-10T10:00:11.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.074+0000 7fc19b400640 1 -- 172.21.15.77:0/3196799030 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1ac004050 con 0x560df3883590 2024-09-10T10:00:11.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.074+0000 7fc19b400640 1 -- 172.21.15.77:0/3196799030 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fc1a00c43e0 con 0x560df3883590 2024-09-10T10:00:11.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.074+0000 7fc1bee03740 10 client.26485 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T10:00:11.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.074+0000 7fc1bee03740 10 client.26485 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T10:00:11.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.074+0000 7fc1bee03740 20 client.26485 populate_metadata read hostname 'smithi077' 2024-09-10T10:00:11.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.074+0000 7fc1bee03740 1 --2- 172.21.15.77:0/3196799030 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x560df38a2300 0x560df38a47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:11.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.074+0000 7fc1bee03740 4 client.26485 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:00:11.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.074+0000 7fc1bee03740 1 -- 172.21.15.77:0/3196799030 --> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560df37f6950 con 0x560df38a2300 2024-09-10T10:00:11.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.074+0000 7fc1b9a00640 1 --2- 172.21.15.77:0/3196799030 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x560df38a2300 0x560df38a47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:11.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.075+0000 7fc1b9a00640 1 --2- 172.21.15.77:0/3196799030 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x560df38a2300 0x560df38a47a0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=390dd5b7925d6b5d server_cookie=4be6de9414a3673c in_seq=0 out_seq=0 2024-09-10T10:00:11.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.075+0000 7fc19b400640 10 client.26485 ms_handle_connect on v2:172.21.15.134:6838/1965117680 2024-09-10T10:00:11.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.076+0000 7fc19b400640 1 -- 172.21.15.77:0/3196799030 <== mds.2 v2:172.21.15.134:6838/1965117680 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560df37f6950 con 0x560df38a2300 2024-09-10T10:00:11.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.076+0000 7fc19b400640 10 client.26485 handle_command_reply: tid=0 2024-09-10T10:00:11.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.076+0000 7fc1bee03740 1 client.26485 shutdown 2024-09-10T10:00:11.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.076+0000 7fc1bee03740 1 -- 172.21.15.77:0/3196799030 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560df3883590 msgr2=0x560df3892890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:11.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.076+0000 7fc1bee03740 1 --2- 172.21.15.77:0/3196799030 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560df3883590 0x560df3892890 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc1a00ecef0 tx=0x7fc1a0031490 comp rx=0 tx=0).stop 2024-09-10T10:00:11.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.076+0000 7fc1bee03740 1 -- 172.21.15.77:0/3196799030 shutdown_connections 2024-09-10T10:00:11.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.076+0000 7fc1bee03740 1 --2- 172.21.15.77:0/3196799030 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x560df38a2300 0x560df38a47a0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:11.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.076+0000 7fc1bee03740 1 --2- 172.21.15.77:0/3196799030 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560df388de50 0x560df3896a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:11.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.076+0000 7fc1bee03740 1 --2- 172.21.15.77:0/3196799030 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560df3883590 0x560df3892890 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:11.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.076+0000 7fc19b400640 0 client.26485 ms_handle_reset on v2:172.21.15.134:6838/1965117680 2024-09-10T10:00:11.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.076+0000 7fc1bee03740 1 --2- 172.21.15.77:0/3196799030 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560df3882c70 0x560df3892350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:11.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.076+0000 7fc1bee03740 1 -- 172.21.15.77:0/3196799030 >> 172.21.15.77:0/3196799030 conn(0x560df387f200 msgr2=0x560df387e380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:11.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.076+0000 7fc1bee03740 1 -- 172.21.15.77:0/3196799030 shutdown_connections 2024-09-10T10:00:11.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.076+0000 7fc1bee03740 1 -- 172.21.15.77:0/3196799030 wait complete. 2024-09-10T10:00:11.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.076+0000 7fc1bee03740 20 client.26485 trim_cache size 0 max 16384 2024-09-10T10:00:11.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.090+0000 7fc1bee03740 1 Processor -- start 2024-09-10T10:00:11.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.090+0000 7fc1bee03740 1 -- start start 2024-09-10T10:00:11.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.091+0000 7fc1bee03740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560df3882c70 0x560df3898510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:11.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.091+0000 7fc1bee03740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560df3883590 0x560df38aa0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:11.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.091+0000 7fc1bee03740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560df388de50 0x560df38aa5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:11.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.091+0000 7fc1bee03740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc19400fcd0 con 0x560df388de50 2024-09-10T10:00:11.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.091+0000 7fc1bee03740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc1940118d0 con 0x560df3882c70 2024-09-10T10:00:11.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.091+0000 7fc1bee03740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc1940103a0 con 0x560df3883590 2024-09-10T10:00:11.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.091+0000 7fc1b9a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560df3882c70 0x560df3898510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:11.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.091+0000 7fc1b9000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560df3883590 0x560df38aa0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:11.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.091+0000 7fc1b9a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560df3882c70 0x560df3898510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:32820/0 (socket says 172.21.15.77:32820) 2024-09-10T10:00:11.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.091+0000 7fc1b9000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560df3883590 0x560df38aa0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:35880/0 (socket says 172.21.15.77:35880) 2024-09-10T10:00:11.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.091+0000 7fc1b9a00640 1 -- 172.21.15.77:0/3842071007 learned_addr learned my addr 172.21.15.77:0/3842071007 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:00:11.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.091+0000 7fc1ba400640 1 --2- 172.21.15.77:0/3842071007 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560df388de50 0x560df38aa5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:11.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.091+0000 7fc1b9a00640 1 -- 172.21.15.77:0/3842071007 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560df3883590 msgr2=0x560df38aa0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:11.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.091+0000 7fc1b9a00640 1 --2- 172.21.15.77:0/3842071007 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560df3883590 0x560df38aa0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:11.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.091+0000 7fc1b9a00640 1 -- 172.21.15.77:0/3842071007 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560df388de50 msgr2=0x560df38aa5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:11.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.091+0000 7fc1b9a00640 1 --2- 172.21.15.77:0/3842071007 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560df388de50 0x560df38aa5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:11.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.091+0000 7fc1b9a00640 1 -- 172.21.15.77:0/3842071007 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560df38a4e20 con 0x560df3882c70 2024-09-10T10:00:11.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.091+0000 7fc1b9000640 1 --2- 172.21.15.77:0/3842071007 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560df3883590 0x560df38aa0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:00:11.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.092+0000 7fc1b9a00640 1 --2- 172.21.15.77:0/3842071007 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560df3882c70 0x560df3898510 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x560df38a5020 tx=0x7fc1a801a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=979f2a0a4687ed9b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:11.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.092+0000 7fc1a6000640 1 -- 172.21.15.77:0/3842071007 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a800fd20 con 0x560df3882c70 2024-09-10T10:00:11.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.092+0000 7fc1a6000640 1 -- 172.21.15.77:0/3842071007 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1a8005500 con 0x560df3882c70 2024-09-10T10:00:11.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.092+0000 7fc1a6000640 1 -- 172.21.15.77:0/3842071007 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a8004a00 con 0x560df3882c70 2024-09-10T10:00:11.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.093+0000 7fc1bee03740 1 -- 172.21.15.77:0/3842071007 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560df3882c70 msgr2=0x560df3898510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:11.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.093+0000 7fc1bee03740 1 --2- 172.21.15.77:0/3842071007 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560df3882c70 0x560df3898510 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x560df38a5020 tx=0x7fc1a801a000 comp rx=0 tx=0).stop 2024-09-10T10:00:11.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.093+0000 7fc1bee03740 1 -- 172.21.15.77:0/3842071007 shutdown_connections 2024-09-10T10:00:11.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.093+0000 7fc1bee03740 1 --2- 172.21.15.77:0/3842071007 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560df388de50 0x560df38aa5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:11.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.093+0000 7fc1bee03740 1 --2- 172.21.15.77:0/3842071007 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560df3883590 0x560df38aa0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:11.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.093+0000 7fc1bee03740 1 --2- 172.21.15.77:0/3842071007 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560df3882c70 0x560df3898510 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:11.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.093+0000 7fc1bee03740 1 -- 172.21.15.77:0/3842071007 >> 172.21.15.77:0/3842071007 conn(0x560df387f200 msgr2=0x560df3884ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:11.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.093+0000 7fc1bee03740 1 -- 172.21.15.77:0/3842071007 shutdown_connections 2024-09-10T10:00:11.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.093+0000 7fc1bee03740 1 -- 172.21.15.77:0/3842071007 wait complete. 2024-09-10T10:00:11.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.093+0000 7fc1bee03740 1 -- start start 2024-09-10T10:00:11.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.093+0000 7fc1bee03740 1 Processor -- start 2024-09-10T10:00:11.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.094+0000 7fc1bee03740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560df3882c70 0x560df389a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:11.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.094+0000 7fc1bee03740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560df3883590 0x560df389abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:11.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.094+0000 7fc1bee03740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560df388de50 0x560df3906130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:11.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.094+0000 7fc1bee03740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc194011140 con 0x560df3882c70 2024-09-10T10:00:11.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.094+0000 7fc1bee03740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc194010a70 con 0x560df388de50 2024-09-10T10:00:11.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.094+0000 7fc1bee03740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc19400dcb0 con 0x560df3883590 2024-09-10T10:00:11.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.094+0000 7fc1bee03740 10 client.0 _ll_register_callbacks cb 0x7fc1a0003f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:00:11.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.094+0000 7fc1b9000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560df3883590 0x560df389abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:11.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.094+0000 7fc1ba400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560df388de50 0x560df3906130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:11.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.094+0000 7fc1b9a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560df3882c70 0x560df389a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:11.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.094+0000 7fc1b9000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560df3883590 0x560df389abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:35882/0 (socket says 172.21.15.77:35882) 2024-09-10T10:00:11.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.094+0000 7fc1b9000640 1 -- 172.21.15.77:0/3879109091 learned_addr learned my addr 172.21.15.77:0/3879109091 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:00:11.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.094+0000 7fc1b9000640 1 -- 172.21.15.77:0/3879109091 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560df388de50 msgr2=0x560df3906130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:11.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.094+0000 7fc1b9000640 1 --2- 172.21.15.77:0/3879109091 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560df388de50 0x560df3906130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:11.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.094+0000 7fc1b9000640 1 -- 172.21.15.77:0/3879109091 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560df3882c70 msgr2=0x560df389a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:11.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.094+0000 7fc1b9000640 1 --2- 172.21.15.77:0/3879109091 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560df3882c70 0x560df389a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:11.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.095+0000 7fc1b9000640 1 -- 172.21.15.77:0/3879109091 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560df3892c00 con 0x560df3883590 2024-09-10T10:00:11.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.095+0000 7fc1b9000640 1 --2- 172.21.15.77:0/3879109091 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560df3883590 0x560df389abb0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc1a00b8060 tx=0x7fc1a00b84a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1655777930d80e2f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:11.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.095+0000 7fc19b400640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-10T10:00:11.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.095+0000 7fc19b400640 1 -- 172.21.15.77:0/3879109091 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a00bb560 con 0x560df3883590 2024-09-10T10:00:11.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.095+0000 7fc19b400640 1 -- 172.21.15.77:0/3879109091 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fc1a00d4e00 con 0x560df3883590 2024-09-10T10:00:11.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.095+0000 7fc1bee03740 1 -- 172.21.15.77:0/3879109091 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1a8005500 con 0x560df3883590 2024-09-10T10:00:11.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.095+0000 7fc19b400640 1 -- 172.21.15.77:0/3879109091 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc1a00bc300 con 0x560df3883590 2024-09-10T10:00:11.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.095+0000 7fc1bee03740 10 client.26491 fetch_fsmap learned FSMap version 32 2024-09-10T10:00:11.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.095+0000 7fc1bee03740 1 -- 172.21.15.77:0/3879109091 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560df387a770 con 0x560df3883590 2024-09-10T10:00:11.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.096+0000 7fc19b400640 1 -- 172.21.15.77:0/3879109091 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1a8005500 con 0x560df3883590 2024-09-10T10:00:11.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.096+0000 7fc19b400640 1 -- 172.21.15.77:0/3879109091 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fc1a00da2b0 con 0x560df3883590 2024-09-10T10:00:11.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.096+0000 7fc1bee03740 10 client.26491 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T10:00:11.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.096+0000 7fc1bee03740 10 client.26491 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T10:00:11.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.096+0000 7fc1bee03740 20 client.26491 populate_metadata read hostname 'smithi077' 2024-09-10T10:00:11.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.096+0000 7fc1bee03740 1 --2- 172.21.15.77:0/3879109091 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x560df390dd40 0x560df3910160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:11.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.096+0000 7fc1bee03740 4 client.26491 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:00:11.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.096+0000 7fc1bee03740 1 -- 172.21.15.77:0/3879109091 --> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560df37f0e60 con 0x560df390dd40 2024-09-10T10:00:11.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.096+0000 7fc1b9a00640 1 --2- 172.21.15.77:0/3879109091 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x560df390dd40 0x560df3910160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:11.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.097+0000 7fc1b9a00640 1 --2- 172.21.15.77:0/3879109091 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x560df390dd40 0x560df3910160 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a3afb6bc3b4fbb6f server_cookie=2f80639cee16d668 in_seq=0 out_seq=0 2024-09-10T10:00:11.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.097+0000 7fc19b400640 10 client.26491 ms_handle_connect on v2:172.21.15.134:6838/1965117680 2024-09-10T10:00:11.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.097+0000 7fc19b400640 1 -- 172.21.15.77:0/3879109091 <== mds.2 v2:172.21.15.134:6838/1965117680 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560df37f0e60 con 0x560df390dd40 2024-09-10T10:00:11.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.097+0000 7fc19b400640 10 client.26491 handle_command_reply: tid=0 2024-09-10T10:00:11.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.097+0000 7fc1bee03740 1 client.26491 shutdown 2024-09-10T10:00:11.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.097+0000 7fc1bee03740 1 -- 172.21.15.77:0/3879109091 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560df3883590 msgr2=0x560df389abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:11.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.097+0000 7fc1bee03740 1 --2- 172.21.15.77:0/3879109091 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560df3883590 0x560df389abb0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc1a00b8060 tx=0x7fc1a00b84a0 comp rx=0 tx=0).stop 2024-09-10T10:00:11.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.097+0000 7fc1bee03740 1 -- 172.21.15.77:0/3879109091 shutdown_connections 2024-09-10T10:00:11.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.097+0000 7fc1bee03740 1 --2- 172.21.15.77:0/3879109091 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x560df390dd40 0x560df3910160 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:11.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.097+0000 7fc1bee03740 1 --2- 172.21.15.77:0/3879109091 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560df388de50 0x560df3906130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:11.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.097+0000 7fc1bee03740 1 --2- 172.21.15.77:0/3879109091 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560df3883590 0x560df389abb0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:11.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.097+0000 7fc1bee03740 1 --2- 172.21.15.77:0/3879109091 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560df3882c70 0x560df389a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:11.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.097+0000 7fc19b400640 0 client.26491 ms_handle_reset on v2:172.21.15.134:6838/1965117680 2024-09-10T10:00:11.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.097+0000 7fc1bee03740 1 -- 172.21.15.77:0/3879109091 >> 172.21.15.77:0/3879109091 conn(0x560df387f200 msgr2=0x560df387db60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:11.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.098+0000 7fc1bee03740 1 -- 172.21.15.77:0/3879109091 shutdown_connections 2024-09-10T10:00:11.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.098+0000 7fc1bee03740 1 -- 172.21.15.77:0/3879109091 wait complete. 2024-09-10T10:00:11.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.098+0000 7fc1bee03740 20 client.26491 trim_cache size 0 max 16384 2024-09-10T10:00:11.112 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-10T10:00:11.113 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.098+0000 7fc1bc400640 1 -- 172.21.15.77:0/3885309671 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fc19c0a2b80 msgr2=0x7fc19c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:11.113 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.098+0000 7fc1bc400640 1 --2- 172.21.15.77:0/3885309671 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fc19c0a2b80 0x7fc19c0a5040 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x560df3880060 tx=0x7fc1a8005f90 comp rx=0 tx=0).stop 2024-09-10T10:00:11.113 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.098+0000 7fc1bc400640 1 -- 172.21.15.77:0/3885309671 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc1b4102790 msgr2=0x7fc1b41989c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:11.113 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.098+0000 7fc1bc400640 1 --2- 172.21.15.77:0/3885309671 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc1b4102790 0x7fc1b41989c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc1a002a910 tx=0x7fc1a0003b20 comp rx=0 tx=0).stop 2024-09-10T10:00:11.113 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.099+0000 7fc1bc400640 1 -- 172.21.15.77:0/3885309671 shutdown_connections 2024-09-10T10:00:11.113 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.099+0000 7fc1bc400640 1 --2- 172.21.15.77:0/3885309671 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fc19c0a2b80 0x7fc19c0a5040 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:11.113 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.099+0000 7fc1bc400640 1 --2- 172.21.15.77:0/3885309671 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1b4106f10 0x7fc1b419cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:11.113 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.099+0000 7fc1bc400640 1 --2- 172.21.15.77:0/3885309671 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc1b4102790 0x7fc1b41989c0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:11.113 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.099+0000 7fc1bc400640 1 --2- 172.21.15.77:0/3885309671 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1b4101de0 0x7fc1b4198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:11.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.099+0000 7fc1bc400640 1 -- 172.21.15.77:0/3885309671 >> 172.21.15.77:0/3885309671 conn(0x7fc1b40fd8c0 msgr2=0x7fc1b40feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:11.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.099+0000 7fc1bc400640 1 -- 172.21.15.77:0/3885309671 shutdown_connections 2024-09-10T10:00:11.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:11.099+0000 7fc1bc400640 1 -- 172.21.15.77:0/3885309671 wait complete. 2024-09-10T10:00:11.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:00:10 smithi077 ceph-mon[30319]: pgmap v541: 129 pgs: 129 active+clean; 97 MiB data, 939 MiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 362 KiB/s wr, 73 op/s 2024-09-10T10:00:11.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:00:10 smithi134 ceph-mon[35674]: pgmap v541: 129 pgs: 129 active+clean; 97 MiB data, 939 MiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 362 KiB/s wr, 73 op/s 2024-09-10T10:00:11.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:00:10 smithi120 ceph-mon[34770]: pgmap v541: 129 pgs: 129 active+clean; 97 MiB data, 939 MiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 362 KiB/s wr, 73 op/s 2024-09-10T10:00:11.939 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T10:00:12.062 INFO:tasks.workunit.client.0.smithi077.stdout:50000 copy from 0x898b8 to 0x92015, (0x875d bytes) at 0xb142c 2024-09-10T10:00:12.062 INFO:tasks.workunit.client.0.smithi077.stdout:All 50000 operations completed A-OK! 2024-09-10T10:00:12.065 INFO:tasks.workunit.client.0.smithi077.stderr:+ ./fsx -z 10MB -N 50000 -p 10000 -l 10485760 2024-09-10T10:00:12.071 INFO:tasks.workunit.client.0.smithi077.stdout:Seed set to 1 2024-09-10T10:00:12.211 INFO:tasks.workunit.client.0.smithi077.stderr:main: filesystem does not support fallocate mode 0, disabling! 2024-09-10T10:00:12.211 INFO:tasks.workunit.client.0.smithi077.stderr:main: filesystem does not support fallocate mode FALLOC_FL_KEEP_SIZE, disabling! 2024-09-10T10:00:12.213 INFO:tasks.workunit.client.0.smithi077.stderr:main: filesystem does not support fallocate mode FALLOC_FL_COLLAPSE_RANGE, disabling! 2024-09-10T10:00:12.213 INFO:tasks.workunit.client.0.smithi077.stderr:main: filesystem does not support fallocate mode FALLOC_FL_INSERT_RANGE, disabling! 2024-09-10T10:00:12.213 INFO:tasks.workunit.client.0.smithi077.stderr:main: filesystem does not support clone range, disabling! 2024-09-10T10:00:12.215 INFO:tasks.workunit.client.0.smithi077.stderr:main: filesystem does not support dedupe range, disabling! 2024-09-10T10:00:12.217 INFO:tasks.workunit.client.0.smithi077.stderr:main: filesystem does not support exchange range, disabling! 2024-09-10T10:00:12.217 INFO:tasks.workunit.client.0.smithi077.stdout:skipping zero length punch hole 2024-09-10T10:00:12.217 INFO:tasks.workunit.client.0.smithi077.stdout:skipping zero length punch hole 2024-09-10T10:00:12.217 INFO:tasks.workunit.client.0.smithi077.stdout:skipping zero length punch hole 2024-09-10T10:00:12.217 INFO:tasks.workunit.client.0.smithi077.stdout:skipping zero size read 2024-09-10T10:00:12.217 INFO:tasks.workunit.client.0.smithi077.stdout:skipping zero length punch hole 2024-09-10T10:00:12.217 INFO:tasks.workunit.client.0.smithi077.stdout:skipping zero size read 2024-09-10T10:00:12.218 INFO:tasks.workunit.client.0.smithi077.stdout:skipping zero length punch hole 2024-09-10T10:00:12.218 INFO:tasks.workunit.client.0.smithi077.stdout:truncating to largest ever: 0x4310e9 2024-09-10T10:00:12.224 INFO:tasks.workunit.client.0.smithi077.stdout:copying to largest ever: 0x954a45 2024-09-10T10:00:12.293 INFO:tasks.workunit.client.0.smithi077.stdout:truncating to largest ever: 0x9bc582 2024-09-10T10:00:12.330 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:00:12.338 INFO:tasks.workunit.client.0.smithi077.stdout:truncating to largest ever: 0x9eecbc 2024-09-10T10:00:12.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.981+0000 7fa596a00640 1 -- 172.21.15.77:0/1246102479 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa590104800 msgr2=0x7fa590108960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:12.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.981+0000 7fa596a00640 1 --2- 172.21.15.77:0/1246102479 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa590104800 0x7fa590108960 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fa57c009900 tx=0x7fa57c02a3f0 comp rx=0 tx=0).stop 2024-09-10T10:00:12.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.982+0000 7fa596a00640 1 -- 172.21.15.77:0/1246102479 shutdown_connections 2024-09-10T10:00:12.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.982+0000 7fa596a00640 1 --2- 172.21.15.77:0/1246102479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa590109090 0x7fa590110c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:12.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.982+0000 7fa596a00640 1 --2- 172.21.15.77:0/1246102479 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa590104800 0x7fa590108960 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:12.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.982+0000 7fa596a00640 1 --2- 172.21.15.77:0/1246102479 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa590103e50 0x7fa590104230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:12.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.982+0000 7fa596a00640 1 -- 172.21.15.77:0/1246102479 >> 172.21.15.77:0/1246102479 conn(0x7fa5900ff8b0 msgr2=0x7fa590101cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:12.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.982+0000 7fa596a00640 1 -- 172.21.15.77:0/1246102479 shutdown_connections 2024-09-10T10:00:12.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.982+0000 7fa596a00640 1 -- 172.21.15.77:0/1246102479 wait complete. 2024-09-10T10:00:12.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.983+0000 7fa596a00640 1 Processor -- start 2024-09-10T10:00:12.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.984+0000 7fa596a00640 1 -- start start 2024-09-10T10:00:12.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.984+0000 7fa596a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa590103e50 0x7fa590110ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:12.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.984+0000 7fa596a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa590104800 0x7fa59010db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:12.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.984+0000 7fa596a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa590109090 0x7fa59010e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:12.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.984+0000 7fa596a00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa590114090 con 0x7fa590109090 2024-09-10T10:00:12.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.984+0000 7fa596a00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa590113f10 con 0x7fa590103e50 2024-09-10T10:00:12.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.984+0000 7fa596a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa590114210 con 0x7fa590104800 2024-09-10T10:00:12.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.985+0000 7fa596000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa590109090 0x7fa59010e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:12.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.985+0000 7fa595600640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa590103e50 0x7fa590110ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:12.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.985+0000 7fa596000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa590109090 0x7fa59010e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57250/0 (socket says 172.21.15.77:57250) 2024-09-10T10:00:12.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.985+0000 7fa594c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa590104800 0x7fa59010db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:12.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.985+0000 7fa596000640 1 -- 172.21.15.77:0/1859739437 learned_addr learned my addr 172.21.15.77:0/1859739437 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:00:12.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.985+0000 7fa595600640 1 -- 172.21.15.77:0/1859739437 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa590104800 msgr2=0x7fa59010db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:12.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.985+0000 7fa595600640 1 --2- 172.21.15.77:0/1859739437 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa590104800 0x7fa59010db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:12.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.985+0000 7fa595600640 1 -- 172.21.15.77:0/1859739437 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa590109090 msgr2=0x7fa59010e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:12.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.985+0000 7fa595600640 1 --2- 172.21.15.77:0/1859739437 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa590109090 0x7fa59010e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:12.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.985+0000 7fa595600640 1 -- 172.21.15.77:0/1859739437 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa59010e9b0 con 0x7fa590103e50 2024-09-10T10:00:12.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.985+0000 7fa596000640 1 --2- 172.21.15.77:0/1859739437 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa590109090 0x7fa59010e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:00:12.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.986+0000 7fa595600640 1 --2- 172.21.15.77:0/1859739437 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa590103e50 0x7fa590110ae0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa58400e9f0 tx=0x7fa58400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:12.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.986+0000 7fa58e000640 1 -- 172.21.15.77:0/1859739437 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa58400ccc0 con 0x7fa590103e50 2024-09-10T10:00:12.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.986+0000 7fa58e000640 1 -- 172.21.15.77:0/1859739437 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa584010910 con 0x7fa590103e50 2024-09-10T10:00:12.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.986+0000 7fa58e000640 1 -- 172.21.15.77:0/1859739437 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa584018960 con 0x7fa590103e50 2024-09-10T10:00:12.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.987+0000 7fa596a00640 1 -- 172.21.15.77:0/1859739437 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5901a26e0 con 0x7fa590103e50 2024-09-10T10:00:12.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.987+0000 7fa596a00640 1 -- 172.21.15.77:0/1859739437 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5901a2c40 con 0x7fa590103e50 2024-09-10T10:00:12.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.990+0000 7fa59aa80740 1 Processor -- start 2024-09-10T10:00:12.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.991+0000 7fa59aa80740 1 -- start start 2024-09-10T10:00:12.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.991+0000 7fa59aa80740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b81362e40 0x559b81363220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:12.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.991+0000 7fa59aa80740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b81363760 0x559b8136dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:12.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.991+0000 7fa59aa80740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b8136e020 0x559b81370410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:12.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.991+0000 7fa58e000640 1 -- 172.21.15.77:0/1859739437 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa584010ac0 con 0x7fa590103e50 2024-09-10T10:00:12.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.991+0000 7fa59aa80740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x559b812c93e0 con 0x559b81362e40 2024-09-10T10:00:12.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.991+0000 7fa59aa80740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x559b81270d80 con 0x559b8136e020 2024-09-10T10:00:12.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.991+0000 7fa59aa80740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559b812c53a0 con 0x559b81363760 2024-09-10T10:00:12.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.991+0000 7fa594c00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b8136e020 0x559b81370410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:12.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.992+0000 7fa596000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b81362e40 0x559b81363220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:12.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.992+0000 7fa595600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b81363760 0x559b8136dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:12.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.992+0000 7fa596000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b81362e40 0x559b81363220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57264/0 (socket says 172.21.15.77:57264) 2024-09-10T10:00:12.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.992+0000 7fa595600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b81363760 0x559b8136dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:33326/0 (socket says 172.21.15.77:33326) 2024-09-10T10:00:12.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.992+0000 7fa595600640 1 -- 172.21.15.77:0/2467234758 learned_addr learned my addr 172.21.15.77:0/2467234758 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:00:12.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.992+0000 7fa595600640 1 -- 172.21.15.77:0/2467234758 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b8136e020 msgr2=0x559b81370410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:12.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.992+0000 7fa595600640 1 --2- 172.21.15.77:0/2467234758 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b8136e020 0x559b81370410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:12.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.992+0000 7fa595600640 1 -- 172.21.15.77:0/2467234758 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b81362e40 msgr2=0x559b81363220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:12.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.992+0000 7fa595600640 1 --2- 172.21.15.77:0/2467234758 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b81362e40 0x559b81363220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:12.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.992+0000 7fa595600640 1 -- 172.21.15.77:0/2467234758 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b8128fae0 con 0x559b81363760 2024-09-10T10:00:12.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.992+0000 7fa594c00640 1 --2- 172.21.15.77:0/2467234758 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b8136e020 0x559b81370410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:00:12.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.993+0000 7fa595600640 1 --2- 172.21.15.77:0/2467234758 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b81363760 0x559b8136dae0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa5840d5c10 tx=0x7fa5840d7690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=80933e88839449a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:12.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.993+0000 7fa596000640 1 --2- 172.21.15.77:0/2467234758 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b81362e40 0x559b81363220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:00:12.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.993+0000 7fa58e000640 1 --2- 172.21.15.77:0/1859739437 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fa5780a2c00 0x7fa5780a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:12.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.994+0000 7fa582600640 1 -- 172.21.15.77:0/2467234758 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5840de070 con 0x559b81363760 2024-09-10T10:00:12.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.994+0000 7fa58e000640 1 -- 172.21.15.77:0/1859739437 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fa5840cf740 con 0x7fa590103e50 2024-09-10T10:00:12.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.994+0000 7fa582600640 1 -- 172.21.15.77:0/2467234758 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa584052b90 con 0x559b81363760 2024-09-10T10:00:12.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.994+0000 7fa582600640 1 -- 172.21.15.77:0/2467234758 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa58409b7a0 con 0x559b81363760 2024-09-10T10:00:12.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.994+0000 7fa596000640 1 --2- 172.21.15.77:0/1859739437 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fa5780a2c00 0x7fa5780a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:12.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.995+0000 7fa59aa80740 1 -- 172.21.15.77:0/2467234758 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b81363760 msgr2=0x559b8136dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:12.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.995+0000 7fa59aa80740 1 --2- 172.21.15.77:0/2467234758 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b81363760 0x559b8136dae0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa5840d5c10 tx=0x7fa5840d7690 comp rx=0 tx=0).stop 2024-09-10T10:00:12.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.995+0000 7fa59aa80740 1 -- 172.21.15.77:0/2467234758 shutdown_connections 2024-09-10T10:00:12.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.995+0000 7fa59aa80740 1 --2- 172.21.15.77:0/2467234758 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b8136e020 0x559b81370410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:12.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.995+0000 7fa59aa80740 1 --2- 172.21.15.77:0/2467234758 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b81363760 0x559b8136dae0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:12.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.995+0000 7fa59aa80740 1 --2- 172.21.15.77:0/2467234758 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b81362e40 0x559b81363220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:12.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.995+0000 7fa59aa80740 1 -- 172.21.15.77:0/2467234758 >> 172.21.15.77:0/2467234758 conn(0x559b8135f3d0 msgr2=0x559b8135f7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:12.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.996+0000 7fa59aa80740 1 -- 172.21.15.77:0/2467234758 shutdown_connections 2024-09-10T10:00:12.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.996+0000 7fa596000640 1 --2- 172.21.15.77:0/1859739437 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fa5780a2c00 0x7fa5780a50c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fa5880049d0 tx=0x7fa588009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:12.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.996+0000 7fa59aa80740 1 -- 172.21.15.77:0/2467234758 wait complete. 2024-09-10T10:00:12.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.996+0000 7fa59aa80740 1 -- start start 2024-09-10T10:00:12.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.997+0000 7fa59aa80740 1 Processor -- start 2024-09-10T10:00:12.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.997+0000 7fa59aa80740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b81362e40 0x559b8136af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:12.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.997+0000 7fa59aa80740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b81363760 0x559b8136b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:12.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.997+0000 7fa59aa80740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b8136e020 0x559b8136b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:12.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.997+0000 7fa59aa80740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x559b81370bf0 con 0x559b8136e020 2024-09-10T10:00:12.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.997+0000 7fa59aa80740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x559b812840f0 con 0x559b81363760 2024-09-10T10:00:12.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.997+0000 7fa59aa80740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559b81370d70 con 0x559b81362e40 2024-09-10T10:00:12.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.997+0000 7fa59aa80740 10 client.0 _ll_register_callbacks cb 0x559b8126bbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:00:12.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.997+0000 7fa595600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b81362e40 0x559b8136af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:12.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.997+0000 7fa595600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b81362e40 0x559b8136af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:33338/0 (socket says 172.21.15.77:33338) 2024-09-10T10:00:12.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.998+0000 7fa595600640 1 -- 172.21.15.77:0/4113064704 learned_addr learned my addr 172.21.15.77:0/4113064704 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:00:12.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.998+0000 7fa594c00640 1 --2- 172.21.15.77:0/4113064704 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b81363760 0x559b8136b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:13.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.998+0000 7fa595600640 1 -- 172.21.15.77:0/4113064704 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b81363760 msgr2=0x559b8136b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:13.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.998+0000 7fa595600640 1 --2- 172.21.15.77:0/4113064704 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b81363760 0x559b8136b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:13.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.998+0000 7fa595600640 1 -- 172.21.15.77:0/4113064704 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b8136e020 msgr2=0x559b8136b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:13.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.998+0000 7fa595600640 1 --2- 172.21.15.77:0/4113064704 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b8136e020 0x559b8136b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:13.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.998+0000 7fa595600640 1 -- 172.21.15.77:0/4113064704 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b8103a3c0 con 0x559b81362e40 2024-09-10T10:00:13.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.998+0000 7fa595600640 1 --2- 172.21.15.77:0/4113064704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b81362e40 0x559b8136af30 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa5840ccea0 tx=0x7fa5840cfa40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=129d698cee83fe81 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:13.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.998+0000 7fa576a00640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-10T10:00:13.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.998+0000 7fa576a00640 1 -- 172.21.15.77:0/4113064704 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5840de070 con 0x559b81362e40 2024-09-10T10:00:13.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.999+0000 7fa576a00640 1 -- 172.21.15.77:0/4113064704 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fa5840cf410 con 0x559b81362e40 2024-09-10T10:00:13.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.999+0000 7fa59aa80740 1 -- 172.21.15.77:0/4113064704 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa584052b90 con 0x559b81362e40 2024-09-10T10:00:13.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.999+0000 7fa576a00640 1 -- 172.21.15.77:0/4113064704 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa58409b5c0 con 0x559b81362e40 2024-09-10T10:00:13.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.999+0000 7fa59aa80740 10 client.26503 fetch_fsmap learned FSMap version 32 2024-09-10T10:00:13.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.999+0000 7fa59aa80740 1 -- 172.21.15.77:0/4113064704 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559b81370bf0 con 0x559b81362e40 2024-09-10T10:00:13.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:12.999+0000 7fa576a00640 1 -- 172.21.15.77:0/4113064704 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5840d5850 con 0x559b81362e40 2024-09-10T10:00:13.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.000+0000 7fa576a00640 1 -- 172.21.15.77:0/4113064704 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fa58405b120 con 0x559b81362e40 2024-09-10T10:00:13.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.000+0000 7fa59aa80740 10 client.26503 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T10:00:13.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.000+0000 7fa59aa80740 10 client.26503 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-10T10:00:13.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.000+0000 7fa59aa80740 20 client.26503 populate_metadata read hostname 'smithi077' 2024-09-10T10:00:13.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.000+0000 7fa59aa80740 1 --2- 172.21.15.77:0/4113064704 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x559b8137dd80 0x559b81380170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:13.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.000+0000 7fa59aa80740 4 client.26503 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:00:13.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.000+0000 7fa59aa80740 1 -- 172.21.15.77:0/4113064704 --> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fa584052b90 con 0x559b8137dd80 2024-09-10T10:00:13.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.000+0000 7fa594c00640 1 --2- 172.21.15.77:0/4113064704 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x559b8137dd80 0x559b81380170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:13.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.001+0000 7fa594c00640 1 --2- 172.21.15.77:0/4113064704 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x559b8137dd80 0x559b81380170 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f896f77d1c4c5242 server_cookie=941640e4b1a654a4 in_seq=0 out_seq=0 2024-09-10T10:00:13.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.001+0000 7fa576a00640 10 client.26503 ms_handle_connect on v2:172.21.15.120:6836/3033280709 2024-09-10T10:00:13.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.002+0000 7fa576a00640 1 -- 172.21.15.77:0/4113064704 <== mds.1 v2:172.21.15.120:6836/3033280709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fa584052b90 con 0x559b8137dd80 2024-09-10T10:00:13.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.002+0000 7fa576a00640 10 client.26503 handle_command_reply: tid=0 2024-09-10T10:00:13.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.002+0000 7fa59aa80740 1 client.26503 shutdown 2024-09-10T10:00:13.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.002+0000 7fa59aa80740 1 -- 172.21.15.77:0/4113064704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b81362e40 msgr2=0x559b8136af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:13.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.002+0000 7fa59aa80740 1 --2- 172.21.15.77:0/4113064704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b81362e40 0x559b8136af30 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa5840ccea0 tx=0x7fa5840cfa40 comp rx=0 tx=0).stop 2024-09-10T10:00:13.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.002+0000 7fa59aa80740 1 -- 172.21.15.77:0/4113064704 shutdown_connections 2024-09-10T10:00:13.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.002+0000 7fa59aa80740 1 --2- 172.21.15.77:0/4113064704 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x559b8137dd80 0x559b81380170 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:13.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.002+0000 7fa59aa80740 1 --2- 172.21.15.77:0/4113064704 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b8136e020 0x559b8136b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:13.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.002+0000 7fa59aa80740 1 --2- 172.21.15.77:0/4113064704 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b81363760 0x559b8136b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:13.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.002+0000 7fa59aa80740 1 --2- 172.21.15.77:0/4113064704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b81362e40 0x559b8136af30 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:13.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.002+0000 7fa59aa80740 1 -- 172.21.15.77:0/4113064704 >> 172.21.15.77:0/4113064704 conn(0x559b8135f3d0 msgr2=0x559b81365100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:13.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.002+0000 7fa576a00640 0 client.26503 ms_handle_reset on v2:172.21.15.120:6836/3033280709 2024-09-10T10:00:13.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.002+0000 7fa59aa80740 1 -- 172.21.15.77:0/4113064704 shutdown_connections 2024-09-10T10:00:13.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.002+0000 7fa59aa80740 1 -- 172.21.15.77:0/4113064704 wait complete. 2024-09-10T10:00:13.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.003+0000 7fa59aa80740 20 client.26503 trim_cache size 0 max 16384 2024-09-10T10:00:13.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.017+0000 7fa59aa80740 1 Processor -- start 2024-09-10T10:00:13.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.017+0000 7fa59aa80740 1 -- start start 2024-09-10T10:00:13.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.018+0000 7fa59aa80740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b81362e40 0x559b8136b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:13.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.018+0000 7fa59aa80740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b81363760 0x559b81385740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:13.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.018+0000 7fa59aa80740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b8136e020 0x559b81385c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:13.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.018+0000 7fa59aa80740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa5901acad0 con 0x559b8136e020 2024-09-10T10:00:13.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.018+0000 7fa59aa80740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa5901ae6d0 con 0x559b81363760 2024-09-10T10:00:13.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.018+0000 7fa59aa80740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa5901ad1a0 con 0x559b81362e40 2024-09-10T10:00:13.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.018+0000 7fa595600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b81362e40 0x559b8136b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:13.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.018+0000 7fa594c00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b81363760 0x559b81385740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:13.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.018+0000 7fa595600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b81362e40 0x559b8136b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:33344/0 (socket says 172.21.15.77:33344) 2024-09-10T10:00:13.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.018+0000 7fa595600640 1 -- 172.21.15.77:0/1734469081 learned_addr learned my addr 172.21.15.77:0/1734469081 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:00:13.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.018+0000 7fa596000640 1 --2- 172.21.15.77:0/1734469081 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b8136e020 0x559b81385c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:13.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.019+0000 7fa595600640 1 -- 172.21.15.77:0/1734469081 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b81363760 msgr2=0x559b81385740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:13.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.019+0000 7fa595600640 1 --2- 172.21.15.77:0/1734469081 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b81363760 0x559b81385740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:13.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.019+0000 7fa595600640 1 -- 172.21.15.77:0/1734469081 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b8136e020 msgr2=0x559b81385c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:13.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.019+0000 7fa595600640 1 --2- 172.21.15.77:0/1734469081 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b8136e020 0x559b81385c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:13.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.019+0000 7fa595600640 1 -- 172.21.15.77:0/1734469081 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b8135ff80 con 0x559b81362e40 2024-09-10T10:00:13.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.019+0000 7fa596000640 1 --2- 172.21.15.77:0/1734469081 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b8136e020 0x559b81385c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:00:13.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.019+0000 7fa594c00640 1 --2- 172.21.15.77:0/1734469081 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b81363760 0x559b81385740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:00:13.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.019+0000 7fa595600640 1 --2- 172.21.15.77:0/1734469081 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b81362e40 0x559b8136b800 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa5840cd1c0 tx=0x7fa5840cfab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e4163d294fce937 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:13.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.019+0000 7fa581c00640 1 -- 172.21.15.77:0/1734469081 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5840de070 con 0x559b81362e40 2024-09-10T10:00:13.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.019+0000 7fa581c00640 1 -- 172.21.15.77:0/1734469081 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5840d5850 con 0x559b81362e40 2024-09-10T10:00:13.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.019+0000 7fa581c00640 1 -- 172.21.15.77:0/1734469081 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5840a4390 con 0x559b81362e40 2024-09-10T10:00:13.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.020+0000 7fa59aa80740 1 -- 172.21.15.77:0/1734469081 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b81362e40 msgr2=0x559b8136b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:13.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.020+0000 7fa59aa80740 1 --2- 172.21.15.77:0/1734469081 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b81362e40 0x559b8136b800 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa5840cd1c0 tx=0x7fa5840cfab0 comp rx=0 tx=0).stop 2024-09-10T10:00:13.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.021+0000 7fa59aa80740 1 -- 172.21.15.77:0/1734469081 shutdown_connections 2024-09-10T10:00:13.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.021+0000 7fa59aa80740 1 --2- 172.21.15.77:0/1734469081 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b8136e020 0x559b81385c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:13.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.021+0000 7fa59aa80740 1 --2- 172.21.15.77:0/1734469081 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b81363760 0x559b81385740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:13.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.021+0000 7fa59aa80740 1 --2- 172.21.15.77:0/1734469081 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b81362e40 0x559b8136b800 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:13.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.021+0000 7fa59aa80740 1 -- 172.21.15.77:0/1734469081 >> 172.21.15.77:0/1734469081 conn(0x559b8135f3d0 msgr2=0x559b813650b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:13.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.021+0000 7fa59aa80740 1 -- 172.21.15.77:0/1734469081 shutdown_connections 2024-09-10T10:00:13.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.021+0000 7fa59aa80740 1 -- 172.21.15.77:0/1734469081 wait complete. 2024-09-10T10:00:13.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.022+0000 7fa59aa80740 1 -- start start 2024-09-10T10:00:13.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.022+0000 7fa59aa80740 1 Processor -- start 2024-09-10T10:00:13.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.023+0000 7fa59aa80740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b81362e40 0x559b81376460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:13.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.023+0000 7fa59aa80740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b81363760 0x559b81371790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:13.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.023+0000 7fa59aa80740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b8136e020 0x559b81371cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:13.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.023+0000 7fa59aa80740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa5901adf40 con 0x559b81362e40 2024-09-10T10:00:13.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.023+0000 7fa59aa80740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa5901ad870 con 0x559b81363760 2024-09-10T10:00:13.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.023+0000 7fa59aa80740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa5901aaab0 con 0x559b8136e020 2024-09-10T10:00:13.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.023+0000 7fa59aa80740 10 client.0 _ll_register_callbacks cb 0x7fa5880093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:00:13.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.023+0000 7fa596000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b8136e020 0x559b81371cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:13.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.023+0000 7fa596000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b8136e020 0x559b81371cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:33348/0 (socket says 172.21.15.77:33348) 2024-09-10T10:00:13.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.023+0000 7fa596000640 1 -- 172.21.15.77:0/590810432 learned_addr learned my addr 172.21.15.77:0/590810432 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:00:13.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.023+0000 7fa594c00640 1 --2- 172.21.15.77:0/590810432 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b81363760 0x559b81371790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:13.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.024+0000 7fa595600640 1 --2- 172.21.15.77:0/590810432 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b81362e40 0x559b81376460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:13.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.024+0000 7fa596000640 1 -- 172.21.15.77:0/590810432 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b81363760 msgr2=0x559b81371790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:13.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.024+0000 7fa596000640 1 --2- 172.21.15.77:0/590810432 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b81363760 0x559b81371790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:13.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.024+0000 7fa596000640 1 -- 172.21.15.77:0/590810432 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b81362e40 msgr2=0x559b81376460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:13.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.024+0000 7fa596000640 1 --2- 172.21.15.77:0/590810432 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b81362e40 0x559b81376460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:13.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.024+0000 7fa596000640 1 -- 172.21.15.77:0/590810432 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b81372210 con 0x559b8136e020 2024-09-10T10:00:13.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.024+0000 7fa594c00640 1 --2- 172.21.15.77:0/590810432 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b81363760 0x559b81371790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:00:13.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.024+0000 7fa595600640 1 --2- 172.21.15.77:0/590810432 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b81362e40 0x559b81376460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:00:13.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.024+0000 7fa596000640 1 --2- 172.21.15.77:0/590810432 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b8136e020 0x559b81371cd0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa58801f6a0 tx=0x7fa588004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89ea6ecbd4788255 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:13.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.024+0000 7fa576a00640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-10T10:00:13.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.025+0000 7fa576a00640 1 -- 172.21.15.77:0/590810432 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa58802c060 con 0x559b8136e020 2024-09-10T10:00:13.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.025+0000 7fa576a00640 1 -- 172.21.15.77:0/590810432 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fa58801d840 con 0x559b8136e020 2024-09-10T10:00:13.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.025+0000 7fa59aa80740 1 -- 172.21.15.77:0/590810432 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5840d5850 con 0x559b8136e020 2024-09-10T10:00:13.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.025+0000 7fa576a00640 1 -- 172.21.15.77:0/590810432 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa58801da40 con 0x559b8136e020 2024-09-10T10:00:13.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.025+0000 7fa59aa80740 10 client.26509 fetch_fsmap learned FSMap version 32 2024-09-10T10:00:13.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.025+0000 7fa59aa80740 1 -- 172.21.15.77:0/590810432 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559b813e8480 con 0x559b8136e020 2024-09-10T10:00:13.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.026+0000 7fa576a00640 1 -- 172.21.15.77:0/590810432 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5840d5850 con 0x559b8136e020 2024-09-10T10:00:13.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.026+0000 7fa576a00640 1 -- 172.21.15.77:0/590810432 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fa588026030 con 0x559b8136e020 2024-09-10T10:00:13.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.026+0000 7fa59aa80740 10 client.26509 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T10:00:13.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.026+0000 7fa59aa80740 10 client.26509 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-10T10:00:13.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.026+0000 7fa59aa80740 20 client.26509 populate_metadata read hostname 'smithi077' 2024-09-10T10:00:13.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.026+0000 7fa59aa80740 1 --2- 172.21.15.77:0/590810432 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x559b813e8c00 0x559b813eb020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:13.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.026+0000 7fa59aa80740 4 client.26509 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:00:13.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.026+0000 7fa59aa80740 1 -- 172.21.15.77:0/590810432 --> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559b812d6950 con 0x559b813e8c00 2024-09-10T10:00:13.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.027+0000 7fa595600640 1 --2- 172.21.15.77:0/590810432 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x559b813e8c00 0x559b813eb020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:13.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.027+0000 7fa595600640 1 --2- 172.21.15.77:0/590810432 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x559b813e8c00 0x559b813eb020 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c893b28d8f1635d server_cookie=4091eb8b55b78002 in_seq=0 out_seq=0 2024-09-10T10:00:13.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.027+0000 7fa576a00640 10 client.26509 ms_handle_connect on v2:172.21.15.120:6836/3033280709 2024-09-10T10:00:13.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.027+0000 7fa576a00640 1 -- 172.21.15.77:0/590810432 <== mds.1 v2:172.21.15.120:6836/3033280709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559b812d6950 con 0x559b813e8c00 2024-09-10T10:00:13.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.027+0000 7fa576a00640 10 client.26509 handle_command_reply: tid=0 2024-09-10T10:00:13.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.028+0000 7fa59aa80740 1 client.26509 shutdown 2024-09-10T10:00:13.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.028+0000 7fa59aa80740 1 -- 172.21.15.77:0/590810432 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b8136e020 msgr2=0x559b81371cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:13.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.028+0000 7fa59aa80740 1 --2- 172.21.15.77:0/590810432 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b8136e020 0x559b81371cd0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa58801f6a0 tx=0x7fa588004ab0 comp rx=0 tx=0).stop 2024-09-10T10:00:13.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.028+0000 7fa59aa80740 1 -- 172.21.15.77:0/590810432 shutdown_connections 2024-09-10T10:00:13.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.028+0000 7fa59aa80740 1 --2- 172.21.15.77:0/590810432 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x559b813e8c00 0x559b813eb020 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:13.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.028+0000 7fa576a00640 0 client.26509 ms_handle_reset on v2:172.21.15.120:6836/3033280709 2024-09-10T10:00:13.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.028+0000 7fa59aa80740 1 --2- 172.21.15.77:0/590810432 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b8136e020 0x559b81371cd0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:13.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.028+0000 7fa59aa80740 1 --2- 172.21.15.77:0/590810432 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b81363760 0x559b81371790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:13.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.028+0000 7fa59aa80740 1 --2- 172.21.15.77:0/590810432 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b81362e40 0x559b81376460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:13.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.028+0000 7fa59aa80740 1 -- 172.21.15.77:0/590810432 >> 172.21.15.77:0/590810432 conn(0x559b8135f3d0 msgr2=0x559b8136b720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:13.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.029+0000 7fa59aa80740 1 -- 172.21.15.77:0/590810432 shutdown_connections 2024-09-10T10:00:13.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.029+0000 7fa59aa80740 1 -- 172.21.15.77:0/590810432 wait complete. 2024-09-10T10:00:13.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.029+0000 7fa59aa80740 20 client.26509 trim_cache size 0 max 16384 2024-09-10T10:00:13.031 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-10T10:00:13.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.030+0000 7fa596a00640 1 -- 172.21.15.77:0/1859739437 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fa5780a2c00 msgr2=0x7fa5780a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:13.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.030+0000 7fa596a00640 1 --2- 172.21.15.77:0/1859739437 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fa5780a2c00 0x7fa5780a50c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fa5880049d0 tx=0x7fa588009500 comp rx=0 tx=0).stop 2024-09-10T10:00:13.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.030+0000 7fa596a00640 1 -- 172.21.15.77:0/1859739437 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa590103e50 msgr2=0x7fa590110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:13.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.030+0000 7fa596a00640 1 --2- 172.21.15.77:0/1859739437 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa590103e50 0x7fa590110ae0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa58400e9f0 tx=0x7fa58400eeb0 comp rx=0 tx=0).stop 2024-09-10T10:00:13.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.030+0000 7fa596a00640 1 -- 172.21.15.77:0/1859739437 shutdown_connections 2024-09-10T10:00:13.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.030+0000 7fa596a00640 1 --2- 172.21.15.77:0/1859739437 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fa5780a2c00 0x7fa5780a50c0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:13.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.030+0000 7fa596a00640 1 --2- 172.21.15.77:0/1859739437 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa590109090 0x7fa59010e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:13.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.030+0000 7fa596a00640 1 --2- 172.21.15.77:0/1859739437 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa590104800 0x7fa59010db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:13.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.030+0000 7fa596a00640 1 --2- 172.21.15.77:0/1859739437 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa590103e50 0x7fa590110ae0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:13.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.030+0000 7fa596a00640 1 -- 172.21.15.77:0/1859739437 >> 172.21.15.77:0/1859739437 conn(0x7fa5900ff8b0 msgr2=0x7fa590100bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:13.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.030+0000 7fa596a00640 1 -- 172.21.15.77:0/1859739437 shutdown_connections 2024-09-10T10:00:13.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:13.030+0000 7fa596a00640 1 -- 172.21.15.77:0/1859739437 wait complete. 2024-09-10T10:00:13.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:00:12 smithi077 ceph-mon[30319]: pgmap v542: 129 pgs: 129 active+clean; 97 MiB data, 966 MiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 353 KiB/s wr, 69 op/s 2024-09-10T10:00:13.225 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:00:12 smithi120 ceph-mon[34770]: pgmap v542: 129 pgs: 129 active+clean; 97 MiB data, 966 MiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 353 KiB/s wr, 69 op/s 2024-09-10T10:00:13.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:00:12 smithi134 ceph-mon[35674]: pgmap v542: 129 pgs: 129 active+clean; 97 MiB data, 966 MiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 353 KiB/s wr, 69 op/s 2024-09-10T10:00:13.810 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T10:00:14.267 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:00:14.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.878+0000 7f1de8e00640 1 -- 172.21.15.77:0/1297591232 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1de4104010 msgr2=0x7f1de41043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:14.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.878+0000 7f1de8e00640 1 --2- 172.21.15.77:0/1297591232 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1de4104010 0x7f1de41043f0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f1dd4009900 tx=0x7f1dd402a410 comp rx=0 tx=0).stop 2024-09-10T10:00:14.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.879+0000 7f1de8e00640 1 -- 172.21.15.77:0/1297591232 shutdown_connections 2024-09-10T10:00:14.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.879+0000 7f1de8e00640 1 --2- 172.21.15.77:0/1297591232 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1de4109140 0x7f1de4110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:14.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.879+0000 7f1de8e00640 1 --2- 172.21.15.77:0/1297591232 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1de41049c0 0x7f1de4108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:14.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.879+0000 7f1de8e00640 1 --2- 172.21.15.77:0/1297591232 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1de4104010 0x7f1de41043f0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:14.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.879+0000 7f1de8e00640 1 -- 172.21.15.77:0/1297591232 >> 172.21.15.77:0/1297591232 conn(0x7f1de40ffa50 msgr2=0x7f1de4101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:14.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.879+0000 7f1de8e00640 1 -- 172.21.15.77:0/1297591232 shutdown_connections 2024-09-10T10:00:14.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.879+0000 7f1de8e00640 1 -- 172.21.15.77:0/1297591232 wait complete. 2024-09-10T10:00:14.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.880+0000 7f1de8e00640 1 Processor -- start 2024-09-10T10:00:14.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.880+0000 7f1de8e00640 1 -- start start 2024-09-10T10:00:14.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.880+0000 7f1de8e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1de4104010 0x7f1de41a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:14.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.880+0000 7f1de8e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1de41049c0 0x7f1de41a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:14.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.880+0000 7f1de8e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1de4109140 0x7f1de410f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:14.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.880+0000 7f1de8e00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1de4077aa0 con 0x7f1de41049c0 2024-09-10T10:00:14.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.880+0000 7f1de8e00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1de4077920 con 0x7f1de4104010 2024-09-10T10:00:14.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.880+0000 7f1de8e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1de4077c20 con 0x7f1de4109140 2024-09-10T10:00:14.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.881+0000 7f1de1e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1de4104010 0x7f1de41a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:14.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.881+0000 7f1de2800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1de4109140 0x7f1de410f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:14.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.881+0000 7f1de1e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1de4104010 0x7f1de41a4810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:41012/0 (socket says 172.21.15.77:41012) 2024-09-10T10:00:14.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.881+0000 7f1de1e00640 1 -- 172.21.15.77:0/2908413562 learned_addr learned my addr 172.21.15.77:0/2908413562 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:00:14.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.881+0000 7f1dd9400640 1 --2- 172.21.15.77:0/2908413562 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1de41049c0 0x7f1de41a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:14.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.881+0000 7f1de1e00640 1 -- 172.21.15.77:0/2908413562 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1de4109140 msgr2=0x7f1de410f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:14.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.881+0000 7f1de1e00640 1 --2- 172.21.15.77:0/2908413562 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1de4109140 0x7f1de410f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:14.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.881+0000 7f1de1e00640 1 -- 172.21.15.77:0/2908413562 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1de41049c0 msgr2=0x7f1de41a4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:14.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.881+0000 7f1de1e00640 1 --2- 172.21.15.77:0/2908413562 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1de41049c0 0x7f1de41a4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:14.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.881+0000 7f1de1e00640 1 -- 172.21.15.77:0/2908413562 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1de4110030 con 0x7f1de4104010 2024-09-10T10:00:14.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.881+0000 7f1dd9400640 1 --2- 172.21.15.77:0/2908413562 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1de41049c0 0x7f1de41a4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:00:14.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.882+0000 7f1de1e00640 1 --2- 172.21.15.77:0/2908413562 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1de4104010 0x7f1de41a4810 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f1dd4004770 tx=0x7f1dd4003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:14.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.882+0000 7f1ddb400640 1 -- 172.21.15.77:0/2908413562 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dd403f070 con 0x7f1de4104010 2024-09-10T10:00:14.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.882+0000 7f1ddb400640 1 -- 172.21.15.77:0/2908413562 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1dd4031ca0 con 0x7f1de4104010 2024-09-10T10:00:14.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.882+0000 7f1ddb400640 1 -- 172.21.15.77:0/2908413562 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dd4035030 con 0x7f1de4104010 2024-09-10T10:00:14.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.882+0000 7f1de8e00640 1 -- 172.21.15.77:0/2908413562 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1de41102c0 con 0x7f1de4104010 2024-09-10T10:00:14.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.882+0000 7f1de8e00640 1 -- 172.21.15.77:0/2908413562 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1de41ab670 con 0x7f1de4104010 2024-09-10T10:00:14.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.886+0000 7f1deb7fe740 1 Processor -- start 2024-09-10T10:00:14.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.886+0000 7f1deb7fe740 1 -- start start 2024-09-10T10:00:14.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.886+0000 7f1ddb400640 1 -- 172.21.15.77:0/2908413562 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1dd4042070 con 0x7f1de4104010 2024-09-10T10:00:14.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.887+0000 7f1deb7fe740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c404d30c70 0x55c404d31050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:14.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.887+0000 7f1deb7fe740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c404d31590 0x55c404d3b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:14.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.887+0000 7f1deb7fe740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c404d3be50 0x55c404d3e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:14.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.887+0000 7f1deb7fe740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c404c973e0 con 0x55c404d3be50 2024-09-10T10:00:14.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.887+0000 7f1deb7fe740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c404c3ed80 con 0x55c404d31590 2024-09-10T10:00:14.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.887+0000 7f1deb7fe740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c404c933a0 con 0x55c404d30c70 2024-09-10T10:00:14.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.887+0000 7f1de2800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c404d30c70 0x55c404d31050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:14.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.887+0000 7f1de2800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c404d30c70 0x55c404d31050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:33382/0 (socket says 172.21.15.77:33382) 2024-09-10T10:00:14.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.887+0000 7f1de2800640 1 -- 172.21.15.77:0/1634419224 learned_addr learned my addr 172.21.15.77:0/1634419224 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:00:14.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.887+0000 7f1dd9400640 1 --2- 172.21.15.77:0/1634419224 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c404d3be50 0x55c404d3e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:14.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.887+0000 7f1de2800640 1 -- 172.21.15.77:0/1634419224 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c404d31590 msgr2=0x55c404d3b910 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T10:00:14.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.887+0000 7f1de2800640 1 --2- 172.21.15.77:0/1634419224 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c404d31590 0x55c404d3b910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:14.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.887+0000 7f1de2800640 1 -- 172.21.15.77:0/1634419224 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c404d3be50 msgr2=0x55c404d3e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:14.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.887+0000 7f1de2800640 1 --2- 172.21.15.77:0/1634419224 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c404d3be50 0x55c404d3e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:14.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.887+0000 7f1de2800640 1 -- 172.21.15.77:0/1634419224 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c404c5dae0 con 0x55c404d30c70 2024-09-10T10:00:14.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.888+0000 7f1de2800640 1 --2- 172.21.15.77:0/1634419224 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c404d30c70 0x55c404d31050 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1ddc009c20 tx=0x7f1ddc011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8a68a2b78daed42 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:14.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.888+0000 7f1dd3400640 1 -- 172.21.15.77:0/1634419224 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ddc01e070 con 0x55c404d30c70 2024-09-10T10:00:14.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.888+0000 7f1dd3400640 1 -- 172.21.15.77:0/1634419224 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ddc019b10 con 0x55c404d30c70 2024-09-10T10:00:14.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.888+0000 7f1dd3400640 1 -- 172.21.15.77:0/1634419224 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ddc00e940 con 0x55c404d30c70 2024-09-10T10:00:14.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.888+0000 7f1ddb400640 1 --2- 172.21.15.77:0/2908413562 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f1dc80a2c80 0x7f1dc80a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:14.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.889+0000 7f1ddb400640 1 -- 172.21.15.77:0/2908413562 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f1dd40efac0 con 0x7f1de4104010 2024-09-10T10:00:14.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.889+0000 7f1de1e00640 1 --2- 172.21.15.77:0/2908413562 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f1dc80a2c80 0x7f1dc80a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:14.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.889+0000 7f1deb7fe740 1 -- 172.21.15.77:0/1634419224 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c404d30c70 msgr2=0x55c404d31050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:14.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.889+0000 7f1deb7fe740 1 --2- 172.21.15.77:0/1634419224 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c404d30c70 0x55c404d31050 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1ddc009c20 tx=0x7f1ddc011810 comp rx=0 tx=0).stop 2024-09-10T10:00:14.892 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.889+0000 7f1deb7fe740 1 -- 172.21.15.77:0/1634419224 shutdown_connections 2024-09-10T10:00:14.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.889+0000 7f1deb7fe740 1 --2- 172.21.15.77:0/1634419224 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c404d3be50 0x55c404d3e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:14.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.889+0000 7f1deb7fe740 1 --2- 172.21.15.77:0/1634419224 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c404d31590 0x55c404d3b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:14.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.889+0000 7f1deb7fe740 1 --2- 172.21.15.77:0/1634419224 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c404d30c70 0x55c404d31050 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:14.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.889+0000 7f1deb7fe740 1 -- 172.21.15.77:0/1634419224 >> 172.21.15.77:0/1634419224 conn(0x55c404d2d200 msgr2=0x55c404d2d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:14.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.889+0000 7f1deb7fe740 1 -- 172.21.15.77:0/1634419224 shutdown_connections 2024-09-10T10:00:14.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.889+0000 7f1deb7fe740 1 -- 172.21.15.77:0/1634419224 wait complete. 2024-09-10T10:00:14.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.890+0000 7f1deb7fe740 1 -- start start 2024-09-10T10:00:14.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.890+0000 7f1deb7fe740 1 Processor -- start 2024-09-10T10:00:14.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.890+0000 7f1de1e00640 1 --2- 172.21.15.77:0/2908413562 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f1dc80a2c80 0x7f1dc80a5140 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f1dd40b8960 tx=0x7f1dd403b890 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:14.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.890+0000 7f1deb7fe740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c404d30c70 0x55c404d38d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:14.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.890+0000 7f1deb7fe740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c404d31590 0x55c404d39280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:14.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.890+0000 7f1deb7fe740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c404d3be50 0x55c404d397c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:14.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.890+0000 7f1deb7fe740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c404d3ea20 con 0x55c404d31590 2024-09-10T10:00:14.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.890+0000 7f1deb7fe740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c404c520f0 con 0x55c404d30c70 2024-09-10T10:00:14.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.890+0000 7f1deb7fe740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c404d3eba0 con 0x55c404d3be50 2024-09-10T10:00:14.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.890+0000 7f1deb7fe740 10 client.0 _ll_register_callbacks cb 0x55c404c39bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:00:14.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.891+0000 7f1dd9400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c404d31590 0x55c404d39280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:14.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.891+0000 7f1dd9400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c404d31590 0x55c404d39280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57330/0 (socket says 172.21.15.77:57330) 2024-09-10T10:00:14.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.891+0000 7f1dd9400640 1 -- 172.21.15.77:0/2178516343 learned_addr learned my addr 172.21.15.77:0/2178516343 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:00:14.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.891+0000 7f1de2800640 1 --2- 172.21.15.77:0/2178516343 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c404d30c70 0x55c404d38d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:14.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.891+0000 7f1de2800640 1 --2- 172.21.15.77:0/2178516343 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c404d3be50 0x55c404d397c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:14.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.891+0000 7f1dd9400640 1 -- 172.21.15.77:0/2178516343 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c404d3be50 msgr2=0x55c404d397c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:14.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.891+0000 7f1dd9400640 1 --2- 172.21.15.77:0/2178516343 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c404d3be50 0x55c404d397c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:14.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.891+0000 7f1dd9400640 1 -- 172.21.15.77:0/2178516343 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c404d30c70 msgr2=0x55c404d38d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:14.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.891+0000 7f1dd9400640 1 --2- 172.21.15.77:0/2178516343 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c404d30c70 0x55c404d38d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:14.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.891+0000 7f1dd9400640 1 -- 172.21.15.77:0/2178516343 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c404a083c0 con 0x55c404d31590 2024-09-10T10:00:14.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.892+0000 7f1de2800640 1 --2- 172.21.15.77:0/2178516343 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c404d3be50 0x55c404d397c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:00:14.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.892+0000 7f1dd9400640 1 --2- 172.21.15.77:0/2178516343 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c404d31590 0x55c404d39280 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc009bc0 tx=0x7f1dcc004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=385b72c9162cc635 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:14.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.892+0000 7f1dc7e00640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-10T10:00:14.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.893+0000 7f1dc7e00640 1 -- 172.21.15.77:0/2178516343 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dcc012860 con 0x55c404d31590 2024-09-10T10:00:14.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.893+0000 7f1dc7e00640 1 -- 172.21.15.77:0/2178516343 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f1dcc012a00 con 0x55c404d31590 2024-09-10T10:00:14.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.893+0000 7f1deb7fe740 1 -- 172.21.15.77:0/2178516343 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ddc019b10 con 0x55c404d31590 2024-09-10T10:00:14.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.894+0000 7f1dc7e00640 1 -- 172.21.15.77:0/2178516343 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1dcc012c00 con 0x55c404d31590 2024-09-10T10:00:14.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.894+0000 7f1deb7fe740 10 client.16482 fetch_fsmap learned FSMap version 32 2024-09-10T10:00:14.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.894+0000 7f1deb7fe740 1 -- 172.21.15.77:0/2178516343 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c404d4aaf0 con 0x55c404d31590 2024-09-10T10:00:14.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.895+0000 7f1dc7e00640 1 -- 172.21.15.77:0/2178516343 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ddc019b10 con 0x55c404d31590 2024-09-10T10:00:14.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.895+0000 7f1dc7e00640 1 -- 172.21.15.77:0/2178516343 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f1dcc025420 con 0x55c404d31590 2024-09-10T10:00:14.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.895+0000 7f1deb7fe740 10 client.16482 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T10:00:14.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.895+0000 7f1deb7fe740 10 client.16482 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T10:00:14.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.895+0000 7f1deb7fe740 20 client.16482 populate_metadata read hostname 'smithi077' 2024-09-10T10:00:14.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.895+0000 7f1deb7fe740 1 --2- 172.21.15.77:0/2178516343 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55c404d4c000 0x55c404d4e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:14.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.895+0000 7f1deb7fe740 4 client.16482 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:00:14.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.895+0000 7f1deb7fe740 1 -- 172.21.15.77:0/2178516343 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c404ca4950 con 0x55c404d4c000 2024-09-10T10:00:14.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.895+0000 7f1de2800640 1 --2- 172.21.15.77:0/2178516343 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55c404d4c000 0x55c404d4e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:14.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.896+0000 7f1de2800640 1 --2- 172.21.15.77:0/2178516343 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55c404d4c000 0x55c404d4e450 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7d2de62eac3aaf4 server_cookie=c89af8d3ca2385e5 in_seq=0 out_seq=0 2024-09-10T10:00:14.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.896+0000 7f1dc7e00640 10 client.16482 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T10:00:14.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.897+0000 7f1dc7e00640 1 -- 172.21.15.77:0/2178516343 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c404ca4950 con 0x55c404d4c000 2024-09-10T10:00:14.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.897+0000 7f1dc7e00640 10 client.16482 handle_command_reply: tid=0 2024-09-10T10:00:14.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.897+0000 7f1deb7fe740 1 client.16482 shutdown 2024-09-10T10:00:14.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.897+0000 7f1deb7fe740 1 -- 172.21.15.77:0/2178516343 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c404d31590 msgr2=0x55c404d39280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:14.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.897+0000 7f1deb7fe740 1 --2- 172.21.15.77:0/2178516343 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c404d31590 0x55c404d39280 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc009bc0 tx=0x7f1dcc004410 comp rx=0 tx=0).stop 2024-09-10T10:00:14.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.897+0000 7f1deb7fe740 1 -- 172.21.15.77:0/2178516343 shutdown_connections 2024-09-10T10:00:14.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.898+0000 7f1deb7fe740 1 --2- 172.21.15.77:0/2178516343 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55c404d4c000 0x55c404d4e450 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:14.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.898+0000 7f1deb7fe740 1 --2- 172.21.15.77:0/2178516343 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c404d3be50 0x55c404d397c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:14.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.898+0000 7f1deb7fe740 1 --2- 172.21.15.77:0/2178516343 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c404d31590 0x55c404d39280 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:14.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.898+0000 7f1deb7fe740 1 --2- 172.21.15.77:0/2178516343 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c404d30c70 0x55c404d38d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:14.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.898+0000 7f1deb7fe740 1 -- 172.21.15.77:0/2178516343 >> 172.21.15.77:0/2178516343 conn(0x55c404d2d200 msgr2=0x55c404d2c380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:14.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.898+0000 7f1dc7e00640 0 client.16482 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T10:00:14.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.898+0000 7f1deb7fe740 1 -- 172.21.15.77:0/2178516343 shutdown_connections 2024-09-10T10:00:14.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.898+0000 7f1deb7fe740 1 -- 172.21.15.77:0/2178516343 wait complete. 2024-09-10T10:00:14.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.898+0000 7f1deb7fe740 20 client.16482 trim_cache size 0 max 16384 2024-09-10T10:00:14.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.918+0000 7f1deb7fe740 1 Processor -- start 2024-09-10T10:00:14.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.918+0000 7f1deb7fe740 1 -- start start 2024-09-10T10:00:14.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.918+0000 7f1deb7fe740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c404d30c70 0x55c404d336d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:14.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.918+0000 7f1deb7fe740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c404d31590 0x55c404d2c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:14.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.919+0000 7f1deb7fe740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c404d3be50 0x55c404d53d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:14.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.919+0000 7f1deb7fe740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1de41b5570 con 0x55c404d31590 2024-09-10T10:00:14.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.919+0000 7f1deb7fe740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1de41b7170 con 0x55c404d3be50 2024-09-10T10:00:14.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.919+0000 7f1deb7fe740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1de41b5c40 con 0x55c404d30c70 2024-09-10T10:00:14.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.919+0000 7f1dd9400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c404d31590 0x55c404d2c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:14.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.919+0000 7f1dd9400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c404d31590 0x55c404d2c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57334/0 (socket says 172.21.15.77:57334) 2024-09-10T10:00:14.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.919+0000 7f1dd9400640 1 -- 172.21.15.77:0/4279601090 learned_addr learned my addr 172.21.15.77:0/4279601090 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:00:14.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.919+0000 7f1de2800640 1 --2- 172.21.15.77:0/4279601090 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c404d30c70 0x55c404d336d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:14.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.919+0000 7f1de2800640 1 --2- 172.21.15.77:0/4279601090 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c404d3be50 0x55c404d53d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:14.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.920+0000 7f1de2800640 1 -- 172.21.15.77:0/4279601090 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c404d3be50 msgr2=0x55c404d53d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:14.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.920+0000 7f1de2800640 1 --2- 172.21.15.77:0/4279601090 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c404d3be50 0x55c404d53d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:14.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.920+0000 7f1de2800640 1 -- 172.21.15.77:0/4279601090 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c404d31590 msgr2=0x55c404d2c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:14.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.920+0000 7f1de2800640 1 --2- 172.21.15.77:0/4279601090 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c404d31590 0x55c404d2c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:14.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.920+0000 7f1de2800640 1 -- 172.21.15.77:0/4279601090 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c404d28770 con 0x55c404d30c70 2024-09-10T10:00:14.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.920+0000 7f1de2800640 1 --2- 172.21.15.77:0/4279601090 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c404d30c70 0x55c404d336d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1ddc009bf0 tx=0x7f1ddc0096a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17d486082405720e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:14.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.921+0000 7f1dd2a00640 1 -- 172.21.15.77:0/4279601090 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ddc01e040 con 0x55c404d30c70 2024-09-10T10:00:14.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.921+0000 7f1dd2a00640 1 -- 172.21.15.77:0/4279601090 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ddc019b10 con 0x55c404d30c70 2024-09-10T10:00:14.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.921+0000 7f1dd2a00640 1 -- 172.21.15.77:0/4279601090 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ddc003de0 con 0x55c404d30c70 2024-09-10T10:00:14.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.922+0000 7f1deb7fe740 1 -- 172.21.15.77:0/4279601090 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c404d30c70 msgr2=0x55c404d336d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:14.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.922+0000 7f1deb7fe740 1 --2- 172.21.15.77:0/4279601090 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c404d30c70 0x55c404d336d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1ddc009bf0 tx=0x7f1ddc0096a0 comp rx=0 tx=0).stop 2024-09-10T10:00:14.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.922+0000 7f1deb7fe740 1 -- 172.21.15.77:0/4279601090 shutdown_connections 2024-09-10T10:00:14.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.922+0000 7f1deb7fe740 1 --2- 172.21.15.77:0/4279601090 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c404d3be50 0x55c404d53d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:14.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.922+0000 7f1deb7fe740 1 --2- 172.21.15.77:0/4279601090 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c404d31590 0x55c404d2c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:14.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.922+0000 7f1deb7fe740 1 --2- 172.21.15.77:0/4279601090 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c404d30c70 0x55c404d336d0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:14.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.922+0000 7f1deb7fe740 1 -- 172.21.15.77:0/4279601090 >> 172.21.15.77:0/4279601090 conn(0x55c404d2d200 msgr2=0x55c404d2bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:14.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.922+0000 7f1deb7fe740 1 -- 172.21.15.77:0/4279601090 shutdown_connections 2024-09-10T10:00:14.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.922+0000 7f1deb7fe740 1 -- 172.21.15.77:0/4279601090 wait complete. 2024-09-10T10:00:14.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.923+0000 7f1deb7fe740 1 -- start start 2024-09-10T10:00:14.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.923+0000 7f1deb7fe740 1 Processor -- start 2024-09-10T10:00:14.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.924+0000 7f1deb7fe740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c404d30c70 0x55c404d40350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:14.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.924+0000 7f1deb7fe740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c404d31590 0x55c404d40890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:14.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.924+0000 7f1deb7fe740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c404d3be50 0x55c404db5d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:14.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.924+0000 7f1deb7fe740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1de41b69e0 con 0x55c404d30c70 2024-09-10T10:00:14.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.924+0000 7f1deb7fe740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1de41b6310 con 0x55c404d31590 2024-09-10T10:00:14.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.924+0000 7f1deb7fe740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1de41b3550 con 0x55c404d3be50 2024-09-10T10:00:14.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.924+0000 7f1deb7fe740 10 client.0 _ll_register_callbacks cb 0x7f1dcc003a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:00:14.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.924+0000 7f1de2800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c404d30c70 0x55c404d40350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:14.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.924+0000 7f1de2800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c404d30c70 0x55c404d40350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57340/0 (socket says 172.21.15.77:57340) 2024-09-10T10:00:14.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.924+0000 7f1de2800640 1 -- 172.21.15.77:0/2595984374 learned_addr learned my addr 172.21.15.77:0/2595984374 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:00:14.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.924+0000 7f1de2800640 1 --2- 172.21.15.77:0/2595984374 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c404d3be50 0x55c404db5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:14.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.924+0000 7f1dd9400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c404d31590 0x55c404d40890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:14.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.925+0000 7f1de2800640 1 -- 172.21.15.77:0/2595984374 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c404d3be50 msgr2=0x55c404db5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:14.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.925+0000 7f1de2800640 1 --2- 172.21.15.77:0/2595984374 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c404d3be50 0x55c404db5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:14.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.925+0000 7f1de2800640 1 -- 172.21.15.77:0/2595984374 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c404d31590 msgr2=0x55c404d40890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:14.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.925+0000 7f1de2800640 1 --2- 172.21.15.77:0/2595984374 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c404d31590 0x55c404d40890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:14.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.925+0000 7f1de2800640 1 -- 172.21.15.77:0/2595984374 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c404db6320 con 0x55c404d30c70 2024-09-10T10:00:14.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.925+0000 7f1de2800640 1 --2- 172.21.15.77:0/2595984374 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c404d30c70 0x55c404d40350 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f1ddc0059f0 tx=0x7f1ddc0039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=316e51243d69d80e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:14.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.925+0000 7f1dc7e00640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-10T10:00:14.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.925+0000 7f1dc7e00640 1 -- 172.21.15.77:0/2595984374 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ddc01e050 con 0x55c404d30c70 2024-09-10T10:00:14.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.926+0000 7f1dc7e00640 1 -- 172.21.15.77:0/2595984374 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f1ddc0035f0 con 0x55c404d30c70 2024-09-10T10:00:14.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.926+0000 7f1deb7fe740 1 -- 172.21.15.77:0/2595984374 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ddc019b10 con 0x55c404d30c70 2024-09-10T10:00:14.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.926+0000 7f1dc7e00640 1 -- 172.21.15.77:0/2595984374 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1ddc045050 con 0x55c404d30c70 2024-09-10T10:00:14.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.926+0000 7f1deb7fe740 10 client.16488 fetch_fsmap learned FSMap version 32 2024-09-10T10:00:14.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.926+0000 7f1deb7fe740 1 -- 172.21.15.77:0/2595984374 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c404db7600 con 0x55c404d30c70 2024-09-10T10:00:14.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.926+0000 7f1dc7e00640 1 -- 172.21.15.77:0/2595984374 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ddc019b10 con 0x55c404d30c70 2024-09-10T10:00:14.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.926+0000 7f1dc7e00640 1 -- 172.21.15.77:0/2595984374 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f1ddc00e530 con 0x55c404d30c70 2024-09-10T10:00:14.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.927+0000 7f1deb7fe740 10 client.16488 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T10:00:14.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.927+0000 7f1deb7fe740 10 client.16488 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T10:00:14.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.927+0000 7f1deb7fe740 20 client.16488 populate_metadata read hostname 'smithi077' 2024-09-10T10:00:14.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.927+0000 7f1deb7fe740 1 --2- 172.21.15.77:0/2595984374 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55c404db7a80 0x55c404db9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:14.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.927+0000 7f1deb7fe740 4 client.16488 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:00:14.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.927+0000 7f1deb7fe740 1 -- 172.21.15.77:0/2595984374 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c404c9ee60 con 0x55c404db7a80 2024-09-10T10:00:14.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.927+0000 7f1dd9400640 1 --2- 172.21.15.77:0/2595984374 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55c404db7a80 0x55c404db9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:14.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.927+0000 7f1dc7e00640 10 client.16488 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T10:00:14.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.927+0000 7f1dd9400640 1 --2- 172.21.15.77:0/2595984374 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55c404db7a80 0x55c404db9ea0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f6d0cc7e3106bb3 server_cookie=94e5fc973abeee92 in_seq=0 out_seq=0 2024-09-10T10:00:14.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.928+0000 7f1dc7e00640 1 -- 172.21.15.77:0/2595984374 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c404c9ee60 con 0x55c404db7a80 2024-09-10T10:00:14.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.928+0000 7f1dc7e00640 10 client.16488 handle_command_reply: tid=0 2024-09-10T10:00:14.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.928+0000 7f1deb7fe740 1 client.16488 shutdown 2024-09-10T10:00:14.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.928+0000 7f1deb7fe740 1 -- 172.21.15.77:0/2595984374 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c404d30c70 msgr2=0x55c404d40350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:14.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.928+0000 7f1deb7fe740 1 --2- 172.21.15.77:0/2595984374 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c404d30c70 0x55c404d40350 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f1ddc0059f0 tx=0x7f1ddc0039c0 comp rx=0 tx=0).stop 2024-09-10T10:00:14.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.929+0000 7f1deb7fe740 1 -- 172.21.15.77:0/2595984374 shutdown_connections 2024-09-10T10:00:14.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.929+0000 7f1deb7fe740 1 --2- 172.21.15.77:0/2595984374 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55c404db7a80 0x55c404db9ea0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:14.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.929+0000 7f1deb7fe740 1 --2- 172.21.15.77:0/2595984374 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c404d3be50 0x55c404db5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:14.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.929+0000 7f1deb7fe740 1 --2- 172.21.15.77:0/2595984374 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c404d31590 0x55c404d40890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:14.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.929+0000 7f1deb7fe740 1 --2- 172.21.15.77:0/2595984374 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c404d30c70 0x55c404d40350 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:14.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.929+0000 7f1deb7fe740 1 -- 172.21.15.77:0/2595984374 >> 172.21.15.77:0/2595984374 conn(0x55c404d2d200 msgr2=0x55c404d39500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:14.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.929+0000 7f1dc7e00640 0 client.16488 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T10:00:14.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.929+0000 7f1deb7fe740 1 -- 172.21.15.77:0/2595984374 shutdown_connections 2024-09-10T10:00:14.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.929+0000 7f1deb7fe740 1 -- 172.21.15.77:0/2595984374 wait complete. 2024-09-10T10:00:14.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.930+0000 7f1deb7fe740 20 client.16488 trim_cache size 0 max 16384 2024-09-10T10:00:14.931 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-10T10:00:14.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.930+0000 7f1de8e00640 1 -- 172.21.15.77:0/2908413562 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f1dc80a2c80 msgr2=0x7f1dc80a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:14.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.930+0000 7f1de8e00640 1 --2- 172.21.15.77:0/2908413562 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f1dc80a2c80 0x7f1dc80a5140 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f1dd40b8960 tx=0x7f1dd403b890 comp rx=0 tx=0).stop 2024-09-10T10:00:14.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.931+0000 7f1de8e00640 1 -- 172.21.15.77:0/2908413562 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1de4104010 msgr2=0x7f1de41a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:14.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.931+0000 7f1de8e00640 1 --2- 172.21.15.77:0/2908413562 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1de4104010 0x7f1de41a4810 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f1dd4004770 tx=0x7f1dd4003c10 comp rx=0 tx=0).stop 2024-09-10T10:00:14.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.931+0000 7f1de8e00640 1 -- 172.21.15.77:0/2908413562 shutdown_connections 2024-09-10T10:00:14.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.931+0000 7f1de8e00640 1 --2- 172.21.15.77:0/2908413562 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f1dc80a2c80 0x7f1dc80a5140 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:14.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.931+0000 7f1de8e00640 1 --2- 172.21.15.77:0/2908413562 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1de4109140 0x7f1de410f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:14.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.931+0000 7f1de8e00640 1 --2- 172.21.15.77:0/2908413562 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1de41049c0 0x7f1de41a4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:14.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.931+0000 7f1de8e00640 1 --2- 172.21.15.77:0/2908413562 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1de4104010 0x7f1de41a4810 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:14.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.931+0000 7f1de8e00640 1 -- 172.21.15.77:0/2908413562 >> 172.21.15.77:0/2908413562 conn(0x7f1de40ffa50 msgr2=0x7f1de407a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:14.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.931+0000 7f1de8e00640 1 -- 172.21.15.77:0/2908413562 shutdown_connections 2024-09-10T10:00:14.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:14.931+0000 7f1de8e00640 1 -- 172.21.15.77:0/2908413562 wait complete. 2024-09-10T10:00:15.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:00:14 smithi077 ceph-mon[30319]: pgmap v543: 129 pgs: 129 active+clean; 99 MiB data, 974 MiB used, 1.0 TiB / 1.0 TiB avail; 957 KiB/s rd, 1.1 MiB/s wr, 198 op/s 2024-09-10T10:00:15.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:00:14 smithi134 ceph-mon[35674]: pgmap v543: 129 pgs: 129 active+clean; 99 MiB data, 974 MiB used, 1.0 TiB / 1.0 TiB avail; 957 KiB/s rd, 1.1 MiB/s wr, 198 op/s 2024-09-10T10:00:15.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:00:14 smithi120 ceph-mon[34770]: pgmap v543: 129 pgs: 129 active+clean; 99 MiB data, 974 MiB used, 1.0 TiB / 1.0 TiB avail; 957 KiB/s rd, 1.1 MiB/s wr, 198 op/s 2024-09-10T10:00:15.673 INFO:tasks.workunit.client.0.smithi077.stdout:copying to largest ever: 0x9fdd84 2024-09-10T10:00:16.760 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T10:00:16.760 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T10:00:17.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:00:16 smithi077 ceph-mon[30319]: pgmap v544: 129 pgs: 129 active+clean; 103 MiB data, 979 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.2 MiB/s wr, 214 op/s 2024-09-10T10:00:17.247 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:00:17.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:00:16 smithi134 ceph-mon[35674]: pgmap v544: 129 pgs: 129 active+clean; 103 MiB data, 979 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.2 MiB/s wr, 214 op/s 2024-09-10T10:00:17.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:00:16 smithi120 ceph-mon[34770]: pgmap v544: 129 pgs: 129 active+clean; 103 MiB data, 979 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.2 MiB/s wr, 214 op/s 2024-09-10T10:00:17.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.915+0000 7fafc0e00640 1 -- 172.21.15.77:0/1150098247 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fafbc076720 msgr2=0x7fafbc076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:17.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.915+0000 7fafc0e00640 1 --2- 172.21.15.77:0/1150098247 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fafbc076720 0x7fafbc076b00 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fafac009860 tx=0x7fafac02a340 comp rx=0 tx=0).stop 2024-09-10T10:00:17.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.915+0000 7fafc0e00640 1 -- 172.21.15.77:0/1150098247 shutdown_connections 2024-09-10T10:00:17.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.915+0000 7fafc0e00640 1 --2- 172.21.15.77:0/1150098247 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fafbc112870 0x7fafbc114c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:17.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.915+0000 7fafc0e00640 1 --2- 172.21.15.77:0/1150098247 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fafbc077040 0x7fafbc112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:17.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.915+0000 7fafc0e00640 1 --2- 172.21.15.77:0/1150098247 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fafbc076720 0x7fafbc076b00 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:17.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.915+0000 7fafc0e00640 1 -- 172.21.15.77:0/1150098247 >> 172.21.15.77:0/1150098247 conn(0x7fafbc0ff8f0 msgr2=0x7fafbc101d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:17.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.915+0000 7fafc0e00640 1 -- 172.21.15.77:0/1150098247 shutdown_connections 2024-09-10T10:00:17.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.915+0000 7fafc0e00640 1 -- 172.21.15.77:0/1150098247 wait complete. 2024-09-10T10:00:17.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.916+0000 7fafc0e00640 1 Processor -- start 2024-09-10T10:00:17.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.916+0000 7fafc0e00640 1 -- start start 2024-09-10T10:00:17.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.917+0000 7fafc0e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fafbc076720 0x7fafbc1a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:17.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.917+0000 7fafc0e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fafbc077040 0x7fafbc1a4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:17.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.917+0000 7fafc0e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fafbc112870 0x7fafbc19e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:17.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.917+0000 7fafc0e00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fafbc118400 con 0x7fafbc076720 2024-09-10T10:00:17.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.917+0000 7fafc0e00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fafbc118280 con 0x7fafbc112870 2024-09-10T10:00:17.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.917+0000 7fafc0e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fafbc118580 con 0x7fafbc077040 2024-09-10T10:00:17.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.917+0000 7fafb3e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fafbc077040 0x7fafbc1a4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:17.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.917+0000 7fafbb400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fafbc076720 0x7fafbc1a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:17.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.917+0000 7fafb3e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fafbc077040 0x7fafbc1a4b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:33426/0 (socket says 172.21.15.77:33426) 2024-09-10T10:00:17.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.917+0000 7fafb3e00640 1 -- 172.21.15.77:0/4228754502 learned_addr learned my addr 172.21.15.77:0/4228754502 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:00:17.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.917+0000 7fafbbe00640 1 --2- 172.21.15.77:0/4228754502 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fafbc112870 0x7fafbc19e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:17.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.918+0000 7fafb3e00640 1 -- 172.21.15.77:0/4228754502 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fafbc112870 msgr2=0x7fafbc19e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:17.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.918+0000 7fafb3e00640 1 --2- 172.21.15.77:0/4228754502 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fafbc112870 0x7fafbc19e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:17.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.918+0000 7fafb3e00640 1 -- 172.21.15.77:0/4228754502 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fafbc076720 msgr2=0x7fafbc1a4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:17.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.918+0000 7fafb3e00640 1 --2- 172.21.15.77:0/4228754502 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fafbc076720 0x7fafbc1a4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:17.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.918+0000 7fafb3e00640 1 -- 172.21.15.77:0/4228754502 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafbc19ef40 con 0x7fafbc077040 2024-09-10T10:00:17.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.918+0000 7fafb3e00640 1 --2- 172.21.15.77:0/4228754502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fafbc077040 0x7fafbc1a4b40 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fafa400e9c0 tx=0x7fafa400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:17.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.918+0000 7fafb8c00640 1 -- 172.21.15.77:0/4228754502 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa400ccc0 con 0x7fafbc077040 2024-09-10T10:00:17.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.919+0000 7fafc0e00640 1 -- 172.21.15.77:0/4228754502 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafbc19f230 con 0x7fafbc077040 2024-09-10T10:00:17.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.919+0000 7fafb8c00640 1 -- 172.21.15.77:0/4228754502 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafa4005500 con 0x7fafbc077040 2024-09-10T10:00:17.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.919+0000 7fafb8c00640 1 -- 172.21.15.77:0/4228754502 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa4010430 con 0x7fafbc077040 2024-09-10T10:00:17.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.919+0000 7fafc0e00640 1 -- 172.21.15.77:0/4228754502 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafbc1ab580 con 0x7fafbc077040 2024-09-10T10:00:17.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.922+0000 7fafc4e4a740 1 Processor -- start 2024-09-10T10:00:17.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.922+0000 7fafc4e4a740 1 -- start start 2024-09-10T10:00:17.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.922+0000 7fafb8c00640 1 -- 172.21.15.77:0/4228754502 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fafa40046c0 con 0x7fafbc077040 2024-09-10T10:00:17.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.923+0000 7fafc4e4a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0b6fb5cb0 0x55c0b6fb6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:17.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.923+0000 7fafc4e4a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c0b6fb65d0 0x55c0b6fc0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:17.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.923+0000 7fafc4e4a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c0b6fc0e90 0x55c0b6fc3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:17.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.923+0000 7fafc4e4a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c0b6f1c290 con 0x55c0b6fb65d0 2024-09-10T10:00:17.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.923+0000 7fafc4e4a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c0b6ec3b40 con 0x55c0b6fc0e90 2024-09-10T10:00:17.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.923+0000 7fafc4e4a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c0b6f18250 con 0x55c0b6fb5cb0 2024-09-10T10:00:17.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.923+0000 7fafbbe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0b6fb5cb0 0x55c0b6fb6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:17.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.923+0000 7fafbb400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c0b6fb65d0 0x55c0b6fc0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:17.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.923+0000 7fafbbe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0b6fb5cb0 0x55c0b6fb6090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:33440/0 (socket says 172.21.15.77:33440) 2024-09-10T10:00:17.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.923+0000 7fafbbe00640 1 -- 172.21.15.77:0/2191472180 learned_addr learned my addr 172.21.15.77:0/2191472180 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:00:17.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.923+0000 7fafb3e00640 1 --2- 172.21.15.77:0/2191472180 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c0b6fc0e90 0x55c0b6fc3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:17.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.924+0000 7fafbb400640 1 -- 172.21.15.77:0/2191472180 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0b6fb5cb0 msgr2=0x55c0b6fb6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:17.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.924+0000 7fafbb400640 1 --2- 172.21.15.77:0/2191472180 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0b6fb5cb0 0x55c0b6fb6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:17.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.924+0000 7fafbb400640 1 -- 172.21.15.77:0/2191472180 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c0b6fc0e90 msgr2=0x55c0b6fc3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:17.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.924+0000 7fafbb400640 1 --2- 172.21.15.77:0/2191472180 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c0b6fc0e90 0x55c0b6fc3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:17.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.924+0000 7fafbb400640 1 -- 172.21.15.77:0/2191472180 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0b6c8d6a0 con 0x55c0b6fb65d0 2024-09-10T10:00:17.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.924+0000 7fafb3e00640 1 --2- 172.21.15.77:0/2191472180 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c0b6fc0e90 0x55c0b6fc3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:00:17.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.924+0000 7fafbb400640 1 --2- 172.21.15.77:0/2191472180 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c0b6fb65d0 0x55c0b6fc0950 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fafac035040 tx=0x7fafac009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=984e6ff129f75815 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:17.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.924+0000 7fafb0c00640 1 -- 172.21.15.77:0/2191472180 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafac041020 con 0x55c0b6fb65d0 2024-09-10T10:00:17.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.925+0000 7fafb0c00640 1 -- 172.21.15.77:0/2191472180 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafac030e50 con 0x55c0b6fb65d0 2024-09-10T10:00:17.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.925+0000 7fafb8c00640 1 --2- 172.21.15.77:0/4228754502 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fafa00a2b80 0x7fafa00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:17.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.925+0000 7fafb8c00640 1 -- 172.21.15.77:0/4228754502 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fafa40cf720 con 0x7fafbc077040 2024-09-10T10:00:17.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.925+0000 7fafbbe00640 1 --2- 172.21.15.77:0/4228754502 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fafa00a2b80 0x7fafa00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:17.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.926+0000 7fafc4e4a740 1 -- 172.21.15.77:0/2191472180 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c0b6fb65d0 msgr2=0x55c0b6fc0950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:17.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.926+0000 7fafc4e4a740 1 --2- 172.21.15.77:0/2191472180 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c0b6fb65d0 0x55c0b6fc0950 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fafac035040 tx=0x7fafac009510 comp rx=0 tx=0).stop 2024-09-10T10:00:17.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.926+0000 7fafc4e4a740 1 -- 172.21.15.77:0/2191472180 shutdown_connections 2024-09-10T10:00:17.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.926+0000 7fafc4e4a740 1 --2- 172.21.15.77:0/2191472180 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c0b6fc0e90 0x55c0b6fc3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:17.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.926+0000 7fafc4e4a740 1 --2- 172.21.15.77:0/2191472180 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c0b6fb65d0 0x55c0b6fc0950 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:17.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.926+0000 7fafc4e4a740 1 --2- 172.21.15.77:0/2191472180 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0b6fb5cb0 0x55c0b6fb6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:17.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.926+0000 7fafc4e4a740 1 -- 172.21.15.77:0/2191472180 >> 172.21.15.77:0/2191472180 conn(0x55c0b6fb2240 msgr2=0x55c0b6fb2620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:17.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.926+0000 7fafc4e4a740 1 -- 172.21.15.77:0/2191472180 shutdown_connections 2024-09-10T10:00:17.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.926+0000 7fafc4e4a740 1 -- 172.21.15.77:0/2191472180 wait complete. 2024-09-10T10:00:17.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.926+0000 7fafbbe00640 1 --2- 172.21.15.77:0/4228754502 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fafa00a2b80 0x7fafa00a5040 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fafb4011630 tx=0x7fafb4009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:17.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.926+0000 7fafc4e4a740 1 -- start start 2024-09-10T10:00:17.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.927+0000 7fafc4e4a740 1 Processor -- start 2024-09-10T10:00:17.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.927+0000 7fafc4e4a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0b6fb5cb0 0x55c0b6fc0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:17.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.927+0000 7fafc4e4a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c0b6fb65d0 0x55c0b6fbd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:17.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.927+0000 7fafc4e4a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c0b6fc0e90 0x55c0b6fbde10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:17.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.927+0000 7fafc4e4a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c0b6fc3b50 con 0x55c0b6fb65d0 2024-09-10T10:00:17.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.927+0000 7fafc4e4a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c0b6ed6fa0 con 0x55c0b6fc0e90 2024-09-10T10:00:17.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.927+0000 7fafc4e4a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c0b6fc3cd0 con 0x55c0b6fb5cb0 2024-09-10T10:00:17.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.927+0000 7fafc4e4a740 10 client.0 _ll_register_callbacks cb 0x55c0b6c2aba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:00:17.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.927+0000 7fafbb400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0b6fb5cb0 0x55c0b6fc0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:17.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.927+0000 7fafbb400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0b6fb5cb0 0x55c0b6fc0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:33444/0 (socket says 172.21.15.77:33444) 2024-09-10T10:00:17.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.927+0000 7fafbb400640 1 -- 172.21.15.77:0/250654121 learned_addr learned my addr 172.21.15.77:0/250654121 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:00:17.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.927+0000 7fafbbe00640 1 --2- 172.21.15.77:0/250654121 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c0b6fc0e90 0x55c0b6fbde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:17.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.927+0000 7fafb3e00640 1 --2- 172.21.15.77:0/250654121 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c0b6fb65d0 0x55c0b6fbd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:17.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.928+0000 7fafbb400640 1 -- 172.21.15.77:0/250654121 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c0b6fc0e90 msgr2=0x55c0b6fbde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:17.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.928+0000 7fafbb400640 1 --2- 172.21.15.77:0/250654121 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c0b6fc0e90 0x55c0b6fbde10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:17.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.928+0000 7fafbb400640 1 -- 172.21.15.77:0/250654121 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c0b6fb65d0 msgr2=0x55c0b6fbd8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:17.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.928+0000 7fafbb400640 1 --2- 172.21.15.77:0/250654121 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c0b6fb65d0 0x55c0b6fbd8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:17.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.928+0000 7fafbb400640 1 -- 172.21.15.77:0/250654121 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0b6fbe500 con 0x55c0b6fb5cb0 2024-09-10T10:00:17.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.928+0000 7fafbb400640 1 --2- 172.21.15.77:0/250654121 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0b6fb5cb0 0x55c0b6fc0620 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fafac005b60 tx=0x7fafac004440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3a56893f257b26d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:17.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.928+0000 7fafa9600640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-10T10:00:17.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.929+0000 7fafa9600640 1 -- 172.21.15.77:0/250654121 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafac041030 con 0x55c0b6fb5cb0 2024-09-10T10:00:17.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.929+0000 7fafc4e4a740 1 -- 172.21.15.77:0/250654121 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fafac030e50 con 0x55c0b6fb5cb0 2024-09-10T10:00:17.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.929+0000 7fafa9600640 1 -- 172.21.15.77:0/250654121 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fafac030760 con 0x55c0b6fb5cb0 2024-09-10T10:00:17.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.929+0000 7fafa9600640 1 -- 172.21.15.77:0/250654121 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fafac030960 con 0x55c0b6fb5cb0 2024-09-10T10:00:17.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.929+0000 7fafc4e4a740 10 client.26539 fetch_fsmap learned FSMap version 32 2024-09-10T10:00:17.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.929+0000 7fafc4e4a740 1 -- 172.21.15.77:0/250654121 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c0b6fc3b50 con 0x55c0b6fb5cb0 2024-09-10T10:00:17.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.929+0000 7fafa9600640 1 -- 172.21.15.77:0/250654121 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafac030e50 con 0x55c0b6fb5cb0 2024-09-10T10:00:17.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.931+0000 7fafa9600640 1 -- 172.21.15.77:0/250654121 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fafac03da20 con 0x55c0b6fb5cb0 2024-09-10T10:00:17.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.931+0000 7fafc4e4a740 10 client.26539 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T10:00:17.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.931+0000 7fafc4e4a740 10 client.26539 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T10:00:17.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.931+0000 7fafc4e4a740 20 client.26539 populate_metadata read hostname 'smithi077' 2024-09-10T10:00:17.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.931+0000 7fafc4e4a740 1 --2- 172.21.15.77:0/250654121 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55c0b6fe1ae0 0x55c0b6fe3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:17.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.931+0000 7fafc4e4a740 4 client.26539 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:00:17.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.931+0000 7fafc4e4a740 1 -- 172.21.15.77:0/250654121 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c0b6f29800 con 0x55c0b6fe1ae0 2024-09-10T10:00:17.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.931+0000 7fafb3e00640 1 --2- 172.21.15.77:0/250654121 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55c0b6fe1ae0 0x55c0b6fe3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:17.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.932+0000 7fafb3e00640 1 --2- 172.21.15.77:0/250654121 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55c0b6fe1ae0 0x55c0b6fe3ed0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1c3606bfcd47a76 server_cookie=d95c1764f49ae1a4 in_seq=0 out_seq=0 2024-09-10T10:00:17.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.932+0000 7fafa9600640 10 client.26539 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T10:00:17.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.933+0000 7fafa9600640 1 -- 172.21.15.77:0/250654121 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c0b6f29800 con 0x55c0b6fe1ae0 2024-09-10T10:00:17.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.933+0000 7fafa9600640 10 client.26539 handle_command_reply: tid=0 2024-09-10T10:00:17.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.933+0000 7fafc4e4a740 1 client.26539 shutdown 2024-09-10T10:00:17.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.933+0000 7fafc4e4a740 1 -- 172.21.15.77:0/250654121 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0b6fb5cb0 msgr2=0x55c0b6fc0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:17.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.933+0000 7fafc4e4a740 1 --2- 172.21.15.77:0/250654121 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0b6fb5cb0 0x55c0b6fc0620 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fafac005b60 tx=0x7fafac004440 comp rx=0 tx=0).stop 2024-09-10T10:00:17.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.934+0000 7fafc4e4a740 1 -- 172.21.15.77:0/250654121 shutdown_connections 2024-09-10T10:00:17.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.934+0000 7fafc4e4a740 1 --2- 172.21.15.77:0/250654121 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55c0b6fe1ae0 0x55c0b6fe3ed0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:17.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.934+0000 7fafc4e4a740 1 --2- 172.21.15.77:0/250654121 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c0b6fc0e90 0x55c0b6fbde10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:17.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.934+0000 7fafc4e4a740 1 --2- 172.21.15.77:0/250654121 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c0b6fb65d0 0x55c0b6fbd8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:17.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.934+0000 7fafc4e4a740 1 --2- 172.21.15.77:0/250654121 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0b6fb5cb0 0x55c0b6fc0620 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:17.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.934+0000 7fafc4e4a740 1 -- 172.21.15.77:0/250654121 >> 172.21.15.77:0/250654121 conn(0x55c0b6fb2240 msgr2=0x55c0b6fb7f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:17.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.934+0000 7fafa9600640 0 client.26539 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T10:00:17.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.934+0000 7fafc4e4a740 1 -- 172.21.15.77:0/250654121 shutdown_connections 2024-09-10T10:00:17.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.934+0000 7fafc4e4a740 1 -- 172.21.15.77:0/250654121 wait complete. 2024-09-10T10:00:17.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.934+0000 7fafc4e4a740 20 client.26539 trim_cache size 0 max 16384 2024-09-10T10:00:17.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.956+0000 7fafc4e4a740 1 Processor -- start 2024-09-10T10:00:17.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.956+0000 7fafc4e4a740 1 -- start start 2024-09-10T10:00:17.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.956+0000 7fafc4e4a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0b6fb2240 0x55c0b6fb9110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:17.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.957+0000 7fafc4e4a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c0b6fb5cb0 0x55c0b6fb1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:17.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.957+0000 7fafc4e4a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c0b6fb65d0 0x55c0b6fe9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:17.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.957+0000 7fafc4e4a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fafbc1aca90 con 0x55c0b6fb5cb0 2024-09-10T10:00:17.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.957+0000 7fafc4e4a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fafbc1ae690 con 0x55c0b6fb65d0 2024-09-10T10:00:17.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.957+0000 7fafc4e4a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fafbc1ad160 con 0x55c0b6fb2240 2024-09-10T10:00:17.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.957+0000 7fafbbe00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c0b6fb65d0 0x55c0b6fe9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:17.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.957+0000 7fafb3e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c0b6fb5cb0 0x55c0b6fb1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:17.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.957+0000 7fafbb400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0b6fb2240 0x55c0b6fb9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:17.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.957+0000 7fafbbe00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c0b6fb65d0 0x55c0b6fe9700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:41082/0 (socket says 172.21.15.77:41082) 2024-09-10T10:00:17.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.957+0000 7fafbbe00640 1 -- 172.21.15.77:0/1991381790 learned_addr learned my addr 172.21.15.77:0/1991381790 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:00:17.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.957+0000 7fafbbe00640 1 -- 172.21.15.77:0/1991381790 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0b6fb2240 msgr2=0x55c0b6fb9110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:17.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.957+0000 7fafbbe00640 1 --2- 172.21.15.77:0/1991381790 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0b6fb2240 0x55c0b6fb9110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:17.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.957+0000 7fafbbe00640 1 -- 172.21.15.77:0/1991381790 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c0b6fb5cb0 msgr2=0x55c0b6fb1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:17.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.957+0000 7fafbbe00640 1 --2- 172.21.15.77:0/1991381790 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c0b6fb5cb0 0x55c0b6fb1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:17.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.957+0000 7fafbbe00640 1 -- 172.21.15.77:0/1991381790 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0b6fcce00 con 0x55c0b6fb65d0 2024-09-10T10:00:17.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.957+0000 7fafb3e00640 1 --2- 172.21.15.77:0/1991381790 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c0b6fb5cb0 0x55c0b6fb1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:00:17.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.958+0000 7fafbbe00640 1 --2- 172.21.15.77:0/1991381790 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c0b6fb65d0 0x55c0b6fe9700 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fafb4003de0 tx=0x7fafb4019000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5af35798d065f5b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:17.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.959+0000 7fafabe00640 1 -- 172.21.15.77:0/1991381790 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb400f4e0 con 0x55c0b6fb65d0 2024-09-10T10:00:17.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.959+0000 7fafabe00640 1 -- 172.21.15.77:0/1991381790 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafb400fb60 con 0x55c0b6fb65d0 2024-09-10T10:00:17.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.959+0000 7fafabe00640 1 -- 172.21.15.77:0/1991381790 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb4029e00 con 0x55c0b6fb65d0 2024-09-10T10:00:17.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.960+0000 7fafc4e4a740 1 -- 172.21.15.77:0/1991381790 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c0b6fb65d0 msgr2=0x55c0b6fe9700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:17.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.960+0000 7fafc4e4a740 1 --2- 172.21.15.77:0/1991381790 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c0b6fb65d0 0x55c0b6fe9700 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fafb4003de0 tx=0x7fafb4019000 comp rx=0 tx=0).stop 2024-09-10T10:00:17.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.960+0000 7fafc4e4a740 1 -- 172.21.15.77:0/1991381790 shutdown_connections 2024-09-10T10:00:17.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.960+0000 7fafc4e4a740 1 --2- 172.21.15.77:0/1991381790 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c0b6fb65d0 0x55c0b6fe9700 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:17.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.960+0000 7fafc4e4a740 1 --2- 172.21.15.77:0/1991381790 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c0b6fb5cb0 0x55c0b6fb1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:17.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.960+0000 7fafc4e4a740 1 --2- 172.21.15.77:0/1991381790 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0b6fb2240 0x55c0b6fb9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:17.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.960+0000 7fafc4e4a740 1 -- 172.21.15.77:0/1991381790 >> 172.21.15.77:0/1991381790 conn(0x55c0b6fe42c0 msgr2=0x55c0b6fb0a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:17.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.960+0000 7fafc4e4a740 1 -- 172.21.15.77:0/1991381790 shutdown_connections 2024-09-10T10:00:17.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.960+0000 7fafc4e4a740 1 -- 172.21.15.77:0/1991381790 wait complete. 2024-09-10T10:00:17.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.961+0000 7fafc4e4a740 1 -- start start 2024-09-10T10:00:17.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.961+0000 7fafc4e4a740 1 Processor -- start 2024-09-10T10:00:17.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.962+0000 7fafc4e4a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0b6fb2240 0x55c0b6fc9160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:17.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.962+0000 7fafc4e4a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c0b6fb5cb0 0x55c0b6fc96a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:17.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.962+0000 7fafc4e4a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c0b6fb65d0 0x55c0b6fc3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:17.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.962+0000 7fafc4e4a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fafbc1adf00 con 0x55c0b6fb65d0 2024-09-10T10:00:17.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.962+0000 7fafc4e4a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fafbc1ad830 con 0x55c0b6fb5cb0 2024-09-10T10:00:17.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.962+0000 7fafc4e4a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fafbc10cf50 con 0x55c0b6fb2240 2024-09-10T10:00:17.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.962+0000 7fafc4e4a740 10 client.0 _ll_register_callbacks cb 0x7fafa40d5420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:00:17.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.962+0000 7fafbb400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0b6fb2240 0x55c0b6fc9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:17.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.962+0000 7fafbb400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0b6fb2240 0x55c0b6fc9160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:33460/0 (socket says 172.21.15.77:33460) 2024-09-10T10:00:17.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.962+0000 7fafbb400640 1 -- 172.21.15.77:0/1052736805 learned_addr learned my addr 172.21.15.77:0/1052736805 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:00:17.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.962+0000 7fafb3e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c0b6fb5cb0 0x55c0b6fc96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:17.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.963+0000 7fafbb400640 1 -- 172.21.15.77:0/1052736805 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c0b6fb5cb0 msgr2=0x55c0b6fc96a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:17.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.963+0000 7fafbb400640 1 --2- 172.21.15.77:0/1052736805 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c0b6fb5cb0 0x55c0b6fc96a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:17.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.963+0000 7fafbb400640 1 -- 172.21.15.77:0/1052736805 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c0b6fb65d0 msgr2=0x55c0b6fc3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:17.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.963+0000 7fafbb400640 1 --2- 172.21.15.77:0/1052736805 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c0b6fb65d0 0x55c0b6fc3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:17.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.963+0000 7fafbb400640 1 -- 172.21.15.77:0/1052736805 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0b6fc45f0 con 0x55c0b6fb2240 2024-09-10T10:00:17.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.963+0000 7fafbb400640 1 --2- 172.21.15.77:0/1052736805 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0b6fb2240 0x55c0b6fc9160 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fafac02a930 tx=0x7fafac032670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e7d6b0ffe0a5e9f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:17.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.963+0000 7fafa9600640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-10T10:00:17.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.964+0000 7fafa9600640 1 -- 172.21.15.77:0/1052736805 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafac041070 con 0x55c0b6fb2240 2024-09-10T10:00:17.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.964+0000 7fafc4e4a740 1 -- 172.21.15.77:0/1052736805 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fafb400fb60 con 0x55c0b6fb2240 2024-09-10T10:00:17.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.964+0000 7fafa9600640 1 -- 172.21.15.77:0/1052736805 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fafac003910 con 0x55c0b6fb2240 2024-09-10T10:00:17.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.964+0000 7fafa9600640 1 -- 172.21.15.77:0/1052736805 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fafac068050 con 0x55c0b6fb2240 2024-09-10T10:00:17.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.964+0000 7fafc4e4a740 10 client.26545 fetch_fsmap learned FSMap version 32 2024-09-10T10:00:17.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.964+0000 7fafc4e4a740 1 -- 172.21.15.77:0/1052736805 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c0b704d0d0 con 0x55c0b6fb2240 2024-09-10T10:00:17.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.964+0000 7fafa9600640 1 -- 172.21.15.77:0/1052736805 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafb400fb60 con 0x55c0b6fb2240 2024-09-10T10:00:17.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.964+0000 7fafa9600640 1 -- 172.21.15.77:0/1052736805 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fafac032ba0 con 0x55c0b6fb2240 2024-09-10T10:00:17.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.964+0000 7fafc4e4a740 10 client.26545 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T10:00:17.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.964+0000 7fafc4e4a740 10 client.26545 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T10:00:17.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.964+0000 7fafc4e4a740 20 client.26545 populate_metadata read hostname 'smithi077' 2024-09-10T10:00:17.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.964+0000 7fafc4e4a740 1 --2- 172.21.15.77:0/1052736805 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55c0b6fc0e90 0x55c0b704f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:17.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.964+0000 7fafc4e4a740 4 client.26545 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T10:00:17.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.964+0000 7fafc4e4a740 1 -- 172.21.15.77:0/1052736805 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c0b6f23d10 con 0x55c0b6fc0e90 2024-09-10T10:00:17.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.965+0000 7fafb3e00640 1 --2- 172.21.15.77:0/1052736805 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55c0b6fc0e90 0x55c0b704f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:17.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.965+0000 7fafb3e00640 1 --2- 172.21.15.77:0/1052736805 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55c0b6fc0e90 0x55c0b704f650 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82495a92328f3563 server_cookie=3fce0996e077cf65 in_seq=0 out_seq=0 2024-09-10T10:00:17.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.965+0000 7fafa9600640 10 client.26545 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T10:00:17.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.966+0000 7fafa9600640 1 -- 172.21.15.77:0/1052736805 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c0b6f23d10 con 0x55c0b6fc0e90 2024-09-10T10:00:17.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.966+0000 7fafa9600640 10 client.26545 handle_command_reply: tid=0 2024-09-10T10:00:17.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.966+0000 7fafc4e4a740 1 client.26545 shutdown 2024-09-10T10:00:17.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.966+0000 7fafc4e4a740 1 -- 172.21.15.77:0/1052736805 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0b6fb2240 msgr2=0x55c0b6fc9160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:17.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.966+0000 7fafc4e4a740 1 --2- 172.21.15.77:0/1052736805 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0b6fb2240 0x55c0b6fc9160 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fafac02a930 tx=0x7fafac032670 comp rx=0 tx=0).stop 2024-09-10T10:00:17.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.966+0000 7fafc4e4a740 1 -- 172.21.15.77:0/1052736805 shutdown_connections 2024-09-10T10:00:17.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.966+0000 7fafc4e4a740 1 --2- 172.21.15.77:0/1052736805 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55c0b6fc0e90 0x55c0b704f650 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:17.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.966+0000 7fafc4e4a740 1 --2- 172.21.15.77:0/1052736805 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c0b6fb65d0 0x55c0b6fc3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:17.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.966+0000 7fafc4e4a740 1 --2- 172.21.15.77:0/1052736805 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c0b6fb5cb0 0x55c0b6fc96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:17.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.966+0000 7fafc4e4a740 1 --2- 172.21.15.77:0/1052736805 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0b6fb2240 0x55c0b6fc9160 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:17.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.966+0000 7fafa9600640 0 client.26545 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T10:00:17.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.966+0000 7fafc4e4a740 1 -- 172.21.15.77:0/1052736805 >> 172.21.15.77:0/1052736805 conn(0x55c0b6fe42c0 msgr2=0x55c0b6fb0a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:17.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.967+0000 7fafc4e4a740 1 -- 172.21.15.77:0/1052736805 shutdown_connections 2024-09-10T10:00:17.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.967+0000 7fafc4e4a740 1 -- 172.21.15.77:0/1052736805 wait complete. 2024-09-10T10:00:17.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.967+0000 7fafc4e4a740 20 client.26545 trim_cache size 0 max 16384 2024-09-10T10:00:17.969 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-10T10:00:17.969 INFO:teuthology.orchestra.run.smithi077.stdout: "return_code": 0, 2024-09-10T10:00:17.969 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_tag": "46cac69b-0aa5-462b-ad21-21f7004d640b", 2024-09-10T10:00:17.969 INFO:teuthology.orchestra.run.smithi077.stdout: "mode": "asynchronous" 2024-09-10T10:00:17.969 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-10T10:00:17.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.968+0000 7fafc0e00640 1 -- 172.21.15.77:0/4228754502 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fafa00a2b80 msgr2=0x7fafa00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:17.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.968+0000 7fafc0e00640 1 --2- 172.21.15.77:0/4228754502 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fafa00a2b80 0x7fafa00a5040 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fafb4011630 tx=0x7fafb4009500 comp rx=0 tx=0).stop 2024-09-10T10:00:17.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.968+0000 7fafc0e00640 1 -- 172.21.15.77:0/4228754502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fafbc077040 msgr2=0x7fafbc1a4b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:17.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.968+0000 7fafc0e00640 1 --2- 172.21.15.77:0/4228754502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fafbc077040 0x7fafbc1a4b40 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fafa400e9c0 tx=0x7fafa400ee80 comp rx=0 tx=0).stop 2024-09-10T10:00:17.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.968+0000 7fafc0e00640 1 -- 172.21.15.77:0/4228754502 shutdown_connections 2024-09-10T10:00:17.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.968+0000 7fafc0e00640 1 --2- 172.21.15.77:0/4228754502 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fafa00a2b80 0x7fafa00a5040 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:17.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.968+0000 7fafc0e00640 1 --2- 172.21.15.77:0/4228754502 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fafbc112870 0x7fafbc19e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:17.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.968+0000 7fafc0e00640 1 --2- 172.21.15.77:0/4228754502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fafbc077040 0x7fafbc1a4b40 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:17.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.968+0000 7fafc0e00640 1 --2- 172.21.15.77:0/4228754502 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fafbc076720 0x7fafbc1a4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:17.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.968+0000 7fafc0e00640 1 -- 172.21.15.77:0/4228754502 >> 172.21.15.77:0/4228754502 conn(0x7fafbc0ff8f0 msgr2=0x7fafbc101ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:17.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.968+0000 7fafc0e00640 1 -- 172.21.15.77:0/4228754502 shutdown_connections 2024-09-10T10:00:17.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:17.968+0000 7fafc0e00640 1 -- 172.21.15.77:0/4228754502 wait complete. 2024-09-10T10:00:18.844 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T10:00:19.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:00:18 smithi077 ceph-mon[30319]: pgmap v545: 129 pgs: 129 active+clean; 103 MiB data, 962 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.3 MiB/s wr, 240 op/s 2024-09-10T10:00:19.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:00:18 smithi077 ceph-mon[30319]: scrub queued for path: / 2024-09-10T10:00:19.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:00:18 smithi077 ceph-mon[30319]: scrub summary: idle+waiting paths [/] 2024-09-10T10:00:19.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:00:18 smithi077 ceph-mon[30319]: scrub summary: active paths [/] 2024-09-10T10:00:19.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:00:18 smithi077 ceph-mon[30319]: scrub summary: active paths [/] 2024-09-10T10:00:19.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:00:18 smithi077 ceph-mon[30319]: scrub summary: active paths [/] 2024-09-10T10:00:19.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:00:18 smithi077 ceph-mon[30319]: scrub summary: active paths [/] 2024-09-10T10:00:19.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:00:18 smithi077 ceph-mon[30319]: scrub summary: active paths [/] 2024-09-10T10:00:19.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:00:18 smithi077 ceph-mon[30319]: scrub summary: idle+waiting paths [/] 2024-09-10T10:00:19.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:00:18 smithi077 ceph-mon[30319]: scrub summary: idle+waiting paths [/] 2024-09-10T10:00:19.191 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:00:18 smithi077 ceph-mon[30319]: scrub summary: idle+waiting paths [/] 2024-09-10T10:00:19.191 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:00:18 smithi077 ceph-mon[30319]: scrub summary: idle+waiting paths [/] 2024-09-10T10:00:19.304 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:00:19.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:00:18 smithi134 ceph-mon[35674]: pgmap v545: 129 pgs: 129 active+clean; 103 MiB data, 962 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.3 MiB/s wr, 240 op/s 2024-09-10T10:00:19.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:00:18 smithi134 ceph-mon[35674]: scrub queued for path: / 2024-09-10T10:00:19.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:00:18 smithi134 ceph-mon[35674]: scrub summary: idle+waiting paths [/] 2024-09-10T10:00:19.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:00:18 smithi134 ceph-mon[35674]: scrub summary: active paths [/] 2024-09-10T10:00:19.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:00:18 smithi134 ceph-mon[35674]: scrub summary: active paths [/] 2024-09-10T10:00:19.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:00:18 smithi134 ceph-mon[35674]: scrub summary: active paths [/] 2024-09-10T10:00:19.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:00:18 smithi134 ceph-mon[35674]: scrub summary: active paths [/] 2024-09-10T10:00:19.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:00:18 smithi134 ceph-mon[35674]: scrub summary: active paths [/] 2024-09-10T10:00:19.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:00:18 smithi134 ceph-mon[35674]: scrub summary: idle+waiting paths [/] 2024-09-10T10:00:19.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:00:18 smithi134 ceph-mon[35674]: scrub summary: idle+waiting paths [/] 2024-09-10T10:00:19.382 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:00:18 smithi134 ceph-mon[35674]: scrub summary: idle+waiting paths [/] 2024-09-10T10:00:19.382 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:00:18 smithi134 ceph-mon[35674]: scrub summary: idle+waiting paths [/] 2024-09-10T10:00:19.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:00:18 smithi120 ceph-mon[34770]: pgmap v545: 129 pgs: 129 active+clean; 103 MiB data, 962 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.3 MiB/s wr, 240 op/s 2024-09-10T10:00:19.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:00:18 smithi120 ceph-mon[34770]: scrub queued for path: / 2024-09-10T10:00:19.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:00:18 smithi120 ceph-mon[34770]: scrub summary: idle+waiting paths [/] 2024-09-10T10:00:19.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:00:18 smithi120 ceph-mon[34770]: scrub summary: active paths [/] 2024-09-10T10:00:19.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:00:18 smithi120 ceph-mon[34770]: scrub summary: active paths [/] 2024-09-10T10:00:19.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:00:18 smithi120 ceph-mon[34770]: scrub summary: active paths [/] 2024-09-10T10:00:19.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:00:18 smithi120 ceph-mon[34770]: scrub summary: active paths [/] 2024-09-10T10:00:19.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:00:18 smithi120 ceph-mon[34770]: scrub summary: active paths [/] 2024-09-10T10:00:19.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:00:18 smithi120 ceph-mon[34770]: scrub summary: idle+waiting paths [/] 2024-09-10T10:00:19.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:00:18 smithi120 ceph-mon[34770]: scrub summary: idle+waiting paths [/] 2024-09-10T10:00:19.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:00:18 smithi120 ceph-mon[34770]: scrub summary: idle+waiting paths [/] 2024-09-10T10:00:19.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:00:18 smithi120 ceph-mon[34770]: scrub summary: idle+waiting paths [/] 2024-09-10T10:00:20.209 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:00:19 smithi077 ceph-mon[30319]: scrub summary: idle+waiting paths [/] 2024-09-10T10:00:20.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.350+0000 7fa2c4600640 1 -- 172.21.15.77:0/1349621718 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa2bc101de0 msgr2=0x7fa2bc1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:20.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.350+0000 7fa2c4600640 1 --2- 172.21.15.77:0/1349621718 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa2bc101de0 0x7fa2bc1021c0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fa2b0009900 tx=0x7fa2b002a400 comp rx=0 tx=0).stop 2024-09-10T10:00:20.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.350+0000 7fa2c4600640 1 -- 172.21.15.77:0/1349621718 shutdown_connections 2024-09-10T10:00:20.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.350+0000 7fa2c4600640 1 --2- 172.21.15.77:0/1349621718 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2bc106f10 0x7fa2bc10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:20.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.350+0000 7fa2c4600640 1 --2- 172.21.15.77:0/1349621718 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa2bc102790 0x7fa2bc1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:20.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.350+0000 7fa2c4600640 1 --2- 172.21.15.77:0/1349621718 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa2bc101de0 0x7fa2bc1021c0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:20.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.350+0000 7fa2c4600640 1 -- 172.21.15.77:0/1349621718 >> 172.21.15.77:0/1349621718 conn(0x7fa2bc0fd880 msgr2=0x7fa2bc0ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:20.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.351+0000 7fa2c4600640 1 -- 172.21.15.77:0/1349621718 shutdown_connections 2024-09-10T10:00:20.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.351+0000 7fa2c4600640 1 -- 172.21.15.77:0/1349621718 wait complete. 2024-09-10T10:00:20.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.351+0000 7fa2c4600640 1 Processor -- start 2024-09-10T10:00:20.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.352+0000 7fa2c4600640 1 -- start start 2024-09-10T10:00:20.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.352+0000 7fa2c4600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2bc101de0 0x7fa2bc198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:20.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.352+0000 7fa2c4600640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa2bc102790 0x7fa2bc198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:20.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.352+0000 7fa2c4600640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa2bc106f10 0x7fa2bc19cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:20.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.352+0000 7fa2c4600640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa2bc111fc0 con 0x7fa2bc106f10 2024-09-10T10:00:20.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.352+0000 7fa2c4600640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa2bc111e40 con 0x7fa2bc102790 2024-09-10T10:00:20.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.352+0000 7fa2c4600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa2bc112140 con 0x7fa2bc101de0 2024-09-10T10:00:20.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.352+0000 7fa2c1200640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa2bc102790 0x7fa2bc198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:20.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.352+0000 7fa2c1c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2bc101de0 0x7fa2bc198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:20.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.352+0000 7fa2c1200640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa2bc102790 0x7fa2bc198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:41104/0 (socket says 172.21.15.77:41104) 2024-09-10T10:00:20.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.352+0000 7fa2c1c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2bc101de0 0x7fa2bc198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:33482/0 (socket says 172.21.15.77:33482) 2024-09-10T10:00:20.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.352+0000 7fa2c1200640 1 -- 172.21.15.77:0/3252903131 learned_addr learned my addr 172.21.15.77:0/3252903131 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:00:20.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.352+0000 7fa2c2600640 1 --2- 172.21.15.77:0/3252903131 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa2bc106f10 0x7fa2bc19cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:20.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.353+0000 7fa2c1200640 1 -- 172.21.15.77:0/3252903131 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2bc101de0 msgr2=0x7fa2bc198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:20.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.353+0000 7fa2c1200640 1 --2- 172.21.15.77:0/3252903131 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2bc101de0 0x7fa2bc198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:20.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.353+0000 7fa2c1200640 1 -- 172.21.15.77:0/3252903131 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa2bc106f10 msgr2=0x7fa2bc19cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:20.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.353+0000 7fa2c1200640 1 --2- 172.21.15.77:0/3252903131 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa2bc106f10 0x7fa2bc19cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:20.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.353+0000 7fa2c1200640 1 -- 172.21.15.77:0/3252903131 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2bc19d400 con 0x7fa2bc102790 2024-09-10T10:00:20.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.353+0000 7fa2c1c00640 1 --2- 172.21.15.77:0/3252903131 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2bc101de0 0x7fa2bc198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:00:20.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.353+0000 7fa2c1200640 1 --2- 172.21.15.77:0/3252903131 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa2bc102790 0x7fa2bc198990 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa2a800e9f0 tx=0x7fa2a800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:20.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.353+0000 7fa2ba000640 1 -- 172.21.15.77:0/3252903131 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a800cd80 con 0x7fa2bc102790 2024-09-10T10:00:20.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.353+0000 7fa2c4600640 1 -- 172.21.15.77:0/3252903131 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2bc19d6f0 con 0x7fa2bc102790 2024-09-10T10:00:20.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.354+0000 7fa2ba000640 1 -- 172.21.15.77:0/3252903131 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2a8010910 con 0x7fa2bc102790 2024-09-10T10:00:20.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.354+0000 7fa2ba000640 1 -- 172.21.15.77:0/3252903131 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a8018980 con 0x7fa2bc102790 2024-09-10T10:00:20.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.354+0000 7fa2c4600640 1 -- 172.21.15.77:0/3252903131 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2bc1a4fd0 con 0x7fa2bc102790 2024-09-10T10:00:20.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.357+0000 7fa2ba000640 1 -- 172.21.15.77:0/3252903131 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa2a8010ac0 con 0x7fa2bc102790 2024-09-10T10:00:20.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.357+0000 7fa2c7222740 1 Processor -- start 2024-09-10T10:00:20.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.358+0000 7fa2c7222740 1 -- start start 2024-09-10T10:00:20.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.358+0000 7fa2c7222740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b80e5f9110 0x55b80e5f7a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:20.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.358+0000 7fa2c7222740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b80e5f7f70 0x55b80e5f83b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:20.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.358+0000 7fa2c7222740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b80e60b410 0x55b80e60d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:20.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.358+0000 7fa2c7222740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55b80e55e3e0 con 0x55b80e5f9110 2024-09-10T10:00:20.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.358+0000 7fa2c7222740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b80e505d80 con 0x55b80e60b410 2024-09-10T10:00:20.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.358+0000 7fa2c7222740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b80e55a3a0 con 0x55b80e5f7f70 2024-09-10T10:00:20.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.359+0000 7fa2c2600640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b80e5f9110 0x55b80e5f7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:20.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.359+0000 7fa2c2600640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b80e5f9110 0x55b80e5f7a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57414/0 (socket says 172.21.15.77:57414) 2024-09-10T10:00:20.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.359+0000 7fa2c2600640 1 -- 172.21.15.77:0/1536370600 learned_addr learned my addr 172.21.15.77:0/1536370600 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:00:20.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.359+0000 7fa2c1200640 1 --2- 172.21.15.77:0/1536370600 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b80e60b410 0x55b80e60d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:20.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.359+0000 7fa2c2600640 1 -- 172.21.15.77:0/1536370600 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b80e5f7f70 msgr2=0x55b80e5f83b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:20.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.359+0000 7fa2c2600640 1 --2- 172.21.15.77:0/1536370600 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b80e5f7f70 0x55b80e5f83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:20.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.359+0000 7fa2c2600640 1 -- 172.21.15.77:0/1536370600 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b80e60b410 msgr2=0x55b80e60d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:20.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.359+0000 7fa2c2600640 1 --2- 172.21.15.77:0/1536370600 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b80e60b410 0x55b80e60d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:20.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.359+0000 7fa2c2600640 1 -- 172.21.15.77:0/1536370600 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b80e524ae0 con 0x55b80e5f9110 2024-09-10T10:00:20.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.359+0000 7fa2c1200640 1 --2- 172.21.15.77:0/1536370600 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b80e60b410 0x55b80e60d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:00:20.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.360+0000 7fa2c2600640 1 --2- 172.21.15.77:0/1536370600 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b80e5f9110 0x55b80e5f7a30 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fa2b4009f40 tx=0x7fa2b4010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfc8a00859e611fe server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:20.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.360+0000 7fa2ba000640 1 --2- 172.21.15.77:0/3252903131 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fa2a40a2c00 0x7fa2a40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:20.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.360+0000 7fa2aea00640 1 -- 172.21.15.77:0/1536370600 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b4003b70 con 0x55b80e5f9110 2024-09-10T10:00:20.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.360+0000 7fa2aea00640 1 -- 172.21.15.77:0/1536370600 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2b4003d10 con 0x55b80e5f9110 2024-09-10T10:00:20.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.360+0000 7fa2aea00640 1 -- 172.21.15.77:0/1536370600 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b4011af0 con 0x55b80e5f9110 2024-09-10T10:00:20.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.360+0000 7fa2ba000640 1 -- 172.21.15.77:0/3252903131 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fa2a80cf780 con 0x7fa2bc102790 2024-09-10T10:00:20.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.360+0000 7fa2c1c00640 1 --2- 172.21.15.77:0/3252903131 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fa2a40a2c00 0x7fa2a40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:20.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.362+0000 7fa2c7222740 1 -- 172.21.15.77:0/1536370600 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b80e5f9110 msgr2=0x55b80e5f7a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:20.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.362+0000 7fa2c1c00640 1 --2- 172.21.15.77:0/3252903131 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fa2a40a2c00 0x7fa2a40a50c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fa2b0000c00 tx=0x7fa2b0005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:20.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.362+0000 7fa2c7222740 1 --2- 172.21.15.77:0/1536370600 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b80e5f9110 0x55b80e5f7a30 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fa2b4009f40 tx=0x7fa2b4010860 comp rx=0 tx=0).stop 2024-09-10T10:00:20.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.362+0000 7fa2c7222740 1 -- 172.21.15.77:0/1536370600 shutdown_connections 2024-09-10T10:00:20.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.362+0000 7fa2c7222740 1 --2- 172.21.15.77:0/1536370600 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b80e60b410 0x55b80e60d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:20.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.362+0000 7fa2c7222740 1 --2- 172.21.15.77:0/1536370600 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b80e5f7f70 0x55b80e5f83b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:20.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.362+0000 7fa2c7222740 1 --2- 172.21.15.77:0/1536370600 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b80e5f9110 0x55b80e5f7a30 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:20.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.362+0000 7fa2c7222740 1 -- 172.21.15.77:0/1536370600 >> 172.21.15.77:0/1536370600 conn(0x55b80e5f4120 msgr2=0x55b80e5f4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:20.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.362+0000 7fa2c7222740 1 -- 172.21.15.77:0/1536370600 shutdown_connections 2024-09-10T10:00:20.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.362+0000 7fa2c7222740 1 -- 172.21.15.77:0/1536370600 wait complete. 2024-09-10T10:00:20.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.362+0000 7fa2c7222740 1 -- start start 2024-09-10T10:00:20.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.363+0000 7fa2c7222740 1 Processor -- start 2024-09-10T10:00:20.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.363+0000 7fa2c7222740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b80e5f7f70 0x55b80e5fb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:20.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.363+0000 7fa2c7222740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b80e5f9110 0x55b80e5fbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:20.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.363+0000 7fa2c7222740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b80e60b410 0x55b80e5fc330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:20.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.363+0000 7fa2c7222740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55b80e60e1e0 con 0x55b80e60b410 2024-09-10T10:00:20.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.363+0000 7fa2c7222740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b80e5190f0 con 0x55b80e5f9110 2024-09-10T10:00:20.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.363+0000 7fa2c7222740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b80e60e360 con 0x55b80e5f7f70 2024-09-10T10:00:20.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.363+0000 7fa2c7222740 10 client.0 _ll_register_callbacks cb 0x55b80e50a9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:00:20.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.363+0000 7fa2c1200640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b80e5f9110 0x55b80e5fbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:20.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.363+0000 7fa2c1200640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b80e5f9110 0x55b80e5fbdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:41124/0 (socket says 172.21.15.77:41124) 2024-09-10T10:00:20.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.364+0000 7fa2c1200640 1 -- 172.21.15.77:0/295799561 learned_addr learned my addr 172.21.15.77:0/295799561 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:00:20.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.364+0000 7fa2c2600640 1 --2- 172.21.15.77:0/295799561 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b80e60b410 0x55b80e5fc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:20.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.364+0000 7fa2c1c00640 1 --2- 172.21.15.77:0/295799561 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b80e5f7f70 0x55b80e5fb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:20.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.364+0000 7fa2c2600640 1 -- 172.21.15.77:0/295799561 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b80e5f7f70 msgr2=0x55b80e5fb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:20.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.364+0000 7fa2c2600640 1 --2- 172.21.15.77:0/295799561 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b80e5f7f70 0x55b80e5fb8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:20.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.364+0000 7fa2c2600640 1 -- 172.21.15.77:0/295799561 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b80e5f9110 msgr2=0x55b80e5fbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:20.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.364+0000 7fa2c2600640 1 --2- 172.21.15.77:0/295799561 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b80e5f9110 0x55b80e5fbdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:20.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.364+0000 7fa2c2600640 1 -- 172.21.15.77:0/295799561 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b80e2cf3c0 con 0x55b80e60b410 2024-09-10T10:00:20.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.364+0000 7fa2c1200640 1 --2- 172.21.15.77:0/295799561 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b80e5f9110 0x55b80e5fbdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:00:20.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.365+0000 7fa2c2600640 1 --2- 172.21.15.77:0/295799561 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b80e60b410 0x55b80e5fc330 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fa2b400a010 tx=0x7fa2b4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63ae00d2dab9a2d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:20.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.365+0000 7fa2a3400640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-10T10:00:20.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.365+0000 7fa2a3400640 1 -- 172.21.15.77:0/295799561 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b4011870 con 0x55b80e60b410 2024-09-10T10:00:20.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.365+0000 7fa2a3400640 1 -- 172.21.15.77:0/295799561 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fa2b4011a10 con 0x55b80e60b410 2024-09-10T10:00:20.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.365+0000 7fa2c7222740 1 -- 172.21.15.77:0/295799561 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2b4003d10 con 0x55b80e60b410 2024-09-10T10:00:20.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.365+0000 7fa2a3400640 1 -- 172.21.15.77:0/295799561 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa2b4011c10 con 0x55b80e60b410 2024-09-10T10:00:20.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.365+0000 7fa2c7222740 10 client.16515 fetch_fsmap learned FSMap version 32 2024-09-10T10:00:20.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.365+0000 7fa2c7222740 1 -- 172.21.15.77:0/295799561 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b80e61a020 con 0x55b80e60b410 2024-09-10T10:00:20.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.366+0000 7fa2a3400640 1 -- 172.21.15.77:0/295799561 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2b4003d10 con 0x55b80e60b410 2024-09-10T10:00:20.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.367+0000 7fa2a3400640 1 -- 172.21.15.77:0/295799561 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fa2b4003ec0 con 0x55b80e60b410 2024-09-10T10:00:20.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.367+0000 7fa2c7222740 10 client.16515 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T10:00:20.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.367+0000 7fa2c7222740 10 client.16515 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T10:00:20.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.367+0000 7fa2c7222740 20 client.16515 populate_metadata read hostname 'smithi077' 2024-09-10T10:00:20.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.367+0000 7fa2c7222740 1 --2- 172.21.15.77:0/295799561 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55b80e61b370 0x55b80e61d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:20.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.367+0000 7fa2c7222740 4 client.16515 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:00:20.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.367+0000 7fa2c7222740 1 -- 172.21.15.77:0/295799561 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b80e56b950 con 0x55b80e61b370 2024-09-10T10:00:20.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.367+0000 7fa2c1c00640 1 --2- 172.21.15.77:0/295799561 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55b80e61b370 0x55b80e61d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:20.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.368+0000 7fa2c1c00640 1 --2- 172.21.15.77:0/295799561 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55b80e61b370 0x55b80e61d760 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c050a42e5b56550d server_cookie=ff657c4ac1f3e53 in_seq=0 out_seq=0 2024-09-10T10:00:20.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.368+0000 7fa2a3400640 10 client.16515 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T10:00:20.370 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.369+0000 7fa2a3400640 1 -- 172.21.15.77:0/295799561 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b80e56b950 con 0x55b80e61b370 2024-09-10T10:00:20.370 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.369+0000 7fa2a3400640 10 client.16515 handle_command_reply: tid=0 2024-09-10T10:00:20.370 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.369+0000 7fa2c7222740 1 client.16515 shutdown 2024-09-10T10:00:20.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.369+0000 7fa2c7222740 1 -- 172.21.15.77:0/295799561 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b80e60b410 msgr2=0x55b80e5fc330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:20.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.369+0000 7fa2c7222740 1 --2- 172.21.15.77:0/295799561 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b80e60b410 0x55b80e5fc330 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fa2b400a010 tx=0x7fa2b4004a40 comp rx=0 tx=0).stop 2024-09-10T10:00:20.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.370+0000 7fa2c7222740 1 -- 172.21.15.77:0/295799561 shutdown_connections 2024-09-10T10:00:20.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.370+0000 7fa2c7222740 1 --2- 172.21.15.77:0/295799561 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55b80e61b370 0x55b80e61d760 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:20.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.370+0000 7fa2c7222740 1 --2- 172.21.15.77:0/295799561 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b80e60b410 0x55b80e5fc330 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:20.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.370+0000 7fa2c7222740 1 --2- 172.21.15.77:0/295799561 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b80e5f9110 0x55b80e5fbdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:20.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.370+0000 7fa2c7222740 1 --2- 172.21.15.77:0/295799561 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b80e5f7f70 0x55b80e5fb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:20.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.370+0000 7fa2c7222740 1 -- 172.21.15.77:0/295799561 >> 172.21.15.77:0/295799561 conn(0x55b80e5f4120 msgr2=0x55b80e5f9e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:20.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.370+0000 7fa2a3400640 0 client.16515 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T10:00:20.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.370+0000 7fa2c7222740 1 -- 172.21.15.77:0/295799561 shutdown_connections 2024-09-10T10:00:20.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.370+0000 7fa2c7222740 1 -- 172.21.15.77:0/295799561 wait complete. 2024-09-10T10:00:20.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.370+0000 7fa2c7222740 20 client.16515 trim_cache size 0 max 16384 2024-09-10T10:00:20.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:00:19 smithi134 ceph-mon[35674]: scrub summary: idle+waiting paths [/] 2024-09-10T10:00:20.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.394+0000 7fa2c7222740 1 Processor -- start 2024-09-10T10:00:20.395 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.394+0000 7fa2c7222740 1 -- start start 2024-09-10T10:00:20.395 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.394+0000 7fa2c7222740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b80e5f7f70 0x55b80e622d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:20.395 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.394+0000 7fa2c7222740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b80e5f9110 0x55b80e6232a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:20.395 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.394+0000 7fa2c7222740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b80e60b410 0x55b80e5fa7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:20.395 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.394+0000 7fa2c7222740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa2bc1acd70 con 0x55b80e60b410 2024-09-10T10:00:20.395 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.394+0000 7fa2c7222740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa2bc1ae970 con 0x55b80e5f9110 2024-09-10T10:00:20.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.394+0000 7fa2c7222740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa2bc1ad440 con 0x55b80e5f7f70 2024-09-10T10:00:20.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.394+0000 7fa2c1c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b80e5f7f70 0x55b80e622d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:20.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.394+0000 7fa2c1c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b80e5f7f70 0x55b80e622d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:33514/0 (socket says 172.21.15.77:33514) 2024-09-10T10:00:20.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.394+0000 7fa2c1c00640 1 -- 172.21.15.77:0/2182014921 learned_addr learned my addr 172.21.15.77:0/2182014921 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:00:20.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.394+0000 7fa2c2600640 1 --2- 172.21.15.77:0/2182014921 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b80e60b410 0x55b80e5fa7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:20.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.394+0000 7fa2c1200640 1 --2- 172.21.15.77:0/2182014921 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b80e5f9110 0x55b80e6232a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:20.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.395+0000 7fa2c1200640 1 -- 172.21.15.77:0/2182014921 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b80e5f7f70 msgr2=0x55b80e622d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:20.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.395+0000 7fa2c1200640 1 --2- 172.21.15.77:0/2182014921 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b80e5f7f70 0x55b80e622d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:20.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.395+0000 7fa2c1200640 1 -- 172.21.15.77:0/2182014921 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b80e60b410 msgr2=0x55b80e5fa7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:20.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.395+0000 7fa2c1200640 1 --2- 172.21.15.77:0/2182014921 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b80e60b410 0x55b80e5fa7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:20.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.395+0000 7fa2c1200640 1 -- 172.21.15.77:0/2182014921 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b80e5f8980 con 0x55b80e5f9110 2024-09-10T10:00:20.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.395+0000 7fa2c1200640 1 --2- 172.21.15.77:0/2182014921 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b80e5f9110 0x55b80e6232a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa2a8098050 tx=0x7fa2a8018420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2304de189d0f63cd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:20.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.396+0000 7fa2ae000640 1 -- 172.21.15.77:0/2182014921 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a809ffa0 con 0x55b80e5f9110 2024-09-10T10:00:20.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.396+0000 7fa2ae000640 1 -- 172.21.15.77:0/2182014921 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2a8086040 con 0x55b80e5f9110 2024-09-10T10:00:20.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.396+0000 7fa2ae000640 1 -- 172.21.15.77:0/2182014921 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a8010430 con 0x55b80e5f9110 2024-09-10T10:00:20.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.397+0000 7fa2c7222740 1 -- 172.21.15.77:0/2182014921 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b80e5f9110 msgr2=0x55b80e6232a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:20.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.397+0000 7fa2c7222740 1 --2- 172.21.15.77:0/2182014921 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b80e5f9110 0x55b80e6232a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa2a8098050 tx=0x7fa2a8018420 comp rx=0 tx=0).stop 2024-09-10T10:00:20.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.397+0000 7fa2c7222740 1 -- 172.21.15.77:0/2182014921 shutdown_connections 2024-09-10T10:00:20.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.397+0000 7fa2c7222740 1 --2- 172.21.15.77:0/2182014921 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b80e60b410 0x55b80e5fa7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:20.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.397+0000 7fa2c7222740 1 --2- 172.21.15.77:0/2182014921 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b80e5f9110 0x55b80e6232a0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:20.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.397+0000 7fa2c7222740 1 --2- 172.21.15.77:0/2182014921 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b80e5f7f70 0x55b80e622d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:20.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.397+0000 7fa2c7222740 1 -- 172.21.15.77:0/2182014921 >> 172.21.15.77:0/2182014921 conn(0x55b80e5f4120 msgr2=0x55b80e5fd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:20.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.397+0000 7fa2c7222740 1 -- 172.21.15.77:0/2182014921 shutdown_connections 2024-09-10T10:00:20.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.398+0000 7fa2c7222740 1 -- 172.21.15.77:0/2182014921 wait complete. 2024-09-10T10:00:20.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.398+0000 7fa2c7222740 1 -- start start 2024-09-10T10:00:20.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.398+0000 7fa2c7222740 1 Processor -- start 2024-09-10T10:00:20.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.399+0000 7fa2c7222740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b80e5f7f70 0x55b80e613ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:20.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.399+0000 7fa2c7222740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b80e5f9110 0x55b80e60ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:20.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.399+0000 7fa2c7222740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b80e60b410 0x55b80e60f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:20.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.399+0000 7fa2c7222740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa2bc1ae1e0 con 0x55b80e5f9110 2024-09-10T10:00:20.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.399+0000 7fa2c7222740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa2bc1adb10 con 0x55b80e5f7f70 2024-09-10T10:00:20.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.399+0000 7fa2c7222740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa2bc1aad50 con 0x55b80e60b410 2024-09-10T10:00:20.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.399+0000 7fa2c7222740 10 client.0 _ll_register_callbacks cb 0x7fa2b40116d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:00:20.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.399+0000 7fa2c2600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b80e60b410 0x55b80e60f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:20.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.399+0000 7fa2c2600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b80e60b410 0x55b80e60f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:33516/0 (socket says 172.21.15.77:33516) 2024-09-10T10:00:20.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.399+0000 7fa2c2600640 1 -- 172.21.15.77:0/273455681 learned_addr learned my addr 172.21.15.77:0/273455681 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:00:20.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.399+0000 7fa2c1c00640 1 --2- 172.21.15.77:0/273455681 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b80e5f7f70 0x55b80e613ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:20.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.400+0000 7fa2c2600640 1 -- 172.21.15.77:0/273455681 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b80e5f7f70 msgr2=0x55b80e613ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:20.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.400+0000 7fa2c2600640 1 --2- 172.21.15.77:0/273455681 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b80e5f7f70 0x55b80e613ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:20.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.400+0000 7fa2c2600640 1 -- 172.21.15.77:0/273455681 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b80e5f9110 msgr2=0x55b80e60ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:20.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.400+0000 7fa2c2600640 1 --2- 172.21.15.77:0/273455681 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b80e5f9110 0x55b80e60ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:20.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.400+0000 7fa2c2600640 1 -- 172.21.15.77:0/273455681 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b80e60f910 con 0x55b80e60b410 2024-09-10T10:00:20.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.401+0000 7fa2c2600640 1 --2- 172.21.15.77:0/273455681 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b80e60b410 0x55b80e60f3a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa2b4009d50 tx=0x7fa2b4010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64969c6f2597b482 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:20.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.401+0000 7fa2a3400640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-10T10:00:20.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.401+0000 7fa2a3400640 1 -- 172.21.15.77:0/273455681 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b4011db0 con 0x55b80e60b410 2024-09-10T10:00:20.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.401+0000 7fa2c7222740 1 -- 172.21.15.77:0/273455681 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2a8086040 con 0x55b80e60b410 2024-09-10T10:00:20.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.401+0000 7fa2a3400640 1 -- 172.21.15.77:0/273455681 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fa2b4022040 con 0x55b80e60b410 2024-09-10T10:00:20.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.402+0000 7fa2a3400640 1 -- 172.21.15.77:0/273455681 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa2b4003f00 con 0x55b80e60b410 2024-09-10T10:00:20.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.402+0000 7fa2c7222740 10 client.26560 fetch_fsmap learned FSMap version 32 2024-09-10T10:00:20.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.402+0000 7fa2c7222740 1 -- 172.21.15.77:0/273455681 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b80e685aa0 con 0x55b80e60b410 2024-09-10T10:00:20.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.402+0000 7fa2a3400640 1 -- 172.21.15.77:0/273455681 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2a8086040 con 0x55b80e60b410 2024-09-10T10:00:20.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.402+0000 7fa2a3400640 1 -- 172.21.15.77:0/273455681 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fa2b4026bf0 con 0x55b80e60b410 2024-09-10T10:00:20.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.403+0000 7fa2c7222740 10 client.26560 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T10:00:20.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.403+0000 7fa2c7222740 10 client.26560 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T10:00:20.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.403+0000 7fa2c7222740 20 client.26560 populate_metadata read hostname 'smithi077' 2024-09-10T10:00:20.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.403+0000 7fa2c7222740 1 --2- 172.21.15.77:0/273455681 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55b80e686180 0x55b80e6885c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:20.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.403+0000 7fa2c7222740 4 client.26560 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T10:00:20.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.403+0000 7fa2c7222740 1 -- 172.21.15.77:0/273455681 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b80e565e60 con 0x55b80e686180 2024-09-10T10:00:20.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.403+0000 7fa2c1c00640 1 --2- 172.21.15.77:0/273455681 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55b80e686180 0x55b80e6885c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:20.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.404+0000 7fa2c1c00640 1 --2- 172.21.15.77:0/273455681 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55b80e686180 0x55b80e6885c0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e44ce4bf574b9886 server_cookie=34ab6b8ef197fda9 in_seq=0 out_seq=0 2024-09-10T10:00:20.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.404+0000 7fa2a3400640 10 client.26560 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T10:00:20.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.404+0000 7fa2a3400640 1 -- 172.21.15.77:0/273455681 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55b80e565e60 con 0x55b80e686180 2024-09-10T10:00:20.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.404+0000 7fa2a3400640 10 client.26560 handle_command_reply: tid=0 2024-09-10T10:00:20.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.404+0000 7fa2c7222740 1 client.26560 shutdown 2024-09-10T10:00:20.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.405+0000 7fa2c7222740 1 -- 172.21.15.77:0/273455681 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b80e60b410 msgr2=0x55b80e60f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:20.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.405+0000 7fa2c7222740 1 --2- 172.21.15.77:0/273455681 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b80e60b410 0x55b80e60f3a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa2b4009d50 tx=0x7fa2b4010860 comp rx=0 tx=0).stop 2024-09-10T10:00:20.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.405+0000 7fa2c7222740 1 -- 172.21.15.77:0/273455681 shutdown_connections 2024-09-10T10:00:20.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.405+0000 7fa2c7222740 1 --2- 172.21.15.77:0/273455681 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55b80e686180 0x55b80e6885c0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:20.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.405+0000 7fa2c7222740 1 --2- 172.21.15.77:0/273455681 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b80e60b410 0x55b80e60f3a0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:20.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.405+0000 7fa2c7222740 1 --2- 172.21.15.77:0/273455681 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b80e5f9110 0x55b80e60ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:20.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.405+0000 7fa2c7222740 1 --2- 172.21.15.77:0/273455681 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b80e5f7f70 0x55b80e613ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:20.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.405+0000 7fa2c7222740 1 -- 172.21.15.77:0/273455681 >> 172.21.15.77:0/273455681 conn(0x55b80e5f4120 msgr2=0x55b80e5fb870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:20.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.405+0000 7fa2a3400640 0 client.26560 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T10:00:20.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.405+0000 7fa2c7222740 1 -- 172.21.15.77:0/273455681 shutdown_connections 2024-09-10T10:00:20.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.405+0000 7fa2c7222740 1 -- 172.21.15.77:0/273455681 wait complete. 2024-09-10T10:00:20.407 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-10T10:00:20.407 INFO:teuthology.orchestra.run.smithi077.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T10:00:20.408 INFO:teuthology.orchestra.run.smithi077.stdout: "scrubs": { 2024-09-10T10:00:20.408 INFO:teuthology.orchestra.run.smithi077.stdout: "46cac69b-0aa5-462b-ad21-21f7004d640b": { 2024-09-10T10:00:20.408 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/", 2024-09-10T10:00:20.408 INFO:teuthology.orchestra.run.smithi077.stdout: "tag": "46cac69b-0aa5-462b-ad21-21f7004d640b", 2024-09-10T10:00:20.408 INFO:teuthology.orchestra.run.smithi077.stdout: "options": "recursive,force" 2024-09-10T10:00:20.408 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:00:20.408 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:00:20.408 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-10T10:00:20.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.405+0000 7fa2c7222740 20 client.26560 trim_cache size 0 max 16384 2024-09-10T10:00:20.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.406+0000 7fa2c4600640 1 -- 172.21.15.77:0/3252903131 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fa2a40a2c00 msgr2=0x7fa2a40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:20.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.406+0000 7fa2c4600640 1 --2- 172.21.15.77:0/3252903131 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fa2a40a2c00 0x7fa2a40a50c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fa2b0000c00 tx=0x7fa2b0005b50 comp rx=0 tx=0).stop 2024-09-10T10:00:20.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.406+0000 7fa2c4600640 1 -- 172.21.15.77:0/3252903131 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa2bc102790 msgr2=0x7fa2bc198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:20.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.406+0000 7fa2c4600640 1 --2- 172.21.15.77:0/3252903131 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa2bc102790 0x7fa2bc198990 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa2a800e9f0 tx=0x7fa2a800eeb0 comp rx=0 tx=0).stop 2024-09-10T10:00:20.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.407+0000 7fa2c4600640 1 -- 172.21.15.77:0/3252903131 shutdown_connections 2024-09-10T10:00:20.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.407+0000 7fa2c4600640 1 --2- 172.21.15.77:0/3252903131 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fa2a40a2c00 0x7fa2a40a50c0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:20.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.407+0000 7fa2c4600640 1 --2- 172.21.15.77:0/3252903131 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa2bc106f10 0x7fa2bc19cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:20.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.407+0000 7fa2c4600640 1 --2- 172.21.15.77:0/3252903131 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa2bc102790 0x7fa2bc198990 secure :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa2a800e9f0 tx=0x7fa2a800eeb0 comp rx=0 tx=0).stop 2024-09-10T10:00:20.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.407+0000 7fa2c4600640 1 --2- 172.21.15.77:0/3252903131 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2bc101de0 0x7fa2bc198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:20.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.407+0000 7fa2c4600640 1 -- 172.21.15.77:0/3252903131 >> 172.21.15.77:0/3252903131 conn(0x7fa2bc0fd880 msgr2=0x7fa2bc0fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:20.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.407+0000 7fa2c4600640 1 -- 172.21.15.77:0/3252903131 shutdown_connections 2024-09-10T10:00:20.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:20.407+0000 7fa2c4600640 1 -- 172.21.15.77:0/3252903131 wait complete. 2024-09-10T10:00:20.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:00:19 smithi120 ceph-mon[34770]: scrub summary: idle+waiting paths [/] 2024-09-10T10:00:21.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:00:20 smithi077 ceph-mon[30319]: pgmap v546: 129 pgs: 129 active+clean; 104 MiB data, 996 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.2 MiB/s wr, 411 op/s 2024-09-10T10:00:21.282 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:00:20 smithi134 ceph-mon[35674]: pgmap v546: 129 pgs: 129 active+clean; 104 MiB data, 996 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.2 MiB/s wr, 411 op/s 2024-09-10T10:00:21.351 INFO:tasks.cephfs.filesystem:scrub status for tag:46cac69b-0aa5-462b-ad21-21f7004d640b - {'path': '/', 'tag': '46cac69b-0aa5-462b-ad21-21f7004d640b', 'options': 'recursive,force'} 2024-09-10T10:00:21.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:00:20 smithi120 ceph-mon[34770]: pgmap v546: 129 pgs: 129 active+clean; 104 MiB data, 996 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.2 MiB/s wr, 411 op/s 2024-09-10T10:00:23.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:00:22 smithi134 ceph-mon[35674]: pgmap v547: 129 pgs: 129 active+clean; 104 MiB data, 1009 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.0 MiB/s wr, 377 op/s 2024-09-10T10:00:23.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:00:22 smithi120 ceph-mon[34770]: pgmap v547: 129 pgs: 129 active+clean; 104 MiB data, 1009 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.0 MiB/s wr, 377 op/s 2024-09-10T10:00:23.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:00:22 smithi077 ceph-mon[30319]: pgmap v547: 129 pgs: 129 active+clean; 104 MiB data, 1009 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.0 MiB/s wr, 377 op/s 2024-09-10T10:00:25.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:00:24 smithi134 ceph-mon[35674]: pgmap v548: 129 pgs: 129 active+clean; 106 MiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 2.5 MiB/s wr, 496 op/s 2024-09-10T10:00:25.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:00:24 smithi120 ceph-mon[34770]: pgmap v548: 129 pgs: 129 active+clean; 106 MiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 2.5 MiB/s wr, 496 op/s 2024-09-10T10:00:25.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:00:24 smithi077 ceph-mon[30319]: pgmap v548: 129 pgs: 129 active+clean; 106 MiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 2.5 MiB/s wr, 496 op/s 2024-09-10T10:00:27.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:00:26 smithi134 ceph-mon[35674]: pgmap v549: 129 pgs: 129 active+clean; 105 MiB data, 996 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 2.1 MiB/s wr, 423 op/s 2024-09-10T10:00:27.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:00:26 smithi120 ceph-mon[34770]: pgmap v549: 129 pgs: 129 active+clean; 105 MiB data, 996 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 2.1 MiB/s wr, 423 op/s 2024-09-10T10:00:27.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:00:26 smithi077 ceph-mon[30319]: pgmap v549: 129 pgs: 129 active+clean; 105 MiB data, 996 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 2.1 MiB/s wr, 423 op/s 2024-09-10T10:00:29.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:00:28 smithi134 ceph-mon[35674]: pgmap v550: 129 pgs: 129 active+clean; 105 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 2.2 MiB/s wr, 434 op/s 2024-09-10T10:00:29.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:00:28 smithi120 ceph-mon[34770]: pgmap v550: 129 pgs: 129 active+clean; 105 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 2.2 MiB/s wr, 434 op/s 2024-09-10T10:00:29.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:00:28 smithi077 ceph-mon[30319]: pgmap v550: 129 pgs: 129 active+clean; 105 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 2.2 MiB/s wr, 434 op/s 2024-09-10T10:00:31.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:00:30 smithi134 ceph-mon[35674]: pgmap v551: 129 pgs: 129 active+clean; 105 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 3.0 MiB/s wr, 574 op/s 2024-09-10T10:00:31.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:00:30 smithi134 ceph-mon[35674]: scrub summary: idle 2024-09-10T10:00:31.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:00:30 smithi134 ceph-mon[35674]: scrub summary: idle 2024-09-10T10:00:31.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:00:30 smithi120 ceph-mon[34770]: pgmap v551: 129 pgs: 129 active+clean; 105 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 3.0 MiB/s wr, 574 op/s 2024-09-10T10:00:31.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:00:30 smithi120 ceph-mon[34770]: scrub summary: idle 2024-09-10T10:00:31.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:00:30 smithi120 ceph-mon[34770]: scrub summary: idle 2024-09-10T10:00:31.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:00:30 smithi077 ceph-mon[30319]: pgmap v551: 129 pgs: 129 active+clean; 105 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 3.0 MiB/s wr, 574 op/s 2024-09-10T10:00:31.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:00:30 smithi077 ceph-mon[30319]: scrub summary: idle 2024-09-10T10:00:31.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:00:30 smithi077 ceph-mon[30319]: scrub summary: idle 2024-09-10T10:00:32.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:00:31 smithi134 ceph-mon[35674]: scrub summary: idle 2024-09-10T10:00:32.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:00:31 smithi134 ceph-mon[35674]: scrub summary: idle 2024-09-10T10:00:32.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:00:31 smithi134 ceph-mon[35674]: scrub summary: idle 2024-09-10T10:00:32.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:00:31 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:00:32.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:00:31 smithi120 ceph-mon[34770]: scrub summary: idle 2024-09-10T10:00:32.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:00:31 smithi120 ceph-mon[34770]: scrub summary: idle 2024-09-10T10:00:32.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:00:31 smithi120 ceph-mon[34770]: scrub summary: idle 2024-09-10T10:00:32.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:00:31 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:00:32.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:00:31 smithi077 ceph-mon[30319]: scrub summary: idle 2024-09-10T10:00:32.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:00:31 smithi077 ceph-mon[30319]: scrub summary: idle 2024-09-10T10:00:32.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:00:31 smithi077 ceph-mon[30319]: scrub summary: idle 2024-09-10T10:00:32.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:00:31 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:00:33.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:00:32 smithi134 ceph-mon[35674]: pgmap v552: 129 pgs: 129 active+clean; 105 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 2.3 MiB/s wr, 431 op/s 2024-09-10T10:00:33.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:00:32 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:00:33.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:00:32 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:00:33.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:00:32 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T10:00:33.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:00:32 smithi120 ceph-mon[34770]: pgmap v552: 129 pgs: 129 active+clean; 105 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 2.3 MiB/s wr, 431 op/s 2024-09-10T10:00:33.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:00:32 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:00:33.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:00:32 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:00:33.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:00:32 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T10:00:33.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:00:32 smithi077 ceph-mon[30319]: pgmap v552: 129 pgs: 129 active+clean; 105 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 2.3 MiB/s wr, 431 op/s 2024-09-10T10:00:33.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:00:32 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:00:33.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:00:32 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:00:33.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:00:32 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T10:00:34.195 INFO:tasks.workunit.client.0.smithi077.stdout:copying to largest ever: 0x9ff773 2024-09-10T10:00:34.866 INFO:tasks.workunit.client.0.smithi077.stdout:20000 punch from 0x34ddcb to 0x35324f, (0x5484 bytes) 2024-09-10T10:00:35.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:00:34 smithi134 ceph-mon[35674]: pgmap v553: 129 pgs: 129 active+clean; 111 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 2.7 MiB/s wr, 514 op/s 2024-09-10T10:00:35.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:00:34 smithi120 ceph-mon[34770]: pgmap v553: 129 pgs: 129 active+clean; 111 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 2.7 MiB/s wr, 514 op/s 2024-09-10T10:00:35.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:00:34 smithi077 ceph-mon[30319]: pgmap v553: 129 pgs: 129 active+clean; 111 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 2.7 MiB/s wr, 514 op/s 2024-09-10T10:00:37.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:00:36 smithi134 ceph-mon[35674]: pgmap v554: 129 pgs: 129 active+clean; 111 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 2.5 MiB/s wr, 441 op/s 2024-09-10T10:00:37.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:00:36 smithi120 ceph-mon[34770]: pgmap v554: 129 pgs: 129 active+clean; 111 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 2.5 MiB/s wr, 441 op/s 2024-09-10T10:00:37.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:00:36 smithi077 ceph-mon[30319]: pgmap v554: 129 pgs: 129 active+clean; 111 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 2.5 MiB/s wr, 441 op/s 2024-09-10T10:00:38.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:00:37 smithi134 ceph-mon[35674]: pgmap v555: 129 pgs: 129 active+clean; 110 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 2.3 MiB/s wr, 411 op/s 2024-09-10T10:00:38.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:00:37 smithi120 ceph-mon[34770]: pgmap v555: 129 pgs: 129 active+clean; 110 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 2.3 MiB/s wr, 411 op/s 2024-09-10T10:00:38.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:00:37 smithi077 ceph-mon[30319]: pgmap v555: 129 pgs: 129 active+clean; 110 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 2.3 MiB/s wr, 411 op/s 2024-09-10T10:00:41.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:00:40 smithi134 ceph-mon[35674]: pgmap v556: 129 pgs: 129 active+clean; 107 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 3.0 MiB/s wr, 547 op/s 2024-09-10T10:00:41.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:00:40 smithi120 ceph-mon[34770]: pgmap v556: 129 pgs: 129 active+clean; 107 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 3.0 MiB/s wr, 547 op/s 2024-09-10T10:00:41.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:00:40 smithi077 ceph-mon[30319]: pgmap v556: 129 pgs: 129 active+clean; 107 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 3.0 MiB/s wr, 547 op/s 2024-09-10T10:00:43.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:00:42 smithi134 ceph-mon[35674]: pgmap v557: 129 pgs: 129 active+clean; 108 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 2.2 MiB/s wr, 409 op/s 2024-09-10T10:00:43.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:00:42 smithi120 ceph-mon[34770]: pgmap v557: 129 pgs: 129 active+clean; 108 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 2.2 MiB/s wr, 409 op/s 2024-09-10T10:00:43.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:00:42 smithi077 ceph-mon[30319]: pgmap v557: 129 pgs: 129 active+clean; 108 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 2.2 MiB/s wr, 409 op/s 2024-09-10T10:00:45.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:00:44 smithi134 ceph-mon[35674]: pgmap v558: 129 pgs: 129 active+clean; 109 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 2.5 MiB/s wr, 491 op/s 2024-09-10T10:00:45.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:00:44 smithi120 ceph-mon[34770]: pgmap v558: 129 pgs: 129 active+clean; 109 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 2.5 MiB/s wr, 491 op/s 2024-09-10T10:00:45.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:00:44 smithi077 ceph-mon[30319]: pgmap v558: 129 pgs: 129 active+clean; 109 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 2.5 MiB/s wr, 491 op/s 2024-09-10T10:00:47.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:00:46 smithi120 ceph-mon[34770]: pgmap v559: 129 pgs: 129 active+clean; 112 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 2.4 MiB/s wr, 424 op/s 2024-09-10T10:00:47.192 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:00:46 smithi077 ceph-mon[30319]: pgmap v559: 129 pgs: 129 active+clean; 112 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 2.4 MiB/s wr, 424 op/s 2024-09-10T10:00:47.217 INFO:tasks.workunit.client.0.smithi077.stdout:30000 trunc from 0x7a5444 to 0x143a5 2024-09-10T10:00:47.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:00:46 smithi134 ceph-mon[35674]: pgmap v559: 129 pgs: 129 active+clean; 112 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 2.4 MiB/s wr, 424 op/s 2024-09-10T10:00:49.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:00:48 smithi120 ceph-mon[34770]: pgmap v560: 129 pgs: 129 active+clean; 111 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 2.2 MiB/s wr, 404 op/s 2024-09-10T10:00:49.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:00:48 smithi077 ceph-mon[30319]: pgmap v560: 129 pgs: 129 active+clean; 111 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 2.2 MiB/s wr, 404 op/s 2024-09-10T10:00:49.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:00:48 smithi134 ceph-mon[35674]: pgmap v560: 129 pgs: 129 active+clean; 111 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 2.2 MiB/s wr, 404 op/s 2024-09-10T10:00:51.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:00:50 smithi120 ceph-mon[34770]: pgmap v561: 129 pgs: 129 active+clean; 115 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 2.9 MiB/s wr, 530 op/s 2024-09-10T10:00:51.182 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:00:50 smithi134 ceph-mon[35674]: pgmap v561: 129 pgs: 129 active+clean; 115 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 2.9 MiB/s wr, 530 op/s 2024-09-10T10:00:51.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:00:50 smithi077 ceph-mon[30319]: pgmap v561: 129 pgs: 129 active+clean; 115 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 2.9 MiB/s wr, 530 op/s 2024-09-10T10:00:51.352 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T10:00:51.820 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:00:52.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.493+0000 7f350f600640 1 -- 172.21.15.77:0/3843872060 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3508103ff0 msgr2=0x7f35081043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:52.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.493+0000 7f350f600640 1 --2- 172.21.15.77:0/3843872060 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3508103ff0 0x7f35081043d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f34fc009900 tx=0x7f34fc02a3f0 comp rx=0 tx=0).stop 2024-09-10T10:00:52.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.493+0000 7f350f600640 1 -- 172.21.15.77:0/3843872060 shutdown_connections 2024-09-10T10:00:52.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.493+0000 7f350f600640 1 --2- 172.21.15.77:0/3843872060 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3508109120 0x7f3508110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:52.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.493+0000 7f350f600640 1 --2- 172.21.15.77:0/3843872060 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35081049a0 0x7f35081089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:52.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.493+0000 7f350f600640 1 --2- 172.21.15.77:0/3843872060 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3508103ff0 0x7f35081043d0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:52.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.493+0000 7f350f600640 1 -- 172.21.15.77:0/3843872060 >> 172.21.15.77:0/3843872060 conn(0x7f35080ffa90 msgr2=0x7f3508101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:52.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.493+0000 7f350f600640 1 -- 172.21.15.77:0/3843872060 shutdown_connections 2024-09-10T10:00:52.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.494+0000 7f350f600640 1 -- 172.21.15.77:0/3843872060 wait complete. 2024-09-10T10:00:52.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.494+0000 7f350f600640 1 Processor -- start 2024-09-10T10:00:52.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.495+0000 7f350f600640 1 -- start start 2024-09-10T10:00:52.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.495+0000 7f350f600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3508103ff0 0x7f350819a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:52.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.495+0000 7f350f600640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35081049a0 0x7f350819abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:52.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.495+0000 7f350f600640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3508109120 0x7f350819ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:52.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.495+0000 7f350f600640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3508114170 con 0x7f35081049a0 2024-09-10T10:00:52.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.495+0000 7f350f600640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3508113ff0 con 0x7f3508109120 2024-09-10T10:00:52.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.495+0000 7f350f600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f35081142f0 con 0x7f3508103ff0 2024-09-10T10:00:52.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.495+0000 7f3507e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35081049a0 0x7f350819abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:52.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.495+0000 7f3507e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35081049a0 0x7f350819abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34154/0 (socket says 172.21.15.77:34154) 2024-09-10T10:00:52.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.495+0000 7f3507e00640 1 -- 172.21.15.77:0/1753920494 learned_addr learned my addr 172.21.15.77:0/1753920494 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:00:52.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.495+0000 7f350d600640 1 --2- 172.21.15.77:0/1753920494 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3508109120 0x7f350819ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:52.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.496+0000 7f350cc00640 1 --2- 172.21.15.77:0/1753920494 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3508103ff0 0x7f350819a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:52.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.496+0000 7f350d600640 1 -- 172.21.15.77:0/1753920494 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3508103ff0 msgr2=0x7f350819a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:52.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.496+0000 7f350d600640 1 --2- 172.21.15.77:0/1753920494 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3508103ff0 0x7f350819a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:52.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.496+0000 7f350d600640 1 -- 172.21.15.77:0/1753920494 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35081049a0 msgr2=0x7f350819abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:52.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.496+0000 7f350d600640 1 --2- 172.21.15.77:0/1753920494 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35081049a0 0x7f350819abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:52.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.496+0000 7f350d600640 1 -- 172.21.15.77:0/1753920494 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f350819f620 con 0x7f3508109120 2024-09-10T10:00:52.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.496+0000 7f3507e00640 1 --2- 172.21.15.77:0/1753920494 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35081049a0 0x7f350819abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:00:52.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.496+0000 7f350d600640 1 --2- 172.21.15.77:0/1753920494 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3508109120 0x7f350819ef40 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3500009fd0 tx=0x7f350000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:52.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.497+0000 7f3505600640 1 -- 172.21.15.77:0/1753920494 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f350000ce20 con 0x7f3508109120 2024-09-10T10:00:52.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.497+0000 7f3505600640 1 -- 172.21.15.77:0/1753920494 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3500005020 con 0x7f3508109120 2024-09-10T10:00:52.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.497+0000 7f3505600640 1 -- 172.21.15.77:0/1753920494 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3500010720 con 0x7f3508109120 2024-09-10T10:00:52.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.497+0000 7f350f600640 1 -- 172.21.15.77:0/1753920494 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f350819f910 con 0x7f3508109120 2024-09-10T10:00:52.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.497+0000 7f350f600640 1 -- 172.21.15.77:0/1753920494 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35081a71f0 con 0x7f3508109120 2024-09-10T10:00:52.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.500+0000 7f3505600640 1 -- 172.21.15.77:0/1753920494 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3500005540 con 0x7f3508109120 2024-09-10T10:00:52.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.502+0000 7f3505600640 1 --2- 172.21.15.77:0/1753920494 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f34f00a2c00 0x7f34f00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:52.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.502+0000 7f3512225740 1 Processor -- start 2024-09-10T10:00:52.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.503+0000 7f3505600640 1 -- 172.21.15.77:0/1753920494 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f35000cfa40 con 0x7f3508109120 2024-09-10T10:00:52.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.503+0000 7f350cc00640 1 --2- 172.21.15.77:0/1753920494 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f34f00a2c00 0x7f34f00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:52.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.503+0000 7f3512225740 1 -- start start 2024-09-10T10:00:52.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.504+0000 7f3512225740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d88d247b90 0x55d88d247f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:52.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.504+0000 7f3512225740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d88d2484b0 0x55d88d252830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:52.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.504+0000 7f350cc00640 1 --2- 172.21.15.77:0/1753920494 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f34f00a2c00 0x7f34f00a50c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f34fc000c00 tx=0x7f34fc005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:52.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.504+0000 7f3512225740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d88d252d70 0x55d88d255160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:52.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.504+0000 7f3512225740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d88d1ae3e0 con 0x55d88d252d70 2024-09-10T10:00:52.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.504+0000 7f3512225740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55d88d155d80 con 0x55d88d2484b0 2024-09-10T10:00:52.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.504+0000 7f3512225740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d88d1aa3a0 con 0x55d88d247b90 2024-09-10T10:00:52.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.504+0000 7f3507e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d88d2484b0 0x55d88d252830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:52.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.504+0000 7f350cc00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d88d247b90 0x55d88d247f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:52.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.504+0000 7f3507e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d88d2484b0 0x55d88d252830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:35966/0 (socket says 172.21.15.77:35966) 2024-09-10T10:00:52.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.504+0000 7f3507e00640 1 -- 172.21.15.77:0/4082472135 learned_addr learned my addr 172.21.15.77:0/4082472135 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:00:52.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.505+0000 7f3507e00640 1 -- 172.21.15.77:0/4082472135 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d88d247b90 msgr2=0x55d88d247f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:52.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.505+0000 7f3507e00640 1 --2- 172.21.15.77:0/4082472135 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d88d247b90 0x55d88d247f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:52.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.505+0000 7f3507e00640 1 -- 172.21.15.77:0/4082472135 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d88d252d70 msgr2=0x55d88d255160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:52.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.505+0000 7f3507e00640 1 --2- 172.21.15.77:0/4082472135 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d88d252d70 0x55d88d255160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:52.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.505+0000 7f3507e00640 1 -- 172.21.15.77:0/4082472135 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d88d174ae0 con 0x55d88d2484b0 2024-09-10T10:00:52.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.505+0000 7f3507e00640 1 --2- 172.21.15.77:0/4082472135 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d88d2484b0 0x55d88d252830 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f34f40115f0 tx=0x7f34f4011ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68fe45ae772a23e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:52.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.505+0000 7f34f9c00640 1 -- 172.21.15.77:0/4082472135 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f401e070 con 0x55d88d2484b0 2024-09-10T10:00:52.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.505+0000 7f34f9c00640 1 -- 172.21.15.77:0/4082472135 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34f400f660 con 0x55d88d2484b0 2024-09-10T10:00:52.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.505+0000 7f34f9c00640 1 -- 172.21.15.77:0/4082472135 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f4003870 con 0x55d88d2484b0 2024-09-10T10:00:52.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.506+0000 7f3512225740 1 -- 172.21.15.77:0/4082472135 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d88d2484b0 msgr2=0x55d88d252830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:52.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.506+0000 7f3512225740 1 --2- 172.21.15.77:0/4082472135 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d88d2484b0 0x55d88d252830 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f34f40115f0 tx=0x7f34f4011ab0 comp rx=0 tx=0).stop 2024-09-10T10:00:52.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.506+0000 7f3512225740 1 -- 172.21.15.77:0/4082472135 shutdown_connections 2024-09-10T10:00:52.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.506+0000 7f3512225740 1 --2- 172.21.15.77:0/4082472135 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d88d252d70 0x55d88d255160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:52.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.506+0000 7f3512225740 1 --2- 172.21.15.77:0/4082472135 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d88d2484b0 0x55d88d252830 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:52.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.506+0000 7f3512225740 1 --2- 172.21.15.77:0/4082472135 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d88d247b90 0x55d88d247f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:52.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.506+0000 7f3512225740 1 -- 172.21.15.77:0/4082472135 >> 172.21.15.77:0/4082472135 conn(0x55d88d244120 msgr2=0x55d88d244500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:52.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.507+0000 7f3512225740 1 -- 172.21.15.77:0/4082472135 shutdown_connections 2024-09-10T10:00:52.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.507+0000 7f3512225740 1 -- 172.21.15.77:0/4082472135 wait complete. 2024-09-10T10:00:52.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.507+0000 7f3512225740 1 -- start start 2024-09-10T10:00:52.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.508+0000 7f3512225740 1 Processor -- start 2024-09-10T10:00:52.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.508+0000 7f3512225740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d88d247b90 0x55d88d24fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:52.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.508+0000 7f3512225740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d88d2484b0 0x55d88d2502b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:52.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.508+0000 7f3512225740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d88d252d70 0x55d88d2507f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:52.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.508+0000 7f3512225740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d88d2559e0 con 0x55d88d252d70 2024-09-10T10:00:52.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.508+0000 7f3512225740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55d88d1690f0 con 0x55d88d2484b0 2024-09-10T10:00:52.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.508+0000 7f3512225740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d88d255b60 con 0x55d88d247b90 2024-09-10T10:00:52.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.509+0000 7f3512225740 10 client.0 _ll_register_callbacks cb 0x55d88d15a9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:00:52.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.509+0000 7f3507e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d88d2484b0 0x55d88d2502b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:52.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.509+0000 7f3507e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d88d2484b0 0x55d88d2502b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:35982/0 (socket says 172.21.15.77:35982) 2024-09-10T10:00:52.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.509+0000 7f3507e00640 1 -- 172.21.15.77:0/1780768673 learned_addr learned my addr 172.21.15.77:0/1780768673 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:00:52.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.509+0000 7f350cc00640 1 --2- 172.21.15.77:0/1780768673 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d88d247b90 0x55d88d24fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:52.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.509+0000 7f350d600640 1 --2- 172.21.15.77:0/1780768673 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d88d252d70 0x55d88d2507f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:52.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.509+0000 7f3507e00640 1 -- 172.21.15.77:0/1780768673 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d88d247b90 msgr2=0x55d88d24fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:52.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.509+0000 7f3507e00640 1 --2- 172.21.15.77:0/1780768673 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d88d247b90 0x55d88d24fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:52.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.509+0000 7f3507e00640 1 -- 172.21.15.77:0/1780768673 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d88d252d70 msgr2=0x55d88d2507f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:52.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.509+0000 7f3507e00640 1 --2- 172.21.15.77:0/1780768673 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d88d252d70 0x55d88d2507f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:52.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.509+0000 7f3507e00640 1 -- 172.21.15.77:0/1780768673 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d88cf1f3c0 con 0x55d88d2484b0 2024-09-10T10:00:52.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.509+0000 7f350d600640 1 --2- 172.21.15.77:0/1780768673 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d88d252d70 0x55d88d2507f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:00:52.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.510+0000 7f3507e00640 1 --2- 172.21.15.77:0/1780768673 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d88d2484b0 0x55d88d2502b0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f34f40052a0 tx=0x7f34f40052d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3226ae044a73fde server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:52.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.510+0000 7f34ee000640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T10:00:52.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.511+0000 7f34ee000640 1 -- 172.21.15.77:0/1780768673 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f401e040 con 0x55d88d2484b0 2024-09-10T10:00:52.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.511+0000 7f34ee000640 1 -- 172.21.15.77:0/1780768673 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f34f40054f0 con 0x55d88d2484b0 2024-09-10T10:00:52.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.511+0000 7f3512225740 1 -- 172.21.15.77:0/1780768673 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34f400f660 con 0x55d88d2484b0 2024-09-10T10:00:52.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.511+0000 7f34ee000640 1 -- 172.21.15.77:0/1780768673 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f34f4030050 con 0x55d88d2484b0 2024-09-10T10:00:52.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.511+0000 7f3512225740 10 client.26621 fetch_fsmap learned FSMap version 32 2024-09-10T10:00:52.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.511+0000 7f3512225740 1 -- 172.21.15.77:0/1780768673 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d88d2617b0 con 0x55d88d2484b0 2024-09-10T10:00:52.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.511+0000 7f34ee000640 1 -- 172.21.15.77:0/1780768673 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34f400f660 con 0x55d88d2484b0 2024-09-10T10:00:52.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.511+0000 7f34ee000640 1 -- 172.21.15.77:0/1780768673 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f34f4021070 con 0x55d88d2484b0 2024-09-10T10:00:52.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.512+0000 7f3512225740 10 client.26621 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T10:00:52.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.512+0000 7f3512225740 10 client.26621 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T10:00:52.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.512+0000 7f3512225740 20 client.26621 populate_metadata read hostname 'smithi077' 2024-09-10T10:00:52.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.512+0000 7f3512225740 1 --2- 172.21.15.77:0/1780768673 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55d88d262b60 0x55d88d264f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:52.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.512+0000 7f3512225740 4 client.26621 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:00:52.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.512+0000 7f3512225740 1 -- 172.21.15.77:0/1780768673 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d88d1bb950 con 0x55d88d262b60 2024-09-10T10:00:52.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.512+0000 7f350cc00640 1 --2- 172.21.15.77:0/1780768673 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55d88d262b60 0x55d88d264f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:52.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.513+0000 7f350cc00640 1 --2- 172.21.15.77:0/1780768673 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55d88d262b60 0x55d88d264f50 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=676b5874d34ce180 server_cookie=9d4f11eaef08dbd5 in_seq=0 out_seq=0 2024-09-10T10:00:52.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.513+0000 7f34ee000640 10 client.26621 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T10:00:52.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.513+0000 7f34ee000640 1 -- 172.21.15.77:0/1780768673 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d88d1bb950 con 0x55d88d262b60 2024-09-10T10:00:52.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.513+0000 7f34ee000640 10 client.26621 handle_command_reply: tid=0 2024-09-10T10:00:52.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.513+0000 7f3512225740 1 client.26621 shutdown 2024-09-10T10:00:52.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.514+0000 7f3512225740 1 -- 172.21.15.77:0/1780768673 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d88d2484b0 msgr2=0x55d88d2502b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:52.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.514+0000 7f3512225740 1 --2- 172.21.15.77:0/1780768673 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d88d2484b0 0x55d88d2502b0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f34f40052a0 tx=0x7f34f40052d0 comp rx=0 tx=0).stop 2024-09-10T10:00:52.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.514+0000 7f3512225740 1 -- 172.21.15.77:0/1780768673 shutdown_connections 2024-09-10T10:00:52.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.514+0000 7f3512225740 1 --2- 172.21.15.77:0/1780768673 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55d88d262b60 0x55d88d264f50 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:52.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.514+0000 7f3512225740 1 --2- 172.21.15.77:0/1780768673 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d88d252d70 0x55d88d2507f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:52.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.514+0000 7f3512225740 1 --2- 172.21.15.77:0/1780768673 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d88d2484b0 0x55d88d2502b0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:52.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.514+0000 7f3512225740 1 --2- 172.21.15.77:0/1780768673 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d88d247b90 0x55d88d24fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:52.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.514+0000 7f3512225740 1 -- 172.21.15.77:0/1780768673 >> 172.21.15.77:0/1780768673 conn(0x55d88d244120 msgr2=0x55d88d249e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:52.517 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.515+0000 7f34ee000640 0 client.26621 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T10:00:52.517 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.515+0000 7f3512225740 1 -- 172.21.15.77:0/1780768673 shutdown_connections 2024-09-10T10:00:52.517 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.515+0000 7f3512225740 1 -- 172.21.15.77:0/1780768673 wait complete. 2024-09-10T10:00:52.517 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.515+0000 7f3512225740 20 client.26621 trim_cache size 0 max 16384 2024-09-10T10:00:52.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.537+0000 7f3512225740 1 Processor -- start 2024-09-10T10:00:52.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.538+0000 7f3512225740 1 -- start start 2024-09-10T10:00:52.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.538+0000 7f3512225740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d88d244120 0x55d88d26a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:52.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.538+0000 7f3512225740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d88d247b90 0x55d88d26ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:52.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.538+0000 7f3512225740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d88d2484b0 0x55d88d24cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:52.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.538+0000 7f3512225740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f35081b1120 con 0x55d88d247b90 2024-09-10T10:00:52.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.538+0000 7f3512225740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f35081b2d20 con 0x55d88d244120 2024-09-10T10:00:52.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.538+0000 7f3512225740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f35081b17f0 con 0x55d88d2484b0 2024-09-10T10:00:52.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.538+0000 7f350cc00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d88d244120 0x55d88d26a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:52.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.538+0000 7f350d600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d88d2484b0 0x55d88d24cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:52.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.538+0000 7f350cc00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d88d244120 0x55d88d26a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:35984/0 (socket says 172.21.15.77:35984) 2024-09-10T10:00:52.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.538+0000 7f350cc00640 1 -- 172.21.15.77:0/823248238 learned_addr learned my addr 172.21.15.77:0/823248238 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:00:52.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.539+0000 7f350cc00640 1 -- 172.21.15.77:0/823248238 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d88d2484b0 msgr2=0x55d88d24cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:52.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.539+0000 7f350cc00640 1 --2- 172.21.15.77:0/823248238 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d88d2484b0 0x55d88d24cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:52.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.539+0000 7f350cc00640 1 -- 172.21.15.77:0/823248238 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d88d247b90 msgr2=0x55d88d26ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:52.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.539+0000 7f350cc00640 1 --2- 172.21.15.77:0/823248238 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d88d247b90 0x55d88d26ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:52.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.539+0000 7f350cc00640 1 -- 172.21.15.77:0/823248238 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d88d24ed20 con 0x55d88d244120 2024-09-10T10:00:52.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.539+0000 7f350cc00640 1 --2- 172.21.15.77:0/823248238 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d88d244120 0x55d88d26a720 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f34fc054fd0 tx=0x7f34fc040000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22a1781da32477d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:52.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.540+0000 7f34f9200640 1 -- 172.21.15.77:0/823248238 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34fc035040 con 0x55d88d244120 2024-09-10T10:00:52.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.540+0000 7f34f9200640 1 -- 172.21.15.77:0/823248238 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34fc05a070 con 0x55d88d244120 2024-09-10T10:00:52.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.540+0000 7f34f9200640 1 -- 172.21.15.77:0/823248238 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34fc032d40 con 0x55d88d244120 2024-09-10T10:00:52.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.541+0000 7f3512225740 1 -- 172.21.15.77:0/823248238 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d88d244120 msgr2=0x55d88d26a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:52.543 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.541+0000 7f3512225740 1 --2- 172.21.15.77:0/823248238 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d88d244120 0x55d88d26a720 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f34fc054fd0 tx=0x7f34fc040000 comp rx=0 tx=0).stop 2024-09-10T10:00:52.543 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.541+0000 7f3512225740 1 -- 172.21.15.77:0/823248238 shutdown_connections 2024-09-10T10:00:52.543 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.541+0000 7f3512225740 1 --2- 172.21.15.77:0/823248238 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d88d2484b0 0x55d88d24cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:52.543 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.541+0000 7f3512225740 1 --2- 172.21.15.77:0/823248238 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d88d247b90 0x55d88d26ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:52.543 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.541+0000 7f3512225740 1 --2- 172.21.15.77:0/823248238 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d88d244120 0x55d88d26a720 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:52.543 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.541+0000 7f3512225740 1 -- 172.21.15.77:0/823248238 >> 172.21.15.77:0/823248238 conn(0x55d88d250950 msgr2=0x55d88d24fd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:52.543 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.541+0000 7f3512225740 1 -- 172.21.15.77:0/823248238 shutdown_connections 2024-09-10T10:00:52.543 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.541+0000 7f3512225740 1 -- 172.21.15.77:0/823248238 wait complete. 2024-09-10T10:00:52.543 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.542+0000 7f3512225740 1 -- start start 2024-09-10T10:00:52.543 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.542+0000 7f3512225740 1 Processor -- start 2024-09-10T10:00:52.544 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.543+0000 7f3512225740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d88d244120 0x55d88d256b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:52.544 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.543+0000 7f3512225740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d88d247b90 0x55d88d257090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:52.544 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.543+0000 7f3512225740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d88d2484b0 0x55d88d2575d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:52.544 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.543+0000 7f3512225740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f35081b2590 con 0x55d88d244120 2024-09-10T10:00:52.544 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.543+0000 7f3512225740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f35081b1ec0 con 0x55d88d2484b0 2024-09-10T10:00:52.544 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.543+0000 7f3512225740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f35081af100 con 0x55d88d247b90 2024-09-10T10:00:52.544 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.543+0000 7f3512225740 10 client.0 _ll_register_callbacks cb 0x7f34f4011440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:00:52.545 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.543+0000 7f3507e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d88d247b90 0x55d88d257090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:52.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.543+0000 7f3507e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d88d247b90 0x55d88d257090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:41526/0 (socket says 172.21.15.77:41526) 2024-09-10T10:00:52.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.543+0000 7f350d600640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d88d2484b0 0x55d88d2575d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:52.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.543+0000 7f3507e00640 1 -- 172.21.15.77:0/1999154289 learned_addr learned my addr 172.21.15.77:0/1999154289 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:00:52.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.544+0000 7f350cc00640 1 --2- 172.21.15.77:0/1999154289 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d88d244120 0x55d88d256b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:52.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.544+0000 7f3507e00640 1 -- 172.21.15.77:0/1999154289 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d88d2484b0 msgr2=0x55d88d2575d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:52.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.544+0000 7f3507e00640 1 --2- 172.21.15.77:0/1999154289 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d88d2484b0 0x55d88d2575d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:52.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.544+0000 7f3507e00640 1 -- 172.21.15.77:0/1999154289 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d88d244120 msgr2=0x55d88d256b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:52.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.544+0000 7f3507e00640 1 --2- 172.21.15.77:0/1999154289 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d88d244120 0x55d88d256b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:52.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.544+0000 7f3507e00640 1 -- 172.21.15.77:0/1999154289 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d88d2cc770 con 0x55d88d247b90 2024-09-10T10:00:52.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.544+0000 7f3507e00640 1 --2- 172.21.15.77:0/1999154289 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d88d247b90 0x55d88d257090 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f34f4005270 tx=0x7f34f4005d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f07a382c0e262e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:52.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.544+0000 7f34ee000640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-10T10:00:52.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.544+0000 7f34ee000640 1 -- 172.21.15.77:0/1999154289 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f401e040 con 0x55d88d247b90 2024-09-10T10:00:52.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.544+0000 7f3512225740 1 -- 172.21.15.77:0/1999154289 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34fc05a070 con 0x55d88d247b90 2024-09-10T10:00:52.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.544+0000 7f34ee000640 1 -- 172.21.15.77:0/1999154289 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f34f4019dc0 con 0x55d88d247b90 2024-09-10T10:00:52.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.545+0000 7f34ee000640 1 -- 172.21.15.77:0/1999154289 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f34f403d050 con 0x55d88d247b90 2024-09-10T10:00:52.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.545+0000 7f3512225740 10 client.26578 fetch_fsmap learned FSMap version 32 2024-09-10T10:00:52.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.545+0000 7f3512225740 1 -- 172.21.15.77:0/1999154289 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d88d2cda20 con 0x55d88d247b90 2024-09-10T10:00:52.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.545+0000 7f34ee000640 1 -- 172.21.15.77:0/1999154289 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34fc05a070 con 0x55d88d247b90 2024-09-10T10:00:52.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.545+0000 7f34ee000640 1 -- 172.21.15.77:0/1999154289 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f34f4021070 con 0x55d88d247b90 2024-09-10T10:00:52.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.545+0000 7f3512225740 10 client.26578 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T10:00:52.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.545+0000 7f3512225740 10 client.26578 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T10:00:52.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.545+0000 7f3512225740 20 client.26578 populate_metadata read hostname 'smithi077' 2024-09-10T10:00:52.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.545+0000 7f3512225740 1 --2- 172.21.15.77:0/1999154289 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55d88d252d70 0x55d88d2d02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:52.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.545+0000 7f3512225740 4 client.26578 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T10:00:52.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.545+0000 7f3512225740 1 -- 172.21.15.77:0/1999154289 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d88d1b5e60 con 0x55d88d252d70 2024-09-10T10:00:52.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.546+0000 7f350cc00640 1 --2- 172.21.15.77:0/1999154289 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55d88d252d70 0x55d88d2d02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:52.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.547+0000 7f350cc00640 1 --2- 172.21.15.77:0/1999154289 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55d88d252d70 0x55d88d2d02a0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f8b9b8d4ef523e2 server_cookie=46e33efac6ed7ed in_seq=0 out_seq=0 2024-09-10T10:00:52.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.547+0000 7f34ee000640 10 client.26578 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T10:00:52.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.547+0000 7f34ee000640 1 -- 172.21.15.77:0/1999154289 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55d88d1b5e60 con 0x55d88d252d70 2024-09-10T10:00:52.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.547+0000 7f34ee000640 10 client.26578 handle_command_reply: tid=0 2024-09-10T10:00:52.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.547+0000 7f3512225740 1 client.26578 shutdown 2024-09-10T10:00:52.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.548+0000 7f3512225740 1 -- 172.21.15.77:0/1999154289 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d88d247b90 msgr2=0x55d88d257090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:52.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.548+0000 7f3512225740 1 --2- 172.21.15.77:0/1999154289 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d88d247b90 0x55d88d257090 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f34f4005270 tx=0x7f34f4005d30 comp rx=0 tx=0).stop 2024-09-10T10:00:52.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.548+0000 7f3512225740 1 -- 172.21.15.77:0/1999154289 shutdown_connections 2024-09-10T10:00:52.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.548+0000 7f3512225740 1 --2- 172.21.15.77:0/1999154289 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55d88d252d70 0x55d88d2d02a0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:52.550 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.548+0000 7f3512225740 1 --2- 172.21.15.77:0/1999154289 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d88d2484b0 0x55d88d2575d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:52.550 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.548+0000 7f34ee000640 0 client.26578 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T10:00:52.550 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.548+0000 7f3512225740 1 --2- 172.21.15.77:0/1999154289 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d88d247b90 0x55d88d257090 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:52.550 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.548+0000 7f3512225740 1 --2- 172.21.15.77:0/1999154289 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d88d244120 0x55d88d256b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:52.550 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.548+0000 7f3512225740 1 -- 172.21.15.77:0/1999154289 >> 172.21.15.77:0/1999154289 conn(0x55d88d250950 msgr2=0x55d88d2515d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:52.550 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.549+0000 7f3512225740 1 -- 172.21.15.77:0/1999154289 shutdown_connections 2024-09-10T10:00:52.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.549+0000 7f3512225740 1 -- 172.21.15.77:0/1999154289 wait complete. 2024-09-10T10:00:52.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.549+0000 7f3512225740 20 client.26578 trim_cache size 0 max 16384 2024-09-10T10:00:52.551 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-10T10:00:52.551 INFO:teuthology.orchestra.run.smithi077.stdout: "status": "no active scrubs running", 2024-09-10T10:00:52.551 INFO:teuthology.orchestra.run.smithi077.stdout: "scrubs": {} 2024-09-10T10:00:52.551 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-10T10:00:52.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.550+0000 7f350f600640 1 -- 172.21.15.77:0/1753920494 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f34f00a2c00 msgr2=0x7f34f00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:52.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.550+0000 7f350f600640 1 --2- 172.21.15.77:0/1753920494 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f34f00a2c00 0x7f34f00a50c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f34fc000c00 tx=0x7f34fc005b50 comp rx=0 tx=0).stop 2024-09-10T10:00:52.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.550+0000 7f350f600640 1 -- 172.21.15.77:0/1753920494 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3508109120 msgr2=0x7f350819ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:52.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.550+0000 7f350f600640 1 --2- 172.21.15.77:0/1753920494 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3508109120 0x7f350819ef40 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3500009fd0 tx=0x7f350000ef10 comp rx=0 tx=0).stop 2024-09-10T10:00:52.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.551+0000 7f350f600640 1 -- 172.21.15.77:0/1753920494 shutdown_connections 2024-09-10T10:00:52.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.551+0000 7f350f600640 1 --2- 172.21.15.77:0/1753920494 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f34f00a2c00 0x7f34f00a50c0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:52.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.551+0000 7f350f600640 1 --2- 172.21.15.77:0/1753920494 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3508109120 0x7f350819ef40 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:52.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.551+0000 7f350f600640 1 --2- 172.21.15.77:0/1753920494 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35081049a0 0x7f350819abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:52.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.551+0000 7f350f600640 1 --2- 172.21.15.77:0/1753920494 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3508103ff0 0x7f350819a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:52.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.551+0000 7f350f600640 1 -- 172.21.15.77:0/1753920494 >> 172.21.15.77:0/1753920494 conn(0x7f35080ffa90 msgr2=0x7f3508100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:52.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.551+0000 7f350f600640 1 -- 172.21.15.77:0/1753920494 shutdown_connections 2024-09-10T10:00:52.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:52.551+0000 7f350f600640 1 -- 172.21.15.77:0/1753920494 wait complete. 2024-09-10T10:00:53.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:00:52 smithi120 ceph-mon[34770]: pgmap v562: 129 pgs: 129 active+clean; 115 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 2.2 MiB/s wr, 387 op/s 2024-09-10T10:00:53.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:00:52 smithi077 ceph-mon[30319]: pgmap v562: 129 pgs: 129 active+clean; 115 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 2.2 MiB/s wr, 387 op/s 2024-09-10T10:00:53.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:00:52 smithi134 ceph-mon[35674]: pgmap v562: 129 pgs: 129 active+clean; 115 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 2.2 MiB/s wr, 387 op/s 2024-09-10T10:00:53.857 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T10:00:53.857 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:00:54.327 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:00:54.367 INFO:tasks.workunit.client.0.smithi077.stdout:truncating to largest ever: 0x9ffb01 2024-09-10T10:00:55.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:55.101+0000 7f68e2800640 1 -- 172.21.15.77:0/2433656914 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f68dc101de0 msgr2=0x7f68dc1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:55.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:55.101+0000 7f68e2800640 1 --2- 172.21.15.77:0/2433656914 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f68dc101de0 0x7f68dc1021c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f68cc009900 tx=0x7f68cc02a3f0 comp rx=0 tx=0).stop 2024-09-10T10:00:55.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:55.102+0000 7f68e2800640 1 -- 172.21.15.77:0/2433656914 shutdown_connections 2024-09-10T10:00:55.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:55.102+0000 7f68e2800640 1 --2- 172.21.15.77:0/2433656914 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f68dc106f10 0x7f68dc10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:55.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:55.102+0000 7f68e2800640 1 --2- 172.21.15.77:0/2433656914 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68dc102790 0x7f68dc1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:55.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:55.102+0000 7f68e2800640 1 --2- 172.21.15.77:0/2433656914 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f68dc101de0 0x7f68dc1021c0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:55.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:55.102+0000 7f68e2800640 1 -- 172.21.15.77:0/2433656914 >> 172.21.15.77:0/2433656914 conn(0x7f68dc0fd8c0 msgr2=0x7f68dc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:55.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:55.103+0000 7f68e2800640 1 -- 172.21.15.77:0/2433656914 shutdown_connections 2024-09-10T10:00:55.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:55.105+0000 7f68e2800640 1 -- 172.21.15.77:0/2433656914 wait complete. 2024-09-10T10:00:55.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:55.106+0000 7f68e2800640 1 Processor -- start 2024-09-10T10:00:55.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:55.106+0000 7f68e2800640 1 -- start start 2024-09-10T10:00:55.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:55.106+0000 7f68e2800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68dc101de0 0x7f68dc10e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:55.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:55.106+0000 7f68e2800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f68dc102790 0x7f68dc10ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:55.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:55.106+0000 7f68e2800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f68dc106f10 0x7f68dc10b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:55.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:55.106+0000 7f68e2800640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f68dc075ab0 con 0x7f68dc102790 2024-09-10T10:00:55.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:55.106+0000 7f68e2800640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f68dc075930 con 0x7f68dc106f10 2024-09-10T10:00:55.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:55.106+0000 7f68e2800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f68dc075c30 con 0x7f68dc101de0 2024-09-10T10:00:55.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:55.107+0000 7f68dbe00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f68dc106f10 0x7f68dc10b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:55.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:55.107+0000 7f68dbe00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f68dc106f10 0x7f68dc10b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:56570/0 (socket says 172.21.15.77:56570) 2024-09-10T10:00:55.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:55.107+0000 7f68dbe00640 1 -- 172.21.15.77:0/2751056343 learned_addr learned my addr 172.21.15.77:0/2751056343 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:00:55.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:55.107+0000 7f68db400640 1 --2- 172.21.15.77:0/2751056343 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68dc101de0 0x7f68dc10e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:55.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:55.107+0000 7f68daa00640 1 --2- 172.21.15.77:0/2751056343 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f68dc102790 0x7f68dc10ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:55.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:55.107+0000 7f68dbe00640 1 -- 172.21.15.77:0/2751056343 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68dc101de0 msgr2=0x7f68dc10e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:55.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:55.107+0000 7f68dbe00640 1 --2- 172.21.15.77:0/2751056343 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68dc101de0 0x7f68dc10e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:55.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:55.107+0000 7f68dbe00640 1 -- 172.21.15.77:0/2751056343 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f68dc102790 msgr2=0x7f68dc10ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:55.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:55.107+0000 7f68dbe00640 1 --2- 172.21.15.77:0/2751056343 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f68dc102790 0x7f68dc10ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:55.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:55.107+0000 7f68dbe00640 1 -- 172.21.15.77:0/2751056343 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68dc10c0e0 con 0x7f68dc106f10 2024-09-10T10:00:55.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:55.107+0000 7f68daa00640 1 --2- 172.21.15.77:0/2751056343 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f68dc102790 0x7f68dc10ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:00:55.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:55.108+0000 7f68dbe00640 1 --2- 172.21.15.77:0/2751056343 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f68dc106f10 0x7f68dc10b820 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f68d400efd0 tx=0x7f68d400c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:55.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:55.108+0000 7f68d3e00640 1 -- 172.21.15.77:0/2751056343 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d4003b70 con 0x7f68dc106f10 2024-09-10T10:00:55.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:55.108+0000 7f68d3e00640 1 -- 172.21.15.77:0/2751056343 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68d4003d10 con 0x7f68dc106f10 2024-09-10T10:00:55.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:55.108+0000 7f68d3e00640 1 -- 172.21.15.77:0/2751056343 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d40109f0 con 0x7f68dc106f10 2024-09-10T10:00:55.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:55.108+0000 7f68e2800640 1 -- 172.21.15.77:0/2751056343 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68dc10c3d0 con 0x7f68dc106f10 2024-09-10T10:00:55.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:55.108+0000 7f68e2800640 1 -- 172.21.15.77:0/2751056343 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68dc1ab4f0 con 0x7f68dc106f10 2024-09-10T10:00:55.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:55.109+0000 7f68e2800640 1 -- 172.21.15.77:0/2751056343 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68bc005290 con 0x7f68dc106f10 2024-09-10T10:00:55.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:55.111+0000 7f68d3e00640 1 -- 172.21.15.77:0/2751056343 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f68d4010b90 con 0x7f68dc106f10 2024-09-10T10:00:55.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:55.113+0000 7f68d3e00640 1 --2- 172.21.15.77:0/2751056343 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f68c00a2c00 0x7f68c00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:55.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:55.113+0000 7f68d3e00640 1 -- 172.21.15.77:0/2751056343 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f68d40cf650 con 0x7f68dc106f10 2024-09-10T10:00:55.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:55.114+0000 7f68db400640 1 --2- 172.21.15.77:0/2751056343 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f68c00a2c00 0x7f68c00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:55.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:55.114+0000 7f68db400640 1 --2- 172.21.15.77:0/2751056343 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f68c00a2c00 0x7f68c00a50c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f68cc000c00 tx=0x7f68cc005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:55.117 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:55.117+0000 7f68d3e00640 1 -- 172.21.15.77:0/2751056343 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f68d409c1c0 con 0x7f68dc106f10 2024-09-10T10:00:55.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:00:54 smithi120 ceph-mon[34770]: pgmap v563: 129 pgs: 129 active+clean; 112 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 2.5 MiB/s wr, 446 op/s 2024-09-10T10:00:55.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:00:54 smithi077 ceph-mon[30319]: pgmap v563: 129 pgs: 129 active+clean; 112 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 2.5 MiB/s wr, 446 op/s 2024-09-10T10:00:55.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:00:54 smithi134 ceph-mon[35674]: pgmap v563: 129 pgs: 129 active+clean; 112 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 2.5 MiB/s wr, 446 op/s 2024-09-10T10:00:55.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:55.399+0000 7f68e2800640 1 -- 172.21.15.77:0/2751056343 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f68bc005580 con 0x7f68dc106f10 2024-09-10T10:00:55.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:55.400+0000 7f68d3e00640 1 -- 172.21.15.77:0/2751056343 <== mon.2 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11996 (secure 0 0 0) 0x7f68d409bac0 con 0x7f68dc106f10 2024-09-10T10:00:55.401 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T10:00:55.402 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":32,"btime":"2024-09-10T09:57:40:888996+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3159289675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3159289675},{"type":"v1","addr":"172.21.15.77:6835","nonce":3159289675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1951962341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1951962341},{"type":"v1","addr":"172.21.15.77:6837","nonce":1951962341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:52:23.368252+0000","modified":"2024-09-10T09:57:40.695191+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24563,"mds_1":24524,"mds_2":24520,"mds_3":24502,"mds_4":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14487":{"gid":14487,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/4291121888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":4291121888},{"type":"v1","addr":"172.21.15.77:6839","nonce":4291121888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14499":{"gid":14499,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6841/445284194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":445284194},{"type":"v1","addr":"172.21.15.77:6841","nonce":445284194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1005054423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1005054423},{"type":"v1","addr":"172.21.15.134:6833","nonce":1005054423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6833/3716307744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":3716307744},{"type":"v1","addr":"172.21.15.120:6833","nonce":3716307744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/1319892544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1319892544},{"type":"v1","addr":"172.21.15.120:6835","nonce":1319892544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.134:6835/3293933699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3293933699},{"type":"v1","addr":"172.21.15.134:6835","nonce":3293933699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.134:6837/2461497091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2461497091},{"type":"v1","addr":"172.21.15.134:6837","nonce":2461497091}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":20,"addr":"172.21.15.134:6839/1965117680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1965117680},{"type":"v1","addr":"172.21.15.134:6839","nonce":1965117680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":20,"addr":"172.21.15.120:6837/3033280709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3033280709},{"type":"v1","addr":"172.21.15.120:6837","nonce":3033280709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.120:6839/1777710157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1777710157},{"type":"v1","addr":"172.21.15.120:6839","nonce":1777710157}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24563,"qdb_cluster":[24563,24524,24520,24502,24496]},"id":1}]} 2024-09-10T10:00:55.402 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 32 2024-09-10T10:00:55.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:55.403+0000 7f68e2800640 1 -- 172.21.15.77:0/2751056343 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f68c00a2c00 msgr2=0x7f68c00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:55.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:55.403+0000 7f68e2800640 1 --2- 172.21.15.77:0/2751056343 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f68c00a2c00 0x7f68c00a50c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f68cc000c00 tx=0x7f68cc005b50 comp rx=0 tx=0).stop 2024-09-10T10:00:55.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:55.403+0000 7f68e2800640 1 -- 172.21.15.77:0/2751056343 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f68dc106f10 msgr2=0x7f68dc10b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:55.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:55.403+0000 7f68e2800640 1 --2- 172.21.15.77:0/2751056343 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f68dc106f10 0x7f68dc10b820 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f68d400efd0 tx=0x7f68d400c6a0 comp rx=0 tx=0).stop 2024-09-10T10:00:55.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:55.403+0000 7f68e2800640 1 -- 172.21.15.77:0/2751056343 shutdown_connections 2024-09-10T10:00:55.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:55.403+0000 7f68e2800640 1 --2- 172.21.15.77:0/2751056343 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f68c00a2c00 0x7f68c00a50c0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:55.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:55.403+0000 7f68e2800640 1 --2- 172.21.15.77:0/2751056343 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f68dc106f10 0x7f68dc10b820 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:55.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:55.403+0000 7f68e2800640 1 --2- 172.21.15.77:0/2751056343 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f68dc102790 0x7f68dc10ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:55.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:55.403+0000 7f68e2800640 1 --2- 172.21.15.77:0/2751056343 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68dc101de0 0x7f68dc10e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:55.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:55.403+0000 7f68e2800640 1 -- 172.21.15.77:0/2751056343 >> 172.21.15.77:0/2751056343 conn(0x7f68dc0fd8c0 msgr2=0x7f68dc0fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:55.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:55.403+0000 7f68e2800640 1 -- 172.21.15.77:0/2751056343 shutdown_connections 2024-09-10T10:00:55.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:55.403+0000 7f68e2800640 1 -- 172.21.15.77:0/2751056343 wait complete. 2024-09-10T10:00:55.942 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:00:55 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/2751056343' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:00:56.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:00:55 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/2751056343' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:00:56.334 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T10:00:56.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:00:55 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/2751056343' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:00:56.795 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:00:57.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:00:56 smithi120 ceph-mon[34770]: pgmap v564: 129 pgs: 129 active+clean; 114 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 2.3 MiB/s wr, 377 op/s 2024-09-10T10:00:57.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:00:56 smithi120 ceph-mon[34770]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T10:00:57.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:00:56 smithi077 ceph-mon[30319]: pgmap v564: 129 pgs: 129 active+clean; 114 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 2.3 MiB/s wr, 377 op/s 2024-09-10T10:00:57.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:00:56 smithi077 ceph-mon[30319]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T10:00:57.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:00:56 smithi134 ceph-mon[35674]: pgmap v564: 129 pgs: 129 active+clean; 114 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 2.3 MiB/s wr, 377 op/s 2024-09-10T10:00:57.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:00:56 smithi134 ceph-mon[35674]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T10:00:57.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.387+0000 7fabbaa00640 1 -- 172.21.15.77:0/1984027207 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fabb4103e10 msgr2=0x7fabb41041f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:57.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.387+0000 7fabbaa00640 1 --2- 172.21.15.77:0/1984027207 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fabb4103e10 0x7fabb41041f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7faba8009900 tx=0x7faba802a3f0 comp rx=0 tx=0).stop 2024-09-10T10:00:57.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.388+0000 7fabbaa00640 1 -- 172.21.15.77:0/1984027207 shutdown_connections 2024-09-10T10:00:57.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.388+0000 7fabbaa00640 1 --2- 172.21.15.77:0/1984027207 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fabb4108f40 0x7fabb4110ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:57.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.388+0000 7fabbaa00640 1 --2- 172.21.15.77:0/1984027207 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fabb41047c0 0x7fabb4108810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:57.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.388+0000 7fabbaa00640 1 --2- 172.21.15.77:0/1984027207 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fabb4103e10 0x7fabb41041f0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:57.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.388+0000 7fabbaa00640 1 -- 172.21.15.77:0/1984027207 >> 172.21.15.77:0/1984027207 conn(0x7fabb40ff8b0 msgr2=0x7fabb4101cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:57.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.388+0000 7fabbaa00640 1 -- 172.21.15.77:0/1984027207 shutdown_connections 2024-09-10T10:00:57.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.388+0000 7fabbaa00640 1 -- 172.21.15.77:0/1984027207 wait complete. 2024-09-10T10:00:57.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.389+0000 7fabbaa00640 1 Processor -- start 2024-09-10T10:00:57.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.389+0000 7fabbaa00640 1 -- start start 2024-09-10T10:00:57.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.389+0000 7fabbaa00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fabb4103e10 0x7fabb4110960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:57.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.389+0000 7fabbaa00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fabb41047c0 0x7fabb410da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:57.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.389+0000 7fabbaa00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fabb4108f40 0x7fabb410df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:57.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.389+0000 7fabbaa00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fabb4113fb0 con 0x7fabb4103e10 2024-09-10T10:00:57.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.389+0000 7fabbaa00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fabb4113e30 con 0x7fabb41047c0 2024-09-10T10:00:57.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.389+0000 7fabbaa00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fabb4114130 con 0x7fabb4108f40 2024-09-10T10:00:57.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.390+0000 7fabba000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fabb4108f40 0x7fabb410df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:57.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.390+0000 7fabba000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fabb4108f40 0x7fabb410df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:47366/0 (socket says 172.21.15.77:47366) 2024-09-10T10:00:57.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.390+0000 7fabba000640 1 -- 172.21.15.77:0/977781414 learned_addr learned my addr 172.21.15.77:0/977781414 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:00:57.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.390+0000 7fabb8c00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fabb41047c0 0x7fabb410da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:57.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.390+0000 7fabb9600640 1 --2- 172.21.15.77:0/977781414 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fabb4103e10 0x7fabb4110960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:57.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.390+0000 7fabba000640 1 -- 172.21.15.77:0/977781414 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fabb41047c0 msgr2=0x7fabb410da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:57.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.390+0000 7fabba000640 1 --2- 172.21.15.77:0/977781414 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fabb41047c0 0x7fabb410da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:57.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.390+0000 7fabba000640 1 -- 172.21.15.77:0/977781414 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fabb4103e10 msgr2=0x7fabb4110960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:57.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.390+0000 7fabba000640 1 --2- 172.21.15.77:0/977781414 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fabb4103e10 0x7fabb4110960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:57.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.390+0000 7fabba000640 1 -- 172.21.15.77:0/977781414 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabb410e810 con 0x7fabb4108f40 2024-09-10T10:00:57.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.391+0000 7fabba000640 1 --2- 172.21.15.77:0/977781414 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fabb4108f40 0x7fabb410df50 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fabac00eb10 tx=0x7fabac00c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:57.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.391+0000 7fabb2000640 1 -- 172.21.15.77:0/977781414 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabac004b20 con 0x7fabb4108f40 2024-09-10T10:00:57.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.391+0000 7fabb2000640 1 -- 172.21.15.77:0/977781414 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabac004cc0 con 0x7fabb4108f40 2024-09-10T10:00:57.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.391+0000 7fabb2000640 1 -- 172.21.15.77:0/977781414 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabac010760 con 0x7fabb4108f40 2024-09-10T10:00:57.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.391+0000 7fabbaa00640 1 -- 172.21.15.77:0/977781414 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabb41a05d0 con 0x7fabb4108f40 2024-09-10T10:00:57.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.391+0000 7fabbaa00640 1 -- 172.21.15.77:0/977781414 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabb41a0b10 con 0x7fabb4108f40 2024-09-10T10:00:57.395 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.394+0000 7fabb2000640 1 -- 172.21.15.77:0/977781414 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fabac005020 con 0x7fabb4108f40 2024-09-10T10:00:57.395 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.395+0000 7fabbec22740 1 Processor -- start 2024-09-10T10:00:57.395 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.395+0000 7fabbec22740 1 -- start start 2024-09-10T10:00:57.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.395+0000 7fabbec22740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560463555e40 0x560463556220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:57.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.395+0000 7fabbec22740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560463556760 0x560463560ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:57.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.395+0000 7fabbec22740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560463561020 0x560463563410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:57.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.395+0000 7fabbec22740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5604634bc3e0 con 0x560463561020 2024-09-10T10:00:57.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.395+0000 7fabbec22740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x560463463d80 con 0x560463555e40 2024-09-10T10:00:57.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.395+0000 7fabbec22740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5604634b83a0 con 0x560463556760 2024-09-10T10:00:57.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.395+0000 7fabb2000640 1 --2- 172.21.15.77:0/977781414 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fab9c0a2c00 0x7fab9c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:57.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.396+0000 7fabb9600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560463556760 0x560463560ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:57.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.396+0000 7fabb9600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560463556760 0x560463560ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:47374/0 (socket says 172.21.15.77:47374) 2024-09-10T10:00:57.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.396+0000 7fabb9600640 1 -- 172.21.15.77:0/565017522 learned_addr learned my addr 172.21.15.77:0/565017522 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:00:57.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.396+0000 7fabb8c00640 1 --2- 172.21.15.77:0/565017522 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560463561020 0x560463563410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:57.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.396+0000 7fabb2000640 1 -- 172.21.15.77:0/977781414 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fabac0cf920 con 0x7fabb4108f40 2024-09-10T10:00:57.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.396+0000 7fabba000640 1 --2- 172.21.15.77:0/977781414 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fab9c0a2c00 0x7fab9c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:57.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.396+0000 7fabb9600640 1 -- 172.21.15.77:0/565017522 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560463555e40 msgr2=0x560463556220 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T10:00:57.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.396+0000 7fabb9600640 1 --2- 172.21.15.77:0/565017522 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560463555e40 0x560463556220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:57.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.396+0000 7fabb9600640 1 -- 172.21.15.77:0/565017522 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560463561020 msgr2=0x560463563410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:57.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.397+0000 7fabba000640 1 --2- 172.21.15.77:0/977781414 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fab9c0a2c00 0x7fab9c0a50c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fabac0a4470 tx=0x7fabac00efd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:57.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.397+0000 7fabb9600640 1 --2- 172.21.15.77:0/565017522 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560463561020 0x560463563410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:57.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.397+0000 7fabb9600640 1 -- 172.21.15.77:0/565017522 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560463482ae0 con 0x560463556760 2024-09-10T10:00:57.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.397+0000 7fabb8c00640 1 --2- 172.21.15.77:0/565017522 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560463561020 0x560463563410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:00:57.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.397+0000 7fabb9600640 1 --2- 172.21.15.77:0/565017522 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560463556760 0x560463560ae0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7faba8009900 tx=0x7faba8005de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ebd70d3637f037a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:57.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.397+0000 7faba6600640 1 -- 172.21.15.77:0/565017522 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba8035040 con 0x560463556760 2024-09-10T10:00:57.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.398+0000 7faba6600640 1 -- 172.21.15.77:0/565017522 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faba8004400 con 0x560463556760 2024-09-10T10:00:57.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.398+0000 7faba6600640 1 -- 172.21.15.77:0/565017522 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba80035f0 con 0x560463556760 2024-09-10T10:00:57.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.399+0000 7fabbec22740 1 -- 172.21.15.77:0/565017522 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560463556760 msgr2=0x560463560ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:57.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.399+0000 7fabbec22740 1 --2- 172.21.15.77:0/565017522 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560463556760 0x560463560ae0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7faba8009900 tx=0x7faba8005de0 comp rx=0 tx=0).stop 2024-09-10T10:00:57.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.399+0000 7fabbec22740 1 -- 172.21.15.77:0/565017522 shutdown_connections 2024-09-10T10:00:57.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.399+0000 7fabbec22740 1 --2- 172.21.15.77:0/565017522 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560463561020 0x560463563410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:57.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.399+0000 7fabbec22740 1 --2- 172.21.15.77:0/565017522 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560463556760 0x560463560ae0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:57.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.399+0000 7fabbec22740 1 --2- 172.21.15.77:0/565017522 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560463555e40 0x560463556220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:57.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.399+0000 7fabbec22740 1 -- 172.21.15.77:0/565017522 >> 172.21.15.77:0/565017522 conn(0x5604635523d0 msgr2=0x5604635527b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:57.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.399+0000 7fabbec22740 1 -- 172.21.15.77:0/565017522 shutdown_connections 2024-09-10T10:00:57.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.399+0000 7fabbec22740 1 -- 172.21.15.77:0/565017522 wait complete. 2024-09-10T10:00:57.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.399+0000 7fabbec22740 1 -- start start 2024-09-10T10:00:57.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.400+0000 7fabbec22740 1 Processor -- start 2024-09-10T10:00:57.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.400+0000 7fabbec22740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560463555e40 0x56046355e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:57.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.400+0000 7fabbec22740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560463556760 0x56046355e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:57.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.400+0000 7fabbec22740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560463561020 0x56046355eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:57.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.400+0000 7fabbec22740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560463563bf0 con 0x560463556760 2024-09-10T10:00:57.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.400+0000 7fabbec22740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5604634770f0 con 0x560463555e40 2024-09-10T10:00:57.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.400+0000 7fabbec22740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560463563d70 con 0x560463561020 2024-09-10T10:00:57.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.400+0000 7fabbec22740 10 client.0 _ll_register_callbacks cb 0x56046345ebd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:00:57.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.401+0000 7fabb8c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560463561020 0x56046355eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:57.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.401+0000 7fabb8c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560463561020 0x56046355eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:47380/0 (socket says 172.21.15.77:47380) 2024-09-10T10:00:57.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.401+0000 7fabb8c00640 1 -- 172.21.15.77:0/4152761123 learned_addr learned my addr 172.21.15.77:0/4152761123 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:00:57.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.401+0000 7fabb8c00640 1 --2- 172.21.15.77:0/4152761123 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560463555e40 0x56046355e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:57.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.401+0000 7fabb8c00640 1 -- 172.21.15.77:0/4152761123 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560463555e40 msgr2=0x56046355e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:57.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.401+0000 7fabb8c00640 1 --2- 172.21.15.77:0/4152761123 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560463555e40 0x56046355e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:57.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.401+0000 7fabb8c00640 1 -- 172.21.15.77:0/4152761123 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560463556760 msgr2=0x56046355e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:57.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.401+0000 7fabb8c00640 1 --2- 172.21.15.77:0/4152761123 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560463556760 0x56046355e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:57.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.401+0000 7fabb8c00640 1 -- 172.21.15.77:0/4152761123 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56046322d3c0 con 0x560463561020 2024-09-10T10:00:57.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.401+0000 7fabb8c00640 1 --2- 172.21.15.77:0/4152761123 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560463561020 0x56046355eac0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7faba0010e90 tx=0x7faba0004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8b0025cf4191084 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:57.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.401+0000 7fab9aa00640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-10T10:00:57.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.402+0000 7fab9aa00640 1 -- 172.21.15.77:0/4152761123 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba0019040 con 0x560463561020 2024-09-10T10:00:57.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.402+0000 7fabbec22740 1 -- 172.21.15.77:0/4152761123 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faba8004400 con 0x560463561020 2024-09-10T10:00:57.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.402+0000 7fab9aa00640 1 -- 172.21.15.77:0/4152761123 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7faba0005020 con 0x560463561020 2024-09-10T10:00:57.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.402+0000 7fab9aa00640 1 -- 172.21.15.77:0/4152761123 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7faba0026050 con 0x560463561020 2024-09-10T10:00:57.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.402+0000 7fabbec22740 10 client.26596 fetch_fsmap learned FSMap version 33 2024-09-10T10:00:57.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.402+0000 7fabbec22740 1 -- 172.21.15.77:0/4152761123 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56046356fe00 con 0x560463561020 2024-09-10T10:00:57.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.402+0000 7fab9aa00640 1 -- 172.21.15.77:0/4152761123 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faba8004400 con 0x560463561020 2024-09-10T10:00:57.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.403+0000 7fab9aa00640 1 -- 172.21.15.77:0/4152761123 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7faba0005220 con 0x560463561020 2024-09-10T10:00:57.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.403+0000 7fabbec22740 10 client.26596 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T10:00:57.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.404+0000 7fabbec22740 10 client.26596 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-10T10:00:57.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.404+0000 7fabbec22740 20 client.26596 populate_metadata read hostname 'smithi077' 2024-09-10T10:00:57.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.404+0000 7fabbec22740 1 --2- 172.21.15.77:0/4152761123 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x560463571150 0x560463573540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:57.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.404+0000 7fabbec22740 4 client.26596 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:00:57.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.404+0000 7fabbec22740 1 -- 172.21.15.77:0/4152761123 --> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5604634c9950 con 0x560463571150 2024-09-10T10:00:57.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.404+0000 7fabb9600640 1 --2- 172.21.15.77:0/4152761123 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x560463571150 0x560463573540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:57.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.406+0000 7fabb9600640 1 --2- 172.21.15.77:0/4152761123 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x560463571150 0x560463573540 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9dea6746189d5b68 server_cookie=497217189ccd8884 in_seq=0 out_seq=0 2024-09-10T10:00:57.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.406+0000 7fab9aa00640 10 client.26596 ms_handle_connect on v2:172.21.15.134:6834/3293933699 2024-09-10T10:00:57.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.406+0000 7fab9aa00640 1 -- 172.21.15.77:0/4152761123 <== mds.4 v2:172.21.15.134:6834/3293933699 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5604634c9950 con 0x560463571150 2024-09-10T10:00:57.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.407+0000 7fab9aa00640 10 client.26596 handle_command_reply: tid=0 2024-09-10T10:00:57.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.407+0000 7fabbec22740 1 client.26596 shutdown 2024-09-10T10:00:57.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.407+0000 7fabbec22740 1 -- 172.21.15.77:0/4152761123 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560463561020 msgr2=0x56046355eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:57.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.407+0000 7fabbec22740 1 --2- 172.21.15.77:0/4152761123 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560463561020 0x56046355eac0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7faba0010e90 tx=0x7faba0004730 comp rx=0 tx=0).stop 2024-09-10T10:00:57.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.408+0000 7fabbec22740 1 -- 172.21.15.77:0/4152761123 shutdown_connections 2024-09-10T10:00:57.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.408+0000 7fabbec22740 1 --2- 172.21.15.77:0/4152761123 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x560463571150 0x560463573540 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:57.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.408+0000 7fab9aa00640 0 client.26596 ms_handle_reset on v2:172.21.15.134:6834/3293933699 2024-09-10T10:00:57.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.408+0000 7fabbec22740 1 --2- 172.21.15.77:0/4152761123 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560463561020 0x56046355eac0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:57.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.408+0000 7fabbec22740 1 --2- 172.21.15.77:0/4152761123 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560463556760 0x56046355e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:57.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.408+0000 7fabbec22740 1 --2- 172.21.15.77:0/4152761123 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560463555e40 0x56046355e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:57.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.408+0000 7fabbec22740 1 -- 172.21.15.77:0/4152761123 >> 172.21.15.77:0/4152761123 conn(0x5604635523d0 msgr2=0x5604635580b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:57.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.408+0000 7fabbec22740 1 -- 172.21.15.77:0/4152761123 shutdown_connections 2024-09-10T10:00:57.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.409+0000 7fabbec22740 1 -- 172.21.15.77:0/4152761123 wait complete. 2024-09-10T10:00:57.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.409+0000 7fabbec22740 20 client.26596 trim_cache size 0 max 16384 2024-09-10T10:00:57.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.426+0000 7fabbec22740 1 Processor -- start 2024-09-10T10:00:57.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.426+0000 7fabbec22740 1 -- start start 2024-09-10T10:00:57.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.427+0000 7fabbec22740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560463555e40 0x5604635588a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:57.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.427+0000 7fabbec22740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560463556760 0x56046355ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:57.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.427+0000 7fabbec22740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560463561020 0x560463578da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:57.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.427+0000 7fabbec22740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fabb41bdbd0 con 0x560463556760 2024-09-10T10:00:57.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.427+0000 7fabbec22740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fabb41bf7a0 con 0x560463555e40 2024-09-10T10:00:57.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.427+0000 7fabbec22740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fabb41be2a0 con 0x560463561020 2024-09-10T10:00:57.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.427+0000 7fabb8c00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560463555e40 0x5604635588a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:57.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.427+0000 7fabb8c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560463561020 0x560463578da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:57.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.427+0000 7fabb9600640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560463556760 0x56046355ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:57.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.427+0000 7fabb8c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560463561020 0x560463578da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:47394/0 (socket says 172.21.15.77:47394) 2024-09-10T10:00:57.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.427+0000 7fabb8c00640 1 -- 172.21.15.77:0/2733591758 learned_addr learned my addr 172.21.15.77:0/2733591758 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:00:57.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.428+0000 7fabb9600640 1 -- 172.21.15.77:0/2733591758 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560463561020 msgr2=0x560463578da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:57.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.428+0000 7fabb9600640 1 --2- 172.21.15.77:0/2733591758 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560463561020 0x560463578da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:57.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.428+0000 7fabb9600640 1 -- 172.21.15.77:0/2733591758 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560463555e40 msgr2=0x5604635588a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:57.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.428+0000 7fabb9600640 1 --2- 172.21.15.77:0/2733591758 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560463555e40 0x5604635588a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:57.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.428+0000 7fabb9600640 1 -- 172.21.15.77:0/2733591758 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604635738b0 con 0x560463556760 2024-09-10T10:00:57.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.428+0000 7fabb8c00640 1 --2- 172.21.15.77:0/2733591758 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560463561020 0x560463578da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:00:57.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.428+0000 7fabb9600640 1 --2- 172.21.15.77:0/2733591758 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560463556760 0x56046355ffb0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x56046355f680 tx=0x7faba8004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2165714e104e501a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:57.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.429+0000 7faba5c00640 1 -- 172.21.15.77:0/2733591758 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba8035040 con 0x560463556760 2024-09-10T10:00:57.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.429+0000 7faba5c00640 1 -- 172.21.15.77:0/2733591758 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faba80046c0 con 0x560463556760 2024-09-10T10:00:57.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.429+0000 7faba5c00640 1 -- 172.21.15.77:0/2733591758 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba80038c0 con 0x560463556760 2024-09-10T10:00:57.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.430+0000 7fabbec22740 1 -- 172.21.15.77:0/2733591758 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560463556760 msgr2=0x56046355ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:57.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.430+0000 7fabbec22740 1 --2- 172.21.15.77:0/2733591758 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560463556760 0x56046355ffb0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x56046355f680 tx=0x7faba8004b40 comp rx=0 tx=0).stop 2024-09-10T10:00:57.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.431+0000 7fabbec22740 1 -- 172.21.15.77:0/2733591758 shutdown_connections 2024-09-10T10:00:57.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.431+0000 7fabbec22740 1 --2- 172.21.15.77:0/2733591758 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560463561020 0x560463578da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:57.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.431+0000 7fabbec22740 1 --2- 172.21.15.77:0/2733591758 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560463556760 0x56046355ffb0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:57.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.431+0000 7fabbec22740 1 --2- 172.21.15.77:0/2733591758 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560463555e40 0x5604635588a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:57.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.431+0000 7fabbec22740 1 -- 172.21.15.77:0/2733591758 >> 172.21.15.77:0/2733591758 conn(0x5604635523d0 msgr2=0x56046355f8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:57.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.431+0000 7fabbec22740 1 -- 172.21.15.77:0/2733591758 shutdown_connections 2024-09-10T10:00:57.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.432+0000 7fabbec22740 1 -- 172.21.15.77:0/2733591758 wait complete. 2024-09-10T10:00:57.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.433+0000 7fabbec22740 1 -- start start 2024-09-10T10:00:57.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.433+0000 7fabbec22740 1 Processor -- start 2024-09-10T10:00:57.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.434+0000 7fabbec22740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560463555e40 0x5604635696f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:57.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.434+0000 7fabbec22740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560463556760 0x560463564a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:57.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.434+0000 7fabbec22740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560463561020 0x560463564f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:57.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.434+0000 7fabbec22740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fabb41bf040 con 0x560463555e40 2024-09-10T10:00:57.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.434+0000 7fabbec22740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fabb41be970 con 0x560463561020 2024-09-10T10:00:57.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.434+0000 7fabbec22740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fabb41bbb50 con 0x560463556760 2024-09-10T10:00:57.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.434+0000 7fabbec22740 10 client.0 _ll_register_callbacks cb 0x7faba0009a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:00:57.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.434+0000 7fabb9600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560463556760 0x560463564a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:57.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.434+0000 7fabb9600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560463556760 0x560463564a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:47410/0 (socket says 172.21.15.77:47410) 2024-09-10T10:00:57.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.434+0000 7fabb9600640 1 -- 172.21.15.77:0/4135477148 learned_addr learned my addr 172.21.15.77:0/4135477148 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:00:57.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.435+0000 7fabb9600640 1 -- 172.21.15.77:0/4135477148 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560463561020 msgr2=0x560463564f60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T10:00:57.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.435+0000 7fabb8c00640 1 --2- 172.21.15.77:0/4135477148 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560463561020 0x560463564f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:57.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.435+0000 7fabb9600640 1 --2- 172.21.15.77:0/4135477148 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560463561020 0x560463564f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:57.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.435+0000 7fabb9600640 1 -- 172.21.15.77:0/4135477148 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560463555e40 msgr2=0x5604635696f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:57.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.435+0000 7fabb9600640 1 --2- 172.21.15.77:0/4135477148 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560463555e40 0x5604635696f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:57.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.435+0000 7fabb9600640 1 -- 172.21.15.77:0/4135477148 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560463565790 con 0x560463556760 2024-09-10T10:00:57.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.435+0000 7fabb8c00640 1 --2- 172.21.15.77:0/4135477148 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560463561020 0x560463564f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:00:57.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.435+0000 7fabb9600640 1 --2- 172.21.15.77:0/4135477148 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560463556760 0x560463564a20 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7faba8009900 tx=0x7faba8009540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ff91d99d8291106 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:57.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.435+0000 7fab9aa00640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-10T10:00:57.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.435+0000 7fab9aa00640 1 -- 172.21.15.77:0/4135477148 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba8035040 con 0x560463556760 2024-09-10T10:00:57.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.435+0000 7fab9aa00640 1 -- 172.21.15.77:0/4135477148 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7faba80317b0 con 0x560463556760 2024-09-10T10:00:57.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.435+0000 7fabbec22740 1 -- 172.21.15.77:0/4135477148 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faba80046c0 con 0x560463556760 2024-09-10T10:00:57.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.436+0000 7fab9aa00640 1 -- 172.21.15.77:0/4135477148 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7faba80319b0 con 0x560463556760 2024-09-10T10:00:57.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.436+0000 7fabbec22740 10 client.26602 fetch_fsmap learned FSMap version 33 2024-09-10T10:00:57.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.436+0000 7fabbec22740 1 -- 172.21.15.77:0/4135477148 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5604635dbd60 con 0x560463556760 2024-09-10T10:00:57.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.436+0000 7fab9aa00640 1 -- 172.21.15.77:0/4135477148 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faba80046c0 con 0x560463556760 2024-09-10T10:00:57.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.436+0000 7fab9aa00640 1 -- 172.21.15.77:0/4135477148 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7faba803c4d0 con 0x560463556760 2024-09-10T10:00:57.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.436+0000 7fabbec22740 10 client.26602 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T10:00:57.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.436+0000 7fabbec22740 10 client.26602 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-10T10:00:57.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.436+0000 7fabbec22740 20 client.26602 populate_metadata read hostname 'smithi077' 2024-09-10T10:00:57.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.437+0000 7fabbec22740 1 --2- 172.21.15.77:0/4135477148 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x5604635dbdb0 0x5604635de1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:57.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.437+0000 7fabbec22740 4 client.26602 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:00:57.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.437+0000 7fabbec22740 1 -- 172.21.15.77:0/4135477148 --> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5604634c3e60 con 0x5604635dbdb0 2024-09-10T10:00:57.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.437+0000 7fabb8c00640 1 --2- 172.21.15.77:0/4135477148 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x5604635dbdb0 0x5604635de1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:57.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.437+0000 7fabb8c00640 1 --2- 172.21.15.77:0/4135477148 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x5604635dbdb0 0x5604635de1a0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=62dd80246c9b240f server_cookie=b7a7502e3467c284 in_seq=0 out_seq=0 2024-09-10T10:00:57.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.437+0000 7fab9aa00640 10 client.26602 ms_handle_connect on v2:172.21.15.134:6834/3293933699 2024-09-10T10:00:57.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.437+0000 7fab9aa00640 1 -- 172.21.15.77:0/4135477148 <== mds.4 v2:172.21.15.134:6834/3293933699 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5604634c3e60 con 0x5604635dbdb0 2024-09-10T10:00:57.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.437+0000 7fab9aa00640 10 client.26602 handle_command_reply: tid=0 2024-09-10T10:00:57.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.438+0000 7fabbec22740 1 client.26602 shutdown 2024-09-10T10:00:57.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.438+0000 7fabbec22740 1 -- 172.21.15.77:0/4135477148 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560463556760 msgr2=0x560463564a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:57.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.438+0000 7fabbec22740 1 --2- 172.21.15.77:0/4135477148 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560463556760 0x560463564a20 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7faba8009900 tx=0x7faba8009540 comp rx=0 tx=0).stop 2024-09-10T10:00:57.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.439+0000 7fabbec22740 1 -- 172.21.15.77:0/4135477148 shutdown_connections 2024-09-10T10:00:57.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.439+0000 7fabbec22740 1 --2- 172.21.15.77:0/4135477148 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x5604635dbdb0 0x5604635de1a0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:57.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.439+0000 7fabbec22740 1 --2- 172.21.15.77:0/4135477148 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560463561020 0x560463564f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:57.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.439+0000 7fabbec22740 1 --2- 172.21.15.77:0/4135477148 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560463556760 0x560463564a20 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:57.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.439+0000 7fab9aa00640 0 client.26602 ms_handle_reset on v2:172.21.15.134:6834/3293933699 2024-09-10T10:00:57.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.439+0000 7fabbec22740 1 --2- 172.21.15.77:0/4135477148 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560463555e40 0x5604635696f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:57.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.439+0000 7fabbec22740 1 -- 172.21.15.77:0/4135477148 >> 172.21.15.77:0/4135477148 conn(0x5604635523d0 msgr2=0x5604635600c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:57.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.439+0000 7fabbec22740 1 -- 172.21.15.77:0/4135477148 shutdown_connections 2024-09-10T10:00:57.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.439+0000 7fabbec22740 1 -- 172.21.15.77:0/4135477148 wait complete. 2024-09-10T10:00:57.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.439+0000 7fabbec22740 20 client.26602 trim_cache size 0 max 16384 2024-09-10T10:00:57.442 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-10T10:00:57.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.440+0000 7fabbaa00640 1 -- 172.21.15.77:0/977781414 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fab9c0a2c00 msgr2=0x7fab9c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:57.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.440+0000 7fabbaa00640 1 --2- 172.21.15.77:0/977781414 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fab9c0a2c00 0x7fab9c0a50c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fabac0a4470 tx=0x7fabac00efd0 comp rx=0 tx=0).stop 2024-09-10T10:00:57.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.440+0000 7fabbaa00640 1 -- 172.21.15.77:0/977781414 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fabb4108f40 msgr2=0x7fabb410df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:57.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.440+0000 7fabbaa00640 1 --2- 172.21.15.77:0/977781414 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fabb4108f40 0x7fabb410df50 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fabac00eb10 tx=0x7fabac00c2d0 comp rx=0 tx=0).stop 2024-09-10T10:00:57.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.440+0000 7fabbaa00640 1 -- 172.21.15.77:0/977781414 shutdown_connections 2024-09-10T10:00:57.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.440+0000 7fabbaa00640 1 --2- 172.21.15.77:0/977781414 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fab9c0a2c00 0x7fab9c0a50c0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:57.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.440+0000 7fabbaa00640 1 --2- 172.21.15.77:0/977781414 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fabb4108f40 0x7fabb410df50 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:57.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.440+0000 7fabbaa00640 1 --2- 172.21.15.77:0/977781414 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fabb41047c0 0x7fabb410da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:57.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.440+0000 7fabbaa00640 1 --2- 172.21.15.77:0/977781414 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fabb4103e10 0x7fabb4110960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:57.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.440+0000 7fabbaa00640 1 -- 172.21.15.77:0/977781414 >> 172.21.15.77:0/977781414 conn(0x7fabb40ff8b0 msgr2=0x7fabb4100c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:57.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.440+0000 7fabbaa00640 1 -- 172.21.15.77:0/977781414 shutdown_connections 2024-09-10T10:00:57.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:57.440+0000 7fabbaa00640 1 -- 172.21.15.77:0/977781414 wait complete. 2024-09-10T10:00:58.216 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T10:00:58.681 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:00:59.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:00:58 smithi120 ceph-mon[34770]: pgmap v565: 129 pgs: 129 active+clean; 114 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.1 MiB/s wr, 354 op/s 2024-09-10T10:00:59.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:00:58 smithi077 ceph-mon[30319]: pgmap v565: 129 pgs: 129 active+clean; 114 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.1 MiB/s wr, 354 op/s 2024-09-10T10:00:59.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.351+0000 7f8381e00640 1 -- 172.21.15.77:0/2752599363 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f837c10c170 msgr2=0x7f837c10e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:59.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.351+0000 7f8381e00640 1 --2- 172.21.15.77:0/2752599363 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f837c10c170 0x7f837c10e610 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8374009900 tx=0x7f837402a480 comp rx=0 tx=0).stop 2024-09-10T10:00:59.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.352+0000 7f8381e00640 1 -- 172.21.15.77:0/2752599363 shutdown_connections 2024-09-10T10:00:59.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.352+0000 7f8381e00640 1 --2- 172.21.15.77:0/2752599363 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f837c10c170 0x7f837c10e610 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:59.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.352+0000 7f8381e00640 1 --2- 172.21.15.77:0/2752599363 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f837c103660 0x7f837c10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:59.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.352+0000 7f8381e00640 1 --2- 172.21.15.77:0/2752599363 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f837c102d40 0x7f837c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:59.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.352+0000 7f8381e00640 1 -- 172.21.15.77:0/2752599363 >> 172.21.15.77:0/2752599363 conn(0x7f837c0fc7a0 msgr2=0x7f837c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:59.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.352+0000 7f8381e00640 1 -- 172.21.15.77:0/2752599363 shutdown_connections 2024-09-10T10:00:59.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.352+0000 7f8381e00640 1 -- 172.21.15.77:0/2752599363 wait complete. 2024-09-10T10:00:59.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.353+0000 7f8381e00640 1 Processor -- start 2024-09-10T10:00:59.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.353+0000 7f8381e00640 1 -- start start 2024-09-10T10:00:59.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.353+0000 7f8381e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f837c102d40 0x7f837c1983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:59.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.353+0000 7f8381e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f837c103660 0x7f837c198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:59.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.353+0000 7f8381e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f837c10c170 0x7f837c19cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:59.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.354+0000 7f8381e00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f837c111fc0 con 0x7f837c103660 2024-09-10T10:00:59.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.354+0000 7f8381e00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f837c111e40 con 0x7f837c10c170 2024-09-10T10:00:59.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.354+0000 7f8381e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f837c112140 con 0x7f837c102d40 2024-09-10T10:00:59.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.354+0000 7f837aa00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f837c102d40 0x7f837c1983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:59.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.354+0000 7f837aa00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f837c102d40 0x7f837c1983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:47416/0 (socket says 172.21.15.77:47416) 2024-09-10T10:00:59.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.354+0000 7f837aa00640 1 -- 172.21.15.77:0/1004092829 learned_addr learned my addr 172.21.15.77:0/1004092829 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:00:59.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.354+0000 7f837b400640 1 --2- 172.21.15.77:0/1004092829 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f837c10c170 0x7f837c19cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:59.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.354+0000 7f837a000640 1 --2- 172.21.15.77:0/1004092829 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f837c103660 0x7f837c198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:59.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.354+0000 7f837aa00640 1 -- 172.21.15.77:0/1004092829 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f837c10c170 msgr2=0x7f837c19cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:59.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.354+0000 7f837aa00640 1 --2- 172.21.15.77:0/1004092829 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f837c10c170 0x7f837c19cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:59.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.354+0000 7f837aa00640 1 -- 172.21.15.77:0/1004092829 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f837c103660 msgr2=0x7f837c198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:59.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.354+0000 7f837aa00640 1 --2- 172.21.15.77:0/1004092829 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f837c103660 0x7f837c198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:59.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.354+0000 7f837aa00640 1 -- 172.21.15.77:0/1004092829 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f837c19d370 con 0x7f837c102d40 2024-09-10T10:00:59.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.354+0000 7f837b400640 1 --2- 172.21.15.77:0/1004092829 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f837c10c170 0x7f837c19cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:00:59.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.355+0000 7f837aa00640 1 --2- 172.21.15.77:0/1004092829 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f837c102d40 0x7f837c1983c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f836c00ebc0 tx=0x7f836c00c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:59.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.355+0000 7f8380a00640 1 -- 172.21.15.77:0/1004092829 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f836c010630 con 0x7f837c102d40 2024-09-10T10:00:59.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.355+0000 7f8380a00640 1 -- 172.21.15.77:0/1004092829 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f836c010cb0 con 0x7f837c102d40 2024-09-10T10:00:59.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.355+0000 7f8381e00640 1 -- 172.21.15.77:0/1004092829 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f837c19d660 con 0x7f837c102d40 2024-09-10T10:00:59.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.355+0000 7f8380a00640 1 -- 172.21.15.77:0/1004092829 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f836c018bd0 con 0x7f837c102d40 2024-09-10T10:00:59.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.355+0000 7f8381e00640 1 -- 172.21.15.77:0/1004092829 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f837c1a4f40 con 0x7f837c102d40 2024-09-10T10:00:59.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.362+0000 7f8384869740 1 Processor -- start 2024-09-10T10:00:59.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.362+0000 7f8384869740 1 -- start start 2024-09-10T10:00:59.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.362+0000 7f8384869740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c29ff3450 0x558c29ff3830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:59.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.362+0000 7f8384869740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558c29ff3d70 0x558c29fedfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:59.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.362+0000 7f8384869740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558c29fee520 0x558c29fee9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:59.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.362+0000 7f8384869740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558c29f543e0 con 0x558c29ff3450 2024-09-10T10:00:59.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.362+0000 7f8384869740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558c29efbd80 con 0x558c29ff3d70 2024-09-10T10:00:59.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.362+0000 7f8384869740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558c29f503a0 con 0x558c29fee520 2024-09-10T10:00:59.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.363+0000 7f837a000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558c29fee520 0x558c29fee9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:59.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.363+0000 7f837a000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558c29fee520 0x558c29fee9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:47424/0 (socket says 172.21.15.77:47424) 2024-09-10T10:00:59.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.363+0000 7f837a000640 1 -- 172.21.15.77:0/658037287 learned_addr learned my addr 172.21.15.77:0/658037287 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:00:59.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.363+0000 7f837a000640 1 -- 172.21.15.77:0/658037287 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558c29ff3d70 msgr2=0x558c29fedfe0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T10:00:59.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.363+0000 7f837a000640 1 --2- 172.21.15.77:0/658037287 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558c29ff3d70 0x558c29fedfe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:59.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.363+0000 7f837a000640 1 -- 172.21.15.77:0/658037287 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c29ff3450 msgr2=0x558c29ff3830 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T10:00:59.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.363+0000 7f8380a00640 1 -- 172.21.15.77:0/1004092829 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f836c0107d0 con 0x7f837c102d40 2024-09-10T10:00:59.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.364+0000 7f837b400640 1 --2- 172.21.15.77:0/658037287 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c29ff3450 0x558c29ff3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:59.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.364+0000 7f837a000640 1 --2- 172.21.15.77:0/658037287 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c29ff3450 0x558c29ff3830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:59.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.364+0000 7f837a000640 1 -- 172.21.15.77:0/658037287 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c29f1aae0 con 0x558c29fee520 2024-09-10T10:00:59.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.365+0000 7f837b400640 1 --2- 172.21.15.77:0/658037287 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c29ff3450 0x558c29ff3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:00:59.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.366+0000 7f837a000640 1 --2- 172.21.15.77:0/658037287 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558c29fee520 0x558c29fee9a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f836800bc20 tx=0x7f83680107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a57ff23a4ff269b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:59.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.366+0000 7f8380a00640 1 --2- 172.21.15.77:0/1004092829 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f83600a2c00 0x7f83600a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:59.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.366+0000 7f8367e00640 1 -- 172.21.15.77:0/658037287 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f836801c480 con 0x558c29fee520 2024-09-10T10:00:59.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.366+0000 7f8380a00640 1 -- 172.21.15.77:0/1004092829 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f836c0cf9f0 con 0x7f837c102d40 2024-09-10T10:00:59.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.366+0000 7f8367e00640 1 -- 172.21.15.77:0/658037287 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f836801cb00 con 0x558c29fee520 2024-09-10T10:00:59.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.366+0000 7f8367e00640 1 -- 172.21.15.77:0/658037287 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8368011b20 con 0x558c29fee520 2024-09-10T10:00:59.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.366+0000 7f837b400640 1 --2- 172.21.15.77:0/1004092829 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f83600a2c00 0x7f83600a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:59.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.367+0000 7f8384869740 1 -- 172.21.15.77:0/658037287 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558c29fee520 msgr2=0x558c29fee9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:59.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.367+0000 7f8384869740 1 --2- 172.21.15.77:0/658037287 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558c29fee520 0x558c29fee9a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f836800bc20 tx=0x7f83680107a0 comp rx=0 tx=0).stop 2024-09-10T10:00:59.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.367+0000 7f8384869740 1 -- 172.21.15.77:0/658037287 shutdown_connections 2024-09-10T10:00:59.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.367+0000 7f8384869740 1 --2- 172.21.15.77:0/658037287 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558c29fee520 0x558c29fee9a0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:59.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.367+0000 7f8384869740 1 --2- 172.21.15.77:0/658037287 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558c29ff3d70 0x558c29fedfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:59.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.367+0000 7f8384869740 1 --2- 172.21.15.77:0/658037287 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c29ff3450 0x558c29ff3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:59.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.367+0000 7f8384869740 1 -- 172.21.15.77:0/658037287 >> 172.21.15.77:0/658037287 conn(0x558c29fe9cb0 msgr2=0x558c29fea090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:59.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.367+0000 7f8384869740 1 -- 172.21.15.77:0/658037287 shutdown_connections 2024-09-10T10:00:59.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.367+0000 7f8384869740 1 -- 172.21.15.77:0/658037287 wait complete. 2024-09-10T10:00:59.370 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.368+0000 7f8384869740 1 -- start start 2024-09-10T10:00:59.370 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.368+0000 7f8384869740 1 Processor -- start 2024-09-10T10:00:59.370 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.368+0000 7f837b400640 1 --2- 172.21.15.77:0/1004092829 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f83600a2c00 0x7f83600a50c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f8374031550 tx=0x7f83740046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:59.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.369+0000 7f8384869740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c29fee520 0x558c2a009820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:59.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.369+0000 7f8384869740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558c29ff3450 0x558c2a009d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:59.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.369+0000 7f8384869740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558c29ff3d70 0x558c2a00a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:59.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.369+0000 7f8384869740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558c2a00ca70 con 0x558c29fee520 2024-09-10T10:00:59.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.369+0000 7f8384869740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558c29f0f0f0 con 0x558c29ff3450 2024-09-10T10:00:59.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.369+0000 7f8384869740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558c2a00cbf0 con 0x558c29ff3d70 2024-09-10T10:00:59.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.369+0000 7f8384869740 10 client.0 _ll_register_callbacks cb 0x558c29ef6bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:00:59.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.369+0000 7f837aa00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c29fee520 0x558c2a009820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:59.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.369+0000 7f837aa00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c29fee520 0x558c2a009820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:41082/0 (socket says 172.21.15.77:41082) 2024-09-10T10:00:59.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.369+0000 7f837aa00640 1 -- 172.21.15.77:0/3755454789 learned_addr learned my addr 172.21.15.77:0/3755454789 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:00:59.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.369+0000 7f837b400640 1 --2- 172.21.15.77:0/3755454789 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558c29ff3d70 0x558c2a00a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:59.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.369+0000 7f837a000640 1 --2- 172.21.15.77:0/3755454789 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558c29ff3450 0x558c2a009d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:59.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.370+0000 7f837b400640 1 -- 172.21.15.77:0/3755454789 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558c29ff3450 msgr2=0x558c2a009d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:59.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.370+0000 7f837b400640 1 --2- 172.21.15.77:0/3755454789 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558c29ff3450 0x558c2a009d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:59.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.370+0000 7f837b400640 1 -- 172.21.15.77:0/3755454789 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c29fee520 msgr2=0x558c2a009820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:59.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.370+0000 7f837b400640 1 --2- 172.21.15.77:0/3755454789 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c29fee520 0x558c2a009820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:59.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.370+0000 7f837b400640 1 -- 172.21.15.77:0/3755454789 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c29cc53c0 con 0x558c29ff3d70 2024-09-10T10:00:59.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.370+0000 7f837a000640 1 --2- 172.21.15.77:0/3755454789 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558c29ff3450 0x558c2a009d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:00:59.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.370+0000 7f837b400640 1 --2- 172.21.15.77:0/3755454789 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558c29ff3d70 0x558c2a00a2a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f8374009230 tx=0x7f8374009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba0dd0a2d9c5078 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:59.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.370+0000 7f8364c00640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-10T10:00:59.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.370+0000 7f8364c00640 1 -- 172.21.15.77:0/3755454789 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f837404d070 con 0x558c29ff3d70 2024-09-10T10:00:59.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.370+0000 7f8384869740 1 -- 172.21.15.77:0/3755454789 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f836801cb00 con 0x558c29ff3d70 2024-09-10T10:00:59.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.370+0000 7f8364c00640 1 -- 172.21.15.77:0/3755454789 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f8374003a00 con 0x558c29ff3d70 2024-09-10T10:00:59.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.371+0000 7f8364c00640 1 -- 172.21.15.77:0/3755454789 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f8374059050 con 0x558c29ff3d70 2024-09-10T10:00:59.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.371+0000 7f8384869740 10 client.26614 fetch_fsmap learned FSMap version 33 2024-09-10T10:00:59.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.371+0000 7f8384869740 1 -- 172.21.15.77:0/3755454789 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x558c2a018bb0 con 0x558c29ff3d70 2024-09-10T10:00:59.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.371+0000 7f8364c00640 1 -- 172.21.15.77:0/3755454789 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f836801cb00 con 0x558c29ff3d70 2024-09-10T10:00:59.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.371+0000 7f8364c00640 1 -- 172.21.15.77:0/3755454789 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7f8374049ca0 con 0x558c29ff3d70 2024-09-10T10:00:59.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.371+0000 7f8384869740 10 client.26614 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T10:00:59.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.371+0000 7f8384869740 10 client.26614 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T10:00:59.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.371+0000 7f8384869740 20 client.26614 populate_metadata read hostname 'smithi077' 2024-09-10T10:00:59.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.371+0000 7f8384869740 1 --2- 172.21.15.77:0/3755454789 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x558c2a019fc0 0x558c2a01c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:59.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.371+0000 7f8384869740 4 client.26614 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:00:59.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.371+0000 7f8384869740 1 -- 172.21.15.77:0/3755454789 --> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558c29f61950 con 0x558c2a019fc0 2024-09-10T10:00:59.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.372+0000 7f837aa00640 1 --2- 172.21.15.77:0/3755454789 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x558c2a019fc0 0x558c2a01c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:59.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.373+0000 7f8364c00640 10 client.26614 ms_handle_connect on v2:172.21.15.134:6836/2461497091 2024-09-10T10:00:59.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.373+0000 7f837aa00640 1 --2- 172.21.15.77:0/3755454789 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x558c2a019fc0 0x558c2a01c3b0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=824195fc36b79fff server_cookie=121051c792910e59 in_seq=0 out_seq=0 2024-09-10T10:00:59.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.373+0000 7f8364c00640 1 -- 172.21.15.77:0/3755454789 <== mds.3 v2:172.21.15.134:6836/2461497091 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558c29f61950 con 0x558c2a019fc0 2024-09-10T10:00:59.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.373+0000 7f8364c00640 10 client.26614 handle_command_reply: tid=0 2024-09-10T10:00:59.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.373+0000 7f8384869740 1 client.26614 shutdown 2024-09-10T10:00:59.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.374+0000 7f8384869740 1 -- 172.21.15.77:0/3755454789 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558c29ff3d70 msgr2=0x558c2a00a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:59.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.374+0000 7f8384869740 1 --2- 172.21.15.77:0/3755454789 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558c29ff3d70 0x558c2a00a2a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f8374009230 tx=0x7f8374009260 comp rx=0 tx=0).stop 2024-09-10T10:00:59.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.374+0000 7f8384869740 1 -- 172.21.15.77:0/3755454789 shutdown_connections 2024-09-10T10:00:59.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.374+0000 7f8384869740 1 --2- 172.21.15.77:0/3755454789 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558c29ff3d70 0x558c2a00a2a0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:59.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.374+0000 7f8384869740 1 --2- 172.21.15.77:0/3755454789 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558c29ff3450 0x558c2a009d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:59.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.374+0000 7f8384869740 1 --2- 172.21.15.77:0/3755454789 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x558c2a019fc0 0x558c2a01c3b0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:59.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.374+0000 7f8384869740 1 --2- 172.21.15.77:0/3755454789 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c29fee520 0x558c2a009820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:59.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.374+0000 7f8364c00640 0 client.26614 ms_handle_reset on v2:172.21.15.134:6836/2461497091 2024-09-10T10:00:59.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.374+0000 7f8384869740 1 -- 172.21.15.77:0/3755454789 >> 172.21.15.77:0/3755454789 conn(0x558c29fe9cb0 msgr2=0x558c29ff6fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:59.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.374+0000 7f8384869740 1 -- 172.21.15.77:0/3755454789 shutdown_connections 2024-09-10T10:00:59.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.374+0000 7f8384869740 1 -- 172.21.15.77:0/3755454789 wait complete. 2024-09-10T10:00:59.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.374+0000 7f8384869740 20 client.26614 trim_cache size 0 max 16384 2024-09-10T10:00:59.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:00:58 smithi134 ceph-mon[35674]: pgmap v565: 129 pgs: 129 active+clean; 114 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.1 MiB/s wr, 354 op/s 2024-09-10T10:00:59.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.392+0000 7f8384869740 1 Processor -- start 2024-09-10T10:00:59.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.392+0000 7f8384869740 1 -- start start 2024-09-10T10:00:59.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.392+0000 7f8384869740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c29fee520 0x558c29ff5c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:59.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.392+0000 7f8384869740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558c29ff3450 0x558c29ff5360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:59.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.392+0000 7f8384869740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558c29ff3d70 0x558c2a021c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:59.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.392+0000 7f8384869740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f837c1accc0 con 0x558c29fee520 2024-09-10T10:00:59.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.392+0000 7f8384869740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f837c1ae890 con 0x558c29ff3450 2024-09-10T10:00:59.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.392+0000 7f8384869740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f837c1ad390 con 0x558c29ff3d70 2024-09-10T10:00:59.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.393+0000 7f837a000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558c29ff3450 0x558c29ff5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:59.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.393+0000 7f837a000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558c29ff3450 0x558c29ff5360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:56680/0 (socket says 172.21.15.77:56680) 2024-09-10T10:00:59.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.393+0000 7f837aa00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c29fee520 0x558c29ff5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:59.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.393+0000 7f837a000640 1 -- 172.21.15.77:0/3215095902 learned_addr learned my addr 172.21.15.77:0/3215095902 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:00:59.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.393+0000 7f837b400640 1 --2- 172.21.15.77:0/3215095902 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558c29ff3d70 0x558c2a021c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:59.395 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.393+0000 7f837a000640 1 -- 172.21.15.77:0/3215095902 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558c29ff3d70 msgr2=0x558c2a021c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:59.395 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.393+0000 7f837a000640 1 --2- 172.21.15.77:0/3215095902 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558c29ff3d70 0x558c2a021c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:59.395 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.393+0000 7f837a000640 1 -- 172.21.15.77:0/3215095902 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c29fee520 msgr2=0x558c29ff5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:59.395 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.393+0000 7f837a000640 1 --2- 172.21.15.77:0/3215095902 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c29fee520 0x558c29ff5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:59.395 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.393+0000 7f837a000640 1 -- 172.21.15.77:0/3215095902 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c2a01c9e0 con 0x558c29ff3450 2024-09-10T10:00:59.395 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.394+0000 7f837a000640 1 --2- 172.21.15.77:0/3215095902 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558c29ff3450 0x558c29ff5360 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f836800bbf0 tx=0x7f836800bb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a46a215dd5dd19e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:59.395 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.394+0000 7f8367400640 1 -- 172.21.15.77:0/3215095902 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8368004020 con 0x558c29ff3450 2024-09-10T10:00:59.395 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.394+0000 7f8367400640 1 -- 172.21.15.77:0/3215095902 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8368005520 con 0x558c29ff3450 2024-09-10T10:00:59.395 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.394+0000 7f8367400640 1 -- 172.21.15.77:0/3215095902 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8368011cf0 con 0x558c29ff3450 2024-09-10T10:00:59.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.395+0000 7f8384869740 1 -- 172.21.15.77:0/3215095902 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558c29ff3450 msgr2=0x558c29ff5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:59.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.395+0000 7f8384869740 1 --2- 172.21.15.77:0/3215095902 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558c29ff3450 0x558c29ff5360 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f836800bbf0 tx=0x7f836800bb20 comp rx=0 tx=0).stop 2024-09-10T10:00:59.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.395+0000 7f8384869740 1 -- 172.21.15.77:0/3215095902 shutdown_connections 2024-09-10T10:00:59.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.395+0000 7f8384869740 1 --2- 172.21.15.77:0/3215095902 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558c29ff3d70 0x558c2a021c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:59.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.395+0000 7f8384869740 1 --2- 172.21.15.77:0/3215095902 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558c29ff3450 0x558c29ff5360 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:59.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.395+0000 7f8384869740 1 --2- 172.21.15.77:0/3215095902 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c29fee520 0x558c29ff5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:59.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.395+0000 7f8384869740 1 -- 172.21.15.77:0/3215095902 >> 172.21.15.77:0/3215095902 conn(0x558c29fe9cb0 msgr2=0x558c2a00a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:59.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.395+0000 7f8384869740 1 -- 172.21.15.77:0/3215095902 shutdown_connections 2024-09-10T10:00:59.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.395+0000 7f8384869740 1 -- 172.21.15.77:0/3215095902 wait complete. 2024-09-10T10:00:59.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.396+0000 7f8384869740 1 -- start start 2024-09-10T10:00:59.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.396+0000 7f8384869740 1 Processor -- start 2024-09-10T10:00:59.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.396+0000 7f8384869740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c29fee520 0x558c2a012030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:59.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.396+0000 7f8384869740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558c29ff3450 0x558c2a012570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:59.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.396+0000 7f8384869740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558c29ff3d70 0x558c2a00ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:59.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.396+0000 7f8384869740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f837c1ae130 con 0x558c29fee520 2024-09-10T10:00:59.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.396+0000 7f8384869740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f837c1ada60 con 0x558c29ff3d70 2024-09-10T10:00:59.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.396+0000 7f8384869740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f837c1aaca0 con 0x558c29ff3450 2024-09-10T10:00:59.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.396+0000 7f8384869740 10 client.0 _ll_register_callbacks cb 0x7f8374009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:00:59.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.397+0000 7f837b400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558c29ff3d70 0x558c2a00ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:59.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.397+0000 7f837a000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558c29ff3450 0x558c2a012570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:59.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.397+0000 7f837b400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558c29ff3d70 0x558c2a00ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:56686/0 (socket says 172.21.15.77:56686) 2024-09-10T10:00:59.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.397+0000 7f837b400640 1 -- 172.21.15.77:0/4001377796 learned_addr learned my addr 172.21.15.77:0/4001377796 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:00:59.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.397+0000 7f837aa00640 1 --2- 172.21.15.77:0/4001377796 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c29fee520 0x558c2a012030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:59.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.397+0000 7f837b400640 1 -- 172.21.15.77:0/4001377796 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558c29ff3450 msgr2=0x558c2a012570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:59.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.397+0000 7f837b400640 1 --2- 172.21.15.77:0/4001377796 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558c29ff3450 0x558c2a012570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:59.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.397+0000 7f837b400640 1 -- 172.21.15.77:0/4001377796 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c29fee520 msgr2=0x558c2a012030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:59.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.397+0000 7f837b400640 1 --2- 172.21.15.77:0/4001377796 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c29fee520 0x558c2a012030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:59.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.397+0000 7f837b400640 1 -- 172.21.15.77:0/4001377796 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c2a00d510 con 0x558c29ff3d70 2024-09-10T10:00:59.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.397+0000 7f837aa00640 1 --2- 172.21.15.77:0/4001377796 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c29fee520 0x558c2a012030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:00:59.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.397+0000 7f837a000640 1 --2- 172.21.15.77:0/4001377796 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558c29ff3450 0x558c2a012570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:00:59.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.397+0000 7f837b400640 1 --2- 172.21.15.77:0/4001377796 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558c29ff3d70 0x558c2a00ce00 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f837403c680 tx=0x7f8374003ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=309e1c5b7637ba1e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:59.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.398+0000 7f8364c00640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T10:00:59.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.398+0000 7f8364c00640 1 -- 172.21.15.77:0/4001377796 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8374061070 con 0x558c29ff3d70 2024-09-10T10:00:59.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.398+0000 7f8384869740 1 -- 172.21.15.77:0/4001377796 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8368005520 con 0x558c29ff3d70 2024-09-10T10:00:59.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.398+0000 7f8364c00640 1 -- 172.21.15.77:0/4001377796 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f837403c900 con 0x558c29ff3d70 2024-09-10T10:00:59.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.398+0000 7f8364c00640 1 -- 172.21.15.77:0/4001377796 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f837404d050 con 0x558c29ff3d70 2024-09-10T10:00:59.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.398+0000 7f8384869740 10 client.26660 fetch_fsmap learned FSMap version 33 2024-09-10T10:00:59.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.398+0000 7f8384869740 1 -- 172.21.15.77:0/4001377796 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x558c2a0855c0 con 0x558c29ff3d70 2024-09-10T10:00:59.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.398+0000 7f8364c00640 1 -- 172.21.15.77:0/4001377796 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8368005520 con 0x558c29ff3d70 2024-09-10T10:00:59.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.399+0000 7f8364c00640 1 -- 172.21.15.77:0/4001377796 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7f8374049930 con 0x558c29ff3d70 2024-09-10T10:00:59.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.399+0000 7f8384869740 10 client.26660 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T10:00:59.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.399+0000 7f8384869740 10 client.26660 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T10:00:59.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.399+0000 7f8384869740 20 client.26660 populate_metadata read hostname 'smithi077' 2024-09-10T10:00:59.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.399+0000 7f8384869740 1 --2- 172.21.15.77:0/4001377796 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x558c2a085d10 0x558c2a088130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:59.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.399+0000 7f8384869740 4 client.26660 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:00:59.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.399+0000 7f8384869740 1 -- 172.21.15.77:0/4001377796 --> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558c29f5be60 con 0x558c2a085d10 2024-09-10T10:00:59.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.399+0000 7f837aa00640 1 --2- 172.21.15.77:0/4001377796 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x558c2a085d10 0x558c2a088130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:59.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.399+0000 7f837aa00640 1 --2- 172.21.15.77:0/4001377796 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x558c2a085d10 0x558c2a088130 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=af7b151d3564b292 server_cookie=4a65db9520d00afb in_seq=0 out_seq=0 2024-09-10T10:00:59.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.399+0000 7f8364c00640 10 client.26660 ms_handle_connect on v2:172.21.15.134:6836/2461497091 2024-09-10T10:00:59.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.400+0000 7f8364c00640 1 -- 172.21.15.77:0/4001377796 <== mds.3 v2:172.21.15.134:6836/2461497091 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558c29f5be60 con 0x558c2a085d10 2024-09-10T10:00:59.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.400+0000 7f8364c00640 10 client.26660 handle_command_reply: tid=0 2024-09-10T10:00:59.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.400+0000 7f8384869740 1 client.26660 shutdown 2024-09-10T10:00:59.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.400+0000 7f8384869740 1 -- 172.21.15.77:0/4001377796 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558c29ff3d70 msgr2=0x558c2a00ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:59.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.400+0000 7f8384869740 1 --2- 172.21.15.77:0/4001377796 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558c29ff3d70 0x558c2a00ce00 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f837403c680 tx=0x7f8374003ab0 comp rx=0 tx=0).stop 2024-09-10T10:00:59.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.400+0000 7f8384869740 1 -- 172.21.15.77:0/4001377796 shutdown_connections 2024-09-10T10:00:59.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.400+0000 7f8384869740 1 --2- 172.21.15.77:0/4001377796 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558c29ff3d70 0x558c2a00ce00 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:59.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.400+0000 7f8384869740 1 --2- 172.21.15.77:0/4001377796 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558c29ff3450 0x558c2a012570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:59.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.400+0000 7f8384869740 1 --2- 172.21.15.77:0/4001377796 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x558c2a085d10 0x558c2a088130 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:59.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.400+0000 7f8384869740 1 --2- 172.21.15.77:0/4001377796 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c29fee520 0x558c2a012030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:59.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.400+0000 7f8364c00640 0 client.26660 ms_handle_reset on v2:172.21.15.134:6836/2461497091 2024-09-10T10:00:59.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.400+0000 7f8384869740 1 -- 172.21.15.77:0/4001377796 >> 172.21.15.77:0/4001377796 conn(0x558c29fe9cb0 msgr2=0x558c2a00b8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:59.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.400+0000 7f8384869740 1 -- 172.21.15.77:0/4001377796 shutdown_connections 2024-09-10T10:00:59.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.400+0000 7f8384869740 1 -- 172.21.15.77:0/4001377796 wait complete. 2024-09-10T10:00:59.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.400+0000 7f8384869740 20 client.26660 trim_cache size 0 max 16384 2024-09-10T10:00:59.404 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-10T10:00:59.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.401+0000 7f8381e00640 1 -- 172.21.15.77:0/1004092829 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f83600a2c00 msgr2=0x7f83600a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:59.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.401+0000 7f8381e00640 1 --2- 172.21.15.77:0/1004092829 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f83600a2c00 0x7f83600a50c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f8374031550 tx=0x7f83740046c0 comp rx=0 tx=0).stop 2024-09-10T10:00:59.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.401+0000 7f8381e00640 1 -- 172.21.15.77:0/1004092829 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f837c102d40 msgr2=0x7f837c1983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:59.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.401+0000 7f8381e00640 1 --2- 172.21.15.77:0/1004092829 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f837c102d40 0x7f837c1983c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f836c00ebc0 tx=0x7f836c00c6a0 comp rx=0 tx=0).stop 2024-09-10T10:00:59.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.401+0000 7f8381e00640 1 -- 172.21.15.77:0/1004092829 shutdown_connections 2024-09-10T10:00:59.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.401+0000 7f8381e00640 1 --2- 172.21.15.77:0/1004092829 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f83600a2c00 0x7f83600a50c0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:59.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.401+0000 7f8381e00640 1 --2- 172.21.15.77:0/1004092829 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f837c10c170 0x7f837c19cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:59.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.401+0000 7f8381e00640 1 --2- 172.21.15.77:0/1004092829 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f837c103660 0x7f837c198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:59.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.401+0000 7f8381e00640 1 --2- 172.21.15.77:0/1004092829 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f837c102d40 0x7f837c1983c0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:59.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.401+0000 7f8381e00640 1 -- 172.21.15.77:0/1004092829 >> 172.21.15.77:0/1004092829 conn(0x7f837c0fc7a0 msgr2=0x7f837c0fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:59.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.402+0000 7f8381e00640 1 -- 172.21.15.77:0/1004092829 shutdown_connections 2024-09-10T10:00:59.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:00:59.402+0000 7f8381e00640 1 -- 172.21.15.77:0/1004092829 wait complete. 2024-09-10T10:01:00.303 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T10:01:00.762 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:01:01.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:01:00 smithi120 ceph-mon[34770]: pgmap v566: 129 pgs: 129 active+clean; 115 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 2.6 MiB/s wr, 466 op/s 2024-09-10T10:01:01.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:01:00 smithi077 ceph-mon[30319]: pgmap v566: 129 pgs: 129 active+clean; 115 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 2.6 MiB/s wr, 466 op/s 2024-09-10T10:01:01.241 INFO:tasks.workunit.client.0.smithi077.stdout:40000 mapwrite 0x1becad thru 0x1c6a92 (0x7de6 bytes) 2024-09-10T10:01:01.259 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:01:00 smithi134 ceph-mon[35674]: pgmap v566: 129 pgs: 129 active+clean; 115 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 2.6 MiB/s wr, 466 op/s 2024-09-10T10:01:01.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.362+0000 7fce4ce00640 1 -- 172.21.15.77:0/3859572211 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fce48104010 msgr2=0x7fce481043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:01.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.362+0000 7fce4ce00640 1 --2- 172.21.15.77:0/3859572211 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fce48104010 0x7fce481043f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fce38009900 tx=0x7fce3802a400 comp rx=0 tx=0).stop 2024-09-10T10:01:01.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.362+0000 7fce4ce00640 1 -- 172.21.15.77:0/3859572211 shutdown_connections 2024-09-10T10:01:01.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.362+0000 7fce4ce00640 1 --2- 172.21.15.77:0/3859572211 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fce48109140 0x7fce48110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:01.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.362+0000 7fce4ce00640 1 --2- 172.21.15.77:0/3859572211 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce481049c0 0x7fce48108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:01.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.362+0000 7fce4ce00640 1 --2- 172.21.15.77:0/3859572211 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fce48104010 0x7fce481043f0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:01.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.362+0000 7fce4ce00640 1 -- 172.21.15.77:0/3859572211 >> 172.21.15.77:0/3859572211 conn(0x7fce480ffa50 msgr2=0x7fce48101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:01.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.363+0000 7fce4ce00640 1 -- 172.21.15.77:0/3859572211 shutdown_connections 2024-09-10T10:01:01.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.363+0000 7fce4ce00640 1 -- 172.21.15.77:0/3859572211 wait complete. 2024-09-10T10:01:01.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.365+0000 7fce4ce00640 1 Processor -- start 2024-09-10T10:01:01.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.365+0000 7fce4ce00640 1 -- start start 2024-09-10T10:01:01.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.365+0000 7fce4ce00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fce48104010 0x7fce48073c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:01.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.365+0000 7fce4ce00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fce481049c0 0x7fce4806dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:01.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.365+0000 7fce4ce00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce48109140 0x7fce4806e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:01.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.365+0000 7fce4ce00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fce4806c660 con 0x7fce481049c0 2024-09-10T10:01:01.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.365+0000 7fce4ce00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fce4806c4e0 con 0x7fce48104010 2024-09-10T10:01:01.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.365+0000 7fce4ce00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fce4806c7e0 con 0x7fce48109140 2024-09-10T10:01:01.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.365+0000 7fce45e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fce48104010 0x7fce48073c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:01.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.366+0000 7fce45e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fce48104010 0x7fce48073c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:56696/0 (socket says 172.21.15.77:56696) 2024-09-10T10:01:01.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.366+0000 7fce45e00640 1 -- 172.21.15.77:0/4258050322 learned_addr learned my addr 172.21.15.77:0/4258050322 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:01:01.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.366+0000 7fce45e00640 1 -- 172.21.15.77:0/4258050322 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce48109140 msgr2=0x7fce4806e1c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T10:01:01.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.366+0000 7fce46800640 1 --2- 172.21.15.77:0/4258050322 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce48109140 0x7fce4806e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:01.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.366+0000 7fce45e00640 1 --2- 172.21.15.77:0/4258050322 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce48109140 0x7fce4806e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:01.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.366+0000 7fce45e00640 1 -- 172.21.15.77:0/4258050322 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fce481049c0 msgr2=0x7fce4806dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:01.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.366+0000 7fce45e00640 1 --2- 172.21.15.77:0/4258050322 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fce481049c0 0x7fce4806dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:01.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.366+0000 7fce45e00640 1 -- 172.21.15.77:0/4258050322 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce4806e990 con 0x7fce48104010 2024-09-10T10:01:01.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.367+0000 7fce45e00640 1 --2- 172.21.15.77:0/4258050322 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fce48104010 0x7fce48073c00 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fce38006010 tx=0x7fce38004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:01.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.367+0000 7fce3ea00640 1 -- 172.21.15.77:0/4258050322 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce3803f030 con 0x7fce48104010 2024-09-10T10:01:01.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.367+0000 7fce3ea00640 1 -- 172.21.15.77:0/4258050322 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce38004440 con 0x7fce48104010 2024-09-10T10:01:01.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.367+0000 7fce4ce00640 1 -- 172.21.15.77:0/4258050322 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce481af440 con 0x7fce48104010 2024-09-10T10:01:01.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.367+0000 7fce3ea00640 1 -- 172.21.15.77:0/4258050322 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce38035070 con 0x7fce48104010 2024-09-10T10:01:01.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.367+0000 7fce4ce00640 1 -- 172.21.15.77:0/4258050322 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce481af920 con 0x7fce48104010 2024-09-10T10:01:01.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.372+0000 7fce4f81b740 1 Processor -- start 2024-09-10T10:01:01.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.372+0000 7fce4f81b740 1 -- start start 2024-09-10T10:01:01.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.372+0000 7fce3ea00640 1 -- 172.21.15.77:0/4258050322 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fce38042030 con 0x7fce48104010 2024-09-10T10:01:01.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.372+0000 7fce4f81b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bfa2e97b90 0x55bfa2e97f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:01.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.372+0000 7fce4f81b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bfa2e984b0 0x55bfa2eab120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:01.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.372+0000 7fce4f81b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bfa2eab660 0x55bfa2eada50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:01.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.372+0000 7fce4f81b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55bfa2dfe3e0 con 0x55bfa2e984b0 2024-09-10T10:01:01.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.372+0000 7fce4f81b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55bfa2da5d80 con 0x55bfa2eab660 2024-09-10T10:01:01.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.372+0000 7fce4f81b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55bfa2dfa3a0 con 0x55bfa2e97b90 2024-09-10T10:01:01.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.373+0000 7fce46800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bfa2e97b90 0x55bfa2e97f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:01.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.373+0000 7fce46800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bfa2e97b90 0x55bfa2e97f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:47482/0 (socket says 172.21.15.77:47482) 2024-09-10T10:01:01.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.373+0000 7fce46800640 1 -- 172.21.15.77:0/142388934 learned_addr learned my addr 172.21.15.77:0/142388934 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:01:01.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.373+0000 7fce45e00640 1 --2- 172.21.15.77:0/142388934 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bfa2e984b0 0x55bfa2eab120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:01.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.373+0000 7fce46800640 1 -- 172.21.15.77:0/142388934 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bfa2eab660 msgr2=0x55bfa2eada50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T10:01:01.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.373+0000 7fce46800640 1 --2- 172.21.15.77:0/142388934 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bfa2eab660 0x55bfa2eada50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:01.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.373+0000 7fce46800640 1 -- 172.21.15.77:0/142388934 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bfa2e984b0 msgr2=0x55bfa2eab120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:01.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.373+0000 7fce46800640 1 --2- 172.21.15.77:0/142388934 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bfa2e984b0 0x55bfa2eab120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:01.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.373+0000 7fce46800640 1 -- 172.21.15.77:0/142388934 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bfa2dc4ae0 con 0x55bfa2e97b90 2024-09-10T10:01:01.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.373+0000 7fce46800640 1 --2- 172.21.15.77:0/142388934 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bfa2e97b90 0x55bfa2e97f70 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fce40009d80 tx=0x7fce4000f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e085b65c52fb8308 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:01.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.374+0000 7fce33400640 1 -- 172.21.15.77:0/142388934 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce40011440 con 0x55bfa2e97b90 2024-09-10T10:01:01.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.374+0000 7fce33400640 1 -- 172.21.15.77:0/142388934 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce40011ac0 con 0x55bfa2e97b90 2024-09-10T10:01:01.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.374+0000 7fce33400640 1 -- 172.21.15.77:0/142388934 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce40010c20 con 0x55bfa2e97b90 2024-09-10T10:01:01.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.374+0000 7fce3ea00640 1 --2- 172.21.15.77:0/4258050322 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fce2c0a2c00 0x7fce2c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:01.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.374+0000 7fce3ea00640 1 -- 172.21.15.77:0/4258050322 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fce380ef5b0 con 0x7fce48104010 2024-09-10T10:01:01.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.374+0000 7fce45e00640 1 --2- 172.21.15.77:0/4258050322 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fce2c0a2c00 0x7fce2c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:01.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.374+0000 7fce4f81b740 1 -- 172.21.15.77:0/142388934 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bfa2e97b90 msgr2=0x55bfa2e97f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:01.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.375+0000 7fce4f81b740 1 --2- 172.21.15.77:0/142388934 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bfa2e97b90 0x55bfa2e97f70 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fce40009d80 tx=0x7fce4000f730 comp rx=0 tx=0).stop 2024-09-10T10:01:01.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.375+0000 7fce4f81b740 1 -- 172.21.15.77:0/142388934 shutdown_connections 2024-09-10T10:01:01.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.375+0000 7fce4f81b740 1 --2- 172.21.15.77:0/142388934 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bfa2eab660 0x55bfa2eada50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:01.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.375+0000 7fce4f81b740 1 --2- 172.21.15.77:0/142388934 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bfa2e984b0 0x55bfa2eab120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:01.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.375+0000 7fce4f81b740 1 --2- 172.21.15.77:0/142388934 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bfa2e97b90 0x55bfa2e97f70 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:01.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.375+0000 7fce4f81b740 1 -- 172.21.15.77:0/142388934 >> 172.21.15.77:0/142388934 conn(0x55bfa2e94200 msgr2=0x55bfa2e945e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:01.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.375+0000 7fce4f81b740 1 -- 172.21.15.77:0/142388934 shutdown_connections 2024-09-10T10:01:01.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.375+0000 7fce4f81b740 1 -- 172.21.15.77:0/142388934 wait complete. 2024-09-10T10:01:01.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.376+0000 7fce4f81b740 1 -- start start 2024-09-10T10:01:01.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.376+0000 7fce4f81b740 1 Processor -- start 2024-09-10T10:01:01.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.376+0000 7fce45e00640 1 --2- 172.21.15.77:0/4258050322 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fce2c0a2c00 0x7fce2c0a50c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fce380eccc0 tx=0x7fce380f50e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:01.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.376+0000 7fce4f81b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bfa2e97b90 0x55bfa2ea8600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:01.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.377+0000 7fce4f81b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bfa2e984b0 0x55bfa2ea8b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:01.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.377+0000 7fce4f81b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bfa2eab660 0x55bfa2ea9080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:01.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.377+0000 7fce46800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bfa2e97b90 0x55bfa2ea8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:01.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.377+0000 7fce46800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bfa2e97b90 0x55bfa2ea8600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:41142/0 (socket says 172.21.15.77:41142) 2024-09-10T10:01:01.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.377+0000 7fce46800640 1 -- 172.21.15.77:0/1805747445 learned_addr learned my addr 172.21.15.77:0/1805747445 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:01:01.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.377+0000 7fce4f81b740 1 -- 172.21.15.77:0/1805747445 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55bfa2eae1d0 con 0x55bfa2e97b90 2024-09-10T10:01:01.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.377+0000 7fce4f81b740 1 -- 172.21.15.77:0/1805747445 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55bfa2db90f0 con 0x55bfa2e984b0 2024-09-10T10:01:01.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.377+0000 7fce4f81b740 1 -- 172.21.15.77:0/1805747445 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55bfa2eae350 con 0x55bfa2eab660 2024-09-10T10:01:01.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.377+0000 7fce45400640 1 --2- 172.21.15.77:0/1805747445 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bfa2e984b0 0x55bfa2ea8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:01.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.377+0000 7fce46800640 1 --2- 172.21.15.77:0/1805747445 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bfa2eab660 0x55bfa2ea9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:01.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.377+0000 7fce4f81b740 10 client.0 _ll_register_callbacks cb 0x55bfa2da0bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:01:01.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.378+0000 7fce46800640 1 -- 172.21.15.77:0/1805747445 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bfa2e984b0 msgr2=0x55bfa2ea8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:01.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.378+0000 7fce46800640 1 --2- 172.21.15.77:0/1805747445 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bfa2e984b0 0x55bfa2ea8b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:01.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.378+0000 7fce46800640 1 -- 172.21.15.77:0/1805747445 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bfa2e97b90 msgr2=0x55bfa2ea8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:01.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.378+0000 7fce46800640 1 --2- 172.21.15.77:0/1805747445 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bfa2e97b90 0x55bfa2ea8600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:01.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.378+0000 7fce46800640 1 -- 172.21.15.77:0/1805747445 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bfa2b6f3c0 con 0x55bfa2eab660 2024-09-10T10:01:01.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.378+0000 7fce46800640 1 --2- 172.21.15.77:0/1805747445 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bfa2eab660 0x55bfa2ea9080 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fce400094d0 tx=0x7fce40009500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9faa0055a5528a2e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:01.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.378+0000 7fce27e00640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-10T10:01:01.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.378+0000 7fce27e00640 1 -- 172.21.15.77:0/1805747445 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce40024040 con 0x55bfa2eab660 2024-09-10T10:01:01.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.378+0000 7fce4f81b740 1 -- 172.21.15.77:0/1805747445 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce40011ac0 con 0x55bfa2eab660 2024-09-10T10:01:01.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.378+0000 7fce27e00640 1 -- 172.21.15.77:0/1805747445 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fce40010db0 con 0x55bfa2eab660 2024-09-10T10:01:01.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.379+0000 7fce27e00640 1 -- 172.21.15.77:0/1805747445 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fce40011440 con 0x55bfa2eab660 2024-09-10T10:01:01.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.379+0000 7fce4f81b740 10 client.26629 fetch_fsmap learned FSMap version 34 2024-09-10T10:01:01.384 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.379+0000 7fce4f81b740 1 -- 172.21.15.77:0/1805747445 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55bfa2eba300 con 0x55bfa2eab660 2024-09-10T10:01:01.384 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.379+0000 7fce27e00640 1 -- 172.21.15.77:0/1805747445 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce40011ac0 con 0x55bfa2eab660 2024-09-10T10:01:01.384 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.380+0000 7fce27e00640 1 -- 172.21.15.77:0/1805747445 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 34) ==== 8374+0+0 (secure 0 0 0) 0x7fce40004040 con 0x55bfa2eab660 2024-09-10T10:01:01.384 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.380+0000 7fce4f81b740 10 client.26629 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T10:01:01.384 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.380+0000 7fce4f81b740 10 client.26629 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T10:01:01.384 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.380+0000 7fce4f81b740 20 client.26629 populate_metadata read hostname 'smithi077' 2024-09-10T10:01:01.384 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.380+0000 7fce4f81b740 1 --2- 172.21.15.77:0/1805747445 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x55bfa2ebb6b0 0x55bfa2ebdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:01.384 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.380+0000 7fce4f81b740 4 client.26629 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:01:01.384 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.380+0000 7fce4f81b740 1 -- 172.21.15.77:0/1805747445 --> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bfa2e0b950 con 0x55bfa2ebb6b0 2024-09-10T10:01:01.384 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.380+0000 7fce45400640 1 --2- 172.21.15.77:0/1805747445 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x55bfa2ebb6b0 0x55bfa2ebdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:01.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.381+0000 7fce45400640 1 --2- 172.21.15.77:0/1805747445 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x55bfa2ebb6b0 0x55bfa2ebdaa0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c7355b2263121aa7 server_cookie=88f530638ea53f4a in_seq=0 out_seq=0 2024-09-10T10:01:01.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.381+0000 7fce27e00640 10 client.26629 ms_handle_connect on v2:172.21.15.134:6838/1965117680 2024-09-10T10:01:01.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.382+0000 7fce27e00640 1 -- 172.21.15.77:0/1805747445 <== mds.2 v2:172.21.15.134:6838/1965117680 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bfa2e0b950 con 0x55bfa2ebb6b0 2024-09-10T10:01:01.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.382+0000 7fce27e00640 10 client.26629 handle_command_reply: tid=0 2024-09-10T10:01:01.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.382+0000 7fce4f81b740 1 client.26629 shutdown 2024-09-10T10:01:01.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.382+0000 7fce4f81b740 1 -- 172.21.15.77:0/1805747445 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bfa2eab660 msgr2=0x55bfa2ea9080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:01.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.382+0000 7fce4f81b740 1 --2- 172.21.15.77:0/1805747445 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bfa2eab660 0x55bfa2ea9080 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fce400094d0 tx=0x7fce40009500 comp rx=0 tx=0).stop 2024-09-10T10:01:01.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.382+0000 7fce4f81b740 1 -- 172.21.15.77:0/1805747445 shutdown_connections 2024-09-10T10:01:01.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.382+0000 7fce4f81b740 1 --2- 172.21.15.77:0/1805747445 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x55bfa2ebb6b0 0x55bfa2ebdaa0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:01.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.382+0000 7fce4f81b740 1 --2- 172.21.15.77:0/1805747445 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bfa2eab660 0x55bfa2ea9080 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:01.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.382+0000 7fce4f81b740 1 --2- 172.21.15.77:0/1805747445 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bfa2e984b0 0x55bfa2ea8b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:01.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.382+0000 7fce4f81b740 1 --2- 172.21.15.77:0/1805747445 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bfa2e97b90 0x55bfa2ea8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:01.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.382+0000 7fce27e00640 0 client.26629 ms_handle_reset on v2:172.21.15.134:6838/1965117680 2024-09-10T10:01:01.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.382+0000 7fce4f81b740 1 -- 172.21.15.77:0/1805747445 >> 172.21.15.77:0/1805747445 conn(0x55bfa2e94200 msgr2=0x55bfa2e93380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:01.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.382+0000 7fce4f81b740 1 -- 172.21.15.77:0/1805747445 shutdown_connections 2024-09-10T10:01:01.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.382+0000 7fce4f81b740 1 -- 172.21.15.77:0/1805747445 wait complete. 2024-09-10T10:01:01.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.383+0000 7fce4f81b740 20 client.26629 trim_cache size 0 max 16384 2024-09-10T10:01:01.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.399+0000 7fce4f81b740 1 Processor -- start 2024-09-10T10:01:01.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.399+0000 7fce4f81b740 1 -- start start 2024-09-10T10:01:01.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.399+0000 7fce4f81b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bfa2e97b90 0x55bfa2ea6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:01.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.399+0000 7fce4f81b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bfa2e984b0 0x55bfa2ea8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:01.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.399+0000 7fce4f81b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bfa2eab660 0x55bfa2ec31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:01.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.399+0000 7fce4f81b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fce481b54e0 con 0x55bfa2e97b90 2024-09-10T10:01:01.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.399+0000 7fce4f81b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fce481b7050 con 0x55bfa2e984b0 2024-09-10T10:01:01.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.399+0000 7fce4f81b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fce481b5bb0 con 0x55bfa2eab660 2024-09-10T10:01:01.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.400+0000 7fce46800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bfa2e97b90 0x55bfa2ea6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:01.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.400+0000 7fce45400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bfa2e984b0 0x55bfa2ea8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:01.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.400+0000 7fce45400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bfa2e984b0 0x55bfa2ea8ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:56718/0 (socket says 172.21.15.77:56718) 2024-09-10T10:01:01.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.400+0000 7fce45400640 1 -- 172.21.15.77:0/3635358658 learned_addr learned my addr 172.21.15.77:0/3635358658 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:01:01.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.400+0000 7fce46800640 1 --2- 172.21.15.77:0/3635358658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bfa2eab660 0x55bfa2ec31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:01.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.400+0000 7fce45400640 1 -- 172.21.15.77:0/3635358658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bfa2eab660 msgr2=0x55bfa2ec31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:01.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.400+0000 7fce45400640 1 --2- 172.21.15.77:0/3635358658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bfa2eab660 0x55bfa2ec31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:01.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.400+0000 7fce45400640 1 -- 172.21.15.77:0/3635358658 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bfa2e97b90 msgr2=0x55bfa2ea6ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:01.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.400+0000 7fce45400640 1 --2- 172.21.15.77:0/3635358658 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bfa2e97b90 0x55bfa2ea6ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:01.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.400+0000 7fce45400640 1 -- 172.21.15.77:0/3635358658 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bfa2eb7fe0 con 0x55bfa2e984b0 2024-09-10T10:01:01.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.400+0000 7fce46800640 1 --2- 172.21.15.77:0/3635358658 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bfa2e97b90 0x55bfa2ea6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:01:01.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.400+0000 7fce45400640 1 --2- 172.21.15.77:0/3635358658 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bfa2e984b0 0x55bfa2ea8ed0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fce34009b30 tx=0x7fce34009370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23d6c3664212558b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:01.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.401+0000 7fce32a00640 1 -- 172.21.15.77:0/3635358658 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce3401b5b0 con 0x55bfa2e984b0 2024-09-10T10:01:01.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.401+0000 7fce32a00640 1 -- 172.21.15.77:0/3635358658 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce34013e20 con 0x55bfa2e984b0 2024-09-10T10:01:01.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.401+0000 7fce32a00640 1 -- 172.21.15.77:0/3635358658 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce34011dc0 con 0x55bfa2e984b0 2024-09-10T10:01:01.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.402+0000 7fce4f81b740 1 -- 172.21.15.77:0/3635358658 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bfa2e984b0 msgr2=0x55bfa2ea8ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:01.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.402+0000 7fce4f81b740 1 --2- 172.21.15.77:0/3635358658 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bfa2e984b0 0x55bfa2ea8ed0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fce34009b30 tx=0x7fce34009370 comp rx=0 tx=0).stop 2024-09-10T10:01:01.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.402+0000 7fce4f81b740 1 -- 172.21.15.77:0/3635358658 shutdown_connections 2024-09-10T10:01:01.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.402+0000 7fce4f81b740 1 --2- 172.21.15.77:0/3635358658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bfa2eab660 0x55bfa2ec31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:01.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.402+0000 7fce4f81b740 1 --2- 172.21.15.77:0/3635358658 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bfa2e984b0 0x55bfa2ea8ed0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:01.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.402+0000 7fce4f81b740 1 --2- 172.21.15.77:0/3635358658 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bfa2e97b90 0x55bfa2ea6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:01.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.402+0000 7fce4f81b740 1 -- 172.21.15.77:0/3635358658 >> 172.21.15.77:0/3635358658 conn(0x55bfa2e94200 msgr2=0x55bfa2eaa680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:01.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.402+0000 7fce4f81b740 1 -- 172.21.15.77:0/3635358658 shutdown_connections 2024-09-10T10:01:01.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.403+0000 7fce4f81b740 1 -- 172.21.15.77:0/3635358658 wait complete. 2024-09-10T10:01:01.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.403+0000 7fce4f81b740 1 -- start start 2024-09-10T10:01:01.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.403+0000 7fce4f81b740 1 Processor -- start 2024-09-10T10:01:01.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.404+0000 7fce4f81b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bfa2e97b90 0x55bfa2eb39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:01.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.405+0000 7fce4f81b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bfa2e984b0 0x55bfa2eb3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:01.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.405+0000 7fce46800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bfa2e97b90 0x55bfa2eb39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:01.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.405+0000 7fce46800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bfa2e97b90 0x55bfa2eb39c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:56730/0 (socket says 172.21.15.77:56730) 2024-09-10T10:01:01.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.405+0000 7fce46800640 1 -- 172.21.15.77:0/501571297 learned_addr learned my addr 172.21.15.77:0/501571297 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:01:01.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.405+0000 7fce4f81b740 1 --2- 172.21.15.77:0/501571297 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bfa2eab660 0x55bfa2eaef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:01.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.405+0000 7fce4f81b740 1 -- 172.21.15.77:0/501571297 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fce481b6950 con 0x55bfa2e984b0 2024-09-10T10:01:01.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.405+0000 7fce4f81b740 1 -- 172.21.15.77:0/501571297 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fce481b6280 con 0x55bfa2e97b90 2024-09-10T10:01:01.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.405+0000 7fce4f81b740 1 -- 172.21.15.77:0/501571297 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fce481b34c0 con 0x55bfa2eab660 2024-09-10T10:01:01.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.405+0000 7fce4f81b740 10 client.0 _ll_register_callbacks cb 0x7fce380f2f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:01:01.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.405+0000 7fce45400640 1 --2- 172.21.15.77:0/501571297 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bfa2e984b0 0x55bfa2eb3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:01.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.406+0000 7fce45400640 1 -- 172.21.15.77:0/501571297 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bfa2eab660 msgr2=0x55bfa2eaef40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T10:01:01.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.406+0000 7fce45400640 1 --2- 172.21.15.77:0/501571297 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bfa2eab660 0x55bfa2eaef40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:01.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.406+0000 7fce45400640 1 -- 172.21.15.77:0/501571297 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bfa2e97b90 msgr2=0x55bfa2eb39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:01.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.406+0000 7fce45400640 1 --2- 172.21.15.77:0/501571297 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bfa2e97b90 0x55bfa2eb39c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:01.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.406+0000 7fce45400640 1 -- 172.21.15.77:0/501571297 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bfa2eaf770 con 0x55bfa2e984b0 2024-09-10T10:01:01.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.406+0000 7fce46800640 1 --2- 172.21.15.77:0/501571297 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bfa2e97b90 0x55bfa2eb39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:01:01.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.407+0000 7fce45400640 1 --2- 172.21.15.77:0/501571297 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bfa2e984b0 0x55bfa2eb3f00 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x55bfa2ea6d30 tx=0x7fce34011620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=427144872ad02ec6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:01.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.407+0000 7fce27e00640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-10T10:01:01.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.407+0000 7fce27e00640 1 -- 172.21.15.77:0/501571297 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce34011a40 con 0x55bfa2e984b0 2024-09-10T10:01:01.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.407+0000 7fce27e00640 1 -- 172.21.15.77:0/501571297 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fce34011be0 con 0x55bfa2e984b0 2024-09-10T10:01:01.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.407+0000 7fce4f81b740 1 -- 172.21.15.77:0/501571297 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce34013e20 con 0x55bfa2e984b0 2024-09-10T10:01:01.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.408+0000 7fce27e00640 1 -- 172.21.15.77:0/501571297 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fce34032050 con 0x55bfa2e984b0 2024-09-10T10:01:01.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.408+0000 7fce4f81b740 10 client.16575 fetch_fsmap learned FSMap version 34 2024-09-10T10:01:01.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.408+0000 7fce4f81b740 1 -- 172.21.15.77:0/501571297 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55bfa2f25e20 con 0x55bfa2e984b0 2024-09-10T10:01:01.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.408+0000 7fce27e00640 1 -- 172.21.15.77:0/501571297 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce34013e20 con 0x55bfa2e984b0 2024-09-10T10:01:01.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.409+0000 7fce27e00640 1 -- 172.21.15.77:0/501571297 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8374+0+0 (secure 0 0 0) 0x7fce34004ce0 con 0x55bfa2e984b0 2024-09-10T10:01:01.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.409+0000 7fce4f81b740 10 client.16575 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T10:01:01.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.409+0000 7fce4f81b740 10 client.16575 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T10:01:01.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.409+0000 7fce4f81b740 20 client.16575 populate_metadata read hostname 'smithi077' 2024-09-10T10:01:01.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.409+0000 7fce4f81b740 1 --2- 172.21.15.77:0/501571297 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x55bfa2f264d0 0x55bfa2f28910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:01.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.409+0000 7fce4f81b740 4 client.16575 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:01:01.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.409+0000 7fce4f81b740 1 -- 172.21.15.77:0/501571297 --> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bfa2e05e60 con 0x55bfa2f264d0 2024-09-10T10:01:01.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.409+0000 7fce46800640 1 --2- 172.21.15.77:0/501571297 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x55bfa2f264d0 0x55bfa2f28910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:01.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.410+0000 7fce46800640 1 --2- 172.21.15.77:0/501571297 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x55bfa2f264d0 0x55bfa2f28910 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=302b39dd8b1bc73c server_cookie=3acaa939b32e585f in_seq=0 out_seq=0 2024-09-10T10:01:01.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.410+0000 7fce27e00640 10 client.16575 ms_handle_connect on v2:172.21.15.134:6838/1965117680 2024-09-10T10:01:01.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.410+0000 7fce27e00640 1 -- 172.21.15.77:0/501571297 <== mds.2 v2:172.21.15.134:6838/1965117680 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bfa2e05e60 con 0x55bfa2f264d0 2024-09-10T10:01:01.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.410+0000 7fce27e00640 10 client.16575 handle_command_reply: tid=0 2024-09-10T10:01:01.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.410+0000 7fce4f81b740 1 client.16575 shutdown 2024-09-10T10:01:01.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.410+0000 7fce4f81b740 1 -- 172.21.15.77:0/501571297 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bfa2e984b0 msgr2=0x55bfa2eb3f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:01.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.410+0000 7fce4f81b740 1 --2- 172.21.15.77:0/501571297 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bfa2e984b0 0x55bfa2eb3f00 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x55bfa2ea6d30 tx=0x7fce34011620 comp rx=0 tx=0).stop 2024-09-10T10:01:01.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.410+0000 7fce4f81b740 1 -- 172.21.15.77:0/501571297 shutdown_connections 2024-09-10T10:01:01.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.410+0000 7fce4f81b740 1 --2- 172.21.15.77:0/501571297 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x55bfa2f264d0 0x55bfa2f28910 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:01.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.410+0000 7fce4f81b740 1 --2- 172.21.15.77:0/501571297 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bfa2eab660 0x55bfa2eaef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:01.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.410+0000 7fce4f81b740 1 --2- 172.21.15.77:0/501571297 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bfa2e984b0 0x55bfa2eb3f00 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:01.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.410+0000 7fce4f81b740 1 --2- 172.21.15.77:0/501571297 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bfa2e97b90 0x55bfa2eb39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:01.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.410+0000 7fce4f81b740 1 -- 172.21.15.77:0/501571297 >> 172.21.15.77:0/501571297 conn(0x55bfa2e94200 msgr2=0x55bfa2e92b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:01.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.410+0000 7fce27e00640 0 client.16575 ms_handle_reset on v2:172.21.15.134:6838/1965117680 2024-09-10T10:01:01.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.411+0000 7fce4f81b740 1 -- 172.21.15.77:0/501571297 shutdown_connections 2024-09-10T10:01:01.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.411+0000 7fce4f81b740 1 -- 172.21.15.77:0/501571297 wait complete. 2024-09-10T10:01:01.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.411+0000 7fce4f81b740 20 client.16575 trim_cache size 0 max 16384 2024-09-10T10:01:01.416 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-10T10:01:01.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.411+0000 7fce4ce00640 1 -- 172.21.15.77:0/4258050322 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fce2c0a2c00 msgr2=0x7fce2c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:01.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.411+0000 7fce4ce00640 1 --2- 172.21.15.77:0/4258050322 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fce2c0a2c00 0x7fce2c0a50c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fce380eccc0 tx=0x7fce380f50e0 comp rx=0 tx=0).stop 2024-09-10T10:01:01.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.412+0000 7fce4ce00640 1 -- 172.21.15.77:0/4258050322 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fce48104010 msgr2=0x7fce48073c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:01.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.412+0000 7fce4ce00640 1 --2- 172.21.15.77:0/4258050322 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fce48104010 0x7fce48073c00 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fce38006010 tx=0x7fce38004040 comp rx=0 tx=0).stop 2024-09-10T10:01:01.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.412+0000 7fce4ce00640 1 -- 172.21.15.77:0/4258050322 shutdown_connections 2024-09-10T10:01:01.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.412+0000 7fce4ce00640 1 --2- 172.21.15.77:0/4258050322 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fce2c0a2c00 0x7fce2c0a50c0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:01.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.412+0000 7fce4ce00640 1 --2- 172.21.15.77:0/4258050322 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce48109140 0x7fce4806e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:01.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.412+0000 7fce4ce00640 1 --2- 172.21.15.77:0/4258050322 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fce481049c0 0x7fce4806dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:01.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.412+0000 7fce4ce00640 1 --2- 172.21.15.77:0/4258050322 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fce48104010 0x7fce48073c00 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:01.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.412+0000 7fce4ce00640 1 -- 172.21.15.77:0/4258050322 >> 172.21.15.77:0/4258050322 conn(0x7fce480ffa50 msgr2=0x7fce48100e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:01.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.412+0000 7fce4ce00640 1 -- 172.21.15.77:0/4258050322 shutdown_connections 2024-09-10T10:01:01.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:01.412+0000 7fce4ce00640 1 -- 172.21.15.77:0/4258050322 wait complete. 2024-09-10T10:01:02.118 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:01:01 smithi077 ceph-mon[30319]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T10:01:02.122 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:01:01 smithi120 ceph-mon[34770]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T10:01:02.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:01:01 smithi134 ceph-mon[35674]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T10:01:02.641 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T10:01:03.099 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:01:03.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:01:02 smithi120 ceph-mon[34770]: pgmap v567: 129 pgs: 129 active+clean; 114 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 1.9 MiB/s wr, 338 op/s 2024-09-10T10:01:03.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:01:02 smithi077 ceph-mon[30319]: pgmap v567: 129 pgs: 129 active+clean; 114 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 1.9 MiB/s wr, 338 op/s 2024-09-10T10:01:03.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:01:02 smithi134 ceph-mon[35674]: pgmap v567: 129 pgs: 129 active+clean; 114 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 1.9 MiB/s wr, 338 op/s 2024-09-10T10:01:03.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.725+0000 7f8001000640 1 -- 172.21.15.77:0/4279463937 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7ffc102790 msgr2=0x7f7ffc1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:03.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.725+0000 7f8001000640 1 --2- 172.21.15.77:0/4279463937 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7ffc102790 0x7f7ffc1067e0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f7fe4009900 tx=0x7f7fe402a3f0 comp rx=0 tx=0).stop 2024-09-10T10:01:03.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.725+0000 7f8001000640 1 -- 172.21.15.77:0/4279463937 shutdown_connections 2024-09-10T10:01:03.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.725+0000 7f8001000640 1 --2- 172.21.15.77:0/4279463937 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ffc106f10 0x7f7ffc10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:03.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.725+0000 7f8001000640 1 --2- 172.21.15.77:0/4279463937 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7ffc102790 0x7f7ffc1067e0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:03.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.725+0000 7f8001000640 1 --2- 172.21.15.77:0/4279463937 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ffc101de0 0x7f7ffc1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:03.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.725+0000 7f8001000640 1 -- 172.21.15.77:0/4279463937 >> 172.21.15.77:0/4279463937 conn(0x7f7ffc0fd8c0 msgr2=0x7f7ffc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:03.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.727+0000 7f8001000640 1 -- 172.21.15.77:0/4279463937 shutdown_connections 2024-09-10T10:01:03.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.727+0000 7f8001000640 1 -- 172.21.15.77:0/4279463937 wait complete. 2024-09-10T10:01:03.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.728+0000 7f8001000640 1 Processor -- start 2024-09-10T10:01:03.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.728+0000 7f8001000640 1 -- start start 2024-09-10T10:01:03.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.728+0000 7f8001000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7ffc101de0 0x7f7ffc19a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:03.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.728+0000 7f8001000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ffc102790 0x7f7ffc19ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:03.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.728+0000 7f8001000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ffc106f10 0x7f7ffc19ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:03.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.728+0000 7f8001000640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7ffc111fc0 con 0x7f7ffc101de0 2024-09-10T10:01:03.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.728+0000 7f8001000640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7ffc111e40 con 0x7f7ffc102790 2024-09-10T10:01:03.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.728+0000 7f8001000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7ffc112140 con 0x7f7ffc106f10 2024-09-10T10:01:03.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.728+0000 7f7ff9e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7ffc101de0 0x7f7ffc19a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:03.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.728+0000 7f7ff3e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ffc102790 0x7f7ffc19ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:03.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.728+0000 7f7ff3e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ffc102790 0x7f7ffc19ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:56296/0 (socket says 172.21.15.77:56296) 2024-09-10T10:01:03.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.728+0000 7f7ff3e00640 1 -- 172.21.15.77:0/1443311759 learned_addr learned my addr 172.21.15.77:0/1443311759 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:01:03.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.729+0000 7f7ffa800640 1 --2- 172.21.15.77:0/1443311759 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ffc106f10 0x7f7ffc19ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:03.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.729+0000 7f7ff3e00640 1 -- 172.21.15.77:0/1443311759 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ffc106f10 msgr2=0x7f7ffc19ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:03.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.729+0000 7f7ff3e00640 1 --2- 172.21.15.77:0/1443311759 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ffc106f10 0x7f7ffc19ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:03.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.729+0000 7f7ff3e00640 1 -- 172.21.15.77:0/1443311759 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7ffc101de0 msgr2=0x7f7ffc19a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:03.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.729+0000 7f7ff3e00640 1 --2- 172.21.15.77:0/1443311759 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7ffc101de0 0x7f7ffc19a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:03.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.729+0000 7f7ff3e00640 1 -- 172.21.15.77:0/1443311759 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ffc19f5f0 con 0x7f7ffc102790 2024-09-10T10:01:03.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.729+0000 7f7ffa800640 1 --2- 172.21.15.77:0/1443311759 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ffc106f10 0x7f7ffc19ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:01:03.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.729+0000 7f7ff3e00640 1 --2- 172.21.15.77:0/1443311759 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ffc102790 0x7f7ffc19ab80 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7fe4004770 tx=0x7f7fe4003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:03.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.730+0000 7f7ff2a00640 1 -- 172.21.15.77:0/1443311759 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe403f070 con 0x7f7ffc102790 2024-09-10T10:01:03.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.730+0000 7f7ff2a00640 1 -- 172.21.15.77:0/1443311759 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fe4031ca0 con 0x7f7ffc102790 2024-09-10T10:01:03.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.730+0000 7f7ff2a00640 1 -- 172.21.15.77:0/1443311759 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe4035030 con 0x7f7ffc102790 2024-09-10T10:01:03.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.730+0000 7f8001000640 1 -- 172.21.15.77:0/1443311759 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ffc19f880 con 0x7f7ffc102790 2024-09-10T10:01:03.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.730+0000 7f8001000640 1 -- 172.21.15.77:0/1443311759 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ffc1a71c0 con 0x7f7ffc102790 2024-09-10T10:01:03.734 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.733+0000 7f7ff2a00640 1 -- 172.21.15.77:0/1443311759 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7fe4042070 con 0x7f7ffc102790 2024-09-10T10:01:03.734 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.733+0000 7f8003a90740 1 Processor -- start 2024-09-10T10:01:03.734 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.733+0000 7f8003a90740 1 -- start start 2024-09-10T10:01:03.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.734+0000 7f8003a90740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e532f62c70 0x55e532f63050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:03.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.734+0000 7f8003a90740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e532f63590 0x55e532f6d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:03.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.734+0000 7f8003a90740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e532f6de50 0x55e532f70240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:03.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.734+0000 7f8003a90740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e532ec93e0 con 0x55e532f63590 2024-09-10T10:01:03.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.734+0000 7f8003a90740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e532e70d80 con 0x55e532f62c70 2024-09-10T10:01:03.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.734+0000 7f8003a90740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e532ec53a0 con 0x55e532f6de50 2024-09-10T10:01:03.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.734+0000 7f7ffa800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e532f62c70 0x55e532f63050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:03.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.734+0000 7f7ff9e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e532f63590 0x55e532f6d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:03.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.734+0000 7f7ffa800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e532f62c70 0x55e532f63050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:56300/0 (socket says 172.21.15.77:56300) 2024-09-10T10:01:03.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.734+0000 7f7ffa800640 1 -- 172.21.15.77:0/2743440483 learned_addr learned my addr 172.21.15.77:0/2743440483 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:01:03.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.734+0000 7f7ff3e00640 1 --2- 172.21.15.77:0/2743440483 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e532f6de50 0x55e532f70240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:03.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.734+0000 7f7ffa800640 1 -- 172.21.15.77:0/2743440483 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e532f6de50 msgr2=0x55e532f70240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:03.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.734+0000 7f7ffa800640 1 --2- 172.21.15.77:0/2743440483 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e532f6de50 0x55e532f70240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:03.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.734+0000 7f7ffa800640 1 -- 172.21.15.77:0/2743440483 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e532f63590 msgr2=0x55e532f6d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:03.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.734+0000 7f7ffa800640 1 --2- 172.21.15.77:0/2743440483 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e532f63590 0x55e532f6d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:03.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.734+0000 7f7ffa800640 1 -- 172.21.15.77:0/2743440483 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e532e8fae0 con 0x55e532f62c70 2024-09-10T10:01:03.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.734+0000 7f7ff3e00640 1 --2- 172.21.15.77:0/2743440483 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e532f6de50 0x55e532f70240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:01:03.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.735+0000 7f7ff2a00640 1 --2- 172.21.15.77:0/1443311759 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f7fe00a2b80 0x7f7fe00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:03.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.735+0000 7f7ffa800640 1 --2- 172.21.15.77:0/2743440483 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e532f62c70 0x55e532f63050 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7ff4005f40 tx=0x7f7ff4010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a04199a6e4994f33 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:03.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.735+0000 7f7ff2a00640 1 -- 172.21.15.77:0/1443311759 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f7fe40ef7a0 con 0x7f7ffc102790 2024-09-10T10:01:03.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.735+0000 7f7feb400640 1 -- 172.21.15.77:0/2743440483 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff4011700 con 0x55e532f62c70 2024-09-10T10:01:03.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.735+0000 7f7ff9e00640 1 --2- 172.21.15.77:0/1443311759 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f7fe00a2b80 0x7f7fe00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:03.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.735+0000 7f7feb400640 1 -- 172.21.15.77:0/2743440483 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ff4011d80 con 0x55e532f62c70 2024-09-10T10:01:03.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.735+0000 7f7feb400640 1 -- 172.21.15.77:0/2743440483 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff400e3c0 con 0x55e532f62c70 2024-09-10T10:01:03.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.736+0000 7f8003a90740 1 -- 172.21.15.77:0/2743440483 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e532f62c70 msgr2=0x55e532f63050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:03.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.736+0000 7f7ff9e00640 1 --2- 172.21.15.77:0/1443311759 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f7fe00a2b80 0x7f7fe00a5040 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x55e532f60060 tx=0x7f7fec005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:03.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.736+0000 7f8003a90740 1 --2- 172.21.15.77:0/2743440483 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e532f62c70 0x55e532f63050 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7ff4005f40 tx=0x7f7ff4010860 comp rx=0 tx=0).stop 2024-09-10T10:01:03.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.736+0000 7f8003a90740 1 -- 172.21.15.77:0/2743440483 shutdown_connections 2024-09-10T10:01:03.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.736+0000 7f8003a90740 1 --2- 172.21.15.77:0/2743440483 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e532f6de50 0x55e532f70240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:03.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.736+0000 7f8003a90740 1 --2- 172.21.15.77:0/2743440483 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e532f63590 0x55e532f6d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:03.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.736+0000 7f8003a90740 1 --2- 172.21.15.77:0/2743440483 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e532f62c70 0x55e532f63050 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:03.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.736+0000 7f8003a90740 1 -- 172.21.15.77:0/2743440483 >> 172.21.15.77:0/2743440483 conn(0x55e532f5f200 msgr2=0x55e532f5f5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:03.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.736+0000 7f8003a90740 1 -- 172.21.15.77:0/2743440483 shutdown_connections 2024-09-10T10:01:03.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.736+0000 7f8003a90740 1 -- 172.21.15.77:0/2743440483 wait complete. 2024-09-10T10:01:03.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.737+0000 7f8003a90740 1 -- start start 2024-09-10T10:01:03.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.737+0000 7f8003a90740 1 Processor -- start 2024-09-10T10:01:03.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.738+0000 7f8003a90740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e532f62c70 0x55e532f6ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:03.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.738+0000 7f8003a90740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e532f63590 0x55e532f6b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:03.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.738+0000 7f8003a90740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e532f6de50 0x55e532f6b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:03.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.738+0000 7f8003a90740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e532f70a20 con 0x55e532f63590 2024-09-10T10:01:03.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.738+0000 7f8003a90740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e532e840f0 con 0x55e532f62c70 2024-09-10T10:01:03.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.738+0000 7f8003a90740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e532f70ba0 con 0x55e532f6de50 2024-09-10T10:01:03.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.738+0000 7f8003a90740 10 client.0 _ll_register_callbacks cb 0x55e532e6bbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:01:03.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.738+0000 7f7ff9e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e532f62c70 0x55e532f6ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:03.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.738+0000 7f7ff9e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e532f62c70 0x55e532f6ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:56314/0 (socket says 172.21.15.77:56314) 2024-09-10T10:01:03.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.738+0000 7f7ff9e00640 1 -- 172.21.15.77:0/3272892556 learned_addr learned my addr 172.21.15.77:0/3272892556 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:01:03.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.738+0000 7f7ff3e00640 1 --2- 172.21.15.77:0/3272892556 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e532f63590 0x55e532f6b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:03.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.738+0000 7f7ffa800640 1 --2- 172.21.15.77:0/3272892556 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e532f6de50 0x55e532f6b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:03.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.738+0000 7f7ff9e00640 1 -- 172.21.15.77:0/3272892556 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e532f6de50 msgr2=0x55e532f6b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:03.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.738+0000 7f7ff9e00640 1 --2- 172.21.15.77:0/3272892556 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e532f6de50 0x55e532f6b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:03.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.738+0000 7f7ff9e00640 1 -- 172.21.15.77:0/3272892556 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e532f63590 msgr2=0x55e532f6b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:03.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.738+0000 7f7ff9e00640 1 --2- 172.21.15.77:0/3272892556 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e532f63590 0x55e532f6b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:03.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.738+0000 7f7ff9e00640 1 -- 172.21.15.77:0/3272892556 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e532c3a3c0 con 0x55e532f62c70 2024-09-10T10:01:03.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.739+0000 7f7ff9e00640 1 --2- 172.21.15.77:0/3272892556 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e532f62c70 0x55e532f6ad60 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f7fec004610 tx=0x7f7fec004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a19b67cd3d123f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:03.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.739+0000 7f7fdfe00640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T10:01:03.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.739+0000 7f7fdfe00640 1 -- 172.21.15.77:0/3272892556 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fec01f610 con 0x55e532f62c70 2024-09-10T10:01:03.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.739+0000 7f8003a90740 1 -- 172.21.15.77:0/3272892556 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ff4011d80 con 0x55e532f62c70 2024-09-10T10:01:03.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.739+0000 7f7fdfe00640 1 -- 172.21.15.77:0/3272892556 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f7fec01f7b0 con 0x55e532f62c70 2024-09-10T10:01:03.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.739+0000 7f7fdfe00640 1 -- 172.21.15.77:0/3272892556 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f7fec01f9b0 con 0x55e532f62c70 2024-09-10T10:01:03.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.739+0000 7f8003a90740 10 client.26684 fetch_fsmap learned FSMap version 34 2024-09-10T10:01:03.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.739+0000 7f8003a90740 1 -- 172.21.15.77:0/3272892556 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55e532f7cae0 con 0x55e532f62c70 2024-09-10T10:01:03.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.740+0000 7f7fdfe00640 1 -- 172.21.15.77:0/3272892556 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ff4011d80 con 0x55e532f62c70 2024-09-10T10:01:03.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.740+0000 7f7fdfe00640 1 -- 172.21.15.77:0/3272892556 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 34) ==== 8374+0+0 (secure 0 0 0) 0x7f7fec027020 con 0x55e532f62c70 2024-09-10T10:01:03.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.740+0000 7f8003a90740 10 client.26684 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T10:01:03.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.740+0000 7f8003a90740 10 client.26684 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-10T10:01:03.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.740+0000 7f8003a90740 20 client.26684 populate_metadata read hostname 'smithi077' 2024-09-10T10:01:03.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.740+0000 7f8003a90740 1 --2- 172.21.15.77:0/3272892556 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x55e532f7def0 0x55e532f80340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:03.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.740+0000 7f8003a90740 4 client.26684 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:01:03.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.740+0000 7f8003a90740 1 -- 172.21.15.77:0/3272892556 --> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e532ed6950 con 0x55e532f7def0 2024-09-10T10:01:03.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.740+0000 7f7ff3e00640 1 --2- 172.21.15.77:0/3272892556 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x55e532f7def0 0x55e532f80340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:03.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.741+0000 7f7ff3e00640 1 --2- 172.21.15.77:0/3272892556 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x55e532f7def0 0x55e532f80340 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=57b315fd7e067455 server_cookie=78887efbb0716cba in_seq=0 out_seq=0 2024-09-10T10:01:03.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.741+0000 7f7fdfe00640 10 client.26684 ms_handle_connect on v2:172.21.15.120:6836/3033280709 2024-09-10T10:01:03.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.742+0000 7f7fdfe00640 1 -- 172.21.15.77:0/3272892556 <== mds.1 v2:172.21.15.120:6836/3033280709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e532ed6950 con 0x55e532f7def0 2024-09-10T10:01:03.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.742+0000 7f7fdfe00640 10 client.26684 handle_command_reply: tid=0 2024-09-10T10:01:03.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.742+0000 7f8003a90740 1 client.26684 shutdown 2024-09-10T10:01:03.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.742+0000 7f8003a90740 1 -- 172.21.15.77:0/3272892556 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e532f62c70 msgr2=0x55e532f6ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:03.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.742+0000 7f8003a90740 1 --2- 172.21.15.77:0/3272892556 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e532f62c70 0x55e532f6ad60 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f7fec004610 tx=0x7f7fec004730 comp rx=0 tx=0).stop 2024-09-10T10:01:03.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.742+0000 7f8003a90740 1 -- 172.21.15.77:0/3272892556 shutdown_connections 2024-09-10T10:01:03.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.742+0000 7f8003a90740 1 --2- 172.21.15.77:0/3272892556 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x55e532f7def0 0x55e532f80340 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:03.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.742+0000 7f8003a90740 1 --2- 172.21.15.77:0/3272892556 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e532f6de50 0x55e532f6b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:03.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.742+0000 7f8003a90740 1 --2- 172.21.15.77:0/3272892556 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e532f63590 0x55e532f6b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:03.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.742+0000 7f8003a90740 1 --2- 172.21.15.77:0/3272892556 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e532f62c70 0x55e532f6ad60 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:03.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.742+0000 7f8003a90740 1 -- 172.21.15.77:0/3272892556 >> 172.21.15.77:0/3272892556 conn(0x55e532f5f200 msgr2=0x55e532f64f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:03.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.742+0000 7f7fdfe00640 0 client.26684 ms_handle_reset on v2:172.21.15.120:6836/3033280709 2024-09-10T10:01:03.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.742+0000 7f8003a90740 1 -- 172.21.15.77:0/3272892556 shutdown_connections 2024-09-10T10:01:03.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.742+0000 7f8003a90740 1 -- 172.21.15.77:0/3272892556 wait complete. 2024-09-10T10:01:03.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.742+0000 7f8003a90740 20 client.26684 trim_cache size 0 max 16384 2024-09-10T10:01:03.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.759+0000 7f8003a90740 1 Processor -- start 2024-09-10T10:01:03.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.759+0000 7f8003a90740 1 -- start start 2024-09-10T10:01:03.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.760+0000 7f8003a90740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e532f62c70 0x55e532f85c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:03.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.760+0000 7f8003a90740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e532f63590 0x55e532f86150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:03.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.760+0000 7f8003a90740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e532f6de50 0x55e532f67fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:03.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.760+0000 7f8003a90740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7ffc1b10a0 con 0x55e532f6de50 2024-09-10T10:01:03.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.760+0000 7f8003a90740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7ffc1b2c10 con 0x55e532f62c70 2024-09-10T10:01:03.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.760+0000 7f8003a90740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7ffc1b1770 con 0x55e532f63590 2024-09-10T10:01:03.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.760+0000 7f7ff3e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e532f63590 0x55e532f86150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:03.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.760+0000 7f7ff9e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e532f62c70 0x55e532f85c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:03.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.760+0000 7f7ff3e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e532f63590 0x55e532f86150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:53062/0 (socket says 172.21.15.77:53062) 2024-09-10T10:01:03.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.760+0000 7f7ff3e00640 1 -- 172.21.15.77:0/1519626688 learned_addr learned my addr 172.21.15.77:0/1519626688 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:01:03.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.760+0000 7f7ff3e00640 1 -- 172.21.15.77:0/1519626688 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e532f62c70 msgr2=0x55e532f85c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:03.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.760+0000 7f7ff3e00640 1 --2- 172.21.15.77:0/1519626688 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e532f62c70 0x55e532f85c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:03.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.760+0000 7f7ff3e00640 1 -- 172.21.15.77:0/1519626688 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e532f6de50 msgr2=0x55e532f67fc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T10:01:03.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.760+0000 7f7ff3e00640 1 --2- 172.21.15.77:0/1519626688 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e532f6de50 0x55e532f67fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:03.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.760+0000 7f7ff3e00640 1 -- 172.21.15.77:0/1519626688 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e532f5f030 con 0x55e532f63590 2024-09-10T10:01:03.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.761+0000 7f7ff3e00640 1 --2- 172.21.15.77:0/1519626688 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e532f63590 0x55e532f86150 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f7fe4000c00 tx=0x7f7fe402a3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4dd5a1b45181721d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:03.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.761+0000 7f7feaa00640 1 -- 172.21.15.77:0/1519626688 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe4049040 con 0x55e532f63590 2024-09-10T10:01:03.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.761+0000 7f7feaa00640 1 -- 172.21.15.77:0/1519626688 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fe40c0020 con 0x55e532f63590 2024-09-10T10:01:03.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.761+0000 7f7feaa00640 1 -- 172.21.15.77:0/1519626688 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe40ef3f0 con 0x55e532f63590 2024-09-10T10:01:03.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.762+0000 7f8003a90740 1 -- 172.21.15.77:0/1519626688 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e532f63590 msgr2=0x55e532f86150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:03.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.762+0000 7f8003a90740 1 --2- 172.21.15.77:0/1519626688 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e532f63590 0x55e532f86150 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f7fe4000c00 tx=0x7f7fe402a3f0 comp rx=0 tx=0).stop 2024-09-10T10:01:03.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.762+0000 7f8003a90740 1 -- 172.21.15.77:0/1519626688 shutdown_connections 2024-09-10T10:01:03.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.762+0000 7f8003a90740 1 --2- 172.21.15.77:0/1519626688 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e532f6de50 0x55e532f67fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:03.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.762+0000 7f8003a90740 1 --2- 172.21.15.77:0/1519626688 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e532f63590 0x55e532f86150 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:03.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.762+0000 7f8003a90740 1 --2- 172.21.15.77:0/1519626688 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e532f62c70 0x55e532f85c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:03.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.762+0000 7f8003a90740 1 -- 172.21.15.77:0/1519626688 >> 172.21.15.77:0/1519626688 conn(0x55e532f5f200 msgr2=0x55e532f64f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:03.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.762+0000 7f8003a90740 1 -- 172.21.15.77:0/1519626688 shutdown_connections 2024-09-10T10:01:03.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.762+0000 7f8003a90740 1 -- 172.21.15.77:0/1519626688 wait complete. 2024-09-10T10:01:03.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.763+0000 7f8003a90740 1 -- start start 2024-09-10T10:01:03.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.763+0000 7f8003a90740 1 Processor -- start 2024-09-10T10:01:03.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.763+0000 7f8003a90740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e532f62c70 0x55e532f71b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:03.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.763+0000 7f8003a90740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e532f63590 0x55e532f72080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:03.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.763+0000 7f8003a90740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e532f6de50 0x55e532f725c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:03.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.763+0000 7f8003a90740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7ffc1b2510 con 0x55e532f63590 2024-09-10T10:01:03.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.764+0000 7f8003a90740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7ffc1b1e40 con 0x55e532f6de50 2024-09-10T10:01:03.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.764+0000 7f8003a90740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7ffc1af080 con 0x55e532f62c70 2024-09-10T10:01:03.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.764+0000 7f8003a90740 10 client.0 _ll_register_callbacks cb 0x7f7fec009ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:01:03.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.764+0000 7f7ff9e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e532f62c70 0x55e532f71b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:03.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.764+0000 7f7ff9e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e532f62c70 0x55e532f71b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:53076/0 (socket says 172.21.15.77:53076) 2024-09-10T10:01:03.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.764+0000 7f7ff9e00640 1 -- 172.21.15.77:0/2009175541 learned_addr learned my addr 172.21.15.77:0/2009175541 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:01:03.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.764+0000 7f7ffa800640 1 --2- 172.21.15.77:0/2009175541 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e532f6de50 0x55e532f725c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:03.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.764+0000 7f7ff9e00640 1 -- 172.21.15.77:0/2009175541 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e532f6de50 msgr2=0x55e532f725c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:03.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.764+0000 7f7ff9e00640 1 --2- 172.21.15.77:0/2009175541 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e532f6de50 0x55e532f725c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:03.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.764+0000 7f7ff9e00640 1 -- 172.21.15.77:0/2009175541 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e532f63590 msgr2=0x55e532f72080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:03.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.764+0000 7f7ff9e00640 1 --2- 172.21.15.77:0/2009175541 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e532f63590 0x55e532f72080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:03.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.764+0000 7f7ff9e00640 1 -- 172.21.15.77:0/2009175541 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e532f5fe90 con 0x55e532f62c70 2024-09-10T10:01:03.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.765+0000 7f7ff9e00640 1 --2- 172.21.15.77:0/2009175541 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e532f62c70 0x55e532f71b40 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f7fec029f30 tx=0x7f7fec01fd80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fbf590497dd52654 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:03.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.765+0000 7f7fdfe00640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-10T10:01:03.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.765+0000 7f7fdfe00640 1 -- 172.21.15.77:0/2009175541 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fec0043d0 con 0x55e532f62c70 2024-09-10T10:01:03.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.765+0000 7f7fdfe00640 1 -- 172.21.15.77:0/2009175541 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f7fec029650 con 0x55e532f62c70 2024-09-10T10:01:03.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.765+0000 7f8003a90740 1 -- 172.21.15.77:0/2009175541 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7fe40c0020 con 0x55e532f62c70 2024-09-10T10:01:03.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.765+0000 7f7fdfe00640 1 -- 172.21.15.77:0/2009175541 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f7fec043050 con 0x55e532f62c70 2024-09-10T10:01:03.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.765+0000 7f8003a90740 10 client.26647 fetch_fsmap learned FSMap version 34 2024-09-10T10:01:03.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.765+0000 7f8003a90740 1 -- 172.21.15.77:0/2009175541 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55e532f602b0 con 0x55e532f62c70 2024-09-10T10:01:03.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.765+0000 7f7fdfe00640 1 -- 172.21.15.77:0/2009175541 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fe40c0020 con 0x55e532f62c70 2024-09-10T10:01:03.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.766+0000 7f7fdfe00640 1 -- 172.21.15.77:0/2009175541 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 34) ==== 8374+0+0 (secure 0 0 0) 0x7f7fec03e030 con 0x55e532f62c70 2024-09-10T10:01:03.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.766+0000 7f8003a90740 10 client.26647 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T10:01:03.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.766+0000 7f8003a90740 10 client.26647 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-10T10:01:03.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.766+0000 7f8003a90740 20 client.26647 populate_metadata read hostname 'smithi077' 2024-09-10T10:01:03.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.766+0000 7f8003a90740 1 --2- 172.21.15.77:0/2009175541 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x55e532ff1b30 0x55e532ff3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:03.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.766+0000 7f8003a90740 4 client.26647 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:01:03.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.766+0000 7f8003a90740 1 -- 172.21.15.77:0/2009175541 --> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e532ed0e60 con 0x55e532ff1b30 2024-09-10T10:01:03.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.766+0000 7f7ff3e00640 1 --2- 172.21.15.77:0/2009175541 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x55e532ff1b30 0x55e532ff3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:03.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.767+0000 7f7ff3e00640 1 --2- 172.21.15.77:0/2009175541 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x55e532ff1b30 0x55e532ff3f70 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=47021b819ab0875e server_cookie=340778e1dd054fff in_seq=0 out_seq=0 2024-09-10T10:01:03.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.767+0000 7f7fdfe00640 10 client.26647 ms_handle_connect on v2:172.21.15.120:6836/3033280709 2024-09-10T10:01:03.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.767+0000 7f7fdfe00640 1 -- 172.21.15.77:0/2009175541 <== mds.1 v2:172.21.15.120:6836/3033280709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e532ed0e60 con 0x55e532ff1b30 2024-09-10T10:01:03.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.767+0000 7f7fdfe00640 10 client.26647 handle_command_reply: tid=0 2024-09-10T10:01:03.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.767+0000 7f8003a90740 1 client.26647 shutdown 2024-09-10T10:01:03.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.767+0000 7f8003a90740 1 -- 172.21.15.77:0/2009175541 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e532f62c70 msgr2=0x55e532f71b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:03.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.767+0000 7f8003a90740 1 --2- 172.21.15.77:0/2009175541 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e532f62c70 0x55e532f71b40 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f7fec029f30 tx=0x7f7fec01fd80 comp rx=0 tx=0).stop 2024-09-10T10:01:03.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.768+0000 7f8003a90740 1 -- 172.21.15.77:0/2009175541 shutdown_connections 2024-09-10T10:01:03.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.768+0000 7f8003a90740 1 --2- 172.21.15.77:0/2009175541 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x55e532ff1b30 0x55e532ff3f70 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:03.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.768+0000 7f8003a90740 1 --2- 172.21.15.77:0/2009175541 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e532f6de50 0x55e532f725c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:03.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.768+0000 7f8003a90740 1 --2- 172.21.15.77:0/2009175541 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e532f63590 0x55e532f72080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:03.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.768+0000 7f8003a90740 1 --2- 172.21.15.77:0/2009175541 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e532f62c70 0x55e532f71b40 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:03.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.768+0000 7f8003a90740 1 -- 172.21.15.77:0/2009175541 >> 172.21.15.77:0/2009175541 conn(0x55e532f5f200 msgr2=0x55e532f5db60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:03.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.768+0000 7f7fdfe00640 0 client.26647 ms_handle_reset on v2:172.21.15.120:6836/3033280709 2024-09-10T10:01:03.772 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-10T10:01:03.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.768+0000 7f8003a90740 1 -- 172.21.15.77:0/2009175541 shutdown_connections 2024-09-10T10:01:03.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.768+0000 7f8003a90740 1 -- 172.21.15.77:0/2009175541 wait complete. 2024-09-10T10:01:03.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.768+0000 7f8003a90740 20 client.26647 trim_cache size 0 max 16384 2024-09-10T10:01:03.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.769+0000 7f8001000640 1 -- 172.21.15.77:0/1443311759 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f7fe00a2b80 msgr2=0x7f7fe00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:03.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.769+0000 7f8001000640 1 --2- 172.21.15.77:0/1443311759 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f7fe00a2b80 0x7f7fe00a5040 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x55e532f60060 tx=0x7f7fec005f90 comp rx=0 tx=0).stop 2024-09-10T10:01:03.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.769+0000 7f8001000640 1 -- 172.21.15.77:0/1443311759 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ffc102790 msgr2=0x7f7ffc19ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:03.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.769+0000 7f8001000640 1 --2- 172.21.15.77:0/1443311759 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ffc102790 0x7f7ffc19ab80 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7fe4004770 tx=0x7f7fe4003c10 comp rx=0 tx=0).stop 2024-09-10T10:01:03.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.769+0000 7f8001000640 1 -- 172.21.15.77:0/1443311759 shutdown_connections 2024-09-10T10:01:03.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.769+0000 7f8001000640 1 --2- 172.21.15.77:0/1443311759 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f7fe00a2b80 0x7f7fe00a5040 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:03.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.769+0000 7f8001000640 1 --2- 172.21.15.77:0/1443311759 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ffc106f10 0x7f7ffc19ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:03.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.769+0000 7f8001000640 1 --2- 172.21.15.77:0/1443311759 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ffc102790 0x7f7ffc19ab80 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:03.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.769+0000 7f8001000640 1 --2- 172.21.15.77:0/1443311759 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7ffc101de0 0x7f7ffc19a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:03.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.769+0000 7f8001000640 1 -- 172.21.15.77:0/1443311759 >> 172.21.15.77:0/1443311759 conn(0x7f7ffc0fd8c0 msgr2=0x7f7ffc0febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:03.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.769+0000 7f8001000640 1 -- 172.21.15.77:0/1443311759 shutdown_connections 2024-09-10T10:01:03.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:03.769+0000 7f8001000640 1 -- 172.21.15.77:0/1443311759 wait complete. 2024-09-10T10:01:04.579 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T10:01:05.051 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:01:05.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:01:04 smithi120 ceph-mon[34770]: pgmap v568: 129 pgs: 129 active+clean; 118 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 2.1 MiB/s wr, 407 op/s 2024-09-10T10:01:05.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:01:04 smithi077 ceph-mon[30319]: pgmap v568: 129 pgs: 129 active+clean; 118 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 2.1 MiB/s wr, 407 op/s 2024-09-10T10:01:05.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:01:04 smithi134 ceph-mon[35674]: pgmap v568: 129 pgs: 129 active+clean; 118 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 2.1 MiB/s wr, 407 op/s 2024-09-10T10:01:05.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.634+0000 7fc9a5200640 1 -- 172.21.15.77:0/309171534 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc9a0077040 msgr2=0x7fc9a01124f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:05.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.634+0000 7fc9a5200640 1 --2- 172.21.15.77:0/309171534 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc9a0077040 0x7fc9a01124f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc990009900 tx=0x7fc99002a410 comp rx=0 tx=0).stop 2024-09-10T10:01:05.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.635+0000 7fc9a5200640 1 -- 172.21.15.77:0/309171534 shutdown_connections 2024-09-10T10:01:05.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.635+0000 7fc9a5200640 1 --2- 172.21.15.77:0/309171534 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9a0112a30 0x7fc9a0114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:05.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.635+0000 7fc9a5200640 1 --2- 172.21.15.77:0/309171534 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc9a0077040 0x7fc9a01124f0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:05.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.635+0000 7fc9a5200640 1 --2- 172.21.15.77:0/309171534 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc9a0076720 0x7fc9a0076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:05.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.635+0000 7fc9a5200640 1 -- 172.21.15.77:0/309171534 >> 172.21.15.77:0/309171534 conn(0x7fc9a00ffa90 msgr2=0x7fc9a0101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:05.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.635+0000 7fc9a5200640 1 -- 172.21.15.77:0/309171534 shutdown_connections 2024-09-10T10:01:05.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.635+0000 7fc9a5200640 1 -- 172.21.15.77:0/309171534 wait complete. 2024-09-10T10:01:05.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.636+0000 7fc9a5200640 1 Processor -- start 2024-09-10T10:01:05.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.636+0000 7fc9a5200640 1 -- start start 2024-09-10T10:01:05.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.636+0000 7fc9a5200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc9a0076720 0x7fc9a01a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:05.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.636+0000 7fc9a5200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9a0077040 0x7fc9a01a4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:05.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.636+0000 7fc9a5200640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc9a0112a30 0x7fc9a019e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:05.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.637+0000 7fc9a5200640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a0118610 con 0x7fc9a0077040 2024-09-10T10:01:05.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.637+0000 7fc9a5200640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a0118490 con 0x7fc9a0112a30 2024-09-10T10:01:05.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.637+0000 7fc9a5200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a0118790 con 0x7fc9a0076720 2024-09-10T10:01:05.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.637+0000 7fc99de00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc9a0076720 0x7fc9a01a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:05.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.637+0000 7fc99de00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc9a0076720 0x7fc9a01a46e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:53096/0 (socket says 172.21.15.77:53096) 2024-09-10T10:01:05.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.637+0000 7fc99e800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc9a0112a30 0x7fc9a019e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:05.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.637+0000 7fc99de00640 1 -- 172.21.15.77:0/2207550361 learned_addr learned my addr 172.21.15.77:0/2207550361 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:01:05.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.637+0000 7fc99de00640 1 -- 172.21.15.77:0/2207550361 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc9a0112a30 msgr2=0x7fc9a019e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:05.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.637+0000 7fc99de00640 1 --2- 172.21.15.77:0/2207550361 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc9a0112a30 0x7fc9a019e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:05.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.637+0000 7fc99de00640 1 -- 172.21.15.77:0/2207550361 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9a0077040 msgr2=0x7fc9a01a4c20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T10:01:05.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.638+0000 7fc99d400640 1 --2- 172.21.15.77:0/2207550361 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9a0077040 0x7fc9a01a4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:05.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.638+0000 7fc99de00640 1 --2- 172.21.15.77:0/2207550361 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9a0077040 0x7fc9a01a4c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:05.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.638+0000 7fc99de00640 1 -- 172.21.15.77:0/2207550361 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9a019efd0 con 0x7fc9a0076720 2024-09-10T10:01:05.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.638+0000 7fc99e800640 1 --2- 172.21.15.77:0/2207550361 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc9a0112a30 0x7fc9a019e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:01:05.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.638+0000 7fc99d400640 1 --2- 172.21.15.77:0/2207550361 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9a0077040 0x7fc9a01a4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:01:05.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.638+0000 7fc99de00640 1 --2- 172.21.15.77:0/2207550361 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc9a0076720 0x7fc9a01a46e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc98800e9f0 tx=0x7fc98800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:05.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.638+0000 7fc996a00640 1 -- 172.21.15.77:0/2207550361 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98800cd80 con 0x7fc9a0076720 2024-09-10T10:01:05.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.638+0000 7fc996a00640 1 -- 172.21.15.77:0/2207550361 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc988005500 con 0x7fc9a0076720 2024-09-10T10:01:05.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.638+0000 7fc996a00640 1 -- 172.21.15.77:0/2207550361 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc988010690 con 0x7fc9a0076720 2024-09-10T10:01:05.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.638+0000 7fc9a5200640 1 -- 172.21.15.77:0/2207550361 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9a019f2c0 con 0x7fc9a0076720 2024-09-10T10:01:05.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.639+0000 7fc9a5200640 1 -- 172.21.15.77:0/2207550361 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9a01ab5b0 con 0x7fc9a0076720 2024-09-10T10:01:05.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.642+0000 7fc9a7c71740 1 Processor -- start 2024-09-10T10:01:05.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.642+0000 7fc9a7c71740 1 -- start start 2024-09-10T10:01:05.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.642+0000 7fc996a00640 1 -- 172.21.15.77:0/2207550361 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc9880046c0 con 0x7fc9a0076720 2024-09-10T10:01:05.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.642+0000 7fc9a7c71740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d2c6d7c70 0x558d2c6d8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:05.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.642+0000 7fc9a7c71740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d2c6d8590 0x558d2c6e2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:05.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.642+0000 7fc9a7c71740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d2c6e2e50 0x558d2c6e5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:05.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.642+0000 7fc9a7c71740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558d2c63e3e0 con 0x558d2c6d7c70 2024-09-10T10:01:05.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.642+0000 7fc9a7c71740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558d2c5e5d80 con 0x558d2c6e2e50 2024-09-10T10:01:05.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.642+0000 7fc9a7c71740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558d2c63a3a0 con 0x558d2c6d8590 2024-09-10T10:01:05.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.643+0000 7fc99d400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d2c6e2e50 0x558d2c6e5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:05.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.643+0000 7fc99d400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d2c6e2e50 0x558d2c6e5240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:56370/0 (socket says 172.21.15.77:56370) 2024-09-10T10:01:05.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.643+0000 7fc99d400640 1 -- 172.21.15.77:0/784303234 learned_addr learned my addr 172.21.15.77:0/784303234 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:01:05.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.643+0000 7fc99d400640 1 -- 172.21.15.77:0/784303234 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d2c6d8590 msgr2=0x558d2c6e2910 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T10:01:05.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.643+0000 7fc99d400640 1 --2- 172.21.15.77:0/784303234 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d2c6d8590 0x558d2c6e2910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:05.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.643+0000 7fc99d400640 1 -- 172.21.15.77:0/784303234 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d2c6d7c70 msgr2=0x558d2c6d8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:05.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.643+0000 7fc99d400640 1 --2- 172.21.15.77:0/784303234 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d2c6d7c70 0x558d2c6d8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:05.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.644+0000 7fc99d400640 1 -- 172.21.15.77:0/784303234 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d2c604ae0 con 0x558d2c6e2e50 2024-09-10T10:01:05.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.645+0000 7fc99d400640 1 --2- 172.21.15.77:0/784303234 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d2c6e2e50 0x558d2c6e5240 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc99002a930 tx=0x7fc9900040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dca5d4f7a3edea82 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:05.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.645+0000 7fc996a00640 1 --2- 172.21.15.77:0/2207550361 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fc9840a2c00 0x7fc9840a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:05.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.645+0000 7fc98f400640 1 -- 172.21.15.77:0/784303234 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc99003f070 con 0x558d2c6e2e50 2024-09-10T10:01:05.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.645+0000 7fc98f400640 1 -- 172.21.15.77:0/784303234 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc990005520 con 0x558d2c6e2e50 2024-09-10T10:01:05.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.645+0000 7fc98f400640 1 -- 172.21.15.77:0/784303234 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc990035030 con 0x558d2c6e2e50 2024-09-10T10:01:05.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.646+0000 7fc996a00640 1 -- 172.21.15.77:0/2207550361 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fc9880cf8c0 con 0x7fc9a0076720 2024-09-10T10:01:05.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.646+0000 7fc9a7c71740 1 -- 172.21.15.77:0/784303234 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d2c6e2e50 msgr2=0x558d2c6e5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:05.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.646+0000 7fc9a7c71740 1 --2- 172.21.15.77:0/784303234 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d2c6e2e50 0x558d2c6e5240 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc99002a930 tx=0x7fc9900040c0 comp rx=0 tx=0).stop 2024-09-10T10:01:05.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.646+0000 7fc99e800640 1 --2- 172.21.15.77:0/2207550361 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fc9840a2c00 0x7fc9840a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:05.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.647+0000 7fc9a7c71740 1 -- 172.21.15.77:0/784303234 shutdown_connections 2024-09-10T10:01:05.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.647+0000 7fc9a7c71740 1 --2- 172.21.15.77:0/784303234 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d2c6e2e50 0x558d2c6e5240 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:05.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.647+0000 7fc9a7c71740 1 --2- 172.21.15.77:0/784303234 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d2c6d8590 0x558d2c6e2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:05.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.647+0000 7fc9a7c71740 1 --2- 172.21.15.77:0/784303234 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d2c6d7c70 0x558d2c6d8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:05.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.647+0000 7fc9a7c71740 1 -- 172.21.15.77:0/784303234 >> 172.21.15.77:0/784303234 conn(0x558d2c6d4200 msgr2=0x558d2c6d45e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:05.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.647+0000 7fc9a7c71740 1 -- 172.21.15.77:0/784303234 shutdown_connections 2024-09-10T10:01:05.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.647+0000 7fc9a7c71740 1 -- 172.21.15.77:0/784303234 wait complete. 2024-09-10T10:01:05.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.648+0000 7fc9a7c71740 1 -- start start 2024-09-10T10:01:05.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.648+0000 7fc99e800640 1 --2- 172.21.15.77:0/2207550361 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fc9840a2c00 0x7fc9840a50c0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fc998009fd0 tx=0x7fc998009e80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:05.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.649+0000 7fc9a7c71740 1 Processor -- start 2024-09-10T10:01:05.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.649+0000 7fc9a7c71740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d2c6d7c70 0x558d2c6dfd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:05.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.649+0000 7fc9a7c71740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d2c6d8590 0x558d2c6e0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:05.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.649+0000 7fc9a7c71740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d2c6e2e50 0x558d2c6e07d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:05.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.649+0000 7fc9a7c71740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558d2c6e5a20 con 0x558d2c6e2e50 2024-09-10T10:01:05.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.649+0000 7fc9a7c71740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558d2c5f90f0 con 0x558d2c6d7c70 2024-09-10T10:01:05.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.649+0000 7fc9a7c71740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558d2c6e5ba0 con 0x558d2c6d8590 2024-09-10T10:01:05.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.650+0000 7fc9a7c71740 10 client.0 _ll_register_callbacks cb 0x558d2c5e0bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:01:05.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.650+0000 7fc99de00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d2c6d7c70 0x558d2c6dfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:05.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.650+0000 7fc99de00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d2c6d7c70 0x558d2c6dfd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:56382/0 (socket says 172.21.15.77:56382) 2024-09-10T10:01:05.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.650+0000 7fc99de00640 1 -- 172.21.15.77:0/2725421576 learned_addr learned my addr 172.21.15.77:0/2725421576 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:01:05.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.650+0000 7fc99d400640 1 --2- 172.21.15.77:0/2725421576 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d2c6d8590 0x558d2c6e0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:05.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.650+0000 7fc99de00640 1 -- 172.21.15.77:0/2725421576 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d2c6d8590 msgr2=0x558d2c6e0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:05.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.650+0000 7fc99de00640 1 --2- 172.21.15.77:0/2725421576 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d2c6d8590 0x558d2c6e0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:05.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.650+0000 7fc99de00640 1 -- 172.21.15.77:0/2725421576 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d2c6e2e50 msgr2=0x558d2c6e07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:05.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.650+0000 7fc99de00640 1 --2- 172.21.15.77:0/2725421576 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d2c6e2e50 0x558d2c6e07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:05.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.650+0000 7fc99de00640 1 -- 172.21.15.77:0/2725421576 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d2c3af3c0 con 0x558d2c6d7c70 2024-09-10T10:01:05.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.651+0000 7fc99de00640 1 --2- 172.21.15.77:0/2725421576 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d2c6d7c70 0x558d2c6dfd50 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc98800c6a0 tx=0x7fc98809b6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f3ae50c4a3d5574 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:05.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.651+0000 7fc983e00640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T10:01:05.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.651+0000 7fc983e00640 1 -- 172.21.15.77:0/2725421576 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9880a46b0 con 0x558d2c6d7c70 2024-09-10T10:01:05.656 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.651+0000 7fc9a7c71740 1 -- 172.21.15.77:0/2725421576 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc990005520 con 0x558d2c6d7c70 2024-09-10T10:01:05.656 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.651+0000 7fc983e00640 1 -- 172.21.15.77:0/2725421576 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fc9880a44b0 con 0x558d2c6d7c70 2024-09-10T10:01:05.656 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.651+0000 7fc983e00640 1 -- 172.21.15.77:0/2725421576 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fc9880a46b0 con 0x558d2c6d7c70 2024-09-10T10:01:05.656 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.651+0000 7fc9a7c71740 10 client.26702 fetch_fsmap learned FSMap version 34 2024-09-10T10:01:05.656 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.651+0000 7fc9a7c71740 1 -- 172.21.15.77:0/2725421576 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x558d2c702b30 con 0x558d2c6d7c70 2024-09-10T10:01:05.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.652+0000 7fc983e00640 1 -- 172.21.15.77:0/2725421576 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc990005520 con 0x558d2c6d7c70 2024-09-10T10:01:05.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.652+0000 7fc983e00640 1 -- 172.21.15.77:0/2725421576 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 34) ==== 8374+0+0 (secure 0 0 0) 0x7fc9880b3020 con 0x558d2c6d7c70 2024-09-10T10:01:05.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.653+0000 7fc9a7c71740 10 client.26702 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T10:01:05.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.653+0000 7fc9a7c71740 10 client.26702 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T10:01:05.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.653+0000 7fc9a7c71740 20 client.26702 populate_metadata read hostname 'smithi077' 2024-09-10T10:01:05.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.653+0000 7fc9a7c71740 1 --2- 172.21.15.77:0/2725421576 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x558d2c703f40 0x558d2c706330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:05.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.653+0000 7fc9a7c71740 4 client.26702 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:01:05.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.653+0000 7fc9a7c71740 1 -- 172.21.15.77:0/2725421576 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558d2c64b950 con 0x558d2c703f40 2024-09-10T10:01:05.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.653+0000 7fc99d400640 1 --2- 172.21.15.77:0/2725421576 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x558d2c703f40 0x558d2c706330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:05.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.654+0000 7fc99d400640 1 --2- 172.21.15.77:0/2725421576 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x558d2c703f40 0x558d2c706330 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b3f853da18bfdff server_cookie=703848eabd96cea3 in_seq=0 out_seq=0 2024-09-10T10:01:05.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.654+0000 7fc983e00640 10 client.26702 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T10:01:05.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.655+0000 7fc983e00640 1 -- 172.21.15.77:0/2725421576 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558d2c64b950 con 0x558d2c703f40 2024-09-10T10:01:05.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.655+0000 7fc983e00640 10 client.26702 handle_command_reply: tid=0 2024-09-10T10:01:05.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.655+0000 7fc9a7c71740 1 client.26702 shutdown 2024-09-10T10:01:05.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.656+0000 7fc9a7c71740 1 -- 172.21.15.77:0/2725421576 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d2c6d7c70 msgr2=0x558d2c6dfd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:05.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.656+0000 7fc9a7c71740 1 --2- 172.21.15.77:0/2725421576 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d2c6d7c70 0x558d2c6dfd50 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc98800c6a0 tx=0x7fc98809b6f0 comp rx=0 tx=0).stop 2024-09-10T10:01:05.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.656+0000 7fc9a7c71740 1 -- 172.21.15.77:0/2725421576 shutdown_connections 2024-09-10T10:01:05.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.656+0000 7fc9a7c71740 1 --2- 172.21.15.77:0/2725421576 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x558d2c703f40 0x558d2c706330 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:05.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.656+0000 7fc9a7c71740 1 --2- 172.21.15.77:0/2725421576 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d2c6e2e50 0x558d2c6e07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:05.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.656+0000 7fc9a7c71740 1 --2- 172.21.15.77:0/2725421576 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d2c6d8590 0x558d2c6e0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:05.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.656+0000 7fc9a7c71740 1 --2- 172.21.15.77:0/2725421576 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d2c6d7c70 0x558d2c6dfd50 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:05.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.656+0000 7fc983e00640 0 client.26702 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T10:01:05.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.656+0000 7fc9a7c71740 1 -- 172.21.15.77:0/2725421576 >> 172.21.15.77:0/2725421576 conn(0x558d2c6d4200 msgr2=0x558d2c6d9f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:05.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.657+0000 7fc9a7c71740 1 -- 172.21.15.77:0/2725421576 shutdown_connections 2024-09-10T10:01:05.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.657+0000 7fc9a7c71740 1 -- 172.21.15.77:0/2725421576 wait complete. 2024-09-10T10:01:05.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.657+0000 7fc9a7c71740 20 client.26702 trim_cache size 0 max 16384 2024-09-10T10:01:05.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.672+0000 7fc9a7c71740 1 Processor -- start 2024-09-10T10:01:05.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.672+0000 7fc9a7c71740 1 -- start start 2024-09-10T10:01:05.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.672+0000 7fc9a7c71740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d2c6d7c70 0x558d2c6da6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:05.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.673+0000 7fc9a7c71740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d2c6d8590 0x558d2c70bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:05.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.673+0000 7fc9a7c71740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d2c6e2e50 0x558d2c70c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:05.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.673+0000 7fc9a7c71740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a01acd40 con 0x558d2c6e2e50 2024-09-10T10:01:05.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.673+0000 7fc9a7c71740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a01ae8b0 con 0x558d2c6d7c70 2024-09-10T10:01:05.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.673+0000 7fc9a7c71740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a01ad410 con 0x558d2c6d8590 2024-09-10T10:01:05.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.673+0000 7fc99e800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d2c6e2e50 0x558d2c70c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:05.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.673+0000 7fc99de00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d2c6d7c70 0x558d2c6da6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:05.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.673+0000 7fc99de00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d2c6d7c70 0x558d2c6da6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:56388/0 (socket says 172.21.15.77:56388) 2024-09-10T10:01:05.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.673+0000 7fc99e800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d2c6e2e50 0x558d2c70c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:55476/0 (socket says 172.21.15.77:55476) 2024-09-10T10:01:05.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.673+0000 7fc99de00640 1 -- 172.21.15.77:0/2361522779 learned_addr learned my addr 172.21.15.77:0/2361522779 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:01:05.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.673+0000 7fc99d400640 1 --2- 172.21.15.77:0/2361522779 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d2c6d8590 0x558d2c70bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:05.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.673+0000 7fc99de00640 1 -- 172.21.15.77:0/2361522779 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d2c6d8590 msgr2=0x558d2c70bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:05.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.673+0000 7fc99de00640 1 --2- 172.21.15.77:0/2361522779 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d2c6d8590 0x558d2c70bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:05.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.673+0000 7fc99de00640 1 -- 172.21.15.77:0/2361522779 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d2c6e2e50 msgr2=0x558d2c70c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:05.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.673+0000 7fc99de00640 1 --2- 172.21.15.77:0/2361522779 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d2c6e2e50 0x558d2c70c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:05.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.673+0000 7fc99de00640 1 -- 172.21.15.77:0/2361522779 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d2c6cf770 con 0x558d2c6d7c70 2024-09-10T10:01:05.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.673+0000 7fc99e800640 1 --2- 172.21.15.77:0/2361522779 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d2c6e2e50 0x558d2c70c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:01:05.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.674+0000 7fc99de00640 1 --2- 172.21.15.77:0/2361522779 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d2c6d7c70 0x558d2c6da6d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fc98809bc60 tx=0x7fc98809a1e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7c690af1b7218df server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:05.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.674+0000 7fc98ea00640 1 -- 172.21.15.77:0/2361522779 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98809baa0 con 0x558d2c6d7c70 2024-09-10T10:01:05.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.674+0000 7fc98ea00640 1 -- 172.21.15.77:0/2361522779 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9880a4940 con 0x558d2c6d7c70 2024-09-10T10:01:05.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.674+0000 7fc98ea00640 1 -- 172.21.15.77:0/2361522779 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9880a4df0 con 0x558d2c6d7c70 2024-09-10T10:01:05.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.675+0000 7fc9a7c71740 1 -- 172.21.15.77:0/2361522779 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d2c6d7c70 msgr2=0x558d2c6da6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:05.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.675+0000 7fc9a7c71740 1 --2- 172.21.15.77:0/2361522779 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d2c6d7c70 0x558d2c6da6d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fc98809bc60 tx=0x7fc98809a1e0 comp rx=0 tx=0).stop 2024-09-10T10:01:05.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.675+0000 7fc9a7c71740 1 -- 172.21.15.77:0/2361522779 shutdown_connections 2024-09-10T10:01:05.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.675+0000 7fc9a7c71740 1 --2- 172.21.15.77:0/2361522779 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d2c6e2e50 0x558d2c70c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:05.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.675+0000 7fc9a7c71740 1 --2- 172.21.15.77:0/2361522779 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d2c6d8590 0x558d2c70bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:05.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.675+0000 7fc9a7c71740 1 --2- 172.21.15.77:0/2361522779 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d2c6d7c70 0x558d2c6da6d0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:05.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.675+0000 7fc9a7c71740 1 -- 172.21.15.77:0/2361522779 >> 172.21.15.77:0/2361522779 conn(0x558d2c6d4200 msgr2=0x558d2c6e1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:05.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.675+0000 7fc9a7c71740 1 -- 172.21.15.77:0/2361522779 shutdown_connections 2024-09-10T10:01:05.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.675+0000 7fc9a7c71740 1 -- 172.21.15.77:0/2361522779 wait complete. 2024-09-10T10:01:05.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.675+0000 7fc9a7c71740 1 -- start start 2024-09-10T10:01:05.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.675+0000 7fc9a7c71740 1 Processor -- start 2024-09-10T10:01:05.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.676+0000 7fc9a7c71740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d2c6d7c70 0x558d2c6e6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:05.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.676+0000 7fc9a7c71740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d2c6d8590 0x558d2c6e74f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:05.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.676+0000 7fc9a7c71740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d2c6e2e50 0x558d2c76dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:05.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.676+0000 7fc9a7c71740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a01ae1b0 con 0x558d2c6e2e50 2024-09-10T10:01:05.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.676+0000 7fc9a7c71740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a01adae0 con 0x558d2c6d8590 2024-09-10T10:01:05.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.676+0000 7fc9a7c71740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a019fdd0 con 0x558d2c6d7c70 2024-09-10T10:01:05.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.676+0000 7fc9a7c71740 10 client.0 _ll_register_callbacks cb 0x7fc98800cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:01:05.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.677+0000 7fc99d400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d2c6d8590 0x558d2c6e74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:05.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.677+0000 7fc99d400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d2c6d8590 0x558d2c6e74f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:56392/0 (socket says 172.21.15.77:56392) 2024-09-10T10:01:05.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.677+0000 7fc99d400640 1 -- 172.21.15.77:0/1687910565 learned_addr learned my addr 172.21.15.77:0/1687910565 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:01:05.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.677+0000 7fc99de00640 1 --2- 172.21.15.77:0/1687910565 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d2c6d7c70 0x558d2c6e6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:05.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.677+0000 7fc99de00640 1 -- 172.21.15.77:0/1687910565 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d2c6d8590 msgr2=0x558d2c6e74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:05.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.677+0000 7fc99de00640 1 --2- 172.21.15.77:0/1687910565 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d2c6d8590 0x558d2c6e74f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:05.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.677+0000 7fc99de00640 1 -- 172.21.15.77:0/1687910565 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d2c6e2e50 msgr2=0x558d2c76dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:05.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.677+0000 7fc99de00640 1 --2- 172.21.15.77:0/1687910565 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d2c6e2e50 0x558d2c76dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:05.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.677+0000 7fc99de00640 1 -- 172.21.15.77:0/1687910565 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d2c6e7dd0 con 0x558d2c6d7c70 2024-09-10T10:01:05.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.678+0000 7fc99de00640 1 --2- 172.21.15.77:0/1687910565 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d2c6d7c70 0x558d2c6e6fb0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc988098650 tx=0x7fc9880a4850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=361e7c293e59db4f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:05.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.678+0000 7fc983e00640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-10T10:01:05.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.679+0000 7fc983e00640 1 -- 172.21.15.77:0/1687910565 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98809baa0 con 0x558d2c6d7c70 2024-09-10T10:01:05.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.679+0000 7fc983e00640 1 -- 172.21.15.77:0/1687910565 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fc9880a4da0 con 0x558d2c6d7c70 2024-09-10T10:01:05.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.679+0000 7fc9a7c71740 1 -- 172.21.15.77:0/1687910565 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9880a4940 con 0x558d2c6d7c70 2024-09-10T10:01:05.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.679+0000 7fc983e00640 1 -- 172.21.15.77:0/1687910565 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fc98809b120 con 0x558d2c6d7c70 2024-09-10T10:01:05.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.679+0000 7fc9a7c71740 10 client.26662 fetch_fsmap learned FSMap version 34 2024-09-10T10:01:05.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.679+0000 7fc9a7c71740 1 -- 172.21.15.77:0/1687910565 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x558d2c6e7dd0 con 0x558d2c6d7c70 2024-09-10T10:01:05.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.680+0000 7fc983e00640 1 -- 172.21.15.77:0/1687910565 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9880a4940 con 0x558d2c6d7c70 2024-09-10T10:01:05.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.680+0000 7fc983e00640 1 -- 172.21.15.77:0/1687910565 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 34) ==== 8374+0+0 (secure 0 0 0) 0x7fc9880b3070 con 0x558d2c6d7c70 2024-09-10T10:01:05.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.680+0000 7fc9a7c71740 10 client.26662 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T10:01:05.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.680+0000 7fc9a7c71740 10 client.26662 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T10:01:05.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.680+0000 7fc9a7c71740 20 client.26662 populate_metadata read hostname 'smithi077' 2024-09-10T10:01:05.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.680+0000 7fc9a7c71740 1 --2- 172.21.15.77:0/1687910565 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x558d2c76f540 0x558d2c771930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:05.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.680+0000 7fc9a7c71740 4 client.26662 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:01:05.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.680+0000 7fc9a7c71740 1 -- 172.21.15.77:0/1687910565 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558d2c645e60 con 0x558d2c76f540 2024-09-10T10:01:05.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.681+0000 7fc99d400640 1 --2- 172.21.15.77:0/1687910565 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x558d2c76f540 0x558d2c771930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:05.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.681+0000 7fc99d400640 1 --2- 172.21.15.77:0/1687910565 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x558d2c76f540 0x558d2c771930 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3822ae79794f5880 server_cookie=1ddcf085b0f27905 in_seq=0 out_seq=0 2024-09-10T10:01:05.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.681+0000 7fc983e00640 10 client.26662 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T10:01:05.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.681+0000 7fc983e00640 1 -- 172.21.15.77:0/1687910565 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558d2c645e60 con 0x558d2c76f540 2024-09-10T10:01:05.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.681+0000 7fc983e00640 10 client.26662 handle_command_reply: tid=0 2024-09-10T10:01:05.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.681+0000 7fc9a7c71740 1 client.26662 shutdown 2024-09-10T10:01:05.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.682+0000 7fc9a7c71740 1 -- 172.21.15.77:0/1687910565 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d2c6d7c70 msgr2=0x558d2c6e6fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:05.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.682+0000 7fc9a7c71740 1 --2- 172.21.15.77:0/1687910565 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d2c6d7c70 0x558d2c6e6fb0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc988098650 tx=0x7fc9880a4850 comp rx=0 tx=0).stop 2024-09-10T10:01:05.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.682+0000 7fc9a7c71740 1 -- 172.21.15.77:0/1687910565 shutdown_connections 2024-09-10T10:01:05.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.682+0000 7fc9a7c71740 1 --2- 172.21.15.77:0/1687910565 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x558d2c76f540 0x558d2c771930 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:05.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.682+0000 7fc9a7c71740 1 --2- 172.21.15.77:0/1687910565 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d2c6e2e50 0x558d2c76dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:05.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.682+0000 7fc983e00640 0 client.26662 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T10:01:05.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.682+0000 7fc9a7c71740 1 --2- 172.21.15.77:0/1687910565 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d2c6d8590 0x558d2c6e74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:05.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.682+0000 7fc9a7c71740 1 --2- 172.21.15.77:0/1687910565 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d2c6d7c70 0x558d2c6e6fb0 secure :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc988098650 tx=0x7fc9880a4850 comp rx=0 tx=0).stop 2024-09-10T10:01:05.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.682+0000 7fc9a7c71740 1 -- 172.21.15.77:0/1687910565 >> 172.21.15.77:0/1687910565 conn(0x558d2c6d4200 msgr2=0x558d2c6d2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:05.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.682+0000 7fc9a7c71740 1 -- 172.21.15.77:0/1687910565 shutdown_connections 2024-09-10T10:01:05.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.682+0000 7fc9a7c71740 1 -- 172.21.15.77:0/1687910565 wait complete. 2024-09-10T10:01:05.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.683+0000 7fc9a7c71740 20 client.26662 trim_cache size 0 max 16384 2024-09-10T10:01:05.685 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-10T10:01:05.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.683+0000 7fc9a5200640 1 -- 172.21.15.77:0/2207550361 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fc9840a2c00 msgr2=0x7fc9840a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:05.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.683+0000 7fc9a5200640 1 --2- 172.21.15.77:0/2207550361 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fc9840a2c00 0x7fc9840a50c0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fc998009fd0 tx=0x7fc998009e80 comp rx=0 tx=0).stop 2024-09-10T10:01:05.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.683+0000 7fc9a5200640 1 -- 172.21.15.77:0/2207550361 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc9a0076720 msgr2=0x7fc9a01a46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:05.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.683+0000 7fc9a5200640 1 --2- 172.21.15.77:0/2207550361 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc9a0076720 0x7fc9a01a46e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc98800e9f0 tx=0x7fc98800edf0 comp rx=0 tx=0).stop 2024-09-10T10:01:05.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.684+0000 7fc9a5200640 1 -- 172.21.15.77:0/2207550361 shutdown_connections 2024-09-10T10:01:05.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.684+0000 7fc9a5200640 1 --2- 172.21.15.77:0/2207550361 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fc9840a2c00 0x7fc9840a50c0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:05.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.684+0000 7fc9a5200640 1 --2- 172.21.15.77:0/2207550361 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc9a0112a30 0x7fc9a019e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:05.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.684+0000 7fc9a5200640 1 --2- 172.21.15.77:0/2207550361 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9a0077040 0x7fc9a01a4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:05.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.684+0000 7fc9a5200640 1 --2- 172.21.15.77:0/2207550361 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc9a0076720 0x7fc9a01a46e0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:05.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.684+0000 7fc9a5200640 1 -- 172.21.15.77:0/2207550361 >> 172.21.15.77:0/2207550361 conn(0x7fc9a00ffa90 msgr2=0x7fc9a0101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:05.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.684+0000 7fc9a5200640 1 -- 172.21.15.77:0/2207550361 shutdown_connections 2024-09-10T10:01:05.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:05.684+0000 7fc9a5200640 1 -- 172.21.15.77:0/2207550361 wait complete. 2024-09-10T10:01:07.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:01:06 smithi077 ceph-mon[30319]: pgmap v569: 129 pgs: 129 active+clean; 118 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 2.0 MiB/s wr, 363 op/s 2024-09-10T10:01:07.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:01:06 smithi077 ceph-mon[30319]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T10:01:07.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:01:06 smithi134 ceph-mon[35674]: pgmap v569: 129 pgs: 129 active+clean; 118 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 2.0 MiB/s wr, 363 op/s 2024-09-10T10:01:07.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:01:06 smithi134 ceph-mon[35674]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T10:01:07.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:01:06 smithi120 ceph-mon[34770]: pgmap v569: 129 pgs: 129 active+clean; 118 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 2.0 MiB/s wr, 363 op/s 2024-09-10T10:01:07.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:01:06 smithi120 ceph-mon[34770]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T10:01:07.557 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T10:01:07.557 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T10:01:08.010 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:01:08.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.613+0000 7fe868c00640 1 -- 172.21.15.77:0/2038061087 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe864104010 msgr2=0x7fe8641043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:08.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.613+0000 7fe868c00640 1 --2- 172.21.15.77:0/2038061087 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe864104010 0x7fe8641043f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fe854009900 tx=0x7fe85402a420 comp rx=0 tx=0).stop 2024-09-10T10:01:08.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.613+0000 7fe868c00640 1 -- 172.21.15.77:0/2038061087 shutdown_connections 2024-09-10T10:01:08.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.613+0000 7fe868c00640 1 --2- 172.21.15.77:0/2038061087 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe864109140 0x7fe864110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:08.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.613+0000 7fe868c00640 1 --2- 172.21.15.77:0/2038061087 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8641049c0 0x7fe864108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:08.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.613+0000 7fe868c00640 1 --2- 172.21.15.77:0/2038061087 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe864104010 0x7fe8641043f0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:08.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.613+0000 7fe868c00640 1 -- 172.21.15.77:0/2038061087 >> 172.21.15.77:0/2038061087 conn(0x7fe8640ffa50 msgr2=0x7fe864101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:08.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.613+0000 7fe868c00640 1 -- 172.21.15.77:0/2038061087 shutdown_connections 2024-09-10T10:01:08.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.614+0000 7fe868c00640 1 -- 172.21.15.77:0/2038061087 wait complete. 2024-09-10T10:01:08.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.614+0000 7fe868c00640 1 Processor -- start 2024-09-10T10:01:08.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.615+0000 7fe868c00640 1 -- start start 2024-09-10T10:01:08.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.615+0000 7fe868c00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe864104010 0x7fe86419a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:08.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.615+0000 7fe868c00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe8641049c0 0x7fe86419ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:08.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.615+0000 7fe868c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe864109140 0x7fe86419efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:08.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.615+0000 7fe868c00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe8641141e0 con 0x7fe864104010 2024-09-10T10:01:08.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.615+0000 7fe868c00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe864114060 con 0x7fe8641049c0 2024-09-10T10:01:08.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.615+0000 7fe868c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe864114360 con 0x7fe864109140 2024-09-10T10:01:08.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.615+0000 7fe861400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe8641049c0 0x7fe86419ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:08.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.616+0000 7fe861400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe8641049c0 0x7fe86419ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:56404/0 (socket says 172.21.15.77:56404) 2024-09-10T10:01:08.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.616+0000 7fe861400640 1 -- 172.21.15.77:0/1684881326 learned_addr learned my addr 172.21.15.77:0/1684881326 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:01:08.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.616+0000 7fe861e00640 1 --2- 172.21.15.77:0/1684881326 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe864104010 0x7fe86419a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:08.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.616+0000 7fe861400640 1 -- 172.21.15.77:0/1684881326 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe864109140 msgr2=0x7fe86419efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:08.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.616+0000 7fe861400640 1 --2- 172.21.15.77:0/1684881326 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe864109140 0x7fe86419efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:08.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.616+0000 7fe861400640 1 -- 172.21.15.77:0/1684881326 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe864104010 msgr2=0x7fe86419a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:08.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.616+0000 7fe861400640 1 --2- 172.21.15.77:0/1684881326 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe864104010 0x7fe86419a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:08.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.616+0000 7fe861400640 1 -- 172.21.15.77:0/1684881326 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe86419f690 con 0x7fe8641049c0 2024-09-10T10:01:08.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.616+0000 7fe861400640 1 --2- 172.21.15.77:0/1684881326 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe8641049c0 0x7fe86419ac20 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe85000e9f0 tx=0x7fe85000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:08.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.616+0000 7fe85aa00640 1 -- 172.21.15.77:0/1684881326 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe85000cd80 con 0x7fe8641049c0 2024-09-10T10:01:08.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.617+0000 7fe85aa00640 1 -- 172.21.15.77:0/1684881326 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe850010910 con 0x7fe8641049c0 2024-09-10T10:01:08.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.617+0000 7fe868c00640 1 -- 172.21.15.77:0/1684881326 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe86419f980 con 0x7fe8641049c0 2024-09-10T10:01:08.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.617+0000 7fe85aa00640 1 -- 172.21.15.77:0/1684881326 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe850018980 con 0x7fe8641049c0 2024-09-10T10:01:08.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.617+0000 7fe868c00640 1 -- 172.21.15.77:0/1684881326 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8641a7260 con 0x7fe8641049c0 2024-09-10T10:01:08.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.620+0000 7fe85aa00640 1 -- 172.21.15.77:0/1684881326 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe850010ac0 con 0x7fe8641049c0 2024-09-10T10:01:08.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.621+0000 7fe86b7e1740 1 Processor -- start 2024-09-10T10:01:08.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.621+0000 7fe86b7e1740 1 -- start start 2024-09-10T10:01:08.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.621+0000 7fe85aa00640 1 --2- 172.21.15.77:0/1684881326 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fe8480a2c00 0x7fe8480a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:08.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.621+0000 7fe85aa00640 1 -- 172.21.15.77:0/1684881326 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fe8500cf870 con 0x7fe8641049c0 2024-09-10T10:01:08.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.622+0000 7fe86b7e1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ab63bbeac0 0x55ab63bbeea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:08.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.622+0000 7fe86b7e1740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab63bbf3e0 0x55ab63bc9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:08.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.622+0000 7fe862800640 1 --2- 172.21.15.77:0/1684881326 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fe8480a2c00 0x7fe8480a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:08.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.622+0000 7fe86b7e1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab63bc9ca0 0x55ab63bcc090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:08.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.622+0000 7fe86b7e1740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ab63b25290 con 0x55ab63bbf3e0 2024-09-10T10:01:08.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.622+0000 7fe86b7e1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ab63accb40 con 0x55ab63bc9ca0 2024-09-10T10:01:08.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.622+0000 7fe86b7e1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ab63b21250 con 0x55ab63bbeac0 2024-09-10T10:01:08.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.622+0000 7fe862800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab63bc9ca0 0x55ab63bcc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:08.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.622+0000 7fe862800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab63bc9ca0 0x55ab63bcc090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:56412/0 (socket says 172.21.15.77:56412) 2024-09-10T10:01:08.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.622+0000 7fe862800640 1 -- 172.21.15.77:0/2305928974 learned_addr learned my addr 172.21.15.77:0/2305928974 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:01:08.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.622+0000 7fe861e00640 1 --2- 172.21.15.77:0/2305928974 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ab63bbeac0 0x55ab63bbeea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:08.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.622+0000 7fe861400640 1 --2- 172.21.15.77:0/2305928974 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab63bbf3e0 0x55ab63bc9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:08.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.623+0000 7fe861e00640 1 -- 172.21.15.77:0/2305928974 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab63bc9ca0 msgr2=0x55ab63bcc090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:08.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.623+0000 7fe861e00640 1 --2- 172.21.15.77:0/2305928974 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab63bc9ca0 0x55ab63bcc090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:08.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.623+0000 7fe861e00640 1 -- 172.21.15.77:0/2305928974 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab63bbf3e0 msgr2=0x55ab63bc9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:08.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.623+0000 7fe862800640 1 --2- 172.21.15.77:0/1684881326 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fe8480a2c00 0x7fe8480a50c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fe85c00e690 tx=0x7fe85c00ab60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:08.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.623+0000 7fe861e00640 1 --2- 172.21.15.77:0/2305928974 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab63bbf3e0 0x55ab63bc9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:08.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.623+0000 7fe861e00640 1 -- 172.21.15.77:0/2305928974 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab638966a0 con 0x55ab63bbeac0 2024-09-10T10:01:08.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.623+0000 7fe861400640 1 --2- 172.21.15.77:0/2305928974 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab63bbf3e0 0x55ab63bc9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:01:08.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.624+0000 7fe861e00640 1 --2- 172.21.15.77:0/2305928974 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ab63bbeac0 0x55ab63bbeea0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fe854009230 tx=0x7fe854005de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=618a7c440fa063f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:08.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.625+0000 7fe84f000640 1 -- 172.21.15.77:0/2305928974 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe85403f070 con 0x55ab63bbeac0 2024-09-10T10:01:08.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.625+0000 7fe84f000640 1 -- 172.21.15.77:0/2305928974 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe854004030 con 0x55ab63bbeac0 2024-09-10T10:01:08.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.625+0000 7fe84f000640 1 -- 172.21.15.77:0/2305928974 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe854035070 con 0x55ab63bbeac0 2024-09-10T10:01:08.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.626+0000 7fe86b7e1740 1 -- 172.21.15.77:0/2305928974 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ab63bbeac0 msgr2=0x55ab63bbeea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:08.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.626+0000 7fe86b7e1740 1 --2- 172.21.15.77:0/2305928974 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ab63bbeac0 0x55ab63bbeea0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fe854009230 tx=0x7fe854005de0 comp rx=0 tx=0).stop 2024-09-10T10:01:08.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.627+0000 7fe86b7e1740 1 -- 172.21.15.77:0/2305928974 shutdown_connections 2024-09-10T10:01:08.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.627+0000 7fe86b7e1740 1 --2- 172.21.15.77:0/2305928974 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab63bc9ca0 0x55ab63bcc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:08.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.627+0000 7fe86b7e1740 1 --2- 172.21.15.77:0/2305928974 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab63bbf3e0 0x55ab63bc9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:08.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.627+0000 7fe86b7e1740 1 --2- 172.21.15.77:0/2305928974 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ab63bbeac0 0x55ab63bbeea0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:08.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.627+0000 7fe86b7e1740 1 -- 172.21.15.77:0/2305928974 >> 172.21.15.77:0/2305928974 conn(0x55ab63bbb050 msgr2=0x55ab63bbb430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:08.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.627+0000 7fe86b7e1740 1 -- 172.21.15.77:0/2305928974 shutdown_connections 2024-09-10T10:01:08.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.627+0000 7fe86b7e1740 1 -- 172.21.15.77:0/2305928974 wait complete. 2024-09-10T10:01:08.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.628+0000 7fe86b7e1740 1 -- start start 2024-09-10T10:01:08.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.628+0000 7fe86b7e1740 1 Processor -- start 2024-09-10T10:01:08.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.628+0000 7fe86b7e1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab63bbeac0 0x55ab63bc9640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:08.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.628+0000 7fe86b7e1740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab63bbf3e0 0x55ab63bc68f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:08.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.628+0000 7fe86b7e1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ab63bc9ca0 0x55ab63bc6e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:08.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.628+0000 7fe86b7e1740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ab63bd5120 con 0x55ab63bbf3e0 2024-09-10T10:01:08.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.628+0000 7fe86b7e1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ab63adffa0 con 0x55ab63bbeac0 2024-09-10T10:01:08.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.628+0000 7fe86b7e1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ab63bd52a0 con 0x55ab63bc9ca0 2024-09-10T10:01:08.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.628+0000 7fe86b7e1740 10 client.0 _ll_register_callbacks cb 0x55ab63833ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:01:08.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.629+0000 7fe861e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab63bbeac0 0x55ab63bc9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:08.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.629+0000 7fe861e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab63bbeac0 0x55ab63bc9640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:56426/0 (socket says 172.21.15.77:56426) 2024-09-10T10:01:08.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.629+0000 7fe861e00640 1 -- 172.21.15.77:0/1674302576 learned_addr learned my addr 172.21.15.77:0/1674302576 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:01:08.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.629+0000 7fe862800640 1 --2- 172.21.15.77:0/1674302576 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ab63bc9ca0 0x55ab63bc6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:08.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.629+0000 7fe861400640 1 --2- 172.21.15.77:0/1674302576 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab63bbf3e0 0x55ab63bc68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:08.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.629+0000 7fe861e00640 1 -- 172.21.15.77:0/1674302576 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ab63bc9ca0 msgr2=0x55ab63bc6e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:08.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.629+0000 7fe861e00640 1 --2- 172.21.15.77:0/1674302576 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ab63bc9ca0 0x55ab63bc6e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:08.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.629+0000 7fe861e00640 1 -- 172.21.15.77:0/1674302576 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab63bbf3e0 msgr2=0x55ab63bc68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:08.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.629+0000 7fe861e00640 1 --2- 172.21.15.77:0/1674302576 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab63bbf3e0 0x55ab63bc68f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:08.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.629+0000 7fe861e00640 1 -- 172.21.15.77:0/1674302576 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab63bc76c0 con 0x55ab63bbeac0 2024-09-10T10:01:08.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.629+0000 7fe862800640 1 --2- 172.21.15.77:0/1674302576 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ab63bc9ca0 0x55ab63bc6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:01:08.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.629+0000 7fe861400640 1 --2- 172.21.15.77:0/1674302576 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab63bbf3e0 0x55ab63bc68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:01:08.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.630+0000 7fe861e00640 1 --2- 172.21.15.77:0/1674302576 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab63bbeac0 0x55ab63bc9640 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fe854009900 tx=0x7fe854032f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be6e376f723ce541 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:08.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.630+0000 7fe843400640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T10:01:08.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.630+0000 7fe843400640 1 -- 172.21.15.77:0/1674302576 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe854035070 con 0x55ab63bbeac0 2024-09-10T10:01:08.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.630+0000 7fe843400640 1 -- 172.21.15.77:0/1674302576 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fe8540315b0 con 0x55ab63bbeac0 2024-09-10T10:01:08.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.630+0000 7fe86b7e1740 1 -- 172.21.15.77:0/1674302576 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe854004030 con 0x55ab63bbeac0 2024-09-10T10:01:08.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.630+0000 7fe843400640 1 -- 172.21.15.77:0/1674302576 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fe854055050 con 0x55ab63bbeac0 2024-09-10T10:01:08.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.630+0000 7fe86b7e1740 10 client.26720 fetch_fsmap learned FSMap version 35 2024-09-10T10:01:08.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.630+0000 7fe86b7e1740 1 -- 172.21.15.77:0/1674302576 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55ab63be10a0 con 0x55ab63bbeac0 2024-09-10T10:01:08.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.631+0000 7fe843400640 1 -- 172.21.15.77:0/1674302576 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe854004030 con 0x55ab63bbeac0 2024-09-10T10:01:08.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.631+0000 7fe843400640 1 -- 172.21.15.77:0/1674302576 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7fe8540035f0 con 0x55ab63bbeac0 2024-09-10T10:01:08.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.631+0000 7fe86b7e1740 10 client.26720 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T10:01:08.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.631+0000 7fe86b7e1740 10 client.26720 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T10:01:08.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.631+0000 7fe86b7e1740 20 client.26720 populate_metadata read hostname 'smithi077' 2024-09-10T10:01:08.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.631+0000 7fe86b7e1740 1 --2- 172.21.15.77:0/1674302576 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55ab63be23f0 0x55ab63be47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:08.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.631+0000 7fe86b7e1740 4 client.26720 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:01:08.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.631+0000 7fe86b7e1740 1 -- 172.21.15.77:0/1674302576 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ab63b32800 con 0x55ab63be23f0 2024-09-10T10:01:08.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.631+0000 7fe861400640 1 --2- 172.21.15.77:0/1674302576 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55ab63be23f0 0x55ab63be47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:08.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.632+0000 7fe843400640 10 client.26720 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T10:01:08.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.632+0000 7fe861400640 1 --2- 172.21.15.77:0/1674302576 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55ab63be23f0 0x55ab63be47e0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cdab1a4a01ecdad8 server_cookie=f8984e934c400245 in_seq=0 out_seq=0 2024-09-10T10:01:08.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.632+0000 7fe843400640 1 -- 172.21.15.77:0/1674302576 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ab63b32800 con 0x55ab63be23f0 2024-09-10T10:01:08.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.632+0000 7fe843400640 10 client.26720 handle_command_reply: tid=0 2024-09-10T10:01:08.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.633+0000 7fe86b7e1740 1 client.26720 shutdown 2024-09-10T10:01:08.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.633+0000 7fe86b7e1740 1 -- 172.21.15.77:0/1674302576 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab63bbeac0 msgr2=0x55ab63bc9640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:08.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.633+0000 7fe86b7e1740 1 --2- 172.21.15.77:0/1674302576 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab63bbeac0 0x55ab63bc9640 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fe854009900 tx=0x7fe854032f60 comp rx=0 tx=0).stop 2024-09-10T10:01:08.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.633+0000 7fe86b7e1740 1 -- 172.21.15.77:0/1674302576 shutdown_connections 2024-09-10T10:01:08.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.633+0000 7fe86b7e1740 1 --2- 172.21.15.77:0/1674302576 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55ab63be23f0 0x55ab63be47e0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:08.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.633+0000 7fe86b7e1740 1 --2- 172.21.15.77:0/1674302576 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ab63bc9ca0 0x55ab63bc6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:08.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.633+0000 7fe86b7e1740 1 --2- 172.21.15.77:0/1674302576 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab63bbf3e0 0x55ab63bc68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:08.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.633+0000 7fe86b7e1740 1 --2- 172.21.15.77:0/1674302576 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab63bbeac0 0x55ab63bc9640 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:08.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.633+0000 7fe843400640 0 client.26720 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T10:01:08.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.633+0000 7fe86b7e1740 1 -- 172.21.15.77:0/1674302576 >> 172.21.15.77:0/1674302576 conn(0x55ab63bbb050 msgr2=0x55ab63bba1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:08.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.633+0000 7fe86b7e1740 1 -- 172.21.15.77:0/1674302576 shutdown_connections 2024-09-10T10:01:08.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.633+0000 7fe86b7e1740 1 -- 172.21.15.77:0/1674302576 wait complete. 2024-09-10T10:01:08.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.634+0000 7fe86b7e1740 20 client.26720 trim_cache size 0 max 16384 2024-09-10T10:01:08.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.668+0000 7fe86b7e1740 1 Processor -- start 2024-09-10T10:01:08.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.668+0000 7fe86b7e1740 1 -- start start 2024-09-10T10:01:08.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.668+0000 7fe86b7e1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ab63bbeac0 0x55ab63bc87c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:08.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.668+0000 7fe86b7e1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab63bbf3e0 0x55ab63be7810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:08.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.668+0000 7fe86b7e1740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab63bc9ca0 0x55ab63be7d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:08.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.668+0000 7fe86b7e1740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe8641acd20 con 0x55ab63bc9ca0 2024-09-10T10:01:08.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.669+0000 7fe86b7e1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe8641ae860 con 0x55ab63bbf3e0 2024-09-10T10:01:08.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.669+0000 7fe86b7e1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe8641ad3f0 con 0x55ab63bbeac0 2024-09-10T10:01:08.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.669+0000 7fe861e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ab63bbeac0 0x55ab63bc87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:08.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.669+0000 7fe861400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab63bbf3e0 0x55ab63be7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:08.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.669+0000 7fe861e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ab63bbeac0 0x55ab63bc87c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:53174/0 (socket says 172.21.15.77:53174) 2024-09-10T10:01:08.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.669+0000 7fe861400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab63bbf3e0 0x55ab63be7810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:56436/0 (socket says 172.21.15.77:56436) 2024-09-10T10:01:08.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.669+0000 7fe861e00640 1 -- 172.21.15.77:0/3326145471 learned_addr learned my addr 172.21.15.77:0/3326145471 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:01:08.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.669+0000 7fe861e00640 1 -- 172.21.15.77:0/3326145471 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab63bbf3e0 msgr2=0x55ab63be7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:08.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.669+0000 7fe862800640 1 --2- 172.21.15.77:0/3326145471 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab63bc9ca0 0x55ab63be7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:08.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.669+0000 7fe861e00640 1 --2- 172.21.15.77:0/3326145471 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab63bbf3e0 0x55ab63be7810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:08.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.669+0000 7fe861e00640 1 -- 172.21.15.77:0/3326145471 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab63bc9ca0 msgr2=0x55ab63be7d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:08.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.669+0000 7fe861e00640 1 --2- 172.21.15.77:0/3326145471 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab63bc9ca0 0x55ab63be7d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:08.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.669+0000 7fe861e00640 1 -- 172.21.15.77:0/3326145471 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab63bc7890 con 0x55ab63bbeac0 2024-09-10T10:01:08.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.669+0000 7fe862800640 1 --2- 172.21.15.77:0/3326145471 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab63bc9ca0 0x55ab63be7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:01:08.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.670+0000 7fe861e00640 1 --2- 172.21.15.77:0/3326145471 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ab63bbeac0 0x55ab63bc87c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fe8540039c0 tx=0x7fe854005e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9bafbfddf03a3b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:08.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.670+0000 7fe84e600640 1 -- 172.21.15.77:0/3326145471 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe854035030 con 0x55ab63bbeac0 2024-09-10T10:01:08.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.670+0000 7fe84e600640 1 -- 172.21.15.77:0/3326145471 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe85404c070 con 0x55ab63bbeac0 2024-09-10T10:01:08.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.670+0000 7fe84e600640 1 -- 172.21.15.77:0/3326145471 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe85403f040 con 0x55ab63bbeac0 2024-09-10T10:01:08.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.671+0000 7fe86b7e1740 1 -- 172.21.15.77:0/3326145471 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ab63bbeac0 msgr2=0x55ab63bc87c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:08.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.671+0000 7fe86b7e1740 1 --2- 172.21.15.77:0/3326145471 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ab63bbeac0 0x55ab63bc87c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fe8540039c0 tx=0x7fe854005e60 comp rx=0 tx=0).stop 2024-09-10T10:01:08.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.671+0000 7fe86b7e1740 1 -- 172.21.15.77:0/3326145471 shutdown_connections 2024-09-10T10:01:08.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.671+0000 7fe86b7e1740 1 --2- 172.21.15.77:0/3326145471 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab63bc9ca0 0x55ab63be7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:08.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.671+0000 7fe86b7e1740 1 --2- 172.21.15.77:0/3326145471 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab63bbf3e0 0x55ab63be7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:08.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.671+0000 7fe86b7e1740 1 --2- 172.21.15.77:0/3326145471 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ab63bbeac0 0x55ab63bc87c0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:08.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.671+0000 7fe86b7e1740 1 -- 172.21.15.77:0/3326145471 >> 172.21.15.77:0/3326145471 conn(0x55ab63bbb050 msgr2=0x55ab63bddd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:08.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.671+0000 7fe86b7e1740 1 -- 172.21.15.77:0/3326145471 shutdown_connections 2024-09-10T10:01:08.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.671+0000 7fe86b7e1740 1 -- 172.21.15.77:0/3326145471 wait complete. 2024-09-10T10:01:08.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.671+0000 7fe86b7e1740 1 -- start start 2024-09-10T10:01:08.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.672+0000 7fe86b7e1740 1 Processor -- start 2024-09-10T10:01:08.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.672+0000 7fe86b7e1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ab63bbeac0 0x55ab63c40cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:08.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.672+0000 7fe86b7e1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab63bbf3e0 0x55ab63c41200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:08.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.672+0000 7fe86b7e1740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab63bc9ca0 0x55ab63c4bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:08.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.672+0000 7fe86b7e1740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe8641ae190 con 0x55ab63bc9ca0 2024-09-10T10:01:08.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.672+0000 7fe86b7e1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe8641adac0 con 0x55ab63bbf3e0 2024-09-10T10:01:08.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.672+0000 7fe86b7e1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe8641aad20 con 0x55ab63bbeac0 2024-09-10T10:01:08.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.672+0000 7fe86b7e1740 10 client.0 _ll_register_callbacks cb 0x7fe85c00ad70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:01:08.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.672+0000 7fe861e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ab63bbeac0 0x55ab63c40cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:08.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.672+0000 7fe861400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab63bbf3e0 0x55ab63c41200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:08.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.672+0000 7fe861e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ab63bbeac0 0x55ab63c40cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:53188/0 (socket says 172.21.15.77:53188) 2024-09-10T10:01:08.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.672+0000 7fe861e00640 1 -- 172.21.15.77:0/1511895148 learned_addr learned my addr 172.21.15.77:0/1511895148 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:01:08.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.672+0000 7fe862800640 1 --2- 172.21.15.77:0/1511895148 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab63bc9ca0 0x55ab63c4bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:08.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.673+0000 7fe861e00640 1 -- 172.21.15.77:0/1511895148 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab63bbf3e0 msgr2=0x55ab63c41200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:08.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.673+0000 7fe861e00640 1 --2- 172.21.15.77:0/1511895148 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab63bbf3e0 0x55ab63c41200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:08.677 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.673+0000 7fe861e00640 1 -- 172.21.15.77:0/1511895148 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab63bc9ca0 msgr2=0x55ab63c4bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:08.677 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.673+0000 7fe861e00640 1 --2- 172.21.15.77:0/1511895148 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab63bc9ca0 0x55ab63c4bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:08.677 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.673+0000 7fe861e00640 1 -- 172.21.15.77:0/1511895148 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab63c4c280 con 0x55ab63bbeac0 2024-09-10T10:01:08.677 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.673+0000 7fe861e00640 1 --2- 172.21.15.77:0/1511895148 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ab63bbeac0 0x55ab63c40cc0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe85402a940 tx=0x7fe854005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=388f4bf42d4acf7e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:08.677 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.673+0000 7fe843400640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-10T10:01:08.677 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.673+0000 7fe843400640 1 -- 172.21.15.77:0/1511895148 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe854035070 con 0x55ab63bbeac0 2024-09-10T10:01:08.677 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.673+0000 7fe86b7e1740 1 -- 172.21.15.77:0/1511895148 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe85404c070 con 0x55ab63bbeac0 2024-09-10T10:01:08.677 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.674+0000 7fe843400640 1 -- 172.21.15.77:0/1511895148 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fe8540317b0 con 0x55ab63bbeac0 2024-09-10T10:01:08.677 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.674+0000 7fe843400640 1 -- 172.21.15.77:0/1511895148 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fe85406c050 con 0x55ab63bbeac0 2024-09-10T10:01:08.677 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.674+0000 7fe86b7e1740 10 client.26674 fetch_fsmap learned FSMap version 35 2024-09-10T10:01:08.677 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.674+0000 7fe86b7e1740 1 -- 172.21.15.77:0/1511895148 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55ab63c4d3a0 con 0x55ab63bbeac0 2024-09-10T10:01:08.677 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.674+0000 7fe843400640 1 -- 172.21.15.77:0/1511895148 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe85404c070 con 0x55ab63bbeac0 2024-09-10T10:01:08.677 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.674+0000 7fe843400640 1 -- 172.21.15.77:0/1511895148 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7fe854051820 con 0x55ab63bbeac0 2024-09-10T10:01:08.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.674+0000 7fe86b7e1740 10 client.26674 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T10:01:08.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.674+0000 7fe86b7e1740 10 client.26674 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T10:01:08.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.674+0000 7fe86b7e1740 20 client.26674 populate_metadata read hostname 'smithi077' 2024-09-10T10:01:08.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.674+0000 7fe86b7e1740 1 --2- 172.21.15.77:0/1511895148 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55ab63c4da30 0x55ab63c4fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:08.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.674+0000 7fe86b7e1740 4 client.26674 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T10:01:08.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.674+0000 7fe86b7e1740 1 -- 172.21.15.77:0/1511895148 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ab63b2cd10 con 0x55ab63c4da30 2024-09-10T10:01:08.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.675+0000 7fe861400640 1 --2- 172.21.15.77:0/1511895148 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55ab63c4da30 0x55ab63c4fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:08.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.676+0000 7fe861400640 1 --2- 172.21.15.77:0/1511895148 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55ab63c4da30 0x55ab63c4fe50 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5e0558dbea2dabd server_cookie=d91f6010d13de71e in_seq=0 out_seq=0 2024-09-10T10:01:08.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.676+0000 7fe843400640 10 client.26674 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T10:01:08.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.678+0000 7fe843400640 1 -- 172.21.15.77:0/1511895148 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ab63b2cd10 con 0x55ab63c4da30 2024-09-10T10:01:08.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.678+0000 7fe843400640 10 client.26674 handle_command_reply: tid=0 2024-09-10T10:01:08.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.678+0000 7fe86b7e1740 1 client.26674 shutdown 2024-09-10T10:01:08.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.678+0000 7fe86b7e1740 1 -- 172.21.15.77:0/1511895148 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ab63bbeac0 msgr2=0x55ab63c40cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:08.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.678+0000 7fe86b7e1740 1 --2- 172.21.15.77:0/1511895148 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ab63bbeac0 0x55ab63c40cc0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe85402a940 tx=0x7fe854005740 comp rx=0 tx=0).stop 2024-09-10T10:01:08.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.679+0000 7fe86b7e1740 1 -- 172.21.15.77:0/1511895148 shutdown_connections 2024-09-10T10:01:08.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.679+0000 7fe86b7e1740 1 --2- 172.21.15.77:0/1511895148 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55ab63c4da30 0x55ab63c4fe50 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:08.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.679+0000 7fe86b7e1740 1 --2- 172.21.15.77:0/1511895148 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab63bc9ca0 0x55ab63c4bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:08.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.679+0000 7fe86b7e1740 1 --2- 172.21.15.77:0/1511895148 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab63bbf3e0 0x55ab63c41200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:08.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.679+0000 7fe86b7e1740 1 --2- 172.21.15.77:0/1511895148 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ab63bbeac0 0x55ab63c40cc0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:08.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.679+0000 7fe86b7e1740 1 -- 172.21.15.77:0/1511895148 >> 172.21.15.77:0/1511895148 conn(0x55ab63bbb050 msgr2=0x55ab63bb99a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:08.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.679+0000 7fe843400640 0 client.26674 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T10:01:08.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.679+0000 7fe86b7e1740 1 -- 172.21.15.77:0/1511895148 shutdown_connections 2024-09-10T10:01:08.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.680+0000 7fe86b7e1740 1 -- 172.21.15.77:0/1511895148 wait complete. 2024-09-10T10:01:08.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.680+0000 7fe86b7e1740 20 client.26674 trim_cache size 0 max 16384 2024-09-10T10:01:08.682 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-10T10:01:08.682 INFO:teuthology.orchestra.run.smithi077.stdout: "return_code": 0, 2024-09-10T10:01:08.682 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_tag": "02112086-8417-4834-835b-5391fc99190c", 2024-09-10T10:01:08.682 INFO:teuthology.orchestra.run.smithi077.stdout: "mode": "asynchronous" 2024-09-10T10:01:08.682 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-10T10:01:08.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.681+0000 7fe868c00640 1 -- 172.21.15.77:0/1684881326 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fe8480a2c00 msgr2=0x7fe8480a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:08.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.681+0000 7fe868c00640 1 --2- 172.21.15.77:0/1684881326 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fe8480a2c00 0x7fe8480a50c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fe85c00e690 tx=0x7fe85c00ab60 comp rx=0 tx=0).stop 2024-09-10T10:01:08.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.681+0000 7fe868c00640 1 -- 172.21.15.77:0/1684881326 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe8641049c0 msgr2=0x7fe86419ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:08.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.681+0000 7fe868c00640 1 --2- 172.21.15.77:0/1684881326 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe8641049c0 0x7fe86419ac20 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe85000e9f0 tx=0x7fe85000eeb0 comp rx=0 tx=0).stop 2024-09-10T10:01:08.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.681+0000 7fe868c00640 1 -- 172.21.15.77:0/1684881326 shutdown_connections 2024-09-10T10:01:08.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.681+0000 7fe868c00640 1 --2- 172.21.15.77:0/1684881326 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fe8480a2c00 0x7fe8480a50c0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:08.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.681+0000 7fe868c00640 1 --2- 172.21.15.77:0/1684881326 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe864109140 0x7fe86419efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:08.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.681+0000 7fe868c00640 1 --2- 172.21.15.77:0/1684881326 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe8641049c0 0x7fe86419ac20 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:08.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.681+0000 7fe868c00640 1 --2- 172.21.15.77:0/1684881326 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe864104010 0x7fe86419a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:08.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.681+0000 7fe868c00640 1 -- 172.21.15.77:0/1684881326 >> 172.21.15.77:0/1684881326 conn(0x7fe8640ffa50 msgr2=0x7fe864100e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:08.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.681+0000 7fe868c00640 1 -- 172.21.15.77:0/1684881326 shutdown_connections 2024-09-10T10:01:08.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:08.681+0000 7fe868c00640 1 -- 172.21.15.77:0/1684881326 wait complete. 2024-09-10T10:01:09.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:01:08 smithi077 ceph-mon[30319]: pgmap v570: 129 pgs: 129 active+clean; 118 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 1.8 MiB/s wr, 343 op/s 2024-09-10T10:01:09.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:01:08 smithi077 ceph-mon[30319]: scrub queued for path: / 2024-09-10T10:01:09.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:01:08 smithi077 ceph-mon[30319]: scrub summary: idle+waiting paths [/] 2024-09-10T10:01:09.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:01:08 smithi077 ceph-mon[30319]: scrub summary: active paths [/] 2024-09-10T10:01:09.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:01:08 smithi077 ceph-mon[30319]: scrub summary: active paths [/] 2024-09-10T10:01:09.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:01:08 smithi077 ceph-mon[30319]: scrub summary: idle+waiting paths [/] 2024-09-10T10:01:09.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:01:08 smithi134 ceph-mon[35674]: pgmap v570: 129 pgs: 129 active+clean; 118 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 1.8 MiB/s wr, 343 op/s 2024-09-10T10:01:09.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:01:08 smithi134 ceph-mon[35674]: scrub queued for path: / 2024-09-10T10:01:09.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:01:08 smithi134 ceph-mon[35674]: scrub summary: idle+waiting paths [/] 2024-09-10T10:01:09.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:01:08 smithi134 ceph-mon[35674]: scrub summary: active paths [/] 2024-09-10T10:01:09.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:01:08 smithi134 ceph-mon[35674]: scrub summary: active paths [/] 2024-09-10T10:01:09.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:01:08 smithi134 ceph-mon[35674]: scrub summary: idle+waiting paths [/] 2024-09-10T10:01:09.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:01:08 smithi120 ceph-mon[34770]: pgmap v570: 129 pgs: 129 active+clean; 118 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 1.8 MiB/s wr, 343 op/s 2024-09-10T10:01:09.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:01:08 smithi120 ceph-mon[34770]: scrub queued for path: / 2024-09-10T10:01:09.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:01:08 smithi120 ceph-mon[34770]: scrub summary: idle+waiting paths [/] 2024-09-10T10:01:09.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:01:08 smithi120 ceph-mon[34770]: scrub summary: active paths [/] 2024-09-10T10:01:09.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:01:08 smithi120 ceph-mon[34770]: scrub summary: active paths [/] 2024-09-10T10:01:09.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:01:08 smithi120 ceph-mon[34770]: scrub summary: idle+waiting paths [/] 2024-09-10T10:01:10.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:01:09 smithi077 ceph-mon[30319]: scrub summary: active paths [/] 2024-09-10T10:01:10.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:01:09 smithi077 ceph-mon[30319]: scrub summary: active paths [/] 2024-09-10T10:01:10.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:01:09 smithi077 ceph-mon[30319]: scrub summary: active paths [/] 2024-09-10T10:01:10.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:01:09 smithi077 ceph-mon[30319]: scrub summary: idle+waiting paths [/] 2024-09-10T10:01:10.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:01:09 smithi077 ceph-mon[30319]: scrub summary: idle+waiting paths [/] 2024-09-10T10:01:10.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:01:09 smithi077 ceph-mon[30319]: scrub summary: idle+waiting paths [/] 2024-09-10T10:01:10.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:01:09 smithi077 ceph-mon[30319]: scrub summary: idle+waiting paths [/] 2024-09-10T10:01:10.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:01:09 smithi134 ceph-mon[35674]: scrub summary: active paths [/] 2024-09-10T10:01:10.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:01:09 smithi134 ceph-mon[35674]: scrub summary: active paths [/] 2024-09-10T10:01:10.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:01:09 smithi134 ceph-mon[35674]: scrub summary: active paths [/] 2024-09-10T10:01:10.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:01:09 smithi134 ceph-mon[35674]: scrub summary: idle+waiting paths [/] 2024-09-10T10:01:10.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:01:09 smithi134 ceph-mon[35674]: scrub summary: idle+waiting paths [/] 2024-09-10T10:01:10.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:01:09 smithi134 ceph-mon[35674]: scrub summary: idle+waiting paths [/] 2024-09-10T10:01:10.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:01:09 smithi134 ceph-mon[35674]: scrub summary: idle+waiting paths [/] 2024-09-10T10:01:10.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:01:09 smithi120 ceph-mon[34770]: scrub summary: active paths [/] 2024-09-10T10:01:10.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:01:09 smithi120 ceph-mon[34770]: scrub summary: active paths [/] 2024-09-10T10:01:10.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:01:09 smithi120 ceph-mon[34770]: scrub summary: active paths [/] 2024-09-10T10:01:10.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:01:09 smithi120 ceph-mon[34770]: scrub summary: idle+waiting paths [/] 2024-09-10T10:01:10.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:01:09 smithi120 ceph-mon[34770]: scrub summary: idle+waiting paths [/] 2024-09-10T10:01:10.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:01:09 smithi120 ceph-mon[34770]: scrub summary: idle+waiting paths [/] 2024-09-10T10:01:10.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:01:09 smithi120 ceph-mon[34770]: scrub summary: idle+waiting paths [/] 2024-09-10T10:01:10.625 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T10:01:11.074 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:01:11.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:01:10 smithi077 ceph-mon[30319]: pgmap v571: 129 pgs: 129 active+clean; 114 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 2.4 MiB/s wr, 456 op/s 2024-09-10T10:01:11.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:01:10 smithi134 ceph-mon[35674]: pgmap v571: 129 pgs: 129 active+clean; 114 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 2.4 MiB/s wr, 456 op/s 2024-09-10T10:01:11.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:01:10 smithi120 ceph-mon[34770]: pgmap v571: 129 pgs: 129 active+clean; 114 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 2.4 MiB/s wr, 456 op/s 2024-09-10T10:01:12.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.063+0000 7f5898200640 1 -- 172.21.15.77:0/383365141 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f58901049c0 msgr2=0x7f5890108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:12.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.063+0000 7f5898200640 1 --2- 172.21.15.77:0/383365141 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f58901049c0 0x7f5890108a10 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f587c009900 tx=0x7f587c02a3f0 comp rx=0 tx=0).stop 2024-09-10T10:01:12.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.063+0000 7f5898200640 1 -- 172.21.15.77:0/383365141 shutdown_connections 2024-09-10T10:01:12.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.063+0000 7f5898200640 1 --2- 172.21.15.77:0/383365141 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5890109140 0x7f5890110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:12.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.063+0000 7f5898200640 1 --2- 172.21.15.77:0/383365141 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f58901049c0 0x7f5890108a10 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:12.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.063+0000 7f5898200640 1 --2- 172.21.15.77:0/383365141 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5890104010 0x7f58901043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:12.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.063+0000 7f5898200640 1 -- 172.21.15.77:0/383365141 >> 172.21.15.77:0/383365141 conn(0x7f58900ffa50 msgr2=0x7f5890101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:12.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.063+0000 7f5898200640 1 -- 172.21.15.77:0/383365141 shutdown_connections 2024-09-10T10:01:12.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.064+0000 7f5898200640 1 -- 172.21.15.77:0/383365141 wait complete. 2024-09-10T10:01:12.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.065+0000 7f5898200640 1 Processor -- start 2024-09-10T10:01:12.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.065+0000 7f5898200640 1 -- start start 2024-09-10T10:01:12.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.065+0000 7f5898200640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5890104010 0x7f589019a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:12.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.065+0000 7f5898200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f58901049c0 0x7f589019ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:12.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.065+0000 7f5898200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5890109140 0x7f589019efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:12.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.065+0000 7f5898200640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f58901141b0 con 0x7f58901049c0 2024-09-10T10:01:12.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.065+0000 7f5898200640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5890114030 con 0x7f5890104010 2024-09-10T10:01:12.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.065+0000 7f5898200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5890114330 con 0x7f5890109140 2024-09-10T10:01:12.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.066+0000 7f5896200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5890109140 0x7f589019efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:12.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.066+0000 7f5896200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5890109140 0x7f589019efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:53208/0 (socket says 172.21.15.77:53208) 2024-09-10T10:01:12.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.066+0000 7f5896200640 1 -- 172.21.15.77:0/655357967 learned_addr learned my addr 172.21.15.77:0/655357967 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:01:12.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.066+0000 7f5894e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f58901049c0 0x7f589019ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:12.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.066+0000 7f5895800640 1 --2- 172.21.15.77:0/655357967 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5890104010 0x7f589019a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:12.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.066+0000 7f5896200640 1 -- 172.21.15.77:0/655357967 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5890104010 msgr2=0x7f589019a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:12.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.066+0000 7f5896200640 1 --2- 172.21.15.77:0/655357967 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5890104010 0x7f589019a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:12.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.066+0000 7f5896200640 1 -- 172.21.15.77:0/655357967 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f58901049c0 msgr2=0x7f589019ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:12.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.066+0000 7f5896200640 1 --2- 172.21.15.77:0/655357967 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f58901049c0 0x7f589019ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:12.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.066+0000 7f5896200640 1 -- 172.21.15.77:0/655357967 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f589019f680 con 0x7f5890109140 2024-09-10T10:01:12.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.066+0000 7f5895800640 1 --2- 172.21.15.77:0/655357967 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5890104010 0x7f589019a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:01:12.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.066+0000 7f5896200640 1 --2- 172.21.15.77:0/655357967 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5890109140 0x7f589019efa0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f588800cce0 tx=0x7f588800a620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:12.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.067+0000 7f588e000640 1 -- 172.21.15.77:0/655357967 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f588800f450 con 0x7f5890109140 2024-09-10T10:01:12.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.067+0000 7f588e000640 1 -- 172.21.15.77:0/655357967 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f588800fad0 con 0x7f5890109140 2024-09-10T10:01:12.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.067+0000 7f588e000640 1 -- 172.21.15.77:0/655357967 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5888018c10 con 0x7f5890109140 2024-09-10T10:01:12.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.067+0000 7f5898200640 1 -- 172.21.15.77:0/655357967 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f589019f970 con 0x7f5890109140 2024-09-10T10:01:12.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.067+0000 7f5898200640 1 -- 172.21.15.77:0/655357967 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58901a7250 con 0x7f5890109140 2024-09-10T10:01:12.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.070+0000 7f589acd8740 1 Processor -- start 2024-09-10T10:01:12.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.071+0000 7f588e000640 1 -- 172.21.15.77:0/655357967 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f588800f5f0 con 0x7f5890109140 2024-09-10T10:01:12.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.071+0000 7f589acd8740 1 -- start start 2024-09-10T10:01:12.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.071+0000 7f589acd8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c92af1db90 0x55c92af1df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:12.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.071+0000 7f589acd8740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c92af1e4b0 0x55c92af28830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:12.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.071+0000 7f589acd8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c92af28d70 0x55c92af2b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:12.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.071+0000 7f589acd8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c92ae843e0 con 0x55c92af28d70 2024-09-10T10:01:12.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.071+0000 7f589acd8740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c92ae2bd80 con 0x55c92af1e4b0 2024-09-10T10:01:12.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.071+0000 7f589acd8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c92ae803a0 con 0x55c92af1db90 2024-09-10T10:01:12.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.072+0000 7f5895800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c92af1e4b0 0x55c92af28830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:12.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.072+0000 7f5895800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c92af1e4b0 0x55c92af28830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:56486/0 (socket says 172.21.15.77:56486) 2024-09-10T10:01:12.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.072+0000 7f5895800640 1 -- 172.21.15.77:0/4039191267 learned_addr learned my addr 172.21.15.77:0/4039191267 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:01:12.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.072+0000 7f5894e00640 1 --2- 172.21.15.77:0/4039191267 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c92af1db90 0x55c92af1df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:12.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.072+0000 7f5894e00640 1 --2- 172.21.15.77:0/4039191267 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c92af28d70 0x55c92af2b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:12.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.072+0000 7f588e000640 1 --2- 172.21.15.77:0/655357967 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f58780a2b80 0x7f58780a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:12.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.072+0000 7f5894e00640 1 -- 172.21.15.77:0/4039191267 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c92af1e4b0 msgr2=0x55c92af28830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:12.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.072+0000 7f5894e00640 1 --2- 172.21.15.77:0/4039191267 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c92af1e4b0 0x55c92af28830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:12.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.072+0000 7f5894e00640 1 -- 172.21.15.77:0/4039191267 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c92af28d70 msgr2=0x55c92af2b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:12.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.072+0000 7f5894e00640 1 --2- 172.21.15.77:0/4039191267 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c92af28d70 0x55c92af2b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:12.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.072+0000 7f5894e00640 1 -- 172.21.15.77:0/4039191267 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c92ae4aae0 con 0x55c92af1db90 2024-09-10T10:01:12.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.073+0000 7f5894e00640 1 --2- 172.21.15.77:0/4039191267 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c92af1db90 0x55c92af1df70 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f587c02a9e0 tx=0x7f587c009560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf5b5fd83278a19f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:12.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.073+0000 7f588e000640 1 -- 172.21.15.77:0/655357967 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f58880cfac0 con 0x7f5890109140 2024-09-10T10:01:12.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.073+0000 7f5896200640 1 --2- 172.21.15.77:0/655357967 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f58780a2b80 0x7f58780a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:12.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.073+0000 7f5882a00640 1 -- 172.21.15.77:0/4039191267 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f587c035030 con 0x55c92af1db90 2024-09-10T10:01:12.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.073+0000 7f5882a00640 1 -- 172.21.15.77:0/4039191267 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f587c004720 con 0x55c92af1db90 2024-09-10T10:01:12.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.073+0000 7f5882a00640 1 -- 172.21.15.77:0/4039191267 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f587c031930 con 0x55c92af1db90 2024-09-10T10:01:12.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.074+0000 7f589acd8740 1 -- 172.21.15.77:0/4039191267 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c92af1db90 msgr2=0x55c92af1df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:12.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.074+0000 7f589acd8740 1 --2- 172.21.15.77:0/4039191267 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c92af1db90 0x55c92af1df70 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f587c02a9e0 tx=0x7f587c009560 comp rx=0 tx=0).stop 2024-09-10T10:01:12.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.074+0000 7f5896200640 1 --2- 172.21.15.77:0/655357967 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f58780a2b80 0x7f58780a5040 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f5888098330 tx=0x7f58880186b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:12.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.075+0000 7f589acd8740 1 -- 172.21.15.77:0/4039191267 shutdown_connections 2024-09-10T10:01:12.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.075+0000 7f589acd8740 1 --2- 172.21.15.77:0/4039191267 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c92af28d70 0x55c92af2b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:12.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.075+0000 7f589acd8740 1 --2- 172.21.15.77:0/4039191267 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c92af1e4b0 0x55c92af28830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:12.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.075+0000 7f589acd8740 1 --2- 172.21.15.77:0/4039191267 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c92af1db90 0x55c92af1df70 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:12.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.075+0000 7f589acd8740 1 -- 172.21.15.77:0/4039191267 >> 172.21.15.77:0/4039191267 conn(0x55c92af1a120 msgr2=0x55c92af1a500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:12.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.075+0000 7f589acd8740 1 -- 172.21.15.77:0/4039191267 shutdown_connections 2024-09-10T10:01:12.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.075+0000 7f589acd8740 1 -- 172.21.15.77:0/4039191267 wait complete. 2024-09-10T10:01:12.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.076+0000 7f589acd8740 1 -- start start 2024-09-10T10:01:12.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.076+0000 7f589acd8740 1 Processor -- start 2024-09-10T10:01:12.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.077+0000 7f589acd8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c92af1db90 0x55c92af25d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:12.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.077+0000 7f589acd8740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c92af1e4b0 0x55c92af26290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:12.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.077+0000 7f589acd8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c92af28d70 0x55c92af267d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:12.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.077+0000 7f589acd8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c92af2b9e0 con 0x55c92af28d70 2024-09-10T10:01:12.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.077+0000 7f589acd8740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c92ae3f0f0 con 0x55c92af1e4b0 2024-09-10T10:01:12.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.077+0000 7f589acd8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c92af2bb60 con 0x55c92af1db90 2024-09-10T10:01:12.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.077+0000 7f589acd8740 10 client.0 _ll_register_callbacks cb 0x55c92ae309e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:01:12.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.077+0000 7f5895800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c92af1e4b0 0x55c92af26290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:12.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.077+0000 7f5894e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c92af1db90 0x55c92af25d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:12.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.077+0000 7f5895800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c92af1e4b0 0x55c92af26290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:56490/0 (socket says 172.21.15.77:56490) 2024-09-10T10:01:12.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.077+0000 7f5895800640 1 -- 172.21.15.77:0/949367135 learned_addr learned my addr 172.21.15.77:0/949367135 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:01:12.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.077+0000 7f5894e00640 1 --2- 172.21.15.77:0/949367135 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c92af28d70 0x55c92af267d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:12.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.078+0000 7f5895800640 1 -- 172.21.15.77:0/949367135 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c92af1db90 msgr2=0x55c92af25d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:12.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.078+0000 7f5895800640 1 --2- 172.21.15.77:0/949367135 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c92af1db90 0x55c92af25d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:12.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.078+0000 7f5895800640 1 -- 172.21.15.77:0/949367135 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c92af28d70 msgr2=0x55c92af267d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:12.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.078+0000 7f5895800640 1 --2- 172.21.15.77:0/949367135 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c92af28d70 0x55c92af267d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:12.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.078+0000 7f5895800640 1 -- 172.21.15.77:0/949367135 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c92abf53c0 con 0x55c92af1e4b0 2024-09-10T10:01:12.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.078+0000 7f5877400640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T10:01:12.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.078+0000 7f5895800640 1 --2- 172.21.15.77:0/949367135 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c92af1e4b0 0x55c92af26290 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5884012840 tx=0x7f5884012d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40a734ba4051b4ae server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:12.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.078+0000 7f5877400640 1 -- 172.21.15.77:0/949367135 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5884011b30 con 0x55c92af1e4b0 2024-09-10T10:01:12.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.078+0000 7f5877400640 1 -- 172.21.15.77:0/949367135 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f5884011cd0 con 0x55c92af1e4b0 2024-09-10T10:01:12.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.078+0000 7f589acd8740 1 -- 172.21.15.77:0/949367135 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f587c004720 con 0x55c92af1e4b0 2024-09-10T10:01:12.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.079+0000 7f5877400640 1 -- 172.21.15.77:0/949367135 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f5884005020 con 0x55c92af1e4b0 2024-09-10T10:01:12.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.079+0000 7f589acd8740 10 client.26738 fetch_fsmap learned FSMap version 35 2024-09-10T10:01:12.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.079+0000 7f589acd8740 1 -- 172.21.15.77:0/949367135 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55c92af3ffe0 con 0x55c92af1e4b0 2024-09-10T10:01:12.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.079+0000 7f5877400640 1 -- 172.21.15.77:0/949367135 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f587c004720 con 0x55c92af1e4b0 2024-09-10T10:01:12.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.079+0000 7f5877400640 1 -- 172.21.15.77:0/949367135 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f588401c470 con 0x55c92af1e4b0 2024-09-10T10:01:12.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.080+0000 7f589acd8740 10 client.26738 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T10:01:12.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.080+0000 7f589acd8740 10 client.26738 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T10:01:12.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.080+0000 7f589acd8740 20 client.26738 populate_metadata read hostname 'smithi077' 2024-09-10T10:01:12.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.080+0000 7f589acd8740 1 --2- 172.21.15.77:0/949367135 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55c92af414a0 0x55c92af43890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:12.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.080+0000 7f589acd8740 4 client.26738 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:01:12.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.080+0000 7f589acd8740 1 -- 172.21.15.77:0/949367135 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c92ae91950 con 0x55c92af414a0 2024-09-10T10:01:12.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.080+0000 7f5894e00640 1 --2- 172.21.15.77:0/949367135 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55c92af414a0 0x55c92af43890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:12.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.081+0000 7f5894e00640 1 --2- 172.21.15.77:0/949367135 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55c92af414a0 0x55c92af43890 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=544d68acfaf22614 server_cookie=73f052c1fdd2f13f in_seq=0 out_seq=0 2024-09-10T10:01:12.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.081+0000 7f5877400640 10 client.26738 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T10:01:12.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.082+0000 7f5877400640 1 -- 172.21.15.77:0/949367135 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c92ae91950 con 0x55c92af414a0 2024-09-10T10:01:12.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.082+0000 7f5877400640 10 client.26738 handle_command_reply: tid=0 2024-09-10T10:01:12.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.082+0000 7f589acd8740 1 client.26738 shutdown 2024-09-10T10:01:12.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.082+0000 7f589acd8740 1 -- 172.21.15.77:0/949367135 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c92af1e4b0 msgr2=0x55c92af26290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:12.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.082+0000 7f589acd8740 1 --2- 172.21.15.77:0/949367135 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c92af1e4b0 0x55c92af26290 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5884012840 tx=0x7f5884012d00 comp rx=0 tx=0).stop 2024-09-10T10:01:12.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.082+0000 7f589acd8740 1 -- 172.21.15.77:0/949367135 shutdown_connections 2024-09-10T10:01:12.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.082+0000 7f589acd8740 1 --2- 172.21.15.77:0/949367135 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55c92af414a0 0x55c92af43890 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:12.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.082+0000 7f589acd8740 1 --2- 172.21.15.77:0/949367135 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c92af28d70 0x55c92af267d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:12.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.082+0000 7f589acd8740 1 --2- 172.21.15.77:0/949367135 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c92af1e4b0 0x55c92af26290 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:12.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.082+0000 7f589acd8740 1 --2- 172.21.15.77:0/949367135 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c92af1db90 0x55c92af25d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:12.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.082+0000 7f5877400640 0 client.26738 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T10:01:12.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.082+0000 7f589acd8740 1 -- 172.21.15.77:0/949367135 >> 172.21.15.77:0/949367135 conn(0x55c92af1a120 msgr2=0x55c92af1fe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:12.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.082+0000 7f589acd8740 1 -- 172.21.15.77:0/949367135 shutdown_connections 2024-09-10T10:01:12.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.082+0000 7f589acd8740 1 -- 172.21.15.77:0/949367135 wait complete. 2024-09-10T10:01:12.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.083+0000 7f589acd8740 20 client.26738 trim_cache size 0 max 16384 2024-09-10T10:01:12.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.105+0000 7f589acd8740 1 Processor -- start 2024-09-10T10:01:12.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.105+0000 7f589acd8740 1 -- start start 2024-09-10T10:01:12.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.105+0000 7f589acd8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c92af1db90 0x55c92af490f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:12.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.105+0000 7f589acd8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c92af1e4b0 0x55c92af49630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:12.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.105+0000 7f589acd8740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c92af28d70 0x55c92af22fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:12.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.105+0000 7f589acd8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f58901accd0 con 0x55c92af1e4b0 2024-09-10T10:01:12.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.105+0000 7f589acd8740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f58901ae810 con 0x55c92af28d70 2024-09-10T10:01:12.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.105+0000 7f589acd8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f58901ad3a0 con 0x55c92af1db90 2024-09-10T10:01:12.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.105+0000 7f5894e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c92af1db90 0x55c92af490f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:12.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.106+0000 7f5894e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c92af1db90 0x55c92af490f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:53222/0 (socket says 172.21.15.77:53222) 2024-09-10T10:01:12.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.106+0000 7f5894e00640 1 -- 172.21.15.77:0/3891986295 learned_addr learned my addr 172.21.15.77:0/3891986295 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:01:12.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.106+0000 7f5894e00640 1 --2- 172.21.15.77:0/3891986295 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c92af28d70 0x55c92af22fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:12.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.106+0000 7f5895800640 1 --2- 172.21.15.77:0/3891986295 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c92af1e4b0 0x55c92af49630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:12.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.106+0000 7f5894e00640 1 -- 172.21.15.77:0/3891986295 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c92af28d70 msgr2=0x55c92af22fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:12.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.106+0000 7f5894e00640 1 --2- 172.21.15.77:0/3891986295 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c92af28d70 0x55c92af22fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:12.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.106+0000 7f5894e00640 1 -- 172.21.15.77:0/3891986295 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c92af1e4b0 msgr2=0x55c92af49630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:12.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.106+0000 7f5894e00640 1 --2- 172.21.15.77:0/3891986295 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c92af1e4b0 0x55c92af49630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:12.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.106+0000 7f5894e00640 1 -- 172.21.15.77:0/3891986295 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c92af43ec0 con 0x55c92af1db90 2024-09-10T10:01:12.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.106+0000 7f5894e00640 1 --2- 172.21.15.77:0/3891986295 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c92af1db90 0x55c92af490f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f587c031550 tx=0x7f587c02a3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27dbb336f2379f15 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:12.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.107+0000 7f5882000640 1 -- 172.21.15.77:0/3891986295 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f587c035040 con 0x55c92af1db90 2024-09-10T10:01:12.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.107+0000 7f5882000640 1 -- 172.21.15.77:0/3891986295 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f587c032250 con 0x55c92af1db90 2024-09-10T10:01:12.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.107+0000 7f5882000640 1 -- 172.21.15.77:0/3891986295 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f587c0319d0 con 0x55c92af1db90 2024-09-10T10:01:12.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.107+0000 7f589acd8740 1 -- 172.21.15.77:0/3891986295 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c92af1db90 msgr2=0x55c92af490f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:12.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.107+0000 7f589acd8740 1 --2- 172.21.15.77:0/3891986295 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c92af1db90 0x55c92af490f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f587c031550 tx=0x7f587c02a3f0 comp rx=0 tx=0).stop 2024-09-10T10:01:12.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.108+0000 7f589acd8740 1 -- 172.21.15.77:0/3891986295 shutdown_connections 2024-09-10T10:01:12.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.108+0000 7f589acd8740 1 --2- 172.21.15.77:0/3891986295 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c92af28d70 0x55c92af22fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:12.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.108+0000 7f589acd8740 1 --2- 172.21.15.77:0/3891986295 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c92af1e4b0 0x55c92af49630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:12.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.108+0000 7f589acd8740 1 --2- 172.21.15.77:0/3891986295 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c92af1db90 0x55c92af490f0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:12.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.108+0000 7f589acd8740 1 -- 172.21.15.77:0/3891986295 >> 172.21.15.77:0/3891986295 conn(0x55c92af1a120 msgr2=0x55c92af27dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:12.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.108+0000 7f589acd8740 1 -- 172.21.15.77:0/3891986295 shutdown_connections 2024-09-10T10:01:12.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.108+0000 7f589acd8740 1 -- 172.21.15.77:0/3891986295 wait complete. 2024-09-10T10:01:12.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.108+0000 7f589acd8740 1 -- start start 2024-09-10T10:01:12.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.108+0000 7f589acd8740 1 Processor -- start 2024-09-10T10:01:12.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.109+0000 7f589acd8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c92af1db90 0x55c92af35ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:12.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.109+0000 7f589acd8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c92af1e4b0 0x55c92af361e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:12.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.109+0000 7f589acd8740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c92af28d70 0x55c92afa5100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:12.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.109+0000 7f589acd8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f58901ae140 con 0x55c92af1db90 2024-09-10T10:01:12.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.109+0000 7f589acd8740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f58901ada70 con 0x55c92af28d70 2024-09-10T10:01:12.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.109+0000 7f589acd8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f58901aacb0 con 0x55c92af1e4b0 2024-09-10T10:01:12.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.109+0000 7f589acd8740 10 client.0 _ll_register_callbacks cb 0x7f5884012450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:01:12.113 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.109+0000 7f5894e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c92af1db90 0x55c92af35ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:12.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.109+0000 7f5895800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c92af1e4b0 0x55c92af361e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:12.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.109+0000 7f5895800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c92af1e4b0 0x55c92af361e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:53236/0 (socket says 172.21.15.77:53236) 2024-09-10T10:01:12.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.109+0000 7f5894e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c92af1db90 0x55c92af35ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:55554/0 (socket says 172.21.15.77:55554) 2024-09-10T10:01:12.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.109+0000 7f5895800640 1 -- 172.21.15.77:0/263907243 learned_addr learned my addr 172.21.15.77:0/263907243 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:01:12.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.110+0000 7f5894e00640 1 --2- 172.21.15.77:0/263907243 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c92af28d70 0x55c92afa5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:12.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.110+0000 7f5895800640 1 -- 172.21.15.77:0/263907243 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c92af28d70 msgr2=0x55c92afa5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:12.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.110+0000 7f5895800640 1 --2- 172.21.15.77:0/263907243 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c92af28d70 0x55c92afa5100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:12.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.110+0000 7f5895800640 1 -- 172.21.15.77:0/263907243 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c92af1db90 msgr2=0x55c92af35ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:12.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.110+0000 7f5895800640 1 --2- 172.21.15.77:0/263907243 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c92af1db90 0x55c92af35ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:12.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.110+0000 7f5895800640 1 -- 172.21.15.77:0/263907243 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c92af199d0 con 0x55c92af1e4b0 2024-09-10T10:01:12.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.110+0000 7f5894e00640 1 --2- 172.21.15.77:0/263907243 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c92af1db90 0x55c92af35ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:01:12.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.110+0000 7f5895800640 1 --2- 172.21.15.77:0/263907243 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c92af1e4b0 0x55c92af361e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f588401c890 tx=0x7f5884011de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1954c17db1848a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:12.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.110+0000 7f5877400640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-10T10:01:12.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.110+0000 7f5877400640 1 -- 172.21.15.77:0/263907243 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f588402b070 con 0x55c92af1e4b0 2024-09-10T10:01:12.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.110+0000 7f589acd8740 1 -- 172.21.15.77:0/263907243 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f587c032250 con 0x55c92af1e4b0 2024-09-10T10:01:12.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.110+0000 7f5877400640 1 -- 172.21.15.77:0/263907243 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f5884009e40 con 0x55c92af1e4b0 2024-09-10T10:01:12.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.111+0000 7f5877400640 1 -- 172.21.15.77:0/263907243 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f5884009530 con 0x55c92af1e4b0 2024-09-10T10:01:12.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.111+0000 7f589acd8740 10 client.26692 fetch_fsmap learned FSMap version 35 2024-09-10T10:01:12.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.111+0000 7f589acd8740 1 -- 172.21.15.77:0/263907243 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55c92afac960 con 0x55c92af1e4b0 2024-09-10T10:01:12.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.111+0000 7f5877400640 1 -- 172.21.15.77:0/263907243 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f587c032250 con 0x55c92af1e4b0 2024-09-10T10:01:12.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.111+0000 7f5877400640 1 -- 172.21.15.77:0/263907243 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f58840049a0 con 0x55c92af1e4b0 2024-09-10T10:01:12.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.111+0000 7f589acd8740 10 client.26692 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T10:01:12.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.111+0000 7f589acd8740 10 client.26692 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T10:01:12.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.111+0000 7f589acd8740 20 client.26692 populate_metadata read hostname 'smithi077' 2024-09-10T10:01:12.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.111+0000 7f589acd8740 1 --2- 172.21.15.77:0/263907243 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55c92afad0d0 0x55c92afaf4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:12.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.111+0000 7f589acd8740 4 client.26692 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T10:01:12.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.111+0000 7f589acd8740 1 -- 172.21.15.77:0/263907243 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c92ae8be60 con 0x55c92afad0d0 2024-09-10T10:01:12.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.111+0000 7f5894e00640 1 --2- 172.21.15.77:0/263907243 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55c92afad0d0 0x55c92afaf4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:12.117 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.112+0000 7f5877400640 10 client.26692 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T10:01:12.117 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.112+0000 7f5894e00640 1 --2- 172.21.15.77:0/263907243 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55c92afad0d0 0x55c92afaf4c0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0ca82ca1b79de83 server_cookie=b0644ac4cf432e12 in_seq=0 out_seq=0 2024-09-10T10:01:12.117 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.112+0000 7f5877400640 1 -- 172.21.15.77:0/263907243 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55c92ae8be60 con 0x55c92afad0d0 2024-09-10T10:01:12.117 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.112+0000 7f5877400640 10 client.26692 handle_command_reply: tid=0 2024-09-10T10:01:12.117 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.112+0000 7f589acd8740 1 client.26692 shutdown 2024-09-10T10:01:12.117 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.112+0000 7f589acd8740 1 -- 172.21.15.77:0/263907243 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c92af1e4b0 msgr2=0x55c92af361e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:12.117 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.112+0000 7f589acd8740 1 --2- 172.21.15.77:0/263907243 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c92af1e4b0 0x55c92af361e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f588401c890 tx=0x7f5884011de0 comp rx=0 tx=0).stop 2024-09-10T10:01:12.117 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.113+0000 7f589acd8740 1 -- 172.21.15.77:0/263907243 shutdown_connections 2024-09-10T10:01:12.117 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.113+0000 7f589acd8740 1 --2- 172.21.15.77:0/263907243 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55c92afad0d0 0x55c92afaf4c0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:12.117 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.113+0000 7f589acd8740 1 --2- 172.21.15.77:0/263907243 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c92af28d70 0x55c92afa5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:12.117 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.113+0000 7f589acd8740 1 --2- 172.21.15.77:0/263907243 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c92af1e4b0 0x55c92af361e0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:12.117 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.113+0000 7f589acd8740 1 --2- 172.21.15.77:0/263907243 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c92af1db90 0x55c92af35ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:12.117 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.113+0000 7f589acd8740 1 -- 172.21.15.77:0/263907243 >> 172.21.15.77:0/263907243 conn(0x55c92af1a120 msgr2=0x55c92af275b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:12.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.113+0000 7f5877400640 0 client.26692 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T10:01:12.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.113+0000 7f589acd8740 1 -- 172.21.15.77:0/263907243 shutdown_connections 2024-09-10T10:01:12.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.113+0000 7f589acd8740 1 -- 172.21.15.77:0/263907243 wait complete. 2024-09-10T10:01:12.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.113+0000 7f589acd8740 20 client.26692 trim_cache size 0 max 16384 2024-09-10T10:01:12.118 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-10T10:01:12.118 INFO:teuthology.orchestra.run.smithi077.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T10:01:12.118 INFO:teuthology.orchestra.run.smithi077.stdout: "scrubs": { 2024-09-10T10:01:12.118 INFO:teuthology.orchestra.run.smithi077.stdout: "02112086-8417-4834-835b-5391fc99190c": { 2024-09-10T10:01:12.118 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/", 2024-09-10T10:01:12.118 INFO:teuthology.orchestra.run.smithi077.stdout: "tag": "02112086-8417-4834-835b-5391fc99190c", 2024-09-10T10:01:12.118 INFO:teuthology.orchestra.run.smithi077.stdout: "options": "recursive,force" 2024-09-10T10:01:12.119 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:01:12.119 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:01:12.119 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-10T10:01:12.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.114+0000 7f5898200640 1 -- 172.21.15.77:0/655357967 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f58780a2b80 msgr2=0x7f58780a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:12.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.114+0000 7f5898200640 1 --2- 172.21.15.77:0/655357967 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f58780a2b80 0x7f58780a5040 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f5888098330 tx=0x7f58880186b0 comp rx=0 tx=0).stop 2024-09-10T10:01:12.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.114+0000 7f5898200640 1 -- 172.21.15.77:0/655357967 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5890109140 msgr2=0x7f589019efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:12.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.114+0000 7f5898200640 1 --2- 172.21.15.77:0/655357967 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5890109140 0x7f589019efa0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f588800cce0 tx=0x7f588800a620 comp rx=0 tx=0).stop 2024-09-10T10:01:12.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.114+0000 7f5898200640 1 -- 172.21.15.77:0/655357967 shutdown_connections 2024-09-10T10:01:12.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.114+0000 7f5898200640 1 --2- 172.21.15.77:0/655357967 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f58780a2b80 0x7f58780a5040 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:12.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.114+0000 7f5898200640 1 --2- 172.21.15.77:0/655357967 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5890109140 0x7f589019efa0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:12.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.114+0000 7f5898200640 1 --2- 172.21.15.77:0/655357967 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f58901049c0 0x7f589019ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:12.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.114+0000 7f5898200640 1 --2- 172.21.15.77:0/655357967 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5890104010 0x7f589019a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:12.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.114+0000 7f5898200640 1 -- 172.21.15.77:0/655357967 >> 172.21.15.77:0/655357967 conn(0x7f58900ffa50 msgr2=0x7f5890100d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:12.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.115+0000 7f5898200640 1 -- 172.21.15.77:0/655357967 shutdown_connections 2024-09-10T10:01:12.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:12.115+0000 7f5898200640 1 -- 172.21.15.77:0/655357967 wait complete. 2024-09-10T10:01:13.065 INFO:tasks.cephfs.filesystem:scrub status for tag:02112086-8417-4834-835b-5391fc99190c - {'path': '/', 'tag': '02112086-8417-4834-835b-5391fc99190c', 'options': 'recursive,force'} 2024-09-10T10:01:13.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:01:12 smithi077 ceph-mon[30319]: pgmap v572: 129 pgs: 129 active+clean; 115 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.9 MiB/s wr, 339 op/s 2024-09-10T10:01:13.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:01:12 smithi134 ceph-mon[35674]: pgmap v572: 129 pgs: 129 active+clean; 115 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.9 MiB/s wr, 339 op/s 2024-09-10T10:01:13.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:01:12 smithi120 ceph-mon[34770]: pgmap v572: 129 pgs: 129 active+clean; 115 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.9 MiB/s wr, 339 op/s 2024-09-10T10:01:15.226 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:01:14 smithi120 ceph-mon[34770]: pgmap v573: 129 pgs: 129 active+clean; 117 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 2.1 MiB/s wr, 401 op/s 2024-09-10T10:01:15.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:01:14 smithi134 ceph-mon[35674]: pgmap v573: 129 pgs: 129 active+clean; 117 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 2.1 MiB/s wr, 401 op/s 2024-09-10T10:01:15.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:01:14 smithi077 ceph-mon[30319]: pgmap v573: 129 pgs: 129 active+clean; 117 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 2.1 MiB/s wr, 401 op/s 2024-09-10T10:01:16.564 INFO:tasks.workunit.client.0.smithi077.stdout:50000 write 0x5304f4 thru 0x53fa11 (0xf51e bytes) 2024-09-10T10:01:16.564 INFO:tasks.workunit.client.0.smithi077.stdout:All 50000 operations completed A-OK! 2024-09-10T10:01:16.567 INFO:tasks.workunit.client.0.smithi077.stderr:+ ./fsx -z 100MB -N 50000 -p 10000 -l 104857600 2024-09-10T10:01:16.572 INFO:tasks.workunit.client.0.smithi077.stdout:Seed set to 1 2024-09-10T10:01:17.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:01:16 smithi134 ceph-mon[35674]: pgmap v574: 129 pgs: 129 active+clean; 119 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.1 MiB/s wr, 351 op/s 2024-09-10T10:01:17.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:01:16 smithi120 ceph-mon[34770]: pgmap v574: 129 pgs: 129 active+clean; 119 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.1 MiB/s wr, 351 op/s 2024-09-10T10:01:17.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:01:16 smithi077 ceph-mon[30319]: pgmap v574: 129 pgs: 129 active+clean; 119 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.1 MiB/s wr, 351 op/s 2024-09-10T10:01:17.842 INFO:tasks.workunit.client.0.smithi077.stderr:main: filesystem does not support fallocate mode 0, disabling! 2024-09-10T10:01:17.842 INFO:tasks.workunit.client.0.smithi077.stderr:main: filesystem does not support fallocate mode FALLOC_FL_KEEP_SIZE, disabling! 2024-09-10T10:01:17.845 INFO:tasks.workunit.client.0.smithi077.stderr:main: filesystem does not support fallocate mode FALLOC_FL_COLLAPSE_RANGE, disabling! 2024-09-10T10:01:17.845 INFO:tasks.workunit.client.0.smithi077.stderr:main: filesystem does not support fallocate mode FALLOC_FL_INSERT_RANGE, disabling! 2024-09-10T10:01:17.846 INFO:tasks.workunit.client.0.smithi077.stderr:main: filesystem does not support clone range, disabling! 2024-09-10T10:01:17.849 INFO:tasks.workunit.client.0.smithi077.stderr:main: filesystem does not support dedupe range, disabling! 2024-09-10T10:01:17.852 INFO:tasks.workunit.client.0.smithi077.stderr:main: filesystem does not support exchange range, disabling! 2024-09-10T10:01:17.852 INFO:tasks.workunit.client.0.smithi077.stdout:skipping zero size read 2024-09-10T10:01:17.869 INFO:tasks.workunit.client.0.smithi077.stdout:truncating to largest ever: 0x1c37b90 2024-09-10T10:01:17.877 INFO:tasks.workunit.client.0.smithi077.stdout:truncating to largest ever: 0x5bd330d 2024-09-10T10:01:17.907 INFO:tasks.workunit.client.0.smithi077.stdout:copying to largest ever: 0x637bf99 2024-09-10T10:01:18.316 INFO:tasks.workunit.client.0.smithi077.stdout:copying to largest ever: 0x63d9b9c 2024-09-10T10:01:19.241 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:01:18 smithi077 ceph-mon[30319]: pgmap v575: 129 pgs: 129 active+clean; 120 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 1.9 MiB/s wr, 327 op/s 2024-09-10T10:01:19.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:01:18 smithi134 ceph-mon[35674]: pgmap v575: 129 pgs: 129 active+clean; 120 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 1.9 MiB/s wr, 327 op/s 2024-09-10T10:01:19.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:01:18 smithi120 ceph-mon[34770]: pgmap v575: 129 pgs: 129 active+clean; 120 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 1.9 MiB/s wr, 327 op/s 2024-09-10T10:01:21.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:01:20 smithi134 ceph-mon[35674]: pgmap v576: 129 pgs: 129 active+clean; 132 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 2.2 MiB/s wr, 405 op/s 2024-09-10T10:01:21.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:01:20 smithi134 ceph-mon[35674]: scrub summary: idle 2024-09-10T10:01:21.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:01:20 smithi134 ceph-mon[35674]: scrub summary: idle 2024-09-10T10:01:21.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:01:20 smithi120 ceph-mon[34770]: pgmap v576: 129 pgs: 129 active+clean; 132 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 2.2 MiB/s wr, 405 op/s 2024-09-10T10:01:21.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:01:20 smithi120 ceph-mon[34770]: scrub summary: idle 2024-09-10T10:01:21.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:01:20 smithi120 ceph-mon[34770]: scrub summary: idle 2024-09-10T10:01:21.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:01:20 smithi077 ceph-mon[30319]: pgmap v576: 129 pgs: 129 active+clean; 132 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 2.2 MiB/s wr, 405 op/s 2024-09-10T10:01:21.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:01:20 smithi077 ceph-mon[30319]: scrub summary: idle 2024-09-10T10:01:21.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:01:20 smithi077 ceph-mon[30319]: scrub summary: idle 2024-09-10T10:01:22.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:01:21 smithi134 ceph-mon[35674]: scrub summary: idle 2024-09-10T10:01:22.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:01:21 smithi134 ceph-mon[35674]: scrub summary: idle 2024-09-10T10:01:22.382 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:01:21 smithi134 ceph-mon[35674]: scrub summary: idle 2024-09-10T10:01:22.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:01:21 smithi120 ceph-mon[34770]: scrub summary: idle 2024-09-10T10:01:22.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:01:21 smithi120 ceph-mon[34770]: scrub summary: idle 2024-09-10T10:01:22.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:01:21 smithi120 ceph-mon[34770]: scrub summary: idle 2024-09-10T10:01:22.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:01:21 smithi077 ceph-mon[30319]: scrub summary: idle 2024-09-10T10:01:22.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:01:21 smithi077 ceph-mon[30319]: scrub summary: idle 2024-09-10T10:01:22.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:01:21 smithi077 ceph-mon[30319]: scrub summary: idle 2024-09-10T10:01:23.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:01:22 smithi134 ceph-mon[35674]: pgmap v577: 129 pgs: 129 active+clean; 133 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.6 MiB/s wr, 292 op/s 2024-09-10T10:01:23.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:01:22 smithi120 ceph-mon[34770]: pgmap v577: 129 pgs: 129 active+clean; 133 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.6 MiB/s wr, 292 op/s 2024-09-10T10:01:23.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:01:22 smithi077 ceph-mon[30319]: pgmap v577: 129 pgs: 129 active+clean; 133 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.6 MiB/s wr, 292 op/s 2024-09-10T10:01:25.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:01:24 smithi134 ceph-mon[35674]: pgmap v578: 129 pgs: 129 active+clean; 141 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 1.7 MiB/s wr, 379 op/s 2024-09-10T10:01:25.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:01:24 smithi120 ceph-mon[34770]: pgmap v578: 129 pgs: 129 active+clean; 141 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 1.7 MiB/s wr, 379 op/s 2024-09-10T10:01:25.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:01:24 smithi077 ceph-mon[30319]: pgmap v578: 129 pgs: 129 active+clean; 141 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 1.7 MiB/s wr, 379 op/s 2024-09-10T10:01:27.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:01:26 smithi134 ceph-mon[35674]: pgmap v579: 129 pgs: 129 active+clean; 142 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.5 MiB/s wr, 339 op/s 2024-09-10T10:01:27.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:01:26 smithi120 ceph-mon[34770]: pgmap v579: 129 pgs: 129 active+clean; 142 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.5 MiB/s wr, 339 op/s 2024-09-10T10:01:27.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:01:26 smithi077 ceph-mon[30319]: pgmap v579: 129 pgs: 129 active+clean; 142 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.5 MiB/s wr, 339 op/s 2024-09-10T10:01:29.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:01:28 smithi134 ceph-mon[35674]: pgmap v580: 129 pgs: 129 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.2 MiB/s wr, 339 op/s 2024-09-10T10:01:29.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:01:28 smithi120 ceph-mon[34770]: pgmap v580: 129 pgs: 129 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.2 MiB/s wr, 339 op/s 2024-09-10T10:01:29.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:01:28 smithi077 ceph-mon[30319]: pgmap v580: 129 pgs: 129 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.2 MiB/s wr, 339 op/s 2024-09-10T10:01:31.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:01:30 smithi134 ceph-mon[35674]: pgmap v581: 129 pgs: 129 active+clean; 138 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 1.5 MiB/s wr, 480 op/s 2024-09-10T10:01:31.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:01:30 smithi120 ceph-mon[34770]: pgmap v581: 129 pgs: 129 active+clean; 138 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 1.5 MiB/s wr, 480 op/s 2024-09-10T10:01:31.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:01:30 smithi077 ceph-mon[30319]: pgmap v581: 129 pgs: 129 active+clean; 138 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 1.5 MiB/s wr, 480 op/s 2024-09-10T10:01:33.149 INFO:tasks.quiescer.fs.[cephfs]:Total duration (40.2) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-10T10:01:33.150 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.1 2024-09-10T10:01:33.150 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:01:33.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:01:32 smithi134 ceph-mon[35674]: pgmap v582: 129 pgs: 129 active+clean; 141 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 1.0 MiB/s wr, 415 op/s 2024-09-10T10:01:33.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:01:32 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:01:33.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:01:32 smithi120 ceph-mon[34770]: pgmap v582: 129 pgs: 129 active+clean; 141 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 1.0 MiB/s wr, 415 op/s 2024-09-10T10:01:33.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:01:32 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:01:33.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:01:32 smithi077 ceph-mon[30319]: pgmap v582: 129 pgs: 129 active+clean; 141 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 1.0 MiB/s wr, 415 op/s 2024-09-10T10:01:33.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:01:32 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:01:33.641 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:01:34.054 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:01:33 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:01:34.055 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:01:33 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:01:34.055 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:01:33 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:01:34.055 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:01:33 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:01:34.055 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:01:33 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:01:34.055 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:01:33 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:01:34.055 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:01:33 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:01:34.055 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:01:33 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:01:34.055 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:01:33 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:01:34.055 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:01:33 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:01:34.055 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:01:33 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:01:34.056 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:01:33 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:01:34.056 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:01:33 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:01:34.056 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:01:33 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:01:34.056 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:01:33 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T10:01:34.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:34.343+0000 7fed5de00640 1 -- 172.21.15.77:0/2761526203 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed5810e210 msgr2=0x7fed58110660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:34.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:34.343+0000 7fed5de00640 1 --2- 172.21.15.77:0/2761526203 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed5810e210 0x7fed58110660 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fed50009ca0 tx=0x7fed5002a950 comp rx=0 tx=0).stop 2024-09-10T10:01:34.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:34.343+0000 7fed5de00640 1 -- 172.21.15.77:0/2761526203 shutdown_connections 2024-09-10T10:01:34.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:34.343+0000 7fed5de00640 1 --2- 172.21.15.77:0/2761526203 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed5810e210 0x7fed58110660 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:34.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:34.343+0000 7fed5de00640 1 --2- 172.21.15.77:0/2761526203 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fed58074f40 0x7fed5810dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:34.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:34.343+0000 7fed5de00640 1 --2- 172.21.15.77:0/2761526203 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fed58074620 0x7fed58074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:34.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:34.343+0000 7fed5de00640 1 -- 172.21.15.77:0/2761526203 >> 172.21.15.77:0/2761526203 conn(0x7fed580fd710 msgr2=0x7fed580ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:34.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:34.343+0000 7fed5de00640 1 -- 172.21.15.77:0/2761526203 shutdown_connections 2024-09-10T10:01:34.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:34.343+0000 7fed5de00640 1 -- 172.21.15.77:0/2761526203 wait complete. 2024-09-10T10:01:34.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:34.344+0000 7fed5de00640 1 Processor -- start 2024-09-10T10:01:34.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:34.345+0000 7fed5de00640 1 -- start start 2024-09-10T10:01:34.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:34.345+0000 7fed5de00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fed58074620 0x7fed58101930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:34.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:34.345+0000 7fed5de00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed58074f40 0x7fed58101e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:34.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:34.345+0000 7fed5de00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fed5810e210 0x7fed581023b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:34.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:34.345+0000 7fed5de00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fed58113ff0 con 0x7fed5810e210 2024-09-10T10:01:34.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:34.345+0000 7fed5de00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fed58113e70 con 0x7fed58074620 2024-09-10T10:01:34.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:34.345+0000 7fed5de00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fed58114170 con 0x7fed58074f40 2024-09-10T10:01:34.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:34.346+0000 7fed57e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed58074f40 0x7fed58101e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:34.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:34.346+0000 7fed5ca00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fed58074620 0x7fed58101930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:34.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:34.346+0000 7fed57e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed58074f40 0x7fed58101e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:43580/0 (socket says 172.21.15.77:43580) 2024-09-10T10:01:34.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:34.346+0000 7fed57e00640 1 -- 172.21.15.77:0/3890786211 learned_addr learned my addr 172.21.15.77:0/3890786211 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:01:34.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:34.346+0000 7fed5d400640 1 --2- 172.21.15.77:0/3890786211 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fed5810e210 0x7fed581023b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:34.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:34.347+0000 7fed5d400640 1 -- 172.21.15.77:0/3890786211 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed58074f40 msgr2=0x7fed58101e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:34.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:34.347+0000 7fed5d400640 1 --2- 172.21.15.77:0/3890786211 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed58074f40 0x7fed58101e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:34.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:34.347+0000 7fed5d400640 1 -- 172.21.15.77:0/3890786211 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fed58074620 msgr2=0x7fed58101930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:34.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:34.347+0000 7fed5d400640 1 --2- 172.21.15.77:0/3890786211 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fed58074620 0x7fed58101930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:34.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:34.347+0000 7fed5d400640 1 -- 172.21.15.77:0/3890786211 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed581aab40 con 0x7fed5810e210 2024-09-10T10:01:34.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:34.347+0000 7fed5d400640 1 --2- 172.21.15.77:0/3890786211 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fed5810e210 0x7fed581023b0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fed5002af40 tx=0x7fed50003b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:34.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:34.348+0000 7fed55600640 1 -- 172.21.15.77:0/3890786211 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed50034040 con 0x7fed5810e210 2024-09-10T10:01:34.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:34.348+0000 7fed55600640 1 -- 172.21.15.77:0/3890786211 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed5002f940 con 0x7fed5810e210 2024-09-10T10:01:34.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:34.348+0000 7fed55600640 1 -- 172.21.15.77:0/3890786211 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed50004610 con 0x7fed5810e210 2024-09-10T10:01:34.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:34.349+0000 7fed5de00640 1 -- 172.21.15.77:0/3890786211 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed581aadd0 con 0x7fed5810e210 2024-09-10T10:01:34.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:34.349+0000 7fed5de00640 1 -- 172.21.15.77:0/3890786211 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed581ab260 con 0x7fed5810e210 2024-09-10T10:01:34.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:34.351+0000 7fed5de00640 1 -- 172.21.15.77:0/3890786211 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed58075cd0 con 0x7fed5810e210 2024-09-10T10:01:34.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:34.355+0000 7fed55600640 1 -- 172.21.15.77:0/3890786211 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fed50037020 con 0x7fed5810e210 2024-09-10T10:01:34.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:34.356+0000 7fed55600640 1 --2- 172.21.15.77:0/3890786211 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fed400a2c00 0x7fed400a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:34.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:34.357+0000 7fed55600640 1 -- 172.21.15.77:0/3890786211 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fed500f0140 con 0x7fed5810e210 2024-09-10T10:01:34.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:34.359+0000 7fed5ca00640 1 --2- 172.21.15.77:0/3890786211 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fed400a2c00 0x7fed400a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:34.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:34.360+0000 7fed5ca00640 1 --2- 172.21.15.77:0/3890786211 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fed400a2c00 0x7fed400a50c0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fed4c009bd0 tx=0x7fed4c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:34.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:34.362+0000 7fed55600640 1 -- 172.21.15.77:0/3890786211 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fed500bcc50 con 0x7fed5810e210 2024-09-10T10:01:34.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:01:33 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:01:34.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:01:33 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:01:34.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:01:33 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:01:34.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:01:33 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:01:34.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:01:33 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:01:34.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:01:33 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:01:34.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:01:33 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:01:34.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:01:33 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:01:34.382 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:01:33 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:01:34.382 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:01:33 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:01:34.382 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:01:33 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:01:34.382 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:01:33 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:01:34.382 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:01:33 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:01:34.382 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:01:33 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:01:34.382 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:01:33 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T10:01:34.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:01:33 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:01:34.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:01:33 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:01:34.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:01:33 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:01:34.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:01:33 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:01:34.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:01:33 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:01:34.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:01:33 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:01:34.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:01:33 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:01:34.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:01:33 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:01:34.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:01:33 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:01:34.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:01:33 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:01:34.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:01:33 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:01:34.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:01:33 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:01:34.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:01:33 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:01:34.425 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:01:33 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:01:34.425 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:01:33 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T10:01:34.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:34.635+0000 7fed5de00640 1 -- 172.21.15.77:0/3890786211 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fed58102e60 con 0x7fed5810e210 2024-09-10T10:01:34.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:34.637+0000 7fed55600640 1 -- 172.21.15.77:0/3890786211 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11989 (secure 0 0 0) 0x7fed500bc550 con 0x7fed5810e210 2024-09-10T10:01:34.639 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T10:01:34.639 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":35,"btime":"2024-09-10T10:01:05:922537+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3159289675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3159289675},{"type":"v1","addr":"172.21.15.77:6835","nonce":3159289675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1951962341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1951962341},{"type":"v1","addr":"172.21.15.77:6837","nonce":1951962341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":35,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:52:23.368252+0000","modified":"2024-09-10T10:01:05.706646+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24563,"mds_1":24524,"mds_2":24520,"mds_3":24502,"mds_4":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14487":{"gid":14487,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/4291121888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":4291121888},{"type":"v1","addr":"172.21.15.77:6839","nonce":4291121888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14499":{"gid":14499,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6841/445284194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":445284194},{"type":"v1","addr":"172.21.15.77:6841","nonce":445284194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1005054423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1005054423},{"type":"v1","addr":"172.21.15.134:6833","nonce":1005054423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6833/3716307744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":3716307744},{"type":"v1","addr":"172.21.15.120:6833","nonce":3716307744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/1319892544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1319892544},{"type":"v1","addr":"172.21.15.120:6835","nonce":1319892544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.134:6835/3293933699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3293933699},{"type":"v1","addr":"172.21.15.134:6835","nonce":3293933699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.134:6837/2461497091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2461497091},{"type":"v1","addr":"172.21.15.134:6837","nonce":2461497091}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":20,"addr":"172.21.15.134:6839/1965117680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1965117680},{"type":"v1","addr":"172.21.15.134:6839","nonce":1965117680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":20,"addr":"172.21.15.120:6837/3033280709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3033280709},{"type":"v1","addr":"172.21.15.120:6837","nonce":3033280709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.120:6839/1777710157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1777710157},{"type":"v1","addr":"172.21.15.120:6839","nonce":1777710157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24563,"qdb_cluster":[24496,24502,24520,24524,24563]},"id":1}]} 2024-09-10T10:01:34.639 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 35 2024-09-10T10:01:34.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:34.639+0000 7fed5de00640 1 -- 172.21.15.77:0/3890786211 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fed400a2c00 msgr2=0x7fed400a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:34.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:34.639+0000 7fed5de00640 1 --2- 172.21.15.77:0/3890786211 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fed400a2c00 0x7fed400a50c0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fed4c009bd0 tx=0x7fed4c005f90 comp rx=0 tx=0).stop 2024-09-10T10:01:34.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:34.639+0000 7fed5de00640 1 -- 172.21.15.77:0/3890786211 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fed5810e210 msgr2=0x7fed581023b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:34.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:34.639+0000 7fed5de00640 1 --2- 172.21.15.77:0/3890786211 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fed5810e210 0x7fed581023b0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fed5002af40 tx=0x7fed50003b80 comp rx=0 tx=0).stop 2024-09-10T10:01:34.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:34.640+0000 7fed5de00640 1 -- 172.21.15.77:0/3890786211 shutdown_connections 2024-09-10T10:01:34.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:34.640+0000 7fed5de00640 1 --2- 172.21.15.77:0/3890786211 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fed400a2c00 0x7fed400a50c0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:34.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:34.640+0000 7fed5de00640 1 --2- 172.21.15.77:0/3890786211 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fed5810e210 0x7fed581023b0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:34.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:34.640+0000 7fed5de00640 1 --2- 172.21.15.77:0/3890786211 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed58074f40 0x7fed58101e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:34.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:34.640+0000 7fed5de00640 1 --2- 172.21.15.77:0/3890786211 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fed58074620 0x7fed58101930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:34.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:34.640+0000 7fed5de00640 1 -- 172.21.15.77:0/3890786211 >> 172.21.15.77:0/3890786211 conn(0x7fed580fd710 msgr2=0x7fed580fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:34.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:34.640+0000 7fed5de00640 1 -- 172.21.15.77:0/3890786211 shutdown_connections 2024-09-10T10:01:34.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:34.640+0000 7fed5de00640 1 -- 172.21.15.77:0/3890786211 wait complete. 2024-09-10T10:01:35.382 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:01:34 smithi134 ceph-mon[35674]: Adjusting osd_memory_target on smithi120 to 176.0M 2024-09-10T10:01:35.382 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:01:34 smithi134 ceph-mon[35674]: Adjusting osd_memory_target on smithi134 to 176.0M 2024-09-10T10:01:35.382 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:01:34 smithi134 ceph-mon[35674]: Unable to set osd_memory_target on smithi120 to 184575283: error parsing value: Value '184575283' is below minimum 939524096 2024-09-10T10:01:35.382 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:01:34 smithi134 ceph-mon[35674]: Unable to set osd_memory_target on smithi134 to 184596787: error parsing value: Value '184596787' is below minimum 939524096 2024-09-10T10:01:35.382 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:01:34 smithi134 ceph-mon[35674]: pgmap v583: 129 pgs: 129 active+clean; 141 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 1.2 MiB/s wr, 511 op/s 2024-09-10T10:01:35.382 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:01:34 smithi134 ceph-mon[35674]: Adjusting osd_memory_target on smithi077 to 176.4M 2024-09-10T10:01:35.382 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:01:34 smithi134 ceph-mon[35674]: Unable to set osd_memory_target on smithi077 to 185021849: error parsing value: Value '185021849' is below minimum 939524096 2024-09-10T10:01:35.382 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:01:34 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/3890786211' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:01:35.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:01:34 smithi120 ceph-mon[34770]: Adjusting osd_memory_target on smithi120 to 176.0M 2024-09-10T10:01:35.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:01:34 smithi120 ceph-mon[34770]: Adjusting osd_memory_target on smithi134 to 176.0M 2024-09-10T10:01:35.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:01:34 smithi120 ceph-mon[34770]: Unable to set osd_memory_target on smithi120 to 184575283: error parsing value: Value '184575283' is below minimum 939524096 2024-09-10T10:01:35.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:01:34 smithi120 ceph-mon[34770]: Unable to set osd_memory_target on smithi134 to 184596787: error parsing value: Value '184596787' is below minimum 939524096 2024-09-10T10:01:35.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:01:34 smithi120 ceph-mon[34770]: pgmap v583: 129 pgs: 129 active+clean; 141 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 1.2 MiB/s wr, 511 op/s 2024-09-10T10:01:35.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:01:34 smithi120 ceph-mon[34770]: Adjusting osd_memory_target on smithi077 to 176.4M 2024-09-10T10:01:35.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:01:34 smithi120 ceph-mon[34770]: Unable to set osd_memory_target on smithi077 to 185021849: error parsing value: Value '185021849' is below minimum 939524096 2024-09-10T10:01:35.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:01:34 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/3890786211' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:01:35.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:01:34 smithi077 ceph-mon[30319]: Adjusting osd_memory_target on smithi120 to 176.0M 2024-09-10T10:01:35.440 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:01:34 smithi077 ceph-mon[30319]: Adjusting osd_memory_target on smithi134 to 176.0M 2024-09-10T10:01:35.440 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:01:34 smithi077 ceph-mon[30319]: Unable to set osd_memory_target on smithi120 to 184575283: error parsing value: Value '184575283' is below minimum 939524096 2024-09-10T10:01:35.440 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:01:34 smithi077 ceph-mon[30319]: Unable to set osd_memory_target on smithi134 to 184596787: error parsing value: Value '184596787' is below minimum 939524096 2024-09-10T10:01:35.440 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:01:34 smithi077 ceph-mon[30319]: pgmap v583: 129 pgs: 129 active+clean; 141 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 1.2 MiB/s wr, 511 op/s 2024-09-10T10:01:35.440 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:01:34 smithi077 ceph-mon[30319]: Adjusting osd_memory_target on smithi077 to 176.4M 2024-09-10T10:01:35.440 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:01:34 smithi077 ceph-mon[30319]: Unable to set osd_memory_target on smithi077 to 185021849: error parsing value: Value '185021849' is below minimum 939524096 2024-09-10T10:01:35.440 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:01:34 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/3890786211' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:01:35.950 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24563 quiesce db / --timeout 90 --expiration 140.1 --await-for 45.0' 2024-09-10T10:01:35.950 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24563 quiesce db / --timeout 90 --expiration 140.1 --await-for 45.0 2024-09-10T10:01:36.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:01:35 smithi077 ceph-mon[30319]: pgmap v584: 129 pgs: 129 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 1.0 MiB/s wr, 458 op/s 2024-09-10T10:01:36.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:01:35 smithi134 ceph-mon[35674]: pgmap v584: 129 pgs: 129 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 1.0 MiB/s wr, 458 op/s 2024-09-10T10:01:36.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:01:35 smithi120 ceph-mon[34770]: pgmap v584: 129 pgs: 129 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 1.0 MiB/s wr, 458 op/s 2024-09-10T10:01:36.457 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:01:37.136 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.132+0000 7fa0d9800640 1 -- 172.21.15.77:0/2722801298 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa0d4102360 msgr2=0x7fa0d410e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:37.137 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.132+0000 7fa0d9800640 1 --2- 172.21.15.77:0/2722801298 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa0d4102360 0x7fa0d410e890 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc009ca0 tx=0x7fa0cc02a8c0 comp rx=0 tx=0).stop 2024-09-10T10:01:37.137 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.132+0000 7fa0d9800640 1 -- 172.21.15.77:0/2722801298 shutdown_connections 2024-09-10T10:01:37.137 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.132+0000 7fa0d9800640 1 --2- 172.21.15.77:0/2722801298 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa0d4102360 0x7fa0d410e890 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:37.137 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.132+0000 7fa0d9800640 1 --2- 172.21.15.77:0/2722801298 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa0d41019c0 0x7fa0d4101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:37.137 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.132+0000 7fa0d9800640 1 --2- 172.21.15.77:0/2722801298 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0d41079c0 0x7fa0d4107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:37.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.132+0000 7fa0d9800640 1 -- 172.21.15.77:0/2722801298 >> 172.21.15.77:0/2722801298 conn(0x7fa0d40fd740 msgr2=0x7fa0d40ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:37.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.132+0000 7fa0d9800640 1 -- 172.21.15.77:0/2722801298 shutdown_connections 2024-09-10T10:01:37.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.132+0000 7fa0d9800640 1 -- 172.21.15.77:0/2722801298 wait complete. 2024-09-10T10:01:37.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.133+0000 7fa0d9800640 1 Processor -- start 2024-09-10T10:01:37.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.134+0000 7fa0d9800640 1 -- start start 2024-09-10T10:01:37.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.134+0000 7fa0d9800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0d41019c0 0x7fa0d419a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:37.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.134+0000 7fa0d9800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa0d4102360 0x7fa0d419a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:37.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.134+0000 7fa0d9800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa0d41079c0 0x7fa0d419ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:37.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.134+0000 7fa0d9800640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d4111e10 con 0x7fa0d4102360 2024-09-10T10:01:37.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.134+0000 7fa0d9800640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d4111c90 con 0x7fa0d41079c0 2024-09-10T10:01:37.139 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.134+0000 7fa0d9800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d4111f90 con 0x7fa0d41019c0 2024-09-10T10:01:37.139 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.134+0000 7fa0d3e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0d41019c0 0x7fa0d419a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:37.139 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.134+0000 7fa0d3e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0d41019c0 0x7fa0d419a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:43606/0 (socket says 172.21.15.77:43606) 2024-09-10T10:01:37.139 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.134+0000 7fa0d3e00640 1 -- 172.21.15.77:0/1009038507 learned_addr learned my addr 172.21.15.77:0/1009038507 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:01:37.139 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.134+0000 7fa0d8e00640 1 --2- 172.21.15.77:0/1009038507 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa0d41079c0 0x7fa0d419ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:37.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.135+0000 7fa0d3400640 1 --2- 172.21.15.77:0/1009038507 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa0d4102360 0x7fa0d419a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:37.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.135+0000 7fa0d8e00640 1 -- 172.21.15.77:0/1009038507 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0d41019c0 msgr2=0x7fa0d419a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:37.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.135+0000 7fa0d8e00640 1 --2- 172.21.15.77:0/1009038507 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0d41019c0 0x7fa0d419a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:37.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.135+0000 7fa0d8e00640 1 -- 172.21.15.77:0/1009038507 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa0d4102360 msgr2=0x7fa0d419a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:37.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.135+0000 7fa0d8e00640 1 --2- 172.21.15.77:0/1009038507 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa0d4102360 0x7fa0d419a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:37.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.135+0000 7fa0d8e00640 1 -- 172.21.15.77:0/1009038507 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0d419f390 con 0x7fa0d41079c0 2024-09-10T10:01:37.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.135+0000 7fa0d3e00640 1 --2- 172.21.15.77:0/1009038507 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0d41019c0 0x7fa0d419a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:01:37.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.136+0000 7fa0d8e00640 1 --2- 172.21.15.77:0/1009038507 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa0d41079c0 0x7fa0d419ecb0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc005e30 tx=0x7fa0cc003f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:37.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.138+0000 7fa0d0c00640 1 -- 172.21.15.77:0/1009038507 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0cc031650 con 0x7fa0d41079c0 2024-09-10T10:01:37.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.138+0000 7fa0d0c00640 1 -- 172.21.15.77:0/1009038507 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0cc0317f0 con 0x7fa0d41079c0 2024-09-10T10:01:37.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.138+0000 7fa0d0c00640 1 -- 172.21.15.77:0/1009038507 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0cc02f970 con 0x7fa0d41079c0 2024-09-10T10:01:37.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.141+0000 7fa0d9800640 1 -- 172.21.15.77:0/1009038507 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0d419f620 con 0x7fa0d41079c0 2024-09-10T10:01:37.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.142+0000 7fa0d9800640 1 -- 172.21.15.77:0/1009038507 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0d419fae0 con 0x7fa0d41079c0 2024-09-10T10:01:37.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.147+0000 7fa0dd89f740 1 Processor -- start 2024-09-10T10:01:37.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.151+0000 7fa0d0c00640 1 -- 172.21.15.77:0/1009038507 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa0cc02fb10 con 0x7fa0d41079c0 2024-09-10T10:01:37.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.153+0000 7fa0dd89f740 1 -- start start 2024-09-10T10:01:37.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.153+0000 7fa0dd89f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e814c86560 0x55e814c86940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:37.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.153+0000 7fa0dd89f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e814c86e80 0x55e814c91200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:37.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.153+0000 7fa0dd89f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e814c91740 0x55e814c93b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:37.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.153+0000 7fa0dd89f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e814bec530 con 0x55e814c91740 2024-09-10T10:01:37.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.153+0000 7fa0dd89f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e814b93860 con 0x55e814c86560 2024-09-10T10:01:37.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.153+0000 7fa0dd89f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e814be84f0 con 0x55e814c86e80 2024-09-10T10:01:37.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.154+0000 7fa0d3e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e814c86e80 0x55e814c91200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:37.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.154+0000 7fa0d3e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e814c86e80 0x55e814c91200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:43612/0 (socket says 172.21.15.77:43612) 2024-09-10T10:01:37.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.154+0000 7fa0d3e00640 1 -- 172.21.15.77:0/802706182 learned_addr learned my addr 172.21.15.77:0/802706182 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:01:37.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.154+0000 7fa0d3400640 1 --2- 172.21.15.77:0/802706182 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e814c86560 0x55e814c86940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:37.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.154+0000 7fa0d3e00640 1 -- 172.21.15.77:0/802706182 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e814c86560 msgr2=0x55e814c86940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:37.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.154+0000 7fa0d3e00640 1 --2- 172.21.15.77:0/802706182 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e814c86560 0x55e814c86940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:37.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.154+0000 7fa0d3e00640 1 -- 172.21.15.77:0/802706182 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e814c91740 msgr2=0x55e814c93b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:37.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.154+0000 7fa0d3e00640 1 --2- 172.21.15.77:0/802706182 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e814c91740 0x55e814c93b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:37.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.154+0000 7fa0d3e00640 1 -- 172.21.15.77:0/802706182 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e81495db90 con 0x55e814c86e80 2024-09-10T10:01:37.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.154+0000 7fa0d0c00640 1 --2- 172.21.15.77:0/1009038507 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fa0b80a2cd0 0x7fa0b80a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:37.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.155+0000 7fa0d0c00640 1 -- 172.21.15.77:0/1009038507 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fa0cc0f06c0 con 0x7fa0d41079c0 2024-09-10T10:01:37.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.155+0000 7fa0d3400640 1 --2- 172.21.15.77:0/1009038507 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fa0b80a2cd0 0x7fa0b80a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:37.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.155+0000 7fa0d3e00640 1 --2- 172.21.15.77:0/802706182 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e814c86e80 0x55e814c91200 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa0c40115f0 tx=0x7fa0c4011ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64b982ff8baa4beb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:37.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.155+0000 7fa0c9600640 1 -- 172.21.15.77:0/802706182 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c401e070 con 0x55e814c86e80 2024-09-10T10:01:37.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.156+0000 7fa0c9600640 1 -- 172.21.15.77:0/802706182 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0c400fdd0 con 0x55e814c86e80 2024-09-10T10:01:37.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.156+0000 7fa0c9600640 1 -- 172.21.15.77:0/802706182 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c400e3b0 con 0x55e814c86e80 2024-09-10T10:01:37.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.156+0000 7fa0dd89f740 1 -- 172.21.15.77:0/802706182 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e814c86e80 msgr2=0x55e814c91200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:37.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.156+0000 7fa0dd89f740 1 --2- 172.21.15.77:0/802706182 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e814c86e80 0x55e814c91200 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa0c40115f0 tx=0x7fa0c4011ab0 comp rx=0 tx=0).stop 2024-09-10T10:01:37.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.156+0000 7fa0d3400640 1 --2- 172.21.15.77:0/1009038507 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fa0b80a2cd0 0x7fa0b80a5190 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x55e814c83e70 tx=0x7fa0c0014040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:37.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.157+0000 7fa0dd89f740 1 -- 172.21.15.77:0/802706182 shutdown_connections 2024-09-10T10:01:37.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.157+0000 7fa0dd89f740 1 --2- 172.21.15.77:0/802706182 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e814c91740 0x55e814c93b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:37.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.157+0000 7fa0dd89f740 1 --2- 172.21.15.77:0/802706182 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e814c86e80 0x55e814c91200 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:37.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.157+0000 7fa0dd89f740 1 --2- 172.21.15.77:0/802706182 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e814c86560 0x55e814c86940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:37.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.157+0000 7fa0dd89f740 1 -- 172.21.15.77:0/802706182 >> 172.21.15.77:0/802706182 conn(0x55e814c82af0 msgr2=0x55e814c82ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:37.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.157+0000 7fa0dd89f740 1 -- 172.21.15.77:0/802706182 shutdown_connections 2024-09-10T10:01:37.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.157+0000 7fa0dd89f740 1 -- 172.21.15.77:0/802706182 wait complete. 2024-09-10T10:01:37.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.157+0000 7fa0dd89f740 1 -- start start 2024-09-10T10:01:37.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.158+0000 7fa0dd89f740 1 Processor -- start 2024-09-10T10:01:37.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.158+0000 7fa0dd89f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e814c86560 0x55e814c8e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:37.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.158+0000 7fa0dd89f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e814c86e80 0x55e814c8ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:37.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.158+0000 7fa0dd89f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e814c91740 0x55e814c8f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:37.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.159+0000 7fa0d3e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e814c86560 0x55e814c8e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:37.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.159+0000 7fa0d8e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e814c91740 0x55e814c8f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:37.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.159+0000 7fa0d8e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e814c91740 0x55e814c8f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:42864/0 (socket says 172.21.15.77:42864) 2024-09-10T10:01:37.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.159+0000 7fa0d8e00640 1 -- 172.21.15.77:0/1646785683 learned_addr learned my addr 172.21.15.77:0/1646785683 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:01:37.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.159+0000 7fa0dd89f740 1 -- 172.21.15.77:0/1646785683 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e814c9cb70 con 0x55e814c86560 2024-09-10T10:01:37.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.159+0000 7fa0dd89f740 1 -- 172.21.15.77:0/1646785683 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e814ba6e60 con 0x55e814c91740 2024-09-10T10:01:37.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.159+0000 7fa0dd89f740 1 -- 172.21.15.77:0/1646785683 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e814c9ccf0 con 0x55e814c86e80 2024-09-10T10:01:37.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.159+0000 7fa0d8e00640 1 -- 172.21.15.77:0/1646785683 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e814c86e80 msgr2=0x55e814c8ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:37.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.159+0000 7fa0d8e00640 1 --2- 172.21.15.77:0/1646785683 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e814c86e80 0x55e814c8ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:37.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.159+0000 7fa0d8e00640 1 -- 172.21.15.77:0/1646785683 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e814c86560 msgr2=0x55e814c8e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:37.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.159+0000 7fa0d8e00640 1 --2- 172.21.15.77:0/1646785683 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e814c86560 0x55e814c8e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:37.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.160+0000 7fa0d8e00640 1 -- 172.21.15.77:0/1646785683 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e814bb2990 con 0x55e814c91740 2024-09-10T10:01:37.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.159+0000 7fa0dd89f740 10 client.0 _ll_register_callbacks cb 0x55e814babcc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:01:37.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.160+0000 7fa0dd89f740 1 -- 172.21.15.77:0/1646785683 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0c400fdd0 con 0x55e814c91740 2024-09-10T10:01:37.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.160+0000 7fa0be000640 10 client.26762 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T10:01:37.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.160+0000 7fa0d8e00640 1 --2- 172.21.15.77:0/1646785683 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e814c91740 0x55e814c8f110 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc0b8f90 tx=0x7fa0cc0bc550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a217011aa04fe41 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:37.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.160+0000 7fa0be000640 1 -- 172.21.15.77:0/1646785683 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0cc0d5070 con 0x55e814c91740 2024-09-10T10:01:37.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.160+0000 7fa0be000640 1 -- 172.21.15.77:0/1646785683 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fa0cc009890 con 0x55e814c91740 2024-09-10T10:01:37.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.161+0000 7fa0be000640 1 -- 172.21.15.77:0/1646785683 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fa0cc0d5070 con 0x55e814c91740 2024-09-10T10:01:37.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.161+0000 7fa0dd89f740 10 client.26762 fetch_fsmap learned FSMap version 35 2024-09-10T10:01:37.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.161+0000 7fa0dd89f740 1 -- 172.21.15.77:0/1646785683 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({fsmap=35}) -- 0x55e814c9cca0 con 0x55e814c91740 2024-09-10T10:01:37.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.161+0000 7fa0be000640 1 -- 172.21.15.77:0/1646785683 <== mon.2 v2:172.21.15.120:3300/0 4 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7fa0cc0b91a0 con 0x55e814c91740 2024-09-10T10:01:37.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.161+0000 7fa0dd89f740 10 client.26762 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T10:01:37.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.161+0000 7fa0dd89f740 10 client.26762 resolve_mds: validated gid 24563 aka daemon mds.b 2024-09-10T10:01:37.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.161+0000 7fa0dd89f740 20 client.26762 populate_metadata read hostname 'smithi077' 2024-09-10T10:01:37.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.161+0000 7fa0dd89f740 1 --2- 172.21.15.77:0/1646785683 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55e814ca9af0 0x55e814cabf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:37.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.161+0000 7fa0dd89f740 4 client.26762 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:01:37.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.161+0000 7fa0dd89f740 1 -- 172.21.15.77:0/1646785683 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e814b8f310 con 0x55e814ca9af0 2024-09-10T10:01:37.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.162+0000 7fa0d3e00640 1 --2- 172.21.15.77:0/1646785683 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55e814ca9af0 0x55e814cabf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:37.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.162+0000 7fa0d3e00640 1 --2- 172.21.15.77:0/1646785683 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55e814ca9af0 0x55e814cabf60 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37cf10301723fa6b server_cookie=c239c607350c2abd in_seq=0 out_seq=0 2024-09-10T10:01:37.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.162+0000 7fa0be000640 10 client.26762 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T10:01:37.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.163+0000 7fa0be000640 1 -- 172.21.15.77:0/1646785683 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e814b8f310 con 0x55e814ca9af0 2024-09-10T10:01:37.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.163+0000 7fa0be000640 10 client.26762 handle_command_reply: tid=0 2024-09-10T10:01:37.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.163+0000 7fa0dd89f740 1 client.26762 shutdown 2024-09-10T10:01:37.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.163+0000 7fa0dd89f740 1 -- 172.21.15.77:0/1646785683 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e814c91740 msgr2=0x55e814c8f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:37.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.163+0000 7fa0dd89f740 1 --2- 172.21.15.77:0/1646785683 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e814c91740 0x55e814c8f110 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc0b8f90 tx=0x7fa0cc0bc550 comp rx=0 tx=0).stop 2024-09-10T10:01:37.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.163+0000 7fa0dd89f740 1 -- 172.21.15.77:0/1646785683 shutdown_connections 2024-09-10T10:01:37.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.163+0000 7fa0dd89f740 1 --2- 172.21.15.77:0/1646785683 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55e814ca9af0 0x55e814cabf60 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:37.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.163+0000 7fa0dd89f740 1 --2- 172.21.15.77:0/1646785683 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e814c91740 0x55e814c8f110 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:37.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.163+0000 7fa0dd89f740 1 --2- 172.21.15.77:0/1646785683 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e814c86e80 0x55e814c8ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:37.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.163+0000 7fa0dd89f740 1 --2- 172.21.15.77:0/1646785683 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e814c86560 0x55e814c8e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:37.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.163+0000 7fa0dd89f740 1 -- 172.21.15.77:0/1646785683 >> 172.21.15.77:0/1646785683 conn(0x55e814c82af0 msgr2=0x55e814c81c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:37.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.163+0000 7fa0be000640 0 client.26762 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T10:01:37.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.164+0000 7fa0dd89f740 1 -- 172.21.15.77:0/1646785683 shutdown_connections 2024-09-10T10:01:37.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.164+0000 7fa0dd89f740 1 -- 172.21.15.77:0/1646785683 wait complete. 2024-09-10T10:01:37.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.164+0000 7fa0dd89f740 20 client.26762 trim_cache size 0 max 16384 2024-09-10T10:01:37.196 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.194+0000 7fa0dd89f740 1 Processor -- start 2024-09-10T10:01:37.196 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.194+0000 7fa0dd89f740 1 -- start start 2024-09-10T10:01:37.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.195+0000 7fa0dd89f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e814c86560 0x55e814c8cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:37.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.195+0000 7fa0dd89f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e814c86e80 0x55e814cb1840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:37.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.195+0000 7fa0dd89f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e814c91740 0x55e814cb1d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:37.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.195+0000 7fa0dd89f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d41aca40 con 0x55e814c86e80 2024-09-10T10:01:37.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.195+0000 7fa0dd89f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d41ae580 con 0x55e814c91740 2024-09-10T10:01:37.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.195+0000 7fa0dd89f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d41ad110 con 0x55e814c86560 2024-09-10T10:01:37.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.195+0000 7fa0d3400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e814c86e80 0x55e814cb1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:37.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.195+0000 7fa0d3400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e814c86e80 0x55e814cb1840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:51024/0 (socket says 172.21.15.77:51024) 2024-09-10T10:01:37.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.195+0000 7fa0d3400640 1 -- 172.21.15.77:0/2219058485 learned_addr learned my addr 172.21.15.77:0/2219058485 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:01:37.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.195+0000 7fa0d8e00640 1 --2- 172.21.15.77:0/2219058485 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e814c91740 0x55e814cb1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:37.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.195+0000 7fa0d3e00640 1 --2- 172.21.15.77:0/2219058485 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e814c86560 0x55e814c8cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:37.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.196+0000 7fa0d3e00640 1 -- 172.21.15.77:0/2219058485 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e814c91740 msgr2=0x55e814cb1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:37.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.196+0000 7fa0d3e00640 1 --2- 172.21.15.77:0/2219058485 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e814c91740 0x55e814cb1d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:37.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.196+0000 7fa0d3e00640 1 -- 172.21.15.77:0/2219058485 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e814c86e80 msgr2=0x55e814cb1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:37.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.196+0000 7fa0d3e00640 1 --2- 172.21.15.77:0/2219058485 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e814c86e80 0x55e814cb1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:37.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.196+0000 7fa0d3e00640 1 -- 172.21.15.77:0/2219058485 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e814cac2d0 con 0x55e814c86560 2024-09-10T10:01:37.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.196+0000 7fa0d3e00640 1 --2- 172.21.15.77:0/2219058485 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e814c86560 0x55e814c8cf60 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa0c401c040 tx=0x7fa0c40195c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76eca8e7fad94d4b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:37.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.196+0000 7fa0c8c00640 1 -- 172.21.15.77:0/2219058485 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c4034070 con 0x55e814c86560 2024-09-10T10:01:37.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.196+0000 7fa0c8c00640 1 -- 172.21.15.77:0/2219058485 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0c401f040 con 0x55e814c86560 2024-09-10T10:01:37.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.196+0000 7fa0c8c00640 1 -- 172.21.15.77:0/2219058485 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c401e040 con 0x55e814c86560 2024-09-10T10:01:37.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.197+0000 7fa0dd89f740 1 -- 172.21.15.77:0/2219058485 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e814c86560 msgr2=0x55e814c8cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:37.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.197+0000 7fa0dd89f740 1 --2- 172.21.15.77:0/2219058485 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e814c86560 0x55e814c8cf60 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa0c401c040 tx=0x7fa0c40195c0 comp rx=0 tx=0).stop 2024-09-10T10:01:37.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.197+0000 7fa0dd89f740 1 -- 172.21.15.77:0/2219058485 shutdown_connections 2024-09-10T10:01:37.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.197+0000 7fa0dd89f740 1 --2- 172.21.15.77:0/2219058485 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e814c91740 0x55e814cb1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:37.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.197+0000 7fa0dd89f740 1 --2- 172.21.15.77:0/2219058485 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e814c86e80 0x55e814cb1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:37.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.197+0000 7fa0dd89f740 1 --2- 172.21.15.77:0/2219058485 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e814c86560 0x55e814c8cf60 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:37.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.197+0000 7fa0dd89f740 1 -- 172.21.15.77:0/2219058485 >> 172.21.15.77:0/2219058485 conn(0x55e814c82af0 msgr2=0x55e814c887d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:37.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.197+0000 7fa0dd89f740 1 -- 172.21.15.77:0/2219058485 shutdown_connections 2024-09-10T10:01:37.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.198+0000 7fa0dd89f740 1 -- 172.21.15.77:0/2219058485 wait complete. 2024-09-10T10:01:37.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.198+0000 7fa0dd89f740 1 -- start start 2024-09-10T10:01:37.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.198+0000 7fa0dd89f740 1 Processor -- start 2024-09-10T10:01:37.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.199+0000 7fa0dd89f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e814c86560 0x55e814c9dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:37.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.199+0000 7fa0dd89f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e814c86e80 0x55e814c9e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:37.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.199+0000 7fa0dd89f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e814c91740 0x55e814d13810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:37.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.199+0000 7fa0dd89f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d41adeb0 con 0x55e814c91740 2024-09-10T10:01:37.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.199+0000 7fa0dd89f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d41ad7e0 con 0x55e814c86560 2024-09-10T10:01:37.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.199+0000 7fa0dd89f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d41aaa00 con 0x55e814c86e80 2024-09-10T10:01:37.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.199+0000 7fa0dd89f740 10 client.0 _ll_register_callbacks cb 0x7fa0cc005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:01:37.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.199+0000 7fa0d3400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e814c86e80 0x55e814c9e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:37.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.199+0000 7fa0d3400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e814c86e80 0x55e814c9e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:43622/0 (socket says 172.21.15.77:43622) 2024-09-10T10:01:37.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.199+0000 7fa0d3400640 1 -- 172.21.15.77:0/2183640109 learned_addr learned my addr 172.21.15.77:0/2183640109 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:01:37.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.199+0000 7fa0d3e00640 1 --2- 172.21.15.77:0/2183640109 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e814c86560 0x55e814c9dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:37.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.199+0000 7fa0d8e00640 1 --2- 172.21.15.77:0/2183640109 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e814c91740 0x55e814d13810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:37.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.200+0000 7fa0d3400640 1 -- 172.21.15.77:0/2183640109 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e814c86560 msgr2=0x55e814c9dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:37.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.200+0000 7fa0d3400640 1 --2- 172.21.15.77:0/2183640109 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e814c86560 0x55e814c9dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:37.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.200+0000 7fa0d3400640 1 -- 172.21.15.77:0/2183640109 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e814c91740 msgr2=0x55e814d13810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:37.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.200+0000 7fa0d3400640 1 --2- 172.21.15.77:0/2183640109 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e814c91740 0x55e814d13810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:37.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.200+0000 7fa0d3400640 1 -- 172.21.15.77:0/2183640109 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e814c9ede0 con 0x55e814c86e80 2024-09-10T10:01:37.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.200+0000 7fa0d8e00640 1 --2- 172.21.15.77:0/2183640109 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e814c91740 0x55e814d13810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:01:37.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.201+0000 7fa0d3400640 1 --2- 172.21.15.77:0/2183640109 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e814c86e80 0x55e814c9e530 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa0c00095f0 tx=0x7fa0c0005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a38ef7736ffbb98 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:37.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.201+0000 7fa0be000640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-10T10:01:37.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.201+0000 7fa0be000640 1 -- 172.21.15.77:0/2183640109 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c0010b10 con 0x55e814c86e80 2024-09-10T10:01:37.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.201+0000 7fa0be000640 1 -- 172.21.15.77:0/2183640109 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fa0c0010cb0 con 0x55e814c86e80 2024-09-10T10:01:37.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.201+0000 7fa0dd89f740 1 -- 172.21.15.77:0/2183640109 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0c401f040 con 0x55e814c86e80 2024-09-10T10:01:37.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.202+0000 7fa0be000640 1 -- 172.21.15.77:0/2183640109 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fa0c0004020 con 0x55e814c86e80 2024-09-10T10:01:37.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.202+0000 7fa0dd89f740 10 client.26713 fetch_fsmap learned FSMap version 35 2024-09-10T10:01:37.206 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.202+0000 7fa0dd89f740 1 -- 172.21.15.77:0/2183640109 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55e814d14e70 con 0x55e814c86e80 2024-09-10T10:01:37.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.202+0000 7fa0be000640 1 -- 172.21.15.77:0/2183640109 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0c401f040 con 0x55e814c86e80 2024-09-10T10:01:37.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.202+0000 7fa0be000640 1 -- 172.21.15.77:0/2183640109 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7fa0c001b070 con 0x55e814c86e80 2024-09-10T10:01:37.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.202+0000 7fa0dd89f740 10 client.26713 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T10:01:37.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.202+0000 7fa0dd89f740 10 client.26713 resolve_mds: validated gid 24563 aka daemon mds.b 2024-09-10T10:01:37.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.202+0000 7fa0dd89f740 20 client.26713 populate_metadata read hostname 'smithi077' 2024-09-10T10:01:37.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.202+0000 7fa0dd89f740 1 --2- 172.21.15.77:0/2183640109 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55e814d15660 0x55e814d17a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:37.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.202+0000 7fa0dd89f740 4 client.26713 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.1, "await_for": 45.0}] 2024-09-10T10:01:37.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.202+0000 7fa0dd89f740 1 -- 172.21.15.77:0/2183640109 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.1, "await_for": 45.0}) -- 0x55e814bf9c60 con 0x55e814d15660 2024-09-10T10:01:37.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.203+0000 7fa0d3e00640 1 --2- 172.21.15.77:0/2183640109 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55e814d15660 0x55e814d17a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:37.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.203+0000 7fa0be000640 10 client.26713 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T10:01:37.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.203+0000 7fa0d3e00640 1 --2- 172.21.15.77:0/2183640109 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55e814d15660 0x55e814d17a80 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c240255179a6d5cd server_cookie=595ef9936f2c0168 in_seq=0 out_seq=0 2024-09-10T10:01:37.805 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.802+0000 7fa0be000640 1 -- 172.21.15.77:0/2183640109 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55e814bf9c60 con 0x55e814d15660 2024-09-10T10:01:37.805 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.802+0000 7fa0be000640 10 client.26713 handle_command_reply: tid=0 2024-09-10T10:01:37.805 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.802+0000 7fa0dd89f740 1 client.26713 shutdown 2024-09-10T10:01:37.805 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.802+0000 7fa0dd89f740 1 -- 172.21.15.77:0/2183640109 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e814c86e80 msgr2=0x55e814c9e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:37.805 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.802+0000 7fa0dd89f740 1 --2- 172.21.15.77:0/2183640109 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e814c86e80 0x55e814c9e530 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa0c00095f0 tx=0x7fa0c0005f90 comp rx=0 tx=0).stop 2024-09-10T10:01:37.805 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.802+0000 7fa0dd89f740 1 -- 172.21.15.77:0/2183640109 shutdown_connections 2024-09-10T10:01:37.805 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.802+0000 7fa0dd89f740 1 --2- 172.21.15.77:0/2183640109 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55e814d15660 0x55e814d17a80 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:37.805 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.802+0000 7fa0dd89f740 1 --2- 172.21.15.77:0/2183640109 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e814c91740 0x55e814d13810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:37.805 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.802+0000 7fa0dd89f740 1 --2- 172.21.15.77:0/2183640109 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e814c86e80 0x55e814c9e530 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:37.805 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.802+0000 7fa0be000640 0 client.26713 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T10:01:37.805 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.802+0000 7fa0dd89f740 1 --2- 172.21.15.77:0/2183640109 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e814c86560 0x55e814c9dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:37.806 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.802+0000 7fa0dd89f740 1 -- 172.21.15.77:0/2183640109 >> 172.21.15.77:0/2183640109 conn(0x55e814c82af0 msgr2=0x55e814c81450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:37.806 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.803+0000 7fa0dd89f740 1 -- 172.21.15.77:0/2183640109 shutdown_connections 2024-09-10T10:01:37.806 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.803+0000 7fa0dd89f740 1 -- 172.21.15.77:0/2183640109 wait complete. 2024-09-10T10:01:37.806 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.803+0000 7fa0dd89f740 20 client.26713 trim_cache size 0 max 16384 2024-09-10T10:01:37.806 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-10T10:01:37.806 INFO:teuthology.orchestra.run.smithi077.stdout: "epoch": 35, 2024-09-10T10:01:37.806 INFO:teuthology.orchestra.run.smithi077.stdout: "leader": 24563, 2024-09-10T10:01:37.806 INFO:teuthology.orchestra.run.smithi077.stdout: "set_version": 18, 2024-09-10T10:01:37.806 INFO:teuthology.orchestra.run.smithi077.stdout: "sets": { 2024-09-10T10:01:37.807 INFO:teuthology.orchestra.run.smithi077.stdout: "f49f355b": { 2024-09-10T10:01:37.807 INFO:teuthology.orchestra.run.smithi077.stdout: "version": 18, 2024-09-10T10:01:37.807 INFO:teuthology.orchestra.run.smithi077.stdout: "age_ref": 0.0, 2024-09-10T10:01:37.807 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-10T10:01:37.807 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "QUIESCED", 2024-09-10T10:01:37.807 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-10T10:01:37.807 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:01:37.807 INFO:teuthology.orchestra.run.smithi077.stdout: "timeout": 90.0, 2024-09-10T10:01:37.807 INFO:teuthology.orchestra.run.smithi077.stdout: "expiration": 140.1, 2024-09-10T10:01:37.807 INFO:teuthology.orchestra.run.smithi077.stdout: "members": { 2024-09-10T10:01:37.807 INFO:teuthology.orchestra.run.smithi077.stdout: "file:/": { 2024-09-10T10:01:37.807 INFO:teuthology.orchestra.run.smithi077.stdout: "excluded": false, 2024-09-10T10:01:37.807 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-10T10:01:37.808 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "QUIESCED", 2024-09-10T10:01:37.808 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-10T10:01:37.808 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:01:37.808 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:01:37.808 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:01:37.808 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:01:37.808 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:01:37.808 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-10T10:01:37.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.804+0000 7fa0d9800640 1 -- 172.21.15.77:0/1009038507 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fa0b80a2cd0 msgr2=0x7fa0b80a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:37.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.804+0000 7fa0d9800640 1 --2- 172.21.15.77:0/1009038507 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fa0b80a2cd0 0x7fa0b80a5190 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x55e814c83e70 tx=0x7fa0c0014040 comp rx=0 tx=0).stop 2024-09-10T10:01:37.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.804+0000 7fa0d9800640 1 -- 172.21.15.77:0/1009038507 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa0d41079c0 msgr2=0x7fa0d419ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:37.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.804+0000 7fa0d9800640 1 --2- 172.21.15.77:0/1009038507 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa0d41079c0 0x7fa0d419ecb0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc005e30 tx=0x7fa0cc003f10 comp rx=0 tx=0).stop 2024-09-10T10:01:37.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.804+0000 7fa0d9800640 1 -- 172.21.15.77:0/1009038507 shutdown_connections 2024-09-10T10:01:37.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.804+0000 7fa0d9800640 1 --2- 172.21.15.77:0/1009038507 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fa0b80a2cd0 0x7fa0b80a5190 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:37.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.804+0000 7fa0d9800640 1 --2- 172.21.15.77:0/1009038507 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa0d41079c0 0x7fa0d419ecb0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:37.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.804+0000 7fa0d9800640 1 --2- 172.21.15.77:0/1009038507 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa0d4102360 0x7fa0d419a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:37.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.804+0000 7fa0d9800640 1 --2- 172.21.15.77:0/1009038507 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0d41019c0 0x7fa0d419a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:37.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.804+0000 7fa0d9800640 1 -- 172.21.15.77:0/1009038507 >> 172.21.15.77:0/1009038507 conn(0x7fa0d40fd740 msgr2=0x7fa0d410cc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:37.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.806+0000 7fa0d9800640 1 -- 172.21.15.77:0/1009038507 shutdown_connections 2024-09-10T10:01:37.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:37.806+0000 7fa0d9800640 1 -- 172.21.15.77:0/1009038507 wait complete. 2024-09-10T10:01:38.751 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'f49f355b', quiesce took 5.6 seconds. Will release after: 14.500000000000002 2024-09-10T10:01:39.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:01:38 smithi120 ceph-mon[34770]: pgmap v585: 129 pgs: 129 active+clean; 136 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 1.0 MiB/s wr, 453 op/s 2024-09-10T10:01:39.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:01:38 smithi077 ceph-mon[30319]: pgmap v585: 129 pgs: 129 active+clean; 136 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 1.0 MiB/s wr, 453 op/s 2024-09-10T10:01:39.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:01:38 smithi134 ceph-mon[35674]: pgmap v585: 129 pgs: 129 active+clean; 136 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 1.0 MiB/s wr, 453 op/s 2024-09-10T10:01:41.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:01:40 smithi120 ceph-mon[34770]: pgmap v586: 129 pgs: 129 active+clean; 136 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.3 MiB/s wr, 532 op/s 2024-09-10T10:01:41.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:01:40 smithi077 ceph-mon[30319]: pgmap v586: 129 pgs: 129 active+clean; 136 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.3 MiB/s wr, 532 op/s 2024-09-10T10:01:41.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:01:40 smithi134 ceph-mon[35674]: pgmap v586: 129 pgs: 129 active+clean; 136 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.3 MiB/s wr, 532 op/s 2024-09-10T10:01:43.066 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T10:01:43.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:01:42 smithi120 ceph-mon[34770]: pgmap v587: 129 pgs: 129 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 946 KiB/s wr, 382 op/s 2024-09-10T10:01:43.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:01:42 smithi077 ceph-mon[30319]: pgmap v587: 129 pgs: 129 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 946 KiB/s wr, 382 op/s 2024-09-10T10:01:43.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:01:42 smithi134 ceph-mon[35674]: pgmap v587: 129 pgs: 129 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 946 KiB/s wr, 382 op/s 2024-09-10T10:01:43.449 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:01:44.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.030+0000 7fabdb000640 1 -- 172.21.15.77:0/1909367047 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fabd4109140 msgr2=0x7fabd4110cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:44.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.030+0000 7fabdb000640 1 --2- 172.21.15.77:0/1909367047 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fabd4109140 0x7fabd4110cc0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fabcc009900 tx=0x7fabcc02a410 comp rx=0 tx=0).stop 2024-09-10T10:01:44.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.030+0000 7fabdb000640 1 -- 172.21.15.77:0/1909367047 shutdown_connections 2024-09-10T10:01:44.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.030+0000 7fabdb000640 1 --2- 172.21.15.77:0/1909367047 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fabd4109140 0x7fabd4110cc0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:44.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.030+0000 7fabdb000640 1 --2- 172.21.15.77:0/1909367047 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fabd41049c0 0x7fabd4108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:44.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.030+0000 7fabdb000640 1 --2- 172.21.15.77:0/1909367047 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fabd4104010 0x7fabd41043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:44.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.030+0000 7fabdb000640 1 -- 172.21.15.77:0/1909367047 >> 172.21.15.77:0/1909367047 conn(0x7fabd40ffa50 msgr2=0x7fabd4101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:44.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.030+0000 7fabdb000640 1 -- 172.21.15.77:0/1909367047 shutdown_connections 2024-09-10T10:01:44.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.030+0000 7fabdb000640 1 -- 172.21.15.77:0/1909367047 wait complete. 2024-09-10T10:01:44.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.031+0000 7fabdb000640 1 Processor -- start 2024-09-10T10:01:44.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.031+0000 7fabdb000640 1 -- start start 2024-09-10T10:01:44.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.032+0000 7fabdb000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fabd4104010 0x7fabd419a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:44.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.032+0000 7fabdb000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fabd41049c0 0x7fabd419ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:44.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.032+0000 7fabdb000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fabd4109140 0x7fabd419ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:44.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.032+0000 7fabdb000640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fabd4114150 con 0x7fabd41049c0 2024-09-10T10:01:44.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.032+0000 7fabdb000640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fabd4113fd0 con 0x7fabd4109140 2024-09-10T10:01:44.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.032+0000 7fabdb000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fabd41142d0 con 0x7fabd4104010 2024-09-10T10:01:44.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.032+0000 7fabcb400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fabd41049c0 0x7fabd419ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:44.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.032+0000 7fabd3e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fabd4104010 0x7fabd419a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:44.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.032+0000 7fabd9000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fabd4109140 0x7fabd419ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:44.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.032+0000 7fabcb400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fabd41049c0 0x7fabd419ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52168/0 (socket says 172.21.15.77:52168) 2024-09-10T10:01:44.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.032+0000 7fabd3e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fabd4104010 0x7fabd419a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:46348/0 (socket says 172.21.15.77:46348) 2024-09-10T10:01:44.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.032+0000 7fabcb400640 1 -- 172.21.15.77:0/2536094289 learned_addr learned my addr 172.21.15.77:0/2536094289 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:01:44.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.033+0000 7fabd3e00640 1 -- 172.21.15.77:0/2536094289 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fabd4109140 msgr2=0x7fabd419ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:44.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.033+0000 7fabd3e00640 1 --2- 172.21.15.77:0/2536094289 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fabd4109140 0x7fabd419ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:44.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.033+0000 7fabd3e00640 1 -- 172.21.15.77:0/2536094289 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fabd41049c0 msgr2=0x7fabd419ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:44.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.033+0000 7fabd3e00640 1 --2- 172.21.15.77:0/2536094289 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fabd41049c0 0x7fabd419ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:44.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.033+0000 7fabd3e00640 1 -- 172.21.15.77:0/2536094289 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabd419f5e0 con 0x7fabd4104010 2024-09-10T10:01:44.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.033+0000 7fabcb400640 1 --2- 172.21.15.77:0/2536094289 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fabd41049c0 0x7fabd419ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:01:44.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.033+0000 7fabd3e00640 1 --2- 172.21.15.77:0/2536094289 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fabd4104010 0x7fabd419a630 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fabc400eb00 tx=0x7fabc400c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:44.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.033+0000 7fabd1600640 1 -- 172.21.15.77:0/2536094289 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabc4010430 con 0x7fabd4104010 2024-09-10T10:01:44.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.033+0000 7fabd1600640 1 -- 172.21.15.77:0/2536094289 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabc4010ab0 con 0x7fabd4104010 2024-09-10T10:01:44.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.033+0000 7fabdb000640 1 -- 172.21.15.77:0/2536094289 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabd419f8d0 con 0x7fabd4104010 2024-09-10T10:01:44.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.033+0000 7fabd1600640 1 -- 172.21.15.77:0/2536094289 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabc4017bb0 con 0x7fabd4104010 2024-09-10T10:01:44.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.033+0000 7fabdb000640 1 -- 172.21.15.77:0/2536094289 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabd41a71b0 con 0x7fabd4104010 2024-09-10T10:01:44.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.036+0000 7fabdd9de740 1 Processor -- start 2024-09-10T10:01:44.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.037+0000 7fabd1600640 1 -- 172.21.15.77:0/2536094289 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fabc4010c60 con 0x7fabd4104010 2024-09-10T10:01:44.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.037+0000 7fabdd9de740 1 -- start start 2024-09-10T10:01:44.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.037+0000 7fabdd9de740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560017ca3b90 0x560017ca3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:44.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.037+0000 7fabdd9de740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560017ca44b0 0x560017cae830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:44.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.037+0000 7fabdd9de740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560017caed70 0x560017cb1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:44.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.037+0000 7fabdd9de740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560017c0a3e0 con 0x560017caed70 2024-09-10T10:01:44.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.037+0000 7fabdd9de740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x560017bb1d80 con 0x560017ca3b90 2024-09-10T10:01:44.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.037+0000 7fabdd9de740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560017c063a0 con 0x560017ca44b0 2024-09-10T10:01:44.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.037+0000 7fabd9000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560017ca3b90 0x560017ca3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:44.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.037+0000 7fabd9000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560017ca3b90 0x560017ca3f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:40428/0 (socket says 172.21.15.77:40428) 2024-09-10T10:01:44.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.037+0000 7fabd9000640 1 -- 172.21.15.77:0/1615539567 learned_addr learned my addr 172.21.15.77:0/1615539567 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:01:44.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.037+0000 7fabd3e00640 1 --2- 172.21.15.77:0/1615539567 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560017ca44b0 0x560017cae830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:44.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.037+0000 7fabcb400640 1 --2- 172.21.15.77:0/1615539567 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560017caed70 0x560017cb1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:44.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.038+0000 7fabd9000640 1 -- 172.21.15.77:0/1615539567 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560017ca44b0 msgr2=0x560017cae830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:44.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.038+0000 7fabd9000640 1 --2- 172.21.15.77:0/1615539567 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560017ca44b0 0x560017cae830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:44.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.038+0000 7fabd9000640 1 -- 172.21.15.77:0/1615539567 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560017caed70 msgr2=0x560017cb1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:44.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.038+0000 7fabd9000640 1 --2- 172.21.15.77:0/1615539567 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560017caed70 0x560017cb1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:44.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.038+0000 7fabd9000640 1 -- 172.21.15.77:0/1615539567 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560017bd0ae0 con 0x560017ca3b90 2024-09-10T10:01:44.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.038+0000 7fabcb400640 1 --2- 172.21.15.77:0/1615539567 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560017caed70 0x560017cb1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:01:44.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.038+0000 7fabd9000640 1 --2- 172.21.15.77:0/1615539567 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560017ca3b90 0x560017ca3f70 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fabcc033000 tx=0x7fabcc005e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d6237efc1045b2b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:44.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.038+0000 7fabc9600640 1 -- 172.21.15.77:0/1615539567 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabcc040070 con 0x560017ca3b90 2024-09-10T10:01:44.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.038+0000 7fabd1600640 1 --2- 172.21.15.77:0/2536094289 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fabb80a2b80 0x7fabb80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:44.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.038+0000 7fabc9600640 1 -- 172.21.15.77:0/1615539567 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabcc0318b0 con 0x560017ca3b90 2024-09-10T10:01:44.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.038+0000 7fabc9600640 1 -- 172.21.15.77:0/1615539567 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabcc035040 con 0x560017ca3b90 2024-09-10T10:01:44.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.039+0000 7fabd1600640 1 -- 172.21.15.77:0/2536094289 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fabc40cfe40 con 0x7fabd4104010 2024-09-10T10:01:44.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.039+0000 7fabd3e00640 1 --2- 172.21.15.77:0/2536094289 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fabb80a2b80 0x7fabb80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:44.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.039+0000 7fabdd9de740 1 -- 172.21.15.77:0/1615539567 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560017ca3b90 msgr2=0x560017ca3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:44.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.039+0000 7fabdd9de740 1 --2- 172.21.15.77:0/1615539567 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560017ca3b90 0x560017ca3f70 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fabcc033000 tx=0x7fabcc005e00 comp rx=0 tx=0).stop 2024-09-10T10:01:44.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.039+0000 7fabdd9de740 1 -- 172.21.15.77:0/1615539567 shutdown_connections 2024-09-10T10:01:44.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.039+0000 7fabdd9de740 1 --2- 172.21.15.77:0/1615539567 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560017caed70 0x560017cb1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:44.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.039+0000 7fabdd9de740 1 --2- 172.21.15.77:0/1615539567 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560017ca44b0 0x560017cae830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:44.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.039+0000 7fabdd9de740 1 --2- 172.21.15.77:0/1615539567 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560017ca3b90 0x560017ca3f70 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:44.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.039+0000 7fabdd9de740 1 -- 172.21.15.77:0/1615539567 >> 172.21.15.77:0/1615539567 conn(0x560017ca0120 msgr2=0x560017ca0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:44.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.039+0000 7fabdd9de740 1 -- 172.21.15.77:0/1615539567 shutdown_connections 2024-09-10T10:01:44.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.039+0000 7fabd3e00640 1 --2- 172.21.15.77:0/2536094289 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fabb80a2b80 0x7fabb80a5040 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fabc40a4990 tx=0x7fabc400cf10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:44.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.039+0000 7fabdd9de740 1 -- 172.21.15.77:0/1615539567 wait complete. 2024-09-10T10:01:44.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.040+0000 7fabdd9de740 1 -- start start 2024-09-10T10:01:44.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.040+0000 7fabdd9de740 1 Processor -- start 2024-09-10T10:01:44.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.040+0000 7fabdd9de740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560017ca3b90 0x560017cabc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:44.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.040+0000 7fabdd9de740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560017ca44b0 0x560017cac1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:44.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.040+0000 7fabdd9de740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560017caed70 0x560017cac6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:44.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.041+0000 7fabdd9de740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560017cb19e0 con 0x560017caed70 2024-09-10T10:01:44.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.041+0000 7fabdd9de740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x560017bc50f0 con 0x560017ca3b90 2024-09-10T10:01:44.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.041+0000 7fabdd9de740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560017cb1b60 con 0x560017ca44b0 2024-09-10T10:01:44.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.041+0000 7fabdd9de740 10 client.0 _ll_register_callbacks cb 0x560017bb69e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:01:44.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.041+0000 7fabd9000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560017caed70 0x560017cac6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:44.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.041+0000 7fabd9000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560017caed70 0x560017cac6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52184/0 (socket says 172.21.15.77:52184) 2024-09-10T10:01:44.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.041+0000 7fabd9000640 1 -- 172.21.15.77:0/1812559188 learned_addr learned my addr 172.21.15.77:0/1812559188 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:01:44.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.041+0000 7fabd9000640 1 --2- 172.21.15.77:0/1812559188 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560017ca3b90 0x560017cabc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:44.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.041+0000 7fabcb400640 1 --2- 172.21.15.77:0/1812559188 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560017ca44b0 0x560017cac1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:44.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.041+0000 7fabd9000640 1 -- 172.21.15.77:0/1812559188 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560017ca44b0 msgr2=0x560017cac1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:44.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.041+0000 7fabd9000640 1 --2- 172.21.15.77:0/1812559188 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560017ca44b0 0x560017cac1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:44.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.041+0000 7fabd9000640 1 -- 172.21.15.77:0/1812559188 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560017ca3b90 msgr2=0x560017cabc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:44.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.041+0000 7fabd9000640 1 --2- 172.21.15.77:0/1812559188 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560017ca3b90 0x560017cabc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:44.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.041+0000 7fabd9000640 1 -- 172.21.15.77:0/1812559188 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56001797b3c0 con 0x560017caed70 2024-09-10T10:01:44.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.042+0000 7fabd9000640 1 --2- 172.21.15.77:0/1812559188 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560017caed70 0x560017cac6f0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fabcc02a930 tx=0x7fabcc005540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5f7cc1a471504f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:44.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.042+0000 7fabc2000640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-10T10:01:44.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.042+0000 7fabc2000640 1 -- 172.21.15.77:0/1812559188 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabcc035070 con 0x560017caed70 2024-09-10T10:01:44.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.042+0000 7fabc2000640 1 -- 172.21.15.77:0/1812559188 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fabcc0316b0 con 0x560017caed70 2024-09-10T10:01:44.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.042+0000 7fabdd9de740 1 -- 172.21.15.77:0/1812559188 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fabcc0318b0 con 0x560017caed70 2024-09-10T10:01:44.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.042+0000 7fabc2000640 1 -- 172.21.15.77:0/1812559188 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fabcc040040 con 0x560017caed70 2024-09-10T10:01:44.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.042+0000 7fabdd9de740 10 client.16650 fetch_fsmap learned FSMap version 35 2024-09-10T10:01:44.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.042+0000 7fabdd9de740 1 -- 172.21.15.77:0/1812559188 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x560017cbd6a0 con 0x560017caed70 2024-09-10T10:01:44.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.043+0000 7fabc2000640 1 -- 172.21.15.77:0/1812559188 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabcc0318b0 con 0x560017caed70 2024-09-10T10:01:44.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.043+0000 7fabc2000640 1 -- 172.21.15.77:0/1812559188 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7fabcc051020 con 0x560017caed70 2024-09-10T10:01:44.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.043+0000 7fabdd9de740 10 client.16650 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T10:01:44.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.043+0000 7fabdd9de740 10 client.16650 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T10:01:44.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.043+0000 7fabdd9de740 20 client.16650 populate_metadata read hostname 'smithi077' 2024-09-10T10:01:44.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.043+0000 7fabdd9de740 1 --2- 172.21.15.77:0/1812559188 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x560017cbe9f0 0x560017cc0de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:44.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.043+0000 7fabdd9de740 4 client.16650 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:01:44.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.043+0000 7fabdd9de740 1 -- 172.21.15.77:0/1812559188 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560017c17950 con 0x560017cbe9f0 2024-09-10T10:01:44.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.043+0000 7fabcb400640 1 --2- 172.21.15.77:0/1812559188 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x560017cbe9f0 0x560017cc0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:44.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.044+0000 7fabcb400640 1 --2- 172.21.15.77:0/1812559188 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x560017cbe9f0 0x560017cc0de0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24c20e3e6d62c292 server_cookie=ce97fb5b7d91df5a in_seq=0 out_seq=0 2024-09-10T10:01:44.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.044+0000 7fabc2000640 10 client.16650 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T10:01:44.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.045+0000 7fabc2000640 1 -- 172.21.15.77:0/1812559188 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560017c17950 con 0x560017cbe9f0 2024-09-10T10:01:44.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.045+0000 7fabc2000640 10 client.16650 handle_command_reply: tid=0 2024-09-10T10:01:44.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.045+0000 7fabdd9de740 1 client.16650 shutdown 2024-09-10T10:01:44.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.045+0000 7fabdd9de740 1 -- 172.21.15.77:0/1812559188 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560017caed70 msgr2=0x560017cac6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:44.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.045+0000 7fabdd9de740 1 --2- 172.21.15.77:0/1812559188 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560017caed70 0x560017cac6f0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fabcc02a930 tx=0x7fabcc005540 comp rx=0 tx=0).stop 2024-09-10T10:01:44.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.045+0000 7fabdd9de740 1 -- 172.21.15.77:0/1812559188 shutdown_connections 2024-09-10T10:01:44.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.045+0000 7fabdd9de740 1 --2- 172.21.15.77:0/1812559188 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x560017cbe9f0 0x560017cc0de0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:44.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.045+0000 7fabdd9de740 1 --2- 172.21.15.77:0/1812559188 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560017caed70 0x560017cac6f0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:44.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.045+0000 7fabdd9de740 1 --2- 172.21.15.77:0/1812559188 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560017ca44b0 0x560017cac1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:44.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.045+0000 7fabc2000640 0 client.16650 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T10:01:44.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.045+0000 7fabdd9de740 1 --2- 172.21.15.77:0/1812559188 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560017ca3b90 0x560017cabc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:44.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.045+0000 7fabdd9de740 1 -- 172.21.15.77:0/1812559188 >> 172.21.15.77:0/1812559188 conn(0x560017ca0120 msgr2=0x560017ca5e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:44.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.045+0000 7fabdd9de740 1 -- 172.21.15.77:0/1812559188 shutdown_connections 2024-09-10T10:01:44.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.046+0000 7fabdd9de740 1 -- 172.21.15.77:0/1812559188 wait complete. 2024-09-10T10:01:44.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.046+0000 7fabdd9de740 20 client.16650 trim_cache size 0 max 16384 2024-09-10T10:01:44.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.063+0000 7fabdd9de740 1 Processor -- start 2024-09-10T10:01:44.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.064+0000 7fabdd9de740 1 -- start start 2024-09-10T10:01:44.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.064+0000 7fabdd9de740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560017ca3b90 0x560017ca65f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:44.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.064+0000 7fabdd9de740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560017ca44b0 0x560017caa560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:44.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.064+0000 7fabdd9de740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560017caed70 0x560017c9f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:44.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.064+0000 7fabdd9de740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fabd41b1120 con 0x560017caed70 2024-09-10T10:01:44.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.064+0000 7fabdd9de740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fabd41b2c60 con 0x560017ca44b0 2024-09-10T10:01:44.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.064+0000 7fabdd9de740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fabd41b17f0 con 0x560017ca3b90 2024-09-10T10:01:44.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.064+0000 7fabd9000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560017caed70 0x560017c9f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:44.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.064+0000 7fabd9000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560017caed70 0x560017c9f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52198/0 (socket says 172.21.15.77:52198) 2024-09-10T10:01:44.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.064+0000 7fabd9000640 1 -- 172.21.15.77:0/3581644737 learned_addr learned my addr 172.21.15.77:0/3581644737 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:01:44.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.064+0000 7fabd9000640 1 --2- 172.21.15.77:0/3581644737 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560017ca3b90 0x560017ca65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:44.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.064+0000 7fabcb400640 1 --2- 172.21.15.77:0/3581644737 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560017ca44b0 0x560017caa560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:44.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.065+0000 7fabd9000640 1 -- 172.21.15.77:0/3581644737 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560017ca3b90 msgr2=0x560017ca65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:44.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.065+0000 7fabd9000640 1 --2- 172.21.15.77:0/3581644737 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560017ca3b90 0x560017ca65f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:44.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.065+0000 7fabd9000640 1 -- 172.21.15.77:0/3581644737 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560017ca44b0 msgr2=0x560017caa560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:44.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.065+0000 7fabd9000640 1 --2- 172.21.15.77:0/3581644737 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560017ca44b0 0x560017caa560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:44.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.065+0000 7fabd9000640 1 -- 172.21.15.77:0/3581644737 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560017cae5e0 con 0x560017caed70 2024-09-10T10:01:44.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.065+0000 7fabd9000640 1 --2- 172.21.15.77:0/3581644737 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560017caed70 0x560017c9f0d0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fabcc003650 tx=0x7fabcc005be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f649ad5f6507cadf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:44.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.065+0000 7fabc8c00640 1 -- 172.21.15.77:0/3581644737 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabcc035040 con 0x560017caed70 2024-09-10T10:01:44.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.065+0000 7fabc8c00640 1 -- 172.21.15.77:0/3581644737 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabcc0039c0 con 0x560017caed70 2024-09-10T10:01:44.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.065+0000 7fabc8c00640 1 -- 172.21.15.77:0/3581644737 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabcc003f20 con 0x560017caed70 2024-09-10T10:01:44.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.066+0000 7fabdd9de740 1 -- 172.21.15.77:0/3581644737 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560017caed70 msgr2=0x560017c9f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:44.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.066+0000 7fabdd9de740 1 --2- 172.21.15.77:0/3581644737 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560017caed70 0x560017c9f0d0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fabcc003650 tx=0x7fabcc005be0 comp rx=0 tx=0).stop 2024-09-10T10:01:44.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.066+0000 7fabdd9de740 1 -- 172.21.15.77:0/3581644737 shutdown_connections 2024-09-10T10:01:44.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.066+0000 7fabdd9de740 1 --2- 172.21.15.77:0/3581644737 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560017caed70 0x560017c9f0d0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:44.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.066+0000 7fabdd9de740 1 --2- 172.21.15.77:0/3581644737 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560017ca44b0 0x560017caa560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:44.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.066+0000 7fabdd9de740 1 --2- 172.21.15.77:0/3581644737 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560017ca3b90 0x560017ca65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:44.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.066+0000 7fabdd9de740 1 -- 172.21.15.77:0/3581644737 >> 172.21.15.77:0/3581644737 conn(0x560017ca0120 msgr2=0x560017c9e9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:44.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.066+0000 7fabdd9de740 1 -- 172.21.15.77:0/3581644737 shutdown_connections 2024-09-10T10:01:44.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.066+0000 7fabdd9de740 1 -- 172.21.15.77:0/3581644737 wait complete. 2024-09-10T10:01:44.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.067+0000 7fabdd9de740 1 -- start start 2024-09-10T10:01:44.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.067+0000 7fabdd9de740 1 Processor -- start 2024-09-10T10:01:44.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.067+0000 7fabdd9de740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560017ca3b90 0x560017cc2390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:44.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.067+0000 7fabdd9de740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560017ca44b0 0x560017cc28d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:44.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.067+0000 7fabdd9de740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560017caed70 0x560017d30b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:44.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.067+0000 7fabdd9de740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fabd41b2590 con 0x560017ca3b90 2024-09-10T10:01:44.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.067+0000 7fabdd9de740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fabd41b1ec0 con 0x560017caed70 2024-09-10T10:01:44.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.067+0000 7fabdd9de740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fabd41af100 con 0x560017ca44b0 2024-09-10T10:01:44.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.067+0000 7fabdd9de740 10 client.0 _ll_register_callbacks cb 0x7fabcc005020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:01:44.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.067+0000 7fabd9000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560017ca3b90 0x560017cc2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:44.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.068+0000 7fabd9000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560017ca3b90 0x560017cc2390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52204/0 (socket says 172.21.15.77:52204) 2024-09-10T10:01:44.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.068+0000 7fabd9000640 1 -- 172.21.15.77:0/4102886770 learned_addr learned my addr 172.21.15.77:0/4102886770 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:01:44.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.068+0000 7fabcb400640 1 --2- 172.21.15.77:0/4102886770 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560017ca44b0 0x560017cc28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:44.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.068+0000 7fabd9000640 1 --2- 172.21.15.77:0/4102886770 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560017caed70 0x560017d30b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:44.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.068+0000 7fabcb400640 1 -- 172.21.15.77:0/4102886770 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560017caed70 msgr2=0x560017d30b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:44.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.068+0000 7fabcb400640 1 --2- 172.21.15.77:0/4102886770 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560017caed70 0x560017d30b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:44.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.068+0000 7fabcb400640 1 -- 172.21.15.77:0/4102886770 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560017ca3b90 msgr2=0x560017cc2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:44.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.068+0000 7fabcb400640 1 --2- 172.21.15.77:0/4102886770 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560017ca3b90 0x560017cc2390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:44.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.068+0000 7fabcb400640 1 -- 172.21.15.77:0/4102886770 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560017d310b0 con 0x560017ca44b0 2024-09-10T10:01:44.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.068+0000 7fabd9000640 1 --2- 172.21.15.77:0/4102886770 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560017caed70 0x560017d30b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:01:44.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.068+0000 7fabcb400640 1 --2- 172.21.15.77:0/4102886770 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560017ca44b0 0x560017cc28d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x560017cab500 tx=0x7fabbc004610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=babbf04f264ef237 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:44.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.068+0000 7fabc2000640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-10T10:01:44.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.069+0000 7fabc2000640 1 -- 172.21.15.77:0/4102886770 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabbc023030 con 0x560017ca44b0 2024-09-10T10:01:44.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.069+0000 7fabdd9de740 1 -- 172.21.15.77:0/4102886770 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fabcc0039c0 con 0x560017ca44b0 2024-09-10T10:01:44.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.069+0000 7fabc2000640 1 -- 172.21.15.77:0/4102886770 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fabbc012020 con 0x560017ca44b0 2024-09-10T10:01:44.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.069+0000 7fabc2000640 1 -- 172.21.15.77:0/4102886770 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fabbc012220 con 0x560017ca44b0 2024-09-10T10:01:44.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.069+0000 7fabdd9de740 10 client.26731 fetch_fsmap learned FSMap version 35 2024-09-10T10:01:44.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.069+0000 7fabdd9de740 1 -- 172.21.15.77:0/4102886770 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x560017d32400 con 0x560017ca44b0 2024-09-10T10:01:44.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.069+0000 7fabc2000640 1 -- 172.21.15.77:0/4102886770 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabcc0039c0 con 0x560017ca44b0 2024-09-10T10:01:44.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.070+0000 7fabc2000640 1 -- 172.21.15.77:0/4102886770 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7fabbc0047a0 con 0x560017ca44b0 2024-09-10T10:01:44.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.070+0000 7fabdd9de740 10 client.26731 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T10:01:44.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.070+0000 7fabdd9de740 10 client.26731 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T10:01:44.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.070+0000 7fabdd9de740 20 client.26731 populate_metadata read hostname 'smithi077' 2024-09-10T10:01:44.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.070+0000 7fabdd9de740 1 --2- 172.21.15.77:0/4102886770 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x560017d32b20 0x560017d34f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:44.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.070+0000 7fabdd9de740 4 client.26731 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T10:01:44.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.070+0000 7fabdd9de740 1 -- 172.21.15.77:0/4102886770 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560017c11e60 con 0x560017d32b20 2024-09-10T10:01:44.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.070+0000 7fabd9000640 1 --2- 172.21.15.77:0/4102886770 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x560017d32b20 0x560017d34f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:44.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.071+0000 7fabd9000640 1 --2- 172.21.15.77:0/4102886770 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x560017d32b20 0x560017d34f10 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3982c73f8bc3d525 server_cookie=f9cedb92902b4239 in_seq=0 out_seq=0 2024-09-10T10:01:44.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.071+0000 7fabc2000640 10 client.26731 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T10:01:44.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.071+0000 7fabc2000640 1 -- 172.21.15.77:0/4102886770 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560017c11e60 con 0x560017d32b20 2024-09-10T10:01:44.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.071+0000 7fabc2000640 10 client.26731 handle_command_reply: tid=0 2024-09-10T10:01:44.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.071+0000 7fabdd9de740 1 client.26731 shutdown 2024-09-10T10:01:44.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.071+0000 7fabdd9de740 1 -- 172.21.15.77:0/4102886770 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560017ca44b0 msgr2=0x560017cc28d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:44.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.071+0000 7fabdd9de740 1 --2- 172.21.15.77:0/4102886770 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560017ca44b0 0x560017cc28d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x560017cab500 tx=0x7fabbc004610 comp rx=0 tx=0).stop 2024-09-10T10:01:44.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.071+0000 7fabdd9de740 1 -- 172.21.15.77:0/4102886770 shutdown_connections 2024-09-10T10:01:44.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.071+0000 7fabdd9de740 1 --2- 172.21.15.77:0/4102886770 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x560017d32b20 0x560017d34f10 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:44.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.071+0000 7fabdd9de740 1 --2- 172.21.15.77:0/4102886770 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560017caed70 0x560017d30b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:44.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.071+0000 7fabdd9de740 1 --2- 172.21.15.77:0/4102886770 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560017ca44b0 0x560017cc28d0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:44.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.071+0000 7fabdd9de740 1 --2- 172.21.15.77:0/4102886770 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560017ca3b90 0x560017cc2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:44.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.071+0000 7fabc2000640 0 client.26731 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T10:01:44.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.071+0000 7fabdd9de740 1 -- 172.21.15.77:0/4102886770 >> 172.21.15.77:0/4102886770 conn(0x560017ca0120 msgr2=0x560017cba5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:44.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.071+0000 7fabdd9de740 1 -- 172.21.15.77:0/4102886770 shutdown_connections 2024-09-10T10:01:44.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.071+0000 7fabdd9de740 1 -- 172.21.15.77:0/4102886770 wait complete. 2024-09-10T10:01:44.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.071+0000 7fabdd9de740 20 client.26731 trim_cache size 0 max 16384 2024-09-10T10:01:44.079 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-10T10:01:44.079 INFO:teuthology.orchestra.run.smithi077.stdout: "status": "no active scrubs running", 2024-09-10T10:01:44.080 INFO:teuthology.orchestra.run.smithi077.stdout: "scrubs": {} 2024-09-10T10:01:44.080 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-10T10:01:44.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.072+0000 7fabdb000640 1 -- 172.21.15.77:0/2536094289 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fabb80a2b80 msgr2=0x7fabb80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:44.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.072+0000 7fabdb000640 1 --2- 172.21.15.77:0/2536094289 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fabb80a2b80 0x7fabb80a5040 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fabc40a4990 tx=0x7fabc400cf10 comp rx=0 tx=0).stop 2024-09-10T10:01:44.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.072+0000 7fabdb000640 1 -- 172.21.15.77:0/2536094289 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fabd4104010 msgr2=0x7fabd419a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:44.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.072+0000 7fabdb000640 1 --2- 172.21.15.77:0/2536094289 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fabd4104010 0x7fabd419a630 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fabc400eb00 tx=0x7fabc400c2d0 comp rx=0 tx=0).stop 2024-09-10T10:01:44.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.072+0000 7fabdb000640 1 -- 172.21.15.77:0/2536094289 shutdown_connections 2024-09-10T10:01:44.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.072+0000 7fabdb000640 1 --2- 172.21.15.77:0/2536094289 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fabb80a2b80 0x7fabb80a5040 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:44.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.072+0000 7fabdb000640 1 --2- 172.21.15.77:0/2536094289 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fabd4109140 0x7fabd419ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:44.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.072+0000 7fabdb000640 1 --2- 172.21.15.77:0/2536094289 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fabd41049c0 0x7fabd419ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:44.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.072+0000 7fabdb000640 1 --2- 172.21.15.77:0/2536094289 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fabd4104010 0x7fabd419a630 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:44.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.072+0000 7fabdb000640 1 -- 172.21.15.77:0/2536094289 >> 172.21.15.77:0/2536094289 conn(0x7fabd40ffa50 msgr2=0x7fabd4100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:44.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.072+0000 7fabdb000640 1 -- 172.21.15.77:0/2536094289 shutdown_connections 2024-09-10T10:01:44.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:44.072+0000 7fabdb000640 1 -- 172.21.15.77:0/2536094289 wait complete. 2024-09-10T10:01:44.930 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T10:01:44.931 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:01:45.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:01:44 smithi120 ceph-mon[34770]: pgmap v588: 129 pgs: 129 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 882 KiB/s wr, 352 op/s 2024-09-10T10:01:45.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:01:44 smithi077 ceph-mon[30319]: pgmap v588: 129 pgs: 129 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 882 KiB/s wr, 352 op/s 2024-09-10T10:01:45.298 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:01:45.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:01:44 smithi134 ceph-mon[35674]: pgmap v588: 129 pgs: 129 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 882 KiB/s wr, 352 op/s 2024-09-10T10:01:45.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:45.864+0000 7f15dda00640 1 -- 172.21.15.77:0/2534287264 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f15d8074ee0 msgr2=0x7f15d810def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:45.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:45.864+0000 7f15dda00640 1 --2- 172.21.15.77:0/2534287264 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f15d8074ee0 0x7f15d810def0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f15c4009900 tx=0x7f15c402a480 comp rx=0 tx=0).stop 2024-09-10T10:01:45.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:45.864+0000 7f15dda00640 1 -- 172.21.15.77:0/2534287264 shutdown_connections 2024-09-10T10:01:45.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:45.864+0000 7f15dda00640 1 --2- 172.21.15.77:0/2534287264 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15d810e430 0x7f15d8110820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:45.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:45.864+0000 7f15dda00640 1 --2- 172.21.15.77:0/2534287264 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f15d8074ee0 0x7f15d810def0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:45.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:45.864+0000 7f15dda00640 1 --2- 172.21.15.77:0/2534287264 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f15d80745c0 0x7f15d80749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:45.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:45.864+0000 7f15dda00640 1 -- 172.21.15.77:0/2534287264 >> 172.21.15.77:0/2534287264 conn(0x7f15d80fd8b0 msgr2=0x7f15d80ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:45.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:45.864+0000 7f15dda00640 1 -- 172.21.15.77:0/2534287264 shutdown_connections 2024-09-10T10:01:45.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:45.864+0000 7f15dda00640 1 -- 172.21.15.77:0/2534287264 wait complete. 2024-09-10T10:01:45.868 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:45.865+0000 7f15dda00640 1 Processor -- start 2024-09-10T10:01:45.868 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:45.865+0000 7f15dda00640 1 -- start start 2024-09-10T10:01:45.868 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:45.865+0000 7f15dda00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15d80745c0 0x7f15d819a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:45.868 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:45.865+0000 7f15dda00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f15d8074ee0 0x7f15d819ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:45.868 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:45.866+0000 7f15dda00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f15d810e430 0x7f15d819eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:45.868 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:45.866+0000 7f15dda00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f15d81141c0 con 0x7f15d810e430 2024-09-10T10:01:45.868 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:45.866+0000 7f15dda00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f15d8114040 con 0x7f15d8074ee0 2024-09-10T10:01:45.868 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:45.866+0000 7f15dda00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f15d8114340 con 0x7f15d80745c0 2024-09-10T10:01:45.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:45.866+0000 7f15d7400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f15d810e430 0x7f15d819eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:45.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:45.866+0000 7f15d6a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15d80745c0 0x7f15d819a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:45.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:45.866+0000 7f15d6000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f15d8074ee0 0x7f15d819ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:45.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:45.866+0000 7f15d6a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15d80745c0 0x7f15d819a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:46384/0 (socket says 172.21.15.77:46384) 2024-09-10T10:01:45.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:45.866+0000 7f15d7400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f15d810e430 0x7f15d819eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52232/0 (socket says 172.21.15.77:52232) 2024-09-10T10:01:45.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:45.866+0000 7f15d6a00640 1 -- 172.21.15.77:0/2348135952 learned_addr learned my addr 172.21.15.77:0/2348135952 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:01:45.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:45.866+0000 7f15d6a00640 1 -- 172.21.15.77:0/2348135952 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f15d8074ee0 msgr2=0x7f15d819ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:45.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:45.866+0000 7f15d6a00640 1 --2- 172.21.15.77:0/2348135952 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f15d8074ee0 0x7f15d819ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:45.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:45.866+0000 7f15d6a00640 1 -- 172.21.15.77:0/2348135952 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f15d810e430 msgr2=0x7f15d819eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:45.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:45.866+0000 7f15d6a00640 1 --2- 172.21.15.77:0/2348135952 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f15d810e430 0x7f15d819eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:45.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:45.866+0000 7f15d6a00640 1 -- 172.21.15.77:0/2348135952 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15d819f590 con 0x7f15d80745c0 2024-09-10T10:01:45.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:45.866+0000 7f15d7400640 1 --2- 172.21.15.77:0/2348135952 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f15d810e430 0x7f15d819eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:01:45.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:45.867+0000 7f15d6a00640 1 --2- 172.21.15.77:0/2348135952 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15d80745c0 0x7f15d819a5e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f15c800e9f0 tx=0x7f15c800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:45.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:45.867+0000 7f15cf400640 1 -- 172.21.15.77:0/2348135952 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c800cd80 con 0x7f15d80745c0 2024-09-10T10:01:45.870 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:45.867+0000 7f15cf400640 1 -- 172.21.15.77:0/2348135952 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15c8010910 con 0x7f15d80745c0 2024-09-10T10:01:45.870 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:45.867+0000 7f15dda00640 1 -- 172.21.15.77:0/2348135952 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15d819f880 con 0x7f15d80745c0 2024-09-10T10:01:45.870 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:45.867+0000 7f15cf400640 1 -- 172.21.15.77:0/2348135952 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c8018990 con 0x7f15d80745c0 2024-09-10T10:01:45.870 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:45.867+0000 7f15dda00640 1 -- 172.21.15.77:0/2348135952 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15d81a7160 con 0x7f15d80745c0 2024-09-10T10:01:45.870 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:45.868+0000 7f15dda00640 1 -- 172.21.15.77:0/2348135952 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15d80749a0 con 0x7f15d80745c0 2024-09-10T10:01:45.871 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:45.870+0000 7f15cf400640 1 -- 172.21.15.77:0/2348135952 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f15c8010ac0 con 0x7f15d80745c0 2024-09-10T10:01:45.872 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:45.871+0000 7f15cf400640 1 --2- 172.21.15.77:0/2348135952 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f15bc0a2b80 0x7f15bc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:45.872 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:45.872+0000 7f15cf400640 1 -- 172.21.15.77:0/2348135952 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f15c80cfbe0 con 0x7f15d80745c0 2024-09-10T10:01:45.872 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:45.872+0000 7f15d6000640 1 --2- 172.21.15.77:0/2348135952 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f15bc0a2b80 0x7f15bc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:45.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:45.872+0000 7f15d6000640 1 --2- 172.21.15.77:0/2348135952 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f15bc0a2b80 0x7f15bc0a5040 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f15c4000c00 tx=0x7f15c4005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:45.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:45.874+0000 7f15cf400640 1 -- 172.21.15.77:0/2348135952 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f15c809c0a0 con 0x7f15d80745c0 2024-09-10T10:01:46.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:46.086+0000 7f15dda00640 1 -- 172.21.15.77:0/2348135952 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f15d819b940 con 0x7f15d80745c0 2024-09-10T10:01:46.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:46.087+0000 7f15cf400640 1 -- 172.21.15.77:0/2348135952 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11989 (secure 0 0 0) 0x7f15c80a03a0 con 0x7f15d80745c0 2024-09-10T10:01:46.089 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T10:01:46.089 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":35,"btime":"2024-09-10T10:01:05:922537+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3159289675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3159289675},{"type":"v1","addr":"172.21.15.77:6835","nonce":3159289675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1951962341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1951962341},{"type":"v1","addr":"172.21.15.77:6837","nonce":1951962341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":35,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:52:23.368252+0000","modified":"2024-09-10T10:01:05.706646+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24563,"mds_1":24524,"mds_2":24520,"mds_3":24502,"mds_4":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14487":{"gid":14487,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/4291121888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":4291121888},{"type":"v1","addr":"172.21.15.77:6839","nonce":4291121888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14499":{"gid":14499,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6841/445284194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":445284194},{"type":"v1","addr":"172.21.15.77:6841","nonce":445284194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1005054423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1005054423},{"type":"v1","addr":"172.21.15.134:6833","nonce":1005054423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6833/3716307744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":3716307744},{"type":"v1","addr":"172.21.15.120:6833","nonce":3716307744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/1319892544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1319892544},{"type":"v1","addr":"172.21.15.120:6835","nonce":1319892544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.134:6835/3293933699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3293933699},{"type":"v1","addr":"172.21.15.134:6835","nonce":3293933699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.134:6837/2461497091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2461497091},{"type":"v1","addr":"172.21.15.134:6837","nonce":2461497091}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":20,"addr":"172.21.15.134:6839/1965117680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1965117680},{"type":"v1","addr":"172.21.15.134:6839","nonce":1965117680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":20,"addr":"172.21.15.120:6837/3033280709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3033280709},{"type":"v1","addr":"172.21.15.120:6837","nonce":3033280709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.120:6839/1777710157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1777710157},{"type":"v1","addr":"172.21.15.120:6839","nonce":1777710157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24563,"qdb_cluster":[24496,24502,24520,24524,24563]},"id":1}]} 2024-09-10T10:01:46.089 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 35 2024-09-10T10:01:46.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:46.090+0000 7f15dda00640 1 -- 172.21.15.77:0/2348135952 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f15bc0a2b80 msgr2=0x7f15bc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:46.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:46.090+0000 7f15dda00640 1 --2- 172.21.15.77:0/2348135952 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f15bc0a2b80 0x7f15bc0a5040 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f15c4000c00 tx=0x7f15c4005b50 comp rx=0 tx=0).stop 2024-09-10T10:01:46.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:46.090+0000 7f15dda00640 1 -- 172.21.15.77:0/2348135952 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15d80745c0 msgr2=0x7f15d819a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:46.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:46.090+0000 7f15dda00640 1 --2- 172.21.15.77:0/2348135952 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15d80745c0 0x7f15d819a5e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f15c800e9f0 tx=0x7f15c800eeb0 comp rx=0 tx=0).stop 2024-09-10T10:01:46.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:46.090+0000 7f15dda00640 1 -- 172.21.15.77:0/2348135952 shutdown_connections 2024-09-10T10:01:46.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:46.090+0000 7f15dda00640 1 --2- 172.21.15.77:0/2348135952 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f15bc0a2b80 0x7f15bc0a5040 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:46.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:46.090+0000 7f15dda00640 1 --2- 172.21.15.77:0/2348135952 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f15d810e430 0x7f15d819eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:46.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:46.090+0000 7f15dda00640 1 --2- 172.21.15.77:0/2348135952 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f15d8074ee0 0x7f15d819ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:46.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:46.090+0000 7f15dda00640 1 --2- 172.21.15.77:0/2348135952 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15d80745c0 0x7f15d819a5e0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:46.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:46.090+0000 7f15dda00640 1 -- 172.21.15.77:0/2348135952 >> 172.21.15.77:0/2348135952 conn(0x7f15d80fd8b0 msgr2=0x7f15d80fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:46.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:46.090+0000 7f15dda00640 1 -- 172.21.15.77:0/2348135952 shutdown_connections 2024-09-10T10:01:46.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:46.090+0000 7f15dda00640 1 -- 172.21.15.77:0/2348135952 wait complete. 2024-09-10T10:01:46.850 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T10:01:47.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:01:46 smithi120 ceph-mon[34770]: pgmap v589: 129 pgs: 129 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 661 KiB/s wr, 234 op/s 2024-09-10T10:01:47.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:01:46 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/2348135952' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:01:47.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:01:46 smithi077 ceph-mon[30319]: pgmap v589: 129 pgs: 129 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 661 KiB/s wr, 234 op/s 2024-09-10T10:01:47.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:01:46 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/2348135952' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:01:47.224 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:01:47.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:01:46 smithi134 ceph-mon[35674]: pgmap v589: 129 pgs: 129 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 661 KiB/s wr, 234 op/s 2024-09-10T10:01:47.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:01:46 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/2348135952' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:01:47.868 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.867+0000 7f8a4e200640 1 -- 172.21.15.77:0/3857805003 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8a481049c0 msgr2=0x7f8a48108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:47.868 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.867+0000 7f8a4e200640 1 --2- 172.21.15.77:0/3857805003 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8a481049c0 0x7f8a48108a10 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f8a34009900 tx=0x7f8a3402a410 comp rx=0 tx=0).stop 2024-09-10T10:01:47.868 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.867+0000 7f8a4e200640 1 -- 172.21.15.77:0/3857805003 shutdown_connections 2024-09-10T10:01:47.868 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.867+0000 7f8a4e200640 1 --2- 172.21.15.77:0/3857805003 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8a48109140 0x7f8a48110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:47.868 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.867+0000 7f8a4e200640 1 --2- 172.21.15.77:0/3857805003 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8a481049c0 0x7f8a48108a10 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:47.868 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.867+0000 7f8a4e200640 1 --2- 172.21.15.77:0/3857805003 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8a48104010 0x7f8a481043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:47.868 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.867+0000 7f8a4e200640 1 -- 172.21.15.77:0/3857805003 >> 172.21.15.77:0/3857805003 conn(0x7f8a480ffa50 msgr2=0x7f8a48101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:47.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.867+0000 7f8a4e200640 1 -- 172.21.15.77:0/3857805003 shutdown_connections 2024-09-10T10:01:47.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.867+0000 7f8a4e200640 1 -- 172.21.15.77:0/3857805003 wait complete. 2024-09-10T10:01:47.870 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.868+0000 7f8a4e200640 1 Processor -- start 2024-09-10T10:01:47.870 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.868+0000 7f8a4e200640 1 -- start start 2024-09-10T10:01:47.870 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.868+0000 7f8a4e200640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8a48104010 0x7f8a4819a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:47.870 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.868+0000 7f8a4e200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8a481049c0 0x7f8a4819aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:47.870 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.868+0000 7f8a4e200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8a48109140 0x7f8a4819ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:47.870 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.868+0000 7f8a4e200640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8a48114160 con 0x7f8a481049c0 2024-09-10T10:01:47.870 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.868+0000 7f8a4e200640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8a48113fe0 con 0x7f8a48104010 2024-09-10T10:01:47.870 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.868+0000 7f8a4e200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8a481142e0 con 0x7f8a48109140 2024-09-10T10:01:47.870 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.869+0000 7f8a46a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8a481049c0 0x7f8a4819aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:47.870 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.869+0000 7f8a47400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8a48104010 0x7f8a4819a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:47.870 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.869+0000 7f8a46a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8a481049c0 0x7f8a4819aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52254/0 (socket says 172.21.15.77:52254) 2024-09-10T10:01:47.871 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.869+0000 7f8a46a00640 1 -- 172.21.15.77:0/697617542 learned_addr learned my addr 172.21.15.77:0/697617542 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:01:47.871 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.869+0000 7f8a47e00640 1 --2- 172.21.15.77:0/697617542 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8a48109140 0x7f8a4819ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:47.871 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.869+0000 7f8a47400640 1 -- 172.21.15.77:0/697617542 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8a48109140 msgr2=0x7f8a4819ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:47.871 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.869+0000 7f8a47400640 1 --2- 172.21.15.77:0/697617542 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8a48109140 0x7f8a4819ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:47.871 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.869+0000 7f8a47400640 1 -- 172.21.15.77:0/697617542 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8a481049c0 msgr2=0x7f8a4819aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:47.871 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.869+0000 7f8a47400640 1 --2- 172.21.15.77:0/697617542 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8a481049c0 0x7f8a4819aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:47.871 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.869+0000 7f8a47400640 1 -- 172.21.15.77:0/697617542 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a4819f610 con 0x7f8a48104010 2024-09-10T10:01:47.872 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.870+0000 7f8a47400640 1 --2- 172.21.15.77:0/697617542 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8a48104010 0x7f8a4819a660 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f8a3800e9f0 tx=0x7f8a3800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:47.872 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.870+0000 7f8a3fe00640 1 -- 172.21.15.77:0/697617542 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a3800ccc0 con 0x7f8a48104010 2024-09-10T10:01:47.872 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.870+0000 7f8a3fe00640 1 -- 172.21.15.77:0/697617542 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a38005020 con 0x7f8a48104010 2024-09-10T10:01:47.872 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.870+0000 7f8a3fe00640 1 -- 172.21.15.77:0/697617542 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a380049e0 con 0x7f8a48104010 2024-09-10T10:01:47.872 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.870+0000 7f8a4e200640 1 -- 172.21.15.77:0/697617542 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a4819f900 con 0x7f8a48104010 2024-09-10T10:01:47.872 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.870+0000 7f8a4e200640 1 -- 172.21.15.77:0/697617542 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a481a71e0 con 0x7f8a48104010 2024-09-10T10:01:47.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.874+0000 7f8a50bf4740 1 Processor -- start 2024-09-10T10:01:47.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.874+0000 7f8a50bf4740 1 -- start start 2024-09-10T10:01:47.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.874+0000 7f8a50bf4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad78346c70 0x55ad78347050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:47.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.874+0000 7f8a50bf4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ad78347590 0x55ad78351910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:47.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.875+0000 7f8a50bf4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad78351e50 0x55ad78354240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:47.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.875+0000 7f8a50bf4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ad782ad3e0 con 0x55ad78347590 2024-09-10T10:01:47.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.875+0000 7f8a50bf4740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ad78254d80 con 0x55ad78351e50 2024-09-10T10:01:47.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.875+0000 7f8a50bf4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ad782a93a0 con 0x55ad78346c70 2024-09-10T10:01:47.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.875+0000 7f8a3fe00640 1 -- 172.21.15.77:0/697617542 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8a38005560 con 0x7f8a48104010 2024-09-10T10:01:47.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.875+0000 7f8a46a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad78351e50 0x55ad78354240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:47.877 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.875+0000 7f8a47e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad78346c70 0x55ad78347050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:47.877 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.875+0000 7f8a46a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad78351e50 0x55ad78354240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:40492/0 (socket says 172.21.15.77:40492) 2024-09-10T10:01:47.877 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.875+0000 7f8a46a00640 1 -- 172.21.15.77:0/1186308335 learned_addr learned my addr 172.21.15.77:0/1186308335 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:01:47.877 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.875+0000 7f8a47400640 1 --2- 172.21.15.77:0/1186308335 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ad78347590 0x55ad78351910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:47.877 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.875+0000 7f8a46a00640 1 -- 172.21.15.77:0/1186308335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad78346c70 msgr2=0x55ad78347050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:47.877 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.875+0000 7f8a46a00640 1 --2- 172.21.15.77:0/1186308335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad78346c70 0x55ad78347050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:47.877 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.875+0000 7f8a46a00640 1 -- 172.21.15.77:0/1186308335 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ad78347590 msgr2=0x55ad78351910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:47.877 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.875+0000 7f8a46a00640 1 --2- 172.21.15.77:0/1186308335 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ad78347590 0x55ad78351910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:47.877 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.875+0000 7f8a46a00640 1 -- 172.21.15.77:0/1186308335 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad78273ae0 con 0x55ad78351e50 2024-09-10T10:01:47.877 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.875+0000 7f8a47400640 1 --2- 172.21.15.77:0/1186308335 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ad78347590 0x55ad78351910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:01:47.877 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.876+0000 7f8a46a00640 1 --2- 172.21.15.77:0/1186308335 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad78351e50 0x55ad78354240 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f8a34000c00 tx=0x7f8a34005dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11f56a1f463f1fae server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:47.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.876+0000 7f8a33e00640 1 -- 172.21.15.77:0/1186308335 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a34040070 con 0x55ad78351e50 2024-09-10T10:01:47.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.876+0000 7f8a33e00640 1 -- 172.21.15.77:0/1186308335 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a34004030 con 0x55ad78351e50 2024-09-10T10:01:47.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.876+0000 7f8a33e00640 1 -- 172.21.15.77:0/1186308335 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a34035040 con 0x55ad78351e50 2024-09-10T10:01:47.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.876+0000 7f8a3fe00640 1 --2- 172.21.15.77:0/697617542 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f8a2c0a2b80 0x7f8a2c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:47.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.877+0000 7f8a3fe00640 1 -- 172.21.15.77:0/697617542 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f8a380cf460 con 0x7f8a48104010 2024-09-10T10:01:47.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.877+0000 7f8a47e00640 1 --2- 172.21.15.77:0/697617542 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f8a2c0a2b80 0x7f8a2c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:47.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.877+0000 7f8a50bf4740 1 -- 172.21.15.77:0/1186308335 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad78351e50 msgr2=0x55ad78354240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:47.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.877+0000 7f8a50bf4740 1 --2- 172.21.15.77:0/1186308335 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad78351e50 0x55ad78354240 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f8a34000c00 tx=0x7f8a34005dd0 comp rx=0 tx=0).stop 2024-09-10T10:01:47.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.877+0000 7f8a50bf4740 1 -- 172.21.15.77:0/1186308335 shutdown_connections 2024-09-10T10:01:47.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.877+0000 7f8a50bf4740 1 --2- 172.21.15.77:0/1186308335 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad78351e50 0x55ad78354240 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:47.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.877+0000 7f8a50bf4740 1 --2- 172.21.15.77:0/1186308335 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ad78347590 0x55ad78351910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:47.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.877+0000 7f8a50bf4740 1 --2- 172.21.15.77:0/1186308335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad78346c70 0x55ad78347050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:47.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.877+0000 7f8a50bf4740 1 -- 172.21.15.77:0/1186308335 >> 172.21.15.77:0/1186308335 conn(0x55ad78343200 msgr2=0x55ad783435e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:47.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.877+0000 7f8a50bf4740 1 -- 172.21.15.77:0/1186308335 shutdown_connections 2024-09-10T10:01:47.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.877+0000 7f8a50bf4740 1 -- 172.21.15.77:0/1186308335 wait complete. 2024-09-10T10:01:47.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.878+0000 7f8a47e00640 1 --2- 172.21.15.77:0/697617542 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f8a2c0a2b80 0x7f8a2c0a5040 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f8a40005fd0 tx=0x7f8a40009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:47.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.878+0000 7f8a50bf4740 1 -- start start 2024-09-10T10:01:47.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.878+0000 7f8a50bf4740 1 Processor -- start 2024-09-10T10:01:47.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.878+0000 7f8a50bf4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad78346c70 0x55ad7834ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:47.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.878+0000 7f8a50bf4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad78347590 0x55ad7834f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:47.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.878+0000 7f8a50bf4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ad78351e50 0x55ad7834f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:47.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.878+0000 7f8a50bf4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ad78354a20 con 0x55ad78351e50 2024-09-10T10:01:47.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.878+0000 7f8a50bf4740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ad782680f0 con 0x55ad78347590 2024-09-10T10:01:47.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.878+0000 7f8a50bf4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ad78354ba0 con 0x55ad78346c70 2024-09-10T10:01:47.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.879+0000 7f8a50bf4740 10 client.0 _ll_register_callbacks cb 0x55ad7824fbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:01:47.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.879+0000 7f8a47e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ad78351e50 0x55ad7834f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:47.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.879+0000 7f8a46a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad78347590 0x55ad7834f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:47.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.879+0000 7f8a47e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ad78351e50 0x55ad7834f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52280/0 (socket says 172.21.15.77:52280) 2024-09-10T10:01:47.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.879+0000 7f8a47e00640 1 -- 172.21.15.77:0/384652493 learned_addr learned my addr 172.21.15.77:0/384652493 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:01:47.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.879+0000 7f8a47400640 1 --2- 172.21.15.77:0/384652493 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad78346c70 0x55ad7834ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:47.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.879+0000 7f8a46a00640 1 -- 172.21.15.77:0/384652493 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad78346c70 msgr2=0x55ad7834ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:47.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.879+0000 7f8a46a00640 1 --2- 172.21.15.77:0/384652493 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad78346c70 0x55ad7834ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:47.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.879+0000 7f8a46a00640 1 -- 172.21.15.77:0/384652493 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ad78351e50 msgr2=0x55ad7834f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:47.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.879+0000 7f8a46a00640 1 --2- 172.21.15.77:0/384652493 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ad78351e50 0x55ad7834f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:47.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.879+0000 7f8a46a00640 1 -- 172.21.15.77:0/384652493 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad7801e3c0 con 0x55ad78347590 2024-09-10T10:01:47.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.880+0000 7f8a46a00640 1 --2- 172.21.15.77:0/384652493 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad78347590 0x55ad7834f270 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8a34005ab0 tx=0x7f8a340095d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad0b717ee0658a67 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:47.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.880+0000 7f8a30c00640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T10:01:47.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.880+0000 7f8a30c00640 1 -- 172.21.15.77:0/384652493 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a34040070 con 0x55ad78347590 2024-09-10T10:01:47.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.880+0000 7f8a50bf4740 1 -- 172.21.15.77:0/384652493 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a34004030 con 0x55ad78347590 2024-09-10T10:01:47.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.880+0000 7f8a30c00640 1 -- 172.21.15.77:0/384652493 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f8a34005520 con 0x55ad78347590 2024-09-10T10:01:47.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.880+0000 7f8a30c00640 1 -- 172.21.15.77:0/384652493 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f8a34057050 con 0x55ad78347590 2024-09-10T10:01:47.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.880+0000 7f8a50bf4740 10 client.26804 fetch_fsmap learned FSMap version 35 2024-09-10T10:01:47.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.880+0000 7f8a50bf4740 1 -- 172.21.15.77:0/384652493 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55ad783609d0 con 0x55ad78347590 2024-09-10T10:01:47.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.881+0000 7f8a30c00640 1 -- 172.21.15.77:0/384652493 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a34004030 con 0x55ad78347590 2024-09-10T10:01:47.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.881+0000 7f8a30c00640 1 -- 172.21.15.77:0/384652493 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f8a3404c020 con 0x55ad78347590 2024-09-10T10:01:47.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.881+0000 7f8a50bf4740 10 client.26804 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T10:01:47.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.881+0000 7f8a50bf4740 10 client.26804 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-10T10:01:47.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.882+0000 7f8a50bf4740 20 client.26804 populate_metadata read hostname 'smithi077' 2024-09-10T10:01:47.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.882+0000 7f8a50bf4740 1 --2- 172.21.15.77:0/384652493 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x55ad78361d20 0x55ad78364110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:47.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.882+0000 7f8a50bf4740 4 client.26804 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:01:47.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.882+0000 7f8a50bf4740 1 -- 172.21.15.77:0/384652493 --> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ad782ba950 con 0x55ad78361d20 2024-09-10T10:01:47.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.882+0000 7f8a47400640 1 --2- 172.21.15.77:0/384652493 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x55ad78361d20 0x55ad78364110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:47.886 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.883+0000 7f8a47400640 1 --2- 172.21.15.77:0/384652493 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x55ad78361d20 0x55ad78364110 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=72f115e097e3f28a server_cookie=65936e274dfd7cb5 in_seq=0 out_seq=0 2024-09-10T10:01:47.886 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.883+0000 7f8a30c00640 10 client.26804 ms_handle_connect on v2:172.21.15.134:6834/3293933699 2024-09-10T10:01:47.886 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.883+0000 7f8a30c00640 1 -- 172.21.15.77:0/384652493 <== mds.4 v2:172.21.15.134:6834/3293933699 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ad782ba950 con 0x55ad78361d20 2024-09-10T10:01:47.886 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.883+0000 7f8a30c00640 10 client.26804 handle_command_reply: tid=0 2024-09-10T10:01:47.886 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.883+0000 7f8a50bf4740 1 client.26804 shutdown 2024-09-10T10:01:47.886 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.884+0000 7f8a50bf4740 1 -- 172.21.15.77:0/384652493 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad78347590 msgr2=0x55ad7834f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:47.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.884+0000 7f8a50bf4740 1 --2- 172.21.15.77:0/384652493 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad78347590 0x55ad7834f270 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8a34005ab0 tx=0x7f8a340095d0 comp rx=0 tx=0).stop 2024-09-10T10:01:47.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.884+0000 7f8a50bf4740 1 -- 172.21.15.77:0/384652493 shutdown_connections 2024-09-10T10:01:47.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.884+0000 7f8a50bf4740 1 --2- 172.21.15.77:0/384652493 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x55ad78361d20 0x55ad78364110 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:47.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.884+0000 7f8a50bf4740 1 --2- 172.21.15.77:0/384652493 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ad78351e50 0x55ad7834f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:47.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.884+0000 7f8a50bf4740 1 --2- 172.21.15.77:0/384652493 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad78347590 0x55ad7834f270 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:47.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.884+0000 7f8a50bf4740 1 --2- 172.21.15.77:0/384652493 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad78346c70 0x55ad7834ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:47.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.884+0000 7f8a30c00640 0 client.26804 ms_handle_reset on v2:172.21.15.134:6834/3293933699 2024-09-10T10:01:47.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.884+0000 7f8a50bf4740 1 -- 172.21.15.77:0/384652493 >> 172.21.15.77:0/384652493 conn(0x55ad78343200 msgr2=0x55ad78348f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:47.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.884+0000 7f8a50bf4740 1 -- 172.21.15.77:0/384652493 shutdown_connections 2024-09-10T10:01:47.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.884+0000 7f8a50bf4740 1 -- 172.21.15.77:0/384652493 wait complete. 2024-09-10T10:01:47.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.884+0000 7f8a50bf4740 20 client.26804 trim_cache size 0 max 16384 2024-09-10T10:01:47.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.898+0000 7f8a50bf4740 1 Processor -- start 2024-09-10T10:01:47.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.898+0000 7f8a50bf4740 1 -- start start 2024-09-10T10:01:47.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.898+0000 7f8a50bf4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ad78346c70 0x55ad783696e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:47.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.898+0000 7f8a50bf4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad78347590 0x55ad78369c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:47.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.898+0000 7f8a50bf4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad78351e50 0x55ad78341b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:47.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.898+0000 7f8a50bf4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8a481b1020 con 0x55ad78346c70 2024-09-10T10:01:47.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.898+0000 7f8a50bf4740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8a481b2b60 con 0x55ad78347590 2024-09-10T10:01:47.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.898+0000 7f8a50bf4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8a481b16f0 con 0x55ad78351e50 2024-09-10T10:01:47.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.899+0000 7f8a46a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad78347590 0x55ad78369c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:47.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.899+0000 7f8a47400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ad78346c70 0x55ad783696e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:47.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.899+0000 7f8a46a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad78347590 0x55ad78369c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:40508/0 (socket says 172.21.15.77:40508) 2024-09-10T10:01:47.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.899+0000 7f8a46a00640 1 -- 172.21.15.77:0/1289719493 learned_addr learned my addr 172.21.15.77:0/1289719493 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:01:47.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.899+0000 7f8a47e00640 1 --2- 172.21.15.77:0/1289719493 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad78351e50 0x55ad78341b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:47.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.899+0000 7f8a46a00640 1 -- 172.21.15.77:0/1289719493 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad78351e50 msgr2=0x55ad78341b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:47.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.899+0000 7f8a46a00640 1 --2- 172.21.15.77:0/1289719493 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad78351e50 0x55ad78341b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:47.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.899+0000 7f8a46a00640 1 -- 172.21.15.77:0/1289719493 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ad78346c70 msgr2=0x55ad783696e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:47.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.899+0000 7f8a46a00640 1 --2- 172.21.15.77:0/1289719493 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ad78346c70 0x55ad783696e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:47.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.899+0000 7f8a46a00640 1 -- 172.21.15.77:0/1289719493 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad78346aa0 con 0x55ad78347590 2024-09-10T10:01:47.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.899+0000 7f8a47400640 1 --2- 172.21.15.77:0/1289719493 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ad78346c70 0x55ad783696e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:01:47.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.900+0000 7f8a46a00640 1 --2- 172.21.15.77:0/1289719493 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad78347590 0x55ad78369c20 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f8a34006010 tx=0x7f8a34003780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d29001758e62f4f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:47.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.900+0000 7f8a33400640 1 -- 172.21.15.77:0/1289719493 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a34040070 con 0x55ad78347590 2024-09-10T10:01:47.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.900+0000 7f8a33400640 1 -- 172.21.15.77:0/1289719493 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a340053f0 con 0x55ad78347590 2024-09-10T10:01:47.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.900+0000 7f8a33400640 1 -- 172.21.15.77:0/1289719493 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a34035040 con 0x55ad78347590 2024-09-10T10:01:47.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.900+0000 7f8a50bf4740 1 -- 172.21.15.77:0/1289719493 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad78347590 msgr2=0x55ad78369c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:47.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.901+0000 7f8a50bf4740 1 --2- 172.21.15.77:0/1289719493 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad78347590 0x55ad78369c20 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f8a34006010 tx=0x7f8a34003780 comp rx=0 tx=0).stop 2024-09-10T10:01:47.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.901+0000 7f8a50bf4740 1 -- 172.21.15.77:0/1289719493 shutdown_connections 2024-09-10T10:01:47.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.901+0000 7f8a50bf4740 1 --2- 172.21.15.77:0/1289719493 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad78351e50 0x55ad78341b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:47.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.901+0000 7f8a50bf4740 1 --2- 172.21.15.77:0/1289719493 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad78347590 0x55ad78369c20 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:47.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.901+0000 7f8a50bf4740 1 --2- 172.21.15.77:0/1289719493 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ad78346c70 0x55ad783696e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:47.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.901+0000 7f8a50bf4740 1 -- 172.21.15.77:0/1289719493 >> 172.21.15.77:0/1289719493 conn(0x55ad78343200 msgr2=0x55ad78350db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:47.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.901+0000 7f8a50bf4740 1 -- 172.21.15.77:0/1289719493 shutdown_connections 2024-09-10T10:01:47.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.901+0000 7f8a50bf4740 1 -- 172.21.15.77:0/1289719493 wait complete. 2024-09-10T10:01:47.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.901+0000 7f8a50bf4740 1 -- start start 2024-09-10T10:01:47.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.901+0000 7f8a50bf4740 1 Processor -- start 2024-09-10T10:01:47.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.902+0000 7f8a50bf4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad78346c70 0x55ad7835a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:47.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.902+0000 7f8a50bf4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ad78347590 0x55ad783558b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:47.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.902+0000 7f8a50bf4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad78351e50 0x55ad78355df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:47.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.902+0000 7f8a50bf4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8a481b2490 con 0x55ad78347590 2024-09-10T10:01:47.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.902+0000 7f8a50bf4740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8a481b1dc0 con 0x55ad78346c70 2024-09-10T10:01:47.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.902+0000 7f8a50bf4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8a481af000 con 0x55ad78351e50 2024-09-10T10:01:47.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.902+0000 7f8a50bf4740 10 client.0 _ll_register_callbacks cb 0x7f8a400099c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:01:47.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.902+0000 7f8a46a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ad78347590 0x55ad783558b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:47.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.902+0000 7f8a47400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad78346c70 0x55ad7835a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:47.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.902+0000 7f8a47e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad78351e50 0x55ad78355df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:47.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.902+0000 7f8a47400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad78346c70 0x55ad7835a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:40518/0 (socket says 172.21.15.77:40518) 2024-09-10T10:01:47.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.902+0000 7f8a47400640 1 -- 172.21.15.77:0/2682275805 learned_addr learned my addr 172.21.15.77:0/2682275805 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:01:47.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.903+0000 7f8a47400640 1 -- 172.21.15.77:0/2682275805 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad78351e50 msgr2=0x55ad78355df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:47.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.903+0000 7f8a47400640 1 --2- 172.21.15.77:0/2682275805 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad78351e50 0x55ad78355df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:47.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.903+0000 7f8a47400640 1 -- 172.21.15.77:0/2682275805 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ad78347590 msgr2=0x55ad783558b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:47.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.903+0000 7f8a47400640 1 --2- 172.21.15.77:0/2682275805 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ad78347590 0x55ad783558b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:47.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.903+0000 7f8a47400640 1 -- 172.21.15.77:0/2682275805 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad78343e90 con 0x55ad78346c70 2024-09-10T10:01:47.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.903+0000 7f8a47e00640 1 --2- 172.21.15.77:0/2682275805 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad78351e50 0x55ad78355df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:01:47.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.903+0000 7f8a47400640 1 --2- 172.21.15.77:0/2682275805 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad78346c70 0x55ad7835a580 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8a3809b580 tx=0x7f8a38005f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eeaac236dec20fd9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:47.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.903+0000 7f8a30c00640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T10:01:47.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.903+0000 7f8a30c00640 1 -- 172.21.15.77:0/2682275805 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a3801c040 con 0x55ad78346c70 2024-09-10T10:01:47.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.903+0000 7f8a30c00640 1 -- 172.21.15.77:0/2682275805 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f8a38097f40 con 0x55ad78346c70 2024-09-10T10:01:47.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.903+0000 7f8a50bf4740 1 -- 172.21.15.77:0/2682275805 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a340053f0 con 0x55ad78346c70 2024-09-10T10:01:47.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.903+0000 7f8a30c00640 1 -- 172.21.15.77:0/2682275805 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f8a38084e60 con 0x55ad78346c70 2024-09-10T10:01:47.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.903+0000 7f8a50bf4740 10 client.26810 fetch_fsmap learned FSMap version 35 2024-09-10T10:01:47.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.903+0000 7f8a50bf4740 1 -- 172.21.15.77:0/2682275805 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55ad7833e770 con 0x55ad78346c70 2024-09-10T10:01:47.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.904+0000 7f8a30c00640 1 -- 172.21.15.77:0/2682275805 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a340053f0 con 0x55ad78346c70 2024-09-10T10:01:47.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.904+0000 7f8a30c00640 1 -- 172.21.15.77:0/2682275805 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f8a38004be0 con 0x55ad78346c70 2024-09-10T10:01:47.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.904+0000 7f8a50bf4740 10 client.26810 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T10:01:47.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.904+0000 7f8a50bf4740 10 client.26810 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-10T10:01:47.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.904+0000 7f8a50bf4740 20 client.26810 populate_metadata read hostname 'smithi077' 2024-09-10T10:01:47.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.904+0000 7f8a50bf4740 1 --2- 172.21.15.77:0/2682275805 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x55ad783cca70 0x55ad783cee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:47.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.904+0000 7f8a50bf4740 4 client.26810 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:01:47.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.904+0000 7f8a50bf4740 1 -- 172.21.15.77:0/2682275805 --> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ad782b4e60 con 0x55ad783cca70 2024-09-10T10:01:47.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.905+0000 7f8a46a00640 1 --2- 172.21.15.77:0/2682275805 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x55ad783cca70 0x55ad783cee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:47.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.905+0000 7f8a30c00640 10 client.26810 ms_handle_connect on v2:172.21.15.134:6834/3293933699 2024-09-10T10:01:47.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.905+0000 7f8a46a00640 1 --2- 172.21.15.77:0/2682275805 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x55ad783cca70 0x55ad783cee60 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bebd3fa7fa4a2f67 server_cookie=46717be86f39ebc in_seq=0 out_seq=0 2024-09-10T10:01:47.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.905+0000 7f8a30c00640 1 -- 172.21.15.77:0/2682275805 <== mds.4 v2:172.21.15.134:6834/3293933699 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ad782b4e60 con 0x55ad783cca70 2024-09-10T10:01:47.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.905+0000 7f8a30c00640 10 client.26810 handle_command_reply: tid=0 2024-09-10T10:01:47.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.905+0000 7f8a50bf4740 1 client.26810 shutdown 2024-09-10T10:01:47.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.906+0000 7f8a50bf4740 1 -- 172.21.15.77:0/2682275805 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad78346c70 msgr2=0x55ad7835a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:47.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.906+0000 7f8a50bf4740 1 --2- 172.21.15.77:0/2682275805 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad78346c70 0x55ad7835a580 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8a3809b580 tx=0x7f8a38005f10 comp rx=0 tx=0).stop 2024-09-10T10:01:47.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.906+0000 7f8a50bf4740 1 -- 172.21.15.77:0/2682275805 shutdown_connections 2024-09-10T10:01:47.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.906+0000 7f8a50bf4740 1 --2- 172.21.15.77:0/2682275805 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x55ad783cca70 0x55ad783cee60 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:47.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.906+0000 7f8a30c00640 0 client.26810 ms_handle_reset on v2:172.21.15.134:6834/3293933699 2024-09-10T10:01:47.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.906+0000 7f8a50bf4740 1 --2- 172.21.15.77:0/2682275805 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad78351e50 0x55ad78355df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:47.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.906+0000 7f8a50bf4740 1 --2- 172.21.15.77:0/2682275805 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ad78347590 0x55ad783558b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:47.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.906+0000 7f8a50bf4740 1 --2- 172.21.15.77:0/2682275805 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad78346c70 0x55ad7835a580 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:47.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.906+0000 7f8a50bf4740 1 -- 172.21.15.77:0/2682275805 >> 172.21.15.77:0/2682275805 conn(0x55ad78343200 msgr2=0x55ad7834ecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:47.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.906+0000 7f8a50bf4740 1 -- 172.21.15.77:0/2682275805 shutdown_connections 2024-09-10T10:01:47.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.906+0000 7f8a50bf4740 1 -- 172.21.15.77:0/2682275805 wait complete. 2024-09-10T10:01:47.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.907+0000 7f8a50bf4740 20 client.26810 trim_cache size 0 max 16384 2024-09-10T10:01:47.911 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-10T10:01:47.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.907+0000 7f8a4e200640 1 -- 172.21.15.77:0/697617542 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f8a2c0a2b80 msgr2=0x7f8a2c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:47.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.907+0000 7f8a4e200640 1 --2- 172.21.15.77:0/697617542 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f8a2c0a2b80 0x7f8a2c0a5040 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f8a40005fd0 tx=0x7f8a40009500 comp rx=0 tx=0).stop 2024-09-10T10:01:47.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.907+0000 7f8a4e200640 1 -- 172.21.15.77:0/697617542 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8a48104010 msgr2=0x7f8a4819a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:47.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.907+0000 7f8a4e200640 1 --2- 172.21.15.77:0/697617542 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8a48104010 0x7f8a4819a660 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f8a3800e9f0 tx=0x7f8a3800eeb0 comp rx=0 tx=0).stop 2024-09-10T10:01:47.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.907+0000 7f8a4e200640 1 -- 172.21.15.77:0/697617542 shutdown_connections 2024-09-10T10:01:47.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.907+0000 7f8a4e200640 1 --2- 172.21.15.77:0/697617542 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f8a2c0a2b80 0x7f8a2c0a5040 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:47.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.907+0000 7f8a4e200640 1 --2- 172.21.15.77:0/697617542 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8a48109140 0x7f8a4819ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:47.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.907+0000 7f8a4e200640 1 --2- 172.21.15.77:0/697617542 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8a481049c0 0x7f8a4819aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:47.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.907+0000 7f8a4e200640 1 --2- 172.21.15.77:0/697617542 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8a48104010 0x7f8a4819a660 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:47.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.907+0000 7f8a4e200640 1 -- 172.21.15.77:0/697617542 >> 172.21.15.77:0/697617542 conn(0x7f8a480ffa50 msgr2=0x7f8a48100db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:47.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.908+0000 7f8a4e200640 1 -- 172.21.15.77:0/697617542 shutdown_connections 2024-09-10T10:01:47.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:47.908+0000 7f8a4e200640 1 -- 172.21.15.77:0/697617542 wait complete. 2024-09-10T10:01:48.818 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T10:01:49.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:01:48 smithi120 ceph-mon[34770]: pgmap v590: 129 pgs: 129 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 506 KiB/s wr, 179 op/s 2024-09-10T10:01:49.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:01:48 smithi077 ceph-mon[30319]: pgmap v590: 129 pgs: 129 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 506 KiB/s wr, 179 op/s 2024-09-10T10:01:49.195 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:01:49.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:01:48 smithi134 ceph-mon[35674]: pgmap v590: 129 pgs: 129 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 506 KiB/s wr, 179 op/s 2024-09-10T10:01:49.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.762+0000 7f6433e00640 1 -- 172.21.15.77:0/3765953847 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6434102360 msgr2=0x7f643410e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:49.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.762+0000 7f6433e00640 1 --2- 172.21.15.77:0/3765953847 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6434102360 0x7f643410e890 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f642c009c10 tx=0x7f642c02a6e0 comp rx=0 tx=0).stop 2024-09-10T10:01:49.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.762+0000 7f6433e00640 1 -- 172.21.15.77:0/3765953847 shutdown_connections 2024-09-10T10:01:49.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.762+0000 7f6433e00640 1 --2- 172.21.15.77:0/3765953847 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6434102360 0x7f643410e890 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:49.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.762+0000 7f6433e00640 1 --2- 172.21.15.77:0/3765953847 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f64341019c0 0x7f6434101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:49.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.762+0000 7f6433e00640 1 --2- 172.21.15.77:0/3765953847 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f64341079c0 0x7f6434107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:49.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.762+0000 7f6433e00640 1 -- 172.21.15.77:0/3765953847 >> 172.21.15.77:0/3765953847 conn(0x7f64340fd740 msgr2=0x7f64340ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:49.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.762+0000 7f6433e00640 1 -- 172.21.15.77:0/3765953847 shutdown_connections 2024-09-10T10:01:49.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.763+0000 7f6433e00640 1 -- 172.21.15.77:0/3765953847 wait complete. 2024-09-10T10:01:49.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.764+0000 7f6433e00640 1 Processor -- start 2024-09-10T10:01:49.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.764+0000 7f6433e00640 1 -- start start 2024-09-10T10:01:49.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.764+0000 7f6433e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f64341019c0 0x7f64341982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:49.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.764+0000 7f6433e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6434102360 0x7f64341987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:49.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.764+0000 7f6433e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f64341079c0 0x7f643419cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:49.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.764+0000 7f6433e00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6434111d70 con 0x7f64341079c0 2024-09-10T10:01:49.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.764+0000 7f6433e00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6434111bf0 con 0x7f64341019c0 2024-09-10T10:01:49.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.764+0000 7f6433e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6434111ef0 con 0x7f6434102360 2024-09-10T10:01:49.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.764+0000 7f6433400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f64341079c0 0x7f643419cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:49.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.765+0000 7f6433400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f64341079c0 0x7f643419cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52304/0 (socket says 172.21.15.77:52304) 2024-09-10T10:01:49.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.765+0000 7f6432000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6434102360 0x7f64341987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:49.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.765+0000 7f6433400640 1 -- 172.21.15.77:0/4057063249 learned_addr learned my addr 172.21.15.77:0/4057063249 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:01:49.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.765+0000 7f6432a00640 1 --2- 172.21.15.77:0/4057063249 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f64341019c0 0x7f64341982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:49.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.765+0000 7f6433400640 1 -- 172.21.15.77:0/4057063249 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6434102360 msgr2=0x7f64341987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:49.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.765+0000 7f6433400640 1 --2- 172.21.15.77:0/4057063249 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6434102360 0x7f64341987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:49.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.765+0000 7f6433400640 1 -- 172.21.15.77:0/4057063249 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f64341019c0 msgr2=0x7f64341982b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:49.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.765+0000 7f6433400640 1 --2- 172.21.15.77:0/4057063249 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f64341019c0 0x7f64341982b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:49.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.765+0000 7f6433400640 1 -- 172.21.15.77:0/4057063249 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f643419d260 con 0x7f64341079c0 2024-09-10T10:01:49.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.765+0000 7f6432a00640 1 --2- 172.21.15.77:0/4057063249 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f64341019c0 0x7f64341982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:01:49.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.765+0000 7f6433400640 1 --2- 172.21.15.77:0/4057063249 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f64341079c0 0x7f643419cb80 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f642c0305d0 tx=0x7f642c030600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:49.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.766+0000 7f642b400640 1 -- 172.21.15.77:0/4057063249 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f642c041020 con 0x7f64341079c0 2024-09-10T10:01:49.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.766+0000 7f642b400640 1 -- 172.21.15.77:0/4057063249 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f642c037040 con 0x7f64341079c0 2024-09-10T10:01:49.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.766+0000 7f6433e00640 1 -- 172.21.15.77:0/4057063249 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f643419d4f0 con 0x7f64341079c0 2024-09-10T10:01:49.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.766+0000 7f6433e00640 1 -- 172.21.15.77:0/4057063249 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64341a4e30 con 0x7f64341079c0 2024-09-10T10:01:49.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.766+0000 7f642b400640 1 -- 172.21.15.77:0/4057063249 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f642c036070 con 0x7f64341079c0 2024-09-10T10:01:49.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.769+0000 7f642b400640 1 -- 172.21.15.77:0/4057063249 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f642c044020 con 0x7f64341079c0 2024-09-10T10:01:49.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.770+0000 7f643c08c740 1 Processor -- start 2024-09-10T10:01:49.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.771+0000 7f643c08c740 1 -- start start 2024-09-10T10:01:49.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.771+0000 7f642b400640 1 --2- 172.21.15.77:0/4057063249 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f64180a2b80 0x7f64180a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:49.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.771+0000 7f642b400640 1 -- 172.21.15.77:0/4057063249 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f642c0eff30 con 0x7f64341079c0 2024-09-10T10:01:49.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.771+0000 7f643c08c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f560d7cd00 0x55f560d7d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:49.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.771+0000 7f6432000640 1 --2- 172.21.15.77:0/4057063249 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f64180a2b80 0x7f64180a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:49.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.771+0000 7f643c08c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f560d7d620 0x55f560d8bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:49.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.771+0000 7f643c08c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f560d8c430 0x55f560d8e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:49.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.771+0000 7f643c08c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f560ce33e0 con 0x55f560d8c430 2024-09-10T10:01:49.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.771+0000 7f643c08c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f560c8ad80 con 0x55f560d7cd00 2024-09-10T10:01:49.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.771+0000 7f643c08c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f560cdf3a0 con 0x55f560d7d620 2024-09-10T10:01:49.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.772+0000 7f6432a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f560d7cd00 0x55f560d7d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:49.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.772+0000 7f6433400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f560d8c430 0x55f560d8e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:49.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.772+0000 7f6432a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f560d7cd00 0x55f560d7d0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:40550/0 (socket says 172.21.15.77:40550) 2024-09-10T10:01:49.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.772+0000 7f6432000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f560d7d620 0x55f560d8bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:49.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.772+0000 7f6432a00640 1 -- 172.21.15.77:0/114338257 learned_addr learned my addr 172.21.15.77:0/114338257 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:01:49.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.772+0000 7f6433400640 1 -- 172.21.15.77:0/114338257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f560d7d620 msgr2=0x55f560d8bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:49.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.772+0000 7f6433400640 1 --2- 172.21.15.77:0/114338257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f560d7d620 0x55f560d8bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:49.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.772+0000 7f6433400640 1 -- 172.21.15.77:0/114338257 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f560d7cd00 msgr2=0x55f560d7d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:49.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.772+0000 7f6433400640 1 --2- 172.21.15.77:0/114338257 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f560d7cd00 0x55f560d7d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:49.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.772+0000 7f6433400640 1 -- 172.21.15.77:0/114338257 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f560ca9ae0 con 0x55f560d8c430 2024-09-10T10:01:49.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.772+0000 7f6432000640 1 --2- 172.21.15.77:0/4057063249 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f64180a2b80 0x7f64180a5040 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f64340ffda0 tx=0x7f6420011920 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:49.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.772+0000 7f6433400640 1 --2- 172.21.15.77:0/114338257 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f560d8c430 0x55f560d8e820 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f642c004af0 tx=0x7f642c0bc1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cdcb6dce960a433 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:49.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.773+0000 7f641f400640 1 -- 172.21.15.77:0/114338257 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f642c003d80 con 0x55f560d8c430 2024-09-10T10:01:49.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.773+0000 7f641f400640 1 -- 172.21.15.77:0/114338257 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f642c0bca40 con 0x55f560d8c430 2024-09-10T10:01:49.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.773+0000 7f641f400640 1 -- 172.21.15.77:0/114338257 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f642c0c4b20 con 0x55f560d8c430 2024-09-10T10:01:49.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.773+0000 7f643c08c740 1 -- 172.21.15.77:0/114338257 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f560d8c430 msgr2=0x55f560d8e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:49.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.773+0000 7f643c08c740 1 --2- 172.21.15.77:0/114338257 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f560d8c430 0x55f560d8e820 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f642c004af0 tx=0x7f642c0bc1e0 comp rx=0 tx=0).stop 2024-09-10T10:01:49.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.773+0000 7f643c08c740 1 -- 172.21.15.77:0/114338257 shutdown_connections 2024-09-10T10:01:49.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.773+0000 7f643c08c740 1 --2- 172.21.15.77:0/114338257 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f560d8c430 0x55f560d8e820 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:49.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.773+0000 7f643c08c740 1 --2- 172.21.15.77:0/114338257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f560d7d620 0x55f560d8bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:49.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.773+0000 7f643c08c740 1 --2- 172.21.15.77:0/114338257 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f560d7cd00 0x55f560d7d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:49.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.773+0000 7f643c08c740 1 -- 172.21.15.77:0/114338257 >> 172.21.15.77:0/114338257 conn(0x55f560d793d0 msgr2=0x55f560d797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:49.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.774+0000 7f643c08c740 1 -- 172.21.15.77:0/114338257 shutdown_connections 2024-09-10T10:01:49.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.774+0000 7f643c08c740 1 -- 172.21.15.77:0/114338257 wait complete. 2024-09-10T10:01:49.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.774+0000 7f643c08c740 1 -- start start 2024-09-10T10:01:49.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.774+0000 7f643c08c740 1 Processor -- start 2024-09-10T10:01:49.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.775+0000 7f643c08c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f560d7cd00 0x55f560d893e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:49.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.775+0000 7f643c08c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f560d7d620 0x55f560d89920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:49.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.775+0000 7f643c08c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f560d8c430 0x55f560d89e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:49.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.775+0000 7f643c08c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f560d8edc0 con 0x55f560d7d620 2024-09-10T10:01:49.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.775+0000 7f643c08c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f560c9e0f0 con 0x55f560d7cd00 2024-09-10T10:01:49.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.775+0000 7f643c08c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f560d8ef40 con 0x55f560d8c430 2024-09-10T10:01:49.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.775+0000 7f643c08c740 10 client.0 _ll_register_callbacks cb 0x55f560c85bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:01:49.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.775+0000 7f6432000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f560d7d620 0x55f560d89920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:49.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.775+0000 7f6433400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f560d8c430 0x55f560d89e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:49.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.775+0000 7f6432000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f560d7d620 0x55f560d89920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52314/0 (socket says 172.21.15.77:52314) 2024-09-10T10:01:49.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.775+0000 7f6432000640 1 -- 172.21.15.77:0/2677650428 learned_addr learned my addr 172.21.15.77:0/2677650428 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:01:49.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.775+0000 7f6432a00640 1 --2- 172.21.15.77:0/2677650428 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f560d7cd00 0x55f560d893e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:49.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.776+0000 7f6432000640 1 -- 172.21.15.77:0/2677650428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f560d8c430 msgr2=0x55f560d89e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:49.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.776+0000 7f6432000640 1 --2- 172.21.15.77:0/2677650428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f560d8c430 0x55f560d89e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:49.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.776+0000 7f6432000640 1 -- 172.21.15.77:0/2677650428 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f560d7cd00 msgr2=0x55f560d893e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:49.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.776+0000 7f6432000640 1 --2- 172.21.15.77:0/2677650428 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f560d7cd00 0x55f560d893e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:49.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.776+0000 7f6432000640 1 -- 172.21.15.77:0/2677650428 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f560a543c0 con 0x55f560d7d620 2024-09-10T10:01:49.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.776+0000 7f6432000640 1 --2- 172.21.15.77:0/2677650428 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f560d7d620 0x55f560d89920 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f642000faa0 tx=0x7f6420004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=957866a7d18361 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:49.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.776+0000 7f6413e00640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-10T10:01:49.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.776+0000 7f6413e00640 1 -- 172.21.15.77:0/2677650428 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6420024070 con 0x55f560d7d620 2024-09-10T10:01:49.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.776+0000 7f6413e00640 1 -- 172.21.15.77:0/2677650428 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f6420028400 con 0x55f560d7d620 2024-09-10T10:01:49.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.776+0000 7f643c08c740 1 -- 172.21.15.77:0/2677650428 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f642c0bca40 con 0x55f560d7d620 2024-09-10T10:01:49.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.777+0000 7f6413e00640 1 -- 172.21.15.77:0/2677650428 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f6420033050 con 0x55f560d7d620 2024-09-10T10:01:49.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.777+0000 7f643c08c740 10 client.16689 fetch_fsmap learned FSMap version 35 2024-09-10T10:01:49.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.777+0000 7f643c08c740 1 -- 172.21.15.77:0/2677650428 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55f560d9adf0 con 0x55f560d7d620 2024-09-10T10:01:49.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.777+0000 7f6413e00640 1 -- 172.21.15.77:0/2677650428 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f642c0bca40 con 0x55f560d7d620 2024-09-10T10:01:49.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.777+0000 7f6413e00640 1 -- 172.21.15.77:0/2677650428 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f642001d030 con 0x55f560d7d620 2024-09-10T10:01:49.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.777+0000 7f643c08c740 10 client.16689 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T10:01:49.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.777+0000 7f643c08c740 10 client.16689 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T10:01:49.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.777+0000 7f643c08c740 20 client.16689 populate_metadata read hostname 'smithi077' 2024-09-10T10:01:49.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.777+0000 7f643c08c740 1 --2- 172.21.15.77:0/2677650428 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x55f560d9c2b0 0x55f560d9e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:49.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.778+0000 7f643c08c740 4 client.16689 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:01:49.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.778+0000 7f643c08c740 1 -- 172.21.15.77:0/2677650428 --> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f560cf0950 con 0x55f560d9c2b0 2024-09-10T10:01:49.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.778+0000 7f6432a00640 1 --2- 172.21.15.77:0/2677650428 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x55f560d9c2b0 0x55f560d9e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:49.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.778+0000 7f6432a00640 1 --2- 172.21.15.77:0/2677650428 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x55f560d9c2b0 0x55f560d9e730 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=daa3db910cbd225 server_cookie=d108bdc0e5a403b2 in_seq=0 out_seq=0 2024-09-10T10:01:49.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.778+0000 7f6413e00640 10 client.16689 ms_handle_connect on v2:172.21.15.134:6836/2461497091 2024-09-10T10:01:49.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.779+0000 7f6413e00640 1 -- 172.21.15.77:0/2677650428 <== mds.3 v2:172.21.15.134:6836/2461497091 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f560cf0950 con 0x55f560d9c2b0 2024-09-10T10:01:49.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.779+0000 7f6413e00640 10 client.16689 handle_command_reply: tid=0 2024-09-10T10:01:49.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.779+0000 7f643c08c740 1 client.16689 shutdown 2024-09-10T10:01:49.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.780+0000 7f643c08c740 1 -- 172.21.15.77:0/2677650428 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f560d7d620 msgr2=0x55f560d89920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:49.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.780+0000 7f643c08c740 1 --2- 172.21.15.77:0/2677650428 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f560d7d620 0x55f560d89920 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f642000faa0 tx=0x7f6420004730 comp rx=0 tx=0).stop 2024-09-10T10:01:49.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.780+0000 7f643c08c740 1 -- 172.21.15.77:0/2677650428 shutdown_connections 2024-09-10T10:01:49.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.780+0000 7f643c08c740 1 --2- 172.21.15.77:0/2677650428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f560d8c430 0x55f560d89e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:49.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.780+0000 7f643c08c740 1 --2- 172.21.15.77:0/2677650428 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x55f560d9c2b0 0x55f560d9e730 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:49.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.780+0000 7f643c08c740 1 --2- 172.21.15.77:0/2677650428 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f560d7d620 0x55f560d89920 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:49.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.780+0000 7f643c08c740 1 --2- 172.21.15.77:0/2677650428 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f560d7cd00 0x55f560d893e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:49.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.780+0000 7f6413e00640 0 client.16689 ms_handle_reset on v2:172.21.15.134:6836/2461497091 2024-09-10T10:01:49.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.780+0000 7f643c08c740 1 -- 172.21.15.77:0/2677650428 >> 172.21.15.77:0/2677650428 conn(0x55f560d793d0 msgr2=0x55f560d7f130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:49.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.780+0000 7f643c08c740 1 -- 172.21.15.77:0/2677650428 shutdown_connections 2024-09-10T10:01:49.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.780+0000 7f643c08c740 1 -- 172.21.15.77:0/2677650428 wait complete. 2024-09-10T10:01:49.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.780+0000 7f643c08c740 20 client.16689 trim_cache size 0 max 16384 2024-09-10T10:01:49.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.794+0000 7f643c08c740 1 Processor -- start 2024-09-10T10:01:49.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.794+0000 7f643c08c740 1 -- start start 2024-09-10T10:01:49.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.795+0000 7f643c08c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f560d7cd00 0x55f560da4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:49.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.795+0000 7f643c08c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f560d7d620 0x55f560da4540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:49.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.795+0000 7f643c08c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f560d8c430 0x55f560d86640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:49.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.795+0000 7f643c08c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f64341acaf0 con 0x55f560d7d620 2024-09-10T10:01:49.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.795+0000 7f643c08c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f64341ae630 con 0x55f560d7cd00 2024-09-10T10:01:49.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.795+0000 7f643c08c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f64341ad1c0 con 0x55f560d8c430 2024-09-10T10:01:49.797 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.795+0000 7f6432000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f560d7d620 0x55f560da4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:49.797 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.795+0000 7f6432000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f560d7d620 0x55f560da4540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52318/0 (socket says 172.21.15.77:52318) 2024-09-10T10:01:49.797 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.795+0000 7f6432000640 1 -- 172.21.15.77:0/2378966611 learned_addr learned my addr 172.21.15.77:0/2378966611 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:01:49.797 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.795+0000 7f6433400640 1 --2- 172.21.15.77:0/2378966611 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f560d8c430 0x55f560d86640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:49.797 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.795+0000 7f6432a00640 1 --2- 172.21.15.77:0/2378966611 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f560d7cd00 0x55f560da4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:49.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.795+0000 7f6432a00640 1 -- 172.21.15.77:0/2378966611 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f560d8c430 msgr2=0x55f560d86640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:49.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.795+0000 7f6432a00640 1 --2- 172.21.15.77:0/2378966611 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f560d8c430 0x55f560d86640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:49.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.795+0000 7f6432a00640 1 -- 172.21.15.77:0/2378966611 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f560d7d620 msgr2=0x55f560da4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:49.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.795+0000 7f6432a00640 1 --2- 172.21.15.77:0/2378966611 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f560d7d620 0x55f560da4540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:49.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.795+0000 7f6432a00640 1 -- 172.21.15.77:0/2378966611 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f560d79200 con 0x55f560d7cd00 2024-09-10T10:01:49.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.796+0000 7f6432a00640 1 --2- 172.21.15.77:0/2378966611 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f560d7cd00 0x55f560da4000 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x55f560d9ef40 tx=0x7f6424005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24e728c174ad7596 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:49.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.796+0000 7f641ea00640 1 -- 172.21.15.77:0/2378966611 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6424005220 con 0x55f560d7cd00 2024-09-10T10:01:49.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.796+0000 7f641ea00640 1 -- 172.21.15.77:0/2378966611 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6424004c80 con 0x55f560d7cd00 2024-09-10T10:01:49.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.796+0000 7f641ea00640 1 -- 172.21.15.77:0/2378966611 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6424010a20 con 0x55f560d7cd00 2024-09-10T10:01:49.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.797+0000 7f643c08c740 1 -- 172.21.15.77:0/2378966611 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f560d7cd00 msgr2=0x55f560da4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:49.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.797+0000 7f643c08c740 1 --2- 172.21.15.77:0/2378966611 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f560d7cd00 0x55f560da4000 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x55f560d9ef40 tx=0x7f6424005f90 comp rx=0 tx=0).stop 2024-09-10T10:01:49.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.797+0000 7f643c08c740 1 -- 172.21.15.77:0/2378966611 shutdown_connections 2024-09-10T10:01:49.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.797+0000 7f643c08c740 1 --2- 172.21.15.77:0/2378966611 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f560d8c430 0x55f560d86640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:49.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.797+0000 7f643c08c740 1 --2- 172.21.15.77:0/2378966611 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f560d7d620 0x55f560da4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:49.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.797+0000 7f643c08c740 1 --2- 172.21.15.77:0/2378966611 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f560d7cd00 0x55f560da4000 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:49.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.797+0000 7f643c08c740 1 -- 172.21.15.77:0/2378966611 >> 172.21.15.77:0/2378966611 conn(0x55f560d793d0 msgr2=0x55f560d89bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:49.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.797+0000 7f643c08c740 1 -- 172.21.15.77:0/2378966611 shutdown_connections 2024-09-10T10:01:49.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.797+0000 7f643c08c740 1 -- 172.21.15.77:0/2378966611 wait complete. 2024-09-10T10:01:49.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.798+0000 7f643c08c740 1 -- start start 2024-09-10T10:01:49.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.798+0000 7f643c08c740 1 Processor -- start 2024-09-10T10:01:49.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.798+0000 7f643c08c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f560d7cd00 0x55f560d90550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:49.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.798+0000 7f643c08c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f560d7d620 0x55f560d90a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:49.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.798+0000 7f643c08c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f560d8c430 0x55f560e05fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:49.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.798+0000 7f643c08c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f64341adf60 con 0x55f560d8c430 2024-09-10T10:01:49.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.798+0000 7f643c08c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f64341ad890 con 0x55f560d7cd00 2024-09-10T10:01:49.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.798+0000 7f643c08c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f64341aaad0 con 0x55f560d7d620 2024-09-10T10:01:49.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.798+0000 7f643c08c740 10 client.0 _ll_register_callbacks cb 0x7f6420009a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:01:49.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.799+0000 7f6432a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f560d7cd00 0x55f560d90550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:49.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.799+0000 7f6432a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f560d7cd00 0x55f560d90550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:40564/0 (socket says 172.21.15.77:40564) 2024-09-10T10:01:49.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.799+0000 7f6432000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f560d7d620 0x55f560d90a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:49.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.799+0000 7f6432a00640 1 -- 172.21.15.77:0/706662387 learned_addr learned my addr 172.21.15.77:0/706662387 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:01:49.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.799+0000 7f6433400640 1 --2- 172.21.15.77:0/706662387 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f560d8c430 0x55f560e05fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:49.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.799+0000 7f6432a00640 1 -- 172.21.15.77:0/706662387 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f560d7d620 msgr2=0x55f560d90a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:49.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.799+0000 7f6432a00640 1 --2- 172.21.15.77:0/706662387 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f560d7d620 0x55f560d90a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:49.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.799+0000 7f6432a00640 1 -- 172.21.15.77:0/706662387 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f560d8c430 msgr2=0x55f560e05fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:49.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.799+0000 7f6432a00640 1 --2- 172.21.15.77:0/706662387 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f560d8c430 0x55f560e05fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:49.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.799+0000 7f6432a00640 1 -- 172.21.15.77:0/706662387 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f560d7a060 con 0x55f560d7cd00 2024-09-10T10:01:49.804 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.799+0000 7f6432a00640 1 --2- 172.21.15.77:0/706662387 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f560d7cd00 0x55f560d90550 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f6424014010 tx=0x7f64240120b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ff103c6b09a5f5a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:49.804 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.799+0000 7f6413e00640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T10:01:49.804 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.800+0000 7f6413e00640 1 -- 172.21.15.77:0/706662387 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6424005000 con 0x55f560d7cd00 2024-09-10T10:01:49.804 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.800+0000 7f643c08c740 1 -- 172.21.15.77:0/706662387 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6424004c80 con 0x55f560d7cd00 2024-09-10T10:01:49.804 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.800+0000 7f6413e00640 1 -- 172.21.15.77:0/706662387 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f64240051a0 con 0x55f560d7cd00 2024-09-10T10:01:49.804 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.800+0000 7f6413e00640 1 -- 172.21.15.77:0/706662387 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f64240053a0 con 0x55f560d7cd00 2024-09-10T10:01:49.804 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.800+0000 7f643c08c740 10 client.26828 fetch_fsmap learned FSMap version 35 2024-09-10T10:01:49.805 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.800+0000 7f643c08c740 1 -- 172.21.15.77:0/706662387 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55f560e075c0 con 0x55f560d7cd00 2024-09-10T10:01:49.805 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.800+0000 7f6413e00640 1 -- 172.21.15.77:0/706662387 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6424004c80 con 0x55f560d7cd00 2024-09-10T10:01:49.805 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.801+0000 7f6413e00640 1 -- 172.21.15.77:0/706662387 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f64240274c0 con 0x55f560d7cd00 2024-09-10T10:01:49.805 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.801+0000 7f643c08c740 10 client.26828 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T10:01:49.805 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.801+0000 7f643c08c740 10 client.26828 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T10:01:49.805 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.801+0000 7f643c08c740 20 client.26828 populate_metadata read hostname 'smithi077' 2024-09-10T10:01:49.805 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.801+0000 7f643c08c740 1 --2- 172.21.15.77:0/706662387 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x55f560e07cb0 0x55f560e0a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:49.805 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.801+0000 7f643c08c740 4 client.26828 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:01:49.805 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.801+0000 7f643c08c740 1 -- 172.21.15.77:0/706662387 --> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f560ceae60 con 0x55f560e07cb0 2024-09-10T10:01:49.805 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.801+0000 7f6432000640 1 --2- 172.21.15.77:0/706662387 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x55f560e07cb0 0x55f560e0a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:49.805 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.802+0000 7f6432000640 1 --2- 172.21.15.77:0/706662387 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x55f560e07cb0 0x55f560e0a0d0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=bc63f4ce1b2d3da0 server_cookie=966667ec60ca75ae in_seq=0 out_seq=0 2024-09-10T10:01:49.805 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.802+0000 7f6413e00640 10 client.26828 ms_handle_connect on v2:172.21.15.134:6836/2461497091 2024-09-10T10:01:49.805 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.802+0000 7f6413e00640 1 -- 172.21.15.77:0/706662387 <== mds.3 v2:172.21.15.134:6836/2461497091 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f560ceae60 con 0x55f560e07cb0 2024-09-10T10:01:49.805 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.802+0000 7f6413e00640 10 client.26828 handle_command_reply: tid=0 2024-09-10T10:01:49.806 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.802+0000 7f643c08c740 1 client.26828 shutdown 2024-09-10T10:01:49.806 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.802+0000 7f643c08c740 1 -- 172.21.15.77:0/706662387 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f560d7cd00 msgr2=0x55f560d90550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:49.806 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.802+0000 7f643c08c740 1 --2- 172.21.15.77:0/706662387 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f560d7cd00 0x55f560d90550 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f6424014010 tx=0x7f64240120b0 comp rx=0 tx=0).stop 2024-09-10T10:01:49.806 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.802+0000 7f643c08c740 1 -- 172.21.15.77:0/706662387 shutdown_connections 2024-09-10T10:01:49.806 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.802+0000 7f643c08c740 1 --2- 172.21.15.77:0/706662387 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x55f560e07cb0 0x55f560e0a0d0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:49.806 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.802+0000 7f643c08c740 1 --2- 172.21.15.77:0/706662387 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f560d8c430 0x55f560e05fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:49.806 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.802+0000 7f643c08c740 1 --2- 172.21.15.77:0/706662387 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f560d7d620 0x55f560d90a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:49.806 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.802+0000 7f643c08c740 1 --2- 172.21.15.77:0/706662387 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f560d7cd00 0x55f560d90550 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:49.806 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.802+0000 7f6413e00640 0 client.26828 ms_handle_reset on v2:172.21.15.134:6836/2461497091 2024-09-10T10:01:49.806 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.802+0000 7f643c08c740 1 -- 172.21.15.77:0/706662387 >> 172.21.15.77:0/706662387 conn(0x55f560d793d0 msgr2=0x55f560d96f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:49.806 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.802+0000 7f643c08c740 1 -- 172.21.15.77:0/706662387 shutdown_connections 2024-09-10T10:01:49.806 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.802+0000 7f643c08c740 1 -- 172.21.15.77:0/706662387 wait complete. 2024-09-10T10:01:49.806 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.802+0000 7f643c08c740 20 client.26828 trim_cache size 0 max 16384 2024-09-10T10:01:49.807 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-10T10:01:49.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.803+0000 7f6433e00640 1 -- 172.21.15.77:0/4057063249 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f64180a2b80 msgr2=0x7f64180a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:49.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.803+0000 7f6433e00640 1 --2- 172.21.15.77:0/4057063249 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f64180a2b80 0x7f64180a5040 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f64340ffda0 tx=0x7f6420011920 comp rx=0 tx=0).stop 2024-09-10T10:01:49.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.803+0000 7f6433e00640 1 -- 172.21.15.77:0/4057063249 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f64341079c0 msgr2=0x7f643419cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:49.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.803+0000 7f6433e00640 1 --2- 172.21.15.77:0/4057063249 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f64341079c0 0x7f643419cb80 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f642c0305d0 tx=0x7f642c030600 comp rx=0 tx=0).stop 2024-09-10T10:01:49.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.803+0000 7f6433e00640 1 -- 172.21.15.77:0/4057063249 shutdown_connections 2024-09-10T10:01:49.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.803+0000 7f6433e00640 1 --2- 172.21.15.77:0/4057063249 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f64180a2b80 0x7f64180a5040 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:49.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.803+0000 7f6433e00640 1 --2- 172.21.15.77:0/4057063249 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f64341079c0 0x7f643419cb80 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:49.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.803+0000 7f6433e00640 1 --2- 172.21.15.77:0/4057063249 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6434102360 0x7f64341987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:49.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.803+0000 7f6433e00640 1 --2- 172.21.15.77:0/4057063249 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f64341019c0 0x7f64341982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:49.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.803+0000 7f6433e00640 1 -- 172.21.15.77:0/4057063249 >> 172.21.15.77:0/4057063249 conn(0x7f64340fd740 msgr2=0x7f64340fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:49.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.804+0000 7f6433e00640 1 -- 172.21.15.77:0/4057063249 shutdown_connections 2024-09-10T10:01:49.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:49.804+0000 7f6433e00640 1 -- 172.21.15.77:0/4057063249 wait complete. 2024-09-10T10:01:50.731 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T10:01:51.104 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:01:51.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:01:50 smithi120 ceph-mon[34770]: pgmap v591: 129 pgs: 129 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 363 KiB/s wr, 142 op/s 2024-09-10T10:01:51.183 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:01:50 smithi134 ceph-mon[35674]: pgmap v591: 129 pgs: 129 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 363 KiB/s wr, 142 op/s 2024-09-10T10:01:51.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:01:50 smithi077 ceph-mon[30319]: pgmap v591: 129 pgs: 129 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 363 KiB/s wr, 142 op/s 2024-09-10T10:01:51.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.676+0000 7f4fc6e00640 1 -- 172.21.15.77:0/4184767533 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fc0106f10 msgr2=0x7f4fc010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:51.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.676+0000 7f4fc6e00640 1 --2- 172.21.15.77:0/4184767533 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fc0106f10 0x7f4fc010ea90 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f4fb8009ca0 tx=0x7f4fb802a8c0 comp rx=0 tx=0).stop 2024-09-10T10:01:51.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.676+0000 7f4fc6e00640 1 -- 172.21.15.77:0/4184767533 shutdown_connections 2024-09-10T10:01:51.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.676+0000 7f4fc6e00640 1 --2- 172.21.15.77:0/4184767533 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fc0106f10 0x7f4fc010ea90 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:51.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.676+0000 7f4fc6e00640 1 --2- 172.21.15.77:0/4184767533 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4fc0102790 0x7f4fc01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:51.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.676+0000 7f4fc6e00640 1 --2- 172.21.15.77:0/4184767533 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4fc0101de0 0x7f4fc01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:51.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.676+0000 7f4fc6e00640 1 -- 172.21.15.77:0/4184767533 >> 172.21.15.77:0/4184767533 conn(0x7f4fc00fd8c0 msgr2=0x7f4fc00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:51.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.676+0000 7f4fc6e00640 1 -- 172.21.15.77:0/4184767533 shutdown_connections 2024-09-10T10:01:51.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.676+0000 7f4fc6e00640 1 -- 172.21.15.77:0/4184767533 wait complete. 2024-09-10T10:01:51.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.677+0000 7f4fc6e00640 1 Processor -- start 2024-09-10T10:01:51.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.677+0000 7f4fc6e00640 1 -- start start 2024-09-10T10:01:51.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.677+0000 7f4fc6e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fc0101de0 0x7f4fc019a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:51.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.678+0000 7f4fc6e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4fc0102790 0x7f4fc019aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:51.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.678+0000 7f4fc6e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4fc0106f10 0x7f4fc019ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:51.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.678+0000 7f4fc6e00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc0111fc0 con 0x7f4fc0101de0 2024-09-10T10:01:51.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.678+0000 7f4fc6e00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc0111e40 con 0x7f4fc0106f10 2024-09-10T10:01:51.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.678+0000 7f4fc6e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc0112140 con 0x7f4fc0102790 2024-09-10T10:01:51.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.678+0000 7f4fbfe00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fc0101de0 0x7f4fc019a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:51.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.678+0000 7f4fbfe00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fc0101de0 0x7f4fc019a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52334/0 (socket says 172.21.15.77:52334) 2024-09-10T10:01:51.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.678+0000 7f4fbfe00640 1 -- 172.21.15.77:0/1658116795 learned_addr learned my addr 172.21.15.77:0/1658116795 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:01:51.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.678+0000 7f4fb7400640 1 --2- 172.21.15.77:0/1658116795 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4fc0102790 0x7f4fc019aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:51.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.678+0000 7f4fc4e00640 1 --2- 172.21.15.77:0/1658116795 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4fc0106f10 0x7f4fc019ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:51.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.678+0000 7f4fbfe00640 1 -- 172.21.15.77:0/1658116795 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4fc0102790 msgr2=0x7f4fc019aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:51.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.678+0000 7f4fbfe00640 1 --2- 172.21.15.77:0/1658116795 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4fc0102790 0x7f4fc019aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:51.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.678+0000 7f4fbfe00640 1 -- 172.21.15.77:0/1658116795 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4fc0106f10 msgr2=0x7f4fc019ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:51.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.678+0000 7f4fbfe00640 1 --2- 172.21.15.77:0/1658116795 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4fc0106f10 0x7f4fc019ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:51.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.678+0000 7f4fbfe00640 1 -- 172.21.15.77:0/1658116795 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fc019f610 con 0x7f4fc0101de0 2024-09-10T10:01:51.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.678+0000 7f4fb7400640 1 --2- 172.21.15.77:0/1658116795 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4fc0102790 0x7f4fc019aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:01:51.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.678+0000 7f4fc4e00640 1 --2- 172.21.15.77:0/1658116795 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4fc0106f10 0x7f4fc019ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:01:51.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.678+0000 7f4fbfe00640 1 --2- 172.21.15.77:0/1658116795 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fc0101de0 0x7f4fc019a660 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f4fb000e9f0 tx=0x7f4fb000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:51.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.679+0000 7f4fbd600640 1 -- 172.21.15.77:0/1658116795 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb000cd80 con 0x7f4fc0101de0 2024-09-10T10:01:51.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.679+0000 7f4fbd600640 1 -- 172.21.15.77:0/1658116795 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fb0010910 con 0x7f4fc0101de0 2024-09-10T10:01:51.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.679+0000 7f4fc6e00640 1 -- 172.21.15.77:0/1658116795 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fc019f900 con 0x7f4fc0101de0 2024-09-10T10:01:51.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.679+0000 7f4fbd600640 1 -- 172.21.15.77:0/1658116795 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb0018990 con 0x7f4fc0101de0 2024-09-10T10:01:51.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.679+0000 7f4fc6e00640 1 -- 172.21.15.77:0/1658116795 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fc01a71e0 con 0x7f4fc0101de0 2024-09-10T10:01:51.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.682+0000 7f4fbd600640 1 -- 172.21.15.77:0/1658116795 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4fb0010ac0 con 0x7f4fc0101de0 2024-09-10T10:01:51.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.682+0000 7f4fc9a52740 1 Processor -- start 2024-09-10T10:01:51.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.682+0000 7f4fc9a52740 1 -- start start 2024-09-10T10:01:51.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.683+0000 7f4fc9a52740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcda8e8c70 0x55dcda8e9050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:51.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.683+0000 7f4fc9a52740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dcda8e9590 0x55dcda8f3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:51.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.683+0000 7f4fc9a52740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dcda8f3e50 0x55dcda8f6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:51.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.683+0000 7f4fc9a52740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55dcda84f3e0 con 0x55dcda8f3e50 2024-09-10T10:01:51.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.683+0000 7f4fc9a52740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55dcda7f6d80 con 0x55dcda8e9590 2024-09-10T10:01:51.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.683+0000 7f4fc9a52740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55dcda84b3a0 con 0x55dcda8e8c70 2024-09-10T10:01:51.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.683+0000 7f4fbfe00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dcda8e9590 0x55dcda8f3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:51.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.683+0000 7f4fc4e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcda8e8c70 0x55dcda8e9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:51.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.683+0000 7f4fbfe00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dcda8e9590 0x55dcda8f3910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:40582/0 (socket says 172.21.15.77:40582) 2024-09-10T10:01:51.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.683+0000 7f4fbfe00640 1 -- 172.21.15.77:0/23684412 learned_addr learned my addr 172.21.15.77:0/23684412 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:01:51.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.683+0000 7f4fb7400640 1 --2- 172.21.15.77:0/23684412 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dcda8f3e50 0x55dcda8f6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:51.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.683+0000 7f4fc4e00640 1 -- 172.21.15.77:0/23684412 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dcda8e9590 msgr2=0x55dcda8f3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:51.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.683+0000 7f4fc4e00640 1 --2- 172.21.15.77:0/23684412 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dcda8e9590 0x55dcda8f3910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:51.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.683+0000 7f4fc4e00640 1 -- 172.21.15.77:0/23684412 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dcda8f3e50 msgr2=0x55dcda8f6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:51.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.683+0000 7f4fc4e00640 1 --2- 172.21.15.77:0/23684412 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dcda8f3e50 0x55dcda8f6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:51.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.684+0000 7f4fc4e00640 1 -- 172.21.15.77:0/23684412 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcda815ae0 con 0x55dcda8e8c70 2024-09-10T10:01:51.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.684+0000 7f4fbd600640 1 --2- 172.21.15.77:0/1658116795 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f4fa40a2b80 0x7f4fa40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:51.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.684+0000 7f4fc4e00640 1 --2- 172.21.15.77:0/23684412 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcda8e8c70 0x55dcda8e9050 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f4fb8006010 tx=0x7f4fb8009800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a4c7fe2bf8f83d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:51.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.684+0000 7f4fbd600640 1 -- 172.21.15.77:0/1658116795 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f4fb00cf900 con 0x7f4fc0101de0 2024-09-10T10:01:51.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.684+0000 7f4fbfe00640 1 --2- 172.21.15.77:0/1658116795 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f4fa40a2b80 0x7f4fa40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:51.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.684+0000 7f4fb5600640 1 -- 172.21.15.77:0/23684412 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb8031db0 con 0x55dcda8e8c70 2024-09-10T10:01:51.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.684+0000 7f4fb5600640 1 -- 172.21.15.77:0/23684412 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fb8036040 con 0x55dcda8e8c70 2024-09-10T10:01:51.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.684+0000 7f4fb5600640 1 -- 172.21.15.77:0/23684412 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb80042a0 con 0x55dcda8e8c70 2024-09-10T10:01:51.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.685+0000 7f4fc9a52740 1 -- 172.21.15.77:0/23684412 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcda8e8c70 msgr2=0x55dcda8e9050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:51.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.685+0000 7f4fbfe00640 1 --2- 172.21.15.77:0/1658116795 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f4fa40a2b80 0x7f4fa40a5040 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f4fb00cfba0 tx=0x7f4fb00d5530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:51.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.685+0000 7f4fc9a52740 1 --2- 172.21.15.77:0/23684412 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcda8e8c70 0x55dcda8e9050 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f4fb8006010 tx=0x7f4fb8009800 comp rx=0 tx=0).stop 2024-09-10T10:01:51.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.685+0000 7f4fc9a52740 1 -- 172.21.15.77:0/23684412 shutdown_connections 2024-09-10T10:01:51.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.685+0000 7f4fc9a52740 1 --2- 172.21.15.77:0/23684412 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dcda8f3e50 0x55dcda8f6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:51.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.685+0000 7f4fc9a52740 1 --2- 172.21.15.77:0/23684412 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dcda8e9590 0x55dcda8f3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:51.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.685+0000 7f4fc9a52740 1 --2- 172.21.15.77:0/23684412 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcda8e8c70 0x55dcda8e9050 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:51.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.685+0000 7f4fc9a52740 1 -- 172.21.15.77:0/23684412 >> 172.21.15.77:0/23684412 conn(0x55dcda8e5200 msgr2=0x55dcda8e55e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:51.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.685+0000 7f4fc9a52740 1 -- 172.21.15.77:0/23684412 shutdown_connections 2024-09-10T10:01:51.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.685+0000 7f4fc9a52740 1 -- 172.21.15.77:0/23684412 wait complete. 2024-09-10T10:01:51.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.686+0000 7f4fc9a52740 1 -- start start 2024-09-10T10:01:51.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.686+0000 7f4fc9a52740 1 Processor -- start 2024-09-10T10:01:51.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.686+0000 7f4fc9a52740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcda8e8c70 0x55dcda8f0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:51.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.686+0000 7f4fc9a52740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dcda8e9590 0x55dcda8f1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:51.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.686+0000 7f4fc9a52740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dcda8f3e50 0x55dcda8f1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:51.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.686+0000 7f4fc9a52740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55dcda8f6a20 con 0x55dcda8f3e50 2024-09-10T10:01:51.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.686+0000 7f4fc9a52740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55dcda80a0f0 con 0x55dcda8e9590 2024-09-10T10:01:51.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.686+0000 7f4fc9a52740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55dcda8f6ba0 con 0x55dcda8e8c70 2024-09-10T10:01:51.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.687+0000 7f4fc9a52740 10 client.0 _ll_register_callbacks cb 0x55dcda7f1bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:01:51.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.687+0000 7f4fc4e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcda8e8c70 0x55dcda8f0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:51.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.687+0000 7f4fb7400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dcda8e9590 0x55dcda8f1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:51.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.687+0000 7f4fb7400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dcda8e9590 0x55dcda8f1220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:40586/0 (socket says 172.21.15.77:40586) 2024-09-10T10:01:51.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.687+0000 7f4fc4e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcda8e8c70 0x55dcda8f0ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:46544/0 (socket says 172.21.15.77:46544) 2024-09-10T10:01:51.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.687+0000 7f4fb7400640 1 -- 172.21.15.77:0/909028231 learned_addr learned my addr 172.21.15.77:0/909028231 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:01:51.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.687+0000 7f4fc4e00640 1 --2- 172.21.15.77:0/909028231 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dcda8f3e50 0x55dcda8f1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:51.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.687+0000 7f4fb7400640 1 -- 172.21.15.77:0/909028231 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcda8e8c70 msgr2=0x55dcda8f0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:51.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.687+0000 7f4fb7400640 1 --2- 172.21.15.77:0/909028231 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcda8e8c70 0x55dcda8f0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:51.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.687+0000 7f4fb7400640 1 -- 172.21.15.77:0/909028231 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dcda8f3e50 msgr2=0x55dcda8f1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:51.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.687+0000 7f4fb7400640 1 --2- 172.21.15.77:0/909028231 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dcda8f3e50 0x55dcda8f1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:51.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.687+0000 7f4fb7400640 1 -- 172.21.15.77:0/909028231 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcda5c03c0 con 0x55dcda8e9590 2024-09-10T10:01:51.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.688+0000 7f4fb7400640 1 --2- 172.21.15.77:0/909028231 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dcda8e9590 0x55dcda8f1220 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4fa80126b0 tx=0x7f4fa8012b70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7c4f31687562568 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:51.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.688+0000 7f4fae000640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T10:01:51.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.688+0000 7f4fae000640 1 -- 172.21.15.77:0/909028231 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fa8011bb0 con 0x55dcda8e9590 2024-09-10T10:01:51.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.688+0000 7f4fc9a52740 1 -- 172.21.15.77:0/909028231 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fb8036040 con 0x55dcda8e9590 2024-09-10T10:01:51.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.688+0000 7f4fae000640 1 -- 172.21.15.77:0/909028231 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f4fa8011d50 con 0x55dcda8e9590 2024-09-10T10:01:51.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.688+0000 7f4fae000640 1 -- 172.21.15.77:0/909028231 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f4fa8005020 con 0x55dcda8e9590 2024-09-10T10:01:51.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.688+0000 7f4fc9a52740 10 client.26840 fetch_fsmap learned FSMap version 35 2024-09-10T10:01:51.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.688+0000 7f4fc9a52740 1 -- 172.21.15.77:0/909028231 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55dcda902aa0 con 0x55dcda8e9590 2024-09-10T10:01:51.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.689+0000 7f4fae000640 1 -- 172.21.15.77:0/909028231 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fb8036040 con 0x55dcda8e9590 2024-09-10T10:01:51.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.689+0000 7f4fae000640 1 -- 172.21.15.77:0/909028231 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f4fa801c470 con 0x55dcda8e9590 2024-09-10T10:01:51.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.689+0000 7f4fc9a52740 10 client.26840 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T10:01:51.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.689+0000 7f4fc9a52740 10 client.26840 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T10:01:51.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.689+0000 7f4fc9a52740 20 client.26840 populate_metadata read hostname 'smithi077' 2024-09-10T10:01:51.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.689+0000 7f4fc9a52740 1 --2- 172.21.15.77:0/909028231 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x55dcda903f00 0x55dcda906320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:51.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.689+0000 7f4fc9a52740 4 client.26840 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:01:51.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.689+0000 7f4fc9a52740 1 -- 172.21.15.77:0/909028231 --> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dcda85c950 con 0x55dcda903f00 2024-09-10T10:01:51.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.689+0000 7f4fc4e00640 1 --2- 172.21.15.77:0/909028231 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x55dcda903f00 0x55dcda906320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:51.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.690+0000 7f4fc4e00640 1 --2- 172.21.15.77:0/909028231 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x55dcda903f00 0x55dcda906320 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bf367869a55fddf8 server_cookie=c516730af55f3cd5 in_seq=0 out_seq=0 2024-09-10T10:01:51.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.690+0000 7f4fae000640 10 client.26840 ms_handle_connect on v2:172.21.15.134:6838/1965117680 2024-09-10T10:01:51.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.691+0000 7f4fae000640 1 -- 172.21.15.77:0/909028231 <== mds.2 v2:172.21.15.134:6838/1965117680 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dcda85c950 con 0x55dcda903f00 2024-09-10T10:01:51.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.691+0000 7f4fae000640 10 client.26840 handle_command_reply: tid=0 2024-09-10T10:01:51.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.691+0000 7f4fc9a52740 1 client.26840 shutdown 2024-09-10T10:01:51.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.691+0000 7f4fc9a52740 1 -- 172.21.15.77:0/909028231 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dcda8e9590 msgr2=0x55dcda8f1220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:51.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.691+0000 7f4fc9a52740 1 --2- 172.21.15.77:0/909028231 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dcda8e9590 0x55dcda8f1220 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4fa80126b0 tx=0x7f4fa8012b70 comp rx=0 tx=0).stop 2024-09-10T10:01:51.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.691+0000 7f4fc9a52740 1 -- 172.21.15.77:0/909028231 shutdown_connections 2024-09-10T10:01:51.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.691+0000 7f4fc9a52740 1 --2- 172.21.15.77:0/909028231 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x55dcda903f00 0x55dcda906320 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:51.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.691+0000 7f4fc9a52740 1 --2- 172.21.15.77:0/909028231 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dcda8f3e50 0x55dcda8f1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:51.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.691+0000 7f4fc9a52740 1 --2- 172.21.15.77:0/909028231 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dcda8e9590 0x55dcda8f1220 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:51.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.691+0000 7f4fc9a52740 1 --2- 172.21.15.77:0/909028231 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcda8e8c70 0x55dcda8f0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:51.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.691+0000 7f4fc9a52740 1 -- 172.21.15.77:0/909028231 >> 172.21.15.77:0/909028231 conn(0x55dcda8e5200 msgr2=0x55dcda8e4380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:51.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.691+0000 7f4fae000640 0 client.26840 ms_handle_reset on v2:172.21.15.134:6838/1965117680 2024-09-10T10:01:51.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.691+0000 7f4fc9a52740 1 -- 172.21.15.77:0/909028231 shutdown_connections 2024-09-10T10:01:51.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.691+0000 7f4fc9a52740 1 -- 172.21.15.77:0/909028231 wait complete. 2024-09-10T10:01:51.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.691+0000 7f4fc9a52740 20 client.26840 trim_cache size 0 max 16384 2024-09-10T10:01:51.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.705+0000 7f4fc9a52740 1 Processor -- start 2024-09-10T10:01:51.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.705+0000 7f4fc9a52740 1 -- start start 2024-09-10T10:01:51.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.705+0000 7f4fc9a52740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dcda8e8c70 0x55dcda8f15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:51.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.705+0000 7f4fc9a52740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcda8e9590 0x55dcda90bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:51.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.705+0000 7f4fc9a52740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dcda8f3e50 0x55dcda90c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:51.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.705+0000 7f4fc9a52740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc01b10e0 con 0x55dcda8f3e50 2024-09-10T10:01:51.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.705+0000 7f4fc9a52740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc01b2c20 con 0x55dcda8e8c70 2024-09-10T10:01:51.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.705+0000 7f4fc9a52740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc01b17b0 con 0x55dcda8e9590 2024-09-10T10:01:51.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.706+0000 7f4fc4e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dcda8f3e50 0x55dcda90c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:51.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.706+0000 7f4fb7400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcda8e9590 0x55dcda90bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:51.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.706+0000 7f4fc4e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dcda8e8c70 0x55dcda8f15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:51.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.706+0000 7f4fb7400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcda8e9590 0x55dcda90bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:46556/0 (socket says 172.21.15.77:46556) 2024-09-10T10:01:51.708 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.706+0000 7f4fb7400640 1 -- 172.21.15.77:0/1842380881 learned_addr learned my addr 172.21.15.77:0/1842380881 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:01:51.708 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.706+0000 7f4fc4e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dcda8e8c70 0x55dcda8f15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:40598/0 (socket says 172.21.15.77:40598) 2024-09-10T10:01:51.708 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.706+0000 7f4fc4e00640 1 -- 172.21.15.77:0/1842380881 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcda8e9590 msgr2=0x55dcda90bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:51.708 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.706+0000 7f4fc4e00640 1 --2- 172.21.15.77:0/1842380881 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcda8e9590 0x55dcda90bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:51.708 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.706+0000 7f4fc4e00640 1 -- 172.21.15.77:0/1842380881 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dcda8f3e50 msgr2=0x55dcda90c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:51.708 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.706+0000 7f4fc4e00640 1 --2- 172.21.15.77:0/1842380881 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dcda8f3e50 0x55dcda90c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:51.708 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.706+0000 7f4fc4e00640 1 -- 172.21.15.77:0/1842380881 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcda906980 con 0x55dcda8e8c70 2024-09-10T10:01:51.708 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.706+0000 7f4fc4e00640 1 --2- 172.21.15.77:0/1842380881 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dcda8e8c70 0x55dcda8f15b0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f4fb8031a70 tx=0x7f4fb8031970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50a16cb8694d3023 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:51.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.707+0000 7f4fb4c00640 1 -- 172.21.15.77:0/1842380881 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb8031530 con 0x55dcda8e8c70 2024-09-10T10:01:51.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.707+0000 7f4fb4c00640 1 -- 172.21.15.77:0/1842380881 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fb8036040 con 0x55dcda8e8c70 2024-09-10T10:01:51.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.707+0000 7f4fb4c00640 1 -- 172.21.15.77:0/1842380881 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb8004540 con 0x55dcda8e8c70 2024-09-10T10:01:51.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.707+0000 7f4fc9a52740 1 -- 172.21.15.77:0/1842380881 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dcda8e8c70 msgr2=0x55dcda8f15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:51.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.707+0000 7f4fc9a52740 1 --2- 172.21.15.77:0/1842380881 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dcda8e8c70 0x55dcda8f15b0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f4fb8031a70 tx=0x7f4fb8031970 comp rx=0 tx=0).stop 2024-09-10T10:01:51.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.707+0000 7f4fc9a52740 1 -- 172.21.15.77:0/1842380881 shutdown_connections 2024-09-10T10:01:51.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.707+0000 7f4fc9a52740 1 --2- 172.21.15.77:0/1842380881 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dcda8f3e50 0x55dcda90c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:51.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.707+0000 7f4fc9a52740 1 --2- 172.21.15.77:0/1842380881 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcda8e9590 0x55dcda90bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:51.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.707+0000 7f4fc9a52740 1 --2- 172.21.15.77:0/1842380881 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dcda8e8c70 0x55dcda8f15b0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:51.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.707+0000 7f4fc9a52740 1 -- 172.21.15.77:0/1842380881 >> 172.21.15.77:0/1842380881 conn(0x55dcda8e5200 msgr2=0x55dcda8f2d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:51.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.707+0000 7f4fc9a52740 1 -- 172.21.15.77:0/1842380881 shutdown_connections 2024-09-10T10:01:51.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.707+0000 7f4fc9a52740 1 -- 172.21.15.77:0/1842380881 wait complete. 2024-09-10T10:01:51.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.708+0000 7f4fc9a52740 1 -- start start 2024-09-10T10:01:51.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.708+0000 7f4fc9a52740 1 Processor -- start 2024-09-10T10:01:51.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.709+0000 7f4fc9a52740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dcda8e8c70 0x55dcda8f7e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:51.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.709+0000 7f4fc9a52740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcda8e9590 0x55dcda8f83a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:51.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.709+0000 7f4fc9a52740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dcda8f3e50 0x55dcda8f8920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:51.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.709+0000 7f4fc9a52740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc01b2550 con 0x55dcda8f3e50 2024-09-10T10:01:51.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.709+0000 7f4fc9a52740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc01b1e80 con 0x55dcda8e8c70 2024-09-10T10:01:51.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.709+0000 7f4fc9a52740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc01af0c0 con 0x55dcda8e9590 2024-09-10T10:01:51.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.709+0000 7f4fc9a52740 10 client.0 _ll_register_callbacks cb 0x7f4fa80097e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:01:51.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.709+0000 7f4fb7400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcda8e9590 0x55dcda8f83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:51.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.709+0000 7f4fb7400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcda8e9590 0x55dcda8f83a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:46572/0 (socket says 172.21.15.77:46572) 2024-09-10T10:01:51.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.709+0000 7f4fb7400640 1 -- 172.21.15.77:0/796941883 learned_addr learned my addr 172.21.15.77:0/796941883 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:01:51.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.709+0000 7f4fc4e00640 1 --2- 172.21.15.77:0/796941883 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dcda8e8c70 0x55dcda8f7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:51.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.709+0000 7f4fc4e00640 1 --2- 172.21.15.77:0/796941883 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dcda8f3e50 0x55dcda8f8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:51.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.709+0000 7f4fb7400640 1 -- 172.21.15.77:0/796941883 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dcda8e8c70 msgr2=0x55dcda8f7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:51.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.709+0000 7f4fb7400640 1 --2- 172.21.15.77:0/796941883 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dcda8e8c70 0x55dcda8f7e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:51.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.709+0000 7f4fb7400640 1 -- 172.21.15.77:0/796941883 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dcda8f3e50 msgr2=0x55dcda8f8920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:51.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.709+0000 7f4fb7400640 1 --2- 172.21.15.77:0/796941883 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dcda8f3e50 0x55dcda8f8920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:51.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.709+0000 7f4fb7400640 1 -- 172.21.15.77:0/796941883 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcda8ebae0 con 0x55dcda8e9590 2024-09-10T10:01:51.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.710+0000 7f4fb7400640 1 --2- 172.21.15.77:0/796941883 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcda8e9590 0x55dcda8f83a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f4fa801ca80 tx=0x7f4fa801cd30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3cf566073250e9d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:51.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.710+0000 7f4fae000640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-10T10:01:51.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.710+0000 7f4fae000640 1 -- 172.21.15.77:0/796941883 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fa8011dc0 con 0x55dcda8e9590 2024-09-10T10:01:51.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.710+0000 7f4fc9a52740 1 -- 172.21.15.77:0/796941883 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fb8036040 con 0x55dcda8e9590 2024-09-10T10:01:51.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.710+0000 7f4fae000640 1 -- 172.21.15.77:0/796941883 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f4fa80051c0 con 0x55dcda8e9590 2024-09-10T10:01:51.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.710+0000 7f4fae000640 1 -- 172.21.15.77:0/796941883 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f4fa80053c0 con 0x55dcda8e9590 2024-09-10T10:01:51.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.710+0000 7f4fc9a52740 10 client.26788 fetch_fsmap learned FSMap version 35 2024-09-10T10:01:51.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.710+0000 7f4fc9a52740 1 -- 172.21.15.77:0/796941883 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55dcda8e5e90 con 0x55dcda8e9590 2024-09-10T10:01:51.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.711+0000 7f4fae000640 1 -- 172.21.15.77:0/796941883 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fb8036040 con 0x55dcda8e9590 2024-09-10T10:01:51.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.711+0000 7f4fae000640 1 -- 172.21.15.77:0/796941883 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f4fa8029420 con 0x55dcda8e9590 2024-09-10T10:01:51.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.711+0000 7f4fc9a52740 10 client.26788 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T10:01:51.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.711+0000 7f4fc9a52740 10 client.26788 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T10:01:51.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.711+0000 7f4fc9a52740 20 client.26788 populate_metadata read hostname 'smithi077' 2024-09-10T10:01:51.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.711+0000 7f4fc9a52740 1 --2- 172.21.15.77:0/796941883 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x55dcda96f4f0 0x55dcda971910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:51.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.711+0000 7f4fc9a52740 4 client.26788 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:01:51.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.711+0000 7f4fc9a52740 1 -- 172.21.15.77:0/796941883 --> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dcda8ec900 con 0x55dcda96f4f0 2024-09-10T10:01:51.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.711+0000 7f4fc4e00640 1 --2- 172.21.15.77:0/796941883 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x55dcda96f4f0 0x55dcda971910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:51.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.712+0000 7f4fc4e00640 1 --2- 172.21.15.77:0/796941883 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x55dcda96f4f0 0x55dcda971910 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7420bd4a52559666 server_cookie=fab1ef1284c96622 in_seq=0 out_seq=0 2024-09-10T10:01:51.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.712+0000 7f4fae000640 10 client.26788 ms_handle_connect on v2:172.21.15.134:6838/1965117680 2024-09-10T10:01:51.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.712+0000 7f4fae000640 1 -- 172.21.15.77:0/796941883 <== mds.2 v2:172.21.15.134:6838/1965117680 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dcda8ec900 con 0x55dcda96f4f0 2024-09-10T10:01:51.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.712+0000 7f4fae000640 10 client.26788 handle_command_reply: tid=0 2024-09-10T10:01:51.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.712+0000 7f4fc9a52740 1 client.26788 shutdown 2024-09-10T10:01:51.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.712+0000 7f4fc9a52740 1 -- 172.21.15.77:0/796941883 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcda8e9590 msgr2=0x55dcda8f83a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:51.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.712+0000 7f4fc9a52740 1 --2- 172.21.15.77:0/796941883 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcda8e9590 0x55dcda8f83a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f4fa801ca80 tx=0x7f4fa801cd30 comp rx=0 tx=0).stop 2024-09-10T10:01:51.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.712+0000 7f4fc9a52740 1 -- 172.21.15.77:0/796941883 shutdown_connections 2024-09-10T10:01:51.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.712+0000 7f4fc9a52740 1 --2- 172.21.15.77:0/796941883 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x55dcda96f4f0 0x55dcda971910 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:51.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.712+0000 7f4fc9a52740 1 --2- 172.21.15.77:0/796941883 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dcda8f3e50 0x55dcda8f8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:51.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.712+0000 7f4fc9a52740 1 --2- 172.21.15.77:0/796941883 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcda8e9590 0x55dcda8f83a0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:51.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.712+0000 7f4fae000640 0 client.26788 ms_handle_reset on v2:172.21.15.134:6838/1965117680 2024-09-10T10:01:51.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.712+0000 7f4fc9a52740 1 --2- 172.21.15.77:0/796941883 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dcda8e8c70 0x55dcda8f7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:51.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.712+0000 7f4fc9a52740 1 -- 172.21.15.77:0/796941883 >> 172.21.15.77:0/796941883 conn(0x55dcda8e5200 msgr2=0x55dcda8e3b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:51.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.712+0000 7f4fc9a52740 1 -- 172.21.15.77:0/796941883 shutdown_connections 2024-09-10T10:01:51.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.712+0000 7f4fc9a52740 1 -- 172.21.15.77:0/796941883 wait complete. 2024-09-10T10:01:51.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.712+0000 7f4fc9a52740 20 client.26788 trim_cache size 0 max 16384 2024-09-10T10:01:51.717 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-10T10:01:51.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.713+0000 7f4fc6e00640 1 -- 172.21.15.77:0/1658116795 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f4fa40a2b80 msgr2=0x7f4fa40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:51.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.713+0000 7f4fc6e00640 1 --2- 172.21.15.77:0/1658116795 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f4fa40a2b80 0x7f4fa40a5040 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f4fb00cfba0 tx=0x7f4fb00d5530 comp rx=0 tx=0).stop 2024-09-10T10:01:51.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.713+0000 7f4fc6e00640 1 -- 172.21.15.77:0/1658116795 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fc0101de0 msgr2=0x7f4fc019a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:51.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.713+0000 7f4fc6e00640 1 --2- 172.21.15.77:0/1658116795 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fc0101de0 0x7f4fc019a660 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f4fb000e9f0 tx=0x7f4fb000eeb0 comp rx=0 tx=0).stop 2024-09-10T10:01:51.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.713+0000 7f4fc6e00640 1 -- 172.21.15.77:0/1658116795 shutdown_connections 2024-09-10T10:01:51.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.713+0000 7f4fc6e00640 1 --2- 172.21.15.77:0/1658116795 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f4fa40a2b80 0x7f4fa40a5040 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:51.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.713+0000 7f4fc6e00640 1 --2- 172.21.15.77:0/1658116795 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4fc0106f10 0x7f4fc019ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:51.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.713+0000 7f4fc6e00640 1 --2- 172.21.15.77:0/1658116795 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4fc0102790 0x7f4fc019aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:51.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.713+0000 7f4fc6e00640 1 --2- 172.21.15.77:0/1658116795 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fc0101de0 0x7f4fc019a660 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:51.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.713+0000 7f4fc6e00640 1 -- 172.21.15.77:0/1658116795 >> 172.21.15.77:0/1658116795 conn(0x7f4fc00fd8c0 msgr2=0x7f4fc00fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:51.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.713+0000 7f4fc6e00640 1 -- 172.21.15.77:0/1658116795 shutdown_connections 2024-09-10T10:01:51.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:51.714+0000 7f4fc6e00640 1 -- 172.21.15.77:0/1658116795 wait complete. 2024-09-10T10:01:52.566 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T10:01:52.935 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:01:53.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:01:52 smithi120 ceph-mon[34770]: pgmap v592: 129 pgs: 129 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 54 KiB/s wr, 20 op/s 2024-09-10T10:01:53.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:01:52 smithi077 ceph-mon[30319]: pgmap v592: 129 pgs: 129 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 54 KiB/s wr, 20 op/s 2024-09-10T10:01:53.252 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:01:53.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:01:52 smithi134 ceph-mon[35674]: pgmap v592: 129 pgs: 129 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 54 KiB/s wr, 20 op/s 2024-09-10T10:01:53.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.540+0000 7f96c8a00640 1 -- 172.21.15.77:0/1571840299 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f96c41079d0 msgr2=0x7f96c4107db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:53.543 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.540+0000 7f96c8a00640 1 --2- 172.21.15.77:0/1571840299 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f96c41079d0 0x7f96c4107db0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f96b4009930 tx=0x7f96b402a420 comp rx=0 tx=0).stop 2024-09-10T10:01:53.543 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.540+0000 7f96c8a00640 1 -- 172.21.15.77:0/1571840299 shutdown_connections 2024-09-10T10:01:53.543 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.540+0000 7f96c8a00640 1 --2- 172.21.15.77:0/1571840299 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f96c4102370 0x7f96c410e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:53.543 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.540+0000 7f96c8a00640 1 --2- 172.21.15.77:0/1571840299 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f96c41019d0 0x7f96c4101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:53.543 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.540+0000 7f96c8a00640 1 --2- 172.21.15.77:0/1571840299 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f96c41079d0 0x7f96c4107db0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:53.543 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.541+0000 7f96c8a00640 1 -- 172.21.15.77:0/1571840299 >> 172.21.15.77:0/1571840299 conn(0x7f96c40fd710 msgr2=0x7f96c40ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:53.543 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.541+0000 7f96c8a00640 1 -- 172.21.15.77:0/1571840299 shutdown_connections 2024-09-10T10:01:53.543 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.541+0000 7f96c8a00640 1 -- 172.21.15.77:0/1571840299 wait complete. 2024-09-10T10:01:53.543 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.542+0000 7f96c8a00640 1 Processor -- start 2024-09-10T10:01:53.543 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.542+0000 7f96c8a00640 1 -- start start 2024-09-10T10:01:53.545 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.542+0000 7f96c8a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f96c41019d0 0x7f96c40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:53.545 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.542+0000 7f96c8a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f96c4102370 0x7f96c4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:53.545 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.542+0000 7f96c8a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f96c41079d0 0x7f96c4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:53.545 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.542+0000 7f96c8a00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f96c4111de0 con 0x7f96c41079d0 2024-09-10T10:01:53.545 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.542+0000 7f96c8a00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f96c4111c60 con 0x7f96c4102370 2024-09-10T10:01:53.545 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.542+0000 7f96c8a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f96c4111f60 con 0x7f96c41019d0 2024-09-10T10:01:53.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.543+0000 7f96c3e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f96c41079d0 0x7f96c4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:53.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.543+0000 7f96c2a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f96c4102370 0x7f96c4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:53.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.543+0000 7f96c3e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f96c41079d0 0x7f96c4074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:51344/0 (socket says 172.21.15.77:51344) 2024-09-10T10:01:53.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.543+0000 7f96c3e00640 1 -- 172.21.15.77:0/2263064917 learned_addr learned my addr 172.21.15.77:0/2263064917 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:01:53.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.543+0000 7f96c3400640 1 --2- 172.21.15.77:0/2263064917 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f96c41019d0 0x7f96c40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:53.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.543+0000 7f96c3e00640 1 -- 172.21.15.77:0/2263064917 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f96c41019d0 msgr2=0x7f96c40744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:53.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.543+0000 7f96c3e00640 1 --2- 172.21.15.77:0/2263064917 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f96c41019d0 0x7f96c40744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:53.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.543+0000 7f96c3e00640 1 -- 172.21.15.77:0/2263064917 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f96c4102370 msgr2=0x7f96c4074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:53.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.543+0000 7f96c3e00640 1 --2- 172.21.15.77:0/2263064917 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f96c4102370 0x7f96c4074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:53.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.543+0000 7f96c3e00640 1 -- 172.21.15.77:0/2263064917 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96c41a6750 con 0x7f96c41079d0 2024-09-10T10:01:53.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.543+0000 7f96c2a00640 1 --2- 172.21.15.77:0/2263064917 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f96c4102370 0x7f96c4074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:01:53.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.544+0000 7f96c3e00640 1 --2- 172.21.15.77:0/2263064917 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f96c41079d0 0x7f96c4074f70 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f96bc00eaa0 tx=0x7f96bc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:53.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.544+0000 7f96bbe00640 1 -- 172.21.15.77:0/2263064917 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96bc00cdb0 con 0x7f96c41079d0 2024-09-10T10:01:53.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.544+0000 7f96bbe00640 1 -- 172.21.15.77:0/2263064917 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96bc005020 con 0x7f96c41079d0 2024-09-10T10:01:53.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.544+0000 7f96bbe00640 1 -- 172.21.15.77:0/2263064917 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96bc0116b0 con 0x7f96c41079d0 2024-09-10T10:01:53.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.544+0000 7f96c8a00640 1 -- 172.21.15.77:0/2263064917 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96c41a6a40 con 0x7f96c41079d0 2024-09-10T10:01:53.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.544+0000 7f96c8a00640 1 -- 172.21.15.77:0/2263064917 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96c41a6f30 con 0x7f96c41079d0 2024-09-10T10:01:53.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.548+0000 7f96bbe00640 1 -- 172.21.15.77:0/2263064917 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f96bc005560 con 0x7f96c41079d0 2024-09-10T10:01:53.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.549+0000 7f96ccc46740 1 Processor -- start 2024-09-10T10:01:53.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.549+0000 7f96ccc46740 1 -- start start 2024-09-10T10:01:53.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.549+0000 7f96ccc46740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x565154643e40 0x565154644220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:53.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.549+0000 7f96bbe00640 1 --2- 172.21.15.77:0/2263064917 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f96a80a2b80 0x7f96a80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:53.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.549+0000 7f96ccc46740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565154644760 0x56515464eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:53.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.549+0000 7f96ccc46740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56515464f020 0x565154651410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:53.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.549+0000 7f96ccc46740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5651545aa3e0 con 0x565154643e40 2024-09-10T10:01:53.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.549+0000 7f96ccc46740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x565154551d80 con 0x565154644760 2024-09-10T10:01:53.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.549+0000 7f96ccc46740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5651545a63a0 con 0x56515464f020 2024-09-10T10:01:53.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.549+0000 7f96bbe00640 1 -- 172.21.15.77:0/2263064917 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f96bc0cf8c0 con 0x7f96c41079d0 2024-09-10T10:01:53.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.550+0000 7f96c3400640 1 --2- 172.21.15.77:0/2263064917 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f96a80a2b80 0x7f96a80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:53.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.550+0000 7f96c3e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56515464f020 0x565154651410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:53.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.550+0000 7f96c3e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56515464f020 0x565154651410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:51924/0 (socket says 172.21.15.77:51924) 2024-09-10T10:01:53.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.550+0000 7f96c3e00640 1 -- 172.21.15.77:0/1228855440 learned_addr learned my addr 172.21.15.77:0/1228855440 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:01:53.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.550+0000 7f96c2a00640 1 --2- 172.21.15.77:0/1228855440 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x565154643e40 0x565154644220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:53.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.550+0000 7f96c3e00640 1 -- 172.21.15.77:0/1228855440 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565154644760 msgr2=0x56515464eae0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T10:01:53.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.550+0000 7f96c3e00640 1 --2- 172.21.15.77:0/1228855440 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565154644760 0x56515464eae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:53.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.550+0000 7f96c3e00640 1 -- 172.21.15.77:0/1228855440 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x565154643e40 msgr2=0x565154644220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:53.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.550+0000 7f96c3e00640 1 --2- 172.21.15.77:0/1228855440 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x565154643e40 0x565154644220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:53.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.550+0000 7f96c3e00640 1 -- 172.21.15.77:0/1228855440 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565154570ae0 con 0x56515464f020 2024-09-10T10:01:53.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.551+0000 7f96c3e00640 1 --2- 172.21.15.77:0/1228855440 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56515464f020 0x565154651410 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f96bc0986a0 tx=0x7f96bc0072f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d16e6aee0aafd8c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:53.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.551+0000 7f96c3400640 1 --2- 172.21.15.77:0/2263064917 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f96a80a2b80 0x7f96a80a5040 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f96b4006010 tx=0x7f96b4000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:53.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.551+0000 7f96afe00640 1 -- 172.21.15.77:0/1228855440 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96bc0a0080 con 0x56515464f020 2024-09-10T10:01:53.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.551+0000 7f96afe00640 1 -- 172.21.15.77:0/1228855440 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96bc0a4940 con 0x56515464f020 2024-09-10T10:01:53.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.551+0000 7f96afe00640 1 -- 172.21.15.77:0/1228855440 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96bc007d60 con 0x56515464f020 2024-09-10T10:01:53.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.552+0000 7f96ccc46740 1 -- 172.21.15.77:0/1228855440 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56515464f020 msgr2=0x565154651410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:53.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.552+0000 7f96ccc46740 1 --2- 172.21.15.77:0/1228855440 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56515464f020 0x565154651410 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f96bc0986a0 tx=0x7f96bc0072f0 comp rx=0 tx=0).stop 2024-09-10T10:01:53.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.552+0000 7f96ccc46740 1 -- 172.21.15.77:0/1228855440 shutdown_connections 2024-09-10T10:01:53.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.552+0000 7f96ccc46740 1 --2- 172.21.15.77:0/1228855440 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56515464f020 0x565154651410 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:53.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.552+0000 7f96ccc46740 1 --2- 172.21.15.77:0/1228855440 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565154644760 0x56515464eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:53.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.552+0000 7f96ccc46740 1 --2- 172.21.15.77:0/1228855440 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x565154643e40 0x565154644220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:53.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.552+0000 7f96ccc46740 1 -- 172.21.15.77:0/1228855440 >> 172.21.15.77:0/1228855440 conn(0x5651546403d0 msgr2=0x5651546407b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:53.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.552+0000 7f96ccc46740 1 -- 172.21.15.77:0/1228855440 shutdown_connections 2024-09-10T10:01:53.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.552+0000 7f96ccc46740 1 -- 172.21.15.77:0/1228855440 wait complete. 2024-09-10T10:01:53.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.552+0000 7f96ccc46740 1 -- start start 2024-09-10T10:01:53.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.553+0000 7f96ccc46740 1 Processor -- start 2024-09-10T10:01:53.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.553+0000 7f96ccc46740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565154643e40 0x56515464bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:53.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.553+0000 7f96ccc46740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565154644760 0x56515464c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:53.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.553+0000 7f96ccc46740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56515464f020 0x56515464ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:53.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.553+0000 7f96ccc46740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x565154651bf0 con 0x56515464f020 2024-09-10T10:01:53.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.553+0000 7f96ccc46740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5651545650f0 con 0x565154644760 2024-09-10T10:01:53.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.553+0000 7f96ccc46740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x565154651d70 con 0x565154643e40 2024-09-10T10:01:53.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.553+0000 7f96ccc46740 10 client.0 _ll_register_callbacks cb 0x56515454cbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:01:53.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.554+0000 7f96c3400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565154643e40 0x56515464bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:53.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.554+0000 7f96c3400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565154643e40 0x56515464bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:51940/0 (socket says 172.21.15.77:51940) 2024-09-10T10:01:53.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.554+0000 7f96c2a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565154644760 0x56515464c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:53.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.554+0000 7f96c3400640 1 -- 172.21.15.77:0/3018295479 learned_addr learned my addr 172.21.15.77:0/3018295479 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:01:53.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.554+0000 7f96c3e00640 1 --2- 172.21.15.77:0/3018295479 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56515464f020 0x56515464ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:53.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.554+0000 7f96c2a00640 1 -- 172.21.15.77:0/3018295479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565154643e40 msgr2=0x56515464bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:53.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.554+0000 7f96c2a00640 1 --2- 172.21.15.77:0/3018295479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565154643e40 0x56515464bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:53.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.554+0000 7f96c2a00640 1 -- 172.21.15.77:0/3018295479 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56515464f020 msgr2=0x56515464ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:53.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.554+0000 7f96c2a00640 1 --2- 172.21.15.77:0/3018295479 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56515464f020 0x56515464ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:53.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.554+0000 7f96c2a00640 1 -- 172.21.15.77:0/3018295479 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56515431b3c0 con 0x565154644760 2024-09-10T10:01:53.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.554+0000 7f96c3e00640 1 --2- 172.21.15.77:0/3018295479 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56515464f020 0x56515464ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:01:53.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.554+0000 7f96c3400640 1 --2- 172.21.15.77:0/3018295479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565154643e40 0x56515464bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:01:53.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.554+0000 7f96c2a00640 1 --2- 172.21.15.77:0/3018295479 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565154644760 0x56515464c510 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x565154641230 tx=0x7f96b0012b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68cc3ff2cfc54756 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:53.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.555+0000 7f96acc00640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T10:01:53.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.555+0000 7f96acc00640 1 -- 172.21.15.77:0/3018295479 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b0010d90 con 0x565154644760 2024-09-10T10:01:53.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.555+0000 7f96ccc46740 1 -- 172.21.15.77:0/3018295479 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96bc0a4940 con 0x565154644760 2024-09-10T10:01:53.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.555+0000 7f96acc00640 1 -- 172.21.15.77:0/3018295479 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f96b0005020 con 0x565154644760 2024-09-10T10:01:53.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.555+0000 7f96acc00640 1 -- 172.21.15.77:0/3018295479 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f96b0005220 con 0x565154644760 2024-09-10T10:01:53.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.555+0000 7f96ccc46740 10 client.26855 fetch_fsmap learned FSMap version 35 2024-09-10T10:01:53.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.555+0000 7f96ccc46740 1 -- 172.21.15.77:0/3018295479 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56515465dda0 con 0x565154644760 2024-09-10T10:01:53.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.555+0000 7f96acc00640 1 -- 172.21.15.77:0/3018295479 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96bc0a4940 con 0x565154644760 2024-09-10T10:01:53.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.556+0000 7f96acc00640 1 -- 172.21.15.77:0/3018295479 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f96b001c4e0 con 0x565154644760 2024-09-10T10:01:53.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.556+0000 7f96ccc46740 10 client.26855 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T10:01:53.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.556+0000 7f96ccc46740 10 client.26855 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-10T10:01:53.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.556+0000 7f96ccc46740 20 client.26855 populate_metadata read hostname 'smithi077' 2024-09-10T10:01:53.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.556+0000 7f96ccc46740 1 --2- 172.21.15.77:0/3018295479 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x56515465f250 0x565154661670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:53.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.556+0000 7f96ccc46740 4 client.26855 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:01:53.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.556+0000 7f96ccc46740 1 -- 172.21.15.77:0/3018295479 --> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5651545b7950 con 0x56515465f250 2024-09-10T10:01:53.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.556+0000 7f96c3400640 1 --2- 172.21.15.77:0/3018295479 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x56515465f250 0x565154661670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:53.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.557+0000 7f96acc00640 10 client.26855 ms_handle_connect on v2:172.21.15.120:6836/3033280709 2024-09-10T10:01:53.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.557+0000 7f96c3400640 1 --2- 172.21.15.77:0/3018295479 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x56515465f250 0x565154661670 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=39eff5a98a137e39 server_cookie=d1233ffffdf3e703 in_seq=0 out_seq=0 2024-09-10T10:01:53.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.557+0000 7f96acc00640 1 -- 172.21.15.77:0/3018295479 <== mds.1 v2:172.21.15.120:6836/3033280709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5651545b7950 con 0x56515465f250 2024-09-10T10:01:53.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.557+0000 7f96acc00640 10 client.26855 handle_command_reply: tid=0 2024-09-10T10:01:53.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.558+0000 7f96ccc46740 1 client.26855 shutdown 2024-09-10T10:01:53.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.558+0000 7f96ccc46740 1 -- 172.21.15.77:0/3018295479 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565154644760 msgr2=0x56515464c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:53.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.558+0000 7f96ccc46740 1 --2- 172.21.15.77:0/3018295479 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565154644760 0x56515464c510 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x565154641230 tx=0x7f96b0012b80 comp rx=0 tx=0).stop 2024-09-10T10:01:53.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.558+0000 7f96ccc46740 1 -- 172.21.15.77:0/3018295479 shutdown_connections 2024-09-10T10:01:53.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.558+0000 7f96ccc46740 1 --2- 172.21.15.77:0/3018295479 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x56515465f250 0x565154661670 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:53.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.558+0000 7f96ccc46740 1 --2- 172.21.15.77:0/3018295479 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56515464f020 0x56515464ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:53.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.558+0000 7f96ccc46740 1 --2- 172.21.15.77:0/3018295479 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565154644760 0x56515464c510 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:53.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.558+0000 7f96ccc46740 1 --2- 172.21.15.77:0/3018295479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565154643e40 0x56515464bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:53.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.558+0000 7f96ccc46740 1 -- 172.21.15.77:0/3018295479 >> 172.21.15.77:0/3018295479 conn(0x5651546403d0 msgr2=0x565154646100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:53.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.558+0000 7f96acc00640 0 client.26855 ms_handle_reset on v2:172.21.15.120:6836/3033280709 2024-09-10T10:01:53.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.558+0000 7f96ccc46740 1 -- 172.21.15.77:0/3018295479 shutdown_connections 2024-09-10T10:01:53.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.558+0000 7f96ccc46740 1 -- 172.21.15.77:0/3018295479 wait complete. 2024-09-10T10:01:53.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.558+0000 7f96ccc46740 20 client.26855 trim_cache size 0 max 16384 2024-09-10T10:01:53.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.572+0000 7f96ccc46740 1 Processor -- start 2024-09-10T10:01:53.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.572+0000 7f96ccc46740 1 -- start start 2024-09-10T10:01:53.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.572+0000 7f96ccc46740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565154643e40 0x56515464c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:53.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.572+0000 7f96ccc46740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565154644760 0x565154666f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:53.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.572+0000 7f96ccc46740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56515464f020 0x565154667460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:53.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.572+0000 7f96ccc46740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f96c41b0f60 con 0x56515464f020 2024-09-10T10:01:53.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.572+0000 7f96ccc46740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f96c41b2aa0 con 0x565154643e40 2024-09-10T10:01:53.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.572+0000 7f96ccc46740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f96c41b1630 con 0x565154644760 2024-09-10T10:01:53.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.572+0000 7f96c2a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565154644760 0x565154666f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:53.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.572+0000 7f96c3e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56515464f020 0x565154667460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:53.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.573+0000 7f96c2a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565154644760 0x565154666f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:51952/0 (socket says 172.21.15.77:51952) 2024-09-10T10:01:53.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.573+0000 7f96c2a00640 1 -- 172.21.15.77:0/1289284527 learned_addr learned my addr 172.21.15.77:0/1289284527 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:01:53.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.573+0000 7f96c3400640 1 --2- 172.21.15.77:0/1289284527 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565154643e40 0x56515464c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:53.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.573+0000 7f96c2a00640 1 -- 172.21.15.77:0/1289284527 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565154643e40 msgr2=0x56515464c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:53.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.573+0000 7f96c2a00640 1 --2- 172.21.15.77:0/1289284527 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565154643e40 0x56515464c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:53.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.573+0000 7f96c2a00640 1 -- 172.21.15.77:0/1289284527 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56515464f020 msgr2=0x565154667460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:53.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.573+0000 7f96c2a00640 1 --2- 172.21.15.77:0/1289284527 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56515464f020 0x565154667460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:53.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.573+0000 7f96c2a00640 1 -- 172.21.15.77:0/1289284527 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565154643bc0 con 0x565154644760 2024-09-10T10:01:53.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.573+0000 7f96c2a00640 1 --2- 172.21.15.77:0/1289284527 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565154644760 0x565154666f20 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f96b001cb40 tx=0x7f96b000bf40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30b8e417322a68ec server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:53.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.574+0000 7f96af400640 1 -- 172.21.15.77:0/1289284527 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b0000e60 con 0x565154644760 2024-09-10T10:01:53.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.574+0000 7f96af400640 1 -- 172.21.15.77:0/1289284527 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96b001cdc0 con 0x565154644760 2024-09-10T10:01:53.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.574+0000 7f96af400640 1 -- 172.21.15.77:0/1289284527 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b002f070 con 0x565154644760 2024-09-10T10:01:53.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.574+0000 7f96ccc46740 1 -- 172.21.15.77:0/1289284527 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565154644760 msgr2=0x565154666f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:53.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.574+0000 7f96ccc46740 1 --2- 172.21.15.77:0/1289284527 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565154644760 0x565154666f20 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f96b001cb40 tx=0x7f96b000bf40 comp rx=0 tx=0).stop 2024-09-10T10:01:53.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.575+0000 7f96ccc46740 1 -- 172.21.15.77:0/1289284527 shutdown_connections 2024-09-10T10:01:53.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.575+0000 7f96ccc46740 1 --2- 172.21.15.77:0/1289284527 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56515464f020 0x565154667460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:53.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.575+0000 7f96ccc46740 1 --2- 172.21.15.77:0/1289284527 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565154644760 0x565154666f20 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:53.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.575+0000 7f96ccc46740 1 --2- 172.21.15.77:0/1289284527 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565154643e40 0x56515464c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:53.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.575+0000 7f96ccc46740 1 -- 172.21.15.77:0/1289284527 >> 172.21.15.77:0/1289284527 conn(0x5651546403d0 msgr2=0x56515464b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:53.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.575+0000 7f96ccc46740 1 -- 172.21.15.77:0/1289284527 shutdown_connections 2024-09-10T10:01:53.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.575+0000 7f96ccc46740 1 -- 172.21.15.77:0/1289284527 wait complete. 2024-09-10T10:01:53.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.576+0000 7f96ccc46740 1 -- start start 2024-09-10T10:01:53.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.576+0000 7f96ccc46740 1 Processor -- start 2024-09-10T10:01:53.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.576+0000 7f96ccc46740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x565154643e40 0x5651546571a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:53.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.576+0000 7f96ccc46740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565154644760 0x5651546576e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:53.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.576+0000 7f96ccc46740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56515464f020 0x565154652560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:53.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.576+0000 7f96ccc46740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f96c41b23d0 con 0x565154643e40 2024-09-10T10:01:53.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.576+0000 7f96ccc46740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f96c41b1d00 con 0x565154644760 2024-09-10T10:01:53.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.576+0000 7f96ccc46740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f96c41aef40 con 0x56515464f020 2024-09-10T10:01:53.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.576+0000 7f96ccc46740 10 client.0 _ll_register_callbacks cb 0x7f96b000bb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:01:53.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.577+0000 7f96c2a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565154644760 0x5651546576e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:53.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.577+0000 7f96c3400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x565154643e40 0x5651546571a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:53.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.577+0000 7f96c2a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565154644760 0x5651546576e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:46626/0 (socket says 172.21.15.77:46626) 2024-09-10T10:01:53.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.577+0000 7f96c2a00640 1 -- 172.21.15.77:0/2199472331 learned_addr learned my addr 172.21.15.77:0/2199472331 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:01:53.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.577+0000 7f96c3e00640 1 --2- 172.21.15.77:0/2199472331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56515464f020 0x565154652560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:53.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.577+0000 7f96c2a00640 1 -- 172.21.15.77:0/2199472331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56515464f020 msgr2=0x565154652560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:53.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.577+0000 7f96c2a00640 1 --2- 172.21.15.77:0/2199472331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56515464f020 0x565154652560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:53.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.577+0000 7f96c2a00640 1 -- 172.21.15.77:0/2199472331 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x565154643e40 msgr2=0x5651546571a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:53.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.577+0000 7f96c2a00640 1 --2- 172.21.15.77:0/2199472331 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x565154643e40 0x5651546571a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:53.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.577+0000 7f96c2a00640 1 -- 172.21.15.77:0/2199472331 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565154652d90 con 0x565154644760 2024-09-10T10:01:53.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.577+0000 7f96c2a00640 1 --2- 172.21.15.77:0/2199472331 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565154644760 0x5651546576e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f96b0000c00 tx=0x7f96b0005650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80174dc9f38ce7ad server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:53.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.577+0000 7f96acc00640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T10:01:53.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.578+0000 7f96acc00640 1 -- 172.21.15.77:0/2199472331 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b0037070 con 0x565154644760 2024-09-10T10:01:53.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.578+0000 7f96ccc46740 1 -- 172.21.15.77:0/2199472331 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96b001cdc0 con 0x565154644760 2024-09-10T10:01:53.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.578+0000 7f96acc00640 1 -- 172.21.15.77:0/2199472331 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f96b0010da0 con 0x565154644760 2024-09-10T10:01:53.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.578+0000 7f96acc00640 1 -- 172.21.15.77:0/2199472331 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f96b0040050 con 0x565154644760 2024-09-10T10:01:53.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.578+0000 7f96ccc46740 10 client.26861 fetch_fsmap learned FSMap version 35 2024-09-10T10:01:53.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.578+0000 7f96ccc46740 1 -- 172.21.15.77:0/2199472331 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5651546c9a20 con 0x565154644760 2024-09-10T10:01:53.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.579+0000 7f96acc00640 1 -- 172.21.15.77:0/2199472331 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96b001cdc0 con 0x565154644760 2024-09-10T10:01:53.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.579+0000 7f96acc00640 1 -- 172.21.15.77:0/2199472331 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f96b003a020 con 0x565154644760 2024-09-10T10:01:53.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.579+0000 7f96ccc46740 10 client.26861 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T10:01:53.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.579+0000 7f96ccc46740 10 client.26861 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-10T10:01:53.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.579+0000 7f96ccc46740 20 client.26861 populate_metadata read hostname 'smithi077' 2024-09-10T10:01:53.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.579+0000 7f96ccc46740 1 --2- 172.21.15.77:0/2199472331 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x5651546ca0d0 0x5651546cc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:53.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.579+0000 7f96ccc46740 4 client.26861 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:01:53.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.579+0000 7f96ccc46740 1 -- 172.21.15.77:0/2199472331 --> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5651545b1e60 con 0x5651546ca0d0 2024-09-10T10:01:53.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.579+0000 7f96c3400640 1 --2- 172.21.15.77:0/2199472331 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x5651546ca0d0 0x5651546cc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:53.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.580+0000 7f96c3400640 1 --2- 172.21.15.77:0/2199472331 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x5651546ca0d0 0x5651546cc4c0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=35b62f95c1043dc1 server_cookie=777e2b0e1a825edc in_seq=0 out_seq=0 2024-09-10T10:01:53.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.580+0000 7f96acc00640 10 client.26861 ms_handle_connect on v2:172.21.15.120:6836/3033280709 2024-09-10T10:01:53.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.580+0000 7f96acc00640 1 -- 172.21.15.77:0/2199472331 <== mds.1 v2:172.21.15.120:6836/3033280709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5651545b1e60 con 0x5651546ca0d0 2024-09-10T10:01:53.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.580+0000 7f96acc00640 10 client.26861 handle_command_reply: tid=0 2024-09-10T10:01:53.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.580+0000 7f96ccc46740 1 client.26861 shutdown 2024-09-10T10:01:53.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.580+0000 7f96ccc46740 1 -- 172.21.15.77:0/2199472331 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565154644760 msgr2=0x5651546576e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:53.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.580+0000 7f96ccc46740 1 --2- 172.21.15.77:0/2199472331 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565154644760 0x5651546576e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f96b0000c00 tx=0x7f96b0005650 comp rx=0 tx=0).stop 2024-09-10T10:01:53.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.580+0000 7f96ccc46740 1 -- 172.21.15.77:0/2199472331 shutdown_connections 2024-09-10T10:01:53.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.580+0000 7f96ccc46740 1 --2- 172.21.15.77:0/2199472331 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x5651546ca0d0 0x5651546cc4c0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:53.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.580+0000 7f96ccc46740 1 --2- 172.21.15.77:0/2199472331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56515464f020 0x565154652560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:53.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.580+0000 7f96ccc46740 1 --2- 172.21.15.77:0/2199472331 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565154644760 0x5651546576e0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:53.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.580+0000 7f96acc00640 0 client.26861 ms_handle_reset on v2:172.21.15.120:6836/3033280709 2024-09-10T10:01:53.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.580+0000 7f96ccc46740 1 --2- 172.21.15.77:0/2199472331 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x565154643e40 0x5651546571a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:53.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.580+0000 7f96ccc46740 1 -- 172.21.15.77:0/2199472331 >> 172.21.15.77:0/2199472331 conn(0x5651546403d0 msgr2=0x5651546458e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:53.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.580+0000 7f96ccc46740 1 -- 172.21.15.77:0/2199472331 shutdown_connections 2024-09-10T10:01:53.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.580+0000 7f96ccc46740 1 -- 172.21.15.77:0/2199472331 wait complete. 2024-09-10T10:01:53.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.581+0000 7f96ccc46740 20 client.26861 trim_cache size 0 max 16384 2024-09-10T10:01:53.584 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-10T10:01:53.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.581+0000 7f96c8a00640 1 -- 172.21.15.77:0/2263064917 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f96a80a2b80 msgr2=0x7f96a80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:53.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.581+0000 7f96c8a00640 1 --2- 172.21.15.77:0/2263064917 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f96a80a2b80 0x7f96a80a5040 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f96b4006010 tx=0x7f96b4000bc0 comp rx=0 tx=0).stop 2024-09-10T10:01:53.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.581+0000 7f96c8a00640 1 -- 172.21.15.77:0/2263064917 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f96c41079d0 msgr2=0x7f96c4074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:53.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.581+0000 7f96c8a00640 1 --2- 172.21.15.77:0/2263064917 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f96c41079d0 0x7f96c4074f70 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f96bc00eaa0 tx=0x7f96bc00ef60 comp rx=0 tx=0).stop 2024-09-10T10:01:53.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.582+0000 7f96c8a00640 1 -- 172.21.15.77:0/2263064917 shutdown_connections 2024-09-10T10:01:53.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.582+0000 7f96c8a00640 1 --2- 172.21.15.77:0/2263064917 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f96a80a2b80 0x7f96a80a5040 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:53.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.582+0000 7f96c8a00640 1 --2- 172.21.15.77:0/2263064917 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f96c41079d0 0x7f96c4074f70 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:53.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.582+0000 7f96c8a00640 1 --2- 172.21.15.77:0/2263064917 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f96c4102370 0x7f96c4074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:53.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.582+0000 7f96c8a00640 1 --2- 172.21.15.77:0/2263064917 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f96c41019d0 0x7f96c40744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:53.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.582+0000 7f96c8a00640 1 -- 172.21.15.77:0/2263064917 >> 172.21.15.77:0/2263064917 conn(0x7f96c40fd710 msgr2=0x7f96c40fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:53.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.582+0000 7f96c8a00640 1 -- 172.21.15.77:0/2263064917 shutdown_connections 2024-09-10T10:01:53.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:53.582+0000 7f96c8a00640 1 -- 172.21.15.77:0/2263064917 wait complete. 2024-09-10T10:01:53.830 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:01:54.437 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T10:01:54.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:54.911+0000 7fdb43000640 1 -- 172.21.15.77:0/3580582504 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdb3c109120 msgr2=0x7fdb3c110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:54.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:54.911+0000 7fdb43000640 1 --2- 172.21.15.77:0/3580582504 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdb3c109120 0x7fdb3c110ca0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fdb34009be0 tx=0x7fdb3402a6c0 comp rx=0 tx=0).stop 2024-09-10T10:01:54.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:54.911+0000 7fdb43000640 1 -- 172.21.15.77:0/3580582504 shutdown_connections 2024-09-10T10:01:54.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:54.911+0000 7fdb43000640 1 --2- 172.21.15.77:0/3580582504 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdb3c109120 0x7fdb3c110ca0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:54.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:54.911+0000 7fdb43000640 1 --2- 172.21.15.77:0/3580582504 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdb3c1049a0 0x7fdb3c1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:54.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:54.911+0000 7fdb43000640 1 --2- 172.21.15.77:0/3580582504 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb3c103ff0 0x7fdb3c1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:54.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:54.911+0000 7fdb43000640 1 -- 172.21.15.77:0/3580582504 >> 172.21.15.77:0/3580582504 conn(0x7fdb3c0ffa90 msgr2=0x7fdb3c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:54.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:54.912+0000 7fdb43000640 1 -- 172.21.15.77:0/3580582504 shutdown_connections 2024-09-10T10:01:54.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:54.912+0000 7fdb43000640 1 -- 172.21.15.77:0/3580582504 wait complete. 2024-09-10T10:01:54.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:54.912+0000 7fdb43000640 1 Processor -- start 2024-09-10T10:01:54.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:54.913+0000 7fdb43000640 1 -- start start 2024-09-10T10:01:54.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:54.913+0000 7fdb43000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdb3c103ff0 0x7fdb3c19a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:54.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:54.913+0000 7fdb43000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb3c1049a0 0x7fdb3c19ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:54.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:54.913+0000 7fdb43000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdb3c109120 0x7fdb3c19ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:54.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:54.913+0000 7fdb43000640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fdb3c114150 con 0x7fdb3c109120 2024-09-10T10:01:54.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:54.913+0000 7fdb43000640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdb3c113fd0 con 0x7fdb3c103ff0 2024-09-10T10:01:54.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:54.913+0000 7fdb43000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdb3c1142d0 con 0x7fdb3c1049a0 2024-09-10T10:01:54.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:54.913+0000 7fdb41000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdb3c109120 0x7fdb3c19ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:54.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:54.913+0000 7fdb33e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb3c1049a0 0x7fdb3c19ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:54.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:54.913+0000 7fdb3be00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdb3c103ff0 0x7fdb3c19a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:54.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:54.913+0000 7fdb41000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdb3c109120 0x7fdb3c19ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:51392/0 (socket says 172.21.15.77:51392) 2024-09-10T10:01:54.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:54.913+0000 7fdb41000640 1 -- 172.21.15.77:0/1687544039 learned_addr learned my addr 172.21.15.77:0/1687544039 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:01:54.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:54.914+0000 7fdb3be00640 1 -- 172.21.15.77:0/1687544039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb3c1049a0 msgr2=0x7fdb3c19ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:54.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:54.914+0000 7fdb3be00640 1 --2- 172.21.15.77:0/1687544039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb3c1049a0 0x7fdb3c19ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:54.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:54.914+0000 7fdb3be00640 1 -- 172.21.15.77:0/1687544039 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdb3c109120 msgr2=0x7fdb3c19ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:54.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:54.914+0000 7fdb3be00640 1 --2- 172.21.15.77:0/1687544039 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdb3c109120 0x7fdb3c19ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:54.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:54.914+0000 7fdb3be00640 1 -- 172.21.15.77:0/1687544039 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb3c19f5e0 con 0x7fdb3c103ff0 2024-09-10T10:01:54.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:54.914+0000 7fdb33e00640 1 --2- 172.21.15.77:0/1687544039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb3c1049a0 0x7fdb3c19ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:01:54.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:54.914+0000 7fdb3be00640 1 --2- 172.21.15.77:0/1687544039 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdb3c103ff0 0x7fdb3c19a630 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c009fd0 tx=0x7fdb2c00ec20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:54.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:54.914+0000 7fdb39600640 1 -- 172.21.15.77:0/1687544039 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb2c00ccc0 con 0x7fdb3c103ff0 2024-09-10T10:01:54.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:54.914+0000 7fdb43000640 1 -- 172.21.15.77:0/1687544039 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb3c19f8d0 con 0x7fdb3c103ff0 2024-09-10T10:01:54.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:54.914+0000 7fdb39600640 1 -- 172.21.15.77:0/1687544039 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb2c00ee70 con 0x7fdb3c103ff0 2024-09-10T10:01:54.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:54.914+0000 7fdb43000640 1 -- 172.21.15.77:0/1687544039 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb3c1a71b0 con 0x7fdb3c103ff0 2024-09-10T10:01:54.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:54.915+0000 7fdb39600640 1 -- 172.21.15.77:0/1687544039 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb2c004a00 con 0x7fdb3c103ff0 2024-09-10T10:01:54.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:54.915+0000 7fdb43000640 1 -- 172.21.15.77:0/1687544039 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb18005290 con 0x7fdb3c103ff0 2024-09-10T10:01:54.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:54.918+0000 7fdb39600640 1 -- 172.21.15.77:0/1687544039 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fdb2c004ba0 con 0x7fdb3c103ff0 2024-09-10T10:01:54.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:54.919+0000 7fdb39600640 1 --2- 172.21.15.77:0/1687544039 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fdb200a2b80 0x7fdb200a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:54.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:54.919+0000 7fdb39600640 1 -- 172.21.15.77:0/1687544039 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fdb2c0cf1f0 con 0x7fdb3c103ff0 2024-09-10T10:01:54.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:54.920+0000 7fdb33e00640 1 --2- 172.21.15.77:0/1687544039 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fdb200a2b80 0x7fdb200a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:54.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:54.920+0000 7fdb33e00640 1 --2- 172.21.15.77:0/1687544039 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fdb200a2b80 0x7fdb200a5040 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c10a9e0 tx=0x7fdb24005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:54.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:54.921+0000 7fdb39600640 1 -- 172.21.15.77:0/1687544039 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdb2c09bd00 con 0x7fdb3c103ff0 2024-09-10T10:01:54.990 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:01:55.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.143+0000 7fdb43000640 1 -- 172.21.15.77:0/1687544039 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdb18005580 con 0x7fdb3c103ff0 2024-09-10T10:01:55.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.144+0000 7fdb39600640 1 -- 172.21.15.77:0/1687544039 <== mon.2 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11989 (secure 0 0 0) 0x7fdb2c09b600 con 0x7fdb3c103ff0 2024-09-10T10:01:55.145 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T10:01:55.145 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":35,"btime":"2024-09-10T10:01:05:922537+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3159289675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3159289675},{"type":"v1","addr":"172.21.15.77:6835","nonce":3159289675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1951962341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1951962341},{"type":"v1","addr":"172.21.15.77:6837","nonce":1951962341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":35,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:52:23.368252+0000","modified":"2024-09-10T10:01:05.706646+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24563,"mds_1":24524,"mds_2":24520,"mds_3":24502,"mds_4":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14487":{"gid":14487,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/4291121888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":4291121888},{"type":"v1","addr":"172.21.15.77:6839","nonce":4291121888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14499":{"gid":14499,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6841/445284194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":445284194},{"type":"v1","addr":"172.21.15.77:6841","nonce":445284194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1005054423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1005054423},{"type":"v1","addr":"172.21.15.134:6833","nonce":1005054423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6833/3716307744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":3716307744},{"type":"v1","addr":"172.21.15.120:6833","nonce":3716307744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/1319892544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1319892544},{"type":"v1","addr":"172.21.15.120:6835","nonce":1319892544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.134:6835/3293933699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3293933699},{"type":"v1","addr":"172.21.15.134:6835","nonce":3293933699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.134:6837/2461497091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2461497091},{"type":"v1","addr":"172.21.15.134:6837","nonce":2461497091}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":20,"addr":"172.21.15.134:6839/1965117680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1965117680},{"type":"v1","addr":"172.21.15.134:6839","nonce":1965117680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":20,"addr":"172.21.15.120:6837/3033280709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3033280709},{"type":"v1","addr":"172.21.15.120:6837","nonce":3033280709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.120:6839/1777710157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1777710157},{"type":"v1","addr":"172.21.15.120:6839","nonce":1777710157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24563,"qdb_cluster":[24496,24502,24520,24524,24563]},"id":1}]} 2024-09-10T10:01:55.146 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 35 2024-09-10T10:01:55.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.146+0000 7fdb43000640 1 -- 172.21.15.77:0/1687544039 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fdb200a2b80 msgr2=0x7fdb200a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:55.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.146+0000 7fdb43000640 1 --2- 172.21.15.77:0/1687544039 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fdb200a2b80 0x7fdb200a5040 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c10a9e0 tx=0x7fdb24005f90 comp rx=0 tx=0).stop 2024-09-10T10:01:55.148 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.146+0000 7fdb43000640 1 -- 172.21.15.77:0/1687544039 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdb3c103ff0 msgr2=0x7fdb3c19a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:55.148 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.146+0000 7fdb43000640 1 --2- 172.21.15.77:0/1687544039 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdb3c103ff0 0x7fdb3c19a630 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c009fd0 tx=0x7fdb2c00ec20 comp rx=0 tx=0).stop 2024-09-10T10:01:55.148 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.146+0000 7fdb43000640 1 -- 172.21.15.77:0/1687544039 shutdown_connections 2024-09-10T10:01:55.148 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.146+0000 7fdb43000640 1 --2- 172.21.15.77:0/1687544039 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fdb200a2b80 0x7fdb200a5040 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:55.148 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.146+0000 7fdb43000640 1 --2- 172.21.15.77:0/1687544039 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdb3c109120 0x7fdb3c19ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:55.148 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.146+0000 7fdb43000640 1 --2- 172.21.15.77:0/1687544039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb3c1049a0 0x7fdb3c19ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:55.148 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.146+0000 7fdb43000640 1 --2- 172.21.15.77:0/1687544039 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdb3c103ff0 0x7fdb3c19a630 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:55.148 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.146+0000 7fdb43000640 1 -- 172.21.15.77:0/1687544039 >> 172.21.15.77:0/1687544039 conn(0x7fdb3c0ffa90 msgr2=0x7fdb3c1005b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:55.148 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.147+0000 7fdb43000640 1 -- 172.21.15.77:0/1687544039 shutdown_connections 2024-09-10T10:01:55.148 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.147+0000 7fdb43000640 1 -- 172.21.15.77:0/1687544039 wait complete. 2024-09-10T10:01:55.162 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:01:54 smithi077 ceph-mon[30319]: pgmap v593: 129 pgs: 129 active+clean; 142 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 40 KiB/s wr, 14 op/s 2024-09-10T10:01:55.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:01:54 smithi120 ceph-mon[34770]: pgmap v593: 129 pgs: 129 active+clean; 142 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 40 KiB/s wr, 14 op/s 2024-09-10T10:01:55.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:01:54 smithi134 ceph-mon[35674]: pgmap v593: 129 pgs: 129 active+clean; 142 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 40 KiB/s wr, 14 op/s 2024-09-10T10:01:55.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.871+0000 7ffbb2e00640 1 -- 172.21.15.77:0/2305572094 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffbac108f60 msgr2=0x7ffbac110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:55.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.871+0000 7ffbb2e00640 1 --2- 172.21.15.77:0/2305572094 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffbac108f60 0x7ffbac110ae0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7ffba4009900 tx=0x7ffba402a3f0 comp rx=0 tx=0).stop 2024-09-10T10:01:55.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.872+0000 7ffbb2e00640 1 -- 172.21.15.77:0/2305572094 shutdown_connections 2024-09-10T10:01:55.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.872+0000 7ffbb2e00640 1 --2- 172.21.15.77:0/2305572094 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffbac108f60 0x7ffbac110ae0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:55.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.872+0000 7ffbb2e00640 1 --2- 172.21.15.77:0/2305572094 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffbac1047e0 0x7ffbac108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:55.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.872+0000 7ffbb2e00640 1 --2- 172.21.15.77:0/2305572094 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffbac103e30 0x7ffbac104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:55.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.872+0000 7ffbb2e00640 1 -- 172.21.15.77:0/2305572094 >> 172.21.15.77:0/2305572094 conn(0x7ffbac0ff8d0 msgr2=0x7ffbac101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:55.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.872+0000 7ffbb2e00640 1 -- 172.21.15.77:0/2305572094 shutdown_connections 2024-09-10T10:01:55.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.872+0000 7ffbb2e00640 1 -- 172.21.15.77:0/2305572094 wait complete. 2024-09-10T10:01:55.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.873+0000 7ffbb2e00640 1 Processor -- start 2024-09-10T10:01:55.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.873+0000 7ffbb2e00640 1 -- start start 2024-09-10T10:01:55.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.873+0000 7ffbb2e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffbac103e30 0x7ffbac19a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:55.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.873+0000 7ffbb2e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffbac1047e0 0x7ffbac19a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:55.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.873+0000 7ffbb2e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffbac108f60 0x7ffbac19ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:55.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.873+0000 7ffbb2e00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ffbac113fc0 con 0x7ffbac103e30 2024-09-10T10:01:55.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.873+0000 7ffbb2e00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ffbac113e40 con 0x7ffbac108f60 2024-09-10T10:01:55.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.873+0000 7ffbb2e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ffbac114140 con 0x7ffbac1047e0 2024-09-10T10:01:55.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.873+0000 7ffbb1a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffbac103e30 0x7ffbac19a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:55.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.874+0000 7ffbb1a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffbac103e30 0x7ffbac19a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:51414/0 (socket says 172.21.15.77:51414) 2024-09-10T10:01:55.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.874+0000 7ffbb1a00640 1 -- 172.21.15.77:0/852799055 learned_addr learned my addr 172.21.15.77:0/852799055 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:01:55.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.874+0000 7ffbb2400640 1 --2- 172.21.15.77:0/852799055 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffbac108f60 0x7ffbac19ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:55.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.874+0000 7ffbb1000640 1 --2- 172.21.15.77:0/852799055 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffbac1047e0 0x7ffbac19a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:55.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.874+0000 7ffbb1a00640 1 -- 172.21.15.77:0/852799055 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffbac1047e0 msgr2=0x7ffbac19a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:55.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.874+0000 7ffbb1a00640 1 --2- 172.21.15.77:0/852799055 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffbac1047e0 0x7ffbac19a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:55.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.874+0000 7ffbb1a00640 1 -- 172.21.15.77:0/852799055 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffbac108f60 msgr2=0x7ffbac19ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:55.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.874+0000 7ffbb1a00640 1 --2- 172.21.15.77:0/852799055 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffbac108f60 0x7ffbac19ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:55.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.874+0000 7ffbb1a00640 1 -- 172.21.15.77:0/852799055 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbac19f3b0 con 0x7ffbac103e30 2024-09-10T10:01:55.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.874+0000 7ffbb2400640 1 --2- 172.21.15.77:0/852799055 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffbac108f60 0x7ffbac19ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:01:55.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.874+0000 7ffbb1a00640 1 --2- 172.21.15.77:0/852799055 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffbac103e30 0x7ffbac19a400 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7ffba000ebc0 tx=0x7ffba000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:55.877 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.875+0000 7ffbaa000640 1 -- 172.21.15.77:0/852799055 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffba0010630 con 0x7ffbac103e30 2024-09-10T10:01:55.877 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.875+0000 7ffbaa000640 1 -- 172.21.15.77:0/852799055 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffba0010cb0 con 0x7ffbac103e30 2024-09-10T10:01:55.877 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.875+0000 7ffbb2e00640 1 -- 172.21.15.77:0/852799055 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbac19f6a0 con 0x7ffbac103e30 2024-09-10T10:01:55.877 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.875+0000 7ffbaa000640 1 -- 172.21.15.77:0/852799055 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffba0018bd0 con 0x7ffbac103e30 2024-09-10T10:01:55.877 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.875+0000 7ffbb2e00640 1 -- 172.21.15.77:0/852799055 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbac1a6f80 con 0x7ffbac103e30 2024-09-10T10:01:55.878 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.878+0000 7ffbaa000640 1 -- 172.21.15.77:0/852799055 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ffba00107d0 con 0x7ffbac103e30 2024-09-10T10:01:55.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.878+0000 7ffbb707c740 1 Processor -- start 2024-09-10T10:01:55.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.878+0000 7ffbb707c740 1 -- start start 2024-09-10T10:01:55.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.879+0000 7ffbb707c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564e99193e40 0x564e99194220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:55.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.879+0000 7ffbb707c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564e99194760 0x564e9919eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:55.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.879+0000 7ffbb707c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564e9919f020 0x564e991a1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:55.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.879+0000 7ffbb707c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x564e990fa3e0 con 0x564e99193e40 2024-09-10T10:01:55.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.879+0000 7ffbb707c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x564e990a1d80 con 0x564e99194760 2024-09-10T10:01:55.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.879+0000 7ffbb707c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x564e990f63a0 con 0x564e9919f020 2024-09-10T10:01:55.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.879+0000 7ffbaa000640 1 --2- 172.21.15.77:0/852799055 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ffb940a2b80 0x7ffb940a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:55.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.879+0000 7ffbb2400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564e99193e40 0x564e99194220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:55.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.879+0000 7ffbb1a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564e99194760 0x564e9919eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:55.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.879+0000 7ffbb1000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564e9919f020 0x564e991a1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:55.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.879+0000 7ffbb1a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564e99194760 0x564e9919eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:46668/0 (socket says 172.21.15.77:46668) 2024-09-10T10:01:55.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.879+0000 7ffbb1a00640 1 -- 172.21.15.77:0/1443903001 learned_addr learned my addr 172.21.15.77:0/1443903001 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:01:55.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.879+0000 7ffbaa000640 1 -- 172.21.15.77:0/852799055 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7ffba00cfab0 con 0x7ffbac103e30 2024-09-10T10:01:55.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.879+0000 7ffbb2400640 1 --2- 172.21.15.77:0/852799055 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ffb940a2b80 0x7ffb940a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:55.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.880+0000 7ffbb1a00640 1 -- 172.21.15.77:0/1443903001 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564e9919f020 msgr2=0x564e991a1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:55.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.880+0000 7ffbb1a00640 1 --2- 172.21.15.77:0/1443903001 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564e9919f020 0x564e991a1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:55.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.880+0000 7ffbb1a00640 1 -- 172.21.15.77:0/1443903001 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564e99193e40 msgr2=0x564e99194220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:55.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.880+0000 7ffbb1a00640 1 --2- 172.21.15.77:0/1443903001 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564e99193e40 0x564e99194220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:55.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.880+0000 7ffbb1a00640 1 -- 172.21.15.77:0/1443903001 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e990c0ae0 con 0x564e99194760 2024-09-10T10:01:55.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.880+0000 7ffbb1a00640 1 --2- 172.21.15.77:0/1443903001 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564e99194760 0x564e9919eae0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ffba00983d0 tx=0x7ffba006f030 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47413d567b404e31 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:55.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.880+0000 7ffb9ea00640 1 -- 172.21.15.77:0/1443903001 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffba0010e60 con 0x564e99194760 2024-09-10T10:01:55.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.880+0000 7ffb9ea00640 1 -- 172.21.15.77:0/1443903001 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffba00a0270 con 0x564e99194760 2024-09-10T10:01:55.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.880+0000 7ffb9ea00640 1 -- 172.21.15.77:0/1443903001 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffba00985e0 con 0x564e99194760 2024-09-10T10:01:55.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.881+0000 7ffbb707c740 1 -- 172.21.15.77:0/1443903001 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564e99194760 msgr2=0x564e9919eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:55.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.881+0000 7ffbb2400640 1 --2- 172.21.15.77:0/852799055 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ffb940a2b80 0x7ffb940a5040 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ffba4031550 tx=0x7ffba402a3f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:55.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.881+0000 7ffbb707c740 1 --2- 172.21.15.77:0/1443903001 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564e99194760 0x564e9919eae0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ffba00983d0 tx=0x7ffba006f030 comp rx=0 tx=0).stop 2024-09-10T10:01:55.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.881+0000 7ffbb707c740 1 -- 172.21.15.77:0/1443903001 shutdown_connections 2024-09-10T10:01:55.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.881+0000 7ffbb707c740 1 --2- 172.21.15.77:0/1443903001 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564e9919f020 0x564e991a1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:55.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.881+0000 7ffbb707c740 1 --2- 172.21.15.77:0/1443903001 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564e99194760 0x564e9919eae0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:55.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.881+0000 7ffbb707c740 1 --2- 172.21.15.77:0/1443903001 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564e99193e40 0x564e99194220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:55.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.881+0000 7ffbb707c740 1 -- 172.21.15.77:0/1443903001 >> 172.21.15.77:0/1443903001 conn(0x564e991903d0 msgr2=0x564e991907b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:55.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.881+0000 7ffbb707c740 1 -- 172.21.15.77:0/1443903001 shutdown_connections 2024-09-10T10:01:55.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.881+0000 7ffbb707c740 1 -- 172.21.15.77:0/1443903001 wait complete. 2024-09-10T10:01:55.886 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.881+0000 7ffbb707c740 1 -- start start 2024-09-10T10:01:55.886 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.882+0000 7ffbb707c740 1 Processor -- start 2024-09-10T10:01:55.886 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.882+0000 7ffbb707c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564e99193e40 0x564e9919beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:55.886 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.882+0000 7ffbb707c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564e99194760 0x564e9919c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:55.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.882+0000 7ffbb707c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564e9919f020 0x564e9919c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:55.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.882+0000 7ffbb707c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x564e991a1bf0 con 0x564e9919f020 2024-09-10T10:01:55.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.882+0000 7ffbb707c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x564e990b50f0 con 0x564e99194760 2024-09-10T10:01:55.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.882+0000 7ffbb707c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x564e991a1d70 con 0x564e99193e40 2024-09-10T10:01:55.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.882+0000 7ffbb707c740 10 client.0 _ll_register_callbacks cb 0x564e9909cbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:01:55.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.882+0000 7ffbb2400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564e9919f020 0x564e9919c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:55.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.882+0000 7ffbb1a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564e99193e40 0x564e9919beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:55.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.883+0000 7ffbb1a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564e99193e40 0x564e9919beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:52016/0 (socket says 172.21.15.77:52016) 2024-09-10T10:01:55.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.883+0000 7ffbb1a00640 1 -- 172.21.15.77:0/3817731459 learned_addr learned my addr 172.21.15.77:0/3817731459 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:01:55.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.883+0000 7ffbb1000640 1 --2- 172.21.15.77:0/3817731459 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564e99194760 0x564e9919c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:55.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.883+0000 7ffbb1a00640 1 -- 172.21.15.77:0/3817731459 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564e99194760 msgr2=0x564e9919c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:55.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.883+0000 7ffbb1a00640 1 --2- 172.21.15.77:0/3817731459 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564e99194760 0x564e9919c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:55.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.883+0000 7ffbb1a00640 1 -- 172.21.15.77:0/3817731459 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564e9919f020 msgr2=0x564e9919c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:55.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.883+0000 7ffbb1a00640 1 --2- 172.21.15.77:0/3817731459 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564e9919f020 0x564e9919c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:55.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.883+0000 7ffbb1a00640 1 -- 172.21.15.77:0/3817731459 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e98e6b3c0 con 0x564e99193e40 2024-09-10T10:01:55.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.883+0000 7ffbb1a00640 1 --2- 172.21.15.77:0/3817731459 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564e99193e40 0x564e9919beb0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ffba0000f80 tx=0x7ffba009a3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=496ccfcbe7cac6e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:55.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.883+0000 7ffb93400640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-10T10:01:55.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.883+0000 7ffb93400640 1 -- 172.21.15.77:0/3817731459 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffba0010e60 con 0x564e99193e40 2024-09-10T10:01:55.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.883+0000 7ffbb707c740 1 -- 172.21.15.77:0/3817731459 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffba00a0270 con 0x564e99193e40 2024-09-10T10:01:55.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.883+0000 7ffb93400640 1 -- 172.21.15.77:0/3817731459 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7ffba009c3d0 con 0x564e99193e40 2024-09-10T10:01:55.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.884+0000 7ffb93400640 1 -- 172.21.15.77:0/3817731459 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7ffba00a4b30 con 0x564e99193e40 2024-09-10T10:01:55.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.884+0000 7ffbb707c740 10 client.26824 fetch_fsmap learned FSMap version 35 2024-09-10T10:01:55.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.884+0000 7ffbb707c740 1 -- 172.21.15.77:0/3817731459 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x564e991a1bf0 con 0x564e99193e40 2024-09-10T10:01:55.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.884+0000 7ffb93400640 1 -- 172.21.15.77:0/3817731459 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffba00a0270 con 0x564e99193e40 2024-09-10T10:01:55.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.884+0000 7ffb93400640 1 -- 172.21.15.77:0/3817731459 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7ffba0096ae0 con 0x564e99193e40 2024-09-10T10:01:55.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.884+0000 7ffbb707c740 10 client.26824 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T10:01:55.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.884+0000 7ffbb707c740 10 client.26824 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T10:01:55.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.884+0000 7ffbb707c740 20 client.26824 populate_metadata read hostname 'smithi077' 2024-09-10T10:01:55.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.884+0000 7ffbb707c740 1 --2- 172.21.15.77:0/3817731459 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x564e991aed40 0x564e991b1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:55.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.884+0000 7ffbb707c740 4 client.26824 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:01:55.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.884+0000 7ffbb707c740 1 -- 172.21.15.77:0/3817731459 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564e99107950 con 0x564e991aed40 2024-09-10T10:01:55.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.885+0000 7ffbb1000640 1 --2- 172.21.15.77:0/3817731459 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x564e991aed40 0x564e991b1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:55.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.885+0000 7ffb93400640 10 client.26824 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T10:01:55.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.885+0000 7ffbb1000640 1 --2- 172.21.15.77:0/3817731459 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x564e991aed40 0x564e991b1130 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2eecee2cce96fe2 server_cookie=67341729fceae23f in_seq=0 out_seq=0 2024-09-10T10:01:55.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.886+0000 7ffb93400640 1 -- 172.21.15.77:0/3817731459 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564e99107950 con 0x564e991aed40 2024-09-10T10:01:55.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.886+0000 7ffb93400640 10 client.26824 handle_command_reply: tid=0 2024-09-10T10:01:55.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.886+0000 7ffbb707c740 1 client.26824 shutdown 2024-09-10T10:01:55.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.887+0000 7ffbb707c740 1 -- 172.21.15.77:0/3817731459 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564e99193e40 msgr2=0x564e9919beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:55.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.887+0000 7ffbb707c740 1 --2- 172.21.15.77:0/3817731459 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564e99193e40 0x564e9919beb0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ffba0000f80 tx=0x7ffba009a3c0 comp rx=0 tx=0).stop 2024-09-10T10:01:55.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.887+0000 7ffbb707c740 1 -- 172.21.15.77:0/3817731459 shutdown_connections 2024-09-10T10:01:55.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.887+0000 7ffbb707c740 1 --2- 172.21.15.77:0/3817731459 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x564e991aed40 0x564e991b1130 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:55.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.887+0000 7ffbb707c740 1 --2- 172.21.15.77:0/3817731459 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564e9919f020 0x564e9919c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:55.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.887+0000 7ffbb707c740 1 --2- 172.21.15.77:0/3817731459 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564e99194760 0x564e9919c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:55.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.887+0000 7ffbb707c740 1 --2- 172.21.15.77:0/3817731459 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564e99193e40 0x564e9919beb0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:55.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.887+0000 7ffb93400640 0 client.26824 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T10:01:55.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.887+0000 7ffbb707c740 1 -- 172.21.15.77:0/3817731459 >> 172.21.15.77:0/3817731459 conn(0x564e991903d0 msgr2=0x564e99196100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:55.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.887+0000 7ffbb707c740 1 -- 172.21.15.77:0/3817731459 shutdown_connections 2024-09-10T10:01:55.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.887+0000 7ffbb707c740 1 -- 172.21.15.77:0/3817731459 wait complete. 2024-09-10T10:01:55.892 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.887+0000 7ffbb707c740 20 client.26824 trim_cache size 0 max 16384 2024-09-10T10:01:55.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.901+0000 7ffbb707c740 1 Processor -- start 2024-09-10T10:01:55.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.901+0000 7ffbb707c740 1 -- start start 2024-09-10T10:01:55.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.901+0000 7ffbb707c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564e99193e40 0x564e991968a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:55.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.901+0000 7ffbb707c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564e99194760 0x564e991b6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:55.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.901+0000 7ffbb707c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564e9919f020 0x564e991b6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:55.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.901+0000 7ffbb707c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ffbac1b0ec0 con 0x564e99193e40 2024-09-10T10:01:55.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.901+0000 7ffbb707c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ffbac1b2a00 con 0x564e99194760 2024-09-10T10:01:55.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.901+0000 7ffbb707c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ffbac1b1590 con 0x564e9919f020 2024-09-10T10:01:55.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.901+0000 7ffbb1a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564e99193e40 0x564e991968a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:55.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.901+0000 7ffbb1a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564e99193e40 0x564e991968a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:51430/0 (socket says 172.21.15.77:51430) 2024-09-10T10:01:55.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.901+0000 7ffbb1a00640 1 -- 172.21.15.77:0/1713110959 learned_addr learned my addr 172.21.15.77:0/1713110959 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:01:55.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.901+0000 7ffbb2400640 1 --2- 172.21.15.77:0/1713110959 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564e9919f020 0x564e991b6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:55.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.901+0000 7ffbb1000640 1 --2- 172.21.15.77:0/1713110959 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564e99194760 0x564e991b6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:55.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.902+0000 7ffbb1a00640 1 -- 172.21.15.77:0/1713110959 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564e9919f020 msgr2=0x564e991b6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:55.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.902+0000 7ffbb1a00640 1 --2- 172.21.15.77:0/1713110959 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564e9919f020 0x564e991b6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:55.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.902+0000 7ffbb1a00640 1 -- 172.21.15.77:0/1713110959 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564e99194760 msgr2=0x564e991b6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:55.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.902+0000 7ffbb1a00640 1 --2- 172.21.15.77:0/1713110959 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564e99194760 0x564e991b6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:55.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.902+0000 7ffbb1a00640 1 -- 172.21.15.77:0/1713110959 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e99190200 con 0x564e99193e40 2024-09-10T10:01:55.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.902+0000 7ffbb1a00640 1 --2- 172.21.15.77:0/1713110959 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564e99193e40 0x564e991968a0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7ffba000cba0 tx=0x7ffba009bd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f92f8067568c012 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:55.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.902+0000 7ffb9e000640 1 -- 172.21.15.77:0/1713110959 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffba0010e60 con 0x564e99193e40 2024-09-10T10:01:55.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.902+0000 7ffb9e000640 1 -- 172.21.15.77:0/1713110959 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffba00a0270 con 0x564e99193e40 2024-09-10T10:01:55.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.902+0000 7ffb9e000640 1 -- 172.21.15.77:0/1713110959 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffba00a46a0 con 0x564e99193e40 2024-09-10T10:01:55.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.903+0000 7ffbb707c740 1 -- 172.21.15.77:0/1713110959 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564e99193e40 msgr2=0x564e991968a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:55.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.903+0000 7ffbb707c740 1 --2- 172.21.15.77:0/1713110959 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564e99193e40 0x564e991968a0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7ffba000cba0 tx=0x7ffba009bd40 comp rx=0 tx=0).stop 2024-09-10T10:01:55.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.903+0000 7ffbb707c740 1 -- 172.21.15.77:0/1713110959 shutdown_connections 2024-09-10T10:01:55.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.903+0000 7ffbb707c740 1 --2- 172.21.15.77:0/1713110959 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564e9919f020 0x564e991b6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:55.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.903+0000 7ffbb707c740 1 --2- 172.21.15.77:0/1713110959 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564e99194760 0x564e991b6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:55.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.903+0000 7ffbb707c740 1 --2- 172.21.15.77:0/1713110959 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564e99193e40 0x564e991968a0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:55.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.903+0000 7ffbb707c740 1 -- 172.21.15.77:0/1713110959 >> 172.21.15.77:0/1713110959 conn(0x564e991903d0 msgr2=0x564e9919df30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:55.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.903+0000 7ffbb707c740 1 -- 172.21.15.77:0/1713110959 shutdown_connections 2024-09-10T10:01:55.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.903+0000 7ffbb707c740 1 -- 172.21.15.77:0/1713110959 wait complete. 2024-09-10T10:01:55.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.904+0000 7ffbb707c740 1 -- start start 2024-09-10T10:01:55.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.904+0000 7ffbb707c740 1 Processor -- start 2024-09-10T10:01:55.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.904+0000 7ffbb707c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564e99193e40 0x564e991a7810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:55.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.904+0000 7ffbb707c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564e99194760 0x564e99198cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:55.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.904+0000 7ffbb707c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564e9919f020 0x564e991992a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:55.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.904+0000 7ffbb707c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ffbac1b2330 con 0x564e99194760 2024-09-10T10:01:55.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.904+0000 7ffbb707c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ffbac1b1c60 con 0x564e9919f020 2024-09-10T10:01:55.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.904+0000 7ffbb707c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ffbac1aeea0 con 0x564e99193e40 2024-09-10T10:01:55.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.904+0000 7ffbb707c740 10 client.0 _ll_register_callbacks cb 0x7ffba009a270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:01:55.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.905+0000 7ffbb1000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564e99194760 0x564e99198cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:55.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.905+0000 7ffbb1000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564e99194760 0x564e99198cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:51442/0 (socket says 172.21.15.77:51442) 2024-09-10T10:01:55.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.905+0000 7ffbb2400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564e9919f020 0x564e991992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:55.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.905+0000 7ffbb1000640 1 -- 172.21.15.77:0/3430402036 learned_addr learned my addr 172.21.15.77:0/3430402036 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:01:55.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.905+0000 7ffbb1a00640 1 --2- 172.21.15.77:0/3430402036 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564e99193e40 0x564e991a7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:55.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.905+0000 7ffbb1000640 1 -- 172.21.15.77:0/3430402036 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564e99193e40 msgr2=0x564e991a7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:55.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.905+0000 7ffbb1000640 1 --2- 172.21.15.77:0/3430402036 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564e99193e40 0x564e991a7810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:55.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.905+0000 7ffbb1000640 1 -- 172.21.15.77:0/3430402036 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564e9919f020 msgr2=0x564e991992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:55.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.905+0000 7ffbb1000640 1 --2- 172.21.15.77:0/3430402036 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564e9919f020 0x564e991992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:55.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.905+0000 7ffbb1000640 1 -- 172.21.15.77:0/3430402036 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e99199ad0 con 0x564e99194760 2024-09-10T10:01:55.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.905+0000 7ffbb2400640 1 --2- 172.21.15.77:0/3430402036 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564e9919f020 0x564e991992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:01:55.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.905+0000 7ffbb1000640 1 --2- 172.21.15.77:0/3430402036 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564e99194760 0x564e99198cd0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7ffb98005940 tx=0x7ffb98011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecdc56dd3827f7fa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:55.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.905+0000 7ffb93400640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-10T10:01:55.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.906+0000 7ffb93400640 1 -- 172.21.15.77:0/3430402036 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb98004020 con 0x564e99194760 2024-09-10T10:01:55.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.906+0000 7ffb93400640 1 -- 172.21.15.77:0/3430402036 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7ffb980041c0 con 0x564e99194760 2024-09-10T10:01:55.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.906+0000 7ffbb707c740 1 -- 172.21.15.77:0/3430402036 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffba00a0270 con 0x564e99194760 2024-09-10T10:01:55.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.906+0000 7ffb93400640 1 -- 172.21.15.77:0/3430402036 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7ffb980043c0 con 0x564e99194760 2024-09-10T10:01:55.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.906+0000 7ffbb707c740 10 client.16755 fetch_fsmap learned FSMap version 35 2024-09-10T10:01:55.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.906+0000 7ffbb707c740 1 -- 172.21.15.77:0/3430402036 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x564e99219890 con 0x564e99194760 2024-09-10T10:01:55.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.906+0000 7ffb93400640 1 -- 172.21.15.77:0/3430402036 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffba00a0270 con 0x564e99194760 2024-09-10T10:01:55.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.907+0000 7ffb93400640 1 -- 172.21.15.77:0/3430402036 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7ffb98005570 con 0x564e99194760 2024-09-10T10:01:55.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.907+0000 7ffbb707c740 10 client.16755 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T10:01:55.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.907+0000 7ffbb707c740 10 client.16755 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T10:01:55.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.907+0000 7ffbb707c740 20 client.16755 populate_metadata read hostname 'smithi077' 2024-09-10T10:01:55.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.907+0000 7ffbb707c740 1 --2- 172.21.15.77:0/3430402036 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x564e99219fe0 0x564e9921c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:55.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.907+0000 7ffbb707c740 4 client.16755 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:01:55.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.907+0000 7ffbb707c740 1 -- 172.21.15.77:0/3430402036 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564e99101e60 con 0x564e99219fe0 2024-09-10T10:01:55.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.907+0000 7ffbb1a00640 1 --2- 172.21.15.77:0/3430402036 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x564e99219fe0 0x564e9921c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:55.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.908+0000 7ffbb1a00640 1 --2- 172.21.15.77:0/3430402036 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x564e99219fe0 0x564e9921c400 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76ad90df1f4fbf70 server_cookie=c98433e9e6cb1ed6 in_seq=0 out_seq=0 2024-09-10T10:01:55.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.908+0000 7ffb93400640 10 client.16755 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T10:01:55.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.908+0000 7ffb93400640 1 -- 172.21.15.77:0/3430402036 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564e99101e60 con 0x564e99219fe0 2024-09-10T10:01:55.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.908+0000 7ffb93400640 10 client.16755 handle_command_reply: tid=0 2024-09-10T10:01:55.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.908+0000 7ffbb707c740 1 client.16755 shutdown 2024-09-10T10:01:55.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.908+0000 7ffbb707c740 1 -- 172.21.15.77:0/3430402036 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564e99194760 msgr2=0x564e99198cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:55.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.908+0000 7ffbb707c740 1 --2- 172.21.15.77:0/3430402036 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564e99194760 0x564e99198cd0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7ffb98005940 tx=0x7ffb98011940 comp rx=0 tx=0).stop 2024-09-10T10:01:55.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.908+0000 7ffbb707c740 1 -- 172.21.15.77:0/3430402036 shutdown_connections 2024-09-10T10:01:55.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.908+0000 7ffbb707c740 1 --2- 172.21.15.77:0/3430402036 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x564e99219fe0 0x564e9921c400 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:55.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.908+0000 7ffbb707c740 1 --2- 172.21.15.77:0/3430402036 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564e9919f020 0x564e991992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:55.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.908+0000 7ffbb707c740 1 --2- 172.21.15.77:0/3430402036 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564e99194760 0x564e99198cd0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:55.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.908+0000 7ffbb707c740 1 --2- 172.21.15.77:0/3430402036 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564e99193e40 0x564e991a7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:55.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.908+0000 7ffbb707c740 1 -- 172.21.15.77:0/3430402036 >> 172.21.15.77:0/3430402036 conn(0x564e991903d0 msgr2=0x564e9918ec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:55.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.908+0000 7ffb93400640 0 client.16755 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T10:01:55.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.908+0000 7ffbb707c740 1 -- 172.21.15.77:0/3430402036 shutdown_connections 2024-09-10T10:01:55.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.908+0000 7ffbb707c740 1 -- 172.21.15.77:0/3430402036 wait complete. 2024-09-10T10:01:55.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.908+0000 7ffbb707c740 20 client.16755 trim_cache size 0 max 16384 2024-09-10T10:01:55.913 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-10T10:01:55.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.909+0000 7ffbb2e00640 1 -- 172.21.15.77:0/852799055 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ffb940a2b80 msgr2=0x7ffb940a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:55.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.909+0000 7ffbb2e00640 1 --2- 172.21.15.77:0/852799055 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ffb940a2b80 0x7ffb940a5040 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ffba4031550 tx=0x7ffba402a3f0 comp rx=0 tx=0).stop 2024-09-10T10:01:55.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.909+0000 7ffbb2e00640 1 -- 172.21.15.77:0/852799055 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffbac103e30 msgr2=0x7ffbac19a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:55.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.909+0000 7ffbb2e00640 1 --2- 172.21.15.77:0/852799055 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffbac103e30 0x7ffbac19a400 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7ffba000ebc0 tx=0x7ffba000c6a0 comp rx=0 tx=0).stop 2024-09-10T10:01:55.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.909+0000 7ffbb2e00640 1 -- 172.21.15.77:0/852799055 shutdown_connections 2024-09-10T10:01:55.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.909+0000 7ffbb2e00640 1 --2- 172.21.15.77:0/852799055 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ffb940a2b80 0x7ffb940a5040 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:55.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.909+0000 7ffbb2e00640 1 --2- 172.21.15.77:0/852799055 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffbac108f60 0x7ffbac19ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:55.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.909+0000 7ffbb2e00640 1 --2- 172.21.15.77:0/852799055 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffbac1047e0 0x7ffbac19a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:55.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.909+0000 7ffbb2e00640 1 --2- 172.21.15.77:0/852799055 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffbac103e30 0x7ffbac19a400 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:55.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.909+0000 7ffbb2e00640 1 -- 172.21.15.77:0/852799055 >> 172.21.15.77:0/852799055 conn(0x7ffbac0ff8d0 msgr2=0x7ffbac100bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:55.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.909+0000 7ffbb2e00640 1 -- 172.21.15.77:0/852799055 shutdown_connections 2024-09-10T10:01:55.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:55.909+0000 7ffbb2e00640 1 -- 172.21.15.77:0/852799055 wait complete. 2024-09-10T10:01:56.135 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:01:55 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/1687544039' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:01:56.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:01:55 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/1687544039' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:01:56.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:01:55 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/1687544039' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:01:56.404 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24563 quiesce db --set-id f49f355b --release --await' 2024-09-10T10:01:56.404 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24563 quiesce db --set-id f49f355b --release --await 2024-09-10T10:01:57.105 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:01:56 smithi077 ceph-mon[30319]: pgmap v594: 129 pgs: 129 active+clean; 146 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 43 KiB/s wr, 10 op/s 2024-09-10T10:01:57.213 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:01:57.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:01:56 smithi134 ceph-mon[35674]: pgmap v594: 129 pgs: 129 active+clean; 146 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 43 KiB/s wr, 10 op/s 2024-09-10T10:01:57.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:01:56 smithi120 ceph-mon[34770]: pgmap v594: 129 pgs: 129 active+clean; 146 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 43 KiB/s wr, 10 op/s 2024-09-10T10:01:57.751 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.750+0000 7f6e11600640 1 -- 172.21.15.77:0/3036957354 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6e0c074510 msgr2=0x7f6e0c0748f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:57.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.750+0000 7f6e11600640 1 --2- 172.21.15.77:0/3036957354 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6e0c074510 0x7f6e0c0748f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc009900 tx=0x7f6dfc02a480 comp rx=0 tx=0).stop 2024-09-10T10:01:57.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.750+0000 7f6e11600640 1 -- 172.21.15.77:0/3036957354 shutdown_connections 2024-09-10T10:01:57.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.750+0000 7f6e11600640 1 --2- 172.21.15.77:0/3036957354 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6e0c10e440 0x7f6e0c110830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:57.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.750+0000 7f6e11600640 1 --2- 172.21.15.77:0/3036957354 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6e0c074e30 0x7f6e0c10df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:57.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.750+0000 7f6e11600640 1 --2- 172.21.15.77:0/3036957354 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6e0c074510 0x7f6e0c0748f0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:57.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.750+0000 7f6e11600640 1 -- 172.21.15.77:0/3036957354 >> 172.21.15.77:0/3036957354 conn(0x7f6e0c0fd8c0 msgr2=0x7f6e0c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:57.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.750+0000 7f6e11600640 1 -- 172.21.15.77:0/3036957354 shutdown_connections 2024-09-10T10:01:57.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.750+0000 7f6e11600640 1 -- 172.21.15.77:0/3036957354 wait complete. 2024-09-10T10:01:57.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.751+0000 7f6e11600640 1 Processor -- start 2024-09-10T10:01:57.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.751+0000 7f6e11600640 1 -- start start 2024-09-10T10:01:57.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.752+0000 7f6e11600640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6e0c074510 0x7f6e0c19a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:57.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.752+0000 7f6e11600640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6e0c074e30 0x7f6e0c19abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:57.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.752+0000 7f6e11600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6e0c10e440 0x7f6e0c19ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:57.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.752+0000 7f6e11600640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6e0c1141c0 con 0x7f6e0c074e30 2024-09-10T10:01:57.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.752+0000 7f6e11600640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6e0c114040 con 0x7f6e0c074510 2024-09-10T10:01:57.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.752+0000 7f6e11600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6e0c114340 con 0x7f6e0c10e440 2024-09-10T10:01:57.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.752+0000 7f6e03e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6e0c074e30 0x7f6e0c19abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:57.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.752+0000 7f6e0aa00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6e0c074510 0x7f6e0c19a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:57.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.752+0000 7f6e0b400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6e0c10e440 0x7f6e0c19ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:57.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.752+0000 7f6e0aa00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6e0c074510 0x7f6e0c19a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:46720/0 (socket says 172.21.15.77:46720) 2024-09-10T10:01:57.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.752+0000 7f6e03e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6e0c074e30 0x7f6e0c19abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:51464/0 (socket says 172.21.15.77:51464) 2024-09-10T10:01:57.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.752+0000 7f6e0aa00640 1 -- 172.21.15.77:0/1444200643 learned_addr learned my addr 172.21.15.77:0/1444200643 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:01:57.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.753+0000 7f6e0aa00640 1 -- 172.21.15.77:0/1444200643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6e0c10e440 msgr2=0x7f6e0c19ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:57.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.753+0000 7f6e0aa00640 1 --2- 172.21.15.77:0/1444200643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6e0c10e440 0x7f6e0c19ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:57.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.753+0000 7f6e0aa00640 1 -- 172.21.15.77:0/1444200643 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6e0c074e30 msgr2=0x7f6e0c19abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:57.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.753+0000 7f6e0aa00640 1 --2- 172.21.15.77:0/1444200643 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6e0c074e30 0x7f6e0c19abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:57.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.753+0000 7f6e0aa00640 1 -- 172.21.15.77:0/1444200643 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e0c19f650 con 0x7f6e0c074510 2024-09-10T10:01:57.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.753+0000 7f6e0b400640 1 --2- 172.21.15.77:0/1444200643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6e0c10e440 0x7f6e0c19ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:01:57.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.753+0000 7f6e03e00640 1 --2- 172.21.15.77:0/1444200643 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6e0c074e30 0x7f6e0c19abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:01:57.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.753+0000 7f6e0aa00640 1 --2- 172.21.15.77:0/1444200643 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6e0c074510 0x7f6e0c19a6a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc004770 tx=0x7f6dfc003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:57.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.753+0000 7f6e03400640 1 -- 172.21.15.77:0/1444200643 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dfc03f070 con 0x7f6e0c074510 2024-09-10T10:01:57.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.753+0000 7f6e11600640 1 -- 172.21.15.77:0/1444200643 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e0c19f8e0 con 0x7f6e0c074510 2024-09-10T10:01:57.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.753+0000 7f6e03400640 1 -- 172.21.15.77:0/1444200643 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6dfc031ca0 con 0x7f6e0c074510 2024-09-10T10:01:57.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.753+0000 7f6e03400640 1 -- 172.21.15.77:0/1444200643 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dfc035030 con 0x7f6e0c074510 2024-09-10T10:01:57.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.753+0000 7f6e11600640 1 -- 172.21.15.77:0/1444200643 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e0c1a7220 con 0x7f6e0c074510 2024-09-10T10:01:57.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.757+0000 7f6e14063740 1 Processor -- start 2024-09-10T10:01:57.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.757+0000 7f6e14063740 1 -- start start 2024-09-10T10:01:57.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.758+0000 7f6e14063740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5564a4107310 0x5564a41076f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:57.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.758+0000 7f6e14063740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5564a4107c30 0x5564a4111fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:57.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.758+0000 7f6e14063740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5564a41124f0 0x5564a41148e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:57.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.758+0000 7f6e14063740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5564a406d650 con 0x5564a4107310 2024-09-10T10:01:57.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.758+0000 7f6e14063740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5564a4014f10 con 0x5564a4107c30 2024-09-10T10:01:57.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.758+0000 7f6e14063740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5564a4069610 con 0x5564a41124f0 2024-09-10T10:01:57.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.758+0000 7f6e03400640 1 -- 172.21.15.77:0/1444200643 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6dfc042070 con 0x7f6e0c074510 2024-09-10T10:01:57.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.758+0000 7f6e0b400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5564a4107310 0x5564a41076f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:57.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.758+0000 7f6e0b400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5564a4107310 0x5564a41076f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:51474/0 (socket says 172.21.15.77:51474) 2024-09-10T10:01:57.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.758+0000 7f6e0b400640 1 -- 172.21.15.77:0/3362789362 learned_addr learned my addr 172.21.15.77:0/3362789362 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:01:57.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.758+0000 7f6e03e00640 1 --2- 172.21.15.77:0/3362789362 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5564a41124f0 0x5564a41148e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:57.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.758+0000 7f6e0b400640 1 -- 172.21.15.77:0/3362789362 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5564a41124f0 msgr2=0x5564a41148e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:57.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.759+0000 7f6e0b400640 1 --2- 172.21.15.77:0/3362789362 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5564a41124f0 0x5564a41148e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:57.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.759+0000 7f6e0b400640 1 -- 172.21.15.77:0/3362789362 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5564a4107c30 msgr2=0x5564a4111fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:57.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.759+0000 7f6e0aa00640 1 --2- 172.21.15.77:0/3362789362 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5564a4107c30 0x5564a4111fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:57.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.759+0000 7f6e0b400640 1 --2- 172.21.15.77:0/3362789362 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5564a4107c30 0x5564a4111fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:57.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.759+0000 7f6e0b400640 1 -- 172.21.15.77:0/3362789362 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564a3d7c580 con 0x5564a4107310 2024-09-10T10:01:57.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.759+0000 7f6e0b400640 1 --2- 172.21.15.77:0/3362789362 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5564a4107310 0x5564a41076f0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f6e04011790 tx=0x7f6e04011c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b56c085ab99ce5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:57.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.759+0000 7f6dfb400640 1 -- 172.21.15.77:0/3362789362 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e0401e070 con 0x5564a4107310 2024-09-10T10:01:57.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.759+0000 7f6dfb400640 1 -- 172.21.15.77:0/3362789362 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e0400fc60 con 0x5564a4107310 2024-09-10T10:01:57.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.759+0000 7f6dfb400640 1 -- 172.21.15.77:0/3362789362 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e0400e610 con 0x5564a4107310 2024-09-10T10:01:57.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.760+0000 7f6e03400640 1 --2- 172.21.15.77:0/1444200643 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f6df00a2c00 0x7f6df00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:57.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.760+0000 7f6e03400640 1 -- 172.21.15.77:0/1444200643 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f6dfc0efa50 con 0x7f6e0c074510 2024-09-10T10:01:57.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.760+0000 7f6e0aa00640 1 --2- 172.21.15.77:0/1444200643 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f6df00a2c00 0x7f6df00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:57.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.760+0000 7f6e14063740 1 -- 172.21.15.77:0/3362789362 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5564a4107310 msgr2=0x5564a41076f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:57.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.760+0000 7f6e14063740 1 --2- 172.21.15.77:0/3362789362 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5564a4107310 0x5564a41076f0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f6e04011790 tx=0x7f6e04011c50 comp rx=0 tx=0).stop 2024-09-10T10:01:57.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.760+0000 7f6e14063740 1 -- 172.21.15.77:0/3362789362 shutdown_connections 2024-09-10T10:01:57.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.760+0000 7f6e14063740 1 --2- 172.21.15.77:0/3362789362 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5564a41124f0 0x5564a41148e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:57.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.760+0000 7f6e14063740 1 --2- 172.21.15.77:0/3362789362 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5564a4107c30 0x5564a4111fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:57.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.760+0000 7f6e14063740 1 --2- 172.21.15.77:0/3362789362 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5564a4107310 0x5564a41076f0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:57.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.760+0000 7f6e14063740 1 -- 172.21.15.77:0/3362789362 >> 172.21.15.77:0/3362789362 conn(0x5564a41038a0 msgr2=0x5564a4103c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:57.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.761+0000 7f6e14063740 1 -- 172.21.15.77:0/3362789362 shutdown_connections 2024-09-10T10:01:57.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.761+0000 7f6e14063740 1 -- 172.21.15.77:0/3362789362 wait complete. 2024-09-10T10:01:57.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.761+0000 7f6e0aa00640 1 --2- 172.21.15.77:0/1444200643 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f6df00a2c00 0x7f6df00a50c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc0bbcd0 tx=0x7f6dfc03b460 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:57.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.761+0000 7f6e14063740 1 -- start start 2024-09-10T10:01:57.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.761+0000 7f6e14063740 1 Processor -- start 2024-09-10T10:01:57.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.762+0000 7f6e14063740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5564a4107310 0x5564a410f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:57.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.762+0000 7f6e14063740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5564a4107c30 0x5564a410f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:57.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.762+0000 7f6e14063740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5564a41124f0 0x5564a410fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:57.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.762+0000 7f6e14063740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5564a41150c0 con 0x5564a41124f0 2024-09-10T10:01:57.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.762+0000 7f6e14063740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5564a4028350 con 0x5564a4107c30 2024-09-10T10:01:57.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.762+0000 7f6e14063740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5564a4115240 con 0x5564a4107310 2024-09-10T10:01:57.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.762+0000 7f6e14063740 10 client.0 _ll_register_callbacks cb 0x5564a404b290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:01:57.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.762+0000 7f6e0b400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5564a41124f0 0x5564a410fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:57.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.762+0000 7f6e03e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5564a4107c30 0x5564a410f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:57.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.762+0000 7f6e0b400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5564a41124f0 0x5564a410fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:51490/0 (socket says 172.21.15.77:51490) 2024-09-10T10:01:57.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.762+0000 7f6e0b400640 1 -- 172.21.15.77:0/3848262802 learned_addr learned my addr 172.21.15.77:0/3848262802 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:01:57.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.762+0000 7f6e03e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5564a4107c30 0x5564a410f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:46738/0 (socket says 172.21.15.77:46738) 2024-09-10T10:01:57.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.762+0000 7f6e0b400640 1 --2- 172.21.15.77:0/3848262802 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5564a4107310 0x5564a410f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:57.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.762+0000 7f6e0b400640 1 -- 172.21.15.77:0/3848262802 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5564a4107310 msgr2=0x5564a410f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:57.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.762+0000 7f6e0b400640 1 --2- 172.21.15.77:0/3848262802 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5564a4107310 0x5564a410f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:57.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.763+0000 7f6e0b400640 1 -- 172.21.15.77:0/3848262802 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5564a4107c30 msgr2=0x5564a410f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:57.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.763+0000 7f6e0b400640 1 --2- 172.21.15.77:0/3848262802 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5564a4107c30 0x5564a410f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:57.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.763+0000 7f6e0b400640 1 -- 172.21.15.77:0/3848262802 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564a3e46960 con 0x5564a41124f0 2024-09-10T10:01:57.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.763+0000 7f6e03e00640 1 --2- 172.21.15.77:0/3848262802 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5564a4107c30 0x5564a410f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:01:57.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.763+0000 7f6e0b400640 1 --2- 172.21.15.77:0/3848262802 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5564a41124f0 0x5564a410fe10 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f6e0400ffd0 tx=0x7f6e04019e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34cc5f13d7bcfb3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:57.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.763+0000 7f6defe00640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-10T10:01:57.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.763+0000 7f6defe00640 1 -- 172.21.15.77:0/3848262802 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e0402d060 con 0x5564a41124f0 2024-09-10T10:01:57.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.763+0000 7f6e14063740 1 -- 172.21.15.77:0/3848262802 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e0400fc60 con 0x5564a41124f0 2024-09-10T10:01:57.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.763+0000 7f6defe00640 1 -- 172.21.15.77:0/3848262802 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f6e04019660 con 0x5564a41124f0 2024-09-10T10:01:57.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.763+0000 7f6defe00640 1 -- 172.21.15.77:0/3848262802 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f6e04034050 con 0x5564a41124f0 2024-09-10T10:01:57.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.763+0000 7f6e14063740 10 client.16767 fetch_fsmap learned FSMap version 35 2024-09-10T10:01:57.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.763+0000 7f6e14063740 1 -- 172.21.15.77:0/3848262802 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5564a4120d20 con 0x5564a41124f0 2024-09-10T10:01:57.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.764+0000 7f6defe00640 1 -- 172.21.15.77:0/3848262802 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e0400fc60 con 0x5564a41124f0 2024-09-10T10:01:57.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.764+0000 7f6defe00640 1 -- 172.21.15.77:0/3848262802 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f6e04010ba0 con 0x5564a41124f0 2024-09-10T10:01:57.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.764+0000 7f6e14063740 10 client.16767 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T10:01:57.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.764+0000 7f6e14063740 10 client.16767 resolve_mds: validated gid 24563 aka daemon mds.b 2024-09-10T10:01:57.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.764+0000 7f6e14063740 20 client.16767 populate_metadata read hostname 'smithi077' 2024-09-10T10:01:57.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.764+0000 7f6e14063740 1 --2- 172.21.15.77:0/3848262802 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x5564a4122130 0x5564a4124520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:57.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.764+0000 7f6e14063740 4 client.16767 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:01:57.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.764+0000 7f6e14063740 1 -- 172.21.15.77:0/3848262802 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5564a407abc0 con 0x5564a4122130 2024-09-10T10:01:57.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.765+0000 7f6e03e00640 1 --2- 172.21.15.77:0/3848262802 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x5564a4122130 0x5564a4124520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:57.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.765+0000 7f6defe00640 10 client.16767 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T10:01:57.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.765+0000 7f6e03e00640 1 --2- 172.21.15.77:0/3848262802 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x5564a4122130 0x5564a4124520 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1804dfd76ff45515 server_cookie=bc11eba788c7a305 in_seq=0 out_seq=0 2024-09-10T10:01:57.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.766+0000 7f6defe00640 1 -- 172.21.15.77:0/3848262802 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5564a407abc0 con 0x5564a4122130 2024-09-10T10:01:57.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.766+0000 7f6defe00640 10 client.16767 handle_command_reply: tid=0 2024-09-10T10:01:57.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.766+0000 7f6e14063740 1 client.16767 shutdown 2024-09-10T10:01:57.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.766+0000 7f6e14063740 1 -- 172.21.15.77:0/3848262802 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5564a41124f0 msgr2=0x5564a410fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:57.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.766+0000 7f6e14063740 1 --2- 172.21.15.77:0/3848262802 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5564a41124f0 0x5564a410fe10 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f6e0400ffd0 tx=0x7f6e04019e70 comp rx=0 tx=0).stop 2024-09-10T10:01:57.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.766+0000 7f6e14063740 1 -- 172.21.15.77:0/3848262802 shutdown_connections 2024-09-10T10:01:57.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.766+0000 7f6e14063740 1 --2- 172.21.15.77:0/3848262802 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x5564a4122130 0x5564a4124520 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:57.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.766+0000 7f6e14063740 1 --2- 172.21.15.77:0/3848262802 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5564a41124f0 0x5564a410fe10 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:57.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.766+0000 7f6e14063740 1 --2- 172.21.15.77:0/3848262802 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5564a4107c30 0x5564a410f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:57.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.766+0000 7f6e14063740 1 --2- 172.21.15.77:0/3848262802 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5564a4107310 0x5564a410f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:57.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.766+0000 7f6e14063740 1 -- 172.21.15.77:0/3848262802 >> 172.21.15.77:0/3848262802 conn(0x5564a41038a0 msgr2=0x5564a4102aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:57.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.766+0000 7f6defe00640 0 client.16767 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T10:01:57.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.767+0000 7f6e14063740 1 -- 172.21.15.77:0/3848262802 shutdown_connections 2024-09-10T10:01:57.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.767+0000 7f6e14063740 1 -- 172.21.15.77:0/3848262802 wait complete. 2024-09-10T10:01:57.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.767+0000 7f6e14063740 20 client.16767 trim_cache size 0 max 16384 2024-09-10T10:01:57.785 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.784+0000 7f6e14063740 1 Processor -- start 2024-09-10T10:01:57.785 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.784+0000 7f6e14063740 1 -- start start 2024-09-10T10:01:57.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.785+0000 7f6e14063740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5564a4107310 0x5564a4109d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:57.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.785+0000 7f6e14063740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5564a4107c30 0x5564a4102aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:57.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.785+0000 7f6e14063740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5564a41124f0 0x5564a4129cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:57.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.785+0000 7f6e14063740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6e0c1b10e0 con 0x5564a4107c30 2024-09-10T10:01:57.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.785+0000 7f6e14063740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6e0c1b2c20 con 0x5564a4107310 2024-09-10T10:01:57.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.785+0000 7f6e14063740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6e0c1b17b0 con 0x5564a41124f0 2024-09-10T10:01:57.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.785+0000 7f6e03e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5564a4107c30 0x5564a4102aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:57.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.785+0000 7f6e0b400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5564a4107310 0x5564a4109d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:57.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.785+0000 7f6e0b400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5564a4107310 0x5564a4109d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:46752/0 (socket says 172.21.15.77:46752) 2024-09-10T10:01:57.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.785+0000 7f6e0b400640 1 -- 172.21.15.77:0/658603605 learned_addr learned my addr 172.21.15.77:0/658603605 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:01:57.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.785+0000 7f6e0b400640 1 -- 172.21.15.77:0/658603605 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5564a41124f0 msgr2=0x5564a4129cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:57.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.785+0000 7f6e0b400640 1 --2- 172.21.15.77:0/658603605 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5564a41124f0 0x5564a4129cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:57.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.785+0000 7f6e0b400640 1 -- 172.21.15.77:0/658603605 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5564a4107c30 msgr2=0x5564a4102aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:57.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.785+0000 7f6e0b400640 1 --2- 172.21.15.77:0/658603605 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5564a4107c30 0x5564a4102aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:57.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.785+0000 7f6e0b400640 1 -- 172.21.15.77:0/658603605 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564a41036d0 con 0x5564a4107310 2024-09-10T10:01:57.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.785+0000 7f6e03e00640 1 --2- 172.21.15.77:0/658603605 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5564a4107c30 0x5564a4102aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:01:57.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.786+0000 7f6e0b400640 1 --2- 172.21.15.77:0/658603605 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5564a4107310 0x5564a4109d30 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6e04032040 tx=0x7f6e04009520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42ec4433bf335ca1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:57.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.786+0000 7f6dfaa00640 1 -- 172.21.15.77:0/658603605 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e0401e030 con 0x5564a4107310 2024-09-10T10:01:57.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.786+0000 7f6dfaa00640 1 -- 172.21.15.77:0/658603605 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e0401f040 con 0x5564a4107310 2024-09-10T10:01:57.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.786+0000 7f6dfaa00640 1 -- 172.21.15.77:0/658603605 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e04003bd0 con 0x5564a4107310 2024-09-10T10:01:57.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.786+0000 7f6e14063740 1 -- 172.21.15.77:0/658603605 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5564a4107310 msgr2=0x5564a4109d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:57.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.787+0000 7f6e14063740 1 --2- 172.21.15.77:0/658603605 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5564a4107310 0x5564a4109d30 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6e04032040 tx=0x7f6e04009520 comp rx=0 tx=0).stop 2024-09-10T10:01:57.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.787+0000 7f6e14063740 1 -- 172.21.15.77:0/658603605 shutdown_connections 2024-09-10T10:01:57.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.787+0000 7f6e14063740 1 --2- 172.21.15.77:0/658603605 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5564a41124f0 0x5564a4129cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:57.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.787+0000 7f6e14063740 1 --2- 172.21.15.77:0/658603605 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5564a4107c30 0x5564a4102aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:57.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.787+0000 7f6e14063740 1 --2- 172.21.15.77:0/658603605 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5564a4107310 0x5564a4109d30 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:57.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.787+0000 7f6e14063740 1 -- 172.21.15.77:0/658603605 >> 172.21.15.77:0/658603605 conn(0x5564a41038a0 msgr2=0x5564a4111380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:57.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.787+0000 7f6e14063740 1 -- 172.21.15.77:0/658603605 shutdown_connections 2024-09-10T10:01:57.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.787+0000 7f6e14063740 1 -- 172.21.15.77:0/658603605 wait complete. 2024-09-10T10:01:57.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.787+0000 7f6e14063740 1 -- start start 2024-09-10T10:01:57.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.787+0000 7f6e14063740 1 Processor -- start 2024-09-10T10:01:57.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.788+0000 7f6e14063740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5564a4107310 0x5564a418d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:57.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.788+0000 7f6e14063740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5564a4107c30 0x5564a418db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:57.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.788+0000 7f6e14063740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5564a41124f0 0x5564a411a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:57.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.788+0000 7f6e14063740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6e0c1b2550 con 0x5564a4107310 2024-09-10T10:01:57.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.788+0000 7f6e14063740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6e0c1b1e80 con 0x5564a4107c30 2024-09-10T10:01:57.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.788+0000 7f6e14063740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6e0c1af0c0 con 0x5564a41124f0 2024-09-10T10:01:57.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.788+0000 7f6e14063740 10 client.0 _ll_register_callbacks cb 0x7f6e0401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:01:57.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.788+0000 7f6e03e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5564a4107c30 0x5564a418db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:57.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.788+0000 7f6e0b400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5564a4107310 0x5564a418d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:57.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.788+0000 7f6e03e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5564a4107c30 0x5564a418db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:46760/0 (socket says 172.21.15.77:46760) 2024-09-10T10:01:57.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.788+0000 7f6e03e00640 1 -- 172.21.15.77:0/3004823713 learned_addr learned my addr 172.21.15.77:0/3004823713 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:01:57.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.788+0000 7f6e0b400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5564a4107310 0x5564a418d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:51514/0 (socket says 172.21.15.77:51514) 2024-09-10T10:01:57.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.788+0000 7f6e0b400640 1 --2- 172.21.15.77:0/3004823713 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5564a41124f0 0x5564a411a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:57.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.788+0000 7f6e03e00640 1 -- 172.21.15.77:0/3004823713 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5564a41124f0 msgr2=0x5564a411a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:57.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.788+0000 7f6e03e00640 1 --2- 172.21.15.77:0/3004823713 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5564a41124f0 0x5564a411a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:57.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.788+0000 7f6e03e00640 1 -- 172.21.15.77:0/3004823713 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5564a4107310 msgr2=0x5564a418d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:57.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.788+0000 7f6e03e00640 1 --2- 172.21.15.77:0/3004823713 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5564a4107310 0x5564a418d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:57.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.788+0000 7f6e03e00640 1 -- 172.21.15.77:0/3004823713 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564a411a840 con 0x5564a4107c30 2024-09-10T10:01:57.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.789+0000 7f6e0b400640 1 --2- 172.21.15.77:0/3004823713 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5564a4107310 0x5564a418d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:01:57.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.789+0000 7f6e03e00640 1 --2- 172.21.15.77:0/3004823713 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5564a4107c30 0x5564a418db70 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x5564a4104c20 tx=0x7f6df40217e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5934362f8107ec5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:57.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.789+0000 7f6defe00640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T10:01:57.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.789+0000 7f6defe00640 1 -- 172.21.15.77:0/3004823713 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df4005020 con 0x5564a4107c30 2024-09-10T10:01:57.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.789+0000 7f6defe00640 1 -- 172.21.15.77:0/3004823713 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f6df40051c0 con 0x5564a4107c30 2024-09-10T10:01:57.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.789+0000 7f6e14063740 1 -- 172.21.15.77:0/3004823713 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e0401f040 con 0x5564a4107c30 2024-09-10T10:01:57.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.790+0000 7f6defe00640 1 -- 172.21.15.77:0/3004823713 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f6df4034050 con 0x5564a4107c30 2024-09-10T10:01:57.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.790+0000 7f6e14063740 10 client.26900 fetch_fsmap learned FSMap version 35 2024-09-10T10:01:57.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.790+0000 7f6e14063740 1 -- 172.21.15.77:0/3004823713 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5564a4194f50 con 0x5564a4107c30 2024-09-10T10:01:57.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.790+0000 7f6defe00640 1 -- 172.21.15.77:0/3004823713 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e0401f040 con 0x5564a4107c30 2024-09-10T10:01:57.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.790+0000 7f6defe00640 1 -- 172.21.15.77:0/3004823713 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f6df4010430 con 0x5564a4107c30 2024-09-10T10:01:57.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.790+0000 7f6e14063740 10 client.26900 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T10:01:57.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.790+0000 7f6e14063740 10 client.26900 resolve_mds: validated gid 24563 aka daemon mds.b 2024-09-10T10:01:57.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.790+0000 7f6e14063740 20 client.26900 populate_metadata read hostname 'smithi077' 2024-09-10T10:01:57.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.790+0000 7f6e14063740 1 --2- 172.21.15.77:0/3004823713 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x5564a4195700 0x5564a4197b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:57.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.790+0000 7f6e14063740 4 client.26900 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "f49f355b", "release": true, "await": true}] 2024-09-10T10:01:57.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.790+0000 7f6e14063740 1 -- 172.21.15.77:0/3004823713 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "quiesce db", "set_id": "f49f355b", "release": true, "await": true}) -- 0x5564a40750d0 con 0x5564a4195700 2024-09-10T10:01:57.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.791+0000 7f6e0b400640 1 --2- 172.21.15.77:0/3004823713 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x5564a4195700 0x5564a4197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:57.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.791+0000 7f6e0b400640 1 --2- 172.21.15.77:0/3004823713 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x5564a4195700 0x5564a4197b20 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=899a0f8f9c8146ab server_cookie=cb3bceb05a11717a in_seq=0 out_seq=0 2024-09-10T10:01:57.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.791+0000 7f6defe00640 10 client.26900 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T10:01:57.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.893+0000 7f6defe00640 1 -- 172.21.15.77:0/3004823713 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5564a40750d0 con 0x5564a4195700 2024-09-10T10:01:57.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.893+0000 7f6defe00640 10 client.26900 handle_command_reply: tid=0 2024-09-10T10:01:57.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.893+0000 7f6e14063740 1 client.26900 shutdown 2024-09-10T10:01:57.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.894+0000 7f6e14063740 1 -- 172.21.15.77:0/3004823713 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5564a4107c30 msgr2=0x5564a418db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:57.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.894+0000 7f6e14063740 1 --2- 172.21.15.77:0/3004823713 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5564a4107c30 0x5564a418db70 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x5564a4104c20 tx=0x7f6df40217e0 comp rx=0 tx=0).stop 2024-09-10T10:01:57.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.894+0000 7f6e14063740 1 -- 172.21.15.77:0/3004823713 shutdown_connections 2024-09-10T10:01:57.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.894+0000 7f6e14063740 1 --2- 172.21.15.77:0/3004823713 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x5564a4195700 0x5564a4197b20 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:57.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.894+0000 7f6e14063740 1 --2- 172.21.15.77:0/3004823713 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5564a41124f0 0x5564a411a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:57.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.894+0000 7f6e14063740 1 --2- 172.21.15.77:0/3004823713 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5564a4107c30 0x5564a418db70 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:57.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.894+0000 7f6e14063740 1 --2- 172.21.15.77:0/3004823713 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5564a4107310 0x5564a418d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:57.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.894+0000 7f6e14063740 1 -- 172.21.15.77:0/3004823713 >> 172.21.15.77:0/3004823713 conn(0x5564a41038a0 msgr2=0x5564a410e350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:57.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.894+0000 7f6defe00640 0 client.26900 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T10:01:57.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.894+0000 7f6e14063740 1 -- 172.21.15.77:0/3004823713 shutdown_connections 2024-09-10T10:01:57.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.894+0000 7f6e14063740 1 -- 172.21.15.77:0/3004823713 wait complete. 2024-09-10T10:01:57.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.894+0000 7f6e14063740 20 client.26900 trim_cache size 0 max 16384 2024-09-10T10:01:57.896 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-10T10:01:57.896 INFO:teuthology.orchestra.run.smithi077.stdout: "epoch": 35, 2024-09-10T10:01:57.896 INFO:teuthology.orchestra.run.smithi077.stdout: "leader": 24563, 2024-09-10T10:01:57.897 INFO:teuthology.orchestra.run.smithi077.stdout: "set_version": 20, 2024-09-10T10:01:57.897 INFO:teuthology.orchestra.run.smithi077.stdout: "sets": { 2024-09-10T10:01:57.897 INFO:teuthology.orchestra.run.smithi077.stdout: "f49f355b": { 2024-09-10T10:01:57.897 INFO:teuthology.orchestra.run.smithi077.stdout: "version": 20, 2024-09-10T10:01:57.897 INFO:teuthology.orchestra.run.smithi077.stdout: "age_ref": 0.0, 2024-09-10T10:01:57.897 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-10T10:01:57.897 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RELEASED", 2024-09-10T10:01:57.897 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-10T10:01:57.897 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:01:57.897 INFO:teuthology.orchestra.run.smithi077.stdout: "timeout": 90.0, 2024-09-10T10:01:57.897 INFO:teuthology.orchestra.run.smithi077.stdout: "expiration": 140.1, 2024-09-10T10:01:57.897 INFO:teuthology.orchestra.run.smithi077.stdout: "members": { 2024-09-10T10:01:57.897 INFO:teuthology.orchestra.run.smithi077.stdout: "file:/": { 2024-09-10T10:01:57.897 INFO:teuthology.orchestra.run.smithi077.stdout: "excluded": false, 2024-09-10T10:01:57.898 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-10T10:01:57.898 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RELEASED", 2024-09-10T10:01:57.898 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-10T10:01:57.898 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:01:57.898 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:01:57.898 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:01:57.898 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:01:57.898 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:01:57.898 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-10T10:01:57.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.895+0000 7f6e11600640 1 -- 172.21.15.77:0/1444200643 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f6df00a2c00 msgr2=0x7f6df00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:57.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.895+0000 7f6e11600640 1 --2- 172.21.15.77:0/1444200643 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f6df00a2c00 0x7f6df00a50c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc0bbcd0 tx=0x7f6dfc03b460 comp rx=0 tx=0).stop 2024-09-10T10:01:57.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.895+0000 7f6e11600640 1 -- 172.21.15.77:0/1444200643 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6e0c074510 msgr2=0x7f6e0c19a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:57.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.895+0000 7f6e11600640 1 --2- 172.21.15.77:0/1444200643 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6e0c074510 0x7f6e0c19a6a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc004770 tx=0x7f6dfc003c10 comp rx=0 tx=0).stop 2024-09-10T10:01:57.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.895+0000 7f6e11600640 1 -- 172.21.15.77:0/1444200643 shutdown_connections 2024-09-10T10:01:57.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.895+0000 7f6e11600640 1 --2- 172.21.15.77:0/1444200643 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f6df00a2c00 0x7f6df00a50c0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:57.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.895+0000 7f6e11600640 1 --2- 172.21.15.77:0/1444200643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6e0c10e440 0x7f6e0c19ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:57.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.895+0000 7f6e11600640 1 --2- 172.21.15.77:0/1444200643 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6e0c074e30 0x7f6e0c19abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:57.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.895+0000 7f6e11600640 1 --2- 172.21.15.77:0/1444200643 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6e0c074510 0x7f6e0c19a6a0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:57.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.895+0000 7f6e11600640 1 -- 172.21.15.77:0/1444200643 >> 172.21.15.77:0/1444200643 conn(0x7f6e0c0fd8c0 msgr2=0x7f6e0c0fe520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:57.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.895+0000 7f6e11600640 1 -- 172.21.15.77:0/1444200643 shutdown_connections 2024-09-10T10:01:57.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:57.895+0000 7f6e11600640 1 -- 172.21.15.77:0/1444200643 wait complete. 2024-09-10T10:01:58.142 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T10:01:58.142 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T10:01:58.823 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'f49f355b', total seconds elapsed: 25.7 2024-09-10T10:01:58.823 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.1 2024-09-10T10:01:58.824 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:01:58.944 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:01:59.101 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:01:58 smithi077 ceph-mon[30319]: pgmap v595: 129 pgs: 129 active+clean; 146 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 43 KiB/s wr, 10 op/s 2024-09-10T10:01:59.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:01:58 smithi134 ceph-mon[35674]: pgmap v595: 129 pgs: 129 active+clean; 146 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 43 KiB/s wr, 10 op/s 2024-09-10T10:01:59.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:01:58 smithi120 ceph-mon[34770]: pgmap v595: 129 pgs: 129 active+clean; 146 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 43 KiB/s wr, 10 op/s 2024-09-10T10:01:59.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.593+0000 7fc336600640 1 -- 172.21.15.77:0/118863379 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc330101de0 msgr2=0x7fc3301021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:59.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.593+0000 7fc336600640 1 --2- 172.21.15.77:0/118863379 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc330101de0 0x7fc3301021c0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fc320009900 tx=0x7fc32002a410 comp rx=0 tx=0).stop 2024-09-10T10:01:59.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.593+0000 7fc336600640 1 -- 172.21.15.77:0/118863379 shutdown_connections 2024-09-10T10:01:59.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.593+0000 7fc336600640 1 --2- 172.21.15.77:0/118863379 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc330106f10 0x7fc33010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:59.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.593+0000 7fc336600640 1 --2- 172.21.15.77:0/118863379 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc330102790 0x7fc3301067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:59.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.593+0000 7fc336600640 1 --2- 172.21.15.77:0/118863379 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc330101de0 0x7fc3301021c0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:59.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.593+0000 7fc336600640 1 -- 172.21.15.77:0/118863379 >> 172.21.15.77:0/118863379 conn(0x7fc3300fd8e0 msgr2=0x7fc3300ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:59.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.593+0000 7fc336600640 1 -- 172.21.15.77:0/118863379 shutdown_connections 2024-09-10T10:01:59.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.593+0000 7fc336600640 1 -- 172.21.15.77:0/118863379 wait complete. 2024-09-10T10:01:59.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.594+0000 7fc336600640 1 Processor -- start 2024-09-10T10:01:59.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.595+0000 7fc336600640 1 -- start start 2024-09-10T10:01:59.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.595+0000 7fc336600640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc330101de0 0x7fc3301984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:59.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.595+0000 7fc336600640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc330102790 0x7fc3301989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:59.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.595+0000 7fc336600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc330106f10 0x7fc33019cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:59.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.595+0000 7fc336600640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc330111f70 con 0x7fc330102790 2024-09-10T10:01:59.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.595+0000 7fc336600640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc330111df0 con 0x7fc330101de0 2024-09-10T10:01:59.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.595+0000 7fc336600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc3301120f0 con 0x7fc330106f10 2024-09-10T10:01:59.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.596+0000 7fc32f400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc330101de0 0x7fc3301984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:59.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.596+0000 7fc32f400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc330101de0 0x7fc3301984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:46766/0 (socket says 172.21.15.77:46766) 2024-09-10T10:01:59.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.596+0000 7fc32f400640 1 -- 172.21.15.77:0/2594183525 learned_addr learned my addr 172.21.15.77:0/2594183525 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:01:59.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.596+0000 7fc32fe00640 1 --2- 172.21.15.77:0/2594183525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc330106f10 0x7fc33019cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:59.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.596+0000 7fc32f400640 1 -- 172.21.15.77:0/2594183525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc330106f10 msgr2=0x7fc33019cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:59.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.596+0000 7fc32f400640 1 --2- 172.21.15.77:0/2594183525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc330106f10 0x7fc33019cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:59.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.596+0000 7fc32f400640 1 -- 172.21.15.77:0/2594183525 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc330102790 msgr2=0x7fc3301989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:59.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.596+0000 7fc32f400640 1 --2- 172.21.15.77:0/2594183525 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc330102790 0x7fc3301989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:59.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.596+0000 7fc32f400640 1 -- 172.21.15.77:0/2594183525 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc33019d460 con 0x7fc330101de0 2024-09-10T10:01:59.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.597+0000 7fc32f400640 1 --2- 172.21.15.77:0/2594183525 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc330101de0 0x7fc3301984b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fc320006010 tx=0x7fc320004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:59.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.597+0000 7fc327e00640 1 -- 172.21.15.77:0/2594183525 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc32003f030 con 0x7fc330101de0 2024-09-10T10:01:59.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.597+0000 7fc327e00640 1 -- 172.21.15.77:0/2594183525 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc320004440 con 0x7fc330101de0 2024-09-10T10:01:59.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.597+0000 7fc336600640 1 -- 172.21.15.77:0/2594183525 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc33019d6f0 con 0x7fc330101de0 2024-09-10T10:01:59.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.597+0000 7fc327e00640 1 -- 172.21.15.77:0/2594183525 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc320035070 con 0x7fc330101de0 2024-09-10T10:01:59.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.597+0000 7fc336600640 1 -- 172.21.15.77:0/2594183525 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc330074420 con 0x7fc330101de0 2024-09-10T10:01:59.602 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.602+0000 7fc339051740 1 Processor -- start 2024-09-10T10:01:59.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.602+0000 7fc327e00640 1 -- 172.21.15.77:0/2594183525 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc320042030 con 0x7fc330101de0 2024-09-10T10:01:59.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.602+0000 7fc339051740 1 -- start start 2024-09-10T10:01:59.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.602+0000 7fc339051740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ad9fe689c0 0x55ad9fe68da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:59.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.602+0000 7fc339051740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad9fe692e0 0x55ad9fe7bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:59.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.602+0000 7fc339051740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad9fe7c4b0 0x55ad9fe7e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:59.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.602+0000 7fc339051740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ad9fdcf290 con 0x55ad9fe689c0 2024-09-10T10:01:59.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.603+0000 7fc339051740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ad9fd76b40 con 0x55ad9fe692e0 2024-09-10T10:01:59.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.603+0000 7fc339051740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ad9fdcb250 con 0x55ad9fe7c4b0 2024-09-10T10:01:59.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.603+0000 7fc32ea00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad9fe7c4b0 0x55ad9fe7e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:59.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.603+0000 7fc32f400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad9fe692e0 0x55ad9fe7bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:59.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.603+0000 7fc32f400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad9fe692e0 0x55ad9fe7bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:46770/0 (socket says 172.21.15.77:46770) 2024-09-10T10:01:59.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.603+0000 7fc32f400640 1 -- 172.21.15.77:0/2179676122 learned_addr learned my addr 172.21.15.77:0/2179676122 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:01:59.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.603+0000 7fc32fe00640 1 --2- 172.21.15.77:0/2179676122 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ad9fe689c0 0x55ad9fe68da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:59.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.604+0000 7fc32f400640 1 -- 172.21.15.77:0/2179676122 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad9fe7c4b0 msgr2=0x55ad9fe7e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:59.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.604+0000 7fc32f400640 1 --2- 172.21.15.77:0/2179676122 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad9fe7c4b0 0x55ad9fe7e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:59.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.604+0000 7fc32f400640 1 -- 172.21.15.77:0/2179676122 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ad9fe689c0 msgr2=0x55ad9fe68da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:59.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.604+0000 7fc327e00640 1 --2- 172.21.15.77:0/2594183525 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fc3140a2c00 0x7fc3140a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:59.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.604+0000 7fc32f400640 1 --2- 172.21.15.77:0/2179676122 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ad9fe689c0 0x55ad9fe68da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:59.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.604+0000 7fc32f400640 1 -- 172.21.15.77:0/2179676122 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad9fb406a0 con 0x55ad9fe692e0 2024-09-10T10:01:59.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.604+0000 7fc32fe00640 1 --2- 172.21.15.77:0/2179676122 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ad9fe689c0 0x55ad9fe68da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:01:59.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.604+0000 7fc327e00640 1 -- 172.21.15.77:0/2594183525 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fc3200ef770 con 0x7fc330101de0 2024-09-10T10:01:59.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.604+0000 7fc32f400640 1 --2- 172.21.15.77:0/2179676122 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad9fe692e0 0x55ad9fe7bf70 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc3200ccd40 tx=0x7fc3200b7c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e4dcb9691ad8c93 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:59.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.605+0000 7fc32ea00640 1 --2- 172.21.15.77:0/2594183525 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fc3140a2c00 0x7fc3140a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:59.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.605+0000 7fc324c00640 1 -- 172.21.15.77:0/2179676122 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc320049030 con 0x55ad9fe692e0 2024-09-10T10:01:59.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.605+0000 7fc324c00640 1 -- 172.21.15.77:0/2179676122 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3200cbbc0 con 0x55ad9fe692e0 2024-09-10T10:01:59.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.605+0000 7fc324c00640 1 -- 172.21.15.77:0/2179676122 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3200bb6c0 con 0x55ad9fe692e0 2024-09-10T10:01:59.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.606+0000 7fc32ea00640 1 --2- 172.21.15.77:0/2594183525 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fc3140a2c00 0x7fc3140a50c0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x55ad9fe6a560 tx=0x7fc31c009620 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:59.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.606+0000 7fc339051740 1 -- 172.21.15.77:0/2179676122 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad9fe692e0 msgr2=0x55ad9fe7bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:59.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.606+0000 7fc339051740 1 --2- 172.21.15.77:0/2179676122 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad9fe692e0 0x55ad9fe7bf70 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc3200ccd40 tx=0x7fc3200b7c50 comp rx=0 tx=0).stop 2024-09-10T10:01:59.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.606+0000 7fc339051740 1 -- 172.21.15.77:0/2179676122 shutdown_connections 2024-09-10T10:01:59.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.606+0000 7fc339051740 1 --2- 172.21.15.77:0/2179676122 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad9fe7c4b0 0x55ad9fe7e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:59.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.606+0000 7fc339051740 1 --2- 172.21.15.77:0/2179676122 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad9fe692e0 0x55ad9fe7bf70 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:59.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.606+0000 7fc339051740 1 --2- 172.21.15.77:0/2179676122 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ad9fe689c0 0x55ad9fe68da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:59.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.606+0000 7fc339051740 1 -- 172.21.15.77:0/2179676122 >> 172.21.15.77:0/2179676122 conn(0x55ad9fe65050 msgr2=0x55ad9fe65430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:59.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.606+0000 7fc339051740 1 -- 172.21.15.77:0/2179676122 shutdown_connections 2024-09-10T10:01:59.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.606+0000 7fc339051740 1 -- 172.21.15.77:0/2179676122 wait complete. 2024-09-10T10:01:59.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.607+0000 7fc339051740 1 -- start start 2024-09-10T10:01:59.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.607+0000 7fc339051740 1 Processor -- start 2024-09-10T10:01:59.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.608+0000 7fc339051740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad9fe689c0 0x55ad9fe75990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:59.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.608+0000 7fc339051740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ad9fe692e0 0x55ad9fe75ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:59.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.608+0000 7fc339051740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad9fe7c4b0 0x55ad9fe7b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:59.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.608+0000 7fc339051740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ad9fe6b7c0 con 0x55ad9fe692e0 2024-09-10T10:01:59.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.608+0000 7fc339051740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ad9fd89fa0 con 0x55ad9fe689c0 2024-09-10T10:01:59.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.608+0000 7fc339051740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ad9fe6b940 con 0x55ad9fe7c4b0 2024-09-10T10:01:59.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.609+0000 7fc32ea00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ad9fe692e0 0x55ad9fe75ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:59.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.609+0000 7fc32ea00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ad9fe692e0 0x55ad9fe75ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:51534/0 (socket says 172.21.15.77:51534) 2024-09-10T10:01:59.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.609+0000 7fc32ea00640 1 -- 172.21.15.77:0/333704570 learned_addr learned my addr 172.21.15.77:0/333704570 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:01:59.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.609+0000 7fc32f400640 1 --2- 172.21.15.77:0/333704570 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad9fe689c0 0x55ad9fe75990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:59.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.608+0000 7fc339051740 10 client.0 _ll_register_callbacks cb 0x55ad9faddba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:01:59.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.609+0000 7fc32fe00640 1 --2- 172.21.15.77:0/333704570 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad9fe7c4b0 0x55ad9fe7b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:59.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.609+0000 7fc32ea00640 1 -- 172.21.15.77:0/333704570 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad9fe7c4b0 msgr2=0x55ad9fe7b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:59.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.609+0000 7fc32ea00640 1 --2- 172.21.15.77:0/333704570 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad9fe7c4b0 0x55ad9fe7b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:59.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.609+0000 7fc32ea00640 1 -- 172.21.15.77:0/333704570 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad9fe689c0 msgr2=0x55ad9fe75990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:59.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.609+0000 7fc32ea00640 1 --2- 172.21.15.77:0/333704570 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad9fe689c0 0x55ad9fe75990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:59.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.609+0000 7fc32ea00640 1 -- 172.21.15.77:0/333704570 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad9fe7bef0 con 0x55ad9fe692e0 2024-09-10T10:01:59.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.610+0000 7fc32ea00640 1 --2- 172.21.15.77:0/333704570 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ad9fe692e0 0x55ad9fe75ed0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fc31c005f60 tx=0x7fc31c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5073d56c87b38ffc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:59.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.610+0000 7fc319600640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-10T10:01:59.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.610+0000 7fc319600640 1 -- 172.21.15.77:0/333704570 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc31c009e60 con 0x55ad9fe692e0 2024-09-10T10:01:59.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.610+0000 7fc319600640 1 -- 172.21.15.77:0/333704570 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fc31c01c420 con 0x55ad9fe692e0 2024-09-10T10:01:59.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.610+0000 7fc339051740 1 -- 172.21.15.77:0/333704570 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ad9fddc800 con 0x55ad9fe692e0 2024-09-10T10:01:59.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.611+0000 7fc319600640 1 -- 172.21.15.77:0/333704570 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fc31c026630 con 0x55ad9fe692e0 2024-09-10T10:01:59.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.611+0000 7fc339051740 10 client.16782 fetch_fsmap learned FSMap version 35 2024-09-10T10:01:59.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.611+0000 7fc339051740 1 -- 172.21.15.77:0/333704570 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55ad9fe935d0 con 0x55ad9fe692e0 2024-09-10T10:01:59.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.611+0000 7fc319600640 1 -- 172.21.15.77:0/333704570 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55ad9fddc800 con 0x55ad9fe692e0 2024-09-10T10:01:59.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.611+0000 7fc319600640 1 -- 172.21.15.77:0/333704570 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7fc31c01cbb0 con 0x55ad9fe692e0 2024-09-10T10:01:59.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.611+0000 7fc339051740 10 client.16782 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T10:01:59.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.611+0000 7fc339051740 10 client.16782 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T10:01:59.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.611+0000 7fc339051740 20 client.16782 populate_metadata read hostname 'smithi077' 2024-09-10T10:01:59.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.611+0000 7fc339051740 1 --2- 172.21.15.77:0/333704570 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55ad9fe94b30 0x55ad9fe96f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:59.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.611+0000 7fc339051740 4 client.16782 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:01:59.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.611+0000 7fc339051740 1 -- 172.21.15.77:0/333704570 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ad9fdd6d10 con 0x55ad9fe94b30 2024-09-10T10:01:59.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.612+0000 7fc32f400640 1 --2- 172.21.15.77:0/333704570 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55ad9fe94b30 0x55ad9fe96f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:59.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.612+0000 7fc32f400640 1 --2- 172.21.15.77:0/333704570 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55ad9fe94b30 0x55ad9fe96f50 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a712073c850835f server_cookie=a1e326d008885b5f in_seq=0 out_seq=0 2024-09-10T10:01:59.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.612+0000 7fc319600640 10 client.16782 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T10:01:59.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.613+0000 7fc319600640 1 -- 172.21.15.77:0/333704570 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ad9fdd6d10 con 0x55ad9fe94b30 2024-09-10T10:01:59.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.613+0000 7fc319600640 10 client.16782 handle_command_reply: tid=0 2024-09-10T10:01:59.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.613+0000 7fc339051740 1 client.16782 shutdown 2024-09-10T10:01:59.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.613+0000 7fc339051740 1 -- 172.21.15.77:0/333704570 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ad9fe692e0 msgr2=0x55ad9fe75ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:59.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.613+0000 7fc339051740 1 --2- 172.21.15.77:0/333704570 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ad9fe692e0 0x55ad9fe75ed0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fc31c005f60 tx=0x7fc31c0046c0 comp rx=0 tx=0).stop 2024-09-10T10:01:59.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.614+0000 7fc339051740 1 -- 172.21.15.77:0/333704570 shutdown_connections 2024-09-10T10:01:59.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.614+0000 7fc339051740 1 --2- 172.21.15.77:0/333704570 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55ad9fe94b30 0x55ad9fe96f50 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:59.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.614+0000 7fc339051740 1 --2- 172.21.15.77:0/333704570 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad9fe7c4b0 0x55ad9fe7b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:59.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.614+0000 7fc339051740 1 --2- 172.21.15.77:0/333704570 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ad9fe692e0 0x55ad9fe75ed0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:59.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.614+0000 7fc339051740 1 --2- 172.21.15.77:0/333704570 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad9fe689c0 0x55ad9fe75990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:59.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.614+0000 7fc339051740 1 -- 172.21.15.77:0/333704570 >> 172.21.15.77:0/333704570 conn(0x55ad9fe65050 msgr2=0x55ad9fe6adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:59.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.614+0000 7fc319600640 0 client.16782 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T10:01:59.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.614+0000 7fc339051740 1 -- 172.21.15.77:0/333704570 shutdown_connections 2024-09-10T10:01:59.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.614+0000 7fc339051740 1 -- 172.21.15.77:0/333704570 wait complete. 2024-09-10T10:01:59.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.614+0000 7fc339051740 20 client.16782 trim_cache size 0 max 16384 2024-09-10T10:01:59.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.640+0000 7fc339051740 1 Processor -- start 2024-09-10T10:01:59.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.640+0000 7fc339051740 1 -- start start 2024-09-10T10:01:59.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.640+0000 7fc339051740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad9fe689c0 0x55ad9fe7b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:59.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.640+0000 7fc339051740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad9fe692e0 0x55ad9fe6aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:59.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.640+0000 7fc339051740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ad9fe7c4b0 0x55ad9fe9c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:59.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.640+0000 7fc339051740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc3301acc90 con 0x55ad9fe7c4b0 2024-09-10T10:01:59.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.640+0000 7fc339051740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc3301ae7d0 con 0x55ad9fe692e0 2024-09-10T10:01:59.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.640+0000 7fc339051740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc3301ad360 con 0x55ad9fe689c0 2024-09-10T10:01:59.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.640+0000 7fc32fe00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ad9fe7c4b0 0x55ad9fe9c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:59.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.640+0000 7fc32f400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad9fe689c0 0x55ad9fe7b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:59.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.640+0000 7fc32fe00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ad9fe7c4b0 0x55ad9fe9c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:51550/0 (socket says 172.21.15.77:51550) 2024-09-10T10:01:59.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.641+0000 7fc32fe00640 1 -- 172.21.15.77:0/3434198062 learned_addr learned my addr 172.21.15.77:0/3434198062 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:01:59.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.641+0000 7fc32f400640 1 -- 172.21.15.77:0/3434198062 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad9fe692e0 msgr2=0x55ad9fe6aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:59.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.641+0000 7fc32f400640 1 --2- 172.21.15.77:0/3434198062 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad9fe692e0 0x55ad9fe6aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:59.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.641+0000 7fc32f400640 1 -- 172.21.15.77:0/3434198062 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ad9fe7c4b0 msgr2=0x55ad9fe9c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:59.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.641+0000 7fc32f400640 1 --2- 172.21.15.77:0/3434198062 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ad9fe7c4b0 0x55ad9fe9c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:59.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.641+0000 7fc32f400640 1 -- 172.21.15.77:0/3434198062 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad9fe64e80 con 0x55ad9fe689c0 2024-09-10T10:01:59.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.641+0000 7fc32f400640 1 --2- 172.21.15.77:0/3434198062 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad9fe689c0 0x55ad9fe7b680 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc320000c00 tx=0x7fc3200f1040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c64aee0da9860e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:59.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.641+0000 7fc31be00640 1 -- 172.21.15.77:0/3434198062 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc320049040 con 0x55ad9fe689c0 2024-09-10T10:01:59.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.642+0000 7fc31be00640 1 -- 172.21.15.77:0/3434198062 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3200bc280 con 0x55ad9fe689c0 2024-09-10T10:01:59.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.642+0000 7fc31be00640 1 -- 172.21.15.77:0/3434198062 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3200bb4e0 con 0x55ad9fe689c0 2024-09-10T10:01:59.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.642+0000 7fc339051740 1 -- 172.21.15.77:0/3434198062 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad9fe689c0 msgr2=0x55ad9fe7b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:59.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.642+0000 7fc339051740 1 --2- 172.21.15.77:0/3434198062 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad9fe689c0 0x55ad9fe7b680 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc320000c00 tx=0x7fc3200f1040 comp rx=0 tx=0).stop 2024-09-10T10:01:59.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.642+0000 7fc339051740 1 -- 172.21.15.77:0/3434198062 shutdown_connections 2024-09-10T10:01:59.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.642+0000 7fc339051740 1 --2- 172.21.15.77:0/3434198062 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ad9fe7c4b0 0x55ad9fe9c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:59.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.642+0000 7fc339051740 1 --2- 172.21.15.77:0/3434198062 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad9fe692e0 0x55ad9fe6aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:59.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.643+0000 7fc339051740 1 --2- 172.21.15.77:0/3434198062 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad9fe689c0 0x55ad9fe7b680 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:59.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.643+0000 7fc339051740 1 -- 172.21.15.77:0/3434198062 >> 172.21.15.77:0/3434198062 conn(0x55ad9fe65050 msgr2=0x55ad9fe6a590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:59.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.643+0000 7fc339051740 1 -- 172.21.15.77:0/3434198062 shutdown_connections 2024-09-10T10:01:59.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.643+0000 7fc339051740 1 -- 172.21.15.77:0/3434198062 wait complete. 2024-09-10T10:01:59.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.643+0000 7fc339051740 1 -- start start 2024-09-10T10:01:59.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.643+0000 7fc339051740 1 Processor -- start 2024-09-10T10:01:59.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.644+0000 7fc339051740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad9fe689c0 0x55ad9fea8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:59.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.644+0000 7fc339051740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad9fe692e0 0x55ad9fea8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:59.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.644+0000 7fc339051740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ad9fe7c4b0 0x55ad9fea34f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:59.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.644+0000 7fc339051740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc3301ae100 con 0x55ad9fe7c4b0 2024-09-10T10:01:59.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.644+0000 7fc339051740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc3301ada30 con 0x55ad9fe689c0 2024-09-10T10:01:59.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.644+0000 7fc339051740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc3301aac70 con 0x55ad9fe692e0 2024-09-10T10:01:59.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.644+0000 7fc339051740 10 client.0 _ll_register_callbacks cb 0x7fc31c009a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:01:59.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.644+0000 7fc32ea00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad9fe692e0 0x55ad9fea8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:59.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.644+0000 7fc32ea00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad9fe692e0 0x55ad9fea8a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:52132/0 (socket says 172.21.15.77:52132) 2024-09-10T10:01:59.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.644+0000 7fc32f400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad9fe689c0 0x55ad9fea8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:59.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.644+0000 7fc32ea00640 1 -- 172.21.15.77:0/3881383746 learned_addr learned my addr 172.21.15.77:0/3881383746 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:01:59.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.644+0000 7fc32fe00640 1 --2- 172.21.15.77:0/3881383746 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ad9fe7c4b0 0x55ad9fea34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:59.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.645+0000 7fc32fe00640 1 -- 172.21.15.77:0/3881383746 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad9fe692e0 msgr2=0x55ad9fea8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:59.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.645+0000 7fc32fe00640 1 --2- 172.21.15.77:0/3881383746 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad9fe692e0 0x55ad9fea8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:59.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.645+0000 7fc32fe00640 1 -- 172.21.15.77:0/3881383746 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad9fe689c0 msgr2=0x55ad9fea8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:59.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.645+0000 7fc32fe00640 1 --2- 172.21.15.77:0/3881383746 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad9fe689c0 0x55ad9fea8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:59.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.645+0000 7fc32fe00640 1 -- 172.21.15.77:0/3881383746 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad9fea9170 con 0x55ad9fe7c4b0 2024-09-10T10:01:59.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.645+0000 7fc32fe00640 1 --2- 172.21.15.77:0/3881383746 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ad9fe7c4b0 0x55ad9fea34f0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fc328012c60 tx=0x7fc328011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79ed06ea090f82f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:59.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.645+0000 7fc319600640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-10T10:01:59.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.646+0000 7fc319600640 1 -- 172.21.15.77:0/3881383746 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc328012e10 con 0x55ad9fe7c4b0 2024-09-10T10:01:59.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.646+0000 7fc319600640 1 -- 172.21.15.77:0/3881383746 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fc328004b20 con 0x55ad9fe7c4b0 2024-09-10T10:01:59.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.646+0000 7fc339051740 1 -- 172.21.15.77:0/3881383746 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3200bc280 con 0x55ad9fe7c4b0 2024-09-10T10:01:59.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.646+0000 7fc319600640 1 -- 172.21.15.77:0/3881383746 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fc32802b050 con 0x55ad9fe7c4b0 2024-09-10T10:01:59.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.646+0000 7fc339051740 10 client.16788 fetch_fsmap learned FSMap version 35 2024-09-10T10:01:59.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.646+0000 7fc339051740 1 -- 172.21.15.77:0/3881383746 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55ad9feffcf0 con 0x55ad9fe7c4b0 2024-09-10T10:01:59.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.647+0000 7fc319600640 1 -- 172.21.15.77:0/3881383746 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3200bc280 con 0x55ad9fe7c4b0 2024-09-10T10:01:59.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.647+0000 7fc319600640 1 -- 172.21.15.77:0/3881383746 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7fc32801e6c0 con 0x55ad9fe7c4b0 2024-09-10T10:01:59.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.647+0000 7fc339051740 10 client.16788 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T10:01:59.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.647+0000 7fc339051740 10 client.16788 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T10:01:59.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.647+0000 7fc339051740 20 client.16788 populate_metadata read hostname 'smithi077' 2024-09-10T10:01:59.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.647+0000 7fc339051740 1 --2- 172.21.15.77:0/3881383746 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55ad9ff004f0 0x55ad9ff02910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:59.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.647+0000 7fc339051740 4 client.16788 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T10:01:59.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.647+0000 7fc339051740 1 -- 172.21.15.77:0/3881383746 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ad9fd52fb0 con 0x55ad9ff004f0 2024-09-10T10:01:59.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.648+0000 7fc32f400640 1 --2- 172.21.15.77:0/3881383746 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55ad9ff004f0 0x55ad9ff02910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:59.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.648+0000 7fc319600640 10 client.16788 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T10:01:59.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.649+0000 7fc32f400640 1 --2- 172.21.15.77:0/3881383746 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55ad9ff004f0 0x55ad9ff02910 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86997b65f2853318 server_cookie=d381042aa8705d39 in_seq=0 out_seq=0 2024-09-10T10:01:59.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.650+0000 7fc319600640 1 -- 172.21.15.77:0/3881383746 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ad9fd52fb0 con 0x55ad9ff004f0 2024-09-10T10:01:59.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.650+0000 7fc319600640 10 client.16788 handle_command_reply: tid=0 2024-09-10T10:01:59.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.650+0000 7fc339051740 1 client.16788 shutdown 2024-09-10T10:01:59.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.650+0000 7fc339051740 1 -- 172.21.15.77:0/3881383746 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ad9fe7c4b0 msgr2=0x55ad9fea34f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:59.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.650+0000 7fc339051740 1 --2- 172.21.15.77:0/3881383746 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ad9fe7c4b0 0x55ad9fea34f0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fc328012c60 tx=0x7fc328011810 comp rx=0 tx=0).stop 2024-09-10T10:01:59.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.650+0000 7fc339051740 1 -- 172.21.15.77:0/3881383746 shutdown_connections 2024-09-10T10:01:59.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.650+0000 7fc339051740 1 --2- 172.21.15.77:0/3881383746 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55ad9ff004f0 0x55ad9ff02910 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:59.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.650+0000 7fc319600640 0 client.16788 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T10:01:59.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.650+0000 7fc339051740 1 --2- 172.21.15.77:0/3881383746 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ad9fe7c4b0 0x55ad9fea34f0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:59.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.650+0000 7fc339051740 1 --2- 172.21.15.77:0/3881383746 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad9fe692e0 0x55ad9fea8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:59.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.650+0000 7fc339051740 1 --2- 172.21.15.77:0/3881383746 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad9fe689c0 0x55ad9fea8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:59.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.650+0000 7fc339051740 1 -- 172.21.15.77:0/3881383746 >> 172.21.15.77:0/3881383746 conn(0x55ad9fe65050 msgr2=0x55ad9fe6adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:59.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.650+0000 7fc339051740 1 -- 172.21.15.77:0/3881383746 shutdown_connections 2024-09-10T10:01:59.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.651+0000 7fc339051740 1 -- 172.21.15.77:0/3881383746 wait complete. 2024-09-10T10:01:59.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.651+0000 7fc339051740 20 client.16788 trim_cache size 0 max 16384 2024-09-10T10:01:59.653 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-10T10:01:59.653 INFO:teuthology.orchestra.run.smithi077.stdout: "return_code": 0, 2024-09-10T10:01:59.653 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_tag": "d5e9b76b-b7eb-4716-995f-80783f23eb2c", 2024-09-10T10:01:59.653 INFO:teuthology.orchestra.run.smithi077.stdout: "mode": "asynchronous" 2024-09-10T10:01:59.654 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-10T10:01:59.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.653+0000 7fc336600640 1 -- 172.21.15.77:0/2594183525 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fc3140a2c00 msgr2=0x7fc3140a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:59.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.653+0000 7fc336600640 1 --2- 172.21.15.77:0/2594183525 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fc3140a2c00 0x7fc3140a50c0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x55ad9fe6a560 tx=0x7fc31c009620 comp rx=0 tx=0).stop 2024-09-10T10:01:59.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.653+0000 7fc336600640 1 -- 172.21.15.77:0/2594183525 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc330101de0 msgr2=0x7fc3301984b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:59.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.653+0000 7fc336600640 1 --2- 172.21.15.77:0/2594183525 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc330101de0 0x7fc3301984b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fc320006010 tx=0x7fc320004040 comp rx=0 tx=0).stop 2024-09-10T10:01:59.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.653+0000 7fc336600640 1 -- 172.21.15.77:0/2594183525 shutdown_connections 2024-09-10T10:01:59.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.653+0000 7fc336600640 1 --2- 172.21.15.77:0/2594183525 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fc3140a2c00 0x7fc3140a50c0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:59.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.653+0000 7fc336600640 1 --2- 172.21.15.77:0/2594183525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc330106f10 0x7fc33019cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:59.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.653+0000 7fc336600640 1 --2- 172.21.15.77:0/2594183525 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc330102790 0x7fc3301989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:59.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.653+0000 7fc336600640 1 --2- 172.21.15.77:0/2594183525 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc330101de0 0x7fc3301984b0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:59.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.653+0000 7fc336600640 1 -- 172.21.15.77:0/2594183525 >> 172.21.15.77:0/2594183525 conn(0x7fc3300fd8e0 msgr2=0x7fc3300fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:59.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.653+0000 7fc336600640 1 -- 172.21.15.77:0/2594183525 shutdown_connections 2024-09-10T10:01:59.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:01:59.654+0000 7fc336600640 1 -- 172.21.15.77:0/2594183525 wait complete. 2024-09-10T10:01:59.657 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:01:59.938 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:01:59 smithi077 ceph-mon[30319]: scrub queued for path: / 2024-09-10T10:01:59.938 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:01:59 smithi077 ceph-mon[30319]: scrub summary: idle+waiting paths [/] 2024-09-10T10:01:59.938 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:01:59 smithi077 ceph-mon[30319]: scrub summary: active paths [/] 2024-09-10T10:01:59.938 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:01:59 smithi077 ceph-mon[30319]: scrub summary: active paths [/] 2024-09-10T10:01:59.938 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:01:59 smithi077 ceph-mon[30319]: scrub summary: idle+waiting paths [/] 2024-09-10T10:02:00.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:01:59 smithi134 ceph-mon[35674]: scrub queued for path: / 2024-09-10T10:02:00.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:01:59 smithi134 ceph-mon[35674]: scrub summary: idle+waiting paths [/] 2024-09-10T10:02:00.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:01:59 smithi134 ceph-mon[35674]: scrub summary: active paths [/] 2024-09-10T10:02:00.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:01:59 smithi134 ceph-mon[35674]: scrub summary: active paths [/] 2024-09-10T10:02:00.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:01:59 smithi134 ceph-mon[35674]: scrub summary: idle+waiting paths [/] 2024-09-10T10:02:00.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:01:59 smithi120 ceph-mon[34770]: scrub queued for path: / 2024-09-10T10:02:00.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:01:59 smithi120 ceph-mon[34770]: scrub summary: idle+waiting paths [/] 2024-09-10T10:02:00.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:01:59 smithi120 ceph-mon[34770]: scrub summary: active paths [/] 2024-09-10T10:02:00.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:01:59 smithi120 ceph-mon[34770]: scrub summary: active paths [/] 2024-09-10T10:02:00.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:01:59 smithi120 ceph-mon[34770]: scrub summary: idle+waiting paths [/] 2024-09-10T10:02:00.994 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T10:02:01.105 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T10:02:01.105 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":35,"btime":"2024-09-10T10:01:05:922537+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3159289675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3159289675},{"type":"v1","addr":"172.21.15.77:6835","nonce":3159289675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1951962341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1951962341},{"type":"v1","addr":"172.21.15.77:6837","nonce":1951962341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":35,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:52:23.368252+0000","modified":"2024-09-10T10:01:05.706646+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24563,"mds_1":24524,"mds_2":24520,"mds_3":24502,"mds_4":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14487":{"gid":14487,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/4291121888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":4291121888},{"type":"v1","addr":"172.21.15.77:6839","nonce":4291121888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14499":{"gid":14499,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6841/445284194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":445284194},{"type":"v1","addr":"172.21.15.77:6841","nonce":445284194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1005054423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1005054423},{"type":"v1","addr":"172.21.15.134:6833","nonce":1005054423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6833/3716307744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":3716307744},{"type":"v1","addr":"172.21.15.120:6833","nonce":3716307744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/1319892544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1319892544},{"type":"v1","addr":"172.21.15.120:6835","nonce":1319892544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.134:6835/3293933699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3293933699},{"type":"v1","addr":"172.21.15.134:6835","nonce":3293933699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.134:6837/2461497091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2461497091},{"type":"v1","addr":"172.21.15.134:6837","nonce":2461497091}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":20,"addr":"172.21.15.134:6839/1965117680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1965117680},{"type":"v1","addr":"172.21.15.134:6839","nonce":1965117680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":20,"addr":"172.21.15.120:6837/3033280709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3033280709},{"type":"v1","addr":"172.21.15.120:6837","nonce":3033280709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.120:6839/1777710157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1777710157},{"type":"v1","addr":"172.21.15.120:6839","nonce":1777710157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24563,"qdb_cluster":[24496,24502,24520,24524,24563]},"id":1}]} 2024-09-10T10:02:01.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:00.613+0000 7f5448a00640 1 -- 172.21.15.77:0/2267291085 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f54441019d0 msgr2=0x7f5444101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:01.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:00.613+0000 7f5448a00640 1 --2- 172.21.15.77:0/2267291085 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f54441019d0 0x7f5444101e30 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f5430009930 tx=0x7f543002a420 comp rx=0 tx=0).stop 2024-09-10T10:02:01.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:00.613+0000 7f5448a00640 1 -- 172.21.15.77:0/2267291085 shutdown_connections 2024-09-10T10:02:01.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:00.613+0000 7f5448a00640 1 --2- 172.21.15.77:0/2267291085 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5444102370 0x7f544410e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:01.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:00.613+0000 7f5448a00640 1 --2- 172.21.15.77:0/2267291085 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f54441019d0 0x7f5444101e30 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:01.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:00.613+0000 7f5448a00640 1 --2- 172.21.15.77:0/2267291085 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54441079d0 0x7f5444107db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:01.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:00.613+0000 7f5448a00640 1 -- 172.21.15.77:0/2267291085 >> 172.21.15.77:0/2267291085 conn(0x7f54440fd710 msgr2=0x7f54440ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:01.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:00.613+0000 7f5448a00640 1 -- 172.21.15.77:0/2267291085 shutdown_connections 2024-09-10T10:02:01.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:00.613+0000 7f5448a00640 1 -- 172.21.15.77:0/2267291085 wait complete. 2024-09-10T10:02:01.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:00.614+0000 7f5448a00640 1 Processor -- start 2024-09-10T10:02:01.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:00.614+0000 7f5448a00640 1 -- start start 2024-09-10T10:02:01.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:00.615+0000 7f5448a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54441019d0 0x7f5444108900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:01.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:00.615+0000 7f5448a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5444102370 0x7f5444108e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:01.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:00.615+0000 7f5448a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f54441079d0 0x7f5444109380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:01.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:00.615+0000 7f5448a00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5444111e20 con 0x7f5444102370 2024-09-10T10:02:01.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:00.615+0000 7f5448a00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5444111ca0 con 0x7f54441079d0 2024-09-10T10:02:01.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:00.615+0000 7f5448a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5444111fa0 con 0x7f54441019d0 2024-09-10T10:02:01.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:00.615+0000 7f5441e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5444102370 0x7f5444108e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:01.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:00.615+0000 7f5442800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54441019d0 0x7f5444108900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:01.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:00.615+0000 7f5441e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5444102370 0x7f5444108e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:51572/0 (socket says 172.21.15.77:51572) 2024-09-10T10:02:01.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:00.615+0000 7f5441e00640 1 -- 172.21.15.77:0/3273915059 learned_addr learned my addr 172.21.15.77:0/3273915059 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:02:01.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:00.616+0000 7f5442800640 1 -- 172.21.15.77:0/3273915059 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f54441079d0 msgr2=0x7f5444109380 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T10:02:01.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:00.616+0000 7f5442800640 1 --2- 172.21.15.77:0/3273915059 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f54441079d0 0x7f5444109380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:01.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:00.616+0000 7f5442800640 1 -- 172.21.15.77:0/3273915059 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5444102370 msgr2=0x7f5444108e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:01.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:00.616+0000 7f5442800640 1 --2- 172.21.15.77:0/3273915059 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5444102370 0x7f5444108e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:01.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:00.616+0000 7f5442800640 1 -- 172.21.15.77:0/3273915059 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54441aaba0 con 0x7f54441019d0 2024-09-10T10:02:01.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:00.616+0000 7f5441e00640 1 --2- 172.21.15.77:0/3273915059 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5444102370 0x7f5444108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:02:01.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:00.616+0000 7f5442800640 1 --2- 172.21.15.77:0/3273915059 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54441019d0 0x7f5444108900 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f543400ea20 tx=0x7f543400eee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:01.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:00.616+0000 7f543b400640 1 -- 172.21.15.77:0/3273915059 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f543400ccc0 con 0x7f54441019d0 2024-09-10T10:02:01.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:00.616+0000 7f543b400640 1 -- 172.21.15.77:0/3273915059 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f543400ce60 con 0x7f54441019d0 2024-09-10T10:02:01.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:00.616+0000 7f543b400640 1 -- 172.21.15.77:0/3273915059 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54340188b0 con 0x7f54441019d0 2024-09-10T10:02:01.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:00.616+0000 7f5448a00640 1 -- 172.21.15.77:0/3273915059 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54441aae90 con 0x7f54441019d0 2024-09-10T10:02:01.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:00.616+0000 7f5448a00640 1 -- 172.21.15.77:0/3273915059 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54441ab320 con 0x7f54441019d0 2024-09-10T10:02:01.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:00.617+0000 7f5448a00640 1 -- 172.21.15.77:0/3273915059 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5444103110 con 0x7f54441019d0 2024-09-10T10:02:01.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:00.619+0000 7f543b400640 1 -- 172.21.15.77:0/3273915059 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5434018a50 con 0x7f54441019d0 2024-09-10T10:02:01.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:00.622+0000 7f543b400640 1 --2- 172.21.15.77:0/3273915059 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f54280a2c00 0x7f54280a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:01.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:00.622+0000 7f543b400640 1 -- 172.21.15.77:0/3273915059 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f54340cfa20 con 0x7f54441019d0 2024-09-10T10:02:01.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:00.622+0000 7f5441e00640 1 --2- 172.21.15.77:0/3273915059 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f54280a2c00 0x7f54280a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:01.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:00.623+0000 7f5441e00640 1 --2- 172.21.15.77:0/3273915059 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f54280a2c00 0x7f54280a50c0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f544410a0c0 tx=0x7f5430031510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:01.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:00.623+0000 7f543b400640 1 -- 172.21.15.77:0/3273915059 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f543409c530 con 0x7f54441019d0 2024-09-10T10:02:01.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:00.930+0000 7f5448a00640 1 -- 172.21.15.77:0/3273915059 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5444101e30 con 0x7f54441019d0 2024-09-10T10:02:01.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:00.931+0000 7f543b400640 1 -- 172.21.15.77:0/3273915059 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11989 (secure 0 0 0) 0x7f5444101e30 con 0x7f54441019d0 2024-09-10T10:02:01.109 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 35 2024-09-10T10:02:01.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:00.935+0000 7f5448a00640 1 -- 172.21.15.77:0/3273915059 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f54280a2c00 msgr2=0x7f54280a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:01.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:00.935+0000 7f5448a00640 1 --2- 172.21.15.77:0/3273915059 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f54280a2c00 0x7f54280a50c0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f544410a0c0 tx=0x7f5430031510 comp rx=0 tx=0).stop 2024-09-10T10:02:01.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:00.935+0000 7f5448a00640 1 -- 172.21.15.77:0/3273915059 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54441019d0 msgr2=0x7f5444108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:01.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:00.935+0000 7f5448a00640 1 --2- 172.21.15.77:0/3273915059 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54441019d0 0x7f5444108900 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f543400ea20 tx=0x7f543400eee0 comp rx=0 tx=0).stop 2024-09-10T10:02:01.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:00.935+0000 7f5448a00640 1 -- 172.21.15.77:0/3273915059 shutdown_connections 2024-09-10T10:02:01.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:00.935+0000 7f5448a00640 1 --2- 172.21.15.77:0/3273915059 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f54280a2c00 0x7f54280a50c0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:01.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:00.935+0000 7f5448a00640 1 --2- 172.21.15.77:0/3273915059 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f54441079d0 0x7f5444109380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:01.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:00.935+0000 7f5448a00640 1 --2- 172.21.15.77:0/3273915059 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5444102370 0x7f5444108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:01.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:00.935+0000 7f5448a00640 1 --2- 172.21.15.77:0/3273915059 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54441019d0 0x7f5444108900 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:01.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:00.935+0000 7f5448a00640 1 -- 172.21.15.77:0/3273915059 >> 172.21.15.77:0/3273915059 conn(0x7f54440fd710 msgr2=0x7f54440fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:01.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:00.935+0000 7f5448a00640 1 -- 172.21.15.77:0/3273915059 shutdown_connections 2024-09-10T10:02:01.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:00.935+0000 7f5448a00640 1 -- 172.21.15.77:0/3273915059 wait complete. 2024-09-10T10:02:01.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:02:00 smithi077 ceph-mon[30319]: scrub summary: active paths [/] 2024-09-10T10:02:01.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:02:00 smithi077 ceph-mon[30319]: scrub summary: active paths [/] 2024-09-10T10:02:01.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:02:00 smithi077 ceph-mon[30319]: scrub summary: active paths [/] 2024-09-10T10:02:01.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:02:00 smithi077 ceph-mon[30319]: pgmap v596: 129 pgs: 129 active+clean; 142 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 104 KiB/s wr, 37 op/s 2024-09-10T10:02:01.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:02:00 smithi077 ceph-mon[30319]: scrub summary: idle+waiting paths [/] 2024-09-10T10:02:01.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:02:00 smithi077 ceph-mon[30319]: scrub summary: idle+waiting paths [/] 2024-09-10T10:02:01.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:02:00 smithi077 ceph-mon[30319]: scrub summary: idle+waiting paths [/] 2024-09-10T10:02:01.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:02:00 smithi077 ceph-mon[30319]: scrub summary: idle+waiting paths [/] 2024-09-10T10:02:01.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:02:00 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/3273915059' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:02:01.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:02:00 smithi134 ceph-mon[35674]: scrub summary: active paths [/] 2024-09-10T10:02:01.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:02:00 smithi134 ceph-mon[35674]: scrub summary: active paths [/] 2024-09-10T10:02:01.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:02:00 smithi134 ceph-mon[35674]: scrub summary: active paths [/] 2024-09-10T10:02:01.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:02:00 smithi134 ceph-mon[35674]: pgmap v596: 129 pgs: 129 active+clean; 142 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 104 KiB/s wr, 37 op/s 2024-09-10T10:02:01.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:02:00 smithi134 ceph-mon[35674]: scrub summary: idle+waiting paths [/] 2024-09-10T10:02:01.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:02:00 smithi134 ceph-mon[35674]: scrub summary: idle+waiting paths [/] 2024-09-10T10:02:01.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:02:00 smithi134 ceph-mon[35674]: scrub summary: idle+waiting paths [/] 2024-09-10T10:02:01.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:02:00 smithi134 ceph-mon[35674]: scrub summary: idle+waiting paths [/] 2024-09-10T10:02:01.382 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:02:00 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/3273915059' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:02:01.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:02:00 smithi120 ceph-mon[34770]: scrub summary: active paths [/] 2024-09-10T10:02:01.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:02:00 smithi120 ceph-mon[34770]: scrub summary: active paths [/] 2024-09-10T10:02:01.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:02:00 smithi120 ceph-mon[34770]: scrub summary: active paths [/] 2024-09-10T10:02:01.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:02:00 smithi120 ceph-mon[34770]: pgmap v596: 129 pgs: 129 active+clean; 142 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 104 KiB/s wr, 37 op/s 2024-09-10T10:02:01.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:02:00 smithi120 ceph-mon[34770]: scrub summary: idle+waiting paths [/] 2024-09-10T10:02:01.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:02:00 smithi120 ceph-mon[34770]: scrub summary: idle+waiting paths [/] 2024-09-10T10:02:01.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:02:00 smithi120 ceph-mon[34770]: scrub summary: idle+waiting paths [/] 2024-09-10T10:02:01.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:02:00 smithi120 ceph-mon[34770]: scrub summary: idle+waiting paths [/] 2024-09-10T10:02:01.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:02:00 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/3273915059' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:02:02.040 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24563 quiesce db / --timeout 90 --expiration 140.1 --await-for 45.0' 2024-09-10T10:02:02.040 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24563 quiesce db / --timeout 90 --expiration 140.1 --await-for 45.0 2024-09-10T10:02:02.154 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:02:02.478 INFO:tasks.workunit.client.0.smithi077.stdout:10000 mapread 0x33cc4e3 thru 0x33cec16 (0x2734 bytes) 2024-09-10T10:02:03.214 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:02:02 smithi077 ceph-mon[30319]: pgmap v597: 129 pgs: 129 active+clean; 139 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 162 KiB/s wr, 54 op/s 2024-09-10T10:02:03.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:02:02 smithi134 ceph-mon[35674]: pgmap v597: 129 pgs: 129 active+clean; 139 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 162 KiB/s wr, 54 op/s 2024-09-10T10:02:03.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:02:02 smithi120 ceph-mon[34770]: pgmap v597: 129 pgs: 129 active+clean; 139 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 162 KiB/s wr, 54 op/s 2024-09-10T10:02:03.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.507+0000 7f59ac600640 1 -- 172.21.15.77:0/1593889390 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f59a4104010 msgr2=0x7f59a41043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:03.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.507+0000 7f59ac600640 1 --2- 172.21.15.77:0/1593889390 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f59a4104010 0x7f59a41043f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f5990009900 tx=0x7f599002a3f0 comp rx=0 tx=0).stop 2024-09-10T10:02:03.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.507+0000 7f59ac600640 1 -- 172.21.15.77:0/1593889390 shutdown_connections 2024-09-10T10:02:03.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.507+0000 7f59ac600640 1 --2- 172.21.15.77:0/1593889390 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59a4109140 0x7f59a4110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:03.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.507+0000 7f59ac600640 1 --2- 172.21.15.77:0/1593889390 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59a41049c0 0x7f59a4108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:03.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.507+0000 7f59ac600640 1 --2- 172.21.15.77:0/1593889390 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f59a4104010 0x7f59a41043f0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:03.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.507+0000 7f59ac600640 1 -- 172.21.15.77:0/1593889390 >> 172.21.15.77:0/1593889390 conn(0x7f59a40ffa50 msgr2=0x7f59a4101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:03.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.507+0000 7f59ac600640 1 -- 172.21.15.77:0/1593889390 shutdown_connections 2024-09-10T10:02:03.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.507+0000 7f59ac600640 1 -- 172.21.15.77:0/1593889390 wait complete. 2024-09-10T10:02:03.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.508+0000 7f59ac600640 1 Processor -- start 2024-09-10T10:02:03.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.509+0000 7f59ac600640 1 -- start start 2024-09-10T10:02:03.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.509+0000 7f59ac600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59a4104010 0x7f59a419a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:03.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.509+0000 7f59ac600640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59a41049c0 0x7f59a419ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:03.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.509+0000 7f59ac600640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f59a4109140 0x7f59a419ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:03.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.509+0000 7f59ac600640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f59a4114190 con 0x7f59a41049c0 2024-09-10T10:02:03.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.509+0000 7f59ac600640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f59a4114010 con 0x7f59a4109140 2024-09-10T10:02:03.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.509+0000 7f59ac600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f59a4114310 con 0x7f59a4104010 2024-09-10T10:02:03.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.510+0000 7f59a9c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59a4104010 0x7f59a419a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:03.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.510+0000 7f59a9c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59a4104010 0x7f59a419a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:59902/0 (socket says 172.21.15.77:59902) 2024-09-10T10:02:03.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.510+0000 7f59a9c00640 1 -- 172.21.15.77:0/1553049878 learned_addr learned my addr 172.21.15.77:0/1553049878 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:02:03.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.510+0000 7f59aa600640 1 --2- 172.21.15.77:0/1553049878 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f59a4109140 0x7f59a419ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:03.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.510+0000 7f59a9200640 1 --2- 172.21.15.77:0/1553049878 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59a41049c0 0x7f59a419ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:03.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.510+0000 7f59a9200640 1 -- 172.21.15.77:0/1553049878 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59a4104010 msgr2=0x7f59a419a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:03.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.510+0000 7f59a9200640 1 --2- 172.21.15.77:0/1553049878 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59a4104010 0x7f59a419a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:03.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.510+0000 7f59a9200640 1 -- 172.21.15.77:0/1553049878 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f59a4109140 msgr2=0x7f59a419ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:03.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.510+0000 7f59a9200640 1 --2- 172.21.15.77:0/1553049878 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f59a4109140 0x7f59a419ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:03.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.510+0000 7f59a9200640 1 -- 172.21.15.77:0/1553049878 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59a419f5e0 con 0x7f59a41049c0 2024-09-10T10:02:03.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.511+0000 7f59a9200640 1 --2- 172.21.15.77:0/1553049878 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59a41049c0 0x7f59a419ab70 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f599801e6f0 tx=0x7f599801ebb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:03.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.511+0000 7f59a2000640 1 -- 172.21.15.77:0/1553049878 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5998028070 con 0x7f59a41049c0 2024-09-10T10:02:03.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.511+0000 7f59a2000640 1 -- 172.21.15.77:0/1553049878 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5998005020 con 0x7f59a41049c0 2024-09-10T10:02:03.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.511+0000 7f59a2000640 1 -- 172.21.15.77:0/1553049878 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5998004d00 con 0x7f59a41049c0 2024-09-10T10:02:03.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.511+0000 7f59ac600640 1 -- 172.21.15.77:0/1553049878 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59a419f8d0 con 0x7f59a41049c0 2024-09-10T10:02:03.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.512+0000 7f59ac600640 1 -- 172.21.15.77:0/1553049878 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59a4076630 con 0x7f59a41049c0 2024-09-10T10:02:03.517 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.515+0000 7f59af24e740 1 Processor -- start 2024-09-10T10:02:03.518 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.515+0000 7f59af24e740 1 -- start start 2024-09-10T10:02:03.518 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.516+0000 7f59af24e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f1d1addb90 0x55f1d1addf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:03.518 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.516+0000 7f59af24e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f1d1ade4b0 0x55f1d1ae8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:03.518 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.516+0000 7f59af24e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f1d1ae8d70 0x55f1d1aeb160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:03.518 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.516+0000 7f59af24e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f1d1a443e0 con 0x55f1d1ae8d70 2024-09-10T10:02:03.518 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.516+0000 7f59af24e740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f1d19ebd80 con 0x55f1d1ade4b0 2024-09-10T10:02:03.518 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.516+0000 7f59af24e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f1d1a403a0 con 0x55f1d1addb90 2024-09-10T10:02:03.518 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.516+0000 7f59a9c00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f1d1ade4b0 0x55f1d1ae8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:03.519 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.516+0000 7f59a9c00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f1d1ade4b0 0x55f1d1ae8830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:50544/0 (socket says 172.21.15.77:50544) 2024-09-10T10:02:03.519 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.516+0000 7f59a9c00640 1 -- 172.21.15.77:0/2892574164 learned_addr learned my addr 172.21.15.77:0/2892574164 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:02:03.519 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.517+0000 7f59aa600640 1 --2- 172.21.15.77:0/2892574164 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f1d1addb90 0x55f1d1addf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:03.519 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.517+0000 7f59a9c00640 1 -- 172.21.15.77:0/2892574164 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f1d1addb90 msgr2=0x55f1d1addf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:03.519 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.517+0000 7f59a9c00640 1 --2- 172.21.15.77:0/2892574164 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f1d1addb90 0x55f1d1addf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:03.519 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.517+0000 7f59a9c00640 1 -- 172.21.15.77:0/2892574164 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f1d1ae8d70 msgr2=0x55f1d1aeb160 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T10:02:03.519 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.517+0000 7f59a9c00640 1 --2- 172.21.15.77:0/2892574164 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f1d1ae8d70 0x55f1d1aeb160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:03.519 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.517+0000 7f59a9c00640 1 -- 172.21.15.77:0/2892574164 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1d1a0aae0 con 0x55f1d1ade4b0 2024-09-10T10:02:03.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.518+0000 7f59aa600640 1 --2- 172.21.15.77:0/2892574164 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f1d1addb90 0x55f1d1addf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:02:03.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.518+0000 7f59a9c00640 1 --2- 172.21.15.77:0/2892574164 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f1d1ade4b0 0x55f1d1ae8830 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f5990005e80 tx=0x7f5990005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d3de0b23ddfc1c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:03.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.518+0000 7f59a2000640 1 -- 172.21.15.77:0/1553049878 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5998009a30 con 0x7f59a41049c0 2024-09-10T10:02:03.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.518+0000 7f5996a00640 1 -- 172.21.15.77:0/2892574164 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5990035040 con 0x55f1d1ade4b0 2024-09-10T10:02:03.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.518+0000 7f5996a00640 1 -- 172.21.15.77:0/2892574164 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59900044b0 con 0x55f1d1ade4b0 2024-09-10T10:02:03.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.519+0000 7f59af24e740 1 -- 172.21.15.77:0/2892574164 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f1d1ade4b0 msgr2=0x55f1d1ae8830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:03.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.519+0000 7f59af24e740 1 --2- 172.21.15.77:0/2892574164 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f1d1ade4b0 0x55f1d1ae8830 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f5990005e80 tx=0x7f5990005c00 comp rx=0 tx=0).stop 2024-09-10T10:02:03.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.519+0000 7f59af24e740 1 -- 172.21.15.77:0/2892574164 shutdown_connections 2024-09-10T10:02:03.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.519+0000 7f59af24e740 1 --2- 172.21.15.77:0/2892574164 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f1d1ae8d70 0x55f1d1aeb160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:03.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.519+0000 7f59af24e740 1 --2- 172.21.15.77:0/2892574164 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f1d1ade4b0 0x55f1d1ae8830 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:03.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.519+0000 7f59af24e740 1 --2- 172.21.15.77:0/2892574164 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f1d1addb90 0x55f1d1addf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:03.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.519+0000 7f59af24e740 1 -- 172.21.15.77:0/2892574164 >> 172.21.15.77:0/2892574164 conn(0x55f1d1ada120 msgr2=0x55f1d1ada500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:03.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.520+0000 7f59af24e740 1 -- 172.21.15.77:0/2892574164 shutdown_connections 2024-09-10T10:02:03.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.520+0000 7f59a2000640 1 --2- 172.21.15.77:0/1553049878 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f598c0a2ae0 0x7f598c0a4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:03.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.520+0000 7f59af24e740 1 -- 172.21.15.77:0/2892574164 wait complete. 2024-09-10T10:02:03.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.520+0000 7f59a2000640 1 -- 172.21.15.77:0/1553049878 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f59980dff00 con 0x7f59a41049c0 2024-09-10T10:02:03.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.520+0000 7f59a9c00640 1 --2- 172.21.15.77:0/1553049878 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f598c0a2ae0 0x7f598c0a4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:03.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.521+0000 7f59af24e740 1 -- start start 2024-09-10T10:02:03.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.521+0000 7f59af24e740 1 Processor -- start 2024-09-10T10:02:03.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.522+0000 7f59a9c00640 1 --2- 172.21.15.77:0/1553049878 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f598c0a2ae0 0x7f598c0a4fa0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f599002a9e0 tx=0x7f5990000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:03.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.522+0000 7f59af24e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f1d1addb90 0x55f1d1ae5b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:03.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.522+0000 7f59af24e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f1d1ade4b0 0x55f1d1ae60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:03.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.522+0000 7f59af24e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f1d1ae8d70 0x55f1d1ae65e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:03.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.522+0000 7f59af24e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f1d1aeba30 con 0x55f1d1ade4b0 2024-09-10T10:02:03.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.522+0000 7f59af24e740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f1d19ff0f0 con 0x55f1d1ae8d70 2024-09-10T10:02:03.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.522+0000 7f59af24e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f1d1aebbb0 con 0x55f1d1addb90 2024-09-10T10:02:03.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.522+0000 7f59af24e740 10 client.0 _ll_register_callbacks cb 0x55f1d19f09e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:02:03.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.522+0000 7f59a9c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f1d1addb90 0x55f1d1ae5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:03.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.522+0000 7f59a9c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f1d1addb90 0x55f1d1ae5b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:59930/0 (socket says 172.21.15.77:59930) 2024-09-10T10:02:03.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.522+0000 7f59a9c00640 1 -- 172.21.15.77:0/3401129565 learned_addr learned my addr 172.21.15.77:0/3401129565 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:02:03.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.522+0000 7f59a9200640 1 --2- 172.21.15.77:0/3401129565 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f1d1ade4b0 0x55f1d1ae60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:03.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.523+0000 7f59a9c00640 1 -- 172.21.15.77:0/3401129565 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f1d1ae8d70 msgr2=0x55f1d1ae65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:03.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.523+0000 7f59aa600640 1 --2- 172.21.15.77:0/3401129565 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f1d1ae8d70 0x55f1d1ae65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:03.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.523+0000 7f59a9c00640 1 --2- 172.21.15.77:0/3401129565 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f1d1ae8d70 0x55f1d1ae65e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:03.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.523+0000 7f59a9c00640 1 -- 172.21.15.77:0/3401129565 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f1d1ade4b0 msgr2=0x55f1d1ae60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:03.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.523+0000 7f59a9c00640 1 --2- 172.21.15.77:0/3401129565 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f1d1ade4b0 0x55f1d1ae60a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:03.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.523+0000 7f59a9c00640 1 -- 172.21.15.77:0/3401129565 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1d17b53c0 con 0x55f1d1addb90 2024-09-10T10:02:03.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.523+0000 7f59a9200640 1 --2- 172.21.15.77:0/3401129565 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f1d1ade4b0 0x55f1d1ae60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:02:03.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.523+0000 7f59a9c00640 1 --2- 172.21.15.77:0/3401129565 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f1d1addb90 0x55f1d1ae5b60 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5990009620 tx=0x7f5990009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3d7a4c535a92532 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:03.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.523+0000 7f598b400640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-10T10:02:03.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.523+0000 7f598b400640 1 -- 172.21.15.77:0/3401129565 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5990059070 con 0x55f1d1addb90 2024-09-10T10:02:03.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.524+0000 7f598b400640 1 -- 172.21.15.77:0/3401129565 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f5990005170 con 0x55f1d1addb90 2024-09-10T10:02:03.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.524+0000 7f59af24e740 1 -- 172.21.15.77:0/3401129565 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59900044b0 con 0x55f1d1addb90 2024-09-10T10:02:03.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.524+0000 7f598b400640 1 -- 172.21.15.77:0/3401129565 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f5990063050 con 0x55f1d1addb90 2024-09-10T10:02:03.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.524+0000 7f59af24e740 10 client.26878 fetch_fsmap learned FSMap version 35 2024-09-10T10:02:03.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.524+0000 7f59af24e740 1 -- 172.21.15.77:0/3401129565 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55f1d1aeba30 con 0x55f1d1addb90 2024-09-10T10:02:03.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.525+0000 7f598b400640 1 -- 172.21.15.77:0/3401129565 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59900044b0 con 0x55f1d1addb90 2024-09-10T10:02:03.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.525+0000 7f598b400640 1 -- 172.21.15.77:0/3401129565 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f599005e440 con 0x55f1d1addb90 2024-09-10T10:02:03.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.525+0000 7f59af24e740 10 client.26878 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T10:02:03.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.525+0000 7f59af24e740 10 client.26878 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T10:02:03.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.525+0000 7f59af24e740 20 client.26878 populate_metadata read hostname 'smithi077' 2024-09-10T10:02:03.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.525+0000 7f59af24e740 1 --2- 172.21.15.77:0/3401129565 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55f1d1af87b0 0x55f1d1afaba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:03.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.525+0000 7f59af24e740 4 client.26878 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:02:03.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.525+0000 7f59af24e740 1 -- 172.21.15.77:0/3401129565 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f1d1a51950 con 0x55f1d1af87b0 2024-09-10T10:02:03.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.526+0000 7f59a9200640 1 --2- 172.21.15.77:0/3401129565 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55f1d1af87b0 0x55f1d1afaba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:03.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.527+0000 7f598b400640 10 client.26878 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T10:02:03.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.527+0000 7f59a9200640 1 --2- 172.21.15.77:0/3401129565 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55f1d1af87b0 0x55f1d1afaba0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e38e45bf94ecfe6 server_cookie=e4f139711e232aa3 in_seq=0 out_seq=0 2024-09-10T10:02:03.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.528+0000 7f598b400640 1 -- 172.21.15.77:0/3401129565 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f1d1a51950 con 0x55f1d1af87b0 2024-09-10T10:02:03.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.528+0000 7f598b400640 10 client.26878 handle_command_reply: tid=0 2024-09-10T10:02:03.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.528+0000 7f59af24e740 1 client.26878 shutdown 2024-09-10T10:02:03.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.528+0000 7f59af24e740 1 -- 172.21.15.77:0/3401129565 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f1d1addb90 msgr2=0x55f1d1ae5b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:03.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.528+0000 7f59af24e740 1 --2- 172.21.15.77:0/3401129565 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f1d1addb90 0x55f1d1ae5b60 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5990009620 tx=0x7f5990009260 comp rx=0 tx=0).stop 2024-09-10T10:02:03.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.528+0000 7f59af24e740 1 -- 172.21.15.77:0/3401129565 shutdown_connections 2024-09-10T10:02:03.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.528+0000 7f59af24e740 1 --2- 172.21.15.77:0/3401129565 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55f1d1af87b0 0x55f1d1afaba0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:03.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.528+0000 7f59af24e740 1 --2- 172.21.15.77:0/3401129565 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f1d1ae8d70 0x55f1d1ae65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:03.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.528+0000 7f59af24e740 1 --2- 172.21.15.77:0/3401129565 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f1d1ade4b0 0x55f1d1ae60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:03.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.528+0000 7f59af24e740 1 --2- 172.21.15.77:0/3401129565 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f1d1addb90 0x55f1d1ae5b60 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:03.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.528+0000 7f598b400640 0 client.26878 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T10:02:03.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.528+0000 7f59af24e740 1 -- 172.21.15.77:0/3401129565 >> 172.21.15.77:0/3401129565 conn(0x55f1d1ada120 msgr2=0x55f1d1adfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:03.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.529+0000 7f59af24e740 1 -- 172.21.15.77:0/3401129565 shutdown_connections 2024-09-10T10:02:03.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.529+0000 7f59af24e740 1 -- 172.21.15.77:0/3401129565 wait complete. 2024-09-10T10:02:03.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.529+0000 7f59af24e740 20 client.26878 trim_cache size 0 max 16384 2024-09-10T10:02:03.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.549+0000 7f59af24e740 1 Processor -- start 2024-09-10T10:02:03.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.550+0000 7f59af24e740 1 -- start start 2024-09-10T10:02:03.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.550+0000 7f59af24e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f1d1addb90 0x55f1d1ae73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:03.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.550+0000 7f59af24e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f1d1ade4b0 0x55f1d1b00440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:03.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.550+0000 7f59af24e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f1d1ae8d70 0x55f1d1b00980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:03.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.550+0000 7f59af24e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f598400fd80 con 0x55f1d1ade4b0 2024-09-10T10:02:03.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.550+0000 7f59af24e740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f59840118c0 con 0x55f1d1ae8d70 2024-09-10T10:02:03.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.550+0000 7f59af24e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5984010450 con 0x55f1d1addb90 2024-09-10T10:02:03.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.550+0000 7f59a9c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f1d1addb90 0x55f1d1ae73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:03.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.550+0000 7f59a9c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f1d1addb90 0x55f1d1ae73c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:59934/0 (socket says 172.21.15.77:59934) 2024-09-10T10:02:03.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.550+0000 7f59a9c00640 1 -- 172.21.15.77:0/596286547 learned_addr learned my addr 172.21.15.77:0/596286547 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:02:03.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.550+0000 7f59aa600640 1 --2- 172.21.15.77:0/596286547 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f1d1ae8d70 0x55f1d1b00980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:03.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.551+0000 7f59a9c00640 1 -- 172.21.15.77:0/596286547 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f1d1ae8d70 msgr2=0x55f1d1b00980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:03.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.551+0000 7f59a9c00640 1 --2- 172.21.15.77:0/596286547 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f1d1ae8d70 0x55f1d1b00980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:03.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.551+0000 7f59a9c00640 1 -- 172.21.15.77:0/596286547 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f1d1ade4b0 msgr2=0x55f1d1b00440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:03.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.551+0000 7f59a9c00640 1 --2- 172.21.15.77:0/596286547 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f1d1ade4b0 0x55f1d1b00440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:03.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.551+0000 7f59a9c00640 1 -- 172.21.15.77:0/596286547 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1d1adb290 con 0x55f1d1addb90 2024-09-10T10:02:03.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.551+0000 7f59a9c00640 1 --2- 172.21.15.77:0/596286547 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f1d1addb90 0x55f1d1ae73c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f599004b040 tx=0x7f5990004e10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41c79fbaaa71e57b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:03.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.551+0000 7f5996000640 1 -- 172.21.15.77:0/596286547 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5990059070 con 0x55f1d1addb90 2024-09-10T10:02:03.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.552+0000 7f5996000640 1 -- 172.21.15.77:0/596286547 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5990004a50 con 0x55f1d1addb90 2024-09-10T10:02:03.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.552+0000 7f5996000640 1 -- 172.21.15.77:0/596286547 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5990035050 con 0x55f1d1addb90 2024-09-10T10:02:03.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.552+0000 7f59af24e740 1 -- 172.21.15.77:0/596286547 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f1d1addb90 msgr2=0x55f1d1ae73c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:03.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.552+0000 7f59af24e740 1 --2- 172.21.15.77:0/596286547 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f1d1addb90 0x55f1d1ae73c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f599004b040 tx=0x7f5990004e10 comp rx=0 tx=0).stop 2024-09-10T10:02:03.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.553+0000 7f59af24e740 1 -- 172.21.15.77:0/596286547 shutdown_connections 2024-09-10T10:02:03.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.553+0000 7f59af24e740 1 --2- 172.21.15.77:0/596286547 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f1d1ae8d70 0x55f1d1b00980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:03.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.553+0000 7f59af24e740 1 --2- 172.21.15.77:0/596286547 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f1d1ade4b0 0x55f1d1b00440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:03.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.553+0000 7f59af24e740 1 --2- 172.21.15.77:0/596286547 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f1d1addb90 0x55f1d1ae73c0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:03.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.553+0000 7f59af24e740 1 -- 172.21.15.77:0/596286547 >> 172.21.15.77:0/596286547 conn(0x55f1d1ada120 msgr2=0x55f1d1ad91e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:03.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.553+0000 7f59af24e740 1 -- 172.21.15.77:0/596286547 shutdown_connections 2024-09-10T10:02:03.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.553+0000 7f59af24e740 1 -- 172.21.15.77:0/596286547 wait complete. 2024-09-10T10:02:03.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.554+0000 7f59af24e740 1 -- start start 2024-09-10T10:02:03.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.554+0000 7f59af24e740 1 Processor -- start 2024-09-10T10:02:03.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.555+0000 7f59af24e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f1d1addb90 0x55f1d1af1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:03.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.555+0000 7f59af24e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f1d1ade4b0 0x55f1d1ae29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:03.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.555+0000 7f59af24e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f1d1ae8d70 0x55f1d1ae3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:03.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.555+0000 7f59af24e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f59840111f0 con 0x55f1d1ade4b0 2024-09-10T10:02:03.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.555+0000 7f59af24e740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5984010b20 con 0x55f1d1addb90 2024-09-10T10:02:03.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.555+0000 7f59af24e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f598400dd60 con 0x55f1d1ae8d70 2024-09-10T10:02:03.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.555+0000 7f59af24e740 10 client.0 _ll_register_callbacks cb 0x7f5990009520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:02:03.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.555+0000 7f59a9200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f1d1ade4b0 0x55f1d1ae29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:03.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.555+0000 7f59aa600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f1d1ae8d70 0x55f1d1ae3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:03.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.555+0000 7f59a9200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f1d1ade4b0 0x55f1d1ae29c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33956/0 (socket says 172.21.15.77:33956) 2024-09-10T10:02:03.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.555+0000 7f59a9200640 1 -- 172.21.15.77:0/1526632179 learned_addr learned my addr 172.21.15.77:0/1526632179 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:02:03.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.556+0000 7f59aa600640 1 -- 172.21.15.77:0/1526632179 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f1d1addb90 msgr2=0x55f1d1af1790 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T10:02:03.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.556+0000 7f59aa600640 1 --2- 172.21.15.77:0/1526632179 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f1d1addb90 0x55f1d1af1790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:03.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.556+0000 7f59aa600640 1 -- 172.21.15.77:0/1526632179 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f1d1ade4b0 msgr2=0x55f1d1ae29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:03.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.556+0000 7f59aa600640 1 --2- 172.21.15.77:0/1526632179 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f1d1ade4b0 0x55f1d1ae29c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:03.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.556+0000 7f59aa600640 1 -- 172.21.15.77:0/1526632179 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1d1aef210 con 0x55f1d1ae8d70 2024-09-10T10:02:03.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.556+0000 7f59aa600640 1 --2- 172.21.15.77:0/1526632179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f1d1ae8d70 0x55f1d1ae3150 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f599c0042c0 tx=0x7f599c003f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a9539b08ba96208 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:03.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.556+0000 7f598b400640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-10T10:02:03.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.556+0000 7f598b400640 1 -- 172.21.15.77:0/1526632179 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f599c013b50 con 0x55f1d1ae8d70 2024-09-10T10:02:03.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.556+0000 7f598b400640 1 -- 172.21.15.77:0/1526632179 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f599c013cf0 con 0x55f1d1ae8d70 2024-09-10T10:02:03.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.556+0000 7f59af24e740 1 -- 172.21.15.77:0/1526632179 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5990004a50 con 0x55f1d1ae8d70 2024-09-10T10:02:03.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.557+0000 7f598b400640 1 -- 172.21.15.77:0/1526632179 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f599c004b20 con 0x55f1d1ae8d70 2024-09-10T10:02:03.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.557+0000 7f59af24e740 10 client.26884 fetch_fsmap learned FSMap version 35 2024-09-10T10:02:03.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.557+0000 7f59af24e740 1 -- 172.21.15.77:0/1526632179 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55f1d1b63590 con 0x55f1d1ae8d70 2024-09-10T10:02:03.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.557+0000 7f598b400640 1 -- 172.21.15.77:0/1526632179 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f599c004cc0 con 0x55f1d1ae8d70 2024-09-10T10:02:03.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.558+0000 7f598b400640 1 -- 172.21.15.77:0/1526632179 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f599c010430 con 0x55f1d1ae8d70 2024-09-10T10:02:03.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.558+0000 7f59af24e740 10 client.26884 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T10:02:03.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.558+0000 7f59af24e740 10 client.26884 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T10:02:03.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.558+0000 7f59af24e740 20 client.26884 populate_metadata read hostname 'smithi077' 2024-09-10T10:02:03.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.558+0000 7f59af24e740 1 --2- 172.21.15.77:0/1526632179 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55f1d1b63d60 0x55f1d1b66180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:03.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.558+0000 7f59af24e740 4 client.26884 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T10:02:03.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.558+0000 7f59af24e740 1 -- 172.21.15.77:0/1526632179 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f5990004a50 con 0x55f1d1b63d60 2024-09-10T10:02:03.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.559+0000 7f59a9c00640 1 --2- 172.21.15.77:0/1526632179 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55f1d1b63d60 0x55f1d1b66180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:03.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.559+0000 7f59a9c00640 1 --2- 172.21.15.77:0/1526632179 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55f1d1b63d60 0x55f1d1b66180 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2aef6c78edf26d3 server_cookie=263a76c4f7adce75 in_seq=0 out_seq=0 2024-09-10T10:02:03.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.559+0000 7f598b400640 10 client.26884 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T10:02:03.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.560+0000 7f598b400640 1 -- 172.21.15.77:0/1526632179 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7f5990004a50 con 0x55f1d1b63d60 2024-09-10T10:02:03.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.560+0000 7f598b400640 10 client.26884 handle_command_reply: tid=0 2024-09-10T10:02:03.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.560+0000 7f59af24e740 1 client.26884 shutdown 2024-09-10T10:02:03.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.560+0000 7f59af24e740 1 -- 172.21.15.77:0/1526632179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f1d1ae8d70 msgr2=0x55f1d1ae3150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:03.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.560+0000 7f59af24e740 1 --2- 172.21.15.77:0/1526632179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f1d1ae8d70 0x55f1d1ae3150 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f599c0042c0 tx=0x7f599c003f90 comp rx=0 tx=0).stop 2024-09-10T10:02:03.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.560+0000 7f59af24e740 1 -- 172.21.15.77:0/1526632179 shutdown_connections 2024-09-10T10:02:03.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.560+0000 7f59af24e740 1 --2- 172.21.15.77:0/1526632179 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55f1d1b63d60 0x55f1d1b66180 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:03.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.560+0000 7f59af24e740 1 --2- 172.21.15.77:0/1526632179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f1d1ae8d70 0x55f1d1ae3150 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:03.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.560+0000 7f598b400640 0 client.26884 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T10:02:03.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.560+0000 7f59af24e740 1 --2- 172.21.15.77:0/1526632179 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f1d1ade4b0 0x55f1d1ae29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:03.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.560+0000 7f59af24e740 1 --2- 172.21.15.77:0/1526632179 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f1d1addb90 0x55f1d1af1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:03.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.560+0000 7f59af24e740 1 -- 172.21.15.77:0/1526632179 >> 172.21.15.77:0/1526632179 conn(0x55f1d1ada120 msgr2=0x55f1d1ae4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:03.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.560+0000 7f59af24e740 1 -- 172.21.15.77:0/1526632179 shutdown_connections 2024-09-10T10:02:03.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.561+0000 7f59af24e740 1 -- 172.21.15.77:0/1526632179 wait complete. 2024-09-10T10:02:03.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.561+0000 7f59af24e740 20 client.26884 trim_cache size 0 max 16384 2024-09-10T10:02:03.564 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-10T10:02:03.564 INFO:teuthology.orchestra.run.smithi077.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T10:02:03.565 INFO:teuthology.orchestra.run.smithi077.stdout: "scrubs": { 2024-09-10T10:02:03.565 INFO:teuthology.orchestra.run.smithi077.stdout: "d5e9b76b-b7eb-4716-995f-80783f23eb2c": { 2024-09-10T10:02:03.565 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/", 2024-09-10T10:02:03.565 INFO:teuthology.orchestra.run.smithi077.stdout: "tag": "d5e9b76b-b7eb-4716-995f-80783f23eb2c", 2024-09-10T10:02:03.565 INFO:teuthology.orchestra.run.smithi077.stdout: "options": "recursive,force" 2024-09-10T10:02:03.565 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:02:03.565 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:02:03.565 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-10T10:02:03.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.562+0000 7f59ac600640 1 -- 172.21.15.77:0/1553049878 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f598c0a2ae0 msgr2=0x7f598c0a4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:03.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.562+0000 7f59ac600640 1 --2- 172.21.15.77:0/1553049878 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f598c0a2ae0 0x7f598c0a4fa0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f599002a9e0 tx=0x7f5990000bc0 comp rx=0 tx=0).stop 2024-09-10T10:02:03.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.562+0000 7f59ac600640 1 -- 172.21.15.77:0/1553049878 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59a41049c0 msgr2=0x7f59a419ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:03.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.562+0000 7f59ac600640 1 --2- 172.21.15.77:0/1553049878 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59a41049c0 0x7f59a419ab70 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f599801e6f0 tx=0x7f599801ebb0 comp rx=0 tx=0).stop 2024-09-10T10:02:03.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.562+0000 7f59ac600640 1 -- 172.21.15.77:0/1553049878 shutdown_connections 2024-09-10T10:02:03.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.562+0000 7f59ac600640 1 --2- 172.21.15.77:0/1553049878 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f598c0a2ae0 0x7f598c0a4fa0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:03.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.562+0000 7f59ac600640 1 --2- 172.21.15.77:0/1553049878 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f59a4109140 0x7f59a419ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:03.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.562+0000 7f59ac600640 1 --2- 172.21.15.77:0/1553049878 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59a41049c0 0x7f59a419ab70 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:03.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.562+0000 7f59ac600640 1 --2- 172.21.15.77:0/1553049878 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59a4104010 0x7f59a419a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:03.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.562+0000 7f59ac600640 1 -- 172.21.15.77:0/1553049878 >> 172.21.15.77:0/1553049878 conn(0x7f59a40ffa50 msgr2=0x7f59a4100df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:03.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.563+0000 7f59ac600640 1 -- 172.21.15.77:0/1553049878 shutdown_connections 2024-09-10T10:02:03.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:03.563+0000 7f59ac600640 1 -- 172.21.15.77:0/1553049878 wait complete. 2024-09-10T10:02:03.572 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:02:04.333 INFO:tasks.cephfs.filesystem:scrub status for tag:d5e9b76b-b7eb-4716-995f-80783f23eb2c - {'path': '/', 'tag': 'd5e9b76b-b7eb-4716-995f-80783f23eb2c', 'options': 'recursive,force'} 2024-09-10T10:02:04.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.911+0000 7f8225e00640 1 -- 172.21.15.77:0/2100372986 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f82201049a0 msgr2=0x7f82201089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:04.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.911+0000 7f8225e00640 1 --2- 172.21.15.77:0/2100372986 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f82201049a0 0x7f82201089f0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f820c009860 tx=0x7f820c02a340 comp rx=0 tx=0).stop 2024-09-10T10:02:04.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.911+0000 7f8225e00640 1 -- 172.21.15.77:0/2100372986 shutdown_connections 2024-09-10T10:02:04.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.911+0000 7f8225e00640 1 --2- 172.21.15.77:0/2100372986 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8220109120 0x7f8220110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:04.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.911+0000 7f8225e00640 1 --2- 172.21.15.77:0/2100372986 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f82201049a0 0x7f82201089f0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:04.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.911+0000 7f8225e00640 1 --2- 172.21.15.77:0/2100372986 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8220103ff0 0x7f82201043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:04.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.911+0000 7f8225e00640 1 -- 172.21.15.77:0/2100372986 >> 172.21.15.77:0/2100372986 conn(0x7f82200ffa90 msgr2=0x7f8220101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:04.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.911+0000 7f8225e00640 1 -- 172.21.15.77:0/2100372986 shutdown_connections 2024-09-10T10:02:04.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.911+0000 7f8225e00640 1 -- 172.21.15.77:0/2100372986 wait complete. 2024-09-10T10:02:04.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.912+0000 7f8225e00640 1 Processor -- start 2024-09-10T10:02:04.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.912+0000 7f8225e00640 1 -- start start 2024-09-10T10:02:04.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.913+0000 7f8225e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8220103ff0 0x7f822019a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:04.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.913+0000 7f8225e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f82201049a0 0x7f822019abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:04.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.913+0000 7f8225e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8220109120 0x7f822019ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:04.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.913+0000 7f8225e00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8220114120 con 0x7f82201049a0 2024-09-10T10:02:04.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.913+0000 7f8225e00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8220113fa0 con 0x7f8220109120 2024-09-10T10:02:04.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.913+0000 7f8225e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f82201142a0 con 0x7f8220103ff0 2024-09-10T10:02:04.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.913+0000 7f821e000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f82201049a0 0x7f822019abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:04.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.913+0000 7f821ea00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8220103ff0 0x7f822019a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:04.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.913+0000 7f821e000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f82201049a0 0x7f822019abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33978/0 (socket says 172.21.15.77:33978) 2024-09-10T10:02:04.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.913+0000 7f821e000640 1 -- 172.21.15.77:0/1179952177 learned_addr learned my addr 172.21.15.77:0/1179952177 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:02:04.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.913+0000 7f821f400640 1 --2- 172.21.15.77:0/1179952177 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8220109120 0x7f822019ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:04.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.914+0000 7f821e000640 1 -- 172.21.15.77:0/1179952177 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8220103ff0 msgr2=0x7f822019a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:04.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.914+0000 7f821e000640 1 --2- 172.21.15.77:0/1179952177 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8220103ff0 0x7f822019a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:04.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.914+0000 7f821e000640 1 -- 172.21.15.77:0/1179952177 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8220109120 msgr2=0x7f822019ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:04.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.914+0000 7f821e000640 1 --2- 172.21.15.77:0/1179952177 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8220109120 0x7f822019ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:04.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.914+0000 7f821e000640 1 -- 172.21.15.77:0/1179952177 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f822019f650 con 0x7f82201049a0 2024-09-10T10:02:04.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.914+0000 7f821e000640 1 --2- 172.21.15.77:0/1179952177 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f82201049a0 0x7f822019abe0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f820c035040 tx=0x7f820c033d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:04.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.914+0000 7f8224a00640 1 -- 172.21.15.77:0/1179952177 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f820c03f020 con 0x7f82201049a0 2024-09-10T10:02:04.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.914+0000 7f8224a00640 1 -- 172.21.15.77:0/1179952177 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f820c032770 con 0x7f82201049a0 2024-09-10T10:02:04.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.914+0000 7f8224a00640 1 -- 172.21.15.77:0/1179952177 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f820c037040 con 0x7f82201049a0 2024-09-10T10:02:04.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.914+0000 7f8225e00640 1 -- 172.21.15.77:0/1179952177 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f822019f8e0 con 0x7f82201049a0 2024-09-10T10:02:04.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.915+0000 7f8225e00640 1 -- 172.21.15.77:0/1179952177 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f822019fd70 con 0x7f82201049a0 2024-09-10T10:02:04.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.918+0000 7f8228882740 1 Processor -- start 2024-09-10T10:02:04.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.918+0000 7f8228882740 1 -- start start 2024-09-10T10:02:04.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.918+0000 7f8228882740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1e042b3b0 0x55a1e042b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:04.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.918+0000 7f8228882740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1e042bcd0 0x55a1e0436050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:04.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.918+0000 7f8228882740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a1e0436590 0x55a1e0438980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:04.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.918+0000 7f8228882740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55a1e0391530 con 0x55a1e0436590 2024-09-10T10:02:04.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.918+0000 7f8228882740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a1e0338860 con 0x55a1e042bcd0 2024-09-10T10:02:04.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.918+0000 7f8228882740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a1e038d4f0 con 0x55a1e042b3b0 2024-09-10T10:02:04.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.918+0000 7f821f400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1e042b3b0 0x55a1e042b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:04.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.918+0000 7f821f400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1e042b3b0 0x55a1e042b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:59960/0 (socket says 172.21.15.77:59960) 2024-09-10T10:02:04.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.918+0000 7f821f400640 1 -- 172.21.15.77:0/1473871603 learned_addr learned my addr 172.21.15.77:0/1473871603 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:02:04.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.918+0000 7f8224a00640 1 -- 172.21.15.77:0/1179952177 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f820c042020 con 0x7f82201049a0 2024-09-10T10:02:04.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.919+0000 7f821ea00640 1 --2- 172.21.15.77:0/1473871603 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1e042bcd0 0x55a1e0436050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:04.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.919+0000 7f821f400640 1 -- 172.21.15.77:0/1473871603 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1e042bcd0 msgr2=0x55a1e0436050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:04.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.919+0000 7f821f400640 1 --2- 172.21.15.77:0/1473871603 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1e042bcd0 0x55a1e0436050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:04.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.919+0000 7f821f400640 1 -- 172.21.15.77:0/1473871603 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a1e0436590 msgr2=0x55a1e0438980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:04.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.919+0000 7f821e000640 1 --2- 172.21.15.77:0/1473871603 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a1e0436590 0x55a1e0438980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:04.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.919+0000 7f821f400640 1 --2- 172.21.15.77:0/1473871603 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a1e0436590 0x55a1e0438980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:04.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.919+0000 7f821f400640 1 -- 172.21.15.77:0/1473871603 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1e0102b90 con 0x55a1e042b3b0 2024-09-10T10:02:04.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.919+0000 7f821e000640 1 --2- 172.21.15.77:0/1473871603 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a1e0436590 0x55a1e0438980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:02:04.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.919+0000 7f821f400640 1 --2- 172.21.15.77:0/1473871603 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1e042b3b0 0x55a1e042b790 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8218009f50 tx=0x7f82180118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90707d9050c79cf9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:04.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.920+0000 7f820be00640 1 -- 172.21.15.77:0/1473871603 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8218004d20 con 0x55a1e042b3b0 2024-09-10T10:02:04.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.920+0000 7f820be00640 1 -- 172.21.15.77:0/1473871603 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8218004050 con 0x55a1e042b3b0 2024-09-10T10:02:04.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.920+0000 7f820be00640 1 -- 172.21.15.77:0/1473871603 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8218005260 con 0x55a1e042b3b0 2024-09-10T10:02:04.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.920+0000 7f8224a00640 1 --2- 172.21.15.77:0/1179952177 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f82040a2b80 0x7f82040a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:04.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.920+0000 7f8224a00640 1 -- 172.21.15.77:0/1179952177 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f820c0eefe0 con 0x7f82201049a0 2024-09-10T10:02:04.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.920+0000 7f821ea00640 1 --2- 172.21.15.77:0/1179952177 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f82040a2b80 0x7f82040a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:04.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.921+0000 7f8228882740 1 -- 172.21.15.77:0/1473871603 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1e042b3b0 msgr2=0x55a1e042b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:04.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.921+0000 7f8228882740 1 --2- 172.21.15.77:0/1473871603 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1e042b3b0 0x55a1e042b790 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8218009f50 tx=0x7f82180118d0 comp rx=0 tx=0).stop 2024-09-10T10:02:04.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.921+0000 7f8228882740 1 -- 172.21.15.77:0/1473871603 shutdown_connections 2024-09-10T10:02:04.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.921+0000 7f8228882740 1 --2- 172.21.15.77:0/1473871603 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a1e0436590 0x55a1e0438980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:04.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.921+0000 7f8228882740 1 --2- 172.21.15.77:0/1473871603 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1e042bcd0 0x55a1e0436050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:04.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.921+0000 7f8228882740 1 --2- 172.21.15.77:0/1473871603 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1e042b3b0 0x55a1e042b790 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:04.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.921+0000 7f8228882740 1 -- 172.21.15.77:0/1473871603 >> 172.21.15.77:0/1473871603 conn(0x55a1e0427940 msgr2=0x55a1e0427d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:04.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.921+0000 7f8228882740 1 -- 172.21.15.77:0/1473871603 shutdown_connections 2024-09-10T10:02:04.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.921+0000 7f8228882740 1 -- 172.21.15.77:0/1473871603 wait complete. 2024-09-10T10:02:04.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.921+0000 7f821ea00640 1 --2- 172.21.15.77:0/1179952177 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f82040a2b80 0x7f82040a5040 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x55a1e04287a0 tx=0x7f8210009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:04.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.922+0000 7f8228882740 1 -- start start 2024-09-10T10:02:04.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.922+0000 7f8228882740 1 Processor -- start 2024-09-10T10:02:04.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.922+0000 7f8228882740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1e042b3b0 0x55a1e0433700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:04.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.922+0000 7f8228882740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a1e042bcd0 0x55a1e0433c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:04.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.922+0000 7f8228882740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1e0436590 0x55a1e0434180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:04.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.922+0000 7f8228882740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55a1e0438fb0 con 0x55a1e042bcd0 2024-09-10T10:02:04.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.922+0000 7f8228882740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a1e034be60 con 0x55a1e0436590 2024-09-10T10:02:04.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.922+0000 7f8228882740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a1e0439130 con 0x55a1e042b3b0 2024-09-10T10:02:04.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.923+0000 7f8228882740 10 client.0 _ll_register_callbacks cb 0x55a1e0350cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:02:04.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.923+0000 7f821ea00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1e042b3b0 0x55a1e0433700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:04.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.923+0000 7f821ea00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1e042b3b0 0x55a1e0433700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:59962/0 (socket says 172.21.15.77:59962) 2024-09-10T10:02:04.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.923+0000 7f821ea00640 1 -- 172.21.15.77:0/316252530 learned_addr learned my addr 172.21.15.77:0/316252530 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:02:04.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.923+0000 7f821e000640 1 --2- 172.21.15.77:0/316252530 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a1e042bcd0 0x55a1e0433c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:04.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.923+0000 7f821f400640 1 --2- 172.21.15.77:0/316252530 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1e0436590 0x55a1e0434180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:04.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.923+0000 7f821ea00640 1 -- 172.21.15.77:0/316252530 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1e0436590 msgr2=0x55a1e0434180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:04.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.923+0000 7f821ea00640 1 --2- 172.21.15.77:0/316252530 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1e0436590 0x55a1e0434180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:04.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.923+0000 7f821ea00640 1 -- 172.21.15.77:0/316252530 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a1e042bcd0 msgr2=0x55a1e0433c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:04.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.923+0000 7f821ea00640 1 --2- 172.21.15.77:0/316252530 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a1e042bcd0 0x55a1e0433c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:04.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.923+0000 7f821ea00640 1 -- 172.21.15.77:0/316252530 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1e0357990 con 0x55a1e042b3b0 2024-09-10T10:02:04.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.923+0000 7f821e000640 1 --2- 172.21.15.77:0/316252530 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a1e042bcd0 0x55a1e0433c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:02:04.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.924+0000 7f821ea00640 1 --2- 172.21.15.77:0/316252530 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1e042b3b0 0x55a1e0433700 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f82100058a0 tx=0x7f8210004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cfef86502cfb28e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:04.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.924+0000 7f8208c00640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-10T10:02:04.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.924+0000 7f8208c00640 1 -- 172.21.15.77:0/316252530 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f821002c060 con 0x55a1e042b3b0 2024-09-10T10:02:04.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.924+0000 7f8208c00640 1 -- 172.21.15.77:0/316252530 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f821001d650 con 0x55a1e042b3b0 2024-09-10T10:02:04.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.924+0000 7f8228882740 1 -- 172.21.15.77:0/316252530 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8218004050 con 0x55a1e042b3b0 2024-09-10T10:02:04.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.924+0000 7f8208c00640 1 -- 172.21.15.77:0/316252530 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f8210033050 con 0x55a1e042b3b0 2024-09-10T10:02:04.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.924+0000 7f8228882740 10 client.26896 fetch_fsmap learned FSMap version 35 2024-09-10T10:02:04.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.924+0000 7f8228882740 1 -- 172.21.15.77:0/316252530 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55a1e0444d60 con 0x55a1e042b3b0 2024-09-10T10:02:04.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.925+0000 7f8208c00640 1 -- 172.21.15.77:0/316252530 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8218004050 con 0x55a1e042b3b0 2024-09-10T10:02:04.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.925+0000 7f8208c00640 1 -- 172.21.15.77:0/316252530 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f8210026030 con 0x55a1e042b3b0 2024-09-10T10:02:04.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.925+0000 7f8228882740 10 client.26896 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T10:02:04.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.925+0000 7f8228882740 10 client.26896 resolve_mds: validated gid 24563 aka daemon mds.b 2024-09-10T10:02:04.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.925+0000 7f8228882740 20 client.26896 populate_metadata read hostname 'smithi077' 2024-09-10T10:02:04.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.925+0000 7f8228882740 1 --2- 172.21.15.77:0/316252530 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55a1e0446220 0x55a1e0448650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:04.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.925+0000 7f8228882740 4 client.26896 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:02:04.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.925+0000 7f8228882740 1 -- 172.21.15.77:0/316252530 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a1e0334310 con 0x55a1e0446220 2024-09-10T10:02:04.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.925+0000 7f821e000640 1 --2- 172.21.15.77:0/316252530 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55a1e0446220 0x55a1e0448650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:04.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.926+0000 7f821e000640 1 --2- 172.21.15.77:0/316252530 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55a1e0446220 0x55a1e0448650 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b5053ebc02bccf4 server_cookie=f3c86a18863d00c7 in_seq=0 out_seq=0 2024-09-10T10:02:04.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.926+0000 7f8208c00640 10 client.26896 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T10:02:04.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.926+0000 7f8208c00640 1 -- 172.21.15.77:0/316252530 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a1e0334310 con 0x55a1e0446220 2024-09-10T10:02:04.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.926+0000 7f8208c00640 10 client.26896 handle_command_reply: tid=0 2024-09-10T10:02:04.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.927+0000 7f8228882740 1 client.26896 shutdown 2024-09-10T10:02:04.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.927+0000 7f8228882740 1 -- 172.21.15.77:0/316252530 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1e042b3b0 msgr2=0x55a1e0433700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:04.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.927+0000 7f8228882740 1 --2- 172.21.15.77:0/316252530 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1e042b3b0 0x55a1e0433700 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f82100058a0 tx=0x7f8210004730 comp rx=0 tx=0).stop 2024-09-10T10:02:04.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.927+0000 7f8228882740 1 -- 172.21.15.77:0/316252530 shutdown_connections 2024-09-10T10:02:04.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.927+0000 7f8228882740 1 --2- 172.21.15.77:0/316252530 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55a1e0446220 0x55a1e0448650 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:04.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.927+0000 7f8228882740 1 --2- 172.21.15.77:0/316252530 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1e0436590 0x55a1e0434180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:04.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.927+0000 7f8228882740 1 --2- 172.21.15.77:0/316252530 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a1e042bcd0 0x55a1e0433c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:04.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.927+0000 7f8228882740 1 --2- 172.21.15.77:0/316252530 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1e042b3b0 0x55a1e0433700 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:04.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.927+0000 7f8228882740 1 -- 172.21.15.77:0/316252530 >> 172.21.15.77:0/316252530 conn(0x55a1e0427940 msgr2=0x55a1e0426c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:04.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.927+0000 7f8208c00640 0 client.26896 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T10:02:04.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.927+0000 7f8228882740 1 -- 172.21.15.77:0/316252530 shutdown_connections 2024-09-10T10:02:04.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.927+0000 7f8228882740 1 -- 172.21.15.77:0/316252530 wait complete. 2024-09-10T10:02:04.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.927+0000 7f8228882740 20 client.26896 trim_cache size 0 max 16384 2024-09-10T10:02:04.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.955+0000 7f8228882740 1 Processor -- start 2024-09-10T10:02:04.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.955+0000 7f8228882740 1 -- start start 2024-09-10T10:02:04.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.956+0000 7f8228882740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a1e042b3b0 0x55a1e044def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:04.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.956+0000 7f8228882740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1e042bcd0 0x55a1e044e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:04.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.957+0000 7f8228882740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1e0436590 0x55a1e04309a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:04.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.957+0000 7f8228882740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f82201b10e0 con 0x55a1e042b3b0 2024-09-10T10:02:04.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.957+0000 7f8228882740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f82201b2c20 con 0x55a1e0436590 2024-09-10T10:02:04.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.957+0000 7f821ea00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a1e042b3b0 0x55a1e044def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:04.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.957+0000 7f8228882740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f82201b17b0 con 0x55a1e042bcd0 2024-09-10T10:02:04.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.957+0000 7f821e000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1e042bcd0 0x55a1e044e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:04.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.957+0000 7f821ea00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a1e042b3b0 0x55a1e044def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33998/0 (socket says 172.21.15.77:33998) 2024-09-10T10:02:04.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.957+0000 7f821ea00640 1 -- 172.21.15.77:0/1151925598 learned_addr learned my addr 172.21.15.77:0/1151925598 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:02:04.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.957+0000 7f821f400640 1 --2- 172.21.15.77:0/1151925598 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1e0436590 0x55a1e04309a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:04.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.958+0000 7f821f400640 1 -- 172.21.15.77:0/1151925598 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1e042bcd0 msgr2=0x55a1e044e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:04.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.958+0000 7f821f400640 1 --2- 172.21.15.77:0/1151925598 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1e042bcd0 0x55a1e044e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:04.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.958+0000 7f821f400640 1 -- 172.21.15.77:0/1151925598 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a1e042b3b0 msgr2=0x55a1e044def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:04.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.958+0000 7f821f400640 1 --2- 172.21.15.77:0/1151925598 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a1e042b3b0 0x55a1e044def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:04.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.958+0000 7f821f400640 1 -- 172.21.15.77:0/1151925598 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1e0438cf0 con 0x55a1e0436590 2024-09-10T10:02:04.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.959+0000 7f821f400640 1 --2- 172.21.15.77:0/1151925598 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1e0436590 0x55a1e04309a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f8218012fd0 tx=0x7f8218005bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4523439607c898e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:04.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.961+0000 7f820b400640 1 -- 172.21.15.77:0/1151925598 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8218004050 con 0x55a1e0436590 2024-09-10T10:02:04.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.961+0000 7f820b400640 1 -- 172.21.15.77:0/1151925598 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8218004820 con 0x55a1e0436590 2024-09-10T10:02:04.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.961+0000 7f820b400640 1 -- 172.21.15.77:0/1151925598 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8218005380 con 0x55a1e0436590 2024-09-10T10:02:04.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.963+0000 7f8228882740 1 -- 172.21.15.77:0/1151925598 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1e0436590 msgr2=0x55a1e04309a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:04.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.963+0000 7f8228882740 1 --2- 172.21.15.77:0/1151925598 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1e0436590 0x55a1e04309a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f8218012fd0 tx=0x7f8218005bd0 comp rx=0 tx=0).stop 2024-09-10T10:02:04.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.963+0000 7f8228882740 1 -- 172.21.15.77:0/1151925598 shutdown_connections 2024-09-10T10:02:04.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.963+0000 7f8228882740 1 --2- 172.21.15.77:0/1151925598 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1e0436590 0x55a1e04309a0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:04.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.963+0000 7f8228882740 1 --2- 172.21.15.77:0/1151925598 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1e042bcd0 0x55a1e044e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:04.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.963+0000 7f8228882740 1 --2- 172.21.15.77:0/1151925598 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a1e042b3b0 0x55a1e044def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:04.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.963+0000 7f8228882740 1 -- 172.21.15.77:0/1151925598 >> 172.21.15.77:0/1151925598 conn(0x55a1e0427940 msgr2=0x55a1e0426420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:04.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.963+0000 7f8228882740 1 -- 172.21.15.77:0/1151925598 shutdown_connections 2024-09-10T10:02:04.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.963+0000 7f8228882740 1 -- 172.21.15.77:0/1151925598 wait complete. 2024-09-10T10:02:04.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.964+0000 7f8228882740 1 -- start start 2024-09-10T10:02:04.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.964+0000 7f8228882740 1 Processor -- start 2024-09-10T10:02:04.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.966+0000 7f8228882740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1e042b3b0 0x55a1e043e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:04.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.966+0000 7f8228882740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1e042bcd0 0x55a1e043b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:04.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.966+0000 7f8228882740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a1e0436590 0x55a1e043b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:04.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.966+0000 7f8228882740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f82201b2550 con 0x55a1e0436590 2024-09-10T10:02:04.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.966+0000 7f8228882740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f82201b1e80 con 0x55a1e042b3b0 2024-09-10T10:02:04.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.966+0000 7f8228882740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f82201af0c0 con 0x55a1e042bcd0 2024-09-10T10:02:04.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.966+0000 7f8228882740 10 client.0 _ll_register_callbacks cb 0x7f8210009de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:02:04.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.966+0000 7f821e000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1e042bcd0 0x55a1e043b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:04.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.966+0000 7f821ea00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1e042b3b0 0x55a1e043e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:04.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.966+0000 7f821e000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1e042bcd0 0x55a1e043b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:59980/0 (socket says 172.21.15.77:59980) 2024-09-10T10:02:04.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.966+0000 7f821e000640 1 -- 172.21.15.77:0/236836692 learned_addr learned my addr 172.21.15.77:0/236836692 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:02:04.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.967+0000 7f821e000640 1 -- 172.21.15.77:0/236836692 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1e042b3b0 msgr2=0x55a1e043e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:04.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.967+0000 7f821e000640 1 --2- 172.21.15.77:0/236836692 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1e042b3b0 0x55a1e043e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:04.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.967+0000 7f821e000640 1 -- 172.21.15.77:0/236836692 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a1e0436590 msgr2=0x55a1e043b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:04.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.967+0000 7f821e000640 1 --2- 172.21.15.77:0/236836692 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a1e0436590 0x55a1e043b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:04.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.967+0000 7f821e000640 1 -- 172.21.15.77:0/236836692 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1e043bce0 con 0x55a1e042bcd0 2024-09-10T10:02:04.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.967+0000 7f821e000640 1 --2- 172.21.15.77:0/236836692 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1e042bcd0 0x55a1e043b260 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f820c0bb3a0 tx=0x7f820c0bad70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3b5c8d8f855df41 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:04.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.967+0000 7f8208c00640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-10T10:02:04.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.967+0000 7f8208c00640 1 -- 172.21.15.77:0/236836692 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f820c0bb410 con 0x55a1e042bcd0 2024-09-10T10:02:04.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.967+0000 7f8228882740 1 -- 172.21.15.77:0/236836692 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8218004820 con 0x55a1e042bcd0 2024-09-10T10:02:04.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.967+0000 7f8208c00640 1 -- 172.21.15.77:0/236836692 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f820c0bf490 con 0x55a1e042bcd0 2024-09-10T10:02:04.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.967+0000 7f8208c00640 1 -- 172.21.15.77:0/236836692 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f820c0bf7c0 con 0x55a1e042bcd0 2024-09-10T10:02:04.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.967+0000 7f8228882740 10 client.26902 fetch_fsmap learned FSMap version 35 2024-09-10T10:02:04.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.967+0000 7f8228882740 1 -- 172.21.15.77:0/236836692 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55a1e04ba3a0 con 0x55a1e042bcd0 2024-09-10T10:02:04.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.968+0000 7f8208c00640 1 -- 172.21.15.77:0/236836692 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8218004820 con 0x55a1e042bcd0 2024-09-10T10:02:04.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.968+0000 7f8208c00640 1 -- 172.21.15.77:0/236836692 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f820c03a030 con 0x55a1e042bcd0 2024-09-10T10:02:04.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.968+0000 7f8228882740 10 client.26902 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T10:02:04.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.968+0000 7f8228882740 10 client.26902 resolve_mds: validated gid 24563 aka daemon mds.b 2024-09-10T10:02:04.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.968+0000 7f8228882740 20 client.26902 populate_metadata read hostname 'smithi077' 2024-09-10T10:02:04.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.968+0000 7f8228882740 1 --2- 172.21.15.77:0/236836692 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55a1e04baae0 0x55a1e04bcf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:04.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.968+0000 7f8228882740 4 client.26902 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.1, "await_for": 45.0}] 2024-09-10T10:02:04.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.968+0000 7f8228882740 1 -- 172.21.15.77:0/236836692 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.1, "await_for": 45.0}) -- 0x55a1e039ec60 con 0x55a1e04baae0 2024-09-10T10:02:04.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.968+0000 7f821ea00640 1 --2- 172.21.15.77:0/236836692 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55a1e04baae0 0x55a1e04bcf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:04.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.969+0000 7f8208c00640 10 client.26902 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T10:02:04.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:04.969+0000 7f821ea00640 1 --2- 172.21.15.77:0/236836692 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55a1e04baae0 0x55a1e04bcf30 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1cf3dc55283dd091 server_cookie=9fd43cbb4e6ff030 in_seq=0 out_seq=0 2024-09-10T10:02:05.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:02:04 smithi077 ceph-mon[30319]: pgmap v598: 129 pgs: 129 active+clean; 142 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 464 KiB/s wr, 152 op/s 2024-09-10T10:02:05.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:02:04 smithi134 ceph-mon[35674]: pgmap v598: 129 pgs: 129 active+clean; 142 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 464 KiB/s wr, 152 op/s 2024-09-10T10:02:05.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:02:04 smithi120 ceph-mon[34770]: pgmap v598: 129 pgs: 129 active+clean; 142 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 464 KiB/s wr, 152 op/s 2024-09-10T10:02:05.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:05.556+0000 7f8208c00640 1 -- 172.21.15.77:0/236836692 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55a1e039ec60 con 0x55a1e04baae0 2024-09-10T10:02:05.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:05.556+0000 7f8208c00640 10 client.26902 handle_command_reply: tid=0 2024-09-10T10:02:05.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:05.556+0000 7f8228882740 1 client.26902 shutdown 2024-09-10T10:02:05.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:05.557+0000 7f8228882740 1 -- 172.21.15.77:0/236836692 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1e042bcd0 msgr2=0x55a1e043b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:05.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:05.557+0000 7f8228882740 1 --2- 172.21.15.77:0/236836692 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1e042bcd0 0x55a1e043b260 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f820c0bb3a0 tx=0x7f820c0bad70 comp rx=0 tx=0).stop 2024-09-10T10:02:05.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:05.557+0000 7f8228882740 1 -- 172.21.15.77:0/236836692 shutdown_connections 2024-09-10T10:02:05.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:05.557+0000 7f8228882740 1 --2- 172.21.15.77:0/236836692 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55a1e04baae0 0x55a1e04bcf30 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:05.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:05.557+0000 7f8228882740 1 --2- 172.21.15.77:0/236836692 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a1e0436590 0x55a1e043b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:05.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:05.557+0000 7f8228882740 1 --2- 172.21.15.77:0/236836692 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1e042bcd0 0x55a1e043b260 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:05.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:05.557+0000 7f8208c00640 0 client.26902 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T10:02:05.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:05.557+0000 7f8228882740 1 --2- 172.21.15.77:0/236836692 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1e042b3b0 0x55a1e043e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:05.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:05.557+0000 7f8228882740 1 -- 172.21.15.77:0/236836692 >> 172.21.15.77:0/236836692 conn(0x55a1e0427940 msgr2=0x55a1e042ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:05.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:05.557+0000 7f8228882740 1 -- 172.21.15.77:0/236836692 shutdown_connections 2024-09-10T10:02:05.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:05.557+0000 7f8228882740 1 -- 172.21.15.77:0/236836692 wait complete. 2024-09-10T10:02:05.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:05.557+0000 7f8228882740 20 client.26902 trim_cache size 0 max 16384 2024-09-10T10:02:05.659 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-10T10:02:05.659 INFO:teuthology.orchestra.run.smithi077.stdout: "epoch": 35, 2024-09-10T10:02:05.659 INFO:teuthology.orchestra.run.smithi077.stdout: "leader": 24563, 2024-09-10T10:02:05.659 INFO:teuthology.orchestra.run.smithi077.stdout: "set_version": 22, 2024-09-10T10:02:05.659 INFO:teuthology.orchestra.run.smithi077.stdout: "sets": { 2024-09-10T10:02:05.659 INFO:teuthology.orchestra.run.smithi077.stdout: "1d58ccc9": { 2024-09-10T10:02:05.659 INFO:teuthology.orchestra.run.smithi077.stdout: "version": 22, 2024-09-10T10:02:05.659 INFO:teuthology.orchestra.run.smithi077.stdout: "age_ref": 0.0, 2024-09-10T10:02:05.659 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-10T10:02:05.659 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "QUIESCED", 2024-09-10T10:02:05.659 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-10T10:02:05.660 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:02:05.660 INFO:teuthology.orchestra.run.smithi077.stdout: "timeout": 90.0, 2024-09-10T10:02:05.660 INFO:teuthology.orchestra.run.smithi077.stdout: "expiration": 140.1, 2024-09-10T10:02:05.660 INFO:teuthology.orchestra.run.smithi077.stdout: "members": { 2024-09-10T10:02:05.660 INFO:teuthology.orchestra.run.smithi077.stdout: "file:/": { 2024-09-10T10:02:05.660 INFO:teuthology.orchestra.run.smithi077.stdout: "excluded": false, 2024-09-10T10:02:05.660 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-10T10:02:05.660 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "QUIESCED", 2024-09-10T10:02:05.660 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-10T10:02:05.660 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:02:05.660 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:02:05.660 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:02:05.660 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:02:05.660 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:02:05.661 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-10T10:02:05.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:05.558+0000 7f8225e00640 1 -- 172.21.15.77:0/1179952177 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f82040a2b80 msgr2=0x7f82040a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:05.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:05.558+0000 7f8225e00640 1 --2- 172.21.15.77:0/1179952177 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f82040a2b80 0x7f82040a5040 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x55a1e04287a0 tx=0x7f8210009260 comp rx=0 tx=0).stop 2024-09-10T10:02:05.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:05.558+0000 7f8225e00640 1 -- 172.21.15.77:0/1179952177 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f82201049a0 msgr2=0x7f822019abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:05.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:05.558+0000 7f8225e00640 1 --2- 172.21.15.77:0/1179952177 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f82201049a0 0x7f822019abe0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f820c035040 tx=0x7f820c033d90 comp rx=0 tx=0).stop 2024-09-10T10:02:05.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:05.558+0000 7f8225e00640 1 -- 172.21.15.77:0/1179952177 shutdown_connections 2024-09-10T10:02:05.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:05.558+0000 7f8225e00640 1 --2- 172.21.15.77:0/1179952177 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f82040a2b80 0x7f82040a5040 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:05.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:05.558+0000 7f8225e00640 1 --2- 172.21.15.77:0/1179952177 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8220109120 0x7f822019ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:05.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:05.558+0000 7f8225e00640 1 --2- 172.21.15.77:0/1179952177 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f82201049a0 0x7f822019abe0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:05.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:05.558+0000 7f8225e00640 1 --2- 172.21.15.77:0/1179952177 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8220103ff0 0x7f822019a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:05.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:05.558+0000 7f8225e00640 1 -- 172.21.15.77:0/1179952177 >> 172.21.15.77:0/1179952177 conn(0x7f82200ffa90 msgr2=0x7f8220100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:05.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:05.558+0000 7f8225e00640 1 -- 172.21.15.77:0/1179952177 shutdown_connections 2024-09-10T10:02:05.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:05.558+0000 7f8225e00640 1 -- 172.21.15.77:0/1179952177 wait complete. 2024-09-10T10:02:06.515 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '1d58ccc9', quiesce took 7.7 seconds. Will release after: 12.400000000000002 2024-09-10T10:02:07.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:02:06 smithi134 ceph-mon[35674]: pgmap v599: 129 pgs: 129 active+clean; 138 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 552 KiB/s wr, 198 op/s 2024-09-10T10:02:07.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:02:06 smithi120 ceph-mon[34770]: pgmap v599: 129 pgs: 129 active+clean; 138 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 552 KiB/s wr, 198 op/s 2024-09-10T10:02:07.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:02:06 smithi077 ceph-mon[30319]: pgmap v599: 129 pgs: 129 active+clean; 138 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 552 KiB/s wr, 198 op/s 2024-09-10T10:02:09.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:02:08 smithi134 ceph-mon[35674]: pgmap v600: 129 pgs: 129 active+clean; 137 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 636 KiB/s wr, 229 op/s 2024-09-10T10:02:09.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:02:08 smithi120 ceph-mon[34770]: pgmap v600: 129 pgs: 129 active+clean; 137 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 636 KiB/s wr, 229 op/s 2024-09-10T10:02:09.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:02:08 smithi077 ceph-mon[30319]: pgmap v600: 129 pgs: 129 active+clean; 137 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 636 KiB/s wr, 229 op/s 2024-09-10T10:02:11.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:02:10 smithi134 ceph-mon[35674]: pgmap v601: 129 pgs: 129 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 771 KiB/s wr, 291 op/s 2024-09-10T10:02:11.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:02:10 smithi134 ceph-mon[35674]: scrub summary: idle 2024-09-10T10:02:11.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:02:10 smithi134 ceph-mon[35674]: scrub summary: idle 2024-09-10T10:02:11.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:02:10 smithi120 ceph-mon[34770]: pgmap v601: 129 pgs: 129 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 771 KiB/s wr, 291 op/s 2024-09-10T10:02:11.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:02:10 smithi120 ceph-mon[34770]: scrub summary: idle 2024-09-10T10:02:11.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:02:10 smithi120 ceph-mon[34770]: scrub summary: idle 2024-09-10T10:02:11.440 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:02:10 smithi077 ceph-mon[30319]: pgmap v601: 129 pgs: 129 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 771 KiB/s wr, 291 op/s 2024-09-10T10:02:11.440 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:02:10 smithi077 ceph-mon[30319]: scrub summary: idle 2024-09-10T10:02:11.441 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:02:10 smithi077 ceph-mon[30319]: scrub summary: idle 2024-09-10T10:02:12.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:02:11 smithi134 ceph-mon[35674]: scrub summary: idle 2024-09-10T10:02:12.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:02:11 smithi134 ceph-mon[35674]: scrub summary: idle 2024-09-10T10:02:12.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:02:11 smithi134 ceph-mon[35674]: scrub summary: idle 2024-09-10T10:02:12.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:02:11 smithi120 ceph-mon[34770]: scrub summary: idle 2024-09-10T10:02:12.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:02:11 smithi120 ceph-mon[34770]: scrub summary: idle 2024-09-10T10:02:12.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:02:11 smithi120 ceph-mon[34770]: scrub summary: idle 2024-09-10T10:02:12.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:02:11 smithi077 ceph-mon[30319]: scrub summary: idle 2024-09-10T10:02:12.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:02:11 smithi077 ceph-mon[30319]: scrub summary: idle 2024-09-10T10:02:12.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:02:11 smithi077 ceph-mon[30319]: scrub summary: idle 2024-09-10T10:02:13.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:02:12 smithi134 ceph-mon[35674]: pgmap v602: 129 pgs: 129 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 710 KiB/s wr, 264 op/s 2024-09-10T10:02:13.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:02:12 smithi120 ceph-mon[34770]: pgmap v602: 129 pgs: 129 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 710 KiB/s wr, 264 op/s 2024-09-10T10:02:13.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:02:12 smithi077 ceph-mon[30319]: pgmap v602: 129 pgs: 129 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 710 KiB/s wr, 264 op/s 2024-09-10T10:02:15.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:02:14 smithi134 ceph-mon[35674]: pgmap v603: 129 pgs: 129 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 651 KiB/s wr, 245 op/s 2024-09-10T10:02:15.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:02:14 smithi120 ceph-mon[34770]: pgmap v603: 129 pgs: 129 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 651 KiB/s wr, 245 op/s 2024-09-10T10:02:15.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:02:14 smithi077 ceph-mon[30319]: pgmap v603: 129 pgs: 129 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 651 KiB/s wr, 245 op/s 2024-09-10T10:02:17.226 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:02:16 smithi120 ceph-mon[34770]: pgmap v604: 129 pgs: 129 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 344 KiB/s wr, 148 op/s 2024-09-10T10:02:17.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:02:16 smithi134 ceph-mon[35674]: pgmap v604: 129 pgs: 129 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 344 KiB/s wr, 148 op/s 2024-09-10T10:02:17.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:02:16 smithi077 ceph-mon[30319]: pgmap v604: 129 pgs: 129 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 344 KiB/s wr, 148 op/s 2024-09-10T10:02:18.916 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:02:19.319 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:02:19.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:02:18 smithi134 ceph-mon[35674]: pgmap v605: 129 pgs: 129 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 228 KiB/s wr, 98 op/s 2024-09-10T10:02:19.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:02:18 smithi120 ceph-mon[34770]: pgmap v605: 129 pgs: 129 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 228 KiB/s wr, 98 op/s 2024-09-10T10:02:19.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:02:18 smithi077 ceph-mon[30319]: pgmap v605: 129 pgs: 129 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 228 KiB/s wr, 98 op/s 2024-09-10T10:02:19.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:19.887+0000 7fea7a800640 1 -- 172.21.15.77:0/2858574602 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fea74102b90 msgr2=0x7fea74102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:19.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:19.887+0000 7fea7a800640 1 --2- 172.21.15.77:0/2858574602 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fea74102b90 0x7fea74102f70 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fea68009900 tx=0x7fea6802a480 comp rx=0 tx=0).stop 2024-09-10T10:02:19.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:19.887+0000 7fea7a800640 1 -- 172.21.15.77:0/2858574602 shutdown_connections 2024-09-10T10:02:19.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:19.887+0000 7fea7a800640 1 --2- 172.21.15.77:0/2858574602 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea7410bfc0 0x7fea7410e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:19.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:19.887+0000 7fea7a800640 1 --2- 172.21.15.77:0/2858574602 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fea741034b0 0x7fea7410b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:19.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:19.887+0000 7fea7a800640 1 --2- 172.21.15.77:0/2858574602 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fea74102b90 0x7fea74102f70 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:19.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:19.887+0000 7fea7a800640 1 -- 172.21.15.77:0/2858574602 >> 172.21.15.77:0/2858574602 conn(0x7fea740fc610 msgr2=0x7fea740fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:19.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:19.887+0000 7fea7a800640 1 -- 172.21.15.77:0/2858574602 shutdown_connections 2024-09-10T10:02:19.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:19.887+0000 7fea7a800640 1 -- 172.21.15.77:0/2858574602 wait complete. 2024-09-10T10:02:19.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:19.888+0000 7fea7a800640 1 Processor -- start 2024-09-10T10:02:19.892 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:19.888+0000 7fea7a800640 1 -- start start 2024-09-10T10:02:19.892 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:19.889+0000 7fea7a800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fea74102b90 0x7fea741982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:19.892 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:19.889+0000 7fea7a800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fea741034b0 0x7fea741987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:19.892 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:19.889+0000 7fea7a800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea7410bfc0 0x7fea7419cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:19.892 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:19.889+0000 7fea7a800640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fea74111dc0 con 0x7fea74102b90 2024-09-10T10:02:19.892 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:19.889+0000 7fea7a800640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fea74111c40 con 0x7fea741034b0 2024-09-10T10:02:19.892 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:19.889+0000 7fea7a800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fea74111f40 con 0x7fea7410bfc0 2024-09-10T10:02:19.892 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:19.889+0000 7fea79400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fea74102b90 0x7fea741982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:19.892 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:19.889+0000 7fea79e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea7410bfc0 0x7fea7419cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:19.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:19.889+0000 7fea79400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fea74102b90 0x7fea741982a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57250/0 (socket says 172.21.15.77:57250) 2024-09-10T10:02:19.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:19.889+0000 7fea79e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea7410bfc0 0x7fea7419cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:48618/0 (socket says 172.21.15.77:48618) 2024-09-10T10:02:19.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:19.889+0000 7fea79400640 1 -- 172.21.15.77:0/1302868992 learned_addr learned my addr 172.21.15.77:0/1302868992 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:02:19.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:19.889+0000 7fea78a00640 1 --2- 172.21.15.77:0/1302868992 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fea741034b0 0x7fea741987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:19.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:19.890+0000 7fea79e00640 1 -- 172.21.15.77:0/1302868992 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fea741034b0 msgr2=0x7fea741987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:19.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:19.890+0000 7fea79e00640 1 --2- 172.21.15.77:0/1302868992 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fea741034b0 0x7fea741987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:19.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:19.890+0000 7fea79e00640 1 -- 172.21.15.77:0/1302868992 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fea74102b90 msgr2=0x7fea741982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:19.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:19.890+0000 7fea79e00640 1 --2- 172.21.15.77:0/1302868992 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fea74102b90 0x7fea741982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:19.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:19.890+0000 7fea79e00640 1 -- 172.21.15.77:0/1302868992 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea7419d250 con 0x7fea7410bfc0 2024-09-10T10:02:19.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:19.890+0000 7fea79e00640 1 --2- 172.21.15.77:0/1302868992 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea7410bfc0 0x7fea7419cb70 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fea6c009fd0 tx=0x7fea6c00ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:19.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:19.890+0000 7fea72000640 1 -- 172.21.15.77:0/1302868992 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea6c00ce20 con 0x7fea7410bfc0 2024-09-10T10:02:19.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:19.891+0000 7fea7a800640 1 -- 172.21.15.77:0/1302868992 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea7419d540 con 0x7fea7410bfc0 2024-09-10T10:02:19.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:19.891+0000 7fea72000640 1 -- 172.21.15.77:0/1302868992 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea6c005020 con 0x7fea7410bfc0 2024-09-10T10:02:19.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:19.891+0000 7fea72000640 1 -- 172.21.15.77:0/1302868992 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea6c010720 con 0x7fea7410bfc0 2024-09-10T10:02:19.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:19.891+0000 7fea7a800640 1 -- 172.21.15.77:0/1302868992 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea741a4e20 con 0x7fea7410bfc0 2024-09-10T10:02:19.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:19.892+0000 7fea7a800640 1 -- 172.21.15.77:0/1302868992 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea74102f70 con 0x7fea7410bfc0 2024-09-10T10:02:19.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:19.895+0000 7fea72000640 1 -- 172.21.15.77:0/1302868992 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fea6c005540 con 0x7fea7410bfc0 2024-09-10T10:02:19.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:19.896+0000 7fea72000640 1 --2- 172.21.15.77:0/1302868992 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fea5c0a2c00 0x7fea5c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:19.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:19.896+0000 7fea72000640 1 -- 172.21.15.77:0/1302868992 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fea6c0cf980 con 0x7fea7410bfc0 2024-09-10T10:02:19.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:19.897+0000 7fea79400640 1 --2- 172.21.15.77:0/1302868992 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fea5c0a2c00 0x7fea5c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:19.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:19.897+0000 7fea79400640 1 --2- 172.21.15.77:0/1302868992 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fea5c0a2c00 0x7fea5c0a50c0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fea680096c0 tx=0x7fea68005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:19.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:19.901+0000 7fea72000640 1 -- 172.21.15.77:0/1302868992 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fea6c09c490 con 0x7fea7410bfc0 2024-09-10T10:02:20.122 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:20.120+0000 7fea7a800640 1 -- 172.21.15.77:0/1302868992 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fea7402d7c0 con 0x7fea7410bfc0 2024-09-10T10:02:20.123 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:20.121+0000 7fea72000640 1 -- 172.21.15.77:0/1302868992 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11989 (secure 0 0 0) 0x7fea6c09bd90 con 0x7fea7410bfc0 2024-09-10T10:02:20.123 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T10:02:20.124 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":35,"btime":"2024-09-10T10:01:05:922537+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3159289675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3159289675},{"type":"v1","addr":"172.21.15.77:6835","nonce":3159289675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1951962341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1951962341},{"type":"v1","addr":"172.21.15.77:6837","nonce":1951962341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":35,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:52:23.368252+0000","modified":"2024-09-10T10:01:05.706646+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24563,"mds_1":24524,"mds_2":24520,"mds_3":24502,"mds_4":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14487":{"gid":14487,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/4291121888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":4291121888},{"type":"v1","addr":"172.21.15.77:6839","nonce":4291121888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14499":{"gid":14499,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6841/445284194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":445284194},{"type":"v1","addr":"172.21.15.77:6841","nonce":445284194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1005054423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1005054423},{"type":"v1","addr":"172.21.15.134:6833","nonce":1005054423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6833/3716307744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":3716307744},{"type":"v1","addr":"172.21.15.120:6833","nonce":3716307744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/1319892544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1319892544},{"type":"v1","addr":"172.21.15.120:6835","nonce":1319892544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.134:6835/3293933699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3293933699},{"type":"v1","addr":"172.21.15.134:6835","nonce":3293933699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.134:6837/2461497091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2461497091},{"type":"v1","addr":"172.21.15.134:6837","nonce":2461497091}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":20,"addr":"172.21.15.134:6839/1965117680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1965117680},{"type":"v1","addr":"172.21.15.134:6839","nonce":1965117680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":20,"addr":"172.21.15.120:6837/3033280709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3033280709},{"type":"v1","addr":"172.21.15.120:6837","nonce":3033280709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.120:6839/1777710157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1777710157},{"type":"v1","addr":"172.21.15.120:6839","nonce":1777710157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24563,"qdb_cluster":[24496,24502,24520,24524,24563]},"id":1}]} 2024-09-10T10:02:20.124 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 35 2024-09-10T10:02:20.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:20.123+0000 7fea7a800640 1 -- 172.21.15.77:0/1302868992 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fea5c0a2c00 msgr2=0x7fea5c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:20.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:20.124+0000 7fea7a800640 1 --2- 172.21.15.77:0/1302868992 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fea5c0a2c00 0x7fea5c0a50c0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fea680096c0 tx=0x7fea68005a20 comp rx=0 tx=0).stop 2024-09-10T10:02:20.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:20.124+0000 7fea7a800640 1 -- 172.21.15.77:0/1302868992 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea7410bfc0 msgr2=0x7fea7419cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:20.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:20.124+0000 7fea7a800640 1 --2- 172.21.15.77:0/1302868992 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea7410bfc0 0x7fea7419cb70 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fea6c009fd0 tx=0x7fea6c00ef10 comp rx=0 tx=0).stop 2024-09-10T10:02:20.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:20.124+0000 7fea7a800640 1 -- 172.21.15.77:0/1302868992 shutdown_connections 2024-09-10T10:02:20.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:20.124+0000 7fea7a800640 1 --2- 172.21.15.77:0/1302868992 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fea5c0a2c00 0x7fea5c0a50c0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:20.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:20.124+0000 7fea7a800640 1 --2- 172.21.15.77:0/1302868992 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea7410bfc0 0x7fea7419cb70 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:20.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:20.124+0000 7fea7a800640 1 --2- 172.21.15.77:0/1302868992 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fea741034b0 0x7fea741987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:20.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:20.124+0000 7fea7a800640 1 --2- 172.21.15.77:0/1302868992 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fea74102b90 0x7fea741982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:20.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:20.124+0000 7fea7a800640 1 -- 172.21.15.77:0/1302868992 >> 172.21.15.77:0/1302868992 conn(0x7fea740fc610 msgr2=0x7fea740fd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:20.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:20.124+0000 7fea7a800640 1 -- 172.21.15.77:0/1302868992 shutdown_connections 2024-09-10T10:02:20.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:20.124+0000 7fea7a800640 1 -- 172.21.15.77:0/1302868992 wait complete. 2024-09-10T10:02:20.953 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24563 quiesce db --set-id 1d58ccc9 --release --await' 2024-09-10T10:02:20.953 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24563 quiesce db --set-id 1d58ccc9 --release --await 2024-09-10T10:02:21.174 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:02:20 smithi077 ceph-mon[30319]: pgmap v606: 129 pgs: 129 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 136 KiB/s wr, 70 op/s 2024-09-10T10:02:21.174 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:02:20 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/1302868992' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:02:21.333 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:02:21.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:02:20 smithi134 ceph-mon[35674]: pgmap v606: 129 pgs: 129 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 136 KiB/s wr, 70 op/s 2024-09-10T10:02:21.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:02:20 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/1302868992' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:02:21.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:02:20 smithi120 ceph-mon[34770]: pgmap v606: 129 pgs: 129 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 136 KiB/s wr, 70 op/s 2024-09-10T10:02:21.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:02:20 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/1302868992' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:02:21.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.909+0000 7eff09200640 1 -- 172.21.15.77:0/653505118 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7eff04102360 msgr2=0x7eff0410e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:21.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.909+0000 7eff09200640 1 --2- 172.21.15.77:0/653505118 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7eff04102360 0x7eff0410e890 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7efefc009cd0 tx=0x7efefc02a8f0 comp rx=0 tx=0).stop 2024-09-10T10:02:21.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.909+0000 7eff09200640 1 -- 172.21.15.77:0/653505118 shutdown_connections 2024-09-10T10:02:21.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.909+0000 7eff09200640 1 --2- 172.21.15.77:0/653505118 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7eff04102360 0x7eff0410e890 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:21.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.909+0000 7eff09200640 1 --2- 172.21.15.77:0/653505118 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff041019c0 0x7eff04101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:21.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.909+0000 7eff09200640 1 --2- 172.21.15.77:0/653505118 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7eff041079c0 0x7eff04107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:21.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.909+0000 7eff09200640 1 -- 172.21.15.77:0/653505118 >> 172.21.15.77:0/653505118 conn(0x7eff040fd740 msgr2=0x7eff040ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:21.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.909+0000 7eff09200640 1 -- 172.21.15.77:0/653505118 shutdown_connections 2024-09-10T10:02:21.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.909+0000 7eff09200640 1 -- 172.21.15.77:0/653505118 wait complete. 2024-09-10T10:02:21.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.910+0000 7eff09200640 1 Processor -- start 2024-09-10T10:02:21.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.910+0000 7eff09200640 1 -- start start 2024-09-10T10:02:21.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.911+0000 7eff09200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7eff041019c0 0x7eff04198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:21.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.911+0000 7eff09200640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7eff04102360 0x7eff04198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:21.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.911+0000 7eff09200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff041079c0 0x7eff0419cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:21.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.911+0000 7eff09200640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7eff04111dc0 con 0x7eff041019c0 2024-09-10T10:02:21.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.911+0000 7eff09200640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7eff04111c40 con 0x7eff04102360 2024-09-10T10:02:21.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.911+0000 7eff09200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7eff04111f40 con 0x7eff041079c0 2024-09-10T10:02:21.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.911+0000 7eff03400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7eff041019c0 0x7eff04198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:21.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.911+0000 7eff03400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7eff041019c0 0x7eff04198240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57264/0 (socket says 172.21.15.77:57264) 2024-09-10T10:02:21.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.911+0000 7eff03400640 1 -- 172.21.15.77:0/1474118660 learned_addr learned my addr 172.21.15.77:0/1474118660 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:02:21.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.911+0000 7eff03e00640 1 --2- 172.21.15.77:0/1474118660 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff041079c0 0x7eff0419cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:21.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.911+0000 7eff02a00640 1 --2- 172.21.15.77:0/1474118660 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7eff04102360 0x7eff04198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:21.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.911+0000 7eff03400640 1 -- 172.21.15.77:0/1474118660 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff041079c0 msgr2=0x7eff0419cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:21.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.911+0000 7eff03400640 1 --2- 172.21.15.77:0/1474118660 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff041079c0 0x7eff0419cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:21.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.911+0000 7eff03400640 1 -- 172.21.15.77:0/1474118660 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7eff04102360 msgr2=0x7eff04198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:21.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.911+0000 7eff03400640 1 --2- 172.21.15.77:0/1474118660 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7eff04102360 0x7eff04198780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:21.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.911+0000 7eff03400640 1 -- 172.21.15.77:0/1474118660 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff0419d1f0 con 0x7eff041019c0 2024-09-10T10:02:21.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.912+0000 7eff03400640 1 --2- 172.21.15.77:0/1474118660 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7eff041019c0 0x7eff04198240 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7efef400e9f0 tx=0x7efef400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:21.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.912+0000 7efefbe00640 1 -- 172.21.15.77:0/1474118660 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef400cd80 con 0x7eff041019c0 2024-09-10T10:02:21.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.912+0000 7eff09200640 1 -- 172.21.15.77:0/1474118660 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff0419d4e0 con 0x7eff041019c0 2024-09-10T10:02:21.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.912+0000 7efefbe00640 1 -- 172.21.15.77:0/1474118660 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efef4010910 con 0x7eff041019c0 2024-09-10T10:02:21.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.912+0000 7eff09200640 1 -- 172.21.15.77:0/1474118660 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff041a4dc0 con 0x7eff041019c0 2024-09-10T10:02:21.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.912+0000 7efefbe00640 1 -- 172.21.15.77:0/1474118660 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef4018910 con 0x7eff041019c0 2024-09-10T10:02:21.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.915+0000 7eff0d207740 1 Processor -- start 2024-09-10T10:02:21.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.915+0000 7efefbe00640 1 -- 172.21.15.77:0/1474118660 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7efef4010ac0 con 0x7eff041019c0 2024-09-10T10:02:21.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.915+0000 7eff0d207740 1 -- start start 2024-09-10T10:02:21.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.915+0000 7eff0d207740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4f8732930 0x55d4f8732d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:21.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.915+0000 7eff0d207740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d4f8733250 0x55d4f873d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:21.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.915+0000 7eff0d207740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4f873db10 0x55d4f873ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:21.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.915+0000 7eff0d207740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d4f8694650 con 0x55d4f8733250 2024-09-10T10:02:21.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.915+0000 7eff0d207740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55d4f863bf10 con 0x55d4f873db10 2024-09-10T10:02:21.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.915+0000 7eff0d207740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d4f8690610 con 0x55d4f8732930 2024-09-10T10:02:21.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.916+0000 7eff02a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4f873db10 0x55d4f873ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:21.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.916+0000 7eff03400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d4f8733250 0x55d4f873d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:21.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.916+0000 7eff02a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4f873db10 0x55d4f873ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:41142/0 (socket says 172.21.15.77:41142) 2024-09-10T10:02:21.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.916+0000 7eff02a00640 1 -- 172.21.15.77:0/3937865735 learned_addr learned my addr 172.21.15.77:0/3937865735 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:02:21.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.916+0000 7eff03e00640 1 --2- 172.21.15.77:0/3937865735 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4f8732930 0x55d4f8732d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:21.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.916+0000 7eff02a00640 1 -- 172.21.15.77:0/3937865735 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4f8732930 msgr2=0x55d4f8732d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:21.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.916+0000 7eff02a00640 1 --2- 172.21.15.77:0/3937865735 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4f8732930 0x55d4f8732d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:21.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.916+0000 7eff02a00640 1 -- 172.21.15.77:0/3937865735 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d4f8733250 msgr2=0x55d4f873d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:21.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.916+0000 7eff02a00640 1 --2- 172.21.15.77:0/3937865735 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d4f8733250 0x55d4f873d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:21.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.916+0000 7eff02a00640 1 -- 172.21.15.77:0/3937865735 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4f83a3580 con 0x55d4f873db10 2024-09-10T10:02:21.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.916+0000 7eff03400640 1 --2- 172.21.15.77:0/3937865735 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d4f8733250 0x55d4f873d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:02:21.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.916+0000 7eff03e00640 1 --2- 172.21.15.77:0/3937865735 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4f8732930 0x55d4f8732d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:02:21.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.917+0000 7eff02a00640 1 --2- 172.21.15.77:0/3937865735 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4f873db10 0x55d4f873ff00 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7efef000dc80 tx=0x7efef000f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c74b395c9a1c4727 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:21.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.917+0000 7efefbe00640 1 --2- 172.21.15.77:0/1474118660 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7efee80a2b80 0x7efee80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:21.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.917+0000 7efef8c00640 1 -- 172.21.15.77:0/3937865735 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef0015070 con 0x55d4f873db10 2024-09-10T10:02:21.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.917+0000 7efefbe00640 1 -- 172.21.15.77:0/1474118660 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7efef40cf820 con 0x7eff041019c0 2024-09-10T10:02:21.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.917+0000 7efef8c00640 1 -- 172.21.15.77:0/3937865735 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efef001a940 con 0x55d4f873db10 2024-09-10T10:02:21.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.917+0000 7efef8c00640 1 -- 172.21.15.77:0/3937865735 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef0011a70 con 0x55d4f873db10 2024-09-10T10:02:21.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.917+0000 7eff03e00640 1 --2- 172.21.15.77:0/1474118660 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7efee80a2b80 0x7efee80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:21.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.918+0000 7eff0d207740 1 -- 172.21.15.77:0/3937865735 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4f873db10 msgr2=0x55d4f873ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:21.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.918+0000 7eff0d207740 1 --2- 172.21.15.77:0/3937865735 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4f873db10 0x55d4f873ff00 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7efef000dc80 tx=0x7efef000f730 comp rx=0 tx=0).stop 2024-09-10T10:02:21.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.918+0000 7eff03e00640 1 --2- 172.21.15.77:0/1474118660 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7efee80a2b80 0x7efee80a5040 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7efefc004af0 tx=0x7efefc004a80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:21.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.918+0000 7eff0d207740 1 -- 172.21.15.77:0/3937865735 shutdown_connections 2024-09-10T10:02:21.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.918+0000 7eff0d207740 1 --2- 172.21.15.77:0/3937865735 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4f873db10 0x55d4f873ff00 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:21.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.918+0000 7eff0d207740 1 --2- 172.21.15.77:0/3937865735 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d4f8733250 0x55d4f873d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:21.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.918+0000 7eff0d207740 1 --2- 172.21.15.77:0/3937865735 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4f8732930 0x55d4f8732d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:21.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.918+0000 7eff0d207740 1 -- 172.21.15.77:0/3937865735 >> 172.21.15.77:0/3937865735 conn(0x55d4f872eec0 msgr2=0x55d4f872f2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:21.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.918+0000 7eff0d207740 1 -- 172.21.15.77:0/3937865735 shutdown_connections 2024-09-10T10:02:21.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.918+0000 7eff0d207740 1 -- 172.21.15.77:0/3937865735 wait complete. 2024-09-10T10:02:21.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.918+0000 7eff0d207740 1 -- start start 2024-09-10T10:02:21.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.919+0000 7eff0d207740 1 Processor -- start 2024-09-10T10:02:21.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.919+0000 7eff0d207740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d4f8732930 0x55d4f873a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:21.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.919+0000 7eff0d207740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4f8733250 0x55d4f873aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:21.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.919+0000 7eff0d207740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4f873db10 0x55d4f873b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:21.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.919+0000 7eff0d207740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d4f87406e0 con 0x55d4f8732930 2024-09-10T10:02:21.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.919+0000 7eff0d207740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55d4f864f350 con 0x55d4f873db10 2024-09-10T10:02:21.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.919+0000 7eff0d207740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d4f8740860 con 0x55d4f8733250 2024-09-10T10:02:21.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.919+0000 7eff0d207740 10 client.0 _ll_register_callbacks cb 0x55d4f8672290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:02:21.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.919+0000 7eff03400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d4f8732930 0x55d4f873a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:21.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.919+0000 7eff03400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d4f8732930 0x55d4f873a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57274/0 (socket says 172.21.15.77:57274) 2024-09-10T10:02:21.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.919+0000 7eff03400640 1 -- 172.21.15.77:0/1165620623 learned_addr learned my addr 172.21.15.77:0/1165620623 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:02:21.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.920+0000 7eff02a00640 1 --2- 172.21.15.77:0/1165620623 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4f8733250 0x55d4f873aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:21.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.920+0000 7eff03e00640 1 --2- 172.21.15.77:0/1165620623 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4f873db10 0x55d4f873b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:21.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.920+0000 7eff03400640 1 -- 172.21.15.77:0/1165620623 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4f8733250 msgr2=0x55d4f873aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:21.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.920+0000 7eff03400640 1 --2- 172.21.15.77:0/1165620623 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4f8733250 0x55d4f873aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:21.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.920+0000 7eff03400640 1 -- 172.21.15.77:0/1165620623 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4f873db10 msgr2=0x55d4f873b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:21.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.920+0000 7eff03400640 1 --2- 172.21.15.77:0/1165620623 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4f873db10 0x55d4f873b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:21.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.920+0000 7eff03400640 1 -- 172.21.15.77:0/1165620623 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4f846d960 con 0x55d4f8732930 2024-09-10T10:02:21.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.920+0000 7eff02a00640 1 --2- 172.21.15.77:0/1165620623 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4f8733250 0x55d4f873aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:02:21.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.920+0000 7eff03400640 1 --2- 172.21.15.77:0/1165620623 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d4f8732930 0x55d4f873a9b0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7efef4098090 tx=0x7efef409baf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74faddeb301568d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:21.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.920+0000 7efeed600640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-10T10:02:21.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.920+0000 7efeed600640 1 -- 172.21.15.77:0/1165620623 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef409bc30 con 0x55d4f8732930 2024-09-10T10:02:21.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.920+0000 7efeed600640 1 -- 172.21.15.77:0/1165620623 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7efef409b080 con 0x55d4f8732930 2024-09-10T10:02:21.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.920+0000 7eff0d207740 1 -- 172.21.15.77:0/1165620623 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efef001a940 con 0x55d4f8732930 2024-09-10T10:02:21.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.921+0000 7efeed600640 1 -- 172.21.15.77:0/1165620623 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7efef409ffe0 con 0x55d4f8732930 2024-09-10T10:02:21.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.921+0000 7eff0d207740 10 client.16836 fetch_fsmap learned FSMap version 35 2024-09-10T10:02:21.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.921+0000 7eff0d207740 1 -- 172.21.15.77:0/1165620623 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55d4f874c310 con 0x55d4f8732930 2024-09-10T10:02:21.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.921+0000 7efeed600640 1 -- 172.21.15.77:0/1165620623 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efef001a940 con 0x55d4f8732930 2024-09-10T10:02:21.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.921+0000 7efeed600640 1 -- 172.21.15.77:0/1165620623 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7efef40d1ce0 con 0x55d4f8732930 2024-09-10T10:02:21.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.921+0000 7eff0d207740 10 client.16836 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T10:02:21.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.921+0000 7eff0d207740 10 client.16836 resolve_mds: validated gid 24563 aka daemon mds.b 2024-09-10T10:02:21.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.921+0000 7eff0d207740 20 client.16836 populate_metadata read hostname 'smithi077' 2024-09-10T10:02:21.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.921+0000 7eff0d207740 1 --2- 172.21.15.77:0/1165620623 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55d4f874d820 0x55d4f874fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:21.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.921+0000 7eff0d207740 4 client.16836 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:02:21.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.921+0000 7eff0d207740 1 -- 172.21.15.77:0/1165620623 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d4f86a1bc0 con 0x55d4f874d820 2024-09-10T10:02:21.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.922+0000 7eff02a00640 1 --2- 172.21.15.77:0/1165620623 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55d4f874d820 0x55d4f874fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:21.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.922+0000 7eff02a00640 1 --2- 172.21.15.77:0/1165620623 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55d4f874d820 0x55d4f874fc70 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=649a0b57e856bd3a server_cookie=590558b777f2ed74 in_seq=0 out_seq=0 2024-09-10T10:02:21.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.922+0000 7efeed600640 10 client.16836 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T10:02:21.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.923+0000 7efeed600640 1 -- 172.21.15.77:0/1165620623 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d4f86a1bc0 con 0x55d4f874d820 2024-09-10T10:02:21.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.923+0000 7efeed600640 10 client.16836 handle_command_reply: tid=0 2024-09-10T10:02:21.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.923+0000 7eff0d207740 1 client.16836 shutdown 2024-09-10T10:02:21.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.923+0000 7eff0d207740 1 -- 172.21.15.77:0/1165620623 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d4f8732930 msgr2=0x55d4f873a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:21.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.923+0000 7eff0d207740 1 --2- 172.21.15.77:0/1165620623 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d4f8732930 0x55d4f873a9b0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7efef4098090 tx=0x7efef409baf0 comp rx=0 tx=0).stop 2024-09-10T10:02:21.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.923+0000 7eff0d207740 1 -- 172.21.15.77:0/1165620623 shutdown_connections 2024-09-10T10:02:21.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.923+0000 7eff0d207740 1 --2- 172.21.15.77:0/1165620623 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55d4f874d820 0x55d4f874fc70 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:21.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.923+0000 7eff0d207740 1 --2- 172.21.15.77:0/1165620623 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4f873db10 0x55d4f873b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:21.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.923+0000 7eff0d207740 1 --2- 172.21.15.77:0/1165620623 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4f8733250 0x55d4f873aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:21.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.923+0000 7eff0d207740 1 --2- 172.21.15.77:0/1165620623 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d4f8732930 0x55d4f873a9b0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:21.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.923+0000 7eff0d207740 1 -- 172.21.15.77:0/1165620623 >> 172.21.15.77:0/1165620623 conn(0x55d4f872eec0 msgr2=0x55d4f8729cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:21.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.923+0000 7efeed600640 0 client.16836 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T10:02:21.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.924+0000 7eff0d207740 1 -- 172.21.15.77:0/1165620623 shutdown_connections 2024-09-10T10:02:21.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.924+0000 7eff0d207740 1 -- 172.21.15.77:0/1165620623 wait complete. 2024-09-10T10:02:21.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.924+0000 7eff0d207740 20 client.16836 trim_cache size 0 max 16384 2024-09-10T10:02:21.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.941+0000 7eff0d207740 1 Processor -- start 2024-09-10T10:02:21.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.941+0000 7eff0d207740 1 -- start start 2024-09-10T10:02:21.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.941+0000 7eff0d207740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d4f8732930 0x55d4f873a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:21.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.941+0000 7eff0d207740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4f8733250 0x55d4f873b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:21.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.941+0000 7eff0d207740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4f873db10 0x55d4f873b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:21.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.942+0000 7eff0d207740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7eff041acbe0 con 0x55d4f8732930 2024-09-10T10:02:21.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.942+0000 7eff0d207740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7eff041ae720 con 0x55d4f873db10 2024-09-10T10:02:21.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.942+0000 7eff0d207740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7eff041ad2b0 con 0x55d4f8733250 2024-09-10T10:02:21.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.942+0000 7eff03400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d4f8732930 0x55d4f873a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:21.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.942+0000 7eff03400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d4f8732930 0x55d4f873a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57278/0 (socket says 172.21.15.77:57278) 2024-09-10T10:02:21.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.942+0000 7eff03400640 1 -- 172.21.15.77:0/2383391391 learned_addr learned my addr 172.21.15.77:0/2383391391 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:02:21.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.942+0000 7eff03e00640 1 --2- 172.21.15.77:0/2383391391 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4f873db10 0x55d4f873b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:21.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.942+0000 7eff02a00640 1 --2- 172.21.15.77:0/2383391391 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4f8733250 0x55d4f873b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:21.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.942+0000 7eff03400640 1 -- 172.21.15.77:0/2383391391 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4f8733250 msgr2=0x55d4f873b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:21.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.942+0000 7eff03400640 1 --2- 172.21.15.77:0/2383391391 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4f8733250 0x55d4f873b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:21.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.942+0000 7eff03400640 1 -- 172.21.15.77:0/2383391391 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4f873db10 msgr2=0x55d4f873b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:21.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.942+0000 7eff03400640 1 --2- 172.21.15.77:0/2383391391 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4f873db10 0x55d4f873b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:21.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.942+0000 7eff03400640 1 -- 172.21.15.77:0/2383391391 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4f872ecf0 con 0x55d4f8732930 2024-09-10T10:02:21.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.942+0000 7eff03e00640 1 --2- 172.21.15.77:0/2383391391 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4f873db10 0x55d4f873b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:02:21.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.943+0000 7eff03400640 1 --2- 172.21.15.77:0/2383391391 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d4f8732930 0x55d4f873a290 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7efef409b930 tx=0x7efef409a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da2b3229ab2b6e6b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:21.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.943+0000 7efeefe00640 1 -- 172.21.15.77:0/2383391391 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef409bc30 con 0x55d4f8732930 2024-09-10T10:02:21.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.943+0000 7efeefe00640 1 -- 172.21.15.77:0/2383391391 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efef40a48a0 con 0x55d4f8732930 2024-09-10T10:02:21.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.943+0000 7efeefe00640 1 -- 172.21.15.77:0/2383391391 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef4010430 con 0x55d4f8732930 2024-09-10T10:02:21.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.943+0000 7eff0d207740 1 -- 172.21.15.77:0/2383391391 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d4f8732930 msgr2=0x55d4f873a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:21.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.943+0000 7eff0d207740 1 --2- 172.21.15.77:0/2383391391 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d4f8732930 0x55d4f873a290 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7efef409b930 tx=0x7efef409a140 comp rx=0 tx=0).stop 2024-09-10T10:02:21.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.944+0000 7eff0d207740 1 -- 172.21.15.77:0/2383391391 shutdown_connections 2024-09-10T10:02:21.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.944+0000 7eff0d207740 1 --2- 172.21.15.77:0/2383391391 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4f873db10 0x55d4f873b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:21.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.944+0000 7eff0d207740 1 --2- 172.21.15.77:0/2383391391 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4f8733250 0x55d4f873b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:21.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.944+0000 7eff0d207740 1 --2- 172.21.15.77:0/2383391391 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d4f8732930 0x55d4f873a290 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:21.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.944+0000 7eff0d207740 1 -- 172.21.15.77:0/2383391391 >> 172.21.15.77:0/2383391391 conn(0x55d4f872eec0 msgr2=0x55d4f873a970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:21.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.944+0000 7eff0d207740 1 -- 172.21.15.77:0/2383391391 shutdown_connections 2024-09-10T10:02:21.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.944+0000 7eff0d207740 1 -- 172.21.15.77:0/2383391391 wait complete. 2024-09-10T10:02:21.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.944+0000 7eff0d207740 1 -- start start 2024-09-10T10:02:21.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.944+0000 7eff0d207740 1 Processor -- start 2024-09-10T10:02:21.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.945+0000 7eff0d207740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4f8732930 0x55d4f87419f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:21.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.945+0000 7eff0d207740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4f8733250 0x55d4f8741f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:21.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.945+0000 7eff0d207740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d4f873db10 0x55d4f8742470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:21.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.945+0000 7eff0d207740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7eff041ae050 con 0x55d4f873db10 2024-09-10T10:02:21.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.945+0000 7eff0d207740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7eff041ad980 con 0x55d4f8733250 2024-09-10T10:02:21.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.945+0000 7eff0d207740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7eff041aabc0 con 0x55d4f8732930 2024-09-10T10:02:21.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.945+0000 7eff0d207740 10 client.0 _ll_register_callbacks cb 0x7efef400e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:02:21.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.945+0000 7eff03400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4f8732930 0x55d4f87419f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:21.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.945+0000 7eff03400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4f8732930 0x55d4f87419f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:48694/0 (socket says 172.21.15.77:48694) 2024-09-10T10:02:21.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.945+0000 7eff03400640 1 -- 172.21.15.77:0/1388578935 learned_addr learned my addr 172.21.15.77:0/1388578935 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:02:21.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.945+0000 7eff02a00640 1 --2- 172.21.15.77:0/1388578935 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4f8733250 0x55d4f8741f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:21.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.945+0000 7eff03e00640 1 --2- 172.21.15.77:0/1388578935 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d4f873db10 0x55d4f8742470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:21.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.945+0000 7eff03400640 1 -- 172.21.15.77:0/1388578935 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4f8733250 msgr2=0x55d4f8741f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:21.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.945+0000 7eff03400640 1 --2- 172.21.15.77:0/1388578935 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4f8733250 0x55d4f8741f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:21.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.945+0000 7eff03400640 1 -- 172.21.15.77:0/1388578935 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d4f873db10 msgr2=0x55d4f8742470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:21.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.945+0000 7eff03400640 1 --2- 172.21.15.77:0/1388578935 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d4f873db10 0x55d4f8742470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:21.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.945+0000 7eff03400640 1 -- 172.21.15.77:0/1388578935 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4f87b76e0 con 0x55d4f8732930 2024-09-10T10:02:21.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.945+0000 7eff03e00640 1 --2- 172.21.15.77:0/1388578935 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d4f873db10 0x55d4f8742470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:02:21.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.946+0000 7eff03400640 1 --2- 172.21.15.77:0/1388578935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4f8732930 0x55d4f87419f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7efef40985b0 tx=0x7efef4097f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fdc5d9a299932890 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:21.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.946+0000 7efeed600640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-10T10:02:21.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.946+0000 7efeed600640 1 -- 172.21.15.77:0/1388578935 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef409bc30 con 0x55d4f8732930 2024-09-10T10:02:21.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.946+0000 7eff0d207740 1 -- 172.21.15.77:0/1388578935 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efef40a48a0 con 0x55d4f8732930 2024-09-10T10:02:21.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.946+0000 7efeed600640 1 -- 172.21.15.77:0/1388578935 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7efef40d5040 con 0x55d4f8732930 2024-09-10T10:02:21.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.946+0000 7efeed600640 1 -- 172.21.15.77:0/1388578935 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7efef4010430 con 0x55d4f8732930 2024-09-10T10:02:21.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.946+0000 7eff0d207740 10 client.26926 fetch_fsmap learned FSMap version 35 2024-09-10T10:02:21.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.946+0000 7eff0d207740 1 -- 172.21.15.77:0/1388578935 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55d4f87b76e0 con 0x55d4f8732930 2024-09-10T10:02:21.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.946+0000 7efeed600640 1 -- 172.21.15.77:0/1388578935 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efef40a48a0 con 0x55d4f8732930 2024-09-10T10:02:21.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.947+0000 7efeed600640 1 -- 172.21.15.77:0/1388578935 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7efef409ab80 con 0x55d4f8732930 2024-09-10T10:02:21.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.947+0000 7eff0d207740 10 client.26926 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T10:02:21.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.947+0000 7eff0d207740 10 client.26926 resolve_mds: validated gid 24563 aka daemon mds.b 2024-09-10T10:02:21.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.947+0000 7eff0d207740 20 client.26926 populate_metadata read hostname 'smithi077' 2024-09-10T10:02:21.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.947+0000 7eff0d207740 1 --2- 172.21.15.77:0/1388578935 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55d4f87b8c80 0x55d4f87bb070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:21.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.947+0000 7eff0d207740 4 client.26926 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "1d58ccc9", "release": true, "await": true}] 2024-09-10T10:02:21.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.947+0000 7eff0d207740 1 -- 172.21.15.77:0/1388578935 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "quiesce db", "set_id": "1d58ccc9", "release": true, "await": true}) -- 0x55d4f869c0d0 con 0x55d4f87b8c80 2024-09-10T10:02:21.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.947+0000 7eff02a00640 1 --2- 172.21.15.77:0/1388578935 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55d4f87b8c80 0x55d4f87bb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:21.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.948+0000 7eff02a00640 1 --2- 172.21.15.77:0/1388578935 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55d4f87b8c80 0x55d4f87bb070 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c84e82ca13c3e0b server_cookie=4d0317a157fdcec7 in_seq=0 out_seq=0 2024-09-10T10:02:21.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:21.948+0000 7efeed600640 10 client.26926 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T10:02:22.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:22.037+0000 7efeed600640 1 -- 172.21.15.77:0/1388578935 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55d4f869c0d0 con 0x55d4f87b8c80 2024-09-10T10:02:22.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:22.037+0000 7efeed600640 10 client.26926 handle_command_reply: tid=0 2024-09-10T10:02:22.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:22.037+0000 7eff0d207740 1 client.26926 shutdown 2024-09-10T10:02:22.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:22.038+0000 7eff0d207740 1 -- 172.21.15.77:0/1388578935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4f8732930 msgr2=0x55d4f87419f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:22.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:22.038+0000 7eff0d207740 1 --2- 172.21.15.77:0/1388578935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4f8732930 0x55d4f87419f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7efef40985b0 tx=0x7efef4097f40 comp rx=0 tx=0).stop 2024-09-10T10:02:22.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:22.038+0000 7eff0d207740 1 -- 172.21.15.77:0/1388578935 shutdown_connections 2024-09-10T10:02:22.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:22.038+0000 7eff0d207740 1 --2- 172.21.15.77:0/1388578935 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55d4f87b8c80 0x55d4f87bb070 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:22.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:22.038+0000 7eff0d207740 1 --2- 172.21.15.77:0/1388578935 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d4f873db10 0x55d4f8742470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:22.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:22.038+0000 7eff0d207740 1 --2- 172.21.15.77:0/1388578935 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4f8733250 0x55d4f8741f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:22.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:22.038+0000 7eff0d207740 1 --2- 172.21.15.77:0/1388578935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4f8732930 0x55d4f87419f0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:22.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:22.038+0000 7efeed600640 0 client.26926 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T10:02:22.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:22.038+0000 7eff0d207740 1 -- 172.21.15.77:0/1388578935 >> 172.21.15.77:0/1388578935 conn(0x55d4f872eec0 msgr2=0x55d4f873b170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:22.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:22.038+0000 7eff0d207740 1 -- 172.21.15.77:0/1388578935 shutdown_connections 2024-09-10T10:02:22.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:22.038+0000 7eff0d207740 1 -- 172.21.15.77:0/1388578935 wait complete. 2024-09-10T10:02:22.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:22.038+0000 7eff0d207740 20 client.26926 trim_cache size 0 max 16384 2024-09-10T10:02:22.041 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-10T10:02:22.042 INFO:teuthology.orchestra.run.smithi077.stdout: "epoch": 35, 2024-09-10T10:02:22.042 INFO:teuthology.orchestra.run.smithi077.stdout: "leader": 24563, 2024-09-10T10:02:22.042 INFO:teuthology.orchestra.run.smithi077.stdout: "set_version": 24, 2024-09-10T10:02:22.042 INFO:teuthology.orchestra.run.smithi077.stdout: "sets": { 2024-09-10T10:02:22.042 INFO:teuthology.orchestra.run.smithi077.stdout: "1d58ccc9": { 2024-09-10T10:02:22.042 INFO:teuthology.orchestra.run.smithi077.stdout: "version": 24, 2024-09-10T10:02:22.042 INFO:teuthology.orchestra.run.smithi077.stdout: "age_ref": 0.0, 2024-09-10T10:02:22.042 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-10T10:02:22.042 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RELEASED", 2024-09-10T10:02:22.042 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-10T10:02:22.042 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:02:22.042 INFO:teuthology.orchestra.run.smithi077.stdout: "timeout": 90.0, 2024-09-10T10:02:22.043 INFO:teuthology.orchestra.run.smithi077.stdout: "expiration": 140.1, 2024-09-10T10:02:22.043 INFO:teuthology.orchestra.run.smithi077.stdout: "members": { 2024-09-10T10:02:22.043 INFO:teuthology.orchestra.run.smithi077.stdout: "file:/": { 2024-09-10T10:02:22.043 INFO:teuthology.orchestra.run.smithi077.stdout: "excluded": false, 2024-09-10T10:02:22.043 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-10T10:02:22.043 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RELEASED", 2024-09-10T10:02:22.043 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-10T10:02:22.043 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:02:22.043 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:02:22.043 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:02:22.043 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:02:22.043 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:02:22.043 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-10T10:02:22.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:22.039+0000 7eff09200640 1 -- 172.21.15.77:0/1474118660 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7efee80a2b80 msgr2=0x7efee80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:22.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:22.039+0000 7eff09200640 1 --2- 172.21.15.77:0/1474118660 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7efee80a2b80 0x7efee80a5040 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7efefc004af0 tx=0x7efefc004a80 comp rx=0 tx=0).stop 2024-09-10T10:02:22.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:22.039+0000 7eff09200640 1 -- 172.21.15.77:0/1474118660 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7eff041019c0 msgr2=0x7eff04198240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:22.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:22.039+0000 7eff09200640 1 --2- 172.21.15.77:0/1474118660 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7eff041019c0 0x7eff04198240 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7efef400e9f0 tx=0x7efef400eeb0 comp rx=0 tx=0).stop 2024-09-10T10:02:22.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:22.039+0000 7eff09200640 1 -- 172.21.15.77:0/1474118660 shutdown_connections 2024-09-10T10:02:22.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:22.039+0000 7eff09200640 1 --2- 172.21.15.77:0/1474118660 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7efee80a2b80 0x7efee80a5040 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:22.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:22.039+0000 7eff09200640 1 --2- 172.21.15.77:0/1474118660 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff041079c0 0x7eff0419cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:22.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:22.039+0000 7eff09200640 1 --2- 172.21.15.77:0/1474118660 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7eff04102360 0x7eff04198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:22.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:22.039+0000 7eff09200640 1 --2- 172.21.15.77:0/1474118660 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7eff041019c0 0x7eff04198240 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:22.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:22.039+0000 7eff09200640 1 -- 172.21.15.77:0/1474118660 >> 172.21.15.77:0/1474118660 conn(0x7eff040fd740 msgr2=0x7eff040feea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:22.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:22.039+0000 7eff09200640 1 -- 172.21.15.77:0/1474118660 shutdown_connections 2024-09-10T10:02:22.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:22.039+0000 7eff09200640 1 -- 172.21.15.77:0/1474118660 wait complete. 2024-09-10T10:02:22.806 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '1d58ccc9', total seconds elapsed: 24.0 2024-09-10T10:02:23.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:02:22 smithi077 ceph-mon[30319]: pgmap v607: 129 pgs: 129 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 511 B/s wr, 9 op/s 2024-09-10T10:02:23.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:02:22 smithi134 ceph-mon[35674]: pgmap v607: 129 pgs: 129 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 511 B/s wr, 9 op/s 2024-09-10T10:02:23.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:02:22 smithi120 ceph-mon[34770]: pgmap v607: 129 pgs: 129 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 511 B/s wr, 9 op/s 2024-09-10T10:02:25.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:02:24 smithi134 ceph-mon[35674]: pgmap v608: 129 pgs: 129 active+clean; 137 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 94 KiB/s wr, 39 op/s 2024-09-10T10:02:25.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:02:24 smithi120 ceph-mon[34770]: pgmap v608: 129 pgs: 129 active+clean; 137 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 94 KiB/s wr, 39 op/s 2024-09-10T10:02:25.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:02:24 smithi077 ceph-mon[30319]: pgmap v608: 129 pgs: 129 active+clean; 137 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 94 KiB/s wr, 39 op/s 2024-09-10T10:02:26.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:02:26 smithi134 ceph-mon[35674]: pgmap v609: 129 pgs: 129 active+clean; 137 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 940 KiB/s rd, 145 KiB/s wr, 56 op/s 2024-09-10T10:02:26.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:02:26 smithi120 ceph-mon[34770]: pgmap v609: 129 pgs: 129 active+clean; 137 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 940 KiB/s rd, 145 KiB/s wr, 56 op/s 2024-09-10T10:02:26.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:02:26 smithi077 ceph-mon[30319]: pgmap v609: 129 pgs: 129 active+clean; 137 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 940 KiB/s rd, 145 KiB/s wr, 56 op/s 2024-09-10T10:02:29.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:02:28 smithi120 ceph-mon[34770]: pgmap v610: 129 pgs: 129 active+clean; 137 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 245 KiB/s wr, 98 op/s 2024-09-10T10:02:29.191 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:02:28 smithi077 ceph-mon[30319]: pgmap v610: 129 pgs: 129 active+clean; 137 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 245 KiB/s wr, 98 op/s 2024-09-10T10:02:29.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:02:28 smithi134 ceph-mon[35674]: pgmap v610: 129 pgs: 129 active+clean; 137 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 245 KiB/s wr, 98 op/s 2024-09-10T10:02:31.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:02:30 smithi120 ceph-mon[34770]: pgmap v611: 129 pgs: 129 active+clean; 136 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 719 KiB/s wr, 263 op/s 2024-09-10T10:02:31.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:02:30 smithi077 ceph-mon[30319]: pgmap v611: 129 pgs: 129 active+clean; 136 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 719 KiB/s wr, 263 op/s 2024-09-10T10:02:31.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:02:30 smithi134 ceph-mon[35674]: pgmap v611: 129 pgs: 129 active+clean; 136 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 719 KiB/s wr, 263 op/s 2024-09-10T10:02:33.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:02:32 smithi120 ceph-mon[34770]: pgmap v612: 129 pgs: 129 active+clean; 144 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 796 KiB/s wr, 292 op/s 2024-09-10T10:02:33.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:02:32 smithi077 ceph-mon[30319]: pgmap v612: 129 pgs: 129 active+clean; 144 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 796 KiB/s wr, 292 op/s 2024-09-10T10:02:33.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:02:32 smithi134 ceph-mon[35674]: pgmap v612: 129 pgs: 129 active+clean; 144 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 796 KiB/s wr, 292 op/s 2024-09-10T10:02:33.532 INFO:tasks.workunit.client.0.smithi077.stdout:truncating to largest ever: 0x63e5d2a 2024-09-10T10:02:34.335 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T10:02:34.826 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:02:35.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:02:34 smithi120 ceph-mon[34770]: pgmap v613: 129 pgs: 129 active+clean; 144 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 1.1 MiB/s wr, 400 op/s 2024-09-10T10:02:35.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:02:34 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:02:35.179 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:02:34 smithi077 ceph-mon[30319]: pgmap v613: 129 pgs: 129 active+clean; 144 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 1.1 MiB/s wr, 400 op/s 2024-09-10T10:02:35.179 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:02:34 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:02:35.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:02:34 smithi134 ceph-mon[35674]: pgmap v613: 129 pgs: 129 active+clean; 144 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 1.1 MiB/s wr, 400 op/s 2024-09-10T10:02:35.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:02:34 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:02:35.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.478+0000 7ffa80e00640 1 -- 172.21.15.77:0/4164615317 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa7c1068d0 msgr2=0x7ffa7c106cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:35.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.478+0000 7ffa80e00640 1 --2- 172.21.15.77:0/4164615317 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa7c1068d0 0x7ffa7c106cb0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c009cd0 tx=0x7ffa6c02a910 comp rx=0 tx=0).stop 2024-09-10T10:02:35.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.478+0000 7ffa80e00640 1 -- 172.21.15.77:0/4164615317 shutdown_connections 2024-09-10T10:02:35.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.478+0000 7ffa80e00640 1 --2- 172.21.15.77:0/4164615317 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa7c101270 0x7ffa7c10d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:35.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.478+0000 7ffa80e00640 1 --2- 172.21.15.77:0/4164615317 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa7c1008d0 0x7ffa7c100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:35.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.478+0000 7ffa80e00640 1 --2- 172.21.15.77:0/4164615317 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa7c1068d0 0x7ffa7c106cb0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:35.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.478+0000 7ffa80e00640 1 -- 172.21.15.77:0/4164615317 >> 172.21.15.77:0/4164615317 conn(0x7ffa7c0fc610 msgr2=0x7ffa7c0fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:35.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.478+0000 7ffa80e00640 1 -- 172.21.15.77:0/4164615317 shutdown_connections 2024-09-10T10:02:35.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.478+0000 7ffa80e00640 1 -- 172.21.15.77:0/4164615317 wait complete. 2024-09-10T10:02:35.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.479+0000 7ffa80e00640 1 Processor -- start 2024-09-10T10:02:35.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.480+0000 7ffa80e00640 1 -- start start 2024-09-10T10:02:35.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.480+0000 7ffa80e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa7c1008d0 0x7ffa7c111c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:35.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.480+0000 7ffa80e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa7c101270 0x7ffa7c1121b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:35.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.480+0000 7ffa80e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa7c1068d0 0x7ffa7c1a43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:35.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.480+0000 7ffa80e00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ffa7c10e4f0 con 0x7ffa7c1068d0 2024-09-10T10:02:35.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.480+0000 7ffa80e00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ffa7c10e370 con 0x7ffa7c101270 2024-09-10T10:02:35.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.480+0000 7ffa80e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ffa7c10e670 con 0x7ffa7c1008d0 2024-09-10T10:02:35.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.480+0000 7ffa7b400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa7c1008d0 0x7ffa7c111c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:35.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.481+0000 7ffa7be00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa7c1068d0 0x7ffa7c1a43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:35.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.481+0000 7ffa7be00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa7c1068d0 0x7ffa7c1a43e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:53636/0 (socket says 172.21.15.77:53636) 2024-09-10T10:02:35.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.481+0000 7ffa7be00640 1 -- 172.21.15.77:0/2905312543 learned_addr learned my addr 172.21.15.77:0/2905312543 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:02:35.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.481+0000 7ffa7aa00640 1 --2- 172.21.15.77:0/2905312543 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa7c101270 0x7ffa7c1121b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:35.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.482+0000 7ffa7be00640 1 -- 172.21.15.77:0/2905312543 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa7c1008d0 msgr2=0x7ffa7c111c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:35.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.482+0000 7ffa7be00640 1 --2- 172.21.15.77:0/2905312543 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa7c1008d0 0x7ffa7c111c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:35.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.482+0000 7ffa7be00640 1 -- 172.21.15.77:0/2905312543 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa7c101270 msgr2=0x7ffa7c1121b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:35.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.482+0000 7ffa7be00640 1 --2- 172.21.15.77:0/2905312543 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa7c101270 0x7ffa7c1121b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:35.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.482+0000 7ffa7be00640 1 -- 172.21.15.77:0/2905312543 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa7c1a4ac0 con 0x7ffa7c1068d0 2024-09-10T10:02:35.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.483+0000 7ffa7be00640 1 --2- 172.21.15.77:0/2905312543 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa7c1068d0 0x7ffa7c1a43e0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7ffa7400e970 tx=0x7ffa7400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:35.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.484+0000 7ffa73e00640 1 -- 172.21.15.77:0/2905312543 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa7400ccc0 con 0x7ffa7c1068d0 2024-09-10T10:02:35.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.484+0000 7ffa73e00640 1 -- 172.21.15.77:0/2905312543 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa7400ce60 con 0x7ffa7c1068d0 2024-09-10T10:02:35.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.484+0000 7ffa73e00640 1 -- 172.21.15.77:0/2905312543 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa74010490 con 0x7ffa7c1068d0 2024-09-10T10:02:35.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.484+0000 7ffa80e00640 1 -- 172.21.15.77:0/2905312543 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa7c1a4d50 con 0x7ffa7c1068d0 2024-09-10T10:02:35.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.484+0000 7ffa80e00640 1 -- 172.21.15.77:0/2905312543 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa7c1ab4d0 con 0x7ffa7c1068d0 2024-09-10T10:02:35.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.489+0000 7ffa84e92740 1 Processor -- start 2024-09-10T10:02:35.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.490+0000 7ffa73e00640 1 -- 172.21.15.77:0/2905312543 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ffa74005000 con 0x7ffa7c1068d0 2024-09-10T10:02:35.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.490+0000 7ffa84e92740 1 -- start start 2024-09-10T10:02:35.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.491+0000 7ffa84e92740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f0a010ed50 0x55f0a010f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:35.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.491+0000 7ffa84e92740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0a010f670 0x55f0a01199f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:35.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.491+0000 7ffa84e92740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0a0119f30 0x55f0a011c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:35.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.491+0000 7ffa84e92740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f0a00753e0 con 0x55f0a0119f30 2024-09-10T10:02:35.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.491+0000 7ffa84e92740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f0a001cd80 con 0x55f0a010f670 2024-09-10T10:02:35.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.491+0000 7ffa84e92740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f0a00713a0 con 0x55f0a010ed50 2024-09-10T10:02:35.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.491+0000 7ffa7aa00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f0a010ed50 0x55f0a010f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:35.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.491+0000 7ffa7b400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0a010f670 0x55f0a01199f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:35.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.491+0000 7ffa7aa00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f0a010ed50 0x55f0a010f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:54662/0 (socket says 172.21.15.77:54662) 2024-09-10T10:02:35.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.491+0000 7ffa7aa00640 1 -- 172.21.15.77:0/2634090067 learned_addr learned my addr 172.21.15.77:0/2634090067 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:02:35.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.491+0000 7ffa7aa00640 1 --2- 172.21.15.77:0/2634090067 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0a0119f30 0x55f0a011c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:35.494 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.492+0000 7ffa7b400640 1 -- 172.21.15.77:0/2634090067 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f0a010ed50 msgr2=0x55f0a010f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:35.494 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.492+0000 7ffa7b400640 1 --2- 172.21.15.77:0/2634090067 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f0a010ed50 0x55f0a010f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:35.494 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.492+0000 7ffa7b400640 1 -- 172.21.15.77:0/2634090067 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0a0119f30 msgr2=0x55f0a011c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:35.494 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.492+0000 7ffa7b400640 1 --2- 172.21.15.77:0/2634090067 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0a0119f30 0x55f0a011c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:35.494 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.492+0000 7ffa7b400640 1 -- 172.21.15.77:0/2634090067 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0a003bae0 con 0x55f0a010f670 2024-09-10T10:02:35.494 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.492+0000 7ffa7b400640 1 --2- 172.21.15.77:0/2634090067 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0a010f670 0x55f0a01199f0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c009ca0 tx=0x7ffa6c02f420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9eaded435eb28e52 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:35.494 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.492+0000 7ffa73e00640 1 --2- 172.21.15.77:0/2905312543 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ffa600a2c00 0x7ffa600a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:35.494 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.493+0000 7ffa70c00640 1 -- 172.21.15.77:0/2634090067 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6c031b40 con 0x55f0a010f670 2024-09-10T10:02:35.494 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.493+0000 7ffa70c00640 1 -- 172.21.15.77:0/2634090067 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa6c036040 con 0x55f0a010f670 2024-09-10T10:02:35.494 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.493+0000 7ffa70c00640 1 -- 172.21.15.77:0/2634090067 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6c02fb50 con 0x55f0a010f670 2024-09-10T10:02:35.494 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.493+0000 7ffa73e00640 1 -- 172.21.15.77:0/2905312543 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7ffa740cf8d0 con 0x7ffa7c1068d0 2024-09-10T10:02:35.494 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.493+0000 7ffa7aa00640 1 --2- 172.21.15.77:0/2905312543 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ffa600a2c00 0x7ffa600a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:35.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.494+0000 7ffa84e92740 1 -- 172.21.15.77:0/2634090067 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0a010f670 msgr2=0x55f0a01199f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:35.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.494+0000 7ffa84e92740 1 --2- 172.21.15.77:0/2634090067 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0a010f670 0x55f0a01199f0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c009ca0 tx=0x7ffa6c02f420 comp rx=0 tx=0).stop 2024-09-10T10:02:35.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.494+0000 7ffa7aa00640 1 --2- 172.21.15.77:0/2905312543 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ffa600a2c00 0x7ffa600a50c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x55f0a010c660 tx=0x7ffa6400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:35.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.494+0000 7ffa84e92740 1 -- 172.21.15.77:0/2634090067 shutdown_connections 2024-09-10T10:02:35.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.494+0000 7ffa84e92740 1 --2- 172.21.15.77:0/2634090067 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0a0119f30 0x55f0a011c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:35.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.494+0000 7ffa84e92740 1 --2- 172.21.15.77:0/2634090067 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0a010f670 0x55f0a01199f0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:35.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.494+0000 7ffa84e92740 1 --2- 172.21.15.77:0/2634090067 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f0a010ed50 0x55f0a010f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:35.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.494+0000 7ffa84e92740 1 -- 172.21.15.77:0/2634090067 >> 172.21.15.77:0/2634090067 conn(0x55f0a010b2e0 msgr2=0x55f0a010b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:35.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.494+0000 7ffa84e92740 1 -- 172.21.15.77:0/2634090067 shutdown_connections 2024-09-10T10:02:35.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.494+0000 7ffa84e92740 1 -- 172.21.15.77:0/2634090067 wait complete. 2024-09-10T10:02:35.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.494+0000 7ffa84e92740 1 -- start start 2024-09-10T10:02:35.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.495+0000 7ffa84e92740 1 Processor -- start 2024-09-10T10:02:35.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.495+0000 7ffa84e92740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f0a010ed50 0x55f0a0116da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:35.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.495+0000 7ffa84e92740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0a010f670 0x55f0a01172e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:35.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.495+0000 7ffa84e92740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0a0119f30 0x55f0a0117820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:35.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.495+0000 7ffa84e92740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f0a011cb40 con 0x55f0a010f670 2024-09-10T10:02:35.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.495+0000 7ffa84e92740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f0a00300f0 con 0x55f0a0119f30 2024-09-10T10:02:35.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.495+0000 7ffa84e92740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f0a011ccc0 con 0x55f0a010ed50 2024-09-10T10:02:35.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.495+0000 7ffa84e92740 10 client.0 _ll_register_callbacks cb 0x55f0a00219e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:02:35.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.495+0000 7ffa7aa00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0a010f670 0x55f0a01172e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:35.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.495+0000 7ffa7aa00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0a010f670 0x55f0a01172e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:53660/0 (socket says 172.21.15.77:53660) 2024-09-10T10:02:35.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.495+0000 7ffa7aa00640 1 -- 172.21.15.77:0/2695223010 learned_addr learned my addr 172.21.15.77:0/2695223010 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:02:35.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.495+0000 7ffa7b400640 1 --2- 172.21.15.77:0/2695223010 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f0a010ed50 0x55f0a0116da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:35.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.496+0000 7ffa7be00640 1 --2- 172.21.15.77:0/2695223010 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0a0119f30 0x55f0a0117820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:35.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.496+0000 7ffa7b400640 1 -- 172.21.15.77:0/2695223010 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0a0119f30 msgr2=0x55f0a0117820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:35.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.496+0000 7ffa7b400640 1 --2- 172.21.15.77:0/2695223010 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0a0119f30 0x55f0a0117820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:35.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.496+0000 7ffa7b400640 1 -- 172.21.15.77:0/2695223010 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0a010f670 msgr2=0x55f0a01172e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:35.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.496+0000 7ffa7b400640 1 --2- 172.21.15.77:0/2695223010 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0a010f670 0x55f0a01172e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:35.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.496+0000 7ffa7b400640 1 -- 172.21.15.77:0/2695223010 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f09fde63c0 con 0x55f0a010ed50 2024-09-10T10:02:35.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.496+0000 7ffa7be00640 1 --2- 172.21.15.77:0/2695223010 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0a0119f30 0x55f0a0117820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:02:35.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.496+0000 7ffa7aa00640 1 --2- 172.21.15.77:0/2695223010 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0a010f670 0x55f0a01172e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:02:35.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.496+0000 7ffa7b400640 1 --2- 172.21.15.77:0/2695223010 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f0a010ed50 0x55f0a0116da0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c02ae30 tx=0x7ffa6c031690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5860beb70dbed4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:35.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.496+0000 7ffa69600640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-10T10:02:35.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.497+0000 7ffa69600640 1 -- 172.21.15.77:0/2695223010 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6c052060 con 0x55f0a010ed50 2024-09-10T10:02:35.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.497+0000 7ffa69600640 1 -- 172.21.15.77:0/2695223010 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7ffa6c040040 con 0x55f0a010ed50 2024-09-10T10:02:35.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.497+0000 7ffa84e92740 1 -- 172.21.15.77:0/2695223010 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa6c036040 con 0x55f0a010ed50 2024-09-10T10:02:35.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.497+0000 7ffa69600640 1 -- 172.21.15.77:0/2695223010 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7ffa6c059050 con 0x55f0a010ed50 2024-09-10T10:02:35.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.497+0000 7ffa84e92740 10 client.26938 fetch_fsmap learned FSMap version 35 2024-09-10T10:02:35.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.497+0000 7ffa84e92740 1 -- 172.21.15.77:0/2695223010 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55f0a011cb40 con 0x55f0a010ed50 2024-09-10T10:02:35.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.498+0000 7ffa69600640 1 -- 172.21.15.77:0/2695223010 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa6c02fdb0 con 0x55f0a010ed50 2024-09-10T10:02:35.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.498+0000 7ffa69600640 1 -- 172.21.15.77:0/2695223010 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7ffa6c04c070 con 0x55f0a010ed50 2024-09-10T10:02:35.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.498+0000 7ffa84e92740 10 client.26938 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T10:02:35.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.498+0000 7ffa84e92740 10 client.26938 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T10:02:35.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.498+0000 7ffa84e92740 20 client.26938 populate_metadata read hostname 'smithi077' 2024-09-10T10:02:35.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.498+0000 7ffa84e92740 1 --2- 172.21.15.77:0/2695223010 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55f0a01298f0 0x55f0a012bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:35.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.498+0000 7ffa84e92740 4 client.26938 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:02:35.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.498+0000 7ffa84e92740 1 -- 172.21.15.77:0/2695223010 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ffa6c036040 con 0x55f0a01298f0 2024-09-10T10:02:35.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.499+0000 7ffa7aa00640 1 --2- 172.21.15.77:0/2695223010 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55f0a01298f0 0x55f0a012bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:35.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.499+0000 7ffa69600640 10 client.26938 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T10:02:35.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.499+0000 7ffa7aa00640 1 --2- 172.21.15.77:0/2695223010 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55f0a01298f0 0x55f0a012bce0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27428ebf6849ed6d server_cookie=eecaefa65f4f40b5 in_seq=0 out_seq=0 2024-09-10T10:02:35.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.500+0000 7ffa69600640 1 -- 172.21.15.77:0/2695223010 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7ffa6c036040 con 0x55f0a01298f0 2024-09-10T10:02:35.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.500+0000 7ffa69600640 10 client.26938 handle_command_reply: tid=0 2024-09-10T10:02:35.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.500+0000 7ffa84e92740 1 client.26938 shutdown 2024-09-10T10:02:35.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.500+0000 7ffa84e92740 1 -- 172.21.15.77:0/2695223010 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f0a010ed50 msgr2=0x55f0a0116da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:35.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.500+0000 7ffa84e92740 1 --2- 172.21.15.77:0/2695223010 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f0a010ed50 0x55f0a0116da0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c02ae30 tx=0x7ffa6c031690 comp rx=0 tx=0).stop 2024-09-10T10:02:35.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.500+0000 7ffa84e92740 1 -- 172.21.15.77:0/2695223010 shutdown_connections 2024-09-10T10:02:35.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.500+0000 7ffa84e92740 1 --2- 172.21.15.77:0/2695223010 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55f0a01298f0 0x55f0a012bce0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:35.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.500+0000 7ffa84e92740 1 --2- 172.21.15.77:0/2695223010 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0a0119f30 0x55f0a0117820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:35.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.500+0000 7ffa84e92740 1 --2- 172.21.15.77:0/2695223010 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0a010f670 0x55f0a01172e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:35.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.500+0000 7ffa84e92740 1 --2- 172.21.15.77:0/2695223010 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f0a010ed50 0x55f0a0116da0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:35.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.501+0000 7ffa84e92740 1 -- 172.21.15.77:0/2695223010 >> 172.21.15.77:0/2695223010 conn(0x55f0a010b2e0 msgr2=0x55f0a0110fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:35.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.501+0000 7ffa69600640 0 client.26938 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T10:02:35.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.501+0000 7ffa84e92740 1 -- 172.21.15.77:0/2695223010 shutdown_connections 2024-09-10T10:02:35.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.501+0000 7ffa84e92740 1 -- 172.21.15.77:0/2695223010 wait complete. 2024-09-10T10:02:35.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.501+0000 7ffa84e92740 20 client.26938 trim_cache size 0 max 16384 2024-09-10T10:02:35.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.528+0000 7ffa84e92740 1 Processor -- start 2024-09-10T10:02:35.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.528+0000 7ffa84e92740 1 -- start start 2024-09-10T10:02:35.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.529+0000 7ffa84e92740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0a010ed50 0x55f0a01117b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:35.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.529+0000 7ffa84e92740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0a010f670 0x55f0a010a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:35.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.529+0000 7ffa84e92740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f0a0119f30 0x55f0a01314a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:35.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.529+0000 7ffa84e92740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ffa7c1b5290 con 0x55f0a010f670 2024-09-10T10:02:35.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.529+0000 7ffa84e92740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ffa7c1b6dd0 con 0x55f0a010ed50 2024-09-10T10:02:35.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.529+0000 7ffa84e92740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ffa7c1b5960 con 0x55f0a0119f30 2024-09-10T10:02:35.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.529+0000 7ffa7b400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0a010ed50 0x55f0a01117b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:35.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.529+0000 7ffa7b400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0a010ed50 0x55f0a01117b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:40648/0 (socket says 172.21.15.77:40648) 2024-09-10T10:02:35.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.529+0000 7ffa7b400640 1 -- 172.21.15.77:0/2893318935 learned_addr learned my addr 172.21.15.77:0/2893318935 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:02:35.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.529+0000 7ffa7be00640 1 --2- 172.21.15.77:0/2893318935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f0a0119f30 0x55f0a01314a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:35.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.529+0000 7ffa7aa00640 1 --2- 172.21.15.77:0/2893318935 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0a010f670 0x55f0a010a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:35.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.530+0000 7ffa7b400640 1 -- 172.21.15.77:0/2893318935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f0a0119f30 msgr2=0x55f0a01314a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:35.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.530+0000 7ffa7b400640 1 --2- 172.21.15.77:0/2893318935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f0a0119f30 0x55f0a01314a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:35.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.530+0000 7ffa7b400640 1 -- 172.21.15.77:0/2893318935 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0a010f670 msgr2=0x55f0a010a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:35.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.530+0000 7ffa7b400640 1 --2- 172.21.15.77:0/2893318935 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0a010f670 0x55f0a010a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:35.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.530+0000 7ffa7b400640 1 -- 172.21.15.77:0/2893318935 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0a010ad80 con 0x55f0a010ed50 2024-09-10T10:02:35.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.530+0000 7ffa7b400640 1 --2- 172.21.15.77:0/2893318935 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0a010ed50 0x55f0a01117b0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c000f80 tx=0x7ffa6c005bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9af798697e92757f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:35.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.530+0000 7ffa6be00640 1 -- 172.21.15.77:0/2893318935 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6c052040 con 0x55f0a010ed50 2024-09-10T10:02:35.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.530+0000 7ffa6be00640 1 -- 172.21.15.77:0/2893318935 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa6c036070 con 0x55f0a010ed50 2024-09-10T10:02:35.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.530+0000 7ffa6be00640 1 -- 172.21.15.77:0/2893318935 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6c043400 con 0x55f0a010ed50 2024-09-10T10:02:35.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.531+0000 7ffa84e92740 1 -- 172.21.15.77:0/2893318935 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0a010ed50 msgr2=0x55f0a01117b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:35.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.531+0000 7ffa84e92740 1 --2- 172.21.15.77:0/2893318935 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0a010ed50 0x55f0a01117b0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c000f80 tx=0x7ffa6c005bd0 comp rx=0 tx=0).stop 2024-09-10T10:02:35.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.531+0000 7ffa84e92740 1 -- 172.21.15.77:0/2893318935 shutdown_connections 2024-09-10T10:02:35.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.531+0000 7ffa84e92740 1 --2- 172.21.15.77:0/2893318935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f0a0119f30 0x55f0a01314a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:35.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.531+0000 7ffa84e92740 1 --2- 172.21.15.77:0/2893318935 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0a010f670 0x55f0a010a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:35.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.531+0000 7ffa84e92740 1 --2- 172.21.15.77:0/2893318935 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0a010ed50 0x55f0a01117b0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:35.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.531+0000 7ffa84e92740 1 -- 172.21.15.77:0/2893318935 >> 172.21.15.77:0/2893318935 conn(0x55f0a010b2e0 msgr2=0x55f0a0109b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:35.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.531+0000 7ffa84e92740 1 -- 172.21.15.77:0/2893318935 shutdown_connections 2024-09-10T10:02:35.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.531+0000 7ffa84e92740 1 -- 172.21.15.77:0/2893318935 wait complete. 2024-09-10T10:02:35.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.532+0000 7ffa84e92740 1 -- start start 2024-09-10T10:02:35.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.532+0000 7ffa84e92740 1 Processor -- start 2024-09-10T10:02:35.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.533+0000 7ffa84e92740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f0a010ed50 0x55f0a01224b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:35.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.533+0000 7ffa84e92740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0a010f670 0x55f0a011d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:35.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.533+0000 7ffa84e92740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0a0119f30 0x55f0a011dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:35.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.533+0000 7ffa84e92740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ffa7c1b6700 con 0x55f0a010f670 2024-09-10T10:02:35.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.533+0000 7ffa84e92740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ffa7c1b6030 con 0x55f0a0119f30 2024-09-10T10:02:35.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.533+0000 7ffa84e92740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ffa7c1b3270 con 0x55f0a010ed50 2024-09-10T10:02:35.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.533+0000 7ffa84e92740 10 client.0 _ll_register_callbacks cb 0x7ffa6c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:02:35.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.533+0000 7ffa7b400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f0a010ed50 0x55f0a01224b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:35.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.533+0000 7ffa7be00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0a0119f30 0x55f0a011dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:35.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.533+0000 7ffa7b400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f0a010ed50 0x55f0a01224b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:54700/0 (socket says 172.21.15.77:54700) 2024-09-10T10:02:35.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.533+0000 7ffa7b400640 1 -- 172.21.15.77:0/3874979946 learned_addr learned my addr 172.21.15.77:0/3874979946 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:02:35.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.533+0000 7ffa7aa00640 1 --2- 172.21.15.77:0/3874979946 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0a010f670 0x55f0a011d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:35.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.534+0000 7ffa7be00640 1 -- 172.21.15.77:0/3874979946 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f0a010ed50 msgr2=0x55f0a01224b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:35.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.534+0000 7ffa7be00640 1 --2- 172.21.15.77:0/3874979946 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f0a010ed50 0x55f0a01224b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:35.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.534+0000 7ffa7be00640 1 -- 172.21.15.77:0/3874979946 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0a010f670 msgr2=0x55f0a011d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:35.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.534+0000 7ffa7be00640 1 --2- 172.21.15.77:0/3874979946 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0a010f670 0x55f0a011d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:35.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.534+0000 7ffa7be00640 1 -- 172.21.15.77:0/3874979946 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0a011e360 con 0x55f0a0119f30 2024-09-10T10:02:35.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.534+0000 7ffa7be00640 1 --2- 172.21.15.77:0/3874979946 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0a0119f30 0x55f0a011dd20 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ffa740ccf50 tx=0x7ffa7409c460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee95504e5f45290e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:35.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.534+0000 7ffa69600640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T10:02:35.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.535+0000 7ffa69600640 1 -- 172.21.15.77:0/3874979946 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa740a4950 con 0x55f0a0119f30 2024-09-10T10:02:35.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.535+0000 7ffa69600640 1 -- 172.21.15.77:0/3874979946 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7ffa740052f0 con 0x55f0a0119f30 2024-09-10T10:02:35.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.535+0000 7ffa84e92740 1 -- 172.21.15.77:0/3874979946 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa6c036070 con 0x55f0a0119f30 2024-09-10T10:02:35.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.535+0000 7ffa69600640 1 -- 172.21.15.77:0/3874979946 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7ffa7407ab50 con 0x55f0a0119f30 2024-09-10T10:02:35.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.536+0000 7ffa84e92740 10 client.26990 fetch_fsmap learned FSMap version 35 2024-09-10T10:02:35.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.536+0000 7ffa84e92740 1 -- 172.21.15.77:0/3874979946 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55f0a01942d0 con 0x55f0a0119f30 2024-09-10T10:02:35.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.536+0000 7ffa69600640 1 -- 172.21.15.77:0/3874979946 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa6c036070 con 0x55f0a0119f30 2024-09-10T10:02:35.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.536+0000 7ffa69600640 1 -- 172.21.15.77:0/3874979946 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7ffa7409ada0 con 0x55f0a0119f30 2024-09-10T10:02:35.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.536+0000 7ffa84e92740 10 client.26990 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T10:02:35.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.536+0000 7ffa84e92740 10 client.26990 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T10:02:35.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.536+0000 7ffa84e92740 20 client.26990 populate_metadata read hostname 'smithi077' 2024-09-10T10:02:35.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.536+0000 7ffa84e92740 1 --2- 172.21.15.77:0/3874979946 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55f0a0194a40 0x55f0a0196e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:35.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.536+0000 7ffa84e92740 4 client.26990 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T10:02:35.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.536+0000 7ffa84e92740 1 -- 172.21.15.77:0/3874979946 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f0a0082950 con 0x55f0a0194a40 2024-09-10T10:02:35.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.538+0000 7ffa7b400640 1 --2- 172.21.15.77:0/3874979946 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55f0a0194a40 0x55f0a0196e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:35.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.538+0000 7ffa7b400640 1 --2- 172.21.15.77:0/3874979946 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55f0a0194a40 0x55f0a0196e30 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89b85cf6a6df5b90 server_cookie=b6f9525e793dc5e in_seq=0 out_seq=0 2024-09-10T10:02:35.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.538+0000 7ffa69600640 10 client.26990 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T10:02:35.543 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.542+0000 7ffa69600640 1 -- 172.21.15.77:0/3874979946 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55f0a0082950 con 0x55f0a0194a40 2024-09-10T10:02:35.543 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.542+0000 7ffa69600640 10 client.26990 handle_command_reply: tid=0 2024-09-10T10:02:35.544 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.542+0000 7ffa84e92740 1 client.26990 shutdown 2024-09-10T10:02:35.544 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.542+0000 7ffa84e92740 1 -- 172.21.15.77:0/3874979946 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0a0119f30 msgr2=0x55f0a011dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:35.544 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.542+0000 7ffa84e92740 1 --2- 172.21.15.77:0/3874979946 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0a0119f30 0x55f0a011dd20 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ffa740ccf50 tx=0x7ffa7409c460 comp rx=0 tx=0).stop 2024-09-10T10:02:35.544 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.543+0000 7ffa84e92740 1 -- 172.21.15.77:0/3874979946 shutdown_connections 2024-09-10T10:02:35.544 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.543+0000 7ffa84e92740 1 --2- 172.21.15.77:0/3874979946 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55f0a0194a40 0x55f0a0196e30 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:35.544 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.543+0000 7ffa69600640 0 client.26990 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T10:02:35.544 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.543+0000 7ffa84e92740 1 --2- 172.21.15.77:0/3874979946 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0a0119f30 0x55f0a011dd20 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:35.545 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.543+0000 7ffa84e92740 1 --2- 172.21.15.77:0/3874979946 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0a010f670 0x55f0a011d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:35.545 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.543+0000 7ffa84e92740 1 --2- 172.21.15.77:0/3874979946 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f0a010ed50 0x55f0a01224b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:35.545 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.543+0000 7ffa84e92740 1 -- 172.21.15.77:0/3874979946 >> 172.21.15.77:0/3874979946 conn(0x55f0a010b2e0 msgr2=0x55f0a01174f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:35.545 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.543+0000 7ffa84e92740 1 -- 172.21.15.77:0/3874979946 shutdown_connections 2024-09-10T10:02:35.545 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.543+0000 7ffa84e92740 1 -- 172.21.15.77:0/3874979946 wait complete. 2024-09-10T10:02:35.545 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.543+0000 7ffa84e92740 20 client.26990 trim_cache size 0 max 16384 2024-09-10T10:02:35.545 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-10T10:02:35.545 INFO:teuthology.orchestra.run.smithi077.stdout: "status": "no active scrubs running", 2024-09-10T10:02:35.546 INFO:teuthology.orchestra.run.smithi077.stdout: "scrubs": {} 2024-09-10T10:02:35.546 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-10T10:02:35.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.546+0000 7ffa80e00640 1 -- 172.21.15.77:0/2905312543 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ffa600a2c00 msgr2=0x7ffa600a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:35.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.546+0000 7ffa80e00640 1 --2- 172.21.15.77:0/2905312543 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ffa600a2c00 0x7ffa600a50c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x55f0a010c660 tx=0x7ffa6400a2d0 comp rx=0 tx=0).stop 2024-09-10T10:02:35.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.546+0000 7ffa80e00640 1 -- 172.21.15.77:0/2905312543 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa7c1068d0 msgr2=0x7ffa7c1a43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:35.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.546+0000 7ffa80e00640 1 --2- 172.21.15.77:0/2905312543 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa7c1068d0 0x7ffa7c1a43e0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7ffa7400e970 tx=0x7ffa7400ed70 comp rx=0 tx=0).stop 2024-09-10T10:02:35.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.546+0000 7ffa80e00640 1 -- 172.21.15.77:0/2905312543 shutdown_connections 2024-09-10T10:02:35.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.546+0000 7ffa80e00640 1 --2- 172.21.15.77:0/2905312543 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ffa600a2c00 0x7ffa600a50c0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:35.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.546+0000 7ffa80e00640 1 --2- 172.21.15.77:0/2905312543 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa7c1068d0 0x7ffa7c1a43e0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:35.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.546+0000 7ffa80e00640 1 --2- 172.21.15.77:0/2905312543 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa7c101270 0x7ffa7c1121b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:35.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.546+0000 7ffa80e00640 1 --2- 172.21.15.77:0/2905312543 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa7c1008d0 0x7ffa7c111c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:35.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.546+0000 7ffa80e00640 1 -- 172.21.15.77:0/2905312543 >> 172.21.15.77:0/2905312543 conn(0x7ffa7c0fc610 msgr2=0x7ffa7c0fd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:35.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.547+0000 7ffa80e00640 1 -- 172.21.15.77:0/2905312543 shutdown_connections 2024-09-10T10:02:35.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:35.547+0000 7ffa80e00640 1 -- 172.21.15.77:0/2905312543 wait complete. 2024-09-10T10:02:36.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:02:35 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:02:36.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:02:35 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:02:36.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:02:35 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T10:02:36.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:02:35 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:02:36.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:02:35 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:02:36.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:02:35 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T10:02:36.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:02:35 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:02:36.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:02:35 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:02:36.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:02:35 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T10:02:36.452 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T10:02:36.452 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:02:36.934 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:02:37.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:02:36 smithi077 ceph-mon[30319]: pgmap v614: 129 pgs: 129 active+clean; 149 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 1.1 MiB/s wr, 420 op/s 2024-09-10T10:02:37.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:02:36 smithi134 ceph-mon[35674]: pgmap v614: 129 pgs: 129 active+clean; 149 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 1.1 MiB/s wr, 420 op/s 2024-09-10T10:02:37.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:02:36 smithi120 ceph-mon[34770]: pgmap v614: 129 pgs: 129 active+clean; 149 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 1.1 MiB/s wr, 420 op/s 2024-09-10T10:02:37.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:37.599+0000 7f871cc00640 1 -- 172.21.15.77:0/2001761867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8718103ff0 msgr2=0x7f87181043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:37.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:37.599+0000 7f871cc00640 1 --2- 172.21.15.77:0/2001761867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8718103ff0 0x7f87181043d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f8708009900 tx=0x7f870802a420 comp rx=0 tx=0).stop 2024-09-10T10:02:37.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:37.600+0000 7f871cc00640 1 -- 172.21.15.77:0/2001761867 shutdown_connections 2024-09-10T10:02:37.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:37.600+0000 7f871cc00640 1 --2- 172.21.15.77:0/2001761867 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8718109120 0x7f8718110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:37.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:37.600+0000 7f871cc00640 1 --2- 172.21.15.77:0/2001761867 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87181049a0 0x7f87181089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:37.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:37.600+0000 7f871cc00640 1 --2- 172.21.15.77:0/2001761867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8718103ff0 0x7f87181043d0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:37.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:37.600+0000 7f871cc00640 1 -- 172.21.15.77:0/2001761867 >> 172.21.15.77:0/2001761867 conn(0x7f87180ffa90 msgr2=0x7f8718101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:37.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:37.600+0000 7f871cc00640 1 -- 172.21.15.77:0/2001761867 shutdown_connections 2024-09-10T10:02:37.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:37.600+0000 7f871cc00640 1 -- 172.21.15.77:0/2001761867 wait complete. 2024-09-10T10:02:37.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:37.602+0000 7f871cc00640 1 Processor -- start 2024-09-10T10:02:37.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:37.602+0000 7f871cc00640 1 -- start start 2024-09-10T10:02:37.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:37.602+0000 7f871cc00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8718103ff0 0x7f871810f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:37.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:37.602+0000 7f871cc00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f87181049a0 0x7f871810f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:37.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:37.602+0000 7f871cc00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8718109120 0x7f87181aa9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:37.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:37.603+0000 7f871cc00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8718077430 con 0x7f8718103ff0 2024-09-10T10:02:37.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:37.603+0000 7f871cc00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f87180772b0 con 0x7f87181049a0 2024-09-10T10:02:37.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:37.603+0000 7f871cc00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f87180775b0 con 0x7f8718109120 2024-09-10T10:02:37.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:37.603+0000 7f8715400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f87181049a0 0x7f871810f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:37.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:37.603+0000 7f8715400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f87181049a0 0x7f871810f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:40670/0 (socket says 172.21.15.77:40670) 2024-09-10T10:02:37.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:37.603+0000 7f8715400640 1 -- 172.21.15.77:0/3686501283 learned_addr learned my addr 172.21.15.77:0/3686501283 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:02:37.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:37.603+0000 7f8716800640 1 --2- 172.21.15.77:0/3686501283 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8718109120 0x7f87181aa9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:37.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:37.603+0000 7f8715e00640 1 --2- 172.21.15.77:0/3686501283 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8718103ff0 0x7f871810f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:37.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:37.604+0000 7f8715400640 1 -- 172.21.15.77:0/3686501283 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8718109120 msgr2=0x7f87181aa9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:37.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:37.604+0000 7f8715400640 1 --2- 172.21.15.77:0/3686501283 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8718109120 0x7f87181aa9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:37.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:37.604+0000 7f8715400640 1 -- 172.21.15.77:0/3686501283 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8718103ff0 msgr2=0x7f871810f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:37.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:37.604+0000 7f8715400640 1 --2- 172.21.15.77:0/3686501283 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8718103ff0 0x7f871810f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:37.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:37.604+0000 7f8715400640 1 -- 172.21.15.77:0/3686501283 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87181aaee0 con 0x7f87181049a0 2024-09-10T10:02:37.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:37.604+0000 7f8715400640 1 --2- 172.21.15.77:0/3686501283 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f87181049a0 0x7f871810f750 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f870400d950 tx=0x7f870400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:37.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:37.605+0000 7f870ea00640 1 -- 172.21.15.77:0/3686501283 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8704017070 con 0x7f87181049a0 2024-09-10T10:02:37.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:37.605+0000 7f870ea00640 1 -- 172.21.15.77:0/3686501283 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8704009c30 con 0x7f87181049a0 2024-09-10T10:02:37.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:37.605+0000 7f870ea00640 1 -- 172.21.15.77:0/3686501283 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f870400f6b0 con 0x7f87181049a0 2024-09-10T10:02:37.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:37.605+0000 7f871cc00640 1 -- 172.21.15.77:0/3686501283 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87181ab170 con 0x7f87181049a0 2024-09-10T10:02:37.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:37.605+0000 7f871cc00640 1 -- 172.21.15.77:0/3686501283 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87181ab630 con 0x7f87181049a0 2024-09-10T10:02:37.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:37.607+0000 7f871cc00640 1 -- 172.21.15.77:0/3686501283 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87181043d0 con 0x7f87181049a0 2024-09-10T10:02:37.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:37.610+0000 7f870ea00640 1 -- 172.21.15.77:0/3686501283 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f87040109d0 con 0x7f87181049a0 2024-09-10T10:02:37.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:37.611+0000 7f870ea00640 1 --2- 172.21.15.77:0/3686501283 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f86fc0a2cd0 0x7f86fc0a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:37.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:37.611+0000 7f870ea00640 1 -- 172.21.15.77:0/3686501283 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f87040cef20 con 0x7f87181049a0 2024-09-10T10:02:37.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:37.612+0000 7f8715e00640 1 --2- 172.21.15.77:0/3686501283 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f86fc0a2cd0 0x7f86fc0a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:37.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:37.612+0000 7f8715e00640 1 --2- 172.21.15.77:0/3686501283 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f86fc0a2cd0 0x7f86fc0a5190 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f8708000c00 tx=0x7f8708005a00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:37.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:37.619+0000 7f870ea00640 1 -- 172.21.15.77:0/3686501283 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8704009de0 con 0x7f87181049a0 2024-09-10T10:02:37.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:37.953+0000 7f871cc00640 1 -- 172.21.15.77:0/3686501283 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f87181ab9f0 con 0x7f87181049a0 2024-09-10T10:02:37.956 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T10:02:37.956 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":35,"btime":"2024-09-10T10:01:05:922537+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3159289675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3159289675},{"type":"v1","addr":"172.21.15.77:6835","nonce":3159289675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1951962341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1951962341},{"type":"v1","addr":"172.21.15.77:6837","nonce":1951962341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":35,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:52:23.368252+0000","modified":"2024-09-10T10:01:05.706646+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24563,"mds_1":24524,"mds_2":24520,"mds_3":24502,"mds_4":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14487":{"gid":14487,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/4291121888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":4291121888},{"type":"v1","addr":"172.21.15.77:6839","nonce":4291121888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14499":{"gid":14499,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6841/445284194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":445284194},{"type":"v1","addr":"172.21.15.77:6841","nonce":445284194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1005054423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1005054423},{"type":"v1","addr":"172.21.15.134:6833","nonce":1005054423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6833/3716307744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":3716307744},{"type":"v1","addr":"172.21.15.120:6833","nonce":3716307744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/1319892544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1319892544},{"type":"v1","addr":"172.21.15.120:6835","nonce":1319892544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.134:6835/3293933699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3293933699},{"type":"v1","addr":"172.21.15.134:6835","nonce":3293933699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.134:6837/2461497091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2461497091},{"type":"v1","addr":"172.21.15.134:6837","nonce":2461497091}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":20,"addr":"172.21.15.134:6839/1965117680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1965117680},{"type":"v1","addr":"172.21.15.134:6839","nonce":1965117680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":20,"addr":"172.21.15.120:6837/3033280709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3033280709},{"type":"v1","addr":"172.21.15.120:6837","nonce":3033280709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.120:6839/1777710157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1777710157},{"type":"v1","addr":"172.21.15.120:6839","nonce":1777710157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24563,"qdb_cluster":[24496,24502,24520,24524,24563]},"id":1}]} 2024-09-10T10:02:37.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:37.955+0000 7f870ea00640 1 -- 172.21.15.77:0/3686501283 <== mon.2 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11989 (secure 0 0 0) 0x7f870409ba90 con 0x7f87181049a0 2024-09-10T10:02:37.956 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 35 2024-09-10T10:02:37.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:37.958+0000 7f871cc00640 1 -- 172.21.15.77:0/3686501283 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f86fc0a2cd0 msgr2=0x7f86fc0a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:37.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:37.958+0000 7f871cc00640 1 --2- 172.21.15.77:0/3686501283 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f86fc0a2cd0 0x7f86fc0a5190 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f8708000c00 tx=0x7f8708005a00 comp rx=0 tx=0).stop 2024-09-10T10:02:37.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:37.958+0000 7f871cc00640 1 -- 172.21.15.77:0/3686501283 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f87181049a0 msgr2=0x7f871810f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:37.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:37.958+0000 7f871cc00640 1 --2- 172.21.15.77:0/3686501283 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f87181049a0 0x7f871810f750 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f870400d950 tx=0x7f870400de10 comp rx=0 tx=0).stop 2024-09-10T10:02:37.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:37.958+0000 7f871cc00640 1 -- 172.21.15.77:0/3686501283 shutdown_connections 2024-09-10T10:02:37.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:37.958+0000 7f871cc00640 1 --2- 172.21.15.77:0/3686501283 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f86fc0a2cd0 0x7f86fc0a5190 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:37.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:37.958+0000 7f871cc00640 1 --2- 172.21.15.77:0/3686501283 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8718109120 0x7f87181aa9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:37.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:37.958+0000 7f871cc00640 1 --2- 172.21.15.77:0/3686501283 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f87181049a0 0x7f871810f750 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:37.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:37.958+0000 7f871cc00640 1 --2- 172.21.15.77:0/3686501283 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8718103ff0 0x7f871810f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:37.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:37.958+0000 7f871cc00640 1 -- 172.21.15.77:0/3686501283 >> 172.21.15.77:0/3686501283 conn(0x7f87180ffa90 msgr2=0x7f8718100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:37.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:37.958+0000 7f871cc00640 1 -- 172.21.15.77:0/3686501283 shutdown_connections 2024-09-10T10:02:37.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:37.958+0000 7f871cc00640 1 -- 172.21.15.77:0/3686501283 wait complete. 2024-09-10T10:02:39.114 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T10:02:39.187 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:02:38 smithi077 ceph-mon[30319]: pgmap v615: 129 pgs: 129 active+clean; 145 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 1.2 MiB/s wr, 440 op/s 2024-09-10T10:02:39.187 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:02:38 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/3686501283' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:02:39.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:02:38 smithi134 ceph-mon[35674]: pgmap v615: 129 pgs: 129 active+clean; 145 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 1.2 MiB/s wr, 440 op/s 2024-09-10T10:02:39.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:02:38 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/3686501283' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:02:39.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:02:38 smithi120 ceph-mon[34770]: pgmap v615: 129 pgs: 129 active+clean; 145 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 1.2 MiB/s wr, 440 op/s 2024-09-10T10:02:39.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:02:38 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/3686501283' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:02:39.566 INFO:tasks.workunit.client.0.smithi077.stdout:truncating to largest ever: 0x63e9b52 2024-09-10T10:02:39.614 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:02:40.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.347+0000 7f858e800640 1 -- 172.21.15.77:0/2791680084 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8588102360 msgr2=0x7f858810e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:40.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.347+0000 7f858e800640 1 --2- 172.21.15.77:0/2791680084 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8588102360 0x7f858810e890 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f8580009cd0 tx=0x7f858002a8e0 comp rx=0 tx=0).stop 2024-09-10T10:02:40.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.348+0000 7f858e800640 1 -- 172.21.15.77:0/2791680084 shutdown_connections 2024-09-10T10:02:40.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.348+0000 7f858e800640 1 --2- 172.21.15.77:0/2791680084 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8588102360 0x7f858810e890 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:40.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.348+0000 7f858e800640 1 --2- 172.21.15.77:0/2791680084 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85881019c0 0x7f8588101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:40.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.348+0000 7f858e800640 1 --2- 172.21.15.77:0/2791680084 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f85881079c0 0x7f8588107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:40.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.348+0000 7f858e800640 1 -- 172.21.15.77:0/2791680084 >> 172.21.15.77:0/2791680084 conn(0x7f85880fd740 msgr2=0x7f85880ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:40.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.348+0000 7f858e800640 1 -- 172.21.15.77:0/2791680084 shutdown_connections 2024-09-10T10:02:40.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.348+0000 7f858e800640 1 -- 172.21.15.77:0/2791680084 wait complete. 2024-09-10T10:02:40.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.349+0000 7f858e800640 1 Processor -- start 2024-09-10T10:02:40.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.349+0000 7f858e800640 1 -- start start 2024-09-10T10:02:40.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.349+0000 7f858e800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f85881019c0 0x7f85880744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:40.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.349+0000 7f858e800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8588102360 0x7f8588074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:40.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.349+0000 7f858e800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f85881079c0 0x7f8588074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:40.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.349+0000 7f858e800640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8588111dc0 con 0x7f85881079c0 2024-09-10T10:02:40.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.349+0000 7f858e800640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8588111c40 con 0x7f85881019c0 2024-09-10T10:02:40.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.349+0000 7f858e800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8588111f40 con 0x7f8588102360 2024-09-10T10:02:40.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.350+0000 7f858d400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f85881019c0 0x7f85880744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:40.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.350+0000 7f858d400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f85881019c0 0x7f85880744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:40690/0 (socket says 172.21.15.77:40690) 2024-09-10T10:02:40.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.350+0000 7f858d400640 1 -- 172.21.15.77:0/3919301067 learned_addr learned my addr 172.21.15.77:0/3919301067 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:02:40.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.350+0000 7f858ca00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8588102360 0x7f8588074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:40.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.350+0000 7f858ca00640 1 -- 172.21.15.77:0/3919301067 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f85881019c0 msgr2=0x7f85880744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:40.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.350+0000 7f858ca00640 1 --2- 172.21.15.77:0/3919301067 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f85881019c0 0x7f85880744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:40.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.350+0000 7f858ca00640 1 -- 172.21.15.77:0/3919301067 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f85881079c0 msgr2=0x7f8588074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:40.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.350+0000 7f858de00640 1 --2- 172.21.15.77:0/3919301067 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f85881079c0 0x7f8588074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:40.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.350+0000 7f858ca00640 1 --2- 172.21.15.77:0/3919301067 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f85881079c0 0x7f8588074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:40.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.350+0000 7f858ca00640 1 -- 172.21.15.77:0/3919301067 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85881a6660 con 0x7f8588102360 2024-09-10T10:02:40.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.350+0000 7f858d400640 1 --2- 172.21.15.77:0/3919301067 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f85881019c0 0x7f85880744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:02:40.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.351+0000 7f858de00640 1 --2- 172.21.15.77:0/3919301067 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f85881079c0 0x7f8588074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:02:40.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.351+0000 7f858ca00640 1 --2- 172.21.15.77:0/3919301067 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8588102360 0x7f8588074a30 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f857400e9f0 tx=0x7f857400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:40.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.351+0000 7f8586000640 1 -- 172.21.15.77:0/3919301067 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f857400ccc0 con 0x7f8588102360 2024-09-10T10:02:40.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.351+0000 7f8586000640 1 -- 172.21.15.77:0/3919301067 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8574010910 con 0x7f8588102360 2024-09-10T10:02:40.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.351+0000 7f8586000640 1 -- 172.21.15.77:0/3919301067 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8574018960 con 0x7f8588102360 2024-09-10T10:02:40.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.351+0000 7f858e800640 1 -- 172.21.15.77:0/3919301067 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85881a6950 con 0x7f8588102360 2024-09-10T10:02:40.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.351+0000 7f858e800640 1 -- 172.21.15.77:0/3919301067 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85881a6ee0 con 0x7f8588102360 2024-09-10T10:02:40.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.354+0000 7f8592a27740 1 Processor -- start 2024-09-10T10:02:40.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.355+0000 7f8592a27740 1 -- start start 2024-09-10T10:02:40.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.355+0000 7f8586000640 1 -- 172.21.15.77:0/3919301067 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8574010ac0 con 0x7f8588102360 2024-09-10T10:02:40.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.355+0000 7f8592a27740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654cba17e40 0x5654cba18220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:40.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.355+0000 7f8592a27740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5654cba18760 0x5654cba22ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:40.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.355+0000 7f8592a27740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654cba23020 0x5654cba25410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:40.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.355+0000 7f8592a27740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5654cb97e3e0 con 0x5654cba18760 2024-09-10T10:02:40.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.355+0000 7f8592a27740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5654cb925d80 con 0x5654cba17e40 2024-09-10T10:02:40.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.355+0000 7f8592a27740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5654cb97a3a0 con 0x5654cba23020 2024-09-10T10:02:40.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.355+0000 7f858de00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654cba17e40 0x5654cba18220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:40.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.355+0000 7f858de00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654cba17e40 0x5654cba18220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:40702/0 (socket says 172.21.15.77:40702) 2024-09-10T10:02:40.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.355+0000 7f858de00640 1 -- 172.21.15.77:0/3335589256 learned_addr learned my addr 172.21.15.77:0/3335589256 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:02:40.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.355+0000 7f858d400640 1 --2- 172.21.15.77:0/3335589256 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5654cba18760 0x5654cba22ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:40.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.355+0000 7f858ca00640 1 --2- 172.21.15.77:0/3335589256 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654cba23020 0x5654cba25410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:40.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.356+0000 7f858ca00640 1 -- 172.21.15.77:0/3335589256 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654cba17e40 msgr2=0x5654cba18220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:40.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.356+0000 7f858ca00640 1 --2- 172.21.15.77:0/3335589256 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654cba17e40 0x5654cba18220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:40.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.356+0000 7f858ca00640 1 -- 172.21.15.77:0/3335589256 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5654cba18760 msgr2=0x5654cba22ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:40.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.356+0000 7f858ca00640 1 --2- 172.21.15.77:0/3335589256 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5654cba18760 0x5654cba22ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:40.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.356+0000 7f858ca00640 1 -- 172.21.15.77:0/3335589256 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654cb944ae0 con 0x5654cba23020 2024-09-10T10:02:40.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.357+0000 7f858ca00640 1 --2- 172.21.15.77:0/3335589256 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654cba23020 0x5654cba25410 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f85740ccea0 tx=0x7f85740d7670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a9408318efc61c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:40.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.357+0000 7f8586000640 1 --2- 172.21.15.77:0/3919301067 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f85700a2c00 0x7f85700a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:40.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.357+0000 7f857a600640 1 -- 172.21.15.77:0/3335589256 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85740de070 con 0x5654cba23020 2024-09-10T10:02:40.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.357+0000 7f857a600640 1 -- 172.21.15.77:0/3335589256 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85740bd580 con 0x5654cba23020 2024-09-10T10:02:40.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.357+0000 7f857a600640 1 -- 172.21.15.77:0/3335589256 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f857409b5c0 con 0x5654cba23020 2024-09-10T10:02:40.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.357+0000 7f8586000640 1 -- 172.21.15.77:0/3919301067 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f85740cf850 con 0x7f8588102360 2024-09-10T10:02:40.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.357+0000 7f858de00640 1 --2- 172.21.15.77:0/3919301067 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f85700a2c00 0x7f85700a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:40.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.358+0000 7f8592a27740 1 -- 172.21.15.77:0/3335589256 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654cba23020 msgr2=0x5654cba25410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:40.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.358+0000 7f858de00640 1 --2- 172.21.15.77:0/3919301067 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f85700a2c00 0x7f85700a50c0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f8580004af0 tx=0x7f8580005dc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:40.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.358+0000 7f8592a27740 1 --2- 172.21.15.77:0/3335589256 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654cba23020 0x5654cba25410 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f85740ccea0 tx=0x7f85740d7670 comp rx=0 tx=0).stop 2024-09-10T10:02:40.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.358+0000 7f8592a27740 1 -- 172.21.15.77:0/3335589256 shutdown_connections 2024-09-10T10:02:40.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.358+0000 7f8592a27740 1 --2- 172.21.15.77:0/3335589256 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654cba23020 0x5654cba25410 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:40.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.358+0000 7f8592a27740 1 --2- 172.21.15.77:0/3335589256 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5654cba18760 0x5654cba22ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:40.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.358+0000 7f8592a27740 1 --2- 172.21.15.77:0/3335589256 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654cba17e40 0x5654cba18220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:40.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.358+0000 7f8592a27740 1 -- 172.21.15.77:0/3335589256 >> 172.21.15.77:0/3335589256 conn(0x5654cba143d0 msgr2=0x5654cba147b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:40.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.359+0000 7f8592a27740 1 -- 172.21.15.77:0/3335589256 shutdown_connections 2024-09-10T10:02:40.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.359+0000 7f8592a27740 1 -- 172.21.15.77:0/3335589256 wait complete. 2024-09-10T10:02:40.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.359+0000 7f8592a27740 1 -- start start 2024-09-10T10:02:40.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.359+0000 7f8592a27740 1 Processor -- start 2024-09-10T10:02:40.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.360+0000 7f8592a27740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654cba17e40 0x5654cba1fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:40.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.360+0000 7f8592a27740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5654cba18760 0x5654cba203b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:40.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.360+0000 7f8592a27740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654cba23020 0x5654cba208f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:40.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.360+0000 7f8592a27740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5654cba25bf0 con 0x5654cba18760 2024-09-10T10:02:40.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.360+0000 7f8592a27740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5654cb9390f0 con 0x5654cba17e40 2024-09-10T10:02:40.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.360+0000 7f8592a27740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5654cba25d70 con 0x5654cba23020 2024-09-10T10:02:40.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.360+0000 7f8592a27740 10 client.0 _ll_register_callbacks cb 0x5654cb920bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:02:40.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.360+0000 7f858d400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654cba17e40 0x5654cba1fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:40.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.360+0000 7f858d400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654cba17e40 0x5654cba1fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:40706/0 (socket says 172.21.15.77:40706) 2024-09-10T10:02:40.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.360+0000 7f858d400640 1 -- 172.21.15.77:0/2530836605 learned_addr learned my addr 172.21.15.77:0/2530836605 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:02:40.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.360+0000 7f858de00640 1 --2- 172.21.15.77:0/2530836605 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654cba23020 0x5654cba208f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:40.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.360+0000 7f858ca00640 1 --2- 172.21.15.77:0/2530836605 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5654cba18760 0x5654cba203b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:40.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.361+0000 7f858d400640 1 -- 172.21.15.77:0/2530836605 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654cba23020 msgr2=0x5654cba208f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:40.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.361+0000 7f858d400640 1 --2- 172.21.15.77:0/2530836605 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654cba23020 0x5654cba208f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:40.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.361+0000 7f858d400640 1 -- 172.21.15.77:0/2530836605 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5654cba18760 msgr2=0x5654cba203b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:40.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.361+0000 7f858d400640 1 --2- 172.21.15.77:0/2530836605 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5654cba18760 0x5654cba203b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:40.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.361+0000 7f858d400640 1 -- 172.21.15.77:0/2530836605 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654cb6ef3c0 con 0x5654cba17e40 2024-09-10T10:02:40.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.361+0000 7f858d400640 1 --2- 172.21.15.77:0/2530836605 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654cba17e40 0x5654cba1fe70 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f857c010d30 tx=0x7f857c00f7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e4ef9b4ec9d9720 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:40.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.361+0000 7f856ea00640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T10:02:40.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.361+0000 7f856ea00640 1 -- 172.21.15.77:0/2530836605 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f857c005020 con 0x5654cba17e40 2024-09-10T10:02:40.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.361+0000 7f8592a27740 1 -- 172.21.15.77:0/2530836605 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85740bd580 con 0x5654cba17e40 2024-09-10T10:02:40.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.361+0000 7f856ea00640 1 -- 172.21.15.77:0/2530836605 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f857c0051c0 con 0x5654cba17e40 2024-09-10T10:02:40.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.361+0000 7f856ea00640 1 -- 172.21.15.77:0/2530836605 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f857c024050 con 0x5654cba17e40 2024-09-10T10:02:40.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.362+0000 7f8592a27740 10 client.27002 fetch_fsmap learned FSMap version 35 2024-09-10T10:02:40.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.362+0000 7f8592a27740 1 -- 172.21.15.77:0/2530836605 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5654cba31cc0 con 0x5654cba17e40 2024-09-10T10:02:40.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.362+0000 7f856ea00640 1 -- 172.21.15.77:0/2530836605 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85740bd580 con 0x5654cba17e40 2024-09-10T10:02:40.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.362+0000 7f856ea00640 1 -- 172.21.15.77:0/2530836605 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f857c00e7f0 con 0x5654cba17e40 2024-09-10T10:02:40.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.362+0000 7f8592a27740 10 client.27002 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T10:02:40.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.362+0000 7f8592a27740 10 client.27002 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-10T10:02:40.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.362+0000 7f8592a27740 20 client.27002 populate_metadata read hostname 'smithi077' 2024-09-10T10:02:40.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.362+0000 7f8592a27740 1 --2- 172.21.15.77:0/2530836605 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x5654cba330d0 0x5654cba35550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:40.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.362+0000 7f8592a27740 4 client.27002 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:02:40.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.362+0000 7f8592a27740 1 -- 172.21.15.77:0/2530836605 --> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5654cb98b950 con 0x5654cba330d0 2024-09-10T10:02:40.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.363+0000 7f858ca00640 1 --2- 172.21.15.77:0/2530836605 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x5654cba330d0 0x5654cba35550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:40.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.363+0000 7f858ca00640 1 --2- 172.21.15.77:0/2530836605 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x5654cba330d0 0x5654cba35550 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f942801014c50670 server_cookie=1c34180b4f5d3468 in_seq=0 out_seq=0 2024-09-10T10:02:40.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.363+0000 7f856ea00640 10 client.27002 ms_handle_connect on v2:172.21.15.134:6834/3293933699 2024-09-10T10:02:40.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.364+0000 7f856ea00640 1 -- 172.21.15.77:0/2530836605 <== mds.4 v2:172.21.15.134:6834/3293933699 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5654cb98b950 con 0x5654cba330d0 2024-09-10T10:02:40.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.364+0000 7f856ea00640 10 client.27002 handle_command_reply: tid=0 2024-09-10T10:02:40.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.364+0000 7f8592a27740 1 client.27002 shutdown 2024-09-10T10:02:40.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.364+0000 7f8592a27740 1 -- 172.21.15.77:0/2530836605 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654cba17e40 msgr2=0x5654cba1fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:40.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.364+0000 7f8592a27740 1 --2- 172.21.15.77:0/2530836605 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654cba17e40 0x5654cba1fe70 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f857c010d30 tx=0x7f857c00f7f0 comp rx=0 tx=0).stop 2024-09-10T10:02:40.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.364+0000 7f8592a27740 1 -- 172.21.15.77:0/2530836605 shutdown_connections 2024-09-10T10:02:40.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.364+0000 7f8592a27740 1 --2- 172.21.15.77:0/2530836605 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x5654cba330d0 0x5654cba35550 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:40.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.364+0000 7f8592a27740 1 --2- 172.21.15.77:0/2530836605 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654cba23020 0x5654cba208f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:40.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.364+0000 7f8592a27740 1 --2- 172.21.15.77:0/2530836605 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5654cba18760 0x5654cba203b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:40.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.364+0000 7f856ea00640 0 client.27002 ms_handle_reset on v2:172.21.15.134:6834/3293933699 2024-09-10T10:02:40.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.364+0000 7f8592a27740 1 --2- 172.21.15.77:0/2530836605 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654cba17e40 0x5654cba1fe70 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:40.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.364+0000 7f8592a27740 1 -- 172.21.15.77:0/2530836605 >> 172.21.15.77:0/2530836605 conn(0x5654cba143d0 msgr2=0x5654cba1a100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:40.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.365+0000 7f8592a27740 1 -- 172.21.15.77:0/2530836605 shutdown_connections 2024-09-10T10:02:40.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.365+0000 7f8592a27740 1 -- 172.21.15.77:0/2530836605 wait complete. 2024-09-10T10:02:40.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.365+0000 7f8592a27740 20 client.27002 trim_cache size 0 max 16384 2024-09-10T10:02:40.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.384+0000 7f8592a27740 1 Processor -- start 2024-09-10T10:02:40.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.384+0000 7f8592a27740 1 -- start start 2024-09-10T10:02:40.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.385+0000 7f8592a27740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654cba17e40 0x5654cba21de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:40.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.385+0000 7f8592a27740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654cba18760 0x5654cba3ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:40.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.385+0000 7f8592a27740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5654cba23020 0x5654cba3b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:40.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.385+0000 7f8592a27740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f85881b0ed0 con 0x5654cba23020 2024-09-10T10:02:40.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.385+0000 7f8592a27740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f85881b2a10 con 0x5654cba17e40 2024-09-10T10:02:40.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.385+0000 7f8592a27740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f85881b15a0 con 0x5654cba18760 2024-09-10T10:02:40.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.385+0000 7f858de00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5654cba23020 0x5654cba3b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:40.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.385+0000 7f858d400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654cba17e40 0x5654cba21de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:40.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.385+0000 7f858d400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654cba17e40 0x5654cba21de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:40720/0 (socket says 172.21.15.77:40720) 2024-09-10T10:02:40.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.385+0000 7f858d400640 1 -- 172.21.15.77:0/21322812 learned_addr learned my addr 172.21.15.77:0/21322812 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:02:40.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.385+0000 7f858ca00640 1 --2- 172.21.15.77:0/21322812 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654cba18760 0x5654cba3ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:40.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.386+0000 7f858d400640 1 -- 172.21.15.77:0/21322812 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654cba18760 msgr2=0x5654cba3ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:40.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.386+0000 7f858d400640 1 --2- 172.21.15.77:0/21322812 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654cba18760 0x5654cba3ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:40.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.386+0000 7f858d400640 1 -- 172.21.15.77:0/21322812 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5654cba23020 msgr2=0x5654cba3b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:40.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.386+0000 7f858d400640 1 --2- 172.21.15.77:0/21322812 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5654cba23020 0x5654cba3b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:40.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.386+0000 7f858d400640 1 -- 172.21.15.77:0/21322812 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654cba14200 con 0x5654cba17e40 2024-09-10T10:02:40.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.386+0000 7f858d400640 1 --2- 172.21.15.77:0/21322812 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654cba17e40 0x5654cba21de0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f857c0107f0 tx=0x7f857c00ab40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b64e648e5020c4e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:40.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.386+0000 7f8579c00640 1 -- 172.21.15.77:0/21322812 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f857c02d070 con 0x5654cba17e40 2024-09-10T10:02:40.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.386+0000 7f8579c00640 1 -- 172.21.15.77:0/21322812 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f857c005030 con 0x5654cba17e40 2024-09-10T10:02:40.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.386+0000 7f8579c00640 1 -- 172.21.15.77:0/21322812 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f857c0048c0 con 0x5654cba17e40 2024-09-10T10:02:40.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.387+0000 7f8592a27740 1 -- 172.21.15.77:0/21322812 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654cba17e40 msgr2=0x5654cba21de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:40.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.387+0000 7f8592a27740 1 --2- 172.21.15.77:0/21322812 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654cba17e40 0x5654cba21de0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f857c0107f0 tx=0x7f857c00ab40 comp rx=0 tx=0).stop 2024-09-10T10:02:40.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.387+0000 7f8592a27740 1 -- 172.21.15.77:0/21322812 shutdown_connections 2024-09-10T10:02:40.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.387+0000 7f8592a27740 1 --2- 172.21.15.77:0/21322812 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5654cba23020 0x5654cba3b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:40.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.387+0000 7f8592a27740 1 --2- 172.21.15.77:0/21322812 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654cba18760 0x5654cba3ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:40.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.387+0000 7f8592a27740 1 --2- 172.21.15.77:0/21322812 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654cba17e40 0x5654cba21de0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:40.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.387+0000 7f8592a27740 1 -- 172.21.15.77:0/21322812 >> 172.21.15.77:0/21322812 conn(0x5654cba143d0 msgr2=0x5654cba216d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:40.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.387+0000 7f8592a27740 1 -- 172.21.15.77:0/21322812 shutdown_connections 2024-09-10T10:02:40.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.387+0000 7f8592a27740 1 -- 172.21.15.77:0/21322812 wait complete. 2024-09-10T10:02:40.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.388+0000 7f8592a27740 1 -- start start 2024-09-10T10:02:40.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.388+0000 7f8592a27740 1 Processor -- start 2024-09-10T10:02:40.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.388+0000 7f8592a27740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5654cba17e40 0x5654cba2b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:40.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.388+0000 7f8592a27740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654cba18760 0x5654cba1ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:40.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.388+0000 7f8592a27740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654cba23020 0x5654cba1cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:40.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.388+0000 7f8592a27740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f85881b2340 con 0x5654cba17e40 2024-09-10T10:02:40.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.389+0000 7f8592a27740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f85881b1c70 con 0x5654cba23020 2024-09-10T10:02:40.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.389+0000 7f8592a27740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f85881aeeb0 con 0x5654cba18760 2024-09-10T10:02:40.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.389+0000 7f8592a27740 10 client.0 _ll_register_callbacks cb 0x7f857c00ac20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:02:40.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.389+0000 7f858ca00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654cba18760 0x5654cba1ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:40.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.389+0000 7f858ca00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654cba18760 0x5654cba1ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:54756/0 (socket says 172.21.15.77:54756) 2024-09-10T10:02:40.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.389+0000 7f858ca00640 1 -- 172.21.15.77:0/3169167891 learned_addr learned my addr 172.21.15.77:0/3169167891 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:02:40.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.389+0000 7f858de00640 1 --2- 172.21.15.77:0/3169167891 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654cba23020 0x5654cba1cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:40.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.389+0000 7f858d400640 1 --2- 172.21.15.77:0/3169167891 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5654cba17e40 0x5654cba2b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:40.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.389+0000 7f858ca00640 1 -- 172.21.15.77:0/3169167891 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654cba23020 msgr2=0x5654cba1cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:40.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.389+0000 7f858ca00640 1 --2- 172.21.15.77:0/3169167891 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654cba23020 0x5654cba1cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:40.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.389+0000 7f858ca00640 1 -- 172.21.15.77:0/3169167891 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5654cba17e40 msgr2=0x5654cba2b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:40.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.389+0000 7f858ca00640 1 --2- 172.21.15.77:0/3169167891 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5654cba17e40 0x5654cba2b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:40.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.389+0000 7f858ca00640 1 -- 172.21.15.77:0/3169167891 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654cba1d6b0 con 0x5654cba18760 2024-09-10T10:02:40.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.389+0000 7f858d400640 1 --2- 172.21.15.77:0/3169167891 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5654cba17e40 0x5654cba2b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:02:40.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.390+0000 7f858ca00640 1 --2- 172.21.15.77:0/3169167891 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654cba18760 0x5654cba1ca80 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f857409b960 tx=0x7f857409b7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae1b44f4482dcbf5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:40.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.390+0000 7f856ea00640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-10T10:02:40.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.390+0000 7f856ea00640 1 -- 172.21.15.77:0/3169167891 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85740c7a00 con 0x5654cba18760 2024-09-10T10:02:40.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.390+0000 7f856ea00640 1 -- 172.21.15.77:0/3169167891 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f85740ccf80 con 0x5654cba18760 2024-09-10T10:02:40.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.390+0000 7f8592a27740 1 -- 172.21.15.77:0/3169167891 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f857c005030 con 0x5654cba18760 2024-09-10T10:02:40.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.390+0000 7f856ea00640 1 -- 172.21.15.77:0/3169167891 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f85740c5b40 con 0x5654cba18760 2024-09-10T10:02:40.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.390+0000 7f8592a27740 10 client.26968 fetch_fsmap learned FSMap version 35 2024-09-10T10:02:40.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.390+0000 7f8592a27740 1 -- 172.21.15.77:0/3169167891 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5654cba9dd00 con 0x5654cba18760 2024-09-10T10:02:40.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.390+0000 7f856ea00640 1 -- 172.21.15.77:0/3169167891 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f857c005030 con 0x5654cba18760 2024-09-10T10:02:40.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.391+0000 7f856ea00640 1 -- 172.21.15.77:0/3169167891 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f85740982d0 con 0x5654cba18760 2024-09-10T10:02:40.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.391+0000 7f8592a27740 10 client.26968 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T10:02:40.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.391+0000 7f8592a27740 10 client.26968 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-10T10:02:40.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.391+0000 7f8592a27740 20 client.26968 populate_metadata read hostname 'smithi077' 2024-09-10T10:02:40.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.391+0000 7f8592a27740 1 --2- 172.21.15.77:0/3169167891 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x5654cba9df90 0x5654cbaa0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:40.395 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.391+0000 7f8592a27740 4 client.26968 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:02:40.395 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.391+0000 7f8592a27740 1 -- 172.21.15.77:0/3169167891 --> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5654cb985e60 con 0x5654cba9df90 2024-09-10T10:02:40.395 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.391+0000 7f858d400640 1 --2- 172.21.15.77:0/3169167891 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x5654cba9df90 0x5654cbaa0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:40.395 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.391+0000 7f858d400640 1 --2- 172.21.15.77:0/3169167891 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x5654cba9df90 0x5654cbaa0380 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=44e1bf8b6e470e14 server_cookie=68f76b3b7366259d in_seq=0 out_seq=0 2024-09-10T10:02:40.395 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.391+0000 7f856ea00640 10 client.26968 ms_handle_connect on v2:172.21.15.134:6834/3293933699 2024-09-10T10:02:40.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.392+0000 7f856ea00640 1 -- 172.21.15.77:0/3169167891 <== mds.4 v2:172.21.15.134:6834/3293933699 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5654cb985e60 con 0x5654cba9df90 2024-09-10T10:02:40.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.392+0000 7f856ea00640 10 client.26968 handle_command_reply: tid=0 2024-09-10T10:02:40.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.392+0000 7f8592a27740 1 client.26968 shutdown 2024-09-10T10:02:40.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.392+0000 7f8592a27740 1 -- 172.21.15.77:0/3169167891 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654cba18760 msgr2=0x5654cba1ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:40.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.392+0000 7f8592a27740 1 --2- 172.21.15.77:0/3169167891 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654cba18760 0x5654cba1ca80 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f857409b960 tx=0x7f857409b7a0 comp rx=0 tx=0).stop 2024-09-10T10:02:40.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.392+0000 7f8592a27740 1 -- 172.21.15.77:0/3169167891 shutdown_connections 2024-09-10T10:02:40.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.392+0000 7f8592a27740 1 --2- 172.21.15.77:0/3169167891 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x5654cba9df90 0x5654cbaa0380 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:40.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.392+0000 7f8592a27740 1 --2- 172.21.15.77:0/3169167891 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654cba23020 0x5654cba1cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:40.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.392+0000 7f8592a27740 1 --2- 172.21.15.77:0/3169167891 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654cba18760 0x5654cba1ca80 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:40.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.392+0000 7f8592a27740 1 --2- 172.21.15.77:0/3169167891 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5654cba17e40 0x5654cba2b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:40.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.392+0000 7f856ea00640 0 client.26968 ms_handle_reset on v2:172.21.15.134:6834/3293933699 2024-09-10T10:02:40.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.392+0000 7f8592a27740 1 -- 172.21.15.77:0/3169167891 >> 172.21.15.77:0/3169167891 conn(0x5654cba143d0 msgr2=0x5654cba1a100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:40.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.392+0000 7f8592a27740 1 -- 172.21.15.77:0/3169167891 shutdown_connections 2024-09-10T10:02:40.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.392+0000 7f8592a27740 1 -- 172.21.15.77:0/3169167891 wait complete. 2024-09-10T10:02:40.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.392+0000 7f8592a27740 20 client.26968 trim_cache size 0 max 16384 2024-09-10T10:02:40.397 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-10T10:02:40.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.393+0000 7f858e800640 1 -- 172.21.15.77:0/3919301067 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f85700a2c00 msgr2=0x7f85700a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:40.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.393+0000 7f858e800640 1 --2- 172.21.15.77:0/3919301067 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f85700a2c00 0x7f85700a50c0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f8580004af0 tx=0x7f8580005dc0 comp rx=0 tx=0).stop 2024-09-10T10:02:40.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.393+0000 7f858e800640 1 -- 172.21.15.77:0/3919301067 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8588102360 msgr2=0x7f8588074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:40.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.393+0000 7f858e800640 1 --2- 172.21.15.77:0/3919301067 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8588102360 0x7f8588074a30 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f857400e9f0 tx=0x7f857400eeb0 comp rx=0 tx=0).stop 2024-09-10T10:02:40.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.393+0000 7f858e800640 1 -- 172.21.15.77:0/3919301067 shutdown_connections 2024-09-10T10:02:40.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.393+0000 7f858e800640 1 --2- 172.21.15.77:0/3919301067 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f85700a2c00 0x7f85700a50c0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:40.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.393+0000 7f858e800640 1 --2- 172.21.15.77:0/3919301067 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f85881079c0 0x7f8588074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:40.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.393+0000 7f858e800640 1 --2- 172.21.15.77:0/3919301067 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8588102360 0x7f8588074a30 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:40.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.393+0000 7f858e800640 1 --2- 172.21.15.77:0/3919301067 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f85881019c0 0x7f85880744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:40.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.393+0000 7f858e800640 1 -- 172.21.15.77:0/3919301067 >> 172.21.15.77:0/3919301067 conn(0x7f85880fd740 msgr2=0x7f85880fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:40.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.393+0000 7f858e800640 1 -- 172.21.15.77:0/3919301067 shutdown_connections 2024-09-10T10:02:40.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:40.393+0000 7f858e800640 1 -- 172.21.15.77:0/3919301067 wait complete. 2024-09-10T10:02:41.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:02:40 smithi077 ceph-mon[30319]: pgmap v616: 129 pgs: 129 active+clean; 139 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.4 MiB/s wr, 530 op/s 2024-09-10T10:02:41.361 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T10:02:41.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:02:40 smithi134 ceph-mon[35674]: pgmap v616: 129 pgs: 129 active+clean; 139 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.4 MiB/s wr, 530 op/s 2024-09-10T10:02:41.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:02:40 smithi120 ceph-mon[34770]: pgmap v616: 129 pgs: 129 active+clean; 139 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.4 MiB/s wr, 530 op/s 2024-09-10T10:02:41.853 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:02:42.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.630+0000 7f9125600640 1 -- 172.21.15.77:0/2130891801 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f91201049c0 msgr2=0x7f9120108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:42.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.630+0000 7f9125600640 1 --2- 172.21.15.77:0/2130891801 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f91201049c0 0x7f9120108a10 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f9108009900 tx=0x7f910802a3f0 comp rx=0 tx=0).stop 2024-09-10T10:02:42.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.630+0000 7f9125600640 1 -- 172.21.15.77:0/2130891801 shutdown_connections 2024-09-10T10:02:42.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.630+0000 7f9125600640 1 --2- 172.21.15.77:0/2130891801 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9120109140 0x7f9120110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:42.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.630+0000 7f9125600640 1 --2- 172.21.15.77:0/2130891801 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f91201049c0 0x7f9120108a10 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:42.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.630+0000 7f9125600640 1 --2- 172.21.15.77:0/2130891801 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9120104010 0x7f91201043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:42.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.630+0000 7f9125600640 1 -- 172.21.15.77:0/2130891801 >> 172.21.15.77:0/2130891801 conn(0x7f91200ffa50 msgr2=0x7f9120101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:42.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.630+0000 7f9125600640 1 -- 172.21.15.77:0/2130891801 shutdown_connections 2024-09-10T10:02:42.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.630+0000 7f9125600640 1 -- 172.21.15.77:0/2130891801 wait complete. 2024-09-10T10:02:42.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.632+0000 7f9125600640 1 Processor -- start 2024-09-10T10:02:42.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.632+0000 7f9125600640 1 -- start start 2024-09-10T10:02:42.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.632+0000 7f9125600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9120104010 0x7f912019a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:42.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.632+0000 7f9125600640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f91201049c0 0x7f912019ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:42.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.632+0000 7f9125600640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9120109140 0x7f912019efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:42.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.632+0000 7f9125600640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9120114190 con 0x7f9120109140 2024-09-10T10:02:42.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.632+0000 7f9125600640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9120114010 con 0x7f91201049c0 2024-09-10T10:02:42.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.632+0000 7f9125600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9120114310 con 0x7f9120104010 2024-09-10T10:02:42.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.632+0000 7f9117e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f91201049c0 0x7f912019ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:42.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.633+0000 7f9117e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f91201049c0 0x7f912019ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:40752/0 (socket says 172.21.15.77:40752) 2024-09-10T10:02:42.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.633+0000 7f9117e00640 1 -- 172.21.15.77:0/2842683586 learned_addr learned my addr 172.21.15.77:0/2842683586 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:02:42.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.633+0000 7f911ea00640 1 --2- 172.21.15.77:0/2842683586 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9120104010 0x7f912019a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:42.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.633+0000 7f911f400640 1 --2- 172.21.15.77:0/2842683586 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9120109140 0x7f912019efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:42.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.633+0000 7f911ea00640 1 -- 172.21.15.77:0/2842683586 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f91201049c0 msgr2=0x7f912019ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:42.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.633+0000 7f911ea00640 1 --2- 172.21.15.77:0/2842683586 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f91201049c0 0x7f912019ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:42.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.633+0000 7f911ea00640 1 -- 172.21.15.77:0/2842683586 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9120109140 msgr2=0x7f912019efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:42.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.633+0000 7f911ea00640 1 --2- 172.21.15.77:0/2842683586 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9120109140 0x7f912019efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:42.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.633+0000 7f911ea00640 1 -- 172.21.15.77:0/2842683586 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f912019f6a0 con 0x7f9120104010 2024-09-10T10:02:42.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.633+0000 7f9117e00640 1 --2- 172.21.15.77:0/2842683586 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f91201049c0 0x7f912019ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:02:42.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.634+0000 7f911ea00640 1 --2- 172.21.15.77:0/2842683586 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9120104010 0x7f912019a6f0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f911000e9f0 tx=0x7f911000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:42.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.634+0000 7f9117400640 1 -- 172.21.15.77:0/2842683586 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f911000cd80 con 0x7f9120104010 2024-09-10T10:02:42.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.634+0000 7f9117400640 1 -- 172.21.15.77:0/2842683586 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91100035f0 con 0x7f9120104010 2024-09-10T10:02:42.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.634+0000 7f9125600640 1 -- 172.21.15.77:0/2842683586 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f912019f990 con 0x7f9120104010 2024-09-10T10:02:42.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.634+0000 7f9117400640 1 -- 172.21.15.77:0/2842683586 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9110010690 con 0x7f9120104010 2024-09-10T10:02:42.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.634+0000 7f9125600640 1 -- 172.21.15.77:0/2842683586 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91201a7270 con 0x7f9120104010 2024-09-10T10:02:42.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.638+0000 7f9117400640 1 -- 172.21.15.77:0/2842683586 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9110004020 con 0x7f9120104010 2024-09-10T10:02:42.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.638+0000 7f9128019740 1 Processor -- start 2024-09-10T10:02:42.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.638+0000 7f9128019740 1 -- start start 2024-09-10T10:02:42.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.639+0000 7f9128019740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610527bcc70 0x5610527bd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:42.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.639+0000 7f9128019740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610527bd590 0x5610527c7910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:42.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.639+0000 7f9128019740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610527c7e50 0x5610527ca240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:42.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.639+0000 7f9128019740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5610527233e0 con 0x5610527c7e50 2024-09-10T10:02:42.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.639+0000 7f9128019740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5610526cad80 con 0x5610527bcc70 2024-09-10T10:02:42.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.639+0000 7f9128019740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56105271f3a0 con 0x5610527bd590 2024-09-10T10:02:42.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.639+0000 7f911ea00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610527bd590 0x5610527c7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:42.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.639+0000 7f911f400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610527bcc70 0x5610527bd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:42.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.639+0000 7f911f400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610527bcc70 0x5610527bd050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:40758/0 (socket says 172.21.15.77:40758) 2024-09-10T10:02:42.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.639+0000 7f911f400640 1 -- 172.21.15.77:0/1015517241 learned_addr learned my addr 172.21.15.77:0/1015517241 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:02:42.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.639+0000 7f9117e00640 1 --2- 172.21.15.77:0/1015517241 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610527c7e50 0x5610527ca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:42.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.640+0000 7f911f400640 1 -- 172.21.15.77:0/1015517241 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610527bd590 msgr2=0x5610527c7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:42.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.640+0000 7f911f400640 1 --2- 172.21.15.77:0/1015517241 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610527bd590 0x5610527c7910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:42.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.640+0000 7f911f400640 1 -- 172.21.15.77:0/1015517241 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610527c7e50 msgr2=0x5610527ca240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:42.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.640+0000 7f911f400640 1 --2- 172.21.15.77:0/1015517241 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610527c7e50 0x5610527ca240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:42.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.640+0000 7f911f400640 1 -- 172.21.15.77:0/1015517241 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610526e9ae0 con 0x5610527bcc70 2024-09-10T10:02:42.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.640+0000 7f9117400640 1 --2- 172.21.15.77:0/2842683586 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f91040a2c00 0x7f91040a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:42.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.640+0000 7f9117400640 1 -- 172.21.15.77:0/2842683586 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f91100cf6a0 con 0x7f9120104010 2024-09-10T10:02:42.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.640+0000 7f911ea00640 1 --2- 172.21.15.77:0/2842683586 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f91040a2c00 0x7f91040a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:42.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.640+0000 7f911f400640 1 --2- 172.21.15.77:0/1015517241 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610527bcc70 0x5610527bd050 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f9118009f50 tx=0x7f91180118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43d79210ec0ff674 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:42.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.640+0000 7f910f400640 1 -- 172.21.15.77:0/1015517241 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9118004d20 con 0x5610527bcc70 2024-09-10T10:02:42.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.640+0000 7f910f400640 1 -- 172.21.15.77:0/1015517241 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9118004050 con 0x5610527bcc70 2024-09-10T10:02:42.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.640+0000 7f910f400640 1 -- 172.21.15.77:0/1015517241 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9118005280 con 0x5610527bcc70 2024-09-10T10:02:42.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.641+0000 7f9128019740 1 -- 172.21.15.77:0/1015517241 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610527bcc70 msgr2=0x5610527bd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:42.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.641+0000 7f9128019740 1 --2- 172.21.15.77:0/1015517241 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610527bcc70 0x5610527bd050 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f9118009f50 tx=0x7f91180118d0 comp rx=0 tx=0).stop 2024-09-10T10:02:42.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.641+0000 7f911ea00640 1 --2- 172.21.15.77:0/2842683586 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f91040a2c00 0x7f91040a50c0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f91100cf940 tx=0x7f91100d5530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:42.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.641+0000 7f9128019740 1 -- 172.21.15.77:0/1015517241 shutdown_connections 2024-09-10T10:02:42.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.641+0000 7f9128019740 1 --2- 172.21.15.77:0/1015517241 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610527c7e50 0x5610527ca240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:42.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.641+0000 7f9128019740 1 --2- 172.21.15.77:0/1015517241 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610527bd590 0x5610527c7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:42.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.641+0000 7f9128019740 1 --2- 172.21.15.77:0/1015517241 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610527bcc70 0x5610527bd050 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:42.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.641+0000 7f9128019740 1 -- 172.21.15.77:0/1015517241 >> 172.21.15.77:0/1015517241 conn(0x5610527b9200 msgr2=0x5610527b95e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:42.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.642+0000 7f9128019740 1 -- 172.21.15.77:0/1015517241 shutdown_connections 2024-09-10T10:02:42.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.642+0000 7f9128019740 1 -- 172.21.15.77:0/1015517241 wait complete. 2024-09-10T10:02:42.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.642+0000 7f9128019740 1 -- start start 2024-09-10T10:02:42.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.644+0000 7f9128019740 1 Processor -- start 2024-09-10T10:02:42.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.645+0000 7f9128019740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610527bcc70 0x5610527c4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:42.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.645+0000 7f9128019740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610527bd590 0x5610527c52d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:42.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.645+0000 7f9128019740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610527c7e50 0x5610527c5810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:42.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.645+0000 7f9128019740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5610527caa20 con 0x5610527bcc70 2024-09-10T10:02:42.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.645+0000 7f9128019740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5610526de0f0 con 0x5610527bd590 2024-09-10T10:02:42.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.645+0000 7f9128019740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5610527caba0 con 0x5610527c7e50 2024-09-10T10:02:42.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.645+0000 7f9128019740 10 client.0 _ll_register_callbacks cb 0x5610526c5bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:02:42.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.646+0000 7f9117e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610527bd590 0x5610527c52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:42.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.646+0000 7f911f400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610527c7e50 0x5610527c5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:42.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.646+0000 7f911f400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610527c7e50 0x5610527c5810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:54788/0 (socket says 172.21.15.77:54788) 2024-09-10T10:02:42.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.646+0000 7f911f400640 1 -- 172.21.15.77:0/3750228817 learned_addr learned my addr 172.21.15.77:0/3750228817 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:02:42.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.647+0000 7f911f400640 1 -- 172.21.15.77:0/3750228817 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610527bd590 msgr2=0x5610527c52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:42.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.647+0000 7f911f400640 1 --2- 172.21.15.77:0/3750228817 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610527bd590 0x5610527c52d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:42.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.647+0000 7f911f400640 1 -- 172.21.15.77:0/3750228817 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610527bcc70 msgr2=0x5610527c4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:42.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.647+0000 7f911f400640 1 --2- 172.21.15.77:0/3750228817 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610527bcc70 0x5610527c4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:42.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.647+0000 7f911f400640 1 -- 172.21.15.77:0/3750228817 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610524943c0 con 0x5610527c7e50 2024-09-10T10:02:42.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.647+0000 7f911f400640 1 --2- 172.21.15.77:0/3750228817 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610527c7e50 0x5610527c5810 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f9118004e30 tx=0x7f9118009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5417a9756a8f7952 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:42.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.652+0000 7f9103e00640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-10T10:02:42.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.652+0000 7f9103e00640 1 -- 172.21.15.77:0/3750228817 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9118028030 con 0x5610527c7e50 2024-09-10T10:02:42.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.652+0000 7f9103e00640 1 -- 172.21.15.77:0/3750228817 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f9118003c40 con 0x5610527c7e50 2024-09-10T10:02:42.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.652+0000 7f9128019740 1 -- 172.21.15.77:0/3750228817 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9118004050 con 0x5610527c7e50 2024-09-10T10:02:42.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.652+0000 7f9103e00640 1 -- 172.21.15.77:0/3750228817 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f9118003e40 con 0x5610527c7e50 2024-09-10T10:02:42.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.652+0000 7f9128019740 10 client.26980 fetch_fsmap learned FSMap version 35 2024-09-10T10:02:42.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.652+0000 7f9128019740 1 -- 172.21.15.77:0/3750228817 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5610527e7ac0 con 0x5610527c7e50 2024-09-10T10:02:42.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.652+0000 7f9103e00640 1 -- 172.21.15.77:0/3750228817 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9118034070 con 0x5610527c7e50 2024-09-10T10:02:42.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.653+0000 7f9103e00640 1 -- 172.21.15.77:0/3750228817 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f911800f530 con 0x5610527c7e50 2024-09-10T10:02:42.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.653+0000 7f9128019740 10 client.26980 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T10:02:42.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.653+0000 7f9128019740 10 client.26980 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T10:02:42.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.653+0000 7f9128019740 20 client.26980 populate_metadata read hostname 'smithi077' 2024-09-10T10:02:42.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.653+0000 7f9128019740 1 --2- 172.21.15.77:0/3750228817 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x5610527e8ec0 0x5610527eb2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:42.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.653+0000 7f9128019740 4 client.26980 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:02:42.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.653+0000 7f9128019740 1 -- 172.21.15.77:0/3750228817 --> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561052730950 con 0x5610527e8ec0 2024-09-10T10:02:42.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.653+0000 7f9117e00640 1 --2- 172.21.15.77:0/3750228817 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x5610527e8ec0 0x5610527eb2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:42.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.654+0000 7f9117e00640 1 --2- 172.21.15.77:0/3750228817 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x5610527e8ec0 0x5610527eb2b0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=27e5ca489ef03eae server_cookie=8d7f9e5934ef790 in_seq=0 out_seq=0 2024-09-10T10:02:42.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.654+0000 7f9103e00640 10 client.26980 ms_handle_connect on v2:172.21.15.134:6836/2461497091 2024-09-10T10:02:42.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.655+0000 7f9103e00640 1 -- 172.21.15.77:0/3750228817 <== mds.3 v2:172.21.15.134:6836/2461497091 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561052730950 con 0x5610527e8ec0 2024-09-10T10:02:42.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.655+0000 7f9103e00640 10 client.26980 handle_command_reply: tid=0 2024-09-10T10:02:42.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.655+0000 7f9128019740 1 client.26980 shutdown 2024-09-10T10:02:42.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.656+0000 7f9128019740 1 -- 172.21.15.77:0/3750228817 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610527c7e50 msgr2=0x5610527c5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:42.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.656+0000 7f9128019740 1 --2- 172.21.15.77:0/3750228817 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610527c7e50 0x5610527c5810 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f9118004e30 tx=0x7f9118009ca0 comp rx=0 tx=0).stop 2024-09-10T10:02:42.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.656+0000 7f9128019740 1 -- 172.21.15.77:0/3750228817 shutdown_connections 2024-09-10T10:02:42.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.656+0000 7f9128019740 1 --2- 172.21.15.77:0/3750228817 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610527c7e50 0x5610527c5810 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:42.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.656+0000 7f9128019740 1 --2- 172.21.15.77:0/3750228817 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610527bd590 0x5610527c52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:42.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.656+0000 7f9128019740 1 --2- 172.21.15.77:0/3750228817 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x5610527e8ec0 0x5610527eb2b0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:42.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.656+0000 7f9128019740 1 --2- 172.21.15.77:0/3750228817 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610527bcc70 0x5610527c4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:42.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.656+0000 7f9128019740 1 -- 172.21.15.77:0/3750228817 >> 172.21.15.77:0/3750228817 conn(0x5610527b9200 msgr2=0x5610527beee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:42.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.656+0000 7f9103e00640 0 client.26980 ms_handle_reset on v2:172.21.15.134:6836/2461497091 2024-09-10T10:02:42.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.657+0000 7f9128019740 1 -- 172.21.15.77:0/3750228817 shutdown_connections 2024-09-10T10:02:42.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.661+0000 7f9128019740 1 -- 172.21.15.77:0/3750228817 wait complete. 2024-09-10T10:02:42.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.662+0000 7f9128019740 20 client.26980 trim_cache size 0 max 16384 2024-09-10T10:02:42.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.681+0000 7f9128019740 1 Processor -- start 2024-09-10T10:02:42.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.681+0000 7f9128019740 1 -- start start 2024-09-10T10:02:42.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.681+0000 7f9128019740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610527bcc70 0x5610527c5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:42.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.681+0000 7f9128019740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610527bd590 0x5610527f09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:42.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.681+0000 7f9128019740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610527c7e50 0x5610527f0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:42.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.681+0000 7f9128019740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f91201a8a50 con 0x5610527bcc70 2024-09-10T10:02:42.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.681+0000 7f9128019740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f91201aa590 con 0x5610527c7e50 2024-09-10T10:02:42.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.681+0000 7f9128019740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f91201a9120 con 0x5610527bd590 2024-09-10T10:02:42.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.682+0000 7f9117e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610527bd590 0x5610527f09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:42.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.682+0000 7f911f400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610527bcc70 0x5610527c5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:42.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.682+0000 7f9117e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610527bd590 0x5610527f09a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:54798/0 (socket says 172.21.15.77:54798) 2024-09-10T10:02:42.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.682+0000 7f9117e00640 1 -- 172.21.15.77:0/1089030711 learned_addr learned my addr 172.21.15.77:0/1089030711 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:02:42.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.682+0000 7f911f400640 1 --2- 172.21.15.77:0/1089030711 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610527c7e50 0x5610527f0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:42.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.682+0000 7f9117e00640 1 -- 172.21.15.77:0/1089030711 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610527c7e50 msgr2=0x5610527f0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:42.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.682+0000 7f9117e00640 1 --2- 172.21.15.77:0/1089030711 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610527c7e50 0x5610527f0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:42.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.682+0000 7f9117e00640 1 -- 172.21.15.77:0/1089030711 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610527bcc70 msgr2=0x5610527c5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:42.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.682+0000 7f9117e00640 1 --2- 172.21.15.77:0/1089030711 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610527bcc70 0x5610527c5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:42.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.682+0000 7f9117e00640 1 -- 172.21.15.77:0/1089030711 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610527bd3c0 con 0x5610527bd590 2024-09-10T10:02:42.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.682+0000 7f911f400640 1 --2- 172.21.15.77:0/1089030711 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610527bcc70 0x5610527c5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:02:42.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.682+0000 7f9117e00640 1 --2- 172.21.15.77:0/1089030711 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610527bd590 0x5610527f09a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f910802a9e0 tx=0x7f91080046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa3e079df3d6c97d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:42.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.682+0000 7f910ea00640 1 -- 172.21.15.77:0/1089030711 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9108035040 con 0x5610527bd590 2024-09-10T10:02:42.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.683+0000 7f910ea00640 1 -- 172.21.15.77:0/1089030711 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9108031d70 con 0x5610527bd590 2024-09-10T10:02:42.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.683+0000 7f910ea00640 1 -- 172.21.15.77:0/1089030711 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9108004a00 con 0x5610527bd590 2024-09-10T10:02:42.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.683+0000 7f9128019740 1 -- 172.21.15.77:0/1089030711 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610527bd590 msgr2=0x5610527f09a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:42.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.683+0000 7f9128019740 1 --2- 172.21.15.77:0/1089030711 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610527bd590 0x5610527f09a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f910802a9e0 tx=0x7f91080046c0 comp rx=0 tx=0).stop 2024-09-10T10:02:42.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.683+0000 7f9128019740 1 -- 172.21.15.77:0/1089030711 shutdown_connections 2024-09-10T10:02:42.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.683+0000 7f9128019740 1 --2- 172.21.15.77:0/1089030711 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610527c7e50 0x5610527f0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:42.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.683+0000 7f9128019740 1 --2- 172.21.15.77:0/1089030711 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610527bd590 0x5610527f09a0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:42.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.683+0000 7f9128019740 1 --2- 172.21.15.77:0/1089030711 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610527bcc70 0x5610527c5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:42.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.683+0000 7f9128019740 1 -- 172.21.15.77:0/1089030711 >> 172.21.15.77:0/1089030711 conn(0x5610527b9200 msgr2=0x5610527c3ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:42.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.684+0000 7f9128019740 1 -- 172.21.15.77:0/1089030711 shutdown_connections 2024-09-10T10:02:42.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.684+0000 7f9128019740 1 -- 172.21.15.77:0/1089030711 wait complete. 2024-09-10T10:02:42.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.684+0000 7f9128019740 1 -- start start 2024-09-10T10:02:42.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.684+0000 7f9128019740 1 Processor -- start 2024-09-10T10:02:42.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.685+0000 7f9128019740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610527bcc70 0x5610527d02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:42.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.685+0000 7f9128019740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610527bd590 0x5610527c18b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:42.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.685+0000 7f9128019740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610527c7e50 0x5610527c1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:42.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.685+0000 7f9128019740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f91201a9ec0 con 0x5610527bcc70 2024-09-10T10:02:42.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.685+0000 7f9128019740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f91201a97f0 con 0x5610527c7e50 2024-09-10T10:02:42.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.685+0000 7f9128019740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f91200739a0 con 0x5610527bd590 2024-09-10T10:02:42.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.685+0000 7f9128019740 10 client.0 _ll_register_callbacks cb 0x7f91100d5400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:02:42.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.685+0000 7f9117e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610527bd590 0x5610527c18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:42.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.685+0000 7f9117e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610527bd590 0x5610527c18b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:54806/0 (socket says 172.21.15.77:54806) 2024-09-10T10:02:42.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.685+0000 7f9117e00640 1 -- 172.21.15.77:0/3622733213 learned_addr learned my addr 172.21.15.77:0/3622733213 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:02:42.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.685+0000 7f911f400640 1 --2- 172.21.15.77:0/3622733213 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610527c7e50 0x5610527c1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:42.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.685+0000 7f911f400640 1 --2- 172.21.15.77:0/3622733213 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610527bcc70 0x5610527d02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:42.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.686+0000 7f9117e00640 1 -- 172.21.15.77:0/3622733213 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610527c7e50 msgr2=0x5610527c1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:42.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.686+0000 7f9117e00640 1 --2- 172.21.15.77:0/3622733213 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610527c7e50 0x5610527c1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:42.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.686+0000 7f9117e00640 1 -- 172.21.15.77:0/3622733213 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610527bcc70 msgr2=0x5610527d02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:42.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.686+0000 7f9117e00640 1 --2- 172.21.15.77:0/3622733213 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610527bcc70 0x5610527d02e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:42.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.686+0000 7f9117e00640 1 -- 172.21.15.77:0/3622733213 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610527c23b0 con 0x5610527bd590 2024-09-10T10:02:42.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.686+0000 7f9117e00640 1 --2- 172.21.15.77:0/3622733213 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610527bd590 0x5610527c18b0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f9108031550 tx=0x7f91080314b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e089a1a955aca74 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:42.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.686+0000 7f9103e00640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-10T10:02:42.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.687+0000 7f9103e00640 1 -- 172.21.15.77:0/3622733213 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9108056060 con 0x5610527bd590 2024-09-10T10:02:42.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.687+0000 7f9128019740 1 -- 172.21.15.77:0/3622733213 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9108031d70 con 0x5610527bd590 2024-09-10T10:02:42.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.687+0000 7f9103e00640 1 -- 172.21.15.77:0/3622733213 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f910804b040 con 0x5610527bd590 2024-09-10T10:02:42.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.688+0000 7f9103e00640 1 -- 172.21.15.77:0/3622733213 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f9108056060 con 0x5610527bd590 2024-09-10T10:02:42.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.688+0000 7f9128019740 10 client.26986 fetch_fsmap learned FSMap version 35 2024-09-10T10:02:42.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.688+0000 7f9128019740 1 -- 172.21.15.77:0/3622733213 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56105284c6e0 con 0x5610527bd590 2024-09-10T10:02:42.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.689+0000 7f9103e00640 1 -- 172.21.15.77:0/3622733213 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9108031d70 con 0x5610527bd590 2024-09-10T10:02:42.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.689+0000 7f9103e00640 1 -- 172.21.15.77:0/3622733213 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f91080516d0 con 0x5610527bd590 2024-09-10T10:02:42.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.689+0000 7f9128019740 10 client.26986 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T10:02:42.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.689+0000 7f9128019740 10 client.26986 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T10:02:42.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.690+0000 7f9128019740 20 client.26986 populate_metadata read hostname 'smithi077' 2024-09-10T10:02:42.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.686+0000 7f911f400640 1 --2- 172.21.15.77:0/3622733213 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610527bcc70 0x5610527d02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:02:42.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.690+0000 7f9128019740 1 --2- 172.21.15.77:0/3622733213 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x56105284ce80 0x56105284f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:42.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.690+0000 7f9128019740 4 client.26986 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:02:42.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.690+0000 7f9128019740 1 -- 172.21.15.77:0/3622733213 --> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56105272ae60 con 0x56105284ce80 2024-09-10T10:02:42.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.690+0000 7f911f400640 1 --2- 172.21.15.77:0/3622733213 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x56105284ce80 0x56105284f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:42.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.692+0000 7f911f400640 1 --2- 172.21.15.77:0/3622733213 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x56105284ce80 0x56105284f340 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3825521053c8e729 server_cookie=512081ab15c2e03 in_seq=0 out_seq=0 2024-09-10T10:02:42.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.692+0000 7f9103e00640 10 client.26986 ms_handle_connect on v2:172.21.15.134:6836/2461497091 2024-09-10T10:02:42.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.693+0000 7f9103e00640 1 -- 172.21.15.77:0/3622733213 <== mds.3 v2:172.21.15.134:6836/2461497091 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56105272ae60 con 0x56105284ce80 2024-09-10T10:02:42.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.693+0000 7f9103e00640 10 client.26986 handle_command_reply: tid=0 2024-09-10T10:02:42.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.693+0000 7f9128019740 1 client.26986 shutdown 2024-09-10T10:02:42.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.693+0000 7f9128019740 1 -- 172.21.15.77:0/3622733213 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610527bd590 msgr2=0x5610527c18b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:42.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.693+0000 7f9128019740 1 --2- 172.21.15.77:0/3622733213 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610527bd590 0x5610527c18b0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f9108031550 tx=0x7f91080314b0 comp rx=0 tx=0).stop 2024-09-10T10:02:42.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.695+0000 7f9128019740 1 -- 172.21.15.77:0/3622733213 shutdown_connections 2024-09-10T10:02:42.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.695+0000 7f9128019740 1 --2- 172.21.15.77:0/3622733213 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610527c7e50 0x5610527c1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:42.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.695+0000 7f9128019740 1 --2- 172.21.15.77:0/3622733213 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610527bd590 0x5610527c18b0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:42.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.695+0000 7f9128019740 1 --2- 172.21.15.77:0/3622733213 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x56105284ce80 0x56105284f340 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:42.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.695+0000 7f9128019740 1 --2- 172.21.15.77:0/3622733213 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610527bcc70 0x5610527d02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:42.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.695+0000 7f9128019740 1 -- 172.21.15.77:0/3622733213 >> 172.21.15.77:0/3622733213 conn(0x5610527b9200 msgr2=0x5610527b7b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:42.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.695+0000 7f9103e00640 0 client.26986 ms_handle_reset on v2:172.21.15.134:6836/2461497091 2024-09-10T10:02:42.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.695+0000 7f9128019740 1 -- 172.21.15.77:0/3622733213 shutdown_connections 2024-09-10T10:02:42.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.695+0000 7f9128019740 1 -- 172.21.15.77:0/3622733213 wait complete. 2024-09-10T10:02:42.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.696+0000 7f9128019740 20 client.26986 trim_cache size 0 max 16384 2024-09-10T10:02:42.697 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-10T10:02:42.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.697+0000 7f9125600640 1 -- 172.21.15.77:0/2842683586 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f91040a2c00 msgr2=0x7f91040a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:42.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.697+0000 7f9125600640 1 --2- 172.21.15.77:0/2842683586 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f91040a2c00 0x7f91040a50c0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f91100cf940 tx=0x7f91100d5530 comp rx=0 tx=0).stop 2024-09-10T10:02:42.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.697+0000 7f9125600640 1 -- 172.21.15.77:0/2842683586 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9120104010 msgr2=0x7f912019a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:42.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.697+0000 7f9125600640 1 --2- 172.21.15.77:0/2842683586 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9120104010 0x7f912019a6f0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f911000e9f0 tx=0x7f911000eeb0 comp rx=0 tx=0).stop 2024-09-10T10:02:42.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.697+0000 7f9125600640 1 -- 172.21.15.77:0/2842683586 shutdown_connections 2024-09-10T10:02:42.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.697+0000 7f9125600640 1 --2- 172.21.15.77:0/2842683586 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f91040a2c00 0x7f91040a50c0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:42.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.697+0000 7f9125600640 1 --2- 172.21.15.77:0/2842683586 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9120109140 0x7f912019efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:42.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.697+0000 7f9125600640 1 --2- 172.21.15.77:0/2842683586 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f91201049c0 0x7f912019ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:42.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.697+0000 7f9125600640 1 --2- 172.21.15.77:0/2842683586 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9120104010 0x7f912019a6f0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:42.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.697+0000 7f9125600640 1 -- 172.21.15.77:0/2842683586 >> 172.21.15.77:0/2842683586 conn(0x7f91200ffa50 msgr2=0x7f9120100dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:42.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.698+0000 7f9125600640 1 -- 172.21.15.77:0/2842683586 shutdown_connections 2024-09-10T10:02:42.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:42.698+0000 7f9125600640 1 -- 172.21.15.77:0/2842683586 wait complete. 2024-09-10T10:02:43.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:02:42 smithi077 ceph-mon[30319]: pgmap v617: 129 pgs: 129 active+clean; 139 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 1.1 MiB/s wr, 423 op/s 2024-09-10T10:02:43.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:02:42 smithi134 ceph-mon[35674]: pgmap v617: 129 pgs: 129 active+clean; 139 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 1.1 MiB/s wr, 423 op/s 2024-09-10T10:02:43.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:02:42 smithi120 ceph-mon[34770]: pgmap v617: 129 pgs: 129 active+clean; 139 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 1.1 MiB/s wr, 423 op/s 2024-09-10T10:02:43.849 INFO:tasks.workunit.client.0.smithi077.stdout:20000 read 0xa6cbf1 thru 0xa6e358 (0x1768 bytes) 2024-09-10T10:02:44.212 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T10:02:44.703 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:02:45.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:02:44 smithi077 ceph-mon[30319]: pgmap v618: 129 pgs: 129 active+clean; 146 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.2 MiB/s wr, 470 op/s 2024-09-10T10:02:45.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:02:44 smithi134 ceph-mon[35674]: pgmap v618: 129 pgs: 129 active+clean; 146 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.2 MiB/s wr, 470 op/s 2024-09-10T10:02:45.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:02:44 smithi120 ceph-mon[34770]: pgmap v618: 129 pgs: 129 active+clean; 146 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.2 MiB/s wr, 470 op/s 2024-09-10T10:02:45.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.519+0000 7fcbd3000640 1 -- 172.21.15.77:0/3448782730 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcbcc074e30 msgr2=0x7fcbcc10dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:45.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.519+0000 7fcbd3000640 1 --2- 172.21.15.77:0/3448782730 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcbcc074e30 0x7fcbcc10dd40 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fcbb8009900 tx=0x7fcbb802a480 comp rx=0 tx=0).stop 2024-09-10T10:02:45.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.519+0000 7fcbd3000640 1 -- 172.21.15.77:0/3448782730 shutdown_connections 2024-09-10T10:02:45.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.519+0000 7fcbd3000640 1 --2- 172.21.15.77:0/3448782730 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcbcc10e280 0x7fcbcc110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:45.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.519+0000 7fcbd3000640 1 --2- 172.21.15.77:0/3448782730 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcbcc074e30 0x7fcbcc10dd40 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:45.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.519+0000 7fcbd3000640 1 --2- 172.21.15.77:0/3448782730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcbcc074510 0x7fcbcc0748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:45.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.519+0000 7fcbd3000640 1 -- 172.21.15.77:0/3448782730 >> 172.21.15.77:0/3448782730 conn(0x7fcbcc0fd740 msgr2=0x7fcbcc0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:45.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.519+0000 7fcbd3000640 1 -- 172.21.15.77:0/3448782730 shutdown_connections 2024-09-10T10:02:45.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.519+0000 7fcbd3000640 1 -- 172.21.15.77:0/3448782730 wait complete. 2024-09-10T10:02:45.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.520+0000 7fcbd3000640 1 Processor -- start 2024-09-10T10:02:45.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.521+0000 7fcbd3000640 1 -- start start 2024-09-10T10:02:45.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.521+0000 7fcbd3000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcbcc074510 0x7fcbcc19a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:45.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.521+0000 7fcbd3000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcbcc074e30 0x7fcbcc19aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:45.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.521+0000 7fcbd3000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcbcc10e280 0x7fcbcc19edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:45.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.521+0000 7fcbd3000640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fcbcc114000 con 0x7fcbcc074e30 2024-09-10T10:02:45.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.521+0000 7fcbd3000640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fcbcc113e80 con 0x7fcbcc074510 2024-09-10T10:02:45.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.521+0000 7fcbd3000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcbcc114180 con 0x7fcbcc10e280 2024-09-10T10:02:45.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.521+0000 7fcbd1c00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcbcc074510 0x7fcbcc19a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:45.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.522+0000 7fcbd2600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcbcc10e280 0x7fcbcc19edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:45.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.522+0000 7fcbd2600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcbcc10e280 0x7fcbcc19edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:37892/0 (socket says 172.21.15.77:37892) 2024-09-10T10:02:45.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.522+0000 7fcbd1c00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcbcc074510 0x7fcbcc19a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:36656/0 (socket says 172.21.15.77:36656) 2024-09-10T10:02:45.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.522+0000 7fcbd2600640 1 -- 172.21.15.77:0/1798461107 learned_addr learned my addr 172.21.15.77:0/1798461107 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:02:45.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.522+0000 7fcbd1200640 1 --2- 172.21.15.77:0/1798461107 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcbcc074e30 0x7fcbcc19aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:45.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.522+0000 7fcbd1c00640 1 -- 172.21.15.77:0/1798461107 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcbcc10e280 msgr2=0x7fcbcc19edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:45.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.522+0000 7fcbd1c00640 1 --2- 172.21.15.77:0/1798461107 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcbcc10e280 0x7fcbcc19edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:45.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.522+0000 7fcbd1c00640 1 -- 172.21.15.77:0/1798461107 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcbcc074e30 msgr2=0x7fcbcc19aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:45.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.522+0000 7fcbd1c00640 1 --2- 172.21.15.77:0/1798461107 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcbcc074e30 0x7fcbcc19aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:45.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.522+0000 7fcbd1c00640 1 -- 172.21.15.77:0/1798461107 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbcc19f4d0 con 0x7fcbcc074510 2024-09-10T10:02:45.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.522+0000 7fcbd1c00640 1 --2- 172.21.15.77:0/1798461107 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcbcc074510 0x7fcbcc19a520 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc009fd0 tx=0x7fcbbc00ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:45.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.523+0000 7fcbca000640 1 -- 172.21.15.77:0/1798461107 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbbc004b20 con 0x7fcbcc074510 2024-09-10T10:02:45.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.523+0000 7fcbca000640 1 -- 172.21.15.77:0/1798461107 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbbc004cc0 con 0x7fcbcc074510 2024-09-10T10:02:45.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.523+0000 7fcbd3000640 1 -- 172.21.15.77:0/1798461107 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbcc19f7c0 con 0x7fcbcc074510 2024-09-10T10:02:45.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.523+0000 7fcbca000640 1 -- 172.21.15.77:0/1798461107 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbbc0107b0 con 0x7fcbcc074510 2024-09-10T10:02:45.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.523+0000 7fcbd3000640 1 -- 172.21.15.77:0/1798461107 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbcc1a70a0 con 0x7fcbcc074510 2024-09-10T10:02:45.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.527+0000 7fcbd7214740 1 Processor -- start 2024-09-10T10:02:45.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.527+0000 7fcbd7214740 1 -- start start 2024-09-10T10:02:45.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.527+0000 7fcbca000640 1 -- 172.21.15.77:0/1798461107 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fcbbc005020 con 0x7fcbcc074510 2024-09-10T10:02:45.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.528+0000 7fcbd7214740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556168e2be40 0x556168e2c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:45.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.528+0000 7fcbd7214740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556168e2c760 0x556168e36ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:45.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.528+0000 7fcbd7214740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556168e37020 0x556168e39410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:45.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.528+0000 7fcbd7214740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x556168d923e0 con 0x556168e2be40 2024-09-10T10:02:45.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.528+0000 7fcbd7214740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x556168d39d80 con 0x556168e37020 2024-09-10T10:02:45.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.528+0000 7fcbd7214740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556168d8e3a0 con 0x556168e2c760 2024-09-10T10:02:45.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.528+0000 7fcbd2600640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556168e2be40 0x556168e2c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:45.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.528+0000 7fcbd1c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556168e2c760 0x556168e36ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:45.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.528+0000 7fcbd1200640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556168e37020 0x556168e39410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:45.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.528+0000 7fcbd2600640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556168e2be40 0x556168e2c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:35264/0 (socket says 172.21.15.77:35264) 2024-09-10T10:02:45.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.528+0000 7fcbd1200640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556168e37020 0x556168e39410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:36666/0 (socket says 172.21.15.77:36666) 2024-09-10T10:02:45.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.528+0000 7fcbd2600640 1 -- 172.21.15.77:0/2046523514 learned_addr learned my addr 172.21.15.77:0/2046523514 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:02:45.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.529+0000 7fcbd1c00640 1 -- 172.21.15.77:0/2046523514 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556168e37020 msgr2=0x556168e39410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:45.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.529+0000 7fcbd1c00640 1 --2- 172.21.15.77:0/2046523514 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556168e37020 0x556168e39410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:45.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.529+0000 7fcbd1c00640 1 -- 172.21.15.77:0/2046523514 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556168e2be40 msgr2=0x556168e2c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:45.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.529+0000 7fcbd1c00640 1 --2- 172.21.15.77:0/2046523514 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556168e2be40 0x556168e2c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:45.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.529+0000 7fcbd1c00640 1 -- 172.21.15.77:0/2046523514 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556168d58ae0 con 0x556168e2c760 2024-09-10T10:02:45.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.529+0000 7fcbd1c00640 1 --2- 172.21.15.77:0/2046523514 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556168e2c760 0x556168e36ae0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc0cd210 tx=0x7fcbbc0d3d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b7bc181c5cde09c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:45.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.529+0000 7fcbc2a00640 1 -- 172.21.15.77:0/2046523514 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbbc0d8bf0 con 0x556168e2c760 2024-09-10T10:02:45.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.529+0000 7fcbca000640 1 --2- 172.21.15.77:0/1798461107 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fcbb00a2c00 0x7fcbb00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:45.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.529+0000 7fcbc2a00640 1 -- 172.21.15.77:0/2046523514 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbbc0e1070 con 0x556168e2c760 2024-09-10T10:02:45.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.529+0000 7fcbc2a00640 1 -- 172.21.15.77:0/2046523514 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbbc09b3f0 con 0x556168e2c760 2024-09-10T10:02:45.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.530+0000 7fcbca000640 1 -- 172.21.15.77:0/1798461107 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fcbbc0cfb30 con 0x7fcbcc074510 2024-09-10T10:02:45.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.530+0000 7fcbd2600640 1 --2- 172.21.15.77:0/1798461107 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fcbb00a2c00 0x7fcbb00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:45.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.530+0000 7fcbd7214740 1 -- 172.21.15.77:0/2046523514 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556168e2c760 msgr2=0x556168e36ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:45.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.530+0000 7fcbd7214740 1 --2- 172.21.15.77:0/2046523514 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556168e2c760 0x556168e36ae0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc0cd210 tx=0x7fcbbc0d3d90 comp rx=0 tx=0).stop 2024-09-10T10:02:45.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.531+0000 7fcbd7214740 1 -- 172.21.15.77:0/2046523514 shutdown_connections 2024-09-10T10:02:45.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.531+0000 7fcbd7214740 1 --2- 172.21.15.77:0/2046523514 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556168e37020 0x556168e39410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:45.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.531+0000 7fcbd7214740 1 --2- 172.21.15.77:0/2046523514 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556168e2c760 0x556168e36ae0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:45.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.531+0000 7fcbd7214740 1 --2- 172.21.15.77:0/2046523514 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556168e2be40 0x556168e2c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:45.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.531+0000 7fcbd7214740 1 -- 172.21.15.77:0/2046523514 >> 172.21.15.77:0/2046523514 conn(0x556168e283d0 msgr2=0x556168e287b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:45.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.531+0000 7fcbd2600640 1 --2- 172.21.15.77:0/1798461107 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fcbb00a2c00 0x7fcbb00a50c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fcbc40095e0 tx=0x7fcbc4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:45.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.531+0000 7fcbd7214740 1 -- 172.21.15.77:0/2046523514 shutdown_connections 2024-09-10T10:02:45.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.531+0000 7fcbd7214740 1 -- 172.21.15.77:0/2046523514 wait complete. 2024-09-10T10:02:45.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.531+0000 7fcbd7214740 1 -- start start 2024-09-10T10:02:45.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.532+0000 7fcbd7214740 1 Processor -- start 2024-09-10T10:02:45.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.532+0000 7fcbd7214740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556168e2be40 0x556168e33e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:45.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.532+0000 7fcbd7214740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556168e2c760 0x556168e34370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:45.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.532+0000 7fcbd7214740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556168e37020 0x556168e348b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:45.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.532+0000 7fcbd7214740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x556168e39bf0 con 0x556168e2c760 2024-09-10T10:02:45.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.532+0000 7fcbd7214740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x556168d4d0f0 con 0x556168e37020 2024-09-10T10:02:45.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.532+0000 7fcbd7214740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556168e39d70 con 0x556168e2be40 2024-09-10T10:02:45.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.532+0000 7fcbd7214740 10 client.0 _ll_register_callbacks cb 0x556168d34bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:02:45.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.532+0000 7fcbd1c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556168e2be40 0x556168e33e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:45.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.532+0000 7fcbd1c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556168e2be40 0x556168e33e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:37900/0 (socket says 172.21.15.77:37900) 2024-09-10T10:02:45.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.532+0000 7fcbd1c00640 1 -- 172.21.15.77:0/2271865225 learned_addr learned my addr 172.21.15.77:0/2271865225 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:02:45.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.533+0000 7fcbd1c00640 1 -- 172.21.15.77:0/2271865225 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556168e37020 msgr2=0x556168e348b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T10:02:45.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.533+0000 7fcbd1c00640 1 --2- 172.21.15.77:0/2271865225 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556168e37020 0x556168e348b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:45.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.533+0000 7fcbd1c00640 1 -- 172.21.15.77:0/2271865225 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556168e2c760 msgr2=0x556168e34370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:45.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.533+0000 7fcbd1200640 1 --2- 172.21.15.77:0/2271865225 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556168e2c760 0x556168e34370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:45.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.533+0000 7fcbd1c00640 1 --2- 172.21.15.77:0/2271865225 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556168e2c760 0x556168e34370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:45.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.533+0000 7fcbd1c00640 1 -- 172.21.15.77:0/2271865225 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556168b033c0 con 0x556168e2be40 2024-09-10T10:02:45.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.533+0000 7fcbd1200640 1 --2- 172.21.15.77:0/2271865225 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556168e2c760 0x556168e34370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:02:45.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.534+0000 7fcbd1c00640 1 --2- 172.21.15.77:0/2271865225 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556168e2be40 0x556168e33e30 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc09bca0 tx=0x7fcbbc09a2c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d9091854642238a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:45.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.534+0000 7fcbb7400640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-10T10:02:45.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.535+0000 7fcbb7400640 1 -- 172.21.15.77:0/2271865225 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbbc099e80 con 0x556168e2be40 2024-09-10T10:02:45.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.535+0000 7fcbb7400640 1 -- 172.21.15.77:0/2271865225 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fcbbc0a4980 con 0x556168e2be40 2024-09-10T10:02:45.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.535+0000 7fcbd7214740 1 -- 172.21.15.77:0/2271865225 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcbbc0e1070 con 0x556168e2be40 2024-09-10T10:02:45.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.535+0000 7fcbb7400640 1 -- 172.21.15.77:0/2271865225 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fcbbc09a0e0 con 0x556168e2be40 2024-09-10T10:02:45.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.535+0000 7fcbd7214740 10 client.26998 fetch_fsmap learned FSMap version 35 2024-09-10T10:02:45.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.535+0000 7fcbd7214740 1 -- 172.21.15.77:0/2271865225 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x556168e39bf0 con 0x556168e2be40 2024-09-10T10:02:45.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.536+0000 7fcbb7400640 1 -- 172.21.15.77:0/2271865225 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbbc0e1070 con 0x556168e2be40 2024-09-10T10:02:45.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.536+0000 7fcbb7400640 1 -- 172.21.15.77:0/2271865225 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7fcbbc098610 con 0x556168e2be40 2024-09-10T10:02:45.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.536+0000 7fcbd7214740 10 client.26998 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T10:02:45.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.536+0000 7fcbd7214740 10 client.26998 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T10:02:45.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.536+0000 7fcbd7214740 20 client.26998 populate_metadata read hostname 'smithi077' 2024-09-10T10:02:45.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.536+0000 7fcbd7214740 1 --2- 172.21.15.77:0/2271865225 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x556168e46cd0 0x556168e490c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:45.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.536+0000 7fcbd7214740 4 client.26998 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:02:45.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.536+0000 7fcbd7214740 1 -- 172.21.15.77:0/2271865225 --> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556168d9f950 con 0x556168e46cd0 2024-09-10T10:02:45.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.536+0000 7fcbd1200640 1 --2- 172.21.15.77:0/2271865225 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x556168e46cd0 0x556168e490c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:45.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.537+0000 7fcbd1200640 1 --2- 172.21.15.77:0/2271865225 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x556168e46cd0 0x556168e490c0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=648aeb6b26a17d4e server_cookie=750cfb93fa7f3517 in_seq=0 out_seq=0 2024-09-10T10:02:45.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.537+0000 7fcbb7400640 10 client.26998 ms_handle_connect on v2:172.21.15.134:6838/1965117680 2024-09-10T10:02:45.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.538+0000 7fcbb7400640 1 -- 172.21.15.77:0/2271865225 <== mds.2 v2:172.21.15.134:6838/1965117680 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556168d9f950 con 0x556168e46cd0 2024-09-10T10:02:45.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.538+0000 7fcbb7400640 10 client.26998 handle_command_reply: tid=0 2024-09-10T10:02:45.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.538+0000 7fcbd7214740 1 client.26998 shutdown 2024-09-10T10:02:45.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.538+0000 7fcbd7214740 1 -- 172.21.15.77:0/2271865225 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556168e2be40 msgr2=0x556168e33e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:45.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.538+0000 7fcbd7214740 1 --2- 172.21.15.77:0/2271865225 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556168e2be40 0x556168e33e30 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc09bca0 tx=0x7fcbbc09a2c0 comp rx=0 tx=0).stop 2024-09-10T10:02:45.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.538+0000 7fcbd7214740 1 -- 172.21.15.77:0/2271865225 shutdown_connections 2024-09-10T10:02:45.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.538+0000 7fcbd7214740 1 --2- 172.21.15.77:0/2271865225 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x556168e46cd0 0x556168e490c0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:45.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.538+0000 7fcbd7214740 1 --2- 172.21.15.77:0/2271865225 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556168e37020 0x556168e348b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:45.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.538+0000 7fcbd7214740 1 --2- 172.21.15.77:0/2271865225 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556168e2c760 0x556168e34370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:45.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.538+0000 7fcbd7214740 1 --2- 172.21.15.77:0/2271865225 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556168e2be40 0x556168e33e30 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:45.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.538+0000 7fcbd7214740 1 -- 172.21.15.77:0/2271865225 >> 172.21.15.77:0/2271865225 conn(0x556168e283d0 msgr2=0x556168e2e100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:45.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.538+0000 7fcbb7400640 0 client.26998 ms_handle_reset on v2:172.21.15.134:6838/1965117680 2024-09-10T10:02:45.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.539+0000 7fcbd7214740 1 -- 172.21.15.77:0/2271865225 shutdown_connections 2024-09-10T10:02:45.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.539+0000 7fcbd7214740 1 -- 172.21.15.77:0/2271865225 wait complete. 2024-09-10T10:02:45.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.539+0000 7fcbd7214740 20 client.26998 trim_cache size 0 max 16384 2024-09-10T10:02:45.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.558+0000 7fcbd7214740 1 Processor -- start 2024-09-10T10:02:45.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.558+0000 7fcbd7214740 1 -- start start 2024-09-10T10:02:45.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.558+0000 7fcbd7214740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556168e2be40 0x556168e36160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:45.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.558+0000 7fcbd7214740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556168e2c760 0x556168e4e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:45.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.558+0000 7fcbd7214740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556168e37020 0x556168e4ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:45.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.558+0000 7fcbd7214740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fcbcc1b0ef0 con 0x556168e37020 2024-09-10T10:02:45.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.558+0000 7fcbd7214740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fcbcc1b2a30 con 0x556168e2c760 2024-09-10T10:02:45.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.558+0000 7fcbd7214740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcbcc1b15c0 con 0x556168e2be40 2024-09-10T10:02:45.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.558+0000 7fcbd1c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556168e2be40 0x556168e36160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:45.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.558+0000 7fcbd1200640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556168e2c760 0x556168e4e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:45.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.558+0000 7fcbd1200640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556168e2c760 0x556168e4e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:36678/0 (socket says 172.21.15.77:36678) 2024-09-10T10:02:45.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.558+0000 7fcbd1200640 1 -- 172.21.15.77:0/3835802498 learned_addr learned my addr 172.21.15.77:0/3835802498 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:02:45.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.558+0000 7fcbd1c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556168e2be40 0x556168e36160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:37912/0 (socket says 172.21.15.77:37912) 2024-09-10T10:02:45.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.559+0000 7fcbd2600640 1 --2- 172.21.15.77:0/3835802498 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556168e37020 0x556168e4ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:45.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.559+0000 7fcbd1200640 1 -- 172.21.15.77:0/3835802498 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556168e2be40 msgr2=0x556168e36160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:45.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.559+0000 7fcbd1200640 1 --2- 172.21.15.77:0/3835802498 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556168e2be40 0x556168e36160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:45.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.559+0000 7fcbd1200640 1 -- 172.21.15.77:0/3835802498 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556168e37020 msgr2=0x556168e4ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:45.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.559+0000 7fcbd1200640 1 --2- 172.21.15.77:0/3835802498 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556168e37020 0x556168e4ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:45.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.559+0000 7fcbd1200640 1 -- 172.21.15.77:0/3835802498 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556168e28200 con 0x556168e2c760 2024-09-10T10:02:45.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.559+0000 7fcbd1c00640 1 --2- 172.21.15.77:0/3835802498 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556168e2be40 0x556168e36160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:02:45.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.559+0000 7fcbd1200640 1 --2- 172.21.15.77:0/3835802498 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556168e2c760 0x556168e4e830 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fcbb8005ec0 tx=0x7fcbb8005b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a512eb3065e6b92 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:45.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.559+0000 7fcbc2000640 1 -- 172.21.15.77:0/3835802498 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb8035040 con 0x556168e2c760 2024-09-10T10:02:45.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.560+0000 7fcbc2000640 1 -- 172.21.15.77:0/3835802498 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbb8004490 con 0x556168e2c760 2024-09-10T10:02:45.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.560+0000 7fcbc2000640 1 -- 172.21.15.77:0/3835802498 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb8004e20 con 0x556168e2c760 2024-09-10T10:02:45.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.560+0000 7fcbd7214740 1 -- 172.21.15.77:0/3835802498 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556168e2c760 msgr2=0x556168e4e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:45.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.560+0000 7fcbd7214740 1 --2- 172.21.15.77:0/3835802498 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556168e2c760 0x556168e4e830 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fcbb8005ec0 tx=0x7fcbb8005b50 comp rx=0 tx=0).stop 2024-09-10T10:02:45.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.560+0000 7fcbd7214740 1 -- 172.21.15.77:0/3835802498 shutdown_connections 2024-09-10T10:02:45.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.560+0000 7fcbd7214740 1 --2- 172.21.15.77:0/3835802498 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556168e37020 0x556168e4ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:45.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.560+0000 7fcbd7214740 1 --2- 172.21.15.77:0/3835802498 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556168e2c760 0x556168e4e830 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:45.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.560+0000 7fcbd7214740 1 --2- 172.21.15.77:0/3835802498 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556168e2be40 0x556168e36160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:45.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.560+0000 7fcbd7214740 1 -- 172.21.15.77:0/3835802498 >> 172.21.15.77:0/3835802498 conn(0x556168e283d0 msgr2=0x556168e34610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:45.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.561+0000 7fcbd7214740 1 -- 172.21.15.77:0/3835802498 shutdown_connections 2024-09-10T10:02:45.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.561+0000 7fcbd7214740 1 -- 172.21.15.77:0/3835802498 wait complete. 2024-09-10T10:02:45.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.561+0000 7fcbd7214740 1 -- start start 2024-09-10T10:02:45.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.561+0000 7fcbd7214740 1 Processor -- start 2024-09-10T10:02:45.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.562+0000 7fcbd7214740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556168e2be40 0x556168e3f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:45.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.562+0000 7fcbd7214740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556168e2c760 0x556168e37fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:45.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.562+0000 7fcbd7214740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556168e37020 0x556168e385a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:45.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.562+0000 7fcbd7214740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fcbcc1b2360 con 0x556168e2c760 2024-09-10T10:02:45.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.562+0000 7fcbd7214740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fcbcc1b1c90 con 0x556168e2be40 2024-09-10T10:02:45.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.562+0000 7fcbd7214740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcbcc1aeed0 con 0x556168e37020 2024-09-10T10:02:45.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.562+0000 7fcbd7214740 10 client.0 _ll_register_callbacks cb 0x7fcbc4009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:02:45.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.562+0000 7fcbd1c00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556168e2be40 0x556168e3f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:45.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.562+0000 7fcbd1c00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556168e2be40 0x556168e3f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:36694/0 (socket says 172.21.15.77:36694) 2024-09-10T10:02:45.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.562+0000 7fcbd1c00640 1 -- 172.21.15.77:0/1716545500 learned_addr learned my addr 172.21.15.77:0/1716545500 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:02:45.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.562+0000 7fcbd1200640 1 --2- 172.21.15.77:0/1716545500 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556168e2c760 0x556168e37fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:45.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.563+0000 7fcbd2600640 1 --2- 172.21.15.77:0/1716545500 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556168e37020 0x556168e385a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:45.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.563+0000 7fcbd1c00640 1 -- 172.21.15.77:0/1716545500 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556168e37020 msgr2=0x556168e385a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:45.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.563+0000 7fcbd1c00640 1 --2- 172.21.15.77:0/1716545500 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556168e37020 0x556168e385a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:45.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.563+0000 7fcbd1c00640 1 -- 172.21.15.77:0/1716545500 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556168e2c760 msgr2=0x556168e37fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:45.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.563+0000 7fcbd1c00640 1 --2- 172.21.15.77:0/1716545500 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556168e2c760 0x556168e37fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:45.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.563+0000 7fcbd1c00640 1 -- 172.21.15.77:0/1716545500 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556168e38da0 con 0x556168e2be40 2024-09-10T10:02:45.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.563+0000 7fcbd2600640 1 --2- 172.21.15.77:0/1716545500 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556168e37020 0x556168e385a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:02:45.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.563+0000 7fcbd1c00640 1 --2- 172.21.15.77:0/1716545500 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556168e2be40 0x556168e3f820 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc098400 tx=0x7fcbbc004e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e30b60a2f3c0d17f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:45.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.563+0000 7fcbb7400640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T10:02:45.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.563+0000 7fcbb7400640 1 -- 172.21.15.77:0/1716545500 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbbc0b4030 con 0x556168e2be40 2024-09-10T10:02:45.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.563+0000 7fcbb7400640 1 -- 172.21.15.77:0/1716545500 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fcbbc005590 con 0x556168e2be40 2024-09-10T10:02:45.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.563+0000 7fcbd7214740 1 -- 172.21.15.77:0/1716545500 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcbb8004490 con 0x556168e2be40 2024-09-10T10:02:45.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.564+0000 7fcbb7400640 1 -- 172.21.15.77:0/1716545500 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fcbbc09a340 con 0x556168e2be40 2024-09-10T10:02:45.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.564+0000 7fcbd7214740 10 client.27041 fetch_fsmap learned FSMap version 35 2024-09-10T10:02:45.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.564+0000 7fcbd7214740 1 -- 172.21.15.77:0/1716545500 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x556168e38da0 con 0x556168e2be40 2024-09-10T10:02:45.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.564+0000 7fcbb7400640 1 -- 172.21.15.77:0/1716545500 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbb8004490 con 0x556168e2be40 2024-09-10T10:02:45.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.564+0000 7fcbb7400640 1 -- 172.21.15.77:0/1716545500 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7fcbbc09e340 con 0x556168e2be40 2024-09-10T10:02:45.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.564+0000 7fcbd7214740 10 client.27041 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T10:02:45.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.564+0000 7fcbd7214740 10 client.27041 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T10:02:45.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.564+0000 7fcbd7214740 20 client.27041 populate_metadata read hostname 'smithi077' 2024-09-10T10:02:45.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.564+0000 7fcbd7214740 1 --2- 172.21.15.77:0/1716545500 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x556168eb1c50 0x556168eb4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:45.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.564+0000 7fcbd7214740 4 client.27041 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:02:45.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.564+0000 7fcbd7214740 1 -- 172.21.15.77:0/1716545500 --> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556168d99e60 con 0x556168eb1c50 2024-09-10T10:02:45.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.565+0000 7fcbd1200640 1 --2- 172.21.15.77:0/1716545500 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x556168eb1c50 0x556168eb4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:45.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.565+0000 7fcbd1200640 1 --2- 172.21.15.77:0/1716545500 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x556168eb1c50 0x556168eb4040 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e5b65c6d38063c45 server_cookie=d21193dcb7c23cda in_seq=0 out_seq=0 2024-09-10T10:02:45.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.565+0000 7fcbb7400640 10 client.27041 ms_handle_connect on v2:172.21.15.134:6838/1965117680 2024-09-10T10:02:45.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.565+0000 7fcbb7400640 1 -- 172.21.15.77:0/1716545500 <== mds.2 v2:172.21.15.134:6838/1965117680 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556168d99e60 con 0x556168eb1c50 2024-09-10T10:02:45.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.565+0000 7fcbb7400640 10 client.27041 handle_command_reply: tid=0 2024-09-10T10:02:45.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.565+0000 7fcbd7214740 1 client.27041 shutdown 2024-09-10T10:02:45.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.566+0000 7fcbd7214740 1 -- 172.21.15.77:0/1716545500 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556168e2be40 msgr2=0x556168e3f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:45.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.566+0000 7fcbd7214740 1 --2- 172.21.15.77:0/1716545500 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556168e2be40 0x556168e3f820 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc098400 tx=0x7fcbbc004e70 comp rx=0 tx=0).stop 2024-09-10T10:02:45.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.566+0000 7fcbd7214740 1 -- 172.21.15.77:0/1716545500 shutdown_connections 2024-09-10T10:02:45.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.566+0000 7fcbd7214740 1 --2- 172.21.15.77:0/1716545500 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x556168eb1c50 0x556168eb4040 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:45.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.566+0000 7fcbd7214740 1 --2- 172.21.15.77:0/1716545500 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556168e37020 0x556168e385a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:45.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.566+0000 7fcbd7214740 1 --2- 172.21.15.77:0/1716545500 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556168e2c760 0x556168e37fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:45.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.566+0000 7fcbb7400640 0 client.27041 ms_handle_reset on v2:172.21.15.134:6838/1965117680 2024-09-10T10:02:45.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.566+0000 7fcbd7214740 1 --2- 172.21.15.77:0/1716545500 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556168e2be40 0x556168e3f820 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:45.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.566+0000 7fcbd7214740 1 -- 172.21.15.77:0/1716545500 >> 172.21.15.77:0/1716545500 conn(0x556168e283d0 msgr2=0x556168e41e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:45.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.566+0000 7fcbd7214740 1 -- 172.21.15.77:0/1716545500 shutdown_connections 2024-09-10T10:02:45.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.566+0000 7fcbd7214740 1 -- 172.21.15.77:0/1716545500 wait complete. 2024-09-10T10:02:45.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.566+0000 7fcbd7214740 20 client.27041 trim_cache size 0 max 16384 2024-09-10T10:02:45.570 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-10T10:02:45.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.567+0000 7fcbd3000640 1 -- 172.21.15.77:0/1798461107 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fcbb00a2c00 msgr2=0x7fcbb00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:45.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.567+0000 7fcbd3000640 1 --2- 172.21.15.77:0/1798461107 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fcbb00a2c00 0x7fcbb00a50c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fcbc40095e0 tx=0x7fcbc4005f90 comp rx=0 tx=0).stop 2024-09-10T10:02:45.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.567+0000 7fcbd3000640 1 -- 172.21.15.77:0/1798461107 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcbcc074510 msgr2=0x7fcbcc19a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:45.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.567+0000 7fcbd3000640 1 --2- 172.21.15.77:0/1798461107 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcbcc074510 0x7fcbcc19a520 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc009fd0 tx=0x7fcbbc00ef90 comp rx=0 tx=0).stop 2024-09-10T10:02:45.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.567+0000 7fcbd3000640 1 -- 172.21.15.77:0/1798461107 shutdown_connections 2024-09-10T10:02:45.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.567+0000 7fcbd3000640 1 --2- 172.21.15.77:0/1798461107 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fcbb00a2c00 0x7fcbb00a50c0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:45.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.567+0000 7fcbd3000640 1 --2- 172.21.15.77:0/1798461107 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcbcc10e280 0x7fcbcc19edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:45.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.567+0000 7fcbd3000640 1 --2- 172.21.15.77:0/1798461107 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcbcc074e30 0x7fcbcc19aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:45.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.567+0000 7fcbd3000640 1 --2- 172.21.15.77:0/1798461107 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcbcc074510 0x7fcbcc19a520 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:45.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.567+0000 7fcbd3000640 1 -- 172.21.15.77:0/1798461107 >> 172.21.15.77:0/1798461107 conn(0x7fcbcc0fd740 msgr2=0x7fcbcc0fe440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:45.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.567+0000 7fcbd3000640 1 -- 172.21.15.77:0/1798461107 shutdown_connections 2024-09-10T10:02:45.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:45.567+0000 7fcbd3000640 1 -- 172.21.15.77:0/1798461107 wait complete. 2024-09-10T10:02:46.378 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T10:02:46.833 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:02:47.226 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:02:46 smithi120 ceph-mon[34770]: pgmap v619: 129 pgs: 129 active+clean; 145 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 1.1 MiB/s wr, 415 op/s 2024-09-10T10:02:47.236 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:02:46 smithi077 ceph-mon[30319]: pgmap v619: 129 pgs: 129 active+clean; 145 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 1.1 MiB/s wr, 415 op/s 2024-09-10T10:02:47.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:02:46 smithi134 ceph-mon[35674]: pgmap v619: 129 pgs: 129 active+clean; 145 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 1.1 MiB/s wr, 415 op/s 2024-09-10T10:02:47.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.678+0000 7f1404c00640 1 -- 172.21.15.77:0/2523773359 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1400102d30 msgr2=0x7f1400103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:47.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.678+0000 7f1404c00640 1 --2- 172.21.15.77:0/2523773359 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1400102d30 0x7f1400103110 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f13f0009930 tx=0x7f13f002a440 comp rx=0 tx=0).stop 2024-09-10T10:02:47.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.678+0000 7f1404c00640 1 -- 172.21.15.77:0/2523773359 shutdown_connections 2024-09-10T10:02:47.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.678+0000 7f1404c00640 1 --2- 172.21.15.77:0/2523773359 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f14001082a0 0x7f140010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:47.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.678+0000 7f1404c00640 1 --2- 172.21.15.77:0/2523773359 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14001036e0 0x7f1400107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:47.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.678+0000 7f1404c00640 1 --2- 172.21.15.77:0/2523773359 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1400102d30 0x7f1400103110 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:47.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.678+0000 7f1404c00640 1 -- 172.21.15.77:0/2523773359 >> 172.21.15.77:0/2523773359 conn(0x7f14000fc7d0 msgr2=0x7f14000febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:47.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.678+0000 7f1404c00640 1 -- 172.21.15.77:0/2523773359 shutdown_connections 2024-09-10T10:02:47.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.678+0000 7f1404c00640 1 -- 172.21.15.77:0/2523773359 wait complete. 2024-09-10T10:02:47.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.679+0000 7f1404c00640 1 Processor -- start 2024-09-10T10:02:47.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.679+0000 7f1404c00640 1 -- start start 2024-09-10T10:02:47.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.680+0000 7f1404c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1400102d30 0x7f1400198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:47.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.680+0000 7f1404c00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f14001036e0 0x7f1400198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:47.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.680+0000 7f1404c00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f14001082a0 0x7f140019ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:47.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.680+0000 7f1404c00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f14001120a0 con 0x7f14001036e0 2024-09-10T10:02:47.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.680+0000 7f1404c00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1400111f20 con 0x7f14001082a0 2024-09-10T10:02:47.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.680+0000 7f1404c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1400112220 con 0x7f1400102d30 2024-09-10T10:02:47.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.680+0000 7f13fde00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1400102d30 0x7f1400198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:47.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.680+0000 7f13fd400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f14001036e0 0x7f1400198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:47.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.680+0000 7f13fde00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1400102d30 0x7f1400198600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:37944/0 (socket says 172.21.15.77:37944) 2024-09-10T10:02:47.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.680+0000 7f13fe800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f14001082a0 0x7f140019ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:47.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.680+0000 7f13fde00640 1 -- 172.21.15.77:0/1161368453 learned_addr learned my addr 172.21.15.77:0/1161368453 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:02:47.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.681+0000 7f13fde00640 1 -- 172.21.15.77:0/1161368453 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f14001082a0 msgr2=0x7f140019ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:47.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.681+0000 7f13fde00640 1 --2- 172.21.15.77:0/1161368453 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f14001082a0 0x7f140019ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:47.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.681+0000 7f13fde00640 1 -- 172.21.15.77:0/1161368453 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f14001036e0 msgr2=0x7f1400198b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:47.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.681+0000 7f13fde00640 1 --2- 172.21.15.77:0/1161368453 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f14001036e0 0x7f1400198b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:47.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.681+0000 7f13fde00640 1 -- 172.21.15.77:0/1161368453 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f140019d5b0 con 0x7f1400102d30 2024-09-10T10:02:47.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.681+0000 7f13fde00640 1 --2- 172.21.15.77:0/1161368453 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1400102d30 0x7f1400198600 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f13f0009230 tx=0x7f13f0004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:47.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.681+0000 7f13f6a00640 1 -- 172.21.15.77:0/1161368453 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f003f030 con 0x7f1400102d30 2024-09-10T10:02:47.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.681+0000 7f13f6a00640 1 -- 172.21.15.77:0/1161368453 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13f0004440 con 0x7f1400102d30 2024-09-10T10:02:47.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.681+0000 7f1404c00640 1 -- 172.21.15.77:0/1161368453 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f140019d840 con 0x7f1400102d30 2024-09-10T10:02:47.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.681+0000 7f13f6a00640 1 -- 172.21.15.77:0/1161368453 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f0035070 con 0x7f1400102d30 2024-09-10T10:02:47.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.681+0000 7f1404c00640 1 -- 172.21.15.77:0/1161368453 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14001008c0 con 0x7f1400102d30 2024-09-10T10:02:47.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.685+0000 7f13f6a00640 1 -- 172.21.15.77:0/1161368453 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f13f0042030 con 0x7f1400102d30 2024-09-10T10:02:47.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.685+0000 7f1407800740 1 Processor -- start 2024-09-10T10:02:47.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.685+0000 7f1407800740 1 -- start start 2024-09-10T10:02:47.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.686+0000 7f1407800740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5558ffa99c70 0x5558ffa9a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:47.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.686+0000 7f1407800740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5558ffa9a590 0x5558ffaa4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:47.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.686+0000 7f1407800740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5558ffaa4e50 0x5558ffaa7240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:47.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.686+0000 7f1407800740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5558ffa003e0 con 0x5558ffaa4e50 2024-09-10T10:02:47.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.686+0000 7f1407800740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5558ff9a7d80 con 0x5558ffa99c70 2024-09-10T10:02:47.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.686+0000 7f1407800740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5558ff9fc3a0 con 0x5558ffa9a590 2024-09-10T10:02:47.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.686+0000 7f13fde00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5558ffa9a590 0x5558ffaa4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:47.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.686+0000 7f13fde00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5558ffa9a590 0x5558ffaa4910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:37960/0 (socket says 172.21.15.77:37960) 2024-09-10T10:02:47.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.686+0000 7f13fde00640 1 -- 172.21.15.77:0/874993592 learned_addr learned my addr 172.21.15.77:0/874993592 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:02:47.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.686+0000 7f13fe800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5558ffa99c70 0x5558ffa9a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:47.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.686+0000 7f13fd400640 1 --2- 172.21.15.77:0/874993592 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5558ffaa4e50 0x5558ffaa7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:47.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.686+0000 7f13fe800640 1 -- 172.21.15.77:0/874993592 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5558ffa9a590 msgr2=0x5558ffaa4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:47.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.686+0000 7f13fe800640 1 --2- 172.21.15.77:0/874993592 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5558ffa9a590 0x5558ffaa4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:47.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.686+0000 7f13fe800640 1 -- 172.21.15.77:0/874993592 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5558ffaa4e50 msgr2=0x5558ffaa7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:47.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.686+0000 7f13fe800640 1 --2- 172.21.15.77:0/874993592 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5558ffaa4e50 0x5558ffaa7240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:47.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.686+0000 7f13fe800640 1 -- 172.21.15.77:0/874993592 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558ff9c6ae0 con 0x5558ffa99c70 2024-09-10T10:02:47.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.687+0000 7f13f6a00640 1 --2- 172.21.15.77:0/1161368453 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f13e40a2c00 0x7f13e40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:47.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.687+0000 7f13fe800640 1 --2- 172.21.15.77:0/874993592 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5558ffa99c70 0x5558ffa9a050 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f13f8009f00 tx=0x7f13f8011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f267d3503328cfe8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:47.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.687+0000 7f13f6a00640 1 -- 172.21.15.77:0/1161368453 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f13f00ef760 con 0x7f1400102d30 2024-09-10T10:02:47.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.687+0000 7f13eb000640 1 -- 172.21.15.77:0/874993592 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f8004d20 con 0x5558ffa99c70 2024-09-10T10:02:47.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.687+0000 7f13eb000640 1 -- 172.21.15.77:0/874993592 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13f8004050 con 0x5558ffa99c70 2024-09-10T10:02:47.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.687+0000 7f13eb000640 1 -- 172.21.15.77:0/874993592 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f8005220 con 0x5558ffa99c70 2024-09-10T10:02:47.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.687+0000 7f13fde00640 1 --2- 172.21.15.77:0/1161368453 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f13e40a2c00 0x7f13e40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:47.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.688+0000 7f1407800740 1 -- 172.21.15.77:0/874993592 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5558ffa99c70 msgr2=0x5558ffa9a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:47.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.688+0000 7f1407800740 1 --2- 172.21.15.77:0/874993592 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5558ffa99c70 0x5558ffa9a050 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f13f8009f00 tx=0x7f13f8011810 comp rx=0 tx=0).stop 2024-09-10T10:02:47.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.688+0000 7f1407800740 1 -- 172.21.15.77:0/874993592 shutdown_connections 2024-09-10T10:02:47.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.688+0000 7f13fde00640 1 --2- 172.21.15.77:0/1161368453 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f13e40a2c00 0x7f13e40a50c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f13f00c42b0 tx=0x7f13f00f5530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:47.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.688+0000 7f1407800740 1 --2- 172.21.15.77:0/874993592 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5558ffaa4e50 0x5558ffaa7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:47.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.688+0000 7f1407800740 1 --2- 172.21.15.77:0/874993592 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5558ffa9a590 0x5558ffaa4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:47.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.688+0000 7f1407800740 1 --2- 172.21.15.77:0/874993592 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5558ffa99c70 0x5558ffa9a050 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:47.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.688+0000 7f1407800740 1 -- 172.21.15.77:0/874993592 >> 172.21.15.77:0/874993592 conn(0x5558ffa96200 msgr2=0x5558ffa965e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:47.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.688+0000 7f1407800740 1 -- 172.21.15.77:0/874993592 shutdown_connections 2024-09-10T10:02:47.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.688+0000 7f1407800740 1 -- 172.21.15.77:0/874993592 wait complete. 2024-09-10T10:02:47.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.689+0000 7f1407800740 1 -- start start 2024-09-10T10:02:47.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.689+0000 7f1407800740 1 Processor -- start 2024-09-10T10:02:47.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.689+0000 7f1407800740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5558ffa99c70 0x5558ffaa1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:47.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.689+0000 7f1407800740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5558ffa9a590 0x5558ffaa21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:47.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.689+0000 7f1407800740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5558ffaa4e50 0x5558ffaa2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:47.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.689+0000 7f1407800740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5558ffaa7a20 con 0x5558ffa9a590 2024-09-10T10:02:47.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.689+0000 7f1407800740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5558ff9bb0f0 con 0x5558ffa99c70 2024-09-10T10:02:47.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.689+0000 7f1407800740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5558ffaa7ba0 con 0x5558ffaa4e50 2024-09-10T10:02:47.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.689+0000 7f1407800740 10 client.0 _ll_register_callbacks cb 0x5558ff9a2bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:02:47.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.690+0000 7f13fd400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5558ffa9a590 0x5558ffaa21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:47.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.690+0000 7f13fd400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5558ffa9a590 0x5558ffaa21e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:35328/0 (socket says 172.21.15.77:35328) 2024-09-10T10:02:47.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.690+0000 7f13fe800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5558ffa99c70 0x5558ffaa1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:47.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.690+0000 7f13fd400640 1 -- 172.21.15.77:0/3581293234 learned_addr learned my addr 172.21.15.77:0/3581293234 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:02:47.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.690+0000 7f13fe800640 1 --2- 172.21.15.77:0/3581293234 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5558ffaa4e50 0x5558ffaa2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:47.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.690+0000 7f13fd400640 1 -- 172.21.15.77:0/3581293234 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5558ffaa4e50 msgr2=0x5558ffaa2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:47.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.690+0000 7f13fd400640 1 --2- 172.21.15.77:0/3581293234 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5558ffaa4e50 0x5558ffaa2720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:47.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.690+0000 7f13fd400640 1 -- 172.21.15.77:0/3581293234 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5558ffa99c70 msgr2=0x5558ffaa1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:47.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.690+0000 7f13fd400640 1 --2- 172.21.15.77:0/3581293234 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5558ffa99c70 0x5558ffaa1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:47.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.690+0000 7f13fd400640 1 -- 172.21.15.77:0/3581293234 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558ff7713c0 con 0x5558ffa9a590 2024-09-10T10:02:47.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.690+0000 7f13fe800640 1 --2- 172.21.15.77:0/3581293234 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5558ffa99c70 0x5558ffaa1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:02:47.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.691+0000 7f13fd400640 1 --2- 172.21.15.77:0/3581293234 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5558ffa9a590 0x5558ffaa21e0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f13ec012730 tx=0x7f13ec012bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41c1bd9326044f8a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:47.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.691+0000 7f13df400640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-10T10:02:47.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.691+0000 7f13df400640 1 -- 172.21.15.77:0/3581293234 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13ec004050 con 0x5558ffa9a590 2024-09-10T10:02:47.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.691+0000 7f1407800740 1 -- 172.21.15.77:0/3581293234 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13f8004050 con 0x5558ffa9a590 2024-09-10T10:02:47.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.691+0000 7f13df400640 1 -- 172.21.15.77:0/3581293234 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f13ec0041f0 con 0x5558ffa9a590 2024-09-10T10:02:47.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.691+0000 7f13df400640 1 -- 172.21.15.77:0/3581293234 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f13ec025050 con 0x5558ffa9a590 2024-09-10T10:02:47.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.691+0000 7f1407800740 10 client.16920 fetch_fsmap learned FSMap version 35 2024-09-10T10:02:47.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.691+0000 7f1407800740 1 -- 172.21.15.77:0/3581293234 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5558ffab3a70 con 0x5558ffa9a590 2024-09-10T10:02:47.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.692+0000 7f13df400640 1 -- 172.21.15.77:0/3581293234 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13f8004050 con 0x5558ffa9a590 2024-09-10T10:02:47.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.692+0000 7f13df400640 1 -- 172.21.15.77:0/3581293234 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f13ec0039d0 con 0x5558ffa9a590 2024-09-10T10:02:47.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.692+0000 7f1407800740 10 client.16920 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T10:02:47.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.692+0000 7f1407800740 10 client.16920 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-10T10:02:47.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.692+0000 7f1407800740 20 client.16920 populate_metadata read hostname 'smithi077' 2024-09-10T10:02:47.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.692+0000 7f1407800740 1 --2- 172.21.15.77:0/3581293234 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x5558ffab4e80 0x5558ffab7270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:47.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.692+0000 7f1407800740 4 client.16920 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:02:47.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.692+0000 7f1407800740 1 -- 172.21.15.77:0/3581293234 --> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5558ffa0d950 con 0x5558ffab4e80 2024-09-10T10:02:47.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.692+0000 7f13fe800640 1 --2- 172.21.15.77:0/3581293234 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x5558ffab4e80 0x5558ffab7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:47.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.693+0000 7f13df400640 10 client.16920 ms_handle_connect on v2:172.21.15.120:6836/3033280709 2024-09-10T10:02:47.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.693+0000 7f13fe800640 1 --2- 172.21.15.77:0/3581293234 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x5558ffab4e80 0x5558ffab7270 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9547cbbce405004d server_cookie=e1f72f004e844feb in_seq=0 out_seq=0 2024-09-10T10:02:47.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.694+0000 7f13df400640 1 -- 172.21.15.77:0/3581293234 <== mds.1 v2:172.21.15.120:6836/3033280709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5558ffa0d950 con 0x5558ffab4e80 2024-09-10T10:02:47.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.694+0000 7f13df400640 10 client.16920 handle_command_reply: tid=0 2024-09-10T10:02:47.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.694+0000 7f1407800740 1 client.16920 shutdown 2024-09-10T10:02:47.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.694+0000 7f1407800740 1 -- 172.21.15.77:0/3581293234 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5558ffa9a590 msgr2=0x5558ffaa21e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:47.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.694+0000 7f1407800740 1 --2- 172.21.15.77:0/3581293234 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5558ffa9a590 0x5558ffaa21e0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f13ec012730 tx=0x7f13ec012bf0 comp rx=0 tx=0).stop 2024-09-10T10:02:47.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.694+0000 7f1407800740 1 -- 172.21.15.77:0/3581293234 shutdown_connections 2024-09-10T10:02:47.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.694+0000 7f1407800740 1 --2- 172.21.15.77:0/3581293234 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x5558ffab4e80 0x5558ffab7270 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:47.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.694+0000 7f1407800740 1 --2- 172.21.15.77:0/3581293234 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5558ffaa4e50 0x5558ffaa2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:47.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.694+0000 7f1407800740 1 --2- 172.21.15.77:0/3581293234 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5558ffa9a590 0x5558ffaa21e0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:47.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.694+0000 7f1407800740 1 --2- 172.21.15.77:0/3581293234 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5558ffa99c70 0x5558ffaa1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:47.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.694+0000 7f13df400640 0 client.16920 ms_handle_reset on v2:172.21.15.120:6836/3033280709 2024-09-10T10:02:47.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.694+0000 7f1407800740 1 -- 172.21.15.77:0/3581293234 >> 172.21.15.77:0/3581293234 conn(0x5558ffa96200 msgr2=0x5558ffa9bee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:47.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.694+0000 7f1407800740 1 -- 172.21.15.77:0/3581293234 shutdown_connections 2024-09-10T10:02:47.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.695+0000 7f1407800740 1 -- 172.21.15.77:0/3581293234 wait complete. 2024-09-10T10:02:47.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.695+0000 7f1407800740 20 client.16920 trim_cache size 0 max 16384 2024-09-10T10:02:47.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.711+0000 7f1407800740 1 Processor -- start 2024-09-10T10:02:47.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.711+0000 7f1407800740 1 -- start start 2024-09-10T10:02:47.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.711+0000 7f1407800740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5558ffa99c70 0x5558ffaa3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:47.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.711+0000 7f1407800740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5558ffa9a590 0x5558ffabcb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:47.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.711+0000 7f1407800740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5558ffaa4e50 0x5558ffabd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:47.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.711+0000 7f1407800740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f14001b1110 con 0x5558ffa9a590 2024-09-10T10:02:47.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.711+0000 7f1407800740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f14001b2c50 con 0x5558ffaa4e50 2024-09-10T10:02:47.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.711+0000 7f1407800740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f14001b17e0 con 0x5558ffa99c70 2024-09-10T10:02:47.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.711+0000 7f13fe800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5558ffa99c70 0x5558ffaa3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:47.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.711+0000 7f13fd400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5558ffa9a590 0x5558ffabcb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:47.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.711+0000 7f13fe800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5558ffa99c70 0x5558ffaa3c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:37992/0 (socket says 172.21.15.77:37992) 2024-09-10T10:02:47.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.711+0000 7f13fe800640 1 -- 172.21.15.77:0/4105350154 learned_addr learned my addr 172.21.15.77:0/4105350154 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:02:47.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.711+0000 7f13fd400640 1 --2- 172.21.15.77:0/4105350154 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5558ffa9a590 0x5558ffabcb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:35330/0 (socket says 172.21.15.77:35330) 2024-09-10T10:02:47.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.712+0000 7f13fd400640 1 -- 172.21.15.77:0/4105350154 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5558ffa99c70 msgr2=0x5558ffaa3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:47.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.712+0000 7f13fd400640 1 --2- 172.21.15.77:0/4105350154 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5558ffa99c70 0x5558ffaa3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:47.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.712+0000 7f13fd400640 1 -- 172.21.15.77:0/4105350154 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5558ffaa4e50 msgr2=0x5558ffabd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:47.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.712+0000 7f13fd400640 1 --2- 172.21.15.77:0/4105350154 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5558ffaa4e50 0x5558ffabd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:47.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.712+0000 7f13fd400640 1 -- 172.21.15.77:0/4105350154 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558ffa9a3c0 con 0x5558ffa9a590 2024-09-10T10:02:47.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.712+0000 7f13fe800640 1 --2- 172.21.15.77:0/4105350154 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5558ffa99c70 0x5558ffaa3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:02:47.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.712+0000 7f13fd400640 1 --2- 172.21.15.77:0/4105350154 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5558ffa9a590 0x5558ffabcb80 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f13ec003d70 tx=0x7f13ec010710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6b0e555347f81be server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:47.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.712+0000 7f13ea600640 1 -- 172.21.15.77:0/4105350154 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13ec0050e0 con 0x5558ffa9a590 2024-09-10T10:02:47.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.713+0000 7f13ea600640 1 -- 172.21.15.77:0/4105350154 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13ec0035f0 con 0x5558ffa9a590 2024-09-10T10:02:47.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.713+0000 7f13ea600640 1 -- 172.21.15.77:0/4105350154 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13ec0109d0 con 0x5558ffa9a590 2024-09-10T10:02:47.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.713+0000 7f1407800740 1 -- 172.21.15.77:0/4105350154 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5558ffa9a590 msgr2=0x5558ffabcb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:47.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.713+0000 7f1407800740 1 --2- 172.21.15.77:0/4105350154 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5558ffa9a590 0x5558ffabcb80 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f13ec003d70 tx=0x7f13ec010710 comp rx=0 tx=0).stop 2024-09-10T10:02:47.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.713+0000 7f1407800740 1 -- 172.21.15.77:0/4105350154 shutdown_connections 2024-09-10T10:02:47.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.713+0000 7f1407800740 1 --2- 172.21.15.77:0/4105350154 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5558ffaa4e50 0x5558ffabd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:47.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.713+0000 7f1407800740 1 --2- 172.21.15.77:0/4105350154 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5558ffa9a590 0x5558ffabcb80 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:47.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.713+0000 7f1407800740 1 --2- 172.21.15.77:0/4105350154 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5558ffa99c70 0x5558ffaa3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:47.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.713+0000 7f1407800740 1 -- 172.21.15.77:0/4105350154 >> 172.21.15.77:0/4105350154 conn(0x5558ffa96200 msgr2=0x5558ffaa3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:47.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.714+0000 7f1407800740 1 -- 172.21.15.77:0/4105350154 shutdown_connections 2024-09-10T10:02:47.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.714+0000 7f1407800740 1 -- 172.21.15.77:0/4105350154 wait complete. 2024-09-10T10:02:47.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.714+0000 7f1407800740 1 -- start start 2024-09-10T10:02:47.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.714+0000 7f1407800740 1 Processor -- start 2024-09-10T10:02:47.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.715+0000 7f1407800740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5558ffa99c70 0x5558ffa9ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:47.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.715+0000 7f1407800740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5558ffa9a590 0x5558ffa9f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:47.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.715+0000 7f1407800740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5558ffaa4e50 0x5558ffa9f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:47.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.715+0000 7f1407800740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f14001b2580 con 0x5558ffaa4e50 2024-09-10T10:02:47.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.715+0000 7f1407800740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f14001b1eb0 con 0x5558ffa9a590 2024-09-10T10:02:47.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.715+0000 7f1407800740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f14001af0f0 con 0x5558ffa99c70 2024-09-10T10:02:47.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.715+0000 7f1407800740 10 client.0 _ll_register_callbacks cb 0x7f13ec009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:02:47.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.715+0000 7f13fd400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5558ffa9a590 0x5558ffa9f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:47.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.715+0000 7f13fd400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5558ffa9a590 0x5558ffa9f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:36744/0 (socket says 172.21.15.77:36744) 2024-09-10T10:02:47.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.715+0000 7f13fd400640 1 -- 172.21.15.77:0/3371084005 learned_addr learned my addr 172.21.15.77:0/3371084005 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:02:47.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.715+0000 7f13fe800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5558ffa99c70 0x5558ffa9ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:47.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.716+0000 7f13fe800640 1 --2- 172.21.15.77:0/3371084005 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5558ffaa4e50 0x5558ffa9f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:47.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.716+0000 7f13fd400640 1 -- 172.21.15.77:0/3371084005 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5558ffa99c70 msgr2=0x5558ffa9ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:47.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.716+0000 7f13fd400640 1 --2- 172.21.15.77:0/3371084005 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5558ffa99c70 0x5558ffa9ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:47.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.716+0000 7f13fd400640 1 -- 172.21.15.77:0/3371084005 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5558ffaa4e50 msgr2=0x5558ffa9f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:47.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.716+0000 7f13fd400640 1 --2- 172.21.15.77:0/3371084005 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5558ffaa4e50 0x5558ffa9f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:47.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.716+0000 7f13fd400640 1 -- 172.21.15.77:0/3371084005 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558ffb1ed50 con 0x5558ffa9a590 2024-09-10T10:02:47.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.717+0000 7f13fd400640 1 --2- 172.21.15.77:0/3371084005 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5558ffa9a590 0x5558ffa9f3b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f13ec006010 tx=0x7f13ec003a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b59cedf5cd49600 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:47.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.717+0000 7f13df400640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T10:02:47.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.717+0000 7f13df400640 1 -- 172.21.15.77:0/3371084005 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13ec036060 con 0x5558ffa9a590 2024-09-10T10:02:47.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.718+0000 7f13df400640 1 -- 172.21.15.77:0/3371084005 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f13ec018040 con 0x5558ffa9a590 2024-09-10T10:02:47.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.718+0000 7f1407800740 1 -- 172.21.15.77:0/3371084005 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13ec0035f0 con 0x5558ffa9a590 2024-09-10T10:02:47.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.718+0000 7f13df400640 1 -- 172.21.15.77:0/3371084005 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f13ec0052d0 con 0x5558ffa9a590 2024-09-10T10:02:47.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.718+0000 7f1407800740 10 client.27056 fetch_fsmap learned FSMap version 35 2024-09-10T10:02:47.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.718+0000 7f1407800740 1 -- 172.21.15.77:0/3371084005 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5558ffb1fff0 con 0x5558ffa9a590 2024-09-10T10:02:47.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.719+0000 7f13df400640 1 -- 172.21.15.77:0/3371084005 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13ec0035f0 con 0x5558ffa9a590 2024-09-10T10:02:47.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.719+0000 7f13df400640 1 -- 172.21.15.77:0/3371084005 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f13ec00f8e0 con 0x5558ffa9a590 2024-09-10T10:02:47.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.719+0000 7f1407800740 10 client.27056 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T10:02:47.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.719+0000 7f1407800740 10 client.27056 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-10T10:02:47.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.719+0000 7f1407800740 20 client.27056 populate_metadata read hostname 'smithi077' 2024-09-10T10:02:47.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.719+0000 7f1407800740 1 --2- 172.21.15.77:0/3371084005 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x5558ffb20660 0x5558ffb22a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:47.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.719+0000 7f1407800740 4 client.27056 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:02:47.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.719+0000 7f1407800740 1 -- 172.21.15.77:0/3371084005 --> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5558ffa07e60 con 0x5558ffb20660 2024-09-10T10:02:47.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.719+0000 7f13fe800640 1 --2- 172.21.15.77:0/3371084005 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x5558ffb20660 0x5558ffb22a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:47.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.720+0000 7f13fe800640 1 --2- 172.21.15.77:0/3371084005 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x5558ffb20660 0x5558ffb22a80 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=57e61c96a926e969 server_cookie=609364d8d7379b63 in_seq=0 out_seq=0 2024-09-10T10:02:47.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.720+0000 7f13df400640 10 client.27056 ms_handle_connect on v2:172.21.15.120:6836/3033280709 2024-09-10T10:02:47.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.720+0000 7f13df400640 1 -- 172.21.15.77:0/3371084005 <== mds.1 v2:172.21.15.120:6836/3033280709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5558ffa07e60 con 0x5558ffb20660 2024-09-10T10:02:47.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.720+0000 7f13df400640 10 client.27056 handle_command_reply: tid=0 2024-09-10T10:02:47.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.720+0000 7f1407800740 1 client.27056 shutdown 2024-09-10T10:02:47.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.720+0000 7f1407800740 1 -- 172.21.15.77:0/3371084005 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5558ffa9a590 msgr2=0x5558ffa9f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:47.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.720+0000 7f1407800740 1 --2- 172.21.15.77:0/3371084005 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5558ffa9a590 0x5558ffa9f3b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f13ec006010 tx=0x7f13ec003a50 comp rx=0 tx=0).stop 2024-09-10T10:02:47.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.721+0000 7f1407800740 1 -- 172.21.15.77:0/3371084005 shutdown_connections 2024-09-10T10:02:47.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.721+0000 7f1407800740 1 --2- 172.21.15.77:0/3371084005 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x5558ffb20660 0x5558ffb22a80 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:47.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.721+0000 7f1407800740 1 --2- 172.21.15.77:0/3371084005 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5558ffaa4e50 0x5558ffa9f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:47.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.721+0000 7f1407800740 1 --2- 172.21.15.77:0/3371084005 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5558ffa9a590 0x5558ffa9f3b0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:47.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.721+0000 7f1407800740 1 --2- 172.21.15.77:0/3371084005 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5558ffa99c70 0x5558ffa9ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:47.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.721+0000 7f1407800740 1 -- 172.21.15.77:0/3371084005 >> 172.21.15.77:0/3371084005 conn(0x5558ffa96200 msgr2=0x5558ffaa2480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:47.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.721+0000 7f13df400640 0 client.27056 ms_handle_reset on v2:172.21.15.120:6836/3033280709 2024-09-10T10:02:47.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.721+0000 7f1407800740 1 -- 172.21.15.77:0/3371084005 shutdown_connections 2024-09-10T10:02:47.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.721+0000 7f1407800740 1 -- 172.21.15.77:0/3371084005 wait complete. 2024-09-10T10:02:47.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.721+0000 7f1407800740 20 client.27056 trim_cache size 0 max 16384 2024-09-10T10:02:47.723 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-10T10:02:47.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.722+0000 7f1404c00640 1 -- 172.21.15.77:0/1161368453 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f13e40a2c00 msgr2=0x7f13e40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:47.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.722+0000 7f1404c00640 1 --2- 172.21.15.77:0/1161368453 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f13e40a2c00 0x7f13e40a50c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f13f00c42b0 tx=0x7f13f00f5530 comp rx=0 tx=0).stop 2024-09-10T10:02:47.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.722+0000 7f1404c00640 1 -- 172.21.15.77:0/1161368453 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1400102d30 msgr2=0x7f1400198600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:47.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.722+0000 7f1404c00640 1 --2- 172.21.15.77:0/1161368453 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1400102d30 0x7f1400198600 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f13f0009230 tx=0x7f13f0004040 comp rx=0 tx=0).stop 2024-09-10T10:02:47.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.722+0000 7f1404c00640 1 -- 172.21.15.77:0/1161368453 shutdown_connections 2024-09-10T10:02:47.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.722+0000 7f1404c00640 1 --2- 172.21.15.77:0/1161368453 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f13e40a2c00 0x7f13e40a50c0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:47.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.722+0000 7f1404c00640 1 --2- 172.21.15.77:0/1161368453 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f14001082a0 0x7f140019ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:47.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.722+0000 7f1404c00640 1 --2- 172.21.15.77:0/1161368453 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f14001036e0 0x7f1400198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:47.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.722+0000 7f1404c00640 1 --2- 172.21.15.77:0/1161368453 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1400102d30 0x7f1400198600 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:47.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.722+0000 7f1404c00640 1 -- 172.21.15.77:0/1161368453 >> 172.21.15.77:0/1161368453 conn(0x7f14000fc7d0 msgr2=0x7f1400105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:47.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.722+0000 7f1404c00640 1 -- 172.21.15.77:0/1161368453 shutdown_connections 2024-09-10T10:02:47.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:47.722+0000 7f1404c00640 1 -- 172.21.15.77:0/1161368453 wait complete. 2024-09-10T10:02:48.055 INFO:tasks.workunit.client.0.smithi077.stdout:copying to largest ever: 0x63ed6b9 2024-09-10T10:02:48.913 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T10:02:49.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:02:48 smithi077 ceph-mon[30319]: pgmap v620: 129 pgs: 129 active+clean; 146 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 1.1 MiB/s wr, 433 op/s 2024-09-10T10:02:49.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:02:48 smithi134 ceph-mon[35674]: pgmap v620: 129 pgs: 129 active+clean; 146 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 1.1 MiB/s wr, 433 op/s 2024-09-10T10:02:49.389 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:02:49.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:02:48 smithi120 ceph-mon[34770]: pgmap v620: 129 pgs: 129 active+clean; 146 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 1.1 MiB/s wr, 433 op/s 2024-09-10T10:02:50.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.786+0000 7f7b6d200640 1 -- 172.21.15.77:0/639298888 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7b6806ea30 msgr2=0x7f7b6806ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:50.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.786+0000 7f7b6d200640 1 --2- 172.21.15.77:0/639298888 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7b6806ea30 0x7f7b6806ee70 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f7b54009900 tx=0x7f7b5402a3f0 comp rx=0 tx=0).stop 2024-09-10T10:02:50.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.787+0000 7f7b6d200640 1 -- 172.21.15.77:0/639298888 shutdown_connections 2024-09-10T10:02:50.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.787+0000 7f7b6d200640 1 --2- 172.21.15.77:0/639298888 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b6811b140 0x7f7b6811d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:50.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.787+0000 7f7b6d200640 1 --2- 172.21.15.77:0/639298888 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7b6806ea30 0x7f7b6806ee70 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:50.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.787+0000 7f7b6d200640 1 --2- 172.21.15.77:0/639298888 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7b68110190 0x7f7b6806e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:50.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.787+0000 7f7b6d200640 1 -- 172.21.15.77:0/639298888 >> 172.21.15.77:0/639298888 conn(0x7f7b68073180 msgr2=0x7f7b68073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:50.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.787+0000 7f7b6d200640 1 -- 172.21.15.77:0/639298888 shutdown_connections 2024-09-10T10:02:50.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.787+0000 7f7b6d200640 1 -- 172.21.15.77:0/639298888 wait complete. 2024-09-10T10:02:50.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.788+0000 7f7b6d200640 1 Processor -- start 2024-09-10T10:02:50.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.788+0000 7f7b6d200640 1 -- start start 2024-09-10T10:02:50.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.788+0000 7f7b6d200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7b6806ea30 0x7f7b681a70e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:50.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.788+0000 7f7b6d200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b68110190 0x7f7b681a7620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:50.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.788+0000 7f7b6d200640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7b6811b140 0x7f7b681ab9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:50.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.788+0000 7f7b6d200640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7b68120be0 con 0x7f7b6806ea30 2024-09-10T10:02:50.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.788+0000 7f7b6d200640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7b68120a60 con 0x7f7b6811b140 2024-09-10T10:02:50.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.788+0000 7f7b6d200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7b68120d60 con 0x7f7b68110190 2024-09-10T10:02:50.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.788+0000 7f7b67e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7b6811b140 0x7f7b681ab9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:50.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.789+0000 7f7b67e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7b6811b140 0x7f7b681ab9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:36772/0 (socket says 172.21.15.77:36772) 2024-09-10T10:02:50.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.789+0000 7f7b67e00640 1 -- 172.21.15.77:0/1433857547 learned_addr learned my addr 172.21.15.77:0/1433857547 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:02:50.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.789+0000 7f7b66a00640 1 --2- 172.21.15.77:0/1433857547 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b68110190 0x7f7b681a7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:50.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.789+0000 7f7b67400640 1 --2- 172.21.15.77:0/1433857547 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7b6806ea30 0x7f7b681a70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:50.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.789+0000 7f7b67e00640 1 -- 172.21.15.77:0/1433857547 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b68110190 msgr2=0x7f7b681a7620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:50.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.789+0000 7f7b67e00640 1 --2- 172.21.15.77:0/1433857547 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b68110190 0x7f7b681a7620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:50.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.789+0000 7f7b67e00640 1 -- 172.21.15.77:0/1433857547 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7b6806ea30 msgr2=0x7f7b681a70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:50.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.789+0000 7f7b67e00640 1 --2- 172.21.15.77:0/1433857547 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7b6806ea30 0x7f7b681a70e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:50.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.789+0000 7f7b67e00640 1 -- 172.21.15.77:0/1433857547 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b681ac090 con 0x7f7b6811b140 2024-09-10T10:02:50.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.789+0000 7f7b66a00640 1 --2- 172.21.15.77:0/1433857547 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b68110190 0x7f7b681a7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:02:50.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.789+0000 7f7b67e00640 1 --2- 172.21.15.77:0/1433857547 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7b6811b140 0x7f7b681ab9b0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f7b6000e920 tx=0x7f7b6000ede0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:50.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.790+0000 7f7b5fe00640 1 -- 172.21.15.77:0/1433857547 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b60004d20 con 0x7f7b6811b140 2024-09-10T10:02:50.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.790+0000 7f7b5fe00640 1 -- 172.21.15.77:0/1433857547 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b60005020 con 0x7f7b6811b140 2024-09-10T10:02:50.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.790+0000 7f7b5fe00640 1 -- 172.21.15.77:0/1433857547 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b60011690 con 0x7f7b6811b140 2024-09-10T10:02:50.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.790+0000 7f7b6d200640 1 -- 172.21.15.77:0/1433857547 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b681ac380 con 0x7f7b6811b140 2024-09-10T10:02:50.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.790+0000 7f7b6d200640 1 -- 172.21.15.77:0/1433857547 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b681b3c60 con 0x7f7b6811b140 2024-09-10T10:02:50.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.793+0000 7f7b5fe00640 1 -- 172.21.15.77:0/1433857547 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7b600051d0 con 0x7f7b6811b140 2024-09-10T10:02:50.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.793+0000 7f7b7123d740 1 Processor -- start 2024-09-10T10:02:50.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.794+0000 7f7b7123d740 1 -- start start 2024-09-10T10:02:50.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.794+0000 7f7b7123d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55be5278fe40 0x55be52790220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:50.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.794+0000 7f7b7123d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55be52790760 0x55be5279aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:50.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.794+0000 7f7b66a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55be5278fe40 0x55be52790220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:50.797 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.794+0000 7f7b66a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55be5278fe40 0x55be52790220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:35382/0 (socket says 172.21.15.77:35382) 2024-09-10T10:02:50.797 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.794+0000 7f7b7123d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55be5279b020 0x55be5279d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:50.797 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.795+0000 7f7b66a00640 1 -- 172.21.15.77:0/3806302650 learned_addr learned my addr 172.21.15.77:0/3806302650 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:02:50.797 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.795+0000 7f7b7123d740 1 -- 172.21.15.77:0/3806302650 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55be526f63e0 con 0x55be5278fe40 2024-09-10T10:02:50.797 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.795+0000 7f7b7123d740 1 -- 172.21.15.77:0/3806302650 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55be5269dd80 con 0x55be5279b020 2024-09-10T10:02:50.797 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.795+0000 7f7b7123d740 1 -- 172.21.15.77:0/3806302650 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55be526f23a0 con 0x55be52790760 2024-09-10T10:02:50.797 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.795+0000 7f7b67400640 1 --2- 172.21.15.77:0/3806302650 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55be52790760 0x55be5279aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:50.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.795+0000 7f7b5fe00640 1 --2- 172.21.15.77:0/1433857547 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f7b4c0a2c00 0x7f7b4c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:50.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.796+0000 7f7b66a00640 1 --2- 172.21.15.77:0/3806302650 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55be5279b020 0x55be5279d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:50.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.796+0000 7f7b67400640 1 -- 172.21.15.77:0/3806302650 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55be5279b020 msgr2=0x55be5279d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:50.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.796+0000 7f7b67400640 1 --2- 172.21.15.77:0/3806302650 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55be5279b020 0x55be5279d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:50.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.796+0000 7f7b67400640 1 -- 172.21.15.77:0/3806302650 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55be5278fe40 msgr2=0x55be52790220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:50.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.796+0000 7f7b67400640 1 --2- 172.21.15.77:0/3806302650 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55be5278fe40 0x55be52790220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:50.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.796+0000 7f7b67400640 1 -- 172.21.15.77:0/3806302650 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be526bcae0 con 0x55be52790760 2024-09-10T10:02:50.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.796+0000 7f7b5fe00640 1 -- 172.21.15.77:0/1433857547 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f7b600cf770 con 0x7f7b6811b140 2024-09-10T10:02:50.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.796+0000 7f7b66a00640 1 --2- 172.21.15.77:0/3806302650 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55be5278fe40 0x55be52790220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:02:50.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.796+0000 7f7b67e00640 1 --2- 172.21.15.77:0/1433857547 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f7b4c0a2c00 0x7f7b4c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:50.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.796+0000 7f7b67400640 1 --2- 172.21.15.77:0/3806302650 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55be52790760 0x55be5279aae0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f7b58005fd0 tx=0x7f7b580107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e87c7c4f679c5e17 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:50.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.797+0000 7f7b5cc00640 1 -- 172.21.15.77:0/3806302650 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b5801c480 con 0x55be52790760 2024-09-10T10:02:50.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.797+0000 7f7b5cc00640 1 -- 172.21.15.77:0/3806302650 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b5801cb00 con 0x55be52790760 2024-09-10T10:02:50.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.797+0000 7f7b5cc00640 1 -- 172.21.15.77:0/3806302650 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b58011bc0 con 0x55be52790760 2024-09-10T10:02:50.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.797+0000 7f7b7123d740 1 -- 172.21.15.77:0/3806302650 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55be52790760 msgr2=0x55be5279aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:50.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.797+0000 7f7b7123d740 1 --2- 172.21.15.77:0/3806302650 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55be52790760 0x55be5279aae0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f7b58005fd0 tx=0x7f7b580107a0 comp rx=0 tx=0).stop 2024-09-10T10:02:50.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.797+0000 7f7b67e00640 1 --2- 172.21.15.77:0/1433857547 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f7b4c0a2c00 0x7f7b4c0a50c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f7b6009b930 tx=0x7f7b60005840 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:50.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.798+0000 7f7b7123d740 1 -- 172.21.15.77:0/3806302650 shutdown_connections 2024-09-10T10:02:50.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.798+0000 7f7b7123d740 1 --2- 172.21.15.77:0/3806302650 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55be5279b020 0x55be5279d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:50.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.798+0000 7f7b7123d740 1 --2- 172.21.15.77:0/3806302650 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55be52790760 0x55be5279aae0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:50.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.798+0000 7f7b7123d740 1 --2- 172.21.15.77:0/3806302650 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55be5278fe40 0x55be52790220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:50.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.798+0000 7f7b7123d740 1 -- 172.21.15.77:0/3806302650 >> 172.21.15.77:0/3806302650 conn(0x55be5278c3d0 msgr2=0x55be5278c7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:50.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.798+0000 7f7b7123d740 1 -- 172.21.15.77:0/3806302650 shutdown_connections 2024-09-10T10:02:50.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.798+0000 7f7b7123d740 1 -- 172.21.15.77:0/3806302650 wait complete. 2024-09-10T10:02:50.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.799+0000 7f7b7123d740 1 -- start start 2024-09-10T10:02:50.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.799+0000 7f7b7123d740 1 Processor -- start 2024-09-10T10:02:50.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.799+0000 7f7b7123d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55be5278fe40 0x55be52797f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:50.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.799+0000 7f7b7123d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55be52790760 0x55be527984c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:50.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.799+0000 7f7b7123d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55be5279b020 0x55be52798a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:50.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.799+0000 7f7b7123d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55be5279dbf0 con 0x55be52790760 2024-09-10T10:02:50.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.800+0000 7f7b7123d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55be526b10f0 con 0x55be5278fe40 2024-09-10T10:02:50.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.800+0000 7f7b7123d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55be5279dd70 con 0x55be5279b020 2024-09-10T10:02:50.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.800+0000 7f7b7123d740 10 client.0 _ll_register_callbacks cb 0x55be52698bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:02:50.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.800+0000 7f7b66a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55be5278fe40 0x55be52797f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:50.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.800+0000 7f7b66a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55be5278fe40 0x55be52797f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:36788/0 (socket says 172.21.15.77:36788) 2024-09-10T10:02:50.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.800+0000 7f7b66a00640 1 -- 172.21.15.77:0/4205294447 learned_addr learned my addr 172.21.15.77:0/4205294447 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:02:50.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.800+0000 7f7b66a00640 1 --2- 172.21.15.77:0/4205294447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55be5279b020 0x55be52798a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:50.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.800+0000 7f7b67400640 1 --2- 172.21.15.77:0/4205294447 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55be52790760 0x55be527984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:50.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.800+0000 7f7b66a00640 1 -- 172.21.15.77:0/4205294447 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55be5278fe40 msgr2=0x55be52797f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:50.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.800+0000 7f7b66a00640 1 --2- 172.21.15.77:0/4205294447 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55be5278fe40 0x55be52797f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:50.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.800+0000 7f7b66a00640 1 -- 172.21.15.77:0/4205294447 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55be52790760 msgr2=0x55be527984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:50.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.800+0000 7f7b66a00640 1 --2- 172.21.15.77:0/4205294447 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55be52790760 0x55be527984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:50.804 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.800+0000 7f7b66a00640 1 -- 172.21.15.77:0/4205294447 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be524673c0 con 0x55be5279b020 2024-09-10T10:02:50.804 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.801+0000 7f7b66a00640 1 --2- 172.21.15.77:0/4205294447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55be5279b020 0x55be52798a00 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f7b54000c00 tx=0x7f7b54009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=79abbaf18c4f635b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:50.805 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.801+0000 7f7b51600640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-10T10:02:50.805 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.801+0000 7f7b51600640 1 -- 172.21.15.77:0/4205294447 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b54035040 con 0x55be5279b020 2024-09-10T10:02:50.805 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.801+0000 7f7b51600640 1 -- 172.21.15.77:0/4205294447 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f7b54004970 con 0x55be5279b020 2024-09-10T10:02:50.805 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.801+0000 7f7b7123d740 1 -- 172.21.15.77:0/4205294447 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b5801cb00 con 0x55be5279b020 2024-09-10T10:02:50.805 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.801+0000 7f7b51600640 1 -- 172.21.15.77:0/4205294447 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f7b54004b70 con 0x55be5279b020 2024-09-10T10:02:50.805 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.801+0000 7f7b7123d740 10 client.27031 fetch_fsmap learned FSMap version 35 2024-09-10T10:02:50.805 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.801+0000 7f7b7123d740 1 -- 172.21.15.77:0/4205294447 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55be527a9ce0 con 0x55be5279b020 2024-09-10T10:02:50.805 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.802+0000 7f7b51600640 1 -- 172.21.15.77:0/4205294447 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b5801cb00 con 0x55be5279b020 2024-09-10T10:02:50.805 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.802+0000 7f7b51600640 1 -- 172.21.15.77:0/4205294447 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f7b5404a020 con 0x55be5279b020 2024-09-10T10:02:50.805 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.802+0000 7f7b7123d740 10 client.27031 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T10:02:50.805 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.802+0000 7f7b7123d740 10 client.27031 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T10:02:50.805 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.802+0000 7f7b7123d740 20 client.27031 populate_metadata read hostname 'smithi077' 2024-09-10T10:02:50.805 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.802+0000 7f7b7123d740 1 --2- 172.21.15.77:0/4205294447 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55be527ab1a0 0x55be527ad590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:50.805 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.802+0000 7f7b7123d740 4 client.27031 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:02:50.806 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.802+0000 7f7b7123d740 1 -- 172.21.15.77:0/4205294447 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55be52703950 con 0x55be527ab1a0 2024-09-10T10:02:50.806 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.803+0000 7f7b67400640 1 --2- 172.21.15.77:0/4205294447 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55be527ab1a0 0x55be527ad590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:50.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.803+0000 7f7b67400640 1 --2- 172.21.15.77:0/4205294447 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55be527ab1a0 0x55be527ad590 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e57c793208035bfc server_cookie=80f4e47397b373b5 in_seq=0 out_seq=0 2024-09-10T10:02:50.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.803+0000 7f7b51600640 10 client.27031 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T10:02:50.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.804+0000 7f7b51600640 1 -- 172.21.15.77:0/4205294447 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55be52703950 con 0x55be527ab1a0 2024-09-10T10:02:50.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.804+0000 7f7b51600640 10 client.27031 handle_command_reply: tid=0 2024-09-10T10:02:50.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.804+0000 7f7b7123d740 1 client.27031 shutdown 2024-09-10T10:02:50.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.804+0000 7f7b7123d740 1 -- 172.21.15.77:0/4205294447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55be5279b020 msgr2=0x55be52798a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:50.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.804+0000 7f7b7123d740 1 --2- 172.21.15.77:0/4205294447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55be5279b020 0x55be52798a00 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f7b54000c00 tx=0x7f7b54009260 comp rx=0 tx=0).stop 2024-09-10T10:02:50.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.804+0000 7f7b7123d740 1 -- 172.21.15.77:0/4205294447 shutdown_connections 2024-09-10T10:02:50.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.804+0000 7f7b7123d740 1 --2- 172.21.15.77:0/4205294447 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55be527ab1a0 0x55be527ad590 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:50.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.804+0000 7f7b7123d740 1 --2- 172.21.15.77:0/4205294447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55be5279b020 0x55be52798a00 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:50.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.804+0000 7f7b7123d740 1 --2- 172.21.15.77:0/4205294447 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55be52790760 0x55be527984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:50.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.804+0000 7f7b7123d740 1 --2- 172.21.15.77:0/4205294447 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55be5278fe40 0x55be52797f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:50.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.804+0000 7f7b51600640 0 client.27031 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T10:02:50.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.804+0000 7f7b7123d740 1 -- 172.21.15.77:0/4205294447 >> 172.21.15.77:0/4205294447 conn(0x55be5278c3d0 msgr2=0x55be52792100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:50.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.804+0000 7f7b7123d740 1 -- 172.21.15.77:0/4205294447 shutdown_connections 2024-09-10T10:02:50.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.805+0000 7f7b7123d740 1 -- 172.21.15.77:0/4205294447 wait complete. 2024-09-10T10:02:50.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.805+0000 7f7b7123d740 20 client.27031 trim_cache size 0 max 16384 2024-09-10T10:02:50.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.820+0000 7f7b7123d740 1 Processor -- start 2024-09-10T10:02:50.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.820+0000 7f7b7123d740 1 -- start start 2024-09-10T10:02:50.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.821+0000 7f7b7123d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55be5278fe40 0x55be527984d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:50.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.821+0000 7f7b7123d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55be52790760 0x55be52798a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:50.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.821+0000 7f7b7123d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55be527b2d50 0x55be527b3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:50.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.821+0000 7f7b7123d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7b681bdb60 con 0x55be5278fe40 2024-09-10T10:02:50.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.821+0000 7f7b7123d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7b681bf6a0 con 0x55be52790760 2024-09-10T10:02:50.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.821+0000 7f7b7123d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7b681be230 con 0x55be527b2d50 2024-09-10T10:02:50.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.821+0000 7f7b67400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55be52790760 0x55be52798a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:50.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.821+0000 7f7b67400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55be52790760 0x55be52798a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:36800/0 (socket says 172.21.15.77:36800) 2024-09-10T10:02:50.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.821+0000 7f7b67400640 1 -- 172.21.15.77:0/3802738233 learned_addr learned my addr 172.21.15.77:0/3802738233 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:02:50.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.821+0000 7f7b66a00640 1 --2- 172.21.15.77:0/3802738233 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55be5278fe40 0x55be527984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:50.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.821+0000 7f7b67400640 1 -- 172.21.15.77:0/3802738233 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55be527b2d50 msgr2=0x55be527b3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:50.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.821+0000 7f7b67400640 1 --2- 172.21.15.77:0/3802738233 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55be527b2d50 0x55be527b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:50.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.821+0000 7f7b67400640 1 -- 172.21.15.77:0/3802738233 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55be5278fe40 msgr2=0x55be527984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:50.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.821+0000 7f7b67400640 1 --2- 172.21.15.77:0/3802738233 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55be5278fe40 0x55be527984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:50.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.821+0000 7f7b67400640 1 -- 172.21.15.77:0/3802738233 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be527adb20 con 0x55be52790760 2024-09-10T10:02:50.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.821+0000 7f7b66a00640 1 --2- 172.21.15.77:0/3802738233 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55be5278fe40 0x55be527984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:02:50.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.822+0000 7f7b67400640 1 --2- 172.21.15.77:0/3802738233 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55be52790760 0x55be52798a10 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x55be5278d230 tx=0x7f7b580116e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4047aedb436cd2d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:50.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.822+0000 7f7b53e00640 1 -- 172.21.15.77:0/3802738233 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b58011890 con 0x55be52790760 2024-09-10T10:02:50.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.822+0000 7f7b53e00640 1 -- 172.21.15.77:0/3802738233 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b58005040 con 0x55be52790760 2024-09-10T10:02:50.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.822+0000 7f7b53e00640 1 -- 172.21.15.77:0/3802738233 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b5801c770 con 0x55be52790760 2024-09-10T10:02:50.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.823+0000 7f7b7123d740 1 -- 172.21.15.77:0/3802738233 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55be52790760 msgr2=0x55be52798a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:50.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.823+0000 7f7b7123d740 1 --2- 172.21.15.77:0/3802738233 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55be52790760 0x55be52798a10 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x55be5278d230 tx=0x7f7b580116e0 comp rx=0 tx=0).stop 2024-09-10T10:02:50.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.823+0000 7f7b7123d740 1 -- 172.21.15.77:0/3802738233 shutdown_connections 2024-09-10T10:02:50.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.823+0000 7f7b7123d740 1 --2- 172.21.15.77:0/3802738233 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55be527b2d50 0x55be527b3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:50.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.823+0000 7f7b7123d740 1 --2- 172.21.15.77:0/3802738233 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55be52790760 0x55be52798a10 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:50.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.823+0000 7f7b7123d740 1 --2- 172.21.15.77:0/3802738233 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55be5278fe40 0x55be527984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:50.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.823+0000 7f7b7123d740 1 -- 172.21.15.77:0/3802738233 >> 172.21.15.77:0/3802738233 conn(0x55be5278c3d0 msgr2=0x55be52792100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:50.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.823+0000 7f7b7123d740 1 -- 172.21.15.77:0/3802738233 shutdown_connections 2024-09-10T10:02:50.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.823+0000 7f7b7123d740 1 -- 172.21.15.77:0/3802738233 wait complete. 2024-09-10T10:02:50.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.823+0000 7f7b7123d740 1 -- start start 2024-09-10T10:02:50.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.823+0000 7f7b7123d740 1 Processor -- start 2024-09-10T10:02:50.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.824+0000 7f7b7123d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55be5278fe40 0x55be527a3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:50.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.824+0000 7f7b7123d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55be52790760 0x55be5279f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:50.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.824+0000 7f7b7123d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55be527b2d50 0x55be5279fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:50.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.824+0000 7f7b7123d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7b681befd0 con 0x55be527b2d50 2024-09-10T10:02:50.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.824+0000 7f7b7123d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7b681be900 con 0x55be52790760 2024-09-10T10:02:50.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.824+0000 7f7b7123d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7b681bbb40 con 0x55be5278fe40 2024-09-10T10:02:50.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.824+0000 7f7b7123d740 10 client.0 _ll_register_callbacks cb 0x7f7b540044d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:02:50.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.824+0000 7f7b66a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55be5278fe40 0x55be527a3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:50.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.824+0000 7f7b67400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55be52790760 0x55be5279f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:50.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.824+0000 7f7b67400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55be52790760 0x55be5279f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:36806/0 (socket says 172.21.15.77:36806) 2024-09-10T10:02:50.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.824+0000 7f7b67400640 1 -- 172.21.15.77:0/1459539810 learned_addr learned my addr 172.21.15.77:0/1459539810 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:02:50.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.824+0000 7f7b66a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55be527b2d50 0x55be5279fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:50.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.825+0000 7f7b67400640 1 -- 172.21.15.77:0/1459539810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55be5278fe40 msgr2=0x55be527a3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:50.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.825+0000 7f7b67400640 1 --2- 172.21.15.77:0/1459539810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55be5278fe40 0x55be527a3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:50.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.825+0000 7f7b67400640 1 -- 172.21.15.77:0/1459539810 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55be527b2d50 msgr2=0x55be5279fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:50.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.825+0000 7f7b67400640 1 --2- 172.21.15.77:0/1459539810 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55be527b2d50 0x55be5279fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:50.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.825+0000 7f7b67400640 1 -- 172.21.15.77:0/1459539810 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be5278d060 con 0x55be52790760 2024-09-10T10:02:50.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.825+0000 7f7b67400640 1 --2- 172.21.15.77:0/1459539810 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55be52790760 0x55be5279f8a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f7b58010ca0 tx=0x7f7b580115a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1bd22cae179e9422 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:50.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.825+0000 7f7b51600640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T10:02:50.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.825+0000 7f7b51600640 1 -- 172.21.15.77:0/1459539810 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b58005600 con 0x55be52790760 2024-09-10T10:02:50.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.825+0000 7f7b51600640 1 -- 172.21.15.77:0/1459539810 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f7b5801c4f0 con 0x55be52790760 2024-09-10T10:02:50.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.825+0000 7f7b7123d740 1 -- 172.21.15.77:0/1459539810 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b58005040 con 0x55be52790760 2024-09-10T10:02:50.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.826+0000 7f7b51600640 1 -- 172.21.15.77:0/1459539810 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f7b58041050 con 0x55be52790760 2024-09-10T10:02:50.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.826+0000 7f7b7123d740 10 client.27074 fetch_fsmap learned FSMap version 35 2024-09-10T10:02:50.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.826+0000 7f7b7123d740 1 -- 172.21.15.77:0/1459539810 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55be52816970 con 0x55be52790760 2024-09-10T10:02:50.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.826+0000 7f7b51600640 1 -- 172.21.15.77:0/1459539810 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b58005040 con 0x55be52790760 2024-09-10T10:02:50.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.826+0000 7f7b51600640 1 -- 172.21.15.77:0/1459539810 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f7b58004680 con 0x55be52790760 2024-09-10T10:02:50.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.826+0000 7f7b7123d740 10 client.27074 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T10:02:50.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.826+0000 7f7b7123d740 10 client.27074 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T10:02:50.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.826+0000 7f7b7123d740 20 client.27074 populate_metadata read hostname 'smithi077' 2024-09-10T10:02:50.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.826+0000 7f7b7123d740 1 --2- 172.21.15.77:0/1459539810 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55be52816e40 0x55be52819260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:50.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.826+0000 7f7b7123d740 4 client.27074 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:02:50.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.826+0000 7f7b7123d740 1 -- 172.21.15.77:0/1459539810 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55be527879a0 con 0x55be52816e40 2024-09-10T10:02:50.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.827+0000 7f7b66a00640 1 --2- 172.21.15.77:0/1459539810 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55be52816e40 0x55be52819260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:50.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.827+0000 7f7b66a00640 1 --2- 172.21.15.77:0/1459539810 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55be52816e40 0x55be52819260 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7b9eb833f452829 server_cookie=ab97547609f9894f in_seq=0 out_seq=0 2024-09-10T10:02:50.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.827+0000 7f7b51600640 10 client.27074 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T10:02:50.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.827+0000 7f7b51600640 1 -- 172.21.15.77:0/1459539810 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55be527879a0 con 0x55be52816e40 2024-09-10T10:02:50.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.827+0000 7f7b51600640 10 client.27074 handle_command_reply: tid=0 2024-09-10T10:02:50.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.827+0000 7f7b7123d740 1 client.27074 shutdown 2024-09-10T10:02:50.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.827+0000 7f7b7123d740 1 -- 172.21.15.77:0/1459539810 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55be52790760 msgr2=0x55be5279f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:50.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.827+0000 7f7b7123d740 1 --2- 172.21.15.77:0/1459539810 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55be52790760 0x55be5279f8a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f7b58010ca0 tx=0x7f7b580115a0 comp rx=0 tx=0).stop 2024-09-10T10:02:50.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.827+0000 7f7b7123d740 1 -- 172.21.15.77:0/1459539810 shutdown_connections 2024-09-10T10:02:50.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.827+0000 7f7b7123d740 1 --2- 172.21.15.77:0/1459539810 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55be52816e40 0x55be52819260 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:50.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.828+0000 7f7b7123d740 1 --2- 172.21.15.77:0/1459539810 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55be527b2d50 0x55be5279fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:50.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.828+0000 7f7b7123d740 1 --2- 172.21.15.77:0/1459539810 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55be52790760 0x55be5279f8a0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:50.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.828+0000 7f7b7123d740 1 --2- 172.21.15.77:0/1459539810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55be5278fe40 0x55be527a3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:50.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.828+0000 7f7b7123d740 1 -- 172.21.15.77:0/1459539810 >> 172.21.15.77:0/1459539810 conn(0x55be5278c3d0 msgr2=0x55be52792100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:50.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.828+0000 7f7b51600640 0 client.27074 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T10:02:50.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.828+0000 7f7b7123d740 1 -- 172.21.15.77:0/1459539810 shutdown_connections 2024-09-10T10:02:50.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.828+0000 7f7b7123d740 1 -- 172.21.15.77:0/1459539810 wait complete. 2024-09-10T10:02:50.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.828+0000 7f7b7123d740 20 client.27074 trim_cache size 0 max 16384 2024-09-10T10:02:50.833 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-10T10:02:50.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.829+0000 7f7b6d200640 1 -- 172.21.15.77:0/1433857547 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f7b4c0a2c00 msgr2=0x7f7b4c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:50.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.829+0000 7f7b6d200640 1 --2- 172.21.15.77:0/1433857547 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f7b4c0a2c00 0x7f7b4c0a50c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f7b6009b930 tx=0x7f7b60005840 comp rx=0 tx=0).stop 2024-09-10T10:02:50.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.829+0000 7f7b6d200640 1 -- 172.21.15.77:0/1433857547 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7b6811b140 msgr2=0x7f7b681ab9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:50.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.829+0000 7f7b6d200640 1 --2- 172.21.15.77:0/1433857547 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7b6811b140 0x7f7b681ab9b0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f7b6000e920 tx=0x7f7b6000ede0 comp rx=0 tx=0).stop 2024-09-10T10:02:50.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.829+0000 7f7b6d200640 1 -- 172.21.15.77:0/1433857547 shutdown_connections 2024-09-10T10:02:50.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.829+0000 7f7b6d200640 1 --2- 172.21.15.77:0/1433857547 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f7b4c0a2c00 0x7f7b4c0a50c0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:50.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.829+0000 7f7b6d200640 1 --2- 172.21.15.77:0/1433857547 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7b6811b140 0x7f7b681ab9b0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:50.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.829+0000 7f7b6d200640 1 --2- 172.21.15.77:0/1433857547 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b68110190 0x7f7b681a7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:50.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.829+0000 7f7b6d200640 1 --2- 172.21.15.77:0/1433857547 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7b6806ea30 0x7f7b681a70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:50.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.829+0000 7f7b6d200640 1 -- 172.21.15.77:0/1433857547 >> 172.21.15.77:0/1433857547 conn(0x7f7b68073180 msgr2=0x7f7b681192d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:50.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.829+0000 7f7b6d200640 1 -- 172.21.15.77:0/1433857547 shutdown_connections 2024-09-10T10:02:50.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:50.829+0000 7f7b6d200640 1 -- 172.21.15.77:0/1433857547 wait complete. 2024-09-10T10:02:51.360 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:02:50 smithi077 ceph-mon[30319]: pgmap v621: 129 pgs: 129 active+clean; 153 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.3 MiB/s wr, 511 op/s 2024-09-10T10:02:51.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:02:50 smithi134 ceph-mon[35674]: pgmap v621: 129 pgs: 129 active+clean; 153 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.3 MiB/s wr, 511 op/s 2024-09-10T10:02:51.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:02:50 smithi120 ceph-mon[34770]: pgmap v621: 129 pgs: 129 active+clean; 153 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.3 MiB/s wr, 511 op/s 2024-09-10T10:02:52.857 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T10:02:52.857 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T10:02:53.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:02:52 smithi077 ceph-mon[30319]: pgmap v622: 129 pgs: 129 active+clean; 151 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 1.1 MiB/s wr, 444 op/s 2024-09-10T10:02:53.333 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:02:53.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:02:52 smithi134 ceph-mon[35674]: pgmap v622: 129 pgs: 129 active+clean; 151 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 1.1 MiB/s wr, 444 op/s 2024-09-10T10:02:53.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:02:52 smithi120 ceph-mon[34770]: pgmap v622: 129 pgs: 129 active+clean; 151 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 1.1 MiB/s wr, 444 op/s 2024-09-10T10:02:54.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.150+0000 7fc14b200640 1 -- 172.21.15.77:0/186571001 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc14c103e30 msgr2=0x7fc14c104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:54.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.150+0000 7fc14b200640 1 --2- 172.21.15.77:0/186571001 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc14c103e30 0x7fc14c104210 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fc13c009900 tx=0x7fc13c02a410 comp rx=0 tx=0).stop 2024-09-10T10:02:54.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.150+0000 7fc14b200640 1 -- 172.21.15.77:0/186571001 shutdown_connections 2024-09-10T10:02:54.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.150+0000 7fc14b200640 1 --2- 172.21.15.77:0/186571001 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc14c108f60 0x7fc14c110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:54.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.150+0000 7fc14b200640 1 --2- 172.21.15.77:0/186571001 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc14c1047e0 0x7fc14c108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:54.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.150+0000 7fc14b200640 1 --2- 172.21.15.77:0/186571001 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc14c103e30 0x7fc14c104210 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:54.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.150+0000 7fc14b200640 1 -- 172.21.15.77:0/186571001 >> 172.21.15.77:0/186571001 conn(0x7fc14c0ff910 msgr2=0x7fc14c101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:54.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.150+0000 7fc14b200640 1 -- 172.21.15.77:0/186571001 shutdown_connections 2024-09-10T10:02:54.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.150+0000 7fc14b200640 1 -- 172.21.15.77:0/186571001 wait complete. 2024-09-10T10:02:54.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.151+0000 7fc14b200640 1 Processor -- start 2024-09-10T10:02:54.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.151+0000 7fc14b200640 1 -- start start 2024-09-10T10:02:54.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.152+0000 7fc14b200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc14c103e30 0x7fc14c19a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:54.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.152+0000 7fc14b200640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc14c1047e0 0x7fc14c19aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:54.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.152+0000 7fc14b200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc14c108f60 0x7fc14c19ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:54.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.152+0000 7fc14b200640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc14c113fc0 con 0x7fc14c103e30 2024-09-10T10:02:54.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.152+0000 7fc14b200640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc14c113e40 con 0x7fc14c1047e0 2024-09-10T10:02:54.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.152+0000 7fc14b200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc14c114140 con 0x7fc14c108f60 2024-09-10T10:02:54.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.152+0000 7fc14a800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc14c108f60 0x7fc14c19ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:54.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.152+0000 7fc149e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc14c103e30 0x7fc14c19a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:54.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.152+0000 7fc14a800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc14c108f60 0x7fc14c19ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:49432/0 (socket says 172.21.15.77:49432) 2024-09-10T10:02:54.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.152+0000 7fc14a800640 1 -- 172.21.15.77:0/3746028290 learned_addr learned my addr 172.21.15.77:0/3746028290 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:02:54.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.152+0000 7fc149400640 1 --2- 172.21.15.77:0/3746028290 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc14c1047e0 0x7fc14c19aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:54.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.153+0000 7fc14a800640 1 -- 172.21.15.77:0/3746028290 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc14c1047e0 msgr2=0x7fc14c19aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:54.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.153+0000 7fc14a800640 1 --2- 172.21.15.77:0/3746028290 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc14c1047e0 0x7fc14c19aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:54.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.153+0000 7fc14a800640 1 -- 172.21.15.77:0/3746028290 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc14c103e30 msgr2=0x7fc14c19a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:54.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.153+0000 7fc14a800640 1 --2- 172.21.15.77:0/3746028290 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc14c103e30 0x7fc14c19a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:54.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.153+0000 7fc14a800640 1 -- 172.21.15.77:0/3746028290 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc14c19f470 con 0x7fc14c108f60 2024-09-10T10:02:54.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.153+0000 7fc14a800640 1 --2- 172.21.15.77:0/3746028290 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc14c108f60 0x7fc14c19ed90 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fc14400eb10 tx=0x7fc14400c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:54.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.153+0000 7fc142a00640 1 -- 172.21.15.77:0/3746028290 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc144004b20 con 0x7fc14c108f60 2024-09-10T10:02:54.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.153+0000 7fc142a00640 1 -- 172.21.15.77:0/3746028290 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc144004cc0 con 0x7fc14c108f60 2024-09-10T10:02:54.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.153+0000 7fc14b200640 1 -- 172.21.15.77:0/3746028290 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc14c19f760 con 0x7fc14c108f60 2024-09-10T10:02:54.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.153+0000 7fc142a00640 1 -- 172.21.15.77:0/3746028290 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc144010720 con 0x7fc14c108f60 2024-09-10T10:02:54.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.153+0000 7fc14b200640 1 -- 172.21.15.77:0/3746028290 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc14c1a7040 con 0x7fc14c108f60 2024-09-10T10:02:54.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.156+0000 7fc142a00640 1 -- 172.21.15.77:0/3746028290 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc144005020 con 0x7fc14c108f60 2024-09-10T10:02:54.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.157+0000 7fc15383e740 1 Processor -- start 2024-09-10T10:02:54.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.157+0000 7fc15383e740 1 -- start start 2024-09-10T10:02:54.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.157+0000 7fc15383e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a42b8ecb0 0x558a42b8f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:54.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.158+0000 7fc15383e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a42b8f5d0 0x558a42b99950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:54.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.158+0000 7fc15383e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a42b99e90 0x558a42b9c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:54.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.158+0000 7fc15383e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558a42af5290 con 0x558a42b8ecb0 2024-09-10T10:02:54.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.158+0000 7fc15383e740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558a42a9cb40 con 0x558a42b99e90 2024-09-10T10:02:54.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.158+0000 7fc15383e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558a42af1250 con 0x558a42b8f5d0 2024-09-10T10:02:54.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.158+0000 7fc149400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a42b8ecb0 0x558a42b8f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:54.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.158+0000 7fc142a00640 1 --2- 172.21.15.77:0/3746028290 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fc1300a2c00 0x7fc1300a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:54.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.158+0000 7fc149400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a42b8ecb0 0x558a42b8f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:45914/0 (socket says 172.21.15.77:45914) 2024-09-10T10:02:54.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.158+0000 7fc149400640 1 -- 172.21.15.77:0/2307001530 learned_addr learned my addr 172.21.15.77:0/2307001530 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:02:54.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.158+0000 7fc149400640 1 --2- 172.21.15.77:0/2307001530 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a42b99e90 0x558a42b9c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:54.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.158+0000 7fc142a00640 1 -- 172.21.15.77:0/3746028290 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fc1440cfa50 con 0x7fc14c108f60 2024-09-10T10:02:54.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.158+0000 7fc149e00640 1 --2- 172.21.15.77:0/2307001530 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a42b8f5d0 0x558a42b99950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:54.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.158+0000 7fc14a800640 1 --2- 172.21.15.77:0/3746028290 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fc1300a2c00 0x7fc1300a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:54.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.159+0000 7fc149400640 1 -- 172.21.15.77:0/2307001530 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a42b8f5d0 msgr2=0x558a42b99950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:54.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.159+0000 7fc149400640 1 --2- 172.21.15.77:0/2307001530 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a42b8f5d0 0x558a42b99950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:54.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.159+0000 7fc149400640 1 -- 172.21.15.77:0/2307001530 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a42b99e90 msgr2=0x558a42b9c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:54.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.159+0000 7fc149400640 1 --2- 172.21.15.77:0/2307001530 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a42b99e90 0x558a42b9c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:54.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.159+0000 7fc149400640 1 -- 172.21.15.77:0/2307001530 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a428666a0 con 0x558a42b8ecb0 2024-09-10T10:02:54.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.159+0000 7fc149400640 1 --2- 172.21.15.77:0/2307001530 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a42b8ecb0 0x558a42b8f090 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fc138013730 tx=0x7fc1380096b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2f6ef4f97daa6d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:54.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.159+0000 7fc14a800640 1 --2- 172.21.15.77:0/3746028290 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fc1300a2c00 0x7fc1300a50c0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fc1440049d0 tx=0x7fc144005b40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:54.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.160+0000 7fc137400640 1 -- 172.21.15.77:0/2307001530 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc138013cb0 con 0x558a42b8ecb0 2024-09-10T10:02:54.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.160+0000 7fc137400640 1 -- 172.21.15.77:0/2307001530 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc138013e50 con 0x558a42b8ecb0 2024-09-10T10:02:54.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.160+0000 7fc137400640 1 -- 172.21.15.77:0/2307001530 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc13800fc60 con 0x558a42b8ecb0 2024-09-10T10:02:54.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.161+0000 7fc15383e740 1 -- 172.21.15.77:0/2307001530 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a42b8ecb0 msgr2=0x558a42b8f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:54.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.161+0000 7fc15383e740 1 --2- 172.21.15.77:0/2307001530 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a42b8ecb0 0x558a42b8f090 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fc138013730 tx=0x7fc1380096b0 comp rx=0 tx=0).stop 2024-09-10T10:02:54.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.161+0000 7fc15383e740 1 -- 172.21.15.77:0/2307001530 shutdown_connections 2024-09-10T10:02:54.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.161+0000 7fc15383e740 1 --2- 172.21.15.77:0/2307001530 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a42b99e90 0x558a42b9c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:54.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.161+0000 7fc15383e740 1 --2- 172.21.15.77:0/2307001530 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a42b8f5d0 0x558a42b99950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:54.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.161+0000 7fc15383e740 1 --2- 172.21.15.77:0/2307001530 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a42b8ecb0 0x558a42b8f090 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:54.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.161+0000 7fc15383e740 1 -- 172.21.15.77:0/2307001530 >> 172.21.15.77:0/2307001530 conn(0x558a42b8b240 msgr2=0x558a42b8b620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:54.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.161+0000 7fc15383e740 1 -- 172.21.15.77:0/2307001530 shutdown_connections 2024-09-10T10:02:54.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.161+0000 7fc15383e740 1 -- 172.21.15.77:0/2307001530 wait complete. 2024-09-10T10:02:54.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.161+0000 7fc15383e740 1 -- start start 2024-09-10T10:02:54.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.162+0000 7fc15383e740 1 Processor -- start 2024-09-10T10:02:54.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.162+0000 7fc15383e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a42b8ecb0 0x558a42b997a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:54.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.162+0000 7fc15383e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a42b8f5d0 0x558a42b96a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:54.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.162+0000 7fc15383e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a42b99e90 0x558a42b96f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:54.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.162+0000 7fc15383e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558a42b9cb00 con 0x558a42b99e90 2024-09-10T10:02:54.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.162+0000 7fc15383e740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558a42aaffa0 con 0x558a42b8f5d0 2024-09-10T10:02:54.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.162+0000 7fc15383e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558a42b9cc80 con 0x558a42b8ecb0 2024-09-10T10:02:54.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.162+0000 7fc15383e740 10 client.0 _ll_register_callbacks cb 0x558a42803ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:02:54.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.162+0000 7fc149400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a42b8ecb0 0x558a42b997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:54.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.163+0000 7fc149e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a42b8f5d0 0x558a42b96a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:54.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.163+0000 7fc149e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a42b8f5d0 0x558a42b96a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:49246/0 (socket says 172.21.15.77:49246) 2024-09-10T10:02:54.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.163+0000 7fc149e00640 1 -- 172.21.15.77:0/4013376295 learned_addr learned my addr 172.21.15.77:0/4013376295 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:02:54.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.163+0000 7fc149400640 1 --2- 172.21.15.77:0/4013376295 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a42b99e90 0x558a42b96f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:54.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.163+0000 7fc149e00640 1 -- 172.21.15.77:0/4013376295 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a42b8ecb0 msgr2=0x558a42b997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:54.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.163+0000 7fc149e00640 1 --2- 172.21.15.77:0/4013376295 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a42b8ecb0 0x558a42b997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:54.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.163+0000 7fc149e00640 1 -- 172.21.15.77:0/4013376295 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a42b99e90 msgr2=0x558a42b96f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:54.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.163+0000 7fc149e00640 1 --2- 172.21.15.77:0/4013376295 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a42b99e90 0x558a42b96f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:54.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.163+0000 7fc149e00640 1 -- 172.21.15.77:0/4013376295 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a42b97790 con 0x558a42b8f5d0 2024-09-10T10:02:54.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.163+0000 7fc149e00640 1 --2- 172.21.15.77:0/4013376295 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a42b8f5d0 0x558a42b96a50 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc13c02a930 tx=0x7fc13c0317e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21f1cb200ad6bd5e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:54.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.163+0000 7fc12be00640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T10:02:54.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.164+0000 7fc12be00640 1 -- 172.21.15.77:0/4013376295 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc13c035040 con 0x558a42b8f5d0 2024-09-10T10:02:54.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.164+0000 7fc12be00640 1 -- 172.21.15.77:0/4013376295 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fc13c004240 con 0x558a42b8f5d0 2024-09-10T10:02:54.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.164+0000 7fc15383e740 1 -- 172.21.15.77:0/4013376295 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc138013e50 con 0x558a42b8f5d0 2024-09-10T10:02:54.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.164+0000 7fc12be00640 1 -- 172.21.15.77:0/4013376295 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fc13c004440 con 0x558a42b8f5d0 2024-09-10T10:02:54.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.164+0000 7fc15383e740 10 client.27086 fetch_fsmap learned FSMap version 35 2024-09-10T10:02:54.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.164+0000 7fc15383e740 1 -- 172.21.15.77:0/4013376295 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x558a42ba89e0 con 0x558a42b8f5d0 2024-09-10T10:02:54.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.165+0000 7fc12be00640 1 -- 172.21.15.77:0/4013376295 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc138013e50 con 0x558a42b8f5d0 2024-09-10T10:02:54.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.165+0000 7fc12be00640 1 -- 172.21.15.77:0/4013376295 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7fc13c004bc0 con 0x558a42b8f5d0 2024-09-10T10:02:54.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.165+0000 7fc15383e740 10 client.27086 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T10:02:54.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.165+0000 7fc15383e740 10 client.27086 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T10:02:54.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.165+0000 7fc15383e740 20 client.27086 populate_metadata read hostname 'smithi077' 2024-09-10T10:02:54.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.165+0000 7fc15383e740 1 --2- 172.21.15.77:0/4013376295 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x558a42ba9ea0 0x558a42bac2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:54.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.165+0000 7fc15383e740 4 client.27086 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:02:54.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.165+0000 7fc15383e740 1 -- 172.21.15.77:0/4013376295 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558a42b02800 con 0x558a42ba9ea0 2024-09-10T10:02:54.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.165+0000 7fc149400640 1 --2- 172.21.15.77:0/4013376295 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x558a42ba9ea0 0x558a42bac2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:54.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.166+0000 7fc149400640 1 --2- 172.21.15.77:0/4013376295 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x558a42ba9ea0 0x558a42bac2c0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35ea639d06badcfc server_cookie=c4e64cd143e481df in_seq=0 out_seq=0 2024-09-10T10:02:54.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.166+0000 7fc12be00640 10 client.27086 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T10:02:54.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.166+0000 7fc12be00640 1 -- 172.21.15.77:0/4013376295 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558a42b02800 con 0x558a42ba9ea0 2024-09-10T10:02:54.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.166+0000 7fc12be00640 10 client.27086 handle_command_reply: tid=0 2024-09-10T10:02:54.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.167+0000 7fc15383e740 1 client.27086 shutdown 2024-09-10T10:02:54.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.167+0000 7fc15383e740 1 -- 172.21.15.77:0/4013376295 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a42b8f5d0 msgr2=0x558a42b96a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:54.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.167+0000 7fc15383e740 1 --2- 172.21.15.77:0/4013376295 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a42b8f5d0 0x558a42b96a50 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc13c02a930 tx=0x7fc13c0317e0 comp rx=0 tx=0).stop 2024-09-10T10:02:54.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.168+0000 7fc15383e740 1 -- 172.21.15.77:0/4013376295 shutdown_connections 2024-09-10T10:02:54.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.168+0000 7fc15383e740 1 --2- 172.21.15.77:0/4013376295 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x558a42ba9ea0 0x558a42bac2c0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:54.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.168+0000 7fc15383e740 1 --2- 172.21.15.77:0/4013376295 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a42b99e90 0x558a42b96f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:54.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.168+0000 7fc15383e740 1 --2- 172.21.15.77:0/4013376295 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a42b8f5d0 0x558a42b96a50 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:54.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.168+0000 7fc15383e740 1 --2- 172.21.15.77:0/4013376295 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a42b8ecb0 0x558a42b997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:54.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.168+0000 7fc15383e740 1 -- 172.21.15.77:0/4013376295 >> 172.21.15.77:0/4013376295 conn(0x558a42b8b240 msgr2=0x558a42b90f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:54.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.168+0000 7fc12be00640 0 client.27086 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T10:02:54.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.168+0000 7fc15383e740 1 -- 172.21.15.77:0/4013376295 shutdown_connections 2024-09-10T10:02:54.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.168+0000 7fc15383e740 1 -- 172.21.15.77:0/4013376295 wait complete. 2024-09-10T10:02:54.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.168+0000 7fc15383e740 20 client.27086 trim_cache size 0 max 16384 2024-09-10T10:02:54.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.189+0000 7fc15383e740 1 Processor -- start 2024-09-10T10:02:54.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.190+0000 7fc15383e740 1 -- start start 2024-09-10T10:02:54.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.191+0000 7fc15383e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a42b8ecb0 0x558a42b92000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:54.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.191+0000 7fc15383e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a42b8f5d0 0x558a42bb1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:54.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.191+0000 7fc15383e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a42b99e90 0x558a42bb20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:54.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.191+0000 7fc15383e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc14c1b0e50 con 0x558a42b8f5d0 2024-09-10T10:02:54.194 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.191+0000 7fc15383e740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc14c1b2990 con 0x558a42b99e90 2024-09-10T10:02:54.194 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.191+0000 7fc15383e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc14c1b1520 con 0x558a42b8ecb0 2024-09-10T10:02:54.194 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.191+0000 7fc149400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a42b8ecb0 0x558a42b92000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:54.194 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.191+0000 7fc149400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a42b8ecb0 0x558a42b92000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:49470/0 (socket says 172.21.15.77:49470) 2024-09-10T10:02:54.194 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.191+0000 7fc149400640 1 -- 172.21.15.77:0/3067768758 learned_addr learned my addr 172.21.15.77:0/3067768758 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:02:54.194 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.191+0000 7fc149e00640 1 --2- 172.21.15.77:0/3067768758 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a42b8f5d0 0x558a42bb1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:54.194 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.192+0000 7fc149400640 1 --2- 172.21.15.77:0/3067768758 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a42b99e90 0x558a42bb20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:54.195 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.192+0000 7fc149400640 1 -- 172.21.15.77:0/3067768758 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a42b99e90 msgr2=0x558a42bb20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:54.195 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.192+0000 7fc149400640 1 --2- 172.21.15.77:0/3067768758 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a42b99e90 0x558a42bb20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:54.195 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.192+0000 7fc149400640 1 -- 172.21.15.77:0/3067768758 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a42b8f5d0 msgr2=0x558a42bb1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:54.196 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.193+0000 7fc149400640 1 --2- 172.21.15.77:0/3067768758 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a42b8f5d0 0x558a42bb1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:54.196 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.193+0000 7fc149400640 1 -- 172.21.15.77:0/3067768758 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a42bac8a0 con 0x558a42b8ecb0 2024-09-10T10:02:54.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.194+0000 7fc149e00640 1 --2- 172.21.15.77:0/3067768758 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a42b8f5d0 0x558a42bb1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:02:54.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.195+0000 7fc149400640 1 --2- 172.21.15.77:0/3067768758 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a42b8ecb0 0x558a42b92000 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fc13800f870 tx=0x7fc13800f800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6174885bcd77e928 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:54.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.195+0000 7fc136a00640 1 -- 172.21.15.77:0/3067768758 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc13802a040 con 0x558a42b8ecb0 2024-09-10T10:02:54.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.195+0000 7fc136a00640 1 -- 172.21.15.77:0/3067768758 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc13800fe90 con 0x558a42b8ecb0 2024-09-10T10:02:54.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.195+0000 7fc136a00640 1 -- 172.21.15.77:0/3067768758 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc138000e60 con 0x558a42b8ecb0 2024-09-10T10:02:54.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.196+0000 7fc15383e740 1 -- 172.21.15.77:0/3067768758 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a42b8ecb0 msgr2=0x558a42b92000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:54.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.196+0000 7fc15383e740 1 --2- 172.21.15.77:0/3067768758 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a42b8ecb0 0x558a42b92000 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fc13800f870 tx=0x7fc13800f800 comp rx=0 tx=0).stop 2024-09-10T10:02:54.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.196+0000 7fc15383e740 1 -- 172.21.15.77:0/3067768758 shutdown_connections 2024-09-10T10:02:54.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.196+0000 7fc15383e740 1 --2- 172.21.15.77:0/3067768758 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a42b99e90 0x558a42bb20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:54.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.196+0000 7fc15383e740 1 --2- 172.21.15.77:0/3067768758 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a42b8f5d0 0x558a42bb1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:54.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.196+0000 7fc15383e740 1 --2- 172.21.15.77:0/3067768758 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a42b8ecb0 0x558a42b92000 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:54.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.196+0000 7fc15383e740 1 -- 172.21.15.77:0/3067768758 >> 172.21.15.77:0/3067768758 conn(0x558a42b8b240 msgr2=0x558a42b918f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:54.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.197+0000 7fc15383e740 1 -- 172.21.15.77:0/3067768758 shutdown_connections 2024-09-10T10:02:54.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.197+0000 7fc15383e740 1 -- 172.21.15.77:0/3067768758 wait complete. 2024-09-10T10:02:54.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.197+0000 7fc15383e740 1 -- start start 2024-09-10T10:02:54.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.197+0000 7fc15383e740 1 Processor -- start 2024-09-10T10:02:54.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.198+0000 7fc15383e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a42b8ecb0 0x558a42ba27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:54.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.198+0000 7fc15383e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a42b8f5d0 0x558a42b93a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:54.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.198+0000 7fc15383e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a42b99e90 0x558a42b94190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:54.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.198+0000 7fc15383e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc14c1b22c0 con 0x558a42b99e90 2024-09-10T10:02:54.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.198+0000 7fc15383e740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc14c1b1bf0 con 0x558a42b8f5d0 2024-09-10T10:02:54.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.198+0000 7fc15383e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc14c1aee30 con 0x558a42b8ecb0 2024-09-10T10:02:54.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.198+0000 7fc15383e740 10 client.0 _ll_register_callbacks cb 0x7fc13c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:02:54.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.198+0000 7fc149e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a42b8f5d0 0x558a42b93a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:54.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.198+0000 7fc149e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a42b8f5d0 0x558a42b93a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:49264/0 (socket says 172.21.15.77:49264) 2024-09-10T10:02:54.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.198+0000 7fc149e00640 1 -- 172.21.15.77:0/2641345639 learned_addr learned my addr 172.21.15.77:0/2641345639 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:02:54.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.199+0000 7fc149400640 1 --2- 172.21.15.77:0/2641345639 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a42b8ecb0 0x558a42ba27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:54.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.199+0000 7fc149e00640 1 -- 172.21.15.77:0/2641345639 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a42b8ecb0 msgr2=0x558a42ba27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:54.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.199+0000 7fc149e00640 1 --2- 172.21.15.77:0/2641345639 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a42b8ecb0 0x558a42ba27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:54.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.199+0000 7fc149e00640 1 -- 172.21.15.77:0/2641345639 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a42b99e90 msgr2=0x558a42b94190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:54.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.199+0000 7fc149e00640 1 --2- 172.21.15.77:0/2641345639 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a42b99e90 0x558a42b94190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:54.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.199+0000 7fc149e00640 1 -- 172.21.15.77:0/2641345639 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a42c1c350 con 0x558a42b8f5d0 2024-09-10T10:02:54.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.199+0000 7fc149400640 1 --2- 172.21.15.77:0/2641345639 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a42b8ecb0 0x558a42ba27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:02:54.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.199+0000 7fc149e00640 1 --2- 172.21.15.77:0/2641345639 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a42b8f5d0 0x558a42b93a00 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fc13c009900 tx=0x7fc13c0039f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12158190bb84fb1e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:54.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.199+0000 7fc12be00640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T10:02:54.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.199+0000 7fc12be00640 1 -- 172.21.15.77:0/2641345639 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc13c058020 con 0x558a42b8f5d0 2024-09-10T10:02:54.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.199+0000 7fc12be00640 1 -- 172.21.15.77:0/2641345639 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fc13c004070 con 0x558a42b8f5d0 2024-09-10T10:02:54.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.199+0000 7fc15383e740 1 -- 172.21.15.77:0/2641345639 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc13800fe90 con 0x558a42b8f5d0 2024-09-10T10:02:54.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.200+0000 7fc12be00640 1 -- 172.21.15.77:0/2641345639 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fc13c063050 con 0x558a42b8f5d0 2024-09-10T10:02:54.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.200+0000 7fc15383e740 10 client.27092 fetch_fsmap learned FSMap version 35 2024-09-10T10:02:54.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.200+0000 7fc15383e740 1 -- 172.21.15.77:0/2641345639 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x558a42c1d570 con 0x558a42b8f5d0 2024-09-10T10:02:54.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.200+0000 7fc12be00640 1 -- 172.21.15.77:0/2641345639 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc13800fe90 con 0x558a42b8f5d0 2024-09-10T10:02:54.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.200+0000 7fc12be00640 1 -- 172.21.15.77:0/2641345639 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7fc13c0046c0 con 0x558a42b8f5d0 2024-09-10T10:02:54.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.200+0000 7fc15383e740 10 client.27092 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T10:02:54.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.200+0000 7fc15383e740 10 client.27092 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T10:02:54.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.200+0000 7fc15383e740 20 client.27092 populate_metadata read hostname 'smithi077' 2024-09-10T10:02:54.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.200+0000 7fc15383e740 1 --2- 172.21.15.77:0/2641345639 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x558a42c1dcb0 0x558a42c200d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:54.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.200+0000 7fc15383e740 4 client.27092 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T10:02:54.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.200+0000 7fc15383e740 1 -- 172.21.15.77:0/2641345639 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x558a42afcd10 con 0x558a42c1dcb0 2024-09-10T10:02:54.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.200+0000 7fc149400640 1 --2- 172.21.15.77:0/2641345639 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x558a42c1dcb0 0x558a42c200d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:54.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.201+0000 7fc149400640 1 --2- 172.21.15.77:0/2641345639 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x558a42c1dcb0 0x558a42c200d0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c3bfe267c66e1e5 server_cookie=13bcd1f0540316eb in_seq=0 out_seq=0 2024-09-10T10:02:54.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.201+0000 7fc12be00640 10 client.27092 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T10:02:54.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.202+0000 7fc12be00640 1 -- 172.21.15.77:0/2641345639 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x558a42afcd10 con 0x558a42c1dcb0 2024-09-10T10:02:54.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.202+0000 7fc12be00640 10 client.27092 handle_command_reply: tid=0 2024-09-10T10:02:54.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.202+0000 7fc15383e740 1 client.27092 shutdown 2024-09-10T10:02:54.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.202+0000 7fc15383e740 1 -- 172.21.15.77:0/2641345639 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a42b8f5d0 msgr2=0x558a42b93a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:54.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.202+0000 7fc15383e740 1 --2- 172.21.15.77:0/2641345639 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a42b8f5d0 0x558a42b93a00 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fc13c009900 tx=0x7fc13c0039f0 comp rx=0 tx=0).stop 2024-09-10T10:02:54.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.202+0000 7fc15383e740 1 -- 172.21.15.77:0/2641345639 shutdown_connections 2024-09-10T10:02:54.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.202+0000 7fc15383e740 1 --2- 172.21.15.77:0/2641345639 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x558a42c1dcb0 0x558a42c200d0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:54.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.202+0000 7fc15383e740 1 --2- 172.21.15.77:0/2641345639 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a42b99e90 0x558a42b94190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:54.206 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.202+0000 7fc15383e740 1 --2- 172.21.15.77:0/2641345639 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a42b8f5d0 0x558a42b93a00 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:54.206 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.202+0000 7fc15383e740 1 --2- 172.21.15.77:0/2641345639 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a42b8ecb0 0x558a42ba27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:54.206 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.202+0000 7fc15383e740 1 -- 172.21.15.77:0/2641345639 >> 172.21.15.77:0/2641345639 conn(0x558a42b8b240 msgr2=0x558a42b92090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:54.206 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.202+0000 7fc12be00640 0 client.27092 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T10:02:54.206 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.203+0000 7fc15383e740 1 -- 172.21.15.77:0/2641345639 shutdown_connections 2024-09-10T10:02:54.206 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.203+0000 7fc15383e740 1 -- 172.21.15.77:0/2641345639 wait complete. 2024-09-10T10:02:54.206 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.204+0000 7fc15383e740 20 client.27092 trim_cache size 0 max 16384 2024-09-10T10:02:54.207 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-10T10:02:54.207 INFO:teuthology.orchestra.run.smithi077.stdout: "return_code": 0, 2024-09-10T10:02:54.207 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_tag": "a6062a4a-43db-471f-af77-435352f50f83", 2024-09-10T10:02:54.207 INFO:teuthology.orchestra.run.smithi077.stdout: "mode": "asynchronous" 2024-09-10T10:02:54.207 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-10T10:02:54.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.204+0000 7fc14b200640 1 -- 172.21.15.77:0/3746028290 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fc1300a2c00 msgr2=0x7fc1300a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:54.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.204+0000 7fc14b200640 1 --2- 172.21.15.77:0/3746028290 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fc1300a2c00 0x7fc1300a50c0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fc1440049d0 tx=0x7fc144005b40 comp rx=0 tx=0).stop 2024-09-10T10:02:54.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.204+0000 7fc14b200640 1 -- 172.21.15.77:0/3746028290 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc14c108f60 msgr2=0x7fc14c19ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:54.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.204+0000 7fc14b200640 1 --2- 172.21.15.77:0/3746028290 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc14c108f60 0x7fc14c19ed90 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fc14400eb10 tx=0x7fc14400c2d0 comp rx=0 tx=0).stop 2024-09-10T10:02:54.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.205+0000 7fc14b200640 1 -- 172.21.15.77:0/3746028290 shutdown_connections 2024-09-10T10:02:54.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.205+0000 7fc14b200640 1 --2- 172.21.15.77:0/3746028290 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fc1300a2c00 0x7fc1300a50c0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:54.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.205+0000 7fc14b200640 1 --2- 172.21.15.77:0/3746028290 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc14c108f60 0x7fc14c19ed90 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:54.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.205+0000 7fc14b200640 1 --2- 172.21.15.77:0/3746028290 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc14c1047e0 0x7fc14c19aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:54.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.205+0000 7fc14b200640 1 --2- 172.21.15.77:0/3746028290 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc14c103e30 0x7fc14c19a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:54.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.205+0000 7fc14b200640 1 -- 172.21.15.77:0/3746028290 >> 172.21.15.77:0/3746028290 conn(0x7fc14c0ff910 msgr2=0x7fc14c100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:54.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.205+0000 7fc14b200640 1 -- 172.21.15.77:0/3746028290 shutdown_connections 2024-09-10T10:02:54.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:54.205+0000 7fc14b200640 1 -- 172.21.15.77:0/3746028290 wait complete. 2024-09-10T10:02:54.962 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T10:02:55.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:02:54 smithi077 ceph-mon[30319]: pgmap v623: 129 pgs: 129 active+clean; 146 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 1.2 MiB/s wr, 451 op/s 2024-09-10T10:02:55.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:02:54 smithi077 ceph-mon[30319]: scrub queued for path: / 2024-09-10T10:02:55.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:02:54 smithi077 ceph-mon[30319]: scrub summary: idle+waiting paths [/] 2024-09-10T10:02:55.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:02:54 smithi077 ceph-mon[30319]: scrub summary: active paths [/] 2024-09-10T10:02:55.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:02:54 smithi077 ceph-mon[30319]: scrub summary: active paths [/] 2024-09-10T10:02:55.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:02:54 smithi077 ceph-mon[30319]: scrub summary: active paths [/] 2024-09-10T10:02:55.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:02:54 smithi077 ceph-mon[30319]: scrub summary: active paths [/] 2024-09-10T10:02:55.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:02:54 smithi077 ceph-mon[30319]: scrub summary: idle+waiting paths [/] 2024-09-10T10:02:55.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:02:54 smithi077 ceph-mon[30319]: scrub summary: idle+waiting paths [/] 2024-09-10T10:02:55.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:02:54 smithi134 ceph-mon[35674]: pgmap v623: 129 pgs: 129 active+clean; 146 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 1.2 MiB/s wr, 451 op/s 2024-09-10T10:02:55.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:02:54 smithi134 ceph-mon[35674]: scrub queued for path: / 2024-09-10T10:02:55.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:02:54 smithi134 ceph-mon[35674]: scrub summary: idle+waiting paths [/] 2024-09-10T10:02:55.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:02:54 smithi134 ceph-mon[35674]: scrub summary: active paths [/] 2024-09-10T10:02:55.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:02:54 smithi134 ceph-mon[35674]: scrub summary: active paths [/] 2024-09-10T10:02:55.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:02:54 smithi134 ceph-mon[35674]: scrub summary: active paths [/] 2024-09-10T10:02:55.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:02:54 smithi134 ceph-mon[35674]: scrub summary: active paths [/] 2024-09-10T10:02:55.382 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:02:54 smithi134 ceph-mon[35674]: scrub summary: idle+waiting paths [/] 2024-09-10T10:02:55.382 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:02:54 smithi134 ceph-mon[35674]: scrub summary: idle+waiting paths [/] 2024-09-10T10:02:55.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:02:54 smithi120 ceph-mon[34770]: pgmap v623: 129 pgs: 129 active+clean; 146 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 1.2 MiB/s wr, 451 op/s 2024-09-10T10:02:55.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:02:54 smithi120 ceph-mon[34770]: scrub queued for path: / 2024-09-10T10:02:55.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:02:54 smithi120 ceph-mon[34770]: scrub summary: idle+waiting paths [/] 2024-09-10T10:02:55.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:02:54 smithi120 ceph-mon[34770]: scrub summary: active paths [/] 2024-09-10T10:02:55.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:02:54 smithi120 ceph-mon[34770]: scrub summary: active paths [/] 2024-09-10T10:02:55.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:02:54 smithi120 ceph-mon[34770]: scrub summary: active paths [/] 2024-09-10T10:02:55.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:02:54 smithi120 ceph-mon[34770]: scrub summary: active paths [/] 2024-09-10T10:02:55.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:02:54 smithi120 ceph-mon[34770]: scrub summary: idle+waiting paths [/] 2024-09-10T10:02:55.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:02:54 smithi120 ceph-mon[34770]: scrub summary: idle+waiting paths [/] 2024-09-10T10:02:55.452 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:02:56.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.076+0000 7fabbbe00640 1 -- 172.21.15.77:0/216180540 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fabbc1079c0 msgr2=0x7fabbc107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:56.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.076+0000 7fabbbe00640 1 --2- 172.21.15.77:0/216180540 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fabbc1079c0 0x7fabbc107da0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fabac009930 tx=0x7fabac02a440 comp rx=0 tx=0).stop 2024-09-10T10:02:56.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.076+0000 7fabbbe00640 1 -- 172.21.15.77:0/216180540 shutdown_connections 2024-09-10T10:02:56.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.076+0000 7fabbbe00640 1 --2- 172.21.15.77:0/216180540 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fabbc102360 0x7fabbc10e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:56.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.076+0000 7fabbbe00640 1 --2- 172.21.15.77:0/216180540 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fabbc1019c0 0x7fabbc101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:56.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.076+0000 7fabbbe00640 1 --2- 172.21.15.77:0/216180540 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fabbc1079c0 0x7fabbc107da0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:56.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.076+0000 7fabbbe00640 1 -- 172.21.15.77:0/216180540 >> 172.21.15.77:0/216180540 conn(0x7fabbc0fd740 msgr2=0x7fabbc0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:56.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.076+0000 7fabbbe00640 1 -- 172.21.15.77:0/216180540 shutdown_connections 2024-09-10T10:02:56.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.076+0000 7fabbbe00640 1 -- 172.21.15.77:0/216180540 wait complete. 2024-09-10T10:02:56.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.077+0000 7fabbbe00640 1 Processor -- start 2024-09-10T10:02:56.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.077+0000 7fabbbe00640 1 -- start start 2024-09-10T10:02:56.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.078+0000 7fabbbe00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fabbc1019c0 0x7fabbc198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:56.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.078+0000 7fabbbe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fabbc102360 0x7fabbc198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:56.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.078+0000 7fabbbe00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fabbc1079c0 0x7fabbc19cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:56.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.078+0000 7fabbbe00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fabbc111e10 con 0x7fabbc1019c0 2024-09-10T10:02:56.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.078+0000 7fabbbe00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fabbc111c90 con 0x7fabbc1079c0 2024-09-10T10:02:56.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.078+0000 7fabbbe00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fabbc111f90 con 0x7fabbc102360 2024-09-10T10:02:56.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.078+0000 7fabbaa00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fabbc1019c0 0x7fabbc198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:56.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.078+0000 7fabbaa00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fabbc1019c0 0x7fabbc198310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:45962/0 (socket says 172.21.15.77:45962) 2024-09-10T10:02:56.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.078+0000 7fabbaa00640 1 -- 172.21.15.77:0/1720069256 learned_addr learned my addr 172.21.15.77:0/1720069256 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:02:56.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.078+0000 7fabba000640 1 --2- 172.21.15.77:0/1720069256 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fabbc102360 0x7fabbc198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:56.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.079+0000 7fabbb400640 1 --2- 172.21.15.77:0/1720069256 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fabbc1079c0 0x7fabbc19cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:56.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.079+0000 7fabbaa00640 1 -- 172.21.15.77:0/1720069256 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fabbc102360 msgr2=0x7fabbc198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:56.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.079+0000 7fabbaa00640 1 --2- 172.21.15.77:0/1720069256 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fabbc102360 0x7fabbc198850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:56.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.079+0000 7fabbaa00640 1 -- 172.21.15.77:0/1720069256 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fabbc1079c0 msgr2=0x7fabbc19cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:56.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.079+0000 7fabbaa00640 1 --2- 172.21.15.77:0/1720069256 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fabbc1079c0 0x7fabbc19cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:56.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.079+0000 7fabbaa00640 1 -- 172.21.15.77:0/1720069256 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabbc19d2c0 con 0x7fabbc1019c0 2024-09-10T10:02:56.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.079+0000 7fabbb400640 1 --2- 172.21.15.77:0/1720069256 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fabbc1079c0 0x7fabbc19cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:02:56.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.079+0000 7fabbaa00640 1 --2- 172.21.15.77:0/1720069256 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fabbc1019c0 0x7fabbc198310 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fabac02a960 tx=0x7fabac003b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:56.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.080+0000 7fabb3400640 1 -- 172.21.15.77:0/1720069256 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabac03f070 con 0x7fabbc1019c0 2024-09-10T10:02:56.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.080+0000 7fabbbe00640 1 -- 172.21.15.77:0/1720069256 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabbc19d550 con 0x7fabbc1019c0 2024-09-10T10:02:56.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.080+0000 7fabb3400640 1 -- 172.21.15.77:0/1720069256 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabac031c90 con 0x7fabbc1019c0 2024-09-10T10:02:56.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.080+0000 7fabb3400640 1 -- 172.21.15.77:0/1720069256 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabac035030 con 0x7fabbc1019c0 2024-09-10T10:02:56.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.080+0000 7fabbbe00640 1 -- 172.21.15.77:0/1720069256 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabbc1a4e90 con 0x7fabbc1019c0 2024-09-10T10:02:56.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.083+0000 7fabb3400640 1 -- 172.21.15.77:0/1720069256 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fabac042070 con 0x7fabbc1019c0 2024-09-10T10:02:56.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.084+0000 7fabc40b9740 1 Processor -- start 2024-09-10T10:02:56.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.084+0000 7fabc40b9740 1 -- start start 2024-09-10T10:02:56.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.084+0000 7fabc40b9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5573de7d6c50 0x5573de7d7030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:56.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.084+0000 7fabc40b9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573de7d7570 0x5573de7e5e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:56.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.084+0000 7fabc40b9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573de7e6340 0x5573de7e8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:56.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.084+0000 7fabc40b9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5573de73d3e0 con 0x5573de7d6c50 2024-09-10T10:02:56.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.084+0000 7fabc40b9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5573de6e4d80 con 0x5573de7d7570 2024-09-10T10:02:56.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.084+0000 7fabc40b9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5573de7393a0 con 0x5573de7e6340 2024-09-10T10:02:56.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.084+0000 7fabb3400640 1 --2- 172.21.15.77:0/1720069256 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7faba00a2c00 0x7faba00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:56.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.085+0000 7fabb3400640 1 -- 172.21.15.77:0/1720069256 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fabac0ef7c0 con 0x7fabbc1019c0 2024-09-10T10:02:56.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.085+0000 7fabbb400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5573de7d6c50 0x5573de7d7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:56.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.085+0000 7fabbb400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5573de7d6c50 0x5573de7d7030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:45976/0 (socket says 172.21.15.77:45976) 2024-09-10T10:02:56.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.085+0000 7fabbb400640 1 -- 172.21.15.77:0/1318985064 learned_addr learned my addr 172.21.15.77:0/1318985064 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:02:56.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.085+0000 7fabbb400640 1 --2- 172.21.15.77:0/1720069256 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7faba00a2c00 0x7faba00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:56.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.085+0000 7fabba000640 1 --2- 172.21.15.77:0/1318985064 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573de7e6340 0x5573de7e8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:56.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.085+0000 7fabbaa00640 1 --2- 172.21.15.77:0/1318985064 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573de7d7570 0x5573de7e5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:56.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.085+0000 7fabbb400640 1 -- 172.21.15.77:0/1318985064 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573de7e6340 msgr2=0x5573de7e8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:56.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.085+0000 7fabbb400640 1 --2- 172.21.15.77:0/1318985064 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573de7e6340 0x5573de7e8730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:56.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.085+0000 7fabbb400640 1 -- 172.21.15.77:0/1318985064 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573de7d7570 msgr2=0x5573de7e5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:56.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.085+0000 7fabbb400640 1 --2- 172.21.15.77:0/1318985064 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573de7d7570 0x5573de7e5e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:56.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.085+0000 7fabbb400640 1 -- 172.21.15.77:0/1318985064 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573de703ae0 con 0x5573de7d6c50 2024-09-10T10:02:56.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.085+0000 7fabba000640 1 --2- 172.21.15.77:0/1318985064 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573de7e6340 0x5573de7e8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:02:56.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.086+0000 7fabbb400640 1 --2- 172.21.15.77:0/1318985064 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5573de7d6c50 0x5573de7d7030 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fabb4012a10 tx=0x7fabb4012ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7003a2063d1e5bd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:56.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.086+0000 7faba7400640 1 -- 172.21.15.77:0/1318985064 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb4021020 con 0x5573de7d6c50 2024-09-10T10:02:56.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.086+0000 7faba7400640 1 -- 172.21.15.77:0/1318985064 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabb4010bf0 con 0x5573de7d6c50 2024-09-10T10:02:56.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.086+0000 7faba7400640 1 -- 172.21.15.77:0/1318985064 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb4005630 con 0x5573de7d6c50 2024-09-10T10:02:56.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.087+0000 7fabc40b9740 1 -- 172.21.15.77:0/1318985064 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5573de7d6c50 msgr2=0x5573de7d7030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:56.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.087+0000 7fabc40b9740 1 --2- 172.21.15.77:0/1318985064 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5573de7d6c50 0x5573de7d7030 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fabb4012a10 tx=0x7fabb4012ed0 comp rx=0 tx=0).stop 2024-09-10T10:02:56.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.087+0000 7fabbb400640 1 --2- 172.21.15.77:0/1720069256 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7faba00a2c00 0x7faba00a50c0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fabb4022000 tx=0x7fabb4020000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:56.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.087+0000 7fabc40b9740 1 -- 172.21.15.77:0/1318985064 shutdown_connections 2024-09-10T10:02:56.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.087+0000 7fabc40b9740 1 --2- 172.21.15.77:0/1318985064 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573de7e6340 0x5573de7e8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:56.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.087+0000 7fabc40b9740 1 --2- 172.21.15.77:0/1318985064 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573de7d7570 0x5573de7e5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:56.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.087+0000 7fabc40b9740 1 --2- 172.21.15.77:0/1318985064 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5573de7d6c50 0x5573de7d7030 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:56.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.087+0000 7fabc40b9740 1 -- 172.21.15.77:0/1318985064 >> 172.21.15.77:0/1318985064 conn(0x5573de7d32e0 msgr2=0x5573de7d36c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:56.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.087+0000 7fabc40b9740 1 -- 172.21.15.77:0/1318985064 shutdown_connections 2024-09-10T10:02:56.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.087+0000 7fabc40b9740 1 -- 172.21.15.77:0/1318985064 wait complete. 2024-09-10T10:02:56.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.088+0000 7fabc40b9740 1 -- start start 2024-09-10T10:02:56.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.088+0000 7fabc40b9740 1 Processor -- start 2024-09-10T10:02:56.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.088+0000 7fabc40b9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5573de7d6c50 0x5573de7e3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:56.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.088+0000 7fabc40b9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573de7d7570 0x5573de7e38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:56.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.088+0000 7fabc40b9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573de7e6340 0x5573de7e3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:56.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.088+0000 7fabc40b9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5573de7e9010 con 0x5573de7d6c50 2024-09-10T10:02:56.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.088+0000 7fabc40b9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5573de6f80f0 con 0x5573de7e6340 2024-09-10T10:02:56.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.088+0000 7fabc40b9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5573de7e9190 con 0x5573de7d7570 2024-09-10T10:02:56.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.088+0000 7fabc40b9740 10 client.0 _ll_register_callbacks cb 0x5573de6e99e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:02:56.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.088+0000 7fabbaa00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5573de7d6c50 0x5573de7e3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:56.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.088+0000 7fabba000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573de7d7570 0x5573de7e38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:56.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.089+0000 7fabbb400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573de7e6340 0x5573de7e3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:56.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.089+0000 7fabba000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573de7d7570 0x5573de7e38a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:49528/0 (socket says 172.21.15.77:49528) 2024-09-10T10:02:56.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.089+0000 7fabba000640 1 -- 172.21.15.77:0/3694798421 learned_addr learned my addr 172.21.15.77:0/3694798421 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:02:56.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.089+0000 7fabbb400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573de7e6340 0x5573de7e3de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:49292/0 (socket says 172.21.15.77:49292) 2024-09-10T10:02:56.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.089+0000 7fabba000640 1 -- 172.21.15.77:0/3694798421 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573de7e6340 msgr2=0x5573de7e3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:56.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.089+0000 7fabba000640 1 --2- 172.21.15.77:0/3694798421 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573de7e6340 0x5573de7e3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:56.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.089+0000 7fabba000640 1 -- 172.21.15.77:0/3694798421 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5573de7d6c50 msgr2=0x5573de7e3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:56.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.089+0000 7fabba000640 1 --2- 172.21.15.77:0/3694798421 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5573de7d6c50 0x5573de7e3360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:56.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.089+0000 7fabba000640 1 -- 172.21.15.77:0/3694798421 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573de4ae3c0 con 0x5573de7d7570 2024-09-10T10:02:56.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.089+0000 7fabba000640 1 --2- 172.21.15.77:0/3694798421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573de7d7570 0x5573de7e38a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7faba8012720 tx=0x7faba8012be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7dccc2f42527015 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:56.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.089+0000 7fab9be00640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-10T10:02:56.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.090+0000 7fab9be00640 1 -- 172.21.15.77:0/3694798421 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba8004050 con 0x5573de7d7570 2024-09-10T10:02:56.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.090+0000 7fabc40b9740 1 -- 172.21.15.77:0/3694798421 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fabb4010bf0 con 0x5573de7d7570 2024-09-10T10:02:56.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.090+0000 7fab9be00640 1 -- 172.21.15.77:0/3694798421 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7faba80041f0 con 0x5573de7d7570 2024-09-10T10:02:56.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.090+0000 7fab9be00640 1 -- 172.21.15.77:0/3694798421 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7faba8025050 con 0x5573de7d7570 2024-09-10T10:02:56.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.090+0000 7fabc40b9740 10 client.27064 fetch_fsmap learned FSMap version 35 2024-09-10T10:02:56.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.090+0000 7fabc40b9740 1 -- 172.21.15.77:0/3694798421 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5573de7f4ee0 con 0x5573de7d7570 2024-09-10T10:02:56.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.090+0000 7fab9be00640 1 -- 172.21.15.77:0/3694798421 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabb4010bf0 con 0x5573de7d7570 2024-09-10T10:02:56.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.090+0000 7fab9be00640 1 -- 172.21.15.77:0/3694798421 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7faba8003a40 con 0x5573de7d7570 2024-09-10T10:02:56.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.091+0000 7fabc40b9740 10 client.27064 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T10:02:56.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.091+0000 7fabc40b9740 10 client.27064 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T10:02:56.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.091+0000 7fabc40b9740 20 client.27064 populate_metadata read hostname 'smithi077' 2024-09-10T10:02:56.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.091+0000 7fabc40b9740 1 --2- 172.21.15.77:0/3694798421 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x5573de7f62f0 0x5573de7f8700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:56.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.091+0000 7fabc40b9740 4 client.27064 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:02:56.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.091+0000 7fabc40b9740 1 -- 172.21.15.77:0/3694798421 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5573de74a950 con 0x5573de7f62f0 2024-09-10T10:02:56.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.091+0000 7fabbaa00640 1 --2- 172.21.15.77:0/3694798421 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x5573de7f62f0 0x5573de7f8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:56.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.091+0000 7fabbaa00640 1 --2- 172.21.15.77:0/3694798421 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x5573de7f62f0 0x5573de7f8700 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d9332309f3c57a0 server_cookie=da4cd152695529c2 in_seq=0 out_seq=0 2024-09-10T10:02:56.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.091+0000 7fab9be00640 10 client.27064 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T10:02:56.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.092+0000 7fab9be00640 1 -- 172.21.15.77:0/3694798421 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5573de74a950 con 0x5573de7f62f0 2024-09-10T10:02:56.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.092+0000 7fab9be00640 10 client.27064 handle_command_reply: tid=0 2024-09-10T10:02:56.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.092+0000 7fabc40b9740 1 client.27064 shutdown 2024-09-10T10:02:56.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.092+0000 7fabc40b9740 1 -- 172.21.15.77:0/3694798421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573de7d7570 msgr2=0x5573de7e38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:56.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.092+0000 7fabc40b9740 1 --2- 172.21.15.77:0/3694798421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573de7d7570 0x5573de7e38a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7faba8012720 tx=0x7faba8012be0 comp rx=0 tx=0).stop 2024-09-10T10:02:56.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.093+0000 7fabc40b9740 1 -- 172.21.15.77:0/3694798421 shutdown_connections 2024-09-10T10:02:56.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.093+0000 7fabc40b9740 1 --2- 172.21.15.77:0/3694798421 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x5573de7f62f0 0x5573de7f8700 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:56.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.093+0000 7fab9be00640 0 client.27064 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T10:02:56.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.093+0000 7fabc40b9740 1 --2- 172.21.15.77:0/3694798421 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573de7e6340 0x5573de7e3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:56.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.093+0000 7fabc40b9740 1 --2- 172.21.15.77:0/3694798421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573de7d7570 0x5573de7e38a0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:56.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.093+0000 7fabc40b9740 1 --2- 172.21.15.77:0/3694798421 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5573de7d6c50 0x5573de7e3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:56.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.093+0000 7fabc40b9740 1 -- 172.21.15.77:0/3694798421 >> 172.21.15.77:0/3694798421 conn(0x5573de7d32e0 msgr2=0x5573de7d9040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:56.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.093+0000 7fabc40b9740 1 -- 172.21.15.77:0/3694798421 shutdown_connections 2024-09-10T10:02:56.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.093+0000 7fabc40b9740 1 -- 172.21.15.77:0/3694798421 wait complete. 2024-09-10T10:02:56.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.093+0000 7fabc40b9740 20 client.27064 trim_cache size 0 max 16384 2024-09-10T10:02:56.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.112+0000 7fabc40b9740 1 Processor -- start 2024-09-10T10:02:56.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.112+0000 7fabc40b9740 1 -- start start 2024-09-10T10:02:56.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.112+0000 7fabc40b9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573de7d32e0 0x5573de7d2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:56.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.112+0000 7fabc40b9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5573de7d6c50 0x5573de7fe010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:56.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.112+0000 7fabc40b9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573de7d7570 0x5573de7fe550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:56.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.112+0000 7fabc40b9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fabbc1acaf0 con 0x5573de7d6c50 2024-09-10T10:02:56.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.112+0000 7fabc40b9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fabbc1ae630 con 0x5573de7d32e0 2024-09-10T10:02:56.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.112+0000 7fabc40b9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fabbc1ad1c0 con 0x5573de7d7570 2024-09-10T10:02:56.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.112+0000 7fabbaa00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573de7d32e0 0x5573de7d2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:56.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.112+0000 7fabbaa00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573de7d32e0 0x5573de7d2290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:49308/0 (socket says 172.21.15.77:49308) 2024-09-10T10:02:56.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.113+0000 7fabbaa00640 1 -- 172.21.15.77:0/2316252374 learned_addr learned my addr 172.21.15.77:0/2316252374 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:02:56.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.113+0000 7fabba000640 1 --2- 172.21.15.77:0/2316252374 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5573de7d6c50 0x5573de7fe010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:56.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.113+0000 7fabbb400640 1 --2- 172.21.15.77:0/2316252374 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573de7d7570 0x5573de7fe550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:56.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.113+0000 7fabbaa00640 1 -- 172.21.15.77:0/2316252374 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573de7d7570 msgr2=0x5573de7fe550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:56.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.113+0000 7fabbaa00640 1 --2- 172.21.15.77:0/2316252374 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573de7d7570 0x5573de7fe550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:56.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.113+0000 7fabbaa00640 1 -- 172.21.15.77:0/2316252374 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5573de7d6c50 msgr2=0x5573de7fe010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:56.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.113+0000 7fabbaa00640 1 --2- 172.21.15.77:0/2316252374 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5573de7d6c50 0x5573de7fe010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:56.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.113+0000 7fabbaa00640 1 -- 172.21.15.77:0/2316252374 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573de7f22b0 con 0x5573de7d32e0 2024-09-10T10:02:56.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.113+0000 7fabbb400640 1 --2- 172.21.15.77:0/2316252374 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573de7d7570 0x5573de7fe550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:02:56.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.113+0000 7fabbaa00640 1 --2- 172.21.15.77:0/2316252374 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573de7d32e0 0x5573de7d2290 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fabac0b8600 tx=0x7fabac009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26654a2eee3e2945 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:56.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.113+0000 7faba6a00640 1 -- 172.21.15.77:0/2316252374 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabac049040 con 0x5573de7d32e0 2024-09-10T10:02:56.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.113+0000 7faba6a00640 1 -- 172.21.15.77:0/2316252374 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabac0bbc30 con 0x5573de7d32e0 2024-09-10T10:02:56.117 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.113+0000 7faba6a00640 1 -- 172.21.15.77:0/2316252374 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabac0b82a0 con 0x5573de7d32e0 2024-09-10T10:02:56.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.114+0000 7fabc40b9740 1 -- 172.21.15.77:0/2316252374 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573de7d32e0 msgr2=0x5573de7d2290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:56.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.114+0000 7fabc40b9740 1 --2- 172.21.15.77:0/2316252374 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573de7d32e0 0x5573de7d2290 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fabac0b8600 tx=0x7fabac009260 comp rx=0 tx=0).stop 2024-09-10T10:02:56.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.114+0000 7fabc40b9740 1 -- 172.21.15.77:0/2316252374 shutdown_connections 2024-09-10T10:02:56.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.114+0000 7fabc40b9740 1 --2- 172.21.15.77:0/2316252374 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573de7d7570 0x5573de7fe550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:56.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.114+0000 7fabc40b9740 1 --2- 172.21.15.77:0/2316252374 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5573de7d6c50 0x5573de7fe010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:56.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.114+0000 7fabc40b9740 1 --2- 172.21.15.77:0/2316252374 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573de7d32e0 0x5573de7d2290 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:56.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.114+0000 7fabc40b9740 1 -- 172.21.15.77:0/2316252374 >> 172.21.15.77:0/2316252374 conn(0x5573de7ce470 msgr2=0x5573de7d1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:56.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.114+0000 7fabc40b9740 1 -- 172.21.15.77:0/2316252374 shutdown_connections 2024-09-10T10:02:56.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.114+0000 7fabc40b9740 1 -- 172.21.15.77:0/2316252374 wait complete. 2024-09-10T10:02:56.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.115+0000 7fabc40b9740 1 -- start start 2024-09-10T10:02:56.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.115+0000 7fabc40b9740 1 Processor -- start 2024-09-10T10:02:56.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.116+0000 7fabc40b9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5573de7d32e0 0x5573de7ea3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:56.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.116+0000 7fabc40b9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573de7d6c50 0x5573de7ea8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:56.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.116+0000 7fabc40b9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573de7d7570 0x5573de7eae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:56.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.116+0000 7fabc40b9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fabbc1adf60 con 0x5573de7d32e0 2024-09-10T10:02:56.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.116+0000 7fabc40b9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fabbc1ad890 con 0x5573de7d7570 2024-09-10T10:02:56.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.116+0000 7fabc40b9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fabbc1aaad0 con 0x5573de7d6c50 2024-09-10T10:02:56.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.116+0000 7fabc40b9740 10 client.0 _ll_register_callbacks cb 0x7faba8009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:02:56.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.116+0000 7fabbaa00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5573de7d32e0 0x5573de7ea3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:56.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.116+0000 7fabba000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573de7d6c50 0x5573de7ea8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:56.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.116+0000 7fabbaa00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5573de7d32e0 0x5573de7ea3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:45994/0 (socket says 172.21.15.77:45994) 2024-09-10T10:02:56.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.116+0000 7fabbaa00640 1 -- 172.21.15.77:0/1978010200 learned_addr learned my addr 172.21.15.77:0/1978010200 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:02:56.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.116+0000 7fabba000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573de7d6c50 0x5573de7ea8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:49542/0 (socket says 172.21.15.77:49542) 2024-09-10T10:02:56.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.116+0000 7fabbb400640 1 --2- 172.21.15.77:0/1978010200 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573de7d7570 0x5573de7eae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:56.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.116+0000 7fabbaa00640 1 -- 172.21.15.77:0/1978010200 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573de7d6c50 msgr2=0x5573de7ea8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:56.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.116+0000 7fabbaa00640 1 --2- 172.21.15.77:0/1978010200 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573de7d6c50 0x5573de7ea8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:56.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.116+0000 7fabbaa00640 1 -- 172.21.15.77:0/1978010200 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573de7d7570 msgr2=0x5573de7eae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:56.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.116+0000 7fabbaa00640 1 --2- 172.21.15.77:0/1978010200 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573de7d7570 0x5573de7eae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:56.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.116+0000 7fabbaa00640 1 -- 172.21.15.77:0/1978010200 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573de860230 con 0x5573de7d32e0 2024-09-10T10:02:56.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.116+0000 7fabba000640 1 --2- 172.21.15.77:0/1978010200 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573de7d6c50 0x5573de7ea8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:02:56.122 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.117+0000 7fabbaa00640 1 --2- 172.21.15.77:0/1978010200 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5573de7d32e0 0x5573de7ea3b0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fabac0bbaf0 tx=0x7fabac003f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5d70c143b5b1534 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:56.122 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.117+0000 7fab9be00640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-10T10:02:56.122 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.117+0000 7fab9be00640 1 -- 172.21.15.77:0/1978010200 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabac049040 con 0x5573de7d32e0 2024-09-10T10:02:56.122 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.117+0000 7fab9be00640 1 -- 172.21.15.77:0/1978010200 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fabac0b82a0 con 0x5573de7d32e0 2024-09-10T10:02:56.122 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.117+0000 7fabc40b9740 1 -- 172.21.15.77:0/1978010200 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fabac0bbc30 con 0x5573de7d32e0 2024-09-10T10:02:56.122 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.117+0000 7fab9be00640 1 -- 172.21.15.77:0/1978010200 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fabac0c48a0 con 0x5573de7d32e0 2024-09-10T10:02:56.123 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.117+0000 7fabc40b9740 10 client.16977 fetch_fsmap learned FSMap version 35 2024-09-10T10:02:56.123 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.117+0000 7fabc40b9740 1 -- 172.21.15.77:0/1978010200 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5573de861500 con 0x5573de7d32e0 2024-09-10T10:02:56.123 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.117+0000 7fab9be00640 1 -- 172.21.15.77:0/1978010200 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabac0bbc30 con 0x5573de7d32e0 2024-09-10T10:02:56.123 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.118+0000 7fab9be00640 1 -- 172.21.15.77:0/1978010200 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7fabac0c4410 con 0x5573de7d32e0 2024-09-10T10:02:56.123 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.118+0000 7fabc40b9740 10 client.16977 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T10:02:56.123 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.118+0000 7fabc40b9740 10 client.16977 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T10:02:56.123 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.118+0000 7fabc40b9740 20 client.16977 populate_metadata read hostname 'smithi077' 2024-09-10T10:02:56.123 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.118+0000 7fabc40b9740 1 --2- 172.21.15.77:0/1978010200 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x5573de7e6340 0x5573de863990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:56.123 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.118+0000 7fabc40b9740 4 client.16977 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T10:02:56.123 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.118+0000 7fabc40b9740 1 -- 172.21.15.77:0/1978010200 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5573de744e60 con 0x5573de7e6340 2024-09-10T10:02:56.123 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.118+0000 7fabba000640 1 --2- 172.21.15.77:0/1978010200 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x5573de7e6340 0x5573de863990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:56.123 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.119+0000 7fabba000640 1 --2- 172.21.15.77:0/1978010200 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x5573de7e6340 0x5573de863990 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1175ed42647ff049 server_cookie=88b4f4819d6f49b4 in_seq=0 out_seq=0 2024-09-10T10:02:56.123 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.119+0000 7fab9be00640 10 client.16977 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T10:02:56.123 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.119+0000 7fab9be00640 1 -- 172.21.15.77:0/1978010200 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5573de744e60 con 0x5573de7e6340 2024-09-10T10:02:56.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.119+0000 7fab9be00640 10 client.16977 handle_command_reply: tid=0 2024-09-10T10:02:56.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.119+0000 7fabc40b9740 1 client.16977 shutdown 2024-09-10T10:02:56.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.119+0000 7fabc40b9740 1 -- 172.21.15.77:0/1978010200 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5573de7d32e0 msgr2=0x5573de7ea3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:56.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.119+0000 7fabc40b9740 1 --2- 172.21.15.77:0/1978010200 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5573de7d32e0 0x5573de7ea3b0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fabac0bbaf0 tx=0x7fabac003f50 comp rx=0 tx=0).stop 2024-09-10T10:02:56.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.119+0000 7fabc40b9740 1 -- 172.21.15.77:0/1978010200 shutdown_connections 2024-09-10T10:02:56.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.119+0000 7fabc40b9740 1 --2- 172.21.15.77:0/1978010200 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x5573de7e6340 0x5573de863990 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:56.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.119+0000 7fabc40b9740 1 --2- 172.21.15.77:0/1978010200 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573de7d7570 0x5573de7eae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:56.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.119+0000 7fabc40b9740 1 --2- 172.21.15.77:0/1978010200 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573de7d6c50 0x5573de7ea8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:56.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.119+0000 7fabc40b9740 1 --2- 172.21.15.77:0/1978010200 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5573de7d32e0 0x5573de7ea3b0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:56.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.119+0000 7fabc40b9740 1 -- 172.21.15.77:0/1978010200 >> 172.21.15.77:0/1978010200 conn(0x5573de7ce470 msgr2=0x5573de7f1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:56.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.119+0000 7fab9be00640 0 client.16977 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T10:02:56.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.119+0000 7fabc40b9740 1 -- 172.21.15.77:0/1978010200 shutdown_connections 2024-09-10T10:02:56.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.119+0000 7fabc40b9740 1 -- 172.21.15.77:0/1978010200 wait complete. 2024-09-10T10:02:56.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.120+0000 7fabc40b9740 20 client.16977 trim_cache size 0 max 16384 2024-09-10T10:02:56.125 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-10T10:02:56.125 INFO:teuthology.orchestra.run.smithi077.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T10:02:56.125 INFO:teuthology.orchestra.run.smithi077.stdout: "scrubs": { 2024-09-10T10:02:56.125 INFO:teuthology.orchestra.run.smithi077.stdout: "a6062a4a-43db-471f-af77-435352f50f83": { 2024-09-10T10:02:56.125 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/", 2024-09-10T10:02:56.125 INFO:teuthology.orchestra.run.smithi077.stdout: "tag": "a6062a4a-43db-471f-af77-435352f50f83", 2024-09-10T10:02:56.125 INFO:teuthology.orchestra.run.smithi077.stdout: "options": "recursive,force" 2024-09-10T10:02:56.125 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:02:56.125 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:02:56.126 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-10T10:02:56.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.120+0000 7fabbbe00640 1 -- 172.21.15.77:0/1720069256 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7faba00a2c00 msgr2=0x7faba00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:56.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.120+0000 7fabbbe00640 1 --2- 172.21.15.77:0/1720069256 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7faba00a2c00 0x7faba00a50c0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fabb4022000 tx=0x7fabb4020000 comp rx=0 tx=0).stop 2024-09-10T10:02:56.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.120+0000 7fabbbe00640 1 -- 172.21.15.77:0/1720069256 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fabbc1019c0 msgr2=0x7fabbc198310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:56.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.120+0000 7fabbbe00640 1 --2- 172.21.15.77:0/1720069256 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fabbc1019c0 0x7fabbc198310 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fabac02a960 tx=0x7fabac003b20 comp rx=0 tx=0).stop 2024-09-10T10:02:56.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.120+0000 7fabbbe00640 1 -- 172.21.15.77:0/1720069256 shutdown_connections 2024-09-10T10:02:56.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.120+0000 7fabbbe00640 1 --2- 172.21.15.77:0/1720069256 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7faba00a2c00 0x7faba00a50c0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:56.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.120+0000 7fabbbe00640 1 --2- 172.21.15.77:0/1720069256 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fabbc1079c0 0x7fabbc19cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:56.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.120+0000 7fabbbe00640 1 --2- 172.21.15.77:0/1720069256 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fabbc102360 0x7fabbc198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:56.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.120+0000 7fabbbe00640 1 --2- 172.21.15.77:0/1720069256 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fabbc1019c0 0x7fabbc198310 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:56.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.120+0000 7fabbbe00640 1 -- 172.21.15.77:0/1720069256 >> 172.21.15.77:0/1720069256 conn(0x7fabbc0fd740 msgr2=0x7fabbc0fe010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:56.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.121+0000 7fabbbe00640 1 -- 172.21.15.77:0/1720069256 shutdown_connections 2024-09-10T10:02:56.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:02:56.121+0000 7fabbbe00640 1 -- 172.21.15.77:0/1720069256 wait complete. 2024-09-10T10:02:56.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:02:55 smithi077 ceph-mon[30319]: scrub summary: active paths [/] 2024-09-10T10:02:56.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:02:55 smithi077 ceph-mon[30319]: scrub summary: idle+waiting paths [/] 2024-09-10T10:02:56.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:02:55 smithi077 ceph-mon[30319]: scrub summary: idle+waiting paths [/] 2024-09-10T10:02:56.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:02:55 smithi077 ceph-mon[30319]: scrub summary: idle+waiting paths [/] 2024-09-10T10:02:56.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:02:55 smithi134 ceph-mon[35674]: scrub summary: active paths [/] 2024-09-10T10:02:56.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:02:55 smithi134 ceph-mon[35674]: scrub summary: idle+waiting paths [/] 2024-09-10T10:02:56.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:02:55 smithi134 ceph-mon[35674]: scrub summary: idle+waiting paths [/] 2024-09-10T10:02:56.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:02:55 smithi134 ceph-mon[35674]: scrub summary: idle+waiting paths [/] 2024-09-10T10:02:56.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:02:55 smithi120 ceph-mon[34770]: scrub summary: active paths [/] 2024-09-10T10:02:56.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:02:55 smithi120 ceph-mon[34770]: scrub summary: idle+waiting paths [/] 2024-09-10T10:02:56.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:02:55 smithi120 ceph-mon[34770]: scrub summary: idle+waiting paths [/] 2024-09-10T10:02:56.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:02:55 smithi120 ceph-mon[34770]: scrub summary: idle+waiting paths [/] 2024-09-10T10:02:57.050 INFO:tasks.cephfs.filesystem:scrub status for tag:a6062a4a-43db-471f-af77-435352f50f83 - {'path': '/', 'tag': 'a6062a4a-43db-471f-af77-435352f50f83', 'options': 'recursive,force'} 2024-09-10T10:02:57.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:02:56 smithi134 ceph-mon[35674]: pgmap v624: 129 pgs: 129 active+clean; 143 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 1.2 MiB/s wr, 423 op/s 2024-09-10T10:02:57.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:02:56 smithi120 ceph-mon[34770]: pgmap v624: 129 pgs: 129 active+clean; 143 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 1.2 MiB/s wr, 423 op/s 2024-09-10T10:02:57.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:02:56 smithi077 ceph-mon[30319]: pgmap v624: 129 pgs: 129 active+clean; 143 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 1.2 MiB/s wr, 423 op/s 2024-09-10T10:02:59.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:02:58 smithi134 ceph-mon[35674]: pgmap v625: 129 pgs: 129 active+clean; 149 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 1.2 MiB/s wr, 435 op/s 2024-09-10T10:02:59.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:02:58 smithi120 ceph-mon[34770]: pgmap v625: 129 pgs: 129 active+clean; 149 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 1.2 MiB/s wr, 435 op/s 2024-09-10T10:02:59.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:02:58 smithi077 ceph-mon[30319]: pgmap v625: 129 pgs: 129 active+clean; 149 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 1.2 MiB/s wr, 435 op/s 2024-09-10T10:03:01.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:03:00 smithi134 ceph-mon[35674]: pgmap v626: 129 pgs: 129 active+clean; 147 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 1.4 MiB/s wr, 489 op/s 2024-09-10T10:03:01.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:03:00 smithi120 ceph-mon[34770]: pgmap v626: 129 pgs: 129 active+clean; 147 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 1.4 MiB/s wr, 489 op/s 2024-09-10T10:03:01.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:03:00 smithi077 ceph-mon[30319]: pgmap v626: 129 pgs: 129 active+clean; 147 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 1.4 MiB/s wr, 489 op/s 2024-09-10T10:03:03.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:03:02 smithi134 ceph-mon[35674]: pgmap v627: 129 pgs: 129 active+clean; 142 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 1.2 MiB/s wr, 446 op/s 2024-09-10T10:03:03.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:03:02 smithi120 ceph-mon[34770]: pgmap v627: 129 pgs: 129 active+clean; 142 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 1.2 MiB/s wr, 446 op/s 2024-09-10T10:03:03.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:03:02 smithi077 ceph-mon[30319]: pgmap v627: 129 pgs: 129 active+clean; 142 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 1.2 MiB/s wr, 446 op/s 2024-09-10T10:03:05.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:03:04 smithi134 ceph-mon[35674]: pgmap v628: 129 pgs: 129 active+clean; 145 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 1.3 MiB/s wr, 456 op/s 2024-09-10T10:03:05.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:03:04 smithi120 ceph-mon[34770]: pgmap v628: 129 pgs: 129 active+clean; 145 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 1.3 MiB/s wr, 456 op/s 2024-09-10T10:03:05.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:03:04 smithi077 ceph-mon[30319]: pgmap v628: 129 pgs: 129 active+clean; 145 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 1.3 MiB/s wr, 456 op/s 2024-09-10T10:03:06.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:03:05 smithi134 ceph-mon[35674]: scrub summary: idle 2024-09-10T10:03:06.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:03:05 smithi134 ceph-mon[35674]: scrub summary: idle 2024-09-10T10:03:06.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:03:05 smithi120 ceph-mon[34770]: scrub summary: idle 2024-09-10T10:03:06.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:03:05 smithi120 ceph-mon[34770]: scrub summary: idle 2024-09-10T10:03:06.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:03:05 smithi077 ceph-mon[30319]: scrub summary: idle 2024-09-10T10:03:06.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:03:05 smithi077 ceph-mon[30319]: scrub summary: idle 2024-09-10T10:03:07.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:03:06 smithi134 ceph-mon[35674]: scrub summary: idle 2024-09-10T10:03:07.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:03:06 smithi134 ceph-mon[35674]: scrub summary: idle 2024-09-10T10:03:07.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:03:06 smithi134 ceph-mon[35674]: scrub summary: idle 2024-09-10T10:03:07.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:03:06 smithi134 ceph-mon[35674]: pgmap v629: 129 pgs: 129 active+clean; 140 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 1.2 MiB/s wr, 446 op/s 2024-09-10T10:03:07.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:03:06 smithi120 ceph-mon[34770]: scrub summary: idle 2024-09-10T10:03:07.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:03:06 smithi120 ceph-mon[34770]: scrub summary: idle 2024-09-10T10:03:07.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:03:06 smithi120 ceph-mon[34770]: scrub summary: idle 2024-09-10T10:03:07.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:03:06 smithi120 ceph-mon[34770]: pgmap v629: 129 pgs: 129 active+clean; 140 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 1.2 MiB/s wr, 446 op/s 2024-09-10T10:03:07.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:03:06 smithi077 ceph-mon[30319]: scrub summary: idle 2024-09-10T10:03:07.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:03:06 smithi077 ceph-mon[30319]: scrub summary: idle 2024-09-10T10:03:07.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:03:06 smithi077 ceph-mon[30319]: scrub summary: idle 2024-09-10T10:03:07.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:03:06 smithi077 ceph-mon[30319]: pgmap v629: 129 pgs: 129 active+clean; 140 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 1.2 MiB/s wr, 446 op/s 2024-09-10T10:03:08.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:03:08 smithi134 ceph-mon[35674]: pgmap v630: 129 pgs: 129 active+clean; 140 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 1.2 MiB/s wr, 465 op/s 2024-09-10T10:03:08.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:03:08 smithi120 ceph-mon[34770]: pgmap v630: 129 pgs: 129 active+clean; 140 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 1.2 MiB/s wr, 465 op/s 2024-09-10T10:03:08.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:03:08 smithi077 ceph-mon[30319]: pgmap v630: 129 pgs: 129 active+clean; 140 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 1.2 MiB/s wr, 465 op/s 2024-09-10T10:03:10.305 INFO:tasks.workunit.client.0.smithi077.stdout:copying to largest ever: 0x63ef9ee 2024-09-10T10:03:11.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:03:10 smithi134 ceph-mon[35674]: pgmap v631: 129 pgs: 129 active+clean; 143 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 1.3 MiB/s wr, 523 op/s 2024-09-10T10:03:11.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:03:10 smithi120 ceph-mon[34770]: pgmap v631: 129 pgs: 129 active+clean; 143 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 1.3 MiB/s wr, 523 op/s 2024-09-10T10:03:11.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:03:10 smithi077 ceph-mon[30319]: pgmap v631: 129 pgs: 129 active+clean; 143 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 1.3 MiB/s wr, 523 op/s 2024-09-10T10:03:13.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:03:12 smithi134 ceph-mon[35674]: pgmap v632: 129 pgs: 129 active+clean; 143 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 1.2 MiB/s wr, 471 op/s 2024-09-10T10:03:13.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:03:12 smithi120 ceph-mon[34770]: pgmap v632: 129 pgs: 129 active+clean; 143 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 1.2 MiB/s wr, 471 op/s 2024-09-10T10:03:13.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:03:12 smithi077 ceph-mon[30319]: pgmap v632: 129 pgs: 129 active+clean; 143 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 1.2 MiB/s wr, 471 op/s 2024-09-10T10:03:15.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:03:14 smithi134 ceph-mon[35674]: pgmap v633: 129 pgs: 129 active+clean; 134 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 1.2 MiB/s wr, 466 op/s 2024-09-10T10:03:15.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:03:14 smithi120 ceph-mon[34770]: pgmap v633: 129 pgs: 129 active+clean; 134 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 1.2 MiB/s wr, 466 op/s 2024-09-10T10:03:15.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:03:14 smithi077 ceph-mon[30319]: pgmap v633: 129 pgs: 129 active+clean; 134 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 1.2 MiB/s wr, 466 op/s 2024-09-10T10:03:17.225 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:03:16 smithi120 ceph-mon[34770]: pgmap v634: 129 pgs: 129 active+clean; 134 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 1.1 MiB/s wr, 445 op/s 2024-09-10T10:03:17.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:03:16 smithi134 ceph-mon[35674]: pgmap v634: 129 pgs: 129 active+clean; 134 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 1.1 MiB/s wr, 445 op/s 2024-09-10T10:03:17.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:03:16 smithi077 ceph-mon[30319]: pgmap v634: 129 pgs: 129 active+clean; 134 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 1.1 MiB/s wr, 445 op/s 2024-09-10T10:03:19.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:03:18 smithi134 ceph-mon[35674]: pgmap v635: 129 pgs: 129 active+clean; 142 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 1.2 MiB/s wr, 461 op/s 2024-09-10T10:03:19.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:03:18 smithi120 ceph-mon[34770]: pgmap v635: 129 pgs: 129 active+clean; 142 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 1.2 MiB/s wr, 461 op/s 2024-09-10T10:03:19.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:03:18 smithi077 ceph-mon[30319]: pgmap v635: 129 pgs: 129 active+clean; 142 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 1.2 MiB/s wr, 461 op/s 2024-09-10T10:03:21.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:03:20 smithi134 ceph-mon[35674]: pgmap v636: 129 pgs: 129 active+clean; 163 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.3 MiB/s wr, 517 op/s 2024-09-10T10:03:21.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:03:20 smithi120 ceph-mon[34770]: pgmap v636: 129 pgs: 129 active+clean; 163 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.3 MiB/s wr, 517 op/s 2024-09-10T10:03:21.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:03:20 smithi077 ceph-mon[30319]: pgmap v636: 129 pgs: 129 active+clean; 163 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.3 MiB/s wr, 517 op/s 2024-09-10T10:03:23.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:03:22 smithi134 ceph-mon[35674]: pgmap v637: 129 pgs: 129 active+clean; 157 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 1.2 MiB/s wr, 464 op/s 2024-09-10T10:03:23.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:03:22 smithi120 ceph-mon[34770]: pgmap v637: 129 pgs: 129 active+clean; 157 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 1.2 MiB/s wr, 464 op/s 2024-09-10T10:03:23.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:03:22 smithi077 ceph-mon[30319]: pgmap v637: 129 pgs: 129 active+clean; 157 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 1.2 MiB/s wr, 464 op/s 2024-09-10T10:03:25.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:03:24 smithi134 ceph-mon[35674]: pgmap v638: 129 pgs: 129 active+clean; 154 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 1.2 MiB/s wr, 467 op/s 2024-09-10T10:03:25.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:03:24 smithi120 ceph-mon[34770]: pgmap v638: 129 pgs: 129 active+clean; 154 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 1.2 MiB/s wr, 467 op/s 2024-09-10T10:03:25.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:03:24 smithi077 ceph-mon[30319]: pgmap v638: 129 pgs: 129 active+clean; 154 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 1.2 MiB/s wr, 467 op/s 2024-09-10T10:03:27.052 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T10:03:27.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:03:26 smithi134 ceph-mon[35674]: pgmap v639: 129 pgs: 129 active+clean; 152 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 1.1 MiB/s wr, 448 op/s 2024-09-10T10:03:27.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:03:26 smithi120 ceph-mon[34770]: pgmap v639: 129 pgs: 129 active+clean; 152 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 1.1 MiB/s wr, 448 op/s 2024-09-10T10:03:27.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:03:26 smithi077 ceph-mon[30319]: pgmap v639: 129 pgs: 129 active+clean; 152 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 1.1 MiB/s wr, 448 op/s 2024-09-10T10:03:27.517 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:03:28.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.123+0000 7f5c62200640 1 -- 172.21.15.77:0/93081684 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5c5c1047e0 msgr2=0x7f5c5c108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:28.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.123+0000 7f5c62200640 1 --2- 172.21.15.77:0/93081684 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5c5c1047e0 0x7f5c5c108830 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f5c48009900 tx=0x7f5c4802a410 comp rx=0 tx=0).stop 2024-09-10T10:03:28.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.123+0000 7f5c62200640 1 -- 172.21.15.77:0/93081684 shutdown_connections 2024-09-10T10:03:28.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.123+0000 7f5c62200640 1 --2- 172.21.15.77:0/93081684 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5c5c108f60 0x7f5c5c110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.123+0000 7f5c62200640 1 --2- 172.21.15.77:0/93081684 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5c5c1047e0 0x7f5c5c108830 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.123+0000 7f5c62200640 1 --2- 172.21.15.77:0/93081684 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5c5c103e30 0x7f5c5c104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.123+0000 7f5c62200640 1 -- 172.21.15.77:0/93081684 >> 172.21.15.77:0/93081684 conn(0x7f5c5c0ff910 msgr2=0x7f5c5c101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:28.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.124+0000 7f5c62200640 1 -- 172.21.15.77:0/93081684 shutdown_connections 2024-09-10T10:03:28.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.124+0000 7f5c62200640 1 -- 172.21.15.77:0/93081684 wait complete. 2024-09-10T10:03:28.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.125+0000 7f5c62200640 1 Processor -- start 2024-09-10T10:03:28.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.126+0000 7f5c62200640 1 -- start start 2024-09-10T10:03:28.128 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.126+0000 7f5c62200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5c5c103e30 0x7f5c5c19a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:28.128 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.126+0000 7f5c62200640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5c5c1047e0 0x7f5c5c19a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:28.128 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.126+0000 7f5c62200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5c5c108f60 0x7f5c5c19ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:28.128 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.126+0000 7f5c62200640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5c5c113fc0 con 0x7f5c5c108f60 2024-09-10T10:03:28.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.126+0000 7f5c62200640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5c5c113e40 con 0x7f5c5c1047e0 2024-09-10T10:03:28.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.126+0000 7f5c62200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5c5c114140 con 0x7f5c5c103e30 2024-09-10T10:03:28.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.126+0000 7f5c60e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5c5c103e30 0x7f5c5c19a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:28.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.126+0000 7f5c5be00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5c5c1047e0 0x7f5c5c19a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:28.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.127+0000 7f5c5be00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5c5c1047e0 0x7f5c5c19a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:43544/0 (socket says 172.21.15.77:43544) 2024-09-10T10:03:28.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.127+0000 7f5c60e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5c5c103e30 0x7f5c5c19a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:53458/0 (socket says 172.21.15.77:53458) 2024-09-10T10:03:28.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.127+0000 7f5c5be00640 1 -- 172.21.15.77:0/2830524389 learned_addr learned my addr 172.21.15.77:0/2830524389 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:03:28.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.127+0000 7f5c60e00640 1 -- 172.21.15.77:0/2830524389 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5c5c1047e0 msgr2=0x7f5c5c19a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:28.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.127+0000 7f5c60e00640 1 --2- 172.21.15.77:0/2830524389 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5c5c1047e0 0x7f5c5c19a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.127+0000 7f5c60e00640 1 -- 172.21.15.77:0/2830524389 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5c5c108f60 msgr2=0x7f5c5c19ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:28.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.127+0000 7f5c60e00640 1 --2- 172.21.15.77:0/2830524389 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5c5c108f60 0x7f5c5c19ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.127+0000 7f5c60e00640 1 -- 172.21.15.77:0/2830524389 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c5c19f430 con 0x7f5c5c103e30 2024-09-10T10:03:28.130 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.128+0000 7f5c60e00640 1 --2- 172.21.15.77:0/2830524389 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5c5c103e30 0x7f5c5c19a480 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c00eb00 tx=0x7f5c4c00c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:28.130 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.128+0000 7f5c59600640 1 -- 172.21.15.77:0/2830524389 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c4c010430 con 0x7f5c5c103e30 2024-09-10T10:03:28.130 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.128+0000 7f5c59600640 1 -- 172.21.15.77:0/2830524389 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c4c010ab0 con 0x7f5c5c103e30 2024-09-10T10:03:28.130 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.128+0000 7f5c59600640 1 -- 172.21.15.77:0/2830524389 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c4c017bb0 con 0x7f5c5c103e30 2024-09-10T10:03:28.130 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.128+0000 7f5c62200640 1 -- 172.21.15.77:0/2830524389 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c5c19f720 con 0x7f5c5c103e30 2024-09-10T10:03:28.130 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.128+0000 7f5c62200640 1 -- 172.21.15.77:0/2830524389 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c5c1a7000 con 0x7f5c5c103e30 2024-09-10T10:03:28.132 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.132+0000 7f5c59600640 1 -- 172.21.15.77:0/2830524389 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5c4c010c60 con 0x7f5c5c103e30 2024-09-10T10:03:28.134 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.133+0000 7f5c6629b740 1 Processor -- start 2024-09-10T10:03:28.134 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.133+0000 7f5c59600640 1 --2- 172.21.15.77:0/2830524389 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f5c400a2c00 0x7f5c400a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:28.134 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.133+0000 7f5c6629b740 1 -- start start 2024-09-10T10:03:28.135 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.133+0000 7f5c59600640 1 -- 172.21.15.77:0/2830524389 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f5c4c0cff10 con 0x7f5c5c103e30 2024-09-10T10:03:28.135 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.133+0000 7f5c61800640 1 --2- 172.21.15.77:0/2830524389 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f5c400a2c00 0x7f5c400a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:28.135 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.133+0000 7f5c6629b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ec4a382d50 0x55ec4a383130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:28.135 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.133+0000 7f5c6629b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec4a383670 0x55ec4a38d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:28.135 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.133+0000 7f5c6629b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ec4a38df30 0x55ec4a390320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:28.135 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.133+0000 7f5c6629b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ec4a2e93e0 con 0x55ec4a382d50 2024-09-10T10:03:28.135 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.133+0000 7f5c6629b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ec4a290d80 con 0x55ec4a38df30 2024-09-10T10:03:28.135 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.133+0000 7f5c6629b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ec4a2e53a0 con 0x55ec4a383670 2024-09-10T10:03:28.135 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.134+0000 7f5c60e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ec4a382d50 0x55ec4a383130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:28.135 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.134+0000 7f5c61800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ec4a38df30 0x55ec4a390320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:28.136 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.134+0000 7f5c61800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ec4a38df30 0x55ec4a390320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:43546/0 (socket says 172.21.15.77:43546) 2024-09-10T10:03:28.136 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.134+0000 7f5c61800640 1 -- 172.21.15.77:0/1061159481 learned_addr learned my addr 172.21.15.77:0/1061159481 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:03:28.136 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.134+0000 7f5c60e00640 1 --2- 172.21.15.77:0/1061159481 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ec4a382d50 0x55ec4a383130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52472/0 (socket says 172.21.15.77:52472) 2024-09-10T10:03:28.136 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.134+0000 7f5c5be00640 1 --2- 172.21.15.77:0/1061159481 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec4a383670 0x55ec4a38d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:28.137 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.134+0000 7f5c60e00640 1 -- 172.21.15.77:0/1061159481 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec4a383670 msgr2=0x55ec4a38d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:28.137 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.134+0000 7f5c60e00640 1 --2- 172.21.15.77:0/1061159481 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec4a383670 0x55ec4a38d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.134+0000 7f5c60e00640 1 -- 172.21.15.77:0/1061159481 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ec4a38df30 msgr2=0x55ec4a390320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:28.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.134+0000 7f5c60e00640 1 --2- 172.21.15.77:0/1061159481 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ec4a38df30 0x55ec4a390320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.134+0000 7f5c60e00640 1 -- 172.21.15.77:0/1061159481 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec4a2afae0 con 0x55ec4a382d50 2024-09-10T10:03:28.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.134+0000 7f5c61800640 1 --2- 172.21.15.77:0/2830524389 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f5c400a2c00 0x7f5c400a50c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f5c5400be80 tx=0x7f5c540147e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:28.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.134+0000 7f5c5be00640 1 --2- 172.21.15.77:0/1061159481 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec4a383670 0x55ec4a38d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:03:28.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.135+0000 7f5c60e00640 1 --2- 172.21.15.77:0/1061159481 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ec4a382d50 0x55ec4a383130 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c0cd5c0 tx=0x7f5c4c09bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4405dfd82b3f88cf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:28.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.135+0000 7f5c52000640 1 -- 172.21.15.77:0/1061159481 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c4c0a06d0 con 0x55ec4a382d50 2024-09-10T10:03:28.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.135+0000 7f5c52000640 1 -- 172.21.15.77:0/1061159481 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c4c0a4f90 con 0x55ec4a382d50 2024-09-10T10:03:28.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.135+0000 7f5c52000640 1 -- 172.21.15.77:0/1061159481 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c4c098990 con 0x55ec4a382d50 2024-09-10T10:03:28.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.136+0000 7f5c6629b740 1 -- 172.21.15.77:0/1061159481 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ec4a382d50 msgr2=0x55ec4a383130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:28.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.136+0000 7f5c6629b740 1 --2- 172.21.15.77:0/1061159481 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ec4a382d50 0x55ec4a383130 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c0cd5c0 tx=0x7f5c4c09bf60 comp rx=0 tx=0).stop 2024-09-10T10:03:28.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.136+0000 7f5c6629b740 1 -- 172.21.15.77:0/1061159481 shutdown_connections 2024-09-10T10:03:28.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.136+0000 7f5c6629b740 1 --2- 172.21.15.77:0/1061159481 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ec4a38df30 0x55ec4a390320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.139 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.136+0000 7f5c6629b740 1 --2- 172.21.15.77:0/1061159481 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec4a383670 0x55ec4a38d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.139 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.136+0000 7f5c6629b740 1 --2- 172.21.15.77:0/1061159481 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ec4a382d50 0x55ec4a383130 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.139 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.136+0000 7f5c6629b740 1 -- 172.21.15.77:0/1061159481 >> 172.21.15.77:0/1061159481 conn(0x55ec4a37f2e0 msgr2=0x55ec4a37f6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:28.139 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.136+0000 7f5c6629b740 1 -- 172.21.15.77:0/1061159481 shutdown_connections 2024-09-10T10:03:28.139 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.136+0000 7f5c6629b740 1 -- 172.21.15.77:0/1061159481 wait complete. 2024-09-10T10:03:28.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.137+0000 7f5c6629b740 1 -- start start 2024-09-10T10:03:28.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.137+0000 7f5c6629b740 1 Processor -- start 2024-09-10T10:03:28.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.138+0000 7f5c6629b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ec4a382d50 0x55ec4a38af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:28.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.138+0000 7f5c6629b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec4a383670 0x55ec4a38b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:28.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.138+0000 7f5c6629b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ec4a38df30 0x55ec4a38b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:28.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.138+0000 7f5c6629b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ec4a390ba0 con 0x55ec4a382d50 2024-09-10T10:03:28.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.138+0000 7f5c6629b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ec4a2a40f0 con 0x55ec4a38df30 2024-09-10T10:03:28.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.138+0000 7f5c6629b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ec4a390d20 con 0x55ec4a383670 2024-09-10T10:03:28.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.138+0000 7f5c6629b740 10 client.0 _ll_register_callbacks cb 0x55ec4a2959e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:03:28.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.139+0000 7f5c61800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ec4a38df30 0x55ec4a38b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:28.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.139+0000 7f5c61800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ec4a38df30 0x55ec4a38b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:43560/0 (socket says 172.21.15.77:43560) 2024-09-10T10:03:28.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.139+0000 7f5c61800640 1 -- 172.21.15.77:0/4186514753 learned_addr learned my addr 172.21.15.77:0/4186514753 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:03:28.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.139+0000 7f5c60e00640 1 --2- 172.21.15.77:0/4186514753 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ec4a382d50 0x55ec4a38af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:28.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.139+0000 7f5c61800640 1 -- 172.21.15.77:0/4186514753 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec4a383670 msgr2=0x55ec4a38b480 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T10:03:28.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.139+0000 7f5c61800640 1 --2- 172.21.15.77:0/4186514753 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec4a383670 0x55ec4a38b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.139+0000 7f5c61800640 1 -- 172.21.15.77:0/4186514753 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ec4a382d50 msgr2=0x55ec4a38af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:28.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.139+0000 7f5c61800640 1 --2- 172.21.15.77:0/4186514753 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ec4a382d50 0x55ec4a38af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.139+0000 7f5c61800640 1 -- 172.21.15.77:0/4186514753 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec4a05a3c0 con 0x55ec4a38df30 2024-09-10T10:03:28.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.140+0000 7f5c61800640 1 --2- 172.21.15.77:0/4186514753 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ec4a38df30 0x55ec4a38b9c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f5c5400bfd0 tx=0x7f5c54005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5c71e15eff6bc02 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:28.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.140+0000 7f5c46a00640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T10:03:28.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.140+0000 7f5c46a00640 1 -- 172.21.15.77:0/4186514753 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c54023070 con 0x55ec4a38df30 2024-09-10T10:03:28.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.140+0000 7f5c6629b740 1 -- 172.21.15.77:0/4186514753 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c4c0a4f90 con 0x55ec4a38df30 2024-09-10T10:03:28.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.140+0000 7f5c46a00640 1 -- 172.21.15.77:0/4186514753 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f5c5401f840 con 0x55ec4a38df30 2024-09-10T10:03:28.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.140+0000 7f5c46a00640 1 -- 172.21.15.77:0/4186514753 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f5c54031050 con 0x55ec4a38df30 2024-09-10T10:03:28.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.140+0000 7f5c6629b740 10 client.27122 fetch_fsmap learned FSMap version 35 2024-09-10T10:03:28.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.141+0000 7f5c6629b740 1 -- 172.21.15.77:0/4186514753 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55ec4a399890 con 0x55ec4a38df30 2024-09-10T10:03:28.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.141+0000 7f5c46a00640 1 -- 172.21.15.77:0/4186514753 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c4c0a4f90 con 0x55ec4a38df30 2024-09-10T10:03:28.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.141+0000 7f5c46a00640 1 -- 172.21.15.77:0/4186514753 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f5c54012be0 con 0x55ec4a38df30 2024-09-10T10:03:28.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.141+0000 7f5c6629b740 10 client.27122 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T10:03:28.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.141+0000 7f5c6629b740 10 client.27122 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T10:03:28.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.141+0000 7f5c6629b740 20 client.27122 populate_metadata read hostname 'smithi077' 2024-09-10T10:03:28.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.141+0000 7f5c6629b740 1 --2- 172.21.15.77:0/4186514753 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55ec4a39aca0 0x55ec4a39d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:28.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.141+0000 7f5c6629b740 4 client.27122 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:03:28.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.141+0000 7f5c6629b740 1 -- 172.21.15.77:0/4186514753 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ec4a2f6950 con 0x55ec4a39aca0 2024-09-10T10:03:28.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.142+0000 7f5c60e00640 1 --2- 172.21.15.77:0/4186514753 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55ec4a39aca0 0x55ec4a39d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:28.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.143+0000 7f5c60e00640 1 --2- 172.21.15.77:0/4186514753 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55ec4a39aca0 0x55ec4a39d0f0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d39d3be8a11eb442 server_cookie=f0f979acd1647c53 in_seq=0 out_seq=0 2024-09-10T10:03:28.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.143+0000 7f5c46a00640 10 client.27122 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T10:03:28.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.143+0000 7f5c46a00640 1 -- 172.21.15.77:0/4186514753 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ec4a2f6950 con 0x55ec4a39aca0 2024-09-10T10:03:28.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.143+0000 7f5c46a00640 10 client.27122 handle_command_reply: tid=0 2024-09-10T10:03:28.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.143+0000 7f5c6629b740 1 client.27122 shutdown 2024-09-10T10:03:28.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.144+0000 7f5c6629b740 1 -- 172.21.15.77:0/4186514753 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ec4a38df30 msgr2=0x55ec4a38b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:28.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.144+0000 7f5c6629b740 1 --2- 172.21.15.77:0/4186514753 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ec4a38df30 0x55ec4a38b9c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f5c5400bfd0 tx=0x7f5c54005850 comp rx=0 tx=0).stop 2024-09-10T10:03:28.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.144+0000 7f5c6629b740 1 -- 172.21.15.77:0/4186514753 shutdown_connections 2024-09-10T10:03:28.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.144+0000 7f5c6629b740 1 --2- 172.21.15.77:0/4186514753 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55ec4a39aca0 0x55ec4a39d0f0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.144+0000 7f5c6629b740 1 --2- 172.21.15.77:0/4186514753 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ec4a38df30 0x55ec4a38b9c0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.144+0000 7f5c6629b740 1 --2- 172.21.15.77:0/4186514753 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec4a383670 0x55ec4a38b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.144+0000 7f5c46a00640 0 client.27122 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T10:03:28.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.144+0000 7f5c6629b740 1 --2- 172.21.15.77:0/4186514753 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ec4a382d50 0x55ec4a38af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.144+0000 7f5c6629b740 1 -- 172.21.15.77:0/4186514753 >> 172.21.15.77:0/4186514753 conn(0x55ec4a37f2e0 msgr2=0x55ec4a384fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:28.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.144+0000 7f5c6629b740 1 -- 172.21.15.77:0/4186514753 shutdown_connections 2024-09-10T10:03:28.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.144+0000 7f5c6629b740 1 -- 172.21.15.77:0/4186514753 wait complete. 2024-09-10T10:03:28.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.144+0000 7f5c6629b740 20 client.27122 trim_cache size 0 max 16384 2024-09-10T10:03:28.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.171+0000 7f5c6629b740 1 Processor -- start 2024-09-10T10:03:28.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.171+0000 7f5c6629b740 1 -- start start 2024-09-10T10:03:28.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.172+0000 7f5c6629b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ec4a382d50 0x55ec4a38b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:28.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.172+0000 7f5c6629b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ec4a383670 0x55ec4a3a29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:28.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.172+0000 7f5c6629b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec4a38df30 0x55ec4a3a2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:28.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.172+0000 7f5c6629b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5c5c1acb60 con 0x55ec4a382d50 2024-09-10T10:03:28.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.172+0000 7f5c6629b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5c5c1ae6a0 con 0x55ec4a383670 2024-09-10T10:03:28.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.172+0000 7f5c6629b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5c5c1ad230 con 0x55ec4a38df30 2024-09-10T10:03:28.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.172+0000 7f5c61800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec4a38df30 0x55ec4a3a2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:28.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.172+0000 7f5c5be00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ec4a383670 0x55ec4a3a29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:28.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.172+0000 7f5c61800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec4a38df30 0x55ec4a3a2f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:53472/0 (socket says 172.21.15.77:53472) 2024-09-10T10:03:28.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.172+0000 7f5c61800640 1 -- 172.21.15.77:0/1176840330 learned_addr learned my addr 172.21.15.77:0/1176840330 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:03:28.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.172+0000 7f5c60e00640 1 --2- 172.21.15.77:0/1176840330 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ec4a382d50 0x55ec4a38b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:28.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.172+0000 7f5c5be00640 1 -- 172.21.15.77:0/1176840330 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec4a38df30 msgr2=0x55ec4a3a2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:28.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.172+0000 7f5c5be00640 1 --2- 172.21.15.77:0/1176840330 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec4a38df30 0x55ec4a3a2f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.172+0000 7f5c5be00640 1 -- 172.21.15.77:0/1176840330 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ec4a382d50 msgr2=0x55ec4a38b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:28.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.172+0000 7f5c5be00640 1 --2- 172.21.15.77:0/1176840330 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ec4a382d50 0x55ec4a38b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.172+0000 7f5c5be00640 1 -- 172.21.15.77:0/1176840330 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec4a3834a0 con 0x55ec4a383670 2024-09-10T10:03:28.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.173+0000 7f5c5be00640 1 --2- 172.21.15.77:0/1176840330 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ec4a383670 0x55ec4a3a29c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c0ffd30 tx=0x7f5c48032c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ceb021d4603ce96d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:28.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.173+0000 7f5c51600640 1 -- 172.21.15.77:0/1176840330 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c48035040 con 0x55ec4a383670 2024-09-10T10:03:28.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.173+0000 7f5c51600640 1 -- 172.21.15.77:0/1176840330 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c48004040 con 0x55ec4a383670 2024-09-10T10:03:28.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.173+0000 7f5c51600640 1 -- 172.21.15.77:0/1176840330 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c480046c0 con 0x55ec4a383670 2024-09-10T10:03:28.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.174+0000 7f5c6629b740 1 -- 172.21.15.77:0/1176840330 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ec4a383670 msgr2=0x55ec4a3a29c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:28.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.174+0000 7f5c6629b740 1 --2- 172.21.15.77:0/1176840330 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ec4a383670 0x55ec4a3a29c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c0ffd30 tx=0x7f5c48032c30 comp rx=0 tx=0).stop 2024-09-10T10:03:28.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.174+0000 7f5c6629b740 1 -- 172.21.15.77:0/1176840330 shutdown_connections 2024-09-10T10:03:28.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.174+0000 7f5c6629b740 1 --2- 172.21.15.77:0/1176840330 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec4a38df30 0x55ec4a3a2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.174+0000 7f5c6629b740 1 --2- 172.21.15.77:0/1176840330 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ec4a383670 0x55ec4a3a29c0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.174+0000 7f5c6629b740 1 --2- 172.21.15.77:0/1176840330 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ec4a382d50 0x55ec4a38b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.174+0000 7f5c6629b740 1 -- 172.21.15.77:0/1176840330 >> 172.21.15.77:0/1176840330 conn(0x55ec4a37f2e0 msgr2=0x55ec4a384fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:28.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.174+0000 7f5c6629b740 1 -- 172.21.15.77:0/1176840330 shutdown_connections 2024-09-10T10:03:28.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.174+0000 7f5c6629b740 1 -- 172.21.15.77:0/1176840330 wait complete. 2024-09-10T10:03:28.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.175+0000 7f5c6629b740 1 -- start start 2024-09-10T10:03:28.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.175+0000 7f5c6629b740 1 Processor -- start 2024-09-10T10:03:28.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.175+0000 7f5c6629b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ec4a382d50 0x55ec4a397500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:28.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.175+0000 7f5c6629b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ec4a383670 0x55ec4a397a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:28.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.175+0000 7f5c6629b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec4a38df30 0x55ec4a391a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:28.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.175+0000 7f5c6629b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5c5c1adfd0 con 0x55ec4a382d50 2024-09-10T10:03:28.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.176+0000 7f5c6629b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5c5c1ad900 con 0x55ec4a383670 2024-09-10T10:03:28.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.176+0000 7f5c6629b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5c5c1aab40 con 0x55ec4a38df30 2024-09-10T10:03:28.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.176+0000 7f5c6629b740 10 client.0 _ll_register_callbacks cb 0x7f5c54014050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:03:28.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.176+0000 7f5c60e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ec4a382d50 0x55ec4a397500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:28.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.176+0000 7f5c60e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ec4a382d50 0x55ec4a397500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52492/0 (socket says 172.21.15.77:52492) 2024-09-10T10:03:28.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.176+0000 7f5c60e00640 1 -- 172.21.15.77:0/910084456 learned_addr learned my addr 172.21.15.77:0/910084456 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:03:28.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.176+0000 7f5c5be00640 1 --2- 172.21.15.77:0/910084456 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ec4a383670 0x55ec4a397a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:28.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.176+0000 7f5c61800640 1 --2- 172.21.15.77:0/910084456 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec4a38df30 0x55ec4a391a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:28.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.176+0000 7f5c60e00640 1 -- 172.21.15.77:0/910084456 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec4a38df30 msgr2=0x55ec4a391a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:28.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.176+0000 7f5c60e00640 1 --2- 172.21.15.77:0/910084456 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec4a38df30 0x55ec4a391a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.176+0000 7f5c60e00640 1 -- 172.21.15.77:0/910084456 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ec4a383670 msgr2=0x55ec4a397a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:28.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.176+0000 7f5c60e00640 1 --2- 172.21.15.77:0/910084456 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ec4a383670 0x55ec4a397a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.176+0000 7f5c60e00640 1 -- 172.21.15.77:0/910084456 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec4a3922a0 con 0x55ec4a382d50 2024-09-10T10:03:28.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.176+0000 7f5c61800640 1 --2- 172.21.15.77:0/910084456 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec4a38df30 0x55ec4a391a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:03:28.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.176+0000 7f5c60e00640 1 --2- 172.21.15.77:0/910084456 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ec4a382d50 0x55ec4a397500 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c09bfd0 tx=0x7f5c4c0985c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87c5cab4f5e98e22 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:28.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.176+0000 7f5c46a00640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-10T10:03:28.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.177+0000 7f5c46a00640 1 -- 172.21.15.77:0/910084456 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c4c01fe70 con 0x55ec4a382d50 2024-09-10T10:03:28.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.177+0000 7f5c46a00640 1 -- 172.21.15.77:0/910084456 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f5c4c098990 con 0x55ec4a382d50 2024-09-10T10:03:28.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.177+0000 7f5c6629b740 1 -- 172.21.15.77:0/910084456 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c48004040 con 0x55ec4a382d50 2024-09-10T10:03:28.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.177+0000 7f5c46a00640 1 -- 172.21.15.77:0/910084456 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f5c4c0a4f90 con 0x55ec4a382d50 2024-09-10T10:03:28.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.177+0000 7f5c6629b740 10 client.16992 fetch_fsmap learned FSMap version 35 2024-09-10T10:03:28.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.177+0000 7f5c6629b740 1 -- 172.21.15.77:0/910084456 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55ec4a42a870 con 0x55ec4a382d50 2024-09-10T10:03:28.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.177+0000 7f5c46a00640 1 -- 172.21.15.77:0/910084456 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c48004040 con 0x55ec4a382d50 2024-09-10T10:03:28.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.178+0000 7f5c46a00640 1 -- 172.21.15.77:0/910084456 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f5c4c0a0170 con 0x55ec4a382d50 2024-09-10T10:03:28.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.178+0000 7f5c6629b740 10 client.16992 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T10:03:28.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.178+0000 7f5c6629b740 10 client.16992 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T10:03:28.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.178+0000 7f5c6629b740 20 client.16992 populate_metadata read hostname 'smithi077' 2024-09-10T10:03:28.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.178+0000 7f5c6629b740 1 --2- 172.21.15.77:0/910084456 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55ec4a42ac70 0x55ec4a42d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:28.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.178+0000 7f5c6629b740 4 client.16992 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T10:03:28.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.178+0000 7f5c6629b740 1 -- 172.21.15.77:0/910084456 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ec4a2f0e60 con 0x55ec4a42ac70 2024-09-10T10:03:28.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.178+0000 7f5c5be00640 1 --2- 172.21.15.77:0/910084456 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55ec4a42ac70 0x55ec4a42d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:28.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.178+0000 7f5c5be00640 1 --2- 172.21.15.77:0/910084456 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55ec4a42ac70 0x55ec4a42d060 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c1fa4b4b27054f3 server_cookie=85ea7ba5330f819a in_seq=0 out_seq=0 2024-09-10T10:03:28.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.178+0000 7f5c46a00640 10 client.16992 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T10:03:28.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.179+0000 7f5c46a00640 1 -- 172.21.15.77:0/910084456 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ec4a2f0e60 con 0x55ec4a42ac70 2024-09-10T10:03:28.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.179+0000 7f5c46a00640 10 client.16992 handle_command_reply: tid=0 2024-09-10T10:03:28.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.179+0000 7f5c6629b740 1 client.16992 shutdown 2024-09-10T10:03:28.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.179+0000 7f5c6629b740 1 -- 172.21.15.77:0/910084456 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ec4a382d50 msgr2=0x55ec4a397500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:28.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.179+0000 7f5c6629b740 1 --2- 172.21.15.77:0/910084456 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ec4a382d50 0x55ec4a397500 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c09bfd0 tx=0x7f5c4c0985c0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.179+0000 7f5c6629b740 1 -- 172.21.15.77:0/910084456 shutdown_connections 2024-09-10T10:03:28.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.179+0000 7f5c6629b740 1 --2- 172.21.15.77:0/910084456 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55ec4a42ac70 0x55ec4a42d060 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.179+0000 7f5c6629b740 1 --2- 172.21.15.77:0/910084456 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec4a38df30 0x55ec4a391a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.179+0000 7f5c6629b740 1 --2- 172.21.15.77:0/910084456 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ec4a383670 0x55ec4a397a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.180+0000 7f5c6629b740 1 --2- 172.21.15.77:0/910084456 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ec4a382d50 0x55ec4a397500 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.180+0000 7f5c6629b740 1 -- 172.21.15.77:0/910084456 >> 172.21.15.77:0/910084456 conn(0x55ec4a37f2e0 msgr2=0x55ec4a389810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:28.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.180+0000 7f5c46a00640 0 client.16992 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T10:03:28.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.180+0000 7f5c6629b740 1 -- 172.21.15.77:0/910084456 shutdown_connections 2024-09-10T10:03:28.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.180+0000 7f5c6629b740 1 -- 172.21.15.77:0/910084456 wait complete. 2024-09-10T10:03:28.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.180+0000 7f5c6629b740 20 client.16992 trim_cache size 0 max 16384 2024-09-10T10:03:28.184 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-10T10:03:28.184 INFO:teuthology.orchestra.run.smithi077.stdout: "status": "no active scrubs running", 2024-09-10T10:03:28.184 INFO:teuthology.orchestra.run.smithi077.stdout: "scrubs": {} 2024-09-10T10:03:28.184 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-10T10:03:28.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.181+0000 7f5c62200640 1 -- 172.21.15.77:0/2830524389 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f5c400a2c00 msgr2=0x7f5c400a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:28.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.181+0000 7f5c62200640 1 --2- 172.21.15.77:0/2830524389 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f5c400a2c00 0x7f5c400a50c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f5c5400be80 tx=0x7f5c540147e0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.181+0000 7f5c62200640 1 -- 172.21.15.77:0/2830524389 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5c5c103e30 msgr2=0x7f5c5c19a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:28.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.181+0000 7f5c62200640 1 --2- 172.21.15.77:0/2830524389 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5c5c103e30 0x7f5c5c19a480 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c00eb00 tx=0x7f5c4c00c2d0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.181+0000 7f5c62200640 1 -- 172.21.15.77:0/2830524389 shutdown_connections 2024-09-10T10:03:28.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.181+0000 7f5c62200640 1 --2- 172.21.15.77:0/2830524389 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f5c400a2c00 0x7f5c400a50c0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.181+0000 7f5c62200640 1 --2- 172.21.15.77:0/2830524389 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5c5c108f60 0x7f5c5c19ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.181+0000 7f5c62200640 1 --2- 172.21.15.77:0/2830524389 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5c5c1047e0 0x7f5c5c19a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.181+0000 7f5c62200640 1 --2- 172.21.15.77:0/2830524389 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5c5c103e30 0x7f5c5c19a480 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.181+0000 7f5c62200640 1 -- 172.21.15.77:0/2830524389 >> 172.21.15.77:0/2830524389 conn(0x7f5c5c0ff910 msgr2=0x7f5c5c100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:28.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.183+0000 7f5c62200640 1 -- 172.21.15.77:0/2830524389 shutdown_connections 2024-09-10T10:03:28.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:28.183+0000 7f5c62200640 1 -- 172.21.15.77:0/2830524389 wait complete. 2024-09-10T10:03:29.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:03:28 smithi077 ceph-mon[30319]: pgmap v640: 129 pgs: 129 active+clean; 152 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 1.2 MiB/s wr, 466 op/s 2024-09-10T10:03:29.325 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T10:03:29.326 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:03:29.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:03:28 smithi134 ceph-mon[35674]: pgmap v640: 129 pgs: 129 active+clean; 152 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 1.2 MiB/s wr, 466 op/s 2024-09-10T10:03:29.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:03:28 smithi120 ceph-mon[34770]: pgmap v640: 129 pgs: 129 active+clean; 152 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 1.2 MiB/s wr, 466 op/s 2024-09-10T10:03:29.809 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:03:30.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:30.434+0000 7f5fddc00640 1 -- 172.21.15.77:0/1558555533 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fd8103ff0 msgr2=0x7f5fd81043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:30.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:30.434+0000 7f5fddc00640 1 --2- 172.21.15.77:0/1558555533 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fd8103ff0 0x7f5fd81043d0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f5fc8009900 tx=0x7f5fc802a3f0 comp rx=0 tx=0).stop 2024-09-10T10:03:30.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:30.434+0000 7f5fddc00640 1 -- 172.21.15.77:0/1558555533 shutdown_connections 2024-09-10T10:03:30.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:30.434+0000 7f5fddc00640 1 --2- 172.21.15.77:0/1558555533 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5fd8109120 0x7f5fd8110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:30.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:30.434+0000 7f5fddc00640 1 --2- 172.21.15.77:0/1558555533 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5fd81049a0 0x7f5fd81089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:30.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:30.434+0000 7f5fddc00640 1 --2- 172.21.15.77:0/1558555533 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fd8103ff0 0x7f5fd81043d0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:30.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:30.434+0000 7f5fddc00640 1 -- 172.21.15.77:0/1558555533 >> 172.21.15.77:0/1558555533 conn(0x7f5fd80ffa90 msgr2=0x7f5fd8101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:30.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:30.435+0000 7f5fddc00640 1 -- 172.21.15.77:0/1558555533 shutdown_connections 2024-09-10T10:03:30.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:30.435+0000 7f5fddc00640 1 -- 172.21.15.77:0/1558555533 wait complete. 2024-09-10T10:03:30.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:30.436+0000 7f5fddc00640 1 Processor -- start 2024-09-10T10:03:30.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:30.436+0000 7f5fddc00640 1 -- start start 2024-09-10T10:03:30.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:30.436+0000 7f5fddc00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5fd8103ff0 0x7f5fd819a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:30.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:30.436+0000 7f5fddc00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5fd81049a0 0x7f5fd819abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:30.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:30.436+0000 7f5fddc00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fd8109120 0x7f5fd819ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:30.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:30.436+0000 7f5fddc00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd8114170 con 0x7f5fd81049a0 2024-09-10T10:03:30.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:30.436+0000 7f5fddc00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd8113ff0 con 0x7f5fd8103ff0 2024-09-10T10:03:30.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:30.436+0000 7f5fddc00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd81142f0 con 0x7f5fd8109120 2024-09-10T10:03:30.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:30.437+0000 7f5fd6a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5fd8103ff0 0x7f5fd819a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:30.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:30.437+0000 7f5fd6a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5fd8103ff0 0x7f5fd819a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:43588/0 (socket says 172.21.15.77:43588) 2024-09-10T10:03:30.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:30.437+0000 7f5fd6a00640 1 -- 172.21.15.77:0/2871694399 learned_addr learned my addr 172.21.15.77:0/2871694399 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:03:30.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:30.437+0000 7f5fd7400640 1 --2- 172.21.15.77:0/2871694399 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fd8109120 0x7f5fd819ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:30.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:30.437+0000 7f5fd6a00640 1 -- 172.21.15.77:0/2871694399 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fd8109120 msgr2=0x7f5fd819ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:30.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:30.437+0000 7f5fd6a00640 1 --2- 172.21.15.77:0/2871694399 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fd8109120 0x7f5fd819ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:30.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:30.437+0000 7f5fd6000640 1 --2- 172.21.15.77:0/2871694399 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5fd81049a0 0x7f5fd819abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:30.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:30.437+0000 7f5fd6a00640 1 -- 172.21.15.77:0/2871694399 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5fd81049a0 msgr2=0x7f5fd819abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:30.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:30.437+0000 7f5fd6a00640 1 --2- 172.21.15.77:0/2871694399 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5fd81049a0 0x7f5fd819abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:30.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:30.437+0000 7f5fd6a00640 1 -- 172.21.15.77:0/2871694399 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fd819f620 con 0x7f5fd8103ff0 2024-09-10T10:03:30.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:30.437+0000 7f5fd6000640 1 --2- 172.21.15.77:0/2871694399 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5fd81049a0 0x7f5fd819abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:03:30.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:30.437+0000 7f5fd6a00640 1 --2- 172.21.15.77:0/2871694399 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5fd8103ff0 0x7f5fd819a670 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f5fc8004770 tx=0x7f5fc8003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:30.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:30.438+0000 7f5fcf400640 1 -- 172.21.15.77:0/2871694399 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc803f070 con 0x7f5fd8103ff0 2024-09-10T10:03:30.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:30.438+0000 7f5fddc00640 1 -- 172.21.15.77:0/2871694399 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fd819f8b0 con 0x7f5fd8103ff0 2024-09-10T10:03:30.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:30.438+0000 7f5fcf400640 1 -- 172.21.15.77:0/2871694399 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fc8031ca0 con 0x7f5fd8103ff0 2024-09-10T10:03:30.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:30.438+0000 7f5fcf400640 1 -- 172.21.15.77:0/2871694399 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc8035030 con 0x7f5fd8103ff0 2024-09-10T10:03:30.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:30.438+0000 7f5fddc00640 1 -- 172.21.15.77:0/2871694399 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fd81a71f0 con 0x7f5fd8103ff0 2024-09-10T10:03:30.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:30.439+0000 7f5fddc00640 1 -- 172.21.15.77:0/2871694399 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fd81043d0 con 0x7f5fd8103ff0 2024-09-10T10:03:30.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:30.441+0000 7f5fcf400640 1 -- 172.21.15.77:0/2871694399 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5fc8042070 con 0x7f5fd8103ff0 2024-09-10T10:03:30.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:30.442+0000 7f5fcf400640 1 --2- 172.21.15.77:0/2871694399 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f5fbc0a2b80 0x7f5fbc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:30.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:30.443+0000 7f5fcf400640 1 -- 172.21.15.77:0/2871694399 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f5fc806de00 con 0x7f5fd8103ff0 2024-09-10T10:03:30.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:30.443+0000 7f5fd6000640 1 --2- 172.21.15.77:0/2871694399 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f5fbc0a2b80 0x7f5fbc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:30.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:30.443+0000 7f5fd6000640 1 --2- 172.21.15.77:0/2871694399 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f5fbc0a2b80 0x7f5fbc0a5040 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f5fd80fff40 tx=0x7f5fc4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:30.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:30.444+0000 7f5fcf400640 1 -- 172.21.15.77:0/2871694399 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5fc80bc820 con 0x7f5fd8103ff0 2024-09-10T10:03:30.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:30.719+0000 7f5fddc00640 1 -- 172.21.15.77:0/2871694399 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5fd810c300 con 0x7f5fd8103ff0 2024-09-10T10:03:30.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:30.721+0000 7f5fcf400640 1 -- 172.21.15.77:0/2871694399 <== mon.2 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11989 (secure 0 0 0) 0x7f5fc80bc120 con 0x7f5fd8103ff0 2024-09-10T10:03:30.723 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T10:03:30.723 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":35,"btime":"2024-09-10T10:01:05:922537+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3159289675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3159289675},{"type":"v1","addr":"172.21.15.77:6835","nonce":3159289675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1951962341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1951962341},{"type":"v1","addr":"172.21.15.77:6837","nonce":1951962341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":35,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:52:23.368252+0000","modified":"2024-09-10T10:01:05.706646+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24563,"mds_1":24524,"mds_2":24520,"mds_3":24502,"mds_4":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14487":{"gid":14487,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/4291121888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":4291121888},{"type":"v1","addr":"172.21.15.77:6839","nonce":4291121888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14499":{"gid":14499,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6841/445284194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":445284194},{"type":"v1","addr":"172.21.15.77:6841","nonce":445284194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1005054423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1005054423},{"type":"v1","addr":"172.21.15.134:6833","nonce":1005054423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6833/3716307744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":3716307744},{"type":"v1","addr":"172.21.15.120:6833","nonce":3716307744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/1319892544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1319892544},{"type":"v1","addr":"172.21.15.120:6835","nonce":1319892544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.134:6835/3293933699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3293933699},{"type":"v1","addr":"172.21.15.134:6835","nonce":3293933699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.134:6837/2461497091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2461497091},{"type":"v1","addr":"172.21.15.134:6837","nonce":2461497091}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":20,"addr":"172.21.15.134:6839/1965117680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1965117680},{"type":"v1","addr":"172.21.15.134:6839","nonce":1965117680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":20,"addr":"172.21.15.120:6837/3033280709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3033280709},{"type":"v1","addr":"172.21.15.120:6837","nonce":3033280709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.120:6839/1777710157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1777710157},{"type":"v1","addr":"172.21.15.120:6839","nonce":1777710157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24563,"qdb_cluster":[24496,24502,24520,24524,24563]},"id":1}]} 2024-09-10T10:03:30.724 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 35 2024-09-10T10:03:30.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:30.726+0000 7f5fddc00640 1 -- 172.21.15.77:0/2871694399 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f5fbc0a2b80 msgr2=0x7f5fbc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:30.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:30.726+0000 7f5fddc00640 1 --2- 172.21.15.77:0/2871694399 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f5fbc0a2b80 0x7f5fbc0a5040 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f5fd80fff40 tx=0x7f5fc4005f90 comp rx=0 tx=0).stop 2024-09-10T10:03:30.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:30.726+0000 7f5fddc00640 1 -- 172.21.15.77:0/2871694399 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5fd8103ff0 msgr2=0x7f5fd819a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:30.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:30.726+0000 7f5fddc00640 1 --2- 172.21.15.77:0/2871694399 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5fd8103ff0 0x7f5fd819a670 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f5fc8004770 tx=0x7f5fc8003c10 comp rx=0 tx=0).stop 2024-09-10T10:03:30.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:30.726+0000 7f5fddc00640 1 -- 172.21.15.77:0/2871694399 shutdown_connections 2024-09-10T10:03:30.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:30.726+0000 7f5fddc00640 1 --2- 172.21.15.77:0/2871694399 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f5fbc0a2b80 0x7f5fbc0a5040 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:30.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:30.726+0000 7f5fddc00640 1 --2- 172.21.15.77:0/2871694399 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fd8109120 0x7f5fd819ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:30.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:30.726+0000 7f5fddc00640 1 --2- 172.21.15.77:0/2871694399 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5fd81049a0 0x7f5fd819abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:30.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:30.726+0000 7f5fddc00640 1 --2- 172.21.15.77:0/2871694399 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5fd8103ff0 0x7f5fd819a670 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:30.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:30.726+0000 7f5fddc00640 1 -- 172.21.15.77:0/2871694399 >> 172.21.15.77:0/2871694399 conn(0x7f5fd80ffa90 msgr2=0x7f5fd8100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:30.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:30.726+0000 7f5fddc00640 1 -- 172.21.15.77:0/2871694399 shutdown_connections 2024-09-10T10:03:30.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:30.727+0000 7f5fddc00640 1 -- 172.21.15.77:0/2871694399 wait complete. 2024-09-10T10:03:31.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:03:30 smithi134 ceph-mon[35674]: pgmap v641: 129 pgs: 129 active+clean; 144 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.3 MiB/s wr, 519 op/s 2024-09-10T10:03:31.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:03:30 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/2871694399' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:03:31.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:03:30 smithi120 ceph-mon[34770]: pgmap v641: 129 pgs: 129 active+clean; 144 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.3 MiB/s wr, 519 op/s 2024-09-10T10:03:31.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:03:30 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/2871694399' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:03:31.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:03:30 smithi077 ceph-mon[30319]: pgmap v641: 129 pgs: 129 active+clean; 144 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.3 MiB/s wr, 519 op/s 2024-09-10T10:03:31.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:03:30 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/2871694399' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:03:31.766 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T10:03:32.234 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:03:32.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.861+0000 7f5f1d600640 1 -- 172.21.15.77:0/4287670510 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5f18074620 msgr2=0x7f5f18074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:32.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.861+0000 7f5f1d600640 1 --2- 172.21.15.77:0/4287670510 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5f18074620 0x7f5f18074a00 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f5f08009900 tx=0x7f5f0802a480 comp rx=0 tx=0).stop 2024-09-10T10:03:32.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.861+0000 7f5f1d600640 1 -- 172.21.15.77:0/4287670510 shutdown_connections 2024-09-10T10:03:32.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.861+0000 7f5f1d600640 1 --2- 172.21.15.77:0/4287670510 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5f1810e3c0 0x7f5f18110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:32.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.861+0000 7f5f1d600640 1 --2- 172.21.15.77:0/4287670510 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5f18074f40 0x7f5f1810de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:32.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.861+0000 7f5f1d600640 1 --2- 172.21.15.77:0/4287670510 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5f18074620 0x7f5f18074a00 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:32.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.861+0000 7f5f1d600640 1 -- 172.21.15.77:0/4287670510 >> 172.21.15.77:0/4287670510 conn(0x7f5f180fd8c0 msgr2=0x7f5f180ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:32.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.861+0000 7f5f1d600640 1 -- 172.21.15.77:0/4287670510 shutdown_connections 2024-09-10T10:03:32.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.861+0000 7f5f1d600640 1 -- 172.21.15.77:0/4287670510 wait complete. 2024-09-10T10:03:32.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.862+0000 7f5f1d600640 1 Processor -- start 2024-09-10T10:03:32.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.862+0000 7f5f1d600640 1 -- start start 2024-09-10T10:03:32.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.862+0000 7f5f1d600640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5f18074620 0x7f5f1819a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:32.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.862+0000 7f5f1d600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5f18074f40 0x7f5f1819ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:32.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.862+0000 7f5f1d600640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5f1810e3c0 0x7f5f1819efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:32.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.862+0000 7f5f1d600640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5f18114150 con 0x7f5f18074620 2024-09-10T10:03:32.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.863+0000 7f5f1d600640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5f18113fd0 con 0x7f5f1810e3c0 2024-09-10T10:03:32.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.863+0000 7f5f1d600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5f181142d0 con 0x7f5f18074f40 2024-09-10T10:03:32.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.863+0000 7f5f16000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5f18074f40 0x7f5f1819ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:32.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.863+0000 7f5f17400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5f1810e3c0 0x7f5f1819efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:32.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.863+0000 7f5f16000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5f18074f40 0x7f5f1819ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:56260/0 (socket says 172.21.15.77:56260) 2024-09-10T10:03:32.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.863+0000 7f5f16000640 1 -- 172.21.15.77:0/1949649886 learned_addr learned my addr 172.21.15.77:0/1949649886 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:03:32.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.863+0000 7f5f16a00640 1 --2- 172.21.15.77:0/1949649886 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5f18074620 0x7f5f1819a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:32.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.863+0000 7f5f17400640 1 -- 172.21.15.77:0/1949649886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5f18074f40 msgr2=0x7f5f1819ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:32.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.863+0000 7f5f17400640 1 --2- 172.21.15.77:0/1949649886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5f18074f40 0x7f5f1819ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:32.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.863+0000 7f5f17400640 1 -- 172.21.15.77:0/1949649886 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5f18074620 msgr2=0x7f5f1819a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:32.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.863+0000 7f5f17400640 1 --2- 172.21.15.77:0/1949649886 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5f18074620 0x7f5f1819a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:32.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.863+0000 7f5f17400640 1 -- 172.21.15.77:0/1949649886 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f1819f6c0 con 0x7f5f1810e3c0 2024-09-10T10:03:32.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.864+0000 7f5f17400640 1 --2- 172.21.15.77:0/1949649886 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5f1810e3c0 0x7f5f1819efe0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f5f10009fd0 tx=0x7f5f1000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:32.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.864+0000 7f5f0f400640 1 -- 172.21.15.77:0/1949649886 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f1000ce20 con 0x7f5f1810e3c0 2024-09-10T10:03:32.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.864+0000 7f5f1d600640 1 -- 172.21.15.77:0/1949649886 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f1819f9b0 con 0x7f5f1810e3c0 2024-09-10T10:03:32.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.864+0000 7f5f0f400640 1 -- 172.21.15.77:0/1949649886 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f10005020 con 0x7f5f1810e3c0 2024-09-10T10:03:32.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.864+0000 7f5f0f400640 1 -- 172.21.15.77:0/1949649886 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f10010720 con 0x7f5f1810e3c0 2024-09-10T10:03:32.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.864+0000 7f5f1d600640 1 -- 172.21.15.77:0/1949649886 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f181a7290 con 0x7f5f1810e3c0 2024-09-10T10:03:32.868 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.867+0000 7f5f0f400640 1 -- 172.21.15.77:0/1949649886 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5f10004b20 con 0x7f5f1810e3c0 2024-09-10T10:03:32.868 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.867+0000 7f5f2003a740 1 Processor -- start 2024-09-10T10:03:32.868 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.867+0000 7f5f2003a740 1 -- start start 2024-09-10T10:03:32.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.868+0000 7f5f2003a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f96c34c70 0x564f96c35050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:32.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.868+0000 7f5f2003a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f96c35590 0x564f96c3f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:32.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.868+0000 7f5f2003a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564f96c3fe50 0x564f96c42240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:32.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.868+0000 7f5f2003a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x564f96b9b3e0 con 0x564f96c3fe50 2024-09-10T10:03:32.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.868+0000 7f5f2003a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x564f96b42d80 con 0x564f96c35590 2024-09-10T10:03:32.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.868+0000 7f5f2003a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x564f96b973a0 con 0x564f96c34c70 2024-09-10T10:03:32.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.868+0000 7f5f16a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f96c35590 0x564f96c3f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:32.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.868+0000 7f5f16000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f96c34c70 0x564f96c35050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:32.870 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.868+0000 7f5f16000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564f96c3fe50 0x564f96c42240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:32.870 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.868+0000 7f5f16a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f96c35590 0x564f96c3f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:46056/0 (socket says 172.21.15.77:46056) 2024-09-10T10:03:32.871 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.868+0000 7f5f16a00640 1 -- 172.21.15.77:0/3412754601 learned_addr learned my addr 172.21.15.77:0/3412754601 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:03:32.871 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.869+0000 7f5f16a00640 1 -- 172.21.15.77:0/3412754601 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f96c34c70 msgr2=0x564f96c35050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:32.871 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.869+0000 7f5f16a00640 1 --2- 172.21.15.77:0/3412754601 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f96c34c70 0x564f96c35050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:32.871 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.869+0000 7f5f16a00640 1 -- 172.21.15.77:0/3412754601 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564f96c3fe50 msgr2=0x564f96c42240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:32.871 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.869+0000 7f5f16a00640 1 --2- 172.21.15.77:0/3412754601 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564f96c3fe50 0x564f96c42240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:32.871 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.869+0000 7f5f16a00640 1 -- 172.21.15.77:0/3412754601 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f96b61ae0 con 0x564f96c35590 2024-09-10T10:03:32.871 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.869+0000 7f5f0f400640 1 --2- 172.21.15.77:0/1949649886 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f5efc0a2b80 0x7f5efc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:32.871 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.869+0000 7f5f16a00640 1 --2- 172.21.15.77:0/3412754601 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f96c35590 0x564f96c3f910 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f5f08009900 tx=0x7f5f08005be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=979a505dc9bdbc39 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:32.871 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.869+0000 7f5f0f400640 1 -- 172.21.15.77:0/1949649886 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f5f100cfa80 con 0x7f5f1810e3c0 2024-09-10T10:03:32.871 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.869+0000 7f5f16000640 1 --2- 172.21.15.77:0/1949649886 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f5efc0a2b80 0x7f5efc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:32.872 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.870+0000 7f5f03400640 1 -- 172.21.15.77:0/3412754601 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f08040070 con 0x564f96c35590 2024-09-10T10:03:32.872 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.870+0000 7f5f03400640 1 -- 172.21.15.77:0/3412754601 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f08031870 con 0x564f96c35590 2024-09-10T10:03:32.872 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.870+0000 7f5f03400640 1 -- 172.21.15.77:0/3412754601 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f08035040 con 0x564f96c35590 2024-09-10T10:03:32.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.870+0000 7f5f2003a740 1 -- 172.21.15.77:0/3412754601 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f96c35590 msgr2=0x564f96c3f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:32.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.870+0000 7f5f2003a740 1 --2- 172.21.15.77:0/3412754601 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f96c35590 0x564f96c3f910 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f5f08009900 tx=0x7f5f08005be0 comp rx=0 tx=0).stop 2024-09-10T10:03:32.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.870+0000 7f5f16000640 1 --2- 172.21.15.77:0/1949649886 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f5efc0a2b80 0x7f5efc0a5040 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x564f96c32580 tx=0x7f5f04004730 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:32.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.871+0000 7f5f2003a740 1 -- 172.21.15.77:0/3412754601 shutdown_connections 2024-09-10T10:03:32.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.871+0000 7f5f2003a740 1 --2- 172.21.15.77:0/3412754601 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564f96c3fe50 0x564f96c42240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:32.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.871+0000 7f5f2003a740 1 --2- 172.21.15.77:0/3412754601 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f96c35590 0x564f96c3f910 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:32.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.871+0000 7f5f2003a740 1 --2- 172.21.15.77:0/3412754601 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f96c34c70 0x564f96c35050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:32.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.871+0000 7f5f2003a740 1 -- 172.21.15.77:0/3412754601 >> 172.21.15.77:0/3412754601 conn(0x564f96c31200 msgr2=0x564f96c315e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:32.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.871+0000 7f5f2003a740 1 -- 172.21.15.77:0/3412754601 shutdown_connections 2024-09-10T10:03:32.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.871+0000 7f5f2003a740 1 -- 172.21.15.77:0/3412754601 wait complete. 2024-09-10T10:03:32.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.871+0000 7f5f2003a740 1 -- start start 2024-09-10T10:03:32.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.871+0000 7f5f2003a740 1 Processor -- start 2024-09-10T10:03:32.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.872+0000 7f5f2003a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564f96c34c70 0x564f96c3ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:32.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.872+0000 7f5f2003a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f96c35590 0x564f96c3d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:32.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.872+0000 7f5f2003a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f96c3fe50 0x564f96c3d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:32.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.872+0000 7f5f2003a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x564f96c42a20 con 0x564f96c34c70 2024-09-10T10:03:32.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.872+0000 7f5f2003a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x564f96b560f0 con 0x564f96c35590 2024-09-10T10:03:32.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.872+0000 7f5f2003a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x564f96c42ba0 con 0x564f96c3fe50 2024-09-10T10:03:32.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.872+0000 7f5f2003a740 10 client.0 _ll_register_callbacks cb 0x564f96b3dbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:03:32.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.872+0000 7f5f16000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f96c35590 0x564f96c3d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:32.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.872+0000 7f5f16a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564f96c34c70 0x564f96c3ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:32.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.872+0000 7f5f16000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f96c35590 0x564f96c3d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:46072/0 (socket says 172.21.15.77:46072) 2024-09-10T10:03:32.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.872+0000 7f5f16000640 1 -- 172.21.15.77:0/1131954175 learned_addr learned my addr 172.21.15.77:0/1131954175 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:03:32.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.872+0000 7f5f17400640 1 --2- 172.21.15.77:0/1131954175 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f96c3fe50 0x564f96c3d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:32.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.873+0000 7f5f17400640 1 -- 172.21.15.77:0/1131954175 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f96c35590 msgr2=0x564f96c3d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:32.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.873+0000 7f5f17400640 1 --2- 172.21.15.77:0/1131954175 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f96c35590 0x564f96c3d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:32.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.873+0000 7f5f17400640 1 -- 172.21.15.77:0/1131954175 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564f96c34c70 msgr2=0x564f96c3ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:32.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.873+0000 7f5f17400640 1 --2- 172.21.15.77:0/1131954175 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564f96c34c70 0x564f96c3ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:32.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.873+0000 7f5f17400640 1 -- 172.21.15.77:0/1131954175 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f9690c3c0 con 0x564f96c3fe50 2024-09-10T10:03:32.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.873+0000 7f5f16000640 1 --2- 172.21.15.77:0/1131954175 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f96c35590 0x564f96c3d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:03:32.877 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.873+0000 7f5f17400640 1 --2- 172.21.15.77:0/1131954175 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f96c3fe50 0x564f96c3d730 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f5f100cd190 tx=0x7f5f1009c6b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4e37cd412a64e33 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:32.877 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.873+0000 7f5ef7e00640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-10T10:03:32.877 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.873+0000 7f5ef7e00640 1 -- 172.21.15.77:0/1131954175 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f100b4020 con 0x564f96c3fe50 2024-09-10T10:03:32.878 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.873+0000 7f5ef7e00640 1 -- 172.21.15.77:0/1131954175 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f5f100051d0 con 0x564f96c3fe50 2024-09-10T10:03:32.878 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.873+0000 7f5f2003a740 1 -- 172.21.15.77:0/1131954175 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f08031870 con 0x564f96c3fe50 2024-09-10T10:03:32.878 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.873+0000 7f5ef7e00640 1 -- 172.21.15.77:0/1131954175 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f5f100a4b60 con 0x564f96c3fe50 2024-09-10T10:03:32.878 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.873+0000 7f5f2003a740 10 client.27100 fetch_fsmap learned FSMap version 35 2024-09-10T10:03:32.878 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.873+0000 7f5f2003a740 1 -- 172.21.15.77:0/1131954175 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x564f96c4ea40 con 0x564f96c3fe50 2024-09-10T10:03:32.878 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.874+0000 7f5ef7e00640 1 -- 172.21.15.77:0/1131954175 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f08031870 con 0x564f96c3fe50 2024-09-10T10:03:32.878 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.874+0000 7f5ef7e00640 1 -- 172.21.15.77:0/1131954175 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f5f100bab30 con 0x564f96c3fe50 2024-09-10T10:03:32.878 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.874+0000 7f5f2003a740 10 client.27100 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T10:03:32.878 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.874+0000 7f5f2003a740 10 client.27100 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-10T10:03:32.878 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.874+0000 7f5f2003a740 20 client.27100 populate_metadata read hostname 'smithi077' 2024-09-10T10:03:32.878 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.874+0000 7f5f2003a740 1 --2- 172.21.15.77:0/1131954175 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x564f96c4fef0 0x564f96c52360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:32.878 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.874+0000 7f5f2003a740 4 client.27100 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:03:32.878 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.874+0000 7f5f2003a740 1 -- 172.21.15.77:0/1131954175 --> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564f96ba8950 con 0x564f96c4fef0 2024-09-10T10:03:32.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.874+0000 7f5f16a00640 1 --2- 172.21.15.77:0/1131954175 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x564f96c4fef0 0x564f96c52360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:32.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.875+0000 7f5f16a00640 1 --2- 172.21.15.77:0/1131954175 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x564f96c4fef0 0x564f96c52360 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=867cb318afb3a413 server_cookie=ee69bf6f878b2b6b in_seq=0 out_seq=0 2024-09-10T10:03:32.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.875+0000 7f5ef7e00640 10 client.27100 ms_handle_connect on v2:172.21.15.134:6834/3293933699 2024-09-10T10:03:32.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.876+0000 7f5ef7e00640 1 -- 172.21.15.77:0/1131954175 <== mds.4 v2:172.21.15.134:6834/3293933699 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564f96ba8950 con 0x564f96c4fef0 2024-09-10T10:03:32.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.876+0000 7f5ef7e00640 10 client.27100 handle_command_reply: tid=0 2024-09-10T10:03:32.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.876+0000 7f5f2003a740 1 client.27100 shutdown 2024-09-10T10:03:32.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.876+0000 7f5f2003a740 1 -- 172.21.15.77:0/1131954175 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f96c3fe50 msgr2=0x564f96c3d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:32.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.876+0000 7f5f2003a740 1 --2- 172.21.15.77:0/1131954175 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f96c3fe50 0x564f96c3d730 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f5f100cd190 tx=0x7f5f1009c6b0 comp rx=0 tx=0).stop 2024-09-10T10:03:32.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.876+0000 7f5f2003a740 1 -- 172.21.15.77:0/1131954175 shutdown_connections 2024-09-10T10:03:32.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.876+0000 7f5f2003a740 1 --2- 172.21.15.77:0/1131954175 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x564f96c4fef0 0x564f96c52360 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:32.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.876+0000 7f5f2003a740 1 --2- 172.21.15.77:0/1131954175 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f96c3fe50 0x564f96c3d730 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:32.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.876+0000 7f5f2003a740 1 --2- 172.21.15.77:0/1131954175 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f96c35590 0x564f96c3d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:32.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.876+0000 7f5f2003a740 1 --2- 172.21.15.77:0/1131954175 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564f96c34c70 0x564f96c3ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:32.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.876+0000 7f5ef7e00640 0 client.27100 ms_handle_reset on v2:172.21.15.134:6834/3293933699 2024-09-10T10:03:32.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.876+0000 7f5f2003a740 1 -- 172.21.15.77:0/1131954175 >> 172.21.15.77:0/1131954175 conn(0x564f96c31200 msgr2=0x564f96c30380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:32.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.877+0000 7f5f2003a740 1 -- 172.21.15.77:0/1131954175 shutdown_connections 2024-09-10T10:03:32.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.877+0000 7f5f2003a740 1 -- 172.21.15.77:0/1131954175 wait complete. 2024-09-10T10:03:32.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.877+0000 7f5f2003a740 20 client.27100 trim_cache size 0 max 16384 2024-09-10T10:03:32.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.891+0000 7f5f2003a740 1 Processor -- start 2024-09-10T10:03:32.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.892+0000 7f5f2003a740 1 -- start start 2024-09-10T10:03:32.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.892+0000 7f5f2003a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f96c34c70 0x564f96c36dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:32.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.892+0000 7f5f2003a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564f96c35590 0x564f96c57c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:32.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.892+0000 7f5f2003a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f96c3fe50 0x564f96c58170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:32.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.892+0000 7f5f2003a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5f181b1080 con 0x564f96c35590 2024-09-10T10:03:32.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.892+0000 7f5f2003a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5f181b2bc0 con 0x564f96c3fe50 2024-09-10T10:03:32.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.892+0000 7f5f2003a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5f181b1750 con 0x564f96c34c70 2024-09-10T10:03:32.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.892+0000 7f5f16000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564f96c35590 0x564f96c57c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:32.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.892+0000 7f5f16000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564f96c35590 0x564f96c57c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58040/0 (socket says 172.21.15.77:58040) 2024-09-10T10:03:32.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.892+0000 7f5f16000640 1 -- 172.21.15.77:0/1644858789 learned_addr learned my addr 172.21.15.77:0/1644858789 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:03:32.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.893+0000 7f5f17400640 1 --2- 172.21.15.77:0/1644858789 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f96c3fe50 0x564f96c58170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:32.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.893+0000 7f5f16000640 1 -- 172.21.15.77:0/1644858789 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f96c34c70 msgr2=0x564f96c36dd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T10:03:32.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.893+0000 7f5f16000640 1 --2- 172.21.15.77:0/1644858789 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f96c34c70 0x564f96c36dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:32.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.893+0000 7f5f16000640 1 -- 172.21.15.77:0/1644858789 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f96c3fe50 msgr2=0x564f96c58170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:32.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.893+0000 7f5f16000640 1 --2- 172.21.15.77:0/1644858789 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f96c3fe50 0x564f96c58170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:32.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.893+0000 7f5f16000640 1 -- 172.21.15.77:0/1644858789 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f96c2c770 con 0x564f96c35590 2024-09-10T10:03:32.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.893+0000 7f5f17400640 1 --2- 172.21.15.77:0/1644858789 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f96c3fe50 0x564f96c58170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:03:32.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.893+0000 7f5f16000640 1 --2- 172.21.15.77:0/1644858789 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564f96c35590 0x564f96c57c30 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f5f04009620 tx=0x7f5f04012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e72f2723f4c247d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:32.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.894+0000 7f5f02a00640 1 -- 172.21.15.77:0/1644858789 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f04031070 con 0x564f96c35590 2024-09-10T10:03:32.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.894+0000 7f5f02a00640 1 -- 172.21.15.77:0/1644858789 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f04010bd0 con 0x564f96c35590 2024-09-10T10:03:32.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.894+0000 7f5f02a00640 1 -- 172.21.15.77:0/1644858789 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f04019040 con 0x564f96c35590 2024-09-10T10:03:32.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.895+0000 7f5f2003a740 1 -- 172.21.15.77:0/1644858789 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564f96c35590 msgr2=0x564f96c57c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:32.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.895+0000 7f5f2003a740 1 --2- 172.21.15.77:0/1644858789 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564f96c35590 0x564f96c57c30 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f5f04009620 tx=0x7f5f04012000 comp rx=0 tx=0).stop 2024-09-10T10:03:32.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.895+0000 7f5f2003a740 1 -- 172.21.15.77:0/1644858789 shutdown_connections 2024-09-10T10:03:32.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.895+0000 7f5f2003a740 1 --2- 172.21.15.77:0/1644858789 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f96c3fe50 0x564f96c58170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:32.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.895+0000 7f5f2003a740 1 --2- 172.21.15.77:0/1644858789 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564f96c35590 0x564f96c57c30 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:32.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.895+0000 7f5f2003a740 1 --2- 172.21.15.77:0/1644858789 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f96c34c70 0x564f96c36dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:32.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.895+0000 7f5f2003a740 1 -- 172.21.15.77:0/1644858789 >> 172.21.15.77:0/1644858789 conn(0x564f96c31200 msgr2=0x564f96c366c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:32.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.895+0000 7f5f2003a740 1 -- 172.21.15.77:0/1644858789 shutdown_connections 2024-09-10T10:03:32.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.895+0000 7f5f2003a740 1 -- 172.21.15.77:0/1644858789 wait complete. 2024-09-10T10:03:32.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.896+0000 7f5f2003a740 1 -- start start 2024-09-10T10:03:32.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.896+0000 7f5f2003a740 1 Processor -- start 2024-09-10T10:03:32.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.896+0000 7f5f2003a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f96c34c70 0x564f96c43d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:32.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.896+0000 7f5f2003a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564f96c35590 0x564f96c442c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:32.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.897+0000 7f5f2003a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f96c3fe50 0x564f96c44800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:32.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.897+0000 7f5f2003a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5f181b24f0 con 0x564f96c35590 2024-09-10T10:03:32.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.897+0000 7f5f2003a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5f181b1e20 con 0x564f96c3fe50 2024-09-10T10:03:32.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.897+0000 7f5f2003a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5f181af060 con 0x564f96c34c70 2024-09-10T10:03:32.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.897+0000 7f5f2003a740 10 client.0 _ll_register_callbacks cb 0x7f5f1009bac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:03:32.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.897+0000 7f5f16a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f96c34c70 0x564f96c43d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:32.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.897+0000 7f5f16a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f96c34c70 0x564f96c43d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:56300/0 (socket says 172.21.15.77:56300) 2024-09-10T10:03:32.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.897+0000 7f5f16a00640 1 -- 172.21.15.77:0/1490450816 learned_addr learned my addr 172.21.15.77:0/1490450816 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:03:32.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.897+0000 7f5f17400640 1 --2- 172.21.15.77:0/1490450816 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f96c3fe50 0x564f96c44800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:32.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.897+0000 7f5f16000640 1 --2- 172.21.15.77:0/1490450816 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564f96c35590 0x564f96c442c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:32.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.897+0000 7f5f16a00640 1 -- 172.21.15.77:0/1490450816 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f96c3fe50 msgr2=0x564f96c44800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:32.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.897+0000 7f5f16a00640 1 --2- 172.21.15.77:0/1490450816 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f96c3fe50 0x564f96c44800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:32.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.897+0000 7f5f16a00640 1 -- 172.21.15.77:0/1490450816 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564f96c35590 msgr2=0x564f96c442c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:32.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.897+0000 7f5f16a00640 1 --2- 172.21.15.77:0/1490450816 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564f96c35590 0x564f96c442c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:32.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.897+0000 7f5f16a00640 1 -- 172.21.15.77:0/1490450816 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f96cb9e50 con 0x564f96c34c70 2024-09-10T10:03:32.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.898+0000 7f5f16000640 1 --2- 172.21.15.77:0/1490450816 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564f96c35590 0x564f96c442c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:03:32.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.898+0000 7f5f16a00640 1 --2- 172.21.15.77:0/1490450816 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f96c34c70 0x564f96c43d80 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f5f08031470 tx=0x7f5f08005420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e20c40be1cafaaed server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:32.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.898+0000 7f5ef7e00640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-10T10:03:32.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.898+0000 7f5ef7e00640 1 -- 172.21.15.77:0/1490450816 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f08040040 con 0x564f96c34c70 2024-09-10T10:03:32.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.898+0000 7f5f2003a740 1 -- 172.21.15.77:0/1490450816 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f04010bd0 con 0x564f96c34c70 2024-09-10T10:03:32.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.898+0000 7f5ef7e00640 1 -- 172.21.15.77:0/1490450816 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f5f08004040 con 0x564f96c34c70 2024-09-10T10:03:32.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.899+0000 7f5ef7e00640 1 -- 172.21.15.77:0/1490450816 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f5f0805d050 con 0x564f96c34c70 2024-09-10T10:03:32.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.899+0000 7f5f2003a740 10 client.27103 fetch_fsmap learned FSMap version 35 2024-09-10T10:03:32.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.899+0000 7f5f2003a740 1 -- 172.21.15.77:0/1490450816 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x564f96cbb080 con 0x564f96c34c70 2024-09-10T10:03:32.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.899+0000 7f5ef7e00640 1 -- 172.21.15.77:0/1490450816 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f04010bd0 con 0x564f96c34c70 2024-09-10T10:03:32.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.899+0000 7f5ef7e00640 1 -- 172.21.15.77:0/1490450816 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f5f08003c90 con 0x564f96c34c70 2024-09-10T10:03:32.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.899+0000 7f5f2003a740 10 client.27103 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T10:03:32.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.899+0000 7f5f2003a740 10 client.27103 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-10T10:03:32.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.899+0000 7f5f2003a740 20 client.27103 populate_metadata read hostname 'smithi077' 2024-09-10T10:03:32.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.899+0000 7f5f2003a740 1 --2- 172.21.15.77:0/1490450816 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x564f96cbb360 0x564f96cbd750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:32.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.899+0000 7f5f2003a740 4 client.27103 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:03:32.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.899+0000 7f5f2003a740 1 -- 172.21.15.77:0/1490450816 --> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564f96ba2e60 con 0x564f96cbb360 2024-09-10T10:03:32.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.900+0000 7f5f16000640 1 --2- 172.21.15.77:0/1490450816 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x564f96cbb360 0x564f96cbd750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:32.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.900+0000 7f5f16000640 1 --2- 172.21.15.77:0/1490450816 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x564f96cbb360 0x564f96cbd750 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2f11fe1350381fb8 server_cookie=950e52e36a90c4b3 in_seq=0 out_seq=0 2024-09-10T10:03:32.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.900+0000 7f5ef7e00640 10 client.27103 ms_handle_connect on v2:172.21.15.134:6834/3293933699 2024-09-10T10:03:32.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.900+0000 7f5ef7e00640 1 -- 172.21.15.77:0/1490450816 <== mds.4 v2:172.21.15.134:6834/3293933699 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564f96ba2e60 con 0x564f96cbb360 2024-09-10T10:03:32.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.900+0000 7f5ef7e00640 10 client.27103 handle_command_reply: tid=0 2024-09-10T10:03:32.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.901+0000 7f5f2003a740 1 client.27103 shutdown 2024-09-10T10:03:32.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.901+0000 7f5f2003a740 1 -- 172.21.15.77:0/1490450816 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f96c34c70 msgr2=0x564f96c43d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:32.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.901+0000 7f5f2003a740 1 --2- 172.21.15.77:0/1490450816 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f96c34c70 0x564f96c43d80 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f5f08031470 tx=0x7f5f08005420 comp rx=0 tx=0).stop 2024-09-10T10:03:32.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.901+0000 7f5f2003a740 1 -- 172.21.15.77:0/1490450816 shutdown_connections 2024-09-10T10:03:32.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.901+0000 7f5f2003a740 1 --2- 172.21.15.77:0/1490450816 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x564f96cbb360 0x564f96cbd750 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:32.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.901+0000 7f5f2003a740 1 --2- 172.21.15.77:0/1490450816 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f96c3fe50 0x564f96c44800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:32.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.901+0000 7f5f2003a740 1 --2- 172.21.15.77:0/1490450816 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564f96c35590 0x564f96c442c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:32.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.901+0000 7f5f2003a740 1 --2- 172.21.15.77:0/1490450816 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f96c34c70 0x564f96c43d80 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:32.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.901+0000 7f5f2003a740 1 -- 172.21.15.77:0/1490450816 >> 172.21.15.77:0/1490450816 conn(0x564f96c31200 msgr2=0x564f96c30380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:32.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.901+0000 7f5ef7e00640 0 client.27103 ms_handle_reset on v2:172.21.15.134:6834/3293933699 2024-09-10T10:03:32.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.901+0000 7f5f2003a740 1 -- 172.21.15.77:0/1490450816 shutdown_connections 2024-09-10T10:03:32.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.901+0000 7f5f2003a740 1 -- 172.21.15.77:0/1490450816 wait complete. 2024-09-10T10:03:32.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.901+0000 7f5f2003a740 20 client.27103 trim_cache size 0 max 16384 2024-09-10T10:03:32.904 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-10T10:03:32.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.902+0000 7f5f1d600640 1 -- 172.21.15.77:0/1949649886 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f5efc0a2b80 msgr2=0x7f5efc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:32.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.902+0000 7f5f1d600640 1 --2- 172.21.15.77:0/1949649886 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f5efc0a2b80 0x7f5efc0a5040 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x564f96c32580 tx=0x7f5f04004730 comp rx=0 tx=0).stop 2024-09-10T10:03:32.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.902+0000 7f5f1d600640 1 -- 172.21.15.77:0/1949649886 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5f1810e3c0 msgr2=0x7f5f1819efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:32.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.902+0000 7f5f1d600640 1 --2- 172.21.15.77:0/1949649886 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5f1810e3c0 0x7f5f1819efe0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f5f10009fd0 tx=0x7f5f1000ef10 comp rx=0 tx=0).stop 2024-09-10T10:03:32.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.903+0000 7f5f1d600640 1 -- 172.21.15.77:0/1949649886 shutdown_connections 2024-09-10T10:03:32.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.903+0000 7f5f1d600640 1 --2- 172.21.15.77:0/1949649886 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f5efc0a2b80 0x7f5efc0a5040 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:32.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.903+0000 7f5f1d600640 1 --2- 172.21.15.77:0/1949649886 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5f1810e3c0 0x7f5f1819efe0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:32.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.903+0000 7f5f1d600640 1 --2- 172.21.15.77:0/1949649886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5f18074f40 0x7f5f1819ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:32.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.903+0000 7f5f1d600640 1 --2- 172.21.15.77:0/1949649886 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5f18074620 0x7f5f1819a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:32.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.903+0000 7f5f1d600640 1 -- 172.21.15.77:0/1949649886 >> 172.21.15.77:0/1949649886 conn(0x7f5f180fd8c0 msgr2=0x7f5f180fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:32.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.903+0000 7f5f1d600640 1 -- 172.21.15.77:0/1949649886 shutdown_connections 2024-09-10T10:03:32.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:32.903+0000 7f5f1d600640 1 -- 172.21.15.77:0/1949649886 wait complete. 2024-09-10T10:03:33.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:03:33 smithi134 ceph-mon[35674]: pgmap v642: 129 pgs: 129 active+clean; 145 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 1.1 MiB/s wr, 455 op/s 2024-09-10T10:03:33.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:03:33 smithi120 ceph-mon[34770]: pgmap v642: 129 pgs: 129 active+clean; 145 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 1.1 MiB/s wr, 455 op/s 2024-09-10T10:03:33.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:03:33 smithi077 ceph-mon[30319]: pgmap v642: 129 pgs: 129 active+clean; 145 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 1.1 MiB/s wr, 455 op/s 2024-09-10T10:03:33.984 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T10:03:34.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:03:34 smithi134 ceph-mon[35674]: pgmap v643: 129 pgs: 129 active+clean; 144 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.1 MiB/s wr, 451 op/s 2024-09-10T10:03:34.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:03:34 smithi120 ceph-mon[34770]: pgmap v643: 129 pgs: 129 active+clean; 144 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.1 MiB/s wr, 451 op/s 2024-09-10T10:03:34.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:03:34 smithi077 ceph-mon[30319]: pgmap v643: 129 pgs: 129 active+clean; 144 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.1 MiB/s wr, 451 op/s 2024-09-10T10:03:34.440 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:03:35.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.058+0000 7f4e55c00640 1 -- 172.21.15.77:0/171171370 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4e50102d30 msgr2=0x7f4e50103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:35.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.058+0000 7f4e55c00640 1 --2- 172.21.15.77:0/171171370 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4e50102d30 0x7f4e50103110 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f4e40009930 tx=0x7f4e4002a440 comp rx=0 tx=0).stop 2024-09-10T10:03:35.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.058+0000 7f4e55c00640 1 -- 172.21.15.77:0/171171370 shutdown_connections 2024-09-10T10:03:35.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.058+0000 7f4e55c00640 1 --2- 172.21.15.77:0/171171370 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4e501082a0 0x7f4e5010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:35.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.058+0000 7f4e55c00640 1 --2- 172.21.15.77:0/171171370 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e501036e0 0x7f4e50107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:35.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.058+0000 7f4e55c00640 1 --2- 172.21.15.77:0/171171370 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4e50102d30 0x7f4e50103110 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:35.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.058+0000 7f4e55c00640 1 -- 172.21.15.77:0/171171370 >> 172.21.15.77:0/171171370 conn(0x7f4e500fc7d0 msgr2=0x7f4e500febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:35.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.058+0000 7f4e55c00640 1 -- 172.21.15.77:0/171171370 shutdown_connections 2024-09-10T10:03:35.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.058+0000 7f4e55c00640 1 -- 172.21.15.77:0/171171370 wait complete. 2024-09-10T10:03:35.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.059+0000 7f4e55c00640 1 Processor -- start 2024-09-10T10:03:35.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.059+0000 7f4e55c00640 1 -- start start 2024-09-10T10:03:35.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.060+0000 7f4e55c00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4e50102d30 0x7f4e501a0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:35.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.060+0000 7f4e55c00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4e501036e0 0x7f4e501a0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:35.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.060+0000 7f4e55c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e501082a0 0x7f4e5019a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:35.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.060+0000 7f4e55c00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4e50112100 con 0x7f4e501036e0 2024-09-10T10:03:35.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.060+0000 7f4e55c00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4e50111f80 con 0x7f4e50102d30 2024-09-10T10:03:35.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.060+0000 7f4e55c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4e50112280 con 0x7f4e501082a0 2024-09-10T10:03:35.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.061+0000 7f4e4ea00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4e50102d30 0x7f4e501a0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:35.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.061+0000 7f4e4ea00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4e50102d30 0x7f4e501a0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:46110/0 (socket says 172.21.15.77:46110) 2024-09-10T10:03:35.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.061+0000 7f4e4ea00640 1 -- 172.21.15.77:0/861975533 learned_addr learned my addr 172.21.15.77:0/861975533 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:03:35.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.062+0000 7f4e4ea00640 1 -- 172.21.15.77:0/861975533 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e501082a0 msgr2=0x7f4e5019a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:35.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.062+0000 7f4e46000640 1 --2- 172.21.15.77:0/861975533 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4e501036e0 0x7f4e501a0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:35.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.062+0000 7f4e4f400640 1 --2- 172.21.15.77:0/861975533 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e501082a0 0x7f4e5019a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:35.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.062+0000 7f4e4ea00640 1 --2- 172.21.15.77:0/861975533 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e501082a0 0x7f4e5019a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:35.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.062+0000 7f4e4ea00640 1 -- 172.21.15.77:0/861975533 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4e501036e0 msgr2=0x7f4e501a0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:35.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.062+0000 7f4e4ea00640 1 --2- 172.21.15.77:0/861975533 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4e501036e0 0x7f4e501a0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:35.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.062+0000 7f4e4ea00640 1 -- 172.21.15.77:0/861975533 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e5019ad20 con 0x7f4e50102d30 2024-09-10T10:03:35.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.063+0000 7f4e46000640 1 --2- 172.21.15.77:0/861975533 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4e501036e0 0x7f4e501a0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:03:35.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.064+0000 7f4e4f400640 1 --2- 172.21.15.77:0/861975533 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e501082a0 0x7f4e5019a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:03:35.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.064+0000 7f4e4ea00640 1 --2- 172.21.15.77:0/861975533 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4e50102d30 0x7f4e501a0430 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f4e4002a960 tx=0x7f4e40003b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:35.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.066+0000 7f4e47e00640 1 -- 172.21.15.77:0/861975533 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e4003f070 con 0x7f4e50102d30 2024-09-10T10:03:35.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.066+0000 7f4e47e00640 1 -- 172.21.15.77:0/861975533 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e40031c90 con 0x7f4e50102d30 2024-09-10T10:03:35.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.066+0000 7f4e47e00640 1 -- 172.21.15.77:0/861975533 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e40035030 con 0x7f4e50102d30 2024-09-10T10:03:35.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.066+0000 7f4e55c00640 1 -- 172.21.15.77:0/861975533 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e5019afb0 con 0x7f4e50102d30 2024-09-10T10:03:35.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.066+0000 7f4e55c00640 1 -- 172.21.15.77:0/861975533 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e5006bc10 con 0x7f4e50102d30 2024-09-10T10:03:35.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.071+0000 7f4e47e00640 1 -- 172.21.15.77:0/861975533 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4e40042070 con 0x7f4e50102d30 2024-09-10T10:03:35.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.072+0000 7f4e58867740 1 Processor -- start 2024-09-10T10:03:35.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.072+0000 7f4e58867740 1 -- start start 2024-09-10T10:03:35.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.072+0000 7f4e58867740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5641a0abbc70 0x5641a0abc050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:35.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.072+0000 7f4e58867740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5641a0abc590 0x5641a0ac6910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:35.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.072+0000 7f4e58867740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5641a0ac6e50 0x5641a0ac9240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:35.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.072+0000 7f4e58867740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5641a0a223e0 con 0x5641a0ac6e50 2024-09-10T10:03:35.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.072+0000 7f4e58867740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5641a09c9d80 con 0x5641a0abbc70 2024-09-10T10:03:35.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.072+0000 7f4e58867740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5641a0a1e3a0 con 0x5641a0abc590 2024-09-10T10:03:35.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.073+0000 7f4e4ea00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5641a0abc590 0x5641a0ac6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:35.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.073+0000 7f4e4ea00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5641a0abc590 0x5641a0ac6910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:56326/0 (socket says 172.21.15.77:56326) 2024-09-10T10:03:35.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.073+0000 7f4e4ea00640 1 -- 172.21.15.77:0/286778087 learned_addr learned my addr 172.21.15.77:0/286778087 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:03:35.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.073+0000 7f4e4f400640 1 --2- 172.21.15.77:0/286778087 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5641a0abbc70 0x5641a0abc050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:35.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.074+0000 7f4e46000640 1 --2- 172.21.15.77:0/286778087 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5641a0ac6e50 0x5641a0ac9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:35.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.074+0000 7f4e4f400640 1 -- 172.21.15.77:0/286778087 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5641a0abc590 msgr2=0x5641a0ac6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:35.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.074+0000 7f4e4f400640 1 --2- 172.21.15.77:0/286778087 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5641a0abc590 0x5641a0ac6910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:35.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.074+0000 7f4e4f400640 1 -- 172.21.15.77:0/286778087 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5641a0ac6e50 msgr2=0x5641a0ac9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:35.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.074+0000 7f4e4f400640 1 --2- 172.21.15.77:0/286778087 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5641a0ac6e50 0x5641a0ac9240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:35.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.074+0000 7f4e4f400640 1 -- 172.21.15.77:0/286778087 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641a09e8ae0 con 0x5641a0abbc70 2024-09-10T10:03:35.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.075+0000 7f4e4f400640 1 --2- 172.21.15.77:0/286778087 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5641a0abbc70 0x5641a0abc050 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f4e48009e40 tx=0x7f4e4800f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93c0d70be3cbb092 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:35.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.075+0000 7f4e47e00640 1 --2- 172.21.15.77:0/861975533 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f4e340a2cd0 0x7f4e340a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:35.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.075+0000 7f4e3fe00640 1 -- 172.21.15.77:0/286778087 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e48003b70 con 0x5641a0abbc70 2024-09-10T10:03:35.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.075+0000 7f4e3fe00640 1 -- 172.21.15.77:0/286778087 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e48003d10 con 0x5641a0abbc70 2024-09-10T10:03:35.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.075+0000 7f4e3fe00640 1 -- 172.21.15.77:0/286778087 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e480119a0 con 0x5641a0abbc70 2024-09-10T10:03:35.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.076+0000 7f4e47e00640 1 -- 172.21.15.77:0/861975533 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f4e400ef6e0 con 0x7f4e50102d30 2024-09-10T10:03:35.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.076+0000 7f4e4ea00640 1 --2- 172.21.15.77:0/861975533 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f4e340a2cd0 0x7f4e340a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:35.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.077+0000 7f4e58867740 1 -- 172.21.15.77:0/286778087 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5641a0abbc70 msgr2=0x5641a0abc050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:35.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.077+0000 7f4e58867740 1 --2- 172.21.15.77:0/286778087 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5641a0abbc70 0x5641a0abc050 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f4e48009e40 tx=0x7f4e4800f730 comp rx=0 tx=0).stop 2024-09-10T10:03:35.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.077+0000 7f4e58867740 1 -- 172.21.15.77:0/286778087 shutdown_connections 2024-09-10T10:03:35.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.077+0000 7f4e58867740 1 --2- 172.21.15.77:0/286778087 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5641a0ac6e50 0x5641a0ac9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:35.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.077+0000 7f4e58867740 1 --2- 172.21.15.77:0/286778087 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5641a0abc590 0x5641a0ac6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:35.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.077+0000 7f4e58867740 1 --2- 172.21.15.77:0/286778087 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5641a0abbc70 0x5641a0abc050 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:35.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.077+0000 7f4e58867740 1 -- 172.21.15.77:0/286778087 >> 172.21.15.77:0/286778087 conn(0x5641a0ab8200 msgr2=0x5641a0ab85e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:35.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.077+0000 7f4e4ea00640 1 --2- 172.21.15.77:0/861975533 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f4e340a2cd0 0x7f4e340a5190 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f4e4002aa00 tx=0x7f4e400f5550 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:35.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.077+0000 7f4e58867740 1 -- 172.21.15.77:0/286778087 shutdown_connections 2024-09-10T10:03:35.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.077+0000 7f4e58867740 1 -- 172.21.15.77:0/286778087 wait complete. 2024-09-10T10:03:35.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.078+0000 7f4e58867740 1 -- start start 2024-09-10T10:03:35.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.078+0000 7f4e58867740 1 Processor -- start 2024-09-10T10:03:35.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.079+0000 7f4e58867740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5641a0abbc70 0x5641a0ac3d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:35.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.079+0000 7f4e58867740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5641a0abc590 0x5641a0ac42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:35.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.079+0000 7f4e58867740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5641a0ac6e50 0x5641a0ac47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:35.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.079+0000 7f4e58867740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5641a0ad2230 con 0x5641a0abc590 2024-09-10T10:03:35.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.079+0000 7f4e58867740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5641a09dd0f0 con 0x5641a0ac6e50 2024-09-10T10:03:35.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.079+0000 7f4e58867740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5641a0ad23b0 con 0x5641a0abbc70 2024-09-10T10:03:35.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.079+0000 7f4e58867740 10 client.0 _ll_register_callbacks cb 0x5641a09c4bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:03:35.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.079+0000 7f4e4f400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5641a0ac6e50 0x5641a0ac47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:35.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.079+0000 7f4e4f400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5641a0ac6e50 0x5641a0ac47e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:46132/0 (socket says 172.21.15.77:46132) 2024-09-10T10:03:35.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.079+0000 7f4e4f400640 1 -- 172.21.15.77:0/531793745 learned_addr learned my addr 172.21.15.77:0/531793745 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:03:35.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.079+0000 7f4e4f400640 1 --2- 172.21.15.77:0/531793745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5641a0abbc70 0x5641a0ac3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:35.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.080+0000 7f4e46000640 1 --2- 172.21.15.77:0/531793745 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5641a0abc590 0x5641a0ac42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:35.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.081+0000 7f4e4f400640 1 -- 172.21.15.77:0/531793745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5641a0abbc70 msgr2=0x5641a0ac3d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:35.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.081+0000 7f4e4f400640 1 --2- 172.21.15.77:0/531793745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5641a0abbc70 0x5641a0ac3d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:35.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.081+0000 7f4e4f400640 1 -- 172.21.15.77:0/531793745 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5641a0abc590 msgr2=0x5641a0ac42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:35.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.081+0000 7f4e4f400640 1 --2- 172.21.15.77:0/531793745 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5641a0abc590 0x5641a0ac42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:35.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.081+0000 7f4e4f400640 1 -- 172.21.15.77:0/531793745 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641a07933c0 con 0x5641a0ac6e50 2024-09-10T10:03:35.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.081+0000 7f4e46000640 1 --2- 172.21.15.77:0/531793745 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5641a0abc590 0x5641a0ac42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:03:35.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.082+0000 7f4e4f400640 1 --2- 172.21.15.77:0/531793745 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5641a0ac6e50 0x5641a0ac47e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f4e48009c50 tx=0x7f4e48011580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84d9e35666530cac server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:35.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.082+0000 7f4e3cc00640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T10:03:35.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.082+0000 7f4e3cc00640 1 -- 172.21.15.77:0/531793745 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e48024040 con 0x5641a0ac6e50 2024-09-10T10:03:35.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.082+0000 7f4e3cc00640 1 -- 172.21.15.77:0/531793745 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f4e480042c0 con 0x5641a0ac6e50 2024-09-10T10:03:35.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.084+0000 7f4e58867740 1 -- 172.21.15.77:0/531793745 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e48003d10 con 0x5641a0ac6e50 2024-09-10T10:03:35.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.084+0000 7f4e3cc00640 1 -- 172.21.15.77:0/531793745 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f4e480044c0 con 0x5641a0ac6e50 2024-09-10T10:03:35.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.084+0000 7f4e58867740 10 client.27161 fetch_fsmap learned FSMap version 35 2024-09-10T10:03:35.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.084+0000 7f4e58867740 1 -- 172.21.15.77:0/531793745 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5641a0ade300 con 0x5641a0ac6e50 2024-09-10T10:03:35.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.085+0000 7f4e3cc00640 1 -- 172.21.15.77:0/531793745 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e48003d10 con 0x5641a0ac6e50 2024-09-10T10:03:35.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.085+0000 7f4e3cc00640 1 -- 172.21.15.77:0/531793745 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f4e4802b4a0 con 0x5641a0ac6e50 2024-09-10T10:03:35.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.085+0000 7f4e58867740 10 client.27161 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T10:03:35.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.085+0000 7f4e58867740 10 client.27161 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T10:03:35.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.085+0000 7f4e58867740 20 client.27161 populate_metadata read hostname 'smithi077' 2024-09-10T10:03:35.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.085+0000 7f4e58867740 1 --2- 172.21.15.77:0/531793745 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x5641a0adf710 0x5641a0ae1b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:35.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.086+0000 7f4e58867740 4 client.27161 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:03:35.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.086+0000 7f4e58867740 1 -- 172.21.15.77:0/531793745 --> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5641a0a2f950 con 0x5641a0adf710 2024-09-10T10:03:35.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.086+0000 7f4e46000640 1 --2- 172.21.15.77:0/531793745 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x5641a0adf710 0x5641a0ae1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:35.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.087+0000 7f4e46000640 1 --2- 172.21.15.77:0/531793745 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x5641a0adf710 0x5641a0ae1b00 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=111346cc4ccd6fc8 server_cookie=58dccf94ae0ea1d2 in_seq=0 out_seq=0 2024-09-10T10:03:35.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.087+0000 7f4e3cc00640 10 client.27161 ms_handle_connect on v2:172.21.15.134:6836/2461497091 2024-09-10T10:03:35.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.088+0000 7f4e3cc00640 1 -- 172.21.15.77:0/531793745 <== mds.3 v2:172.21.15.134:6836/2461497091 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5641a0a2f950 con 0x5641a0adf710 2024-09-10T10:03:35.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.088+0000 7f4e3cc00640 10 client.27161 handle_command_reply: tid=0 2024-09-10T10:03:35.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.088+0000 7f4e58867740 1 client.27161 shutdown 2024-09-10T10:03:35.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.089+0000 7f4e58867740 1 -- 172.21.15.77:0/531793745 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5641a0ac6e50 msgr2=0x5641a0ac47e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:35.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.089+0000 7f4e58867740 1 --2- 172.21.15.77:0/531793745 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5641a0ac6e50 0x5641a0ac47e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f4e48009c50 tx=0x7f4e48011580 comp rx=0 tx=0).stop 2024-09-10T10:03:35.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.089+0000 7f4e58867740 1 -- 172.21.15.77:0/531793745 shutdown_connections 2024-09-10T10:03:35.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.089+0000 7f4e58867740 1 --2- 172.21.15.77:0/531793745 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5641a0ac6e50 0x5641a0ac47e0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:35.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.089+0000 7f4e58867740 1 --2- 172.21.15.77:0/531793745 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x5641a0adf710 0x5641a0ae1b00 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:35.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.089+0000 7f4e58867740 1 --2- 172.21.15.77:0/531793745 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5641a0abc590 0x5641a0ac42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:35.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.089+0000 7f4e58867740 1 --2- 172.21.15.77:0/531793745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5641a0abbc70 0x5641a0ac3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:35.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.089+0000 7f4e58867740 1 -- 172.21.15.77:0/531793745 >> 172.21.15.77:0/531793745 conn(0x5641a0ab8200 msgr2=0x5641a0ab7380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:35.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.089+0000 7f4e3cc00640 0 client.27161 ms_handle_reset on v2:172.21.15.134:6836/2461497091 2024-09-10T10:03:35.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.092+0000 7f4e58867740 1 -- 172.21.15.77:0/531793745 shutdown_connections 2024-09-10T10:03:35.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.092+0000 7f4e58867740 1 -- 172.21.15.77:0/531793745 wait complete. 2024-09-10T10:03:35.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.092+0000 7f4e58867740 20 client.27161 trim_cache size 0 max 16384 2024-09-10T10:03:35.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.113+0000 7f4e58867740 1 Processor -- start 2024-09-10T10:03:35.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.113+0000 7f4e58867740 1 -- start start 2024-09-10T10:03:35.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.114+0000 7f4e58867740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5641a0abbc70 0x5641a0ac2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:35.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.114+0000 7f4e58867740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5641a0abc590 0x5641a0ac4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:35.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.114+0000 7f4e58867740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5641a0ac6e50 0x5641a0ae71f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:35.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.114+0000 7f4e58867740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4e500716d0 con 0x5641a0abc590 2024-09-10T10:03:35.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.114+0000 7f4e58867740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4e50073210 con 0x5641a0ac6e50 2024-09-10T10:03:35.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.114+0000 7f4e58867740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4e50071da0 con 0x5641a0abbc70 2024-09-10T10:03:35.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.114+0000 7f4e46000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5641a0abc590 0x5641a0ac4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:35.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.114+0000 7f4e4f400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5641a0abbc70 0x5641a0ac2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:35.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.114+0000 7f4e4f400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5641a0abbc70 0x5641a0ac2630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:56336/0 (socket says 172.21.15.77:56336) 2024-09-10T10:03:35.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.114+0000 7f4e4f400640 1 -- 172.21.15.77:0/1169734495 learned_addr learned my addr 172.21.15.77:0/1169734495 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:03:35.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.114+0000 7f4e4f400640 1 --2- 172.21.15.77:0/1169734495 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5641a0ac6e50 0x5641a0ae71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:35.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.115+0000 7f4e46000640 1 -- 172.21.15.77:0/1169734495 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5641a0abbc70 msgr2=0x5641a0ac2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:35.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.115+0000 7f4e46000640 1 --2- 172.21.15.77:0/1169734495 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5641a0abbc70 0x5641a0ac2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:35.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.115+0000 7f4e46000640 1 -- 172.21.15.77:0/1169734495 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5641a0ac6e50 msgr2=0x5641a0ae71f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:35.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.115+0000 7f4e46000640 1 --2- 172.21.15.77:0/1169734495 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5641a0ac6e50 0x5641a0ae71f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:35.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.115+0000 7f4e46000640 1 -- 172.21.15.77:0/1169734495 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641a0ab92b0 con 0x5641a0abc590 2024-09-10T10:03:35.117 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.115+0000 7f4e46000640 1 --2- 172.21.15.77:0/1169734495 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5641a0abc590 0x5641a0ac4630 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x5641a0ab9580 tx=0x7f4e38014a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d95f5c8ca1f471a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:35.117 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.115+0000 7f4e3f400640 1 -- 172.21.15.77:0/1169734495 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e38014190 con 0x5641a0abc590 2024-09-10T10:03:35.117 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.115+0000 7f4e3f400640 1 -- 172.21.15.77:0/1169734495 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e38005500 con 0x5641a0abc590 2024-09-10T10:03:35.117 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.116+0000 7f4e3f400640 1 -- 172.21.15.77:0/1169734495 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e38011500 con 0x5641a0abc590 2024-09-10T10:03:35.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.117+0000 7f4e58867740 1 -- 172.21.15.77:0/1169734495 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5641a0abc590 msgr2=0x5641a0ac4630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:35.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.117+0000 7f4e58867740 1 --2- 172.21.15.77:0/1169734495 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5641a0abc590 0x5641a0ac4630 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x5641a0ab9580 tx=0x7f4e38014a10 comp rx=0 tx=0).stop 2024-09-10T10:03:35.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.117+0000 7f4e58867740 1 -- 172.21.15.77:0/1169734495 shutdown_connections 2024-09-10T10:03:35.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.117+0000 7f4e58867740 1 --2- 172.21.15.77:0/1169734495 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5641a0ac6e50 0x5641a0ae71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:35.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.117+0000 7f4e58867740 1 --2- 172.21.15.77:0/1169734495 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5641a0abc590 0x5641a0ac4630 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:35.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.117+0000 7f4e58867740 1 --2- 172.21.15.77:0/1169734495 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5641a0abbc70 0x5641a0ac2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:35.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.117+0000 7f4e58867740 1 -- 172.21.15.77:0/1169734495 >> 172.21.15.77:0/1169734495 conn(0x5641a0ab8200 msgr2=0x5641a0ac5de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:35.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.118+0000 7f4e58867740 1 -- 172.21.15.77:0/1169734495 shutdown_connections 2024-09-10T10:03:35.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.118+0000 7f4e58867740 1 -- 172.21.15.77:0/1169734495 wait complete. 2024-09-10T10:03:35.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.118+0000 7f4e58867740 1 -- start start 2024-09-10T10:03:35.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.119+0000 7f4e58867740 1 Processor -- start 2024-09-10T10:03:35.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.120+0000 7f4e58867740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5641a0abbc70 0x5641a0ad7c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:35.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.120+0000 7f4e58867740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5641a0abc590 0x5641a0ad2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:35.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.120+0000 7f4e58867740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5641a0ac6e50 0x5641a0ad34f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:35.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.120+0000 7f4e58867740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4e50072b40 con 0x5641a0abbc70 2024-09-10T10:03:35.122 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.120+0000 7f4e58867740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4e50072470 con 0x5641a0abc590 2024-09-10T10:03:35.122 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.120+0000 7f4e58867740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4e5006f6b0 con 0x5641a0ac6e50 2024-09-10T10:03:35.122 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.120+0000 7f4e4f400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5641a0abbc70 0x5641a0ad7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:35.122 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.120+0000 7f4e58867740 10 client.0 _ll_register_callbacks cb 0x7f4e48015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:03:35.122 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.120+0000 7f4e4f400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5641a0abbc70 0x5641a0ad7c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58084/0 (socket says 172.21.15.77:58084) 2024-09-10T10:03:35.122 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.120+0000 7f4e4f400640 1 -- 172.21.15.77:0/1477215850 learned_addr learned my addr 172.21.15.77:0/1477215850 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:03:35.122 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.120+0000 7f4e46000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5641a0abc590 0x5641a0ad2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:35.122 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.121+0000 7f4e4f400640 1 --2- 172.21.15.77:0/1477215850 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5641a0ac6e50 0x5641a0ad34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:35.123 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.121+0000 7f4e4f400640 1 -- 172.21.15.77:0/1477215850 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5641a0ac6e50 msgr2=0x5641a0ad34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:35.123 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.121+0000 7f4e4f400640 1 --2- 172.21.15.77:0/1477215850 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5641a0ac6e50 0x5641a0ad34f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:35.123 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.121+0000 7f4e4f400640 1 -- 172.21.15.77:0/1477215850 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5641a0abc590 msgr2=0x5641a0ad2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:35.123 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.121+0000 7f4e4f400640 1 --2- 172.21.15.77:0/1477215850 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5641a0abc590 0x5641a0ad2fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:35.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.121+0000 7f4e4f400640 1 -- 172.21.15.77:0/1477215850 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641a0ab3770 con 0x5641a0abbc70 2024-09-10T10:03:35.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.121+0000 7f4e4f400640 1 --2- 172.21.15.77:0/1477215850 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5641a0abbc70 0x5641a0ad7c80 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f4e48009540 tx=0x7f4e48011580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4fcc19f88274d08 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:35.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.121+0000 7f4e3cc00640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-10T10:03:35.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.122+0000 7f4e3cc00640 1 -- 172.21.15.77:0/1477215850 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e4801f040 con 0x5641a0abbc70 2024-09-10T10:03:35.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.122+0000 7f4e3cc00640 1 -- 172.21.15.77:0/1477215850 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f4e4802b430 con 0x5641a0abbc70 2024-09-10T10:03:35.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.122+0000 7f4e58867740 1 -- 172.21.15.77:0/1477215850 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e38005500 con 0x5641a0abbc70 2024-09-10T10:03:35.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.122+0000 7f4e3cc00640 1 -- 172.21.15.77:0/1477215850 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f4e4802b650 con 0x5641a0abbc70 2024-09-10T10:03:35.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.122+0000 7f4e58867740 10 client.17025 fetch_fsmap learned FSMap version 35 2024-09-10T10:03:35.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.122+0000 7f4e58867740 1 -- 172.21.15.77:0/1477215850 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5641a0b49f10 con 0x5641a0abbc70 2024-09-10T10:03:35.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.122+0000 7f4e3cc00640 1 -- 172.21.15.77:0/1477215850 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e38005500 con 0x5641a0abbc70 2024-09-10T10:03:35.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.123+0000 7f4e3cc00640 1 -- 172.21.15.77:0/1477215850 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f4e48004bd0 con 0x5641a0abbc70 2024-09-10T10:03:35.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.123+0000 7f4e58867740 10 client.17025 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T10:03:35.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.123+0000 7f4e58867740 10 client.17025 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T10:03:35.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.123+0000 7f4e58867740 20 client.17025 populate_metadata read hostname 'smithi077' 2024-09-10T10:03:35.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.123+0000 7f4e58867740 1 --2- 172.21.15.77:0/1477215850 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x5641a0b4a660 0x5641a0b4ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:35.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.123+0000 7f4e58867740 4 client.17025 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:03:35.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.123+0000 7f4e58867740 1 -- 172.21.15.77:0/1477215850 --> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5641a0a29e60 con 0x5641a0b4a660 2024-09-10T10:03:35.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.123+0000 7f4e46000640 1 --2- 172.21.15.77:0/1477215850 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x5641a0b4a660 0x5641a0b4ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:35.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.124+0000 7f4e3cc00640 10 client.17025 ms_handle_connect on v2:172.21.15.134:6836/2461497091 2024-09-10T10:03:35.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.124+0000 7f4e46000640 1 --2- 172.21.15.77:0/1477215850 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x5641a0b4a660 0x5641a0b4ca80 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a3a8ffc449bdefb8 server_cookie=f6bb8a594a059998 in_seq=0 out_seq=0 2024-09-10T10:03:35.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.124+0000 7f4e3cc00640 1 -- 172.21.15.77:0/1477215850 <== mds.3 v2:172.21.15.134:6836/2461497091 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5641a0a29e60 con 0x5641a0b4a660 2024-09-10T10:03:35.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.124+0000 7f4e3cc00640 10 client.17025 handle_command_reply: tid=0 2024-09-10T10:03:35.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.124+0000 7f4e58867740 1 client.17025 shutdown 2024-09-10T10:03:35.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.124+0000 7f4e58867740 1 -- 172.21.15.77:0/1477215850 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5641a0abbc70 msgr2=0x5641a0ad7c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:35.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.124+0000 7f4e58867740 1 --2- 172.21.15.77:0/1477215850 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5641a0abbc70 0x5641a0ad7c80 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f4e48009540 tx=0x7f4e48011580 comp rx=0 tx=0).stop 2024-09-10T10:03:35.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.125+0000 7f4e58867740 1 -- 172.21.15.77:0/1477215850 shutdown_connections 2024-09-10T10:03:35.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.125+0000 7f4e58867740 1 --2- 172.21.15.77:0/1477215850 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5641a0ac6e50 0x5641a0ad34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:35.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.125+0000 7f4e58867740 1 --2- 172.21.15.77:0/1477215850 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5641a0abc590 0x5641a0ad2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:35.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.125+0000 7f4e58867740 1 --2- 172.21.15.77:0/1477215850 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x5641a0b4a660 0x5641a0b4ca80 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:35.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.125+0000 7f4e58867740 1 --2- 172.21.15.77:0/1477215850 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5641a0abbc70 0x5641a0ad7c80 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:35.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.125+0000 7f4e58867740 1 -- 172.21.15.77:0/1477215850 >> 172.21.15.77:0/1477215850 conn(0x5641a0ab8200 msgr2=0x5641a0abd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:35.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.125+0000 7f4e3cc00640 0 client.17025 ms_handle_reset on v2:172.21.15.134:6836/2461497091 2024-09-10T10:03:35.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.125+0000 7f4e58867740 1 -- 172.21.15.77:0/1477215850 shutdown_connections 2024-09-10T10:03:35.128 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.125+0000 7f4e58867740 1 -- 172.21.15.77:0/1477215850 wait complete. 2024-09-10T10:03:35.128 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.125+0000 7f4e58867740 20 client.17025 trim_cache size 0 max 16384 2024-09-10T10:03:35.128 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-10T10:03:35.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.126+0000 7f4e55c00640 1 -- 172.21.15.77:0/861975533 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f4e340a2cd0 msgr2=0x7f4e340a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:35.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.126+0000 7f4e55c00640 1 --2- 172.21.15.77:0/861975533 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f4e340a2cd0 0x7f4e340a5190 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f4e4002aa00 tx=0x7f4e400f5550 comp rx=0 tx=0).stop 2024-09-10T10:03:35.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.126+0000 7f4e55c00640 1 -- 172.21.15.77:0/861975533 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4e50102d30 msgr2=0x7f4e501a0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:35.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.126+0000 7f4e55c00640 1 --2- 172.21.15.77:0/861975533 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4e50102d30 0x7f4e501a0430 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f4e4002a960 tx=0x7f4e40003b20 comp rx=0 tx=0).stop 2024-09-10T10:03:35.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.126+0000 7f4e55c00640 1 -- 172.21.15.77:0/861975533 shutdown_connections 2024-09-10T10:03:35.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.126+0000 7f4e55c00640 1 --2- 172.21.15.77:0/861975533 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f4e340a2cd0 0x7f4e340a5190 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:35.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.126+0000 7f4e55c00640 1 --2- 172.21.15.77:0/861975533 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e501082a0 0x7f4e5019a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:35.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.126+0000 7f4e55c00640 1 --2- 172.21.15.77:0/861975533 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4e501036e0 0x7f4e501a0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:35.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.126+0000 7f4e55c00640 1 --2- 172.21.15.77:0/861975533 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4e50102d30 0x7f4e501a0430 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:35.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.126+0000 7f4e55c00640 1 -- 172.21.15.77:0/861975533 >> 172.21.15.77:0/861975533 conn(0x7f4e500fc7d0 msgr2=0x7f4e501014c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:35.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.127+0000 7f4e55c00640 1 -- 172.21.15.77:0/861975533 shutdown_connections 2024-09-10T10:03:35.130 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:35.127+0000 7f4e55c00640 1 -- 172.21.15.77:0/861975533 wait complete. 2024-09-10T10:03:35.621 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:03:35 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:03:35.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:03:35 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:03:35.673 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:03:35 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:03:36.043 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T10:03:36.524 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:03:36.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:03:36 smithi134 ceph-mon[35674]: pgmap v644: 129 pgs: 129 active+clean; 147 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 1.0 MiB/s wr, 425 op/s 2024-09-10T10:03:36.673 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:03:36 smithi120 ceph-mon[34770]: pgmap v644: 129 pgs: 129 active+clean; 147 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 1.0 MiB/s wr, 425 op/s 2024-09-10T10:03:36.684 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:03:36 smithi077 ceph-mon[30319]: pgmap v644: 129 pgs: 129 active+clean; 147 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 1.0 MiB/s wr, 425 op/s 2024-09-10T10:03:37.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.166+0000 7f885d600640 1 -- 172.21.15.77:0/3677613771 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8858104010 msgr2=0x7f88581043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:37.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.166+0000 7f885d600640 1 --2- 172.21.15.77:0/3677613771 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8858104010 0x7f88581043f0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f8848009900 tx=0x7f884802a3f0 comp rx=0 tx=0).stop 2024-09-10T10:03:37.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.166+0000 7f885d600640 1 -- 172.21.15.77:0/3677613771 shutdown_connections 2024-09-10T10:03:37.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.166+0000 7f885d600640 1 --2- 172.21.15.77:0/3677613771 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8858109140 0x7f8858110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:37.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.166+0000 7f885d600640 1 --2- 172.21.15.77:0/3677613771 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f88581049c0 0x7f8858108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:37.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.166+0000 7f885d600640 1 --2- 172.21.15.77:0/3677613771 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8858104010 0x7f88581043f0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:37.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.166+0000 7f885d600640 1 -- 172.21.15.77:0/3677613771 >> 172.21.15.77:0/3677613771 conn(0x7f88580ffa50 msgr2=0x7f8858101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:37.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.166+0000 7f885d600640 1 -- 172.21.15.77:0/3677613771 shutdown_connections 2024-09-10T10:03:37.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.166+0000 7f885d600640 1 -- 172.21.15.77:0/3677613771 wait complete. 2024-09-10T10:03:37.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.167+0000 7f885d600640 1 Processor -- start 2024-09-10T10:03:37.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.168+0000 7f885d600640 1 -- start start 2024-09-10T10:03:37.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.168+0000 7f885d600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8858104010 0x7f885819a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:37.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.168+0000 7f885d600640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f88581049c0 0x7f885819ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:37.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.168+0000 7f885d600640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8858109140 0x7f885819eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:37.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.168+0000 7f885d600640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8858114140 con 0x7f88581049c0 2024-09-10T10:03:37.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.168+0000 7f885d600640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8858113fc0 con 0x7f8858109140 2024-09-10T10:03:37.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.168+0000 7f885d600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f88581142c0 con 0x7f8858104010 2024-09-10T10:03:37.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.168+0000 7f8856a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8858104010 0x7f885819a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:37.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.168+0000 7f884e000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f88581049c0 0x7f885819ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:37.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.168+0000 7f8856a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8858104010 0x7f885819a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:56364/0 (socket says 172.21.15.77:56364) 2024-09-10T10:03:37.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.168+0000 7f8856a00640 1 -- 172.21.15.77:0/168642277 learned_addr learned my addr 172.21.15.77:0/168642277 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:03:37.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.168+0000 7f8857400640 1 --2- 172.21.15.77:0/168642277 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8858109140 0x7f885819eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:37.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.169+0000 7f8857400640 1 -- 172.21.15.77:0/168642277 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8858104010 msgr2=0x7f885819a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:37.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.169+0000 7f8857400640 1 --2- 172.21.15.77:0/168642277 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8858104010 0x7f885819a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:37.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.169+0000 7f8857400640 1 -- 172.21.15.77:0/168642277 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f88581049c0 msgr2=0x7f885819ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:37.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.169+0000 7f8857400640 1 --2- 172.21.15.77:0/168642277 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f88581049c0 0x7f885819ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:37.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.169+0000 7f8857400640 1 -- 172.21.15.77:0/168642277 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f885819f5b0 con 0x7f8858109140 2024-09-10T10:03:37.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.169+0000 7f8857400640 1 --2- 172.21.15.77:0/168642277 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8858109140 0x7f885819eed0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f8850009fd0 tx=0x7f885000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:37.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.169+0000 7f884fe00640 1 -- 172.21.15.77:0/168642277 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f885000ce20 con 0x7f8858109140 2024-09-10T10:03:37.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.169+0000 7f885d600640 1 -- 172.21.15.77:0/168642277 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f885819f8a0 con 0x7f8858109140 2024-09-10T10:03:37.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.169+0000 7f884fe00640 1 -- 172.21.15.77:0/168642277 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8850005020 con 0x7f8858109140 2024-09-10T10:03:37.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.170+0000 7f884fe00640 1 -- 172.21.15.77:0/168642277 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8850010720 con 0x7f8858109140 2024-09-10T10:03:37.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.170+0000 7f885d600640 1 -- 172.21.15.77:0/168642277 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88581a7180 con 0x7f8858109140 2024-09-10T10:03:37.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.173+0000 7f884fe00640 1 -- 172.21.15.77:0/168642277 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f88500051d0 con 0x7f8858109140 2024-09-10T10:03:37.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.173+0000 7f88601f0740 1 Processor -- start 2024-09-10T10:03:37.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.174+0000 7f88601f0740 1 -- start start 2024-09-10T10:03:37.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.174+0000 7f88601f0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2f07f9c70 0x55a2f07fa050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:37.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.174+0000 7f88601f0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2f07fa590 0x55a2f0804910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:37.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.174+0000 7f88601f0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a2f0804e50 0x55a2f0807240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:37.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.174+0000 7f88601f0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55a2f07603e0 con 0x55a2f0804e50 2024-09-10T10:03:37.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.174+0000 7f88601f0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a2f0707d80 con 0x55a2f07fa590 2024-09-10T10:03:37.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.174+0000 7f88601f0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a2f075c3a0 con 0x55a2f07f9c70 2024-09-10T10:03:37.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.175+0000 7f8856a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2f07fa590 0x55a2f0804910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:37.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.175+0000 7f8856a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2f07fa590 0x55a2f0804910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:46186/0 (socket says 172.21.15.77:46186) 2024-09-10T10:03:37.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.175+0000 7f8856a00640 1 -- 172.21.15.77:0/2615658578 learned_addr learned my addr 172.21.15.77:0/2615658578 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:03:37.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.175+0000 7f884e000640 1 --2- 172.21.15.77:0/2615658578 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2f07f9c70 0x55a2f07fa050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:37.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.175+0000 7f884fe00640 1 --2- 172.21.15.77:0/168642277 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f883c0a2b80 0x7f883c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:37.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.175+0000 7f884fe00640 1 -- 172.21.15.77:0/168642277 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f88500cfb30 con 0x7f8858109140 2024-09-10T10:03:37.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.175+0000 7f884e000640 1 --2- 172.21.15.77:0/2615658578 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a2f0804e50 0x55a2f0807240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:37.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.175+0000 7f8856a00640 1 -- 172.21.15.77:0/2615658578 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2f07f9c70 msgr2=0x55a2f07fa050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:37.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.175+0000 7f8856a00640 1 --2- 172.21.15.77:0/2615658578 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2f07f9c70 0x55a2f07fa050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:37.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.175+0000 7f8856a00640 1 -- 172.21.15.77:0/2615658578 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a2f0804e50 msgr2=0x55a2f0807240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:37.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.175+0000 7f8856a00640 1 --2- 172.21.15.77:0/2615658578 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a2f0804e50 0x55a2f0807240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:37.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.175+0000 7f8856a00640 1 -- 172.21.15.77:0/2615658578 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2f0726ae0 con 0x55a2f07fa590 2024-09-10T10:03:37.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.175+0000 7f8857400640 1 --2- 172.21.15.77:0/168642277 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f883c0a2b80 0x7f883c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:37.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.175+0000 7f884e000640 1 --2- 172.21.15.77:0/2615658578 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a2f0804e50 0x55a2f0807240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:03:37.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.176+0000 7f8856a00640 1 --2- 172.21.15.77:0/2615658578 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2f07fa590 0x55a2f0804910 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f8848009a20 tx=0x7f8848005dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41d759643aff4d0a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:37.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.176+0000 7f8857400640 1 --2- 172.21.15.77:0/168642277 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f883c0a2b80 0x7f883c0a5040 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f885009bed0 tx=0x7f8850005f70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:37.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.176+0000 7f8847400640 1 -- 172.21.15.77:0/2615658578 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8848040070 con 0x55a2f07fa590 2024-09-10T10:03:37.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.176+0000 7f8847400640 1 -- 172.21.15.77:0/2615658578 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8848004030 con 0x55a2f07fa590 2024-09-10T10:03:37.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.176+0000 7f8847400640 1 -- 172.21.15.77:0/2615658578 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8848035040 con 0x55a2f07fa590 2024-09-10T10:03:37.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.177+0000 7f88601f0740 1 -- 172.21.15.77:0/2615658578 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2f07fa590 msgr2=0x55a2f0804910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:37.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.177+0000 7f88601f0740 1 --2- 172.21.15.77:0/2615658578 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2f07fa590 0x55a2f0804910 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f8848009a20 tx=0x7f8848005dd0 comp rx=0 tx=0).stop 2024-09-10T10:03:37.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.177+0000 7f88601f0740 1 -- 172.21.15.77:0/2615658578 shutdown_connections 2024-09-10T10:03:37.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.177+0000 7f88601f0740 1 --2- 172.21.15.77:0/2615658578 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a2f0804e50 0x55a2f0807240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:37.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.177+0000 7f88601f0740 1 --2- 172.21.15.77:0/2615658578 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2f07fa590 0x55a2f0804910 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:37.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.177+0000 7f88601f0740 1 --2- 172.21.15.77:0/2615658578 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2f07f9c70 0x55a2f07fa050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:37.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.177+0000 7f88601f0740 1 -- 172.21.15.77:0/2615658578 >> 172.21.15.77:0/2615658578 conn(0x55a2f07f6200 msgr2=0x55a2f07f65e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:37.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.178+0000 7f88601f0740 1 -- 172.21.15.77:0/2615658578 shutdown_connections 2024-09-10T10:03:37.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.178+0000 7f88601f0740 1 -- 172.21.15.77:0/2615658578 wait complete. 2024-09-10T10:03:37.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.178+0000 7f88601f0740 1 -- start start 2024-09-10T10:03:37.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.178+0000 7f88601f0740 1 Processor -- start 2024-09-10T10:03:37.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.179+0000 7f88601f0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2f07f9c70 0x55a2f0801d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:37.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.179+0000 7f88601f0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2f07fa590 0x55a2f0802270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:37.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.179+0000 7f88601f0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a2f0804e50 0x55a2f08027b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:37.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.179+0000 7f88601f0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55a2f0807a20 con 0x55a2f0804e50 2024-09-10T10:03:37.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.179+0000 7f88601f0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a2f071b0f0 con 0x55a2f07f9c70 2024-09-10T10:03:37.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.179+0000 7f88601f0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a2f0807ba0 con 0x55a2f07fa590 2024-09-10T10:03:37.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.179+0000 7f88601f0740 10 client.0 _ll_register_callbacks cb 0x55a2f0702bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:03:37.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.179+0000 7f884e000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2f07f9c70 0x55a2f0801d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:37.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.179+0000 7f884e000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2f07f9c70 0x55a2f0801d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:46196/0 (socket says 172.21.15.77:46196) 2024-09-10T10:03:37.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.179+0000 7f884e000640 1 -- 172.21.15.77:0/3300789472 learned_addr learned my addr 172.21.15.77:0/3300789472 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:03:37.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.180+0000 7f8856a00640 1 --2- 172.21.15.77:0/3300789472 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2f07fa590 0x55a2f0802270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:37.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.180+0000 7f884e000640 1 --2- 172.21.15.77:0/3300789472 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a2f0804e50 0x55a2f08027b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:37.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.180+0000 7f8856a00640 1 -- 172.21.15.77:0/3300789472 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2f07f9c70 msgr2=0x55a2f0801d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:37.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.180+0000 7f8856a00640 1 --2- 172.21.15.77:0/3300789472 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2f07f9c70 0x55a2f0801d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:37.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.180+0000 7f8856a00640 1 -- 172.21.15.77:0/3300789472 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a2f0804e50 msgr2=0x55a2f08027b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:37.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.180+0000 7f8856a00640 1 --2- 172.21.15.77:0/3300789472 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a2f0804e50 0x55a2f08027b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:37.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.180+0000 7f8856a00640 1 -- 172.21.15.77:0/3300789472 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2f04d13c0 con 0x55a2f07fa590 2024-09-10T10:03:37.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.181+0000 7f8856a00640 1 --2- 172.21.15.77:0/3300789472 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2f07fa590 0x55a2f0802270 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f8848005ab0 tx=0x7f8848009680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cebfc9a7e7a4a720 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:37.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.181+0000 7f883be00640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-10T10:03:37.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.181+0000 7f883be00640 1 -- 172.21.15.77:0/3300789472 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8848040070 con 0x55a2f07fa590 2024-09-10T10:03:37.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.181+0000 7f883be00640 1 -- 172.21.15.77:0/3300789472 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f88480043f0 con 0x55a2f07fa590 2024-09-10T10:03:37.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.182+0000 7f88601f0740 1 -- 172.21.15.77:0/3300789472 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8848004030 con 0x55a2f07fa590 2024-09-10T10:03:37.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.182+0000 7f883be00640 1 -- 172.21.15.77:0/3300789472 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f8848035040 con 0x55a2f07fa590 2024-09-10T10:03:37.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.182+0000 7f88601f0740 10 client.27130 fetch_fsmap learned FSMap version 35 2024-09-10T10:03:37.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.182+0000 7f88601f0740 1 -- 172.21.15.77:0/3300789472 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55a2f0807a20 con 0x55a2f07fa590 2024-09-10T10:03:37.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.182+0000 7f883be00640 1 -- 172.21.15.77:0/3300789472 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8848004030 con 0x55a2f07fa590 2024-09-10T10:03:37.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.182+0000 7f883be00640 1 -- 172.21.15.77:0/3300789472 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f884804c030 con 0x55a2f07fa590 2024-09-10T10:03:37.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.183+0000 7f88601f0740 10 client.27130 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T10:03:37.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.183+0000 7f88601f0740 10 client.27130 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T10:03:37.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.183+0000 7f88601f0740 20 client.27130 populate_metadata read hostname 'smithi077' 2024-09-10T10:03:37.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.183+0000 7f88601f0740 1 --2- 172.21.15.77:0/3300789472 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x55a2f0814b50 0x55a2f0816f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:37.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.183+0000 7f88601f0740 4 client.27130 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:03:37.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.183+0000 7f88601f0740 1 -- 172.21.15.77:0/3300789472 --> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a2f076d950 con 0x55a2f0814b50 2024-09-10T10:03:37.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.183+0000 7f884e000640 1 --2- 172.21.15.77:0/3300789472 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x55a2f0814b50 0x55a2f0816f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:37.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.183+0000 7f884e000640 1 --2- 172.21.15.77:0/3300789472 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x55a2f0814b50 0x55a2f0816f40 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=551b80082c78b9db server_cookie=758ad24683bc9cb1 in_seq=0 out_seq=0 2024-09-10T10:03:37.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.183+0000 7f883be00640 10 client.27130 ms_handle_connect on v2:172.21.15.134:6838/1965117680 2024-09-10T10:03:37.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.184+0000 7f883be00640 1 -- 172.21.15.77:0/3300789472 <== mds.2 v2:172.21.15.134:6838/1965117680 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a2f076d950 con 0x55a2f0814b50 2024-09-10T10:03:37.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.184+0000 7f883be00640 10 client.27130 handle_command_reply: tid=0 2024-09-10T10:03:37.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.184+0000 7f88601f0740 1 client.27130 shutdown 2024-09-10T10:03:37.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.185+0000 7f88601f0740 1 -- 172.21.15.77:0/3300789472 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2f07fa590 msgr2=0x55a2f0802270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:37.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.185+0000 7f88601f0740 1 --2- 172.21.15.77:0/3300789472 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2f07fa590 0x55a2f0802270 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f8848005ab0 tx=0x7f8848009680 comp rx=0 tx=0).stop 2024-09-10T10:03:37.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.185+0000 7f88601f0740 1 -- 172.21.15.77:0/3300789472 shutdown_connections 2024-09-10T10:03:37.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.185+0000 7f88601f0740 1 --2- 172.21.15.77:0/3300789472 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x55a2f0814b50 0x55a2f0816f40 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:37.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.185+0000 7f88601f0740 1 --2- 172.21.15.77:0/3300789472 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a2f0804e50 0x55a2f08027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:37.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.185+0000 7f88601f0740 1 --2- 172.21.15.77:0/3300789472 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2f07fa590 0x55a2f0802270 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:37.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.185+0000 7f88601f0740 1 --2- 172.21.15.77:0/3300789472 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2f07f9c70 0x55a2f0801d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:37.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.185+0000 7f88601f0740 1 -- 172.21.15.77:0/3300789472 >> 172.21.15.77:0/3300789472 conn(0x55a2f07f6200 msgr2=0x55a2f07f5380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:37.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.185+0000 7f883be00640 0 client.27130 ms_handle_reset on v2:172.21.15.134:6838/1965117680 2024-09-10T10:03:37.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.185+0000 7f88601f0740 1 -- 172.21.15.77:0/3300789472 shutdown_connections 2024-09-10T10:03:37.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.185+0000 7f88601f0740 1 -- 172.21.15.77:0/3300789472 wait complete. 2024-09-10T10:03:37.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.185+0000 7f88601f0740 20 client.27130 trim_cache size 0 max 16384 2024-09-10T10:03:37.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.203+0000 7f88601f0740 1 Processor -- start 2024-09-10T10:03:37.206 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.205+0000 7f88601f0740 1 -- start start 2024-09-10T10:03:37.206 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.205+0000 7f88601f0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2f07f9c70 0x55a2f0800600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:37.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.205+0000 7f88601f0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a2f07fa590 0x55a2f0802600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:37.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.205+0000 7f88601f0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2f0804e50 0x55a2f081c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:37.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.205+0000 7f88601f0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f88581b0f80 con 0x55a2f07fa590 2024-09-10T10:03:37.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.205+0000 7f88601f0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f88581b2ac0 con 0x55a2f0804e50 2024-09-10T10:03:37.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.205+0000 7f88601f0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f88581b1650 con 0x55a2f07f9c70 2024-09-10T10:03:37.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.206+0000 7f884e000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2f0804e50 0x55a2f081c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:37.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.206+0000 7f8856a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a2f07fa590 0x55a2f0802600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:37.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.206+0000 7f884e000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2f0804e50 0x55a2f081c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:46198/0 (socket says 172.21.15.77:46198) 2024-09-10T10:03:37.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.206+0000 7f884e000640 1 -- 172.21.15.77:0/427985661 learned_addr learned my addr 172.21.15.77:0/427985661 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:03:37.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.206+0000 7f8856a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a2f07fa590 0x55a2f0802600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58126/0 (socket says 172.21.15.77:58126) 2024-09-10T10:03:37.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.206+0000 7f884e000640 1 --2- 172.21.15.77:0/427985661 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2f07f9c70 0x55a2f0800600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:37.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.207+0000 7f8856a00640 1 -- 172.21.15.77:0/427985661 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2f07f9c70 msgr2=0x55a2f0800600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:37.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.207+0000 7f8856a00640 1 --2- 172.21.15.77:0/427985661 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2f07f9c70 0x55a2f0800600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:37.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.207+0000 7f8856a00640 1 -- 172.21.15.77:0/427985661 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2f0804e50 msgr2=0x55a2f081c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:37.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.207+0000 7f8856a00640 1 --2- 172.21.15.77:0/427985661 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2f0804e50 0x55a2f081c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:37.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.207+0000 7f8856a00640 1 -- 172.21.15.77:0/427985661 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2f0800ce0 con 0x55a2f07fa590 2024-09-10T10:03:37.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.207+0000 7f8856a00640 1 --2- 172.21.15.77:0/427985661 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a2f07fa590 0x55a2f0802600 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f8848005040 tx=0x7f8848005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ae5e7a20bfe90b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:37.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.208+0000 7f8846a00640 1 -- 172.21.15.77:0/427985661 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8848040070 con 0x55a2f07fa590 2024-09-10T10:03:37.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.208+0000 7f8846a00640 1 -- 172.21.15.77:0/427985661 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88480043f0 con 0x55a2f07fa590 2024-09-10T10:03:37.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.208+0000 7f8846a00640 1 -- 172.21.15.77:0/427985661 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88480049a0 con 0x55a2f07fa590 2024-09-10T10:03:37.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.209+0000 7f88601f0740 1 -- 172.21.15.77:0/427985661 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a2f07fa590 msgr2=0x55a2f0802600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:37.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.209+0000 7f88601f0740 1 --2- 172.21.15.77:0/427985661 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a2f07fa590 0x55a2f0802600 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f8848005040 tx=0x7f8848005dd0 comp rx=0 tx=0).stop 2024-09-10T10:03:37.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.210+0000 7f88601f0740 1 -- 172.21.15.77:0/427985661 shutdown_connections 2024-09-10T10:03:37.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.210+0000 7f88601f0740 1 --2- 172.21.15.77:0/427985661 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2f0804e50 0x55a2f081c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:37.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.210+0000 7f88601f0740 1 --2- 172.21.15.77:0/427985661 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a2f07fa590 0x55a2f0802600 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:37.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.210+0000 7f88601f0740 1 --2- 172.21.15.77:0/427985661 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2f07f9c70 0x55a2f0800600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:37.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.210+0000 7f88601f0740 1 -- 172.21.15.77:0/427985661 >> 172.21.15.77:0/427985661 conn(0x55a2f07f6200 msgr2=0x55a2f0803db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:37.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.210+0000 7f88601f0740 1 -- 172.21.15.77:0/427985661 shutdown_connections 2024-09-10T10:03:37.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.210+0000 7f88601f0740 1 -- 172.21.15.77:0/427985661 wait complete. 2024-09-10T10:03:37.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.211+0000 7f88601f0740 1 -- start start 2024-09-10T10:03:37.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.211+0000 7f88601f0740 1 Processor -- start 2024-09-10T10:03:37.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.213+0000 7f88601f0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2f07f9c70 0x55a2f080cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:37.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.213+0000 7f88601f0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2f07fa590 0x55a2f080d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:37.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.213+0000 7f88601f0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a2f0804e50 0x55a2f0878520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:37.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.213+0000 7f88601f0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f88581b23f0 con 0x55a2f0804e50 2024-09-10T10:03:37.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.213+0000 7f88601f0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f88581b1d20 con 0x55a2f07f9c70 2024-09-10T10:03:37.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.213+0000 7f88601f0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f88581aef60 con 0x55a2f07fa590 2024-09-10T10:03:37.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.213+0000 7f88601f0740 10 client.0 _ll_register_callbacks cb 0x7f8848009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:03:37.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.213+0000 7f884e000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2f07f9c70 0x55a2f080cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:37.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.213+0000 7f884e000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2f07f9c70 0x55a2f080cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:46210/0 (socket says 172.21.15.77:46210) 2024-09-10T10:03:37.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.213+0000 7f884e000640 1 -- 172.21.15.77:0/1066725257 learned_addr learned my addr 172.21.15.77:0/1066725257 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:03:37.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.214+0000 7f884e000640 1 --2- 172.21.15.77:0/1066725257 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a2f0804e50 0x55a2f0878520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:37.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.214+0000 7f884e000640 1 -- 172.21.15.77:0/1066725257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2f07fa590 msgr2=0x55a2f080d330 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T10:03:37.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.214+0000 7f884e000640 1 --2- 172.21.15.77:0/1066725257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2f07fa590 0x55a2f080d330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:37.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.214+0000 7f884e000640 1 -- 172.21.15.77:0/1066725257 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a2f0804e50 msgr2=0x55a2f0878520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:37.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.214+0000 7f884e000640 1 --2- 172.21.15.77:0/1066725257 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a2f0804e50 0x55a2f0878520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:37.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.214+0000 7f884e000640 1 -- 172.21.15.77:0/1066725257 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2f087b280 con 0x55a2f07f9c70 2024-09-10T10:03:37.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.215+0000 7f883be00640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T10:03:37.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.215+0000 7f884e000640 1 --2- 172.21.15.77:0/1066725257 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2f07f9c70 0x55a2f080cdf0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f8840005c10 tx=0x7f88400096c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a6ec242d16ccd22 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:37.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.215+0000 7f883be00640 1 -- 172.21.15.77:0/1066725257 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8840010d80 con 0x55a2f07f9c70 2024-09-10T10:03:37.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.215+0000 7f883be00640 1 -- 172.21.15.77:0/1066725257 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f884001a040 con 0x55a2f07f9c70 2024-09-10T10:03:37.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.215+0000 7f88601f0740 1 -- 172.21.15.77:0/1066725257 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88480043f0 con 0x55a2f07f9c70 2024-09-10T10:03:37.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.216+0000 7f883be00640 1 -- 172.21.15.77:0/1066725257 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f8840004020 con 0x55a2f07f9c70 2024-09-10T10:03:37.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.216+0000 7f88601f0740 10 client.27185 fetch_fsmap learned FSMap version 35 2024-09-10T10:03:37.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.216+0000 7f88601f0740 1 -- 172.21.15.77:0/1066725257 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55a2f0888320 con 0x55a2f07f9c70 2024-09-10T10:03:37.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.217+0000 7f883be00640 1 -- 172.21.15.77:0/1066725257 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88480043f0 con 0x55a2f07f9c70 2024-09-10T10:03:37.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.217+0000 7f883be00640 1 -- 172.21.15.77:0/1066725257 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f8840004830 con 0x55a2f07f9c70 2024-09-10T10:03:37.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.218+0000 7f88601f0740 10 client.27185 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T10:03:37.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.218+0000 7f88601f0740 10 client.27185 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T10:03:37.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.218+0000 7f88601f0740 20 client.27185 populate_metadata read hostname 'smithi077' 2024-09-10T10:03:37.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.218+0000 7f88601f0740 1 --2- 172.21.15.77:0/1066725257 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x55a2f0888ab0 0x55a2f088aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:37.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.218+0000 7f88601f0740 4 client.27185 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:03:37.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.218+0000 7f88601f0740 1 -- 172.21.15.77:0/1066725257 --> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a2f0767e60 con 0x55a2f0888ab0 2024-09-10T10:03:37.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.218+0000 7f8856a00640 1 --2- 172.21.15.77:0/1066725257 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x55a2f0888ab0 0x55a2f088aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:37.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.219+0000 7f8856a00640 1 --2- 172.21.15.77:0/1066725257 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x55a2f0888ab0 0x55a2f088aed0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ebe133306c4e619c server_cookie=32f514c9a47bd650 in_seq=0 out_seq=0 2024-09-10T10:03:37.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.219+0000 7f883be00640 10 client.27185 ms_handle_connect on v2:172.21.15.134:6838/1965117680 2024-09-10T10:03:37.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.219+0000 7f883be00640 1 -- 172.21.15.77:0/1066725257 <== mds.2 v2:172.21.15.134:6838/1965117680 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a2f0767e60 con 0x55a2f0888ab0 2024-09-10T10:03:37.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.219+0000 7f883be00640 10 client.27185 handle_command_reply: tid=0 2024-09-10T10:03:37.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.219+0000 7f88601f0740 1 client.27185 shutdown 2024-09-10T10:03:37.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.219+0000 7f88601f0740 1 -- 172.21.15.77:0/1066725257 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2f07f9c70 msgr2=0x55a2f080cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:37.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.219+0000 7f88601f0740 1 --2- 172.21.15.77:0/1066725257 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2f07f9c70 0x55a2f080cdf0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f8840005c10 tx=0x7f88400096c0 comp rx=0 tx=0).stop 2024-09-10T10:03:37.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.220+0000 7f88601f0740 1 -- 172.21.15.77:0/1066725257 shutdown_connections 2024-09-10T10:03:37.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.220+0000 7f88601f0740 1 --2- 172.21.15.77:0/1066725257 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x55a2f0888ab0 0x55a2f088aed0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:37.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.220+0000 7f88601f0740 1 --2- 172.21.15.77:0/1066725257 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a2f0804e50 0x55a2f0878520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:37.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.220+0000 7f88601f0740 1 --2- 172.21.15.77:0/1066725257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2f07fa590 0x55a2f080d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:37.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.220+0000 7f883be00640 0 client.27185 ms_handle_reset on v2:172.21.15.134:6838/1965117680 2024-09-10T10:03:37.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.220+0000 7f88601f0740 1 --2- 172.21.15.77:0/1066725257 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2f07f9c70 0x55a2f080cdf0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:37.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.220+0000 7f88601f0740 1 -- 172.21.15.77:0/1066725257 >> 172.21.15.77:0/1066725257 conn(0x55a2f07f6200 msgr2=0x55a2f07fb6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:37.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.220+0000 7f88601f0740 1 -- 172.21.15.77:0/1066725257 shutdown_connections 2024-09-10T10:03:37.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.220+0000 7f88601f0740 1 -- 172.21.15.77:0/1066725257 wait complete. 2024-09-10T10:03:37.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.220+0000 7f88601f0740 20 client.27185 trim_cache size 0 max 16384 2024-09-10T10:03:37.222 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-10T10:03:37.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.221+0000 7f885d600640 1 -- 172.21.15.77:0/168642277 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f883c0a2b80 msgr2=0x7f883c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:37.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.221+0000 7f885d600640 1 --2- 172.21.15.77:0/168642277 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f883c0a2b80 0x7f883c0a5040 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f885009bed0 tx=0x7f8850005f70 comp rx=0 tx=0).stop 2024-09-10T10:03:37.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.221+0000 7f885d600640 1 -- 172.21.15.77:0/168642277 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8858109140 msgr2=0x7f885819eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:37.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.221+0000 7f885d600640 1 --2- 172.21.15.77:0/168642277 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8858109140 0x7f885819eed0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f8850009fd0 tx=0x7f885000ef10 comp rx=0 tx=0).stop 2024-09-10T10:03:37.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.221+0000 7f885d600640 1 -- 172.21.15.77:0/168642277 shutdown_connections 2024-09-10T10:03:37.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.221+0000 7f885d600640 1 --2- 172.21.15.77:0/168642277 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f883c0a2b80 0x7f883c0a5040 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:37.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.221+0000 7f885d600640 1 --2- 172.21.15.77:0/168642277 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8858109140 0x7f885819eed0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:37.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.221+0000 7f885d600640 1 --2- 172.21.15.77:0/168642277 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f88581049c0 0x7f885819ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:37.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.221+0000 7f885d600640 1 --2- 172.21.15.77:0/168642277 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8858104010 0x7f885819a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:37.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.221+0000 7f885d600640 1 -- 172.21.15.77:0/168642277 >> 172.21.15.77:0/168642277 conn(0x7f88580ffa50 msgr2=0x7f8858100e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:37.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.221+0000 7f885d600640 1 -- 172.21.15.77:0/168642277 shutdown_connections 2024-09-10T10:03:37.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:37.221+0000 7f885d600640 1 -- 172.21.15.77:0/168642277 wait complete. 2024-09-10T10:03:37.464 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 13.4 2024-09-10T10:03:37.464 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:03:37.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:03:37 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:03:37.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:03:37 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:03:37.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:03:37 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T10:03:37.663 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:03:37 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:03:37.663 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:03:37 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:03:37.663 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:03:37 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T10:03:37.673 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:03:37 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:03:37.673 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:03:37 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:03:37.673 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:03:37 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T10:03:38.058 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T10:03:38.205 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:03:38.580 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:03:38 smithi077 ceph-mon[30319]: pgmap v645: 129 pgs: 129 active+clean; 147 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 1.1 MiB/s wr, 435 op/s 2024-09-10T10:03:38.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:03:38 smithi134 ceph-mon[35674]: pgmap v645: 129 pgs: 129 active+clean; 147 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 1.1 MiB/s wr, 435 op/s 2024-09-10T10:03:38.673 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:03:38 smithi120 ceph-mon[34770]: pgmap v645: 129 pgs: 129 active+clean; 147 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 1.1 MiB/s wr, 435 op/s 2024-09-10T10:03:38.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:38.820+0000 7febc1400640 1 -- 172.21.15.77:0/1380852178 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7febbc104010 msgr2=0x7febbc1043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:38.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:38.820+0000 7febc1400640 1 --2- 172.21.15.77:0/1380852178 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7febbc104010 0x7febbc1043f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7febac009860 tx=0x7febac02a340 comp rx=0 tx=0).stop 2024-09-10T10:03:38.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:38.820+0000 7febc1400640 1 -- 172.21.15.77:0/1380852178 shutdown_connections 2024-09-10T10:03:38.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:38.820+0000 7febc1400640 1 --2- 172.21.15.77:0/1380852178 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7febbc109140 0x7febbc110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:38.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:38.820+0000 7febc1400640 1 --2- 172.21.15.77:0/1380852178 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7febbc1049c0 0x7febbc108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:38.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:38.820+0000 7febc1400640 1 --2- 172.21.15.77:0/1380852178 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7febbc104010 0x7febbc1043f0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:38.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:38.820+0000 7febc1400640 1 -- 172.21.15.77:0/1380852178 >> 172.21.15.77:0/1380852178 conn(0x7febbc0ffa50 msgr2=0x7febbc101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:38.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:38.820+0000 7febc1400640 1 -- 172.21.15.77:0/1380852178 shutdown_connections 2024-09-10T10:03:38.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:38.820+0000 7febc1400640 1 -- 172.21.15.77:0/1380852178 wait complete. 2024-09-10T10:03:38.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:38.821+0000 7febc1400640 1 Processor -- start 2024-09-10T10:03:38.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:38.821+0000 7febc1400640 1 -- start start 2024-09-10T10:03:38.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:38.821+0000 7febc1400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7febbc104010 0x7febbc1966f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:38.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:38.821+0000 7febc1400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7febbc1049c0 0x7febbc196c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:38.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:38.821+0000 7febc1400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7febbc109140 0x7febbc19ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:38.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:38.821+0000 7febc1400640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7febbc078fc0 con 0x7febbc1049c0 2024-09-10T10:03:38.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:38.821+0000 7febc1400640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7febbc078e40 con 0x7febbc104010 2024-09-10T10:03:38.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:38.821+0000 7febc1400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7febbc079140 con 0x7febbc109140 2024-09-10T10:03:38.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:38.822+0000 7febb1e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7febbc1049c0 0x7febbc196c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:38.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:38.822+0000 7febbb200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7febbc109140 0x7febbc19ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:38.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:38.822+0000 7febb1e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7febbc1049c0 0x7febbc196c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58160/0 (socket says 172.21.15.77:58160) 2024-09-10T10:03:38.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:38.822+0000 7febb1e00640 1 -- 172.21.15.77:0/2899516808 learned_addr learned my addr 172.21.15.77:0/2899516808 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:03:38.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:38.822+0000 7febbb200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7febbc109140 0x7febbc19ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:56430/0 (socket says 172.21.15.77:56430) 2024-09-10T10:03:38.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:38.822+0000 7febba800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7febbc104010 0x7febbc1966f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:38.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:38.822+0000 7febbb200640 1 -- 172.21.15.77:0/2899516808 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7febbc104010 msgr2=0x7febbc1966f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:38.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:38.822+0000 7febbb200640 1 --2- 172.21.15.77:0/2899516808 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7febbc104010 0x7febbc1966f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:38.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:38.822+0000 7febbb200640 1 -- 172.21.15.77:0/2899516808 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7febbc1049c0 msgr2=0x7febbc196c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:38.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:38.822+0000 7febbb200640 1 --2- 172.21.15.77:0/2899516808 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7febbc1049c0 0x7febbc196c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:38.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:38.822+0000 7febbb200640 1 -- 172.21.15.77:0/2899516808 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febbc19b1f0 con 0x7febbc109140 2024-09-10T10:03:38.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:38.822+0000 7febb1e00640 1 --2- 172.21.15.77:0/2899516808 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7febbc1049c0 0x7febbc196c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:03:38.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:38.823+0000 7febbb200640 1 --2- 172.21.15.77:0/2899516808 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7febbc109140 0x7febbc19ab10 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7febb4009fd0 tx=0x7febb400ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:38.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:38.823+0000 7febb3e00640 1 -- 172.21.15.77:0/2899516808 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb400ce20 con 0x7febbc109140 2024-09-10T10:03:38.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:38.823+0000 7febb3e00640 1 -- 172.21.15.77:0/2899516808 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7febb4005020 con 0x7febbc109140 2024-09-10T10:03:38.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:38.823+0000 7febb3e00640 1 -- 172.21.15.77:0/2899516808 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb4010720 con 0x7febbc109140 2024-09-10T10:03:38.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:38.823+0000 7febc1400640 1 -- 172.21.15.77:0/2899516808 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febbc19b4e0 con 0x7febbc109140 2024-09-10T10:03:38.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:38.823+0000 7febc1400640 1 -- 172.21.15.77:0/2899516808 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febbc1a2dc0 con 0x7febbc109140 2024-09-10T10:03:38.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:38.824+0000 7febc1400640 1 -- 172.21.15.77:0/2899516808 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febbc1043f0 con 0x7febbc109140 2024-09-10T10:03:38.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:38.827+0000 7febb3e00640 1 -- 172.21.15.77:0/2899516808 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7febb4005540 con 0x7febbc109140 2024-09-10T10:03:38.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:38.829+0000 7febb3e00640 1 --2- 172.21.15.77:0/2899516808 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7feba00a2c00 0x7feba00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:38.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:38.829+0000 7febb3e00640 1 -- 172.21.15.77:0/2899516808 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7febb40d00e0 con 0x7febbc109140 2024-09-10T10:03:38.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:38.830+0000 7febba800640 1 --2- 172.21.15.77:0/2899516808 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7feba00a2c00 0x7feba00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:38.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:38.830+0000 7febb3e00640 1 -- 172.21.15.77:0/2899516808 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7febb401d060 con 0x7febbc109140 2024-09-10T10:03:38.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:38.830+0000 7febba800640 1 --2- 172.21.15.77:0/2899516808 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7feba00a2c00 0x7feba00a50c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7febac02a930 tx=0x7febac0056d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:38.867 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:03:39.122 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.121+0000 7febc1400640 1 -- 172.21.15.77:0/2899516808 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7febbc1a30c0 con 0x7febbc109140 2024-09-10T10:03:39.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.122+0000 7febb3e00640 1 -- 172.21.15.77:0/2899516808 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11989 (secure 0 0 0) 0x7febb409cbf0 con 0x7febbc109140 2024-09-10T10:03:39.124 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T10:03:39.124 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":35,"btime":"2024-09-10T10:01:05:922537+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3159289675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3159289675},{"type":"v1","addr":"172.21.15.77:6835","nonce":3159289675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1951962341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1951962341},{"type":"v1","addr":"172.21.15.77:6837","nonce":1951962341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":35,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:52:23.368252+0000","modified":"2024-09-10T10:01:05.706646+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24563,"mds_1":24524,"mds_2":24520,"mds_3":24502,"mds_4":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14487":{"gid":14487,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/4291121888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":4291121888},{"type":"v1","addr":"172.21.15.77:6839","nonce":4291121888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14499":{"gid":14499,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6841/445284194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":445284194},{"type":"v1","addr":"172.21.15.77:6841","nonce":445284194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1005054423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1005054423},{"type":"v1","addr":"172.21.15.134:6833","nonce":1005054423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6833/3716307744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":3716307744},{"type":"v1","addr":"172.21.15.120:6833","nonce":3716307744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/1319892544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1319892544},{"type":"v1","addr":"172.21.15.120:6835","nonce":1319892544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.134:6835/3293933699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3293933699},{"type":"v1","addr":"172.21.15.134:6835","nonce":3293933699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.134:6837/2461497091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2461497091},{"type":"v1","addr":"172.21.15.134:6837","nonce":2461497091}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":20,"addr":"172.21.15.134:6839/1965117680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1965117680},{"type":"v1","addr":"172.21.15.134:6839","nonce":1965117680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":20,"addr":"172.21.15.120:6837/3033280709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3033280709},{"type":"v1","addr":"172.21.15.120:6837","nonce":3033280709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.120:6839/1777710157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1777710157},{"type":"v1","addr":"172.21.15.120:6839","nonce":1777710157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24563,"qdb_cluster":[24496,24502,24520,24524,24563]},"id":1}]} 2024-09-10T10:03:39.124 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 35 2024-09-10T10:03:39.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.124+0000 7febc1400640 1 -- 172.21.15.77:0/2899516808 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7feba00a2c00 msgr2=0x7feba00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:39.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.124+0000 7febc1400640 1 --2- 172.21.15.77:0/2899516808 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7feba00a2c00 0x7feba00a50c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7febac02a930 tx=0x7febac0056d0 comp rx=0 tx=0).stop 2024-09-10T10:03:39.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.124+0000 7febc1400640 1 -- 172.21.15.77:0/2899516808 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7febbc109140 msgr2=0x7febbc19ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:39.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.124+0000 7febc1400640 1 --2- 172.21.15.77:0/2899516808 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7febbc109140 0x7febbc19ab10 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7febb4009fd0 tx=0x7febb400ef10 comp rx=0 tx=0).stop 2024-09-10T10:03:39.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.124+0000 7febc1400640 1 -- 172.21.15.77:0/2899516808 shutdown_connections 2024-09-10T10:03:39.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.124+0000 7febc1400640 1 --2- 172.21.15.77:0/2899516808 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7feba00a2c00 0x7feba00a50c0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:39.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.124+0000 7febc1400640 1 --2- 172.21.15.77:0/2899516808 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7febbc109140 0x7febbc19ab10 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:39.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.124+0000 7febc1400640 1 --2- 172.21.15.77:0/2899516808 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7febbc1049c0 0x7febbc196c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:39.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.124+0000 7febc1400640 1 --2- 172.21.15.77:0/2899516808 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7febbc104010 0x7febbc1966f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:39.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.124+0000 7febc1400640 1 -- 172.21.15.77:0/2899516808 >> 172.21.15.77:0/2899516808 conn(0x7febbc0ffa50 msgr2=0x7febbc1079f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:39.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.124+0000 7febc1400640 1 -- 172.21.15.77:0/2899516808 shutdown_connections 2024-09-10T10:03:39.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.124+0000 7febc1400640 1 -- 172.21.15.77:0/2899516808 wait complete. 2024-09-10T10:03:39.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:03:39 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/2899516808' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:03:39.660 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:03:39 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/2899516808' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:03:39.674 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:03:39 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/2899516808' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:03:39.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.691+0000 7f160a200640 1 -- 172.21.15.77:0/2988801191 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1604102790 msgr2=0x7f16041067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:39.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.691+0000 7f160a200640 1 --2- 172.21.15.77:0/2988801191 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1604102790 0x7f16041067e0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f15f0009900 tx=0x7f15f002a410 comp rx=0 tx=0).stop 2024-09-10T10:03:39.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.691+0000 7f160a200640 1 -- 172.21.15.77:0/2988801191 shutdown_connections 2024-09-10T10:03:39.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.691+0000 7f160a200640 1 --2- 172.21.15.77:0/2988801191 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1604106f10 0x7f160410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:39.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.691+0000 7f160a200640 1 --2- 172.21.15.77:0/2988801191 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1604102790 0x7f16041067e0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:39.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.691+0000 7f160a200640 1 --2- 172.21.15.77:0/2988801191 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1604101de0 0x7f16041021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:39.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.691+0000 7f160a200640 1 -- 172.21.15.77:0/2988801191 >> 172.21.15.77:0/2988801191 conn(0x7f16040fd8c0 msgr2=0x7f16040ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:39.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.691+0000 7f160a200640 1 -- 172.21.15.77:0/2988801191 shutdown_connections 2024-09-10T10:03:39.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.691+0000 7f160a200640 1 -- 172.21.15.77:0/2988801191 wait complete. 2024-09-10T10:03:39.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.692+0000 7f160a200640 1 Processor -- start 2024-09-10T10:03:39.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.692+0000 7f160a200640 1 -- start start 2024-09-10T10:03:39.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.693+0000 7f160a200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1604101de0 0x7f16040744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:39.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.693+0000 7f160a200640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1604102790 0x7f1604074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:39.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.693+0000 7f160a200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1604106f10 0x7f1604074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:39.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.693+0000 7f160a200640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1604111fc0 con 0x7f1604106f10 2024-09-10T10:03:39.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.693+0000 7f160a200640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1604111e40 con 0x7f1604102790 2024-09-10T10:03:39.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.693+0000 7f160a200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1604112140 con 0x7f1604101de0 2024-09-10T10:03:39.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.693+0000 7f1603400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1604101de0 0x7f16040744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:39.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.693+0000 7f1603400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1604101de0 0x7f16040744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:56452/0 (socket says 172.21.15.77:56452) 2024-09-10T10:03:39.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.693+0000 7f1603400640 1 -- 172.21.15.77:0/3416207332 learned_addr learned my addr 172.21.15.77:0/3416207332 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:03:39.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.693+0000 7f1603e00640 1 --2- 172.21.15.77:0/3416207332 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1604106f10 0x7f1604074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:39.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.693+0000 7f1602a00640 1 --2- 172.21.15.77:0/3416207332 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1604102790 0x7f1604074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:39.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.694+0000 7f1603400640 1 -- 172.21.15.77:0/3416207332 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1604102790 msgr2=0x7f1604074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:39.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.694+0000 7f1603400640 1 --2- 172.21.15.77:0/3416207332 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1604102790 0x7f1604074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:39.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.694+0000 7f1603400640 1 -- 172.21.15.77:0/3416207332 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1604106f10 msgr2=0x7f1604074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:39.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.694+0000 7f1603400640 1 --2- 172.21.15.77:0/3416207332 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1604106f10 0x7f1604074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:39.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.694+0000 7f1603400640 1 -- 172.21.15.77:0/3416207332 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16041a6990 con 0x7f1604101de0 2024-09-10T10:03:39.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.694+0000 7f1603400640 1 --2- 172.21.15.77:0/3416207332 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1604101de0 0x7f16040744f0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f15f400e9f0 tx=0x7f15f400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:39.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.694+0000 7f15fbe00640 1 -- 172.21.15.77:0/3416207332 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f400cd80 con 0x7f1604101de0 2024-09-10T10:03:39.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.694+0000 7f15fbe00640 1 -- 172.21.15.77:0/3416207332 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15f4010910 con 0x7f1604101de0 2024-09-10T10:03:39.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.694+0000 7f160a200640 1 -- 172.21.15.77:0/3416207332 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16041a6c80 con 0x7f1604101de0 2024-09-10T10:03:39.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.694+0000 7f15fbe00640 1 -- 172.21.15.77:0/3416207332 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f4018990 con 0x7f1604101de0 2024-09-10T10:03:39.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.694+0000 7f160a200640 1 -- 172.21.15.77:0/3416207332 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16041a7190 con 0x7f1604101de0 2024-09-10T10:03:39.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.699+0000 7f160cc1e740 1 Processor -- start 2024-09-10T10:03:39.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.699+0000 7f160cc1e740 1 -- start start 2024-09-10T10:03:39.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.699+0000 7f15fbe00640 1 -- 172.21.15.77:0/3416207332 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f15f4010ac0 con 0x7f1604101de0 2024-09-10T10:03:39.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.699+0000 7f160cc1e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5575c5b33c70 0x5575c5b34050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:39.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.699+0000 7f160cc1e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5575c5b34590 0x5575c5b3e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:39.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.699+0000 7f160cc1e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5575c5b3ee50 0x5575c5b41240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:39.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.699+0000 7f160cc1e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5575c5a9a3e0 con 0x5575c5b3ee50 2024-09-10T10:03:39.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.699+0000 7f160cc1e740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5575c5a41d80 con 0x5575c5b33c70 2024-09-10T10:03:39.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.699+0000 7f160cc1e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5575c5a963a0 con 0x5575c5b34590 2024-09-10T10:03:39.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.700+0000 7f1603e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5575c5b33c70 0x5575c5b34050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:39.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.700+0000 7f1603e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5575c5b33c70 0x5575c5b34050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:46242/0 (socket says 172.21.15.77:46242) 2024-09-10T10:03:39.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.700+0000 7f1603e00640 1 -- 172.21.15.77:0/790932550 learned_addr learned my addr 172.21.15.77:0/790932550 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:03:39.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.700+0000 7f1602a00640 1 --2- 172.21.15.77:0/790932550 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5575c5b3ee50 0x5575c5b41240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:39.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.700+0000 7f1603e00640 1 -- 172.21.15.77:0/790932550 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5575c5b34590 msgr2=0x5575c5b3e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:39.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.700+0000 7f1603e00640 1 --2- 172.21.15.77:0/790932550 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5575c5b34590 0x5575c5b3e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:39.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.700+0000 7f1603e00640 1 -- 172.21.15.77:0/790932550 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5575c5b3ee50 msgr2=0x5575c5b41240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:39.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.700+0000 7f1603e00640 1 --2- 172.21.15.77:0/790932550 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5575c5b3ee50 0x5575c5b41240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:39.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.700+0000 7f1603e00640 1 -- 172.21.15.77:0/790932550 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575c5a60ae0 con 0x5575c5b33c70 2024-09-10T10:03:39.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.700+0000 7f1602a00640 1 --2- 172.21.15.77:0/790932550 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5575c5b3ee50 0x5575c5b41240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:03:39.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.700+0000 7f1603e00640 1 --2- 172.21.15.77:0/790932550 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5575c5b33c70 0x5575c5b34050 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f15fc009f50 tx=0x7f15fc0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9101c02fd011102 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:39.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.701+0000 7f15efe00640 1 -- 172.21.15.77:0/790932550 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15fc004b20 con 0x5575c5b33c70 2024-09-10T10:03:39.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.701+0000 7f15efe00640 1 -- 172.21.15.77:0/790932550 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15fc004cc0 con 0x5575c5b33c70 2024-09-10T10:03:39.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.701+0000 7f15efe00640 1 -- 172.21.15.77:0/790932550 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15fc005020 con 0x5575c5b33c70 2024-09-10T10:03:39.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.701+0000 7f15fbe00640 1 --2- 172.21.15.77:0/3416207332 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f15e80a2c00 0x7f15e80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:39.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.701+0000 7f15fbe00640 1 -- 172.21.15.77:0/3416207332 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f15f40cf6a0 con 0x7f1604101de0 2024-09-10T10:03:39.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.701+0000 7f1603400640 1 --2- 172.21.15.77:0/3416207332 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f15e80a2c00 0x7f15e80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:39.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.702+0000 7f160cc1e740 1 -- 172.21.15.77:0/790932550 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5575c5b33c70 msgr2=0x5575c5b34050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:39.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.702+0000 7f160cc1e740 1 --2- 172.21.15.77:0/790932550 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5575c5b33c70 0x5575c5b34050 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f15fc009f50 tx=0x7f15fc0118d0 comp rx=0 tx=0).stop 2024-09-10T10:03:39.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.702+0000 7f160cc1e740 1 -- 172.21.15.77:0/790932550 shutdown_connections 2024-09-10T10:03:39.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.702+0000 7f160cc1e740 1 --2- 172.21.15.77:0/790932550 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5575c5b3ee50 0x5575c5b41240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:39.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.702+0000 7f160cc1e740 1 --2- 172.21.15.77:0/790932550 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5575c5b34590 0x5575c5b3e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:39.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.702+0000 7f160cc1e740 1 --2- 172.21.15.77:0/790932550 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5575c5b33c70 0x5575c5b34050 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:39.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.702+0000 7f160cc1e740 1 -- 172.21.15.77:0/790932550 >> 172.21.15.77:0/790932550 conn(0x5575c5b30200 msgr2=0x5575c5b305e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:39.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.702+0000 7f160cc1e740 1 -- 172.21.15.77:0/790932550 shutdown_connections 2024-09-10T10:03:39.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.702+0000 7f160cc1e740 1 -- 172.21.15.77:0/790932550 wait complete. 2024-09-10T10:03:39.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.702+0000 7f1603400640 1 --2- 172.21.15.77:0/3416207332 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f15e80a2c00 0x7f15e80a50c0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f15f409b8c0 tx=0x7f15f40977c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:39.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.703+0000 7f160cc1e740 1 -- start start 2024-09-10T10:03:39.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.703+0000 7f160cc1e740 1 Processor -- start 2024-09-10T10:03:39.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.703+0000 7f160cc1e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5575c5b33c70 0x5575c5b3bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:39.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.704+0000 7f160cc1e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5575c5b34590 0x5575c5b3c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:39.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.704+0000 7f160cc1e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5575c5b3ee50 0x5575c5b3c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:39.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.704+0000 7f160cc1e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5575c5b41a70 con 0x5575c5b34590 2024-09-10T10:03:39.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.704+0000 7f160cc1e740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5575c5a550f0 con 0x5575c5b33c70 2024-09-10T10:03:39.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.704+0000 7f160cc1e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5575c5b41bf0 con 0x5575c5b3ee50 2024-09-10T10:03:39.708 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.704+0000 7f160cc1e740 10 client.0 _ll_register_callbacks cb 0x5575c5a3cbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:03:39.708 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.704+0000 7f1603e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5575c5b33c70 0x5575c5b3bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:39.708 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.704+0000 7f1603e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5575c5b33c70 0x5575c5b3bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:46254/0 (socket says 172.21.15.77:46254) 2024-09-10T10:03:39.708 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.704+0000 7f1603e00640 1 -- 172.21.15.77:0/3219641966 learned_addr learned my addr 172.21.15.77:0/3219641966 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:03:39.708 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.704+0000 7f1602a00640 1 --2- 172.21.15.77:0/3219641966 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5575c5b34590 0x5575c5b3c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:39.708 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.704+0000 7f1603e00640 1 --2- 172.21.15.77:0/3219641966 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5575c5b3ee50 0x5575c5b3c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:39.708 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.704+0000 7f1602a00640 1 -- 172.21.15.77:0/3219641966 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5575c5b3ee50 msgr2=0x5575c5b3c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:39.708 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.704+0000 7f1602a00640 1 --2- 172.21.15.77:0/3219641966 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5575c5b3ee50 0x5575c5b3c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:39.708 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.704+0000 7f1602a00640 1 -- 172.21.15.77:0/3219641966 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5575c5b33c70 msgr2=0x5575c5b3bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:39.708 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.704+0000 7f1602a00640 1 --2- 172.21.15.77:0/3219641966 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5575c5b33c70 0x5575c5b3bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:39.708 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.705+0000 7f1602a00640 1 -- 172.21.15.77:0/3219641966 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575c580b3c0 con 0x5575c5b34590 2024-09-10T10:03:39.708 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.705+0000 7f1603e00640 1 --2- 172.21.15.77:0/3219641966 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5575c5b3ee50 0x5575c5b3c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:03:39.708 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.705+0000 7f1602a00640 1 --2- 172.21.15.77:0/3219641966 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5575c5b34590 0x5575c5b3c210 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f15f002a930 tx=0x7f15f0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6d10da0f26aaff7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:39.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.705+0000 7f15ecc00640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-10T10:03:39.710 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.706+0000 7f15ecc00640 1 -- 172.21.15.77:0/3219641966 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f0035040 con 0x5575c5b34590 2024-09-10T10:03:39.710 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.706+0000 7f15ecc00640 1 -- 172.21.15.77:0/3219641966 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f15f0005230 con 0x5575c5b34590 2024-09-10T10:03:39.710 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.706+0000 7f160cc1e740 1 -- 172.21.15.77:0/3219641966 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15fc004cc0 con 0x5575c5b34590 2024-09-10T10:03:39.710 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.706+0000 7f15ecc00640 1 -- 172.21.15.77:0/3219641966 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f15f0005430 con 0x5575c5b34590 2024-09-10T10:03:39.710 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.706+0000 7f160cc1e740 10 client.17058 fetch_fsmap learned FSMap version 35 2024-09-10T10:03:39.710 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.706+0000 7f160cc1e740 1 -- 172.21.15.77:0/3219641966 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5575c5b56250 con 0x5575c5b34590 2024-09-10T10:03:39.710 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.706+0000 7f15ecc00640 1 -- 172.21.15.77:0/3219641966 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15fc004cc0 con 0x5575c5b34590 2024-09-10T10:03:39.710 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.707+0000 7f15ecc00640 1 -- 172.21.15.77:0/3219641966 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f15f0031b20 con 0x5575c5b34590 2024-09-10T10:03:39.710 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.707+0000 7f160cc1e740 10 client.17058 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T10:03:39.710 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.707+0000 7f160cc1e740 10 client.17058 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-10T10:03:39.710 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.707+0000 7f160cc1e740 20 client.17058 populate_metadata read hostname 'smithi077' 2024-09-10T10:03:39.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.707+0000 7f160cc1e740 1 --2- 172.21.15.77:0/3219641966 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x5575c5b57710 0x5575c5b59b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:39.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.707+0000 7f160cc1e740 4 client.17058 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:03:39.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.707+0000 7f160cc1e740 1 -- 172.21.15.77:0/3219641966 --> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5575c5aa7950 con 0x5575c5b57710 2024-09-10T10:03:39.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.707+0000 7f1603e00640 1 --2- 172.21.15.77:0/3219641966 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x5575c5b57710 0x5575c5b59b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:39.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.708+0000 7f15ecc00640 10 client.17058 ms_handle_connect on v2:172.21.15.120:6836/3033280709 2024-09-10T10:03:39.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.708+0000 7f1603e00640 1 --2- 172.21.15.77:0/3219641966 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x5575c5b57710 0x5575c5b59b00 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3f38b1f34973cf09 server_cookie=a639c2669b2f5cd3 in_seq=0 out_seq=0 2024-09-10T10:03:39.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.708+0000 7f15ecc00640 1 -- 172.21.15.77:0/3219641966 <== mds.1 v2:172.21.15.120:6836/3033280709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5575c5aa7950 con 0x5575c5b57710 2024-09-10T10:03:39.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.708+0000 7f15ecc00640 10 client.17058 handle_command_reply: tid=0 2024-09-10T10:03:39.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.709+0000 7f160cc1e740 1 client.17058 shutdown 2024-09-10T10:03:39.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.709+0000 7f160cc1e740 1 -- 172.21.15.77:0/3219641966 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5575c5b34590 msgr2=0x5575c5b3c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:39.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.709+0000 7f160cc1e740 1 --2- 172.21.15.77:0/3219641966 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5575c5b34590 0x5575c5b3c210 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f15f002a930 tx=0x7f15f0004270 comp rx=0 tx=0).stop 2024-09-10T10:03:39.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.709+0000 7f160cc1e740 1 -- 172.21.15.77:0/3219641966 shutdown_connections 2024-09-10T10:03:39.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.709+0000 7f160cc1e740 1 --2- 172.21.15.77:0/3219641966 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x5575c5b57710 0x5575c5b59b00 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:39.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.709+0000 7f15ecc00640 0 client.17058 ms_handle_reset on v2:172.21.15.120:6836/3033280709 2024-09-10T10:03:39.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.709+0000 7f160cc1e740 1 --2- 172.21.15.77:0/3219641966 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5575c5b3ee50 0x5575c5b3c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:39.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.709+0000 7f160cc1e740 1 --2- 172.21.15.77:0/3219641966 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5575c5b34590 0x5575c5b3c210 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:39.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.709+0000 7f160cc1e740 1 --2- 172.21.15.77:0/3219641966 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5575c5b33c70 0x5575c5b3bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:39.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.709+0000 7f160cc1e740 1 -- 172.21.15.77:0/3219641966 >> 172.21.15.77:0/3219641966 conn(0x5575c5b30200 msgr2=0x5575c5b35ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:39.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.709+0000 7f160cc1e740 1 -- 172.21.15.77:0/3219641966 shutdown_connections 2024-09-10T10:03:39.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.710+0000 7f160cc1e740 1 -- 172.21.15.77:0/3219641966 wait complete. 2024-09-10T10:03:39.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.710+0000 7f160cc1e740 20 client.17058 trim_cache size 0 max 16384 2024-09-10T10:03:39.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.728+0000 7f160cc1e740 1 Processor -- start 2024-09-10T10:03:39.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.728+0000 7f160cc1e740 1 -- start start 2024-09-10T10:03:39.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.728+0000 7f160cc1e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5575c5b33c70 0x5575c5b366d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:39.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.728+0000 7f160cc1e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5575c5b34590 0x5575c5b3dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:39.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.728+0000 7f160cc1e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5575c5b3ee50 0x5575c5b5f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:39.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.728+0000 7f160cc1e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f16041acc70 con 0x5575c5b34590 2024-09-10T10:03:39.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.728+0000 7f160cc1e740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f16041ae7b0 con 0x5575c5b3ee50 2024-09-10T10:03:39.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.728+0000 7f160cc1e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f16041ad340 con 0x5575c5b33c70 2024-09-10T10:03:39.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.729+0000 7f1603e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5575c5b33c70 0x5575c5b366d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:39.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.729+0000 7f1603e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5575c5b33c70 0x5575c5b366d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:56488/0 (socket says 172.21.15.77:56488) 2024-09-10T10:03:39.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.729+0000 7f1603e00640 1 -- 172.21.15.77:0/3207493839 learned_addr learned my addr 172.21.15.77:0/3207493839 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:03:39.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.729+0000 7f1602a00640 1 --2- 172.21.15.77:0/3207493839 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5575c5b34590 0x5575c5b3dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:39.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.729+0000 7f1603e00640 1 --2- 172.21.15.77:0/3207493839 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5575c5b3ee50 0x5575c5b5f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:39.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.729+0000 7f1603e00640 1 -- 172.21.15.77:0/3207493839 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5575c5b3ee50 msgr2=0x5575c5b5f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:39.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.729+0000 7f1603e00640 1 --2- 172.21.15.77:0/3207493839 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5575c5b3ee50 0x5575c5b5f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:39.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.729+0000 7f1603e00640 1 -- 172.21.15.77:0/3207493839 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5575c5b34590 msgr2=0x5575c5b3dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:39.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.729+0000 7f1603e00640 1 --2- 172.21.15.77:0/3207493839 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5575c5b34590 0x5575c5b3dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:39.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.729+0000 7f1603e00640 1 -- 172.21.15.77:0/3207493839 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575c5b30030 con 0x5575c5b33c70 2024-09-10T10:03:39.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.730+0000 7f1603e00640 1 --2- 172.21.15.77:0/3207493839 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5575c5b33c70 0x5575c5b366d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f15fc004e80 tx=0x7f15fc0044a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2314b267bb472fb3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:39.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.730+0000 7f15ef400640 1 -- 172.21.15.77:0/3207493839 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15fc004670 con 0x5575c5b33c70 2024-09-10T10:03:39.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.730+0000 7f15ef400640 1 -- 172.21.15.77:0/3207493839 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15fc004810 con 0x5575c5b33c70 2024-09-10T10:03:39.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.730+0000 7f15ef400640 1 -- 172.21.15.77:0/3207493839 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15fc005560 con 0x5575c5b33c70 2024-09-10T10:03:39.734 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.731+0000 7f160cc1e740 1 -- 172.21.15.77:0/3207493839 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5575c5b33c70 msgr2=0x5575c5b366d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:39.734 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.731+0000 7f160cc1e740 1 --2- 172.21.15.77:0/3207493839 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5575c5b33c70 0x5575c5b366d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f15fc004e80 tx=0x7f15fc0044a0 comp rx=0 tx=0).stop 2024-09-10T10:03:39.734 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.731+0000 7f160cc1e740 1 -- 172.21.15.77:0/3207493839 shutdown_connections 2024-09-10T10:03:39.734 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.731+0000 7f160cc1e740 1 --2- 172.21.15.77:0/3207493839 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5575c5b3ee50 0x5575c5b5f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:39.734 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.731+0000 7f160cc1e740 1 --2- 172.21.15.77:0/3207493839 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5575c5b34590 0x5575c5b3dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:39.734 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.731+0000 7f160cc1e740 1 --2- 172.21.15.77:0/3207493839 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5575c5b33c70 0x5575c5b366d0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:39.734 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.731+0000 7f160cc1e740 1 -- 172.21.15.77:0/3207493839 >> 172.21.15.77:0/3207493839 conn(0x5575c5b30200 msgr2=0x5575c5b3d530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:39.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.731+0000 7f160cc1e740 1 -- 172.21.15.77:0/3207493839 shutdown_connections 2024-09-10T10:03:39.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.731+0000 7f160cc1e740 1 -- 172.21.15.77:0/3207493839 wait complete. 2024-09-10T10:03:39.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.732+0000 7f160cc1e740 1 -- start start 2024-09-10T10:03:39.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.732+0000 7f160cc1e740 1 Processor -- start 2024-09-10T10:03:39.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.733+0000 7f160cc1e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5575c5b33c70 0x5575c5b4f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:39.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.733+0000 7f160cc1e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5575c5b34590 0x5575c5b4fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:39.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.733+0000 7f160cc1e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5575c5b3ee50 0x5575c5b4c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:39.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.733+0000 7f160cc1e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f16041ae0e0 con 0x5575c5b34590 2024-09-10T10:03:39.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.733+0000 7f160cc1e740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f16041ada10 con 0x5575c5b3ee50 2024-09-10T10:03:39.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.733+0000 7f160cc1e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f16041aac50 con 0x5575c5b33c70 2024-09-10T10:03:39.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.733+0000 7f160cc1e740 10 client.0 _ll_register_callbacks cb 0x7f15fc0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:03:39.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.734+0000 7f1603e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5575c5b33c70 0x5575c5b4f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:39.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.734+0000 7f1603e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5575c5b33c70 0x5575c5b4f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:56496/0 (socket says 172.21.15.77:56496) 2024-09-10T10:03:39.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.734+0000 7f1603e00640 1 -- 172.21.15.77:0/347552528 learned_addr learned my addr 172.21.15.77:0/347552528 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:03:39.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.734+0000 7f1603e00640 1 --2- 172.21.15.77:0/347552528 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5575c5b3ee50 0x5575c5b4c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:39.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.734+0000 7f1602a00640 1 --2- 172.21.15.77:0/347552528 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5575c5b34590 0x5575c5b4fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:39.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.735+0000 7f1603e00640 1 -- 172.21.15.77:0/347552528 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5575c5b3ee50 msgr2=0x5575c5b4c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:39.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.735+0000 7f1603e00640 1 --2- 172.21.15.77:0/347552528 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5575c5b3ee50 0x5575c5b4c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:39.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.735+0000 7f1603e00640 1 -- 172.21.15.77:0/347552528 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5575c5b34590 msgr2=0x5575c5b4fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:39.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.735+0000 7f1603e00640 1 --2- 172.21.15.77:0/347552528 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5575c5b34590 0x5575c5b4fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:39.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.735+0000 7f1603e00640 1 -- 172.21.15.77:0/347552528 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575c5b2b770 con 0x5575c5b33c70 2024-09-10T10:03:39.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.735+0000 7f1603e00640 1 --2- 172.21.15.77:0/347552528 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5575c5b33c70 0x5575c5b4f6d0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f15fc006010 tx=0x7f15fc004df0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c39c695d953e09e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:39.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.735+0000 7f15ecc00640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-10T10:03:39.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.735+0000 7f15ecc00640 1 -- 172.21.15.77:0/347552528 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15fc004fc0 con 0x5575c5b33c70 2024-09-10T10:03:39.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.735+0000 7f15ecc00640 1 -- 172.21.15.77:0/347552528 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f15fc005160 con 0x5575c5b33c70 2024-09-10T10:03:39.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.735+0000 7f160cc1e740 1 -- 172.21.15.77:0/347552528 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15fc004810 con 0x5575c5b33c70 2024-09-10T10:03:39.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.736+0000 7f15ecc00640 1 -- 172.21.15.77:0/347552528 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f15fc003a30 con 0x5575c5b33c70 2024-09-10T10:03:39.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.736+0000 7f160cc1e740 10 client.27154 fetch_fsmap learned FSMap version 35 2024-09-10T10:03:39.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.736+0000 7f160cc1e740 1 -- 172.21.15.77:0/347552528 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5575c5bc2750 con 0x5575c5b33c70 2024-09-10T10:03:39.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.737+0000 7f15ecc00640 1 -- 172.21.15.77:0/347552528 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15fc004810 con 0x5575c5b33c70 2024-09-10T10:03:39.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.737+0000 7f15ecc00640 1 -- 172.21.15.77:0/347552528 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f15fc034070 con 0x5575c5b33c70 2024-09-10T10:03:39.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.737+0000 7f160cc1e740 10 client.27154 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T10:03:39.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.737+0000 7f160cc1e740 10 client.27154 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-10T10:03:39.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.737+0000 7f160cc1e740 20 client.27154 populate_metadata read hostname 'smithi077' 2024-09-10T10:03:39.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.737+0000 7f160cc1e740 1 --2- 172.21.15.77:0/347552528 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x5575c5b57710 0x5575c5bc4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:39.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.737+0000 7f160cc1e740 4 client.27154 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:03:39.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.737+0000 7f160cc1e740 1 -- 172.21.15.77:0/347552528 --> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5575c5aa1e60 con 0x5575c5b57710 2024-09-10T10:03:39.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.737+0000 7f1602a00640 1 --2- 172.21.15.77:0/347552528 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x5575c5b57710 0x5575c5bc4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:39.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.738+0000 7f1602a00640 1 --2- 172.21.15.77:0/347552528 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x5575c5b57710 0x5575c5bc4ae0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a9d83be23f28b0bc server_cookie=5d50b31fc4fe8d91 in_seq=0 out_seq=0 2024-09-10T10:03:39.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.738+0000 7f15ecc00640 10 client.27154 ms_handle_connect on v2:172.21.15.120:6836/3033280709 2024-09-10T10:03:39.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.738+0000 7f15ecc00640 1 -- 172.21.15.77:0/347552528 <== mds.1 v2:172.21.15.120:6836/3033280709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5575c5aa1e60 con 0x5575c5b57710 2024-09-10T10:03:39.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.738+0000 7f15ecc00640 10 client.27154 handle_command_reply: tid=0 2024-09-10T10:03:39.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.738+0000 7f160cc1e740 1 client.27154 shutdown 2024-09-10T10:03:39.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.739+0000 7f160cc1e740 1 -- 172.21.15.77:0/347552528 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5575c5b33c70 msgr2=0x5575c5b4f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:39.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.739+0000 7f160cc1e740 1 --2- 172.21.15.77:0/347552528 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5575c5b33c70 0x5575c5b4f6d0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f15fc006010 tx=0x7f15fc004df0 comp rx=0 tx=0).stop 2024-09-10T10:03:39.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.739+0000 7f160cc1e740 1 -- 172.21.15.77:0/347552528 shutdown_connections 2024-09-10T10:03:39.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.739+0000 7f160cc1e740 1 --2- 172.21.15.77:0/347552528 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x5575c5b57710 0x5575c5bc4ae0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:39.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.739+0000 7f160cc1e740 1 --2- 172.21.15.77:0/347552528 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5575c5b3ee50 0x5575c5b4c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:39.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.739+0000 7f160cc1e740 1 --2- 172.21.15.77:0/347552528 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5575c5b34590 0x5575c5b4fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:39.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.739+0000 7f160cc1e740 1 --2- 172.21.15.77:0/347552528 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5575c5b33c70 0x5575c5b4f6d0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:39.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.739+0000 7f160cc1e740 1 -- 172.21.15.77:0/347552528 >> 172.21.15.77:0/347552528 conn(0x5575c5b30200 msgr2=0x5575c5b3d530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:39.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.739+0000 7f15ecc00640 0 client.27154 ms_handle_reset on v2:172.21.15.120:6836/3033280709 2024-09-10T10:03:39.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.739+0000 7f160cc1e740 1 -- 172.21.15.77:0/347552528 shutdown_connections 2024-09-10T10:03:39.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.739+0000 7f160cc1e740 1 -- 172.21.15.77:0/347552528 wait complete. 2024-09-10T10:03:39.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.739+0000 7f160cc1e740 20 client.27154 trim_cache size 0 max 16384 2024-09-10T10:03:39.743 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-10T10:03:39.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.740+0000 7f160a200640 1 -- 172.21.15.77:0/3416207332 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f15e80a2c00 msgr2=0x7f15e80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:39.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.740+0000 7f160a200640 1 --2- 172.21.15.77:0/3416207332 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f15e80a2c00 0x7f15e80a50c0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f15f409b8c0 tx=0x7f15f40977c0 comp rx=0 tx=0).stop 2024-09-10T10:03:39.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.740+0000 7f160a200640 1 -- 172.21.15.77:0/3416207332 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1604101de0 msgr2=0x7f16040744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:39.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.740+0000 7f160a200640 1 --2- 172.21.15.77:0/3416207332 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1604101de0 0x7f16040744f0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f15f400e9f0 tx=0x7f15f400eeb0 comp rx=0 tx=0).stop 2024-09-10T10:03:39.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.740+0000 7f160a200640 1 -- 172.21.15.77:0/3416207332 shutdown_connections 2024-09-10T10:03:39.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.740+0000 7f160a200640 1 --2- 172.21.15.77:0/3416207332 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f15e80a2c00 0x7f15e80a50c0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:39.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.740+0000 7f160a200640 1 --2- 172.21.15.77:0/3416207332 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1604106f10 0x7f1604074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:39.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.740+0000 7f160a200640 1 --2- 172.21.15.77:0/3416207332 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1604102790 0x7f1604074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:39.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.740+0000 7f160a200640 1 --2- 172.21.15.77:0/3416207332 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1604101de0 0x7f16040744f0 secure :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f15f400e9f0 tx=0x7f15f400eeb0 comp rx=0 tx=0).stop 2024-09-10T10:03:39.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.740+0000 7f160a200640 1 -- 172.21.15.77:0/3416207332 >> 172.21.15.77:0/3416207332 conn(0x7f16040fd8c0 msgr2=0x7f16040fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:39.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.741+0000 7f160a200640 1 -- 172.21.15.77:0/3416207332 shutdown_connections 2024-09-10T10:03:39.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:39.741+0000 7f160a200640 1 -- 172.21.15.77:0/3416207332 wait complete. 2024-09-10T10:03:40.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:03:40 smithi134 ceph-mon[35674]: pgmap v646: 129 pgs: 129 active+clean; 154 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.2 MiB/s wr, 476 op/s 2024-09-10T10:03:40.637 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:03:40 smithi077 ceph-mon[30319]: pgmap v646: 129 pgs: 129 active+clean; 154 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.2 MiB/s wr, 476 op/s 2024-09-10T10:03:40.673 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:03:40 smithi120 ceph-mon[34770]: pgmap v646: 129 pgs: 129 active+clean; 154 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.2 MiB/s wr, 476 op/s 2024-09-10T10:03:40.681 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T10:03:41.077 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24563 quiesce db / --timeout 90 --expiration 133.4 --await-for 45.0' 2024-09-10T10:03:41.077 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24563 quiesce db / --timeout 90 --expiration 133.4 --await-for 45.0 2024-09-10T10:03:41.201 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:03:41.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.776+0000 7f86f6000640 1 -- 172.21.15.77:0/3563643050 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86f0101dd0 msgr2=0x7f86f01021b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:41.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.776+0000 7f86f6000640 1 --2- 172.21.15.77:0/3563643050 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86f0101dd0 0x7f86f01021b0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f86e0009900 tx=0x7f86e002a3f0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.776+0000 7f86f6000640 1 -- 172.21.15.77:0/3563643050 shutdown_connections 2024-09-10T10:03:41.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.776+0000 7f86f6000640 1 --2- 172.21.15.77:0/3563643050 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86f0106f00 0x7f86f010ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.776+0000 7f86f6000640 1 --2- 172.21.15.77:0/3563643050 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f86f0102780 0x7f86f01067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.776+0000 7f86f6000640 1 --2- 172.21.15.77:0/3563643050 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86f0101dd0 0x7f86f01021b0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.776+0000 7f86f6000640 1 -- 172.21.15.77:0/3563643050 >> 172.21.15.77:0/3563643050 conn(0x7f86f00fd890 msgr2=0x7f86f00ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:41.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.778+0000 7f86f6000640 1 -- 172.21.15.77:0/3563643050 shutdown_connections 2024-09-10T10:03:41.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.778+0000 7f86f6000640 1 -- 172.21.15.77:0/3563643050 wait complete. 2024-09-10T10:03:41.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.779+0000 7f86f6000640 1 Processor -- start 2024-09-10T10:03:41.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.779+0000 7f86f6000640 1 -- start start 2024-09-10T10:03:41.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.779+0000 7f86f6000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f86f0101dd0 0x7f86f00744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:41.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.779+0000 7f86f6000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86f0102780 0x7f86f0074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:41.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.779+0000 7f86f6000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86f0106f00 0x7f86f0074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:41.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.780+0000 7f86f6000640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f86f0111f70 con 0x7f86f0106f00 2024-09-10T10:03:41.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.780+0000 7f86f6000640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f86f0111df0 con 0x7f86f0101dd0 2024-09-10T10:03:41.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.780+0000 7f86f6000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f86f01120f0 con 0x7f86f0102780 2024-09-10T10:03:41.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.780+0000 7f86e7e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86f0102780 0x7f86f0074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:41.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.780+0000 7f86ef400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f86f0101dd0 0x7f86f00744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:41.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.780+0000 7f86e7e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86f0102780 0x7f86f0074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:56526/0 (socket says 172.21.15.77:56526) 2024-09-10T10:03:41.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.780+0000 7f86e7e00640 1 -- 172.21.15.77:0/3589910547 learned_addr learned my addr 172.21.15.77:0/3589910547 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:03:41.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.780+0000 7f86efe00640 1 --2- 172.21.15.77:0/3589910547 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86f0106f00 0x7f86f0074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:41.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.780+0000 7f86ef400640 1 -- 172.21.15.77:0/3589910547 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86f0102780 msgr2=0x7f86f0074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:41.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.780+0000 7f86ef400640 1 --2- 172.21.15.77:0/3589910547 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86f0102780 0x7f86f0074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.780+0000 7f86ef400640 1 -- 172.21.15.77:0/3589910547 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86f0106f00 msgr2=0x7f86f0074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:41.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.780+0000 7f86ef400640 1 --2- 172.21.15.77:0/3589910547 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86f0106f00 0x7f86f0074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.780+0000 7f86ef400640 1 -- 172.21.15.77:0/3589910547 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86f01a6930 con 0x7f86f0101dd0 2024-09-10T10:03:41.784 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.781+0000 7f86ef400640 1 --2- 172.21.15.77:0/3589910547 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f86f0101dd0 0x7f86f00744f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f86e002a910 tx=0x7f86e0003b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:41.784 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.781+0000 7f86ecc00640 1 -- 172.21.15.77:0/3589910547 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e003f070 con 0x7f86f0101dd0 2024-09-10T10:03:41.784 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.781+0000 7f86ecc00640 1 -- 172.21.15.77:0/3589910547 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86e0031c90 con 0x7f86f0101dd0 2024-09-10T10:03:41.784 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.781+0000 7f86ecc00640 1 -- 172.21.15.77:0/3589910547 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e0035030 con 0x7f86f0101dd0 2024-09-10T10:03:41.784 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.781+0000 7f86f6000640 1 -- 172.21.15.77:0/3589910547 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86f01a6bc0 con 0x7f86f0101dd0 2024-09-10T10:03:41.784 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.781+0000 7f86f6000640 1 -- 172.21.15.77:0/3589910547 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86f01a70f0 con 0x7f86f0101dd0 2024-09-10T10:03:41.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.784+0000 7f86ecc00640 1 -- 172.21.15.77:0/3589910547 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f86e0042070 con 0x7f86f0101dd0 2024-09-10T10:03:41.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.785+0000 7f86f8a07740 1 Processor -- start 2024-09-10T10:03:41.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.785+0000 7f86f8a07740 1 -- start start 2024-09-10T10:03:41.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.786+0000 7f86f8a07740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eb7a252c70 0x55eb7a253050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:41.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.786+0000 7f86f8a07740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb7a253590 0x55eb7a25d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:41.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.786+0000 7f86f8a07740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eb7a25de50 0x55eb7a260240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:41.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.786+0000 7f86f8a07740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55eb7a1b93e0 con 0x55eb7a253590 2024-09-10T10:03:41.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.786+0000 7f86f8a07740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55eb7a160d80 con 0x55eb7a252c70 2024-09-10T10:03:41.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.786+0000 7f86f8a07740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55eb7a1b53a0 con 0x55eb7a25de50 2024-09-10T10:03:41.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.786+0000 7f86ef400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb7a253590 0x55eb7a25d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:41.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.786+0000 7f86ef400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb7a253590 0x55eb7a25d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58228/0 (socket says 172.21.15.77:58228) 2024-09-10T10:03:41.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.786+0000 7f86e7e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eb7a25de50 0x55eb7a260240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:41.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.786+0000 7f86ef400640 1 -- 172.21.15.77:0/2545703026 learned_addr learned my addr 172.21.15.77:0/2545703026 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:03:41.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.786+0000 7f86efe00640 1 --2- 172.21.15.77:0/2545703026 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eb7a252c70 0x55eb7a253050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:41.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.786+0000 7f86ecc00640 1 --2- 172.21.15.77:0/3589910547 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f86d40a2c00 0x7f86d40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:41.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.787+0000 7f86ecc00640 1 -- 172.21.15.77:0/3589910547 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f86e00efda0 con 0x7f86f0101dd0 2024-09-10T10:03:41.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.787+0000 7f86ef400640 1 -- 172.21.15.77:0/2545703026 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eb7a25de50 msgr2=0x55eb7a260240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:41.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.787+0000 7f86ef400640 1 --2- 172.21.15.77:0/2545703026 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eb7a25de50 0x55eb7a260240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.787+0000 7f86ef400640 1 -- 172.21.15.77:0/2545703026 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eb7a252c70 msgr2=0x55eb7a253050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:41.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.787+0000 7f86ef400640 1 --2- 172.21.15.77:0/2545703026 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eb7a252c70 0x55eb7a253050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.787+0000 7f86ef400640 1 -- 172.21.15.77:0/2545703026 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb7a17fae0 con 0x55eb7a253590 2024-09-10T10:03:41.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.787+0000 7f86efe00640 1 --2- 172.21.15.77:0/3589910547 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f86d40a2c00 0x7f86d40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:41.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.787+0000 7f86ef400640 1 --2- 172.21.15.77:0/2545703026 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb7a253590 0x55eb7a25d910 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f86e00b8610 tx=0x7f86e002a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acc75d0df641ce77 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:41.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.787+0000 7f86dfe00640 1 -- 172.21.15.77:0/2545703026 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e0049070 con 0x55eb7a253590 2024-09-10T10:03:41.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.788+0000 7f86dfe00640 1 -- 172.21.15.77:0/2545703026 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86e0031e40 con 0x55eb7a253590 2024-09-10T10:03:41.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.788+0000 7f86f8a07740 1 -- 172.21.15.77:0/2545703026 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb7a253590 msgr2=0x55eb7a25d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:41.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.788+0000 7f86efe00640 1 --2- 172.21.15.77:0/3589910547 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f86d40a2c00 0x7f86d40a50c0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f86e80094d0 tx=0x7f86e8004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:41.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.788+0000 7f86f8a07740 1 --2- 172.21.15.77:0/2545703026 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb7a253590 0x55eb7a25d910 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f86e00b8610 tx=0x7f86e002a3f0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.789+0000 7f86f8a07740 1 -- 172.21.15.77:0/2545703026 shutdown_connections 2024-09-10T10:03:41.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.789+0000 7f86f8a07740 1 --2- 172.21.15.77:0/2545703026 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eb7a25de50 0x55eb7a260240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.789+0000 7f86f8a07740 1 --2- 172.21.15.77:0/2545703026 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb7a253590 0x55eb7a25d910 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.789+0000 7f86f8a07740 1 --2- 172.21.15.77:0/2545703026 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eb7a252c70 0x55eb7a253050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.789+0000 7f86f8a07740 1 -- 172.21.15.77:0/2545703026 >> 172.21.15.77:0/2545703026 conn(0x55eb7a24f200 msgr2=0x55eb7a24f5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:41.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.789+0000 7f86f8a07740 1 -- 172.21.15.77:0/2545703026 shutdown_connections 2024-09-10T10:03:41.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.789+0000 7f86f8a07740 1 -- 172.21.15.77:0/2545703026 wait complete. 2024-09-10T10:03:41.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.789+0000 7f86f8a07740 1 -- start start 2024-09-10T10:03:41.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.790+0000 7f86f8a07740 1 Processor -- start 2024-09-10T10:03:41.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.790+0000 7f86f8a07740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eb7a252c70 0x55eb7a25ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:41.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.790+0000 7f86f8a07740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb7a253590 0x55eb7a25b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:41.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.790+0000 7f86f8a07740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eb7a25de50 0x55eb7a25b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:41.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.791+0000 7f86f8a07740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55eb7a2607e0 con 0x55eb7a253590 2024-09-10T10:03:41.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.791+0000 7f86f8a07740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55eb7a1740f0 con 0x55eb7a252c70 2024-09-10T10:03:41.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.791+0000 7f86f8a07740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55eb7a260960 con 0x55eb7a25de50 2024-09-10T10:03:41.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.791+0000 7f86f8a07740 10 client.0 _ll_register_callbacks cb 0x55eb7a15bbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:03:41.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.791+0000 7f86efe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eb7a25de50 0x55eb7a25b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:41.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.791+0000 7f86e7e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb7a253590 0x55eb7a25b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:41.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.791+0000 7f86efe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eb7a25de50 0x55eb7a25b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:56536/0 (socket says 172.21.15.77:56536) 2024-09-10T10:03:41.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.791+0000 7f86efe00640 1 -- 172.21.15.77:0/1947306141 learned_addr learned my addr 172.21.15.77:0/1947306141 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:03:41.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.792+0000 7f86efe00640 1 -- 172.21.15.77:0/1947306141 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eb7a252c70 msgr2=0x55eb7a25ac00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T10:03:41.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.792+0000 7f86efe00640 1 --2- 172.21.15.77:0/1947306141 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eb7a252c70 0x55eb7a25ac00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.792+0000 7f86efe00640 1 -- 172.21.15.77:0/1947306141 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb7a253590 msgr2=0x55eb7a25b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:41.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.792+0000 7f86efe00640 1 --2- 172.21.15.77:0/1947306141 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb7a253590 0x55eb7a25b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.792+0000 7f86efe00640 1 -- 172.21.15.77:0/1947306141 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb79f2a3c0 con 0x55eb7a25de50 2024-09-10T10:03:41.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.793+0000 7f86efe00640 1 --2- 172.21.15.77:0/1947306141 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eb7a25de50 0x55eb7a25b680 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f86e8011cb0 tx=0x7f86e8004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=392e8086d336f9fa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:41.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.793+0000 7f86dcc00640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-10T10:03:41.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.793+0000 7f86dcc00640 1 -- 172.21.15.77:0/1947306141 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e8027640 con 0x55eb7a25de50 2024-09-10T10:03:41.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.793+0000 7f86f8a07740 1 -- 172.21.15.77:0/1947306141 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55eb7a1c6950 con 0x55eb7a25de50 2024-09-10T10:03:41.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.793+0000 7f86dcc00640 1 -- 172.21.15.77:0/1947306141 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f86e801b410 con 0x55eb7a25de50 2024-09-10T10:03:41.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.794+0000 7f86dcc00640 1 -- 172.21.15.77:0/1947306141 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f86e80277e0 con 0x55eb7a25de50 2024-09-10T10:03:41.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.794+0000 7f86f8a07740 10 client.27166 fetch_fsmap learned FSMap version 35 2024-09-10T10:03:41.797 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.794+0000 7f86f8a07740 1 -- 172.21.15.77:0/1947306141 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55eb7a27d800 con 0x55eb7a25de50 2024-09-10T10:03:41.797 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.794+0000 7f86dcc00640 1 -- 172.21.15.77:0/1947306141 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55eb7a1c6950 con 0x55eb7a25de50 2024-09-10T10:03:41.797 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.795+0000 7f86dcc00640 1 -- 172.21.15.77:0/1947306141 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f86e8025020 con 0x55eb7a25de50 2024-09-10T10:03:41.797 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.795+0000 7f86f8a07740 10 client.27166 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T10:03:41.797 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.795+0000 7f86f8a07740 10 client.27166 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T10:03:41.797 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.795+0000 7f86f8a07740 20 client.27166 populate_metadata read hostname 'smithi077' 2024-09-10T10:03:41.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.795+0000 7f86f8a07740 1 --2- 172.21.15.77:0/1947306141 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55eb7a27ecc0 0x55eb7a281130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:41.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.795+0000 7f86f8a07740 4 client.27166 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:03:41.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.795+0000 7f86f8a07740 1 -- 172.21.15.77:0/1947306141 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eb7a1c0e60 con 0x55eb7a27ecc0 2024-09-10T10:03:41.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.796+0000 7f86ef400640 1 --2- 172.21.15.77:0/1947306141 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55eb7a27ecc0 0x55eb7a281130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:41.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.797+0000 7f86ef400640 1 --2- 172.21.15.77:0/1947306141 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55eb7a27ecc0 0x55eb7a281130 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71f17064fc4fe951 server_cookie=c97b8e9ff7b67969 in_seq=0 out_seq=0 2024-09-10T10:03:41.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.797+0000 7f86dcc00640 10 client.27166 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T10:03:41.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.797+0000 7f86dcc00640 1 -- 172.21.15.77:0/1947306141 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55eb7a1c0e60 con 0x55eb7a27ecc0 2024-09-10T10:03:41.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.797+0000 7f86dcc00640 10 client.27166 handle_command_reply: tid=0 2024-09-10T10:03:41.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.797+0000 7f86f8a07740 1 client.27166 shutdown 2024-09-10T10:03:41.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.798+0000 7f86f8a07740 1 -- 172.21.15.77:0/1947306141 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eb7a25de50 msgr2=0x55eb7a25b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:41.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.798+0000 7f86f8a07740 1 --2- 172.21.15.77:0/1947306141 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eb7a25de50 0x55eb7a25b680 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f86e8011cb0 tx=0x7f86e8004ab0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.798+0000 7f86f8a07740 1 -- 172.21.15.77:0/1947306141 shutdown_connections 2024-09-10T10:03:41.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.798+0000 7f86f8a07740 1 --2- 172.21.15.77:0/1947306141 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55eb7a27ecc0 0x55eb7a281130 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.798+0000 7f86f8a07740 1 --2- 172.21.15.77:0/1947306141 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eb7a25de50 0x55eb7a25b680 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.798+0000 7f86f8a07740 1 --2- 172.21.15.77:0/1947306141 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb7a253590 0x55eb7a25b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.798+0000 7f86f8a07740 1 --2- 172.21.15.77:0/1947306141 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eb7a252c70 0x55eb7a25ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.798+0000 7f86dcc00640 0 client.27166 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T10:03:41.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.798+0000 7f86f8a07740 1 -- 172.21.15.77:0/1947306141 >> 172.21.15.77:0/1947306141 conn(0x55eb7a24f200 msgr2=0x55eb7a254f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:41.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.798+0000 7f86f8a07740 1 -- 172.21.15.77:0/1947306141 shutdown_connections 2024-09-10T10:03:41.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.798+0000 7f86f8a07740 1 -- 172.21.15.77:0/1947306141 wait complete. 2024-09-10T10:03:41.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.798+0000 7f86f8a07740 20 client.27166 trim_cache size 0 max 16384 2024-09-10T10:03:41.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.825+0000 7f86f8a07740 1 Processor -- start 2024-09-10T10:03:41.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.825+0000 7f86f8a07740 1 -- start start 2024-09-10T10:03:41.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.825+0000 7f86f8a07740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eb7a252c70 0x55eb7a25cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:41.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.825+0000 7f86f8a07740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb7a253590 0x55eb7a286a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:41.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.825+0000 7f86f8a07740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eb7a25de50 0x55eb7a286f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:41.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.825+0000 7f86f8a07740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f86f01a8850 con 0x55eb7a253590 2024-09-10T10:03:41.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.825+0000 7f86f8a07740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f86f01aa390 con 0x55eb7a252c70 2024-09-10T10:03:41.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.825+0000 7f86f8a07740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f86f01a8f20 con 0x55eb7a25de50 2024-09-10T10:03:41.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.826+0000 7f86ef400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eb7a252c70 0x55eb7a25cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:41.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.826+0000 7f86ef400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eb7a252c70 0x55eb7a25cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:46314/0 (socket says 172.21.15.77:46314) 2024-09-10T10:03:41.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.826+0000 7f86ef400640 1 -- 172.21.15.77:0/1179054156 learned_addr learned my addr 172.21.15.77:0/1179054156 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:03:41.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.826+0000 7f86e7e00640 1 --2- 172.21.15.77:0/1179054156 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb7a253590 0x55eb7a286a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:41.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.826+0000 7f86ef400640 1 -- 172.21.15.77:0/1179054156 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eb7a25de50 msgr2=0x55eb7a286f40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T10:03:41.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.826+0000 7f86ef400640 1 --2- 172.21.15.77:0/1179054156 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eb7a25de50 0x55eb7a286f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.826+0000 7f86ef400640 1 -- 172.21.15.77:0/1179054156 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb7a253590 msgr2=0x55eb7a286a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:41.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.826+0000 7f86ef400640 1 --2- 172.21.15.77:0/1179054156 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb7a253590 0x55eb7a286a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.826+0000 7f86ef400640 1 -- 172.21.15.77:0/1179054156 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb7a24f030 con 0x55eb7a252c70 2024-09-10T10:03:41.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.827+0000 7f86e7e00640 1 --2- 172.21.15.77:0/1179054156 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb7a253590 0x55eb7a286a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:03:41.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.827+0000 7f86ef400640 1 --2- 172.21.15.77:0/1179054156 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eb7a252c70 0x55eb7a25cb70 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f86e00bc140 tx=0x7f86e0031490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4aee4c7f9a3e476 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:41.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.827+0000 7f86df400640 1 -- 172.21.15.77:0/1179054156 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e0049040 con 0x55eb7a252c70 2024-09-10T10:03:41.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.827+0000 7f86df400640 1 -- 172.21.15.77:0/1179054156 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86e00f3630 con 0x55eb7a252c70 2024-09-10T10:03:41.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.827+0000 7f86df400640 1 -- 172.21.15.77:0/1179054156 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e00dc0c0 con 0x55eb7a252c70 2024-09-10T10:03:41.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.828+0000 7f86f8a07740 1 -- 172.21.15.77:0/1179054156 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eb7a252c70 msgr2=0x55eb7a25cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:41.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.828+0000 7f86f8a07740 1 --2- 172.21.15.77:0/1179054156 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eb7a252c70 0x55eb7a25cb70 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f86e00bc140 tx=0x7f86e0031490 comp rx=0 tx=0).stop 2024-09-10T10:03:41.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.828+0000 7f86f8a07740 1 -- 172.21.15.77:0/1179054156 shutdown_connections 2024-09-10T10:03:41.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.828+0000 7f86f8a07740 1 --2- 172.21.15.77:0/1179054156 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eb7a25de50 0x55eb7a286f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.828+0000 7f86f8a07740 1 --2- 172.21.15.77:0/1179054156 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb7a253590 0x55eb7a286a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.828+0000 7f86f8a07740 1 --2- 172.21.15.77:0/1179054156 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eb7a252c70 0x55eb7a25cb70 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.828+0000 7f86f8a07740 1 -- 172.21.15.77:0/1179054156 >> 172.21.15.77:0/1179054156 conn(0x55eb7a24f200 msgr2=0x55eb7a25c460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:41.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.829+0000 7f86f8a07740 1 -- 172.21.15.77:0/1179054156 shutdown_connections 2024-09-10T10:03:41.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.829+0000 7f86f8a07740 1 -- 172.21.15.77:0/1179054156 wait complete. 2024-09-10T10:03:41.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.829+0000 7f86f8a07740 1 -- start start 2024-09-10T10:03:41.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.830+0000 7f86f8a07740 1 Processor -- start 2024-09-10T10:03:41.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.830+0000 7f86f8a07740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eb7a252c70 0x55eb7a265c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:41.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.830+0000 7f86f8a07740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb7a253590 0x55eb7a2661a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:41.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.830+0000 7f86f8a07740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eb7a25de50 0x55eb7a260c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:41.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.830+0000 7f86f8a07740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f86f01a9cc0 con 0x55eb7a253590 2024-09-10T10:03:41.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.830+0000 7f86f8a07740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f86f01a95f0 con 0x55eb7a25de50 2024-09-10T10:03:41.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.830+0000 7f86f8a07740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f86f0073990 con 0x55eb7a252c70 2024-09-10T10:03:41.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.830+0000 7f86f8a07740 10 client.0 _ll_register_callbacks cb 0x7f86e8009d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:03:41.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.831+0000 7f86efe00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eb7a25de50 0x55eb7a260c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:41.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.831+0000 7f86efe00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eb7a25de50 0x55eb7a260c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:46324/0 (socket says 172.21.15.77:46324) 2024-09-10T10:03:41.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.831+0000 7f86efe00640 1 -- 172.21.15.77:0/3975633167 learned_addr learned my addr 172.21.15.77:0/3975633167 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:03:41.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.831+0000 7f86ef400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eb7a252c70 0x55eb7a265c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:41.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.832+0000 7f86efe00640 1 -- 172.21.15.77:0/3975633167 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eb7a252c70 msgr2=0x55eb7a265c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:41.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.832+0000 7f86efe00640 1 --2- 172.21.15.77:0/3975633167 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eb7a252c70 0x55eb7a265c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.832+0000 7f86efe00640 1 -- 172.21.15.77:0/3975633167 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb7a253590 msgr2=0x55eb7a2661a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:41.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.832+0000 7f86efe00640 1 --2- 172.21.15.77:0/3975633167 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb7a253590 0x55eb7a2661a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.832+0000 7f86efe00640 1 -- 172.21.15.77:0/3975633167 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb7a269ae0 con 0x55eb7a25de50 2024-09-10T10:03:41.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.832+0000 7f86efe00640 1 --2- 172.21.15.77:0/3975633167 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eb7a25de50 0x55eb7a260c10 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f86e801b9c0 tx=0x7f86e8027670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b57d764329cb3fb8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:41.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.832+0000 7f86dcc00640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T10:03:41.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.832+0000 7f86dcc00640 1 -- 172.21.15.77:0/3975633167 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e8039060 con 0x55eb7a25de50 2024-09-10T10:03:41.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.833+0000 7f86dcc00640 1 -- 172.21.15.77:0/3975633167 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f86e801bcf0 con 0x55eb7a25de50 2024-09-10T10:03:41.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.833+0000 7f86f8a07740 1 -- 172.21.15.77:0/3975633167 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86e00f3630 con 0x55eb7a25de50 2024-09-10T10:03:41.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.833+0000 7f86dcc00640 1 -- 172.21.15.77:0/3975633167 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f86e8011830 con 0x55eb7a25de50 2024-09-10T10:03:41.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.833+0000 7f86f8a07740 10 client.27224 fetch_fsmap learned FSMap version 35 2024-09-10T10:03:41.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.833+0000 7f86f8a07740 1 -- 172.21.15.77:0/3975633167 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55eb7a24a770 con 0x55eb7a25de50 2024-09-10T10:03:41.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.834+0000 7f86dcc00640 1 -- 172.21.15.77:0/3975633167 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86e00f3630 con 0x55eb7a25de50 2024-09-10T10:03:41.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.835+0000 7f86dcc00640 1 -- 172.21.15.77:0/3975633167 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f86e8025070 con 0x55eb7a25de50 2024-09-10T10:03:41.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.835+0000 7f86f8a07740 10 client.27224 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T10:03:41.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.835+0000 7f86f8a07740 10 client.27224 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T10:03:41.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.835+0000 7f86f8a07740 20 client.27224 populate_metadata read hostname 'smithi077' 2024-09-10T10:03:41.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.835+0000 7f86f8a07740 1 --2- 172.21.15.77:0/3975633167 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55eb7a2f2ac0 0x55eb7a2f4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:41.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.835+0000 7f86f8a07740 4 client.27224 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:03:41.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.835+0000 7f86f8a07740 1 -- 172.21.15.77:0/3975633167 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55eb7a13d010 con 0x55eb7a2f2ac0 2024-09-10T10:03:41.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.836+0000 7f86ef400640 1 --2- 172.21.15.77:0/3975633167 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55eb7a2f2ac0 0x55eb7a2f4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:41.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.836+0000 7f86ef400640 1 --2- 172.21.15.77:0/3975633167 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55eb7a2f2ac0 0x55eb7a2f4ee0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5554ba0305b035f server_cookie=7dc38f8fecfff8df in_seq=0 out_seq=0 2024-09-10T10:03:41.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.836+0000 7f86dcc00640 10 client.27224 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T10:03:41.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.836+0000 7f86dcc00640 1 -- 172.21.15.77:0/3975633167 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55eb7a13d010 con 0x55eb7a2f2ac0 2024-09-10T10:03:41.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.837+0000 7f86dcc00640 10 client.27224 handle_command_reply: tid=0 2024-09-10T10:03:41.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.837+0000 7f86f8a07740 1 client.27224 shutdown 2024-09-10T10:03:41.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.837+0000 7f86f8a07740 1 -- 172.21.15.77:0/3975633167 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eb7a25de50 msgr2=0x55eb7a260c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:41.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.837+0000 7f86f8a07740 1 --2- 172.21.15.77:0/3975633167 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eb7a25de50 0x55eb7a260c10 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f86e801b9c0 tx=0x7f86e8027670 comp rx=0 tx=0).stop 2024-09-10T10:03:41.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.837+0000 7f86f8a07740 1 -- 172.21.15.77:0/3975633167 shutdown_connections 2024-09-10T10:03:41.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.837+0000 7f86f8a07740 1 --2- 172.21.15.77:0/3975633167 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55eb7a2f2ac0 0x55eb7a2f4ee0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.837+0000 7f86dcc00640 0 client.27224 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T10:03:41.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.837+0000 7f86f8a07740 1 --2- 172.21.15.77:0/3975633167 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eb7a25de50 0x55eb7a260c10 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.837+0000 7f86f8a07740 1 --2- 172.21.15.77:0/3975633167 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb7a253590 0x55eb7a2661a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.837+0000 7f86f8a07740 1 --2- 172.21.15.77:0/3975633167 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eb7a252c70 0x55eb7a265c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.837+0000 7f86f8a07740 1 -- 172.21.15.77:0/3975633167 >> 172.21.15.77:0/3975633167 conn(0x55eb7a24f200 msgr2=0x55eb7a2692f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:41.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.837+0000 7f86f8a07740 1 -- 172.21.15.77:0/3975633167 shutdown_connections 2024-09-10T10:03:41.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.838+0000 7f86f8a07740 1 -- 172.21.15.77:0/3975633167 wait complete. 2024-09-10T10:03:41.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.838+0000 7f86f8a07740 20 client.27224 trim_cache size 0 max 16384 2024-09-10T10:03:41.842 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-10T10:03:41.842 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.839+0000 7f86f6000640 1 -- 172.21.15.77:0/3589910547 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f86d40a2c00 msgr2=0x7f86d40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:41.842 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.839+0000 7f86f6000640 1 --2- 172.21.15.77:0/3589910547 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f86d40a2c00 0x7f86d40a50c0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f86e80094d0 tx=0x7f86e8004a40 comp rx=0 tx=0).stop 2024-09-10T10:03:41.842 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.839+0000 7f86f6000640 1 -- 172.21.15.77:0/3589910547 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f86f0101dd0 msgr2=0x7f86f00744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:41.842 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.839+0000 7f86f6000640 1 --2- 172.21.15.77:0/3589910547 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f86f0101dd0 0x7f86f00744f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f86e002a910 tx=0x7f86e0003b20 comp rx=0 tx=0).stop 2024-09-10T10:03:41.842 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.839+0000 7f86f6000640 1 -- 172.21.15.77:0/3589910547 shutdown_connections 2024-09-10T10:03:41.842 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.839+0000 7f86f6000640 1 --2- 172.21.15.77:0/3589910547 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f86d40a2c00 0x7f86d40a50c0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.839+0000 7f86f6000640 1 --2- 172.21.15.77:0/3589910547 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86f0106f00 0x7f86f0074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.839+0000 7f86f6000640 1 --2- 172.21.15.77:0/3589910547 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86f0102780 0x7f86f0074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.839+0000 7f86f6000640 1 --2- 172.21.15.77:0/3589910547 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f86f0101dd0 0x7f86f00744f0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.839+0000 7f86f6000640 1 -- 172.21.15.77:0/3589910547 >> 172.21.15.77:0/3589910547 conn(0x7f86f00fd890 msgr2=0x7f86f00fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:41.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.839+0000 7f86f6000640 1 -- 172.21.15.77:0/3589910547 shutdown_connections 2024-09-10T10:03:41.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:41.839+0000 7f86f6000640 1 -- 172.21.15.77:0/3589910547 wait complete. 2024-09-10T10:03:42.615 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:03:43.181 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:03:42 smithi077 ceph-mon[30319]: pgmap v647: 129 pgs: 129 active+clean; 154 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 1.0 MiB/s wr, 417 op/s 2024-09-10T10:03:43.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.220+0000 7f87a8400640 1 -- 172.21.15.77:0/1268474111 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f87a0104970 msgr2=0x7f87a01089c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:43.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.220+0000 7f87a8400640 1 --2- 172.21.15.77:0/1268474111 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f87a0104970 0x7f87a01089c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f878c009900 tx=0x7f878c02a3f0 comp rx=0 tx=0).stop 2024-09-10T10:03:43.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.220+0000 7f87a8400640 1 -- 172.21.15.77:0/1268474111 shutdown_connections 2024-09-10T10:03:43.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.220+0000 7f87a8400640 1 --2- 172.21.15.77:0/1268474111 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f87a01090f0 0x7f87a0110c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:43.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.220+0000 7f87a8400640 1 --2- 172.21.15.77:0/1268474111 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f87a0104970 0x7f87a01089c0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:43.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.220+0000 7f87a8400640 1 --2- 172.21.15.77:0/1268474111 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87a0103fc0 0x7f87a01043a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:43.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.220+0000 7f87a8400640 1 -- 172.21.15.77:0/1268474111 >> 172.21.15.77:0/1268474111 conn(0x7f87a00ffa20 msgr2=0x7f87a0101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:43.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.220+0000 7f87a8400640 1 -- 172.21.15.77:0/1268474111 shutdown_connections 2024-09-10T10:03:43.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.220+0000 7f87a8400640 1 -- 172.21.15.77:0/1268474111 wait complete. 2024-09-10T10:03:43.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.222+0000 7f87a8400640 1 Processor -- start 2024-09-10T10:03:43.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.222+0000 7f87a8400640 1 -- start start 2024-09-10T10:03:43.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.223+0000 7f87a8400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f87a0103fc0 0x7f87a019a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:43.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.223+0000 7f87a8400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87a0104970 0x7f87a019ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:43.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.223+0000 7f87a8400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f87a01090f0 0x7f87a019ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:43.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.223+0000 7f87a8400640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f87a0114140 con 0x7f87a0104970 2024-09-10T10:03:43.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.223+0000 7f87a8400640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f87a0113fc0 con 0x7f87a0103fc0 2024-09-10T10:03:43.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.223+0000 7f87a8400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f87a01142c0 con 0x7f87a01090f0 2024-09-10T10:03:43.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.224+0000 7f87a5a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f87a0103fc0 0x7f87a019a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:43.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.224+0000 7f87a5a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f87a0103fc0 0x7f87a019a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:53830/0 (socket says 172.21.15.77:53830) 2024-09-10T10:03:43.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.224+0000 7f87a5a00640 1 -- 172.21.15.77:0/3398624848 learned_addr learned my addr 172.21.15.77:0/3398624848 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:03:43.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.224+0000 7f87a5000640 1 --2- 172.21.15.77:0/3398624848 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87a0104970 0x7f87a019ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:43.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.224+0000 7f87a6400640 1 --2- 172.21.15.77:0/3398624848 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f87a01090f0 0x7f87a019ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:43.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.224+0000 7f87a5a00640 1 -- 172.21.15.77:0/3398624848 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f87a01090f0 msgr2=0x7f87a019ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:43.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.224+0000 7f87a5a00640 1 --2- 172.21.15.77:0/3398624848 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f87a01090f0 0x7f87a019ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:43.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.224+0000 7f87a5a00640 1 -- 172.21.15.77:0/3398624848 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87a0104970 msgr2=0x7f87a019ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:43.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.224+0000 7f87a5a00640 1 --2- 172.21.15.77:0/3398624848 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87a0104970 0x7f87a019ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:43.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.224+0000 7f87a5a00640 1 -- 172.21.15.77:0/3398624848 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87a019f600 con 0x7f87a0103fc0 2024-09-10T10:03:43.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.225+0000 7f87a5a00640 1 --2- 172.21.15.77:0/3398624848 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f87a0103fc0 0x7f87a019a650 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f879800eab0 tx=0x7f879800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:43.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.225+0000 7f879e000640 1 -- 172.21.15.77:0/3398624848 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f879800cd80 con 0x7f87a0103fc0 2024-09-10T10:03:43.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.225+0000 7f879e000640 1 -- 172.21.15.77:0/3398624848 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8798011950 con 0x7f87a0103fc0 2024-09-10T10:03:43.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.225+0000 7f87a8400640 1 -- 172.21.15.77:0/3398624848 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87a019f8f0 con 0x7f87a0103fc0 2024-09-10T10:03:43.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.225+0000 7f879e000640 1 -- 172.21.15.77:0/3398624848 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87980109e0 con 0x7f87a0103fc0 2024-09-10T10:03:43.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.225+0000 7f87a8400640 1 -- 172.21.15.77:0/3398624848 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87a01a71d0 con 0x7f87a0103fc0 2024-09-10T10:03:43.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.229+0000 7f87aae82740 1 Processor -- start 2024-09-10T10:03:43.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.229+0000 7f879e000640 1 -- 172.21.15.77:0/3398624848 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8798011b00 con 0x7f87a0103fc0 2024-09-10T10:03:43.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.229+0000 7f87aae82740 1 -- start start 2024-09-10T10:03:43.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.230+0000 7f87aae82740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5607fa3583b0 0x5607fa358790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:43.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.230+0000 7f87aae82740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5607fa358cd0 0x5607fa363050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:43.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.230+0000 7f87aae82740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5607fa363590 0x5607fa365980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:43.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.230+0000 7f87aae82740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5607fa2be530 con 0x5607fa3583b0 2024-09-10T10:03:43.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.230+0000 7f87aae82740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5607fa265860 con 0x5607fa363590 2024-09-10T10:03:43.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.230+0000 7f87aae82740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5607fa2ba4f0 con 0x5607fa358cd0 2024-09-10T10:03:43.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.230+0000 7f87a5a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5607fa358cd0 0x5607fa363050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:43.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.230+0000 7f87a5a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5607fa358cd0 0x5607fa363050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:56432/0 (socket says 172.21.15.77:56432) 2024-09-10T10:03:43.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.230+0000 7f87a5a00640 1 -- 172.21.15.77:0/2349083418 learned_addr learned my addr 172.21.15.77:0/2349083418 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:03:43.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.230+0000 7f87a5000640 1 --2- 172.21.15.77:0/2349083418 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5607fa363590 0x5607fa365980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:43.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.231+0000 7f87a5a00640 1 -- 172.21.15.77:0/2349083418 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5607fa363590 msgr2=0x5607fa365980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:43.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.231+0000 7f87a5a00640 1 --2- 172.21.15.77:0/2349083418 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5607fa363590 0x5607fa365980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:43.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.231+0000 7f87a5a00640 1 -- 172.21.15.77:0/2349083418 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5607fa3583b0 msgr2=0x5607fa358790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:43.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.231+0000 7f87a5a00640 1 --2- 172.21.15.77:0/2349083418 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5607fa3583b0 0x5607fa358790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:43.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.231+0000 7f87a5a00640 1 -- 172.21.15.77:0/2349083418 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607fa02fb90 con 0x5607fa358cd0 2024-09-10T10:03:43.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.231+0000 7f87a5000640 1 --2- 172.21.15.77:0/2349083418 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5607fa363590 0x5607fa365980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:03:43.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.231+0000 7f87a5a00640 1 --2- 172.21.15.77:0/2349083418 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5607fa358cd0 0x5607fa363050 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f87980d5c20 tx=0x7f87980d5d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83b9b8651d0d4edb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:43.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.231+0000 7f8792a00640 1 -- 172.21.15.77:0/2349083418 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87980de070 con 0x5607fa358cd0 2024-09-10T10:03:43.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.231+0000 7f879e000640 1 --2- 172.21.15.77:0/3398624848 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f87880a2c00 0x7f87880a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:43.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.231+0000 7f8792a00640 1 -- 172.21.15.77:0/2349083418 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87980d7c90 con 0x5607fa358cd0 2024-09-10T10:03:43.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.231+0000 7f8792a00640 1 -- 172.21.15.77:0/2349083418 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87980d96f0 con 0x5607fa358cd0 2024-09-10T10:03:43.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.232+0000 7f87a6400640 1 --2- 172.21.15.77:0/3398624848 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f87880a2c00 0x7f87880a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:43.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.232+0000 7f879e000640 1 -- 172.21.15.77:0/3398624848 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f87980cf9a0 con 0x7f87a0103fc0 2024-09-10T10:03:43.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.232+0000 7f87aae82740 1 -- 172.21.15.77:0/2349083418 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5607fa358cd0 msgr2=0x5607fa363050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:43.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.232+0000 7f87aae82740 1 --2- 172.21.15.77:0/2349083418 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5607fa358cd0 0x5607fa363050 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f87980d5c20 tx=0x7f87980d5d10 comp rx=0 tx=0).stop 2024-09-10T10:03:43.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.233+0000 7f87aae82740 1 -- 172.21.15.77:0/2349083418 shutdown_connections 2024-09-10T10:03:43.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.233+0000 7f87aae82740 1 --2- 172.21.15.77:0/2349083418 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5607fa363590 0x5607fa365980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:43.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.233+0000 7f87aae82740 1 --2- 172.21.15.77:0/2349083418 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5607fa358cd0 0x5607fa363050 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:43.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.233+0000 7f87aae82740 1 --2- 172.21.15.77:0/2349083418 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5607fa3583b0 0x5607fa358790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:43.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.233+0000 7f87aae82740 1 -- 172.21.15.77:0/2349083418 >> 172.21.15.77:0/2349083418 conn(0x5607fa354940 msgr2=0x5607fa354d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:43.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.233+0000 7f87aae82740 1 -- 172.21.15.77:0/2349083418 shutdown_connections 2024-09-10T10:03:43.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.233+0000 7f87a6400640 1 --2- 172.21.15.77:0/3398624848 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f87880a2c00 0x7f87880a50c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f8794009c30 tx=0x7f87940094d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:43.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.233+0000 7f87aae82740 1 -- 172.21.15.77:0/2349083418 wait complete. 2024-09-10T10:03:43.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.234+0000 7f87aae82740 1 -- start start 2024-09-10T10:03:43.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.234+0000 7f87aae82740 1 Processor -- start 2024-09-10T10:03:43.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.235+0000 7f87aae82740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5607fa3583b0 0x5607fa3606b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:43.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.235+0000 7f87aae82740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5607fa358cd0 0x5607fa360bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:43.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.235+0000 7f87aae82740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5607fa363590 0x5607fa361130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:43.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.235+0000 7f87aae82740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5607fa365fb0 con 0x5607fa358cd0 2024-09-10T10:03:43.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.235+0000 7f87aae82740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5607fa278e60 con 0x5607fa363590 2024-09-10T10:03:43.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.235+0000 7f87aae82740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5607fa366130 con 0x5607fa3583b0 2024-09-10T10:03:43.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.235+0000 7f87aae82740 10 client.0 _ll_register_callbacks cb 0x5607fa27dcc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:03:43.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.235+0000 7f87a5a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5607fa3583b0 0x5607fa3606b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:43.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.235+0000 7f87a5a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5607fa3583b0 0x5607fa3606b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:56434/0 (socket says 172.21.15.77:56434) 2024-09-10T10:03:43.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.235+0000 7f87a5a00640 1 -- 172.21.15.77:0/783502315 learned_addr learned my addr 172.21.15.77:0/783502315 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:03:43.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.236+0000 7f87a6400640 1 --2- 172.21.15.77:0/783502315 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5607fa363590 0x5607fa361130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:43.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.235+0000 7f87a5000640 1 --2- 172.21.15.77:0/783502315 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5607fa358cd0 0x5607fa360bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:43.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.236+0000 7f87a6400640 1 -- 172.21.15.77:0/783502315 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5607fa3583b0 msgr2=0x5607fa3606b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:43.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.236+0000 7f87a6400640 1 --2- 172.21.15.77:0/783502315 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5607fa3583b0 0x5607fa3606b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:43.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.236+0000 7f87a6400640 1 -- 172.21.15.77:0/783502315 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5607fa358cd0 msgr2=0x5607fa360bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:43.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.236+0000 7f87a6400640 1 --2- 172.21.15.77:0/783502315 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5607fa358cd0 0x5607fa360bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:43.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.236+0000 7f87a6400640 1 -- 172.21.15.77:0/783502315 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607fa284990 con 0x5607fa363590 2024-09-10T10:03:43.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.236+0000 7f87a5000640 1 --2- 172.21.15.77:0/783502315 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5607fa358cd0 0x5607fa360bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:03:43.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.236+0000 7f87a5a00640 1 --2- 172.21.15.77:0/783502315 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5607fa3583b0 0x5607fa3606b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:03:43.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.236+0000 7f87a6400640 1 --2- 172.21.15.77:0/783502315 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5607fa363590 0x5607fa361130 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f8794011ca0 tx=0x7f8794005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=593f9ddbda2c97bd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:43.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.236+0000 7f8787400640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T10:03:43.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.237+0000 7f8787400640 1 -- 172.21.15.77:0/783502315 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8794004220 con 0x5607fa363590 2024-09-10T10:03:43.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.237+0000 7f8787400640 1 -- 172.21.15.77:0/783502315 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f87940043c0 con 0x5607fa363590 2024-09-10T10:03:43.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.237+0000 7f87aae82740 1 -- 172.21.15.77:0/783502315 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87980d7c90 con 0x5607fa363590 2024-09-10T10:03:43.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.237+0000 7f8787400640 1 -- 172.21.15.77:0/783502315 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f8794030050 con 0x5607fa363590 2024-09-10T10:03:43.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.237+0000 7f87aae82740 10 client.27233 fetch_fsmap learned FSMap version 35 2024-09-10T10:03:43.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.237+0000 7f87aae82740 1 -- 172.21.15.77:0/783502315 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5607fa37a610 con 0x5607fa363590 2024-09-10T10:03:43.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.237+0000 7f8787400640 1 -- 172.21.15.77:0/783502315 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87980d7c90 con 0x5607fa363590 2024-09-10T10:03:43.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.238+0000 7f8787400640 1 -- 172.21.15.77:0/783502315 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f8794003a30 con 0x5607fa363590 2024-09-10T10:03:43.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.238+0000 7f87aae82740 10 client.27233 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T10:03:43.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.238+0000 7f87aae82740 10 client.27233 resolve_mds: validated gid 24563 aka daemon mds.b 2024-09-10T10:03:43.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.238+0000 7f87aae82740 20 client.27233 populate_metadata read hostname 'smithi077' 2024-09-10T10:03:43.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.238+0000 7f87aae82740 1 --2- 172.21.15.77:0/783502315 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x5607fa37bad0 0x5607fa37dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:43.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.238+0000 7f87aae82740 4 client.27233 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:03:43.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.238+0000 7f87aae82740 1 -- 172.21.15.77:0/783502315 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5607fa261310 con 0x5607fa37bad0 2024-09-10T10:03:43.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.238+0000 7f87a5a00640 1 --2- 172.21.15.77:0/783502315 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x5607fa37bad0 0x5607fa37dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:43.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.239+0000 7f87a5a00640 1 --2- 172.21.15.77:0/783502315 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x5607fa37bad0 0x5607fa37dec0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=774f8d3e32479232 server_cookie=effbc4413437c616 in_seq=0 out_seq=0 2024-09-10T10:03:43.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.239+0000 7f8787400640 10 client.27233 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T10:03:43.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.239+0000 7f8787400640 1 -- 172.21.15.77:0/783502315 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5607fa261310 con 0x5607fa37bad0 2024-09-10T10:03:43.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.239+0000 7f8787400640 10 client.27233 handle_command_reply: tid=0 2024-09-10T10:03:43.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.240+0000 7f87aae82740 1 client.27233 shutdown 2024-09-10T10:03:43.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.240+0000 7f87aae82740 1 -- 172.21.15.77:0/783502315 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5607fa363590 msgr2=0x5607fa361130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:43.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.240+0000 7f87aae82740 1 --2- 172.21.15.77:0/783502315 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5607fa363590 0x5607fa361130 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f8794011ca0 tx=0x7f8794005f90 comp rx=0 tx=0).stop 2024-09-10T10:03:43.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.240+0000 7f87aae82740 1 -- 172.21.15.77:0/783502315 shutdown_connections 2024-09-10T10:03:43.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.240+0000 7f87aae82740 1 --2- 172.21.15.77:0/783502315 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x5607fa37bad0 0x5607fa37dec0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:43.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.240+0000 7f87aae82740 1 --2- 172.21.15.77:0/783502315 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5607fa363590 0x5607fa361130 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:43.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.240+0000 7f87aae82740 1 --2- 172.21.15.77:0/783502315 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5607fa358cd0 0x5607fa360bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:43.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.240+0000 7f8787400640 0 client.27233 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T10:03:43.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.240+0000 7f87aae82740 1 --2- 172.21.15.77:0/783502315 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5607fa3583b0 0x5607fa3606b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:43.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.240+0000 7f87aae82740 1 -- 172.21.15.77:0/783502315 >> 172.21.15.77:0/783502315 conn(0x5607fa354940 msgr2=0x5607fa353c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:43.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.240+0000 7f87aae82740 1 -- 172.21.15.77:0/783502315 shutdown_connections 2024-09-10T10:03:43.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.240+0000 7f87aae82740 1 -- 172.21.15.77:0/783502315 wait complete. 2024-09-10T10:03:43.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.240+0000 7f87aae82740 20 client.27233 trim_cache size 0 max 16384 2024-09-10T10:03:43.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.259+0000 7f87aae82740 1 Processor -- start 2024-09-10T10:03:43.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.259+0000 7f87aae82740 1 -- start start 2024-09-10T10:03:43.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.259+0000 7f87aae82740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5607fa3583b0 0x5607fa360f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:43.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.259+0000 7f87aae82740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5607fa358cd0 0x5607fa383720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:43.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.259+0000 7f87aae82740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5607fa363590 0x5607fa383c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:43.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.259+0000 7f87aae82740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f87a01acce0 con 0x5607fa3583b0 2024-09-10T10:03:43.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.259+0000 7f87aae82740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f87a01ae820 con 0x5607fa358cd0 2024-09-10T10:03:43.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.259+0000 7f87aae82740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f87a01ad3b0 con 0x5607fa363590 2024-09-10T10:03:43.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.260+0000 7f87a5000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5607fa358cd0 0x5607fa383720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:43.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.260+0000 7f87a5a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5607fa3583b0 0x5607fa360f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:43.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.260+0000 7f87a5000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5607fa358cd0 0x5607fa383720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:53854/0 (socket says 172.21.15.77:53854) 2024-09-10T10:03:43.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.260+0000 7f87a5000640 1 -- 172.21.15.77:0/1358278002 learned_addr learned my addr 172.21.15.77:0/1358278002 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:03:43.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.260+0000 7f87a6400640 1 --2- 172.21.15.77:0/1358278002 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5607fa363590 0x5607fa383c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:43.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.260+0000 7f87a5a00640 1 -- 172.21.15.77:0/1358278002 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5607fa363590 msgr2=0x5607fa383c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:43.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.260+0000 7f87a5a00640 1 --2- 172.21.15.77:0/1358278002 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5607fa363590 0x5607fa383c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:43.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.260+0000 7f87a5a00640 1 -- 172.21.15.77:0/1358278002 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5607fa358cd0 msgr2=0x5607fa383720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:43.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.260+0000 7f87a5a00640 1 --2- 172.21.15.77:0/1358278002 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5607fa358cd0 0x5607fa383720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:43.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.260+0000 7f87a5a00640 1 -- 172.21.15.77:0/1358278002 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607fa350380 con 0x5607fa3583b0 2024-09-10T10:03:43.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.260+0000 7f87a5000640 1 --2- 172.21.15.77:0/1358278002 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5607fa358cd0 0x5607fa383720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:03:43.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.260+0000 7f87a6400640 1 --2- 172.21.15.77:0/1358278002 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5607fa363590 0x5607fa383c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:03:43.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.261+0000 7f87a5a00640 1 --2- 172.21.15.77:0/1358278002 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5607fa3583b0 0x5607fa360f80 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f8798098210 tx=0x7f87980d9380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a157f554a604e18 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:43.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.261+0000 7f8792000640 1 -- 172.21.15.77:0/1358278002 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87980de040 con 0x5607fa3583b0 2024-09-10T10:03:43.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.261+0000 7f8792000640 1 -- 172.21.15.77:0/1358278002 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f879809f020 con 0x5607fa3583b0 2024-09-10T10:03:43.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.261+0000 7f8792000640 1 -- 172.21.15.77:0/1358278002 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f879809c4b0 con 0x5607fa3583b0 2024-09-10T10:03:43.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.262+0000 7f87aae82740 1 -- 172.21.15.77:0/1358278002 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5607fa3583b0 msgr2=0x5607fa360f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:43.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.262+0000 7f87aae82740 1 --2- 172.21.15.77:0/1358278002 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5607fa3583b0 0x5607fa360f80 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f8798098210 tx=0x7f87980d9380 comp rx=0 tx=0).stop 2024-09-10T10:03:43.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.262+0000 7f87aae82740 1 -- 172.21.15.77:0/1358278002 shutdown_connections 2024-09-10T10:03:43.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.262+0000 7f87aae82740 1 --2- 172.21.15.77:0/1358278002 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5607fa363590 0x5607fa383c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:43.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.262+0000 7f87aae82740 1 --2- 172.21.15.77:0/1358278002 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5607fa358cd0 0x5607fa383720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:43.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.262+0000 7f87aae82740 1 --2- 172.21.15.77:0/1358278002 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5607fa3583b0 0x5607fa360f80 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:43.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.262+0000 7f87aae82740 1 -- 172.21.15.77:0/1358278002 >> 172.21.15.77:0/1358278002 conn(0x5607fa354940 msgr2=0x5607fa353c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:43.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.262+0000 7f87aae82740 1 -- 172.21.15.77:0/1358278002 shutdown_connections 2024-09-10T10:03:43.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.263+0000 7f87aae82740 1 -- 172.21.15.77:0/1358278002 wait complete. 2024-09-10T10:03:43.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.263+0000 7f87aae82740 1 -- start start 2024-09-10T10:03:43.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.263+0000 7f87aae82740 1 Processor -- start 2024-09-10T10:03:43.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.264+0000 7f87aae82740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5607fa3583b0 0x5607fa373c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:43.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.264+0000 7f87aae82740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5607fa358cd0 0x5607fa3741a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:43.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.264+0000 7f87aae82740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5607fa363590 0x5607fa370c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:43.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.264+0000 7f87aae82740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f87a01ae150 con 0x5607fa363590 2024-09-10T10:03:43.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.264+0000 7f87aae82740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f87a01ada80 con 0x5607fa358cd0 2024-09-10T10:03:43.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.264+0000 7f87aae82740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f87a01aacc0 con 0x5607fa3583b0 2024-09-10T10:03:43.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.264+0000 7f87aae82740 10 client.0 _ll_register_callbacks cb 0x7f8794009540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:03:43.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.264+0000 7f87a5000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5607fa358cd0 0x5607fa3741a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:43.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.264+0000 7f87a5000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5607fa358cd0 0x5607fa3741a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:53862/0 (socket says 172.21.15.77:53862) 2024-09-10T10:03:43.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.264+0000 7f87a5000640 1 -- 172.21.15.77:0/2501730721 learned_addr learned my addr 172.21.15.77:0/2501730721 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:03:43.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.265+0000 7f87a5a00640 1 --2- 172.21.15.77:0/2501730721 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5607fa3583b0 0x5607fa373c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:43.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.265+0000 7f87a6400640 1 --2- 172.21.15.77:0/2501730721 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5607fa363590 0x5607fa370c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:43.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.265+0000 7f87a5000640 1 -- 172.21.15.77:0/2501730721 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5607fa3583b0 msgr2=0x5607fa373c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:43.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.265+0000 7f87a5000640 1 --2- 172.21.15.77:0/2501730721 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5607fa3583b0 0x5607fa373c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:43.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.265+0000 7f87a5000640 1 -- 172.21.15.77:0/2501730721 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5607fa363590 msgr2=0x5607fa370c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:43.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.265+0000 7f87a5000640 1 --2- 172.21.15.77:0/2501730721 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5607fa363590 0x5607fa370c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:43.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.265+0000 7f87a5000640 1 -- 172.21.15.77:0/2501730721 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607fa374830 con 0x5607fa358cd0 2024-09-10T10:03:43.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.266+0000 7f87a5a00640 1 --2- 172.21.15.77:0/2501730721 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5607fa3583b0 0x5607fa373c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:03:43.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.266+0000 7f87a5000640 1 --2- 172.21.15.77:0/2501730721 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5607fa358cd0 0x5607fa3741a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f878c009900 tx=0x7f878c0045e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75169df7eb615209 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:43.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.266+0000 7f8787400640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T10:03:43.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.266+0000 7f8787400640 1 -- 172.21.15.77:0/2501730721 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f878c035040 con 0x5607fa358cd0 2024-09-10T10:03:43.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.266+0000 7f8787400640 1 -- 172.21.15.77:0/2501730721 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f878c004740 con 0x5607fa358cd0 2024-09-10T10:03:43.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.267+0000 7f87aae82740 1 -- 172.21.15.77:0/2501730721 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f879809f020 con 0x5607fa358cd0 2024-09-10T10:03:43.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.267+0000 7f8787400640 1 -- 172.21.15.77:0/2501730721 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f878c005610 con 0x5607fa358cd0 2024-09-10T10:03:43.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.267+0000 7f87aae82740 10 client.27239 fetch_fsmap learned FSMap version 35 2024-09-10T10:03:43.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.267+0000 7f87aae82740 1 -- 172.21.15.77:0/2501730721 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5607fa3e6ca0 con 0x5607fa358cd0 2024-09-10T10:03:43.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.267+0000 7f8787400640 1 -- 172.21.15.77:0/2501730721 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f879809f020 con 0x5607fa358cd0 2024-09-10T10:03:43.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.268+0000 7f8787400640 1 -- 172.21.15.77:0/2501730721 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f878c044020 con 0x5607fa358cd0 2024-09-10T10:03:43.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.268+0000 7f87aae82740 10 client.27239 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T10:03:43.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.268+0000 7f87aae82740 10 client.27239 resolve_mds: validated gid 24563 aka daemon mds.b 2024-09-10T10:03:43.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.268+0000 7f87aae82740 20 client.27239 populate_metadata read hostname 'smithi077' 2024-09-10T10:03:43.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.268+0000 7f87aae82740 1 --2- 172.21.15.77:0/2501730721 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x5607fa37bad0 0x5607fa3e94b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:43.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.268+0000 7f87aae82740 4 client.27239 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 133.4, "await_for": 45.0}] 2024-09-10T10:03:43.272 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.268+0000 7f87aae82740 1 -- 172.21.15.77:0/2501730721 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 133.4, "await_for": 45.0}) -- 0x5607fa2cbc60 con 0x5607fa37bad0 2024-09-10T10:03:43.272 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.268+0000 7f87a5a00640 1 --2- 172.21.15.77:0/2501730721 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x5607fa37bad0 0x5607fa3e94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:43.272 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.269+0000 7f8787400640 10 client.27239 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T10:03:43.272 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.270+0000 7f87a5a00640 1 --2- 172.21.15.77:0/2501730721 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x5607fa37bad0 0x5607fa3e94b0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6df9f4b5327b145f server_cookie=48036efa12c952bb in_seq=0 out_seq=0 2024-09-10T10:03:43.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:03:42 smithi134 ceph-mon[35674]: pgmap v647: 129 pgs: 129 active+clean; 154 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 1.0 MiB/s wr, 417 op/s 2024-09-10T10:03:43.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:03:42 smithi120 ceph-mon[34770]: pgmap v647: 129 pgs: 129 active+clean; 154 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 1.0 MiB/s wr, 417 op/s 2024-09-10T10:03:43.657 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T10:03:43.657 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T10:03:43.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.872+0000 7f8787400640 1 -- 172.21.15.77:0/2501730721 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5607fa2cbc60 con 0x5607fa37bad0 2024-09-10T10:03:43.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.872+0000 7f8787400640 10 client.27239 handle_command_reply: tid=0 2024-09-10T10:03:43.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.872+0000 7f87aae82740 1 client.27239 shutdown 2024-09-10T10:03:43.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.872+0000 7f87aae82740 1 -- 172.21.15.77:0/2501730721 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5607fa358cd0 msgr2=0x5607fa3741a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:43.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.872+0000 7f87aae82740 1 --2- 172.21.15.77:0/2501730721 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5607fa358cd0 0x5607fa3741a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f878c009900 tx=0x7f878c0045e0 comp rx=0 tx=0).stop 2024-09-10T10:03:43.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.872+0000 7f87aae82740 1 -- 172.21.15.77:0/2501730721 shutdown_connections 2024-09-10T10:03:43.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.872+0000 7f87aae82740 1 --2- 172.21.15.77:0/2501730721 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x5607fa37bad0 0x5607fa3e94b0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:43.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.872+0000 7f87aae82740 1 --2- 172.21.15.77:0/2501730721 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5607fa363590 0x5607fa370c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:43.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.872+0000 7f87aae82740 1 --2- 172.21.15.77:0/2501730721 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5607fa358cd0 0x5607fa3741a0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:43.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.872+0000 7f87aae82740 1 --2- 172.21.15.77:0/2501730721 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5607fa3583b0 0x5607fa373c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:43.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.872+0000 7f87aae82740 1 -- 172.21.15.77:0/2501730721 >> 172.21.15.77:0/2501730721 conn(0x5607fa354940 msgr2=0x5607fa376ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:43.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.873+0000 7f8787400640 0 client.27239 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T10:03:43.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.873+0000 7f87aae82740 1 -- 172.21.15.77:0/2501730721 shutdown_connections 2024-09-10T10:03:43.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.873+0000 7f87aae82740 1 -- 172.21.15.77:0/2501730721 wait complete. 2024-09-10T10:03:43.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.873+0000 7f87aae82740 20 client.27239 trim_cache size 0 max 16384 2024-09-10T10:03:43.876 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-10T10:03:43.876 INFO:teuthology.orchestra.run.smithi077.stdout: "epoch": 35, 2024-09-10T10:03:43.876 INFO:teuthology.orchestra.run.smithi077.stdout: "leader": 24563, 2024-09-10T10:03:43.877 INFO:teuthology.orchestra.run.smithi077.stdout: "set_version": 26, 2024-09-10T10:03:43.877 INFO:teuthology.orchestra.run.smithi077.stdout: "sets": { 2024-09-10T10:03:43.877 INFO:teuthology.orchestra.run.smithi077.stdout: "1f5021c0": { 2024-09-10T10:03:43.877 INFO:teuthology.orchestra.run.smithi077.stdout: "version": 26, 2024-09-10T10:03:43.877 INFO:teuthology.orchestra.run.smithi077.stdout: "age_ref": 0.0, 2024-09-10T10:03:43.877 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-10T10:03:43.877 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "QUIESCED", 2024-09-10T10:03:43.877 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-10T10:03:43.877 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:03:43.877 INFO:teuthology.orchestra.run.smithi077.stdout: "timeout": 90.0, 2024-09-10T10:03:43.877 INFO:teuthology.orchestra.run.smithi077.stdout: "expiration": 133.4, 2024-09-10T10:03:43.877 INFO:teuthology.orchestra.run.smithi077.stdout: "members": { 2024-09-10T10:03:43.877 INFO:teuthology.orchestra.run.smithi077.stdout: "file:/": { 2024-09-10T10:03:43.878 INFO:teuthology.orchestra.run.smithi077.stdout: "excluded": false, 2024-09-10T10:03:43.878 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-10T10:03:43.878 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "QUIESCED", 2024-09-10T10:03:43.878 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-10T10:03:43.878 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:03:43.878 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:03:43.878 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:03:43.878 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:03:43.878 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:03:43.878 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-10T10:03:43.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.874+0000 7f87a8400640 1 -- 172.21.15.77:0/3398624848 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f87880a2c00 msgr2=0x7f87880a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:43.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.874+0000 7f87a8400640 1 --2- 172.21.15.77:0/3398624848 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f87880a2c00 0x7f87880a50c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f8794009c30 tx=0x7f87940094d0 comp rx=0 tx=0).stop 2024-09-10T10:03:43.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.874+0000 7f87a8400640 1 -- 172.21.15.77:0/3398624848 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f87a0103fc0 msgr2=0x7f87a019a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:43.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.874+0000 7f87a8400640 1 --2- 172.21.15.77:0/3398624848 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f87a0103fc0 0x7f87a019a650 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f879800eab0 tx=0x7f879800eeb0 comp rx=0 tx=0).stop 2024-09-10T10:03:43.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.875+0000 7f87a8400640 1 -- 172.21.15.77:0/3398624848 shutdown_connections 2024-09-10T10:03:43.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.875+0000 7f87a8400640 1 --2- 172.21.15.77:0/3398624848 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f87880a2c00 0x7f87880a50c0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:43.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.875+0000 7f87a8400640 1 --2- 172.21.15.77:0/3398624848 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f87a01090f0 0x7f87a019ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:43.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.875+0000 7f87a8400640 1 --2- 172.21.15.77:0/3398624848 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87a0104970 0x7f87a019ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:43.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.875+0000 7f87a8400640 1 --2- 172.21.15.77:0/3398624848 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f87a0103fc0 0x7f87a019a650 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:43.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.875+0000 7f87a8400640 1 -- 172.21.15.77:0/3398624848 >> 172.21.15.77:0/3398624848 conn(0x7f87a00ffa20 msgr2=0x7f87a0100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:43.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.876+0000 7f87a8400640 1 -- 172.21.15.77:0/3398624848 shutdown_connections 2024-09-10T10:03:43.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:43.876+0000 7f87a8400640 1 -- 172.21.15.77:0/3398624848 wait complete. 2024-09-10T10:03:44.655 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:03:44.684 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '1f5021c0', quiesce took 7.2 seconds. Will release after: 6.2 2024-09-10T10:03:45.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:03:44 smithi077 ceph-mon[30319]: pgmap v648: 129 pgs: 129 active+clean; 148 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 1.1 MiB/s wr, 419 op/s 2024-09-10T10:03:45.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.219+0000 7f8386600640 1 -- 172.21.15.77:0/2670165310 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f83801047e0 msgr2=0x7f8380108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:45.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.219+0000 7f8386600640 1 --2- 172.21.15.77:0/2670165310 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f83801047e0 0x7f8380108830 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f836c009900 tx=0x7f836c02a3f0 comp rx=0 tx=0).stop 2024-09-10T10:03:45.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.219+0000 7f8386600640 1 -- 172.21.15.77:0/2670165310 shutdown_connections 2024-09-10T10:03:45.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.219+0000 7f8386600640 1 --2- 172.21.15.77:0/2670165310 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8380108f60 0x7f8380110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:45.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.219+0000 7f8386600640 1 --2- 172.21.15.77:0/2670165310 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f83801047e0 0x7f8380108830 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:45.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.219+0000 7f8386600640 1 --2- 172.21.15.77:0/2670165310 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8380103e30 0x7f8380104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:45.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.219+0000 7f8386600640 1 -- 172.21.15.77:0/2670165310 >> 172.21.15.77:0/2670165310 conn(0x7f83800ff910 msgr2=0x7f8380101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:45.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.219+0000 7f8386600640 1 -- 172.21.15.77:0/2670165310 shutdown_connections 2024-09-10T10:03:45.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.219+0000 7f8386600640 1 -- 172.21.15.77:0/2670165310 wait complete. 2024-09-10T10:03:45.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.220+0000 7f8386600640 1 Processor -- start 2024-09-10T10:03:45.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.220+0000 7f8386600640 1 -- start start 2024-09-10T10:03:45.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.220+0000 7f8386600640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8380103e30 0x7f838019a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:45.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.221+0000 7f8386600640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f83801047e0 0x7f838019a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:45.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.221+0000 7f8386600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8380108f60 0x7f838019ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:45.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.221+0000 7f8386600640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8380113fc0 con 0x7f8380103e30 2024-09-10T10:03:45.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.221+0000 7f8386600640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8380113e40 con 0x7f83801047e0 2024-09-10T10:03:45.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.221+0000 7f8386600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8380114140 con 0x7f8380108f60 2024-09-10T10:03:45.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.221+0000 7f8385200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8380103e30 0x7f838019a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:45.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.221+0000 7f8385200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8380103e30 0x7f838019a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:38180/0 (socket says 172.21.15.77:38180) 2024-09-10T10:03:45.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.221+0000 7f8385200640 1 -- 172.21.15.77:0/2286838672 learned_addr learned my addr 172.21.15.77:0/2286838672 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:03:45.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.221+0000 7f837fe00640 1 --2- 172.21.15.77:0/2286838672 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f83801047e0 0x7f838019a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:45.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.221+0000 7f8385c00640 1 --2- 172.21.15.77:0/2286838672 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8380108f60 0x7f838019ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:45.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.221+0000 7f8385200640 1 -- 172.21.15.77:0/2286838672 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8380108f60 msgr2=0x7f838019ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:45.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.221+0000 7f8385200640 1 --2- 172.21.15.77:0/2286838672 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8380108f60 0x7f838019ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:45.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.221+0000 7f8385200640 1 -- 172.21.15.77:0/2286838672 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f83801047e0 msgr2=0x7f838019a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:45.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.221+0000 7f8385200640 1 --2- 172.21.15.77:0/2286838672 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f83801047e0 0x7f838019a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:45.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.221+0000 7f8385200640 1 -- 172.21.15.77:0/2286838672 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f838019f410 con 0x7f8380103e30 2024-09-10T10:03:45.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.221+0000 7f8385c00640 1 --2- 172.21.15.77:0/2286838672 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8380108f60 0x7f838019ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:03:45.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.221+0000 7f837fe00640 1 --2- 172.21.15.77:0/2286838672 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f83801047e0 0x7f838019a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:03:45.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.222+0000 7f8385200640 1 --2- 172.21.15.77:0/2286838672 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8380103e30 0x7f838019a460 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f837400e9f0 tx=0x7f837400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:45.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.222+0000 7f837d600640 1 -- 172.21.15.77:0/2286838672 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f837400cd80 con 0x7f8380103e30 2024-09-10T10:03:45.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.222+0000 7f837d600640 1 -- 172.21.15.77:0/2286838672 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8374010910 con 0x7f8380103e30 2024-09-10T10:03:45.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.222+0000 7f837d600640 1 -- 172.21.15.77:0/2286838672 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8374018990 con 0x7f8380103e30 2024-09-10T10:03:45.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.222+0000 7f8386600640 1 -- 172.21.15.77:0/2286838672 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f838019f700 con 0x7f8380103e30 2024-09-10T10:03:45.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.222+0000 7f8386600640 1 -- 172.21.15.77:0/2286838672 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83801a6fe0 con 0x7f8380103e30 2024-09-10T10:03:45.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.225+0000 7f837d600640 1 -- 172.21.15.77:0/2286838672 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8374010ac0 con 0x7f8380103e30 2024-09-10T10:03:45.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.225+0000 7f838a81e740 1 Processor -- start 2024-09-10T10:03:45.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.225+0000 7f838a81e740 1 -- start start 2024-09-10T10:03:45.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.226+0000 7f838a81e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b6480f6cb0 0x55b6480f7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:45.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.226+0000 7f838a81e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b6480f75d0 0x55b648101950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:45.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.226+0000 7f838a81e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b648101e90 0x55b648104280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:45.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.226+0000 7f838a81e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55b64805d290 con 0x55b648101e90 2024-09-10T10:03:45.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.226+0000 7f838a81e740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b648004b40 con 0x55b6480f6cb0 2024-09-10T10:03:45.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.226+0000 7f838a81e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b648059250 con 0x55b6480f75d0 2024-09-10T10:03:45.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.226+0000 7f837fe00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b648101e90 0x55b648104280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:45.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.226+0000 7f837fe00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b648101e90 0x55b648104280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:38196/0 (socket says 172.21.15.77:38196) 2024-09-10T10:03:45.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.226+0000 7f8385200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b6480f75d0 0x55b648101950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:45.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.226+0000 7f837fe00640 1 -- 172.21.15.77:0/2300492229 learned_addr learned my addr 172.21.15.77:0/2300492229 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:03:45.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.226+0000 7f8385c00640 1 --2- 172.21.15.77:0/2300492229 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b6480f6cb0 0x55b6480f7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:45.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.227+0000 7f837fe00640 1 -- 172.21.15.77:0/2300492229 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b6480f75d0 msgr2=0x55b648101950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:45.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.227+0000 7f837fe00640 1 --2- 172.21.15.77:0/2300492229 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b6480f75d0 0x55b648101950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:45.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.227+0000 7f837fe00640 1 -- 172.21.15.77:0/2300492229 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b6480f6cb0 msgr2=0x55b6480f7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:45.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.227+0000 7f837fe00640 1 --2- 172.21.15.77:0/2300492229 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b6480f6cb0 0x55b6480f7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:45.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.227+0000 7f837fe00640 1 -- 172.21.15.77:0/2300492229 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b647dce6a0 con 0x55b648101e90 2024-09-10T10:03:45.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.227+0000 7f8385c00640 1 --2- 172.21.15.77:0/2300492229 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b6480f6cb0 0x55b6480f7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:03:45.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.227+0000 7f837d600640 1 --2- 172.21.15.77:0/2286838672 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f83680a2c00 0x7f83680a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:45.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.227+0000 7f837fe00640 1 --2- 172.21.15.77:0/2300492229 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b648101e90 0x55b648104280 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f836c009900 tx=0x7f836c005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5423e2e5af7ca564 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:45.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.227+0000 7f837d600640 1 -- 172.21.15.77:0/2286838672 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f83740cf7b0 con 0x7f8380103e30 2024-09-10T10:03:45.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.227+0000 7f8385c00640 1 --2- 172.21.15.77:0/2286838672 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f83680a2c00 0x7f83680a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:45.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.227+0000 7f8372000640 1 -- 172.21.15.77:0/2300492229 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f836c035040 con 0x55b648101e90 2024-09-10T10:03:45.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.228+0000 7f8372000640 1 -- 172.21.15.77:0/2300492229 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f836c031830 con 0x55b648101e90 2024-09-10T10:03:45.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.228+0000 7f8372000640 1 -- 172.21.15.77:0/2300492229 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f836c0047a0 con 0x55b648101e90 2024-09-10T10:03:45.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.228+0000 7f838a81e740 1 -- 172.21.15.77:0/2300492229 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b648101e90 msgr2=0x55b648104280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:45.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.228+0000 7f8385c00640 1 --2- 172.21.15.77:0/2286838672 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f83680a2c00 0x7f83680a50c0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f8378005fd0 tx=0x7f8378009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:45.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.228+0000 7f838a81e740 1 --2- 172.21.15.77:0/2300492229 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b648101e90 0x55b648104280 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f836c009900 tx=0x7f836c005d50 comp rx=0 tx=0).stop 2024-09-10T10:03:45.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.228+0000 7f838a81e740 1 -- 172.21.15.77:0/2300492229 shutdown_connections 2024-09-10T10:03:45.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.228+0000 7f838a81e740 1 --2- 172.21.15.77:0/2300492229 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b648101e90 0x55b648104280 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:45.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.228+0000 7f838a81e740 1 --2- 172.21.15.77:0/2300492229 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b6480f75d0 0x55b648101950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:45.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.228+0000 7f838a81e740 1 --2- 172.21.15.77:0/2300492229 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b6480f6cb0 0x55b6480f7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:45.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.228+0000 7f838a81e740 1 -- 172.21.15.77:0/2300492229 >> 172.21.15.77:0/2300492229 conn(0x55b6480f3240 msgr2=0x55b6480f3620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:45.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.229+0000 7f838a81e740 1 -- 172.21.15.77:0/2300492229 shutdown_connections 2024-09-10T10:03:45.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.229+0000 7f838a81e740 1 -- 172.21.15.77:0/2300492229 wait complete. 2024-09-10T10:03:45.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.229+0000 7f838a81e740 1 -- start start 2024-09-10T10:03:45.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.229+0000 7f838a81e740 1 Processor -- start 2024-09-10T10:03:45.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.230+0000 7f838a81e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b6480f6cb0 0x55b6481017a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:45.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.230+0000 7f838a81e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b6480f75d0 0x55b6480fea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:45.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.230+0000 7f838a81e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b648101e90 0x55b6480fef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:45.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.230+0000 7f838a81e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55b648104b00 con 0x55b6480f6cb0 2024-09-10T10:03:45.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.230+0000 7f838a81e740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b648017fa0 con 0x55b6480f75d0 2024-09-10T10:03:45.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.230+0000 7f838a81e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b648104c80 con 0x55b648101e90 2024-09-10T10:03:45.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.230+0000 7f838a81e740 10 client.0 _ll_register_callbacks cb 0x55b647d6bba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:03:45.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.230+0000 7f8385200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b6480f6cb0 0x55b6481017a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:45.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.230+0000 7f837fe00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b6480f75d0 0x55b6480fea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:45.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.230+0000 7f8385200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b6480f6cb0 0x55b6481017a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:38206/0 (socket says 172.21.15.77:38206) 2024-09-10T10:03:45.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.230+0000 7f837fe00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b6480f75d0 0x55b6480fea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:53890/0 (socket says 172.21.15.77:53890) 2024-09-10T10:03:45.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.230+0000 7f8385200640 1 -- 172.21.15.77:0/3412473102 learned_addr learned my addr 172.21.15.77:0/3412473102 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:03:45.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.230+0000 7f8385c00640 1 --2- 172.21.15.77:0/3412473102 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b648101e90 0x55b6480fef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:45.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.230+0000 7f837fe00640 1 -- 172.21.15.77:0/3412473102 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b648101e90 msgr2=0x55b6480fef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:45.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.230+0000 7f837fe00640 1 --2- 172.21.15.77:0/3412473102 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b648101e90 0x55b6480fef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:45.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.230+0000 7f837fe00640 1 -- 172.21.15.77:0/3412473102 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b6480f6cb0 msgr2=0x55b6481017a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:45.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.230+0000 7f837fe00640 1 --2- 172.21.15.77:0/3412473102 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b6480f6cb0 0x55b6481017a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:45.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.231+0000 7f837fe00640 1 -- 172.21.15.77:0/3412473102 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6480ff790 con 0x55b6480f75d0 2024-09-10T10:03:45.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.231+0000 7f8385200640 1 --2- 172.21.15.77:0/3412473102 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b6480f6cb0 0x55b6481017a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:03:45.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.231+0000 7f837fe00640 1 --2- 172.21.15.77:0/3412473102 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b6480f75d0 0x55b6480fea50 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f836c006010 tx=0x7f836c031a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f1f305d54c631e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:45.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.231+0000 7f8366a00640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T10:03:45.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.231+0000 7f8366a00640 1 -- 172.21.15.77:0/3412473102 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f836c035040 con 0x55b6480f75d0 2024-09-10T10:03:45.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.231+0000 7f838a81e740 1 -- 172.21.15.77:0/3412473102 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f836c031830 con 0x55b6480f75d0 2024-09-10T10:03:45.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.231+0000 7f8366a00640 1 -- 172.21.15.77:0/3412473102 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f836c004250 con 0x55b6480f75d0 2024-09-10T10:03:45.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.231+0000 7f8366a00640 1 -- 172.21.15.77:0/3412473102 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f836c058050 con 0x55b6480f75d0 2024-09-10T10:03:45.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.231+0000 7f838a81e740 10 client.27251 fetch_fsmap learned FSMap version 35 2024-09-10T10:03:45.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.231+0000 7f838a81e740 1 -- 172.21.15.77:0/3412473102 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55b6481109e0 con 0x55b6480f75d0 2024-09-10T10:03:45.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.232+0000 7f8366a00640 1 -- 172.21.15.77:0/3412473102 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f836c031830 con 0x55b6480f75d0 2024-09-10T10:03:45.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.232+0000 7f8366a00640 1 -- 172.21.15.77:0/3412473102 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f836c046570 con 0x55b6480f75d0 2024-09-10T10:03:45.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.232+0000 7f838a81e740 10 client.27251 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T10:03:45.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.232+0000 7f838a81e740 10 client.27251 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T10:03:45.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.232+0000 7f838a81e740 20 client.27251 populate_metadata read hostname 'smithi077' 2024-09-10T10:03:45.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.232+0000 7f838a81e740 1 --2- 172.21.15.77:0/3412473102 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55b648111d30 0x55b648114120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:45.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.232+0000 7f838a81e740 4 client.27251 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:03:45.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.232+0000 7f838a81e740 1 -- 172.21.15.77:0/3412473102 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b64806a800 con 0x55b648111d30 2024-09-10T10:03:45.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.232+0000 7f8385200640 1 --2- 172.21.15.77:0/3412473102 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55b648111d30 0x55b648114120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:45.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.233+0000 7f8385200640 1 --2- 172.21.15.77:0/3412473102 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55b648111d30 0x55b648114120 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed15873a8c6fcbbc server_cookie=a6084cfbbdcdaab2 in_seq=0 out_seq=0 2024-09-10T10:03:45.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.233+0000 7f8366a00640 10 client.27251 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T10:03:45.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.234+0000 7f8366a00640 1 -- 172.21.15.77:0/3412473102 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b64806a800 con 0x55b648111d30 2024-09-10T10:03:45.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.234+0000 7f8366a00640 10 client.27251 handle_command_reply: tid=0 2024-09-10T10:03:45.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.234+0000 7f838a81e740 1 client.27251 shutdown 2024-09-10T10:03:45.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.234+0000 7f838a81e740 1 -- 172.21.15.77:0/3412473102 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b6480f75d0 msgr2=0x55b6480fea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:45.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.234+0000 7f838a81e740 1 --2- 172.21.15.77:0/3412473102 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b6480f75d0 0x55b6480fea50 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f836c006010 tx=0x7f836c031a50 comp rx=0 tx=0).stop 2024-09-10T10:03:45.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.234+0000 7f838a81e740 1 -- 172.21.15.77:0/3412473102 shutdown_connections 2024-09-10T10:03:45.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.234+0000 7f838a81e740 1 --2- 172.21.15.77:0/3412473102 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55b648111d30 0x55b648114120 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:45.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.234+0000 7f838a81e740 1 --2- 172.21.15.77:0/3412473102 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b648101e90 0x55b6480fef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:45.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.234+0000 7f838a81e740 1 --2- 172.21.15.77:0/3412473102 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b6480f75d0 0x55b6480fea50 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:45.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.234+0000 7f838a81e740 1 --2- 172.21.15.77:0/3412473102 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b6480f6cb0 0x55b6481017a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:45.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.234+0000 7f8366a00640 0 client.27251 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T10:03:45.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.234+0000 7f838a81e740 1 -- 172.21.15.77:0/3412473102 >> 172.21.15.77:0/3412473102 conn(0x55b6480f3240 msgr2=0x55b6480f8f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:45.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.234+0000 7f838a81e740 1 -- 172.21.15.77:0/3412473102 shutdown_connections 2024-09-10T10:03:45.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.234+0000 7f838a81e740 1 -- 172.21.15.77:0/3412473102 wait complete. 2024-09-10T10:03:45.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.234+0000 7f838a81e740 20 client.27251 trim_cache size 0 max 16384 2024-09-10T10:03:45.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.252+0000 7f838a81e740 1 Processor -- start 2024-09-10T10:03:45.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.252+0000 7f838a81e740 1 -- start start 2024-09-10T10:03:45.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.252+0000 7f838a81e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b6480f6cb0 0x55b6480f8e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:45.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.252+0000 7f838a81e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b6480f75d0 0x55b6481196c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:45.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.252+0000 7f838a81e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b648101e90 0x55b648119c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:45.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.252+0000 7f838a81e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f83801b0ec0 con 0x55b6480f75d0 2024-09-10T10:03:45.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.252+0000 7f838a81e740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f83801b2a00 con 0x55b648101e90 2024-09-10T10:03:45.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.252+0000 7f838a81e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f83801b1590 con 0x55b6480f6cb0 2024-09-10T10:03:45.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.253+0000 7f837fe00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b6480f75d0 0x55b6481196c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:45.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.253+0000 7f837fe00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b6480f75d0 0x55b6481196c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:38218/0 (socket says 172.21.15.77:38218) 2024-09-10T10:03:45.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.253+0000 7f837fe00640 1 -- 172.21.15.77:0/445353418 learned_addr learned my addr 172.21.15.77:0/445353418 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:03:45.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.253+0000 7f8385c00640 1 --2- 172.21.15.77:0/445353418 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b648101e90 0x55b648119c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:45.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.253+0000 7f8385200640 1 --2- 172.21.15.77:0/445353418 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b6480f6cb0 0x55b6480f8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:45.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.253+0000 7f837fe00640 1 -- 172.21.15.77:0/445353418 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b6480f6cb0 msgr2=0x55b6480f8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:45.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.253+0000 7f837fe00640 1 --2- 172.21.15.77:0/445353418 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b6480f6cb0 0x55b6480f8e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:45.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.253+0000 7f837fe00640 1 -- 172.21.15.77:0/445353418 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b648101e90 msgr2=0x55b648119c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:45.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.253+0000 7f837fe00640 1 --2- 172.21.15.77:0/445353418 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b648101e90 0x55b648119c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:45.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.253+0000 7f837fe00640 1 -- 172.21.15.77:0/445353418 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6480fdfb0 con 0x55b6480f75d0 2024-09-10T10:03:45.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.253+0000 7f8385c00640 1 --2- 172.21.15.77:0/445353418 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b648101e90 0x55b648119c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:03:45.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.253+0000 7f837fe00640 1 --2- 172.21.15.77:0/445353418 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b6480f75d0 0x55b6481196c0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f836c046990 tx=0x7f836c005460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d2ce272a6db104b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:45.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.254+0000 7f8371600640 1 -- 172.21.15.77:0/445353418 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f836c005850 con 0x55b6480f75d0 2024-09-10T10:03:45.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.254+0000 7f8371600640 1 -- 172.21.15.77:0/445353418 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f836c004250 con 0x55b6480f75d0 2024-09-10T10:03:45.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.254+0000 7f8371600640 1 -- 172.21.15.77:0/445353418 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f836c003ad0 con 0x55b6480f75d0 2024-09-10T10:03:45.258 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.254+0000 7f838a81e740 1 -- 172.21.15.77:0/445353418 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b6480f75d0 msgr2=0x55b6481196c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:45.258 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.254+0000 7f838a81e740 1 --2- 172.21.15.77:0/445353418 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b6480f75d0 0x55b6481196c0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f836c046990 tx=0x7f836c005460 comp rx=0 tx=0).stop 2024-09-10T10:03:45.258 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.254+0000 7f838a81e740 1 -- 172.21.15.77:0/445353418 shutdown_connections 2024-09-10T10:03:45.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.254+0000 7f838a81e740 1 --2- 172.21.15.77:0/445353418 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b648101e90 0x55b648119c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:45.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.254+0000 7f838a81e740 1 --2- 172.21.15.77:0/445353418 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b6480f75d0 0x55b6481196c0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:45.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.254+0000 7f838a81e740 1 --2- 172.21.15.77:0/445353418 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b6480f6cb0 0x55b6480f8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:45.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.254+0000 7f838a81e740 1 -- 172.21.15.77:0/445353418 >> 172.21.15.77:0/445353418 conn(0x55b6480f3240 msgr2=0x55b6480f8700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:45.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.254+0000 7f838a81e740 1 -- 172.21.15.77:0/445353418 shutdown_connections 2024-09-10T10:03:45.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.255+0000 7f838a81e740 1 -- 172.21.15.77:0/445353418 wait complete. 2024-09-10T10:03:45.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.255+0000 7f838a81e740 1 -- start start 2024-09-10T10:03:45.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.255+0000 7f838a81e740 1 Processor -- start 2024-09-10T10:03:45.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.256+0000 7f838a81e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b6480f6cb0 0x55b648105ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:45.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.256+0000 7f838a81e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b6480f75d0 0x55b648106420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:45.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.256+0000 7f838a81e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b648101e90 0x55b648106960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:45.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.256+0000 7f838a81e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f83801b2330 con 0x55b6480f75d0 2024-09-10T10:03:45.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.256+0000 7f838a81e740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f83801b1c60 con 0x55b648101e90 2024-09-10T10:03:45.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.256+0000 7f838a81e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f83801aeea0 con 0x55b6480f6cb0 2024-09-10T10:03:45.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.256+0000 7f838a81e740 10 client.0 _ll_register_callbacks cb 0x7f836c040070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:03:45.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.256+0000 7f837fe00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b6480f75d0 0x55b648106420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:45.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.256+0000 7f837fe00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b6480f75d0 0x55b648106420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:38226/0 (socket says 172.21.15.77:38226) 2024-09-10T10:03:45.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.256+0000 7f837fe00640 1 -- 172.21.15.77:0/735504758 learned_addr learned my addr 172.21.15.77:0/735504758 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:03:45.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.256+0000 7f8385200640 1 --2- 172.21.15.77:0/735504758 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b6480f6cb0 0x55b648105ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:45.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.256+0000 7f8385c00640 1 --2- 172.21.15.77:0/735504758 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b648101e90 0x55b648106960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:45.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.256+0000 7f837fe00640 1 -- 172.21.15.77:0/735504758 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b6480f6cb0 msgr2=0x55b648105ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:45.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.256+0000 7f837fe00640 1 --2- 172.21.15.77:0/735504758 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b6480f6cb0 0x55b648105ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:45.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.256+0000 7f837fe00640 1 -- 172.21.15.77:0/735504758 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b648101e90 msgr2=0x55b648106960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:45.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.256+0000 7f837fe00640 1 --2- 172.21.15.77:0/735504758 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b648101e90 0x55b648106960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:45.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.256+0000 7f837fe00640 1 -- 172.21.15.77:0/735504758 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b64817b930 con 0x55b6480f75d0 2024-09-10T10:03:45.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.257+0000 7f837fe00640 1 --2- 172.21.15.77:0/735504758 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b6480f75d0 0x55b648106420 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f836c046b00 tx=0x7f836c005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d004cacd952d2dc4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:45.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.257+0000 7f8366a00640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-10T10:03:45.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.257+0000 7f8366a00640 1 -- 172.21.15.77:0/735504758 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f836c035030 con 0x55b6480f75d0 2024-09-10T10:03:45.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.257+0000 7f8366a00640 1 -- 172.21.15.77:0/735504758 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f836c046650 con 0x55b6480f75d0 2024-09-10T10:03:45.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.257+0000 7f838a81e740 1 -- 172.21.15.77:0/735504758 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f836c004250 con 0x55b6480f75d0 2024-09-10T10:03:45.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.257+0000 7f8366a00640 1 -- 172.21.15.77:0/735504758 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f836c04a040 con 0x55b6480f75d0 2024-09-10T10:03:45.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.257+0000 7f838a81e740 10 client.17109 fetch_fsmap learned FSMap version 35 2024-09-10T10:03:45.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.257+0000 7f838a81e740 1 -- 172.21.15.77:0/735504758 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55b64817cc70 con 0x55b6480f75d0 2024-09-10T10:03:45.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.258+0000 7f8366a00640 1 -- 172.21.15.77:0/735504758 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f836c004250 con 0x55b6480f75d0 2024-09-10T10:03:45.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.258+0000 7f8366a00640 1 -- 172.21.15.77:0/735504758 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f836c003a00 con 0x55b6480f75d0 2024-09-10T10:03:45.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.258+0000 7f838a81e740 10 client.17109 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T10:03:45.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.258+0000 7f838a81e740 10 client.17109 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T10:03:45.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.258+0000 7f838a81e740 20 client.17109 populate_metadata read hostname 'smithi077' 2024-09-10T10:03:45.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.258+0000 7f838a81e740 1 --2- 172.21.15.77:0/735504758 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55b64817d270 0x55b64817f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:45.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.258+0000 7f838a81e740 4 client.17109 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T10:03:45.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.258+0000 7f838a81e740 1 -- 172.21.15.77:0/735504758 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55b6480ff320 con 0x55b64817d270 2024-09-10T10:03:45.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.258+0000 7f8385200640 1 --2- 172.21.15.77:0/735504758 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55b64817d270 0x55b64817f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:45.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.259+0000 7f8385200640 1 --2- 172.21.15.77:0/735504758 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55b64817d270 0x55b64817f690 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83440b04c350cefd server_cookie=a8dde8615e11cb17 in_seq=0 out_seq=0 2024-09-10T10:03:45.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.259+0000 7f8366a00640 10 client.17109 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T10:03:45.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.260+0000 7f8366a00640 1 -- 172.21.15.77:0/735504758 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b6480ff320 con 0x55b64817d270 2024-09-10T10:03:45.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.260+0000 7f8366a00640 10 client.17109 handle_command_reply: tid=0 2024-09-10T10:03:45.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.260+0000 7f838a81e740 1 client.17109 shutdown 2024-09-10T10:03:45.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.260+0000 7f838a81e740 1 -- 172.21.15.77:0/735504758 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b6480f75d0 msgr2=0x55b648106420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:45.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.260+0000 7f838a81e740 1 --2- 172.21.15.77:0/735504758 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b6480f75d0 0x55b648106420 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f836c046b00 tx=0x7f836c005850 comp rx=0 tx=0).stop 2024-09-10T10:03:45.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.260+0000 7f838a81e740 1 -- 172.21.15.77:0/735504758 shutdown_connections 2024-09-10T10:03:45.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.260+0000 7f838a81e740 1 --2- 172.21.15.77:0/735504758 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x55b64817d270 0x55b64817f690 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:45.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.260+0000 7f838a81e740 1 --2- 172.21.15.77:0/735504758 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b648101e90 0x55b648106960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:45.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.260+0000 7f838a81e740 1 --2- 172.21.15.77:0/735504758 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b6480f75d0 0x55b648106420 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:45.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.260+0000 7f838a81e740 1 --2- 172.21.15.77:0/735504758 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b6480f6cb0 0x55b648105ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:45.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.260+0000 7f838a81e740 1 -- 172.21.15.77:0/735504758 >> 172.21.15.77:0/735504758 conn(0x55b6480f3240 msgr2=0x55b6480f2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:45.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.260+0000 7f8366a00640 0 client.17109 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T10:03:45.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.260+0000 7f838a81e740 1 -- 172.21.15.77:0/735504758 shutdown_connections 2024-09-10T10:03:45.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.261+0000 7f838a81e740 1 -- 172.21.15.77:0/735504758 wait complete. 2024-09-10T10:03:45.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.261+0000 7f838a81e740 20 client.17109 trim_cache size 0 max 16384 2024-09-10T10:03:45.265 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-10T10:03:45.265 INFO:teuthology.orchestra.run.smithi077.stdout: "return_code": 0, 2024-09-10T10:03:45.265 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_tag": "bcf9d927-6c1e-4ab2-b6b8-55801c8be666", 2024-09-10T10:03:45.266 INFO:teuthology.orchestra.run.smithi077.stdout: "mode": "asynchronous" 2024-09-10T10:03:45.266 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-10T10:03:45.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.261+0000 7f8386600640 1 -- 172.21.15.77:0/2286838672 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f83680a2c00 msgr2=0x7f83680a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:45.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.261+0000 7f8386600640 1 --2- 172.21.15.77:0/2286838672 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f83680a2c00 0x7f83680a50c0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f8378005fd0 tx=0x7f8378009500 comp rx=0 tx=0).stop 2024-09-10T10:03:45.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.261+0000 7f8386600640 1 -- 172.21.15.77:0/2286838672 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8380103e30 msgr2=0x7f838019a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:45.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.261+0000 7f8386600640 1 --2- 172.21.15.77:0/2286838672 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8380103e30 0x7f838019a460 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f837400e9f0 tx=0x7f837400eeb0 comp rx=0 tx=0).stop 2024-09-10T10:03:45.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.261+0000 7f8386600640 1 -- 172.21.15.77:0/2286838672 shutdown_connections 2024-09-10T10:03:45.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.261+0000 7f8386600640 1 --2- 172.21.15.77:0/2286838672 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f83680a2c00 0x7f83680a50c0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:45.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.261+0000 7f8386600640 1 --2- 172.21.15.77:0/2286838672 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8380108f60 0x7f838019ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:45.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.261+0000 7f8386600640 1 --2- 172.21.15.77:0/2286838672 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f83801047e0 0x7f838019a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:45.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.261+0000 7f8386600640 1 --2- 172.21.15.77:0/2286838672 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8380103e30 0x7f838019a460 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:45.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.261+0000 7f8386600640 1 -- 172.21.15.77:0/2286838672 >> 172.21.15.77:0/2286838672 conn(0x7f83800ff910 msgr2=0x7f8380100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:45.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.262+0000 7f8386600640 1 -- 172.21.15.77:0/2286838672 shutdown_connections 2024-09-10T10:03:45.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:45.262+0000 7f8386600640 1 -- 172.21.15.77:0/2286838672 wait complete. 2024-09-10T10:03:45.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:03:44 smithi134 ceph-mon[35674]: pgmap v648: 129 pgs: 129 active+clean; 148 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 1.1 MiB/s wr, 419 op/s 2024-09-10T10:03:45.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:03:44 smithi120 ceph-mon[34770]: pgmap v648: 129 pgs: 129 active+clean; 148 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 1.1 MiB/s wr, 419 op/s 2024-09-10T10:03:46.041 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T10:03:46.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:03:45 smithi134 ceph-mon[35674]: scrub queued for path: / 2024-09-10T10:03:46.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:03:45 smithi134 ceph-mon[35674]: scrub summary: idle+waiting paths [/] 2024-09-10T10:03:46.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:03:45 smithi134 ceph-mon[35674]: scrub summary: active paths [/] 2024-09-10T10:03:46.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:03:45 smithi134 ceph-mon[35674]: scrub summary: active paths [/] 2024-09-10T10:03:46.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:03:45 smithi134 ceph-mon[35674]: scrub summary: active paths [/] 2024-09-10T10:03:46.382 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:03:45 smithi134 ceph-mon[35674]: scrub summary: active paths [/] 2024-09-10T10:03:46.382 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:03:45 smithi134 ceph-mon[35674]: scrub summary: idle+waiting paths [/] 2024-09-10T10:03:46.382 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:03:45 smithi134 ceph-mon[35674]: scrub summary: idle+waiting paths [/] 2024-09-10T10:03:46.407 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:03:46.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:03:45 smithi120 ceph-mon[34770]: scrub queued for path: / 2024-09-10T10:03:46.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:03:45 smithi120 ceph-mon[34770]: scrub summary: idle+waiting paths [/] 2024-09-10T10:03:46.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:03:45 smithi120 ceph-mon[34770]: scrub summary: active paths [/] 2024-09-10T10:03:46.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:03:45 smithi120 ceph-mon[34770]: scrub summary: active paths [/] 2024-09-10T10:03:46.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:03:45 smithi120 ceph-mon[34770]: scrub summary: active paths [/] 2024-09-10T10:03:46.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:03:45 smithi120 ceph-mon[34770]: scrub summary: active paths [/] 2024-09-10T10:03:46.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:03:45 smithi120 ceph-mon[34770]: scrub summary: idle+waiting paths [/] 2024-09-10T10:03:46.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:03:45 smithi120 ceph-mon[34770]: scrub summary: idle+waiting paths [/] 2024-09-10T10:03:46.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:03:45 smithi077 ceph-mon[30319]: scrub queued for path: / 2024-09-10T10:03:46.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:03:45 smithi077 ceph-mon[30319]: scrub summary: idle+waiting paths [/] 2024-09-10T10:03:46.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:03:45 smithi077 ceph-mon[30319]: scrub summary: active paths [/] 2024-09-10T10:03:46.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:03:45 smithi077 ceph-mon[30319]: scrub summary: active paths [/] 2024-09-10T10:03:46.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:03:45 smithi077 ceph-mon[30319]: scrub summary: active paths [/] 2024-09-10T10:03:46.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:03:45 smithi077 ceph-mon[30319]: scrub summary: active paths [/] 2024-09-10T10:03:46.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:03:45 smithi077 ceph-mon[30319]: scrub summary: idle+waiting paths [/] 2024-09-10T10:03:46.440 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:03:45 smithi077 ceph-mon[30319]: scrub summary: idle+waiting paths [/] 2024-09-10T10:03:46.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.985+0000 7f2dbd600640 1 -- 172.21.15.77:0/1525030939 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2db8106f10 msgr2=0x7f2db810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:46.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.985+0000 7f2dbd600640 1 --2- 172.21.15.77:0/1525030939 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2db8106f10 0x7f2db810ea90 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f2db0009ca0 tx=0x7f2db002a8c0 comp rx=0 tx=0).stop 2024-09-10T10:03:46.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.985+0000 7f2dbd600640 1 -- 172.21.15.77:0/1525030939 shutdown_connections 2024-09-10T10:03:46.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.986+0000 7f2dbd600640 1 --2- 172.21.15.77:0/1525030939 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2db8106f10 0x7f2db810ea90 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:46.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.986+0000 7f2dbd600640 1 --2- 172.21.15.77:0/1525030939 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2db8102790 0x7f2db81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:46.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.986+0000 7f2dbd600640 1 --2- 172.21.15.77:0/1525030939 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2db8101de0 0x7f2db81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:46.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.986+0000 7f2dbd600640 1 -- 172.21.15.77:0/1525030939 >> 172.21.15.77:0/1525030939 conn(0x7f2db80fd8c0 msgr2=0x7f2db80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:46.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.986+0000 7f2dbd600640 1 -- 172.21.15.77:0/1525030939 shutdown_connections 2024-09-10T10:03:46.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.986+0000 7f2dbd600640 1 -- 172.21.15.77:0/1525030939 wait complete. 2024-09-10T10:03:46.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.987+0000 7f2dbd600640 1 Processor -- start 2024-09-10T10:03:46.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.987+0000 7f2dbd600640 1 -- start start 2024-09-10T10:03:46.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.987+0000 7f2dbd600640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2db8101de0 0x7f2db80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:46.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.987+0000 7f2dbd600640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2db8102790 0x7f2db8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:46.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.987+0000 7f2dbd600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2db8106f10 0x7f2db8074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:46.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.987+0000 7f2dbd600640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2db8111fc0 con 0x7f2db8102790 2024-09-10T10:03:46.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.987+0000 7f2dbd600640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2db8111e40 con 0x7f2db8101de0 2024-09-10T10:03:46.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.988+0000 7f2dbd600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2db8112140 con 0x7f2db8106f10 2024-09-10T10:03:46.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.988+0000 7f2db6000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2db8102790 0x7f2db8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:46.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.988+0000 7f2db6a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2db8101de0 0x7f2db80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:46.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.988+0000 7f2db6a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2db8101de0 0x7f2db80744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:53920/0 (socket says 172.21.15.77:53920) 2024-09-10T10:03:46.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.988+0000 7f2db6000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2db8102790 0x7f2db8074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:38246/0 (socket says 172.21.15.77:38246) 2024-09-10T10:03:46.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.988+0000 7f2db6a00640 1 -- 172.21.15.77:0/1522201959 learned_addr learned my addr 172.21.15.77:0/1522201959 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:03:46.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.988+0000 7f2db7400640 1 --2- 172.21.15.77:0/1522201959 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2db8106f10 0x7f2db8074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:46.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.988+0000 7f2db6a00640 1 -- 172.21.15.77:0/1522201959 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2db8106f10 msgr2=0x7f2db8074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:46.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.988+0000 7f2db6a00640 1 --2- 172.21.15.77:0/1522201959 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2db8106f10 0x7f2db8074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:46.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.988+0000 7f2db6a00640 1 -- 172.21.15.77:0/1522201959 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2db8102790 msgr2=0x7f2db8074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:46.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.988+0000 7f2db6a00640 1 --2- 172.21.15.77:0/1522201959 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2db8102790 0x7f2db8074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:46.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.988+0000 7f2db6a00640 1 -- 172.21.15.77:0/1522201959 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2db81a68d0 con 0x7f2db8101de0 2024-09-10T10:03:46.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.989+0000 7f2db6a00640 1 --2- 172.21.15.77:0/1522201959 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2db8101de0 0x7f2db80744f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f2da800e9f0 tx=0x7f2da800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:46.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.989+0000 7f2daf400640 1 -- 172.21.15.77:0/1522201959 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2da800cd80 con 0x7f2db8101de0 2024-09-10T10:03:46.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.989+0000 7f2dbd600640 1 -- 172.21.15.77:0/1522201959 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2db81a6bc0 con 0x7f2db8101de0 2024-09-10T10:03:46.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.989+0000 7f2daf400640 1 -- 172.21.15.77:0/1522201959 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2da8005500 con 0x7f2db8101de0 2024-09-10T10:03:46.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.989+0000 7f2daf400640 1 -- 172.21.15.77:0/1522201959 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2da8010690 con 0x7f2db8101de0 2024-09-10T10:03:46.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.989+0000 7f2dbd600640 1 -- 172.21.15.77:0/1522201959 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2db81a70b0 con 0x7f2db8101de0 2024-09-10T10:03:46.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.992+0000 7f2daf400640 1 -- 172.21.15.77:0/1522201959 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2da80046c0 con 0x7f2db8101de0 2024-09-10T10:03:46.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.992+0000 7f2dc001e740 1 Processor -- start 2024-09-10T10:03:46.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.992+0000 7f2dc001e740 1 -- start start 2024-09-10T10:03:46.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.993+0000 7f2dc001e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556ffe7c3b90 0x556ffe7c3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:46.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.993+0000 7f2dc001e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556ffe7c44b0 0x556ffe7ce830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:46.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.993+0000 7f2dc001e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556ffe7ced70 0x556ffe7d1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:46.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.993+0000 7f2dc001e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x556ffe72a3e0 con 0x556ffe7c44b0 2024-09-10T10:03:46.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.993+0000 7f2dc001e740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x556ffe6d1d80 con 0x556ffe7ced70 2024-09-10T10:03:46.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.993+0000 7f2dc001e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556ffe7263a0 con 0x556ffe7c3b90 2024-09-10T10:03:46.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.993+0000 7f2db6a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556ffe7c44b0 0x556ffe7ce830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:46.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.993+0000 7f2db7400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556ffe7c3b90 0x556ffe7c3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:46.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.993+0000 7f2db6a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556ffe7c44b0 0x556ffe7ce830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:38260/0 (socket says 172.21.15.77:38260) 2024-09-10T10:03:46.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.993+0000 7f2db6a00640 1 -- 172.21.15.77:0/429755493 learned_addr learned my addr 172.21.15.77:0/429755493 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:03:46.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.993+0000 7f2db6000640 1 --2- 172.21.15.77:0/429755493 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556ffe7ced70 0x556ffe7d1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:46.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.993+0000 7f2db7400640 1 -- 172.21.15.77:0/429755493 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556ffe7ced70 msgr2=0x556ffe7d1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:46.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.993+0000 7f2db7400640 1 --2- 172.21.15.77:0/429755493 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556ffe7ced70 0x556ffe7d1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:46.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.993+0000 7f2db7400640 1 -- 172.21.15.77:0/429755493 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556ffe7c44b0 msgr2=0x556ffe7ce830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:46.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.993+0000 7f2db7400640 1 --2- 172.21.15.77:0/429755493 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556ffe7c44b0 0x556ffe7ce830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:46.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.993+0000 7f2db7400640 1 -- 172.21.15.77:0/429755493 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ffe6f0ae0 con 0x556ffe7c3b90 2024-09-10T10:03:46.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.994+0000 7f2daf400640 1 --2- 172.21.15.77:0/1522201959 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f2d9c0a2b80 0x7f2d9c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:46.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.994+0000 7f2daf400640 1 -- 172.21.15.77:0/1522201959 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f2da80cf8b0 con 0x7f2db8101de0 2024-09-10T10:03:46.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.994+0000 7f2db7400640 1 --2- 172.21.15.77:0/429755493 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556ffe7c3b90 0x556ffe7c3f70 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f2db0005e30 tx=0x7f2db00093e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1af5d955997f54a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:46.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.994+0000 7f2db6a00640 1 --2- 172.21.15.77:0/1522201959 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f2d9c0a2b80 0x7f2d9c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:46.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.994+0000 7f2da3400640 1 -- 172.21.15.77:0/429755493 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db0031db0 con 0x556ffe7c3b90 2024-09-10T10:03:46.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.994+0000 7f2da3400640 1 -- 172.21.15.77:0/429755493 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2db0036030 con 0x556ffe7c3b90 2024-09-10T10:03:46.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.994+0000 7f2da3400640 1 -- 172.21.15.77:0/429755493 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db00042a0 con 0x556ffe7c3b90 2024-09-10T10:03:47.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.995+0000 7f2dc001e740 1 -- 172.21.15.77:0/429755493 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556ffe7c3b90 msgr2=0x556ffe7c3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:47.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.995+0000 7f2db6a00640 1 --2- 172.21.15.77:0/1522201959 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f2d9c0a2b80 0x7f2d9c0a5040 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f2da80cfb50 tx=0x7f2da80d5530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:47.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.995+0000 7f2dc001e740 1 --2- 172.21.15.77:0/429755493 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556ffe7c3b90 0x556ffe7c3f70 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f2db0005e30 tx=0x7f2db00093e0 comp rx=0 tx=0).stop 2024-09-10T10:03:47.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.995+0000 7f2dc001e740 1 -- 172.21.15.77:0/429755493 shutdown_connections 2024-09-10T10:03:47.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.995+0000 7f2dc001e740 1 --2- 172.21.15.77:0/429755493 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556ffe7ced70 0x556ffe7d1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:47.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.995+0000 7f2dc001e740 1 --2- 172.21.15.77:0/429755493 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556ffe7c44b0 0x556ffe7ce830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:47.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.995+0000 7f2dc001e740 1 --2- 172.21.15.77:0/429755493 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556ffe7c3b90 0x556ffe7c3f70 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:47.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.995+0000 7f2dc001e740 1 -- 172.21.15.77:0/429755493 >> 172.21.15.77:0/429755493 conn(0x556ffe7c0120 msgr2=0x556ffe7c0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:47.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.995+0000 7f2dc001e740 1 -- 172.21.15.77:0/429755493 shutdown_connections 2024-09-10T10:03:47.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.995+0000 7f2dc001e740 1 -- 172.21.15.77:0/429755493 wait complete. 2024-09-10T10:03:47.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.996+0000 7f2dc001e740 1 -- start start 2024-09-10T10:03:47.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.996+0000 7f2dc001e740 1 Processor -- start 2024-09-10T10:03:47.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.996+0000 7f2dc001e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556ffe7c3b90 0x556ffe7cbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:47.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.996+0000 7f2dc001e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556ffe7c44b0 0x556ffe7cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:47.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.996+0000 7f2dc001e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556ffe7ced70 0x556ffe7cc750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:47.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.996+0000 7f2dc001e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x556ffe7d1860 con 0x556ffe7c44b0 2024-09-10T10:03:47.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.996+0000 7f2dc001e740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x556ffe6e50f0 con 0x556ffe7c3b90 2024-09-10T10:03:47.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.996+0000 7f2dc001e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556ffe7d19e0 con 0x556ffe7ced70 2024-09-10T10:03:47.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.996+0000 7f2dc001e740 10 client.0 _ll_register_callbacks cb 0x556ffe6d69e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:03:47.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.997+0000 7f2db6000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556ffe7c44b0 0x556ffe7cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:47.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.997+0000 7f2db7400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556ffe7c3b90 0x556ffe7cbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:47.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.997+0000 7f2db6000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556ffe7c44b0 0x556ffe7cc210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:38266/0 (socket says 172.21.15.77:38266) 2024-09-10T10:03:47.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.997+0000 7f2db6000640 1 -- 172.21.15.77:0/4191854983 learned_addr learned my addr 172.21.15.77:0/4191854983 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:03:47.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.997+0000 7f2db7400640 1 --2- 172.21.15.77:0/4191854983 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556ffe7ced70 0x556ffe7cc750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:47.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.997+0000 7f2db6000640 1 -- 172.21.15.77:0/4191854983 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556ffe7ced70 msgr2=0x556ffe7cc750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:47.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.997+0000 7f2db6000640 1 --2- 172.21.15.77:0/4191854983 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556ffe7ced70 0x556ffe7cc750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:47.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.997+0000 7f2db6000640 1 -- 172.21.15.77:0/4191854983 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556ffe7c3b90 msgr2=0x556ffe7cbcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:47.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.997+0000 7f2db6000640 1 --2- 172.21.15.77:0/4191854983 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556ffe7c3b90 0x556ffe7cbcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:47.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.997+0000 7f2db6000640 1 -- 172.21.15.77:0/4191854983 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ffe49b3c0 con 0x556ffe7c44b0 2024-09-10T10:03:47.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.997+0000 7f2db7400640 1 --2- 172.21.15.77:0/4191854983 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556ffe7c3b90 0x556ffe7cbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:03:47.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.997+0000 7f2db6000640 1 --2- 172.21.15.77:0/4191854983 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556ffe7c44b0 0x556ffe7cc210 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f2da4012700 tx=0x7f2da4012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a61f0277e34611a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:47.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.997+0000 7f2d97e00640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-10T10:03:47.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.998+0000 7f2d97e00640 1 -- 172.21.15.77:0/4191854983 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2da400f8c0 con 0x556ffe7c44b0 2024-09-10T10:03:47.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.998+0000 7f2dc001e740 1 -- 172.21.15.77:0/4191854983 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2db0036030 con 0x556ffe7c44b0 2024-09-10T10:03:47.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.998+0000 7f2d97e00640 1 -- 172.21.15.77:0/4191854983 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f2da400fa60 con 0x556ffe7c44b0 2024-09-10T10:03:47.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.998+0000 7f2d97e00640 1 -- 172.21.15.77:0/4191854983 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f2da400fc60 con 0x556ffe7c44b0 2024-09-10T10:03:47.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.998+0000 7f2dc001e740 10 client.17121 fetch_fsmap learned FSMap version 35 2024-09-10T10:03:47.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.998+0000 7f2dc001e740 1 -- 172.21.15.77:0/4191854983 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x556ffe7dd640 con 0x556ffe7c44b0 2024-09-10T10:03:47.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.998+0000 7f2d97e00640 1 -- 172.21.15.77:0/4191854983 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2db0036030 con 0x556ffe7c44b0 2024-09-10T10:03:47.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.998+0000 7f2d97e00640 1 -- 172.21.15.77:0/4191854983 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f2da4010530 con 0x556ffe7c44b0 2024-09-10T10:03:47.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.999+0000 7f2dc001e740 10 client.17121 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T10:03:47.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.999+0000 7f2dc001e740 10 client.17121 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T10:03:47.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.999+0000 7f2dc001e740 20 client.17121 populate_metadata read hostname 'smithi077' 2024-09-10T10:03:47.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.999+0000 7f2dc001e740 1 --2- 172.21.15.77:0/4191854983 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x556ffe7dea50 0x556ffe7e0e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:47.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.999+0000 7f2dc001e740 4 client.17121 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:03:47.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.999+0000 7f2dc001e740 1 -- 172.21.15.77:0/4191854983 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556ffe737950 con 0x556ffe7dea50 2024-09-10T10:03:47.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:46.999+0000 7f2db7400640 1 --2- 172.21.15.77:0/4191854983 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x556ffe7dea50 0x556ffe7e0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:47.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.000+0000 7f2db7400640 1 --2- 172.21.15.77:0/4191854983 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x556ffe7dea50 0x556ffe7e0e60 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6d6d4c4a8fdcd38 server_cookie=ec656f9a3d71d522 in_seq=0 out_seq=0 2024-09-10T10:03:47.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.000+0000 7f2d97e00640 10 client.17121 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T10:03:47.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.000+0000 7f2d97e00640 1 -- 172.21.15.77:0/4191854983 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556ffe737950 con 0x556ffe7dea50 2024-09-10T10:03:47.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.000+0000 7f2d97e00640 10 client.17121 handle_command_reply: tid=0 2024-09-10T10:03:47.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.000+0000 7f2dc001e740 1 client.17121 shutdown 2024-09-10T10:03:47.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.001+0000 7f2dc001e740 1 -- 172.21.15.77:0/4191854983 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556ffe7c44b0 msgr2=0x556ffe7cc210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:47.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.001+0000 7f2dc001e740 1 --2- 172.21.15.77:0/4191854983 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556ffe7c44b0 0x556ffe7cc210 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f2da4012700 tx=0x7f2da4012bc0 comp rx=0 tx=0).stop 2024-09-10T10:03:47.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.001+0000 7f2dc001e740 1 -- 172.21.15.77:0/4191854983 shutdown_connections 2024-09-10T10:03:47.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.001+0000 7f2dc001e740 1 --2- 172.21.15.77:0/4191854983 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x556ffe7dea50 0x556ffe7e0e60 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:47.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.001+0000 7f2dc001e740 1 --2- 172.21.15.77:0/4191854983 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556ffe7ced70 0x556ffe7cc750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:47.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.001+0000 7f2dc001e740 1 --2- 172.21.15.77:0/4191854983 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556ffe7c44b0 0x556ffe7cc210 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:47.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.001+0000 7f2dc001e740 1 --2- 172.21.15.77:0/4191854983 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556ffe7c3b90 0x556ffe7cbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:47.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.001+0000 7f2d97e00640 0 client.17121 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T10:03:47.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.001+0000 7f2dc001e740 1 -- 172.21.15.77:0/4191854983 >> 172.21.15.77:0/4191854983 conn(0x556ffe7c0120 msgr2=0x556ffe7c5e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:47.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.001+0000 7f2dc001e740 1 -- 172.21.15.77:0/4191854983 shutdown_connections 2024-09-10T10:03:47.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.001+0000 7f2dc001e740 1 -- 172.21.15.77:0/4191854983 wait complete. 2024-09-10T10:03:47.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.001+0000 7f2dc001e740 20 client.17121 trim_cache size 0 max 16384 2024-09-10T10:03:47.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.019+0000 7f2dc001e740 1 Processor -- start 2024-09-10T10:03:47.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.019+0000 7f2dc001e740 1 -- start start 2024-09-10T10:03:47.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.019+0000 7f2dc001e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556ffe7c3b90 0x556ffe7e6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:47.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.019+0000 7f2dc001e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556ffe7c44b0 0x556ffe7e6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:47.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.019+0000 7f2dc001e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556ffe7ced70 0x556ffe7c8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:47.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.019+0000 7f2dc001e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2db81b0fa0 con 0x556ffe7c3b90 2024-09-10T10:03:47.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.019+0000 7f2dc001e740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2db81b2ae0 con 0x556ffe7ced70 2024-09-10T10:03:47.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.019+0000 7f2dc001e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2db81b1670 con 0x556ffe7c44b0 2024-09-10T10:03:47.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.019+0000 7f2db7400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556ffe7c3b90 0x556ffe7e6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:47.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.020+0000 7f2db7400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556ffe7c3b90 0x556ffe7e6770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:38272/0 (socket says 172.21.15.77:38272) 2024-09-10T10:03:47.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.020+0000 7f2db7400640 1 -- 172.21.15.77:0/3630301134 learned_addr learned my addr 172.21.15.77:0/3630301134 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:03:47.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.020+0000 7f2db6000640 1 --2- 172.21.15.77:0/3630301134 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556ffe7c44b0 0x556ffe7e6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:47.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.020+0000 7f2db7400640 1 --2- 172.21.15.77:0/3630301134 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556ffe7ced70 0x556ffe7c8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:47.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.020+0000 7f2db7400640 1 -- 172.21.15.77:0/3630301134 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556ffe7c44b0 msgr2=0x556ffe7e6cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:47.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.020+0000 7f2db7400640 1 --2- 172.21.15.77:0/3630301134 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556ffe7c44b0 0x556ffe7e6cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:47.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.020+0000 7f2db7400640 1 -- 172.21.15.77:0/3630301134 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556ffe7ced70 msgr2=0x556ffe7c8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:47.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.020+0000 7f2db7400640 1 --2- 172.21.15.77:0/3630301134 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556ffe7ced70 0x556ffe7c8f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:47.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.020+0000 7f2db7400640 1 -- 172.21.15.77:0/3630301134 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ffe7ba3c0 con 0x556ffe7c3b90 2024-09-10T10:03:47.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.020+0000 7f2db7400640 1 --2- 172.21.15.77:0/3630301134 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556ffe7c3b90 0x556ffe7e6770 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f2db0006010 tx=0x7f2db0005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0519cf67aece8ed server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:47.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.020+0000 7f2da2a00640 1 -- 172.21.15.77:0/3630301134 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db004a040 con 0x556ffe7c3b90 2024-09-10T10:03:47.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.021+0000 7f2da2a00640 1 -- 172.21.15.77:0/3630301134 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2db0036070 con 0x556ffe7c3b90 2024-09-10T10:03:47.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.021+0000 7f2da2a00640 1 -- 172.21.15.77:0/3630301134 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db0004570 con 0x556ffe7c3b90 2024-09-10T10:03:47.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.021+0000 7f2dc001e740 1 -- 172.21.15.77:0/3630301134 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556ffe7c3b90 msgr2=0x556ffe7e6770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:47.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.021+0000 7f2dc001e740 1 --2- 172.21.15.77:0/3630301134 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556ffe7c3b90 0x556ffe7e6770 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f2db0006010 tx=0x7f2db0005d80 comp rx=0 tx=0).stop 2024-09-10T10:03:47.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.021+0000 7f2dc001e740 1 -- 172.21.15.77:0/3630301134 shutdown_connections 2024-09-10T10:03:47.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.021+0000 7f2dc001e740 1 --2- 172.21.15.77:0/3630301134 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556ffe7ced70 0x556ffe7c8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:47.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.021+0000 7f2dc001e740 1 --2- 172.21.15.77:0/3630301134 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556ffe7c44b0 0x556ffe7e6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:47.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.021+0000 7f2dc001e740 1 --2- 172.21.15.77:0/3630301134 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556ffe7c3b90 0x556ffe7e6770 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:47.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.021+0000 7f2dc001e740 1 -- 172.21.15.77:0/3630301134 >> 172.21.15.77:0/3630301134 conn(0x556ffe7c0120 msgr2=0x556ffe7ca5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:47.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.021+0000 7f2dc001e740 1 -- 172.21.15.77:0/3630301134 shutdown_connections 2024-09-10T10:03:47.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.021+0000 7f2dc001e740 1 -- 172.21.15.77:0/3630301134 wait complete. 2024-09-10T10:03:47.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.022+0000 7f2dc001e740 1 -- start start 2024-09-10T10:03:47.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.022+0000 7f2dc001e740 1 Processor -- start 2024-09-10T10:03:47.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.022+0000 7f2dc001e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556ffe7c3b90 0x556ffe7d6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:47.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.022+0000 7f2dc001e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556ffe7c44b0 0x556ffe7d7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:47.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.022+0000 7f2dc001e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556ffe7ced70 0x556ffe7d3c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:47.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.022+0000 7f2dc001e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2db81b2410 con 0x556ffe7ced70 2024-09-10T10:03:47.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.022+0000 7f2dc001e740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2db81b1d40 con 0x556ffe7c3b90 2024-09-10T10:03:47.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.022+0000 7f2dc001e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2db81aef80 con 0x556ffe7c44b0 2024-09-10T10:03:47.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.022+0000 7f2dc001e740 10 client.0 _ll_register_callbacks cb 0x7f2da4009e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:03:47.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.023+0000 7f2db7400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556ffe7c3b90 0x556ffe7d6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:47.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.023+0000 7f2db7400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556ffe7ced70 0x556ffe7d3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:47.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.023+0000 7f2db6000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556ffe7c44b0 0x556ffe7d7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:47.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.023+0000 7f2db7400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556ffe7ced70 0x556ffe7d3c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:38278/0 (socket says 172.21.15.77:38278) 2024-09-10T10:03:47.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.023+0000 7f2db7400640 1 -- 172.21.15.77:0/2144569962 learned_addr learned my addr 172.21.15.77:0/2144569962 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:03:47.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.023+0000 7f2db6000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556ffe7c44b0 0x556ffe7d7350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:56562/0 (socket says 172.21.15.77:56562) 2024-09-10T10:03:47.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.023+0000 7f2db6000640 1 -- 172.21.15.77:0/2144569962 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556ffe7c3b90 msgr2=0x556ffe7d6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:47.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.023+0000 7f2db6000640 1 --2- 172.21.15.77:0/2144569962 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556ffe7c3b90 0x556ffe7d6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:47.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.023+0000 7f2db6000640 1 -- 172.21.15.77:0/2144569962 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556ffe7ced70 msgr2=0x556ffe7d3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:47.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.023+0000 7f2db6000640 1 --2- 172.21.15.77:0/2144569962 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556ffe7ced70 0x556ffe7d3c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:47.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.023+0000 7f2db6000640 1 -- 172.21.15.77:0/2144569962 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ffe7d4290 con 0x556ffe7c44b0 2024-09-10T10:03:47.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.023+0000 7f2db7400640 1 --2- 172.21.15.77:0/2144569962 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556ffe7ced70 0x556ffe7d3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:03:47.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.024+0000 7f2db6000640 1 --2- 172.21.15.77:0/2144569962 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556ffe7c44b0 0x556ffe7d7350 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f2da40109d0 tx=0x7f2da40096c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=157eff17b33f5ca4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:47.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.024+0000 7f2d97e00640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-10T10:03:47.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.024+0000 7f2d97e00640 1 -- 172.21.15.77:0/2144569962 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2da402e060 con 0x556ffe7c44b0 2024-09-10T10:03:47.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.024+0000 7f2dc001e740 1 -- 172.21.15.77:0/2144569962 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2db0036070 con 0x556ffe7c44b0 2024-09-10T10:03:47.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.024+0000 7f2d97e00640 1 -- 172.21.15.77:0/2144569962 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f2da400fa20 con 0x556ffe7c44b0 2024-09-10T10:03:47.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.024+0000 7f2d97e00640 1 -- 172.21.15.77:0/2144569962 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f2da4010430 con 0x556ffe7c44b0 2024-09-10T10:03:47.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.024+0000 7f2dc001e740 10 client.27223 fetch_fsmap learned FSMap version 35 2024-09-10T10:03:47.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.024+0000 7f2dc001e740 1 -- 172.21.15.77:0/2144569962 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x556ffe84a0e0 con 0x556ffe7c44b0 2024-09-10T10:03:47.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.025+0000 7f2d97e00640 1 -- 172.21.15.77:0/2144569962 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2db0036070 con 0x556ffe7c44b0 2024-09-10T10:03:47.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.025+0000 7f2d97e00640 1 -- 172.21.15.77:0/2144569962 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f2da4003850 con 0x556ffe7c44b0 2024-09-10T10:03:47.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.025+0000 7f2dc001e740 10 client.27223 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T10:03:47.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.025+0000 7f2dc001e740 10 client.27223 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T10:03:47.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.025+0000 7f2dc001e740 20 client.27223 populate_metadata read hostname 'smithi077' 2024-09-10T10:03:47.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.025+0000 7f2dc001e740 1 --2- 172.21.15.77:0/2144569962 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x556ffe84a860 0x556ffe84cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:47.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.025+0000 7f2dc001e740 4 client.27223 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T10:03:47.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.025+0000 7f2dc001e740 1 -- 172.21.15.77:0/2144569962 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556ffe731e60 con 0x556ffe84a860 2024-09-10T10:03:47.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.025+0000 7f2db7400640 1 --2- 172.21.15.77:0/2144569962 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x556ffe84a860 0x556ffe84cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:47.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.026+0000 7f2db7400640 1 --2- 172.21.15.77:0/2144569962 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x556ffe84a860 0x556ffe84cc80 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6b5fa71cfc6bb7 server_cookie=d69e94242fa8ecb0 in_seq=0 out_seq=0 2024-09-10T10:03:47.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.026+0000 7f2d97e00640 10 client.27223 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T10:03:47.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.026+0000 7f2d97e00640 1 -- 172.21.15.77:0/2144569962 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x556ffe731e60 con 0x556ffe84a860 2024-09-10T10:03:47.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.026+0000 7f2d97e00640 10 client.27223 handle_command_reply: tid=0 2024-09-10T10:03:47.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.026+0000 7f2dc001e740 1 client.27223 shutdown 2024-09-10T10:03:47.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.026+0000 7f2dc001e740 1 -- 172.21.15.77:0/2144569962 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556ffe7c44b0 msgr2=0x556ffe7d7350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:47.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.026+0000 7f2dc001e740 1 --2- 172.21.15.77:0/2144569962 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556ffe7c44b0 0x556ffe7d7350 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f2da40109d0 tx=0x7f2da40096c0 comp rx=0 tx=0).stop 2024-09-10T10:03:47.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.026+0000 7f2dc001e740 1 -- 172.21.15.77:0/2144569962 shutdown_connections 2024-09-10T10:03:47.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.026+0000 7f2dc001e740 1 --2- 172.21.15.77:0/2144569962 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x556ffe84a860 0x556ffe84cc80 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:47.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.026+0000 7f2dc001e740 1 --2- 172.21.15.77:0/2144569962 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556ffe7ced70 0x556ffe7d3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:47.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.026+0000 7f2dc001e740 1 --2- 172.21.15.77:0/2144569962 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556ffe7c44b0 0x556ffe7d7350 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:47.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.026+0000 7f2d97e00640 0 client.27223 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T10:03:47.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.026+0000 7f2dc001e740 1 --2- 172.21.15.77:0/2144569962 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556ffe7c3b90 0x556ffe7d6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:47.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.026+0000 7f2dc001e740 1 -- 172.21.15.77:0/2144569962 >> 172.21.15.77:0/2144569962 conn(0x556ffe7c0120 msgr2=0x556ffe7cd530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:47.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.027+0000 7f2dc001e740 1 -- 172.21.15.77:0/2144569962 shutdown_connections 2024-09-10T10:03:47.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.027+0000 7f2dc001e740 1 -- 172.21.15.77:0/2144569962 wait complete. 2024-09-10T10:03:47.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.027+0000 7f2dc001e740 20 client.27223 trim_cache size 0 max 16384 2024-09-10T10:03:47.032 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-10T10:03:47.032 INFO:teuthology.orchestra.run.smithi077.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T10:03:47.032 INFO:teuthology.orchestra.run.smithi077.stdout: "scrubs": { 2024-09-10T10:03:47.032 INFO:teuthology.orchestra.run.smithi077.stdout: "bcf9d927-6c1e-4ab2-b6b8-55801c8be666": { 2024-09-10T10:03:47.032 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/", 2024-09-10T10:03:47.032 INFO:teuthology.orchestra.run.smithi077.stdout: "tag": "bcf9d927-6c1e-4ab2-b6b8-55801c8be666", 2024-09-10T10:03:47.032 INFO:teuthology.orchestra.run.smithi077.stdout: "options": "recursive,force" 2024-09-10T10:03:47.033 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:03:47.033 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:03:47.033 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-10T10:03:47.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.027+0000 7f2dbd600640 1 -- 172.21.15.77:0/1522201959 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f2d9c0a2b80 msgr2=0x7f2d9c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:47.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.027+0000 7f2dbd600640 1 --2- 172.21.15.77:0/1522201959 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f2d9c0a2b80 0x7f2d9c0a5040 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f2da80cfb50 tx=0x7f2da80d5530 comp rx=0 tx=0).stop 2024-09-10T10:03:47.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.027+0000 7f2dbd600640 1 -- 172.21.15.77:0/1522201959 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2db8101de0 msgr2=0x7f2db80744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:47.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.027+0000 7f2dbd600640 1 --2- 172.21.15.77:0/1522201959 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2db8101de0 0x7f2db80744f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f2da800e9f0 tx=0x7f2da800edf0 comp rx=0 tx=0).stop 2024-09-10T10:03:47.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.027+0000 7f2dbd600640 1 -- 172.21.15.77:0/1522201959 shutdown_connections 2024-09-10T10:03:47.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.027+0000 7f2dbd600640 1 --2- 172.21.15.77:0/1522201959 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f2d9c0a2b80 0x7f2d9c0a5040 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:47.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.027+0000 7f2dbd600640 1 --2- 172.21.15.77:0/1522201959 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2db8106f10 0x7f2db8074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:47.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.027+0000 7f2dbd600640 1 --2- 172.21.15.77:0/1522201959 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2db8102790 0x7f2db8074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:47.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.027+0000 7f2dbd600640 1 --2- 172.21.15.77:0/1522201959 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2db8101de0 0x7f2db80744f0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:47.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.027+0000 7f2dbd600640 1 -- 172.21.15.77:0/1522201959 >> 172.21.15.77:0/1522201959 conn(0x7f2db80fd8c0 msgr2=0x7f2db80fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:47.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.028+0000 7f2dbd600640 1 -- 172.21.15.77:0/1522201959 shutdown_connections 2024-09-10T10:03:47.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:47.028+0000 7f2dbd600640 1 -- 172.21.15.77:0/1522201959 wait complete. 2024-09-10T10:03:47.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:03:46 smithi077 ceph-mon[30319]: scrub summary: active paths [/] 2024-09-10T10:03:47.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:03:46 smithi077 ceph-mon[30319]: scrub summary: idle+waiting paths [/] 2024-09-10T10:03:47.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:03:46 smithi077 ceph-mon[30319]: scrub summary: idle+waiting paths [/] 2024-09-10T10:03:47.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:03:46 smithi077 ceph-mon[30319]: scrub summary: idle+waiting paths [/] 2024-09-10T10:03:47.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:03:46 smithi077 ceph-mon[30319]: pgmap v649: 129 pgs: 129 active+clean; 140 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 1018 KiB/s wr, 399 op/s 2024-09-10T10:03:47.325 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:03:46 smithi120 ceph-mon[34770]: scrub summary: active paths [/] 2024-09-10T10:03:47.325 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:03:46 smithi120 ceph-mon[34770]: scrub summary: idle+waiting paths [/] 2024-09-10T10:03:47.325 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:03:46 smithi120 ceph-mon[34770]: scrub summary: idle+waiting paths [/] 2024-09-10T10:03:47.325 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:03:46 smithi120 ceph-mon[34770]: scrub summary: idle+waiting paths [/] 2024-09-10T10:03:47.325 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:03:46 smithi120 ceph-mon[34770]: pgmap v649: 129 pgs: 129 active+clean; 140 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 1018 KiB/s wr, 399 op/s 2024-09-10T10:03:47.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:03:46 smithi134 ceph-mon[35674]: scrub summary: active paths [/] 2024-09-10T10:03:47.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:03:46 smithi134 ceph-mon[35674]: scrub summary: idle+waiting paths [/] 2024-09-10T10:03:47.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:03:46 smithi134 ceph-mon[35674]: scrub summary: idle+waiting paths [/] 2024-09-10T10:03:47.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:03:46 smithi134 ceph-mon[35674]: scrub summary: idle+waiting paths [/] 2024-09-10T10:03:47.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:03:46 smithi134 ceph-mon[35674]: pgmap v649: 129 pgs: 129 active+clean; 140 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 1018 KiB/s wr, 399 op/s 2024-09-10T10:03:47.787 INFO:tasks.cephfs.filesystem:scrub status for tag:bcf9d927-6c1e-4ab2-b6b8-55801c8be666 - {'path': '/', 'tag': 'bcf9d927-6c1e-4ab2-b6b8-55801c8be666', 'options': 'recursive,force'} 2024-09-10T10:03:49.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:03:48 smithi134 ceph-mon[35674]: pgmap v650: 129 pgs: 129 active+clean; 149 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 972 KiB/s wr, 382 op/s 2024-09-10T10:03:49.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:03:48 smithi120 ceph-mon[34770]: pgmap v650: 129 pgs: 129 active+clean; 149 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 972 KiB/s wr, 382 op/s 2024-09-10T10:03:49.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:03:48 smithi077 ceph-mon[30319]: pgmap v650: 129 pgs: 129 active+clean; 149 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 972 KiB/s wr, 382 op/s 2024-09-10T10:03:50.886 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:03:51.261 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:03:51.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:03:50 smithi134 ceph-mon[35674]: pgmap v651: 129 pgs: 129 active+clean; 150 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 835 KiB/s wr, 331 op/s 2024-09-10T10:03:51.400 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:03:50 smithi077 ceph-mon[30319]: pgmap v651: 129 pgs: 129 active+clean; 150 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 835 KiB/s wr, 331 op/s 2024-09-10T10:03:51.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:03:50 smithi120 ceph-mon[34770]: pgmap v651: 129 pgs: 129 active+clean; 150 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 835 KiB/s wr, 331 op/s 2024-09-10T10:03:51.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:51.852+0000 7fa0e7400640 1 -- 172.21.15.77:0/1534764744 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa0e01049a0 msgr2=0x7fa0e01089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:51.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:51.852+0000 7fa0e7400640 1 --2- 172.21.15.77:0/1534764744 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa0e01049a0 0x7fa0e01089f0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc009900 tx=0x7fa0cc02a3f0 comp rx=0 tx=0).stop 2024-09-10T10:03:51.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:51.852+0000 7fa0e7400640 1 -- 172.21.15.77:0/1534764744 shutdown_connections 2024-09-10T10:03:51.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:51.852+0000 7fa0e7400640 1 --2- 172.21.15.77:0/1534764744 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa0e0109120 0x7fa0e0110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:51.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:51.852+0000 7fa0e7400640 1 --2- 172.21.15.77:0/1534764744 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa0e01049a0 0x7fa0e01089f0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:51.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:51.852+0000 7fa0e7400640 1 --2- 172.21.15.77:0/1534764744 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0e0103ff0 0x7fa0e01043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:51.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:51.852+0000 7fa0e7400640 1 -- 172.21.15.77:0/1534764744 >> 172.21.15.77:0/1534764744 conn(0x7fa0e00ffa90 msgr2=0x7fa0e0101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:51.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:51.852+0000 7fa0e7400640 1 -- 172.21.15.77:0/1534764744 shutdown_connections 2024-09-10T10:03:51.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:51.852+0000 7fa0e7400640 1 -- 172.21.15.77:0/1534764744 wait complete. 2024-09-10T10:03:51.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:51.853+0000 7fa0e7400640 1 Processor -- start 2024-09-10T10:03:51.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:51.853+0000 7fa0e7400640 1 -- start start 2024-09-10T10:03:51.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:51.854+0000 7fa0e7400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa0e0103ff0 0x7fa0e019a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:51.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:51.854+0000 7fa0e7400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0e01049a0 0x7fa0e019ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:51.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:51.854+0000 7fa0e7400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa0e0109120 0x7fa0e019ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:51.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:51.854+0000 7fa0e7400640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e0114170 con 0x7fa0e0103ff0 2024-09-10T10:03:51.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:51.854+0000 7fa0e7400640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e0113ff0 con 0x7fa0e0109120 2024-09-10T10:03:51.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:51.854+0000 7fa0e7400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e01142f0 con 0x7fa0e01049a0 2024-09-10T10:03:51.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:51.854+0000 7fa0e4a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa0e0103ff0 0x7fa0e019a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:51.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:51.854+0000 7fa0e4a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa0e0103ff0 0x7fa0e019a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:38288/0 (socket says 172.21.15.77:38288) 2024-09-10T10:03:51.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:51.854+0000 7fa0e4a00640 1 -- 172.21.15.77:0/2802945446 learned_addr learned my addr 172.21.15.77:0/2802945446 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:03:51.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:51.854+0000 7fa0dfe00640 1 --2- 172.21.15.77:0/2802945446 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0e01049a0 0x7fa0e019ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:51.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:51.854+0000 7fa0e5400640 1 --2- 172.21.15.77:0/2802945446 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa0e0109120 0x7fa0e019ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:51.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:51.855+0000 7fa0e4a00640 1 -- 172.21.15.77:0/2802945446 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0e01049a0 msgr2=0x7fa0e019ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:51.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:51.855+0000 7fa0e4a00640 1 --2- 172.21.15.77:0/2802945446 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0e01049a0 0x7fa0e019ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:51.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:51.855+0000 7fa0e4a00640 1 -- 172.21.15.77:0/2802945446 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa0e0109120 msgr2=0x7fa0e019ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:51.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:51.855+0000 7fa0e4a00640 1 --2- 172.21.15.77:0/2802945446 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa0e0109120 0x7fa0e019ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:51.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:51.855+0000 7fa0e4a00640 1 -- 172.21.15.77:0/2802945446 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0e019f600 con 0x7fa0e0103ff0 2024-09-10T10:03:51.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:51.855+0000 7fa0e4a00640 1 --2- 172.21.15.77:0/2802945446 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa0e0103ff0 0x7fa0e019a650 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fa0d400d950 tx=0x7fa0d400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:51.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:51.855+0000 7fa0dd600640 1 -- 172.21.15.77:0/2802945446 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d4017030 con 0x7fa0e0103ff0 2024-09-10T10:03:51.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:51.855+0000 7fa0e7400640 1 -- 172.21.15.77:0/2802945446 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0e019f8f0 con 0x7fa0e0103ff0 2024-09-10T10:03:51.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:51.855+0000 7fa0dd600640 1 -- 172.21.15.77:0/2802945446 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0d4004020 con 0x7fa0e0103ff0 2024-09-10T10:03:51.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:51.855+0000 7fa0dd600640 1 -- 172.21.15.77:0/2802945446 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d40047a0 con 0x7fa0e0103ff0 2024-09-10T10:03:51.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:51.855+0000 7fa0e7400640 1 -- 172.21.15.77:0/2802945446 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0e01a71d0 con 0x7fa0e0103ff0 2024-09-10T10:03:51.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:51.857+0000 7fa0e7400640 1 -- 172.21.15.77:0/2802945446 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0e01043d0 con 0x7fa0e0103ff0 2024-09-10T10:03:51.860 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:51.858+0000 7fa0dd600640 1 -- 172.21.15.77:0/2802945446 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa0d40041d0 con 0x7fa0e0103ff0 2024-09-10T10:03:51.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:51.860+0000 7fa0dd600640 1 --2- 172.21.15.77:0/2802945446 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fa0c80a2b80 0x7fa0c80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:51.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:51.860+0000 7fa0dd600640 1 -- 172.21.15.77:0/2802945446 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fa0d40cedc0 con 0x7fa0e0103ff0 2024-09-10T10:03:51.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:51.860+0000 7fa0dfe00640 1 --2- 172.21.15.77:0/2802945446 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fa0c80a2b80 0x7fa0c80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:51.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:51.861+0000 7fa0dfe00640 1 --2- 172.21.15.77:0/2802945446 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fa0c80a2b80 0x7fa0c80a5040 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc000c00 tx=0x7fa0cc005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:51.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:51.864+0000 7fa0dd600640 1 -- 172.21.15.77:0/2802945446 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa0d409b8d0 con 0x7fa0e0103ff0 2024-09-10T10:03:52.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:52.074+0000 7fa0e7400640 1 -- 172.21.15.77:0/2802945446 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa0e010c3c0 con 0x7fa0e0103ff0 2024-09-10T10:03:52.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:52.074+0000 7fa0dd600640 1 -- 172.21.15.77:0/2802945446 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11989 (secure 0 0 0) 0x7fa0d409b1d0 con 0x7fa0e0103ff0 2024-09-10T10:03:52.077 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T10:03:52.077 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":35,"btime":"2024-09-10T10:01:05:922537+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3159289675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3159289675},{"type":"v1","addr":"172.21.15.77:6835","nonce":3159289675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1951962341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1951962341},{"type":"v1","addr":"172.21.15.77:6837","nonce":1951962341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":35,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:52:23.368252+0000","modified":"2024-09-10T10:01:05.706646+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24563,"mds_1":24524,"mds_2":24520,"mds_3":24502,"mds_4":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14487":{"gid":14487,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/4291121888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":4291121888},{"type":"v1","addr":"172.21.15.77:6839","nonce":4291121888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14499":{"gid":14499,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6841/445284194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":445284194},{"type":"v1","addr":"172.21.15.77:6841","nonce":445284194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1005054423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1005054423},{"type":"v1","addr":"172.21.15.134:6833","nonce":1005054423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6833/3716307744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":3716307744},{"type":"v1","addr":"172.21.15.120:6833","nonce":3716307744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/1319892544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1319892544},{"type":"v1","addr":"172.21.15.120:6835","nonce":1319892544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.134:6835/3293933699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3293933699},{"type":"v1","addr":"172.21.15.134:6835","nonce":3293933699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.134:6837/2461497091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2461497091},{"type":"v1","addr":"172.21.15.134:6837","nonce":2461497091}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":20,"addr":"172.21.15.134:6839/1965117680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1965117680},{"type":"v1","addr":"172.21.15.134:6839","nonce":1965117680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":20,"addr":"172.21.15.120:6837/3033280709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3033280709},{"type":"v1","addr":"172.21.15.120:6837","nonce":3033280709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.120:6839/1777710157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1777710157},{"type":"v1","addr":"172.21.15.120:6839","nonce":1777710157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24563,"qdb_cluster":[24496,24502,24520,24524,24563]},"id":1}]} 2024-09-10T10:03:52.077 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 35 2024-09-10T10:03:52.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:52.077+0000 7fa0e7400640 1 -- 172.21.15.77:0/2802945446 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fa0c80a2b80 msgr2=0x7fa0c80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:52.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:52.077+0000 7fa0e7400640 1 --2- 172.21.15.77:0/2802945446 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fa0c80a2b80 0x7fa0c80a5040 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc000c00 tx=0x7fa0cc005b50 comp rx=0 tx=0).stop 2024-09-10T10:03:52.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:52.077+0000 7fa0e7400640 1 -- 172.21.15.77:0/2802945446 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa0e0103ff0 msgr2=0x7fa0e019a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:52.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:52.077+0000 7fa0e7400640 1 --2- 172.21.15.77:0/2802945446 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa0e0103ff0 0x7fa0e019a650 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fa0d400d950 tx=0x7fa0d400de10 comp rx=0 tx=0).stop 2024-09-10T10:03:52.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:52.077+0000 7fa0e7400640 1 -- 172.21.15.77:0/2802945446 shutdown_connections 2024-09-10T10:03:52.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:52.077+0000 7fa0e7400640 1 --2- 172.21.15.77:0/2802945446 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fa0c80a2b80 0x7fa0c80a5040 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:52.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:52.077+0000 7fa0e7400640 1 --2- 172.21.15.77:0/2802945446 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa0e0109120 0x7fa0e019ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:52.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:52.077+0000 7fa0e7400640 1 --2- 172.21.15.77:0/2802945446 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0e01049a0 0x7fa0e019ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:52.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:52.077+0000 7fa0e7400640 1 --2- 172.21.15.77:0/2802945446 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa0e0103ff0 0x7fa0e019a650 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:52.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:52.077+0000 7fa0e7400640 1 -- 172.21.15.77:0/2802945446 >> 172.21.15.77:0/2802945446 conn(0x7fa0e00ffa90 msgr2=0x7fa0e0100da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:52.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:52.077+0000 7fa0e7400640 1 -- 172.21.15.77:0/2802945446 shutdown_connections 2024-09-10T10:03:52.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:52.077+0000 7fa0e7400640 1 -- 172.21.15.77:0/2802945446 wait complete. 2024-09-10T10:03:53.283 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:03:52 smithi077 ceph-mon[30319]: pgmap v652: 129 pgs: 129 active+clean; 150 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 572 KiB/s wr, 219 op/s 2024-09-10T10:03:53.284 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:03:52 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/2802945446' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:03:53.327 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24563 quiesce db --set-id 1f5021c0 --release --await' 2024-09-10T10:03:53.327 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24563 quiesce db --set-id 1f5021c0 --release --await 2024-09-10T10:03:53.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:03:52 smithi134 ceph-mon[35674]: pgmap v652: 129 pgs: 129 active+clean; 150 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 572 KiB/s wr, 219 op/s 2024-09-10T10:03:53.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:03:52 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/2802945446' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:03:53.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:03:52 smithi120 ceph-mon[34770]: pgmap v652: 129 pgs: 129 active+clean; 150 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 572 KiB/s wr, 219 op/s 2024-09-10T10:03:53.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:03:52 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/2802945446' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:03:53.697 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:03:54.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.655+0000 7f896ac00640 1 -- 172.21.15.77:0/3579435735 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8964103490 msgr2=0x7f896410b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:54.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.655+0000 7f896ac00640 1 --2- 172.21.15.77:0/3579435735 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8964103490 0x7f896410b980 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f8950009900 tx=0x7f895002a480 comp rx=0 tx=0).stop 2024-09-10T10:03:54.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.655+0000 7f896ac00640 1 -- 172.21.15.77:0/3579435735 shutdown_connections 2024-09-10T10:03:54.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.655+0000 7f896ac00640 1 --2- 172.21.15.77:0/3579435735 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f896410bfa0 0x7f896410e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:54.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.655+0000 7f896ac00640 1 --2- 172.21.15.77:0/3579435735 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8964103490 0x7f896410b980 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:54.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.655+0000 7f896ac00640 1 --2- 172.21.15.77:0/3579435735 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8964102b70 0x7f8964102f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:54.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.655+0000 7f896ac00640 1 -- 172.21.15.77:0/3579435735 >> 172.21.15.77:0/3579435735 conn(0x7f89640fc5d0 msgr2=0x7f89640fe9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:54.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.655+0000 7f896ac00640 1 -- 172.21.15.77:0/3579435735 shutdown_connections 2024-09-10T10:03:54.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.655+0000 7f896ac00640 1 -- 172.21.15.77:0/3579435735 wait complete. 2024-09-10T10:03:54.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.656+0000 7f896ac00640 1 Processor -- start 2024-09-10T10:03:54.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.657+0000 7f896ac00640 1 -- start start 2024-09-10T10:03:54.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.657+0000 7f896ac00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8964102b70 0x7f896419a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:54.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.657+0000 7f896ac00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8964103490 0x7f896419aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:54.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.657+0000 7f896ac00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f896410bfa0 0x7f896419ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:54.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.657+0000 7f896ac00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8964111df0 con 0x7f8964102b70 2024-09-10T10:03:54.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.657+0000 7f896ac00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8964111c70 con 0x7f8964103490 2024-09-10T10:03:54.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.657+0000 7f896ac00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8964111f70 con 0x7f896410bfa0 2024-09-10T10:03:54.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.657+0000 7f8969800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8964102b70 0x7f896419a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:54.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.657+0000 7f8969800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8964102b70 0x7f896419a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:36130/0 (socket says 172.21.15.77:36130) 2024-09-10T10:03:54.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.657+0000 7f8969800640 1 -- 172.21.15.77:0/1093996953 learned_addr learned my addr 172.21.15.77:0/1093996953 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:03:54.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.657+0000 7f8968e00640 1 --2- 172.21.15.77:0/1093996953 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8964103490 0x7f896419aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:54.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.657+0000 7f896a200640 1 --2- 172.21.15.77:0/1093996953 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f896410bfa0 0x7f896419ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:54.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.658+0000 7f8969800640 1 -- 172.21.15.77:0/1093996953 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f896410bfa0 msgr2=0x7f896419ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:54.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.658+0000 7f8969800640 1 --2- 172.21.15.77:0/1093996953 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f896410bfa0 0x7f896419ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:54.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.658+0000 7f8969800640 1 -- 172.21.15.77:0/1093996953 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8964103490 msgr2=0x7f896419aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:54.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.658+0000 7f8969800640 1 --2- 172.21.15.77:0/1093996953 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8964103490 0x7f896419aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:54.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.658+0000 7f8969800640 1 -- 172.21.15.77:0/1093996953 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f896419f540 con 0x7f8964102b70 2024-09-10T10:03:54.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.658+0000 7f896a200640 1 --2- 172.21.15.77:0/1093996953 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f896410bfa0 0x7f896419ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:03:54.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.658+0000 7f8969800640 1 --2- 172.21.15.77:0/1093996953 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8964102b70 0x7f896419a590 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f895800ea60 tx=0x7f895800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:54.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.658+0000 7f8962000640 1 -- 172.21.15.77:0/1093996953 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f895800cde0 con 0x7f8964102b70 2024-09-10T10:03:54.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.658+0000 7f896ac00640 1 -- 172.21.15.77:0/1093996953 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f896419f830 con 0x7f8964102b70 2024-09-10T10:03:54.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.658+0000 7f8962000640 1 -- 172.21.15.77:0/1093996953 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89580035f0 con 0x7f8964102b70 2024-09-10T10:03:54.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.658+0000 7f8962000640 1 -- 172.21.15.77:0/1093996953 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8958010690 con 0x7f8964102b70 2024-09-10T10:03:54.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.659+0000 7f896ac00640 1 -- 172.21.15.77:0/1093996953 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89641a7110 con 0x7f8964102b70 2024-09-10T10:03:54.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.661+0000 7f896eca1740 1 Processor -- start 2024-09-10T10:03:54.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.661+0000 7f8962000640 1 -- 172.21.15.77:0/1093996953 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f89580037a0 con 0x7f8964102b70 2024-09-10T10:03:54.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.662+0000 7f896eca1740 1 -- start start 2024-09-10T10:03:54.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.662+0000 7f896eca1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561fef748520 0x561fef748900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:54.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.662+0000 7f896eca1740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561fef748e40 0x561fef7531c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:54.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.662+0000 7f896eca1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fef753700 0x561fef755af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:54.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.662+0000 7f896eca1740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x561fef6ae650 con 0x561fef748e40 2024-09-10T10:03:54.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.662+0000 7f896eca1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x561fef655f10 con 0x561fef748520 2024-09-10T10:03:54.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.662+0000 7f896eca1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x561fef6aa610 con 0x561fef753700 2024-09-10T10:03:54.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.662+0000 7f896a200640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561fef748520 0x561fef748900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:54.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.662+0000 7f8969800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561fef748e40 0x561fef7531c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:54.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.662+0000 7f896a200640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561fef748520 0x561fef748900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:56440/0 (socket says 172.21.15.77:56440) 2024-09-10T10:03:54.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.662+0000 7f896a200640 1 -- 172.21.15.77:0/2557361904 learned_addr learned my addr 172.21.15.77:0/2557361904 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:03:54.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.662+0000 7f8968e00640 1 --2- 172.21.15.77:0/2557361904 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fef753700 0x561fef755af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:54.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.663+0000 7f896a200640 1 -- 172.21.15.77:0/2557361904 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fef753700 msgr2=0x561fef755af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:54.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.663+0000 7f896a200640 1 --2- 172.21.15.77:0/2557361904 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fef753700 0x561fef755af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:54.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.663+0000 7f896a200640 1 -- 172.21.15.77:0/2557361904 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561fef748e40 msgr2=0x561fef7531c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:54.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.663+0000 7f896a200640 1 --2- 172.21.15.77:0/2557361904 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561fef748e40 0x561fef7531c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:54.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.663+0000 7f896a200640 1 -- 172.21.15.77:0/2557361904 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fef3bd580 con 0x561fef748520 2024-09-10T10:03:54.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.663+0000 7f8962000640 1 --2- 172.21.15.77:0/1093996953 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f894c0a2b80 0x7f894c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:54.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.663+0000 7f896a200640 1 --2- 172.21.15.77:0/2557361904 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561fef748520 0x561fef748900 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f895c009f30 tx=0x7f895c010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cfbb0c52c3636e90 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:54.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.663+0000 7f8962000640 1 -- 172.21.15.77:0/1093996953 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f89580cf910 con 0x7f8964102b70 2024-09-10T10:03:54.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.663+0000 7f8956a00640 1 -- 172.21.15.77:0/2557361904 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f895c016070 con 0x561fef748520 2024-09-10T10:03:54.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.664+0000 7f8956a00640 1 -- 172.21.15.77:0/2557361904 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f895c01bb40 con 0x561fef748520 2024-09-10T10:03:54.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.664+0000 7f8956a00640 1 -- 172.21.15.77:0/2557361904 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f895c012c50 con 0x561fef748520 2024-09-10T10:03:54.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.664+0000 7f8969800640 1 --2- 172.21.15.77:0/1093996953 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f894c0a2b80 0x7f894c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:54.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.664+0000 7f896eca1740 1 -- 172.21.15.77:0/2557361904 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561fef748520 msgr2=0x561fef748900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:54.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.664+0000 7f8969800640 1 --2- 172.21.15.77:0/1093996953 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f894c0a2b80 0x7f894c0a5040 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f89580a4460 tx=0x7f89580d5530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:54.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.664+0000 7f896eca1740 1 --2- 172.21.15.77:0/2557361904 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561fef748520 0x561fef748900 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f895c009f30 tx=0x7f895c010800 comp rx=0 tx=0).stop 2024-09-10T10:03:54.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.664+0000 7f896eca1740 1 -- 172.21.15.77:0/2557361904 shutdown_connections 2024-09-10T10:03:54.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.664+0000 7f896eca1740 1 --2- 172.21.15.77:0/2557361904 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fef753700 0x561fef755af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:54.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.664+0000 7f896eca1740 1 --2- 172.21.15.77:0/2557361904 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561fef748e40 0x561fef7531c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:54.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.664+0000 7f896eca1740 1 --2- 172.21.15.77:0/2557361904 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561fef748520 0x561fef748900 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:54.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.664+0000 7f896eca1740 1 -- 172.21.15.77:0/2557361904 >> 172.21.15.77:0/2557361904 conn(0x561fef744ab0 msgr2=0x561fef744e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:54.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.665+0000 7f896eca1740 1 -- 172.21.15.77:0/2557361904 shutdown_connections 2024-09-10T10:03:54.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.665+0000 7f896eca1740 1 -- 172.21.15.77:0/2557361904 wait complete. 2024-09-10T10:03:54.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.665+0000 7f896eca1740 1 -- start start 2024-09-10T10:03:54.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.665+0000 7f896eca1740 1 Processor -- start 2024-09-10T10:03:54.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.666+0000 7f896eca1740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561fef748520 0x561fef750640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:54.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.666+0000 7f896eca1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fef748e40 0x561fef750b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:54.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.666+0000 7f896eca1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561fef753700 0x561fef7510c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:54.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.666+0000 7f896eca1740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x561fef7562d0 con 0x561fef748520 2024-09-10T10:03:54.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.666+0000 7f896eca1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x561fef669350 con 0x561fef753700 2024-09-10T10:03:54.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.666+0000 7f896eca1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x561fef756450 con 0x561fef748e40 2024-09-10T10:03:54.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.666+0000 7f896eca1740 10 client.0 _ll_register_callbacks cb 0x561fef68c290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:03:54.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.666+0000 7f896a200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561fef748520 0x561fef750640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:54.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.666+0000 7f8968e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fef748e40 0x561fef750b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:54.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.666+0000 7f896a200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561fef748520 0x561fef750640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:36140/0 (socket says 172.21.15.77:36140) 2024-09-10T10:03:54.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.666+0000 7f896a200640 1 -- 172.21.15.77:0/1711040886 learned_addr learned my addr 172.21.15.77:0/1711040886 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:03:54.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.666+0000 7f896a200640 1 --2- 172.21.15.77:0/1711040886 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561fef753700 0x561fef7510c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:54.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.666+0000 7f8968e00640 1 -- 172.21.15.77:0/1711040886 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561fef753700 msgr2=0x561fef7510c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:54.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.666+0000 7f8968e00640 1 --2- 172.21.15.77:0/1711040886 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561fef753700 0x561fef7510c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:54.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.666+0000 7f8968e00640 1 -- 172.21.15.77:0/1711040886 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561fef748520 msgr2=0x561fef750640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:54.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.667+0000 7f8968e00640 1 --2- 172.21.15.77:0/1711040886 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561fef748520 0x561fef750640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:54.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.667+0000 7f8968e00640 1 -- 172.21.15.77:0/1711040886 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fef487960 con 0x561fef748e40 2024-09-10T10:03:54.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.667+0000 7f896a200640 1 --2- 172.21.15.77:0/1711040886 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561fef753700 0x561fef7510c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:03:54.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.667+0000 7f8968e00640 1 --2- 172.21.15.77:0/1711040886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fef748e40 0x561fef750b80 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f895002aa70 tx=0x7f8950004440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed1a01b0586593dd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:54.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.667+0000 7f894b400640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-10T10:03:54.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.667+0000 7f894b400640 1 -- 172.21.15.77:0/1711040886 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8950035040 con 0x561fef748e40 2024-09-10T10:03:54.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.667+0000 7f896eca1740 1 -- 172.21.15.77:0/1711040886 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f895c01bb40 con 0x561fef748e40 2024-09-10T10:03:54.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.667+0000 7f894b400640 1 -- 172.21.15.77:0/1711040886 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f8950005450 con 0x561fef748e40 2024-09-10T10:03:54.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.668+0000 7f894b400640 1 -- 172.21.15.77:0/1711040886 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f895004f050 con 0x561fef748e40 2024-09-10T10:03:54.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.668+0000 7f896eca1740 10 client.27241 fetch_fsmap learned FSMap version 35 2024-09-10T10:03:54.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.668+0000 7f896eca1740 1 -- 172.21.15.77:0/1711040886 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x561fef761f80 con 0x561fef748e40 2024-09-10T10:03:54.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.668+0000 7f894b400640 1 -- 172.21.15.77:0/1711040886 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f895c01bb40 con 0x561fef748e40 2024-09-10T10:03:54.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.668+0000 7f894b400640 1 -- 172.21.15.77:0/1711040886 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f8950003920 con 0x561fef748e40 2024-09-10T10:03:54.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.668+0000 7f896eca1740 10 client.27241 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T10:03:54.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.668+0000 7f896eca1740 10 client.27241 resolve_mds: validated gid 24563 aka daemon mds.b 2024-09-10T10:03:54.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.668+0000 7f896eca1740 20 client.27241 populate_metadata read hostname 'smithi077' 2024-09-10T10:03:54.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.668+0000 7f896eca1740 1 --2- 172.21.15.77:0/1711040886 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x561fef763390 0x561fef765780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:54.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.668+0000 7f896eca1740 4 client.27241 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:03:54.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.668+0000 7f896eca1740 1 -- 172.21.15.77:0/1711040886 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561fef6bbbc0 con 0x561fef763390 2024-09-10T10:03:54.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.669+0000 7f896a200640 1 --2- 172.21.15.77:0/1711040886 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x561fef763390 0x561fef765780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:54.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.669+0000 7f896a200640 1 --2- 172.21.15.77:0/1711040886 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x561fef763390 0x561fef765780 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa309f42138f8ad7 server_cookie=13cce0d61046d5d2 in_seq=0 out_seq=0 2024-09-10T10:03:54.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.669+0000 7f894b400640 10 client.27241 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T10:03:54.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.670+0000 7f894b400640 1 -- 172.21.15.77:0/1711040886 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561fef6bbbc0 con 0x561fef763390 2024-09-10T10:03:54.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.670+0000 7f894b400640 10 client.27241 handle_command_reply: tid=0 2024-09-10T10:03:54.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.670+0000 7f896eca1740 1 client.27241 shutdown 2024-09-10T10:03:54.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.670+0000 7f896eca1740 1 -- 172.21.15.77:0/1711040886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fef748e40 msgr2=0x561fef750b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:54.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.670+0000 7f896eca1740 1 --2- 172.21.15.77:0/1711040886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fef748e40 0x561fef750b80 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f895002aa70 tx=0x7f8950004440 comp rx=0 tx=0).stop 2024-09-10T10:03:54.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.670+0000 7f896eca1740 1 -- 172.21.15.77:0/1711040886 shutdown_connections 2024-09-10T10:03:54.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.670+0000 7f896eca1740 1 --2- 172.21.15.77:0/1711040886 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x561fef763390 0x561fef765780 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:54.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.670+0000 7f896eca1740 1 --2- 172.21.15.77:0/1711040886 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561fef753700 0x561fef7510c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:54.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.670+0000 7f896eca1740 1 --2- 172.21.15.77:0/1711040886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fef748e40 0x561fef750b80 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:54.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.670+0000 7f896eca1740 1 --2- 172.21.15.77:0/1711040886 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561fef748520 0x561fef750640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:54.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.670+0000 7f896eca1740 1 -- 172.21.15.77:0/1711040886 >> 172.21.15.77:0/1711040886 conn(0x561fef744ab0 msgr2=0x561fef743cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:54.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.670+0000 7f894b400640 0 client.27241 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T10:03:54.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.671+0000 7f896eca1740 1 -- 172.21.15.77:0/1711040886 shutdown_connections 2024-09-10T10:03:54.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.671+0000 7f896eca1740 1 -- 172.21.15.77:0/1711040886 wait complete. 2024-09-10T10:03:54.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.671+0000 7f896eca1740 20 client.27241 trim_cache size 0 max 16384 2024-09-10T10:03:54.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.688+0000 7f896eca1740 1 Processor -- start 2024-09-10T10:03:54.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.688+0000 7f896eca1740 1 -- start start 2024-09-10T10:03:54.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.688+0000 7f896eca1740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561fef748520 0x561fef74af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:54.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.688+0000 7f896eca1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561fef748e40 0x561fef76b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:54.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.689+0000 7f896eca1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fef753700 0x561fef76b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:54.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.689+0000 7f896eca1740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f89641b0ed0 con 0x561fef748520 2024-09-10T10:03:54.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.689+0000 7f896eca1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f89641b2a10 con 0x561fef748e40 2024-09-10T10:03:54.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.689+0000 7f896eca1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f89641b15a0 con 0x561fef753700 2024-09-10T10:03:54.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.689+0000 7f896a200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561fef748520 0x561fef74af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:54.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.689+0000 7f896a200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561fef748520 0x561fef74af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:36154/0 (socket says 172.21.15.77:36154) 2024-09-10T10:03:54.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.689+0000 7f896a200640 1 -- 172.21.15.77:0/904677286 learned_addr learned my addr 172.21.15.77:0/904677286 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:03:54.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.689+0000 7f8968e00640 1 --2- 172.21.15.77:0/904677286 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561fef748e40 0x561fef76b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:54.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.689+0000 7f896a200640 1 -- 172.21.15.77:0/904677286 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fef753700 msgr2=0x561fef76b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:54.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.689+0000 7f896a200640 1 --2- 172.21.15.77:0/904677286 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fef753700 0x561fef76b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:54.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.689+0000 7f896a200640 1 -- 172.21.15.77:0/904677286 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561fef748e40 msgr2=0x561fef76b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:54.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.689+0000 7f896a200640 1 --2- 172.21.15.77:0/904677286 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561fef748e40 0x561fef76b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:54.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.689+0000 7f896a200640 1 -- 172.21.15.77:0/904677286 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fef765de0 con 0x561fef748520 2024-09-10T10:03:54.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.689+0000 7f896a200640 1 --2- 172.21.15.77:0/904677286 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561fef748520 0x561fef74af80 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f895c010d00 tx=0x7f895c012920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a685caae996699f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:54.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.690+0000 7f8956000640 1 -- 172.21.15.77:0/904677286 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f895c016030 con 0x561fef748520 2024-09-10T10:03:54.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.690+0000 7f8956000640 1 -- 172.21.15.77:0/904677286 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f895c025040 con 0x561fef748520 2024-09-10T10:03:54.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.690+0000 7f8956000640 1 -- 172.21.15.77:0/904677286 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f895c024040 con 0x561fef748520 2024-09-10T10:03:54.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.690+0000 7f896eca1740 1 -- 172.21.15.77:0/904677286 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561fef748520 msgr2=0x561fef74af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:54.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.690+0000 7f896eca1740 1 --2- 172.21.15.77:0/904677286 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561fef748520 0x561fef74af80 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f895c010d00 tx=0x7f895c012920 comp rx=0 tx=0).stop 2024-09-10T10:03:54.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.690+0000 7f896eca1740 1 -- 172.21.15.77:0/904677286 shutdown_connections 2024-09-10T10:03:54.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.690+0000 7f896eca1740 1 --2- 172.21.15.77:0/904677286 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fef753700 0x561fef76b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:54.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.690+0000 7f896eca1740 1 --2- 172.21.15.77:0/904677286 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561fef748e40 0x561fef76b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:54.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.690+0000 7f896eca1740 1 --2- 172.21.15.77:0/904677286 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561fef748520 0x561fef74af80 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:54.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.690+0000 7f896eca1740 1 -- 172.21.15.77:0/904677286 >> 172.21.15.77:0/904677286 conn(0x561fef744ab0 msgr2=0x561fef743cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:54.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.691+0000 7f896eca1740 1 -- 172.21.15.77:0/904677286 shutdown_connections 2024-09-10T10:03:54.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.691+0000 7f896eca1740 1 -- 172.21.15.77:0/904677286 wait complete. 2024-09-10T10:03:54.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.691+0000 7f896eca1740 1 -- start start 2024-09-10T10:03:54.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.691+0000 7f896eca1740 1 Processor -- start 2024-09-10T10:03:54.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.691+0000 7f896eca1740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561fef748520 0x561fef75bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:54.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.692+0000 7f896eca1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fef748e40 0x561fef7572d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:54.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.692+0000 7f896eca1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561fef753700 0x561fef757a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:54.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.692+0000 7f896eca1740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f89641b2340 con 0x561fef748520 2024-09-10T10:03:54.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.692+0000 7f896eca1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f89641b1c70 con 0x561fef753700 2024-09-10T10:03:54.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.692+0000 7f896eca1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f89641aeeb0 con 0x561fef748e40 2024-09-10T10:03:54.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.692+0000 7f896eca1740 10 client.0 _ll_register_callbacks cb 0x7f8950009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:03:54.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.692+0000 7f896a200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561fef748520 0x561fef75bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:54.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.692+0000 7f896a200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561fef748520 0x561fef75bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:36162/0 (socket says 172.21.15.77:36162) 2024-09-10T10:03:54.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.692+0000 7f896a200640 1 -- 172.21.15.77:0/1844473025 learned_addr learned my addr 172.21.15.77:0/1844473025 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:03:54.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.692+0000 7f8968e00640 1 --2- 172.21.15.77:0/1844473025 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fef748e40 0x561fef7572d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:54.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.692+0000 7f896a200640 1 --2- 172.21.15.77:0/1844473025 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561fef753700 0x561fef757a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:54.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.692+0000 7f896a200640 1 -- 172.21.15.77:0/1844473025 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fef748e40 msgr2=0x561fef7572d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:54.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.692+0000 7f896a200640 1 --2- 172.21.15.77:0/1844473025 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fef748e40 0x561fef7572d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:54.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.692+0000 7f896a200640 1 -- 172.21.15.77:0/1844473025 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561fef753700 msgr2=0x561fef757a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:54.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.692+0000 7f896a200640 1 --2- 172.21.15.77:0/1844473025 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561fef753700 0x561fef757a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:54.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.692+0000 7f896a200640 1 -- 172.21.15.77:0/1844473025 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fef7ccfe0 con 0x561fef748520 2024-09-10T10:03:54.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.693+0000 7f896a200640 1 --2- 172.21.15.77:0/1844473025 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561fef748520 0x561fef75bfa0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f895c009900 tx=0x7f895c012da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31c78baf8029e597 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:54.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.693+0000 7f894b400640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-10T10:03:54.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.693+0000 7f894b400640 1 -- 172.21.15.77:0/1844473025 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f895c024070 con 0x561fef748520 2024-09-10T10:03:54.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.693+0000 7f894b400640 1 -- 172.21.15.77:0/1844473025 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f895c004c00 con 0x561fef748520 2024-09-10T10:03:54.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.693+0000 7f896eca1740 1 -- 172.21.15.77:0/1844473025 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f895c025040 con 0x561fef748520 2024-09-10T10:03:54.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.693+0000 7f894b400640 1 -- 172.21.15.77:0/1844473025 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f895c044050 con 0x561fef748520 2024-09-10T10:03:54.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.693+0000 7f896eca1740 10 client.17151 fetch_fsmap learned FSMap version 35 2024-09-10T10:03:54.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.693+0000 7f896eca1740 1 -- 172.21.15.77:0/1844473025 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x561fef7ce1e0 con 0x561fef748520 2024-09-10T10:03:54.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.694+0000 7f894b400640 1 -- 172.21.15.77:0/1844473025 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f895c025040 con 0x561fef748520 2024-09-10T10:03:54.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.694+0000 7f894b400640 1 -- 172.21.15.77:0/1844473025 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f895c00f040 con 0x561fef748520 2024-09-10T10:03:54.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.694+0000 7f896eca1740 10 client.17151 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T10:03:54.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.694+0000 7f896eca1740 10 client.17151 resolve_mds: validated gid 24563 aka daemon mds.b 2024-09-10T10:03:54.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.694+0000 7f896eca1740 20 client.17151 populate_metadata read hostname 'smithi077' 2024-09-10T10:03:54.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.694+0000 7f896eca1740 1 --2- 172.21.15.77:0/1844473025 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x561fef7ce6a0 0x561fef7d0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:54.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.694+0000 7f896eca1740 4 client.17151 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "1f5021c0", "release": true, "await": true}] 2024-09-10T10:03:54.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.694+0000 7f896eca1740 1 -- 172.21.15.77:0/1844473025 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "quiesce db", "set_id": "1f5021c0", "release": true, "await": true}) -- 0x561fef6b60d0 con 0x561fef7ce6a0 2024-09-10T10:03:54.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.694+0000 7f8968e00640 1 --2- 172.21.15.77:0/1844473025 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x561fef7ce6a0 0x561fef7d0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:54.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.695+0000 7f8968e00640 1 --2- 172.21.15.77:0/1844473025 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x561fef7ce6a0 0x561fef7d0a90 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f06e3d85e95f726 server_cookie=e8064612c7af78fb in_seq=0 out_seq=0 2024-09-10T10:03:54.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.695+0000 7f894b400640 10 client.17151 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T10:03:54.784 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.781+0000 7f894b400640 1 -- 172.21.15.77:0/1844473025 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x561fef6b60d0 con 0x561fef7ce6a0 2024-09-10T10:03:54.784 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.781+0000 7f894b400640 10 client.17151 handle_command_reply: tid=0 2024-09-10T10:03:54.785 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.781+0000 7f896eca1740 1 client.17151 shutdown 2024-09-10T10:03:54.785 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.782+0000 7f896eca1740 1 -- 172.21.15.77:0/1844473025 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561fef748520 msgr2=0x561fef75bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:54.785 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.782+0000 7f896eca1740 1 --2- 172.21.15.77:0/1844473025 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561fef748520 0x561fef75bfa0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f895c009900 tx=0x7f895c012da0 comp rx=0 tx=0).stop 2024-09-10T10:03:54.785 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.782+0000 7f896eca1740 1 -- 172.21.15.77:0/1844473025 shutdown_connections 2024-09-10T10:03:54.785 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.782+0000 7f896eca1740 1 --2- 172.21.15.77:0/1844473025 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x561fef7ce6a0 0x561fef7d0a90 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:54.785 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.782+0000 7f896eca1740 1 --2- 172.21.15.77:0/1844473025 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561fef753700 0x561fef757a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:54.785 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.782+0000 7f896eca1740 1 --2- 172.21.15.77:0/1844473025 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fef748e40 0x561fef7572d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:54.785 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.782+0000 7f896eca1740 1 --2- 172.21.15.77:0/1844473025 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561fef748520 0x561fef75bfa0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:54.785 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.782+0000 7f896eca1740 1 -- 172.21.15.77:0/1844473025 >> 172.21.15.77:0/1844473025 conn(0x561fef744ab0 msgr2=0x561fef751cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:54.785 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.782+0000 7f894b400640 0 client.17151 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T10:03:54.785 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.782+0000 7f896eca1740 1 -- 172.21.15.77:0/1844473025 shutdown_connections 2024-09-10T10:03:54.785 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.782+0000 7f896eca1740 1 -- 172.21.15.77:0/1844473025 wait complete. 2024-09-10T10:03:54.785 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.782+0000 7f896eca1740 20 client.17151 trim_cache size 0 max 16384 2024-09-10T10:03:54.786 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-10T10:03:54.786 INFO:teuthology.orchestra.run.smithi077.stdout: "epoch": 35, 2024-09-10T10:03:54.786 INFO:teuthology.orchestra.run.smithi077.stdout: "leader": 24563, 2024-09-10T10:03:54.786 INFO:teuthology.orchestra.run.smithi077.stdout: "set_version": 28, 2024-09-10T10:03:54.786 INFO:teuthology.orchestra.run.smithi077.stdout: "sets": { 2024-09-10T10:03:54.786 INFO:teuthology.orchestra.run.smithi077.stdout: "1f5021c0": { 2024-09-10T10:03:54.786 INFO:teuthology.orchestra.run.smithi077.stdout: "version": 28, 2024-09-10T10:03:54.786 INFO:teuthology.orchestra.run.smithi077.stdout: "age_ref": 0.0, 2024-09-10T10:03:54.786 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-10T10:03:54.787 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RELEASED", 2024-09-10T10:03:54.787 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-10T10:03:54.787 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:03:54.787 INFO:teuthology.orchestra.run.smithi077.stdout: "timeout": 90.0, 2024-09-10T10:03:54.787 INFO:teuthology.orchestra.run.smithi077.stdout: "expiration": 133.4, 2024-09-10T10:03:54.787 INFO:teuthology.orchestra.run.smithi077.stdout: "members": { 2024-09-10T10:03:54.787 INFO:teuthology.orchestra.run.smithi077.stdout: "file:/": { 2024-09-10T10:03:54.787 INFO:teuthology.orchestra.run.smithi077.stdout: "excluded": false, 2024-09-10T10:03:54.787 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-10T10:03:54.787 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RELEASED", 2024-09-10T10:03:54.787 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-10T10:03:54.787 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:03:54.787 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:03:54.787 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:03:54.788 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:03:54.788 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:03:54.788 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-10T10:03:54.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.783+0000 7f896ac00640 1 -- 172.21.15.77:0/1093996953 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f894c0a2b80 msgr2=0x7f894c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:54.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.783+0000 7f896ac00640 1 --2- 172.21.15.77:0/1093996953 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f894c0a2b80 0x7f894c0a5040 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f89580a4460 tx=0x7f89580d5530 comp rx=0 tx=0).stop 2024-09-10T10:03:54.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.783+0000 7f896ac00640 1 -- 172.21.15.77:0/1093996953 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8964102b70 msgr2=0x7f896419a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:54.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.783+0000 7f896ac00640 1 --2- 172.21.15.77:0/1093996953 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8964102b70 0x7f896419a590 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f895800ea60 tx=0x7f895800ef20 comp rx=0 tx=0).stop 2024-09-10T10:03:54.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.783+0000 7f896ac00640 1 -- 172.21.15.77:0/1093996953 shutdown_connections 2024-09-10T10:03:54.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.783+0000 7f896ac00640 1 --2- 172.21.15.77:0/1093996953 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f894c0a2b80 0x7f894c0a5040 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:54.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.783+0000 7f896ac00640 1 --2- 172.21.15.77:0/1093996953 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f896410bfa0 0x7f896419ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:54.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.783+0000 7f896ac00640 1 --2- 172.21.15.77:0/1093996953 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8964103490 0x7f896419aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:54.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.783+0000 7f896ac00640 1 --2- 172.21.15.77:0/1093996953 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8964102b70 0x7f896419a590 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:54.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.783+0000 7f896ac00640 1 -- 172.21.15.77:0/1093996953 >> 172.21.15.77:0/1093996953 conn(0x7f89640fc5d0 msgr2=0x7f8964100700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:54.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.784+0000 7f896ac00640 1 -- 172.21.15.77:0/1093996953 shutdown_connections 2024-09-10T10:03:54.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:03:54.784+0000 7f896ac00640 1 -- 172.21.15.77:0/1093996953 wait complete. 2024-09-10T10:03:55.156 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:03:54 smithi077 ceph-mon[30319]: pgmap v653: 129 pgs: 129 active+clean; 150 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 384 KiB/s wr, 158 op/s 2024-09-10T10:03:55.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:03:54 smithi134 ceph-mon[35674]: pgmap v653: 129 pgs: 129 active+clean; 150 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 384 KiB/s wr, 158 op/s 2024-09-10T10:03:55.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:03:54 smithi120 ceph-mon[34770]: pgmap v653: 129 pgs: 129 active+clean; 150 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 384 KiB/s wr, 158 op/s 2024-09-10T10:03:55.572 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '1f5021c0', total seconds elapsed: 18.1 2024-09-10T10:03:56.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:03:55 smithi134 ceph-mon[35674]: scrub summary: idle 2024-09-10T10:03:56.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:03:55 smithi134 ceph-mon[35674]: scrub summary: idle 2024-09-10T10:03:56.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:03:55 smithi120 ceph-mon[34770]: scrub summary: idle 2024-09-10T10:03:56.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:03:55 smithi120 ceph-mon[34770]: scrub summary: idle 2024-09-10T10:03:56.440 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:03:55 smithi077 ceph-mon[30319]: scrub summary: idle 2024-09-10T10:03:56.440 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:03:55 smithi077 ceph-mon[30319]: scrub summary: idle 2024-09-10T10:03:57.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:03:56 smithi134 ceph-mon[35674]: scrub summary: idle 2024-09-10T10:03:57.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:03:56 smithi134 ceph-mon[35674]: scrub summary: idle 2024-09-10T10:03:57.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:03:56 smithi134 ceph-mon[35674]: scrub summary: idle 2024-09-10T10:03:57.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:03:56 smithi134 ceph-mon[35674]: pgmap v654: 129 pgs: 129 active+clean; 150 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 204 KiB/s wr, 92 op/s 2024-09-10T10:03:57.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:03:56 smithi120 ceph-mon[34770]: scrub summary: idle 2024-09-10T10:03:57.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:03:56 smithi120 ceph-mon[34770]: scrub summary: idle 2024-09-10T10:03:57.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:03:56 smithi120 ceph-mon[34770]: scrub summary: idle 2024-09-10T10:03:57.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:03:56 smithi120 ceph-mon[34770]: pgmap v654: 129 pgs: 129 active+clean; 150 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 204 KiB/s wr, 92 op/s 2024-09-10T10:03:57.441 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:03:56 smithi077 ceph-mon[30319]: scrub summary: idle 2024-09-10T10:03:57.441 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:03:56 smithi077 ceph-mon[30319]: scrub summary: idle 2024-09-10T10:03:57.441 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:03:56 smithi077 ceph-mon[30319]: scrub summary: idle 2024-09-10T10:03:57.441 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:03:56 smithi077 ceph-mon[30319]: pgmap v654: 129 pgs: 129 active+clean; 150 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 204 KiB/s wr, 92 op/s 2024-09-10T10:03:59.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:03:58 smithi134 ceph-mon[35674]: pgmap v655: 129 pgs: 129 active+clean; 145 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 801 KiB/s rd, 191 KiB/s wr, 70 op/s 2024-09-10T10:03:59.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:03:58 smithi120 ceph-mon[34770]: pgmap v655: 129 pgs: 129 active+clean; 145 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 801 KiB/s rd, 191 KiB/s wr, 70 op/s 2024-09-10T10:03:59.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:03:58 smithi077 ceph-mon[30319]: pgmap v655: 129 pgs: 129 active+clean; 145 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 801 KiB/s rd, 191 KiB/s wr, 70 op/s 2024-09-10T10:04:01.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:04:00 smithi134 ceph-mon[35674]: pgmap v656: 129 pgs: 129 active+clean; 148 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 384 KiB/s wr, 129 op/s 2024-09-10T10:04:01.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:04:00 smithi120 ceph-mon[34770]: pgmap v656: 129 pgs: 129 active+clean; 148 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 384 KiB/s wr, 129 op/s 2024-09-10T10:04:01.441 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:04:00 smithi077 ceph-mon[30319]: pgmap v656: 129 pgs: 129 active+clean; 148 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 384 KiB/s wr, 129 op/s 2024-09-10T10:04:03.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:04:02 smithi134 ceph-mon[35674]: pgmap v657: 129 pgs: 129 active+clean; 146 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 529 KiB/s wr, 185 op/s 2024-09-10T10:04:03.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:04:02 smithi120 ceph-mon[34770]: pgmap v657: 129 pgs: 129 active+clean; 146 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 529 KiB/s wr, 185 op/s 2024-09-10T10:04:03.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:04:02 smithi077 ceph-mon[30319]: pgmap v657: 129 pgs: 129 active+clean; 146 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 529 KiB/s wr, 185 op/s 2024-09-10T10:04:05.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:04:04 smithi134 ceph-mon[35674]: pgmap v658: 129 pgs: 129 active+clean; 143 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 715 KiB/s wr, 255 op/s 2024-09-10T10:04:05.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:04:04 smithi120 ceph-mon[34770]: pgmap v658: 129 pgs: 129 active+clean; 143 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 715 KiB/s wr, 255 op/s 2024-09-10T10:04:05.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:04:04 smithi077 ceph-mon[30319]: pgmap v658: 129 pgs: 129 active+clean; 143 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 715 KiB/s wr, 255 op/s 2024-09-10T10:04:07.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:04:06 smithi134 ceph-mon[35674]: pgmap v659: 129 pgs: 129 active+clean; 146 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 844 KiB/s wr, 304 op/s 2024-09-10T10:04:07.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:04:06 smithi120 ceph-mon[34770]: pgmap v659: 129 pgs: 129 active+clean; 146 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 844 KiB/s wr, 304 op/s 2024-09-10T10:04:07.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:04:06 smithi077 ceph-mon[30319]: pgmap v659: 129 pgs: 129 active+clean; 146 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 844 KiB/s wr, 304 op/s 2024-09-10T10:04:09.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:04:08 smithi134 ceph-mon[35674]: pgmap v660: 129 pgs: 129 active+clean; 147 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 1015 KiB/s wr, 368 op/s 2024-09-10T10:04:09.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:04:08 smithi120 ceph-mon[34770]: pgmap v660: 129 pgs: 129 active+clean; 147 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 1015 KiB/s wr, 368 op/s 2024-09-10T10:04:09.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:04:08 smithi077 ceph-mon[30319]: pgmap v660: 129 pgs: 129 active+clean; 147 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 1015 KiB/s wr, 368 op/s 2024-09-10T10:04:10.686 INFO:tasks.workunit.client.0.smithi077.stdout:All 50000 operations completed A-OK! 2024-09-10T10:04:10.722 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-10T10:04:10.722 DEBUG:teuthology.orchestra.run.smithi077:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-10T10:04:11.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:04:10 smithi077 ceph-mon[30319]: pgmap v661: 129 pgs: 129 active+clean; 157 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 1.2 MiB/s wr, 466 op/s 2024-09-10T10:04:11.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:04:10 smithi134 ceph-mon[35674]: pgmap v661: 129 pgs: 129 active+clean; 157 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 1.2 MiB/s wr, 466 op/s 2024-09-10T10:04:11.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:04:10 smithi120 ceph-mon[34770]: pgmap v661: 129 pgs: 129 active+clean; 157 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 1.2 MiB/s wr, 466 op/s 2024-09-10T10:04:13.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:04:12 smithi134 ceph-mon[35674]: pgmap v662: 129 pgs: 129 active+clean; 158 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 1.1 MiB/s wr, 432 op/s 2024-09-10T10:04:13.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:04:12 smithi120 ceph-mon[34770]: pgmap v662: 129 pgs: 129 active+clean; 158 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 1.1 MiB/s wr, 432 op/s 2024-09-10T10:04:13.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:04:12 smithi077 ceph-mon[30319]: pgmap v662: 129 pgs: 129 active+clean; 158 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 1.1 MiB/s wr, 432 op/s 2024-09-10T10:04:14.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:04:14 smithi134 ceph-mon[35674]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T10:04:14.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:04:14 smithi120 ceph-mon[34770]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T10:04:14.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:04:14 smithi077 ceph-mon[30319]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T10:04:15.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:04:15 smithi134 ceph-mon[35674]: pgmap v663: 129 pgs: 129 active+clean; 150 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 1.1 MiB/s wr, 421 op/s 2024-09-10T10:04:15.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:04:15 smithi120 ceph-mon[34770]: pgmap v663: 129 pgs: 129 active+clean; 150 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 1.1 MiB/s wr, 421 op/s 2024-09-10T10:04:15.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:04:15 smithi077 ceph-mon[30319]: pgmap v663: 129 pgs: 129 active+clean; 150 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 1.1 MiB/s wr, 421 op/s 2024-09-10T10:04:17.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:04:17 smithi134 ceph-mon[35674]: pgmap v664: 129 pgs: 129 active+clean; 143 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 1019 KiB/s wr, 377 op/s 2024-09-10T10:04:17.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:04:17 smithi134 ceph-mon[35674]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T10:04:17.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:04:17 smithi120 ceph-mon[34770]: pgmap v664: 129 pgs: 129 active+clean; 143 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 1019 KiB/s wr, 377 op/s 2024-09-10T10:04:17.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:04:17 smithi120 ceph-mon[34770]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T10:04:17.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:04:17 smithi077 ceph-mon[30319]: pgmap v664: 129 pgs: 129 active+clean; 143 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 1019 KiB/s wr, 377 op/s 2024-09-10T10:04:17.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:04:17 smithi077 ceph-mon[30319]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T10:04:17.789 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T10:04:18.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:04:18 smithi077 ceph-mon[30319]: pgmap v665: 129 pgs: 129 active+clean; 143 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.1 MiB/s wr, 369 op/s 2024-09-10T10:04:18.232 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:04:18.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:04:18 smithi134 ceph-mon[35674]: pgmap v665: 129 pgs: 129 active+clean; 143 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.1 MiB/s wr, 369 op/s 2024-09-10T10:04:18.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:04:18 smithi120 ceph-mon[34770]: pgmap v665: 129 pgs: 129 active+clean; 143 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.1 MiB/s wr, 369 op/s 2024-09-10T10:04:18.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.958+0000 7f9f48600640 1 -- 172.21.15.77:0/3693040904 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9f4010e420 msgr2=0x7f9f40110810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:18.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.958+0000 7f9f48600640 1 --2- 172.21.15.77:0/3693040904 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9f4010e420 0x7f9f40110810 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c009860 tx=0x7f9f2c02a3d0 comp rx=0 tx=0).stop 2024-09-10T10:04:18.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.959+0000 7f9f48600640 1 -- 172.21.15.77:0/3693040904 shutdown_connections 2024-09-10T10:04:18.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.959+0000 7f9f48600640 1 --2- 172.21.15.77:0/3693040904 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9f4010e420 0x7f9f40110810 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:18.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.959+0000 7f9f48600640 1 --2- 172.21.15.77:0/3693040904 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9f40074ee0 0x7f9f4010dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:18.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.959+0000 7f9f48600640 1 --2- 172.21.15.77:0/3693040904 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9f400745c0 0x7f9f400749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:18.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.959+0000 7f9f48600640 1 -- 172.21.15.77:0/3693040904 >> 172.21.15.77:0/3693040904 conn(0x7f9f400fd8e0 msgr2=0x7f9f400ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:18.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.959+0000 7f9f48600640 1 -- 172.21.15.77:0/3693040904 shutdown_connections 2024-09-10T10:04:18.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.959+0000 7f9f48600640 1 -- 172.21.15.77:0/3693040904 wait complete. 2024-09-10T10:04:18.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.960+0000 7f9f48600640 1 Processor -- start 2024-09-10T10:04:18.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.961+0000 7f9f48600640 1 -- start start 2024-09-10T10:04:18.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.963+0000 7f9f48600640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9f400745c0 0x7f9f4019a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:18.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.963+0000 7f9f48600640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9f40074ee0 0x7f9f4019ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:18.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.963+0000 7f9f48600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9f4010e420 0x7f9f4019eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:18.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.963+0000 7f9f48600640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9f401141a0 con 0x7f9f40074ee0 2024-09-10T10:04:18.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.963+0000 7f9f48600640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9f40114020 con 0x7f9f400745c0 2024-09-10T10:04:18.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.963+0000 7f9f48600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9f40114320 con 0x7f9f4010e420 2024-09-10T10:04:18.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.964+0000 7f9f45c00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9f400745c0 0x7f9f4019a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:18.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.964+0000 7f9f46600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9f4010e420 0x7f9f4019eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:18.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.964+0000 7f9f46600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9f4010e420 0x7f9f4019eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:52108/0 (socket says 172.21.15.77:52108) 2024-09-10T10:04:18.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.964+0000 7f9f46600640 1 -- 172.21.15.77:0/2365483737 learned_addr learned my addr 172.21.15.77:0/2365483737 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:04:18.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.965+0000 7f9f45c00640 1 -- 172.21.15.77:0/2365483737 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9f4010e420 msgr2=0x7f9f4019eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:18.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.965+0000 7f9f45c00640 1 --2- 172.21.15.77:0/2365483737 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9f4010e420 0x7f9f4019eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:18.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.965+0000 7f9f45c00640 1 -- 172.21.15.77:0/2365483737 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9f40074ee0 msgr2=0x7f9f4019ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:18.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.965+0000 7f9f45c00640 1 --2- 172.21.15.77:0/2365483737 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9f40074ee0 0x7f9f4019ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:18.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.965+0000 7f9f45c00640 1 -- 172.21.15.77:0/2365483737 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f4019f5b0 con 0x7f9f400745c0 2024-09-10T10:04:18.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.966+0000 7f9f45c00640 1 --2- 172.21.15.77:0/2365483737 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9f400745c0 0x7f9f4019a600 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f9f3801fd80 tx=0x7f9f3801fdb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:18.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.967+0000 7f9f3e000640 1 -- 172.21.15.77:0/2365483737 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f3802d070 con 0x7f9f400745c0 2024-09-10T10:04:18.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.968+0000 7f9f3e000640 1 -- 172.21.15.77:0/2365483737 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f3800cd00 con 0x7f9f400745c0 2024-09-10T10:04:18.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.968+0000 7f9f48600640 1 -- 172.21.15.77:0/2365483737 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f4019f8a0 con 0x7f9f400745c0 2024-09-10T10:04:18.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.968+0000 7f9f3e000640 1 -- 172.21.15.77:0/2365483737 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f38020870 con 0x7f9f400745c0 2024-09-10T10:04:18.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.968+0000 7f9f48600640 1 -- 172.21.15.77:0/2365483737 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f4006bc10 con 0x7f9f400745c0 2024-09-10T10:04:18.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.973+0000 7f9f4b07f740 1 Processor -- start 2024-09-10T10:04:18.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.973+0000 7f9f4b07f740 1 -- start start 2024-09-10T10:04:18.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.974+0000 7f9f4b07f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558288ec4b90 0x558288ec4f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:18.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.974+0000 7f9f4b07f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558288ec54b0 0x558288ecf830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:18.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.974+0000 7f9f4b07f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558288ecfd70 0x558288ed2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:18.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.974+0000 7f9f4b07f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558288e2b3e0 con 0x558288ec54b0 2024-09-10T10:04:18.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.974+0000 7f9f4b07f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558288dd2d80 con 0x558288ecfd70 2024-09-10T10:04:18.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.974+0000 7f9f4b07f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558288e273a0 con 0x558288ec4b90 2024-09-10T10:04:18.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.974+0000 7f9f46600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558288ec4b90 0x558288ec4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:18.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.974+0000 7f9f46600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558288ec4b90 0x558288ec4f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:52118/0 (socket says 172.21.15.77:52118) 2024-09-10T10:04:18.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.974+0000 7f9f46600640 1 -- 172.21.15.77:0/4128147696 learned_addr learned my addr 172.21.15.77:0/4128147696 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:04:18.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.976+0000 7f9f46600640 1 -- 172.21.15.77:0/4128147696 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558288ecfd70 msgr2=0x558288ed2160 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T10:04:18.982 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.976+0000 7f9f46600640 1 --2- 172.21.15.77:0/4128147696 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558288ecfd70 0x558288ed2160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:18.982 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.976+0000 7f9f3e000640 1 -- 172.21.15.77:0/2365483737 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9f38030070 con 0x7f9f400745c0 2024-09-10T10:04:18.982 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.976+0000 7f9f45c00640 1 --2- 172.21.15.77:0/4128147696 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558288ec54b0 0x558288ecf830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:18.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.979+0000 7f9f3e000640 1 --2- 172.21.15.77:0/2365483737 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f9f280a2c80 0x7f9f280a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:18.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.979+0000 7f9f3e000640 1 -- 172.21.15.77:0/2365483737 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f9f380e0010 con 0x7f9f400745c0 2024-09-10T10:04:18.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.980+0000 7f9f46600640 1 -- 172.21.15.77:0/4128147696 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558288ec54b0 msgr2=0x558288ecf830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:18.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.980+0000 7f9f46600640 1 --2- 172.21.15.77:0/4128147696 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558288ec54b0 0x558288ecf830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:18.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.980+0000 7f9f46600640 1 -- 172.21.15.77:0/4128147696 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558288df1ae0 con 0x558288ec4b90 2024-09-10T10:04:18.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.980+0000 7f9f45c00640 1 --2- 172.21.15.77:0/4128147696 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558288ec54b0 0x558288ecf830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:04:18.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.981+0000 7f9f46600640 1 --2- 172.21.15.77:0/4128147696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558288ec4b90 0x558288ec4f70 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c030bf0 tx=0x7f9f2c030c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ef523e9443e3596 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:18.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.982+0000 7f9f32a00640 1 -- 172.21.15.77:0/4128147696 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c004250 con 0x558288ec4b90 2024-09-10T10:04:18.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.982+0000 7f9f32a00640 1 -- 172.21.15.77:0/4128147696 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f2c041070 con 0x558288ec4b90 2024-09-10T10:04:18.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.982+0000 7f9f32a00640 1 -- 172.21.15.77:0/4128147696 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c0046c0 con 0x558288ec4b90 2024-09-10T10:04:18.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.982+0000 7f9f46600640 1 --2- 172.21.15.77:0/2365483737 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f9f280a2c80 0x7f9f280a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:18.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.984+0000 7f9f4b07f740 1 -- 172.21.15.77:0/4128147696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558288ec4b90 msgr2=0x558288ec4f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:18.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.984+0000 7f9f4b07f740 1 --2- 172.21.15.77:0/4128147696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558288ec4b90 0x558288ec4f70 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c030bf0 tx=0x7f9f2c030c20 comp rx=0 tx=0).stop 2024-09-10T10:04:18.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.984+0000 7f9f46600640 1 --2- 172.21.15.77:0/2365483737 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f9f280a2c80 0x7f9f280a5140 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c046000 tx=0x7f9f2c004860 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:18.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.987+0000 7f9f4b07f740 1 -- 172.21.15.77:0/4128147696 shutdown_connections 2024-09-10T10:04:18.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.987+0000 7f9f4b07f740 1 --2- 172.21.15.77:0/4128147696 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558288ecfd70 0x558288ed2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:18.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.987+0000 7f9f4b07f740 1 --2- 172.21.15.77:0/4128147696 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558288ec54b0 0x558288ecf830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:18.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.987+0000 7f9f4b07f740 1 --2- 172.21.15.77:0/4128147696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558288ec4b90 0x558288ec4f70 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:18.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.987+0000 7f9f4b07f740 1 -- 172.21.15.77:0/4128147696 >> 172.21.15.77:0/4128147696 conn(0x558288ec1120 msgr2=0x558288ec1500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:18.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.987+0000 7f9f4b07f740 1 -- 172.21.15.77:0/4128147696 shutdown_connections 2024-09-10T10:04:18.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.987+0000 7f9f4b07f740 1 -- 172.21.15.77:0/4128147696 wait complete. 2024-09-10T10:04:18.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.988+0000 7f9f4b07f740 1 -- start start 2024-09-10T10:04:18.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.988+0000 7f9f4b07f740 1 Processor -- start 2024-09-10T10:04:18.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.989+0000 7f9f4b07f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558288ec4b90 0x558288ecccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:18.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.989+0000 7f9f4b07f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558288ec54b0 0x558288ecd230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:18.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.989+0000 7f9f4b07f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558288ecfd70 0x558288ecd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:18.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.989+0000 7f9f4b07f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558288edb1f0 con 0x558288ec4b90 2024-09-10T10:04:18.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.989+0000 7f9f4b07f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558288de60f0 con 0x558288ecfd70 2024-09-10T10:04:18.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.989+0000 7f9f4b07f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558288edb370 con 0x558288ec54b0 2024-09-10T10:04:18.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.989+0000 7f9f4b07f740 10 client.0 _ll_register_callbacks cb 0x558288dd79e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:04:18.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.990+0000 7f9f46600640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558288ecfd70 0x558288ecd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:18.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.990+0000 7f9f46600640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558288ecfd70 0x558288ecd770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:41028/0 (socket says 172.21.15.77:41028) 2024-09-10T10:04:18.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.990+0000 7f9f46600640 1 -- 172.21.15.77:0/3469317446 learned_addr learned my addr 172.21.15.77:0/3469317446 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:04:18.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.990+0000 7f9f45c00640 1 --2- 172.21.15.77:0/3469317446 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558288ec4b90 0x558288ecccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:18.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.990+0000 7f9f45200640 1 --2- 172.21.15.77:0/3469317446 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558288ec54b0 0x558288ecd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:18.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.991+0000 7f9f46600640 1 -- 172.21.15.77:0/3469317446 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558288ec54b0 msgr2=0x558288ecd230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:18.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.991+0000 7f9f46600640 1 --2- 172.21.15.77:0/3469317446 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558288ec54b0 0x558288ecd230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:18.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.991+0000 7f9f46600640 1 -- 172.21.15.77:0/3469317446 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558288ec4b90 msgr2=0x558288ecccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:18.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.991+0000 7f9f46600640 1 --2- 172.21.15.77:0/3469317446 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558288ec4b90 0x558288ecccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:18.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.991+0000 7f9f46600640 1 -- 172.21.15.77:0/3469317446 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558288b9c3c0 con 0x558288ecfd70 2024-09-10T10:04:18.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.991+0000 7f9f46600640 1 --2- 172.21.15.77:0/3469317446 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558288ecfd70 0x558288ecd770 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c0045b0 tx=0x7f9f2c04f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dae975433e80e10d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:18.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.991+0000 7f9f27400640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T10:04:18.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.992+0000 7f9f27400640 1 -- 172.21.15.77:0/3469317446 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c004c90 con 0x558288ecfd70 2024-09-10T10:04:18.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.992+0000 7f9f27400640 1 -- 172.21.15.77:0/3469317446 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f9f2c003c00 con 0x558288ecfd70 2024-09-10T10:04:18.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.993+0000 7f9f4b07f740 1 -- 172.21.15.77:0/3469317446 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f2c041070 con 0x558288ecfd70 2024-09-10T10:04:18.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.993+0000 7f9f27400640 1 -- 172.21.15.77:0/3469317446 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f9f2c067050 con 0x558288ecfd70 2024-09-10T10:04:18.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.993+0000 7f9f4b07f740 10 client.27308 fetch_fsmap learned FSMap version 38 2024-09-10T10:04:18.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.993+0000 7f9f4b07f740 1 -- 172.21.15.77:0/3469317446 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x558288edb1f0 con 0x558288ecfd70 2024-09-10T10:04:18.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.993+0000 7f9f27400640 1 -- 172.21.15.77:0/3469317446 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f2c041070 con 0x558288ecfd70 2024-09-10T10:04:19.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.994+0000 7f9f27400640 1 -- 172.21.15.77:0/3469317446 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 38) ==== 8390+0+0 (secure 0 0 0) 0x7f9f2c047480 con 0x558288ecfd70 2024-09-10T10:04:19.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.995+0000 7f9f4b07f740 10 client.27308 fetch_fsmap finished waiting for FSMap version 38 2024-09-10T10:04:19.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.995+0000 7f9f4b07f740 10 client.27308 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T10:04:19.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.995+0000 7f9f4b07f740 20 client.27308 populate_metadata read hostname 'smithi077' 2024-09-10T10:04:19.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.995+0000 7f9f4b07f740 1 --2- 172.21.15.77:0/3469317446 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x558288ee8010 0x558288eea400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:19.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.995+0000 7f9f4b07f740 4 client.27308 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:04:19.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.995+0000 7f9f4b07f740 1 -- 172.21.15.77:0/3469317446 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558288e38950 con 0x558288ee8010 2024-09-10T10:04:19.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.995+0000 7f9f45c00640 1 --2- 172.21.15.77:0/3469317446 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x558288ee8010 0x558288eea400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:19.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.996+0000 7f9f27400640 10 client.27308 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T10:04:19.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.996+0000 7f9f45c00640 1 --2- 172.21.15.77:0/3469317446 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x558288ee8010 0x558288eea400 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c26011215892560 server_cookie=6ee33131cfda8871 in_seq=0 out_seq=0 2024-09-10T10:04:19.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.997+0000 7f9f27400640 1 -- 172.21.15.77:0/3469317446 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558288e38950 con 0x558288ee8010 2024-09-10T10:04:19.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.997+0000 7f9f27400640 10 client.27308 handle_command_reply: tid=0 2024-09-10T10:04:19.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.997+0000 7f9f4b07f740 1 client.27308 shutdown 2024-09-10T10:04:19.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.998+0000 7f9f4b07f740 1 -- 172.21.15.77:0/3469317446 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558288ecfd70 msgr2=0x558288ecd770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:19.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.998+0000 7f9f4b07f740 1 --2- 172.21.15.77:0/3469317446 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558288ecfd70 0x558288ecd770 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c0045b0 tx=0x7f9f2c04f000 comp rx=0 tx=0).stop 2024-09-10T10:04:19.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.998+0000 7f9f4b07f740 1 -- 172.21.15.77:0/3469317446 shutdown_connections 2024-09-10T10:04:19.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.998+0000 7f9f4b07f740 1 --2- 172.21.15.77:0/3469317446 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x558288ee8010 0x558288eea400 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:19.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.998+0000 7f9f4b07f740 1 --2- 172.21.15.77:0/3469317446 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558288ecfd70 0x558288ecd770 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:19.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.998+0000 7f9f4b07f740 1 --2- 172.21.15.77:0/3469317446 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558288ec54b0 0x558288ecd230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:19.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.998+0000 7f9f4b07f740 1 --2- 172.21.15.77:0/3469317446 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558288ec4b90 0x558288ecccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:19.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.998+0000 7f9f4b07f740 1 -- 172.21.15.77:0/3469317446 >> 172.21.15.77:0/3469317446 conn(0x558288ec1120 msgr2=0x558288ec6e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:19.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.998+0000 7f9f27400640 0 client.27308 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T10:04:19.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.999+0000 7f9f4b07f740 1 -- 172.21.15.77:0/3469317446 shutdown_connections 2024-09-10T10:04:19.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.999+0000 7f9f4b07f740 1 -- 172.21.15.77:0/3469317446 wait complete. 2024-09-10T10:04:19.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:18.999+0000 7f9f4b07f740 20 client.27308 trim_cache size 0 max 16384 2024-09-10T10:04:19.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:19.035+0000 7f9f4b07f740 1 Processor -- start 2024-09-10T10:04:19.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:19.035+0000 7f9f4b07f740 1 -- start start 2024-09-10T10:04:19.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:19.036+0000 7f9f4b07f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558288ec4b90 0x558288ecb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:19.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:19.036+0000 7f9f4b07f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558288ec54b0 0x558288ecd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:19.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:19.036+0000 7f9f4b07f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558288ecfd70 0x558288ec00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:19.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:19.036+0000 7f9f4b07f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9f40071720 con 0x558288ecfd70 2024-09-10T10:04:19.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:19.036+0000 7f9f4b07f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9f40073320 con 0x558288ec4b90 2024-09-10T10:04:19.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:19.036+0000 7f9f4b07f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9f40071e50 con 0x558288ec54b0 2024-09-10T10:04:19.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:19.037+0000 7f9f46600640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558288ecfd70 0x558288ec00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:19.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:19.037+0000 7f9f46600640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558288ecfd70 0x558288ec00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56778/0 (socket says 172.21.15.77:56778) 2024-09-10T10:04:19.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:19.037+0000 7f9f46600640 1 -- 172.21.15.77:0/2202516294 learned_addr learned my addr 172.21.15.77:0/2202516294 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:04:19.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:19.037+0000 7f9f45c00640 1 --2- 172.21.15.77:0/2202516294 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558288ec4b90 0x558288ecb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:19.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:19.037+0000 7f9f45200640 1 --2- 172.21.15.77:0/2202516294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558288ec54b0 0x558288ecd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:19.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:19.037+0000 7f9f46600640 1 -- 172.21.15.77:0/2202516294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558288ec54b0 msgr2=0x558288ecd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:19.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:19.037+0000 7f9f46600640 1 --2- 172.21.15.77:0/2202516294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558288ec54b0 0x558288ecd5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:19.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:19.037+0000 7f9f46600640 1 -- 172.21.15.77:0/2202516294 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558288ec4b90 msgr2=0x558288ecb5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:19.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:19.037+0000 7f9f46600640 1 --2- 172.21.15.77:0/2202516294 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558288ec4b90 0x558288ecb5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:19.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:19.038+0000 7f9f46600640 1 -- 172.21.15.77:0/2202516294 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558288eea770 con 0x558288ecfd70 2024-09-10T10:04:19.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:19.038+0000 7f9f45200640 1 --2- 172.21.15.77:0/2202516294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558288ec54b0 0x558288ecd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:04:19.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:19.038+0000 7f9f45c00640 1 --2- 172.21.15.77:0/2202516294 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558288ec4b90 0x558288ecb5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:04:19.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:19.038+0000 7f9f46600640 1 --2- 172.21.15.77:0/2202516294 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558288ecfd70 0x558288ec00d0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c009780 tx=0x7f9f2c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e09cf64bbf6b8cdb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:19.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:19.038+0000 7f9f32000640 1 -- 172.21.15.77:0/2202516294 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c0046c0 con 0x558288ecfd70 2024-09-10T10:04:19.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:19.039+0000 7f9f32000640 1 -- 172.21.15.77:0/2202516294 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f2c041040 con 0x558288ecfd70 2024-09-10T10:04:19.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:19.039+0000 7f9f32000640 1 -- 172.21.15.77:0/2202516294 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c005040 con 0x558288ecfd70 2024-09-10T10:04:19.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:19.040+0000 7f9f4b07f740 1 -- 172.21.15.77:0/2202516294 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558288ecfd70 msgr2=0x558288ec00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:19.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:19.040+0000 7f9f4b07f740 1 --2- 172.21.15.77:0/2202516294 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558288ecfd70 0x558288ec00d0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c009780 tx=0x7f9f2c000bc0 comp rx=0 tx=0).stop 2024-09-10T10:04:19.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:19.040+0000 7f9f4b07f740 1 -- 172.21.15.77:0/2202516294 shutdown_connections 2024-09-10T10:04:19.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:19.040+0000 7f9f4b07f740 1 --2- 172.21.15.77:0/2202516294 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558288ecfd70 0x558288ec00d0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:19.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:19.040+0000 7f9f4b07f740 1 --2- 172.21.15.77:0/2202516294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558288ec54b0 0x558288ecd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:19.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:19.040+0000 7f9f4b07f740 1 --2- 172.21.15.77:0/2202516294 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558288ec4b90 0x558288ecb5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:19.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:19.040+0000 7f9f4b07f740 1 -- 172.21.15.77:0/2202516294 >> 172.21.15.77:0/2202516294 conn(0x558288ec1120 msgr2=0x558288ebf9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:19.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:19.041+0000 7f9f4b07f740 1 -- 172.21.15.77:0/2202516294 shutdown_connections 2024-09-10T10:04:19.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:19.041+0000 7f9f4b07f740 1 -- 172.21.15.77:0/2202516294 wait complete. 2024-09-10T10:04:19.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:19.042+0000 7f9f4b07f740 1 -- start start 2024-09-10T10:04:19.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:19.042+0000 7f9f4b07f740 1 Processor -- start 2024-09-10T10:04:19.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:19.043+0000 7f9f4b07f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558288ec4b90 0x558288ee0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:19.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:19.044+0000 7f9f4b07f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558288ec54b0 0x558288ec9920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:19.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:19.044+0000 7f9f4b07f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558288ecfd70 0x558288ec9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:19.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:19.044+0000 7f9f4b07f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9f40072c50 con 0x558288ec54b0 2024-09-10T10:04:19.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:19.044+0000 7f9f4b07f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9f40072550 con 0x558288ec4b90 2024-09-10T10:04:19.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:19.044+0000 7f9f4b07f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9f4006f720 con 0x558288ecfd70 2024-09-10T10:04:19.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:19.044+0000 7f9f4b07f740 10 client.0 _ll_register_callbacks cb 0x7f9f2c030780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:04:19.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:19.044+0000 7f9f46600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558288ecfd70 0x558288ec9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:19.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:19.044+0000 7f9f45c00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558288ec4b90 0x558288ee0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:19.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:19.044+0000 7f9f46600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558288ecfd70 0x558288ec9e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:52152/0 (socket says 172.21.15.77:52152) 2024-09-10T10:04:19.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:19.044+0000 7f9f46600640 1 -- 172.21.15.77:0/1933630073 learned_addr learned my addr 172.21.15.77:0/1933630073 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:04:19.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:19.044+0000 7f9f46600640 1 -- 172.21.15.77:0/1933630073 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558288ec4b90 msgr2=0x558288ee0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:19.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:19.044+0000 7f9f46600640 1 --2- 172.21.15.77:0/1933630073 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558288ec4b90 0x558288ee0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:19.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:19.044+0000 7f9f46600640 1 -- 172.21.15.77:0/1933630073 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558288ec54b0 msgr2=0x558288ec9920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:19.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:19.044+0000 7f9f46600640 1 --2- 172.21.15.77:0/1933630073 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558288ec54b0 0x558288ec9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:19.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:19.044+0000 7f9f46600640 1 -- 172.21.15.77:0/1933630073 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558288eca550 con 0x558288ecfd70 2024-09-10T10:04:19.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:19.045+0000 7f9f46600640 1 --2- 172.21.15.77:0/1933630073 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558288ecfd70 0x558288ec9e60 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c030730 tx=0x7f9f2c0477a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b775c66a4c96a3a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:19.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:19.045+0000 7f9f27400640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-10T10:04:19.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:19.045+0000 7f9f27400640 1 -- 172.21.15.77:0/1933630073 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c047480 con 0x558288ecfd70 2024-09-10T10:04:19.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:19.046+0000 7f9f4b07f740 1 -- 172.21.15.77:0/1933630073 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f2c041040 con 0x558288ecfd70 2024-09-10T10:04:19.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:19.046+0000 7f9f27400640 1 -- 172.21.15.77:0/1933630073 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f9f2c05a040 con 0x558288ecfd70 2024-09-10T10:04:19.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:19.046+0000 7f9f27400640 1 -- 172.21.15.77:0/1933630073 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f9f2c07d050 con 0x558288ecfd70 2024-09-10T10:04:19.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:19.046+0000 7f9f4b07f740 10 client.27259 fetch_fsmap learned FSMap version 38 2024-09-10T10:04:19.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:19.046+0000 7f9f4b07f740 1 -- 172.21.15.77:0/1933630073 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x558288f52a30 con 0x558288ecfd70 2024-09-10T10:04:19.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:19.046+0000 7f9f27400640 1 -- 172.21.15.77:0/1933630073 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f2c041040 con 0x558288ecfd70 2024-09-10T10:04:19.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:19.047+0000 7f9f27400640 1 -- 172.21.15.77:0/1933630073 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 38) ==== 8390+0+0 (secure 0 0 0) 0x7f9f2c0040e0 con 0x558288ecfd70 2024-09-10T10:04:19.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:19.047+0000 7f9f4b07f740 10 client.27259 fetch_fsmap finished waiting for FSMap version 38 2024-09-10T10:04:19.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:19.047+0000 7f9f4b07f740 10 client.27259 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T10:04:19.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:19.047+0000 7f9f4b07f740 20 client.27259 populate_metadata read hostname 'smithi077' 2024-09-10T10:04:19.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:19.047+0000 7f9f4b07f740 1 --2- 172.21.15.77:0/1933630073 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x558288f530d0 0x558288f554f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:19.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:19.047+0000 7f9f4b07f740 4 client.27259 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T10:04:19.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:19.047+0000 7f9f4b07f740 1 -- 172.21.15.77:0/1933630073 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558288e32e60 con 0x558288f530d0 2024-09-10T10:04:19.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:19.047+0000 7f9f45c00640 1 --2- 172.21.15.77:0/1933630073 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x558288f530d0 0x558288f554f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:19.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:19.048+0000 7f9f45c00640 1 --2- 172.21.15.77:0/1933630073 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x558288f530d0 0x558288f554f0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b3401f5fa0ad75 server_cookie=53f4fef3a349a99a in_seq=0 out_seq=0 2024-09-10T10:04:19.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:19.048+0000 7f9f27400640 10 client.27259 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T10:04:19.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:19.048+0000 7f9f27400640 1 -- 172.21.15.77:0/1933630073 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x558288e32e60 con 0x558288f530d0 2024-09-10T10:04:19.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:19.048+0000 7f9f27400640 10 client.27259 handle_command_reply: tid=0 2024-09-10T10:04:19.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:19.048+0000 7f9f4b07f740 1 client.27259 shutdown 2024-09-10T10:04:19.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:19.049+0000 7f9f4b07f740 1 -- 172.21.15.77:0/1933630073 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558288ecfd70 msgr2=0x558288ec9e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:19.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:19.049+0000 7f9f4b07f740 1 --2- 172.21.15.77:0/1933630073 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558288ecfd70 0x558288ec9e60 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c030730 tx=0x7f9f2c0477a0 comp rx=0 tx=0).stop 2024-09-10T10:04:19.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:19.049+0000 7f9f4b07f740 1 -- 172.21.15.77:0/1933630073 shutdown_connections 2024-09-10T10:04:19.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:19.049+0000 7f9f4b07f740 1 --2- 172.21.15.77:0/1933630073 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x558288f530d0 0x558288f554f0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:19.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:19.049+0000 7f9f27400640 0 client.27259 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T10:04:19.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:19.049+0000 7f9f4b07f740 1 --2- 172.21.15.77:0/1933630073 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558288ecfd70 0x558288ec9e60 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:19.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:19.049+0000 7f9f4b07f740 1 --2- 172.21.15.77:0/1933630073 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558288ec54b0 0x558288ec9920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:19.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:19.049+0000 7f9f4b07f740 1 --2- 172.21.15.77:0/1933630073 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558288ec4b90 0x558288ee0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:19.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:19.049+0000 7f9f4b07f740 1 -- 172.21.15.77:0/1933630073 >> 172.21.15.77:0/1933630073 conn(0x558288ec1120 msgr2=0x558288ec01e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:19.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:19.049+0000 7f9f4b07f740 1 -- 172.21.15.77:0/1933630073 shutdown_connections 2024-09-10T10:04:19.052 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-10T10:04:19.052 INFO:teuthology.orchestra.run.smithi077.stdout: "status": "no active scrubs running", 2024-09-10T10:04:19.052 INFO:teuthology.orchestra.run.smithi077.stdout: "scrubs": {} 2024-09-10T10:04:19.052 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-10T10:04:19.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:19.049+0000 7f9f4b07f740 1 -- 172.21.15.77:0/1933630073 wait complete. 2024-09-10T10:04:19.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:19.049+0000 7f9f4b07f740 20 client.27259 trim_cache size 0 max 16384 2024-09-10T10:04:19.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:19.050+0000 7f9f48600640 1 -- 172.21.15.77:0/2365483737 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f9f280a2c80 msgr2=0x7f9f280a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:19.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:19.050+0000 7f9f48600640 1 --2- 172.21.15.77:0/2365483737 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f9f280a2c80 0x7f9f280a5140 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c046000 tx=0x7f9f2c004860 comp rx=0 tx=0).stop 2024-09-10T10:04:19.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:19.050+0000 7f9f48600640 1 -- 172.21.15.77:0/2365483737 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9f400745c0 msgr2=0x7f9f4019a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:19.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:19.050+0000 7f9f48600640 1 --2- 172.21.15.77:0/2365483737 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9f400745c0 0x7f9f4019a600 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f9f3801fd80 tx=0x7f9f3801fdb0 comp rx=0 tx=0).stop 2024-09-10T10:04:19.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:19.050+0000 7f9f48600640 1 -- 172.21.15.77:0/2365483737 shutdown_connections 2024-09-10T10:04:19.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:19.050+0000 7f9f48600640 1 --2- 172.21.15.77:0/2365483737 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f9f280a2c80 0x7f9f280a5140 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:19.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:19.050+0000 7f9f48600640 1 --2- 172.21.15.77:0/2365483737 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9f4010e420 0x7f9f4019eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:19.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:19.050+0000 7f9f48600640 1 --2- 172.21.15.77:0/2365483737 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9f40074ee0 0x7f9f4019ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:19.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:19.050+0000 7f9f48600640 1 --2- 172.21.15.77:0/2365483737 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9f400745c0 0x7f9f4019a600 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:19.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:19.050+0000 7f9f48600640 1 -- 172.21.15.77:0/2365483737 >> 172.21.15.77:0/2365483737 conn(0x7f9f400fd8e0 msgr2=0x7f9f400fe4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:19.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:19.051+0000 7f9f48600640 1 -- 172.21.15.77:0/2365483737 shutdown_connections 2024-09-10T10:04:19.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:19.051+0000 7f9f48600640 1 -- 172.21.15.77:0/2365483737 wait complete. 2024-09-10T10:04:19.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:04:19 smithi134 ceph-mon[35674]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T10:04:19.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:04:19 smithi120 ceph-mon[34770]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T10:04:19.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:04:19 smithi077 ceph-mon[30319]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T10:04:19.910 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T10:04:19.910 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:04:20.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:04:20 smithi077 ceph-mon[30319]: pgmap v666: 129 pgs: 129 active+clean; 141 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.2 MiB/s wr, 410 op/s 2024-09-10T10:04:20.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:04:20 smithi134 ceph-mon[35674]: pgmap v666: 129 pgs: 129 active+clean; 141 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.2 MiB/s wr, 410 op/s 2024-09-10T10:04:20.397 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:04:20.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:04:20 smithi120 ceph-mon[34770]: pgmap v666: 129 pgs: 129 active+clean; 141 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.2 MiB/s wr, 410 op/s 2024-09-10T10:04:20.459 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 24.6 2024-09-10T10:04:20.459 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:04:21.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:21.140+0000 7f0548a00640 1 -- 172.21.15.77:0/1816550424 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0544109b10 msgr2=0x7f0544114570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:21.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:21.140+0000 7f0548a00640 1 --2- 172.21.15.77:0/1816550424 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0544109b10 0x7f0544114570 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f0530009900 tx=0x7f053002a480 comp rx=0 tx=0).stop 2024-09-10T10:04:21.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:21.140+0000 7f0548a00640 1 -- 172.21.15.77:0/1816550424 shutdown_connections 2024-09-10T10:04:21.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:21.140+0000 7f0548a00640 1 --2- 172.21.15.77:0/1816550424 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0544114ab0 0x7f0544116ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:21.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:21.140+0000 7f0548a00640 1 --2- 172.21.15.77:0/1816550424 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0544109b10 0x7f0544114570 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:21.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:21.141+0000 7f0548a00640 1 --2- 172.21.15.77:0/1816550424 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05441091f0 0x7f05441095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:21.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:21.141+0000 7f0548a00640 1 -- 172.21.15.77:0/1816550424 >> 172.21.15.77:0/1816550424 conn(0x7f054406e140 msgr2=0x7f054406e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:21.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:21.141+0000 7f0548a00640 1 -- 172.21.15.77:0/1816550424 shutdown_connections 2024-09-10T10:04:21.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:21.141+0000 7f0548a00640 1 -- 172.21.15.77:0/1816550424 wait complete. 2024-09-10T10:04:21.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:21.142+0000 7f0548a00640 1 Processor -- start 2024-09-10T10:04:21.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:21.142+0000 7f0548a00640 1 -- start start 2024-09-10T10:04:21.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:21.143+0000 7f0548a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f05441091f0 0x7f05441140a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:21.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:21.143+0000 7f0548a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0544109b10 0x7f054410e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:21.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:21.143+0000 7f0548a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0544114ab0 0x7f054410e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:21.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:21.144+0000 7f0548a00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f05441185f0 con 0x7f05441091f0 2024-09-10T10:04:21.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:21.144+0000 7f0548a00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0544118470 con 0x7f0544114ab0 2024-09-10T10:04:21.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:21.144+0000 7f0548a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0544118770 con 0x7f0544109b10 2024-09-10T10:04:21.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:21.144+0000 7f0542800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0544114ab0 0x7f054410e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:21.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:21.144+0000 7f0542800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0544114ab0 0x7f054410e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:41048/0 (socket says 172.21.15.77:41048) 2024-09-10T10:04:21.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:21.144+0000 7f0542800640 1 -- 172.21.15.77:0/1882081068 learned_addr learned my addr 172.21.15.77:0/1882081068 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:04:21.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:21.144+0000 7f0541400640 1 --2- 172.21.15.77:0/1882081068 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0544109b10 0x7f054410e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:21.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:21.145+0000 7f0542800640 1 -- 172.21.15.77:0/1882081068 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0544109b10 msgr2=0x7f054410e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:21.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:21.145+0000 7f0542800640 1 --2- 172.21.15.77:0/1882081068 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0544109b10 0x7f054410e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:21.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:21.145+0000 7f0542800640 1 -- 172.21.15.77:0/1882081068 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f05441091f0 msgr2=0x7f05441140a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:21.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:21.145+0000 7f0542800640 1 --2- 172.21.15.77:0/1882081068 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f05441091f0 0x7f05441140a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:21.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:21.145+0000 7f0542800640 1 -- 172.21.15.77:0/1882081068 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f054410ec50 con 0x7f0544114ab0 2024-09-10T10:04:21.148 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:21.146+0000 7f0542800640 1 --2- 172.21.15.77:0/1882081068 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0544114ab0 0x7f054410e660 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f053c00dac0 tx=0x7f053c00df80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:21.148 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:21.147+0000 7f053aa00640 1 -- 172.21.15.77:0/1882081068 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f053c017030 con 0x7f0544114ab0 2024-09-10T10:04:21.148 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:21.147+0000 7f053aa00640 1 -- 172.21.15.77:0/1882081068 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f053c003e50 con 0x7f0544114ab0 2024-09-10T10:04:21.148 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:21.147+0000 7f053aa00640 1 -- 172.21.15.77:0/1882081068 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f053c010460 con 0x7f0544114ab0 2024-09-10T10:04:21.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:21.148+0000 7f0548a00640 1 -- 172.21.15.77:0/1882081068 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f054410ef40 con 0x7f0544114ab0 2024-09-10T10:04:21.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:21.148+0000 7f0548a00640 1 -- 172.21.15.77:0/1882081068 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05441bc670 con 0x7f0544114ab0 2024-09-10T10:04:21.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:21.154+0000 7f053aa00640 1 -- 172.21.15.77:0/1882081068 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f053c01a070 con 0x7f0544114ab0 2024-09-10T10:04:21.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:21.158+0000 7f0548a00640 1 -- 172.21.15.77:0/1882081068 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0544073a40 con 0x7f0544114ab0 2024-09-10T10:04:21.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:21.159+0000 7f053aa00640 1 --2- 172.21.15.77:0/1882081068 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f05280a2cc0 0x7f05280a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:21.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:21.160+0000 7f053aa00640 1 -- 172.21.15.77:0/1882081068 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f053c0cf1b0 con 0x7f0544114ab0 2024-09-10T10:04:21.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:21.161+0000 7f0541e00640 1 --2- 172.21.15.77:0/1882081068 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f05280a2cc0 0x7f05280a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:21.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:21.162+0000 7f0541e00640 1 --2- 172.21.15.77:0/1882081068 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f05280a2cc0 0x7f05280a5180 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f05340058c0 tx=0x7f0534005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:21.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:21.166+0000 7f053aa00640 1 -- 172.21.15.77:0/1882081068 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f053c09bd20 con 0x7f0544114ab0 2024-09-10T10:04:21.350 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:04:21.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:21.491+0000 7f0548a00640 1 -- 172.21.15.77:0/1882081068 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f054411c4b0 con 0x7f0544114ab0 2024-09-10T10:04:21.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:21.492+0000 7f053aa00640 1 -- 172.21.15.77:0/1882081068 <== mon.2 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11997 (secure 0 0 0) 0x7f053c09b620 con 0x7f0544114ab0 2024-09-10T10:04:21.493 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T10:04:21.493 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":39,"btime":"2024-09-10T10:04:21:022253+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3159289675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3159289675},{"type":"v1","addr":"172.21.15.77:6835","nonce":3159289675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1951962341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1951962341},{"type":"v1","addr":"172.21.15.77:6837","nonce":1951962341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:52:23.368252+0000","modified":"2024-09-10T10:04:20.266231+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24563,"mds_1":24524,"mds_2":24520,"mds_3":24502,"mds_4":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14487":{"gid":14487,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/4291121888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":4291121888},{"type":"v1","addr":"172.21.15.77:6839","nonce":4291121888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14499":{"gid":14499,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6841/445284194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":445284194},{"type":"v1","addr":"172.21.15.77:6841","nonce":445284194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1005054423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1005054423},{"type":"v1","addr":"172.21.15.134:6833","nonce":1005054423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6833/3716307744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":3716307744},{"type":"v1","addr":"172.21.15.120:6833","nonce":3716307744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/1319892544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1319892544},{"type":"v1","addr":"172.21.15.120:6835","nonce":1319892544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.134:6835/3293933699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3293933699},{"type":"v1","addr":"172.21.15.134:6835","nonce":3293933699}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.134:6837/2461497091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2461497091},{"type":"v1","addr":"172.21.15.134:6837","nonce":2461497091}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":20,"addr":"172.21.15.134:6839/1965117680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1965117680},{"type":"v1","addr":"172.21.15.134:6839","nonce":1965117680}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":20,"addr":"172.21.15.120:6837/3033280709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3033280709},{"type":"v1","addr":"172.21.15.120:6837","nonce":3033280709}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.120:6839/1777710157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1777710157},{"type":"v1","addr":"172.21.15.120:6839","nonce":1777710157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24563,"qdb_cluster":[24496,24502,24520,24524,24563]},"id":1}]} 2024-09-10T10:04:21.494 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 39 2024-09-10T10:04:21.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:21.494+0000 7f0548a00640 1 -- 172.21.15.77:0/1882081068 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f05280a2cc0 msgr2=0x7f05280a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:21.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:21.494+0000 7f0548a00640 1 --2- 172.21.15.77:0/1882081068 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f05280a2cc0 0x7f05280a5180 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f05340058c0 tx=0x7f0534005850 comp rx=0 tx=0).stop 2024-09-10T10:04:21.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:21.494+0000 7f0548a00640 1 -- 172.21.15.77:0/1882081068 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0544114ab0 msgr2=0x7f054410e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:21.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:21.494+0000 7f0548a00640 1 --2- 172.21.15.77:0/1882081068 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0544114ab0 0x7f054410e660 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f053c00dac0 tx=0x7f053c00df80 comp rx=0 tx=0).stop 2024-09-10T10:04:21.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:21.494+0000 7f0548a00640 1 -- 172.21.15.77:0/1882081068 shutdown_connections 2024-09-10T10:04:21.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:21.494+0000 7f0548a00640 1 --2- 172.21.15.77:0/1882081068 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f05280a2cc0 0x7f05280a5180 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:21.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:21.494+0000 7f0548a00640 1 --2- 172.21.15.77:0/1882081068 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0544114ab0 0x7f054410e660 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:21.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:21.494+0000 7f0548a00640 1 --2- 172.21.15.77:0/1882081068 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0544109b10 0x7f054410e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:21.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:21.494+0000 7f0548a00640 1 --2- 172.21.15.77:0/1882081068 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f05441091f0 0x7f05441140a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:21.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:21.494+0000 7f0548a00640 1 -- 172.21.15.77:0/1882081068 >> 172.21.15.77:0/1882081068 conn(0x7f054406e140 msgr2=0x7f0544071dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:21.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:21.495+0000 7f0548a00640 1 -- 172.21.15.77:0/1882081068 shutdown_connections 2024-09-10T10:04:21.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:21.495+0000 7f0548a00640 1 -- 172.21.15.77:0/1882081068 wait complete. 2024-09-10T10:04:22.317 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:04:22 smithi077 ceph-mon[30319]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T10:04:22.317 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:04:22 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/1882081068' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:04:22.317 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:04:22 smithi077 ceph-mon[30319]: pgmap v667: 129 pgs: 129 active+clean; 143 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.2 MiB/s wr, 343 op/s 2024-09-10T10:04:22.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:04:22 smithi134 ceph-mon[35674]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T10:04:22.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:04:22 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/1882081068' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:04:22.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:04:22 smithi134 ceph-mon[35674]: pgmap v667: 129 pgs: 129 active+clean; 143 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.2 MiB/s wr, 343 op/s 2024-09-10T10:04:22.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:04:22 smithi120 ceph-mon[34770]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T10:04:22.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:04:22 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/1882081068' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:04:22.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:04:22 smithi120 ceph-mon[34770]: pgmap v667: 129 pgs: 129 active+clean; 143 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.2 MiB/s wr, 343 op/s 2024-09-10T10:04:22.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:22.551+0000 7fed00400640 1 -- 172.21.15.77:0/944474831 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fecf8074eb0 msgr2=0x7fecf8112650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:22.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:22.551+0000 7fed00400640 1 --2- 172.21.15.77:0/944474831 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fecf8074eb0 0x7fecf8112650 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fece4009900 tx=0x7fece402a480 comp rx=0 tx=0).stop 2024-09-10T10:04:22.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:22.552+0000 7fed00400640 1 -- 172.21.15.77:0/944474831 shutdown_connections 2024-09-10T10:04:22.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:22.552+0000 7fed00400640 1 --2- 172.21.15.77:0/944474831 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fecf8112b90 0x7fecf8114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:22.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:22.552+0000 7fed00400640 1 --2- 172.21.15.77:0/944474831 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fecf8074eb0 0x7fecf8112650 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:22.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:22.552+0000 7fed00400640 1 --2- 172.21.15.77:0/944474831 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fecf8074590 0x7fecf8074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:22.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:22.552+0000 7fed00400640 1 -- 172.21.15.77:0/944474831 >> 172.21.15.77:0/944474831 conn(0x7fecf80fd8c0 msgr2=0x7fecf80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:22.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:22.552+0000 7fed00400640 1 -- 172.21.15.77:0/944474831 shutdown_connections 2024-09-10T10:04:22.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:22.552+0000 7fed00400640 1 -- 172.21.15.77:0/944474831 wait complete. 2024-09-10T10:04:22.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:22.553+0000 7fed00400640 1 Processor -- start 2024-09-10T10:04:22.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:22.553+0000 7fed00400640 1 -- start start 2024-09-10T10:04:22.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:22.553+0000 7fed00400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fecf8074590 0x7fecf819eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:22.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:22.553+0000 7fed00400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fecf8074eb0 0x7fecf819f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:22.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:22.553+0000 7fed00400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fecf8112b90 0x7fecf81a3410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:22.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:22.553+0000 7fed00400640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fecf8118540 con 0x7fecf8074590 2024-09-10T10:04:22.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:22.553+0000 7fed00400640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fecf81183c0 con 0x7fecf8112b90 2024-09-10T10:04:22.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:22.553+0000 7fed00400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fecf81186c0 con 0x7fecf8074eb0 2024-09-10T10:04:22.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:22.553+0000 7fecfd000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fecf8074eb0 0x7fecf819f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:22.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:22.553+0000 7fecfda00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fecf8074590 0x7fecf819eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:22.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:22.553+0000 7fecfd000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fecf8074eb0 0x7fecf819f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:52186/0 (socket says 172.21.15.77:52186) 2024-09-10T10:04:22.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:22.553+0000 7fecfd000640 1 -- 172.21.15.77:0/925878704 learned_addr learned my addr 172.21.15.77:0/925878704 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:04:22.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:22.554+0000 7fecfe400640 1 --2- 172.21.15.77:0/925878704 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fecf8112b90 0x7fecf81a3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:22.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:22.554+0000 7fecfda00640 1 -- 172.21.15.77:0/925878704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fecf8074eb0 msgr2=0x7fecf819f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:22.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:22.554+0000 7fecfda00640 1 --2- 172.21.15.77:0/925878704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fecf8074eb0 0x7fecf819f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:22.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:22.554+0000 7fecfda00640 1 -- 172.21.15.77:0/925878704 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fecf8112b90 msgr2=0x7fecf81a3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:22.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:22.554+0000 7fecfda00640 1 --2- 172.21.15.77:0/925878704 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fecf8112b90 0x7fecf81a3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:22.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:22.554+0000 7fecfda00640 1 -- 172.21.15.77:0/925878704 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecf81a3b40 con 0x7fecf8074590 2024-09-10T10:04:22.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:22.554+0000 7fecfd000640 1 --2- 172.21.15.77:0/925878704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fecf8074eb0 0x7fecf819f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:04:22.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:22.554+0000 7fecfda00640 1 --2- 172.21.15.77:0/925878704 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fecf8074590 0x7fecf819eb10 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fecec00e9f0 tx=0x7fecec00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:22.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:22.554+0000 7fecf6000640 1 -- 172.21.15.77:0/925878704 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecec00cd80 con 0x7fecf8074590 2024-09-10T10:04:22.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:22.554+0000 7fecf6000640 1 -- 172.21.15.77:0/925878704 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fecec0035f0 con 0x7fecf8074590 2024-09-10T10:04:22.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:22.555+0000 7fed00400640 1 -- 172.21.15.77:0/925878704 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecf81a3e30 con 0x7fecf8074590 2024-09-10T10:04:22.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:22.555+0000 7fecf6000640 1 -- 172.21.15.77:0/925878704 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecec010690 con 0x7fecf8074590 2024-09-10T10:04:22.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:22.555+0000 7fed00400640 1 -- 172.21.15.77:0/925878704 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecf81ab6c0 con 0x7fecf8074590 2024-09-10T10:04:22.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:22.556+0000 7fed00400640 1 -- 172.21.15.77:0/925878704 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecf8075cd0 con 0x7fecf8074590 2024-09-10T10:04:22.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:22.558+0000 7fecf6000640 1 -- 172.21.15.77:0/925878704 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fecec003b30 con 0x7fecf8074590 2024-09-10T10:04:22.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:22.560+0000 7fecf6000640 1 --2- 172.21.15.77:0/925878704 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fece00a2b80 0x7fece00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:22.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:22.560+0000 7fecf6000640 1 -- 172.21.15.77:0/925878704 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fecec0cf770 con 0x7fecf8074590 2024-09-10T10:04:22.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:22.560+0000 7fecfd000640 1 --2- 172.21.15.77:0/925878704 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fece00a2b80 0x7fece00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:22.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:22.561+0000 7fecfd000640 1 --2- 172.21.15.77:0/925878704 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fece00a2b80 0x7fece00a5040 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fecf819fa80 tx=0x7fece4005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:22.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:22.564+0000 7fecf6000640 1 -- 172.21.15.77:0/925878704 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fecec09c280 con 0x7fecf8074590 2024-09-10T10:04:22.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:22.893+0000 7fed00400640 1 -- 172.21.15.77:0/925878704 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fecf8074970 con 0x7fecf8074590 2024-09-10T10:04:22.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:22.895+0000 7fecf6000640 1 -- 172.21.15.77:0/925878704 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11997 (secure 0 0 0) 0x7fecec09bb80 con 0x7fecf8074590 2024-09-10T10:04:22.896 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T10:04:22.896 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":39,"btime":"2024-09-10T10:04:21:022253+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3159289675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3159289675},{"type":"v1","addr":"172.21.15.77:6835","nonce":3159289675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1951962341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1951962341},{"type":"v1","addr":"172.21.15.77:6837","nonce":1951962341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:52:23.368252+0000","modified":"2024-09-10T10:04:20.266231+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24563,"mds_1":24524,"mds_2":24520,"mds_3":24502,"mds_4":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14487":{"gid":14487,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/4291121888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":4291121888},{"type":"v1","addr":"172.21.15.77:6839","nonce":4291121888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14499":{"gid":14499,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6841/445284194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":445284194},{"type":"v1","addr":"172.21.15.77:6841","nonce":445284194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1005054423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1005054423},{"type":"v1","addr":"172.21.15.134:6833","nonce":1005054423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6833/3716307744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":3716307744},{"type":"v1","addr":"172.21.15.120:6833","nonce":3716307744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/1319892544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1319892544},{"type":"v1","addr":"172.21.15.120:6835","nonce":1319892544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.134:6835/3293933699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3293933699},{"type":"v1","addr":"172.21.15.134:6835","nonce":3293933699}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.134:6837/2461497091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2461497091},{"type":"v1","addr":"172.21.15.134:6837","nonce":2461497091}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":20,"addr":"172.21.15.134:6839/1965117680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1965117680},{"type":"v1","addr":"172.21.15.134:6839","nonce":1965117680}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":20,"addr":"172.21.15.120:6837/3033280709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3033280709},{"type":"v1","addr":"172.21.15.120:6837","nonce":3033280709}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.120:6839/1777710157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1777710157},{"type":"v1","addr":"172.21.15.120:6839","nonce":1777710157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24563,"qdb_cluster":[24496,24502,24520,24524,24563]},"id":1}]} 2024-09-10T10:04:22.897 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 39 2024-09-10T10:04:22.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:22.899+0000 7fed00400640 1 -- 172.21.15.77:0/925878704 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fece00a2b80 msgr2=0x7fece00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:22.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:22.899+0000 7fed00400640 1 --2- 172.21.15.77:0/925878704 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fece00a2b80 0x7fece00a5040 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fecf819fa80 tx=0x7fece4005a20 comp rx=0 tx=0).stop 2024-09-10T10:04:22.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:22.899+0000 7fed00400640 1 -- 172.21.15.77:0/925878704 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fecf8074590 msgr2=0x7fecf819eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:22.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:22.899+0000 7fed00400640 1 --2- 172.21.15.77:0/925878704 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fecf8074590 0x7fecf819eb10 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fecec00e9f0 tx=0x7fecec00eeb0 comp rx=0 tx=0).stop 2024-09-10T10:04:22.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:22.899+0000 7fed00400640 1 -- 172.21.15.77:0/925878704 shutdown_connections 2024-09-10T10:04:22.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:22.899+0000 7fed00400640 1 --2- 172.21.15.77:0/925878704 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fece00a2b80 0x7fece00a5040 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:22.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:22.899+0000 7fed00400640 1 --2- 172.21.15.77:0/925878704 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fecf8112b90 0x7fecf81a3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:22.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:22.899+0000 7fed00400640 1 --2- 172.21.15.77:0/925878704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fecf8074eb0 0x7fecf819f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:22.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:22.899+0000 7fed00400640 1 --2- 172.21.15.77:0/925878704 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fecf8074590 0x7fecf819eb10 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:22.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:22.899+0000 7fed00400640 1 -- 172.21.15.77:0/925878704 >> 172.21.15.77:0/925878704 conn(0x7fecf80fd8c0 msgr2=0x7fecf80ff100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:22.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:22.900+0000 7fed00400640 1 -- 172.21.15.77:0/925878704 shutdown_connections 2024-09-10T10:04:22.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:22.900+0000 7fed00400640 1 -- 172.21.15.77:0/925878704 wait complete. 2024-09-10T10:04:23.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:04:23 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/925878704' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:04:23.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:04:23 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/925878704' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:04:23.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:04:23 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/925878704' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:04:23.553 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T10:04:24.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:04:24 smithi134 ceph-mon[35674]: pgmap v668: 129 pgs: 129 active+clean; 139 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.2 MiB/s wr, 398 op/s 2024-09-10T10:04:24.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:04:24 smithi120 ceph-mon[34770]: pgmap v668: 129 pgs: 129 active+clean; 139 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.2 MiB/s wr, 398 op/s 2024-09-10T10:04:24.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:04:24 smithi077 ceph-mon[30319]: pgmap v668: 129 pgs: 129 active+clean; 139 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.2 MiB/s wr, 398 op/s 2024-09-10T10:04:24.625 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24563 quiesce db / --timeout 90 --expiration 144.6 --await-for 45.0' 2024-09-10T10:04:24.625 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24563 quiesce db / --timeout 90 --expiration 144.6 --await-for 45.0 2024-09-10T10:04:24.756 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:04:24.858 INFO:tasks.workunit:Stopping ['suites/fsx.sh'] on client.0... 2024-09-10T10:04:24.858 DEBUG:teuthology.orchestra.run.smithi077:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-10T10:04:25.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.752+0000 7f0674a00640 1 -- 172.21.15.77:0/3045815737 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06680319b0 con 0x7f0678104980 2024-09-10T10:04:25.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.752+0000 7f067ca00640 1 -- 172.21.15.77:0/3045815737 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0678104980 msgr2=0x7f06781089d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:25.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.752+0000 7f067ca00640 1 --2- 172.21.15.77:0/3045815737 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0678104980 0x7f06781089d0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f0668009900 tx=0x7f066802a420 comp rx=0 tx=0).stop 2024-09-10T10:04:25.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.752+0000 7f067ca00640 1 -- 172.21.15.77:0/3045815737 shutdown_connections 2024-09-10T10:04:25.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.752+0000 7f067ca00640 1 --2- 172.21.15.77:0/3045815737 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0678109100 0x7f0678110c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:25.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.752+0000 7f067ca00640 1 --2- 172.21.15.77:0/3045815737 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0678104980 0x7f06781089d0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:25.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.752+0000 7f067ca00640 1 --2- 172.21.15.77:0/3045815737 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0678103fd0 0x7f06781043b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:25.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.752+0000 7f067ca00640 1 -- 172.21.15.77:0/3045815737 >> 172.21.15.77:0/3045815737 conn(0x7f06780ffa30 msgr2=0x7f0678101e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:25.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.752+0000 7f067ca00640 1 -- 172.21.15.77:0/3045815737 shutdown_connections 2024-09-10T10:04:25.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.752+0000 7f067ca00640 1 -- 172.21.15.77:0/3045815737 wait complete. 2024-09-10T10:04:25.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.753+0000 7f067ca00640 1 Processor -- start 2024-09-10T10:04:25.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.753+0000 7f067ca00640 1 -- start start 2024-09-10T10:04:25.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.753+0000 7f067ca00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0678103fd0 0x7f067819a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:25.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.753+0000 7f067ca00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0678104980 0x7f067819ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:25.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.753+0000 7f067ca00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0678109100 0x7f067819eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:25.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.753+0000 7f067ca00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0678114150 con 0x7f0678109100 2024-09-10T10:04:25.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.753+0000 7f067ca00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0678113fd0 con 0x7f0678103fd0 2024-09-10T10:04:25.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.753+0000 7f067ca00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f06781142d0 con 0x7f0678104980 2024-09-10T10:04:25.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.754+0000 7f0676800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0678109100 0x7f067819eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:25.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.754+0000 7f0675400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0678104980 0x7f067819ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:25.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.754+0000 7f0675400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0678104980 0x7f067819ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:33142/0 (socket says 172.21.15.77:33142) 2024-09-10T10:04:25.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.754+0000 7f0676800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0678109100 0x7f067819eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:39346/0 (socket says 172.21.15.77:39346) 2024-09-10T10:04:25.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.754+0000 7f0675400640 1 -- 172.21.15.77:0/4275487576 learned_addr learned my addr 172.21.15.77:0/4275487576 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:04:25.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.754+0000 7f0675e00640 1 --2- 172.21.15.77:0/4275487576 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0678103fd0 0x7f067819a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:25.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.754+0000 7f0675e00640 1 -- 172.21.15.77:0/4275487576 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0678104980 msgr2=0x7f067819ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:25.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.754+0000 7f0675e00640 1 --2- 172.21.15.77:0/4275487576 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0678104980 0x7f067819ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:25.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.754+0000 7f0675e00640 1 -- 172.21.15.77:0/4275487576 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0678109100 msgr2=0x7f067819eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:25.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.754+0000 7f0675e00640 1 --2- 172.21.15.77:0/4275487576 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0678109100 0x7f067819eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:25.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.754+0000 7f0675e00640 1 -- 172.21.15.77:0/4275487576 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f067819f5d0 con 0x7f0678103fd0 2024-09-10T10:04:25.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.755+0000 7f0675e00640 1 --2- 172.21.15.77:0/4275487576 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0678103fd0 0x7f067819a620 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f067000e9f0 tx=0x7f067000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:25.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.755+0000 7f066ea00640 1 -- 172.21.15.77:0/4275487576 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f067000cd80 con 0x7f0678103fd0 2024-09-10T10:04:25.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.755+0000 7f066ea00640 1 -- 172.21.15.77:0/4275487576 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0670011950 con 0x7f0678103fd0 2024-09-10T10:04:25.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.755+0000 7f067ca00640 1 -- 172.21.15.77:0/4275487576 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f067819f8c0 con 0x7f0678103fd0 2024-09-10T10:04:25.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.755+0000 7f066ea00640 1 -- 172.21.15.77:0/4275487576 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0670010a50 con 0x7f0678103fd0 2024-09-10T10:04:25.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.755+0000 7f067ca00640 1 -- 172.21.15.77:0/4275487576 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06781a71a0 con 0x7f0678103fd0 2024-09-10T10:04:25.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.758+0000 7f066ea00640 1 -- 172.21.15.77:0/4275487576 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0670011b00 con 0x7f0678103fd0 2024-09-10T10:04:25.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.760+0000 7f067f5ea740 1 Processor -- start 2024-09-10T10:04:25.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.760+0000 7f067f5ea740 1 -- start start 2024-09-10T10:04:25.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.760+0000 7f066ea00640 1 --2- 172.21.15.77:0/4275487576 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f065c0a2b80 0x7f065c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:25.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.760+0000 7f067f5ea740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555dbc040c70 0x555dbc041050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:25.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.760+0000 7f067f5ea740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555dbc041590 0x555dbc04b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:25.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.760+0000 7f066ea00640 1 -- 172.21.15.77:0/4275487576 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f06700cf920 con 0x7f0678103fd0 2024-09-10T10:04:25.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.760+0000 7f067f5ea740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555dbc04be50 0x555dbc04e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:25.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.760+0000 7f067f5ea740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x555dbbfa73e0 con 0x555dbc041590 2024-09-10T10:04:25.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.760+0000 7f067f5ea740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x555dbbf4ed80 con 0x555dbc04be50 2024-09-10T10:04:25.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.760+0000 7f067f5ea740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x555dbbfa33a0 con 0x555dbc040c70 2024-09-10T10:04:25.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.760+0000 7f0676800640 1 --2- 172.21.15.77:0/4275487576 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f065c0a2b80 0x7f065c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:25.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.761+0000 7f0675e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555dbc040c70 0x555dbc041050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:25.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.761+0000 7f0675e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555dbc040c70 0x555dbc041050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:33156/0 (socket says 172.21.15.77:33156) 2024-09-10T10:04:25.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.761+0000 7f0675e00640 1 -- 172.21.15.77:0/2048122011 learned_addr learned my addr 172.21.15.77:0/2048122011 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:04:25.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.761+0000 7f0675400640 1 --2- 172.21.15.77:0/2048122011 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555dbc041590 0x555dbc04b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:25.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.761+0000 7f0676800640 1 --2- 172.21.15.77:0/2048122011 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555dbc04be50 0x555dbc04e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:25.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.761+0000 7f0675e00640 1 -- 172.21.15.77:0/2048122011 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555dbc04be50 msgr2=0x555dbc04e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:25.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.761+0000 7f0675e00640 1 --2- 172.21.15.77:0/2048122011 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555dbc04be50 0x555dbc04e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:25.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.761+0000 7f0675e00640 1 -- 172.21.15.77:0/2048122011 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555dbc041590 msgr2=0x555dbc04b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:25.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.761+0000 7f0675e00640 1 --2- 172.21.15.77:0/2048122011 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555dbc041590 0x555dbc04b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:25.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.761+0000 7f0675e00640 1 -- 172.21.15.77:0/2048122011 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555dbbf6dae0 con 0x555dbc040c70 2024-09-10T10:04:25.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.761+0000 7f0675400640 1 --2- 172.21.15.77:0/2048122011 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555dbc041590 0x555dbc04b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:04:25.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.762+0000 7f0675e00640 1 --2- 172.21.15.77:0/2048122011 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555dbc040c70 0x555dbc041050 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f06700a4470 tx=0x7f067009b750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5bc7f1f7d8dabce server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:25.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.762+0000 7f0676800640 1 --2- 172.21.15.77:0/4275487576 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f065c0a2b80 0x7f065c0a5040 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x555dbc03e580 tx=0x7f0660004140 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:25.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.762+0000 7f0667000640 1 -- 172.21.15.77:0/2048122011 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06700a00e0 con 0x555dbc040c70 2024-09-10T10:04:25.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.762+0000 7f0667000640 1 -- 172.21.15.77:0/2048122011 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06700a49a0 con 0x555dbc040c70 2024-09-10T10:04:25.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.762+0000 7f0667000640 1 -- 172.21.15.77:0/2048122011 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0670010e50 con 0x555dbc040c70 2024-09-10T10:04:25.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.763+0000 7f067f5ea740 1 -- 172.21.15.77:0/2048122011 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555dbc040c70 msgr2=0x555dbc041050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:25.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.763+0000 7f067f5ea740 1 --2- 172.21.15.77:0/2048122011 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555dbc040c70 0x555dbc041050 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f06700a4470 tx=0x7f067009b750 comp rx=0 tx=0).stop 2024-09-10T10:04:25.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.763+0000 7f067f5ea740 1 -- 172.21.15.77:0/2048122011 shutdown_connections 2024-09-10T10:04:25.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.763+0000 7f067f5ea740 1 --2- 172.21.15.77:0/2048122011 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555dbc04be50 0x555dbc04e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:25.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.763+0000 7f067f5ea740 1 --2- 172.21.15.77:0/2048122011 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555dbc041590 0x555dbc04b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:25.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.763+0000 7f067f5ea740 1 --2- 172.21.15.77:0/2048122011 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555dbc040c70 0x555dbc041050 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:25.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.763+0000 7f067f5ea740 1 -- 172.21.15.77:0/2048122011 >> 172.21.15.77:0/2048122011 conn(0x555dbc03d200 msgr2=0x555dbc03d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:25.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.763+0000 7f067f5ea740 1 -- 172.21.15.77:0/2048122011 shutdown_connections 2024-09-10T10:04:25.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.763+0000 7f067f5ea740 1 -- 172.21.15.77:0/2048122011 wait complete. 2024-09-10T10:04:25.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.764+0000 7f067f5ea740 1 -- start start 2024-09-10T10:04:25.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.764+0000 7f067f5ea740 1 Processor -- start 2024-09-10T10:04:25.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.765+0000 7f067f5ea740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555dbc040c70 0x555dbc048d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:25.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.765+0000 7f067f5ea740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555dbc041590 0x555dbc049270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:25.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.765+0000 7f067f5ea740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555dbc04be50 0x555dbc0497b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:25.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.765+0000 7f067f5ea740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x555dbc04ea20 con 0x555dbc04be50 2024-09-10T10:04:25.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.765+0000 7f067f5ea740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x555dbbf620f0 con 0x555dbc040c70 2024-09-10T10:04:25.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.765+0000 7f067f5ea740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x555dbc04eba0 con 0x555dbc041590 2024-09-10T10:04:25.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.765+0000 7f067f5ea740 10 client.0 _ll_register_callbacks cb 0x555dbbf49bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:04:25.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.765+0000 7f0675e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555dbc040c70 0x555dbc048d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:25.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.765+0000 7f0675e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555dbc040c70 0x555dbc048d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:57608/0 (socket says 172.21.15.77:57608) 2024-09-10T10:04:25.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.765+0000 7f0675e00640 1 -- 172.21.15.77:0/1775545243 learned_addr learned my addr 172.21.15.77:0/1775545243 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:04:25.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.765+0000 7f0675400640 1 --2- 172.21.15.77:0/1775545243 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555dbc041590 0x555dbc049270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:25.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.765+0000 7f0676800640 1 --2- 172.21.15.77:0/1775545243 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555dbc04be50 0x555dbc0497b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:25.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.766+0000 7f0675e00640 1 -- 172.21.15.77:0/1775545243 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555dbc041590 msgr2=0x555dbc049270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:25.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.766+0000 7f0675e00640 1 --2- 172.21.15.77:0/1775545243 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555dbc041590 0x555dbc049270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:25.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.766+0000 7f0675e00640 1 -- 172.21.15.77:0/1775545243 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555dbc04be50 msgr2=0x555dbc0497b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:25.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.766+0000 7f0675e00640 1 --2- 172.21.15.77:0/1775545243 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555dbc04be50 0x555dbc0497b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:25.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.766+0000 7f0675e00640 1 -- 172.21.15.77:0/1775545243 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555dbbd183c0 con 0x555dbc040c70 2024-09-10T10:04:25.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.766+0000 7f0675400640 1 --2- 172.21.15.77:0/1775545243 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555dbc041590 0x555dbc049270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:04:25.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.766+0000 7f0675e00640 1 --2- 172.21.15.77:0/1775545243 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555dbc040c70 0x555dbc048d30 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f067000cf60 tx=0x7f06700a46c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=269cf8e57cbed2c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:25.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.766+0000 7f065b400640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T10:04:25.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.766+0000 7f065b400640 1 -- 172.21.15.77:0/1775545243 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f067007e070 con 0x555dbc040c70 2024-09-10T10:04:25.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.766+0000 7f065b400640 1 -- 172.21.15.77:0/1775545243 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f0670010bf0 con 0x555dbc040c70 2024-09-10T10:04:25.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.767+0000 7f067f5ea740 1 -- 172.21.15.77:0/1775545243 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06700a49a0 con 0x555dbc040c70 2024-09-10T10:04:25.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.767+0000 7f065b400640 1 -- 172.21.15.77:0/1775545243 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f067009a0f0 con 0x555dbc040c70 2024-09-10T10:04:25.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.767+0000 7f067f5ea740 10 client.27335 fetch_fsmap learned FSMap version 39 2024-09-10T10:04:25.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.767+0000 7f067f5ea740 1 -- 172.21.15.77:0/1775545243 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x555dbc05aa60 con 0x555dbc040c70 2024-09-10T10:04:25.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.767+0000 7f065b400640 1 -- 172.21.15.77:0/1775545243 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06700a49a0 con 0x555dbc040c70 2024-09-10T10:04:25.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.767+0000 7f065b400640 1 -- 172.21.15.77:0/1775545243 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8394+0+0 (secure 0 0 0) 0x7f067000f040 con 0x555dbc040c70 2024-09-10T10:04:25.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.767+0000 7f067f5ea740 10 client.27335 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T10:04:25.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.767+0000 7f067f5ea740 10 client.27335 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-10T10:04:25.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.767+0000 7f067f5ea740 20 client.27335 populate_metadata read hostname 'smithi077' 2024-09-10T10:04:25.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.767+0000 7f067f5ea740 1 --2- 172.21.15.77:0/1775545243 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x555dbc05bdb0 0x555dbc05e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:25.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.767+0000 7f067f5ea740 4 client.27335 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:04:25.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.767+0000 7f067f5ea740 1 -- 172.21.15.77:0/1775545243 --> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555dbbfb4950 con 0x555dbc05bdb0 2024-09-10T10:04:25.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.768+0000 7f0675400640 1 --2- 172.21.15.77:0/1775545243 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x555dbc05bdb0 0x555dbc05e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:25.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.768+0000 7f065b400640 10 client.27335 ms_handle_connect on v2:172.21.15.134:6834/3293933699 2024-09-10T10:04:25.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.768+0000 7f0675400640 1 --2- 172.21.15.77:0/1775545243 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x555dbc05bdb0 0x555dbc05e1a0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9f1a3de1eb42f812 server_cookie=8c991bbafd7510d1 in_seq=0 out_seq=0 2024-09-10T10:04:25.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.770+0000 7f065b400640 1 -- 172.21.15.77:0/1775545243 <== mds.4 v2:172.21.15.134:6834/3293933699 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555dbbfb4950 con 0x555dbc05bdb0 2024-09-10T10:04:25.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.770+0000 7f065b400640 10 client.27335 handle_command_reply: tid=0 2024-09-10T10:04:25.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.770+0000 7f067f5ea740 1 client.27335 shutdown 2024-09-10T10:04:25.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.770+0000 7f067f5ea740 1 -- 172.21.15.77:0/1775545243 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555dbc040c70 msgr2=0x555dbc048d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:25.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.770+0000 7f067f5ea740 1 --2- 172.21.15.77:0/1775545243 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555dbc040c70 0x555dbc048d30 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f067000cf60 tx=0x7f06700a46c0 comp rx=0 tx=0).stop 2024-09-10T10:04:25.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.770+0000 7f067f5ea740 1 -- 172.21.15.77:0/1775545243 shutdown_connections 2024-09-10T10:04:25.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.770+0000 7f067f5ea740 1 --2- 172.21.15.77:0/1775545243 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x555dbc05bdb0 0x555dbc05e1a0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:25.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.770+0000 7f067f5ea740 1 --2- 172.21.15.77:0/1775545243 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555dbc04be50 0x555dbc0497b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:25.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.770+0000 7f067f5ea740 1 --2- 172.21.15.77:0/1775545243 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555dbc041590 0x555dbc049270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:25.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.770+0000 7f067f5ea740 1 --2- 172.21.15.77:0/1775545243 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555dbc040c70 0x555dbc048d30 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:25.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.770+0000 7f067f5ea740 1 -- 172.21.15.77:0/1775545243 >> 172.21.15.77:0/1775545243 conn(0x555dbc03d200 msgr2=0x555dbc042f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:25.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.771+0000 7f065b400640 0 client.27335 ms_handle_reset on v2:172.21.15.134:6834/3293933699 2024-09-10T10:04:25.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.771+0000 7f067f5ea740 1 -- 172.21.15.77:0/1775545243 shutdown_connections 2024-09-10T10:04:25.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.771+0000 7f067f5ea740 1 -- 172.21.15.77:0/1775545243 wait complete. 2024-09-10T10:04:25.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.771+0000 7f067f5ea740 20 client.27335 trim_cache size 0 max 16384 2024-09-10T10:04:25.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.785+0000 7f067f5ea740 1 Processor -- start 2024-09-10T10:04:25.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.786+0000 7f067f5ea740 1 -- start start 2024-09-10T10:04:25.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.786+0000 7f067f5ea740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555dbc040c70 0x555dbc0436d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:25.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.786+0000 7f067f5ea740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555dbc041590 0x555dbc047600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:25.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.786+0000 7f067f5ea740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555dbc04be50 0x555dbc0611c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:25.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.786+0000 7f067f5ea740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f06781b1180 con 0x555dbc041590 2024-09-10T10:04:25.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.786+0000 7f067f5ea740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f06781b2d80 con 0x555dbc04be50 2024-09-10T10:04:25.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.786+0000 7f067f5ea740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f06781b18b0 con 0x555dbc040c70 2024-09-10T10:04:25.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.786+0000 7f0676800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555dbc04be50 0x555dbc0611c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:25.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.786+0000 7f0676800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555dbc04be50 0x555dbc0611c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:57620/0 (socket says 172.21.15.77:57620) 2024-09-10T10:04:25.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.786+0000 7f0676800640 1 -- 172.21.15.77:0/1927716453 learned_addr learned my addr 172.21.15.77:0/1927716453 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:04:25.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.787+0000 7f0676800640 1 -- 172.21.15.77:0/1927716453 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555dbc040c70 msgr2=0x555dbc0436d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T10:04:25.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.787+0000 7f0676800640 1 --2- 172.21.15.77:0/1927716453 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555dbc040c70 0x555dbc0436d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:25.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.787+0000 7f0676800640 1 -- 172.21.15.77:0/1927716453 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555dbc041590 msgr2=0x555dbc047600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:25.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.787+0000 7f0676800640 1 --2- 172.21.15.77:0/1927716453 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555dbc041590 0x555dbc047600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:25.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.787+0000 7f0676800640 1 -- 172.21.15.77:0/1927716453 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555dbc04eb00 con 0x555dbc04be50 2024-09-10T10:04:25.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.787+0000 7f0676800640 1 --2- 172.21.15.77:0/1927716453 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555dbc04be50 0x555dbc0611c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f0660004650 tx=0x7f0660004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bcdefa7332a04ed0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:25.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.787+0000 7f0666600640 1 -- 172.21.15.77:0/1927716453 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f066001f640 con 0x555dbc04be50 2024-09-10T10:04:25.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.787+0000 7f0666600640 1 -- 172.21.15.77:0/1927716453 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0660010e40 con 0x555dbc04be50 2024-09-10T10:04:25.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.787+0000 7f0666600640 1 -- 172.21.15.77:0/1927716453 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0660027b10 con 0x555dbc04be50 2024-09-10T10:04:25.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.788+0000 7f067f5ea740 1 -- 172.21.15.77:0/1927716453 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555dbc04be50 msgr2=0x555dbc0611c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:25.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.788+0000 7f067f5ea740 1 --2- 172.21.15.77:0/1927716453 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555dbc04be50 0x555dbc0611c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f0660004650 tx=0x7f0660004730 comp rx=0 tx=0).stop 2024-09-10T10:04:25.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.789+0000 7f067f5ea740 1 -- 172.21.15.77:0/1927716453 shutdown_connections 2024-09-10T10:04:25.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.789+0000 7f067f5ea740 1 --2- 172.21.15.77:0/1927716453 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555dbc04be50 0x555dbc0611c0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:25.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.789+0000 7f067f5ea740 1 --2- 172.21.15.77:0/1927716453 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555dbc041590 0x555dbc047600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:25.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.789+0000 7f067f5ea740 1 --2- 172.21.15.77:0/1927716453 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555dbc040c70 0x555dbc0436d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:25.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.789+0000 7f067f5ea740 1 -- 172.21.15.77:0/1927716453 >> 172.21.15.77:0/1927716453 conn(0x555dbc03d200 msgr2=0x555dbc057610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:25.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.789+0000 7f067f5ea740 1 -- 172.21.15.77:0/1927716453 shutdown_connections 2024-09-10T10:04:25.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.789+0000 7f067f5ea740 1 -- 172.21.15.77:0/1927716453 wait complete. 2024-09-10T10:04:25.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.789+0000 7f067f5ea740 1 -- start start 2024-09-10T10:04:25.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.790+0000 7f067f5ea740 1 Processor -- start 2024-09-10T10:04:25.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.790+0000 7f067f5ea740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555dbc040c70 0x555dbc05f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:25.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.790+0000 7f067f5ea740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555dbc041590 0x555dbc05f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:25.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.790+0000 7f067f5ea740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555dbc04be50 0x555dbc05fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:25.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.790+0000 7f067f5ea740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f06781b26b0 con 0x555dbc041590 2024-09-10T10:04:25.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.790+0000 7f067f5ea740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f06781b1fb0 con 0x555dbc04be50 2024-09-10T10:04:25.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.790+0000 7f067f5ea740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f06781af160 con 0x555dbc040c70 2024-09-10T10:04:25.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.790+0000 7f067f5ea740 10 client.0 _ll_register_callbacks cb 0x7f067009b900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:04:25.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.791+0000 7f0675400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555dbc041590 0x555dbc05f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:25.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.791+0000 7f0676800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555dbc04be50 0x555dbc05fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:25.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.791+0000 7f0675e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555dbc040c70 0x555dbc05f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:25.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.791+0000 7f0676800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555dbc04be50 0x555dbc05fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:57626/0 (socket says 172.21.15.77:57626) 2024-09-10T10:04:25.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.791+0000 7f0676800640 1 -- 172.21.15.77:0/2701691379 learned_addr learned my addr 172.21.15.77:0/2701691379 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:04:25.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.791+0000 7f0676800640 1 -- 172.21.15.77:0/2701691379 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555dbc040c70 msgr2=0x555dbc05f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:25.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.791+0000 7f0676800640 1 --2- 172.21.15.77:0/2701691379 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555dbc040c70 0x555dbc05f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:25.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.791+0000 7f0676800640 1 -- 172.21.15.77:0/2701691379 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555dbc041590 msgr2=0x555dbc05f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:25.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.791+0000 7f0676800640 1 --2- 172.21.15.77:0/2701691379 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555dbc041590 0x555dbc05f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:25.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.791+0000 7f0676800640 1 -- 172.21.15.77:0/2701691379 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555dbc038770 con 0x555dbc04be50 2024-09-10T10:04:25.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.791+0000 7f0676800640 1 --2- 172.21.15.77:0/2701691379 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555dbc04be50 0x555dbc05fac0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f0660004620 tx=0x7f0660003d40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89a5546863fe3871 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:25.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.791+0000 7f065b400640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T10:04:25.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.792+0000 7f065b400640 1 -- 172.21.15.77:0/2701691379 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0660037060 con 0x555dbc04be50 2024-09-10T10:04:25.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.792+0000 7f067f5ea740 1 -- 172.21.15.77:0/2701691379 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555dbbfaee60 con 0x555dbc04be50 2024-09-10T10:04:25.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.792+0000 7f065b400640 1 -- 172.21.15.77:0/2701691379 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f0660027aa0 con 0x555dbc04be50 2024-09-10T10:04:25.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.792+0000 7f065b400640 1 -- 172.21.15.77:0/2701691379 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f066003e050 con 0x555dbc04be50 2024-09-10T10:04:25.797 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.793+0000 7f067f5ea740 10 client.27341 fetch_fsmap learned FSMap version 39 2024-09-10T10:04:25.797 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.793+0000 7f067f5ea740 1 -- 172.21.15.77:0/2701691379 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x555dbc03e2b0 con 0x555dbc04be50 2024-09-10T10:04:25.797 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.793+0000 7f065b400640 1 -- 172.21.15.77:0/2701691379 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x555dbbfaee60 con 0x555dbc04be50 2024-09-10T10:04:25.797 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.793+0000 7f065b400640 1 -- 172.21.15.77:0/2701691379 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8394+0+0 (secure 0 0 0) 0x7f066001c040 con 0x555dbc04be50 2024-09-10T10:04:25.797 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.793+0000 7f067f5ea740 10 client.27341 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T10:04:25.797 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.793+0000 7f067f5ea740 10 client.27341 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-10T10:04:25.797 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.793+0000 7f067f5ea740 20 client.27341 populate_metadata read hostname 'smithi077' 2024-09-10T10:04:25.797 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.793+0000 7f067f5ea740 1 --2- 172.21.15.77:0/2701691379 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x555dbc0c6de0 0x555dbc0c9200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:25.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.793+0000 7f067f5ea740 4 client.27341 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:04:25.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.793+0000 7f067f5ea740 1 -- 172.21.15.77:0/2701691379 --> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555dbbf2b010 con 0x555dbc0c6de0 2024-09-10T10:04:25.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.794+0000 7f0675e00640 1 --2- 172.21.15.77:0/2701691379 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x555dbc0c6de0 0x555dbc0c9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:25.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.794+0000 7f0675e00640 1 --2- 172.21.15.77:0/2701691379 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x555dbc0c6de0 0x555dbc0c9200 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f147c6c9797c5c60 server_cookie=cf3b728752cfa25 in_seq=0 out_seq=0 2024-09-10T10:04:25.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.794+0000 7f065b400640 10 client.27341 ms_handle_connect on v2:172.21.15.134:6834/3293933699 2024-09-10T10:04:25.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.794+0000 7f065b400640 1 -- 172.21.15.77:0/2701691379 <== mds.4 v2:172.21.15.134:6834/3293933699 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555dbbf2b010 con 0x555dbc0c6de0 2024-09-10T10:04:25.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.795+0000 7f065b400640 10 client.27341 handle_command_reply: tid=0 2024-09-10T10:04:25.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.795+0000 7f067f5ea740 1 client.27341 shutdown 2024-09-10T10:04:25.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.795+0000 7f067f5ea740 1 -- 172.21.15.77:0/2701691379 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555dbc04be50 msgr2=0x555dbc05fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:25.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.795+0000 7f067f5ea740 1 --2- 172.21.15.77:0/2701691379 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555dbc04be50 0x555dbc05fac0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f0660004620 tx=0x7f0660003d40 comp rx=0 tx=0).stop 2024-09-10T10:04:25.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.795+0000 7f067f5ea740 1 -- 172.21.15.77:0/2701691379 shutdown_connections 2024-09-10T10:04:25.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.795+0000 7f067f5ea740 1 --2- 172.21.15.77:0/2701691379 >> [v2:172.21.15.134:6834/3293933699,v1:172.21.15.134:6835/3293933699] conn(0x555dbc0c6de0 0x555dbc0c9200 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:25.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.795+0000 7f067f5ea740 1 --2- 172.21.15.77:0/2701691379 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555dbc04be50 0x555dbc05fac0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:25.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.795+0000 7f067f5ea740 1 --2- 172.21.15.77:0/2701691379 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555dbc041590 0x555dbc05f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:25.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.795+0000 7f067f5ea740 1 --2- 172.21.15.77:0/2701691379 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555dbc040c70 0x555dbc05f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:25.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.795+0000 7f067f5ea740 1 -- 172.21.15.77:0/2701691379 >> 172.21.15.77:0/2701691379 conn(0x555dbc03d200 msgr2=0x555dbc048cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:25.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.795+0000 7f065b400640 0 client.27341 ms_handle_reset on v2:172.21.15.134:6834/3293933699 2024-09-10T10:04:25.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.795+0000 7f067f5ea740 1 -- 172.21.15.77:0/2701691379 shutdown_connections 2024-09-10T10:04:25.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.795+0000 7f067f5ea740 1 -- 172.21.15.77:0/2701691379 wait complete. 2024-09-10T10:04:25.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.795+0000 7f067f5ea740 20 client.27341 trim_cache size 0 max 16384 2024-09-10T10:04:25.799 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-10T10:04:25.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.796+0000 7f067ca00640 1 -- 172.21.15.77:0/4275487576 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f065c0a2b80 msgr2=0x7f065c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:25.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.796+0000 7f067ca00640 1 --2- 172.21.15.77:0/4275487576 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f065c0a2b80 0x7f065c0a5040 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x555dbc03e580 tx=0x7f0660004140 comp rx=0 tx=0).stop 2024-09-10T10:04:25.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.796+0000 7f067ca00640 1 -- 172.21.15.77:0/4275487576 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0678103fd0 msgr2=0x7f067819a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:25.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.796+0000 7f067ca00640 1 --2- 172.21.15.77:0/4275487576 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0678103fd0 0x7f067819a620 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f067000e9f0 tx=0x7f067000eeb0 comp rx=0 tx=0).stop 2024-09-10T10:04:25.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.796+0000 7f067ca00640 1 -- 172.21.15.77:0/4275487576 shutdown_connections 2024-09-10T10:04:25.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.796+0000 7f067ca00640 1 --2- 172.21.15.77:0/4275487576 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f065c0a2b80 0x7f065c0a5040 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:25.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.796+0000 7f067ca00640 1 --2- 172.21.15.77:0/4275487576 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0678109100 0x7f067819eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:25.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.796+0000 7f067ca00640 1 --2- 172.21.15.77:0/4275487576 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0678104980 0x7f067819ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:25.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.796+0000 7f067ca00640 1 --2- 172.21.15.77:0/4275487576 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0678103fd0 0x7f067819a620 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:25.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.796+0000 7f067ca00640 1 -- 172.21.15.77:0/4275487576 >> 172.21.15.77:0/4275487576 conn(0x7f06780ffa30 msgr2=0x7f0678100da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:25.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.797+0000 7f067ca00640 1 -- 172.21.15.77:0/4275487576 shutdown_connections 2024-09-10T10:04:25.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:25.797+0000 7f067ca00640 1 -- 172.21.15.77:0/4275487576 wait complete. 2024-09-10T10:04:26.063 DEBUG:teuthology.parallel:result is None 2024-09-10T10:04:26.063 DEBUG:teuthology.orchestra.run.smithi077:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-10T10:04:26.127 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-10T10:04:26.127 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-09-10T10:04:26.138 INFO:tasks.quiescer:joining Quiescers 2024-09-10T10:04:26.138 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-09-10T10:04:26.138 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:04:26.153 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:04:27.007 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T10:04:27.202 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:04:26 smithi077 ceph-mon[30319]: pgmap v669: 129 pgs: 129 active+clean; 139 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.0 MiB/s wr, 382 op/s 2024-09-10T10:04:27.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:04:26 smithi134 ceph-mon[35674]: pgmap v669: 129 pgs: 129 active+clean; 139 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.0 MiB/s wr, 382 op/s 2024-09-10T10:04:27.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:04:26 smithi120 ceph-mon[34770]: pgmap v669: 129 pgs: 129 active+clean; 139 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.0 MiB/s wr, 382 op/s 2024-09-10T10:04:27.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.466+0000 7fbc51200640 1 -- 172.21.15.77:0/1571508913 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc4c108f60 msgr2=0x7fbc4c110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:27.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.466+0000 7fbc51200640 1 --2- 172.21.15.77:0/1571508913 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc4c108f60 0x7fbc4c110ae0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fbc44009ca0 tx=0x7fbc4402a8b0 comp rx=0 tx=0).stop 2024-09-10T10:04:27.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.466+0000 7fbc51200640 1 -- 172.21.15.77:0/1571508913 shutdown_connections 2024-09-10T10:04:27.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.466+0000 7fbc51200640 1 --2- 172.21.15.77:0/1571508913 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc4c108f60 0x7fbc4c110ae0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:27.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.466+0000 7fbc51200640 1 --2- 172.21.15.77:0/1571508913 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc4c1047e0 0x7fbc4c108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:27.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.466+0000 7fbc51200640 1 --2- 172.21.15.77:0/1571508913 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbc4c103e30 0x7fbc4c104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:27.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.466+0000 7fbc51200640 1 -- 172.21.15.77:0/1571508913 >> 172.21.15.77:0/1571508913 conn(0x7fbc4c0ff8d0 msgr2=0x7fbc4c101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:27.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.467+0000 7fbc51200640 1 -- 172.21.15.77:0/1571508913 shutdown_connections 2024-09-10T10:04:27.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.467+0000 7fbc51200640 1 -- 172.21.15.77:0/1571508913 wait complete. 2024-09-10T10:04:27.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.468+0000 7fbc51200640 1 Processor -- start 2024-09-10T10:04:27.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.468+0000 7fbc51200640 1 -- start start 2024-09-10T10:04:27.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.468+0000 7fbc51200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc4c103e30 0x7fbc4c19a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:27.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.468+0000 7fbc51200640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbc4c1047e0 0x7fbc4c19a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:27.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.468+0000 7fbc51200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc4c108f60 0x7fbc4c19ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:27.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.468+0000 7fbc51200640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbc4c114010 con 0x7fbc4c108f60 2024-09-10T10:04:27.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.468+0000 7fbc51200640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbc4c113e90 con 0x7fbc4c1047e0 2024-09-10T10:04:27.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.468+0000 7fbc51200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbc4c114190 con 0x7fbc4c103e30 2024-09-10T10:04:27.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.469+0000 7fbc4b400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc4c103e30 0x7fbc4c19a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:27.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.469+0000 7fbc4b400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc4c103e30 0x7fbc4c19a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:33194/0 (socket says 172.21.15.77:33194) 2024-09-10T10:04:27.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.469+0000 7fbc4b400640 1 -- 172.21.15.77:0/4259916544 learned_addr learned my addr 172.21.15.77:0/4259916544 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:04:27.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.469+0000 7fbc42a00640 1 --2- 172.21.15.77:0/4259916544 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbc4c1047e0 0x7fbc4c19a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:27.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.469+0000 7fbc4be00640 1 --2- 172.21.15.77:0/4259916544 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc4c108f60 0x7fbc4c19ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:27.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.470+0000 7fbc4be00640 1 -- 172.21.15.77:0/4259916544 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc4c103e30 msgr2=0x7fbc4c19a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:27.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.470+0000 7fbc4be00640 1 --2- 172.21.15.77:0/4259916544 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc4c103e30 0x7fbc4c19a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:27.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.470+0000 7fbc4be00640 1 -- 172.21.15.77:0/4259916544 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbc4c1047e0 msgr2=0x7fbc4c19a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:27.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.470+0000 7fbc4be00640 1 --2- 172.21.15.77:0/4259916544 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbc4c1047e0 0x7fbc4c19a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:27.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.470+0000 7fbc4be00640 1 -- 172.21.15.77:0/4259916544 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc4c19f3c0 con 0x7fbc4c108f60 2024-09-10T10:04:27.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.470+0000 7fbc4b400640 1 --2- 172.21.15.77:0/4259916544 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc4c103e30 0x7fbc4c19a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:04:27.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.470+0000 7fbc4be00640 1 --2- 172.21.15.77:0/4259916544 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc4c108f60 0x7fbc4c19ece0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fbc44005e30 tx=0x7fbc44003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:27.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.470+0000 7fbc48c00640 1 -- 172.21.15.77:0/4259916544 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc44030dc0 con 0x7fbc4c108f60 2024-09-10T10:04:27.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.470+0000 7fbc51200640 1 -- 172.21.15.77:0/4259916544 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc4c19f650 con 0x7fbc4c108f60 2024-09-10T10:04:27.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.471+0000 7fbc48c00640 1 -- 172.21.15.77:0/4259916544 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc44040070 con 0x7fbc4c108f60 2024-09-10T10:04:27.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.471+0000 7fbc51200640 1 -- 172.21.15.77:0/4259916544 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc4c076630 con 0x7fbc4c108f60 2024-09-10T10:04:27.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.471+0000 7fbc48c00640 1 -- 172.21.15.77:0/4259916544 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc44045a70 con 0x7fbc4c108f60 2024-09-10T10:04:27.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.474+0000 7fbc48c00640 1 -- 172.21.15.77:0/4259916544 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fbc44045c10 con 0x7fbc4c108f60 2024-09-10T10:04:27.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.474+0000 7fbc5541d740 1 Processor -- start 2024-09-10T10:04:27.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.474+0000 7fbc5541d740 1 -- start start 2024-09-10T10:04:27.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.474+0000 7fbc5541d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561466b6d560 0x561466b6d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:27.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.474+0000 7fbc5541d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561466b6de80 0x561466b78200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:27.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.474+0000 7fbc5541d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561466b78740 0x561466b7ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:27.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.474+0000 7fbc5541d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x561466ad3530 con 0x561466b6de80 2024-09-10T10:04:27.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.474+0000 7fbc5541d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x561466a7a860 con 0x561466b6d560 2024-09-10T10:04:27.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.474+0000 7fbc5541d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x561466acf4f0 con 0x561466b78740 2024-09-10T10:04:27.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.475+0000 7fbc42a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561466b6d560 0x561466b6d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:27.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.475+0000 7fbc42a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561466b6d560 0x561466b6d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:57658/0 (socket says 172.21.15.77:57658) 2024-09-10T10:04:27.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.475+0000 7fbc42a00640 1 -- 172.21.15.77:0/3594742639 learned_addr learned my addr 172.21.15.77:0/3594742639 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:04:27.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.475+0000 7fbc42a00640 1 --2- 172.21.15.77:0/3594742639 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561466b78740 0x561466b7ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:27.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.475+0000 7fbc4b400640 1 --2- 172.21.15.77:0/3594742639 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561466b6de80 0x561466b78200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:27.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.475+0000 7fbc42a00640 1 -- 172.21.15.77:0/3594742639 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561466b6d560 msgr2=0x561466b6d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:27.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.475+0000 7fbc42a00640 1 --2- 172.21.15.77:0/3594742639 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561466b6d560 0x561466b6d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:27.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.475+0000 7fbc42a00640 1 -- 172.21.15.77:0/3594742639 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561466b6de80 msgr2=0x561466b78200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:27.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.475+0000 7fbc42a00640 1 --2- 172.21.15.77:0/3594742639 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561466b6de80 0x561466b78200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:27.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.475+0000 7fbc42a00640 1 -- 172.21.15.77:0/3594742639 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561466844b90 con 0x561466b78740 2024-09-10T10:04:27.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.475+0000 7fbc4b400640 1 --2- 172.21.15.77:0/3594742639 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561466b6de80 0x561466b78200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:04:27.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.475+0000 7fbc48c00640 1 --2- 172.21.15.77:0/4259916544 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fbc300a2b80 0x7fbc300a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:27.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.476+0000 7fbc42a00640 1 --2- 172.21.15.77:0/3594742639 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561466b78740 0x561466b7ab30 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fbc340137a0 tx=0x7fbc34004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d03c6f321e62208a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:27.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.476+0000 7fbc48c00640 1 -- 172.21.15.77:0/4259916544 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fbc440f09a0 con 0x7fbc4c108f60 2024-09-10T10:04:27.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.476+0000 7fbc4b400640 1 --2- 172.21.15.77:0/4259916544 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fbc300a2b80 0x7fbc300a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:27.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.476+0000 7fbc40c00640 1 -- 172.21.15.77:0/3594742639 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc34013050 con 0x561466b78740 2024-09-10T10:04:27.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.476+0000 7fbc40c00640 1 -- 172.21.15.77:0/3594742639 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc340131f0 con 0x561466b78740 2024-09-10T10:04:27.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.476+0000 7fbc40c00640 1 -- 172.21.15.77:0/3594742639 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc340108f0 con 0x561466b78740 2024-09-10T10:04:27.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.477+0000 7fbc5541d740 1 -- 172.21.15.77:0/3594742639 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561466b78740 msgr2=0x561466b7ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:27.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.477+0000 7fbc4b400640 1 --2- 172.21.15.77:0/4259916544 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fbc300a2b80 0x7fbc300a5040 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x561466b6a950 tx=0x7fbc3c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:27.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.477+0000 7fbc5541d740 1 --2- 172.21.15.77:0/3594742639 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561466b78740 0x561466b7ab30 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fbc340137a0 tx=0x7fbc34004730 comp rx=0 tx=0).stop 2024-09-10T10:04:27.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.477+0000 7fbc5541d740 1 -- 172.21.15.77:0/3594742639 shutdown_connections 2024-09-10T10:04:27.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.477+0000 7fbc5541d740 1 --2- 172.21.15.77:0/3594742639 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561466b78740 0x561466b7ab30 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:27.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.477+0000 7fbc5541d740 1 --2- 172.21.15.77:0/3594742639 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561466b6de80 0x561466b78200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:27.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.477+0000 7fbc5541d740 1 --2- 172.21.15.77:0/3594742639 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561466b6d560 0x561466b6d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:27.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.477+0000 7fbc5541d740 1 -- 172.21.15.77:0/3594742639 >> 172.21.15.77:0/3594742639 conn(0x561466b69af0 msgr2=0x561466b69ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:27.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.477+0000 7fbc5541d740 1 -- 172.21.15.77:0/3594742639 shutdown_connections 2024-09-10T10:04:27.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.477+0000 7fbc5541d740 1 -- 172.21.15.77:0/3594742639 wait complete. 2024-09-10T10:04:27.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.478+0000 7fbc5541d740 1 -- start start 2024-09-10T10:04:27.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.478+0000 7fbc5541d740 1 Processor -- start 2024-09-10T10:04:27.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.478+0000 7fbc5541d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561466b6d560 0x561466b75670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:27.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.478+0000 7fbc5541d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561466b6de80 0x561466b75bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:27.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.478+0000 7fbc5541d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561466b78740 0x561466b760f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:27.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.478+0000 7fbc5541d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x561466b7b310 con 0x561466b78740 2024-09-10T10:04:27.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.478+0000 7fbc5541d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x561466a8de60 con 0x561466b6d560 2024-09-10T10:04:27.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.478+0000 7fbc5541d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x561466b7b490 con 0x561466b6de80 2024-09-10T10:04:27.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.478+0000 7fbc5541d740 10 client.0 _ll_register_callbacks cb 0x561466a92cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:04:27.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.479+0000 7fbc4b400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561466b6d560 0x561466b75670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:27.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.479+0000 7fbc42a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561466b6de80 0x561466b75bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:27.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.479+0000 7fbc4b400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561466b6d560 0x561466b75670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:57666/0 (socket says 172.21.15.77:57666) 2024-09-10T10:04:27.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.479+0000 7fbc4b400640 1 -- 172.21.15.77:0/3566072182 learned_addr learned my addr 172.21.15.77:0/3566072182 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:04:27.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.479+0000 7fbc4be00640 1 --2- 172.21.15.77:0/3566072182 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561466b78740 0x561466b760f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:27.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.479+0000 7fbc4b400640 1 -- 172.21.15.77:0/3566072182 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561466b6de80 msgr2=0x561466b75bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:27.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.479+0000 7fbc4b400640 1 --2- 172.21.15.77:0/3566072182 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561466b6de80 0x561466b75bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:27.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.479+0000 7fbc4b400640 1 -- 172.21.15.77:0/3566072182 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561466b78740 msgr2=0x561466b760f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:27.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.479+0000 7fbc4b400640 1 --2- 172.21.15.77:0/3566072182 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561466b78740 0x561466b760f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:27.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.479+0000 7fbc4b400640 1 -- 172.21.15.77:0/3566072182 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561466a99990 con 0x561466b6d560 2024-09-10T10:04:27.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.479+0000 7fbc4be00640 1 --2- 172.21.15.77:0/3566072182 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561466b78740 0x561466b760f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:04:27.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.480+0000 7fbc4b400640 1 --2- 172.21.15.77:0/3566072182 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561466b6d560 0x561466b75670 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c011cf0 tx=0x7fbc3c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=340f2678cd11c2ad server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:27.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.480+0000 7fbc39600640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T10:04:27.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.481+0000 7fbc39600640 1 -- 172.21.15.77:0/3566072182 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc3c011ea0 con 0x561466b6d560 2024-09-10T10:04:27.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.481+0000 7fbc5541d740 1 -- 172.21.15.77:0/3566072182 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc340131f0 con 0x561466b6d560 2024-09-10T10:04:27.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.481+0000 7fbc39600640 1 -- 172.21.15.77:0/3566072182 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fbc3c01fd30 con 0x561466b6d560 2024-09-10T10:04:27.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.481+0000 7fbc39600640 1 -- 172.21.15.77:0/3566072182 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fbc3c034050 con 0x561466b6d560 2024-09-10T10:04:27.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.481+0000 7fbc5541d740 10 client.27353 fetch_fsmap learned FSMap version 39 2024-09-10T10:04:27.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.481+0000 7fbc5541d740 1 -- 172.21.15.77:0/3566072182 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x561466b86f70 con 0x561466b6d560 2024-09-10T10:04:27.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.481+0000 7fbc39600640 1 -- 172.21.15.77:0/3566072182 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc340131f0 con 0x561466b6d560 2024-09-10T10:04:27.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.482+0000 7fbc39600640 1 -- 172.21.15.77:0/3566072182 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8394+0+0 (secure 0 0 0) 0x7fbc3c029620 con 0x561466b6d560 2024-09-10T10:04:27.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.482+0000 7fbc5541d740 10 client.27353 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T10:04:27.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.482+0000 7fbc5541d740 10 client.27353 resolve_mds: validated gid 24563 aka daemon mds.b 2024-09-10T10:04:27.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.482+0000 7fbc5541d740 20 client.27353 populate_metadata read hostname 'smithi077' 2024-09-10T10:04:27.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.482+0000 7fbc5541d740 1 --2- 172.21.15.77:0/3566072182 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x561466b883d0 0x561466b8a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:27.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.482+0000 7fbc5541d740 4 client.27353 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:04:27.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.482+0000 7fbc5541d740 1 -- 172.21.15.77:0/3566072182 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561466a76310 con 0x561466b883d0 2024-09-10T10:04:27.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.482+0000 7fbc42a00640 1 --2- 172.21.15.77:0/3566072182 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x561466b883d0 0x561466b8a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:27.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.483+0000 7fbc42a00640 1 --2- 172.21.15.77:0/3566072182 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x561466b883d0 0x561466b8a830 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6c953583bdbde4d server_cookie=56e7c8f33ad610a1 in_seq=0 out_seq=0 2024-09-10T10:04:27.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.483+0000 7fbc39600640 10 client.27353 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T10:04:27.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.483+0000 7fbc39600640 1 -- 172.21.15.77:0/3566072182 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561466a76310 con 0x561466b883d0 2024-09-10T10:04:27.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.483+0000 7fbc39600640 10 client.27353 handle_command_reply: tid=0 2024-09-10T10:04:27.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.484+0000 7fbc5541d740 1 client.27353 shutdown 2024-09-10T10:04:27.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.484+0000 7fbc5541d740 1 -- 172.21.15.77:0/3566072182 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561466b6d560 msgr2=0x561466b75670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:27.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.484+0000 7fbc5541d740 1 --2- 172.21.15.77:0/3566072182 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561466b6d560 0x561466b75670 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c011cf0 tx=0x7fbc3c0046c0 comp rx=0 tx=0).stop 2024-09-10T10:04:27.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.484+0000 7fbc5541d740 1 -- 172.21.15.77:0/3566072182 shutdown_connections 2024-09-10T10:04:27.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.484+0000 7fbc5541d740 1 --2- 172.21.15.77:0/3566072182 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x561466b883d0 0x561466b8a830 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:27.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.484+0000 7fbc5541d740 1 --2- 172.21.15.77:0/3566072182 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561466b78740 0x561466b760f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:27.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.484+0000 7fbc5541d740 1 --2- 172.21.15.77:0/3566072182 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561466b6de80 0x561466b75bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:27.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.484+0000 7fbc5541d740 1 --2- 172.21.15.77:0/3566072182 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561466b6d560 0x561466b75670 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:27.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.484+0000 7fbc5541d740 1 -- 172.21.15.77:0/3566072182 >> 172.21.15.77:0/3566072182 conn(0x561466b69af0 msgr2=0x561466b6f820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:27.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.484+0000 7fbc39600640 0 client.27353 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T10:04:27.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.484+0000 7fbc5541d740 1 -- 172.21.15.77:0/3566072182 shutdown_connections 2024-09-10T10:04:27.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.484+0000 7fbc5541d740 1 -- 172.21.15.77:0/3566072182 wait complete. 2024-09-10T10:04:27.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.484+0000 7fbc5541d740 20 client.27353 trim_cache size 0 max 16384 2024-09-10T10:04:27.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.503+0000 7fbc5541d740 1 Processor -- start 2024-09-10T10:04:27.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.503+0000 7fbc5541d740 1 -- start start 2024-09-10T10:04:27.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.503+0000 7fbc5541d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561466b6d560 0x561466b73f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:27.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.504+0000 7fbc5541d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561466b6de80 0x561466b90100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:27.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.504+0000 7fbc5541d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561466b78740 0x561466b90640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:27.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.504+0000 7fbc5541d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbc4c1b52b0 con 0x561466b78740 2024-09-10T10:04:27.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.504+0000 7fbc5541d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbc4c1b6eb0 con 0x561466b6de80 2024-09-10T10:04:27.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.504+0000 7fbc5541d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbc4c1b59e0 con 0x561466b6d560 2024-09-10T10:04:27.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.504+0000 7fbc42a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561466b6de80 0x561466b90100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:27.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.504+0000 7fbc4be00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561466b78740 0x561466b90640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:27.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.504+0000 7fbc42a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561466b6de80 0x561466b90100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:57678/0 (socket says 172.21.15.77:57678) 2024-09-10T10:04:27.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.504+0000 7fbc4be00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561466b78740 0x561466b90640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:39408/0 (socket says 172.21.15.77:39408) 2024-09-10T10:04:27.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.504+0000 7fbc42a00640 1 -- 172.21.15.77:0/3783045667 learned_addr learned my addr 172.21.15.77:0/3783045667 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:04:27.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.504+0000 7fbc4b400640 1 --2- 172.21.15.77:0/3783045667 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561466b6d560 0x561466b73f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:27.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.504+0000 7fbc42a00640 1 -- 172.21.15.77:0/3783045667 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561466b6d560 msgr2=0x561466b73f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:27.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.504+0000 7fbc42a00640 1 --2- 172.21.15.77:0/3783045667 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561466b6d560 0x561466b73f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:27.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.504+0000 7fbc42a00640 1 -- 172.21.15.77:0/3783045667 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561466b78740 msgr2=0x561466b90640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:27.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.504+0000 7fbc42a00640 1 --2- 172.21.15.77:0/3783045667 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561466b78740 0x561466b90640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:27.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.504+0000 7fbc42a00640 1 -- 172.21.15.77:0/3783045667 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561466b71060 con 0x561466b6de80 2024-09-10T10:04:27.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.505+0000 7fbc42a00640 1 --2- 172.21.15.77:0/3783045667 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561466b6de80 0x561466b90100 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fbc340137a0 tx=0x7fbc340104d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f00262b0995b9b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:27.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.505+0000 7fbc3be00640 1 -- 172.21.15.77:0/3783045667 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc34004240 con 0x561466b6de80 2024-09-10T10:04:27.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.505+0000 7fbc3be00640 1 -- 172.21.15.77:0/3783045667 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc340043e0 con 0x561466b6de80 2024-09-10T10:04:27.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.505+0000 7fbc3be00640 1 -- 172.21.15.77:0/3783045667 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc34010bd0 con 0x561466b6de80 2024-09-10T10:04:27.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.506+0000 7fbc5541d740 1 -- 172.21.15.77:0/3783045667 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561466b6de80 msgr2=0x561466b90100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:27.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.506+0000 7fbc5541d740 1 --2- 172.21.15.77:0/3783045667 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561466b6de80 0x561466b90100 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fbc340137a0 tx=0x7fbc340104d0 comp rx=0 tx=0).stop 2024-09-10T10:04:27.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.506+0000 7fbc5541d740 1 -- 172.21.15.77:0/3783045667 shutdown_connections 2024-09-10T10:04:27.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.506+0000 7fbc5541d740 1 --2- 172.21.15.77:0/3783045667 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561466b78740 0x561466b90640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:27.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.506+0000 7fbc5541d740 1 --2- 172.21.15.77:0/3783045667 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561466b6de80 0x561466b90100 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:27.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.506+0000 7fbc5541d740 1 --2- 172.21.15.77:0/3783045667 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561466b6d560 0x561466b73f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:27.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.506+0000 7fbc5541d740 1 -- 172.21.15.77:0/3783045667 >> 172.21.15.77:0/3783045667 conn(0x561466b69af0 msgr2=0x561466b6f820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:27.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.507+0000 7fbc5541d740 1 -- 172.21.15.77:0/3783045667 shutdown_connections 2024-09-10T10:04:27.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.507+0000 7fbc5541d740 1 -- 172.21.15.77:0/3783045667 wait complete. 2024-09-10T10:04:27.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.507+0000 7fbc5541d740 1 -- start start 2024-09-10T10:04:27.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.507+0000 7fbc5541d740 1 Processor -- start 2024-09-10T10:04:27.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.508+0000 7fbc5541d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561466b6d560 0x561466b806e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:27.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.508+0000 7fbc5541d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561466b6de80 0x561466b80c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:27.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.508+0000 7fbc5541d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561466b78740 0x561466b7d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:27.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.508+0000 7fbc5541d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbc4c1b67e0 con 0x561466b6de80 2024-09-10T10:04:27.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.508+0000 7fbc5541d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbc4c1b60e0 con 0x561466b6d560 2024-09-10T10:04:27.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.508+0000 7fbc5541d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbc4c1b3290 con 0x561466b78740 2024-09-10T10:04:27.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.508+0000 7fbc5541d740 10 client.0 _ll_register_callbacks cb 0x7fbc3c011bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:04:27.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.508+0000 7fbc42a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561466b6de80 0x561466b80c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:27.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.509+0000 7fbc42a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561466b6de80 0x561466b80c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:39418/0 (socket says 172.21.15.77:39418) 2024-09-10T10:04:27.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.509+0000 7fbc42a00640 1 -- 172.21.15.77:0/3919680390 learned_addr learned my addr 172.21.15.77:0/3919680390 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:04:27.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.509+0000 7fbc4b400640 1 --2- 172.21.15.77:0/3919680390 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561466b6d560 0x561466b806e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:27.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.509+0000 7fbc4b400640 1 -- 172.21.15.77:0/3919680390 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561466b78740 msgr2=0x561466b7d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:27.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.509+0000 7fbc4b400640 1 --2- 172.21.15.77:0/3919680390 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561466b78740 0x561466b7d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:27.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.509+0000 7fbc4b400640 1 -- 172.21.15.77:0/3919680390 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561466b6de80 msgr2=0x561466b80c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:27.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.509+0000 7fbc4b400640 1 --2- 172.21.15.77:0/3919680390 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561466b6de80 0x561466b80c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:27.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.509+0000 7fbc4b400640 1 -- 172.21.15.77:0/3919680390 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561466b74640 con 0x561466b6d560 2024-09-10T10:04:27.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.509+0000 7fbc4b400640 1 --2- 172.21.15.77:0/3919680390 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561466b6d560 0x561466b806e0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c029c30 tx=0x7fbc3c01fde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e44a45cdff3453da server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:27.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.509+0000 7fbc39600640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T10:04:27.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.510+0000 7fbc39600640 1 -- 172.21.15.77:0/3919680390 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc3c029800 con 0x561466b6d560 2024-09-10T10:04:27.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.510+0000 7fbc5541d740 1 -- 172.21.15.77:0/3919680390 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc340043e0 con 0x561466b6d560 2024-09-10T10:04:27.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.510+0000 7fbc39600640 1 -- 172.21.15.77:0/3919680390 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fbc3c004030 con 0x561466b6d560 2024-09-10T10:04:27.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.510+0000 7fbc39600640 1 -- 172.21.15.77:0/3919680390 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fbc3c045050 con 0x561466b6d560 2024-09-10T10:04:27.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.510+0000 7fbc5541d740 10 client.27359 fetch_fsmap learned FSMap version 39 2024-09-10T10:04:27.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.510+0000 7fbc5541d740 1 -- 172.21.15.77:0/3919680390 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x561466bf3460 con 0x561466b6d560 2024-09-10T10:04:27.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.510+0000 7fbc39600640 1 -- 172.21.15.77:0/3919680390 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc340043e0 con 0x561466b6d560 2024-09-10T10:04:27.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.511+0000 7fbc39600640 1 -- 172.21.15.77:0/3919680390 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8394+0+0 (secure 0 0 0) 0x7fbc3c040030 con 0x561466b6d560 2024-09-10T10:04:27.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.511+0000 7fbc5541d740 10 client.27359 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T10:04:27.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.511+0000 7fbc5541d740 10 client.27359 resolve_mds: validated gid 24563 aka daemon mds.b 2024-09-10T10:04:27.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.511+0000 7fbc5541d740 20 client.27359 populate_metadata read hostname 'smithi077' 2024-09-10T10:04:27.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.511+0000 7fbc5541d740 1 --2- 172.21.15.77:0/3919680390 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x561466bf3cd0 0x561466bf6110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:27.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.511+0000 7fbc5541d740 4 client.27359 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.6, "await_for": 45.0}] 2024-09-10T10:04:27.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.511+0000 7fbc5541d740 1 -- 172.21.15.77:0/3919680390 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.6, "await_for": 45.0}) -- 0x561466ae0c60 con 0x561466bf3cd0 2024-09-10T10:04:27.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.512+0000 7fbc42a00640 1 --2- 172.21.15.77:0/3919680390 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x561466bf3cd0 0x561466bf6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:27.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.512+0000 7fbc42a00640 1 --2- 172.21.15.77:0/3919680390 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x561466bf3cd0 0x561466bf6110 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a17925f32b730410 server_cookie=d6ebdf92539a04cb in_seq=0 out_seq=0 2024-09-10T10:04:27.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.512+0000 7fbc39600640 10 client.27359 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T10:04:27.558 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:04:27.562 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:04:27.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.967+0000 7fbc39600640 1 -- 172.21.15.77:0/3919680390 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x561466ae0c60 con 0x561466bf3cd0 2024-09-10T10:04:27.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.967+0000 7fbc39600640 10 client.27359 handle_command_reply: tid=0 2024-09-10T10:04:27.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.967+0000 7fbc5541d740 1 client.27359 shutdown 2024-09-10T10:04:27.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.967+0000 7fbc5541d740 1 -- 172.21.15.77:0/3919680390 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561466b6d560 msgr2=0x561466b806e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:27.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.967+0000 7fbc5541d740 1 --2- 172.21.15.77:0/3919680390 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561466b6d560 0x561466b806e0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c029c30 tx=0x7fbc3c01fde0 comp rx=0 tx=0).stop 2024-09-10T10:04:27.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.967+0000 7fbc5541d740 1 -- 172.21.15.77:0/3919680390 shutdown_connections 2024-09-10T10:04:27.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.967+0000 7fbc5541d740 1 --2- 172.21.15.77:0/3919680390 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x561466bf3cd0 0x561466bf6110 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:27.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.967+0000 7fbc5541d740 1 --2- 172.21.15.77:0/3919680390 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561466b78740 0x561466b7d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:27.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.967+0000 7fbc5541d740 1 --2- 172.21.15.77:0/3919680390 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561466b6de80 0x561466b80c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:27.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.967+0000 7fbc5541d740 1 --2- 172.21.15.77:0/3919680390 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561466b6d560 0x561466b806e0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:27.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.967+0000 7fbc5541d740 1 -- 172.21.15.77:0/3919680390 >> 172.21.15.77:0/3919680390 conn(0x561466b69af0 msgr2=0x561466b68450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:27.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.967+0000 7fbc39600640 0 client.27359 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T10:04:27.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.967+0000 7fbc5541d740 1 -- 172.21.15.77:0/3919680390 shutdown_connections 2024-09-10T10:04:27.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.968+0000 7fbc5541d740 1 -- 172.21.15.77:0/3919680390 wait complete. 2024-09-10T10:04:27.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.968+0000 7fbc5541d740 20 client.27359 trim_cache size 0 max 16384 2024-09-10T10:04:27.971 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-10T10:04:27.971 INFO:teuthology.orchestra.run.smithi077.stdout: "epoch": 40, 2024-09-10T10:04:27.971 INFO:teuthology.orchestra.run.smithi077.stdout: "leader": 24563, 2024-09-10T10:04:27.971 INFO:teuthology.orchestra.run.smithi077.stdout: "set_version": 30, 2024-09-10T10:04:27.971 INFO:teuthology.orchestra.run.smithi077.stdout: "sets": { 2024-09-10T10:04:27.971 INFO:teuthology.orchestra.run.smithi077.stdout: "41f52be3": { 2024-09-10T10:04:27.972 INFO:teuthology.orchestra.run.smithi077.stdout: "version": 30, 2024-09-10T10:04:27.972 INFO:teuthology.orchestra.run.smithi077.stdout: "age_ref": 0.0, 2024-09-10T10:04:27.972 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-10T10:04:27.972 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "QUIESCED", 2024-09-10T10:04:27.972 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-10T10:04:27.972 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:27.972 INFO:teuthology.orchestra.run.smithi077.stdout: "timeout": 90.0, 2024-09-10T10:04:27.972 INFO:teuthology.orchestra.run.smithi077.stdout: "expiration": 144.6, 2024-09-10T10:04:27.972 INFO:teuthology.orchestra.run.smithi077.stdout: "members": { 2024-09-10T10:04:27.972 INFO:teuthology.orchestra.run.smithi077.stdout: "file:/": { 2024-09-10T10:04:27.972 INFO:teuthology.orchestra.run.smithi077.stdout: "excluded": false, 2024-09-10T10:04:27.972 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-10T10:04:27.972 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "QUIESCED", 2024-09-10T10:04:27.972 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-10T10:04:27.972 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:27.973 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:27.973 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:27.973 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:27.973 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:27.973 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-10T10:04:27.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.968+0000 7fbc51200640 1 -- 172.21.15.77:0/4259916544 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fbc300a2b80 msgr2=0x7fbc300a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:27.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.968+0000 7fbc51200640 1 --2- 172.21.15.77:0/4259916544 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fbc300a2b80 0x7fbc300a5040 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x561466b6a950 tx=0x7fbc3c005f90 comp rx=0 tx=0).stop 2024-09-10T10:04:27.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.968+0000 7fbc51200640 1 -- 172.21.15.77:0/4259916544 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc4c108f60 msgr2=0x7fbc4c19ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:27.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.968+0000 7fbc51200640 1 --2- 172.21.15.77:0/4259916544 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc4c108f60 0x7fbc4c19ece0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fbc44005e30 tx=0x7fbc44003f10 comp rx=0 tx=0).stop 2024-09-10T10:04:27.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.969+0000 7fbc51200640 1 -- 172.21.15.77:0/4259916544 shutdown_connections 2024-09-10T10:04:27.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.969+0000 7fbc51200640 1 --2- 172.21.15.77:0/4259916544 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fbc300a2b80 0x7fbc300a5040 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:27.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.969+0000 7fbc51200640 1 --2- 172.21.15.77:0/4259916544 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc4c108f60 0x7fbc4c19ece0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:27.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.969+0000 7fbc51200640 1 --2- 172.21.15.77:0/4259916544 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbc4c1047e0 0x7fbc4c19a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:27.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.969+0000 7fbc51200640 1 --2- 172.21.15.77:0/4259916544 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc4c103e30 0x7fbc4c19a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:27.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.969+0000 7fbc51200640 1 -- 172.21.15.77:0/4259916544 >> 172.21.15.77:0/4259916544 conn(0x7fbc4c0ff8d0 msgr2=0x7fbc4c100b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:27.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.969+0000 7fbc51200640 1 -- 172.21.15.77:0/4259916544 shutdown_connections 2024-09-10T10:04:27.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:27.969+0000 7fbc51200640 1 -- 172.21.15.77:0/4259916544 wait complete. 2024-09-10T10:04:29.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:04:28 smithi134 ceph-mon[35674]: pgmap v670: 129 pgs: 129 active+clean; 135 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 407 op/s 2024-09-10T10:04:29.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:04:28 smithi134 ceph-mon[35674]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T10:04:29.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:04:28 smithi120 ceph-mon[34770]: pgmap v670: 129 pgs: 129 active+clean; 135 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 407 op/s 2024-09-10T10:04:29.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:04:28 smithi120 ceph-mon[34770]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T10:04:29.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:04:28 smithi077 ceph-mon[30319]: pgmap v670: 129 pgs: 129 active+clean; 135 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 407 op/s 2024-09-10T10:04:29.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:04:28 smithi077 ceph-mon[30319]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T10:04:29.722 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '41f52be3', quiesce took 9.3 seconds. Will release after: 15.3 2024-09-10T10:04:29.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:29.987+0000 7f62bbc00640 1 -- 172.21.15.77:0/2646582320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62b41049c0 msgr2=0x7f62b4108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:29.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:29.987+0000 7f62bbc00640 1 --2- 172.21.15.77:0/2646582320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62b41049c0 0x7f62b4108a10 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f629c009900 tx=0x7f629c02a3f0 comp rx=0 tx=0).stop 2024-09-10T10:04:29.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:29.987+0000 7f62bbc00640 1 -- 172.21.15.77:0/2646582320 shutdown_connections 2024-09-10T10:04:29.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:29.987+0000 7f62bbc00640 1 --2- 172.21.15.77:0/2646582320 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f62b4109140 0x7f62b4110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:29.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:29.987+0000 7f62bbc00640 1 --2- 172.21.15.77:0/2646582320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62b41049c0 0x7f62b4108a10 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:29.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:29.987+0000 7f62bbc00640 1 --2- 172.21.15.77:0/2646582320 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f62b4104010 0x7f62b41043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:29.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:29.987+0000 7f62bbc00640 1 -- 172.21.15.77:0/2646582320 >> 172.21.15.77:0/2646582320 conn(0x7f62b40ffa50 msgr2=0x7f62b4101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:29.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:29.987+0000 7f62bbc00640 1 -- 172.21.15.77:0/2646582320 shutdown_connections 2024-09-10T10:04:29.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:29.987+0000 7f62bbc00640 1 -- 172.21.15.77:0/2646582320 wait complete. 2024-09-10T10:04:29.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:29.988+0000 7f62bbc00640 1 Processor -- start 2024-09-10T10:04:29.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:29.988+0000 7f62bbc00640 1 -- start start 2024-09-10T10:04:29.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:29.988+0000 7f62bbc00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f62b4104010 0x7f62b419a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:29.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:29.988+0000 7f62bbc00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62b41049c0 0x7f62b419ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:29.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:29.988+0000 7f62bbc00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f62b4109140 0x7f62b419efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:29.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:29.988+0000 7f62bbc00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f62b4114190 con 0x7f62b4109140 2024-09-10T10:04:29.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:29.988+0000 7f62bbc00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f62b4114010 con 0x7f62b4104010 2024-09-10T10:04:29.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:29.988+0000 7f62bbc00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f62b4114310 con 0x7f62b41049c0 2024-09-10T10:04:29.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:29.989+0000 7f62b9200640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f62b4104010 0x7f62b419a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:29.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:29.989+0000 7f62abe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62b41049c0 0x7f62b419ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:29.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:29.989+0000 7f62b9200640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f62b4104010 0x7f62b419a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:57710/0 (socket says 172.21.15.77:57710) 2024-09-10T10:04:29.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:29.989+0000 7f62b9200640 1 -- 172.21.15.77:0/2904391250 learned_addr learned my addr 172.21.15.77:0/2904391250 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:04:29.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:29.989+0000 7f62b9c00640 1 --2- 172.21.15.77:0/2904391250 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f62b4109140 0x7f62b419efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:29.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:29.989+0000 7f62b9200640 1 -- 172.21.15.77:0/2904391250 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62b41049c0 msgr2=0x7f62b419ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:29.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:29.989+0000 7f62b9200640 1 --2- 172.21.15.77:0/2904391250 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62b41049c0 0x7f62b419ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:29.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:29.989+0000 7f62b9200640 1 -- 172.21.15.77:0/2904391250 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f62b4109140 msgr2=0x7f62b419efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:29.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:29.989+0000 7f62b9200640 1 --2- 172.21.15.77:0/2904391250 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f62b4109140 0x7f62b419efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:29.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:29.989+0000 7f62b9200640 1 -- 172.21.15.77:0/2904391250 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62b419f6c0 con 0x7f62b4104010 2024-09-10T10:04:29.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:29.989+0000 7f62b9200640 1 --2- 172.21.15.77:0/2904391250 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f62b4104010 0x7f62b419a710 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f62a400e9f0 tx=0x7f62a400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:29.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:29.990+0000 7f62b2000640 1 -- 172.21.15.77:0/2904391250 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a400ccc0 con 0x7f62b4104010 2024-09-10T10:04:29.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:29.990+0000 7f62b2000640 1 -- 172.21.15.77:0/2904391250 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62a4010910 con 0x7f62b4104010 2024-09-10T10:04:29.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:29.990+0000 7f62b2000640 1 -- 172.21.15.77:0/2904391250 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a40188a0 con 0x7f62b4104010 2024-09-10T10:04:29.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:29.990+0000 7f62bbc00640 1 -- 172.21.15.77:0/2904391250 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62b419f9b0 con 0x7f62b4104010 2024-09-10T10:04:29.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:29.990+0000 7f62bbc00640 1 -- 172.21.15.77:0/2904391250 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62b41a7290 con 0x7f62b4104010 2024-09-10T10:04:29.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:29.991+0000 7f62bbc00640 1 -- 172.21.15.77:0/2904391250 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62b41043f0 con 0x7f62b4104010 2024-09-10T10:04:29.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:29.993+0000 7f62b2000640 1 -- 172.21.15.77:0/2904391250 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f62a4010ac0 con 0x7f62b4104010 2024-09-10T10:04:29.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:29.994+0000 7f62b2000640 1 --2- 172.21.15.77:0/2904391250 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f62980a2c00 0x7f62980a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:29.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:29.994+0000 7f62b2000640 1 -- 172.21.15.77:0/2904391250 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f62a40cfbb0 con 0x7f62b4104010 2024-09-10T10:04:29.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:29.995+0000 7f62abe00640 1 --2- 172.21.15.77:0/2904391250 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f62980a2c00 0x7f62980a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:29.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:29.995+0000 7f62abe00640 1 --2- 172.21.15.77:0/2904391250 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f62980a2c00 0x7f62980a50c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f629c000c00 tx=0x7f629c005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:29.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:29.996+0000 7f62b2000640 1 -- 172.21.15.77:0/2904391250 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f62a409c6c0 con 0x7f62b4104010 2024-09-10T10:04:30.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.210+0000 7f62bbc00640 1 -- 172.21.15.77:0/2904391250 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f62b410c380 con 0x7f62b4104010 2024-09-10T10:04:30.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.211+0000 7f62b2000640 1 -- 172.21.15.77:0/2904391250 <== mon.2 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+11999 (secure 0 0 0) 0x7f62a409bfc0 con 0x7f62b4104010 2024-09-10T10:04:30.213 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T10:04:30.214 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 40 2024-09-10T10:04:30.214 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":40,"btime":"2024-09-10T10:04:27:940036+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3159289675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3159289675},{"type":"v1","addr":"172.21.15.77:6835","nonce":3159289675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1951962341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1951962341},{"type":"v1","addr":"172.21.15.77:6837","nonce":1951962341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":40,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:52:23.368252+0000","modified":"2024-09-10T10:04:27.788564+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24563,"mds_1":24524,"mds_2":24520,"mds_3":24502,"mds_4":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14487":{"gid":14487,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/4291121888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":4291121888},{"type":"v1","addr":"172.21.15.77:6839","nonce":4291121888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14499":{"gid":14499,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6841/445284194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":445284194},{"type":"v1","addr":"172.21.15.77:6841","nonce":445284194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1005054423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1005054423},{"type":"v1","addr":"172.21.15.134:6833","nonce":1005054423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6833/3716307744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":3716307744},{"type":"v1","addr":"172.21.15.120:6833","nonce":3716307744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/1319892544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1319892544},{"type":"v1","addr":"172.21.15.120:6835","nonce":1319892544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.134:6835/3293933699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3293933699},{"type":"v1","addr":"172.21.15.134:6835","nonce":3293933699}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.134:6837/2461497091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2461497091},{"type":"v1","addr":"172.21.15.134:6837","nonce":2461497091}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":20,"addr":"172.21.15.134:6839/1965117680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1965117680},{"type":"v1","addr":"172.21.15.134:6839","nonce":1965117680}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":20,"addr":"172.21.15.120:6837/3033280709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3033280709},{"type":"v1","addr":"172.21.15.120:6837","nonce":3033280709}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.120:6839/1777710157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1777710157},{"type":"v1","addr":"172.21.15.120:6839","nonce":1777710157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24563,"qdb_cluster":[24563,24524,24520,24502,24496]},"id":1}]} 2024-09-10T10:04:30.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.213+0000 7f62bbc00640 1 -- 172.21.15.77:0/2904391250 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f62980a2c00 msgr2=0x7f62980a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:30.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.213+0000 7f62bbc00640 1 --2- 172.21.15.77:0/2904391250 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f62980a2c00 0x7f62980a50c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f629c000c00 tx=0x7f629c005b50 comp rx=0 tx=0).stop 2024-09-10T10:04:30.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.213+0000 7f62bbc00640 1 -- 172.21.15.77:0/2904391250 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f62b4104010 msgr2=0x7f62b419a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:30.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.213+0000 7f62bbc00640 1 --2- 172.21.15.77:0/2904391250 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f62b4104010 0x7f62b419a710 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f62a400e9f0 tx=0x7f62a400edf0 comp rx=0 tx=0).stop 2024-09-10T10:04:30.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.213+0000 7f62bbc00640 1 -- 172.21.15.77:0/2904391250 shutdown_connections 2024-09-10T10:04:30.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.213+0000 7f62bbc00640 1 --2- 172.21.15.77:0/2904391250 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f62980a2c00 0x7f62980a50c0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:30.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.213+0000 7f62bbc00640 1 --2- 172.21.15.77:0/2904391250 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f62b4109140 0x7f62b419efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:30.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.213+0000 7f62bbc00640 1 --2- 172.21.15.77:0/2904391250 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62b41049c0 0x7f62b419ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:30.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.213+0000 7f62bbc00640 1 --2- 172.21.15.77:0/2904391250 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f62b4104010 0x7f62b419a710 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:30.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.213+0000 7f62bbc00640 1 -- 172.21.15.77:0/2904391250 >> 172.21.15.77:0/2904391250 conn(0x7f62b40ffa50 msgr2=0x7f62b4100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:30.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.214+0000 7f62bbc00640 1 -- 172.21.15.77:0/2904391250 shutdown_connections 2024-09-10T10:04:30.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.214+0000 7f62bbc00640 1 -- 172.21.15.77:0/2904391250 wait complete. 2024-09-10T10:04:30.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.269+0000 7f01fae00640 1 -- 172.21.15.77:0/1199777793 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f01f41047e0 msgr2=0x7f01f4108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:30.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.269+0000 7f01fae00640 1 --2- 172.21.15.77:0/1199777793 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f01f41047e0 0x7f01f4108830 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f01e0009900 tx=0x7f01e002a3f0 comp rx=0 tx=0).stop 2024-09-10T10:04:30.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.269+0000 7f01fae00640 1 -- 172.21.15.77:0/1199777793 shutdown_connections 2024-09-10T10:04:30.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.269+0000 7f01fae00640 1 --2- 172.21.15.77:0/1199777793 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01f4108f60 0x7f01f4110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:30.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.269+0000 7f01fae00640 1 --2- 172.21.15.77:0/1199777793 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f01f41047e0 0x7f01f4108830 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:30.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.269+0000 7f01fae00640 1 --2- 172.21.15.77:0/1199777793 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f01f4103e30 0x7f01f4104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:30.272 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.269+0000 7f01fae00640 1 -- 172.21.15.77:0/1199777793 >> 172.21.15.77:0/1199777793 conn(0x7f01f40ff910 msgr2=0x7f01f4101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:30.272 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.269+0000 7f01fae00640 1 -- 172.21.15.77:0/1199777793 shutdown_connections 2024-09-10T10:04:30.272 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.269+0000 7f01fae00640 1 -- 172.21.15.77:0/1199777793 wait complete. 2024-09-10T10:04:30.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.270+0000 7f01fae00640 1 Processor -- start 2024-09-10T10:04:30.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.270+0000 7f01fae00640 1 -- start start 2024-09-10T10:04:30.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.270+0000 7f01fae00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f01f4103e30 0x7f01f419a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:30.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.270+0000 7f01fae00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f01f41047e0 0x7f01f419a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:30.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.270+0000 7f01fae00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01f4108f60 0x7f01f419ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:30.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.271+0000 7f01fae00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f01f4113fd0 con 0x7f01f41047e0 2024-09-10T10:04:30.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.271+0000 7f01fae00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f01f4113e50 con 0x7f01f4103e30 2024-09-10T10:04:30.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.271+0000 7f01fae00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f01f4114150 con 0x7f01f4108f60 2024-09-10T10:04:30.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.271+0000 7f01f9a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f01f4103e30 0x7f01f419a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:30.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.271+0000 7f01f9a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f01f4103e30 0x7f01f419a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:57718/0 (socket says 172.21.15.77:57718) 2024-09-10T10:04:30.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.271+0000 7f01f9a00640 1 -- 172.21.15.77:0/566369547 learned_addr learned my addr 172.21.15.77:0/566369547 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:04:30.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.271+0000 7f01f9000640 1 --2- 172.21.15.77:0/566369547 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f01f41047e0 0x7f01f419a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:30.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.271+0000 7f01fa400640 1 --2- 172.21.15.77:0/566369547 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01f4108f60 0x7f01f419ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:30.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.271+0000 7f01f9a00640 1 -- 172.21.15.77:0/566369547 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01f4108f60 msgr2=0x7f01f419ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:30.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.271+0000 7f01f9a00640 1 --2- 172.21.15.77:0/566369547 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01f4108f60 0x7f01f419ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:30.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.271+0000 7f01f9a00640 1 -- 172.21.15.77:0/566369547 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f01f41047e0 msgr2=0x7f01f419a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:30.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.271+0000 7f01f9a00640 1 --2- 172.21.15.77:0/566369547 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f01f41047e0 0x7f01f419a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:30.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.271+0000 7f01f9a00640 1 -- 172.21.15.77:0/566369547 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01f419f440 con 0x7f01f4103e30 2024-09-10T10:04:30.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.271+0000 7f01fa400640 1 --2- 172.21.15.77:0/566369547 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01f4108f60 0x7f01f419ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:04:30.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.272+0000 7f01f9a00640 1 --2- 172.21.15.77:0/566369547 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f01f4103e30 0x7f01f419a490 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f01e800ea60 tx=0x7f01e800ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:30.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.272+0000 7f01f2000640 1 -- 172.21.15.77:0/566369547 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e800cde0 con 0x7f01f4103e30 2024-09-10T10:04:30.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.272+0000 7f01fae00640 1 -- 172.21.15.77:0/566369547 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01f419f730 con 0x7f01f4103e30 2024-09-10T10:04:30.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.272+0000 7f01f2000640 1 -- 172.21.15.77:0/566369547 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01e80035f0 con 0x7f01f4103e30 2024-09-10T10:04:30.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.272+0000 7f01fae00640 1 -- 172.21.15.77:0/566369547 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01f41a7010 con 0x7f01f4103e30 2024-09-10T10:04:30.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.272+0000 7f01f2000640 1 -- 172.21.15.77:0/566369547 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e8018690 con 0x7f01f4103e30 2024-09-10T10:04:30.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.275+0000 7f01fedff740 1 Processor -- start 2024-09-10T10:04:30.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.275+0000 7f01fedff740 1 -- start start 2024-09-10T10:04:30.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.275+0000 7f01fedff740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561ccd497e40 0x561ccd498220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:30.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.275+0000 7f01fedff740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561ccd498760 0x561ccd4a2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:30.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.275+0000 7f01fedff740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ccd4a3020 0x561ccd4a5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:30.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.275+0000 7f01fedff740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x561ccd3fe3e0 con 0x561ccd498760 2024-09-10T10:04:30.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.275+0000 7f01fedff740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x561ccd3a5d80 con 0x561ccd497e40 2024-09-10T10:04:30.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.275+0000 7f01fedff740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x561ccd3fa3a0 con 0x561ccd4a3020 2024-09-10T10:04:30.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.275+0000 7f01f2000640 1 -- 172.21.15.77:0/566369547 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f01e8003a80 con 0x7f01f4103e30 2024-09-10T10:04:30.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.276+0000 7f01fa400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561ccd497e40 0x561ccd498220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:30.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.276+0000 7f01fa400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561ccd497e40 0x561ccd498220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:57726/0 (socket says 172.21.15.77:57726) 2024-09-10T10:04:30.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.276+0000 7f01fa400640 1 -- 172.21.15.77:0/730740663 learned_addr learned my addr 172.21.15.77:0/730740663 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:04:30.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.276+0000 7f01f9a00640 1 --2- 172.21.15.77:0/730740663 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561ccd498760 0x561ccd4a2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:30.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.276+0000 7f01f9000640 1 --2- 172.21.15.77:0/730740663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ccd4a3020 0x561ccd4a5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:30.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.276+0000 7f01fa400640 1 -- 172.21.15.77:0/730740663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ccd4a3020 msgr2=0x561ccd4a5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:30.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.276+0000 7f01fa400640 1 --2- 172.21.15.77:0/730740663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ccd4a3020 0x561ccd4a5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:30.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.276+0000 7f01fa400640 1 -- 172.21.15.77:0/730740663 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561ccd498760 msgr2=0x561ccd4a2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:30.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.276+0000 7f01fa400640 1 --2- 172.21.15.77:0/730740663 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561ccd498760 0x561ccd4a2ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:30.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.276+0000 7f01fa400640 1 -- 172.21.15.77:0/730740663 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ccd3c4ae0 con 0x561ccd497e40 2024-09-10T10:04:30.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.276+0000 7f01fa400640 1 --2- 172.21.15.77:0/730740663 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561ccd497e40 0x561ccd498220 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f01ec009f10 tx=0x7f01ec00f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d880dfc487e7c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:30.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.277+0000 7f01e6a00640 1 -- 172.21.15.77:0/730740663 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01ec003b70 con 0x561ccd497e40 2024-09-10T10:04:30.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.277+0000 7f01e6a00640 1 -- 172.21.15.77:0/730740663 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01ec003d10 con 0x561ccd497e40 2024-09-10T10:04:30.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.277+0000 7f01e6a00640 1 -- 172.21.15.77:0/730740663 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01ec011a10 con 0x561ccd497e40 2024-09-10T10:04:30.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.277+0000 7f01f2000640 1 --2- 172.21.15.77:0/566369547 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f01dc0a2b80 0x7f01dc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:30.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.277+0000 7f01f2000640 1 -- 172.21.15.77:0/566369547 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f01e80cf8b0 con 0x7f01f4103e30 2024-09-10T10:04:30.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.278+0000 7f01f9a00640 1 --2- 172.21.15.77:0/566369547 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f01dc0a2b80 0x7f01dc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:30.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.278+0000 7f01fedff740 1 -- 172.21.15.77:0/730740663 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561ccd497e40 msgr2=0x561ccd498220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:30.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.278+0000 7f01fedff740 1 --2- 172.21.15.77:0/730740663 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561ccd497e40 0x561ccd498220 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f01ec009f10 tx=0x7f01ec00f730 comp rx=0 tx=0).stop 2024-09-10T10:04:30.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.278+0000 7f01fedff740 1 -- 172.21.15.77:0/730740663 shutdown_connections 2024-09-10T10:04:30.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.278+0000 7f01fedff740 1 --2- 172.21.15.77:0/730740663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ccd4a3020 0x561ccd4a5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:30.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.278+0000 7f01fedff740 1 --2- 172.21.15.77:0/730740663 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561ccd498760 0x561ccd4a2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:30.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.278+0000 7f01fedff740 1 --2- 172.21.15.77:0/730740663 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561ccd497e40 0x561ccd498220 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:30.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.278+0000 7f01fedff740 1 -- 172.21.15.77:0/730740663 >> 172.21.15.77:0/730740663 conn(0x561ccd4943d0 msgr2=0x561ccd4947b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:30.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.278+0000 7f01f9a00640 1 --2- 172.21.15.77:0/566369547 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f01dc0a2b80 0x7f01dc0a5040 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f01e8098690 tx=0x7f01e800ef90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:30.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.278+0000 7f01fedff740 1 -- 172.21.15.77:0/730740663 shutdown_connections 2024-09-10T10:04:30.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.279+0000 7f01fedff740 1 -- 172.21.15.77:0/730740663 wait complete. 2024-09-10T10:04:30.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.279+0000 7f01fedff740 1 -- start start 2024-09-10T10:04:30.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.279+0000 7f01fedff740 1 Processor -- start 2024-09-10T10:04:30.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.280+0000 7f01fedff740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561ccd497e40 0x561ccd49fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:30.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.280+0000 7f01fedff740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561ccd498760 0x561ccd4a0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:30.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.280+0000 7f01fedff740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ccd4a3020 0x561ccd4a0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:30.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.280+0000 7f01fedff740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x561ccd4a5bf0 con 0x561ccd498760 2024-09-10T10:04:30.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.280+0000 7f01fedff740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x561ccd3b90f0 con 0x561ccd497e40 2024-09-10T10:04:30.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.280+0000 7f01fedff740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x561ccd4a5d70 con 0x561ccd4a3020 2024-09-10T10:04:30.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.280+0000 7f01fedff740 10 client.0 _ll_register_callbacks cb 0x561ccd3a0bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:04:30.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.280+0000 7f01fa400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561ccd497e40 0x561ccd49fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:30.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.280+0000 7f01f9000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561ccd498760 0x561ccd4a0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:30.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.280+0000 7f01fa400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561ccd497e40 0x561ccd49fef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:57736/0 (socket says 172.21.15.77:57736) 2024-09-10T10:04:30.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.280+0000 7f01fa400640 1 -- 172.21.15.77:0/1660106585 learned_addr learned my addr 172.21.15.77:0/1660106585 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:04:30.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.280+0000 7f01fa400640 1 --2- 172.21.15.77:0/1660106585 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ccd4a3020 0x561ccd4a0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:30.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.281+0000 7f01fa400640 1 -- 172.21.15.77:0/1660106585 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ccd4a3020 msgr2=0x561ccd4a0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:30.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.281+0000 7f01fa400640 1 --2- 172.21.15.77:0/1660106585 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ccd4a3020 0x561ccd4a0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:30.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.281+0000 7f01fa400640 1 -- 172.21.15.77:0/1660106585 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561ccd498760 msgr2=0x561ccd4a0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:30.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.281+0000 7f01fa400640 1 --2- 172.21.15.77:0/1660106585 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561ccd498760 0x561ccd4a0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:30.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.281+0000 7f01fa400640 1 -- 172.21.15.77:0/1660106585 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ccd16f3c0 con 0x561ccd497e40 2024-09-10T10:04:30.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.281+0000 7f01fa400640 1 --2- 172.21.15.77:0/1660106585 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561ccd497e40 0x561ccd49fef0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f01ec009910 tx=0x7f01ec0115f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2f19bae9c84b4ce server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:30.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.281+0000 7f01db400640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T10:04:30.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.281+0000 7f01db400640 1 -- 172.21.15.77:0/1660106585 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01ec024040 con 0x561ccd497e40 2024-09-10T10:04:30.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.281+0000 7f01fedff740 1 -- 172.21.15.77:0/1660106585 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01ec003d10 con 0x561ccd497e40 2024-09-10T10:04:30.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.281+0000 7f01db400640 1 -- 172.21.15.77:0/1660106585 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f01ec004c40 con 0x561ccd497e40 2024-09-10T10:04:30.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.281+0000 7f01db400640 1 -- 172.21.15.77:0/1660106585 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f01ec033050 con 0x561ccd497e40 2024-09-10T10:04:30.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.281+0000 7f01fedff740 10 client.27377 fetch_fsmap learned FSMap version 40 2024-09-10T10:04:30.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.281+0000 7f01fedff740 1 -- 172.21.15.77:0/1660106585 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x561ccd4b1c70 con 0x561ccd497e40 2024-09-10T10:04:30.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.282+0000 7f01db400640 1 -- 172.21.15.77:0/1660106585 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01ec003d10 con 0x561ccd497e40 2024-09-10T10:04:30.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.282+0000 7f01db400640 1 -- 172.21.15.77:0/1660106585 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 40) ==== 8398+0+0 (secure 0 0 0) 0x7f01ec0043e0 con 0x561ccd497e40 2024-09-10T10:04:30.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.282+0000 7f01fedff740 10 client.27377 fetch_fsmap finished waiting for FSMap version 40 2024-09-10T10:04:30.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.282+0000 7f01fedff740 10 client.27377 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T10:04:30.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.282+0000 7f01fedff740 20 client.27377 populate_metadata read hostname 'smithi077' 2024-09-10T10:04:30.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.282+0000 7f01fedff740 1 --2- 172.21.15.77:0/1660106585 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x561ccd4b3080 0x561ccd4b5470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:30.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.282+0000 7f01fedff740 4 client.27377 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:04:30.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.282+0000 7f01fedff740 1 -- 172.21.15.77:0/1660106585 --> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561ccd40b950 con 0x561ccd4b3080 2024-09-10T10:04:30.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.283+0000 7f01f9000640 1 --2- 172.21.15.77:0/1660106585 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x561ccd4b3080 0x561ccd4b5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:30.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.283+0000 7f01f9000640 1 --2- 172.21.15.77:0/1660106585 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x561ccd4b3080 0x561ccd4b5470 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f733a9446490cb01 server_cookie=56c8a444420fe8eb in_seq=0 out_seq=0 2024-09-10T10:04:30.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.283+0000 7f01db400640 10 client.27377 ms_handle_connect on v2:172.21.15.134:6836/2461497091 2024-09-10T10:04:30.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.284+0000 7f01db400640 1 -- 172.21.15.77:0/1660106585 <== mds.3 v2:172.21.15.134:6836/2461497091 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561ccd40b950 con 0x561ccd4b3080 2024-09-10T10:04:30.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.284+0000 7f01db400640 10 client.27377 handle_command_reply: tid=0 2024-09-10T10:04:30.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.284+0000 7f01fedff740 1 client.27377 shutdown 2024-09-10T10:04:30.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.284+0000 7f01fedff740 1 -- 172.21.15.77:0/1660106585 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561ccd497e40 msgr2=0x561ccd49fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:30.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.284+0000 7f01fedff740 1 --2- 172.21.15.77:0/1660106585 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561ccd497e40 0x561ccd49fef0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f01ec009910 tx=0x7f01ec0115f0 comp rx=0 tx=0).stop 2024-09-10T10:04:30.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.284+0000 7f01fedff740 1 -- 172.21.15.77:0/1660106585 shutdown_connections 2024-09-10T10:04:30.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.284+0000 7f01fedff740 1 --2- 172.21.15.77:0/1660106585 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ccd4a3020 0x561ccd4a0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:30.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.284+0000 7f01fedff740 1 --2- 172.21.15.77:0/1660106585 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x561ccd4b3080 0x561ccd4b5470 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:30.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.284+0000 7f01fedff740 1 --2- 172.21.15.77:0/1660106585 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561ccd498760 0x561ccd4a0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:30.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.284+0000 7f01fedff740 1 --2- 172.21.15.77:0/1660106585 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561ccd497e40 0x561ccd49fef0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:30.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.284+0000 7f01db400640 0 client.27377 ms_handle_reset on v2:172.21.15.134:6836/2461497091 2024-09-10T10:04:30.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.284+0000 7f01fedff740 1 -- 172.21.15.77:0/1660106585 >> 172.21.15.77:0/1660106585 conn(0x561ccd4943d0 msgr2=0x561ccd49a100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:30.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.284+0000 7f01fedff740 1 -- 172.21.15.77:0/1660106585 shutdown_connections 2024-09-10T10:04:30.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.284+0000 7f01fedff740 1 -- 172.21.15.77:0/1660106585 wait complete. 2024-09-10T10:04:30.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.285+0000 7f01fedff740 20 client.27377 trim_cache size 0 max 16384 2024-09-10T10:04:30.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.298+0000 7f01fedff740 1 Processor -- start 2024-09-10T10:04:30.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.298+0000 7f01fedff740 1 -- start start 2024-09-10T10:04:30.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.298+0000 7f01fedff740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561ccd497e40 0x561ccd4babb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:30.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.298+0000 7f01fedff740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ccd498760 0x561ccd4bb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:30.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.298+0000 7f01fedff740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561ccd4a3020 0x561ccd492c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:30.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.298+0000 7f01fedff740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f01f41b0fc0 con 0x561ccd497e40 2024-09-10T10:04:30.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.299+0000 7f01fedff740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f01f41b2bf0 con 0x561ccd4a3020 2024-09-10T10:04:30.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.299+0000 7f01fedff740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f01f41b16f0 con 0x561ccd498760 2024-09-10T10:04:30.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.299+0000 7f01fa400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561ccd497e40 0x561ccd4babb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:30.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.299+0000 7f01fa400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561ccd497e40 0x561ccd4babb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:39462/0 (socket says 172.21.15.77:39462) 2024-09-10T10:04:30.311 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.299+0000 7f01fa400640 1 -- 172.21.15.77:0/2380897777 learned_addr learned my addr 172.21.15.77:0/2380897777 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:04:30.311 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.299+0000 7f01f9000640 1 --2- 172.21.15.77:0/2380897777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ccd498760 0x561ccd4bb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:30.311 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.299+0000 7f01fa400640 1 --2- 172.21.15.77:0/2380897777 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561ccd4a3020 0x561ccd492c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:30.311 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.299+0000 7f01fa400640 1 -- 172.21.15.77:0/2380897777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ccd498760 msgr2=0x561ccd4bb0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:30.311 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.299+0000 7f01fa400640 1 --2- 172.21.15.77:0/2380897777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ccd498760 0x561ccd4bb0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:30.311 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.299+0000 7f01fa400640 1 -- 172.21.15.77:0/2380897777 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561ccd4a3020 msgr2=0x561ccd492c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:30.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.299+0000 7f01fa400640 1 --2- 172.21.15.77:0/2380897777 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561ccd4a3020 0x561ccd492c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:30.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.299+0000 7f01fa400640 1 -- 172.21.15.77:0/2380897777 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ccd494200 con 0x561ccd497e40 2024-09-10T10:04:30.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.300+0000 7f01fa400640 1 --2- 172.21.15.77:0/2380897777 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561ccd497e40 0x561ccd4babb0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f01ec009730 tx=0x7f01ec005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f87f4da17989e637 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:30.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.300+0000 7f01e6000640 1 -- 172.21.15.77:0/2380897777 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01ec0099c0 con 0x561ccd497e40 2024-09-10T10:04:30.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.300+0000 7f01e6000640 1 -- 172.21.15.77:0/2380897777 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01ec009b60 con 0x561ccd497e40 2024-09-10T10:04:30.313 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.300+0000 7f01e6000640 1 -- 172.21.15.77:0/2380897777 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01ec004c20 con 0x561ccd497e40 2024-09-10T10:04:30.313 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.300+0000 7f01fedff740 1 -- 172.21.15.77:0/2380897777 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561ccd497e40 msgr2=0x561ccd4babb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:30.313 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.300+0000 7f01fedff740 1 --2- 172.21.15.77:0/2380897777 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561ccd497e40 0x561ccd4babb0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f01ec009730 tx=0x7f01ec005bd0 comp rx=0 tx=0).stop 2024-09-10T10:04:30.313 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.301+0000 7f01fedff740 1 -- 172.21.15.77:0/2380897777 shutdown_connections 2024-09-10T10:04:30.313 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.301+0000 7f01fedff740 1 --2- 172.21.15.77:0/2380897777 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561ccd4a3020 0x561ccd492c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:30.313 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.301+0000 7f01fedff740 1 --2- 172.21.15.77:0/2380897777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ccd498760 0x561ccd4bb0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:30.314 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.301+0000 7f01fedff740 1 --2- 172.21.15.77:0/2380897777 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561ccd497e40 0x561ccd4babb0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:30.314 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.301+0000 7f01fedff740 1 -- 172.21.15.77:0/2380897777 >> 172.21.15.77:0/2380897777 conn(0x561ccd4943d0 msgr2=0x561ccd49a0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:30.314 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.301+0000 7f01fedff740 1 -- 172.21.15.77:0/2380897777 shutdown_connections 2024-09-10T10:04:30.314 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.301+0000 7f01fedff740 1 -- 172.21.15.77:0/2380897777 wait complete. 2024-09-10T10:04:30.314 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.301+0000 7f01fedff740 1 -- start start 2024-09-10T10:04:30.314 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.301+0000 7f01fedff740 1 Processor -- start 2024-09-10T10:04:30.315 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.302+0000 7f01fedff740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561ccd497e40 0x561ccd4ab3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:30.315 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.302+0000 7f01fedff740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561ccd498760 0x561ccd4a6720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:30.315 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.302+0000 7f01fedff740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ccd4a3020 0x561ccd4a6c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:30.315 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.302+0000 7f01fedff740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f01f41b2520 con 0x561ccd498760 2024-09-10T10:04:30.315 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.302+0000 7f01fedff740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f01f41b1df0 con 0x561ccd497e40 2024-09-10T10:04:30.315 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.302+0000 7f01fedff740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f01f41aefa0 con 0x561ccd4a3020 2024-09-10T10:04:30.315 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.302+0000 7f01fedff740 10 client.0 _ll_register_callbacks cb 0x7f01ec015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:04:30.315 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.302+0000 7f01f9000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561ccd498760 0x561ccd4a6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:30.316 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.302+0000 7f01f9000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561ccd498760 0x561ccd4a6720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:39476/0 (socket says 172.21.15.77:39476) 2024-09-10T10:04:30.316 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.302+0000 7f01f9000640 1 -- 172.21.15.77:0/1319169393 learned_addr learned my addr 172.21.15.77:0/1319169393 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:04:30.316 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.302+0000 7f01fa400640 1 --2- 172.21.15.77:0/1319169393 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561ccd497e40 0x561ccd4ab3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:30.316 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.302+0000 7f01fa400640 1 --2- 172.21.15.77:0/1319169393 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ccd4a3020 0x561ccd4a6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:30.316 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.303+0000 7f01f9000640 1 -- 172.21.15.77:0/1319169393 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ccd4a3020 msgr2=0x561ccd4a6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:30.316 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.303+0000 7f01f9000640 1 --2- 172.21.15.77:0/1319169393 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ccd4a3020 0x561ccd4a6c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:30.316 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.303+0000 7f01f9000640 1 -- 172.21.15.77:0/1319169393 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561ccd497e40 msgr2=0x561ccd4ab3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:30.316 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.303+0000 7f01f9000640 1 --2- 172.21.15.77:0/1319169393 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561ccd497e40 0x561ccd4ab3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:30.316 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.303+0000 7f01f9000640 1 -- 172.21.15.77:0/1319169393 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ccd495060 con 0x561ccd498760 2024-09-10T10:04:30.316 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.303+0000 7f01f9000640 1 --2- 172.21.15.77:0/1319169393 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561ccd498760 0x561ccd4a6720 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f01e0031550 tx=0x7f01e0004350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23c44175c43ab473 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:30.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.303+0000 7f01db400640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-10T10:04:30.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.303+0000 7f01db400640 1 -- 172.21.15.77:0/1319169393 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e0049070 con 0x561ccd498760 2024-09-10T10:04:30.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.303+0000 7f01db400640 1 -- 172.21.15.77:0/1319169393 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f01e0005450 con 0x561ccd498760 2024-09-10T10:04:30.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.303+0000 7f01fedff740 1 -- 172.21.15.77:0/1319169393 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01ec009b60 con 0x561ccd498760 2024-09-10T10:04:30.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.304+0000 7f01db400640 1 -- 172.21.15.77:0/1319169393 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f01e0055050 con 0x561ccd498760 2024-09-10T10:04:30.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.304+0000 7f01fedff740 10 client.17217 fetch_fsmap learned FSMap version 40 2024-09-10T10:04:30.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.304+0000 7f01fedff740 1 -- 172.21.15.77:0/1319169393 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x561ccd51d730 con 0x561ccd498760 2024-09-10T10:04:30.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.304+0000 7f01db400640 1 -- 172.21.15.77:0/1319169393 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01ec009b60 con 0x561ccd498760 2024-09-10T10:04:30.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.304+0000 7f01db400640 1 -- 172.21.15.77:0/1319169393 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 40) ==== 8398+0+0 (secure 0 0 0) 0x7f01e00317f0 con 0x561ccd498760 2024-09-10T10:04:30.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.304+0000 7f01fedff740 10 client.17217 fetch_fsmap finished waiting for FSMap version 40 2024-09-10T10:04:30.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.304+0000 7f01fedff740 10 client.17217 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T10:04:30.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.304+0000 7f01fedff740 20 client.17217 populate_metadata read hostname 'smithi077' 2024-09-10T10:04:30.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.304+0000 7f01fedff740 1 --2- 172.21.15.77:0/1319169393 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x561ccd51dec0 0x561ccd5202e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:30.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.304+0000 7f01fedff740 4 client.17217 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:04:30.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.304+0000 7f01fedff740 1 -- 172.21.15.77:0/1319169393 --> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561ccd405e60 con 0x561ccd51dec0 2024-09-10T10:04:30.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.305+0000 7f01fa400640 1 --2- 172.21.15.77:0/1319169393 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x561ccd51dec0 0x561ccd5202e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:30.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.305+0000 7f01fa400640 1 --2- 172.21.15.77:0/1319169393 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x561ccd51dec0 0x561ccd5202e0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d7e80822a53fb113 server_cookie=fe6b20fbdcdc53f3 in_seq=0 out_seq=0 2024-09-10T10:04:30.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.305+0000 7f01db400640 10 client.17217 ms_handle_connect on v2:172.21.15.134:6836/2461497091 2024-09-10T10:04:30.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.306+0000 7f01db400640 1 -- 172.21.15.77:0/1319169393 <== mds.3 v2:172.21.15.134:6836/2461497091 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561ccd405e60 con 0x561ccd51dec0 2024-09-10T10:04:30.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.306+0000 7f01db400640 10 client.17217 handle_command_reply: tid=0 2024-09-10T10:04:30.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.306+0000 7f01fedff740 1 client.17217 shutdown 2024-09-10T10:04:30.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.306+0000 7f01fedff740 1 -- 172.21.15.77:0/1319169393 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561ccd498760 msgr2=0x561ccd4a6720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:30.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.306+0000 7f01fedff740 1 --2- 172.21.15.77:0/1319169393 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561ccd498760 0x561ccd4a6720 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f01e0031550 tx=0x7f01e0004350 comp rx=0 tx=0).stop 2024-09-10T10:04:30.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.306+0000 7f01fedff740 1 -- 172.21.15.77:0/1319169393 shutdown_connections 2024-09-10T10:04:30.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.306+0000 7f01fedff740 1 --2- 172.21.15.77:0/1319169393 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ccd4a3020 0x561ccd4a6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:30.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.306+0000 7f01fedff740 1 --2- 172.21.15.77:0/1319169393 >> [v2:172.21.15.134:6836/2461497091,v1:172.21.15.134:6837/2461497091] conn(0x561ccd51dec0 0x561ccd5202e0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:30.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.306+0000 7f01fedff740 1 --2- 172.21.15.77:0/1319169393 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561ccd498760 0x561ccd4a6720 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:30.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.306+0000 7f01fedff740 1 --2- 172.21.15.77:0/1319169393 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561ccd497e40 0x561ccd4ab3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:30.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.306+0000 7f01fedff740 1 -- 172.21.15.77:0/1319169393 >> 172.21.15.77:0/1319169393 conn(0x561ccd4943d0 msgr2=0x561ccd49e690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:30.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.306+0000 7f01db400640 0 client.17217 ms_handle_reset on v2:172.21.15.134:6836/2461497091 2024-09-10T10:04:30.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.306+0000 7f01fedff740 1 -- 172.21.15.77:0/1319169393 shutdown_connections 2024-09-10T10:04:30.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.306+0000 7f01fedff740 1 -- 172.21.15.77:0/1319169393 wait complete. 2024-09-10T10:04:30.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.306+0000 7f01fedff740 20 client.17217 trim_cache size 0 max 16384 2024-09-10T10:04:30.319 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-10T10:04:30.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.307+0000 7f01fae00640 1 -- 172.21.15.77:0/566369547 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f01dc0a2b80 msgr2=0x7f01dc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:30.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.307+0000 7f01fae00640 1 --2- 172.21.15.77:0/566369547 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f01dc0a2b80 0x7f01dc0a5040 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f01e8098690 tx=0x7f01e800ef90 comp rx=0 tx=0).stop 2024-09-10T10:04:30.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.307+0000 7f01fae00640 1 -- 172.21.15.77:0/566369547 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f01f4103e30 msgr2=0x7f01f419a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:30.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.307+0000 7f01fae00640 1 --2- 172.21.15.77:0/566369547 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f01f4103e30 0x7f01f419a490 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f01e800ea60 tx=0x7f01e800ef20 comp rx=0 tx=0).stop 2024-09-10T10:04:30.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.307+0000 7f01fae00640 1 -- 172.21.15.77:0/566369547 shutdown_connections 2024-09-10T10:04:30.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.307+0000 7f01fae00640 1 --2- 172.21.15.77:0/566369547 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f01dc0a2b80 0x7f01dc0a5040 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:30.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.307+0000 7f01fae00640 1 --2- 172.21.15.77:0/566369547 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01f4108f60 0x7f01f419ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:30.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.307+0000 7f01fae00640 1 --2- 172.21.15.77:0/566369547 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f01f41047e0 0x7f01f419a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:30.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.307+0000 7f01fae00640 1 --2- 172.21.15.77:0/566369547 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f01f4103e30 0x7f01f419a490 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:30.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.307+0000 7f01fae00640 1 -- 172.21.15.77:0/566369547 >> 172.21.15.77:0/566369547 conn(0x7f01f40ff910 msgr2=0x7f01f4100c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:30.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.307+0000 7f01fae00640 1 -- 172.21.15.77:0/566369547 shutdown_connections 2024-09-10T10:04:30.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:30.307+0000 7f01fae00640 1 -- 172.21.15.77:0/566369547 wait complete. 2024-09-10T10:04:31.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:04:30 smithi077 ceph-mon[30319]: pgmap v671: 129 pgs: 129 active+clean; 120 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 407 op/s 2024-09-10T10:04:31.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:04:30 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/2904391250' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:04:31.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:04:30 smithi134 ceph-mon[35674]: pgmap v671: 129 pgs: 129 active+clean; 120 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 407 op/s 2024-09-10T10:04:31.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:04:30 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/2904391250' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:04:31.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:04:30 smithi120 ceph-mon[34770]: pgmap v671: 129 pgs: 129 active+clean; 120 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 407 op/s 2024-09-10T10:04:31.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:04:30 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/2904391250' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:04:31.785 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T10:04:32.623 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24563 quiesce db --cancel --all' 2024-09-10T10:04:32.623 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24563 quiesce db --cancel --all 2024-09-10T10:04:32.702 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:04:33.178 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:04:32 smithi077 ceph-mon[30319]: pgmap v672: 129 pgs: 129 active+clean; 120 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 806 KiB/s rd, 748 KiB/s wr, 305 op/s 2024-09-10T10:04:33.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.288+0000 7f48fec00640 1 -- 172.21.15.77:0/1518796639 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f48f81049c0 msgr2=0x7f48f8108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:33.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.288+0000 7f48fec00640 1 --2- 172.21.15.77:0/1518796639 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f48f81049c0 0x7f48f8108a10 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f48e4009860 tx=0x7f48e402a340 comp rx=0 tx=0).stop 2024-09-10T10:04:33.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.288+0000 7f48fec00640 1 -- 172.21.15.77:0/1518796639 shutdown_connections 2024-09-10T10:04:33.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.288+0000 7f48fec00640 1 --2- 172.21.15.77:0/1518796639 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f48f8109140 0x7f48f8110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:33.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.288+0000 7f48fec00640 1 --2- 172.21.15.77:0/1518796639 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f48f81049c0 0x7f48f8108a10 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:33.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.288+0000 7f48fec00640 1 --2- 172.21.15.77:0/1518796639 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48f8104010 0x7f48f81043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:33.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.288+0000 7f48fec00640 1 -- 172.21.15.77:0/1518796639 >> 172.21.15.77:0/1518796639 conn(0x7f48f80ffa50 msgr2=0x7f48f8101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:33.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.288+0000 7f48fec00640 1 -- 172.21.15.77:0/1518796639 shutdown_connections 2024-09-10T10:04:33.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.288+0000 7f48fec00640 1 -- 172.21.15.77:0/1518796639 wait complete. 2024-09-10T10:04:33.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.290+0000 7f48fec00640 1 Processor -- start 2024-09-10T10:04:33.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.290+0000 7f48fec00640 1 -- start start 2024-09-10T10:04:33.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.290+0000 7f48fec00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48f8104010 0x7f48f819a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:33.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.290+0000 7f48fec00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f48f81049c0 0x7f48f819abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:33.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.290+0000 7f48fec00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f48f8109140 0x7f48f819ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:33.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.290+0000 7f48fec00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f48f8114160 con 0x7f48f81049c0 2024-09-10T10:04:33.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.290+0000 7f48fec00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f48f8113fe0 con 0x7f48f8109140 2024-09-10T10:04:33.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.290+0000 7f48fec00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f48f81142e0 con 0x7f48f8104010 2024-09-10T10:04:33.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.290+0000 7f48f7e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48f8104010 0x7f48f819a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:33.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.290+0000 7f48f7400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f48f81049c0 0x7f48f819abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:33.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.290+0000 7f48fcc00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f48f8109140 0x7f48f819ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:33.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.290+0000 7f48f7e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48f8104010 0x7f48f819a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:54296/0 (socket says 172.21.15.77:54296) 2024-09-10T10:04:33.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.290+0000 7f48f7400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f48f81049c0 0x7f48f819abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60578/0 (socket says 172.21.15.77:60578) 2024-09-10T10:04:33.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.290+0000 7f48f7e00640 1 -- 172.21.15.77:0/963511546 learned_addr learned my addr 172.21.15.77:0/963511546 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:04:33.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.291+0000 7f48fcc00640 1 -- 172.21.15.77:0/963511546 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48f8104010 msgr2=0x7f48f819a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:33.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.291+0000 7f48fcc00640 1 --2- 172.21.15.77:0/963511546 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48f8104010 0x7f48f819a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:33.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.291+0000 7f48fcc00640 1 -- 172.21.15.77:0/963511546 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f48f81049c0 msgr2=0x7f48f819abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:33.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.291+0000 7f48fcc00640 1 --2- 172.21.15.77:0/963511546 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f48f81049c0 0x7f48f819abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:33.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.291+0000 7f48fcc00640 1 -- 172.21.15.77:0/963511546 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48f819f640 con 0x7f48f8109140 2024-09-10T10:04:33.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.291+0000 7f48f7400640 1 --2- 172.21.15.77:0/963511546 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f48f81049c0 0x7f48f819abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:04:33.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.291+0000 7f48fcc00640 1 --2- 172.21.15.77:0/963511546 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f48f8109140 0x7f48f819ef60 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f48f000efc0 tx=0x7f48f000c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:33.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.292+0000 7f48f4c00640 1 -- 172.21.15.77:0/963511546 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f0003970 con 0x7f48f8109140 2024-09-10T10:04:33.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.292+0000 7f48fec00640 1 -- 172.21.15.77:0/963511546 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48f819f930 con 0x7f48f8109140 2024-09-10T10:04:33.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.292+0000 7f48f4c00640 1 -- 172.21.15.77:0/963511546 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48f0004020 con 0x7f48f8109140 2024-09-10T10:04:33.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.292+0000 7f48f4c00640 1 -- 172.21.15.77:0/963511546 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f000b9f0 con 0x7f48f8109140 2024-09-10T10:04:33.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.292+0000 7f48fec00640 1 -- 172.21.15.77:0/963511546 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48f81a7210 con 0x7f48f8109140 2024-09-10T10:04:33.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.295+0000 7f490164b740 1 Processor -- start 2024-09-10T10:04:33.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.296+0000 7f48f4c00640 1 -- 172.21.15.77:0/963511546 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f48f001d020 con 0x7f48f8109140 2024-09-10T10:04:33.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.297+0000 7f490164b740 1 -- start start 2024-09-10T10:04:33.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.297+0000 7f490164b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cb388aec70 0x55cb388af050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:33.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.297+0000 7f490164b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb388af590 0x55cb388b9910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:33.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.297+0000 7f490164b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cb388b9e50 0x55cb388bc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:33.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.297+0000 7f490164b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55cb388153e0 con 0x55cb388b9e50 2024-09-10T10:04:33.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.297+0000 7f490164b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55cb387bcd80 con 0x55cb388aec70 2024-09-10T10:04:33.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.297+0000 7f490164b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55cb388113a0 con 0x55cb388af590 2024-09-10T10:04:33.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.298+0000 7f48f7400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cb388aec70 0x55cb388af050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:33.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.298+0000 7f48f7e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb388af590 0x55cb388b9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:33.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.298+0000 7f48f7400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cb388aec70 0x55cb388af050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:36740/0 (socket says 172.21.15.77:36740) 2024-09-10T10:04:33.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.298+0000 7f48f7400640 1 -- 172.21.15.77:0/1844737280 learned_addr learned my addr 172.21.15.77:0/1844737280 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:04:33.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.298+0000 7f48f7400640 1 --2- 172.21.15.77:0/1844737280 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cb388b9e50 0x55cb388bc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:33.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.298+0000 7f48f4c00640 1 --2- 172.21.15.77:0/963511546 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f48dc0a2b80 0x7f48dc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:33.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.298+0000 7f48f7400640 1 -- 172.21.15.77:0/1844737280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb388af590 msgr2=0x55cb388b9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:33.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.298+0000 7f48f7400640 1 --2- 172.21.15.77:0/1844737280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb388af590 0x55cb388b9910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:33.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.298+0000 7f48f7400640 1 -- 172.21.15.77:0/1844737280 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cb388b9e50 msgr2=0x55cb388bc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:33.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.298+0000 7f48f7400640 1 --2- 172.21.15.77:0/1844737280 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cb388b9e50 0x55cb388bc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:33.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.298+0000 7f48f7400640 1 -- 172.21.15.77:0/1844737280 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb387dbae0 con 0x55cb388aec70 2024-09-10T10:04:33.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.298+0000 7f48f4c00640 1 -- 172.21.15.77:0/963511546 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f48f00cf240 con 0x7f48f8109140 2024-09-10T10:04:33.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.298+0000 7f48f7400640 1 --2- 172.21.15.77:0/1844737280 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cb388aec70 0x55cb388af050 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f48e4005e00 tx=0x7f48e4005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0bcc8ddf15f10c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:33.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.299+0000 7f48fcc00640 1 --2- 172.21.15.77:0/963511546 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f48dc0a2b80 0x7f48dc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:33.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.299+0000 7f48ed200640 1 -- 172.21.15.77:0/1844737280 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e4047070 con 0x55cb388aec70 2024-09-10T10:04:33.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.299+0000 7f48ed200640 1 -- 172.21.15.77:0/1844737280 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48e4030750 con 0x55cb388aec70 2024-09-10T10:04:33.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.299+0000 7f48ed200640 1 -- 172.21.15.77:0/1844737280 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e4037040 con 0x55cb388aec70 2024-09-10T10:04:33.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.300+0000 7f48fcc00640 1 --2- 172.21.15.77:0/963511546 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f48dc0a2b80 0x7f48dc0a5040 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f48f000cd00 tx=0x7f48f000ceb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:33.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.300+0000 7f490164b740 1 -- 172.21.15.77:0/1844737280 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cb388aec70 msgr2=0x55cb388af050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:33.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.300+0000 7f490164b740 1 --2- 172.21.15.77:0/1844737280 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cb388aec70 0x55cb388af050 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f48e4005e00 tx=0x7f48e4005f90 comp rx=0 tx=0).stop 2024-09-10T10:04:33.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.300+0000 7f490164b740 1 -- 172.21.15.77:0/1844737280 shutdown_connections 2024-09-10T10:04:33.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.300+0000 7f490164b740 1 --2- 172.21.15.77:0/1844737280 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cb388b9e50 0x55cb388bc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:33.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.300+0000 7f490164b740 1 --2- 172.21.15.77:0/1844737280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb388af590 0x55cb388b9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:33.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.300+0000 7f490164b740 1 --2- 172.21.15.77:0/1844737280 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cb388aec70 0x55cb388af050 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:33.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.300+0000 7f490164b740 1 -- 172.21.15.77:0/1844737280 >> 172.21.15.77:0/1844737280 conn(0x55cb388ab200 msgr2=0x55cb388ab5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:33.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.300+0000 7f490164b740 1 -- 172.21.15.77:0/1844737280 shutdown_connections 2024-09-10T10:04:33.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.300+0000 7f490164b740 1 -- 172.21.15.77:0/1844737280 wait complete. 2024-09-10T10:04:33.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.301+0000 7f490164b740 1 -- start start 2024-09-10T10:04:33.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.301+0000 7f490164b740 1 Processor -- start 2024-09-10T10:04:33.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.302+0000 7f490164b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cb388aec70 0x55cb388b6e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:33.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.302+0000 7f490164b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb388af590 0x55cb388b7360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:33.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.302+0000 7f490164b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cb388b9e50 0x55cb388b78a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:33.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.302+0000 7f490164b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55cb388bca20 con 0x55cb388aec70 2024-09-10T10:04:33.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.302+0000 7f490164b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55cb387d00f0 con 0x55cb388b9e50 2024-09-10T10:04:33.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.302+0000 7f490164b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55cb388bcba0 con 0x55cb388af590 2024-09-10T10:04:33.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.302+0000 7f490164b740 10 client.0 _ll_register_callbacks cb 0x55cb387b7bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:04:33.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.302+0000 7f48f7400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cb388aec70 0x55cb388b6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:33.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.302+0000 7f48f7400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cb388aec70 0x55cb388b6e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60602/0 (socket says 172.21.15.77:60602) 2024-09-10T10:04:33.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.302+0000 7f48f7400640 1 -- 172.21.15.77:0/3905279719 learned_addr learned my addr 172.21.15.77:0/3905279719 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:04:33.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.302+0000 7f48f7e00640 1 --2- 172.21.15.77:0/3905279719 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb388af590 0x55cb388b7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:33.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.302+0000 7f48f7400640 1 --2- 172.21.15.77:0/3905279719 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cb388b9e50 0x55cb388b78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:33.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.303+0000 7f48f7400640 1 -- 172.21.15.77:0/3905279719 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb388af590 msgr2=0x55cb388b7360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:33.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.303+0000 7f48f7400640 1 --2- 172.21.15.77:0/3905279719 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb388af590 0x55cb388b7360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:33.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.303+0000 7f48f7400640 1 -- 172.21.15.77:0/3905279719 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cb388b9e50 msgr2=0x55cb388b78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:33.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.303+0000 7f48f7400640 1 --2- 172.21.15.77:0/3905279719 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cb388b9e50 0x55cb388b78a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:33.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.303+0000 7f48f7400640 1 -- 172.21.15.77:0/3905279719 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb385863c0 con 0x55cb388aec70 2024-09-10T10:04:33.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.303+0000 7f48f7e00640 1 --2- 172.21.15.77:0/3905279719 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb388af590 0x55cb388b7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:04:33.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.303+0000 7f48f7400640 1 --2- 172.21.15.77:0/3905279719 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cb388aec70 0x55cb388b6e20 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f48e4045040 tx=0x7f48e40056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98eaada44ab3c78 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:33.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.303+0000 7f48e1600640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-10T10:04:33.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.303+0000 7f48e1600640 1 -- 172.21.15.77:0/3905279719 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e4037070 con 0x55cb388aec70 2024-09-10T10:04:33.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.303+0000 7f48e1600640 1 -- 172.21.15.77:0/3905279719 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f48e4003d50 con 0x55cb388aec70 2024-09-10T10:04:33.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.303+0000 7f490164b740 1 -- 172.21.15.77:0/3905279719 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48e4030750 con 0x55cb388aec70 2024-09-10T10:04:33.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.304+0000 7f48e1600640 1 -- 172.21.15.77:0/3905279719 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f48e4047040 con 0x55cb388aec70 2024-09-10T10:04:33.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.304+0000 7f490164b740 10 client.17229 fetch_fsmap learned FSMap version 40 2024-09-10T10:04:33.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.304+0000 7f490164b740 1 -- 172.21.15.77:0/3905279719 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55cb388c8b30 con 0x55cb388aec70 2024-09-10T10:04:33.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.304+0000 7f48e1600640 1 -- 172.21.15.77:0/3905279719 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48e4030750 con 0x55cb388aec70 2024-09-10T10:04:33.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.304+0000 7f48e1600640 1 -- 172.21.15.77:0/3905279719 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 40) ==== 8398+0+0 (secure 0 0 0) 0x7f48e404d9e0 con 0x55cb388aec70 2024-09-10T10:04:33.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.304+0000 7f490164b740 10 client.17229 fetch_fsmap finished waiting for FSMap version 40 2024-09-10T10:04:33.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.304+0000 7f490164b740 10 client.17229 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T10:04:33.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.304+0000 7f490164b740 20 client.17229 populate_metadata read hostname 'smithi077' 2024-09-10T10:04:33.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.304+0000 7f490164b740 1 --2- 172.21.15.77:0/3905279719 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x55cb388c9e80 0x55cb388cc270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:33.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.304+0000 7f490164b740 4 client.17229 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:04:33.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.304+0000 7f490164b740 1 -- 172.21.15.77:0/3905279719 --> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cb38822950 con 0x55cb388c9e80 2024-09-10T10:04:33.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.305+0000 7f48f7e00640 1 --2- 172.21.15.77:0/3905279719 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x55cb388c9e80 0x55cb388cc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:33.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.306+0000 7f48f7e00640 1 --2- 172.21.15.77:0/3905279719 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x55cb388c9e80 0x55cb388cc270 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b057ed07f7ba6ba server_cookie=4ef91c6f4d909e56 in_seq=0 out_seq=0 2024-09-10T10:04:33.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.306+0000 7f48e1600640 10 client.17229 ms_handle_connect on v2:172.21.15.134:6838/1965117680 2024-09-10T10:04:33.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.306+0000 7f48e1600640 1 -- 172.21.15.77:0/3905279719 <== mds.2 v2:172.21.15.134:6838/1965117680 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cb38822950 con 0x55cb388c9e80 2024-09-10T10:04:33.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.306+0000 7f48e1600640 10 client.17229 handle_command_reply: tid=0 2024-09-10T10:04:33.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.306+0000 7f490164b740 1 client.17229 shutdown 2024-09-10T10:04:33.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.307+0000 7f490164b740 1 -- 172.21.15.77:0/3905279719 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cb388aec70 msgr2=0x55cb388b6e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:33.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.307+0000 7f490164b740 1 --2- 172.21.15.77:0/3905279719 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cb388aec70 0x55cb388b6e20 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f48e4045040 tx=0x7f48e40056d0 comp rx=0 tx=0).stop 2024-09-10T10:04:33.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.307+0000 7f490164b740 1 -- 172.21.15.77:0/3905279719 shutdown_connections 2024-09-10T10:04:33.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.307+0000 7f490164b740 1 --2- 172.21.15.77:0/3905279719 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x55cb388c9e80 0x55cb388cc270 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:33.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.307+0000 7f490164b740 1 --2- 172.21.15.77:0/3905279719 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cb388b9e50 0x55cb388b78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:33.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.307+0000 7f490164b740 1 --2- 172.21.15.77:0/3905279719 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb388af590 0x55cb388b7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:33.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.307+0000 7f490164b740 1 --2- 172.21.15.77:0/3905279719 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cb388aec70 0x55cb388b6e20 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:33.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.307+0000 7f490164b740 1 -- 172.21.15.77:0/3905279719 >> 172.21.15.77:0/3905279719 conn(0x55cb388ab200 msgr2=0x55cb388b0ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:33.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.307+0000 7f48e1600640 0 client.17229 ms_handle_reset on v2:172.21.15.134:6838/1965117680 2024-09-10T10:04:33.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.307+0000 7f490164b740 1 -- 172.21.15.77:0/3905279719 shutdown_connections 2024-09-10T10:04:33.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.307+0000 7f490164b740 1 -- 172.21.15.77:0/3905279719 wait complete. 2024-09-10T10:04:33.311 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.307+0000 7f490164b740 20 client.17229 trim_cache size 0 max 16384 2024-09-10T10:04:33.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.322+0000 7f490164b740 1 Processor -- start 2024-09-10T10:04:33.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.322+0000 7f490164b740 1 -- start start 2024-09-10T10:04:33.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.322+0000 7f490164b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cb388aec70 0x55cb388cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:33.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.322+0000 7f490164b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cb388af590 0x55cb388cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:33.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.322+0000 7f490164b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb388b9e50 0x55cb388b40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:33.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.322+0000 7f490164b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f48f81b1060 con 0x55cb388aec70 2024-09-10T10:04:33.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.322+0000 7f490164b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f48f81b2c90 con 0x55cb388af590 2024-09-10T10:04:33.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.322+0000 7f490164b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f48f81b1790 con 0x55cb388b9e50 2024-09-10T10:04:33.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.323+0000 7f48f7400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cb388aec70 0x55cb388cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:33.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.323+0000 7f48f7400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cb388aec70 0x55cb388cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60614/0 (socket says 172.21.15.77:60614) 2024-09-10T10:04:33.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.323+0000 7f48f7400640 1 -- 172.21.15.77:0/2677995160 learned_addr learned my addr 172.21.15.77:0/2677995160 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:04:33.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.323+0000 7f48f7e00640 1 --2- 172.21.15.77:0/2677995160 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cb388af590 0x55cb388cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:33.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.323+0000 7f48f7400640 1 --2- 172.21.15.77:0/2677995160 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb388b9e50 0x55cb388b40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:33.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.323+0000 7f48f7e00640 1 -- 172.21.15.77:0/2677995160 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb388b9e50 msgr2=0x55cb388b40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:33.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.323+0000 7f48f7e00640 1 --2- 172.21.15.77:0/2677995160 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb388b9e50 0x55cb388b40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:33.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.323+0000 7f48f7e00640 1 -- 172.21.15.77:0/2677995160 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cb388aec70 msgr2=0x55cb388cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:33.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.323+0000 7f48f7e00640 1 --2- 172.21.15.77:0/2677995160 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cb388aec70 0x55cb388cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:33.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.323+0000 7f48f7e00640 1 -- 172.21.15.77:0/2677995160 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb388a6770 con 0x55cb388af590 2024-09-10T10:04:33.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.323+0000 7f48f7400640 1 --2- 172.21.15.77:0/2677995160 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cb388aec70 0x55cb388cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:04:33.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.324+0000 7f48f7e00640 1 --2- 172.21.15.77:0/2677995160 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cb388af590 0x55cb388cf7d0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x55cb388bcc20 tx=0x7f48e8005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6198d81b6cc5f528 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:33.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.324+0000 7f48e3e00640 1 -- 172.21.15.77:0/2677995160 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e8010b60 con 0x55cb388af590 2024-09-10T10:04:33.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.324+0000 7f48e3e00640 1 -- 172.21.15.77:0/2677995160 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48e8005500 con 0x55cb388af590 2024-09-10T10:04:33.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.324+0000 7f48e3e00640 1 -- 172.21.15.77:0/2677995160 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e8004e00 con 0x55cb388af590 2024-09-10T10:04:33.328 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.325+0000 7f490164b740 1 -- 172.21.15.77:0/2677995160 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cb388af590 msgr2=0x55cb388cf7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:33.329 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.325+0000 7f490164b740 1 --2- 172.21.15.77:0/2677995160 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cb388af590 0x55cb388cf7d0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x55cb388bcc20 tx=0x7f48e8005f90 comp rx=0 tx=0).stop 2024-09-10T10:04:33.329 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.325+0000 7f490164b740 1 -- 172.21.15.77:0/2677995160 shutdown_connections 2024-09-10T10:04:33.329 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.325+0000 7f490164b740 1 --2- 172.21.15.77:0/2677995160 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb388b9e50 0x55cb388b40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:33.329 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.325+0000 7f490164b740 1 --2- 172.21.15.77:0/2677995160 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cb388af590 0x55cb388cf7d0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:33.329 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.325+0000 7f490164b740 1 --2- 172.21.15.77:0/2677995160 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cb388aec70 0x55cb388cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:33.329 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.325+0000 7f490164b740 1 -- 172.21.15.77:0/2677995160 >> 172.21.15.77:0/2677995160 conn(0x55cb388ab200 msgr2=0x55cb388b8ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:33.329 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.325+0000 7f490164b740 1 -- 172.21.15.77:0/2677995160 shutdown_connections 2024-09-10T10:04:33.329 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.325+0000 7f490164b740 1 -- 172.21.15.77:0/2677995160 wait complete. 2024-09-10T10:04:33.329 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.325+0000 7f490164b740 1 -- start start 2024-09-10T10:04:33.329 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.326+0000 7f490164b740 1 Processor -- start 2024-09-10T10:04:33.329 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.326+0000 7f490164b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb388aec70 0x55cb388cd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:33.329 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.326+0000 7f490164b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cb388af590 0x55cb388cd890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:33.329 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.326+0000 7f490164b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cb388b9e50 0x55cb3893bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:33.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.326+0000 7f490164b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f48f81b25c0 con 0x55cb388af590 2024-09-10T10:04:33.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.326+0000 7f490164b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f48f81b1e90 con 0x55cb388b9e50 2024-09-10T10:04:33.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.326+0000 7f490164b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f48f81af040 con 0x55cb388aec70 2024-09-10T10:04:33.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.326+0000 7f490164b740 10 client.0 _ll_register_callbacks cb 0x7f48e4045070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:04:33.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.326+0000 7f48f7e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cb388af590 0x55cb388cd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:33.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.326+0000 7f48f7400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb388aec70 0x55cb388cd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:33.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.326+0000 7f48f7400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb388aec70 0x55cb388cd350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:54332/0 (socket says 172.21.15.77:54332) 2024-09-10T10:04:33.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.326+0000 7f48f7400640 1 -- 172.21.15.77:0/327461916 learned_addr learned my addr 172.21.15.77:0/327461916 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:04:33.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.327+0000 7f48f7400640 1 --2- 172.21.15.77:0/327461916 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cb388b9e50 0x55cb3893bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:33.332 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.327+0000 7f48f7400640 1 -- 172.21.15.77:0/327461916 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cb388b9e50 msgr2=0x55cb3893bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:33.332 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.327+0000 7f48f7400640 1 --2- 172.21.15.77:0/327461916 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cb388b9e50 0x55cb3893bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:33.332 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.327+0000 7f48f7400640 1 -- 172.21.15.77:0/327461916 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cb388af590 msgr2=0x55cb388cd890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:33.332 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.327+0000 7f48f7400640 1 --2- 172.21.15.77:0/327461916 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cb388af590 0x55cb388cd890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:33.332 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.327+0000 7f48f7400640 1 -- 172.21.15.77:0/327461916 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb388abe90 con 0x55cb388aec70 2024-09-10T10:04:33.333 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.327+0000 7f48f7400640 1 --2- 172.21.15.77:0/327461916 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb388aec70 0x55cb388cd350 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f48e4005d70 tx=0x7f48e40043d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f836c75447d9acab server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:33.333 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.327+0000 7f48e1600640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-10T10:04:33.333 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.328+0000 7f48e1600640 1 -- 172.21.15.77:0/327461916 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e4037040 con 0x55cb388aec70 2024-09-10T10:04:33.333 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.328+0000 7f48e1600640 1 -- 172.21.15.77:0/327461916 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f48e404dab0 con 0x55cb388aec70 2024-09-10T10:04:33.333 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.328+0000 7f490164b740 1 -- 172.21.15.77:0/327461916 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48e8005500 con 0x55cb388aec70 2024-09-10T10:04:33.333 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.328+0000 7f48e1600640 1 -- 172.21.15.77:0/327461916 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f48e4009450 con 0x55cb388aec70 2024-09-10T10:04:33.333 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.328+0000 7f490164b740 10 client.27340 fetch_fsmap learned FSMap version 40 2024-09-10T10:04:33.333 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.328+0000 7f490164b740 1 -- 172.21.15.77:0/327461916 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55cb3893d400 con 0x55cb388aec70 2024-09-10T10:04:33.333 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.328+0000 7f48e1600640 1 -- 172.21.15.77:0/327461916 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48e8005500 con 0x55cb388aec70 2024-09-10T10:04:33.333 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.328+0000 7f48e1600640 1 -- 172.21.15.77:0/327461916 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 40) ==== 8398+0+0 (secure 0 0 0) 0x7f48e40035f0 con 0x55cb388aec70 2024-09-10T10:04:33.333 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.329+0000 7f490164b740 10 client.27340 fetch_fsmap finished waiting for FSMap version 40 2024-09-10T10:04:33.333 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.329+0000 7f490164b740 10 client.27340 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T10:04:33.333 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.329+0000 7f490164b740 20 client.27340 populate_metadata read hostname 'smithi077' 2024-09-10T10:04:33.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.329+0000 7f490164b740 1 --2- 172.21.15.77:0/327461916 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x55cb3893d8e0 0x55cb3893fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:33.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.329+0000 7f490164b740 4 client.27340 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:04:33.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.329+0000 7f490164b740 1 -- 172.21.15.77:0/327461916 --> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cb3881ce60 con 0x55cb3893d8e0 2024-09-10T10:04:33.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.329+0000 7f48f7e00640 1 --2- 172.21.15.77:0/327461916 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x55cb3893d8e0 0x55cb3893fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:33.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.329+0000 7f48f7e00640 1 --2- 172.21.15.77:0/327461916 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x55cb3893d8e0 0x55cb3893fcd0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=72e44b070864031e server_cookie=f114064eef080942 in_seq=0 out_seq=0 2024-09-10T10:04:33.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.329+0000 7f48e1600640 10 client.27340 ms_handle_connect on v2:172.21.15.134:6838/1965117680 2024-09-10T10:04:33.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.330+0000 7f48e1600640 1 -- 172.21.15.77:0/327461916 <== mds.2 v2:172.21.15.134:6838/1965117680 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cb3881ce60 con 0x55cb3893d8e0 2024-09-10T10:04:33.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.330+0000 7f48e1600640 10 client.27340 handle_command_reply: tid=0 2024-09-10T10:04:33.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.330+0000 7f490164b740 1 client.27340 shutdown 2024-09-10T10:04:33.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.330+0000 7f490164b740 1 -- 172.21.15.77:0/327461916 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb388aec70 msgr2=0x55cb388cd350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:33.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.330+0000 7f490164b740 1 --2- 172.21.15.77:0/327461916 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb388aec70 0x55cb388cd350 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f48e4005d70 tx=0x7f48e40043d0 comp rx=0 tx=0).stop 2024-09-10T10:04:33.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.330+0000 7f490164b740 1 -- 172.21.15.77:0/327461916 shutdown_connections 2024-09-10T10:04:33.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.330+0000 7f490164b740 1 --2- 172.21.15.77:0/327461916 >> [v2:172.21.15.134:6838/1965117680,v1:172.21.15.134:6839/1965117680] conn(0x55cb3893d8e0 0x55cb3893fcd0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:33.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.330+0000 7f490164b740 1 --2- 172.21.15.77:0/327461916 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cb388b9e50 0x55cb3893bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:33.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.330+0000 7f490164b740 1 --2- 172.21.15.77:0/327461916 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cb388af590 0x55cb388cd890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:33.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.330+0000 7f490164b740 1 --2- 172.21.15.77:0/327461916 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb388aec70 0x55cb388cd350 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:33.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.330+0000 7f48e1600640 0 client.27340 ms_handle_reset on v2:172.21.15.134:6838/1965117680 2024-09-10T10:04:33.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.330+0000 7f490164b740 1 -- 172.21.15.77:0/327461916 >> 172.21.15.77:0/327461916 conn(0x55cb388ab200 msgr2=0x55cb388c4df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:33.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.331+0000 7f490164b740 1 -- 172.21.15.77:0/327461916 shutdown_connections 2024-09-10T10:04:33.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.331+0000 7f490164b740 1 -- 172.21.15.77:0/327461916 wait complete. 2024-09-10T10:04:33.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.331+0000 7f490164b740 20 client.27340 trim_cache size 0 max 16384 2024-09-10T10:04:33.336 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-10T10:04:33.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.332+0000 7f48fec00640 1 -- 172.21.15.77:0/963511546 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f48dc0a2b80 msgr2=0x7f48dc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:33.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.332+0000 7f48fec00640 1 --2- 172.21.15.77:0/963511546 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f48dc0a2b80 0x7f48dc0a5040 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f48f000cd00 tx=0x7f48f000ceb0 comp rx=0 tx=0).stop 2024-09-10T10:04:33.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.332+0000 7f48fec00640 1 -- 172.21.15.77:0/963511546 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f48f8109140 msgr2=0x7f48f819ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:33.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.332+0000 7f48fec00640 1 --2- 172.21.15.77:0/963511546 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f48f8109140 0x7f48f819ef60 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f48f000efc0 tx=0x7f48f000c800 comp rx=0 tx=0).stop 2024-09-10T10:04:33.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.332+0000 7f48fec00640 1 -- 172.21.15.77:0/963511546 shutdown_connections 2024-09-10T10:04:33.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.332+0000 7f48fec00640 1 --2- 172.21.15.77:0/963511546 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f48dc0a2b80 0x7f48dc0a5040 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:33.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.332+0000 7f48fec00640 1 --2- 172.21.15.77:0/963511546 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f48f8109140 0x7f48f819ef60 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:33.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.332+0000 7f48fec00640 1 --2- 172.21.15.77:0/963511546 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f48f81049c0 0x7f48f819abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:33.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.332+0000 7f48fec00640 1 --2- 172.21.15.77:0/963511546 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48f8104010 0x7f48f819a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:33.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.332+0000 7f48fec00640 1 -- 172.21.15.77:0/963511546 >> 172.21.15.77:0/963511546 conn(0x7f48f80ffa50 msgr2=0x7f48f8100e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:33.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.334+0000 7f48fec00640 1 -- 172.21.15.77:0/963511546 shutdown_connections 2024-09-10T10:04:33.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:33.334+0000 7f48fec00640 1 -- 172.21.15.77:0/963511546 wait complete. 2024-09-10T10:04:33.356 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:04:33.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:04:32 smithi134 ceph-mon[35674]: pgmap v672: 129 pgs: 129 active+clean; 120 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 806 KiB/s rd, 748 KiB/s wr, 305 op/s 2024-09-10T10:04:33.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:04:32 smithi120 ceph-mon[34770]: pgmap v672: 129 pgs: 129 active+clean; 120 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 806 KiB/s rd, 748 KiB/s wr, 305 op/s 2024-09-10T10:04:34.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.401+0000 7f7001400640 1 -- 172.21.15.77:0/150001576 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ffc106f20 msgr2=0x7f6ffc10eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:34.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.401+0000 7f7001400640 1 --2- 172.21.15.77:0/150001576 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ffc106f20 0x7f6ffc10eaa0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f6ff4009ca0 tx=0x7f6ff402a8c0 comp rx=0 tx=0).stop 2024-09-10T10:04:34.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.401+0000 7f7001400640 1 -- 172.21.15.77:0/150001576 shutdown_connections 2024-09-10T10:04:34.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.401+0000 7f7001400640 1 --2- 172.21.15.77:0/150001576 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ffc106f20 0x7f6ffc10eaa0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:34.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.401+0000 7f7001400640 1 --2- 172.21.15.77:0/150001576 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ffc1027a0 0x7f6ffc1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:34.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.401+0000 7f7001400640 1 --2- 172.21.15.77:0/150001576 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6ffc101df0 0x7f6ffc1021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:34.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.401+0000 7f7001400640 1 -- 172.21.15.77:0/150001576 >> 172.21.15.77:0/150001576 conn(0x7f6ffc0fd8b0 msgr2=0x7f6ffc0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:34.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.401+0000 7f7001400640 1 -- 172.21.15.77:0/150001576 shutdown_connections 2024-09-10T10:04:34.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.401+0000 7f7001400640 1 -- 172.21.15.77:0/150001576 wait complete. 2024-09-10T10:04:34.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.402+0000 7f7001400640 1 Processor -- start 2024-09-10T10:04:34.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.402+0000 7f7001400640 1 -- start start 2024-09-10T10:04:34.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.402+0000 7f7001400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ffc101df0 0x7f6ffc19a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:34.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.402+0000 7f7001400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ffc1027a0 0x7f6ffc19ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:34.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.402+0000 7f7001400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6ffc106f20 0x7f6ffc19eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:34.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.402+0000 7f7001400640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6ffc111f80 con 0x7f6ffc101df0 2024-09-10T10:04:34.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.402+0000 7f7001400640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6ffc111e00 con 0x7f6ffc106f20 2024-09-10T10:04:34.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.402+0000 7f7001400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6ffc112100 con 0x7f6ffc1027a0 2024-09-10T10:04:34.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.402+0000 7f6ffa800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ffc101df0 0x7f6ffc19a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:34.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.402+0000 7f6ffa800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ffc101df0 0x7f6ffc19a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60642/0 (socket says 172.21.15.77:60642) 2024-09-10T10:04:34.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.402+0000 7f6ffa800640 1 -- 172.21.15.77:0/3022878397 learned_addr learned my addr 172.21.15.77:0/3022878397 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:04:34.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.403+0000 7f6ff3e00640 1 --2- 172.21.15.77:0/3022878397 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ffc1027a0 0x7f6ffc19ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:34.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.403+0000 7f6ffb200640 1 --2- 172.21.15.77:0/3022878397 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6ffc106f20 0x7f6ffc19eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:34.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.403+0000 7f6ffa800640 1 -- 172.21.15.77:0/3022878397 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ffc1027a0 msgr2=0x7f6ffc19ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:34.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.403+0000 7f6ffa800640 1 --2- 172.21.15.77:0/3022878397 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ffc1027a0 0x7f6ffc19ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:34.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.403+0000 7f6ffa800640 1 -- 172.21.15.77:0/3022878397 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6ffc106f20 msgr2=0x7f6ffc19eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:34.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.403+0000 7f6ffa800640 1 --2- 172.21.15.77:0/3022878397 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6ffc106f20 0x7f6ffc19eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:34.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.403+0000 7f6ffa800640 1 -- 172.21.15.77:0/3022878397 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ffc19f5d0 con 0x7f6ffc101df0 2024-09-10T10:04:34.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.403+0000 7f6ffa800640 1 --2- 172.21.15.77:0/3022878397 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ffc101df0 0x7f6ffc19a620 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f6fec00e9f0 tx=0x7f6fec00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:34.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.404+0000 7f6ff3400640 1 -- 172.21.15.77:0/3022878397 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fec00cd80 con 0x7f6ffc101df0 2024-09-10T10:04:34.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.404+0000 7f6ff3400640 1 -- 172.21.15.77:0/3022878397 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6fec010910 con 0x7f6ffc101df0 2024-09-10T10:04:34.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.404+0000 7f7001400640 1 -- 172.21.15.77:0/3022878397 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ffc19f8c0 con 0x7f6ffc101df0 2024-09-10T10:04:34.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.404+0000 7f6ff3400640 1 -- 172.21.15.77:0/3022878397 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fec018990 con 0x7f6ffc101df0 2024-09-10T10:04:34.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.404+0000 7f7001400640 1 -- 172.21.15.77:0/3022878397 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ffc1a71a0 con 0x7f6ffc101df0 2024-09-10T10:04:34.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.407+0000 7f7003dce740 1 Processor -- start 2024-09-10T10:04:34.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.407+0000 7f6ff3400640 1 -- 172.21.15.77:0/3022878397 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6fec010ac0 con 0x7f6ffc101df0 2024-09-10T10:04:34.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.407+0000 7f7003dce740 1 -- start start 2024-09-10T10:04:34.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.407+0000 7f7003dce740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610cbe0a6d0 0x5610cbe0aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:34.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.407+0000 7f7003dce740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610cbe0aff0 0x5610cbe15370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:34.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.407+0000 7f7003dce740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610cbe158b0 0x5610cbe17ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:34.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.407+0000 7f7003dce740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5610cbd71340 con 0x5610cbe158b0 2024-09-10T10:04:34.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.407+0000 7f7003dce740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5610cbd18bf0 con 0x5610cbe0a6d0 2024-09-10T10:04:34.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.407+0000 7f7003dce740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5610cbd6d300 con 0x5610cbe0aff0 2024-09-10T10:04:34.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.407+0000 7f6ffb200640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610cbe0a6d0 0x5610cbe0aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:34.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.407+0000 7f6ffb200640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610cbe0a6d0 0x5610cbe0aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:36792/0 (socket says 172.21.15.77:36792) 2024-09-10T10:04:34.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.408+0000 7f6ffb200640 1 -- 172.21.15.77:0/2793152657 learned_addr learned my addr 172.21.15.77:0/2793152657 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:04:34.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.407+0000 7f6ffa800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610cbe0aff0 0x5610cbe15370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:34.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.408+0000 7f6ff3e00640 1 --2- 172.21.15.77:0/2793152657 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610cbe158b0 0x5610cbe17ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:34.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.408+0000 7f6ffb200640 1 -- 172.21.15.77:0/2793152657 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610cbe0aff0 msgr2=0x5610cbe15370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:34.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.408+0000 7f6ffb200640 1 --2- 172.21.15.77:0/2793152657 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610cbe0aff0 0x5610cbe15370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:34.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.408+0000 7f6ffb200640 1 -- 172.21.15.77:0/2793152657 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610cbe158b0 msgr2=0x5610cbe17ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:34.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.408+0000 7f6ffb200640 1 --2- 172.21.15.77:0/2793152657 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610cbe158b0 0x5610cbe17ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:34.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.408+0000 7f6ffb200640 1 -- 172.21.15.77:0/2793152657 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610cbae2620 con 0x5610cbe0a6d0 2024-09-10T10:04:34.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.408+0000 7f6ff3e00640 1 --2- 172.21.15.77:0/2793152657 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610cbe158b0 0x5610cbe17ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:04:34.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.408+0000 7f6ffb200640 1 --2- 172.21.15.77:0/2793152657 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610cbe0a6d0 0x5610cbe0aab0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f6ff4009ca0 tx=0x7f6ff4005f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ea5a44a5ef46098 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:34.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.408+0000 7f6ff3400640 1 --2- 172.21.15.77:0/3022878397 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f6fe00a2b80 0x7f6fe00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:34.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.409+0000 7f6feb400640 1 -- 172.21.15.77:0/2793152657 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff4031db0 con 0x5610cbe0a6d0 2024-09-10T10:04:34.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.409+0000 7f6feb400640 1 -- 172.21.15.77:0/2793152657 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ff4036040 con 0x5610cbe0a6d0 2024-09-10T10:04:34.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.409+0000 7f6feb400640 1 -- 172.21.15.77:0/2793152657 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff4004330 con 0x5610cbe0a6d0 2024-09-10T10:04:34.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.409+0000 7f6ff3400640 1 -- 172.21.15.77:0/3022878397 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f6fec0cf7a0 con 0x7f6ffc101df0 2024-09-10T10:04:34.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.409+0000 7f6ffa800640 1 --2- 172.21.15.77:0/3022878397 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f6fe00a2b80 0x7f6fe00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:34.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.409+0000 7f7003dce740 1 -- 172.21.15.77:0/2793152657 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610cbe0a6d0 msgr2=0x5610cbe0aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:34.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.409+0000 7f7003dce740 1 --2- 172.21.15.77:0/2793152657 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610cbe0a6d0 0x5610cbe0aab0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f6ff4009ca0 tx=0x7f6ff4005f10 comp rx=0 tx=0).stop 2024-09-10T10:04:34.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.409+0000 7f6ffa800640 1 --2- 172.21.15.77:0/3022878397 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f6fe00a2b80 0x7f6fe00a5040 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f6fec098630 tx=0x7f6fec0d5530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:34.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.410+0000 7f7003dce740 1 -- 172.21.15.77:0/2793152657 shutdown_connections 2024-09-10T10:04:34.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.410+0000 7f7003dce740 1 --2- 172.21.15.77:0/2793152657 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610cbe158b0 0x5610cbe17ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:34.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.410+0000 7f7003dce740 1 --2- 172.21.15.77:0/2793152657 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610cbe0aff0 0x5610cbe15370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:34.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.410+0000 7f7003dce740 1 --2- 172.21.15.77:0/2793152657 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610cbe0a6d0 0x5610cbe0aab0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:34.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.410+0000 7f7003dce740 1 -- 172.21.15.77:0/2793152657 >> 172.21.15.77:0/2793152657 conn(0x5610cbe06c60 msgr2=0x5610cbe07040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:34.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.410+0000 7f7003dce740 1 -- 172.21.15.77:0/2793152657 shutdown_connections 2024-09-10T10:04:34.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.410+0000 7f7003dce740 1 -- 172.21.15.77:0/2793152657 wait complete. 2024-09-10T10:04:34.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.410+0000 7f7003dce740 1 -- start start 2024-09-10T10:04:34.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.410+0000 7f7003dce740 1 Processor -- start 2024-09-10T10:04:34.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.411+0000 7f7003dce740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610cbe0a6d0 0x5610cbe127f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:34.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.411+0000 7f7003dce740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610cbe0aff0 0x5610cbe12d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:34.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.411+0000 7f7003dce740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610cbe158b0 0x5610cbe13270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:34.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.411+0000 7f7003dce740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5610cbe18300 con 0x5610cbe0aff0 2024-09-10T10:04:34.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.411+0000 7f7003dce740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5610cbd2c060 con 0x5610cbe158b0 2024-09-10T10:04:34.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.411+0000 7f7003dce740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5610cbe18480 con 0x5610cbe0a6d0 2024-09-10T10:04:34.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.411+0000 7f7003dce740 10 client.0 _ll_register_callbacks cb 0x5610cbd1aaf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:04:34.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.411+0000 7f6ff3e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610cbe0aff0 0x5610cbe12d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:34.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.411+0000 7f6ff3e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610cbe0aff0 0x5610cbe12d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60658/0 (socket says 172.21.15.77:60658) 2024-09-10T10:04:34.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.411+0000 7f6ff3e00640 1 -- 172.21.15.77:0/2240170924 learned_addr learned my addr 172.21.15.77:0/2240170924 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:04:34.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.411+0000 7f6ffb200640 1 --2- 172.21.15.77:0/2240170924 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610cbe158b0 0x5610cbe13270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:34.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.411+0000 7f6ffb200640 1 --2- 172.21.15.77:0/2240170924 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610cbe0a6d0 0x5610cbe127f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:34.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.412+0000 7f6ff3e00640 1 -- 172.21.15.77:0/2240170924 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610cbe0a6d0 msgr2=0x5610cbe127f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:34.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.412+0000 7f6ff3e00640 1 --2- 172.21.15.77:0/2240170924 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610cbe0a6d0 0x5610cbe127f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:34.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.412+0000 7f6ff3e00640 1 -- 172.21.15.77:0/2240170924 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610cbe158b0 msgr2=0x5610cbe13270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:34.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.412+0000 7f6ff3e00640 1 --2- 172.21.15.77:0/2240170924 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610cbe158b0 0x5610cbe13270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:34.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.412+0000 7f6ff3e00640 1 -- 172.21.15.77:0/2240170924 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610cbe21840 con 0x5610cbe0aff0 2024-09-10T10:04:34.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.412+0000 7f6ff3e00640 1 --2- 172.21.15.77:0/2240170924 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610cbe0aff0 0x5610cbe12d30 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f6fe4012710 tx=0x7f6fe4012bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38d7d246defddb3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:34.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.412+0000 7f6fdfe00640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-10T10:04:34.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.412+0000 7f6fdfe00640 1 -- 172.21.15.77:0/2240170924 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe400f920 con 0x5610cbe0aff0 2024-09-10T10:04:34.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.412+0000 7f7003dce740 1 -- 172.21.15.77:0/2240170924 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ff4036040 con 0x5610cbe0aff0 2024-09-10T10:04:34.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.412+0000 7f6fdfe00640 1 -- 172.21.15.77:0/2240170924 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f6fe400fac0 con 0x5610cbe0aff0 2024-09-10T10:04:34.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.413+0000 7f6fdfe00640 1 -- 172.21.15.77:0/2240170924 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f6fe400fcc0 con 0x5610cbe0aff0 2024-09-10T10:04:34.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.413+0000 7f7003dce740 10 client.17247 fetch_fsmap learned FSMap version 40 2024-09-10T10:04:34.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.413+0000 7f7003dce740 1 -- 172.21.15.77:0/2240170924 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5610cbe24280 con 0x5610cbe0aff0 2024-09-10T10:04:34.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.413+0000 7f6fdfe00640 1 -- 172.21.15.77:0/2240170924 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ff4036040 con 0x5610cbe0aff0 2024-09-10T10:04:34.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.413+0000 7f6fdfe00640 1 -- 172.21.15.77:0/2240170924 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 40) ==== 8398+0+0 (secure 0 0 0) 0x7f6fe401c420 con 0x5610cbe0aff0 2024-09-10T10:04:34.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.413+0000 7f7003dce740 10 client.17247 fetch_fsmap finished waiting for FSMap version 40 2024-09-10T10:04:34.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.413+0000 7f7003dce740 10 client.17247 resolve_mds: validated gid 24563 aka daemon mds.b 2024-09-10T10:04:34.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.413+0000 7f7003dce740 20 client.17247 populate_metadata read hostname 'smithi077' 2024-09-10T10:04:34.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.413+0000 7f7003dce740 1 --2- 172.21.15.77:0/2240170924 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x5610cbe25730 0x5610cbe27b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:34.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.413+0000 7f7003dce740 4 client.17247 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:04:34.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.413+0000 7f7003dce740 1 -- 172.21.15.77:0/2240170924 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5610cbd7e8b0 con 0x5610cbe25730 2024-09-10T10:04:34.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.414+0000 7f6ffb200640 1 --2- 172.21.15.77:0/2240170924 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x5610cbe25730 0x5610cbe27b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:34.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.414+0000 7f6ffb200640 1 --2- 172.21.15.77:0/2240170924 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x5610cbe25730 0x5610cbe27b50 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35e9d8debf938810 server_cookie=2fdc208ffa4bce04 in_seq=0 out_seq=0 2024-09-10T10:04:34.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.414+0000 7f6fdfe00640 10 client.17247 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T10:04:34.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.415+0000 7f6fdfe00640 1 -- 172.21.15.77:0/2240170924 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5610cbd7e8b0 con 0x5610cbe25730 2024-09-10T10:04:34.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.415+0000 7f6fdfe00640 10 client.17247 handle_command_reply: tid=0 2024-09-10T10:04:34.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.415+0000 7f7003dce740 1 client.17247 shutdown 2024-09-10T10:04:34.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.416+0000 7f7003dce740 1 -- 172.21.15.77:0/2240170924 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610cbe0aff0 msgr2=0x5610cbe12d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:34.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.416+0000 7f7003dce740 1 --2- 172.21.15.77:0/2240170924 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610cbe0aff0 0x5610cbe12d30 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f6fe4012710 tx=0x7f6fe4012bd0 comp rx=0 tx=0).stop 2024-09-10T10:04:34.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.416+0000 7f7003dce740 1 -- 172.21.15.77:0/2240170924 shutdown_connections 2024-09-10T10:04:34.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.416+0000 7f7003dce740 1 --2- 172.21.15.77:0/2240170924 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x5610cbe25730 0x5610cbe27b50 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:34.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.416+0000 7f7003dce740 1 --2- 172.21.15.77:0/2240170924 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610cbe158b0 0x5610cbe13270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:34.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.416+0000 7f7003dce740 1 --2- 172.21.15.77:0/2240170924 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610cbe0aff0 0x5610cbe12d30 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:34.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.416+0000 7f6fdfe00640 0 client.17247 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T10:04:34.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.416+0000 7f7003dce740 1 --2- 172.21.15.77:0/2240170924 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610cbe0a6d0 0x5610cbe127f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:34.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.416+0000 7f7003dce740 1 -- 172.21.15.77:0/2240170924 >> 172.21.15.77:0/2240170924 conn(0x5610cbe06c60 msgr2=0x5610cbe0c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:34.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.416+0000 7f7003dce740 1 -- 172.21.15.77:0/2240170924 shutdown_connections 2024-09-10T10:04:34.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.416+0000 7f7003dce740 1 -- 172.21.15.77:0/2240170924 wait complete. 2024-09-10T10:04:34.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.416+0000 7f7003dce740 20 client.17247 trim_cache size 0 max 16384 2024-09-10T10:04:34.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.434+0000 7f7003dce740 1 Processor -- start 2024-09-10T10:04:34.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.434+0000 7f7003dce740 1 -- start start 2024-09-10T10:04:34.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.434+0000 7f7003dce740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610cbe0a6d0 0x5610cbe07270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:34.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.434+0000 7f7003dce740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610cbe0aff0 0x5610cbe11750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:34.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.434+0000 7f7003dce740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610cbe158b0 0x5610cbe2d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:34.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.434+0000 7f7003dce740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6ffc1b0fd0 con 0x5610cbe0aff0 2024-09-10T10:04:34.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.434+0000 7f7003dce740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6ffc1b2c00 con 0x5610cbe0a6d0 2024-09-10T10:04:34.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.434+0000 7f7003dce740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6ffc1b1700 con 0x5610cbe158b0 2024-09-10T10:04:34.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.434+0000 7f6ff3e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610cbe0aff0 0x5610cbe11750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:34.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.434+0000 7f6ff3e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610cbe0aff0 0x5610cbe11750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60660/0 (socket says 172.21.15.77:60660) 2024-09-10T10:04:34.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.434+0000 7f6ff3e00640 1 -- 172.21.15.77:0/4106514400 learned_addr learned my addr 172.21.15.77:0/4106514400 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:04:34.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.434+0000 7f6ffb200640 1 --2- 172.21.15.77:0/4106514400 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610cbe0a6d0 0x5610cbe07270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:34.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.435+0000 7f6ffb200640 1 --2- 172.21.15.77:0/4106514400 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610cbe158b0 0x5610cbe2d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:34.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.435+0000 7f6ff3e00640 1 -- 172.21.15.77:0/4106514400 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610cbe158b0 msgr2=0x5610cbe2d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:34.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.435+0000 7f6ff3e00640 1 --2- 172.21.15.77:0/4106514400 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610cbe158b0 0x5610cbe2d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:34.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.435+0000 7f6ff3e00640 1 -- 172.21.15.77:0/4106514400 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610cbe0a6d0 msgr2=0x5610cbe07270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:34.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.435+0000 7f6ff3e00640 1 --2- 172.21.15.77:0/4106514400 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610cbe0a6d0 0x5610cbe07270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:34.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.435+0000 7f6ff3e00640 1 -- 172.21.15.77:0/4106514400 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610cbe18010 con 0x5610cbe0aff0 2024-09-10T10:04:34.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.435+0000 7f6ffb200640 1 --2- 172.21.15.77:0/4106514400 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610cbe158b0 0x5610cbe2d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:04:34.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.435+0000 7f6ff3e00640 1 --2- 172.21.15.77:0/4106514400 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610cbe0aff0 0x5610cbe11750 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f6fe4012420 tx=0x7f6fe4005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0bcfd92cc6647f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:34.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.435+0000 7f6feaa00640 1 -- 172.21.15.77:0/4106514400 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe4009e80 con 0x5610cbe0aff0 2024-09-10T10:04:34.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.435+0000 7f6feaa00640 1 -- 172.21.15.77:0/4106514400 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6fe400f920 con 0x5610cbe0aff0 2024-09-10T10:04:34.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.435+0000 7f6feaa00640 1 -- 172.21.15.77:0/4106514400 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe401c680 con 0x5610cbe0aff0 2024-09-10T10:04:34.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.436+0000 7f7003dce740 1 -- 172.21.15.77:0/4106514400 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610cbe0aff0 msgr2=0x5610cbe11750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:34.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.436+0000 7f7003dce740 1 --2- 172.21.15.77:0/4106514400 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610cbe0aff0 0x5610cbe11750 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f6fe4012420 tx=0x7f6fe4005db0 comp rx=0 tx=0).stop 2024-09-10T10:04:34.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.436+0000 7f7003dce740 1 -- 172.21.15.77:0/4106514400 shutdown_connections 2024-09-10T10:04:34.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.436+0000 7f7003dce740 1 --2- 172.21.15.77:0/4106514400 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610cbe158b0 0x5610cbe2d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:34.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.436+0000 7f7003dce740 1 --2- 172.21.15.77:0/4106514400 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610cbe0aff0 0x5610cbe11750 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:34.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.436+0000 7f7003dce740 1 --2- 172.21.15.77:0/4106514400 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610cbe0a6d0 0x5610cbe07270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:34.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.436+0000 7f7003dce740 1 -- 172.21.15.77:0/4106514400 >> 172.21.15.77:0/4106514400 conn(0x5610cbe06c60 msgr2=0x5610cbe14870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:34.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.436+0000 7f7003dce740 1 -- 172.21.15.77:0/4106514400 shutdown_connections 2024-09-10T10:04:34.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.436+0000 7f7003dce740 1 -- 172.21.15.77:0/4106514400 wait complete. 2024-09-10T10:04:34.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.437+0000 7f7003dce740 1 -- start start 2024-09-10T10:04:34.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.437+0000 7f7003dce740 1 Processor -- start 2024-09-10T10:04:34.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.437+0000 7f7003dce740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610cbe0a6d0 0x5610cbe1d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:34.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.437+0000 7f7003dce740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610cbe0aff0 0x5610cbe1dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:34.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.437+0000 7f7003dce740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610cbe158b0 0x5610cbe1a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:34.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.437+0000 7f7003dce740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6ffc1b2530 con 0x5610cbe0a6d0 2024-09-10T10:04:34.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.437+0000 7f7003dce740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6ffc1b1e00 con 0x5610cbe0aff0 2024-09-10T10:04:34.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.437+0000 7f7003dce740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6ffc1aefb0 con 0x5610cbe158b0 2024-09-10T10:04:34.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.437+0000 7f7003dce740 10 client.0 _ll_register_callbacks cb 0x7f6fe4009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:04:34.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.438+0000 7f6ffb200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610cbe0a6d0 0x5610cbe1d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:34.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.438+0000 7f6ffb200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610cbe0a6d0 0x5610cbe1d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60664/0 (socket says 172.21.15.77:60664) 2024-09-10T10:04:34.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.438+0000 7f6ffb200640 1 -- 172.21.15.77:0/2193238582 learned_addr learned my addr 172.21.15.77:0/2193238582 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:04:34.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.438+0000 7f6ff3e00640 1 --2- 172.21.15.77:0/2193238582 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610cbe0aff0 0x5610cbe1dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:34.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.438+0000 7f6ffb200640 1 --2- 172.21.15.77:0/2193238582 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610cbe158b0 0x5610cbe1a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:34.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.438+0000 7f6ffb200640 1 -- 172.21.15.77:0/2193238582 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610cbe158b0 msgr2=0x5610cbe1a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:34.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.438+0000 7f6ffb200640 1 --2- 172.21.15.77:0/2193238582 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610cbe158b0 0x5610cbe1a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:34.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.438+0000 7f6ffb200640 1 -- 172.21.15.77:0/2193238582 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610cbe0aff0 msgr2=0x5610cbe1dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:34.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.438+0000 7f6ffb200640 1 --2- 172.21.15.77:0/2193238582 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610cbe0aff0 0x5610cbe1dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:34.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.438+0000 7f6ffb200640 1 -- 172.21.15.77:0/2193238582 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610cbe1e300 con 0x5610cbe0a6d0 2024-09-10T10:04:34.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.438+0000 7f6ffb200640 1 --2- 172.21.15.77:0/2193238582 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610cbe0a6d0 0x5610cbe1d6d0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f6ff402a900 tx=0x7f6ff4004a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c446e2c102ab824 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:34.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.438+0000 7f6fdfe00640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-10T10:04:34.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.439+0000 7f6fdfe00640 1 -- 172.21.15.77:0/2193238582 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff4003970 con 0x5610cbe0a6d0 2024-09-10T10:04:34.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.439+0000 7f6fdfe00640 1 -- 172.21.15.77:0/2193238582 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f6ff4003b10 con 0x5610cbe0a6d0 2024-09-10T10:04:34.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.439+0000 7f7003dce740 1 -- 172.21.15.77:0/2193238582 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6fe400f920 con 0x5610cbe0a6d0 2024-09-10T10:04:34.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.439+0000 7f6fdfe00640 1 -- 172.21.15.77:0/2193238582 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f6ff4003d10 con 0x5610cbe0a6d0 2024-09-10T10:04:34.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.439+0000 7f7003dce740 10 client.17253 fetch_fsmap learned FSMap version 40 2024-09-10T10:04:34.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.439+0000 7f7003dce740 1 -- 172.21.15.77:0/2193238582 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5610cbe909c0 con 0x5610cbe0a6d0 2024-09-10T10:04:34.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.439+0000 7f6fdfe00640 1 -- 172.21.15.77:0/2193238582 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6fe400f920 con 0x5610cbe0a6d0 2024-09-10T10:04:34.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.439+0000 7f6fdfe00640 1 -- 172.21.15.77:0/2193238582 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 40) ==== 8398+0+0 (secure 0 0 0) 0x7f6ff405b070 con 0x5610cbe0a6d0 2024-09-10T10:04:34.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.439+0000 7f7003dce740 10 client.17253 fetch_fsmap finished waiting for FSMap version 40 2024-09-10T10:04:34.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.439+0000 7f7003dce740 10 client.17253 resolve_mds: validated gid 24563 aka daemon mds.b 2024-09-10T10:04:34.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.439+0000 7f7003dce740 20 client.17253 populate_metadata read hostname 'smithi077' 2024-09-10T10:04:34.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.439+0000 7f7003dce740 1 --2- 172.21.15.77:0/2193238582 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x5610cbe90d60 0x5610cbe93150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:34.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.439+0000 7f7003dce740 4 client.17253 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-09-10T10:04:34.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.439+0000 7f7003dce740 1 -- 172.21.15.77:0/2193238582 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x5610cbd78dc0 con 0x5610cbe90d60 2024-09-10T10:04:34.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.440+0000 7f6ff3e00640 1 --2- 172.21.15.77:0/2193238582 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x5610cbe90d60 0x5610cbe93150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:34.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.440+0000 7f6ff3e00640 1 --2- 172.21.15.77:0/2193238582 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x5610cbe90d60 0x5610cbe93150 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7bcce3e9a2052d0 server_cookie=8250b8cc79e861d4 in_seq=0 out_seq=0 2024-09-10T10:04:34.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.440+0000 7f6fdfe00640 10 client.17253 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T10:04:34.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.441+0000 7f6fdfe00640 1 -- 172.21.15.77:0/2193238582 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x5610cbd78dc0 con 0x5610cbe90d60 2024-09-10T10:04:34.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.441+0000 7f6fdfe00640 10 client.17253 handle_command_reply: tid=0 2024-09-10T10:04:34.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.441+0000 7f7003dce740 1 client.17253 shutdown 2024-09-10T10:04:34.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.441+0000 7f7003dce740 1 -- 172.21.15.77:0/2193238582 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610cbe0a6d0 msgr2=0x5610cbe1d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:34.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.441+0000 7f7003dce740 1 --2- 172.21.15.77:0/2193238582 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610cbe0a6d0 0x5610cbe1d6d0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f6ff402a900 tx=0x7f6ff4004a80 comp rx=0 tx=0).stop 2024-09-10T10:04:34.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.441+0000 7f7003dce740 1 -- 172.21.15.77:0/2193238582 shutdown_connections 2024-09-10T10:04:34.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.441+0000 7f7003dce740 1 --2- 172.21.15.77:0/2193238582 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x5610cbe90d60 0x5610cbe93150 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:34.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.441+0000 7f7003dce740 1 --2- 172.21.15.77:0/2193238582 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610cbe158b0 0x5610cbe1a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:34.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.441+0000 7f7003dce740 1 --2- 172.21.15.77:0/2193238582 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610cbe0aff0 0x5610cbe1dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:34.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.441+0000 7f7003dce740 1 --2- 172.21.15.77:0/2193238582 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610cbe0a6d0 0x5610cbe1d6d0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:34.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.441+0000 7f6fdfe00640 0 client.17253 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T10:04:34.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.441+0000 7f7003dce740 1 -- 172.21.15.77:0/2193238582 >> 172.21.15.77:0/2193238582 conn(0x5610cbe06c60 msgr2=0x5610cbe0c120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:34.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.441+0000 7f7003dce740 1 -- 172.21.15.77:0/2193238582 shutdown_connections 2024-09-10T10:04:34.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.441+0000 7f7003dce740 1 -- 172.21.15.77:0/2193238582 wait complete. 2024-09-10T10:04:34.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.441+0000 7f7003dce740 20 client.17253 trim_cache size 0 max 16384 2024-09-10T10:04:34.447 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-10T10:04:34.447 INFO:teuthology.orchestra.run.smithi077.stdout: "epoch": 40, 2024-09-10T10:04:34.447 INFO:teuthology.orchestra.run.smithi077.stdout: "leader": 24563, 2024-09-10T10:04:34.447 INFO:teuthology.orchestra.run.smithi077.stdout: "set_version": 31, 2024-09-10T10:04:34.448 INFO:teuthology.orchestra.run.smithi077.stdout: "sets": {} 2024-09-10T10:04:34.448 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-10T10:04:34.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.442+0000 7f7001400640 1 -- 172.21.15.77:0/3022878397 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f6fe00a2b80 msgr2=0x7f6fe00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:34.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.442+0000 7f7001400640 1 --2- 172.21.15.77:0/3022878397 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f6fe00a2b80 0x7f6fe00a5040 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f6fec098630 tx=0x7f6fec0d5530 comp rx=0 tx=0).stop 2024-09-10T10:04:34.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.442+0000 7f7001400640 1 -- 172.21.15.77:0/3022878397 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ffc101df0 msgr2=0x7f6ffc19a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:34.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.442+0000 7f7001400640 1 --2- 172.21.15.77:0/3022878397 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ffc101df0 0x7f6ffc19a620 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f6fec00e9f0 tx=0x7f6fec00eeb0 comp rx=0 tx=0).stop 2024-09-10T10:04:34.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.442+0000 7f7001400640 1 -- 172.21.15.77:0/3022878397 shutdown_connections 2024-09-10T10:04:34.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.442+0000 7f7001400640 1 --2- 172.21.15.77:0/3022878397 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f6fe00a2b80 0x7f6fe00a5040 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:34.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.442+0000 7f7001400640 1 --2- 172.21.15.77:0/3022878397 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6ffc106f20 0x7f6ffc19eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:34.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.442+0000 7f7001400640 1 --2- 172.21.15.77:0/3022878397 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ffc1027a0 0x7f6ffc19ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:34.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.442+0000 7f7001400640 1 --2- 172.21.15.77:0/3022878397 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ffc101df0 0x7f6ffc19a620 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:34.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.442+0000 7f7001400640 1 -- 172.21.15.77:0/3022878397 >> 172.21.15.77:0/3022878397 conn(0x7f6ffc0fd8b0 msgr2=0x7f6ffc0ffc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:34.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.442+0000 7f7001400640 1 -- 172.21.15.77:0/3022878397 shutdown_connections 2024-09-10T10:04:34.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:34.442+0000 7f7001400640 1 -- 172.21.15.77:0/3022878397 wait complete. 2024-09-10T10:04:34.785 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T10:04:35.133 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:04:34 smithi077 ceph-mon[30319]: pgmap v673: 129 pgs: 129 active+clean; 115 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 488 KiB/s wr, 255 op/s 2024-09-10T10:04:35.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:04:34 smithi134 ceph-mon[35674]: pgmap v673: 129 pgs: 129 active+clean; 115 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 488 KiB/s wr, 255 op/s 2024-09-10T10:04:35.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:04:34 smithi120 ceph-mon[34770]: pgmap v673: 129 pgs: 129 active+clean; 115 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 488 KiB/s wr, 255 op/s 2024-09-10T10:04:35.912 INFO:tasks.quiescer:done joining Quiescers 2024-09-10T10:04:35.912 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-09-10T10:04:35.922 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-09-10T10:04:35.997 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:04:36.517 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.515+0000 7f4775c00640 1 -- 172.21.15.77:0/4272034270 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4770106f10 msgr2=0x7f477010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:36.517 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.515+0000 7f4775c00640 1 --2- 172.21.15.77:0/4272034270 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4770106f10 0x7f477010ea90 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f4768009ca0 tx=0x7f476802a8c0 comp rx=0 tx=0).stop 2024-09-10T10:04:36.517 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.515+0000 7f4775c00640 1 -- 172.21.15.77:0/4272034270 shutdown_connections 2024-09-10T10:04:36.518 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.515+0000 7f4775c00640 1 --2- 172.21.15.77:0/4272034270 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4770106f10 0x7f477010ea90 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:36.518 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.515+0000 7f4775c00640 1 --2- 172.21.15.77:0/4272034270 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4770102790 0x7f47701067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:36.518 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.515+0000 7f4775c00640 1 --2- 172.21.15.77:0/4272034270 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4770101de0 0x7f47701021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:36.518 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.515+0000 7f4775c00640 1 -- 172.21.15.77:0/4272034270 >> 172.21.15.77:0/4272034270 conn(0x7f47700fd8c0 msgr2=0x7f47700ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:36.518 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.515+0000 7f4775c00640 1 -- 172.21.15.77:0/4272034270 shutdown_connections 2024-09-10T10:04:36.518 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.515+0000 7f4775c00640 1 -- 172.21.15.77:0/4272034270 wait complete. 2024-09-10T10:04:36.519 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.516+0000 7f4775c00640 1 Processor -- start 2024-09-10T10:04:36.519 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.516+0000 7f4775c00640 1 -- start start 2024-09-10T10:04:36.519 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.516+0000 7f4775c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4770101de0 0x7f477019a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:36.519 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.516+0000 7f4775c00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4770102790 0x7f477019ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:36.519 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.516+0000 7f4775c00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4770106f10 0x7f477019ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:36.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.516+0000 7f4775c00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4770111f90 con 0x7f4770106f10 2024-09-10T10:04:36.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.516+0000 7f4775c00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4770111e10 con 0x7f4770102790 2024-09-10T10:04:36.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.516+0000 7f4775c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4770112110 con 0x7f4770101de0 2024-09-10T10:04:36.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.517+0000 7f476f400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4770106f10 0x7f477019ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:36.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.517+0000 7f476f400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4770106f10 0x7f477019ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60682/0 (socket says 172.21.15.77:60682) 2024-09-10T10:04:36.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.517+0000 7f476e000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4770102790 0x7f477019ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:36.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.517+0000 7f476f400640 1 -- 172.21.15.77:0/3746838428 learned_addr learned my addr 172.21.15.77:0/3746838428 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:04:36.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.517+0000 7f476ea00640 1 --2- 172.21.15.77:0/3746838428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4770101de0 0x7f477019a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:36.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.517+0000 7f476e000640 1 -- 172.21.15.77:0/3746838428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4770101de0 msgr2=0x7f477019a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:36.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.517+0000 7f476e000640 1 --2- 172.21.15.77:0/3746838428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4770101de0 0x7f477019a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:36.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.517+0000 7f476e000640 1 -- 172.21.15.77:0/3746838428 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4770106f10 msgr2=0x7f477019ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:36.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.517+0000 7f476e000640 1 --2- 172.21.15.77:0/3746838428 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4770106f10 0x7f477019ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:36.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.517+0000 7f476e000640 1 -- 172.21.15.77:0/3746838428 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f477019f5e0 con 0x7f4770102790 2024-09-10T10:04:36.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.517+0000 7f476e000640 1 --2- 172.21.15.77:0/3746838428 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4770102790 0x7f477019ab70 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f475c00e9f0 tx=0x7f475c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:36.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.518+0000 7f4767400640 1 -- 172.21.15.77:0/3746838428 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f475c00ccc0 con 0x7f4770102790 2024-09-10T10:04:36.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.518+0000 7f4767400640 1 -- 172.21.15.77:0/3746838428 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f475c010910 con 0x7f4770102790 2024-09-10T10:04:36.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.518+0000 7f4775c00640 1 -- 172.21.15.77:0/3746838428 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f477019f8d0 con 0x7f4770102790 2024-09-10T10:04:36.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.518+0000 7f4767400640 1 -- 172.21.15.77:0/3746838428 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f475c018800 con 0x7f4770102790 2024-09-10T10:04:36.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.518+0000 7f4775c00640 1 -- 172.21.15.77:0/3746838428 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47701a71b0 con 0x7f4770102790 2024-09-10T10:04:36.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.521+0000 7f4778818740 1 Processor -- start 2024-09-10T10:04:36.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.521+0000 7f4778818740 1 -- start start 2024-09-10T10:04:36.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.521+0000 7f4767400640 1 -- 172.21.15.77:0/3746838428 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f475c010ac0 con 0x7f4770102790 2024-09-10T10:04:36.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.521+0000 7f4778818740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576e32a3c70 0x5576e32a4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:36.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.521+0000 7f4778818740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576e32a4590 0x5576e32ae910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:36.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.521+0000 7f4778818740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576e32aee50 0x5576e32b1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:36.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.521+0000 7f4778818740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5576e320a3e0 con 0x5576e32a3c70 2024-09-10T10:04:36.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.521+0000 7f4778818740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5576e31b1d80 con 0x5576e32a4590 2024-09-10T10:04:36.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.521+0000 7f4778818740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5576e32063a0 con 0x5576e32aee50 2024-09-10T10:04:36.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.522+0000 7f476f400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576e32a3c70 0x5576e32a4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:36.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.522+0000 7f476f400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576e32a3c70 0x5576e32a4050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60694/0 (socket says 172.21.15.77:60694) 2024-09-10T10:04:36.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.522+0000 7f476f400640 1 -- 172.21.15.77:0/3751580041 learned_addr learned my addr 172.21.15.77:0/3751580041 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:04:36.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.522+0000 7f476e000640 1 --2- 172.21.15.77:0/3751580041 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576e32aee50 0x5576e32b1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:36.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.522+0000 7f476ea00640 1 --2- 172.21.15.77:0/3751580041 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576e32a4590 0x5576e32ae910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:36.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.522+0000 7f476f400640 1 -- 172.21.15.77:0/3751580041 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576e32aee50 msgr2=0x5576e32b1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:36.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.522+0000 7f476f400640 1 --2- 172.21.15.77:0/3751580041 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576e32aee50 0x5576e32b1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:36.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.522+0000 7f476f400640 1 -- 172.21.15.77:0/3751580041 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576e32a4590 msgr2=0x5576e32ae910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:36.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.522+0000 7f476f400640 1 --2- 172.21.15.77:0/3751580041 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576e32a4590 0x5576e32ae910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:36.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.522+0000 7f476f400640 1 -- 172.21.15.77:0/3751580041 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576e31d0ae0 con 0x5576e32a3c70 2024-09-10T10:04:36.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.522+0000 7f476ea00640 1 --2- 172.21.15.77:0/3751580041 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576e32a4590 0x5576e32ae910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:04:36.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.523+0000 7f476f400640 1 --2- 172.21.15.77:0/3751580041 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576e32a3c70 0x5576e32a4050 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f4768009950 tx=0x7f4768009980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=800fc3de95cbc8b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:36.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.523+0000 7f475be00640 1 -- 172.21.15.77:0/3751580041 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4768003970 con 0x5576e32a3c70 2024-09-10T10:04:36.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.523+0000 7f4767400640 1 --2- 172.21.15.77:0/3746838428 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f47540a2c00 0x7f47540a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:36.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.523+0000 7f475be00640 1 -- 172.21.15.77:0/3751580041 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4768036040 con 0x5576e32a3c70 2024-09-10T10:04:36.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.523+0000 7f475be00640 1 -- 172.21.15.77:0/3751580041 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47680043c0 con 0x5576e32a3c70 2024-09-10T10:04:36.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.523+0000 7f4767400640 1 -- 172.21.15.77:0/3746838428 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f475c0cf670 con 0x7f4770102790 2024-09-10T10:04:36.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.523+0000 7f476ea00640 1 --2- 172.21.15.77:0/3746838428 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f47540a2c00 0x7f47540a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:36.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.524+0000 7f4778818740 1 -- 172.21.15.77:0/3751580041 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576e32a3c70 msgr2=0x5576e32a4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:36.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.524+0000 7f4778818740 1 --2- 172.21.15.77:0/3751580041 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576e32a3c70 0x5576e32a4050 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f4768009950 tx=0x7f4768009980 comp rx=0 tx=0).stop 2024-09-10T10:04:36.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.524+0000 7f476ea00640 1 --2- 172.21.15.77:0/3746838428 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f47540a2c00 0x7f47540a50c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x5576e32a1060 tx=0x7f4760009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:36.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.524+0000 7f4778818740 1 -- 172.21.15.77:0/3751580041 shutdown_connections 2024-09-10T10:04:36.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.524+0000 7f4778818740 1 --2- 172.21.15.77:0/3751580041 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576e32aee50 0x5576e32b1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:36.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.524+0000 7f4778818740 1 --2- 172.21.15.77:0/3751580041 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576e32a4590 0x5576e32ae910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:36.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.524+0000 7f4778818740 1 --2- 172.21.15.77:0/3751580041 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576e32a3c70 0x5576e32a4050 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:36.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.524+0000 7f4778818740 1 -- 172.21.15.77:0/3751580041 >> 172.21.15.77:0/3751580041 conn(0x5576e32a0200 msgr2=0x5576e32a05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:36.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.524+0000 7f4778818740 1 -- 172.21.15.77:0/3751580041 shutdown_connections 2024-09-10T10:04:36.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.524+0000 7f4778818740 1 -- 172.21.15.77:0/3751580041 wait complete. 2024-09-10T10:04:36.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.525+0000 7f4778818740 1 -- start start 2024-09-10T10:04:36.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.525+0000 7f4778818740 1 Processor -- start 2024-09-10T10:04:36.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.525+0000 7f4778818740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576e32a3c70 0x5576e32abcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:36.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.525+0000 7f4778818740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576e32a4590 0x5576e32ac1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:36.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.525+0000 7f4778818740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576e32aee50 0x5576e32ac730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:36.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.525+0000 7f4778818740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5576e32b18a0 con 0x5576e32a4590 2024-09-10T10:04:36.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.525+0000 7f4778818740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5576e31c50f0 con 0x5576e32aee50 2024-09-10T10:04:36.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.525+0000 7f4778818740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5576e32b1a20 con 0x5576e32a3c70 2024-09-10T10:04:36.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.525+0000 7f4778818740 10 client.0 _ll_register_callbacks cb 0x5576e31acbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:04:36.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.526+0000 7f476e000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576e32a4590 0x5576e32ac1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:36.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.526+0000 7f476ea00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576e32a3c70 0x5576e32abcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:36.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.526+0000 7f476f400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576e32aee50 0x5576e32ac730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:36.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.526+0000 7f476ea00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576e32a3c70 0x5576e32abcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:54418/0 (socket says 172.21.15.77:54418) 2024-09-10T10:04:36.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.526+0000 7f476ea00640 1 -- 172.21.15.77:0/1139402884 learned_addr learned my addr 172.21.15.77:0/1139402884 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:04:36.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.526+0000 7f476e000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576e32a4590 0x5576e32ac1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60710/0 (socket says 172.21.15.77:60710) 2024-09-10T10:04:36.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.526+0000 7f476f400640 1 -- 172.21.15.77:0/1139402884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576e32a3c70 msgr2=0x5576e32abcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:36.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.526+0000 7f476f400640 1 --2- 172.21.15.77:0/1139402884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576e32a3c70 0x5576e32abcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:36.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.526+0000 7f476f400640 1 -- 172.21.15.77:0/1139402884 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576e32a4590 msgr2=0x5576e32ac1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:36.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.526+0000 7f476f400640 1 --2- 172.21.15.77:0/1139402884 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576e32a4590 0x5576e32ac1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:36.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.526+0000 7f476f400640 1 -- 172.21.15.77:0/1139402884 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576e2f7b3c0 con 0x5576e32aee50 2024-09-10T10:04:36.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.526+0000 7f476e000640 1 --2- 172.21.15.77:0/1139402884 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576e32a4590 0x5576e32ac1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:04:36.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.526+0000 7f476f400640 1 --2- 172.21.15.77:0/1139402884 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576e32aee50 0x5576e32ac730 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f4768009920 tx=0x7f4768009a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6475cb06d0f2b2ab server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:36.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.526+0000 7f4758c00640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T10:04:36.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.527+0000 7f4758c00640 1 -- 172.21.15.77:0/1139402884 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4768004120 con 0x5576e32aee50 2024-09-10T10:04:36.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.527+0000 7f4778818740 1 -- 172.21.15.77:0/1139402884 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4768036040 con 0x5576e32aee50 2024-09-10T10:04:36.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.527+0000 7f4758c00640 1 -- 172.21.15.77:0/1139402884 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f47680042c0 con 0x5576e32aee50 2024-09-10T10:04:36.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.527+0000 7f4758c00640 1 -- 172.21.15.77:0/1139402884 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f476805c050 con 0x5576e32aee50 2024-09-10T10:04:36.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.527+0000 7f4778818740 10 client.27431 fetch_fsmap learned FSMap version 40 2024-09-10T10:04:36.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.527+0000 7f4778818740 1 -- 172.21.15.77:0/1139402884 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5576e32b18a0 con 0x5576e32aee50 2024-09-10T10:04:36.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.527+0000 7f4758c00640 1 -- 172.21.15.77:0/1139402884 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4768036040 con 0x5576e32aee50 2024-09-10T10:04:36.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.528+0000 7f4758c00640 1 -- 172.21.15.77:0/1139402884 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 40) ==== 8398+0+0 (secure 0 0 0) 0x7f4768057030 con 0x5576e32aee50 2024-09-10T10:04:36.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.528+0000 7f4778818740 10 client.27431 fetch_fsmap finished waiting for FSMap version 40 2024-09-10T10:04:36.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.528+0000 7f4778818740 10 client.27431 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-10T10:04:36.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.528+0000 7f4778818740 20 client.27431 populate_metadata read hostname 'smithi077' 2024-09-10T10:04:36.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.528+0000 7f4778818740 1 --2- 172.21.15.77:0/1139402884 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x5576e32bea60 0x5576e32c0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:36.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.528+0000 7f4778818740 4 client.27431 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:04:36.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.528+0000 7f4778818740 1 -- 172.21.15.77:0/1139402884 --> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5576e3217950 con 0x5576e32bea60 2024-09-10T10:04:36.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.528+0000 7f476ea00640 1 --2- 172.21.15.77:0/1139402884 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x5576e32bea60 0x5576e32c0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:36.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.529+0000 7f476ea00640 1 --2- 172.21.15.77:0/1139402884 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x5576e32bea60 0x5576e32c0e50 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d0cd422f6436d6e6 server_cookie=25931d8da4a69d3e in_seq=0 out_seq=0 2024-09-10T10:04:36.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.529+0000 7f4758c00640 10 client.27431 ms_handle_connect on v2:172.21.15.120:6836/3033280709 2024-09-10T10:04:36.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.529+0000 7f4758c00640 1 -- 172.21.15.77:0/1139402884 <== mds.1 v2:172.21.15.120:6836/3033280709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5576e3217950 con 0x5576e32bea60 2024-09-10T10:04:36.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.529+0000 7f4758c00640 10 client.27431 handle_command_reply: tid=0 2024-09-10T10:04:36.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.530+0000 7f4778818740 1 client.27431 shutdown 2024-09-10T10:04:36.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.530+0000 7f4778818740 1 -- 172.21.15.77:0/1139402884 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576e32aee50 msgr2=0x5576e32ac730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:36.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.530+0000 7f4778818740 1 --2- 172.21.15.77:0/1139402884 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576e32aee50 0x5576e32ac730 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f4768009920 tx=0x7f4768009a20 comp rx=0 tx=0).stop 2024-09-10T10:04:36.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.530+0000 7f4778818740 1 -- 172.21.15.77:0/1139402884 shutdown_connections 2024-09-10T10:04:36.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.530+0000 7f4778818740 1 --2- 172.21.15.77:0/1139402884 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x5576e32bea60 0x5576e32c0e50 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:36.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.530+0000 7f4778818740 1 --2- 172.21.15.77:0/1139402884 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576e32aee50 0x5576e32ac730 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:36.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.530+0000 7f4778818740 1 --2- 172.21.15.77:0/1139402884 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576e32a4590 0x5576e32ac1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:36.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.530+0000 7f4778818740 1 --2- 172.21.15.77:0/1139402884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576e32a3c70 0x5576e32abcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:36.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.530+0000 7f4778818740 1 -- 172.21.15.77:0/1139402884 >> 172.21.15.77:0/1139402884 conn(0x5576e32a0200 msgr2=0x5576e329f380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:36.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.530+0000 7f4758c00640 0 client.27431 ms_handle_reset on v2:172.21.15.120:6836/3033280709 2024-09-10T10:04:36.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.530+0000 7f4778818740 1 -- 172.21.15.77:0/1139402884 shutdown_connections 2024-09-10T10:04:36.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.530+0000 7f4778818740 1 -- 172.21.15.77:0/1139402884 wait complete. 2024-09-10T10:04:36.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.530+0000 7f4778818740 20 client.27431 trim_cache size 0 max 16384 2024-09-10T10:04:36.545 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.543+0000 7f4778818740 1 Processor -- start 2024-09-10T10:04:36.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.544+0000 7f4778818740 1 -- start start 2024-09-10T10:04:36.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.544+0000 7f4778818740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576e32a3c70 0x5576e32a66d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:36.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.544+0000 7f4778818740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576e32a4590 0x5576e32adfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:36.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.544+0000 7f4778818740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576e32aee50 0x5576e32c6700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:36.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.544+0000 7f4778818740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f47701b1050 con 0x5576e32aee50 2024-09-10T10:04:36.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.544+0000 7f4778818740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f47701b2c80 con 0x5576e32a3c70 2024-09-10T10:04:36.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.544+0000 7f4778818740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f47701b1780 con 0x5576e32a4590 2024-09-10T10:04:36.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.544+0000 7f476f400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576e32aee50 0x5576e32c6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:36.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.544+0000 7f476f400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576e32aee50 0x5576e32c6700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60712/0 (socket says 172.21.15.77:60712) 2024-09-10T10:04:36.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.544+0000 7f476f400640 1 -- 172.21.15.77:0/1543432683 learned_addr learned my addr 172.21.15.77:0/1543432683 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:04:36.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.544+0000 7f476e000640 1 --2- 172.21.15.77:0/1543432683 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576e32a4590 0x5576e32adfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:36.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.544+0000 7f476ea00640 1 --2- 172.21.15.77:0/1543432683 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576e32a3c70 0x5576e32a66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:36.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.545+0000 7f476f400640 1 -- 172.21.15.77:0/1543432683 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576e32a4590 msgr2=0x5576e32adfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:36.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.545+0000 7f476f400640 1 --2- 172.21.15.77:0/1543432683 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576e32a4590 0x5576e32adfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:36.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.545+0000 7f476f400640 1 -- 172.21.15.77:0/1543432683 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576e32a3c70 msgr2=0x5576e32a66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:36.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.545+0000 7f476f400640 1 --2- 172.21.15.77:0/1543432683 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576e32a3c70 0x5576e32a66d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:36.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.545+0000 7f476f400640 1 -- 172.21.15.77:0/1543432683 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576e32a0030 con 0x5576e32aee50 2024-09-10T10:04:36.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.545+0000 7f476f400640 1 --2- 172.21.15.77:0/1543432683 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576e32aee50 0x5576e32c6700 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f4768002230 tx=0x7f4768000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df216ea61b4ca4ff server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:36.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.545+0000 7f475b400640 1 -- 172.21.15.77:0/1543432683 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4768003b80 con 0x5576e32aee50 2024-09-10T10:04:36.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.545+0000 7f475b400640 1 -- 172.21.15.77:0/1543432683 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4768036070 con 0x5576e32aee50 2024-09-10T10:04:36.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.545+0000 7f475b400640 1 -- 172.21.15.77:0/1543432683 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4768004470 con 0x5576e32aee50 2024-09-10T10:04:36.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.546+0000 7f4778818740 1 -- 172.21.15.77:0/1543432683 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576e32aee50 msgr2=0x5576e32c6700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:36.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.546+0000 7f4778818740 1 --2- 172.21.15.77:0/1543432683 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576e32aee50 0x5576e32c6700 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f4768002230 tx=0x7f4768000f40 comp rx=0 tx=0).stop 2024-09-10T10:04:36.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.546+0000 7f4778818740 1 -- 172.21.15.77:0/1543432683 shutdown_connections 2024-09-10T10:04:36.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.546+0000 7f4778818740 1 --2- 172.21.15.77:0/1543432683 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576e32aee50 0x5576e32c6700 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:36.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.546+0000 7f4778818740 1 --2- 172.21.15.77:0/1543432683 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576e32a4590 0x5576e32adfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:36.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.546+0000 7f4778818740 1 --2- 172.21.15.77:0/1543432683 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576e32a3c70 0x5576e32a66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:36.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.546+0000 7f4778818740 1 -- 172.21.15.77:0/1543432683 >> 172.21.15.77:0/1543432683 conn(0x5576e32a0200 msgr2=0x5576e32aac70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:36.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.546+0000 7f4778818740 1 -- 172.21.15.77:0/1543432683 shutdown_connections 2024-09-10T10:04:36.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.546+0000 7f4778818740 1 -- 172.21.15.77:0/1543432683 wait complete. 2024-09-10T10:04:36.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.546+0000 7f4778818740 1 -- start start 2024-09-10T10:04:36.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.547+0000 7f4778818740 1 Processor -- start 2024-09-10T10:04:36.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.547+0000 7f4778818740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576e32a3c70 0x5576e32b74e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:36.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.547+0000 7f4778818740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576e32a4590 0x5576e32b2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:36.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.547+0000 7f4778818740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576e32aee50 0x5576e32b2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:36.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.547+0000 7f4778818740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f47701b25b0 con 0x5576e32a4590 2024-09-10T10:04:36.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.547+0000 7f4778818740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f47701b1e80 con 0x5576e32aee50 2024-09-10T10:04:36.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.547+0000 7f4778818740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f47701af030 con 0x5576e32a3c70 2024-09-10T10:04:36.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.547+0000 7f4778818740 10 client.0 _ll_register_callbacks cb 0x7f47600093c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:04:36.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.547+0000 7f476e000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576e32a4590 0x5576e32b2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:36.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.547+0000 7f476e000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576e32a4590 0x5576e32b2810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60720/0 (socket says 172.21.15.77:60720) 2024-09-10T10:04:36.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.547+0000 7f476e000640 1 -- 172.21.15.77:0/2557497663 learned_addr learned my addr 172.21.15.77:0/2557497663 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:04:36.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.547+0000 7f476ea00640 1 --2- 172.21.15.77:0/2557497663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576e32a3c70 0x5576e32b74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:36.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.548+0000 7f476f400640 1 --2- 172.21.15.77:0/2557497663 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576e32aee50 0x5576e32b2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:36.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.548+0000 7f476e000640 1 -- 172.21.15.77:0/2557497663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576e32a3c70 msgr2=0x5576e32b74e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:36.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.548+0000 7f476e000640 1 --2- 172.21.15.77:0/2557497663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576e32a3c70 0x5576e32b74e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:36.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.548+0000 7f476e000640 1 -- 172.21.15.77:0/2557497663 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576e32aee50 msgr2=0x5576e32b2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:36.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.548+0000 7f476e000640 1 --2- 172.21.15.77:0/2557497663 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576e32aee50 0x5576e32b2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:36.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.548+0000 7f476e000640 1 -- 172.21.15.77:0/2557497663 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576e32a0e90 con 0x5576e32a4590 2024-09-10T10:04:36.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.548+0000 7f476e000640 1 --2- 172.21.15.77:0/2557497663 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576e32a4590 0x5576e32b2810 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f475c00c6a0 tx=0x7f475c010ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83b240d5c8b650d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:36.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.548+0000 7f4758c00640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-10T10:04:36.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.548+0000 7f4758c00640 1 -- 172.21.15.77:0/2557497663 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f475c09b440 con 0x5576e32a4590 2024-09-10T10:04:36.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.548+0000 7f4758c00640 1 -- 172.21.15.77:0/2557497663 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f475c0cfb20 con 0x5576e32a4590 2024-09-10T10:04:36.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.548+0000 7f4778818740 1 -- 172.21.15.77:0/2557497663 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4768036070 con 0x5576e32a4590 2024-09-10T10:04:36.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.549+0000 7f4758c00640 1 -- 172.21.15.77:0/2557497663 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f475c09c1e0 con 0x5576e32a4590 2024-09-10T10:04:36.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.549+0000 7f4778818740 10 client.17271 fetch_fsmap learned FSMap version 40 2024-09-10T10:04:36.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.549+0000 7f4778818740 1 -- 172.21.15.77:0/2557497663 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5576e32a12b0 con 0x5576e32a4590 2024-09-10T10:04:36.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.549+0000 7f4758c00640 1 -- 172.21.15.77:0/2557497663 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4768036070 con 0x5576e32a4590 2024-09-10T10:04:36.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.549+0000 7f4758c00640 1 -- 172.21.15.77:0/2557497663 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 40) ==== 8398+0+0 (secure 0 0 0) 0x7f475c097830 con 0x5576e32a4590 2024-09-10T10:04:36.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.549+0000 7f4778818740 10 client.17271 fetch_fsmap finished waiting for FSMap version 40 2024-09-10T10:04:36.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.549+0000 7f4778818740 10 client.17271 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-10T10:04:36.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.549+0000 7f4778818740 20 client.17271 populate_metadata read hostname 'smithi077' 2024-09-10T10:04:36.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.549+0000 7f4778818740 1 --2- 172.21.15.77:0/2557497663 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x5576e3329c00 0x5576e332c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:36.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.549+0000 7f4778818740 4 client.17271 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:04:36.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.549+0000 7f4778818740 1 -- 172.21.15.77:0/2557497663 --> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5576e3211e60 con 0x5576e3329c00 2024-09-10T10:04:36.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.550+0000 7f476ea00640 1 --2- 172.21.15.77:0/2557497663 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x5576e3329c00 0x5576e332c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:36.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.550+0000 7f476ea00640 1 --2- 172.21.15.77:0/2557497663 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x5576e3329c00 0x5576e332c020 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1d8c127d37f3f0bd server_cookie=5ed18d0423bd78a9 in_seq=0 out_seq=0 2024-09-10T10:04:36.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.550+0000 7f4758c00640 10 client.17271 ms_handle_connect on v2:172.21.15.120:6836/3033280709 2024-09-10T10:04:36.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.550+0000 7f4758c00640 1 -- 172.21.15.77:0/2557497663 <== mds.1 v2:172.21.15.120:6836/3033280709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5576e3211e60 con 0x5576e3329c00 2024-09-10T10:04:36.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.550+0000 7f4758c00640 10 client.17271 handle_command_reply: tid=0 2024-09-10T10:04:36.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.550+0000 7f4778818740 1 client.17271 shutdown 2024-09-10T10:04:36.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.550+0000 7f4778818740 1 -- 172.21.15.77:0/2557497663 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576e32a4590 msgr2=0x5576e32b2810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:36.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.550+0000 7f4778818740 1 --2- 172.21.15.77:0/2557497663 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576e32a4590 0x5576e32b2810 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f475c00c6a0 tx=0x7f475c010ee0 comp rx=0 tx=0).stop 2024-09-10T10:04:36.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.550+0000 7f4778818740 1 -- 172.21.15.77:0/2557497663 shutdown_connections 2024-09-10T10:04:36.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.550+0000 7f4778818740 1 --2- 172.21.15.77:0/2557497663 >> [v2:172.21.15.120:6836/3033280709,v1:172.21.15.120:6837/3033280709] conn(0x5576e3329c00 0x5576e332c020 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:36.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.551+0000 7f4778818740 1 --2- 172.21.15.77:0/2557497663 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576e32aee50 0x5576e32b2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:36.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.551+0000 7f4778818740 1 --2- 172.21.15.77:0/2557497663 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576e32a4590 0x5576e32b2810 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:36.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.551+0000 7f4778818740 1 --2- 172.21.15.77:0/2557497663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576e32a3c70 0x5576e32b74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:36.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.551+0000 7f4778818740 1 -- 172.21.15.77:0/2557497663 >> 172.21.15.77:0/2557497663 conn(0x5576e32a0200 msgr2=0x5576e32aa450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:36.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.551+0000 7f4758c00640 0 client.17271 ms_handle_reset on v2:172.21.15.120:6836/3033280709 2024-09-10T10:04:36.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.551+0000 7f4778818740 1 -- 172.21.15.77:0/2557497663 shutdown_connections 2024-09-10T10:04:36.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.551+0000 7f4778818740 1 -- 172.21.15.77:0/2557497663 wait complete. 2024-09-10T10:04:36.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.551+0000 7f4778818740 20 client.17271 trim_cache size 0 max 16384 2024-09-10T10:04:36.557 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-10T10:04:36.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.551+0000 7f4775c00640 1 -- 172.21.15.77:0/3746838428 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f47540a2c00 msgr2=0x7f47540a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:36.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.551+0000 7f4775c00640 1 --2- 172.21.15.77:0/3746838428 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f47540a2c00 0x7f47540a50c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x5576e32a1060 tx=0x7f4760009110 comp rx=0 tx=0).stop 2024-09-10T10:04:36.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.551+0000 7f4775c00640 1 -- 172.21.15.77:0/3746838428 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4770102790 msgr2=0x7f477019ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:36.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.551+0000 7f4775c00640 1 --2- 172.21.15.77:0/3746838428 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4770102790 0x7f477019ab70 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f475c00e9f0 tx=0x7f475c00edf0 comp rx=0 tx=0).stop 2024-09-10T10:04:36.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.552+0000 7f4775c00640 1 -- 172.21.15.77:0/3746838428 shutdown_connections 2024-09-10T10:04:36.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.552+0000 7f4775c00640 1 --2- 172.21.15.77:0/3746838428 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f47540a2c00 0x7f47540a50c0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:36.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.552+0000 7f4775c00640 1 --2- 172.21.15.77:0/3746838428 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4770106f10 0x7f477019ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:36.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.552+0000 7f4775c00640 1 --2- 172.21.15.77:0/3746838428 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4770102790 0x7f477019ab70 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:36.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.552+0000 7f4775c00640 1 --2- 172.21.15.77:0/3746838428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4770101de0 0x7f477019a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:36.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.552+0000 7f4775c00640 1 -- 172.21.15.77:0/3746838428 >> 172.21.15.77:0/3746838428 conn(0x7f47700fd8c0 msgr2=0x7f47700feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:36.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.552+0000 7f4775c00640 1 -- 172.21.15.77:0/3746838428 shutdown_connections 2024-09-10T10:04:36.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:36.552+0000 7f4775c00640 1 -- 172.21.15.77:0/3746838428 wait complete. 2024-09-10T10:04:37.260 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:04:36 smithi077 ceph-mon[30319]: pgmap v674: 129 pgs: 129 active+clean; 115 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 321 KiB/s wr, 144 op/s 2024-09-10T10:04:37.260 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:04:36 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:04:37.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:04:36 smithi134 ceph-mon[35674]: pgmap v674: 129 pgs: 129 active+clean; 115 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 321 KiB/s wr, 144 op/s 2024-09-10T10:04:37.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:04:36 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:04:37.417 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:04:36 smithi120 ceph-mon[34770]: pgmap v674: 129 pgs: 129 active+clean; 115 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 321 KiB/s wr, 144 op/s 2024-09-10T10:04:37.417 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:04:36 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:04:37.591 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T10:04:38.723 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:04:39.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:04:38 smithi077 ceph-mon[30319]: pgmap v675: 129 pgs: 129 active+clean; 115 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 321 KiB/s wr, 106 op/s 2024-09-10T10:04:39.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:04:38 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T10:04:39.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:04:38 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T10:04:39.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:04:38 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T10:04:39.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:04:38 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T10:04:39.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.327+0000 7f0b36600640 1 -- 172.21.15.77:0/2835272896 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0b30101de0 msgr2=0x7f0b301021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:39.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.327+0000 7f0b36600640 1 --2- 172.21.15.77:0/2835272896 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0b30101de0 0x7f0b301021c0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f0b20009900 tx=0x7f0b2002a400 comp rx=0 tx=0).stop 2024-09-10T10:04:39.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.327+0000 7f0b36600640 1 -- 172.21.15.77:0/2835272896 shutdown_connections 2024-09-10T10:04:39.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.327+0000 7f0b36600640 1 --2- 172.21.15.77:0/2835272896 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0b30106f10 0x7f0b3010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:39.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.327+0000 7f0b36600640 1 --2- 172.21.15.77:0/2835272896 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b30102790 0x7f0b301067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:39.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.327+0000 7f0b36600640 1 --2- 172.21.15.77:0/2835272896 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0b30101de0 0x7f0b301021c0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:39.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.327+0000 7f0b36600640 1 -- 172.21.15.77:0/2835272896 >> 172.21.15.77:0/2835272896 conn(0x7f0b300fd8c0 msgr2=0x7f0b300ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:39.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.327+0000 7f0b36600640 1 -- 172.21.15.77:0/2835272896 shutdown_connections 2024-09-10T10:04:39.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.327+0000 7f0b36600640 1 -- 172.21.15.77:0/2835272896 wait complete. 2024-09-10T10:04:39.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.328+0000 7f0b36600640 1 Processor -- start 2024-09-10T10:04:39.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.328+0000 7f0b36600640 1 -- start start 2024-09-10T10:04:39.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.329+0000 7f0b36600640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0b30101de0 0x7f0b3019a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:39.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.329+0000 7f0b36600640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0b30102790 0x7f0b3019abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:39.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.329+0000 7f0b36600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b30106f10 0x7f0b3019ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:39.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.329+0000 7f0b36600640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0b30111fc0 con 0x7f0b30101de0 2024-09-10T10:04:39.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.329+0000 7f0b36600640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0b30111e40 con 0x7f0b30102790 2024-09-10T10:04:39.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.329+0000 7f0b36600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0b30112140 con 0x7f0b30106f10 2024-09-10T10:04:39.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.329+0000 7f0b2f400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0b30101de0 0x7f0b3019a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:39.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.329+0000 7f0b2f400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0b30101de0 0x7f0b3019a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60732/0 (socket says 172.21.15.77:60732) 2024-09-10T10:04:39.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.329+0000 7f0b2f400640 1 -- 172.21.15.77:0/3873389812 learned_addr learned my addr 172.21.15.77:0/3873389812 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:04:39.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.329+0000 7f0b2ea00640 1 --2- 172.21.15.77:0/3873389812 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0b30102790 0x7f0b3019abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:39.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.329+0000 7f0b2fe00640 1 --2- 172.21.15.77:0/3873389812 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b30106f10 0x7f0b3019ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:39.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.329+0000 7f0b2f400640 1 -- 172.21.15.77:0/3873389812 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b30106f10 msgr2=0x7f0b3019ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:39.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.329+0000 7f0b2f400640 1 --2- 172.21.15.77:0/3873389812 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b30106f10 0x7f0b3019ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:39.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.329+0000 7f0b2f400640 1 -- 172.21.15.77:0/3873389812 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0b30102790 msgr2=0x7f0b3019abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:39.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.329+0000 7f0b2f400640 1 --2- 172.21.15.77:0/3873389812 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0b30102790 0x7f0b3019abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:39.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.329+0000 7f0b2f400640 1 -- 172.21.15.77:0/3873389812 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b3019f620 con 0x7f0b30101de0 2024-09-10T10:04:39.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.330+0000 7f0b2f400640 1 --2- 172.21.15.77:0/3873389812 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0b30101de0 0x7f0b3019a670 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f0b20004770 tx=0x7f0b20003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:39.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.330+0000 7f0b27e00640 1 -- 172.21.15.77:0/3873389812 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b2003f070 con 0x7f0b30101de0 2024-09-10T10:04:39.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.330+0000 7f0b27e00640 1 -- 172.21.15.77:0/3873389812 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b20031ca0 con 0x7f0b30101de0 2024-09-10T10:04:39.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.330+0000 7f0b36600640 1 -- 172.21.15.77:0/3873389812 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b3019f8b0 con 0x7f0b30101de0 2024-09-10T10:04:39.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.330+0000 7f0b27e00640 1 -- 172.21.15.77:0/3873389812 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b20035030 con 0x7f0b30101de0 2024-09-10T10:04:39.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.330+0000 7f0b36600640 1 -- 172.21.15.77:0/3873389812 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b301a71f0 con 0x7f0b30101de0 2024-09-10T10:04:39.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.333+0000 7f0b27e00640 1 -- 172.21.15.77:0/3873389812 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0b20042070 con 0x7f0b30101de0 2024-09-10T10:04:39.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.333+0000 7f0b39248740 1 Processor -- start 2024-09-10T10:04:39.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.334+0000 7f0b39248740 1 -- start start 2024-09-10T10:04:39.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.334+0000 7f0b39248740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c3348cc70 0x557c3348d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:39.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.334+0000 7f0b39248740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c3348d590 0x557c33497910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:39.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.334+0000 7f0b39248740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557c33497e50 0x557c3349a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:39.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.334+0000 7f0b39248740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557c333f33e0 con 0x557c33497e50 2024-09-10T10:04:39.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.334+0000 7f0b39248740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x557c3339ad80 con 0x557c3348d590 2024-09-10T10:04:39.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.334+0000 7f0b39248740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557c333ef3a0 con 0x557c3348cc70 2024-09-10T10:04:39.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.334+0000 7f0b2fe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c3348cc70 0x557c3348d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:39.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.334+0000 7f0b2fe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c3348cc70 0x557c3348d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:54458/0 (socket says 172.21.15.77:54458) 2024-09-10T10:04:39.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.334+0000 7f0b2fe00640 1 -- 172.21.15.77:0/1321311081 learned_addr learned my addr 172.21.15.77:0/1321311081 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:04:39.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.334+0000 7f0b2f400640 1 --2- 172.21.15.77:0/1321311081 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c3348d590 0x557c33497910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:39.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.335+0000 7f0b2ea00640 1 --2- 172.21.15.77:0/1321311081 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557c33497e50 0x557c3349a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:39.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.335+0000 7f0b2fe00640 1 -- 172.21.15.77:0/1321311081 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c3348d590 msgr2=0x557c33497910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:39.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.335+0000 7f0b2fe00640 1 --2- 172.21.15.77:0/1321311081 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c3348d590 0x557c33497910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:39.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.335+0000 7f0b2fe00640 1 -- 172.21.15.77:0/1321311081 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557c33497e50 msgr2=0x557c3349a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:39.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.335+0000 7f0b2fe00640 1 --2- 172.21.15.77:0/1321311081 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557c33497e50 0x557c3349a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:39.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.335+0000 7f0b2fe00640 1 -- 172.21.15.77:0/1321311081 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c333b9ae0 con 0x557c3348cc70 2024-09-10T10:04:39.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.335+0000 7f0b2ea00640 1 --2- 172.21.15.77:0/1321311081 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557c33497e50 0x557c3349a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:04:39.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.335+0000 7f0b27e00640 1 --2- 172.21.15.77:0/3873389812 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f0b140a2c00 0x7f0b140a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:39.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.335+0000 7f0b2fe00640 1 --2- 172.21.15.77:0/1321311081 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c3348cc70 0x557c3348d050 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f0b28009f50 tx=0x7f0b280118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d21f75aad672bf8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:39.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.335+0000 7f0b27e00640 1 -- 172.21.15.77:0/3873389812 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f0b200ef660 con 0x7f0b30101de0 2024-09-10T10:04:39.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.335+0000 7f0b2f400640 1 --2- 172.21.15.77:0/3873389812 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f0b140a2c00 0x7f0b140a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:39.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.335+0000 7f0b24c00640 1 -- 172.21.15.77:0/1321311081 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b28004d20 con 0x557c3348cc70 2024-09-10T10:04:39.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.336+0000 7f0b24c00640 1 -- 172.21.15.77:0/1321311081 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b28004050 con 0x557c3348cc70 2024-09-10T10:04:39.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.336+0000 7f0b24c00640 1 -- 172.21.15.77:0/1321311081 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b28005280 con 0x557c3348cc70 2024-09-10T10:04:39.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.336+0000 7f0b39248740 1 -- 172.21.15.77:0/1321311081 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c3348cc70 msgr2=0x557c3348d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:39.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.336+0000 7f0b39248740 1 --2- 172.21.15.77:0/1321311081 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c3348cc70 0x557c3348d050 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f0b28009f50 tx=0x7f0b280118d0 comp rx=0 tx=0).stop 2024-09-10T10:04:39.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.336+0000 7f0b39248740 1 -- 172.21.15.77:0/1321311081 shutdown_connections 2024-09-10T10:04:39.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.336+0000 7f0b39248740 1 --2- 172.21.15.77:0/1321311081 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557c33497e50 0x557c3349a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:39.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.336+0000 7f0b39248740 1 --2- 172.21.15.77:0/1321311081 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c3348d590 0x557c33497910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:39.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.336+0000 7f0b39248740 1 --2- 172.21.15.77:0/1321311081 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c3348cc70 0x557c3348d050 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:39.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.336+0000 7f0b39248740 1 -- 172.21.15.77:0/1321311081 >> 172.21.15.77:0/1321311081 conn(0x557c33489200 msgr2=0x557c334895e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:39.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.337+0000 7f0b39248740 1 -- 172.21.15.77:0/1321311081 shutdown_connections 2024-09-10T10:04:39.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.337+0000 7f0b39248740 1 -- 172.21.15.77:0/1321311081 wait complete. 2024-09-10T10:04:39.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.337+0000 7f0b2f400640 1 --2- 172.21.15.77:0/3873389812 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f0b140a2c00 0x7f0b140a50c0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f0b200b85b0 tx=0x7f0b200f5110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:39.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.337+0000 7f0b39248740 1 -- start start 2024-09-10T10:04:39.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.337+0000 7f0b39248740 1 Processor -- start 2024-09-10T10:04:39.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.338+0000 7f0b39248740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c3348cc70 0x557c33494d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:39.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.338+0000 7f0b39248740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c3348d590 0x557c33495270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:39.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.338+0000 7f0b39248740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557c33497e50 0x557c334957b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:39.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.338+0000 7f0b39248740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557c3349aa20 con 0x557c33497e50 2024-09-10T10:04:39.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.338+0000 7f0b39248740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x557c333ae0f0 con 0x557c3348d590 2024-09-10T10:04:39.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.338+0000 7f0b39248740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557c3349aba0 con 0x557c3348cc70 2024-09-10T10:04:39.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.338+0000 7f0b39248740 10 client.0 _ll_register_callbacks cb 0x557c33395bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:04:39.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.338+0000 7f0b2ea00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c3348d590 0x557c33495270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:39.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.338+0000 7f0b2ea00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c3348d590 0x557c33495270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:36912/0 (socket says 172.21.15.77:36912) 2024-09-10T10:04:39.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.338+0000 7f0b2ea00640 1 -- 172.21.15.77:0/3366528525 learned_addr learned my addr 172.21.15.77:0/3366528525 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:04:39.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.338+0000 7f0b2fe00640 1 --2- 172.21.15.77:0/3366528525 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557c33497e50 0x557c334957b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:39.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.338+0000 7f0b2fe00640 1 --2- 172.21.15.77:0/3366528525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c3348cc70 0x557c33494d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:39.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.339+0000 7f0b2ea00640 1 -- 172.21.15.77:0/3366528525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c3348cc70 msgr2=0x557c33494d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:39.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.339+0000 7f0b2ea00640 1 --2- 172.21.15.77:0/3366528525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c3348cc70 0x557c33494d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:39.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.339+0000 7f0b2ea00640 1 -- 172.21.15.77:0/3366528525 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557c33497e50 msgr2=0x557c334957b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:39.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.339+0000 7f0b2ea00640 1 --2- 172.21.15.77:0/3366528525 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557c33497e50 0x557c334957b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:39.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.339+0000 7f0b2ea00640 1 -- 172.21.15.77:0/3366528525 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c331643c0 con 0x557c3348d590 2024-09-10T10:04:39.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.339+0000 7f0b2ea00640 1 --2- 172.21.15.77:0/3366528525 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c3348d590 0x557c33495270 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c012700 tx=0x7f0b1c012b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e60538352963863 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:39.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.339+0000 7f0b19600640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T10:04:39.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.339+0000 7f0b19600640 1 -- 172.21.15.77:0/3366528525 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b1c004050 con 0x557c3348d590 2024-09-10T10:04:39.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.339+0000 7f0b39248740 1 -- 172.21.15.77:0/3366528525 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b28004050 con 0x557c3348d590 2024-09-10T10:04:39.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.339+0000 7f0b19600640 1 -- 172.21.15.77:0/3366528525 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f0b1c0041f0 con 0x557c3348d590 2024-09-10T10:04:39.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.339+0000 7f0b19600640 1 -- 172.21.15.77:0/3366528525 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f0b1c025050 con 0x557c3348d590 2024-09-10T10:04:39.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.339+0000 7f0b39248740 10 client.27449 fetch_fsmap learned FSMap version 40 2024-09-10T10:04:39.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.339+0000 7f0b39248740 1 -- 172.21.15.77:0/3366528525 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x557c334a6ac0 con 0x557c3348d590 2024-09-10T10:04:39.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.340+0000 7f0b19600640 1 -- 172.21.15.77:0/3366528525 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b28004050 con 0x557c3348d590 2024-09-10T10:04:39.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.340+0000 7f0b19600640 1 -- 172.21.15.77:0/3366528525 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 40) ==== 8398+0+0 (secure 0 0 0) 0x7f0b1c003710 con 0x557c3348d590 2024-09-10T10:04:39.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.340+0000 7f0b39248740 10 client.27449 fetch_fsmap finished waiting for FSMap version 40 2024-09-10T10:04:39.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.340+0000 7f0b39248740 10 client.27449 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T10:04:39.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.340+0000 7f0b39248740 20 client.27449 populate_metadata read hostname 'smithi077' 2024-09-10T10:04:39.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.340+0000 7f0b39248740 1 --2- 172.21.15.77:0/3366528525 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x557c334a8020 0x557c334aa440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:39.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.340+0000 7f0b39248740 4 client.27449 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:04:39.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.340+0000 7f0b39248740 1 -- 172.21.15.77:0/3366528525 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557c33400950 con 0x557c334a8020 2024-09-10T10:04:39.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.340+0000 7f0b2fe00640 1 --2- 172.21.15.77:0/3366528525 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x557c334a8020 0x557c334aa440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:39.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.341+0000 7f0b2fe00640 1 --2- 172.21.15.77:0/3366528525 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x557c334a8020 0x557c334aa440 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=624a4a5bed87fe03 server_cookie=31bff24d1a7a2796 in_seq=0 out_seq=0 2024-09-10T10:04:39.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.341+0000 7f0b19600640 10 client.27449 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T10:04:39.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.342+0000 7f0b19600640 1 -- 172.21.15.77:0/3366528525 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557c33400950 con 0x557c334a8020 2024-09-10T10:04:39.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.342+0000 7f0b19600640 10 client.27449 handle_command_reply: tid=0 2024-09-10T10:04:39.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.342+0000 7f0b39248740 1 client.27449 shutdown 2024-09-10T10:04:39.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.342+0000 7f0b39248740 1 -- 172.21.15.77:0/3366528525 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c3348d590 msgr2=0x557c33495270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:39.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.342+0000 7f0b39248740 1 --2- 172.21.15.77:0/3366528525 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c3348d590 0x557c33495270 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c012700 tx=0x7f0b1c012b00 comp rx=0 tx=0).stop 2024-09-10T10:04:39.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.342+0000 7f0b39248740 1 -- 172.21.15.77:0/3366528525 shutdown_connections 2024-09-10T10:04:39.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.342+0000 7f0b39248740 1 --2- 172.21.15.77:0/3366528525 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x557c334a8020 0x557c334aa440 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:39.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.342+0000 7f0b39248740 1 --2- 172.21.15.77:0/3366528525 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557c33497e50 0x557c334957b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:39.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.342+0000 7f0b39248740 1 --2- 172.21.15.77:0/3366528525 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c3348d590 0x557c33495270 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:39.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.342+0000 7f0b39248740 1 --2- 172.21.15.77:0/3366528525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c3348cc70 0x557c33494d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:39.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.342+0000 7f0b19600640 0 client.27449 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T10:04:39.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.342+0000 7f0b39248740 1 -- 172.21.15.77:0/3366528525 >> 172.21.15.77:0/3366528525 conn(0x557c33489200 msgr2=0x557c33488380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:39.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.342+0000 7f0b39248740 1 -- 172.21.15.77:0/3366528525 shutdown_connections 2024-09-10T10:04:39.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.342+0000 7f0b39248740 1 -- 172.21.15.77:0/3366528525 wait complete. 2024-09-10T10:04:39.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.343+0000 7f0b39248740 20 client.27449 trim_cache size 0 max 16384 2024-09-10T10:04:39.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.358+0000 7f0b39248740 1 Processor -- start 2024-09-10T10:04:39.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.358+0000 7f0b39248740 1 -- start start 2024-09-10T10:04:39.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.359+0000 7f0b39248740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c3348cc70 0x557c334afcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:39.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.359+0000 7f0b39248740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557c3348d590 0x557c334b0230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:39.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.359+0000 7f0b39248740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c33497e50 0x557c33491f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:39.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.359+0000 7f0b39248740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0b301b1080 con 0x557c3348d590 2024-09-10T10:04:39.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.359+0000 7f0b39248740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0b301b2cb0 con 0x557c3348cc70 2024-09-10T10:04:39.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.359+0000 7f0b39248740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0b301b17b0 con 0x557c33497e50 2024-09-10T10:04:39.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.359+0000 7f0b2ea00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557c3348d590 0x557c334b0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:39.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.359+0000 7f0b2ea00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557c3348d590 0x557c334b0230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60760/0 (socket says 172.21.15.77:60760) 2024-09-10T10:04:39.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.359+0000 7f0b2ea00640 1 -- 172.21.15.77:0/1866613551 learned_addr learned my addr 172.21.15.77:0/1866613551 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:04:39.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.359+0000 7f0b2fe00640 1 --2- 172.21.15.77:0/1866613551 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c3348cc70 0x557c334afcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:39.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.359+0000 7f0b2fe00640 1 --2- 172.21.15.77:0/1866613551 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c33497e50 0x557c33491f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:39.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.359+0000 7f0b2fe00640 1 -- 172.21.15.77:0/1866613551 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c33497e50 msgr2=0x557c33491f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:39.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.359+0000 7f0b2fe00640 1 --2- 172.21.15.77:0/1866613551 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c33497e50 0x557c33491f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:39.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.359+0000 7f0b2fe00640 1 -- 172.21.15.77:0/1866613551 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557c3348d590 msgr2=0x557c334b0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:39.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.359+0000 7f0b2fe00640 1 --2- 172.21.15.77:0/1866613551 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557c3348d590 0x557c334b0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:39.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.360+0000 7f0b2fe00640 1 -- 172.21.15.77:0/1866613551 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c33484770 con 0x557c3348cc70 2024-09-10T10:04:39.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.360+0000 7f0b2fe00640 1 --2- 172.21.15.77:0/1866613551 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c3348cc70 0x557c334afcf0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f0b28009250 tx=0x7f0b28009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c009c2338135d2dc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:39.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.360+0000 7f0b1be00640 1 -- 172.21.15.77:0/1866613551 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b28004050 con 0x557c3348cc70 2024-09-10T10:04:39.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.360+0000 7f0b1be00640 1 -- 172.21.15.77:0/1866613551 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b28027440 con 0x557c3348cc70 2024-09-10T10:04:39.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.360+0000 7f0b1be00640 1 -- 172.21.15.77:0/1866613551 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b2800f6e0 con 0x557c3348cc70 2024-09-10T10:04:39.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.361+0000 7f0b39248740 1 -- 172.21.15.77:0/1866613551 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c3348cc70 msgr2=0x557c334afcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:39.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.361+0000 7f0b39248740 1 --2- 172.21.15.77:0/1866613551 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c3348cc70 0x557c334afcf0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f0b28009250 tx=0x7f0b28009500 comp rx=0 tx=0).stop 2024-09-10T10:04:39.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.361+0000 7f0b39248740 1 -- 172.21.15.77:0/1866613551 shutdown_connections 2024-09-10T10:04:39.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.361+0000 7f0b39248740 1 --2- 172.21.15.77:0/1866613551 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c33497e50 0x557c33491f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:39.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.361+0000 7f0b39248740 1 --2- 172.21.15.77:0/1866613551 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557c3348d590 0x557c334b0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:39.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.361+0000 7f0b39248740 1 --2- 172.21.15.77:0/1866613551 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c3348cc70 0x557c334afcf0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:39.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.361+0000 7f0b39248740 1 -- 172.21.15.77:0/1866613551 >> 172.21.15.77:0/1866613551 conn(0x557c33489200 msgr2=0x557c33496db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:39.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.361+0000 7f0b39248740 1 -- 172.21.15.77:0/1866613551 shutdown_connections 2024-09-10T10:04:39.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.361+0000 7f0b39248740 1 -- 172.21.15.77:0/1866613551 wait complete. 2024-09-10T10:04:39.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.362+0000 7f0b39248740 1 -- start start 2024-09-10T10:04:39.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.362+0000 7f0b39248740 1 Processor -- start 2024-09-10T10:04:39.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.362+0000 7f0b39248740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c3348cc70 0x557c3349c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:39.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.362+0000 7f0b39248740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557c3348d590 0x557c3349c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:39.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.362+0000 7f0b39248740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c33497e50 0x557c33511cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:39.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.362+0000 7f0b39248740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0b301b25e0 con 0x557c3348d590 2024-09-10T10:04:39.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.362+0000 7f0b39248740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0b301b1eb0 con 0x557c33497e50 2024-09-10T10:04:39.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.362+0000 7f0b39248740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0b301af060 con 0x557c3348cc70 2024-09-10T10:04:39.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.362+0000 7f0b39248740 10 client.0 _ll_register_callbacks cb 0x7f0b1c009860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:04:39.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.362+0000 7f0b2ea00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557c3348d590 0x557c3349c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:39.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.363+0000 7f0b2ea00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557c3348d590 0x557c3349c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60776/0 (socket says 172.21.15.77:60776) 2024-09-10T10:04:39.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.363+0000 7f0b2fe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c3348cc70 0x557c3349c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:39.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.363+0000 7f0b2ea00640 1 -- 172.21.15.77:0/178967328 learned_addr learned my addr 172.21.15.77:0/178967328 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:04:39.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.363+0000 7f0b2fe00640 1 --2- 172.21.15.77:0/178967328 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c33497e50 0x557c33511cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:39.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.363+0000 7f0b2ea00640 1 -- 172.21.15.77:0/178967328 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c3348cc70 msgr2=0x557c3349c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:39.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.363+0000 7f0b2ea00640 1 --2- 172.21.15.77:0/178967328 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c3348cc70 0x557c3349c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:39.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.363+0000 7f0b2ea00640 1 -- 172.21.15.77:0/178967328 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c33497e50 msgr2=0x557c33511cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:39.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.363+0000 7f0b2ea00640 1 --2- 172.21.15.77:0/178967328 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c33497e50 0x557c33511cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:39.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.363+0000 7f0b2ea00640 1 -- 172.21.15.77:0/178967328 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c33512200 con 0x557c3348d590 2024-09-10T10:04:39.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.363+0000 7f0b2fe00640 1 --2- 172.21.15.77:0/178967328 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c33497e50 0x557c33511cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:04:39.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.363+0000 7f0b2ea00640 1 --2- 172.21.15.77:0/178967328 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557c3348d590 0x557c3349c720 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c000c00 tx=0x7f0b1c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=649647b57a244a63 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:39.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.363+0000 7f0b19600640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-10T10:04:39.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.364+0000 7f0b19600640 1 -- 172.21.15.77:0/178967328 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b1c022040 con 0x557c3348d590 2024-09-10T10:04:39.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.364+0000 7f0b39248740 1 -- 172.21.15.77:0/178967328 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b28027440 con 0x557c3348d590 2024-09-10T10:04:39.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.364+0000 7f0b19600640 1 -- 172.21.15.77:0/178967328 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f0b1c003800 con 0x557c3348d590 2024-09-10T10:04:39.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.364+0000 7f0b19600640 1 -- 172.21.15.77:0/178967328 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f0b1c03a050 con 0x557c3348d590 2024-09-10T10:04:39.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.364+0000 7f0b39248740 10 client.17289 fetch_fsmap learned FSMap version 40 2024-09-10T10:04:39.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.364+0000 7f0b39248740 1 -- 172.21.15.77:0/178967328 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x557c33513400 con 0x557c3348d590 2024-09-10T10:04:39.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.364+0000 7f0b19600640 1 -- 172.21.15.77:0/178967328 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b28027440 con 0x557c3348d590 2024-09-10T10:04:39.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.365+0000 7f0b19600640 1 -- 172.21.15.77:0/178967328 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 40) ==== 8398+0+0 (secure 0 0 0) 0x7f0b1c00fb60 con 0x557c3348d590 2024-09-10T10:04:39.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.365+0000 7f0b39248740 10 client.17289 fetch_fsmap finished waiting for FSMap version 40 2024-09-10T10:04:39.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.365+0000 7f0b39248740 10 client.17289 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T10:04:39.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.365+0000 7f0b39248740 20 client.17289 populate_metadata read hostname 'smithi077' 2024-09-10T10:04:39.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.365+0000 7f0b39248740 1 --2- 172.21.15.77:0/178967328 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x557c33513ac0 0x557c33515eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:39.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.365+0000 7f0b39248740 4 client.17289 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:04:39.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.365+0000 7f0b39248740 1 -- 172.21.15.77:0/178967328 --> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557c333fae60 con 0x557c33513ac0 2024-09-10T10:04:39.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.365+0000 7f0b2fe00640 1 --2- 172.21.15.77:0/178967328 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x557c33513ac0 0x557c33515eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:39.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.366+0000 7f0b2fe00640 1 --2- 172.21.15.77:0/178967328 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x557c33513ac0 0x557c33515eb0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b6b5f7e5ee8ba70 server_cookie=3dda25720238ebb2 in_seq=0 out_seq=0 2024-09-10T10:04:39.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.366+0000 7f0b19600640 10 client.17289 ms_handle_connect on v2:172.21.15.120:6838/1777710157 2024-09-10T10:04:39.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.366+0000 7f0b19600640 1 -- 172.21.15.77:0/178967328 <== mds.0 v2:172.21.15.120:6838/1777710157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557c333fae60 con 0x557c33513ac0 2024-09-10T10:04:39.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.366+0000 7f0b19600640 10 client.17289 handle_command_reply: tid=0 2024-09-10T10:04:39.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.366+0000 7f0b39248740 1 client.17289 shutdown 2024-09-10T10:04:39.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.366+0000 7f0b39248740 1 -- 172.21.15.77:0/178967328 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557c3348d590 msgr2=0x557c3349c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:39.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.366+0000 7f0b39248740 1 --2- 172.21.15.77:0/178967328 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557c3348d590 0x557c3349c720 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c000c00 tx=0x7f0b1c009e30 comp rx=0 tx=0).stop 2024-09-10T10:04:39.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.366+0000 7f0b39248740 1 -- 172.21.15.77:0/178967328 shutdown_connections 2024-09-10T10:04:39.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.366+0000 7f0b39248740 1 --2- 172.21.15.77:0/178967328 >> [v2:172.21.15.120:6838/1777710157,v1:172.21.15.120:6839/1777710157] conn(0x557c33513ac0 0x557c33515eb0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:39.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.366+0000 7f0b39248740 1 --2- 172.21.15.77:0/178967328 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c33497e50 0x557c33511cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:39.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.366+0000 7f0b19600640 0 client.17289 ms_handle_reset on v2:172.21.15.120:6838/1777710157 2024-09-10T10:04:39.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.366+0000 7f0b39248740 1 --2- 172.21.15.77:0/178967328 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557c3348d590 0x557c3349c720 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:39.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.366+0000 7f0b39248740 1 --2- 172.21.15.77:0/178967328 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c3348cc70 0x557c3349c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:39.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.366+0000 7f0b39248740 1 -- 172.21.15.77:0/178967328 >> 172.21.15.77:0/178967328 conn(0x557c33489200 msgr2=0x557c33487b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:39.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.367+0000 7f0b39248740 1 -- 172.21.15.77:0/178967328 shutdown_connections 2024-09-10T10:04:39.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.367+0000 7f0b39248740 1 -- 172.21.15.77:0/178967328 wait complete. 2024-09-10T10:04:39.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.367+0000 7f0b39248740 20 client.17289 trim_cache size 0 max 16384 2024-09-10T10:04:39.375 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-10T10:04:39.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.367+0000 7f0b36600640 1 -- 172.21.15.77:0/3873389812 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f0b140a2c00 msgr2=0x7f0b140a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:39.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.367+0000 7f0b36600640 1 --2- 172.21.15.77:0/3873389812 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f0b140a2c00 0x7f0b140a50c0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f0b200b85b0 tx=0x7f0b200f5110 comp rx=0 tx=0).stop 2024-09-10T10:04:39.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.367+0000 7f0b36600640 1 -- 172.21.15.77:0/3873389812 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0b30101de0 msgr2=0x7f0b3019a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:39.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.367+0000 7f0b36600640 1 --2- 172.21.15.77:0/3873389812 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0b30101de0 0x7f0b3019a670 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f0b20004770 tx=0x7f0b20003c10 comp rx=0 tx=0).stop 2024-09-10T10:04:39.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.367+0000 7f0b36600640 1 -- 172.21.15.77:0/3873389812 shutdown_connections 2024-09-10T10:04:39.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.367+0000 7f0b36600640 1 --2- 172.21.15.77:0/3873389812 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f0b140a2c00 0x7f0b140a50c0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:39.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.367+0000 7f0b36600640 1 --2- 172.21.15.77:0/3873389812 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b30106f10 0x7f0b3019ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:39.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.367+0000 7f0b36600640 1 --2- 172.21.15.77:0/3873389812 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0b30102790 0x7f0b3019abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:39.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.367+0000 7f0b36600640 1 --2- 172.21.15.77:0/3873389812 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0b30101de0 0x7f0b3019a670 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:39.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.367+0000 7f0b36600640 1 -- 172.21.15.77:0/3873389812 >> 172.21.15.77:0/3873389812 conn(0x7f0b300fd8c0 msgr2=0x7f0b300ffc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:39.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.368+0000 7f0b36600640 1 -- 172.21.15.77:0/3873389812 shutdown_connections 2024-09-10T10:04:39.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:39.368+0000 7f0b36600640 1 -- 172.21.15.77:0/3873389812 wait complete. 2024-09-10T10:04:39.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:04:38 smithi134 ceph-mon[35674]: pgmap v675: 129 pgs: 129 active+clean; 115 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 321 KiB/s wr, 106 op/s 2024-09-10T10:04:39.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:04:38 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T10:04:39.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:04:38 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T10:04:39.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:04:38 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T10:04:39.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:04:38 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T10:04:39.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:04:38 smithi120 ceph-mon[34770]: pgmap v675: 129 pgs: 129 active+clean; 115 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 321 KiB/s wr, 106 op/s 2024-09-10T10:04:39.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:04:38 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T10:04:39.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:04:38 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T10:04:39.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:04:38 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T10:04:39.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:04:38 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T10:04:40.289 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa/tasks/thrasher.py", line 47, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-09-10T10:04:40.290 INFO:tasks.fwd_scrub:done joining 2024-09-10T10:04:40.290 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-09-10T10:04:40.300 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:04:40.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:40.472+0000 7fd78d400640 1 -- 172.21.15.77:0/2038968248 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd788104d10 msgr2=0x7fd788107100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:40.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:40.472+0000 7fd78d400640 1 --2- 172.21.15.77:0/2038968248 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd788104d10 0x7fd788107100 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fd774009900 tx=0x7fd77402a410 comp rx=0 tx=0).stop 2024-09-10T10:04:40.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:40.472+0000 7fd78d400640 1 -- 172.21.15.77:0/2038968248 shutdown_connections 2024-09-10T10:04:40.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:40.472+0000 7fd78d400640 1 --2- 172.21.15.77:0/2038968248 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd788107640 0x7fd788109ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:40.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:40.472+0000 7fd78d400640 1 --2- 172.21.15.77:0/2038968248 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd788104d10 0x7fd788107100 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:40.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:40.472+0000 7fd78d400640 1 --2- 172.21.15.77:0/2038968248 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd7881023e0 0x7fd7881047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:40.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:40.472+0000 7fd78d400640 1 -- 172.21.15.77:0/2038968248 >> 172.21.15.77:0/2038968248 conn(0x7fd7880fc000 msgr2=0x7fd7880fe420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:40.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:40.472+0000 7fd78d400640 1 -- 172.21.15.77:0/2038968248 shutdown_connections 2024-09-10T10:04:40.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:40.472+0000 7fd78d400640 1 -- 172.21.15.77:0/2038968248 wait complete. 2024-09-10T10:04:40.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:40.473+0000 7fd78d400640 1 Processor -- start 2024-09-10T10:04:40.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:40.474+0000 7fd78d400640 1 -- start start 2024-09-10T10:04:40.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:40.474+0000 7fd78d400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd7881023e0 0x7fd788197d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:40.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:40.474+0000 7fd78d400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd788104d10 0x7fd788198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:40.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:40.474+0000 7fd78d400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd788107640 0x7fd78819f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:40.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:40.474+0000 7fd78d400640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd78810d640 con 0x7fd7881023e0 2024-09-10T10:04:40.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:40.474+0000 7fd78d400640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd78810d4c0 con 0x7fd788104d10 2024-09-10T10:04:40.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:40.474+0000 7fd78d400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd78810d7c0 con 0x7fd788107640 2024-09-10T10:04:40.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:40.474+0000 7fd787e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd7881023e0 0x7fd788197d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:40.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:40.474+0000 7fd787e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd7881023e0 0x7fd788197d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60794/0 (socket says 172.21.15.77:60794) 2024-09-10T10:04:40.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:40.474+0000 7fd787e00640 1 -- 172.21.15.77:0/3573034558 learned_addr learned my addr 172.21.15.77:0/3573034558 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:04:40.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:40.474+0000 7fd787400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd788104d10 0x7fd788198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:40.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:40.474+0000 7fd78ca00640 1 --2- 172.21.15.77:0/3573034558 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd788107640 0x7fd78819f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:40.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:40.475+0000 7fd787e00640 1 -- 172.21.15.77:0/3573034558 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd788107640 msgr2=0x7fd78819f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:40.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:40.475+0000 7fd787e00640 1 --2- 172.21.15.77:0/3573034558 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd788107640 0x7fd78819f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:40.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:40.475+0000 7fd787e00640 1 -- 172.21.15.77:0/3573034558 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd788104d10 msgr2=0x7fd788198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:40.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:40.475+0000 7fd787e00640 1 --2- 172.21.15.77:0/3573034558 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd788104d10 0x7fd788198260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:40.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:40.475+0000 7fd787e00640 1 -- 172.21.15.77:0/3573034558 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd78819f930 con 0x7fd7881023e0 2024-09-10T10:04:40.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:40.475+0000 7fd787400640 1 --2- 172.21.15.77:0/3573034558 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd788104d10 0x7fd788198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:04:40.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:40.475+0000 7fd787e00640 1 --2- 172.21.15.77:0/3573034558 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd7881023e0 0x7fd788197d20 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fd77800d950 tx=0x7fd77800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:40.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:40.475+0000 7fd784c00640 1 -- 172.21.15.77:0/3573034558 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd778017030 con 0x7fd7881023e0 2024-09-10T10:04:40.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:40.476+0000 7fd78d400640 1 -- 172.21.15.77:0/3573034558 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd78819fc20 con 0x7fd7881023e0 2024-09-10T10:04:40.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:40.476+0000 7fd784c00640 1 -- 172.21.15.77:0/3573034558 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd778004020 con 0x7fd7881023e0 2024-09-10T10:04:40.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:40.476+0000 7fd784c00640 1 -- 172.21.15.77:0/3573034558 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7780047a0 con 0x7fd7881023e0 2024-09-10T10:04:40.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:40.476+0000 7fd78d400640 1 -- 172.21.15.77:0/3573034558 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7881a01c0 con 0x7fd7881023e0 2024-09-10T10:04:40.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:40.477+0000 7fd78d400640 1 -- 172.21.15.77:0/3573034558 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7880fdaf0 con 0x7fd7881023e0 2024-09-10T10:04:40.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:40.480+0000 7fd784c00640 1 -- 172.21.15.77:0/3573034558 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd778009a30 con 0x7fd7881023e0 2024-09-10T10:04:40.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:40.482+0000 7fd784c00640 1 --2- 172.21.15.77:0/3573034558 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fd76c0a2b90 0x7fd76c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:40.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:40.482+0000 7fd784c00640 1 -- 172.21.15.77:0/3573034558 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fd7780ceef0 con 0x7fd7881023e0 2024-09-10T10:04:40.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:40.482+0000 7fd787400640 1 --2- 172.21.15.77:0/3573034558 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fd76c0a2b90 0x7fd76c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:40.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:40.483+0000 7fd787400640 1 --2- 172.21.15.77:0/3573034558 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fd76c0a2b90 0x7fd76c0a5050 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fd77402aa00 tx=0x7fd7740315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:40.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:40.486+0000 7fd784c00640 1 -- 172.21.15.77:0/3573034558 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd77809ba00 con 0x7fd7881023e0 2024-09-10T10:04:40.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:40.687+0000 7fd78d400640 1 -- 172.21.15.77:0/3573034558 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd788062e90 con 0x7fd7881023e0 2024-09-10T10:04:40.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:40.690+0000 7fd784c00640 1 -- 172.21.15.77:0/3573034558 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+24941 (secure 0 0 0) 0x7fd77809b300 con 0x7fd7881023e0 2024-09-10T10:04:40.692 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T10:04:40.692 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":96,"fsid":"14a55226-6f59-11ef-bcea-c7b262605968","created":"2024-09-10T09:44:07.359542+0000","modified":"2024-09-10T09:54:00.837074+0000","last_up_change":"2024-09-10T09:51:26.731971+0000","last_in_change":"2024-09-10T09:51:03.857873+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:47:24.174660+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T09:52:18.766738+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T09:52:21.284745+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T09:52:29.846594+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8d46f634-114c-4926-afdc-f8fa37e40336","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":1509649079},{"type":"v1","addr":"172.21.15.77:6803","nonce":1509649079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":1509649079},{"type":"v1","addr":"172.21.15.77:6805","nonce":1509649079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":1509649079},{"type":"v1","addr":"172.21.15.77:6809","nonce":1509649079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":1509649079},{"type":"v1","addr":"172.21.15.77:6807","nonce":1509649079}]},"public_addr":"172.21.15.77:6803/1509649079","cluster_addr":"172.21.15.77:6805/1509649079","heartbeat_back_addr":"172.21.15.77:6809/1509649079","heartbeat_front_addr":"172.21.15.77:6807/1509649079","state":["exists","up"]},{"osd":1,"uuid":"b6570764-736a-49e4-877a-1965e5812d1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":4243920698},{"type":"v1","addr":"172.21.15.120:6801","nonce":4243920698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":4243920698},{"type":"v1","addr":"172.21.15.120:6803","nonce":4243920698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":4243920698},{"type":"v1","addr":"172.21.15.120:6807","nonce":4243920698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":4243920698},{"type":"v1","addr":"172.21.15.120:6805","nonce":4243920698}]},"public_addr":"172.21.15.120:6801/4243920698","cluster_addr":"172.21.15.120:6803/4243920698","heartbeat_back_addr":"172.21.15.120:6807/4243920698","heartbeat_front_addr":"172.21.15.120:6805/4243920698","state":["exists","up"]},{"osd":2,"uuid":"5e83fe13-d1ee-483e-a575-e8041ec836f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":3280466823},{"type":"v1","addr":"172.21.15.134:6801","nonce":3280466823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":3280466823},{"type":"v1","addr":"172.21.15.134:6803","nonce":3280466823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":3280466823},{"type":"v1","addr":"172.21.15.134:6807","nonce":3280466823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":3280466823},{"type":"v1","addr":"172.21.15.134:6805","nonce":3280466823}]},"public_addr":"172.21.15.134:6801/3280466823","cluster_addr":"172.21.15.134:6803/3280466823","heartbeat_back_addr":"172.21.15.134:6807/3280466823","heartbeat_front_addr":"172.21.15.134:6805/3280466823","state":["exists","up"]},{"osd":3,"uuid":"5affc6a7-1cae-48fe-b7db-befc2fcafecb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":1629416786},{"type":"v1","addr":"172.21.15.77:6811","nonce":1629416786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":1629416786},{"type":"v1","addr":"172.21.15.77:6813","nonce":1629416786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":1629416786},{"type":"v1","addr":"172.21.15.77:6817","nonce":1629416786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":1629416786},{"type":"v1","addr":"172.21.15.77:6815","nonce":1629416786}]},"public_addr":"172.21.15.77:6811/1629416786","cluster_addr":"172.21.15.77:6813/1629416786","heartbeat_back_addr":"172.21.15.77:6817/1629416786","heartbeat_front_addr":"172.21.15.77:6815/1629416786","state":["exists","up"]},{"osd":4,"uuid":"c826978f-40b0-43ae-adbb-5c93b32a68ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":3162658038},{"type":"v1","addr":"172.21.15.120:6809","nonce":3162658038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":3162658038},{"type":"v1","addr":"172.21.15.120:6811","nonce":3162658038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":3162658038},{"type":"v1","addr":"172.21.15.120:6815","nonce":3162658038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":3162658038},{"type":"v1","addr":"172.21.15.120:6813","nonce":3162658038}]},"public_addr":"172.21.15.120:6809/3162658038","cluster_addr":"172.21.15.120:6811/3162658038","heartbeat_back_addr":"172.21.15.120:6815/3162658038","heartbeat_front_addr":"172.21.15.120:6813/3162658038","state":["exists","up"]},{"osd":5,"uuid":"be51c7fb-2765-46a3-9861-c950f63de148","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":4081444621},{"type":"v1","addr":"172.21.15.134:6809","nonce":4081444621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":4081444621},{"type":"v1","addr":"172.21.15.134:6811","nonce":4081444621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":4081444621},{"type":"v1","addr":"172.21.15.134:6815","nonce":4081444621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":4081444621},{"type":"v1","addr":"172.21.15.134:6813","nonce":4081444621}]},"public_addr":"172.21.15.134:6809/4081444621","cluster_addr":"172.21.15.134:6811/4081444621","heartbeat_back_addr":"172.21.15.134:6815/4081444621","heartbeat_front_addr":"172.21.15.134:6813/4081444621","state":["exists","up"]},{"osd":6,"uuid":"4eb403ab-473b-41cc-803d-18c908f768a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":879565639},{"type":"v1","addr":"172.21.15.77:6819","nonce":879565639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":879565639},{"type":"v1","addr":"172.21.15.77:6821","nonce":879565639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":879565639},{"type":"v1","addr":"172.21.15.77:6825","nonce":879565639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":879565639},{"type":"v1","addr":"172.21.15.77:6823","nonce":879565639}]},"public_addr":"172.21.15.77:6819/879565639","cluster_addr":"172.21.15.77:6821/879565639","heartbeat_back_addr":"172.21.15.77:6825/879565639","heartbeat_front_addr":"172.21.15.77:6823/879565639","state":["exists","up"]},{"osd":7,"uuid":"ba982bd0-c236-456c-9889-61087d79c1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":2688667400},{"type":"v1","addr":"172.21.15.120:6817","nonce":2688667400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":2688667400},{"type":"v1","addr":"172.21.15.120:6819","nonce":2688667400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":2688667400},{"type":"v1","addr":"172.21.15.120:6823","nonce":2688667400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2688667400},{"type":"v1","addr":"172.21.15.120:6821","nonce":2688667400}]},"public_addr":"172.21.15.120:6817/2688667400","cluster_addr":"172.21.15.120:6819/2688667400","heartbeat_back_addr":"172.21.15.120:6823/2688667400","heartbeat_front_addr":"172.21.15.120:6821/2688667400","state":["exists","up"]},{"osd":8,"uuid":"f2e8ca43-57c9-4933-b5f7-73f4d1b3e08b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2081686544},{"type":"v1","addr":"172.21.15.134:6817","nonce":2081686544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2081686544},{"type":"v1","addr":"172.21.15.134:6819","nonce":2081686544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2081686544},{"type":"v1","addr":"172.21.15.134:6823","nonce":2081686544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2081686544},{"type":"v1","addr":"172.21.15.134:6821","nonce":2081686544}]},"public_addr":"172.21.15.134:6817/2081686544","cluster_addr":"172.21.15.134:6819/2081686544","heartbeat_back_addr":"172.21.15.134:6823/2081686544","heartbeat_front_addr":"172.21.15.134:6821/2081686544","state":["exists","up"]},{"osd":9,"uuid":"9486f8b5-0614-4208-87ec-117741f6b622","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":1210619387},{"type":"v1","addr":"172.21.15.77:6827","nonce":1210619387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":1210619387},{"type":"v1","addr":"172.21.15.77:6829","nonce":1210619387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":1210619387},{"type":"v1","addr":"172.21.15.77:6833","nonce":1210619387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":1210619387},{"type":"v1","addr":"172.21.15.77:6831","nonce":1210619387}]},"public_addr":"172.21.15.77:6827/1210619387","cluster_addr":"172.21.15.77:6829/1210619387","heartbeat_back_addr":"172.21.15.77:6833/1210619387","heartbeat_front_addr":"172.21.15.77:6831/1210619387","state":["exists","up"]},{"osd":10,"uuid":"8e8192b0-1398-4d4d-9756-762b2f874d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":754733885},{"type":"v1","addr":"172.21.15.120:6825","nonce":754733885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":754733885},{"type":"v1","addr":"172.21.15.120:6827","nonce":754733885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":754733885},{"type":"v1","addr":"172.21.15.120:6831","nonce":754733885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":754733885},{"type":"v1","addr":"172.21.15.120:6829","nonce":754733885}]},"public_addr":"172.21.15.120:6825/754733885","cluster_addr":"172.21.15.120:6827/754733885","heartbeat_back_addr":"172.21.15.120:6831/754733885","heartbeat_front_addr":"172.21.15.120:6829/754733885","state":["exists","up"]},{"osd":11,"uuid":"85c1ccd4-5147-4995-896e-fe977f73f749","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1602213143},{"type":"v1","addr":"172.21.15.134:6825","nonce":1602213143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1602213143},{"type":"v1","addr":"172.21.15.134:6827","nonce":1602213143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1602213143},{"type":"v1","addr":"172.21.15.134:6831","nonce":1602213143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1602213143},{"type":"v1","addr":"172.21.15.134:6829","nonce":1602213143}]},"public_addr":"172.21.15.134:6825/1602213143","cluster_addr":"172.21.15.134:6827/1602213143","heartbeat_back_addr":"172.21.15.134:6831/1602213143","heartbeat_front_addr":"172.21.15.134:6829/1602213143","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:46:30.293451+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:46:56.554801+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:47:21.476225+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:47:49.737332+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:48:16.092181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:48:41.240815+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:49:11.683950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:49:38.859255+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:50:04.105401+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:50:33.847810+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:50:59.901797+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:51:25.241507+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.2","mappings":[{"from":7,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":7}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":5,"to":8}]},{"pgid":"4.6","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.77:0/1697982655":"2024-09-11T09:44:59.575663+0000","172.21.15.77:6801/1527070442":"2024-09-11T09:44:59.575663+0000","172.21.15.77:0/3146688196":"2024-09-11T09:44:32.185342+0000","172.21.15.77:0/371773177":"2024-09-11T09:44:59.575663+0000","172.21.15.77:0/1925456910":"2024-09-11T09:44:59.575663+0000","172.21.15.77:0/138545832":"2024-09-11T09:44:32.185342+0000","172.21.15.77:6801/3235900503":"2024-09-11T09:44:32.185342+0000","172.21.15.77:6800/1527070442":"2024-09-11T09:44:59.575663+0000","172.21.15.77:6800/3235900503":"2024-09-11T09:44:32.185342+0000","172.21.15.77:0/1647092548":"2024-09-11T09:44:32.185342+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:04:40.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:40.692+0000 7fd78d400640 1 -- 172.21.15.77:0/3573034558 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fd76c0a2b90 msgr2=0x7fd76c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:40.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:40.692+0000 7fd78d400640 1 --2- 172.21.15.77:0/3573034558 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fd76c0a2b90 0x7fd76c0a5050 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fd77402aa00 tx=0x7fd7740315b0 comp rx=0 tx=0).stop 2024-09-10T10:04:40.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:40.692+0000 7fd78d400640 1 -- 172.21.15.77:0/3573034558 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd7881023e0 msgr2=0x7fd788197d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:40.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:40.692+0000 7fd78d400640 1 --2- 172.21.15.77:0/3573034558 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd7881023e0 0x7fd788197d20 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fd77800d950 tx=0x7fd77800de10 comp rx=0 tx=0).stop 2024-09-10T10:04:40.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:40.692+0000 7fd78d400640 1 -- 172.21.15.77:0/3573034558 shutdown_connections 2024-09-10T10:04:40.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:40.692+0000 7fd78d400640 1 --2- 172.21.15.77:0/3573034558 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fd76c0a2b90 0x7fd76c0a5050 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:40.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:40.692+0000 7fd78d400640 1 --2- 172.21.15.77:0/3573034558 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd788107640 0x7fd78819f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:40.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:40.692+0000 7fd78d400640 1 --2- 172.21.15.77:0/3573034558 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd788104d10 0x7fd788198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:40.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:40.692+0000 7fd78d400640 1 --2- 172.21.15.77:0/3573034558 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd7881023e0 0x7fd788197d20 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:40.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:40.692+0000 7fd78d400640 1 -- 172.21.15.77:0/3573034558 >> 172.21.15.77:0/3573034558 conn(0x7fd7880fc000 msgr2=0x7fd788102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:40.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:40.692+0000 7fd78d400640 1 -- 172.21.15.77:0/3573034558 shutdown_connections 2024-09-10T10:04:40.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:40.692+0000 7fd78d400640 1 -- 172.21.15.77:0/3573034558 wait complete. 2024-09-10T10:04:40.711 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T09:47:24.174660+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-10T09:52:18.766738+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-10T09:52:21.284745+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-10T09:52:29.846594+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '92', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-10T10:04:40.712 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T10:04:40.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:40.873+0000 7fa167e00640 1 -- 172.21.15.77:0/2599015959 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa1681035b0 msgr2=0x7fa168109e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:40.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:40.873+0000 7fa167e00640 1 --2- 172.21.15.77:0/2599015959 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa1681035b0 0x7fa168109e40 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fa160009900 tx=0x7fa16002a3f0 comp rx=0 tx=0).stop 2024-09-10T10:04:40.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:40.874+0000 7fa167e00640 1 -- 172.21.15.77:0/2599015959 shutdown_connections 2024-09-10T10:04:40.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:40.874+0000 7fa167e00640 1 --2- 172.21.15.77:0/2599015959 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa1681035b0 0x7fa168109e40 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:40.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:40.874+0000 7fa167e00640 1 --2- 172.21.15.77:0/2599015959 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa168102bf0 0x7fa168103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:40.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:40.874+0000 7fa167e00640 1 --2- 172.21.15.77:0/2599015959 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa168101a00 0x7fa168101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:40.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:40.874+0000 7fa167e00640 1 -- 172.21.15.77:0/2599015959 >> 172.21.15.77:0/2599015959 conn(0x7fa1680fd180 msgr2=0x7fa1680ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:40.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:40.874+0000 7fa167e00640 1 -- 172.21.15.77:0/2599015959 shutdown_connections 2024-09-10T10:04:40.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:40.874+0000 7fa167e00640 1 -- 172.21.15.77:0/2599015959 wait complete. 2024-09-10T10:04:40.878 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:40.875+0000 7fa167e00640 1 Processor -- start 2024-09-10T10:04:40.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:40.875+0000 7fa167e00640 1 -- start start 2024-09-10T10:04:40.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:40.875+0000 7fa167e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa168101a00 0x7fa168197ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:40.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:40.875+0000 7fa167e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa168102bf0 0x7fa1681981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:40.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:40.875+0000 7fa167e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa1681035b0 0x7fa16819f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:40.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:40.875+0000 7fa167e00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa16810d6d0 con 0x7fa168102bf0 2024-09-10T10:04:40.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:40.875+0000 7fa167e00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa16810d550 con 0x7fa1681035b0 2024-09-10T10:04:40.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:40.875+0000 7fa167e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa16810d850 con 0x7fa168101a00 2024-09-10T10:04:40.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:40.875+0000 7fa15e000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa168102bf0 0x7fa1681981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:40.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:40.875+0000 7fa15e000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa168102bf0 0x7fa1681981e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60804/0 (socket says 172.21.15.77:60804) 2024-09-10T10:04:40.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:40.875+0000 7fa15e000640 1 -- 172.21.15.77:0/1864591244 learned_addr learned my addr 172.21.15.77:0/1864591244 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:04:40.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:40.875+0000 7fa166a00640 1 --2- 172.21.15.77:0/1864591244 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa168101a00 0x7fa168197ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:40.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:40.876+0000 7fa167400640 1 --2- 172.21.15.77:0/1864591244 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa1681035b0 0x7fa16819f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:40.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:40.876+0000 7fa15e000640 1 -- 172.21.15.77:0/1864591244 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa168101a00 msgr2=0x7fa168197ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:40.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:40.876+0000 7fa15e000640 1 --2- 172.21.15.77:0/1864591244 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa168101a00 0x7fa168197ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:40.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:40.876+0000 7fa15e000640 1 -- 172.21.15.77:0/1864591244 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa1681035b0 msgr2=0x7fa16819f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:40.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:40.876+0000 7fa15e000640 1 --2- 172.21.15.77:0/1864591244 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa1681035b0 0x7fa16819f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:40.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:40.876+0000 7fa15e000640 1 -- 172.21.15.77:0/1864591244 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa16819f8e0 con 0x7fa168102bf0 2024-09-10T10:04:40.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:40.876+0000 7fa167400640 1 --2- 172.21.15.77:0/1864591244 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa1681035b0 0x7fa16819f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:04:40.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:40.876+0000 7fa15e000640 1 --2- 172.21.15.77:0/1864591244 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa168102bf0 0x7fa1681981e0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fa15000e9f0 tx=0x7fa15000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:40.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:40.876+0000 7fa15fe00640 1 -- 172.21.15.77:0/1864591244 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa15000cd80 con 0x7fa168102bf0 2024-09-10T10:04:40.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:40.876+0000 7fa15fe00640 1 -- 172.21.15.77:0/1864591244 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa150010910 con 0x7fa168102bf0 2024-09-10T10:04:40.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:40.876+0000 7fa167e00640 1 -- 172.21.15.77:0/1864591244 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa16819fbd0 con 0x7fa168102bf0 2024-09-10T10:04:40.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:40.876+0000 7fa15fe00640 1 -- 172.21.15.77:0/1864591244 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa150018990 con 0x7fa168102bf0 2024-09-10T10:04:40.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:40.877+0000 7fa167e00640 1 -- 172.21.15.77:0/1864591244 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1681a0120 con 0x7fa168102bf0 2024-09-10T10:04:40.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:40.878+0000 7fa167e00640 1 -- 172.21.15.77:0/1864591244 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa168108da0 con 0x7fa168102bf0 2024-09-10T10:04:40.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:40.880+0000 7fa15fe00640 1 -- 172.21.15.77:0/1864591244 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa150010ac0 con 0x7fa168102bf0 2024-09-10T10:04:40.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:40.881+0000 7fa15fe00640 1 --2- 172.21.15.77:0/1864591244 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fa14c0a2b90 0x7fa14c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:40.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:40.882+0000 7fa15fe00640 1 -- 172.21.15.77:0/1864591244 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fa1500cf830 con 0x7fa168102bf0 2024-09-10T10:04:40.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:40.882+0000 7fa166a00640 1 --2- 172.21.15.77:0/1864591244 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fa14c0a2b90 0x7fa14c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:40.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:40.882+0000 7fa166a00640 1 --2- 172.21.15.77:0/1864591244 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fa14c0a2b90 0x7fa14c0a5050 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fa168198b70 tx=0x7fa158009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:40.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:40.885+0000 7fa15fe00640 1 -- 172.21.15.77:0/1864591244 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa15009c340 con 0x7fa168102bf0 2024-09-10T10:04:41.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.068+0000 7fa167e00640 1 -- 172.21.15.77:0/1864591244 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa168062df0 con 0x7fa168102bf0 2024-09-10T10:04:41.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.069+0000 7fa15fe00640 1 -- 172.21.15.77:0/1864591244 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v96) ==== 93+0+10 (secure 0 0 0) 0x7fa15009bc40 con 0x7fa168102bf0 2024-09-10T10:04:41.071 INFO:teuthology.orchestra.run.smithi077.stdout:pg_num: 1 2024-09-10T10:04:41.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.071+0000 7fa167e00640 1 -- 172.21.15.77:0/1864591244 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fa14c0a2b90 msgr2=0x7fa14c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:41.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.071+0000 7fa167e00640 1 --2- 172.21.15.77:0/1864591244 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fa14c0a2b90 0x7fa14c0a5050 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fa168198b70 tx=0x7fa158009500 comp rx=0 tx=0).stop 2024-09-10T10:04:41.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.071+0000 7fa167e00640 1 -- 172.21.15.77:0/1864591244 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa168102bf0 msgr2=0x7fa1681981e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:41.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.071+0000 7fa167e00640 1 --2- 172.21.15.77:0/1864591244 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa168102bf0 0x7fa1681981e0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fa15000e9f0 tx=0x7fa15000eeb0 comp rx=0 tx=0).stop 2024-09-10T10:04:41.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.071+0000 7fa167e00640 1 -- 172.21.15.77:0/1864591244 shutdown_connections 2024-09-10T10:04:41.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.071+0000 7fa167e00640 1 --2- 172.21.15.77:0/1864591244 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fa14c0a2b90 0x7fa14c0a5050 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:41.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.071+0000 7fa167e00640 1 --2- 172.21.15.77:0/1864591244 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa1681035b0 0x7fa16819f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:41.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.071+0000 7fa167e00640 1 --2- 172.21.15.77:0/1864591244 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa168102bf0 0x7fa1681981e0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:41.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.071+0000 7fa167e00640 1 --2- 172.21.15.77:0/1864591244 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa168101a00 0x7fa168197ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:41.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.071+0000 7fa167e00640 1 -- 172.21.15.77:0/1864591244 >> 172.21.15.77:0/1864591244 conn(0x7fa1680fd180 msgr2=0x7fa168105f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:41.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.072+0000 7fa167e00640 1 -- 172.21.15.77:0/1864591244 shutdown_connections 2024-09-10T10:04:41.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.072+0000 7fa167e00640 1 -- 172.21.15.77:0/1864591244 wait complete. 2024-09-10T10:04:41.090 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-10T10:04:41.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.275+0000 7f387de00640 1 -- 172.21.15.77:0/2360876192 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f387810e100 msgr2=0x7f38781104f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:41.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.275+0000 7f387de00640 1 --2- 172.21.15.77:0/2360876192 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f387810e100 0x7f38781104f0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f3870009be0 tx=0x7f387002a6d0 comp rx=0 tx=0).stop 2024-09-10T10:04:41.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.275+0000 7f387de00640 1 -- 172.21.15.77:0/2360876192 shutdown_connections 2024-09-10T10:04:41.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.275+0000 7f387de00640 1 --2- 172.21.15.77:0/2360876192 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f387810e100 0x7f38781104f0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:41.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.275+0000 7f387de00640 1 --2- 172.21.15.77:0/2360876192 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3878074c50 0x7f38780750b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:41.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.275+0000 7f387de00640 1 --2- 172.21.15.77:0/2360876192 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3878076230 0x7f3878074680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:41.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.275+0000 7f387de00640 1 -- 172.21.15.77:0/2360876192 >> 172.21.15.77:0/2360876192 conn(0x7f38780fd2b0 msgr2=0x7f38780ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:41.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.276+0000 7f387de00640 1 -- 172.21.15.77:0/2360876192 shutdown_connections 2024-09-10T10:04:41.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.276+0000 7f387de00640 1 -- 172.21.15.77:0/2360876192 wait complete. 2024-09-10T10:04:41.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.277+0000 7f387de00640 1 Processor -- start 2024-09-10T10:04:41.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.277+0000 7f387de00640 1 -- start start 2024-09-10T10:04:41.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.277+0000 7f387de00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3878074c50 0x7f387819e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:41.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.278+0000 7f387de00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3878076230 0x7f387819e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:41.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.278+0000 7f387de00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f387810e100 0x7f38781a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:41.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.278+0000 7f387de00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3878113d70 con 0x7f387810e100 2024-09-10T10:04:41.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.278+0000 7f387de00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3878113bf0 con 0x7f3878074c50 2024-09-10T10:04:41.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.278+0000 7f387de00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3878113ef0 con 0x7f3878076230 2024-09-10T10:04:41.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.278+0000 7f3876a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3878074c50 0x7f387819e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:41.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.278+0000 7f3877400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f387810e100 0x7f38781a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:41.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.278+0000 7f3876a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3878074c50 0x7f387819e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:36978/0 (socket says 172.21.15.77:36978) 2024-09-10T10:04:41.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.278+0000 7f3876a00640 1 -- 172.21.15.77:0/2331523628 learned_addr learned my addr 172.21.15.77:0/2331523628 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:04:41.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.278+0000 7f3876000640 1 --2- 172.21.15.77:0/2331523628 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3878076230 0x7f387819e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:41.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.278+0000 7f3876a00640 1 -- 172.21.15.77:0/2331523628 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3878076230 msgr2=0x7f387819e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:41.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.278+0000 7f3876a00640 1 --2- 172.21.15.77:0/2331523628 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3878076230 0x7f387819e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:41.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.278+0000 7f3876a00640 1 -- 172.21.15.77:0/2331523628 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f387810e100 msgr2=0x7f38781a59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:41.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.278+0000 7f3876a00640 1 --2- 172.21.15.77:0/2331523628 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f387810e100 0x7f38781a59a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:41.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.279+0000 7f3876a00640 1 -- 172.21.15.77:0/2331523628 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38781a6010 con 0x7f3878074c50 2024-09-10T10:04:41.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.279+0000 7f3876a00640 1 --2- 172.21.15.77:0/2331523628 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3878074c50 0x7f387819e3f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f386800e9f0 tx=0x7f386800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:41.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.279+0000 7f387ca00640 1 -- 172.21.15.77:0/2331523628 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f386800cd80 con 0x7f3878074c50 2024-09-10T10:04:41.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.279+0000 7f387de00640 1 -- 172.21.15.77:0/2331523628 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38781a6300 con 0x7f3878074c50 2024-09-10T10:04:41.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.279+0000 7f387ca00640 1 -- 172.21.15.77:0/2331523628 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3868010910 con 0x7f3878074c50 2024-09-10T10:04:41.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.279+0000 7f387ca00640 1 -- 172.21.15.77:0/2331523628 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38680188f0 con 0x7f3878074c50 2024-09-10T10:04:41.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.279+0000 7f387de00640 1 -- 172.21.15.77:0/2331523628 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38781a68a0 con 0x7f3878074c50 2024-09-10T10:04:41.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.280+0000 7f387de00640 1 -- 172.21.15.77:0/2331523628 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3878103aa0 con 0x7f3878074c50 2024-09-10T10:04:41.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.282+0000 7f387ca00640 1 -- 172.21.15.77:0/2331523628 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3868010ac0 con 0x7f3878074c50 2024-09-10T10:04:41.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.284+0000 7f387ca00640 1 --2- 172.21.15.77:0/2331523628 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f385c0a2b90 0x7f385c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:41.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.284+0000 7f387ca00640 1 -- 172.21.15.77:0/2331523628 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f38680cfd80 con 0x7f3878074c50 2024-09-10T10:04:41.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.284+0000 7f3876000640 1 --2- 172.21.15.77:0/2331523628 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f385c0a2b90 0x7f385c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:41.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.285+0000 7f3876000640 1 --2- 172.21.15.77:0/2331523628 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f385c0a2b90 0x7f385c0a5050 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f387819f900 tx=0x7f3864009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:41.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.286+0000 7f387ca00640 1 -- 172.21.15.77:0/2331523628 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f386809c240 con 0x7f3878074c50 2024-09-10T10:04:41.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:04:40 smithi134 ceph-mon[35674]: pgmap v676: 129 pgs: 129 active+clean; 115 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 95 KiB/s wr, 55 op/s 2024-09-10T10:04:41.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:04:40 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T10:04:41.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:04:40 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T10:04:41.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:04:40 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/3573034558' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:04:41.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:04:40 smithi120 ceph-mon[34770]: pgmap v676: 129 pgs: 129 active+clean; 115 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 95 KiB/s wr, 55 op/s 2024-09-10T10:04:41.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:04:40 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T10:04:41.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:04:40 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T10:04:41.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:04:40 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/3573034558' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:04:41.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:04:40 smithi077 ceph-mon[30319]: pgmap v676: 129 pgs: 129 active+clean; 115 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 95 KiB/s wr, 55 op/s 2024-09-10T10:04:41.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:04:40 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T10:04:41.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:04:40 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T10:04:41.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:04:40 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/3573034558' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:04:41.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.469+0000 7f387de00640 1 -- 172.21.15.77:0/2331523628 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f3878062df0 con 0x7f3878074c50 2024-09-10T10:04:41.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.470+0000 7f387ca00640 1 -- 172.21.15.77:0/2331523628 <== mon.2 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v96) ==== 104+0+11 (secure 0 0 0) 0x7f38680a0540 con 0x7f3878074c50 2024-09-10T10:04:41.472 INFO:teuthology.orchestra.run.smithi077.stdout:pg_num: 32 2024-09-10T10:04:41.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.472+0000 7f387de00640 1 -- 172.21.15.77:0/2331523628 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f385c0a2b90 msgr2=0x7f385c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:41.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.472+0000 7f387de00640 1 --2- 172.21.15.77:0/2331523628 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f385c0a2b90 0x7f385c0a5050 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f387819f900 tx=0x7f3864009260 comp rx=0 tx=0).stop 2024-09-10T10:04:41.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.472+0000 7f387de00640 1 -- 172.21.15.77:0/2331523628 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3878074c50 msgr2=0x7f387819e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:41.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.472+0000 7f387de00640 1 --2- 172.21.15.77:0/2331523628 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3878074c50 0x7f387819e3f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f386800e9f0 tx=0x7f386800eeb0 comp rx=0 tx=0).stop 2024-09-10T10:04:41.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.472+0000 7f387de00640 1 -- 172.21.15.77:0/2331523628 shutdown_connections 2024-09-10T10:04:41.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.472+0000 7f387de00640 1 --2- 172.21.15.77:0/2331523628 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f385c0a2b90 0x7f385c0a5050 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:41.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.472+0000 7f387de00640 1 --2- 172.21.15.77:0/2331523628 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f387810e100 0x7f38781a59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:41.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.472+0000 7f387de00640 1 --2- 172.21.15.77:0/2331523628 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3878076230 0x7f387819e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:41.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.472+0000 7f387de00640 1 --2- 172.21.15.77:0/2331523628 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3878074c50 0x7f387819e3f0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:41.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.472+0000 7f387de00640 1 -- 172.21.15.77:0/2331523628 >> 172.21.15.77:0/2331523628 conn(0x7f38780fd2b0 msgr2=0x7f38780ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:41.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.472+0000 7f387de00640 1 -- 172.21.15.77:0/2331523628 shutdown_connections 2024-09-10T10:04:41.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.472+0000 7f387de00640 1 -- 172.21.15.77:0/2331523628 wait complete. 2024-09-10T10:04:41.490 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-10T10:04:41.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.645+0000 7f35dfa00640 1 -- 172.21.15.77:0/1056177118 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35d80ffa80 msgr2=0x7f35d80ffe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:41.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.645+0000 7f35dfa00640 1 --2- 172.21.15.77:0/1056177118 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35d80ffa80 0x7f35d80ffe80 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f35c8009900 tx=0x7f35c802a3f0 comp rx=0 tx=0).stop 2024-09-10T10:04:41.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.645+0000 7f35dfa00640 1 -- 172.21.15.77:0/1056177118 shutdown_connections 2024-09-10T10:04:41.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.645+0000 7f35dfa00640 1 --2- 172.21.15.77:0/1056177118 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f35d8101490 0x7f35d8101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:41.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.645+0000 7f35dfa00640 1 --2- 172.21.15.77:0/1056177118 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35d8100af0 0x7f35d8100f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:41.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.645+0000 7f35dfa00640 1 --2- 172.21.15.77:0/1056177118 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35d80ffa80 0x7f35d80ffe80 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:41.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.645+0000 7f35dfa00640 1 -- 172.21.15.77:0/1056177118 >> 172.21.15.77:0/1056177118 conn(0x7f35d80fb4b0 msgr2=0x7f35d80fd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:41.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.645+0000 7f35dfa00640 1 -- 172.21.15.77:0/1056177118 shutdown_connections 2024-09-10T10:04:41.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.646+0000 7f35dfa00640 1 -- 172.21.15.77:0/1056177118 wait complete. 2024-09-10T10:04:41.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.646+0000 7f35dfa00640 1 Processor -- start 2024-09-10T10:04:41.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.646+0000 7f35dfa00640 1 -- start start 2024-09-10T10:04:41.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.647+0000 7f35dfa00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f35d80ffa80 0x7f35d819a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:41.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.647+0000 7f35dfa00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35d8100af0 0x7f35d819a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:41.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.647+0000 7f35dfa00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35d8101490 0x7f35d81a1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:41.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.647+0000 7f35dfa00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f35d810fa00 con 0x7f35d8100af0 2024-09-10T10:04:41.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.647+0000 7f35dfa00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f35d810f880 con 0x7f35d80ffa80 2024-09-10T10:04:41.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.647+0000 7f35dfa00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f35d810fb80 con 0x7f35d8101490 2024-09-10T10:04:41.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.647+0000 7f35cfe00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35d8100af0 0x7f35d819a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:41.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.647+0000 7f35cfe00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35d8100af0 0x7f35d819a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60844/0 (socket says 172.21.15.77:60844) 2024-09-10T10:04:41.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.647+0000 7f35cfe00640 1 -- 172.21.15.77:0/64856131 learned_addr learned my addr 172.21.15.77:0/64856131 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:04:41.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.647+0000 7f35dd000640 1 --2- 172.21.15.77:0/64856131 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f35d80ffa80 0x7f35d819a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:41.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.647+0000 7f35dda00640 1 --2- 172.21.15.77:0/64856131 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35d8101490 0x7f35d81a1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:41.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.647+0000 7f35cfe00640 1 -- 172.21.15.77:0/64856131 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35d8101490 msgr2=0x7f35d81a1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:41.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.647+0000 7f35cfe00640 1 --2- 172.21.15.77:0/64856131 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35d8101490 0x7f35d81a1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:41.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.647+0000 7f35cfe00640 1 -- 172.21.15.77:0/64856131 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f35d80ffa80 msgr2=0x7f35d819a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:41.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.647+0000 7f35cfe00640 1 --2- 172.21.15.77:0/64856131 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f35d80ffa80 0x7f35d819a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:41.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.647+0000 7f35cfe00640 1 -- 172.21.15.77:0/64856131 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35d81a1c90 con 0x7f35d8100af0 2024-09-10T10:04:41.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.647+0000 7f35dda00640 1 --2- 172.21.15.77:0/64856131 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35d8101490 0x7f35d81a1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:04:41.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.648+0000 7f35cfe00640 1 --2- 172.21.15.77:0/64856131 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35d8100af0 0x7f35d819a5b0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f35c000e9f0 tx=0x7f35c000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:41.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.648+0000 7f35d6000640 1 -- 172.21.15.77:0/64856131 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c000cd80 con 0x7f35d8100af0 2024-09-10T10:04:41.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.648+0000 7f35d6000640 1 -- 172.21.15.77:0/64856131 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35c0005500 con 0x7f35d8100af0 2024-09-10T10:04:41.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.648+0000 7f35dfa00640 1 -- 172.21.15.77:0/64856131 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35d81a1f80 con 0x7f35d8100af0 2024-09-10T10:04:41.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.648+0000 7f35d6000640 1 -- 172.21.15.77:0/64856131 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c0010690 con 0x7f35d8100af0 2024-09-10T10:04:41.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.648+0000 7f35dfa00640 1 -- 172.21.15.77:0/64856131 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35d81a23f0 con 0x7f35d8100af0 2024-09-10T10:04:41.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.649+0000 7f35dfa00640 1 -- 172.21.15.77:0/64856131 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35d810bf20 con 0x7f35d8100af0 2024-09-10T10:04:41.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.651+0000 7f35d6000640 1 -- 172.21.15.77:0/64856131 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f35c00046c0 con 0x7f35d8100af0 2024-09-10T10:04:41.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.652+0000 7f35d6000640 1 --2- 172.21.15.77:0/64856131 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f35bc0a2b90 0x7f35bc0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:41.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.653+0000 7f35d6000640 1 -- 172.21.15.77:0/64856131 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f35c00cf8b0 con 0x7f35d8100af0 2024-09-10T10:04:41.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.653+0000 7f35dd000640 1 --2- 172.21.15.77:0/64856131 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f35bc0a2b90 0x7f35bc0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:41.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.653+0000 7f35dd000640 1 --2- 172.21.15.77:0/64856131 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f35bc0a2b90 0x7f35bc0a5050 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f35c8031440 tx=0x7f35c80315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:41.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.656+0000 7f35d6000640 1 -- 172.21.15.77:0/64856131 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f35c009c420 con 0x7f35d8100af0 2024-09-10T10:04:41.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.830+0000 7f35dfa00640 1 -- 172.21.15.77:0/64856131 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f35d8062e90 con 0x7f35d8100af0 2024-09-10T10:04:41.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.830+0000 7f35d6000640 1 -- 172.21.15.77:0/64856131 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v96) ==== 100+0+11 (secure 0 0 0) 0x7f35c009bd20 con 0x7f35d8100af0 2024-09-10T10:04:41.832 INFO:teuthology.orchestra.run.smithi077.stdout:pg_num: 64 2024-09-10T10:04:41.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.832+0000 7f35dfa00640 1 -- 172.21.15.77:0/64856131 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f35bc0a2b90 msgr2=0x7f35bc0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:41.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.832+0000 7f35dfa00640 1 --2- 172.21.15.77:0/64856131 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f35bc0a2b90 0x7f35bc0a5050 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f35c8031440 tx=0x7f35c80315b0 comp rx=0 tx=0).stop 2024-09-10T10:04:41.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.832+0000 7f35dfa00640 1 -- 172.21.15.77:0/64856131 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35d8100af0 msgr2=0x7f35d819a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:41.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.832+0000 7f35dfa00640 1 --2- 172.21.15.77:0/64856131 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35d8100af0 0x7f35d819a5b0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f35c000e9f0 tx=0x7f35c000edf0 comp rx=0 tx=0).stop 2024-09-10T10:04:41.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.833+0000 7f35dfa00640 1 -- 172.21.15.77:0/64856131 shutdown_connections 2024-09-10T10:04:41.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.833+0000 7f35dfa00640 1 --2- 172.21.15.77:0/64856131 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f35bc0a2b90 0x7f35bc0a5050 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:41.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.833+0000 7f35dfa00640 1 --2- 172.21.15.77:0/64856131 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35d8101490 0x7f35d81a1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:41.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.833+0000 7f35dfa00640 1 --2- 172.21.15.77:0/64856131 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35d8100af0 0x7f35d819a5b0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:41.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.833+0000 7f35dfa00640 1 --2- 172.21.15.77:0/64856131 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f35d80ffa80 0x7f35d819a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:41.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.833+0000 7f35dfa00640 1 -- 172.21.15.77:0/64856131 >> 172.21.15.77:0/64856131 conn(0x7f35d80fb4b0 msgr2=0x7f35d80fcc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:41.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.833+0000 7f35dfa00640 1 -- 172.21.15.77:0/64856131 shutdown_connections 2024-09-10T10:04:41.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:41.833+0000 7f35dfa00640 1 -- 172.21.15.77:0/64856131 wait complete. 2024-09-10T10:04:41.852 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-10T10:04:42.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.009+0000 7f3326800640 1 -- 172.21.15.77:0/1482993521 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3320104e00 msgr2=0x7f3320105280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:42.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.009+0000 7f3326800640 1 --2- 172.21.15.77:0/1482993521 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3320104e00 0x7f3320105280 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f330c009900 tx=0x7f330c02a3f0 comp rx=0 tx=0).stop 2024-09-10T10:04:42.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.009+0000 7f3326800640 1 -- 172.21.15.77:0/1482993521 shutdown_connections 2024-09-10T10:04:42.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.009+0000 7f3326800640 1 --2- 172.21.15.77:0/1482993521 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f33201057c0 0x7f332010c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:42.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.009+0000 7f3326800640 1 --2- 172.21.15.77:0/1482993521 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3320104e00 0x7f3320105280 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:42.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.009+0000 7f3326800640 1 --2- 172.21.15.77:0/1482993521 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3320103c10 0x7f3320104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:42.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.009+0000 7f3326800640 1 -- 172.21.15.77:0/1482993521 >> 172.21.15.77:0/1482993521 conn(0x7f33200ff390 msgr2=0x7f33201017b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:42.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.009+0000 7f3326800640 1 -- 172.21.15.77:0/1482993521 shutdown_connections 2024-09-10T10:04:42.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.009+0000 7f3326800640 1 -- 172.21.15.77:0/1482993521 wait complete. 2024-09-10T10:04:42.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.010+0000 7f3326800640 1 Processor -- start 2024-09-10T10:04:42.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.010+0000 7f3326800640 1 -- start start 2024-09-10T10:04:42.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.010+0000 7f3326800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3320103c10 0x7f3320199f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:42.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.010+0000 7f3326800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3320104e00 0x7f332019a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:42.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.010+0000 7f3326800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f33201057c0 0x7f33201a14e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:42.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.010+0000 7f3326800640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f332010f8e0 con 0x7f33201057c0 2024-09-10T10:04:42.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.010+0000 7f3326800640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f332010f760 con 0x7f3320103c10 2024-09-10T10:04:42.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.010+0000 7f3326800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f332010fa60 con 0x7f3320104e00 2024-09-10T10:04:42.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.011+0000 7f3325e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f33201057c0 0x7f33201a14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:42.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.011+0000 7f3325400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3320103c10 0x7f3320199f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:42.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.011+0000 7f3325e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f33201057c0 0x7f33201a14e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60864/0 (socket says 172.21.15.77:60864) 2024-09-10T10:04:42.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.011+0000 7f3324a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3320104e00 0x7f332019a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:42.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.011+0000 7f3325e00640 1 -- 172.21.15.77:0/1582998527 learned_addr learned my addr 172.21.15.77:0/1582998527 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:04:42.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.011+0000 7f3325e00640 1 -- 172.21.15.77:0/1582998527 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3320104e00 msgr2=0x7f332019a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:42.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.011+0000 7f3325e00640 1 --2- 172.21.15.77:0/1582998527 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3320104e00 0x7f332019a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:42.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.011+0000 7f3325e00640 1 -- 172.21.15.77:0/1582998527 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3320103c10 msgr2=0x7f3320199f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:42.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.011+0000 7f3325e00640 1 --2- 172.21.15.77:0/1582998527 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3320103c10 0x7f3320199f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:42.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.011+0000 7f3325e00640 1 -- 172.21.15.77:0/1582998527 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33201a1b50 con 0x7f33201057c0 2024-09-10T10:04:42.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.011+0000 7f3325400640 1 --2- 172.21.15.77:0/1582998527 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3320103c10 0x7f3320199f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:04:42.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.011+0000 7f3325e00640 1 --2- 172.21.15.77:0/1582998527 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f33201057c0 0x7f33201a14e0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f331800eb10 tx=0x7f331800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:42.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.012+0000 7f331e000640 1 -- 172.21.15.77:0/1582998527 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3318004b20 con 0x7f33201057c0 2024-09-10T10:04:42.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.012+0000 7f3326800640 1 -- 172.21.15.77:0/1582998527 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33201a1e40 con 0x7f33201057c0 2024-09-10T10:04:42.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.012+0000 7f331e000640 1 -- 172.21.15.77:0/1582998527 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3318004cc0 con 0x7f33201057c0 2024-09-10T10:04:42.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.012+0000 7f331e000640 1 -- 172.21.15.77:0/1582998527 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3318010760 con 0x7f33201057c0 2024-09-10T10:04:42.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.012+0000 7f3326800640 1 -- 172.21.15.77:0/1582998527 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33201a2300 con 0x7f33201057c0 2024-09-10T10:04:42.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.013+0000 7f3326800640 1 -- 172.21.15.77:0/1582998527 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f332010b000 con 0x7f33201057c0 2024-09-10T10:04:42.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.015+0000 7f331e000640 1 -- 172.21.15.77:0/1582998527 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3318005020 con 0x7f33201057c0 2024-09-10T10:04:42.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.016+0000 7f331e000640 1 --2- 172.21.15.77:0/1582998527 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f33080a2b90 0x7f33080a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:42.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.016+0000 7f331e000640 1 -- 172.21.15.77:0/1582998527 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f33180cfb30 con 0x7f33201057c0 2024-09-10T10:04:42.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.016+0000 7f3325400640 1 --2- 172.21.15.77:0/1582998527 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f33080a2b90 0x7f33080a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:42.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.017+0000 7f3325400640 1 --2- 172.21.15.77:0/1582998527 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f33080a2b90 0x7f33080a5050 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f3320104c60 tx=0x7f3314005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:42.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.020+0000 7f331e000640 1 -- 172.21.15.77:0/1582998527 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f331809c640 con 0x7f33201057c0 2024-09-10T10:04:42.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:04:41 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/1864591244' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T10:04:42.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:04:41 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:04:42.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:04:41 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:04:42.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:04:41 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T10:04:42.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:04:41 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/2331523628' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-10T10:04:42.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:04:41 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/64856131' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-10T10:04:42.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.192+0000 7f3326800640 1 -- 172.21.15.77:0/1582998527 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f3320062df0 con 0x7f33201057c0 2024-09-10T10:04:42.194 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.192+0000 7f331e000640 1 -- 172.21.15.77:0/1582998527 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v96) ==== 103+0+11 (secure 0 0 0) 0x7f331809bf40 con 0x7f33201057c0 2024-09-10T10:04:42.194 INFO:teuthology.orchestra.run.smithi077.stdout:pg_num: 32 2024-09-10T10:04:42.196 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.194+0000 7f3326800640 1 -- 172.21.15.77:0/1582998527 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f33080a2b90 msgr2=0x7f33080a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:42.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.194+0000 7f3326800640 1 --2- 172.21.15.77:0/1582998527 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f33080a2b90 0x7f33080a5050 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f3320104c60 tx=0x7f3314005f90 comp rx=0 tx=0).stop 2024-09-10T10:04:42.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.194+0000 7f3326800640 1 -- 172.21.15.77:0/1582998527 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f33201057c0 msgr2=0x7f33201a14e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:42.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.194+0000 7f3326800640 1 --2- 172.21.15.77:0/1582998527 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f33201057c0 0x7f33201a14e0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f331800eb10 tx=0x7f331800c2d0 comp rx=0 tx=0).stop 2024-09-10T10:04:42.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.194+0000 7f3326800640 1 -- 172.21.15.77:0/1582998527 shutdown_connections 2024-09-10T10:04:42.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.194+0000 7f3326800640 1 --2- 172.21.15.77:0/1582998527 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f33080a2b90 0x7f33080a5050 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:42.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.194+0000 7f3326800640 1 --2- 172.21.15.77:0/1582998527 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f33201057c0 0x7f33201a14e0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:42.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.194+0000 7f3326800640 1 --2- 172.21.15.77:0/1582998527 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3320104e00 0x7f332019a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:42.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.194+0000 7f3326800640 1 --2- 172.21.15.77:0/1582998527 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3320103c10 0x7f3320199f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:42.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.194+0000 7f3326800640 1 -- 172.21.15.77:0/1582998527 >> 172.21.15.77:0/1582998527 conn(0x7f33200ff390 msgr2=0x7f33201081d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:42.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.194+0000 7f3326800640 1 -- 172.21.15.77:0/1582998527 shutdown_connections 2024-09-10T10:04:42.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.195+0000 7f3326800640 1 -- 172.21.15.77:0/1582998527 wait complete. 2024-09-10T10:04:42.213 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-10T10:04:42.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.366+0000 7f61d1c00640 1 -- 172.21.15.77:0/59533863 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f61cc1058e0 msgr2=0x7f61cc10c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:42.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.366+0000 7f61d1c00640 1 --2- 172.21.15.77:0/59533863 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f61cc1058e0 0x7f61cc10c170 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f61c4009ca0 tx=0x7f61c402a8c0 comp rx=0 tx=0).stop 2024-09-10T10:04:42.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.366+0000 7f61d1c00640 1 -- 172.21.15.77:0/59533863 shutdown_connections 2024-09-10T10:04:42.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.366+0000 7f61d1c00640 1 --2- 172.21.15.77:0/59533863 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f61cc1058e0 0x7f61cc10c170 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:42.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.366+0000 7f61d1c00640 1 --2- 172.21.15.77:0/59533863 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f61cc104f20 0x7f61cc1053a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:42.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.366+0000 7f61d1c00640 1 --2- 172.21.15.77:0/59533863 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f61cc103d30 0x7f61cc104130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:42.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.366+0000 7f61d1c00640 1 -- 172.21.15.77:0/59533863 >> 172.21.15.77:0/59533863 conn(0x7f61cc0ff4b0 msgr2=0x7f61cc1018d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:42.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.366+0000 7f61d1c00640 1 -- 172.21.15.77:0/59533863 shutdown_connections 2024-09-10T10:04:42.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.366+0000 7f61d1c00640 1 -- 172.21.15.77:0/59533863 wait complete. 2024-09-10T10:04:42.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.367+0000 7f61d1c00640 1 Processor -- start 2024-09-10T10:04:42.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.367+0000 7f61d1c00640 1 -- start start 2024-09-10T10:04:42.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.368+0000 7f61d1c00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f61cc103d30 0x7f61cc19a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:42.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.368+0000 7f61d1c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f61cc104f20 0x7f61cc19a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:42.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.368+0000 7f61d1c00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f61cc1058e0 0x7f61cc1a15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:42.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.368+0000 7f61d1c00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f61cc10fa00 con 0x7f61cc1058e0 2024-09-10T10:04:42.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.368+0000 7f61d1c00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f61cc10f880 con 0x7f61cc103d30 2024-09-10T10:04:42.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.368+0000 7f61d1c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f61cc10fb80 con 0x7f61cc104f20 2024-09-10T10:04:42.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.368+0000 7f61caa00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f61cc103d30 0x7f61cc19a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:42.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.368+0000 7f61cb400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f61cc1058e0 0x7f61cc1a15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:42.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.368+0000 7f61c2000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f61cc104f20 0x7f61cc19a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:42.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.368+0000 7f61cb400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f61cc1058e0 0x7f61cc1a15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60884/0 (socket says 172.21.15.77:60884) 2024-09-10T10:04:42.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.368+0000 7f61c2000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f61cc104f20 0x7f61cc19a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.77:54590/0 (socket says 172.21.15.77:54590) 2024-09-10T10:04:42.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.368+0000 7f61cb400640 1 -- 172.21.15.77:0/4247775583 learned_addr learned my addr 172.21.15.77:0/4247775583 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:04:42.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.368+0000 7f61caa00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f61cc103d30 0x7f61cc19a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:37022/0 (socket says 172.21.15.77:37022) 2024-09-10T10:04:42.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.368+0000 7f61caa00640 1 -- 172.21.15.77:0/4247775583 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f61cc104f20 msgr2=0x7f61cc19a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:42.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.368+0000 7f61caa00640 1 --2- 172.21.15.77:0/4247775583 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f61cc104f20 0x7f61cc19a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:42.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.368+0000 7f61caa00640 1 -- 172.21.15.77:0/4247775583 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f61cc1058e0 msgr2=0x7f61cc1a15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:42.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.368+0000 7f61caa00640 1 --2- 172.21.15.77:0/4247775583 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f61cc1058e0 0x7f61cc1a15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:42.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.368+0000 7f61caa00640 1 -- 172.21.15.77:0/4247775583 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61cc1a1c20 con 0x7f61cc103d30 2024-09-10T10:04:42.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.368+0000 7f61cb400640 1 --2- 172.21.15.77:0/4247775583 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f61cc1058e0 0x7f61cc1a15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:04:42.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.369+0000 7f61caa00640 1 --2- 172.21.15.77:0/4247775583 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f61cc103d30 0x7f61cc19a000 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f61bc00e9c0 tx=0x7f61bc00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:42.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.369+0000 7f61c3e00640 1 -- 172.21.15.77:0/4247775583 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61bc00ccc0 con 0x7f61cc103d30 2024-09-10T10:04:42.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.369+0000 7f61c3e00640 1 -- 172.21.15.77:0/4247775583 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61bc0035f0 con 0x7f61cc103d30 2024-09-10T10:04:42.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.369+0000 7f61d1c00640 1 -- 172.21.15.77:0/4247775583 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61cc1a1f10 con 0x7f61cc103d30 2024-09-10T10:04:42.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.369+0000 7f61c3e00640 1 -- 172.21.15.77:0/4247775583 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61bc00ce60 con 0x7f61cc103d30 2024-09-10T10:04:42.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.369+0000 7f61d1c00640 1 -- 172.21.15.77:0/4247775583 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61cc1a2370 con 0x7f61cc103d30 2024-09-10T10:04:42.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.370+0000 7f61d1c00640 1 -- 172.21.15.77:0/4247775583 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61cc10b050 con 0x7f61cc103d30 2024-09-10T10:04:42.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.372+0000 7f61c3e00640 1 -- 172.21.15.77:0/4247775583 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f61bc0037a0 con 0x7f61cc103d30 2024-09-10T10:04:42.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.373+0000 7f61c3e00640 1 --2- 172.21.15.77:0/4247775583 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f61b00a2b90 0x7f61b00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:42.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.373+0000 7f61c3e00640 1 -- 172.21.15.77:0/4247775583 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f61bc0cf980 con 0x7f61cc103d30 2024-09-10T10:04:42.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.374+0000 7f61c2000640 1 --2- 172.21.15.77:0/4247775583 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f61b00a2b90 0x7f61b00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:42.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.374+0000 7f61c2000640 1 --2- 172.21.15.77:0/4247775583 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f61b00a2b90 0x7f61b00a5050 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f61cc19b510 tx=0x7f61b4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:42.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.375+0000 7f61c3e00640 1 -- 172.21.15.77:0/4247775583 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f61bc09c490 con 0x7f61cc103d30 2024-09-10T10:04:42.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:04:41 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/1864591244' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T10:04:42.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:04:41 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:04:42.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:04:41 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:04:42.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:04:41 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T10:04:42.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:04:41 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/2331523628' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-10T10:04:42.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:04:41 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/64856131' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-10T10:04:42.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:04:41 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/1864591244' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T10:04:42.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:04:41 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:04:42.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:04:41 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:04:42.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:04:41 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' 2024-09-10T10:04:42.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:04:41 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/2331523628' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-10T10:04:42.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:04:41 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/64856131' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-10T10:04:42.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.594+0000 7f61d1c00640 1 -- 172.21.15.77:0/4247775583 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f61cc02d9e0 con 0x7f61cc103d30 2024-09-10T10:04:42.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.604+0000 7f61c3e00640 1 -- 172.21.15.77:0/4247775583 <== mon.2 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v18) ==== 81+0+740370 (secure 0 0 0) 0x7f61bc09bd90 con 0x7f61cc103d30 2024-09-10T10:04:42.617 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T10:04:42.617 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-10T10:04:42.617 INFO:teuthology.orchestra.run.smithi077.stdout: "epoch": 18, 2024-09-10T10:04:42.617 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.617 INFO:teuthology.orchestra.run.smithi077.stdout: "active_gid": 14150, 2024-09-10T10:04:42.617 INFO:teuthology.orchestra.run.smithi077.stdout: "active_name": "x", 2024-09-10T10:04:42.617 INFO:teuthology.orchestra.run.smithi077.stdout: "active_addrs": { 2024-09-10T10:04:42.617 INFO:teuthology.orchestra.run.smithi077.stdout: "addrvec": [ 2024-09-10T10:04:42.617 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:42.617 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "v2", 2024-09-10T10:04:42.617 INFO:teuthology.orchestra.run.smithi077.stdout: "addr": "172.21.15.77:6800", 2024-09-10T10:04:42.617 INFO:teuthology.orchestra.run.smithi077.stdout: "nonce": 4261324650 2024-09-10T10:04:42.618 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.618 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:42.618 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "v1", 2024-09-10T10:04:42.618 INFO:teuthology.orchestra.run.smithi077.stdout: "addr": "172.21.15.77:6801", 2024-09-10T10:04:42.618 INFO:teuthology.orchestra.run.smithi077.stdout: "nonce": 4261324650 2024-09-10T10:04:42.618 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:42.618 INFO:teuthology.orchestra.run.smithi077.stdout: ] 2024-09-10T10:04:42.618 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.618 INFO:teuthology.orchestra.run.smithi077.stdout: "active_addr": "172.21.15.77:6801/4261324650", 2024-09-10T10:04:42.618 INFO:teuthology.orchestra.run.smithi077.stdout: "active_change": "2024-09-10T09:44:59.575771+0000", 2024-09-10T10:04:42.618 INFO:teuthology.orchestra.run.smithi077.stdout: "active_mgr_features": 4540701547738038271, 2024-09-10T10:04:42.618 INFO:teuthology.orchestra.run.smithi077.stdout: "available": true, 2024-09-10T10:04:42.619 INFO:teuthology.orchestra.run.smithi077.stdout: "standbys": [ 2024-09-10T10:04:42.619 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:42.619 INFO:teuthology.orchestra.run.smithi077.stdout: "gid": 24109, 2024-09-10T10:04:42.619 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "z", 2024-09-10T10:04:42.619 INFO:teuthology.orchestra.run.smithi077.stdout: "mgr_features": 4540701547738038271, 2024-09-10T10:04:42.619 INFO:teuthology.orchestra.run.smithi077.stdout: "available_modules": [ 2024-09-10T10:04:42.619 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:42.619 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "alerts", 2024-09-10T10:04:42.619 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:42.619 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:42.619 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:42.619 INFO:teuthology.orchestra.run.smithi077.stdout: "interval": { 2024-09-10T10:04:42.619 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "interval", 2024-09-10T10:04:42.619 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-10T10:04:42.620 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.620 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.620 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "60", 2024-09-10T10:04:42.620 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.620 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.620 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.620 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "How frequently to reexamine health status", 2024-09-10T10:04:42.620 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.620 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.620 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.620 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.620 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:42.620 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:42.620 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.621 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.621 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.621 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:42.621 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.621 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.621 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:42.621 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:42.621 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:42.621 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:42.621 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:42.621 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:42.621 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:42.621 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:42.622 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.622 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.622 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.622 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.622 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.622 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:42.622 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:42.622 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.622 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.622 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.622 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:42.622 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.622 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.622 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.623 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.623 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.623 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.623 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.623 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.623 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:42.623 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:42.623 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.623 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.623 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.623 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:42.623 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.623 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.623 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:42.624 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:42.624 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:42.624 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:42.624 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:42.624 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:42.624 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:42.624 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:42.624 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.624 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.624 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.624 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.624 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.624 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:42.625 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:42.625 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.625 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.625 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.625 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:42.625 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.625 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.625 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.625 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.625 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.625 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.625 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.625 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.625 INFO:teuthology.orchestra.run.smithi077.stdout: "smtp_destination": { 2024-09-10T10:04:42.626 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smtp_destination", 2024-09-10T10:04:42.626 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.626 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.626 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.626 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:42.626 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.626 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.626 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.626 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Email address to send alerts to", 2024-09-10T10:04:42.626 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.626 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.626 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.626 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.627 INFO:teuthology.orchestra.run.smithi077.stdout: "smtp_from_name": { 2024-09-10T10:04:42.627 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smtp_from_name", 2024-09-10T10:04:42.627 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.627 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.627 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.627 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "Ceph", 2024-09-10T10:04:42.627 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.627 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.627 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.627 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Email From: name", 2024-09-10T10:04:42.627 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.627 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.627 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.627 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.628 INFO:teuthology.orchestra.run.smithi077.stdout: "smtp_host": { 2024-09-10T10:04:42.628 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smtp_host", 2024-09-10T10:04:42.628 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.628 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.628 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.628 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:42.628 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.628 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.628 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.628 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "SMTP server", 2024-09-10T10:04:42.628 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.628 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.629 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.629 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.629 INFO:teuthology.orchestra.run.smithi077.stdout: "smtp_password": { 2024-09-10T10:04:42.629 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smtp_password", 2024-09-10T10:04:42.629 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.629 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.629 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.629 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:42.629 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.629 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.629 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.629 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Password to authenticate with", 2024-09-10T10:04:42.629 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.630 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.630 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.630 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.630 INFO:teuthology.orchestra.run.smithi077.stdout: "smtp_port": { 2024-09-10T10:04:42.630 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smtp_port", 2024-09-10T10:04:42.630 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:42.630 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.630 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.630 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "465", 2024-09-10T10:04:42.630 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.630 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.630 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.630 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "SMTP port", 2024-09-10T10:04:42.630 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.631 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.631 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.631 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.631 INFO:teuthology.orchestra.run.smithi077.stdout: "smtp_sender": { 2024-09-10T10:04:42.631 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smtp_sender", 2024-09-10T10:04:42.631 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.631 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.631 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.631 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:42.631 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.631 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.631 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.631 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "SMTP envelope sender", 2024-09-10T10:04:42.631 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.631 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.632 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.632 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.632 INFO:teuthology.orchestra.run.smithi077.stdout: "smtp_ssl": { 2024-09-10T10:04:42.632 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smtp_ssl", 2024-09-10T10:04:42.632 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.632 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.632 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.632 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:42.632 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.632 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.632 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.632 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-10T10:04:42.633 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.633 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.633 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.633 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.633 INFO:teuthology.orchestra.run.smithi077.stdout: "smtp_user": { 2024-09-10T10:04:42.633 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smtp_user", 2024-09-10T10:04:42.633 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.633 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.633 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.633 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:42.633 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.633 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.633 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.634 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "User to authenticate as", 2024-09-10T10:04:42.634 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.634 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.634 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.634 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:42.634 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:42.634 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.634 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:42.634 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "balancer", 2024-09-10T10:04:42.634 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:42.634 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:42.634 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:42.634 INFO:teuthology.orchestra.run.smithi077.stdout: "active": { 2024-09-10T10:04:42.635 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "active", 2024-09-10T10:04:42.635 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.635 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.635 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.635 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:42.635 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.635 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.635 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.635 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "automatically balance PGs across cluster", 2024-09-10T10:04:42.635 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.635 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.635 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.636 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.636 INFO:teuthology.orchestra.run.smithi077.stdout: "begin_time": { 2024-09-10T10:04:42.636 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "begin_time", 2024-09-10T10:04:42.636 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.636 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.636 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.636 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0000", 2024-09-10T10:04:42.636 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.636 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.636 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.636 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "beginning time of day to automatically balance", 2024-09-10T10:04:42.636 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-10T10:04:42.636 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.637 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.637 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.637 INFO:teuthology.orchestra.run.smithi077.stdout: "begin_weekday": { 2024-09-10T10:04:42.637 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "begin_weekday", 2024-09-10T10:04:42.637 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "uint", 2024-09-10T10:04:42.637 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.637 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.637 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-09-10T10:04:42.637 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "0", 2024-09-10T10:04:42.637 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "6", 2024-09-10T10:04:42.637 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.637 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-10T10:04:42.638 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-10T10:04:42.638 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.638 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.638 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.638 INFO:teuthology.orchestra.run.smithi077.stdout: "crush_compat_max_iterations": { 2024-09-10T10:04:42.638 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "crush_compat_max_iterations", 2024-09-10T10:04:42.638 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "uint", 2024-09-10T10:04:42.638 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.638 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.638 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "25", 2024-09-10T10:04:42.638 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "1", 2024-09-10T10:04:42.638 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "250", 2024-09-10T10:04:42.638 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.639 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-10T10:04:42.639 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.639 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.639 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.639 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.639 INFO:teuthology.orchestra.run.smithi077.stdout: "crush_compat_metrics": { 2024-09-10T10:04:42.639 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "crush_compat_metrics", 2024-09-10T10:04:42.639 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.639 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.639 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.639 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "pgs,objects,bytes", 2024-09-10T10:04:42.639 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.639 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.639 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.640 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-10T10:04:42.640 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-10T10:04:42.640 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.640 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.640 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.640 INFO:teuthology.orchestra.run.smithi077.stdout: "crush_compat_step": { 2024-09-10T10:04:42.640 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "crush_compat_step", 2024-09-10T10:04:42.640 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "float", 2024-09-10T10:04:42.640 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.640 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.640 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0.5", 2024-09-10T10:04:42.640 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "0.001", 2024-09-10T10:04:42.640 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "0.999", 2024-09-10T10:04:42.641 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.641 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "aggressiveness of optimization", 2024-09-10T10:04:42.641 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-10T10:04:42.641 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.641 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.641 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.641 INFO:teuthology.orchestra.run.smithi077.stdout: "end_time": { 2024-09-10T10:04:42.641 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "end_time", 2024-09-10T10:04:42.641 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.641 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.641 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.641 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "2359", 2024-09-10T10:04:42.641 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.642 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.642 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.642 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "ending time of day to automatically balance", 2024-09-10T10:04:42.642 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-10T10:04:42.642 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.642 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.642 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.642 INFO:teuthology.orchestra.run.smithi077.stdout: "end_weekday": { 2024-09-10T10:04:42.642 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "end_weekday", 2024-09-10T10:04:42.642 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "uint", 2024-09-10T10:04:42.642 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.642 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.642 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-09-10T10:04:42.642 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "0", 2024-09-10T10:04:42.643 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "6", 2024-09-10T10:04:42.643 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.643 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-10T10:04:42.643 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-10T10:04:42.643 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.643 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.643 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.643 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:42.643 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:42.643 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.643 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.643 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.644 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:42.644 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.644 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.644 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:42.644 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:42.644 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:42.644 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:42.644 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:42.644 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:42.644 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:42.644 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:42.644 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.645 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.645 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.645 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.645 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.645 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:42.645 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:42.645 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.645 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.645 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.646 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:42.646 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.646 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.646 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.646 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.646 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.646 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.646 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.646 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.646 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:42.646 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:42.646 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.646 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.647 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.647 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:42.647 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.647 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.647 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:42.647 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:42.647 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:42.647 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:42.647 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:42.647 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:42.647 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:42.647 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:42.648 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.648 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.648 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.648 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.648 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.648 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:42.648 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:42.648 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.648 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.648 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.648 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:42.648 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.648 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.649 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.649 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.649 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.649 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.649 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.649 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.649 INFO:teuthology.orchestra.run.smithi077.stdout: "min_score": { 2024-09-10T10:04:42.649 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "min_score", 2024-09-10T10:04:42.649 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "float", 2024-09-10T10:04:42.649 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.649 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.649 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-09-10T10:04:42.649 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.650 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.650 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.650 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-10T10:04:42.650 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.650 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.650 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.650 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.650 INFO:teuthology.orchestra.run.smithi077.stdout: "mode": { 2024-09-10T10:04:42.650 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "mode", 2024-09-10T10:04:42.650 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.651 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.651 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.651 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "upmap", 2024-09-10T10:04:42.651 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.651 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.651 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:42.651 INFO:teuthology.orchestra.run.smithi077.stdout: "crush-compat", 2024-09-10T10:04:42.651 INFO:teuthology.orchestra.run.smithi077.stdout: "none", 2024-09-10T10:04:42.651 INFO:teuthology.orchestra.run.smithi077.stdout: "read", 2024-09-10T10:04:42.651 INFO:teuthology.orchestra.run.smithi077.stdout: "upmap", 2024-09-10T10:04:42.651 INFO:teuthology.orchestra.run.smithi077.stdout: "upmap-read" 2024-09-10T10:04:42.651 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:42.651 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Balancer mode", 2024-09-10T10:04:42.652 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.652 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.652 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.652 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.652 INFO:teuthology.orchestra.run.smithi077.stdout: "pool_ids": { 2024-09-10T10:04:42.652 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "pool_ids", 2024-09-10T10:04:42.652 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.652 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.652 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.652 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:42.652 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.652 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.653 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.653 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-10T10:04:42.653 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.653 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.653 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.653 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.653 INFO:teuthology.orchestra.run.smithi077.stdout: "sleep_interval": { 2024-09-10T10:04:42.653 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sleep_interval", 2024-09-10T10:04:42.653 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-10T10:04:42.653 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.653 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.653 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "60", 2024-09-10T10:04:42.653 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.653 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.654 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.654 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-10T10:04:42.654 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.654 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.654 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.654 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.654 INFO:teuthology.orchestra.run.smithi077.stdout: "upmap_max_deviation": { 2024-09-10T10:04:42.654 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "upmap_max_deviation", 2024-09-10T10:04:42.654 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:42.654 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.654 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.654 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "5", 2024-09-10T10:04:42.654 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "1", 2024-09-10T10:04:42.654 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.655 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.655 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-10T10:04:42.655 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-10T10:04:42.655 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.655 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.655 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.655 INFO:teuthology.orchestra.run.smithi077.stdout: "upmap_max_optimizations": { 2024-09-10T10:04:42.655 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "upmap_max_optimizations", 2024-09-10T10:04:42.655 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "uint", 2024-09-10T10:04:42.655 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.655 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.655 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10", 2024-09-10T10:04:42.655 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.655 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.656 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.656 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-10T10:04:42.656 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.656 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.656 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.656 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:42.656 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:42.656 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.656 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:42.656 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "cephadm", 2024-09-10T10:04:42.656 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:42.656 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:42.656 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:42.656 INFO:teuthology.orchestra.run.smithi077.stdout: "agent_down_multiplier": { 2024-09-10T10:04:42.656 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "agent_down_multiplier", 2024-09-10T10:04:42.657 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "float", 2024-09-10T10:04:42.657 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.657 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.657 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "3.0", 2024-09-10T10:04:42.657 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.657 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.657 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.657 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-10T10:04:42.657 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.657 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.657 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.657 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.657 INFO:teuthology.orchestra.run.smithi077.stdout: "agent_refresh_rate": { 2024-09-10T10:04:42.658 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "agent_refresh_rate", 2024-09-10T10:04:42.658 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-10T10:04:42.658 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.658 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.658 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "20", 2024-09-10T10:04:42.658 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.658 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.658 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.658 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-10T10:04:42.658 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.658 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.658 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.658 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.658 INFO:teuthology.orchestra.run.smithi077.stdout: "agent_starting_port": { 2024-09-10T10:04:42.658 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "agent_starting_port", 2024-09-10T10:04:42.659 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:42.659 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.659 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.659 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "4721", 2024-09-10T10:04:42.659 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.659 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.659 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.659 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-10T10:04:42.660 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.660 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.660 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.660 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.660 INFO:teuthology.orchestra.run.smithi077.stdout: "allow_ptrace": { 2024-09-10T10:04:42.660 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "allow_ptrace", 2024-09-10T10:04:42.660 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.660 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.660 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.660 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:42.660 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.660 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.661 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.661 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-10T10:04:42.661 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-10T10:04:42.661 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.661 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.661 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.661 INFO:teuthology.orchestra.run.smithi077.stdout: "autotune_interval": { 2024-09-10T10:04:42.661 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "autotune_interval", 2024-09-10T10:04:42.661 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-10T10:04:42.661 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.661 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.662 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "600", 2024-09-10T10:04:42.662 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.662 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.662 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.662 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-10T10:04:42.662 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.662 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.662 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.662 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.662 INFO:teuthology.orchestra.run.smithi077.stdout: "autotune_memory_target_ratio": { 2024-09-10T10:04:42.662 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "autotune_memory_target_ratio", 2024-09-10T10:04:42.662 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "float", 2024-09-10T10:04:42.662 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.663 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.663 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0.7", 2024-09-10T10:04:42.663 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.663 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.663 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.663 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-10T10:04:42.663 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.663 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.663 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.663 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.663 INFO:teuthology.orchestra.run.smithi077.stdout: "cephadm_log_destination": { 2024-09-10T10:04:42.663 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "cephadm_log_destination", 2024-09-10T10:04:42.663 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.664 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.664 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.664 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:42.664 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.664 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.664 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:42.664 INFO:teuthology.orchestra.run.smithi077.stdout: "file", 2024-09-10T10:04:42.664 INFO:teuthology.orchestra.run.smithi077.stdout: "file,syslog", 2024-09-10T10:04:42.664 INFO:teuthology.orchestra.run.smithi077.stdout: "syslog" 2024-09-10T10:04:42.664 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:42.664 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-10T10:04:42.664 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.664 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.664 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.664 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.665 INFO:teuthology.orchestra.run.smithi077.stdout: "cgroups_split": { 2024-09-10T10:04:42.665 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "cgroups_split", 2024-09-10T10:04:42.665 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.665 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.665 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.665 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:42.665 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.665 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.665 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.665 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-10T10:04:42.665 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.665 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.665 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.665 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.665 INFO:teuthology.orchestra.run.smithi077.stdout: "config_checks_enabled": { 2024-09-10T10:04:42.666 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "config_checks_enabled", 2024-09-10T10:04:42.666 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.666 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.666 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.666 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:42.666 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.666 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.666 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.666 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-10T10:04:42.666 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.666 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.666 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.666 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.666 INFO:teuthology.orchestra.run.smithi077.stdout: "config_dashboard": { 2024-09-10T10:04:42.666 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "config_dashboard", 2024-09-10T10:04:42.667 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.667 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.667 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.667 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:42.667 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.667 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.667 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.667 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-10T10:04:42.667 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.667 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.667 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.667 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.667 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_alertmanager": { 2024-09-10T10:04:42.667 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_alertmanager", 2024-09-10T10:04:42.668 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.668 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.668 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.668 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-10T10:04:42.668 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.668 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.668 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.668 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Prometheus container image", 2024-09-10T10:04:42.668 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.668 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.668 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.668 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.668 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_base": { 2024-09-10T10:04:42.669 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_base", 2024-09-10T10:04:42.669 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.669 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.669 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.669 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-10T10:04:42.669 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.669 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.669 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.669 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Container image name, without the tag", 2024-09-10T10:04:42.669 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.669 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.669 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.669 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.670 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_elasticsearch": { 2024-09-10T10:04:42.670 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_elasticsearch", 2024-09-10T10:04:42.670 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.670 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.670 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.670 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-10T10:04:42.670 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.670 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.670 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.670 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "elasticsearch container image", 2024-09-10T10:04:42.670 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.670 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.670 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.670 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.670 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_grafana": { 2024-09-10T10:04:42.671 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_grafana", 2024-09-10T10:04:42.671 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.671 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.671 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.671 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-10T10:04:42.671 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.671 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.671 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.671 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Prometheus container image", 2024-09-10T10:04:42.671 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.671 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.671 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.671 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.671 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_haproxy": { 2024-09-10T10:04:42.671 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_haproxy", 2024-09-10T10:04:42.672 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.672 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.672 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.672 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-10T10:04:42.672 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.672 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.672 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.672 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "HAproxy container image", 2024-09-10T10:04:42.672 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.672 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.672 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.672 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.672 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_jaeger_agent": { 2024-09-10T10:04:42.673 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_jaeger_agent", 2024-09-10T10:04:42.673 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.673 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.673 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.673 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-10T10:04:42.673 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.673 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.673 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.673 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Jaeger agent container image", 2024-09-10T10:04:42.673 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.673 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.673 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.673 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.673 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_jaeger_collector": { 2024-09-10T10:04:42.674 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_jaeger_collector", 2024-09-10T10:04:42.674 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.674 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.674 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.674 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-10T10:04:42.674 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.674 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.674 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.674 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Jaeger collector container image", 2024-09-10T10:04:42.674 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.674 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.674 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.674 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.674 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_jaeger_query": { 2024-09-10T10:04:42.675 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_jaeger_query", 2024-09-10T10:04:42.675 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.675 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.675 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.675 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-10T10:04:42.675 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.675 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.675 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.675 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Jaeger query container image", 2024-09-10T10:04:42.675 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.675 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.675 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.675 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.675 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_keepalived": { 2024-09-10T10:04:42.675 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_keepalived", 2024-09-10T10:04:42.676 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.676 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.676 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.676 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-10T10:04:42.676 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.676 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.676 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.676 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Keepalived container image", 2024-09-10T10:04:42.676 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.676 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.676 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.676 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.676 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_loki": { 2024-09-10T10:04:42.676 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_loki", 2024-09-10T10:04:42.677 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.677 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.677 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.677 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-10T10:04:42.677 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.677 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.677 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.677 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Loki container image", 2024-09-10T10:04:42.677 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.677 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.677 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.677 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.677 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_node_exporter": { 2024-09-10T10:04:42.677 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_node_exporter", 2024-09-10T10:04:42.678 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.678 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.678 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.678 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-10T10:04:42.678 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.678 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.678 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.678 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Prometheus container image", 2024-09-10T10:04:42.678 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.678 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.678 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.678 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.678 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_nvmeof": { 2024-09-10T10:04:42.678 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_nvmeof", 2024-09-10T10:04:42.678 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.679 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.679 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.679 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-10T10:04:42.679 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.679 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.679 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.679 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Nvme-of container image", 2024-09-10T10:04:42.679 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.679 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.679 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.679 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.679 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_prometheus": { 2024-09-10T10:04:42.679 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_prometheus", 2024-09-10T10:04:42.679 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.679 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.680 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.680 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-10T10:04:42.680 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.680 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.680 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.680 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Prometheus container image", 2024-09-10T10:04:42.680 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.680 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.680 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.680 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.680 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_promtail": { 2024-09-10T10:04:42.680 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_promtail", 2024-09-10T10:04:42.680 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.681 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.681 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.681 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-10T10:04:42.681 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.681 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.681 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.681 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Promtail container image", 2024-09-10T10:04:42.681 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.681 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.681 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.681 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.681 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_samba": { 2024-09-10T10:04:42.681 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_samba", 2024-09-10T10:04:42.681 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.681 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.682 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.682 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-10T10:04:42.682 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.682 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.682 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.682 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Samba/SMB container image", 2024-09-10T10:04:42.682 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.682 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.682 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.682 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.682 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_snmp_gateway": { 2024-09-10T10:04:42.682 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_snmp_gateway", 2024-09-10T10:04:42.682 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.683 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.683 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.683 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-10T10:04:42.683 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.683 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.683 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.683 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "SNMP Gateway container image", 2024-09-10T10:04:42.683 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.683 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.683 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.683 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.683 INFO:teuthology.orchestra.run.smithi077.stdout: "container_init": { 2024-09-10T10:04:42.683 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_init", 2024-09-10T10:04:42.683 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.683 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.684 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.684 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:42.684 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.684 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.684 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.684 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Run podman/docker with `--init`", 2024-09-10T10:04:42.684 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.684 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.684 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.684 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.685 INFO:teuthology.orchestra.run.smithi077.stdout: "daemon_cache_timeout": { 2024-09-10T10:04:42.685 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "daemon_cache_timeout", 2024-09-10T10:04:42.685 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-10T10:04:42.685 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.685 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.685 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "600", 2024-09-10T10:04:42.685 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.685 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.685 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.685 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-10T10:04:42.685 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.685 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.686 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.686 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.686 INFO:teuthology.orchestra.run.smithi077.stdout: "default_cephadm_command_timeout": { 2024-09-10T10:04:42.686 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "default_cephadm_command_timeout", 2024-09-10T10:04:42.686 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:42.686 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.686 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.686 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "900", 2024-09-10T10:04:42.686 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.686 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.687 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.687 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-10T10:04:42.687 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.687 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.687 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.687 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.687 INFO:teuthology.orchestra.run.smithi077.stdout: "default_registry": { 2024-09-10T10:04:42.687 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "default_registry", 2024-09-10T10:04:42.687 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.687 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.687 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.687 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "docker.io", 2024-09-10T10:04:42.688 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.688 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.688 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.688 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-10T10:04:42.688 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.688 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.688 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.688 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.688 INFO:teuthology.orchestra.run.smithi077.stdout: "device_cache_timeout": { 2024-09-10T10:04:42.688 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "device_cache_timeout", 2024-09-10T10:04:42.688 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-10T10:04:42.689 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.689 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.689 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "1800", 2024-09-10T10:04:42.689 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.689 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.689 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.689 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "seconds to cache device inventory", 2024-09-10T10:04:42.689 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.689 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.689 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.689 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.689 INFO:teuthology.orchestra.run.smithi077.stdout: "device_enhanced_scan": { 2024-09-10T10:04:42.690 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "device_enhanced_scan", 2024-09-10T10:04:42.690 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.690 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.690 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.690 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:42.690 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.690 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.690 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.690 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-10T10:04:42.690 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.690 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.691 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.691 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.691 INFO:teuthology.orchestra.run.smithi077.stdout: "facts_cache_timeout": { 2024-09-10T10:04:42.691 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "facts_cache_timeout", 2024-09-10T10:04:42.691 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-10T10:04:42.691 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.691 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.691 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "60", 2024-09-10T10:04:42.691 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.691 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.691 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.692 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "seconds to cache host facts data", 2024-09-10T10:04:42.692 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.692 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.692 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.692 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.692 INFO:teuthology.orchestra.run.smithi077.stdout: "grafana_dashboards_path": { 2024-09-10T10:04:42.692 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "grafana_dashboards_path", 2024-09-10T10:04:42.692 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.692 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.692 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.692 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-10T10:04:42.692 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.693 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.693 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.693 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-10T10:04:42.693 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.693 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.693 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.693 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.693 INFO:teuthology.orchestra.run.smithi077.stdout: "host_check_interval": { 2024-09-10T10:04:42.693 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "host_check_interval", 2024-09-10T10:04:42.693 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-10T10:04:42.693 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.694 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.694 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "600", 2024-09-10T10:04:42.694 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.694 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.694 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.694 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "how frequently to perform a host check", 2024-09-10T10:04:42.694 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.694 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.694 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.694 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.694 INFO:teuthology.orchestra.run.smithi077.stdout: "hw_monitoring": { 2024-09-10T10:04:42.694 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "hw_monitoring", 2024-09-10T10:04:42.695 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.695 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.695 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.695 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:42.695 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.695 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.695 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.695 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-10T10:04:42.695 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.695 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.695 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.695 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.696 INFO:teuthology.orchestra.run.smithi077.stdout: "inventory_list_all": { 2024-09-10T10:04:42.696 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "inventory_list_all", 2024-09-10T10:04:42.696 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.696 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.696 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.696 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:42.696 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.696 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.696 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.696 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-10T10:04:42.696 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.696 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.696 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.697 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.697 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:42.697 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:42.697 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.697 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.697 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.697 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:42.697 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.697 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.697 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:42.697 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:42.697 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:42.697 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:42.697 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:42.698 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:42.698 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:42.698 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:42.698 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.698 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.698 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.698 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.698 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.698 INFO:teuthology.orchestra.run.smithi077.stdout: "log_refresh_metadata": { 2024-09-10T10:04:42.698 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_refresh_metadata", 2024-09-10T10:04:42.699 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.699 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.699 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.699 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:42.699 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.699 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.699 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.699 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-10T10:04:42.699 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.699 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.699 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.699 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.700 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:42.700 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:42.700 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.700 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.700 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.700 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:42.700 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.700 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.700 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.700 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-10T10:04:42.700 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.700 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.700 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.700 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.701 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:42.701 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:42.701 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.701 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.701 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.701 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:42.701 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.701 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.701 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:42.701 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:42.701 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:42.701 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:42.701 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:42.702 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:42.702 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:42.702 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:42.702 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.702 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.702 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.702 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.702 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.702 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:42.702 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:42.702 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.702 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.702 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.702 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:42.703 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.703 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.703 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.703 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.703 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.703 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.703 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.703 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.703 INFO:teuthology.orchestra.run.smithi077.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-10T10:04:42.703 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-10T10:04:42.703 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.703 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.703 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.703 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:42.704 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.704 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.704 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.704 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-10T10:04:42.704 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.704 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.704 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.704 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.704 INFO:teuthology.orchestra.run.smithi077.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-10T10:04:42.704 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-10T10:04:42.704 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.704 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.704 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.704 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "*", 2024-09-10T10:04:42.705 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.705 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.705 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.705 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-10T10:04:42.705 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.705 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.705 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.705 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.705 INFO:teuthology.orchestra.run.smithi077.stdout: "max_count_per_host": { 2024-09-10T10:04:42.705 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "max_count_per_host", 2024-09-10T10:04:42.705 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:42.705 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.705 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.705 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10", 2024-09-10T10:04:42.706 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.706 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.706 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.706 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "max number of daemons per service per host", 2024-09-10T10:04:42.706 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.706 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.706 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.706 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.706 INFO:teuthology.orchestra.run.smithi077.stdout: "max_osd_draining_count": { 2024-09-10T10:04:42.706 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "max_osd_draining_count", 2024-09-10T10:04:42.706 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:42.706 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.706 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.706 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10", 2024-09-10T10:04:42.707 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.707 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.707 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.707 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-10T10:04:42.707 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.707 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.707 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.707 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.707 INFO:teuthology.orchestra.run.smithi077.stdout: "migration_current": { 2024-09-10T10:04:42.707 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "migration_current", 2024-09-10T10:04:42.707 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:42.707 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.707 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.708 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:42.708 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.708 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.708 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.708 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "internal - do not modify", 2024-09-10T10:04:42.708 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.708 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.708 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.708 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.708 INFO:teuthology.orchestra.run.smithi077.stdout: "mode": { 2024-09-10T10:04:42.708 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "mode", 2024-09-10T10:04:42.708 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.708 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.708 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.708 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "root", 2024-09-10T10:04:42.709 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.709 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.709 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:42.709 INFO:teuthology.orchestra.run.smithi077.stdout: "cephadm-package", 2024-09-10T10:04:42.709 INFO:teuthology.orchestra.run.smithi077.stdout: "root" 2024-09-10T10:04:42.709 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:42.709 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "mode for remote execution of cephadm", 2024-09-10T10:04:42.709 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.709 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.709 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.709 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.709 INFO:teuthology.orchestra.run.smithi077.stdout: "oob_default_addr": { 2024-09-10T10:04:42.709 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "oob_default_addr", 2024-09-10T10:04:42.709 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.710 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.710 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.710 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "169.254.1.1", 2024-09-10T10:04:42.710 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.710 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.710 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.710 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-10T10:04:42.710 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.710 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.710 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.710 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.710 INFO:teuthology.orchestra.run.smithi077.stdout: "prometheus_alerts_path": { 2024-09-10T10:04:42.710 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "prometheus_alerts_path", 2024-09-10T10:04:42.710 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.711 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.711 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.711 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-10T10:04:42.711 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.711 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.711 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.711 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-10T10:04:42.711 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.711 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.711 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.711 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.711 INFO:teuthology.orchestra.run.smithi077.stdout: "registry_insecure": { 2024-09-10T10:04:42.711 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "registry_insecure", 2024-09-10T10:04:42.711 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.712 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.712 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.712 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:42.712 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.712 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.712 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.712 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-10T10:04:42.712 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.712 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.712 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.712 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.712 INFO:teuthology.orchestra.run.smithi077.stdout: "registry_password": { 2024-09-10T10:04:42.713 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "registry_password", 2024-09-10T10:04:42.713 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.713 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.713 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.713 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:42.713 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.713 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.713 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.713 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-10T10:04:42.713 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.713 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.713 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.714 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.714 INFO:teuthology.orchestra.run.smithi077.stdout: "registry_url": { 2024-09-10T10:04:42.714 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "registry_url", 2024-09-10T10:04:42.714 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.714 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.714 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.714 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:42.714 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.714 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.714 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.714 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-10T10:04:42.714 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.714 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.714 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.714 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.715 INFO:teuthology.orchestra.run.smithi077.stdout: "registry_username": { 2024-09-10T10:04:42.715 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "registry_username", 2024-09-10T10:04:42.715 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.715 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.715 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.715 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:42.715 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.715 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.715 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.715 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-10T10:04:42.715 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.715 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.715 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.715 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.716 INFO:teuthology.orchestra.run.smithi077.stdout: "secure_monitoring_stack": { 2024-09-10T10:04:42.716 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "secure_monitoring_stack", 2024-09-10T10:04:42.716 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.716 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.716 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.716 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:42.716 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.716 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.716 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.716 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-10T10:04:42.716 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.716 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.716 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.716 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.717 INFO:teuthology.orchestra.run.smithi077.stdout: "service_discovery_port": { 2024-09-10T10:04:42.717 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "service_discovery_port", 2024-09-10T10:04:42.717 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:42.717 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.717 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.717 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "8765", 2024-09-10T10:04:42.717 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.717 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.717 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.717 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "cephadm service discovery port", 2024-09-10T10:04:42.717 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.717 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.717 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.718 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.718 INFO:teuthology.orchestra.run.smithi077.stdout: "ssh_config_file": { 2024-09-10T10:04:42.718 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ssh_config_file", 2024-09-10T10:04:42.718 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.718 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.718 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.718 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:42.718 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.718 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.718 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.718 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-10T10:04:42.718 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.718 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.718 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.718 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.718 INFO:teuthology.orchestra.run.smithi077.stdout: "use_agent": { 2024-09-10T10:04:42.719 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "use_agent", 2024-09-10T10:04:42.719 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.719 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.719 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.719 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:42.719 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.719 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.719 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.719 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-10T10:04:42.719 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.719 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.719 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.719 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.720 INFO:teuthology.orchestra.run.smithi077.stdout: "use_repo_digest": { 2024-09-10T10:04:42.720 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "use_repo_digest", 2024-09-10T10:04:42.720 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.720 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.720 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.720 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:42.720 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.720 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.720 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.720 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-10T10:04:42.720 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.720 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.720 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.721 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.721 INFO:teuthology.orchestra.run.smithi077.stdout: "warn_on_failed_host_check": { 2024-09-10T10:04:42.721 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "warn_on_failed_host_check", 2024-09-10T10:04:42.721 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.721 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.721 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.721 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:42.721 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.721 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.721 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.721 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "raise a health warning if the host check fails", 2024-09-10T10:04:42.721 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.721 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.722 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.722 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.722 INFO:teuthology.orchestra.run.smithi077.stdout: "warn_on_stray_daemons": { 2024-09-10T10:04:42.722 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "warn_on_stray_daemons", 2024-09-10T10:04:42.722 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.722 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.722 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.722 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:42.722 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.722 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.722 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.722 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-10T10:04:42.722 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.722 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.723 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.723 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.723 INFO:teuthology.orchestra.run.smithi077.stdout: "warn_on_stray_hosts": { 2024-09-10T10:04:42.723 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "warn_on_stray_hosts", 2024-09-10T10:04:42.723 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.723 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.723 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.723 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:42.723 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.723 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.723 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.723 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-10T10:04:42.723 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.724 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.724 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.724 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:42.724 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:42.724 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.724 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:42.724 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "crash", 2024-09-10T10:04:42.724 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:42.724 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:42.724 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:42.725 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:42.725 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:42.725 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.725 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.725 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.725 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:42.725 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.725 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.725 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:42.725 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:42.725 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:42.725 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:42.726 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:42.726 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:42.726 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:42.726 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:42.726 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.726 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.726 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.726 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.726 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.726 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:42.726 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:42.726 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.726 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.727 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.727 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:42.727 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.727 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.727 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.727 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.727 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.727 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.727 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.727 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.727 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:42.727 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:42.727 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.727 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.728 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.728 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:42.728 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.728 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.728 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:42.728 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:42.728 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:42.728 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:42.728 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:42.728 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:42.728 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:42.728 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:42.728 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.729 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.729 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.729 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.729 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.729 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:42.729 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:42.729 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.729 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.729 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.729 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:42.729 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.729 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.729 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.729 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.729 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.730 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.730 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.730 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.730 INFO:teuthology.orchestra.run.smithi077.stdout: "retain_interval": { 2024-09-10T10:04:42.730 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "retain_interval", 2024-09-10T10:04:42.730 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-10T10:04:42.730 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.730 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.730 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "31536000", 2024-09-10T10:04:42.730 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.730 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.730 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.730 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-10T10:04:42.731 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.731 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.731 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.731 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.731 INFO:teuthology.orchestra.run.smithi077.stdout: "warn_recent_interval": { 2024-09-10T10:04:42.731 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "warn_recent_interval", 2024-09-10T10:04:42.731 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-10T10:04:42.731 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.731 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.731 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "1209600", 2024-09-10T10:04:42.731 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.731 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.731 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.731 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-10T10:04:42.732 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.732 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.732 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.732 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:42.732 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:42.732 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.732 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:42.732 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "dashboard", 2024-09-10T10:04:42.732 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:42.732 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:42.732 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:42.732 INFO:teuthology.orchestra.run.smithi077.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-10T10:04:42.732 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-10T10:04:42.732 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:42.733 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.733 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.733 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10", 2024-09-10T10:04:42.733 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.733 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.733 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.733 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.733 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.733 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.733 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.733 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.734 INFO:teuthology.orchestra.run.smithi077.stdout: "ALERTMANAGER_API_HOST": { 2024-09-10T10:04:42.734 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-10T10:04:42.734 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.734 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.734 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.734 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:42.734 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.734 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.734 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.734 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.734 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.735 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.735 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.735 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.735 INFO:teuthology.orchestra.run.smithi077.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-10T10:04:42.735 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-10T10:04:42.735 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.735 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.735 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.735 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:42.735 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.735 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.735 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.735 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.736 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.736 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.736 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.736 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.736 INFO:teuthology.orchestra.run.smithi077.stdout: "AUDIT_API_ENABLED": { 2024-09-10T10:04:42.736 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "AUDIT_API_ENABLED", 2024-09-10T10:04:42.736 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.736 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.736 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.736 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:42.736 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.737 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.737 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.737 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.737 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.737 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.737 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.737 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.737 INFO:teuthology.orchestra.run.smithi077.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-10T10:04:42.737 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-10T10:04:42.737 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.737 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.737 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.737 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:42.738 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.738 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.738 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.738 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.738 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.738 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.738 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.738 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.738 INFO:teuthology.orchestra.run.smithi077.stdout: "ENABLE_BROWSABLE_API": { 2024-09-10T10:04:42.738 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-10T10:04:42.738 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.739 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.739 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.739 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:42.739 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.739 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.739 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.739 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.739 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.739 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.739 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.739 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.740 INFO:teuthology.orchestra.run.smithi077.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-10T10:04:42.740 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-10T10:04:42.740 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.740 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.740 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.740 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:42.740 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.740 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.740 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.740 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.740 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.741 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.741 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.741 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.741 INFO:teuthology.orchestra.run.smithi077.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-10T10:04:42.741 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-10T10:04:42.741 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.741 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.741 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.741 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:42.741 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.741 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.742 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.742 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.742 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.742 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.742 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.742 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.742 INFO:teuthology.orchestra.run.smithi077.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-10T10:04:42.742 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-10T10:04:42.742 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.742 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.742 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.743 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:42.743 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.743 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.743 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.743 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.743 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.743 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.743 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.743 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.743 INFO:teuthology.orchestra.run.smithi077.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-10T10:04:42.743 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-10T10:04:42.743 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.744 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.744 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.744 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:42.744 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.744 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.744 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.744 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.744 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.744 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.744 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.744 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.744 INFO:teuthology.orchestra.run.smithi077.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-10T10:04:42.744 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-10T10:04:42.745 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.745 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.745 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.745 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:42.745 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.745 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.745 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.745 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.745 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.745 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.745 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.745 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.745 INFO:teuthology.orchestra.run.smithi077.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-10T10:04:42.746 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-10T10:04:42.746 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.746 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.746 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.746 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:42.746 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.746 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.746 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.746 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.746 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.746 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.746 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.746 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.746 INFO:teuthology.orchestra.run.smithi077.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-10T10:04:42.747 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-10T10:04:42.747 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.747 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.747 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.747 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:42.747 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.747 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.747 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.747 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.747 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.747 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.747 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.747 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.747 INFO:teuthology.orchestra.run.smithi077.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-10T10:04:42.748 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-10T10:04:42.748 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.748 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.748 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.748 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:42.748 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.748 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.748 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.748 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.748 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.748 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.748 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.748 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.748 INFO:teuthology.orchestra.run.smithi077.stdout: "GRAFANA_API_PASSWORD": { 2024-09-10T10:04:42.749 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-10T10:04:42.749 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.749 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.749 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.749 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "admin", 2024-09-10T10:04:42.749 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.749 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.749 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.749 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.749 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.749 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.749 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.749 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.749 INFO:teuthology.orchestra.run.smithi077.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-10T10:04:42.749 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-10T10:04:42.750 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.750 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.750 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.750 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:42.750 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.750 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.750 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.750 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.750 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.750 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.750 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.750 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.750 INFO:teuthology.orchestra.run.smithi077.stdout: "GRAFANA_API_URL": { 2024-09-10T10:04:42.750 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "GRAFANA_API_URL", 2024-09-10T10:04:42.751 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.751 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.751 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.751 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:42.751 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.751 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.751 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.751 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.751 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.751 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.751 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.751 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.751 INFO:teuthology.orchestra.run.smithi077.stdout: "GRAFANA_API_USERNAME": { 2024-09-10T10:04:42.751 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-10T10:04:42.751 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.752 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.752 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.752 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "admin", 2024-09-10T10:04:42.752 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.752 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.752 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.752 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.752 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.752 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.752 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.752 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.752 INFO:teuthology.orchestra.run.smithi077.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-10T10:04:42.752 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-10T10:04:42.752 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.753 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.753 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.753 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:42.753 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.753 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.753 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.753 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.753 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.753 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.753 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.753 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.753 INFO:teuthology.orchestra.run.smithi077.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-10T10:04:42.753 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-10T10:04:42.753 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.754 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.754 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.754 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:42.754 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.754 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.754 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.754 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.754 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.754 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.754 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.754 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.754 INFO:teuthology.orchestra.run.smithi077.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-10T10:04:42.754 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-10T10:04:42.755 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.755 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.755 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.755 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:42.755 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.755 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.755 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.755 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.755 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.755 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.755 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.755 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.755 INFO:teuthology.orchestra.run.smithi077.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-10T10:04:42.755 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-10T10:04:42.756 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.756 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.756 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.756 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:42.756 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.756 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.756 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.756 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.756 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.756 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.756 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.756 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.757 INFO:teuthology.orchestra.run.smithi077.stdout: "PROMETHEUS_API_HOST": { 2024-09-10T10:04:42.757 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-10T10:04:42.757 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.757 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.757 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.757 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:42.757 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.757 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.757 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.757 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.757 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.757 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.757 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.758 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.758 INFO:teuthology.orchestra.run.smithi077.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-10T10:04:42.758 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-10T10:04:42.758 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.758 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.758 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.758 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:42.758 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.758 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.758 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.759 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.759 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.759 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.759 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.759 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.759 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-10T10:04:42.759 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-10T10:04:42.759 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.760 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.760 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.760 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:42.760 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.760 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.760 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.760 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.760 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.760 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.760 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.760 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.760 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-10T10:04:42.761 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-10T10:04:42.761 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.761 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.761 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.761 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:42.761 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.761 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.761 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.761 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.761 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.761 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.761 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.761 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.762 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-10T10:04:42.762 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-10T10:04:42.762 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.762 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.762 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.762 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:42.762 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.762 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.762 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.762 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.762 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.762 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.762 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.763 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.763 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-10T10:04:42.763 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-10T10:04:42.763 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.763 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.763 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.763 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:42.763 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.763 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.763 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.763 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.763 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.763 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.764 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.764 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.764 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-10T10:04:42.764 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-10T10:04:42.764 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.764 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.764 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.764 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:42.764 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.764 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.764 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.764 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.764 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.765 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.765 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.765 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.765 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-10T10:04:42.765 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-10T10:04:42.765 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.765 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.765 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.765 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:42.765 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.765 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.765 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.765 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.766 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.766 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.766 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.766 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.766 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-10T10:04:42.766 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-10T10:04:42.766 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.766 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.766 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.766 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:42.766 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.766 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.767 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.767 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.767 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.767 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.767 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.767 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.767 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_ENABLED": { 2024-09-10T10:04:42.767 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-10T10:04:42.767 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.767 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.767 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.767 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:42.767 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.767 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.767 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.768 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.768 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.768 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.768 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.768 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.768 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-10T10:04:42.768 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-10T10:04:42.768 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.768 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.768 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.768 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-10T10:04:42.768 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.768 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.769 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.769 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.769 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.769 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.769 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.769 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.769 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-10T10:04:42.769 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-10T10:04:42.769 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:42.769 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.769 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.769 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10", 2024-09-10T10:04:42.769 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.769 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.770 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.770 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.770 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.770 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.770 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.770 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.770 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-10T10:04:42.770 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-10T10:04:42.770 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:42.770 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.770 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.770 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "8", 2024-09-10T10:04:42.770 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.770 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.770 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.771 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.771 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.771 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.771 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.771 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.771 INFO:teuthology.orchestra.run.smithi077.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-10T10:04:42.771 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-10T10:04:42.771 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:42.771 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.771 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.771 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "45", 2024-09-10T10:04:42.771 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.771 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.772 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.772 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.772 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.772 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.772 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.772 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.772 INFO:teuthology.orchestra.run.smithi077.stdout: "RGW_API_ACCESS_KEY": { 2024-09-10T10:04:42.772 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-10T10:04:42.772 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.772 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.772 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.772 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:42.772 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.772 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.773 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.773 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.773 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.773 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.773 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.773 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.773 INFO:teuthology.orchestra.run.smithi077.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-10T10:04:42.773 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-10T10:04:42.773 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.773 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.773 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.773 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "admin", 2024-09-10T10:04:42.773 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.773 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.774 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.774 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.774 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.774 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.774 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.774 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.774 INFO:teuthology.orchestra.run.smithi077.stdout: "RGW_API_SECRET_KEY": { 2024-09-10T10:04:42.774 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-10T10:04:42.774 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.774 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.774 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.774 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:42.774 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.774 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.774 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.774 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.775 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.775 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.775 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.775 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.775 INFO:teuthology.orchestra.run.smithi077.stdout: "RGW_API_SSL_VERIFY": { 2024-09-10T10:04:42.775 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-10T10:04:42.775 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.775 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.775 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.775 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:42.775 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.775 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.775 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.776 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.776 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.776 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.776 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.776 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.776 INFO:teuthology.orchestra.run.smithi077.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-10T10:04:42.776 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-10T10:04:42.776 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:42.776 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.776 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.776 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-09-10T10:04:42.776 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.776 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.776 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.777 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.777 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.777 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.777 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.777 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.777 INFO:teuthology.orchestra.run.smithi077.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-10T10:04:42.777 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-10T10:04:42.777 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:42.777 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.777 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.777 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10", 2024-09-10T10:04:42.777 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.777 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.777 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.778 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.778 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.778 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.778 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.778 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.778 INFO:teuthology.orchestra.run.smithi077.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-10T10:04:42.778 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-10T10:04:42.778 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:42.778 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.778 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.778 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "5", 2024-09-10T10:04:42.778 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.779 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.779 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.779 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.779 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.779 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.779 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.779 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.779 INFO:teuthology.orchestra.run.smithi077.stdout: "cross_origin_url": { 2024-09-10T10:04:42.779 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "cross_origin_url", 2024-09-10T10:04:42.779 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.779 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.779 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.779 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:42.779 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.779 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.780 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.780 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.780 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.780 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.780 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.780 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.780 INFO:teuthology.orchestra.run.smithi077.stdout: "crt_file": { 2024-09-10T10:04:42.780 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "crt_file", 2024-09-10T10:04:42.780 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.780 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.780 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.780 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:42.780 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.781 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.781 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.781 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.781 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.781 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.781 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.781 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.781 INFO:teuthology.orchestra.run.smithi077.stdout: "debug": { 2024-09-10T10:04:42.781 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "debug", 2024-09-10T10:04:42.781 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.781 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.781 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.782 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:42.782 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.782 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.782 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.782 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Enable/disable debug options", 2024-09-10T10:04:42.782 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.782 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.782 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.782 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.782 INFO:teuthology.orchestra.run.smithi077.stdout: "jwt_token_ttl": { 2024-09-10T10:04:42.782 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "jwt_token_ttl", 2024-09-10T10:04:42.782 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:42.782 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.782 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.783 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "28800", 2024-09-10T10:04:42.783 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.783 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.783 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.783 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.783 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.783 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.783 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.783 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.783 INFO:teuthology.orchestra.run.smithi077.stdout: "key_file": { 2024-09-10T10:04:42.783 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "key_file", 2024-09-10T10:04:42.783 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.783 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.784 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.784 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:42.784 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.784 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.784 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.784 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.784 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.784 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.784 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.784 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.784 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:42.784 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:42.784 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.785 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.785 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.785 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:42.785 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.785 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.785 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:42.785 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:42.785 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:42.785 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:42.785 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:42.785 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:42.785 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:42.785 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:42.786 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.786 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.786 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.786 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.786 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.786 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:42.786 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:42.786 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.786 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.786 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.786 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:42.786 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.786 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.786 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.787 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.787 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.787 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.787 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.787 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.787 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:42.787 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:42.787 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.787 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.787 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.787 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:42.787 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.787 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.788 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:42.788 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:42.788 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:42.788 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:42.788 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:42.788 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:42.788 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:42.788 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:42.788 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.788 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.788 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.788 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.788 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.788 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:42.788 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:42.789 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.789 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.789 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.789 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:42.789 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.789 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.789 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.789 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.789 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.789 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.789 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.789 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.789 INFO:teuthology.orchestra.run.smithi077.stdout: "motd": { 2024-09-10T10:04:42.789 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "motd", 2024-09-10T10:04:42.790 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.790 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.790 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.790 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:42.790 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.790 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.790 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.790 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "The message of the day", 2024-09-10T10:04:42.790 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.790 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.790 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.790 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.790 INFO:teuthology.orchestra.run.smithi077.stdout: "redirect_resolve_ip_addr": { 2024-09-10T10:04:42.790 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "redirect_resolve_ip_addr", 2024-09-10T10:04:42.791 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.791 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.791 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.791 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:42.791 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.791 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.791 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.791 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.791 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.791 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.791 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.791 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.791 INFO:teuthology.orchestra.run.smithi077.stdout: "server_addr": { 2024-09-10T10:04:42.791 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "server_addr", 2024-09-10T10:04:42.792 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.792 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.792 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.792 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "::", 2024-09-10T10:04:42.792 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.792 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.792 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.792 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.792 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.792 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.792 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.792 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.792 INFO:teuthology.orchestra.run.smithi077.stdout: "server_port": { 2024-09-10T10:04:42.792 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "server_port", 2024-09-10T10:04:42.792 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:42.793 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.793 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.793 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "8080", 2024-09-10T10:04:42.793 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.793 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.793 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.793 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.793 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.793 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.793 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.793 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.793 INFO:teuthology.orchestra.run.smithi077.stdout: "ssl": { 2024-09-10T10:04:42.793 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ssl", 2024-09-10T10:04:42.793 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.794 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.794 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.794 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:42.794 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.794 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.794 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.794 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.794 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.794 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.794 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.794 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.794 INFO:teuthology.orchestra.run.smithi077.stdout: "ssl_server_port": { 2024-09-10T10:04:42.794 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ssl_server_port", 2024-09-10T10:04:42.794 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:42.794 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.795 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.795 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "8443", 2024-09-10T10:04:42.795 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.795 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.795 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.795 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.795 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.795 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.795 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.795 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.795 INFO:teuthology.orchestra.run.smithi077.stdout: "standby_behaviour": { 2024-09-10T10:04:42.795 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "standby_behaviour", 2024-09-10T10:04:42.795 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.795 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.795 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.796 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "redirect", 2024-09-10T10:04:42.796 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.796 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.796 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:42.796 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:42.796 INFO:teuthology.orchestra.run.smithi077.stdout: "redirect" 2024-09-10T10:04:42.796 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:42.796 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.796 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.796 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.796 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.797 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.797 INFO:teuthology.orchestra.run.smithi077.stdout: "standby_error_status_code": { 2024-09-10T10:04:42.797 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "standby_error_status_code", 2024-09-10T10:04:42.797 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:42.797 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.797 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.797 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "500", 2024-09-10T10:04:42.797 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "400", 2024-09-10T10:04:42.797 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "599", 2024-09-10T10:04:42.797 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.797 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.797 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.798 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.798 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.798 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.798 INFO:teuthology.orchestra.run.smithi077.stdout: "url_prefix": { 2024-09-10T10:04:42.798 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "url_prefix", 2024-09-10T10:04:42.798 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.798 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.798 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.798 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:42.798 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.798 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.798 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.798 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.799 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.799 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.799 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.799 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:42.799 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:42.799 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.799 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:42.799 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "devicehealth", 2024-09-10T10:04:42.799 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:42.799 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:42.799 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:42.799 INFO:teuthology.orchestra.run.smithi077.stdout: "enable_monitoring": { 2024-09-10T10:04:42.799 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "enable_monitoring", 2024-09-10T10:04:42.800 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.800 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.800 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.800 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:42.800 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.800 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.800 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.800 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "monitor device health metrics", 2024-09-10T10:04:42.800 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.800 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.800 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.800 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.800 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:42.801 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:42.801 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.801 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.801 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.801 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:42.801 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.801 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.801 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:42.801 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:42.801 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:42.801 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:42.801 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:42.801 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:42.802 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:42.802 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:42.802 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.802 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.802 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.802 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.802 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.802 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:42.802 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:42.802 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.802 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.802 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.802 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:42.802 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.803 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.803 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.803 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.803 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.803 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.803 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.803 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.803 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:42.803 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:42.803 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.803 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.803 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.803 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:42.803 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.804 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.804 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:42.804 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:42.804 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:42.804 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:42.804 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:42.804 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:42.804 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:42.804 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:42.804 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.804 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.804 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.804 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.805 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.805 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:42.805 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:42.805 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.805 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.805 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.805 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:42.805 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.805 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.805 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.805 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.805 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.805 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.805 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.806 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.806 INFO:teuthology.orchestra.run.smithi077.stdout: "mark_out_threshold": { 2024-09-10T10:04:42.806 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "mark_out_threshold", 2024-09-10T10:04:42.806 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-10T10:04:42.806 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.806 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.806 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "2419200", 2024-09-10T10:04:42.806 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.806 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.806 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.806 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-10T10:04:42.806 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.806 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.807 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.807 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.807 INFO:teuthology.orchestra.run.smithi077.stdout: "pool_name": { 2024-09-10T10:04:42.807 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "pool_name", 2024-09-10T10:04:42.807 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.807 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.807 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.807 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "device_health_metrics", 2024-09-10T10:04:42.807 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.807 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.807 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.807 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-10T10:04:42.807 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.807 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.808 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.808 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.808 INFO:teuthology.orchestra.run.smithi077.stdout: "retention_period": { 2024-09-10T10:04:42.808 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "retention_period", 2024-09-10T10:04:42.808 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-10T10:04:42.808 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.808 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.808 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "15552000", 2024-09-10T10:04:42.808 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.808 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.808 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.808 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "how long to retain device health metrics", 2024-09-10T10:04:42.808 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.809 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.809 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.809 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.809 INFO:teuthology.orchestra.run.smithi077.stdout: "scrape_frequency": { 2024-09-10T10:04:42.809 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "scrape_frequency", 2024-09-10T10:04:42.809 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-10T10:04:42.809 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.809 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.809 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "86400", 2024-09-10T10:04:42.809 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.809 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.809 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.810 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-10T10:04:42.810 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.810 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.810 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.810 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.810 INFO:teuthology.orchestra.run.smithi077.stdout: "self_heal": { 2024-09-10T10:04:42.810 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "self_heal", 2024-09-10T10:04:42.810 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.810 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.810 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.810 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:42.810 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.810 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.810 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.811 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-10T10:04:42.811 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.811 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.811 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.811 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.811 INFO:teuthology.orchestra.run.smithi077.stdout: "sleep_interval": { 2024-09-10T10:04:42.811 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sleep_interval", 2024-09-10T10:04:42.811 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-10T10:04:42.811 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.811 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.811 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "600", 2024-09-10T10:04:42.811 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.811 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.811 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.812 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "how frequently to wake up and check device health", 2024-09-10T10:04:42.812 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.812 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.812 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.812 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.812 INFO:teuthology.orchestra.run.smithi077.stdout: "warn_threshold": { 2024-09-10T10:04:42.812 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "warn_threshold", 2024-09-10T10:04:42.812 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-10T10:04:42.812 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.812 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.812 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "7257600", 2024-09-10T10:04:42.813 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.813 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.813 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.813 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-10T10:04:42.813 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.813 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.813 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.813 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:42.813 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:42.813 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.813 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:42.813 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "diskprediction_local", 2024-09-10T10:04:42.813 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:42.813 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:42.814 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:42.814 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:42.814 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:42.814 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.814 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.814 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.814 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:42.814 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.814 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.814 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:42.814 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:42.814 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:42.814 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:42.815 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:42.815 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:42.815 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:42.815 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:42.815 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.815 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.815 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.815 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.815 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.815 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:42.815 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:42.815 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.815 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.815 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.816 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:42.816 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.816 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.816 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.816 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.816 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.816 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.816 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.816 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.816 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:42.816 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:42.816 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.816 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.816 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.817 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:42.817 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.817 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.817 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:42.817 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:42.817 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:42.817 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:42.817 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:42.817 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:42.817 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:42.817 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:42.817 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.817 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.818 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.818 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.818 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.818 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:42.818 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:42.818 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.818 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.818 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.818 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:42.818 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.818 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.818 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.818 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.818 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.818 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.819 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.819 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.819 INFO:teuthology.orchestra.run.smithi077.stdout: "predict_interval": { 2024-09-10T10:04:42.819 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "predict_interval", 2024-09-10T10:04:42.819 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.819 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.819 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.819 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "86400", 2024-09-10T10:04:42.819 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.819 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.819 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.819 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.820 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.820 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.820 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.820 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.820 INFO:teuthology.orchestra.run.smithi077.stdout: "predictor_model": { 2024-09-10T10:04:42.820 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "predictor_model", 2024-09-10T10:04:42.820 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.820 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.820 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.820 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "prophetstor", 2024-09-10T10:04:42.820 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.821 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.821 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.821 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.821 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.821 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.821 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.821 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.821 INFO:teuthology.orchestra.run.smithi077.stdout: "sleep_interval": { 2024-09-10T10:04:42.821 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sleep_interval", 2024-09-10T10:04:42.821 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.821 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.821 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.821 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "600", 2024-09-10T10:04:42.821 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.821 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.822 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.822 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.822 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.822 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.822 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.822 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:42.822 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:42.822 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.822 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:42.822 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "influx", 2024-09-10T10:04:42.822 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": false, 2024-09-10T10:04:42.822 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "influxdb python module not found", 2024-09-10T10:04:42.822 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:42.822 INFO:teuthology.orchestra.run.smithi077.stdout: "batch_size": { 2024-09-10T10:04:42.823 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "batch_size", 2024-09-10T10:04:42.823 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:42.823 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.823 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.823 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "5000", 2024-09-10T10:04:42.823 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.823 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.823 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.823 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-10T10:04:42.823 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.823 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.823 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.823 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.823 INFO:teuthology.orchestra.run.smithi077.stdout: "database": { 2024-09-10T10:04:42.824 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "database", 2024-09-10T10:04:42.824 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.824 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.824 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.824 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "ceph", 2024-09-10T10:04:42.824 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.824 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.824 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.824 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-10T10:04:42.824 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.824 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.824 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.824 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.824 INFO:teuthology.orchestra.run.smithi077.stdout: "hostname": { 2024-09-10T10:04:42.825 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "hostname", 2024-09-10T10:04:42.825 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.825 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.825 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.825 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:42.825 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.825 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.825 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.825 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "InfluxDB server hostname", 2024-09-10T10:04:42.825 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.825 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.825 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.825 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.825 INFO:teuthology.orchestra.run.smithi077.stdout: "interval": { 2024-09-10T10:04:42.826 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "interval", 2024-09-10T10:04:42.826 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-10T10:04:42.826 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.826 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.826 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "30", 2024-09-10T10:04:42.826 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "5", 2024-09-10T10:04:42.826 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.826 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.826 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-10T10:04:42.826 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.826 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.826 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.826 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.826 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:42.827 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:42.827 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.827 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.827 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.827 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:42.827 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.827 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.827 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:42.827 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:42.827 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:42.827 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:42.827 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:42.827 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:42.828 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:42.828 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:42.828 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.828 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.828 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.828 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.828 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.828 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:42.828 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:42.828 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.828 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.828 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.828 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:42.828 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.829 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.829 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.829 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.829 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.829 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.829 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.829 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.829 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:42.829 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:42.829 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.829 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.829 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.829 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:42.830 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.830 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.830 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:42.830 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:42.830 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:42.830 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:42.830 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:42.830 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:42.830 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:42.830 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:42.830 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.830 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.830 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.830 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.831 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.831 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:42.831 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:42.831 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.831 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.831 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.831 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:42.831 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.831 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.831 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.831 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.831 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.831 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.831 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.832 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.832 INFO:teuthology.orchestra.run.smithi077.stdout: "password": { 2024-09-10T10:04:42.832 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "password", 2024-09-10T10:04:42.832 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.832 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.832 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.832 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:42.832 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.832 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.832 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.832 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "password of InfluxDB server user", 2024-09-10T10:04:42.832 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.832 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.832 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.833 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.833 INFO:teuthology.orchestra.run.smithi077.stdout: "port": { 2024-09-10T10:04:42.833 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "port", 2024-09-10T10:04:42.833 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:42.833 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.833 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.833 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "8086", 2024-09-10T10:04:42.833 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.833 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.833 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.833 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "InfluxDB server port", 2024-09-10T10:04:42.833 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.833 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.834 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.834 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.834 INFO:teuthology.orchestra.run.smithi077.stdout: "ssl": { 2024-09-10T10:04:42.834 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ssl", 2024-09-10T10:04:42.834 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.834 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.834 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.834 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "false", 2024-09-10T10:04:42.834 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.834 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.834 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.834 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-10T10:04:42.834 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.834 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.834 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.834 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.835 INFO:teuthology.orchestra.run.smithi077.stdout: "threads": { 2024-09-10T10:04:42.835 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "threads", 2024-09-10T10:04:42.835 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:42.835 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.835 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.835 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "5", 2024-09-10T10:04:42.835 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "1", 2024-09-10T10:04:42.835 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "32", 2024-09-10T10:04:42.835 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.835 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-10T10:04:42.835 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.835 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.835 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.835 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.835 INFO:teuthology.orchestra.run.smithi077.stdout: "username": { 2024-09-10T10:04:42.836 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "username", 2024-09-10T10:04:42.836 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.836 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.836 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.836 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:42.836 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.836 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.836 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.836 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "username of InfluxDB server user", 2024-09-10T10:04:42.836 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.836 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.836 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.836 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.837 INFO:teuthology.orchestra.run.smithi077.stdout: "verify_ssl": { 2024-09-10T10:04:42.837 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "verify_ssl", 2024-09-10T10:04:42.837 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.837 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.837 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.837 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "true", 2024-09-10T10:04:42.837 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.837 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.837 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.837 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-10T10:04:42.837 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.837 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.837 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.838 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:42.838 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:42.838 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.838 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:42.838 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "insights", 2024-09-10T10:04:42.838 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:42.838 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:42.838 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:42.838 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:42.838 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:42.838 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.838 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.838 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.838 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:42.838 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.839 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.839 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:42.839 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:42.839 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:42.839 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:42.839 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:42.839 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:42.839 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:42.839 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:42.839 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.839 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.839 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.839 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.840 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.840 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:42.840 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:42.840 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.840 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.840 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.840 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:42.840 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.840 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.840 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.840 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.840 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.840 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.841 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.841 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.841 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:42.841 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:42.841 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.841 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.841 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.841 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:42.841 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.841 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.841 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:42.841 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:42.841 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:42.841 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:42.842 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:42.842 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:42.842 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:42.842 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:42.842 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.842 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.842 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.842 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.842 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.842 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:42.842 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:42.842 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.842 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.843 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.843 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:42.843 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.843 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.843 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.843 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.843 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.843 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.843 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.843 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:42.843 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:42.843 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.843 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:42.843 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "iostat", 2024-09-10T10:04:42.844 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:42.844 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:42.844 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:42.844 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:42.844 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:42.844 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.844 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.844 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.844 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:42.844 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.844 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.844 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:42.845 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:42.845 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:42.845 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:42.845 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:42.845 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:42.845 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:42.845 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:42.845 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.845 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.845 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.845 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.845 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.845 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:42.845 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:42.846 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.846 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.846 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.846 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:42.846 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.846 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.846 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.846 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.846 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.846 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.846 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.846 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.846 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:42.847 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:42.847 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.847 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.847 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.847 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:42.847 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.847 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.847 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:42.847 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:42.847 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:42.847 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:42.847 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:42.847 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:42.847 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:42.848 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:42.848 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.848 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.848 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.848 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.848 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.848 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:42.848 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:42.848 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.848 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.848 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.848 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:42.848 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.849 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.849 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.849 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.849 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.849 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.849 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.849 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:42.849 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:42.849 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.849 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:42.849 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "k8sevents", 2024-09-10T10:04:42.849 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:42.849 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:42.849 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:42.849 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_event_retention_days": { 2024-09-10T10:04:42.850 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ceph_event_retention_days", 2024-09-10T10:04:42.850 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:42.850 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.850 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.850 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "7", 2024-09-10T10:04:42.850 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.850 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.850 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.850 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-10T10:04:42.850 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.850 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.850 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.850 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.850 INFO:teuthology.orchestra.run.smithi077.stdout: "config_check_secs": { 2024-09-10T10:04:42.850 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "config_check_secs", 2024-09-10T10:04:42.851 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:42.851 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.851 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.851 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10", 2024-09-10T10:04:42.851 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "10", 2024-09-10T10:04:42.851 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.851 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.851 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-10T10:04:42.851 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.851 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.851 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.851 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.851 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:42.851 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:42.852 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.852 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.852 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.852 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:42.852 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.852 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.852 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:42.852 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:42.852 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:42.852 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:42.852 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:42.852 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:42.852 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:42.852 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:42.853 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.853 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.853 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.853 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.853 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.853 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:42.853 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:42.853 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.853 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.853 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.853 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:42.853 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.853 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.853 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.853 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.854 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.854 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.854 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.854 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.854 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:42.854 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:42.854 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.854 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.854 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.854 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:42.854 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.854 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.854 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:42.854 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:42.854 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:42.855 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:42.855 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:42.855 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:42.855 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:42.855 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:42.855 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.855 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.855 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.855 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.855 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.855 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:42.855 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:42.855 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.856 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.856 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.856 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:42.856 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.856 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.856 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.856 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.856 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.856 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.856 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.856 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:42.856 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:42.856 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.856 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:42.857 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "localpool", 2024-09-10T10:04:42.857 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:42.857 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:42.857 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:42.857 INFO:teuthology.orchestra.run.smithi077.stdout: "failure_domain": { 2024-09-10T10:04:42.857 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "failure_domain", 2024-09-10T10:04:42.857 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.857 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.857 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.857 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "host", 2024-09-10T10:04:42.857 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.857 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.857 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.858 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "failure domain for any created local pool", 2024-09-10T10:04:42.858 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-10T10:04:42.858 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.858 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.858 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.858 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:42.858 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:42.858 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.858 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.858 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.858 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:42.858 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.858 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.858 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:42.858 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:42.859 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:42.859 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:42.859 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:42.859 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:42.859 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:42.859 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:42.859 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.859 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.859 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.859 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.859 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.859 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:42.859 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:42.859 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.860 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.860 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.860 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:42.860 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.860 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.860 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.860 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.860 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.860 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.861 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.861 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.861 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:42.861 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:42.861 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.861 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.861 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.861 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:42.861 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.861 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.861 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:42.861 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:42.861 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:42.862 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:42.862 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:42.862 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:42.862 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:42.862 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:42.862 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.862 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.862 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.862 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.862 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.862 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:42.862 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:42.862 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.862 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.863 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.863 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:42.863 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.863 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.863 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.863 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.863 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.863 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.863 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.863 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.863 INFO:teuthology.orchestra.run.smithi077.stdout: "min_size": { 2024-09-10T10:04:42.863 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "min_size", 2024-09-10T10:04:42.863 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:42.864 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.864 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.864 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:42.864 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.864 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.864 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.864 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "default min_size for any created local pool", 2024-09-10T10:04:42.864 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-10T10:04:42.864 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.864 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.864 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.864 INFO:teuthology.orchestra.run.smithi077.stdout: "num_rep": { 2024-09-10T10:04:42.864 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "num_rep", 2024-09-10T10:04:42.865 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:42.865 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.865 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.865 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "3", 2024-09-10T10:04:42.865 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.865 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.865 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.865 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "default replica count for any created local pool", 2024-09-10T10:04:42.865 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.865 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.865 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.865 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.865 INFO:teuthology.orchestra.run.smithi077.stdout: "pg_num": { 2024-09-10T10:04:42.865 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "pg_num", 2024-09-10T10:04:42.866 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:42.866 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.866 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.866 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "128", 2024-09-10T10:04:42.866 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.866 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.866 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.866 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "default pg_num for any created local pool", 2024-09-10T10:04:42.866 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.866 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.866 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.866 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.866 INFO:teuthology.orchestra.run.smithi077.stdout: "prefix": { 2024-09-10T10:04:42.866 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "prefix", 2024-09-10T10:04:42.867 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.867 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.867 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.867 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:42.867 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.867 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.867 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.867 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "name prefix for any created local pool", 2024-09-10T10:04:42.867 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.867 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.867 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.867 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.867 INFO:teuthology.orchestra.run.smithi077.stdout: "subtree": { 2024-09-10T10:04:42.868 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "subtree", 2024-09-10T10:04:42.868 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.868 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.868 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.868 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "rack", 2024-09-10T10:04:42.868 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.868 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.868 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.868 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-10T10:04:42.868 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-10T10:04:42.868 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.868 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.868 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:42.868 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:42.869 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.869 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:42.869 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "mds_autoscaler", 2024-09-10T10:04:42.869 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:42.869 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:42.869 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:42.869 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:42.869 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:42.869 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.869 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.869 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.869 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:42.869 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.869 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.870 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:42.870 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:42.870 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:42.870 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:42.870 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:42.870 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:42.870 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:42.870 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:42.870 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.870 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.870 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.870 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.870 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.870 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:42.871 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:42.871 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.871 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.871 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.871 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:42.871 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.871 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.871 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.871 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.871 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.871 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.871 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.871 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.872 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:42.872 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:42.872 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.872 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.872 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.872 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:42.872 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.872 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.872 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:42.872 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:42.872 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:42.872 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:42.872 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:42.872 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:42.873 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:42.873 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:42.873 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.873 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.873 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.873 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.873 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.873 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:42.873 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:42.873 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.873 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.873 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.873 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:42.873 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.874 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.874 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.874 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.874 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.874 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.874 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.874 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:42.874 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:42.874 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.874 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:42.874 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "mirroring", 2024-09-10T10:04:42.874 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:42.874 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:42.875 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:42.875 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:42.875 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:42.875 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.875 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.875 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.875 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:42.875 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.875 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.875 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:42.875 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:42.875 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:42.875 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:42.876 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:42.876 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:42.876 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:42.876 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:42.876 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.876 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.876 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.876 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.876 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.876 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:42.876 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:42.877 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.877 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.877 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.877 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:42.877 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.877 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.877 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.877 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.877 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.877 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.877 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.877 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.878 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:42.878 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:42.878 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.878 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.878 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.878 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:42.878 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.878 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.878 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:42.878 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:42.878 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:42.878 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:42.878 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:42.878 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:42.879 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:42.879 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:42.879 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.879 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.879 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.879 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.879 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.879 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:42.879 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:42.879 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.879 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.879 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.879 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:42.880 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.880 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.880 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.880 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.880 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.880 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.880 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.880 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:42.881 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:42.881 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.881 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:42.881 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "nfs", 2024-09-10T10:04:42.881 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:42.881 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:42.881 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:42.881 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:42.881 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:42.881 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.881 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.882 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.882 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:42.882 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.882 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.882 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:42.882 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:42.882 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:42.882 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:42.882 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:42.882 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:42.883 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:42.883 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:42.883 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.883 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.883 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.883 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.883 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.883 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:42.883 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:42.883 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.883 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.884 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.884 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:42.884 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.884 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.884 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.884 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.884 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.884 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.884 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.884 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.885 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:42.885 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:42.885 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.885 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.885 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.885 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:42.885 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.885 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.885 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:42.885 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:42.885 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:42.886 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:42.886 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:42.886 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:42.886 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:42.886 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:42.886 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.886 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.886 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.886 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.886 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.886 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:42.886 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:42.886 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.887 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.887 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.887 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:42.887 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.887 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.887 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.887 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.887 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.887 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.887 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.887 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:42.887 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:42.888 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.888 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:42.888 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "orchestrator", 2024-09-10T10:04:42.888 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:42.888 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:42.888 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:42.888 INFO:teuthology.orchestra.run.smithi077.stdout: "fail_fs": { 2024-09-10T10:04:42.888 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "fail_fs", 2024-09-10T10:04:42.888 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.888 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.888 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.888 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:42.889 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.889 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.889 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.889 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-10T10:04:42.889 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.889 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.889 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.889 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.889 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:42.889 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:42.889 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.889 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.889 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.889 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:42.890 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.890 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.890 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:42.890 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:42.890 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:42.890 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:42.890 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:42.890 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:42.890 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:42.890 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:42.890 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.890 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.890 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.890 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.891 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.891 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:42.891 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:42.891 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.891 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.891 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.891 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:42.891 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.891 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.891 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.891 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.891 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.891 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.892 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.892 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.892 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:42.892 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:42.892 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.892 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.892 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.892 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:42.892 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.892 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.892 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:42.892 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:42.892 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:42.893 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:42.893 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:42.893 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:42.893 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:42.893 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:42.893 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.893 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.893 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.893 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.893 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.893 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:42.893 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:42.893 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.893 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.893 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.894 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:42.894 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.894 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.894 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.894 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.894 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.894 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.894 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.894 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.894 INFO:teuthology.orchestra.run.smithi077.stdout: "orchestrator": { 2024-09-10T10:04:42.894 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "orchestrator", 2024-09-10T10:04:42.894 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.894 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.894 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.895 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:42.895 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.895 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.895 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:42.895 INFO:teuthology.orchestra.run.smithi077.stdout: "cephadm", 2024-09-10T10:04:42.895 INFO:teuthology.orchestra.run.smithi077.stdout: "rook", 2024-09-10T10:04:42.895 INFO:teuthology.orchestra.run.smithi077.stdout: "test_orchestrator" 2024-09-10T10:04:42.895 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:42.895 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Orchestrator backend", 2024-09-10T10:04:42.895 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.895 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.895 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.895 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:42.895 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:42.896 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.896 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:42.896 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "osd_perf_query", 2024-09-10T10:04:42.896 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:42.896 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:42.896 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:42.896 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:42.896 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:42.896 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.896 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.896 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.896 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:42.896 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.897 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.897 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:42.897 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:42.897 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:42.897 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:42.897 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:42.897 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:42.897 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:42.897 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:42.897 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.897 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.897 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.897 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.898 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.898 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:42.898 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:42.898 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.898 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.898 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.898 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:42.898 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.898 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.898 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.898 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.898 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.898 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.898 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.899 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.899 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:42.899 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:42.899 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.899 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.899 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.899 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:42.899 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.899 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.899 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:42.899 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:42.899 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:42.899 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:42.900 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:42.900 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:42.900 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:42.900 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:42.900 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.900 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.900 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.900 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.900 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.900 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:42.900 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:42.900 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.900 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.900 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.901 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:42.901 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.901 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.901 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.901 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.901 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.901 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.901 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.901 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:42.901 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:42.901 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.901 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:42.901 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "osd_support", 2024-09-10T10:04:42.902 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:42.902 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:42.902 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:42.902 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:42.902 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:42.902 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.902 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.902 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.902 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:42.902 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.902 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.902 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:42.902 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:42.902 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:42.902 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:42.903 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:42.903 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:42.903 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:42.903 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:42.903 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.903 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.903 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.903 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.903 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.903 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:42.903 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:42.903 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.903 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.903 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.903 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:42.904 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.904 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.904 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.904 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.904 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.904 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.904 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.904 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.904 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:42.904 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:42.904 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.904 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.904 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.904 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:42.904 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.905 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.905 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:42.905 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:42.905 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:42.905 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:42.905 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:42.905 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:42.905 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:42.905 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:42.905 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.905 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.905 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.905 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.905 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.906 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:42.906 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:42.906 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.906 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.906 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.906 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:42.906 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.906 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.906 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.906 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.906 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.906 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.906 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.906 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:42.907 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:42.907 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.907 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:42.907 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "pg_autoscaler", 2024-09-10T10:04:42.907 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:42.907 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:42.907 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:42.907 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:42.907 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:42.907 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.907 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.907 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.907 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:42.907 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.908 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.908 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:42.908 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:42.908 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:42.908 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:42.908 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:42.908 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:42.908 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:42.908 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:42.908 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.908 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.908 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.908 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.908 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.909 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:42.909 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:42.909 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.909 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.909 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.909 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:42.909 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.909 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.909 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.909 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.909 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.909 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.909 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.910 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.910 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:42.910 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:42.910 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.910 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.910 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.910 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:42.910 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.910 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.910 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:42.910 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:42.910 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:42.910 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:42.911 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:42.911 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:42.911 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:42.911 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:42.911 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.911 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.911 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.911 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.911 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.911 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:42.911 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:42.911 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.911 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.912 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.912 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:42.912 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.912 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.912 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.912 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.912 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.912 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.912 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.912 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.912 INFO:teuthology.orchestra.run.smithi077.stdout: "sleep_interval": { 2024-09-10T10:04:42.912 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sleep_interval", 2024-09-10T10:04:42.912 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-10T10:04:42.913 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.913 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.913 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "60", 2024-09-10T10:04:42.913 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.913 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.913 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.913 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.913 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.913 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.913 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.913 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.913 INFO:teuthology.orchestra.run.smithi077.stdout: "threshold": { 2024-09-10T10:04:42.913 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "threshold", 2024-09-10T10:04:42.913 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "float", 2024-09-10T10:04:42.914 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.914 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.914 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "3.0", 2024-09-10T10:04:42.914 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "1.0", 2024-09-10T10:04:42.914 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.914 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.914 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "scaling threshold", 2024-09-10T10:04:42.914 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-10T10:04:42.914 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.914 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.914 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:42.914 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:42.914 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.914 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:42.914 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "progress", 2024-09-10T10:04:42.915 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:42.915 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:42.915 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:42.915 INFO:teuthology.orchestra.run.smithi077.stdout: "allow_pg_recovery_event": { 2024-09-10T10:04:42.915 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "allow_pg_recovery_event", 2024-09-10T10:04:42.915 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.915 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.915 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.915 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:42.915 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.915 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.915 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.915 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-10T10:04:42.915 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.916 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.916 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.916 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.916 INFO:teuthology.orchestra.run.smithi077.stdout: "enabled": { 2024-09-10T10:04:42.916 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "enabled", 2024-09-10T10:04:42.916 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.916 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.916 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.916 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:42.916 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.916 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.916 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.916 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.916 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.916 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.917 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.917 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.917 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:42.917 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:42.917 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.917 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.917 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.917 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:42.917 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.917 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.917 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:42.917 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:42.917 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:42.918 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:42.918 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:42.918 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:42.918 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:42.918 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:42.918 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.918 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.918 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.918 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.918 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.918 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:42.918 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:42.918 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.918 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.918 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.919 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:42.919 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.919 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.919 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.919 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.919 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.919 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.919 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.919 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.919 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:42.919 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:42.919 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.919 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.919 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.919 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:42.920 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.920 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.920 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:42.920 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:42.920 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:42.920 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:42.920 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:42.920 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:42.920 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:42.920 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:42.920 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.920 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.920 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.920 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.920 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.921 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:42.921 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:42.921 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.921 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.921 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.921 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:42.921 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.921 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.921 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.921 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.921 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.921 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.921 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.921 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.921 INFO:teuthology.orchestra.run.smithi077.stdout: "max_completed_events": { 2024-09-10T10:04:42.921 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "max_completed_events", 2024-09-10T10:04:42.922 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:42.922 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.922 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.922 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "50", 2024-09-10T10:04:42.922 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.922 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.922 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.922 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "number of past completed events to remember", 2024-09-10T10:04:42.922 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.922 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.922 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.922 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.922 INFO:teuthology.orchestra.run.smithi077.stdout: "sleep_interval": { 2024-09-10T10:04:42.922 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sleep_interval", 2024-09-10T10:04:42.923 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-10T10:04:42.923 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.923 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.923 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "5", 2024-09-10T10:04:42.923 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.923 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.923 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.923 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "how long the module is going to sleep", 2024-09-10T10:04:42.923 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.923 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.923 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.923 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:42.923 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:42.924 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.924 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:42.924 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "prometheus", 2024-09-10T10:04:42.924 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:42.924 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:42.924 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:42.924 INFO:teuthology.orchestra.run.smithi077.stdout: "cache": { 2024-09-10T10:04:42.924 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "cache", 2024-09-10T10:04:42.924 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.924 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.924 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.924 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:42.924 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.924 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.924 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.925 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.925 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.925 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.925 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.925 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.925 INFO:teuthology.orchestra.run.smithi077.stdout: "exclude_perf_counters": { 2024-09-10T10:04:42.925 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "exclude_perf_counters", 2024-09-10T10:04:42.925 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.925 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.925 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.925 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:42.925 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.925 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.925 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.925 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-10T10:04:42.926 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-10T10:04:42.926 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.926 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.926 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.926 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:42.926 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:42.926 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.926 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.926 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.926 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:42.926 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.926 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.926 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:42.926 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:42.926 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:42.927 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:42.927 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:42.927 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:42.927 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:42.927 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:42.927 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.927 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.927 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.927 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.927 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.927 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:42.927 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:42.927 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.928 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.928 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.928 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:42.928 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.928 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.928 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.928 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.928 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.928 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.928 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.928 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.928 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:42.928 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:42.928 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.929 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.929 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.929 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:42.929 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.929 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.929 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:42.929 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:42.929 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:42.929 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:42.929 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:42.929 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:42.929 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:42.929 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:42.930 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.930 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.930 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.930 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.930 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.930 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:42.930 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:42.930 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.930 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.930 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.930 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:42.930 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.930 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.930 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.931 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.931 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.931 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.931 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.931 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.931 INFO:teuthology.orchestra.run.smithi077.stdout: "rbd_stats_pools": { 2024-09-10T10:04:42.931 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rbd_stats_pools", 2024-09-10T10:04:42.931 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.931 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.931 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.931 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:42.931 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.931 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.931 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.932 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.932 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.932 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.932 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.932 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.932 INFO:teuthology.orchestra.run.smithi077.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-10T10:04:42.932 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-10T10:04:42.932 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:42.932 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.932 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.932 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "300", 2024-09-10T10:04:42.932 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.932 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.933 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.933 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.933 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.933 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.933 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.933 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.933 INFO:teuthology.orchestra.run.smithi077.stdout: "scrape_interval": { 2024-09-10T10:04:42.933 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "scrape_interval", 2024-09-10T10:04:42.933 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "float", 2024-09-10T10:04:42.933 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.933 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.933 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "15.0", 2024-09-10T10:04:42.933 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.934 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.934 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.934 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.934 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.934 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.934 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.934 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.934 INFO:teuthology.orchestra.run.smithi077.stdout: "server_addr": { 2024-09-10T10:04:42.934 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "server_addr", 2024-09-10T10:04:42.934 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.934 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.934 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.934 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "::", 2024-09-10T10:04:42.934 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.934 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.934 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.935 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-10T10:04:42.935 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.935 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.935 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.935 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.935 INFO:teuthology.orchestra.run.smithi077.stdout: "server_port": { 2024-09-10T10:04:42.935 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "server_port", 2024-09-10T10:04:42.935 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:42.935 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.935 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.935 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "9283", 2024-09-10T10:04:42.935 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.935 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.935 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.935 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-10T10:04:42.936 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.936 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.936 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.936 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.936 INFO:teuthology.orchestra.run.smithi077.stdout: "stale_cache_strategy": { 2024-09-10T10:04:42.936 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "stale_cache_strategy", 2024-09-10T10:04:42.936 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.936 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.936 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.936 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "log", 2024-09-10T10:04:42.936 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.936 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.936 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.936 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.937 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.937 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.937 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.937 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.937 INFO:teuthology.orchestra.run.smithi077.stdout: "standby_behaviour": { 2024-09-10T10:04:42.937 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "standby_behaviour", 2024-09-10T10:04:42.937 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.937 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.937 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.937 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "default", 2024-09-10T10:04:42.937 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.937 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.937 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:42.937 INFO:teuthology.orchestra.run.smithi077.stdout: "default", 2024-09-10T10:04:42.938 INFO:teuthology.orchestra.run.smithi077.stdout: "error" 2024-09-10T10:04:42.938 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:42.938 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.938 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.938 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.938 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.938 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.938 INFO:teuthology.orchestra.run.smithi077.stdout: "standby_error_status_code": { 2024-09-10T10:04:42.938 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "standby_error_status_code", 2024-09-10T10:04:42.938 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:42.938 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.938 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.938 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "500", 2024-09-10T10:04:42.938 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "400", 2024-09-10T10:04:42.938 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "599", 2024-09-10T10:04:42.939 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.939 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.939 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.939 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.939 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.939 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:42.939 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:42.939 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.939 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:42.939 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rbd_support", 2024-09-10T10:04:42.939 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:42.939 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:42.939 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:42.939 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:42.939 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:42.940 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.940 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.940 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.940 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:42.940 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.940 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.940 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:42.940 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:42.940 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:42.940 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:42.940 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:42.940 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:42.940 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:42.940 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:42.940 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.941 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.941 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.941 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.941 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.941 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:42.941 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:42.941 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.941 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.941 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.941 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:42.941 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.941 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.941 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.941 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.942 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.942 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.942 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.942 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.942 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:42.942 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:42.942 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.942 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.942 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.942 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:42.942 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.942 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.942 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:42.942 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:42.943 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:42.943 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:42.943 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:42.943 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:42.943 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:42.943 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:42.943 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.943 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.943 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.943 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.943 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.943 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:42.943 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:42.943 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.943 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.944 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.944 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:42.944 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.944 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.944 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.944 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.944 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.944 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.944 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.944 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.944 INFO:teuthology.orchestra.run.smithi077.stdout: "max_concurrent_snap_create": { 2024-09-10T10:04:42.944 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "max_concurrent_snap_create", 2024-09-10T10:04:42.944 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:42.944 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.944 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.945 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10", 2024-09-10T10:04:42.945 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.945 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.945 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.945 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.945 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.945 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.945 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.945 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.945 INFO:teuthology.orchestra.run.smithi077.stdout: "mirror_snapshot_schedule": { 2024-09-10T10:04:42.945 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "mirror_snapshot_schedule", 2024-09-10T10:04:42.945 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.945 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.945 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.946 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:42.946 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.946 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.946 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.946 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.946 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.946 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.946 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.946 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.946 INFO:teuthology.orchestra.run.smithi077.stdout: "trash_purge_schedule": { 2024-09-10T10:04:42.946 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "trash_purge_schedule", 2024-09-10T10:04:42.946 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.946 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.946 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.946 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:42.947 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.947 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.947 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.947 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.947 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.947 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.947 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.947 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:42.947 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:42.947 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.947 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:42.947 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "restful", 2024-09-10T10:04:42.947 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:42.948 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:42.948 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:42.948 INFO:teuthology.orchestra.run.smithi077.stdout: "enable_auth": { 2024-09-10T10:04:42.948 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "enable_auth", 2024-09-10T10:04:42.948 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.948 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.948 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.948 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:42.948 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.948 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.948 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.948 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.948 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.948 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.948 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.949 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.949 INFO:teuthology.orchestra.run.smithi077.stdout: "key_file": { 2024-09-10T10:04:42.949 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "key_file", 2024-09-10T10:04:42.949 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.949 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.949 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.949 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:42.949 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.949 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.949 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.949 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.949 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.950 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.950 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.950 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.950 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:42.950 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:42.950 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.950 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.950 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.950 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:42.950 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.950 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.950 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:42.950 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:42.950 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:42.951 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:42.951 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:42.951 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:42.951 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:42.951 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:42.951 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.951 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.951 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.951 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.951 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.951 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:42.951 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:42.951 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.951 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.952 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.952 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:42.952 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.952 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.952 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.952 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.952 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.952 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.952 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.952 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.952 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:42.952 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:42.952 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.953 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.953 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.953 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:42.953 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.953 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.953 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:42.953 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:42.953 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:42.953 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:42.953 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:42.953 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:42.953 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:42.953 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:42.953 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.953 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.954 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.954 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.954 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.954 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:42.954 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:42.954 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.954 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.954 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.954 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:42.954 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.954 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.954 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.954 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.954 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.954 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.955 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.955 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.955 INFO:teuthology.orchestra.run.smithi077.stdout: "server_addr": { 2024-09-10T10:04:42.955 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "server_addr", 2024-09-10T10:04:42.955 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.955 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.955 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.955 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:42.955 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.955 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.955 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.955 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.955 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.955 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.956 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.956 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.956 INFO:teuthology.orchestra.run.smithi077.stdout: "server_port": { 2024-09-10T10:04:42.956 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "server_port", 2024-09-10T10:04:42.956 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.956 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.956 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.956 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:42.956 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.956 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.956 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.956 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.956 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.956 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.957 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.957 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:42.957 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:42.957 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.957 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:42.957 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rgw", 2024-09-10T10:04:42.957 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:42.957 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:42.957 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:42.957 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:42.957 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:42.958 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.958 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.958 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.958 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:42.958 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.958 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.958 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:42.958 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:42.958 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:42.958 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:42.958 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:42.958 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:42.958 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:42.958 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:42.958 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.959 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.959 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.959 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.959 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.959 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:42.959 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:42.959 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.959 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.959 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.959 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:42.959 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.959 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.959 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.959 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.960 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.960 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.960 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.960 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.960 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:42.960 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:42.960 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.960 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.960 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.960 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:42.960 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.960 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.961 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:42.961 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:42.961 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:42.961 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:42.961 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:42.961 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:42.961 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:42.961 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:42.961 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.961 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.961 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.961 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.961 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.961 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:42.962 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:42.962 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.962 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.962 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.962 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:42.962 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.962 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.962 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.962 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.962 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.962 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.962 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.962 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:42.963 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:42.963 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.963 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:42.963 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rook", 2024-09-10T10:04:42.963 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:42.963 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:42.963 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:42.963 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:42.963 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:42.963 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.963 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.963 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.963 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:42.963 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.964 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.964 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:42.964 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:42.964 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:42.964 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:42.964 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:42.964 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:42.964 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:42.964 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:42.964 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.964 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.964 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.965 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.965 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.965 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:42.965 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:42.965 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.965 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.965 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.965 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:42.965 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.965 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.965 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.965 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.965 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.966 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.966 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.966 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.966 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:42.966 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:42.966 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.966 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.966 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.966 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:42.966 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.966 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.966 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:42.966 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:42.966 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:42.967 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:42.967 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:42.967 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:42.967 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:42.967 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:42.967 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.967 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.967 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.967 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.967 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.967 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:42.967 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:42.967 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.968 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.968 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.968 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:42.968 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.968 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.968 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.968 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.968 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.968 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.968 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.968 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.968 INFO:teuthology.orchestra.run.smithi077.stdout: "storage_class": { 2024-09-10T10:04:42.968 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "storage_class", 2024-09-10T10:04:42.968 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.969 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.969 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.969 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "local", 2024-09-10T10:04:42.969 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.969 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.969 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.969 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-10T10:04:42.969 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.969 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.969 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.969 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:42.969 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:42.969 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.969 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:42.970 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "selftest", 2024-09-10T10:04:42.970 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:42.970 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:42.970 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:42.970 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:42.970 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:42.970 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.970 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.970 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.970 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:42.970 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.970 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.970 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:42.971 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:42.971 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:42.971 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:42.971 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:42.971 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:42.971 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:42.971 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:42.971 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.971 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.971 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.971 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.971 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.971 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:42.971 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:42.971 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.972 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.972 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.972 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:42.972 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.972 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.972 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.972 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.972 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.972 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.972 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.972 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.972 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:42.972 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:42.973 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.973 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.973 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.973 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:42.973 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.973 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.973 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:42.973 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:42.973 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:42.973 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:42.973 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:42.973 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:42.973 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:42.973 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:42.973 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.974 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.974 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.974 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.974 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.974 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:42.974 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:42.974 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.974 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.974 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.974 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:42.974 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.974 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.974 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.974 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.975 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.975 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.975 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.975 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.975 INFO:teuthology.orchestra.run.smithi077.stdout: "roption1": { 2024-09-10T10:04:42.975 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "roption1", 2024-09-10T10:04:42.975 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.975 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.975 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.975 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:42.975 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.975 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.975 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.975 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.976 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.976 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.976 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.976 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.976 INFO:teuthology.orchestra.run.smithi077.stdout: "roption2": { 2024-09-10T10:04:42.976 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "roption2", 2024-09-10T10:04:42.976 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.976 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.976 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.976 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "xyz", 2024-09-10T10:04:42.976 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.976 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.977 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.977 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.977 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.977 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.977 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.977 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.977 INFO:teuthology.orchestra.run.smithi077.stdout: "rwoption1": { 2024-09-10T10:04:42.977 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rwoption1", 2024-09-10T10:04:42.977 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.977 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.977 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.977 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:42.977 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.977 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.978 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.978 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.978 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.978 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.978 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.978 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.978 INFO:teuthology.orchestra.run.smithi077.stdout: "rwoption2": { 2024-09-10T10:04:42.978 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rwoption2", 2024-09-10T10:04:42.978 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:42.978 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.978 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.978 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:42.978 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.978 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.979 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.979 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.979 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.979 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.979 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.979 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.979 INFO:teuthology.orchestra.run.smithi077.stdout: "rwoption3": { 2024-09-10T10:04:42.979 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rwoption3", 2024-09-10T10:04:42.979 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "float", 2024-09-10T10:04:42.979 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.979 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.979 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:42.979 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.979 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.980 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.980 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.980 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.980 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.980 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.980 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.980 INFO:teuthology.orchestra.run.smithi077.stdout: "rwoption4": { 2024-09-10T10:04:42.980 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rwoption4", 2024-09-10T10:04:42.980 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.980 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.980 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.980 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:42.980 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.980 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.981 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.981 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.981 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.981 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.981 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.981 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.981 INFO:teuthology.orchestra.run.smithi077.stdout: "rwoption5": { 2024-09-10T10:04:42.981 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rwoption5", 2024-09-10T10:04:42.981 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.981 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.981 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.981 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:42.981 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.981 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.982 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.982 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.982 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.982 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.982 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.982 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.982 INFO:teuthology.orchestra.run.smithi077.stdout: "rwoption6": { 2024-09-10T10:04:42.982 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rwoption6", 2024-09-10T10:04:42.982 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.982 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.982 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.982 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:42.982 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.982 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.983 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.983 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.983 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.983 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.983 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.983 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.983 INFO:teuthology.orchestra.run.smithi077.stdout: "rwoption7": { 2024-09-10T10:04:42.983 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rwoption7", 2024-09-10T10:04:42.983 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:42.983 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.983 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.983 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:42.983 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "1", 2024-09-10T10:04:42.983 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "42", 2024-09-10T10:04:42.984 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.984 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.984 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.984 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.984 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.984 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.984 INFO:teuthology.orchestra.run.smithi077.stdout: "testkey": { 2024-09-10T10:04:42.984 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "testkey", 2024-09-10T10:04:42.984 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.984 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.984 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.984 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:42.984 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.984 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.984 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.985 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.985 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.985 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.985 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.985 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.985 INFO:teuthology.orchestra.run.smithi077.stdout: "testlkey": { 2024-09-10T10:04:42.985 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "testlkey", 2024-09-10T10:04:42.985 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.985 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.985 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.985 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:42.985 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.985 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.985 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.985 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.986 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.986 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.986 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.986 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.986 INFO:teuthology.orchestra.run.smithi077.stdout: "testnewline": { 2024-09-10T10:04:42.986 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "testnewline", 2024-09-10T10:04:42.986 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.986 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.986 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:42.986 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:42.986 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.986 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.986 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.987 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.987 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.987 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.987 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.987 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:42.987 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:42.987 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.987 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:42.987 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "snap_schedule", 2024-09-10T10:04:42.987 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:42.987 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:42.987 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:42.987 INFO:teuthology.orchestra.run.smithi077.stdout: "allow_m_granularity": { 2024-09-10T10:04:42.987 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "allow_m_granularity", 2024-09-10T10:04:42.987 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.988 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.988 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.988 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:42.988 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.988 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.988 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.988 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "allow minute scheduled snapshots", 2024-09-10T10:04:42.988 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.988 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.988 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.988 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.988 INFO:teuthology.orchestra.run.smithi077.stdout: "dump_on_update": { 2024-09-10T10:04:42.988 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "dump_on_update", 2024-09-10T10:04:42.988 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.988 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.989 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.989 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:42.989 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.989 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.989 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.989 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "dump database to debug log on update", 2024-09-10T10:04:42.989 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.989 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.989 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.989 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.989 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:42.989 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:42.989 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.989 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.989 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.990 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:42.990 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.990 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.990 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:42.990 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:42.990 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:42.990 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:42.990 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:42.990 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:42.990 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:42.990 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:42.990 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.990 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.990 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.990 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.991 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.991 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:42.991 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:42.991 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.991 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.991 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.991 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:42.991 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.991 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.991 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.991 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.991 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.992 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.992 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.992 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.992 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:42.992 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:42.992 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.992 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.992 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.992 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:42.992 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.992 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.992 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:42.992 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:42.993 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:42.993 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:42.993 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:42.993 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:42.993 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:42.993 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:42.993 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.993 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.993 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.993 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.993 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.993 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:42.993 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:42.993 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.994 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.994 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.994 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:42.994 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.994 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.994 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.994 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.994 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.994 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.994 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.994 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:42.994 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:42.994 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.994 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:42.994 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "stats", 2024-09-10T10:04:42.995 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:42.995 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:42.995 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:42.995 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:42.995 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:42.995 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.995 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.995 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.995 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:42.995 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.995 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.995 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:42.995 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:42.996 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:42.996 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:42.996 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:42.996 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:42.996 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:42.996 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:42.996 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.996 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.996 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.996 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.996 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.996 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:42.996 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:42.997 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.997 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.997 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.997 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:42.997 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.997 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.997 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:42.997 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.997 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.997 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.997 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.997 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.997 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:42.998 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:42.998 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:42.998 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.998 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.998 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:42.998 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:42.998 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:42.998 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:42.998 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:42.998 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:42.998 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:42.998 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:42.998 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:42.999 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:42.999 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:42.999 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:42.999 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:42.999 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:42.999 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:42.999 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:42.999 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:42.999 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:42.999 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:42.999 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:42.999 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:42.999 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:42.999 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.000 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.000 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.000 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.000 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.000 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.000 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.000 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.000 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.000 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.000 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:43.000 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "status", 2024-09-10T10:04:43.000 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:43.000 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:43.001 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:43.001 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:43.001 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:43.001 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.001 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.001 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.001 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.001 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.001 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.001 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.001 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.001 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.001 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.002 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.002 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.002 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.002 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.002 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.002 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.002 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.002 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.002 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.002 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:43.002 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:43.002 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.003 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.003 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.003 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.003 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.003 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.003 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.003 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.003 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.003 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.003 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.003 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.003 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:43.003 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:43.004 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.004 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.004 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.004 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:43.004 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.004 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.004 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.004 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.004 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.004 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.004 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.005 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.005 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.005 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.005 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.005 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.005 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.005 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.005 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.005 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:43.005 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:43.005 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.005 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.006 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.006 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.006 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.006 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.006 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.006 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.006 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.006 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.006 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.006 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.006 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.006 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.007 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:43.007 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "telegraf", 2024-09-10T10:04:43.007 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:43.007 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:43.007 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:43.007 INFO:teuthology.orchestra.run.smithi077.stdout: "address": { 2024-09-10T10:04:43.007 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "address", 2024-09-10T10:04:43.007 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.007 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.007 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.007 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-10T10:04:43.007 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.008 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.008 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.008 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.008 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.008 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.008 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.008 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.008 INFO:teuthology.orchestra.run.smithi077.stdout: "interval": { 2024-09-10T10:04:43.008 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "interval", 2024-09-10T10:04:43.008 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-10T10:04:43.009 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.009 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.009 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "15", 2024-09-10T10:04:43.009 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.009 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.009 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.009 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.009 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.009 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.009 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.009 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.009 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:43.010 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:43.010 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.010 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.010 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.010 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.010 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.010 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.010 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.010 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.010 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.010 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.010 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.010 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.011 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.011 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.011 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.011 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.011 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.011 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.011 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.011 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:43.011 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:43.011 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.011 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.011 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.011 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.012 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.012 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.012 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.012 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.012 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.012 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.012 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.012 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.012 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:43.012 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:43.012 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.012 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.012 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.012 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:43.012 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.013 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.013 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.013 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.013 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.013 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.013 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.013 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.013 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.013 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.013 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.013 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.013 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.013 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.013 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.014 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:43.014 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:43.014 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.014 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.014 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.014 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.014 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.014 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.014 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.014 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.014 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.014 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.014 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.014 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.014 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.015 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.015 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:43.015 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "telemetry", 2024-09-10T10:04:43.015 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:43.015 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:43.015 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:43.015 INFO:teuthology.orchestra.run.smithi077.stdout: "channel_basic": { 2024-09-10T10:04:43.015 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "channel_basic", 2024-09-10T10:04:43.015 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.015 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.015 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.015 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:43.015 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.016 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.016 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.016 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-10T10:04:43.016 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.016 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.016 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.016 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.016 INFO:teuthology.orchestra.run.smithi077.stdout: "channel_crash": { 2024-09-10T10:04:43.016 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "channel_crash", 2024-09-10T10:04:43.016 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.016 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.016 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.017 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:43.017 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.017 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.017 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.017 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-10T10:04:43.017 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.017 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.017 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.017 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.017 INFO:teuthology.orchestra.run.smithi077.stdout: "channel_device": { 2024-09-10T10:04:43.017 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "channel_device", 2024-09-10T10:04:43.017 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.017 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.018 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.018 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:43.018 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.018 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.018 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.018 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-10T10:04:43.018 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.018 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.018 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.018 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.018 INFO:teuthology.orchestra.run.smithi077.stdout: "channel_ident": { 2024-09-10T10:04:43.018 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "channel_ident", 2024-09-10T10:04:43.018 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.018 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.019 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.019 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.019 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.019 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.019 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.019 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-10T10:04:43.019 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.019 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.019 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.019 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.019 INFO:teuthology.orchestra.run.smithi077.stdout: "channel_perf": { 2024-09-10T10:04:43.019 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "channel_perf", 2024-09-10T10:04:43.019 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.020 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.020 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.020 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.020 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.020 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.020 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.020 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-10T10:04:43.020 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.020 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.020 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.020 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.020 INFO:teuthology.orchestra.run.smithi077.stdout: "contact": { 2024-09-10T10:04:43.020 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "contact", 2024-09-10T10:04:43.021 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.021 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.021 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.021 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.021 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.021 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.021 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.021 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.021 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.021 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.022 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.022 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.022 INFO:teuthology.orchestra.run.smithi077.stdout: "description": { 2024-09-10T10:04:43.022 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "description", 2024-09-10T10:04:43.022 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.022 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.022 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.022 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.022 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.022 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.022 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.022 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.022 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.023 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.023 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.023 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.023 INFO:teuthology.orchestra.run.smithi077.stdout: "device_url": { 2024-09-10T10:04:43.023 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "device_url", 2024-09-10T10:04:43.023 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.023 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.023 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.023 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-10T10:04:43.023 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.023 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.023 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.023 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.024 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.024 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.024 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.024 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.024 INFO:teuthology.orchestra.run.smithi077.stdout: "enabled": { 2024-09-10T10:04:43.024 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "enabled", 2024-09-10T10:04:43.024 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.024 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.024 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.024 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.024 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.024 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.024 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.025 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.025 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.025 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.025 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.025 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.025 INFO:teuthology.orchestra.run.smithi077.stdout: "interval": { 2024-09-10T10:04:43.025 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "interval", 2024-09-10T10:04:43.025 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:43.025 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.025 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.025 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "24", 2024-09-10T10:04:43.025 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "8", 2024-09-10T10:04:43.025 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.026 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.026 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.026 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.026 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.026 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.026 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.026 INFO:teuthology.orchestra.run.smithi077.stdout: "last_opt_revision": { 2024-09-10T10:04:43.026 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "last_opt_revision", 2024-09-10T10:04:43.026 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:43.026 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.026 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.026 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "1", 2024-09-10T10:04:43.026 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.026 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.027 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.027 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.027 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.027 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.027 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.027 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.027 INFO:teuthology.orchestra.run.smithi077.stdout: "leaderboard": { 2024-09-10T10:04:43.027 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "leaderboard", 2024-09-10T10:04:43.027 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.027 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.027 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.027 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.027 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.028 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.028 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.028 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.028 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.028 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.028 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.028 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.028 INFO:teuthology.orchestra.run.smithi077.stdout: "leaderboard_description": { 2024-09-10T10:04:43.028 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "leaderboard_description", 2024-09-10T10:04:43.028 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.028 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.028 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.028 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.029 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.029 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.029 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.029 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.029 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.029 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.029 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.029 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.029 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:43.029 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:43.029 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.029 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.029 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.029 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.030 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.030 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.030 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.030 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.030 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.030 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.030 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.030 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.030 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.030 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.030 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.030 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.030 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.031 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.031 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.031 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:43.031 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:43.031 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.031 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.031 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.031 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.031 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.031 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.031 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.031 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.031 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.031 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.031 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.032 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.032 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:43.032 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:43.032 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.032 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.032 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.032 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:43.032 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.032 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.032 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.032 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.032 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.032 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.032 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.033 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.033 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.033 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.033 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.033 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.033 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.033 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.033 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.033 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:43.033 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:43.033 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.034 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.034 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.034 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.034 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.034 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.034 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.034 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.034 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.034 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.034 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.034 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.034 INFO:teuthology.orchestra.run.smithi077.stdout: "organization": { 2024-09-10T10:04:43.034 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "organization", 2024-09-10T10:04:43.035 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.035 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.035 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.035 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.035 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.035 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.035 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.035 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.035 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.035 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.035 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.035 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.035 INFO:teuthology.orchestra.run.smithi077.stdout: "proxy": { 2024-09-10T10:04:43.036 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "proxy", 2024-09-10T10:04:43.036 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.036 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.036 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.036 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.036 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.036 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.036 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.036 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.036 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.036 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.036 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.037 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.037 INFO:teuthology.orchestra.run.smithi077.stdout: "url": { 2024-09-10T10:04:43.037 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "url", 2024-09-10T10:04:43.037 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.037 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.037 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.037 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-10T10:04:43.037 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.037 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.037 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.037 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.037 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.038 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.038 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.038 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.038 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.038 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.038 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:43.038 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "test_orchestrator", 2024-09-10T10:04:43.038 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:43.038 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:43.038 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:43.038 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:43.038 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:43.039 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.039 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.039 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.039 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.039 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.039 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.039 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.039 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.039 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.039 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.039 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.039 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.039 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.040 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.040 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.040 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.040 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.040 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.040 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.040 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:43.040 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:43.040 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.040 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.040 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.040 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.041 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.041 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.041 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.041 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.041 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.041 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.041 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.041 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.041 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:43.041 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:43.041 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.041 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.042 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.042 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:43.042 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.042 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.042 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.042 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.042 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.042 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.042 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.042 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.042 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.043 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.043 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.043 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.043 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.043 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.043 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.043 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:43.043 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:43.043 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.043 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.043 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.043 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.043 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.043 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.044 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.044 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.044 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.044 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.044 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.044 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.044 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.044 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.044 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:43.044 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "volumes", 2024-09-10T10:04:43.044 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:43.044 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:43.044 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:43.044 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:43.045 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:43.045 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.045 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.045 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.045 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.045 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.045 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.045 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.045 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.045 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.045 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.045 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.045 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.045 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.046 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.046 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.046 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.046 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.046 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.046 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.046 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:43.046 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:43.046 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.046 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.046 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.046 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.046 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.047 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.047 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.047 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.047 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.047 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.047 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.047 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.047 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:43.047 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:43.047 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.047 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.047 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.047 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:43.048 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.048 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.048 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.048 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.048 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.048 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.048 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.048 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.048 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.048 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.048 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.048 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.048 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.048 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.049 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.049 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:43.049 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:43.049 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.049 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.049 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.049 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.049 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.049 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.049 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.049 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.049 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.049 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.049 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.050 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.050 INFO:teuthology.orchestra.run.smithi077.stdout: "max_concurrent_clones": { 2024-09-10T10:04:43.050 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "max_concurrent_clones", 2024-09-10T10:04:43.050 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:43.050 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.050 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.050 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "4", 2024-09-10T10:04:43.050 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.050 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.050 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.050 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-10T10:04:43.050 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.050 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.050 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.051 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.051 INFO:teuthology.orchestra.run.smithi077.stdout: "periodic_async_work": { 2024-09-10T10:04:43.051 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "periodic_async_work", 2024-09-10T10:04:43.051 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.051 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.051 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.051 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.051 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.051 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.051 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.051 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Periodically check for async work", 2024-09-10T10:04:43.051 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.051 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.051 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.052 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.052 INFO:teuthology.orchestra.run.smithi077.stdout: "snapshot_clone_delay": { 2024-09-10T10:04:43.052 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "snapshot_clone_delay", 2024-09-10T10:04:43.052 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:43.052 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.052 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.052 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-09-10T10:04:43.052 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.052 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.052 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.052 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-10T10:04:43.052 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.052 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.052 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.053 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.053 INFO:teuthology.orchestra.run.smithi077.stdout: "snapshot_clone_no_wait": { 2024-09-10T10:04:43.053 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "snapshot_clone_no_wait", 2024-09-10T10:04:43.053 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.053 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.053 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.053 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:43.053 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.053 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.053 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.053 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-10T10:04:43.053 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.053 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.054 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.054 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.054 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.054 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.054 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:43.054 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "zabbix", 2024-09-10T10:04:43.054 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:43.054 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:43.054 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:43.054 INFO:teuthology.orchestra.run.smithi077.stdout: "discovery_interval": { 2024-09-10T10:04:43.054 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "discovery_interval", 2024-09-10T10:04:43.054 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "uint", 2024-09-10T10:04:43.054 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.054 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.054 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "100", 2024-09-10T10:04:43.054 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.055 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.055 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.055 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.055 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.055 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.055 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.055 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.055 INFO:teuthology.orchestra.run.smithi077.stdout: "identifier": { 2024-09-10T10:04:43.055 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "identifier", 2024-09-10T10:04:43.055 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.055 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.055 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.055 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.055 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.055 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.056 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.056 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.056 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.056 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.056 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.056 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.056 INFO:teuthology.orchestra.run.smithi077.stdout: "interval": { 2024-09-10T10:04:43.056 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "interval", 2024-09-10T10:04:43.056 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-10T10:04:43.056 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.056 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.056 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "60", 2024-09-10T10:04:43.056 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.056 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.057 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.057 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.057 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.057 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.057 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.057 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.057 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:43.057 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:43.057 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.057 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.057 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.057 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.057 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.058 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.058 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.058 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.058 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.058 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.058 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.058 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.058 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.058 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.058 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.058 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.058 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.058 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.058 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.058 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:43.059 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:43.059 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.059 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.059 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.059 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.059 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.059 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.059 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.059 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.059 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.059 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.059 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.059 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.059 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:43.059 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:43.059 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.060 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.060 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.060 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:43.060 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.060 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.060 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.060 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.060 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.060 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.060 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.060 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.060 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.061 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.061 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.061 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.061 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.061 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.061 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.061 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:43.061 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:43.061 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.061 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.061 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.061 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.061 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.061 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.062 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.062 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.062 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.062 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.062 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.062 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.062 INFO:teuthology.orchestra.run.smithi077.stdout: "zabbix_host": { 2024-09-10T10:04:43.062 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "zabbix_host", 2024-09-10T10:04:43.062 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.062 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.062 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.062 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.062 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.063 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.063 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.063 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.063 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.063 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.063 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.063 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.063 INFO:teuthology.orchestra.run.smithi077.stdout: "zabbix_port": { 2024-09-10T10:04:43.063 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "zabbix_port", 2024-09-10T10:04:43.063 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:43.063 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.063 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.063 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10051", 2024-09-10T10:04:43.064 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.064 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.064 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.064 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.064 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.064 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.064 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.064 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.064 INFO:teuthology.orchestra.run.smithi077.stdout: "zabbix_sender": { 2024-09-10T10:04:43.064 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "zabbix_sender", 2024-09-10T10:04:43.064 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.064 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.064 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.065 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-10T10:04:43.065 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.065 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.065 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.065 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.065 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.065 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.065 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.065 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.065 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.065 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.065 INFO:teuthology.orchestra.run.smithi077.stdout: ] 2024-09-10T10:04:43.065 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.066 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:43.066 INFO:teuthology.orchestra.run.smithi077.stdout: "gid": 24113, 2024-09-10T10:04:43.066 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "y", 2024-09-10T10:04:43.066 INFO:teuthology.orchestra.run.smithi077.stdout: "mgr_features": 4540701547738038271, 2024-09-10T10:04:43.066 INFO:teuthology.orchestra.run.smithi077.stdout: "available_modules": [ 2024-09-10T10:04:43.066 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:43.066 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "alerts", 2024-09-10T10:04:43.066 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:43.066 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:43.066 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:43.066 INFO:teuthology.orchestra.run.smithi077.stdout: "interval": { 2024-09-10T10:04:43.066 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "interval", 2024-09-10T10:04:43.066 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-10T10:04:43.066 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.067 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.067 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "60", 2024-09-10T10:04:43.067 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.067 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.067 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.067 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "How frequently to reexamine health status", 2024-09-10T10:04:43.067 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.067 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.067 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.067 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.067 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:43.067 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:43.067 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.068 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.068 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.068 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.068 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.068 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.068 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.068 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.068 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.068 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.068 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.068 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.068 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.068 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.068 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.069 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.069 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.069 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.069 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.069 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:43.069 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:43.069 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.069 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.069 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.069 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.069 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.069 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.069 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.070 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.070 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.070 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.070 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.070 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.070 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:43.070 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:43.070 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.070 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.070 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.070 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:43.070 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.070 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.071 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.071 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.071 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.071 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.071 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.071 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.071 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.071 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.071 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.071 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.071 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.071 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.071 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.071 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:43.071 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:43.072 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.072 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.072 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.072 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.072 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.072 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.072 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.072 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.072 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.072 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.072 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.072 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.072 INFO:teuthology.orchestra.run.smithi077.stdout: "smtp_destination": { 2024-09-10T10:04:43.072 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smtp_destination", 2024-09-10T10:04:43.073 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.073 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.073 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.073 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.073 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.073 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.073 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.073 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Email address to send alerts to", 2024-09-10T10:04:43.073 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.073 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.073 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.073 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.073 INFO:teuthology.orchestra.run.smithi077.stdout: "smtp_from_name": { 2024-09-10T10:04:43.073 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smtp_from_name", 2024-09-10T10:04:43.074 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.074 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.074 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.074 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "Ceph", 2024-09-10T10:04:43.074 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.074 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.074 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.074 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Email From: name", 2024-09-10T10:04:43.074 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.074 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.074 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.074 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.074 INFO:teuthology.orchestra.run.smithi077.stdout: "smtp_host": { 2024-09-10T10:04:43.074 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smtp_host", 2024-09-10T10:04:43.075 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.075 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.075 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.075 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.075 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.075 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.075 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.075 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "SMTP server", 2024-09-10T10:04:43.075 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.075 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.075 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.075 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.075 INFO:teuthology.orchestra.run.smithi077.stdout: "smtp_password": { 2024-09-10T10:04:43.076 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smtp_password", 2024-09-10T10:04:43.076 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.076 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.076 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.076 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.076 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.076 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.076 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.076 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Password to authenticate with", 2024-09-10T10:04:43.076 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.076 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.076 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.077 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.077 INFO:teuthology.orchestra.run.smithi077.stdout: "smtp_port": { 2024-09-10T10:04:43.077 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smtp_port", 2024-09-10T10:04:43.077 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:43.077 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.077 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.077 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "465", 2024-09-10T10:04:43.077 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.077 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.077 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.077 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "SMTP port", 2024-09-10T10:04:43.077 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.078 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.078 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.078 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.078 INFO:teuthology.orchestra.run.smithi077.stdout: "smtp_sender": { 2024-09-10T10:04:43.078 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smtp_sender", 2024-09-10T10:04:43.078 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.078 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.078 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.078 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.078 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.078 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.078 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.078 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "SMTP envelope sender", 2024-09-10T10:04:43.079 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.079 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.079 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.079 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.079 INFO:teuthology.orchestra.run.smithi077.stdout: "smtp_ssl": { 2024-09-10T10:04:43.079 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smtp_ssl", 2024-09-10T10:04:43.079 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.079 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.079 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.079 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:43.079 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.079 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.079 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.079 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-10T10:04:43.080 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.080 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.080 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.080 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.080 INFO:teuthology.orchestra.run.smithi077.stdout: "smtp_user": { 2024-09-10T10:04:43.080 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smtp_user", 2024-09-10T10:04:43.080 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.080 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.080 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.080 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.080 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.081 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.081 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.081 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "User to authenticate as", 2024-09-10T10:04:43.081 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.081 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.081 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.081 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.081 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.081 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.081 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:43.081 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "balancer", 2024-09-10T10:04:43.081 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:43.082 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:43.082 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:43.082 INFO:teuthology.orchestra.run.smithi077.stdout: "active": { 2024-09-10T10:04:43.082 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "active", 2024-09-10T10:04:43.082 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.082 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.082 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.082 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:43.082 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.082 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.082 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.082 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "automatically balance PGs across cluster", 2024-09-10T10:04:43.082 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.083 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.083 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.083 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.083 INFO:teuthology.orchestra.run.smithi077.stdout: "begin_time": { 2024-09-10T10:04:43.083 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "begin_time", 2024-09-10T10:04:43.083 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.083 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.083 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.083 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0000", 2024-09-10T10:04:43.083 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.083 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.083 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.083 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "beginning time of day to automatically balance", 2024-09-10T10:04:43.083 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-10T10:04:43.084 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.084 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.084 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.084 INFO:teuthology.orchestra.run.smithi077.stdout: "begin_weekday": { 2024-09-10T10:04:43.084 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "begin_weekday", 2024-09-10T10:04:43.084 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "uint", 2024-09-10T10:04:43.084 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.084 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.084 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-09-10T10:04:43.084 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "0", 2024-09-10T10:04:43.084 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "6", 2024-09-10T10:04:43.084 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.084 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-10T10:04:43.084 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-10T10:04:43.085 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.085 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.085 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.085 INFO:teuthology.orchestra.run.smithi077.stdout: "crush_compat_max_iterations": { 2024-09-10T10:04:43.085 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "crush_compat_max_iterations", 2024-09-10T10:04:43.085 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "uint", 2024-09-10T10:04:43.085 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.085 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.085 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "25", 2024-09-10T10:04:43.085 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "1", 2024-09-10T10:04:43.085 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "250", 2024-09-10T10:04:43.085 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.085 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-10T10:04:43.086 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.086 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.086 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.086 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.086 INFO:teuthology.orchestra.run.smithi077.stdout: "crush_compat_metrics": { 2024-09-10T10:04:43.086 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "crush_compat_metrics", 2024-09-10T10:04:43.086 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.086 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.086 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.086 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "pgs,objects,bytes", 2024-09-10T10:04:43.086 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.086 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.086 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.086 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-10T10:04:43.087 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-10T10:04:43.087 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.087 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.087 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.087 INFO:teuthology.orchestra.run.smithi077.stdout: "crush_compat_step": { 2024-09-10T10:04:43.087 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "crush_compat_step", 2024-09-10T10:04:43.087 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "float", 2024-09-10T10:04:43.087 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.087 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.087 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0.5", 2024-09-10T10:04:43.087 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "0.001", 2024-09-10T10:04:43.087 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "0.999", 2024-09-10T10:04:43.088 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.088 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "aggressiveness of optimization", 2024-09-10T10:04:43.088 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-10T10:04:43.088 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.088 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.088 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.088 INFO:teuthology.orchestra.run.smithi077.stdout: "end_time": { 2024-09-10T10:04:43.088 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "end_time", 2024-09-10T10:04:43.088 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.088 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.088 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.088 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "2359", 2024-09-10T10:04:43.089 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.089 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.089 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.089 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "ending time of day to automatically balance", 2024-09-10T10:04:43.089 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-10T10:04:43.089 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.089 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.089 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.089 INFO:teuthology.orchestra.run.smithi077.stdout: "end_weekday": { 2024-09-10T10:04:43.089 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "end_weekday", 2024-09-10T10:04:43.089 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "uint", 2024-09-10T10:04:43.089 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.089 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.089 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-09-10T10:04:43.090 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "0", 2024-09-10T10:04:43.090 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "6", 2024-09-10T10:04:43.090 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.090 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-10T10:04:43.090 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-10T10:04:43.090 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.090 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.090 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.090 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:43.090 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:43.090 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.091 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.091 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.091 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.091 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.091 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.091 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.091 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.091 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.091 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.091 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.091 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.091 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.091 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.092 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.092 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.092 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.092 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.092 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.092 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:43.092 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:43.092 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.092 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.092 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.092 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.092 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.092 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.092 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.093 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.093 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.093 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.093 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.093 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.093 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:43.093 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:43.093 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.093 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.093 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.093 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:43.093 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.093 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.093 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.094 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.094 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.094 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.094 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.094 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.094 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.094 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.094 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.094 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.094 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.094 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.094 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.094 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:43.094 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:43.095 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.095 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.095 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.095 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.095 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.095 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.095 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.095 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.095 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.095 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.095 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.095 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.095 INFO:teuthology.orchestra.run.smithi077.stdout: "min_score": { 2024-09-10T10:04:43.095 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "min_score", 2024-09-10T10:04:43.096 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "float", 2024-09-10T10:04:43.096 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.096 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.096 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-09-10T10:04:43.096 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.096 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.096 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.096 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-10T10:04:43.096 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.096 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.096 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.096 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.097 INFO:teuthology.orchestra.run.smithi077.stdout: "mode": { 2024-09-10T10:04:43.097 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "mode", 2024-09-10T10:04:43.097 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.097 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.097 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.097 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "upmap", 2024-09-10T10:04:43.097 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.097 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.097 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.097 INFO:teuthology.orchestra.run.smithi077.stdout: "crush-compat", 2024-09-10T10:04:43.097 INFO:teuthology.orchestra.run.smithi077.stdout: "none", 2024-09-10T10:04:43.097 INFO:teuthology.orchestra.run.smithi077.stdout: "read", 2024-09-10T10:04:43.097 INFO:teuthology.orchestra.run.smithi077.stdout: "upmap", 2024-09-10T10:04:43.098 INFO:teuthology.orchestra.run.smithi077.stdout: "upmap-read" 2024-09-10T10:04:43.098 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.098 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Balancer mode", 2024-09-10T10:04:43.098 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.098 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.098 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.098 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.098 INFO:teuthology.orchestra.run.smithi077.stdout: "pool_ids": { 2024-09-10T10:04:43.098 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "pool_ids", 2024-09-10T10:04:43.098 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.098 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.098 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.098 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.098 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.099 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.099 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.099 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-10T10:04:43.099 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.099 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.099 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.099 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.099 INFO:teuthology.orchestra.run.smithi077.stdout: "sleep_interval": { 2024-09-10T10:04:43.099 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sleep_interval", 2024-09-10T10:04:43.099 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-10T10:04:43.099 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.099 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.099 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "60", 2024-09-10T10:04:43.099 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.100 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.100 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.100 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-10T10:04:43.100 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.100 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.100 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.100 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.100 INFO:teuthology.orchestra.run.smithi077.stdout: "upmap_max_deviation": { 2024-09-10T10:04:43.100 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "upmap_max_deviation", 2024-09-10T10:04:43.100 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:43.100 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.100 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.100 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "5", 2024-09-10T10:04:43.101 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "1", 2024-09-10T10:04:43.101 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.101 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.101 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-10T10:04:43.101 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-10T10:04:43.101 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.101 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.101 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.101 INFO:teuthology.orchestra.run.smithi077.stdout: "upmap_max_optimizations": { 2024-09-10T10:04:43.101 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "upmap_max_optimizations", 2024-09-10T10:04:43.101 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "uint", 2024-09-10T10:04:43.101 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.101 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.101 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10", 2024-09-10T10:04:43.102 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.102 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.102 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.102 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-10T10:04:43.102 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.102 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.102 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.102 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.102 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.102 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.102 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:43.102 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "cephadm", 2024-09-10T10:04:43.102 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:43.102 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:43.103 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:43.103 INFO:teuthology.orchestra.run.smithi077.stdout: "agent_down_multiplier": { 2024-09-10T10:04:43.103 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "agent_down_multiplier", 2024-09-10T10:04:43.103 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "float", 2024-09-10T10:04:43.103 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.103 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.103 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "3.0", 2024-09-10T10:04:43.103 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.103 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.103 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.103 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-10T10:04:43.103 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.103 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.104 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.104 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.104 INFO:teuthology.orchestra.run.smithi077.stdout: "agent_refresh_rate": { 2024-09-10T10:04:43.104 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "agent_refresh_rate", 2024-09-10T10:04:43.104 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-10T10:04:43.104 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.104 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.104 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "20", 2024-09-10T10:04:43.104 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.104 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.104 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.104 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-10T10:04:43.104 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.104 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.105 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.105 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.105 INFO:teuthology.orchestra.run.smithi077.stdout: "agent_starting_port": { 2024-09-10T10:04:43.105 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "agent_starting_port", 2024-09-10T10:04:43.105 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:43.105 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.105 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.105 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "4721", 2024-09-10T10:04:43.105 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.105 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.105 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.105 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-10T10:04:43.105 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.105 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.106 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.106 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.106 INFO:teuthology.orchestra.run.smithi077.stdout: "allow_ptrace": { 2024-09-10T10:04:43.106 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "allow_ptrace", 2024-09-10T10:04:43.106 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.106 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.106 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.106 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.106 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.106 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.106 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.106 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-10T10:04:43.106 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-10T10:04:43.107 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.107 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.107 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.107 INFO:teuthology.orchestra.run.smithi077.stdout: "autotune_interval": { 2024-09-10T10:04:43.107 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "autotune_interval", 2024-09-10T10:04:43.107 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-10T10:04:43.107 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.107 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.107 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "600", 2024-09-10T10:04:43.107 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.107 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.107 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.107 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-10T10:04:43.108 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.108 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.108 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.108 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.108 INFO:teuthology.orchestra.run.smithi077.stdout: "autotune_memory_target_ratio": { 2024-09-10T10:04:43.108 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "autotune_memory_target_ratio", 2024-09-10T10:04:43.108 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "float", 2024-09-10T10:04:43.108 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.108 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.108 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0.7", 2024-09-10T10:04:43.108 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.108 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.108 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.108 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-10T10:04:43.109 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.109 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.109 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.109 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.109 INFO:teuthology.orchestra.run.smithi077.stdout: "cephadm_log_destination": { 2024-09-10T10:04:43.109 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "cephadm_log_destination", 2024-09-10T10:04:43.109 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.109 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.109 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.109 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.109 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.109 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.109 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.110 INFO:teuthology.orchestra.run.smithi077.stdout: "file", 2024-09-10T10:04:43.110 INFO:teuthology.orchestra.run.smithi077.stdout: "file,syslog", 2024-09-10T10:04:43.110 INFO:teuthology.orchestra.run.smithi077.stdout: "syslog" 2024-09-10T10:04:43.110 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.110 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-10T10:04:43.110 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.110 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.110 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.110 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.110 INFO:teuthology.orchestra.run.smithi077.stdout: "cgroups_split": { 2024-09-10T10:04:43.110 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "cgroups_split", 2024-09-10T10:04:43.110 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.110 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.110 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.111 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:43.111 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.111 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.111 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.111 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-10T10:04:43.111 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.111 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.111 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.111 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.111 INFO:teuthology.orchestra.run.smithi077.stdout: "config_checks_enabled": { 2024-09-10T10:04:43.111 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "config_checks_enabled", 2024-09-10T10:04:43.111 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.111 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.111 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.111 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.112 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.112 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.112 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.112 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-10T10:04:43.112 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.112 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.112 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.112 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.112 INFO:teuthology.orchestra.run.smithi077.stdout: "config_dashboard": { 2024-09-10T10:04:43.112 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "config_dashboard", 2024-09-10T10:04:43.112 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.112 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.112 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.112 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:43.113 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.113 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.113 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.113 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-10T10:04:43.113 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.113 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.113 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.113 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.113 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_alertmanager": { 2024-09-10T10:04:43.113 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_alertmanager", 2024-09-10T10:04:43.113 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.113 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.113 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.113 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-10T10:04:43.114 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.114 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.114 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.114 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Prometheus container image", 2024-09-10T10:04:43.114 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.114 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.114 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.114 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.114 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_base": { 2024-09-10T10:04:43.114 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_base", 2024-09-10T10:04:43.114 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.114 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.114 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.114 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-10T10:04:43.114 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.115 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.115 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.115 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Container image name, without the tag", 2024-09-10T10:04:43.115 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.115 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.115 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.115 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.115 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_elasticsearch": { 2024-09-10T10:04:43.115 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_elasticsearch", 2024-09-10T10:04:43.115 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.115 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.115 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.115 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-10T10:04:43.115 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.115 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.116 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.116 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "elasticsearch container image", 2024-09-10T10:04:43.116 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.116 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.116 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.116 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.116 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_grafana": { 2024-09-10T10:04:43.116 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_grafana", 2024-09-10T10:04:43.116 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.116 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.116 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.116 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-10T10:04:43.116 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.116 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.117 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.117 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Prometheus container image", 2024-09-10T10:04:43.117 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.117 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.117 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.117 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.117 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_haproxy": { 2024-09-10T10:04:43.117 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_haproxy", 2024-09-10T10:04:43.117 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.117 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.117 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.117 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-10T10:04:43.117 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.118 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.118 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.118 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "HAproxy container image", 2024-09-10T10:04:43.118 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.118 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.118 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.118 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.118 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_jaeger_agent": { 2024-09-10T10:04:43.118 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_jaeger_agent", 2024-09-10T10:04:43.118 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.118 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.118 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.118 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-10T10:04:43.118 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.119 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.119 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.119 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Jaeger agent container image", 2024-09-10T10:04:43.119 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.119 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.119 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.119 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.119 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_jaeger_collector": { 2024-09-10T10:04:43.119 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_jaeger_collector", 2024-09-10T10:04:43.119 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.119 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.119 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.119 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-10T10:04:43.119 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.120 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.120 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.120 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Jaeger collector container image", 2024-09-10T10:04:43.120 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.120 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.120 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.120 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.120 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_jaeger_query": { 2024-09-10T10:04:43.120 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_jaeger_query", 2024-09-10T10:04:43.120 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.120 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.120 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.120 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-10T10:04:43.120 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.121 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.121 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.121 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Jaeger query container image", 2024-09-10T10:04:43.121 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.121 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.121 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.121 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.121 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_keepalived": { 2024-09-10T10:04:43.121 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_keepalived", 2024-09-10T10:04:43.121 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.121 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.121 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.121 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-10T10:04:43.121 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.122 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.122 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.122 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Keepalived container image", 2024-09-10T10:04:43.122 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.122 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.122 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.122 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.122 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_loki": { 2024-09-10T10:04:43.122 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_loki", 2024-09-10T10:04:43.122 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.122 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.122 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.122 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-10T10:04:43.122 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.123 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.123 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.123 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Loki container image", 2024-09-10T10:04:43.123 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.123 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.123 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.123 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.123 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_node_exporter": { 2024-09-10T10:04:43.123 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_node_exporter", 2024-09-10T10:04:43.123 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.123 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.123 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.123 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-10T10:04:43.123 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.123 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.124 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.124 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Prometheus container image", 2024-09-10T10:04:43.124 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.124 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.124 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.124 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.124 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_nvmeof": { 2024-09-10T10:04:43.124 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_nvmeof", 2024-09-10T10:04:43.124 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.124 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.124 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.124 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-10T10:04:43.124 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.124 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.125 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.125 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Nvme-of container image", 2024-09-10T10:04:43.125 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.125 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.125 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.125 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.125 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_prometheus": { 2024-09-10T10:04:43.125 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_prometheus", 2024-09-10T10:04:43.125 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.125 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.125 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.125 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-10T10:04:43.125 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.125 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.125 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.126 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Prometheus container image", 2024-09-10T10:04:43.126 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.126 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.126 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.126 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.126 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_promtail": { 2024-09-10T10:04:43.126 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_promtail", 2024-09-10T10:04:43.126 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.126 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.126 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.126 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-10T10:04:43.126 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.126 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.126 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.126 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Promtail container image", 2024-09-10T10:04:43.127 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.127 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.127 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.127 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.127 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_samba": { 2024-09-10T10:04:43.127 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_samba", 2024-09-10T10:04:43.127 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.127 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.127 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.127 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-10T10:04:43.127 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.127 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.127 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.127 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Samba/SMB container image", 2024-09-10T10:04:43.127 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.128 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.128 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.128 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.128 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_snmp_gateway": { 2024-09-10T10:04:43.128 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_snmp_gateway", 2024-09-10T10:04:43.128 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.128 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.128 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.128 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-10T10:04:43.128 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.128 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.128 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.128 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "SNMP Gateway container image", 2024-09-10T10:04:43.128 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.129 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.129 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.129 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.129 INFO:teuthology.orchestra.run.smithi077.stdout: "container_init": { 2024-09-10T10:04:43.129 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_init", 2024-09-10T10:04:43.129 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.129 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.129 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.129 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:43.129 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.129 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.129 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.129 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Run podman/docker with `--init`", 2024-09-10T10:04:43.129 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.129 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.130 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.130 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.130 INFO:teuthology.orchestra.run.smithi077.stdout: "daemon_cache_timeout": { 2024-09-10T10:04:43.130 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "daemon_cache_timeout", 2024-09-10T10:04:43.130 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-10T10:04:43.130 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.130 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.130 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "600", 2024-09-10T10:04:43.130 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.130 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.130 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.131 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-10T10:04:43.131 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.131 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.131 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.131 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.131 INFO:teuthology.orchestra.run.smithi077.stdout: "default_cephadm_command_timeout": { 2024-09-10T10:04:43.131 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "default_cephadm_command_timeout", 2024-09-10T10:04:43.131 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:43.131 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.131 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.131 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "900", 2024-09-10T10:04:43.131 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.131 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.131 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.132 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-10T10:04:43.132 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.132 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.132 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.132 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.132 INFO:teuthology.orchestra.run.smithi077.stdout: "default_registry": { 2024-09-10T10:04:43.132 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "default_registry", 2024-09-10T10:04:43.132 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.132 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.132 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.132 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "docker.io", 2024-09-10T10:04:43.132 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.132 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.132 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.133 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-10T10:04:43.133 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.133 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.133 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.133 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.133 INFO:teuthology.orchestra.run.smithi077.stdout: "device_cache_timeout": { 2024-09-10T10:04:43.133 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "device_cache_timeout", 2024-09-10T10:04:43.133 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-10T10:04:43.133 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.133 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.133 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "1800", 2024-09-10T10:04:43.133 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.133 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.133 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.133 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "seconds to cache device inventory", 2024-09-10T10:04:43.133 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.134 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.134 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.134 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.134 INFO:teuthology.orchestra.run.smithi077.stdout: "device_enhanced_scan": { 2024-09-10T10:04:43.134 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "device_enhanced_scan", 2024-09-10T10:04:43.134 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.134 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.134 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.134 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.134 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.134 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.134 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.134 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-10T10:04:43.134 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.135 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.135 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.135 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.135 INFO:teuthology.orchestra.run.smithi077.stdout: "facts_cache_timeout": { 2024-09-10T10:04:43.135 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "facts_cache_timeout", 2024-09-10T10:04:43.135 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-10T10:04:43.135 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.135 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.135 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "60", 2024-09-10T10:04:43.135 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.135 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.135 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.135 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "seconds to cache host facts data", 2024-09-10T10:04:43.135 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.135 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.135 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.136 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.136 INFO:teuthology.orchestra.run.smithi077.stdout: "grafana_dashboards_path": { 2024-09-10T10:04:43.136 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "grafana_dashboards_path", 2024-09-10T10:04:43.136 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.136 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.136 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.136 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-10T10:04:43.136 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.136 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.136 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.136 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-10T10:04:43.136 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.136 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.136 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.136 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.137 INFO:teuthology.orchestra.run.smithi077.stdout: "host_check_interval": { 2024-09-10T10:04:43.137 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "host_check_interval", 2024-09-10T10:04:43.137 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-10T10:04:43.137 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.137 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.137 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "600", 2024-09-10T10:04:43.137 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.137 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.137 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.137 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "how frequently to perform a host check", 2024-09-10T10:04:43.137 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.137 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.137 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.137 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.137 INFO:teuthology.orchestra.run.smithi077.stdout: "hw_monitoring": { 2024-09-10T10:04:43.138 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "hw_monitoring", 2024-09-10T10:04:43.138 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.138 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.138 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.138 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.138 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.138 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.138 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.138 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-10T10:04:43.138 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.138 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.138 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.138 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.138 INFO:teuthology.orchestra.run.smithi077.stdout: "inventory_list_all": { 2024-09-10T10:04:43.138 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "inventory_list_all", 2024-09-10T10:04:43.139 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.139 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.139 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.139 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.139 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.139 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.139 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.139 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-10T10:04:43.139 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.139 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.139 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.139 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.139 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:43.139 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:43.139 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.140 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.140 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.140 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.140 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.140 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.140 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.140 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.140 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.140 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.140 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.140 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.140 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.140 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.140 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.140 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.141 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.141 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.141 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.141 INFO:teuthology.orchestra.run.smithi077.stdout: "log_refresh_metadata": { 2024-09-10T10:04:43.141 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_refresh_metadata", 2024-09-10T10:04:43.141 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.141 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.141 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.141 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.141 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.141 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.141 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.141 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-10T10:04:43.141 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.141 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.142 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.142 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.142 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:43.142 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:43.142 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.142 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.142 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.142 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:43.142 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.142 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.142 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.142 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-10T10:04:43.142 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.143 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.143 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.143 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.143 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:43.143 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:43.143 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.143 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.143 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.143 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:43.143 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.143 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.143 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.143 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.144 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.144 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.144 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.144 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.144 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.144 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.144 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.144 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.144 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.144 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.144 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.144 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:43.144 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:43.144 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.145 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.145 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.145 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.145 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.145 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.145 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.145 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.145 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.145 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.145 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.145 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.145 INFO:teuthology.orchestra.run.smithi077.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-10T10:04:43.145 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-10T10:04:43.145 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.145 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.146 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.146 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.146 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.146 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.146 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.146 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-10T10:04:43.146 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.146 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.146 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.146 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.146 INFO:teuthology.orchestra.run.smithi077.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-10T10:04:43.146 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-10T10:04:43.146 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.146 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.146 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.147 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "*", 2024-09-10T10:04:43.147 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.147 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.147 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.147 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-10T10:04:43.147 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.147 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.147 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.147 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.147 INFO:teuthology.orchestra.run.smithi077.stdout: "max_count_per_host": { 2024-09-10T10:04:43.147 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "max_count_per_host", 2024-09-10T10:04:43.147 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:43.147 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.147 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.147 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10", 2024-09-10T10:04:43.148 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.148 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.148 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.148 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "max number of daemons per service per host", 2024-09-10T10:04:43.148 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.148 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.148 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.148 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.148 INFO:teuthology.orchestra.run.smithi077.stdout: "max_osd_draining_count": { 2024-09-10T10:04:43.148 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "max_osd_draining_count", 2024-09-10T10:04:43.148 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:43.148 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.148 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.148 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10", 2024-09-10T10:04:43.148 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.149 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.149 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.149 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-10T10:04:43.149 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.149 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.149 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.149 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.149 INFO:teuthology.orchestra.run.smithi077.stdout: "migration_current": { 2024-09-10T10:04:43.149 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "migration_current", 2024-09-10T10:04:43.149 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:43.149 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.149 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.149 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.149 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.150 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.150 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.150 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "internal - do not modify", 2024-09-10T10:04:43.150 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.150 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.150 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.150 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.150 INFO:teuthology.orchestra.run.smithi077.stdout: "mode": { 2024-09-10T10:04:43.150 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "mode", 2024-09-10T10:04:43.150 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.150 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.150 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.150 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "root", 2024-09-10T10:04:43.150 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.151 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.151 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.151 INFO:teuthology.orchestra.run.smithi077.stdout: "cephadm-package", 2024-09-10T10:04:43.151 INFO:teuthology.orchestra.run.smithi077.stdout: "root" 2024-09-10T10:04:43.151 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.151 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "mode for remote execution of cephadm", 2024-09-10T10:04:43.151 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.151 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.151 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.151 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.151 INFO:teuthology.orchestra.run.smithi077.stdout: "oob_default_addr": { 2024-09-10T10:04:43.151 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "oob_default_addr", 2024-09-10T10:04:43.151 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.151 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.151 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.152 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "169.254.1.1", 2024-09-10T10:04:43.152 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.152 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.152 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.152 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-10T10:04:43.152 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.152 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.152 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.152 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.152 INFO:teuthology.orchestra.run.smithi077.stdout: "prometheus_alerts_path": { 2024-09-10T10:04:43.152 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "prometheus_alerts_path", 2024-09-10T10:04:43.152 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.152 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.152 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.152 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-10T10:04:43.153 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.153 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.153 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.153 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-10T10:04:43.153 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.153 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.153 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.153 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.153 INFO:teuthology.orchestra.run.smithi077.stdout: "registry_insecure": { 2024-09-10T10:04:43.153 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "registry_insecure", 2024-09-10T10:04:43.153 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.153 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.153 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.153 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.153 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.154 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.154 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.154 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-10T10:04:43.154 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.154 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.154 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.154 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.154 INFO:teuthology.orchestra.run.smithi077.stdout: "registry_password": { 2024-09-10T10:04:43.154 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "registry_password", 2024-09-10T10:04:43.154 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.154 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.154 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.154 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.154 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.154 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.155 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.155 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-10T10:04:43.155 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.155 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.155 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.155 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.155 INFO:teuthology.orchestra.run.smithi077.stdout: "registry_url": { 2024-09-10T10:04:43.155 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "registry_url", 2024-09-10T10:04:43.155 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.155 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.155 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.155 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.155 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.155 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.156 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.156 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-10T10:04:43.156 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.156 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.156 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.156 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.156 INFO:teuthology.orchestra.run.smithi077.stdout: "registry_username": { 2024-09-10T10:04:43.156 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "registry_username", 2024-09-10T10:04:43.156 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.156 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.156 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.156 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.156 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.157 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.157 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.157 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-10T10:04:43.157 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.157 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.157 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.157 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.157 INFO:teuthology.orchestra.run.smithi077.stdout: "secure_monitoring_stack": { 2024-09-10T10:04:43.157 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "secure_monitoring_stack", 2024-09-10T10:04:43.157 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.157 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.157 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.157 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.158 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.158 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.158 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.158 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-10T10:04:43.158 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.158 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.158 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.158 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.158 INFO:teuthology.orchestra.run.smithi077.stdout: "service_discovery_port": { 2024-09-10T10:04:43.158 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "service_discovery_port", 2024-09-10T10:04:43.158 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:43.158 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.158 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.158 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "8765", 2024-09-10T10:04:43.158 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.159 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.159 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.159 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "cephadm service discovery port", 2024-09-10T10:04:43.159 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.159 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.159 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.159 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.159 INFO:teuthology.orchestra.run.smithi077.stdout: "ssh_config_file": { 2024-09-10T10:04:43.159 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ssh_config_file", 2024-09-10T10:04:43.159 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.159 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.159 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.159 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.159 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.159 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.160 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.160 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-10T10:04:43.160 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.160 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.160 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.160 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.160 INFO:teuthology.orchestra.run.smithi077.stdout: "use_agent": { 2024-09-10T10:04:43.160 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "use_agent", 2024-09-10T10:04:43.160 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.160 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.160 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.160 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.160 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.160 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.161 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.161 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-10T10:04:43.161 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.161 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.161 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.161 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.161 INFO:teuthology.orchestra.run.smithi077.stdout: "use_repo_digest": { 2024-09-10T10:04:43.161 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "use_repo_digest", 2024-09-10T10:04:43.161 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.161 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.161 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.161 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:43.161 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.162 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.162 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.162 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-10T10:04:43.162 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.162 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.162 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.162 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.162 INFO:teuthology.orchestra.run.smithi077.stdout: "warn_on_failed_host_check": { 2024-09-10T10:04:43.162 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "warn_on_failed_host_check", 2024-09-10T10:04:43.162 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.162 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.162 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.162 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:43.162 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.162 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.163 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.163 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "raise a health warning if the host check fails", 2024-09-10T10:04:43.163 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.163 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.163 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.163 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.163 INFO:teuthology.orchestra.run.smithi077.stdout: "warn_on_stray_daemons": { 2024-09-10T10:04:43.163 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "warn_on_stray_daemons", 2024-09-10T10:04:43.163 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.163 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.163 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.163 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:43.163 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.163 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.163 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.164 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-10T10:04:43.164 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.164 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.164 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.164 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.164 INFO:teuthology.orchestra.run.smithi077.stdout: "warn_on_stray_hosts": { 2024-09-10T10:04:43.164 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "warn_on_stray_hosts", 2024-09-10T10:04:43.164 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.164 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.164 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.164 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:43.164 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.164 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.164 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.164 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-10T10:04:43.165 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.165 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.165 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.165 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.165 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.165 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.165 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:43.165 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "crash", 2024-09-10T10:04:43.165 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:43.165 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:43.165 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:43.165 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:43.165 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:43.165 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.165 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.165 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.166 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.166 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.166 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.166 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.166 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.166 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.166 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.166 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.166 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.166 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.167 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.167 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.167 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.167 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.167 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.167 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.167 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:43.167 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:43.167 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.167 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.167 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.167 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.167 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.167 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.168 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.168 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.168 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.168 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.168 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.168 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.168 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:43.168 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:43.168 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.168 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.168 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.168 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:43.168 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.168 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.169 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.169 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.169 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.169 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.169 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.169 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.169 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.169 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.169 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.169 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.169 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.169 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.169 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.170 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:43.170 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:43.170 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.170 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.170 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.170 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.170 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.170 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.170 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.170 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.170 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.170 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.170 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.170 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.171 INFO:teuthology.orchestra.run.smithi077.stdout: "retain_interval": { 2024-09-10T10:04:43.171 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "retain_interval", 2024-09-10T10:04:43.171 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-10T10:04:43.171 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.171 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.171 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "31536000", 2024-09-10T10:04:43.171 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.171 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.171 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.171 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-10T10:04:43.171 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.171 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.171 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.171 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.172 INFO:teuthology.orchestra.run.smithi077.stdout: "warn_recent_interval": { 2024-09-10T10:04:43.172 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "warn_recent_interval", 2024-09-10T10:04:43.172 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-10T10:04:43.172 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.172 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.172 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "1209600", 2024-09-10T10:04:43.172 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.172 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.172 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.172 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-10T10:04:43.172 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.172 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.172 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.173 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.173 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.173 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.173 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:43.173 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "dashboard", 2024-09-10T10:04:43.173 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:43.173 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:43.173 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:43.173 INFO:teuthology.orchestra.run.smithi077.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-10T10:04:43.173 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-10T10:04:43.173 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:43.173 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.173 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.174 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10", 2024-09-10T10:04:43.174 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.174 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.174 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.174 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.174 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.174 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.174 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.174 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.174 INFO:teuthology.orchestra.run.smithi077.stdout: "ALERTMANAGER_API_HOST": { 2024-09-10T10:04:43.174 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-10T10:04:43.174 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.174 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.174 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.175 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.175 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.175 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.175 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.175 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.175 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.175 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.175 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.175 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.175 INFO:teuthology.orchestra.run.smithi077.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-10T10:04:43.175 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-10T10:04:43.175 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.175 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.175 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.175 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:43.176 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.176 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.176 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.176 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.176 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.176 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.176 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.176 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.176 INFO:teuthology.orchestra.run.smithi077.stdout: "AUDIT_API_ENABLED": { 2024-09-10T10:04:43.176 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "AUDIT_API_ENABLED", 2024-09-10T10:04:43.176 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.176 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.176 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.176 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.177 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.177 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.177 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.177 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.177 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.177 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.177 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.177 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.177 INFO:teuthology.orchestra.run.smithi077.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-10T10:04:43.177 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-10T10:04:43.177 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.177 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.177 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.177 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:43.177 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.178 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.178 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.178 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.178 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.178 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.178 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.178 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.178 INFO:teuthology.orchestra.run.smithi077.stdout: "ENABLE_BROWSABLE_API": { 2024-09-10T10:04:43.178 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-10T10:04:43.178 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.178 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.178 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.178 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:43.178 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.178 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.179 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.179 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.179 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.179 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.179 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.179 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.179 INFO:teuthology.orchestra.run.smithi077.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-10T10:04:43.179 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-10T10:04:43.179 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.179 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.179 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.179 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:43.179 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.179 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.180 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.180 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.180 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.180 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.180 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.180 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.180 INFO:teuthology.orchestra.run.smithi077.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-10T10:04:43.180 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-10T10:04:43.180 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.180 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.180 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.180 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:43.180 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.180 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.181 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.181 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.181 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.181 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.181 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.181 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.181 INFO:teuthology.orchestra.run.smithi077.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-10T10:04:43.181 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-10T10:04:43.181 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.181 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.181 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.181 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:43.181 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.181 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.182 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.182 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.182 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.182 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.182 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.182 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.182 INFO:teuthology.orchestra.run.smithi077.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-10T10:04:43.182 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-10T10:04:43.182 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.182 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.182 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.182 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:43.182 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.183 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.183 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.183 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.183 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.183 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.183 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.183 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.183 INFO:teuthology.orchestra.run.smithi077.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-10T10:04:43.183 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-10T10:04:43.183 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.183 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.183 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.183 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:43.184 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.184 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.184 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.184 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.184 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.184 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.184 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.184 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.184 INFO:teuthology.orchestra.run.smithi077.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-10T10:04:43.184 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-10T10:04:43.184 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.184 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.184 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.184 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:43.185 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.185 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.185 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.185 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.185 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.185 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.185 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.185 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.185 INFO:teuthology.orchestra.run.smithi077.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-10T10:04:43.185 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-10T10:04:43.185 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.186 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.186 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.186 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:43.186 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.186 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.186 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.186 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.186 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.186 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.186 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.186 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.186 INFO:teuthology.orchestra.run.smithi077.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-10T10:04:43.187 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-10T10:04:43.187 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.187 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.187 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.187 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.187 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.187 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.187 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.187 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.187 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.187 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.187 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.187 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.188 INFO:teuthology.orchestra.run.smithi077.stdout: "GRAFANA_API_PASSWORD": { 2024-09-10T10:04:43.188 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-10T10:04:43.188 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.188 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.188 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.188 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "admin", 2024-09-10T10:04:43.188 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.188 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.188 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.188 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.188 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.188 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.188 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.188 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.189 INFO:teuthology.orchestra.run.smithi077.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-10T10:04:43.189 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-10T10:04:43.189 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.189 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.189 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.189 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:43.189 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.189 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.189 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.189 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.189 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.189 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.189 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.189 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.189 INFO:teuthology.orchestra.run.smithi077.stdout: "GRAFANA_API_URL": { 2024-09-10T10:04:43.190 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "GRAFANA_API_URL", 2024-09-10T10:04:43.190 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.190 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.190 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.190 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.190 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.190 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.190 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.190 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.190 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.190 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.190 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.190 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.190 INFO:teuthology.orchestra.run.smithi077.stdout: "GRAFANA_API_USERNAME": { 2024-09-10T10:04:43.190 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-10T10:04:43.191 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.191 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.191 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.191 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "admin", 2024-09-10T10:04:43.191 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.191 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.191 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.191 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.191 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.191 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.191 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.191 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.191 INFO:teuthology.orchestra.run.smithi077.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-10T10:04:43.191 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-10T10:04:43.192 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.192 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.192 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.192 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.192 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.192 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.192 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.192 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.192 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.192 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.192 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.192 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.192 INFO:teuthology.orchestra.run.smithi077.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-10T10:04:43.192 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-10T10:04:43.193 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.193 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.193 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.193 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.193 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.193 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.193 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.193 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.193 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.193 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.193 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.193 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.193 INFO:teuthology.orchestra.run.smithi077.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-10T10:04:43.193 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-10T10:04:43.194 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.194 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.194 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.194 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:43.194 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.194 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.194 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.194 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.194 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.194 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.194 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.194 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.194 INFO:teuthology.orchestra.run.smithi077.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-10T10:04:43.194 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-10T10:04:43.194 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.195 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.195 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.195 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.195 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.195 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.195 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.195 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.195 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.195 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.195 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.195 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.195 INFO:teuthology.orchestra.run.smithi077.stdout: "PROMETHEUS_API_HOST": { 2024-09-10T10:04:43.195 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-10T10:04:43.195 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.195 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.196 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.196 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.196 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.196 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.196 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.196 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.196 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.196 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.196 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.196 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.196 INFO:teuthology.orchestra.run.smithi077.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-10T10:04:43.196 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-10T10:04:43.196 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.197 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.197 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.197 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:43.197 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.197 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.197 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.197 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.197 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.197 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.197 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.197 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.197 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-10T10:04:43.197 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-10T10:04:43.197 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.198 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.198 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.198 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.198 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.198 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.198 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.198 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.198 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.198 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.198 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.198 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.198 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-10T10:04:43.198 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-10T10:04:43.198 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.198 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.199 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.199 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.199 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.199 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.199 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.199 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.199 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.199 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.199 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.199 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.199 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-10T10:04:43.199 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-10T10:04:43.199 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.199 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.199 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.200 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:43.200 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.200 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.200 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.200 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.200 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.200 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.200 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.200 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.200 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-10T10:04:43.200 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-10T10:04:43.200 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.200 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.200 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.201 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:43.201 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.201 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.201 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.201 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.201 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.201 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.201 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.201 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.201 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-10T10:04:43.201 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-10T10:04:43.201 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.201 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.202 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.202 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.202 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.202 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.202 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.202 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.202 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.202 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.202 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.202 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.202 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-10T10:04:43.202 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-10T10:04:43.202 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.203 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.203 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.203 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.203 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.203 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.203 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.203 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.203 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.203 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.203 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.203 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.203 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-10T10:04:43.203 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-10T10:04:43.203 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.204 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.204 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.204 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.204 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.204 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.204 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.204 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.204 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.204 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.204 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.204 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.204 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_ENABLED": { 2024-09-10T10:04:43.204 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-10T10:04:43.204 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.204 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.205 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.205 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:43.205 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.205 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.205 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.205 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.205 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.205 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.205 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.205 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.205 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-10T10:04:43.205 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-10T10:04:43.205 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.205 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.205 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.206 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-10T10:04:43.206 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.206 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.206 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.206 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.206 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.206 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.206 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.206 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.206 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-10T10:04:43.206 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-10T10:04:43.206 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:43.207 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.207 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.207 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10", 2024-09-10T10:04:43.207 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.207 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.207 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.207 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.207 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.207 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.207 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.207 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.207 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-10T10:04:43.207 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-10T10:04:43.208 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:43.208 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.208 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.208 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "8", 2024-09-10T10:04:43.208 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.208 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.208 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.208 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.208 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.208 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.208 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.208 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.208 INFO:teuthology.orchestra.run.smithi077.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-10T10:04:43.209 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-10T10:04:43.209 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:43.209 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.209 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.209 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "45", 2024-09-10T10:04:43.209 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.209 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.209 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.209 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.209 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.209 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.209 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.209 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.210 INFO:teuthology.orchestra.run.smithi077.stdout: "RGW_API_ACCESS_KEY": { 2024-09-10T10:04:43.210 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-10T10:04:43.210 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.210 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.210 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.210 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.210 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.210 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.210 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.210 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.210 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.210 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.210 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.211 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.211 INFO:teuthology.orchestra.run.smithi077.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-10T10:04:43.211 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-10T10:04:43.211 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.211 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.211 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.211 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "admin", 2024-09-10T10:04:43.211 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.211 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.211 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.211 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.211 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.211 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.211 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.212 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.212 INFO:teuthology.orchestra.run.smithi077.stdout: "RGW_API_SECRET_KEY": { 2024-09-10T10:04:43.212 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-10T10:04:43.212 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.212 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.212 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.212 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.212 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.212 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.212 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.212 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.213 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.213 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.213 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.213 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.213 INFO:teuthology.orchestra.run.smithi077.stdout: "RGW_API_SSL_VERIFY": { 2024-09-10T10:04:43.213 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-10T10:04:43.213 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.213 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.213 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.213 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:43.213 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.213 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.213 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.214 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.214 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.214 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.214 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.214 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.214 INFO:teuthology.orchestra.run.smithi077.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-10T10:04:43.214 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-10T10:04:43.214 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:43.214 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.214 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.214 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-09-10T10:04:43.214 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.214 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.214 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.215 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.215 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.215 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.215 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.215 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.215 INFO:teuthology.orchestra.run.smithi077.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-10T10:04:43.215 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-10T10:04:43.215 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:43.215 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.215 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.215 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10", 2024-09-10T10:04:43.215 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.215 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.216 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.216 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.216 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.216 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.216 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.216 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.216 INFO:teuthology.orchestra.run.smithi077.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-10T10:04:43.216 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-10T10:04:43.216 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:43.216 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.216 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.216 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "5", 2024-09-10T10:04:43.216 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.216 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.216 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.217 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.217 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.217 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.217 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.217 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.217 INFO:teuthology.orchestra.run.smithi077.stdout: "cross_origin_url": { 2024-09-10T10:04:43.217 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "cross_origin_url", 2024-09-10T10:04:43.217 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.217 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.217 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.217 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.217 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.217 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.217 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.218 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.218 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.218 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.218 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.218 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.218 INFO:teuthology.orchestra.run.smithi077.stdout: "crt_file": { 2024-09-10T10:04:43.218 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "crt_file", 2024-09-10T10:04:43.218 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.218 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.218 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.218 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.218 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.218 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.218 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.218 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.218 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.219 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.219 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.219 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.219 INFO:teuthology.orchestra.run.smithi077.stdout: "debug": { 2024-09-10T10:04:43.219 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "debug", 2024-09-10T10:04:43.219 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.219 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.219 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.219 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.219 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.219 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.219 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.219 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Enable/disable debug options", 2024-09-10T10:04:43.219 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.219 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.220 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.220 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.220 INFO:teuthology.orchestra.run.smithi077.stdout: "jwt_token_ttl": { 2024-09-10T10:04:43.220 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "jwt_token_ttl", 2024-09-10T10:04:43.220 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:43.220 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.220 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.220 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "28800", 2024-09-10T10:04:43.220 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.220 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.220 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.220 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.220 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.220 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.221 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.221 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.221 INFO:teuthology.orchestra.run.smithi077.stdout: "key_file": { 2024-09-10T10:04:43.221 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "key_file", 2024-09-10T10:04:43.221 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.221 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.221 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.221 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.221 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.221 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.221 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.221 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.221 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.221 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.221 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.222 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.222 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:43.222 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:43.222 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.222 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.222 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.222 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.222 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.222 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.222 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.222 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.222 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.222 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.222 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.222 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.223 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.223 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.223 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.223 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.223 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.223 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.223 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.223 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:43.223 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:43.223 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.223 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.223 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.223 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.223 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.223 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.224 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.224 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.224 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.224 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.224 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.224 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.224 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:43.224 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:43.224 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.224 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.224 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.224 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:43.224 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.225 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.225 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.225 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.225 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.225 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.225 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.225 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.225 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.225 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.225 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.225 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.225 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.225 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.225 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.226 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:43.226 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:43.226 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.226 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.226 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.226 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.226 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.226 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.226 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.226 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.226 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.226 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.226 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.226 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.226 INFO:teuthology.orchestra.run.smithi077.stdout: "motd": { 2024-09-10T10:04:43.227 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "motd", 2024-09-10T10:04:43.227 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.227 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.227 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.227 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.227 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.227 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.227 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.227 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "The message of the day", 2024-09-10T10:04:43.227 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.227 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.227 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.228 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.228 INFO:teuthology.orchestra.run.smithi077.stdout: "redirect_resolve_ip_addr": { 2024-09-10T10:04:43.228 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "redirect_resolve_ip_addr", 2024-09-10T10:04:43.228 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.228 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.228 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.228 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.228 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.228 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.228 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.228 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.228 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.229 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.229 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.229 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.229 INFO:teuthology.orchestra.run.smithi077.stdout: "server_addr": { 2024-09-10T10:04:43.229 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "server_addr", 2024-09-10T10:04:43.229 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.229 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.229 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.229 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "::", 2024-09-10T10:04:43.229 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.229 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.230 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.230 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.230 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.230 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.230 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.230 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.230 INFO:teuthology.orchestra.run.smithi077.stdout: "server_port": { 2024-09-10T10:04:43.230 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "server_port", 2024-09-10T10:04:43.230 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:43.230 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.230 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.230 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "8080", 2024-09-10T10:04:43.230 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.231 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.231 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.231 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.231 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.231 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.231 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.231 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.231 INFO:teuthology.orchestra.run.smithi077.stdout: "ssl": { 2024-09-10T10:04:43.231 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ssl", 2024-09-10T10:04:43.231 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.231 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.231 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.231 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:43.232 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.232 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.232 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.232 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.232 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.232 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.232 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.232 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.232 INFO:teuthology.orchestra.run.smithi077.stdout: "ssl_server_port": { 2024-09-10T10:04:43.232 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ssl_server_port", 2024-09-10T10:04:43.232 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:43.232 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.233 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.233 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "8443", 2024-09-10T10:04:43.233 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.233 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.233 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.233 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.233 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.233 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.233 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.233 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.233 INFO:teuthology.orchestra.run.smithi077.stdout: "standby_behaviour": { 2024-09-10T10:04:43.233 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "standby_behaviour", 2024-09-10T10:04:43.234 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.234 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.234 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.234 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "redirect", 2024-09-10T10:04:43.234 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.234 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.234 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.234 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.234 INFO:teuthology.orchestra.run.smithi077.stdout: "redirect" 2024-09-10T10:04:43.234 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.235 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.235 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.235 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.235 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.235 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.235 INFO:teuthology.orchestra.run.smithi077.stdout: "standby_error_status_code": { 2024-09-10T10:04:43.235 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "standby_error_status_code", 2024-09-10T10:04:43.235 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:43.235 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.235 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.235 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "500", 2024-09-10T10:04:43.235 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "400", 2024-09-10T10:04:43.236 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "599", 2024-09-10T10:04:43.236 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.236 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.236 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.236 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.236 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.236 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.236 INFO:teuthology.orchestra.run.smithi077.stdout: "url_prefix": { 2024-09-10T10:04:43.236 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "url_prefix", 2024-09-10T10:04:43.236 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.236 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.237 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.237 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.237 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.237 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.237 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.237 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.237 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.237 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.237 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.237 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.237 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.237 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.237 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:43.238 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "devicehealth", 2024-09-10T10:04:43.238 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:43.238 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:43.238 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:43.238 INFO:teuthology.orchestra.run.smithi077.stdout: "enable_monitoring": { 2024-09-10T10:04:43.238 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "enable_monitoring", 2024-09-10T10:04:43.238 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.238 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.238 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.238 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:43.238 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.238 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.238 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.239 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "monitor device health metrics", 2024-09-10T10:04:43.239 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.239 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.239 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.239 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.239 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:43.239 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:43.239 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.239 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.239 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.239 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.239 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.240 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.240 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.240 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.240 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.240 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.240 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.240 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.240 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.240 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.240 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.240 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.240 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.240 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.241 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.241 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:43.241 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:43.241 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.241 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.241 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.241 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.241 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.241 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.241 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.241 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.241 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.241 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.242 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.242 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.242 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:43.242 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:43.242 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.242 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.242 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.242 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:43.242 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.242 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.242 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.242 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.242 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.242 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.243 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.243 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.243 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.243 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.243 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.243 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.243 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.243 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.243 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.243 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:43.243 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:43.243 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.243 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.243 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.244 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.244 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.244 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.244 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.244 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.244 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.244 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.244 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.244 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.244 INFO:teuthology.orchestra.run.smithi077.stdout: "mark_out_threshold": { 2024-09-10T10:04:43.244 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "mark_out_threshold", 2024-09-10T10:04:43.244 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-10T10:04:43.244 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.245 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.245 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "2419200", 2024-09-10T10:04:43.245 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.245 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.245 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.245 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-10T10:04:43.245 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.245 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.245 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.245 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.245 INFO:teuthology.orchestra.run.smithi077.stdout: "pool_name": { 2024-09-10T10:04:43.245 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "pool_name", 2024-09-10T10:04:43.245 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.245 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.246 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.246 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "device_health_metrics", 2024-09-10T10:04:43.246 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.246 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.246 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.246 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-10T10:04:43.246 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.246 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.246 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.246 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.246 INFO:teuthology.orchestra.run.smithi077.stdout: "retention_period": { 2024-09-10T10:04:43.246 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "retention_period", 2024-09-10T10:04:43.246 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-10T10:04:43.246 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.247 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.247 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "15552000", 2024-09-10T10:04:43.247 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.247 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.247 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.247 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "how long to retain device health metrics", 2024-09-10T10:04:43.247 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.247 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.247 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.247 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.247 INFO:teuthology.orchestra.run.smithi077.stdout: "scrape_frequency": { 2024-09-10T10:04:43.247 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "scrape_frequency", 2024-09-10T10:04:43.247 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-10T10:04:43.248 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.248 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.248 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "86400", 2024-09-10T10:04:43.248 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.248 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.248 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.248 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-10T10:04:43.248 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.248 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.248 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.248 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.248 INFO:teuthology.orchestra.run.smithi077.stdout: "self_heal": { 2024-09-10T10:04:43.248 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "self_heal", 2024-09-10T10:04:43.248 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.249 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.249 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.249 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:43.249 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.249 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.249 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.249 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-10T10:04:43.249 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.249 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.249 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.249 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.249 INFO:teuthology.orchestra.run.smithi077.stdout: "sleep_interval": { 2024-09-10T10:04:43.250 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sleep_interval", 2024-09-10T10:04:43.250 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-10T10:04:43.250 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.250 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.250 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "600", 2024-09-10T10:04:43.250 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.250 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.250 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.250 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "how frequently to wake up and check device health", 2024-09-10T10:04:43.250 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.250 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.250 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.250 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.250 INFO:teuthology.orchestra.run.smithi077.stdout: "warn_threshold": { 2024-09-10T10:04:43.251 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "warn_threshold", 2024-09-10T10:04:43.251 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-10T10:04:43.251 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.251 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.251 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "7257600", 2024-09-10T10:04:43.251 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.251 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.251 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.251 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-10T10:04:43.251 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.251 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.251 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.251 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.251 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.252 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.252 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:43.252 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "diskprediction_local", 2024-09-10T10:04:43.252 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:43.252 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:43.252 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:43.252 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:43.252 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:43.252 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.252 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.252 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.252 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.252 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.252 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.253 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.253 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.253 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.253 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.253 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.253 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.253 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.253 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.253 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.253 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.253 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.253 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.253 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.253 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:43.254 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:43.254 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.254 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.254 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.254 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.254 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.254 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.254 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.254 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.254 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.254 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.254 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.254 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.254 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:43.254 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:43.255 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.255 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.255 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.255 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:43.255 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.255 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.255 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.255 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.255 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.255 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.255 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.255 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.255 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.255 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.255 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.255 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.256 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.256 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.256 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.256 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:43.256 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:43.256 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.256 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.256 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.256 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.256 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.256 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.256 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.256 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.257 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.257 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.257 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.257 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.257 INFO:teuthology.orchestra.run.smithi077.stdout: "predict_interval": { 2024-09-10T10:04:43.257 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "predict_interval", 2024-09-10T10:04:43.257 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.257 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.257 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.257 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "86400", 2024-09-10T10:04:43.257 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.257 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.257 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.258 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.258 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.258 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.258 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.258 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.258 INFO:teuthology.orchestra.run.smithi077.stdout: "predictor_model": { 2024-09-10T10:04:43.258 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "predictor_model", 2024-09-10T10:04:43.258 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.258 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.258 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.258 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "prophetstor", 2024-09-10T10:04:43.258 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.258 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.258 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.259 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.259 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.259 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.259 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.259 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.259 INFO:teuthology.orchestra.run.smithi077.stdout: "sleep_interval": { 2024-09-10T10:04:43.259 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sleep_interval", 2024-09-10T10:04:43.259 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.259 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.259 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.259 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "600", 2024-09-10T10:04:43.259 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.259 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.259 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.260 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.260 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.260 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.260 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.260 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.260 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.260 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.260 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:43.260 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "influx", 2024-09-10T10:04:43.260 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": false, 2024-09-10T10:04:43.260 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "influxdb python module not found", 2024-09-10T10:04:43.260 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:43.260 INFO:teuthology.orchestra.run.smithi077.stdout: "batch_size": { 2024-09-10T10:04:43.260 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "batch_size", 2024-09-10T10:04:43.261 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:43.261 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.261 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.261 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "5000", 2024-09-10T10:04:43.261 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.261 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.261 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.261 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-10T10:04:43.261 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.261 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.261 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.261 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.261 INFO:teuthology.orchestra.run.smithi077.stdout: "database": { 2024-09-10T10:04:43.262 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "database", 2024-09-10T10:04:43.262 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.262 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.262 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.262 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "ceph", 2024-09-10T10:04:43.262 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.262 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.262 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.262 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-10T10:04:43.262 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.262 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.262 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.262 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.262 INFO:teuthology.orchestra.run.smithi077.stdout: "hostname": { 2024-09-10T10:04:43.263 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "hostname", 2024-09-10T10:04:43.263 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.263 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.263 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.263 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.263 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.263 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.263 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.263 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "InfluxDB server hostname", 2024-09-10T10:04:43.263 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.263 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.263 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.263 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.263 INFO:teuthology.orchestra.run.smithi077.stdout: "interval": { 2024-09-10T10:04:43.264 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "interval", 2024-09-10T10:04:43.264 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-10T10:04:43.264 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.264 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.264 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "30", 2024-09-10T10:04:43.264 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "5", 2024-09-10T10:04:43.264 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.264 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.264 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-10T10:04:43.264 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.264 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.264 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.264 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.265 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:43.265 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:43.265 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.265 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.265 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.265 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.265 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.265 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.265 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.265 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.265 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.265 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.266 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.266 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.266 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.266 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.266 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.266 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.266 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.266 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.266 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.266 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:43.266 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:43.266 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.266 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.267 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.267 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.267 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.267 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.267 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.267 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.267 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.267 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.267 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.267 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.267 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:43.267 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:43.268 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.268 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.268 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.268 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:43.268 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.268 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.268 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.268 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.268 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.268 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.268 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.268 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.268 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.269 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.269 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.269 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.269 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.269 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.269 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.269 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:43.269 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:43.269 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.269 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.269 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.269 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.270 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.270 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.270 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.270 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.270 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.270 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.270 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.270 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.270 INFO:teuthology.orchestra.run.smithi077.stdout: "password": { 2024-09-10T10:04:43.270 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "password", 2024-09-10T10:04:43.270 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.270 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.270 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.270 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.271 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.271 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.271 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.271 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "password of InfluxDB server user", 2024-09-10T10:04:43.271 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.271 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.271 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.271 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.271 INFO:teuthology.orchestra.run.smithi077.stdout: "port": { 2024-09-10T10:04:43.271 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "port", 2024-09-10T10:04:43.271 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:43.271 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.271 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.272 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "8086", 2024-09-10T10:04:43.272 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.272 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.272 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.272 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "InfluxDB server port", 2024-09-10T10:04:43.272 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.272 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.272 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.272 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.272 INFO:teuthology.orchestra.run.smithi077.stdout: "ssl": { 2024-09-10T10:04:43.272 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ssl", 2024-09-10T10:04:43.272 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.272 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.273 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.273 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "false", 2024-09-10T10:04:43.273 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.273 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.273 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.273 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-10T10:04:43.273 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.273 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.273 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.273 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.273 INFO:teuthology.orchestra.run.smithi077.stdout: "threads": { 2024-09-10T10:04:43.273 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "threads", 2024-09-10T10:04:43.273 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:43.273 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.274 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.274 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "5", 2024-09-10T10:04:43.274 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "1", 2024-09-10T10:04:43.274 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "32", 2024-09-10T10:04:43.274 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.274 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-10T10:04:43.274 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.274 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.274 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.274 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.274 INFO:teuthology.orchestra.run.smithi077.stdout: "username": { 2024-09-10T10:04:43.274 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "username", 2024-09-10T10:04:43.274 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.274 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.274 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.275 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.275 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.275 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.275 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.275 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "username of InfluxDB server user", 2024-09-10T10:04:43.275 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.275 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.275 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.275 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.275 INFO:teuthology.orchestra.run.smithi077.stdout: "verify_ssl": { 2024-09-10T10:04:43.275 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "verify_ssl", 2024-09-10T10:04:43.275 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.275 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.275 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.275 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "true", 2024-09-10T10:04:43.275 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.276 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.276 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.276 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-10T10:04:43.276 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.276 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.276 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.276 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.276 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.276 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.276 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:43.276 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "insights", 2024-09-10T10:04:43.276 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:43.277 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:43.277 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:43.277 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:43.277 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:43.277 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.277 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.277 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.277 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.277 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.277 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.277 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.277 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.277 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.277 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.278 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.278 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.278 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.278 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.278 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.278 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.278 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.278 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.278 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.278 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:43.278 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:43.278 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.278 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.278 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.278 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.279 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.279 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.279 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.279 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.279 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.279 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.279 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.279 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.279 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:43.279 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:43.279 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.279 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.279 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.279 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:43.280 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.280 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.280 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.280 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.280 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.280 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.280 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.280 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.280 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.280 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.280 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.280 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.280 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.280 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.280 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.281 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:43.281 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:43.281 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.281 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.281 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.281 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.281 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.281 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.281 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.281 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.281 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.281 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.281 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.282 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.282 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.282 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.282 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:43.282 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "iostat", 2024-09-10T10:04:43.282 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:43.282 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:43.282 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:43.282 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:43.282 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:43.282 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.282 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.282 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.282 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.282 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.283 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.283 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.283 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.283 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.283 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.283 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.283 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.283 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.283 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.283 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.283 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.283 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.283 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.283 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.284 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:43.284 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:43.284 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.284 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.284 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.284 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.284 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.284 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.284 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.284 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.284 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.284 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.284 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.284 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.285 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:43.285 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:43.285 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.285 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.285 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.285 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:43.285 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.285 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.285 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.285 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.285 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.285 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.285 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.285 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.285 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.286 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.286 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.286 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.286 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.286 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.286 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.286 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:43.286 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:43.286 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.286 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.286 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.286 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.286 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.287 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.287 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.287 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.287 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.287 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.287 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.287 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.287 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.287 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.287 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:43.287 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "k8sevents", 2024-09-10T10:04:43.287 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:43.287 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:43.288 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:43.288 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_event_retention_days": { 2024-09-10T10:04:43.288 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ceph_event_retention_days", 2024-09-10T10:04:43.288 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:43.288 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.288 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.288 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "7", 2024-09-10T10:04:43.288 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.288 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.288 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.288 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-10T10:04:43.288 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.288 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.289 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.289 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.289 INFO:teuthology.orchestra.run.smithi077.stdout: "config_check_secs": { 2024-09-10T10:04:43.289 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "config_check_secs", 2024-09-10T10:04:43.289 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:43.289 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.289 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.289 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10", 2024-09-10T10:04:43.289 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "10", 2024-09-10T10:04:43.289 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.289 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.289 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-10T10:04:43.289 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.289 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.290 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.290 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.290 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:43.290 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:43.290 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.290 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.290 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.290 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.290 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.290 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.290 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.290 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.290 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.290 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.291 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.291 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.291 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.291 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.291 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.291 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.291 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.291 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.291 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.291 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:43.291 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:43.291 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.291 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.291 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.292 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.292 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.292 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.292 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.292 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.292 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.292 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.292 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.292 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.292 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:43.292 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:43.292 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.292 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.292 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.293 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:43.293 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.293 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.293 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.293 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.293 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.293 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.293 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.293 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.293 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.293 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.293 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.293 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.294 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.294 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.294 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.294 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:43.294 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:43.294 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.294 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.294 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.294 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.294 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.294 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.294 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.294 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.294 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.295 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.295 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.295 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.295 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.295 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.295 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:43.295 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "localpool", 2024-09-10T10:04:43.295 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:43.295 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:43.295 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:43.295 INFO:teuthology.orchestra.run.smithi077.stdout: "failure_domain": { 2024-09-10T10:04:43.295 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "failure_domain", 2024-09-10T10:04:43.295 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.295 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.296 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.296 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "host", 2024-09-10T10:04:43.296 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.296 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.296 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.296 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "failure domain for any created local pool", 2024-09-10T10:04:43.296 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-10T10:04:43.296 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.296 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.296 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.296 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:43.296 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:43.296 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.297 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.297 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.297 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.297 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.297 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.297 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.297 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.297 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.297 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.297 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.297 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.297 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.297 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.297 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.298 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.298 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.298 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.298 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.298 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:43.298 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:43.298 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.298 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.298 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.298 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.298 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.298 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.298 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.299 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.299 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.299 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.299 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.299 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.299 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:43.299 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:43.299 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.299 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.299 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.299 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:43.299 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.299 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.299 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.300 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.300 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.300 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.300 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.300 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.300 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.300 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.300 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.300 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.300 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.300 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.300 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.300 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:43.300 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:43.301 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.301 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.301 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.301 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.301 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.301 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.301 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.301 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.301 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.301 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.301 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.301 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.301 INFO:teuthology.orchestra.run.smithi077.stdout: "min_size": { 2024-09-10T10:04:43.301 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "min_size", 2024-09-10T10:04:43.302 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:43.302 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.302 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.302 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.302 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.302 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.302 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.302 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "default min_size for any created local pool", 2024-09-10T10:04:43.302 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-10T10:04:43.302 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.302 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.302 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.302 INFO:teuthology.orchestra.run.smithi077.stdout: "num_rep": { 2024-09-10T10:04:43.302 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "num_rep", 2024-09-10T10:04:43.302 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:43.303 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.303 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.303 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "3", 2024-09-10T10:04:43.303 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.303 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.303 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.303 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "default replica count for any created local pool", 2024-09-10T10:04:43.303 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.303 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.303 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.303 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.303 INFO:teuthology.orchestra.run.smithi077.stdout: "pg_num": { 2024-09-10T10:04:43.303 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "pg_num", 2024-09-10T10:04:43.303 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:43.304 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.304 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.304 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "128", 2024-09-10T10:04:43.304 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.304 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.304 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.304 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "default pg_num for any created local pool", 2024-09-10T10:04:43.304 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.304 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.305 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.305 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.305 INFO:teuthology.orchestra.run.smithi077.stdout: "prefix": { 2024-09-10T10:04:43.305 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "prefix", 2024-09-10T10:04:43.305 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.305 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.305 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.305 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.305 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.305 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.305 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.305 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "name prefix for any created local pool", 2024-09-10T10:04:43.305 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.305 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.306 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.306 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.306 INFO:teuthology.orchestra.run.smithi077.stdout: "subtree": { 2024-09-10T10:04:43.306 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "subtree", 2024-09-10T10:04:43.306 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.306 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.306 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.306 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "rack", 2024-09-10T10:04:43.306 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.306 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.306 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.306 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-10T10:04:43.306 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-10T10:04:43.307 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.307 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.307 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.307 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.307 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.307 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:43.307 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "mds_autoscaler", 2024-09-10T10:04:43.307 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:43.307 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:43.307 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:43.307 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:43.307 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:43.307 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.307 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.308 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.308 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.308 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.308 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.308 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.308 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.308 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.308 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.308 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.308 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.308 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.308 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.308 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.308 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.309 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.309 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.309 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.309 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:43.309 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:43.309 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.309 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.309 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.309 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.309 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.309 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.309 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.309 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.309 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.310 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.310 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.310 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.310 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:43.310 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:43.310 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.310 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.310 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.310 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:43.310 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.310 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.310 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.310 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.310 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.311 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.311 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.311 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.311 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.311 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.311 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.311 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.311 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.311 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.311 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.311 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:43.311 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:43.311 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.311 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.311 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.312 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.312 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.312 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.312 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.312 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.312 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.312 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.312 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.312 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.312 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.312 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.312 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:43.312 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "mirroring", 2024-09-10T10:04:43.313 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:43.313 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:43.313 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:43.313 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:43.313 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:43.313 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.313 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.313 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.313 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.313 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.313 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.313 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.314 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.314 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.314 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.314 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.314 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.314 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.314 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.314 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.314 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.314 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.314 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.314 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.314 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:43.315 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:43.315 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.315 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.315 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.315 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.315 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.315 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.315 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.315 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.315 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.315 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.315 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.315 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.315 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:43.316 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:43.316 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.316 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.316 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.316 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:43.316 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.316 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.316 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.316 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.316 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.316 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.316 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.316 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.316 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.317 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.317 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.317 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.317 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.317 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.317 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.317 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:43.317 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:43.317 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.317 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.317 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.317 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.317 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.317 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.318 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.318 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.318 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.318 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.318 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.318 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.318 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.318 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.318 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:43.318 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "nfs", 2024-09-10T10:04:43.318 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:43.318 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:43.318 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:43.318 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:43.319 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:43.319 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.319 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.319 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.319 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.319 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.319 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.319 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.319 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.319 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.319 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.319 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.319 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.319 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.319 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.320 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.320 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.320 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.320 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.320 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.320 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:43.320 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:43.320 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.320 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.320 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.320 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.320 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.320 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.320 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.321 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.321 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.321 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.321 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.321 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.321 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:43.321 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:43.321 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.321 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.321 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.321 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:43.321 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.321 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.321 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.322 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.322 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.322 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.322 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.322 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.322 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.322 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.322 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.322 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.322 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.322 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.322 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.322 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:43.322 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:43.323 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.323 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.323 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.323 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.323 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.323 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.323 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.323 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.323 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.323 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.323 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.323 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.323 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.323 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.324 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:43.324 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "orchestrator", 2024-09-10T10:04:43.324 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:43.324 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:43.324 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:43.324 INFO:teuthology.orchestra.run.smithi077.stdout: "fail_fs": { 2024-09-10T10:04:43.324 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "fail_fs", 2024-09-10T10:04:43.324 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.324 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.324 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.324 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.324 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.324 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.324 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.324 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-10T10:04:43.325 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.325 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.325 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.325 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.325 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:43.325 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:43.325 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.325 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.325 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.325 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.325 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.325 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.325 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.325 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.326 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.326 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.326 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.326 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.326 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.326 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.326 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.326 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.326 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.326 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.326 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.326 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:43.326 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:43.327 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.327 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.327 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.327 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.327 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.327 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.327 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.327 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.327 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.327 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.327 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.327 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.327 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:43.327 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:43.328 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.328 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.328 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.328 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:43.328 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.328 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.328 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.328 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.328 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.328 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.328 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.328 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.328 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.329 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.329 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.329 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.329 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.329 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.329 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.329 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:43.329 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:43.329 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.329 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.329 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.329 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.329 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.329 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.330 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.330 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.330 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.330 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.330 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.330 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.330 INFO:teuthology.orchestra.run.smithi077.stdout: "orchestrator": { 2024-09-10T10:04:43.330 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "orchestrator", 2024-09-10T10:04:43.330 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.330 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.330 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.330 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.330 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.330 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.331 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.331 INFO:teuthology.orchestra.run.smithi077.stdout: "cephadm", 2024-09-10T10:04:43.331 INFO:teuthology.orchestra.run.smithi077.stdout: "rook", 2024-09-10T10:04:43.331 INFO:teuthology.orchestra.run.smithi077.stdout: "test_orchestrator" 2024-09-10T10:04:43.331 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.331 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Orchestrator backend", 2024-09-10T10:04:43.331 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.331 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.331 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.331 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.331 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.331 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.331 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:43.332 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "osd_perf_query", 2024-09-10T10:04:43.332 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:43.332 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:43.332 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:43.332 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:43.332 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:43.332 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.332 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.332 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.332 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.332 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.332 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.332 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.333 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.333 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.333 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.333 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.333 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.333 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.333 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.333 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.333 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.333 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.333 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.333 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.333 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:43.333 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:43.334 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.334 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.334 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.334 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.334 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.334 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.334 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.334 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.334 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.334 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.334 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.334 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.334 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:43.334 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:43.335 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.335 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.335 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.335 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:43.335 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.335 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.335 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.335 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.335 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.335 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.335 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.335 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.335 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.335 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.336 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.336 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.336 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.336 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.336 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.336 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:43.336 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:43.336 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.336 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.336 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.336 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.336 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.336 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.336 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.337 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.337 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.337 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.337 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.337 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.337 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.337 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.337 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:43.337 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "osd_support", 2024-09-10T10:04:43.337 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:43.337 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:43.337 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:43.337 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:43.338 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:43.338 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.338 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.338 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.338 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.338 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.338 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.338 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.338 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.338 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.338 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.338 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.338 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.338 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.338 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.339 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.339 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.339 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.339 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.339 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.339 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:43.339 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:43.339 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.339 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.339 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.339 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.339 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.339 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.339 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.339 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.339 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.340 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.340 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.340 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.340 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:43.340 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:43.340 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.340 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.340 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.340 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:43.340 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.340 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.340 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.340 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.340 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.341 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.341 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.341 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.341 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.341 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.341 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.341 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.341 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.341 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.341 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.341 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:43.341 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:43.342 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.342 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.342 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.342 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.342 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.342 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.342 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.342 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.342 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.342 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.342 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.342 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.342 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.342 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.343 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:43.343 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "pg_autoscaler", 2024-09-10T10:04:43.343 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:43.343 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:43.343 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:43.343 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:43.343 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:43.343 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.343 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.343 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.343 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.343 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.343 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.344 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.344 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.344 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.344 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.344 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.344 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.344 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.344 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.344 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.344 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.344 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.344 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.344 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.344 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:43.345 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:43.345 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.345 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.345 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.345 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.345 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.345 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.345 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.345 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.345 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.345 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.345 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.345 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.345 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:43.345 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:43.346 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.346 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.346 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.346 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:43.346 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.346 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.346 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.346 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.346 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.346 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.346 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.346 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.347 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.347 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.347 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.347 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.347 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.347 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.347 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.347 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:43.347 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:43.347 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.347 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.347 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.347 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.348 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.348 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.348 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.348 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.348 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.348 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.348 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.348 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.348 INFO:teuthology.orchestra.run.smithi077.stdout: "sleep_interval": { 2024-09-10T10:04:43.348 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sleep_interval", 2024-09-10T10:04:43.348 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-10T10:04:43.348 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.348 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.349 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "60", 2024-09-10T10:04:43.349 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.349 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.349 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.349 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.349 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.349 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.349 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.349 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.349 INFO:teuthology.orchestra.run.smithi077.stdout: "threshold": { 2024-09-10T10:04:43.349 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "threshold", 2024-09-10T10:04:43.349 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "float", 2024-09-10T10:04:43.349 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.349 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.350 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "3.0", 2024-09-10T10:04:43.350 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "1.0", 2024-09-10T10:04:43.350 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.350 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.350 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "scaling threshold", 2024-09-10T10:04:43.350 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-10T10:04:43.350 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.350 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.350 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.350 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.350 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.350 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:43.350 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "progress", 2024-09-10T10:04:43.350 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:43.351 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:43.351 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:43.351 INFO:teuthology.orchestra.run.smithi077.stdout: "allow_pg_recovery_event": { 2024-09-10T10:04:43.351 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "allow_pg_recovery_event", 2024-09-10T10:04:43.351 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.351 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.351 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.351 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.351 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.351 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.351 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.351 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-10T10:04:43.351 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.351 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.351 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.351 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.352 INFO:teuthology.orchestra.run.smithi077.stdout: "enabled": { 2024-09-10T10:04:43.352 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "enabled", 2024-09-10T10:04:43.352 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.352 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.352 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.352 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:43.352 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.352 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.352 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.352 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.352 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.352 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.352 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.352 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.353 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:43.353 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:43.353 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.353 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.353 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.353 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.353 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.353 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.353 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.353 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.353 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.353 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.353 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.353 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.354 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.354 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.354 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.354 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.354 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.354 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.354 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.354 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:43.354 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:43.354 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.354 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.354 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.354 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.354 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.354 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.355 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.355 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.355 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.355 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.355 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.355 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.355 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:43.355 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:43.355 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.355 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.355 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.355 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:43.355 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.355 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.355 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.356 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.356 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.356 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.356 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.356 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.356 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.356 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.356 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.356 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.356 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.356 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.356 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.357 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:43.357 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:43.357 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.357 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.357 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.357 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.357 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.357 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.357 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.357 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.357 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.357 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.357 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.358 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.358 INFO:teuthology.orchestra.run.smithi077.stdout: "max_completed_events": { 2024-09-10T10:04:43.358 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "max_completed_events", 2024-09-10T10:04:43.358 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:43.358 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.358 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.358 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "50", 2024-09-10T10:04:43.358 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.358 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.358 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.358 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "number of past completed events to remember", 2024-09-10T10:04:43.358 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.358 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.358 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.358 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.359 INFO:teuthology.orchestra.run.smithi077.stdout: "sleep_interval": { 2024-09-10T10:04:43.359 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sleep_interval", 2024-09-10T10:04:43.359 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-10T10:04:43.359 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.359 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.359 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "5", 2024-09-10T10:04:43.359 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.359 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.359 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.359 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "how long the module is going to sleep", 2024-09-10T10:04:43.359 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.359 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.359 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.359 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.360 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.360 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.360 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:43.360 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "prometheus", 2024-09-10T10:04:43.360 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:43.360 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:43.360 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:43.360 INFO:teuthology.orchestra.run.smithi077.stdout: "cache": { 2024-09-10T10:04:43.360 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "cache", 2024-09-10T10:04:43.360 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.360 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.360 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.360 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:43.360 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.361 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.361 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.361 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.361 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.361 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.361 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.361 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.361 INFO:teuthology.orchestra.run.smithi077.stdout: "exclude_perf_counters": { 2024-09-10T10:04:43.361 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "exclude_perf_counters", 2024-09-10T10:04:43.361 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.361 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.361 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.361 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:43.361 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.362 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.362 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.362 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-10T10:04:43.362 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-10T10:04:43.362 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.362 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.362 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.362 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:43.362 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:43.362 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.362 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.362 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.362 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.363 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.363 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.363 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.363 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.363 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.363 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.363 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.363 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.363 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.363 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.363 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.363 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.363 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.363 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.363 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.364 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:43.364 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:43.364 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.364 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.364 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.364 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.364 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.364 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.364 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.364 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.364 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.364 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.364 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.364 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.364 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:43.365 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:43.365 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.365 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.365 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.365 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:43.365 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.365 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.365 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.365 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.365 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.365 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.365 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.366 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.366 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.366 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.366 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.366 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.366 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.366 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.366 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.366 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:43.366 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:43.366 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.366 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.366 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.367 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.367 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.367 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.367 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.367 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.367 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.367 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.367 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.367 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.367 INFO:teuthology.orchestra.run.smithi077.stdout: "rbd_stats_pools": { 2024-09-10T10:04:43.367 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rbd_stats_pools", 2024-09-10T10:04:43.367 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.368 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.368 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.368 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.368 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.368 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.368 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.368 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.368 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.368 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.368 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.368 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.368 INFO:teuthology.orchestra.run.smithi077.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-10T10:04:43.369 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-10T10:04:43.369 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:43.369 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.369 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.369 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "300", 2024-09-10T10:04:43.369 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.369 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.369 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.369 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.369 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.369 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.369 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.369 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.369 INFO:teuthology.orchestra.run.smithi077.stdout: "scrape_interval": { 2024-09-10T10:04:43.369 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "scrape_interval", 2024-09-10T10:04:43.370 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "float", 2024-09-10T10:04:43.370 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.370 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.370 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "15.0", 2024-09-10T10:04:43.370 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.370 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.370 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.370 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.370 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.370 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.370 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.370 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.370 INFO:teuthology.orchestra.run.smithi077.stdout: "server_addr": { 2024-09-10T10:04:43.370 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "server_addr", 2024-09-10T10:04:43.371 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.371 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.371 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.371 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "::", 2024-09-10T10:04:43.371 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.371 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.371 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.371 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-10T10:04:43.371 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.371 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.371 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.371 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.371 INFO:teuthology.orchestra.run.smithi077.stdout: "server_port": { 2024-09-10T10:04:43.371 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "server_port", 2024-09-10T10:04:43.371 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:43.372 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.372 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.372 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "9283", 2024-09-10T10:04:43.372 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.372 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.372 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.372 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-10T10:04:43.372 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.372 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.372 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.372 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.372 INFO:teuthology.orchestra.run.smithi077.stdout: "stale_cache_strategy": { 2024-09-10T10:04:43.372 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "stale_cache_strategy", 2024-09-10T10:04:43.372 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.372 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.373 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.373 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "log", 2024-09-10T10:04:43.373 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.373 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.373 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.373 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.373 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.373 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.373 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.373 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.373 INFO:teuthology.orchestra.run.smithi077.stdout: "standby_behaviour": { 2024-09-10T10:04:43.373 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "standby_behaviour", 2024-09-10T10:04:43.374 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.374 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.374 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.374 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "default", 2024-09-10T10:04:43.374 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.374 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.374 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.374 INFO:teuthology.orchestra.run.smithi077.stdout: "default", 2024-09-10T10:04:43.374 INFO:teuthology.orchestra.run.smithi077.stdout: "error" 2024-09-10T10:04:43.374 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.374 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.374 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.374 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.374 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.375 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.375 INFO:teuthology.orchestra.run.smithi077.stdout: "standby_error_status_code": { 2024-09-10T10:04:43.375 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "standby_error_status_code", 2024-09-10T10:04:43.375 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:43.375 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.375 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.375 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "500", 2024-09-10T10:04:43.375 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "400", 2024-09-10T10:04:43.375 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "599", 2024-09-10T10:04:43.375 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.375 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.375 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.375 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.375 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.376 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.376 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.376 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.376 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:43.376 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rbd_support", 2024-09-10T10:04:43.376 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:43.376 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:43.376 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:43.376 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:43.376 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:43.376 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.376 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.376 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.377 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.377 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.377 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.377 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.377 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.377 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.377 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.377 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.377 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.377 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.377 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.377 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.377 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.377 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.378 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.378 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.378 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:43.378 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:43.378 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.378 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.378 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.378 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.378 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.378 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.378 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.378 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.378 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.378 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.379 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.379 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.379 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:43.379 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:43.379 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.379 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.379 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.379 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:43.379 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.379 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.379 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.379 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.379 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.379 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.380 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.380 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.380 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.380 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.380 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.380 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.380 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.380 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.380 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.380 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:43.380 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:43.380 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.381 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.381 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.381 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.381 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.381 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.381 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.381 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.381 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.381 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.381 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.381 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.381 INFO:teuthology.orchestra.run.smithi077.stdout: "max_concurrent_snap_create": { 2024-09-10T10:04:43.381 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "max_concurrent_snap_create", 2024-09-10T10:04:43.381 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:43.382 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.382 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.382 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10", 2024-09-10T10:04:43.382 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.382 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.382 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.382 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.382 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.382 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.382 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.382 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.382 INFO:teuthology.orchestra.run.smithi077.stdout: "mirror_snapshot_schedule": { 2024-09-10T10:04:43.382 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "mirror_snapshot_schedule", 2024-09-10T10:04:43.382 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.383 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.383 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.383 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.383 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.383 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.383 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.383 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.383 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.383 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.383 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.383 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.383 INFO:teuthology.orchestra.run.smithi077.stdout: "trash_purge_schedule": { 2024-09-10T10:04:43.383 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "trash_purge_schedule", 2024-09-10T10:04:43.383 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.383 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.384 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.384 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.384 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.384 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.384 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.384 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.384 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.384 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.384 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.384 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.384 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.384 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.384 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:43.385 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "restful", 2024-09-10T10:04:43.385 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:43.385 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:43.385 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:43.385 INFO:teuthology.orchestra.run.smithi077.stdout: "enable_auth": { 2024-09-10T10:04:43.385 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "enable_auth", 2024-09-10T10:04:43.385 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.385 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.385 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.385 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:43.385 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.385 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.385 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.385 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.386 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.386 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.386 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.386 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.386 INFO:teuthology.orchestra.run.smithi077.stdout: "key_file": { 2024-09-10T10:04:43.386 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "key_file", 2024-09-10T10:04:43.386 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.386 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.386 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.386 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.386 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.386 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.386 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.386 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.387 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.387 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.387 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.387 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.387 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:43.387 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:43.387 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.387 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.387 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.387 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.387 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.387 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.387 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.388 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.388 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.388 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.388 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.388 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.388 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.388 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.388 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.388 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.388 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.388 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.388 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.388 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:43.388 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:43.389 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.389 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.389 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.389 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.389 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.389 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.389 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.389 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.389 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.389 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.389 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.389 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.389 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:43.389 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:43.390 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.390 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.390 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.390 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:43.390 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.390 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.390 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.390 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.390 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.390 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.390 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.390 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.390 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.390 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.390 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.391 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.391 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.391 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.391 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.391 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:43.391 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:43.391 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.391 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.391 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.391 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.391 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.391 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.391 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.392 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.392 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.392 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.392 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.392 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.392 INFO:teuthology.orchestra.run.smithi077.stdout: "server_addr": { 2024-09-10T10:04:43.392 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "server_addr", 2024-09-10T10:04:43.392 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.392 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.392 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.392 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.392 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.393 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.393 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.393 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.393 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.393 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.393 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.393 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.393 INFO:teuthology.orchestra.run.smithi077.stdout: "server_port": { 2024-09-10T10:04:43.393 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "server_port", 2024-09-10T10:04:43.393 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.393 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.393 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.393 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.394 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.394 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.394 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.394 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.394 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.394 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.394 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.394 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.394 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.394 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.394 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:43.394 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rgw", 2024-09-10T10:04:43.394 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:43.394 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:43.394 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:43.395 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:43.395 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:43.395 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.395 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.395 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.395 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.395 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.395 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.395 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.395 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.395 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.395 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.395 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.396 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.396 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.396 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.396 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.396 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.396 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.396 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.396 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.396 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:43.396 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:43.396 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.396 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.397 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.397 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.397 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.397 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.397 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.397 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.397 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.397 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.397 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.397 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.397 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:43.397 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:43.397 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.398 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.398 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.398 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:43.398 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.398 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.398 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.398 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.398 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.398 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.398 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.398 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.398 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.398 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.399 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.399 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.399 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.399 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.399 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.399 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:43.399 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:43.399 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.399 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.399 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.399 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.399 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.399 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.400 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.400 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.400 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.400 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.400 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.400 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.400 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.400 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.400 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:43.400 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rook", 2024-09-10T10:04:43.400 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:43.400 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:43.401 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:43.401 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:43.401 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:43.401 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.401 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.401 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.401 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.401 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.401 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.401 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.401 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.401 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.401 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.401 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.402 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.402 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.402 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.402 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.402 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.402 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.402 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.402 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.402 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:43.402 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:43.402 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.402 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.402 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.402 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.403 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.403 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.403 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.403 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.403 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.403 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.403 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.403 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.403 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:43.403 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:43.403 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.403 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.404 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.404 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:43.404 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.404 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.404 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.404 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.404 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.404 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.404 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.404 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.404 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.404 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.404 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.405 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.405 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.405 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.405 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.405 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:43.405 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:43.405 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.405 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.405 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.405 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.405 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.405 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.405 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.406 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.406 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.406 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.406 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.406 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.406 INFO:teuthology.orchestra.run.smithi077.stdout: "storage_class": { 2024-09-10T10:04:43.406 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "storage_class", 2024-09-10T10:04:43.406 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.406 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.406 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.406 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "local", 2024-09-10T10:04:43.406 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.407 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.407 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.407 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-10T10:04:43.407 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.407 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.407 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.407 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.407 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.407 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.407 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:43.407 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "selftest", 2024-09-10T10:04:43.407 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:43.408 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:43.408 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:43.408 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:43.408 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:43.408 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.408 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.408 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.408 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.408 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.408 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.408 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.408 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.408 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.409 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.409 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.409 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.409 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.409 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.409 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.409 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.409 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.409 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.409 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.409 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:43.409 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:43.409 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.410 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.410 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.410 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.410 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.410 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.410 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.410 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.410 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.410 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.410 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.410 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.410 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:43.410 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:43.410 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.411 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.411 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.411 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:43.411 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.411 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.411 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.411 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.411 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.411 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.411 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.411 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.411 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.411 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.411 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.411 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.411 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.412 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.412 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.412 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:43.412 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:43.412 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.412 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.412 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.412 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.412 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.412 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.412 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.412 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.412 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.412 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.413 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.413 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.413 INFO:teuthology.orchestra.run.smithi077.stdout: "roption1": { 2024-09-10T10:04:43.413 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "roption1", 2024-09-10T10:04:43.413 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.413 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.413 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.413 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.413 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.413 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.413 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.413 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.413 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.413 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.413 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.414 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.414 INFO:teuthology.orchestra.run.smithi077.stdout: "roption2": { 2024-09-10T10:04:43.414 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "roption2", 2024-09-10T10:04:43.414 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.414 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.414 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.414 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "xyz", 2024-09-10T10:04:43.414 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.414 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.414 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.414 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.414 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.414 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.414 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.414 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.414 INFO:teuthology.orchestra.run.smithi077.stdout: "rwoption1": { 2024-09-10T10:04:43.415 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rwoption1", 2024-09-10T10:04:43.415 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.415 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.415 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.415 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.415 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.415 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.415 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.415 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.415 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.415 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.415 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.415 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.416 INFO:teuthology.orchestra.run.smithi077.stdout: "rwoption2": { 2024-09-10T10:04:43.416 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rwoption2", 2024-09-10T10:04:43.416 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:43.416 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.416 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.416 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.416 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.416 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.416 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.416 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.416 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.416 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.416 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.416 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.416 INFO:teuthology.orchestra.run.smithi077.stdout: "rwoption3": { 2024-09-10T10:04:43.417 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rwoption3", 2024-09-10T10:04:43.417 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "float", 2024-09-10T10:04:43.417 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.417 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.417 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.417 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.417 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.417 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.417 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.417 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.417 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.417 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.417 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.418 INFO:teuthology.orchestra.run.smithi077.stdout: "rwoption4": { 2024-09-10T10:04:43.418 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rwoption4", 2024-09-10T10:04:43.418 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.418 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.418 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.418 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.418 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.418 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.418 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.418 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.418 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.418 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.418 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.418 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.419 INFO:teuthology.orchestra.run.smithi077.stdout: "rwoption5": { 2024-09-10T10:04:43.419 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rwoption5", 2024-09-10T10:04:43.419 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.419 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.419 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.419 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.419 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.419 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.419 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.419 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.419 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.419 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.419 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.419 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.420 INFO:teuthology.orchestra.run.smithi077.stdout: "rwoption6": { 2024-09-10T10:04:43.420 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rwoption6", 2024-09-10T10:04:43.420 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.420 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.420 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.420 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:43.420 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.420 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.420 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.420 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.420 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.420 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.420 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.420 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.421 INFO:teuthology.orchestra.run.smithi077.stdout: "rwoption7": { 2024-09-10T10:04:43.421 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rwoption7", 2024-09-10T10:04:43.421 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:43.421 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.421 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.421 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.421 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "1", 2024-09-10T10:04:43.421 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "42", 2024-09-10T10:04:43.421 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.421 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.421 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.421 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.421 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.421 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.422 INFO:teuthology.orchestra.run.smithi077.stdout: "testkey": { 2024-09-10T10:04:43.422 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "testkey", 2024-09-10T10:04:43.422 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.422 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.422 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.422 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.422 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.422 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.422 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.422 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.422 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.422 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.422 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.422 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.423 INFO:teuthology.orchestra.run.smithi077.stdout: "testlkey": { 2024-09-10T10:04:43.423 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "testlkey", 2024-09-10T10:04:43.423 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.423 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.423 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.423 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.423 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.423 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.423 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.423 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.423 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.423 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.423 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.423 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.423 INFO:teuthology.orchestra.run.smithi077.stdout: "testnewline": { 2024-09-10T10:04:43.424 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "testnewline", 2024-09-10T10:04:43.424 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.424 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.424 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.424 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.424 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.424 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.424 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.424 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.424 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.424 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.424 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.424 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.424 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.424 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.425 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:43.425 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "snap_schedule", 2024-09-10T10:04:43.425 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:43.425 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:43.425 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:43.425 INFO:teuthology.orchestra.run.smithi077.stdout: "allow_m_granularity": { 2024-09-10T10:04:43.425 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "allow_m_granularity", 2024-09-10T10:04:43.425 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.425 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.425 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.425 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.425 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.425 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.425 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.426 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "allow minute scheduled snapshots", 2024-09-10T10:04:43.426 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.426 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.426 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.426 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.426 INFO:teuthology.orchestra.run.smithi077.stdout: "dump_on_update": { 2024-09-10T10:04:43.426 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "dump_on_update", 2024-09-10T10:04:43.426 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.426 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.426 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.426 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.426 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.426 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.427 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.427 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "dump database to debug log on update", 2024-09-10T10:04:43.427 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.427 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.427 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.427 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.427 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:43.427 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:43.427 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.427 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.427 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.427 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.427 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.428 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.428 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.428 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.428 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.428 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.428 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.428 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.428 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.428 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.428 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.428 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.428 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.428 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.428 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.429 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:43.429 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:43.429 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.429 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.429 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.429 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.429 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.429 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.429 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.429 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.429 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.429 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.429 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.429 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.430 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:43.430 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:43.430 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.430 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.430 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.430 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:43.430 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.430 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.430 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.430 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.430 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.430 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.430 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.431 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.431 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.431 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.431 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.431 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.431 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.431 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.431 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.431 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:43.431 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:43.431 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.431 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.431 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.432 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.432 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.432 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.432 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.432 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.432 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.432 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.432 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.432 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.432 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.432 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.433 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:43.433 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "stats", 2024-09-10T10:04:43.433 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:43.433 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:43.433 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:43.433 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:43.433 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:43.433 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.433 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.433 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.433 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.433 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.433 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.434 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.434 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.434 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.434 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.434 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.434 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.434 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.434 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.434 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.434 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.434 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.434 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.434 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.435 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:43.435 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:43.435 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.435 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.435 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.435 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.435 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.435 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.435 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.435 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.435 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.435 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.435 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.435 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.436 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:43.436 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:43.436 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.436 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.436 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.436 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:43.436 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.436 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.436 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.436 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.436 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.436 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.436 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.436 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.437 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.437 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.437 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.437 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.437 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.437 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.437 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.437 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:43.437 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:43.437 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.437 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.438 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.438 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.438 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.438 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.438 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.438 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.438 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.438 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.438 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.438 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.438 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.438 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.438 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:43.438 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "status", 2024-09-10T10:04:43.439 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:43.439 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:43.439 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:43.439 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:43.439 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:43.439 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.439 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.439 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.439 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.439 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.439 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.439 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.439 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.440 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.440 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.440 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.440 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.440 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.440 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.440 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.440 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.440 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.440 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.440 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.440 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:43.441 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:43.441 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.441 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.441 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.441 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.441 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.441 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.441 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.441 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.441 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.441 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.441 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.441 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.442 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:43.442 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:43.442 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.442 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.442 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.442 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:43.442 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.442 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.442 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.442 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.442 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.442 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.443 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.443 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.443 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.443 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.443 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.443 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.443 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.443 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.443 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.443 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:43.443 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:43.443 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.443 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.444 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.444 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.444 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.444 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.444 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.444 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.444 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.444 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.444 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.444 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.444 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.445 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.445 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:43.445 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "telegraf", 2024-09-10T10:04:43.445 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:43.445 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:43.445 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:43.445 INFO:teuthology.orchestra.run.smithi077.stdout: "address": { 2024-09-10T10:04:43.445 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "address", 2024-09-10T10:04:43.445 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.445 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.445 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.445 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-10T10:04:43.445 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.445 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.446 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.446 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.446 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.446 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.446 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.446 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.446 INFO:teuthology.orchestra.run.smithi077.stdout: "interval": { 2024-09-10T10:04:43.446 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "interval", 2024-09-10T10:04:43.446 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-10T10:04:43.446 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.446 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.446 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "15", 2024-09-10T10:04:43.446 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.446 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.447 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.447 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.447 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.447 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.447 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.447 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.447 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:43.447 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:43.447 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.447 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.447 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.447 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.448 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.448 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.448 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.448 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.448 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.448 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.448 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.448 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.448 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.448 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.448 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.448 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.448 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.449 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.449 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.449 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:43.449 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:43.449 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.449 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.449 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.449 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.449 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.449 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.449 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.449 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.450 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.450 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.450 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.450 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.450 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:43.450 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:43.450 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.450 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.450 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.450 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:43.450 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.450 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.450 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.450 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.451 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.451 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.451 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.451 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.451 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.451 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.451 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.451 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.451 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.451 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.451 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.451 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:43.451 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:43.451 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.452 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.452 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.452 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.452 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.452 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.452 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.452 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.452 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.452 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.452 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.452 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.452 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.452 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.452 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:43.453 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "telemetry", 2024-09-10T10:04:43.453 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:43.453 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:43.453 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:43.453 INFO:teuthology.orchestra.run.smithi077.stdout: "channel_basic": { 2024-09-10T10:04:43.453 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "channel_basic", 2024-09-10T10:04:43.453 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.453 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.453 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.453 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:43.453 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.453 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.453 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.453 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-10T10:04:43.454 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.454 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.454 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.454 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.454 INFO:teuthology.orchestra.run.smithi077.stdout: "channel_crash": { 2024-09-10T10:04:43.454 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "channel_crash", 2024-09-10T10:04:43.454 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.454 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.454 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.454 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:43.454 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.454 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.454 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.454 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-10T10:04:43.454 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.454 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.455 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.455 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.455 INFO:teuthology.orchestra.run.smithi077.stdout: "channel_device": { 2024-09-10T10:04:43.455 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "channel_device", 2024-09-10T10:04:43.455 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.455 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.455 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.455 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:43.455 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.455 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.455 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.455 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-10T10:04:43.455 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.455 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.456 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.456 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.456 INFO:teuthology.orchestra.run.smithi077.stdout: "channel_ident": { 2024-09-10T10:04:43.456 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "channel_ident", 2024-09-10T10:04:43.456 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.456 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.456 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.456 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.456 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.456 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.456 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.456 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-10T10:04:43.456 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.456 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.457 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.457 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.457 INFO:teuthology.orchestra.run.smithi077.stdout: "channel_perf": { 2024-09-10T10:04:43.457 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "channel_perf", 2024-09-10T10:04:43.457 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.457 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.457 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.457 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.457 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.457 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.457 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.457 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-10T10:04:43.458 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.458 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.458 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.458 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.458 INFO:teuthology.orchestra.run.smithi077.stdout: "contact": { 2024-09-10T10:04:43.458 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "contact", 2024-09-10T10:04:43.458 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.458 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.458 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.458 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.458 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.458 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.459 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.459 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.459 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.459 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.459 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.459 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.459 INFO:teuthology.orchestra.run.smithi077.stdout: "description": { 2024-09-10T10:04:43.459 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "description", 2024-09-10T10:04:43.459 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.459 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.459 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.459 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.459 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.459 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.460 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.460 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.460 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.460 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.460 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.460 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.460 INFO:teuthology.orchestra.run.smithi077.stdout: "device_url": { 2024-09-10T10:04:43.460 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "device_url", 2024-09-10T10:04:43.460 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.460 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.460 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.461 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-10T10:04:43.461 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.461 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.461 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.461 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.461 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.461 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.461 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.461 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.461 INFO:teuthology.orchestra.run.smithi077.stdout: "enabled": { 2024-09-10T10:04:43.464 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "enabled", 2024-09-10T10:04:43.464 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.464 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.464 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.464 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.464 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.464 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.464 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.464 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.464 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.464 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.464 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.465 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.465 INFO:teuthology.orchestra.run.smithi077.stdout: "interval": { 2024-09-10T10:04:43.465 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "interval", 2024-09-10T10:04:43.465 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:43.465 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.465 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.465 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "24", 2024-09-10T10:04:43.465 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "8", 2024-09-10T10:04:43.465 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.465 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.465 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.465 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.465 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.466 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.466 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.466 INFO:teuthology.orchestra.run.smithi077.stdout: "last_opt_revision": { 2024-09-10T10:04:43.466 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "last_opt_revision", 2024-09-10T10:04:43.466 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:43.466 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.466 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.466 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "1", 2024-09-10T10:04:43.466 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.466 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.466 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.467 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.467 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.467 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.467 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.467 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.467 INFO:teuthology.orchestra.run.smithi077.stdout: "leaderboard": { 2024-09-10T10:04:43.467 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "leaderboard", 2024-09-10T10:04:43.467 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.467 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.467 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.467 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.467 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.467 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.467 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.467 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.468 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.468 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.468 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.468 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.468 INFO:teuthology.orchestra.run.smithi077.stdout: "leaderboard_description": { 2024-09-10T10:04:43.468 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "leaderboard_description", 2024-09-10T10:04:43.468 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.468 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.468 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.468 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.468 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.468 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.468 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.468 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.468 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.469 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.469 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.469 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.469 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:43.469 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:43.469 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.469 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.469 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.469 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.469 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.469 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.469 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.469 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.469 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.470 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.470 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.470 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.470 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.470 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.470 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.470 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.470 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.470 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.470 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.470 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:43.470 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:43.470 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.471 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.471 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.471 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.471 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.471 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.471 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.471 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.471 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.471 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.471 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.471 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.471 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:43.471 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:43.471 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.471 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.471 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.472 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:43.472 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.472 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.472 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.472 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.472 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.472 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.472 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.472 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.472 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.472 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.472 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.472 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.472 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.473 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.473 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.473 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:43.473 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:43.473 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.473 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.473 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.473 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.473 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.473 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.473 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.473 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.473 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.473 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.474 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.474 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.474 INFO:teuthology.orchestra.run.smithi077.stdout: "organization": { 2024-09-10T10:04:43.474 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "organization", 2024-09-10T10:04:43.474 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.474 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.474 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.474 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.474 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.474 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.474 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.474 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.474 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.474 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.475 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.475 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.475 INFO:teuthology.orchestra.run.smithi077.stdout: "proxy": { 2024-09-10T10:04:43.475 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "proxy", 2024-09-10T10:04:43.475 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.475 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.475 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.475 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.475 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.475 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.475 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.475 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.475 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.475 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.475 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.476 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.476 INFO:teuthology.orchestra.run.smithi077.stdout: "url": { 2024-09-10T10:04:43.476 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "url", 2024-09-10T10:04:43.476 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.476 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.476 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.476 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-10T10:04:43.476 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.476 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.476 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.476 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.476 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.476 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.477 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.477 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.477 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.477 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.477 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:43.477 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "test_orchestrator", 2024-09-10T10:04:43.477 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:43.477 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:43.477 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:43.477 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:43.477 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:43.477 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.477 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.477 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.478 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.478 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.478 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.478 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.478 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.478 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.478 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.478 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.478 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.478 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.478 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.478 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.478 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.478 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.478 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.479 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.479 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:43.479 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:43.479 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.479 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.479 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.479 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.479 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.479 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.479 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.479 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.479 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.480 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.480 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.480 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.480 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:43.480 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:43.480 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.480 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.480 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.480 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:43.480 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.480 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.480 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.480 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.480 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.481 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.481 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.481 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.481 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.481 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.481 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.481 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.481 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.481 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.481 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.481 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:43.481 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:43.481 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.481 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.481 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.482 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.482 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.482 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.482 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.482 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.482 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.482 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.482 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.482 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.482 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.482 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.482 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:43.482 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "volumes", 2024-09-10T10:04:43.482 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:43.482 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:43.483 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:43.483 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:43.483 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:43.483 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.483 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.483 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.483 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.483 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.483 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.483 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.483 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.483 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.483 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.483 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.484 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.484 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.484 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.484 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.484 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.484 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.484 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.484 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.484 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:43.484 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:43.484 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.484 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.484 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.485 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.485 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.485 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.485 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.485 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.485 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.485 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.485 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.485 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.485 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:43.485 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:43.485 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.485 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.486 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.486 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:43.486 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.486 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.486 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.486 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.486 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.486 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.486 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.486 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.486 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.486 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.486 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.487 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.487 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.487 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.487 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.487 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:43.487 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:43.487 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.487 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.487 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.487 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.487 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.487 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.488 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.488 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.488 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.488 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.488 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.488 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.488 INFO:teuthology.orchestra.run.smithi077.stdout: "max_concurrent_clones": { 2024-09-10T10:04:43.488 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "max_concurrent_clones", 2024-09-10T10:04:43.488 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:43.488 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.488 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.488 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "4", 2024-09-10T10:04:43.488 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.489 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.489 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.489 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-10T10:04:43.489 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.489 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.489 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.489 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.489 INFO:teuthology.orchestra.run.smithi077.stdout: "periodic_async_work": { 2024-09-10T10:04:43.489 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "periodic_async_work", 2024-09-10T10:04:43.489 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.489 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.489 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.489 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.490 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.490 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.490 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.490 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Periodically check for async work", 2024-09-10T10:04:43.490 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.490 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.490 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.490 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.490 INFO:teuthology.orchestra.run.smithi077.stdout: "snapshot_clone_delay": { 2024-09-10T10:04:43.490 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "snapshot_clone_delay", 2024-09-10T10:04:43.490 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:43.490 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.490 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.490 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-09-10T10:04:43.491 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.491 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.491 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.491 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-10T10:04:43.491 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.491 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.491 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.491 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.491 INFO:teuthology.orchestra.run.smithi077.stdout: "snapshot_clone_no_wait": { 2024-09-10T10:04:43.491 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "snapshot_clone_no_wait", 2024-09-10T10:04:43.491 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.491 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.492 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.492 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:43.492 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.492 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.492 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.492 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-10T10:04:43.492 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.492 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.492 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.492 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.492 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.492 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.492 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:43.492 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "zabbix", 2024-09-10T10:04:43.493 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:43.493 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:43.493 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:43.493 INFO:teuthology.orchestra.run.smithi077.stdout: "discovery_interval": { 2024-09-10T10:04:43.493 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "discovery_interval", 2024-09-10T10:04:43.493 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "uint", 2024-09-10T10:04:43.493 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.493 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.493 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "100", 2024-09-10T10:04:43.493 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.493 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.493 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.493 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.494 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.494 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.494 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.494 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.494 INFO:teuthology.orchestra.run.smithi077.stdout: "identifier": { 2024-09-10T10:04:43.494 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "identifier", 2024-09-10T10:04:43.494 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.494 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.494 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.494 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.494 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.495 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.495 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.495 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.495 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.495 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.495 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.495 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.495 INFO:teuthology.orchestra.run.smithi077.stdout: "interval": { 2024-09-10T10:04:43.495 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "interval", 2024-09-10T10:04:43.495 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-10T10:04:43.495 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.495 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.495 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "60", 2024-09-10T10:04:43.496 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.496 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.496 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.496 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.496 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.496 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.496 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.496 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.496 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:43.496 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:43.496 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.496 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.496 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.496 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.497 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.497 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.497 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.497 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.497 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.497 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.497 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.497 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.497 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.497 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.497 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.497 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.497 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.497 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.498 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.498 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:43.498 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:43.498 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.498 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.498 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.498 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.498 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.498 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.498 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.498 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.498 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.498 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.498 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.498 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.499 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:43.499 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:43.499 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.499 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.499 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.499 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:43.499 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.499 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.499 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.499 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.499 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.499 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.499 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.499 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.500 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.500 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.500 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.500 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.500 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.500 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.500 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.500 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:43.500 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:43.500 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.500 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.500 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.500 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.500 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.501 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.501 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.501 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.501 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.501 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.501 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.501 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.501 INFO:teuthology.orchestra.run.smithi077.stdout: "zabbix_host": { 2024-09-10T10:04:43.501 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "zabbix_host", 2024-09-10T10:04:43.501 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.501 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.501 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.501 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.501 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.501 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.502 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.502 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.502 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.502 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.502 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.502 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.502 INFO:teuthology.orchestra.run.smithi077.stdout: "zabbix_port": { 2024-09-10T10:04:43.502 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "zabbix_port", 2024-09-10T10:04:43.502 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:43.502 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.502 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.502 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10051", 2024-09-10T10:04:43.502 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.502 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.502 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.503 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.503 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.503 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.503 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.503 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.503 INFO:teuthology.orchestra.run.smithi077.stdout: "zabbix_sender": { 2024-09-10T10:04:43.503 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "zabbix_sender", 2024-09-10T10:04:43.503 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.503 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.503 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.503 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-10T10:04:43.503 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.503 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.503 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.504 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.504 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.504 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.504 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.504 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.504 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.504 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.504 INFO:teuthology.orchestra.run.smithi077.stdout: ] 2024-09-10T10:04:43.504 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.504 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.504 INFO:teuthology.orchestra.run.smithi077.stdout: "modules": [ 2024-09-10T10:04:43.504 INFO:teuthology.orchestra.run.smithi077.stdout: "cephadm", 2024-09-10T10:04:43.504 INFO:teuthology.orchestra.run.smithi077.stdout: "dashboard", 2024-09-10T10:04:43.504 INFO:teuthology.orchestra.run.smithi077.stdout: "iostat", 2024-09-10T10:04:43.505 INFO:teuthology.orchestra.run.smithi077.stdout: "nfs", 2024-09-10T10:04:43.505 INFO:teuthology.orchestra.run.smithi077.stdout: "restful" 2024-09-10T10:04:43.505 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.505 INFO:teuthology.orchestra.run.smithi077.stdout: "available_modules": [ 2024-09-10T10:04:43.505 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:43.505 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "alerts", 2024-09-10T10:04:43.505 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:43.505 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:43.505 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:43.505 INFO:teuthology.orchestra.run.smithi077.stdout: "interval": { 2024-09-10T10:04:43.505 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "interval", 2024-09-10T10:04:43.505 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-10T10:04:43.505 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.506 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.506 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "60", 2024-09-10T10:04:43.506 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.506 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.506 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.506 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "How frequently to reexamine health status", 2024-09-10T10:04:43.506 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.506 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.506 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.506 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.506 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:43.506 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:43.506 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.507 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.507 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.507 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.507 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.507 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.507 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.507 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.507 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.507 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.507 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.507 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.507 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.508 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.508 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.508 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.508 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.508 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.508 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.508 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:43.508 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:43.508 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.508 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.508 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.508 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.508 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.508 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.509 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.509 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.509 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.509 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.509 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.509 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.509 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:43.509 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:43.509 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.509 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.509 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.509 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:43.509 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.509 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.509 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.510 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.510 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.510 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.510 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.510 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.510 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.510 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.510 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.510 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.510 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.510 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.510 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.510 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:43.510 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:43.510 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.510 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.511 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.511 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.511 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.511 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.511 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.511 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.511 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.511 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.511 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.511 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.511 INFO:teuthology.orchestra.run.smithi077.stdout: "smtp_destination": { 2024-09-10T10:04:43.511 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smtp_destination", 2024-09-10T10:04:43.511 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.511 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.511 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.512 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.512 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.512 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.512 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.512 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Email address to send alerts to", 2024-09-10T10:04:43.512 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.512 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.512 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.512 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.512 INFO:teuthology.orchestra.run.smithi077.stdout: "smtp_from_name": { 2024-09-10T10:04:43.512 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smtp_from_name", 2024-09-10T10:04:43.512 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.512 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.512 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.512 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "Ceph", 2024-09-10T10:04:43.513 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.513 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.513 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.513 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Email From: name", 2024-09-10T10:04:43.513 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.513 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.513 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.513 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.513 INFO:teuthology.orchestra.run.smithi077.stdout: "smtp_host": { 2024-09-10T10:04:43.513 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smtp_host", 2024-09-10T10:04:43.513 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.513 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.513 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.513 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.513 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.514 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.514 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.514 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "SMTP server", 2024-09-10T10:04:43.514 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.514 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.514 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.514 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.514 INFO:teuthology.orchestra.run.smithi077.stdout: "smtp_password": { 2024-09-10T10:04:43.514 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smtp_password", 2024-09-10T10:04:43.514 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.514 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.514 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.514 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.514 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.515 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.515 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.515 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Password to authenticate with", 2024-09-10T10:04:43.515 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.515 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.515 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.515 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.515 INFO:teuthology.orchestra.run.smithi077.stdout: "smtp_port": { 2024-09-10T10:04:43.515 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smtp_port", 2024-09-10T10:04:43.515 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:43.515 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.515 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.515 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "465", 2024-09-10T10:04:43.515 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.516 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.516 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.516 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "SMTP port", 2024-09-10T10:04:43.516 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.516 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.516 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.516 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.516 INFO:teuthology.orchestra.run.smithi077.stdout: "smtp_sender": { 2024-09-10T10:04:43.516 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smtp_sender", 2024-09-10T10:04:43.516 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.516 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.516 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.516 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.516 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.516 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.517 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.517 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "SMTP envelope sender", 2024-09-10T10:04:43.517 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.517 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.517 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.517 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.517 INFO:teuthology.orchestra.run.smithi077.stdout: "smtp_ssl": { 2024-09-10T10:04:43.517 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smtp_ssl", 2024-09-10T10:04:43.517 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.517 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.517 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.517 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:43.517 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.517 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.517 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.518 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-10T10:04:43.518 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.518 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.518 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.518 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.518 INFO:teuthology.orchestra.run.smithi077.stdout: "smtp_user": { 2024-09-10T10:04:43.518 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smtp_user", 2024-09-10T10:04:43.518 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.518 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.518 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.518 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.518 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.518 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.518 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.518 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "User to authenticate as", 2024-09-10T10:04:43.519 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.519 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.519 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.519 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.519 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.519 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.519 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:43.519 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "balancer", 2024-09-10T10:04:43.519 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:43.519 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:43.519 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:43.519 INFO:teuthology.orchestra.run.smithi077.stdout: "active": { 2024-09-10T10:04:43.519 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "active", 2024-09-10T10:04:43.519 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.519 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.519 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.520 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:43.520 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.520 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.520 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.520 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "automatically balance PGs across cluster", 2024-09-10T10:04:43.520 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.520 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.520 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.520 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.520 INFO:teuthology.orchestra.run.smithi077.stdout: "begin_time": { 2024-09-10T10:04:43.520 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "begin_time", 2024-09-10T10:04:43.520 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.520 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.520 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.520 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0000", 2024-09-10T10:04:43.521 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.521 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.521 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.521 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "beginning time of day to automatically balance", 2024-09-10T10:04:43.521 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-10T10:04:43.521 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.521 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.521 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.521 INFO:teuthology.orchestra.run.smithi077.stdout: "begin_weekday": { 2024-09-10T10:04:43.521 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "begin_weekday", 2024-09-10T10:04:43.521 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "uint", 2024-09-10T10:04:43.521 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.521 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.521 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-09-10T10:04:43.521 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "0", 2024-09-10T10:04:43.522 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "6", 2024-09-10T10:04:43.522 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.522 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-10T10:04:43.522 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-10T10:04:43.522 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.522 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.522 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.522 INFO:teuthology.orchestra.run.smithi077.stdout: "crush_compat_max_iterations": { 2024-09-10T10:04:43.522 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "crush_compat_max_iterations", 2024-09-10T10:04:43.522 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "uint", 2024-09-10T10:04:43.522 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.522 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.522 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "25", 2024-09-10T10:04:43.522 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "1", 2024-09-10T10:04:43.523 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "250", 2024-09-10T10:04:43.523 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.523 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-10T10:04:43.523 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.523 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.523 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.523 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.523 INFO:teuthology.orchestra.run.smithi077.stdout: "crush_compat_metrics": { 2024-09-10T10:04:43.523 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "crush_compat_metrics", 2024-09-10T10:04:43.523 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.523 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.523 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.523 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "pgs,objects,bytes", 2024-09-10T10:04:43.523 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.523 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.524 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.524 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-10T10:04:43.524 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-10T10:04:43.524 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.524 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.524 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.524 INFO:teuthology.orchestra.run.smithi077.stdout: "crush_compat_step": { 2024-09-10T10:04:43.524 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "crush_compat_step", 2024-09-10T10:04:43.524 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "float", 2024-09-10T10:04:43.524 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.524 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.524 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0.5", 2024-09-10T10:04:43.524 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "0.001", 2024-09-10T10:04:43.524 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "0.999", 2024-09-10T10:04:43.524 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.525 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "aggressiveness of optimization", 2024-09-10T10:04:43.525 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-10T10:04:43.525 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.525 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.525 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.525 INFO:teuthology.orchestra.run.smithi077.stdout: "end_time": { 2024-09-10T10:04:43.525 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "end_time", 2024-09-10T10:04:43.525 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.525 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.525 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.525 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "2359", 2024-09-10T10:04:43.525 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.525 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.525 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.526 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "ending time of day to automatically balance", 2024-09-10T10:04:43.526 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-10T10:04:43.526 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.526 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.526 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.526 INFO:teuthology.orchestra.run.smithi077.stdout: "end_weekday": { 2024-09-10T10:04:43.526 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "end_weekday", 2024-09-10T10:04:43.526 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "uint", 2024-09-10T10:04:43.526 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.526 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.526 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-09-10T10:04:43.526 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "0", 2024-09-10T10:04:43.526 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "6", 2024-09-10T10:04:43.526 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.527 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-10T10:04:43.527 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-10T10:04:43.527 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.527 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.527 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.527 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:43.527 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:43.527 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.527 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.527 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.527 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.527 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.527 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.527 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.528 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.528 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.528 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.528 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.528 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.528 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.528 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.528 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.528 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.528 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.528 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.528 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.528 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:43.528 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:43.529 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.529 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.529 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.529 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.529 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.529 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.529 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.529 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.529 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.529 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.529 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.529 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.529 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:43.529 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:43.530 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.530 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.530 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.530 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:43.530 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.530 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.530 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.530 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.530 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.530 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.530 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.530 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.530 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.530 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.531 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.531 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.531 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.531 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.531 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.531 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:43.531 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:43.531 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.531 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.531 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.531 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.531 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.531 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.532 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.532 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.532 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.532 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.532 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.532 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.532 INFO:teuthology.orchestra.run.smithi077.stdout: "min_score": { 2024-09-10T10:04:43.532 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "min_score", 2024-09-10T10:04:43.532 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "float", 2024-09-10T10:04:43.532 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.532 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.532 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-09-10T10:04:43.532 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.532 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.532 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.533 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-10T10:04:43.533 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.533 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.533 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.533 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.533 INFO:teuthology.orchestra.run.smithi077.stdout: "mode": { 2024-09-10T10:04:43.533 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "mode", 2024-09-10T10:04:43.533 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.533 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.533 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.533 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "upmap", 2024-09-10T10:04:43.533 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.533 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.533 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.533 INFO:teuthology.orchestra.run.smithi077.stdout: "crush-compat", 2024-09-10T10:04:43.534 INFO:teuthology.orchestra.run.smithi077.stdout: "none", 2024-09-10T10:04:43.534 INFO:teuthology.orchestra.run.smithi077.stdout: "read", 2024-09-10T10:04:43.534 INFO:teuthology.orchestra.run.smithi077.stdout: "upmap", 2024-09-10T10:04:43.534 INFO:teuthology.orchestra.run.smithi077.stdout: "upmap-read" 2024-09-10T10:04:43.534 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.534 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Balancer mode", 2024-09-10T10:04:43.534 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.534 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.534 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.534 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.534 INFO:teuthology.orchestra.run.smithi077.stdout: "pool_ids": { 2024-09-10T10:04:43.534 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "pool_ids", 2024-09-10T10:04:43.534 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.534 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.534 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.535 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.535 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.535 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.535 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.535 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-10T10:04:43.535 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.535 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.535 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.535 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.535 INFO:teuthology.orchestra.run.smithi077.stdout: "sleep_interval": { 2024-09-10T10:04:43.535 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sleep_interval", 2024-09-10T10:04:43.535 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-10T10:04:43.535 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.535 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.535 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "60", 2024-09-10T10:04:43.536 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.536 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.536 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.536 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-10T10:04:43.536 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.536 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.536 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.536 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.536 INFO:teuthology.orchestra.run.smithi077.stdout: "upmap_max_deviation": { 2024-09-10T10:04:43.536 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "upmap_max_deviation", 2024-09-10T10:04:43.536 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:43.536 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.536 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.536 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "5", 2024-09-10T10:04:43.537 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "1", 2024-09-10T10:04:43.537 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.537 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.537 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-10T10:04:43.537 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-10T10:04:43.537 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.537 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.537 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.537 INFO:teuthology.orchestra.run.smithi077.stdout: "upmap_max_optimizations": { 2024-09-10T10:04:43.537 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "upmap_max_optimizations", 2024-09-10T10:04:43.537 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "uint", 2024-09-10T10:04:43.537 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.537 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.537 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10", 2024-09-10T10:04:43.538 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.538 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.538 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.538 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-10T10:04:43.538 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.538 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.538 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.538 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.538 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.538 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.538 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:43.538 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "cephadm", 2024-09-10T10:04:43.538 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:43.538 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:43.538 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:43.538 INFO:teuthology.orchestra.run.smithi077.stdout: "agent_down_multiplier": { 2024-09-10T10:04:43.539 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "agent_down_multiplier", 2024-09-10T10:04:43.539 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "float", 2024-09-10T10:04:43.539 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.539 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.539 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "3.0", 2024-09-10T10:04:43.539 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.539 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.539 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.539 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-10T10:04:43.539 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.539 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.539 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.539 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.539 INFO:teuthology.orchestra.run.smithi077.stdout: "agent_refresh_rate": { 2024-09-10T10:04:43.539 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "agent_refresh_rate", 2024-09-10T10:04:43.540 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-10T10:04:43.540 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.540 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.540 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "20", 2024-09-10T10:04:43.540 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.540 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.540 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.540 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-10T10:04:43.540 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.540 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.540 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.540 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.540 INFO:teuthology.orchestra.run.smithi077.stdout: "agent_starting_port": { 2024-09-10T10:04:43.540 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "agent_starting_port", 2024-09-10T10:04:43.541 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:43.541 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.541 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.541 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "4721", 2024-09-10T10:04:43.541 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.541 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.541 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.541 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-10T10:04:43.541 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.541 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.541 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.541 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.541 INFO:teuthology.orchestra.run.smithi077.stdout: "allow_ptrace": { 2024-09-10T10:04:43.541 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "allow_ptrace", 2024-09-10T10:04:43.541 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.542 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.542 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.542 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.542 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.542 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.542 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.542 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-10T10:04:43.542 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-10T10:04:43.542 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.542 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.542 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.542 INFO:teuthology.orchestra.run.smithi077.stdout: "autotune_interval": { 2024-09-10T10:04:43.542 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "autotune_interval", 2024-09-10T10:04:43.542 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-10T10:04:43.542 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.543 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.543 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "600", 2024-09-10T10:04:43.543 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.543 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.543 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.543 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-10T10:04:43.543 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.543 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.543 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.543 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.543 INFO:teuthology.orchestra.run.smithi077.stdout: "autotune_memory_target_ratio": { 2024-09-10T10:04:43.543 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "autotune_memory_target_ratio", 2024-09-10T10:04:43.543 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "float", 2024-09-10T10:04:43.543 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.543 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.544 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0.7", 2024-09-10T10:04:43.544 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.544 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.544 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.544 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-10T10:04:43.544 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.544 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.544 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.544 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.544 INFO:teuthology.orchestra.run.smithi077.stdout: "cephadm_log_destination": { 2024-09-10T10:04:43.544 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "cephadm_log_destination", 2024-09-10T10:04:43.544 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.544 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.545 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.545 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.545 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.545 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.545 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.545 INFO:teuthology.orchestra.run.smithi077.stdout: "file", 2024-09-10T10:04:43.545 INFO:teuthology.orchestra.run.smithi077.stdout: "file,syslog", 2024-09-10T10:04:43.545 INFO:teuthology.orchestra.run.smithi077.stdout: "syslog" 2024-09-10T10:04:43.545 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.545 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-10T10:04:43.545 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.545 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.545 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.545 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.545 INFO:teuthology.orchestra.run.smithi077.stdout: "cgroups_split": { 2024-09-10T10:04:43.545 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "cgroups_split", 2024-09-10T10:04:43.546 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.546 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.546 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.546 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:43.546 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.546 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.546 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.546 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-10T10:04:43.546 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.546 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.546 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.546 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.546 INFO:teuthology.orchestra.run.smithi077.stdout: "config_checks_enabled": { 2024-09-10T10:04:43.546 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "config_checks_enabled", 2024-09-10T10:04:43.546 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.547 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.547 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.547 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.547 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.547 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.547 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.547 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-10T10:04:43.547 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.547 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.547 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.547 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.547 INFO:teuthology.orchestra.run.smithi077.stdout: "config_dashboard": { 2024-09-10T10:04:43.547 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "config_dashboard", 2024-09-10T10:04:43.548 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.548 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.548 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.548 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:43.548 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.548 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.548 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.548 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-10T10:04:43.548 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.548 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.548 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.548 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.548 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_alertmanager": { 2024-09-10T10:04:43.548 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_alertmanager", 2024-09-10T10:04:43.548 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.549 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.549 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.549 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-10T10:04:43.549 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.549 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.549 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.549 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Prometheus container image", 2024-09-10T10:04:43.549 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.549 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.549 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.549 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.549 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_base": { 2024-09-10T10:04:43.549 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_base", 2024-09-10T10:04:43.549 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.549 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.550 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.550 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-10T10:04:43.550 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.550 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.550 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.550 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Container image name, without the tag", 2024-09-10T10:04:43.550 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.550 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.550 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.550 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.550 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_elasticsearch": { 2024-09-10T10:04:43.550 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_elasticsearch", 2024-09-10T10:04:43.550 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.550 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.551 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.551 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-10T10:04:43.551 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.551 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.551 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.551 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "elasticsearch container image", 2024-09-10T10:04:43.551 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.551 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.551 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.551 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.551 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_grafana": { 2024-09-10T10:04:43.551 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_grafana", 2024-09-10T10:04:43.551 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.551 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.552 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.552 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-10T10:04:43.552 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.552 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.552 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.552 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Prometheus container image", 2024-09-10T10:04:43.552 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.552 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.552 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.552 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.552 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_haproxy": { 2024-09-10T10:04:43.552 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_haproxy", 2024-09-10T10:04:43.552 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.552 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.553 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.553 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-10T10:04:43.553 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.553 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.553 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.553 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "HAproxy container image", 2024-09-10T10:04:43.553 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.553 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.553 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.553 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.553 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_jaeger_agent": { 2024-09-10T10:04:43.553 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_jaeger_agent", 2024-09-10T10:04:43.553 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.553 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.554 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.554 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-10T10:04:43.554 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.554 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.554 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.554 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Jaeger agent container image", 2024-09-10T10:04:43.554 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.554 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.554 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.554 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.554 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_jaeger_collector": { 2024-09-10T10:04:43.554 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_jaeger_collector", 2024-09-10T10:04:43.554 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.554 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.554 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.555 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-10T10:04:43.555 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.555 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.555 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.555 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Jaeger collector container image", 2024-09-10T10:04:43.555 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.555 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.555 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.555 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.555 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_jaeger_query": { 2024-09-10T10:04:43.555 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_jaeger_query", 2024-09-10T10:04:43.555 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.555 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.555 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.556 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-10T10:04:43.556 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.556 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.556 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.556 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Jaeger query container image", 2024-09-10T10:04:43.556 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.556 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.556 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.556 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.556 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_keepalived": { 2024-09-10T10:04:43.556 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_keepalived", 2024-09-10T10:04:43.556 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.556 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.556 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.557 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-10T10:04:43.557 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.557 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.557 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.557 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Keepalived container image", 2024-09-10T10:04:43.557 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.557 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.557 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.557 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.557 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_loki": { 2024-09-10T10:04:43.557 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_loki", 2024-09-10T10:04:43.557 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.557 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.558 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.558 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-10T10:04:43.558 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.558 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.558 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.558 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Loki container image", 2024-09-10T10:04:43.558 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.558 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.558 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.558 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.558 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_node_exporter": { 2024-09-10T10:04:43.558 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_node_exporter", 2024-09-10T10:04:43.558 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.558 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.558 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.558 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-10T10:04:43.559 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.559 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.559 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.559 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Prometheus container image", 2024-09-10T10:04:43.559 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.559 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.559 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.559 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.559 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_nvmeof": { 2024-09-10T10:04:43.559 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_nvmeof", 2024-09-10T10:04:43.559 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.559 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.559 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.559 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-10T10:04:43.559 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.560 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.560 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.560 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Nvme-of container image", 2024-09-10T10:04:43.560 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.560 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.560 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.560 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.560 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_prometheus": { 2024-09-10T10:04:43.560 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_prometheus", 2024-09-10T10:04:43.560 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.560 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.560 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.560 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-10T10:04:43.560 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.561 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.561 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.561 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Prometheus container image", 2024-09-10T10:04:43.561 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.561 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.561 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.561 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.561 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_promtail": { 2024-09-10T10:04:43.561 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_promtail", 2024-09-10T10:04:43.561 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.561 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.561 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.561 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-10T10:04:43.561 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.561 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.562 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.562 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Promtail container image", 2024-09-10T10:04:43.562 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.562 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.562 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.562 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.562 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_samba": { 2024-09-10T10:04:43.562 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_samba", 2024-09-10T10:04:43.562 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.562 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.562 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.562 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-10T10:04:43.562 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.562 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.563 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.563 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Samba/SMB container image", 2024-09-10T10:04:43.563 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.563 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.563 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.563 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.563 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_snmp_gateway": { 2024-09-10T10:04:43.563 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_snmp_gateway", 2024-09-10T10:04:43.563 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.563 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.563 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.564 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-10T10:04:43.564 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.564 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.564 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.564 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "SNMP Gateway container image", 2024-09-10T10:04:43.564 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.564 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.564 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.564 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.564 INFO:teuthology.orchestra.run.smithi077.stdout: "container_init": { 2024-09-10T10:04:43.564 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_init", 2024-09-10T10:04:43.564 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.564 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.564 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.565 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:43.565 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.565 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.565 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.565 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Run podman/docker with `--init`", 2024-09-10T10:04:43.565 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.565 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.565 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.565 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.565 INFO:teuthology.orchestra.run.smithi077.stdout: "daemon_cache_timeout": { 2024-09-10T10:04:43.565 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "daemon_cache_timeout", 2024-09-10T10:04:43.565 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-10T10:04:43.565 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.565 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.565 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "600", 2024-09-10T10:04:43.566 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.566 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.566 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.566 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-10T10:04:43.566 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.566 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.566 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.566 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.566 INFO:teuthology.orchestra.run.smithi077.stdout: "default_cephadm_command_timeout": { 2024-09-10T10:04:43.566 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "default_cephadm_command_timeout", 2024-09-10T10:04:43.566 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:43.566 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.566 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.566 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "900", 2024-09-10T10:04:43.566 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.567 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.567 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.567 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-10T10:04:43.567 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.567 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.567 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.567 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.567 INFO:teuthology.orchestra.run.smithi077.stdout: "default_registry": { 2024-09-10T10:04:43.567 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "default_registry", 2024-09-10T10:04:43.567 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.567 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.567 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.567 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "docker.io", 2024-09-10T10:04:43.568 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.568 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.568 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.568 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-10T10:04:43.568 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.568 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.568 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.568 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.568 INFO:teuthology.orchestra.run.smithi077.stdout: "device_cache_timeout": { 2024-09-10T10:04:43.568 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "device_cache_timeout", 2024-09-10T10:04:43.568 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-10T10:04:43.568 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.568 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.569 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "1800", 2024-09-10T10:04:43.569 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.569 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.569 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.569 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "seconds to cache device inventory", 2024-09-10T10:04:43.569 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.569 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.569 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.569 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.569 INFO:teuthology.orchestra.run.smithi077.stdout: "device_enhanced_scan": { 2024-09-10T10:04:43.569 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "device_enhanced_scan", 2024-09-10T10:04:43.569 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.569 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.569 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.570 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.570 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.570 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.570 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.570 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-10T10:04:43.570 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.570 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.570 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.570 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.570 INFO:teuthology.orchestra.run.smithi077.stdout: "facts_cache_timeout": { 2024-09-10T10:04:43.570 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "facts_cache_timeout", 2024-09-10T10:04:43.570 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-10T10:04:43.570 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.570 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.571 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "60", 2024-09-10T10:04:43.571 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.571 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.571 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.571 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "seconds to cache host facts data", 2024-09-10T10:04:43.571 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.571 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.571 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.571 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.571 INFO:teuthology.orchestra.run.smithi077.stdout: "grafana_dashboards_path": { 2024-09-10T10:04:43.571 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "grafana_dashboards_path", 2024-09-10T10:04:43.571 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.571 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.571 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.572 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-10T10:04:43.572 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.572 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.572 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.572 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-10T10:04:43.572 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.572 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.572 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.572 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.572 INFO:teuthology.orchestra.run.smithi077.stdout: "host_check_interval": { 2024-09-10T10:04:43.572 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "host_check_interval", 2024-09-10T10:04:43.572 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-10T10:04:43.572 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.573 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.573 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "600", 2024-09-10T10:04:43.573 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.573 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.573 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.573 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "how frequently to perform a host check", 2024-09-10T10:04:43.573 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.573 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.573 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.573 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.573 INFO:teuthology.orchestra.run.smithi077.stdout: "hw_monitoring": { 2024-09-10T10:04:43.573 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "hw_monitoring", 2024-09-10T10:04:43.573 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.573 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.574 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.574 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.574 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.574 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.574 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.574 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-10T10:04:43.574 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.574 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.574 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.574 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.574 INFO:teuthology.orchestra.run.smithi077.stdout: "inventory_list_all": { 2024-09-10T10:04:43.574 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "inventory_list_all", 2024-09-10T10:04:43.574 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.574 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.575 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.575 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.575 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.575 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.575 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.575 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-10T10:04:43.575 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.575 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.575 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.575 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.575 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:43.575 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:43.575 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.575 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.576 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.576 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.576 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.576 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.576 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.576 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.576 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.576 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.576 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.576 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.576 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.576 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.576 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.577 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.577 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.577 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.577 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.577 INFO:teuthology.orchestra.run.smithi077.stdout: "log_refresh_metadata": { 2024-09-10T10:04:43.577 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_refresh_metadata", 2024-09-10T10:04:43.577 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.577 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.577 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.577 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.577 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.577 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.577 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.578 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-10T10:04:43.578 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.578 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.578 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.578 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.578 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:43.578 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:43.578 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.578 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.578 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.578 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:43.578 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.579 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.579 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.579 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-10T10:04:43.579 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.579 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.579 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.579 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.579 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:43.579 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:43.579 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.579 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.579 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.579 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:43.580 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.580 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.580 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.580 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.580 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.580 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.580 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.580 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.580 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.580 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.580 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.580 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.580 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.580 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.581 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.581 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:43.581 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:43.581 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.581 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.581 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.581 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.581 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.581 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.581 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.581 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.581 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.581 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.582 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.582 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.582 INFO:teuthology.orchestra.run.smithi077.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-10T10:04:43.582 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-10T10:04:43.582 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.582 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.582 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.582 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.582 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.582 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.582 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.582 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-10T10:04:43.582 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.583 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.583 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.583 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.583 INFO:teuthology.orchestra.run.smithi077.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-10T10:04:43.583 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-10T10:04:43.583 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.583 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.583 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.583 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "*", 2024-09-10T10:04:43.583 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.583 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.583 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.583 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-10T10:04:43.584 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.584 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.584 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.584 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.584 INFO:teuthology.orchestra.run.smithi077.stdout: "max_count_per_host": { 2024-09-10T10:04:43.584 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "max_count_per_host", 2024-09-10T10:04:43.584 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:43.584 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.584 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.584 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10", 2024-09-10T10:04:43.584 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.585 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.585 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.585 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "max number of daemons per service per host", 2024-09-10T10:04:43.585 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.585 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.585 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.585 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.585 INFO:teuthology.orchestra.run.smithi077.stdout: "max_osd_draining_count": { 2024-09-10T10:04:43.585 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "max_osd_draining_count", 2024-09-10T10:04:43.585 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:43.585 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.585 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.586 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10", 2024-09-10T10:04:43.586 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.586 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.586 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.586 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-10T10:04:43.586 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.586 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.586 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.586 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.586 INFO:teuthology.orchestra.run.smithi077.stdout: "migration_current": { 2024-09-10T10:04:43.586 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "migration_current", 2024-09-10T10:04:43.586 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:43.586 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.586 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.586 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.587 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.587 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.587 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.587 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "internal - do not modify", 2024-09-10T10:04:43.587 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.587 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.587 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.587 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.587 INFO:teuthology.orchestra.run.smithi077.stdout: "mode": { 2024-09-10T10:04:43.587 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "mode", 2024-09-10T10:04:43.587 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.587 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.587 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.588 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "root", 2024-09-10T10:04:43.588 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.588 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.588 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.588 INFO:teuthology.orchestra.run.smithi077.stdout: "cephadm-package", 2024-09-10T10:04:43.588 INFO:teuthology.orchestra.run.smithi077.stdout: "root" 2024-09-10T10:04:43.588 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.588 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "mode for remote execution of cephadm", 2024-09-10T10:04:43.588 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.588 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.588 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.588 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.589 INFO:teuthology.orchestra.run.smithi077.stdout: "oob_default_addr": { 2024-09-10T10:04:43.589 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "oob_default_addr", 2024-09-10T10:04:43.589 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.589 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.589 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.589 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "169.254.1.1", 2024-09-10T10:04:43.589 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.589 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.589 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.589 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-10T10:04:43.589 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.590 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.590 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.590 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.590 INFO:teuthology.orchestra.run.smithi077.stdout: "prometheus_alerts_path": { 2024-09-10T10:04:43.590 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "prometheus_alerts_path", 2024-09-10T10:04:43.590 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.590 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.590 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.590 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-10T10:04:43.590 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.590 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.590 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.591 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-10T10:04:43.591 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.591 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.591 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.591 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.591 INFO:teuthology.orchestra.run.smithi077.stdout: "registry_insecure": { 2024-09-10T10:04:43.591 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "registry_insecure", 2024-09-10T10:04:43.591 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.591 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.591 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.591 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.592 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.592 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.592 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.592 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-10T10:04:43.592 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.592 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.592 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.592 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.592 INFO:teuthology.orchestra.run.smithi077.stdout: "registry_password": { 2024-09-10T10:04:43.592 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "registry_password", 2024-09-10T10:04:43.593 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.593 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.593 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.593 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.593 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.593 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.593 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.593 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-10T10:04:43.593 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.593 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.593 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.593 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.594 INFO:teuthology.orchestra.run.smithi077.stdout: "registry_url": { 2024-09-10T10:04:43.594 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "registry_url", 2024-09-10T10:04:43.594 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.594 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.594 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.594 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.594 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.594 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.594 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.594 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-10T10:04:43.594 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.594 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.594 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.595 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.595 INFO:teuthology.orchestra.run.smithi077.stdout: "registry_username": { 2024-09-10T10:04:43.595 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "registry_username", 2024-09-10T10:04:43.595 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.595 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.595 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.595 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.595 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.595 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.595 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.595 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-10T10:04:43.596 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.596 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.596 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.596 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.596 INFO:teuthology.orchestra.run.smithi077.stdout: "secure_monitoring_stack": { 2024-09-10T10:04:43.596 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "secure_monitoring_stack", 2024-09-10T10:04:43.596 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.596 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.596 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.596 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.596 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.596 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.597 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.597 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-10T10:04:43.597 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.597 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.597 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.597 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.597 INFO:teuthology.orchestra.run.smithi077.stdout: "service_discovery_port": { 2024-09-10T10:04:43.597 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "service_discovery_port", 2024-09-10T10:04:43.597 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:43.597 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.597 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.598 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "8765", 2024-09-10T10:04:43.598 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.598 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.598 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.598 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "cephadm service discovery port", 2024-09-10T10:04:43.598 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.598 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.598 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.598 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.598 INFO:teuthology.orchestra.run.smithi077.stdout: "ssh_config_file": { 2024-09-10T10:04:43.598 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ssh_config_file", 2024-09-10T10:04:43.598 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.598 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.598 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.599 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.599 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.599 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.599 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.599 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-10T10:04:43.599 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.599 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.599 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.599 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.599 INFO:teuthology.orchestra.run.smithi077.stdout: "use_agent": { 2024-09-10T10:04:43.599 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "use_agent", 2024-09-10T10:04:43.599 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.599 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.599 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.599 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.600 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.600 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.600 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.600 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-10T10:04:43.600 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.600 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.600 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.600 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.600 INFO:teuthology.orchestra.run.smithi077.stdout: "use_repo_digest": { 2024-09-10T10:04:43.600 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "use_repo_digest", 2024-09-10T10:04:43.600 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.600 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.600 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.601 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:43.601 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.601 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.601 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.601 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-10T10:04:43.601 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.601 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.601 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.601 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.601 INFO:teuthology.orchestra.run.smithi077.stdout: "warn_on_failed_host_check": { 2024-09-10T10:04:43.601 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "warn_on_failed_host_check", 2024-09-10T10:04:43.601 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.601 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.601 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.602 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:43.602 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.602 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.602 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.602 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "raise a health warning if the host check fails", 2024-09-10T10:04:43.602 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.602 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.602 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.602 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.602 INFO:teuthology.orchestra.run.smithi077.stdout: "warn_on_stray_daemons": { 2024-09-10T10:04:43.602 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "warn_on_stray_daemons", 2024-09-10T10:04:43.602 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.602 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.602 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.602 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:43.603 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.603 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.603 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.603 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-10T10:04:43.603 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.603 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.603 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.603 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.603 INFO:teuthology.orchestra.run.smithi077.stdout: "warn_on_stray_hosts": { 2024-09-10T10:04:43.603 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "warn_on_stray_hosts", 2024-09-10T10:04:43.603 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.603 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.603 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.603 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:43.604 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.604 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.604 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.604 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-10T10:04:43.604 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.604 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.604 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.604 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.604 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.604 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.604 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:43.605 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "crash", 2024-09-10T10:04:43.605 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:43.605 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:43.605 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:43.605 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:43.605 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:43.605 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.605 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.605 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.605 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.605 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.606 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.606 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.606 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.606 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.606 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.606 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.606 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.606 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.606 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.606 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.606 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.606 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.606 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.607 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.607 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:43.607 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:43.607 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.607 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.607 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.607 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.607 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.607 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.607 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.608 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.608 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.608 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.608 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.608 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.608 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:43.608 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:43.608 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.608 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.608 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.608 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:43.608 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.609 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.609 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.609 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.609 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.609 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.609 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.609 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.609 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.609 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.609 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.609 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.609 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.610 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.610 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.610 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:43.610 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:43.610 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.610 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.610 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.610 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.610 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.610 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.611 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.611 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.611 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.611 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.611 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.611 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.611 INFO:teuthology.orchestra.run.smithi077.stdout: "retain_interval": { 2024-09-10T10:04:43.611 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "retain_interval", 2024-09-10T10:04:43.611 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-10T10:04:43.611 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.611 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.611 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "31536000", 2024-09-10T10:04:43.611 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.611 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.612 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.612 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-10T10:04:43.612 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.612 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.612 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.612 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.612 INFO:teuthology.orchestra.run.smithi077.stdout: "warn_recent_interval": { 2024-09-10T10:04:43.612 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "warn_recent_interval", 2024-09-10T10:04:43.612 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-10T10:04:43.612 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.612 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.612 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "1209600", 2024-09-10T10:04:43.612 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.612 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.613 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.613 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-10T10:04:43.613 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.613 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.613 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.613 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.613 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.613 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.613 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:43.613 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "dashboard", 2024-09-10T10:04:43.613 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:43.613 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:43.613 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:43.614 INFO:teuthology.orchestra.run.smithi077.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-10T10:04:43.614 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-10T10:04:43.614 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:43.614 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.614 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.614 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10", 2024-09-10T10:04:43.614 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.614 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.614 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.614 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.614 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.614 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.614 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.615 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.615 INFO:teuthology.orchestra.run.smithi077.stdout: "ALERTMANAGER_API_HOST": { 2024-09-10T10:04:43.615 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-10T10:04:43.615 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.615 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.615 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.615 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.615 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.615 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.615 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.615 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.615 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.615 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.616 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.616 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.616 INFO:teuthology.orchestra.run.smithi077.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-10T10:04:43.616 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-10T10:04:43.616 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.616 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.616 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.616 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:43.616 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.616 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.616 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.616 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.617 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.617 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.617 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.617 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.617 INFO:teuthology.orchestra.run.smithi077.stdout: "AUDIT_API_ENABLED": { 2024-09-10T10:04:43.617 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "AUDIT_API_ENABLED", 2024-09-10T10:04:43.617 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.617 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.617 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.617 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.617 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.617 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.617 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.618 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.618 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.618 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.618 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.618 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.618 INFO:teuthology.orchestra.run.smithi077.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-10T10:04:43.618 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-10T10:04:43.618 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.618 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.618 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.618 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:43.618 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.618 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.618 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.618 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.619 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.619 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.619 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.619 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.619 INFO:teuthology.orchestra.run.smithi077.stdout: "ENABLE_BROWSABLE_API": { 2024-09-10T10:04:43.619 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-10T10:04:43.619 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.619 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.619 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.619 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:43.619 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.619 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.619 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.619 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.620 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.620 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.620 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.620 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.620 INFO:teuthology.orchestra.run.smithi077.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-10T10:04:43.620 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-10T10:04:43.620 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.620 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.620 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.620 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:43.620 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.620 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.620 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.620 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.621 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.621 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.621 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.621 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.621 INFO:teuthology.orchestra.run.smithi077.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-10T10:04:43.621 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-10T10:04:43.621 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.621 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.621 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.621 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:43.621 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.621 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.621 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.621 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.622 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.622 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.622 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.622 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.622 INFO:teuthology.orchestra.run.smithi077.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-10T10:04:43.622 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-10T10:04:43.622 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.622 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.622 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.622 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:43.622 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.622 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.622 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.622 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.622 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.623 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.623 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.623 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.623 INFO:teuthology.orchestra.run.smithi077.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-10T10:04:43.623 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-10T10:04:43.623 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.623 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.623 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.623 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:43.623 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.623 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.623 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.623 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.623 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.624 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.624 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.624 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.624 INFO:teuthology.orchestra.run.smithi077.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-10T10:04:43.624 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-10T10:04:43.624 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.624 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.624 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.624 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:43.624 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.624 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.624 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.625 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.625 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.625 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.625 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.625 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.625 INFO:teuthology.orchestra.run.smithi077.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-10T10:04:43.625 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-10T10:04:43.625 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.625 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.625 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.625 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:43.625 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.625 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.626 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.626 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.626 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.626 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.626 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.626 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.626 INFO:teuthology.orchestra.run.smithi077.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-10T10:04:43.626 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-10T10:04:43.626 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.626 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.626 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.626 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:43.626 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.627 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.627 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.627 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.627 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.627 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.627 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.627 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.627 INFO:teuthology.orchestra.run.smithi077.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-10T10:04:43.627 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-10T10:04:43.627 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.627 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.627 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.627 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.628 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.628 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.628 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.628 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.628 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.628 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.628 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.628 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.628 INFO:teuthology.orchestra.run.smithi077.stdout: "GRAFANA_API_PASSWORD": { 2024-09-10T10:04:43.628 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-10T10:04:43.628 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.628 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.628 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.629 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "admin", 2024-09-10T10:04:43.629 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.629 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.629 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.629 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.629 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.629 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.629 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.629 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.629 INFO:teuthology.orchestra.run.smithi077.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-10T10:04:43.629 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-10T10:04:43.629 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.629 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.629 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.630 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:43.630 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.630 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.630 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.630 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.630 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.630 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.630 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.630 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.630 INFO:teuthology.orchestra.run.smithi077.stdout: "GRAFANA_API_URL": { 2024-09-10T10:04:43.630 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "GRAFANA_API_URL", 2024-09-10T10:04:43.630 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.630 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.630 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.631 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.631 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.631 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.631 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.631 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.631 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.631 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.631 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.631 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.631 INFO:teuthology.orchestra.run.smithi077.stdout: "GRAFANA_API_USERNAME": { 2024-09-10T10:04:43.631 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-10T10:04:43.631 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.631 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.631 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.632 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "admin", 2024-09-10T10:04:43.632 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.632 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.632 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.632 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.632 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.632 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.632 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.632 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.632 INFO:teuthology.orchestra.run.smithi077.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-10T10:04:43.632 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-10T10:04:43.632 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.632 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.633 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.633 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.633 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.633 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.633 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.633 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.633 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.633 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.633 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.633 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.633 INFO:teuthology.orchestra.run.smithi077.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-10T10:04:43.633 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-10T10:04:43.633 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.634 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.634 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.634 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.634 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.634 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.634 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.634 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.634 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.634 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.634 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.634 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.634 INFO:teuthology.orchestra.run.smithi077.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-10T10:04:43.634 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-10T10:04:43.635 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.635 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.635 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.635 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:43.635 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.635 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.635 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.635 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.635 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.635 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.635 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.636 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.636 INFO:teuthology.orchestra.run.smithi077.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-10T10:04:43.636 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-10T10:04:43.636 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.636 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.636 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.636 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.636 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.636 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.636 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.636 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.636 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.636 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.637 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.637 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.637 INFO:teuthology.orchestra.run.smithi077.stdout: "PROMETHEUS_API_HOST": { 2024-09-10T10:04:43.637 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-10T10:04:43.637 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.637 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.637 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.637 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.637 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.637 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.637 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.637 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.637 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.638 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.638 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.638 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.638 INFO:teuthology.orchestra.run.smithi077.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-10T10:04:43.638 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-10T10:04:43.638 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.638 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.638 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.638 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:43.638 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.638 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.638 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.638 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.638 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.638 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.639 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.639 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.639 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-10T10:04:43.639 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-10T10:04:43.639 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.639 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.639 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.639 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.639 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.639 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.639 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.639 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.639 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.639 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.640 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.640 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.640 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-10T10:04:43.640 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-10T10:04:43.640 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.640 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.640 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.640 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.640 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.640 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.640 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.640 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.640 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.640 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.641 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.641 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.641 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-10T10:04:43.641 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-10T10:04:43.641 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.641 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.641 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.641 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:43.641 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.641 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.641 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.641 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.641 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.641 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.642 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.642 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.642 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-10T10:04:43.642 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-10T10:04:43.642 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.642 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.642 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.642 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:43.642 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.642 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.642 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.642 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.642 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.642 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.642 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.643 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.643 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-10T10:04:43.643 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-10T10:04:43.643 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.643 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.643 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.643 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.643 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.643 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.643 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.643 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.643 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.643 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.643 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.643 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.643 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-10T10:04:43.644 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-10T10:04:43.644 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.644 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.644 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.644 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.644 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.644 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.644 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.644 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.644 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.644 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.644 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.644 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.645 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-10T10:04:43.645 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-10T10:04:43.645 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.645 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.645 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.645 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.645 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.645 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.645 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.645 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.645 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.645 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.645 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.645 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.646 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_ENABLED": { 2024-09-10T10:04:43.646 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-10T10:04:43.646 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.646 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.646 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.646 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:43.646 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.646 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.646 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.646 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.646 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.646 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.647 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.647 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.647 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-10T10:04:43.647 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-10T10:04:43.647 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.647 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.647 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.647 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-10T10:04:43.647 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.647 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.647 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.647 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.647 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.647 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.648 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.648 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.648 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-10T10:04:43.648 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-10T10:04:43.648 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:43.648 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.648 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.648 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10", 2024-09-10T10:04:43.648 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.648 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.648 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.648 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.648 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.648 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.648 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.649 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.649 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-10T10:04:43.649 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-10T10:04:43.649 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:43.649 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.649 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.649 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "8", 2024-09-10T10:04:43.649 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.649 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.649 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.649 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.649 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.649 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.650 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.650 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.650 INFO:teuthology.orchestra.run.smithi077.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-10T10:04:43.650 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-10T10:04:43.650 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:43.650 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.650 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.650 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "45", 2024-09-10T10:04:43.650 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.650 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.650 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.650 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.650 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.650 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.651 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.651 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.651 INFO:teuthology.orchestra.run.smithi077.stdout: "RGW_API_ACCESS_KEY": { 2024-09-10T10:04:43.651 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-10T10:04:43.651 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.651 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.651 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.651 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.651 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.651 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.651 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.651 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.651 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.652 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.652 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.652 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.652 INFO:teuthology.orchestra.run.smithi077.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-10T10:04:43.652 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-10T10:04:43.652 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.652 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.652 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.652 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "admin", 2024-09-10T10:04:43.652 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.652 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.652 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.652 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.653 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.653 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.653 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.653 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.653 INFO:teuthology.orchestra.run.smithi077.stdout: "RGW_API_SECRET_KEY": { 2024-09-10T10:04:43.653 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-10T10:04:43.653 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.653 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.653 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.653 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.653 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.653 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.653 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.653 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.654 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.654 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.654 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.654 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.654 INFO:teuthology.orchestra.run.smithi077.stdout: "RGW_API_SSL_VERIFY": { 2024-09-10T10:04:43.654 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-10T10:04:43.654 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.654 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.654 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.654 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:43.654 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.654 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.654 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.654 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.655 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.655 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.655 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.655 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.655 INFO:teuthology.orchestra.run.smithi077.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-10T10:04:43.655 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-10T10:04:43.655 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:43.655 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.655 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.655 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-09-10T10:04:43.655 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.655 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.655 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.655 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.656 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.656 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.656 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.656 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.656 INFO:teuthology.orchestra.run.smithi077.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-10T10:04:43.656 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-10T10:04:43.656 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:43.656 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.656 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.656 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10", 2024-09-10T10:04:43.656 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.656 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.657 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.657 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.657 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.657 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.657 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.657 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.657 INFO:teuthology.orchestra.run.smithi077.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-10T10:04:43.657 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-10T10:04:43.657 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:43.657 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.657 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.657 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "5", 2024-09-10T10:04:43.657 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.658 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.658 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.658 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.658 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.658 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.658 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.658 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.658 INFO:teuthology.orchestra.run.smithi077.stdout: "cross_origin_url": { 2024-09-10T10:04:43.658 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "cross_origin_url", 2024-09-10T10:04:43.658 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.658 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.658 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.658 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.658 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.659 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.659 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.659 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.659 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.659 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.659 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.659 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.659 INFO:teuthology.orchestra.run.smithi077.stdout: "crt_file": { 2024-09-10T10:04:43.659 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "crt_file", 2024-09-10T10:04:43.659 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.659 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.659 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.659 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.659 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.659 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.660 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.660 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.660 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.660 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.660 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.660 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.660 INFO:teuthology.orchestra.run.smithi077.stdout: "debug": { 2024-09-10T10:04:43.660 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "debug", 2024-09-10T10:04:43.660 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.660 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.660 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.660 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.660 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.661 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.661 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.661 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Enable/disable debug options", 2024-09-10T10:04:43.661 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.661 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.661 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.661 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.661 INFO:teuthology.orchestra.run.smithi077.stdout: "jwt_token_ttl": { 2024-09-10T10:04:43.661 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "jwt_token_ttl", 2024-09-10T10:04:43.661 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:43.661 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.661 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.661 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "28800", 2024-09-10T10:04:43.661 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.662 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.662 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.662 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.662 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.662 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.662 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.662 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.662 INFO:teuthology.orchestra.run.smithi077.stdout: "key_file": { 2024-09-10T10:04:43.662 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "key_file", 2024-09-10T10:04:43.662 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.662 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.662 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.662 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.662 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.663 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.663 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.663 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.663 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.663 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.663 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.663 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.663 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:43.663 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:43.663 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.663 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.663 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.663 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.664 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.664 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.664 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.664 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.664 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.664 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.664 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.664 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.664 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.664 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.664 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.664 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.664 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.664 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.664 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.665 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:43.665 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:43.665 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.665 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.665 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.665 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.665 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.665 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.665 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.665 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.665 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.665 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.665 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.665 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.666 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:43.666 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:43.666 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.666 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.666 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.666 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:43.666 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.666 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.666 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.666 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.666 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.666 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.666 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.666 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.667 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.667 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.667 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.667 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.667 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.667 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.667 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.667 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:43.667 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:43.667 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.667 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.667 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.667 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.668 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.668 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.668 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.668 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.668 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.668 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.668 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.668 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.668 INFO:teuthology.orchestra.run.smithi077.stdout: "motd": { 2024-09-10T10:04:43.668 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "motd", 2024-09-10T10:04:43.668 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.668 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.668 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.668 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.669 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.669 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.669 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.669 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "The message of the day", 2024-09-10T10:04:43.669 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.669 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.669 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.669 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.669 INFO:teuthology.orchestra.run.smithi077.stdout: "redirect_resolve_ip_addr": { 2024-09-10T10:04:43.669 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "redirect_resolve_ip_addr", 2024-09-10T10:04:43.669 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.669 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.669 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.669 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.670 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.670 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.670 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.670 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.670 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.670 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.670 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.670 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.670 INFO:teuthology.orchestra.run.smithi077.stdout: "server_addr": { 2024-09-10T10:04:43.670 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "server_addr", 2024-09-10T10:04:43.670 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.670 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.670 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.670 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "::", 2024-09-10T10:04:43.670 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.671 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.671 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.671 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.671 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.671 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.671 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.671 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.671 INFO:teuthology.orchestra.run.smithi077.stdout: "server_port": { 2024-09-10T10:04:43.671 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "server_port", 2024-09-10T10:04:43.671 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:43.671 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.671 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.671 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "8080", 2024-09-10T10:04:43.671 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.671 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.672 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.672 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.672 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.672 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.672 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.672 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.672 INFO:teuthology.orchestra.run.smithi077.stdout: "ssl": { 2024-09-10T10:04:43.672 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ssl", 2024-09-10T10:04:43.672 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.672 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.672 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.672 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:43.672 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.672 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.672 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.673 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.673 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.673 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.673 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.673 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.673 INFO:teuthology.orchestra.run.smithi077.stdout: "ssl_server_port": { 2024-09-10T10:04:43.673 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ssl_server_port", 2024-09-10T10:04:43.673 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:43.673 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.673 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.673 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "8443", 2024-09-10T10:04:43.673 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.673 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.673 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.673 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.674 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.674 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.674 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.674 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.674 INFO:teuthology.orchestra.run.smithi077.stdout: "standby_behaviour": { 2024-09-10T10:04:43.674 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "standby_behaviour", 2024-09-10T10:04:43.674 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.674 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.674 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.674 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "redirect", 2024-09-10T10:04:43.674 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.674 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.674 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.675 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.675 INFO:teuthology.orchestra.run.smithi077.stdout: "redirect" 2024-09-10T10:04:43.675 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.675 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.675 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.675 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.675 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.675 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.675 INFO:teuthology.orchestra.run.smithi077.stdout: "standby_error_status_code": { 2024-09-10T10:04:43.675 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "standby_error_status_code", 2024-09-10T10:04:43.675 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:43.675 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.675 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.675 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "500", 2024-09-10T10:04:43.675 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "400", 2024-09-10T10:04:43.676 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "599", 2024-09-10T10:04:43.676 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.676 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.676 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.676 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.676 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.676 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.676 INFO:teuthology.orchestra.run.smithi077.stdout: "url_prefix": { 2024-09-10T10:04:43.676 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "url_prefix", 2024-09-10T10:04:43.676 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.676 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.676 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.677 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.677 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.677 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.677 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.677 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.677 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.677 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.677 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.677 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.677 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.677 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.677 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:43.678 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "devicehealth", 2024-09-10T10:04:43.678 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:43.678 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:43.678 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:43.678 INFO:teuthology.orchestra.run.smithi077.stdout: "enable_monitoring": { 2024-09-10T10:04:43.678 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "enable_monitoring", 2024-09-10T10:04:43.678 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.678 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.678 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.678 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:43.678 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.678 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.678 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.678 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "monitor device health metrics", 2024-09-10T10:04:43.679 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.679 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.679 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.679 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.679 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:43.679 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:43.679 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.679 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.679 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.679 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.679 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.679 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.679 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.679 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.679 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.679 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.680 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.680 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.680 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.680 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.680 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.680 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.680 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.680 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.680 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.680 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:43.680 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:43.680 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.680 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.681 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.681 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.681 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.681 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.681 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.681 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.681 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.681 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.681 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.681 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.681 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:43.681 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:43.681 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.681 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.682 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.682 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:43.682 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.682 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.682 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.682 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.682 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.682 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.682 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.682 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.682 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.682 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.682 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.682 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.682 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.683 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.683 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.683 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:43.683 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:43.683 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.683 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.683 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.683 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.683 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.683 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.683 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.683 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.683 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.683 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.683 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.684 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.684 INFO:teuthology.orchestra.run.smithi077.stdout: "mark_out_threshold": { 2024-09-10T10:04:43.684 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "mark_out_threshold", 2024-09-10T10:04:43.684 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-10T10:04:43.684 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.684 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.684 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "2419200", 2024-09-10T10:04:43.684 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.684 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.684 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.684 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-10T10:04:43.684 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.684 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.684 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.685 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.685 INFO:teuthology.orchestra.run.smithi077.stdout: "pool_name": { 2024-09-10T10:04:43.685 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "pool_name", 2024-09-10T10:04:43.685 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.685 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.685 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.685 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "device_health_metrics", 2024-09-10T10:04:43.685 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.685 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.685 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.685 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-10T10:04:43.685 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.685 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.685 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.685 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.686 INFO:teuthology.orchestra.run.smithi077.stdout: "retention_period": { 2024-09-10T10:04:43.686 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "retention_period", 2024-09-10T10:04:43.686 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-10T10:04:43.686 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.686 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.686 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "15552000", 2024-09-10T10:04:43.686 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.686 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.686 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.686 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "how long to retain device health metrics", 2024-09-10T10:04:43.686 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.686 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.686 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.686 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.686 INFO:teuthology.orchestra.run.smithi077.stdout: "scrape_frequency": { 2024-09-10T10:04:43.687 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "scrape_frequency", 2024-09-10T10:04:43.687 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-10T10:04:43.687 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.687 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.687 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "86400", 2024-09-10T10:04:43.687 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.687 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.687 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.687 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-10T10:04:43.687 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.687 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.687 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.687 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.687 INFO:teuthology.orchestra.run.smithi077.stdout: "self_heal": { 2024-09-10T10:04:43.688 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "self_heal", 2024-09-10T10:04:43.688 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.688 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.688 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.688 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:43.688 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.688 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.688 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.688 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-10T10:04:43.688 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.688 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.688 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.688 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.688 INFO:teuthology.orchestra.run.smithi077.stdout: "sleep_interval": { 2024-09-10T10:04:43.689 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sleep_interval", 2024-09-10T10:04:43.689 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-10T10:04:43.689 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.689 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.689 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "600", 2024-09-10T10:04:43.689 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.689 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.689 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.689 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "how frequently to wake up and check device health", 2024-09-10T10:04:43.689 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.689 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.689 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.689 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.689 INFO:teuthology.orchestra.run.smithi077.stdout: "warn_threshold": { 2024-09-10T10:04:43.689 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "warn_threshold", 2024-09-10T10:04:43.690 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-10T10:04:43.690 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.690 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.690 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "7257600", 2024-09-10T10:04:43.690 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.690 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.690 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.690 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-10T10:04:43.690 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.690 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.690 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.690 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.690 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.690 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.690 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:43.691 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "diskprediction_local", 2024-09-10T10:04:43.691 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:43.691 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:43.691 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:43.691 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:43.691 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:43.691 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.691 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.691 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.691 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.691 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.691 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.691 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.691 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.691 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.692 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.692 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.692 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.692 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.692 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.692 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.692 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.692 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.692 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.692 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.692 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:43.692 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:43.692 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.692 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.692 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.693 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.693 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.693 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.693 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.693 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.693 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.693 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.693 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.693 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.693 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:43.693 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:43.693 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.693 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.693 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.693 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:43.694 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.694 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.694 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.694 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.694 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.694 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.694 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.694 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.694 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.694 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.694 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.694 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.694 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.694 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.695 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.695 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:43.695 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:43.695 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.695 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.695 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.695 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.695 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.695 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.695 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.695 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.695 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.695 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.695 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.695 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.696 INFO:teuthology.orchestra.run.smithi077.stdout: "predict_interval": { 2024-09-10T10:04:43.696 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "predict_interval", 2024-09-10T10:04:43.696 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.696 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.696 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.696 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "86400", 2024-09-10T10:04:43.696 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.696 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.696 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.696 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.697 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.697 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.697 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.697 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.697 INFO:teuthology.orchestra.run.smithi077.stdout: "predictor_model": { 2024-09-10T10:04:43.697 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "predictor_model", 2024-09-10T10:04:43.697 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.697 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.697 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.697 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "prophetstor", 2024-09-10T10:04:43.697 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.698 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.698 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.698 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.698 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.698 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.698 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.698 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.698 INFO:teuthology.orchestra.run.smithi077.stdout: "sleep_interval": { 2024-09-10T10:04:43.698 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sleep_interval", 2024-09-10T10:04:43.698 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.698 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.698 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.698 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "600", 2024-09-10T10:04:43.699 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.699 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.699 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.699 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.699 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.699 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.699 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.699 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.699 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.699 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.699 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:43.699 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "influx", 2024-09-10T10:04:43.699 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": false, 2024-09-10T10:04:43.699 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "influxdb python module not found", 2024-09-10T10:04:43.700 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:43.700 INFO:teuthology.orchestra.run.smithi077.stdout: "batch_size": { 2024-09-10T10:04:43.700 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "batch_size", 2024-09-10T10:04:43.700 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:43.700 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.700 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.700 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "5000", 2024-09-10T10:04:43.700 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.700 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.700 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.700 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-10T10:04:43.700 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.700 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.701 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.701 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.701 INFO:teuthology.orchestra.run.smithi077.stdout: "database": { 2024-09-10T10:04:43.701 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "database", 2024-09-10T10:04:43.701 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.701 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.701 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.701 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "ceph", 2024-09-10T10:04:43.701 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.701 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.701 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.701 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-10T10:04:43.701 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.702 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.702 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.702 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.702 INFO:teuthology.orchestra.run.smithi077.stdout: "hostname": { 2024-09-10T10:04:43.702 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "hostname", 2024-09-10T10:04:43.702 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.702 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.702 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.702 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.702 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.702 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.702 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.702 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "InfluxDB server hostname", 2024-09-10T10:04:43.703 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.703 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.703 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.703 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.703 INFO:teuthology.orchestra.run.smithi077.stdout: "interval": { 2024-09-10T10:04:43.703 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "interval", 2024-09-10T10:04:43.703 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-10T10:04:43.703 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.703 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.703 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "30", 2024-09-10T10:04:43.703 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "5", 2024-09-10T10:04:43.703 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.703 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.703 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-10T10:04:43.704 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.704 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.704 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.704 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.704 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:43.704 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:43.704 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.704 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.704 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.704 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.704 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.704 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.704 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.704 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.705 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.705 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.705 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.705 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.705 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.705 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.705 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.705 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.705 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.705 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.705 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.705 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:43.705 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:43.705 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.705 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.706 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.706 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.706 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.706 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.706 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.706 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.706 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.706 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.706 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.706 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.706 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:43.706 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:43.706 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.706 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.707 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.707 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:43.707 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.707 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.707 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.707 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.707 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.707 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.707 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.707 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.707 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.707 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.707 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.707 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.708 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.708 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.708 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.708 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:43.708 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:43.708 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.708 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.708 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.708 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.708 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.708 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.708 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.708 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.708 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.709 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.709 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.709 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.709 INFO:teuthology.orchestra.run.smithi077.stdout: "password": { 2024-09-10T10:04:43.709 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "password", 2024-09-10T10:04:43.709 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.709 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.709 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.709 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.709 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.709 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.709 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.709 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "password of InfluxDB server user", 2024-09-10T10:04:43.709 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.710 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.710 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.710 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.710 INFO:teuthology.orchestra.run.smithi077.stdout: "port": { 2024-09-10T10:04:43.710 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "port", 2024-09-10T10:04:43.710 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:43.710 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.710 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.710 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "8086", 2024-09-10T10:04:43.710 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.710 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.710 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.710 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "InfluxDB server port", 2024-09-10T10:04:43.710 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.711 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.711 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.711 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.711 INFO:teuthology.orchestra.run.smithi077.stdout: "ssl": { 2024-09-10T10:04:43.711 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ssl", 2024-09-10T10:04:43.711 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.711 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.711 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.711 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "false", 2024-09-10T10:04:43.711 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.711 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.711 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.711 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-10T10:04:43.711 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.712 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.712 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.712 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.712 INFO:teuthology.orchestra.run.smithi077.stdout: "threads": { 2024-09-10T10:04:43.712 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "threads", 2024-09-10T10:04:43.712 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:43.712 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.712 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.712 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "5", 2024-09-10T10:04:43.712 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "1", 2024-09-10T10:04:43.712 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "32", 2024-09-10T10:04:43.712 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.712 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-10T10:04:43.712 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.713 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.713 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.713 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.713 INFO:teuthology.orchestra.run.smithi077.stdout: "username": { 2024-09-10T10:04:43.713 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "username", 2024-09-10T10:04:43.713 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.713 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.713 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.713 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.713 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.713 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.713 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.713 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "username of InfluxDB server user", 2024-09-10T10:04:43.714 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.714 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.714 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.714 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.714 INFO:teuthology.orchestra.run.smithi077.stdout: "verify_ssl": { 2024-09-10T10:04:43.714 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "verify_ssl", 2024-09-10T10:04:43.714 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.714 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.714 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.714 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "true", 2024-09-10T10:04:43.714 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.714 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.714 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.714 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-10T10:04:43.715 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.715 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.715 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.715 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.715 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.715 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.715 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:43.715 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "insights", 2024-09-10T10:04:43.715 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:43.715 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:43.715 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:43.715 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:43.715 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:43.715 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.716 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.716 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.716 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.716 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.716 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.716 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.716 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.716 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.716 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.716 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.716 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.716 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.716 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.717 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.717 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.717 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.717 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.717 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.717 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:43.717 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:43.717 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.717 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.717 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.717 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.717 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.717 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.717 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.718 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.718 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.718 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.718 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.718 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.718 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:43.718 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:43.718 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.718 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.718 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.718 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:43.718 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.718 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.718 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.719 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.719 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.719 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.719 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.719 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.719 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.719 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.719 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.719 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.719 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.719 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.719 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.719 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:43.719 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:43.720 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.720 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.720 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.720 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.720 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.720 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.720 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.720 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.720 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.720 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.720 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.720 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.720 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.721 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.721 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:43.721 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "iostat", 2024-09-10T10:04:43.721 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:43.721 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:43.721 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:43.721 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:43.721 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:43.721 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.721 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.721 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.721 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.721 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.721 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.722 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.722 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.722 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.722 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.722 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.722 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.722 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.722 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.722 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.722 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.722 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.722 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.722 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.722 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:43.723 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:43.723 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.723 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.723 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.723 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.723 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.723 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.723 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.723 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.723 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.723 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.723 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.723 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.723 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:43.724 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:43.724 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.724 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.724 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.724 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:43.724 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.724 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.724 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.724 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.724 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.724 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.724 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.724 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.724 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.725 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.725 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.725 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.725 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.725 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.725 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.725 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:43.725 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:43.725 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.725 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.725 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.725 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.725 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.725 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.726 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.726 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.726 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.726 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.726 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.726 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.726 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.726 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.726 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:43.726 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "k8sevents", 2024-09-10T10:04:43.726 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:43.726 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:43.726 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:43.727 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_event_retention_days": { 2024-09-10T10:04:43.727 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ceph_event_retention_days", 2024-09-10T10:04:43.727 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:43.727 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.727 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.727 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "7", 2024-09-10T10:04:43.727 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.727 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.727 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.727 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-10T10:04:43.727 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.727 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.727 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.727 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.728 INFO:teuthology.orchestra.run.smithi077.stdout: "config_check_secs": { 2024-09-10T10:04:43.728 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "config_check_secs", 2024-09-10T10:04:43.728 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:43.728 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.728 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.728 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10", 2024-09-10T10:04:43.728 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "10", 2024-09-10T10:04:43.728 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.728 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.728 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-10T10:04:43.728 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.728 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.728 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.728 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.728 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:43.729 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:43.729 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.729 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.729 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.729 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.729 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.729 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.729 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.729 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.729 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.729 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.729 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.729 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.729 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.730 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.730 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.730 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.730 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.730 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.730 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.730 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:43.730 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:43.730 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.730 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.730 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.730 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.730 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.730 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.731 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.731 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.731 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.731 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.731 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.731 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.731 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:43.731 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:43.731 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.731 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.731 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.731 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:43.731 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.732 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.732 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.732 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.732 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.732 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.732 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.732 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.732 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.732 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.732 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.732 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.732 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.732 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.732 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.733 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:43.733 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:43.733 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.733 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.733 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.733 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.733 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.733 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.733 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.733 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.733 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.733 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.733 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.733 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.734 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.734 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.734 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:43.734 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "localpool", 2024-09-10T10:04:43.734 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:43.734 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:43.734 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:43.734 INFO:teuthology.orchestra.run.smithi077.stdout: "failure_domain": { 2024-09-10T10:04:43.734 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "failure_domain", 2024-09-10T10:04:43.734 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.734 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.734 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.734 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "host", 2024-09-10T10:04:43.734 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.735 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.735 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.735 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "failure domain for any created local pool", 2024-09-10T10:04:43.735 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-10T10:04:43.735 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.735 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.735 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.735 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:43.735 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:43.735 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.735 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.735 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.735 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.735 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.736 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.736 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.736 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.736 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.736 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.736 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.736 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.736 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.736 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.736 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.736 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.736 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.736 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.736 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.737 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:43.737 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:43.737 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.737 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.737 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.737 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.737 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.737 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.737 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.737 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.737 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.737 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.738 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.738 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.738 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:43.738 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:43.738 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.738 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.738 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.738 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:43.738 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.738 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.738 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.738 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.738 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.738 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.739 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.739 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.739 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.739 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.739 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.739 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.739 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.739 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.739 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.739 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:43.739 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:43.739 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.739 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.739 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.740 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.740 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.740 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.740 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.740 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.740 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.740 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.740 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.740 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.740 INFO:teuthology.orchestra.run.smithi077.stdout: "min_size": { 2024-09-10T10:04:43.740 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "min_size", 2024-09-10T10:04:43.740 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:43.740 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.740 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.741 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.741 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.741 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.741 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.741 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "default min_size for any created local pool", 2024-09-10T10:04:43.741 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-10T10:04:43.741 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.741 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.741 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.741 INFO:teuthology.orchestra.run.smithi077.stdout: "num_rep": { 2024-09-10T10:04:43.741 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "num_rep", 2024-09-10T10:04:43.741 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:43.741 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.741 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.741 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "3", 2024-09-10T10:04:43.741 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.742 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.742 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.742 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "default replica count for any created local pool", 2024-09-10T10:04:43.742 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.742 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.742 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.742 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.742 INFO:teuthology.orchestra.run.smithi077.stdout: "pg_num": { 2024-09-10T10:04:43.742 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "pg_num", 2024-09-10T10:04:43.742 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:43.742 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.742 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.742 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "128", 2024-09-10T10:04:43.742 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.742 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.743 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.743 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "default pg_num for any created local pool", 2024-09-10T10:04:43.743 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.743 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.743 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.743 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.743 INFO:teuthology.orchestra.run.smithi077.stdout: "prefix": { 2024-09-10T10:04:43.743 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "prefix", 2024-09-10T10:04:43.743 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.743 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.743 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.743 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.743 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.743 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.744 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.744 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "name prefix for any created local pool", 2024-09-10T10:04:43.744 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.744 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.744 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.744 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.744 INFO:teuthology.orchestra.run.smithi077.stdout: "subtree": { 2024-09-10T10:04:43.744 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "subtree", 2024-09-10T10:04:43.744 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.744 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.744 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.744 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "rack", 2024-09-10T10:04:43.744 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.744 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.744 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.745 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-10T10:04:43.745 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-10T10:04:43.745 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.745 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.745 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.745 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.745 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.745 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:43.745 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "mds_autoscaler", 2024-09-10T10:04:43.745 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:43.745 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:43.745 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:43.745 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:43.746 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:43.746 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.746 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.746 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.746 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.746 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.746 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.746 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.746 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.746 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.746 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.746 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.746 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.746 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.746 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.747 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.747 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.747 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.747 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.747 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.747 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:43.747 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:43.747 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.747 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.747 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.747 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.747 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.747 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.747 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.748 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.748 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.748 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.748 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.748 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.748 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:43.748 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:43.748 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.748 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.748 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.748 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:43.748 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.748 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.748 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.749 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.749 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.749 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.749 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.749 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.749 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.749 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.749 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.749 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.749 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.749 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.749 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.749 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:43.749 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:43.749 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.750 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.750 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.750 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.750 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.750 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.750 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.750 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.750 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.750 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.750 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.750 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.750 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.750 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.750 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:43.750 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "mirroring", 2024-09-10T10:04:43.751 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:43.751 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:43.751 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:43.751 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:43.751 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:43.751 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.751 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.751 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.751 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.751 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.751 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.751 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.751 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.751 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.752 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.752 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.752 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.752 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.752 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.752 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.752 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.752 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.752 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.752 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.752 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:43.752 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:43.752 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.752 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.753 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.753 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.753 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.753 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.753 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.753 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.753 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.753 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.753 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.753 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.753 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:43.753 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:43.754 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.754 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.754 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.754 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:43.754 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.754 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.754 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.754 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.754 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.754 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.754 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.754 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.754 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.754 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.754 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.755 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.755 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.755 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.755 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.755 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:43.755 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:43.755 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.755 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.755 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.755 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.755 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.755 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.755 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.755 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.755 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.756 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.756 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.756 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.756 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.756 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.756 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:43.756 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "nfs", 2024-09-10T10:04:43.756 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:43.756 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:43.756 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:43.756 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:43.756 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:43.756 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.757 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.757 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.757 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.757 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.757 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.757 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.757 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.757 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.757 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.757 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.757 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.757 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.758 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.758 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.758 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.758 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.758 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.758 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.758 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:43.758 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:43.758 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.758 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.758 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.758 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.758 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.758 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.758 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.759 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.759 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.759 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.759 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.759 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.759 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:43.759 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:43.759 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.759 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.759 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.759 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:43.759 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.759 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.759 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.760 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.760 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.760 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.760 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.760 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.760 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.760 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.760 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.760 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.760 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.760 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.760 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.760 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:43.760 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:43.761 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.761 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.761 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.761 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.761 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.761 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.761 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.761 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.761 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.761 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.761 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.761 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.761 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.761 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.762 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:43.762 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "orchestrator", 2024-09-10T10:04:43.762 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:43.762 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:43.762 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:43.762 INFO:teuthology.orchestra.run.smithi077.stdout: "fail_fs": { 2024-09-10T10:04:43.762 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "fail_fs", 2024-09-10T10:04:43.762 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.762 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.762 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.762 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.762 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.762 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.762 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.763 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-10T10:04:43.763 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.763 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.763 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.763 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.763 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:43.763 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:43.763 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.763 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.763 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.763 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.763 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.763 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.764 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.764 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.764 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.764 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.764 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.764 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.764 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.764 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.764 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.764 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.764 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.764 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.764 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.765 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:43.765 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:43.765 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.765 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.765 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.765 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.765 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.765 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.765 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.765 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.765 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.765 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.766 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.766 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.766 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:43.766 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:43.766 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.766 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.766 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.766 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:43.766 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.766 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.766 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.766 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.766 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.767 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.767 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.767 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.767 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.767 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.767 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.767 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.767 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.767 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.767 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.767 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:43.767 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:43.767 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.767 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.768 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.768 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.768 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.768 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.768 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.768 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.768 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.768 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.768 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.768 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.768 INFO:teuthology.orchestra.run.smithi077.stdout: "orchestrator": { 2024-09-10T10:04:43.768 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "orchestrator", 2024-09-10T10:04:43.768 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.768 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.769 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.769 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.769 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.769 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.769 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.769 INFO:teuthology.orchestra.run.smithi077.stdout: "cephadm", 2024-09-10T10:04:43.769 INFO:teuthology.orchestra.run.smithi077.stdout: "rook", 2024-09-10T10:04:43.769 INFO:teuthology.orchestra.run.smithi077.stdout: "test_orchestrator" 2024-09-10T10:04:43.769 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.769 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Orchestrator backend", 2024-09-10T10:04:43.769 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.769 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.769 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.769 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.769 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.770 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.770 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:43.770 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "osd_perf_query", 2024-09-10T10:04:43.770 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:43.770 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:43.770 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:43.770 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:43.770 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:43.770 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.770 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.770 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.770 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.770 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.771 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.771 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.771 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.771 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.771 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.771 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.771 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.771 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.771 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.771 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.771 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.771 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.771 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.771 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.772 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:43.772 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:43.772 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.772 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.772 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.772 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.772 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.772 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.772 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.772 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.772 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.772 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.772 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.772 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.772 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:43.773 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:43.773 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.773 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.773 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.773 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:43.773 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.773 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.773 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.773 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.773 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.773 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.773 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.773 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.773 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.774 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.774 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.774 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.774 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.774 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.774 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.774 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:43.774 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:43.774 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.774 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.774 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.774 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.774 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.774 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.775 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.775 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.775 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.775 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.775 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.775 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.775 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.775 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.775 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:43.775 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "osd_support", 2024-09-10T10:04:43.775 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:43.775 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:43.776 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:43.776 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:43.776 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:43.776 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.776 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.776 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.776 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.776 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.776 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.776 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.776 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.776 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.776 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.776 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.777 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.777 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.777 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.777 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.777 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.777 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.777 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.777 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.777 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:43.777 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:43.777 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.777 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.777 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.777 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.778 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.778 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.778 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.778 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.778 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.778 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.778 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.778 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.778 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:43.778 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:43.778 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.778 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.778 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.779 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:43.779 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.779 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.779 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.779 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.779 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.779 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.779 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.779 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.779 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.779 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.779 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.779 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.779 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.780 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.780 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.780 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:43.780 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:43.780 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.780 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.780 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.780 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.780 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.780 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.780 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.780 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.780 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.780 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.781 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.781 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.781 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.781 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.781 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:43.781 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "pg_autoscaler", 2024-09-10T10:04:43.781 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:43.781 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:43.781 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:43.781 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:43.781 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:43.781 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.781 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.782 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.782 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.782 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.782 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.782 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.782 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.782 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.782 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.782 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.782 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.782 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.782 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.782 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.783 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.783 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.783 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.783 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.783 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:43.783 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:43.783 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.783 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.783 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.783 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.783 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.783 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.783 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.784 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.784 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.784 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.784 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.784 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.784 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:43.784 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:43.784 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.784 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.784 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.784 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:43.784 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.785 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.785 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.785 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.785 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.785 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.785 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.785 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.785 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.785 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.785 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.785 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.785 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.785 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.785 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.785 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:43.786 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:43.786 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.786 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.786 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.786 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.786 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.786 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.786 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.786 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.786 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.786 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.787 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.787 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.787 INFO:teuthology.orchestra.run.smithi077.stdout: "sleep_interval": { 2024-09-10T10:04:43.787 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sleep_interval", 2024-09-10T10:04:43.787 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-10T10:04:43.787 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.787 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.787 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "60", 2024-09-10T10:04:43.787 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.787 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.787 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.787 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.787 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.788 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.788 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.788 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.788 INFO:teuthology.orchestra.run.smithi077.stdout: "threshold": { 2024-09-10T10:04:43.788 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "threshold", 2024-09-10T10:04:43.788 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "float", 2024-09-10T10:04:43.788 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.788 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.788 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "3.0", 2024-09-10T10:04:43.788 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "1.0", 2024-09-10T10:04:43.788 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.788 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.788 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "scaling threshold", 2024-09-10T10:04:43.789 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-10T10:04:43.789 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.789 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.789 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.789 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.789 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.789 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:43.789 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "progress", 2024-09-10T10:04:43.789 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:43.789 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:43.789 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:43.789 INFO:teuthology.orchestra.run.smithi077.stdout: "allow_pg_recovery_event": { 2024-09-10T10:04:43.789 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "allow_pg_recovery_event", 2024-09-10T10:04:43.790 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.790 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.790 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.790 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.790 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.790 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.790 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.790 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-10T10:04:43.790 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.790 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.790 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.790 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.790 INFO:teuthology.orchestra.run.smithi077.stdout: "enabled": { 2024-09-10T10:04:43.790 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "enabled", 2024-09-10T10:04:43.791 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.791 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.791 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.791 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:43.791 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.791 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.791 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.791 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.791 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.791 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.791 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.791 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.792 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:43.792 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:43.792 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.792 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.792 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.792 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.792 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.792 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.792 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.792 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.792 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.792 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.792 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.792 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.793 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.793 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.793 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.793 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.793 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.793 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.793 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.793 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:43.793 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:43.793 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.793 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.793 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.794 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.794 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.794 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.794 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.794 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.794 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.794 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.794 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.794 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.794 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:43.794 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:43.794 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.794 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.795 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.795 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:43.795 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.795 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.795 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.795 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.795 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.795 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.795 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.795 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.795 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.795 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.795 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.796 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.796 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.796 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.796 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.796 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:43.796 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:43.796 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.796 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.796 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.796 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.797 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.797 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.797 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.797 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.797 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.797 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.797 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.797 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.797 INFO:teuthology.orchestra.run.smithi077.stdout: "max_completed_events": { 2024-09-10T10:04:43.797 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "max_completed_events", 2024-09-10T10:04:43.797 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:43.797 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.797 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.797 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "50", 2024-09-10T10:04:43.798 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.798 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.798 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.798 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "number of past completed events to remember", 2024-09-10T10:04:43.798 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.798 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.798 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.798 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.798 INFO:teuthology.orchestra.run.smithi077.stdout: "sleep_interval": { 2024-09-10T10:04:43.798 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sleep_interval", 2024-09-10T10:04:43.798 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-10T10:04:43.798 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.798 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.799 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "5", 2024-09-10T10:04:43.799 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.799 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.799 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.799 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "how long the module is going to sleep", 2024-09-10T10:04:43.799 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.799 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.799 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.799 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.799 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.799 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.799 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:43.800 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "prometheus", 2024-09-10T10:04:43.800 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:43.800 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:43.800 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:43.800 INFO:teuthology.orchestra.run.smithi077.stdout: "cache": { 2024-09-10T10:04:43.800 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "cache", 2024-09-10T10:04:43.800 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.800 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.800 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.800 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:43.800 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.800 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.800 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.800 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.801 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.801 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.801 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.801 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.801 INFO:teuthology.orchestra.run.smithi077.stdout: "exclude_perf_counters": { 2024-09-10T10:04:43.801 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "exclude_perf_counters", 2024-09-10T10:04:43.801 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.801 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.801 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.801 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:43.801 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.801 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.801 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.802 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-10T10:04:43.802 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-10T10:04:43.802 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.802 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.802 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.802 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:43.802 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:43.802 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.802 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.802 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.802 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.802 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.802 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.803 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.803 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.803 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.803 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.803 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.803 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.803 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.803 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.803 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.803 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.803 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.803 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.803 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.804 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:43.804 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:43.804 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.804 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.804 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.804 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.804 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.804 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.804 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.804 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.804 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.804 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.804 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.805 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.805 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:43.805 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:43.805 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.805 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.805 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.805 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:43.805 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.805 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.805 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.806 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.806 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.806 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.806 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.806 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.806 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.806 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.806 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.806 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.806 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.806 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.806 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.807 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:43.807 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:43.807 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.807 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.807 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.807 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.807 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.807 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.807 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.807 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.807 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.807 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.807 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.807 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.808 INFO:teuthology.orchestra.run.smithi077.stdout: "rbd_stats_pools": { 2024-09-10T10:04:43.808 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rbd_stats_pools", 2024-09-10T10:04:43.808 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.808 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.808 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.808 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.808 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.808 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.808 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.808 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.808 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.808 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.808 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.809 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.809 INFO:teuthology.orchestra.run.smithi077.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-10T10:04:43.809 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-10T10:04:43.809 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:43.809 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.809 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.809 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "300", 2024-09-10T10:04:43.809 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.809 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.809 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.809 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.809 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.809 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.809 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.810 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.810 INFO:teuthology.orchestra.run.smithi077.stdout: "scrape_interval": { 2024-09-10T10:04:43.810 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "scrape_interval", 2024-09-10T10:04:43.810 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "float", 2024-09-10T10:04:43.810 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.810 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.810 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "15.0", 2024-09-10T10:04:43.810 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.810 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.810 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.810 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.810 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.810 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.810 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.811 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.811 INFO:teuthology.orchestra.run.smithi077.stdout: "server_addr": { 2024-09-10T10:04:43.811 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "server_addr", 2024-09-10T10:04:43.811 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.811 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.811 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.811 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "::", 2024-09-10T10:04:43.811 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.811 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.811 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.811 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-10T10:04:43.811 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.811 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.811 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.812 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.812 INFO:teuthology.orchestra.run.smithi077.stdout: "server_port": { 2024-09-10T10:04:43.812 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "server_port", 2024-09-10T10:04:43.812 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:43.812 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.812 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.812 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "9283", 2024-09-10T10:04:43.812 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.812 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.812 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.812 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-10T10:04:43.812 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.813 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.813 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.813 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.813 INFO:teuthology.orchestra.run.smithi077.stdout: "stale_cache_strategy": { 2024-09-10T10:04:43.813 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "stale_cache_strategy", 2024-09-10T10:04:43.813 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.813 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.813 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.813 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "log", 2024-09-10T10:04:43.813 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.813 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.813 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.813 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.813 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.814 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.814 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.814 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.814 INFO:teuthology.orchestra.run.smithi077.stdout: "standby_behaviour": { 2024-09-10T10:04:43.814 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "standby_behaviour", 2024-09-10T10:04:43.814 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.814 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.814 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.814 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "default", 2024-09-10T10:04:43.814 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.814 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.814 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.815 INFO:teuthology.orchestra.run.smithi077.stdout: "default", 2024-09-10T10:04:43.815 INFO:teuthology.orchestra.run.smithi077.stdout: "error" 2024-09-10T10:04:43.815 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.815 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.815 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.815 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.815 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.815 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.815 INFO:teuthology.orchestra.run.smithi077.stdout: "standby_error_status_code": { 2024-09-10T10:04:43.815 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "standby_error_status_code", 2024-09-10T10:04:43.815 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:43.815 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.815 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.816 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "500", 2024-09-10T10:04:43.816 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "400", 2024-09-10T10:04:43.816 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "599", 2024-09-10T10:04:43.816 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.816 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.816 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.816 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.816 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.816 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.816 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.816 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.816 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:43.816 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rbd_support", 2024-09-10T10:04:43.816 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:43.817 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:43.817 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:43.817 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:43.817 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:43.817 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.817 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.817 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.817 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.817 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.817 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.817 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.817 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.817 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.818 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.818 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.818 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.818 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.818 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.818 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.818 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.818 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.818 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.818 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.818 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:43.818 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:43.818 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.818 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.819 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.819 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.819 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.819 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.819 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.819 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.819 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.819 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.819 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.819 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.819 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:43.819 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:43.819 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.820 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.820 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.820 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:43.820 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.820 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.820 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.820 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.820 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.820 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.820 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.820 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.820 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.821 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.821 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.821 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.821 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.821 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.821 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.821 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:43.821 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:43.821 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.821 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.821 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.821 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.821 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.822 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.822 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.822 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.822 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.822 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.822 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.822 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.822 INFO:teuthology.orchestra.run.smithi077.stdout: "max_concurrent_snap_create": { 2024-09-10T10:04:43.822 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "max_concurrent_snap_create", 2024-09-10T10:04:43.822 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:43.822 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.822 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.822 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10", 2024-09-10T10:04:43.823 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.823 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.823 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.823 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.823 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.823 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.823 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.823 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.823 INFO:teuthology.orchestra.run.smithi077.stdout: "mirror_snapshot_schedule": { 2024-09-10T10:04:43.823 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "mirror_snapshot_schedule", 2024-09-10T10:04:43.823 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.823 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.823 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.823 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.824 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.824 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.824 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.824 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.824 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.824 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.824 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.824 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.824 INFO:teuthology.orchestra.run.smithi077.stdout: "trash_purge_schedule": { 2024-09-10T10:04:43.824 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "trash_purge_schedule", 2024-09-10T10:04:43.824 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.824 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.824 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.825 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.825 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.825 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.825 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.825 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.825 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.825 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.825 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.825 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.825 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.825 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.825 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:43.825 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "restful", 2024-09-10T10:04:43.825 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:43.826 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:43.826 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:43.826 INFO:teuthology.orchestra.run.smithi077.stdout: "enable_auth": { 2024-09-10T10:04:43.826 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "enable_auth", 2024-09-10T10:04:43.826 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.826 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.826 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.826 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:43.826 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.826 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.826 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.826 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.826 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.826 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.826 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.827 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.827 INFO:teuthology.orchestra.run.smithi077.stdout: "key_file": { 2024-09-10T10:04:43.827 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "key_file", 2024-09-10T10:04:43.827 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.827 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.827 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.827 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.827 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.827 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.827 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.827 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.827 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.828 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.828 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.828 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.828 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:43.828 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:43.828 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.828 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.828 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.828 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.828 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.828 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.828 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.828 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.829 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.829 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.829 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.829 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.829 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.829 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.829 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.829 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.829 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.829 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.829 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.829 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:43.829 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:43.829 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.830 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.830 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.830 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.830 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.830 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.830 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.830 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.830 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.830 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.830 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.830 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.830 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:43.830 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:43.830 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.830 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.831 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.831 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:43.831 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.831 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.831 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.831 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.831 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.831 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.831 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.831 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.831 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.831 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.831 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.831 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.832 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.832 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.832 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.832 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:43.832 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:43.832 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.832 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.832 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.832 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.832 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.832 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.832 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.832 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.832 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.833 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.833 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.833 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.833 INFO:teuthology.orchestra.run.smithi077.stdout: "server_addr": { 2024-09-10T10:04:43.833 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "server_addr", 2024-09-10T10:04:43.833 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.833 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.833 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.833 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.833 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.833 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.833 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.833 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.833 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.833 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.834 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.834 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.834 INFO:teuthology.orchestra.run.smithi077.stdout: "server_port": { 2024-09-10T10:04:43.834 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "server_port", 2024-09-10T10:04:43.834 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.834 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.834 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.834 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.834 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.834 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.834 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.834 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.834 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.834 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.834 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.835 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.835 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.835 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.835 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:43.835 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rgw", 2024-09-10T10:04:43.835 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:43.835 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:43.835 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:43.835 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:43.835 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:43.835 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.835 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.835 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.835 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.836 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.836 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.836 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.836 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.836 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.836 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.836 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.836 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.836 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.836 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.836 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.836 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.836 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.837 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.837 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.837 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:43.837 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:43.837 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.837 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.837 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.837 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.837 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.837 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.837 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.837 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.837 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.838 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.838 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.838 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.838 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:43.838 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:43.838 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.838 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.838 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.838 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:43.838 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.838 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.838 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.838 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.838 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.838 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.839 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.839 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.839 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.839 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.839 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.839 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.839 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.839 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.839 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.839 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:43.839 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:43.839 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.839 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.839 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.839 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.840 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.840 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.840 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.840 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.840 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.840 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.840 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.840 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.840 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.840 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.840 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:43.840 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rook", 2024-09-10T10:04:43.840 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:43.840 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:43.840 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:43.841 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:43.841 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:43.841 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.841 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.841 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.841 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.841 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.841 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.841 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.841 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.841 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.841 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.841 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.841 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.842 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.842 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.842 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.842 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.842 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.842 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.842 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.842 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:43.842 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:43.842 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.842 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.842 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.842 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.842 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.842 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.843 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.843 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.843 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.843 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.843 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.843 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.843 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:43.843 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:43.843 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.843 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.843 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.843 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:43.843 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.843 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.843 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.843 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.844 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.844 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.844 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.844 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.844 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.844 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.844 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.844 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.844 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.844 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.844 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.844 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:43.844 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:43.845 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.845 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.845 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.845 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.845 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.845 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.845 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.845 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.845 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.845 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.845 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.845 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.845 INFO:teuthology.orchestra.run.smithi077.stdout: "storage_class": { 2024-09-10T10:04:43.846 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "storage_class", 2024-09-10T10:04:43.846 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.846 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.846 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.846 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "local", 2024-09-10T10:04:43.846 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.846 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.846 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.846 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-10T10:04:43.846 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.846 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.846 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.846 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.847 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.847 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.847 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:43.847 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "selftest", 2024-09-10T10:04:43.847 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:43.847 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:43.847 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:43.847 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:43.847 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:43.847 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.847 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.847 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.847 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.848 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.848 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.848 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.848 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.848 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.848 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.848 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.848 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.848 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.848 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.848 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.848 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.848 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.849 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.849 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.849 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:43.849 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:43.849 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.849 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.849 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.849 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.849 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.849 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.849 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.850 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.850 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.850 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.850 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.850 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.850 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:43.850 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:43.850 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.850 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.850 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.850 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:43.850 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.850 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.851 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.851 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.851 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.851 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.851 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.851 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.851 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.851 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.851 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.851 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.851 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.852 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.852 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.852 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:43.852 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:43.852 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.852 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.852 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.852 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.852 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.852 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.852 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.852 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.853 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.853 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.853 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.853 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.853 INFO:teuthology.orchestra.run.smithi077.stdout: "roption1": { 2024-09-10T10:04:43.853 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "roption1", 2024-09-10T10:04:43.853 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.853 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.853 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.853 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.853 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.854 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.854 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.854 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.854 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.854 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.854 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.854 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.854 INFO:teuthology.orchestra.run.smithi077.stdout: "roption2": { 2024-09-10T10:04:43.854 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "roption2", 2024-09-10T10:04:43.854 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.854 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.854 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.854 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "xyz", 2024-09-10T10:04:43.855 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.855 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.855 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.855 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.855 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.855 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.855 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.855 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.855 INFO:teuthology.orchestra.run.smithi077.stdout: "rwoption1": { 2024-09-10T10:04:43.855 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rwoption1", 2024-09-10T10:04:43.855 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.855 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.855 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.856 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.856 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.856 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.856 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.856 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.856 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.856 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.856 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.856 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.856 INFO:teuthology.orchestra.run.smithi077.stdout: "rwoption2": { 2024-09-10T10:04:43.856 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rwoption2", 2024-09-10T10:04:43.856 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:43.856 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.857 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.857 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.857 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.857 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.857 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.857 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.857 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.857 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.857 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.857 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.857 INFO:teuthology.orchestra.run.smithi077.stdout: "rwoption3": { 2024-09-10T10:04:43.857 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rwoption3", 2024-09-10T10:04:43.857 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "float", 2024-09-10T10:04:43.857 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.858 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.858 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.858 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.858 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.858 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.858 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.858 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.858 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.858 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.858 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.858 INFO:teuthology.orchestra.run.smithi077.stdout: "rwoption4": { 2024-09-10T10:04:43.858 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rwoption4", 2024-09-10T10:04:43.859 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.859 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.859 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.859 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.859 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.859 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.859 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.859 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.859 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.859 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.859 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.859 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.860 INFO:teuthology.orchestra.run.smithi077.stdout: "rwoption5": { 2024-09-10T10:04:43.860 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rwoption5", 2024-09-10T10:04:43.860 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.860 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.860 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.860 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.860 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.860 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.860 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.860 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.860 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.860 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.860 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.861 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.861 INFO:teuthology.orchestra.run.smithi077.stdout: "rwoption6": { 2024-09-10T10:04:43.861 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rwoption6", 2024-09-10T10:04:43.861 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.861 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.861 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.861 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:43.861 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.861 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.861 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.861 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.861 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.861 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.861 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.862 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.862 INFO:teuthology.orchestra.run.smithi077.stdout: "rwoption7": { 2024-09-10T10:04:43.862 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rwoption7", 2024-09-10T10:04:43.862 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:43.862 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.862 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.862 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.862 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "1", 2024-09-10T10:04:43.862 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "42", 2024-09-10T10:04:43.862 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.862 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.862 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.862 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.863 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.863 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.863 INFO:teuthology.orchestra.run.smithi077.stdout: "testkey": { 2024-09-10T10:04:43.863 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "testkey", 2024-09-10T10:04:43.863 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.863 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.863 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.863 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.863 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.863 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.863 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.863 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.863 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.864 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.864 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.864 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.864 INFO:teuthology.orchestra.run.smithi077.stdout: "testlkey": { 2024-09-10T10:04:43.864 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "testlkey", 2024-09-10T10:04:43.864 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.864 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.864 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.864 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.864 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.864 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.864 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.864 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.864 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.865 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.865 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.865 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.865 INFO:teuthology.orchestra.run.smithi077.stdout: "testnewline": { 2024-09-10T10:04:43.865 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "testnewline", 2024-09-10T10:04:43.865 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.865 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.865 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.865 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.865 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.865 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.865 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.865 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.865 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.866 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.866 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.866 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.866 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.866 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.866 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:43.866 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "snap_schedule", 2024-09-10T10:04:43.866 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:43.866 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:43.866 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:43.866 INFO:teuthology.orchestra.run.smithi077.stdout: "allow_m_granularity": { 2024-09-10T10:04:43.866 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "allow_m_granularity", 2024-09-10T10:04:43.866 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.866 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.867 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.867 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.867 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.867 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.867 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.867 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "allow minute scheduled snapshots", 2024-09-10T10:04:43.867 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.867 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.867 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.867 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.867 INFO:teuthology.orchestra.run.smithi077.stdout: "dump_on_update": { 2024-09-10T10:04:43.867 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "dump_on_update", 2024-09-10T10:04:43.867 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.867 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.868 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.868 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.868 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.868 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.868 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.868 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "dump database to debug log on update", 2024-09-10T10:04:43.868 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.868 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.868 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.868 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.868 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:43.868 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:43.868 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.869 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.869 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.869 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.869 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.869 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.869 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.869 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.869 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.869 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.869 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.869 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.869 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.869 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.870 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.870 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.870 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.870 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.870 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.870 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:43.870 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:43.870 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.870 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.870 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.870 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.870 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.870 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.871 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.871 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.871 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.871 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.871 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.871 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.871 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:43.871 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:43.871 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.871 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.871 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.871 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:43.872 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.872 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.872 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.872 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.872 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.872 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.872 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.872 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.872 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.872 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.872 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.872 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.872 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.872 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.873 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.873 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:43.873 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:43.873 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.873 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.873 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.873 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.873 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.873 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.873 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.873 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.873 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.873 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.874 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.874 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.874 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.874 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.874 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:43.874 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "stats", 2024-09-10T10:04:43.874 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:43.874 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:43.874 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:43.874 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:43.874 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:43.874 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.874 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.874 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.874 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.875 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.875 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.875 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.875 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.875 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.875 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.875 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.875 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.875 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.875 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.875 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.875 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.875 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.875 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.876 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.876 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:43.876 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:43.876 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.876 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.876 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.876 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.876 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.876 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.876 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.876 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.876 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.876 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.877 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.877 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.877 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:43.877 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:43.877 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.877 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.877 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.877 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:43.877 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.877 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.877 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.877 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.877 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.878 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.878 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.878 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.878 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.878 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.878 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.878 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.878 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.878 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.878 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.878 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:43.878 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:43.878 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.878 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.878 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.878 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.879 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.879 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.879 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.879 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.879 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.879 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.879 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.879 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.879 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.879 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.879 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:43.879 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "status", 2024-09-10T10:04:43.879 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:43.879 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:43.880 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:43.880 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:43.880 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:43.880 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.880 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.880 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.880 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.880 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.880 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.880 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.880 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.880 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.880 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.881 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.881 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.881 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.881 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.881 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.881 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.881 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.881 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.881 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.881 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:43.881 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:43.881 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.882 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.882 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.882 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.882 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.882 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.882 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.882 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.882 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.882 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.882 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.882 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.882 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:43.882 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:43.882 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.883 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.883 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.883 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:43.883 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.883 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.883 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.883 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.883 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.883 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.883 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.883 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.883 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.883 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.883 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.883 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.884 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.884 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.884 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.884 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:43.884 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:43.884 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.884 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.884 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.884 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.884 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.884 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.884 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.884 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.885 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.885 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.885 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.885 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.885 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.885 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.885 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:43.885 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "telegraf", 2024-09-10T10:04:43.885 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:43.885 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:43.885 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:43.885 INFO:teuthology.orchestra.run.smithi077.stdout: "address": { 2024-09-10T10:04:43.885 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "address", 2024-09-10T10:04:43.886 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.886 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.886 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.886 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-10T10:04:43.886 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.886 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.886 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.886 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.886 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.886 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.886 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.886 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.886 INFO:teuthology.orchestra.run.smithi077.stdout: "interval": { 2024-09-10T10:04:43.886 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "interval", 2024-09-10T10:04:43.886 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-10T10:04:43.887 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.887 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.887 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "15", 2024-09-10T10:04:43.887 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.887 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.887 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.887 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.887 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.887 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.887 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.887 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.887 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:43.887 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:43.887 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.888 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.888 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.888 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.888 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.888 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.888 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.888 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.888 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.888 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.888 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.888 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.888 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.889 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.889 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.889 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.889 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.889 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.889 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.889 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:43.889 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:43.889 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.889 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.889 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.889 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.889 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.890 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.890 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.890 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.890 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.890 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.890 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.890 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.890 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:43.890 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:43.890 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.890 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.890 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.890 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:43.890 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.891 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.891 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.891 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.891 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.891 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.891 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.891 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.891 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.891 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.891 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.891 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.891 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.891 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.891 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.892 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:43.892 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:43.892 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.892 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.892 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.892 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.892 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.892 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.892 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.892 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.892 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.892 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.892 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.892 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.893 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.893 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.893 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:43.893 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "telemetry", 2024-09-10T10:04:43.893 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:43.893 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:43.893 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:43.893 INFO:teuthology.orchestra.run.smithi077.stdout: "channel_basic": { 2024-09-10T10:04:43.893 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "channel_basic", 2024-09-10T10:04:43.893 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.893 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.893 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.893 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:43.893 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.894 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.894 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.894 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-10T10:04:43.894 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.894 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.894 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.894 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.894 INFO:teuthology.orchestra.run.smithi077.stdout: "channel_crash": { 2024-09-10T10:04:43.894 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "channel_crash", 2024-09-10T10:04:43.894 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.894 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.894 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.894 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:43.894 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.895 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.895 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.895 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-10T10:04:43.895 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.895 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.895 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.895 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.895 INFO:teuthology.orchestra.run.smithi077.stdout: "channel_device": { 2024-09-10T10:04:43.895 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "channel_device", 2024-09-10T10:04:43.895 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.895 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.895 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.895 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:43.895 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.895 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.896 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.896 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-10T10:04:43.896 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.896 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.896 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.896 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.896 INFO:teuthology.orchestra.run.smithi077.stdout: "channel_ident": { 2024-09-10T10:04:43.896 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "channel_ident", 2024-09-10T10:04:43.896 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.896 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.896 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.896 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.896 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.896 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.897 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.897 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-10T10:04:43.897 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.897 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.897 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.897 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.897 INFO:teuthology.orchestra.run.smithi077.stdout: "channel_perf": { 2024-09-10T10:04:43.897 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "channel_perf", 2024-09-10T10:04:43.897 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.897 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.897 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.897 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.897 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.898 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.898 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.898 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-10T10:04:43.898 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.898 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.898 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.898 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.898 INFO:teuthology.orchestra.run.smithi077.stdout: "contact": { 2024-09-10T10:04:43.898 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "contact", 2024-09-10T10:04:43.898 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.898 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.898 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.898 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.898 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.899 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.899 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.899 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.899 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.899 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.899 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.899 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.899 INFO:teuthology.orchestra.run.smithi077.stdout: "description": { 2024-09-10T10:04:43.899 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "description", 2024-09-10T10:04:43.899 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.899 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.899 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.899 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.899 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.900 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.900 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.900 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.900 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.900 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.900 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.900 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.900 INFO:teuthology.orchestra.run.smithi077.stdout: "device_url": { 2024-09-10T10:04:43.900 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "device_url", 2024-09-10T10:04:43.900 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.900 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.900 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.900 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-10T10:04:43.900 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.900 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.901 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.901 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.901 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.901 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.901 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.901 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.901 INFO:teuthology.orchestra.run.smithi077.stdout: "enabled": { 2024-09-10T10:04:43.901 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "enabled", 2024-09-10T10:04:43.901 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.901 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.901 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.901 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.901 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.902 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.902 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.902 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.902 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.902 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.902 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.902 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.902 INFO:teuthology.orchestra.run.smithi077.stdout: "interval": { 2024-09-10T10:04:43.902 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "interval", 2024-09-10T10:04:43.902 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:43.902 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.902 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.902 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "24", 2024-09-10T10:04:43.902 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "8", 2024-09-10T10:04:43.902 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.903 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.903 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.903 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.903 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.903 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.903 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.903 INFO:teuthology.orchestra.run.smithi077.stdout: "last_opt_revision": { 2024-09-10T10:04:43.903 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "last_opt_revision", 2024-09-10T10:04:43.903 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:43.903 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.903 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.903 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "1", 2024-09-10T10:04:43.903 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.903 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.903 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.904 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.904 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.904 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.904 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.904 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.904 INFO:teuthology.orchestra.run.smithi077.stdout: "leaderboard": { 2024-09-10T10:04:43.904 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "leaderboard", 2024-09-10T10:04:43.904 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.904 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.904 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.904 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.904 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.904 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.904 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.905 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.905 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.905 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.905 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.905 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.905 INFO:teuthology.orchestra.run.smithi077.stdout: "leaderboard_description": { 2024-09-10T10:04:43.905 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "leaderboard_description", 2024-09-10T10:04:43.905 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.905 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.905 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.905 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.905 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.905 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.905 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.905 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.906 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.906 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.906 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.906 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.906 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:43.906 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:43.906 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.906 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.906 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.906 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.906 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.906 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.906 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.906 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.907 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.907 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.907 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.907 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.907 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.907 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.907 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.907 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.907 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.907 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.907 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.907 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:43.907 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:43.908 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.908 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.908 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.908 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.908 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.908 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.908 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.908 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.908 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.908 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.908 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.908 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.908 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:43.909 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:43.909 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.909 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.909 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.909 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:43.909 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.909 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.909 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.909 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.909 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.909 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.909 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.909 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.909 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.910 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.910 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.910 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.910 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.910 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.910 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.910 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:43.910 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:43.910 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.910 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.910 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.910 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.910 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.911 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.911 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.911 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.911 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.911 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.911 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.911 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.911 INFO:teuthology.orchestra.run.smithi077.stdout: "organization": { 2024-09-10T10:04:43.911 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "organization", 2024-09-10T10:04:43.911 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.911 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.911 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.911 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.912 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.912 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.912 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.912 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.912 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.912 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.912 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.912 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.912 INFO:teuthology.orchestra.run.smithi077.stdout: "proxy": { 2024-09-10T10:04:43.912 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "proxy", 2024-09-10T10:04:43.912 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.912 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.912 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.912 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.912 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.913 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.913 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.913 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.913 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.913 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.913 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.913 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.913 INFO:teuthology.orchestra.run.smithi077.stdout: "url": { 2024-09-10T10:04:43.913 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "url", 2024-09-10T10:04:43.913 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.913 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.913 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.913 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-10T10:04:43.914 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.914 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.914 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.914 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.914 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.914 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.914 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.914 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.914 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.914 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.914 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:43.914 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "test_orchestrator", 2024-09-10T10:04:43.914 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:43.914 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:43.914 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:43.915 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:43.915 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:43.915 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.915 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.915 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.915 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.915 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.915 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.915 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.915 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.915 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.915 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.915 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.915 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.916 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.916 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.916 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.916 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.916 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.916 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.916 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.916 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:43.916 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:43.916 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.916 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.916 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.916 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.916 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.917 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.917 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.917 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.917 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.917 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.917 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.917 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.917 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:43.917 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:43.917 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.917 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.917 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.917 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:43.917 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.917 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.918 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.918 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.918 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.918 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.918 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.918 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.918 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.918 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.918 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.918 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.918 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.918 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.918 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.918 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:43.918 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:43.919 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.919 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.919 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.919 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.919 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.919 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.919 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.919 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.919 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.919 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.919 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.919 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.919 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.919 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.920 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:43.920 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "volumes", 2024-09-10T10:04:43.920 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:43.920 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:43.920 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:43.920 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:43.920 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:43.920 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.920 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.920 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.920 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.920 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.920 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.920 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.920 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.921 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.921 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.921 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.921 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.921 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.921 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.921 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.921 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.921 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.921 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.921 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.921 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:43.921 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:43.921 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.921 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.921 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.922 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.922 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.922 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.922 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.922 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.922 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.922 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.922 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.922 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.922 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:43.922 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:43.922 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.922 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.922 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.923 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:43.923 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.923 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.923 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.923 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.923 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.923 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.923 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.923 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.923 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.923 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.923 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.923 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.923 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.923 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.924 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.924 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:43.924 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:43.924 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.924 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.924 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.924 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.924 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.924 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.924 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.924 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.924 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.924 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.924 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.925 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.925 INFO:teuthology.orchestra.run.smithi077.stdout: "max_concurrent_clones": { 2024-09-10T10:04:43.925 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "max_concurrent_clones", 2024-09-10T10:04:43.925 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:43.925 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.925 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.925 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "4", 2024-09-10T10:04:43.925 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.925 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.925 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.925 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-10T10:04:43.925 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.925 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.925 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.926 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.926 INFO:teuthology.orchestra.run.smithi077.stdout: "periodic_async_work": { 2024-09-10T10:04:43.926 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "periodic_async_work", 2024-09-10T10:04:43.926 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.926 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.926 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.926 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.926 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.926 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.926 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.926 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Periodically check for async work", 2024-09-10T10:04:43.926 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.926 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.926 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.927 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.927 INFO:teuthology.orchestra.run.smithi077.stdout: "snapshot_clone_delay": { 2024-09-10T10:04:43.927 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "snapshot_clone_delay", 2024-09-10T10:04:43.927 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:43.927 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.927 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.927 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-09-10T10:04:43.927 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.927 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.927 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.927 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-10T10:04:43.927 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.927 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.927 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.928 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.928 INFO:teuthology.orchestra.run.smithi077.stdout: "snapshot_clone_no_wait": { 2024-09-10T10:04:43.928 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "snapshot_clone_no_wait", 2024-09-10T10:04:43.928 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.928 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.928 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.928 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-10T10:04:43.928 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.928 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.928 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.928 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-10T10:04:43.928 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.928 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.928 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.928 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.929 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.929 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.929 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:43.929 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "zabbix", 2024-09-10T10:04:43.929 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-10T10:04:43.929 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-10T10:04:43.929 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-10T10:04:43.929 INFO:teuthology.orchestra.run.smithi077.stdout: "discovery_interval": { 2024-09-10T10:04:43.929 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "discovery_interval", 2024-09-10T10:04:43.929 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "uint", 2024-09-10T10:04:43.929 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.929 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.930 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "100", 2024-09-10T10:04:43.930 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.930 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.930 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.930 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.930 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.930 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.930 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.930 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.930 INFO:teuthology.orchestra.run.smithi077.stdout: "identifier": { 2024-09-10T10:04:43.930 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "identifier", 2024-09-10T10:04:43.930 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.930 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.930 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.931 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.931 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.931 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.931 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.931 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.931 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.931 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.931 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.931 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.931 INFO:teuthology.orchestra.run.smithi077.stdout: "interval": { 2024-09-10T10:04:43.931 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "interval", 2024-09-10T10:04:43.931 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-10T10:04:43.931 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.932 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.932 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "60", 2024-09-10T10:04:43.932 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.932 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.932 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.932 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.932 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.932 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.932 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.932 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.932 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-10T10:04:43.932 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-10T10:04:43.932 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.933 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.933 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.933 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.933 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.933 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.933 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.933 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.933 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.933 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.933 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.933 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.933 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.933 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.933 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.934 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.934 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.934 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.934 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.934 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-10T10:04:43.934 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-10T10:04:43.934 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.934 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.934 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.934 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.934 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.934 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.934 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.934 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.934 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.935 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.935 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.935 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.935 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-10T10:04:43.935 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:43.935 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.935 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.935 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.935 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-10T10:04:43.935 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.935 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.935 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-10T10:04:43.935 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-10T10:04:43.935 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-10T10:04:43.935 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-10T10:04:43.936 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-10T10:04:43.936 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-10T10:04:43.936 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-10T10:04:43.936 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.936 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.936 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.936 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.936 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.936 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.936 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-10T10:04:43.936 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-10T10:04:43.936 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-10T10:04:43.936 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.937 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-10T10:04:43.937 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-10T10:04:43.937 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.937 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.937 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.937 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.937 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.937 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.937 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.937 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.937 INFO:teuthology.orchestra.run.smithi077.stdout: "zabbix_host": { 2024-09-10T10:04:43.937 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "zabbix_host", 2024-09-10T10:04:43.938 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.938 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.938 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.938 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-10T10:04:43.938 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.938 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.938 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.938 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.938 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.938 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.938 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.939 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.939 INFO:teuthology.orchestra.run.smithi077.stdout: "zabbix_port": { 2024-09-10T10:04:43.939 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "zabbix_port", 2024-09-10T10:04:43.939 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-10T10:04:43.939 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.939 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.939 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10051", 2024-09-10T10:04:43.939 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.939 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.939 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.939 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.939 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.939 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.940 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.940 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.940 INFO:teuthology.orchestra.run.smithi077.stdout: "zabbix_sender": { 2024-09-10T10:04:43.940 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "zabbix_sender", 2024-09-10T10:04:43.940 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-10T10:04:43.940 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-10T10:04:43.940 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-10T10:04:43.940 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-10T10:04:43.940 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-10T10:04:43.940 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-10T10:04:43.940 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-10T10:04:43.940 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-10T10:04:43.940 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-10T10:04:43.940 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-10T10:04:43.941 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-10T10:04:43.941 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.941 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.941 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.941 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.941 INFO:teuthology.orchestra.run.smithi077.stdout: "services": { 2024-09-10T10:04:43.941 INFO:teuthology.orchestra.run.smithi077.stdout: "dashboard": "https://172.21.15.77:8443/" 2024-09-10T10:04:43.941 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.941 INFO:teuthology.orchestra.run.smithi077.stdout: "always_on_modules": { 2024-09-10T10:04:43.941 INFO:teuthology.orchestra.run.smithi077.stdout: "octopus": [ 2024-09-10T10:04:43.941 INFO:teuthology.orchestra.run.smithi077.stdout: "balancer", 2024-09-10T10:04:43.941 INFO:teuthology.orchestra.run.smithi077.stdout: "crash", 2024-09-10T10:04:43.941 INFO:teuthology.orchestra.run.smithi077.stdout: "devicehealth", 2024-09-10T10:04:43.941 INFO:teuthology.orchestra.run.smithi077.stdout: "orchestrator", 2024-09-10T10:04:43.942 INFO:teuthology.orchestra.run.smithi077.stdout: "pg_autoscaler", 2024-09-10T10:04:43.942 INFO:teuthology.orchestra.run.smithi077.stdout: "progress", 2024-09-10T10:04:43.942 INFO:teuthology.orchestra.run.smithi077.stdout: "rbd_support", 2024-09-10T10:04:43.942 INFO:teuthology.orchestra.run.smithi077.stdout: "status", 2024-09-10T10:04:43.942 INFO:teuthology.orchestra.run.smithi077.stdout: "telemetry", 2024-09-10T10:04:43.942 INFO:teuthology.orchestra.run.smithi077.stdout: "volumes" 2024-09-10T10:04:43.942 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.942 INFO:teuthology.orchestra.run.smithi077.stdout: "pacific": [ 2024-09-10T10:04:43.942 INFO:teuthology.orchestra.run.smithi077.stdout: "balancer", 2024-09-10T10:04:43.942 INFO:teuthology.orchestra.run.smithi077.stdout: "crash", 2024-09-10T10:04:43.942 INFO:teuthology.orchestra.run.smithi077.stdout: "devicehealth", 2024-09-10T10:04:43.942 INFO:teuthology.orchestra.run.smithi077.stdout: "orchestrator", 2024-09-10T10:04:43.943 INFO:teuthology.orchestra.run.smithi077.stdout: "pg_autoscaler", 2024-09-10T10:04:43.943 INFO:teuthology.orchestra.run.smithi077.stdout: "progress", 2024-09-10T10:04:43.943 INFO:teuthology.orchestra.run.smithi077.stdout: "rbd_support", 2024-09-10T10:04:43.943 INFO:teuthology.orchestra.run.smithi077.stdout: "status", 2024-09-10T10:04:43.943 INFO:teuthology.orchestra.run.smithi077.stdout: "telemetry", 2024-09-10T10:04:43.943 INFO:teuthology.orchestra.run.smithi077.stdout: "volumes" 2024-09-10T10:04:43.943 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.943 INFO:teuthology.orchestra.run.smithi077.stdout: "quincy": [ 2024-09-10T10:04:43.943 INFO:teuthology.orchestra.run.smithi077.stdout: "balancer", 2024-09-10T10:04:43.943 INFO:teuthology.orchestra.run.smithi077.stdout: "crash", 2024-09-10T10:04:43.943 INFO:teuthology.orchestra.run.smithi077.stdout: "devicehealth", 2024-09-10T10:04:43.943 INFO:teuthology.orchestra.run.smithi077.stdout: "orchestrator", 2024-09-10T10:04:43.943 INFO:teuthology.orchestra.run.smithi077.stdout: "pg_autoscaler", 2024-09-10T10:04:43.943 INFO:teuthology.orchestra.run.smithi077.stdout: "progress", 2024-09-10T10:04:43.943 INFO:teuthology.orchestra.run.smithi077.stdout: "rbd_support", 2024-09-10T10:04:43.944 INFO:teuthology.orchestra.run.smithi077.stdout: "status", 2024-09-10T10:04:43.944 INFO:teuthology.orchestra.run.smithi077.stdout: "telemetry", 2024-09-10T10:04:43.944 INFO:teuthology.orchestra.run.smithi077.stdout: "volumes" 2024-09-10T10:04:43.944 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.944 INFO:teuthology.orchestra.run.smithi077.stdout: "reef": [ 2024-09-10T10:04:43.944 INFO:teuthology.orchestra.run.smithi077.stdout: "balancer", 2024-09-10T10:04:43.944 INFO:teuthology.orchestra.run.smithi077.stdout: "crash", 2024-09-10T10:04:43.944 INFO:teuthology.orchestra.run.smithi077.stdout: "devicehealth", 2024-09-10T10:04:43.944 INFO:teuthology.orchestra.run.smithi077.stdout: "orchestrator", 2024-09-10T10:04:43.944 INFO:teuthology.orchestra.run.smithi077.stdout: "pg_autoscaler", 2024-09-10T10:04:43.944 INFO:teuthology.orchestra.run.smithi077.stdout: "progress", 2024-09-10T10:04:43.944 INFO:teuthology.orchestra.run.smithi077.stdout: "rbd_support", 2024-09-10T10:04:43.944 INFO:teuthology.orchestra.run.smithi077.stdout: "status", 2024-09-10T10:04:43.944 INFO:teuthology.orchestra.run.smithi077.stdout: "telemetry", 2024-09-10T10:04:43.945 INFO:teuthology.orchestra.run.smithi077.stdout: "volumes" 2024-09-10T10:04:43.945 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-10T10:04:43.945 INFO:teuthology.orchestra.run.smithi077.stdout: "squid": [ 2024-09-10T10:04:43.945 INFO:teuthology.orchestra.run.smithi077.stdout: "balancer", 2024-09-10T10:04:43.945 INFO:teuthology.orchestra.run.smithi077.stdout: "crash", 2024-09-10T10:04:43.945 INFO:teuthology.orchestra.run.smithi077.stdout: "devicehealth", 2024-09-10T10:04:43.945 INFO:teuthology.orchestra.run.smithi077.stdout: "orchestrator", 2024-09-10T10:04:43.945 INFO:teuthology.orchestra.run.smithi077.stdout: "pg_autoscaler", 2024-09-10T10:04:43.945 INFO:teuthology.orchestra.run.smithi077.stdout: "progress", 2024-09-10T10:04:43.945 INFO:teuthology.orchestra.run.smithi077.stdout: "rbd_support", 2024-09-10T10:04:43.945 INFO:teuthology.orchestra.run.smithi077.stdout: "status", 2024-09-10T10:04:43.945 INFO:teuthology.orchestra.run.smithi077.stdout: "telemetry", 2024-09-10T10:04:43.946 INFO:teuthology.orchestra.run.smithi077.stdout: "volumes" 2024-09-10T10:04:43.946 INFO:teuthology.orchestra.run.smithi077.stdout: ] 2024-09-10T10:04:43.946 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.946 INFO:teuthology.orchestra.run.smithi077.stdout: "last_failure_osd_epoch": 3, 2024-09-10T10:04:43.946 INFO:teuthology.orchestra.run.smithi077.stdout: "active_clients": [ 2024-09-10T10:04:43.946 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:43.946 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "devicehealth", 2024-09-10T10:04:43.946 INFO:teuthology.orchestra.run.smithi077.stdout: "addrvec": [ 2024-09-10T10:04:43.946 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:43.946 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "v2", 2024-09-10T10:04:43.946 INFO:teuthology.orchestra.run.smithi077.stdout: "addr": "172.21.15.77:0", 2024-09-10T10:04:43.947 INFO:teuthology.orchestra.run.smithi077.stdout: "nonce": 1084675112 2024-09-10T10:04:43.947 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.947 INFO:teuthology.orchestra.run.smithi077.stdout: ] 2024-09-10T10:04:43.947 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.947 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:43.947 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "libcephsqlite", 2024-09-10T10:04:43.947 INFO:teuthology.orchestra.run.smithi077.stdout: "addrvec": [ 2024-09-10T10:04:43.947 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:43.947 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "v2", 2024-09-10T10:04:43.947 INFO:teuthology.orchestra.run.smithi077.stdout: "addr": "172.21.15.77:0", 2024-09-10T10:04:43.947 INFO:teuthology.orchestra.run.smithi077.stdout: "nonce": 1630899152 2024-09-10T10:04:43.947 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.947 INFO:teuthology.orchestra.run.smithi077.stdout: ] 2024-09-10T10:04:43.948 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.948 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:43.948 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rbd_support", 2024-09-10T10:04:43.948 INFO:teuthology.orchestra.run.smithi077.stdout: "addrvec": [ 2024-09-10T10:04:43.948 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:43.948 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "v2", 2024-09-10T10:04:43.948 INFO:teuthology.orchestra.run.smithi077.stdout: "addr": "172.21.15.77:0", 2024-09-10T10:04:43.948 INFO:teuthology.orchestra.run.smithi077.stdout: "nonce": 589729730 2024-09-10T10:04:43.948 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.948 INFO:teuthology.orchestra.run.smithi077.stdout: ] 2024-09-10T10:04:43.948 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:43.948 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:43.948 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "volumes", 2024-09-10T10:04:43.949 INFO:teuthology.orchestra.run.smithi077.stdout: "addrvec": [ 2024-09-10T10:04:43.949 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-10T10:04:43.949 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "v2", 2024-09-10T10:04:43.949 INFO:teuthology.orchestra.run.smithi077.stdout: "addr": "172.21.15.77:0", 2024-09-10T10:04:43.949 INFO:teuthology.orchestra.run.smithi077.stdout: "nonce": 2998673251 2024-09-10T10:04:43.949 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.949 INFO:teuthology.orchestra.run.smithi077.stdout: ] 2024-09-10T10:04:43.949 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:43.949 INFO:teuthology.orchestra.run.smithi077.stdout: ] 2024-09-10T10:04:43.949 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-10T10:04:43.951 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:04:42 smithi120 ceph-mon[34770]: pgmap v677: 129 pgs: 129 active+clean; 115 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 4.7 KiB/s wr, 14 op/s 2024-09-10T10:04:43.951 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:04:42 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/1582998527' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-10T10:04:43.951 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:04:42 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/4247775583' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-10T10:04:43.951 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:04:42 smithi134 ceph-mon[35674]: pgmap v677: 129 pgs: 129 active+clean; 115 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 4.7 KiB/s wr, 14 op/s 2024-09-10T10:04:43.951 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:04:42 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/1582998527' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-10T10:04:43.951 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:04:42 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/4247775583' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-10T10:04:43.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.611+0000 7f61d1c00640 1 -- 172.21.15.77:0/4247775583 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f61b00a2b90 msgr2=0x7f61b00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:43.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.611+0000 7f61d1c00640 1 --2- 172.21.15.77:0/4247775583 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f61b00a2b90 0x7f61b00a5050 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f61cc19b510 tx=0x7f61b4005f90 comp rx=0 tx=0).stop 2024-09-10T10:04:43.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.611+0000 7f61d1c00640 1 -- 172.21.15.77:0/4247775583 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f61cc103d30 msgr2=0x7f61cc19a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:43.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.611+0000 7f61d1c00640 1 --2- 172.21.15.77:0/4247775583 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f61cc103d30 0x7f61cc19a000 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f61bc00e9c0 tx=0x7f61bc00ee80 comp rx=0 tx=0).stop 2024-09-10T10:04:43.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.611+0000 7f61d1c00640 1 -- 172.21.15.77:0/4247775583 shutdown_connections 2024-09-10T10:04:43.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.611+0000 7f61d1c00640 1 --2- 172.21.15.77:0/4247775583 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f61b00a2b90 0x7f61b00a5050 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:43.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.611+0000 7f61d1c00640 1 --2- 172.21.15.77:0/4247775583 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f61cc1058e0 0x7f61cc1a15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:43.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.611+0000 7f61d1c00640 1 --2- 172.21.15.77:0/4247775583 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f61cc104f20 0x7f61cc19a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:43.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.611+0000 7f61d1c00640 1 --2- 172.21.15.77:0/4247775583 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f61cc103d30 0x7f61cc19a000 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:43.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.611+0000 7f61d1c00640 1 -- 172.21.15.77:0/4247775583 >> 172.21.15.77:0/4247775583 conn(0x7f61cc0ff4b0 msgr2=0x7f61cc108220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:43.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.611+0000 7f61d1c00640 1 -- 172.21.15.77:0/4247775583 shutdown_connections 2024-09-10T10:04:43.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:04:42.611+0000 7f61d1c00640 1 -- 172.21.15.77:0/4247775583 wait complete. 2024-09-10T10:04:43.958 DEBUG:tasks.check_counter:Getting stats from a 2024-09-10T10:04:43.958 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph daemon mds.a perf dump 2024-09-10T10:04:43.959 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:04:42 smithi077 ceph-mon[30319]: pgmap v677: 129 pgs: 129 active+clean; 115 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 4.7 KiB/s wr, 14 op/s 2024-09-10T10:04:43.959 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:04:42 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/1582998527' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-10T10:04:43.959 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:04:42 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/4247775583' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-10T10:04:44.329 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:04:44.944 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-10T10:04:44.944 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T10:04:44.944 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 914, 2024-09-10T10:04:44.944 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 899, 2024-09-10T10:04:44.944 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 804592, 2024-09-10T10:04:44.944 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 345154, 2024-09-10T10:04:44.944 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 5, 2024-09-10T10:04:44.944 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 1, 2024-09-10T10:04:44.945 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 0.126127834, 2024-09-10T10:04:44.945 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 0.053081566, 2024-09-10T10:04:44.945 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 0.042945315, 2024-09-10T10:04:44.945 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.023363987, 2024-09-10T10:04:44.945 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:04:44.945 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 898, 2024-09-10T10:04:44.945 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.062801875, 2024-09-10T10:04:44.945 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000069935 2024-09-10T10:04:44.945 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:44.945 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:04:44.945 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-10T10:04:44.945 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-10T10:04:44.945 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-10T10:04:44.945 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:44.946 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 573760, 2024-09-10T10:04:44.946 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 126688 2024-09-10T10:04:44.946 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:44.946 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T10:04:44.946 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 442, 2024-09-10T10:04:44.946 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 596, 2024-09-10T10:04:44.946 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 3413091, 2024-09-10T10:04:44.946 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 478354, 2024-09-10T10:04:44.946 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 5, 2024-09-10T10:04:44.946 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 4, 2024-09-10T10:04:44.946 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 0.067581954, 2024-09-10T10:04:44.946 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 0.026133742, 2024-09-10T10:04:44.946 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 0.018862323, 2024-09-10T10:04:44.947 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.016011902, 2024-09-10T10:04:44.947 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:04:44.947 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 594, 2024-09-10T10:04:44.947 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.032533802, 2024-09-10T10:04:44.947 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000054770 2024-09-10T10:04:44.947 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:44.947 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:04:44.947 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-10T10:04:44.947 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-10T10:04:44.947 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-10T10:04:44.947 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:44.947 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-10T10:04:44.947 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 338048 2024-09-10T10:04:44.948 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:44.948 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T10:04:44.948 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 739, 2024-09-10T10:04:44.948 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 739, 2024-09-10T10:04:44.948 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 4394708, 2024-09-10T10:04:44.948 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 234179, 2024-09-10T10:04:44.948 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 5, 2024-09-10T10:04:44.948 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 4, 2024-09-10T10:04:44.948 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 0.083960206, 2024-09-10T10:04:44.948 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 0.024328122, 2024-09-10T10:04:44.948 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 0.031272883, 2024-09-10T10:04:44.948 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.021101170, 2024-09-10T10:04:44.948 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:04:44.948 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 738, 2024-09-10T10:04:44.949 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.048634329, 2024-09-10T10:04:44.949 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000065900 2024-09-10T10:04:44.949 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:44.949 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:04:44.949 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-10T10:04:44.949 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-10T10:04:44.949 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-10T10:04:44.949 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:44.949 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-09-10T10:04:44.949 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 448 2024-09-10T10:04:44.949 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:44.949 INFO:teuthology.orchestra.run.smithi077.stdout: "cct": { 2024-09-10T10:04:44.950 INFO:teuthology.orchestra.run.smithi077.stdout: "total_workers": 1, 2024-09-10T10:04:44.950 INFO:teuthology.orchestra.run.smithi077.stdout: "unhealthy_workers": 0 2024-09-10T10:04:44.950 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:44.950 INFO:teuthology.orchestra.run.smithi077.stdout: "finisher-MDSRank": { 2024-09-10T10:04:44.950 INFO:teuthology.orchestra.run.smithi077.stdout: "queue_len": 0, 2024-09-10T10:04:44.950 INFO:teuthology.orchestra.run.smithi077.stdout: "complete_latency": { 2024-09-10T10:04:44.950 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 3172, 2024-09-10T10:04:44.950 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.090891429, 2024-09-10T10:04:44.950 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000028654 2024-09-10T10:04:44.950 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:44.950 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:44.950 INFO:teuthology.orchestra.run.smithi077.stdout: "finisher-PurgeQueue": { 2024-09-10T10:04:44.950 INFO:teuthology.orchestra.run.smithi077.stdout: "queue_len": 0, 2024-09-10T10:04:44.951 INFO:teuthology.orchestra.run.smithi077.stdout: "complete_latency": { 2024-09-10T10:04:44.951 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-10T10:04:44.951 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-10T10:04:44.951 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-10T10:04:44.951 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:44.951 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:44.951 INFO:teuthology.orchestra.run.smithi077.stdout: "mds": { 2024-09-10T10:04:44.951 INFO:teuthology.orchestra.run.smithi077.stdout: "request": 0, 2024-09-10T10:04:44.951 INFO:teuthology.orchestra.run.smithi077.stdout: "reply": 0, 2024-09-10T10:04:44.951 INFO:teuthology.orchestra.run.smithi077.stdout: "reply_latency": { 2024-09-10T10:04:44.951 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-10T10:04:44.951 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-10T10:04:44.951 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-10T10:04:44.952 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:44.952 INFO:teuthology.orchestra.run.smithi077.stdout: "slow_reply": 0, 2024-09-10T10:04:44.952 INFO:teuthology.orchestra.run.smithi077.stdout: "forward": 0, 2024-09-10T10:04:44.952 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_fetch_complete": 0, 2024-09-10T10:04:44.952 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_fetch_keys": 0, 2024-09-10T10:04:44.952 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_commit": 0, 2024-09-10T10:04:44.952 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_split": 0, 2024-09-10T10:04:44.952 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_merge": 0, 2024-09-10T10:04:44.952 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes": 1332, 2024-09-10T10:04:44.952 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_top": 0, 2024-09-10T10:04:44.952 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_bottom": 1332, 2024-09-10T10:04:44.952 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_pin_tail": 0, 2024-09-10T10:04:44.952 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_pinned": 1332, 2024-09-10T10:04:44.952 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_expired": 0, 2024-09-10T10:04:44.953 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_with_caps": 0, 2024-09-10T10:04:44.953 INFO:teuthology.orchestra.run.smithi077.stdout: "caps": 0, 2024-09-10T10:04:44.953 INFO:teuthology.orchestra.run.smithi077.stdout: "subtrees": 1, 2024-09-10T10:04:44.953 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse": 0, 2024-09-10T10:04:44.953 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_hit": 0, 2024-09-10T10:04:44.953 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_forward": 0, 2024-09-10T10:04:44.953 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_discover": 0, 2024-09-10T10:04:44.953 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_dir_fetch": 0, 2024-09-10T10:04:44.953 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_remote_ino": 0, 2024-09-10T10:04:44.953 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_lock": 0, 2024-09-10T10:04:44.953 INFO:teuthology.orchestra.run.smithi077.stdout: "load_cent": 0, 2024-09-10T10:04:44.953 INFO:teuthology.orchestra.run.smithi077.stdout: "q": 0, 2024-09-10T10:04:44.953 INFO:teuthology.orchestra.run.smithi077.stdout: "exported": 0, 2024-09-10T10:04:44.953 INFO:teuthology.orchestra.run.smithi077.stdout: "exported_inodes": 0, 2024-09-10T10:04:44.954 INFO:teuthology.orchestra.run.smithi077.stdout: "imported": 0, 2024-09-10T10:04:44.954 INFO:teuthology.orchestra.run.smithi077.stdout: "imported_inodes": 0, 2024-09-10T10:04:44.954 INFO:teuthology.orchestra.run.smithi077.stdout: "openino_dir_fetch": 0, 2024-09-10T10:04:44.954 INFO:teuthology.orchestra.run.smithi077.stdout: "openino_backtrace_fetch": 0, 2024-09-10T10:04:44.954 INFO:teuthology.orchestra.run.smithi077.stdout: "openino_peer_discover": 0, 2024-09-10T10:04:44.954 INFO:teuthology.orchestra.run.smithi077.stdout: "root_rfiles": 0, 2024-09-10T10:04:44.954 INFO:teuthology.orchestra.run.smithi077.stdout: "root_rbytes": 0, 2024-09-10T10:04:44.954 INFO:teuthology.orchestra.run.smithi077.stdout: "root_rsnaps": 0, 2024-09-10T10:04:44.954 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_backtrace_fetch": 0, 2024-09-10T10:04:44.954 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_set_tag": 0, 2024-09-10T10:04:44.954 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T10:04:44.954 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_inotable_repaired": 0, 2024-09-10T10:04:44.954 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_dir_inodes": 0, 2024-09-10T10:04:44.954 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_dir_base_inodes": 0, 2024-09-10T10:04:44.954 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_dirfrag_rstats": 0, 2024-09-10T10:04:44.955 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_file_inodes": 0, 2024-09-10T10:04:44.955 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_inode_file_caps": 0, 2024-09-10T10:04:44.955 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_revoke": 0, 2024-09-10T10:04:44.955 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_grant": 0, 2024-09-10T10:04:44.955 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T10:04:44.955 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T10:04:44.955 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T10:04:44.955 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_caps": 0, 2024-09-10T10:04:44.955 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_caps_dirty": 0, 2024-09-10T10:04:44.955 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_cap_release": 0, 2024-09-10T10:04:44.955 INFO:teuthology.orchestra.run.smithi077.stdout: "process_request_cap_release": 0 2024-09-10T10:04:44.955 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:44.955 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_cache": { 2024-09-10T10:04:44.955 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_update": 0, 2024-09-10T10:04:44.955 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_update_receipt": 0, 2024-09-10T10:04:44.956 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_try_discover": 0, 2024-09-10T10:04:44.956 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_send_discover": 0, 2024-09-10T10:04:44.956 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_handle_discover": 0, 2024-09-10T10:04:44.956 INFO:teuthology.orchestra.run.smithi077.stdout: "num_strays": 0, 2024-09-10T10:04:44.956 INFO:teuthology.orchestra.run.smithi077.stdout: "num_strays_delayed": 0, 2024-09-10T10:04:44.956 INFO:teuthology.orchestra.run.smithi077.stdout: "num_strays_enqueuing": 0, 2024-09-10T10:04:44.956 INFO:teuthology.orchestra.run.smithi077.stdout: "strays_created": 0, 2024-09-10T10:04:44.956 INFO:teuthology.orchestra.run.smithi077.stdout: "strays_enqueued": 0, 2024-09-10T10:04:44.956 INFO:teuthology.orchestra.run.smithi077.stdout: "strays_reintegrated": 0, 2024-09-10T10:04:44.956 INFO:teuthology.orchestra.run.smithi077.stdout: "strays_migrated": 0, 2024-09-10T10:04:44.956 INFO:teuthology.orchestra.run.smithi077.stdout: "num_recovering_processing": 0, 2024-09-10T10:04:44.956 INFO:teuthology.orchestra.run.smithi077.stdout: "num_recovering_enqueued": 0, 2024-09-10T10:04:44.956 INFO:teuthology.orchestra.run.smithi077.stdout: "num_recovering_prioritized": 0, 2024-09-10T10:04:44.956 INFO:teuthology.orchestra.run.smithi077.stdout: "recovery_started": 0, 2024-09-10T10:04:44.956 INFO:teuthology.orchestra.run.smithi077.stdout: "recovery_completed": 0, 2024-09-10T10:04:44.957 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_quiesce_path": 0, 2024-09-10T10:04:44.957 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_quiesce_inode": 0, 2024-09-10T10:04:44.957 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_enqueue_scrub": 0, 2024-09-10T10:04:44.957 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_exportdir": 0, 2024-09-10T10:04:44.957 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_flush": 0, 2024-09-10T10:04:44.957 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_fragmentdir": 0, 2024-09-10T10:04:44.957 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_fragstats": 0, 2024-09-10T10:04:44.957 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_inodestats": 0 2024-09-10T10:04:44.957 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:44.957 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_log": { 2024-09-10T10:04:44.957 INFO:teuthology.orchestra.run.smithi077.stdout: "evlrg": 0, 2024-09-10T10:04:44.957 INFO:teuthology.orchestra.run.smithi077.stdout: "evadd": 0, 2024-09-10T10:04:44.957 INFO:teuthology.orchestra.run.smithi077.stdout: "evex": 0, 2024-09-10T10:04:44.957 INFO:teuthology.orchestra.run.smithi077.stdout: "evtrm": 0, 2024-09-10T10:04:44.958 INFO:teuthology.orchestra.run.smithi077.stdout: "ev": 4674, 2024-09-10T10:04:44.958 INFO:teuthology.orchestra.run.smithi077.stdout: "evexg": 0, 2024-09-10T10:04:44.958 INFO:teuthology.orchestra.run.smithi077.stdout: "evexd": 0, 2024-09-10T10:04:44.958 INFO:teuthology.orchestra.run.smithi077.stdout: "segadd": 0, 2024-09-10T10:04:44.958 INFO:teuthology.orchestra.run.smithi077.stdout: "segex": 0, 2024-09-10T10:04:44.958 INFO:teuthology.orchestra.run.smithi077.stdout: "segtrm": 0, 2024-09-10T10:04:44.958 INFO:teuthology.orchestra.run.smithi077.stdout: "seg": 7, 2024-09-10T10:04:44.958 INFO:teuthology.orchestra.run.smithi077.stdout: "segmjr": 2, 2024-09-10T10:04:44.958 INFO:teuthology.orchestra.run.smithi077.stdout: "segexg": 0, 2024-09-10T10:04:44.958 INFO:teuthology.orchestra.run.smithi077.stdout: "segexd": 0, 2024-09-10T10:04:44.958 INFO:teuthology.orchestra.run.smithi077.stdout: "expos": 4194304, 2024-09-10T10:04:44.958 INFO:teuthology.orchestra.run.smithi077.stdout: "wrpos": 11691921, 2024-09-10T10:04:44.958 INFO:teuthology.orchestra.run.smithi077.stdout: "rdpos": 11690783, 2024-09-10T10:04:44.958 INFO:teuthology.orchestra.run.smithi077.stdout: "jlat": { 2024-09-10T10:04:44.958 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-10T10:04:44.959 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-10T10:04:44.959 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-10T10:04:44.959 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:44.959 INFO:teuthology.orchestra.run.smithi077.stdout: "replayed": 4674 2024-09-10T10:04:44.959 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:44.959 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_mem": { 2024-09-10T10:04:44.959 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 8, 2024-09-10T10:04:44.959 INFO:teuthology.orchestra.run.smithi077.stdout: "ino+": 1371, 2024-09-10T10:04:44.959 INFO:teuthology.orchestra.run.smithi077.stdout: "ino-": 1363, 2024-09-10T10:04:44.959 INFO:teuthology.orchestra.run.smithi077.stdout: "dir": 6, 2024-09-10T10:04:44.959 INFO:teuthology.orchestra.run.smithi077.stdout: "dir+": 70, 2024-09-10T10:04:44.959 INFO:teuthology.orchestra.run.smithi077.stdout: "dir-": 64, 2024-09-10T10:04:44.959 INFO:teuthology.orchestra.run.smithi077.stdout: "dn": 1332, 2024-09-10T10:04:44.959 INFO:teuthology.orchestra.run.smithi077.stdout: "dn+": 2506, 2024-09-10T10:04:44.959 INFO:teuthology.orchestra.run.smithi077.stdout: "dn-": 1174, 2024-09-10T10:04:44.959 INFO:teuthology.orchestra.run.smithi077.stdout: "cap": 0, 2024-09-10T10:04:44.960 INFO:teuthology.orchestra.run.smithi077.stdout: "cap+": 0, 2024-09-10T10:04:44.960 INFO:teuthology.orchestra.run.smithi077.stdout: "cap-": 0, 2024-09-10T10:04:44.960 INFO:teuthology.orchestra.run.smithi077.stdout: "rss": 51964, 2024-09-10T10:04:44.960 INFO:teuthology.orchestra.run.smithi077.stdout: "heap": 215328 2024-09-10T10:04:44.960 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:44.960 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_server": { 2024-09-10T10:04:44.960 INFO:teuthology.orchestra.run.smithi077.stdout: "dispatch_client_request": 0, 2024-09-10T10:04:44.960 INFO:teuthology.orchestra.run.smithi077.stdout: "dispatch_server_request": 0, 2024-09-10T10:04:44.960 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_request": 0, 2024-09-10T10:04:44.960 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_session": 0, 2024-09-10T10:04:44.960 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_peer_request": 0, 2024-09-10T10:04:44.961 INFO:teuthology.orchestra.run.smithi077.stdout: "req_create_latency": { 2024-09-10T10:04:44.961 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-10T10:04:44.961 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-10T10:04:44.961 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-10T10:04:44.961 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:44.961 INFO:teuthology.orchestra.run.smithi077.stdout: "req_getattr_latency": { 2024-09-10T10:04:44.961 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-10T10:04:44.961 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-10T10:04:44.961 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-10T10:04:44.961 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:44.962 INFO:teuthology.orchestra.run.smithi077.stdout: "req_getfilelock_latency": { 2024-09-10T10:04:44.962 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-10T10:04:44.962 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-10T10:04:44.962 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-10T10:04:44.962 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:44.962 INFO:teuthology.orchestra.run.smithi077.stdout: "req_link_latency": { 2024-09-10T10:04:44.962 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-10T10:04:44.962 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-10T10:04:44.962 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-10T10:04:44.962 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:44.962 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookup_latency": { 2024-09-10T10:04:44.962 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-10T10:04:44.962 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-10T10:04:44.963 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-10T10:04:44.963 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:44.963 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookuphash_latency": { 2024-09-10T10:04:44.963 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-10T10:04:44.963 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-10T10:04:44.963 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-10T10:04:44.963 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:44.963 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookupino_latency": { 2024-09-10T10:04:44.963 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-10T10:04:44.963 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-10T10:04:44.963 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-10T10:04:44.963 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:44.963 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookupname_latency": { 2024-09-10T10:04:44.964 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-10T10:04:44.964 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-10T10:04:44.964 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-10T10:04:44.964 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:44.964 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookupparent_latency": { 2024-09-10T10:04:44.964 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-10T10:04:44.964 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-10T10:04:44.964 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-10T10:04:44.964 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:44.964 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookupsnap_latency": { 2024-09-10T10:04:44.964 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-10T10:04:44.964 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-10T10:04:44.964 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-10T10:04:44.964 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:44.965 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lssnap_latency": { 2024-09-10T10:04:44.965 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-10T10:04:44.965 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-10T10:04:44.965 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-10T10:04:44.965 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:44.965 INFO:teuthology.orchestra.run.smithi077.stdout: "req_mkdir_latency": { 2024-09-10T10:04:44.966 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-10T10:04:44.966 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-10T10:04:44.966 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-10T10:04:44.966 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:44.966 INFO:teuthology.orchestra.run.smithi077.stdout: "req_mknod_latency": { 2024-09-10T10:04:44.966 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-10T10:04:44.966 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-10T10:04:44.966 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-10T10:04:44.967 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:44.967 INFO:teuthology.orchestra.run.smithi077.stdout: "req_mksnap_latency": { 2024-09-10T10:04:44.967 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-10T10:04:44.967 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-10T10:04:44.967 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-10T10:04:44.967 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:44.967 INFO:teuthology.orchestra.run.smithi077.stdout: "req_open_latency": { 2024-09-10T10:04:44.967 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-10T10:04:44.967 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-10T10:04:44.967 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-10T10:04:44.967 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:44.967 INFO:teuthology.orchestra.run.smithi077.stdout: "req_readdir_latency": { 2024-09-10T10:04:44.967 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-10T10:04:44.968 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-10T10:04:44.968 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-10T10:04:44.968 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:44.968 INFO:teuthology.orchestra.run.smithi077.stdout: "req_rename_latency": { 2024-09-10T10:04:44.968 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-10T10:04:44.968 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-10T10:04:44.968 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-10T10:04:44.968 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:44.968 INFO:teuthology.orchestra.run.smithi077.stdout: "req_renamesnap_latency": { 2024-09-10T10:04:44.968 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-10T10:04:44.968 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-10T10:04:44.968 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-10T10:04:44.969 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:44.969 INFO:teuthology.orchestra.run.smithi077.stdout: "req_snapdiff_latency": { 2024-09-10T10:04:44.969 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-10T10:04:44.969 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-10T10:04:44.969 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-10T10:04:44.969 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:44.969 INFO:teuthology.orchestra.run.smithi077.stdout: "req_rmdir_latency": { 2024-09-10T10:04:44.969 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-10T10:04:44.969 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-10T10:04:44.969 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-10T10:04:44.969 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:44.969 INFO:teuthology.orchestra.run.smithi077.stdout: "req_rmsnap_latency": { 2024-09-10T10:04:44.969 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-10T10:04:44.969 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-10T10:04:44.970 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-10T10:04:44.970 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:44.970 INFO:teuthology.orchestra.run.smithi077.stdout: "req_rmxattr_latency": { 2024-09-10T10:04:44.970 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-10T10:04:44.970 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-10T10:04:44.970 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-10T10:04:44.970 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:44.970 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setattr_latency": { 2024-09-10T10:04:44.970 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-10T10:04:44.970 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-10T10:04:44.970 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-10T10:04:44.970 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:44.970 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setdirlayout_latency": { 2024-09-10T10:04:44.970 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-10T10:04:44.971 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-10T10:04:44.971 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-10T10:04:44.971 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:44.971 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setfilelock_latency": { 2024-09-10T10:04:44.971 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-10T10:04:44.971 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-10T10:04:44.971 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-10T10:04:44.971 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:44.971 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setlayout_latency": { 2024-09-10T10:04:44.971 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-10T10:04:44.971 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-10T10:04:44.971 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-10T10:04:44.971 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:44.971 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setxattr_latency": { 2024-09-10T10:04:44.972 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-10T10:04:44.972 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-10T10:04:44.972 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-10T10:04:44.972 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:44.972 INFO:teuthology.orchestra.run.smithi077.stdout: "req_symlink_latency": { 2024-09-10T10:04:44.972 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-10T10:04:44.972 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-10T10:04:44.972 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-10T10:04:44.972 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:44.972 INFO:teuthology.orchestra.run.smithi077.stdout: "req_unlink_latency": { 2024-09-10T10:04:44.972 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-10T10:04:44.972 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-10T10:04:44.972 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-10T10:04:44.973 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:44.973 INFO:teuthology.orchestra.run.smithi077.stdout: "cap_revoke_eviction": 0, 2024-09-10T10:04:44.973 INFO:teuthology.orchestra.run.smithi077.stdout: "cap_acquisition_throttle": 0, 2024-09-10T10:04:44.973 INFO:teuthology.orchestra.run.smithi077.stdout: "req_getvxattr_latency": { 2024-09-10T10:04:44.973 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-10T10:04:44.973 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-10T10:04:44.973 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-10T10:04:44.973 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:44.973 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:44.973 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_sessions": { 2024-09-10T10:04:44.973 INFO:teuthology.orchestra.run.smithi077.stdout: "session_count": 1, 2024-09-10T10:04:44.973 INFO:teuthology.orchestra.run.smithi077.stdout: "session_add": 1, 2024-09-10T10:04:44.973 INFO:teuthology.orchestra.run.smithi077.stdout: "session_remove": 0, 2024-09-10T10:04:44.973 INFO:teuthology.orchestra.run.smithi077.stdout: "sessions_open": 1, 2024-09-10T10:04:44.974 INFO:teuthology.orchestra.run.smithi077.stdout: "sessions_stale": 0, 2024-09-10T10:04:44.974 INFO:teuthology.orchestra.run.smithi077.stdout: "total_load": 0, 2024-09-10T10:04:44.974 INFO:teuthology.orchestra.run.smithi077.stdout: "average_load": 0, 2024-09-10T10:04:44.974 INFO:teuthology.orchestra.run.smithi077.stdout: "avg_session_uptime": 1713, 2024-09-10T10:04:44.974 INFO:teuthology.orchestra.run.smithi077.stdout: "mdthresh_evicted": 0 2024-09-10T10:04:44.974 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:44.974 INFO:teuthology.orchestra.run.smithi077.stdout: "mempool": { 2024-09-10T10:04:44.974 INFO:teuthology.orchestra.run.smithi077.stdout: "bloom_filter_bytes": 0, 2024-09-10T10:04:44.974 INFO:teuthology.orchestra.run.smithi077.stdout: "bloom_filter_items": 0, 2024-09-10T10:04:44.974 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T10:04:44.974 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_alloc_items": 0, 2024-09-10T10:04:44.974 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T10:04:44.974 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_data_items": 0, 2024-09-10T10:04:44.975 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T10:04:44.975 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T10:04:44.975 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T10:04:44.975 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T10:04:44.975 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T10:04:44.975 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_other_items": 0, 2024-09-10T10:04:44.975 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T10:04:44.975 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T10:04:44.975 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_extent_bytes": 0, 2024-09-10T10:04:44.975 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_extent_items": 0, 2024-09-10T10:04:44.975 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_blob_bytes": 0, 2024-09-10T10:04:44.975 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_blob_items": 0, 2024-09-10T10:04:44.975 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T10:04:44.975 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T10:04:44.976 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T10:04:44.976 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T10:04:44.976 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T10:04:44.976 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_fsck_items": 0, 2024-09-10T10:04:44.976 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_txc_bytes": 0, 2024-09-10T10:04:44.976 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_txc_items": 0, 2024-09-10T10:04:44.976 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T10:04:44.976 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T10:04:44.976 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_bytes": 0, 2024-09-10T10:04:44.976 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_items": 0, 2024-09-10T10:04:44.976 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_bytes": 0, 2024-09-10T10:04:44.976 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_items": 0, 2024-09-10T10:04:44.976 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T10:04:44.976 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_reader_items": 0, 2024-09-10T10:04:44.977 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T10:04:44.977 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_writer_items": 0, 2024-09-10T10:04:44.977 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_anon_bytes": 198494, 2024-09-10T10:04:44.977 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_anon_items": 55, 2024-09-10T10:04:44.977 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_meta_bytes": 0, 2024-09-10T10:04:44.977 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_meta_items": 0, 2024-09-10T10:04:44.977 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_bytes": 0, 2024-09-10T10:04:44.977 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_items": 0, 2024-09-10T10:04:44.977 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_mapbl_bytes": 0, 2024-09-10T10:04:44.977 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_mapbl_items": 0, 2024-09-10T10:04:44.977 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_pglog_bytes": 0, 2024-09-10T10:04:44.977 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_pglog_items": 0, 2024-09-10T10:04:44.977 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_bytes": 9144, 2024-09-10T10:04:44.977 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_items": 172, 2024-09-10T10:04:44.978 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T10:04:44.978 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_mapping_items": 0, 2024-09-10T10:04:44.978 INFO:teuthology.orchestra.run.smithi077.stdout: "pgmap_bytes": 0, 2024-09-10T10:04:44.978 INFO:teuthology.orchestra.run.smithi077.stdout: "pgmap_items": 0, 2024-09-10T10:04:44.978 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_co_bytes": 963800, 2024-09-10T10:04:44.978 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_co_items": 5401, 2024-09-10T10:04:44.978 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_1_bytes": 0, 2024-09-10T10:04:44.978 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_1_items": 0, 2024-09-10T10:04:44.978 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_2_bytes": 0, 2024-09-10T10:04:44.978 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_2_items": 0 2024-09-10T10:04:44.978 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:44.978 INFO:teuthology.orchestra.run.smithi077.stdout: "objecter": { 2024-09-10T10:04:44.978 INFO:teuthology.orchestra.run.smithi077.stdout: "op_active": 0, 2024-09-10T10:04:44.978 INFO:teuthology.orchestra.run.smithi077.stdout: "op_laggy": 0, 2024-09-10T10:04:44.979 INFO:teuthology.orchestra.run.smithi077.stdout: "op_send": 1863, 2024-09-10T10:04:44.979 INFO:teuthology.orchestra.run.smithi077.stdout: "op_send_bytes": 21, 2024-09-10T10:04:44.979 INFO:teuthology.orchestra.run.smithi077.stdout: "op_resend": 3, 2024-09-10T10:04:44.979 INFO:teuthology.orchestra.run.smithi077.stdout: "op_reply": 1863, 2024-09-10T10:04:44.979 INFO:teuthology.orchestra.run.smithi077.stdout: "op_latency": { 2024-09-10T10:04:44.979 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 1863, 2024-09-10T10:04:44.979 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 1.229974224, 2024-09-10T10:04:44.979 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000660211 2024-09-10T10:04:44.979 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:44.979 INFO:teuthology.orchestra.run.smithi077.stdout: "op_inflight": 0, 2024-09-10T10:04:44.979 INFO:teuthology.orchestra.run.smithi077.stdout: "oplen_avg": { 2024-09-10T10:04:44.979 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 1863, 2024-09-10T10:04:44.979 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 1864 2024-09-10T10:04:44.979 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:44.980 INFO:teuthology.orchestra.run.smithi077.stdout: "op": 1863, 2024-09-10T10:04:44.980 INFO:teuthology.orchestra.run.smithi077.stdout: "op_r": 1863, 2024-09-10T10:04:44.980 INFO:teuthology.orchestra.run.smithi077.stdout: "op_w": 0, 2024-09-10T10:04:44.980 INFO:teuthology.orchestra.run.smithi077.stdout: "op_rmw": 0, 2024-09-10T10:04:44.980 INFO:teuthology.orchestra.run.smithi077.stdout: "op_pg": 0, 2024-09-10T10:04:44.980 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_stat": 1115, 2024-09-10T10:04:44.980 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_create": 0, 2024-09-10T10:04:44.980 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_read": 746, 2024-09-10T10:04:44.980 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_write": 0, 2024-09-10T10:04:44.980 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_writefull": 0, 2024-09-10T10:04:44.980 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_writesame": 0, 2024-09-10T10:04:44.980 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_append": 0, 2024-09-10T10:04:44.980 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_zero": 0, 2024-09-10T10:04:44.980 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_truncate": 0, 2024-09-10T10:04:44.980 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_delete": 0, 2024-09-10T10:04:44.981 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_mapext": 0, 2024-09-10T10:04:44.981 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_sparse_read": 0, 2024-09-10T10:04:44.981 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_clonerange": 0, 2024-09-10T10:04:44.981 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_getxattr": 1, 2024-09-10T10:04:44.981 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_setxattr": 0, 2024-09-10T10:04:44.981 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_cmpxattr": 0, 2024-09-10T10:04:44.981 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_rmxattr": 0, 2024-09-10T10:04:44.981 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_resetxattrs": 0, 2024-09-10T10:04:44.981 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_call": 0, 2024-09-10T10:04:44.981 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_watch": 0, 2024-09-10T10:04:44.981 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_notify": 0, 2024-09-10T10:04:44.981 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T10:04:44.981 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_pgls": 0, 2024-09-10T10:04:44.981 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_pgls_filter": 0, 2024-09-10T10:04:44.982 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_other": 0, 2024-09-10T10:04:44.982 INFO:teuthology.orchestra.run.smithi077.stdout: "linger_active": 0, 2024-09-10T10:04:44.982 INFO:teuthology.orchestra.run.smithi077.stdout: "linger_send": 0, 2024-09-10T10:04:44.982 INFO:teuthology.orchestra.run.smithi077.stdout: "linger_resend": 0, 2024-09-10T10:04:44.982 INFO:teuthology.orchestra.run.smithi077.stdout: "linger_ping": 0, 2024-09-10T10:04:44.982 INFO:teuthology.orchestra.run.smithi077.stdout: "poolop_active": 0, 2024-09-10T10:04:44.982 INFO:teuthology.orchestra.run.smithi077.stdout: "poolop_send": 0, 2024-09-10T10:04:44.982 INFO:teuthology.orchestra.run.smithi077.stdout: "poolop_resend": 0, 2024-09-10T10:04:44.982 INFO:teuthology.orchestra.run.smithi077.stdout: "poolstat_active": 0, 2024-09-10T10:04:44.982 INFO:teuthology.orchestra.run.smithi077.stdout: "poolstat_send": 0, 2024-09-10T10:04:44.982 INFO:teuthology.orchestra.run.smithi077.stdout: "poolstat_resend": 0, 2024-09-10T10:04:44.982 INFO:teuthology.orchestra.run.smithi077.stdout: "statfs_active": 0, 2024-09-10T10:04:44.982 INFO:teuthology.orchestra.run.smithi077.stdout: "statfs_send": 0, 2024-09-10T10:04:44.982 INFO:teuthology.orchestra.run.smithi077.stdout: "statfs_resend": 0, 2024-09-10T10:04:44.983 INFO:teuthology.orchestra.run.smithi077.stdout: "command_active": 0, 2024-09-10T10:04:44.983 INFO:teuthology.orchestra.run.smithi077.stdout: "command_send": 0, 2024-09-10T10:04:44.983 INFO:teuthology.orchestra.run.smithi077.stdout: "command_resend": 0, 2024-09-10T10:04:44.983 INFO:teuthology.orchestra.run.smithi077.stdout: "map_epoch": 96, 2024-09-10T10:04:44.983 INFO:teuthology.orchestra.run.smithi077.stdout: "map_full": 0, 2024-09-10T10:04:44.983 INFO:teuthology.orchestra.run.smithi077.stdout: "map_inc": 2, 2024-09-10T10:04:44.983 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_sessions": 6, 2024-09-10T10:04:44.983 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_session_open": 6, 2024-09-10T10:04:44.983 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_session_close": 0, 2024-09-10T10:04:44.983 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_laggy": 0, 2024-09-10T10:04:44.983 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_wr": 0, 2024-09-10T10:04:44.983 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_rd": 2, 2024-09-10T10:04:44.983 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_del": 0 2024-09-10T10:04:44.983 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:44.983 INFO:teuthology.orchestra.run.smithi077.stdout: "oft": { 2024-09-10T10:04:44.984 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_total_objs": 0, 2024-09-10T10:04:44.984 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_total_kv_pairs": 0, 2024-09-10T10:04:44.984 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_total_updates": 0, 2024-09-10T10:04:44.984 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_total_removes": 0 2024-09-10T10:04:44.984 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:44.984 INFO:teuthology.orchestra.run.smithi077.stdout: "osd-slow-ops": { 2024-09-10T10:04:44.984 INFO:teuthology.orchestra.run.smithi077.stdout: "slow_ops_count": 0 2024-09-10T10:04:44.984 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:44.984 INFO:teuthology.orchestra.run.smithi077.stdout: "purge_queue": { 2024-09-10T10:04:44.984 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executing_ops": 0, 2024-09-10T10:04:44.984 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executing_ops_high_water": 0, 2024-09-10T10:04:44.984 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executing": 0, 2024-09-10T10:04:44.984 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executing_high_water": 0, 2024-09-10T10:04:44.984 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executed_ops": 0, 2024-09-10T10:04:44.984 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executed": 0, 2024-09-10T10:04:44.985 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_item_in_journal": 0 2024-09-10T10:04:44.985 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:44.985 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T10:04:44.985 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-09-10T10:04:44.985 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 104857600, 2024-09-10T10:04:44.985 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-09-10T10:04:44.985 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 2091, 2024-09-10T10:04:44.985 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 8409589, 2024-09-10T10:04:44.985 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-09-10T10:04:44.985 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 2091, 2024-09-10T10:04:44.985 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 0, 2024-09-10T10:04:44.985 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 0, 2024-09-10T10:04:44.985 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 2091, 2024-09-10T10:04:44.985 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 8409589, 2024-09-10T10:04:44.986 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-09-10T10:04:44.986 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-10T10:04:44.986 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-10T10:04:44.986 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-10T10:04:44.986 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:44.986 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:44.986 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-objecter_bytes": { 2024-09-10T10:04:44.986 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-09-10T10:04:44.986 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 104857600, 2024-09-10T10:04:44.986 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-09-10T10:04:44.986 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 0, 2024-09-10T10:04:44.986 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 0, 2024-09-10T10:04:44.986 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-09-10T10:04:44.986 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 0, 2024-09-10T10:04:44.986 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 1863, 2024-09-10T10:04:44.987 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 7503415, 2024-09-10T10:04:44.987 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 1863, 2024-09-10T10:04:44.987 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 7503415, 2024-09-10T10:04:44.987 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-09-10T10:04:44.987 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-10T10:04:44.987 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-10T10:04:44.987 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-10T10:04:44.987 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:44.987 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:44.987 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-objecter_ops": { 2024-09-10T10:04:44.987 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-09-10T10:04:44.987 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 1024, 2024-09-10T10:04:44.987 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-09-10T10:04:44.987 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 0, 2024-09-10T10:04:44.988 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 0, 2024-09-10T10:04:44.988 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-09-10T10:04:44.988 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 0, 2024-09-10T10:04:44.988 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 1863, 2024-09-10T10:04:44.988 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 1863, 2024-09-10T10:04:44.988 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 1863, 2024-09-10T10:04:44.988 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 1863, 2024-09-10T10:04:44.988 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-09-10T10:04:44.988 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-10T10:04:44.988 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-10T10:04:44.988 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-10T10:04:44.988 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:44.988 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:44.989 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-write_buf_throttle": { 2024-09-10T10:04:44.989 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-09-10T10:04:44.989 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 3758096384, 2024-09-10T10:04:44.989 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-09-10T10:04:44.989 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 0, 2024-09-10T10:04:44.989 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 0, 2024-09-10T10:04:44.989 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-09-10T10:04:44.989 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 0, 2024-09-10T10:04:44.989 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 0, 2024-09-10T10:04:44.989 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 0, 2024-09-10T10:04:44.989 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 0, 2024-09-10T10:04:44.990 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 0, 2024-09-10T10:04:44.990 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-09-10T10:04:44.990 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-10T10:04:44.990 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-10T10:04:44.990 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-10T10:04:44.990 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:44.990 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:44.990 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-write_buf_throttle-0x5559a5d2cbe0": { 2024-09-10T10:04:44.990 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-09-10T10:04:44.990 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 3758096384, 2024-09-10T10:04:44.990 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-09-10T10:04:44.990 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 0, 2024-09-10T10:04:44.990 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 0, 2024-09-10T10:04:44.990 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-09-10T10:04:44.991 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 0, 2024-09-10T10:04:44.991 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 0, 2024-09-10T10:04:44.991 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 0, 2024-09-10T10:04:44.991 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 0, 2024-09-10T10:04:44.991 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 0, 2024-09-10T10:04:44.991 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-09-10T10:04:44.991 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-10T10:04:44.991 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-10T10:04:44.991 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-10T10:04:44.991 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:44.991 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:44.991 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-10T10:04:45.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:04:44 smithi077 ceph-mon[30319]: pgmap v678: 129 pgs: 129 active+clean; 115 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1023 B/s wr, 12 op/s 2024-09-10T10:04:45.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:04:44 smithi134 ceph-mon[35674]: pgmap v678: 129 pgs: 129 active+clean; 115 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1023 B/s wr, 12 op/s 2024-09-10T10:04:45.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:04:44 smithi120 ceph-mon[34770]: pgmap v678: 129 pgs: 129 active+clean; 115 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1023 B/s wr, 12 op/s 2024-09-10T10:04:45.859 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-09-10T10:04:45.860 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-09-10T10:04:45.860 INFO:tasks.check_counter:Daemon mds.a mds.dir_split=0 2024-09-10T10:04:45.860 DEBUG:tasks.check_counter:Getting stats from d 2024-09-10T10:04:45.860 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph daemon mds.d perf dump 2024-09-10T10:04:46.230 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:04:46.810 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-10T10:04:46.810 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T10:04:46.810 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 213, 2024-09-10T10:04:46.810 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 210, 2024-09-10T10:04:46.810 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 476608, 2024-09-10T10:04:46.810 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 128704, 2024-09-10T10:04:46.811 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 4, 2024-09-10T10:04:46.811 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 0, 2024-09-10T10:04:46.811 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 0.045113962, 2024-09-10T10:04:46.811 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 0.022626246, 2024-09-10T10:04:46.811 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 0.022670834, 2024-09-10T10:04:46.811 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T10:04:46.811 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:04:46.811 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 209, 2024-09-10T10:04:46.811 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.015860080, 2024-09-10T10:04:46.811 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000075885 2024-09-10T10:04:46.811 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:46.811 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:04:46.811 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-10T10:04:46.811 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-10T10:04:46.812 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-10T10:04:46.812 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:46.812 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 476608, 2024-09-10T10:04:46.812 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 128704 2024-09-10T10:04:46.812 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:46.812 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T10:04:46.812 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 1, 2024-09-10T10:04:46.812 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 159, 2024-09-10T10:04:46.812 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 160, 2024-09-10T10:04:46.812 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 106464, 2024-09-10T10:04:46.812 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 3, 2024-09-10T10:04:46.812 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 2, 2024-09-10T10:04:46.813 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 0.017110168, 2024-09-10T10:04:46.813 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 0.012982569, 2024-09-10T10:04:46.813 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 0.002201625, 2024-09-10T10:04:46.813 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T10:04:46.813 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:04:46.813 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 157, 2024-09-10T10:04:46.813 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.009351848, 2024-09-10T10:04:46.813 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000059565 2024-09-10T10:04:46.813 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:46.813 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:04:46.813 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-10T10:04:46.813 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-10T10:04:46.813 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-10T10:04:46.813 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:46.814 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-10T10:04:46.814 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 106464 2024-09-10T10:04:46.814 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:46.814 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T10:04:46.814 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 4, 2024-09-10T10:04:46.814 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 4, 2024-09-10T10:04:46.814 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 2992, 2024-09-10T10:04:46.814 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 448, 2024-09-10T10:04:46.814 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 2, 2024-09-10T10:04:46.814 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 1, 2024-09-10T10:04:46.814 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 0.003574954, 2024-09-10T10:04:46.814 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 0.000073042, 2024-09-10T10:04:46.814 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 0.003451018, 2024-09-10T10:04:46.814 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T10:04:46.814 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:04:46.814 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 3, 2024-09-10T10:04:46.815 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.005880547, 2024-09-10T10:04:46.815 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.001960182 2024-09-10T10:04:46.815 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:46.815 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:04:46.815 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-10T10:04:46.815 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-10T10:04:46.815 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-10T10:04:46.815 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:46.815 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-09-10T10:04:46.815 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 448 2024-09-10T10:04:46.815 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:46.815 INFO:teuthology.orchestra.run.smithi077.stdout: "cct": { 2024-09-10T10:04:46.815 INFO:teuthology.orchestra.run.smithi077.stdout: "total_workers": 0, 2024-09-10T10:04:46.815 INFO:teuthology.orchestra.run.smithi077.stdout: "unhealthy_workers": 0 2024-09-10T10:04:46.815 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:46.815 INFO:teuthology.orchestra.run.smithi077.stdout: "mempool": { 2024-09-10T10:04:46.816 INFO:teuthology.orchestra.run.smithi077.stdout: "bloom_filter_bytes": 0, 2024-09-10T10:04:46.816 INFO:teuthology.orchestra.run.smithi077.stdout: "bloom_filter_items": 0, 2024-09-10T10:04:46.816 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T10:04:46.816 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_alloc_items": 0, 2024-09-10T10:04:46.816 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T10:04:46.816 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_data_items": 0, 2024-09-10T10:04:46.816 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T10:04:46.816 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T10:04:46.816 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T10:04:46.816 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T10:04:46.816 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T10:04:46.816 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_other_items": 0, 2024-09-10T10:04:46.816 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T10:04:46.816 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T10:04:46.816 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_extent_bytes": 0, 2024-09-10T10:04:46.817 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_extent_items": 0, 2024-09-10T10:04:46.817 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_blob_bytes": 0, 2024-09-10T10:04:46.817 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_blob_items": 0, 2024-09-10T10:04:46.817 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T10:04:46.817 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T10:04:46.817 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T10:04:46.817 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T10:04:46.817 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T10:04:46.817 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_fsck_items": 0, 2024-09-10T10:04:46.817 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_txc_bytes": 0, 2024-09-10T10:04:46.817 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_txc_items": 0, 2024-09-10T10:04:46.817 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T10:04:46.818 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T10:04:46.818 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_bytes": 0, 2024-09-10T10:04:46.818 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_items": 0, 2024-09-10T10:04:46.818 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_bytes": 0, 2024-09-10T10:04:46.818 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_items": 0, 2024-09-10T10:04:46.818 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T10:04:46.818 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_reader_items": 0, 2024-09-10T10:04:46.818 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T10:04:46.818 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_writer_items": 0, 2024-09-10T10:04:46.818 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_anon_bytes": 148967, 2024-09-10T10:04:46.818 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_anon_items": 29, 2024-09-10T10:04:46.818 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_meta_bytes": 0, 2024-09-10T10:04:46.818 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_meta_items": 0, 2024-09-10T10:04:46.818 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_bytes": 0, 2024-09-10T10:04:46.818 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_items": 0, 2024-09-10T10:04:46.819 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_mapbl_bytes": 0, 2024-09-10T10:04:46.819 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_mapbl_items": 0, 2024-09-10T10:04:46.819 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_pglog_bytes": 0, 2024-09-10T10:04:46.819 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_pglog_items": 0, 2024-09-10T10:04:46.819 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_bytes": 0, 2024-09-10T10:04:46.819 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_items": 0, 2024-09-10T10:04:46.819 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T10:04:46.819 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_mapping_items": 0, 2024-09-10T10:04:46.819 INFO:teuthology.orchestra.run.smithi077.stdout: "pgmap_bytes": 0, 2024-09-10T10:04:46.819 INFO:teuthology.orchestra.run.smithi077.stdout: "pgmap_items": 0, 2024-09-10T10:04:46.819 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_co_bytes": 928, 2024-09-10T10:04:46.820 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_co_items": 2, 2024-09-10T10:04:46.820 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_1_bytes": 0, 2024-09-10T10:04:46.820 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_1_items": 0, 2024-09-10T10:04:46.820 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_2_bytes": 0, 2024-09-10T10:04:46.820 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_2_items": 0 2024-09-10T10:04:46.820 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:46.820 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T10:04:46.820 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-09-10T10:04:46.820 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 104857600, 2024-09-10T10:04:46.820 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-09-10T10:04:46.820 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 214, 2024-09-10T10:04:46.821 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 446508, 2024-09-10T10:04:46.821 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-09-10T10:04:46.821 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 214, 2024-09-10T10:04:46.821 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 0, 2024-09-10T10:04:46.821 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 0, 2024-09-10T10:04:46.821 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 214, 2024-09-10T10:04:46.821 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 446508, 2024-09-10T10:04:46.821 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-09-10T10:04:46.821 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-10T10:04:46.821 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-10T10:04:46.821 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-10T10:04:46.821 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:46.821 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:46.821 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-10T10:04:47.226 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:04:46 smithi120 ceph-mon[34770]: pgmap v679: 129 pgs: 129 active+clean; 115 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 9 op/s 2024-09-10T10:04:47.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:04:46 smithi134 ceph-mon[35674]: pgmap v679: 129 pgs: 129 active+clean; 115 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 9 op/s 2024-09-10T10:04:47.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:04:46 smithi077 ceph-mon[30319]: pgmap v679: 129 pgs: 129 active+clean; 115 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 9 op/s 2024-09-10T10:04:47.570 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-09-10T10:04:47.571 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-09-10T10:04:47.571 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2024-09-10T10:04:47.571 DEBUG:tasks.check_counter:Getting stats from g 2024-09-10T10:04:47.571 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph daemon mds.g perf dump 2024-09-10T10:04:47.938 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:04:48.519 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-10T10:04:48.519 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T10:04:48.519 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 919, 2024-09-10T10:04:48.519 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 904, 2024-09-10T10:04:48.519 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 806144, 2024-09-10T10:04:48.519 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 347388, 2024-09-10T10:04:48.519 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 5, 2024-09-10T10:04:48.520 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 1, 2024-09-10T10:04:48.520 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 0.139320099, 2024-09-10T10:04:48.520 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 0.069742783, 2024-09-10T10:04:48.520 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 0.041666171, 2024-09-10T10:04:48.520 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.021114212, 2024-09-10T10:04:48.520 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:04:48.520 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 903, 2024-09-10T10:04:48.520 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.068931274, 2024-09-10T10:04:48.520 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000076335 2024-09-10T10:04:48.520 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:48.520 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:04:48.520 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-10T10:04:48.520 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-10T10:04:48.521 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-10T10:04:48.521 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:48.521 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 574304, 2024-09-10T10:04:48.521 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 127968 2024-09-10T10:04:48.521 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:48.521 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T10:04:48.521 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 1319, 2024-09-10T10:04:48.521 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 1474, 2024-09-10T10:04:48.521 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 6625464, 2024-09-10T10:04:48.521 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 757405, 2024-09-10T10:04:48.521 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 5, 2024-09-10T10:04:48.521 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 4, 2024-09-10T10:04:48.522 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 0.139044485, 2024-09-10T10:04:48.522 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 0.047389292, 2024-09-10T10:04:48.522 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 0.045732628, 2024-09-10T10:04:48.522 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.036330705, 2024-09-10T10:04:48.522 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:04:48.522 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 1472, 2024-09-10T10:04:48.522 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.074023359, 2024-09-10T10:04:48.522 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000050287 2024-09-10T10:04:48.522 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:48.522 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:04:48.522 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-10T10:04:48.522 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-10T10:04:48.523 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-10T10:04:48.523 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:48.523 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-10T10:04:48.523 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 338224 2024-09-10T10:04:48.523 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:48.523 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T10:04:48.523 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 8, 2024-09-10T10:04:48.523 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 8, 2024-09-10T10:04:48.523 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 4621, 2024-09-10T10:04:48.523 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 1732, 2024-09-10T10:04:48.523 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 5, 2024-09-10T10:04:48.523 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 4, 2024-09-10T10:04:48.523 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 0.005331055, 2024-09-10T10:04:48.524 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 0.000192099, 2024-09-10T10:04:48.524 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 0.004882595, 2024-09-10T10:04:48.524 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.000117754, 2024-09-10T10:04:48.524 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:04:48.524 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 7, 2024-09-10T10:04:48.524 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.009691823, 2024-09-10T10:04:48.524 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.001384546 2024-09-10T10:04:48.524 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:48.524 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:04:48.524 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-10T10:04:48.524 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-10T10:04:48.524 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-10T10:04:48.524 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:48.525 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-09-10T10:04:48.525 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 448 2024-09-10T10:04:48.525 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:48.525 INFO:teuthology.orchestra.run.smithi077.stdout: "cct": { 2024-09-10T10:04:48.525 INFO:teuthology.orchestra.run.smithi077.stdout: "total_workers": 1, 2024-09-10T10:04:48.525 INFO:teuthology.orchestra.run.smithi077.stdout: "unhealthy_workers": 0 2024-09-10T10:04:48.525 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:48.525 INFO:teuthology.orchestra.run.smithi077.stdout: "finisher-MDSRank": { 2024-09-10T10:04:48.525 INFO:teuthology.orchestra.run.smithi077.stdout: "queue_len": 0, 2024-09-10T10:04:48.525 INFO:teuthology.orchestra.run.smithi077.stdout: "complete_latency": { 2024-09-10T10:04:48.525 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 3362, 2024-09-10T10:04:48.525 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.088417574, 2024-09-10T10:04:48.525 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000026299 2024-09-10T10:04:48.526 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:48.526 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:48.526 INFO:teuthology.orchestra.run.smithi077.stdout: "finisher-PurgeQueue": { 2024-09-10T10:04:48.526 INFO:teuthology.orchestra.run.smithi077.stdout: "queue_len": 0, 2024-09-10T10:04:48.526 INFO:teuthology.orchestra.run.smithi077.stdout: "complete_latency": { 2024-09-10T10:04:48.526 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-10T10:04:48.526 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-10T10:04:48.526 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-10T10:04:48.526 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:48.526 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:48.526 INFO:teuthology.orchestra.run.smithi077.stdout: "mds": { 2024-09-10T10:04:48.526 INFO:teuthology.orchestra.run.smithi077.stdout: "request": 0, 2024-09-10T10:04:48.526 INFO:teuthology.orchestra.run.smithi077.stdout: "reply": 0, 2024-09-10T10:04:48.526 INFO:teuthology.orchestra.run.smithi077.stdout: "reply_latency": { 2024-09-10T10:04:48.527 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-10T10:04:48.527 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-10T10:04:48.527 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-10T10:04:48.527 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:48.527 INFO:teuthology.orchestra.run.smithi077.stdout: "slow_reply": 0, 2024-09-10T10:04:48.527 INFO:teuthology.orchestra.run.smithi077.stdout: "forward": 0, 2024-09-10T10:04:48.527 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_fetch_complete": 0, 2024-09-10T10:04:48.527 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_fetch_keys": 0, 2024-09-10T10:04:48.527 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_commit": 0, 2024-09-10T10:04:48.527 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_split": 0, 2024-09-10T10:04:48.527 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_merge": 0, 2024-09-10T10:04:48.527 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes": 1149, 2024-09-10T10:04:48.527 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_top": 0, 2024-09-10T10:04:48.527 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_bottom": 1149, 2024-09-10T10:04:48.528 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_pin_tail": 0, 2024-09-10T10:04:48.528 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_pinned": 1149, 2024-09-10T10:04:48.528 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_expired": 0, 2024-09-10T10:04:48.528 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_with_caps": 0, 2024-09-10T10:04:48.528 INFO:teuthology.orchestra.run.smithi077.stdout: "caps": 0, 2024-09-10T10:04:48.528 INFO:teuthology.orchestra.run.smithi077.stdout: "subtrees": 1, 2024-09-10T10:04:48.528 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse": 0, 2024-09-10T10:04:48.528 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_hit": 0, 2024-09-10T10:04:48.528 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_forward": 0, 2024-09-10T10:04:48.528 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_discover": 0, 2024-09-10T10:04:48.528 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_dir_fetch": 0, 2024-09-10T10:04:48.528 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_remote_ino": 0, 2024-09-10T10:04:48.528 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_lock": 0, 2024-09-10T10:04:48.528 INFO:teuthology.orchestra.run.smithi077.stdout: "load_cent": 0, 2024-09-10T10:04:48.528 INFO:teuthology.orchestra.run.smithi077.stdout: "q": 0, 2024-09-10T10:04:48.529 INFO:teuthology.orchestra.run.smithi077.stdout: "exported": 0, 2024-09-10T10:04:48.529 INFO:teuthology.orchestra.run.smithi077.stdout: "exported_inodes": 0, 2024-09-10T10:04:48.529 INFO:teuthology.orchestra.run.smithi077.stdout: "imported": 0, 2024-09-10T10:04:48.529 INFO:teuthology.orchestra.run.smithi077.stdout: "imported_inodes": 0, 2024-09-10T10:04:48.529 INFO:teuthology.orchestra.run.smithi077.stdout: "openino_dir_fetch": 0, 2024-09-10T10:04:48.529 INFO:teuthology.orchestra.run.smithi077.stdout: "openino_backtrace_fetch": 0, 2024-09-10T10:04:48.529 INFO:teuthology.orchestra.run.smithi077.stdout: "openino_peer_discover": 0, 2024-09-10T10:04:48.529 INFO:teuthology.orchestra.run.smithi077.stdout: "root_rfiles": 0, 2024-09-10T10:04:48.529 INFO:teuthology.orchestra.run.smithi077.stdout: "root_rbytes": 0, 2024-09-10T10:04:48.529 INFO:teuthology.orchestra.run.smithi077.stdout: "root_rsnaps": 0, 2024-09-10T10:04:48.529 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_backtrace_fetch": 0, 2024-09-10T10:04:48.529 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_set_tag": 0, 2024-09-10T10:04:48.529 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T10:04:48.529 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_inotable_repaired": 0, 2024-09-10T10:04:48.530 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_dir_inodes": 0, 2024-09-10T10:04:48.530 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_dir_base_inodes": 0, 2024-09-10T10:04:48.530 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_dirfrag_rstats": 0, 2024-09-10T10:04:48.530 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_file_inodes": 0, 2024-09-10T10:04:48.530 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_inode_file_caps": 0, 2024-09-10T10:04:48.530 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_revoke": 0, 2024-09-10T10:04:48.530 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_grant": 0, 2024-09-10T10:04:48.530 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T10:04:48.530 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T10:04:48.530 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T10:04:48.530 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_caps": 0, 2024-09-10T10:04:48.530 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_caps_dirty": 0, 2024-09-10T10:04:48.531 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_cap_release": 0, 2024-09-10T10:04:48.531 INFO:teuthology.orchestra.run.smithi077.stdout: "process_request_cap_release": 0 2024-09-10T10:04:48.531 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:48.531 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_cache": { 2024-09-10T10:04:48.531 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_update": 0, 2024-09-10T10:04:48.531 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_update_receipt": 0, 2024-09-10T10:04:48.531 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_try_discover": 0, 2024-09-10T10:04:48.531 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_send_discover": 0, 2024-09-10T10:04:48.531 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_handle_discover": 0, 2024-09-10T10:04:48.531 INFO:teuthology.orchestra.run.smithi077.stdout: "num_strays": 0, 2024-09-10T10:04:48.531 INFO:teuthology.orchestra.run.smithi077.stdout: "num_strays_delayed": 0, 2024-09-10T10:04:48.531 INFO:teuthology.orchestra.run.smithi077.stdout: "num_strays_enqueuing": 0, 2024-09-10T10:04:48.531 INFO:teuthology.orchestra.run.smithi077.stdout: "strays_created": 0, 2024-09-10T10:04:48.531 INFO:teuthology.orchestra.run.smithi077.stdout: "strays_enqueued": 0, 2024-09-10T10:04:48.532 INFO:teuthology.orchestra.run.smithi077.stdout: "strays_reintegrated": 0, 2024-09-10T10:04:48.532 INFO:teuthology.orchestra.run.smithi077.stdout: "strays_migrated": 0, 2024-09-10T10:04:48.532 INFO:teuthology.orchestra.run.smithi077.stdout: "num_recovering_processing": 0, 2024-09-10T10:04:48.532 INFO:teuthology.orchestra.run.smithi077.stdout: "num_recovering_enqueued": 0, 2024-09-10T10:04:48.532 INFO:teuthology.orchestra.run.smithi077.stdout: "num_recovering_prioritized": 0, 2024-09-10T10:04:48.532 INFO:teuthology.orchestra.run.smithi077.stdout: "recovery_started": 0, 2024-09-10T10:04:48.532 INFO:teuthology.orchestra.run.smithi077.stdout: "recovery_completed": 0, 2024-09-10T10:04:48.532 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_quiesce_path": 0, 2024-09-10T10:04:48.532 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_quiesce_inode": 0, 2024-09-10T10:04:48.532 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_enqueue_scrub": 0, 2024-09-10T10:04:48.532 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_exportdir": 0, 2024-09-10T10:04:48.532 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_flush": 0, 2024-09-10T10:04:48.532 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_fragmentdir": 0, 2024-09-10T10:04:48.532 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_fragstats": 0, 2024-09-10T10:04:48.532 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_inodestats": 0 2024-09-10T10:04:48.533 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:48.533 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_log": { 2024-09-10T10:04:48.533 INFO:teuthology.orchestra.run.smithi077.stdout: "evlrg": 0, 2024-09-10T10:04:48.533 INFO:teuthology.orchestra.run.smithi077.stdout: "evadd": 0, 2024-09-10T10:04:48.533 INFO:teuthology.orchestra.run.smithi077.stdout: "evex": 0, 2024-09-10T10:04:48.533 INFO:teuthology.orchestra.run.smithi077.stdout: "evtrm": 0, 2024-09-10T10:04:48.533 INFO:teuthology.orchestra.run.smithi077.stdout: "ev": 3931, 2024-09-10T10:04:48.533 INFO:teuthology.orchestra.run.smithi077.stdout: "evexg": 0, 2024-09-10T10:04:48.533 INFO:teuthology.orchestra.run.smithi077.stdout: "evexd": 0, 2024-09-10T10:04:48.533 INFO:teuthology.orchestra.run.smithi077.stdout: "segadd": 0, 2024-09-10T10:04:48.533 INFO:teuthology.orchestra.run.smithi077.stdout: "segex": 0, 2024-09-10T10:04:48.533 INFO:teuthology.orchestra.run.smithi077.stdout: "segtrm": 0, 2024-09-10T10:04:48.533 INFO:teuthology.orchestra.run.smithi077.stdout: "seg": 6, 2024-09-10T10:04:48.533 INFO:teuthology.orchestra.run.smithi077.stdout: "segmjr": 2, 2024-09-10T10:04:48.533 INFO:teuthology.orchestra.run.smithi077.stdout: "segexg": 0, 2024-09-10T10:04:48.533 INFO:teuthology.orchestra.run.smithi077.stdout: "segexd": 0, 2024-09-10T10:04:48.534 INFO:teuthology.orchestra.run.smithi077.stdout: "expos": 4194304, 2024-09-10T10:04:48.534 INFO:teuthology.orchestra.run.smithi077.stdout: "wrpos": 10484084, 2024-09-10T10:04:48.534 INFO:teuthology.orchestra.run.smithi077.stdout: "rdpos": 10482946, 2024-09-10T10:04:48.534 INFO:teuthology.orchestra.run.smithi077.stdout: "jlat": { 2024-09-10T10:04:48.534 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-10T10:04:48.534 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-10T10:04:48.534 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-10T10:04:48.534 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:48.534 INFO:teuthology.orchestra.run.smithi077.stdout: "replayed": 3931 2024-09-10T10:04:48.534 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:48.534 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_mem": { 2024-09-10T10:04:48.534 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 7, 2024-09-10T10:04:48.534 INFO:teuthology.orchestra.run.smithi077.stdout: "ino+": 1172, 2024-09-10T10:04:48.535 INFO:teuthology.orchestra.run.smithi077.stdout: "ino-": 1165, 2024-09-10T10:04:48.535 INFO:teuthology.orchestra.run.smithi077.stdout: "dir": 5, 2024-09-10T10:04:48.535 INFO:teuthology.orchestra.run.smithi077.stdout: "dir+": 75, 2024-09-10T10:04:48.535 INFO:teuthology.orchestra.run.smithi077.stdout: "dir-": 70, 2024-09-10T10:04:48.535 INFO:teuthology.orchestra.run.smithi077.stdout: "dn": 1149, 2024-09-10T10:04:48.535 INFO:teuthology.orchestra.run.smithi077.stdout: "dn+": 2341, 2024-09-10T10:04:48.535 INFO:teuthology.orchestra.run.smithi077.stdout: "dn-": 1192, 2024-09-10T10:04:48.535 INFO:teuthology.orchestra.run.smithi077.stdout: "cap": 0, 2024-09-10T10:04:48.535 INFO:teuthology.orchestra.run.smithi077.stdout: "cap+": 0, 2024-09-10T10:04:48.535 INFO:teuthology.orchestra.run.smithi077.stdout: "cap-": 0, 2024-09-10T10:04:48.535 INFO:teuthology.orchestra.run.smithi077.stdout: "rss": 51484, 2024-09-10T10:04:48.535 INFO:teuthology.orchestra.run.smithi077.stdout: "heap": 215328 2024-09-10T10:04:48.535 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:48.535 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_server": { 2024-09-10T10:04:48.535 INFO:teuthology.orchestra.run.smithi077.stdout: "dispatch_client_request": 0, 2024-09-10T10:04:48.536 INFO:teuthology.orchestra.run.smithi077.stdout: "dispatch_server_request": 0, 2024-09-10T10:04:48.536 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_request": 0, 2024-09-10T10:04:48.536 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_session": 0, 2024-09-10T10:04:48.536 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_peer_request": 0, 2024-09-10T10:04:48.536 INFO:teuthology.orchestra.run.smithi077.stdout: "req_create_latency": { 2024-09-10T10:04:48.536 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-10T10:04:48.536 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-10T10:04:48.536 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-10T10:04:48.536 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:48.536 INFO:teuthology.orchestra.run.smithi077.stdout: "req_getattr_latency": { 2024-09-10T10:04:48.536 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-10T10:04:48.536 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-10T10:04:48.536 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-10T10:04:48.536 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:48.537 INFO:teuthology.orchestra.run.smithi077.stdout: "req_getfilelock_latency": { 2024-09-10T10:04:48.537 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-10T10:04:48.537 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-10T10:04:48.537 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-10T10:04:48.537 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:48.537 INFO:teuthology.orchestra.run.smithi077.stdout: "req_link_latency": { 2024-09-10T10:04:48.537 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-10T10:04:48.537 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-10T10:04:48.537 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-10T10:04:48.537 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:48.537 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookup_latency": { 2024-09-10T10:04:48.537 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-10T10:04:48.537 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-10T10:04:48.538 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-10T10:04:48.538 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:48.538 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookuphash_latency": { 2024-09-10T10:04:48.538 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-10T10:04:48.538 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-10T10:04:48.538 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-10T10:04:48.538 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:48.538 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookupino_latency": { 2024-09-10T10:04:48.538 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-10T10:04:48.538 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-10T10:04:48.538 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-10T10:04:48.538 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:48.538 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookupname_latency": { 2024-09-10T10:04:48.538 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-10T10:04:48.539 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-10T10:04:48.539 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-10T10:04:48.539 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:48.539 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookupparent_latency": { 2024-09-10T10:04:48.539 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-10T10:04:48.539 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-10T10:04:48.539 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-10T10:04:48.539 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:48.539 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookupsnap_latency": { 2024-09-10T10:04:48.539 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-10T10:04:48.539 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-10T10:04:48.539 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-10T10:04:48.540 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:48.540 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lssnap_latency": { 2024-09-10T10:04:48.540 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-10T10:04:48.540 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-10T10:04:48.540 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-10T10:04:48.540 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:48.540 INFO:teuthology.orchestra.run.smithi077.stdout: "req_mkdir_latency": { 2024-09-10T10:04:48.540 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-10T10:04:48.541 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-10T10:04:48.541 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-10T10:04:48.541 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:48.541 INFO:teuthology.orchestra.run.smithi077.stdout: "req_mknod_latency": { 2024-09-10T10:04:48.541 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-10T10:04:48.541 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-10T10:04:48.541 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-10T10:04:48.541 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:48.541 INFO:teuthology.orchestra.run.smithi077.stdout: "req_mksnap_latency": { 2024-09-10T10:04:48.541 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-10T10:04:48.541 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-10T10:04:48.541 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-10T10:04:48.541 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:48.542 INFO:teuthology.orchestra.run.smithi077.stdout: "req_open_latency": { 2024-09-10T10:04:48.542 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-10T10:04:48.542 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-10T10:04:48.542 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-10T10:04:48.542 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:48.542 INFO:teuthology.orchestra.run.smithi077.stdout: "req_readdir_latency": { 2024-09-10T10:04:48.542 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-10T10:04:48.542 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-10T10:04:48.542 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-10T10:04:48.542 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:48.542 INFO:teuthology.orchestra.run.smithi077.stdout: "req_rename_latency": { 2024-09-10T10:04:48.542 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-10T10:04:48.542 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-10T10:04:48.542 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-10T10:04:48.543 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:48.543 INFO:teuthology.orchestra.run.smithi077.stdout: "req_renamesnap_latency": { 2024-09-10T10:04:48.543 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-10T10:04:48.543 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-10T10:04:48.543 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-10T10:04:48.543 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:48.543 INFO:teuthology.orchestra.run.smithi077.stdout: "req_snapdiff_latency": { 2024-09-10T10:04:48.543 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-10T10:04:48.543 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-10T10:04:48.543 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-10T10:04:48.543 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:48.543 INFO:teuthology.orchestra.run.smithi077.stdout: "req_rmdir_latency": { 2024-09-10T10:04:48.543 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-10T10:04:48.543 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-10T10:04:48.544 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-10T10:04:48.544 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:48.544 INFO:teuthology.orchestra.run.smithi077.stdout: "req_rmsnap_latency": { 2024-09-10T10:04:48.544 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-10T10:04:48.544 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-10T10:04:48.544 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-10T10:04:48.544 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:48.544 INFO:teuthology.orchestra.run.smithi077.stdout: "req_rmxattr_latency": { 2024-09-10T10:04:48.544 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-10T10:04:48.544 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-10T10:04:48.544 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-10T10:04:48.544 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:48.544 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setattr_latency": { 2024-09-10T10:04:48.544 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-10T10:04:48.544 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-10T10:04:48.545 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-10T10:04:48.545 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:48.545 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setdirlayout_latency": { 2024-09-10T10:04:48.545 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-10T10:04:48.545 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-10T10:04:48.545 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-10T10:04:48.545 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:48.545 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setfilelock_latency": { 2024-09-10T10:04:48.545 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-10T10:04:48.545 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-10T10:04:48.545 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-10T10:04:48.545 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:48.545 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setlayout_latency": { 2024-09-10T10:04:48.545 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-10T10:04:48.546 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-10T10:04:48.546 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-10T10:04:48.546 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:48.546 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setxattr_latency": { 2024-09-10T10:04:48.546 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-10T10:04:48.546 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-10T10:04:48.546 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-10T10:04:48.546 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:48.546 INFO:teuthology.orchestra.run.smithi077.stdout: "req_symlink_latency": { 2024-09-10T10:04:48.546 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-10T10:04:48.546 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-10T10:04:48.546 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-10T10:04:48.546 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:48.547 INFO:teuthology.orchestra.run.smithi077.stdout: "req_unlink_latency": { 2024-09-10T10:04:48.547 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-10T10:04:48.547 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-10T10:04:48.547 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-10T10:04:48.547 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:48.547 INFO:teuthology.orchestra.run.smithi077.stdout: "cap_revoke_eviction": 0, 2024-09-10T10:04:48.547 INFO:teuthology.orchestra.run.smithi077.stdout: "cap_acquisition_throttle": 0, 2024-09-10T10:04:48.547 INFO:teuthology.orchestra.run.smithi077.stdout: "req_getvxattr_latency": { 2024-09-10T10:04:48.547 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-10T10:04:48.547 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-10T10:04:48.547 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-10T10:04:48.547 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:48.547 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:48.548 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_sessions": { 2024-09-10T10:04:48.548 INFO:teuthology.orchestra.run.smithi077.stdout: "session_count": 1, 2024-09-10T10:04:48.548 INFO:teuthology.orchestra.run.smithi077.stdout: "session_add": 1, 2024-09-10T10:04:48.548 INFO:teuthology.orchestra.run.smithi077.stdout: "session_remove": 0, 2024-09-10T10:04:48.548 INFO:teuthology.orchestra.run.smithi077.stdout: "sessions_open": 1, 2024-09-10T10:04:48.548 INFO:teuthology.orchestra.run.smithi077.stdout: "sessions_stale": 0, 2024-09-10T10:04:48.548 INFO:teuthology.orchestra.run.smithi077.stdout: "total_load": 0, 2024-09-10T10:04:48.548 INFO:teuthology.orchestra.run.smithi077.stdout: "average_load": 0, 2024-09-10T10:04:48.548 INFO:teuthology.orchestra.run.smithi077.stdout: "avg_session_uptime": 1713, 2024-09-10T10:04:48.548 INFO:teuthology.orchestra.run.smithi077.stdout: "mdthresh_evicted": 0 2024-09-10T10:04:48.548 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:48.548 INFO:teuthology.orchestra.run.smithi077.stdout: "mempool": { 2024-09-10T10:04:48.548 INFO:teuthology.orchestra.run.smithi077.stdout: "bloom_filter_bytes": 0, 2024-09-10T10:04:48.548 INFO:teuthology.orchestra.run.smithi077.stdout: "bloom_filter_items": 0, 2024-09-10T10:04:48.549 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T10:04:48.549 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_alloc_items": 0, 2024-09-10T10:04:48.549 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T10:04:48.549 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_data_items": 0, 2024-09-10T10:04:48.549 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T10:04:48.549 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T10:04:48.549 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T10:04:48.549 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T10:04:48.549 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T10:04:48.549 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_other_items": 0, 2024-09-10T10:04:48.549 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T10:04:48.549 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T10:04:48.549 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_extent_bytes": 0, 2024-09-10T10:04:48.549 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_extent_items": 0, 2024-09-10T10:04:48.550 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_blob_bytes": 0, 2024-09-10T10:04:48.550 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_blob_items": 0, 2024-09-10T10:04:48.550 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T10:04:48.550 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T10:04:48.550 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T10:04:48.550 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T10:04:48.550 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T10:04:48.550 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_fsck_items": 0, 2024-09-10T10:04:48.550 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_txc_bytes": 0, 2024-09-10T10:04:48.550 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_txc_items": 0, 2024-09-10T10:04:48.550 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T10:04:48.551 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T10:04:48.551 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_bytes": 0, 2024-09-10T10:04:48.551 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_items": 0, 2024-09-10T10:04:48.551 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_bytes": 0, 2024-09-10T10:04:48.551 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_items": 0, 2024-09-10T10:04:48.551 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T10:04:48.551 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_reader_items": 0, 2024-09-10T10:04:48.551 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T10:04:48.551 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_writer_items": 0, 2024-09-10T10:04:48.552 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_anon_bytes": 197463, 2024-09-10T10:04:48.552 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_anon_items": 54, 2024-09-10T10:04:48.552 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_meta_bytes": 0, 2024-09-10T10:04:48.552 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_meta_items": 0, 2024-09-10T10:04:48.552 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_bytes": 0, 2024-09-10T10:04:48.552 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_items": 0, 2024-09-10T10:04:48.552 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_mapbl_bytes": 0, 2024-09-10T10:04:48.552 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_mapbl_items": 0, 2024-09-10T10:04:48.552 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_pglog_bytes": 0, 2024-09-10T10:04:48.552 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_pglog_items": 0, 2024-09-10T10:04:48.552 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_bytes": 9144, 2024-09-10T10:04:48.552 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_items": 172, 2024-09-10T10:04:48.552 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T10:04:48.552 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_mapping_items": 0, 2024-09-10T10:04:48.553 INFO:teuthology.orchestra.run.smithi077.stdout: "pgmap_bytes": 0, 2024-09-10T10:04:48.553 INFO:teuthology.orchestra.run.smithi077.stdout: "pgmap_items": 0, 2024-09-10T10:04:48.553 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_co_bytes": 831464, 2024-09-10T10:04:48.553 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_co_items": 4658, 2024-09-10T10:04:48.553 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_1_bytes": 0, 2024-09-10T10:04:48.553 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_1_items": 0, 2024-09-10T10:04:48.553 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_2_bytes": 0, 2024-09-10T10:04:48.553 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_2_items": 0 2024-09-10T10:04:48.553 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:48.553 INFO:teuthology.orchestra.run.smithi077.stdout: "objecter": { 2024-09-10T10:04:48.553 INFO:teuthology.orchestra.run.smithi077.stdout: "op_active": 0, 2024-09-10T10:04:48.553 INFO:teuthology.orchestra.run.smithi077.stdout: "op_laggy": 0, 2024-09-10T10:04:48.554 INFO:teuthology.orchestra.run.smithi077.stdout: "op_send": 2012, 2024-09-10T10:04:48.554 INFO:teuthology.orchestra.run.smithi077.stdout: "op_send_bytes": 21, 2024-09-10T10:04:48.554 INFO:teuthology.orchestra.run.smithi077.stdout: "op_resend": 3, 2024-09-10T10:04:48.554 INFO:teuthology.orchestra.run.smithi077.stdout: "op_reply": 2012, 2024-09-10T10:04:48.554 INFO:teuthology.orchestra.run.smithi077.stdout: "op_latency": { 2024-09-10T10:04:48.554 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 2012, 2024-09-10T10:04:48.554 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 2.036957334, 2024-09-10T10:04:48.554 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.001012404 2024-09-10T10:04:48.554 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:48.554 INFO:teuthology.orchestra.run.smithi077.stdout: "op_inflight": 0, 2024-09-10T10:04:48.554 INFO:teuthology.orchestra.run.smithi077.stdout: "oplen_avg": { 2024-09-10T10:04:48.554 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 2012, 2024-09-10T10:04:48.554 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 2013 2024-09-10T10:04:48.554 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:48.554 INFO:teuthology.orchestra.run.smithi077.stdout: "op": 2012, 2024-09-10T10:04:48.554 INFO:teuthology.orchestra.run.smithi077.stdout: "op_r": 2012, 2024-09-10T10:04:48.555 INFO:teuthology.orchestra.run.smithi077.stdout: "op_w": 0, 2024-09-10T10:04:48.555 INFO:teuthology.orchestra.run.smithi077.stdout: "op_rmw": 0, 2024-09-10T10:04:48.555 INFO:teuthology.orchestra.run.smithi077.stdout: "op_pg": 0, 2024-09-10T10:04:48.555 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_stat": 1264, 2024-09-10T10:04:48.555 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_create": 0, 2024-09-10T10:04:48.555 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_read": 746, 2024-09-10T10:04:48.555 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_write": 0, 2024-09-10T10:04:48.555 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_writefull": 0, 2024-09-10T10:04:48.555 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_writesame": 0, 2024-09-10T10:04:48.555 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_append": 0, 2024-09-10T10:04:48.555 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_zero": 0, 2024-09-10T10:04:48.555 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_truncate": 0, 2024-09-10T10:04:48.555 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_delete": 0, 2024-09-10T10:04:48.555 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_mapext": 0, 2024-09-10T10:04:48.555 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_sparse_read": 0, 2024-09-10T10:04:48.556 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_clonerange": 0, 2024-09-10T10:04:48.556 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_getxattr": 1, 2024-09-10T10:04:48.556 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_setxattr": 0, 2024-09-10T10:04:48.556 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_cmpxattr": 0, 2024-09-10T10:04:48.556 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_rmxattr": 0, 2024-09-10T10:04:48.556 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_resetxattrs": 0, 2024-09-10T10:04:48.556 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_call": 0, 2024-09-10T10:04:48.556 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_watch": 0, 2024-09-10T10:04:48.556 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_notify": 0, 2024-09-10T10:04:48.556 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T10:04:48.556 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_pgls": 0, 2024-09-10T10:04:48.556 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_pgls_filter": 0, 2024-09-10T10:04:48.556 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_other": 0, 2024-09-10T10:04:48.556 INFO:teuthology.orchestra.run.smithi077.stdout: "linger_active": 0, 2024-09-10T10:04:48.557 INFO:teuthology.orchestra.run.smithi077.stdout: "linger_send": 0, 2024-09-10T10:04:48.557 INFO:teuthology.orchestra.run.smithi077.stdout: "linger_resend": 0, 2024-09-10T10:04:48.557 INFO:teuthology.orchestra.run.smithi077.stdout: "linger_ping": 0, 2024-09-10T10:04:48.557 INFO:teuthology.orchestra.run.smithi077.stdout: "poolop_active": 0, 2024-09-10T10:04:48.557 INFO:teuthology.orchestra.run.smithi077.stdout: "poolop_send": 0, 2024-09-10T10:04:48.557 INFO:teuthology.orchestra.run.smithi077.stdout: "poolop_resend": 0, 2024-09-10T10:04:48.557 INFO:teuthology.orchestra.run.smithi077.stdout: "poolstat_active": 0, 2024-09-10T10:04:48.557 INFO:teuthology.orchestra.run.smithi077.stdout: "poolstat_send": 0, 2024-09-10T10:04:48.557 INFO:teuthology.orchestra.run.smithi077.stdout: "poolstat_resend": 0, 2024-09-10T10:04:48.557 INFO:teuthology.orchestra.run.smithi077.stdout: "statfs_active": 0, 2024-09-10T10:04:48.557 INFO:teuthology.orchestra.run.smithi077.stdout: "statfs_send": 0, 2024-09-10T10:04:48.557 INFO:teuthology.orchestra.run.smithi077.stdout: "statfs_resend": 0, 2024-09-10T10:04:48.557 INFO:teuthology.orchestra.run.smithi077.stdout: "command_active": 0, 2024-09-10T10:04:48.557 INFO:teuthology.orchestra.run.smithi077.stdout: "command_send": 0, 2024-09-10T10:04:48.558 INFO:teuthology.orchestra.run.smithi077.stdout: "command_resend": 0, 2024-09-10T10:04:48.558 INFO:teuthology.orchestra.run.smithi077.stdout: "map_epoch": 96, 2024-09-10T10:04:48.558 INFO:teuthology.orchestra.run.smithi077.stdout: "map_full": 0, 2024-09-10T10:04:48.558 INFO:teuthology.orchestra.run.smithi077.stdout: "map_inc": 2, 2024-09-10T10:04:48.558 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_sessions": 6, 2024-09-10T10:04:48.558 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_session_open": 6, 2024-09-10T10:04:48.558 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_session_close": 0, 2024-09-10T10:04:48.558 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_laggy": 0, 2024-09-10T10:04:48.558 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_wr": 0, 2024-09-10T10:04:48.558 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_rd": 2, 2024-09-10T10:04:48.558 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_del": 0 2024-09-10T10:04:48.558 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:48.558 INFO:teuthology.orchestra.run.smithi077.stdout: "oft": { 2024-09-10T10:04:48.558 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_total_objs": 0, 2024-09-10T10:04:48.558 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_total_kv_pairs": 0, 2024-09-10T10:04:48.559 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_total_updates": 0, 2024-09-10T10:04:48.559 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_total_removes": 0 2024-09-10T10:04:48.559 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:48.559 INFO:teuthology.orchestra.run.smithi077.stdout: "osd-slow-ops": { 2024-09-10T10:04:48.559 INFO:teuthology.orchestra.run.smithi077.stdout: "slow_ops_count": 0 2024-09-10T10:04:48.559 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:48.559 INFO:teuthology.orchestra.run.smithi077.stdout: "purge_queue": { 2024-09-10T10:04:48.559 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executing_ops": 0, 2024-09-10T10:04:48.559 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executing_ops_high_water": 0, 2024-09-10T10:04:48.559 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executing": 0, 2024-09-10T10:04:48.559 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executing_high_water": 0, 2024-09-10T10:04:48.559 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executed_ops": 0, 2024-09-10T10:04:48.559 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executed": 0, 2024-09-10T10:04:48.559 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_item_in_journal": 0 2024-09-10T10:04:48.559 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:48.559 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T10:04:48.560 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-09-10T10:04:48.560 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 104857600, 2024-09-10T10:04:48.560 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-09-10T10:04:48.560 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 2242, 2024-09-10T10:04:48.560 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 7219733, 2024-09-10T10:04:48.560 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-09-10T10:04:48.560 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 2242, 2024-09-10T10:04:48.560 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 0, 2024-09-10T10:04:48.560 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 0, 2024-09-10T10:04:48.560 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 2242, 2024-09-10T10:04:48.560 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 7219733, 2024-09-10T10:04:48.560 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-09-10T10:04:48.560 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-10T10:04:48.560 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-10T10:04:48.560 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-10T10:04:48.561 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:48.561 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:48.561 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-objecter_bytes": { 2024-09-10T10:04:48.561 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-09-10T10:04:48.561 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 104857600, 2024-09-10T10:04:48.561 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-09-10T10:04:48.561 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 0, 2024-09-10T10:04:48.561 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 0, 2024-09-10T10:04:48.561 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-09-10T10:04:48.561 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 0, 2024-09-10T10:04:48.561 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 2012, 2024-09-10T10:04:48.561 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 6289785, 2024-09-10T10:04:48.561 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 2012, 2024-09-10T10:04:48.561 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 6289785, 2024-09-10T10:04:48.561 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-09-10T10:04:48.562 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-10T10:04:48.562 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-10T10:04:48.562 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-10T10:04:48.562 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:48.562 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:48.562 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-objecter_ops": { 2024-09-10T10:04:48.562 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-09-10T10:04:48.562 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 1024, 2024-09-10T10:04:48.562 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-09-10T10:04:48.562 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 0, 2024-09-10T10:04:48.562 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 0, 2024-09-10T10:04:48.562 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-09-10T10:04:48.562 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 0, 2024-09-10T10:04:48.563 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 2012, 2024-09-10T10:04:48.563 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 2012, 2024-09-10T10:04:48.563 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 2012, 2024-09-10T10:04:48.563 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 2012, 2024-09-10T10:04:48.563 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-09-10T10:04:48.563 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-10T10:04:48.563 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-10T10:04:48.563 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-10T10:04:48.563 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:48.563 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:48.563 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-write_buf_throttle": { 2024-09-10T10:04:48.563 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-09-10T10:04:48.563 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 3758096384, 2024-09-10T10:04:48.563 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-09-10T10:04:48.563 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 0, 2024-09-10T10:04:48.564 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 0, 2024-09-10T10:04:48.564 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-09-10T10:04:48.564 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 0, 2024-09-10T10:04:48.564 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 0, 2024-09-10T10:04:48.564 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 0, 2024-09-10T10:04:48.564 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 0, 2024-09-10T10:04:48.564 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 0, 2024-09-10T10:04:48.564 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-09-10T10:04:48.564 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-10T10:04:48.564 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-10T10:04:48.564 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-10T10:04:48.564 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:48.564 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:48.564 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-write_buf_throttle-0x55d849394be0": { 2024-09-10T10:04:48.565 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-09-10T10:04:48.565 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 3758096384, 2024-09-10T10:04:48.565 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-09-10T10:04:48.565 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 0, 2024-09-10T10:04:48.565 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 0, 2024-09-10T10:04:48.565 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-09-10T10:04:48.565 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 0, 2024-09-10T10:04:48.565 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 0, 2024-09-10T10:04:48.565 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 0, 2024-09-10T10:04:48.565 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 0, 2024-09-10T10:04:48.565 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 0, 2024-09-10T10:04:48.565 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-09-10T10:04:48.565 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-10T10:04:48.565 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-10T10:04:48.566 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-10T10:04:48.566 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:48.566 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:48.566 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-10T10:04:49.268 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:04:48 smithi077 ceph-mon[30319]: pgmap v680: 129 pgs: 129 active+clean; 115 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-10T10:04:49.306 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-09-10T10:04:49.306 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-09-10T10:04:49.306 INFO:tasks.check_counter:Daemon mds.g mds.dir_split=0 2024-09-10T10:04:49.306 DEBUG:tasks.check_counter:Getting stats from j 2024-09-10T10:04:49.307 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph daemon mds.j perf dump 2024-09-10T10:04:49.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:04:48 smithi134 ceph-mon[35674]: pgmap v680: 129 pgs: 129 active+clean; 115 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-10T10:04:49.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:04:48 smithi120 ceph-mon[34770]: pgmap v680: 129 pgs: 129 active+clean; 115 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-10T10:04:49.674 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:04:50.246 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-10T10:04:50.247 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T10:04:50.247 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 0, 2024-09-10T10:04:50.247 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 2, 2024-09-10T10:04:50.247 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 0, 2024-09-10T10:04:50.247 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 0, 2024-09-10T10:04:50.247 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 4, 2024-09-10T10:04:50.247 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 0, 2024-09-10T10:04:50.247 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 0.003173189, 2024-09-10T10:04:50.247 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 0.000000000, 2024-09-10T10:04:50.247 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 0.000668425, 2024-09-10T10:04:50.247 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T10:04:50.247 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:04:50.248 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-10T10:04:50.248 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-10T10:04:50.248 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-10T10:04:50.248 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:50.248 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:04:50.248 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-10T10:04:50.248 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-10T10:04:50.248 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-10T10:04:50.248 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:50.248 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-10T10:04:50.248 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 0 2024-09-10T10:04:50.248 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:50.248 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T10:04:50.249 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 1, 2024-09-10T10:04:50.249 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 161, 2024-09-10T10:04:50.249 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 160, 2024-09-10T10:04:50.249 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 106816, 2024-09-10T10:04:50.249 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 3, 2024-09-10T10:04:50.249 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 2, 2024-09-10T10:04:50.249 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 0.018405817, 2024-09-10T10:04:50.249 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 0.014146784, 2024-09-10T10:04:50.249 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 0.002004272, 2024-09-10T10:04:50.249 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T10:04:50.249 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:04:50.249 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 159, 2024-09-10T10:04:50.249 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.012348875, 2024-09-10T10:04:50.250 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000077665 2024-09-10T10:04:50.250 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:50.250 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:04:50.250 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-10T10:04:50.250 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-10T10:04:50.250 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-10T10:04:50.250 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:50.250 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-10T10:04:50.250 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 106816 2024-09-10T10:04:50.250 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:50.250 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T10:04:50.250 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 219, 2024-09-10T10:04:50.251 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 214, 2024-09-10T10:04:50.251 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 480144, 2024-09-10T10:04:50.251 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 130432, 2024-09-10T10:04:50.251 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 2, 2024-09-10T10:04:50.251 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 1, 2024-09-10T10:04:50.251 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 0.043834887, 2024-09-10T10:04:50.251 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 0.023257363, 2024-09-10T10:04:50.251 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 0.024573342, 2024-09-10T10:04:50.251 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T10:04:50.251 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:04:50.251 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 214, 2024-09-10T10:04:50.251 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.021864238, 2024-09-10T10:04:50.251 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000102169 2024-09-10T10:04:50.252 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:50.252 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:04:50.252 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-10T10:04:50.252 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-10T10:04:50.252 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-10T10:04:50.252 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:50.252 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 480144, 2024-09-10T10:04:50.252 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 130432 2024-09-10T10:04:50.252 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:50.252 INFO:teuthology.orchestra.run.smithi077.stdout: "cct": { 2024-09-10T10:04:50.252 INFO:teuthology.orchestra.run.smithi077.stdout: "total_workers": 0, 2024-09-10T10:04:50.252 INFO:teuthology.orchestra.run.smithi077.stdout: "unhealthy_workers": 0 2024-09-10T10:04:50.253 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:50.253 INFO:teuthology.orchestra.run.smithi077.stdout: "mempool": { 2024-09-10T10:04:50.253 INFO:teuthology.orchestra.run.smithi077.stdout: "bloom_filter_bytes": 0, 2024-09-10T10:04:50.253 INFO:teuthology.orchestra.run.smithi077.stdout: "bloom_filter_items": 0, 2024-09-10T10:04:50.253 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T10:04:50.253 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_alloc_items": 0, 2024-09-10T10:04:50.253 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T10:04:50.253 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_data_items": 0, 2024-09-10T10:04:50.253 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T10:04:50.253 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T10:04:50.253 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T10:04:50.253 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T10:04:50.253 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T10:04:50.253 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_other_items": 0, 2024-09-10T10:04:50.254 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T10:04:50.254 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T10:04:50.254 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_extent_bytes": 0, 2024-09-10T10:04:50.254 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_extent_items": 0, 2024-09-10T10:04:50.254 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_blob_bytes": 0, 2024-09-10T10:04:50.254 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_blob_items": 0, 2024-09-10T10:04:50.254 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T10:04:50.254 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T10:04:50.254 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T10:04:50.254 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T10:04:50.254 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T10:04:50.254 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_fsck_items": 0, 2024-09-10T10:04:50.254 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_txc_bytes": 0, 2024-09-10T10:04:50.254 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_txc_items": 0, 2024-09-10T10:04:50.254 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T10:04:50.255 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T10:04:50.255 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_bytes": 0, 2024-09-10T10:04:50.255 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_items": 0, 2024-09-10T10:04:50.255 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_bytes": 0, 2024-09-10T10:04:50.255 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_items": 0, 2024-09-10T10:04:50.255 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T10:04:50.255 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_reader_items": 0, 2024-09-10T10:04:50.255 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T10:04:50.255 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_writer_items": 0, 2024-09-10T10:04:50.255 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_anon_bytes": 148967, 2024-09-10T10:04:50.255 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_anon_items": 29, 2024-09-10T10:04:50.255 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_meta_bytes": 0, 2024-09-10T10:04:50.255 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_meta_items": 0, 2024-09-10T10:04:50.255 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_bytes": 0, 2024-09-10T10:04:50.256 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_items": 0, 2024-09-10T10:04:50.256 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_mapbl_bytes": 0, 2024-09-10T10:04:50.256 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_mapbl_items": 0, 2024-09-10T10:04:50.256 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_pglog_bytes": 0, 2024-09-10T10:04:50.256 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_pglog_items": 0, 2024-09-10T10:04:50.256 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_bytes": 0, 2024-09-10T10:04:50.256 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_items": 0, 2024-09-10T10:04:50.256 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T10:04:50.256 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_mapping_items": 0, 2024-09-10T10:04:50.256 INFO:teuthology.orchestra.run.smithi077.stdout: "pgmap_bytes": 0, 2024-09-10T10:04:50.256 INFO:teuthology.orchestra.run.smithi077.stdout: "pgmap_items": 0, 2024-09-10T10:04:50.256 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_co_bytes": 928, 2024-09-10T10:04:50.256 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_co_items": 2, 2024-09-10T10:04:50.256 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_1_bytes": 0, 2024-09-10T10:04:50.256 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_1_items": 0, 2024-09-10T10:04:50.257 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_2_bytes": 0, 2024-09-10T10:04:50.257 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_2_items": 0 2024-09-10T10:04:50.257 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-10T10:04:50.257 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T10:04:50.257 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-09-10T10:04:50.257 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 104857600, 2024-09-10T10:04:50.257 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-09-10T10:04:50.257 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 216, 2024-09-10T10:04:50.257 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 446768, 2024-09-10T10:04:50.257 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-09-10T10:04:50.257 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 216, 2024-09-10T10:04:50.257 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 0, 2024-09-10T10:04:50.257 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 0, 2024-09-10T10:04:50.257 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 216, 2024-09-10T10:04:50.257 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 446768, 2024-09-10T10:04:50.258 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-09-10T10:04:50.258 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-10T10:04:50.258 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-10T10:04:50.258 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-10T10:04:50.258 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:50.258 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-10T10:04:50.258 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-10T10:04:50.964 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-09-10T10:04:50.964 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-09-10T10:04:50.964 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2024-09-10T10:04:50.964 DEBUG:tasks.check_counter:Getting stats from b 2024-09-10T10:04:50.965 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph daemon mds.b perf dump 2024-09-10T10:04:51.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:04:50 smithi077 ceph-mon[30319]: pgmap v681: 129 pgs: 129 active+clean; 115 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-10T10:04:51.223 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T10:04:51.344 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:04:50 smithi120 ceph-mon[34770]: pgmap v681: 129 pgs: 129 active+clean; 115 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-10T10:04:51.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:04:50 smithi134 ceph-mon[35674]: pgmap v681: 129 pgs: 129 active+clean; 115 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-10T10:04:51.761 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:04:51.762 INFO:teuthology.orchestra.run.smithi120.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T10:04:51.762 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_messages": 66005, 2024-09-10T10:04:51.762 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages": 118104, 2024-09-10T10:04:51.762 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_bytes": 19618928, 2024-09-10T10:04:51.762 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_bytes": 78933357, 2024-09-10T10:04:51.762 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_created_connections": 17, 2024-09-10T10:04:51.762 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_active_connections": 3, 2024-09-10T10:04:51.762 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_total_time": 4.159195180, 2024-09-10T10:04:51.762 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_send_time": 1.712261054, 2024-09-10T10:04:51.762 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_recv_time": 2.216622672, 2024-09-10T10:04:51.762 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_fast_dispatch_time": 0.683079997, 2024-09-10T10:04:51.762 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:04:51.762 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 118102, 2024-09-10T10:04:51.762 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 3.483725337, 2024-09-10T10:04:51.763 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000029497 2024-09-10T10:04:51.763 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:51.763 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:04:51.763 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 71388, 2024-09-10T10:04:51.763 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.021095372, 2024-09-10T10:04:51.763 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000295 2024-09-10T10:04:51.763 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:51.763 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_encrypted_bytes": 48416, 2024-09-10T10:04:51.763 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_encrypted_bytes": 29744 2024-09-10T10:04:51.763 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:51.763 INFO:teuthology.orchestra.run.smithi120.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T10:04:51.763 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_messages": 24007, 2024-09-10T10:04:51.763 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages": 23539, 2024-09-10T10:04:51.763 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_bytes": 6531681, 2024-09-10T10:04:51.764 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_bytes": 53122472, 2024-09-10T10:04:51.764 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_created_connections": 11, 2024-09-10T10:04:51.764 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_active_connections": 7, 2024-09-10T10:04:51.764 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_total_time": 1.496346682, 2024-09-10T10:04:51.764 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_send_time": 0.536124681, 2024-09-10T10:04:51.764 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_recv_time": 0.442026384, 2024-09-10T10:04:51.764 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_fast_dispatch_time": 0.449719303, 2024-09-10T10:04:51.764 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:04:51.764 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 23534, 2024-09-10T10:04:51.764 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.765411926, 2024-09-10T10:04:51.764 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000032523 2024-09-10T10:04:51.764 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:51.764 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:04:51.764 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 5503, 2024-09-10T10:04:51.765 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000877427, 2024-09-10T10:04:51.765 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000159 2024-09-10T10:04:51.765 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:51.765 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-10T10:04:51.765 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_encrypted_bytes": 361376 2024-09-10T10:04:51.765 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:51.765 INFO:teuthology.orchestra.run.smithi120.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T10:04:51.765 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_messages": 27244, 2024-09-10T10:04:51.765 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages": 26340, 2024-09-10T10:04:51.765 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_bytes": 7840682, 2024-09-10T10:04:51.765 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_bytes": 24876549, 2024-09-10T10:04:51.765 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_created_connections": 136, 2024-09-10T10:04:51.765 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_active_connections": 5, 2024-09-10T10:04:51.765 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_total_time": 1.727792328, 2024-09-10T10:04:51.766 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_send_time": 0.553654488, 2024-09-10T10:04:51.766 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_recv_time": 0.710912380, 2024-09-10T10:04:51.766 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_fast_dispatch_time": 0.462388054, 2024-09-10T10:04:51.766 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:04:51.766 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 26343, 2024-09-10T10:04:51.766 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.845573801, 2024-09-10T10:04:51.766 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000032098 2024-09-10T10:04:51.766 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:51.766 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:04:51.766 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 11570, 2024-09-10T10:04:51.766 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.001848287, 2024-09-10T10:04:51.766 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000159 2024-09-10T10:04:51.766 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:51.766 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_encrypted_bytes": 621600, 2024-09-10T10:04:51.767 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_encrypted_bytes": 148688 2024-09-10T10:04:51.767 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:51.767 INFO:teuthology.orchestra.run.smithi120.stdout: "cct": { 2024-09-10T10:04:51.767 INFO:teuthology.orchestra.run.smithi120.stdout: "total_workers": 1, 2024-09-10T10:04:51.767 INFO:teuthology.orchestra.run.smithi120.stdout: "unhealthy_workers": 0 2024-09-10T10:04:51.767 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:51.767 INFO:teuthology.orchestra.run.smithi120.stdout: "finisher-MDSRank": { 2024-09-10T10:04:51.767 INFO:teuthology.orchestra.run.smithi120.stdout: "queue_len": 0, 2024-09-10T10:04:51.767 INFO:teuthology.orchestra.run.smithi120.stdout: "complete_latency": { 2024-09-10T10:04:51.767 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 90190, 2024-09-10T10:04:51.767 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 24.641486424, 2024-09-10T10:04:51.767 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000273217 2024-09-10T10:04:51.767 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:04:51.767 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:51.768 INFO:teuthology.orchestra.run.smithi120.stdout: "finisher-PurgeQueue": { 2024-09-10T10:04:51.768 INFO:teuthology.orchestra.run.smithi120.stdout: "queue_len": 0, 2024-09-10T10:04:51.768 INFO:teuthology.orchestra.run.smithi120.stdout: "complete_latency": { 2024-09-10T10:04:51.768 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 4354, 2024-09-10T10:04:51.768 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.578525484, 2024-09-10T10:04:51.768 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000132872 2024-09-10T10:04:51.768 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:04:51.768 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:51.768 INFO:teuthology.orchestra.run.smithi120.stdout: "mds": { 2024-09-10T10:04:51.768 INFO:teuthology.orchestra.run.smithi120.stdout: "request": 19267, 2024-09-10T10:04:51.768 INFO:teuthology.orchestra.run.smithi120.stdout: "reply": 18839, 2024-09-10T10:04:51.768 INFO:teuthology.orchestra.run.smithi120.stdout: "reply_latency": { 2024-09-10T10:04:51.768 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 18839, 2024-09-10T10:04:51.768 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 118.151386776, 2024-09-10T10:04:51.769 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.006271637 2024-09-10T10:04:51.769 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:51.769 INFO:teuthology.orchestra.run.smithi120.stdout: "slow_reply": 0, 2024-09-10T10:04:51.769 INFO:teuthology.orchestra.run.smithi120.stdout: "forward": 428, 2024-09-10T10:04:51.769 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_fetch_complete": 0, 2024-09-10T10:04:51.769 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_fetch_keys": 0, 2024-09-10T10:04:51.769 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_commit": 221, 2024-09-10T10:04:51.769 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_split": 25, 2024-09-10T10:04:51.769 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_merge": 3, 2024-09-10T10:04:51.769 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes": 819, 2024-09-10T10:04:51.769 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_top": 555, 2024-09-10T10:04:51.769 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_bottom": 239, 2024-09-10T10:04:51.769 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_pin_tail": 25, 2024-09-10T10:04:51.769 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_pinned": 26, 2024-09-10T10:04:51.770 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_expired": 5079, 2024-09-10T10:04:51.770 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_with_caps": 2, 2024-09-10T10:04:51.770 INFO:teuthology.orchestra.run.smithi120.stdout: "caps": 2, 2024-09-10T10:04:51.770 INFO:teuthology.orchestra.run.smithi120.stdout: "subtrees": 5, 2024-09-10T10:04:51.770 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse": 20699, 2024-09-10T10:04:51.770 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_hit": 19830, 2024-09-10T10:04:51.770 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_forward": 292, 2024-09-10T10:04:51.770 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_discover": 0, 2024-09-10T10:04:51.770 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_dir_fetch": 0, 2024-09-10T10:04:51.770 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_remote_ino": 0, 2024-09-10T10:04:51.770 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_lock": 1, 2024-09-10T10:04:51.770 INFO:teuthology.orchestra.run.smithi120.stdout: "load_cent": 8, 2024-09-10T10:04:51.770 INFO:teuthology.orchestra.run.smithi120.stdout: "q": 0, 2024-09-10T10:04:51.770 INFO:teuthology.orchestra.run.smithi120.stdout: "exported": 163, 2024-09-10T10:04:51.770 INFO:teuthology.orchestra.run.smithi120.stdout: "exported_inodes": 4877, 2024-09-10T10:04:51.771 INFO:teuthology.orchestra.run.smithi120.stdout: "imported": 1, 2024-09-10T10:04:51.771 INFO:teuthology.orchestra.run.smithi120.stdout: "imported_inodes": 3, 2024-09-10T10:04:51.771 INFO:teuthology.orchestra.run.smithi120.stdout: "openino_dir_fetch": 0, 2024-09-10T10:04:51.771 INFO:teuthology.orchestra.run.smithi120.stdout: "openino_backtrace_fetch": 1907, 2024-09-10T10:04:51.771 INFO:teuthology.orchestra.run.smithi120.stdout: "openino_peer_discover": 0, 2024-09-10T10:04:51.771 INFO:teuthology.orchestra.run.smithi120.stdout: "root_rfiles": 4, 2024-09-10T10:04:51.771 INFO:teuthology.orchestra.run.smithi120.stdout: "root_rbytes": 234, 2024-09-10T10:04:51.771 INFO:teuthology.orchestra.run.smithi120.stdout: "root_rsnaps": 0, 2024-09-10T10:04:51.771 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_backtrace_fetch": 1907, 2024-09-10T10:04:51.771 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_set_tag": 0, 2024-09-10T10:04:51.771 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T10:04:51.771 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_inotable_repaired": 0, 2024-09-10T10:04:51.771 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_dir_inodes": 142, 2024-09-10T10:04:51.771 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_dir_base_inodes": 12, 2024-09-10T10:04:51.772 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_dirfrag_rstats": 130, 2024-09-10T10:04:51.772 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_file_inodes": 1765, 2024-09-10T10:04:51.772 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_inode_file_caps": 27, 2024-09-10T10:04:51.772 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_cap_op_revoke": 8670, 2024-09-10T10:04:51.772 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_cap_op_grant": 18082, 2024-09-10T10:04:51.772 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_cap_op_trunc": 7863, 2024-09-10T10:04:51.772 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T10:04:51.772 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T10:04:51.772 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_client_caps": 15263, 2024-09-10T10:04:51.772 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_client_caps_dirty": 11297, 2024-09-10T10:04:51.772 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_client_cap_release": 425, 2024-09-10T10:04:51.772 INFO:teuthology.orchestra.run.smithi120.stdout: "process_request_cap_release": 11564 2024-09-10T10:04:51.772 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:51.772 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_cache": { 2024-09-10T10:04:51.772 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_update": 0, 2024-09-10T10:04:51.773 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_update_receipt": 0, 2024-09-10T10:04:51.773 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_try_discover": 0, 2024-09-10T10:04:51.773 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_send_discover": 0, 2024-09-10T10:04:51.773 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_handle_discover": 195, 2024-09-10T10:04:51.773 INFO:teuthology.orchestra.run.smithi120.stdout: "num_strays": 0, 2024-09-10T10:04:51.773 INFO:teuthology.orchestra.run.smithi120.stdout: "num_strays_delayed": 0, 2024-09-10T10:04:51.773 INFO:teuthology.orchestra.run.smithi120.stdout: "num_strays_enqueuing": 0, 2024-09-10T10:04:51.773 INFO:teuthology.orchestra.run.smithi120.stdout: "strays_created": 2283, 2024-09-10T10:04:51.773 INFO:teuthology.orchestra.run.smithi120.stdout: "strays_enqueued": 2280, 2024-09-10T10:04:51.773 INFO:teuthology.orchestra.run.smithi120.stdout: "strays_reintegrated": 3, 2024-09-10T10:04:51.773 INFO:teuthology.orchestra.run.smithi120.stdout: "strays_migrated": 0, 2024-09-10T10:04:51.773 INFO:teuthology.orchestra.run.smithi120.stdout: "num_recovering_processing": 0, 2024-09-10T10:04:51.773 INFO:teuthology.orchestra.run.smithi120.stdout: "num_recovering_enqueued": 0, 2024-09-10T10:04:51.773 INFO:teuthology.orchestra.run.smithi120.stdout: "num_recovering_prioritized": 0, 2024-09-10T10:04:51.774 INFO:teuthology.orchestra.run.smithi120.stdout: "recovery_started": 0, 2024-09-10T10:04:51.774 INFO:teuthology.orchestra.run.smithi120.stdout: "recovery_completed": 0, 2024-09-10T10:04:51.774 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_quiesce_path": 8, 2024-09-10T10:04:51.774 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_quiesce_inode": 668, 2024-09-10T10:04:51.774 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_enqueue_scrub": 12, 2024-09-10T10:04:51.774 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_exportdir": 163, 2024-09-10T10:04:51.774 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_flush": 0, 2024-09-10T10:04:51.774 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_fragmentdir": 28, 2024-09-10T10:04:51.774 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_fragstats": 0, 2024-09-10T10:04:51.774 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_inodestats": 0 2024-09-10T10:04:51.774 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:51.774 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_log": { 2024-09-10T10:04:51.774 INFO:teuthology.orchestra.run.smithi120.stdout: "evlrg": 0, 2024-09-10T10:04:51.774 INFO:teuthology.orchestra.run.smithi120.stdout: "evadd": 42186, 2024-09-10T10:04:51.774 INFO:teuthology.orchestra.run.smithi120.stdout: "evex": 0, 2024-09-10T10:04:51.775 INFO:teuthology.orchestra.run.smithi120.stdout: "evtrm": 0, 2024-09-10T10:04:51.775 INFO:teuthology.orchestra.run.smithi120.stdout: "ev": 42186, 2024-09-10T10:04:51.775 INFO:teuthology.orchestra.run.smithi120.stdout: "evexg": 0, 2024-09-10T10:04:51.775 INFO:teuthology.orchestra.run.smithi120.stdout: "evexd": 0, 2024-09-10T10:04:51.775 INFO:teuthology.orchestra.run.smithi120.stdout: "segadd": 50, 2024-09-10T10:04:51.775 INFO:teuthology.orchestra.run.smithi120.stdout: "segex": 0, 2024-09-10T10:04:51.775 INFO:teuthology.orchestra.run.smithi120.stdout: "segtrm": 0, 2024-09-10T10:04:51.775 INFO:teuthology.orchestra.run.smithi120.stdout: "seg": 50, 2024-09-10T10:04:51.775 INFO:teuthology.orchestra.run.smithi120.stdout: "segmjr": 5, 2024-09-10T10:04:51.775 INFO:teuthology.orchestra.run.smithi120.stdout: "segexg": 0, 2024-09-10T10:04:51.775 INFO:teuthology.orchestra.run.smithi120.stdout: "segexd": 0, 2024-09-10T10:04:51.775 INFO:teuthology.orchestra.run.smithi120.stdout: "expos": 4194304, 2024-09-10T10:04:51.775 INFO:teuthology.orchestra.run.smithi120.stdout: "wrpos": 95049273, 2024-09-10T10:04:51.775 INFO:teuthology.orchestra.run.smithi120.stdout: "rdpos": 0, 2024-09-10T10:04:51.775 INFO:teuthology.orchestra.run.smithi120.stdout: "jlat": { 2024-09-10T10:04:51.776 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 24948, 2024-09-10T10:04:51.776 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 60.022851075, 2024-09-10T10:04:51.776 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.002405918 2024-09-10T10:04:51.776 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:51.776 INFO:teuthology.orchestra.run.smithi120.stdout: "replayed": 0 2024-09-10T10:04:51.776 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:51.776 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_mem": { 2024-09-10T10:04:51.776 INFO:teuthology.orchestra.run.smithi120.stdout: "ino": 30, 2024-09-10T10:04:51.776 INFO:teuthology.orchestra.run.smithi120.stdout: "ino+": 6786, 2024-09-10T10:04:51.776 INFO:teuthology.orchestra.run.smithi120.stdout: "ino-": 6756, 2024-09-10T10:04:51.776 INFO:teuthology.orchestra.run.smithi120.stdout: "dir": 25, 2024-09-10T10:04:51.776 INFO:teuthology.orchestra.run.smithi120.stdout: "dir+": 291, 2024-09-10T10:04:51.776 INFO:teuthology.orchestra.run.smithi120.stdout: "dir-": 266, 2024-09-10T10:04:51.776 INFO:teuthology.orchestra.run.smithi120.stdout: "dn": 819, 2024-09-10T10:04:51.777 INFO:teuthology.orchestra.run.smithi120.stdout: "dn+": 7461, 2024-09-10T10:04:51.777 INFO:teuthology.orchestra.run.smithi120.stdout: "dn-": 6642, 2024-09-10T10:04:51.777 INFO:teuthology.orchestra.run.smithi120.stdout: "cap": 2, 2024-09-10T10:04:51.777 INFO:teuthology.orchestra.run.smithi120.stdout: "cap+": 6766, 2024-09-10T10:04:51.777 INFO:teuthology.orchestra.run.smithi120.stdout: "cap-": 6764, 2024-09-10T10:04:51.777 INFO:teuthology.orchestra.run.smithi120.stdout: "rss": 94916, 2024-09-10T10:04:51.777 INFO:teuthology.orchestra.run.smithi120.stdout: "heap": 231712 2024-09-10T10:04:51.777 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:51.777 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_server": { 2024-09-10T10:04:51.777 INFO:teuthology.orchestra.run.smithi120.stdout: "dispatch_client_request": 37569, 2024-09-10T10:04:51.777 INFO:teuthology.orchestra.run.smithi120.stdout: "dispatch_server_request": 944, 2024-09-10T10:04:51.777 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_client_request": 19267, 2024-09-10T10:04:51.777 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_client_session": 183, 2024-09-10T10:04:51.777 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_peer_request": 1885, 2024-09-10T10:04:51.778 INFO:teuthology.orchestra.run.smithi120.stdout: "req_create_latency": { 2024-09-10T10:04:51.778 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 6678, 2024-09-10T10:04:51.778 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 18.097798599, 2024-09-10T10:04:51.778 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.002710062 2024-09-10T10:04:51.778 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:51.778 INFO:teuthology.orchestra.run.smithi120.stdout: "req_getattr_latency": { 2024-09-10T10:04:51.778 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 218, 2024-09-10T10:04:51.778 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 1.361970590, 2024-09-10T10:04:51.778 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.006247571 2024-09-10T10:04:51.778 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:51.778 INFO:teuthology.orchestra.run.smithi120.stdout: "req_getfilelock_latency": { 2024-09-10T10:04:51.778 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:51.778 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:51.778 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:51.779 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:51.779 INFO:teuthology.orchestra.run.smithi120.stdout: "req_link_latency": { 2024-09-10T10:04:51.779 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 3, 2024-09-10T10:04:51.779 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.007369776, 2024-09-10T10:04:51.779 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.002456592 2024-09-10T10:04:51.779 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:51.779 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookup_latency": { 2024-09-10T10:04:51.779 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 173, 2024-09-10T10:04:51.779 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.053190359, 2024-09-10T10:04:51.779 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000307458 2024-09-10T10:04:51.779 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:51.779 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookuphash_latency": { 2024-09-10T10:04:51.779 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:51.779 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:51.779 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:51.780 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:51.780 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookupino_latency": { 2024-09-10T10:04:51.780 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 1, 2024-09-10T10:04:51.780 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000377645, 2024-09-10T10:04:51.780 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000377645 2024-09-10T10:04:51.780 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:51.780 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookupname_latency": { 2024-09-10T10:04:51.780 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:51.780 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:51.780 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:51.780 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:51.780 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookupparent_latency": { 2024-09-10T10:04:51.780 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:51.780 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:51.781 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:51.781 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:51.781 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookupsnap_latency": { 2024-09-10T10:04:51.781 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:51.781 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:51.781 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:51.781 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:51.781 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lssnap_latency": { 2024-09-10T10:04:51.781 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:51.781 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:51.781 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:51.782 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:51.782 INFO:teuthology.orchestra.run.smithi120.stdout: "req_mkdir_latency": { 2024-09-10T10:04:51.782 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 69, 2024-09-10T10:04:51.782 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 24.503407601, 2024-09-10T10:04:51.782 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.355121849 2024-09-10T10:04:51.782 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:51.782 INFO:teuthology.orchestra.run.smithi120.stdout: "req_mknod_latency": { 2024-09-10T10:04:51.782 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:51.782 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:51.782 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:51.782 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:51.782 INFO:teuthology.orchestra.run.smithi120.stdout: "req_mksnap_latency": { 2024-09-10T10:04:51.782 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:51.782 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:51.783 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:51.783 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:51.783 INFO:teuthology.orchestra.run.smithi120.stdout: "req_open_latency": { 2024-09-10T10:04:51.783 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 392, 2024-09-10T10:04:51.783 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.069965272, 2024-09-10T10:04:51.783 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000178482 2024-09-10T10:04:51.783 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:51.783 INFO:teuthology.orchestra.run.smithi120.stdout: "req_readdir_latency": { 2024-09-10T10:04:51.783 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 114, 2024-09-10T10:04:51.783 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.696261048, 2024-09-10T10:04:51.783 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.006107553 2024-09-10T10:04:51.783 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:51.783 INFO:teuthology.orchestra.run.smithi120.stdout: "req_rename_latency": { 2024-09-10T10:04:51.783 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 91, 2024-09-10T10:04:51.783 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.144039288, 2024-09-10T10:04:51.784 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.001582849 2024-09-10T10:04:51.784 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:51.784 INFO:teuthology.orchestra.run.smithi120.stdout: "req_renamesnap_latency": { 2024-09-10T10:04:51.784 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:51.784 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:51.784 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:51.784 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:51.784 INFO:teuthology.orchestra.run.smithi120.stdout: "req_snapdiff_latency": { 2024-09-10T10:04:51.784 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:51.784 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:51.784 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:51.784 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:51.784 INFO:teuthology.orchestra.run.smithi120.stdout: "req_rmdir_latency": { 2024-09-10T10:04:51.784 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 15, 2024-09-10T10:04:51.785 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.101125316, 2024-09-10T10:04:51.785 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.006741687 2024-09-10T10:04:51.785 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:51.785 INFO:teuthology.orchestra.run.smithi120.stdout: "req_rmsnap_latency": { 2024-09-10T10:04:51.785 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:51.785 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:51.785 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:51.785 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:51.785 INFO:teuthology.orchestra.run.smithi120.stdout: "req_rmxattr_latency": { 2024-09-10T10:04:51.785 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:51.785 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:51.785 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:51.785 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:51.785 INFO:teuthology.orchestra.run.smithi120.stdout: "req_setattr_latency": { 2024-09-10T10:04:51.785 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 8848, 2024-09-10T10:04:51.786 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 69.263231250, 2024-09-10T10:04:51.786 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.007828122 2024-09-10T10:04:51.786 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:51.786 INFO:teuthology.orchestra.run.smithi120.stdout: "req_setdirlayout_latency": { 2024-09-10T10:04:51.786 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:51.786 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:51.786 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:51.786 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:51.786 INFO:teuthology.orchestra.run.smithi120.stdout: "req_setfilelock_latency": { 2024-09-10T10:04:51.786 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 8, 2024-09-10T10:04:51.786 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.002608520, 2024-09-10T10:04:51.786 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000326065 2024-09-10T10:04:51.786 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:51.786 INFO:teuthology.orchestra.run.smithi120.stdout: "req_setlayout_latency": { 2024-09-10T10:04:51.787 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:51.787 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:51.787 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:51.787 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:51.787 INFO:teuthology.orchestra.run.smithi120.stdout: "req_setxattr_latency": { 2024-09-10T10:04:51.787 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 8, 2024-09-10T10:04:51.787 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.021158949, 2024-09-10T10:04:51.787 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.002644868 2024-09-10T10:04:51.787 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:51.787 INFO:teuthology.orchestra.run.smithi120.stdout: "req_symlink_latency": { 2024-09-10T10:04:51.787 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 6, 2024-09-10T10:04:51.787 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.004538926, 2024-09-10T10:04:51.787 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000756487 2024-09-10T10:04:51.787 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:51.787 INFO:teuthology.orchestra.run.smithi120.stdout: "req_unlink_latency": { 2024-09-10T10:04:51.788 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 2215, 2024-09-10T10:04:51.788 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 3.824343637, 2024-09-10T10:04:51.788 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.001726565 2024-09-10T10:04:51.788 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:51.788 INFO:teuthology.orchestra.run.smithi120.stdout: "cap_revoke_eviction": 0, 2024-09-10T10:04:51.788 INFO:teuthology.orchestra.run.smithi120.stdout: "cap_acquisition_throttle": 0, 2024-09-10T10:04:51.788 INFO:teuthology.orchestra.run.smithi120.stdout: "req_getvxattr_latency": { 2024-09-10T10:04:51.788 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:51.788 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:51.788 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:51.788 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:04:51.788 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:51.789 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_sessions": { 2024-09-10T10:04:51.789 INFO:teuthology.orchestra.run.smithi120.stdout: "session_count": 1, 2024-09-10T10:04:51.789 INFO:teuthology.orchestra.run.smithi120.stdout: "session_add": 3, 2024-09-10T10:04:51.789 INFO:teuthology.orchestra.run.smithi120.stdout: "session_remove": 2, 2024-09-10T10:04:51.789 INFO:teuthology.orchestra.run.smithi120.stdout: "sessions_open": 1, 2024-09-10T10:04:51.789 INFO:teuthology.orchestra.run.smithi120.stdout: "sessions_stale": 0, 2024-09-10T10:04:51.789 INFO:teuthology.orchestra.run.smithi120.stdout: "total_load": 1445, 2024-09-10T10:04:51.789 INFO:teuthology.orchestra.run.smithi120.stdout: "average_load": 1445, 2024-09-10T10:04:51.789 INFO:teuthology.orchestra.run.smithi120.stdout: "avg_session_uptime": 577, 2024-09-10T10:04:51.789 INFO:teuthology.orchestra.run.smithi120.stdout: "mdthresh_evicted": 0 2024-09-10T10:04:51.789 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:51.789 INFO:teuthology.orchestra.run.smithi120.stdout: "mempool": { 2024-09-10T10:04:51.789 INFO:teuthology.orchestra.run.smithi120.stdout: "bloom_filter_bytes": 0, 2024-09-10T10:04:51.789 INFO:teuthology.orchestra.run.smithi120.stdout: "bloom_filter_items": 0, 2024-09-10T10:04:51.790 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T10:04:51.790 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_alloc_items": 0, 2024-09-10T10:04:51.790 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T10:04:51.790 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_data_items": 0, 2024-09-10T10:04:51.790 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T10:04:51.790 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T10:04:51.790 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T10:04:51.790 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T10:04:51.790 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T10:04:51.790 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_other_items": 0, 2024-09-10T10:04:51.790 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T10:04:51.790 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T10:04:51.790 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_extent_bytes": 0, 2024-09-10T10:04:51.791 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_extent_items": 0, 2024-09-10T10:04:51.791 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_blob_bytes": 0, 2024-09-10T10:04:51.791 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_blob_items": 0, 2024-09-10T10:04:51.791 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T10:04:51.791 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T10:04:51.791 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T10:04:51.791 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T10:04:51.791 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T10:04:51.791 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_fsck_items": 0, 2024-09-10T10:04:51.791 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_txc_bytes": 0, 2024-09-10T10:04:51.791 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_txc_items": 0, 2024-09-10T10:04:51.792 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T10:04:51.792 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T10:04:51.792 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_writing_bytes": 0, 2024-09-10T10:04:51.792 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_writing_items": 0, 2024-09-10T10:04:51.792 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_bytes": 0, 2024-09-10T10:04:51.792 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_items": 0, 2024-09-10T10:04:51.792 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T10:04:51.792 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_file_reader_items": 0, 2024-09-10T10:04:51.792 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T10:04:51.792 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_file_writer_items": 0, 2024-09-10T10:04:51.792 INFO:teuthology.orchestra.run.smithi120.stdout: "buffer_anon_bytes": 310549, 2024-09-10T10:04:51.792 INFO:teuthology.orchestra.run.smithi120.stdout: "buffer_anon_items": 103, 2024-09-10T10:04:51.792 INFO:teuthology.orchestra.run.smithi120.stdout: "buffer_meta_bytes": 0, 2024-09-10T10:04:51.792 INFO:teuthology.orchestra.run.smithi120.stdout: "buffer_meta_items": 0, 2024-09-10T10:04:51.793 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_bytes": 0, 2024-09-10T10:04:51.793 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_items": 0, 2024-09-10T10:04:51.793 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_mapbl_bytes": 0, 2024-09-10T10:04:51.793 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_mapbl_items": 0, 2024-09-10T10:04:51.793 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pglog_bytes": 0, 2024-09-10T10:04:51.793 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pglog_items": 0, 2024-09-10T10:04:51.793 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_bytes": 9144, 2024-09-10T10:04:51.793 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_items": 172, 2024-09-10T10:04:51.793 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T10:04:51.793 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_mapping_items": 0, 2024-09-10T10:04:51.793 INFO:teuthology.orchestra.run.smithi120.stdout: "pgmap_bytes": 0, 2024-09-10T10:04:51.793 INFO:teuthology.orchestra.run.smithi120.stdout: "pgmap_items": 0, 2024-09-10T10:04:51.793 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_co_bytes": 725666, 2024-09-10T10:04:51.794 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_co_items": 9493, 2024-09-10T10:04:51.794 INFO:teuthology.orchestra.run.smithi120.stdout: "unittest_1_bytes": 0, 2024-09-10T10:04:51.794 INFO:teuthology.orchestra.run.smithi120.stdout: "unittest_1_items": 0, 2024-09-10T10:04:51.794 INFO:teuthology.orchestra.run.smithi120.stdout: "unittest_2_bytes": 0, 2024-09-10T10:04:51.794 INFO:teuthology.orchestra.run.smithi120.stdout: "unittest_2_items": 0 2024-09-10T10:04:51.794 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:51.794 INFO:teuthology.orchestra.run.smithi120.stdout: "objecter": { 2024-09-10T10:04:51.794 INFO:teuthology.orchestra.run.smithi120.stdout: "op_active": 0, 2024-09-10T10:04:51.794 INFO:teuthology.orchestra.run.smithi120.stdout: "op_laggy": 0, 2024-09-10T10:04:51.794 INFO:teuthology.orchestra.run.smithi120.stdout: "op_send": 69235, 2024-09-10T10:04:51.794 INFO:teuthology.orchestra.run.smithi120.stdout: "op_send_bytes": 94807168, 2024-09-10T10:04:51.794 INFO:teuthology.orchestra.run.smithi120.stdout: "op_resend": 33, 2024-09-10T10:04:51.794 INFO:teuthology.orchestra.run.smithi120.stdout: "op_reply": 69235, 2024-09-10T10:04:51.794 INFO:teuthology.orchestra.run.smithi120.stdout: "op_latency": { 2024-09-10T10:04:51.795 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 69235, 2024-09-10T10:04:51.795 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 369.092339156, 2024-09-10T10:04:51.795 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.005331008 2024-09-10T10:04:51.795 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:51.795 INFO:teuthology.orchestra.run.smithi120.stdout: "op_inflight": 0, 2024-09-10T10:04:51.795 INFO:teuthology.orchestra.run.smithi120.stdout: "oplen_avg": { 2024-09-10T10:04:51.795 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 69235, 2024-09-10T10:04:51.795 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 69568 2024-09-10T10:04:51.795 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:51.795 INFO:teuthology.orchestra.run.smithi120.stdout: "op": 69235, 2024-09-10T10:04:51.795 INFO:teuthology.orchestra.run.smithi120.stdout: "op_r": 1971, 2024-09-10T10:04:51.795 INFO:teuthology.orchestra.run.smithi120.stdout: "op_w": 67264, 2024-09-10T10:04:51.795 INFO:teuthology.orchestra.run.smithi120.stdout: "op_rmw": 0, 2024-09-10T10:04:51.795 INFO:teuthology.orchestra.run.smithi120.stdout: "op_pg": 0, 2024-09-10T10:04:51.796 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_stat": 0, 2024-09-10T10:04:51.796 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_create": 11, 2024-09-10T10:04:51.796 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_read": 52, 2024-09-10T10:04:51.796 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_write": 24988, 2024-09-10T10:04:51.796 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_writefull": 64, 2024-09-10T10:04:51.796 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_writesame": 0, 2024-09-10T10:04:51.796 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_append": 0, 2024-09-10T10:04:51.796 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_zero": 0, 2024-09-10T10:04:51.796 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_truncate": 9, 2024-09-10T10:04:51.796 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_delete": 5146, 2024-09-10T10:04:51.796 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_mapext": 0, 2024-09-10T10:04:51.796 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_sparse_read": 0, 2024-09-10T10:04:51.796 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_clonerange": 0, 2024-09-10T10:04:51.796 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_getxattr": 1919, 2024-09-10T10:04:51.797 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_setxattr": 22, 2024-09-10T10:04:51.797 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_cmpxattr": 0, 2024-09-10T10:04:51.797 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_rmxattr": 0, 2024-09-10T10:04:51.797 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_resetxattrs": 0, 2024-09-10T10:04:51.797 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_call": 0, 2024-09-10T10:04:51.797 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_watch": 0, 2024-09-10T10:04:51.797 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_notify": 0, 2024-09-10T10:04:51.797 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T10:04:51.797 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_pgls": 0, 2024-09-10T10:04:51.797 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_pgls_filter": 0, 2024-09-10T10:04:51.797 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_other": 36717, 2024-09-10T10:04:51.797 INFO:teuthology.orchestra.run.smithi120.stdout: "linger_active": 0, 2024-09-10T10:04:51.797 INFO:teuthology.orchestra.run.smithi120.stdout: "linger_send": 0, 2024-09-10T10:04:51.797 INFO:teuthology.orchestra.run.smithi120.stdout: "linger_resend": 0, 2024-09-10T10:04:51.797 INFO:teuthology.orchestra.run.smithi120.stdout: "linger_ping": 0, 2024-09-10T10:04:51.798 INFO:teuthology.orchestra.run.smithi120.stdout: "poolop_active": 0, 2024-09-10T10:04:51.798 INFO:teuthology.orchestra.run.smithi120.stdout: "poolop_send": 0, 2024-09-10T10:04:51.798 INFO:teuthology.orchestra.run.smithi120.stdout: "poolop_resend": 0, 2024-09-10T10:04:51.798 INFO:teuthology.orchestra.run.smithi120.stdout: "poolstat_active": 0, 2024-09-10T10:04:51.798 INFO:teuthology.orchestra.run.smithi120.stdout: "poolstat_send": 0, 2024-09-10T10:04:51.798 INFO:teuthology.orchestra.run.smithi120.stdout: "poolstat_resend": 0, 2024-09-10T10:04:51.798 INFO:teuthology.orchestra.run.smithi120.stdout: "statfs_active": 0, 2024-09-10T10:04:51.798 INFO:teuthology.orchestra.run.smithi120.stdout: "statfs_send": 0, 2024-09-10T10:04:51.798 INFO:teuthology.orchestra.run.smithi120.stdout: "statfs_resend": 0, 2024-09-10T10:04:51.798 INFO:teuthology.orchestra.run.smithi120.stdout: "command_active": 0, 2024-09-10T10:04:51.798 INFO:teuthology.orchestra.run.smithi120.stdout: "command_send": 0, 2024-09-10T10:04:51.798 INFO:teuthology.orchestra.run.smithi120.stdout: "command_resend": 0, 2024-09-10T10:04:51.798 INFO:teuthology.orchestra.run.smithi120.stdout: "map_epoch": 96, 2024-09-10T10:04:51.798 INFO:teuthology.orchestra.run.smithi120.stdout: "map_full": 0, 2024-09-10T10:04:51.799 INFO:teuthology.orchestra.run.smithi120.stdout: "map_inc": 11, 2024-09-10T10:04:51.799 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_sessions": 12, 2024-09-10T10:04:51.799 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_session_open": 12, 2024-09-10T10:04:51.799 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_session_close": 0, 2024-09-10T10:04:51.799 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_laggy": 0, 2024-09-10T10:04:51.799 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_wr": 593, 2024-09-10T10:04:51.799 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_rd": 0, 2024-09-10T10:04:51.799 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_del": 47 2024-09-10T10:04:51.799 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:51.799 INFO:teuthology.orchestra.run.smithi120.stdout: "oft": { 2024-09-10T10:04:51.799 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_total_objs": 1, 2024-09-10T10:04:51.799 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_total_kv_pairs": 0, 2024-09-10T10:04:51.799 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_total_updates": 0, 2024-09-10T10:04:51.799 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_total_removes": 0 2024-09-10T10:04:51.799 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:51.799 INFO:teuthology.orchestra.run.smithi120.stdout: "osd-slow-ops": { 2024-09-10T10:04:51.800 INFO:teuthology.orchestra.run.smithi120.stdout: "slow_ops_count": 667 2024-09-10T10:04:51.800 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:51.800 INFO:teuthology.orchestra.run.smithi120.stdout: "purge_queue": { 2024-09-10T10:04:51.800 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executing_ops": 0, 2024-09-10T10:04:51.800 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executing_ops_high_water": 34, 2024-09-10T10:04:51.800 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executing": 0, 2024-09-10T10:04:51.800 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executing_high_water": 6, 2024-09-10T10:04:51.800 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executed_ops": 4573, 2024-09-10T10:04:51.800 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executed": 2280, 2024-09-10T10:04:51.800 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_item_in_journal": 0 2024-09-10T10:04:51.800 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:51.800 INFO:teuthology.orchestra.run.smithi120.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T10:04:51.800 INFO:teuthology.orchestra.run.smithi120.stdout: "val": 0, 2024-09-10T10:04:51.800 INFO:teuthology.orchestra.run.smithi120.stdout: "max": 104857600, 2024-09-10T10:04:51.801 INFO:teuthology.orchestra.run.smithi120.stdout: "get_started": 0, 2024-09-10T10:04:51.801 INFO:teuthology.orchestra.run.smithi120.stdout: "get": 117252, 2024-09-10T10:04:51.801 INFO:teuthology.orchestra.run.smithi120.stdout: "get_sum": 23984529, 2024-09-10T10:04:51.801 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_fail": 0, 2024-09-10T10:04:51.801 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_success": 117252, 2024-09-10T10:04:51.801 INFO:teuthology.orchestra.run.smithi120.stdout: "take": 0, 2024-09-10T10:04:51.801 INFO:teuthology.orchestra.run.smithi120.stdout: "take_sum": 0, 2024-09-10T10:04:51.801 INFO:teuthology.orchestra.run.smithi120.stdout: "put": 117252, 2024-09-10T10:04:51.801 INFO:teuthology.orchestra.run.smithi120.stdout: "put_sum": 23984529, 2024-09-10T10:04:51.801 INFO:teuthology.orchestra.run.smithi120.stdout: "wait": { 2024-09-10T10:04:51.801 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:51.801 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:51.801 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:51.801 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:04:51.802 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:51.802 INFO:teuthology.orchestra.run.smithi120.stdout: "throttle-objecter_bytes": { 2024-09-10T10:04:51.802 INFO:teuthology.orchestra.run.smithi120.stdout: "val": 0, 2024-09-10T10:04:51.802 INFO:teuthology.orchestra.run.smithi120.stdout: "max": 104857600, 2024-09-10T10:04:51.802 INFO:teuthology.orchestra.run.smithi120.stdout: "get_started": 0, 2024-09-10T10:04:51.802 INFO:teuthology.orchestra.run.smithi120.stdout: "get": 0, 2024-09-10T10:04:51.802 INFO:teuthology.orchestra.run.smithi120.stdout: "get_sum": 0, 2024-09-10T10:04:51.802 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_fail": 0, 2024-09-10T10:04:51.802 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_success": 0, 2024-09-10T10:04:51.802 INFO:teuthology.orchestra.run.smithi120.stdout: "take": 69235, 2024-09-10T10:04:51.802 INFO:teuthology.orchestra.run.smithi120.stdout: "take_sum": 95055576, 2024-09-10T10:04:51.802 INFO:teuthology.orchestra.run.smithi120.stdout: "put": 69235, 2024-09-10T10:04:51.802 INFO:teuthology.orchestra.run.smithi120.stdout: "put_sum": 95055576, 2024-09-10T10:04:51.802 INFO:teuthology.orchestra.run.smithi120.stdout: "wait": { 2024-09-10T10:04:51.802 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:51.803 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:51.803 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:51.803 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:04:51.803 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:51.803 INFO:teuthology.orchestra.run.smithi120.stdout: "throttle-objecter_ops": { 2024-09-10T10:04:51.803 INFO:teuthology.orchestra.run.smithi120.stdout: "val": 0, 2024-09-10T10:04:51.803 INFO:teuthology.orchestra.run.smithi120.stdout: "max": 1024, 2024-09-10T10:04:51.803 INFO:teuthology.orchestra.run.smithi120.stdout: "get_started": 0, 2024-09-10T10:04:51.803 INFO:teuthology.orchestra.run.smithi120.stdout: "get": 0, 2024-09-10T10:04:51.803 INFO:teuthology.orchestra.run.smithi120.stdout: "get_sum": 0, 2024-09-10T10:04:51.803 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_fail": 0, 2024-09-10T10:04:51.803 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_success": 0, 2024-09-10T10:04:51.803 INFO:teuthology.orchestra.run.smithi120.stdout: "take": 69235, 2024-09-10T10:04:51.803 INFO:teuthology.orchestra.run.smithi120.stdout: "take_sum": 69235, 2024-09-10T10:04:51.803 INFO:teuthology.orchestra.run.smithi120.stdout: "put": 69235, 2024-09-10T10:04:51.804 INFO:teuthology.orchestra.run.smithi120.stdout: "put_sum": 69235, 2024-09-10T10:04:51.804 INFO:teuthology.orchestra.run.smithi120.stdout: "wait": { 2024-09-10T10:04:51.804 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:51.804 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:51.804 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:51.804 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:04:51.804 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:51.804 INFO:teuthology.orchestra.run.smithi120.stdout: "throttle-write_buf_throttle": { 2024-09-10T10:04:51.804 INFO:teuthology.orchestra.run.smithi120.stdout: "val": 0, 2024-09-10T10:04:51.804 INFO:teuthology.orchestra.run.smithi120.stdout: "max": 3758096384, 2024-09-10T10:04:51.804 INFO:teuthology.orchestra.run.smithi120.stdout: "get_started": 0, 2024-09-10T10:04:51.804 INFO:teuthology.orchestra.run.smithi120.stdout: "get": 2280, 2024-09-10T10:04:51.804 INFO:teuthology.orchestra.run.smithi120.stdout: "get_sum": 248408, 2024-09-10T10:04:51.804 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_fail": 0, 2024-09-10T10:04:51.805 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_success": 2280, 2024-09-10T10:04:51.805 INFO:teuthology.orchestra.run.smithi120.stdout: "take": 0, 2024-09-10T10:04:51.805 INFO:teuthology.orchestra.run.smithi120.stdout: "take_sum": 0, 2024-09-10T10:04:51.805 INFO:teuthology.orchestra.run.smithi120.stdout: "put": 40, 2024-09-10T10:04:51.805 INFO:teuthology.orchestra.run.smithi120.stdout: "put_sum": 248408, 2024-09-10T10:04:51.805 INFO:teuthology.orchestra.run.smithi120.stdout: "wait": { 2024-09-10T10:04:51.805 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:51.805 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:51.805 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:51.805 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:04:51.805 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:51.805 INFO:teuthology.orchestra.run.smithi120.stdout: "throttle-write_buf_throttle-0x55aad34c0960": { 2024-09-10T10:04:51.805 INFO:teuthology.orchestra.run.smithi120.stdout: "val": 0, 2024-09-10T10:04:51.805 INFO:teuthology.orchestra.run.smithi120.stdout: "max": 3758096384, 2024-09-10T10:04:51.805 INFO:teuthology.orchestra.run.smithi120.stdout: "get_started": 0, 2024-09-10T10:04:51.806 INFO:teuthology.orchestra.run.smithi120.stdout: "get": 42186, 2024-09-10T10:04:51.806 INFO:teuthology.orchestra.run.smithi120.stdout: "get_sum": 90854969, 2024-09-10T10:04:51.806 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_fail": 0, 2024-09-10T10:04:51.806 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_success": 42186, 2024-09-10T10:04:51.806 INFO:teuthology.orchestra.run.smithi120.stdout: "take": 0, 2024-09-10T10:04:51.806 INFO:teuthology.orchestra.run.smithi120.stdout: "take_sum": 0, 2024-09-10T10:04:51.806 INFO:teuthology.orchestra.run.smithi120.stdout: "put": 24948, 2024-09-10T10:04:51.806 INFO:teuthology.orchestra.run.smithi120.stdout: "put_sum": 90854969, 2024-09-10T10:04:51.806 INFO:teuthology.orchestra.run.smithi120.stdout: "wait": { 2024-09-10T10:04:51.806 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:51.806 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:51.806 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:51.806 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:04:51.807 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:04:51.807 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:04:52.602 INFO:tasks.check_counter:Daemon mds.b mds.exported=163 2024-09-10T10:04:52.602 INFO:tasks.check_counter:Daemon mds.b mds.imported=1 2024-09-10T10:04:52.602 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=25 2024-09-10T10:04:52.602 DEBUG:tasks.check_counter:Getting stats from e 2024-09-10T10:04:52.602 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph daemon mds.e perf dump 2024-09-10T10:04:52.847 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T10:04:53.238 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:04:52 smithi120 ceph-mon[34770]: pgmap v682: 129 pgs: 129 active+clean; 115 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 511 B/s wr, 10 op/s 2024-09-10T10:04:53.339 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:04:53.340 INFO:teuthology.orchestra.run.smithi120.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T10:04:53.340 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_messages": 1340, 2024-09-10T10:04:53.340 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages": 1342, 2024-09-10T10:04:53.340 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_bytes": 29761088, 2024-09-10T10:04:53.340 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_bytes": 426130, 2024-09-10T10:04:53.340 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_created_connections": 8, 2024-09-10T10:04:53.340 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_active_connections": 4, 2024-09-10T10:04:53.340 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_total_time": 0.126154612, 2024-09-10T10:04:53.340 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_send_time": 0.041713458, 2024-09-10T10:04:53.340 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_recv_time": 0.042763958, 2024-09-10T10:04:53.340 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_fast_dispatch_time": 0.031294681, 2024-09-10T10:04:53.340 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:04:53.340 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 1340, 2024-09-10T10:04:53.341 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.046723834, 2024-09-10T10:04:53.341 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000034868 2024-09-10T10:04:53.341 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:53.341 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:04:53.341 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:53.341 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:53.341 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:53.341 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:53.341 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-10T10:04:53.341 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_encrypted_bytes": 0 2024-09-10T10:04:53.341 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:53.341 INFO:teuthology.orchestra.run.smithi120.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T10:04:53.341 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_messages": 594, 2024-09-10T10:04:53.341 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages": 753, 2024-09-10T10:04:53.342 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_bytes": 36482327, 2024-09-10T10:04:53.342 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_bytes": 529184, 2024-09-10T10:04:53.342 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_created_connections": 6, 2024-09-10T10:04:53.342 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_active_connections": 5, 2024-09-10T10:04:53.342 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_total_time": 0.067621107, 2024-09-10T10:04:53.342 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_send_time": 0.015024227, 2024-09-10T10:04:53.342 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_recv_time": 0.035105438, 2024-09-10T10:04:53.342 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_fast_dispatch_time": 0.013289022, 2024-09-10T10:04:53.342 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:04:53.342 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 751, 2024-09-10T10:04:53.342 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.027578512, 2024-09-10T10:04:53.342 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000036722 2024-09-10T10:04:53.342 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:53.342 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:04:53.343 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:53.343 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:53.343 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:53.343 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:53.343 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-10T10:04:53.343 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_encrypted_bytes": 340544 2024-09-10T10:04:53.343 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:53.343 INFO:teuthology.orchestra.run.smithi120.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T10:04:53.343 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_messages": 787, 2024-09-10T10:04:53.343 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages": 770, 2024-09-10T10:04:53.343 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_bytes": 25881893, 2024-09-10T10:04:53.343 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_bytes": 305719, 2024-09-10T10:04:53.343 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_created_connections": 5, 2024-09-10T10:04:53.343 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_active_connections": 4, 2024-09-10T10:04:53.344 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_total_time": 0.084173086, 2024-09-10T10:04:53.344 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_send_time": 0.025060531, 2024-09-10T10:04:53.344 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_recv_time": 0.046929943, 2024-09-10T10:04:53.344 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_fast_dispatch_time": 0.014280147, 2024-09-10T10:04:53.344 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:04:53.344 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 770, 2024-09-10T10:04:53.344 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.041925361, 2024-09-10T10:04:53.344 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000054448 2024-09-10T10:04:53.344 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:53.344 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:04:53.344 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:53.344 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:53.344 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:53.344 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:53.344 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_encrypted_bytes": 578544, 2024-09-10T10:04:53.345 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_encrypted_bytes": 131136 2024-09-10T10:04:53.345 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:53.345 INFO:teuthology.orchestra.run.smithi120.stdout: "cct": { 2024-09-10T10:04:53.345 INFO:teuthology.orchestra.run.smithi120.stdout: "total_workers": 1, 2024-09-10T10:04:53.345 INFO:teuthology.orchestra.run.smithi120.stdout: "unhealthy_workers": 0 2024-09-10T10:04:53.345 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:53.345 INFO:teuthology.orchestra.run.smithi120.stdout: "finisher-MDSRank": { 2024-09-10T10:04:53.345 INFO:teuthology.orchestra.run.smithi120.stdout: "queue_len": 0, 2024-09-10T10:04:53.345 INFO:teuthology.orchestra.run.smithi120.stdout: "complete_latency": { 2024-09-10T10:04:53.345 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 4161, 2024-09-10T10:04:53.345 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.086836992, 2024-09-10T10:04:53.345 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000020869 2024-09-10T10:04:53.345 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:04:53.345 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:53.346 INFO:teuthology.orchestra.run.smithi120.stdout: "finisher-PurgeQueue": { 2024-09-10T10:04:53.346 INFO:teuthology.orchestra.run.smithi120.stdout: "queue_len": 0, 2024-09-10T10:04:53.346 INFO:teuthology.orchestra.run.smithi120.stdout: "complete_latency": { 2024-09-10T10:04:53.346 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:53.346 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:53.346 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:53.346 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:04:53.346 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:53.346 INFO:teuthology.orchestra.run.smithi120.stdout: "mds": { 2024-09-10T10:04:53.346 INFO:teuthology.orchestra.run.smithi120.stdout: "request": 0, 2024-09-10T10:04:53.346 INFO:teuthology.orchestra.run.smithi120.stdout: "reply": 0, 2024-09-10T10:04:53.346 INFO:teuthology.orchestra.run.smithi120.stdout: "reply_latency": { 2024-09-10T10:04:53.346 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:53.346 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:53.346 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:53.347 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:53.347 INFO:teuthology.orchestra.run.smithi120.stdout: "slow_reply": 0, 2024-09-10T10:04:53.347 INFO:teuthology.orchestra.run.smithi120.stdout: "forward": 0, 2024-09-10T10:04:53.347 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_fetch_complete": 0, 2024-09-10T10:04:53.347 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_fetch_keys": 0, 2024-09-10T10:04:53.347 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_commit": 0, 2024-09-10T10:04:53.347 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_split": 0, 2024-09-10T10:04:53.347 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_merge": 0, 2024-09-10T10:04:53.347 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes": 2305, 2024-09-10T10:04:53.347 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_top": 0, 2024-09-10T10:04:53.347 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_bottom": 2305, 2024-09-10T10:04:53.347 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_pin_tail": 0, 2024-09-10T10:04:53.347 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_pinned": 2305, 2024-09-10T10:04:53.347 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_expired": 0, 2024-09-10T10:04:53.348 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_with_caps": 0, 2024-09-10T10:04:53.348 INFO:teuthology.orchestra.run.smithi120.stdout: "caps": 0, 2024-09-10T10:04:53.348 INFO:teuthology.orchestra.run.smithi120.stdout: "subtrees": 2, 2024-09-10T10:04:53.348 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse": 0, 2024-09-10T10:04:53.348 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_hit": 0, 2024-09-10T10:04:53.348 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_forward": 0, 2024-09-10T10:04:53.348 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_discover": 0, 2024-09-10T10:04:53.348 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_dir_fetch": 0, 2024-09-10T10:04:53.348 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_remote_ino": 0, 2024-09-10T10:04:53.348 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_lock": 0, 2024-09-10T10:04:53.348 INFO:teuthology.orchestra.run.smithi120.stdout: "load_cent": 0, 2024-09-10T10:04:53.348 INFO:teuthology.orchestra.run.smithi120.stdout: "q": 0, 2024-09-10T10:04:53.348 INFO:teuthology.orchestra.run.smithi120.stdout: "exported": 0, 2024-09-10T10:04:53.348 INFO:teuthology.orchestra.run.smithi120.stdout: "exported_inodes": 0, 2024-09-10T10:04:53.349 INFO:teuthology.orchestra.run.smithi120.stdout: "imported": 0, 2024-09-10T10:04:53.349 INFO:teuthology.orchestra.run.smithi120.stdout: "imported_inodes": 0, 2024-09-10T10:04:53.349 INFO:teuthology.orchestra.run.smithi120.stdout: "openino_dir_fetch": 0, 2024-09-10T10:04:53.349 INFO:teuthology.orchestra.run.smithi120.stdout: "openino_backtrace_fetch": 0, 2024-09-10T10:04:53.349 INFO:teuthology.orchestra.run.smithi120.stdout: "openino_peer_discover": 0, 2024-09-10T10:04:53.349 INFO:teuthology.orchestra.run.smithi120.stdout: "root_rfiles": 4, 2024-09-10T10:04:53.349 INFO:teuthology.orchestra.run.smithi120.stdout: "root_rbytes": 234, 2024-09-10T10:04:53.349 INFO:teuthology.orchestra.run.smithi120.stdout: "root_rsnaps": 0, 2024-09-10T10:04:53.349 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_backtrace_fetch": 0, 2024-09-10T10:04:53.349 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_set_tag": 0, 2024-09-10T10:04:53.349 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T10:04:53.349 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_inotable_repaired": 0, 2024-09-10T10:04:53.349 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_dir_inodes": 0, 2024-09-10T10:04:53.349 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_dir_base_inodes": 0, 2024-09-10T10:04:53.350 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_dirfrag_rstats": 0, 2024-09-10T10:04:53.350 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_file_inodes": 0, 2024-09-10T10:04:53.350 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_inode_file_caps": 0, 2024-09-10T10:04:53.350 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_cap_op_revoke": 0, 2024-09-10T10:04:53.350 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_cap_op_grant": 0, 2024-09-10T10:04:53.350 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T10:04:53.350 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T10:04:53.350 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T10:04:53.350 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_client_caps": 0, 2024-09-10T10:04:53.350 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_client_caps_dirty": 0, 2024-09-10T10:04:53.350 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_client_cap_release": 0, 2024-09-10T10:04:53.350 INFO:teuthology.orchestra.run.smithi120.stdout: "process_request_cap_release": 0 2024-09-10T10:04:53.350 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:53.350 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_cache": { 2024-09-10T10:04:53.350 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_update": 0, 2024-09-10T10:04:53.351 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_update_receipt": 0, 2024-09-10T10:04:53.351 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_try_discover": 0, 2024-09-10T10:04:53.351 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_send_discover": 0, 2024-09-10T10:04:53.351 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_handle_discover": 0, 2024-09-10T10:04:53.351 INFO:teuthology.orchestra.run.smithi120.stdout: "num_strays": 0, 2024-09-10T10:04:53.351 INFO:teuthology.orchestra.run.smithi120.stdout: "num_strays_delayed": 0, 2024-09-10T10:04:53.351 INFO:teuthology.orchestra.run.smithi120.stdout: "num_strays_enqueuing": 0, 2024-09-10T10:04:53.351 INFO:teuthology.orchestra.run.smithi120.stdout: "strays_created": 0, 2024-09-10T10:04:53.351 INFO:teuthology.orchestra.run.smithi120.stdout: "strays_enqueued": 0, 2024-09-10T10:04:53.351 INFO:teuthology.orchestra.run.smithi120.stdout: "strays_reintegrated": 0, 2024-09-10T10:04:53.351 INFO:teuthology.orchestra.run.smithi120.stdout: "strays_migrated": 0, 2024-09-10T10:04:53.351 INFO:teuthology.orchestra.run.smithi120.stdout: "num_recovering_processing": 0, 2024-09-10T10:04:53.351 INFO:teuthology.orchestra.run.smithi120.stdout: "num_recovering_enqueued": 0, 2024-09-10T10:04:53.351 INFO:teuthology.orchestra.run.smithi120.stdout: "num_recovering_prioritized": 0, 2024-09-10T10:04:53.351 INFO:teuthology.orchestra.run.smithi120.stdout: "recovery_started": 0, 2024-09-10T10:04:53.351 INFO:teuthology.orchestra.run.smithi120.stdout: "recovery_completed": 0, 2024-09-10T10:04:53.352 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_quiesce_path": 0, 2024-09-10T10:04:53.352 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_quiesce_inode": 0, 2024-09-10T10:04:53.352 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_enqueue_scrub": 0, 2024-09-10T10:04:53.352 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_exportdir": 0, 2024-09-10T10:04:53.352 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_flush": 0, 2024-09-10T10:04:53.352 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_fragmentdir": 0, 2024-09-10T10:04:53.352 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_fragstats": 0, 2024-09-10T10:04:53.352 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_inodestats": 0 2024-09-10T10:04:53.352 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:53.352 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_log": { 2024-09-10T10:04:53.352 INFO:teuthology.orchestra.run.smithi120.stdout: "evlrg": 0, 2024-09-10T10:04:53.352 INFO:teuthology.orchestra.run.smithi120.stdout: "evadd": 0, 2024-09-10T10:04:53.352 INFO:teuthology.orchestra.run.smithi120.stdout: "evex": 0, 2024-09-10T10:04:53.352 INFO:teuthology.orchestra.run.smithi120.stdout: "evtrm": 0, 2024-09-10T10:04:53.352 INFO:teuthology.orchestra.run.smithi120.stdout: "ev": 42186, 2024-09-10T10:04:53.352 INFO:teuthology.orchestra.run.smithi120.stdout: "evexg": 0, 2024-09-10T10:04:53.353 INFO:teuthology.orchestra.run.smithi120.stdout: "evexd": 0, 2024-09-10T10:04:53.353 INFO:teuthology.orchestra.run.smithi120.stdout: "segadd": 0, 2024-09-10T10:04:53.353 INFO:teuthology.orchestra.run.smithi120.stdout: "segex": 0, 2024-09-10T10:04:53.353 INFO:teuthology.orchestra.run.smithi120.stdout: "segtrm": 0, 2024-09-10T10:04:53.353 INFO:teuthology.orchestra.run.smithi120.stdout: "seg": 50, 2024-09-10T10:04:53.353 INFO:teuthology.orchestra.run.smithi120.stdout: "segmjr": 5, 2024-09-10T10:04:53.353 INFO:teuthology.orchestra.run.smithi120.stdout: "segexg": 0, 2024-09-10T10:04:53.353 INFO:teuthology.orchestra.run.smithi120.stdout: "segexd": 0, 2024-09-10T10:04:53.353 INFO:teuthology.orchestra.run.smithi120.stdout: "expos": 4194304, 2024-09-10T10:04:53.353 INFO:teuthology.orchestra.run.smithi120.stdout: "wrpos": 95049273, 2024-09-10T10:04:53.353 INFO:teuthology.orchestra.run.smithi120.stdout: "rdpos": 95044972, 2024-09-10T10:04:53.353 INFO:teuthology.orchestra.run.smithi120.stdout: "jlat": { 2024-09-10T10:04:53.353 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:53.353 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:53.354 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:53.354 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:53.354 INFO:teuthology.orchestra.run.smithi120.stdout: "replayed": 42186 2024-09-10T10:04:53.354 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:53.354 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_mem": { 2024-09-10T10:04:53.354 INFO:teuthology.orchestra.run.smithi120.stdout: "ino": 22, 2024-09-10T10:04:53.354 INFO:teuthology.orchestra.run.smithi120.stdout: "ino+": 6765, 2024-09-10T10:04:53.354 INFO:teuthology.orchestra.run.smithi120.stdout: "ino-": 6743, 2024-09-10T10:04:53.354 INFO:teuthology.orchestra.run.smithi120.stdout: "dir": 17, 2024-09-10T10:04:53.354 INFO:teuthology.orchestra.run.smithi120.stdout: "dir+": 283, 2024-09-10T10:04:53.354 INFO:teuthology.orchestra.run.smithi120.stdout: "dir-": 266, 2024-09-10T10:04:53.354 INFO:teuthology.orchestra.run.smithi120.stdout: "dn": 2305, 2024-09-10T10:04:53.354 INFO:teuthology.orchestra.run.smithi120.stdout: "dn+": 6937, 2024-09-10T10:04:53.354 INFO:teuthology.orchestra.run.smithi120.stdout: "dn-": 4632, 2024-09-10T10:04:53.354 INFO:teuthology.orchestra.run.smithi120.stdout: "cap": 0, 2024-09-10T10:04:53.354 INFO:teuthology.orchestra.run.smithi120.stdout: "cap+": 0, 2024-09-10T10:04:53.355 INFO:teuthology.orchestra.run.smithi120.stdout: "cap-": 0, 2024-09-10T10:04:53.355 INFO:teuthology.orchestra.run.smithi120.stdout: "rss": 66860, 2024-09-10T10:04:53.355 INFO:teuthology.orchestra.run.smithi120.stdout: "heap": 215328 2024-09-10T10:04:53.355 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:53.355 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_server": { 2024-09-10T10:04:53.355 INFO:teuthology.orchestra.run.smithi120.stdout: "dispatch_client_request": 0, 2024-09-10T10:04:53.355 INFO:teuthology.orchestra.run.smithi120.stdout: "dispatch_server_request": 0, 2024-09-10T10:04:53.355 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_client_request": 0, 2024-09-10T10:04:53.355 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_client_session": 0, 2024-09-10T10:04:53.355 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_peer_request": 0, 2024-09-10T10:04:53.355 INFO:teuthology.orchestra.run.smithi120.stdout: "req_create_latency": { 2024-09-10T10:04:53.355 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:53.355 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:53.355 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:53.356 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:53.356 INFO:teuthology.orchestra.run.smithi120.stdout: "req_getattr_latency": { 2024-09-10T10:04:53.356 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:53.356 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:53.356 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:53.356 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:53.356 INFO:teuthology.orchestra.run.smithi120.stdout: "req_getfilelock_latency": { 2024-09-10T10:04:53.356 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:53.356 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:53.356 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:53.356 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:53.356 INFO:teuthology.orchestra.run.smithi120.stdout: "req_link_latency": { 2024-09-10T10:04:53.356 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:53.356 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:53.356 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:53.357 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:53.357 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookup_latency": { 2024-09-10T10:04:53.357 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:53.357 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:53.357 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:53.357 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:53.357 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookuphash_latency": { 2024-09-10T10:04:53.357 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:53.357 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:53.357 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:53.357 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:53.357 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookupino_latency": { 2024-09-10T10:04:53.357 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:53.357 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:53.358 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:53.358 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:53.358 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookupname_latency": { 2024-09-10T10:04:53.358 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:53.358 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:53.358 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:53.358 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:53.358 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookupparent_latency": { 2024-09-10T10:04:53.358 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:53.358 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:53.358 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:53.358 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:53.358 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookupsnap_latency": { 2024-09-10T10:04:53.358 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:53.358 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:53.358 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:53.358 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:53.359 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lssnap_latency": { 2024-09-10T10:04:53.359 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:53.359 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:53.359 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:53.359 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:53.359 INFO:teuthology.orchestra.run.smithi120.stdout: "req_mkdir_latency": { 2024-09-10T10:04:53.359 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:53.359 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:53.359 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:53.360 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:53.360 INFO:teuthology.orchestra.run.smithi120.stdout: "req_mknod_latency": { 2024-09-10T10:04:53.360 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:53.360 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:53.360 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:53.360 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:53.360 INFO:teuthology.orchestra.run.smithi120.stdout: "req_mksnap_latency": { 2024-09-10T10:04:53.360 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:53.360 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:53.360 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:53.360 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:53.360 INFO:teuthology.orchestra.run.smithi120.stdout: "req_open_latency": { 2024-09-10T10:04:53.361 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:53.361 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:53.361 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:53.361 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:53.361 INFO:teuthology.orchestra.run.smithi120.stdout: "req_readdir_latency": { 2024-09-10T10:04:53.361 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:53.361 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:53.361 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:53.361 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:53.361 INFO:teuthology.orchestra.run.smithi120.stdout: "req_rename_latency": { 2024-09-10T10:04:53.361 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:53.361 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:53.361 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:53.361 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:53.361 INFO:teuthology.orchestra.run.smithi120.stdout: "req_renamesnap_latency": { 2024-09-10T10:04:53.362 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:53.362 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:53.362 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:53.362 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:53.362 INFO:teuthology.orchestra.run.smithi120.stdout: "req_snapdiff_latency": { 2024-09-10T10:04:53.362 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:53.362 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:53.362 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:53.362 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:53.362 INFO:teuthology.orchestra.run.smithi120.stdout: "req_rmdir_latency": { 2024-09-10T10:04:53.362 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:53.362 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:53.362 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:53.362 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:53.363 INFO:teuthology.orchestra.run.smithi120.stdout: "req_rmsnap_latency": { 2024-09-10T10:04:53.363 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:53.363 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:53.363 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:53.363 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:53.363 INFO:teuthology.orchestra.run.smithi120.stdout: "req_rmxattr_latency": { 2024-09-10T10:04:53.363 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:53.363 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:53.363 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:53.363 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:53.363 INFO:teuthology.orchestra.run.smithi120.stdout: "req_setattr_latency": { 2024-09-10T10:04:53.363 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:53.363 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:53.363 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:53.364 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:53.364 INFO:teuthology.orchestra.run.smithi120.stdout: "req_setdirlayout_latency": { 2024-09-10T10:04:53.364 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:53.364 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:53.364 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:53.364 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:53.364 INFO:teuthology.orchestra.run.smithi120.stdout: "req_setfilelock_latency": { 2024-09-10T10:04:53.364 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:53.364 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:53.364 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:53.364 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:53.364 INFO:teuthology.orchestra.run.smithi120.stdout: "req_setlayout_latency": { 2024-09-10T10:04:53.364 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:53.364 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:53.365 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:53.365 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:53.365 INFO:teuthology.orchestra.run.smithi120.stdout: "req_setxattr_latency": { 2024-09-10T10:04:53.365 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:53.365 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:53.365 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:53.365 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:53.365 INFO:teuthology.orchestra.run.smithi120.stdout: "req_symlink_latency": { 2024-09-10T10:04:53.365 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:53.365 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:53.365 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:53.365 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:53.365 INFO:teuthology.orchestra.run.smithi120.stdout: "req_unlink_latency": { 2024-09-10T10:04:53.365 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:53.365 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:53.366 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:53.366 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:53.366 INFO:teuthology.orchestra.run.smithi120.stdout: "cap_revoke_eviction": 0, 2024-09-10T10:04:53.366 INFO:teuthology.orchestra.run.smithi120.stdout: "cap_acquisition_throttle": 0, 2024-09-10T10:04:53.366 INFO:teuthology.orchestra.run.smithi120.stdout: "req_getvxattr_latency": { 2024-09-10T10:04:53.366 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:53.366 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:53.366 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:53.366 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:04:53.366 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:53.366 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_sessions": { 2024-09-10T10:04:53.366 INFO:teuthology.orchestra.run.smithi120.stdout: "session_count": 1, 2024-09-10T10:04:53.366 INFO:teuthology.orchestra.run.smithi120.stdout: "session_add": 3, 2024-09-10T10:04:53.366 INFO:teuthology.orchestra.run.smithi120.stdout: "session_remove": 2, 2024-09-10T10:04:53.366 INFO:teuthology.orchestra.run.smithi120.stdout: "sessions_open": 1, 2024-09-10T10:04:53.366 INFO:teuthology.orchestra.run.smithi120.stdout: "sessions_stale": 0, 2024-09-10T10:04:53.367 INFO:teuthology.orchestra.run.smithi120.stdout: "total_load": 0, 2024-09-10T10:04:53.367 INFO:teuthology.orchestra.run.smithi120.stdout: "average_load": 0, 2024-09-10T10:04:53.367 INFO:teuthology.orchestra.run.smithi120.stdout: "avg_session_uptime": 1945, 2024-09-10T10:04:53.367 INFO:teuthology.orchestra.run.smithi120.stdout: "mdthresh_evicted": 0 2024-09-10T10:04:53.367 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:53.367 INFO:teuthology.orchestra.run.smithi120.stdout: "mempool": { 2024-09-10T10:04:53.367 INFO:teuthology.orchestra.run.smithi120.stdout: "bloom_filter_bytes": 0, 2024-09-10T10:04:53.367 INFO:teuthology.orchestra.run.smithi120.stdout: "bloom_filter_items": 0, 2024-09-10T10:04:53.367 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T10:04:53.367 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_alloc_items": 0, 2024-09-10T10:04:53.367 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T10:04:53.367 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_data_items": 0, 2024-09-10T10:04:53.367 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T10:04:53.367 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T10:04:53.367 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T10:04:53.368 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T10:04:53.368 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T10:04:53.368 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_other_items": 0, 2024-09-10T10:04:53.368 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T10:04:53.368 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T10:04:53.368 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_extent_bytes": 0, 2024-09-10T10:04:53.368 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_extent_items": 0, 2024-09-10T10:04:53.368 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_blob_bytes": 0, 2024-09-10T10:04:53.368 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_blob_items": 0, 2024-09-10T10:04:53.368 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T10:04:53.368 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T10:04:53.368 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T10:04:53.368 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T10:04:53.368 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T10:04:53.368 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_fsck_items": 0, 2024-09-10T10:04:53.369 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_txc_bytes": 0, 2024-09-10T10:04:53.369 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_txc_items": 0, 2024-09-10T10:04:53.369 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T10:04:53.369 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T10:04:53.369 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_writing_bytes": 0, 2024-09-10T10:04:53.369 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_writing_items": 0, 2024-09-10T10:04:53.369 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_bytes": 0, 2024-09-10T10:04:53.369 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_items": 0, 2024-09-10T10:04:53.369 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T10:04:53.369 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_file_reader_items": 0, 2024-09-10T10:04:53.369 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T10:04:53.369 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_file_writer_items": 0, 2024-09-10T10:04:53.369 INFO:teuthology.orchestra.run.smithi120.stdout: "buffer_anon_bytes": 232113, 2024-09-10T10:04:53.369 INFO:teuthology.orchestra.run.smithi120.stdout: "buffer_anon_items": 76, 2024-09-10T10:04:53.370 INFO:teuthology.orchestra.run.smithi120.stdout: "buffer_meta_bytes": 0, 2024-09-10T10:04:53.370 INFO:teuthology.orchestra.run.smithi120.stdout: "buffer_meta_items": 0, 2024-09-10T10:04:53.370 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_bytes": 0, 2024-09-10T10:04:53.370 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_items": 0, 2024-09-10T10:04:53.370 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_mapbl_bytes": 0, 2024-09-10T10:04:53.370 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_mapbl_items": 0, 2024-09-10T10:04:53.370 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pglog_bytes": 0, 2024-09-10T10:04:53.370 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pglog_items": 0, 2024-09-10T10:04:53.370 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_bytes": 9144, 2024-09-10T10:04:53.370 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_items": 172, 2024-09-10T10:04:53.370 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T10:04:53.370 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_mapping_items": 0, 2024-09-10T10:04:53.370 INFO:teuthology.orchestra.run.smithi120.stdout: "pgmap_bytes": 0, 2024-09-10T10:04:53.371 INFO:teuthology.orchestra.run.smithi120.stdout: "pgmap_items": 0, 2024-09-10T10:04:53.371 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_co_bytes": 1693770, 2024-09-10T10:04:53.371 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_co_items": 9502, 2024-09-10T10:04:53.371 INFO:teuthology.orchestra.run.smithi120.stdout: "unittest_1_bytes": 0, 2024-09-10T10:04:53.371 INFO:teuthology.orchestra.run.smithi120.stdout: "unittest_1_items": 0, 2024-09-10T10:04:53.371 INFO:teuthology.orchestra.run.smithi120.stdout: "unittest_2_bytes": 0, 2024-09-10T10:04:53.371 INFO:teuthology.orchestra.run.smithi120.stdout: "unittest_2_items": 0 2024-09-10T10:04:53.371 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:53.371 INFO:teuthology.orchestra.run.smithi120.stdout: "objecter": { 2024-09-10T10:04:53.371 INFO:teuthology.orchestra.run.smithi120.stdout: "op_active": 0, 2024-09-10T10:04:53.371 INFO:teuthology.orchestra.run.smithi120.stdout: "op_laggy": 0, 2024-09-10T10:04:53.371 INFO:teuthology.orchestra.run.smithi120.stdout: "op_send": 2482, 2024-09-10T10:04:53.371 INFO:teuthology.orchestra.run.smithi120.stdout: "op_send_bytes": 26, 2024-09-10T10:04:53.371 INFO:teuthology.orchestra.run.smithi120.stdout: "op_resend": 4, 2024-09-10T10:04:53.371 INFO:teuthology.orchestra.run.smithi120.stdout: "op_reply": 2482, 2024-09-10T10:04:53.372 INFO:teuthology.orchestra.run.smithi120.stdout: "op_latency": { 2024-09-10T10:04:53.372 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 2482, 2024-09-10T10:04:53.372 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 2.355945119, 2024-09-10T10:04:53.372 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000949212 2024-09-10T10:04:53.372 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:53.372 INFO:teuthology.orchestra.run.smithi120.stdout: "op_inflight": 0, 2024-09-10T10:04:53.372 INFO:teuthology.orchestra.run.smithi120.stdout: "oplen_avg": { 2024-09-10T10:04:53.372 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 2482, 2024-09-10T10:04:53.372 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 2483 2024-09-10T10:04:53.372 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:53.372 INFO:teuthology.orchestra.run.smithi120.stdout: "op": 2482, 2024-09-10T10:04:53.372 INFO:teuthology.orchestra.run.smithi120.stdout: "op_r": 2482, 2024-09-10T10:04:53.372 INFO:teuthology.orchestra.run.smithi120.stdout: "op_w": 0, 2024-09-10T10:04:53.372 INFO:teuthology.orchestra.run.smithi120.stdout: "op_rmw": 0, 2024-09-10T10:04:53.373 INFO:teuthology.orchestra.run.smithi120.stdout: "op_pg": 0, 2024-09-10T10:04:53.373 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_stat": 1420, 2024-09-10T10:04:53.373 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_create": 0, 2024-09-10T10:04:53.373 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_read": 1059, 2024-09-10T10:04:53.373 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_write": 0, 2024-09-10T10:04:53.373 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_writefull": 0, 2024-09-10T10:04:53.373 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_writesame": 0, 2024-09-10T10:04:53.373 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_append": 0, 2024-09-10T10:04:53.373 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_zero": 0, 2024-09-10T10:04:53.373 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_truncate": 0, 2024-09-10T10:04:53.373 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_delete": 0, 2024-09-10T10:04:53.373 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_mapext": 0, 2024-09-10T10:04:53.373 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_sparse_read": 0, 2024-09-10T10:04:53.373 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_clonerange": 0, 2024-09-10T10:04:53.373 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_getxattr": 2, 2024-09-10T10:04:53.374 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_setxattr": 0, 2024-09-10T10:04:53.374 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_cmpxattr": 0, 2024-09-10T10:04:53.374 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_rmxattr": 0, 2024-09-10T10:04:53.374 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_resetxattrs": 0, 2024-09-10T10:04:53.374 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_call": 0, 2024-09-10T10:04:53.374 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_watch": 0, 2024-09-10T10:04:53.374 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_notify": 0, 2024-09-10T10:04:53.374 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T10:04:53.374 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_pgls": 0, 2024-09-10T10:04:53.374 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_pgls_filter": 0, 2024-09-10T10:04:53.374 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_other": 0, 2024-09-10T10:04:53.374 INFO:teuthology.orchestra.run.smithi120.stdout: "linger_active": 0, 2024-09-10T10:04:53.374 INFO:teuthology.orchestra.run.smithi120.stdout: "linger_send": 0, 2024-09-10T10:04:53.374 INFO:teuthology.orchestra.run.smithi120.stdout: "linger_resend": 0, 2024-09-10T10:04:53.374 INFO:teuthology.orchestra.run.smithi120.stdout: "linger_ping": 0, 2024-09-10T10:04:53.375 INFO:teuthology.orchestra.run.smithi120.stdout: "poolop_active": 0, 2024-09-10T10:04:53.375 INFO:teuthology.orchestra.run.smithi120.stdout: "poolop_send": 0, 2024-09-10T10:04:53.375 INFO:teuthology.orchestra.run.smithi120.stdout: "poolop_resend": 0, 2024-09-10T10:04:53.375 INFO:teuthology.orchestra.run.smithi120.stdout: "poolstat_active": 0, 2024-09-10T10:04:53.375 INFO:teuthology.orchestra.run.smithi120.stdout: "poolstat_send": 0, 2024-09-10T10:04:53.375 INFO:teuthology.orchestra.run.smithi120.stdout: "poolstat_resend": 0, 2024-09-10T10:04:53.375 INFO:teuthology.orchestra.run.smithi120.stdout: "statfs_active": 0, 2024-09-10T10:04:53.375 INFO:teuthology.orchestra.run.smithi120.stdout: "statfs_send": 0, 2024-09-10T10:04:53.375 INFO:teuthology.orchestra.run.smithi120.stdout: "statfs_resend": 0, 2024-09-10T10:04:53.375 INFO:teuthology.orchestra.run.smithi120.stdout: "command_active": 0, 2024-09-10T10:04:53.375 INFO:teuthology.orchestra.run.smithi120.stdout: "command_send": 0, 2024-09-10T10:04:53.376 INFO:teuthology.orchestra.run.smithi120.stdout: "command_resend": 0, 2024-09-10T10:04:53.376 INFO:teuthology.orchestra.run.smithi120.stdout: "map_epoch": 96, 2024-09-10T10:04:53.376 INFO:teuthology.orchestra.run.smithi120.stdout: "map_full": 0, 2024-09-10T10:04:53.376 INFO:teuthology.orchestra.run.smithi120.stdout: "map_inc": 2, 2024-09-10T10:04:53.376 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_sessions": 10, 2024-09-10T10:04:53.376 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_session_open": 10, 2024-09-10T10:04:53.376 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_session_close": 0, 2024-09-10T10:04:53.376 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_laggy": 0, 2024-09-10T10:04:53.376 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_wr": 0, 2024-09-10T10:04:53.376 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_rd": 2, 2024-09-10T10:04:53.376 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_del": 0 2024-09-10T10:04:53.376 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:53.376 INFO:teuthology.orchestra.run.smithi120.stdout: "oft": { 2024-09-10T10:04:53.376 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_total_objs": 0, 2024-09-10T10:04:53.377 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_total_kv_pairs": 0, 2024-09-10T10:04:53.377 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_total_updates": 0, 2024-09-10T10:04:53.377 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_total_removes": 0 2024-09-10T10:04:53.377 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:53.377 INFO:teuthology.orchestra.run.smithi120.stdout: "osd-slow-ops": { 2024-09-10T10:04:53.377 INFO:teuthology.orchestra.run.smithi120.stdout: "slow_ops_count": 0 2024-09-10T10:04:53.377 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:53.377 INFO:teuthology.orchestra.run.smithi120.stdout: "purge_queue": { 2024-09-10T10:04:53.377 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executing_ops": 0, 2024-09-10T10:04:53.377 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executing_ops_high_water": 0, 2024-09-10T10:04:53.377 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executing": 0, 2024-09-10T10:04:53.377 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executing_high_water": 0, 2024-09-10T10:04:53.377 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executed_ops": 0, 2024-09-10T10:04:53.377 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executed": 0, 2024-09-10T10:04:53.378 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_item_in_journal": 0 2024-09-10T10:04:53.378 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:53.378 INFO:teuthology.orchestra.run.smithi120.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T10:04:53.378 INFO:teuthology.orchestra.run.smithi120.stdout: "val": 0, 2024-09-10T10:04:53.378 INFO:teuthology.orchestra.run.smithi120.stdout: "max": 104857600, 2024-09-10T10:04:53.378 INFO:teuthology.orchestra.run.smithi120.stdout: "get_started": 0, 2024-09-10T10:04:53.378 INFO:teuthology.orchestra.run.smithi120.stdout: "get": 2717, 2024-09-10T10:04:53.378 INFO:teuthology.orchestra.run.smithi120.stdout: "get_sum": 91865808, 2024-09-10T10:04:53.378 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_fail": 0, 2024-09-10T10:04:53.378 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_success": 2717, 2024-09-10T10:04:53.378 INFO:teuthology.orchestra.run.smithi120.stdout: "take": 0, 2024-09-10T10:04:53.378 INFO:teuthology.orchestra.run.smithi120.stdout: "take_sum": 0, 2024-09-10T10:04:53.378 INFO:teuthology.orchestra.run.smithi120.stdout: "put": 2717, 2024-09-10T10:04:53.378 INFO:teuthology.orchestra.run.smithi120.stdout: "put_sum": 91865808, 2024-09-10T10:04:53.378 INFO:teuthology.orchestra.run.smithi120.stdout: "wait": { 2024-09-10T10:04:53.378 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:53.379 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:53.379 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:53.379 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:04:53.379 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:53.379 INFO:teuthology.orchestra.run.smithi120.stdout: "throttle-objecter_bytes": { 2024-09-10T10:04:53.379 INFO:teuthology.orchestra.run.smithi120.stdout: "val": 0, 2024-09-10T10:04:53.379 INFO:teuthology.orchestra.run.smithi120.stdout: "max": 104857600, 2024-09-10T10:04:53.379 INFO:teuthology.orchestra.run.smithi120.stdout: "get_started": 0, 2024-09-10T10:04:53.379 INFO:teuthology.orchestra.run.smithi120.stdout: "get": 0, 2024-09-10T10:04:53.379 INFO:teuthology.orchestra.run.smithi120.stdout: "get_sum": 0, 2024-09-10T10:04:53.379 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_fail": 0, 2024-09-10T10:04:53.379 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_success": 0, 2024-09-10T10:04:53.379 INFO:teuthology.orchestra.run.smithi120.stdout: "take": 2482, 2024-09-10T10:04:53.379 INFO:teuthology.orchestra.run.smithi120.stdout: "take_sum": 90858440, 2024-09-10T10:04:53.379 INFO:teuthology.orchestra.run.smithi120.stdout: "put": 2482, 2024-09-10T10:04:53.380 INFO:teuthology.orchestra.run.smithi120.stdout: "put_sum": 90858440, 2024-09-10T10:04:53.380 INFO:teuthology.orchestra.run.smithi120.stdout: "wait": { 2024-09-10T10:04:53.380 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:53.380 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:53.380 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:53.380 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:04:53.380 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:53.380 INFO:teuthology.orchestra.run.smithi120.stdout: "throttle-objecter_ops": { 2024-09-10T10:04:53.380 INFO:teuthology.orchestra.run.smithi120.stdout: "val": 0, 2024-09-10T10:04:53.380 INFO:teuthology.orchestra.run.smithi120.stdout: "max": 1024, 2024-09-10T10:04:53.380 INFO:teuthology.orchestra.run.smithi120.stdout: "get_started": 0, 2024-09-10T10:04:53.380 INFO:teuthology.orchestra.run.smithi120.stdout: "get": 0, 2024-09-10T10:04:53.381 INFO:teuthology.orchestra.run.smithi120.stdout: "get_sum": 0, 2024-09-10T10:04:53.381 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_fail": 0, 2024-09-10T10:04:53.381 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_success": 0, 2024-09-10T10:04:53.381 INFO:teuthology.orchestra.run.smithi120.stdout: "take": 2482, 2024-09-10T10:04:53.381 INFO:teuthology.orchestra.run.smithi120.stdout: "take_sum": 2482, 2024-09-10T10:04:53.381 INFO:teuthology.orchestra.run.smithi120.stdout: "put": 2482, 2024-09-10T10:04:53.381 INFO:teuthology.orchestra.run.smithi120.stdout: "put_sum": 2482, 2024-09-10T10:04:53.381 INFO:teuthology.orchestra.run.smithi120.stdout: "wait": { 2024-09-10T10:04:53.381 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:53.381 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:53.381 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:53.381 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:04:53.381 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:53.381 INFO:teuthology.orchestra.run.smithi120.stdout: "throttle-write_buf_throttle": { 2024-09-10T10:04:53.382 INFO:teuthology.orchestra.run.smithi120.stdout: "val": 0, 2024-09-10T10:04:53.382 INFO:teuthology.orchestra.run.smithi120.stdout: "max": 3758096384, 2024-09-10T10:04:53.382 INFO:teuthology.orchestra.run.smithi120.stdout: "get_started": 0, 2024-09-10T10:04:53.382 INFO:teuthology.orchestra.run.smithi120.stdout: "get": 0, 2024-09-10T10:04:53.382 INFO:teuthology.orchestra.run.smithi120.stdout: "get_sum": 0, 2024-09-10T10:04:53.382 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_fail": 0, 2024-09-10T10:04:53.382 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_success": 0, 2024-09-10T10:04:53.382 INFO:teuthology.orchestra.run.smithi120.stdout: "take": 0, 2024-09-10T10:04:53.382 INFO:teuthology.orchestra.run.smithi120.stdout: "take_sum": 0, 2024-09-10T10:04:53.382 INFO:teuthology.orchestra.run.smithi120.stdout: "put": 0, 2024-09-10T10:04:53.382 INFO:teuthology.orchestra.run.smithi120.stdout: "put_sum": 0, 2024-09-10T10:04:53.382 INFO:teuthology.orchestra.run.smithi120.stdout: "wait": { 2024-09-10T10:04:53.383 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:53.383 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:53.383 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:53.383 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:04:53.383 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:53.383 INFO:teuthology.orchestra.run.smithi120.stdout: "throttle-write_buf_throttle-0x5634c84fcb40": { 2024-09-10T10:04:53.383 INFO:teuthology.orchestra.run.smithi120.stdout: "val": 0, 2024-09-10T10:04:53.383 INFO:teuthology.orchestra.run.smithi120.stdout: "max": 3758096384, 2024-09-10T10:04:53.383 INFO:teuthology.orchestra.run.smithi120.stdout: "get_started": 0, 2024-09-10T10:04:53.383 INFO:teuthology.orchestra.run.smithi120.stdout: "get": 0, 2024-09-10T10:04:53.383 INFO:teuthology.orchestra.run.smithi120.stdout: "get_sum": 0, 2024-09-10T10:04:53.383 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_fail": 0, 2024-09-10T10:04:53.383 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_success": 0, 2024-09-10T10:04:53.384 INFO:teuthology.orchestra.run.smithi120.stdout: "take": 0, 2024-09-10T10:04:53.384 INFO:teuthology.orchestra.run.smithi120.stdout: "take_sum": 0, 2024-09-10T10:04:53.384 INFO:teuthology.orchestra.run.smithi120.stdout: "put": 0, 2024-09-10T10:04:53.384 INFO:teuthology.orchestra.run.smithi120.stdout: "put_sum": 0, 2024-09-10T10:04:53.384 INFO:teuthology.orchestra.run.smithi120.stdout: "wait": { 2024-09-10T10:04:53.384 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:53.384 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:53.384 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:53.384 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:04:53.384 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:04:53.384 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:04:53.385 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:04:52 smithi134 ceph-mon[35674]: pgmap v682: 129 pgs: 129 active+clean; 115 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 511 B/s wr, 10 op/s 2024-09-10T10:04:53.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:04:52 smithi077 ceph-mon[30319]: pgmap v682: 129 pgs: 129 active+clean; 115 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 511 B/s wr, 10 op/s 2024-09-10T10:04:54.187 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-09-10T10:04:54.187 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-09-10T10:04:54.187 INFO:tasks.check_counter:Daemon mds.e mds.dir_split=0 2024-09-10T10:04:54.187 DEBUG:tasks.check_counter:Getting stats from h 2024-09-10T10:04:54.188 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph daemon mds.h perf dump 2024-09-10T10:04:54.433 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T10:04:54.933 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:04:54.933 INFO:teuthology.orchestra.run.smithi120.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T10:04:54.933 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_messages": 8464, 2024-09-10T10:04:54.934 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages": 11878, 2024-09-10T10:04:54.934 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_bytes": 2989957, 2024-09-10T10:04:54.934 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_bytes": 12470421, 2024-09-10T10:04:54.934 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_created_connections": 22, 2024-09-10T10:04:54.934 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_active_connections": 6, 2024-09-10T10:04:54.934 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_total_time": 0.508628965, 2024-09-10T10:04:54.934 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_send_time": 0.206279423, 2024-09-10T10:04:54.934 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_recv_time": 0.230769411, 2024-09-10T10:04:54.934 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_fast_dispatch_time": 0.089434140, 2024-09-10T10:04:54.934 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:04:54.934 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 11873, 2024-09-10T10:04:54.934 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.448124372, 2024-09-10T10:04:54.935 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000037743 2024-09-10T10:04:54.935 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:54.935 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:04:54.935 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 5003, 2024-09-10T10:04:54.935 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.001418544, 2024-09-10T10:04:54.935 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000283 2024-09-10T10:04:54.935 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:54.935 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_encrypted_bytes": 606096, 2024-09-10T10:04:54.935 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_encrypted_bytes": 143008 2024-09-10T10:04:54.935 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:54.935 INFO:teuthology.orchestra.run.smithi120.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T10:04:54.936 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_messages": 5433, 2024-09-10T10:04:54.936 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages": 5857, 2024-09-10T10:04:54.936 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_bytes": 2962518, 2024-09-10T10:04:54.936 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_bytes": 2265471, 2024-09-10T10:04:54.936 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_created_connections": 10, 2024-09-10T10:04:54.936 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_active_connections": 6, 2024-09-10T10:04:54.936 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_total_time": 0.380988964, 2024-09-10T10:04:54.936 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_send_time": 0.153847777, 2024-09-10T10:04:54.936 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_recv_time": 0.185223555, 2024-09-10T10:04:54.936 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_fast_dispatch_time": 0.057086790, 2024-09-10T10:04:54.936 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:04:54.936 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 5858, 2024-09-10T10:04:54.936 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.195448127, 2024-09-10T10:04:54.937 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000033364 2024-09-10T10:04:54.937 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:54.937 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:04:54.937 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 6342, 2024-09-10T10:04:54.937 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.001150639, 2024-09-10T10:04:54.937 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000181 2024-09-10T10:04:54.937 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:54.937 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-09-10T10:04:54.937 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_encrypted_bytes": 342944 2024-09-10T10:04:54.937 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:54.937 INFO:teuthology.orchestra.run.smithi120.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T10:04:54.937 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_messages": 7679, 2024-09-10T10:04:54.937 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages": 8638, 2024-09-10T10:04:54.937 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_bytes": 2612843, 2024-09-10T10:04:54.938 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_bytes": 3217715, 2024-09-10T10:04:54.938 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_created_connections": 29, 2024-09-10T10:04:54.938 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_active_connections": 7, 2024-09-10T10:04:54.938 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_total_time": 0.465583597, 2024-09-10T10:04:54.938 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_send_time": 0.164867759, 2024-09-10T10:04:54.938 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_recv_time": 0.195921359, 2024-09-10T10:04:54.938 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_fast_dispatch_time": 0.104115921, 2024-09-10T10:04:54.938 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:04:54.938 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 8638, 2024-09-10T10:04:54.938 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.316809369, 2024-09-10T10:04:54.938 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000036676 2024-09-10T10:04:54.938 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:54.938 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:04:54.938 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 6205, 2024-09-10T10:04:54.938 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.001018617, 2024-09-10T10:04:54.939 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000164 2024-09-10T10:04:54.939 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:54.939 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_encrypted_bytes": 928, 2024-09-10T10:04:54.939 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_encrypted_bytes": 736 2024-09-10T10:04:54.939 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:54.939 INFO:teuthology.orchestra.run.smithi120.stdout: "cct": { 2024-09-10T10:04:54.939 INFO:teuthology.orchestra.run.smithi120.stdout: "total_workers": 1, 2024-09-10T10:04:54.939 INFO:teuthology.orchestra.run.smithi120.stdout: "unhealthy_workers": 0 2024-09-10T10:04:54.939 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:54.939 INFO:teuthology.orchestra.run.smithi120.stdout: "finisher-MDSRank": { 2024-09-10T10:04:54.939 INFO:teuthology.orchestra.run.smithi120.stdout: "queue_len": 0, 2024-09-10T10:04:54.939 INFO:teuthology.orchestra.run.smithi120.stdout: "complete_latency": { 2024-09-10T10:04:54.939 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 7264, 2024-09-10T10:04:54.939 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 2.214361478, 2024-09-10T10:04:54.939 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000304840 2024-09-10T10:04:54.940 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:04:54.940 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:54.940 INFO:teuthology.orchestra.run.smithi120.stdout: "finisher-PurgeQueue": { 2024-09-10T10:04:54.940 INFO:teuthology.orchestra.run.smithi120.stdout: "queue_len": 0, 2024-09-10T10:04:54.940 INFO:teuthology.orchestra.run.smithi120.stdout: "complete_latency": { 2024-09-10T10:04:54.940 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 2337, 2024-09-10T10:04:54.940 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.190930014, 2024-09-10T10:04:54.940 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000081698 2024-09-10T10:04:54.940 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:04:54.940 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:54.940 INFO:teuthology.orchestra.run.smithi120.stdout: "mds": { 2024-09-10T10:04:54.940 INFO:teuthology.orchestra.run.smithi120.stdout: "request": 2726, 2024-09-10T10:04:54.940 INFO:teuthology.orchestra.run.smithi120.stdout: "reply": 2702, 2024-09-10T10:04:54.941 INFO:teuthology.orchestra.run.smithi120.stdout: "reply_latency": { 2024-09-10T10:04:54.941 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 2702, 2024-09-10T10:04:54.941 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 2.914071100, 2024-09-10T10:04:54.941 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.001078486 2024-09-10T10:04:54.941 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:54.941 INFO:teuthology.orchestra.run.smithi120.stdout: "slow_reply": 0, 2024-09-10T10:04:54.941 INFO:teuthology.orchestra.run.smithi120.stdout: "forward": 24, 2024-09-10T10:04:54.941 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_fetch_complete": 0, 2024-09-10T10:04:54.941 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_fetch_keys": 0, 2024-09-10T10:04:54.941 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_commit": 11, 2024-09-10T10:04:54.941 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_split": 0, 2024-09-10T10:04:54.941 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_merge": 0, 2024-09-10T10:04:54.941 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes": 17, 2024-09-10T10:04:54.941 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_top": 2, 2024-09-10T10:04:54.942 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_bottom": 1, 2024-09-10T10:04:54.942 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_pin_tail": 14, 2024-09-10T10:04:54.942 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_pinned": 14, 2024-09-10T10:04:54.942 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_expired": 442, 2024-09-10T10:04:54.942 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_with_caps": 1, 2024-09-10T10:04:54.942 INFO:teuthology.orchestra.run.smithi120.stdout: "caps": 1, 2024-09-10T10:04:54.942 INFO:teuthology.orchestra.run.smithi120.stdout: "subtrees": 4, 2024-09-10T10:04:54.942 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse": 5258, 2024-09-10T10:04:54.942 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_hit": 3929, 2024-09-10T10:04:54.942 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_forward": 17, 2024-09-10T10:04:54.942 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_discover": 10, 2024-09-10T10:04:54.942 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_dir_fetch": 0, 2024-09-10T10:04:54.942 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_remote_ino": 0, 2024-09-10T10:04:54.942 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_lock": 0, 2024-09-10T10:04:54.943 INFO:teuthology.orchestra.run.smithi120.stdout: "load_cent": 0, 2024-09-10T10:04:54.943 INFO:teuthology.orchestra.run.smithi120.stdout: "q": 0, 2024-09-10T10:04:54.943 INFO:teuthology.orchestra.run.smithi120.stdout: "exported": 35, 2024-09-10T10:04:54.943 INFO:teuthology.orchestra.run.smithi120.stdout: "exported_inodes": 421, 2024-09-10T10:04:54.943 INFO:teuthology.orchestra.run.smithi120.stdout: "imported": 55, 2024-09-10T10:04:54.943 INFO:teuthology.orchestra.run.smithi120.stdout: "imported_inodes": 1225, 2024-09-10T10:04:54.943 INFO:teuthology.orchestra.run.smithi120.stdout: "openino_dir_fetch": 0, 2024-09-10T10:04:54.943 INFO:teuthology.orchestra.run.smithi120.stdout: "openino_backtrace_fetch": 8440, 2024-09-10T10:04:54.943 INFO:teuthology.orchestra.run.smithi120.stdout: "openino_peer_discover": 0, 2024-09-10T10:04:54.943 INFO:teuthology.orchestra.run.smithi120.stdout: "root_rfiles": 4215, 2024-09-10T10:04:54.943 INFO:teuthology.orchestra.run.smithi120.stdout: "root_rbytes": 81947162, 2024-09-10T10:04:54.944 INFO:teuthology.orchestra.run.smithi120.stdout: "root_rsnaps": 0, 2024-09-10T10:04:54.944 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_backtrace_fetch": 8440, 2024-09-10T10:04:54.944 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_set_tag": 0, 2024-09-10T10:04:54.944 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T10:04:54.944 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_inotable_repaired": 0, 2024-09-10T10:04:54.944 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_dir_inodes": 127, 2024-09-10T10:04:54.944 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_dir_base_inodes": 0, 2024-09-10T10:04:54.944 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_dirfrag_rstats": 127, 2024-09-10T10:04:54.944 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_file_inodes": 8313, 2024-09-10T10:04:54.944 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_inode_file_caps": 32, 2024-09-10T10:04:54.944 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_cap_op_revoke": 281, 2024-09-10T10:04:54.944 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_cap_op_grant": 1448, 2024-09-10T10:04:54.944 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T10:04:54.944 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T10:04:54.945 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T10:04:54.945 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_client_caps": 1322, 2024-09-10T10:04:54.945 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_client_caps_dirty": 701, 2024-09-10T10:04:54.945 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_client_cap_release": 33, 2024-09-10T10:04:54.945 INFO:teuthology.orchestra.run.smithi120.stdout: "process_request_cap_release": 2346 2024-09-10T10:04:54.945 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:54.945 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_cache": { 2024-09-10T10:04:54.945 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_update": 0, 2024-09-10T10:04:54.945 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_update_receipt": 0, 2024-09-10T10:04:54.946 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_try_discover": 0, 2024-09-10T10:04:54.946 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_send_discover": 41, 2024-09-10T10:04:54.946 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_handle_discover": 34, 2024-09-10T10:04:54.946 INFO:teuthology.orchestra.run.smithi120.stdout: "num_strays": 0, 2024-09-10T10:04:54.946 INFO:teuthology.orchestra.run.smithi120.stdout: "num_strays_delayed": 0, 2024-09-10T10:04:54.946 INFO:teuthology.orchestra.run.smithi120.stdout: "num_strays_enqueuing": 0, 2024-09-10T10:04:54.946 INFO:teuthology.orchestra.run.smithi120.stdout: "strays_created": 1184, 2024-09-10T10:04:54.946 INFO:teuthology.orchestra.run.smithi120.stdout: "strays_enqueued": 1184, 2024-09-10T10:04:54.946 INFO:teuthology.orchestra.run.smithi120.stdout: "strays_reintegrated": 0, 2024-09-10T10:04:54.946 INFO:teuthology.orchestra.run.smithi120.stdout: "strays_migrated": 0, 2024-09-10T10:04:54.946 INFO:teuthology.orchestra.run.smithi120.stdout: "num_recovering_processing": 0, 2024-09-10T10:04:54.946 INFO:teuthology.orchestra.run.smithi120.stdout: "num_recovering_enqueued": 0, 2024-09-10T10:04:54.946 INFO:teuthology.orchestra.run.smithi120.stdout: "num_recovering_prioritized": 0, 2024-09-10T10:04:54.946 INFO:teuthology.orchestra.run.smithi120.stdout: "recovery_started": 0, 2024-09-10T10:04:54.947 INFO:teuthology.orchestra.run.smithi120.stdout: "recovery_completed": 0, 2024-09-10T10:04:54.947 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_quiesce_path": 8, 2024-09-10T10:04:54.947 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_quiesce_inode": 5698, 2024-09-10T10:04:54.947 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_enqueue_scrub": 0, 2024-09-10T10:04:54.947 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_exportdir": 35, 2024-09-10T10:04:54.947 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_flush": 0, 2024-09-10T10:04:54.947 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_fragmentdir": 0, 2024-09-10T10:04:54.947 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_fragstats": 0, 2024-09-10T10:04:54.947 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_inodestats": 0 2024-09-10T10:04:54.947 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:54.947 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_log": { 2024-09-10T10:04:54.947 INFO:teuthology.orchestra.run.smithi120.stdout: "evlrg": 0, 2024-09-10T10:04:54.947 INFO:teuthology.orchestra.run.smithi120.stdout: "evadd": 3695, 2024-09-10T10:04:54.947 INFO:teuthology.orchestra.run.smithi120.stdout: "evex": 0, 2024-09-10T10:04:54.948 INFO:teuthology.orchestra.run.smithi120.stdout: "evtrm": 0, 2024-09-10T10:04:54.948 INFO:teuthology.orchestra.run.smithi120.stdout: "ev": 3695, 2024-09-10T10:04:54.948 INFO:teuthology.orchestra.run.smithi120.stdout: "evexg": 0, 2024-09-10T10:04:54.948 INFO:teuthology.orchestra.run.smithi120.stdout: "evexd": 0, 2024-09-10T10:04:54.948 INFO:teuthology.orchestra.run.smithi120.stdout: "segadd": 6, 2024-09-10T10:04:54.948 INFO:teuthology.orchestra.run.smithi120.stdout: "segex": 0, 2024-09-10T10:04:54.948 INFO:teuthology.orchestra.run.smithi120.stdout: "segtrm": 0, 2024-09-10T10:04:54.948 INFO:teuthology.orchestra.run.smithi120.stdout: "seg": 6, 2024-09-10T10:04:54.948 INFO:teuthology.orchestra.run.smithi120.stdout: "segmjr": 2, 2024-09-10T10:04:54.948 INFO:teuthology.orchestra.run.smithi120.stdout: "segexg": 0, 2024-09-10T10:04:54.948 INFO:teuthology.orchestra.run.smithi120.stdout: "segexd": 0, 2024-09-10T10:04:54.948 INFO:teuthology.orchestra.run.smithi120.stdout: "expos": 4194304, 2024-09-10T10:04:54.948 INFO:teuthology.orchestra.run.smithi120.stdout: "wrpos": 10600988, 2024-09-10T10:04:54.948 INFO:teuthology.orchestra.run.smithi120.stdout: "rdpos": 0, 2024-09-10T10:04:54.949 INFO:teuthology.orchestra.run.smithi120.stdout: "jlat": { 2024-09-10T10:04:54.949 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 241, 2024-09-10T10:04:54.949 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 1.465671630, 2024-09-10T10:04:54.949 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.006081625 2024-09-10T10:04:54.949 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:54.949 INFO:teuthology.orchestra.run.smithi120.stdout: "replayed": 0 2024-09-10T10:04:54.949 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:54.949 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_mem": { 2024-09-10T10:04:54.949 INFO:teuthology.orchestra.run.smithi120.stdout: "ino": 22, 2024-09-10T10:04:54.949 INFO:teuthology.orchestra.run.smithi120.stdout: "ino+": 1215, 2024-09-10T10:04:54.949 INFO:teuthology.orchestra.run.smithi120.stdout: "ino-": 1193, 2024-09-10T10:04:54.949 INFO:teuthology.orchestra.run.smithi120.stdout: "dir": 21, 2024-09-10T10:04:54.949 INFO:teuthology.orchestra.run.smithi120.stdout: "dir+": 86, 2024-09-10T10:04:54.949 INFO:teuthology.orchestra.run.smithi120.stdout: "dir-": 65, 2024-09-10T10:04:54.949 INFO:teuthology.orchestra.run.smithi120.stdout: "dn": 17, 2024-09-10T10:04:54.950 INFO:teuthology.orchestra.run.smithi120.stdout: "dn+": 2505, 2024-09-10T10:04:54.950 INFO:teuthology.orchestra.run.smithi120.stdout: "dn-": 2488, 2024-09-10T10:04:54.950 INFO:teuthology.orchestra.run.smithi120.stdout: "cap": 1, 2024-09-10T10:04:54.950 INFO:teuthology.orchestra.run.smithi120.stdout: "cap+": 2300, 2024-09-10T10:04:54.950 INFO:teuthology.orchestra.run.smithi120.stdout: "cap-": 2299, 2024-09-10T10:04:54.950 INFO:teuthology.orchestra.run.smithi120.stdout: "rss": 68324, 2024-09-10T10:04:54.950 INFO:teuthology.orchestra.run.smithi120.stdout: "heap": 223520 2024-09-10T10:04:54.950 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:54.950 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_server": { 2024-09-10T10:04:54.950 INFO:teuthology.orchestra.run.smithi120.stdout: "dispatch_client_request": 3972, 2024-09-10T10:04:54.950 INFO:teuthology.orchestra.run.smithi120.stdout: "dispatch_server_request": 379, 2024-09-10T10:04:54.950 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_client_request": 2726, 2024-09-10T10:04:54.950 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_client_session": 12, 2024-09-10T10:04:54.950 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_peer_request": 1913, 2024-09-10T10:04:54.950 INFO:teuthology.orchestra.run.smithi120.stdout: "req_create_latency": { 2024-09-10T10:04:54.951 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 79, 2024-09-10T10:04:54.951 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.115503853, 2024-09-10T10:04:54.951 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.001462074 2024-09-10T10:04:54.951 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:54.951 INFO:teuthology.orchestra.run.smithi120.stdout: "req_getattr_latency": { 2024-09-10T10:04:54.951 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:54.951 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:54.951 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:54.951 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:54.951 INFO:teuthology.orchestra.run.smithi120.stdout: "req_getfilelock_latency": { 2024-09-10T10:04:54.951 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:54.951 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:54.951 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:54.951 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:54.952 INFO:teuthology.orchestra.run.smithi120.stdout: "req_link_latency": { 2024-09-10T10:04:54.952 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:54.952 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:54.952 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:54.952 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:54.952 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookup_latency": { 2024-09-10T10:04:54.952 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 1351, 2024-09-10T10:04:54.952 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.420633863, 2024-09-10T10:04:54.952 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000311350 2024-09-10T10:04:54.952 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:54.952 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookuphash_latency": { 2024-09-10T10:04:54.952 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:54.952 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:54.952 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:54.953 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:54.953 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookupino_latency": { 2024-09-10T10:04:54.953 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 1, 2024-09-10T10:04:54.953 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000132175, 2024-09-10T10:04:54.953 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000132175 2024-09-10T10:04:54.953 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:54.953 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookupname_latency": { 2024-09-10T10:04:54.953 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:54.953 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:54.953 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:54.953 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:54.953 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookupparent_latency": { 2024-09-10T10:04:54.953 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:54.953 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:54.954 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:54.954 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:54.954 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookupsnap_latency": { 2024-09-10T10:04:54.954 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:54.954 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:54.954 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:54.954 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:54.954 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lssnap_latency": { 2024-09-10T10:04:54.954 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:54.954 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:54.954 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:54.954 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:54.955 INFO:teuthology.orchestra.run.smithi120.stdout: "req_mkdir_latency": { 2024-09-10T10:04:54.955 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 1, 2024-09-10T10:04:54.955 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000914109, 2024-09-10T10:04:54.955 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000914109 2024-09-10T10:04:54.955 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:54.955 INFO:teuthology.orchestra.run.smithi120.stdout: "req_mknod_latency": { 2024-09-10T10:04:54.955 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:54.955 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:54.955 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:54.956 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:54.956 INFO:teuthology.orchestra.run.smithi120.stdout: "req_mksnap_latency": { 2024-09-10T10:04:54.956 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:54.956 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:54.956 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:54.956 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:54.956 INFO:teuthology.orchestra.run.smithi120.stdout: "req_open_latency": { 2024-09-10T10:04:54.956 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 6, 2024-09-10T10:04:54.956 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.001466117, 2024-09-10T10:04:54.956 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000244352 2024-09-10T10:04:54.956 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:54.956 INFO:teuthology.orchestra.run.smithi120.stdout: "req_readdir_latency": { 2024-09-10T10:04:54.956 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 80, 2024-09-10T10:04:54.956 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.063612011, 2024-09-10T10:04:54.956 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000795150 2024-09-10T10:04:54.957 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:54.957 INFO:teuthology.orchestra.run.smithi120.stdout: "req_rename_latency": { 2024-09-10T10:04:54.957 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:54.957 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:54.957 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:54.957 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:54.957 INFO:teuthology.orchestra.run.smithi120.stdout: "req_renamesnap_latency": { 2024-09-10T10:04:54.957 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:54.957 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:54.957 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:54.957 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:54.957 INFO:teuthology.orchestra.run.smithi120.stdout: "req_snapdiff_latency": { 2024-09-10T10:04:54.957 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:54.958 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:54.958 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:54.958 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:54.958 INFO:teuthology.orchestra.run.smithi120.stdout: "req_rmdir_latency": { 2024-09-10T10:04:54.958 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 18, 2024-09-10T10:04:54.958 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.473823731, 2024-09-10T10:04:54.958 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.026323540 2024-09-10T10:04:54.958 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:54.958 INFO:teuthology.orchestra.run.smithi120.stdout: "req_rmsnap_latency": { 2024-09-10T10:04:54.958 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:54.958 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:54.958 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:54.958 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:54.958 INFO:teuthology.orchestra.run.smithi120.stdout: "req_rmxattr_latency": { 2024-09-10T10:04:54.959 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:54.959 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:54.959 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:54.959 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:54.959 INFO:teuthology.orchestra.run.smithi120.stdout: "req_setattr_latency": { 2024-09-10T10:04:54.959 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:54.959 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:54.959 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:54.959 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:54.959 INFO:teuthology.orchestra.run.smithi120.stdout: "req_setdirlayout_latency": { 2024-09-10T10:04:54.959 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:54.959 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:54.959 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:54.959 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:54.959 INFO:teuthology.orchestra.run.smithi120.stdout: "req_setfilelock_latency": { 2024-09-10T10:04:54.960 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:54.960 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:54.960 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:54.960 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:54.960 INFO:teuthology.orchestra.run.smithi120.stdout: "req_setlayout_latency": { 2024-09-10T10:04:54.960 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:54.960 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:54.960 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:54.960 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:54.960 INFO:teuthology.orchestra.run.smithi120.stdout: "req_setxattr_latency": { 2024-09-10T10:04:54.960 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:54.960 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:54.960 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:54.961 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:54.961 INFO:teuthology.orchestra.run.smithi120.stdout: "req_symlink_latency": { 2024-09-10T10:04:54.961 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:54.961 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:54.961 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:54.961 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:54.961 INFO:teuthology.orchestra.run.smithi120.stdout: "req_unlink_latency": { 2024-09-10T10:04:54.961 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 1166, 2024-09-10T10:04:54.961 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 1.837985241, 2024-09-10T10:04:54.961 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.001576316 2024-09-10T10:04:54.961 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:54.961 INFO:teuthology.orchestra.run.smithi120.stdout: "cap_revoke_eviction": 0, 2024-09-10T10:04:54.961 INFO:teuthology.orchestra.run.smithi120.stdout: "cap_acquisition_throttle": 0, 2024-09-10T10:04:54.962 INFO:teuthology.orchestra.run.smithi120.stdout: "req_getvxattr_latency": { 2024-09-10T10:04:54.962 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:54.962 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:54.962 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:54.962 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:04:54.962 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:54.962 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_sessions": { 2024-09-10T10:04:54.962 INFO:teuthology.orchestra.run.smithi120.stdout: "session_count": 1, 2024-09-10T10:04:54.962 INFO:teuthology.orchestra.run.smithi120.stdout: "session_add": 2, 2024-09-10T10:04:54.962 INFO:teuthology.orchestra.run.smithi120.stdout: "session_remove": 1, 2024-09-10T10:04:54.962 INFO:teuthology.orchestra.run.smithi120.stdout: "sessions_open": 1, 2024-09-10T10:04:54.962 INFO:teuthology.orchestra.run.smithi120.stdout: "sessions_stale": 0, 2024-09-10T10:04:54.962 INFO:teuthology.orchestra.run.smithi120.stdout: "total_load": 2116, 2024-09-10T10:04:54.962 INFO:teuthology.orchestra.run.smithi120.stdout: "average_load": 2116, 2024-09-10T10:04:54.963 INFO:teuthology.orchestra.run.smithi120.stdout: "avg_session_uptime": 553, 2024-09-10T10:04:54.963 INFO:teuthology.orchestra.run.smithi120.stdout: "mdthresh_evicted": 0 2024-09-10T10:04:54.963 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:54.963 INFO:teuthology.orchestra.run.smithi120.stdout: "mempool": { 2024-09-10T10:04:54.963 INFO:teuthology.orchestra.run.smithi120.stdout: "bloom_filter_bytes": 0, 2024-09-10T10:04:54.963 INFO:teuthology.orchestra.run.smithi120.stdout: "bloom_filter_items": 0, 2024-09-10T10:04:54.963 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T10:04:54.963 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_alloc_items": 0, 2024-09-10T10:04:54.963 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T10:04:54.963 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_data_items": 0, 2024-09-10T10:04:54.963 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T10:04:54.964 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T10:04:54.964 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T10:04:54.964 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T10:04:54.964 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T10:04:54.964 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_other_items": 0, 2024-09-10T10:04:54.964 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T10:04:54.964 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T10:04:54.964 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_extent_bytes": 0, 2024-09-10T10:04:54.964 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_extent_items": 0, 2024-09-10T10:04:54.964 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_blob_bytes": 0, 2024-09-10T10:04:54.964 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_blob_items": 0, 2024-09-10T10:04:54.964 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T10:04:54.964 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T10:04:54.965 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T10:04:54.965 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T10:04:54.965 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T10:04:54.965 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_fsck_items": 0, 2024-09-10T10:04:54.965 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_txc_bytes": 0, 2024-09-10T10:04:54.965 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_txc_items": 0, 2024-09-10T10:04:54.965 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T10:04:54.965 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T10:04:54.965 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_writing_bytes": 0, 2024-09-10T10:04:54.965 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_writing_items": 0, 2024-09-10T10:04:54.965 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_bytes": 0, 2024-09-10T10:04:54.965 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_items": 0, 2024-09-10T10:04:54.965 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T10:04:54.965 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_file_reader_items": 0, 2024-09-10T10:04:54.966 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T10:04:54.966 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_file_writer_items": 0, 2024-09-10T10:04:54.966 INFO:teuthology.orchestra.run.smithi120.stdout: "buffer_anon_bytes": 287562, 2024-09-10T10:04:54.966 INFO:teuthology.orchestra.run.smithi120.stdout: "buffer_anon_items": 103, 2024-09-10T10:04:54.966 INFO:teuthology.orchestra.run.smithi120.stdout: "buffer_meta_bytes": 0, 2024-09-10T10:04:54.966 INFO:teuthology.orchestra.run.smithi120.stdout: "buffer_meta_items": 0, 2024-09-10T10:04:54.966 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_bytes": 0, 2024-09-10T10:04:54.966 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_items": 0, 2024-09-10T10:04:54.966 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_mapbl_bytes": 0, 2024-09-10T10:04:54.966 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_mapbl_items": 0, 2024-09-10T10:04:54.966 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pglog_bytes": 0, 2024-09-10T10:04:54.966 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pglog_items": 0, 2024-09-10T10:04:54.966 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_bytes": 9144, 2024-09-10T10:04:54.966 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_items": 172, 2024-09-10T10:04:54.967 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T10:04:54.967 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_mapping_items": 0, 2024-09-10T10:04:54.967 INFO:teuthology.orchestra.run.smithi120.stdout: "pgmap_bytes": 0, 2024-09-10T10:04:54.967 INFO:teuthology.orchestra.run.smithi120.stdout: "pgmap_items": 0, 2024-09-10T10:04:54.967 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_co_bytes": 201014, 2024-09-10T10:04:54.967 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_co_items": 2005, 2024-09-10T10:04:54.967 INFO:teuthology.orchestra.run.smithi120.stdout: "unittest_1_bytes": 0, 2024-09-10T10:04:54.967 INFO:teuthology.orchestra.run.smithi120.stdout: "unittest_1_items": 0, 2024-09-10T10:04:54.967 INFO:teuthology.orchestra.run.smithi120.stdout: "unittest_2_bytes": 0, 2024-09-10T10:04:54.967 INFO:teuthology.orchestra.run.smithi120.stdout: "unittest_2_items": 0 2024-09-10T10:04:54.967 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:54.967 INFO:teuthology.orchestra.run.smithi120.stdout: "objecter": { 2024-09-10T10:04:54.968 INFO:teuthology.orchestra.run.smithi120.stdout: "op_active": 0, 2024-09-10T10:04:54.968 INFO:teuthology.orchestra.run.smithi120.stdout: "op_laggy": 0, 2024-09-10T10:04:54.968 INFO:teuthology.orchestra.run.smithi120.stdout: "op_send": 11222, 2024-09-10T10:04:54.968 INFO:teuthology.orchestra.run.smithi120.stdout: "op_send_bytes": 6667824, 2024-09-10T10:04:54.968 INFO:teuthology.orchestra.run.smithi120.stdout: "op_resend": 33, 2024-09-10T10:04:54.968 INFO:teuthology.orchestra.run.smithi120.stdout: "op_reply": 11222, 2024-09-10T10:04:54.968 INFO:teuthology.orchestra.run.smithi120.stdout: "op_latency": { 2024-09-10T10:04:54.968 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 11222, 2024-09-10T10:04:54.968 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 25.086411409, 2024-09-10T10:04:54.968 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.002235467 2024-09-10T10:04:54.968 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:54.968 INFO:teuthology.orchestra.run.smithi120.stdout: "op_inflight": 0, 2024-09-10T10:04:54.969 INFO:teuthology.orchestra.run.smithi120.stdout: "oplen_avg": { 2024-09-10T10:04:54.969 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 11222, 2024-09-10T10:04:54.969 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 11310 2024-09-10T10:04:54.969 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:54.969 INFO:teuthology.orchestra.run.smithi120.stdout: "op": 11222, 2024-09-10T10:04:54.969 INFO:teuthology.orchestra.run.smithi120.stdout: "op_r": 8469, 2024-09-10T10:04:54.969 INFO:teuthology.orchestra.run.smithi120.stdout: "op_w": 2753, 2024-09-10T10:04:54.969 INFO:teuthology.orchestra.run.smithi120.stdout: "op_rmw": 0, 2024-09-10T10:04:54.969 INFO:teuthology.orchestra.run.smithi120.stdout: "op_pg": 0, 2024-09-10T10:04:54.969 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_stat": 0, 2024-09-10T10:04:54.969 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_create": 10, 2024-09-10T10:04:54.969 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_read": 29, 2024-09-10T10:04:54.969 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_write": 270, 2024-09-10T10:04:54.969 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_writefull": 35, 2024-09-10T10:04:54.969 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_writesame": 0, 2024-09-10T10:04:54.970 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_append": 0, 2024-09-10T10:04:54.970 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_zero": 0, 2024-09-10T10:04:54.970 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_truncate": 0, 2024-09-10T10:04:54.970 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_delete": 2370, 2024-09-10T10:04:54.970 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_mapext": 0, 2024-09-10T10:04:54.970 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_sparse_read": 0, 2024-09-10T10:04:54.970 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_clonerange": 0, 2024-09-10T10:04:54.970 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_getxattr": 8440, 2024-09-10T10:04:54.970 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_setxattr": 20, 2024-09-10T10:04:54.970 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_cmpxattr": 0, 2024-09-10T10:04:54.970 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_rmxattr": 0, 2024-09-10T10:04:54.970 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_resetxattrs": 0, 2024-09-10T10:04:54.970 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_call": 0, 2024-09-10T10:04:54.971 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_watch": 0, 2024-09-10T10:04:54.971 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_notify": 0, 2024-09-10T10:04:54.971 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T10:04:54.971 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_pgls": 0, 2024-09-10T10:04:54.971 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_pgls_filter": 0, 2024-09-10T10:04:54.971 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_other": 0, 2024-09-10T10:04:54.971 INFO:teuthology.orchestra.run.smithi120.stdout: "linger_active": 0, 2024-09-10T10:04:54.971 INFO:teuthology.orchestra.run.smithi120.stdout: "linger_send": 0, 2024-09-10T10:04:54.971 INFO:teuthology.orchestra.run.smithi120.stdout: "linger_resend": 0, 2024-09-10T10:04:54.971 INFO:teuthology.orchestra.run.smithi120.stdout: "linger_ping": 0, 2024-09-10T10:04:54.971 INFO:teuthology.orchestra.run.smithi120.stdout: "poolop_active": 0, 2024-09-10T10:04:54.972 INFO:teuthology.orchestra.run.smithi120.stdout: "poolop_send": 0, 2024-09-10T10:04:54.972 INFO:teuthology.orchestra.run.smithi120.stdout: "poolop_resend": 0, 2024-09-10T10:04:54.972 INFO:teuthology.orchestra.run.smithi120.stdout: "poolstat_active": 0, 2024-09-10T10:04:54.972 INFO:teuthology.orchestra.run.smithi120.stdout: "poolstat_send": 0, 2024-09-10T10:04:54.972 INFO:teuthology.orchestra.run.smithi120.stdout: "poolstat_resend": 0, 2024-09-10T10:04:54.972 INFO:teuthology.orchestra.run.smithi120.stdout: "statfs_active": 0, 2024-09-10T10:04:54.972 INFO:teuthology.orchestra.run.smithi120.stdout: "statfs_send": 0, 2024-09-10T10:04:54.972 INFO:teuthology.orchestra.run.smithi120.stdout: "statfs_resend": 0, 2024-09-10T10:04:54.972 INFO:teuthology.orchestra.run.smithi120.stdout: "command_active": 0, 2024-09-10T10:04:54.972 INFO:teuthology.orchestra.run.smithi120.stdout: "command_send": 0, 2024-09-10T10:04:54.972 INFO:teuthology.orchestra.run.smithi120.stdout: "command_resend": 0, 2024-09-10T10:04:54.972 INFO:teuthology.orchestra.run.smithi120.stdout: "map_epoch": 96, 2024-09-10T10:04:54.972 INFO:teuthology.orchestra.run.smithi120.stdout: "map_full": 0, 2024-09-10T10:04:54.972 INFO:teuthology.orchestra.run.smithi120.stdout: "map_inc": 2, 2024-09-10T10:04:54.972 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_sessions": 12, 2024-09-10T10:04:54.974 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_session_open": 12, 2024-09-10T10:04:54.974 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_session_close": 0, 2024-09-10T10:04:54.974 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_laggy": 0, 2024-09-10T10:04:54.974 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_wr": 109, 2024-09-10T10:04:54.974 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_rd": 0, 2024-09-10T10:04:54.974 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_del": 27 2024-09-10T10:04:54.975 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:54.975 INFO:teuthology.orchestra.run.smithi120.stdout: "oft": { 2024-09-10T10:04:54.975 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_total_objs": 1, 2024-09-10T10:04:54.975 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_total_kv_pairs": 0, 2024-09-10T10:04:54.975 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_total_updates": 0, 2024-09-10T10:04:54.975 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_total_removes": 0 2024-09-10T10:04:54.975 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:54.975 INFO:teuthology.orchestra.run.smithi120.stdout: "osd-slow-ops": { 2024-09-10T10:04:54.975 INFO:teuthology.orchestra.run.smithi120.stdout: "slow_ops_count": 5699 2024-09-10T10:04:54.976 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:54.976 INFO:teuthology.orchestra.run.smithi120.stdout: "purge_queue": { 2024-09-10T10:04:54.976 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executing_ops": 0, 2024-09-10T10:04:54.976 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executing_ops_high_water": 10, 2024-09-10T10:04:54.976 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executing": 0, 2024-09-10T10:04:54.976 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executing_high_water": 5, 2024-09-10T10:04:54.976 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executed_ops": 2357, 2024-09-10T10:04:54.976 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executed": 1184, 2024-09-10T10:04:54.977 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_item_in_journal": 0 2024-09-10T10:04:54.977 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:54.977 INFO:teuthology.orchestra.run.smithi120.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T10:04:54.977 INFO:teuthology.orchestra.run.smithi120.stdout: "val": 0, 2024-09-10T10:04:54.977 INFO:teuthology.orchestra.run.smithi120.stdout: "max": 104857600, 2024-09-10T10:04:54.977 INFO:teuthology.orchestra.run.smithi120.stdout: "get_started": 0, 2024-09-10T10:04:54.977 INFO:teuthology.orchestra.run.smithi120.stdout: "get": 21572, 2024-09-10T10:04:54.977 INFO:teuthology.orchestra.run.smithi120.stdout: "get_sum": 6673176, 2024-09-10T10:04:54.978 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_fail": 0, 2024-09-10T10:04:54.978 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_success": 21572, 2024-09-10T10:04:54.978 INFO:teuthology.orchestra.run.smithi120.stdout: "take": 0, 2024-09-10T10:04:54.978 INFO:teuthology.orchestra.run.smithi120.stdout: "take_sum": 0, 2024-09-10T10:04:54.978 INFO:teuthology.orchestra.run.smithi120.stdout: "put": 21572, 2024-09-10T10:04:54.978 INFO:teuthology.orchestra.run.smithi120.stdout: "put_sum": 6673176, 2024-09-10T10:04:54.978 INFO:teuthology.orchestra.run.smithi120.stdout: "wait": { 2024-09-10T10:04:54.979 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:54.979 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:54.979 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:54.979 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:04:54.979 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:54.979 INFO:teuthology.orchestra.run.smithi120.stdout: "throttle-objecter_bytes": { 2024-09-10T10:04:54.979 INFO:teuthology.orchestra.run.smithi120.stdout: "val": 0, 2024-09-10T10:04:54.979 INFO:teuthology.orchestra.run.smithi120.stdout: "max": 104857600, 2024-09-10T10:04:54.980 INFO:teuthology.orchestra.run.smithi120.stdout: "get_started": 0, 2024-09-10T10:04:54.980 INFO:teuthology.orchestra.run.smithi120.stdout: "get": 0, 2024-09-10T10:04:54.980 INFO:teuthology.orchestra.run.smithi120.stdout: "get_sum": 0, 2024-09-10T10:04:54.980 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_fail": 0, 2024-09-10T10:04:54.980 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_success": 0, 2024-09-10T10:04:54.980 INFO:teuthology.orchestra.run.smithi120.stdout: "take": 11222, 2024-09-10T10:04:54.980 INFO:teuthology.orchestra.run.smithi120.stdout: "take_sum": 6796736, 2024-09-10T10:04:54.980 INFO:teuthology.orchestra.run.smithi120.stdout: "put": 11222, 2024-09-10T10:04:54.980 INFO:teuthology.orchestra.run.smithi120.stdout: "put_sum": 6796736, 2024-09-10T10:04:54.980 INFO:teuthology.orchestra.run.smithi120.stdout: "wait": { 2024-09-10T10:04:54.980 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:54.980 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:54.980 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:54.981 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:04:54.981 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:54.981 INFO:teuthology.orchestra.run.smithi120.stdout: "throttle-objecter_ops": { 2024-09-10T10:04:54.981 INFO:teuthology.orchestra.run.smithi120.stdout: "val": 0, 2024-09-10T10:04:54.981 INFO:teuthology.orchestra.run.smithi120.stdout: "max": 1024, 2024-09-10T10:04:54.981 INFO:teuthology.orchestra.run.smithi120.stdout: "get_started": 0, 2024-09-10T10:04:54.981 INFO:teuthology.orchestra.run.smithi120.stdout: "get": 0, 2024-09-10T10:04:54.981 INFO:teuthology.orchestra.run.smithi120.stdout: "get_sum": 0, 2024-09-10T10:04:54.981 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_fail": 0, 2024-09-10T10:04:54.981 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_success": 0, 2024-09-10T10:04:54.981 INFO:teuthology.orchestra.run.smithi120.stdout: "take": 11222, 2024-09-10T10:04:54.981 INFO:teuthology.orchestra.run.smithi120.stdout: "take_sum": 11222, 2024-09-10T10:04:54.981 INFO:teuthology.orchestra.run.smithi120.stdout: "put": 11222, 2024-09-10T10:04:54.982 INFO:teuthology.orchestra.run.smithi120.stdout: "put_sum": 11222, 2024-09-10T10:04:54.982 INFO:teuthology.orchestra.run.smithi120.stdout: "wait": { 2024-09-10T10:04:54.982 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:54.982 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:54.982 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:54.982 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:04:54.982 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:54.982 INFO:teuthology.orchestra.run.smithi120.stdout: "throttle-write_buf_throttle": { 2024-09-10T10:04:54.983 INFO:teuthology.orchestra.run.smithi120.stdout: "val": 0, 2024-09-10T10:04:54.983 INFO:teuthology.orchestra.run.smithi120.stdout: "max": 3758096384, 2024-09-10T10:04:54.983 INFO:teuthology.orchestra.run.smithi120.stdout: "get_started": 0, 2024-09-10T10:04:54.983 INFO:teuthology.orchestra.run.smithi120.stdout: "get": 1184, 2024-09-10T10:04:54.983 INFO:teuthology.orchestra.run.smithi120.stdout: "get_sum": 128912, 2024-09-10T10:04:54.983 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_fail": 0, 2024-09-10T10:04:54.983 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_success": 1184, 2024-09-10T10:04:54.983 INFO:teuthology.orchestra.run.smithi120.stdout: "take": 0, 2024-09-10T10:04:54.983 INFO:teuthology.orchestra.run.smithi120.stdout: "take_sum": 0, 2024-09-10T10:04:54.984 INFO:teuthology.orchestra.run.smithi120.stdout: "put": 29, 2024-09-10T10:04:54.984 INFO:teuthology.orchestra.run.smithi120.stdout: "put_sum": 128912, 2024-09-10T10:04:54.984 INFO:teuthology.orchestra.run.smithi120.stdout: "wait": { 2024-09-10T10:04:54.984 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:54.984 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:54.984 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:54.984 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:04:54.984 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:54.984 INFO:teuthology.orchestra.run.smithi120.stdout: "throttle-write_buf_throttle-0x55c0e1900960": { 2024-09-10T10:04:54.984 INFO:teuthology.orchestra.run.smithi120.stdout: "val": 0, 2024-09-10T10:04:54.984 INFO:teuthology.orchestra.run.smithi120.stdout: "max": 3758096384, 2024-09-10T10:04:54.984 INFO:teuthology.orchestra.run.smithi120.stdout: "get_started": 0, 2024-09-10T10:04:54.984 INFO:teuthology.orchestra.run.smithi120.stdout: "get": 3695, 2024-09-10T10:04:54.984 INFO:teuthology.orchestra.run.smithi120.stdout: "get_sum": 6406684, 2024-09-10T10:04:54.984 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_fail": 0, 2024-09-10T10:04:54.985 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_success": 3695, 2024-09-10T10:04:54.985 INFO:teuthology.orchestra.run.smithi120.stdout: "take": 0, 2024-09-10T10:04:54.985 INFO:teuthology.orchestra.run.smithi120.stdout: "take_sum": 0, 2024-09-10T10:04:54.985 INFO:teuthology.orchestra.run.smithi120.stdout: "put": 241, 2024-09-10T10:04:54.985 INFO:teuthology.orchestra.run.smithi120.stdout: "put_sum": 6406684, 2024-09-10T10:04:54.985 INFO:teuthology.orchestra.run.smithi120.stdout: "wait": { 2024-09-10T10:04:54.985 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:54.985 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:54.986 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:54.986 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:04:54.986 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:04:54.986 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:04:55.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:04:54 smithi120 ceph-mon[34770]: pgmap v683: 129 pgs: 129 active+clean; 115 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 85 B/s wr, 10 op/s 2024-09-10T10:04:55.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:04:54 smithi134 ceph-mon[35674]: pgmap v683: 129 pgs: 129 active+clean; 115 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 85 B/s wr, 10 op/s 2024-09-10T10:04:55.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:04:54 smithi077 ceph-mon[30319]: pgmap v683: 129 pgs: 129 active+clean; 115 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 85 B/s wr, 10 op/s 2024-09-10T10:04:55.757 INFO:tasks.check_counter:Daemon mds.h mds.exported=35 2024-09-10T10:04:55.757 INFO:tasks.check_counter:Daemon mds.h mds.imported=55 2024-09-10T10:04:55.757 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=0 2024-09-10T10:04:55.757 DEBUG:tasks.check_counter:Getting stats from k 2024-09-10T10:04:55.757 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph daemon mds.k perf dump 2024-09-10T10:04:56.000 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T10:04:56.695 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:04:56.695 INFO:teuthology.orchestra.run.smithi120.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T10:04:56.695 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_messages": 2091, 2024-09-10T10:04:56.695 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages": 2093, 2024-09-10T10:04:56.695 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_bytes": 6995931, 2024-09-10T10:04:56.695 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_bytes": 664945, 2024-09-10T10:04:56.695 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_created_connections": 6, 2024-09-10T10:04:56.695 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_active_connections": 2, 2024-09-10T10:04:56.695 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_total_time": 0.143431644, 2024-09-10T10:04:56.695 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_send_time": 0.045136678, 2024-09-10T10:04:56.696 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_recv_time": 0.039623900, 2024-09-10T10:04:56.696 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_fast_dispatch_time": 0.047887318, 2024-09-10T10:04:56.696 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:04:56.696 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 2091, 2024-09-10T10:04:56.696 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.053867462, 2024-09-10T10:04:56.696 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000025761 2024-09-10T10:04:56.696 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:56.696 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:04:56.696 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:56.696 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:56.697 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:56.697 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:56.697 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-10T10:04:56.697 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_encrypted_bytes": 0 2024-09-10T10:04:56.697 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:56.697 INFO:teuthology.orchestra.run.smithi120.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T10:04:56.697 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_messages": 243, 2024-09-10T10:04:56.697 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages": 226, 2024-09-10T10:04:56.697 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_bytes": 580028, 2024-09-10T10:04:56.697 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_bytes": 134234, 2024-09-10T10:04:56.698 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_created_connections": 3, 2024-09-10T10:04:56.698 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_active_connections": 2, 2024-09-10T10:04:56.698 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_total_time": 0.033012045, 2024-09-10T10:04:56.698 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_send_time": 0.013801180, 2024-09-10T10:04:56.698 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_recv_time": 0.023810252, 2024-09-10T10:04:56.698 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_fast_dispatch_time": 0.000052371, 2024-09-10T10:04:56.698 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:04:56.698 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 226, 2024-09-10T10:04:56.698 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.016301547, 2024-09-10T10:04:56.698 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000072130 2024-09-10T10:04:56.698 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:56.698 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:04:56.698 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:56.699 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:56.699 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:56.699 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:56.699 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_encrypted_bytes": 579472, 2024-09-10T10:04:56.699 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_encrypted_bytes": 133536 2024-09-10T10:04:56.699 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:56.699 INFO:teuthology.orchestra.run.smithi120.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T10:04:56.699 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_messages": 2, 2024-09-10T10:04:56.699 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages": 164, 2024-09-10T10:04:56.699 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_bytes": 428, 2024-09-10T10:04:56.699 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_bytes": 343006, 2024-09-10T10:04:56.700 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_created_connections": 4, 2024-09-10T10:04:56.700 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_active_connections": 3, 2024-09-10T10:04:56.700 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_total_time": 0.009166447, 2024-09-10T10:04:56.700 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_send_time": 0.006523813, 2024-09-10T10:04:56.700 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_recv_time": 0.001124339, 2024-09-10T10:04:56.700 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_fast_dispatch_time": 0.000022068, 2024-09-10T10:04:56.700 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:04:56.700 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 162, 2024-09-10T10:04:56.700 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.009598798, 2024-09-10T10:04:56.700 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000059251 2024-09-10T10:04:56.700 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:56.700 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:04:56.700 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:56.701 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:56.701 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:56.701 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:56.701 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-10T10:04:56.701 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_encrypted_bytes": 342688 2024-09-10T10:04:56.701 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:56.701 INFO:teuthology.orchestra.run.smithi120.stdout: "cct": { 2024-09-10T10:04:56.701 INFO:teuthology.orchestra.run.smithi120.stdout: "total_workers": 1, 2024-09-10T10:04:56.701 INFO:teuthology.orchestra.run.smithi120.stdout: "unhealthy_workers": 0 2024-09-10T10:04:56.701 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:56.701 INFO:teuthology.orchestra.run.smithi120.stdout: "finisher-MDSRank": { 2024-09-10T10:04:56.701 INFO:teuthology.orchestra.run.smithi120.stdout: "queue_len": 0, 2024-09-10T10:04:56.701 INFO:teuthology.orchestra.run.smithi120.stdout: "complete_latency": { 2024-09-10T10:04:56.701 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 3541, 2024-09-10T10:04:56.702 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.078124853, 2024-09-10T10:04:56.702 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000022062 2024-09-10T10:04:56.702 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:04:56.702 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:56.702 INFO:teuthology.orchestra.run.smithi120.stdout: "finisher-PurgeQueue": { 2024-09-10T10:04:56.702 INFO:teuthology.orchestra.run.smithi120.stdout: "queue_len": 0, 2024-09-10T10:04:56.702 INFO:teuthology.orchestra.run.smithi120.stdout: "complete_latency": { 2024-09-10T10:04:56.702 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:56.702 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:56.702 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:56.702 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:04:56.702 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:56.702 INFO:teuthology.orchestra.run.smithi120.stdout: "mds": { 2024-09-10T10:04:56.702 INFO:teuthology.orchestra.run.smithi120.stdout: "request": 0, 2024-09-10T10:04:56.703 INFO:teuthology.orchestra.run.smithi120.stdout: "reply": 0, 2024-09-10T10:04:56.703 INFO:teuthology.orchestra.run.smithi120.stdout: "reply_latency": { 2024-09-10T10:04:56.703 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:56.703 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:56.703 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:56.703 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:56.703 INFO:teuthology.orchestra.run.smithi120.stdout: "slow_reply": 0, 2024-09-10T10:04:56.703 INFO:teuthology.orchestra.run.smithi120.stdout: "forward": 0, 2024-09-10T10:04:56.703 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_fetch_complete": 0, 2024-09-10T10:04:56.703 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_fetch_keys": 0, 2024-09-10T10:04:56.703 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_commit": 0, 2024-09-10T10:04:56.703 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_split": 0, 2024-09-10T10:04:56.703 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_merge": 0, 2024-09-10T10:04:56.703 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes": 1188, 2024-09-10T10:04:56.703 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_top": 0, 2024-09-10T10:04:56.704 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_bottom": 1188, 2024-09-10T10:04:56.704 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_pin_tail": 0, 2024-09-10T10:04:56.704 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_pinned": 1188, 2024-09-10T10:04:56.704 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_expired": 0, 2024-09-10T10:04:56.704 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_with_caps": 0, 2024-09-10T10:04:56.704 INFO:teuthology.orchestra.run.smithi120.stdout: "caps": 0, 2024-09-10T10:04:56.704 INFO:teuthology.orchestra.run.smithi120.stdout: "subtrees": 1, 2024-09-10T10:04:56.704 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse": 0, 2024-09-10T10:04:56.704 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_hit": 0, 2024-09-10T10:04:56.704 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_forward": 0, 2024-09-10T10:04:56.704 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_discover": 0, 2024-09-10T10:04:56.704 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_dir_fetch": 0, 2024-09-10T10:04:56.704 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_remote_ino": 0, 2024-09-10T10:04:56.704 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_lock": 0, 2024-09-10T10:04:56.704 INFO:teuthology.orchestra.run.smithi120.stdout: "load_cent": 0, 2024-09-10T10:04:56.705 INFO:teuthology.orchestra.run.smithi120.stdout: "q": 0, 2024-09-10T10:04:56.705 INFO:teuthology.orchestra.run.smithi120.stdout: "exported": 0, 2024-09-10T10:04:56.705 INFO:teuthology.orchestra.run.smithi120.stdout: "exported_inodes": 0, 2024-09-10T10:04:56.705 INFO:teuthology.orchestra.run.smithi120.stdout: "imported": 0, 2024-09-10T10:04:56.705 INFO:teuthology.orchestra.run.smithi120.stdout: "imported_inodes": 0, 2024-09-10T10:04:56.705 INFO:teuthology.orchestra.run.smithi120.stdout: "openino_dir_fetch": 0, 2024-09-10T10:04:56.705 INFO:teuthology.orchestra.run.smithi120.stdout: "openino_backtrace_fetch": 0, 2024-09-10T10:04:56.705 INFO:teuthology.orchestra.run.smithi120.stdout: "openino_peer_discover": 0, 2024-09-10T10:04:56.705 INFO:teuthology.orchestra.run.smithi120.stdout: "root_rfiles": 0, 2024-09-10T10:04:56.705 INFO:teuthology.orchestra.run.smithi120.stdout: "root_rbytes": 0, 2024-09-10T10:04:56.706 INFO:teuthology.orchestra.run.smithi120.stdout: "root_rsnaps": 0, 2024-09-10T10:04:56.706 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_backtrace_fetch": 0, 2024-09-10T10:04:56.706 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_set_tag": 0, 2024-09-10T10:04:56.706 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T10:04:56.706 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_inotable_repaired": 0, 2024-09-10T10:04:56.706 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_dir_inodes": 0, 2024-09-10T10:04:56.706 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_dir_base_inodes": 0, 2024-09-10T10:04:56.706 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_dirfrag_rstats": 0, 2024-09-10T10:04:56.706 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_file_inodes": 0, 2024-09-10T10:04:56.706 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_inode_file_caps": 0, 2024-09-10T10:04:56.706 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_cap_op_revoke": 0, 2024-09-10T10:04:56.707 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_cap_op_grant": 0, 2024-09-10T10:04:56.707 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T10:04:56.707 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T10:04:56.707 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T10:04:56.707 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_client_caps": 0, 2024-09-10T10:04:56.707 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_client_caps_dirty": 0, 2024-09-10T10:04:56.707 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_client_cap_release": 0, 2024-09-10T10:04:56.707 INFO:teuthology.orchestra.run.smithi120.stdout: "process_request_cap_release": 0 2024-09-10T10:04:56.707 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:56.707 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_cache": { 2024-09-10T10:04:56.707 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_update": 0, 2024-09-10T10:04:56.707 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_update_receipt": 0, 2024-09-10T10:04:56.707 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_try_discover": 0, 2024-09-10T10:04:56.707 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_send_discover": 0, 2024-09-10T10:04:56.708 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_handle_discover": 0, 2024-09-10T10:04:56.708 INFO:teuthology.orchestra.run.smithi120.stdout: "num_strays": 0, 2024-09-10T10:04:56.708 INFO:teuthology.orchestra.run.smithi120.stdout: "num_strays_delayed": 0, 2024-09-10T10:04:56.708 INFO:teuthology.orchestra.run.smithi120.stdout: "num_strays_enqueuing": 0, 2024-09-10T10:04:56.708 INFO:teuthology.orchestra.run.smithi120.stdout: "strays_created": 0, 2024-09-10T10:04:56.708 INFO:teuthology.orchestra.run.smithi120.stdout: "strays_enqueued": 0, 2024-09-10T10:04:56.708 INFO:teuthology.orchestra.run.smithi120.stdout: "strays_reintegrated": 0, 2024-09-10T10:04:56.708 INFO:teuthology.orchestra.run.smithi120.stdout: "strays_migrated": 0, 2024-09-10T10:04:56.708 INFO:teuthology.orchestra.run.smithi120.stdout: "num_recovering_processing": 0, 2024-09-10T10:04:56.708 INFO:teuthology.orchestra.run.smithi120.stdout: "num_recovering_enqueued": 0, 2024-09-10T10:04:56.708 INFO:teuthology.orchestra.run.smithi120.stdout: "num_recovering_prioritized": 0, 2024-09-10T10:04:56.708 INFO:teuthology.orchestra.run.smithi120.stdout: "recovery_started": 0, 2024-09-10T10:04:56.708 INFO:teuthology.orchestra.run.smithi120.stdout: "recovery_completed": 0, 2024-09-10T10:04:56.709 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_quiesce_path": 0, 2024-09-10T10:04:56.709 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_quiesce_inode": 0, 2024-09-10T10:04:56.709 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_enqueue_scrub": 0, 2024-09-10T10:04:56.709 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_exportdir": 0, 2024-09-10T10:04:56.709 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_flush": 0, 2024-09-10T10:04:56.709 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_fragmentdir": 0, 2024-09-10T10:04:56.709 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_fragstats": 0, 2024-09-10T10:04:56.709 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_inodestats": 0 2024-09-10T10:04:56.709 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:56.709 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_log": { 2024-09-10T10:04:56.709 INFO:teuthology.orchestra.run.smithi120.stdout: "evlrg": 0, 2024-09-10T10:04:56.709 INFO:teuthology.orchestra.run.smithi120.stdout: "evadd": 0, 2024-09-10T10:04:56.709 INFO:teuthology.orchestra.run.smithi120.stdout: "evex": 0, 2024-09-10T10:04:56.709 INFO:teuthology.orchestra.run.smithi120.stdout: "evtrm": 0, 2024-09-10T10:04:56.710 INFO:teuthology.orchestra.run.smithi120.stdout: "ev": 3695, 2024-09-10T10:04:56.710 INFO:teuthology.orchestra.run.smithi120.stdout: "evexg": 0, 2024-09-10T10:04:56.710 INFO:teuthology.orchestra.run.smithi120.stdout: "evexd": 0, 2024-09-10T10:04:56.710 INFO:teuthology.orchestra.run.smithi120.stdout: "segadd": 0, 2024-09-10T10:04:56.710 INFO:teuthology.orchestra.run.smithi120.stdout: "segex": 0, 2024-09-10T10:04:56.710 INFO:teuthology.orchestra.run.smithi120.stdout: "segtrm": 0, 2024-09-10T10:04:56.710 INFO:teuthology.orchestra.run.smithi120.stdout: "seg": 6, 2024-09-10T10:04:56.710 INFO:teuthology.orchestra.run.smithi120.stdout: "segmjr": 2, 2024-09-10T10:04:56.710 INFO:teuthology.orchestra.run.smithi120.stdout: "segexg": 0, 2024-09-10T10:04:56.710 INFO:teuthology.orchestra.run.smithi120.stdout: "segexd": 0, 2024-09-10T10:04:56.710 INFO:teuthology.orchestra.run.smithi120.stdout: "expos": 4194304, 2024-09-10T10:04:56.710 INFO:teuthology.orchestra.run.smithi120.stdout: "wrpos": 10600988, 2024-09-10T10:04:56.710 INFO:teuthology.orchestra.run.smithi120.stdout: "rdpos": 10598704, 2024-09-10T10:04:56.710 INFO:teuthology.orchestra.run.smithi120.stdout: "jlat": { 2024-09-10T10:04:56.710 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:56.711 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:56.711 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:56.711 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:56.711 INFO:teuthology.orchestra.run.smithi120.stdout: "replayed": 3695 2024-09-10T10:04:56.711 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:56.711 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_mem": { 2024-09-10T10:04:56.711 INFO:teuthology.orchestra.run.smithi120.stdout: "ino": 7, 2024-09-10T10:04:56.711 INFO:teuthology.orchestra.run.smithi120.stdout: "ino+": 1215, 2024-09-10T10:04:56.711 INFO:teuthology.orchestra.run.smithi120.stdout: "ino-": 1208, 2024-09-10T10:04:56.711 INFO:teuthology.orchestra.run.smithi120.stdout: "dir": 5, 2024-09-10T10:04:56.711 INFO:teuthology.orchestra.run.smithi120.stdout: "dir+": 84, 2024-09-10T10:04:56.711 INFO:teuthology.orchestra.run.smithi120.stdout: "dir-": 79, 2024-09-10T10:04:56.711 INFO:teuthology.orchestra.run.smithi120.stdout: "dn": 1188, 2024-09-10T10:04:56.712 INFO:teuthology.orchestra.run.smithi120.stdout: "dn+": 2460, 2024-09-10T10:04:56.712 INFO:teuthology.orchestra.run.smithi120.stdout: "dn-": 1272, 2024-09-10T10:04:56.712 INFO:teuthology.orchestra.run.smithi120.stdout: "cap": 0, 2024-09-10T10:04:56.712 INFO:teuthology.orchestra.run.smithi120.stdout: "cap+": 0, 2024-09-10T10:04:56.712 INFO:teuthology.orchestra.run.smithi120.stdout: "cap-": 0, 2024-09-10T10:04:56.712 INFO:teuthology.orchestra.run.smithi120.stdout: "rss": 51684, 2024-09-10T10:04:56.712 INFO:teuthology.orchestra.run.smithi120.stdout: "heap": 215328 2024-09-10T10:04:56.712 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:56.712 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_server": { 2024-09-10T10:04:56.713 INFO:teuthology.orchestra.run.smithi120.stdout: "dispatch_client_request": 0, 2024-09-10T10:04:56.713 INFO:teuthology.orchestra.run.smithi120.stdout: "dispatch_server_request": 0, 2024-09-10T10:04:56.713 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_client_request": 0, 2024-09-10T10:04:56.713 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_client_session": 0, 2024-09-10T10:04:56.713 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_peer_request": 0, 2024-09-10T10:04:56.713 INFO:teuthology.orchestra.run.smithi120.stdout: "req_create_latency": { 2024-09-10T10:04:56.713 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:56.713 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:56.713 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:56.713 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:56.713 INFO:teuthology.orchestra.run.smithi120.stdout: "req_getattr_latency": { 2024-09-10T10:04:56.713 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:56.713 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:56.713 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:56.714 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:56.714 INFO:teuthology.orchestra.run.smithi120.stdout: "req_getfilelock_latency": { 2024-09-10T10:04:56.714 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:56.714 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:56.714 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:56.714 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:56.714 INFO:teuthology.orchestra.run.smithi120.stdout: "req_link_latency": { 2024-09-10T10:04:56.714 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:56.714 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:56.714 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:56.714 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:56.714 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookup_latency": { 2024-09-10T10:04:56.714 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:56.714 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:56.714 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:56.715 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:56.715 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookuphash_latency": { 2024-09-10T10:04:56.715 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:56.715 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:56.715 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:56.715 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:56.715 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookupino_latency": { 2024-09-10T10:04:56.715 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:56.715 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:56.715 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:56.715 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:56.715 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookupname_latency": { 2024-09-10T10:04:56.716 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:56.716 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:56.716 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:56.716 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:56.716 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookupparent_latency": { 2024-09-10T10:04:56.716 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:56.716 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:56.716 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:56.716 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:56.716 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookupsnap_latency": { 2024-09-10T10:04:56.717 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:56.717 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:56.717 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:56.717 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:56.717 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lssnap_latency": { 2024-09-10T10:04:56.717 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:56.717 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:56.717 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:56.717 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:56.717 INFO:teuthology.orchestra.run.smithi120.stdout: "req_mkdir_latency": { 2024-09-10T10:04:56.718 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:56.718 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:56.718 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:56.718 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:56.718 INFO:teuthology.orchestra.run.smithi120.stdout: "req_mknod_latency": { 2024-09-10T10:04:56.718 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:56.718 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:56.718 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:56.719 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:56.719 INFO:teuthology.orchestra.run.smithi120.stdout: "req_mksnap_latency": { 2024-09-10T10:04:56.719 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:56.719 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:56.719 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:56.719 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:56.719 INFO:teuthology.orchestra.run.smithi120.stdout: "req_open_latency": { 2024-09-10T10:04:56.719 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:56.719 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:56.719 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:56.719 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:56.719 INFO:teuthology.orchestra.run.smithi120.stdout: "req_readdir_latency": { 2024-09-10T10:04:56.719 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:56.719 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:56.719 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:56.720 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:56.720 INFO:teuthology.orchestra.run.smithi120.stdout: "req_rename_latency": { 2024-09-10T10:04:56.720 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:56.720 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:56.720 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:56.720 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:56.720 INFO:teuthology.orchestra.run.smithi120.stdout: "req_renamesnap_latency": { 2024-09-10T10:04:56.720 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:56.720 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:56.720 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:56.720 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:56.720 INFO:teuthology.orchestra.run.smithi120.stdout: "req_snapdiff_latency": { 2024-09-10T10:04:56.720 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:56.720 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:56.721 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:56.721 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:56.721 INFO:teuthology.orchestra.run.smithi120.stdout: "req_rmdir_latency": { 2024-09-10T10:04:56.721 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:56.721 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:56.721 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:56.721 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:56.721 INFO:teuthology.orchestra.run.smithi120.stdout: "req_rmsnap_latency": { 2024-09-10T10:04:56.721 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:56.721 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:56.721 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:56.721 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:56.721 INFO:teuthology.orchestra.run.smithi120.stdout: "req_rmxattr_latency": { 2024-09-10T10:04:56.721 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:56.722 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:56.722 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:56.722 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:56.722 INFO:teuthology.orchestra.run.smithi120.stdout: "req_setattr_latency": { 2024-09-10T10:04:56.722 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:56.722 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:56.722 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:56.722 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:56.722 INFO:teuthology.orchestra.run.smithi120.stdout: "req_setdirlayout_latency": { 2024-09-10T10:04:56.722 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:56.722 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:56.722 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:56.722 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:56.722 INFO:teuthology.orchestra.run.smithi120.stdout: "req_setfilelock_latency": { 2024-09-10T10:04:56.722 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:56.723 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:56.723 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:56.723 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:56.723 INFO:teuthology.orchestra.run.smithi120.stdout: "req_setlayout_latency": { 2024-09-10T10:04:56.723 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:56.723 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:56.723 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:56.723 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:56.723 INFO:teuthology.orchestra.run.smithi120.stdout: "req_setxattr_latency": { 2024-09-10T10:04:56.723 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:56.723 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:56.723 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:56.723 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:56.723 INFO:teuthology.orchestra.run.smithi120.stdout: "req_symlink_latency": { 2024-09-10T10:04:56.724 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:56.724 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:56.724 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:56.724 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:56.724 INFO:teuthology.orchestra.run.smithi120.stdout: "req_unlink_latency": { 2024-09-10T10:04:56.724 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:56.724 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:56.724 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:56.724 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:56.724 INFO:teuthology.orchestra.run.smithi120.stdout: "cap_revoke_eviction": 0, 2024-09-10T10:04:56.724 INFO:teuthology.orchestra.run.smithi120.stdout: "cap_acquisition_throttle": 0, 2024-09-10T10:04:56.724 INFO:teuthology.orchestra.run.smithi120.stdout: "req_getvxattr_latency": { 2024-09-10T10:04:56.724 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:56.724 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:56.724 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:56.725 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:04:56.725 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:56.725 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_sessions": { 2024-09-10T10:04:56.725 INFO:teuthology.orchestra.run.smithi120.stdout: "session_count": 1, 2024-09-10T10:04:56.725 INFO:teuthology.orchestra.run.smithi120.stdout: "session_add": 2, 2024-09-10T10:04:56.725 INFO:teuthology.orchestra.run.smithi120.stdout: "session_remove": 1, 2024-09-10T10:04:56.725 INFO:teuthology.orchestra.run.smithi120.stdout: "sessions_open": 1, 2024-09-10T10:04:56.725 INFO:teuthology.orchestra.run.smithi120.stdout: "sessions_stale": 0, 2024-09-10T10:04:56.725 INFO:teuthology.orchestra.run.smithi120.stdout: "total_load": 0, 2024-09-10T10:04:56.725 INFO:teuthology.orchestra.run.smithi120.stdout: "average_load": 0, 2024-09-10T10:04:56.725 INFO:teuthology.orchestra.run.smithi120.stdout: "avg_session_uptime": 1949, 2024-09-10T10:04:56.725 INFO:teuthology.orchestra.run.smithi120.stdout: "mdthresh_evicted": 0 2024-09-10T10:04:56.725 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:56.726 INFO:teuthology.orchestra.run.smithi120.stdout: "mempool": { 2024-09-10T10:04:56.726 INFO:teuthology.orchestra.run.smithi120.stdout: "bloom_filter_bytes": 0, 2024-09-10T10:04:56.726 INFO:teuthology.orchestra.run.smithi120.stdout: "bloom_filter_items": 0, 2024-09-10T10:04:56.726 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T10:04:56.726 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_alloc_items": 0, 2024-09-10T10:04:56.726 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T10:04:56.726 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_data_items": 0, 2024-09-10T10:04:56.726 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T10:04:56.726 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T10:04:56.726 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T10:04:56.726 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T10:04:56.726 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T10:04:56.726 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_other_items": 0, 2024-09-10T10:04:56.726 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T10:04:56.727 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T10:04:56.727 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_extent_bytes": 0, 2024-09-10T10:04:56.727 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_extent_items": 0, 2024-09-10T10:04:56.727 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_blob_bytes": 0, 2024-09-10T10:04:56.727 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_blob_items": 0, 2024-09-10T10:04:56.727 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T10:04:56.727 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T10:04:56.727 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T10:04:56.727 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T10:04:56.727 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T10:04:56.727 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_fsck_items": 0, 2024-09-10T10:04:56.727 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_txc_bytes": 0, 2024-09-10T10:04:56.727 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_txc_items": 0, 2024-09-10T10:04:56.727 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T10:04:56.727 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T10:04:56.727 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_writing_bytes": 0, 2024-09-10T10:04:56.728 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_writing_items": 0, 2024-09-10T10:04:56.728 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_bytes": 0, 2024-09-10T10:04:56.728 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_items": 0, 2024-09-10T10:04:56.728 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T10:04:56.728 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_file_reader_items": 0, 2024-09-10T10:04:56.728 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T10:04:56.728 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_file_writer_items": 0, 2024-09-10T10:04:56.728 INFO:teuthology.orchestra.run.smithi120.stdout: "buffer_anon_bytes": 181343, 2024-09-10T10:04:56.728 INFO:teuthology.orchestra.run.smithi120.stdout: "buffer_anon_items": 47, 2024-09-10T10:04:56.728 INFO:teuthology.orchestra.run.smithi120.stdout: "buffer_meta_bytes": 0, 2024-09-10T10:04:56.728 INFO:teuthology.orchestra.run.smithi120.stdout: "buffer_meta_items": 0, 2024-09-10T10:04:56.728 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_bytes": 0, 2024-09-10T10:04:56.728 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_items": 0, 2024-09-10T10:04:56.728 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_mapbl_bytes": 0, 2024-09-10T10:04:56.728 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_mapbl_items": 0, 2024-09-10T10:04:56.729 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pglog_bytes": 0, 2024-09-10T10:04:56.729 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pglog_items": 0, 2024-09-10T10:04:56.729 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_bytes": 9144, 2024-09-10T10:04:56.729 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_items": 172, 2024-09-10T10:04:56.729 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T10:04:56.729 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_mapping_items": 0, 2024-09-10T10:04:56.729 INFO:teuthology.orchestra.run.smithi120.stdout: "pgmap_bytes": 0, 2024-09-10T10:04:56.729 INFO:teuthology.orchestra.run.smithi120.stdout: "pgmap_items": 0, 2024-09-10T10:04:56.729 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_co_bytes": 858920, 2024-09-10T10:04:56.729 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_co_items": 4814, 2024-09-10T10:04:56.729 INFO:teuthology.orchestra.run.smithi120.stdout: "unittest_1_bytes": 0, 2024-09-10T10:04:56.729 INFO:teuthology.orchestra.run.smithi120.stdout: "unittest_1_items": 0, 2024-09-10T10:04:56.730 INFO:teuthology.orchestra.run.smithi120.stdout: "unittest_2_bytes": 0, 2024-09-10T10:04:56.730 INFO:teuthology.orchestra.run.smithi120.stdout: "unittest_2_items": 0 2024-09-10T10:04:56.730 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:56.730 INFO:teuthology.orchestra.run.smithi120.stdout: "objecter": { 2024-09-10T10:04:56.730 INFO:teuthology.orchestra.run.smithi120.stdout: "op_active": 0, 2024-09-10T10:04:56.730 INFO:teuthology.orchestra.run.smithi120.stdout: "op_laggy": 0, 2024-09-10T10:04:56.730 INFO:teuthology.orchestra.run.smithi120.stdout: "op_send": 2094, 2024-09-10T10:04:56.730 INFO:teuthology.orchestra.run.smithi120.stdout: "op_send_bytes": 21, 2024-09-10T10:04:56.730 INFO:teuthology.orchestra.run.smithi120.stdout: "op_resend": 3, 2024-09-10T10:04:56.730 INFO:teuthology.orchestra.run.smithi120.stdout: "op_reply": 2094, 2024-09-10T10:04:56.730 INFO:teuthology.orchestra.run.smithi120.stdout: "op_latency": { 2024-09-10T10:04:56.730 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 2094, 2024-09-10T10:04:56.730 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 1.088974644, 2024-09-10T10:04:56.730 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000520045 2024-09-10T10:04:56.731 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:56.731 INFO:teuthology.orchestra.run.smithi120.stdout: "op_inflight": 0, 2024-09-10T10:04:56.731 INFO:teuthology.orchestra.run.smithi120.stdout: "oplen_avg": { 2024-09-10T10:04:56.731 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 2094, 2024-09-10T10:04:56.731 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 2095 2024-09-10T10:04:56.731 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:56.731 INFO:teuthology.orchestra.run.smithi120.stdout: "op": 2094, 2024-09-10T10:04:56.731 INFO:teuthology.orchestra.run.smithi120.stdout: "op_r": 2094, 2024-09-10T10:04:56.731 INFO:teuthology.orchestra.run.smithi120.stdout: "op_w": 0, 2024-09-10T10:04:56.731 INFO:teuthology.orchestra.run.smithi120.stdout: "op_rmw": 0, 2024-09-10T10:04:56.731 INFO:teuthology.orchestra.run.smithi120.stdout: "op_pg": 0, 2024-09-10T10:04:56.731 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_stat": 1325, 2024-09-10T10:04:56.731 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_create": 0, 2024-09-10T10:04:56.731 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_read": 767, 2024-09-10T10:04:56.732 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_write": 0, 2024-09-10T10:04:56.732 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_writefull": 0, 2024-09-10T10:04:56.732 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_writesame": 0, 2024-09-10T10:04:56.732 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_append": 0, 2024-09-10T10:04:56.732 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_zero": 0, 2024-09-10T10:04:56.732 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_truncate": 0, 2024-09-10T10:04:56.732 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_delete": 0, 2024-09-10T10:04:56.732 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_mapext": 0, 2024-09-10T10:04:56.732 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_sparse_read": 0, 2024-09-10T10:04:56.732 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_clonerange": 0, 2024-09-10T10:04:56.732 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_getxattr": 1, 2024-09-10T10:04:56.732 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_setxattr": 0, 2024-09-10T10:04:56.732 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_cmpxattr": 0, 2024-09-10T10:04:56.732 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_rmxattr": 0, 2024-09-10T10:04:56.733 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_resetxattrs": 0, 2024-09-10T10:04:56.733 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_call": 0, 2024-09-10T10:04:56.733 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_watch": 0, 2024-09-10T10:04:56.733 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_notify": 0, 2024-09-10T10:04:56.733 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T10:04:56.733 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_pgls": 0, 2024-09-10T10:04:56.733 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_pgls_filter": 0, 2024-09-10T10:04:56.733 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_other": 0, 2024-09-10T10:04:56.733 INFO:teuthology.orchestra.run.smithi120.stdout: "linger_active": 0, 2024-09-10T10:04:56.733 INFO:teuthology.orchestra.run.smithi120.stdout: "linger_send": 0, 2024-09-10T10:04:56.734 INFO:teuthology.orchestra.run.smithi120.stdout: "linger_resend": 0, 2024-09-10T10:04:56.734 INFO:teuthology.orchestra.run.smithi120.stdout: "linger_ping": 0, 2024-09-10T10:04:56.734 INFO:teuthology.orchestra.run.smithi120.stdout: "poolop_active": 0, 2024-09-10T10:04:56.734 INFO:teuthology.orchestra.run.smithi120.stdout: "poolop_send": 0, 2024-09-10T10:04:56.734 INFO:teuthology.orchestra.run.smithi120.stdout: "poolop_resend": 0, 2024-09-10T10:04:56.734 INFO:teuthology.orchestra.run.smithi120.stdout: "poolstat_active": 0, 2024-09-10T10:04:56.734 INFO:teuthology.orchestra.run.smithi120.stdout: "poolstat_send": 0, 2024-09-10T10:04:56.734 INFO:teuthology.orchestra.run.smithi120.stdout: "poolstat_resend": 0, 2024-09-10T10:04:56.734 INFO:teuthology.orchestra.run.smithi120.stdout: "statfs_active": 0, 2024-09-10T10:04:56.734 INFO:teuthology.orchestra.run.smithi120.stdout: "statfs_send": 0, 2024-09-10T10:04:56.734 INFO:teuthology.orchestra.run.smithi120.stdout: "statfs_resend": 0, 2024-09-10T10:04:56.734 INFO:teuthology.orchestra.run.smithi120.stdout: "command_active": 0, 2024-09-10T10:04:56.735 INFO:teuthology.orchestra.run.smithi120.stdout: "command_send": 0, 2024-09-10T10:04:56.735 INFO:teuthology.orchestra.run.smithi120.stdout: "command_resend": 0, 2024-09-10T10:04:56.735 INFO:teuthology.orchestra.run.smithi120.stdout: "map_epoch": 96, 2024-09-10T10:04:56.735 INFO:teuthology.orchestra.run.smithi120.stdout: "map_full": 0, 2024-09-10T10:04:56.735 INFO:teuthology.orchestra.run.smithi120.stdout: "map_inc": 2, 2024-09-10T10:04:56.735 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_sessions": 4, 2024-09-10T10:04:56.735 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_session_open": 4, 2024-09-10T10:04:56.735 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_session_close": 0, 2024-09-10T10:04:56.735 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_laggy": 0, 2024-09-10T10:04:56.735 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_wr": 0, 2024-09-10T10:04:56.735 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_rd": 2, 2024-09-10T10:04:56.735 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_del": 0 2024-09-10T10:04:56.735 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:56.735 INFO:teuthology.orchestra.run.smithi120.stdout: "oft": { 2024-09-10T10:04:56.736 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_total_objs": 0, 2024-09-10T10:04:56.736 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_total_kv_pairs": 0, 2024-09-10T10:04:56.736 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_total_updates": 0, 2024-09-10T10:04:56.736 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_total_removes": 0 2024-09-10T10:04:56.736 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:56.736 INFO:teuthology.orchestra.run.smithi120.stdout: "osd-slow-ops": { 2024-09-10T10:04:56.736 INFO:teuthology.orchestra.run.smithi120.stdout: "slow_ops_count": 0 2024-09-10T10:04:56.736 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:56.736 INFO:teuthology.orchestra.run.smithi120.stdout: "purge_queue": { 2024-09-10T10:04:56.736 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executing_ops": 0, 2024-09-10T10:04:56.736 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executing_ops_high_water": 0, 2024-09-10T10:04:56.736 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executing": 0, 2024-09-10T10:04:56.736 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executing_high_water": 0, 2024-09-10T10:04:56.736 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executed_ops": 0, 2024-09-10T10:04:56.736 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executed": 0, 2024-09-10T10:04:56.737 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_item_in_journal": 0 2024-09-10T10:04:56.737 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:56.737 INFO:teuthology.orchestra.run.smithi120.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T10:04:56.737 INFO:teuthology.orchestra.run.smithi120.stdout: "val": 0, 2024-09-10T10:04:56.737 INFO:teuthology.orchestra.run.smithi120.stdout: "max": 104857600, 2024-09-10T10:04:56.737 INFO:teuthology.orchestra.run.smithi120.stdout: "get_started": 0, 2024-09-10T10:04:56.737 INFO:teuthology.orchestra.run.smithi120.stdout: "get": 2332, 2024-09-10T10:04:56.737 INFO:teuthology.orchestra.run.smithi120.stdout: "get_sum": 7351375, 2024-09-10T10:04:56.737 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_fail": 0, 2024-09-10T10:04:56.737 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_success": 2332, 2024-09-10T10:04:56.737 INFO:teuthology.orchestra.run.smithi120.stdout: "take": 0, 2024-09-10T10:04:56.737 INFO:teuthology.orchestra.run.smithi120.stdout: "take_sum": 0, 2024-09-10T10:04:56.737 INFO:teuthology.orchestra.run.smithi120.stdout: "put": 2332, 2024-09-10T10:04:56.737 INFO:teuthology.orchestra.run.smithi120.stdout: "put_sum": 7351375, 2024-09-10T10:04:56.738 INFO:teuthology.orchestra.run.smithi120.stdout: "wait": { 2024-09-10T10:04:56.738 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:56.738 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:56.738 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:56.738 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:04:56.738 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:56.738 INFO:teuthology.orchestra.run.smithi120.stdout: "throttle-objecter_bytes": { 2024-09-10T10:04:56.738 INFO:teuthology.orchestra.run.smithi120.stdout: "val": 0, 2024-09-10T10:04:56.738 INFO:teuthology.orchestra.run.smithi120.stdout: "max": 104857600, 2024-09-10T10:04:56.738 INFO:teuthology.orchestra.run.smithi120.stdout: "get_started": 0, 2024-09-10T10:04:56.738 INFO:teuthology.orchestra.run.smithi120.stdout: "get": 0, 2024-09-10T10:04:56.738 INFO:teuthology.orchestra.run.smithi120.stdout: "get_sum": 0, 2024-09-10T10:04:56.738 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_fail": 0, 2024-09-10T10:04:56.738 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_success": 0, 2024-09-10T10:04:56.738 INFO:teuthology.orchestra.run.smithi120.stdout: "take": 2094, 2024-09-10T10:04:56.738 INFO:teuthology.orchestra.run.smithi120.stdout: "take_sum": 6406689, 2024-09-10T10:04:56.739 INFO:teuthology.orchestra.run.smithi120.stdout: "put": 2094, 2024-09-10T10:04:56.739 INFO:teuthology.orchestra.run.smithi120.stdout: "put_sum": 6406689, 2024-09-10T10:04:56.739 INFO:teuthology.orchestra.run.smithi120.stdout: "wait": { 2024-09-10T10:04:56.739 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:56.739 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:56.739 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:56.739 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:04:56.739 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:56.739 INFO:teuthology.orchestra.run.smithi120.stdout: "throttle-objecter_ops": { 2024-09-10T10:04:56.739 INFO:teuthology.orchestra.run.smithi120.stdout: "val": 0, 2024-09-10T10:04:56.739 INFO:teuthology.orchestra.run.smithi120.stdout: "max": 1024, 2024-09-10T10:04:56.739 INFO:teuthology.orchestra.run.smithi120.stdout: "get_started": 0, 2024-09-10T10:04:56.739 INFO:teuthology.orchestra.run.smithi120.stdout: "get": 0, 2024-09-10T10:04:56.739 INFO:teuthology.orchestra.run.smithi120.stdout: "get_sum": 0, 2024-09-10T10:04:56.739 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_fail": 0, 2024-09-10T10:04:56.739 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_success": 0, 2024-09-10T10:04:56.740 INFO:teuthology.orchestra.run.smithi120.stdout: "take": 2094, 2024-09-10T10:04:56.740 INFO:teuthology.orchestra.run.smithi120.stdout: "take_sum": 2094, 2024-09-10T10:04:56.740 INFO:teuthology.orchestra.run.smithi120.stdout: "put": 2094, 2024-09-10T10:04:56.740 INFO:teuthology.orchestra.run.smithi120.stdout: "put_sum": 2094, 2024-09-10T10:04:56.740 INFO:teuthology.orchestra.run.smithi120.stdout: "wait": { 2024-09-10T10:04:56.740 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:56.740 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:56.740 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:56.740 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:04:56.740 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:56.741 INFO:teuthology.orchestra.run.smithi120.stdout: "throttle-write_buf_throttle": { 2024-09-10T10:04:56.741 INFO:teuthology.orchestra.run.smithi120.stdout: "val": 0, 2024-09-10T10:04:56.741 INFO:teuthology.orchestra.run.smithi120.stdout: "max": 3758096384, 2024-09-10T10:04:56.741 INFO:teuthology.orchestra.run.smithi120.stdout: "get_started": 0, 2024-09-10T10:04:56.741 INFO:teuthology.orchestra.run.smithi120.stdout: "get": 0, 2024-09-10T10:04:56.741 INFO:teuthology.orchestra.run.smithi120.stdout: "get_sum": 0, 2024-09-10T10:04:56.741 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_fail": 0, 2024-09-10T10:04:56.741 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_success": 0, 2024-09-10T10:04:56.741 INFO:teuthology.orchestra.run.smithi120.stdout: "take": 0, 2024-09-10T10:04:56.741 INFO:teuthology.orchestra.run.smithi120.stdout: "take_sum": 0, 2024-09-10T10:04:56.741 INFO:teuthology.orchestra.run.smithi120.stdout: "put": 0, 2024-09-10T10:04:56.741 INFO:teuthology.orchestra.run.smithi120.stdout: "put_sum": 0, 2024-09-10T10:04:56.741 INFO:teuthology.orchestra.run.smithi120.stdout: "wait": { 2024-09-10T10:04:56.742 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:56.742 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:56.742 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:56.742 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:04:56.742 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:04:56.742 INFO:teuthology.orchestra.run.smithi120.stdout: "throttle-write_buf_throttle-0x5648fec28b40": { 2024-09-10T10:04:56.742 INFO:teuthology.orchestra.run.smithi120.stdout: "val": 0, 2024-09-10T10:04:56.742 INFO:teuthology.orchestra.run.smithi120.stdout: "max": 3758096384, 2024-09-10T10:04:56.742 INFO:teuthology.orchestra.run.smithi120.stdout: "get_started": 0, 2024-09-10T10:04:56.742 INFO:teuthology.orchestra.run.smithi120.stdout: "get": 0, 2024-09-10T10:04:56.742 INFO:teuthology.orchestra.run.smithi120.stdout: "get_sum": 0, 2024-09-10T10:04:56.743 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_fail": 0, 2024-09-10T10:04:56.743 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_success": 0, 2024-09-10T10:04:56.743 INFO:teuthology.orchestra.run.smithi120.stdout: "take": 0, 2024-09-10T10:04:56.743 INFO:teuthology.orchestra.run.smithi120.stdout: "take_sum": 0, 2024-09-10T10:04:56.743 INFO:teuthology.orchestra.run.smithi120.stdout: "put": 0, 2024-09-10T10:04:56.743 INFO:teuthology.orchestra.run.smithi120.stdout: "put_sum": 0, 2024-09-10T10:04:56.743 INFO:teuthology.orchestra.run.smithi120.stdout: "wait": { 2024-09-10T10:04:56.743 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-10T10:04:56.743 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-10T10:04:56.743 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-10T10:04:56.743 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:04:56.743 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:04:56.743 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:04:57.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:04:56 smithi134 ceph-mon[35674]: pgmap v684: 129 pgs: 129 active+clean; 115 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T10:04:57.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:04:56 smithi120 ceph-mon[34770]: pgmap v684: 129 pgs: 129 active+clean; 115 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T10:04:57.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:04:56 smithi077 ceph-mon[30319]: pgmap v684: 129 pgs: 129 active+clean; 115 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T10:04:57.525 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-09-10T10:04:57.525 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-09-10T10:04:57.526 INFO:tasks.check_counter:Daemon mds.k mds.dir_split=0 2024-09-10T10:04:57.526 DEBUG:tasks.check_counter:Getting stats from c 2024-09-10T10:04:57.526 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph daemon mds.c perf dump 2024-09-10T10:04:57.786 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T10:04:58.356 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:04:58.356 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T10:04:58.356 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 6072, 2024-09-10T10:04:58.357 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 6622, 2024-09-10T10:04:58.357 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 3192561, 2024-09-10T10:04:58.357 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 6516610, 2024-09-10T10:04:58.357 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 11, 2024-09-10T10:04:58.357 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 4, 2024-09-10T10:04:58.357 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.465095236, 2024-09-10T10:04:58.357 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.178178545, 2024-09-10T10:04:58.357 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.189547302, 2024-09-10T10:04:58.358 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.091851661, 2024-09-10T10:04:58.358 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:04:58.358 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 6621, 2024-09-10T10:04:58.358 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.511722730, 2024-09-10T10:04:58.358 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000077287 2024-09-10T10:04:58.358 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:04:58.358 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:04:58.358 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 5900, 2024-09-10T10:04:58.358 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.001073797, 2024-09-10T10:04:58.358 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000181 2024-09-10T10:04:58.358 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:04:58.358 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-09-10T10:04:58.359 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 448 2024-09-10T10:04:58.359 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:04:58.359 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T10:04:58.359 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 11158, 2024-09-10T10:04:58.359 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 15061, 2024-09-10T10:04:58.359 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 3357011, 2024-09-10T10:04:58.359 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 11064630, 2024-09-10T10:04:58.359 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 23, 2024-09-10T10:04:58.359 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 5, 2024-09-10T10:04:58.360 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.661369898, 2024-09-10T10:04:58.360 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.261466539, 2024-09-10T10:04:58.360 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.316254803, 2024-09-10T10:04:58.360 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.119071421, 2024-09-10T10:04:58.360 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:04:58.360 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 15059, 2024-09-10T10:04:58.360 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 1.156880308, 2024-09-10T10:04:58.360 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000076823 2024-09-10T10:04:58.360 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:04:58.360 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:04:58.360 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 8343, 2024-09-10T10:04:58.360 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.001996493, 2024-09-10T10:04:58.360 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000239 2024-09-10T10:04:58.361 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:04:58.361 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-10T10:04:58.361 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 344416 2024-09-10T10:04:58.361 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:04:58.361 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T10:04:58.361 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 5270, 2024-09-10T10:04:58.361 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 6084, 2024-09-10T10:04:58.361 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 2354412, 2024-09-10T10:04:58.361 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 2307885, 2024-09-10T10:04:58.361 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 26, 2024-09-10T10:04:58.361 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 6, 2024-09-10T10:04:58.361 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.490871552, 2024-09-10T10:04:58.361 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.222088487, 2024-09-10T10:04:58.362 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.157530732, 2024-09-10T10:04:58.362 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.097721773, 2024-09-10T10:04:58.362 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:04:58.362 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 6083, 2024-09-10T10:04:58.362 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.466330730, 2024-09-10T10:04:58.362 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000076661 2024-09-10T10:04:58.362 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:04:58.362 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:04:58.362 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 3608, 2024-09-10T10:04:58.363 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000576041, 2024-09-10T10:04:58.363 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000159 2024-09-10T10:04:58.363 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:04:58.363 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 597168, 2024-09-10T10:04:58.363 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 142656 2024-09-10T10:04:58.363 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:04:58.363 INFO:teuthology.orchestra.run.smithi134.stdout: "cct": { 2024-09-10T10:04:58.363 INFO:teuthology.orchestra.run.smithi134.stdout: "total_workers": 1, 2024-09-10T10:04:58.363 INFO:teuthology.orchestra.run.smithi134.stdout: "unhealthy_workers": 0 2024-09-10T10:04:58.363 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:04:58.363 INFO:teuthology.orchestra.run.smithi134.stdout: "finisher-MDSRank": { 2024-09-10T10:04:58.363 INFO:teuthology.orchestra.run.smithi134.stdout: "queue_len": 0, 2024-09-10T10:04:58.363 INFO:teuthology.orchestra.run.smithi134.stdout: "complete_latency": { 2024-09-10T10:04:58.363 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 5901, 2024-09-10T10:04:58.364 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 2.818850445, 2024-09-10T10:04:58.364 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000477690 2024-09-10T10:04:58.364 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:04:58.364 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:04:58.364 INFO:teuthology.orchestra.run.smithi134.stdout: "finisher-PurgeQueue": { 2024-09-10T10:04:58.364 INFO:teuthology.orchestra.run.smithi134.stdout: "queue_len": 0, 2024-09-10T10:04:58.364 INFO:teuthology.orchestra.run.smithi134.stdout: "complete_latency": { 2024-09-10T10:04:58.364 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 2548, 2024-09-10T10:04:58.364 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.243820504, 2024-09-10T10:04:58.364 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000095690 2024-09-10T10:04:58.364 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:04:58.364 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:04:58.364 INFO:teuthology.orchestra.run.smithi134.stdout: "mds": { 2024-09-10T10:04:58.365 INFO:teuthology.orchestra.run.smithi134.stdout: "request": 3535, 2024-09-10T10:04:58.365 INFO:teuthology.orchestra.run.smithi134.stdout: "reply": 3531, 2024-09-10T10:04:58.365 INFO:teuthology.orchestra.run.smithi134.stdout: "reply_latency": { 2024-09-10T10:04:58.365 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 3531, 2024-09-10T10:04:58.365 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 2.757513466, 2024-09-10T10:04:58.365 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000780944 2024-09-10T10:04:58.365 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:04:58.365 INFO:teuthology.orchestra.run.smithi134.stdout: "slow_reply": 0, 2024-09-10T10:04:58.365 INFO:teuthology.orchestra.run.smithi134.stdout: "forward": 4, 2024-09-10T10:04:58.365 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_fetch_complete": 0, 2024-09-10T10:04:58.365 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_fetch_keys": 0, 2024-09-10T10:04:58.366 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_commit": 11, 2024-09-10T10:04:58.366 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_split": 0, 2024-09-10T10:04:58.366 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_merge": 0, 2024-09-10T10:04:58.366 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes": 20, 2024-09-10T10:04:58.366 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_top": 4, 2024-09-10T10:04:58.366 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_bottom": 3, 2024-09-10T10:04:58.366 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_pin_tail": 13, 2024-09-10T10:04:58.366 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_pinned": 13, 2024-09-10T10:04:58.366 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_expired": 108, 2024-09-10T10:04:58.366 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_with_caps": 0, 2024-09-10T10:04:58.366 INFO:teuthology.orchestra.run.smithi134.stdout: "caps": 0, 2024-09-10T10:04:58.366 INFO:teuthology.orchestra.run.smithi134.stdout: "subtrees": 5, 2024-09-10T10:04:58.366 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse": 7157, 2024-09-10T10:04:58.366 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_hit": 5044, 2024-09-10T10:04:58.366 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_forward": 4, 2024-09-10T10:04:58.367 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_discover": 23, 2024-09-10T10:04:58.367 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_dir_fetch": 0, 2024-09-10T10:04:58.367 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_remote_ino": 0, 2024-09-10T10:04:58.367 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_lock": 0, 2024-09-10T10:04:58.367 INFO:teuthology.orchestra.run.smithi134.stdout: "load_cent": 0, 2024-09-10T10:04:58.367 INFO:teuthology.orchestra.run.smithi134.stdout: "q": 0, 2024-09-10T10:04:58.367 INFO:teuthology.orchestra.run.smithi134.stdout: "exported": 37, 2024-09-10T10:04:58.367 INFO:teuthology.orchestra.run.smithi134.stdout: "exported_inodes": 37, 2024-09-10T10:04:58.367 INFO:teuthology.orchestra.run.smithi134.stdout: "imported": 37, 2024-09-10T10:04:58.367 INFO:teuthology.orchestra.run.smithi134.stdout: "imported_inodes": 1097, 2024-09-10T10:04:58.367 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_dir_fetch": 0, 2024-09-10T10:04:58.367 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_backtrace_fetch": 8399, 2024-09-10T10:04:58.368 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_peer_discover": 0, 2024-09-10T10:04:58.368 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rfiles": 4215, 2024-09-10T10:04:58.368 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rbytes": 81947162, 2024-09-10T10:04:58.368 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rsnaps": 0, 2024-09-10T10:04:58.368 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_backtrace_fetch": 8399, 2024-09-10T10:04:58.368 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_set_tag": 0, 2024-09-10T10:04:58.368 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T10:04:58.368 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_inotable_repaired": 0, 2024-09-10T10:04:58.368 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dir_inodes": 21, 2024-09-10T10:04:58.368 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dir_base_inodes": 0, 2024-09-10T10:04:58.368 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dirfrag_rstats": 21, 2024-09-10T10:04:58.368 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_file_inodes": 8378, 2024-09-10T10:04:58.369 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_inode_file_caps": 0, 2024-09-10T10:04:58.369 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_revoke": 139, 2024-09-10T10:04:58.369 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_grant": 1734, 2024-09-10T10:04:58.369 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T10:04:58.369 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T10:04:58.369 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T10:04:58.369 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_caps": 1648, 2024-09-10T10:04:58.369 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_caps_dirty": 1011, 2024-09-10T10:04:58.369 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_cap_release": 48, 2024-09-10T10:04:58.369 INFO:teuthology.orchestra.run.smithi134.stdout: "process_request_cap_release": 2750 2024-09-10T10:04:58.369 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:04:58.369 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_cache": { 2024-09-10T10:04:58.369 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_update": 0, 2024-09-10T10:04:58.370 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_update_receipt": 0, 2024-09-10T10:04:58.370 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_try_discover": 0, 2024-09-10T10:04:58.370 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_send_discover": 23, 2024-09-10T10:04:58.370 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_handle_discover": 37, 2024-09-10T10:04:58.370 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays": 0, 2024-09-10T10:04:58.370 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays_delayed": 0, 2024-09-10T10:04:58.370 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays_enqueuing": 0, 2024-09-10T10:04:58.370 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_created": 1327, 2024-09-10T10:04:58.370 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_enqueued": 1327, 2024-09-10T10:04:58.370 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_reintegrated": 0, 2024-09-10T10:04:58.370 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_migrated": 0, 2024-09-10T10:04:58.370 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_processing": 0, 2024-09-10T10:04:58.370 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_enqueued": 0, 2024-09-10T10:04:58.370 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_prioritized": 0, 2024-09-10T10:04:58.370 INFO:teuthology.orchestra.run.smithi134.stdout: "recovery_started": 0, 2024-09-10T10:04:58.371 INFO:teuthology.orchestra.run.smithi134.stdout: "recovery_completed": 0, 2024-09-10T10:04:58.371 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_quiesce_path": 8, 2024-09-10T10:04:58.371 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_quiesce_inode": 5593, 2024-09-10T10:04:58.371 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_enqueue_scrub": 0, 2024-09-10T10:04:58.371 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_exportdir": 37, 2024-09-10T10:04:58.371 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_flush": 0, 2024-09-10T10:04:58.371 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_fragmentdir": 0, 2024-09-10T10:04:58.371 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_fragstats": 0, 2024-09-10T10:04:58.371 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_inodestats": 0 2024-09-10T10:04:58.371 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:04:58.371 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_log": { 2024-09-10T10:04:58.371 INFO:teuthology.orchestra.run.smithi134.stdout: "evlrg": 0, 2024-09-10T10:04:58.371 INFO:teuthology.orchestra.run.smithi134.stdout: "evadd": 4674, 2024-09-10T10:04:58.371 INFO:teuthology.orchestra.run.smithi134.stdout: "evex": 0, 2024-09-10T10:04:58.371 INFO:teuthology.orchestra.run.smithi134.stdout: "evtrm": 0, 2024-09-10T10:04:58.371 INFO:teuthology.orchestra.run.smithi134.stdout: "ev": 4674, 2024-09-10T10:04:58.372 INFO:teuthology.orchestra.run.smithi134.stdout: "evexg": 0, 2024-09-10T10:04:58.372 INFO:teuthology.orchestra.run.smithi134.stdout: "evexd": 0, 2024-09-10T10:04:58.372 INFO:teuthology.orchestra.run.smithi134.stdout: "segadd": 7, 2024-09-10T10:04:58.372 INFO:teuthology.orchestra.run.smithi134.stdout: "segex": 0, 2024-09-10T10:04:58.372 INFO:teuthology.orchestra.run.smithi134.stdout: "segtrm": 0, 2024-09-10T10:04:58.372 INFO:teuthology.orchestra.run.smithi134.stdout: "seg": 7, 2024-09-10T10:04:58.372 INFO:teuthology.orchestra.run.smithi134.stdout: "segmjr": 2, 2024-09-10T10:04:58.372 INFO:teuthology.orchestra.run.smithi134.stdout: "segexg": 0, 2024-09-10T10:04:58.372 INFO:teuthology.orchestra.run.smithi134.stdout: "segexd": 0, 2024-09-10T10:04:58.372 INFO:teuthology.orchestra.run.smithi134.stdout: "expos": 4194304, 2024-09-10T10:04:58.372 INFO:teuthology.orchestra.run.smithi134.stdout: "wrpos": 11691921, 2024-09-10T10:04:58.372 INFO:teuthology.orchestra.run.smithi134.stdout: "rdpos": 0, 2024-09-10T10:04:58.372 INFO:teuthology.orchestra.run.smithi134.stdout: "jlat": { 2024-09-10T10:04:58.372 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 191, 2024-09-10T10:04:58.373 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 2.699714057, 2024-09-10T10:04:58.373 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.014134628 2024-09-10T10:04:58.373 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:04:58.373 INFO:teuthology.orchestra.run.smithi134.stdout: "replayed": 0 2024-09-10T10:04:58.373 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:04:58.373 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_mem": { 2024-09-10T10:04:58.373 INFO:teuthology.orchestra.run.smithi134.stdout: "ino": 26, 2024-09-10T10:04:58.373 INFO:teuthology.orchestra.run.smithi134.stdout: "ino+": 1370, 2024-09-10T10:04:58.373 INFO:teuthology.orchestra.run.smithi134.stdout: "ino-": 1344, 2024-09-10T10:04:58.373 INFO:teuthology.orchestra.run.smithi134.stdout: "dir": 25, 2024-09-10T10:04:58.373 INFO:teuthology.orchestra.run.smithi134.stdout: "dir+": 70, 2024-09-10T10:04:58.373 INFO:teuthology.orchestra.run.smithi134.stdout: "dir-": 45, 2024-09-10T10:04:58.373 INFO:teuthology.orchestra.run.smithi134.stdout: "dn": 20, 2024-09-10T10:04:58.373 INFO:teuthology.orchestra.run.smithi134.stdout: "dn+": 2627, 2024-09-10T10:04:58.374 INFO:teuthology.orchestra.run.smithi134.stdout: "dn-": 2607, 2024-09-10T10:04:58.374 INFO:teuthology.orchestra.run.smithi134.stdout: "cap": 0, 2024-09-10T10:04:58.374 INFO:teuthology.orchestra.run.smithi134.stdout: "cap+": 2432, 2024-09-10T10:04:58.374 INFO:teuthology.orchestra.run.smithi134.stdout: "cap-": 2432, 2024-09-10T10:04:58.374 INFO:teuthology.orchestra.run.smithi134.stdout: "rss": 69848, 2024-09-10T10:04:58.374 INFO:teuthology.orchestra.run.smithi134.stdout: "heap": 223520 2024-09-10T10:04:58.374 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:04:58.374 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_server": { 2024-09-10T10:04:58.374 INFO:teuthology.orchestra.run.smithi134.stdout: "dispatch_client_request": 5319, 2024-09-10T10:04:58.374 INFO:teuthology.orchestra.run.smithi134.stdout: "dispatch_server_request": 15, 2024-09-10T10:04:58.374 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_request": 3535, 2024-09-10T10:04:58.374 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_session": 7, 2024-09-10T10:04:58.374 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_peer_request": 1627, 2024-09-10T10:04:58.375 INFO:teuthology.orchestra.run.smithi134.stdout: "req_create_latency": { 2024-09-10T10:04:58.375 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 261, 2024-09-10T10:04:58.375 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.334439680, 2024-09-10T10:04:58.375 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.001281378 2024-09-10T10:04:58.375 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:04:58.375 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getattr_latency": { 2024-09-10T10:04:58.375 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:04:58.375 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:04:58.375 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:04:58.375 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:04:58.375 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getfilelock_latency": { 2024-09-10T10:04:58.375 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:04:58.375 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:04:58.375 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:04:58.375 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:04:58.376 INFO:teuthology.orchestra.run.smithi134.stdout: "req_link_latency": { 2024-09-10T10:04:58.376 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:04:58.376 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:04:58.376 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:04:58.376 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:04:58.376 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookup_latency": { 2024-09-10T10:04:58.376 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 1867, 2024-09-10T10:04:58.376 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.509654389, 2024-09-10T10:04:58.376 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000272980 2024-09-10T10:04:58.376 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:04:58.376 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookuphash_latency": { 2024-09-10T10:04:58.376 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:04:58.376 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:04:58.376 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:04:58.377 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:04:58.377 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupino_latency": { 2024-09-10T10:04:58.377 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:04:58.377 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:04:58.377 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:04:58.377 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:04:58.377 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupname_latency": { 2024-09-10T10:04:58.377 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:04:58.377 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:04:58.377 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:04:58.377 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:04:58.377 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupparent_latency": { 2024-09-10T10:04:58.378 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:04:58.378 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:04:58.378 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:04:58.378 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:04:58.378 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupsnap_latency": { 2024-09-10T10:04:58.378 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:04:58.378 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:04:58.378 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:04:58.378 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:04:58.378 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lssnap_latency": { 2024-09-10T10:04:58.378 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:04:58.379 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:04:58.379 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:04:58.379 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:04:58.379 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mkdir_latency": { 2024-09-10T10:04:58.379 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:04:58.379 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:04:58.379 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:04:58.379 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:04:58.379 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mknod_latency": { 2024-09-10T10:04:58.379 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:04:58.380 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:04:58.380 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:04:58.380 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:04:58.380 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mksnap_latency": { 2024-09-10T10:04:58.380 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:04:58.380 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:04:58.380 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:04:58.380 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:04:58.380 INFO:teuthology.orchestra.run.smithi134.stdout: "req_open_latency": { 2024-09-10T10:04:58.380 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 3, 2024-09-10T10:04:58.381 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000979173, 2024-09-10T10:04:58.381 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000326391 2024-09-10T10:04:58.381 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:04:58.381 INFO:teuthology.orchestra.run.smithi134.stdout: "req_readdir_latency": { 2024-09-10T10:04:58.381 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 73, 2024-09-10T10:04:58.381 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.070183854, 2024-09-10T10:04:58.381 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000961422 2024-09-10T10:04:58.381 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:04:58.381 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rename_latency": { 2024-09-10T10:04:58.381 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:04:58.381 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:04:58.381 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:04:58.382 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:04:58.382 INFO:teuthology.orchestra.run.smithi134.stdout: "req_renamesnap_latency": { 2024-09-10T10:04:58.382 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:04:58.382 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:04:58.382 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:04:58.382 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:04:58.382 INFO:teuthology.orchestra.run.smithi134.stdout: "req_snapdiff_latency": { 2024-09-10T10:04:58.382 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:04:58.382 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:04:58.382 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:04:58.382 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:04:58.382 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmdir_latency": { 2024-09-10T10:04:58.382 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 3, 2024-09-10T10:04:58.382 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.025517902, 2024-09-10T10:04:58.382 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.008505967 2024-09-10T10:04:58.383 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:04:58.383 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmsnap_latency": { 2024-09-10T10:04:58.383 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:04:58.383 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:04:58.383 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:04:58.383 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:04:58.383 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmxattr_latency": { 2024-09-10T10:04:58.383 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:04:58.383 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:04:58.383 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:04:58.383 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:04:58.383 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setattr_latency": { 2024-09-10T10:04:58.383 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:04:58.383 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:04:58.384 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:04:58.384 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:04:58.384 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setdirlayout_latency": { 2024-09-10T10:04:58.384 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:04:58.384 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:04:58.384 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:04:58.384 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:04:58.384 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setfilelock_latency": { 2024-09-10T10:04:58.384 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:04:58.384 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:04:58.384 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:04:58.384 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:04:58.384 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setlayout_latency": { 2024-09-10T10:04:58.384 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:04:58.385 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:04:58.385 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:04:58.385 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:04:58.385 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setxattr_latency": { 2024-09-10T10:04:58.385 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:04:58.385 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:04:58.385 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:04:58.385 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:04:58.385 INFO:teuthology.orchestra.run.smithi134.stdout: "req_symlink_latency": { 2024-09-10T10:04:58.385 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:04:58.385 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:04:58.385 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:04:58.385 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:04:58.386 INFO:teuthology.orchestra.run.smithi134.stdout: "req_unlink_latency": { 2024-09-10T10:04:58.386 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 1324, 2024-09-10T10:04:58.386 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 1.816738468, 2024-09-10T10:04:58.386 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.001372158 2024-09-10T10:04:58.386 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:04:58.386 INFO:teuthology.orchestra.run.smithi134.stdout: "cap_revoke_eviction": 0, 2024-09-10T10:04:58.386 INFO:teuthology.orchestra.run.smithi134.stdout: "cap_acquisition_throttle": 0, 2024-09-10T10:04:58.386 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getvxattr_latency": { 2024-09-10T10:04:58.386 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:04:58.386 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:04:58.386 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:04:58.386 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:04:58.386 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:04:58.386 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_sessions": { 2024-09-10T10:04:58.387 INFO:teuthology.orchestra.run.smithi134.stdout: "session_count": 1, 2024-09-10T10:04:58.387 INFO:teuthology.orchestra.run.smithi134.stdout: "session_add": 1, 2024-09-10T10:04:58.387 INFO:teuthology.orchestra.run.smithi134.stdout: "session_remove": 0, 2024-09-10T10:04:58.387 INFO:teuthology.orchestra.run.smithi134.stdout: "sessions_open": 1, 2024-09-10T10:04:58.387 INFO:teuthology.orchestra.run.smithi134.stdout: "sessions_stale": 0, 2024-09-10T10:04:58.387 INFO:teuthology.orchestra.run.smithi134.stdout: "total_load": 2065, 2024-09-10T10:04:58.387 INFO:teuthology.orchestra.run.smithi134.stdout: "average_load": 2065, 2024-09-10T10:04:58.387 INFO:teuthology.orchestra.run.smithi134.stdout: "avg_session_uptime": 1951, 2024-09-10T10:04:58.387 INFO:teuthology.orchestra.run.smithi134.stdout: "mdthresh_evicted": 0 2024-09-10T10:04:58.387 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:04:58.387 INFO:teuthology.orchestra.run.smithi134.stdout: "mempool": { 2024-09-10T10:04:58.387 INFO:teuthology.orchestra.run.smithi134.stdout: "bloom_filter_bytes": 0, 2024-09-10T10:04:58.388 INFO:teuthology.orchestra.run.smithi134.stdout: "bloom_filter_items": 0, 2024-09-10T10:04:58.388 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T10:04:58.388 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_alloc_items": 0, 2024-09-10T10:04:58.388 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T10:04:58.388 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_data_items": 0, 2024-09-10T10:04:58.388 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T10:04:58.388 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T10:04:58.388 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T10:04:58.388 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T10:04:58.388 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T10:04:58.388 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_other_items": 0, 2024-09-10T10:04:58.388 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T10:04:58.388 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T10:04:58.389 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_extent_bytes": 0, 2024-09-10T10:04:58.389 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_extent_items": 0, 2024-09-10T10:04:58.389 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_blob_bytes": 0, 2024-09-10T10:04:58.389 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_blob_items": 0, 2024-09-10T10:04:58.389 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T10:04:58.389 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T10:04:58.389 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T10:04:58.389 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T10:04:58.389 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T10:04:58.389 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_fsck_items": 0, 2024-09-10T10:04:58.389 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_txc_bytes": 0, 2024-09-10T10:04:58.389 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_txc_items": 0, 2024-09-10T10:04:58.389 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T10:04:58.389 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T10:04:58.389 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_bytes": 0, 2024-09-10T10:04:58.390 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_items": 0, 2024-09-10T10:04:58.390 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_bytes": 0, 2024-09-10T10:04:58.390 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_items": 0, 2024-09-10T10:04:58.390 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T10:04:58.390 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_reader_items": 0, 2024-09-10T10:04:58.390 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T10:04:58.390 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_writer_items": 0, 2024-09-10T10:04:58.390 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_anon_bytes": 311525, 2024-09-10T10:04:58.390 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_anon_items": 109, 2024-09-10T10:04:58.390 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_meta_bytes": 0, 2024-09-10T10:04:58.392 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_meta_items": 0, 2024-09-10T10:04:58.392 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_bytes": 0, 2024-09-10T10:04:58.392 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_items": 0, 2024-09-10T10:04:58.392 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_mapbl_bytes": 0, 2024-09-10T10:04:58.392 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_mapbl_items": 0, 2024-09-10T10:04:58.393 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_pglog_bytes": 0, 2024-09-10T10:04:58.393 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_pglog_items": 0, 2024-09-10T10:04:58.393 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_bytes": 9144, 2024-09-10T10:04:58.393 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_items": 172, 2024-09-10T10:04:58.393 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T10:04:58.393 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_mapping_items": 0, 2024-09-10T10:04:58.393 INFO:teuthology.orchestra.run.smithi134.stdout: "pgmap_bytes": 0, 2024-09-10T10:04:58.393 INFO:teuthology.orchestra.run.smithi134.stdout: "pgmap_items": 0, 2024-09-10T10:04:58.393 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_co_bytes": 105021, 2024-09-10T10:04:58.393 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_co_items": 416, 2024-09-10T10:04:58.393 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_1_bytes": 0, 2024-09-10T10:04:58.393 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_1_items": 0, 2024-09-10T10:04:58.393 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_2_bytes": 0, 2024-09-10T10:04:58.394 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_2_items": 0 2024-09-10T10:04:58.394 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:04:58.394 INFO:teuthology.orchestra.run.smithi134.stdout: "objecter": { 2024-09-10T10:04:58.394 INFO:teuthology.orchestra.run.smithi134.stdout: "op_active": 0, 2024-09-10T10:04:58.394 INFO:teuthology.orchestra.run.smithi134.stdout: "op_laggy": 0, 2024-09-10T10:04:58.394 INFO:teuthology.orchestra.run.smithi134.stdout: "op_send": 11415, 2024-09-10T10:04:58.394 INFO:teuthology.orchestra.run.smithi134.stdout: "op_send_bytes": 7813045, 2024-09-10T10:04:58.394 INFO:teuthology.orchestra.run.smithi134.stdout: "op_resend": 33, 2024-09-10T10:04:58.394 INFO:teuthology.orchestra.run.smithi134.stdout: "op_reply": 11415, 2024-09-10T10:04:58.394 INFO:teuthology.orchestra.run.smithi134.stdout: "op_latency": { 2024-09-10T10:04:58.394 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 11415, 2024-09-10T10:04:58.394 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 26.324361111, 2024-09-10T10:04:58.394 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.002306120 2024-09-10T10:04:58.394 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:04:58.394 INFO:teuthology.orchestra.run.smithi134.stdout: "op_inflight": 0, 2024-09-10T10:04:58.395 INFO:teuthology.orchestra.run.smithi134.stdout: "oplen_avg": { 2024-09-10T10:04:58.395 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 11415, 2024-09-10T10:04:58.395 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 11496 2024-09-10T10:04:58.395 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:04:58.395 INFO:teuthology.orchestra.run.smithi134.stdout: "op": 11415, 2024-09-10T10:04:58.395 INFO:teuthology.orchestra.run.smithi134.stdout: "op_r": 8429, 2024-09-10T10:04:58.395 INFO:teuthology.orchestra.run.smithi134.stdout: "op_w": 2986, 2024-09-10T10:04:58.395 INFO:teuthology.orchestra.run.smithi134.stdout: "op_rmw": 0, 2024-09-10T10:04:58.395 INFO:teuthology.orchestra.run.smithi134.stdout: "op_pg": 0, 2024-09-10T10:04:58.395 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_stat": 0, 2024-09-10T10:04:58.395 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_create": 10, 2024-09-10T10:04:58.395 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_read": 30, 2024-09-10T10:04:58.395 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_write": 221, 2024-09-10T10:04:58.395 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_writefull": 28, 2024-09-10T10:04:58.395 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_writesame": 0, 2024-09-10T10:04:58.396 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_append": 0, 2024-09-10T10:04:58.396 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_zero": 0, 2024-09-10T10:04:58.396 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_truncate": 0, 2024-09-10T10:04:58.396 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_delete": 2664, 2024-09-10T10:04:58.396 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_mapext": 0, 2024-09-10T10:04:58.396 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_sparse_read": 0, 2024-09-10T10:04:58.396 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_clonerange": 0, 2024-09-10T10:04:58.396 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_getxattr": 8399, 2024-09-10T10:04:58.396 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_setxattr": 20, 2024-09-10T10:04:58.396 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_cmpxattr": 0, 2024-09-10T10:04:58.396 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_rmxattr": 0, 2024-09-10T10:04:58.396 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_resetxattrs": 0, 2024-09-10T10:04:58.396 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_call": 0, 2024-09-10T10:04:58.396 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_watch": 0, 2024-09-10T10:04:58.397 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_notify": 0, 2024-09-10T10:04:58.397 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T10:04:58.397 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_pgls": 0, 2024-09-10T10:04:58.397 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_pgls_filter": 0, 2024-09-10T10:04:58.397 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_other": 0, 2024-09-10T10:04:58.397 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_active": 0, 2024-09-10T10:04:58.397 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_send": 0, 2024-09-10T10:04:58.397 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_resend": 0, 2024-09-10T10:04:58.397 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_ping": 0, 2024-09-10T10:04:58.397 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_active": 0, 2024-09-10T10:04:58.397 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_send": 0, 2024-09-10T10:04:58.397 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_resend": 0, 2024-09-10T10:04:58.397 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_active": 0, 2024-09-10T10:04:58.397 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_send": 0, 2024-09-10T10:04:58.398 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_resend": 0, 2024-09-10T10:04:58.398 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_active": 0, 2024-09-10T10:04:58.398 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_send": 0, 2024-09-10T10:04:58.398 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_resend": 0, 2024-09-10T10:04:58.398 INFO:teuthology.orchestra.run.smithi134.stdout: "command_active": 0, 2024-09-10T10:04:58.398 INFO:teuthology.orchestra.run.smithi134.stdout: "command_send": 0, 2024-09-10T10:04:58.398 INFO:teuthology.orchestra.run.smithi134.stdout: "command_resend": 0, 2024-09-10T10:04:58.398 INFO:teuthology.orchestra.run.smithi134.stdout: "map_epoch": 96, 2024-09-10T10:04:58.398 INFO:teuthology.orchestra.run.smithi134.stdout: "map_full": 0, 2024-09-10T10:04:58.398 INFO:teuthology.orchestra.run.smithi134.stdout: "map_inc": 2, 2024-09-10T10:04:58.398 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_sessions": 12, 2024-09-10T10:04:58.398 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_session_open": 12, 2024-09-10T10:04:58.398 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_session_close": 0, 2024-09-10T10:04:58.398 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_laggy": 0, 2024-09-10T10:04:58.398 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_wr": 98, 2024-09-10T10:04:58.399 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_rd": 0, 2024-09-10T10:04:58.399 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_del": 26 2024-09-10T10:04:58.399 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:04:58.399 INFO:teuthology.orchestra.run.smithi134.stdout: "oft": { 2024-09-10T10:04:58.399 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_objs": 1, 2024-09-10T10:04:58.399 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_kv_pairs": 0, 2024-09-10T10:04:58.399 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_updates": 0, 2024-09-10T10:04:58.399 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_removes": 0 2024-09-10T10:04:58.399 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:04:58.399 INFO:teuthology.orchestra.run.smithi134.stdout: "osd-slow-ops": { 2024-09-10T10:04:58.399 INFO:teuthology.orchestra.run.smithi134.stdout: "slow_ops_count": 5594 2024-09-10T10:04:58.399 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:04:58.399 INFO:teuthology.orchestra.run.smithi134.stdout: "purge_queue": { 2024-09-10T10:04:58.399 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_ops": 0, 2024-09-10T10:04:58.399 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_ops_high_water": 10, 2024-09-10T10:04:58.399 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing": 0, 2024-09-10T10:04:58.400 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_high_water": 6, 2024-09-10T10:04:58.400 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executed_ops": 2651, 2024-09-10T10:04:58.400 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executed": 1327, 2024-09-10T10:04:58.400 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_item_in_journal": 0 2024-09-10T10:04:58.400 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:04:58.400 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T10:04:58.400 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-10T10:04:58.400 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 104857600, 2024-09-10T10:04:58.400 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-10T10:04:58.400 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 22496, 2024-09-10T10:04:58.400 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 6950954, 2024-09-10T10:04:58.400 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-10T10:04:58.400 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 22496, 2024-09-10T10:04:58.400 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-09-10T10:04:58.401 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-09-10T10:04:58.401 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 22496, 2024-09-10T10:04:58.401 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 6950954, 2024-09-10T10:04:58.401 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-10T10:04:58.401 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:04:58.401 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:04:58.401 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:04:58.401 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:04:58.401 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:04:58.401 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-objecter_bytes": { 2024-09-10T10:04:58.401 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-10T10:04:58.401 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 104857600, 2024-09-10T10:04:58.401 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-10T10:04:58.401 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-09-10T10:04:58.401 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-09-10T10:04:58.402 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-10T10:04:58.402 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-09-10T10:04:58.402 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 11415, 2024-09-10T10:04:58.402 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 7957664, 2024-09-10T10:04:58.402 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 11415, 2024-09-10T10:04:58.402 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 7957664, 2024-09-10T10:04:58.402 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-10T10:04:58.402 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:04:58.402 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:04:58.402 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:04:58.402 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:04:58.402 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:04:58.402 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-objecter_ops": { 2024-09-10T10:04:58.402 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-10T10:04:58.402 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 1024, 2024-09-10T10:04:58.403 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-10T10:04:58.403 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-09-10T10:04:58.403 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-09-10T10:04:58.403 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-10T10:04:58.403 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-09-10T10:04:58.403 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 11415, 2024-09-10T10:04:58.403 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 11415, 2024-09-10T10:04:58.403 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 11415, 2024-09-10T10:04:58.403 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 11415, 2024-09-10T10:04:58.403 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-10T10:04:58.403 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:04:58.403 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:04:58.403 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:04:58.403 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:04:58.404 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:04:58.404 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-write_buf_throttle": { 2024-09-10T10:04:58.404 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-10T10:04:58.404 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 3758096384, 2024-09-10T10:04:58.404 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-10T10:04:58.404 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 1327, 2024-09-10T10:04:58.404 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 144619, 2024-09-10T10:04:58.405 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-10T10:04:58.405 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 1327, 2024-09-10T10:04:58.405 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-09-10T10:04:58.405 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-09-10T10:04:58.405 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 30, 2024-09-10T10:04:58.405 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 144619, 2024-09-10T10:04:58.405 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-10T10:04:58.405 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:04:58.405 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:04:58.405 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:04:58.405 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:04:58.405 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:04:58.405 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-write_buf_throttle-0x55dab46d4960": { 2024-09-10T10:04:58.406 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-10T10:04:58.406 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 3758096384, 2024-09-10T10:04:58.406 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-10T10:04:58.406 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 4674, 2024-09-10T10:04:58.406 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 7497617, 2024-09-10T10:04:58.406 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-10T10:04:58.406 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 4674, 2024-09-10T10:04:58.406 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-09-10T10:04:58.406 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-09-10T10:04:58.406 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 191, 2024-09-10T10:04:58.406 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 7497617, 2024-09-10T10:04:58.406 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-10T10:04:58.406 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:04:58.407 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:04:58.407 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:04:58.407 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:04:58.407 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:04:58.407 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:04:59.273 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:04:58 smithi134 ceph-mon[35674]: pgmap v685: 129 pgs: 129 active+clean; 115 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T10:04:59.299 INFO:tasks.check_counter:Daemon mds.c mds.exported=37 2024-09-10T10:04:59.299 INFO:tasks.check_counter:Daemon mds.c mds.imported=37 2024-09-10T10:04:59.299 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=0 2024-09-10T10:04:59.299 DEBUG:tasks.check_counter:Getting stats from f 2024-09-10T10:04:59.300 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph daemon mds.f perf dump 2024-09-10T10:04:59.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:04:58 smithi120 ceph-mon[34770]: pgmap v685: 129 pgs: 129 active+clean; 115 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T10:04:59.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:04:58 smithi077 ceph-mon[30319]: pgmap v685: 129 pgs: 129 active+clean; 115 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T10:04:59.546 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T10:05:00.086 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:05:00.086 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T10:05:00.086 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 5166, 2024-09-10T10:05:00.086 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 5227, 2024-09-10T10:05:00.086 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 1531701, 2024-09-10T10:05:00.087 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 3980527, 2024-09-10T10:05:00.087 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 11, 2024-09-10T10:05:00.087 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 6, 2024-09-10T10:05:00.087 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.417890902, 2024-09-10T10:05:00.087 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.159863703, 2024-09-10T10:05:00.087 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.097013605, 2024-09-10T10:05:00.087 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.121158672, 2024-09-10T10:05:00.087 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:05:00.087 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 5223, 2024-09-10T10:05:00.087 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.410343892, 2024-09-10T10:05:00.087 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000078564 2024-09-10T10:05:00.087 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:00.088 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:05:00.088 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 842, 2024-09-10T10:05:00.088 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000147191, 2024-09-10T10:05:00.088 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000174 2024-09-10T10:05:00.088 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:00.088 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-10T10:05:00.088 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 0 2024-09-10T10:05:00.088 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:00.088 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T10:05:00.088 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 6183, 2024-09-10T10:05:00.088 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 7910, 2024-09-10T10:05:00.088 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 3508182, 2024-09-10T10:05:00.088 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 2963273, 2024-09-10T10:05:00.089 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 9, 2024-09-10T10:05:00.089 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 5, 2024-09-10T10:05:00.089 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.491271532, 2024-09-10T10:05:00.089 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.210324652, 2024-09-10T10:05:00.089 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.231756568, 2024-09-10T10:05:00.089 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.061677687, 2024-09-10T10:05:00.089 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:05:00.089 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 7912, 2024-09-10T10:05:00.089 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.595638020, 2024-09-10T10:05:00.089 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000075282 2024-09-10T10:05:00.089 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:00.089 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:05:00.089 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 9258, 2024-09-10T10:05:00.090 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.001817417, 2024-09-10T10:05:00.090 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000196 2024-09-10T10:05:00.090 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:00.090 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-10T10:05:00.090 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 344496 2024-09-10T10:05:00.090 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:00.090 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T10:05:00.090 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 8754, 2024-09-10T10:05:00.090 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 12068, 2024-09-10T10:05:00.090 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 3034790, 2024-09-10T10:05:00.090 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 10702074, 2024-09-10T10:05:00.090 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 40, 2024-09-10T10:05:00.090 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 6, 2024-09-10T10:05:00.090 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.667917034, 2024-09-10T10:05:00.091 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.289408676, 2024-09-10T10:05:00.091 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.265252366, 2024-09-10T10:05:00.091 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.124028106, 2024-09-10T10:05:00.091 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:05:00.091 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 12066, 2024-09-10T10:05:00.091 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.952893963, 2024-09-10T10:05:00.091 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000078973 2024-09-10T10:05:00.091 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:00.091 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:05:00.091 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 4972, 2024-09-10T10:05:00.091 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.001480116, 2024-09-10T10:05:00.091 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000297 2024-09-10T10:05:00.091 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:00.091 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 594592, 2024-09-10T10:05:00.092 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 145024 2024-09-10T10:05:00.092 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:00.092 INFO:teuthology.orchestra.run.smithi134.stdout: "cct": { 2024-09-10T10:05:00.092 INFO:teuthology.orchestra.run.smithi134.stdout: "total_workers": 1, 2024-09-10T10:05:00.092 INFO:teuthology.orchestra.run.smithi134.stdout: "unhealthy_workers": 0 2024-09-10T10:05:00.092 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:00.092 INFO:teuthology.orchestra.run.smithi134.stdout: "finisher-MDSRank": { 2024-09-10T10:05:00.092 INFO:teuthology.orchestra.run.smithi134.stdout: "queue_len": 0, 2024-09-10T10:05:00.092 INFO:teuthology.orchestra.run.smithi134.stdout: "complete_latency": { 2024-09-10T10:05:00.092 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 6122, 2024-09-10T10:05:00.092 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 2.692262023, 2024-09-10T10:05:00.092 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000439768 2024-09-10T10:05:00.092 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:05:00.092 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:00.093 INFO:teuthology.orchestra.run.smithi134.stdout: "finisher-PurgeQueue": { 2024-09-10T10:05:00.093 INFO:teuthology.orchestra.run.smithi134.stdout: "queue_len": 0, 2024-09-10T10:05:00.093 INFO:teuthology.orchestra.run.smithi134.stdout: "complete_latency": { 2024-09-10T10:05:00.093 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 2155, 2024-09-10T10:05:00.093 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.300840062, 2024-09-10T10:05:00.093 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000139600 2024-09-10T10:05:00.093 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:05:00.093 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:00.093 INFO:teuthology.orchestra.run.smithi134.stdout: "mds": { 2024-09-10T10:05:00.093 INFO:teuthology.orchestra.run.smithi134.stdout: "request": 2651, 2024-09-10T10:05:00.093 INFO:teuthology.orchestra.run.smithi134.stdout: "reply": 2650, 2024-09-10T10:05:00.093 INFO:teuthology.orchestra.run.smithi134.stdout: "reply_latency": { 2024-09-10T10:05:00.094 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 2650, 2024-09-10T10:05:00.094 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 2.796193406, 2024-09-10T10:05:00.094 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.001055167 2024-09-10T10:05:00.094 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:00.094 INFO:teuthology.orchestra.run.smithi134.stdout: "slow_reply": 0, 2024-09-10T10:05:00.094 INFO:teuthology.orchestra.run.smithi134.stdout: "forward": 1, 2024-09-10T10:05:00.094 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_fetch_complete": 0, 2024-09-10T10:05:00.094 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_fetch_keys": 0, 2024-09-10T10:05:00.094 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_commit": 11, 2024-09-10T10:05:00.094 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_split": 0, 2024-09-10T10:05:00.094 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_merge": 0, 2024-09-10T10:05:00.094 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes": 18, 2024-09-10T10:05:00.094 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_top": 1, 2024-09-10T10:05:00.095 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_bottom": 3, 2024-09-10T10:05:00.095 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_pin_tail": 14, 2024-09-10T10:05:00.095 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_pinned": 14, 2024-09-10T10:05:00.095 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_expired": 322, 2024-09-10T10:05:00.095 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_with_caps": 1, 2024-09-10T10:05:00.095 INFO:teuthology.orchestra.run.smithi134.stdout: "caps": 1, 2024-09-10T10:05:00.095 INFO:teuthology.orchestra.run.smithi134.stdout: "subtrees": 5, 2024-09-10T10:05:00.095 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse": 5325, 2024-09-10T10:05:00.095 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_hit": 3578, 2024-09-10T10:05:00.095 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_forward": 0, 2024-09-10T10:05:00.095 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_discover": 41, 2024-09-10T10:05:00.095 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_dir_fetch": 0, 2024-09-10T10:05:00.095 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_remote_ino": 0, 2024-09-10T10:05:00.095 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_lock": 0, 2024-09-10T10:05:00.096 INFO:teuthology.orchestra.run.smithi134.stdout: "load_cent": 0, 2024-09-10T10:05:00.096 INFO:teuthology.orchestra.run.smithi134.stdout: "q": 0, 2024-09-10T10:05:00.096 INFO:teuthology.orchestra.run.smithi134.stdout: "exported": 50, 2024-09-10T10:05:00.096 INFO:teuthology.orchestra.run.smithi134.stdout: "exported_inodes": 304, 2024-09-10T10:05:00.096 INFO:teuthology.orchestra.run.smithi134.stdout: "imported": 50, 2024-09-10T10:05:00.096 INFO:teuthology.orchestra.run.smithi134.stdout: "imported_inodes": 1092, 2024-09-10T10:05:00.096 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_dir_fetch": 0, 2024-09-10T10:05:00.096 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_backtrace_fetch": 8741, 2024-09-10T10:05:00.096 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_peer_discover": 0, 2024-09-10T10:05:00.096 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rfiles": 4215, 2024-09-10T10:05:00.096 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rbytes": 81947162, 2024-09-10T10:05:00.096 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rsnaps": 0, 2024-09-10T10:05:00.096 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_backtrace_fetch": 8741, 2024-09-10T10:05:00.096 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_set_tag": 0, 2024-09-10T10:05:00.097 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T10:05:00.097 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_inotable_repaired": 0, 2024-09-10T10:05:00.097 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dir_inodes": 23, 2024-09-10T10:05:00.097 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dir_base_inodes": 0, 2024-09-10T10:05:00.097 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dirfrag_rstats": 23, 2024-09-10T10:05:00.097 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_file_inodes": 8718, 2024-09-10T10:05:00.097 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_inode_file_caps": 0, 2024-09-10T10:05:00.097 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_revoke": 151, 2024-09-10T10:05:00.097 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_grant": 1355, 2024-09-10T10:05:00.097 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T10:05:00.097 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T10:05:00.097 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T10:05:00.097 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_caps": 1207, 2024-09-10T10:05:00.097 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_caps_dirty": 776, 2024-09-10T10:05:00.098 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_cap_release": 35, 2024-09-10T10:05:00.098 INFO:teuthology.orchestra.run.smithi134.stdout: "process_request_cap_release": 2411 2024-09-10T10:05:00.098 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:00.098 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_cache": { 2024-09-10T10:05:00.098 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_update": 0, 2024-09-10T10:05:00.098 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_update_receipt": 0, 2024-09-10T10:05:00.098 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_try_discover": 0, 2024-09-10T10:05:00.098 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_send_discover": 14, 2024-09-10T10:05:00.098 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_handle_discover": 50, 2024-09-10T10:05:00.098 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays": 0, 2024-09-10T10:05:00.098 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays_delayed": 0, 2024-09-10T10:05:00.098 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays_enqueuing": 0, 2024-09-10T10:05:00.098 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_created": 1145, 2024-09-10T10:05:00.098 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_enqueued": 1145, 2024-09-10T10:05:00.098 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_reintegrated": 0, 2024-09-10T10:05:00.099 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_migrated": 0, 2024-09-10T10:05:00.099 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_processing": 0, 2024-09-10T10:05:00.099 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_enqueued": 0, 2024-09-10T10:05:00.099 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_prioritized": 0, 2024-09-10T10:05:00.099 INFO:teuthology.orchestra.run.smithi134.stdout: "recovery_started": 0, 2024-09-10T10:05:00.099 INFO:teuthology.orchestra.run.smithi134.stdout: "recovery_completed": 0, 2024-09-10T10:05:00.099 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_quiesce_path": 8, 2024-09-10T10:05:00.099 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_quiesce_inode": 5806, 2024-09-10T10:05:00.099 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_enqueue_scrub": 0, 2024-09-10T10:05:00.099 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_exportdir": 50, 2024-09-10T10:05:00.099 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_flush": 0, 2024-09-10T10:05:00.099 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_fragmentdir": 0, 2024-09-10T10:05:00.099 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_fragstats": 0, 2024-09-10T10:05:00.099 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_inodestats": 0 2024-09-10T10:05:00.100 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:00.100 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_log": { 2024-09-10T10:05:00.100 INFO:teuthology.orchestra.run.smithi134.stdout: "evlrg": 0, 2024-09-10T10:05:00.100 INFO:teuthology.orchestra.run.smithi134.stdout: "evadd": 3931, 2024-09-10T10:05:00.100 INFO:teuthology.orchestra.run.smithi134.stdout: "evex": 0, 2024-09-10T10:05:00.100 INFO:teuthology.orchestra.run.smithi134.stdout: "evtrm": 0, 2024-09-10T10:05:00.100 INFO:teuthology.orchestra.run.smithi134.stdout: "ev": 3931, 2024-09-10T10:05:00.100 INFO:teuthology.orchestra.run.smithi134.stdout: "evexg": 0, 2024-09-10T10:05:00.100 INFO:teuthology.orchestra.run.smithi134.stdout: "evexd": 0, 2024-09-10T10:05:00.100 INFO:teuthology.orchestra.run.smithi134.stdout: "segadd": 6, 2024-09-10T10:05:00.100 INFO:teuthology.orchestra.run.smithi134.stdout: "segex": 0, 2024-09-10T10:05:00.100 INFO:teuthology.orchestra.run.smithi134.stdout: "segtrm": 0, 2024-09-10T10:05:00.100 INFO:teuthology.orchestra.run.smithi134.stdout: "seg": 6, 2024-09-10T10:05:00.100 INFO:teuthology.orchestra.run.smithi134.stdout: "segmjr": 2, 2024-09-10T10:05:00.101 INFO:teuthology.orchestra.run.smithi134.stdout: "segexg": 0, 2024-09-10T10:05:00.101 INFO:teuthology.orchestra.run.smithi134.stdout: "segexd": 0, 2024-09-10T10:05:00.101 INFO:teuthology.orchestra.run.smithi134.stdout: "expos": 4194304, 2024-09-10T10:05:00.101 INFO:teuthology.orchestra.run.smithi134.stdout: "wrpos": 10484084, 2024-09-10T10:05:00.101 INFO:teuthology.orchestra.run.smithi134.stdout: "rdpos": 0, 2024-09-10T10:05:00.101 INFO:teuthology.orchestra.run.smithi134.stdout: "jlat": { 2024-09-10T10:05:00.101 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 213, 2024-09-10T10:05:00.101 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 4.001561878, 2024-09-10T10:05:00.101 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.018786675 2024-09-10T10:05:00.101 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:00.101 INFO:teuthology.orchestra.run.smithi134.stdout: "replayed": 0 2024-09-10T10:05:00.101 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:00.101 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_mem": { 2024-09-10T10:05:00.101 INFO:teuthology.orchestra.run.smithi134.stdout: "ino": 24, 2024-09-10T10:05:00.101 INFO:teuthology.orchestra.run.smithi134.stdout: "ino+": 1179, 2024-09-10T10:05:00.102 INFO:teuthology.orchestra.run.smithi134.stdout: "ino-": 1155, 2024-09-10T10:05:00.102 INFO:teuthology.orchestra.run.smithi134.stdout: "dir": 23, 2024-09-10T10:05:00.102 INFO:teuthology.orchestra.run.smithi134.stdout: "dir+": 85, 2024-09-10T10:05:00.102 INFO:teuthology.orchestra.run.smithi134.stdout: "dir-": 62, 2024-09-10T10:05:00.102 INFO:teuthology.orchestra.run.smithi134.stdout: "dn": 18, 2024-09-10T10:05:00.102 INFO:teuthology.orchestra.run.smithi134.stdout: "dn+": 2437, 2024-09-10T10:05:00.102 INFO:teuthology.orchestra.run.smithi134.stdout: "dn-": 2419, 2024-09-10T10:05:00.102 INFO:teuthology.orchestra.run.smithi134.stdout: "cap": 1, 2024-09-10T10:05:00.102 INFO:teuthology.orchestra.run.smithi134.stdout: "cap+": 2292, 2024-09-10T10:05:00.102 INFO:teuthology.orchestra.run.smithi134.stdout: "cap-": 2291, 2024-09-10T10:05:00.102 INFO:teuthology.orchestra.run.smithi134.stdout: "rss": 68748, 2024-09-10T10:05:00.102 INFO:teuthology.orchestra.run.smithi134.stdout: "heap": 223520 2024-09-10T10:05:00.102 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:00.102 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_server": { 2024-09-10T10:05:00.103 INFO:teuthology.orchestra.run.smithi134.stdout: "dispatch_client_request": 4090, 2024-09-10T10:05:00.103 INFO:teuthology.orchestra.run.smithi134.stdout: "dispatch_server_request": 6, 2024-09-10T10:05:00.103 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_request": 2651, 2024-09-10T10:05:00.103 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_session": 8, 2024-09-10T10:05:00.103 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_peer_request": 1199, 2024-09-10T10:05:00.103 INFO:teuthology.orchestra.run.smithi134.stdout: "req_create_latency": { 2024-09-10T10:05:00.103 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 79, 2024-09-10T10:05:00.103 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.201655760, 2024-09-10T10:05:00.103 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.002552604 2024-09-10T10:05:00.103 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:00.103 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getattr_latency": { 2024-09-10T10:05:00.103 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 1, 2024-09-10T10:05:00.103 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.001023348, 2024-09-10T10:05:00.103 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.001023348 2024-09-10T10:05:00.104 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:00.104 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getfilelock_latency": { 2024-09-10T10:05:00.104 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:05:00.104 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:05:00.104 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:05:00.104 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:00.104 INFO:teuthology.orchestra.run.smithi134.stdout: "req_link_latency": { 2024-09-10T10:05:00.104 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:05:00.104 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:05:00.104 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:05:00.104 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:00.104 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookup_latency": { 2024-09-10T10:05:00.104 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 1296, 2024-09-10T10:05:00.104 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.439134886, 2024-09-10T10:05:00.105 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000338838 2024-09-10T10:05:00.105 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:00.105 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookuphash_latency": { 2024-09-10T10:05:00.105 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:05:00.105 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:05:00.105 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:05:00.105 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:00.105 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupino_latency": { 2024-09-10T10:05:00.105 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:05:00.105 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:05:00.105 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:05:00.105 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:00.105 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupname_latency": { 2024-09-10T10:05:00.105 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:05:00.105 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:05:00.106 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:05:00.106 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:00.106 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupparent_latency": { 2024-09-10T10:05:00.106 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:05:00.106 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:05:00.106 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:05:00.106 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:00.106 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupsnap_latency": { 2024-09-10T10:05:00.106 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:05:00.106 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:05:00.106 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:05:00.106 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:00.106 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lssnap_latency": { 2024-09-10T10:05:00.106 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:05:00.107 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:05:00.107 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:05:00.107 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:00.107 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mkdir_latency": { 2024-09-10T10:05:00.107 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 1, 2024-09-10T10:05:00.107 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000812648, 2024-09-10T10:05:00.107 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000812648 2024-09-10T10:05:00.107 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:00.107 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mknod_latency": { 2024-09-10T10:05:00.107 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:05:00.107 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:05:00.107 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:05:00.107 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:00.107 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mksnap_latency": { 2024-09-10T10:05:00.108 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:05:00.108 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:05:00.108 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:05:00.108 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:00.108 INFO:teuthology.orchestra.run.smithi134.stdout: "req_open_latency": { 2024-09-10T10:05:00.108 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 18, 2024-09-10T10:05:00.108 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.004321355, 2024-09-10T10:05:00.108 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000240075 2024-09-10T10:05:00.108 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:00.108 INFO:teuthology.orchestra.run.smithi134.stdout: "req_readdir_latency": { 2024-09-10T10:05:00.108 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 110, 2024-09-10T10:05:00.108 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.084599656, 2024-09-10T10:05:00.108 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000769087 2024-09-10T10:05:00.108 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:00.109 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rename_latency": { 2024-09-10T10:05:00.109 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:05:00.109 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:05:00.109 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:05:00.109 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:00.109 INFO:teuthology.orchestra.run.smithi134.stdout: "req_renamesnap_latency": { 2024-09-10T10:05:00.109 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:05:00.109 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:05:00.109 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:05:00.109 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:00.109 INFO:teuthology.orchestra.run.smithi134.stdout: "req_snapdiff_latency": { 2024-09-10T10:05:00.109 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:05:00.109 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:05:00.109 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:05:00.109 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:00.110 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmdir_latency": { 2024-09-10T10:05:00.110 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 3, 2024-09-10T10:05:00.110 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.076317999, 2024-09-10T10:05:00.110 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.025439333 2024-09-10T10:05:00.110 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:00.110 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmsnap_latency": { 2024-09-10T10:05:00.110 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:05:00.110 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:05:00.110 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:05:00.110 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:00.110 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmxattr_latency": { 2024-09-10T10:05:00.110 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:05:00.110 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:05:00.110 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:05:00.110 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:00.111 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setattr_latency": { 2024-09-10T10:05:00.111 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:05:00.111 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:05:00.111 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:05:00.111 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:00.111 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setdirlayout_latency": { 2024-09-10T10:05:00.111 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:05:00.111 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:05:00.111 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:05:00.111 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:00.111 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setfilelock_latency": { 2024-09-10T10:05:00.111 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:05:00.111 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:05:00.111 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:05:00.112 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:00.112 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setlayout_latency": { 2024-09-10T10:05:00.112 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:05:00.112 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:05:00.112 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:05:00.112 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:00.112 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setxattr_latency": { 2024-09-10T10:05:00.112 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:05:00.112 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:05:00.112 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:05:00.112 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:00.112 INFO:teuthology.orchestra.run.smithi134.stdout: "req_symlink_latency": { 2024-09-10T10:05:00.112 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:05:00.112 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:05:00.113 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:05:00.113 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:00.113 INFO:teuthology.orchestra.run.smithi134.stdout: "req_unlink_latency": { 2024-09-10T10:05:00.113 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 1142, 2024-09-10T10:05:00.113 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 1.988327754, 2024-09-10T10:05:00.113 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.001741092 2024-09-10T10:05:00.113 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:00.113 INFO:teuthology.orchestra.run.smithi134.stdout: "cap_revoke_eviction": 0, 2024-09-10T10:05:00.113 INFO:teuthology.orchestra.run.smithi134.stdout: "cap_acquisition_throttle": 0, 2024-09-10T10:05:00.113 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getvxattr_latency": { 2024-09-10T10:05:00.113 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:05:00.113 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:05:00.113 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:05:00.113 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:05:00.113 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:00.114 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_sessions": { 2024-09-10T10:05:00.114 INFO:teuthology.orchestra.run.smithi134.stdout: "session_count": 1, 2024-09-10T10:05:00.114 INFO:teuthology.orchestra.run.smithi134.stdout: "session_add": 1, 2024-09-10T10:05:00.114 INFO:teuthology.orchestra.run.smithi134.stdout: "session_remove": 0, 2024-09-10T10:05:00.114 INFO:teuthology.orchestra.run.smithi134.stdout: "sessions_open": 1, 2024-09-10T10:05:00.114 INFO:teuthology.orchestra.run.smithi134.stdout: "sessions_stale": 0, 2024-09-10T10:05:00.114 INFO:teuthology.orchestra.run.smithi134.stdout: "total_load": 2107, 2024-09-10T10:05:00.114 INFO:teuthology.orchestra.run.smithi134.stdout: "average_load": 2107, 2024-09-10T10:05:00.114 INFO:teuthology.orchestra.run.smithi134.stdout: "avg_session_uptime": 584, 2024-09-10T10:05:00.114 INFO:teuthology.orchestra.run.smithi134.stdout: "mdthresh_evicted": 0 2024-09-10T10:05:00.114 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:00.114 INFO:teuthology.orchestra.run.smithi134.stdout: "mempool": { 2024-09-10T10:05:00.114 INFO:teuthology.orchestra.run.smithi134.stdout: "bloom_filter_bytes": 0, 2024-09-10T10:05:00.114 INFO:teuthology.orchestra.run.smithi134.stdout: "bloom_filter_items": 0, 2024-09-10T10:05:00.115 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T10:05:00.115 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_alloc_items": 0, 2024-09-10T10:05:00.115 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T10:05:00.115 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_data_items": 0, 2024-09-10T10:05:00.115 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T10:05:00.115 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T10:05:00.115 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T10:05:00.115 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T10:05:00.115 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T10:05:00.115 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_other_items": 0, 2024-09-10T10:05:00.115 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T10:05:00.115 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T10:05:00.115 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_extent_bytes": 0, 2024-09-10T10:05:00.115 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_extent_items": 0, 2024-09-10T10:05:00.116 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_blob_bytes": 0, 2024-09-10T10:05:00.116 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_blob_items": 0, 2024-09-10T10:05:00.116 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T10:05:00.116 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T10:05:00.116 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T10:05:00.116 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T10:05:00.116 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T10:05:00.116 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_fsck_items": 0, 2024-09-10T10:05:00.116 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_txc_bytes": 0, 2024-09-10T10:05:00.116 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_txc_items": 0, 2024-09-10T10:05:00.116 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T10:05:00.116 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T10:05:00.116 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_bytes": 0, 2024-09-10T10:05:00.116 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_items": 0, 2024-09-10T10:05:00.116 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_bytes": 0, 2024-09-10T10:05:00.117 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_items": 0, 2024-09-10T10:05:00.117 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T10:05:00.117 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_reader_items": 0, 2024-09-10T10:05:00.117 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T10:05:00.117 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_writer_items": 0, 2024-09-10T10:05:00.117 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_anon_bytes": 295413, 2024-09-10T10:05:00.117 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_anon_items": 102, 2024-09-10T10:05:00.117 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_meta_bytes": 0, 2024-09-10T10:05:00.117 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_meta_items": 0, 2024-09-10T10:05:00.117 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_bytes": 0, 2024-09-10T10:05:00.117 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_items": 0, 2024-09-10T10:05:00.117 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_mapbl_bytes": 0, 2024-09-10T10:05:00.117 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_mapbl_items": 0, 2024-09-10T10:05:00.117 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_pglog_bytes": 0, 2024-09-10T10:05:00.118 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_pglog_items": 0, 2024-09-10T10:05:00.118 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_bytes": 9144, 2024-09-10T10:05:00.118 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_items": 172, 2024-09-10T10:05:00.118 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T10:05:00.118 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_mapping_items": 0, 2024-09-10T10:05:00.118 INFO:teuthology.orchestra.run.smithi134.stdout: "pgmap_bytes": 0, 2024-09-10T10:05:00.118 INFO:teuthology.orchestra.run.smithi134.stdout: "pgmap_items": 0, 2024-09-10T10:05:00.118 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_co_bytes": 97101, 2024-09-10T10:05:00.118 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_co_items": 404, 2024-09-10T10:05:00.118 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_1_bytes": 0, 2024-09-10T10:05:00.118 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_1_items": 0, 2024-09-10T10:05:00.118 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_2_bytes": 0, 2024-09-10T10:05:00.118 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_2_items": 0 2024-09-10T10:05:00.118 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:00.118 INFO:teuthology.orchestra.run.smithi134.stdout: "objecter": { 2024-09-10T10:05:00.119 INFO:teuthology.orchestra.run.smithi134.stdout: "op_active": 0, 2024-09-10T10:05:00.119 INFO:teuthology.orchestra.run.smithi134.stdout: "op_laggy": 0, 2024-09-10T10:05:00.119 INFO:teuthology.orchestra.run.smithi134.stdout: "op_send": 11391, 2024-09-10T10:05:00.119 INFO:teuthology.orchestra.run.smithi134.stdout: "op_send_bytes": 6545611, 2024-09-10T10:05:00.119 INFO:teuthology.orchestra.run.smithi134.stdout: "op_resend": 33, 2024-09-10T10:05:00.119 INFO:teuthology.orchestra.run.smithi134.stdout: "op_reply": 11391, 2024-09-10T10:05:00.119 INFO:teuthology.orchestra.run.smithi134.stdout: "op_latency": { 2024-09-10T10:05:00.119 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 11391, 2024-09-10T10:05:00.119 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 26.384359167, 2024-09-10T10:05:00.119 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.002316246 2024-09-10T10:05:00.119 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:00.119 INFO:teuthology.orchestra.run.smithi134.stdout: "op_inflight": 0, 2024-09-10T10:05:00.120 INFO:teuthology.orchestra.run.smithi134.stdout: "oplen_avg": { 2024-09-10T10:05:00.120 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 11391, 2024-09-10T10:05:00.120 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 11470 2024-09-10T10:05:00.120 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:00.120 INFO:teuthology.orchestra.run.smithi134.stdout: "op": 11391, 2024-09-10T10:05:00.120 INFO:teuthology.orchestra.run.smithi134.stdout: "op_r": 8761, 2024-09-10T10:05:00.120 INFO:teuthology.orchestra.run.smithi134.stdout: "op_w": 2630, 2024-09-10T10:05:00.120 INFO:teuthology.orchestra.run.smithi134.stdout: "op_rmw": 0, 2024-09-10T10:05:00.120 INFO:teuthology.orchestra.run.smithi134.stdout: "op_pg": 0, 2024-09-10T10:05:00.120 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_stat": 0, 2024-09-10T10:05:00.120 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_create": 10, 2024-09-10T10:05:00.120 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_read": 20, 2024-09-10T10:05:00.120 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_write": 233, 2024-09-10T10:05:00.120 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_writefull": 23, 2024-09-10T10:05:00.120 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_writesame": 0, 2024-09-10T10:05:00.121 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_append": 0, 2024-09-10T10:05:00.121 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_zero": 0, 2024-09-10T10:05:00.121 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_truncate": 0, 2024-09-10T10:05:00.121 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_delete": 2300, 2024-09-10T10:05:00.121 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_mapext": 0, 2024-09-10T10:05:00.121 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_sparse_read": 0, 2024-09-10T10:05:00.121 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_clonerange": 0, 2024-09-10T10:05:00.121 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_getxattr": 8741, 2024-09-10T10:05:00.121 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_setxattr": 20, 2024-09-10T10:05:00.121 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_cmpxattr": 0, 2024-09-10T10:05:00.121 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_rmxattr": 0, 2024-09-10T10:05:00.121 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_resetxattrs": 0, 2024-09-10T10:05:00.122 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_call": 0, 2024-09-10T10:05:00.122 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_watch": 0, 2024-09-10T10:05:00.122 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_notify": 0, 2024-09-10T10:05:00.122 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T10:05:00.122 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_pgls": 0, 2024-09-10T10:05:00.122 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_pgls_filter": 0, 2024-09-10T10:05:00.122 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_other": 0, 2024-09-10T10:05:00.122 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_active": 0, 2024-09-10T10:05:00.122 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_send": 0, 2024-09-10T10:05:00.122 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_resend": 0, 2024-09-10T10:05:00.122 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_ping": 0, 2024-09-10T10:05:00.122 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_active": 0, 2024-09-10T10:05:00.122 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_send": 0, 2024-09-10T10:05:00.122 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_resend": 0, 2024-09-10T10:05:00.123 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_active": 0, 2024-09-10T10:05:00.123 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_send": 0, 2024-09-10T10:05:00.123 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_resend": 0, 2024-09-10T10:05:00.123 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_active": 0, 2024-09-10T10:05:00.123 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_send": 0, 2024-09-10T10:05:00.123 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_resend": 0, 2024-09-10T10:05:00.123 INFO:teuthology.orchestra.run.smithi134.stdout: "command_active": 0, 2024-09-10T10:05:00.123 INFO:teuthology.orchestra.run.smithi134.stdout: "command_send": 0, 2024-09-10T10:05:00.123 INFO:teuthology.orchestra.run.smithi134.stdout: "command_resend": 0, 2024-09-10T10:05:00.124 INFO:teuthology.orchestra.run.smithi134.stdout: "map_epoch": 96, 2024-09-10T10:05:00.124 INFO:teuthology.orchestra.run.smithi134.stdout: "map_full": 0, 2024-09-10T10:05:00.124 INFO:teuthology.orchestra.run.smithi134.stdout: "map_inc": 2, 2024-09-10T10:05:00.124 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_sessions": 12, 2024-09-10T10:05:00.124 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_session_open": 12, 2024-09-10T10:05:00.124 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_session_close": 0, 2024-09-10T10:05:00.124 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_laggy": 0, 2024-09-10T10:05:00.124 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_wr": 102, 2024-09-10T10:05:00.124 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_rd": 0, 2024-09-10T10:05:00.124 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_del": 21 2024-09-10T10:05:00.124 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:00.124 INFO:teuthology.orchestra.run.smithi134.stdout: "oft": { 2024-09-10T10:05:00.124 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_objs": 1, 2024-09-10T10:05:00.125 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_kv_pairs": 0, 2024-09-10T10:05:00.125 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_updates": 0, 2024-09-10T10:05:00.125 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_removes": 0 2024-09-10T10:05:00.125 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:00.125 INFO:teuthology.orchestra.run.smithi134.stdout: "osd-slow-ops": { 2024-09-10T10:05:00.125 INFO:teuthology.orchestra.run.smithi134.stdout: "slow_ops_count": 5807 2024-09-10T10:05:00.125 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:00.125 INFO:teuthology.orchestra.run.smithi134.stdout: "purge_queue": { 2024-09-10T10:05:00.125 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_ops": 0, 2024-09-10T10:05:00.125 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_ops_high_water": 10, 2024-09-10T10:05:00.125 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing": 0, 2024-09-10T10:05:00.125 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_high_water": 5, 2024-09-10T10:05:00.125 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executed_ops": 2287, 2024-09-10T10:05:00.126 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executed": 1145, 2024-09-10T10:05:00.126 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_item_in_journal": 0 2024-09-10T10:05:00.126 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:00.126 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T10:05:00.126 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-10T10:05:00.126 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 104857600, 2024-09-10T10:05:00.126 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-10T10:05:00.126 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 20099, 2024-09-10T10:05:00.126 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 6311047, 2024-09-10T10:05:00.126 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-10T10:05:00.126 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 20099, 2024-09-10T10:05:00.126 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-09-10T10:05:00.126 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-09-10T10:05:00.126 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 20099, 2024-09-10T10:05:00.126 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 6311047, 2024-09-10T10:05:00.127 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-10T10:05:00.127 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:05:00.127 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:05:00.127 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:05:00.127 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:05:00.127 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:00.127 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-objecter_bytes": { 2024-09-10T10:05:00.127 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-10T10:05:00.127 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 104857600, 2024-09-10T10:05:00.127 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-10T10:05:00.127 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-09-10T10:05:00.127 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-09-10T10:05:00.127 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-10T10:05:00.127 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-09-10T10:05:00.128 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 11391, 2024-09-10T10:05:00.128 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 6670392, 2024-09-10T10:05:00.128 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 11391, 2024-09-10T10:05:00.128 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 6670392, 2024-09-10T10:05:00.128 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-10T10:05:00.128 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:05:00.128 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:05:00.128 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:05:00.128 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:05:00.128 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:00.128 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-objecter_ops": { 2024-09-10T10:05:00.128 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-10T10:05:00.128 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 1024, 2024-09-10T10:05:00.128 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-10T10:05:00.129 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-09-10T10:05:00.129 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-09-10T10:05:00.129 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-10T10:05:00.129 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-09-10T10:05:00.129 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 11391, 2024-09-10T10:05:00.129 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 11391, 2024-09-10T10:05:00.129 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 11391, 2024-09-10T10:05:00.129 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 11391, 2024-09-10T10:05:00.129 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-10T10:05:00.129 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:05:00.129 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:05:00.129 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:05:00.129 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:05:00.129 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:00.129 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-write_buf_throttle": { 2024-09-10T10:05:00.130 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-10T10:05:00.130 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 3758096384, 2024-09-10T10:05:00.130 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-10T10:05:00.130 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 1145, 2024-09-10T10:05:00.130 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 124781, 2024-09-10T10:05:00.130 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-10T10:05:00.130 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 1145, 2024-09-10T10:05:00.130 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-09-10T10:05:00.130 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-09-10T10:05:00.130 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 20, 2024-09-10T10:05:00.130 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 124781, 2024-09-10T10:05:00.130 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-10T10:05:00.130 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:05:00.130 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:05:00.130 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:05:00.131 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:05:00.131 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:00.131 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-write_buf_throttle-0x559e471d6960": { 2024-09-10T10:05:00.131 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-10T10:05:00.131 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 3758096384, 2024-09-10T10:05:00.131 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-10T10:05:00.131 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 3931, 2024-09-10T10:05:00.131 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 6289780, 2024-09-10T10:05:00.131 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-10T10:05:00.131 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 3931, 2024-09-10T10:05:00.131 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-09-10T10:05:00.131 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-09-10T10:05:00.131 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 213, 2024-09-10T10:05:00.131 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 6289780, 2024-09-10T10:05:00.131 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-10T10:05:00.132 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:05:00.132 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:05:00.132 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:05:00.132 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:05:00.132 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:05:00.132 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:05:00.897 INFO:tasks.check_counter:Daemon mds.f mds.exported=50 2024-09-10T10:05:00.897 INFO:tasks.check_counter:Daemon mds.f mds.imported=50 2024-09-10T10:05:00.897 INFO:tasks.check_counter:Daemon mds.f mds.dir_split=0 2024-09-10T10:05:00.897 DEBUG:tasks.check_counter:Getting stats from i 2024-09-10T10:05:00.898 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph daemon mds.i perf dump 2024-09-10T10:05:01.145 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T10:05:01.265 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:05:00 smithi134 ceph-mon[35674]: pgmap v686: 129 pgs: 129 active+clean; 115 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-10T10:05:01.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:05:00 smithi120 ceph-mon[34770]: pgmap v686: 129 pgs: 129 active+clean; 115 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-10T10:05:01.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:05:00 smithi077 ceph-mon[30319]: pgmap v686: 129 pgs: 129 active+clean; 115 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-10T10:05:01.665 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:05:01.665 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T10:05:01.665 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 11762, 2024-09-10T10:05:01.665 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 14739, 2024-09-10T10:05:01.665 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 4013495, 2024-09-10T10:05:01.665 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 10748811, 2024-09-10T10:05:01.666 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 25, 2024-09-10T10:05:01.666 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 7, 2024-09-10T10:05:01.666 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.737724090, 2024-09-10T10:05:01.666 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.292014764, 2024-09-10T10:05:01.666 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.375720403, 2024-09-10T10:05:01.666 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.117161959, 2024-09-10T10:05:01.666 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:05:01.666 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 14734, 2024-09-10T10:05:01.666 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 1.356951832, 2024-09-10T10:05:01.666 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000092096 2024-09-10T10:05:01.666 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:01.666 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:05:01.666 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 9448, 2024-09-10T10:05:01.667 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.002315277, 2024-09-10T10:05:01.667 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000245 2024-09-10T10:05:01.667 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:01.667 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 604128, 2024-09-10T10:05:01.667 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 144400 2024-09-10T10:05:01.667 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:01.667 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T10:05:01.667 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 7235, 2024-09-10T10:05:01.667 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 7453, 2024-09-10T10:05:01.667 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 3554053, 2024-09-10T10:05:01.667 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 6209156, 2024-09-10T10:05:01.667 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 12, 2024-09-10T10:05:01.667 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 7, 2024-09-10T10:05:01.668 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.586468272, 2024-09-10T10:05:01.668 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.245707041, 2024-09-10T10:05:01.668 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.200797744, 2024-09-10T10:05:01.668 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.128794581, 2024-09-10T10:05:01.668 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:05:01.668 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 7452, 2024-09-10T10:05:01.668 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.525422423, 2024-09-10T10:05:01.668 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000070507 2024-09-10T10:05:01.668 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:01.668 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:05:01.668 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 4648, 2024-09-10T10:05:01.668 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.001046198, 2024-09-10T10:05:01.668 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000225 2024-09-10T10:05:01.669 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:01.669 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-09-10T10:05:01.669 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 346544 2024-09-10T10:05:01.669 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:01.669 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T10:05:01.669 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 8349, 2024-09-10T10:05:01.669 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 6655, 2024-09-10T10:05:01.669 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 2512121, 2024-09-10T10:05:01.669 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 3256625, 2024-09-10T10:05:01.669 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 24, 2024-09-10T10:05:01.669 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 6, 2024-09-10T10:05:01.669 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.516796763, 2024-09-10T10:05:01.669 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.193829712, 2024-09-10T10:05:01.669 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.269983049, 2024-09-10T10:05:01.670 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.089284932, 2024-09-10T10:05:01.670 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:05:01.670 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 6657, 2024-09-10T10:05:01.670 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.456060068, 2024-09-10T10:05:01.670 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000068508 2024-09-10T10:05:01.670 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:01.670 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:05:01.670 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 8095, 2024-09-10T10:05:01.670 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.001210423, 2024-09-10T10:05:01.670 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000149 2024-09-10T10:05:01.670 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:01.670 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-10T10:05:01.670 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 0 2024-09-10T10:05:01.670 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:01.671 INFO:teuthology.orchestra.run.smithi134.stdout: "cct": { 2024-09-10T10:05:01.671 INFO:teuthology.orchestra.run.smithi134.stdout: "total_workers": 1, 2024-09-10T10:05:01.671 INFO:teuthology.orchestra.run.smithi134.stdout: "unhealthy_workers": 0 2024-09-10T10:05:01.671 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:01.671 INFO:teuthology.orchestra.run.smithi134.stdout: "finisher-MDSRank": { 2024-09-10T10:05:01.671 INFO:teuthology.orchestra.run.smithi134.stdout: "queue_len": 0, 2024-09-10T10:05:01.671 INFO:teuthology.orchestra.run.smithi134.stdout: "complete_latency": { 2024-09-10T10:05:01.671 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 6745, 2024-09-10T10:05:01.671 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 3.069769839, 2024-09-10T10:05:01.671 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000455117 2024-09-10T10:05:01.671 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:05:01.671 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:01.671 INFO:teuthology.orchestra.run.smithi134.stdout: "finisher-PurgeQueue": { 2024-09-10T10:05:01.672 INFO:teuthology.orchestra.run.smithi134.stdout: "queue_len": 0, 2024-09-10T10:05:01.672 INFO:teuthology.orchestra.run.smithi134.stdout: "complete_latency": { 2024-09-10T10:05:01.672 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 2551, 2024-09-10T10:05:01.672 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.359872039, 2024-09-10T10:05:01.672 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000141070 2024-09-10T10:05:01.672 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:05:01.672 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:01.672 INFO:teuthology.orchestra.run.smithi134.stdout: "mds": { 2024-09-10T10:05:01.672 INFO:teuthology.orchestra.run.smithi134.stdout: "request": 2922, 2024-09-10T10:05:01.672 INFO:teuthology.orchestra.run.smithi134.stdout: "reply": 2911, 2024-09-10T10:05:01.673 INFO:teuthology.orchestra.run.smithi134.stdout: "reply_latency": { 2024-09-10T10:05:01.673 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 2911, 2024-09-10T10:05:01.673 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 4.067506364, 2024-09-10T10:05:01.673 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.001397288 2024-09-10T10:05:01.673 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:01.673 INFO:teuthology.orchestra.run.smithi134.stdout: "slow_reply": 0, 2024-09-10T10:05:01.673 INFO:teuthology.orchestra.run.smithi134.stdout: "forward": 11, 2024-09-10T10:05:01.673 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_fetch_complete": 0, 2024-09-10T10:05:01.673 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_fetch_keys": 0, 2024-09-10T10:05:01.673 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_commit": 11, 2024-09-10T10:05:01.673 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_split": 0, 2024-09-10T10:05:01.673 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_merge": 0, 2024-09-10T10:05:01.673 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes": 275, 2024-09-10T10:05:01.673 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_top": 2, 2024-09-10T10:05:01.674 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_bottom": 258, 2024-09-10T10:05:01.674 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_pin_tail": 15, 2024-09-10T10:05:01.674 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_pinned": 15, 2024-09-10T10:05:01.674 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_expired": 16, 2024-09-10T10:05:01.674 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_with_caps": 0, 2024-09-10T10:05:01.674 INFO:teuthology.orchestra.run.smithi134.stdout: "caps": 0, 2024-09-10T10:05:01.674 INFO:teuthology.orchestra.run.smithi134.stdout: "subtrees": 4, 2024-09-10T10:05:01.674 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse": 4640, 2024-09-10T10:05:01.674 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_hit": 4136, 2024-09-10T10:05:01.674 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_forward": 10, 2024-09-10T10:05:01.674 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_discover": 25, 2024-09-10T10:05:01.674 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_dir_fetch": 0, 2024-09-10T10:05:01.674 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_remote_ino": 0, 2024-09-10T10:05:01.674 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_lock": 0, 2024-09-10T10:05:01.674 INFO:teuthology.orchestra.run.smithi134.stdout: "load_cent": 0, 2024-09-10T10:05:01.675 INFO:teuthology.orchestra.run.smithi134.stdout: "q": 0, 2024-09-10T10:05:01.675 INFO:teuthology.orchestra.run.smithi134.stdout: "exported": 3, 2024-09-10T10:05:01.675 INFO:teuthology.orchestra.run.smithi134.stdout: "exported_inodes": 2, 2024-09-10T10:05:01.675 INFO:teuthology.orchestra.run.smithi134.stdout: "imported": 145, 2024-09-10T10:05:01.675 INFO:teuthology.orchestra.run.smithi134.stdout: "imported_inodes": 2224, 2024-09-10T10:05:01.675 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_dir_fetch": 0, 2024-09-10T10:05:01.675 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_backtrace_fetch": 9389, 2024-09-10T10:05:01.675 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_peer_discover": 0, 2024-09-10T10:05:01.675 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rfiles": 4215, 2024-09-10T10:05:01.675 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rbytes": 81947162, 2024-09-10T10:05:01.675 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rsnaps": 0, 2024-09-10T10:05:01.675 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_backtrace_fetch": 9389, 2024-09-10T10:05:01.675 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_set_tag": 0, 2024-09-10T10:05:01.675 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T10:05:01.676 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_inotable_repaired": 0, 2024-09-10T10:05:01.676 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dir_inodes": 255, 2024-09-10T10:05:01.676 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dir_base_inodes": 0, 2024-09-10T10:05:01.676 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dirfrag_rstats": 255, 2024-09-10T10:05:01.676 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_file_inodes": 9134, 2024-09-10T10:05:01.676 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_inode_file_caps": 35, 2024-09-10T10:05:01.676 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_revoke": 437, 2024-09-10T10:05:01.676 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_grant": 1665, 2024-09-10T10:05:01.676 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T10:05:01.676 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T10:05:01.676 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T10:05:01.676 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_caps": 1551, 2024-09-10T10:05:01.676 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_caps_dirty": 747, 2024-09-10T10:05:01.677 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_cap_release": 38, 2024-09-10T10:05:01.677 INFO:teuthology.orchestra.run.smithi134.stdout: "process_request_cap_release": 2586 2024-09-10T10:05:01.677 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:01.677 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_cache": { 2024-09-10T10:05:01.677 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_update": 0, 2024-09-10T10:05:01.677 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_update_receipt": 0, 2024-09-10T10:05:01.677 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_try_discover": 0, 2024-09-10T10:05:01.677 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_send_discover": 239, 2024-09-10T10:05:01.677 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_handle_discover": 1, 2024-09-10T10:05:01.677 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays": 2, 2024-09-10T10:05:01.677 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays_delayed": 0, 2024-09-10T10:05:01.677 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays_enqueuing": 0, 2024-09-10T10:05:01.678 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_created": 1340, 2024-09-10T10:05:01.678 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_enqueued": 1338, 2024-09-10T10:05:01.678 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_reintegrated": 0, 2024-09-10T10:05:01.678 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_migrated": 0, 2024-09-10T10:05:01.678 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_processing": 0, 2024-09-10T10:05:01.678 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_enqueued": 0, 2024-09-10T10:05:01.678 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_prioritized": 0, 2024-09-10T10:05:01.678 INFO:teuthology.orchestra.run.smithi134.stdout: "recovery_started": 0, 2024-09-10T10:05:01.678 INFO:teuthology.orchestra.run.smithi134.stdout: "recovery_completed": 0, 2024-09-10T10:05:01.678 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_quiesce_path": 8, 2024-09-10T10:05:01.678 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_quiesce_inode": 6310, 2024-09-10T10:05:01.678 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_enqueue_scrub": 0, 2024-09-10T10:05:01.678 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_exportdir": 3, 2024-09-10T10:05:01.678 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_flush": 0, 2024-09-10T10:05:01.678 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_fragmentdir": 0, 2024-09-10T10:05:01.678 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_fragstats": 0, 2024-09-10T10:05:01.679 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_inodestats": 0 2024-09-10T10:05:01.679 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:01.679 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_log": { 2024-09-10T10:05:01.679 INFO:teuthology.orchestra.run.smithi134.stdout: "evlrg": 0, 2024-09-10T10:05:01.679 INFO:teuthology.orchestra.run.smithi134.stdout: "evadd": 4184, 2024-09-10T10:05:01.679 INFO:teuthology.orchestra.run.smithi134.stdout: "evex": 0, 2024-09-10T10:05:01.679 INFO:teuthology.orchestra.run.smithi134.stdout: "evtrm": 0, 2024-09-10T10:05:01.679 INFO:teuthology.orchestra.run.smithi134.stdout: "ev": 4184, 2024-09-10T10:05:01.679 INFO:teuthology.orchestra.run.smithi134.stdout: "evexg": 0, 2024-09-10T10:05:01.679 INFO:teuthology.orchestra.run.smithi134.stdout: "evexd": 0, 2024-09-10T10:05:01.679 INFO:teuthology.orchestra.run.smithi134.stdout: "segadd": 6, 2024-09-10T10:05:01.679 INFO:teuthology.orchestra.run.smithi134.stdout: "segex": 0, 2024-09-10T10:05:01.679 INFO:teuthology.orchestra.run.smithi134.stdout: "segtrm": 0, 2024-09-10T10:05:01.679 INFO:teuthology.orchestra.run.smithi134.stdout: "seg": 6, 2024-09-10T10:05:01.679 INFO:teuthology.orchestra.run.smithi134.stdout: "segmjr": 2, 2024-09-10T10:05:01.679 INFO:teuthology.orchestra.run.smithi134.stdout: "segexg": 0, 2024-09-10T10:05:01.680 INFO:teuthology.orchestra.run.smithi134.stdout: "segexd": 0, 2024-09-10T10:05:01.680 INFO:teuthology.orchestra.run.smithi134.stdout: "expos": 4194304, 2024-09-10T10:05:01.680 INFO:teuthology.orchestra.run.smithi134.stdout: "wrpos": 12175396, 2024-09-10T10:05:01.680 INFO:teuthology.orchestra.run.smithi134.stdout: "rdpos": 0, 2024-09-10T10:05:01.680 INFO:teuthology.orchestra.run.smithi134.stdout: "jlat": { 2024-09-10T10:05:01.680 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 272, 2024-09-10T10:05:01.680 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 9.291303286, 2024-09-10T10:05:01.680 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.034159203 2024-09-10T10:05:01.680 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:01.680 INFO:teuthology.orchestra.run.smithi134.stdout: "replayed": 0 2024-09-10T10:05:01.680 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:01.680 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_mem": { 2024-09-10T10:05:01.680 INFO:teuthology.orchestra.run.smithi134.stdout: "ino": 23, 2024-09-10T10:05:01.680 INFO:teuthology.orchestra.run.smithi134.stdout: "ino+": 1368, 2024-09-10T10:05:01.680 INFO:teuthology.orchestra.run.smithi134.stdout: "ino-": 1345, 2024-09-10T10:05:01.680 INFO:teuthology.orchestra.run.smithi134.stdout: "dir": 92, 2024-09-10T10:05:01.681 INFO:teuthology.orchestra.run.smithi134.stdout: "dir+": 209, 2024-09-10T10:05:01.681 INFO:teuthology.orchestra.run.smithi134.stdout: "dir-": 117, 2024-09-10T10:05:01.681 INFO:teuthology.orchestra.run.smithi134.stdout: "dn": 275, 2024-09-10T10:05:01.681 INFO:teuthology.orchestra.run.smithi134.stdout: "dn+": 3654, 2024-09-10T10:05:01.681 INFO:teuthology.orchestra.run.smithi134.stdout: "dn-": 3379, 2024-09-10T10:05:01.681 INFO:teuthology.orchestra.run.smithi134.stdout: "cap": 0, 2024-09-10T10:05:01.681 INFO:teuthology.orchestra.run.smithi134.stdout: "cap+": 2565, 2024-09-10T10:05:01.681 INFO:teuthology.orchestra.run.smithi134.stdout: "cap-": 2565, 2024-09-10T10:05:01.681 INFO:teuthology.orchestra.run.smithi134.stdout: "rss": 71272, 2024-09-10T10:05:01.681 INFO:teuthology.orchestra.run.smithi134.stdout: "heap": 223520 2024-09-10T10:05:01.681 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:01.681 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_server": { 2024-09-10T10:05:01.681 INFO:teuthology.orchestra.run.smithi134.stdout: "dispatch_client_request": 3385, 2024-09-10T10:05:01.682 INFO:teuthology.orchestra.run.smithi134.stdout: "dispatch_server_request": 2863, 2024-09-10T10:05:01.682 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_request": 2922, 2024-09-10T10:05:01.682 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_session": 11, 2024-09-10T10:05:01.682 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_peer_request": 6023, 2024-09-10T10:05:01.682 INFO:teuthology.orchestra.run.smithi134.stdout: "req_create_latency": { 2024-09-10T10:05:01.682 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 110, 2024-09-10T10:05:01.682 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.320118363, 2024-09-10T10:05:01.682 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.002910166 2024-09-10T10:05:01.682 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:01.682 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getattr_latency": { 2024-09-10T10:05:01.682 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 1, 2024-09-10T10:05:01.682 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000163338, 2024-09-10T10:05:01.682 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000163338 2024-09-10T10:05:01.682 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:01.682 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getfilelock_latency": { 2024-09-10T10:05:01.682 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:05:01.683 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:05:01.683 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:05:01.683 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:01.683 INFO:teuthology.orchestra.run.smithi134.stdout: "req_link_latency": { 2024-09-10T10:05:01.683 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:05:01.683 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:05:01.683 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:05:01.683 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:01.683 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookup_latency": { 2024-09-10T10:05:01.683 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 1339, 2024-09-10T10:05:01.683 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.437149112, 2024-09-10T10:05:01.683 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000326474 2024-09-10T10:05:01.683 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:01.683 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookuphash_latency": { 2024-09-10T10:05:01.683 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:05:01.683 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:05:01.684 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:05:01.684 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:01.684 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupino_latency": { 2024-09-10T10:05:01.684 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:05:01.684 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:05:01.684 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:05:01.684 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:01.684 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupname_latency": { 2024-09-10T10:05:01.684 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:05:01.684 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:05:01.684 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:05:01.684 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:01.684 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupparent_latency": { 2024-09-10T10:05:01.684 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:05:01.685 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:05:01.685 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:05:01.685 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:01.685 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupsnap_latency": { 2024-09-10T10:05:01.685 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:05:01.685 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:05:01.685 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:05:01.685 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:01.685 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lssnap_latency": { 2024-09-10T10:05:01.686 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:05:01.686 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:05:01.686 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:05:01.686 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:01.686 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mkdir_latency": { 2024-09-10T10:05:01.686 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 2, 2024-09-10T10:05:01.686 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.005023960, 2024-09-10T10:05:01.686 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.002511980 2024-09-10T10:05:01.686 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:01.686 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mknod_latency": { 2024-09-10T10:05:01.687 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:05:01.687 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:05:01.687 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:05:01.687 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:01.687 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mksnap_latency": { 2024-09-10T10:05:01.687 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:05:01.687 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:05:01.687 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:05:01.687 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:01.687 INFO:teuthology.orchestra.run.smithi134.stdout: "req_open_latency": { 2024-09-10T10:05:01.687 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 8, 2024-09-10T10:05:01.687 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.001685081, 2024-09-10T10:05:01.688 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000210635 2024-09-10T10:05:01.688 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:01.688 INFO:teuthology.orchestra.run.smithi134.stdout: "req_readdir_latency": { 2024-09-10T10:05:01.688 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 110, 2024-09-10T10:05:01.688 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.089831139, 2024-09-10T10:05:01.688 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000816646 2024-09-10T10:05:01.688 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:01.688 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rename_latency": { 2024-09-10T10:05:01.688 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 3, 2024-09-10T10:05:01.688 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.009195022, 2024-09-10T10:05:01.688 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.003065007 2024-09-10T10:05:01.688 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:01.689 INFO:teuthology.orchestra.run.smithi134.stdout: "req_renamesnap_latency": { 2024-09-10T10:05:01.689 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:05:01.689 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:05:01.689 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:05:01.689 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:01.689 INFO:teuthology.orchestra.run.smithi134.stdout: "req_snapdiff_latency": { 2024-09-10T10:05:01.689 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:05:01.689 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:05:01.689 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:05:01.689 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:01.689 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmdir_latency": { 2024-09-10T10:05:01.689 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 34, 2024-09-10T10:05:01.689 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 1.437219105, 2024-09-10T10:05:01.690 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.042271150 2024-09-10T10:05:01.690 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:01.690 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmsnap_latency": { 2024-09-10T10:05:01.690 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:05:01.690 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:05:01.690 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:05:01.690 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:01.690 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmxattr_latency": { 2024-09-10T10:05:01.690 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:05:01.690 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:05:01.690 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:05:01.690 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:01.690 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setattr_latency": { 2024-09-10T10:05:01.691 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:05:01.691 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:05:01.691 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:05:01.691 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:01.691 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setdirlayout_latency": { 2024-09-10T10:05:01.691 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:05:01.691 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:05:01.691 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:05:01.691 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:01.691 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setfilelock_latency": { 2024-09-10T10:05:01.691 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:05:01.691 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:05:01.691 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:05:01.691 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:01.692 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setlayout_latency": { 2024-09-10T10:05:01.692 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:05:01.692 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:05:01.692 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:05:01.692 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:01.692 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setxattr_latency": { 2024-09-10T10:05:01.692 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:05:01.692 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:05:01.692 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:05:01.692 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:01.692 INFO:teuthology.orchestra.run.smithi134.stdout: "req_symlink_latency": { 2024-09-10T10:05:01.692 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:05:01.692 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:05:01.693 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:05:01.693 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:01.693 INFO:teuthology.orchestra.run.smithi134.stdout: "req_unlink_latency": { 2024-09-10T10:05:01.693 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 1304, 2024-09-10T10:05:01.693 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 1.767121244, 2024-09-10T10:05:01.693 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.001355154 2024-09-10T10:05:01.693 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:01.693 INFO:teuthology.orchestra.run.smithi134.stdout: "cap_revoke_eviction": 0, 2024-09-10T10:05:01.693 INFO:teuthology.orchestra.run.smithi134.stdout: "cap_acquisition_throttle": 0, 2024-09-10T10:05:01.693 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getvxattr_latency": { 2024-09-10T10:05:01.693 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:05:01.693 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:05:01.693 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:05:01.694 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:05:01.694 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:01.694 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_sessions": { 2024-09-10T10:05:01.694 INFO:teuthology.orchestra.run.smithi134.stdout: "session_count": 1, 2024-09-10T10:05:01.694 INFO:teuthology.orchestra.run.smithi134.stdout: "session_add": 2, 2024-09-10T10:05:01.694 INFO:teuthology.orchestra.run.smithi134.stdout: "session_remove": 1, 2024-09-10T10:05:01.694 INFO:teuthology.orchestra.run.smithi134.stdout: "sessions_open": 1, 2024-09-10T10:05:01.694 INFO:teuthology.orchestra.run.smithi134.stdout: "sessions_stale": 0, 2024-09-10T10:05:01.694 INFO:teuthology.orchestra.run.smithi134.stdout: "total_load": 2172, 2024-09-10T10:05:01.694 INFO:teuthology.orchestra.run.smithi134.stdout: "average_load": 2172, 2024-09-10T10:05:01.694 INFO:teuthology.orchestra.run.smithi134.stdout: "avg_session_uptime": 1956, 2024-09-10T10:05:01.694 INFO:teuthology.orchestra.run.smithi134.stdout: "mdthresh_evicted": 0 2024-09-10T10:05:01.694 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:01.694 INFO:teuthology.orchestra.run.smithi134.stdout: "mempool": { 2024-09-10T10:05:01.695 INFO:teuthology.orchestra.run.smithi134.stdout: "bloom_filter_bytes": 0, 2024-09-10T10:05:01.695 INFO:teuthology.orchestra.run.smithi134.stdout: "bloom_filter_items": 0, 2024-09-10T10:05:01.695 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T10:05:01.695 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_alloc_items": 0, 2024-09-10T10:05:01.695 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T10:05:01.695 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_data_items": 0, 2024-09-10T10:05:01.695 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T10:05:01.695 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T10:05:01.695 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T10:05:01.695 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T10:05:01.695 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T10:05:01.695 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_other_items": 0, 2024-09-10T10:05:01.695 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T10:05:01.695 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T10:05:01.695 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_extent_bytes": 0, 2024-09-10T10:05:01.696 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_extent_items": 0, 2024-09-10T10:05:01.696 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_blob_bytes": 0, 2024-09-10T10:05:01.696 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_blob_items": 0, 2024-09-10T10:05:01.696 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T10:05:01.696 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T10:05:01.696 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T10:05:01.696 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T10:05:01.696 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T10:05:01.696 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_fsck_items": 0, 2024-09-10T10:05:01.696 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_txc_bytes": 0, 2024-09-10T10:05:01.696 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_txc_items": 0, 2024-09-10T10:05:01.696 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T10:05:01.696 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T10:05:01.696 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_bytes": 0, 2024-09-10T10:05:01.696 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_items": 0, 2024-09-10T10:05:01.697 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_bytes": 0, 2024-09-10T10:05:01.697 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_items": 0, 2024-09-10T10:05:01.697 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T10:05:01.697 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_reader_items": 0, 2024-09-10T10:05:01.697 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T10:05:01.697 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_writer_items": 0, 2024-09-10T10:05:01.697 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_anon_bytes": 279519, 2024-09-10T10:05:01.697 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_anon_items": 99, 2024-09-10T10:05:01.697 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_meta_bytes": 0, 2024-09-10T10:05:01.697 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_meta_items": 0, 2024-09-10T10:05:01.697 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_bytes": 0, 2024-09-10T10:05:01.697 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_items": 0, 2024-09-10T10:05:01.697 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_mapbl_bytes": 0, 2024-09-10T10:05:01.698 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_mapbl_items": 0, 2024-09-10T10:05:01.698 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_pglog_bytes": 0, 2024-09-10T10:05:01.698 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_pglog_items": 0, 2024-09-10T10:05:01.698 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_bytes": 9144, 2024-09-10T10:05:01.698 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_items": 172, 2024-09-10T10:05:01.698 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T10:05:01.698 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_mapping_items": 0, 2024-09-10T10:05:01.698 INFO:teuthology.orchestra.run.smithi134.stdout: "pgmap_bytes": 0, 2024-09-10T10:05:01.698 INFO:teuthology.orchestra.run.smithi134.stdout: "pgmap_items": 0, 2024-09-10T10:05:01.698 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_co_bytes": 381015, 2024-09-10T10:05:01.698 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_co_items": 2795, 2024-09-10T10:05:01.698 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_1_bytes": 0, 2024-09-10T10:05:01.698 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_1_items": 0, 2024-09-10T10:05:01.698 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_2_bytes": 0, 2024-09-10T10:05:01.698 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_2_items": 0 2024-09-10T10:05:01.699 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:01.699 INFO:teuthology.orchestra.run.smithi134.stdout: "objecter": { 2024-09-10T10:05:01.699 INFO:teuthology.orchestra.run.smithi134.stdout: "op_active": 0, 2024-09-10T10:05:01.699 INFO:teuthology.orchestra.run.smithi134.stdout: "op_laggy": 0, 2024-09-10T10:05:01.699 INFO:teuthology.orchestra.run.smithi134.stdout: "op_send": 12571, 2024-09-10T10:05:01.699 INFO:teuthology.orchestra.run.smithi134.stdout: "op_send_bytes": 8273342, 2024-09-10T10:05:01.699 INFO:teuthology.orchestra.run.smithi134.stdout: "op_resend": 32, 2024-09-10T10:05:01.699 INFO:teuthology.orchestra.run.smithi134.stdout: "op_reply": 12571, 2024-09-10T10:05:01.699 INFO:teuthology.orchestra.run.smithi134.stdout: "op_latency": { 2024-09-10T10:05:01.699 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 12571, 2024-09-10T10:05:01.699 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 29.635280335, 2024-09-10T10:05:01.699 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.002357432 2024-09-10T10:05:01.699 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:01.699 INFO:teuthology.orchestra.run.smithi134.stdout: "op_inflight": 0, 2024-09-10T10:05:01.699 INFO:teuthology.orchestra.run.smithi134.stdout: "oplen_avg": { 2024-09-10T10:05:01.699 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 12571, 2024-09-10T10:05:01.700 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 12652 2024-09-10T10:05:01.700 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:01.700 INFO:teuthology.orchestra.run.smithi134.stdout: "op": 12571, 2024-09-10T10:05:01.700 INFO:teuthology.orchestra.run.smithi134.stdout: "op_r": 9421, 2024-09-10T10:05:01.700 INFO:teuthology.orchestra.run.smithi134.stdout: "op_w": 3150, 2024-09-10T10:05:01.700 INFO:teuthology.orchestra.run.smithi134.stdout: "op_rmw": 0, 2024-09-10T10:05:01.700 INFO:teuthology.orchestra.run.smithi134.stdout: "op_pg": 0, 2024-09-10T10:05:01.700 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_stat": 0, 2024-09-10T10:05:01.700 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_create": 10, 2024-09-10T10:05:01.700 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_read": 32, 2024-09-10T10:05:01.700 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_write": 304, 2024-09-10T10:05:01.700 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_writefull": 32, 2024-09-10T10:05:01.700 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_writesame": 0, 2024-09-10T10:05:01.700 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_append": 0, 2024-09-10T10:05:01.701 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_zero": 0, 2024-09-10T10:05:01.701 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_truncate": 0, 2024-09-10T10:05:01.701 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_delete": 2740, 2024-09-10T10:05:01.701 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_mapext": 0, 2024-09-10T10:05:01.701 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_sparse_read": 0, 2024-09-10T10:05:01.701 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_clonerange": 0, 2024-09-10T10:05:01.701 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_getxattr": 9389, 2024-09-10T10:05:01.701 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_setxattr": 20, 2024-09-10T10:05:01.701 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_cmpxattr": 0, 2024-09-10T10:05:01.701 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_rmxattr": 0, 2024-09-10T10:05:01.701 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_resetxattrs": 0, 2024-09-10T10:05:01.701 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_call": 0, 2024-09-10T10:05:01.701 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_watch": 0, 2024-09-10T10:05:01.701 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_notify": 0, 2024-09-10T10:05:01.701 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T10:05:01.702 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_pgls": 0, 2024-09-10T10:05:01.702 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_pgls_filter": 0, 2024-09-10T10:05:01.702 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_other": 0, 2024-09-10T10:05:01.702 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_active": 0, 2024-09-10T10:05:01.702 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_send": 0, 2024-09-10T10:05:01.702 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_resend": 0, 2024-09-10T10:05:01.702 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_ping": 0, 2024-09-10T10:05:01.702 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_active": 0, 2024-09-10T10:05:01.702 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_send": 0, 2024-09-10T10:05:01.702 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_resend": 0, 2024-09-10T10:05:01.702 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_active": 0, 2024-09-10T10:05:01.702 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_send": 0, 2024-09-10T10:05:01.702 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_resend": 0, 2024-09-10T10:05:01.702 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_active": 0, 2024-09-10T10:05:01.702 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_send": 0, 2024-09-10T10:05:01.703 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_resend": 0, 2024-09-10T10:05:01.703 INFO:teuthology.orchestra.run.smithi134.stdout: "command_active": 0, 2024-09-10T10:05:01.703 INFO:teuthology.orchestra.run.smithi134.stdout: "command_send": 0, 2024-09-10T10:05:01.703 INFO:teuthology.orchestra.run.smithi134.stdout: "command_resend": 0, 2024-09-10T10:05:01.703 INFO:teuthology.orchestra.run.smithi134.stdout: "map_epoch": 96, 2024-09-10T10:05:01.703 INFO:teuthology.orchestra.run.smithi134.stdout: "map_full": 0, 2024-09-10T10:05:01.703 INFO:teuthology.orchestra.run.smithi134.stdout: "map_inc": 2, 2024-09-10T10:05:01.703 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_sessions": 12, 2024-09-10T10:05:01.703 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_session_open": 12, 2024-09-10T10:05:01.703 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_session_close": 0, 2024-09-10T10:05:01.703 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_laggy": 0, 2024-09-10T10:05:01.703 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_wr": 102, 2024-09-10T10:05:01.703 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_rd": 0, 2024-09-10T10:05:01.703 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_del": 23 2024-09-10T10:05:01.703 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:01.704 INFO:teuthology.orchestra.run.smithi134.stdout: "oft": { 2024-09-10T10:05:01.704 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_objs": 1, 2024-09-10T10:05:01.704 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_kv_pairs": 0, 2024-09-10T10:05:01.704 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_updates": 0, 2024-09-10T10:05:01.704 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_removes": 0 2024-09-10T10:05:01.704 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:01.704 INFO:teuthology.orchestra.run.smithi134.stdout: "osd-slow-ops": { 2024-09-10T10:05:01.704 INFO:teuthology.orchestra.run.smithi134.stdout: "slow_ops_count": 6311 2024-09-10T10:05:01.704 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:01.704 INFO:teuthology.orchestra.run.smithi134.stdout: "purge_queue": { 2024-09-10T10:05:01.704 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_ops": 0, 2024-09-10T10:05:01.704 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_ops_high_water": 65, 2024-09-10T10:05:01.704 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing": 0, 2024-09-10T10:05:01.704 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_high_water": 7, 2024-09-10T10:05:01.705 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executed_ops": 2727, 2024-09-10T10:05:01.705 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executed": 1338, 2024-09-10T10:05:01.705 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_item_in_journal": 0 2024-09-10T10:05:01.705 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:01.705 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T10:05:01.705 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-10T10:05:01.705 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 104857600, 2024-09-10T10:05:01.705 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-10T10:05:01.705 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 27342, 2024-09-10T10:05:01.705 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 7675674, 2024-09-10T10:05:01.705 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-10T10:05:01.705 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 27342, 2024-09-10T10:05:01.705 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-09-10T10:05:01.706 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-09-10T10:05:01.706 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 27342, 2024-09-10T10:05:01.706 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 7675674, 2024-09-10T10:05:01.706 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-10T10:05:01.706 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:05:01.706 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:05:01.706 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:05:01.706 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:05:01.706 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:01.706 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-objecter_bytes": { 2024-09-10T10:05:01.706 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-10T10:05:01.706 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 104857600, 2024-09-10T10:05:01.706 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-10T10:05:01.706 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-09-10T10:05:01.706 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-09-10T10:05:01.707 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-10T10:05:01.707 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-09-10T10:05:01.707 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 12571, 2024-09-10T10:05:01.707 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 8418952, 2024-09-10T10:05:01.707 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 12571, 2024-09-10T10:05:01.707 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 8418952, 2024-09-10T10:05:01.707 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-10T10:05:01.707 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:05:01.707 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:05:01.707 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:05:01.707 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:05:01.707 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:01.707 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-objecter_ops": { 2024-09-10T10:05:01.707 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-10T10:05:01.707 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 1024, 2024-09-10T10:05:01.708 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-10T10:05:01.708 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-09-10T10:05:01.708 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-09-10T10:05:01.708 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-10T10:05:01.708 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-09-10T10:05:01.708 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 12571, 2024-09-10T10:05:01.708 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 12571, 2024-09-10T10:05:01.708 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 12571, 2024-09-10T10:05:01.708 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 12571, 2024-09-10T10:05:01.708 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-10T10:05:01.708 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:05:01.708 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:05:01.708 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:05:01.708 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:05:01.709 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:01.709 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-write_buf_throttle": { 2024-09-10T10:05:01.709 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-10T10:05:01.709 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 3758096384, 2024-09-10T10:05:01.709 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-10T10:05:01.709 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 1338, 2024-09-10T10:05:01.709 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 145610, 2024-09-10T10:05:01.709 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-10T10:05:01.709 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 1338, 2024-09-10T10:05:01.709 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-09-10T10:05:01.709 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-09-10T10:05:01.709 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 32, 2024-09-10T10:05:01.709 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 145610, 2024-09-10T10:05:01.709 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-10T10:05:01.710 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:05:01.710 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:05:01.710 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:05:01.710 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:05:01.710 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:01.710 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-write_buf_throttle-0x5568d943c960": { 2024-09-10T10:05:01.710 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-10T10:05:01.710 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 3758096384, 2024-09-10T10:05:01.710 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-10T10:05:01.710 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 4184, 2024-09-10T10:05:01.710 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 7981092, 2024-09-10T10:05:01.710 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-10T10:05:01.710 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 4184, 2024-09-10T10:05:01.710 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-09-10T10:05:01.711 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-09-10T10:05:01.711 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 272, 2024-09-10T10:05:01.711 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 7981092, 2024-09-10T10:05:01.711 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-10T10:05:01.711 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:05:01.711 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:05:01.711 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:05:01.711 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:05:01.711 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:05:01.711 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:05:02.438 INFO:tasks.check_counter:Daemon mds.i mds.exported=3 2024-09-10T10:05:02.438 INFO:tasks.check_counter:Daemon mds.i mds.imported=145 2024-09-10T10:05:02.438 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=0 2024-09-10T10:05:02.438 DEBUG:tasks.check_counter:Getting stats from l 2024-09-10T10:05:02.439 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph daemon mds.l perf dump 2024-09-10T10:05:02.683 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T10:05:03.092 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:05:03 smithi134 ceph-mon[35674]: pgmap v687: 129 pgs: 129 active+clean; 115 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T10:05:03.208 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:05:03.208 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T10:05:03.208 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 726, 2024-09-10T10:05:03.208 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 726, 2024-09-10T10:05:03.208 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 4387664, 2024-09-10T10:05:03.208 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 230108, 2024-09-10T10:05:03.208 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 6, 2024-09-10T10:05:03.208 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 2, 2024-09-10T10:05:03.209 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.067288430, 2024-09-10T10:05:03.209 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.019506814, 2024-09-10T10:05:03.209 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.022693136, 2024-09-10T10:05:03.209 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.018515244, 2024-09-10T10:05:03.209 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:05:03.209 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 725, 2024-09-10T10:05:03.209 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.032065520, 2024-09-10T10:05:03.209 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000044228 2024-09-10T10:05:03.209 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:03.209 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:05:03.209 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:05:03.209 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:05:03.209 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:05:03.209 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:03.209 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-09-10T10:05:03.210 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 448 2024-09-10T10:05:03.210 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:03.210 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T10:05:03.210 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 883, 2024-09-10T10:05:03.210 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 868, 2024-09-10T10:05:03.210 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 4523010, 2024-09-10T10:05:03.210 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 339165, 2024-09-10T10:05:03.210 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 4, 2024-09-10T10:05:03.210 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 3, 2024-09-10T10:05:03.210 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.090268480, 2024-09-10T10:05:03.210 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.029101694, 2024-09-10T10:05:03.210 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.038900413, 2024-09-10T10:05:03.210 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.022193055, 2024-09-10T10:05:03.210 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:05:03.210 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 867, 2024-09-10T10:05:03.211 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.033755847, 2024-09-10T10:05:03.211 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000038934 2024-09-10T10:05:03.211 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:03.211 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:05:03.211 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:05:03.211 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:05:03.211 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:05:03.211 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:03.211 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 577024, 2024-09-10T10:05:03.211 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 134368 2024-09-10T10:05:03.211 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:03.211 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T10:05:03.211 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 739, 2024-09-10T10:05:03.211 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 903, 2024-09-10T10:05:03.211 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 245158, 2024-09-10T10:05:03.212 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 579340, 2024-09-10T10:05:03.212 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 5, 2024-09-10T10:05:03.212 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 4, 2024-09-10T10:05:03.212 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.090465965, 2024-09-10T10:05:03.212 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.031934040, 2024-09-10T10:05:03.212 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.022520173, 2024-09-10T10:05:03.212 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.027058805, 2024-09-10T10:05:03.212 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:05:03.212 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 901, 2024-09-10T10:05:03.212 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.045884284, 2024-09-10T10:05:03.212 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000050925 2024-09-10T10:05:03.212 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:03.212 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:05:03.212 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:05:03.212 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:05:03.213 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:05:03.213 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:03.213 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-10T10:05:03.213 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 344656 2024-09-10T10:05:03.213 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:03.213 INFO:teuthology.orchestra.run.smithi134.stdout: "cct": { 2024-09-10T10:05:03.213 INFO:teuthology.orchestra.run.smithi134.stdout: "total_workers": 1, 2024-09-10T10:05:03.213 INFO:teuthology.orchestra.run.smithi134.stdout: "unhealthy_workers": 0 2024-09-10T10:05:03.213 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:03.213 INFO:teuthology.orchestra.run.smithi134.stdout: "finisher-MDSRank": { 2024-09-10T10:05:03.213 INFO:teuthology.orchestra.run.smithi134.stdout: "queue_len": 0, 2024-09-10T10:05:03.213 INFO:teuthology.orchestra.run.smithi134.stdout: "complete_latency": { 2024-09-10T10:05:03.213 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 3498, 2024-09-10T10:05:03.213 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.100589242, 2024-09-10T10:05:03.214 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000028756 2024-09-10T10:05:03.214 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:05:03.214 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:03.214 INFO:teuthology.orchestra.run.smithi134.stdout: "finisher-PurgeQueue": { 2024-09-10T10:05:03.214 INFO:teuthology.orchestra.run.smithi134.stdout: "queue_len": 0, 2024-09-10T10:05:03.214 INFO:teuthology.orchestra.run.smithi134.stdout: "complete_latency": { 2024-09-10T10:05:03.214 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:05:03.214 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:05:03.214 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:05:03.214 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:05:03.214 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:03.214 INFO:teuthology.orchestra.run.smithi134.stdout: "mds": { 2024-09-10T10:05:03.214 INFO:teuthology.orchestra.run.smithi134.stdout: "request": 0, 2024-09-10T10:05:03.214 INFO:teuthology.orchestra.run.smithi134.stdout: "reply": 0, 2024-09-10T10:05:03.214 INFO:teuthology.orchestra.run.smithi134.stdout: "reply_latency": { 2024-09-10T10:05:03.215 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:05:03.215 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:05:03.215 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:05:03.215 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:03.215 INFO:teuthology.orchestra.run.smithi134.stdout: "slow_reply": 0, 2024-09-10T10:05:03.215 INFO:teuthology.orchestra.run.smithi134.stdout: "forward": 0, 2024-09-10T10:05:03.215 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_fetch_complete": 0, 2024-09-10T10:05:03.215 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_fetch_keys": 0, 2024-09-10T10:05:03.215 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_commit": 0, 2024-09-10T10:05:03.215 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_split": 0, 2024-09-10T10:05:03.215 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_merge": 0, 2024-09-10T10:05:03.215 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes": 2442, 2024-09-10T10:05:03.215 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_top": 0, 2024-09-10T10:05:03.215 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_bottom": 2442, 2024-09-10T10:05:03.215 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_pin_tail": 0, 2024-09-10T10:05:03.216 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_pinned": 2442, 2024-09-10T10:05:03.216 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_expired": 0, 2024-09-10T10:05:03.216 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_with_caps": 0, 2024-09-10T10:05:03.216 INFO:teuthology.orchestra.run.smithi134.stdout: "caps": 0, 2024-09-10T10:05:03.216 INFO:teuthology.orchestra.run.smithi134.stdout: "subtrees": 1, 2024-09-10T10:05:03.216 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse": 0, 2024-09-10T10:05:03.216 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_hit": 0, 2024-09-10T10:05:03.216 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_forward": 0, 2024-09-10T10:05:03.216 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_discover": 0, 2024-09-10T10:05:03.216 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_dir_fetch": 0, 2024-09-10T10:05:03.216 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_remote_ino": 0, 2024-09-10T10:05:03.216 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_lock": 0, 2024-09-10T10:05:03.216 INFO:teuthology.orchestra.run.smithi134.stdout: "load_cent": 0, 2024-09-10T10:05:03.216 INFO:teuthology.orchestra.run.smithi134.stdout: "q": 0, 2024-09-10T10:05:03.216 INFO:teuthology.orchestra.run.smithi134.stdout: "exported": 0, 2024-09-10T10:05:03.216 INFO:teuthology.orchestra.run.smithi134.stdout: "exported_inodes": 0, 2024-09-10T10:05:03.217 INFO:teuthology.orchestra.run.smithi134.stdout: "imported": 0, 2024-09-10T10:05:03.217 INFO:teuthology.orchestra.run.smithi134.stdout: "imported_inodes": 0, 2024-09-10T10:05:03.217 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_dir_fetch": 0, 2024-09-10T10:05:03.217 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_backtrace_fetch": 0, 2024-09-10T10:05:03.217 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_peer_discover": 0, 2024-09-10T10:05:03.217 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rfiles": 0, 2024-09-10T10:05:03.217 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rbytes": 0, 2024-09-10T10:05:03.217 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rsnaps": 0, 2024-09-10T10:05:03.217 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_backtrace_fetch": 0, 2024-09-10T10:05:03.217 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_set_tag": 0, 2024-09-10T10:05:03.217 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T10:05:03.217 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_inotable_repaired": 0, 2024-09-10T10:05:03.217 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dir_inodes": 0, 2024-09-10T10:05:03.217 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dir_base_inodes": 0, 2024-09-10T10:05:03.217 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dirfrag_rstats": 0, 2024-09-10T10:05:03.218 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_file_inodes": 0, 2024-09-10T10:05:03.218 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_inode_file_caps": 0, 2024-09-10T10:05:03.218 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_revoke": 0, 2024-09-10T10:05:03.218 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_grant": 0, 2024-09-10T10:05:03.218 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T10:05:03.218 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T10:05:03.218 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T10:05:03.218 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_caps": 0, 2024-09-10T10:05:03.218 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_caps_dirty": 0, 2024-09-10T10:05:03.218 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_cap_release": 0, 2024-09-10T10:05:03.218 INFO:teuthology.orchestra.run.smithi134.stdout: "process_request_cap_release": 0 2024-09-10T10:05:03.218 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:03.218 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_cache": { 2024-09-10T10:05:03.218 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_update": 0, 2024-09-10T10:05:03.218 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_update_receipt": 0, 2024-09-10T10:05:03.219 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_try_discover": 0, 2024-09-10T10:05:03.219 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_send_discover": 0, 2024-09-10T10:05:03.219 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_handle_discover": 0, 2024-09-10T10:05:03.219 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays": 0, 2024-09-10T10:05:03.219 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays_delayed": 0, 2024-09-10T10:05:03.219 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays_enqueuing": 0, 2024-09-10T10:05:03.219 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_created": 0, 2024-09-10T10:05:03.219 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_enqueued": 0, 2024-09-10T10:05:03.219 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_reintegrated": 0, 2024-09-10T10:05:03.219 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_migrated": 0, 2024-09-10T10:05:03.219 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_processing": 0, 2024-09-10T10:05:03.219 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_enqueued": 0, 2024-09-10T10:05:03.219 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_prioritized": 0, 2024-09-10T10:05:03.219 INFO:teuthology.orchestra.run.smithi134.stdout: "recovery_started": 0, 2024-09-10T10:05:03.219 INFO:teuthology.orchestra.run.smithi134.stdout: "recovery_completed": 0, 2024-09-10T10:05:03.219 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_quiesce_path": 0, 2024-09-10T10:05:03.220 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_quiesce_inode": 0, 2024-09-10T10:05:03.220 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_enqueue_scrub": 0, 2024-09-10T10:05:03.220 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_exportdir": 0, 2024-09-10T10:05:03.220 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_flush": 0, 2024-09-10T10:05:03.220 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_fragmentdir": 0, 2024-09-10T10:05:03.220 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_fragstats": 0, 2024-09-10T10:05:03.220 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_inodestats": 0 2024-09-10T10:05:03.220 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:03.220 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_log": { 2024-09-10T10:05:03.220 INFO:teuthology.orchestra.run.smithi134.stdout: "evlrg": 0, 2024-09-10T10:05:03.220 INFO:teuthology.orchestra.run.smithi134.stdout: "evadd": 0, 2024-09-10T10:05:03.220 INFO:teuthology.orchestra.run.smithi134.stdout: "evex": 0, 2024-09-10T10:05:03.220 INFO:teuthology.orchestra.run.smithi134.stdout: "evtrm": 0, 2024-09-10T10:05:03.220 INFO:teuthology.orchestra.run.smithi134.stdout: "ev": 4184, 2024-09-10T10:05:03.220 INFO:teuthology.orchestra.run.smithi134.stdout: "evexg": 0, 2024-09-10T10:05:03.221 INFO:teuthology.orchestra.run.smithi134.stdout: "evexd": 0, 2024-09-10T10:05:03.221 INFO:teuthology.orchestra.run.smithi134.stdout: "segadd": 0, 2024-09-10T10:05:03.221 INFO:teuthology.orchestra.run.smithi134.stdout: "segex": 0, 2024-09-10T10:05:03.221 INFO:teuthology.orchestra.run.smithi134.stdout: "segtrm": 0, 2024-09-10T10:05:03.221 INFO:teuthology.orchestra.run.smithi134.stdout: "seg": 6, 2024-09-10T10:05:03.221 INFO:teuthology.orchestra.run.smithi134.stdout: "segmjr": 2, 2024-09-10T10:05:03.221 INFO:teuthology.orchestra.run.smithi134.stdout: "segexg": 0, 2024-09-10T10:05:03.221 INFO:teuthology.orchestra.run.smithi134.stdout: "segexd": 0, 2024-09-10T10:05:03.221 INFO:teuthology.orchestra.run.smithi134.stdout: "expos": 4194304, 2024-09-10T10:05:03.221 INFO:teuthology.orchestra.run.smithi134.stdout: "wrpos": 12175396, 2024-09-10T10:05:03.221 INFO:teuthology.orchestra.run.smithi134.stdout: "rdpos": 12174258, 2024-09-10T10:05:03.221 INFO:teuthology.orchestra.run.smithi134.stdout: "jlat": { 2024-09-10T10:05:03.221 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:05:03.221 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:05:03.221 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:05:03.221 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:03.222 INFO:teuthology.orchestra.run.smithi134.stdout: "replayed": 4184 2024-09-10T10:05:03.222 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:03.222 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_mem": { 2024-09-10T10:05:03.222 INFO:teuthology.orchestra.run.smithi134.stdout: "ino": 10, 2024-09-10T10:05:03.222 INFO:teuthology.orchestra.run.smithi134.stdout: "ino+": 1364, 2024-09-10T10:05:03.222 INFO:teuthology.orchestra.run.smithi134.stdout: "ino-": 1354, 2024-09-10T10:05:03.222 INFO:teuthology.orchestra.run.smithi134.stdout: "dir": 78, 2024-09-10T10:05:03.222 INFO:teuthology.orchestra.run.smithi134.stdout: "dir+": 204, 2024-09-10T10:05:03.222 INFO:teuthology.orchestra.run.smithi134.stdout: "dir-": 126, 2024-09-10T10:05:03.222 INFO:teuthology.orchestra.run.smithi134.stdout: "dn": 2442, 2024-09-10T10:05:03.222 INFO:teuthology.orchestra.run.smithi134.stdout: "dn+": 3608, 2024-09-10T10:05:03.222 INFO:teuthology.orchestra.run.smithi134.stdout: "dn-": 1166, 2024-09-10T10:05:03.222 INFO:teuthology.orchestra.run.smithi134.stdout: "cap": 0, 2024-09-10T10:05:03.222 INFO:teuthology.orchestra.run.smithi134.stdout: "cap+": 0, 2024-09-10T10:05:03.222 INFO:teuthology.orchestra.run.smithi134.stdout: "cap-": 0, 2024-09-10T10:05:03.222 INFO:teuthology.orchestra.run.smithi134.stdout: "rss": 54188, 2024-09-10T10:05:03.223 INFO:teuthology.orchestra.run.smithi134.stdout: "heap": 215328 2024-09-10T10:05:03.223 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:03.223 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_server": { 2024-09-10T10:05:03.223 INFO:teuthology.orchestra.run.smithi134.stdout: "dispatch_client_request": 0, 2024-09-10T10:05:03.223 INFO:teuthology.orchestra.run.smithi134.stdout: "dispatch_server_request": 0, 2024-09-10T10:05:03.223 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_request": 0, 2024-09-10T10:05:03.223 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_session": 0, 2024-09-10T10:05:03.223 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_peer_request": 0, 2024-09-10T10:05:03.223 INFO:teuthology.orchestra.run.smithi134.stdout: "req_create_latency": { 2024-09-10T10:05:03.223 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:05:03.223 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:05:03.223 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:05:03.223 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:03.223 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getattr_latency": { 2024-09-10T10:05:03.223 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:05:03.224 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:05:03.224 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:05:03.224 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:03.224 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getfilelock_latency": { 2024-09-10T10:05:03.224 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:05:03.224 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:05:03.224 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:05:03.224 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:03.224 INFO:teuthology.orchestra.run.smithi134.stdout: "req_link_latency": { 2024-09-10T10:05:03.224 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:05:03.224 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:05:03.224 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:05:03.224 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:03.224 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookup_latency": { 2024-09-10T10:05:03.224 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:05:03.225 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:05:03.225 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:05:03.225 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:03.225 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookuphash_latency": { 2024-09-10T10:05:03.225 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:05:03.225 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:05:03.225 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:05:03.225 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:03.225 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupino_latency": { 2024-09-10T10:05:03.225 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:05:03.225 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:05:03.225 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:05:03.225 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:03.225 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupname_latency": { 2024-09-10T10:05:03.225 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:05:03.225 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:05:03.226 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:05:03.226 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:03.226 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupparent_latency": { 2024-09-10T10:05:03.226 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:05:03.226 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:05:03.226 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:05:03.226 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:03.226 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupsnap_latency": { 2024-09-10T10:05:03.226 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:05:03.226 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:05:03.226 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:05:03.226 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:03.226 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lssnap_latency": { 2024-09-10T10:05:03.226 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:05:03.226 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:05:03.226 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:05:03.227 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:03.227 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mkdir_latency": { 2024-09-10T10:05:03.227 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:05:03.227 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:05:03.227 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:05:03.227 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:03.227 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mknod_latency": { 2024-09-10T10:05:03.227 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:05:03.227 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:05:03.227 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:05:03.227 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:03.227 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mksnap_latency": { 2024-09-10T10:05:03.227 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:05:03.227 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:05:03.228 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:05:03.228 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:03.228 INFO:teuthology.orchestra.run.smithi134.stdout: "req_open_latency": { 2024-09-10T10:05:03.228 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:05:03.228 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:05:03.228 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:05:03.228 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:03.228 INFO:teuthology.orchestra.run.smithi134.stdout: "req_readdir_latency": { 2024-09-10T10:05:03.228 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:05:03.228 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:05:03.228 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:05:03.228 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:03.228 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rename_latency": { 2024-09-10T10:05:03.228 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:05:03.229 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:05:03.229 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:05:03.229 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:03.229 INFO:teuthology.orchestra.run.smithi134.stdout: "req_renamesnap_latency": { 2024-09-10T10:05:03.229 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:05:03.229 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:05:03.229 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:05:03.229 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:03.229 INFO:teuthology.orchestra.run.smithi134.stdout: "req_snapdiff_latency": { 2024-09-10T10:05:03.229 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:05:03.229 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:05:03.229 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:05:03.229 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:03.229 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmdir_latency": { 2024-09-10T10:05:03.229 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:05:03.230 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:05:03.230 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:05:03.230 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:03.230 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmsnap_latency": { 2024-09-10T10:05:03.230 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:05:03.230 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:05:03.230 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:05:03.230 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:03.230 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmxattr_latency": { 2024-09-10T10:05:03.230 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:05:03.230 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:05:03.230 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:05:03.230 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:03.230 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setattr_latency": { 2024-09-10T10:05:03.230 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:05:03.230 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:05:03.231 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:05:03.231 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:03.231 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setdirlayout_latency": { 2024-09-10T10:05:03.231 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:05:03.231 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:05:03.231 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:05:03.231 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:03.231 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setfilelock_latency": { 2024-09-10T10:05:03.231 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:05:03.231 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:05:03.231 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:05:03.231 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:03.231 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setlayout_latency": { 2024-09-10T10:05:03.231 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:05:03.231 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:05:03.231 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:05:03.232 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:03.232 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setxattr_latency": { 2024-09-10T10:05:03.232 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:05:03.232 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:05:03.232 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:05:03.232 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:03.232 INFO:teuthology.orchestra.run.smithi134.stdout: "req_symlink_latency": { 2024-09-10T10:05:03.232 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:05:03.232 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:05:03.232 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:05:03.232 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:03.232 INFO:teuthology.orchestra.run.smithi134.stdout: "req_unlink_latency": { 2024-09-10T10:05:03.232 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:05:03.232 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:05:03.232 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:05:03.232 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:03.233 INFO:teuthology.orchestra.run.smithi134.stdout: "cap_revoke_eviction": 0, 2024-09-10T10:05:03.233 INFO:teuthology.orchestra.run.smithi134.stdout: "cap_acquisition_throttle": 0, 2024-09-10T10:05:03.233 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getvxattr_latency": { 2024-09-10T10:05:03.233 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:05:03.233 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:05:03.233 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:05:03.233 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:05:03.233 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:03.233 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_sessions": { 2024-09-10T10:05:03.233 INFO:teuthology.orchestra.run.smithi134.stdout: "session_count": 1, 2024-09-10T10:05:03.233 INFO:teuthology.orchestra.run.smithi134.stdout: "session_add": 2, 2024-09-10T10:05:03.233 INFO:teuthology.orchestra.run.smithi134.stdout: "session_remove": 1, 2024-09-10T10:05:03.233 INFO:teuthology.orchestra.run.smithi134.stdout: "sessions_open": 1, 2024-09-10T10:05:03.233 INFO:teuthology.orchestra.run.smithi134.stdout: "sessions_stale": 0, 2024-09-10T10:05:03.233 INFO:teuthology.orchestra.run.smithi134.stdout: "total_load": 0, 2024-09-10T10:05:03.234 INFO:teuthology.orchestra.run.smithi134.stdout: "average_load": 0, 2024-09-10T10:05:03.234 INFO:teuthology.orchestra.run.smithi134.stdout: "avg_session_uptime": 1957, 2024-09-10T10:05:03.234 INFO:teuthology.orchestra.run.smithi134.stdout: "mdthresh_evicted": 0 2024-09-10T10:05:03.234 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:03.234 INFO:teuthology.orchestra.run.smithi134.stdout: "mempool": { 2024-09-10T10:05:03.234 INFO:teuthology.orchestra.run.smithi134.stdout: "bloom_filter_bytes": 0, 2024-09-10T10:05:03.234 INFO:teuthology.orchestra.run.smithi134.stdout: "bloom_filter_items": 0, 2024-09-10T10:05:03.234 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T10:05:03.234 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_alloc_items": 0, 2024-09-10T10:05:03.234 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T10:05:03.234 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_data_items": 0, 2024-09-10T10:05:03.234 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T10:05:03.234 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T10:05:03.234 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T10:05:03.234 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T10:05:03.235 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T10:05:03.235 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_other_items": 0, 2024-09-10T10:05:03.235 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T10:05:03.235 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T10:05:03.235 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_extent_bytes": 0, 2024-09-10T10:05:03.235 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_extent_items": 0, 2024-09-10T10:05:03.235 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_blob_bytes": 0, 2024-09-10T10:05:03.235 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_blob_items": 0, 2024-09-10T10:05:03.235 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T10:05:03.235 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T10:05:03.235 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T10:05:03.235 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T10:05:03.235 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T10:05:03.235 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_fsck_items": 0, 2024-09-10T10:05:03.235 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_txc_bytes": 0, 2024-09-10T10:05:03.235 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_txc_items": 0, 2024-09-10T10:05:03.236 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T10:05:03.236 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T10:05:03.236 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_bytes": 0, 2024-09-10T10:05:03.236 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_items": 0, 2024-09-10T10:05:03.236 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_bytes": 0, 2024-09-10T10:05:03.236 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_items": 0, 2024-09-10T10:05:03.236 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T10:05:03.236 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_reader_items": 0, 2024-09-10T10:05:03.236 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T10:05:03.236 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_writer_items": 0, 2024-09-10T10:05:03.236 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_anon_bytes": 197571, 2024-09-10T10:05:03.236 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_anon_items": 58, 2024-09-10T10:05:03.236 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_meta_bytes": 0, 2024-09-10T10:05:03.236 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_meta_items": 0, 2024-09-10T10:05:03.236 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_bytes": 0, 2024-09-10T10:05:03.237 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_items": 0, 2024-09-10T10:05:03.237 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_mapbl_bytes": 0, 2024-09-10T10:05:03.237 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_mapbl_items": 0, 2024-09-10T10:05:03.237 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_pglog_bytes": 0, 2024-09-10T10:05:03.237 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_pglog_items": 0, 2024-09-10T10:05:03.237 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_bytes": 9144, 2024-09-10T10:05:03.237 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_items": 172, 2024-09-10T10:05:03.237 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T10:05:03.237 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_mapping_items": 0, 2024-09-10T10:05:03.237 INFO:teuthology.orchestra.run.smithi134.stdout: "pgmap_bytes": 0, 2024-09-10T10:05:03.237 INFO:teuthology.orchestra.run.smithi134.stdout: "pgmap_items": 0, 2024-09-10T10:05:03.237 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_co_bytes": 1846402, 2024-09-10T10:05:03.237 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_co_items": 9739, 2024-09-10T10:05:03.237 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_1_bytes": 0, 2024-09-10T10:05:03.238 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_1_items": 0, 2024-09-10T10:05:03.238 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_2_bytes": 0, 2024-09-10T10:05:03.238 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_2_items": 0 2024-09-10T10:05:03.238 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:03.238 INFO:teuthology.orchestra.run.smithi134.stdout: "objecter": { 2024-09-10T10:05:03.238 INFO:teuthology.orchestra.run.smithi134.stdout: "op_active": 0, 2024-09-10T10:05:03.238 INFO:teuthology.orchestra.run.smithi134.stdout: "op_laggy": 0, 2024-09-10T10:05:03.238 INFO:teuthology.orchestra.run.smithi134.stdout: "op_send": 2104, 2024-09-10T10:05:03.238 INFO:teuthology.orchestra.run.smithi134.stdout: "op_send_bytes": 21, 2024-09-10T10:05:03.238 INFO:teuthology.orchestra.run.smithi134.stdout: "op_resend": 3, 2024-09-10T10:05:03.238 INFO:teuthology.orchestra.run.smithi134.stdout: "op_reply": 2104, 2024-09-10T10:05:03.238 INFO:teuthology.orchestra.run.smithi134.stdout: "op_latency": { 2024-09-10T10:05:03.238 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 2104, 2024-09-10T10:05:03.238 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 1.263969405, 2024-09-10T10:05:03.238 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000600745 2024-09-10T10:05:03.239 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:03.239 INFO:teuthology.orchestra.run.smithi134.stdout: "op_inflight": 0, 2024-09-10T10:05:03.239 INFO:teuthology.orchestra.run.smithi134.stdout: "oplen_avg": { 2024-09-10T10:05:03.239 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 2104, 2024-09-10T10:05:03.239 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 2105 2024-09-10T10:05:03.239 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:03.239 INFO:teuthology.orchestra.run.smithi134.stdout: "op": 2104, 2024-09-10T10:05:03.239 INFO:teuthology.orchestra.run.smithi134.stdout: "op_r": 2104, 2024-09-10T10:05:03.239 INFO:teuthology.orchestra.run.smithi134.stdout: "op_w": 0, 2024-09-10T10:05:03.239 INFO:teuthology.orchestra.run.smithi134.stdout: "op_rmw": 0, 2024-09-10T10:05:03.239 INFO:teuthology.orchestra.run.smithi134.stdout: "op_pg": 0, 2024-09-10T10:05:03.239 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_stat": 1337, 2024-09-10T10:05:03.239 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_create": 0, 2024-09-10T10:05:03.239 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_read": 765, 2024-09-10T10:05:03.239 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_write": 0, 2024-09-10T10:05:03.239 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_writefull": 0, 2024-09-10T10:05:03.239 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_writesame": 0, 2024-09-10T10:05:03.240 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_append": 0, 2024-09-10T10:05:03.240 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_zero": 0, 2024-09-10T10:05:03.240 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_truncate": 0, 2024-09-10T10:05:03.240 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_delete": 0, 2024-09-10T10:05:03.240 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_mapext": 0, 2024-09-10T10:05:03.240 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_sparse_read": 0, 2024-09-10T10:05:03.240 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_clonerange": 0, 2024-09-10T10:05:03.240 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_getxattr": 1, 2024-09-10T10:05:03.240 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_setxattr": 0, 2024-09-10T10:05:03.240 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_cmpxattr": 0, 2024-09-10T10:05:03.240 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_rmxattr": 0, 2024-09-10T10:05:03.240 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_resetxattrs": 0, 2024-09-10T10:05:03.240 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_call": 0, 2024-09-10T10:05:03.240 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_watch": 0, 2024-09-10T10:05:03.240 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_notify": 0, 2024-09-10T10:05:03.241 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T10:05:03.241 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_pgls": 0, 2024-09-10T10:05:03.241 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_pgls_filter": 0, 2024-09-10T10:05:03.241 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_other": 0, 2024-09-10T10:05:03.241 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_active": 0, 2024-09-10T10:05:03.241 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_send": 0, 2024-09-10T10:05:03.241 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_resend": 0, 2024-09-10T10:05:03.241 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_ping": 0, 2024-09-10T10:05:03.241 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_active": 0, 2024-09-10T10:05:03.241 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_send": 0, 2024-09-10T10:05:03.241 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_resend": 0, 2024-09-10T10:05:03.241 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_active": 0, 2024-09-10T10:05:03.241 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_send": 0, 2024-09-10T10:05:03.241 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_resend": 0, 2024-09-10T10:05:03.241 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_active": 0, 2024-09-10T10:05:03.241 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_send": 0, 2024-09-10T10:05:03.242 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_resend": 0, 2024-09-10T10:05:03.242 INFO:teuthology.orchestra.run.smithi134.stdout: "command_active": 0, 2024-09-10T10:05:03.242 INFO:teuthology.orchestra.run.smithi134.stdout: "command_send": 0, 2024-09-10T10:05:03.242 INFO:teuthology.orchestra.run.smithi134.stdout: "command_resend": 0, 2024-09-10T10:05:03.242 INFO:teuthology.orchestra.run.smithi134.stdout: "map_epoch": 96, 2024-09-10T10:05:03.242 INFO:teuthology.orchestra.run.smithi134.stdout: "map_full": 0, 2024-09-10T10:05:03.242 INFO:teuthology.orchestra.run.smithi134.stdout: "map_inc": 2, 2024-09-10T10:05:03.242 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_sessions": 6, 2024-09-10T10:05:03.242 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_session_open": 6, 2024-09-10T10:05:03.242 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_session_close": 0, 2024-09-10T10:05:03.242 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_laggy": 0, 2024-09-10T10:05:03.242 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_wr": 0, 2024-09-10T10:05:03.242 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_rd": 2, 2024-09-10T10:05:03.242 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_del": 0 2024-09-10T10:05:03.242 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:03.242 INFO:teuthology.orchestra.run.smithi134.stdout: "oft": { 2024-09-10T10:05:03.243 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_objs": 0, 2024-09-10T10:05:03.243 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_kv_pairs": 0, 2024-09-10T10:05:03.243 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_updates": 0, 2024-09-10T10:05:03.243 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_removes": 0 2024-09-10T10:05:03.243 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:03.243 INFO:teuthology.orchestra.run.smithi134.stdout: "osd-slow-ops": { 2024-09-10T10:05:03.243 INFO:teuthology.orchestra.run.smithi134.stdout: "slow_ops_count": 0 2024-09-10T10:05:03.243 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:03.243 INFO:teuthology.orchestra.run.smithi134.stdout: "purge_queue": { 2024-09-10T10:05:03.243 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_ops": 0, 2024-09-10T10:05:03.243 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_ops_high_water": 0, 2024-09-10T10:05:03.243 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing": 0, 2024-09-10T10:05:03.243 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_high_water": 0, 2024-09-10T10:05:03.243 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executed_ops": 0, 2024-09-10T10:05:03.243 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executed": 0, 2024-09-10T10:05:03.243 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_item_in_journal": 0 2024-09-10T10:05:03.243 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:03.244 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T10:05:03.244 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-10T10:05:03.244 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 104857600, 2024-09-10T10:05:03.244 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-10T10:05:03.244 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 2344, 2024-09-10T10:05:03.244 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 8929636, 2024-09-10T10:05:03.244 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-10T10:05:03.244 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 2344, 2024-09-10T10:05:03.244 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-09-10T10:05:03.244 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-09-10T10:05:03.244 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 2344, 2024-09-10T10:05:03.244 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 8929636, 2024-09-10T10:05:03.244 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-10T10:05:03.244 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:05:03.244 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:05:03.244 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:05:03.245 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:05:03.245 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:03.245 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-objecter_bytes": { 2024-09-10T10:05:03.245 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-10T10:05:03.245 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 104857600, 2024-09-10T10:05:03.245 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-10T10:05:03.245 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-09-10T10:05:03.245 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-09-10T10:05:03.245 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-10T10:05:03.245 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-09-10T10:05:03.245 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 2104, 2024-09-10T10:05:03.245 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 7982414, 2024-09-10T10:05:03.245 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 2104, 2024-09-10T10:05:03.245 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 7982414, 2024-09-10T10:05:03.245 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-10T10:05:03.246 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:05:03.246 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:05:03.246 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:05:03.246 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:05:03.246 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:03.246 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-objecter_ops": { 2024-09-10T10:05:03.246 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-10T10:05:03.246 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 1024, 2024-09-10T10:05:03.246 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-10T10:05:03.246 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-09-10T10:05:03.246 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-09-10T10:05:03.246 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-10T10:05:03.246 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-09-10T10:05:03.246 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 2104, 2024-09-10T10:05:03.246 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 2104, 2024-09-10T10:05:03.246 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 2104, 2024-09-10T10:05:03.247 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 2104, 2024-09-10T10:05:03.247 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-10T10:05:03.247 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:05:03.247 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:05:03.247 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:05:03.247 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:05:03.247 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:03.247 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-write_buf_throttle": { 2024-09-10T10:05:03.247 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-10T10:05:03.247 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 3758096384, 2024-09-10T10:05:03.247 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-10T10:05:03.247 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-09-10T10:05:03.247 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-09-10T10:05:03.247 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-10T10:05:03.247 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-09-10T10:05:03.247 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-09-10T10:05:03.248 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-09-10T10:05:03.248 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 0, 2024-09-10T10:05:03.248 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 0, 2024-09-10T10:05:03.248 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-10T10:05:03.248 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:05:03.248 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:05:03.248 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:05:03.248 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:05:03.248 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:05:03.248 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-write_buf_throttle-0x55dd6c6fab40": { 2024-09-10T10:05:03.248 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-10T10:05:03.248 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 3758096384, 2024-09-10T10:05:03.248 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-10T10:05:03.248 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-09-10T10:05:03.248 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-09-10T10:05:03.248 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-10T10:05:03.249 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-09-10T10:05:03.249 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-09-10T10:05:03.249 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-09-10T10:05:03.249 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 0, 2024-09-10T10:05:03.249 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 0, 2024-09-10T10:05:03.249 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-10T10:05:03.249 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-10T10:05:03.249 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-10T10:05:03.249 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-10T10:05:03.249 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:05:03.249 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:05:03.249 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:05:03.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:05:03 smithi120 ceph-mon[34770]: pgmap v687: 129 pgs: 129 active+clean; 115 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T10:05:03.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:05:03 smithi077 ceph-mon[30319]: pgmap v687: 129 pgs: 129 active+clean; 115 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T10:05:04.059 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-09-10T10:05:04.059 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-09-10T10:05:04.059 INFO:tasks.check_counter:Daemon mds.l mds.dir_split=0 2024-09-10T10:05:04.059 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-10T10:05:04.069 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-10T10:05:04.069 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-09-10T10:05:04.070 DEBUG:teuthology.orchestra.run.smithi077:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:05:04.113 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-09-10T10:05:04.114 DEBUG:teuthology.orchestra.run.smithi077:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:05:04.193 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph osd blocklist ls 2024-09-10T10:05:04.602 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:05:05.113 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:05.106+0000 7f8e53000640 1 -- 172.21.15.77:0/4290970285 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e4c1047e0 msgr2=0x7f8e4c108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:05.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:05.106+0000 7f8e53000640 1 --2- 172.21.15.77:0/4290970285 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e4c1047e0 0x7f8e4c108830 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f8e38009900 tx=0x7f8e3802a410 comp rx=0 tx=0).stop 2024-09-10T10:05:05.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:05.106+0000 7f8e53000640 1 -- 172.21.15.77:0/4290970285 shutdown_connections 2024-09-10T10:05:05.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:05.106+0000 7f8e53000640 1 --2- 172.21.15.77:0/4290970285 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8e4c108f60 0x7f8e4c110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:05.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:05.106+0000 7f8e53000640 1 --2- 172.21.15.77:0/4290970285 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e4c1047e0 0x7f8e4c108830 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:05.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:05.106+0000 7f8e53000640 1 --2- 172.21.15.77:0/4290970285 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e4c103e30 0x7f8e4c104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:05.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:05.106+0000 7f8e53000640 1 -- 172.21.15.77:0/4290970285 >> 172.21.15.77:0/4290970285 conn(0x7f8e4c0ff8d0 msgr2=0x7f8e4c101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:05.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:05.106+0000 7f8e53000640 1 -- 172.21.15.77:0/4290970285 shutdown_connections 2024-09-10T10:05:05.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:05.106+0000 7f8e53000640 1 -- 172.21.15.77:0/4290970285 wait complete. 2024-09-10T10:05:05.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:05.108+0000 7f8e53000640 1 Processor -- start 2024-09-10T10:05:05.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:05.108+0000 7f8e53000640 1 -- start start 2024-09-10T10:05:05.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:05.108+0000 7f8e53000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8e4c103e30 0x7f8e4c19a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:05.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:05.108+0000 7f8e53000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e4c1047e0 0x7f8e4c19a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:05.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:05.108+0000 7f8e53000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e4c108f60 0x7f8e4c19ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:05.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:05.108+0000 7f8e53000640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8e4c113f80 con 0x7f8e4c108f60 2024-09-10T10:05:05.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:05.108+0000 7f8e53000640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8e4c113e00 con 0x7f8e4c103e30 2024-09-10T10:05:05.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:05.108+0000 7f8e53000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8e4c114100 con 0x7f8e4c1047e0 2024-09-10T10:05:05.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:05.109+0000 7f8e52600640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e4c108f60 0x7f8e4c19ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:05.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:05.109+0000 7f8e51200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e4c1047e0 0x7f8e4c19a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:05.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:05.109+0000 7f8e52600640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e4c108f60 0x7f8e4c19ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60200/0 (socket says 172.21.15.77:60200) 2024-09-10T10:05:05.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:05.109+0000 7f8e52600640 1 -- 172.21.15.77:0/843228188 learned_addr learned my addr 172.21.15.77:0/843228188 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:05:05.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:05.109+0000 7f8e51c00640 1 --2- 172.21.15.77:0/843228188 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8e4c103e30 0x7f8e4c19a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:05.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:05.109+0000 7f8e52600640 1 -- 172.21.15.77:0/843228188 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e4c1047e0 msgr2=0x7f8e4c19a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:05.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:05.109+0000 7f8e52600640 1 --2- 172.21.15.77:0/843228188 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e4c1047e0 0x7f8e4c19a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:05.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:05.109+0000 7f8e52600640 1 -- 172.21.15.77:0/843228188 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8e4c103e30 msgr2=0x7f8e4c19a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:05.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:05.109+0000 7f8e52600640 1 --2- 172.21.15.77:0/843228188 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8e4c103e30 0x7f8e4c19a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:05.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:05.109+0000 7f8e52600640 1 -- 172.21.15.77:0/843228188 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e4c19f400 con 0x7f8e4c108f60 2024-09-10T10:05:05.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:05.109+0000 7f8e52600640 1 --2- 172.21.15.77:0/843228188 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e4c108f60 0x7f8e4c19ed20 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f8e4400e970 tx=0x7f8e4400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:05.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:05.110+0000 7f8e4a000640 1 -- 172.21.15.77:0/843228188 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e4400ccc0 con 0x7f8e4c108f60 2024-09-10T10:05:05.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:05.110+0000 7f8e53000640 1 -- 172.21.15.77:0/843228188 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e4c19f6f0 con 0x7f8e4c108f60 2024-09-10T10:05:05.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:05.110+0000 7f8e4a000640 1 -- 172.21.15.77:0/843228188 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e4400ce60 con 0x7f8e4c108f60 2024-09-10T10:05:05.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:05.110+0000 7f8e4a000640 1 -- 172.21.15.77:0/843228188 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e44018630 con 0x7f8e4c108f60 2024-09-10T10:05:05.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:05.110+0000 7f8e53000640 1 -- 172.21.15.77:0/843228188 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e4c1a6fd0 con 0x7f8e4c108f60 2024-09-10T10:05:05.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:05.111+0000 7f8e53000640 1 -- 172.21.15.77:0/843228188 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e4c104210 con 0x7f8e4c108f60 2024-09-10T10:05:05.117 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:05.113+0000 7f8e4a000640 1 -- 172.21.15.77:0/843228188 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8e440187d0 con 0x7f8e4c108f60 2024-09-10T10:05:05.117 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:05.114+0000 7f8e4a000640 1 --2- 172.21.15.77:0/843228188 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f8e340a2b80 0x7f8e340a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:05.117 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:05.114+0000 7f8e4a000640 1 -- 172.21.15.77:0/843228188 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f8e440cf9c0 con 0x7f8e4c108f60 2024-09-10T10:05:05.117 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:05.114+0000 7f8e51c00640 1 --2- 172.21.15.77:0/843228188 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f8e340a2b80 0x7f8e340a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:05.117 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:05.115+0000 7f8e51c00640 1 --2- 172.21.15.77:0/843228188 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f8e340a2b80 0x7f8e340a5040 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f8e40005fd0 tx=0x7f8e40009680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:05.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:05.118+0000 7f8e4a000640 1 -- 172.21.15.77:0/843228188 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8e4409c4d0 con 0x7f8e4c108f60 2024-09-10T10:05:05.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:05.304+0000 7f8e53000640 1 -- 172.21.15.77:0/843228188 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f8e4c02d7c0 con 0x7f8e4c108f60 2024-09-10T10:05:05.306 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:05:05 smithi077 ceph-mon[30319]: pgmap v688: 129 pgs: 129 active+clean; 115 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-10T10:05:05.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:05.305+0000 7f8e4a000640 1 -- 172.21.15.77:0/843228188 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v96) ==== 81+0+590 (secure 0 0 0) 0x7f8e4409bdd0 con 0x7f8e4c108f60 2024-09-10T10:05:05.306 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:0/1697982655 2024-09-11T09:44:59.575663+0000 2024-09-10T10:05:05.306 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:6801/1527070442 2024-09-11T09:44:59.575663+0000 2024-09-10T10:05:05.307 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:0/3146688196 2024-09-11T09:44:32.185342+0000 2024-09-10T10:05:05.307 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:0/371773177 2024-09-11T09:44:59.575663+0000 2024-09-10T10:05:05.307 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:0/1925456910 2024-09-11T09:44:59.575663+0000 2024-09-10T10:05:05.307 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:0/138545832 2024-09-11T09:44:32.185342+0000 2024-09-10T10:05:05.307 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:6801/3235900503 2024-09-11T09:44:32.185342+0000 2024-09-10T10:05:05.307 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:6800/1527070442 2024-09-11T09:44:59.575663+0000 2024-09-10T10:05:05.307 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:6800/3235900503 2024-09-11T09:44:32.185342+0000 2024-09-10T10:05:05.307 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:0/1647092548 2024-09-11T09:44:32.185342+0000 2024-09-10T10:05:05.307 INFO:teuthology.orchestra.run.smithi077.stderr:listed 10 entries 2024-09-10T10:05:05.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:05.307+0000 7f8e53000640 1 -- 172.21.15.77:0/843228188 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f8e340a2b80 msgr2=0x7f8e340a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:05.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:05.307+0000 7f8e53000640 1 --2- 172.21.15.77:0/843228188 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f8e340a2b80 0x7f8e340a5040 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f8e40005fd0 tx=0x7f8e40009680 comp rx=0 tx=0).stop 2024-09-10T10:05:05.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:05.307+0000 7f8e53000640 1 -- 172.21.15.77:0/843228188 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e4c108f60 msgr2=0x7f8e4c19ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:05.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:05.307+0000 7f8e53000640 1 --2- 172.21.15.77:0/843228188 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e4c108f60 0x7f8e4c19ed20 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f8e4400e970 tx=0x7f8e4400ed70 comp rx=0 tx=0).stop 2024-09-10T10:05:05.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:05.307+0000 7f8e53000640 1 -- 172.21.15.77:0/843228188 shutdown_connections 2024-09-10T10:05:05.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:05.307+0000 7f8e53000640 1 --2- 172.21.15.77:0/843228188 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f8e340a2b80 0x7f8e340a5040 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:05.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:05.307+0000 7f8e53000640 1 --2- 172.21.15.77:0/843228188 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e4c108f60 0x7f8e4c19ed20 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:05.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:05.307+0000 7f8e53000640 1 --2- 172.21.15.77:0/843228188 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e4c1047e0 0x7f8e4c19a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:05.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:05.307+0000 7f8e53000640 1 --2- 172.21.15.77:0/843228188 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8e4c103e30 0x7f8e4c19a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:05.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:05.307+0000 7f8e53000640 1 -- 172.21.15.77:0/843228188 >> 172.21.15.77:0/843228188 conn(0x7f8e4c0ff8d0 msgr2=0x7f8e4c100c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:05.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:05.308+0000 7f8e53000640 1 -- 172.21.15.77:0/843228188 shutdown_connections 2024-09-10T10:05:05.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:05.308+0000 7f8e53000640 1 -- 172.21.15.77:0/843228188 wait complete. 2024-09-10T10:05:05.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:05:05 smithi134 ceph-mon[35674]: pgmap v688: 129 pgs: 129 active+clean; 115 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-10T10:05:05.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:05:05 smithi120 ceph-mon[34770]: pgmap v688: 129 pgs: 129 active+clean; 115 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-10T10:05:06.140 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T10:05:06.140 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:05:06.140 DEBUG:teuthology.orchestra.run.smithi077:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-10T10:05:06.263 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:05:06 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/843228188' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T10:05:06.267 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi077.front.sepia.ceph.com 2024-09-10T10:05:06.267 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:05:06.267 DEBUG:teuthology.orchestra.run.smithi077:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T10:05:06.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:05:06.350 DEBUG:teuthology.orchestra.run.smithi077:> ip netns list 2024-09-10T10:05:06.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:05:06 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/843228188' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T10:05:06.421 INFO:teuthology.orchestra.run.smithi077.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T10:05:06.422 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:05:06.422 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T10:05:06.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:05:06 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/843228188' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T10:05:06.495 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:05:06.495 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip link delete ceph-brx 2024-09-10T10:05:06.843 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-10T10:05:06.855 INFO:tasks.cephadm:Teardown begin 2024-09-10T10:05:06.856 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T10:05:06.856 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:05:07.229 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:05:07.367 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:05:07 smithi077 ceph-mon[30319]: pgmap v689: 129 pgs: 129 active+clean; 115 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T10:05:07.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:05:07 smithi134 ceph-mon[35674]: pgmap v689: 129 pgs: 129 active+clean; 115 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T10:05:07.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:05:07 smithi120 ceph-mon[34770]: pgmap v689: 129 pgs: 129 active+clean; 115 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T10:05:07.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:07.801+0000 7fda1cc00640 1 -- 172.21.15.77:0/1642500869 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fda181068b0 msgr2=0x7fda18106c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:07.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:07.801+0000 7fda1cc00640 1 --2- 172.21.15.77:0/1642500869 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fda181068b0 0x7fda18106c90 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fda08009930 tx=0x7fda0802a420 comp rx=0 tx=0).stop 2024-09-10T10:05:07.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:07.801+0000 7fda1cc00640 1 -- 172.21.15.77:0/1642500869 shutdown_connections 2024-09-10T10:05:07.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:07.801+0000 7fda1cc00640 1 --2- 172.21.15.77:0/1642500869 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fda18101250 0x7fda1810d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:07.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:07.801+0000 7fda1cc00640 1 --2- 172.21.15.77:0/1642500869 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda181008b0 0x7fda18100d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:07.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:07.801+0000 7fda1cc00640 1 --2- 172.21.15.77:0/1642500869 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fda181068b0 0x7fda18106c90 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:07.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:07.801+0000 7fda1cc00640 1 -- 172.21.15.77:0/1642500869 >> 172.21.15.77:0/1642500869 conn(0x7fda180fc5d0 msgr2=0x7fda180fe9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:07.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:07.801+0000 7fda1cc00640 1 -- 172.21.15.77:0/1642500869 shutdown_connections 2024-09-10T10:05:07.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:07.801+0000 7fda1cc00640 1 -- 172.21.15.77:0/1642500869 wait complete. 2024-09-10T10:05:07.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:07.802+0000 7fda1cc00640 1 Processor -- start 2024-09-10T10:05:07.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:07.802+0000 7fda1cc00640 1 -- start start 2024-09-10T10:05:07.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:07.803+0000 7fda1cc00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fda181008b0 0x7fda18198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:07.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:07.803+0000 7fda1cc00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda18101250 0x7fda18198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:07.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:07.803+0000 7fda1cc00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fda181068b0 0x7fda1819cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:07.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:07.803+0000 7fda1cc00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fda18111dc0 con 0x7fda181068b0 2024-09-10T10:05:07.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:07.803+0000 7fda1cc00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fda18111c40 con 0x7fda181008b0 2024-09-10T10:05:07.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:07.803+0000 7fda1cc00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fda18111f40 con 0x7fda18101250 2024-09-10T10:05:07.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:07.803+0000 7fda17e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fda181068b0 0x7fda1819cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:07.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:07.803+0000 7fda17e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fda181068b0 0x7fda1819cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60222/0 (socket says 172.21.15.77:60222) 2024-09-10T10:05:07.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:07.803+0000 7fda17e00640 1 -- 172.21.15.77:0/2275368667 learned_addr learned my addr 172.21.15.77:0/2275368667 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:05:07.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:07.803+0000 7fda0ea00640 1 --2- 172.21.15.77:0/2275368667 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda18101250 0x7fda18198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:07.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:07.803+0000 7fda17400640 1 --2- 172.21.15.77:0/2275368667 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fda181008b0 0x7fda18198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:07.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:07.804+0000 7fda17e00640 1 -- 172.21.15.77:0/2275368667 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda18101250 msgr2=0x7fda18198840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:07.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:07.804+0000 7fda17e00640 1 --2- 172.21.15.77:0/2275368667 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda18101250 0x7fda18198840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:07.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:07.804+0000 7fda17e00640 1 -- 172.21.15.77:0/2275368667 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fda181008b0 msgr2=0x7fda18198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:07.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:07.804+0000 7fda17e00640 1 --2- 172.21.15.77:0/2275368667 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fda181008b0 0x7fda18198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:07.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:07.804+0000 7fda17e00640 1 -- 172.21.15.77:0/2275368667 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda1819d2b0 con 0x7fda181068b0 2024-09-10T10:05:07.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:07.804+0000 7fda17e00640 1 --2- 172.21.15.77:0/2275368667 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fda181068b0 0x7fda1819cbd0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fda1000e970 tx=0x7fda1000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:07.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:07.804+0000 7fda14c00640 1 -- 172.21.15.77:0/2275368667 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda1000ccc0 con 0x7fda181068b0 2024-09-10T10:05:07.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:07.804+0000 7fda14c00640 1 -- 172.21.15.77:0/2275368667 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda1000ce60 con 0x7fda181068b0 2024-09-10T10:05:07.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:07.804+0000 7fda1cc00640 1 -- 172.21.15.77:0/2275368667 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda1819d5a0 con 0x7fda181068b0 2024-09-10T10:05:07.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:07.804+0000 7fda14c00640 1 -- 172.21.15.77:0/2275368667 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda10010490 con 0x7fda181068b0 2024-09-10T10:05:07.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:07.804+0000 7fda1cc00640 1 -- 172.21.15.77:0/2275368667 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda181a4e80 con 0x7fda181068b0 2024-09-10T10:05:07.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:07.805+0000 7fda1cc00640 1 -- 172.21.15.77:0/2275368667 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda18101ff0 con 0x7fda181068b0 2024-09-10T10:05:07.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:07.807+0000 7fda14c00640 1 -- 172.21.15.77:0/2275368667 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fda10005020 con 0x7fda181068b0 2024-09-10T10:05:07.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:07.808+0000 7fda14c00640 1 --2- 172.21.15.77:0/2275368667 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fd9fc0a2b80 0x7fd9fc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:07.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:07.809+0000 7fda14c00640 1 -- 172.21.15.77:0/2275368667 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fda100cf8b0 con 0x7fda181068b0 2024-09-10T10:05:07.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:07.809+0000 7fda17400640 1 --2- 172.21.15.77:0/2275368667 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fd9fc0a2b80 0x7fd9fc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:07.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:07.809+0000 7fda17400640 1 --2- 172.21.15.77:0/2275368667 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fd9fc0a2b80 0x7fd9fc0a5040 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fda08009900 tx=0x7fda080058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:07.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:07.812+0000 7fda14c00640 1 -- 172.21.15.77:0/2275368667 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fda1009c3c0 con 0x7fda181068b0 2024-09-10T10:05:08.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:08.033+0000 7fda1cc00640 1 -- 172.21.15.77:0/2275368667 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fda18100d10 con 0x7fda181068b0 2024-09-10T10:05:08.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:08.034+0000 7fda14c00640 1 -- 172.21.15.77:0/2275368667 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+11999 (secure 0 0 0) 0x7fda18100d10 con 0x7fda181068b0 2024-09-10T10:05:08.035 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T10:05:08.036 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":40,"btime":"2024-09-10T10:04:27:940036+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3159289675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3159289675},{"type":"v1","addr":"172.21.15.77:6835","nonce":3159289675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1951962341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1951962341},{"type":"v1","addr":"172.21.15.77:6837","nonce":1951962341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":40,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:52:23.368252+0000","modified":"2024-09-10T10:04:27.788564+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24563,"mds_1":24524,"mds_2":24520,"mds_3":24502,"mds_4":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14487":{"gid":14487,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/4291121888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":4291121888},{"type":"v1","addr":"172.21.15.77:6839","nonce":4291121888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14499":{"gid":14499,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6841/445284194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":445284194},{"type":"v1","addr":"172.21.15.77:6841","nonce":445284194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1005054423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1005054423},{"type":"v1","addr":"172.21.15.134:6833","nonce":1005054423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6833/3716307744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":3716307744},{"type":"v1","addr":"172.21.15.120:6833","nonce":3716307744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/1319892544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1319892544},{"type":"v1","addr":"172.21.15.120:6835","nonce":1319892544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.134:6835/3293933699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3293933699},{"type":"v1","addr":"172.21.15.134:6835","nonce":3293933699}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.134:6837/2461497091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2461497091},{"type":"v1","addr":"172.21.15.134:6837","nonce":2461497091}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":20,"addr":"172.21.15.134:6839/1965117680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1965117680},{"type":"v1","addr":"172.21.15.134:6839","nonce":1965117680}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":20,"addr":"172.21.15.120:6837/3033280709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3033280709},{"type":"v1","addr":"172.21.15.120:6837","nonce":3033280709}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.120:6839/1777710157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1777710157},{"type":"v1","addr":"172.21.15.120:6839","nonce":1777710157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24563,"qdb_cluster":[24563,24524,24520,24502,24496]},"id":1}]} 2024-09-10T10:05:08.036 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 40 2024-09-10T10:05:08.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:08.037+0000 7fda1cc00640 1 -- 172.21.15.77:0/2275368667 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fd9fc0a2b80 msgr2=0x7fd9fc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:08.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:08.037+0000 7fda1cc00640 1 --2- 172.21.15.77:0/2275368667 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fd9fc0a2b80 0x7fd9fc0a5040 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fda08009900 tx=0x7fda080058d0 comp rx=0 tx=0).stop 2024-09-10T10:05:08.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:08.037+0000 7fda1cc00640 1 -- 172.21.15.77:0/2275368667 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fda181068b0 msgr2=0x7fda1819cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:08.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:08.037+0000 7fda1cc00640 1 --2- 172.21.15.77:0/2275368667 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fda181068b0 0x7fda1819cbd0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fda1000e970 tx=0x7fda1000ed70 comp rx=0 tx=0).stop 2024-09-10T10:05:08.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:08.037+0000 7fda1cc00640 1 -- 172.21.15.77:0/2275368667 shutdown_connections 2024-09-10T10:05:08.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:08.037+0000 7fda1cc00640 1 --2- 172.21.15.77:0/2275368667 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fd9fc0a2b80 0x7fd9fc0a5040 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:08.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:08.037+0000 7fda1cc00640 1 --2- 172.21.15.77:0/2275368667 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fda181068b0 0x7fda1819cbd0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:08.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:08.037+0000 7fda1cc00640 1 --2- 172.21.15.77:0/2275368667 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda18101250 0x7fda18198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:08.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:08.037+0000 7fda1cc00640 1 --2- 172.21.15.77:0/2275368667 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fda181008b0 0x7fda18198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:08.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:08.037+0000 7fda1cc00640 1 -- 172.21.15.77:0/2275368667 >> 172.21.15.77:0/2275368667 conn(0x7fda180fc5d0 msgr2=0x7fda180fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:08.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:08.037+0000 7fda1cc00640 1 -- 172.21.15.77:0/2275368667 shutdown_connections 2024-09-10T10:05:08.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:08.037+0000 7fda1cc00640 1 -- 172.21.15.77:0/2275368667 wait complete. 2024-09-10T10:05:08.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:05:08 smithi134 ceph-mon[35674]: pgmap v690: 129 pgs: 129 active+clean; 115 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-10T10:05:08.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:05:08 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/2275368667' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:05:08.393 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:05:08 smithi077 ceph-mon[30319]: pgmap v690: 129 pgs: 129 active+clean; 115 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-10T10:05:08.393 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:05:08 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/2275368667' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:05:08.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:05:08 smithi120 ceph-mon[34770]: pgmap v690: 129 pgs: 129 active+clean; 115 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-10T10:05:08.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:05:08 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/2275368667' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:05:08.920 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:05:09.289 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:05:09.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:09.835+0000 7fdf2fc00640 1 -- 172.21.15.77:0/2665071011 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdf28101de0 msgr2=0x7fdf281021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:09.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:09.835+0000 7fdf2fc00640 1 --2- 172.21.15.77:0/2665071011 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdf28101de0 0x7fdf281021c0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c009900 tx=0x7fdf1c02a410 comp rx=0 tx=0).stop 2024-09-10T10:05:09.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:09.835+0000 7fdf2fc00640 1 -- 172.21.15.77:0/2665071011 shutdown_connections 2024-09-10T10:05:09.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:09.835+0000 7fdf2fc00640 1 --2- 172.21.15.77:0/2665071011 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdf28106f10 0x7fdf2810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:09.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:09.835+0000 7fdf2fc00640 1 --2- 172.21.15.77:0/2665071011 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdf28102790 0x7fdf281067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:09.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:09.835+0000 7fdf2fc00640 1 --2- 172.21.15.77:0/2665071011 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdf28101de0 0x7fdf281021c0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:09.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:09.835+0000 7fdf2fc00640 1 -- 172.21.15.77:0/2665071011 >> 172.21.15.77:0/2665071011 conn(0x7fdf280fd8c0 msgr2=0x7fdf280ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:09.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:09.835+0000 7fdf2fc00640 1 -- 172.21.15.77:0/2665071011 shutdown_connections 2024-09-10T10:05:09.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:09.835+0000 7fdf2fc00640 1 -- 172.21.15.77:0/2665071011 wait complete. 2024-09-10T10:05:09.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:09.836+0000 7fdf2fc00640 1 Processor -- start 2024-09-10T10:05:09.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:09.836+0000 7fdf2fc00640 1 -- start start 2024-09-10T10:05:09.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:09.837+0000 7fdf2fc00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdf28101de0 0x7fdf2819a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:09.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:09.837+0000 7fdf2fc00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdf28102790 0x7fdf2819ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:09.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:09.837+0000 7fdf2fc00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdf28106f10 0x7fdf2819ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:09.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:09.837+0000 7fdf2fc00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fdf28111f70 con 0x7fdf28102790 2024-09-10T10:05:09.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:09.837+0000 7fdf2fc00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdf28111df0 con 0x7fdf28101de0 2024-09-10T10:05:09.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:09.837+0000 7fdf2fc00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdf281120f0 con 0x7fdf28106f10 2024-09-10T10:05:09.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:09.837+0000 7fdf27e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdf28102790 0x7fdf2819ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:09.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:09.837+0000 7fdf2d200640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdf28101de0 0x7fdf2819a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:09.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:09.837+0000 7fdf27e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdf28102790 0x7fdf2819ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60238/0 (socket says 172.21.15.77:60238) 2024-09-10T10:05:09.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:09.837+0000 7fdf27e00640 1 -- 172.21.15.77:0/919310482 learned_addr learned my addr 172.21.15.77:0/919310482 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:05:09.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:09.837+0000 7fdf2dc00640 1 --2- 172.21.15.77:0/919310482 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdf28106f10 0x7fdf2819ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:09.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:09.837+0000 7fdf2d200640 1 -- 172.21.15.77:0/919310482 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdf28106f10 msgr2=0x7fdf2819ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:09.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:09.837+0000 7fdf2d200640 1 --2- 172.21.15.77:0/919310482 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdf28106f10 0x7fdf2819ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:09.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:09.837+0000 7fdf2d200640 1 -- 172.21.15.77:0/919310482 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdf28102790 msgr2=0x7fdf2819ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:09.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:09.837+0000 7fdf2d200640 1 --2- 172.21.15.77:0/919310482 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdf28102790 0x7fdf2819ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:09.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:09.837+0000 7fdf2d200640 1 -- 172.21.15.77:0/919310482 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf2819f600 con 0x7fdf28101de0 2024-09-10T10:05:09.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:09.837+0000 7fdf27e00640 1 --2- 172.21.15.77:0/919310482 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdf28102790 0x7fdf2819ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:05:09.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:09.837+0000 7fdf2dc00640 1 --2- 172.21.15.77:0/919310482 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdf28106f10 0x7fdf2819ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:05:09.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:09.838+0000 7fdf2d200640 1 --2- 172.21.15.77:0/919310482 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdf28101de0 0x7fdf2819a650 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c005940 tx=0x7fdf1c0040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:09.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:09.838+0000 7fdf25600640 1 -- 172.21.15.77:0/919310482 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c03f030 con 0x7fdf28101de0 2024-09-10T10:05:09.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:09.838+0000 7fdf2fc00640 1 -- 172.21.15.77:0/919310482 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf2819f890 con 0x7fdf28101de0 2024-09-10T10:05:09.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:09.838+0000 7fdf25600640 1 -- 172.21.15.77:0/919310482 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf1c0044c0 con 0x7fdf28101de0 2024-09-10T10:05:09.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:09.838+0000 7fdf25600640 1 -- 172.21.15.77:0/919310482 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c035070 con 0x7fdf28101de0 2024-09-10T10:05:09.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:09.838+0000 7fdf2fc00640 1 -- 172.21.15.77:0/919310482 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf281a71d0 con 0x7fdf28101de0 2024-09-10T10:05:09.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:09.839+0000 7fdf2fc00640 1 -- 172.21.15.77:0/919310482 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf281021c0 con 0x7fdf28101de0 2024-09-10T10:05:09.842 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:09.841+0000 7fdf25600640 1 -- 172.21.15.77:0/919310482 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fdf1c042030 con 0x7fdf28101de0 2024-09-10T10:05:09.844 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:09.843+0000 7fdf25600640 1 --2- 172.21.15.77:0/919310482 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fdf100a2b80 0x7fdf100a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:09.844 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:09.843+0000 7fdf25600640 1 -- 172.21.15.77:0/919310482 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7fdf1c0efac0 con 0x7fdf28101de0 2024-09-10T10:05:09.844 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:09.843+0000 7fdf27e00640 1 --2- 172.21.15.77:0/919310482 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fdf100a2b80 0x7fdf100a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:09.844 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:09.843+0000 7fdf27e00640 1 --2- 172.21.15.77:0/919310482 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fdf100a2b80 0x7fdf100a5040 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fdf2810ee50 tx=0x7fdf14005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:09.845 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:09.845+0000 7fdf25600640 1 -- 172.21.15.77:0/919310482 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdf1c0bc5d0 con 0x7fdf28101de0 2024-09-10T10:05:10.058 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:10.057+0000 7fdf2fc00640 1 -- 172.21.15.77:0/919310482 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdf2819b9b0 con 0x7fdf28101de0 2024-09-10T10:05:10.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:10.058+0000 7fdf25600640 1 -- 172.21.15.77:0/919310482 <== mon.2 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+11999 (secure 0 0 0) 0x7fdf1c0bbed0 con 0x7fdf28101de0 2024-09-10T10:05:10.060 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T10:05:10.060 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":40,"btime":"2024-09-10T10:04:27:940036+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3159289675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3159289675},{"type":"v1","addr":"172.21.15.77:6835","nonce":3159289675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1951962341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1951962341},{"type":"v1","addr":"172.21.15.77:6837","nonce":1951962341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":40,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:52:23.368252+0000","modified":"2024-09-10T10:04:27.788564+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24563,"mds_1":24524,"mds_2":24520,"mds_3":24502,"mds_4":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14487":{"gid":14487,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/4291121888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":4291121888},{"type":"v1","addr":"172.21.15.77:6839","nonce":4291121888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14499":{"gid":14499,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6841/445284194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":445284194},{"type":"v1","addr":"172.21.15.77:6841","nonce":445284194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1005054423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1005054423},{"type":"v1","addr":"172.21.15.134:6833","nonce":1005054423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6833/3716307744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":3716307744},{"type":"v1","addr":"172.21.15.120:6833","nonce":3716307744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/1319892544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1319892544},{"type":"v1","addr":"172.21.15.120:6835","nonce":1319892544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.134:6835/3293933699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3293933699},{"type":"v1","addr":"172.21.15.134:6835","nonce":3293933699}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.134:6837/2461497091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2461497091},{"type":"v1","addr":"172.21.15.134:6837","nonce":2461497091}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":20,"addr":"172.21.15.134:6839/1965117680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1965117680},{"type":"v1","addr":"172.21.15.134:6839","nonce":1965117680}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":20,"addr":"172.21.15.120:6837/3033280709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3033280709},{"type":"v1","addr":"172.21.15.120:6837","nonce":3033280709}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.120:6839/1777710157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1777710157},{"type":"v1","addr":"172.21.15.120:6839","nonce":1777710157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24563,"qdb_cluster":[24563,24524,24520,24502,24496]},"id":1}]} 2024-09-10T10:05:10.060 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 40 2024-09-10T10:05:10.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:10.062+0000 7fdf2fc00640 1 -- 172.21.15.77:0/919310482 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fdf100a2b80 msgr2=0x7fdf100a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:10.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:10.062+0000 7fdf2fc00640 1 --2- 172.21.15.77:0/919310482 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fdf100a2b80 0x7fdf100a5040 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fdf2810ee50 tx=0x7fdf14005f90 comp rx=0 tx=0).stop 2024-09-10T10:05:10.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:10.062+0000 7fdf2fc00640 1 -- 172.21.15.77:0/919310482 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdf28101de0 msgr2=0x7fdf2819a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:10.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:10.062+0000 7fdf2fc00640 1 --2- 172.21.15.77:0/919310482 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdf28101de0 0x7fdf2819a650 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c005940 tx=0x7fdf1c0040c0 comp rx=0 tx=0).stop 2024-09-10T10:05:10.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:10.062+0000 7fdf2fc00640 1 -- 172.21.15.77:0/919310482 shutdown_connections 2024-09-10T10:05:10.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:10.062+0000 7fdf2fc00640 1 --2- 172.21.15.77:0/919310482 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fdf100a2b80 0x7fdf100a5040 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:10.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:10.062+0000 7fdf2fc00640 1 --2- 172.21.15.77:0/919310482 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdf28106f10 0x7fdf2819ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:10.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:10.062+0000 7fdf2fc00640 1 --2- 172.21.15.77:0/919310482 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdf28102790 0x7fdf2819ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:10.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:10.062+0000 7fdf2fc00640 1 --2- 172.21.15.77:0/919310482 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdf28101de0 0x7fdf2819a650 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:10.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:10.062+0000 7fdf2fc00640 1 -- 172.21.15.77:0/919310482 >> 172.21.15.77:0/919310482 conn(0x7fdf280fd8c0 msgr2=0x7fdf280ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:10.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:10.062+0000 7fdf2fc00640 1 -- 172.21.15.77:0/919310482 shutdown_connections 2024-09-10T10:05:10.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:10.062+0000 7fdf2fc00640 1 -- 172.21.15.77:0/919310482 wait complete. 2024-09-10T10:05:10.950 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T10:05:11.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:05:10 smithi077 ceph-mon[30319]: pgmap v691: 129 pgs: 129 active+clean; 115 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-09-10T10:05:11.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:05:10 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/919310482' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:05:11.317 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:05:11.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:05:10 smithi134 ceph-mon[35674]: pgmap v691: 129 pgs: 129 active+clean; 115 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-09-10T10:05:11.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:05:10 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/919310482' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:05:11.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:05:10 smithi120 ceph-mon[34770]: pgmap v691: 129 pgs: 129 active+clean; 115 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-09-10T10:05:11.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:05:10 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/919310482' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:05:11.848 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:11.847+0000 7f3959c00640 1 -- 172.21.15.77:0/3644342370 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3954103e30 msgr2=0x7f3954104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:11.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:11.847+0000 7f3959c00640 1 --2- 172.21.15.77:0/3644342370 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3954103e30 0x7f3954104210 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f3944009900 tx=0x7f394402a410 comp rx=0 tx=0).stop 2024-09-10T10:05:11.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:11.847+0000 7f3959c00640 1 -- 172.21.15.77:0/3644342370 shutdown_connections 2024-09-10T10:05:11.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:11.847+0000 7f3959c00640 1 --2- 172.21.15.77:0/3644342370 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3954108f60 0x7f3954110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:11.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:11.847+0000 7f3959c00640 1 --2- 172.21.15.77:0/3644342370 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39541047e0 0x7f3954108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:11.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:11.847+0000 7f3959c00640 1 --2- 172.21.15.77:0/3644342370 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3954103e30 0x7f3954104210 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:11.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:11.847+0000 7f3959c00640 1 -- 172.21.15.77:0/3644342370 >> 172.21.15.77:0/3644342370 conn(0x7f39540ff910 msgr2=0x7f3954101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:11.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:11.847+0000 7f3959c00640 1 -- 172.21.15.77:0/3644342370 shutdown_connections 2024-09-10T10:05:11.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:11.847+0000 7f3959c00640 1 -- 172.21.15.77:0/3644342370 wait complete. 2024-09-10T10:05:11.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:11.848+0000 7f3959c00640 1 Processor -- start 2024-09-10T10:05:11.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:11.848+0000 7f3959c00640 1 -- start start 2024-09-10T10:05:11.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:11.848+0000 7f3959c00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3954103e30 0x7f395419a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:11.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:11.849+0000 7f3959c00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f39541047e0 0x7f395419a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:11.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:11.849+0000 7f3959c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3954108f60 0x7f395419ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:11.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:11.849+0000 7f3959c00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3954113f80 con 0x7f3954103e30 2024-09-10T10:05:11.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:11.849+0000 7f3959c00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3954113e00 con 0x7f39541047e0 2024-09-10T10:05:11.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:11.849+0000 7f3959c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3954114100 con 0x7f3954108f60 2024-09-10T10:05:11.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:11.849+0000 7f3953e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3954103e30 0x7f395419a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:11.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:11.849+0000 7f3953e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3954103e30 0x7f395419a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60260/0 (socket says 172.21.15.77:60260) 2024-09-10T10:05:11.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:11.849+0000 7f3953e00640 1 -- 172.21.15.77:0/3544292278 learned_addr learned my addr 172.21.15.77:0/3544292278 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:05:11.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:11.849+0000 7f394b400640 1 --2- 172.21.15.77:0/3544292278 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f39541047e0 0x7f395419a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:11.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:11.849+0000 7f3959200640 1 --2- 172.21.15.77:0/3544292278 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3954108f60 0x7f395419ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:11.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:11.849+0000 7f3953e00640 1 -- 172.21.15.77:0/3544292278 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3954108f60 msgr2=0x7f395419ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:11.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:11.849+0000 7f3953e00640 1 --2- 172.21.15.77:0/3544292278 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3954108f60 0x7f395419ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:11.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:11.849+0000 7f3953e00640 1 -- 172.21.15.77:0/3544292278 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f39541047e0 msgr2=0x7f395419a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:11.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:11.849+0000 7f3953e00640 1 --2- 172.21.15.77:0/3544292278 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f39541047e0 0x7f395419a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:11.852 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:11.849+0000 7f3953e00640 1 -- 172.21.15.77:0/3544292278 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f395419f440 con 0x7f3954103e30 2024-09-10T10:05:11.852 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:11.850+0000 7f3953e00640 1 --2- 172.21.15.77:0/3544292278 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3954103e30 0x7f395419a490 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f3944006010 tx=0x7f3944004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:11.852 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:11.850+0000 7f3951600640 1 -- 172.21.15.77:0/3544292278 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f394403f030 con 0x7f3954103e30 2024-09-10T10:05:11.852 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:11.850+0000 7f3951600640 1 -- 172.21.15.77:0/3544292278 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3944004440 con 0x7f3954103e30 2024-09-10T10:05:11.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:11.850+0000 7f3951600640 1 -- 172.21.15.77:0/3544292278 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3944035070 con 0x7f3954103e30 2024-09-10T10:05:11.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:11.850+0000 7f3959c00640 1 -- 172.21.15.77:0/3544292278 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f395419f6d0 con 0x7f3954103e30 2024-09-10T10:05:11.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:11.850+0000 7f3959c00640 1 -- 172.21.15.77:0/3544292278 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f395419fb60 con 0x7f3954103e30 2024-09-10T10:05:11.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:11.851+0000 7f3959c00640 1 -- 172.21.15.77:0/3544292278 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3954105310 con 0x7f3954103e30 2024-09-10T10:05:11.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:11.853+0000 7f3951600640 1 -- 172.21.15.77:0/3544292278 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3944042030 con 0x7f3954103e30 2024-09-10T10:05:11.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:11.855+0000 7f3951600640 1 --2- 172.21.15.77:0/3544292278 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f39380a2b80 0x7f39380a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:11.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:11.856+0000 7f3951600640 1 -- 172.21.15.77:0/3544292278 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7f39440efe20 con 0x7f3954103e30 2024-09-10T10:05:11.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:11.856+0000 7f394b400640 1 --2- 172.21.15.77:0/3544292278 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f39380a2b80 0x7f39380a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:11.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:11.856+0000 7f394b400640 1 --2- 172.21.15.77:0/3544292278 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f39380a2b80 0x7f39380a5040 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f393c0095f0 tx=0x7f393c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:11.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:11.858+0000 7f3951600640 1 -- 172.21.15.77:0/3544292278 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3944049030 con 0x7f3954103e30 2024-09-10T10:05:12.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:12.037+0000 7f3959c00640 1 -- 172.21.15.77:0/3544292278 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f395410c140 con 0x7f3954103e30 2024-09-10T10:05:12.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:12.038+0000 7f3951600640 1 -- 172.21.15.77:0/3544292278 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+24941 (secure 0 0 0) 0x7f39440bc9f0 con 0x7f3954103e30 2024-09-10T10:05:12.040 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-10T10:05:12.040 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":96,"fsid":"14a55226-6f59-11ef-bcea-c7b262605968","created":"2024-09-10T09:44:07.359542+0000","modified":"2024-09-10T09:54:00.837074+0000","last_up_change":"2024-09-10T09:51:26.731971+0000","last_in_change":"2024-09-10T09:51:03.857873+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:47:24.174660+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T09:52:18.766738+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T09:52:21.284745+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T09:52:29.846594+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8d46f634-114c-4926-afdc-f8fa37e40336","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":1509649079},{"type":"v1","addr":"172.21.15.77:6803","nonce":1509649079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":1509649079},{"type":"v1","addr":"172.21.15.77:6805","nonce":1509649079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":1509649079},{"type":"v1","addr":"172.21.15.77:6809","nonce":1509649079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":1509649079},{"type":"v1","addr":"172.21.15.77:6807","nonce":1509649079}]},"public_addr":"172.21.15.77:6803/1509649079","cluster_addr":"172.21.15.77:6805/1509649079","heartbeat_back_addr":"172.21.15.77:6809/1509649079","heartbeat_front_addr":"172.21.15.77:6807/1509649079","state":["exists","up"]},{"osd":1,"uuid":"b6570764-736a-49e4-877a-1965e5812d1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":4243920698},{"type":"v1","addr":"172.21.15.120:6801","nonce":4243920698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":4243920698},{"type":"v1","addr":"172.21.15.120:6803","nonce":4243920698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":4243920698},{"type":"v1","addr":"172.21.15.120:6807","nonce":4243920698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":4243920698},{"type":"v1","addr":"172.21.15.120:6805","nonce":4243920698}]},"public_addr":"172.21.15.120:6801/4243920698","cluster_addr":"172.21.15.120:6803/4243920698","heartbeat_back_addr":"172.21.15.120:6807/4243920698","heartbeat_front_addr":"172.21.15.120:6805/4243920698","state":["exists","up"]},{"osd":2,"uuid":"5e83fe13-d1ee-483e-a575-e8041ec836f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":3280466823},{"type":"v1","addr":"172.21.15.134:6801","nonce":3280466823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":3280466823},{"type":"v1","addr":"172.21.15.134:6803","nonce":3280466823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":3280466823},{"type":"v1","addr":"172.21.15.134:6807","nonce":3280466823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":3280466823},{"type":"v1","addr":"172.21.15.134:6805","nonce":3280466823}]},"public_addr":"172.21.15.134:6801/3280466823","cluster_addr":"172.21.15.134:6803/3280466823","heartbeat_back_addr":"172.21.15.134:6807/3280466823","heartbeat_front_addr":"172.21.15.134:6805/3280466823","state":["exists","up"]},{"osd":3,"uuid":"5affc6a7-1cae-48fe-b7db-befc2fcafecb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":1629416786},{"type":"v1","addr":"172.21.15.77:6811","nonce":1629416786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":1629416786},{"type":"v1","addr":"172.21.15.77:6813","nonce":1629416786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":1629416786},{"type":"v1","addr":"172.21.15.77:6817","nonce":1629416786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":1629416786},{"type":"v1","addr":"172.21.15.77:6815","nonce":1629416786}]},"public_addr":"172.21.15.77:6811/1629416786","cluster_addr":"172.21.15.77:6813/1629416786","heartbeat_back_addr":"172.21.15.77:6817/1629416786","heartbeat_front_addr":"172.21.15.77:6815/1629416786","state":["exists","up"]},{"osd":4,"uuid":"c826978f-40b0-43ae-adbb-5c93b32a68ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":3162658038},{"type":"v1","addr":"172.21.15.120:6809","nonce":3162658038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":3162658038},{"type":"v1","addr":"172.21.15.120:6811","nonce":3162658038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":3162658038},{"type":"v1","addr":"172.21.15.120:6815","nonce":3162658038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":3162658038},{"type":"v1","addr":"172.21.15.120:6813","nonce":3162658038}]},"public_addr":"172.21.15.120:6809/3162658038","cluster_addr":"172.21.15.120:6811/3162658038","heartbeat_back_addr":"172.21.15.120:6815/3162658038","heartbeat_front_addr":"172.21.15.120:6813/3162658038","state":["exists","up"]},{"osd":5,"uuid":"be51c7fb-2765-46a3-9861-c950f63de148","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":4081444621},{"type":"v1","addr":"172.21.15.134:6809","nonce":4081444621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":4081444621},{"type":"v1","addr":"172.21.15.134:6811","nonce":4081444621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":4081444621},{"type":"v1","addr":"172.21.15.134:6815","nonce":4081444621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":4081444621},{"type":"v1","addr":"172.21.15.134:6813","nonce":4081444621}]},"public_addr":"172.21.15.134:6809/4081444621","cluster_addr":"172.21.15.134:6811/4081444621","heartbeat_back_addr":"172.21.15.134:6815/4081444621","heartbeat_front_addr":"172.21.15.134:6813/4081444621","state":["exists","up"]},{"osd":6,"uuid":"4eb403ab-473b-41cc-803d-18c908f768a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":879565639},{"type":"v1","addr":"172.21.15.77:6819","nonce":879565639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":879565639},{"type":"v1","addr":"172.21.15.77:6821","nonce":879565639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":879565639},{"type":"v1","addr":"172.21.15.77:6825","nonce":879565639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":879565639},{"type":"v1","addr":"172.21.15.77:6823","nonce":879565639}]},"public_addr":"172.21.15.77:6819/879565639","cluster_addr":"172.21.15.77:6821/879565639","heartbeat_back_addr":"172.21.15.77:6825/879565639","heartbeat_front_addr":"172.21.15.77:6823/879565639","state":["exists","up"]},{"osd":7,"uuid":"ba982bd0-c236-456c-9889-61087d79c1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":2688667400},{"type":"v1","addr":"172.21.15.120:6817","nonce":2688667400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":2688667400},{"type":"v1","addr":"172.21.15.120:6819","nonce":2688667400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":2688667400},{"type":"v1","addr":"172.21.15.120:6823","nonce":2688667400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2688667400},{"type":"v1","addr":"172.21.15.120:6821","nonce":2688667400}]},"public_addr":"172.21.15.120:6817/2688667400","cluster_addr":"172.21.15.120:6819/2688667400","heartbeat_back_addr":"172.21.15.120:6823/2688667400","heartbeat_front_addr":"172.21.15.120:6821/2688667400","state":["exists","up"]},{"osd":8,"uuid":"f2e8ca43-57c9-4933-b5f7-73f4d1b3e08b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2081686544},{"type":"v1","addr":"172.21.15.134:6817","nonce":2081686544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2081686544},{"type":"v1","addr":"172.21.15.134:6819","nonce":2081686544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2081686544},{"type":"v1","addr":"172.21.15.134:6823","nonce":2081686544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2081686544},{"type":"v1","addr":"172.21.15.134:6821","nonce":2081686544}]},"public_addr":"172.21.15.134:6817/2081686544","cluster_addr":"172.21.15.134:6819/2081686544","heartbeat_back_addr":"172.21.15.134:6823/2081686544","heartbeat_front_addr":"172.21.15.134:6821/2081686544","state":["exists","up"]},{"osd":9,"uuid":"9486f8b5-0614-4208-87ec-117741f6b622","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":1210619387},{"type":"v1","addr":"172.21.15.77:6827","nonce":1210619387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":1210619387},{"type":"v1","addr":"172.21.15.77:6829","nonce":1210619387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":1210619387},{"type":"v1","addr":"172.21.15.77:6833","nonce":1210619387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":1210619387},{"type":"v1","addr":"172.21.15.77:6831","nonce":1210619387}]},"public_addr":"172.21.15.77:6827/1210619387","cluster_addr":"172.21.15.77:6829/1210619387","heartbeat_back_addr":"172.21.15.77:6833/1210619387","heartbeat_front_addr":"172.21.15.77:6831/1210619387","state":["exists","up"]},{"osd":10,"uuid":"8e8192b0-1398-4d4d-9756-762b2f874d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":754733885},{"type":"v1","addr":"172.21.15.120:6825","nonce":754733885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":754733885},{"type":"v1","addr":"172.21.15.120:6827","nonce":754733885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":754733885},{"type":"v1","addr":"172.21.15.120:6831","nonce":754733885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":754733885},{"type":"v1","addr":"172.21.15.120:6829","nonce":754733885}]},"public_addr":"172.21.15.120:6825/754733885","cluster_addr":"172.21.15.120:6827/754733885","heartbeat_back_addr":"172.21.15.120:6831/754733885","heartbeat_front_addr":"172.21.15.120:6829/754733885","state":["exists","up"]},{"osd":11,"uuid":"85c1ccd4-5147-4995-896e-fe977f73f749","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1602213143},{"type":"v1","addr":"172.21.15.134:6825","nonce":1602213143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1602213143},{"type":"v1","addr":"172.21.15.134:6827","nonce":1602213143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1602213143},{"type":"v1","addr":"172.21.15.134:6831","nonce":1602213143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1602213143},{"type":"v1","addr":"172.21.15.134:6829","nonce":1602213143}]},"public_addr":"172.21.15.134:6825/1602213143","cluster_addr":"172.21.15.134:6827/1602213143","heartbeat_back_addr":"172.21.15.134:6831/1602213143","heartbeat_front_addr":"172.21.15.134:6829/1602213143","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:46:30.293451+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:46:56.554801+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:47:21.476225+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:47:49.737332+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:48:16.092181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:48:41.240815+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:49:11.683950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:49:38.859255+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:50:04.105401+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:50:33.847810+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:50:59.901797+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:51:25.241507+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.2","mappings":[{"from":7,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":7}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":5,"to":8}]},{"pgid":"4.6","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.77:0/1697982655":"2024-09-11T09:44:59.575663+0000","172.21.15.77:6801/1527070442":"2024-09-11T09:44:59.575663+0000","172.21.15.77:0/3146688196":"2024-09-11T09:44:32.185342+0000","172.21.15.77:0/371773177":"2024-09-11T09:44:59.575663+0000","172.21.15.77:0/1925456910":"2024-09-11T09:44:59.575663+0000","172.21.15.77:0/138545832":"2024-09-11T09:44:32.185342+0000","172.21.15.77:6801/3235900503":"2024-09-11T09:44:32.185342+0000","172.21.15.77:6800/1527070442":"2024-09-11T09:44:59.575663+0000","172.21.15.77:6800/3235900503":"2024-09-11T09:44:32.185342+0000","172.21.15.77:0/1647092548":"2024-09-11T09:44:32.185342+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:05:12.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:12.041+0000 7f3959c00640 1 -- 172.21.15.77:0/3544292278 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f39380a2b80 msgr2=0x7f39380a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:12.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:12.041+0000 7f3959c00640 1 --2- 172.21.15.77:0/3544292278 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f39380a2b80 0x7f39380a5040 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f393c0095f0 tx=0x7f393c005f90 comp rx=0 tx=0).stop 2024-09-10T10:05:12.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:12.041+0000 7f3959c00640 1 -- 172.21.15.77:0/3544292278 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3954103e30 msgr2=0x7f395419a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:12.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:12.041+0000 7f3959c00640 1 --2- 172.21.15.77:0/3544292278 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3954103e30 0x7f395419a490 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f3944006010 tx=0x7f3944004040 comp rx=0 tx=0).stop 2024-09-10T10:05:12.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:12.041+0000 7f3959c00640 1 -- 172.21.15.77:0/3544292278 shutdown_connections 2024-09-10T10:05:12.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:12.041+0000 7f3959c00640 1 --2- 172.21.15.77:0/3544292278 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f39380a2b80 0x7f39380a5040 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:12.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:12.041+0000 7f3959c00640 1 --2- 172.21.15.77:0/3544292278 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3954108f60 0x7f395419ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:12.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:12.041+0000 7f3959c00640 1 --2- 172.21.15.77:0/3544292278 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f39541047e0 0x7f395419a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:12.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:12.041+0000 7f3959c00640 1 --2- 172.21.15.77:0/3544292278 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3954103e30 0x7f395419a490 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:12.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:12.041+0000 7f3959c00640 1 -- 172.21.15.77:0/3544292278 >> 172.21.15.77:0/3544292278 conn(0x7f39540ff910 msgr2=0x7f3954100ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:12.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:12.041+0000 7f3959c00640 1 -- 172.21.15.77:0/3544292278 shutdown_connections 2024-09-10T10:05:12.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:12.041+0000 7f3959c00640 1 -- 172.21.15.77:0/3544292278 wait complete. 2024-09-10T10:05:12.821 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph fs fail cephfs 2024-09-10T10:05:13.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:05:12 smithi077 ceph-mon[30319]: pgmap v692: 129 pgs: 129 active+clean; 115 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-09-10T10:05:13.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:05:12 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/3544292278' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:05:13.192 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:05:13.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:05:12 smithi134 ceph-mon[35674]: pgmap v692: 129 pgs: 129 active+clean; 115 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-09-10T10:05:13.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:05:12 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/3544292278' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:05:13.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:05:12 smithi120 ceph-mon[34770]: pgmap v692: 129 pgs: 129 active+clean; 115 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-09-10T10:05:13.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:05:12 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/3544292278' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:05:13.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:13.720+0000 7ff4b2400640 1 -- 172.21.15.77:0/18361400 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff4ac112870 msgr2=0x7ff4ac114c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:13.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:13.720+0000 7ff4b2400640 1 --2- 172.21.15.77:0/18361400 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff4ac112870 0x7ff4ac114c60 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7ff4a4009ca0 tx=0x7ff4a402a8c0 comp rx=0 tx=0).stop 2024-09-10T10:05:13.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:13.720+0000 7ff4b2400640 1 -- 172.21.15.77:0/18361400 shutdown_connections 2024-09-10T10:05:13.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:13.720+0000 7ff4b2400640 1 --2- 172.21.15.77:0/18361400 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff4ac112870 0x7ff4ac114c60 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:13.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:13.720+0000 7ff4b2400640 1 --2- 172.21.15.77:0/18361400 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4ac077040 0x7ff4ac112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:13.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:13.720+0000 7ff4b2400640 1 --2- 172.21.15.77:0/18361400 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff4ac076720 0x7ff4ac076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:13.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:13.720+0000 7ff4b2400640 1 -- 172.21.15.77:0/18361400 >> 172.21.15.77:0/18361400 conn(0x7ff4ac0ff910 msgr2=0x7ff4ac101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:13.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:13.721+0000 7ff4b2400640 1 -- 172.21.15.77:0/18361400 shutdown_connections 2024-09-10T10:05:13.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:13.721+0000 7ff4b2400640 1 -- 172.21.15.77:0/18361400 wait complete. 2024-09-10T10:05:13.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:13.721+0000 7ff4b2400640 1 Processor -- start 2024-09-10T10:05:13.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:13.722+0000 7ff4b2400640 1 -- start start 2024-09-10T10:05:13.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:13.722+0000 7ff4b2400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4ac076720 0x7ff4ac1a4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:13.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:13.722+0000 7ff4b2400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff4ac077040 0x7ff4ac1a4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:13.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:13.722+0000 7ff4b2400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff4ac112870 0x7ff4ac19e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:13.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:13.722+0000 7ff4b2400640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff4ac1183b0 con 0x7ff4ac112870 2024-09-10T10:05:13.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:13.722+0000 7ff4b2400640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff4ac118230 con 0x7ff4ac077040 2024-09-10T10:05:13.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:13.722+0000 7ff4b2400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff4ac118530 con 0x7ff4ac076720 2024-09-10T10:05:13.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:13.722+0000 7ff4b1a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff4ac112870 0x7ff4ac19e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:13.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:13.722+0000 7ff4b1a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff4ac112870 0x7ff4ac19e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49244/0 (socket says 172.21.15.77:49244) 2024-09-10T10:05:13.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:13.722+0000 7ff4b1a00640 1 -- 172.21.15.77:0/2162298423 learned_addr learned my addr 172.21.15.77:0/2162298423 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:05:13.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:13.722+0000 7ff4b1000640 1 --2- 172.21.15.77:0/2162298423 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4ac076720 0x7ff4ac1a4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:13.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:13.722+0000 7ff4abe00640 1 --2- 172.21.15.77:0/2162298423 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff4ac077040 0x7ff4ac1a4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:13.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:13.723+0000 7ff4b1a00640 1 -- 172.21.15.77:0/2162298423 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4ac076720 msgr2=0x7ff4ac1a4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:13.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:13.723+0000 7ff4b1a00640 1 --2- 172.21.15.77:0/2162298423 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4ac076720 0x7ff4ac1a4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:13.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:13.723+0000 7ff4b1a00640 1 -- 172.21.15.77:0/2162298423 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff4ac077040 msgr2=0x7ff4ac1a4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:13.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:13.723+0000 7ff4b1a00640 1 --2- 172.21.15.77:0/2162298423 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff4ac077040 0x7ff4ac1a4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:13.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:13.723+0000 7ff4b1a00640 1 -- 172.21.15.77:0/2162298423 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4ac19ef70 con 0x7ff4ac112870 2024-09-10T10:05:13.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:13.723+0000 7ff4b1a00640 1 --2- 172.21.15.77:0/2162298423 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff4ac112870 0x7ff4ac19e700 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7ff4a402ade0 tx=0x7ff4a40319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:13.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:13.723+0000 7ff4a9600640 1 -- 172.21.15.77:0/2162298423 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a4003b70 con 0x7ff4ac112870 2024-09-10T10:05:13.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:13.723+0000 7ff4a9600640 1 -- 172.21.15.77:0/2162298423 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4a4003d10 con 0x7ff4ac112870 2024-09-10T10:05:13.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:13.723+0000 7ff4a9600640 1 -- 172.21.15.77:0/2162298423 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a402fc40 con 0x7ff4ac112870 2024-09-10T10:05:13.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:13.723+0000 7ff4b2400640 1 -- 172.21.15.77:0/2162298423 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4ac19f200 con 0x7ff4ac112870 2024-09-10T10:05:13.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:13.723+0000 7ff4b2400640 1 -- 172.21.15.77:0/2162298423 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4ac1ab3e0 con 0x7ff4ac112870 2024-09-10T10:05:13.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:13.724+0000 7ff4b2400640 1 -- 172.21.15.77:0/2162298423 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4ac077e60 con 0x7ff4ac112870 2024-09-10T10:05:13.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:13.726+0000 7ff4a9600640 1 -- 172.21.15.77:0/2162298423 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff4a4004050 con 0x7ff4ac112870 2024-09-10T10:05:13.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:13.727+0000 7ff4a9600640 1 --2- 172.21.15.77:0/2162298423 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ff4940a2c00 0x7ff4940a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:13.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:13.728+0000 7ff4a9600640 1 -- 172.21.15.77:0/2162298423 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9237+0+0 (secure 0 0 0) 0x7ff4a40f0590 con 0x7ff4ac112870 2024-09-10T10:05:13.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:13.728+0000 7ff4b1000640 1 --2- 172.21.15.77:0/2162298423 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ff4940a2c00 0x7ff4940a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:13.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:13.729+0000 7ff4b1000640 1 --2- 172.21.15.77:0/2162298423 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ff4940a2c00 0x7ff4940a50c0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7ff4a0009bd0 tx=0x7ff4a0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:13.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:13.732+0000 7ff4a9600640 1 -- 172.21.15.77:0/2162298423 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff4a40bd1c0 con 0x7ff4ac112870 2024-09-10T10:05:13.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:13.942+0000 7ff4b2400640 1 -- 172.21.15.77:0/2162298423 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7ff4ac10da00 con 0x7ff4ac112870 2024-09-10T10:05:13.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:13.945+0000 7ff4a9600640 1 -- 172.21.15.77:0/2162298423 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v41) ==== 161+0+0 (secure 0 0 0) 0x7ff4a40bcac0 con 0x7ff4ac112870 2024-09-10T10:05:13.947 INFO:teuthology.orchestra.run.smithi077.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T10:05:13.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:13.950+0000 7ff4b2400640 1 -- 172.21.15.77:0/2162298423 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ff4940a2c00 msgr2=0x7ff4940a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:13.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:13.950+0000 7ff4b2400640 1 --2- 172.21.15.77:0/2162298423 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ff4940a2c00 0x7ff4940a50c0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7ff4a0009bd0 tx=0x7ff4a0005f90 comp rx=0 tx=0).stop 2024-09-10T10:05:13.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:13.950+0000 7ff4b2400640 1 -- 172.21.15.77:0/2162298423 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff4ac112870 msgr2=0x7ff4ac19e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:13.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:13.950+0000 7ff4b2400640 1 --2- 172.21.15.77:0/2162298423 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff4ac112870 0x7ff4ac19e700 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7ff4a402ade0 tx=0x7ff4a40319f0 comp rx=0 tx=0).stop 2024-09-10T10:05:13.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:13.952+0000 7ff4b2400640 1 -- 172.21.15.77:0/2162298423 shutdown_connections 2024-09-10T10:05:13.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:13.952+0000 7ff4b2400640 1 --2- 172.21.15.77:0/2162298423 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7ff4940a2c00 0x7ff4940a50c0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:13.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:13.952+0000 7ff4b2400640 1 --2- 172.21.15.77:0/2162298423 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff4ac112870 0x7ff4ac19e700 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:13.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:13.952+0000 7ff4b2400640 1 --2- 172.21.15.77:0/2162298423 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff4ac077040 0x7ff4ac1a4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:13.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:13.952+0000 7ff4b2400640 1 --2- 172.21.15.77:0/2162298423 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4ac076720 0x7ff4ac1a4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:13.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:13.952+0000 7ff4b2400640 1 -- 172.21.15.77:0/2162298423 >> 172.21.15.77:0/2162298423 conn(0x7ff4ac0ff910 msgr2=0x7ff4ac101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:13.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:13.952+0000 7ff4b2400640 1 -- 172.21.15.77:0/2162298423 shutdown_connections 2024-09-10T10:05:13.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:13.952+0000 7ff4b2400640 1 -- 172.21.15.77:0/2162298423 wait complete. 2024-09-10T10:05:14.321 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:05:13 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mon-a[30313]: 2024-09-10T10:05:13.943+0000 7f6b26200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T10:05:14.321 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:05:13 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/2162298423' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-10T10:05:14.321 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:05:13 smithi077 ceph-mon[30319]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-10T10:05:14.321 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:05:13 smithi077 ceph-mon[30319]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T10:05:14.321 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:05:13 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/2162298423' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-10T10:05:14.322 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:05:13 smithi077 ceph-mon[30319]: osdmap e97: 12 total, 12 up, 12 in 2024-09-10T10:05:14.322 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:05:13 smithi077 ceph-mon[30319]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-10T10:05:14.322 INFO:journalctl@ceph.mds.a.smithi077.stdout:Sep 10 10:05:14 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-a[70880]: ignoring --setuser ceph since I am not root 2024-09-10T10:05:14.322 INFO:journalctl@ceph.mds.a.smithi077.stdout:Sep 10 10:05:14 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-a[70880]: ignoring --setgroup ceph since I am not root 2024-09-10T10:05:14.322 INFO:journalctl@ceph.mds.a.smithi077.stdout:Sep 10 10:05:14 smithi077 ceph-mds[70884]: -- 172.21.15.77:0/2879716555 <== mon.0 v2:172.21.15.77:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55791e517860 con 0x55791e4f5000 2024-09-10T10:05:14.322 INFO:journalctl@ceph.mds.a.smithi077.stdout:Sep 10 10:05:14 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-a[70880]: starting mds.a at 2024-09-10T10:05:14.323 INFO:journalctl@ceph.mds.g.smithi077.stdout:Sep 10 10:05:14 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-g[70511]: ignoring --setuser ceph since I am not root 2024-09-10T10:05:14.323 INFO:journalctl@ceph.mds.g.smithi077.stdout:Sep 10 10:05:14 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-g[70511]: ignoring --setgroup ceph since I am not root 2024-09-10T10:05:14.323 INFO:journalctl@ceph.mds.g.smithi077.stdout:Sep 10 10:05:14 smithi077 ceph-mds[70515]: -- 172.21.15.77:0/2235544742 <== mon.0 v2:172.21.15.77:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564ac8f19860 con 0x564ac8ef6800 2024-09-10T10:05:14.323 INFO:journalctl@ceph.mds.g.smithi077.stdout:Sep 10 10:05:14 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-g[70511]: starting mds.g at 2024-09-10T10:05:14.381 INFO:journalctl@ceph.mds.c.smithi134.stdout:Sep 10 10:05:13 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-c[66875]: ignoring --setuser ceph since I am not root 2024-09-10T10:05:14.381 INFO:journalctl@ceph.mds.c.smithi134.stdout:Sep 10 10:05:13 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-c[66875]: ignoring --setgroup ceph since I am not root 2024-09-10T10:05:14.381 INFO:journalctl@ceph.mds.c.smithi134.stdout:Sep 10 10:05:13 smithi134 ceph-mds[66879]: -- 172.21.15.134:0/1669735419 <== mon.2 v2:172.21.15.120:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d0e8ebb860 con 0x55d0e8e98800 2024-09-10T10:05:14.381 INFO:journalctl@ceph.mds.c.smithi134.stdout:Sep 10 10:05:13 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-c[66875]: starting mds.c at 2024-09-10T10:05:14.382 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:05:13 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/2162298423' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-10T10:05:14.382 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:05:13 smithi134 ceph-mon[35674]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-10T10:05:14.382 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:05:13 smithi134 ceph-mon[35674]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T10:05:14.382 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:05:13 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/2162298423' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-10T10:05:14.382 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:05:13 smithi134 ceph-mon[35674]: osdmap e97: 12 total, 12 up, 12 in 2024-09-10T10:05:14.382 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:05:13 smithi134 ceph-mon[35674]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-10T10:05:14.383 INFO:journalctl@ceph.mds.i.smithi134.stdout:Sep 10 10:05:13 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-i[67177]: ignoring --setuser ceph since I am not root 2024-09-10T10:05:14.383 INFO:journalctl@ceph.mds.i.smithi134.stdout:Sep 10 10:05:13 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-i[67177]: ignoring --setgroup ceph since I am not root 2024-09-10T10:05:14.383 INFO:journalctl@ceph.mds.i.smithi134.stdout:Sep 10 10:05:13 smithi134 ceph-mds[67181]: -- 172.21.15.134:0/284162489 <== mon.1 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e25387d860 con 0x55e25385b400 2024-09-10T10:05:14.383 INFO:journalctl@ceph.mds.i.smithi134.stdout:Sep 10 10:05:13 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-i[67177]: starting mds.i at 2024-09-10T10:05:14.383 INFO:journalctl@ceph.mds.l.smithi134.stdout:Sep 10 10:05:13 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-l[66271]: ignoring --setuser ceph since I am not root 2024-09-10T10:05:14.384 INFO:journalctl@ceph.mds.l.smithi134.stdout:Sep 10 10:05:13 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-l[66271]: ignoring --setgroup ceph since I am not root 2024-09-10T10:05:14.384 INFO:journalctl@ceph.mds.l.smithi134.stdout:Sep 10 10:05:13 smithi134 ceph-mds[66275]: -- 172.21.15.134:0/2913518712 <== mon.1 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b0e8303860 con 0x55b0e82e0800 2024-09-10T10:05:14.384 INFO:journalctl@ceph.mds.l.smithi134.stdout:Sep 10 10:05:13 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-l[66271]: starting mds.l at 2024-09-10T10:05:14.384 INFO:journalctl@ceph.mds.f.smithi134.stdout:Sep 10 10:05:13 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-f[66571]: ignoring --setuser ceph since I am not root 2024-09-10T10:05:14.384 INFO:journalctl@ceph.mds.f.smithi134.stdout:Sep 10 10:05:13 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-f[66571]: ignoring --setgroup ceph since I am not root 2024-09-10T10:05:14.384 INFO:journalctl@ceph.mds.f.smithi134.stdout:Sep 10 10:05:13 smithi134 ceph-mds[66575]: -- 172.21.15.134:0/2146712664 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x556a90270680 con 0x556a90244800 2024-09-10T10:05:14.384 INFO:journalctl@ceph.mds.f.smithi134.stdout:Sep 10 10:05:13 smithi134 ceph-mds[66575]: -- 172.21.15.134:0/2146712664 <== mon.2 v2:172.21.15.120:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556a90267860 con 0x556a90244800 2024-09-10T10:05:14.384 INFO:journalctl@ceph.mds.f.smithi134.stdout:Sep 10 10:05:13 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-f[66571]: starting mds.f at 2024-09-10T10:05:14.424 INFO:journalctl@ceph.mds.k.smithi120.stdout:Sep 10 10:05:13 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-k[64012]: ignoring --setuser ceph since I am not root 2024-09-10T10:05:14.424 INFO:journalctl@ceph.mds.k.smithi120.stdout:Sep 10 10:05:13 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-k[64012]: ignoring --setgroup ceph since I am not root 2024-09-10T10:05:14.424 INFO:journalctl@ceph.mds.k.smithi120.stdout:Sep 10 10:05:13 smithi120 ceph-mds[64016]: -- 172.21.15.120:0/3126257142 <== mon.2 v2:172.21.15.120:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f4eef2d860 con 0x55f4eef0a800 2024-09-10T10:05:14.424 INFO:journalctl@ceph.mds.k.smithi120.stdout:Sep 10 10:05:13 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-k[64012]: starting mds.k at 2024-09-10T10:05:14.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:05:13 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/2162298423' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-10T10:05:14.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:05:13 smithi120 ceph-mon[34770]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-10T10:05:14.425 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:05:13 smithi120 ceph-mon[34770]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T10:05:14.425 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:05:13 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/2162298423' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-10T10:05:14.425 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:05:13 smithi120 ceph-mon[34770]: osdmap e97: 12 total, 12 up, 12 in 2024-09-10T10:05:14.425 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:05:13 smithi120 ceph-mon[34770]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-10T10:05:14.425 INFO:journalctl@ceph.mds.e.smithi120.stdout:Sep 10 10:05:13 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-e[64330]: ignoring --setuser ceph since I am not root 2024-09-10T10:05:14.425 INFO:journalctl@ceph.mds.e.smithi120.stdout:Sep 10 10:05:13 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-e[64330]: ignoring --setgroup ceph since I am not root 2024-09-10T10:05:14.425 INFO:journalctl@ceph.mds.e.smithi120.stdout:Sep 10 10:05:13 smithi120 ceph-mds[64334]: -- 172.21.15.120:0/971665249 <== mon.2 v2:172.21.15.120:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5654f6b69c20 con 0x5654f6b47400 2024-09-10T10:05:14.425 INFO:journalctl@ceph.mds.e.smithi120.stdout:Sep 10 10:05:13 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-e[64330]: starting mds.e at 2024-09-10T10:05:14.426 INFO:journalctl@ceph.mds.h.smithi120.stdout:Sep 10 10:05:13 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-h[64633]: ignoring --setuser ceph since I am not root 2024-09-10T10:05:14.426 INFO:journalctl@ceph.mds.h.smithi120.stdout:Sep 10 10:05:13 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-h[64633]: ignoring --setgroup ceph since I am not root 2024-09-10T10:05:14.426 INFO:journalctl@ceph.mds.h.smithi120.stdout:Sep 10 10:05:14 smithi120 ceph-mds[64637]: -- 172.21.15.120:0/4287352686 <== mon.2 v2:172.21.15.120:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f7c3c3bc20 con 0x55f7c3c19000 2024-09-10T10:05:14.426 INFO:journalctl@ceph.mds.h.smithi120.stdout:Sep 10 10:05:14 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-h[64633]: starting mds.h at 2024-09-10T10:05:14.426 INFO:journalctl@ceph.mds.b.smithi120.stdout:Sep 10 10:05:14 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-b[64932]: ignoring --setuser ceph since I am not root 2024-09-10T10:05:14.426 INFO:journalctl@ceph.mds.b.smithi120.stdout:Sep 10 10:05:14 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-b[64932]: ignoring --setgroup ceph since I am not root 2024-09-10T10:05:14.426 INFO:journalctl@ceph.mds.b.smithi120.stdout:Sep 10 10:05:14 smithi120 ceph-mds[64936]: -- 172.21.15.120:0/1951451897 <== mon.1 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5638f03ef860 con 0x5638f03cd000 2024-09-10T10:05:14.426 INFO:journalctl@ceph.mds.b.smithi120.stdout:Sep 10 10:05:14 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-b[64932]: starting mds.b at 2024-09-10T10:05:14.812 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T10:05:15.187 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:05:15.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:05:14 smithi077 ceph-mon[30319]: pgmap v693: 129 pgs: 129 active+clean; 115 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-09-10T10:05:15.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:05:14 smithi134 ceph-mon[35674]: pgmap v693: 129 pgs: 129 active+clean; 115 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-09-10T10:05:15.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:05:14 smithi120 ceph-mon[34770]: pgmap v693: 129 pgs: 129 active+clean; 115 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-09-10T10:05:15.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:15.721+0000 7f02d7e00640 1 -- 172.21.15.77:0/246575557 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f02d8103e30 msgr2=0x7f02d8104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:15.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:15.721+0000 7f02d7e00640 1 --2- 172.21.15.77:0/246575557 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f02d8103e30 0x7f02d8104210 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f02c8009900 tx=0x7f02c802a3f0 comp rx=0 tx=0).stop 2024-09-10T10:05:15.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:15.721+0000 7f02d7e00640 1 -- 172.21.15.77:0/246575557 shutdown_connections 2024-09-10T10:05:15.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:15.721+0000 7f02d7e00640 1 --2- 172.21.15.77:0/246575557 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f02d8108f60 0x7f02d8110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:15.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:15.721+0000 7f02d7e00640 1 --2- 172.21.15.77:0/246575557 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f02d81047e0 0x7f02d8108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:15.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:15.721+0000 7f02d7e00640 1 --2- 172.21.15.77:0/246575557 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f02d8103e30 0x7f02d8104210 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:15.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:15.721+0000 7f02d7e00640 1 -- 172.21.15.77:0/246575557 >> 172.21.15.77:0/246575557 conn(0x7f02d80ff8d0 msgr2=0x7f02d8101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:15.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:15.721+0000 7f02d7e00640 1 -- 172.21.15.77:0/246575557 shutdown_connections 2024-09-10T10:05:15.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:15.721+0000 7f02d7e00640 1 -- 172.21.15.77:0/246575557 wait complete. 2024-09-10T10:05:15.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:15.722+0000 7f02d7e00640 1 Processor -- start 2024-09-10T10:05:15.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:15.722+0000 7f02d7e00640 1 -- start start 2024-09-10T10:05:15.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:15.722+0000 7f02d7e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f02d8103e30 0x7f02d819a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:15.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:15.722+0000 7f02d7e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f02d81047e0 0x7f02d819a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:15.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:15.722+0000 7f02d7e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f02d8108f60 0x7f02d819ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:15.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:15.723+0000 7f02d7e00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f02d8114010 con 0x7f02d8103e30 2024-09-10T10:05:15.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:15.723+0000 7f02d7e00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f02d8113e90 con 0x7f02d81047e0 2024-09-10T10:05:15.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:15.723+0000 7f02d7e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f02d8114190 con 0x7f02d8108f60 2024-09-10T10:05:15.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:15.723+0000 7f02d6a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f02d8103e30 0x7f02d819a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:15.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:15.723+0000 7f02d6000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f02d81047e0 0x7f02d819a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:15.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:15.723+0000 7f02d6000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f02d81047e0 0x7f02d819a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:50270/0 (socket says 172.21.15.77:50270) 2024-09-10T10:05:15.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:15.723+0000 7f02d6000640 1 -- 172.21.15.77:0/2989204654 learned_addr learned my addr 172.21.15.77:0/2989204654 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:05:15.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:15.723+0000 7f02d7400640 1 --2- 172.21.15.77:0/2989204654 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f02d8108f60 0x7f02d819ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:15.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:15.723+0000 7f02d6000640 1 -- 172.21.15.77:0/2989204654 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f02d8108f60 msgr2=0x7f02d819ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:15.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:15.723+0000 7f02d6000640 1 --2- 172.21.15.77:0/2989204654 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f02d8108f60 0x7f02d819ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:15.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:15.723+0000 7f02d6000640 1 -- 172.21.15.77:0/2989204654 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f02d8103e30 msgr2=0x7f02d819a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:15.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:15.723+0000 7f02d6000640 1 --2- 172.21.15.77:0/2989204654 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f02d8103e30 0x7f02d819a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:15.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:15.723+0000 7f02d6000640 1 -- 172.21.15.77:0/2989204654 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02d819f3a0 con 0x7f02d81047e0 2024-09-10T10:05:15.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:15.723+0000 7f02d7400640 1 --2- 172.21.15.77:0/2989204654 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f02d8108f60 0x7f02d819ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:05:15.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:15.724+0000 7f02d6000640 1 --2- 172.21.15.77:0/2989204654 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f02d81047e0 0x7f02d819a930 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f02c400e9f0 tx=0x7f02c400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:15.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:15.724+0000 7f02cf400640 1 -- 172.21.15.77:0/2989204654 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02c400cd80 con 0x7f02d81047e0 2024-09-10T10:05:15.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:15.724+0000 7f02d7e00640 1 -- 172.21.15.77:0/2989204654 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02d819f690 con 0x7f02d81047e0 2024-09-10T10:05:15.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:15.724+0000 7f02cf400640 1 -- 172.21.15.77:0/2989204654 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02c4010910 con 0x7f02d81047e0 2024-09-10T10:05:15.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:15.724+0000 7f02cf400640 1 -- 172.21.15.77:0/2989204654 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02c40188f0 con 0x7f02d81047e0 2024-09-10T10:05:15.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:15.724+0000 7f02d7e00640 1 -- 172.21.15.77:0/2989204654 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02d81a6f70 con 0x7f02d81047e0 2024-09-10T10:05:15.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:15.725+0000 7f02d7e00640 1 -- 172.21.15.77:0/2989204654 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02d8104210 con 0x7f02d81047e0 2024-09-10T10:05:15.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:15.728+0000 7f02cf400640 1 -- 172.21.15.77:0/2989204654 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f02c4010ac0 con 0x7f02d81047e0 2024-09-10T10:05:15.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:15.729+0000 7f02cf400640 1 --2- 172.21.15.77:0/2989204654 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f02bc0a2b80 0x7f02bc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:15.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:15.729+0000 7f02cf400640 1 -- 172.21.15.77:0/2989204654 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9667+0+0 (secure 0 0 0) 0x7f02c40cfe80 con 0x7f02d81047e0 2024-09-10T10:05:15.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:15.729+0000 7f02d6a00640 1 --2- 172.21.15.77:0/2989204654 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f02bc0a2b80 0x7f02bc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:15.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:15.730+0000 7f02d6a00640 1 --2- 172.21.15.77:0/2989204654 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f02bc0a2b80 0x7f02bc0a5040 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f02c8000c00 tx=0x7f02c8005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:15.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:15.730+0000 7f02cf400640 1 -- 172.21.15.77:0/2989204654 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f02c409c7e0 con 0x7f02d81047e0 2024-09-10T10:05:15.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:15.942+0000 7f02d7e00640 1 -- 172.21.15.77:0/2989204654 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f02d810c1a0 con 0x7f02d81047e0 2024-09-10T10:05:15.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:15.982+0000 7f02cf400640 1 -- 172.21.15.77:0/2989204654 <== mon.2 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v43) ==== 104+0+0 (secure 0 0 0) 0x7f02c409c0e0 con 0x7f02d81047e0 2024-09-10T10:05:15.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:15.985+0000 7f02d7e00640 1 -- 172.21.15.77:0/2989204654 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f02bc0a2b80 msgr2=0x7f02bc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:15.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:15.985+0000 7f02d7e00640 1 --2- 172.21.15.77:0/2989204654 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f02bc0a2b80 0x7f02bc0a5040 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f02c8000c00 tx=0x7f02c8005b50 comp rx=0 tx=0).stop 2024-09-10T10:05:15.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:15.985+0000 7f02d7e00640 1 -- 172.21.15.77:0/2989204654 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f02d81047e0 msgr2=0x7f02d819a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:15.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:15.985+0000 7f02d7e00640 1 --2- 172.21.15.77:0/2989204654 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f02d81047e0 0x7f02d819a930 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f02c400e9f0 tx=0x7f02c400eeb0 comp rx=0 tx=0).stop 2024-09-10T10:05:15.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:15.985+0000 7f02d7e00640 1 -- 172.21.15.77:0/2989204654 shutdown_connections 2024-09-10T10:05:15.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:15.985+0000 7f02d7e00640 1 --2- 172.21.15.77:0/2989204654 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f02bc0a2b80 0x7f02bc0a5040 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:15.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:15.985+0000 7f02d7e00640 1 --2- 172.21.15.77:0/2989204654 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f02d8108f60 0x7f02d819ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:15.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:15.985+0000 7f02d7e00640 1 --2- 172.21.15.77:0/2989204654 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f02d81047e0 0x7f02d819a930 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:15.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:15.985+0000 7f02d7e00640 1 --2- 172.21.15.77:0/2989204654 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f02d8103e30 0x7f02d819a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:15.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:15.985+0000 7f02d7e00640 1 -- 172.21.15.77:0/2989204654 >> 172.21.15.77:0/2989204654 conn(0x7f02d80ff8d0 msgr2=0x7f02d8100b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:15.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:15.985+0000 7f02d7e00640 1 -- 172.21.15.77:0/2989204654 shutdown_connections 2024-09-10T10:05:15.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:15.985+0000 7f02d7e00640 1 -- 172.21.15.77:0/2989204654 wait complete. 2024-09-10T10:05:16.324 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:05:15 smithi077 ceph-mon[30319]: mds.? [v2:172.21.15.120:6832/54012532,v1:172.21.15.120:6834/54012532] up:boot 2024-09-10T10:05:16.324 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:05:15 smithi077 ceph-mon[30319]: mds.? [v2:172.21.15.120:6833/1050936676,v1:172.21.15.120:6835/1050936676] up:boot 2024-09-10T10:05:16.324 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:05:15 smithi077 ceph-mon[30319]: mds.? [v2:172.21.15.134:6834/2119862369,v1:172.21.15.134:6836/2119862369] up:boot 2024-09-10T10:05:16.324 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:05:15 smithi077 ceph-mon[30319]: mds.? [v2:172.21.15.134:6833/1209900022,v1:172.21.15.134:6835/1209900022] up:boot 2024-09-10T10:05:16.325 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:05:15 smithi077 ceph-mon[30319]: mds.? [v2:172.21.15.134:6832/1793583193,v1:172.21.15.134:6837/1793583193] up:boot 2024-09-10T10:05:16.325 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:05:15 smithi077 ceph-mon[30319]: mds.? [v2:172.21.15.134:6838/3222373014,v1:172.21.15.134:6839/3222373014] up:boot 2024-09-10T10:05:16.325 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:05:15 smithi077 ceph-mon[30319]: mds.? [v2:172.21.15.120:6836/1293843309,v1:172.21.15.120:6837/1293843309] up:boot 2024-09-10T10:05:16.325 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:05:15 smithi077 ceph-mon[30319]: mds.? [v2:172.21.15.120:6838/2344099825,v1:172.21.15.120:6839/2344099825] up:boot 2024-09-10T10:05:16.325 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:05:15 smithi077 ceph-mon[30319]: mds.? [v2:172.21.15.77:6839/79599258,v1:172.21.15.77:6841/79599258] up:boot 2024-09-10T10:05:16.325 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:05:15 smithi077 ceph-mon[30319]: mds.? [v2:172.21.15.77:6838/3968333462,v1:172.21.15.77:6840/3968333462] up:boot 2024-09-10T10:05:16.325 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:05:15 smithi077 ceph-mon[30319]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-10T10:05:16.325 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:05:15 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-10T10:05:16.325 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:05:15 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-10T10:05:16.325 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:05:15 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-10T10:05:16.325 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:05:15 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-10T10:05:16.325 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:05:15 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-10T10:05:16.326 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:05:15 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-10T10:05:16.326 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:05:15 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-10T10:05:16.326 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:05:15 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-10T10:05:16.326 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:05:15 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-10T10:05:16.326 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:05:15 smithi077 ceph-mon[30319]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-10T10:05:16.326 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:05:15 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/2989204654' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T10:05:16.326 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:05:15 smithi077 ceph-mon[30319]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T10:05:16.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:05:15 smithi134 ceph-mon[35674]: mds.? [v2:172.21.15.120:6832/54012532,v1:172.21.15.120:6834/54012532] up:boot 2024-09-10T10:05:16.382 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:05:15 smithi134 ceph-mon[35674]: mds.? [v2:172.21.15.120:6833/1050936676,v1:172.21.15.120:6835/1050936676] up:boot 2024-09-10T10:05:16.382 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:05:15 smithi134 ceph-mon[35674]: mds.? [v2:172.21.15.134:6834/2119862369,v1:172.21.15.134:6836/2119862369] up:boot 2024-09-10T10:05:16.382 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:05:15 smithi134 ceph-mon[35674]: mds.? [v2:172.21.15.134:6833/1209900022,v1:172.21.15.134:6835/1209900022] up:boot 2024-09-10T10:05:16.382 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:05:15 smithi134 ceph-mon[35674]: mds.? [v2:172.21.15.134:6832/1793583193,v1:172.21.15.134:6837/1793583193] up:boot 2024-09-10T10:05:16.382 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:05:15 smithi134 ceph-mon[35674]: mds.? [v2:172.21.15.134:6838/3222373014,v1:172.21.15.134:6839/3222373014] up:boot 2024-09-10T10:05:16.382 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:05:15 smithi134 ceph-mon[35674]: mds.? [v2:172.21.15.120:6836/1293843309,v1:172.21.15.120:6837/1293843309] up:boot 2024-09-10T10:05:16.382 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:05:15 smithi134 ceph-mon[35674]: mds.? [v2:172.21.15.120:6838/2344099825,v1:172.21.15.120:6839/2344099825] up:boot 2024-09-10T10:05:16.382 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:05:15 smithi134 ceph-mon[35674]: mds.? [v2:172.21.15.77:6839/79599258,v1:172.21.15.77:6841/79599258] up:boot 2024-09-10T10:05:16.382 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:05:15 smithi134 ceph-mon[35674]: mds.? [v2:172.21.15.77:6838/3968333462,v1:172.21.15.77:6840/3968333462] up:boot 2024-09-10T10:05:16.382 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:05:15 smithi134 ceph-mon[35674]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-10T10:05:16.382 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:05:15 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-10T10:05:16.383 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:05:15 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-10T10:05:16.383 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:05:15 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-10T10:05:16.383 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:05:15 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-10T10:05:16.383 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:05:15 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-10T10:05:16.383 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:05:15 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-10T10:05:16.383 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:05:15 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-10T10:05:16.383 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:05:15 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-10T10:05:16.383 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:05:15 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-10T10:05:16.383 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:05:15 smithi134 ceph-mon[35674]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-10T10:05:16.383 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:05:15 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/2989204654' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T10:05:16.383 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:05:15 smithi134 ceph-mon[35674]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T10:05:16.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:05:15 smithi120 ceph-mon[34770]: mds.? [v2:172.21.15.120:6832/54012532,v1:172.21.15.120:6834/54012532] up:boot 2024-09-10T10:05:16.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:05:15 smithi120 ceph-mon[34770]: mds.? [v2:172.21.15.120:6833/1050936676,v1:172.21.15.120:6835/1050936676] up:boot 2024-09-10T10:05:16.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:05:15 smithi120 ceph-mon[34770]: mds.? [v2:172.21.15.134:6834/2119862369,v1:172.21.15.134:6836/2119862369] up:boot 2024-09-10T10:05:16.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:05:15 smithi120 ceph-mon[34770]: mds.? [v2:172.21.15.134:6833/1209900022,v1:172.21.15.134:6835/1209900022] up:boot 2024-09-10T10:05:16.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:05:15 smithi120 ceph-mon[34770]: mds.? [v2:172.21.15.134:6832/1793583193,v1:172.21.15.134:6837/1793583193] up:boot 2024-09-10T10:05:16.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:05:15 smithi120 ceph-mon[34770]: mds.? [v2:172.21.15.134:6838/3222373014,v1:172.21.15.134:6839/3222373014] up:boot 2024-09-10T10:05:16.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:05:15 smithi120 ceph-mon[34770]: mds.? [v2:172.21.15.120:6836/1293843309,v1:172.21.15.120:6837/1293843309] up:boot 2024-09-10T10:05:16.425 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:05:15 smithi120 ceph-mon[34770]: mds.? [v2:172.21.15.120:6838/2344099825,v1:172.21.15.120:6839/2344099825] up:boot 2024-09-10T10:05:16.425 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:05:15 smithi120 ceph-mon[34770]: mds.? [v2:172.21.15.77:6839/79599258,v1:172.21.15.77:6841/79599258] up:boot 2024-09-10T10:05:16.425 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:05:15 smithi120 ceph-mon[34770]: mds.? [v2:172.21.15.77:6838/3968333462,v1:172.21.15.77:6840/3968333462] up:boot 2024-09-10T10:05:16.425 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:05:15 smithi120 ceph-mon[34770]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-10T10:05:16.425 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:05:15 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-10T10:05:16.425 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:05:15 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-10T10:05:16.425 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:05:15 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-10T10:05:16.425 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:05:15 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-10T10:05:16.425 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:05:15 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-10T10:05:16.425 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:05:15 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-10T10:05:16.425 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:05:15 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-10T10:05:16.425 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:05:15 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-10T10:05:16.426 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:05:15 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-10T10:05:16.426 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:05:15 smithi120 ceph-mon[34770]: from='mgr.14150 172.21.15.77:0/360589625' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-10T10:05:16.426 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:05:15 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/2989204654' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T10:05:16.426 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:05:15 smithi120 ceph-mon[34770]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T10:05:16.829 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T10:05:17.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:05:16 smithi077 ceph-mon[30319]: pgmap v695: 129 pgs: 129 active+clean; 115 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-09-10T10:05:17.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:05:16 smithi077 ceph-mon[30319]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-10T10:05:17.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:05:16 smithi077 ceph-mon[30319]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-10T10:05:17.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:05:16 smithi077 ceph-mon[30319]: Cluster is now healthy 2024-09-10T10:05:17.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:05:16 smithi077 ceph-mon[30319]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-10T10:05:17.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:05:16 smithi077 ceph-mon[30319]: fsmap 12 up:standby 2024-09-10T10:05:17.199 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:05:17.325 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:05:16 smithi120 ceph-mon[34770]: pgmap v695: 129 pgs: 129 active+clean; 115 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-09-10T10:05:17.325 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:05:16 smithi120 ceph-mon[34770]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-10T10:05:17.325 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:05:16 smithi120 ceph-mon[34770]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-10T10:05:17.325 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:05:16 smithi120 ceph-mon[34770]: Cluster is now healthy 2024-09-10T10:05:17.325 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:05:16 smithi120 ceph-mon[34770]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-10T10:05:17.325 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:05:16 smithi120 ceph-mon[34770]: fsmap 12 up:standby 2024-09-10T10:05:17.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:05:16 smithi134 ceph-mon[35674]: pgmap v695: 129 pgs: 129 active+clean; 115 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-09-10T10:05:17.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:05:16 smithi134 ceph-mon[35674]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-10T10:05:17.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:05:16 smithi134 ceph-mon[35674]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-10T10:05:17.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:05:16 smithi134 ceph-mon[35674]: Cluster is now healthy 2024-09-10T10:05:17.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:05:16 smithi134 ceph-mon[35674]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-10T10:05:17.382 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:05:16 smithi134 ceph-mon[35674]: fsmap 12 up:standby 2024-09-10T10:05:17.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:17.757+0000 7fd358200640 1 -- 172.21.15.77:0/3383780559 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd350106f10 msgr2=0x7fd35010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:17.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:17.757+0000 7fd358200640 1 --2- 172.21.15.77:0/3383780559 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd350106f10 0x7fd35010ea90 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fd348009ca0 tx=0x7fd34802a8c0 comp rx=0 tx=0).stop 2024-09-10T10:05:17.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:17.757+0000 7fd358200640 1 -- 172.21.15.77:0/3383780559 shutdown_connections 2024-09-10T10:05:17.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:17.757+0000 7fd358200640 1 --2- 172.21.15.77:0/3383780559 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd350106f10 0x7fd35010ea90 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:17.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:17.757+0000 7fd358200640 1 --2- 172.21.15.77:0/3383780559 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd350102790 0x7fd3501067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:17.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:17.757+0000 7fd358200640 1 --2- 172.21.15.77:0/3383780559 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd350101de0 0x7fd3501021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:17.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:17.757+0000 7fd358200640 1 -- 172.21.15.77:0/3383780559 >> 172.21.15.77:0/3383780559 conn(0x7fd3500fd8c0 msgr2=0x7fd3500ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:17.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:17.757+0000 7fd358200640 1 -- 172.21.15.77:0/3383780559 shutdown_connections 2024-09-10T10:05:17.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:17.757+0000 7fd358200640 1 -- 172.21.15.77:0/3383780559 wait complete. 2024-09-10T10:05:17.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:17.758+0000 7fd358200640 1 Processor -- start 2024-09-10T10:05:17.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:17.758+0000 7fd358200640 1 -- start start 2024-09-10T10:05:17.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:17.759+0000 7fd358200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd350101de0 0x7fd35019a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:17.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:17.759+0000 7fd358200640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd350102790 0x7fd35019ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:17.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:17.759+0000 7fd358200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd350106f10 0x7fd35019eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:17.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:17.759+0000 7fd358200640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd350111fc0 con 0x7fd350106f10 2024-09-10T10:05:17.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:17.759+0000 7fd358200640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd350111e40 con 0x7fd350102790 2024-09-10T10:05:17.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:17.759+0000 7fd358200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd350112140 con 0x7fd350101de0 2024-09-10T10:05:17.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:17.759+0000 7fd356200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd350106f10 0x7fd35019eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:17.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:17.759+0000 7fd354e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd350102790 0x7fd35019ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:17.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:17.759+0000 7fd354e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd350102790 0x7fd35019ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:50302/0 (socket says 172.21.15.77:50302) 2024-09-10T10:05:17.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:17.759+0000 7fd356200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd350106f10 0x7fd35019eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49346/0 (socket says 172.21.15.77:49346) 2024-09-10T10:05:17.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:17.759+0000 7fd354e00640 1 -- 172.21.15.77:0/73505854 learned_addr learned my addr 172.21.15.77:0/73505854 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:05:17.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:17.759+0000 7fd355800640 1 --2- 172.21.15.77:0/73505854 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd350101de0 0x7fd35019a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:17.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:17.760+0000 7fd354e00640 1 -- 172.21.15.77:0/73505854 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd350101de0 msgr2=0x7fd35019a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:17.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:17.760+0000 7fd354e00640 1 --2- 172.21.15.77:0/73505854 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd350101de0 0x7fd35019a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:17.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:17.760+0000 7fd354e00640 1 -- 172.21.15.77:0/73505854 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd350106f10 msgr2=0x7fd35019eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:17.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:17.760+0000 7fd354e00640 1 --2- 172.21.15.77:0/73505854 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd350106f10 0x7fd35019eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:17.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:17.760+0000 7fd354e00640 1 -- 172.21.15.77:0/73505854 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd35019f590 con 0x7fd350102790 2024-09-10T10:05:17.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:17.760+0000 7fd354e00640 1 --2- 172.21.15.77:0/73505854 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd350102790 0x7fd35019ab20 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fd33c00ea60 tx=0x7fd33c00ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:17.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:17.760+0000 7fd34e000640 1 -- 172.21.15.77:0/73505854 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd33c00cde0 con 0x7fd350102790 2024-09-10T10:05:17.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:17.760+0000 7fd34e000640 1 -- 172.21.15.77:0/73505854 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd33c0035f0 con 0x7fd350102790 2024-09-10T10:05:17.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:17.760+0000 7fd358200640 1 -- 172.21.15.77:0/73505854 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd35019f880 con 0x7fd350102790 2024-09-10T10:05:17.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:17.760+0000 7fd34e000640 1 -- 172.21.15.77:0/73505854 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd33c010670 con 0x7fd350102790 2024-09-10T10:05:17.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:17.760+0000 7fd358200640 1 -- 172.21.15.77:0/73505854 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3501a7160 con 0x7fd350102790 2024-09-10T10:05:17.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:17.761+0000 7fd358200640 1 -- 172.21.15.77:0/73505854 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3501021c0 con 0x7fd350102790 2024-09-10T10:05:17.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:17.764+0000 7fd34e000640 1 -- 172.21.15.77:0/73505854 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd33c0037a0 con 0x7fd350102790 2024-09-10T10:05:17.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:17.765+0000 7fd34e000640 1 --2- 172.21.15.77:0/73505854 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fd3380a2b80 0x7fd3380a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:17.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:17.765+0000 7fd34e000640 1 -- 172.21.15.77:0/73505854 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9667+0+0 (secure 0 0 0) 0x7fd33c0cfa80 con 0x7fd350102790 2024-09-10T10:05:17.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:17.765+0000 7fd355800640 1 --2- 172.21.15.77:0/73505854 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fd3380a2b80 0x7fd3380a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:17.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:17.766+0000 7fd355800640 1 --2- 172.21.15.77:0/73505854 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fd3380a2b80 0x7fd3380a5040 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fd344009bd0 tx=0x7fd344005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:17.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:17.771+0000 7fd34e000640 1 -- 172.21.15.77:0/73505854 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd33c0541b0 con 0x7fd350102790 2024-09-10T10:05:17.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:17.948+0000 7fd358200640 1 -- 172.21.15.77:0/73505854 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fd35019b940 con 0x7fd350102790 2024-09-10T10:05:17.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:17.986+0000 7fd34e000640 1 -- 172.21.15.77:0/73505854 <== mon.2 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v98) ==== 181+0+0 (secure 0 0 0) 0x7fd35019b940 con 0x7fd350102790 2024-09-10T10:05:17.988 INFO:teuthology.orchestra.run.smithi077.stderr:pool 'cephfs_metadata' removed 2024-09-10T10:05:17.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:17.989+0000 7fd358200640 1 -- 172.21.15.77:0/73505854 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fd3380a2b80 msgr2=0x7fd3380a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:17.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:17.989+0000 7fd358200640 1 --2- 172.21.15.77:0/73505854 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fd3380a2b80 0x7fd3380a5040 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fd344009bd0 tx=0x7fd344005f90 comp rx=0 tx=0).stop 2024-09-10T10:05:17.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:17.989+0000 7fd358200640 1 -- 172.21.15.77:0/73505854 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd350102790 msgr2=0x7fd35019ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:17.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:17.989+0000 7fd358200640 1 --2- 172.21.15.77:0/73505854 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd350102790 0x7fd35019ab20 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fd33c00ea60 tx=0x7fd33c00ef20 comp rx=0 tx=0).stop 2024-09-10T10:05:17.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:17.989+0000 7fd358200640 1 -- 172.21.15.77:0/73505854 shutdown_connections 2024-09-10T10:05:17.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:17.989+0000 7fd358200640 1 --2- 172.21.15.77:0/73505854 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fd3380a2b80 0x7fd3380a5040 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:17.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:17.989+0000 7fd358200640 1 --2- 172.21.15.77:0/73505854 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd350106f10 0x7fd35019eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:17.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:17.989+0000 7fd358200640 1 --2- 172.21.15.77:0/73505854 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd350102790 0x7fd35019ab20 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:17.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:17.989+0000 7fd358200640 1 --2- 172.21.15.77:0/73505854 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd350101de0 0x7fd35019a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:17.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:17.989+0000 7fd358200640 1 -- 172.21.15.77:0/73505854 >> 172.21.15.77:0/73505854 conn(0x7fd3500fd8c0 msgr2=0x7fd3500fe770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:17.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:17.990+0000 7fd358200640 1 -- 172.21.15.77:0/73505854 shutdown_connections 2024-09-10T10:05:17.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:17.990+0000 7fd358200640 1 -- 172.21.15.77:0/73505854 wait complete. 2024-09-10T10:05:18.333 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:05:17 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/73505854' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T10:05:18.333 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:05:17 smithi077 ceph-mon[30319]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T10:05:18.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:05:17 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/73505854' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T10:05:18.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:05:17 smithi134 ceph-mon[35674]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T10:05:18.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:05:17 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/73505854' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T10:05:18.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:05:17 smithi120 ceph-mon[34770]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T10:05:18.823 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T10:05:19.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:05:18 smithi077 ceph-mon[30319]: pgmap v696: 129 pgs: 129 active+clean; 115 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 818 B/s wr, 10 op/s 2024-09-10T10:05:19.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:05:18 smithi077 ceph-mon[30319]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T10:05:19.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:05:18 smithi077 ceph-mon[30319]: osdmap e98: 12 total, 12 up, 12 in 2024-09-10T10:05:19.199 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:05:19.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:05:18 smithi134 ceph-mon[35674]: pgmap v696: 129 pgs: 129 active+clean; 115 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 818 B/s wr, 10 op/s 2024-09-10T10:05:19.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:05:18 smithi134 ceph-mon[35674]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T10:05:19.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:05:18 smithi134 ceph-mon[35674]: osdmap e98: 12 total, 12 up, 12 in 2024-09-10T10:05:19.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:05:18 smithi120 ceph-mon[34770]: pgmap v696: 129 pgs: 129 active+clean; 115 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 818 B/s wr, 10 op/s 2024-09-10T10:05:19.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:05:18 smithi120 ceph-mon[34770]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T10:05:19.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:05:18 smithi120 ceph-mon[34770]: osdmap e98: 12 total, 12 up, 12 in 2024-09-10T10:05:19.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:19.851+0000 7f97baa00640 1 -- 172.21.15.77:0/2946249117 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f97b4103e30 msgr2=0x7f97b4104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:19.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:19.851+0000 7f97baa00640 1 --2- 172.21.15.77:0/2946249117 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f97b4103e30 0x7f97b4104210 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f97a8009900 tx=0x7f97a802a3f0 comp rx=0 tx=0).stop 2024-09-10T10:05:19.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:19.851+0000 7f97baa00640 1 -- 172.21.15.77:0/2946249117 shutdown_connections 2024-09-10T10:05:19.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:19.851+0000 7f97baa00640 1 --2- 172.21.15.77:0/2946249117 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f97b4108f60 0x7f97b4110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:19.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:19.851+0000 7f97baa00640 1 --2- 172.21.15.77:0/2946249117 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f97b41047e0 0x7f97b4108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:19.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:19.851+0000 7f97baa00640 1 --2- 172.21.15.77:0/2946249117 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f97b4103e30 0x7f97b4104210 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:19.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:19.851+0000 7f97baa00640 1 -- 172.21.15.77:0/2946249117 >> 172.21.15.77:0/2946249117 conn(0x7f97b40ff8d0 msgr2=0x7f97b4101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:19.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:19.851+0000 7f97baa00640 1 -- 172.21.15.77:0/2946249117 shutdown_connections 2024-09-10T10:05:19.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:19.851+0000 7f97baa00640 1 -- 172.21.15.77:0/2946249117 wait complete. 2024-09-10T10:05:19.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:19.852+0000 7f97baa00640 1 Processor -- start 2024-09-10T10:05:19.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:19.853+0000 7f97baa00640 1 -- start start 2024-09-10T10:05:19.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:19.853+0000 7f97baa00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f97b4103e30 0x7f97b419a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:19.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:19.853+0000 7f97baa00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f97b41047e0 0x7f97b419aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:19.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:19.853+0000 7f97baa00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f97b4108f60 0x7f97b419ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:19.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:19.853+0000 7f97baa00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f97b4114010 con 0x7f97b4108f60 2024-09-10T10:05:19.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:19.853+0000 7f97baa00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f97b4113e90 con 0x7f97b4103e30 2024-09-10T10:05:19.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:19.853+0000 7f97baa00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f97b4114190 con 0x7f97b41047e0 2024-09-10T10:05:19.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:19.853+0000 7f97b9600640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f97b4103e30 0x7f97b419a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:19.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:19.853+0000 7f97b9600640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f97b4103e30 0x7f97b419a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.77:50324/0 (socket says 172.21.15.77:50324) 2024-09-10T10:05:19.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:19.853+0000 7f97b9600640 1 -- 172.21.15.77:0/4259613950 learned_addr learned my addr 172.21.15.77:0/4259613950 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:05:19.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:19.853+0000 7f97ba000640 1 --2- 172.21.15.77:0/4259613950 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f97b4108f60 0x7f97b419ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:19.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:19.853+0000 7f97b8c00640 1 --2- 172.21.15.77:0/4259613950 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f97b41047e0 0x7f97b419aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:19.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:19.854+0000 7f97b9600640 1 -- 172.21.15.77:0/4259613950 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f97b41047e0 msgr2=0x7f97b419aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:19.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:19.854+0000 7f97b9600640 1 --2- 172.21.15.77:0/4259613950 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f97b41047e0 0x7f97b419aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:19.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:19.854+0000 7f97b9600640 1 -- 172.21.15.77:0/4259613950 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f97b4108f60 msgr2=0x7f97b419ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:19.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:19.854+0000 7f97b9600640 1 --2- 172.21.15.77:0/4259613950 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f97b4108f60 0x7f97b419ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:19.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:19.854+0000 7f97b9600640 1 -- 172.21.15.77:0/4259613950 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97b419f470 con 0x7f97b4103e30 2024-09-10T10:05:19.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:19.854+0000 7f97b8c00640 1 --2- 172.21.15.77:0/4259613950 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f97b41047e0 0x7f97b419aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:05:19.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:19.854+0000 7f97b9600640 1 --2- 172.21.15.77:0/4259613950 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f97b4103e30 0x7f97b419a4c0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f97a8004770 tx=0x7f97a8003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:19.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:19.854+0000 7f97b2000640 1 -- 172.21.15.77:0/4259613950 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97a803f070 con 0x7f97b4103e30 2024-09-10T10:05:19.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:19.854+0000 7f97b2000640 1 -- 172.21.15.77:0/4259613950 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97a8031ca0 con 0x7f97b4103e30 2024-09-10T10:05:19.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:19.854+0000 7f97baa00640 1 -- 172.21.15.77:0/4259613950 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97b419f700 con 0x7f97b4103e30 2024-09-10T10:05:19.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:19.854+0000 7f97b2000640 1 -- 172.21.15.77:0/4259613950 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97a8035030 con 0x7f97b4103e30 2024-09-10T10:05:19.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:19.854+0000 7f97baa00640 1 -- 172.21.15.77:0/4259613950 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97b41a7040 con 0x7f97b4103e30 2024-09-10T10:05:19.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:19.855+0000 7f97baa00640 1 -- 172.21.15.77:0/4259613950 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97b4105310 con 0x7f97b4103e30 2024-09-10T10:05:19.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:19.857+0000 7f97b2000640 1 -- 172.21.15.77:0/4259613950 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f97a8042070 con 0x7f97b4103e30 2024-09-10T10:05:19.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:19.859+0000 7f97b2000640 1 --2- 172.21.15.77:0/4259613950 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f979c0a2b80 0x7f979c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:19.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:19.859+0000 7f97b2000640 1 -- 172.21.15.77:0/4259613950 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9084+0+0 (secure 0 0 0) 0x7f97a80f0250 con 0x7f97b4103e30 2024-09-10T10:05:19.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:19.859+0000 7f97b8c00640 1 --2- 172.21.15.77:0/4259613950 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f979c0a2b80 0x7f979c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:19.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:19.859+0000 7f97b8c00640 1 --2- 172.21.15.77:0/4259613950 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f979c0a2b80 0x7f979c0a5040 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f97b40ffd60 tx=0x7f97a0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:19.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:19.861+0000 7f97b2000640 1 -- 172.21.15.77:0/4259613950 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f97a80bce00 con 0x7f97b4103e30 2024-09-10T10:05:20.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:20.038+0000 7f97baa00640 1 -- 172.21.15.77:0/4259613950 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f97b410c140 con 0x7f97b4103e30 2024-09-10T10:05:21.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:20.997+0000 7f97b2000640 1 -- 172.21.15.77:0/4259613950 <== mon.2 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v99) ==== 169+0+0 (secure 0 0 0) 0x7f97a80bc700 con 0x7f97b4103e30 2024-09-10T10:05:21.000 INFO:teuthology.orchestra.run.smithi077.stderr:pool 'cephfs_data' removed 2024-09-10T10:05:21.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:21.001+0000 7f97baa00640 1 -- 172.21.15.77:0/4259613950 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f979c0a2b80 msgr2=0x7f979c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:21.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:21.001+0000 7f97baa00640 1 --2- 172.21.15.77:0/4259613950 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f979c0a2b80 0x7f979c0a5040 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f97b40ffd60 tx=0x7f97a0005f90 comp rx=0 tx=0).stop 2024-09-10T10:05:21.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:21.001+0000 7f97baa00640 1 -- 172.21.15.77:0/4259613950 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f97b4103e30 msgr2=0x7f97b419a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:21.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:21.001+0000 7f97baa00640 1 --2- 172.21.15.77:0/4259613950 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f97b4103e30 0x7f97b419a4c0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f97a8004770 tx=0x7f97a8003c10 comp rx=0 tx=0).stop 2024-09-10T10:05:21.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:21.004+0000 7f97baa00640 1 -- 172.21.15.77:0/4259613950 shutdown_connections 2024-09-10T10:05:21.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:21.004+0000 7f97baa00640 1 --2- 172.21.15.77:0/4259613950 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7f979c0a2b80 0x7f979c0a5040 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:21.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:21.004+0000 7f97baa00640 1 --2- 172.21.15.77:0/4259613950 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f97b4108f60 0x7f97b419ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:21.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:21.004+0000 7f97baa00640 1 --2- 172.21.15.77:0/4259613950 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f97b41047e0 0x7f97b419aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:21.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:21.004+0000 7f97baa00640 1 --2- 172.21.15.77:0/4259613950 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f97b4103e30 0x7f97b419a4c0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:21.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:21.004+0000 7f97baa00640 1 -- 172.21.15.77:0/4259613950 >> 172.21.15.77:0/4259613950 conn(0x7f97b40ff8d0 msgr2=0x7f97b4100b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:21.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:21.004+0000 7f97baa00640 1 -- 172.21.15.77:0/4259613950 shutdown_connections 2024-09-10T10:05:21.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:21.004+0000 7f97baa00640 1 -- 172.21.15.77:0/4259613950 wait complete. 2024-09-10T10:05:21.351 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:05:20 smithi077 ceph-mon[30319]: pgmap v698: 97 pgs: 97 active+clean; 577 KiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:05:21.351 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:05:20 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/4259613950' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T10:05:21.351 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:05:20 smithi077 ceph-mon[30319]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T10:05:21.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:05:20 smithi134 ceph-mon[35674]: pgmap v698: 97 pgs: 97 active+clean; 577 KiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:05:21.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:05:20 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/4259613950' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T10:05:21.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:05:20 smithi134 ceph-mon[35674]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T10:05:21.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:05:20 smithi120 ceph-mon[34770]: pgmap v698: 97 pgs: 97 active+clean; 577 KiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:05:21.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:05:20 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/4259613950' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T10:05:21.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:05:20 smithi120 ceph-mon[34770]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T10:05:21.834 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 14a55226-6f59-11ef-bcea-c7b262605968 -- ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-09-10T10:05:22.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:05:21 smithi077 ceph-mon[30319]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T10:05:22.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:05:22 smithi077 ceph-mon[30319]: osdmap e99: 12 total, 12 up, 12 in 2024-09-10T10:05:22.205 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:05:22.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:05:22 smithi134 ceph-mon[35674]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T10:05:22.381 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:05:22 smithi134 ceph-mon[35674]: osdmap e99: 12 total, 12 up, 12 in 2024-09-10T10:05:22.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:05:22 smithi120 ceph-mon[34770]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T10:05:22.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:05:22 smithi120 ceph-mon[34770]: osdmap e99: 12 total, 12 up, 12 in 2024-09-10T10:05:22.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:22.757+0000 7fd60aa00640 1 -- 172.21.15.77:0/4281146926 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd604074ee0 msgr2=0x7fd60410dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:22.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:22.757+0000 7fd60aa00640 1 --2- 172.21.15.77:0/4281146926 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd604074ee0 0x7fd60410dd20 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fd5f0009900 tx=0x7fd5f002a480 comp rx=0 tx=0).stop 2024-09-10T10:05:22.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:22.757+0000 7fd60aa00640 1 -- 172.21.15.77:0/4281146926 shutdown_connections 2024-09-10T10:05:22.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:22.757+0000 7fd60aa00640 1 --2- 172.21.15.77:0/4281146926 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd60410e260 0x7fd604110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:22.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:22.757+0000 7fd60aa00640 1 --2- 172.21.15.77:0/4281146926 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd604074ee0 0x7fd60410dd20 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:22.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:22.757+0000 7fd60aa00640 1 --2- 172.21.15.77:0/4281146926 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd6040745c0 0x7fd6040749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:22.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:22.757+0000 7fd60aa00640 1 -- 172.21.15.77:0/4281146926 >> 172.21.15.77:0/4281146926 conn(0x7fd6040fd740 msgr2=0x7fd6040ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:22.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:22.757+0000 7fd60aa00640 1 -- 172.21.15.77:0/4281146926 shutdown_connections 2024-09-10T10:05:22.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:22.757+0000 7fd60aa00640 1 -- 172.21.15.77:0/4281146926 wait complete. 2024-09-10T10:05:22.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:22.758+0000 7fd60aa00640 1 Processor -- start 2024-09-10T10:05:22.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:22.758+0000 7fd60aa00640 1 -- start start 2024-09-10T10:05:22.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:22.758+0000 7fd60aa00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd6040745c0 0x7fd60419a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:22.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:22.758+0000 7fd60aa00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd604074ee0 0x7fd60419a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:22.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:22.758+0000 7fd60aa00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd60410e260 0x7fd60419ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:22.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:22.758+0000 7fd60aa00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd604113fb0 con 0x7fd604074ee0 2024-09-10T10:05:22.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:22.758+0000 7fd60aa00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd604113e30 con 0x7fd60410e260 2024-09-10T10:05:22.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:22.758+0000 7fd60aa00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd604114130 con 0x7fd6040745c0 2024-09-10T10:05:22.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:22.759+0000 7fd608c00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd604074ee0 0x7fd60419a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:22.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:22.759+0000 7fd608c00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd604074ee0 0x7fd60419a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46704/0 (socket says 172.21.15.77:46704) 2024-09-10T10:05:22.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:22.759+0000 7fd608c00640 1 -- 172.21.15.77:0/2925054807 learned_addr learned my addr 172.21.15.77:0/2925054807 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-10T10:05:22.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:22.759+0000 7fd60a000640 1 --2- 172.21.15.77:0/2925054807 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd60410e260 0x7fd60419ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:22.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:22.759+0000 7fd609600640 1 --2- 172.21.15.77:0/2925054807 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd6040745c0 0x7fd60419a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:22.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:22.759+0000 7fd60a000640 1 -- 172.21.15.77:0/2925054807 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd6040745c0 msgr2=0x7fd60419a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:22.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:22.759+0000 7fd60a000640 1 --2- 172.21.15.77:0/2925054807 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd6040745c0 0x7fd60419a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:22.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:22.759+0000 7fd60a000640 1 -- 172.21.15.77:0/2925054807 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd604074ee0 msgr2=0x7fd60419a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:22.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:22.759+0000 7fd60a000640 1 --2- 172.21.15.77:0/2925054807 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd604074ee0 0x7fd60419a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:22.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:22.759+0000 7fd60a000640 1 -- 172.21.15.77:0/2925054807 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd60419f460 con 0x7fd60410e260 2024-09-10T10:05:22.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:22.760+0000 7fd60a000640 1 --2- 172.21.15.77:0/2925054807 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd60410e260 0x7fd60419ed80 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc00e940 tx=0x7fd5fc00ee00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:22.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:22.760+0000 7fd602000640 1 -- 172.21.15.77:0/2925054807 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5fc00ccc0 con 0x7fd60410e260 2024-09-10T10:05:22.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:22.760+0000 7fd60aa00640 1 -- 172.21.15.77:0/2925054807 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd60419f750 con 0x7fd60410e260 2024-09-10T10:05:22.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:22.760+0000 7fd602000640 1 -- 172.21.15.77:0/2925054807 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5fc005020 con 0x7fd60410e260 2024-09-10T10:05:22.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:22.760+0000 7fd602000640 1 -- 172.21.15.77:0/2925054807 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5fc004a00 con 0x7fd60410e260 2024-09-10T10:05:22.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:22.760+0000 7fd60aa00640 1 -- 172.21.15.77:0/2925054807 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6041a7030 con 0x7fd60410e260 2024-09-10T10:05:22.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:22.761+0000 7fd60aa00640 1 -- 172.21.15.77:0/2925054807 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6040749a0 con 0x7fd60410e260 2024-09-10T10:05:22.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:22.763+0000 7fd602000640 1 -- 172.21.15.77:0/2925054807 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd5fc0051d0 con 0x7fd60410e260 2024-09-10T10:05:22.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:22.764+0000 7fd602000640 1 --2- 172.21.15.77:0/2925054807 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fd5ec0a2b80 0x7fd5ec0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:22.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:22.764+0000 7fd602000640 1 -- 172.21.15.77:0/2925054807 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8496+0+0 (secure 0 0 0) 0x7fd5fc0ceff0 con 0x7fd60410e260 2024-09-10T10:05:22.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:22.765+0000 7fd609600640 1 --2- 172.21.15.77:0/2925054807 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fd5ec0a2b80 0x7fd5ec0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:22.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:22.765+0000 7fd609600640 1 --2- 172.21.15.77:0/2925054807 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fd5ec0a2b80 0x7fd5ec0a5040 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fd5f8005fd0 tx=0x7fd5f8009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:22.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:22.766+0000 7fd602000640 1 -- 172.21.15.77:0/2925054807 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd5fc09be50 con 0x7fd60410e260 2024-09-10T10:05:22.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:22.941+0000 7fd60aa00640 1 -- 172.21.15.77:0/2925054807 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7fd604109440 con 0x7fd60410e260 2024-09-10T10:05:23.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:23.012+0000 7fd602000640 1 -- 172.21.15.77:0/2925054807 <== mon.2 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v100) ==== 178+0+0 (secure 0 0 0) 0x7fd5fc09b750 con 0x7fd60410e260 2024-09-10T10:05:23.014 INFO:teuthology.orchestra.run.smithi077.stderr:pool 'cephfs_data_ec' removed 2024-09-10T10:05:23.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:23.015+0000 7fd60aa00640 1 -- 172.21.15.77:0/2925054807 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fd5ec0a2b80 msgr2=0x7fd5ec0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:23.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:23.015+0000 7fd60aa00640 1 --2- 172.21.15.77:0/2925054807 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fd5ec0a2b80 0x7fd5ec0a5040 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fd5f8005fd0 tx=0x7fd5f8009500 comp rx=0 tx=0).stop 2024-09-10T10:05:23.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:23.015+0000 7fd60aa00640 1 -- 172.21.15.77:0/2925054807 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd60410e260 msgr2=0x7fd60419ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:23.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:23.015+0000 7fd60aa00640 1 --2- 172.21.15.77:0/2925054807 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd60410e260 0x7fd60419ed80 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc00e940 tx=0x7fd5fc00ee00 comp rx=0 tx=0).stop 2024-09-10T10:05:23.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:23.015+0000 7fd60aa00640 1 -- 172.21.15.77:0/2925054807 shutdown_connections 2024-09-10T10:05:23.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:23.015+0000 7fd60aa00640 1 --2- 172.21.15.77:0/2925054807 >> [v2:172.21.15.77:6800/4261324650,v1:172.21.15.77:6801/4261324650] conn(0x7fd5ec0a2b80 0x7fd5ec0a5040 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:23.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:23.015+0000 7fd60aa00640 1 --2- 172.21.15.77:0/2925054807 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd60410e260 0x7fd60419ed80 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:23.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:23.015+0000 7fd60aa00640 1 --2- 172.21.15.77:0/2925054807 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd604074ee0 0x7fd60419a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:23.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:23.015+0000 7fd60aa00640 1 --2- 172.21.15.77:0/2925054807 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd6040745c0 0x7fd60419a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:23.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:23.015+0000 7fd60aa00640 1 -- 172.21.15.77:0/2925054807 >> 172.21.15.77:0/2925054807 conn(0x7fd6040fd740 msgr2=0x7fd6040fe440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:23.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:23.018+0000 7fd60aa00640 1 -- 172.21.15.77:0/2925054807 shutdown_connections 2024-09-10T10:05:23.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-10T10:05:23.018+0000 7fd60aa00640 1 -- 172.21.15.77:0/2925054807 wait complete. 2024-09-10T10:05:23.282 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:05:23 smithi134 ceph-mon[35674]: pgmap v700: 33 pgs: 33 active+clean; 577 KiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:05:23.283 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:05:23 smithi134 ceph-mon[35674]: from='client.? 172.21.15.77:0/2925054807' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T10:05:23.283 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:05:23 smithi134 ceph-mon[35674]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T10:05:23.389 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:05:23 smithi077 ceph-mon[30319]: pgmap v700: 33 pgs: 33 active+clean; 577 KiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:05:23.390 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:05:23 smithi077 ceph-mon[30319]: from='client.? 172.21.15.77:0/2925054807' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T10:05:23.390 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:05:23 smithi077 ceph-mon[30319]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T10:05:23.423 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:05:23 smithi120 ceph-mon[34770]: pgmap v700: 33 pgs: 33 active+clean; 577 KiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:05:23.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:05:23 smithi120 ceph-mon[34770]: from='client.? 172.21.15.77:0/2925054807' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T10:05:23.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:05:23 smithi120 ceph-mon[34770]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T10:05:23.746 DEBUG:teuthology.orchestra.run.smithi077:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T10:05:23.803 DEBUG:teuthology.orchestra.run.smithi120:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T10:05:23.838 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T10:05:23.870 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-10T10:05:23.870 DEBUG:teuthology.orchestra.run.smithi077:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-10T10:05:23.905 DEBUG:teuthology.orchestra.run.smithi120:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-10T10:05:23.924 DEBUG:teuthology.orchestra.run.smithi134:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-10T10:05:23.939 INFO:tasks.cephadm:Stopping all daemons... 2024-09-10T10:05:23.939 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-10T10:05:23.940 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ceph-14a55226-6f59-11ef-bcea-c7b262605968@mon.a 2024-09-10T10:05:24.111 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:05:24 smithi134 ceph-mon[35674]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T10:05:24.112 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:05:24 smithi134 ceph-mon[35674]: osdmap e100: 12 total, 12 up, 12 in 2024-09-10T10:05:24.173 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:05:24 smithi120 ceph-mon[34770]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T10:05:24.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:05:24 smithi120 ceph-mon[34770]: osdmap e100: 12 total, 12 up, 12 in 2024-09-10T10:05:24.334 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:05:24 smithi077 ceph-mon[30319]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T10:05:24.334 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:05:24 smithi077 ceph-mon[30319]: osdmap e100: 12 total, 12 up, 12 in 2024-09-10T10:05:24.334 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:05:24 smithi077 systemd[1]: Stopping Ceph mon.a for 14a55226-6f59-11ef-bcea-c7b262605968... 2024-09-10T10:05:24.590 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:05:24 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mon-a[30313]: 2024-09-10T10:05:24.331+0000 7f6b2d000640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:05:24.590 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:05:24 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mon-a[30313]: 2024-09-10T10:05:24.331+0000 7f6b2d000640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-10T10:05:24.906 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:05:24 smithi077 podman[127452]: 2024-09-10 10:05:24.589893481 +0000 UTC m=+0.364776383 container died f32e55b70c78a46742af30d82a80ca5f00f5d5478a5be5faf690e79bd873f941 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-mon-a, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T10:05:25.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:05:25 smithi077 podman[127452]: 2024-09-10 10:05:25.029238659 +0000 UTC m=+0.804121557 container cleanup f32e55b70c78a46742af30d82a80ca5f00f5d5478a5be5faf690e79bd873f941 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-mon-a, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-10T10:05:25.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:05:25 smithi077 bash[127452]: ceph-14a55226-6f59-11ef-bcea-c7b262605968-mon-a 2024-09-10T10:05:25.515 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 10 10:05:25 smithi077 podman[127466]: 2024-09-10 10:05:25.264734754 +0000 UTC m=+0.666856648 container remove f32e55b70c78a46742af30d82a80ca5f00f5d5478a5be5faf690e79bd873f941 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-mon-a, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-10T10:05:25.525 DEBUG:teuthology.orchestra.run.smithi077:> sudo pkill -f 'journalctl -f -n 0 -u ceph-14a55226-6f59-11ef-bcea-c7b262605968@mon.a.service' 2024-09-10T10:05:25.616 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:05:25.616 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-10T10:05:25.616 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-09-10T10:05:25.617 DEBUG:teuthology.orchestra.run.smithi120:> sudo systemctl stop ceph-14a55226-6f59-11ef-bcea-c7b262605968@mon.b 2024-09-10T10:05:25.924 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:05:25 smithi120 systemd[1]: Stopping Ceph mon.b for 14a55226-6f59-11ef-bcea-c7b262605968... 2024-09-10T10:05:26.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:05:26 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mon-b[34766]: 2024-09-10T10:05:26.017+0000 7fa50ee80640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:05:26.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:05:26 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mon-b[34766]: 2024-09-10T10:05:26.017+0000 7fa50ee80640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-09-10T10:05:26.424 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:05:26 smithi120 podman[69448]: 2024-09-10 10:05:26.150602006 +0000 UTC m=+0.335093129 container died fc48b4c41882bdb7707c2ee05be81d6817e86874c6152a0c3f947b34ffc360b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-mon-b, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T10:05:26.804 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:05:26 smithi120 podman[69448]: 2024-09-10 10:05:26.560383249 +0000 UTC m=+0.744874365 container cleanup fc48b4c41882bdb7707c2ee05be81d6817e86874c6152a0c3f947b34ffc360b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-mon-b, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-10T10:05:26.804 INFO:journalctl@ceph.mon.b.smithi120.stdout:Sep 10 10:05:26 smithi120 bash[69448]: ceph-14a55226-6f59-11ef-bcea-c7b262605968-mon-b 2024-09-10T10:05:26.932 DEBUG:teuthology.orchestra.run.smithi120:> sudo pkill -f 'journalctl -f -n 0 -u ceph-14a55226-6f59-11ef-bcea-c7b262605968@mon.b.service' 2024-09-10T10:05:27.010 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:05:27.011 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-09-10T10:05:27.011 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-09-10T10:05:27.011 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-14a55226-6f59-11ef-bcea-c7b262605968@mon.c 2024-09-10T10:05:27.361 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:05:27 smithi134 systemd[1]: Stopping Ceph mon.c for 14a55226-6f59-11ef-bcea-c7b262605968... 2024-09-10T10:05:27.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:05:27 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mon-c[35670]: 2024-09-10T10:05:27.358+0000 7f01d9dc8640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:05:27.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:05:27 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mon-c[35670]: 2024-09-10T10:05:27.358+0000 7f01d9dc8640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-09-10T10:05:27.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:05:27 smithi134 podman[71356]: 2024-09-10 10:05:27.502560567 +0000 UTC m=+0.297121681 container died aa9effc066dae1348e4b2ead4d4c29a101fbef8980fe658dc90cd9f0c82ff9d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-mon-c, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-10T10:05:28.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:05:27 smithi134 podman[71356]: 2024-09-10 10:05:27.893630544 +0000 UTC m=+0.688191693 container cleanup aa9effc066dae1348e4b2ead4d4c29a101fbef8980fe658dc90cd9f0c82ff9d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-mon-c, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T10:05:28.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 10 10:05:27 smithi134 bash[71356]: ceph-14a55226-6f59-11ef-bcea-c7b262605968-mon-c 2024-09-10T10:05:28.317 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-14a55226-6f59-11ef-bcea-c7b262605968@mon.c.service' 2024-09-10T10:05:28.363 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:05:28.363 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-09-10T10:05:28.364 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-10T10:05:28.364 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ceph-14a55226-6f59-11ef-bcea-c7b262605968@mgr.x 2024-09-10T10:05:28.689 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 10:05:28 smithi077 systemd[1]: Stopping Ceph mgr.x for 14a55226-6f59-11ef-bcea-c7b262605968... 2024-09-10T10:05:29.187 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 10:05:28 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: 2024-09-10T10:05:28.786+0000 7f28a3c00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:05:29.187 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 10:05:28 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x[30550]: 2024-09-10T10:05:28.786+0000 7f28a3c00640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-09-10T10:05:29.187 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 10:05:28 smithi077 podman[127572]: 2024-09-10 10:05:28.871177764 +0000 UTC m=+0.212730435 container died 688587491f1e07a2e7e61da7d709e0072094a3f2899b67821b5c7be390199740 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-10T10:05:29.439 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 10:05:29 smithi077 podman[127572]: 2024-09-10 10:05:29.343112098 +0000 UTC m=+0.684664768 container cleanup 688587491f1e07a2e7e61da7d709e0072094a3f2899b67821b5c7be390199740 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-10T10:05:29.439 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 10:05:29 smithi077 bash[127572]: ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x 2024-09-10T10:05:29.854 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 10 10:05:29 smithi077 podman[127583]: 2024-09-10 10:05:29.578545496 +0000 UTC m=+0.697873640 container remove 688587491f1e07a2e7e61da7d709e0072094a3f2899b67821b5c7be390199740 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-x, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-10T10:05:29.873 DEBUG:teuthology.orchestra.run.smithi077:> sudo pkill -f 'journalctl -f -n 0 -u ceph-14a55226-6f59-11ef-bcea-c7b262605968@mgr.x.service' 2024-09-10T10:05:29.947 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:05:29.947 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-10T10:05:29.947 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-10T10:05:29.947 DEBUG:teuthology.orchestra.run.smithi120:> sudo systemctl stop ceph-14a55226-6f59-11ef-bcea-c7b262605968@mgr.y 2024-09-10T10:05:30.423 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Sep 10 10:05:30 smithi120 systemd[1]: Stopping Ceph mgr.y for 14a55226-6f59-11ef-bcea-c7b262605968... 2024-09-10T10:05:30.816 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Sep 10 10:05:30 smithi120 podman[69573]: 2024-09-10 10:05:30.519639668 +0000 UTC m=+0.372400468 container died 09458c250d4107ff8decf443fc76e0fbc61d33d65eb4ef9437fba99958ff970c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-10T10:05:31.174 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Sep 10 10:05:30 smithi120 podman[69573]: 2024-09-10 10:05:30.907305541 +0000 UTC m=+0.760066341 container cleanup 09458c250d4107ff8decf443fc76e0fbc61d33d65eb4ef9437fba99958ff970c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-10T10:05:31.174 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Sep 10 10:05:30 smithi120 bash[69573]: ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-y 2024-09-10T10:05:31.345 DEBUG:teuthology.orchestra.run.smithi120:> sudo pkill -f 'journalctl -f -n 0 -u ceph-14a55226-6f59-11ef-bcea-c7b262605968@mgr.y.service' 2024-09-10T10:05:31.379 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:05:31.379 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-10T10:05:31.380 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-09-10T10:05:31.380 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-14a55226-6f59-11ef-bcea-c7b262605968@mgr.z 2024-09-10T10:05:31.720 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Sep 10 10:05:31 smithi134 systemd[1]: Stopping Ceph mgr.z for 14a55226-6f59-11ef-bcea-c7b262605968... 2024-09-10T10:05:32.026 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Sep 10 10:05:31 smithi134 podman[71479]: 2024-09-10 10:05:31.719792073 +0000 UTC m=+0.145564369 container died 84161fdf796da7108462a7154931c9fcf956bb7068d7f5cb39404e71f5b08d9c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-z, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-10T10:05:32.360 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Sep 10 10:05:32 smithi134 podman[71479]: 2024-09-10 10:05:32.116221707 +0000 UTC m=+0.541993997 container cleanup 84161fdf796da7108462a7154931c9fcf956bb7068d7f5cb39404e71f5b08d9c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-z, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T10:05:32.360 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Sep 10 10:05:32 smithi134 bash[71479]: ceph-14a55226-6f59-11ef-bcea-c7b262605968-mgr-z 2024-09-10T10:05:32.504 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-14a55226-6f59-11ef-bcea-c7b262605968@mgr.z.service' 2024-09-10T10:05:32.538 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:05:32.538 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-09-10T10:05:32.539 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-10T10:05:32.539 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ceph-14a55226-6f59-11ef-bcea-c7b262605968@osd.0 2024-09-10T10:05:32.939 INFO:journalctl@ceph.osd.0.smithi077.stdout:Sep 10 10:05:32 smithi077 systemd[1]: Stopping Ceph osd.0 for 14a55226-6f59-11ef-bcea-c7b262605968... 2024-09-10T10:05:33.439 INFO:journalctl@ceph.osd.0.smithi077.stdout:Sep 10 10:05:32 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-0[44004]: 2024-09-10T10:05:32.958+0000 7f40ec200640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:05:33.439 INFO:journalctl@ceph.osd.0.smithi077.stdout:Sep 10 10:05:32 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-0[44004]: 2024-09-10T10:05:32.958+0000 7f40ec200640 -1 osd.0 100 *** Got signal Terminated *** 2024-09-10T10:05:33.440 INFO:journalctl@ceph.osd.0.smithi077.stdout:Sep 10 10:05:32 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-0[44004]: 2024-09-10T10:05:32.958+0000 7f40ec200640 -1 osd.0 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T10:05:38.689 INFO:journalctl@ceph.osd.0.smithi077.stdout:Sep 10 10:05:38 smithi077 podman[127693]: 2024-09-10 10:05:38.349829836 +0000 UTC m=+5.523738613 container died 9f906f9f1f3e7a6e80f84b27a6f1bd0484b1aa0312111afdcaf8ba65389f9b31 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T10:05:39.134 INFO:journalctl@ceph.osd.0.smithi077.stdout:Sep 10 10:05:38 smithi077 podman[127693]: 2024-09-10 10:05:38.915223624 +0000 UTC m=+6.089132396 container cleanup 9f906f9f1f3e7a6e80f84b27a6f1bd0484b1aa0312111afdcaf8ba65389f9b31 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD) 2024-09-10T10:05:39.134 INFO:journalctl@ceph.osd.0.smithi077.stdout:Sep 10 10:05:38 smithi077 bash[127693]: ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-0 2024-09-10T10:05:39.439 INFO:journalctl@ceph.osd.0.smithi077.stdout:Sep 10 10:05:39 smithi077 podman[127708]: 2024-09-10 10:05:39.133421188 +0000 UTC m=+0.774640091 container remove 9f906f9f1f3e7a6e80f84b27a6f1bd0484b1aa0312111afdcaf8ba65389f9b31 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T10:05:39.938 INFO:journalctl@ceph.osd.0.smithi077.stdout:Sep 10 10:05:39 smithi077 podman[127795]: 2024-09-10 10:05:39.676250537 +0000 UTC m=+0.120623793 container create 433b62c6faf09ef4b3e1df1f7f41e1c0464027550eacb236a73448d12546989f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T10:05:39.938 INFO:journalctl@ceph.osd.0.smithi077.stdout:Sep 10 10:05:39 smithi077 podman[127795]: 2024-09-10 10:05:39.583040813 +0000 UTC m=+0.027414029 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T10:05:39.938 INFO:journalctl@ceph.osd.0.smithi077.stdout:Sep 10 10:05:39 smithi077 podman[127795]: 2024-09-10 10:05:39.937363197 +0000 UTC m=+0.381736413 container init 433b62c6faf09ef4b3e1df1f7f41e1c0464027550eacb236a73448d12546989f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T10:05:40.189 INFO:journalctl@ceph.osd.0.smithi077.stdout:Sep 10 10:05:39 smithi077 podman[127795]: 2024-09-10 10:05:39.944126007 +0000 UTC m=+0.388499233 container start 433b62c6faf09ef4b3e1df1f7f41e1c0464027550eacb236a73448d12546989f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731) 2024-09-10T10:05:40.189 INFO:journalctl@ceph.osd.0.smithi077.stdout:Sep 10 10:05:40 smithi077 podman[127795]: 2024-09-10 10:05:40.045384485 +0000 UTC m=+0.489757727 container attach 433b62c6faf09ef4b3e1df1f7f41e1c0464027550eacb236a73448d12546989f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-10T10:05:40.607 INFO:journalctl@ceph.osd.0.smithi077.stdout:Sep 10 10:05:40 smithi077 podman[127795]: 2024-09-10 10:05:40.200806974 +0000 UTC m=+0.645180199 container died 433b62c6faf09ef4b3e1df1f7f41e1c0464027550eacb236a73448d12546989f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux ) 2024-09-10T10:05:40.972 DEBUG:teuthology.orchestra.run.smithi077:> sudo pkill -f 'journalctl -f -n 0 -u ceph-14a55226-6f59-11ef-bcea-c7b262605968@osd.0.service' 2024-09-10T10:05:41.047 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:05:41.047 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-10T10:05:41.048 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-10T10:05:41.048 DEBUG:teuthology.orchestra.run.smithi120:> sudo systemctl stop ceph-14a55226-6f59-11ef-bcea-c7b262605968@osd.1 2024-09-10T10:05:41.372 INFO:journalctl@ceph.osd.1.smithi120.stdout:Sep 10 10:05:41 smithi120 systemd[1]: Stopping Ceph osd.1 for 14a55226-6f59-11ef-bcea-c7b262605968... 2024-09-10T10:05:41.673 INFO:journalctl@ceph.osd.1.smithi120.stdout:Sep 10 10:05:41 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-1[39249]: 2024-09-10T10:05:41.370+0000 7f81c56c4640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:05:41.674 INFO:journalctl@ceph.osd.1.smithi120.stdout:Sep 10 10:05:41 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-1[39249]: 2024-09-10T10:05:41.370+0000 7f81c56c4640 -1 osd.1 100 *** Got signal Terminated *** 2024-09-10T10:05:41.674 INFO:journalctl@ceph.osd.1.smithi120.stdout:Sep 10 10:05:41 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-1[39249]: 2024-09-10T10:05:41.370+0000 7f81c56c4640 -1 osd.1 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T10:05:47.174 INFO:journalctl@ceph.osd.1.smithi120.stdout:Sep 10 10:05:46 smithi120 podman[69698]: 2024-09-10 10:05:46.879061273 +0000 UTC m=+5.632905712 container died 9f3aa9a398fd29757304248afb93d09c0dbfed99b3879aef4af7f4cf878d571a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-1, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T10:05:47.555 INFO:journalctl@ceph.osd.1.smithi120.stdout:Sep 10 10:05:47 smithi120 podman[69698]: 2024-09-10 10:05:47.278378452 +0000 UTC m=+6.032222890 container cleanup 9f3aa9a398fd29757304248afb93d09c0dbfed99b3879aef4af7f4cf878d571a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True) 2024-09-10T10:05:47.556 INFO:journalctl@ceph.osd.1.smithi120.stdout:Sep 10 10:05:47 smithi120 bash[69698]: ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-1 2024-09-10T10:05:47.907 INFO:journalctl@ceph.osd.1.smithi120.stdout:Sep 10 10:05:47 smithi120 podman[69713]: 2024-09-10 10:05:47.55496944 +0000 UTC m=+0.675326871 container remove 9f3aa9a398fd29757304248afb93d09c0dbfed99b3879aef4af7f4cf878d571a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-1, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0) 2024-09-10T10:05:48.168 INFO:journalctl@ceph.osd.1.smithi120.stdout:Sep 10 10:05:47 smithi120 podman[69811]: 2024-09-10 10:05:47.809112145 +0000 UTC m=+0.016109436 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T10:05:48.168 INFO:journalctl@ceph.osd.1.smithi120.stdout:Sep 10 10:05:47 smithi120 podman[69811]: 2024-09-10 10:05:47.93123935 +0000 UTC m=+0.138236628 container create b545449a4071fe9e9be57240ed007e87ac74e87bd774f7a789418ff1036a9f61 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-10T10:05:48.168 INFO:journalctl@ceph.osd.1.smithi120.stdout:Sep 10 10:05:48 smithi120 podman[69811]: 2024-09-10 10:05:48.167653942 +0000 UTC m=+0.374651266 container init b545449a4071fe9e9be57240ed007e87ac74e87bd774f7a789418ff1036a9f61 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T10:05:48.424 INFO:journalctl@ceph.osd.1.smithi120.stdout:Sep 10 10:05:48 smithi120 podman[69811]: 2024-09-10 10:05:48.171617143 +0000 UTC m=+0.378614430 container start b545449a4071fe9e9be57240ed007e87ac74e87bd774f7a789418ff1036a9f61 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-1-deactivate, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T10:05:48.424 INFO:journalctl@ceph.osd.1.smithi120.stdout:Sep 10 10:05:48 smithi120 podman[69811]: 2024-09-10 10:05:48.250658679 +0000 UTC m=+0.457655975 container attach b545449a4071fe9e9be57240ed007e87ac74e87bd774f7a789418ff1036a9f61 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-10T10:05:48.424 INFO:journalctl@ceph.osd.1.smithi120.stdout:Sep 10 10:05:48 smithi120 podman[69811]: 2024-09-10 10:05:48.372608046 +0000 UTC m=+0.579605335 container died b545449a4071fe9e9be57240ed007e87ac74e87bd774f7a789418ff1036a9f61 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731) 2024-09-10T10:05:49.065 DEBUG:teuthology.orchestra.run.smithi120:> sudo pkill -f 'journalctl -f -n 0 -u ceph-14a55226-6f59-11ef-bcea-c7b262605968@osd.1.service' 2024-09-10T10:05:49.104 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:05:49.104 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-10T10:05:49.105 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-10T10:05:49.105 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-14a55226-6f59-11ef-bcea-c7b262605968@osd.2 2024-09-10T10:05:49.413 INFO:journalctl@ceph.osd.2.smithi134.stdout:Sep 10 10:05:49 smithi134 systemd[1]: Stopping Ceph osd.2 for 14a55226-6f59-11ef-bcea-c7b262605968... 2024-09-10T10:05:49.881 INFO:journalctl@ceph.osd.2.smithi134.stdout:Sep 10 10:05:49 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-2[40655]: 2024-09-10T10:05:49.411+0000 7f3e6b9ef640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:05:49.881 INFO:journalctl@ceph.osd.2.smithi134.stdout:Sep 10 10:05:49 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-2[40655]: 2024-09-10T10:05:49.411+0000 7f3e6b9ef640 -1 osd.2 100 *** Got signal Terminated *** 2024-09-10T10:05:49.881 INFO:journalctl@ceph.osd.2.smithi134.stdout:Sep 10 10:05:49 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-2[40655]: 2024-09-10T10:05:49.411+0000 7f3e6b9ef640 -1 osd.2 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T10:05:55.231 INFO:journalctl@ceph.osd.2.smithi134.stdout:Sep 10 10:05:54 smithi134 podman[71603]: 2024-09-10 10:05:54.914280197 +0000 UTC m=+5.620895570 container died 9fea96d4a0b41b26db69577cf76142d5108abd7c32440e0e2ecac89a350fc8ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux ) 2024-09-10T10:05:55.557 INFO:journalctl@ceph.osd.2.smithi134.stdout:Sep 10 10:05:55 smithi134 podman[71603]: 2024-09-10 10:05:55.337686656 +0000 UTC m=+6.044302027 container cleanup 9fea96d4a0b41b26db69577cf76142d5108abd7c32440e0e2ecac89a350fc8ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T10:05:55.557 INFO:journalctl@ceph.osd.2.smithi134.stdout:Sep 10 10:05:55 smithi134 bash[71603]: ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-2 2024-09-10T10:05:55.881 INFO:journalctl@ceph.osd.2.smithi134.stdout:Sep 10 10:05:55 smithi134 podman[71618]: 2024-09-10 10:05:55.556299485 +0000 UTC m=+0.641535578 container remove 9fea96d4a0b41b26db69577cf76142d5108abd7c32440e0e2ecac89a350fc8ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-10T10:05:56.235 INFO:journalctl@ceph.osd.2.smithi134.stdout:Sep 10 10:05:55 smithi134 podman[71709]: 2024-09-10 10:05:55.85815225 +0000 UTC m=+0.015854130 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T10:05:56.235 INFO:journalctl@ceph.osd.2.smithi134.stdout:Sep 10 10:05:55 smithi134 podman[71709]: 2024-09-10 10:05:55.982199749 +0000 UTC m=+0.139901621 container create 395ace8cca01bff304d2fa453642b50afcdd53bf6aec6eb81f59631e0eb66a80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-10T10:05:56.235 INFO:journalctl@ceph.osd.2.smithi134.stdout:Sep 10 10:05:56 smithi134 podman[71709]: 2024-09-10 10:05:56.234839193 +0000 UTC m=+0.392541066 container init 395ace8cca01bff304d2fa453642b50afcdd53bf6aec6eb81f59631e0eb66a80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-10T10:05:56.631 INFO:journalctl@ceph.osd.2.smithi134.stdout:Sep 10 10:05:56 smithi134 podman[71709]: 2024-09-10 10:05:56.238035242 +0000 UTC m=+0.395737121 container start 395ace8cca01bff304d2fa453642b50afcdd53bf6aec6eb81f59631e0eb66a80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-10T10:05:56.631 INFO:journalctl@ceph.osd.2.smithi134.stdout:Sep 10 10:05:56 smithi134 podman[71709]: 2024-09-10 10:05:56.335077524 +0000 UTC m=+0.492779405 container attach 395ace8cca01bff304d2fa453642b50afcdd53bf6aec6eb81f59631e0eb66a80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-10T10:05:56.632 INFO:journalctl@ceph.osd.2.smithi134.stdout:Sep 10 10:05:56 smithi134 podman[71709]: 2024-09-10 10:05:56.433573669 +0000 UTC m=+0.591275546 container died 395ace8cca01bff304d2fa453642b50afcdd53bf6aec6eb81f59631e0eb66a80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-10T10:05:57.125 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-14a55226-6f59-11ef-bcea-c7b262605968@osd.2.service' 2024-09-10T10:05:57.165 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:05:57.165 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-10T10:05:57.165 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-10T10:05:57.165 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ceph-14a55226-6f59-11ef-bcea-c7b262605968@osd.3 2024-09-10T10:05:57.573 INFO:journalctl@ceph.osd.3.smithi077.stdout:Sep 10 10:05:57 smithi077 systemd[1]: Stopping Ceph osd.3 for 14a55226-6f59-11ef-bcea-c7b262605968... 2024-09-10T10:05:57.939 INFO:journalctl@ceph.osd.3.smithi077.stdout:Sep 10 10:05:57 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-3[50752]: 2024-09-10T10:05:57.571+0000 7fceb0a00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:05:57.939 INFO:journalctl@ceph.osd.3.smithi077.stdout:Sep 10 10:05:57 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-3[50752]: 2024-09-10T10:05:57.571+0000 7fceb0a00640 -1 osd.3 100 *** Got signal Terminated *** 2024-09-10T10:05:57.939 INFO:journalctl@ceph.osd.3.smithi077.stdout:Sep 10 10:05:57 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-3[50752]: 2024-09-10T10:05:57.571+0000 7fceb0a00640 -1 osd.3 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T10:06:00.132 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:05:59 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:05:59.812+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:34.146460+0000 front 2024-09-10T10:05:34.146398+0000 (oldest deadline 2024-09-10T10:05:59.446520+0000) 2024-09-10T10:06:00.631 INFO:journalctl@ceph.osd.5.smithi134.stdout:Sep 10 10:06:00 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-5[47540]: 2024-09-10T10:06:00.134+0000 7f371db00640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:36.760837+0000 front 2024-09-10T10:05:36.760719+0000 (oldest deadline 2024-09-10T10:05:59.660697+0000) 2024-09-10T10:06:00.689 INFO:journalctl@ceph.osd.3.smithi077.stdout:Sep 10 10:06:00 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-3[50752]: 2024-09-10T10:06:00.299+0000 7fcea7400640 -1 osd.3 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.280563+0000 front 2024-09-10T10:05:37.280597+0000 (oldest deadline 2024-09-10T10:05:59.580825+0000) 2024-09-10T10:06:01.131 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:00 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:00.850+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:34.146460+0000 front 2024-09-10T10:05:34.146398+0000 (oldest deadline 2024-09-10T10:05:59.446520+0000) 2024-09-10T10:06:01.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:00 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:00.988+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.045437+0000 front 2024-09-10T10:05:37.045486+0000 (oldest deadline 2024-09-10T10:05:59.945393+0000) 2024-09-10T10:06:01.632 INFO:journalctl@ceph.osd.5.smithi134.stdout:Sep 10 10:06:01 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-5[47540]: 2024-09-10T10:06:01.130+0000 7f371db00640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:36.760837+0000 front 2024-09-10T10:05:36.760719+0000 (oldest deadline 2024-09-10T10:05:59.660697+0000) 2024-09-10T10:06:01.689 INFO:journalctl@ceph.osd.3.smithi077.stdout:Sep 10 10:06:01 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-3[50752]: 2024-09-10T10:06:01.261+0000 7fcea7400640 -1 osd.3 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.280563+0000 front 2024-09-10T10:05:37.280597+0000 (oldest deadline 2024-09-10T10:05:59.580825+0000) 2024-09-10T10:06:02.179 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:01 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:01.897+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:34.146460+0000 front 2024-09-10T10:05:34.146398+0000 (oldest deadline 2024-09-10T10:05:59.446520+0000) 2024-09-10T10:06:02.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:01 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:01.961+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.045437+0000 front 2024-09-10T10:05:37.045486+0000 (oldest deadline 2024-09-10T10:05:59.945393+0000) 2024-09-10T10:06:02.631 INFO:journalctl@ceph.osd.5.smithi134.stdout:Sep 10 10:06:02 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-5[47540]: 2024-09-10T10:06:02.177+0000 7f371db00640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:36.760837+0000 front 2024-09-10T10:05:36.760719+0000 (oldest deadline 2024-09-10T10:05:59.660697+0000) 2024-09-10T10:06:02.689 INFO:journalctl@ceph.osd.3.smithi077.stdout:Sep 10 10:06:02 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-3[50752]: 2024-09-10T10:06:02.310+0000 7fcea7400640 -1 osd.3 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.280563+0000 front 2024-09-10T10:05:37.280597+0000 (oldest deadline 2024-09-10T10:05:59.580825+0000) 2024-09-10T10:06:03.362 INFO:journalctl@ceph.osd.3.smithi077.stdout:Sep 10 10:06:03 smithi077 podman[127897]: 2024-09-10 10:06:03.058511512 +0000 UTC m=+5.616125690 container died 1e8f5b36d2938f4dc06ce142551cf796d31d4083be67e2e22e1f83adbf806130 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-3, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2) 2024-09-10T10:06:03.381 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:02 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:02.943+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:34.146460+0000 front 2024-09-10T10:05:34.146398+0000 (oldest deadline 2024-09-10T10:05:59.446520+0000) 2024-09-10T10:06:03.381 INFO:journalctl@ceph.osd.5.smithi134.stdout:Sep 10 10:06:03 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-5[47540]: 2024-09-10T10:06:03.181+0000 7f371db00640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:36.760837+0000 front 2024-09-10T10:05:36.760719+0000 (oldest deadline 2024-09-10T10:05:59.660697+0000) 2024-09-10T10:06:03.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:03 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:03.007+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.045437+0000 front 2024-09-10T10:05:37.045486+0000 (oldest deadline 2024-09-10T10:05:59.945393+0000) 2024-09-10T10:06:03.622 INFO:journalctl@ceph.osd.3.smithi077.stdout:Sep 10 10:06:03 smithi077 podman[127897]: 2024-09-10 10:06:03.602448063 +0000 UTC m=+6.160062234 container cleanup 1e8f5b36d2938f4dc06ce142551cf796d31d4083be67e2e22e1f83adbf806130 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-3, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-10T10:06:03.622 INFO:journalctl@ceph.osd.3.smithi077.stdout:Sep 10 10:06:03 smithi077 bash[127897]: ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-3 2024-09-10T10:06:04.189 INFO:journalctl@ceph.osd.3.smithi077.stdout:Sep 10 10:06:03 smithi077 podman[127915]: 2024-09-10 10:06:03.820867667 +0000 UTC m=+0.750199451 container remove 1e8f5b36d2938f4dc06ce142551cf796d31d4083be67e2e22e1f83adbf806130 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-3, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-10T10:06:04.189 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:03 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:03.839+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.086443+0000 front 2024-09-10T10:05:37.086361+0000 (oldest deadline 2024-09-10T10:06:02.887395+0000) 2024-09-10T10:06:04.381 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:03 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:03.952+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:34.146460+0000 front 2024-09-10T10:05:34.146398+0000 (oldest deadline 2024-09-10T10:05:59.446520+0000) 2024-09-10T10:06:04.381 INFO:journalctl@ceph.osd.5.smithi134.stdout:Sep 10 10:06:04 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-5[47540]: 2024-09-10T10:06:04.157+0000 7f371db00640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:36.760837+0000 front 2024-09-10T10:05:36.760719+0000 (oldest deadline 2024-09-10T10:05:59.660697+0000) 2024-09-10T10:06:04.423 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:04 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:04.031+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.045437+0000 front 2024-09-10T10:05:37.045486+0000 (oldest deadline 2024-09-10T10:05:59.945393+0000) 2024-09-10T10:06:04.666 INFO:journalctl@ceph.osd.3.smithi077.stdout:Sep 10 10:06:04 smithi077 podman[128006]: 2024-09-10 10:06:04.354781091 +0000 UTC m=+0.115424093 container create f1f996a5aa9f61e18fd44b3b0655a0593447e7f985ee334aab45d5d7b1b138b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731) 2024-09-10T10:06:04.667 INFO:journalctl@ceph.osd.3.smithi077.stdout:Sep 10 10:06:04 smithi077 podman[128006]: 2024-09-10 10:06:04.266482024 +0000 UTC m=+0.027125020 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T10:06:04.667 INFO:journalctl@ceph.osd.3.smithi077.stdout:Sep 10 10:06:04 smithi077 podman[128006]: 2024-09-10 10:06:04.666068166 +0000 UTC m=+0.426711175 container init f1f996a5aa9f61e18fd44b3b0655a0593447e7f985ee334aab45d5d7b1b138b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T10:06:04.930 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:04 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:04.842+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.086443+0000 front 2024-09-10T10:05:37.086361+0000 (oldest deadline 2024-09-10T10:06:02.887395+0000) 2024-09-10T10:06:04.930 INFO:journalctl@ceph.osd.3.smithi077.stdout:Sep 10 10:06:04 smithi077 podman[128006]: 2024-09-10 10:06:04.673044139 +0000 UTC m=+0.433687144 container start f1f996a5aa9f61e18fd44b3b0655a0593447e7f985ee334aab45d5d7b1b138b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-3-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, ceph=True) 2024-09-10T10:06:04.930 INFO:journalctl@ceph.osd.3.smithi077.stdout:Sep 10 10:06:04 smithi077 podman[128006]: 2024-09-10 10:06:04.773925565 +0000 UTC m=+0.534568573 container attach f1f996a5aa9f61e18fd44b3b0655a0593447e7f985ee334aab45d5d7b1b138b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T10:06:05.189 INFO:journalctl@ceph.osd.3.smithi077.stdout:Sep 10 10:06:04 smithi077 podman[128006]: 2024-09-10 10:06:04.930041556 +0000 UTC m=+0.690684566 container died f1f996a5aa9f61e18fd44b3b0655a0593447e7f985ee334aab45d5d7b1b138b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-10T10:06:05.381 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:04 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:04.986+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:34.146460+0000 front 2024-09-10T10:05:34.146398+0000 (oldest deadline 2024-09-10T10:05:59.446520+0000) 2024-09-10T10:06:05.381 INFO:journalctl@ceph.osd.5.smithi134.stdout:Sep 10 10:06:05 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-5[47540]: 2024-09-10T10:06:05.163+0000 7f371db00640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:36.760837+0000 front 2024-09-10T10:05:36.760719+0000 (oldest deadline 2024-09-10T10:05:59.660697+0000) 2024-09-10T10:06:05.423 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:05 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:04.999+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.045437+0000 front 2024-09-10T10:05:37.045486+0000 (oldest deadline 2024-09-10T10:05:59.945393+0000) 2024-09-10T10:06:05.748 DEBUG:teuthology.orchestra.run.smithi077:> sudo pkill -f 'journalctl -f -n 0 -u ceph-14a55226-6f59-11ef-bcea-c7b262605968@osd.3.service' 2024-09-10T10:06:05.822 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:06:05.822 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-10T10:06:05.823 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-10T10:06:05.823 DEBUG:teuthology.orchestra.run.smithi120:> sudo systemctl stop ceph-14a55226-6f59-11ef-bcea-c7b262605968@osd.4 2024-09-10T10:06:06.116 INFO:journalctl@ceph.osd.4.smithi120.stdout:Sep 10 10:06:05 smithi120 systemd[1]: Stopping Ceph osd.4 for 14a55226-6f59-11ef-bcea-c7b262605968... 2024-09-10T10:06:06.117 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:06 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:06.027+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.045437+0000 front 2024-09-10T10:05:37.045486+0000 (oldest deadline 2024-09-10T10:05:59.945393+0000) 2024-09-10T10:06:06.189 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:05 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:05.824+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.086443+0000 front 2024-09-10T10:05:37.086361+0000 (oldest deadline 2024-09-10T10:06:02.887395+0000) 2024-09-10T10:06:06.381 INFO:journalctl@ceph.osd.5.smithi134.stdout:Sep 10 10:06:06 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-5[47540]: 2024-09-10T10:06:06.130+0000 7f371db00640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:36.760837+0000 front 2024-09-10T10:05:36.760719+0000 (oldest deadline 2024-09-10T10:05:59.660697+0000) 2024-09-10T10:06:06.381 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:05 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:05.956+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:34.146460+0000 front 2024-09-10T10:05:34.146398+0000 (oldest deadline 2024-09-10T10:05:59.446520+0000) 2024-09-10T10:06:06.424 INFO:journalctl@ceph.osd.4.smithi120.stdout:Sep 10 10:06:06 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-4[45711]: 2024-09-10T10:06:06.114+0000 7fcb1b40a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:06:06.424 INFO:journalctl@ceph.osd.4.smithi120.stdout:Sep 10 10:06:06 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-4[45711]: 2024-09-10T10:06:06.114+0000 7fcb1b40a640 -1 osd.4 100 *** Got signal Terminated *** 2024-09-10T10:06:06.424 INFO:journalctl@ceph.osd.4.smithi120.stdout:Sep 10 10:06:06 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-4[45711]: 2024-09-10T10:06:06.114+0000 7fcb1b40a640 -1 osd.4 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T10:06:07.189 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:06 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:06.810+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.086443+0000 front 2024-09-10T10:05:37.086361+0000 (oldest deadline 2024-09-10T10:06:02.887395+0000) 2024-09-10T10:06:07.381 INFO:journalctl@ceph.osd.5.smithi134.stdout:Sep 10 10:06:07 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-5[47540]: 2024-09-10T10:06:07.087+0000 7f371db00640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:36.760837+0000 front 2024-09-10T10:05:36.760719+0000 (oldest deadline 2024-09-10T10:05:59.660697+0000) 2024-09-10T10:06:07.381 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:06 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:06.918+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:34.146460+0000 front 2024-09-10T10:05:34.146398+0000 (oldest deadline 2024-09-10T10:05:59.446520+0000) 2024-09-10T10:06:07.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:07 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:07.066+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.045437+0000 front 2024-09-10T10:05:37.045486+0000 (oldest deadline 2024-09-10T10:05:59.945393+0000) 2024-09-10T10:06:07.802 INFO:journalctl@ceph.osd.6.smithi077.stdout:Sep 10 10:06:07 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-6[57286]: 2024-09-10T10:06:07.505+0000 7f174e000640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:42.001143+0000 front 2024-09-10T10:05:42.001163+0000 (oldest deadline 2024-09-10T10:06:07.301413+0000) 2024-09-10T10:06:08.131 INFO:journalctl@ceph.osd.5.smithi134.stdout:Sep 10 10:06:08 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-5[47540]: 2024-09-10T10:06:08.069+0000 7f371db00640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:36.760837+0000 front 2024-09-10T10:05:36.760719+0000 (oldest deadline 2024-09-10T10:05:59.660697+0000) 2024-09-10T10:06:08.131 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:07 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:07.875+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:34.146460+0000 front 2024-09-10T10:05:34.146398+0000 (oldest deadline 2024-09-10T10:05:59.446520+0000) 2024-09-10T10:06:08.189 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:07 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:07.800+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.086443+0000 front 2024-09-10T10:05:37.086361+0000 (oldest deadline 2024-09-10T10:06:02.887395+0000) 2024-09-10T10:06:08.189 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:07 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:07.800+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:42.888131+0000 front 2024-09-10T10:05:42.888100+0000 (oldest deadline 2024-09-10T10:06:07.588327+0000) 2024-09-10T10:06:08.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:08 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:08.114+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.045437+0000 front 2024-09-10T10:05:37.045486+0000 (oldest deadline 2024-09-10T10:05:59.945393+0000) 2024-09-10T10:06:08.792 INFO:journalctl@ceph.osd.6.smithi077.stdout:Sep 10 10:06:08 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-6[57286]: 2024-09-10T10:06:08.503+0000 7f174e000640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:42.001143+0000 front 2024-09-10T10:05:42.001163+0000 (oldest deadline 2024-09-10T10:06:07.301413+0000) 2024-09-10T10:06:08.924 INFO:journalctl@ceph.osd.7.smithi120.stdout:Sep 10 10:06:08 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-7[52351]: 2024-09-10T10:06:08.548+0000 7f3f3f6e4640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:42.865355+0000 front 2024-09-10T10:05:42.865379+0000 (oldest deadline 2024-09-10T10:06:08.165545+0000) 2024-09-10T10:06:09.131 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:08 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:08.826+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:34.146460+0000 front 2024-09-10T10:05:34.146398+0000 (oldest deadline 2024-09-10T10:05:59.446520+0000) 2024-09-10T10:06:09.132 INFO:journalctl@ceph.osd.5.smithi134.stdout:Sep 10 10:06:09 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-5[47540]: 2024-09-10T10:06:09.073+0000 7f371db00640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:36.760837+0000 front 2024-09-10T10:05:36.760719+0000 (oldest deadline 2024-09-10T10:05:59.660697+0000) 2024-09-10T10:06:09.132 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:08 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8[54506]: 2024-09-10T10:06:08.911+0000 7fb09d2dc640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:45.855359+0000 front 2024-09-10T10:05:45.855515+0000 (oldest deadline 2024-09-10T10:06:08.155427+0000) 2024-09-10T10:06:09.189 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:08 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:08.791+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.086443+0000 front 2024-09-10T10:05:37.086361+0000 (oldest deadline 2024-09-10T10:06:02.887395+0000) 2024-09-10T10:06:09.189 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:08 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:08.791+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:42.888131+0000 front 2024-09-10T10:05:42.888100+0000 (oldest deadline 2024-09-10T10:06:07.588327+0000) 2024-09-10T10:06:09.423 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:09 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:09.070+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.045437+0000 front 2024-09-10T10:05:37.045486+0000 (oldest deadline 2024-09-10T10:05:59.945393+0000) 2024-09-10T10:06:09.923 INFO:journalctl@ceph.osd.7.smithi120.stdout:Sep 10 10:06:09 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-7[52351]: 2024-09-10T10:06:09.567+0000 7f3f3f6e4640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:42.865355+0000 front 2024-09-10T10:05:42.865379+0000 (oldest deadline 2024-09-10T10:06:08.165545+0000) 2024-09-10T10:06:09.939 INFO:journalctl@ceph.osd.6.smithi077.stdout:Sep 10 10:06:09 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-6[57286]: 2024-09-10T10:06:09.552+0000 7f174e000640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:42.001143+0000 front 2024-09-10T10:05:42.001163+0000 (oldest deadline 2024-09-10T10:06:07.301413+0000) 2024-09-10T10:06:09.939 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:09 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:09.787+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.086443+0000 front 2024-09-10T10:05:37.086361+0000 (oldest deadline 2024-09-10T10:06:02.887395+0000) 2024-09-10T10:06:09.939 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:09 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:09.787+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:42.888131+0000 front 2024-09-10T10:05:42.888100+0000 (oldest deadline 2024-09-10T10:06:07.588327+0000) 2024-09-10T10:06:10.104 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:09 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8[54506]: 2024-09-10T10:06:09.930+0000 7fb09d2dc640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:45.855359+0000 front 2024-09-10T10:05:45.855515+0000 (oldest deadline 2024-09-10T10:06:08.155427+0000) 2024-09-10T10:06:10.104 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:09 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:09.809+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:34.146460+0000 front 2024-09-10T10:05:34.146398+0000 (oldest deadline 2024-09-10T10:05:59.446520+0000) 2024-09-10T10:06:10.381 INFO:journalctl@ceph.osd.5.smithi134.stdout:Sep 10 10:06:10 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-5[47540]: 2024-09-10T10:06:10.102+0000 7f371db00640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:36.760837+0000 front 2024-09-10T10:05:36.760719+0000 (oldest deadline 2024-09-10T10:05:59.660697+0000) 2024-09-10T10:06:10.381 INFO:journalctl@ceph.osd.5.smithi134.stdout:Sep 10 10:06:10 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-5[47540]: 2024-09-10T10:06:10.102+0000 7f371db00640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:44.862455+0000 front 2024-09-10T10:05:44.862498+0000 (oldest deadline 2024-09-10T10:06:09.562195+0000) 2024-09-10T10:06:10.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:10 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:10.039+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.045437+0000 front 2024-09-10T10:05:37.045486+0000 (oldest deadline 2024-09-10T10:05:59.945393+0000) 2024-09-10T10:06:10.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:10 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:10.039+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:43.446497+0000 front 2024-09-10T10:05:43.446655+0000 (oldest deadline 2024-09-10T10:06:09.346712+0000) 2024-09-10T10:06:10.924 INFO:journalctl@ceph.osd.7.smithi120.stdout:Sep 10 10:06:10 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-7[52351]: 2024-09-10T10:06:10.524+0000 7f3f3f6e4640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:42.865355+0000 front 2024-09-10T10:05:42.865379+0000 (oldest deadline 2024-09-10T10:06:08.165545+0000) 2024-09-10T10:06:10.939 INFO:journalctl@ceph.osd.6.smithi077.stdout:Sep 10 10:06:10 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-6[57286]: 2024-09-10T10:06:10.540+0000 7f174e000640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:42.001143+0000 front 2024-09-10T10:05:42.001163+0000 (oldest deadline 2024-09-10T10:06:07.301413+0000) 2024-09-10T10:06:10.939 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:10 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:10.752+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.086443+0000 front 2024-09-10T10:05:37.086361+0000 (oldest deadline 2024-09-10T10:06:02.887395+0000) 2024-09-10T10:06:10.939 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:10 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:10.752+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:42.888131+0000 front 2024-09-10T10:05:42.888100+0000 (oldest deadline 2024-09-10T10:06:07.588327+0000) 2024-09-10T10:06:11.117 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:10 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8[54506]: 2024-09-10T10:06:10.967+0000 7fb09d2dc640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:45.855359+0000 front 2024-09-10T10:05:45.855515+0000 (oldest deadline 2024-09-10T10:06:08.155427+0000) 2024-09-10T10:06:11.118 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:10 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:10.848+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:34.146460+0000 front 2024-09-10T10:05:34.146398+0000 (oldest deadline 2024-09-10T10:05:59.446520+0000) 2024-09-10T10:06:11.381 INFO:journalctl@ceph.osd.5.smithi134.stdout:Sep 10 10:06:11 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-5[47540]: 2024-09-10T10:06:11.116+0000 7f371db00640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:36.760837+0000 front 2024-09-10T10:05:36.760719+0000 (oldest deadline 2024-09-10T10:05:59.660697+0000) 2024-09-10T10:06:11.381 INFO:journalctl@ceph.osd.5.smithi134.stdout:Sep 10 10:06:11 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-5[47540]: 2024-09-10T10:06:11.116+0000 7f371db00640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:44.862455+0000 front 2024-09-10T10:05:44.862498+0000 (oldest deadline 2024-09-10T10:06:09.562195+0000) 2024-09-10T10:06:11.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:11 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:11.035+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.045437+0000 front 2024-09-10T10:05:37.045486+0000 (oldest deadline 2024-09-10T10:05:59.945393+0000) 2024-09-10T10:06:11.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:11 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:11.035+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:43.446497+0000 front 2024-09-10T10:05:43.446655+0000 (oldest deadline 2024-09-10T10:06:09.346712+0000) 2024-09-10T10:06:11.924 INFO:journalctl@ceph.osd.7.smithi120.stdout:Sep 10 10:06:11 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-7[52351]: 2024-09-10T10:06:11.569+0000 7f3f3f6e4640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:42.865355+0000 front 2024-09-10T10:05:42.865379+0000 (oldest deadline 2024-09-10T10:06:08.165545+0000) 2024-09-10T10:06:11.924 INFO:journalctl@ceph.osd.4.smithi120.stdout:Sep 10 10:06:11 smithi120 podman[69917]: 2024-09-10 10:06:11.622688639 +0000 UTC m=+5.605009955 container died e8252a80001dbb4a38976720cbf2a55a291a49270344ceb541aa328add48c3d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-4, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-10T10:06:11.939 INFO:journalctl@ceph.osd.6.smithi077.stdout:Sep 10 10:06:11 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-6[57286]: 2024-09-10T10:06:11.575+0000 7f174e000640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:42.001143+0000 front 2024-09-10T10:05:42.001163+0000 (oldest deadline 2024-09-10T10:06:07.301413+0000) 2024-09-10T10:06:11.939 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:11 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:11.705+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.086443+0000 front 2024-09-10T10:05:37.086361+0000 (oldest deadline 2024-09-10T10:06:02.887395+0000) 2024-09-10T10:06:11.939 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:11 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:11.705+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:42.888131+0000 front 2024-09-10T10:05:42.888100+0000 (oldest deadline 2024-09-10T10:06:07.588327+0000) 2024-09-10T10:06:12.131 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:12 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8[54506]: 2024-09-10T10:06:12.001+0000 7fb09d2dc640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:45.855359+0000 front 2024-09-10T10:05:45.855515+0000 (oldest deadline 2024-09-10T10:06:08.155427+0000) 2024-09-10T10:06:12.131 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:11 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:11.817+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:34.146460+0000 front 2024-09-10T10:05:34.146398+0000 (oldest deadline 2024-09-10T10:05:59.446520+0000) 2024-09-10T10:06:12.267 INFO:journalctl@ceph.osd.4.smithi120.stdout:Sep 10 10:06:12 smithi120 podman[69917]: 2024-09-10 10:06:12.0152931 +0000 UTC m=+5.997614401 container cleanup e8252a80001dbb4a38976720cbf2a55a291a49270344ceb541aa328add48c3d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-4, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-10T10:06:12.267 INFO:journalctl@ceph.osd.4.smithi120.stdout:Sep 10 10:06:12 smithi120 bash[69917]: ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-4 2024-09-10T10:06:12.267 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:12 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:12.034+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.045437+0000 front 2024-09-10T10:05:37.045486+0000 (oldest deadline 2024-09-10T10:05:59.945393+0000) 2024-09-10T10:06:12.267 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:12 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:12.034+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:43.446497+0000 front 2024-09-10T10:05:43.446655+0000 (oldest deadline 2024-09-10T10:06:09.346712+0000) 2024-09-10T10:06:12.547 INFO:journalctl@ceph.osd.4.smithi120.stdout:Sep 10 10:06:12 smithi120 podman[69933]: 2024-09-10 10:06:12.26665893 +0000 UTC m=+0.641873127 container remove e8252a80001dbb4a38976720cbf2a55a291a49270344ceb541aa328add48c3d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-10T10:06:12.631 INFO:journalctl@ceph.osd.5.smithi134.stdout:Sep 10 10:06:12 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-5[47540]: 2024-09-10T10:06:12.158+0000 7f371db00640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:36.760837+0000 front 2024-09-10T10:05:36.760719+0000 (oldest deadline 2024-09-10T10:05:59.660697+0000) 2024-09-10T10:06:12.631 INFO:journalctl@ceph.osd.5.smithi134.stdout:Sep 10 10:06:12 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-5[47540]: 2024-09-10T10:06:12.158+0000 7f371db00640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:44.862455+0000 front 2024-09-10T10:05:44.862498+0000 (oldest deadline 2024-09-10T10:06:09.562195+0000) 2024-09-10T10:06:12.838 INFO:journalctl@ceph.osd.4.smithi120.stdout:Sep 10 10:06:12 smithi120 podman[70022]: 2024-09-10 10:06:12.485566544 +0000 UTC m=+0.015951739 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T10:06:12.838 INFO:journalctl@ceph.osd.4.smithi120.stdout:Sep 10 10:06:12 smithi120 podman[70022]: 2024-09-10 10:06:12.600948181 +0000 UTC m=+0.131333408 container create 4ed88aef214785458668c72398d71f8d8cbef88f67ffa65716cbbbfbace318dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-4-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T10:06:12.838 INFO:journalctl@ceph.osd.4.smithi120.stdout:Sep 10 10:06:12 smithi120 podman[70022]: 2024-09-10 10:06:12.837520273 +0000 UTC m=+0.367905466 container init 4ed88aef214785458668c72398d71f8d8cbef88f67ffa65716cbbbfbace318dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-10T10:06:12.839 INFO:journalctl@ceph.osd.7.smithi120.stdout:Sep 10 10:06:12 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-7[52351]: 2024-09-10T10:06:12.545+0000 7f3f3f6e4640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:42.865355+0000 front 2024-09-10T10:05:42.865379+0000 (oldest deadline 2024-09-10T10:06:08.165545+0000) 2024-09-10T10:06:12.939 INFO:journalctl@ceph.osd.6.smithi077.stdout:Sep 10 10:06:12 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-6[57286]: 2024-09-10T10:06:12.528+0000 7f174e000640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:42.001143+0000 front 2024-09-10T10:05:42.001163+0000 (oldest deadline 2024-09-10T10:06:07.301413+0000) 2024-09-10T10:06:12.940 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:12 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:12.686+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.086443+0000 front 2024-09-10T10:05:37.086361+0000 (oldest deadline 2024-09-10T10:06:02.887395+0000) 2024-09-10T10:06:12.940 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:12 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:12.686+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:42.888131+0000 front 2024-09-10T10:05:42.888100+0000 (oldest deadline 2024-09-10T10:06:07.588327+0000) 2024-09-10T10:06:13.131 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:13 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8[54506]: 2024-09-10T10:06:13.001+0000 7fb09d2dc640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:45.855359+0000 front 2024-09-10T10:05:45.855515+0000 (oldest deadline 2024-09-10T10:06:08.155427+0000) 2024-09-10T10:06:13.131 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:12 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:12.856+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:34.146460+0000 front 2024-09-10T10:05:34.146398+0000 (oldest deadline 2024-09-10T10:05:59.446520+0000) 2024-09-10T10:06:13.174 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:13 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:13.066+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.045437+0000 front 2024-09-10T10:05:37.045486+0000 (oldest deadline 2024-09-10T10:05:59.945393+0000) 2024-09-10T10:06:13.174 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:13 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:13.066+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:43.446497+0000 front 2024-09-10T10:05:43.446655+0000 (oldest deadline 2024-09-10T10:06:09.346712+0000) 2024-09-10T10:06:13.174 INFO:journalctl@ceph.osd.4.smithi120.stdout:Sep 10 10:06:12 smithi120 podman[70022]: 2024-09-10 10:06:12.84104891 +0000 UTC m=+0.371434097 container start 4ed88aef214785458668c72398d71f8d8cbef88f67ffa65716cbbbfbace318dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-10T10:06:13.174 INFO:journalctl@ceph.osd.4.smithi120.stdout:Sep 10 10:06:12 smithi120 podman[70022]: 2024-09-10 10:06:12.936995013 +0000 UTC m=+0.467380221 container attach 4ed88aef214785458668c72398d71f8d8cbef88f67ffa65716cbbbfbace318dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-10T10:06:13.174 INFO:journalctl@ceph.osd.4.smithi120.stdout:Sep 10 10:06:13 smithi120 podman[70022]: 2024-09-10 10:06:13.036716121 +0000 UTC m=+0.567101306 container died 4ed88aef214785458668c72398d71f8d8cbef88f67ffa65716cbbbfbace318dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T10:06:13.631 INFO:journalctl@ceph.osd.5.smithi134.stdout:Sep 10 10:06:13 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-5[47540]: 2024-09-10T10:06:13.195+0000 7f371db00640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:36.760837+0000 front 2024-09-10T10:05:36.760719+0000 (oldest deadline 2024-09-10T10:05:59.660697+0000) 2024-09-10T10:06:13.631 INFO:journalctl@ceph.osd.5.smithi134.stdout:Sep 10 10:06:13 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-5[47540]: 2024-09-10T10:06:13.195+0000 7f371db00640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:44.862455+0000 front 2024-09-10T10:05:44.862498+0000 (oldest deadline 2024-09-10T10:06:09.562195+0000) 2024-09-10T10:06:13.673 INFO:journalctl@ceph.osd.7.smithi120.stdout:Sep 10 10:06:13 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-7[52351]: 2024-09-10T10:06:13.567+0000 7f3f3f6e4640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:42.865355+0000 front 2024-09-10T10:05:42.865379+0000 (oldest deadline 2024-09-10T10:06:08.165545+0000) 2024-09-10T10:06:13.742 DEBUG:teuthology.orchestra.run.smithi120:> sudo pkill -f 'journalctl -f -n 0 -u ceph-14a55226-6f59-11ef-bcea-c7b262605968@osd.4.service' 2024-09-10T10:06:13.780 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:06:13.781 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-10T10:06:13.781 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-10T10:06:13.781 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-14a55226-6f59-11ef-bcea-c7b262605968@osd.5 2024-09-10T10:06:13.939 INFO:journalctl@ceph.osd.6.smithi077.stdout:Sep 10 10:06:13 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-6[57286]: 2024-09-10T10:06:13.548+0000 7f174e000640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:42.001143+0000 front 2024-09-10T10:05:42.001163+0000 (oldest deadline 2024-09-10T10:06:07.301413+0000) 2024-09-10T10:06:13.939 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:13 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:13.728+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.086443+0000 front 2024-09-10T10:05:37.086361+0000 (oldest deadline 2024-09-10T10:06:02.887395+0000) 2024-09-10T10:06:13.940 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:13 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:13.728+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:42.888131+0000 front 2024-09-10T10:05:42.888100+0000 (oldest deadline 2024-09-10T10:06:07.588327+0000) 2024-09-10T10:06:14.093 INFO:journalctl@ceph.osd.5.smithi134.stdout:Sep 10 10:06:13 smithi134 systemd[1]: Stopping Ceph osd.5 for 14a55226-6f59-11ef-bcea-c7b262605968... 2024-09-10T10:06:14.093 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:13 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:13.885+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:34.146460+0000 front 2024-09-10T10:05:34.146398+0000 (oldest deadline 2024-09-10T10:05:59.446520+0000) 2024-09-10T10:06:14.094 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:13 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8[54506]: 2024-09-10T10:06:13.994+0000 7fb09d2dc640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:45.855359+0000 front 2024-09-10T10:05:45.855515+0000 (oldest deadline 2024-09-10T10:06:08.155427+0000) 2024-09-10T10:06:14.381 INFO:journalctl@ceph.osd.5.smithi134.stdout:Sep 10 10:06:14 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-5[47540]: 2024-09-10T10:06:14.092+0000 7f37214ea640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:06:14.381 INFO:journalctl@ceph.osd.5.smithi134.stdout:Sep 10 10:06:14 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-5[47540]: 2024-09-10T10:06:14.092+0000 7f37214ea640 -1 osd.5 100 *** Got signal Terminated *** 2024-09-10T10:06:14.381 INFO:journalctl@ceph.osd.5.smithi134.stdout:Sep 10 10:06:14 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-5[47540]: 2024-09-10T10:06:14.092+0000 7f37214ea640 -1 osd.5 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T10:06:14.382 INFO:journalctl@ceph.osd.5.smithi134.stdout:Sep 10 10:06:14 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-5[47540]: 2024-09-10T10:06:14.237+0000 7f371db00640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:36.760837+0000 front 2024-09-10T10:05:36.760719+0000 (oldest deadline 2024-09-10T10:05:59.660697+0000) 2024-09-10T10:06:14.382 INFO:journalctl@ceph.osd.5.smithi134.stdout:Sep 10 10:06:14 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-5[47540]: 2024-09-10T10:06:14.237+0000 7f371db00640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:44.862455+0000 front 2024-09-10T10:05:44.862498+0000 (oldest deadline 2024-09-10T10:06:09.562195+0000) 2024-09-10T10:06:14.423 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:14 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:14.101+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.045437+0000 front 2024-09-10T10:05:37.045486+0000 (oldest deadline 2024-09-10T10:05:59.945393+0000) 2024-09-10T10:06:14.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:14 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:14.101+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:43.446497+0000 front 2024-09-10T10:05:43.446655+0000 (oldest deadline 2024-09-10T10:06:09.346712+0000) 2024-09-10T10:06:14.924 INFO:journalctl@ceph.osd.7.smithi120.stdout:Sep 10 10:06:14 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-7[52351]: 2024-09-10T10:06:14.522+0000 7f3f3f6e4640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:42.865355+0000 front 2024-09-10T10:05:42.865379+0000 (oldest deadline 2024-09-10T10:06:08.165545+0000) 2024-09-10T10:06:14.939 INFO:journalctl@ceph.osd.6.smithi077.stdout:Sep 10 10:06:14 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-6[57286]: 2024-09-10T10:06:14.499+0000 7f174e000640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:42.001143+0000 front 2024-09-10T10:05:42.001163+0000 (oldest deadline 2024-09-10T10:06:07.301413+0000) 2024-09-10T10:06:14.939 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:14 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:14.747+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.086443+0000 front 2024-09-10T10:05:37.086361+0000 (oldest deadline 2024-09-10T10:06:02.887395+0000) 2024-09-10T10:06:14.939 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:14 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:14.747+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:42.888131+0000 front 2024-09-10T10:05:42.888100+0000 (oldest deadline 2024-09-10T10:06:07.588327+0000) 2024-09-10T10:06:15.190 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:15 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8[54506]: 2024-09-10T10:06:15.012+0000 7fb09d2dc640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:45.855359+0000 front 2024-09-10T10:05:45.855515+0000 (oldest deadline 2024-09-10T10:06:08.155427+0000) 2024-09-10T10:06:15.190 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:14 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:14.894+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:34.146460+0000 front 2024-09-10T10:05:34.146398+0000 (oldest deadline 2024-09-10T10:05:59.446520+0000) 2024-09-10T10:06:15.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:15 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:15.098+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.045437+0000 front 2024-09-10T10:05:37.045486+0000 (oldest deadline 2024-09-10T10:05:59.945393+0000) 2024-09-10T10:06:15.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:15 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:15.098+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:43.446497+0000 front 2024-09-10T10:05:43.446655+0000 (oldest deadline 2024-09-10T10:06:09.346712+0000) 2024-09-10T10:06:15.631 INFO:journalctl@ceph.osd.5.smithi134.stdout:Sep 10 10:06:15 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-5[47540]: 2024-09-10T10:06:15.188+0000 7f371db00640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:36.760837+0000 front 2024-09-10T10:05:36.760719+0000 (oldest deadline 2024-09-10T10:05:59.660697+0000) 2024-09-10T10:06:15.631 INFO:journalctl@ceph.osd.5.smithi134.stdout:Sep 10 10:06:15 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-5[47540]: 2024-09-10T10:06:15.188+0000 7f371db00640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:44.862455+0000 front 2024-09-10T10:05:44.862498+0000 (oldest deadline 2024-09-10T10:06:09.562195+0000) 2024-09-10T10:06:15.923 INFO:journalctl@ceph.osd.7.smithi120.stdout:Sep 10 10:06:15 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-7[52351]: 2024-09-10T10:06:15.527+0000 7f3f3f6e4640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:42.865355+0000 front 2024-09-10T10:05:42.865379+0000 (oldest deadline 2024-09-10T10:06:08.165545+0000) 2024-09-10T10:06:15.939 INFO:journalctl@ceph.osd.6.smithi077.stdout:Sep 10 10:06:15 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-6[57286]: 2024-09-10T10:06:15.505+0000 7f174e000640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:42.001143+0000 front 2024-09-10T10:05:42.001163+0000 (oldest deadline 2024-09-10T10:06:07.301413+0000) 2024-09-10T10:06:15.939 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:15 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:15.721+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.086443+0000 front 2024-09-10T10:05:37.086361+0000 (oldest deadline 2024-09-10T10:06:02.887395+0000) 2024-09-10T10:06:15.939 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:15 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:15.721+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:42.888131+0000 front 2024-09-10T10:05:42.888100+0000 (oldest deadline 2024-09-10T10:06:07.588327+0000) 2024-09-10T10:06:15.940 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:15 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:15.721+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:50.489601+0000 front 2024-09-10T10:05:50.489747+0000 (oldest deadline 2024-09-10T10:06:15.690110+0000) 2024-09-10T10:06:16.131 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:15 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:15.871+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:34.146460+0000 front 2024-09-10T10:05:34.146398+0000 (oldest deadline 2024-09-10T10:05:59.446520+0000) 2024-09-10T10:06:16.131 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:16 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8[54506]: 2024-09-10T10:06:16.022+0000 7fb09d2dc640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:45.855359+0000 front 2024-09-10T10:05:45.855515+0000 (oldest deadline 2024-09-10T10:06:08.155427+0000) 2024-09-10T10:06:16.131 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:16 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8[54506]: 2024-09-10T10:06:16.022+0000 7fb09d2dc640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:50.456414+0000 front 2024-09-10T10:05:50.456398+0000 (oldest deadline 2024-09-10T10:06:15.156538+0000) 2024-09-10T10:06:16.423 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:16 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:16.125+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.045437+0000 front 2024-09-10T10:05:37.045486+0000 (oldest deadline 2024-09-10T10:05:59.945393+0000) 2024-09-10T10:06:16.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:16 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:16.125+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:43.446497+0000 front 2024-09-10T10:05:43.446655+0000 (oldest deadline 2024-09-10T10:06:09.346712+0000) 2024-09-10T10:06:16.631 INFO:journalctl@ceph.osd.5.smithi134.stdout:Sep 10 10:06:16 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-5[47540]: 2024-09-10T10:06:16.220+0000 7f371db00640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:36.760837+0000 front 2024-09-10T10:05:36.760719+0000 (oldest deadline 2024-09-10T10:05:59.660697+0000) 2024-09-10T10:06:16.631 INFO:journalctl@ceph.osd.5.smithi134.stdout:Sep 10 10:06:16 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-5[47540]: 2024-09-10T10:06:16.220+0000 7f371db00640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:44.862455+0000 front 2024-09-10T10:05:44.862498+0000 (oldest deadline 2024-09-10T10:06:09.562195+0000) 2024-09-10T10:06:16.631 INFO:journalctl@ceph.osd.5.smithi134.stdout:Sep 10 10:06:16 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-5[47540]: 2024-09-10T10:06:16.220+0000 7f371db00640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:51.863031+0000 front 2024-09-10T10:05:51.863069+0000 (oldest deadline 2024-09-10T10:06:15.963026+0000) 2024-09-10T10:06:16.924 INFO:journalctl@ceph.osd.7.smithi120.stdout:Sep 10 10:06:16 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-7[52351]: 2024-09-10T10:06:16.511+0000 7f3f3f6e4640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:42.865355+0000 front 2024-09-10T10:05:42.865379+0000 (oldest deadline 2024-09-10T10:06:08.165545+0000) 2024-09-10T10:06:16.939 INFO:journalctl@ceph.osd.6.smithi077.stdout:Sep 10 10:06:16 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-6[57286]: 2024-09-10T10:06:16.461+0000 7f174e000640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:42.001143+0000 front 2024-09-10T10:05:42.001163+0000 (oldest deadline 2024-09-10T10:06:07.301413+0000) 2024-09-10T10:06:16.939 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:16 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:16.705+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.086443+0000 front 2024-09-10T10:05:37.086361+0000 (oldest deadline 2024-09-10T10:06:02.887395+0000) 2024-09-10T10:06:16.939 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:16 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:16.705+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:42.888131+0000 front 2024-09-10T10:05:42.888100+0000 (oldest deadline 2024-09-10T10:06:07.588327+0000) 2024-09-10T10:06:16.939 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:16 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:16.705+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:50.489601+0000 front 2024-09-10T10:05:50.489747+0000 (oldest deadline 2024-09-10T10:06:15.690110+0000) 2024-09-10T10:06:17.131 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:17 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8[54506]: 2024-09-10T10:06:17.060+0000 7fb09d2dc640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:45.855359+0000 front 2024-09-10T10:05:45.855515+0000 (oldest deadline 2024-09-10T10:06:08.155427+0000) 2024-09-10T10:06:17.131 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:17 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8[54506]: 2024-09-10T10:06:17.060+0000 7fb09d2dc640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:50.456414+0000 front 2024-09-10T10:05:50.456398+0000 (oldest deadline 2024-09-10T10:06:15.156538+0000) 2024-09-10T10:06:17.132 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:16 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:16.825+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:34.146460+0000 front 2024-09-10T10:05:34.146398+0000 (oldest deadline 2024-09-10T10:05:59.446520+0000) 2024-09-10T10:06:17.423 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:17 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:17.110+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.045437+0000 front 2024-09-10T10:05:37.045486+0000 (oldest deadline 2024-09-10T10:05:59.945393+0000) 2024-09-10T10:06:17.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:17 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:17.110+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:43.446497+0000 front 2024-09-10T10:05:43.446655+0000 (oldest deadline 2024-09-10T10:06:09.346712+0000) 2024-09-10T10:06:17.631 INFO:journalctl@ceph.osd.5.smithi134.stdout:Sep 10 10:06:17 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-5[47540]: 2024-09-10T10:06:17.267+0000 7f371db00640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:36.760837+0000 front 2024-09-10T10:05:36.760719+0000 (oldest deadline 2024-09-10T10:05:59.660697+0000) 2024-09-10T10:06:17.631 INFO:journalctl@ceph.osd.5.smithi134.stdout:Sep 10 10:06:17 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-5[47540]: 2024-09-10T10:06:17.267+0000 7f371db00640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:44.862455+0000 front 2024-09-10T10:05:44.862498+0000 (oldest deadline 2024-09-10T10:06:09.562195+0000) 2024-09-10T10:06:17.631 INFO:journalctl@ceph.osd.5.smithi134.stdout:Sep 10 10:06:17 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-5[47540]: 2024-09-10T10:06:17.267+0000 7f371db00640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:51.863031+0000 front 2024-09-10T10:05:51.863069+0000 (oldest deadline 2024-09-10T10:06:15.963026+0000) 2024-09-10T10:06:17.726 INFO:journalctl@ceph.osd.6.smithi077.stdout:Sep 10 10:06:17 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-6[57286]: 2024-09-10T10:06:17.448+0000 7f174e000640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:42.001143+0000 front 2024-09-10T10:05:42.001163+0000 (oldest deadline 2024-09-10T10:06:07.301413+0000) 2024-09-10T10:06:17.924 INFO:journalctl@ceph.osd.7.smithi120.stdout:Sep 10 10:06:17 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-7[52351]: 2024-09-10T10:06:17.547+0000 7f3f3f6e4640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:42.865355+0000 front 2024-09-10T10:05:42.865379+0000 (oldest deadline 2024-09-10T10:06:08.165545+0000) 2024-09-10T10:06:18.101 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:17 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:17.829+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:34.146460+0000 front 2024-09-10T10:05:34.146398+0000 (oldest deadline 2024-09-10T10:05:59.446520+0000) 2024-09-10T10:06:18.189 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:17 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:17.724+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.086443+0000 front 2024-09-10T10:05:37.086361+0000 (oldest deadline 2024-09-10T10:06:02.887395+0000) 2024-09-10T10:06:18.189 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:17 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:17.724+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:42.888131+0000 front 2024-09-10T10:05:42.888100+0000 (oldest deadline 2024-09-10T10:06:07.588327+0000) 2024-09-10T10:06:18.190 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:17 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:17.724+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:50.489601+0000 front 2024-09-10T10:05:50.489747+0000 (oldest deadline 2024-09-10T10:06:15.690110+0000) 2024-09-10T10:06:18.381 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:18 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8[54506]: 2024-09-10T10:06:18.100+0000 7fb09d2dc640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:45.855359+0000 front 2024-09-10T10:05:45.855515+0000 (oldest deadline 2024-09-10T10:06:08.155427+0000) 2024-09-10T10:06:18.381 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:18 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8[54506]: 2024-09-10T10:06:18.100+0000 7fb09d2dc640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:50.456414+0000 front 2024-09-10T10:05:50.456398+0000 (oldest deadline 2024-09-10T10:06:15.156538+0000) 2024-09-10T10:06:18.382 INFO:journalctl@ceph.osd.5.smithi134.stdout:Sep 10 10:06:18 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-5[47540]: 2024-09-10T10:06:18.301+0000 7f371db00640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:36.760837+0000 front 2024-09-10T10:05:36.760719+0000 (oldest deadline 2024-09-10T10:05:59.660697+0000) 2024-09-10T10:06:18.382 INFO:journalctl@ceph.osd.5.smithi134.stdout:Sep 10 10:06:18 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-5[47540]: 2024-09-10T10:06:18.301+0000 7f371db00640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:44.862455+0000 front 2024-09-10T10:05:44.862498+0000 (oldest deadline 2024-09-10T10:06:09.562195+0000) 2024-09-10T10:06:18.382 INFO:journalctl@ceph.osd.5.smithi134.stdout:Sep 10 10:06:18 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-5[47540]: 2024-09-10T10:06:18.301+0000 7f371db00640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:51.863031+0000 front 2024-09-10T10:05:51.863069+0000 (oldest deadline 2024-09-10T10:06:15.963026+0000) 2024-09-10T10:06:18.423 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:18 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:18.142+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.045437+0000 front 2024-09-10T10:05:37.045486+0000 (oldest deadline 2024-09-10T10:05:59.945393+0000) 2024-09-10T10:06:18.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:18 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:18.142+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:43.446497+0000 front 2024-09-10T10:05:43.446655+0000 (oldest deadline 2024-09-10T10:06:09.346712+0000) 2024-09-10T10:06:18.689 INFO:journalctl@ceph.osd.6.smithi077.stdout:Sep 10 10:06:18 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-6[57286]: 2024-09-10T10:06:18.426+0000 7f174e000640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:42.001143+0000 front 2024-09-10T10:05:42.001163+0000 (oldest deadline 2024-09-10T10:06:07.301413+0000) 2024-09-10T10:06:18.689 INFO:journalctl@ceph.osd.6.smithi077.stdout:Sep 10 10:06:18 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-6[57286]: 2024-09-10T10:06:18.426+0000 7f174e000640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:54.303272+0000 front 2024-09-10T10:05:54.303284+0000 (oldest deadline 2024-09-10T10:06:17.803077+0000) 2024-09-10T10:06:18.699 INFO:journalctl@ceph.osd.7.smithi120.stdout:Sep 10 10:06:18 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-7[52351]: 2024-09-10T10:06:18.564+0000 7f3f3f6e4640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:42.865355+0000 front 2024-09-10T10:05:42.865379+0000 (oldest deadline 2024-09-10T10:06:08.165545+0000) 2024-09-10T10:06:19.130 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:18 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:18.870+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:34.146460+0000 front 2024-09-10T10:05:34.146398+0000 (oldest deadline 2024-09-10T10:05:59.446520+0000) 2024-09-10T10:06:19.189 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:18 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:18.720+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.086443+0000 front 2024-09-10T10:05:37.086361+0000 (oldest deadline 2024-09-10T10:06:02.887395+0000) 2024-09-10T10:06:19.189 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:18 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:18.720+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:42.888131+0000 front 2024-09-10T10:05:42.888100+0000 (oldest deadline 2024-09-10T10:06:07.588327+0000) 2024-09-10T10:06:19.189 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:18 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:18.720+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:50.489601+0000 front 2024-09-10T10:05:50.489747+0000 (oldest deadline 2024-09-10T10:06:15.690110+0000) 2024-09-10T10:06:19.381 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:19 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8[54506]: 2024-09-10T10:06:19.128+0000 7fb09d2dc640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:45.855359+0000 front 2024-09-10T10:05:45.855515+0000 (oldest deadline 2024-09-10T10:06:08.155427+0000) 2024-09-10T10:06:19.381 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:19 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8[54506]: 2024-09-10T10:06:19.128+0000 7fb09d2dc640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:50.456414+0000 front 2024-09-10T10:05:50.456398+0000 (oldest deadline 2024-09-10T10:06:15.156538+0000) 2024-09-10T10:06:19.423 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:19 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:19.106+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.045437+0000 front 2024-09-10T10:05:37.045486+0000 (oldest deadline 2024-09-10T10:05:59.945393+0000) 2024-09-10T10:06:19.423 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:19 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:19.106+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:43.446497+0000 front 2024-09-10T10:05:43.446655+0000 (oldest deadline 2024-09-10T10:06:09.346712+0000) 2024-09-10T10:06:19.827 INFO:journalctl@ceph.osd.5.smithi134.stdout:Sep 10 10:06:19 smithi134 podman[71824]: 2024-09-10 10:06:19.525000848 +0000 UTC m=+5.544894059 container died e64a17ce88a27302444be44647ff36127f2814714dfc275112a8acd1da4f7012 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-5, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-10T10:06:19.923 INFO:journalctl@ceph.osd.7.smithi120.stdout:Sep 10 10:06:19 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-7[52351]: 2024-09-10T10:06:19.552+0000 7f3f3f6e4640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:42.865355+0000 front 2024-09-10T10:05:42.865379+0000 (oldest deadline 2024-09-10T10:06:08.165545+0000) 2024-09-10T10:06:19.924 INFO:journalctl@ceph.osd.7.smithi120.stdout:Sep 10 10:06:19 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-7[52351]: 2024-09-10T10:06:19.552+0000 7f3f3f6e4640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:54.066777+0000 front 2024-09-10T10:05:54.066812+0000 (oldest deadline 2024-09-10T10:06:19.267357+0000) 2024-09-10T10:06:19.939 INFO:journalctl@ceph.osd.6.smithi077.stdout:Sep 10 10:06:19 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-6[57286]: 2024-09-10T10:06:19.470+0000 7f174e000640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:42.001143+0000 front 2024-09-10T10:05:42.001163+0000 (oldest deadline 2024-09-10T10:06:07.301413+0000) 2024-09-10T10:06:19.939 INFO:journalctl@ceph.osd.6.smithi077.stdout:Sep 10 10:06:19 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-6[57286]: 2024-09-10T10:06:19.470+0000 7f174e000640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:54.303272+0000 front 2024-09-10T10:05:54.303284+0000 (oldest deadline 2024-09-10T10:06:17.803077+0000) 2024-09-10T10:06:19.939 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:19 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:19.688+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.086443+0000 front 2024-09-10T10:05:37.086361+0000 (oldest deadline 2024-09-10T10:06:02.887395+0000) 2024-09-10T10:06:19.939 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:19 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:19.688+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:42.888131+0000 front 2024-09-10T10:05:42.888100+0000 (oldest deadline 2024-09-10T10:06:07.588327+0000) 2024-09-10T10:06:19.940 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:19 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:19.688+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:50.489601+0000 front 2024-09-10T10:05:50.489747+0000 (oldest deadline 2024-09-10T10:06:15.690110+0000) 2024-09-10T10:06:20.131 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:19 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:19.830+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:34.146460+0000 front 2024-09-10T10:05:34.146398+0000 (oldest deadline 2024-09-10T10:05:59.446520+0000) 2024-09-10T10:06:20.131 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:19 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:19.830+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:53.949416+0000 front 2024-09-10T10:05:53.949272+0000 (oldest deadline 2024-09-10T10:06:19.749872+0000) 2024-09-10T10:06:20.132 INFO:journalctl@ceph.osd.5.smithi134.stdout:Sep 10 10:06:19 smithi134 podman[71824]: 2024-09-10 10:06:19.917414826 +0000 UTC m=+5.937308049 container cleanup e64a17ce88a27302444be44647ff36127f2814714dfc275112a8acd1da4f7012 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-10T10:06:20.132 INFO:journalctl@ceph.osd.5.smithi134.stdout:Sep 10 10:06:19 smithi134 bash[71824]: ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-5 2024-09-10T10:06:20.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:20 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:20.155+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.045437+0000 front 2024-09-10T10:05:37.045486+0000 (oldest deadline 2024-09-10T10:05:59.945393+0000) 2024-09-10T10:06:20.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:20 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:20.155+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:43.446497+0000 front 2024-09-10T10:05:43.446655+0000 (oldest deadline 2024-09-10T10:06:09.346712+0000) 2024-09-10T10:06:20.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:20 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:20.155+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:49.347200+0000 front 2024-09-10T10:05:49.347266+0000 (oldest deadline 2024-09-10T10:06:19.947685+0000) 2024-09-10T10:06:20.477 INFO:journalctl@ceph.osd.5.smithi134.stdout:Sep 10 10:06:20 smithi134 podman[71841]: 2024-09-10 10:06:20.159583615 +0000 UTC m=+0.637604150 container remove e64a17ce88a27302444be44647ff36127f2814714dfc275112a8acd1da4f7012 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True) 2024-09-10T10:06:20.478 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:20 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8[54506]: 2024-09-10T10:06:20.134+0000 7fb09d2dc640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:45.855359+0000 front 2024-09-10T10:05:45.855515+0000 (oldest deadline 2024-09-10T10:06:08.155427+0000) 2024-09-10T10:06:20.478 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:20 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8[54506]: 2024-09-10T10:06:20.134+0000 7fb09d2dc640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:50.456414+0000 front 2024-09-10T10:05:50.456398+0000 (oldest deadline 2024-09-10T10:06:15.156538+0000) 2024-09-10T10:06:20.756 INFO:journalctl@ceph.osd.5.smithi134.stdout:Sep 10 10:06:20 smithi134 podman[71930]: 2024-09-10 10:06:20.379658919 +0000 UTC m=+0.015731298 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T10:06:20.757 INFO:journalctl@ceph.osd.5.smithi134.stdout:Sep 10 10:06:20 smithi134 podman[71930]: 2024-09-10 10:06:20.503494819 +0000 UTC m=+0.139567264 container create ab3e482cfe232c545ba6ccc59fb86861e277ba7a8e11f278e822aceddf55405b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-10T10:06:20.924 INFO:journalctl@ceph.osd.7.smithi120.stdout:Sep 10 10:06:20 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-7[52351]: 2024-09-10T10:06:20.531+0000 7f3f3f6e4640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:42.865355+0000 front 2024-09-10T10:05:42.865379+0000 (oldest deadline 2024-09-10T10:06:08.165545+0000) 2024-09-10T10:06:20.924 INFO:journalctl@ceph.osd.7.smithi120.stdout:Sep 10 10:06:20 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-7[52351]: 2024-09-10T10:06:20.531+0000 7f3f3f6e4640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:54.066777+0000 front 2024-09-10T10:05:54.066812+0000 (oldest deadline 2024-09-10T10:06:19.267357+0000) 2024-09-10T10:06:20.939 INFO:journalctl@ceph.osd.6.smithi077.stdout:Sep 10 10:06:20 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-6[57286]: 2024-09-10T10:06:20.500+0000 7f174e000640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:42.001143+0000 front 2024-09-10T10:05:42.001163+0000 (oldest deadline 2024-09-10T10:06:07.301413+0000) 2024-09-10T10:06:20.939 INFO:journalctl@ceph.osd.6.smithi077.stdout:Sep 10 10:06:20 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-6[57286]: 2024-09-10T10:06:20.500+0000 7f174e000640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:54.303272+0000 front 2024-09-10T10:05:54.303284+0000 (oldest deadline 2024-09-10T10:06:17.803077+0000) 2024-09-10T10:06:20.939 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:20 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:20.683+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.086443+0000 front 2024-09-10T10:05:37.086361+0000 (oldest deadline 2024-09-10T10:06:02.887395+0000) 2024-09-10T10:06:20.939 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:20 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:20.683+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:42.888131+0000 front 2024-09-10T10:05:42.888100+0000 (oldest deadline 2024-09-10T10:06:07.588327+0000) 2024-09-10T10:06:20.939 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:20 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:20.683+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:50.489601+0000 front 2024-09-10T10:05:50.489747+0000 (oldest deadline 2024-09-10T10:06:15.690110+0000) 2024-09-10T10:06:21.132 INFO:journalctl@ceph.osd.5.smithi134.stdout:Sep 10 10:06:20 smithi134 podman[71930]: 2024-09-10 10:06:20.756078436 +0000 UTC m=+0.392150825 container init ab3e482cfe232c545ba6ccc59fb86861e277ba7a8e11f278e822aceddf55405b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T10:06:21.132 INFO:journalctl@ceph.osd.5.smithi134.stdout:Sep 10 10:06:20 smithi134 podman[71930]: 2024-09-10 10:06:20.759330314 +0000 UTC m=+0.395402709 container start ab3e482cfe232c545ba6ccc59fb86861e277ba7a8e11f278e822aceddf55405b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T10:06:21.132 INFO:journalctl@ceph.osd.5.smithi134.stdout:Sep 10 10:06:20 smithi134 podman[71930]: 2024-09-10 10:06:20.864538745 +0000 UTC m=+0.500611136 container attach ab3e482cfe232c545ba6ccc59fb86861e277ba7a8e11f278e822aceddf55405b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-5-deactivate, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-10T10:06:21.132 INFO:journalctl@ceph.osd.5.smithi134.stdout:Sep 10 10:06:20 smithi134 podman[71930]: 2024-09-10 10:06:20.95146707 +0000 UTC m=+0.587539451 container died ab3e482cfe232c545ba6ccc59fb86861e277ba7a8e11f278e822aceddf55405b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-10T10:06:21.132 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:20 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:20.791+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:34.146460+0000 front 2024-09-10T10:05:34.146398+0000 (oldest deadline 2024-09-10T10:05:59.446520+0000) 2024-09-10T10:06:21.132 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:20 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:20.791+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:53.949416+0000 front 2024-09-10T10:05:53.949272+0000 (oldest deadline 2024-09-10T10:06:19.749872+0000) 2024-09-10T10:06:21.539 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:21 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:21.196+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.045437+0000 front 2024-09-10T10:05:37.045486+0000 (oldest deadline 2024-09-10T10:05:59.945393+0000) 2024-09-10T10:06:21.539 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:21 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:21.196+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:43.446497+0000 front 2024-09-10T10:05:43.446655+0000 (oldest deadline 2024-09-10T10:06:09.346712+0000) 2024-09-10T10:06:21.539 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:21 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:21.196+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:49.347200+0000 front 2024-09-10T10:05:49.347266+0000 (oldest deadline 2024-09-10T10:06:19.947685+0000) 2024-09-10T10:06:21.594 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:21 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8[54506]: 2024-09-10T10:06:21.135+0000 7fb09d2dc640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:45.855359+0000 front 2024-09-10T10:05:45.855515+0000 (oldest deadline 2024-09-10T10:06:08.155427+0000) 2024-09-10T10:06:21.594 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:21 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8[54506]: 2024-09-10T10:06:21.135+0000 7fb09d2dc640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:50.456414+0000 front 2024-09-10T10:05:50.456398+0000 (oldest deadline 2024-09-10T10:06:15.156538+0000) 2024-09-10T10:06:21.616 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-14a55226-6f59-11ef-bcea-c7b262605968@osd.5.service' 2024-09-10T10:06:21.659 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:06:21.659 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-10T10:06:21.659 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-10T10:06:21.659 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ceph-14a55226-6f59-11ef-bcea-c7b262605968@osd.6 2024-09-10T10:06:21.881 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:21 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:21.829+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:34.146460+0000 front 2024-09-10T10:05:34.146398+0000 (oldest deadline 2024-09-10T10:05:59.446520+0000) 2024-09-10T10:06:21.881 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:21 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:21.829+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:53.949416+0000 front 2024-09-10T10:05:53.949272+0000 (oldest deadline 2024-09-10T10:06:19.749872+0000) 2024-09-10T10:06:21.924 INFO:journalctl@ceph.osd.7.smithi120.stdout:Sep 10 10:06:21 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-7[52351]: 2024-09-10T10:06:21.537+0000 7f3f3f6e4640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:42.865355+0000 front 2024-09-10T10:05:42.865379+0000 (oldest deadline 2024-09-10T10:06:08.165545+0000) 2024-09-10T10:06:21.924 INFO:journalctl@ceph.osd.7.smithi120.stdout:Sep 10 10:06:21 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-7[52351]: 2024-09-10T10:06:21.537+0000 7f3f3f6e4640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:54.066777+0000 front 2024-09-10T10:05:54.066812+0000 (oldest deadline 2024-09-10T10:06:19.267357+0000) 2024-09-10T10:06:21.939 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:21 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:21.656+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.086443+0000 front 2024-09-10T10:05:37.086361+0000 (oldest deadline 2024-09-10T10:06:02.887395+0000) 2024-09-10T10:06:21.939 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:21 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:21.656+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:42.888131+0000 front 2024-09-10T10:05:42.888100+0000 (oldest deadline 2024-09-10T10:06:07.588327+0000) 2024-09-10T10:06:21.939 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:21 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:21.656+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:50.489601+0000 front 2024-09-10T10:05:50.489747+0000 (oldest deadline 2024-09-10T10:06:15.690110+0000) 2024-09-10T10:06:21.940 INFO:journalctl@ceph.osd.6.smithi077.stdout:Sep 10 10:06:21 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-6[57286]: 2024-09-10T10:06:21.503+0000 7f174e000640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:42.001143+0000 front 2024-09-10T10:05:42.001163+0000 (oldest deadline 2024-09-10T10:06:07.301413+0000) 2024-09-10T10:06:21.940 INFO:journalctl@ceph.osd.6.smithi077.stdout:Sep 10 10:06:21 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-6[57286]: 2024-09-10T10:06:21.503+0000 7f174e000640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:54.303272+0000 front 2024-09-10T10:05:54.303284+0000 (oldest deadline 2024-09-10T10:06:17.803077+0000) 2024-09-10T10:06:21.940 INFO:journalctl@ceph.osd.6.smithi077.stdout:Sep 10 10:06:21 smithi077 systemd[1]: Stopping Ceph osd.6 for 14a55226-6f59-11ef-bcea-c7b262605968... 2024-09-10T10:06:22.439 INFO:journalctl@ceph.osd.6.smithi077.stdout:Sep 10 10:06:22 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-6[57286]: 2024-09-10T10:06:22.066+0000 7f1757600640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:06:22.439 INFO:journalctl@ceph.osd.6.smithi077.stdout:Sep 10 10:06:22 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-6[57286]: 2024-09-10T10:06:22.066+0000 7f1757600640 -1 osd.6 100 *** Got signal Terminated *** 2024-09-10T10:06:22.439 INFO:journalctl@ceph.osd.6.smithi077.stdout:Sep 10 10:06:22 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-6[57286]: 2024-09-10T10:06:22.066+0000 7f1757600640 -1 osd.6 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T10:06:22.531 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:22 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:22.186+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.045437+0000 front 2024-09-10T10:05:37.045486+0000 (oldest deadline 2024-09-10T10:05:59.945393+0000) 2024-09-10T10:06:22.531 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:22 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:22.186+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:43.446497+0000 front 2024-09-10T10:05:43.446655+0000 (oldest deadline 2024-09-10T10:06:09.346712+0000) 2024-09-10T10:06:22.532 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:22 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:22.186+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:49.347200+0000 front 2024-09-10T10:05:49.347266+0000 (oldest deadline 2024-09-10T10:06:19.947685+0000) 2024-09-10T10:06:22.631 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:22 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8[54506]: 2024-09-10T10:06:22.150+0000 7fb09d2dc640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:45.855359+0000 front 2024-09-10T10:05:45.855515+0000 (oldest deadline 2024-09-10T10:06:08.155427+0000) 2024-09-10T10:06:22.631 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:22 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8[54506]: 2024-09-10T10:06:22.150+0000 7fb09d2dc640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:50.456414+0000 front 2024-09-10T10:05:50.456398+0000 (oldest deadline 2024-09-10T10:06:15.156538+0000) 2024-09-10T10:06:22.924 INFO:journalctl@ceph.osd.7.smithi120.stdout:Sep 10 10:06:22 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-7[52351]: 2024-09-10T10:06:22.530+0000 7f3f3f6e4640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:42.865355+0000 front 2024-09-10T10:05:42.865379+0000 (oldest deadline 2024-09-10T10:06:08.165545+0000) 2024-09-10T10:06:22.924 INFO:journalctl@ceph.osd.7.smithi120.stdout:Sep 10 10:06:22 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-7[52351]: 2024-09-10T10:06:22.530+0000 7f3f3f6e4640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:54.066777+0000 front 2024-09-10T10:05:54.066812+0000 (oldest deadline 2024-09-10T10:06:19.267357+0000) 2024-09-10T10:06:22.939 INFO:journalctl@ceph.osd.6.smithi077.stdout:Sep 10 10:06:22 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-6[57286]: 2024-09-10T10:06:22.531+0000 7f174e000640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:42.001143+0000 front 2024-09-10T10:05:42.001163+0000 (oldest deadline 2024-09-10T10:06:07.301413+0000) 2024-09-10T10:06:22.939 INFO:journalctl@ceph.osd.6.smithi077.stdout:Sep 10 10:06:22 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-6[57286]: 2024-09-10T10:06:22.532+0000 7f174e000640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:54.303272+0000 front 2024-09-10T10:05:54.303284+0000 (oldest deadline 2024-09-10T10:06:17.803077+0000) 2024-09-10T10:06:22.940 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:22 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:22.621+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.086443+0000 front 2024-09-10T10:05:37.086361+0000 (oldest deadline 2024-09-10T10:06:02.887395+0000) 2024-09-10T10:06:22.940 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:22 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:22.621+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:42.888131+0000 front 2024-09-10T10:05:42.888100+0000 (oldest deadline 2024-09-10T10:06:07.588327+0000) 2024-09-10T10:06:22.940 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:22 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:22.621+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:50.489601+0000 front 2024-09-10T10:05:50.489747+0000 (oldest deadline 2024-09-10T10:06:15.690110+0000) 2024-09-10T10:06:23.113 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:22 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:22.806+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:34.146460+0000 front 2024-09-10T10:05:34.146398+0000 (oldest deadline 2024-09-10T10:05:59.446520+0000) 2024-09-10T10:06:23.113 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:22 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:22.806+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:53.949416+0000 front 2024-09-10T10:05:53.949272+0000 (oldest deadline 2024-09-10T10:06:19.749872+0000) 2024-09-10T10:06:23.381 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:23 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8[54506]: 2024-09-10T10:06:23.112+0000 7fb09d2dc640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:45.855359+0000 front 2024-09-10T10:05:45.855515+0000 (oldest deadline 2024-09-10T10:06:08.155427+0000) 2024-09-10T10:06:23.381 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:23 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8[54506]: 2024-09-10T10:06:23.112+0000 7fb09d2dc640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:50.456414+0000 front 2024-09-10T10:05:50.456398+0000 (oldest deadline 2024-09-10T10:06:15.156538+0000) 2024-09-10T10:06:23.562 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:23 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:23.176+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.045437+0000 front 2024-09-10T10:05:37.045486+0000 (oldest deadline 2024-09-10T10:05:59.945393+0000) 2024-09-10T10:06:23.562 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:23 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:23.176+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:43.446497+0000 front 2024-09-10T10:05:43.446655+0000 (oldest deadline 2024-09-10T10:06:09.346712+0000) 2024-09-10T10:06:23.562 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:23 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:23.176+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:49.347200+0000 front 2024-09-10T10:05:49.347266+0000 (oldest deadline 2024-09-10T10:06:19.947685+0000) 2024-09-10T10:06:23.924 INFO:journalctl@ceph.osd.7.smithi120.stdout:Sep 10 10:06:23 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-7[52351]: 2024-09-10T10:06:23.561+0000 7f3f3f6e4640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:42.865355+0000 front 2024-09-10T10:05:42.865379+0000 (oldest deadline 2024-09-10T10:06:08.165545+0000) 2024-09-10T10:06:23.924 INFO:journalctl@ceph.osd.7.smithi120.stdout:Sep 10 10:06:23 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-7[52351]: 2024-09-10T10:06:23.561+0000 7f3f3f6e4640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:54.066777+0000 front 2024-09-10T10:05:54.066812+0000 (oldest deadline 2024-09-10T10:06:19.267357+0000) 2024-09-10T10:06:23.939 INFO:journalctl@ceph.osd.6.smithi077.stdout:Sep 10 10:06:23 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-6[57286]: 2024-09-10T10:06:23.545+0000 7f174e000640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:42.001143+0000 front 2024-09-10T10:05:42.001163+0000 (oldest deadline 2024-09-10T10:06:07.301413+0000) 2024-09-10T10:06:23.939 INFO:journalctl@ceph.osd.6.smithi077.stdout:Sep 10 10:06:23 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-6[57286]: 2024-09-10T10:06:23.545+0000 7f174e000640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:54.303272+0000 front 2024-09-10T10:05:54.303284+0000 (oldest deadline 2024-09-10T10:06:17.803077+0000) 2024-09-10T10:06:23.939 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:23 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:23.627+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.086443+0000 front 2024-09-10T10:05:37.086361+0000 (oldest deadline 2024-09-10T10:06:02.887395+0000) 2024-09-10T10:06:23.940 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:23 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:23.627+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:42.888131+0000 front 2024-09-10T10:05:42.888100+0000 (oldest deadline 2024-09-10T10:06:07.588327+0000) 2024-09-10T10:06:23.940 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:23 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:23.627+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:50.489601+0000 front 2024-09-10T10:05:50.489747+0000 (oldest deadline 2024-09-10T10:06:15.690110+0000) 2024-09-10T10:06:24.102 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:23 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:23.805+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:34.146460+0000 front 2024-09-10T10:05:34.146398+0000 (oldest deadline 2024-09-10T10:05:59.446520+0000) 2024-09-10T10:06:24.102 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:23 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:23.805+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:53.949416+0000 front 2024-09-10T10:05:53.949272+0000 (oldest deadline 2024-09-10T10:06:19.749872+0000) 2024-09-10T10:06:24.381 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:24 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8[54506]: 2024-09-10T10:06:24.101+0000 7fb09d2dc640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:45.855359+0000 front 2024-09-10T10:05:45.855515+0000 (oldest deadline 2024-09-10T10:06:08.155427+0000) 2024-09-10T10:06:24.381 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:24 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8[54506]: 2024-09-10T10:06:24.101+0000 7fb09d2dc640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:50.456414+0000 front 2024-09-10T10:05:50.456398+0000 (oldest deadline 2024-09-10T10:06:15.156538+0000) 2024-09-10T10:06:24.423 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:24 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:24.167+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.045437+0000 front 2024-09-10T10:05:37.045486+0000 (oldest deadline 2024-09-10T10:05:59.945393+0000) 2024-09-10T10:06:24.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:24 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:24.167+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:43.446497+0000 front 2024-09-10T10:05:43.446655+0000 (oldest deadline 2024-09-10T10:06:09.346712+0000) 2024-09-10T10:06:24.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:24 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:24.167+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:49.347200+0000 front 2024-09-10T10:05:49.347266+0000 (oldest deadline 2024-09-10T10:06:19.947685+0000) 2024-09-10T10:06:24.924 INFO:journalctl@ceph.osd.7.smithi120.stdout:Sep 10 10:06:24 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-7[52351]: 2024-09-10T10:06:24.596+0000 7f3f3f6e4640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:42.865355+0000 front 2024-09-10T10:05:42.865379+0000 (oldest deadline 2024-09-10T10:06:08.165545+0000) 2024-09-10T10:06:24.924 INFO:journalctl@ceph.osd.7.smithi120.stdout:Sep 10 10:06:24 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-7[52351]: 2024-09-10T10:06:24.596+0000 7f3f3f6e4640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:54.066777+0000 front 2024-09-10T10:05:54.066812+0000 (oldest deadline 2024-09-10T10:06:19.267357+0000) 2024-09-10T10:06:24.939 INFO:journalctl@ceph.osd.6.smithi077.stdout:Sep 10 10:06:24 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-6[57286]: 2024-09-10T10:06:24.498+0000 7f174e000640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:42.001143+0000 front 2024-09-10T10:05:42.001163+0000 (oldest deadline 2024-09-10T10:06:07.301413+0000) 2024-09-10T10:06:24.939 INFO:journalctl@ceph.osd.6.smithi077.stdout:Sep 10 10:06:24 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-6[57286]: 2024-09-10T10:06:24.498+0000 7f174e000640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:54.303272+0000 front 2024-09-10T10:05:54.303284+0000 (oldest deadline 2024-09-10T10:06:17.803077+0000) 2024-09-10T10:06:24.939 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:24 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:24.655+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.086443+0000 front 2024-09-10T10:05:37.086361+0000 (oldest deadline 2024-09-10T10:06:02.887395+0000) 2024-09-10T10:06:24.940 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:24 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:24.655+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:42.888131+0000 front 2024-09-10T10:05:42.888100+0000 (oldest deadline 2024-09-10T10:06:07.588327+0000) 2024-09-10T10:06:24.940 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:24 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:24.655+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:50.489601+0000 front 2024-09-10T10:05:50.489747+0000 (oldest deadline 2024-09-10T10:06:15.690110+0000) 2024-09-10T10:06:25.100 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:24 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:24.849+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:34.146460+0000 front 2024-09-10T10:05:34.146398+0000 (oldest deadline 2024-09-10T10:05:59.446520+0000) 2024-09-10T10:06:25.101 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:24 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:24.849+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:53.949416+0000 front 2024-09-10T10:05:53.949272+0000 (oldest deadline 2024-09-10T10:06:19.749872+0000) 2024-09-10T10:06:25.381 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:25 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8[54506]: 2024-09-10T10:06:25.099+0000 7fb09d2dc640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:45.855359+0000 front 2024-09-10T10:05:45.855515+0000 (oldest deadline 2024-09-10T10:06:08.155427+0000) 2024-09-10T10:06:25.381 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:25 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8[54506]: 2024-09-10T10:06:25.099+0000 7fb09d2dc640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:50.456414+0000 front 2024-09-10T10:05:50.456398+0000 (oldest deadline 2024-09-10T10:06:15.156538+0000) 2024-09-10T10:06:25.580 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:25 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:25.180+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.045437+0000 front 2024-09-10T10:05:37.045486+0000 (oldest deadline 2024-09-10T10:05:59.945393+0000) 2024-09-10T10:06:25.581 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:25 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:25.180+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:43.446497+0000 front 2024-09-10T10:05:43.446655+0000 (oldest deadline 2024-09-10T10:06:09.346712+0000) 2024-09-10T10:06:25.581 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:25 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:25.180+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:49.347200+0000 front 2024-09-10T10:05:49.347266+0000 (oldest deadline 2024-09-10T10:06:19.947685+0000) 2024-09-10T10:06:25.924 INFO:journalctl@ceph.osd.7.smithi120.stdout:Sep 10 10:06:25 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-7[52351]: 2024-09-10T10:06:25.579+0000 7f3f3f6e4640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:42.865355+0000 front 2024-09-10T10:05:42.865379+0000 (oldest deadline 2024-09-10T10:06:08.165545+0000) 2024-09-10T10:06:25.924 INFO:journalctl@ceph.osd.7.smithi120.stdout:Sep 10 10:06:25 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-7[52351]: 2024-09-10T10:06:25.579+0000 7f3f3f6e4640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:54.066777+0000 front 2024-09-10T10:05:54.066812+0000 (oldest deadline 2024-09-10T10:06:19.267357+0000) 2024-09-10T10:06:25.924 INFO:journalctl@ceph.osd.7.smithi120.stdout:Sep 10 10:06:25 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-7[52351]: 2024-09-10T10:06:25.579+0000 7f3f3f6e4640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:06:00.968626+0000 front 2024-09-10T10:06:00.968441+0000 (oldest deadline 2024-09-10T10:06:25.569031+0000) 2024-09-10T10:06:25.939 INFO:journalctl@ceph.osd.6.smithi077.stdout:Sep 10 10:06:25 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-6[57286]: 2024-09-10T10:06:25.469+0000 7f174e000640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:42.001143+0000 front 2024-09-10T10:05:42.001163+0000 (oldest deadline 2024-09-10T10:06:07.301413+0000) 2024-09-10T10:06:25.939 INFO:journalctl@ceph.osd.6.smithi077.stdout:Sep 10 10:06:25 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-6[57286]: 2024-09-10T10:06:25.469+0000 7f174e000640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:54.303272+0000 front 2024-09-10T10:05:54.303284+0000 (oldest deadline 2024-09-10T10:06:17.803077+0000) 2024-09-10T10:06:25.940 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:25 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:25.681+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.086443+0000 front 2024-09-10T10:05:37.086361+0000 (oldest deadline 2024-09-10T10:06:02.887395+0000) 2024-09-10T10:06:25.940 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:25 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:25.681+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:42.888131+0000 front 2024-09-10T10:05:42.888100+0000 (oldest deadline 2024-09-10T10:06:07.588327+0000) 2024-09-10T10:06:25.940 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:25 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:25.681+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:50.489601+0000 front 2024-09-10T10:05:50.489747+0000 (oldest deadline 2024-09-10T10:06:15.690110+0000) 2024-09-10T10:06:26.113 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:25 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:25.804+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:34.146460+0000 front 2024-09-10T10:05:34.146398+0000 (oldest deadline 2024-09-10T10:05:59.446520+0000) 2024-09-10T10:06:26.113 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:25 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:25.804+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:53.949416+0000 front 2024-09-10T10:05:53.949272+0000 (oldest deadline 2024-09-10T10:06:19.749872+0000) 2024-09-10T10:06:26.113 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:25 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:25.804+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:06:00.850855+0000 front 2024-09-10T10:06:00.850768+0000 (oldest deadline 2024-09-10T10:06:25.550818+0000) 2024-09-10T10:06:26.367 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:26 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8[54506]: 2024-09-10T10:06:26.145+0000 7fb09d2dc640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:45.855359+0000 front 2024-09-10T10:05:45.855515+0000 (oldest deadline 2024-09-10T10:06:08.155427+0000) 2024-09-10T10:06:26.367 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:26 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8[54506]: 2024-09-10T10:06:26.145+0000 7fb09d2dc640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:50.456414+0000 front 2024-09-10T10:05:50.456398+0000 (oldest deadline 2024-09-10T10:06:15.156538+0000) 2024-09-10T10:06:26.605 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:26 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:26.224+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.045437+0000 front 2024-09-10T10:05:37.045486+0000 (oldest deadline 2024-09-10T10:05:59.945393+0000) 2024-09-10T10:06:26.606 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:26 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:26.224+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:43.446497+0000 front 2024-09-10T10:05:43.446655+0000 (oldest deadline 2024-09-10T10:06:09.346712+0000) 2024-09-10T10:06:26.606 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:26 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:26.224+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:49.347200+0000 front 2024-09-10T10:05:49.347266+0000 (oldest deadline 2024-09-10T10:06:19.947685+0000) 2024-09-10T10:06:26.689 INFO:journalctl@ceph.osd.6.smithi077.stdout:Sep 10 10:06:26 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-6[57286]: 2024-09-10T10:06:26.427+0000 7f174e000640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:42.001143+0000 front 2024-09-10T10:05:42.001163+0000 (oldest deadline 2024-09-10T10:06:07.301413+0000) 2024-09-10T10:06:26.689 INFO:journalctl@ceph.osd.6.smithi077.stdout:Sep 10 10:06:26 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-6[57286]: 2024-09-10T10:06:26.427+0000 7f174e000640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:54.303272+0000 front 2024-09-10T10:05:54.303284+0000 (oldest deadline 2024-09-10T10:06:17.803077+0000) 2024-09-10T10:06:26.689 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:26 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:26.649+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.086443+0000 front 2024-09-10T10:05:37.086361+0000 (oldest deadline 2024-09-10T10:06:02.887395+0000) 2024-09-10T10:06:26.689 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:26 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:26.649+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:42.888131+0000 front 2024-09-10T10:05:42.888100+0000 (oldest deadline 2024-09-10T10:06:07.588327+0000) 2024-09-10T10:06:26.690 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:26 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:26.649+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:50.489601+0000 front 2024-09-10T10:05:50.489747+0000 (oldest deadline 2024-09-10T10:06:15.690110+0000) 2024-09-10T10:06:26.923 INFO:journalctl@ceph.osd.7.smithi120.stdout:Sep 10 10:06:26 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-7[52351]: 2024-09-10T10:06:26.604+0000 7f3f3f6e4640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:42.865355+0000 front 2024-09-10T10:05:42.865379+0000 (oldest deadline 2024-09-10T10:06:08.165545+0000) 2024-09-10T10:06:26.924 INFO:journalctl@ceph.osd.7.smithi120.stdout:Sep 10 10:06:26 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-7[52351]: 2024-09-10T10:06:26.604+0000 7f3f3f6e4640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:54.066777+0000 front 2024-09-10T10:05:54.066812+0000 (oldest deadline 2024-09-10T10:06:19.267357+0000) 2024-09-10T10:06:26.924 INFO:journalctl@ceph.osd.7.smithi120.stdout:Sep 10 10:06:26 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-7[52351]: 2024-09-10T10:06:26.604+0000 7f3f3f6e4640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:06:00.968626+0000 front 2024-09-10T10:06:00.968441+0000 (oldest deadline 2024-09-10T10:06:25.569031+0000) 2024-09-10T10:06:27.129 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:26 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:26.777+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:34.146460+0000 front 2024-09-10T10:05:34.146398+0000 (oldest deadline 2024-09-10T10:05:59.446520+0000) 2024-09-10T10:06:27.129 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:26 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:26.777+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:53.949416+0000 front 2024-09-10T10:05:53.949272+0000 (oldest deadline 2024-09-10T10:06:19.749872+0000) 2024-09-10T10:06:27.129 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:26 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:26.777+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:06:00.850855+0000 front 2024-09-10T10:06:00.850768+0000 (oldest deadline 2024-09-10T10:06:25.550818+0000) 2024-09-10T10:06:27.381 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:27 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8[54506]: 2024-09-10T10:06:27.127+0000 7fb09d2dc640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:45.855359+0000 front 2024-09-10T10:05:45.855515+0000 (oldest deadline 2024-09-10T10:06:08.155427+0000) 2024-09-10T10:06:27.381 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:27 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8[54506]: 2024-09-10T10:06:27.127+0000 7fb09d2dc640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:50.456414+0000 front 2024-09-10T10:05:50.456398+0000 (oldest deadline 2024-09-10T10:06:15.156538+0000) 2024-09-10T10:06:27.585 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:27 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:27.186+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.045437+0000 front 2024-09-10T10:05:37.045486+0000 (oldest deadline 2024-09-10T10:05:59.945393+0000) 2024-09-10T10:06:27.585 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:27 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:27.186+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:43.446497+0000 front 2024-09-10T10:05:43.446655+0000 (oldest deadline 2024-09-10T10:06:09.346712+0000) 2024-09-10T10:06:27.585 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:27 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:27.186+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:49.347200+0000 front 2024-09-10T10:05:49.347266+0000 (oldest deadline 2024-09-10T10:06:19.947685+0000) 2024-09-10T10:06:27.585 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:27 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:27.186+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:05:59.948402+0000 front 2024-09-10T10:05:59.948527+0000 (oldest deadline 2024-09-10T10:06:26.348781+0000) 2024-09-10T10:06:27.924 INFO:journalctl@ceph.osd.7.smithi120.stdout:Sep 10 10:06:27 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-7[52351]: 2024-09-10T10:06:27.584+0000 7f3f3f6e4640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:42.865355+0000 front 2024-09-10T10:05:42.865379+0000 (oldest deadline 2024-09-10T10:06:08.165545+0000) 2024-09-10T10:06:27.924 INFO:journalctl@ceph.osd.7.smithi120.stdout:Sep 10 10:06:27 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-7[52351]: 2024-09-10T10:06:27.584+0000 7f3f3f6e4640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:54.066777+0000 front 2024-09-10T10:05:54.066812+0000 (oldest deadline 2024-09-10T10:06:19.267357+0000) 2024-09-10T10:06:27.924 INFO:journalctl@ceph.osd.7.smithi120.stdout:Sep 10 10:06:27 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-7[52351]: 2024-09-10T10:06:27.584+0000 7f3f3f6e4640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:06:00.968626+0000 front 2024-09-10T10:06:00.968441+0000 (oldest deadline 2024-09-10T10:06:25.569031+0000) 2024-09-10T10:06:27.939 INFO:journalctl@ceph.osd.6.smithi077.stdout:Sep 10 10:06:27 smithi077 podman[128104]: 2024-09-10 10:06:27.641774069 +0000 UTC m=+5.704582190 container died fed7996e61fc3e4723fadc49d9b35f46f66bf8f779344b7b2356a05292097faf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-6, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, ceph=True) 2024-09-10T10:06:27.939 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:27 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:27.642+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.086443+0000 front 2024-09-10T10:05:37.086361+0000 (oldest deadline 2024-09-10T10:06:02.887395+0000) 2024-09-10T10:06:27.939 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:27 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:27.642+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:42.888131+0000 front 2024-09-10T10:05:42.888100+0000 (oldest deadline 2024-09-10T10:06:07.588327+0000) 2024-09-10T10:06:27.939 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:27 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:27.642+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:50.489601+0000 front 2024-09-10T10:05:50.489747+0000 (oldest deadline 2024-09-10T10:06:15.690110+0000) 2024-09-10T10:06:28.090 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:27 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:27.777+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:34.146460+0000 front 2024-09-10T10:05:34.146398+0000 (oldest deadline 2024-09-10T10:05:59.446520+0000) 2024-09-10T10:06:28.090 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:27 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:27.777+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:53.949416+0000 front 2024-09-10T10:05:53.949272+0000 (oldest deadline 2024-09-10T10:06:19.749872+0000) 2024-09-10T10:06:28.090 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:27 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:27.777+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:06:00.850855+0000 front 2024-09-10T10:06:00.850768+0000 (oldest deadline 2024-09-10T10:06:25.550818+0000) 2024-09-10T10:06:28.381 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:28 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8[54506]: 2024-09-10T10:06:28.088+0000 7fb09d2dc640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:45.855359+0000 front 2024-09-10T10:05:45.855515+0000 (oldest deadline 2024-09-10T10:06:08.155427+0000) 2024-09-10T10:06:28.381 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:28 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8[54506]: 2024-09-10T10:06:28.088+0000 7fb09d2dc640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:50.456414+0000 front 2024-09-10T10:05:50.456398+0000 (oldest deadline 2024-09-10T10:06:15.156538+0000) 2024-09-10T10:06:28.439 INFO:journalctl@ceph.osd.6.smithi077.stdout:Sep 10 10:06:28 smithi077 podman[128104]: 2024-09-10 10:06:28.147295816 +0000 UTC m=+6.210103935 container cleanup fed7996e61fc3e4723fadc49d9b35f46f66bf8f779344b7b2356a05292097faf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-6, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731) 2024-09-10T10:06:28.439 INFO:journalctl@ceph.osd.6.smithi077.stdout:Sep 10 10:06:28 smithi077 bash[128104]: ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-6 2024-09-10T10:06:28.540 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:28 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:28.202+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.045437+0000 front 2024-09-10T10:05:37.045486+0000 (oldest deadline 2024-09-10T10:05:59.945393+0000) 2024-09-10T10:06:28.541 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:28 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:28.202+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:43.446497+0000 front 2024-09-10T10:05:43.446655+0000 (oldest deadline 2024-09-10T10:06:09.346712+0000) 2024-09-10T10:06:28.541 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:28 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:28.202+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:49.347200+0000 front 2024-09-10T10:05:49.347266+0000 (oldest deadline 2024-09-10T10:06:19.947685+0000) 2024-09-10T10:06:28.541 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:28 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:28.202+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:05:59.948402+0000 front 2024-09-10T10:05:59.948527+0000 (oldest deadline 2024-09-10T10:06:26.348781+0000) 2024-09-10T10:06:28.874 INFO:journalctl@ceph.osd.6.smithi077.stdout:Sep 10 10:06:28 smithi077 podman[128121]: 2024-09-10 10:06:28.44046947 +0000 UTC m=+0.788528186 container remove fed7996e61fc3e4723fadc49d9b35f46f66bf8f779344b7b2356a05292097faf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-6, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731) 2024-09-10T10:06:28.874 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:28 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:28.595+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.086443+0000 front 2024-09-10T10:05:37.086361+0000 (oldest deadline 2024-09-10T10:06:02.887395+0000) 2024-09-10T10:06:28.875 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:28 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:28.595+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:42.888131+0000 front 2024-09-10T10:05:42.888100+0000 (oldest deadline 2024-09-10T10:06:07.588327+0000) 2024-09-10T10:06:28.875 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:28 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:28.595+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:50.489601+0000 front 2024-09-10T10:05:50.489747+0000 (oldest deadline 2024-09-10T10:06:15.690110+0000) 2024-09-10T10:06:28.875 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:28 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:28.595+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:06:02.492771+0000 front 2024-09-10T10:06:02.492798+0000 (oldest deadline 2024-09-10T10:06:27.792793+0000) 2024-09-10T10:06:28.923 INFO:journalctl@ceph.osd.7.smithi120.stdout:Sep 10 10:06:28 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-7[52351]: 2024-09-10T10:06:28.539+0000 7f3f3f6e4640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:42.865355+0000 front 2024-09-10T10:05:42.865379+0000 (oldest deadline 2024-09-10T10:06:08.165545+0000) 2024-09-10T10:06:28.924 INFO:journalctl@ceph.osd.7.smithi120.stdout:Sep 10 10:06:28 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-7[52351]: 2024-09-10T10:06:28.539+0000 7f3f3f6e4640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:54.066777+0000 front 2024-09-10T10:05:54.066812+0000 (oldest deadline 2024-09-10T10:06:19.267357+0000) 2024-09-10T10:06:28.924 INFO:journalctl@ceph.osd.7.smithi120.stdout:Sep 10 10:06:28 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-7[52351]: 2024-09-10T10:06:28.539+0000 7f3f3f6e4640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:06:00.968626+0000 front 2024-09-10T10:06:00.968441+0000 (oldest deadline 2024-09-10T10:06:25.569031+0000) 2024-09-10T10:06:29.051 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:28 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:28.769+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:34.146460+0000 front 2024-09-10T10:05:34.146398+0000 (oldest deadline 2024-09-10T10:05:59.446520+0000) 2024-09-10T10:06:29.051 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:28 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:28.769+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:53.949416+0000 front 2024-09-10T10:05:53.949272+0000 (oldest deadline 2024-09-10T10:06:19.749872+0000) 2024-09-10T10:06:29.052 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:28 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:28.769+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:06:00.850855+0000 front 2024-09-10T10:06:00.850768+0000 (oldest deadline 2024-09-10T10:06:25.550818+0000) 2024-09-10T10:06:29.161 INFO:journalctl@ceph.osd.6.smithi077.stdout:Sep 10 10:06:28 smithi077 podman[128206]: 2024-09-10 10:06:28.777549478 +0000 UTC m=+0.026800707 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T10:06:29.161 INFO:journalctl@ceph.osd.6.smithi077.stdout:Sep 10 10:06:28 smithi077 podman[128206]: 2024-09-10 10:06:28.883174902 +0000 UTC m=+0.132426132 container create 09db957995ece2eb4407b1175c5180a063f6d37d94d5f634e0a9ce46d9e7e692 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-6-deactivate, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-10T10:06:29.161 INFO:journalctl@ceph.osd.6.smithi077.stdout:Sep 10 10:06:29 smithi077 podman[128206]: 2024-09-10 10:06:29.160733666 +0000 UTC m=+0.409984896 container init 09db957995ece2eb4407b1175c5180a063f6d37d94d5f634e0a9ce46d9e7e692 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-6-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-10T10:06:29.381 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:29 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8[54506]: 2024-09-10T10:06:29.050+0000 7fb09d2dc640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:45.855359+0000 front 2024-09-10T10:05:45.855515+0000 (oldest deadline 2024-09-10T10:06:08.155427+0000) 2024-09-10T10:06:29.381 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:29 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8[54506]: 2024-09-10T10:06:29.050+0000 7fb09d2dc640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:50.456414+0000 front 2024-09-10T10:05:50.456398+0000 (oldest deadline 2024-09-10T10:06:15.156538+0000) 2024-09-10T10:06:29.381 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:29 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8[54506]: 2024-09-10T10:06:29.050+0000 7fb09d2dc640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:06:00.957951+0000 front 2024-09-10T10:06:00.957990+0000 (oldest deadline 2024-09-10T10:06:28.558458+0000) 2024-09-10T10:06:29.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:29 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:29.243+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.045437+0000 front 2024-09-10T10:05:37.045486+0000 (oldest deadline 2024-09-10T10:05:59.945393+0000) 2024-09-10T10:06:29.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:29 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:29.243+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:43.446497+0000 front 2024-09-10T10:05:43.446655+0000 (oldest deadline 2024-09-10T10:06:09.346712+0000) 2024-09-10T10:06:29.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:29 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:29.243+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:49.347200+0000 front 2024-09-10T10:05:49.347266+0000 (oldest deadline 2024-09-10T10:06:19.947685+0000) 2024-09-10T10:06:29.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:29 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:29.243+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:05:59.948402+0000 front 2024-09-10T10:05:59.948527+0000 (oldest deadline 2024-09-10T10:06:26.348781+0000) 2024-09-10T10:06:29.431 INFO:journalctl@ceph.osd.6.smithi077.stdout:Sep 10 10:06:29 smithi077 podman[128206]: 2024-09-10 10:06:29.167505195 +0000 UTC m=+0.416756425 container start 09db957995ece2eb4407b1175c5180a063f6d37d94d5f634e0a9ce46d9e7e692 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-10T10:06:29.431 INFO:journalctl@ceph.osd.6.smithi077.stdout:Sep 10 10:06:29 smithi077 podman[128206]: 2024-09-10 10:06:29.252130864 +0000 UTC m=+0.501382125 container attach 09db957995ece2eb4407b1175c5180a063f6d37d94d5f634e0a9ce46d9e7e692 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-6-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-10T10:06:29.689 INFO:journalctl@ceph.osd.6.smithi077.stdout:Sep 10 10:06:29 smithi077 podman[128229]: 2024-09-10 10:06:29.469842035 +0000 UTC m=+0.025500287 container died 09db957995ece2eb4407b1175c5180a063f6d37d94d5f634e0a9ce46d9e7e692 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-6-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-10T10:06:29.690 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:29 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:29.601+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.086443+0000 front 2024-09-10T10:05:37.086361+0000 (oldest deadline 2024-09-10T10:06:02.887395+0000) 2024-09-10T10:06:29.690 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:29 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:29.601+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:42.888131+0000 front 2024-09-10T10:05:42.888100+0000 (oldest deadline 2024-09-10T10:06:07.588327+0000) 2024-09-10T10:06:29.690 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:29 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:29.601+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:50.489601+0000 front 2024-09-10T10:05:50.489747+0000 (oldest deadline 2024-09-10T10:06:15.690110+0000) 2024-09-10T10:06:29.690 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:29 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:29.601+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:06:02.492771+0000 front 2024-09-10T10:06:02.492798+0000 (oldest deadline 2024-09-10T10:06:27.792793+0000) 2024-09-10T10:06:29.924 INFO:journalctl@ceph.osd.7.smithi120.stdout:Sep 10 10:06:29 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-7[52351]: 2024-09-10T10:06:29.508+0000 7f3f3f6e4640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:42.865355+0000 front 2024-09-10T10:05:42.865379+0000 (oldest deadline 2024-09-10T10:06:08.165545+0000) 2024-09-10T10:06:29.924 INFO:journalctl@ceph.osd.7.smithi120.stdout:Sep 10 10:06:29 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-7[52351]: 2024-09-10T10:06:29.508+0000 7f3f3f6e4640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:54.066777+0000 front 2024-09-10T10:05:54.066812+0000 (oldest deadline 2024-09-10T10:06:19.267357+0000) 2024-09-10T10:06:29.924 INFO:journalctl@ceph.osd.7.smithi120.stdout:Sep 10 10:06:29 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-7[52351]: 2024-09-10T10:06:29.508+0000 7f3f3f6e4640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:06:00.968626+0000 front 2024-09-10T10:06:00.968441+0000 (oldest deadline 2024-09-10T10:06:25.569031+0000) 2024-09-10T10:06:30.015 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:29 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:29.736+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:34.146460+0000 front 2024-09-10T10:05:34.146398+0000 (oldest deadline 2024-09-10T10:05:59.446520+0000) 2024-09-10T10:06:30.015 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:29 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:29.736+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:53.949416+0000 front 2024-09-10T10:05:53.949272+0000 (oldest deadline 2024-09-10T10:06:19.749872+0000) 2024-09-10T10:06:30.015 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:29 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:29.736+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:06:00.850855+0000 front 2024-09-10T10:06:00.850768+0000 (oldest deadline 2024-09-10T10:06:25.550818+0000) 2024-09-10T10:06:30.179 DEBUG:teuthology.orchestra.run.smithi077:> sudo pkill -f 'journalctl -f -n 0 -u ceph-14a55226-6f59-11ef-bcea-c7b262605968@osd.6.service' 2024-09-10T10:06:30.253 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:06:30.254 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-10T10:06:30.254 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-10T10:06:30.254 DEBUG:teuthology.orchestra.run.smithi120:> sudo systemctl stop ceph-14a55226-6f59-11ef-bcea-c7b262605968@osd.7 2024-09-10T10:06:30.381 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:30 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8[54506]: 2024-09-10T10:06:30.014+0000 7fb09d2dc640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:45.855359+0000 front 2024-09-10T10:05:45.855515+0000 (oldest deadline 2024-09-10T10:06:08.155427+0000) 2024-09-10T10:06:30.381 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:30 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8[54506]: 2024-09-10T10:06:30.014+0000 7fb09d2dc640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:50.456414+0000 front 2024-09-10T10:05:50.456398+0000 (oldest deadline 2024-09-10T10:06:15.156538+0000) 2024-09-10T10:06:30.381 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:30 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8[54506]: 2024-09-10T10:06:30.014+0000 7fb09d2dc640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:06:00.957951+0000 front 2024-09-10T10:06:00.957990+0000 (oldest deadline 2024-09-10T10:06:28.558458+0000) 2024-09-10T10:06:30.518 INFO:journalctl@ceph.osd.7.smithi120.stdout:Sep 10 10:06:30 smithi120 systemd[1]: Stopping Ceph osd.7 for 14a55226-6f59-11ef-bcea-c7b262605968... 2024-09-10T10:06:30.519 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:30 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:30.238+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.045437+0000 front 2024-09-10T10:05:37.045486+0000 (oldest deadline 2024-09-10T10:05:59.945393+0000) 2024-09-10T10:06:30.519 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:30 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:30.238+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:43.446497+0000 front 2024-09-10T10:05:43.446655+0000 (oldest deadline 2024-09-10T10:06:09.346712+0000) 2024-09-10T10:06:30.519 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:30 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:30.238+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:49.347200+0000 front 2024-09-10T10:05:49.347266+0000 (oldest deadline 2024-09-10T10:06:19.947685+0000) 2024-09-10T10:06:30.519 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:30 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:30.238+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:05:59.948402+0000 front 2024-09-10T10:05:59.948527+0000 (oldest deadline 2024-09-10T10:06:26.348781+0000) 2024-09-10T10:06:30.924 INFO:journalctl@ceph.osd.7.smithi120.stdout:Sep 10 10:06:30 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-7[52351]: 2024-09-10T10:06:30.517+0000 7f3f3f6e4640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:42.865355+0000 front 2024-09-10T10:05:42.865379+0000 (oldest deadline 2024-09-10T10:06:08.165545+0000) 2024-09-10T10:06:30.924 INFO:journalctl@ceph.osd.7.smithi120.stdout:Sep 10 10:06:30 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-7[52351]: 2024-09-10T10:06:30.517+0000 7f3f3f6e4640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:54.066777+0000 front 2024-09-10T10:05:54.066812+0000 (oldest deadline 2024-09-10T10:06:19.267357+0000) 2024-09-10T10:06:30.924 INFO:journalctl@ceph.osd.7.smithi120.stdout:Sep 10 10:06:30 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-7[52351]: 2024-09-10T10:06:30.517+0000 7f3f3f6e4640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:06:00.968626+0000 front 2024-09-10T10:06:00.968441+0000 (oldest deadline 2024-09-10T10:06:25.569031+0000) 2024-09-10T10:06:30.924 INFO:journalctl@ceph.osd.7.smithi120.stdout:Sep 10 10:06:30 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-7[52351]: 2024-09-10T10:06:30.584+0000 7f3f430ce640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:06:30.924 INFO:journalctl@ceph.osd.7.smithi120.stdout:Sep 10 10:06:30 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-7[52351]: 2024-09-10T10:06:30.584+0000 7f3f430ce640 -1 osd.7 100 *** Got signal Terminated *** 2024-09-10T10:06:30.924 INFO:journalctl@ceph.osd.7.smithi120.stdout:Sep 10 10:06:30 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-7[52351]: 2024-09-10T10:06:30.584+0000 7f3f430ce640 -1 osd.7 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T10:06:30.939 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:30 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:30.571+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.086443+0000 front 2024-09-10T10:05:37.086361+0000 (oldest deadline 2024-09-10T10:06:02.887395+0000) 2024-09-10T10:06:30.939 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:30 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:30.571+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:42.888131+0000 front 2024-09-10T10:05:42.888100+0000 (oldest deadline 2024-09-10T10:06:07.588327+0000) 2024-09-10T10:06:30.939 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:30 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:30.571+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:50.489601+0000 front 2024-09-10T10:05:50.489747+0000 (oldest deadline 2024-09-10T10:06:15.690110+0000) 2024-09-10T10:06:30.940 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:30 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:30.571+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:06:02.492771+0000 front 2024-09-10T10:06:02.492798+0000 (oldest deadline 2024-09-10T10:06:27.792793+0000) 2024-09-10T10:06:31.058 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:30 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:30.733+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:34.146460+0000 front 2024-09-10T10:05:34.146398+0000 (oldest deadline 2024-09-10T10:05:59.446520+0000) 2024-09-10T10:06:31.058 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:30 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:30.733+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:53.949416+0000 front 2024-09-10T10:05:53.949272+0000 (oldest deadline 2024-09-10T10:06:19.749872+0000) 2024-09-10T10:06:31.058 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:30 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:30.733+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:06:00.850855+0000 front 2024-09-10T10:06:00.850768+0000 (oldest deadline 2024-09-10T10:06:25.550818+0000) 2024-09-10T10:06:31.381 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:31 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8[54506]: 2024-09-10T10:06:31.057+0000 7fb09d2dc640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:45.855359+0000 front 2024-09-10T10:05:45.855515+0000 (oldest deadline 2024-09-10T10:06:08.155427+0000) 2024-09-10T10:06:31.381 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:31 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8[54506]: 2024-09-10T10:06:31.057+0000 7fb09d2dc640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:50.456414+0000 front 2024-09-10T10:05:50.456398+0000 (oldest deadline 2024-09-10T10:06:15.156538+0000) 2024-09-10T10:06:31.381 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:31 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8[54506]: 2024-09-10T10:06:31.057+0000 7fb09d2dc640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:06:00.957951+0000 front 2024-09-10T10:06:00.957990+0000 (oldest deadline 2024-09-10T10:06:28.558458+0000) 2024-09-10T10:06:31.566 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:31 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:31.267+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.045437+0000 front 2024-09-10T10:05:37.045486+0000 (oldest deadline 2024-09-10T10:05:59.945393+0000) 2024-09-10T10:06:31.566 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:31 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:31.267+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:43.446497+0000 front 2024-09-10T10:05:43.446655+0000 (oldest deadline 2024-09-10T10:06:09.346712+0000) 2024-09-10T10:06:31.566 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:31 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:31.267+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:49.347200+0000 front 2024-09-10T10:05:49.347266+0000 (oldest deadline 2024-09-10T10:06:19.947685+0000) 2024-09-10T10:06:31.566 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:31 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:31.267+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:05:59.948402+0000 front 2024-09-10T10:05:59.948527+0000 (oldest deadline 2024-09-10T10:06:26.348781+0000) 2024-09-10T10:06:31.924 INFO:journalctl@ceph.osd.7.smithi120.stdout:Sep 10 10:06:31 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-7[52351]: 2024-09-10T10:06:31.565+0000 7f3f3f6e4640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:42.865355+0000 front 2024-09-10T10:05:42.865379+0000 (oldest deadline 2024-09-10T10:06:08.165545+0000) 2024-09-10T10:06:31.924 INFO:journalctl@ceph.osd.7.smithi120.stdout:Sep 10 10:06:31 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-7[52351]: 2024-09-10T10:06:31.565+0000 7f3f3f6e4640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:54.066777+0000 front 2024-09-10T10:05:54.066812+0000 (oldest deadline 2024-09-10T10:06:19.267357+0000) 2024-09-10T10:06:31.924 INFO:journalctl@ceph.osd.7.smithi120.stdout:Sep 10 10:06:31 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-7[52351]: 2024-09-10T10:06:31.565+0000 7f3f3f6e4640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:06:00.968626+0000 front 2024-09-10T10:06:00.968441+0000 (oldest deadline 2024-09-10T10:06:25.569031+0000) 2024-09-10T10:06:31.939 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:31 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:31.547+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.086443+0000 front 2024-09-10T10:05:37.086361+0000 (oldest deadline 2024-09-10T10:06:02.887395+0000) 2024-09-10T10:06:31.939 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:31 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:31.547+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:42.888131+0000 front 2024-09-10T10:05:42.888100+0000 (oldest deadline 2024-09-10T10:06:07.588327+0000) 2024-09-10T10:06:31.939 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:31 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:31.547+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:50.489601+0000 front 2024-09-10T10:05:50.489747+0000 (oldest deadline 2024-09-10T10:06:15.690110+0000) 2024-09-10T10:06:31.939 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:31 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:31.547+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:06:02.492771+0000 front 2024-09-10T10:06:02.492798+0000 (oldest deadline 2024-09-10T10:06:27.792793+0000) 2024-09-10T10:06:32.063 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:31 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:31.781+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:34.146460+0000 front 2024-09-10T10:05:34.146398+0000 (oldest deadline 2024-09-10T10:05:59.446520+0000) 2024-09-10T10:06:32.063 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:31 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:31.781+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:53.949416+0000 front 2024-09-10T10:05:53.949272+0000 (oldest deadline 2024-09-10T10:06:19.749872+0000) 2024-09-10T10:06:32.063 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:31 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:31.781+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:06:00.850855+0000 front 2024-09-10T10:06:00.850768+0000 (oldest deadline 2024-09-10T10:06:25.550818+0000) 2024-09-10T10:06:32.381 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:32 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8[54506]: 2024-09-10T10:06:32.062+0000 7fb09d2dc640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:45.855359+0000 front 2024-09-10T10:05:45.855515+0000 (oldest deadline 2024-09-10T10:06:08.155427+0000) 2024-09-10T10:06:32.381 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:32 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8[54506]: 2024-09-10T10:06:32.062+0000 7fb09d2dc640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:50.456414+0000 front 2024-09-10T10:05:50.456398+0000 (oldest deadline 2024-09-10T10:06:15.156538+0000) 2024-09-10T10:06:32.381 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:32 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8[54506]: 2024-09-10T10:06:32.062+0000 7fb09d2dc640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:06:00.957951+0000 front 2024-09-10T10:06:00.957990+0000 (oldest deadline 2024-09-10T10:06:28.558458+0000) 2024-09-10T10:06:32.572 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:32 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:32.278+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.045437+0000 front 2024-09-10T10:05:37.045486+0000 (oldest deadline 2024-09-10T10:05:59.945393+0000) 2024-09-10T10:06:32.573 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:32 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:32.278+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:43.446497+0000 front 2024-09-10T10:05:43.446655+0000 (oldest deadline 2024-09-10T10:06:09.346712+0000) 2024-09-10T10:06:32.573 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:32 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:32.278+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:49.347200+0000 front 2024-09-10T10:05:49.347266+0000 (oldest deadline 2024-09-10T10:06:19.947685+0000) 2024-09-10T10:06:32.573 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:32 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:32.278+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:05:59.948402+0000 front 2024-09-10T10:05:59.948527+0000 (oldest deadline 2024-09-10T10:06:26.348781+0000) 2024-09-10T10:06:32.923 INFO:journalctl@ceph.osd.7.smithi120.stdout:Sep 10 10:06:32 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-7[52351]: 2024-09-10T10:06:32.571+0000 7f3f3f6e4640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:42.865355+0000 front 2024-09-10T10:05:42.865379+0000 (oldest deadline 2024-09-10T10:06:08.165545+0000) 2024-09-10T10:06:32.924 INFO:journalctl@ceph.osd.7.smithi120.stdout:Sep 10 10:06:32 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-7[52351]: 2024-09-10T10:06:32.571+0000 7f3f3f6e4640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:54.066777+0000 front 2024-09-10T10:05:54.066812+0000 (oldest deadline 2024-09-10T10:06:19.267357+0000) 2024-09-10T10:06:32.924 INFO:journalctl@ceph.osd.7.smithi120.stdout:Sep 10 10:06:32 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-7[52351]: 2024-09-10T10:06:32.571+0000 7f3f3f6e4640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:06:00.968626+0000 front 2024-09-10T10:06:00.968441+0000 (oldest deadline 2024-09-10T10:06:25.569031+0000) 2024-09-10T10:06:32.939 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:32 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:32.520+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.086443+0000 front 2024-09-10T10:05:37.086361+0000 (oldest deadline 2024-09-10T10:06:02.887395+0000) 2024-09-10T10:06:32.939 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:32 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:32.520+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:42.888131+0000 front 2024-09-10T10:05:42.888100+0000 (oldest deadline 2024-09-10T10:06:07.588327+0000) 2024-09-10T10:06:32.940 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:32 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:32.520+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:50.489601+0000 front 2024-09-10T10:05:50.489747+0000 (oldest deadline 2024-09-10T10:06:15.690110+0000) 2024-09-10T10:06:32.940 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:32 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:32.520+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:06:02.492771+0000 front 2024-09-10T10:06:02.492798+0000 (oldest deadline 2024-09-10T10:06:27.792793+0000) 2024-09-10T10:06:32.940 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:32 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:32.520+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-09-10T10:06:07.793699+0000 front 2024-09-10T10:06:07.793464+0000 (oldest deadline 2024-09-10T10:06:31.893248+0000) 2024-09-10T10:06:33.066 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:32 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:32.801+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:34.146460+0000 front 2024-09-10T10:05:34.146398+0000 (oldest deadline 2024-09-10T10:05:59.446520+0000) 2024-09-10T10:06:33.066 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:32 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:32.801+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:53.949416+0000 front 2024-09-10T10:05:53.949272+0000 (oldest deadline 2024-09-10T10:06:19.749872+0000) 2024-09-10T10:06:33.066 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:32 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:32.801+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:06:00.850855+0000 front 2024-09-10T10:06:00.850768+0000 (oldest deadline 2024-09-10T10:06:25.550818+0000) 2024-09-10T10:06:33.381 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:33 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8[54506]: 2024-09-10T10:06:33.065+0000 7fb09d2dc640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:45.855359+0000 front 2024-09-10T10:05:45.855515+0000 (oldest deadline 2024-09-10T10:06:08.155427+0000) 2024-09-10T10:06:33.381 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:33 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8[54506]: 2024-09-10T10:06:33.065+0000 7fb09d2dc640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:50.456414+0000 front 2024-09-10T10:05:50.456398+0000 (oldest deadline 2024-09-10T10:06:15.156538+0000) 2024-09-10T10:06:33.381 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:33 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8[54506]: 2024-09-10T10:06:33.065+0000 7fb09d2dc640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:06:00.957951+0000 front 2024-09-10T10:06:00.957990+0000 (oldest deadline 2024-09-10T10:06:28.558458+0000) 2024-09-10T10:06:33.597 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:33 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:33.274+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.045437+0000 front 2024-09-10T10:05:37.045486+0000 (oldest deadline 2024-09-10T10:05:59.945393+0000) 2024-09-10T10:06:33.597 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:33 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:33.274+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:43.446497+0000 front 2024-09-10T10:05:43.446655+0000 (oldest deadline 2024-09-10T10:06:09.346712+0000) 2024-09-10T10:06:33.597 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:33 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:33.274+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:49.347200+0000 front 2024-09-10T10:05:49.347266+0000 (oldest deadline 2024-09-10T10:06:19.947685+0000) 2024-09-10T10:06:33.597 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:33 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:33.274+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:05:59.948402+0000 front 2024-09-10T10:05:59.948527+0000 (oldest deadline 2024-09-10T10:06:26.348781+0000) 2024-09-10T10:06:33.924 INFO:journalctl@ceph.osd.7.smithi120.stdout:Sep 10 10:06:33 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-7[52351]: 2024-09-10T10:06:33.596+0000 7f3f3f6e4640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:42.865355+0000 front 2024-09-10T10:05:42.865379+0000 (oldest deadline 2024-09-10T10:06:08.165545+0000) 2024-09-10T10:06:33.924 INFO:journalctl@ceph.osd.7.smithi120.stdout:Sep 10 10:06:33 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-7[52351]: 2024-09-10T10:06:33.596+0000 7f3f3f6e4640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:54.066777+0000 front 2024-09-10T10:05:54.066812+0000 (oldest deadline 2024-09-10T10:06:19.267357+0000) 2024-09-10T10:06:33.924 INFO:journalctl@ceph.osd.7.smithi120.stdout:Sep 10 10:06:33 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-7[52351]: 2024-09-10T10:06:33.596+0000 7f3f3f6e4640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:06:00.968626+0000 front 2024-09-10T10:06:00.968441+0000 (oldest deadline 2024-09-10T10:06:25.569031+0000) 2024-09-10T10:06:33.939 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:33 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:33.488+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.086443+0000 front 2024-09-10T10:05:37.086361+0000 (oldest deadline 2024-09-10T10:06:02.887395+0000) 2024-09-10T10:06:33.939 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:33 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:33.488+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:42.888131+0000 front 2024-09-10T10:05:42.888100+0000 (oldest deadline 2024-09-10T10:06:07.588327+0000) 2024-09-10T10:06:33.939 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:33 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:33.488+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:50.489601+0000 front 2024-09-10T10:05:50.489747+0000 (oldest deadline 2024-09-10T10:06:15.690110+0000) 2024-09-10T10:06:33.939 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:33 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:33.488+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:06:02.492771+0000 front 2024-09-10T10:06:02.492798+0000 (oldest deadline 2024-09-10T10:06:27.792793+0000) 2024-09-10T10:06:33.939 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:33 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:33.488+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-09-10T10:06:07.793699+0000 front 2024-09-10T10:06:07.793464+0000 (oldest deadline 2024-09-10T10:06:31.893248+0000) 2024-09-10T10:06:34.047 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:33 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:33.759+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:34.146460+0000 front 2024-09-10T10:05:34.146398+0000 (oldest deadline 2024-09-10T10:05:59.446520+0000) 2024-09-10T10:06:34.047 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:33 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:33.759+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:53.949416+0000 front 2024-09-10T10:05:53.949272+0000 (oldest deadline 2024-09-10T10:06:19.749872+0000) 2024-09-10T10:06:34.047 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:33 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:33.759+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:06:00.850855+0000 front 2024-09-10T10:06:00.850768+0000 (oldest deadline 2024-09-10T10:06:25.550818+0000) 2024-09-10T10:06:34.381 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:34 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8[54506]: 2024-09-10T10:06:34.046+0000 7fb09d2dc640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:45.855359+0000 front 2024-09-10T10:05:45.855515+0000 (oldest deadline 2024-09-10T10:06:08.155427+0000) 2024-09-10T10:06:34.381 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:34 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8[54506]: 2024-09-10T10:06:34.046+0000 7fb09d2dc640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:50.456414+0000 front 2024-09-10T10:05:50.456398+0000 (oldest deadline 2024-09-10T10:06:15.156538+0000) 2024-09-10T10:06:34.381 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:34 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8[54506]: 2024-09-10T10:06:34.046+0000 7fb09d2dc640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:06:00.957951+0000 front 2024-09-10T10:06:00.957990+0000 (oldest deadline 2024-09-10T10:06:28.558458+0000) 2024-09-10T10:06:34.571 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:34 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:34.235+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.045437+0000 front 2024-09-10T10:05:37.045486+0000 (oldest deadline 2024-09-10T10:05:59.945393+0000) 2024-09-10T10:06:34.571 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:34 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:34.235+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:43.446497+0000 front 2024-09-10T10:05:43.446655+0000 (oldest deadline 2024-09-10T10:06:09.346712+0000) 2024-09-10T10:06:34.571 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:34 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:34.235+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:49.347200+0000 front 2024-09-10T10:05:49.347266+0000 (oldest deadline 2024-09-10T10:06:19.947685+0000) 2024-09-10T10:06:34.571 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:34 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:34.235+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:05:59.948402+0000 front 2024-09-10T10:05:59.948527+0000 (oldest deadline 2024-09-10T10:06:26.348781+0000) 2024-09-10T10:06:34.571 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:34 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:34.235+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-09-10T10:06:10.349977+0000 front 2024-09-10T10:06:10.350027+0000 (oldest deadline 2024-09-10T10:06:33.750499+0000) 2024-09-10T10:06:34.923 INFO:journalctl@ceph.osd.7.smithi120.stdout:Sep 10 10:06:34 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-7[52351]: 2024-09-10T10:06:34.570+0000 7f3f3f6e4640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:42.865355+0000 front 2024-09-10T10:05:42.865379+0000 (oldest deadline 2024-09-10T10:06:08.165545+0000) 2024-09-10T10:06:34.924 INFO:journalctl@ceph.osd.7.smithi120.stdout:Sep 10 10:06:34 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-7[52351]: 2024-09-10T10:06:34.570+0000 7f3f3f6e4640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:54.066777+0000 front 2024-09-10T10:05:54.066812+0000 (oldest deadline 2024-09-10T10:06:19.267357+0000) 2024-09-10T10:06:34.924 INFO:journalctl@ceph.osd.7.smithi120.stdout:Sep 10 10:06:34 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-7[52351]: 2024-09-10T10:06:34.570+0000 7f3f3f6e4640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:06:00.968626+0000 front 2024-09-10T10:06:00.968441+0000 (oldest deadline 2024-09-10T10:06:25.569031+0000) 2024-09-10T10:06:34.939 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:34 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:34.477+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.086443+0000 front 2024-09-10T10:05:37.086361+0000 (oldest deadline 2024-09-10T10:06:02.887395+0000) 2024-09-10T10:06:34.939 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:34 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:34.477+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:42.888131+0000 front 2024-09-10T10:05:42.888100+0000 (oldest deadline 2024-09-10T10:06:07.588327+0000) 2024-09-10T10:06:34.939 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:34 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:34.477+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:50.489601+0000 front 2024-09-10T10:05:50.489747+0000 (oldest deadline 2024-09-10T10:06:15.690110+0000) 2024-09-10T10:06:34.939 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:34 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:34.477+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:06:02.492771+0000 front 2024-09-10T10:06:02.492798+0000 (oldest deadline 2024-09-10T10:06:27.792793+0000) 2024-09-10T10:06:34.940 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:34 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:34.477+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-09-10T10:06:07.793699+0000 front 2024-09-10T10:06:07.793464+0000 (oldest deadline 2024-09-10T10:06:31.893248+0000) 2024-09-10T10:06:35.131 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:35 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8[54506]: 2024-09-10T10:06:35.027+0000 7fb09d2dc640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:45.855359+0000 front 2024-09-10T10:05:45.855515+0000 (oldest deadline 2024-09-10T10:06:08.155427+0000) 2024-09-10T10:06:35.131 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:35 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8[54506]: 2024-09-10T10:06:35.027+0000 7fb09d2dc640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:50.456414+0000 front 2024-09-10T10:05:50.456398+0000 (oldest deadline 2024-09-10T10:06:15.156538+0000) 2024-09-10T10:06:35.131 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:35 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8[54506]: 2024-09-10T10:06:35.027+0000 7fb09d2dc640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:06:00.957951+0000 front 2024-09-10T10:06:00.957990+0000 (oldest deadline 2024-09-10T10:06:28.558458+0000) 2024-09-10T10:06:35.131 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:35 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8[54506]: 2024-09-10T10:06:35.027+0000 7fb09d2dc640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-09-10T10:06:08.559135+0000 front 2024-09-10T10:06:08.559098+0000 (oldest deadline 2024-09-10T10:06:34.458944+0000) 2024-09-10T10:06:35.132 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:34 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:34.782+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:34.146460+0000 front 2024-09-10T10:05:34.146398+0000 (oldest deadline 2024-09-10T10:05:59.446520+0000) 2024-09-10T10:06:35.132 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:34 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:34.782+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:53.949416+0000 front 2024-09-10T10:05:53.949272+0000 (oldest deadline 2024-09-10T10:06:19.749872+0000) 2024-09-10T10:06:35.132 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:34 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:34.782+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:06:00.850855+0000 front 2024-09-10T10:06:00.850768+0000 (oldest deadline 2024-09-10T10:06:25.550818+0000) 2024-09-10T10:06:35.132 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:34 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:34.782+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-09-10T10:06:09.552349+0000 front 2024-09-10T10:06:09.552369+0000 (oldest deadline 2024-09-10T10:06:34.252329+0000) 2024-09-10T10:06:35.674 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:35 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:35.239+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.045437+0000 front 2024-09-10T10:05:37.045486+0000 (oldest deadline 2024-09-10T10:05:59.945393+0000) 2024-09-10T10:06:35.674 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:35 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:35.239+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:43.446497+0000 front 2024-09-10T10:05:43.446655+0000 (oldest deadline 2024-09-10T10:06:09.346712+0000) 2024-09-10T10:06:35.674 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:35 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:35.239+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:49.347200+0000 front 2024-09-10T10:05:49.347266+0000 (oldest deadline 2024-09-10T10:06:19.947685+0000) 2024-09-10T10:06:35.674 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:35 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:35.239+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:05:59.948402+0000 front 2024-09-10T10:05:59.948527+0000 (oldest deadline 2024-09-10T10:06:26.348781+0000) 2024-09-10T10:06:35.674 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:35 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:35.239+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-09-10T10:06:10.349977+0000 front 2024-09-10T10:06:10.350027+0000 (oldest deadline 2024-09-10T10:06:33.750499+0000) 2024-09-10T10:06:35.939 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:35 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:35.486+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.086443+0000 front 2024-09-10T10:05:37.086361+0000 (oldest deadline 2024-09-10T10:06:02.887395+0000) 2024-09-10T10:06:35.939 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:35 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:35.486+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:42.888131+0000 front 2024-09-10T10:05:42.888100+0000 (oldest deadline 2024-09-10T10:06:07.588327+0000) 2024-09-10T10:06:35.939 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:35 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:35.486+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:50.489601+0000 front 2024-09-10T10:05:50.489747+0000 (oldest deadline 2024-09-10T10:06:15.690110+0000) 2024-09-10T10:06:35.940 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:35 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:35.486+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:06:02.492771+0000 front 2024-09-10T10:06:02.492798+0000 (oldest deadline 2024-09-10T10:06:27.792793+0000) 2024-09-10T10:06:35.940 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:35 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:35.486+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-09-10T10:06:07.793699+0000 front 2024-09-10T10:06:07.793464+0000 (oldest deadline 2024-09-10T10:06:31.893248+0000) 2024-09-10T10:06:36.075 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:35 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:35.763+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:34.146460+0000 front 2024-09-10T10:05:34.146398+0000 (oldest deadline 2024-09-10T10:05:59.446520+0000) 2024-09-10T10:06:36.075 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:35 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:35.763+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:53.949416+0000 front 2024-09-10T10:05:53.949272+0000 (oldest deadline 2024-09-10T10:06:19.749872+0000) 2024-09-10T10:06:36.075 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:35 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:35.763+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:06:00.850855+0000 front 2024-09-10T10:06:00.850768+0000 (oldest deadline 2024-09-10T10:06:25.550818+0000) 2024-09-10T10:06:36.075 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:35 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:35.763+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-09-10T10:06:09.552349+0000 front 2024-09-10T10:06:09.552369+0000 (oldest deadline 2024-09-10T10:06:34.252329+0000) 2024-09-10T10:06:36.320 INFO:journalctl@ceph.osd.7.smithi120.stdout:Sep 10 10:06:36 smithi120 podman[70138]: 2024-09-10 10:06:36.020347791 +0000 UTC m=+5.574112330 container died c7c192ebe569246fe4d0c95a048805486844d89cd9d5db5e78b9d9701c27c99a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-7, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T10:06:36.320 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:36 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:36.194+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.045437+0000 front 2024-09-10T10:05:37.045486+0000 (oldest deadline 2024-09-10T10:05:59.945393+0000) 2024-09-10T10:06:36.321 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:36 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:36.194+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:43.446497+0000 front 2024-09-10T10:05:43.446655+0000 (oldest deadline 2024-09-10T10:06:09.346712+0000) 2024-09-10T10:06:36.321 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:36 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:36.194+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:49.347200+0000 front 2024-09-10T10:05:49.347266+0000 (oldest deadline 2024-09-10T10:06:19.947685+0000) 2024-09-10T10:06:36.321 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:36 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:36.194+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:05:59.948402+0000 front 2024-09-10T10:05:59.948527+0000 (oldest deadline 2024-09-10T10:06:26.348781+0000) 2024-09-10T10:06:36.321 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:36 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:36.194+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-09-10T10:06:10.349977+0000 front 2024-09-10T10:06:10.350027+0000 (oldest deadline 2024-09-10T10:06:33.750499+0000) 2024-09-10T10:06:36.381 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:36 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8[54506]: 2024-09-10T10:06:36.074+0000 7fb09d2dc640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:45.855359+0000 front 2024-09-10T10:05:45.855515+0000 (oldest deadline 2024-09-10T10:06:08.155427+0000) 2024-09-10T10:06:36.381 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:36 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8[54506]: 2024-09-10T10:06:36.074+0000 7fb09d2dc640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:50.456414+0000 front 2024-09-10T10:05:50.456398+0000 (oldest deadline 2024-09-10T10:06:15.156538+0000) 2024-09-10T10:06:36.381 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:36 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8[54506]: 2024-09-10T10:06:36.074+0000 7fb09d2dc640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:06:00.957951+0000 front 2024-09-10T10:06:00.957990+0000 (oldest deadline 2024-09-10T10:06:28.558458+0000) 2024-09-10T10:06:36.382 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:36 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8[54506]: 2024-09-10T10:06:36.074+0000 7fb09d2dc640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-09-10T10:06:08.559135+0000 front 2024-09-10T10:06:08.559098+0000 (oldest deadline 2024-09-10T10:06:34.458944+0000) 2024-09-10T10:06:36.653 INFO:journalctl@ceph.osd.7.smithi120.stdout:Sep 10 10:06:36 smithi120 podman[70138]: 2024-09-10 10:06:36.418062035 +0000 UTC m=+5.971826562 container cleanup c7c192ebe569246fe4d0c95a048805486844d89cd9d5db5e78b9d9701c27c99a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-7, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-10T10:06:36.654 INFO:journalctl@ceph.osd.7.smithi120.stdout:Sep 10 10:06:36 smithi120 bash[70138]: ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-7 2024-09-10T10:06:36.924 INFO:journalctl@ceph.osd.7.smithi120.stdout:Sep 10 10:06:36 smithi120 podman[70157]: 2024-09-10 10:06:36.652609275 +0000 UTC m=+0.621634136 container remove c7c192ebe569246fe4d0c95a048805486844d89cd9d5db5e78b9d9701c27c99a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True) 2024-09-10T10:06:36.939 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:36 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:36.514+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.086443+0000 front 2024-09-10T10:05:37.086361+0000 (oldest deadline 2024-09-10T10:06:02.887395+0000) 2024-09-10T10:06:36.939 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:36 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:36.514+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:42.888131+0000 front 2024-09-10T10:05:42.888100+0000 (oldest deadline 2024-09-10T10:06:07.588327+0000) 2024-09-10T10:06:36.939 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:36 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:36.514+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:50.489601+0000 front 2024-09-10T10:05:50.489747+0000 (oldest deadline 2024-09-10T10:06:15.690110+0000) 2024-09-10T10:06:36.940 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:36 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:36.514+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:06:02.492771+0000 front 2024-09-10T10:06:02.492798+0000 (oldest deadline 2024-09-10T10:06:27.792793+0000) 2024-09-10T10:06:36.940 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:36 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:36.514+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-09-10T10:06:07.793699+0000 front 2024-09-10T10:06:07.793464+0000 (oldest deadline 2024-09-10T10:06:31.893248+0000) 2024-09-10T10:06:37.052 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:36 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:36.747+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:34.146460+0000 front 2024-09-10T10:05:34.146398+0000 (oldest deadline 2024-09-10T10:05:59.446520+0000) 2024-09-10T10:06:37.053 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:36 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:36.747+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:53.949416+0000 front 2024-09-10T10:05:53.949272+0000 (oldest deadline 2024-09-10T10:06:19.749872+0000) 2024-09-10T10:06:37.053 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:36 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:36.747+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:06:00.850855+0000 front 2024-09-10T10:06:00.850768+0000 (oldest deadline 2024-09-10T10:06:25.550818+0000) 2024-09-10T10:06:37.053 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:36 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:36.747+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-09-10T10:06:09.552349+0000 front 2024-09-10T10:06:09.552369+0000 (oldest deadline 2024-09-10T10:06:34.252329+0000) 2024-09-10T10:06:37.264 INFO:journalctl@ceph.osd.7.smithi120.stdout:Sep 10 10:06:36 smithi120 podman[70245]: 2024-09-10 10:06:36.888488222 +0000 UTC m=+0.016043131 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T10:06:37.265 INFO:journalctl@ceph.osd.7.smithi120.stdout:Sep 10 10:06:37 smithi120 podman[70245]: 2024-09-10 10:06:37.011663044 +0000 UTC m=+0.139217944 container create 543228fec0f7a88635cb43762a9785770ceba0535711b585c8aeb7c7e65e77ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-10T10:06:37.265 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:37 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:37.168+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.045437+0000 front 2024-09-10T10:05:37.045486+0000 (oldest deadline 2024-09-10T10:05:59.945393+0000) 2024-09-10T10:06:37.265 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:37 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:37.168+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:43.446497+0000 front 2024-09-10T10:05:43.446655+0000 (oldest deadline 2024-09-10T10:06:09.346712+0000) 2024-09-10T10:06:37.265 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:37 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:37.168+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:49.347200+0000 front 2024-09-10T10:05:49.347266+0000 (oldest deadline 2024-09-10T10:06:19.947685+0000) 2024-09-10T10:06:37.265 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:37 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:37.168+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:05:59.948402+0000 front 2024-09-10T10:05:59.948527+0000 (oldest deadline 2024-09-10T10:06:26.348781+0000) 2024-09-10T10:06:37.265 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:37 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:37.168+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-09-10T10:06:10.349977+0000 front 2024-09-10T10:06:10.350027+0000 (oldest deadline 2024-09-10T10:06:33.750499+0000) 2024-09-10T10:06:37.381 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:37 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8[54506]: 2024-09-10T10:06:37.051+0000 7fb09d2dc640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:45.855359+0000 front 2024-09-10T10:05:45.855515+0000 (oldest deadline 2024-09-10T10:06:08.155427+0000) 2024-09-10T10:06:37.382 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:37 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8[54506]: 2024-09-10T10:06:37.051+0000 7fb09d2dc640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:50.456414+0000 front 2024-09-10T10:05:50.456398+0000 (oldest deadline 2024-09-10T10:06:15.156538+0000) 2024-09-10T10:06:37.382 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:37 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8[54506]: 2024-09-10T10:06:37.051+0000 7fb09d2dc640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:06:00.957951+0000 front 2024-09-10T10:06:00.957990+0000 (oldest deadline 2024-09-10T10:06:28.558458+0000) 2024-09-10T10:06:37.382 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:37 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8[54506]: 2024-09-10T10:06:37.051+0000 7fb09d2dc640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-09-10T10:06:08.559135+0000 front 2024-09-10T10:06:08.559098+0000 (oldest deadline 2024-09-10T10:06:34.458944+0000) 2024-09-10T10:06:37.674 INFO:journalctl@ceph.osd.7.smithi120.stdout:Sep 10 10:06:37 smithi120 podman[70245]: 2024-09-10 10:06:37.26416571 +0000 UTC m=+0.391720610 container init 543228fec0f7a88635cb43762a9785770ceba0535711b585c8aeb7c7e65e77ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-10T10:06:37.674 INFO:journalctl@ceph.osd.7.smithi120.stdout:Sep 10 10:06:37 smithi120 podman[70245]: 2024-09-10 10:06:37.267791655 +0000 UTC m=+0.395346558 container start 543228fec0f7a88635cb43762a9785770ceba0535711b585c8aeb7c7e65e77ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-10T10:06:37.674 INFO:journalctl@ceph.osd.7.smithi120.stdout:Sep 10 10:06:37 smithi120 podman[70245]: 2024-09-10 10:06:37.406101709 +0000 UTC m=+0.533656624 container attach 543228fec0f7a88635cb43762a9785770ceba0535711b585c8aeb7c7e65e77ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-10T10:06:37.674 INFO:journalctl@ceph.osd.7.smithi120.stdout:Sep 10 10:06:37 smithi120 podman[70245]: 2024-09-10 10:06:37.458143817 +0000 UTC m=+0.585698718 container died 543228fec0f7a88635cb43762a9785770ceba0535711b585c8aeb7c7e65e77ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-10T10:06:37.939 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:37 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:37.530+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.086443+0000 front 2024-09-10T10:05:37.086361+0000 (oldest deadline 2024-09-10T10:06:02.887395+0000) 2024-09-10T10:06:37.940 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:37 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:37.530+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:42.888131+0000 front 2024-09-10T10:05:42.888100+0000 (oldest deadline 2024-09-10T10:06:07.588327+0000) 2024-09-10T10:06:37.940 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:37 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:37.530+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:50.489601+0000 front 2024-09-10T10:05:50.489747+0000 (oldest deadline 2024-09-10T10:06:15.690110+0000) 2024-09-10T10:06:37.940 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:37 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:37.530+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:06:02.492771+0000 front 2024-09-10T10:06:02.492798+0000 (oldest deadline 2024-09-10T10:06:27.792793+0000) 2024-09-10T10:06:37.940 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:37 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:37.530+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-09-10T10:06:07.793699+0000 front 2024-09-10T10:06:07.793464+0000 (oldest deadline 2024-09-10T10:06:31.893248+0000) 2024-09-10T10:06:38.056 DEBUG:teuthology.orchestra.run.smithi120:> sudo pkill -f 'journalctl -f -n 0 -u ceph-14a55226-6f59-11ef-bcea-c7b262605968@osd.7.service' 2024-09-10T10:06:38.073 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:37 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:37.726+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:34.146460+0000 front 2024-09-10T10:05:34.146398+0000 (oldest deadline 2024-09-10T10:05:59.446520+0000) 2024-09-10T10:06:38.073 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:37 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:37.726+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:53.949416+0000 front 2024-09-10T10:05:53.949272+0000 (oldest deadline 2024-09-10T10:06:19.749872+0000) 2024-09-10T10:06:38.073 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:37 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:37.726+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:06:00.850855+0000 front 2024-09-10T10:06:00.850768+0000 (oldest deadline 2024-09-10T10:06:25.550818+0000) 2024-09-10T10:06:38.073 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:37 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:37.726+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-09-10T10:06:09.552349+0000 front 2024-09-10T10:06:09.552369+0000 (oldest deadline 2024-09-10T10:06:34.252329+0000) 2024-09-10T10:06:38.098 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:06:38.098 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-10T10:06:38.099 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-09-10T10:06:38.099 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-14a55226-6f59-11ef-bcea-c7b262605968@osd.8 2024-09-10T10:06:38.309 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:38 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:38.148+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.045437+0000 front 2024-09-10T10:05:37.045486+0000 (oldest deadline 2024-09-10T10:05:59.945393+0000) 2024-09-10T10:06:38.309 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:38 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:38.148+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:43.446497+0000 front 2024-09-10T10:05:43.446655+0000 (oldest deadline 2024-09-10T10:06:09.346712+0000) 2024-09-10T10:06:38.309 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:38 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:38.148+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:49.347200+0000 front 2024-09-10T10:05:49.347266+0000 (oldest deadline 2024-09-10T10:06:19.947685+0000) 2024-09-10T10:06:38.309 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:38 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:38.148+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:05:59.948402+0000 front 2024-09-10T10:05:59.948527+0000 (oldest deadline 2024-09-10T10:06:26.348781+0000) 2024-09-10T10:06:38.309 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:38 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:38.148+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-09-10T10:06:10.349977+0000 front 2024-09-10T10:06:10.350027+0000 (oldest deadline 2024-09-10T10:06:33.750499+0000) 2024-09-10T10:06:38.381 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:38 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8[54506]: 2024-09-10T10:06:38.072+0000 7fb09d2dc640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:45.855359+0000 front 2024-09-10T10:05:45.855515+0000 (oldest deadline 2024-09-10T10:06:08.155427+0000) 2024-09-10T10:06:38.382 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:38 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8[54506]: 2024-09-10T10:06:38.072+0000 7fb09d2dc640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:50.456414+0000 front 2024-09-10T10:05:50.456398+0000 (oldest deadline 2024-09-10T10:06:15.156538+0000) 2024-09-10T10:06:38.382 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:38 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8[54506]: 2024-09-10T10:06:38.072+0000 7fb09d2dc640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:06:00.957951+0000 front 2024-09-10T10:06:00.957990+0000 (oldest deadline 2024-09-10T10:06:28.558458+0000) 2024-09-10T10:06:38.382 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:38 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8[54506]: 2024-09-10T10:06:38.072+0000 7fb09d2dc640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-09-10T10:06:08.559135+0000 front 2024-09-10T10:06:08.559098+0000 (oldest deadline 2024-09-10T10:06:34.458944+0000) 2024-09-10T10:06:38.382 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:38 smithi134 systemd[1]: Stopping Ceph osd.8 for 14a55226-6f59-11ef-bcea-c7b262605968... 2024-09-10T10:06:38.772 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:38 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8[54506]: 2024-09-10T10:06:38.430+0000 7fb0a4cce640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:06:38.773 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:38 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8[54506]: 2024-09-10T10:06:38.430+0000 7fb0a4cce640 -1 osd.8 100 *** Got signal Terminated *** 2024-09-10T10:06:38.773 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:38 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8[54506]: 2024-09-10T10:06:38.430+0000 7fb0a4cce640 -1 osd.8 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T10:06:38.939 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:38 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:38.489+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.086443+0000 front 2024-09-10T10:05:37.086361+0000 (oldest deadline 2024-09-10T10:06:02.887395+0000) 2024-09-10T10:06:38.939 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:38 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:38.489+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:42.888131+0000 front 2024-09-10T10:05:42.888100+0000 (oldest deadline 2024-09-10T10:06:07.588327+0000) 2024-09-10T10:06:38.939 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:38 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:38.489+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:50.489601+0000 front 2024-09-10T10:05:50.489747+0000 (oldest deadline 2024-09-10T10:06:15.690110+0000) 2024-09-10T10:06:38.940 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:38 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:38.489+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:06:02.492771+0000 front 2024-09-10T10:06:02.492798+0000 (oldest deadline 2024-09-10T10:06:27.792793+0000) 2024-09-10T10:06:38.940 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:38 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:38.489+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-09-10T10:06:07.793699+0000 front 2024-09-10T10:06:07.793464+0000 (oldest deadline 2024-09-10T10:06:31.893248+0000) 2024-09-10T10:06:39.097 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:38 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:38.771+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:34.146460+0000 front 2024-09-10T10:05:34.146398+0000 (oldest deadline 2024-09-10T10:05:59.446520+0000) 2024-09-10T10:06:39.098 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:38 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:38.771+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:53.949416+0000 front 2024-09-10T10:05:53.949272+0000 (oldest deadline 2024-09-10T10:06:19.749872+0000) 2024-09-10T10:06:39.098 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:38 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:38.771+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:06:00.850855+0000 front 2024-09-10T10:06:00.850768+0000 (oldest deadline 2024-09-10T10:06:25.550818+0000) 2024-09-10T10:06:39.098 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:38 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:38.771+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-09-10T10:06:09.552349+0000 front 2024-09-10T10:06:09.552369+0000 (oldest deadline 2024-09-10T10:06:34.252329+0000) 2024-09-10T10:06:39.381 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:39 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8[54506]: 2024-09-10T10:06:39.097+0000 7fb09d2dc640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:45.855359+0000 front 2024-09-10T10:05:45.855515+0000 (oldest deadline 2024-09-10T10:06:08.155427+0000) 2024-09-10T10:06:39.381 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:39 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8[54506]: 2024-09-10T10:06:39.097+0000 7fb09d2dc640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:50.456414+0000 front 2024-09-10T10:05:50.456398+0000 (oldest deadline 2024-09-10T10:06:15.156538+0000) 2024-09-10T10:06:39.381 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:39 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8[54506]: 2024-09-10T10:06:39.097+0000 7fb09d2dc640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:06:00.957951+0000 front 2024-09-10T10:06:00.957990+0000 (oldest deadline 2024-09-10T10:06:28.558458+0000) 2024-09-10T10:06:39.381 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:39 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8[54506]: 2024-09-10T10:06:39.097+0000 7fb09d2dc640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-09-10T10:06:08.559135+0000 front 2024-09-10T10:06:08.559098+0000 (oldest deadline 2024-09-10T10:06:34.458944+0000) 2024-09-10T10:06:39.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:39 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:39.156+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.045437+0000 front 2024-09-10T10:05:37.045486+0000 (oldest deadline 2024-09-10T10:05:59.945393+0000) 2024-09-10T10:06:39.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:39 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:39.156+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:43.446497+0000 front 2024-09-10T10:05:43.446655+0000 (oldest deadline 2024-09-10T10:06:09.346712+0000) 2024-09-10T10:06:39.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:39 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:39.156+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:49.347200+0000 front 2024-09-10T10:05:49.347266+0000 (oldest deadline 2024-09-10T10:06:19.947685+0000) 2024-09-10T10:06:39.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:39 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:39.156+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:05:59.948402+0000 front 2024-09-10T10:05:59.948527+0000 (oldest deadline 2024-09-10T10:06:26.348781+0000) 2024-09-10T10:06:39.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:39 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:39.156+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-09-10T10:06:10.349977+0000 front 2024-09-10T10:06:10.350027+0000 (oldest deadline 2024-09-10T10:06:33.750499+0000) 2024-09-10T10:06:39.939 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:39 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:39.498+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.086443+0000 front 2024-09-10T10:05:37.086361+0000 (oldest deadline 2024-09-10T10:06:02.887395+0000) 2024-09-10T10:06:39.939 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:39 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:39.498+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:42.888131+0000 front 2024-09-10T10:05:42.888100+0000 (oldest deadline 2024-09-10T10:06:07.588327+0000) 2024-09-10T10:06:39.939 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:39 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:39.498+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:50.489601+0000 front 2024-09-10T10:05:50.489747+0000 (oldest deadline 2024-09-10T10:06:15.690110+0000) 2024-09-10T10:06:39.939 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:39 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:39.498+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:06:02.492771+0000 front 2024-09-10T10:06:02.492798+0000 (oldest deadline 2024-09-10T10:06:27.792793+0000) 2024-09-10T10:06:39.939 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:39 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:39.498+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-09-10T10:06:07.793699+0000 front 2024-09-10T10:06:07.793464+0000 (oldest deadline 2024-09-10T10:06:31.893248+0000) 2024-09-10T10:06:40.131 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:39 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:39.816+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:34.146460+0000 front 2024-09-10T10:05:34.146398+0000 (oldest deadline 2024-09-10T10:05:59.446520+0000) 2024-09-10T10:06:40.131 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:39 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:39.816+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:53.949416+0000 front 2024-09-10T10:05:53.949272+0000 (oldest deadline 2024-09-10T10:06:19.749872+0000) 2024-09-10T10:06:40.131 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:39 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:39.816+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:06:00.850855+0000 front 2024-09-10T10:06:00.850768+0000 (oldest deadline 2024-09-10T10:06:25.550818+0000) 2024-09-10T10:06:40.131 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:39 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:39.816+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-09-10T10:06:09.552349+0000 front 2024-09-10T10:06:09.552369+0000 (oldest deadline 2024-09-10T10:06:34.252329+0000) 2024-09-10T10:06:40.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:40 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:40.136+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.045437+0000 front 2024-09-10T10:05:37.045486+0000 (oldest deadline 2024-09-10T10:05:59.945393+0000) 2024-09-10T10:06:40.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:40 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:40.136+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:43.446497+0000 front 2024-09-10T10:05:43.446655+0000 (oldest deadline 2024-09-10T10:06:09.346712+0000) 2024-09-10T10:06:40.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:40 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:40.136+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:49.347200+0000 front 2024-09-10T10:05:49.347266+0000 (oldest deadline 2024-09-10T10:06:19.947685+0000) 2024-09-10T10:06:40.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:40 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:40.136+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:05:59.948402+0000 front 2024-09-10T10:05:59.948527+0000 (oldest deadline 2024-09-10T10:06:26.348781+0000) 2024-09-10T10:06:40.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:40 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:40.136+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-09-10T10:06:10.349977+0000 front 2024-09-10T10:06:10.350027+0000 (oldest deadline 2024-09-10T10:06:33.750499+0000) 2024-09-10T10:06:40.631 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:40 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8[54506]: 2024-09-10T10:06:40.136+0000 7fb09d2dc640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:45.855359+0000 front 2024-09-10T10:05:45.855515+0000 (oldest deadline 2024-09-10T10:06:08.155427+0000) 2024-09-10T10:06:40.631 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:40 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8[54506]: 2024-09-10T10:06:40.136+0000 7fb09d2dc640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:50.456414+0000 front 2024-09-10T10:05:50.456398+0000 (oldest deadline 2024-09-10T10:06:15.156538+0000) 2024-09-10T10:06:40.631 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:40 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8[54506]: 2024-09-10T10:06:40.136+0000 7fb09d2dc640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:06:00.957951+0000 front 2024-09-10T10:06:00.957990+0000 (oldest deadline 2024-09-10T10:06:28.558458+0000) 2024-09-10T10:06:40.632 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:40 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8[54506]: 2024-09-10T10:06:40.136+0000 7fb09d2dc640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-09-10T10:06:08.559135+0000 front 2024-09-10T10:06:08.559098+0000 (oldest deadline 2024-09-10T10:06:34.458944+0000) 2024-09-10T10:06:40.632 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:40 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8[54506]: 2024-09-10T10:06:40.136+0000 7fb09d2dc640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-10T10:06:14.459442+0000 front 2024-09-10T10:06:14.459397+0000 (oldest deadline 2024-09-10T10:06:39.659951+0000) 2024-09-10T10:06:40.840 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:40 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:40.477+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.086443+0000 front 2024-09-10T10:05:37.086361+0000 (oldest deadline 2024-09-10T10:06:02.887395+0000) 2024-09-10T10:06:40.841 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:40 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:40.477+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:42.888131+0000 front 2024-09-10T10:05:42.888100+0000 (oldest deadline 2024-09-10T10:06:07.588327+0000) 2024-09-10T10:06:40.841 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:40 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:40.477+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:50.489601+0000 front 2024-09-10T10:05:50.489747+0000 (oldest deadline 2024-09-10T10:06:15.690110+0000) 2024-09-10T10:06:40.841 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:40 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:40.477+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:06:02.492771+0000 front 2024-09-10T10:06:02.492798+0000 (oldest deadline 2024-09-10T10:06:27.792793+0000) 2024-09-10T10:06:40.841 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:40 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:40.477+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-09-10T10:06:07.793699+0000 front 2024-09-10T10:06:07.793464+0000 (oldest deadline 2024-09-10T10:06:31.893248+0000) 2024-09-10T10:06:41.109 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:40 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:40.840+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:34.146460+0000 front 2024-09-10T10:05:34.146398+0000 (oldest deadline 2024-09-10T10:05:59.446520+0000) 2024-09-10T10:06:41.109 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:40 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:40.840+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:53.949416+0000 front 2024-09-10T10:05:53.949272+0000 (oldest deadline 2024-09-10T10:06:19.749872+0000) 2024-09-10T10:06:41.109 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:40 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:40.840+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:06:00.850855+0000 front 2024-09-10T10:06:00.850768+0000 (oldest deadline 2024-09-10T10:06:25.550818+0000) 2024-09-10T10:06:41.109 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:40 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:40.840+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-09-10T10:06:09.552349+0000 front 2024-09-10T10:06:09.552369+0000 (oldest deadline 2024-09-10T10:06:34.252329+0000) 2024-09-10T10:06:41.109 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:40 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:40.840+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-10T10:06:14.252778+0000 front 2024-09-10T10:06:14.252825+0000 (oldest deadline 2024-09-10T10:06:40.152981+0000) 2024-09-10T10:06:41.381 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:41 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8[54506]: 2024-09-10T10:06:41.108+0000 7fb09d2dc640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:45.855359+0000 front 2024-09-10T10:05:45.855515+0000 (oldest deadline 2024-09-10T10:06:08.155427+0000) 2024-09-10T10:06:41.381 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:41 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8[54506]: 2024-09-10T10:06:41.108+0000 7fb09d2dc640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:50.456414+0000 front 2024-09-10T10:05:50.456398+0000 (oldest deadline 2024-09-10T10:06:15.156538+0000) 2024-09-10T10:06:41.381 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:41 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8[54506]: 2024-09-10T10:06:41.108+0000 7fb09d2dc640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:06:00.957951+0000 front 2024-09-10T10:06:00.957990+0000 (oldest deadline 2024-09-10T10:06:28.558458+0000) 2024-09-10T10:06:41.381 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:41 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8[54506]: 2024-09-10T10:06:41.108+0000 7fb09d2dc640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-09-10T10:06:08.559135+0000 front 2024-09-10T10:06:08.559098+0000 (oldest deadline 2024-09-10T10:06:34.458944+0000) 2024-09-10T10:06:41.382 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:41 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8[54506]: 2024-09-10T10:06:41.108+0000 7fb09d2dc640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-10T10:06:14.459442+0000 front 2024-09-10T10:06:14.459397+0000 (oldest deadline 2024-09-10T10:06:39.659951+0000) 2024-09-10T10:06:41.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:41 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:41.114+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.045437+0000 front 2024-09-10T10:05:37.045486+0000 (oldest deadline 2024-09-10T10:05:59.945393+0000) 2024-09-10T10:06:41.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:41 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:41.114+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:43.446497+0000 front 2024-09-10T10:05:43.446655+0000 (oldest deadline 2024-09-10T10:06:09.346712+0000) 2024-09-10T10:06:41.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:41 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:41.114+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:49.347200+0000 front 2024-09-10T10:05:49.347266+0000 (oldest deadline 2024-09-10T10:06:19.947685+0000) 2024-09-10T10:06:41.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:41 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:41.114+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:05:59.948402+0000 front 2024-09-10T10:05:59.948527+0000 (oldest deadline 2024-09-10T10:06:26.348781+0000) 2024-09-10T10:06:41.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:41 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:41.114+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-09-10T10:06:10.349977+0000 front 2024-09-10T10:06:10.350027+0000 (oldest deadline 2024-09-10T10:06:33.750499+0000) 2024-09-10T10:06:41.689 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:41 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:41.430+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.086443+0000 front 2024-09-10T10:05:37.086361+0000 (oldest deadline 2024-09-10T10:06:02.887395+0000) 2024-09-10T10:06:41.690 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:41 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:41.430+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:42.888131+0000 front 2024-09-10T10:05:42.888100+0000 (oldest deadline 2024-09-10T10:06:07.588327+0000) 2024-09-10T10:06:41.690 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:41 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:41.430+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:50.489601+0000 front 2024-09-10T10:05:50.489747+0000 (oldest deadline 2024-09-10T10:06:15.690110+0000) 2024-09-10T10:06:41.690 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:41 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:41.430+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:06:02.492771+0000 front 2024-09-10T10:06:02.492798+0000 (oldest deadline 2024-09-10T10:06:27.792793+0000) 2024-09-10T10:06:41.690 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:41 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:41.430+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-09-10T10:06:07.793699+0000 front 2024-09-10T10:06:07.793464+0000 (oldest deadline 2024-09-10T10:06:31.893248+0000) 2024-09-10T10:06:42.131 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:41 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:41.833+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:34.146460+0000 front 2024-09-10T10:05:34.146398+0000 (oldest deadline 2024-09-10T10:05:59.446520+0000) 2024-09-10T10:06:42.131 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:41 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:41.833+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:53.949416+0000 front 2024-09-10T10:05:53.949272+0000 (oldest deadline 2024-09-10T10:06:19.749872+0000) 2024-09-10T10:06:42.131 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:41 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:41.833+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:06:00.850855+0000 front 2024-09-10T10:06:00.850768+0000 (oldest deadline 2024-09-10T10:06:25.550818+0000) 2024-09-10T10:06:42.131 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:41 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:41.833+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-09-10T10:06:09.552349+0000 front 2024-09-10T10:06:09.552369+0000 (oldest deadline 2024-09-10T10:06:34.252329+0000) 2024-09-10T10:06:42.132 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:41 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:41.833+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-10T10:06:14.252778+0000 front 2024-09-10T10:06:14.252825+0000 (oldest deadline 2024-09-10T10:06:40.152981+0000) 2024-09-10T10:06:42.132 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:42 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8[54506]: 2024-09-10T10:06:42.059+0000 7fb09d2dc640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:45.855359+0000 front 2024-09-10T10:05:45.855515+0000 (oldest deadline 2024-09-10T10:06:08.155427+0000) 2024-09-10T10:06:42.132 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:42 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8[54506]: 2024-09-10T10:06:42.059+0000 7fb09d2dc640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:50.456414+0000 front 2024-09-10T10:05:50.456398+0000 (oldest deadline 2024-09-10T10:06:15.156538+0000) 2024-09-10T10:06:42.132 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:42 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8[54506]: 2024-09-10T10:06:42.059+0000 7fb09d2dc640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:06:00.957951+0000 front 2024-09-10T10:06:00.957990+0000 (oldest deadline 2024-09-10T10:06:28.558458+0000) 2024-09-10T10:06:42.132 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:42 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8[54506]: 2024-09-10T10:06:42.059+0000 7fb09d2dc640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-09-10T10:06:08.559135+0000 front 2024-09-10T10:06:08.559098+0000 (oldest deadline 2024-09-10T10:06:34.458944+0000) 2024-09-10T10:06:42.132 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:42 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8[54506]: 2024-09-10T10:06:42.059+0000 7fb09d2dc640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-10T10:06:14.459442+0000 front 2024-09-10T10:06:14.459397+0000 (oldest deadline 2024-09-10T10:06:39.659951+0000) 2024-09-10T10:06:42.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:42 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:42.070+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.045437+0000 front 2024-09-10T10:05:37.045486+0000 (oldest deadline 2024-09-10T10:05:59.945393+0000) 2024-09-10T10:06:42.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:42 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:42.070+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:43.446497+0000 front 2024-09-10T10:05:43.446655+0000 (oldest deadline 2024-09-10T10:06:09.346712+0000) 2024-09-10T10:06:42.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:42 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:42.070+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:49.347200+0000 front 2024-09-10T10:05:49.347266+0000 (oldest deadline 2024-09-10T10:06:19.947685+0000) 2024-09-10T10:06:42.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:42 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:42.070+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:05:59.948402+0000 front 2024-09-10T10:05:59.948527+0000 (oldest deadline 2024-09-10T10:06:26.348781+0000) 2024-09-10T10:06:42.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:42 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:42.070+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-09-10T10:06:10.349977+0000 front 2024-09-10T10:06:10.350027+0000 (oldest deadline 2024-09-10T10:06:33.750499+0000) 2024-09-10T10:06:42.689 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:42 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:42.450+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.086443+0000 front 2024-09-10T10:05:37.086361+0000 (oldest deadline 2024-09-10T10:06:02.887395+0000) 2024-09-10T10:06:42.689 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:42 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:42.450+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:42.888131+0000 front 2024-09-10T10:05:42.888100+0000 (oldest deadline 2024-09-10T10:06:07.588327+0000) 2024-09-10T10:06:42.689 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:42 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:42.450+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:50.489601+0000 front 2024-09-10T10:05:50.489747+0000 (oldest deadline 2024-09-10T10:06:15.690110+0000) 2024-09-10T10:06:42.689 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:42 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:42.450+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:06:02.492771+0000 front 2024-09-10T10:06:02.492798+0000 (oldest deadline 2024-09-10T10:06:27.792793+0000) 2024-09-10T10:06:42.690 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:42 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:42.450+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-09-10T10:06:07.793699+0000 front 2024-09-10T10:06:07.793464+0000 (oldest deadline 2024-09-10T10:06:31.893248+0000) 2024-09-10T10:06:43.088 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:42 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:42.828+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:34.146460+0000 front 2024-09-10T10:05:34.146398+0000 (oldest deadline 2024-09-10T10:05:59.446520+0000) 2024-09-10T10:06:43.088 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:42 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:42.828+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:53.949416+0000 front 2024-09-10T10:05:53.949272+0000 (oldest deadline 2024-09-10T10:06:19.749872+0000) 2024-09-10T10:06:43.088 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:42 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:42.828+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:06:00.850855+0000 front 2024-09-10T10:06:00.850768+0000 (oldest deadline 2024-09-10T10:06:25.550818+0000) 2024-09-10T10:06:43.089 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:42 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:42.828+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-09-10T10:06:09.552349+0000 front 2024-09-10T10:06:09.552369+0000 (oldest deadline 2024-09-10T10:06:34.252329+0000) 2024-09-10T10:06:43.089 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:42 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:42.828+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-10T10:06:14.252778+0000 front 2024-09-10T10:06:14.252825+0000 (oldest deadline 2024-09-10T10:06:40.152981+0000) 2024-09-10T10:06:43.381 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:43 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8[54506]: 2024-09-10T10:06:43.086+0000 7fb09d2dc640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:45.855359+0000 front 2024-09-10T10:05:45.855515+0000 (oldest deadline 2024-09-10T10:06:08.155427+0000) 2024-09-10T10:06:43.381 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:43 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8[54506]: 2024-09-10T10:06:43.086+0000 7fb09d2dc640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:50.456414+0000 front 2024-09-10T10:05:50.456398+0000 (oldest deadline 2024-09-10T10:06:15.156538+0000) 2024-09-10T10:06:43.381 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:43 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8[54506]: 2024-09-10T10:06:43.086+0000 7fb09d2dc640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:06:00.957951+0000 front 2024-09-10T10:06:00.957990+0000 (oldest deadline 2024-09-10T10:06:28.558458+0000) 2024-09-10T10:06:43.382 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:43 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8[54506]: 2024-09-10T10:06:43.086+0000 7fb09d2dc640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-09-10T10:06:08.559135+0000 front 2024-09-10T10:06:08.559098+0000 (oldest deadline 2024-09-10T10:06:34.458944+0000) 2024-09-10T10:06:43.382 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:43 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8[54506]: 2024-09-10T10:06:43.086+0000 7fb09d2dc640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-10T10:06:14.459442+0000 front 2024-09-10T10:06:14.459397+0000 (oldest deadline 2024-09-10T10:06:39.659951+0000) 2024-09-10T10:06:43.423 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:43 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:43.024+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.045437+0000 front 2024-09-10T10:05:37.045486+0000 (oldest deadline 2024-09-10T10:05:59.945393+0000) 2024-09-10T10:06:43.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:43 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:43.024+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:43.446497+0000 front 2024-09-10T10:05:43.446655+0000 (oldest deadline 2024-09-10T10:06:09.346712+0000) 2024-09-10T10:06:43.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:43 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:43.024+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:49.347200+0000 front 2024-09-10T10:05:49.347266+0000 (oldest deadline 2024-09-10T10:06:19.947685+0000) 2024-09-10T10:06:43.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:43 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:43.024+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:05:59.948402+0000 front 2024-09-10T10:05:59.948527+0000 (oldest deadline 2024-09-10T10:06:26.348781+0000) 2024-09-10T10:06:43.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:43 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:43.024+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-09-10T10:06:10.349977+0000 front 2024-09-10T10:06:10.350027+0000 (oldest deadline 2024-09-10T10:06:33.750499+0000) 2024-09-10T10:06:43.689 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:43 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:43.430+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.086443+0000 front 2024-09-10T10:05:37.086361+0000 (oldest deadline 2024-09-10T10:06:02.887395+0000) 2024-09-10T10:06:43.689 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:43 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:43.430+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:42.888131+0000 front 2024-09-10T10:05:42.888100+0000 (oldest deadline 2024-09-10T10:06:07.588327+0000) 2024-09-10T10:06:43.690 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:43 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:43.430+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:50.489601+0000 front 2024-09-10T10:05:50.489747+0000 (oldest deadline 2024-09-10T10:06:15.690110+0000) 2024-09-10T10:06:43.690 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:43 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:43.430+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:06:02.492771+0000 front 2024-09-10T10:06:02.492798+0000 (oldest deadline 2024-09-10T10:06:27.792793+0000) 2024-09-10T10:06:43.690 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:43 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:43.430+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-09-10T10:06:07.793699+0000 front 2024-09-10T10:06:07.793464+0000 (oldest deadline 2024-09-10T10:06:31.893248+0000) 2024-09-10T10:06:43.690 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:43 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:43.430+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-10T10:06:16.594588+0000 front 2024-09-10T10:06:16.594617+0000 (oldest deadline 2024-09-10T10:06:42.494249+0000) 2024-09-10T10:06:44.082 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:43 smithi134 podman[72046]: 2024-09-10 10:06:43.79493305 +0000 UTC m=+5.500525078 container died 86c47bc8e5767c94704abb71ed8e98e8c9e5a0e745e404292adbcf88836b7cf1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-10T10:06:44.083 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:43 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:43.844+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:34.146460+0000 front 2024-09-10T10:05:34.146398+0000 (oldest deadline 2024-09-10T10:05:59.446520+0000) 2024-09-10T10:06:44.083 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:43 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:43.844+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:53.949416+0000 front 2024-09-10T10:05:53.949272+0000 (oldest deadline 2024-09-10T10:06:19.749872+0000) 2024-09-10T10:06:44.083 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:43 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:43.844+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:06:00.850855+0000 front 2024-09-10T10:06:00.850768+0000 (oldest deadline 2024-09-10T10:06:25.550818+0000) 2024-09-10T10:06:44.083 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:43 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:43.844+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-09-10T10:06:09.552349+0000 front 2024-09-10T10:06:09.552369+0000 (oldest deadline 2024-09-10T10:06:34.252329+0000) 2024-09-10T10:06:44.083 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:43 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:43.844+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-10T10:06:14.252778+0000 front 2024-09-10T10:06:14.252825+0000 (oldest deadline 2024-09-10T10:06:40.152981+0000) 2024-09-10T10:06:44.340 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:44 smithi134 podman[72046]: 2024-09-10 10:06:44.172053168 +0000 UTC m=+5.877645193 container cleanup 86c47bc8e5767c94704abb71ed8e98e8c9e5a0e745e404292adbcf88836b7cf1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-10T10:06:44.341 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:44 smithi134 bash[72046]: ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8 2024-09-10T10:06:44.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:44 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:44.005+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.045437+0000 front 2024-09-10T10:05:37.045486+0000 (oldest deadline 2024-09-10T10:05:59.945393+0000) 2024-09-10T10:06:44.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:44 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:44.005+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:43.446497+0000 front 2024-09-10T10:05:43.446655+0000 (oldest deadline 2024-09-10T10:06:09.346712+0000) 2024-09-10T10:06:44.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:44 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:44.005+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:49.347200+0000 front 2024-09-10T10:05:49.347266+0000 (oldest deadline 2024-09-10T10:06:19.947685+0000) 2024-09-10T10:06:44.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:44 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:44.005+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:05:59.948402+0000 front 2024-09-10T10:05:59.948527+0000 (oldest deadline 2024-09-10T10:06:26.348781+0000) 2024-09-10T10:06:44.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:44 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:44.005+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-09-10T10:06:10.349977+0000 front 2024-09-10T10:06:10.350027+0000 (oldest deadline 2024-09-10T10:06:33.750499+0000) 2024-09-10T10:06:44.631 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:44 smithi134 podman[72062]: 2024-09-10 10:06:44.339967227 +0000 UTC m=+0.544673529 container remove 86c47bc8e5767c94704abb71ed8e98e8c9e5a0e745e404292adbcf88836b7cf1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-10T10:06:44.689 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:44 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:44.427+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.086443+0000 front 2024-09-10T10:05:37.086361+0000 (oldest deadline 2024-09-10T10:06:02.887395+0000) 2024-09-10T10:06:44.689 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:44 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:44.427+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:42.888131+0000 front 2024-09-10T10:05:42.888100+0000 (oldest deadline 2024-09-10T10:06:07.588327+0000) 2024-09-10T10:06:44.689 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:44 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:44.427+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:50.489601+0000 front 2024-09-10T10:05:50.489747+0000 (oldest deadline 2024-09-10T10:06:15.690110+0000) 2024-09-10T10:06:44.689 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:44 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:44.427+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:06:02.492771+0000 front 2024-09-10T10:06:02.492798+0000 (oldest deadline 2024-09-10T10:06:27.792793+0000) 2024-09-10T10:06:44.689 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:44 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:44.427+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-09-10T10:06:07.793699+0000 front 2024-09-10T10:06:07.793464+0000 (oldest deadline 2024-09-10T10:06:31.893248+0000) 2024-09-10T10:06:44.690 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:44 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:44.427+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-10T10:06:16.594588+0000 front 2024-09-10T10:06:16.594617+0000 (oldest deadline 2024-09-10T10:06:42.494249+0000) 2024-09-10T10:06:45.002 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:44 smithi134 podman[72147]: 2024-09-10 10:06:44.640968457 +0000 UTC m=+0.016243003 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T10:06:45.003 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:44 smithi134 podman[72147]: 2024-09-10 10:06:44.757716674 +0000 UTC m=+0.132991210 container create 6da06cabc4f1f447bc830e82ad2f30d61cb3ea71cd12e62d9ff169c90f8e128b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8-deactivate, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-10T10:06:45.003 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:45 smithi134 podman[72147]: 2024-09-10 10:06:45.001959019 +0000 UTC m=+0.377233558 container init 6da06cabc4f1f447bc830e82ad2f30d61cb3ea71cd12e62d9ff169c90f8e128b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731) 2024-09-10T10:06:45.003 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:44 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:44.827+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:34.146460+0000 front 2024-09-10T10:05:34.146398+0000 (oldest deadline 2024-09-10T10:05:59.446520+0000) 2024-09-10T10:06:45.003 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:44 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:44.827+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:53.949416+0000 front 2024-09-10T10:05:53.949272+0000 (oldest deadline 2024-09-10T10:06:19.749872+0000) 2024-09-10T10:06:45.003 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:44 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:44.827+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:06:00.850855+0000 front 2024-09-10T10:06:00.850768+0000 (oldest deadline 2024-09-10T10:06:25.550818+0000) 2024-09-10T10:06:45.003 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:44 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:44.827+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-09-10T10:06:09.552349+0000 front 2024-09-10T10:06:09.552369+0000 (oldest deadline 2024-09-10T10:06:34.252329+0000) 2024-09-10T10:06:45.003 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:44 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:44.827+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-10T10:06:14.252778+0000 front 2024-09-10T10:06:14.252825+0000 (oldest deadline 2024-09-10T10:06:40.152981+0000) 2024-09-10T10:06:45.381 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:45 smithi134 podman[72147]: 2024-09-10 10:06:45.005389348 +0000 UTC m=+0.380663892 container start 6da06cabc4f1f447bc830e82ad2f30d61cb3ea71cd12e62d9ff169c90f8e128b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-10T10:06:45.381 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:45 smithi134 podman[72147]: 2024-09-10 10:06:45.14368622 +0000 UTC m=+0.518960770 container attach 6da06cabc4f1f447bc830e82ad2f30d61cb3ea71cd12e62d9ff169c90f8e128b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8-deactivate, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T10:06:45.382 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 10 10:06:45 smithi134 podman[72147]: 2024-09-10 10:06:45.191229804 +0000 UTC m=+0.566504342 container died 6da06cabc4f1f447bc830e82ad2f30d61cb3ea71cd12e62d9ff169c90f8e128b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-10T10:06:45.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:45 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:45.027+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.045437+0000 front 2024-09-10T10:05:37.045486+0000 (oldest deadline 2024-09-10T10:05:59.945393+0000) 2024-09-10T10:06:45.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:45 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:45.027+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:43.446497+0000 front 2024-09-10T10:05:43.446655+0000 (oldest deadline 2024-09-10T10:06:09.346712+0000) 2024-09-10T10:06:45.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:45 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:45.027+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:49.347200+0000 front 2024-09-10T10:05:49.347266+0000 (oldest deadline 2024-09-10T10:06:19.947685+0000) 2024-09-10T10:06:45.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:45 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:45.027+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:05:59.948402+0000 front 2024-09-10T10:05:59.948527+0000 (oldest deadline 2024-09-10T10:06:26.348781+0000) 2024-09-10T10:06:45.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:45 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:45.027+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-09-10T10:06:10.349977+0000 front 2024-09-10T10:06:10.350027+0000 (oldest deadline 2024-09-10T10:06:33.750499+0000) 2024-09-10T10:06:45.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:45 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:45.027+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-10T10:06:18.352105+0000 front 2024-09-10T10:06:18.352025+0000 (oldest deadline 2024-09-10T10:06:44.251983+0000) 2024-09-10T10:06:45.689 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:45 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:45.430+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.086443+0000 front 2024-09-10T10:05:37.086361+0000 (oldest deadline 2024-09-10T10:06:02.887395+0000) 2024-09-10T10:06:45.689 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:45 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:45.430+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:42.888131+0000 front 2024-09-10T10:05:42.888100+0000 (oldest deadline 2024-09-10T10:06:07.588327+0000) 2024-09-10T10:06:45.689 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:45 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:45.430+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:50.489601+0000 front 2024-09-10T10:05:50.489747+0000 (oldest deadline 2024-09-10T10:06:15.690110+0000) 2024-09-10T10:06:45.689 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:45 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:45.430+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:06:02.492771+0000 front 2024-09-10T10:06:02.492798+0000 (oldest deadline 2024-09-10T10:06:27.792793+0000) 2024-09-10T10:06:45.689 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:45 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:45.430+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-09-10T10:06:07.793699+0000 front 2024-09-10T10:06:07.793464+0000 (oldest deadline 2024-09-10T10:06:31.893248+0000) 2024-09-10T10:06:45.690 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:45 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:45.430+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-10T10:06:16.594588+0000 front 2024-09-10T10:06:16.594617+0000 (oldest deadline 2024-09-10T10:06:42.494249+0000) 2024-09-10T10:06:45.837 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-14a55226-6f59-11ef-bcea-c7b262605968@osd.8.service' 2024-09-10T10:06:45.877 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:06:45.877 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-09-10T10:06:45.877 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-09-10T10:06:45.878 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ceph-14a55226-6f59-11ef-bcea-c7b262605968@osd.9 2024-09-10T10:06:46.131 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:45 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:45.819+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:34.146460+0000 front 2024-09-10T10:05:34.146398+0000 (oldest deadline 2024-09-10T10:05:59.446520+0000) 2024-09-10T10:06:46.131 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:45 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:45.819+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:53.949416+0000 front 2024-09-10T10:05:53.949272+0000 (oldest deadline 2024-09-10T10:06:19.749872+0000) 2024-09-10T10:06:46.131 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:45 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:45.819+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:06:00.850855+0000 front 2024-09-10T10:06:00.850768+0000 (oldest deadline 2024-09-10T10:06:25.550818+0000) 2024-09-10T10:06:46.131 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:45 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:45.819+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-09-10T10:06:09.552349+0000 front 2024-09-10T10:06:09.552369+0000 (oldest deadline 2024-09-10T10:06:34.252329+0000) 2024-09-10T10:06:46.131 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:45 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:45.819+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-10T10:06:14.252778+0000 front 2024-09-10T10:06:14.252825+0000 (oldest deadline 2024-09-10T10:06:40.152981+0000) 2024-09-10T10:06:46.189 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:45 smithi077 systemd[1]: Stopping Ceph osd.9 for 14a55226-6f59-11ef-bcea-c7b262605968... 2024-09-10T10:06:46.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:46 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:46.022+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.045437+0000 front 2024-09-10T10:05:37.045486+0000 (oldest deadline 2024-09-10T10:05:59.945393+0000) 2024-09-10T10:06:46.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:46 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:46.022+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:43.446497+0000 front 2024-09-10T10:05:43.446655+0000 (oldest deadline 2024-09-10T10:06:09.346712+0000) 2024-09-10T10:06:46.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:46 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:46.022+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:49.347200+0000 front 2024-09-10T10:05:49.347266+0000 (oldest deadline 2024-09-10T10:06:19.947685+0000) 2024-09-10T10:06:46.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:46 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:46.022+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:05:59.948402+0000 front 2024-09-10T10:05:59.948527+0000 (oldest deadline 2024-09-10T10:06:26.348781+0000) 2024-09-10T10:06:46.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:46 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:46.022+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-09-10T10:06:10.349977+0000 front 2024-09-10T10:06:10.350027+0000 (oldest deadline 2024-09-10T10:06:33.750499+0000) 2024-09-10T10:06:46.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:46 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:46.022+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-10T10:06:18.352105+0000 front 2024-09-10T10:06:18.352025+0000 (oldest deadline 2024-09-10T10:06:44.251983+0000) 2024-09-10T10:06:46.690 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:46 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:46.295+0000 7fcc38c00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:06:46.690 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:46 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:46.295+0000 7fcc38c00640 -1 osd.9 100 *** Got signal Terminated *** 2024-09-10T10:06:46.690 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:46 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:46.295+0000 7fcc38c00640 -1 osd.9 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T10:06:46.690 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:46 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:46.409+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.086443+0000 front 2024-09-10T10:05:37.086361+0000 (oldest deadline 2024-09-10T10:06:02.887395+0000) 2024-09-10T10:06:46.690 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:46 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:46.409+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:42.888131+0000 front 2024-09-10T10:05:42.888100+0000 (oldest deadline 2024-09-10T10:06:07.588327+0000) 2024-09-10T10:06:46.690 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:46 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:46.409+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:50.489601+0000 front 2024-09-10T10:05:50.489747+0000 (oldest deadline 2024-09-10T10:06:15.690110+0000) 2024-09-10T10:06:46.690 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:46 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:46.409+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:06:02.492771+0000 front 2024-09-10T10:06:02.492798+0000 (oldest deadline 2024-09-10T10:06:27.792793+0000) 2024-09-10T10:06:46.690 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:46 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:46.409+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-09-10T10:06:07.793699+0000 front 2024-09-10T10:06:07.793464+0000 (oldest deadline 2024-09-10T10:06:31.893248+0000) 2024-09-10T10:06:46.690 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:46 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:46.409+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-10T10:06:16.594588+0000 front 2024-09-10T10:06:16.594617+0000 (oldest deadline 2024-09-10T10:06:42.494249+0000) 2024-09-10T10:06:47.131 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:46 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:46.770+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:34.146460+0000 front 2024-09-10T10:05:34.146398+0000 (oldest deadline 2024-09-10T10:05:59.446520+0000) 2024-09-10T10:06:47.131 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:46 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:46.770+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:53.949416+0000 front 2024-09-10T10:05:53.949272+0000 (oldest deadline 2024-09-10T10:06:19.749872+0000) 2024-09-10T10:06:47.131 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:46 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:46.770+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:06:00.850855+0000 front 2024-09-10T10:06:00.850768+0000 (oldest deadline 2024-09-10T10:06:25.550818+0000) 2024-09-10T10:06:47.131 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:46 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:46.770+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-09-10T10:06:09.552349+0000 front 2024-09-10T10:06:09.552369+0000 (oldest deadline 2024-09-10T10:06:34.252329+0000) 2024-09-10T10:06:47.131 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:46 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:46.770+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-10T10:06:14.252778+0000 front 2024-09-10T10:06:14.252825+0000 (oldest deadline 2024-09-10T10:06:40.152981+0000) 2024-09-10T10:06:47.325 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:47 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:47.010+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.045437+0000 front 2024-09-10T10:05:37.045486+0000 (oldest deadline 2024-09-10T10:05:59.945393+0000) 2024-09-10T10:06:47.325 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:47 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:47.010+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:43.446497+0000 front 2024-09-10T10:05:43.446655+0000 (oldest deadline 2024-09-10T10:06:09.346712+0000) 2024-09-10T10:06:47.325 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:47 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:47.010+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:49.347200+0000 front 2024-09-10T10:05:49.347266+0000 (oldest deadline 2024-09-10T10:06:19.947685+0000) 2024-09-10T10:06:47.325 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:47 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:47.010+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:05:59.948402+0000 front 2024-09-10T10:05:59.948527+0000 (oldest deadline 2024-09-10T10:06:26.348781+0000) 2024-09-10T10:06:47.325 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:47 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:47.010+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-09-10T10:06:10.349977+0000 front 2024-09-10T10:06:10.350027+0000 (oldest deadline 2024-09-10T10:06:33.750499+0000) 2024-09-10T10:06:47.326 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:47 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:47.010+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-10T10:06:18.352105+0000 front 2024-09-10T10:06:18.352025+0000 (oldest deadline 2024-09-10T10:06:44.251983+0000) 2024-09-10T10:06:47.689 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:47 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:47.375+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.086443+0000 front 2024-09-10T10:05:37.086361+0000 (oldest deadline 2024-09-10T10:06:02.887395+0000) 2024-09-10T10:06:47.689 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:47 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:47.375+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:42.888131+0000 front 2024-09-10T10:05:42.888100+0000 (oldest deadline 2024-09-10T10:06:07.588327+0000) 2024-09-10T10:06:47.690 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:47 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:47.375+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:50.489601+0000 front 2024-09-10T10:05:50.489747+0000 (oldest deadline 2024-09-10T10:06:15.690110+0000) 2024-09-10T10:06:47.690 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:47 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:47.375+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:06:02.492771+0000 front 2024-09-10T10:06:02.492798+0000 (oldest deadline 2024-09-10T10:06:27.792793+0000) 2024-09-10T10:06:47.690 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:47 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:47.375+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-09-10T10:06:07.793699+0000 front 2024-09-10T10:06:07.793464+0000 (oldest deadline 2024-09-10T10:06:31.893248+0000) 2024-09-10T10:06:47.690 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:47 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:47.375+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-10T10:06:16.594588+0000 front 2024-09-10T10:06:16.594617+0000 (oldest deadline 2024-09-10T10:06:42.494249+0000) 2024-09-10T10:06:48.131 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:47 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:47.769+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:34.146460+0000 front 2024-09-10T10:05:34.146398+0000 (oldest deadline 2024-09-10T10:05:59.446520+0000) 2024-09-10T10:06:48.131 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:47 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:47.769+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:53.949416+0000 front 2024-09-10T10:05:53.949272+0000 (oldest deadline 2024-09-10T10:06:19.749872+0000) 2024-09-10T10:06:48.131 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:47 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:47.769+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:06:00.850855+0000 front 2024-09-10T10:06:00.850768+0000 (oldest deadline 2024-09-10T10:06:25.550818+0000) 2024-09-10T10:06:48.131 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:47 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:47.769+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-09-10T10:06:09.552349+0000 front 2024-09-10T10:06:09.552369+0000 (oldest deadline 2024-09-10T10:06:34.252329+0000) 2024-09-10T10:06:48.131 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:47 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:47.769+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-10T10:06:14.252778+0000 front 2024-09-10T10:06:14.252825+0000 (oldest deadline 2024-09-10T10:06:40.152981+0000) 2024-09-10T10:06:48.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:48 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:48.035+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.045437+0000 front 2024-09-10T10:05:37.045486+0000 (oldest deadline 2024-09-10T10:05:59.945393+0000) 2024-09-10T10:06:48.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:48 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:48.035+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:43.446497+0000 front 2024-09-10T10:05:43.446655+0000 (oldest deadline 2024-09-10T10:06:09.346712+0000) 2024-09-10T10:06:48.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:48 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:48.035+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:49.347200+0000 front 2024-09-10T10:05:49.347266+0000 (oldest deadline 2024-09-10T10:06:19.947685+0000) 2024-09-10T10:06:48.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:48 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:48.035+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:05:59.948402+0000 front 2024-09-10T10:05:59.948527+0000 (oldest deadline 2024-09-10T10:06:26.348781+0000) 2024-09-10T10:06:48.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:48 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:48.035+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-09-10T10:06:10.349977+0000 front 2024-09-10T10:06:10.350027+0000 (oldest deadline 2024-09-10T10:06:33.750499+0000) 2024-09-10T10:06:48.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:48 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:48.035+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-10T10:06:18.352105+0000 front 2024-09-10T10:06:18.352025+0000 (oldest deadline 2024-09-10T10:06:44.251983+0000) 2024-09-10T10:06:48.689 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:48 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:48.349+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.086443+0000 front 2024-09-10T10:05:37.086361+0000 (oldest deadline 2024-09-10T10:06:02.887395+0000) 2024-09-10T10:06:48.689 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:48 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:48.349+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:42.888131+0000 front 2024-09-10T10:05:42.888100+0000 (oldest deadline 2024-09-10T10:06:07.588327+0000) 2024-09-10T10:06:48.689 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:48 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:48.349+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:50.489601+0000 front 2024-09-10T10:05:50.489747+0000 (oldest deadline 2024-09-10T10:06:15.690110+0000) 2024-09-10T10:06:48.690 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:48 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:48.349+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:06:02.492771+0000 front 2024-09-10T10:06:02.492798+0000 (oldest deadline 2024-09-10T10:06:27.792793+0000) 2024-09-10T10:06:48.690 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:48 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:48.349+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-09-10T10:06:07.793699+0000 front 2024-09-10T10:06:07.793464+0000 (oldest deadline 2024-09-10T10:06:31.893248+0000) 2024-09-10T10:06:48.690 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:48 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:48.349+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-10T10:06:16.594588+0000 front 2024-09-10T10:06:16.594617+0000 (oldest deadline 2024-09-10T10:06:42.494249+0000) 2024-09-10T10:06:49.131 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:48 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:48.803+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:34.146460+0000 front 2024-09-10T10:05:34.146398+0000 (oldest deadline 2024-09-10T10:05:59.446520+0000) 2024-09-10T10:06:49.131 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:48 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:48.803+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:53.949416+0000 front 2024-09-10T10:05:53.949272+0000 (oldest deadline 2024-09-10T10:06:19.749872+0000) 2024-09-10T10:06:49.131 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:48 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:48.803+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:06:00.850855+0000 front 2024-09-10T10:06:00.850768+0000 (oldest deadline 2024-09-10T10:06:25.550818+0000) 2024-09-10T10:06:49.131 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:48 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:48.803+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-09-10T10:06:09.552349+0000 front 2024-09-10T10:06:09.552369+0000 (oldest deadline 2024-09-10T10:06:34.252329+0000) 2024-09-10T10:06:49.131 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:48 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:48.803+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-10T10:06:14.252778+0000 front 2024-09-10T10:06:14.252825+0000 (oldest deadline 2024-09-10T10:06:40.152981+0000) 2024-09-10T10:06:49.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:49 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:49.004+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.045437+0000 front 2024-09-10T10:05:37.045486+0000 (oldest deadline 2024-09-10T10:05:59.945393+0000) 2024-09-10T10:06:49.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:49 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:49.004+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:43.446497+0000 front 2024-09-10T10:05:43.446655+0000 (oldest deadline 2024-09-10T10:06:09.346712+0000) 2024-09-10T10:06:49.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:49 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:49.004+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:49.347200+0000 front 2024-09-10T10:05:49.347266+0000 (oldest deadline 2024-09-10T10:06:19.947685+0000) 2024-09-10T10:06:49.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:49 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:49.004+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:05:59.948402+0000 front 2024-09-10T10:05:59.948527+0000 (oldest deadline 2024-09-10T10:06:26.348781+0000) 2024-09-10T10:06:49.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:49 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:49.004+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-09-10T10:06:10.349977+0000 front 2024-09-10T10:06:10.350027+0000 (oldest deadline 2024-09-10T10:06:33.750499+0000) 2024-09-10T10:06:49.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:49 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:49.004+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-10T10:06:18.352105+0000 front 2024-09-10T10:06:18.352025+0000 (oldest deadline 2024-09-10T10:06:44.251983+0000) 2024-09-10T10:06:49.689 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:49 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:49.321+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.086443+0000 front 2024-09-10T10:05:37.086361+0000 (oldest deadline 2024-09-10T10:06:02.887395+0000) 2024-09-10T10:06:49.689 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:49 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:49.321+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:42.888131+0000 front 2024-09-10T10:05:42.888100+0000 (oldest deadline 2024-09-10T10:06:07.588327+0000) 2024-09-10T10:06:49.689 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:49 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:49.321+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:50.489601+0000 front 2024-09-10T10:05:50.489747+0000 (oldest deadline 2024-09-10T10:06:15.690110+0000) 2024-09-10T10:06:49.689 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:49 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:49.321+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:06:02.492771+0000 front 2024-09-10T10:06:02.492798+0000 (oldest deadline 2024-09-10T10:06:27.792793+0000) 2024-09-10T10:06:49.690 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:49 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:49.321+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-09-10T10:06:07.793699+0000 front 2024-09-10T10:06:07.793464+0000 (oldest deadline 2024-09-10T10:06:31.893248+0000) 2024-09-10T10:06:49.690 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:49 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:49.321+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-10T10:06:16.594588+0000 front 2024-09-10T10:06:16.594617+0000 (oldest deadline 2024-09-10T10:06:42.494249+0000) 2024-09-10T10:06:50.131 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:49 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:49.756+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:34.146460+0000 front 2024-09-10T10:05:34.146398+0000 (oldest deadline 2024-09-10T10:05:59.446520+0000) 2024-09-10T10:06:50.132 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:49 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:49.756+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:53.949416+0000 front 2024-09-10T10:05:53.949272+0000 (oldest deadline 2024-09-10T10:06:19.749872+0000) 2024-09-10T10:06:50.132 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:49 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:49.756+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:06:00.850855+0000 front 2024-09-10T10:06:00.850768+0000 (oldest deadline 2024-09-10T10:06:25.550818+0000) 2024-09-10T10:06:50.132 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:49 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:49.756+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-09-10T10:06:09.552349+0000 front 2024-09-10T10:06:09.552369+0000 (oldest deadline 2024-09-10T10:06:34.252329+0000) 2024-09-10T10:06:50.132 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:49 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:49.756+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-10T10:06:14.252778+0000 front 2024-09-10T10:06:14.252825+0000 (oldest deadline 2024-09-10T10:06:40.152981+0000) 2024-09-10T10:06:50.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:50 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:50.013+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.045437+0000 front 2024-09-10T10:05:37.045486+0000 (oldest deadline 2024-09-10T10:05:59.945393+0000) 2024-09-10T10:06:50.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:50 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:50.013+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:43.446497+0000 front 2024-09-10T10:05:43.446655+0000 (oldest deadline 2024-09-10T10:06:09.346712+0000) 2024-09-10T10:06:50.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:50 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:50.013+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:49.347200+0000 front 2024-09-10T10:05:49.347266+0000 (oldest deadline 2024-09-10T10:06:19.947685+0000) 2024-09-10T10:06:50.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:50 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:50.013+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:05:59.948402+0000 front 2024-09-10T10:05:59.948527+0000 (oldest deadline 2024-09-10T10:06:26.348781+0000) 2024-09-10T10:06:50.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:50 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:50.013+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-09-10T10:06:10.349977+0000 front 2024-09-10T10:06:10.350027+0000 (oldest deadline 2024-09-10T10:06:33.750499+0000) 2024-09-10T10:06:50.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:50 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:50.013+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-10T10:06:18.352105+0000 front 2024-09-10T10:06:18.352025+0000 (oldest deadline 2024-09-10T10:06:44.251983+0000) 2024-09-10T10:06:50.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:50 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:50.013+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6822 osd.6 since back 2024-09-10T10:06:24.252782+0000 front 2024-09-10T10:06:24.252834+0000 (oldest deadline 2024-09-10T10:06:49.552548+0000) 2024-09-10T10:06:50.689 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:50 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:50.325+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.086443+0000 front 2024-09-10T10:05:37.086361+0000 (oldest deadline 2024-09-10T10:06:02.887395+0000) 2024-09-10T10:06:50.689 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:50 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:50.325+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:42.888131+0000 front 2024-09-10T10:05:42.888100+0000 (oldest deadline 2024-09-10T10:06:07.588327+0000) 2024-09-10T10:06:50.690 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:50 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:50.325+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:50.489601+0000 front 2024-09-10T10:05:50.489747+0000 (oldest deadline 2024-09-10T10:06:15.690110+0000) 2024-09-10T10:06:50.690 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:50 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:50.325+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:06:02.492771+0000 front 2024-09-10T10:06:02.492798+0000 (oldest deadline 2024-09-10T10:06:27.792793+0000) 2024-09-10T10:06:50.690 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:50 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:50.325+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-09-10T10:06:07.793699+0000 front 2024-09-10T10:06:07.793464+0000 (oldest deadline 2024-09-10T10:06:31.893248+0000) 2024-09-10T10:06:50.690 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:50 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9[64003]: 2024-09-10T10:06:50.325+0000 7fcc2f600640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-10T10:06:16.594588+0000 front 2024-09-10T10:06:16.594617+0000 (oldest deadline 2024-09-10T10:06:42.494249+0000) 2024-09-10T10:06:51.131 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:50 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:50.791+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:34.146460+0000 front 2024-09-10T10:05:34.146398+0000 (oldest deadline 2024-09-10T10:05:59.446520+0000) 2024-09-10T10:06:51.132 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:50 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:50.791+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:53.949416+0000 front 2024-09-10T10:05:53.949272+0000 (oldest deadline 2024-09-10T10:06:19.749872+0000) 2024-09-10T10:06:51.132 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:50 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:50.791+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:06:00.850855+0000 front 2024-09-10T10:06:00.850768+0000 (oldest deadline 2024-09-10T10:06:25.550818+0000) 2024-09-10T10:06:51.132 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:50 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:50.791+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-09-10T10:06:09.552349+0000 front 2024-09-10T10:06:09.552369+0000 (oldest deadline 2024-09-10T10:06:34.252329+0000) 2024-09-10T10:06:51.132 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:50 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:50.791+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-10T10:06:14.252778+0000 front 2024-09-10T10:06:14.252825+0000 (oldest deadline 2024-09-10T10:06:40.152981+0000) 2024-09-10T10:06:51.132 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:50 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:50.791+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6822 osd.6 since back 2024-09-10T10:06:25.454162+0000 front 2024-09-10T10:06:25.454072+0000 (oldest deadline 2024-09-10T10:06:50.153921+0000) 2024-09-10T10:06:51.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:51 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:51.041+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.045437+0000 front 2024-09-10T10:05:37.045486+0000 (oldest deadline 2024-09-10T10:05:59.945393+0000) 2024-09-10T10:06:51.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:51 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:51.041+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:43.446497+0000 front 2024-09-10T10:05:43.446655+0000 (oldest deadline 2024-09-10T10:06:09.346712+0000) 2024-09-10T10:06:51.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:51 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:51.041+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:49.347200+0000 front 2024-09-10T10:05:49.347266+0000 (oldest deadline 2024-09-10T10:06:19.947685+0000) 2024-09-10T10:06:51.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:51 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:51.041+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:05:59.948402+0000 front 2024-09-10T10:05:59.948527+0000 (oldest deadline 2024-09-10T10:06:26.348781+0000) 2024-09-10T10:06:51.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:51 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:51.041+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-09-10T10:06:10.349977+0000 front 2024-09-10T10:06:10.350027+0000 (oldest deadline 2024-09-10T10:06:33.750499+0000) 2024-09-10T10:06:51.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:51 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:51.041+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-10T10:06:18.352105+0000 front 2024-09-10T10:06:18.352025+0000 (oldest deadline 2024-09-10T10:06:44.251983+0000) 2024-09-10T10:06:51.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:51 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:51.041+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6822 osd.6 since back 2024-09-10T10:06:24.252782+0000 front 2024-09-10T10:06:24.252834+0000 (oldest deadline 2024-09-10T10:06:49.552548+0000) 2024-09-10T10:06:51.985 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:51 smithi077 podman[128321]: 2024-09-10 10:06:51.716912881 +0000 UTC m=+5.549725358 container died ac095d943f808ec214de42e8f5ae6f0ac364223f0bb1e4961933078f4c5c389e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-10T10:06:52.131 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:51 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:51.832+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:34.146460+0000 front 2024-09-10T10:05:34.146398+0000 (oldest deadline 2024-09-10T10:05:59.446520+0000) 2024-09-10T10:06:52.131 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:51 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:51.832+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:53.949416+0000 front 2024-09-10T10:05:53.949272+0000 (oldest deadline 2024-09-10T10:06:19.749872+0000) 2024-09-10T10:06:52.131 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:51 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:51.832+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:06:00.850855+0000 front 2024-09-10T10:06:00.850768+0000 (oldest deadline 2024-09-10T10:06:25.550818+0000) 2024-09-10T10:06:52.132 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:51 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:51.832+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-09-10T10:06:09.552349+0000 front 2024-09-10T10:06:09.552369+0000 (oldest deadline 2024-09-10T10:06:34.252329+0000) 2024-09-10T10:06:52.132 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:51 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:51.832+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-10T10:06:14.252778+0000 front 2024-09-10T10:06:14.252825+0000 (oldest deadline 2024-09-10T10:06:40.152981+0000) 2024-09-10T10:06:52.132 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:51 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:51.832+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6822 osd.6 since back 2024-09-10T10:06:25.454162+0000 front 2024-09-10T10:06:25.454072+0000 (oldest deadline 2024-09-10T10:06:50.153921+0000) 2024-09-10T10:06:52.396 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:52 smithi077 podman[128321]: 2024-09-10 10:06:52.183324213 +0000 UTC m=+6.016136743 container cleanup ac095d943f808ec214de42e8f5ae6f0ac364223f0bb1e4961933078f4c5c389e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T10:06:52.397 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:52 smithi077 bash[128321]: ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9 2024-09-10T10:06:52.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:52 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:52.034+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.045437+0000 front 2024-09-10T10:05:37.045486+0000 (oldest deadline 2024-09-10T10:05:59.945393+0000) 2024-09-10T10:06:52.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:52 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:52.034+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:43.446497+0000 front 2024-09-10T10:05:43.446655+0000 (oldest deadline 2024-09-10T10:06:09.346712+0000) 2024-09-10T10:06:52.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:52 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:52.034+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:49.347200+0000 front 2024-09-10T10:05:49.347266+0000 (oldest deadline 2024-09-10T10:06:19.947685+0000) 2024-09-10T10:06:52.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:52 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:52.034+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:05:59.948402+0000 front 2024-09-10T10:05:59.948527+0000 (oldest deadline 2024-09-10T10:06:26.348781+0000) 2024-09-10T10:06:52.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:52 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:52.034+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-09-10T10:06:10.349977+0000 front 2024-09-10T10:06:10.350027+0000 (oldest deadline 2024-09-10T10:06:33.750499+0000) 2024-09-10T10:06:52.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:52 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:52.034+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-10T10:06:18.352105+0000 front 2024-09-10T10:06:18.352025+0000 (oldest deadline 2024-09-10T10:06:44.251983+0000) 2024-09-10T10:06:52.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:52 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:52.034+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6822 osd.6 since back 2024-09-10T10:06:24.252782+0000 front 2024-09-10T10:06:24.252834+0000 (oldest deadline 2024-09-10T10:06:49.552548+0000) 2024-09-10T10:06:52.689 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:52 smithi077 podman[128337]: 2024-09-10 10:06:52.517935073 +0000 UTC m=+0.790961500 container remove ac095d943f808ec214de42e8f5ae6f0ac364223f0bb1e4961933078f4c5c389e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-10T10:06:53.255 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:52 smithi077 podman[128427]: 2024-09-10 10:06:52.897320168 +0000 UTC m=+0.028578583 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T10:06:53.255 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:53 smithi077 podman[128427]: 2024-09-10 10:06:53.002036563 +0000 UTC m=+0.133294972 container create 256b1779f4095aebfb54b026858145c79cf96a150bf37508526ccb055be1806c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0) 2024-09-10T10:06:53.255 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:53 smithi077 podman[128427]: 2024-09-10 10:06:53.254413205 +0000 UTC m=+0.385671636 container init 256b1779f4095aebfb54b026858145c79cf96a150bf37508526ccb055be1806c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9-deactivate, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T10:06:53.381 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:52 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:52.881+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:34.146460+0000 front 2024-09-10T10:05:34.146398+0000 (oldest deadline 2024-09-10T10:05:59.446520+0000) 2024-09-10T10:06:53.381 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:52 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:52.881+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:53.949416+0000 front 2024-09-10T10:05:53.949272+0000 (oldest deadline 2024-09-10T10:06:19.749872+0000) 2024-09-10T10:06:53.381 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:52 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:52.881+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:06:00.850855+0000 front 2024-09-10T10:06:00.850768+0000 (oldest deadline 2024-09-10T10:06:25.550818+0000) 2024-09-10T10:06:53.381 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:52 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:52.881+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-09-10T10:06:09.552349+0000 front 2024-09-10T10:06:09.552369+0000 (oldest deadline 2024-09-10T10:06:34.252329+0000) 2024-09-10T10:06:53.382 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:52 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:52.881+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-10T10:06:14.252778+0000 front 2024-09-10T10:06:14.252825+0000 (oldest deadline 2024-09-10T10:06:40.152981+0000) 2024-09-10T10:06:53.382 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:52 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:52.881+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6822 osd.6 since back 2024-09-10T10:06:25.454162+0000 front 2024-09-10T10:06:25.454072+0000 (oldest deadline 2024-09-10T10:06:50.153921+0000) 2024-09-10T10:06:53.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:53 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:53.029+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.045437+0000 front 2024-09-10T10:05:37.045486+0000 (oldest deadline 2024-09-10T10:05:59.945393+0000) 2024-09-10T10:06:53.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:53 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:53.029+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:43.446497+0000 front 2024-09-10T10:05:43.446655+0000 (oldest deadline 2024-09-10T10:06:09.346712+0000) 2024-09-10T10:06:53.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:53 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:53.029+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:49.347200+0000 front 2024-09-10T10:05:49.347266+0000 (oldest deadline 2024-09-10T10:06:19.947685+0000) 2024-09-10T10:06:53.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:53 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:53.029+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:05:59.948402+0000 front 2024-09-10T10:05:59.948527+0000 (oldest deadline 2024-09-10T10:06:26.348781+0000) 2024-09-10T10:06:53.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:53 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:53.029+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-09-10T10:06:10.349977+0000 front 2024-09-10T10:06:10.350027+0000 (oldest deadline 2024-09-10T10:06:33.750499+0000) 2024-09-10T10:06:53.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:53 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:53.029+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-10T10:06:18.352105+0000 front 2024-09-10T10:06:18.352025+0000 (oldest deadline 2024-09-10T10:06:44.251983+0000) 2024-09-10T10:06:53.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:53 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:53.029+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6822 osd.6 since back 2024-09-10T10:06:24.252782+0000 front 2024-09-10T10:06:24.252834+0000 (oldest deadline 2024-09-10T10:06:49.552548+0000) 2024-09-10T10:06:53.515 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:53 smithi077 podman[128427]: 2024-09-10 10:06:53.261686371 +0000 UTC m=+0.392944787 container start 256b1779f4095aebfb54b026858145c79cf96a150bf37508526ccb055be1806c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T10:06:53.515 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:53 smithi077 podman[128427]: 2024-09-10 10:06:53.362693585 +0000 UTC m=+0.493951998 container attach 256b1779f4095aebfb54b026858145c79cf96a150bf37508526ccb055be1806c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9-deactivate, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux ) 2024-09-10T10:06:53.908 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 10 10:06:53 smithi077 podman[128427]: 2024-09-10 10:06:53.515235685 +0000 UTC m=+0.646494105 container died 256b1779f4095aebfb54b026858145c79cf96a150bf37508526ccb055be1806c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T10:06:54.131 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:53 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:53.844+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:34.146460+0000 front 2024-09-10T10:05:34.146398+0000 (oldest deadline 2024-09-10T10:05:59.446520+0000) 2024-09-10T10:06:54.131 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:53 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:53.844+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:53.949416+0000 front 2024-09-10T10:05:53.949272+0000 (oldest deadline 2024-09-10T10:06:19.749872+0000) 2024-09-10T10:06:54.132 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:53 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:53.844+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:06:00.850855+0000 front 2024-09-10T10:06:00.850768+0000 (oldest deadline 2024-09-10T10:06:25.550818+0000) 2024-09-10T10:06:54.132 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:53 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:53.844+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-09-10T10:06:09.552349+0000 front 2024-09-10T10:06:09.552369+0000 (oldest deadline 2024-09-10T10:06:34.252329+0000) 2024-09-10T10:06:54.132 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:53 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:53.844+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-10T10:06:14.252778+0000 front 2024-09-10T10:06:14.252825+0000 (oldest deadline 2024-09-10T10:06:40.152981+0000) 2024-09-10T10:06:54.132 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:53 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:53.844+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6822 osd.6 since back 2024-09-10T10:06:25.454162+0000 front 2024-09-10T10:06:25.454072+0000 (oldest deadline 2024-09-10T10:06:50.153921+0000) 2024-09-10T10:06:54.255 DEBUG:teuthology.orchestra.run.smithi077:> sudo pkill -f 'journalctl -f -n 0 -u ceph-14a55226-6f59-11ef-bcea-c7b262605968@osd.9.service' 2024-09-10T10:06:54.330 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:06:54.330 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-09-10T10:06:54.330 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-09-10T10:06:54.330 DEBUG:teuthology.orchestra.run.smithi120:> sudo systemctl stop ceph-14a55226-6f59-11ef-bcea-c7b262605968@osd.10 2024-09-10T10:06:54.358 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:54 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:54.034+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.045437+0000 front 2024-09-10T10:05:37.045486+0000 (oldest deadline 2024-09-10T10:05:59.945393+0000) 2024-09-10T10:06:54.358 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:54 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:54.034+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:43.446497+0000 front 2024-09-10T10:05:43.446655+0000 (oldest deadline 2024-09-10T10:06:09.346712+0000) 2024-09-10T10:06:54.358 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:54 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:54.034+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:49.347200+0000 front 2024-09-10T10:05:49.347266+0000 (oldest deadline 2024-09-10T10:06:19.947685+0000) 2024-09-10T10:06:54.358 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:54 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:54.034+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:05:59.948402+0000 front 2024-09-10T10:05:59.948527+0000 (oldest deadline 2024-09-10T10:06:26.348781+0000) 2024-09-10T10:06:54.359 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:54 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:54.034+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-09-10T10:06:10.349977+0000 front 2024-09-10T10:06:10.350027+0000 (oldest deadline 2024-09-10T10:06:33.750499+0000) 2024-09-10T10:06:54.359 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:54 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:54.034+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-10T10:06:18.352105+0000 front 2024-09-10T10:06:18.352025+0000 (oldest deadline 2024-09-10T10:06:44.251983+0000) 2024-09-10T10:06:54.359 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:54 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:54.034+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6822 osd.6 since back 2024-09-10T10:06:24.252782+0000 front 2024-09-10T10:06:24.252834+0000 (oldest deadline 2024-09-10T10:06:49.552548+0000) 2024-09-10T10:06:54.646 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:54 smithi120 systemd[1]: Stopping Ceph osd.10 for 14a55226-6f59-11ef-bcea-c7b262605968... 2024-09-10T10:06:54.646 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:54 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:54.643+0000 7f7592bee640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:06:54.646 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:54 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:54.643+0000 7f7592bee640 -1 osd.10 100 *** Got signal Terminated *** 2024-09-10T10:06:54.646 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:54 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:54.643+0000 7f7592bee640 -1 osd.10 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T10:06:55.131 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:54 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:54.833+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:34.146460+0000 front 2024-09-10T10:05:34.146398+0000 (oldest deadline 2024-09-10T10:05:59.446520+0000) 2024-09-10T10:06:55.131 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:54 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:54.833+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:53.949416+0000 front 2024-09-10T10:05:53.949272+0000 (oldest deadline 2024-09-10T10:06:19.749872+0000) 2024-09-10T10:06:55.131 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:54 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:54.833+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:06:00.850855+0000 front 2024-09-10T10:06:00.850768+0000 (oldest deadline 2024-09-10T10:06:25.550818+0000) 2024-09-10T10:06:55.131 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:54 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:54.833+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-09-10T10:06:09.552349+0000 front 2024-09-10T10:06:09.552369+0000 (oldest deadline 2024-09-10T10:06:34.252329+0000) 2024-09-10T10:06:55.132 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:54 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:54.833+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-10T10:06:14.252778+0000 front 2024-09-10T10:06:14.252825+0000 (oldest deadline 2024-09-10T10:06:40.152981+0000) 2024-09-10T10:06:55.132 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:54 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:54.833+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6822 osd.6 since back 2024-09-10T10:06:25.454162+0000 front 2024-09-10T10:06:25.454072+0000 (oldest deadline 2024-09-10T10:06:50.153921+0000) 2024-09-10T10:06:55.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:54 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:54.984+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.045437+0000 front 2024-09-10T10:05:37.045486+0000 (oldest deadline 2024-09-10T10:05:59.945393+0000) 2024-09-10T10:06:55.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:54 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:54.984+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:43.446497+0000 front 2024-09-10T10:05:43.446655+0000 (oldest deadline 2024-09-10T10:06:09.346712+0000) 2024-09-10T10:06:55.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:54 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:54.984+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:49.347200+0000 front 2024-09-10T10:05:49.347266+0000 (oldest deadline 2024-09-10T10:06:19.947685+0000) 2024-09-10T10:06:55.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:54 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:54.984+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:05:59.948402+0000 front 2024-09-10T10:05:59.948527+0000 (oldest deadline 2024-09-10T10:06:26.348781+0000) 2024-09-10T10:06:55.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:54 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:54.984+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-09-10T10:06:10.349977+0000 front 2024-09-10T10:06:10.350027+0000 (oldest deadline 2024-09-10T10:06:33.750499+0000) 2024-09-10T10:06:55.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:54 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:54.984+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-10T10:06:18.352105+0000 front 2024-09-10T10:06:18.352025+0000 (oldest deadline 2024-09-10T10:06:44.251983+0000) 2024-09-10T10:06:55.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:54 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:54.984+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6822 osd.6 since back 2024-09-10T10:06:24.252782+0000 front 2024-09-10T10:06:24.252834+0000 (oldest deadline 2024-09-10T10:06:49.552548+0000) 2024-09-10T10:06:56.131 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:55 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:55.787+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:34.146460+0000 front 2024-09-10T10:05:34.146398+0000 (oldest deadline 2024-09-10T10:05:59.446520+0000) 2024-09-10T10:06:56.131 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:55 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:55.787+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:53.949416+0000 front 2024-09-10T10:05:53.949272+0000 (oldest deadline 2024-09-10T10:06:19.749872+0000) 2024-09-10T10:06:56.131 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:55 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:55.787+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:06:00.850855+0000 front 2024-09-10T10:06:00.850768+0000 (oldest deadline 2024-09-10T10:06:25.550818+0000) 2024-09-10T10:06:56.131 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:55 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:55.787+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-09-10T10:06:09.552349+0000 front 2024-09-10T10:06:09.552369+0000 (oldest deadline 2024-09-10T10:06:34.252329+0000) 2024-09-10T10:06:56.131 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:55 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:55.787+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-10T10:06:14.252778+0000 front 2024-09-10T10:06:14.252825+0000 (oldest deadline 2024-09-10T10:06:40.152981+0000) 2024-09-10T10:06:56.132 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:55 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:55.787+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6822 osd.6 since back 2024-09-10T10:06:25.454162+0000 front 2024-09-10T10:06:25.454072+0000 (oldest deadline 2024-09-10T10:06:50.153921+0000) 2024-09-10T10:06:56.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:55 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:55.956+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.045437+0000 front 2024-09-10T10:05:37.045486+0000 (oldest deadline 2024-09-10T10:05:59.945393+0000) 2024-09-10T10:06:56.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:55 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:55.956+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:43.446497+0000 front 2024-09-10T10:05:43.446655+0000 (oldest deadline 2024-09-10T10:06:09.346712+0000) 2024-09-10T10:06:56.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:55 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:55.956+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:49.347200+0000 front 2024-09-10T10:05:49.347266+0000 (oldest deadline 2024-09-10T10:06:19.947685+0000) 2024-09-10T10:06:56.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:55 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:55.956+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:05:59.948402+0000 front 2024-09-10T10:05:59.948527+0000 (oldest deadline 2024-09-10T10:06:26.348781+0000) 2024-09-10T10:06:56.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:55 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:55.956+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-09-10T10:06:10.349977+0000 front 2024-09-10T10:06:10.350027+0000 (oldest deadline 2024-09-10T10:06:33.750499+0000) 2024-09-10T10:06:56.425 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:55 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:55.956+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-10T10:06:18.352105+0000 front 2024-09-10T10:06:18.352025+0000 (oldest deadline 2024-09-10T10:06:44.251983+0000) 2024-09-10T10:06:56.425 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:55 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:55.956+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6822 osd.6 since back 2024-09-10T10:06:24.252782+0000 front 2024-09-10T10:06:24.252834+0000 (oldest deadline 2024-09-10T10:06:49.552548+0000) 2024-09-10T10:06:57.131 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:56 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:56.812+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:34.146460+0000 front 2024-09-10T10:05:34.146398+0000 (oldest deadline 2024-09-10T10:05:59.446520+0000) 2024-09-10T10:06:57.131 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:56 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:56.812+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:53.949416+0000 front 2024-09-10T10:05:53.949272+0000 (oldest deadline 2024-09-10T10:06:19.749872+0000) 2024-09-10T10:06:57.131 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:56 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:56.812+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:06:00.850855+0000 front 2024-09-10T10:06:00.850768+0000 (oldest deadline 2024-09-10T10:06:25.550818+0000) 2024-09-10T10:06:57.131 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:56 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:56.812+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-09-10T10:06:09.552349+0000 front 2024-09-10T10:06:09.552369+0000 (oldest deadline 2024-09-10T10:06:34.252329+0000) 2024-09-10T10:06:57.131 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:56 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:56.812+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-10T10:06:14.252778+0000 front 2024-09-10T10:06:14.252825+0000 (oldest deadline 2024-09-10T10:06:40.152981+0000) 2024-09-10T10:06:57.131 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:56 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:56.812+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6822 osd.6 since back 2024-09-10T10:06:25.454162+0000 front 2024-09-10T10:06:25.454072+0000 (oldest deadline 2024-09-10T10:06:50.153921+0000) 2024-09-10T10:06:57.377 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:56 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:56.971+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.045437+0000 front 2024-09-10T10:05:37.045486+0000 (oldest deadline 2024-09-10T10:05:59.945393+0000) 2024-09-10T10:06:57.377 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:56 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:56.971+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:43.446497+0000 front 2024-09-10T10:05:43.446655+0000 (oldest deadline 2024-09-10T10:06:09.346712+0000) 2024-09-10T10:06:57.377 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:56 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:56.971+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:49.347200+0000 front 2024-09-10T10:05:49.347266+0000 (oldest deadline 2024-09-10T10:06:19.947685+0000) 2024-09-10T10:06:57.377 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:56 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:56.971+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:05:59.948402+0000 front 2024-09-10T10:05:59.948527+0000 (oldest deadline 2024-09-10T10:06:26.348781+0000) 2024-09-10T10:06:57.377 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:56 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:56.971+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-09-10T10:06:10.349977+0000 front 2024-09-10T10:06:10.350027+0000 (oldest deadline 2024-09-10T10:06:33.750499+0000) 2024-09-10T10:06:57.377 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:56 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:56.971+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-10T10:06:18.352105+0000 front 2024-09-10T10:06:18.352025+0000 (oldest deadline 2024-09-10T10:06:44.251983+0000) 2024-09-10T10:06:57.377 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:56 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:56.971+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6822 osd.6 since back 2024-09-10T10:06:24.252782+0000 front 2024-09-10T10:06:24.252834+0000 (oldest deadline 2024-09-10T10:06:49.552548+0000) 2024-09-10T10:06:58.131 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:57 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:57.847+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:34.146460+0000 front 2024-09-10T10:05:34.146398+0000 (oldest deadline 2024-09-10T10:05:59.446520+0000) 2024-09-10T10:06:58.131 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:57 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:57.847+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:53.949416+0000 front 2024-09-10T10:05:53.949272+0000 (oldest deadline 2024-09-10T10:06:19.749872+0000) 2024-09-10T10:06:58.131 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:57 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:57.847+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:06:00.850855+0000 front 2024-09-10T10:06:00.850768+0000 (oldest deadline 2024-09-10T10:06:25.550818+0000) 2024-09-10T10:06:58.132 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:57 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:57.847+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-09-10T10:06:09.552349+0000 front 2024-09-10T10:06:09.552369+0000 (oldest deadline 2024-09-10T10:06:34.252329+0000) 2024-09-10T10:06:58.132 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:57 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:57.847+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-10T10:06:14.252778+0000 front 2024-09-10T10:06:14.252825+0000 (oldest deadline 2024-09-10T10:06:40.152981+0000) 2024-09-10T10:06:58.132 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:57 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:57.847+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6822 osd.6 since back 2024-09-10T10:06:25.454162+0000 front 2024-09-10T10:06:25.454072+0000 (oldest deadline 2024-09-10T10:06:50.153921+0000) 2024-09-10T10:06:58.425 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:57 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:57.958+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.045437+0000 front 2024-09-10T10:05:37.045486+0000 (oldest deadline 2024-09-10T10:05:59.945393+0000) 2024-09-10T10:06:58.425 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:57 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:57.958+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:43.446497+0000 front 2024-09-10T10:05:43.446655+0000 (oldest deadline 2024-09-10T10:06:09.346712+0000) 2024-09-10T10:06:58.425 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:57 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:57.958+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:49.347200+0000 front 2024-09-10T10:05:49.347266+0000 (oldest deadline 2024-09-10T10:06:19.947685+0000) 2024-09-10T10:06:58.425 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:57 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:57.958+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:05:59.948402+0000 front 2024-09-10T10:05:59.948527+0000 (oldest deadline 2024-09-10T10:06:26.348781+0000) 2024-09-10T10:06:58.426 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:57 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:57.958+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-09-10T10:06:10.349977+0000 front 2024-09-10T10:06:10.350027+0000 (oldest deadline 2024-09-10T10:06:33.750499+0000) 2024-09-10T10:06:58.426 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:57 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:57.958+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-10T10:06:18.352105+0000 front 2024-09-10T10:06:18.352025+0000 (oldest deadline 2024-09-10T10:06:44.251983+0000) 2024-09-10T10:06:58.426 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:57 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:57.958+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6822 osd.6 since back 2024-09-10T10:06:24.252782+0000 front 2024-09-10T10:06:24.252834+0000 (oldest deadline 2024-09-10T10:06:49.552548+0000) 2024-09-10T10:06:59.131 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:58 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:58.866+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:34.146460+0000 front 2024-09-10T10:05:34.146398+0000 (oldest deadline 2024-09-10T10:05:59.446520+0000) 2024-09-10T10:06:59.132 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:58 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:58.866+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:53.949416+0000 front 2024-09-10T10:05:53.949272+0000 (oldest deadline 2024-09-10T10:06:19.749872+0000) 2024-09-10T10:06:59.132 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:58 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:58.866+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:06:00.850855+0000 front 2024-09-10T10:06:00.850768+0000 (oldest deadline 2024-09-10T10:06:25.550818+0000) 2024-09-10T10:06:59.132 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:58 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:58.866+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-09-10T10:06:09.552349+0000 front 2024-09-10T10:06:09.552369+0000 (oldest deadline 2024-09-10T10:06:34.252329+0000) 2024-09-10T10:06:59.132 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:58 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:58.866+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-10T10:06:14.252778+0000 front 2024-09-10T10:06:14.252825+0000 (oldest deadline 2024-09-10T10:06:40.152981+0000) 2024-09-10T10:06:59.132 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:58 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:58.866+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6822 osd.6 since back 2024-09-10T10:06:25.454162+0000 front 2024-09-10T10:06:25.454072+0000 (oldest deadline 2024-09-10T10:06:50.153921+0000) 2024-09-10T10:06:59.132 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:58 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:58.866+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.120:6820 osd.7 since back 2024-09-10T10:06:35.355082+0000 front 2024-09-10T10:06:35.355177+0000 (oldest deadline 2024-09-10T10:06:58.255037+0000) 2024-09-10T10:06:59.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:58 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:58.997+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:37.045437+0000 front 2024-09-10T10:05:37.045486+0000 (oldest deadline 2024-09-10T10:05:59.945393+0000) 2024-09-10T10:06:59.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:58 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:58.997+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-09-10T10:05:43.446497+0000 front 2024-09-10T10:05:43.446655+0000 (oldest deadline 2024-09-10T10:06:09.346712+0000) 2024-09-10T10:06:59.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:58 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:58.997+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:49.347200+0000 front 2024-09-10T10:05:49.347266+0000 (oldest deadline 2024-09-10T10:06:19.947685+0000) 2024-09-10T10:06:59.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:58 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:58.997+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:05:59.948402+0000 front 2024-09-10T10:05:59.948527+0000 (oldest deadline 2024-09-10T10:06:26.348781+0000) 2024-09-10T10:06:59.424 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:58 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:58.997+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-09-10T10:06:10.349977+0000 front 2024-09-10T10:06:10.350027+0000 (oldest deadline 2024-09-10T10:06:33.750499+0000) 2024-09-10T10:06:59.425 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:58 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:58.997+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-10T10:06:18.352105+0000 front 2024-09-10T10:06:18.352025+0000 (oldest deadline 2024-09-10T10:06:44.251983+0000) 2024-09-10T10:06:59.425 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:06:58 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10[59286]: 2024-09-10T10:06:58.997+0000 7f758b1fc640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.77:6822 osd.6 since back 2024-09-10T10:06:24.252782+0000 front 2024-09-10T10:06:24.252834+0000 (oldest deadline 2024-09-10T10:06:49.552548+0000) 2024-09-10T10:07:00.131 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:59 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:59.858+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:34.146460+0000 front 2024-09-10T10:05:34.146398+0000 (oldest deadline 2024-09-10T10:05:59.446520+0000) 2024-09-10T10:07:00.131 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:59 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:59.858+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:53.949416+0000 front 2024-09-10T10:05:53.949272+0000 (oldest deadline 2024-09-10T10:06:19.749872+0000) 2024-09-10T10:07:00.132 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:59 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:59.858+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:06:00.850855+0000 front 2024-09-10T10:06:00.850768+0000 (oldest deadline 2024-09-10T10:06:25.550818+0000) 2024-09-10T10:07:00.132 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:59 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:59.858+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-09-10T10:06:09.552349+0000 front 2024-09-10T10:06:09.552369+0000 (oldest deadline 2024-09-10T10:06:34.252329+0000) 2024-09-10T10:07:00.132 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:59 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:59.858+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-10T10:06:14.252778+0000 front 2024-09-10T10:06:14.252825+0000 (oldest deadline 2024-09-10T10:06:40.152981+0000) 2024-09-10T10:07:00.132 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:59 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:59.858+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6822 osd.6 since back 2024-09-10T10:06:25.454162+0000 front 2024-09-10T10:06:25.454072+0000 (oldest deadline 2024-09-10T10:06:50.153921+0000) 2024-09-10T10:07:00.132 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:06:59 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:06:59.858+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.120:6820 osd.7 since back 2024-09-10T10:06:35.355082+0000 front 2024-09-10T10:06:35.355177+0000 (oldest deadline 2024-09-10T10:06:58.255037+0000) 2024-09-10T10:07:00.305 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:07:00 smithi120 podman[70359]: 2024-09-10 10:07:00.035236293 +0000 UTC m=+5.507925064 container died fc2b0b00e00137bc48c8938c4607679164b1f05dee2c6e67e3ba5a997d69070e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-10T10:07:00.663 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:07:00 smithi120 podman[70359]: 2024-09-10 10:07:00.394271709 +0000 UTC m=+5.866960478 container cleanup fc2b0b00e00137bc48c8938c4607679164b1f05dee2c6e67e3ba5a997d69070e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T10:07:00.663 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:07:00 smithi120 bash[70359]: ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10 2024-09-10T10:07:00.923 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:07:00 smithi120 podman[70374]: 2024-09-10 10:07:00.66249463 +0000 UTC m=+0.618329388 container remove fc2b0b00e00137bc48c8938c4607679164b1f05dee2c6e67e3ba5a997d69070e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T10:07:01.131 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:07:00 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:07:00.838+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:34.146460+0000 front 2024-09-10T10:05:34.146398+0000 (oldest deadline 2024-09-10T10:05:59.446520+0000) 2024-09-10T10:07:01.131 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:07:00 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:07:00.838+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:53.949416+0000 front 2024-09-10T10:05:53.949272+0000 (oldest deadline 2024-09-10T10:06:19.749872+0000) 2024-09-10T10:07:01.131 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:07:00 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:07:00.838+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:06:00.850855+0000 front 2024-09-10T10:06:00.850768+0000 (oldest deadline 2024-09-10T10:06:25.550818+0000) 2024-09-10T10:07:01.131 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:07:00 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:07:00.838+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-09-10T10:06:09.552349+0000 front 2024-09-10T10:06:09.552369+0000 (oldest deadline 2024-09-10T10:06:34.252329+0000) 2024-09-10T10:07:01.131 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:07:00 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:07:00.838+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-10T10:06:14.252778+0000 front 2024-09-10T10:06:14.252825+0000 (oldest deadline 2024-09-10T10:06:40.152981+0000) 2024-09-10T10:07:01.132 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:07:00 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:07:00.838+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6822 osd.6 since back 2024-09-10T10:06:25.454162+0000 front 2024-09-10T10:06:25.454072+0000 (oldest deadline 2024-09-10T10:06:50.153921+0000) 2024-09-10T10:07:01.132 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:07:00 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:07:00.838+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.120:6820 osd.7 since back 2024-09-10T10:06:35.355082+0000 front 2024-09-10T10:06:35.355177+0000 (oldest deadline 2024-09-10T10:06:58.255037+0000) 2024-09-10T10:07:01.324 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:07:01 smithi120 podman[70460]: 2024-09-10 10:07:00.911552488 +0000 UTC m=+0.016032006 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T10:07:01.324 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:07:01 smithi120 podman[70460]: 2024-09-10 10:07:01.02990311 +0000 UTC m=+0.134382617 container create a753b16b361789e54251857b64b1394d115c85c0fc16d89291d6bac63fe67607 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731) 2024-09-10T10:07:01.324 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:07:01 smithi120 podman[70460]: 2024-09-10 10:07:01.240492548 +0000 UTC m=+0.344972078 container init a753b16b361789e54251857b64b1394d115c85c0fc16d89291d6bac63fe67607 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-10T10:07:01.324 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:07:01 smithi120 podman[70460]: 2024-09-10 10:07:01.243782988 +0000 UTC m=+0.348262496 container start a753b16b361789e54251857b64b1394d115c85c0fc16d89291d6bac63fe67607 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T10:07:01.324 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:07:01 smithi120 podman[70460]: 2024-09-10 10:07:01.323719226 +0000 UTC m=+0.428198743 container attach a753b16b361789e54251857b64b1394d115c85c0fc16d89291d6bac63fe67607 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T10:07:01.674 INFO:journalctl@ceph.osd.10.smithi120.stdout:Sep 10 10:07:01 smithi120 podman[70460]: 2024-09-10 10:07:01.429491204 +0000 UTC m=+0.533970713 container died a753b16b361789e54251857b64b1394d115c85c0fc16d89291d6bac63fe67607 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-10-deactivate, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-10T10:07:02.000 DEBUG:teuthology.orchestra.run.smithi120:> sudo pkill -f 'journalctl -f -n 0 -u ceph-14a55226-6f59-11ef-bcea-c7b262605968@osd.10.service' 2024-09-10T10:07:02.078 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:07:02.079 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-09-10T10:07:02.079 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-09-10T10:07:02.079 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-14a55226-6f59-11ef-bcea-c7b262605968@osd.11 2024-09-10T10:07:02.107 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:07:01 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:07:01.803+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:34.146460+0000 front 2024-09-10T10:05:34.146398+0000 (oldest deadline 2024-09-10T10:05:59.446520+0000) 2024-09-10T10:07:02.108 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:07:01 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:07:01.803+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:53.949416+0000 front 2024-09-10T10:05:53.949272+0000 (oldest deadline 2024-09-10T10:06:19.749872+0000) 2024-09-10T10:07:02.108 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:07:01 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:07:01.803+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:06:00.850855+0000 front 2024-09-10T10:06:00.850768+0000 (oldest deadline 2024-09-10T10:06:25.550818+0000) 2024-09-10T10:07:02.108 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:07:01 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:07:01.803+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-09-10T10:06:09.552349+0000 front 2024-09-10T10:06:09.552369+0000 (oldest deadline 2024-09-10T10:06:34.252329+0000) 2024-09-10T10:07:02.108 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:07:01 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:07:01.803+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-10T10:06:14.252778+0000 front 2024-09-10T10:06:14.252825+0000 (oldest deadline 2024-09-10T10:06:40.152981+0000) 2024-09-10T10:07:02.108 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:07:01 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:07:01.803+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6822 osd.6 since back 2024-09-10T10:06:25.454162+0000 front 2024-09-10T10:06:25.454072+0000 (oldest deadline 2024-09-10T10:06:50.153921+0000) 2024-09-10T10:07:02.108 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:07:01 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:07:01.803+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.120:6820 osd.7 since back 2024-09-10T10:06:35.355082+0000 front 2024-09-10T10:06:35.355177+0000 (oldest deadline 2024-09-10T10:06:58.255037+0000) 2024-09-10T10:07:02.381 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:07:02 smithi134 systemd[1]: Stopping Ceph osd.11 for 14a55226-6f59-11ef-bcea-c7b262605968... 2024-09-10T10:07:02.773 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:07:02 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:07:02.399+0000 7f5bfe969640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:07:02.773 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:07:02 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:07:02.399+0000 7f5bfe969640 -1 osd.11 100 *** Got signal Terminated *** 2024-09-10T10:07:02.773 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:07:02 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:07:02.399+0000 7f5bfe969640 -1 osd.11 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T10:07:02.773 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:07:02 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:07:02.772+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:34.146460+0000 front 2024-09-10T10:05:34.146398+0000 (oldest deadline 2024-09-10T10:05:59.446520+0000) 2024-09-10T10:07:02.773 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:07:02 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:07:02.772+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:53.949416+0000 front 2024-09-10T10:05:53.949272+0000 (oldest deadline 2024-09-10T10:06:19.749872+0000) 2024-09-10T10:07:02.774 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:07:02 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:07:02.772+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:06:00.850855+0000 front 2024-09-10T10:06:00.850768+0000 (oldest deadline 2024-09-10T10:06:25.550818+0000) 2024-09-10T10:07:03.131 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:07:02 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:07:02.772+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-09-10T10:06:09.552349+0000 front 2024-09-10T10:06:09.552369+0000 (oldest deadline 2024-09-10T10:06:34.252329+0000) 2024-09-10T10:07:03.131 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:07:02 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:07:02.772+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-10T10:06:14.252778+0000 front 2024-09-10T10:06:14.252825+0000 (oldest deadline 2024-09-10T10:06:40.152981+0000) 2024-09-10T10:07:03.131 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:07:02 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:07:02.772+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6822 osd.6 since back 2024-09-10T10:06:25.454162+0000 front 2024-09-10T10:06:25.454072+0000 (oldest deadline 2024-09-10T10:06:50.153921+0000) 2024-09-10T10:07:03.131 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:07:02 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:07:02.772+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.120:6820 osd.7 since back 2024-09-10T10:06:35.355082+0000 front 2024-09-10T10:06:35.355177+0000 (oldest deadline 2024-09-10T10:06:58.255037+0000) 2024-09-10T10:07:04.131 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:07:03 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:07:03.797+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:34.146460+0000 front 2024-09-10T10:05:34.146398+0000 (oldest deadline 2024-09-10T10:05:59.446520+0000) 2024-09-10T10:07:04.131 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:07:03 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:07:03.797+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:53.949416+0000 front 2024-09-10T10:05:53.949272+0000 (oldest deadline 2024-09-10T10:06:19.749872+0000) 2024-09-10T10:07:04.131 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:07:03 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:07:03.797+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:06:00.850855+0000 front 2024-09-10T10:06:00.850768+0000 (oldest deadline 2024-09-10T10:06:25.550818+0000) 2024-09-10T10:07:04.131 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:07:03 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:07:03.797+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-09-10T10:06:09.552349+0000 front 2024-09-10T10:06:09.552369+0000 (oldest deadline 2024-09-10T10:06:34.252329+0000) 2024-09-10T10:07:04.131 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:07:03 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:07:03.797+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-10T10:06:14.252778+0000 front 2024-09-10T10:06:14.252825+0000 (oldest deadline 2024-09-10T10:06:40.152981+0000) 2024-09-10T10:07:04.132 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:07:03 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:07:03.797+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6822 osd.6 since back 2024-09-10T10:06:25.454162+0000 front 2024-09-10T10:06:25.454072+0000 (oldest deadline 2024-09-10T10:06:50.153921+0000) 2024-09-10T10:07:04.132 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:07:03 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:07:03.797+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.120:6820 osd.7 since back 2024-09-10T10:06:35.355082+0000 front 2024-09-10T10:06:35.355177+0000 (oldest deadline 2024-09-10T10:06:58.255037+0000) 2024-09-10T10:07:05.131 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:07:04 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:07:04.788+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:34.146460+0000 front 2024-09-10T10:05:34.146398+0000 (oldest deadline 2024-09-10T10:05:59.446520+0000) 2024-09-10T10:07:05.131 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:07:04 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:07:04.788+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:53.949416+0000 front 2024-09-10T10:05:53.949272+0000 (oldest deadline 2024-09-10T10:06:19.749872+0000) 2024-09-10T10:07:05.131 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:07:04 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:07:04.788+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:06:00.850855+0000 front 2024-09-10T10:06:00.850768+0000 (oldest deadline 2024-09-10T10:06:25.550818+0000) 2024-09-10T10:07:05.132 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:07:04 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:07:04.788+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-09-10T10:06:09.552349+0000 front 2024-09-10T10:06:09.552369+0000 (oldest deadline 2024-09-10T10:06:34.252329+0000) 2024-09-10T10:07:05.132 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:07:04 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:07:04.788+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-10T10:06:14.252778+0000 front 2024-09-10T10:06:14.252825+0000 (oldest deadline 2024-09-10T10:06:40.152981+0000) 2024-09-10T10:07:05.132 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:07:04 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:07:04.788+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6822 osd.6 since back 2024-09-10T10:06:25.454162+0000 front 2024-09-10T10:06:25.454072+0000 (oldest deadline 2024-09-10T10:06:50.153921+0000) 2024-09-10T10:07:05.132 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:07:04 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:07:04.788+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.120:6820 osd.7 since back 2024-09-10T10:06:35.355082+0000 front 2024-09-10T10:06:35.355177+0000 (oldest deadline 2024-09-10T10:06:58.255037+0000) 2024-09-10T10:07:06.131 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:07:05 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:07:05.745+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:34.146460+0000 front 2024-09-10T10:05:34.146398+0000 (oldest deadline 2024-09-10T10:05:59.446520+0000) 2024-09-10T10:07:06.131 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:07:05 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:07:05.745+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:53.949416+0000 front 2024-09-10T10:05:53.949272+0000 (oldest deadline 2024-09-10T10:06:19.749872+0000) 2024-09-10T10:07:06.131 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:07:05 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:07:05.745+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:06:00.850855+0000 front 2024-09-10T10:06:00.850768+0000 (oldest deadline 2024-09-10T10:06:25.550818+0000) 2024-09-10T10:07:06.131 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:07:05 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:07:05.745+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-09-10T10:06:09.552349+0000 front 2024-09-10T10:06:09.552369+0000 (oldest deadline 2024-09-10T10:06:34.252329+0000) 2024-09-10T10:07:06.132 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:07:05 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:07:05.745+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-10T10:06:14.252778+0000 front 2024-09-10T10:06:14.252825+0000 (oldest deadline 2024-09-10T10:06:40.152981+0000) 2024-09-10T10:07:06.132 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:07:05 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:07:05.745+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6822 osd.6 since back 2024-09-10T10:06:25.454162+0000 front 2024-09-10T10:06:25.454072+0000 (oldest deadline 2024-09-10T10:06:50.153921+0000) 2024-09-10T10:07:06.132 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:07:05 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:07:05.745+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.120:6820 osd.7 since back 2024-09-10T10:06:35.355082+0000 front 2024-09-10T10:06:35.355177+0000 (oldest deadline 2024-09-10T10:06:58.255037+0000) 2024-09-10T10:07:06.132 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:07:05 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:07:05.745+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.134:6820 osd.8 since back 2024-09-10T10:06:39.955747+0000 front 2024-09-10T10:06:39.955698+0000 (oldest deadline 2024-09-10T10:07:05.255741+0000) 2024-09-10T10:07:07.131 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:07:06 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:07:06.736+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-10T10:05:34.146460+0000 front 2024-09-10T10:05:34.146398+0000 (oldest deadline 2024-09-10T10:05:59.446520+0000) 2024-09-10T10:07:07.132 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:07:06 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:07:06.736+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-10T10:05:53.949416+0000 front 2024-09-10T10:05:53.949272+0000 (oldest deadline 2024-09-10T10:06:19.749872+0000) 2024-09-10T10:07:07.132 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:07:06 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:07:06.736+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-10T10:06:00.850855+0000 front 2024-09-10T10:06:00.850768+0000 (oldest deadline 2024-09-10T10:06:25.550818+0000) 2024-09-10T10:07:07.132 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:07:06 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:07:06.736+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-09-10T10:06:09.552349+0000 front 2024-09-10T10:06:09.552369+0000 (oldest deadline 2024-09-10T10:06:34.252329+0000) 2024-09-10T10:07:07.132 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:07:06 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:07:06.736+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-10T10:06:14.252778+0000 front 2024-09-10T10:06:14.252825+0000 (oldest deadline 2024-09-10T10:06:40.152981+0000) 2024-09-10T10:07:07.132 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:07:06 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:07:06.736+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.77:6822 osd.6 since back 2024-09-10T10:06:25.454162+0000 front 2024-09-10T10:06:25.454072+0000 (oldest deadline 2024-09-10T10:06:50.153921+0000) 2024-09-10T10:07:07.132 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:07:06 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:07:06.736+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.120:6820 osd.7 since back 2024-09-10T10:06:35.355082+0000 front 2024-09-10T10:06:35.355177+0000 (oldest deadline 2024-09-10T10:06:58.255037+0000) 2024-09-10T10:07:07.132 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:07:06 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11[61515]: 2024-09-10T10:07:06.736+0000 7f5bfaf7f640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.134:6820 osd.8 since back 2024-09-10T10:06:39.955747+0000 front 2024-09-10T10:06:39.955698+0000 (oldest deadline 2024-09-10T10:07:05.255741+0000) 2024-09-10T10:07:08.085 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:07:07 smithi134 podman[72258]: 2024-09-10 10:07:07.808177538 +0000 UTC m=+5.531492039 container died 3288454b05148c919ca971f7b0ae30a87afb71c3113cd26102e5728ea822827a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T10:07:08.381 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:07:08 smithi134 podman[72258]: 2024-09-10 10:07:08.166418047 +0000 UTC m=+5.889732549 container cleanup 3288454b05148c919ca971f7b0ae30a87afb71c3113cd26102e5728ea822827a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-10T10:07:08.381 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:07:08 smithi134 bash[72258]: ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11 2024-09-10T10:07:08.782 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:07:08 smithi134 podman[72275]: 2024-09-10 10:07:08.442531937 +0000 UTC m=+0.630614417 container remove 3288454b05148c919ca971f7b0ae30a87afb71c3113cd26102e5728ea822827a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-10T10:07:09.085 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:07:08 smithi134 podman[72362]: 2024-09-10 10:07:08.684406708 +0000 UTC m=+0.015854579 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T10:07:09.085 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:07:08 smithi134 podman[72362]: 2024-09-10 10:07:08.784760041 +0000 UTC m=+0.116207899 container create 31068b8207ce77365c27d20d50c561d502ac8936a3416722ab7926b9981eafc9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731) 2024-09-10T10:07:09.085 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:07:08 smithi134 podman[72362]: 2024-09-10 10:07:08.995494989 +0000 UTC m=+0.326942846 container init 31068b8207ce77365c27d20d50c561d502ac8936a3416722ab7926b9981eafc9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T10:07:09.085 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:07:08 smithi134 podman[72362]: 2024-09-10 10:07:08.999050886 +0000 UTC m=+0.330498743 container start 31068b8207ce77365c27d20d50c561d502ac8936a3416722ab7926b9981eafc9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11-deactivate, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-10T10:07:09.085 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:07:09 smithi134 podman[72362]: 2024-09-10 10:07:09.084206864 +0000 UTC m=+0.415654735 container attach 31068b8207ce77365c27d20d50c561d502ac8936a3416722ab7926b9981eafc9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11-deactivate, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD) 2024-09-10T10:07:09.381 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 10 10:07:09 smithi134 podman[72362]: 2024-09-10 10:07:09.188715099 +0000 UTC m=+0.520162966 container died 31068b8207ce77365c27d20d50c561d502ac8936a3416722ab7926b9981eafc9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-osd-11-deactivate, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True) 2024-09-10T10:07:09.806 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-14a55226-6f59-11ef-bcea-c7b262605968@osd.11.service' 2024-09-10T10:07:09.884 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:07:09.884 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-09-10T10:07:09.885 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-09-10T10:07:09.885 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ceph-14a55226-6f59-11ef-bcea-c7b262605968@mds.a 2024-09-10T10:07:10.189 INFO:journalctl@ceph.mds.a.smithi077.stdout:Sep 10 10:07:09 smithi077 systemd[1]: Stopping Ceph mds.a for 14a55226-6f59-11ef-bcea-c7b262605968... 2024-09-10T10:07:10.689 INFO:journalctl@ceph.mds.a.smithi077.stdout:Sep 10 10:07:10 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-a[70880]: 2024-09-10T10:07:10.336+0000 7fc47e800640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:07:10.689 INFO:journalctl@ceph.mds.a.smithi077.stdout:Sep 10 10:07:10 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-a[70880]: 2024-09-10T10:07:10.336+0000 7fc47e800640 -1 mds.a *** got signal Terminated *** 2024-09-10T10:07:14.370 INFO:journalctl@ceph.mds.a.smithi077.stdout:Sep 10 10:07:14 smithi077 podman[128530]: 2024-09-10 10:07:14.046583079 +0000 UTC m=+3.870602871 container died 465fb4b642d38bf0e9dc99e652b20257d33bb7e915b79a1bb81ac473cf2126cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-a, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True) 2024-09-10T10:07:14.689 INFO:journalctl@ceph.mds.a.smithi077.stdout:Sep 10 10:07:14 smithi077 podman[128530]: 2024-09-10 10:07:14.550817235 +0000 UTC m=+4.374837027 container cleanup 465fb4b642d38bf0e9dc99e652b20257d33bb7e915b79a1bb81ac473cf2126cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-a, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T10:07:14.689 INFO:journalctl@ceph.mds.a.smithi077.stdout:Sep 10 10:07:14 smithi077 bash[128530]: ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-a 2024-09-10T10:07:14.990 INFO:journalctl@ceph.mds.a.smithi077.stdout:Sep 10 10:07:14 smithi077 podman[128554]: 2024-09-10 10:07:14.735608401 +0000 UTC m=+0.675867277 container remove 465fb4b642d38bf0e9dc99e652b20257d33bb7e915b79a1bb81ac473cf2126cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-10T10:07:14.999 DEBUG:teuthology.orchestra.run.smithi077:> sudo pkill -f 'journalctl -f -n 0 -u ceph-14a55226-6f59-11ef-bcea-c7b262605968@mds.a.service' 2024-09-10T10:07:15.074 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:07:15.074 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-09-10T10:07:15.074 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-09-10T10:07:15.074 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ceph-14a55226-6f59-11ef-bcea-c7b262605968@mds.d 2024-09-10T10:07:15.439 INFO:journalctl@ceph.mds.d.smithi077.stdout:Sep 10 10:07:15 smithi077 systemd[1]: Stopping Ceph mds.d for 14a55226-6f59-11ef-bcea-c7b262605968... 2024-09-10T10:07:15.939 INFO:journalctl@ceph.mds.d.smithi077.stdout:Sep 10 10:07:15 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-d[69746]: 2024-09-10T10:07:15.541+0000 7f106d400640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:07:15.939 INFO:journalctl@ceph.mds.d.smithi077.stdout:Sep 10 10:07:15 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-d[69746]: 2024-09-10T10:07:15.541+0000 7f106d400640 -1 mds.d *** got signal Terminated *** 2024-09-10T10:07:19.390 INFO:journalctl@ceph.mds.d.smithi077.stdout:Sep 10 10:07:19 smithi077 podman[128661]: 2024-09-10 10:07:19.111301096 +0000 UTC m=+3.719722673 container died 3cf98e5ddefe69a9845bf0b41f0a16bad998e948b4adda79635a9cb4f1add85e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-d, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-10T10:07:19.689 INFO:journalctl@ceph.mds.d.smithi077.stdout:Sep 10 10:07:19 smithi077 podman[128661]: 2024-09-10 10:07:19.496625471 +0000 UTC m=+4.105047041 container cleanup 3cf98e5ddefe69a9845bf0b41f0a16bad998e948b4adda79635a9cb4f1add85e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-d, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T10:07:19.689 INFO:journalctl@ceph.mds.d.smithi077.stdout:Sep 10 10:07:19 smithi077 bash[128661]: ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-d 2024-09-10T10:07:19.977 DEBUG:teuthology.orchestra.run.smithi077:> sudo pkill -f 'journalctl -f -n 0 -u ceph-14a55226-6f59-11ef-bcea-c7b262605968@mds.d.service' 2024-09-10T10:07:20.025 INFO:journalctl@ceph.mds.d.smithi077.stdout:Sep 10 10:07:19 smithi077 podman[128673]: 2024-09-10 10:07:19.773105597 +0000 UTC m=+0.646946846 container remove 3cf98e5ddefe69a9845bf0b41f0a16bad998e948b4adda79635a9cb4f1add85e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-d, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-10T10:07:20.025 INFO:journalctl@ceph.mds.d.smithi077.stdout:Sep 10 10:07:19 smithi077 systemd[1]: ceph-14a55226-6f59-11ef-bcea-c7b262605968@mds.d.service: Deactivated successfully. 2024-09-10T10:07:20.025 INFO:journalctl@ceph.mds.d.smithi077.stdout:Sep 10 10:07:19 smithi077 systemd[1]: Stopped Ceph mds.d for 14a55226-6f59-11ef-bcea-c7b262605968. 2024-09-10T10:07:20.025 INFO:journalctl@ceph.mds.d.smithi077.stdout:Sep 10 10:07:19 smithi077 systemd[1]: ceph-14a55226-6f59-11ef-bcea-c7b262605968@mds.d.service: Consumed 1.585s CPU time. 2024-09-10T10:07:20.047 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:07:20.047 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-09-10T10:07:20.047 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-09-10T10:07:20.047 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ceph-14a55226-6f59-11ef-bcea-c7b262605968@mds.g 2024-09-10T10:07:20.439 INFO:journalctl@ceph.mds.g.smithi077.stdout:Sep 10 10:07:20 smithi077 systemd[1]: Stopping Ceph mds.g for 14a55226-6f59-11ef-bcea-c7b262605968... 2024-09-10T10:07:20.939 INFO:journalctl@ceph.mds.g.smithi077.stdout:Sep 10 10:07:20 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-g[70511]: 2024-09-10T10:07:20.444+0000 7f800f800640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:07:20.939 INFO:journalctl@ceph.mds.g.smithi077.stdout:Sep 10 10:07:20 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-g[70511]: 2024-09-10T10:07:20.444+0000 7f800f800640 -1 mds.g *** got signal Terminated *** 2024-09-10T10:07:22.318 INFO:journalctl@ceph.mds.g.smithi077.stdout:Sep 10 10:07:22 smithi077 podman[128780]: 2024-09-10 10:07:22.046007443 +0000 UTC m=+1.732354265 container died 289cc8276997f59b4128aca7bd0d7dade1ef669fb28373c8bc7f212e388de7ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-g, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-10T10:07:22.689 INFO:journalctl@ceph.mds.g.smithi077.stdout:Sep 10 10:07:22 smithi077 podman[128780]: 2024-09-10 10:07:22.525545283 +0000 UTC m=+2.211892106 container cleanup 289cc8276997f59b4128aca7bd0d7dade1ef669fb28373c8bc7f212e388de7ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-g, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True) 2024-09-10T10:07:22.689 INFO:journalctl@ceph.mds.g.smithi077.stdout:Sep 10 10:07:22 smithi077 bash[128780]: ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-g 2024-09-10T10:07:23.041 INFO:journalctl@ceph.mds.g.smithi077.stdout:Sep 10 10:07:22 smithi077 podman[128793]: 2024-09-10 10:07:22.777261328 +0000 UTC m=+0.719400950 container remove 289cc8276997f59b4128aca7bd0d7dade1ef669fb28373c8bc7f212e388de7ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-g, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-10T10:07:23.061 DEBUG:teuthology.orchestra.run.smithi077:> sudo pkill -f 'journalctl -f -n 0 -u ceph-14a55226-6f59-11ef-bcea-c7b262605968@mds.g.service' 2024-09-10T10:07:23.130 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:07:23.130 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-09-10T10:07:23.130 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-09-10T10:07:23.131 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ceph-14a55226-6f59-11ef-bcea-c7b262605968@mds.j 2024-09-10T10:07:23.439 INFO:journalctl@ceph.mds.j.smithi077.stdout:Sep 10 10:07:23 smithi077 systemd[1]: Stopping Ceph mds.j for 14a55226-6f59-11ef-bcea-c7b262605968... 2024-09-10T10:07:23.941 INFO:journalctl@ceph.mds.j.smithi077.stdout:Sep 10 10:07:23 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-j[69324]: 2024-09-10T10:07:23.549+0000 7f10aa200640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:07:23.941 INFO:journalctl@ceph.mds.j.smithi077.stdout:Sep 10 10:07:23 smithi077 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-j[69324]: 2024-09-10T10:07:23.549+0000 7f10aa200640 -1 mds.j *** got signal Terminated *** 2024-09-10T10:07:27.091 INFO:journalctl@ceph.mds.j.smithi077.stdout:Sep 10 10:07:26 smithi077 podman[128900]: 2024-09-10 10:07:26.794932069 +0000 UTC m=+3.398283069 container died 3646333f4cb18cad1b3cb6d41960e086f202513768325b8e01c53f423a16f6dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-j, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T10:07:27.439 INFO:journalctl@ceph.mds.j.smithi077.stdout:Sep 10 10:07:27 smithi077 podman[128900]: 2024-09-10 10:07:27.204909968 +0000 UTC m=+3.808260973 container cleanup 3646333f4cb18cad1b3cb6d41960e086f202513768325b8e01c53f423a16f6dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-j, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2) 2024-09-10T10:07:27.439 INFO:journalctl@ceph.mds.j.smithi077.stdout:Sep 10 10:07:27 smithi077 bash[128900]: ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-j 2024-09-10T10:07:27.704 INFO:journalctl@ceph.mds.j.smithi077.stdout:Sep 10 10:07:27 smithi077 podman[128924]: 2024-09-10 10:07:27.439288352 +0000 UTC m=+0.636602254 container remove 3646333f4cb18cad1b3cb6d41960e086f202513768325b8e01c53f423a16f6dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T10:07:27.713 DEBUG:teuthology.orchestra.run.smithi077:> sudo pkill -f 'journalctl -f -n 0 -u ceph-14a55226-6f59-11ef-bcea-c7b262605968@mds.j.service' 2024-09-10T10:07:27.783 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:07:27.783 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-09-10T10:07:27.784 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-09-10T10:07:27.784 DEBUG:teuthology.orchestra.run.smithi120:> sudo systemctl stop ceph-14a55226-6f59-11ef-bcea-c7b262605968@mds.b 2024-09-10T10:07:28.120 INFO:journalctl@ceph.mds.b.smithi120.stdout:Sep 10 10:07:27 smithi120 systemd[1]: Stopping Ceph mds.b for 14a55226-6f59-11ef-bcea-c7b262605968... 2024-09-10T10:07:28.374 INFO:journalctl@ceph.mds.b.smithi120.stdout:Sep 10 10:07:28 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-b[64932]: 2024-09-10T10:07:28.118+0000 7f9f5c864640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:07:28.374 INFO:journalctl@ceph.mds.b.smithi120.stdout:Sep 10 10:07:28 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-b[64932]: 2024-09-10T10:07:28.118+0000 7f9f5c864640 -1 mds.b *** got signal Terminated *** 2024-09-10T10:07:30.424 INFO:journalctl@ceph.mds.b.smithi120.stdout:Sep 10 10:07:30 smithi120 podman[70563]: 2024-09-10 10:07:30.024602709 +0000 UTC m=+2.044211771 container died cb2fab542c1d156ee13a0bcf13bd9a9fb1782e52e9e21d462c8dae3f46acc487 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-b, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-10T10:07:30.754 INFO:journalctl@ceph.mds.b.smithi120.stdout:Sep 10 10:07:30 smithi120 podman[70563]: 2024-09-10 10:07:30.560370603 +0000 UTC m=+2.579979654 container cleanup cb2fab542c1d156ee13a0bcf13bd9a9fb1782e52e9e21d462c8dae3f46acc487 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T10:07:30.754 INFO:journalctl@ceph.mds.b.smithi120.stdout:Sep 10 10:07:30 smithi120 bash[70563]: ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-b 2024-09-10T10:07:30.992 DEBUG:teuthology.orchestra.run.smithi120:> sudo pkill -f 'journalctl -f -n 0 -u ceph-14a55226-6f59-11ef-bcea-c7b262605968@mds.b.service' 2024-09-10T10:07:31.014 INFO:journalctl@ceph.mds.b.smithi120.stdout:Sep 10 10:07:30 smithi120 podman[70589]: 2024-09-10 10:07:30.753043016 +0000 UTC m=+0.726389497 container remove cb2fab542c1d156ee13a0bcf13bd9a9fb1782e52e9e21d462c8dae3f46acc487 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-b, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-10T10:07:31.014 INFO:journalctl@ceph.mds.b.smithi120.stdout:Sep 10 10:07:30 smithi120 systemd[1]: ceph-14a55226-6f59-11ef-bcea-c7b262605968@mds.b.service: Deactivated successfully. 2024-09-10T10:07:31.014 INFO:journalctl@ceph.mds.b.smithi120.stdout:Sep 10 10:07:30 smithi120 systemd[1]: Stopped Ceph mds.b for 14a55226-6f59-11ef-bcea-c7b262605968. 2024-09-10T10:07:31.014 INFO:journalctl@ceph.mds.b.smithi120.stdout:Sep 10 10:07:30 smithi120 systemd[1]: ceph-14a55226-6f59-11ef-bcea-c7b262605968@mds.b.service: Consumed 1min 21.521s CPU time. 2024-09-10T10:07:31.028 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:07:31.028 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-09-10T10:07:31.028 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-09-10T10:07:31.028 DEBUG:teuthology.orchestra.run.smithi120:> sudo systemctl stop ceph-14a55226-6f59-11ef-bcea-c7b262605968@mds.e 2024-09-10T10:07:31.348 INFO:journalctl@ceph.mds.e.smithi120.stdout:Sep 10 10:07:31 smithi120 systemd[1]: Stopping Ceph mds.e for 14a55226-6f59-11ef-bcea-c7b262605968... 2024-09-10T10:07:31.673 INFO:journalctl@ceph.mds.e.smithi120.stdout:Sep 10 10:07:31 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-e[64330]: 2024-09-10T10:07:31.346+0000 7efd98c6c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:07:31.674 INFO:journalctl@ceph.mds.e.smithi120.stdout:Sep 10 10:07:31 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-e[64330]: 2024-09-10T10:07:31.346+0000 7efd98c6c640 -1 mds.e *** got signal Terminated *** 2024-09-10T10:07:34.289 INFO:journalctl@ceph.mds.e.smithi120.stdout:Sep 10 10:07:34 smithi120 podman[70696]: 2024-09-10 10:07:34.013342419 +0000 UTC m=+2.769771207 container died 4041cb8ecc89493653c1ff68c88b19ee6003d773aff2273e897afab7369b59ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-e, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-10T10:07:34.631 INFO:journalctl@ceph.mds.e.smithi120.stdout:Sep 10 10:07:34 smithi120 podman[70696]: 2024-09-10 10:07:34.380165473 +0000 UTC m=+3.136594303 container cleanup 4041cb8ecc89493653c1ff68c88b19ee6003d773aff2273e897afab7369b59ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-e, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T10:07:34.632 INFO:journalctl@ceph.mds.e.smithi120.stdout:Sep 10 10:07:34 smithi120 bash[70696]: ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-e 2024-09-10T10:07:34.773 DEBUG:teuthology.orchestra.run.smithi120:> sudo pkill -f 'journalctl -f -n 0 -u ceph-14a55226-6f59-11ef-bcea-c7b262605968@mds.e.service' 2024-09-10T10:07:34.806 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:07:34.806 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-09-10T10:07:34.806 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-09-10T10:07:34.806 DEBUG:teuthology.orchestra.run.smithi120:> sudo systemctl stop ceph-14a55226-6f59-11ef-bcea-c7b262605968@mds.h 2024-09-10T10:07:35.174 INFO:journalctl@ceph.mds.h.smithi120.stdout:Sep 10 10:07:34 smithi120 systemd[1]: Stopping Ceph mds.h for 14a55226-6f59-11ef-bcea-c7b262605968... 2024-09-10T10:07:35.174 INFO:journalctl@ceph.mds.h.smithi120.stdout:Sep 10 10:07:35 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-h[64633]: 2024-09-10T10:07:35.125+0000 7fc135f60640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:07:35.174 INFO:journalctl@ceph.mds.h.smithi120.stdout:Sep 10 10:07:35 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-h[64633]: 2024-09-10T10:07:35.125+0000 7fc135f60640 -1 mds.h *** got signal Terminated *** 2024-09-10T10:07:38.360 INFO:journalctl@ceph.mds.h.smithi120.stdout:Sep 10 10:07:38 smithi120 podman[70816]: 2024-09-10 10:07:38.028538773 +0000 UTC m=+3.009175724 container died bfdb151a8d5157d138ac1536508a71cd6354770553ca2e908fced2a3fdc048a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-h, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T10:07:38.660 INFO:journalctl@ceph.mds.h.smithi120.stdout:Sep 10 10:07:38 smithi120 podman[70816]: 2024-09-10 10:07:38.458861539 +0000 UTC m=+3.439498477 container cleanup bfdb151a8d5157d138ac1536508a71cd6354770553ca2e908fced2a3fdc048a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-h, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-10T10:07:38.661 INFO:journalctl@ceph.mds.h.smithi120.stdout:Sep 10 10:07:38 smithi120 bash[70816]: ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-h 2024-09-10T10:07:38.898 DEBUG:teuthology.orchestra.run.smithi120:> sudo pkill -f 'journalctl -f -n 0 -u ceph-14a55226-6f59-11ef-bcea-c7b262605968@mds.h.service' 2024-09-10T10:07:38.919 INFO:journalctl@ceph.mds.h.smithi120.stdout:Sep 10 10:07:38 smithi120 podman[70828]: 2024-09-10 10:07:38.659939934 +0000 UTC m=+0.633657228 container remove bfdb151a8d5157d138ac1536508a71cd6354770553ca2e908fced2a3fdc048a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-h, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True) 2024-09-10T10:07:38.919 INFO:journalctl@ceph.mds.h.smithi120.stdout:Sep 10 10:07:38 smithi120 systemd[1]: ceph-14a55226-6f59-11ef-bcea-c7b262605968@mds.h.service: Deactivated successfully. 2024-09-10T10:07:38.919 INFO:journalctl@ceph.mds.h.smithi120.stdout:Sep 10 10:07:38 smithi120 systemd[1]: Stopped Ceph mds.h for 14a55226-6f59-11ef-bcea-c7b262605968. 2024-09-10T10:07:38.919 INFO:journalctl@ceph.mds.h.smithi120.stdout:Sep 10 10:07:38 smithi120 systemd[1]: ceph-14a55226-6f59-11ef-bcea-c7b262605968@mds.h.service: Consumed 15.648s CPU time. 2024-09-10T10:07:38.930 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:07:38.930 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-09-10T10:07:38.930 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-09-10T10:07:38.931 DEBUG:teuthology.orchestra.run.smithi120:> sudo systemctl stop ceph-14a55226-6f59-11ef-bcea-c7b262605968@mds.k 2024-09-10T10:07:39.175 INFO:journalctl@ceph.mds.k.smithi120.stdout:Sep 10 10:07:39 smithi120 systemd[1]: Stopping Ceph mds.k for 14a55226-6f59-11ef-bcea-c7b262605968... 2024-09-10T10:07:39.673 INFO:journalctl@ceph.mds.k.smithi120.stdout:Sep 10 10:07:39 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-k[64012]: 2024-09-10T10:07:39.328+0000 7f74857f3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:07:39.674 INFO:journalctl@ceph.mds.k.smithi120.stdout:Sep 10 10:07:39 smithi120 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-k[64012]: 2024-09-10T10:07:39.328+0000 7f74857f3640 -1 mds.k *** got signal Terminated *** 2024-09-10T10:07:42.272 INFO:journalctl@ceph.mds.k.smithi120.stdout:Sep 10 10:07:42 smithi120 podman[70936]: 2024-09-10 10:07:42.010663646 +0000 UTC m=+2.866471529 container died 4a976ae355b021cb76066d452e2b735d2242a5d69a95f336e7cf947107cc7626 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-10T10:07:42.647 INFO:journalctl@ceph.mds.k.smithi120.stdout:Sep 10 10:07:42 smithi120 podman[70936]: 2024-09-10 10:07:42.37056528 +0000 UTC m=+3.226373163 container cleanup 4a976ae355b021cb76066d452e2b735d2242a5d69a95f336e7cf947107cc7626 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-k, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T10:07:42.647 INFO:journalctl@ceph.mds.k.smithi120.stdout:Sep 10 10:07:42 smithi120 bash[70936]: ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-k 2024-09-10T10:07:42.799 DEBUG:teuthology.orchestra.run.smithi120:> sudo pkill -f 'journalctl -f -n 0 -u ceph-14a55226-6f59-11ef-bcea-c7b262605968@mds.k.service' 2024-09-10T10:07:42.831 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:07:42.831 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-09-10T10:07:42.831 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-09-10T10:07:42.831 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-14a55226-6f59-11ef-bcea-c7b262605968@mds.c 2024-09-10T10:07:43.131 INFO:journalctl@ceph.mds.c.smithi134.stdout:Sep 10 10:07:42 smithi134 systemd[1]: Stopping Ceph mds.c for 14a55226-6f59-11ef-bcea-c7b262605968... 2024-09-10T10:07:43.881 INFO:journalctl@ceph.mds.c.smithi134.stdout:Sep 10 10:07:43 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-c[66875]: 2024-09-10T10:07:43.448+0000 7f0864b6b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:07:43.882 INFO:journalctl@ceph.mds.c.smithi134.stdout:Sep 10 10:07:43 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-c[66875]: 2024-09-10T10:07:43.448+0000 7f0864b6b640 -1 mds.c *** got signal Terminated *** 2024-09-10T10:07:46.266 INFO:journalctl@ceph.mds.c.smithi134.stdout:Sep 10 10:07:46 smithi134 podman[72464]: 2024-09-10 10:07:46.01383046 +0000 UTC m=+2.982860906 container died 5864e6ac715cecdacce9bff622395ca4ab8710a454503aee4714e943533ebe77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.license=GPLv2) 2024-09-10T10:07:46.552 INFO:journalctl@ceph.mds.c.smithi134.stdout:Sep 10 10:07:46 smithi134 podman[72464]: 2024-09-10 10:07:46.358738945 +0000 UTC m=+3.327769433 container cleanup 5864e6ac715cecdacce9bff622395ca4ab8710a454503aee4714e943533ebe77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-c, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T10:07:46.552 INFO:journalctl@ceph.mds.c.smithi134.stdout:Sep 10 10:07:46 smithi134 bash[72464]: ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-c 2024-09-10T10:07:46.786 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-14a55226-6f59-11ef-bcea-c7b262605968@mds.c.service' 2024-09-10T10:07:46.808 INFO:journalctl@ceph.mds.c.smithi134.stdout:Sep 10 10:07:46 smithi134 podman[72490]: 2024-09-10 10:07:46.551307986 +0000 UTC m=+0.529478825 container remove 5864e6ac715cecdacce9bff622395ca4ab8710a454503aee4714e943533ebe77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-c, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-10T10:07:46.808 INFO:journalctl@ceph.mds.c.smithi134.stdout:Sep 10 10:07:46 smithi134 systemd[1]: ceph-14a55226-6f59-11ef-bcea-c7b262605968@mds.c.service: Deactivated successfully. 2024-09-10T10:07:46.808 INFO:journalctl@ceph.mds.c.smithi134.stdout:Sep 10 10:07:46 smithi134 systemd[1]: Stopped Ceph mds.c for 14a55226-6f59-11ef-bcea-c7b262605968. 2024-09-10T10:07:46.809 INFO:journalctl@ceph.mds.c.smithi134.stdout:Sep 10 10:07:46 smithi134 systemd[1]: ceph-14a55226-6f59-11ef-bcea-c7b262605968@mds.c.service: Consumed 17.081s CPU time. 2024-09-10T10:07:46.822 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:07:46.822 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-09-10T10:07:46.823 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-09-10T10:07:46.823 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-14a55226-6f59-11ef-bcea-c7b262605968@mds.f 2024-09-10T10:07:47.131 INFO:journalctl@ceph.mds.f.smithi134.stdout:Sep 10 10:07:46 smithi134 systemd[1]: Stopping Ceph mds.f for 14a55226-6f59-11ef-bcea-c7b262605968... 2024-09-10T10:07:47.631 INFO:journalctl@ceph.mds.f.smithi134.stdout:Sep 10 10:07:47 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-f[66571]: 2024-09-10T10:07:47.137+0000 7f14b314a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:07:47.631 INFO:journalctl@ceph.mds.f.smithi134.stdout:Sep 10 10:07:47 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-f[66571]: 2024-09-10T10:07:47.137+0000 7f14b314a640 -1 mds.f *** got signal Terminated *** 2024-09-10T10:07:50.272 INFO:journalctl@ceph.mds.f.smithi134.stdout:Sep 10 10:07:50 smithi134 podman[72597]: 2024-09-10 10:07:50.017299074 +0000 UTC m=+2.978424431 container died 221d5b459a693b4cf60023f43b887461723dd8066b6eb952f70c00ba012444ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-f, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-10T10:07:50.531 INFO:journalctl@ceph.mds.f.smithi134.stdout:Sep 10 10:07:50 smithi134 podman[72597]: 2024-09-10 10:07:50.362051643 +0000 UTC m=+3.323176999 container cleanup 221d5b459a693b4cf60023f43b887461723dd8066b6eb952f70c00ba012444ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-f, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T10:07:50.531 INFO:journalctl@ceph.mds.f.smithi134.stdout:Sep 10 10:07:50 smithi134 bash[72597]: ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-f 2024-09-10T10:07:50.786 INFO:journalctl@ceph.mds.f.smithi134.stdout:Sep 10 10:07:50 smithi134 podman[72610]: 2024-09-10 10:07:50.529921771 +0000 UTC m=+0.511047512 container remove 221d5b459a693b4cf60023f43b887461723dd8066b6eb952f70c00ba012444ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-f, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T10:07:50.790 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-14a55226-6f59-11ef-bcea-c7b262605968@mds.f.service' 2024-09-10T10:07:50.823 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:07:50.824 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-09-10T10:07:50.824 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-09-10T10:07:50.824 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-14a55226-6f59-11ef-bcea-c7b262605968@mds.i 2024-09-10T10:07:51.132 INFO:journalctl@ceph.mds.i.smithi134.stdout:Sep 10 10:07:50 smithi134 systemd[1]: Stopping Ceph mds.i for 14a55226-6f59-11ef-bcea-c7b262605968... 2024-09-10T10:07:51.631 INFO:journalctl@ceph.mds.i.smithi134.stdout:Sep 10 10:07:51 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-i[67177]: 2024-09-10T10:07:51.156+0000 7f03df312640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:07:51.631 INFO:journalctl@ceph.mds.i.smithi134.stdout:Sep 10 10:07:51 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-i[67177]: 2024-09-10T10:07:51.156+0000 7f03df312640 -1 mds.i *** got signal Terminated *** 2024-09-10T10:07:54.131 INFO:journalctl@ceph.mds.i.smithi134.stdout:Sep 10 10:07:54 smithi134 podman[72719]: 2024-09-10 10:07:54.012123882 +0000 UTC m=+2.969522010 container died 35a8e19d47456f286e9b6bf0c764fb254a082a8d381d58363cc581e45b94f2dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True) 2024-09-10T10:07:54.631 INFO:journalctl@ceph.mds.i.smithi134.stdout:Sep 10 10:07:54 smithi134 podman[72719]: 2024-09-10 10:07:54.382024754 +0000 UTC m=+3.339422879 container cleanup 35a8e19d47456f286e9b6bf0c764fb254a082a8d381d58363cc581e45b94f2dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-i, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2) 2024-09-10T10:07:54.631 INFO:journalctl@ceph.mds.i.smithi134.stdout:Sep 10 10:07:54 smithi134 bash[72719]: ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-i 2024-09-10T10:07:54.804 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-14a55226-6f59-11ef-bcea-c7b262605968@mds.i.service' 2024-09-10T10:07:54.836 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:07:54.837 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-09-10T10:07:54.837 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-09-10T10:07:54.837 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-14a55226-6f59-11ef-bcea-c7b262605968@mds.l 2024-09-10T10:07:55.381 INFO:journalctl@ceph.mds.l.smithi134.stdout:Sep 10 10:07:54 smithi134 systemd[1]: Stopping Ceph mds.l for 14a55226-6f59-11ef-bcea-c7b262605968... 2024-09-10T10:07:55.381 INFO:journalctl@ceph.mds.l.smithi134.stdout:Sep 10 10:07:55 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-l[66271]: 2024-09-10T10:07:55.152+0000 7f24b8909640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:07:55.381 INFO:journalctl@ceph.mds.l.smithi134.stdout:Sep 10 10:07:55 smithi134 ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-l[66271]: 2024-09-10T10:07:55.152+0000 7f24b8909640 -1 mds.l *** got signal Terminated *** 2024-09-10T10:07:58.272 INFO:journalctl@ceph.mds.l.smithi134.stdout:Sep 10 10:07:58 smithi134 podman[72839]: 2024-09-10 10:07:58.019939178 +0000 UTC m=+2.969769551 container died 91a33ae0180337dc06f8e1a40d500adc71cca72b0402b1d9e8f6f7cef04ce34a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-l, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2) 2024-09-10T10:07:58.537 INFO:journalctl@ceph.mds.l.smithi134.stdout:Sep 10 10:07:58 smithi134 podman[72839]: 2024-09-10 10:07:58.335628905 +0000 UTC m=+3.285459268 container cleanup 91a33ae0180337dc06f8e1a40d500adc71cca72b0402b1d9e8f6f7cef04ce34a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-l, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0) 2024-09-10T10:07:58.537 INFO:journalctl@ceph.mds.l.smithi134.stdout:Sep 10 10:07:58 smithi134 bash[72839]: ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-l 2024-09-10T10:07:58.764 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-14a55226-6f59-11ef-bcea-c7b262605968@mds.l.service' 2024-09-10T10:07:58.787 INFO:journalctl@ceph.mds.l.smithi134.stdout:Sep 10 10:07:58 smithi134 podman[72851]: 2024-09-10 10:07:58.536484626 +0000 UTC m=+0.517425012 container remove 91a33ae0180337dc06f8e1a40d500adc71cca72b0402b1d9e8f6f7cef04ce34a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-14a55226-6f59-11ef-bcea-c7b262605968-mds-l, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T10:07:58.787 INFO:journalctl@ceph.mds.l.smithi134.stdout:Sep 10 10:07:58 smithi134 systemd[1]: ceph-14a55226-6f59-11ef-bcea-c7b262605968@mds.l.service: Deactivated successfully. 2024-09-10T10:07:58.787 INFO:journalctl@ceph.mds.l.smithi134.stdout:Sep 10 10:07:58 smithi134 systemd[1]: Stopped Ceph mds.l for 14a55226-6f59-11ef-bcea-c7b262605968. 2024-09-10T10:07:58.787 INFO:journalctl@ceph.mds.l.smithi134.stdout:Sep 10 10:07:58 smithi134 systemd[1]: ceph-14a55226-6f59-11ef-bcea-c7b262605968@mds.l.service: Consumed 2.818s CPU time. 2024-09-10T10:07:58.796 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:07:58.797 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-09-10T10:07:58.797 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 14a55226-6f59-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-10T10:07:59.068 INFO:teuthology.orchestra.run.smithi077.stdout:Deleting cluster with fsid: 14a55226-6f59-11ef-bcea-c7b262605968 2024-09-10T10:08:03.062 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 14a55226-6f59-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-10T10:08:03.263 INFO:teuthology.orchestra.run.smithi120.stdout:Deleting cluster with fsid: 14a55226-6f59-11ef-bcea-c7b262605968 2024-09-10T10:08:05.787 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 14a55226-6f59-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-10T10:08:05.987 INFO:teuthology.orchestra.run.smithi134.stdout:Deleting cluster with fsid: 14a55226-6f59-11ef-bcea-c7b262605968 2024-09-10T10:08:08.401 DEBUG:teuthology.orchestra.run.smithi077:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T10:08:08.466 DEBUG:teuthology.orchestra.run.smithi120:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T10:08:08.499 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T10:08:08.526 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-10T10:08:08.527 DEBUG:teuthology.misc:Transferring archived files from smithi077:/var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7898997/remote/smithi077/crash 2024-09-10T10:08:08.528 DEBUG:teuthology.orchestra.run.smithi077:> sudo tar c -f - -C /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/crash -- . 2024-09-10T10:08:08.593 INFO:teuthology.orchestra.run.smithi077.stderr:tar: /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-10T10:08:08.593 INFO:teuthology.orchestra.run.smithi077.stderr:tar: Error is not recoverable: exiting now 2024-09-10T10:08:08.596 DEBUG:teuthology.misc:Transferring archived files from smithi120:/var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7898997/remote/smithi120/crash 2024-09-10T10:08:08.597 DEBUG:teuthology.orchestra.run.smithi120:> sudo tar c -f - -C /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/crash -- . 2024-09-10T10:08:08.632 INFO:teuthology.orchestra.run.smithi120.stderr:tar: /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-10T10:08:08.632 INFO:teuthology.orchestra.run.smithi120.stderr:tar: Error is not recoverable: exiting now 2024-09-10T10:08:08.634 DEBUG:teuthology.misc:Transferring archived files from smithi134:/var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7898997/remote/smithi134/crash 2024-09-10T10:08:08.635 DEBUG:teuthology.orchestra.run.smithi134:> sudo tar c -f - -C /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/crash -- . 2024-09-10T10:08:08.684 INFO:teuthology.orchestra.run.smithi134.stderr:tar: /var/lib/ceph/14a55226-6f59-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-10T10:08:08.684 INFO:teuthology.orchestra.run.smithi134.stderr:tar: Error is not recoverable: exiting now 2024-09-10T10:08:08.685 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-10T10:08:08.685 DEBUG:teuthology.orchestra.run.smithi077:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-10T10:08:08.763 INFO:tasks.cephadm:Compressing logs... 2024-09-10T10:08:08.764 DEBUG:teuthology.orchestra.run.smithi077:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-10T10:08:08.768 DEBUG:teuthology.orchestra.run.smithi120:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-10T10:08:08.770 DEBUG:teuthology.orchestra.run.smithi134:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-10T10:08:08.795 INFO:teuthology.orchestra.run.smithi134.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-10T10:08:08.795 INFO:teuthology.orchestra.run.smithi134.stderr:: No such file or directory 2024-09-10T10:08:08.795 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-10T10:08:08.795 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-mds.i.log 2024-09-10T10:08:08.796 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-mon.c.log 2024-09-10T10:08:08.798 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-10T10:08:08.798 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-osd.11.log 2024-09-10T10:08:08.798 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-mon.c.log: /var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-osd.2.log 2024-09-10T10:08:08.798 INFO:teuthology.orchestra.run.smithi134.stderr: 82.2% -- replaced with /var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-10T10:08:08.799 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-osd.5.log 2024-09-10T10:08:08.799 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5/var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-osd.2.log: --verbose -- /var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph.log 2024-09-10T10:08:08.799 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-mds.l.log 2024-09-10T10:08:08.799 INFO:teuthology.orchestra.run.smithi134.stderr: 90.4% -- replaced with /var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-10T10:08:08.800 INFO:teuthology.orchestra.run.smithi120.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-osd.10.log 2024-09-10T10:08:08.800 INFO:teuthology.orchestra.run.smithi120.stderr:: No such file or directory 2024-09-10T10:08:08.800 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-osd.1.log 2024-09-10T10:08:08.800 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-osd.7.log 2024-09-10T10:08:08.800 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-osd.10.log: /var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-10T10:08:08.802 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-mds.b.log 2024-09-10T10:08:08.802 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-mds.k.log 2024-09-10T10:08:08.803 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-osd.4.log 2024-09-10T10:08:08.803 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-mds.b.log: /var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-10T10:08:08.803 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-10T10:08:08.803 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph.log: 86.9% -- replaced with /var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-10T10:08:08.803 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-osd.8.log 2024-09-10T10:08:08.803 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-mgr.z.log 2024-09-10T10:08:08.804 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph.audit.log: 90.3% -- replaced with /var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-10T10:08:08.804 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-mgr.y.log 2024-09-10T10:08:08.804 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph.cephadm.log: 82.3% -- replaced with /var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-10T10:08:08.805 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph.log 2024-09-10T10:08:08.808 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-osd.8.log: 91.0% -- replaced with /var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-10T10:08:08.811 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-mds.c.log 2024-09-10T10:08:08.812 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-mds.h.log 2024-09-10T10:08:08.813 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph.log: 91.0% -- replaced with /var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-10T10:08:08.813 INFO:teuthology.orchestra.run.smithi120.stderr: 92.8% -- replaced with /var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-mgr.y.log.gz 2024-09-10T10:08:08.814 INFO:teuthology.orchestra.run.smithi120.stderr: 86.9% -- replaced with /var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-10T10:08:08.816 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-mon.b.log 2024-09-10T10:08:08.817 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-mds.e.log 2024-09-10T10:08:08.817 INFO:teuthology.orchestra.run.smithi077.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-10T10:08:08.817 INFO:teuthology.orchestra.run.smithi077.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-09-10T10:08:08.819 INFO:teuthology.orchestra.run.smithi077.stderr:gzip -5 --verbose -- /var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-osd.0.log 2024-09-10T10:08:08.819 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/ceph-client.admin.log: gzip -5 --verbose -- /var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-osd.9.log 2024-09-10T10:08:08.820 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-mgr.z.log: 92.9% -- replaced with /var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-mgr.z.log.gz 2024-09-10T10:08:08.820 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-mds.f.log 2024-09-10T10:08:08.821 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-10T10:08:08.821 INFO:teuthology.orchestra.run.smithi077.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-09-10T10:08:08.821 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-10T10:08:08.822 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-mds.a.log 2024-09-10T10:08:08.823 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-10T10:08:08.824 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-mon.a.log 2024-09-10T10:08:08.825 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-mds.g.log 2024-09-10T10:08:08.826 INFO:teuthology.orchestra.run.smithi077.stderr: 82.3% -- replaced with /var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-10T10:08:08.826 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-mon.a.log: 90.2% -- replaced with /var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-10T10:08:08.826 INFO:teuthology.orchestra.run.smithi077.stderr:gzip -5 --verbose -- /var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-osd.6.log 2024-09-10T10:08:08.828 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph.log 2024-09-10T10:08:08.830 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-10T10:08:08.831 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-osd.6.log: /var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-mds.d.log 2024-09-10T10:08:08.833 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-10T10:08:08.835 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-mgr.x.log 2024-09-10T10:08:08.837 INFO:teuthology.orchestra.run.smithi077.stderr:gzip 91.0% -- replaced with /var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-10T10:08:08.839 INFO:teuthology.orchestra.run.smithi077.stderr: -5 --verbose -- /var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-mds.j.log 2024-09-10T10:08:08.840 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-mgr.x.log: 86.9% -- replaced with /var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-10T10:08:08.844 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-mds.j.log: 94.8% -- replaced with /var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-mds.d.log.gz 2024-09-10T10:08:08.847 INFO:teuthology.orchestra.run.smithi077.stderr:gzip -5 --verbose -- /var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-osd.3.log 2024-09-10T10:08:08.847 INFO:teuthology.orchestra.run.smithi077.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-10T10:08:08.851 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-mds.e.log: /var/log/ceph/cephadm.log: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-10T10:08:08.854 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-mds.f.log: /var/log/ceph/cephadm.log: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-10T10:08:08.855 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-osd.3.log: /var/log/ceph/cephadm.log: 94.8% -- replaced with /var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-mds.j.log.gz 2024-09-10T10:08:08.876 INFO:teuthology.orchestra.run.smithi077.stderr: 94.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-10T10:08:09.155 INFO:teuthology.orchestra.run.smithi077.stderr: 90.4% -- replaced with /var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-mgr.x.log.gz 2024-09-10T10:08:09.162 INFO:teuthology.orchestra.run.smithi120.stderr: 94.8% -- replaced with /var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-mds.k.log.gz 2024-09-10T10:08:09.205 INFO:teuthology.orchestra.run.smithi077.stderr: 94.8% -- replaced with /var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-mds.g.log.gz 2024-09-10T10:08:09.230 INFO:teuthology.orchestra.run.smithi134.stderr: 95.1% -- replaced with /var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-mds.l.log.gz 2024-09-10T10:08:09.246 INFO:teuthology.orchestra.run.smithi077.stderr: 94.9% -- replaced with /var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-mds.a.log.gz 2024-09-10T10:08:09.433 INFO:teuthology.orchestra.run.smithi134.stderr: 91.5% -- replaced with /var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-mon.c.log.gz 2024-09-10T10:08:09.442 INFO:teuthology.orchestra.run.smithi120.stderr: 91.6% -- replaced with /var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-mon.b.log.gz 2024-09-10T10:08:10.112 INFO:teuthology.orchestra.run.smithi077.stderr: 90.3% -- replaced with /var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-mon.a.log.gz 2024-09-10T10:08:11.644 INFO:teuthology.orchestra.run.smithi120.stderr: 96.0% -- replaced with /var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-mds.e.log.gz 2024-09-10T10:08:13.413 INFO:teuthology.orchestra.run.smithi120.stderr: 95.0% -- replaced with /var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-mds.h.log.gz 2024-09-10T10:08:13.819 INFO:teuthology.orchestra.run.smithi134.stderr: 94.9% -- replaced with /var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-mds.c.log.gz 2024-09-10T10:08:15.374 INFO:teuthology.orchestra.run.smithi134.stderr: 95.0% -- replaced with /var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-mds.f.log.gz 2024-09-10T10:08:15.569 INFO:teuthology.orchestra.run.smithi134.stderr: 95.0% -- replaced with /var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-mds.i.log.gz 2024-09-10T10:08:18.491 INFO:teuthology.orchestra.run.smithi134.stderr: 91.7% -- replaced with /var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-osd.11.log.gz 2024-09-10T10:08:21.303 INFO:teuthology.orchestra.run.smithi120.stderr: 92.0% -- replaced with /var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-osd.10.log.gz 2024-09-10T10:08:21.409 INFO:teuthology.orchestra.run.smithi077.stderr: 92.0% -- replaced with /var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-osd.3.log.gz 2024-09-10T10:08:23.745 INFO:teuthology.orchestra.run.smithi077.stderr: 91.8% -- replaced with /var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-osd.6.log.gz 2024-09-10T10:08:24.338 INFO:teuthology.orchestra.run.smithi134.stderr: 91.6% -- replaced with /var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-osd.8.log.gz 2024-09-10T10:08:24.836 INFO:teuthology.orchestra.run.smithi077.stderr: 91.8% -- replaced with /var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-osd.0.log.gz 2024-09-10T10:08:26.260 INFO:teuthology.orchestra.run.smithi077.stderr: 91.7% -- replaced with /var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-osd.9.log.gz 2024-09-10T10:08:26.263 INFO:teuthology.orchestra.run.smithi077.stderr: 2024-09-10T10:08:26.264 INFO:teuthology.orchestra.run.smithi077.stderr:real 0m17.467s 2024-09-10T10:08:26.264 INFO:teuthology.orchestra.run.smithi077.stderr:user 0m58.398s 2024-09-10T10:08:26.264 INFO:teuthology.orchestra.run.smithi077.stderr:sys 0m5.032s 2024-09-10T10:08:26.460 INFO:teuthology.orchestra.run.smithi120.stderr: 91.7% -- replaced with /var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-osd.1.log.gz 2024-09-10T10:08:27.404 INFO:teuthology.orchestra.run.smithi134.stderr: 91.7% -- replaced with /var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-osd.2.log.gz 2024-09-10T10:08:31.096 INFO:teuthology.orchestra.run.smithi120.stderr: 91.6% -- replaced with /var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-osd.7.log.gz 2024-09-10T10:08:31.677 INFO:teuthology.orchestra.run.smithi120.stderr: 92.0% -- replaced with /var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-osd.4.log.gz 2024-09-10T10:08:38.992 INFO:teuthology.orchestra.run.smithi120.stderr: 93.4% -- replaced with /var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-mds.b.log.gz 2024-09-10T10:08:38.994 INFO:teuthology.orchestra.run.smithi120.stderr: 2024-09-10T10:08:38.994 INFO:teuthology.orchestra.run.smithi120.stderr:real 0m30.208s 2024-09-10T10:08:38.994 INFO:teuthology.orchestra.run.smithi120.stderr:user 1m49.978s 2024-09-10T10:08:38.994 INFO:teuthology.orchestra.run.smithi120.stderr:sys 0m3.295s 2024-09-10T10:08:40.817 INFO:teuthology.orchestra.run.smithi134.stderr: 91.4% -- replaced with /var/log/ceph/14a55226-6f59-11ef-bcea-c7b262605968/ceph-osd.5.log.gz 2024-09-10T10:08:40.938 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-10T10:08:40.938 INFO:teuthology.orchestra.run.smithi134.stderr:real 0m32.034s 2024-09-10T10:08:40.938 INFO:teuthology.orchestra.run.smithi134.stderr:user 1m31.829s 2024-09-10T10:08:40.939 INFO:teuthology.orchestra.run.smithi134.stderr:sys 0m2.802s 2024-09-10T10:08:40.939 INFO:tasks.cephadm:Archiving logs... 2024-09-10T10:08:41.023 DEBUG:teuthology.misc:Transferring archived files from smithi077:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7898997/remote/smithi077/log 2024-09-10T10:08:41.053 DEBUG:teuthology.orchestra.run.smithi077:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-10T10:08:53.141 DEBUG:teuthology.misc:Transferring archived files from smithi120:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7898997/remote/smithi120/log 2024-09-10T10:08:53.142 DEBUG:teuthology.orchestra.run.smithi120:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-10T10:09:08.653 DEBUG:teuthology.misc:Transferring archived files from smithi134:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7898997/remote/smithi134/log 2024-09-10T10:09:08.654 DEBUG:teuthology.orchestra.run.smithi134:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-10T10:09:22.752 INFO:tasks.cephadm:Removing cluster... 2024-09-10T10:09:22.753 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 14a55226-6f59-11ef-bcea-c7b262605968 --force 2024-09-10T10:09:23.028 INFO:teuthology.orchestra.run.smithi077.stdout:Deleting cluster with fsid: 14a55226-6f59-11ef-bcea-c7b262605968 2024-09-10T10:09:23.700 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 14a55226-6f59-11ef-bcea-c7b262605968 --force 2024-09-10T10:09:23.902 INFO:teuthology.orchestra.run.smithi120.stdout:Deleting cluster with fsid: 14a55226-6f59-11ef-bcea-c7b262605968 2024-09-10T10:09:24.371 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 14a55226-6f59-11ef-bcea-c7b262605968 --force 2024-09-10T10:09:24.567 INFO:teuthology.orchestra.run.smithi134.stdout:Deleting cluster with fsid: 14a55226-6f59-11ef-bcea-c7b262605968 2024-09-10T10:09:25.104 INFO:tasks.cephadm:Removing cephadm ... 2024-09-10T10:09:25.104 DEBUG:teuthology.orchestra.run.smithi077:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-10T10:09:25.144 DEBUG:teuthology.orchestra.run.smithi120:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-10T10:09:25.160 DEBUG:teuthology.orchestra.run.smithi134:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-10T10:09:25.176 INFO:tasks.cephadm:Teardown complete 2024-09-10T10:09:25.176 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-10T10:09:25.186 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-10T10:09:25.187 DEBUG:teuthology.orchestra.run.smithi077:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-10T10:09:25.191 DEBUG:teuthology.orchestra.run.smithi120:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-10T10:09:25.202 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-10T10:09:25.324 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-10T10:09:25.335 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-10T10:09:25.335 DEBUG:teuthology.orchestra.run.smithi077:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T10:09:25.340 DEBUG:teuthology.orchestra.run.smithi120:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T10:09:25.343 DEBUG:teuthology.orchestra.run.smithi134:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T10:09:25.354 INFO:teuthology.orchestra.run.smithi120.stderr:bash: line 1: ntpq: command not found 2024-09-10T10:09:25.357 INFO:teuthology.orchestra.run.smithi134.stderr:bash: line 1: ntpq: command not found 2024-09-10T10:09:25.359 INFO:teuthology.orchestra.run.smithi120.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T10:09:25.359 INFO:teuthology.orchestra.run.smithi120.stdout:=============================================================================== 2024-09-10T10:09:25.359 INFO:teuthology.orchestra.run.smithi120.stdout:^* hv01.front.sepia.ceph.com 3 6 377 0 -345us[ -335us] +/- 48ms 2024-09-10T10:09:25.359 INFO:teuthology.orchestra.run.smithi120.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 1 +509us[ +518us] +/- 62ms 2024-09-10T10:09:25.359 INFO:teuthology.orchestra.run.smithi120.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 64 -99us[ -87us] +/- 89ms 2024-09-10T10:09:25.359 INFO:teuthology.orchestra.run.smithi120.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T10:09:25.362 INFO:teuthology.orchestra.run.smithi134.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T10:09:25.362 INFO:teuthology.orchestra.run.smithi134.stdout:=============================================================================== 2024-09-10T10:09:25.362 INFO:teuthology.orchestra.run.smithi134.stdout:^* hv01.front.sepia.ceph.com 3 6 377 3 -331us[ -318us] +/- 48ms 2024-09-10T10:09:25.362 INFO:teuthology.orchestra.run.smithi134.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 3 +514us[ +514us] +/- 62ms 2024-09-10T10:09:25.362 INFO:teuthology.orchestra.run.smithi134.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 2 -36us[ -36us] +/- 90ms 2024-09-10T10:09:25.362 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T10:09:25.370 INFO:teuthology.orchestra.run.smithi077.stderr:bash: line 1: ntpq: command not found 2024-09-10T10:09:25.377 INFO:teuthology.orchestra.run.smithi077.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T10:09:25.378 INFO:teuthology.orchestra.run.smithi077.stdout:=============================================================================== 2024-09-10T10:09:25.378 INFO:teuthology.orchestra.run.smithi077.stdout:^* hv01.front.sepia.ceph.com 3 6 377 3 -314us[ -278us] +/- 48ms 2024-09-10T10:09:25.378 INFO:teuthology.orchestra.run.smithi077.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 3 +509us[ +509us] +/- 62ms 2024-09-10T10:09:25.378 INFO:teuthology.orchestra.run.smithi077.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 3 -21us[ -21us] +/- 90ms 2024-09-10T10:09:25.378 INFO:teuthology.orchestra.run.smithi077.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T10:09:25.379 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-10T10:09:25.391 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-10T10:09:25.391 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-10T10:09:25.403 DEBUG:teuthology.orchestra.run.smithi077:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-10T10:09:25.555 DEBUG:teuthology.orchestra.run.smithi120:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-10T10:09:25.621 DEBUG:teuthology.orchestra.run.smithi134:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-10T10:09:25.689 DEBUG:teuthology.orchestra.run.smithi077:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-10T10:09:25.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T10:09:25.750 DEBUG:teuthology.orchestra.run.smithi120:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-10T10:09:25.778 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T10:09:25.778 DEBUG:teuthology.orchestra.run.smithi134:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-10T10:09:25.806 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T10:09:25.806 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-10T10:09:25.816 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-10T10:09:25.869 INFO:teuthology.task.internal:Duration was 1968.694719 seconds 2024-09-10T10:09:25.869 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-10T10:09:25.902 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-10T10:09:25.902 DEBUG:teuthology.orchestra.run.smithi077:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-10T10:09:25.906 DEBUG:teuthology.orchestra.run.smithi120:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-10T10:09:25.909 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-10T10:09:25.945 INFO:teuthology.orchestra.run.smithi120.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T10:09:25.948 INFO:teuthology.orchestra.run.smithi134.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T10:09:25.989 INFO:teuthology.orchestra.run.smithi077.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T10:09:26.515 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-10T10:09:26.515 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi077.front.sepia.ceph.com 2024-09-10T10:09:26.515 DEBUG:teuthology.orchestra.run.smithi077:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-10T10:09:26.564 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi120.front.sepia.ceph.com 2024-09-10T10:09:26.565 DEBUG:teuthology.orchestra.run.smithi120:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-10T10:09:26.587 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi134.front.sepia.ceph.com 2024-09-10T10:09:26.587 DEBUG:teuthology.orchestra.run.smithi134:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-10T10:09:26.617 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-10T10:09:26.617 DEBUG:teuthology.orchestra.run.smithi077:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-10T10:09:26.621 DEBUG:teuthology.orchestra.run.smithi120:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-10T10:09:26.630 DEBUG:teuthology.orchestra.run.smithi134:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-10T10:09:26.686 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-10T10:09:26.686 DEBUG:teuthology.orchestra.run.smithi077:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-10T10:09:26.690 DEBUG:teuthology.orchestra.run.smithi120:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-10T10:09:26.697 DEBUG:teuthology.orchestra.run.smithi134:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-10T10:09:28.003 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-10T10:09:28.014 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-10T10:09:28.015 DEBUG:teuthology.orchestra.run.smithi077:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-10T10:09:28.070 DEBUG:teuthology.orchestra.run.smithi120:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-10T10:09:28.100 DEBUG:teuthology.orchestra.run.smithi134:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-10T10:09:28.131 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-10T10:09:28.143 DEBUG:teuthology.orchestra.run.smithi077:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-10T10:09:28.147 DEBUG:teuthology.orchestra.run.smithi120:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-10T10:09:28.150 DEBUG:teuthology.orchestra.run.smithi134:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-10T10:09:28.175 INFO:teuthology.orchestra.run.smithi120.stdout:kernel.core_pattern = core 2024-09-10T10:09:28.199 INFO:teuthology.orchestra.run.smithi134.stdout:kernel.core_pattern = core 2024-09-10T10:09:28.214 INFO:teuthology.orchestra.run.smithi077.stdout:kernel.core_pattern = core 2024-09-10T10:09:28.244 DEBUG:teuthology.orchestra.run.smithi077:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-10T10:09:28.277 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T10:09:28.277 DEBUG:teuthology.orchestra.run.smithi120:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-10T10:09:28.293 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T10:09:28.294 DEBUG:teuthology.orchestra.run.smithi134:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-10T10:09:28.310 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T10:09:28.311 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-10T10:09:28.322 INFO:teuthology.task.internal:Transferring archived files... 2024-09-10T10:09:28.323 DEBUG:teuthology.misc:Transferring archived files from smithi077:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7898997/remote/smithi077 2024-09-10T10:09:28.323 DEBUG:teuthology.orchestra.run.smithi077:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-10T10:09:28.419 DEBUG:teuthology.misc:Transferring archived files from smithi120:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7898997/remote/smithi120 2024-09-10T10:09:28.419 DEBUG:teuthology.orchestra.run.smithi120:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-10T10:09:28.458 DEBUG:teuthology.misc:Transferring archived files from smithi134:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7898997/remote/smithi134 2024-09-10T10:09:28.458 DEBUG:teuthology.orchestra.run.smithi134:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-10T10:09:28.498 INFO:teuthology.task.internal:Removing archive directory... 2024-09-10T10:09:28.499 DEBUG:teuthology.orchestra.run.smithi077:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-10T10:09:28.503 DEBUG:teuthology.orchestra.run.smithi120:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-10T10:09:28.505 DEBUG:teuthology.orchestra.run.smithi134:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-10T10:09:28.551 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-10T10:09:28.564 INFO:teuthology.task.internal:Not uploading archives. 2024-09-10T10:09:28.564 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-10T10:09:28.575 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-10T10:09:28.575 DEBUG:teuthology.orchestra.run.smithi077:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-10T10:09:28.579 DEBUG:teuthology.orchestra.run.smithi120:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-10T10:09:28.581 DEBUG:teuthology.orchestra.run.smithi134:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-10T10:09:28.595 INFO:teuthology.orchestra.run.smithi120.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 10 10:09 /home/ubuntu/cephtest 2024-09-10T10:09:28.608 INFO:teuthology.orchestra.run.smithi134.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 10 10:09 /home/ubuntu/cephtest 2024-09-10T10:09:28.610 INFO:teuthology.orchestra.run.smithi077.stdout: 393253 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 10 10:09 /home/ubuntu/cephtest 2024-09-10T10:09:28.612 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-10T10:09:28.624 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-10T10:09:28.649 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 2ca44a2f35ca8f36983aaaa46d49312239ef519a description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-quiesce/with-quiesce 6-workunit/suites/fsx}} duration: 1968.6947193145752 flavor: default owner: scheduled_rishabh@teuthology success: true 2024-09-10T10:09:28.649 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-10T10:09:28.742 INFO:teuthology.run:pass